2024-03-28T00:00:59.238 INFO:root:teuthology version: 0.0.1.dev262+ge691533 2024-03-28T00:00:59.239 DEBUG:teuthology.run:Teuthology command: teuthology --name mchangir-2024-03-27_03:45:23-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi --owner mchangir --verbose --archive /home/teuthworker/archive/mchangir-2024-03-27_03:45:23-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7625222 --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/no 5-workunit/fs/misc}} -- /home/teuthworker/archive/mchangir-2024-03-27_03:45:23-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7625222/orig.config.yaml 2024-03-28T00:00:59.306 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-28T00:00:59.399 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/mchangir-2024-03-27_03:45:23-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7625222 branch: wip-mchangir-testing-main-20240318.032620 description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/no 5-workunit/fs/misc}} email: mchangir@redhat.com first_in_suite: false job_id: '7625222' kernel: client: branch: testing flavor: default kdb: true sha1: 1df0c7e8a2354c42d738e5454ba83ea7332deac2 ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: mchangir-2024-03-27_03:45:23-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-mchangir-testing-main-20240318.032620 ceph: cephfs: max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN sha1: 3be9a5c9bc793e11e2800a8c0c696e8b46742033 subvols: create: 2 subvol_options: --size 25000000000 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount - mds.dir_split install: ceph: flavor: default sha1: 3be9a5c9bc793e11e2800a8c0c696e8b46742033 kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=legacy - nowsync syntax: v2 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-mchangir-testing-main-20240318.032620 sha1: 3be9a5c9bc793e11e2800a8c0c696e8b46742033 owner: mchangir priority: 100 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 3645 sha1: 3be9a5c9bc793e11e2800a8c0c696e8b46742033 sleep_before_teardown: 0 subset: 132/256 suite: fs suite_branch: wip-mchangir-testing-main-20240318.032620 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_3be9a5c9bc793e11e2800a8c0c696e8b46742033/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 3be9a5c9bc793e11e2800a8c0c696e8b46742033 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi119.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEd29ajJHtaj7DsIRisAwrNhnYTOIHHKP3komPhXBxj3BOJIQ5qd5WmnRRFVWc5g7t0POdHgNq4oOzNy6uYrIws= smithi132.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAD8uE5unwtIM7H8gS8emVpmLElwB2ef1MrmO1+cnZryHOfIacQvr7hrIn8lM0o82Q5O3kinVH+4Ncr5A2yOMv4= smithi165.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLJ2qwb9kad4dYQAG+VSN9XqP+9Kl9k3R4j1cNPXAhwRYinunU5QtlJjgomp9+bRwSYpYLMKSmLGdK42XMlTBmU= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate true - check-counter: {} - exec: mon.a: - ceph mgr module enable snap_schedule - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - workunit: clients: all: - fs/misc teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: e691533f9cbb33d85b2187bba20d7102f098636d timestamp: 2024-03-27_03:45:23 tube: smithi user: mchangir verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-03-28T00:00:59.400 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_3be9a5c9bc793e11e2800a8c0c696e8b46742033/qa; will attempt to use it 2024-03-28T00:00:59.400 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_3be9a5c9bc793e11e2800a8c0c696e8b46742033/qa/tasks 2024-03-28T00:00:59.400 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-03-28T00:00:59.402 INFO:teuthology.task.internal:Checking packages... 2024-03-28T00:00:59.430 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '3be9a5c9bc793e11e2800a8c0c696e8b46742033' 2024-03-28T00:00:59.430 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-28T00:00:59.430 INFO:teuthology.packaging:ref: None 2024-03-28T00:00:59.430 INFO:teuthology.packaging:tag: None 2024-03-28T00:00:59.430 INFO:teuthology.packaging:branch: wip-mchangir-testing-main-20240318.032620 2024-03-28T00:00:59.430 INFO:teuthology.packaging:sha1: 3be9a5c9bc793e11e2800a8c0c696e8b46742033 2024-03-28T00:00:59.430 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-mchangir-testing-main-20240318.032620 2024-03-28T00:00:59.675 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2145.g3be9a5c9 2024-03-28T00:00:59.676 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-03-28T00:00:59.735 INFO:teuthology.task.internal:no buildpackages task found 2024-03-28T00:00:59.735 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-03-28T00:00:59.947 INFO:teuthology.task.internal:Saving configuration 2024-03-28T00:00:59.969 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-03-28T00:00:59.978 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-28T00:01:00.005 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi119.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2024-03-27_03:45:23-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7625222', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-27 23:55:58.810273', 'locked_by': 'mchangir', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEd29ajJHtaj7DsIRisAwrNhnYTOIHHKP3komPhXBxj3BOJIQ5qd5WmnRRFVWc5g7t0POdHgNq4oOzNy6uYrIws='} 2024-03-28T00:01:00.028 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi132.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2024-03-27_03:45:23-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7625222', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-27 23:55:58.813947', 'locked_by': 'mchangir', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAD8uE5unwtIM7H8gS8emVpmLElwB2ef1MrmO1+cnZryHOfIacQvr7hrIn8lM0o82Q5O3kinVH+4Ncr5A2yOMv4='} 2024-03-28T00:01:00.051 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi165.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2024-03-27_03:45:23-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7625222', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-27 23:55:58.812706', 'locked_by': 'mchangir', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLJ2qwb9kad4dYQAG+VSN9XqP+9Kl9k3R4j1cNPXAhwRYinunU5QtlJjgomp9+bRwSYpYLMKSmLGdK42XMlTBmU='} 2024-03-28T00:01:00.051 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-03-28T00:01:00.062 INFO:teuthology.task.internal:roles: ubuntu@smithi119.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-03-28T00:01:00.063 INFO:teuthology.task.internal:roles: ubuntu@smithi132.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-03-28T00:01:00.063 INFO:teuthology.task.internal:roles: ubuntu@smithi165.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-03-28T00:01:00.063 INFO:teuthology.run_tasks:Running task console_log... 2024-03-28T00:01:00.182 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fc7700100d0>, signals=[15]) 2024-03-28T00:01:00.182 INFO:teuthology.run_tasks:Running task internal.connect... 2024-03-28T00:01:00.229 INFO:teuthology.task.internal:Opening connections... 2024-03-28T00:01:00.230 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi119.front.sepia.ceph.com 2024-03-28T00:01:00.233 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-28T00:01:00.322 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi132.front.sepia.ceph.com 2024-03-28T00:01:00.323 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi132.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-28T00:01:00.412 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi165.front.sepia.ceph.com 2024-03-28T00:01:00.413 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi165.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-28T00:01:00.505 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-03-28T00:01:00.534 DEBUG:teuthology.orchestra.run.smithi119:> uname -m 2024-03-28T00:01:00.552 INFO:teuthology.orchestra.run.smithi119.stdout:x86_64 2024-03-28T00:01:00.552 DEBUG:teuthology.orchestra.run.smithi119:> cat /etc/os-release 2024-03-28T00:01:00.610 INFO:teuthology.orchestra.run.smithi119.stdout:NAME="CentOS Stream" 2024-03-28T00:01:00.611 INFO:teuthology.orchestra.run.smithi119.stdout:VERSION="9" 2024-03-28T00:01:00.611 INFO:teuthology.orchestra.run.smithi119.stdout:ID="centos" 2024-03-28T00:01:00.611 INFO:teuthology.orchestra.run.smithi119.stdout:ID_LIKE="rhel fedora" 2024-03-28T00:01:00.611 INFO:teuthology.orchestra.run.smithi119.stdout:VERSION_ID="9" 2024-03-28T00:01:00.611 INFO:teuthology.orchestra.run.smithi119.stdout:PLATFORM_ID="platform:el9" 2024-03-28T00:01:00.611 INFO:teuthology.orchestra.run.smithi119.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-28T00:01:00.611 INFO:teuthology.orchestra.run.smithi119.stdout:ANSI_COLOR="0;31" 2024-03-28T00:01:00.611 INFO:teuthology.orchestra.run.smithi119.stdout:LOGO="fedora-logo-icon" 2024-03-28T00:01:00.611 INFO:teuthology.orchestra.run.smithi119.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-28T00:01:00.611 INFO:teuthology.orchestra.run.smithi119.stdout:HOME_URL="https://centos.org/" 2024-03-28T00:01:00.611 INFO:teuthology.orchestra.run.smithi119.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-28T00:01:00.611 INFO:teuthology.orchestra.run.smithi119.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-28T00:01:00.611 INFO:teuthology.orchestra.run.smithi119.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-28T00:01:00.612 INFO:teuthology.lock.ops:Updating smithi119.front.sepia.ceph.com on lock server 2024-03-28T00:01:00.633 DEBUG:teuthology.orchestra.run.smithi132:> uname -m 2024-03-28T00:01:00.651 INFO:teuthology.orchestra.run.smithi132.stdout:x86_64 2024-03-28T00:01:00.651 DEBUG:teuthology.orchestra.run.smithi132:> cat /etc/os-release 2024-03-28T00:01:00.708 INFO:teuthology.orchestra.run.smithi132.stdout:NAME="CentOS Stream" 2024-03-28T00:01:00.708 INFO:teuthology.orchestra.run.smithi132.stdout:VERSION="9" 2024-03-28T00:01:00.708 INFO:teuthology.orchestra.run.smithi132.stdout:ID="centos" 2024-03-28T00:01:00.708 INFO:teuthology.orchestra.run.smithi132.stdout:ID_LIKE="rhel fedora" 2024-03-28T00:01:00.708 INFO:teuthology.orchestra.run.smithi132.stdout:VERSION_ID="9" 2024-03-28T00:01:00.709 INFO:teuthology.orchestra.run.smithi132.stdout:PLATFORM_ID="platform:el9" 2024-03-28T00:01:00.709 INFO:teuthology.orchestra.run.smithi132.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-28T00:01:00.709 INFO:teuthology.orchestra.run.smithi132.stdout:ANSI_COLOR="0;31" 2024-03-28T00:01:00.709 INFO:teuthology.orchestra.run.smithi132.stdout:LOGO="fedora-logo-icon" 2024-03-28T00:01:00.709 INFO:teuthology.orchestra.run.smithi132.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-28T00:01:00.709 INFO:teuthology.orchestra.run.smithi132.stdout:HOME_URL="https://centos.org/" 2024-03-28T00:01:00.709 INFO:teuthology.orchestra.run.smithi132.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-28T00:01:00.709 INFO:teuthology.orchestra.run.smithi132.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-28T00:01:00.709 INFO:teuthology.orchestra.run.smithi132.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-28T00:01:00.709 INFO:teuthology.lock.ops:Updating smithi132.front.sepia.ceph.com on lock server 2024-03-28T00:01:00.732 DEBUG:teuthology.orchestra.run.smithi165:> uname -m 2024-03-28T00:01:00.750 INFO:teuthology.orchestra.run.smithi165.stdout:x86_64 2024-03-28T00:01:00.750 DEBUG:teuthology.orchestra.run.smithi165:> cat /etc/os-release 2024-03-28T00:01:00.807 INFO:teuthology.orchestra.run.smithi165.stdout:NAME="CentOS Stream" 2024-03-28T00:01:00.807 INFO:teuthology.orchestra.run.smithi165.stdout:VERSION="9" 2024-03-28T00:01:00.807 INFO:teuthology.orchestra.run.smithi165.stdout:ID="centos" 2024-03-28T00:01:00.807 INFO:teuthology.orchestra.run.smithi165.stdout:ID_LIKE="rhel fedora" 2024-03-28T00:01:00.807 INFO:teuthology.orchestra.run.smithi165.stdout:VERSION_ID="9" 2024-03-28T00:01:00.807 INFO:teuthology.orchestra.run.smithi165.stdout:PLATFORM_ID="platform:el9" 2024-03-28T00:01:00.807 INFO:teuthology.orchestra.run.smithi165.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-28T00:01:00.807 INFO:teuthology.orchestra.run.smithi165.stdout:ANSI_COLOR="0;31" 2024-03-28T00:01:00.807 INFO:teuthology.orchestra.run.smithi165.stdout:LOGO="fedora-logo-icon" 2024-03-28T00:01:00.807 INFO:teuthology.orchestra.run.smithi165.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-28T00:01:00.807 INFO:teuthology.orchestra.run.smithi165.stdout:HOME_URL="https://centos.org/" 2024-03-28T00:01:00.808 INFO:teuthology.orchestra.run.smithi165.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-28T00:01:00.808 INFO:teuthology.orchestra.run.smithi165.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-28T00:01:00.808 INFO:teuthology.orchestra.run.smithi165.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-28T00:01:00.808 INFO:teuthology.lock.ops:Updating smithi165.front.sepia.ceph.com on lock server 2024-03-28T00:01:00.824 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-03-28T00:01:00.839 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-03-28T00:01:00.868 INFO:teuthology.task.internal:Checking for old test directory... 2024-03-28T00:01:00.869 DEBUG:teuthology.orchestra.run.smithi119:> test '!' -e /home/ubuntu/cephtest 2024-03-28T00:01:00.871 DEBUG:teuthology.orchestra.run.smithi132:> test '!' -e /home/ubuntu/cephtest 2024-03-28T00:01:00.873 DEBUG:teuthology.orchestra.run.smithi165:> test '!' -e /home/ubuntu/cephtest 2024-03-28T00:01:00.889 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-03-28T00:01:00.895 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-03-28T00:01:00.895 DEBUG:teuthology.orchestra.run.smithi119:> test -z $(ls -A /var/lib/ceph) 2024-03-28T00:01:00.927 DEBUG:teuthology.orchestra.run.smithi132:> test -z $(ls -A /var/lib/ceph) 2024-03-28T00:01:00.930 DEBUG:teuthology.orchestra.run.smithi165:> test -z $(ls -A /var/lib/ceph) 2024-03-28T00:01:00.960 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-03-28T00:01:01.055 INFO:teuthology.run_tasks:Running task kernel... 2024-03-28T00:01:01.080 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'flavor': 'default', 'kdb': True, 'sha1': '1df0c7e8a2354c42d738e5454ba83ea7332deac2'} 2024-03-28T00:01:01.081 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}} 2024-03-28T00:01:01.081 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}}, timeout 300 2024-03-28T00:01:01.081 DEBUG:teuthology.orchestra.run.smithi119:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-28T00:01:01.098 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-28T00:01:01.098 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-03-28T00:01:01.334 DEBUG:teuthology.task.kernel:sha1 for client.0 is 1df0c7e8a2354c42d738e5454ba83ea7332deac2 2024-03-28T00:01:01.334 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "1df0c7e8a2354c42d738e5454ba83ea7332deac2"... 2024-03-28T00:01:01.334 DEBUG:teuthology.orchestra.run.smithi119:> uname -r 2024-03-28T00:01:01.351 INFO:teuthology.orchestra.run.smithi119.stdout:5.14.0-361.el9.x86_64 2024-03-28T00:01:01.352 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-361.el9.x86_64 vs 1df0c7e8a2354c42d738e5454ba83ea7332deac2 2024-03-28T00:01:01.352 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-03-28T00:01:01.353 DEBUG:teuthology.orchestra.run.smithi119:> sudo yum upgrade -y linux-firmware 2024-03-28T00:01:03.490 INFO:teuthology.orchestra.run.smithi119.stdout:CentOS-9 - AppStream 13 MB/s | 6.7 MB 00:00 2024-03-28T00:01:05.124 INFO:teuthology.orchestra.run.smithi119.stdout:CentOS-9 - BaseOS 3.7 MB/s | 2.0 MB 00:00 2024-03-28T00:01:14.638 INFO:teuthology.orchestra.run.smithi119.stdout:CentOS Stream 9 - BaseOS 917 kB/s | 8.0 MB 00:08 2024-03-28T00:01:16.464 INFO:teuthology.orchestra.run.smithi119.stdout:CentOS Stream 9 - AppStream 24 MB/s | 19 MB 00:00 2024-03-28T00:01:20.806 INFO:teuthology.orchestra.run.smithi119.stdout:CentOS Stream 9 - CRB 6.7 MB/s | 6.1 MB 00:00 2024-03-28T00:01:23.065 INFO:teuthology.orchestra.run.smithi119.stdout:CentOS Stream 9 - Extras packages 14 kB/s | 16 kB 00:01 2024-03-28T00:01:24.009 INFO:teuthology.orchestra.run.smithi119.stdout:Extra Packages for Enterprise Linux 25 MB/s | 21 MB 00:00 2024-03-28T00:01:28.867 INFO:teuthology.orchestra.run.smithi119.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-03-28T00:01:31.115 INFO:teuthology.orchestra.run.smithi119.stdout:Dependencies resolved. 2024-03-28T00:01:31.125 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2024-03-28T00:01:31.125 INFO:teuthology.orchestra.run.smithi119.stdout: Package Arch Version Repository Size 2024-03-28T00:01:31.125 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2024-03-28T00:01:31.125 INFO:teuthology.orchestra.run.smithi119.stdout:Upgrading: 2024-03-28T00:01:31.125 INFO:teuthology.orchestra.run.smithi119.stdout: linux-firmware noarch 20240219-143.el9 CentOS-BaseOS 375 M 2024-03-28T00:01:31.125 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-28T00:01:31.125 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction Summary 2024-03-28T00:01:31.125 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2024-03-28T00:01:31.125 INFO:teuthology.orchestra.run.smithi119.stdout:Upgrade 1 Package 2024-03-28T00:01:31.125 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-28T00:01:31.125 INFO:teuthology.orchestra.run.smithi119.stdout:Total download size: 375 M 2024-03-28T00:01:31.126 INFO:teuthology.orchestra.run.smithi119.stdout:Downloading Packages: 2024-03-28T00:01:38.717 INFO:teuthology.orchestra.run.smithi119.stdout:linux-firmware-20240219-143.el9.noarch.rpm 49 MB/s | 375 MB 00:07 2024-03-28T00:01:38.718 INFO:teuthology.orchestra.run.smithi119.stdout:-------------------------------------------------------------------------------- 2024-03-28T00:01:38.719 INFO:teuthology.orchestra.run.smithi119.stdout:Total 49 MB/s | 375 MB 00:07 2024-03-28T00:01:38.719 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction check 2024-03-28T00:01:38.825 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction check succeeded. 2024-03-28T00:01:38.825 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction test 2024-03-28T00:01:41.186 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction test succeeded. 2024-03-28T00:01:41.186 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction 2024-03-28T00:01:47.319 INFO:teuthology.orchestra.run.smithi119.stdout: Preparing : 1/1 2024-03-28T00:01:47.556 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : linux-firmware-20240219-143.el9.noarch 1/2 2024-03-28T00:01:47.653 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : linux-firmware-20230814-139.el9.noarch 2/2 2024-03-28T00:01:49.051 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: linux-firmware-20230814-139.el9.noarch 2/2 2024-03-28T00:01:49.051 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : linux-firmware-20240219-143.el9.noarch 1/2 2024-03-28T00:01:49.463 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : linux-firmware-20230814-139.el9.noarch 2/2 2024-03-28T00:01:49.463 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-28T00:01:49.463 INFO:teuthology.orchestra.run.smithi119.stdout:Upgraded: 2024-03-28T00:01:49.463 INFO:teuthology.orchestra.run.smithi119.stdout: linux-firmware-20240219-143.el9.noarch 2024-03-28T00:01:49.463 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-28T00:01:49.463 INFO:teuthology.orchestra.run.smithi119.stdout:Complete! 2024-03-28T00:01:49.754 INFO:teuthology.task.kernel:Downloading kernel 1df0c7e8a2354c42d738e5454ba83ea7332deac2 on client.0... 2024-03-28T00:01:49.755 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1df0c7e8a2354c42d738e5454ba83ea7332deac2 2024-03-28T00:01:49.980 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/1df0c7e8a2354c42d738e5454ba83ea7332deac2/centos/9/flavors/default/x86_64/ 2024-03-28T00:01:49.980 DEBUG:teuthology.orchestra.run.smithi119:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.8.0_g1df0c7e8a235-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/1df0c7e8a2354c42d738e5454ba83ea7332deac2/centos/9/flavors/default/x86_64/ --input-file=- 2024-03-28T00:01:49.983 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi119.front.sepia.ceph.com to complete... 2024-03-28T00:02:29.181 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28 00:02:29 URL:https://3.chacra.ceph.com/r/kernel/testing/1df0c7e8a2354c42d738e5454ba83ea7332deac2/centos/9/flavors/default/x86_64/kernel-6.8.0_g1df0c7e8a235-1.x86_64.rpm [336317861/336317861] -> "/tmp/kernel.x86_64.rpm" [1] 2024-03-28T00:02:29.182 INFO:teuthology.orchestra.run.smithi119.stderr:FINISHED --2024-03-28 00:02:29-- 2024-03-28T00:02:29.182 INFO:teuthology.orchestra.run.smithi119.stderr:Total wall clock time: 39s 2024-03-28T00:02:29.182 INFO:teuthology.orchestra.run.smithi119.stderr:Downloaded: 1 files, 321M in 39s (8.32 MB/s) 2024-03-28T00:02:29.185 INFO:teuthology.task.kernel:Installing kernel 1df0c7e8a2354c42d738e5454ba83ea7332deac2 on client.0... 2024-03-28T00:02:29.185 DEBUG:teuthology.orchestra.run.smithi119:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-03-28T00:02:31.104 INFO:teuthology.orchestra.run.smithi119.stdout:Verifying... ######################################## 2024-03-28T00:02:31.167 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing... ######################################## 2024-03-28T00:02:31.247 INFO:teuthology.orchestra.run.smithi119.stdout:Updating / installing... 2024-03-28T00:02:38.754 INFO:teuthology.orchestra.run.smithi119.stdout:kernel-6.8.0_g1df0c7e8a235-1 ######################################## 2024-03-28T00:02:53.363 INFO:teuthology.orchestra.run.smithi119.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-28T00:03:15.728 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi119.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-03-28T00:03:15.729 DEBUG:teuthology.orchestra.run.smithi119:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-03-28T00:03:15.802 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235 2024-03-28T00:03:15.802 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/System.map 2024-03-28T00:03:15.802 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/config 2024-03-28T00:03:15.802 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel 2024-03-28T00:03:15.802 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch 2024-03-28T00:03:15.802 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86 2024-03-28T00:03:15.802 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto 2024-03-28T00:03:15.802 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-03-28T00:03:15.803 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-03-28T00:03:15.803 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-03-28T00:03:15.803 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-03-28T00:03:15.803 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-03-28T00:03:15.803 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-03-28T00:03:15.803 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-03-28T00:03:15.803 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/crc32c-intel.ko 2024-03-28T00:03:15.803 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-03-28T00:03:15.803 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-03-28T00:03:15.803 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-03-28T00:03:15.803 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-03-28T00:03:15.803 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/serpent-avx2.ko 2024-03-28T00:03:15.803 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-03-28T00:03:15.804 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-03-28T00:03:15.804 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-03-28T00:03:15.804 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-03-28T00:03:15.804 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-03-28T00:03:15.804 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-03-28T00:03:15.804 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-03-28T00:03:15.804 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel 2024-03-28T00:03:15.804 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel/cpu 2024-03-28T00:03:15.804 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel/cpu/mce 2024-03-28T00:03:15.804 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-03-28T00:03:15.804 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kvm 2024-03-28T00:03:15.804 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kvm/kvm-amd.ko 2024-03-28T00:03:15.804 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kvm/kvm-intel.ko 2024-03-28T00:03:15.805 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kvm/kvm.ko 2024-03-28T00:03:15.805 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto 2024-03-28T00:03:15.805 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/ansi_cprng.ko 2024-03-28T00:03:15.805 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/anubis.ko 2024-03-28T00:03:15.805 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/arc4.ko 2024-03-28T00:03:15.805 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx 2024-03-28T00:03:15.805 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_memcpy.ko 2024-03-28T00:03:15.805 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_pq.ko 2024-03-28T00:03:15.805 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_raid6_recov.ko 2024-03-28T00:03:15.805 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_tx.ko 2024-03-28T00:03:15.805 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_xor.ko 2024-03-28T00:03:15.805 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/raid6test.ko 2024-03-28T00:03:15.805 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/authenc.ko 2024-03-28T00:03:15.806 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/authencesn.ko 2024-03-28T00:03:15.806 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/blake2b_generic.ko 2024-03-28T00:03:15.806 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/blowfish_common.ko 2024-03-28T00:03:15.806 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/blowfish_generic.ko 2024-03-28T00:03:15.806 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/camellia_generic.ko 2024-03-28T00:03:15.806 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cast5_generic.ko 2024-03-28T00:03:15.806 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cast6_generic.ko 2024-03-28T00:03:15.806 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cast_common.ko 2024-03-28T00:03:15.806 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/ccm.ko 2024-03-28T00:03:15.806 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cmac.ko 2024-03-28T00:03:15.806 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/crc32_generic.ko 2024-03-28T00:03:15.807 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/crypto_engine.ko 2024-03-28T00:03:15.807 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/crypto_user.ko 2024-03-28T00:03:15.807 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cts.ko 2024-03-28T00:03:15.807 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/des_generic.ko 2024-03-28T00:03:15.807 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/dh_generic.ko 2024-03-28T00:03:15.807 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/echainiv.ko 2024-03-28T00:03:15.807 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/essiv.ko 2024-03-28T00:03:15.807 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/fcrypt.ko 2024-03-28T00:03:15.807 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/khazad.ko 2024-03-28T00:03:15.807 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/lz4.ko 2024-03-28T00:03:15.807 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/lz4hc.ko 2024-03-28T00:03:15.807 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/md4.ko 2024-03-28T00:03:15.808 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/michael_mic.ko 2024-03-28T00:03:15.808 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/pcbc.ko 2024-03-28T00:03:15.808 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/pcrypt.ko 2024-03-28T00:03:15.808 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/rmd160.ko 2024-03-28T00:03:15.808 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/seed.ko 2024-03-28T00:03:15.808 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/serpent_generic.ko 2024-03-28T00:03:15.808 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/tcrypt.ko 2024-03-28T00:03:15.808 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/tea.ko 2024-03-28T00:03:15.808 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/twofish_common.ko 2024-03-28T00:03:15.808 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/twofish_generic.ko 2024-03-28T00:03:15.808 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/vmac.ko 2024-03-28T00:03:15.808 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/wp512.ko 2024-03-28T00:03:15.808 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/xcbc.ko 2024-03-28T00:03:15.808 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/xor.ko 2024-03-28T00:03:15.808 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/xxhash_generic.ko 2024-03-28T00:03:15.808 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers 2024-03-28T00:03:15.809 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi 2024-03-28T00:03:15.809 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/acpi_ipmi.ko 2024-03-28T00:03:15.809 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/acpi_pad.ko 2024-03-28T00:03:15.809 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/custom_method.ko 2024-03-28T00:03:15.809 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/ec_sys.ko 2024-03-28T00:03:15.809 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/platform_profile.ko 2024-03-28T00:03:15.809 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/sbs.ko 2024-03-28T00:03:15.809 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/sbshc.ko 2024-03-28T00:03:15.809 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/video.ko 2024-03-28T00:03:15.809 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata 2024-03-28T00:03:15.809 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/acard-ahci.ko 2024-03-28T00:03:15.809 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/ahci_platform.ko 2024-03-28T00:03:15.809 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/ata_generic.ko 2024-03-28T00:03:15.809 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/libahci_platform.ko 2024-03-28T00:03:15.809 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_acpi.ko 2024-03-28T00:03:15.810 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_ali.ko 2024-03-28T00:03:15.810 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_amd.ko 2024-03-28T00:03:15.810 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_artop.ko 2024-03-28T00:03:15.810 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_atiixp.ko 2024-03-28T00:03:15.810 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_atp867x.ko 2024-03-28T00:03:15.810 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_cmd640.ko 2024-03-28T00:03:15.810 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_cmd64x.ko 2024-03-28T00:03:15.810 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_cypress.ko 2024-03-28T00:03:15.810 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_efar.ko 2024-03-28T00:03:15.810 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_hpt366.ko 2024-03-28T00:03:15.810 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_hpt37x.ko 2024-03-28T00:03:15.810 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_hpt3x2n.ko 2024-03-28T00:03:15.810 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_hpt3x3.ko 2024-03-28T00:03:15.810 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_it8213.ko 2024-03-28T00:03:15.811 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_it821x.ko 2024-03-28T00:03:15.811 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_jmicron.ko 2024-03-28T00:03:15.811 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_marvell.ko 2024-03-28T00:03:15.811 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_mpiix.ko 2024-03-28T00:03:15.811 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_netcell.ko 2024-03-28T00:03:15.811 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_ninja32.ko 2024-03-28T00:03:15.811 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_ns87410.ko 2024-03-28T00:03:15.811 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_ns87415.ko 2024-03-28T00:03:15.811 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_oldpiix.ko 2024-03-28T00:03:15.811 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_opti.ko 2024-03-28T00:03:15.811 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_optidma.ko 2024-03-28T00:03:15.811 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_pcmcia.ko 2024-03-28T00:03:15.811 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_pdc2027x.ko 2024-03-28T00:03:15.811 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-03-28T00:03:15.812 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_piccolo.ko 2024-03-28T00:03:15.812 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_rdc.ko 2024-03-28T00:03:15.812 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_sch.ko 2024-03-28T00:03:15.812 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_serverworks.ko 2024-03-28T00:03:15.812 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_sil680.ko 2024-03-28T00:03:15.812 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_sis.ko 2024-03-28T00:03:15.812 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_sl82c105.ko 2024-03-28T00:03:15.812 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_triflex.ko 2024-03-28T00:03:15.812 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_via.ko 2024-03-28T00:03:15.812 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pdc_adma.ko 2024-03-28T00:03:15.812 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_inic162x.ko 2024-03-28T00:03:15.812 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_mv.ko 2024-03-28T00:03:15.812 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_nv.ko 2024-03-28T00:03:15.812 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_promise.ko 2024-03-28T00:03:15.812 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_qstor.ko 2024-03-28T00:03:15.813 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_sil.ko 2024-03-28T00:03:15.813 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_sil24.ko 2024-03-28T00:03:15.813 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_sis.ko 2024-03-28T00:03:15.813 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_svw.ko 2024-03-28T00:03:15.813 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_sx4.ko 2024-03-28T00:03:15.813 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_uli.ko 2024-03-28T00:03:15.813 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_via.ko 2024-03-28T00:03:15.813 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_vsc.ko 2024-03-28T00:03:15.813 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/auxdisplay 2024-03-28T00:03:15.813 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/auxdisplay/ks0108.ko 2024-03-28T00:03:15.813 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/base 2024-03-28T00:03:15.813 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/base/regmap 2024-03-28T00:03:15.813 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/base/regmap/regmap-i2c.ko 2024-03-28T00:03:15.813 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/base/regmap/regmap-mmio.ko 2024-03-28T00:03:15.814 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/bcma 2024-03-28T00:03:15.814 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/bcma/bcma.ko 2024-03-28T00:03:15.814 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block 2024-03-28T00:03:15.814 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/aoe 2024-03-28T00:03:15.814 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/aoe/aoe.ko 2024-03-28T00:03:15.814 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/brd.ko 2024-03-28T00:03:15.814 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/drbd 2024-03-28T00:03:15.814 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/drbd/drbd.ko 2024-03-28T00:03:15.814 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/floppy.ko 2024-03-28T00:03:15.814 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/loop.ko 2024-03-28T00:03:15.814 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/mtip32xx 2024-03-28T00:03:15.814 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-03-28T00:03:15.814 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/nbd.ko 2024-03-28T00:03:15.814 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/null_blk 2024-03-28T00:03:15.815 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/null_blk/null_blk.ko 2024-03-28T00:03:15.815 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/pktcdvd.ko 2024-03-28T00:03:15.815 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/rbd.ko 2024-03-28T00:03:15.815 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/virtio_blk.ko 2024-03-28T00:03:15.815 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/xen-blkback 2024-03-28T00:03:15.815 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-03-28T00:03:15.815 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/xen-blkfront.ko 2024-03-28T00:03:15.815 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/zram 2024-03-28T00:03:15.815 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/zram/zram.ko 2024-03-28T00:03:15.815 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char 2024-03-28T00:03:15.815 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hangcheck-timer.ko 2024-03-28T00:03:15.815 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random 2024-03-28T00:03:15.815 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/amd-rng.ko 2024-03-28T00:03:15.815 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/intel-rng.ko 2024-03-28T00:03:15.816 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-03-28T00:03:15.816 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/via-rng.ko 2024-03-28T00:03:15.816 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/virtio-rng.ko 2024-03-28T00:03:15.816 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi 2024-03-28T00:03:15.816 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-03-28T00:03:15.816 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-03-28T00:03:15.816 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-03-28T00:03:15.816 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_si.ko 2024-03-28T00:03:15.816 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-03-28T00:03:15.816 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-03-28T00:03:15.816 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/lp.ko 2024-03-28T00:03:15.816 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/mwave 2024-03-28T00:03:15.816 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/mwave/mwave.ko 2024-03-28T00:03:15.817 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ppdev.ko 2024-03-28T00:03:15.817 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tlclk.ko 2024-03-28T00:03:15.817 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm 2024-03-28T00:03:15.817 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm.ko 2024-03-28T00:03:15.817 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_atmel.ko 2024-03-28T00:03:15.817 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_infineon.ko 2024-03-28T00:03:15.817 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_nsc.ko 2024-03-28T00:03:15.817 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_tis.ko 2024-03-28T00:03:15.817 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-03-28T00:03:15.817 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/uv_mmtimer.ko 2024-03-28T00:03:15.817 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/virtio_console.ko 2024-03-28T00:03:15.817 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq 2024-03-28T00:03:15.817 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-03-28T00:03:15.817 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-03-28T00:03:15.817 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/p4-clockmod.ko 2024-03-28T00:03:15.818 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-03-28T00:03:15.818 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/powernow-k8.ko 2024-03-28T00:03:15.818 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/speedstep-lib.ko 2024-03-28T00:03:15.818 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto 2024-03-28T00:03:15.818 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/ccp 2024-03-28T00:03:15.818 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-03-28T00:03:15.818 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/ccp/ccp.ko 2024-03-28T00:03:15.818 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel 2024-03-28T00:03:15.818 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat 2024-03-28T00:03:15.818 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat/qat_common 2024-03-28T00:03:15.818 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-03-28T00:03:15.818 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-03-28T00:03:15.818 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-03-28T00:03:15.819 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/padlock-aes.ko 2024-03-28T00:03:15.819 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/padlock-sha.ko 2024-03-28T00:03:15.819 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/virtio 2024-03-28T00:03:15.819 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-03-28T00:03:15.819 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dca 2024-03-28T00:03:15.819 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dca/dca.ko 2024-03-28T00:03:15.819 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/devfreq 2024-03-28T00:03:15.819 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-03-28T00:03:15.819 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma 2024-03-28T00:03:15.819 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma/dw 2024-03-28T00:03:15.819 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma/dw/dw_dmac.ko 2024-03-28T00:03:15.819 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma/ioat 2024-03-28T00:03:15.819 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma/ioat/ioatdma.ko 2024-03-28T00:03:15.819 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac 2024-03-28T00:03:15.819 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/amd64_edac.ko 2024-03-28T00:03:15.820 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/e752x_edac.ko 2024-03-28T00:03:15.820 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/edac_mce_amd.ko 2024-03-28T00:03:15.820 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i3000_edac.ko 2024-03-28T00:03:15.820 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i3200_edac.ko 2024-03-28T00:03:15.820 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i5100_edac.ko 2024-03-28T00:03:15.820 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i5400_edac.ko 2024-03-28T00:03:15.820 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i7300_edac.ko 2024-03-28T00:03:15.820 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i7core_edac.ko 2024-03-28T00:03:15.820 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i82975x_edac.ko 2024-03-28T00:03:15.820 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/ie31200_edac.ko 2024-03-28T00:03:15.820 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/sb_edac.ko 2024-03-28T00:03:15.820 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/x38_edac.ko 2024-03-28T00:03:15.820 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/firmware 2024-03-28T00:03:15.821 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/firmware/edd.ko 2024-03-28T00:03:15.821 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/firmware/iscsi_ibft.ko 2024-03-28T00:03:15.821 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio 2024-03-28T00:03:15.821 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-ich.ko 2024-03-28T00:03:15.821 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-viperboard.ko 2024-03-28T00:03:15.821 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid 2024-03-28T00:03:15.821 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-appleir.ko 2024-03-28T00:03:15.821 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-aureal.ko 2024-03-28T00:03:15.821 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-axff.ko 2024-03-28T00:03:15.821 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-dr.ko 2024-03-28T00:03:15.821 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-elecom.ko 2024-03-28T00:03:15.821 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-elo.ko 2024-03-28T00:03:15.821 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-emsff.ko 2024-03-28T00:03:15.821 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-gaff.ko 2024-03-28T00:03:15.821 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-gt683r.ko 2024-03-28T00:03:15.822 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-gyration.ko 2024-03-28T00:03:15.822 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-holtek-kbd.ko 2024-03-28T00:03:15.822 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-holtek-mouse.ko 2024-03-28T00:03:15.822 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-holtekff.ko 2024-03-28T00:03:15.822 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-hyperv.ko 2024-03-28T00:03:15.822 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-icade.ko 2024-03-28T00:03:15.822 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-keytouch.ko 2024-03-28T00:03:15.822 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-kye.ko 2024-03-28T00:03:15.822 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-lcpower.ko 2024-03-28T00:03:15.822 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-led.ko 2024-03-28T00:03:15.822 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-lenovo.ko 2024-03-28T00:03:15.822 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-logitech-dj.ko 2024-03-28T00:03:15.822 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-03-28T00:03:15.823 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-multitouch.ko 2024-03-28T00:03:15.823 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-ortek.ko 2024-03-28T00:03:15.823 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-penmount.ko 2024-03-28T00:03:15.823 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-petalynx.ko 2024-03-28T00:03:15.823 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-picolcd.ko 2024-03-28T00:03:15.823 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-pl.ko 2024-03-28T00:03:15.823 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-plantronics.ko 2024-03-28T00:03:15.823 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-primax.ko 2024-03-28T00:03:15.823 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-rmi.ko 2024-03-28T00:03:15.823 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-arvo.ko 2024-03-28T00:03:15.823 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-common.ko 2024-03-28T00:03:15.823 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-isku.ko 2024-03-28T00:03:15.823 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-kone.ko 2024-03-28T00:03:15.824 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-03-28T00:03:15.824 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-konepure.ko 2024-03-28T00:03:15.824 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-03-28T00:03:15.824 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-lua.ko 2024-03-28T00:03:15.824 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-pyra.ko 2024-03-28T00:03:15.824 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-ryos.ko 2024-03-28T00:03:15.824 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-savu.ko 2024-03-28T00:03:15.824 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat.ko 2024-03-28T00:03:15.824 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-saitek.ko 2024-03-28T00:03:15.824 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-samsung.ko 2024-03-28T00:03:15.824 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-sensor-hub.ko 2024-03-28T00:03:15.824 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-sjoy.ko 2024-03-28T00:03:15.825 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-sony.ko 2024-03-28T00:03:15.825 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-speedlink.ko 2024-03-28T00:03:15.825 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-steelseries.ko 2024-03-28T00:03:15.825 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-sunplus.ko 2024-03-28T00:03:15.825 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-thrustmaster.ko 2024-03-28T00:03:15.825 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-tivo.ko 2024-03-28T00:03:15.825 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-tmff.ko 2024-03-28T00:03:15.825 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-topseed.ko 2024-03-28T00:03:15.825 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-twinhan.ko 2024-03-28T00:03:15.825 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-uclogic.ko 2024-03-28T00:03:15.825 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-waltop.ko 2024-03-28T00:03:15.825 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-wiimote.ko 2024-03-28T00:03:15.825 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-xinmo.ko 2024-03-28T00:03:15.826 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-zpff.ko 2024-03-28T00:03:15.826 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-zydacron.ko 2024-03-28T00:03:15.826 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/uhid.ko 2024-03-28T00:03:15.826 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/wacom.ko 2024-03-28T00:03:15.826 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hv 2024-03-28T00:03:15.826 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hv/hv_balloon.ko 2024-03-28T00:03:15.826 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hv/hv_utils.ko 2024-03-28T00:03:15.826 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hv/hv_vmbus.ko 2024-03-28T00:03:15.826 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon 2024-03-28T00:03:15.826 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/abituguru.ko 2024-03-28T00:03:15.826 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/abituguru3.ko 2024-03-28T00:03:15.826 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/acpi_power_meter.ko 2024-03-28T00:03:15.827 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ad7414.ko 2024-03-28T00:03:15.827 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ad7418.ko 2024-03-28T00:03:15.827 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adc128d818.ko 2024-03-28T00:03:15.827 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm1025.ko 2024-03-28T00:03:15.827 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm1026.ko 2024-03-28T00:03:15.827 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm1029.ko 2024-03-28T00:03:15.827 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm1031.ko 2024-03-28T00:03:15.827 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm9240.ko 2024-03-28T00:03:15.827 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ads7828.ko 2024-03-28T00:03:15.827 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7410.ko 2024-03-28T00:03:15.827 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7411.ko 2024-03-28T00:03:15.827 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7462.ko 2024-03-28T00:03:15.827 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7470.ko 2024-03-28T00:03:15.827 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7475.ko 2024-03-28T00:03:15.828 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7x10.ko 2024-03-28T00:03:15.828 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/amc6821.ko 2024-03-28T00:03:15.828 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/applesmc.ko 2024-03-28T00:03:15.828 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/asb100.ko 2024-03-28T00:03:15.828 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/asc7621.ko 2024-03-28T00:03:15.828 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/asus_atk0110.ko 2024-03-28T00:03:15.828 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/atxp1.ko 2024-03-28T00:03:15.828 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/coretemp.ko 2024-03-28T00:03:15.828 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-03-28T00:03:15.828 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/dme1737.ko 2024-03-28T00:03:15.828 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ds1621.ko 2024-03-28T00:03:15.828 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ds620.ko 2024-03-28T00:03:15.828 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/emc1403.ko 2024-03-28T00:03:15.828 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/emc6w201.ko 2024-03-28T00:03:15.828 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/f71805f.ko 2024-03-28T00:03:15.829 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/f71882fg.ko 2024-03-28T00:03:15.829 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/f75375s.ko 2024-03-28T00:03:15.829 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/fam15h_power.ko 2024-03-28T00:03:15.829 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/fschmd.ko 2024-03-28T00:03:15.829 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/g760a.ko 2024-03-28T00:03:15.829 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/g762.ko 2024-03-28T00:03:15.829 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/gl518sm.ko 2024-03-28T00:03:15.829 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/gl520sm.ko 2024-03-28T00:03:15.829 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/hwmon-vid.ko 2024-03-28T00:03:15.829 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/i5500_temp.ko 2024-03-28T00:03:15.829 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/i5k_amb.ko 2024-03-28T00:03:15.829 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ibmaem.ko 2024-03-28T00:03:15.830 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ibmpex.ko 2024-03-28T00:03:15.830 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ina209.ko 2024-03-28T00:03:15.830 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ina2xx.ko 2024-03-28T00:03:15.830 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/it87.ko 2024-03-28T00:03:15.830 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/k10temp.ko 2024-03-28T00:03:15.830 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/k8temp.ko 2024-03-28T00:03:15.830 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lineage-pem.ko 2024-03-28T00:03:15.830 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm63.ko 2024-03-28T00:03:15.830 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm73.ko 2024-03-28T00:03:15.830 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm75.ko 2024-03-28T00:03:15.830 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm77.ko 2024-03-28T00:03:15.830 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm78.ko 2024-03-28T00:03:15.830 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm80.ko 2024-03-28T00:03:15.830 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm83.ko 2024-03-28T00:03:15.831 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm85.ko 2024-03-28T00:03:15.831 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm87.ko 2024-03-28T00:03:15.831 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm90.ko 2024-03-28T00:03:15.831 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm92.ko 2024-03-28T00:03:15.831 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm93.ko 2024-03-28T00:03:15.831 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm95234.ko 2024-03-28T00:03:15.831 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm95241.ko 2024-03-28T00:03:15.831 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm95245.ko 2024-03-28T00:03:15.831 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc2945.ko 2024-03-28T00:03:15.831 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4151.ko 2024-03-28T00:03:15.831 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4215.ko 2024-03-28T00:03:15.831 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4222.ko 2024-03-28T00:03:15.831 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4245.ko 2024-03-28T00:03:15.831 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4260.ko 2024-03-28T00:03:15.832 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4261.ko 2024-03-28T00:03:15.832 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max16065.ko 2024-03-28T00:03:15.832 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max1619.ko 2024-03-28T00:03:15.832 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max1668.ko 2024-03-28T00:03:15.832 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max197.ko 2024-03-28T00:03:15.832 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max6639.ko 2024-03-28T00:03:15.832 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max6650.ko 2024-03-28T00:03:15.832 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max6697.ko 2024-03-28T00:03:15.832 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/mcp3021.ko 2024-03-28T00:03:15.832 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/nct6683.ko 2024-03-28T00:03:15.832 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/nct6775-core.ko 2024-03-28T00:03:15.832 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/nct6775.ko 2024-03-28T00:03:15.832 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/nct7802.ko 2024-03-28T00:03:15.832 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pc87360.ko 2024-03-28T00:03:15.832 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pc87427.ko 2024-03-28T00:03:15.833 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pcf8591.ko 2024-03-28T00:03:15.833 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus 2024-03-28T00:03:15.833 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-03-28T00:03:15.833 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-03-28T00:03:15.833 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-03-28T00:03:15.833 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/max16064.ko 2024-03-28T00:03:15.833 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/max34440.ko 2024-03-28T00:03:15.833 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/max8688.ko 2024-03-28T00:03:15.833 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-03-28T00:03:15.833 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-03-28T00:03:15.833 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-03-28T00:03:15.833 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-03-28T00:03:15.834 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-03-28T00:03:15.834 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-03-28T00:03:15.834 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/powr1220.ko 2024-03-28T00:03:15.834 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sch5627.ko 2024-03-28T00:03:15.834 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sch5636.ko 2024-03-28T00:03:15.834 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sch56xx-common.ko 2024-03-28T00:03:15.834 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sht15.ko 2024-03-28T00:03:15.834 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sht21.ko 2024-03-28T00:03:15.834 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/shtc1.ko 2024-03-28T00:03:15.834 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sis5595.ko 2024-03-28T00:03:15.834 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/smsc47b397.ko 2024-03-28T00:03:15.834 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/smsc47m1.ko 2024-03-28T00:03:15.834 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/smsc47m192.ko 2024-03-28T00:03:15.835 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/thmc50.ko 2024-03-28T00:03:15.835 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/tmp102.ko 2024-03-28T00:03:15.835 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/tmp103.ko 2024-03-28T00:03:15.835 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/tmp401.ko 2024-03-28T00:03:15.835 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/tmp421.ko 2024-03-28T00:03:15.835 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/via-cputemp.ko 2024-03-28T00:03:15.835 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/via686a.ko 2024-03-28T00:03:15.835 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/vt1211.ko 2024-03-28T00:03:15.835 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/vt8231.ko 2024-03-28T00:03:15.835 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83627ehf.ko 2024-03-28T00:03:15.835 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83627hf.ko 2024-03-28T00:03:15.835 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83781d.ko 2024-03-28T00:03:15.835 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83791d.ko 2024-03-28T00:03:15.835 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83792d.ko 2024-03-28T00:03:15.836 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83793.ko 2024-03-28T00:03:15.836 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83795.ko 2024-03-28T00:03:15.836 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83l785ts.ko 2024-03-28T00:03:15.836 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83l786ng.ko 2024-03-28T00:03:15.836 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c 2024-03-28T00:03:15.836 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/algos 2024-03-28T00:03:15.836 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-03-28T00:03:15.836 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-03-28T00:03:15.836 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses 2024-03-28T00:03:15.836 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-03-28T00:03:15.836 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-03-28T00:03:15.836 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-03-28T00:03:15.836 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-03-28T00:03:15.837 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-03-28T00:03:15.837 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-03-28T00:03:15.837 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-03-28T00:03:15.837 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-03-28T00:03:15.837 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-i801.ko 2024-03-28T00:03:15.837 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-isch.ko 2024-03-28T00:03:15.837 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-03-28T00:03:15.837 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-03-28T00:03:15.837 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-03-28T00:03:15.837 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-parport.ko 2024-03-28T00:03:15.837 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-03-28T00:03:15.837 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-03-28T00:03:15.837 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-03-28T00:03:15.837 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-03-28T00:03:15.838 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-03-28T00:03:15.838 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-03-28T00:03:15.838 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-via.ko 2024-03-28T00:03:15.838 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-03-28T00:03:15.838 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-03-28T00:03:15.838 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-dev.ko 2024-03-28T00:03:15.838 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-mux.ko 2024-03-28T00:03:15.838 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-03-28T00:03:15.838 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-smbus.ko 2024-03-28T00:03:15.838 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-stub.ko 2024-03-28T00:03:15.838 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband 2024-03-28T00:03:15.838 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core 2024-03-28T00:03:15.838 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/ib_cm.ko 2024-03-28T00:03:15.839 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/ib_core.ko 2024-03-28T00:03:15.839 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/ib_umad.ko 2024-03-28T00:03:15.839 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-03-28T00:03:15.839 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/iw_cm.ko 2024-03-28T00:03:15.839 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/rdma_cm.ko 2024-03-28T00:03:15.839 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-03-28T00:03:15.839 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw 2024-03-28T00:03:15.839 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/cxgb4 2024-03-28T00:03:15.839 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-03-28T00:03:15.839 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mlx4 2024-03-28T00:03:15.839 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-03-28T00:03:15.839 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mlx5 2024-03-28T00:03:15.839 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-03-28T00:03:15.839 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mthca 2024-03-28T00:03:15.840 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-03-28T00:03:15.840 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp 2024-03-28T00:03:15.840 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/ipoib 2024-03-28T00:03:15.840 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-03-28T00:03:15.840 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/iser 2024-03-28T00:03:15.840 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-03-28T00:03:15.840 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/isert 2024-03-28T00:03:15.840 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-03-28T00:03:15.840 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/srp 2024-03-28T00:03:15.840 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-03-28T00:03:15.840 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/srpt 2024-03-28T00:03:15.840 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-03-28T00:03:15.840 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input 2024-03-28T00:03:15.840 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/gameport 2024-03-28T00:03:15.840 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-03-28T00:03:15.841 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/gameport/fm801-gp.ko 2024-03-28T00:03:15.841 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/gameport/gameport.ko 2024-03-28T00:03:15.841 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/joydev.ko 2024-03-28T00:03:15.841 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard 2024-03-28T00:03:15.841 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard/gpio_keys.ko 2024-03-28T00:03:15.841 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse 2024-03-28T00:03:15.841 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/appletouch.ko 2024-03-28T00:03:15.841 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/bcm5974.ko 2024-03-28T00:03:15.841 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/cyapatp.ko 2024-03-28T00:03:15.841 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/elan_i2c.ko 2024-03-28T00:03:15.841 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/sermouse.ko 2024-03-28T00:03:15.841 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-03-28T00:03:15.841 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/synaptics_usb.ko 2024-03-28T00:03:15.841 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/vsxxxaa.ko 2024-03-28T00:03:15.842 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/rmi4 2024-03-28T00:03:15.842 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/rmi4/rmi_core.ko 2024-03-28T00:03:15.842 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio 2024-03-28T00:03:15.842 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/altera_ps2.ko 2024-03-28T00:03:15.842 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/arc_ps2.ko 2024-03-28T00:03:15.842 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-03-28T00:03:15.842 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/serio_raw.ko 2024-03-28T00:03:15.842 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/sparse-keymap.ko 2024-03-28T00:03:15.842 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds 2024-03-28T00:03:15.842 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-blinkm.ko 2024-03-28T00:03:15.842 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-lm3530.ko 2024-03-28T00:03:15.842 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-lp3944.ko 2024-03-28T00:03:15.842 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-ss4200.ko 2024-03-28T00:03:15.843 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger 2024-03-28T00:03:15.843 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-03-28T00:03:15.843 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-03-28T00:03:15.843 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-03-28T00:03:15.843 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-03-28T00:03:15.843 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-03-28T00:03:15.843 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-03-28T00:03:15.843 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-03-28T00:03:15.843 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-03-28T00:03:15.843 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-03-28T00:03:15.843 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md 2024-03-28T00:03:15.843 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/bcache 2024-03-28T00:03:15.843 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/bcache/bcache.ko 2024-03-28T00:03:15.843 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-bio-prison.ko 2024-03-28T00:03:15.843 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-cache-smq.ko 2024-03-28T00:03:15.844 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-cache.ko 2024-03-28T00:03:15.844 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-crypt.ko 2024-03-28T00:03:15.844 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-delay.ko 2024-03-28T00:03:15.844 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-flakey.ko 2024-03-28T00:03:15.844 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-log-userspace.ko 2024-03-28T00:03:15.844 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-multipath.ko 2024-03-28T00:03:15.844 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-queue-length.ko 2024-03-28T00:03:15.844 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-raid.ko 2024-03-28T00:03:15.844 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-round-robin.ko 2024-03-28T00:03:15.844 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-service-time.ko 2024-03-28T00:03:15.844 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-switch.ko 2024-03-28T00:03:15.844 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-thin-pool.ko 2024-03-28T00:03:15.844 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-verity.ko 2024-03-28T00:03:15.844 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/persistent-data 2024-03-28T00:03:15.844 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-03-28T00:03:15.845 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/raid0.ko 2024-03-28T00:03:15.845 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/raid1.ko 2024-03-28T00:03:15.845 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/raid10.ko 2024-03-28T00:03:15.845 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/raid456.ko 2024-03-28T00:03:15.845 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message 2024-03-28T00:03:15.845 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion 2024-03-28T00:03:15.845 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptbase.ko 2024-03-28T00:03:15.845 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptctl.ko 2024-03-28T00:03:15.845 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptfc.ko 2024-03-28T00:03:15.845 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptlan.ko 2024-03-28T00:03:15.845 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptsas.ko 2024-03-28T00:03:15.845 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptscsih.ko 2024-03-28T00:03:15.845 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptspi.ko 2024-03-28T00:03:15.846 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd 2024-03-28T00:03:15.846 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/lpc_ich.ko 2024-03-28T00:03:15.846 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/lpc_sch.ko 2024-03-28T00:03:15.846 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/mfd-core.ko 2024-03-28T00:03:15.846 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/sm501.ko 2024-03-28T00:03:15.846 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/viperboard.ko 2024-03-28T00:03:15.846 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/vx855.ko 2024-03-28T00:03:15.846 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/wl1273-core.ko 2024-03-28T00:03:15.846 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc 2024-03-28T00:03:15.846 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/altera-stapl 2024-03-28T00:03:15.846 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-03-28T00:03:15.846 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/apds9802als.ko 2024-03-28T00:03:15.847 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/apds990x.ko 2024-03-28T00:03:15.847 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/bh1770glc.ko 2024-03-28T00:03:15.847 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/cb710 2024-03-28T00:03:15.847 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/cb710/cb710.ko 2024-03-28T00:03:15.847 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/eeprom 2024-03-28T00:03:15.847 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/eeprom/at24.ko 2024-03-28T00:03:15.847 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-03-28T00:03:15.847 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/eeprom/max6875.ko 2024-03-28T00:03:15.847 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/enclosure.ko 2024-03-28T00:03:15.847 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/hpilo.ko 2024-03-28T00:03:15.847 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/isl29003.ko 2024-03-28T00:03:15.847 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/isl29020.ko 2024-03-28T00:03:15.847 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/lis3lv02d 2024-03-28T00:03:15.847 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-03-28T00:03:15.848 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-03-28T00:03:15.848 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/mei 2024-03-28T00:03:15.848 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/mei/mei-me.ko 2024-03-28T00:03:15.848 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/mei/mei-txe.ko 2024-03-28T00:03:15.848 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/mei/mei.ko 2024-03-28T00:03:15.848 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-gru 2024-03-28T00:03:15.848 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-gru/gru.ko 2024-03-28T00:03:15.848 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-xp 2024-03-28T00:03:15.848 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-xp/xp.ko 2024-03-28T00:03:15.848 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-xp/xpc.ko 2024-03-28T00:03:15.848 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-03-28T00:03:15.848 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/tifm_7xx1.ko 2024-03-28T00:03:15.848 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/tifm_core.ko 2024-03-28T00:03:15.848 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/tsl2550.ko 2024-03-28T00:03:15.849 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/vmw_balloon.ko 2024-03-28T00:03:15.849 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/vmw_vmci 2024-03-28T00:03:15.849 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-03-28T00:03:15.849 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net 2024-03-28T00:03:15.849 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/bonding 2024-03-28T00:03:15.849 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/bonding/bonding.ko 2024-03-28T00:03:15.849 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa 2024-03-28T00:03:15.849 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/b53 2024-03-28T00:03:15.849 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/b53/b53_common.ko 2024-03-28T00:03:15.849 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/bcm-sf2.ko 2024-03-28T00:03:15.849 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/mv88e6060.ko 2024-03-28T00:03:15.849 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/mv88e6xxx 2024-03-28T00:03:15.849 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-03-28T00:03:15.849 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dummy.ko 2024-03-28T00:03:15.849 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/eql.ko 2024-03-28T00:03:15.850 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet 2024-03-28T00:03:15.850 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com 2024-03-28T00:03:15.850 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-03-28T00:03:15.850 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-03-28T00:03:15.850 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-03-28T00:03:15.850 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-03-28T00:03:15.850 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390 2024-03-28T00:03:15.850 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390/8390.ko 2024-03-28T00:03:15.850 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-03-28T00:03:15.850 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-03-28T00:03:15.850 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-03-28T00:03:15.850 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/adaptec 2024-03-28T00:03:15.850 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-03-28T00:03:15.851 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/agere 2024-03-28T00:03:15.851 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/agere/et131x.ko 2024-03-28T00:03:15.851 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/alteon 2024-03-28T00:03:15.851 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-03-28T00:03:15.851 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/altera 2024-03-28T00:03:15.851 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-03-28T00:03:15.851 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/amd 2024-03-28T00:03:15.851 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-03-28T00:03:15.851 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-03-28T00:03:15.851 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-03-28T00:03:15.851 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros 2024-03-28T00:03:15.851 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/alx 2024-03-28T00:03:15.851 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-03-28T00:03:15.851 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atl1c 2024-03-28T00:03:15.851 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-03-28T00:03:15.852 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atl1e 2024-03-28T00:03:15.852 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-03-28T00:03:15.852 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atlx 2024-03-28T00:03:15.852 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-03-28T00:03:15.852 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-03-28T00:03:15.852 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom 2024-03-28T00:03:15.852 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-03-28T00:03:15.852 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-03-28T00:03:15.852 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-03-28T00:03:15.852 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-03-28T00:03:15.852 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-03-28T00:03:15.852 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/genet 2024-03-28T00:03:15.852 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-03-28T00:03:15.852 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-03-28T00:03:15.853 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/brocade 2024-03-28T00:03:15.853 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/brocade/bna 2024-03-28T00:03:15.853 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-03-28T00:03:15.853 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio 2024-03-28T00:03:15.853 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb 2024-03-28T00:03:15.853 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-03-28T00:03:15.853 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-03-28T00:03:15.853 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-03-28T00:03:15.853 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-03-28T00:03:15.853 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-03-28T00:03:15.853 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-03-28T00:03:15.853 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-03-28T00:03:15.853 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-03-28T00:03:15.853 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-03-28T00:03:15.854 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/cisco 2024-03-28T00:03:15.854 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/cisco/enic 2024-03-28T00:03:15.854 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-03-28T00:03:15.854 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec 2024-03-28T00:03:15.854 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip 2024-03-28T00:03:15.854 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-03-28T00:03:15.854 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-03-28T00:03:15.854 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-03-28T00:03:15.854 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-03-28T00:03:15.854 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-03-28T00:03:15.854 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-03-28T00:03:15.854 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dlink 2024-03-28T00:03:15.854 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-03-28T00:03:15.855 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-03-28T00:03:15.855 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dnet.ko 2024-03-28T00:03:15.855 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/emulex 2024-03-28T00:03:15.855 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/emulex/benet 2024-03-28T00:03:15.855 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-03-28T00:03:15.855 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/ethoc.ko 2024-03-28T00:03:15.855 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/fealnx.ko 2024-03-28T00:03:15.855 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel 2024-03-28T00:03:15.855 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e100.ko 2024-03-28T00:03:15.855 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e1000 2024-03-28T00:03:15.855 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-03-28T00:03:15.855 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e1000e 2024-03-28T00:03:15.855 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-03-28T00:03:15.856 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/fm10k 2024-03-28T00:03:15.856 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-03-28T00:03:15.856 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/i40e 2024-03-28T00:03:15.856 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-03-28T00:03:15.856 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/iavf 2024-03-28T00:03:15.856 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-03-28T00:03:15.856 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/igb 2024-03-28T00:03:15.856 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-03-28T00:03:15.856 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/igbvf 2024-03-28T00:03:15.856 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-03-28T00:03:15.856 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/ixgbe 2024-03-28T00:03:15.856 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-03-28T00:03:15.856 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/ixgbevf 2024-03-28T00:03:15.856 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-03-28T00:03:15.857 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/jme.ko 2024-03-28T00:03:15.857 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/marvell 2024-03-28T00:03:15.857 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-03-28T00:03:15.857 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/marvell/skge.ko 2024-03-28T00:03:15.857 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-03-28T00:03:15.857 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox 2024-03-28T00:03:15.857 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx4 2024-03-28T00:03:15.857 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-03-28T00:03:15.857 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-03-28T00:03:15.857 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx5 2024-03-28T00:03:15.857 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-03-28T00:03:15.857 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-03-28T00:03:15.857 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/micrel 2024-03-28T00:03:15.857 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-03-28T00:03:15.858 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/myricom 2024-03-28T00:03:15.858 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/myricom/myri10ge 2024-03-28T00:03:15.858 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-03-28T00:03:15.858 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/natsemi 2024-03-28T00:03:15.858 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-03-28T00:03:15.858 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-03-28T00:03:15.858 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/neterion 2024-03-28T00:03:15.858 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-03-28T00:03:15.858 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/nvidia 2024-03-28T00:03:15.858 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-03-28T00:03:15.858 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/packetengines 2024-03-28T00:03:15.858 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-03-28T00:03:15.858 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-03-28T00:03:15.859 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic 2024-03-28T00:03:15.859 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/netxen 2024-03-28T00:03:15.859 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-03-28T00:03:15.859 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-03-28T00:03:15.859 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-03-28T00:03:15.859 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-03-28T00:03:15.859 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/rdc 2024-03-28T00:03:15.859 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-03-28T00:03:15.859 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek 2024-03-28T00:03:15.859 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-03-28T00:03:15.859 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-03-28T00:03:15.859 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek/atp.ko 2024-03-28T00:03:15.859 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-03-28T00:03:15.860 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/rocker 2024-03-28T00:03:15.860 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-03-28T00:03:15.860 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sfc 2024-03-28T00:03:15.860 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-03-28T00:03:15.860 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/silan 2024-03-28T00:03:15.860 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-03-28T00:03:15.860 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sis 2024-03-28T00:03:15.860 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sis/sis190.ko 2024-03-28T00:03:15.860 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sis/sis900.ko 2024-03-28T00:03:15.860 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc 2024-03-28T00:03:15.860 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-03-28T00:03:15.860 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-03-28T00:03:15.860 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-03-28T00:03:15.861 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-03-28T00:03:15.861 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro 2024-03-28T00:03:15.861 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro/stmmac 2024-03-28T00:03:15.861 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-03-28T00:03:15.861 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-03-28T00:03:15.861 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun 2024-03-28T00:03:15.861 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun/cassini.ko 2024-03-28T00:03:15.861 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun/niu.ko 2024-03-28T00:03:15.861 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun/sungem.ko 2024-03-28T00:03:15.861 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-03-28T00:03:15.861 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/tehuti 2024-03-28T00:03:15.861 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-03-28T00:03:15.861 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/ti 2024-03-28T00:03:15.861 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/ti/tlan.ko 2024-03-28T00:03:15.861 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/via 2024-03-28T00:03:15.861 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-03-28T00:03:15.862 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-03-28T00:03:15.862 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/wiznet 2024-03-28T00:03:15.862 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-03-28T00:03:15.862 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-03-28T00:03:15.862 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/xircom 2024-03-28T00:03:15.862 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-03-28T00:03:15.862 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/geneve.ko 2024-03-28T00:03:15.862 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/hyperv 2024-03-28T00:03:15.862 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-03-28T00:03:15.862 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ieee802154 2024-03-28T00:03:15.862 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ieee802154/fakelb.ko 2024-03-28T00:03:15.862 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ifb.ko 2024-03-28T00:03:15.862 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ipvlan 2024-03-28T00:03:15.862 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ipvlan/ipvlan.ko 2024-03-28T00:03:15.863 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/macvlan.ko 2024-03-28T00:03:15.863 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/macvtap.ko 2024-03-28T00:03:15.863 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio 2024-03-28T00:03:15.863 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio.ko 2024-03-28T00:03:15.863 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-03-28T00:03:15.863 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-03-28T00:03:15.863 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio/mdio-regmap.ko 2024-03-28T00:03:15.863 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mii.ko 2024-03-28T00:03:15.863 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/net_failover.ko 2024-03-28T00:03:15.863 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/netconsole.ko 2024-03-28T00:03:15.863 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/nlmon.ko 2024-03-28T00:03:15.863 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/pcs 2024-03-28T00:03:15.863 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/pcs/pcs-lynx.ko 2024-03-28T00:03:15.863 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-03-28T00:03:15.863 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy 2024-03-28T00:03:15.864 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/amd.ko 2024-03-28T00:03:15.864 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-03-28T00:03:15.864 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-03-28T00:03:15.864 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/bcm7xxx.ko 2024-03-28T00:03:15.864 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/bcm87xx.ko 2024-03-28T00:03:15.864 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/broadcom.ko 2024-03-28T00:03:15.864 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/cicada.ko 2024-03-28T00:03:15.864 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/davicom.ko 2024-03-28T00:03:15.864 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/dp83640.ko 2024-03-28T00:03:15.864 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/et1011c.ko 2024-03-28T00:03:15.864 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/icplus.ko 2024-03-28T00:03:15.864 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/lxt.ko 2024-03-28T00:03:15.864 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/marvell.ko 2024-03-28T00:03:15.865 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/micrel.ko 2024-03-28T00:03:15.865 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/national.ko 2024-03-28T00:03:15.865 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/phylink.ko 2024-03-28T00:03:15.865 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/qsemi.ko 2024-03-28T00:03:15.865 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/realtek.ko 2024-03-28T00:03:15.865 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/smsc.ko 2024-03-28T00:03:15.865 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/ste10Xp.ko 2024-03-28T00:03:15.865 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/vitesse.ko 2024-03-28T00:03:15.865 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp 2024-03-28T00:03:15.865 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/bsd_comp.ko 2024-03-28T00:03:15.865 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_async.ko 2024-03-28T00:03:15.865 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_deflate.ko 2024-03-28T00:03:15.865 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_generic.ko 2024-03-28T00:03:15.865 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_mppe.ko 2024-03-28T00:03:15.865 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_synctty.ko 2024-03-28T00:03:15.866 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/pppoe.ko 2024-03-28T00:03:15.866 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/pppox.ko 2024-03-28T00:03:15.866 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/pptp.ko 2024-03-28T00:03:15.866 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/slip 2024-03-28T00:03:15.866 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/slip/slhc.ko 2024-03-28T00:03:15.866 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/slip/slip.ko 2024-03-28T00:03:15.866 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/sungem_phy.ko 2024-03-28T00:03:15.866 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/tap.ko 2024-03-28T00:03:15.866 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team 2024-03-28T00:03:15.866 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team.ko 2024-03-28T00:03:15.866 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_activebackup.ko 2024-03-28T00:03:15.866 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_broadcast.ko 2024-03-28T00:03:15.866 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-03-28T00:03:15.867 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_random.ko 2024-03-28T00:03:15.867 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-03-28T00:03:15.867 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/tun.ko 2024-03-28T00:03:15.867 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/veth.ko 2024-03-28T00:03:15.867 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/virtio_net.ko 2024-03-28T00:03:15.867 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/vmxnet3 2024-03-28T00:03:15.867 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-03-28T00:03:15.867 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/vxlan 2024-03-28T00:03:15.867 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/vxlan/vxlan.ko 2024-03-28T00:03:15.867 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/xen-netback 2024-03-28T00:03:15.867 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/xen-netback/xen-netback.ko 2024-03-28T00:03:15.867 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/xen-netfront.ko 2024-03-28T00:03:15.867 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ntb 2024-03-28T00:03:15.867 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ntb/ntb.ko 2024-03-28T00:03:15.868 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/nvme 2024-03-28T00:03:15.868 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/nvme/host 2024-03-28T00:03:15.868 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/nvme/host/nvme-core.ko 2024-03-28T00:03:15.868 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/nvme/host/nvme.ko 2024-03-28T00:03:15.868 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport 2024-03-28T00:03:15.868 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport/parport.ko 2024-03-28T00:03:15.868 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport/parport_cs.ko 2024-03-28T00:03:15.868 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport/parport_pc.ko 2024-03-28T00:03:15.868 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport/parport_serial.ko 2024-03-28T00:03:15.868 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci 2024-03-28T00:03:15.868 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/hotplug 2024-03-28T00:03:15.868 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-03-28T00:03:15.868 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/pcie 2024-03-28T00:03:15.868 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/pcie/aer_inject.ko 2024-03-28T00:03:15.868 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/xen-pcifront.ko 2024-03-28T00:03:15.869 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia 2024-03-28T00:03:15.869 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia/i82092.ko 2024-03-28T00:03:15.869 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia/pd6729.ko 2024-03-28T00:03:15.869 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia/yenta_socket.ko 2024-03-28T00:03:15.869 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pinctrl 2024-03-28T00:03:15.869 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pinctrl/intel 2024-03-28T00:03:15.869 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-03-28T00:03:15.869 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform 2024-03-28T00:03:15.869 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86 2024-03-28T00:03:15.869 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/acer-wmi.ko 2024-03-28T00:03:15.869 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/acerhdf.ko 2024-03-28T00:03:15.869 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/apple-gmux.ko 2024-03-28T00:03:15.869 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/asus-laptop.ko 2024-03-28T00:03:15.869 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-03-28T00:03:15.869 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/asus-wmi.ko 2024-03-28T00:03:15.870 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/classmate-laptop.ko 2024-03-28T00:03:15.870 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-03-28T00:03:15.870 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-03-28T00:03:15.870 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-03-28T00:03:15.870 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-03-28T00:03:15.870 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/hdaps.ko 2024-03-28T00:03:15.870 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/intel 2024-03-28T00:03:15.870 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-03-28T00:03:15.870 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/intel_ips.ko 2024-03-28T00:03:15.870 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/msi-wmi.ko 2024-03-28T00:03:15.870 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/mxm-wmi.ko 2024-03-28T00:03:15.870 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-03-28T00:03:15.870 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/samsung-laptop.ko 2024-03-28T00:03:15.870 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/samsung-q10.ko 2024-03-28T00:03:15.871 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/topstar-laptop.ko 2024-03-28T00:03:15.871 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-03-28T00:03:15.871 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/toshiba_haps.ko 2024-03-28T00:03:15.871 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/wmi-bmof.ko 2024-03-28T00:03:15.871 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/wmi.ko 2024-03-28T00:03:15.871 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/powercap 2024-03-28T00:03:15.871 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/powercap/intel_rapl_common.ko 2024-03-28T00:03:15.871 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/powercap/intel_rapl_msr.ko 2024-03-28T00:03:15.871 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps 2024-03-28T00:03:15.871 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/clients 2024-03-28T00:03:15.871 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/clients/pps-gpio.ko 2024-03-28T00:03:15.871 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/clients/pps-ldisc.ko 2024-03-28T00:03:15.871 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/clients/pps_parport.ko 2024-03-28T00:03:15.872 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/pps_core.ko 2024-03-28T00:03:15.872 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ptp 2024-03-28T00:03:15.872 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ptp/ptp.ko 2024-03-28T00:03:15.872 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ptp/ptp_kvm.ko 2024-03-28T00:03:15.872 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc 2024-03-28T00:03:15.872 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-bq32k.ko 2024-03-28T00:03:15.872 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1286.ko 2024-03-28T00:03:15.872 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1307.ko 2024-03-28T00:03:15.872 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1374.ko 2024-03-28T00:03:15.872 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1511.ko 2024-03-28T00:03:15.872 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1553.ko 2024-03-28T00:03:15.872 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1672.ko 2024-03-28T00:03:15.872 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1742.ko 2024-03-28T00:03:15.872 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds2404.ko 2024-03-28T00:03:15.873 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds3232.ko 2024-03-28T00:03:15.873 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-em3027.ko 2024-03-28T00:03:15.873 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-fm3130.ko 2024-03-28T00:03:15.873 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-isl12022.ko 2024-03-28T00:03:15.873 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-isl1208.ko 2024-03-28T00:03:15.873 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-m41t80.ko 2024-03-28T00:03:15.873 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-m48t35.ko 2024-03-28T00:03:15.873 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-m48t59.ko 2024-03-28T00:03:15.873 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-max6900.ko 2024-03-28T00:03:15.873 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-msm6242.ko 2024-03-28T00:03:15.873 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf2127.ko 2024-03-28T00:03:15.873 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf85063.ko 2024-03-28T00:03:15.873 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf8523.ko 2024-03-28T00:03:15.874 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf8563.ko 2024-03-28T00:03:15.874 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf8583.ko 2024-03-28T00:03:15.874 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rp5c01.ko 2024-03-28T00:03:15.874 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rs5c372.ko 2024-03-28T00:03:15.874 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-03-28T00:03:15.874 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rx8025.ko 2024-03-28T00:03:15.874 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rx8581.ko 2024-03-28T00:03:15.874 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-03-28T00:03:15.874 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-x1205.ko 2024-03-28T00:03:15.874 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi 2024-03-28T00:03:15.874 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/3w-9xxx.ko 2024-03-28T00:03:15.874 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/3w-sas.ko 2024-03-28T00:03:15.874 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/3w-xxxx.ko 2024-03-28T00:03:15.875 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/BusLogic.ko 2024-03-28T00:03:15.875 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/a100u2w.ko 2024-03-28T00:03:15.875 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aacraid 2024-03-28T00:03:15.875 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aacraid/aacraid.ko 2024-03-28T00:03:15.875 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/advansys.ko 2024-03-28T00:03:15.875 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aic7xxx 2024-03-28T00:03:15.875 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-03-28T00:03:15.875 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-03-28T00:03:15.875 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/am53c974.ko 2024-03-28T00:03:15.875 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/arcmsr 2024-03-28T00:03:15.875 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-03-28T00:03:15.875 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/atp870u.ko 2024-03-28T00:03:15.875 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/be2iscsi 2024-03-28T00:03:15.876 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-03-28T00:03:15.876 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bfa 2024-03-28T00:03:15.876 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bfa/bfa.ko 2024-03-28T00:03:15.876 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bnx2fc 2024-03-28T00:03:15.876 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-03-28T00:03:15.876 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bnx2i 2024-03-28T00:03:15.876 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-03-28T00:03:15.876 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ch.ko 2024-03-28T00:03:15.876 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/csiostor 2024-03-28T00:03:15.876 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/csiostor/csiostor.ko 2024-03-28T00:03:15.876 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi 2024-03-28T00:03:15.876 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/cxgb3i 2024-03-28T00:03:15.876 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-03-28T00:03:15.877 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/cxgb4i 2024-03-28T00:03:15.877 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-03-28T00:03:15.877 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-03-28T00:03:15.877 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/dc395x.ko 2024-03-28T00:03:15.877 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler 2024-03-28T00:03:15.877 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-03-28T00:03:15.877 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-03-28T00:03:15.877 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-03-28T00:03:15.877 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-03-28T00:03:15.877 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/dmx3191d.ko 2024-03-28T00:03:15.877 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/esas2r 2024-03-28T00:03:15.877 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/esas2r/esas2r.ko 2024-03-28T00:03:15.878 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/esp_scsi.ko 2024-03-28T00:03:15.878 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fcoe 2024-03-28T00:03:15.878 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fcoe/fcoe.ko 2024-03-28T00:03:15.878 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-03-28T00:03:15.878 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fnic 2024-03-28T00:03:15.878 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fnic/fnic.ko 2024-03-28T00:03:15.878 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/hpsa.ko 2024-03-28T00:03:15.878 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/hptiop.ko 2024-03-28T00:03:15.878 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/hv_storvsc.ko 2024-03-28T00:03:15.878 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/initio.ko 2024-03-28T00:03:15.878 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ipr.ko 2024-03-28T00:03:15.878 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ips.ko 2024-03-28T00:03:15.879 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/isci 2024-03-28T00:03:15.879 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/isci/isci.ko 2024-03-28T00:03:15.879 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-03-28T00:03:15.879 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/iscsi_tcp.ko 2024-03-28T00:03:15.879 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libfc 2024-03-28T00:03:15.879 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libfc/libfc.ko 2024-03-28T00:03:15.879 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libiscsi.ko 2024-03-28T00:03:15.879 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libiscsi_tcp.ko 2024-03-28T00:03:15.879 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libsas 2024-03-28T00:03:15.879 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libsas/libsas.ko 2024-03-28T00:03:15.879 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/lpfc 2024-03-28T00:03:15.879 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/lpfc/lpfc.ko 2024-03-28T00:03:15.879 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid 2024-03-28T00:03:15.879 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid.ko 2024-03-28T00:03:15.879 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-03-28T00:03:15.880 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-03-28T00:03:15.880 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-03-28T00:03:15.880 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mpt3sas 2024-03-28T00:03:15.880 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-03-28T00:03:15.880 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mvsas 2024-03-28T00:03:15.880 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mvsas/mvsas.ko 2024-03-28T00:03:15.880 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mvumi.ko 2024-03-28T00:03:15.880 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/pm8001 2024-03-28T00:03:15.880 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-03-28T00:03:15.880 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/pmcraid.ko 2024-03-28T00:03:15.880 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla1280.ko 2024-03-28T00:03:15.880 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla2xxx 2024-03-28T00:03:15.880 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-03-28T00:03:15.880 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-03-28T00:03:15.880 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla4xxx 2024-03-28T00:03:15.881 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-03-28T00:03:15.881 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/raid_class.ko 2024-03-28T00:03:15.881 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_debug.ko 2024-03-28T00:03:15.881 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_fc.ko 2024-03-28T00:03:15.881 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-03-28T00:03:15.881 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_sas.ko 2024-03-28T00:03:15.881 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_spi.ko 2024-03-28T00:03:15.881 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_srp.ko 2024-03-28T00:03:15.881 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ses.ko 2024-03-28T00:03:15.881 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/st.ko 2024-03-28T00:03:15.881 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/stex.ko 2024-03-28T00:03:15.881 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/sym53c8xx_2 2024-03-28T00:03:15.881 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-03-28T00:03:15.881 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/virtio_scsi.ko 2024-03-28T00:03:15.882 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/vmw_pvscsi.ko 2024-03-28T00:03:15.882 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/wd719x.ko 2024-03-28T00:03:15.882 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ssb 2024-03-28T00:03:15.882 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ssb/ssb.ko 2024-03-28T00:03:15.882 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target 2024-03-28T00:03:15.882 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/iscsi 2024-03-28T00:03:15.882 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-03-28T00:03:15.882 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/loopback 2024-03-28T00:03:15.882 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/loopback/tcm_loop.ko 2024-03-28T00:03:15.882 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_file.ko 2024-03-28T00:03:15.882 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_iblock.ko 2024-03-28T00:03:15.882 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_mod.ko 2024-03-28T00:03:15.883 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_pscsi.ko 2024-03-28T00:03:15.883 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_user.ko 2024-03-28T00:03:15.883 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/tcm_fc 2024-03-28T00:03:15.883 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-03-28T00:03:15.883 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal 2024-03-28T00:03:15.883 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel 2024-03-28T00:03:15.883 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal 2024-03-28T00:03:15.883 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-03-28T00:03:15.883 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-03-28T00:03:15.883 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-03-28T00:03:15.883 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-03-28T00:03:15.883 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-03-28T00:03:15.883 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-03-28T00:03:15.884 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-03-28T00:03:15.884 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-03-28T00:03:15.884 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-03-28T00:03:15.884 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-03-28T00:03:15.884 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-03-28T00:03:15.884 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-03-28T00:03:15.884 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-03-28T00:03:15.884 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-03-28T00:03:15.884 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-03-28T00:03:15.884 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-03-28T00:03:15.884 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-03-28T00:03:15.884 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-03-28T00:03:15.884 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty 2024-03-28T00:03:15.885 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/ipwireless 2024-03-28T00:03:15.885 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-03-28T00:03:15.885 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/n_gsm.ko 2024-03-28T00:03:15.885 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/n_hdlc.ko 2024-03-28T00:03:15.885 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/nozomi.ko 2024-03-28T00:03:15.885 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial 2024-03-28T00:03:15.885 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/8250 2024-03-28T00:03:15.885 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-03-28T00:03:15.885 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/arc_uart.ko 2024-03-28T00:03:15.885 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/jsm 2024-03-28T00:03:15.885 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/jsm/jsm.ko 2024-03-28T00:03:15.885 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs 2024-03-28T00:03:15.885 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/core 2024-03-28T00:03:15.886 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/core/ufshcd-core.ko 2024-03-28T00:03:15.886 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/host 2024-03-28T00:03:15.886 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-03-28T00:03:15.886 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio 2024-03-28T00:03:15.886 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio.ko 2024-03-28T00:03:15.886 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_aec.ko 2024-03-28T00:03:15.886 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_cif.ko 2024-03-28T00:03:15.886 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_pci_generic.ko 2024-03-28T00:03:15.886 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_sercos3.ko 2024-03-28T00:03:15.886 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb 2024-03-28T00:03:15.886 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class 2024-03-28T00:03:15.886 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class/cdc-acm.ko 2024-03-28T00:03:15.886 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class/cdc-wdm.ko 2024-03-28T00:03:15.887 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class/usblp.ko 2024-03-28T00:03:15.887 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class/usbtmc.ko 2024-03-28T00:03:15.887 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/host 2024-03-28T00:03:15.887 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/host/sl811-hcd.ko 2024-03-28T00:03:15.887 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/image 2024-03-28T00:03:15.887 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/image/mdc800.ko 2024-03-28T00:03:15.887 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/image/microtek.ko 2024-03-28T00:03:15.887 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc 2024-03-28T00:03:15.887 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/adutux.ko 2024-03-28T00:03:15.887 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/appledisplay.ko 2024-03-28T00:03:15.887 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/emi26.ko 2024-03-28T00:03:15.887 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/emi62.ko 2024-03-28T00:03:15.887 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/ezusb.ko 2024-03-28T00:03:15.887 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/idmouse.ko 2024-03-28T00:03:15.888 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/iowarrior.ko 2024-03-28T00:03:15.888 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/isight_firmware.ko 2024-03-28T00:03:15.888 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/ldusb.ko 2024-03-28T00:03:15.888 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/legousbtower.ko 2024-03-28T00:03:15.888 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/sisusbvga 2024-03-28T00:03:15.888 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-03-28T00:03:15.888 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/trancevibrator.ko 2024-03-28T00:03:15.888 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/usb3503.ko 2024-03-28T00:03:15.888 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/usblcd.ko 2024-03-28T00:03:15.888 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/usbsevseg.ko 2024-03-28T00:03:15.888 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/uss720.ko 2024-03-28T00:03:15.888 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/yurex.ko 2024-03-28T00:03:15.888 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/phy 2024-03-28T00:03:15.888 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/phy/phy-generic.ko 2024-03-28T00:03:15.889 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial 2024-03-28T00:03:15.889 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/aircable.ko 2024-03-28T00:03:15.889 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ark3116.ko 2024-03-28T00:03:15.889 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/belkin_sa.ko 2024-03-28T00:03:15.889 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ch341.ko 2024-03-28T00:03:15.889 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/cp210x.ko 2024-03-28T00:03:15.889 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/cyberjack.ko 2024-03-28T00:03:15.889 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/cypress_m8.ko 2024-03-28T00:03:15.889 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/digi_acceleport.ko 2024-03-28T00:03:15.889 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/empeg.ko 2024-03-28T00:03:15.889 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ftdi_sio.ko 2024-03-28T00:03:15.889 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/garmin_gps.ko 2024-03-28T00:03:15.889 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/io_edgeport.ko 2024-03-28T00:03:15.889 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/io_ti.ko 2024-03-28T00:03:15.890 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ipaq.ko 2024-03-28T00:03:15.890 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ipw.ko 2024-03-28T00:03:15.890 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ir-usb.ko 2024-03-28T00:03:15.890 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-03-28T00:03:15.890 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/keyspan.ko 2024-03-28T00:03:15.890 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/keyspan_pda.ko 2024-03-28T00:03:15.890 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/kl5kusb105.ko 2024-03-28T00:03:15.890 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/kobil_sct.ko 2024-03-28T00:03:15.890 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/mct_u232.ko 2024-03-28T00:03:15.890 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/mos7720.ko 2024-03-28T00:03:15.890 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/mos7840.ko 2024-03-28T00:03:15.890 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/navman.ko 2024-03-28T00:03:15.890 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/omninet.ko 2024-03-28T00:03:15.891 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/opticon.ko 2024-03-28T00:03:15.891 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/option.ko 2024-03-28T00:03:15.891 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/oti6858.ko 2024-03-28T00:03:15.891 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/pl2303.ko 2024-03-28T00:03:15.891 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/qcaux.ko 2024-03-28T00:03:15.891 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/qcserial.ko 2024-03-28T00:03:15.891 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/quatech2.ko 2024-03-28T00:03:15.891 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/safe_serial.ko 2024-03-28T00:03:15.891 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/sierra.ko 2024-03-28T00:03:15.891 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/spcp8x5.ko 2024-03-28T00:03:15.891 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ssu100.ko 2024-03-28T00:03:15.891 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/symbolserial.ko 2024-03-28T00:03:15.892 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-03-28T00:03:15.892 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-03-28T00:03:15.892 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/usb_debug.ko 2024-03-28T00:03:15.892 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/usb_wwan.ko 2024-03-28T00:03:15.892 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/visor.ko 2024-03-28T00:03:15.892 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/whiteheat.ko 2024-03-28T00:03:15.892 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/xsens_mt.ko 2024-03-28T00:03:15.892 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage 2024-03-28T00:03:15.892 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/uas.ko 2024-03-28T00:03:15.892 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-alauda.ko 2024-03-28T00:03:15.892 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-cypress.ko 2024-03-28T00:03:15.892 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-datafab.ko 2024-03-28T00:03:15.892 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-03-28T00:03:15.892 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-freecom.ko 2024-03-28T00:03:15.892 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-isd200.ko 2024-03-28T00:03:15.893 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-03-28T00:03:15.893 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-karma.ko 2024-03-28T00:03:15.893 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-onetouch.ko 2024-03-28T00:03:15.893 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-realtek.ko 2024-03-28T00:03:15.893 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-sddr09.ko 2024-03-28T00:03:15.893 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-sddr55.ko 2024-03-28T00:03:15.893 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-usbat.ko 2024-03-28T00:03:15.893 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/usb-storage.ko 2024-03-28T00:03:15.893 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/usbip 2024-03-28T00:03:15.893 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/usbip/usbip-core.ko 2024-03-28T00:03:15.893 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/usbip/usbip-host.ko 2024-03-28T00:03:15.893 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-03-28T00:03:15.893 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio 2024-03-28T00:03:15.893 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/pci 2024-03-28T00:03:15.894 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-03-28T00:03:15.894 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/pci/vfio-pci.ko 2024-03-28T00:03:15.894 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/vfio.ko 2024-03-28T00:03:15.894 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-03-28T00:03:15.894 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost 2024-03-28T00:03:15.894 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost/vhost.ko 2024-03-28T00:03:15.894 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost/vhost_iotlb.ko 2024-03-28T00:03:15.894 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost/vhost_net.ko 2024-03-28T00:03:15.894 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost/vhost_scsi.ko 2024-03-28T00:03:15.894 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video 2024-03-28T00:03:15.894 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight 2024-03-28T00:03:15.894 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/apple_bl.ko 2024-03-28T00:03:15.894 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/lcd.ko 2024-03-28T00:03:15.894 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/platform_lcd.ko 2024-03-28T00:03:15.895 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio 2024-03-28T00:03:15.895 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio.ko 2024-03-28T00:03:15.895 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_balloon.ko 2024-03-28T00:03:15.895 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_mmio.ko 2024-03-28T00:03:15.895 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_pci.ko 2024-03-28T00:03:15.895 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-03-28T00:03:15.895 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-03-28T00:03:15.895 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_ring.ko 2024-03-28T00:03:15.895 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1 2024-03-28T00:03:15.895 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/masters 2024-03-28T00:03:15.895 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/masters/ds2482.ko 2024-03-28T00:03:15.895 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/masters/ds2490.ko 2024-03-28T00:03:15.895 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves 2024-03-28T00:03:15.895 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-03-28T00:03:15.896 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-03-28T00:03:15.896 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-03-28T00:03:15.896 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-03-28T00:03:15.896 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-03-28T00:03:15.896 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-03-28T00:03:15.896 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-03-28T00:03:15.896 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-03-28T00:03:15.896 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-03-28T00:03:15.896 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_smem.ko 2024-03-28T00:03:15.896 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_therm.ko 2024-03-28T00:03:15.896 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/wire.ko 2024-03-28T00:03:15.896 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog 2024-03-28T00:03:15.896 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/alim1535_wdt.ko 2024-03-28T00:03:15.896 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/alim7101_wdt.ko 2024-03-28T00:03:15.897 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/f71808e_wdt.ko 2024-03-28T00:03:15.897 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/hpwdt.ko 2024-03-28T00:03:15.897 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/i6300esb.ko 2024-03-28T00:03:15.897 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-03-28T00:03:15.897 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/iTCO_wdt.ko 2024-03-28T00:03:15.897 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/ib700wdt.ko 2024-03-28T00:03:15.897 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/ibmasr.ko 2024-03-28T00:03:15.897 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-03-28T00:03:15.897 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/it8712f_wdt.ko 2024-03-28T00:03:15.897 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/it87_wdt.ko 2024-03-28T00:03:15.897 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/machzwd.ko 2024-03-28T00:03:15.897 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/nv_tco.ko 2024-03-28T00:03:15.897 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/pcwd_pci.ko 2024-03-28T00:03:15.897 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/pcwd_usb.ko 2024-03-28T00:03:15.897 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-03-28T00:03:15.898 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/sch311x_wdt.ko 2024-03-28T00:03:15.898 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/softdog.ko 2024-03-28T00:03:15.898 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/sp5100_tco.ko 2024-03-28T00:03:15.898 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/via_wdt.ko 2024-03-28T00:03:15.898 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-03-28T00:03:15.898 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/w83877f_wdt.ko 2024-03-28T00:03:15.898 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/w83977f_wdt.ko 2024-03-28T00:03:15.898 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/wdt_pci.ko 2024-03-28T00:03:15.898 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/xen_wdt.ko 2024-03-28T00:03:15.898 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen 2024-03-28T00:03:15.898 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-acpi-processor.ko 2024-03-28T00:03:15.898 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-evtchn.ko 2024-03-28T00:03:15.898 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-gntalloc.ko 2024-03-28T00:03:15.899 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-gntdev.ko 2024-03-28T00:03:15.899 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-pciback 2024-03-28T00:03:15.899 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-03-28T00:03:15.899 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-privcmd.ko 2024-03-28T00:03:15.899 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xenfs 2024-03-28T00:03:15.899 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xenfs/xenfs.ko 2024-03-28T00:03:15.899 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs 2024-03-28T00:03:15.899 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/9p 2024-03-28T00:03:15.899 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/9p/9p.ko 2024-03-28T00:03:15.899 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/affs 2024-03-28T00:03:15.899 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/affs/affs.ko 2024-03-28T00:03:15.899 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/befs 2024-03-28T00:03:15.899 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/befs/befs.ko 2024-03-28T00:03:15.899 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/binfmt_misc.ko 2024-03-28T00:03:15.899 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/btrfs 2024-03-28T00:03:15.900 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/btrfs/btrfs.ko 2024-03-28T00:03:15.900 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/cachefiles 2024-03-28T00:03:15.900 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/cachefiles/cachefiles.ko 2024-03-28T00:03:15.900 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ceph 2024-03-28T00:03:15.900 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ceph/ceph.ko 2024-03-28T00:03:15.900 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/coda 2024-03-28T00:03:15.900 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/coda/coda.ko 2024-03-28T00:03:15.900 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/cramfs 2024-03-28T00:03:15.900 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/cramfs/cramfs.ko 2024-03-28T00:03:15.900 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/dlm 2024-03-28T00:03:15.900 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/dlm/dlm.ko 2024-03-28T00:03:15.900 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ecryptfs 2024-03-28T00:03:15.900 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ecryptfs/ecryptfs.ko 2024-03-28T00:03:15.900 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/f2fs 2024-03-28T00:03:15.900 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/f2fs/f2fs.ko 2024-03-28T00:03:15.901 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fat 2024-03-28T00:03:15.901 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fat/fat.ko 2024-03-28T00:03:15.901 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fat/msdos.ko 2024-03-28T00:03:15.901 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fat/vfat.ko 2024-03-28T00:03:15.901 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fuse 2024-03-28T00:03:15.901 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fuse/cuse.ko 2024-03-28T00:03:15.901 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fuse/fuse.ko 2024-03-28T00:03:15.901 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/gfs2 2024-03-28T00:03:15.901 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/gfs2/gfs2.ko 2024-03-28T00:03:15.901 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hfs 2024-03-28T00:03:15.901 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hfs/hfs.ko 2024-03-28T00:03:15.901 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hfsplus 2024-03-28T00:03:15.901 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hfsplus/hfsplus.ko 2024-03-28T00:03:15.901 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/isofs 2024-03-28T00:03:15.901 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/isofs/isofs.ko 2024-03-28T00:03:15.901 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/jfs 2024-03-28T00:03:15.902 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/jfs/jfs.ko 2024-03-28T00:03:15.902 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/lockd 2024-03-28T00:03:15.902 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/lockd/lockd.ko 2024-03-28T00:03:15.902 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/minix 2024-03-28T00:03:15.902 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/minix/minix.ko 2024-03-28T00:03:15.902 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/netfs 2024-03-28T00:03:15.902 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/netfs/netfs.ko 2024-03-28T00:03:15.902 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs 2024-03-28T00:03:15.902 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/blocklayout 2024-03-28T00:03:15.902 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-03-28T00:03:15.902 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/filelayout 2024-03-28T00:03:15.902 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-03-28T00:03:15.902 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/flexfilelayout 2024-03-28T00:03:15.902 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-03-28T00:03:15.902 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/nfs.ko 2024-03-28T00:03:15.903 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/nfsv3.ko 2024-03-28T00:03:15.903 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/nfsv4.ko 2024-03-28T00:03:15.903 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs_common 2024-03-28T00:03:15.903 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs_common/grace.ko 2024-03-28T00:03:15.903 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs_common/nfs_acl.ko 2024-03-28T00:03:15.903 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfsd 2024-03-28T00:03:15.903 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfsd/nfsd.ko 2024-03-28T00:03:15.903 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nilfs2 2024-03-28T00:03:15.903 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nilfs2/nilfs2.ko 2024-03-28T00:03:15.903 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls 2024-03-28T00:03:15.903 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-celtic.ko 2024-03-28T00:03:15.903 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-centeuro.ko 2024-03-28T00:03:15.903 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-croatian.ko 2024-03-28T00:03:15.903 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-cyrillic.ko 2024-03-28T00:03:15.903 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-gaelic.ko 2024-03-28T00:03:15.903 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-greek.ko 2024-03-28T00:03:15.904 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-iceland.ko 2024-03-28T00:03:15.904 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-inuit.ko 2024-03-28T00:03:15.904 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-roman.ko 2024-03-28T00:03:15.904 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-romanian.ko 2024-03-28T00:03:15.904 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-turkish.ko 2024-03-28T00:03:15.904 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp1250.ko 2024-03-28T00:03:15.904 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp1251.ko 2024-03-28T00:03:15.904 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp1255.ko 2024-03-28T00:03:15.904 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp737.ko 2024-03-28T00:03:15.904 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp775.ko 2024-03-28T00:03:15.904 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp850.ko 2024-03-28T00:03:15.904 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp852.ko 2024-03-28T00:03:15.904 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp855.ko 2024-03-28T00:03:15.904 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp857.ko 2024-03-28T00:03:15.904 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp860.ko 2024-03-28T00:03:15.904 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp861.ko 2024-03-28T00:03:15.905 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp862.ko 2024-03-28T00:03:15.905 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp863.ko 2024-03-28T00:03:15.905 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp864.ko 2024-03-28T00:03:15.905 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp865.ko 2024-03-28T00:03:15.905 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp866.ko 2024-03-28T00:03:15.905 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp869.ko 2024-03-28T00:03:15.905 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp874.ko 2024-03-28T00:03:15.905 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp932.ko 2024-03-28T00:03:15.905 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp936.ko 2024-03-28T00:03:15.905 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp949.ko 2024-03-28T00:03:15.905 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp950.ko 2024-03-28T00:03:15.905 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_euc-jp.ko 2024-03-28T00:03:15.905 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-1.ko 2024-03-28T00:03:15.905 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-13.ko 2024-03-28T00:03:15.905 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-14.ko 2024-03-28T00:03:15.906 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-15.ko 2024-03-28T00:03:15.906 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-2.ko 2024-03-28T00:03:15.906 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-3.ko 2024-03-28T00:03:15.906 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-4.ko 2024-03-28T00:03:15.906 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-5.ko 2024-03-28T00:03:15.906 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-6.ko 2024-03-28T00:03:15.906 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-7.ko 2024-03-28T00:03:15.906 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-9.ko 2024-03-28T00:03:15.906 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_koi8-r.ko 2024-03-28T00:03:15.906 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_koi8-ru.ko 2024-03-28T00:03:15.906 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_koi8-u.ko 2024-03-28T00:03:15.906 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_ucs2_utils.ko 2024-03-28T00:03:15.906 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_utf8.ko 2024-03-28T00:03:15.906 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2 2024-03-28T00:03:15.907 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/cluster 2024-03-28T00:03:15.907 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-03-28T00:03:15.907 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/dlm 2024-03-28T00:03:15.907 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-03-28T00:03:15.907 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/dlmfs 2024-03-28T00:03:15.907 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-03-28T00:03:15.907 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/ocfs2.ko 2024-03-28T00:03:15.907 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-03-28T00:03:15.907 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-03-28T00:03:15.907 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-03-28T00:03:15.907 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/overlayfs 2024-03-28T00:03:15.907 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/overlayfs/overlay.ko 2024-03-28T00:03:15.907 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/pstore 2024-03-28T00:03:15.907 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/pstore/ramoops.ko 2024-03-28T00:03:15.908 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/reiserfs 2024-03-28T00:03:15.908 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/reiserfs/reiserfs.ko 2024-03-28T00:03:15.908 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/romfs 2024-03-28T00:03:15.908 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/romfs/romfs.ko 2024-03-28T00:03:15.908 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb 2024-03-28T00:03:15.908 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/client 2024-03-28T00:03:15.908 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/client/cifs.ko 2024-03-28T00:03:15.908 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/common 2024-03-28T00:03:15.908 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/common/cifs_arc4.ko 2024-03-28T00:03:15.908 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/common/cifs_md4.ko 2024-03-28T00:03:15.908 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/squashfs 2024-03-28T00:03:15.908 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/squashfs/squashfs.ko 2024-03-28T00:03:15.908 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/sysv 2024-03-28T00:03:15.908 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/sysv/sysv.ko 2024-03-28T00:03:15.908 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/udf 2024-03-28T00:03:15.908 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/udf/udf.ko 2024-03-28T00:03:15.909 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ufs 2024-03-28T00:03:15.909 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ufs/ufs.ko 2024-03-28T00:03:15.909 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/xfs 2024-03-28T00:03:15.909 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/xfs/xfs.ko 2024-03-28T00:03:15.909 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel 2024-03-28T00:03:15.909 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/rcu 2024-03-28T00:03:15.909 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/rcu/rcutorture.ko 2024-03-28T00:03:15.909 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/torture.ko 2024-03-28T00:03:15.909 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/trace 2024-03-28T00:03:15.909 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-03-28T00:03:15.909 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib 2024-03-28T00:03:15.909 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/asn1_encoder.ko 2024-03-28T00:03:15.909 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crc-itu-t.ko 2024-03-28T00:03:15.909 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crc8.ko 2024-03-28T00:03:15.909 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crypto 2024-03-28T00:03:15.910 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crypto/libarc4.ko 2024-03-28T00:03:15.910 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crypto/libdes.ko 2024-03-28T00:03:15.910 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/libcrc32c.ko 2024-03-28T00:03:15.910 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/lru_cache.ko 2024-03-28T00:03:15.910 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/lz4 2024-03-28T00:03:15.910 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/lz4/lz4_compress.ko 2024-03-28T00:03:15.910 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/lz4/lz4hc_compress.ko 2024-03-28T00:03:15.910 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/math 2024-03-28T00:03:15.910 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/math/cordic.ko 2024-03-28T00:03:15.910 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/raid6 2024-03-28T00:03:15.910 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/raid6/raid6_pq.ko 2024-03-28T00:03:15.910 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/reed_solomon 2024-03-28T00:03:15.910 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/reed_solomon/reed_solomon.ko 2024-03-28T00:03:15.910 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/ts_bm.ko 2024-03-28T00:03:15.911 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/ts_fsm.ko 2024-03-28T00:03:15.911 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/ts_kmp.ko 2024-03-28T00:03:15.911 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/zstd 2024-03-28T00:03:15.911 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/zstd/zstd_compress.ko 2024-03-28T00:03:15.911 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/mm 2024-03-28T00:03:15.911 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/mm/hwpoison-inject.ko 2024-03-28T00:03:15.911 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net 2024-03-28T00:03:15.911 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan 2024-03-28T00:03:15.911 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/6lowpan.ko 2024-03-28T00:03:15.911 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_dest.ko 2024-03-28T00:03:15.911 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_fragment.ko 2024-03-28T00:03:15.911 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_hop.ko 2024-03-28T00:03:15.911 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_ipv6.ko 2024-03-28T00:03:15.911 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_mobility.ko 2024-03-28T00:03:15.912 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_routing.ko 2024-03-28T00:03:15.912 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_udp.ko 2024-03-28T00:03:15.912 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802 2024-03-28T00:03:15.912 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/garp.ko 2024-03-28T00:03:15.912 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/mrp.ko 2024-03-28T00:03:15.912 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/p8022.ko 2024-03-28T00:03:15.912 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/psnap.ko 2024-03-28T00:03:15.912 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/stp.ko 2024-03-28T00:03:15.912 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/8021q 2024-03-28T00:03:15.912 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/8021q/8021q.ko 2024-03-28T00:03:15.912 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p 2024-03-28T00:03:15.912 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p/9pnet.ko 2024-03-28T00:03:15.912 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p/9pnet_fd.ko 2024-03-28T00:03:15.912 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p/9pnet_rdma.ko 2024-03-28T00:03:15.912 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p/9pnet_virtio.ko 2024-03-28T00:03:15.912 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/batman-adv 2024-03-28T00:03:15.913 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/batman-adv/batman-adv.ko 2024-03-28T00:03:15.913 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge 2024-03-28T00:03:15.913 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/br_netfilter.ko 2024-03-28T00:03:15.913 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/bridge.ko 2024-03-28T00:03:15.913 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter 2024-03-28T00:03:15.913 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-03-28T00:03:15.913 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_among.ko 2024-03-28T00:03:15.913 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_arp.ko 2024-03-28T00:03:15.913 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-03-28T00:03:15.913 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-03-28T00:03:15.913 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_ip.ko 2024-03-28T00:03:15.913 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-03-28T00:03:15.913 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_limit.ko 2024-03-28T00:03:15.913 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_log.ko 2024-03-28T00:03:15.913 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_mark.ko 2024-03-28T00:03:15.914 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-03-28T00:03:15.914 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-03-28T00:03:15.914 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-03-28T00:03:15.914 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-03-28T00:03:15.914 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_snat.ko 2024-03-28T00:03:15.914 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_stp.ko 2024-03-28T00:03:15.914 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-03-28T00:03:15.914 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-03-28T00:03:15.914 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-03-28T00:03:15.914 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-03-28T00:03:15.914 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebtables.ko 2024-03-28T00:03:15.914 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-03-28T00:03:15.914 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-03-28T00:03:15.914 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-03-28T00:03:15.914 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ceph 2024-03-28T00:03:15.915 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ceph/libceph.ko 2024-03-28T00:03:15.915 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/core 2024-03-28T00:03:15.915 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/core/failover.ko 2024-03-28T00:03:15.915 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/core/pktgen.ko 2024-03-28T00:03:15.915 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp 2024-03-28T00:03:15.915 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp/dccp.ko 2024-03-28T00:03:15.915 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp/dccp_diag.ko 2024-03-28T00:03:15.915 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp/dccp_ipv4.ko 2024-03-28T00:03:15.915 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp/dccp_ipv6.ko 2024-03-28T00:03:15.915 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dns_resolver 2024-03-28T00:03:15.915 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dns_resolver/dns_resolver.ko 2024-03-28T00:03:15.915 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa 2024-03-28T00:03:15.915 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/dsa_core.ko 2024-03-28T00:03:15.915 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/tag_brcm.ko 2024-03-28T00:03:15.915 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/tag_dsa.ko 2024-03-28T00:03:15.915 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/tag_none.ko 2024-03-28T00:03:15.916 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/tag_trailer.ko 2024-03-28T00:03:15.916 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154 2024-03-28T00:03:15.916 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154/6lowpan 2024-03-28T00:03:15.916 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-03-28T00:03:15.916 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154/ieee802154.ko 2024-03-28T00:03:15.916 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154/ieee802154_socket.ko 2024-03-28T00:03:15.916 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4 2024-03-28T00:03:15.916 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ah4.ko 2024-03-28T00:03:15.916 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/esp4.ko 2024-03-28T00:03:15.916 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/fou.ko 2024-03-28T00:03:15.916 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/gre.ko 2024-03-28T00:03:15.916 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/inet_diag.ko 2024-03-28T00:03:15.916 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ip_gre.ko 2024-03-28T00:03:15.916 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ip_tunnel.ko 2024-03-28T00:03:15.916 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ip_vti.ko 2024-03-28T00:03:15.917 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ipcomp.ko 2024-03-28T00:03:15.917 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ipip.ko 2024-03-28T00:03:15.917 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter 2024-03-28T00:03:15.917 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/arp_tables.ko 2024-03-28T00:03:15.917 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-03-28T00:03:15.917 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-03-28T00:03:15.917 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-03-28T00:03:15.917 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-03-28T00:03:15.917 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-03-28T00:03:15.917 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-03-28T00:03:15.917 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-03-28T00:03:15.917 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-03-28T00:03:15.917 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-03-28T00:03:15.918 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/iptable_security.ko 2024-03-28T00:03:15.918 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-03-28T00:03:15.918 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-03-28T00:03:15.918 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-03-28T00:03:15.918 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-03-28T00:03:15.918 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-03-28T00:03:15.918 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-03-28T00:03:15.918 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-03-28T00:03:15.918 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-03-28T00:03:15.918 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-03-28T00:03:15.918 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-03-28T00:03:15.918 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_bic.ko 2024-03-28T00:03:15.918 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_dctcp.ko 2024-03-28T00:03:15.918 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_diag.ko 2024-03-28T00:03:15.919 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_highspeed.ko 2024-03-28T00:03:15.919 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_htcp.ko 2024-03-28T00:03:15.919 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_hybla.ko 2024-03-28T00:03:15.919 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_illinois.ko 2024-03-28T00:03:15.919 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_lp.ko 2024-03-28T00:03:15.919 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_scalable.ko 2024-03-28T00:03:15.919 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_vegas.ko 2024-03-28T00:03:15.919 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_veno.ko 2024-03-28T00:03:15.919 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_westwood.ko 2024-03-28T00:03:15.919 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_yeah.ko 2024-03-28T00:03:15.919 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tunnel4.ko 2024-03-28T00:03:15.919 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/udp_diag.ko 2024-03-28T00:03:15.919 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/udp_tunnel.ko 2024-03-28T00:03:15.919 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/xfrm4_tunnel.ko 2024-03-28T00:03:15.920 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6 2024-03-28T00:03:15.920 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ah6.ko 2024-03-28T00:03:15.920 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/esp6.ko 2024-03-28T00:03:15.920 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/fou6.ko 2024-03-28T00:03:15.920 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ip6_tunnel.ko 2024-03-28T00:03:15.920 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-03-28T00:03:15.920 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ip6_vti.ko 2024-03-28T00:03:15.920 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ipcomp6.ko 2024-03-28T00:03:15.920 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter 2024-03-28T00:03:15.920 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-03-28T00:03:15.920 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-03-28T00:03:15.920 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-03-28T00:03:15.920 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-03-28T00:03:15.920 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-03-28T00:03:15.921 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-03-28T00:03:15.921 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-03-28T00:03:15.921 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-03-28T00:03:15.921 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-03-28T00:03:15.921 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-03-28T00:03:15.921 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-03-28T00:03:15.921 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-03-28T00:03:15.921 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-03-28T00:03:15.921 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-03-28T00:03:15.921 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-03-28T00:03:15.921 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-03-28T00:03:15.921 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-03-28T00:03:15.921 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-03-28T00:03:15.922 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-03-28T00:03:15.922 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-03-28T00:03:15.922 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-03-28T00:03:15.922 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-03-28T00:03:15.922 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-03-28T00:03:15.922 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-03-28T00:03:15.922 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/sit.ko 2024-03-28T00:03:15.922 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/tunnel6.ko 2024-03-28T00:03:15.922 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/xfrm6_tunnel.ko 2024-03-28T00:03:15.922 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/key 2024-03-28T00:03:15.922 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/key/af_key.ko 2024-03-28T00:03:15.922 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp 2024-03-28T00:03:15.922 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_core.ko 2024-03-28T00:03:15.923 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_debugfs.ko 2024-03-28T00:03:15.923 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_eth.ko 2024-03-28T00:03:15.923 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_ip.ko 2024-03-28T00:03:15.923 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_ip6.ko 2024-03-28T00:03:15.923 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_netlink.ko 2024-03-28T00:03:15.923 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_ppp.ko 2024-03-28T00:03:15.923 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/llc 2024-03-28T00:03:15.923 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/llc/llc.ko 2024-03-28T00:03:15.923 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/mac802154 2024-03-28T00:03:15.923 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/mac802154/mac802154.ko 2024-03-28T00:03:15.923 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/mpls 2024-03-28T00:03:15.923 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/mpls/mpls_gso.ko 2024-03-28T00:03:15.924 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter 2024-03-28T00:03:15.924 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset 2024-03-28T00:03:15.924 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set.ko 2024-03-28T00:03:15.924 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-03-28T00:03:15.924 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-03-28T00:03:15.924 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-03-28T00:03:15.924 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-03-28T00:03:15.924 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-03-28T00:03:15.924 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-03-28T00:03:15.924 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-03-28T00:03:15.924 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-03-28T00:03:15.924 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-03-28T00:03:15.924 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-03-28T00:03:15.924 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-03-28T00:03:15.925 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-03-28T00:03:15.925 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-03-28T00:03:15.925 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-03-28T00:03:15.925 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-03-28T00:03:15.925 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs 2024-03-28T00:03:15.925 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs.ko 2024-03-28T00:03:15.925 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-03-28T00:03:15.925 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-03-28T00:03:15.925 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-03-28T00:03:15.925 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-03-28T00:03:15.925 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-03-28T00:03:15.925 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-03-28T00:03:15.925 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-03-28T00:03:15.925 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-03-28T00:03:15.925 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-03-28T00:03:15.926 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-03-28T00:03:15.926 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-03-28T00:03:15.926 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-03-28T00:03:15.926 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-03-28T00:03:15.926 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conncount.ko 2024-03-28T00:03:15.926 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack.ko 2024-03-28T00:03:15.926 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-03-28T00:03:15.926 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-03-28T00:03:15.926 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-03-28T00:03:15.926 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_h323.ko 2024-03-28T00:03:15.926 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_irc.ko 2024-03-28T00:03:15.926 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-03-28T00:03:15.926 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-03-28T00:03:15.926 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-03-28T00:03:15.927 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_sane.ko 2024-03-28T00:03:15.927 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_sip.ko 2024-03-28T00:03:15.927 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-03-28T00:03:15.927 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-03-28T00:03:15.927 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_dup_netdev.ko 2024-03-28T00:03:15.927 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_flow_table.ko 2024-03-28T00:03:15.927 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_flow_table_inet.ko 2024-03-28T00:03:15.927 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_log_syslog.ko 2024-03-28T00:03:15.927 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat.ko 2024-03-28T00:03:15.927 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_amanda.ko 2024-03-28T00:03:15.927 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_ftp.ko 2024-03-28T00:03:15.927 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_irc.ko 2024-03-28T00:03:15.927 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_sip.ko 2024-03-28T00:03:15.927 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_tftp.ko 2024-03-28T00:03:15.928 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_synproxy_core.ko 2024-03-28T00:03:15.928 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_tables.ko 2024-03-28T00:03:15.928 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink.ko 2024-03-28T00:03:15.928 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink_acct.ko 2024-03-28T00:03:15.928 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink_log.ko 2024-03-28T00:03:15.928 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink_osf.ko 2024-03-28T00:03:15.928 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink_queue.ko 2024-03-28T00:03:15.928 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_chain_nat.ko 2024-03-28T00:03:15.928 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_compat.ko 2024-03-28T00:03:15.928 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_ct.ko 2024-03-28T00:03:15.928 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_dup_netdev.ko 2024-03-28T00:03:15.928 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_fib.ko 2024-03-28T00:03:15.928 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_fwd_netdev.ko 2024-03-28T00:03:15.928 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_hash.ko 2024-03-28T00:03:15.928 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_limit.ko 2024-03-28T00:03:15.929 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_log.ko 2024-03-28T00:03:15.929 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_masq.ko 2024-03-28T00:03:15.929 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_nat.ko 2024-03-28T00:03:15.929 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_numgen.ko 2024-03-28T00:03:15.929 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_queue.ko 2024-03-28T00:03:15.929 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_quota.ko 2024-03-28T00:03:15.929 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_redir.ko 2024-03-28T00:03:15.929 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_reject.ko 2024-03-28T00:03:15.929 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_reject_inet.ko 2024-03-28T00:03:15.929 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_socket.ko 2024-03-28T00:03:15.929 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_tunnel.ko 2024-03-28T00:03:15.929 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_AUDIT.ko 2024-03-28T00:03:15.929 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_CHECKSUM.ko 2024-03-28T00:03:15.929 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_CLASSIFY.ko 2024-03-28T00:03:15.930 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-03-28T00:03:15.930 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_CT.ko 2024-03-28T00:03:15.930 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_DSCP.ko 2024-03-28T00:03:15.930 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_HL.ko 2024-03-28T00:03:15.930 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_HMARK.ko 2024-03-28T00:03:15.930 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_IDLETIMER.ko 2024-03-28T00:03:15.930 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_LED.ko 2024-03-28T00:03:15.930 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_LOG.ko 2024-03-28T00:03:15.930 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_MASQUERADE.ko 2024-03-28T00:03:15.930 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_NETMAP.ko 2024-03-28T00:03:15.930 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_NFLOG.ko 2024-03-28T00:03:15.930 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_NFQUEUE.ko 2024-03-28T00:03:15.930 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_RATEEST.ko 2024-03-28T00:03:15.931 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_REDIRECT.ko 2024-03-28T00:03:15.931 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_SECMARK.ko 2024-03-28T00:03:15.931 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TCPMSS.ko 2024-03-28T00:03:15.931 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-03-28T00:03:15.931 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TEE.ko 2024-03-28T00:03:15.931 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TPROXY.ko 2024-03-28T00:03:15.931 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TRACE.ko 2024-03-28T00:03:15.931 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_addrtype.ko 2024-03-28T00:03:15.931 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_bpf.ko 2024-03-28T00:03:15.931 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_cgroup.ko 2024-03-28T00:03:15.931 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_cluster.ko 2024-03-28T00:03:15.931 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_comment.ko 2024-03-28T00:03:15.931 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_connbytes.ko 2024-03-28T00:03:15.931 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_connlabel.ko 2024-03-28T00:03:15.931 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_connlimit.ko 2024-03-28T00:03:15.932 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_connmark.ko 2024-03-28T00:03:15.932 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_conntrack.ko 2024-03-28T00:03:15.932 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_cpu.ko 2024-03-28T00:03:15.932 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_dccp.ko 2024-03-28T00:03:15.932 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_devgroup.ko 2024-03-28T00:03:15.932 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_dscp.ko 2024-03-28T00:03:15.932 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_ecn.ko 2024-03-28T00:03:15.932 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_esp.ko 2024-03-28T00:03:15.932 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_hashlimit.ko 2024-03-28T00:03:15.932 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_helper.ko 2024-03-28T00:03:15.932 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_hl.ko 2024-03-28T00:03:15.932 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_ipcomp.ko 2024-03-28T00:03:15.932 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_iprange.ko 2024-03-28T00:03:15.932 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_ipvs.ko 2024-03-28T00:03:15.932 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_l2tp.ko 2024-03-28T00:03:15.933 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_length.ko 2024-03-28T00:03:15.933 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_limit.ko 2024-03-28T00:03:15.933 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_mac.ko 2024-03-28T00:03:15.933 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_mark.ko 2024-03-28T00:03:15.933 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_multiport.ko 2024-03-28T00:03:15.933 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_nat.ko 2024-03-28T00:03:15.933 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_nfacct.ko 2024-03-28T00:03:15.933 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_osf.ko 2024-03-28T00:03:15.933 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_owner.ko 2024-03-28T00:03:15.933 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_physdev.ko 2024-03-28T00:03:15.933 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_pkttype.ko 2024-03-28T00:03:15.933 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_policy.ko 2024-03-28T00:03:15.933 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_quota.ko 2024-03-28T00:03:15.933 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_rateest.ko 2024-03-28T00:03:15.933 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_realm.ko 2024-03-28T00:03:15.933 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_recent.ko 2024-03-28T00:03:15.934 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_sctp.ko 2024-03-28T00:03:15.934 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_set.ko 2024-03-28T00:03:15.934 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_socket.ko 2024-03-28T00:03:15.934 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_state.ko 2024-03-28T00:03:15.934 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_statistic.ko 2024-03-28T00:03:15.934 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_string.ko 2024-03-28T00:03:15.934 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_tcpmss.ko 2024-03-28T00:03:15.934 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_time.ko 2024-03-28T00:03:15.934 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_u32.ko 2024-03-28T00:03:15.934 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netlink 2024-03-28T00:03:15.934 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netlink/netlink_diag.ko 2024-03-28T00:03:15.934 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/nsh 2024-03-28T00:03:15.934 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/nsh/nsh.ko 2024-03-28T00:03:15.934 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch 2024-03-28T00:03:15.935 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch/openvswitch.ko 2024-03-28T00:03:15.935 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch/vport-geneve.ko 2024-03-28T00:03:15.935 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch/vport-gre.ko 2024-03-28T00:03:15.935 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch/vport-vxlan.ko 2024-03-28T00:03:15.935 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/packet 2024-03-28T00:03:15.935 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/packet/af_packet_diag.ko 2024-03-28T00:03:15.935 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rds 2024-03-28T00:03:15.935 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rds/rds.ko 2024-03-28T00:03:15.935 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rds/rds_rdma.ko 2024-03-28T00:03:15.935 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rds/rds_tcp.ko 2024-03-28T00:03:15.935 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched 2024-03-28T00:03:15.935 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_csum.ko 2024-03-28T00:03:15.935 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_gact.ko 2024-03-28T00:03:15.935 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_mirred.ko 2024-03-28T00:03:15.936 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_nat.ko 2024-03-28T00:03:15.936 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_pedit.ko 2024-03-28T00:03:15.936 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_police.ko 2024-03-28T00:03:15.936 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_simple.ko 2024-03-28T00:03:15.936 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_skbedit.ko 2024-03-28T00:03:15.936 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_vlan.ko 2024-03-28T00:03:15.936 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_basic.ko 2024-03-28T00:03:15.936 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_bpf.ko 2024-03-28T00:03:15.936 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_flow.ko 2024-03-28T00:03:15.936 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_fw.ko 2024-03-28T00:03:15.936 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_route.ko 2024-03-28T00:03:15.936 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_u32.ko 2024-03-28T00:03:15.936 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_cmp.ko 2024-03-28T00:03:15.937 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_ipset.ko 2024-03-28T00:03:15.937 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_meta.ko 2024-03-28T00:03:15.937 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_nbyte.ko 2024-03-28T00:03:15.937 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_text.ko 2024-03-28T00:03:15.937 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_u32.ko 2024-03-28T00:03:15.937 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_choke.ko 2024-03-28T00:03:15.937 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_codel.ko 2024-03-28T00:03:15.937 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_drr.ko 2024-03-28T00:03:15.937 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_fq.ko 2024-03-28T00:03:15.937 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_gred.ko 2024-03-28T00:03:15.937 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_hfsc.ko 2024-03-28T00:03:15.937 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_hhf.ko 2024-03-28T00:03:15.937 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_htb.ko 2024-03-28T00:03:15.937 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_ingress.ko 2024-03-28T00:03:15.937 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_mqprio.ko 2024-03-28T00:03:15.937 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_mqprio_lib.ko 2024-03-28T00:03:15.938 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_multiq.ko 2024-03-28T00:03:15.938 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_netem.ko 2024-03-28T00:03:15.938 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_pie.ko 2024-03-28T00:03:15.938 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_plug.ko 2024-03-28T00:03:15.938 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_prio.ko 2024-03-28T00:03:15.938 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_qfq.ko 2024-03-28T00:03:15.938 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_red.ko 2024-03-28T00:03:15.938 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_sfb.ko 2024-03-28T00:03:15.938 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_sfq.ko 2024-03-28T00:03:15.938 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_tbf.ko 2024-03-28T00:03:15.938 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_teql.ko 2024-03-28T00:03:15.938 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sctp 2024-03-28T00:03:15.938 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sctp/sctp.ko 2024-03-28T00:03:15.938 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sctp/sctp_diag.ko 2024-03-28T00:03:15.939 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc 2024-03-28T00:03:15.939 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/auth_gss 2024-03-28T00:03:15.939 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-03-28T00:03:15.939 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-03-28T00:03:15.939 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/sunrpc.ko 2024-03-28T00:03:15.939 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/xprtrdma 2024-03-28T00:03:15.939 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-03-28T00:03:15.939 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/tipc 2024-03-28T00:03:15.939 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/tipc/diag.ko 2024-03-28T00:03:15.939 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/tipc/tipc.ko 2024-03-28T00:03:15.939 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/unix 2024-03-28T00:03:15.939 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/unix/unix_diag.ko 2024-03-28T00:03:15.939 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock 2024-03-28T00:03:15.939 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-03-28T00:03:15.940 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-03-28T00:03:15.940 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vsock.ko 2024-03-28T00:03:15.940 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vsock_diag.ko 2024-03-28T00:03:15.940 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vsock_loopback.ko 2024-03-28T00:03:15.940 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/xfrm 2024-03-28T00:03:15.940 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/xfrm/xfrm_ipcomp.ko 2024-03-28T00:03:15.940 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security 2024-03-28T00:03:15.940 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys 2024-03-28T00:03:15.940 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys/encrypted-keys 2024-03-28T00:03:15.940 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-03-28T00:03:15.940 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys/trusted-keys 2024-03-28T00:03:15.940 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys/trusted-keys/trusted.ko 2024-03-28T00:03:15.940 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/virt 2024-03-28T00:03:15.940 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/virt/lib 2024-03-28T00:03:15.940 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/virt/lib/irqbypass.ko 2024-03-28T00:03:15.941 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.alias 2024-03-28T00:03:15.941 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.alias.bin 2024-03-28T00:03:15.941 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.builtin 2024-03-28T00:03:15.941 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.builtin.alias.bin 2024-03-28T00:03:15.941 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.builtin.bin 2024-03-28T00:03:15.941 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.builtin.modinfo 2024-03-28T00:03:15.941 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.dep 2024-03-28T00:03:15.941 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.dep.bin 2024-03-28T00:03:15.941 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.devname 2024-03-28T00:03:15.941 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.order 2024-03-28T00:03:15.941 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.softdep 2024-03-28T00:03:15.941 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.symbols 2024-03-28T00:03:15.941 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.symbols.bin 2024-03-28T00:03:15.941 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.8.0-g1df0c7e8a235/vmlinuz 2024-03-28T00:03:15.945 DEBUG:teuthology.task.kernel:get_image_version: 6.8.0-g1df0c7e8a235 2024-03-28T00:03:15.946 DEBUG:teuthology.orchestra.run.smithi119:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-03-28T00:03:15.968 INFO:teuthology.orchestra.run.smithi119.stdout:postinstall scriptlet (using /bin/sh): 2024-03-28T00:03:15.968 INFO:teuthology.orchestra.run.smithi119.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-03-28T00:03:15.968 INFO:teuthology.orchestra.run.smithi119.stdout: /usr/bin/kernel-install add 6.8.0-g1df0c7e8a235 /lib/modules/6.8.0-g1df0c7e8a235/vmlinuz 2024-03-28T00:03:15.968 INFO:teuthology.orchestra.run.smithi119.stdout:fi 2024-03-28T00:03:15.968 INFO:teuthology.orchestra.run.smithi119.stdout:for file in vmlinuz System.map config; do 2024-03-28T00:03:15.968 INFO:teuthology.orchestra.run.smithi119.stdout: if ! cmp --silent "/lib/modules/6.8.0-g1df0c7e8a235/${file}" "/boot/${file}-6.8.0-g1df0c7e8a235"; then 2024-03-28T00:03:15.968 INFO:teuthology.orchestra.run.smithi119.stdout: cp "/lib/modules/6.8.0-g1df0c7e8a235/${file}" "/boot/${file}-6.8.0-g1df0c7e8a235" 2024-03-28T00:03:15.968 INFO:teuthology.orchestra.run.smithi119.stdout: fi 2024-03-28T00:03:15.969 INFO:teuthology.orchestra.run.smithi119.stdout:done 2024-03-28T00:03:15.969 INFO:teuthology.orchestra.run.smithi119.stdout:preuninstall scriptlet (using /bin/sh): 2024-03-28T00:03:15.969 INFO:teuthology.orchestra.run.smithi119.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2024-03-28T00:03:15.969 INFO:teuthology.orchestra.run.smithi119.stdout:new-kernel-pkg --remove 6.8.0-g1df0c7e8a235 --rminitrd --initrdfile=/boot/initramfs-6.8.0-g1df0c7e8a235.img 2024-03-28T00:03:15.969 INFO:teuthology.orchestra.run.smithi119.stdout:elif [ -x /usr/bin/kernel-install ]; then 2024-03-28T00:03:15.969 INFO:teuthology.orchestra.run.smithi119.stdout:kernel-install remove 6.8.0-g1df0c7e8a235 2024-03-28T00:03:15.969 INFO:teuthology.orchestra.run.smithi119.stdout:fi 2024-03-28T00:03:15.969 INFO:teuthology.orchestra.run.smithi119.stdout:postuninstall scriptlet (using /bin/sh): 2024-03-28T00:03:15.969 INFO:teuthology.orchestra.run.smithi119.stdout:if [ -x /sbin/update-bootloader ]; then 2024-03-28T00:03:15.969 INFO:teuthology.orchestra.run.smithi119.stdout:/sbin/update-bootloader --remove 6.8.0-g1df0c7e8a235 2024-03-28T00:03:15.969 INFO:teuthology.orchestra.run.smithi119.stdout:fi 2024-03-28T00:03:15.970 DEBUG:teuthology.orchestra.run.smithi119:> sudo rpm -qi grub2-tools 2024-03-28T00:03:16.044 INFO:teuthology.orchestra.run.smithi119.stdout:Name : grub2-tools 2024-03-28T00:03:16.045 INFO:teuthology.orchestra.run.smithi119.stdout:Epoch : 1 2024-03-28T00:03:16.045 INFO:teuthology.orchestra.run.smithi119.stdout:Version : 2.06 2024-03-28T00:03:16.045 INFO:teuthology.orchestra.run.smithi119.stdout:Release : 68.el9 2024-03-28T00:03:16.045 INFO:teuthology.orchestra.run.smithi119.stdout:Architecture: x86_64 2024-03-28T00:03:16.045 INFO:teuthology.orchestra.run.smithi119.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-03-28T00:03:16.045 INFO:teuthology.orchestra.run.smithi119.stdout:Group : Unspecified 2024-03-28T00:03:16.045 INFO:teuthology.orchestra.run.smithi119.stdout:Size : 8264254 2024-03-28T00:03:16.045 INFO:teuthology.orchestra.run.smithi119.stdout:License : GPLv3+ 2024-03-28T00:03:16.045 INFO:teuthology.orchestra.run.smithi119.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-03-28T00:03:16.045 INFO:teuthology.orchestra.run.smithi119.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-03-28T00:03:16.045 INFO:teuthology.orchestra.run.smithi119.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-03-28T00:03:16.045 INFO:teuthology.orchestra.run.smithi119.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-03-28T00:03:16.045 INFO:teuthology.orchestra.run.smithi119.stdout:Packager : builder@centos.org 2024-03-28T00:03:16.045 INFO:teuthology.orchestra.run.smithi119.stdout:Vendor : CentOS 2024-03-28T00:03:16.046 INFO:teuthology.orchestra.run.smithi119.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-28T00:03:16.046 INFO:teuthology.orchestra.run.smithi119.stdout:Summary : Support tools for GRUB. 2024-03-28T00:03:16.046 INFO:teuthology.orchestra.run.smithi119.stdout:Description : 2024-03-28T00:03:16.046 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-28T00:03:16.046 INFO:teuthology.orchestra.run.smithi119.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-28T00:03:16.046 INFO:teuthology.orchestra.run.smithi119.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-28T00:03:16.046 INFO:teuthology.orchestra.run.smithi119.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-28T00:03:16.046 INFO:teuthology.orchestra.run.smithi119.stdout:hardware devices. 2024-03-28T00:03:16.046 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-28T00:03:16.046 INFO:teuthology.orchestra.run.smithi119.stdout:This subpackage provides tools for support of all platforms. 2024-03-28T00:03:16.047 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-28T00:03:16.047 INFO:teuthology.task.kernel:Updating grub on smithi119 to boot 6.8.0-g1df0c7e8a235 2024-03-28T00:03:16.047 DEBUG:teuthology.orchestra.run.smithi119:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-28T00:03:16.627 INFO:teuthology.orchestra.run.smithi119.stderr:Generating grub configuration file ... 2024-03-28T00:03:17.999 INFO:teuthology.orchestra.run.smithi119.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-03-28T00:03:18.037 INFO:teuthology.orchestra.run.smithi119.stderr:done 2024-03-28T00:03:18.039 DEBUG:teuthology.orchestra.run.smithi119:> mktemp 2024-03-28T00:03:18.053 INFO:teuthology.orchestra.run.smithi119.stdout:/tmp/tmp.3VeR1MRXiF 2024-03-28T00:03:18.054 DEBUG:teuthology.orchestra.run.smithi119:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.3VeR1MRXiF 2024-03-28T00:03:18.129 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod 0666 /tmp/tmp.3VeR1MRXiF 2024-03-28T00:03:18.265 DEBUG:teuthology.orchestra.remote:smithi119:/tmp/tmp.3VeR1MRXiF is 6KB 2024-03-28T00:03:18.314 DEBUG:teuthology.orchestra.run.smithi119:> rm -fr /tmp/tmp.3VeR1MRXiF 2024-03-28T00:03:18.329 DEBUG:teuthology.orchestra.run.smithi119:> sudo /bin/ls /boot/loader/entries || true 2024-03-28T00:03:18.393 INFO:teuthology.orchestra.run.smithi119.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-03-28T00:03:18.394 INFO:teuthology.orchestra.run.smithi119.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-03-28T00:03:18.394 INFO:teuthology.orchestra.run.smithi119.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-03-28T00:03:18.394 INFO:teuthology.orchestra.run.smithi119.stdout:15949e3c88704300882482eddfd7baae-6.8.0-g1df0c7e8a235.conf 2024-03-28T00:03:18.395 DEBUG:teuthology.orchestra.run.smithi119:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-6.8.0-g1df0c7e8a235 2024-03-28T00:03:18.549 DEBUG:teuthology.orchestra.run.smithi119:> sudo shutdown -r now 2024-03-28T00:03:48.552 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-03-28T00:03:48.553 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi119.front.sepia.ceph.com' 2024-03-28T00:03:48.554 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-28T00:04:23.203 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.119 2024-03-28T00:04:32.204 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi119.front.sepia.ceph.com' 2024-03-28T00:04:32.205 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-28T00:04:50.590 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.119 2024-03-28T00:05:02.592 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi119.front.sepia.ceph.com' 2024-03-28T00:05:02.593 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-28T00:05:05.662 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.119 2024-03-28T00:05:20.663 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi119.front.sepia.ceph.com' 2024-03-28T00:05:20.664 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-28T00:05:21.340 DEBUG:teuthology.orchestra.run.smithi119:> true 2024-03-28T00:05:22.177 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi119.front.sepia.ceph.com' 2024-03-28T00:05:22.177 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.8.0-g1df0c7e8a235"... 2024-03-28T00:05:22.177 DEBUG:teuthology.orchestra.run.smithi119:> uname -r 2024-03-28T00:05:22.259 INFO:teuthology.orchestra.run.smithi119.stdout:6.8.0-g1df0c7e8a235 2024-03-28T00:05:22.259 DEBUG:teuthology.task.kernel:current kernel version is 6.8.0-g1df0c7e8a235 vs 6.8.0-g1df0c7e8a235 2024-03-28T00:05:22.259 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-28T00:05:22.259 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-03-28T00:05:23.260 DEBUG:teuthology.parallel:result is None 2024-03-28T00:05:23.260 INFO:teuthology.run_tasks:Running task internal.base... 2024-03-28T00:05:23.286 INFO:teuthology.task.internal:Creating test directory... 2024-03-28T00:05:23.286 DEBUG:teuthology.orchestra.run.smithi119:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-28T00:05:23.294 DEBUG:teuthology.orchestra.run.smithi132:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-28T00:05:23.296 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-28T00:05:23.330 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-03-28T00:05:23.369 INFO:teuthology.run_tasks:Running task internal.archive... 2024-03-28T00:05:23.438 INFO:teuthology.task.internal:Creating archive directory... 2024-03-28T00:05:23.438 DEBUG:teuthology.orchestra.run.smithi119:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-28T00:05:23.445 DEBUG:teuthology.orchestra.run.smithi132:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-28T00:05:23.448 DEBUG:teuthology.orchestra.run.smithi165:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-28T00:05:23.556 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-03-28T00:05:23.563 INFO:teuthology.task.internal:Enabling coredump saving... 2024-03-28T00:05:23.563 DEBUG:teuthology.orchestra.run.smithi132:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-28T00:05:23.579 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-28T00:05:23.580 DEBUG:teuthology.orchestra.run.smithi165:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-28T00:05:23.596 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-28T00:05:23.596 DEBUG:teuthology.orchestra.run.smithi119:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-28T00:05:23.603 DEBUG:teuthology.orchestra.run.smithi132:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-28T00:05:23.622 DEBUG:teuthology.orchestra.run.smithi165:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-28T00:05:23.668 INFO:teuthology.orchestra.run.smithi132.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-28T00:05:23.692 INFO:teuthology.orchestra.run.smithi132.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-28T00:05:23.693 INFO:teuthology.orchestra.run.smithi165.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-28T00:05:23.714 INFO:teuthology.orchestra.run.smithi165.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-28T00:05:23.767 INFO:teuthology.orchestra.run.smithi119.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-28T00:05:23.838 INFO:teuthology.orchestra.run.smithi119.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-28T00:05:23.842 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-03-28T00:05:23.849 INFO:teuthology.task.internal:Configuring sudo... 2024-03-28T00:05:23.850 DEBUG:teuthology.orchestra.run.smithi119:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-28T00:05:23.855 DEBUG:teuthology.orchestra.run.smithi132:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-28T00:05:23.857 DEBUG:teuthology.orchestra.run.smithi165:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-28T00:05:23.937 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-03-28T00:05:23.950 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-03-28T00:05:23.950 DEBUG:teuthology.orchestra.run.smithi119:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-28T00:05:23.956 DEBUG:teuthology.orchestra.run.smithi132:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-28T00:05:23.959 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-28T00:05:23.998 DEBUG:teuthology.orchestra.run.smithi119:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-28T00:05:24.101 DEBUG:teuthology.orchestra.run.smithi119:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-28T00:05:24.201 DEBUG:teuthology.orchestra.run.smithi119:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-28T00:05:24.269 DEBUG:teuthology.orchestra.run.smithi119:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-28T00:05:24.349 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-03-28T00:05:24.349 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-28T00:05:24.425 DEBUG:teuthology.orchestra.run.smithi132:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-28T00:05:24.469 DEBUG:teuthology.orchestra.run.smithi132:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-28T00:05:24.531 DEBUG:teuthology.orchestra.run.smithi132:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-28T00:05:24.572 DEBUG:teuthology.orchestra.run.smithi132:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-28T00:05:24.619 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-03-28T00:05:24.619 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-28T00:05:24.687 DEBUG:teuthology.orchestra.run.smithi165:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-28T00:05:24.729 DEBUG:teuthology.orchestra.run.smithi165:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-28T00:05:24.798 DEBUG:teuthology.orchestra.run.smithi165:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-28T00:05:24.841 DEBUG:teuthology.orchestra.run.smithi165:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-28T00:05:24.885 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-03-28T00:05:24.885 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-28T00:05:24.953 DEBUG:teuthology.orchestra.run.smithi119:> sudo service rsyslog restart 2024-03-28T00:05:24.961 DEBUG:teuthology.orchestra.run.smithi132:> sudo service rsyslog restart 2024-03-28T00:05:24.964 DEBUG:teuthology.orchestra.run.smithi165:> sudo service rsyslog restart 2024-03-28T00:05:24.998 INFO:teuthology.orchestra.run.smithi132.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-28T00:05:25.032 INFO:teuthology.orchestra.run.smithi165.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-28T00:05:25.057 INFO:teuthology.orchestra.run.smithi119.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-28T00:05:25.484 INFO:teuthology.run_tasks:Running task internal.timer... 2024-03-28T00:05:25.496 INFO:teuthology.task.internal:Starting timer... 2024-03-28T00:05:25.496 INFO:teuthology.run_tasks:Running task pcp... 2024-03-28T00:05:25.552 INFO:teuthology.run_tasks:Running task selinux... 2024-03-28T00:05:25.590 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-03-28T00:05:25.591 DEBUG:teuthology.orchestra.run.smithi119:> sudo service auditd rotate 2024-03-28T00:05:25.732 INFO:teuthology.orchestra.run.smithi119.stdout:Rotating logs: 2024-03-28T00:05:25.738 DEBUG:teuthology.orchestra.run.smithi132:> sudo service auditd rotate 2024-03-28T00:05:25.792 INFO:teuthology.orchestra.run.smithi132.stdout:Rotating logs: 2024-03-28T00:05:25.794 DEBUG:teuthology.orchestra.run.smithi165:> sudo service auditd rotate 2024-03-28T00:05:25.852 INFO:teuthology.orchestra.run.smithi165.stdout:Rotating logs: 2024-03-28T00:05:25.853 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-03-28T00:05:25.853 DEBUG:teuthology.orchestra.run.smithi119:> /usr/sbin/getenforce 2024-03-28T00:05:25.910 INFO:teuthology.orchestra.run.smithi119.stdout:Permissive 2024-03-28T00:05:25.911 DEBUG:teuthology.orchestra.run.smithi132:> /usr/sbin/getenforce 2024-03-28T00:05:25.930 INFO:teuthology.orchestra.run.smithi132.stdout:Permissive 2024-03-28T00:05:25.930 DEBUG:teuthology.orchestra.run.smithi165:> /usr/sbin/getenforce 2024-03-28T00:05:25.953 INFO:teuthology.orchestra.run.smithi165.stdout:Permissive 2024-03-28T00:05:25.953 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi119.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi132.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi165.front.sepia.ceph.com': 'permissive'} 2024-03-28T00:05:25.953 DEBUG:teuthology.orchestra.run.smithi119:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-28T00:05:26.041 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-28T00:05:26.042 DEBUG:teuthology.orchestra.run.smithi132:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-28T00:05:26.073 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-28T00:05:26.074 DEBUG:teuthology.orchestra.run.smithi165:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-28T00:05:26.103 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-28T00:05:26.103 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-03-28T00:05:26.104 DEBUG:teuthology.orchestra.run.smithi119:> sudo /usr/sbin/setenforce permissive 2024-03-28T00:05:26.199 DEBUG:teuthology.orchestra.run.smithi132:> sudo /usr/sbin/setenforce permissive 2024-03-28T00:05:26.240 DEBUG:teuthology.orchestra.run.smithi165:> sudo /usr/sbin/setenforce permissive 2024-03-28T00:05:26.281 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-03-28T00:05:26.329 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-03-28T00:05:26.346 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-03-28T00:05:26.347 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi119.front.sepia.ceph.com,smithi132.front.sepia.ceph.com,smithi165.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-03-28T00:11:32.571 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi119.front.sepia.ceph.com'), Remote(name='ubuntu@smithi132.front.sepia.ceph.com'), Remote(name='ubuntu@smithi165.front.sepia.ceph.com')] 2024-03-28T00:11:32.572 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi119.front.sepia.ceph.com' 2024-03-28T00:11:32.573 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-28T00:11:32.719 DEBUG:teuthology.orchestra.run.smithi119:> true 2024-03-28T00:11:32.862 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi119.front.sepia.ceph.com' 2024-03-28T00:11:32.862 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi132.front.sepia.ceph.com' 2024-03-28T00:11:32.862 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi132.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-28T00:11:32.952 DEBUG:teuthology.orchestra.run.smithi132:> true 2024-03-28T00:11:33.024 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi132.front.sepia.ceph.com' 2024-03-28T00:11:33.024 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi165.front.sepia.ceph.com' 2024-03-28T00:11:33.025 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi165.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-28T00:11:33.112 DEBUG:teuthology.orchestra.run.smithi165:> true 2024-03-28T00:11:33.181 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi165.front.sepia.ceph.com' 2024-03-28T00:11:33.182 INFO:teuthology.run_tasks:Running task clock... 2024-03-28T00:11:33.329 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-03-28T00:11:33.329 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-28T00:11:33.329 DEBUG:teuthology.orchestra.run.smithi119:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-28T00:11:33.337 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-28T00:11:33.337 DEBUG:teuthology.orchestra.run.smithi132:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-28T00:11:33.339 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-28T00:11:33.339 DEBUG:teuthology.orchestra.run.smithi165:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-28T00:11:33.367 INFO:teuthology.orchestra.run.smithi132.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-28T00:11:33.371 INFO:teuthology.orchestra.run.smithi165.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-28T00:11:33.384 INFO:teuthology.orchestra.run.smithi132.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-28T00:11:33.389 INFO:teuthology.orchestra.run.smithi165.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-28T00:11:33.413 INFO:teuthology.orchestra.run.smithi132.stderr:sudo: ntpd: command not found 2024-03-28T00:11:33.418 INFO:teuthology.orchestra.run.smithi165.stderr:sudo: ntpd: command not found 2024-03-28T00:11:33.427 INFO:teuthology.orchestra.run.smithi132.stdout:506 Cannot talk to daemon 2024-03-28T00:11:33.431 INFO:teuthology.orchestra.run.smithi119.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-28T00:11:33.432 INFO:teuthology.orchestra.run.smithi165.stdout:506 Cannot talk to daemon 2024-03-28T00:11:33.444 INFO:teuthology.orchestra.run.smithi132.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-28T00:11:33.449 INFO:teuthology.orchestra.run.smithi165.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-28T00:11:33.460 INFO:teuthology.orchestra.run.smithi132.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-28T00:11:33.465 INFO:teuthology.orchestra.run.smithi165.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-28T00:11:33.483 INFO:teuthology.orchestra.run.smithi119.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-28T00:11:33.504 INFO:teuthology.orchestra.run.smithi132.stderr:bash: line 1: ntpq: command not found 2024-03-28T00:11:33.507 INFO:teuthology.orchestra.run.smithi132.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-28T00:11:33.507 INFO:teuthology.orchestra.run.smithi132.stdout:=============================================================================== 2024-03-28T00:11:33.507 INFO:teuthology.orchestra.run.smithi132.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-28T00:11:33.507 INFO:teuthology.orchestra.run.smithi132.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-28T00:11:33.507 INFO:teuthology.orchestra.run.smithi132.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-28T00:11:33.507 INFO:teuthology.orchestra.run.smithi132.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-28T00:11:33.509 INFO:teuthology.orchestra.run.smithi165.stderr:bash: line 1: ntpq: command not found 2024-03-28T00:11:33.512 INFO:teuthology.orchestra.run.smithi165.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-28T00:11:33.512 INFO:teuthology.orchestra.run.smithi165.stdout:=============================================================================== 2024-03-28T00:11:33.512 INFO:teuthology.orchestra.run.smithi165.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-28T00:11:33.512 INFO:teuthology.orchestra.run.smithi165.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-28T00:11:33.512 INFO:teuthology.orchestra.run.smithi165.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-28T00:11:33.512 INFO:teuthology.orchestra.run.smithi165.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-28T00:11:33.572 INFO:teuthology.orchestra.run.smithi119.stderr:sudo: ntpd: command not found 2024-03-28T00:11:33.614 INFO:teuthology.orchestra.run.smithi119.stdout:506 Cannot talk to daemon 2024-03-28T00:11:33.663 INFO:teuthology.orchestra.run.smithi119.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-28T00:11:33.716 INFO:teuthology.orchestra.run.smithi119.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-28T00:11:33.836 INFO:teuthology.orchestra.run.smithi119.stderr:bash: line 1: ntpq: command not found 2024-03-28T00:11:33.843 INFO:teuthology.orchestra.run.smithi119.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-28T00:11:33.843 INFO:teuthology.orchestra.run.smithi119.stdout:=============================================================================== 2024-03-28T00:11:33.844 INFO:teuthology.orchestra.run.smithi119.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-28T00:11:33.844 INFO:teuthology.orchestra.run.smithi119.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-28T00:11:33.844 INFO:teuthology.orchestra.run.smithi119.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-28T00:11:33.844 INFO:teuthology.orchestra.run.smithi119.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-28T00:11:33.845 INFO:teuthology.run_tasks:Running task install... 2024-03-28T00:11:33.862 DEBUG:teuthology.task.install:project ceph 2024-03-28T00:11:33.863 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '3be9a5c9bc793e11e2800a8c0c696e8b46742033'}} 2024-03-28T00:11:33.863 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '3be9a5c9bc793e11e2800a8c0c696e8b46742033'} 2024-03-28T00:11:33.863 INFO:teuthology.task.install:Using flavor: default 2024-03-28T00:11:33.871 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-03-28T00:11:33.871 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-03-28T00:11:33.872 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '3be9a5c9bc793e11e2800a8c0c696e8b46742033', 'tag': None, 'wait_for_package': False} 2024-03-28T00:11:33.873 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3be9a5c9bc793e11e2800a8c0c696e8b46742033 2024-03-28T00:11:33.876 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '3be9a5c9bc793e11e2800a8c0c696e8b46742033', 'tag': None, 'wait_for_package': False} 2024-03-28T00:11:33.876 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3be9a5c9bc793e11e2800a8c0c696e8b46742033 2024-03-28T00:11:33.878 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '3be9a5c9bc793e11e2800a8c0c696e8b46742033', 'tag': None, 'wait_for_package': False} 2024-03-28T00:11:33.878 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3be9a5c9bc793e11e2800a8c0c696e8b46742033 2024-03-28T00:11:34.109 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-mchangir-testing-main-20240318.032620/3be9a5c9bc793e11e2800a8c0c696e8b46742033/centos/9/flavors/default/ 2024-03-28T00:11:34.109 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2145.g3be9a5c9 2024-03-28T00:11:34.112 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-mchangir-testing-main-20240318.032620/3be9a5c9bc793e11e2800a8c0c696e8b46742033/centos/9/flavors/default/ 2024-03-28T00:11:34.113 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2145.g3be9a5c9 2024-03-28T00:11:34.121 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-mchangir-testing-main-20240318.032620/3be9a5c9bc793e11e2800a8c0c696e8b46742033/centos/9/flavors/default/ 2024-03-28T00:11:34.121 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2145.g3be9a5c9 2024-03-28T00:11:34.224 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-mchangir-testing-main-20240318.032620/3be9a5c9bc793e11e2800a8c0c696e8b46742033/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-mchangir-testing-main-20240318.032620/3be9a5c9bc793e11e2800a8c0c696e8b46742033/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-mchangir-testing-main-20240318.032620/3be9a5c9bc793e11e2800a8c0c696e8b46742033/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-28T00:11:34.225 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-03-28T00:11:34.225 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-28T00:11:34.232 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-mchangir-testing-main-20240318.032620/3be9a5c9bc793e11e2800a8c0c696e8b46742033/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-mchangir-testing-main-20240318.032620/3be9a5c9bc793e11e2800a8c0c696e8b46742033/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-mchangir-testing-main-20240318.032620/3be9a5c9bc793e11e2800a8c0c696e8b46742033/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-28T00:11:34.232 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-03-28T00:11:34.232 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-28T00:11:34.254 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-28T00:11:34.255 DEBUG:teuthology.orchestra.run.smithi165:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/3be9a5c9bc793e11e2800a8c0c696e8b46742033/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-28T00:11:34.261 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-28T00:11:34.262 DEBUG:teuthology.orchestra.run.smithi132:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/3be9a5c9bc793e11e2800a8c0c696e8b46742033/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-28T00:11:34.280 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-mchangir-testing-main-20240318.032620/3be9a5c9bc793e11e2800a8c0c696e8b46742033/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-mchangir-testing-main-20240318.032620/3be9a5c9bc793e11e2800a8c0c696e8b46742033/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-mchangir-testing-main-20240318.032620/3be9a5c9bc793e11e2800a8c0c696e8b46742033/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-28T00:11:34.281 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-03-28T00:11:34.281 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-28T00:11:34.323 DEBUG:teuthology.orchestra.run.smithi165:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-28T00:11:34.330 DEBUG:teuthology.orchestra.run.smithi132:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-28T00:11:34.379 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-28T00:11:34.380 DEBUG:teuthology.orchestra.run.smithi119:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/3be9a5c9bc793e11e2800a8c0c696e8b46742033/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-28T00:11:34.404 DEBUG:teuthology.orchestra.run.smithi165:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-28T00:11:34.410 DEBUG:teuthology.orchestra.run.smithi132:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-28T00:11:34.470 INFO:teuthology.orchestra.run.smithi165.stdout:check_obsoletes = 1 2024-03-28T00:11:34.472 DEBUG:teuthology.orchestra.run.smithi165:> sudo yum clean all 2024-03-28T00:11:34.477 INFO:teuthology.orchestra.run.smithi132.stdout:check_obsoletes = 1 2024-03-28T00:11:34.478 DEBUG:teuthology.orchestra.run.smithi132:> sudo yum clean all 2024-03-28T00:11:34.517 DEBUG:teuthology.orchestra.run.smithi119:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-28T00:11:34.648 DEBUG:teuthology.orchestra.run.smithi119:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-28T00:11:34.708 INFO:teuthology.orchestra.run.smithi165.stdout:74 files removed 2024-03-28T00:11:34.712 INFO:teuthology.orchestra.run.smithi132.stdout:74 files removed 2024-03-28T00:11:34.732 INFO:teuthology.orchestra.run.smithi119.stdout:check_obsoletes = 1 2024-03-28T00:11:34.735 DEBUG:teuthology.orchestra.run.smithi165:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-28T00:11:34.736 DEBUG:teuthology.orchestra.run.smithi119:> sudo yum clean all 2024-03-28T00:11:34.738 DEBUG:teuthology.orchestra.run.smithi132:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-28T00:11:35.082 INFO:teuthology.orchestra.run.smithi119.stdout:74 files removed 2024-03-28T00:11:35.120 DEBUG:teuthology.orchestra.run.smithi119:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-28T00:11:35.380 INFO:teuthology.orchestra.run.smithi165.stdout:ceph packages for x86_64 236 kB/s | 83 kB 00:00 2024-03-28T00:11:35.395 INFO:teuthology.orchestra.run.smithi132.stdout:ceph packages for x86_64 226 kB/s | 83 kB 00:00 2024-03-28T00:11:35.754 INFO:teuthology.orchestra.run.smithi132.stdout:ceph noarch packages 32 kB/s | 10 kB 00:00 2024-03-28T00:11:35.789 INFO:teuthology.orchestra.run.smithi165.stdout:ceph noarch packages 27 kB/s | 10 kB 00:00 2024-03-28T00:11:36.054 INFO:teuthology.orchestra.run.smithi132.stdout:ceph source packages 7.0 kB/s | 1.9 kB 00:00 2024-03-28T00:11:36.137 INFO:teuthology.orchestra.run.smithi119.stdout:ceph packages for x86_64 164 kB/s | 83 kB 00:00 2024-03-28T00:11:36.206 INFO:teuthology.orchestra.run.smithi165.stdout:ceph source packages 4.9 kB/s | 1.9 kB 00:00 2024-03-28T00:11:36.432 INFO:teuthology.orchestra.run.smithi132.stdout:CentOS-9 - AppStream 19 MB/s | 6.7 MB 00:00 2024-03-28T00:11:36.658 INFO:teuthology.orchestra.run.smithi165.stdout:CentOS-9 - AppStream 16 MB/s | 6.7 MB 00:00 2024-03-28T00:11:36.660 INFO:teuthology.orchestra.run.smithi119.stdout:ceph noarch packages 23 kB/s | 10 kB 00:00 2024-03-28T00:11:37.094 INFO:teuthology.orchestra.run.smithi119.stdout:ceph source packages 5.2 kB/s | 1.9 kB 00:00 2024-03-28T00:11:37.666 INFO:teuthology.orchestra.run.smithi119.stdout:CentOS-9 - AppStream 13 MB/s | 6.7 MB 00:00 2024-03-28T00:11:37.982 INFO:teuthology.orchestra.run.smithi132.stdout:CentOS-9 - BaseOS 4.5 MB/s | 2.0 MB 00:00 2024-03-28T00:11:38.059 INFO:teuthology.orchestra.run.smithi165.stdout:CentOS-9 - BaseOS 6.9 MB/s | 2.0 MB 00:00 2024-03-28T00:11:39.006 INFO:teuthology.orchestra.run.smithi132.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 8.0 MB 00:00 2024-03-28T00:11:39.399 INFO:teuthology.orchestra.run.smithi119.stdout:CentOS-9 - BaseOS 4.3 MB/s | 2.0 MB 00:00 2024-03-28T00:11:39.566 INFO:teuthology.orchestra.run.smithi165.stdout:CentOS Stream 9 - BaseOS 6.8 MB/s | 8.0 MB 00:01 2024-03-28T00:11:41.069 INFO:teuthology.orchestra.run.smithi119.stdout:CentOS Stream 9 - BaseOS 6.4 MB/s | 8.0 MB 00:01 2024-03-28T00:11:43.112 INFO:teuthology.orchestra.run.smithi119.stdout:CentOS Stream 9 - AppStream 22 MB/s | 19 MB 00:00 2024-03-28T00:11:43.992 INFO:teuthology.orchestra.run.smithi165.stdout:CentOS Stream 9 - AppStream 5.6 MB/s | 19 MB 00:03 2024-03-28T00:11:47.655 INFO:teuthology.orchestra.run.smithi119.stdout:CentOS Stream 9 - CRB 8.4 MB/s | 6.1 MB 00:00 2024-03-28T00:11:48.195 INFO:teuthology.orchestra.run.smithi165.stdout:CentOS Stream 9 - CRB 7.9 MB/s | 6.1 MB 00:00 2024-03-28T00:11:49.214 INFO:teuthology.orchestra.run.smithi119.stdout:CentOS Stream 9 - Extras packages 68 kB/s | 16 kB 00:00 2024-03-28T00:11:49.635 INFO:teuthology.orchestra.run.smithi165.stdout:CentOS Stream 9 - Extras packages 52 kB/s | 16 kB 00:00 2024-03-28T00:11:50.416 INFO:teuthology.orchestra.run.smithi119.stdout:Extra Packages for Enterprise Linux 20 MB/s | 21 MB 00:01 2024-03-28T00:11:56.149 INFO:teuthology.orchestra.run.smithi119.stdout:lab-extras 39 kB/s | 1.7 kB 00:00 2024-03-28T00:11:58.773 INFO:teuthology.orchestra.run.smithi119.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-28T00:11:58.775 INFO:teuthology.orchestra.run.smithi119.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-28T00:11:58.797 INFO:teuthology.orchestra.run.smithi119.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-28T00:11:58.799 INFO:teuthology.orchestra.run.smithi119.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-28T00:11:58.800 INFO:teuthology.orchestra.run.smithi119.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-28T00:11:58.802 INFO:teuthology.orchestra.run.smithi119.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-28T00:11:58.804 INFO:teuthology.orchestra.run.smithi119.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-28T00:11:58.808 INFO:teuthology.orchestra.run.smithi119.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-28T00:11:58.809 INFO:teuthology.orchestra.run.smithi119.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-28T00:11:58.897 INFO:teuthology.orchestra.run.smithi119.stdout:Dependencies resolved. 2024-03-28T00:11:58.922 INFO:teuthology.orchestra.run.smithi119.stdout:============================================================================================ 2024-03-28T00:11:58.922 INFO:teuthology.orchestra.run.smithi119.stdout: Package Arch Version Repository Size 2024-03-28T00:11:58.922 INFO:teuthology.orchestra.run.smithi119.stdout:============================================================================================ 2024-03-28T00:11:58.922 INFO:teuthology.orchestra.run.smithi119.stdout:Installing: 2024-03-28T00:11:58.922 INFO:teuthology.orchestra.run.smithi119.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-28T00:11:58.922 INFO:teuthology.orchestra.run.smithi119.stdout: ceph x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 6.6 k 2024-03-28T00:11:58.922 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-base x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 5.4 M 2024-03-28T00:11:58.922 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-fuse x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 890 k 2024-03-28T00:11:58.922 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 145 k 2024-03-28T00:11:58.922 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 1.5 M 2024-03-28T00:11:58.923 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 139 k 2024-03-28T00:11:58.923 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 1.8 M 2024-03-28T00:11:58.923 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 7.4 M 2024-03-28T00:11:58.923 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-rook noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 49 k 2024-03-28T00:11:58.923 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-radosgw x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 9.9 M 2024-03-28T00:11:58.923 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-test x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 48 M 2024-03-28T00:11:58.923 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-volume noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 264 k 2024-03-28T00:11:58.923 INFO:teuthology.orchestra.run.smithi119.stdout: cephadm noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 748 k 2024-03-28T00:11:58.923 INFO:teuthology.orchestra.run.smithi119.stdout: cephfs-mirror x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 220 k 2024-03-28T00:11:58.923 INFO:teuthology.orchestra.run.smithi119.stdout: cephfs-shell x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 27 k 2024-03-28T00:11:58.923 INFO:teuthology.orchestra.run.smithi119.stdout: cephfs-top noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 18 k 2024-03-28T00:11:58.923 INFO:teuthology.orchestra.run.smithi119.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-03-28T00:11:58.924 INFO:teuthology.orchestra.run.smithi119.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-28T00:11:58.924 INFO:teuthology.orchestra.run.smithi119.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-28T00:11:58.924 INFO:teuthology.orchestra.run.smithi119.stdout: libcephfs-devel x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 31 k 2024-03-28T00:11:58.924 INFO:teuthology.orchestra.run.smithi119.stdout: libcephfs2 x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 746 k 2024-03-28T00:11:58.924 INFO:teuthology.orchestra.run.smithi119.stdout: librados-devel x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 127 k 2024-03-28T00:11:58.924 INFO:teuthology.orchestra.run.smithi119.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-28T00:11:58.924 INFO:teuthology.orchestra.run.smithi119.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-03-28T00:11:58.924 INFO:teuthology.orchestra.run.smithi119.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-03-28T00:11:58.924 INFO:teuthology.orchestra.run.smithi119.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-03-28T00:11:58.924 INFO:teuthology.orchestra.run.smithi119.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-03-28T00:11:58.924 INFO:teuthology.orchestra.run.smithi119.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-03-28T00:11:58.924 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cephfs x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 162 k 2024-03-28T00:11:58.924 INFO:teuthology.orchestra.run.smithi119.stdout: python3-rados x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 321 k 2024-03-28T00:11:58.925 INFO:teuthology.orchestra.run.smithi119.stdout: python3-rbd x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 298 k 2024-03-28T00:11:58.925 INFO:teuthology.orchestra.run.smithi119.stdout: python3-rgw x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 99 k 2024-03-28T00:11:58.925 INFO:teuthology.orchestra.run.smithi119.stdout: rbd-fuse x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 85 k 2024-03-28T00:11:58.925 INFO:teuthology.orchestra.run.smithi119.stdout: rbd-mirror x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 3.1 M 2024-03-28T00:11:58.925 INFO:teuthology.orchestra.run.smithi119.stdout: rbd-nbd x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 170 k 2024-03-28T00:11:58.925 INFO:teuthology.orchestra.run.smithi119.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-03-28T00:11:58.925 INFO:teuthology.orchestra.run.smithi119.stdout:Upgrading: 2024-03-28T00:11:58.925 INFO:teuthology.orchestra.run.smithi119.stdout: NetworkManager x86_64 1:1.47.2-1.el9 baseos 2.3 M 2024-03-28T00:11:58.925 INFO:teuthology.orchestra.run.smithi119.stdout: NetworkManager-libnm x86_64 1:1.47.2-1.el9 baseos 1.8 M 2024-03-28T00:11:58.925 INFO:teuthology.orchestra.run.smithi119.stdout: NetworkManager-team x86_64 1:1.47.2-1.el9 baseos 42 k 2024-03-28T00:11:58.925 INFO:teuthology.orchestra.run.smithi119.stdout: NetworkManager-tui x86_64 1:1.47.2-1.el9 baseos 248 k 2024-03-28T00:11:58.925 INFO:teuthology.orchestra.run.smithi119.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-03-28T00:11:58.925 INFO:teuthology.orchestra.run.smithi119.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-03-28T00:11:58.926 INFO:teuthology.orchestra.run.smithi119.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-03-28T00:11:58.926 INFO:teuthology.orchestra.run.smithi119.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-03-28T00:11:58.926 INFO:teuthology.orchestra.run.smithi119.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-03-28T00:11:58.926 INFO:teuthology.orchestra.run.smithi119.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-03-28T00:11:58.926 INFO:teuthology.orchestra.run.smithi119.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-03-28T00:11:58.926 INFO:teuthology.orchestra.run.smithi119.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-03-28T00:11:58.926 INFO:teuthology.orchestra.run.smithi119.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-03-28T00:11:58.926 INFO:teuthology.orchestra.run.smithi119.stdout: librados2 x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 3.4 M 2024-03-28T00:11:58.926 INFO:teuthology.orchestra.run.smithi119.stdout: librbd1 x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 3.1 M 2024-03-28T00:11:58.926 INFO:teuthology.orchestra.run.smithi119.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-03-28T00:11:58.926 INFO:teuthology.orchestra.run.smithi119.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-03-28T00:11:58.926 INFO:teuthology.orchestra.run.smithi119.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-03-28T00:11:58.927 INFO:teuthology.orchestra.run.smithi119.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-03-28T00:11:58.927 INFO:teuthology.orchestra.run.smithi119.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-03-28T00:11:58.927 INFO:teuthology.orchestra.run.smithi119.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-03-28T00:11:58.927 INFO:teuthology.orchestra.run.smithi119.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-03-28T00:11:58.927 INFO:teuthology.orchestra.run.smithi119.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-28T00:11:58.927 INFO:teuthology.orchestra.run.smithi119.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-03-28T00:11:58.927 INFO:teuthology.orchestra.run.smithi119.stdout:Installing dependencies: 2024-03-28T00:11:58.927 INFO:teuthology.orchestra.run.smithi119.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-28T00:11:58.927 INFO:teuthology.orchestra.run.smithi119.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-28T00:11:58.927 INFO:teuthology.orchestra.run.smithi119.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-28T00:11:58.927 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-common x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 20 M 2024-03-28T00:11:58.927 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 34 k 2024-03-28T00:11:58.927 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mds x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 2.3 M 2024-03-28T00:11:58.928 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 248 k 2024-03-28T00:11:58.928 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mon x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 4.7 M 2024-03-28T00:11:58.928 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-osd x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 17 M 2024-03-28T00:11:58.928 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 16 k 2024-03-28T00:11:58.928 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-selinux x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 25 k 2024-03-28T00:11:58.928 INFO:teuthology.orchestra.run.smithi119.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-28T00:11:58.928 INFO:teuthology.orchestra.run.smithi119.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-28T00:11:58.928 INFO:teuthology.orchestra.run.smithi119.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-28T00:11:58.928 INFO:teuthology.orchestra.run.smithi119.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-28T00:11:58.928 INFO:teuthology.orchestra.run.smithi119.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-28T00:11:58.928 INFO:teuthology.orchestra.run.smithi119.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-28T00:11:58.928 INFO:teuthology.orchestra.run.smithi119.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-28T00:11:58.928 INFO:teuthology.orchestra.run.smithi119.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-28T00:11:58.929 INFO:teuthology.orchestra.run.smithi119.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-28T00:11:58.929 INFO:teuthology.orchestra.run.smithi119.stdout: libcephsqlite x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 163 k 2024-03-28T00:11:58.929 INFO:teuthology.orchestra.run.smithi119.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-28T00:11:58.929 INFO:teuthology.orchestra.run.smithi119.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-28T00:11:58.929 INFO:teuthology.orchestra.run.smithi119.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-28T00:11:58.929 INFO:teuthology.orchestra.run.smithi119.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-28T00:11:58.929 INFO:teuthology.orchestra.run.smithi119.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-28T00:11:58.929 INFO:teuthology.orchestra.run.smithi119.stdout: libradosstriper1 x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 501 k 2024-03-28T00:11:58.929 INFO:teuthology.orchestra.run.smithi119.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-28T00:11:58.929 INFO:teuthology.orchestra.run.smithi119.stdout: librgw2 x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 5.1 M 2024-03-28T00:11:58.929 INFO:teuthology.orchestra.run.smithi119.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-28T00:11:58.929 INFO:teuthology.orchestra.run.smithi119.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-28T00:11:58.929 INFO:teuthology.orchestra.run.smithi119.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-28T00:11:58.929 INFO:teuthology.orchestra.run.smithi119.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-28T00:11:58.930 INFO:teuthology.orchestra.run.smithi119.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-28T00:11:58.930 INFO:teuthology.orchestra.run.smithi119.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-28T00:11:58.930 INFO:teuthology.orchestra.run.smithi119.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-28T00:11:58.930 INFO:teuthology.orchestra.run.smithi119.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-28T00:11:58.930 INFO:teuthology.orchestra.run.smithi119.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-28T00:11:58.930 INFO:teuthology.orchestra.run.smithi119.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-28T00:11:58.930 INFO:teuthology.orchestra.run.smithi119.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-03-28T00:11:58.930 INFO:teuthology.orchestra.run.smithi119.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-28T00:11:58.930 INFO:teuthology.orchestra.run.smithi119.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-28T00:11:58.930 INFO:teuthology.orchestra.run.smithi119.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-03-28T00:11:58.930 INFO:teuthology.orchestra.run.smithi119.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-03-28T00:11:58.930 INFO:teuthology.orchestra.run.smithi119.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-28T00:11:58.930 INFO:teuthology.orchestra.run.smithi119.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-28T00:11:58.931 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-28T00:11:58.931 INFO:teuthology.orchestra.run.smithi119.stdout: python3-ceph-argparse x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 45 k 2024-03-28T00:11:58.931 INFO:teuthology.orchestra.run.smithi119.stdout: python3-ceph-common x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 131 k 2024-03-28T00:11:58.931 INFO:teuthology.orchestra.run.smithi119.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-28T00:11:58.931 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-28T00:11:58.931 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-28T00:11:58.931 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-28T00:11:58.931 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-03-28T00:11:58.931 INFO:teuthology.orchestra.run.smithi119.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-03-28T00:11:58.931 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-28T00:11:58.931 INFO:teuthology.orchestra.run.smithi119.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-28T00:11:58.931 INFO:teuthology.orchestra.run.smithi119.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-03-28T00:11:58.931 INFO:teuthology.orchestra.run.smithi119.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-28T00:11:58.932 INFO:teuthology.orchestra.run.smithi119.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-28T00:11:58.932 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-28T00:11:58.932 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-28T00:11:58.932 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-28T00:11:58.932 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-28T00:11:58.932 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-28T00:11:58.932 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-28T00:11:58.932 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-28T00:11:58.932 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-28T00:11:58.932 INFO:teuthology.orchestra.run.smithi119.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-28T00:11:58.932 INFO:teuthology.orchestra.run.smithi119.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-28T00:11:58.932 INFO:teuthology.orchestra.run.smithi119.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-28T00:11:58.932 INFO:teuthology.orchestra.run.smithi119.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-28T00:11:58.932 INFO:teuthology.orchestra.run.smithi119.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-28T00:11:58.933 INFO:teuthology.orchestra.run.smithi119.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-28T00:11:58.933 INFO:teuthology.orchestra.run.smithi119.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-28T00:11:58.933 INFO:teuthology.orchestra.run.smithi119.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-28T00:11:58.933 INFO:teuthology.orchestra.run.smithi119.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-28T00:11:58.933 INFO:teuthology.orchestra.run.smithi119.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-28T00:11:58.933 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-28T00:11:58.933 INFO:teuthology.orchestra.run.smithi119.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-28T00:11:58.933 INFO:teuthology.orchestra.run.smithi119.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-28T00:11:58.933 INFO:teuthology.orchestra.run.smithi119.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-28T00:11:58.933 INFO:teuthology.orchestra.run.smithi119.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-28T00:11:58.933 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-28T00:11:58.933 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-28T00:11:58.933 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-28T00:11:58.933 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-28T00:11:58.934 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-03-28T00:11:58.934 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-28T00:11:58.934 INFO:teuthology.orchestra.run.smithi119.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-28T00:11:58.934 INFO:teuthology.orchestra.run.smithi119.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-28T00:11:58.934 INFO:teuthology.orchestra.run.smithi119.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-28T00:11:58.934 INFO:teuthology.orchestra.run.smithi119.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-28T00:11:58.934 INFO:teuthology.orchestra.run.smithi119.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-28T00:11:58.934 INFO:teuthology.orchestra.run.smithi119.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-28T00:11:58.934 INFO:teuthology.orchestra.run.smithi119.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-28T00:11:58.934 INFO:teuthology.orchestra.run.smithi119.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-28T00:11:58.934 INFO:teuthology.orchestra.run.smithi119.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-28T00:11:58.934 INFO:teuthology.orchestra.run.smithi119.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-28T00:11:58.934 INFO:teuthology.orchestra.run.smithi119.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-03-28T00:11:58.934 INFO:teuthology.orchestra.run.smithi119.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-28T00:11:58.935 INFO:teuthology.orchestra.run.smithi119.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-28T00:11:58.935 INFO:teuthology.orchestra.run.smithi119.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-28T00:11:58.935 INFO:teuthology.orchestra.run.smithi119.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-28T00:11:58.935 INFO:teuthology.orchestra.run.smithi119.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-03-28T00:11:58.935 INFO:teuthology.orchestra.run.smithi119.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-03-28T00:11:58.935 INFO:teuthology.orchestra.run.smithi119.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-28T00:11:58.935 INFO:teuthology.orchestra.run.smithi119.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-28T00:11:58.935 INFO:teuthology.orchestra.run.smithi119.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-28T00:11:58.935 INFO:teuthology.orchestra.run.smithi119.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-28T00:11:58.935 INFO:teuthology.orchestra.run.smithi119.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-28T00:11:58.935 INFO:teuthology.orchestra.run.smithi119.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-28T00:11:58.935 INFO:teuthology.orchestra.run.smithi119.stdout:Installing weak dependencies: 2024-03-28T00:11:58.935 INFO:teuthology.orchestra.run.smithi119.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-03-28T00:11:58.935 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-28T00:11:58.936 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction Summary 2024-03-28T00:11:58.936 INFO:teuthology.orchestra.run.smithi119.stdout:============================================================================================ 2024-03-28T00:11:58.936 INFO:teuthology.orchestra.run.smithi119.stdout:Install 150 Packages 2024-03-28T00:11:58.936 INFO:teuthology.orchestra.run.smithi119.stdout:Upgrade 24 Packages 2024-03-28T00:11:58.936 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-28T00:11:58.936 INFO:teuthology.orchestra.run.smithi119.stdout:Total download size: 225 M 2024-03-28T00:11:58.936 INFO:teuthology.orchestra.run.smithi119.stdout:Downloading Packages: 2024-03-28T00:11:59.750 INFO:teuthology.orchestra.run.smithi119.stdout:(1/174): ceph-19.0.0-2145.g3be9a5c9.el9.x86_64. 22 kB/s | 6.6 kB 00:00 2024-03-28T00:12:00.259 INFO:teuthology.orchestra.run.smithi119.stdout:(2/174): ceph-base-19.0.0-2145.g3be9a5c9.el9.x8 6.6 MB/s | 5.4 MB 00:00 2024-03-28T00:12:00.301 INFO:teuthology.orchestra.run.smithi119.stdout:(3/174): ceph-fuse-19.0.0-2145.g3be9a5c9.el9.x8 1.6 MB/s | 890 kB 00:00 2024-03-28T00:12:00.334 INFO:teuthology.orchestra.run.smithi119.stdout:(4/174): ceph-immutable-object-cache-19.0.0-214 1.9 MB/s | 145 kB 00:00 2024-03-28T00:12:00.460 INFO:teuthology.orchestra.run.smithi119.stdout:(5/174): ceph-mgr-19.0.0-2145.g3be9a5c9.el9.x86 12 MB/s | 1.5 MB 00:00 2024-03-28T00:12:00.802 INFO:teuthology.orchestra.run.smithi119.stdout:(6/174): ceph-mon-19.0.0-2145.g3be9a5c9.el9.x86 14 MB/s | 4.7 MB 00:00 2024-03-28T00:12:01.185 INFO:teuthology.orchestra.run.smithi119.stdout:(7/174): ceph-mds-19.0.0-2145.g3be9a5c9.el9.x86 2.6 MB/s | 2.3 MB 00:00 2024-03-28T00:12:01.213 INFO:teuthology.orchestra.run.smithi132.stdout:CentOS Stream 9 - AppStream 913 kB/s | 19 MB 00:21 2024-03-28T00:12:01.844 INFO:teuthology.orchestra.run.smithi119.stdout:(8/174): ceph-osd-19.0.0-2145.g3be9a5c9.el9.x86 16 MB/s | 17 MB 00:01 2024-03-28T00:12:01.911 INFO:teuthology.orchestra.run.smithi119.stdout:(9/174): ceph-selinux-19.0.0-2145.g3be9a5c9.el9 382 kB/s | 25 kB 00:00 2024-03-28T00:12:04.202 INFO:teuthology.orchestra.run.smithi119.stdout:(10/174): ceph-common-19.0.0-2145.g3be9a5c9.el9 4.3 MB/s | 20 MB 00:04 2024-03-28T00:12:04.303 INFO:teuthology.orchestra.run.smithi119.stdout:(11/174): cephfs-mirror-19.0.0-2145.g3be9a5c9.e 2.2 MB/s | 220 kB 00:00 2024-03-28T00:12:04.803 INFO:teuthology.orchestra.run.smithi119.stdout:(12/174): ceph-test-19.0.0-2145.g3be9a5c9.el9.x 16 MB/s | 48 MB 00:02 2024-03-28T00:12:04.828 INFO:teuthology.orchestra.run.smithi119.stdout:(13/174): cephfs-shell-19.0.0-2145.g3be9a5c9.el 52 kB/s | 27 kB 00:00 2024-03-28T00:12:05.012 INFO:teuthology.orchestra.run.smithi119.stdout:(14/174): ceph-radosgw-19.0.0-2145.g3be9a5c9.el 2.6 MB/s | 9.9 MB 00:03 2024-03-28T00:12:05.046 INFO:teuthology.orchestra.run.smithi119.stdout:(15/174): libcephfs-devel-19.0.0-2145.g3be9a5c9 129 kB/s | 31 kB 00:00 2024-03-28T00:12:05.104 INFO:teuthology.orchestra.run.smithi119.stdout:(16/174): libcephfs2-19.0.0-2145.g3be9a5c9.el9. 2.6 MB/s | 746 kB 00:00 2024-03-28T00:12:05.138 INFO:teuthology.orchestra.run.smithi119.stdout:(17/174): libcephsqlite-19.0.0-2145.g3be9a5c9.e 1.3 MB/s | 163 kB 00:00 2024-03-28T00:12:05.163 INFO:teuthology.orchestra.run.smithi119.stdout:(18/174): librados-devel-19.0.0-2145.g3be9a5c9. 1.1 MB/s | 127 kB 00:00 2024-03-28T00:12:05.239 INFO:teuthology.orchestra.run.smithi119.stdout:(19/174): libradosstriper1-19.0.0-2145.g3be9a5c 3.7 MB/s | 501 kB 00:00 2024-03-28T00:12:05.255 INFO:teuthology.orchestra.run.smithi119.stdout:(20/174): python3-ceph-argparse-19.0.0-2145.g3b 497 kB/s | 45 kB 00:00 2024-03-28T00:12:05.306 INFO:teuthology.orchestra.run.smithi119.stdout:(21/174): python3-ceph-common-19.0.0-2145.g3be9 1.9 MB/s | 131 kB 00:00 2024-03-28T00:12:05.339 INFO:teuthology.orchestra.run.smithi119.stdout:(22/174): python3-cephfs-19.0.0-2145.g3be9a5c9. 1.9 MB/s | 162 kB 00:00 2024-03-28T00:12:05.382 INFO:teuthology.orchestra.run.smithi119.stdout:(23/174): python3-rados-19.0.0-2145.g3be9a5c9.e 4.2 MB/s | 321 kB 00:00 2024-03-28T00:12:05.415 INFO:teuthology.orchestra.run.smithi119.stdout:(24/174): python3-rbd-19.0.0-2145.g3be9a5c9.el9 3.9 MB/s | 298 kB 00:00 2024-03-28T00:12:05.440 INFO:teuthology.orchestra.run.smithi119.stdout:(25/174): python3-rgw-19.0.0-2145.g3be9a5c9.el9 1.7 MB/s | 99 kB 00:00 2024-03-28T00:12:05.499 INFO:teuthology.orchestra.run.smithi119.stdout:(26/174): rbd-fuse-19.0.0-2145.g3be9a5c9.el9.x8 1.0 MB/s | 85 kB 00:00 2024-03-28T00:12:05.582 INFO:teuthology.orchestra.run.smithi119.stdout:(27/174): rbd-nbd-19.0.0-2145.g3be9a5c9.el9.x86 2.0 MB/s | 170 kB 00:00 2024-03-28T00:12:05.650 INFO:teuthology.orchestra.run.smithi119.stdout:(28/174): ceph-grafana-dashboards-19.0.0-2145.g 514 kB/s | 34 kB 00:00 2024-03-28T00:12:05.742 INFO:teuthology.orchestra.run.smithi119.stdout:(29/174): ceph-mgr-cephadm-19.0.0-2145.g3be9a5c 1.5 MB/s | 139 kB 00:00 2024-03-28T00:12:05.925 INFO:teuthology.orchestra.run.smithi119.stdout:(30/174): ceph-mgr-dashboard-19.0.0-2145.g3be9a 9.8 MB/s | 1.8 MB 00:00 2024-03-28T00:12:06.000 INFO:teuthology.orchestra.run.smithi119.stdout:(31/174): rbd-mirror-19.0.0-2145.g3be9a5c9.el9. 5.5 MB/s | 3.1 MB 00:00 2024-03-28T00:12:06.401 INFO:teuthology.orchestra.run.smithi119.stdout:(32/174): ceph-mgr-diskprediction-local-19.0.0- 16 MB/s | 7.4 MB 00:00 2024-03-28T00:12:06.493 INFO:teuthology.orchestra.run.smithi119.stdout:(33/174): ceph-mgr-rook-19.0.0-2145.g3be9a5c9.e 542 kB/s | 49 kB 00:00 2024-03-28T00:12:06.601 INFO:teuthology.orchestra.run.smithi119.stdout:(34/174): librgw2-19.0.0-2145.g3be9a5c9.el9.x86 3.6 MB/s | 5.1 MB 00:01 2024-03-28T00:12:06.626 INFO:teuthology.orchestra.run.smithi119.stdout:(35/174): ceph-mgr-modules-core-19.0.0-2145.g3b 397 kB/s | 248 kB 00:00 2024-03-28T00:12:06.647 INFO:teuthology.orchestra.run.smithi132.stdout:CentOS Stream 9 - CRB 3.1 MB/s | 6.1 MB 00:01 2024-03-28T00:12:06.652 INFO:teuthology.orchestra.run.smithi119.stdout:(36/174): ceph-prometheus-alerts-19.0.0-2145.g3 104 kB/s | 16 kB 00:00 2024-03-28T00:12:06.769 INFO:teuthology.orchestra.run.smithi119.stdout:(37/174): cephadm-19.0.0-2145.g3be9a5c9.el9.noa 6.3 MB/s | 748 kB 00:00 2024-03-28T00:12:06.794 INFO:teuthology.orchestra.run.smithi119.stdout:(38/174): cephfs-top-19.0.0-2145.g3be9a5c9.el9. 129 kB/s | 18 kB 00:00 2024-03-28T00:12:06.819 INFO:teuthology.orchestra.run.smithi119.stdout:(39/174): ceph-volume-19.0.0-2145.g3be9a5c9.el9 1.2 MB/s | 264 kB 00:00 2024-03-28T00:12:07.136 INFO:teuthology.orchestra.run.smithi119.stdout:(40/174): boost-program-options-1.75.0-8.el9.x8 335 kB/s | 106 kB 00:00 2024-03-28T00:12:07.211 INFO:teuthology.orchestra.run.smithi119.stdout:(41/174): automake-1.16.2-8.el9.noarch.rpm 1.5 MB/s | 662 kB 00:00 2024-03-28T00:12:07.254 INFO:teuthology.orchestra.run.smithi119.stdout:(42/174): bison-3.7.4-5.el9.x86_64.rpm 2.0 MB/s | 920 kB 00:00 2024-03-28T00:12:07.287 INFO:teuthology.orchestra.run.smithi119.stdout:(43/174): flex-2.6.4-9.el9.x86_64.rpm 2.0 MB/s | 307 kB 00:00 2024-03-28T00:12:07.312 INFO:teuthology.orchestra.run.smithi119.stdout:(44/174): flexiblas-3.0.4-8.el9.x86_64.rpm 328 kB/s | 32 kB 00:00 2024-03-28T00:12:07.338 INFO:teuthology.orchestra.run.smithi119.stdout:(45/174): flexiblas-openblas-openmp-3.0.4-8.el9 351 kB/s | 17 kB 00:00 2024-03-28T00:12:07.396 INFO:teuthology.orchestra.run.smithi119.stdout:(46/174): libaio-devel-0.3.111-13.el9.x86_64.rp 121 kB/s | 10 kB 00:00 2024-03-28T00:12:07.505 INFO:teuthology.orchestra.run.smithi119.stdout:(47/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 14 MB/s | 3.0 MB 00:00 2024-03-28T00:12:07.530 INFO:teuthology.orchestra.run.smithi119.stdout:(48/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 867 kB/s | 159 kB 00:00 2024-03-28T00:12:07.547 INFO:teuthology.orchestra.run.smithi119.stdout:(49/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 310 kB/s | 44 kB 00:00 2024-03-28T00:12:07.597 INFO:teuthology.orchestra.run.smithi119.stdout:(50/174): librdkafka-1.6.1-102.el9.x86_64.rpm 7.2 MB/s | 661 kB 00:00 2024-03-28T00:12:07.639 INFO:teuthology.orchestra.run.smithi119.stdout:(51/174): libtool-2.4.6-45.el9.x86_64.rpm 6.6 MB/s | 578 kB 00:00 2024-03-28T00:12:07.656 INFO:teuthology.orchestra.run.smithi119.stdout:(52/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 455 kB/s | 46 kB 00:00 2024-03-28T00:12:07.690 INFO:teuthology.orchestra.run.smithi119.stdout:(53/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.2 MB/s | 282 kB 00:00 2024-03-28T00:12:07.723 INFO:teuthology.orchestra.run.smithi119.stdout:(54/174): lua-5.4.4-4.el9.x86_64.rpm 2.9 MB/s | 187 kB 00:00 2024-03-28T00:12:07.757 INFO:teuthology.orchestra.run.smithi119.stdout:(55/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 2.2 MB/s | 217 kB 00:00 2024-03-28T00:12:07.773 INFO:teuthology.orchestra.run.smithi119.stdout:(56/174): openblas-0.3.21-2.el9.x86_64.rpm 410 kB/s | 34 kB 00:00 2024-03-28T00:12:08.016 INFO:teuthology.orchestra.run.smithi119.stdout:(57/174): openblas-openmp-0.3.21-2.el9.x86_64.r 16 MB/s | 4.7 MB 00:00 2024-03-28T00:12:08.041 INFO:teuthology.orchestra.run.smithi119.stdout:(58/174): python3-attrs-20.3.0-7.el9.noarch.rpm 309 kB/s | 78 kB 00:00 2024-03-28T00:12:08.091 INFO:teuthology.orchestra.run.smithi119.stdout:(59/174): protobuf-3.14.0-13.el9.x86_64.rpm 3.3 MB/s | 1.0 MB 00:00 2024-03-28T00:12:08.108 INFO:teuthology.orchestra.run.smithi119.stdout:(60/174): python3-jsonpatch-1.21-16.el9.noarch. 426 kB/s | 24 kB 00:00 2024-03-28T00:12:08.264 INFO:teuthology.orchestra.run.smithi132.stdout:CentOS Stream 9 - Extras packages 32 kB/s | 16 kB 00:00 2024-03-28T00:12:08.358 INFO:teuthology.orchestra.run.smithi119.stdout:(61/174): python3-babel-2.9.1-2.el9.noarch.rpm 19 MB/s | 5.8 MB 00:00 2024-03-28T00:12:08.375 INFO:teuthology.orchestra.run.smithi119.stdout:(62/174): python3-jsonpointer-2.0-4.el9.noarch. 64 kB/s | 18 kB 00:00 2024-03-28T00:12:08.417 INFO:teuthology.orchestra.run.smithi119.stdout:(63/174): python3-mako-1.1.4-6.el9.noarch.rpm 488 kB/s | 144 kB 00:00 2024-03-28T00:12:08.442 INFO:teuthology.orchestra.run.smithi119.stdout:(64/174): python3-markupsafe-1.1.1-12.el9.x86_6 481 kB/s | 32 kB 00:00 2024-03-28T00:12:08.979 INFO:teuthology.orchestra.run.smithi119.stdout:(65/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 7.9 MB/s | 4.8 MB 00:00 2024-03-28T00:12:09.009 INFO:teuthology.orchestra.run.smithi119.stdout:(66/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 371 kB/s | 217 kB 00:00 2024-03-28T00:12:09.034 INFO:teuthology.orchestra.run.smithi119.stdout:(67/174): python3-oauthlib-3.1.1-5.el9.noarch.r 300 kB/s | 174 kB 00:00 2024-03-28T00:12:09.076 INFO:teuthology.orchestra.run.smithi119.stdout:(68/174): python3-prettytable-0.7.2-27.el9.noar 1.0 MB/s | 41 kB 00:00 2024-03-28T00:12:09.102 INFO:teuthology.orchestra.run.smithi119.stdout:(69/174): python3-packaging-20.9-5.el9.noarch.r 1.0 MB/s | 69 kB 00:00 2024-03-28T00:12:09.127 INFO:teuthology.orchestra.run.smithi119.stdout:(70/174): python3-protobuf-3.14.0-13.el9.noarch 2.5 MB/s | 238 kB 00:00 2024-03-28T00:12:09.152 INFO:teuthology.orchestra.run.smithi119.stdout:(71/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.8 MB/s | 132 kB 00:00 2024-03-28T00:12:09.186 INFO:teuthology.orchestra.run.smithi119.stdout:(72/174): python3-pytz-2021.1-5.el9.noarch.rpm 842 kB/s | 47 kB 00:00 2024-03-28T00:12:09.219 INFO:teuthology.orchestra.run.smithi119.stdout:(73/174): python3-pyasn1-modules-0.4.8-6.el9.no 2.3 MB/s | 211 kB 00:00 2024-03-28T00:12:09.244 INFO:teuthology.orchestra.run.smithi119.stdout:(74/174): python3-requests-oauthlib-1.3.0-12.el 467 kB/s | 43 kB 00:00 2024-03-28T00:12:09.645 INFO:teuthology.orchestra.run.smithi119.stdout:(75/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 104 kB/s | 41 kB 00:00 2024-03-28T00:12:09.670 INFO:teuthology.orchestra.run.smithi119.stdout:(76/174): python3-toml-0.10.2-6.el9.noarch.rpm 92 kB/s | 39 kB 00:00 2024-03-28T00:12:09.896 INFO:teuthology.orchestra.run.smithi119.stdout:(77/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 22 MB/s | 16 MB 00:00 2024-03-28T00:12:09.954 INFO:teuthology.orchestra.run.smithi119.stdout:(78/174): socat-1.7.4.1-5.el9.x86_64.rpm 974 kB/s | 300 kB 00:00 2024-03-28T00:12:09.979 INFO:teuthology.orchestra.run.smithi119.stdout:(79/174): uuid-1.6.2-55.el9.x86_64.rpm 185 kB/s | 56 kB 00:00 2024-03-28T00:12:10.005 INFO:teuthology.orchestra.run.smithi119.stdout:(80/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 591 kB/s | 63 kB 00:00 2024-03-28T00:12:10.038 INFO:teuthology.orchestra.run.smithi119.stdout:(81/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 774 kB/s | 44 kB 00:00 2024-03-28T00:12:10.064 INFO:teuthology.orchestra.run.smithi119.stdout:(82/174): fuse-2.9.9-15.el9.x86_64.rpm 958 kB/s | 79 kB 00:00 2024-03-28T00:12:10.106 INFO:teuthology.orchestra.run.smithi119.stdout:(83/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.4 MB/s | 241 kB 00:00 2024-03-28T00:12:10.273 INFO:teuthology.orchestra.run.smithi119.stdout:(84/174): python3-cryptography-36.0.1-4.el9.x86 4.9 MB/s | 1.1 MB 00:00 2024-03-28T00:12:10.298 INFO:teuthology.orchestra.run.smithi119.stdout:(85/174): python3-ply-3.11-14.el9.noarch.rpm 445 kB/s | 103 kB 00:00 2024-03-28T00:12:10.323 INFO:teuthology.orchestra.run.smithi119.stdout:(86/174): python3-pycparser-2.20-6.el9.noarch.r 592 kB/s | 124 kB 00:00 2024-03-28T00:12:10.327 INFO:teuthology.orchestra.run.smithi132.stdout:Extra Packages for Enterprise Linux 11 MB/s | 21 MB 00:01 2024-03-28T00:12:10.357 INFO:teuthology.orchestra.run.smithi119.stdout:(87/174): python3-urllib3-1.26.5-5.el9.noarch.r 2.2 MB/s | 187 kB 00:00 2024-03-28T00:12:10.491 INFO:teuthology.orchestra.run.smithi119.stdout:(88/174): jq-1.6-16.el9.x86_64.rpm 1.1 MB/s | 187 kB 00:00 2024-03-28T00:12:10.524 INFO:teuthology.orchestra.run.smithi119.stdout:(89/174): libquadmath-11.4.1-3.el9.x86_64.rpm 1.1 MB/s | 195 kB 00:00 2024-03-28T00:12:10.558 INFO:teuthology.orchestra.run.smithi119.stdout:(90/174): python3-requests-2.25.1-8.el9.noarch. 1.8 MB/s | 125 kB 00:00 2024-03-28T00:12:10.617 INFO:teuthology.orchestra.run.smithi119.stdout:(91/174): libgfortran-11.4.1-3.el9.x86_64.rpm 2.7 MB/s | 803 kB 00:00 2024-03-28T00:12:10.708 INFO:teuthology.orchestra.run.smithi119.stdout:(92/174): elfutils-libelf-devel-0.190-2.el9.x86 117 kB/s | 21 kB 00:00 2024-03-28T00:12:10.759 INFO:teuthology.orchestra.run.smithi119.stdout:(93/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 127 kB/s | 25 kB 00:00 2024-03-28T00:12:10.886 INFO:teuthology.orchestra.run.smithi165.stdout:Extra Packages for Enterprise Linux 1.0 MB/s | 21 MB 00:21 2024-03-28T00:12:11.276 INFO:teuthology.orchestra.run.smithi119.stdout:(94/174): postgresql-contrib-13.14-1.el9.x86_64 1.7 MB/s | 911 kB 00:00 2024-03-28T00:12:11.368 INFO:teuthology.orchestra.run.smithi119.stdout:(95/174): postgresql-13.14-1.el9.x86_64.rpm 2.4 MB/s | 1.6 MB 00:00 2024-03-28T00:12:11.426 INFO:teuthology.orchestra.run.smithi119.stdout:(96/174): postgresql-private-libs-13.14-1.el9.x 892 kB/s | 133 kB 00:00 2024-03-28T00:12:11.535 INFO:teuthology.orchestra.run.smithi119.stdout:(97/174): openssl-devel-3.0.7-27.el9.x86_64.rpm 4.5 MB/s | 4.1 MB 00:00 2024-03-28T00:12:11.610 INFO:teuthology.orchestra.run.smithi119.stdout:(98/174): python3-devel-3.9.17-2.el9.x86_64.rpm 1.3 MB/s | 245 kB 00:00 2024-03-28T00:12:11.644 INFO:teuthology.orchestra.run.smithi119.stdout:(99/174): python3-jinja2-2.11.3-5.el9.noarch.rp 2.2 MB/s | 248 kB 00:00 2024-03-28T00:12:11.727 INFO:teuthology.orchestra.run.smithi119.stdout:(100/174): qatlib-service-24.02.0-1.el9.x86_64. 434 kB/s | 36 kB 00:00 2024-03-28T00:12:11.752 INFO:teuthology.orchestra.run.smithi119.stdout:(101/174): qatlib-24.02.0-1.el9.x86_64.rpm 1.5 MB/s | 221 kB 00:00 2024-03-28T00:12:11.803 INFO:teuthology.orchestra.run.smithi119.stdout:(102/174): qatzip-libs-1.2.0-1.el9.x86_64.rpm 636 kB/s | 47 kB 00:00 2024-03-28T00:12:11.837 INFO:teuthology.orchestra.run.smithi119.stdout:(103/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.1 MB/s | 94 kB 00:00 2024-03-28T00:12:12.112 INFO:teuthology.orchestra.run.smithi119.stdout:(104/174): lua-devel-5.4.4-4.el9.x86_64.rpm 72 kB/s | 22 kB 00:00 2024-03-28T00:12:12.304 INFO:teuthology.orchestra.run.smithi119.stdout:(105/174): abseil-cpp-20211102.0-3.el9.x86_64.r 2.8 MB/s | 548 kB 00:00 2024-03-28T00:12:12.381 INFO:teuthology.orchestra.run.smithi119.stdout:(106/174): gperftools-libs-2.9.1-2.el9.x86_64.r 4.0 MB/s | 309 kB 00:00 2024-03-28T00:12:12.496 INFO:teuthology.orchestra.run.smithi119.stdout:(107/174): postgresql-server-13.14-1.el9.x86_64 5.1 MB/s | 5.8 MB 00:01 2024-03-28T00:12:12.513 INFO:teuthology.orchestra.run.smithi119.stdout:(108/174): grpc-data-1.46.7-10.el9.noarch.rpm 148 kB/s | 19 kB 00:00 2024-03-28T00:12:12.563 INFO:teuthology.orchestra.run.smithi119.stdout:(109/174): protobuf-compiler-3.14.0-13.el9.x86_ 1.2 MB/s | 863 kB 00:00 2024-03-28T00:12:12.580 INFO:teuthology.orchestra.run.smithi119.stdout:(110/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 394 kB/s | 26 kB 00:00 2024-03-28T00:12:12.631 INFO:teuthology.orchestra.run.smithi119.stdout:(111/174): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-03-28T00:12:12.664 INFO:teuthology.orchestra.run.smithi119.stdout:(112/174): liboath-2.6.7-2.el9.x86_64.rpm 485 kB/s | 49 kB 00:00 2024-03-28T00:12:12.706 INFO:teuthology.orchestra.run.smithi119.stdout:(113/174): luarocks-3.9.2-1.el9.noarch.rpm 2.0 MB/s | 151 kB 00:00 2024-03-28T00:12:12.773 INFO:teuthology.orchestra.run.smithi119.stdout:(114/174): python3-asyncssh-2.13.2-5.el9.noarch 8.1 MB/s | 548 kB 00:00 2024-03-28T00:12:12.824 INFO:teuthology.orchestra.run.smithi119.stdout:(115/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 883 kB/s | 43 kB 00:00 2024-03-28T00:12:12.891 INFO:teuthology.orchestra.run.smithi119.stdout:(116/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 3.6 MB/s | 838 kB 00:00 2024-03-28T00:12:12.991 INFO:teuthology.orchestra.run.smithi119.stdout:(117/174): libarrow-9.0.0-10.el9.x86_64.rpm 9.0 MB/s | 4.4 MB 00:00 2024-03-28T00:12:13.008 INFO:teuthology.orchestra.run.smithi119.stdout:(118/174): python3-cachetools-4.2.4-1.el9.noarc 176 kB/s | 32 kB 00:00 2024-03-28T00:12:13.025 INFO:teuthology.orchestra.run.smithi119.stdout:(119/174): python3-certifi-2021.10.8-2.el9.noar 113 kB/s | 15 kB 00:00 2024-03-28T00:12:13.058 INFO:teuthology.orchestra.run.smithi119.stdout:(120/174): python3-cheroot-8.6.0-4.el9.noarch.r 2.5 MB/s | 172 kB 00:00 2024-03-28T00:12:13.100 INFO:teuthology.orchestra.run.smithi119.stdout:(121/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 4.0 MB/s | 303 kB 00:00 2024-03-28T00:12:13.126 INFO:teuthology.orchestra.run.smithi119.stdout:(122/174): python3-colorama-0.4.4-12.el9.noarch 533 kB/s | 35 kB 00:00 2024-03-28T00:12:13.159 INFO:teuthology.orchestra.run.smithi119.stdout:(123/174): python3-cherrypy-18.6.1-2.el9.noarch 2.6 MB/s | 358 kB 00:00 2024-03-28T00:12:13.192 INFO:teuthology.orchestra.run.smithi119.stdout:(124/174): python3-google-auth-2.28.2-1.el9.noa 2.2 MB/s | 203 kB 00:00 2024-03-28T00:12:13.226 INFO:teuthology.orchestra.run.smithi119.stdout:(125/174): python3-grpcio-tools-1.46.7-10.el9.x 2.1 MB/s | 144 kB 00:00 2024-03-28T00:12:13.252 INFO:teuthology.orchestra.run.smithi119.stdout:(126/174): python3-jaraco-8.2.1-3.el9.noarch.rp 184 kB/s | 11 kB 00:00 2024-03-28T00:12:13.277 INFO:teuthology.orchestra.run.smithi119.stdout:(127/174): python3-jaraco-classes-3.2.1-5.el9.n 357 kB/s | 18 kB 00:00 2024-03-28T00:12:13.302 INFO:teuthology.orchestra.run.smithi119.stdout:(128/174): python3-jaraco-collections-3.0.0-8.e 468 kB/s | 23 kB 00:00 2024-03-28T00:12:13.362 INFO:teuthology.orchestra.run.smithi119.stdout:(129/174): python3-jaraco-functools-3.5.0-2.el9 232 kB/s | 19 kB 00:00 2024-03-28T00:12:13.395 INFO:teuthology.orchestra.run.smithi119.stdout:(130/174): python3-jaraco-text-3.2.0-6.el9.noar 214 kB/s | 20 kB 00:00 2024-03-28T00:12:13.486 INFO:teuthology.orchestra.run.smithi119.stdout:(131/174): python3-grpcio-1.46.7-10.el9.x86_64. 5.7 MB/s | 2.0 MB 00:00 2024-03-28T00:12:13.512 INFO:teuthology.orchestra.run.smithi119.stdout:(132/174): python3-logutils-0.3.5-21.el9.noarch 400 kB/s | 46 kB 00:00 2024-03-28T00:12:13.554 INFO:teuthology.orchestra.run.smithi119.stdout:(133/174): python3-kubernetes-26.1.0-2.el9.noar 5.4 MB/s | 1.0 MB 00:00 2024-03-28T00:12:13.579 INFO:teuthology.orchestra.run.smithi119.stdout:(134/174): python3-natsort-7.1.1-5.el9.noarch.r 870 kB/s | 58 kB 00:00 2024-03-28T00:12:13.605 INFO:teuthology.orchestra.run.smithi119.stdout:(135/174): python3-more-itertools-8.12.0-2.el9. 673 kB/s | 79 kB 00:00 2024-03-28T00:12:13.630 INFO:teuthology.orchestra.run.smithi119.stdout:(136/174): python3-portend-3.1.0-2.el9.noarch.r 330 kB/s | 16 kB 00:00 2024-03-28T00:12:13.655 INFO:teuthology.orchestra.run.smithi119.stdout:(137/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.8 MB/s | 90 kB 00:00 2024-03-28T00:12:13.672 INFO:teuthology.orchestra.run.smithi119.stdout:(138/174): python3-pyperclip-1.8.2-2.el9.noarch 687 kB/s | 28 kB 00:00 2024-03-28T00:12:13.706 INFO:teuthology.orchestra.run.smithi119.stdout:(139/174): python3-pecan-1.4.2-3.el9.noarch.rpm 1.8 MB/s | 272 kB 00:00 2024-03-28T00:12:13.731 INFO:teuthology.orchestra.run.smithi119.stdout:(140/174): python3-repoze-lru-0.7-16.el9.noarch 411 kB/s | 31 kB 00:00 2024-03-28T00:12:13.756 INFO:teuthology.orchestra.run.smithi119.stdout:(141/174): python3-rsa-4.9-2.el9.noarch.rpm 1.2 MB/s | 59 kB 00:00 2024-03-28T00:12:13.773 INFO:teuthology.orchestra.run.smithi119.stdout:(142/174): python3-tempora-5.0.0-2.el9.noarch.r 863 kB/s | 36 kB 00:00 2024-03-28T00:12:13.799 INFO:teuthology.orchestra.run.smithi119.stdout:(143/174): python3-routes-2.5.1-5.el9.noarch.rp 1.5 MB/s | 188 kB 00:00 2024-03-28T00:12:13.816 INFO:teuthology.orchestra.run.smithi119.stdout:(144/174): python3-typing-extensions-4.4.0-2.el 873 kB/s | 51 kB 00:00 2024-03-28T00:12:13.874 INFO:teuthology.orchestra.run.smithi119.stdout:(145/174): python3-webob-1.8.7-6.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-03-28T00:12:13.900 INFO:teuthology.orchestra.run.smithi119.stdout:(146/174): python3-websocket-client-1.2.3-2.el9 899 kB/s | 90 kB 00:00 2024-03-28T00:12:13.942 INFO:teuthology.orchestra.run.smithi119.stdout:(147/174): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-03-28T00:12:13.958 INFO:teuthology.orchestra.run.smithi119.stdout:(148/174): python3-zc-lockfile-2.0-10.el9.noarc 240 kB/s | 20 kB 00:00 2024-03-28T00:12:13.992 INFO:teuthology.orchestra.run.smithi119.stdout:(149/174): re2-20211101-3.el9.x86_64.rpm 2.0 MB/s | 192 kB 00:00 2024-03-28T00:12:14.134 INFO:teuthology.orchestra.run.smithi119.stdout:(150/174): thrift-0.15.0-2.el9.x86_64.rpm 8.3 MB/s | 1.6 MB 00:00 2024-03-28T00:12:14.376 INFO:teuthology.orchestra.run.smithi119.stdout:(151/174): xfsdump-3.1.12-4.el9.x86_64.rpm 1.7 MB/s | 321 kB 00:00 2024-03-28T00:12:14.543 INFO:teuthology.orchestra.run.smithi119.stdout:(152/174): librados2-19.0.0-2145.g3be9a5c9.el9. 5.8 MB/s | 3.4 MB 00:00 2024-03-28T00:12:14.838 INFO:teuthology.orchestra.run.smithi119.stdout:(153/174): NetworkManager-libnm-1.47.2-1.el9.x8 6.2 MB/s | 1.8 MB 00:00 2024-03-28T00:12:14.919 INFO:teuthology.orchestra.run.smithi119.stdout:(154/174): NetworkManager-1.47.2-1.el9.x86_64.r 4.3 MB/s | 2.3 MB 00:00 2024-03-28T00:12:14.944 INFO:teuthology.orchestra.run.smithi119.stdout:(155/174): NetworkManager-team-1.47.2-1.el9.x86 401 kB/s | 42 kB 00:00 2024-03-28T00:12:14.994 INFO:teuthology.orchestra.run.smithi119.stdout:(156/174): acl-2.3.1-4.el9.x86_64.rpm 1.4 MB/s | 71 kB 00:00 2024-03-28T00:12:15.028 INFO:teuthology.orchestra.run.smithi119.stdout:(157/174): NetworkManager-tui-1.47.2-1.el9.x86_ 2.2 MB/s | 248 kB 00:00 2024-03-28T00:12:15.053 INFO:teuthology.orchestra.run.smithi119.stdout:(158/174): elfutils-debuginfod-client-0.190-2.e 631 kB/s | 36 kB 00:00 2024-03-28T00:12:15.087 INFO:teuthology.orchestra.run.smithi119.stdout:(159/174): elfutils-libelf-0.190-2.el9.x86_64.r 3.3 MB/s | 192 kB 00:00 2024-03-28T00:12:15.120 INFO:teuthology.orchestra.run.smithi119.stdout:(160/174): elfutils-libs-0.190-2.el9.x86_64.rpm 3.8 MB/s | 256 kB 00:00 2024-03-28T00:12:15.146 INFO:teuthology.orchestra.run.smithi119.stdout:(161/174): libacl-2.3.1-4.el9.x86_64.rpm 398 kB/s | 23 kB 00:00 2024-03-28T00:12:15.171 INFO:teuthology.orchestra.run.smithi119.stdout:(162/174): libblkid-2.37.4-18.el9.x86_64.rpm 2.1 MB/s | 108 kB 00:00 2024-03-28T00:12:15.221 INFO:teuthology.orchestra.run.smithi119.stdout:(163/174): libfdisk-2.37.4-18.el9.x86_64.rpm 2.0 MB/s | 155 kB 00:00 2024-03-28T00:12:15.224 INFO:teuthology.orchestra.run.smithi132.stdout:lab-extras 26 kB/s | 1.7 kB 00:00 2024-03-28T00:12:15.255 INFO:teuthology.orchestra.run.smithi119.stdout:(164/174): libmount-2.37.4-18.el9.x86_64.rpm 1.6 MB/s | 136 kB 00:00 2024-03-28T00:12:15.389 INFO:teuthology.orchestra.run.smithi119.stdout:(165/174): librbd1-19.0.0-2145.g3be9a5c9.el9.x8 2.4 MB/s | 3.1 MB 00:01 2024-03-28T00:12:15.414 INFO:teuthology.orchestra.run.smithi119.stdout:(166/174): libuuid-2.37.4-18.el9.x86_64.rpm 181 kB/s | 29 kB 00:00 2024-03-28T00:12:15.439 INFO:teuthology.orchestra.run.smithi119.stdout:(167/174): libsmartcols-2.37.4-18.el9.x86_64.rp 294 kB/s | 64 kB 00:00 2024-03-28T00:12:15.623 INFO:teuthology.orchestra.run.smithi119.stdout:(168/174): openssl-3.0.7-27.el9.x86_64.rpm 5.2 MB/s | 1.2 MB 00:00 2024-03-28T00:12:15.716 INFO:teuthology.orchestra.run.smithi119.stdout:(169/174): util-linux-2.37.4-18.el9.x86_64.rpm 8.3 MB/s | 2.3 MB 00:00 2024-03-28T00:12:15.757 INFO:teuthology.orchestra.run.smithi119.stdout:(170/174): util-linux-user-2.37.4-18.el9.x86_64 762 kB/s | 31 kB 00:00 2024-03-28T00:12:15.791 INFO:teuthology.orchestra.run.smithi119.stdout:(171/174): util-linux-core-2.37.4-18.el9.x86_64 2.7 MB/s | 465 kB 00:00 2024-03-28T00:12:15.866 INFO:teuthology.orchestra.run.smithi119.stdout:(172/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 4.8 MB/s | 2.2 MB 00:00 2024-03-28T00:12:15.941 INFO:teuthology.orchestra.run.smithi119.stdout:(173/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 5.9 MB/s | 1.1 MB 00:00 2024-03-28T00:12:16.016 INFO:teuthology.orchestra.run.smithi119.stdout:(174/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 331 kB/s | 74 kB 00:00 2024-03-28T00:12:16.026 INFO:teuthology.orchestra.run.smithi119.stdout:-------------------------------------------------------------------------------- 2024-03-28T00:12:16.026 INFO:teuthology.orchestra.run.smithi119.stdout:Total 13 MB/s | 225 MB 00:17 2024-03-28T00:12:16.032 INFO:teuthology.orchestra.run.smithi165.stdout:lab-extras 12 kB/s | 1.7 kB 00:00 2024-03-28T00:12:17.606 INFO:teuthology.orchestra.run.smithi132.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-28T00:12:17.607 INFO:teuthology.orchestra.run.smithi132.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-28T00:12:17.622 INFO:teuthology.orchestra.run.smithi132.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-28T00:12:17.623 INFO:teuthology.orchestra.run.smithi132.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-28T00:12:17.624 INFO:teuthology.orchestra.run.smithi132.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-28T00:12:17.625 INFO:teuthology.orchestra.run.smithi132.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-28T00:12:17.627 INFO:teuthology.orchestra.run.smithi132.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-28T00:12:17.630 INFO:teuthology.orchestra.run.smithi132.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-28T00:12:17.631 INFO:teuthology.orchestra.run.smithi132.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-28T00:12:17.664 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction check 2024-03-28T00:12:17.712 INFO:teuthology.orchestra.run.smithi132.stdout:Dependencies resolved. 2024-03-28T00:12:17.732 INFO:teuthology.orchestra.run.smithi132.stdout:============================================================================================ 2024-03-28T00:12:17.732 INFO:teuthology.orchestra.run.smithi132.stdout: Package Arch Version Repository Size 2024-03-28T00:12:17.733 INFO:teuthology.orchestra.run.smithi132.stdout:============================================================================================ 2024-03-28T00:12:17.733 INFO:teuthology.orchestra.run.smithi132.stdout:Installing: 2024-03-28T00:12:17.733 INFO:teuthology.orchestra.run.smithi132.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-28T00:12:17.733 INFO:teuthology.orchestra.run.smithi132.stdout: ceph x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 6.6 k 2024-03-28T00:12:17.733 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-base x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 5.4 M 2024-03-28T00:12:17.733 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-fuse x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 890 k 2024-03-28T00:12:17.733 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 145 k 2024-03-28T00:12:17.733 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mgr x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 1.5 M 2024-03-28T00:12:17.733 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 139 k 2024-03-28T00:12:17.733 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 1.8 M 2024-03-28T00:12:17.733 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 7.4 M 2024-03-28T00:12:17.733 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mgr-rook noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 49 k 2024-03-28T00:12:17.733 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-radosgw x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 9.9 M 2024-03-28T00:12:17.733 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-test x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 48 M 2024-03-28T00:12:17.734 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-volume noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 264 k 2024-03-28T00:12:17.734 INFO:teuthology.orchestra.run.smithi132.stdout: cephadm noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 748 k 2024-03-28T00:12:17.734 INFO:teuthology.orchestra.run.smithi132.stdout: cephfs-mirror x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 220 k 2024-03-28T00:12:17.734 INFO:teuthology.orchestra.run.smithi132.stdout: cephfs-shell x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 27 k 2024-03-28T00:12:17.734 INFO:teuthology.orchestra.run.smithi132.stdout: cephfs-top noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 18 k 2024-03-28T00:12:17.734 INFO:teuthology.orchestra.run.smithi132.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-03-28T00:12:17.734 INFO:teuthology.orchestra.run.smithi132.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-28T00:12:17.734 INFO:teuthology.orchestra.run.smithi132.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-28T00:12:17.734 INFO:teuthology.orchestra.run.smithi132.stdout: libcephfs-devel x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 31 k 2024-03-28T00:12:17.734 INFO:teuthology.orchestra.run.smithi132.stdout: libcephfs2 x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 746 k 2024-03-28T00:12:17.734 INFO:teuthology.orchestra.run.smithi132.stdout: librados-devel x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 127 k 2024-03-28T00:12:17.734 INFO:teuthology.orchestra.run.smithi132.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-28T00:12:17.734 INFO:teuthology.orchestra.run.smithi132.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-03-28T00:12:17.735 INFO:teuthology.orchestra.run.smithi132.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-03-28T00:12:17.735 INFO:teuthology.orchestra.run.smithi132.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-03-28T00:12:17.735 INFO:teuthology.orchestra.run.smithi132.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-03-28T00:12:17.735 INFO:teuthology.orchestra.run.smithi132.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-03-28T00:12:17.735 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cephfs x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 162 k 2024-03-28T00:12:17.735 INFO:teuthology.orchestra.run.smithi132.stdout: python3-rados x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 321 k 2024-03-28T00:12:17.735 INFO:teuthology.orchestra.run.smithi132.stdout: python3-rbd x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 298 k 2024-03-28T00:12:17.735 INFO:teuthology.orchestra.run.smithi132.stdout: python3-rgw x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 99 k 2024-03-28T00:12:17.735 INFO:teuthology.orchestra.run.smithi132.stdout: rbd-fuse x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 85 k 2024-03-28T00:12:17.735 INFO:teuthology.orchestra.run.smithi132.stdout: rbd-mirror x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 3.1 M 2024-03-28T00:12:17.735 INFO:teuthology.orchestra.run.smithi132.stdout: rbd-nbd x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 170 k 2024-03-28T00:12:17.735 INFO:teuthology.orchestra.run.smithi132.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-03-28T00:12:17.735 INFO:teuthology.orchestra.run.smithi132.stdout:Upgrading: 2024-03-28T00:12:17.735 INFO:teuthology.orchestra.run.smithi132.stdout: NetworkManager x86_64 1:1.47.2-1.el9 baseos 2.3 M 2024-03-28T00:12:17.736 INFO:teuthology.orchestra.run.smithi132.stdout: NetworkManager-libnm x86_64 1:1.47.2-1.el9 baseos 1.8 M 2024-03-28T00:12:17.736 INFO:teuthology.orchestra.run.smithi132.stdout: NetworkManager-team x86_64 1:1.47.2-1.el9 baseos 42 k 2024-03-28T00:12:17.736 INFO:teuthology.orchestra.run.smithi132.stdout: NetworkManager-tui x86_64 1:1.47.2-1.el9 baseos 248 k 2024-03-28T00:12:17.736 INFO:teuthology.orchestra.run.smithi132.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-03-28T00:12:17.736 INFO:teuthology.orchestra.run.smithi132.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-03-28T00:12:17.736 INFO:teuthology.orchestra.run.smithi132.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-03-28T00:12:17.736 INFO:teuthology.orchestra.run.smithi132.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-03-28T00:12:17.736 INFO:teuthology.orchestra.run.smithi132.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-03-28T00:12:17.736 INFO:teuthology.orchestra.run.smithi132.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-03-28T00:12:17.736 INFO:teuthology.orchestra.run.smithi132.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-03-28T00:12:17.736 INFO:teuthology.orchestra.run.smithi132.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-03-28T00:12:17.736 INFO:teuthology.orchestra.run.smithi132.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-03-28T00:12:17.736 INFO:teuthology.orchestra.run.smithi132.stdout: librados2 x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 3.4 M 2024-03-28T00:12:17.737 INFO:teuthology.orchestra.run.smithi132.stdout: librbd1 x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 3.1 M 2024-03-28T00:12:17.737 INFO:teuthology.orchestra.run.smithi132.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-03-28T00:12:17.737 INFO:teuthology.orchestra.run.smithi132.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-03-28T00:12:17.737 INFO:teuthology.orchestra.run.smithi132.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-03-28T00:12:17.737 INFO:teuthology.orchestra.run.smithi132.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-03-28T00:12:17.737 INFO:teuthology.orchestra.run.smithi132.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-03-28T00:12:17.737 INFO:teuthology.orchestra.run.smithi132.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-03-28T00:12:17.737 INFO:teuthology.orchestra.run.smithi132.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-03-28T00:12:17.737 INFO:teuthology.orchestra.run.smithi132.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-28T00:12:17.737 INFO:teuthology.orchestra.run.smithi132.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-03-28T00:12:17.737 INFO:teuthology.orchestra.run.smithi132.stdout:Installing dependencies: 2024-03-28T00:12:17.737 INFO:teuthology.orchestra.run.smithi132.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-28T00:12:17.737 INFO:teuthology.orchestra.run.smithi132.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-28T00:12:17.737 INFO:teuthology.orchestra.run.smithi132.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-28T00:12:17.737 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-common x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 20 M 2024-03-28T00:12:17.738 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 34 k 2024-03-28T00:12:17.738 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mds x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 2.3 M 2024-03-28T00:12:17.738 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 248 k 2024-03-28T00:12:17.738 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mon x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 4.7 M 2024-03-28T00:12:17.738 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-osd x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 17 M 2024-03-28T00:12:17.738 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 16 k 2024-03-28T00:12:17.738 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-selinux x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 25 k 2024-03-28T00:12:17.738 INFO:teuthology.orchestra.run.smithi132.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-28T00:12:17.738 INFO:teuthology.orchestra.run.smithi132.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-28T00:12:17.738 INFO:teuthology.orchestra.run.smithi132.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-28T00:12:17.738 INFO:teuthology.orchestra.run.smithi132.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-28T00:12:17.738 INFO:teuthology.orchestra.run.smithi132.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-28T00:12:17.738 INFO:teuthology.orchestra.run.smithi132.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-28T00:12:17.738 INFO:teuthology.orchestra.run.smithi132.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-28T00:12:17.739 INFO:teuthology.orchestra.run.smithi132.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-28T00:12:17.739 INFO:teuthology.orchestra.run.smithi132.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-28T00:12:17.739 INFO:teuthology.orchestra.run.smithi132.stdout: libcephsqlite x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 163 k 2024-03-28T00:12:17.739 INFO:teuthology.orchestra.run.smithi132.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-28T00:12:17.739 INFO:teuthology.orchestra.run.smithi132.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-28T00:12:17.739 INFO:teuthology.orchestra.run.smithi132.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-28T00:12:17.739 INFO:teuthology.orchestra.run.smithi132.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-28T00:12:17.739 INFO:teuthology.orchestra.run.smithi132.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-28T00:12:17.739 INFO:teuthology.orchestra.run.smithi132.stdout: libradosstriper1 x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 501 k 2024-03-28T00:12:17.739 INFO:teuthology.orchestra.run.smithi132.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-28T00:12:17.739 INFO:teuthology.orchestra.run.smithi132.stdout: librgw2 x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 5.1 M 2024-03-28T00:12:17.739 INFO:teuthology.orchestra.run.smithi132.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-28T00:12:17.739 INFO:teuthology.orchestra.run.smithi132.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-28T00:12:17.739 INFO:teuthology.orchestra.run.smithi132.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-28T00:12:17.739 INFO:teuthology.orchestra.run.smithi132.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-28T00:12:17.740 INFO:teuthology.orchestra.run.smithi132.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-28T00:12:17.740 INFO:teuthology.orchestra.run.smithi132.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-28T00:12:17.740 INFO:teuthology.orchestra.run.smithi132.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-28T00:12:17.740 INFO:teuthology.orchestra.run.smithi132.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-28T00:12:17.740 INFO:teuthology.orchestra.run.smithi132.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-28T00:12:17.740 INFO:teuthology.orchestra.run.smithi132.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-28T00:12:17.740 INFO:teuthology.orchestra.run.smithi132.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-03-28T00:12:17.740 INFO:teuthology.orchestra.run.smithi132.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-28T00:12:17.740 INFO:teuthology.orchestra.run.smithi132.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-28T00:12:17.740 INFO:teuthology.orchestra.run.smithi132.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-03-28T00:12:17.740 INFO:teuthology.orchestra.run.smithi132.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-03-28T00:12:17.740 INFO:teuthology.orchestra.run.smithi132.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-28T00:12:17.740 INFO:teuthology.orchestra.run.smithi132.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-28T00:12:17.740 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-28T00:12:17.741 INFO:teuthology.orchestra.run.smithi132.stdout: python3-ceph-argparse x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 45 k 2024-03-28T00:12:17.741 INFO:teuthology.orchestra.run.smithi132.stdout: python3-ceph-common x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 131 k 2024-03-28T00:12:17.741 INFO:teuthology.orchestra.run.smithi132.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-28T00:12:17.741 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-28T00:12:17.741 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-28T00:12:17.741 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-28T00:12:17.741 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-03-28T00:12:17.741 INFO:teuthology.orchestra.run.smithi132.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-03-28T00:12:17.741 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-28T00:12:17.741 INFO:teuthology.orchestra.run.smithi132.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-28T00:12:17.741 INFO:teuthology.orchestra.run.smithi132.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-03-28T00:12:17.741 INFO:teuthology.orchestra.run.smithi132.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-28T00:12:17.741 INFO:teuthology.orchestra.run.smithi132.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-28T00:12:17.741 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-28T00:12:17.742 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-28T00:12:17.742 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-28T00:12:17.742 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-28T00:12:17.742 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-28T00:12:17.742 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-28T00:12:17.742 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-28T00:12:17.742 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-28T00:12:17.742 INFO:teuthology.orchestra.run.smithi132.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-28T00:12:17.742 INFO:teuthology.orchestra.run.smithi132.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-28T00:12:17.742 INFO:teuthology.orchestra.run.smithi132.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-28T00:12:17.742 INFO:teuthology.orchestra.run.smithi132.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-28T00:12:17.742 INFO:teuthology.orchestra.run.smithi132.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-28T00:12:17.742 INFO:teuthology.orchestra.run.smithi132.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-28T00:12:17.742 INFO:teuthology.orchestra.run.smithi132.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-28T00:12:17.743 INFO:teuthology.orchestra.run.smithi132.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-28T00:12:17.743 INFO:teuthology.orchestra.run.smithi132.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-28T00:12:17.743 INFO:teuthology.orchestra.run.smithi132.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-28T00:12:17.743 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-28T00:12:17.743 INFO:teuthology.orchestra.run.smithi132.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-28T00:12:17.743 INFO:teuthology.orchestra.run.smithi132.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-28T00:12:17.743 INFO:teuthology.orchestra.run.smithi132.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-28T00:12:17.743 INFO:teuthology.orchestra.run.smithi132.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-28T00:12:17.743 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-28T00:12:17.743 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-28T00:12:17.743 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-28T00:12:17.743 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-28T00:12:17.743 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-03-28T00:12:17.743 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-28T00:12:17.744 INFO:teuthology.orchestra.run.smithi132.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-28T00:12:17.744 INFO:teuthology.orchestra.run.smithi132.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-28T00:12:17.744 INFO:teuthology.orchestra.run.smithi132.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-28T00:12:17.744 INFO:teuthology.orchestra.run.smithi132.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-28T00:12:17.744 INFO:teuthology.orchestra.run.smithi132.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-28T00:12:17.744 INFO:teuthology.orchestra.run.smithi132.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-28T00:12:17.744 INFO:teuthology.orchestra.run.smithi132.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-28T00:12:17.744 INFO:teuthology.orchestra.run.smithi132.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-28T00:12:17.744 INFO:teuthology.orchestra.run.smithi132.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-28T00:12:17.744 INFO:teuthology.orchestra.run.smithi132.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-28T00:12:17.744 INFO:teuthology.orchestra.run.smithi132.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-03-28T00:12:17.744 INFO:teuthology.orchestra.run.smithi132.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-28T00:12:17.744 INFO:teuthology.orchestra.run.smithi132.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-28T00:12:17.744 INFO:teuthology.orchestra.run.smithi132.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-28T00:12:17.744 INFO:teuthology.orchestra.run.smithi132.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-28T00:12:17.745 INFO:teuthology.orchestra.run.smithi132.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-03-28T00:12:17.745 INFO:teuthology.orchestra.run.smithi132.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-03-28T00:12:17.745 INFO:teuthology.orchestra.run.smithi132.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-28T00:12:17.745 INFO:teuthology.orchestra.run.smithi132.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-28T00:12:17.745 INFO:teuthology.orchestra.run.smithi132.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-28T00:12:17.745 INFO:teuthology.orchestra.run.smithi132.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-28T00:12:17.745 INFO:teuthology.orchestra.run.smithi132.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-28T00:12:17.745 INFO:teuthology.orchestra.run.smithi132.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-28T00:12:17.745 INFO:teuthology.orchestra.run.smithi132.stdout:Installing weak dependencies: 2024-03-28T00:12:17.745 INFO:teuthology.orchestra.run.smithi132.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-03-28T00:12:17.745 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-28T00:12:17.745 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction Summary 2024-03-28T00:12:17.745 INFO:teuthology.orchestra.run.smithi132.stdout:============================================================================================ 2024-03-28T00:12:17.745 INFO:teuthology.orchestra.run.smithi132.stdout:Install 150 Packages 2024-03-28T00:12:17.746 INFO:teuthology.orchestra.run.smithi132.stdout:Upgrade 24 Packages 2024-03-28T00:12:17.746 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-28T00:12:17.746 INFO:teuthology.orchestra.run.smithi132.stdout:Total download size: 225 M 2024-03-28T00:12:17.746 INFO:teuthology.orchestra.run.smithi132.stdout:Downloading Packages: 2024-03-28T00:12:18.044 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction check succeeded. 2024-03-28T00:12:18.044 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction test 2024-03-28T00:12:18.432 INFO:teuthology.orchestra.run.smithi165.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-28T00:12:18.433 INFO:teuthology.orchestra.run.smithi165.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-28T00:12:18.447 INFO:teuthology.orchestra.run.smithi165.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-28T00:12:18.448 INFO:teuthology.orchestra.run.smithi165.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-28T00:12:18.449 INFO:teuthology.orchestra.run.smithi165.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-28T00:12:18.450 INFO:teuthology.orchestra.run.smithi165.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-28T00:12:18.452 INFO:teuthology.orchestra.run.smithi165.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-28T00:12:18.455 INFO:teuthology.orchestra.run.smithi165.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-28T00:12:18.456 INFO:teuthology.orchestra.run.smithi165.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-28T00:12:18.537 INFO:teuthology.orchestra.run.smithi165.stdout:Dependencies resolved. 2024-03-28T00:12:18.560 INFO:teuthology.orchestra.run.smithi165.stdout:============================================================================================ 2024-03-28T00:12:18.560 INFO:teuthology.orchestra.run.smithi165.stdout: Package Arch Version Repository Size 2024-03-28T00:12:18.560 INFO:teuthology.orchestra.run.smithi165.stdout:============================================================================================ 2024-03-28T00:12:18.560 INFO:teuthology.orchestra.run.smithi165.stdout:Installing: 2024-03-28T00:12:18.560 INFO:teuthology.orchestra.run.smithi165.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-28T00:12:18.560 INFO:teuthology.orchestra.run.smithi165.stdout: ceph x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 6.6 k 2024-03-28T00:12:18.560 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-base x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 5.4 M 2024-03-28T00:12:18.560 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-fuse x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 890 k 2024-03-28T00:12:18.560 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 145 k 2024-03-28T00:12:18.560 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 1.5 M 2024-03-28T00:12:18.560 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 139 k 2024-03-28T00:12:18.561 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 1.8 M 2024-03-28T00:12:18.561 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 7.4 M 2024-03-28T00:12:18.561 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr-rook noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 49 k 2024-03-28T00:12:18.561 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-radosgw x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 9.9 M 2024-03-28T00:12:18.561 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-test x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 48 M 2024-03-28T00:12:18.561 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-volume noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 264 k 2024-03-28T00:12:18.561 INFO:teuthology.orchestra.run.smithi165.stdout: cephadm noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 748 k 2024-03-28T00:12:18.561 INFO:teuthology.orchestra.run.smithi165.stdout: cephfs-mirror x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 220 k 2024-03-28T00:12:18.561 INFO:teuthology.orchestra.run.smithi165.stdout: cephfs-shell x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 27 k 2024-03-28T00:12:18.561 INFO:teuthology.orchestra.run.smithi165.stdout: cephfs-top noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 18 k 2024-03-28T00:12:18.561 INFO:teuthology.orchestra.run.smithi165.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-03-28T00:12:18.561 INFO:teuthology.orchestra.run.smithi165.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-28T00:12:18.561 INFO:teuthology.orchestra.run.smithi165.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-28T00:12:18.562 INFO:teuthology.orchestra.run.smithi165.stdout: libcephfs-devel x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 31 k 2024-03-28T00:12:18.562 INFO:teuthology.orchestra.run.smithi165.stdout: libcephfs2 x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 746 k 2024-03-28T00:12:18.562 INFO:teuthology.orchestra.run.smithi165.stdout: librados-devel x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 127 k 2024-03-28T00:12:18.562 INFO:teuthology.orchestra.run.smithi165.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-28T00:12:18.562 INFO:teuthology.orchestra.run.smithi165.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-03-28T00:12:18.562 INFO:teuthology.orchestra.run.smithi165.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-03-28T00:12:18.562 INFO:teuthology.orchestra.run.smithi165.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-03-28T00:12:18.562 INFO:teuthology.orchestra.run.smithi165.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-03-28T00:12:18.562 INFO:teuthology.orchestra.run.smithi165.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-03-28T00:12:18.562 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cephfs x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 162 k 2024-03-28T00:12:18.562 INFO:teuthology.orchestra.run.smithi165.stdout: python3-rados x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 321 k 2024-03-28T00:12:18.562 INFO:teuthology.orchestra.run.smithi165.stdout: python3-rbd x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 298 k 2024-03-28T00:12:18.563 INFO:teuthology.orchestra.run.smithi165.stdout: python3-rgw x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 99 k 2024-03-28T00:12:18.563 INFO:teuthology.orchestra.run.smithi165.stdout: rbd-fuse x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 85 k 2024-03-28T00:12:18.563 INFO:teuthology.orchestra.run.smithi165.stdout: rbd-mirror x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 3.1 M 2024-03-28T00:12:18.563 INFO:teuthology.orchestra.run.smithi165.stdout: rbd-nbd x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 170 k 2024-03-28T00:12:18.563 INFO:teuthology.orchestra.run.smithi165.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-03-28T00:12:18.563 INFO:teuthology.orchestra.run.smithi165.stdout:Upgrading: 2024-03-28T00:12:18.563 INFO:teuthology.orchestra.run.smithi165.stdout: NetworkManager x86_64 1:1.47.2-1.el9 baseos 2.3 M 2024-03-28T00:12:18.563 INFO:teuthology.orchestra.run.smithi165.stdout: NetworkManager-libnm x86_64 1:1.47.2-1.el9 baseos 1.8 M 2024-03-28T00:12:18.563 INFO:teuthology.orchestra.run.smithi165.stdout: NetworkManager-team x86_64 1:1.47.2-1.el9 baseos 42 k 2024-03-28T00:12:18.563 INFO:teuthology.orchestra.run.smithi165.stdout: NetworkManager-tui x86_64 1:1.47.2-1.el9 baseos 248 k 2024-03-28T00:12:18.563 INFO:teuthology.orchestra.run.smithi165.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-03-28T00:12:18.564 INFO:teuthology.orchestra.run.smithi165.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-03-28T00:12:18.564 INFO:teuthology.orchestra.run.smithi165.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-03-28T00:12:18.564 INFO:teuthology.orchestra.run.smithi165.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-03-28T00:12:18.564 INFO:teuthology.orchestra.run.smithi165.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-03-28T00:12:18.564 INFO:teuthology.orchestra.run.smithi165.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-03-28T00:12:18.564 INFO:teuthology.orchestra.run.smithi165.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-03-28T00:12:18.564 INFO:teuthology.orchestra.run.smithi165.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-03-28T00:12:18.564 INFO:teuthology.orchestra.run.smithi165.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-03-28T00:12:18.564 INFO:teuthology.orchestra.run.smithi165.stdout: librados2 x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 3.4 M 2024-03-28T00:12:18.564 INFO:teuthology.orchestra.run.smithi165.stdout: librbd1 x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 3.1 M 2024-03-28T00:12:18.564 INFO:teuthology.orchestra.run.smithi165.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-03-28T00:12:18.564 INFO:teuthology.orchestra.run.smithi165.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-03-28T00:12:18.564 INFO:teuthology.orchestra.run.smithi165.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-03-28T00:12:18.565 INFO:teuthology.orchestra.run.smithi165.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-03-28T00:12:18.565 INFO:teuthology.orchestra.run.smithi165.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-03-28T00:12:18.565 INFO:teuthology.orchestra.run.smithi165.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-03-28T00:12:18.565 INFO:teuthology.orchestra.run.smithi165.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-03-28T00:12:18.565 INFO:teuthology.orchestra.run.smithi165.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-28T00:12:18.565 INFO:teuthology.orchestra.run.smithi165.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-03-28T00:12:18.565 INFO:teuthology.orchestra.run.smithi165.stdout:Installing dependencies: 2024-03-28T00:12:18.565 INFO:teuthology.orchestra.run.smithi165.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-28T00:12:18.565 INFO:teuthology.orchestra.run.smithi165.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-28T00:12:18.565 INFO:teuthology.orchestra.run.smithi165.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-28T00:12:18.565 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-common x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 20 M 2024-03-28T00:12:18.565 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 34 k 2024-03-28T00:12:18.565 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mds x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 2.3 M 2024-03-28T00:12:18.566 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 248 k 2024-03-28T00:12:18.566 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mon x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 4.7 M 2024-03-28T00:12:18.566 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-osd x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 17 M 2024-03-28T00:12:18.566 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 16 k 2024-03-28T00:12:18.566 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-selinux x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 25 k 2024-03-28T00:12:18.566 INFO:teuthology.orchestra.run.smithi165.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-28T00:12:18.566 INFO:teuthology.orchestra.run.smithi165.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-28T00:12:18.566 INFO:teuthology.orchestra.run.smithi165.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-28T00:12:18.566 INFO:teuthology.orchestra.run.smithi165.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-28T00:12:18.566 INFO:teuthology.orchestra.run.smithi165.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-28T00:12:18.567 INFO:teuthology.orchestra.run.smithi165.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-28T00:12:18.567 INFO:teuthology.orchestra.run.smithi165.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-28T00:12:18.567 INFO:teuthology.orchestra.run.smithi165.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-28T00:12:18.567 INFO:teuthology.orchestra.run.smithi165.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-28T00:12:18.567 INFO:teuthology.orchestra.run.smithi165.stdout: libcephsqlite x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 163 k 2024-03-28T00:12:18.567 INFO:teuthology.orchestra.run.smithi165.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-28T00:12:18.567 INFO:teuthology.orchestra.run.smithi165.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-28T00:12:18.567 INFO:teuthology.orchestra.run.smithi165.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-28T00:12:18.567 INFO:teuthology.orchestra.run.smithi165.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-28T00:12:18.567 INFO:teuthology.orchestra.run.smithi165.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-28T00:12:18.567 INFO:teuthology.orchestra.run.smithi165.stdout: libradosstriper1 x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 501 k 2024-03-28T00:12:18.567 INFO:teuthology.orchestra.run.smithi165.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-28T00:12:18.567 INFO:teuthology.orchestra.run.smithi165.stdout: librgw2 x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 5.1 M 2024-03-28T00:12:18.567 INFO:teuthology.orchestra.run.smithi165.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-28T00:12:18.568 INFO:teuthology.orchestra.run.smithi165.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-28T00:12:18.568 INFO:teuthology.orchestra.run.smithi165.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-28T00:12:18.568 INFO:teuthology.orchestra.run.smithi165.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-28T00:12:18.568 INFO:teuthology.orchestra.run.smithi165.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-28T00:12:18.568 INFO:teuthology.orchestra.run.smithi165.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-28T00:12:18.568 INFO:teuthology.orchestra.run.smithi165.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-28T00:12:18.568 INFO:teuthology.orchestra.run.smithi165.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-28T00:12:18.568 INFO:teuthology.orchestra.run.smithi165.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-28T00:12:18.568 INFO:teuthology.orchestra.run.smithi165.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-28T00:12:18.568 INFO:teuthology.orchestra.run.smithi165.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-03-28T00:12:18.568 INFO:teuthology.orchestra.run.smithi165.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-28T00:12:18.568 INFO:teuthology.orchestra.run.smithi165.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-28T00:12:18.568 INFO:teuthology.orchestra.run.smithi165.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-03-28T00:12:18.568 INFO:teuthology.orchestra.run.smithi165.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-03-28T00:12:18.569 INFO:teuthology.orchestra.run.smithi165.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-28T00:12:18.569 INFO:teuthology.orchestra.run.smithi165.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-28T00:12:18.569 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-28T00:12:18.569 INFO:teuthology.orchestra.run.smithi165.stdout: python3-ceph-argparse x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 45 k 2024-03-28T00:12:18.569 INFO:teuthology.orchestra.run.smithi165.stdout: python3-ceph-common x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 131 k 2024-03-28T00:12:18.569 INFO:teuthology.orchestra.run.smithi165.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-28T00:12:18.569 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-28T00:12:18.569 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-28T00:12:18.569 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-28T00:12:18.569 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-03-28T00:12:18.569 INFO:teuthology.orchestra.run.smithi165.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-03-28T00:12:18.569 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-28T00:12:18.569 INFO:teuthology.orchestra.run.smithi165.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-28T00:12:18.569 INFO:teuthology.orchestra.run.smithi165.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-03-28T00:12:18.570 INFO:teuthology.orchestra.run.smithi165.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-28T00:12:18.570 INFO:teuthology.orchestra.run.smithi165.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-28T00:12:18.570 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-28T00:12:18.570 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-28T00:12:18.570 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-28T00:12:18.570 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-28T00:12:18.570 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-28T00:12:18.570 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-28T00:12:18.570 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-28T00:12:18.570 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-28T00:12:18.570 INFO:teuthology.orchestra.run.smithi165.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-28T00:12:18.570 INFO:teuthology.orchestra.run.smithi165.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-28T00:12:18.570 INFO:teuthology.orchestra.run.smithi165.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-28T00:12:18.570 INFO:teuthology.orchestra.run.smithi165.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-28T00:12:18.571 INFO:teuthology.orchestra.run.smithi165.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-28T00:12:18.571 INFO:teuthology.orchestra.run.smithi165.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-28T00:12:18.571 INFO:teuthology.orchestra.run.smithi165.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-28T00:12:18.571 INFO:teuthology.orchestra.run.smithi165.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-28T00:12:18.571 INFO:teuthology.orchestra.run.smithi165.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-28T00:12:18.571 INFO:teuthology.orchestra.run.smithi165.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-28T00:12:18.571 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-28T00:12:18.571 INFO:teuthology.orchestra.run.smithi165.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-28T00:12:18.571 INFO:teuthology.orchestra.run.smithi165.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-28T00:12:18.571 INFO:teuthology.orchestra.run.smithi165.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-28T00:12:18.571 INFO:teuthology.orchestra.run.smithi165.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-28T00:12:18.571 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-28T00:12:18.571 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-28T00:12:18.571 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-28T00:12:18.572 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-28T00:12:18.572 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-03-28T00:12:18.572 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-28T00:12:18.572 INFO:teuthology.orchestra.run.smithi165.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-28T00:12:18.572 INFO:teuthology.orchestra.run.smithi165.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-28T00:12:18.572 INFO:teuthology.orchestra.run.smithi165.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-28T00:12:18.572 INFO:teuthology.orchestra.run.smithi165.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-28T00:12:18.572 INFO:teuthology.orchestra.run.smithi165.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-28T00:12:18.572 INFO:teuthology.orchestra.run.smithi165.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-28T00:12:18.572 INFO:teuthology.orchestra.run.smithi165.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-28T00:12:18.572 INFO:teuthology.orchestra.run.smithi165.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-28T00:12:18.572 INFO:teuthology.orchestra.run.smithi165.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-28T00:12:18.572 INFO:teuthology.orchestra.run.smithi165.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-28T00:12:18.572 INFO:teuthology.orchestra.run.smithi165.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-03-28T00:12:18.573 INFO:teuthology.orchestra.run.smithi165.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-28T00:12:18.573 INFO:teuthology.orchestra.run.smithi165.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-28T00:12:18.573 INFO:teuthology.orchestra.run.smithi165.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-28T00:12:18.573 INFO:teuthology.orchestra.run.smithi165.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-28T00:12:18.573 INFO:teuthology.orchestra.run.smithi165.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-03-28T00:12:18.573 INFO:teuthology.orchestra.run.smithi165.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-03-28T00:12:18.573 INFO:teuthology.orchestra.run.smithi165.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-28T00:12:18.573 INFO:teuthology.orchestra.run.smithi165.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-28T00:12:18.573 INFO:teuthology.orchestra.run.smithi165.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-28T00:12:18.573 INFO:teuthology.orchestra.run.smithi165.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-28T00:12:18.573 INFO:teuthology.orchestra.run.smithi165.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-28T00:12:18.573 INFO:teuthology.orchestra.run.smithi165.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-28T00:12:18.573 INFO:teuthology.orchestra.run.smithi165.stdout:Installing weak dependencies: 2024-03-28T00:12:18.573 INFO:teuthology.orchestra.run.smithi165.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-03-28T00:12:18.574 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-03-28T00:12:18.574 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction Summary 2024-03-28T00:12:18.574 INFO:teuthology.orchestra.run.smithi165.stdout:============================================================================================ 2024-03-28T00:12:18.574 INFO:teuthology.orchestra.run.smithi165.stdout:Install 150 Packages 2024-03-28T00:12:18.574 INFO:teuthology.orchestra.run.smithi165.stdout:Upgrade 24 Packages 2024-03-28T00:12:18.574 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-03-28T00:12:18.574 INFO:teuthology.orchestra.run.smithi165.stdout:Total download size: 225 M 2024-03-28T00:12:18.575 INFO:teuthology.orchestra.run.smithi165.stdout:Downloading Packages: 2024-03-28T00:12:18.892 INFO:teuthology.orchestra.run.smithi132.stdout:(1/174): ceph-19.0.0-2145.g3be9a5c9.el9.x86_64. 37 kB/s | 6.6 kB 00:00 2024-03-28T00:12:19.084 INFO:teuthology.orchestra.run.smithi132.stdout:(2/174): ceph-fuse-19.0.0-2145.g3be9a5c9.el9.x8 4.5 MB/s | 890 kB 00:00 2024-03-28T00:12:19.176 INFO:teuthology.orchestra.run.smithi132.stdout:(3/174): ceph-immutable-object-cache-19.0.0-214 1.5 MB/s | 145 kB 00:00 2024-03-28T00:12:19.268 INFO:teuthology.orchestra.run.smithi132.stdout:(4/174): ceph-base-19.0.0-2145.g3be9a5c9.el9.x8 9.7 MB/s | 5.4 MB 00:00 2024-03-28T00:12:19.385 INFO:teuthology.orchestra.run.smithi132.stdout:(5/174): ceph-mgr-19.0.0-2145.g3be9a5c9.el9.x86 13 MB/s | 1.5 MB 00:00 2024-03-28T00:12:19.435 INFO:teuthology.orchestra.run.smithi132.stdout:(6/174): ceph-mds-19.0.0-2145.g3be9a5c9.el9.x86 8.9 MB/s | 2.3 MB 00:00 2024-03-28T00:12:19.450 INFO:teuthology.orchestra.run.smithi165.stdout:(1/174): ceph-19.0.0-2145.g3be9a5c9.el9.x86_64. 22 kB/s | 6.6 kB 00:00 2024-03-28T00:12:19.694 INFO:teuthology.orchestra.run.smithi132.stdout:(7/174): ceph-mon-19.0.0-2145.g3be9a5c9.el9.x86 15 MB/s | 4.7 MB 00:00 2024-03-28T00:12:20.334 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction test succeeded. 2024-03-28T00:12:20.335 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction 2024-03-28T00:12:20.536 INFO:teuthology.orchestra.run.smithi132.stdout:(8/174): ceph-radosgw-19.0.0-2145.g3be9a5c9.el9 12 MB/s | 9.9 MB 00:00 2024-03-28T00:12:20.586 INFO:teuthology.orchestra.run.smithi132.stdout:(9/174): ceph-selinux-19.0.0-2145.g3be9a5c9.el9 506 kB/s | 25 kB 00:00 2024-03-28T00:12:20.850 INFO:teuthology.orchestra.run.smithi165.stdout:(2/174): ceph-fuse-19.0.0-2145.g3be9a5c9.el9.x8 636 kB/s | 890 kB 00:01 2024-03-28T00:12:21.226 INFO:teuthology.orchestra.run.smithi165.stdout:(3/174): ceph-immutable-object-cache-19.0.0-214 387 kB/s | 145 kB 00:00 2024-03-28T00:12:22.326 INFO:teuthology.orchestra.run.smithi165.stdout:(4/174): ceph-base-19.0.0-2145.g3be9a5c9.el9.x8 1.7 MB/s | 5.4 MB 00:03 2024-03-28T00:12:22.742 INFO:teuthology.orchestra.run.smithi165.stdout:(5/174): ceph-mgr-19.0.0-2145.g3be9a5c9.el9.x86 3.6 MB/s | 1.5 MB 00:00 2024-03-28T00:12:22.769 INFO:teuthology.orchestra.run.smithi132.stdout:(10/174): ceph-common-19.0.0-2145.g3be9a5c9.el9 5.1 MB/s | 20 MB 00:04 2024-03-28T00:12:22.861 INFO:teuthology.orchestra.run.smithi132.stdout:(11/174): cephfs-mirror-19.0.0-2145.g3be9a5c9.e 2.4 MB/s | 220 kB 00:00 2024-03-28T00:12:22.936 INFO:teuthology.orchestra.run.smithi132.stdout:(12/174): cephfs-shell-19.0.0-2145.g3be9a5c9.el 361 kB/s | 27 kB 00:00 2024-03-28T00:12:23.103 INFO:teuthology.orchestra.run.smithi132.stdout:(13/174): ceph-osd-19.0.0-2145.g3be9a5c9.el9.x8 4.6 MB/s | 17 MB 00:03 2024-03-28T00:12:23.120 INFO:teuthology.orchestra.run.smithi132.stdout:(14/174): libcephfs-devel-19.0.0-2145.g3be9a5c9 170 kB/s | 31 kB 00:00 2024-03-28T00:12:23.212 INFO:teuthology.orchestra.run.smithi132.stdout:(15/174): libcephsqlite-19.0.0-2145.g3be9a5c9.e 1.7 MB/s | 163 kB 00:00 2024-03-28T00:12:23.304 INFO:teuthology.orchestra.run.smithi132.stdout:(16/174): librados-devel-19.0.0-2145.g3be9a5c9. 1.4 MB/s | 127 kB 00:00 2024-03-28T00:12:23.329 INFO:teuthology.orchestra.run.smithi132.stdout:(17/174): libcephfs2-19.0.0-2145.g3be9a5c9.el9. 3.2 MB/s | 746 kB 00:00 2024-03-28T00:12:23.463 INFO:teuthology.orchestra.run.smithi132.stdout:(18/174): libradosstriper1-19.0.0-2145.g3be9a5c 3.1 MB/s | 501 kB 00:00 2024-03-28T00:12:23.546 INFO:teuthology.orchestra.run.smithi132.stdout:(19/174): python3-ceph-argparse-19.0.0-2145.g3b 542 kB/s | 45 kB 00:00 2024-03-28T00:12:23.584 INFO:teuthology.orchestra.run.smithi165.stdout:(6/174): ceph-mon-19.0.0-2145.g3be9a5c9.el9.x86 5.6 MB/s | 4.7 MB 00:00 2024-03-28T00:12:23.630 INFO:teuthology.orchestra.run.smithi132.stdout:(20/174): python3-ceph-common-19.0.0-2145.g3be9 1.5 MB/s | 131 kB 00:00 2024-03-28T00:12:23.730 INFO:teuthology.orchestra.run.smithi132.stdout:(21/174): python3-cephfs-19.0.0-2145.g3be9a5c9. 1.6 MB/s | 162 kB 00:00 2024-03-28T00:12:23.830 INFO:teuthology.orchestra.run.smithi132.stdout:(22/174): python3-rados-19.0.0-2145.g3be9a5c9.e 3.1 MB/s | 321 kB 00:00 2024-03-28T00:12:23.930 INFO:teuthology.orchestra.run.smithi132.stdout:(23/174): python3-rbd-19.0.0-2145.g3be9a5c9.el9 2.9 MB/s | 298 kB 00:00 2024-03-28T00:12:24.014 INFO:teuthology.orchestra.run.smithi132.stdout:(24/174): python3-rgw-19.0.0-2145.g3be9a5c9.el9 1.2 MB/s | 99 kB 00:00 2024-03-28T00:12:24.106 INFO:teuthology.orchestra.run.smithi132.stdout:(25/174): rbd-fuse-19.0.0-2145.g3be9a5c9.el9.x8 930 kB/s | 85 kB 00:00 2024-03-28T00:12:24.176 INFO:teuthology.orchestra.run.smithi165.stdout:(7/174): ceph-mds-19.0.0-2145.g3be9a5c9.el9.x86 802 kB/s | 2.3 MB 00:02 2024-03-28T00:12:24.976 INFO:teuthology.orchestra.run.smithi165.stdout:(8/174): ceph-common-19.0.0-2145.g3be9a5c9.el9. 3.5 MB/s | 20 MB 00:05 2024-03-28T00:12:25.051 INFO:teuthology.orchestra.run.smithi165.stdout:(9/174): ceph-selinux-19.0.0-2145.g3be9a5c9.el9 337 kB/s | 25 kB 00:00 2024-03-28T00:12:25.090 INFO:teuthology.orchestra.run.smithi132.stdout:(26/174): ceph-test-19.0.0-2145.g3be9a5c9.el9.x 11 MB/s | 48 MB 00:04 2024-03-28T00:12:25.157 INFO:teuthology.orchestra.run.smithi132.stdout:(27/174): librgw2-19.0.0-2145.g3be9a5c9.el9.x86 2.8 MB/s | 5.1 MB 00:01 2024-03-28T00:12:25.182 INFO:teuthology.orchestra.run.smithi132.stdout:(28/174): rbd-nbd-19.0.0-2145.g3be9a5c9.el9.x86 1.8 MB/s | 170 kB 00:00 2024-03-28T00:12:25.207 INFO:teuthology.orchestra.run.smithi132.stdout:(29/174): ceph-grafana-dashboards-19.0.0-2145.g 678 kB/s | 34 kB 00:00 2024-03-28T00:12:25.257 INFO:teuthology.orchestra.run.smithi132.stdout:(30/174): rbd-mirror-19.0.0-2145.g3be9a5c9.el9. 2.7 MB/s | 3.1 MB 00:01 2024-03-28T00:12:25.283 INFO:teuthology.orchestra.run.smithi132.stdout:(31/174): ceph-mgr-cephadm-19.0.0-2145.g3be9a5c 1.4 MB/s | 139 kB 00:00 2024-03-28T00:12:25.391 INFO:teuthology.orchestra.run.smithi132.stdout:(32/174): ceph-mgr-modules-core-19.0.0-2145.g3b 2.2 MB/s | 248 kB 00:00 2024-03-28T00:12:25.393 INFO:teuthology.orchestra.run.smithi165.stdout:(10/174): ceph-osd-19.0.0-2145.g3be9a5c9.el9.x8 9.3 MB/s | 17 MB 00:01 2024-03-28T00:12:25.425 INFO:teuthology.orchestra.run.smithi132.stdout:(33/174): ceph-mgr-dashboard-19.0.0-2145.g3be9a 8.3 MB/s | 1.8 MB 00:00 2024-03-28T00:12:25.450 INFO:teuthology.orchestra.run.smithi132.stdout:(34/174): ceph-mgr-rook-19.0.0-2145.g3be9a5c9.e 843 kB/s | 49 kB 00:00 2024-03-28T00:12:25.460 INFO:teuthology.orchestra.run.smithi165.stdout:(11/174): cephfs-mirror-19.0.0-2145.g3be9a5c9.e 3.2 MB/s | 220 kB 00:00 2024-03-28T00:12:25.475 INFO:teuthology.orchestra.run.smithi132.stdout:(35/174): ceph-prometheus-alerts-19.0.0-2145.g3 328 kB/s | 16 kB 00:00 2024-03-28T00:12:25.526 INFO:teuthology.orchestra.run.smithi132.stdout:(36/174): ceph-volume-19.0.0-2145.g3be9a5c9.el9 3.4 MB/s | 264 kB 00:00 2024-03-28T00:12:25.527 INFO:teuthology.orchestra.run.smithi165.stdout:(12/174): cephfs-shell-19.0.0-2145.g3be9a5c9.el 406 kB/s | 27 kB 00:00 2024-03-28T00:12:25.576 INFO:teuthology.orchestra.run.smithi132.stdout:(37/174): cephadm-19.0.0-2145.g3be9a5c9.el9.noa 7.3 MB/s | 748 kB 00:00 2024-03-28T00:12:25.594 INFO:teuthology.orchestra.run.smithi165.stdout:(13/174): libcephfs-devel-19.0.0-2145.g3be9a5c9 471 kB/s | 31 kB 00:00 2024-03-28T00:12:25.601 INFO:teuthology.orchestra.run.smithi132.stdout:(38/174): cephfs-top-19.0.0-2145.g3be9a5c9.el9. 242 kB/s | 18 kB 00:00 2024-03-28T00:12:25.711 INFO:teuthology.orchestra.run.smithi165.stdout:(14/174): libcephfs2-19.0.0-2145.g3be9a5c9.el9. 6.2 MB/s | 746 kB 00:00 2024-03-28T00:12:25.752 INFO:teuthology.orchestra.run.smithi132.stdout:(39/174): automake-1.16.2-8.el9.noarch.rpm 3.7 MB/s | 662 kB 00:00 2024-03-28T00:12:25.785 INFO:teuthology.orchestra.run.smithi132.stdout:(40/174): bison-3.7.4-5.el9.x86_64.rpm 4.9 MB/s | 920 kB 00:00 2024-03-28T00:12:25.786 INFO:teuthology.orchestra.run.smithi165.stdout:(15/174): libcephsqlite-19.0.0-2145.g3be9a5c9.e 2.1 MB/s | 163 kB 00:00 2024-03-28T00:12:25.811 INFO:teuthology.orchestra.run.smithi132.stdout:(41/174): boost-program-options-1.75.0-8.el9.x8 1.8 MB/s | 106 kB 00:00 2024-03-28T00:12:25.836 INFO:teuthology.orchestra.run.smithi132.stdout:(42/174): flex-2.6.4-9.el9.x86_64.rpm 6.0 MB/s | 307 kB 00:00 2024-03-28T00:12:25.858 INFO:teuthology.orchestra.run.smithi132.stdout:(43/174): flexiblas-3.0.4-8.el9.x86_64.rpm 676 kB/s | 32 kB 00:00 2024-03-28T00:12:25.861 INFO:teuthology.orchestra.run.smithi165.stdout:(16/174): librados-devel-19.0.0-2145.g3be9a5c9. 1.7 MB/s | 127 kB 00:00 2024-03-28T00:12:25.945 INFO:teuthology.orchestra.run.smithi165.stdout:(17/174): libradosstriper1-19.0.0-2145.g3be9a5c 5.9 MB/s | 501 kB 00:00 2024-03-28T00:12:25.950 INFO:teuthology.orchestra.run.smithi132.stdout:(44/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 26 MB/s | 3.0 MB 00:00 2024-03-28T00:12:25.975 INFO:teuthology.orchestra.run.smithi132.stdout:(45/174): flexiblas-openblas-openmp-3.0.4-8.el9 148 kB/s | 17 kB 00:00 2024-03-28T00:12:26.001 INFO:teuthology.orchestra.run.smithi132.stdout:(46/174): libaio-devel-0.3.111-13.el9.x86_64.rp 201 kB/s | 10 kB 00:00 2024-03-28T00:12:26.026 INFO:teuthology.orchestra.run.smithi132.stdout:(47/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 159 kB 00:00 2024-03-28T00:12:26.052 INFO:teuthology.orchestra.run.smithi132.stdout:(48/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 876 kB/s | 44 kB 00:00 2024-03-28T00:12:26.077 INFO:teuthology.orchestra.run.smithi132.stdout:(49/174): librdkafka-1.6.1-102.el9.x86_64.rpm 13 MB/s | 661 kB 00:00 2024-03-28T00:12:26.127 INFO:teuthology.orchestra.run.smithi132.stdout:(50/174): libtool-2.4.6-45.el9.x86_64.rpm 7.6 MB/s | 578 kB 00:00 2024-03-28T00:12:26.153 INFO:teuthology.orchestra.run.smithi132.stdout:(51/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 622 kB/s | 46 kB 00:00 2024-03-28T00:12:26.178 INFO:teuthology.orchestra.run.smithi132.stdout:(52/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.5 MB/s | 282 kB 00:00 2024-03-28T00:12:26.203 INFO:teuthology.orchestra.run.smithi132.stdout:(53/174): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 187 kB 00:00 2024-03-28T00:12:26.228 INFO:teuthology.orchestra.run.smithi132.stdout:(54/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 4.2 MB/s | 217 kB 00:00 2024-03-28T00:12:26.253 INFO:teuthology.orchestra.run.smithi132.stdout:(55/174): openblas-0.3.21-2.el9.x86_64.rpm 687 kB/s | 34 kB 00:00 2024-03-28T00:12:26.346 INFO:teuthology.orchestra.run.smithi132.stdout:(56/174): ceph-mgr-diskprediction-local-19.0.0- 6.8 MB/s | 7.4 MB 00:01 2024-03-28T00:12:26.378 INFO:teuthology.orchestra.run.smithi165.stdout:(18/174): librgw2-19.0.0-2145.g3be9a5c9.el9.x86 12 MB/s | 5.1 MB 00:00 2024-03-28T00:12:26.421 INFO:teuthology.orchestra.run.smithi132.stdout:(57/174): openblas-openmp-0.3.21-2.el9.x86_64.r 25 MB/s | 4.7 MB 00:00 2024-03-28T00:12:26.454 INFO:teuthology.orchestra.run.smithi165.stdout:(19/174): python3-ceph-argparse-19.0.0-2145.g3b 604 kB/s | 45 kB 00:00 2024-03-28T00:12:26.455 INFO:teuthology.orchestra.run.smithi132.stdout:(58/174): protobuf-3.14.0-13.el9.x86_64.rpm 5.1 MB/s | 1.0 MB 00:00 2024-03-28T00:12:26.480 INFO:teuthology.orchestra.run.smithi132.stdout:(59/174): python3-attrs-20.3.0-7.el9.noarch.rpm 590 kB/s | 78 kB 00:00 2024-03-28T00:12:26.505 INFO:teuthology.orchestra.run.smithi132.stdout:(60/174): python3-jsonpatch-1.21-16.el9.noarch. 492 kB/s | 24 kB 00:00 2024-03-28T00:12:26.520 INFO:teuthology.orchestra.run.smithi165.stdout:(20/174): python3-ceph-common-19.0.0-2145.g3be9 1.9 MB/s | 131 kB 00:00 2024-03-28T00:12:26.579 INFO:teuthology.orchestra.run.smithi165.stdout:(21/174): python3-cephfs-19.0.0-2145.g3be9a5c9. 2.7 MB/s | 162 kB 00:00 2024-03-28T00:12:26.654 INFO:teuthology.orchestra.run.smithi165.stdout:(22/174): python3-rados-19.0.0-2145.g3be9a5c9.e 4.2 MB/s | 321 kB 00:00 2024-03-28T00:12:26.714 INFO:teuthology.orchestra.run.smithi132.stdout:(61/174): python3-babel-2.9.1-2.el9.noarch.rpm 22 MB/s | 5.8 MB 00:00 2024-03-28T00:12:26.729 INFO:teuthology.orchestra.run.smithi165.stdout:(23/174): python3-rbd-19.0.0-2145.g3be9a5c9.el9 3.9 MB/s | 298 kB 00:00 2024-03-28T00:12:26.739 INFO:teuthology.orchestra.run.smithi132.stdout:(62/174): python3-jsonpointer-2.0-4.el9.noarch. 70 kB/s | 18 kB 00:00 2024-03-28T00:12:26.764 INFO:teuthology.orchestra.run.smithi132.stdout:(63/174): python3-mako-1.1.4-6.el9.noarch.rpm 560 kB/s | 144 kB 00:00 2024-03-28T00:12:26.789 INFO:teuthology.orchestra.run.smithi132.stdout:(64/174): python3-markupsafe-1.1.1-12.el9.x86_6 422 kB/s | 32 kB 00:00 2024-03-28T00:12:26.805 INFO:teuthology.orchestra.run.smithi165.stdout:(24/174): python3-rgw-19.0.0-2145.g3be9a5c9.el9 1.3 MB/s | 99 kB 00:00 2024-03-28T00:12:26.815 INFO:teuthology.orchestra.run.smithi132.stdout:(65/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.3 MB/s | 217 kB 00:00 2024-03-28T00:12:26.872 INFO:teuthology.orchestra.run.smithi165.stdout:(25/174): rbd-fuse-19.0.0-2145.g3be9a5c9.el9.x8 1.3 MB/s | 85 kB 00:00 2024-03-28T00:12:26.907 INFO:teuthology.orchestra.run.smithi132.stdout:(66/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 34 MB/s | 4.8 MB 00:00 2024-03-28T00:12:26.932 INFO:teuthology.orchestra.run.smithi132.stdout:(67/174): python3-oauthlib-3.1.1-5.el9.noarch.r 1.2 MB/s | 174 kB 00:00 2024-03-28T00:12:26.957 INFO:teuthology.orchestra.run.smithi132.stdout:(68/174): python3-packaging-20.9-5.el9.noarch.r 494 kB/s | 69 kB 00:00 2024-03-28T00:12:26.982 INFO:teuthology.orchestra.run.smithi132.stdout:(69/174): python3-prettytable-0.7.2-27.el9.noar 550 kB/s | 41 kB 00:00 2024-03-28T00:12:27.008 INFO:teuthology.orchestra.run.smithi132.stdout:(70/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.6 MB/s | 132 kB 00:00 2024-03-28T00:12:27.033 INFO:teuthology.orchestra.run.smithi132.stdout:(71/174): python3-protobuf-3.14.0-13.el9.noarch 3.1 MB/s | 238 kB 00:00 2024-03-28T00:12:27.058 INFO:teuthology.orchestra.run.smithi132.stdout:(72/174): python3-pyasn1-modules-0.4.8-6.el9.no 2.7 MB/s | 211 kB 00:00 2024-03-28T00:12:27.075 INFO:teuthology.orchestra.run.smithi132.stdout:(73/174): python3-pytz-2021.1-5.el9.noarch.rpm 714 kB/s | 47 kB 00:00 2024-03-28T00:12:27.092 INFO:teuthology.orchestra.run.smithi132.stdout:(74/174): python3-requests-oauthlib-1.3.0-12.el 1.3 MB/s | 43 kB 00:00 2024-03-28T00:12:27.147 INFO:teuthology.orchestra.run.smithi165.stdout:(26/174): rbd-mirror-19.0.0-2145.g3be9a5c9.el9. 11 MB/s | 3.1 MB 00:00 2024-03-28T00:12:27.184 INFO:teuthology.orchestra.run.smithi132.stdout:(75/174): python3-toml-0.10.2-6.el9.noarch.rpm 358 kB/s | 39 kB 00:00 2024-03-28T00:12:27.231 INFO:teuthology.orchestra.run.smithi165.stdout:(27/174): rbd-nbd-19.0.0-2145.g3be9a5c9.el9.x86 2.0 MB/s | 170 kB 00:00 2024-03-28T00:12:27.268 INFO:teuthology.orchestra.run.smithi132.stdout:(76/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 238 kB/s | 41 kB 00:00 2024-03-28T00:12:27.289 INFO:teuthology.orchestra.run.smithi165.stdout:(28/174): ceph-grafana-dashboards-19.0.0-2145.g 583 kB/s | 34 kB 00:00 2024-03-28T00:12:27.356 INFO:teuthology.orchestra.run.smithi165.stdout:(29/174): ceph-mgr-cephadm-19.0.0-2145.g3be9a5c 2.0 MB/s | 139 kB 00:00 2024-03-28T00:12:27.426 INFO:teuthology.orchestra.run.smithi132.stdout:(77/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 43 MB/s | 16 MB 00:00 2024-03-28T00:12:27.451 INFO:teuthology.orchestra.run.smithi132.stdout:(78/174): socat-1.7.4.1-5.el9.x86_64.rpm 1.1 MB/s | 300 kB 00:00 2024-03-28T00:12:27.468 INFO:teuthology.orchestra.run.smithi132.stdout:(79/174): uuid-1.6.2-55.el9.x86_64.rpm 284 kB/s | 56 kB 00:00 2024-03-28T00:12:27.485 INFO:teuthology.orchestra.run.smithi132.stdout:(80/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 1.3 MB/s | 44 kB 00:00 2024-03-28T00:12:27.502 INFO:teuthology.orchestra.run.smithi132.stdout:(81/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 63 kB 00:00 2024-03-28T00:12:27.515 INFO:teuthology.orchestra.run.smithi165.stdout:(30/174): ceph-mgr-dashboard-19.0.0-2145.g3be9a 11 MB/s | 1.8 MB 00:00 2024-03-28T00:12:27.527 INFO:teuthology.orchestra.run.smithi132.stdout:(82/174): fuse-2.9.9-15.el9.x86_64.rpm 1.3 MB/s | 79 kB 00:00 2024-03-28T00:12:27.552 INFO:teuthology.orchestra.run.smithi132.stdout:(83/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.5 MB/s | 241 kB 00:00 2024-03-28T00:12:27.578 INFO:teuthology.orchestra.run.smithi132.stdout:(84/174): python3-ply-3.11-14.el9.noarch.rpm 2.0 MB/s | 103 kB 00:00 2024-03-28T00:12:27.620 INFO:teuthology.orchestra.run.smithi132.stdout:(85/174): python3-cryptography-36.0.1-4.el9.x86 12 MB/s | 1.1 MB 00:00 2024-03-28T00:12:27.645 INFO:teuthology.orchestra.run.smithi132.stdout:(86/174): python3-pycparser-2.20-6.el9.noarch.r 1.3 MB/s | 124 kB 00:00 2024-03-28T00:12:27.670 INFO:teuthology.orchestra.run.smithi132.stdout:(87/174): python3-urllib3-1.26.5-5.el9.noarch.r 2.0 MB/s | 187 kB 00:00 2024-03-28T00:12:27.820 INFO:teuthology.orchestra.run.smithi132.stdout:(88/174): jq-1.6-16.el9.x86_64.rpm 1.0 MB/s | 187 kB 00:00 2024-03-28T00:12:27.845 INFO:teuthology.orchestra.run.smithi132.stdout:(89/174): libquadmath-11.4.1-3.el9.x86_64.rpm 1.1 MB/s | 195 kB 00:00 2024-03-28T00:12:27.846 INFO:teuthology.orchestra.run.smithi119.stdout: Preparing : 1/1 2024-03-28T00:12:27.888 INFO:teuthology.orchestra.run.smithi132.stdout:(90/174): libgfortran-11.4.1-3.el9.x86_64.rpm 3.2 MB/s | 803 kB 00:00 2024-03-28T00:12:27.913 INFO:teuthology.orchestra.run.smithi132.stdout:(91/174): elfutils-libelf-devel-0.190-2.el9.x86 319 kB/s | 21 kB 00:00 2024-03-28T00:12:27.938 INFO:teuthology.orchestra.run.smithi132.stdout:(92/174): python3-requests-2.25.1-8.el9.noarch. 1.0 MB/s | 125 kB 00:00 2024-03-28T00:12:27.944 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-03-28T00:12:27.963 INFO:teuthology.orchestra.run.smithi132.stdout:(93/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 336 kB/s | 25 kB 00:00 2024-03-28T00:12:27.967 INFO:teuthology.orchestra.run.smithi132.stdout:[MIRROR] postgresql-13.14-1.el9.x86_64.rpm: Status code: 404 for http://ftpmirror.your.org/pub/CentOS-Stream-All/9-stream/AppStream/x86_64/os/Packages/postgresql-13.14-1.el9.x86_64.rpm (IP: 163.237.219.7) 2024-03-28T00:12:27.992 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-03-28T00:12:27.992 INFO:teuthology.orchestra.run.smithi132.stdout:[MIRROR] postgresql-contrib-13.14-1.el9.x86_64.rpm: Status code: 404 for http://ftpmirror.your.org/pub/CentOS-Stream-All/9-stream/AppStream/x86_64/os/Packages/postgresql-contrib-13.14-1.el9.x86_64.rpm (IP: 163.237.219.7) 2024-03-28T00:12:28.023 INFO:teuthology.orchestra.run.smithi165.stdout:(31/174): ceph-mgr-diskprediction-local-19.0.0- 15 MB/s | 7.4 MB 00:00 2024-03-28T00:12:28.047 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-28T00:12:28.087 INFO:teuthology.orchestra.run.smithi132.stdout:[MIRROR] postgresql-13.14-1.el9.x86_64.rpm: Status code: 404 for https://ftpmirror.your.org/pub/CentOS-Stream-All/9-stream/AppStream/x86_64/os/Packages/postgresql-13.14-1.el9.x86_64.rpm (IP: 163.237.219.7) 2024-03-28T00:12:28.091 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-28T00:12:28.098 INFO:teuthology.orchestra.run.smithi165.stdout:(32/174): ceph-mgr-modules-core-19.0.0-2145.g3b 3.2 MB/s | 248 kB 00:00 2024-03-28T00:12:28.113 INFO:teuthology.orchestra.run.smithi132.stdout:[MIRROR] postgresql-contrib-13.14-1.el9.x86_64.rpm: Status code: 404 for https://ftpmirror.your.org/pub/CentOS-Stream-All/9-stream/AppStream/x86_64/os/Packages/postgresql-contrib-13.14-1.el9.x86_64.rpm (IP: 163.237.219.7) 2024-03-28T00:12:28.165 INFO:teuthology.orchestra.run.smithi165.stdout:(33/174): ceph-mgr-rook-19.0.0-2145.g3be9a5c9.e 741 kB/s | 49 kB 00:00 2024-03-28T00:12:28.177 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-03-28T00:12:28.224 INFO:teuthology.orchestra.run.smithi165.stdout:(34/174): ceph-prometheus-alerts-19.0.0-2145.g3 282 kB/s | 16 kB 00:00 2024-03-28T00:12:28.283 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-03-28T00:12:28.291 INFO:teuthology.orchestra.run.smithi165.stdout:(35/174): ceph-volume-19.0.0-2145.g3be9a5c9.el9 3.9 MB/s | 264 kB 00:00 2024-03-28T00:12:28.337 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-03-28T00:12:28.347 INFO:teuthology.orchestra.run.smithi132.stdout:(94/174): openssl-devel-3.0.7-27.el9.x86_64.rpm 9.5 MB/s | 4.1 MB 00:00 2024-03-28T00:12:28.376 INFO:teuthology.orchestra.run.smithi132.stdout:[MIRROR] postgresql-private-libs-13.14-1.el9.x86_64.rpm: Status code: 404 for https://ftpmirror.your.org/pub/CentOS-Stream-All/9-stream/AppStream/x86_64/os/Packages/postgresql-private-libs-13.14-1.el9.x86_64.rpm (IP: 163.237.219.7) 2024-03-28T00:12:28.379 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-03-28T00:12:28.383 INFO:teuthology.orchestra.run.smithi165.stdout:(36/174): cephadm-19.0.0-2145.g3be9a5c9.el9.noa 8.0 MB/s | 748 kB 00:00 2024-03-28T00:12:28.405 INFO:teuthology.orchestra.run.smithi132.stdout:[MIRROR] postgresql-private-libs-13.14-1.el9.x86_64.rpm: Status code: 404 for http://ftpmirror.your.org/pub/CentOS-Stream-All/9-stream/AppStream/x86_64/os/Packages/postgresql-private-libs-13.14-1.el9.x86_64.rpm (IP: 163.237.219.7) 2024-03-28T00:12:28.450 INFO:teuthology.orchestra.run.smithi165.stdout:(37/174): cephfs-top-19.0.0-2145.g3be9a5c9.el9. 274 kB/s | 18 kB 00:00 2024-03-28T00:12:28.584 INFO:teuthology.orchestra.run.smithi165.stdout:(38/174): automake-1.16.2-8.el9.noarch.rpm 4.9 MB/s | 662 kB 00:00 2024-03-28T00:12:28.642 INFO:teuthology.orchestra.run.smithi165.stdout:(39/174): bison-3.7.4-5.el9.x86_64.rpm 15 MB/s | 920 kB 00:00 2024-03-28T00:12:28.667 INFO:teuthology.orchestra.run.smithi165.stdout:(40/174): boost-program-options-1.75.0-8.el9.x8 4.2 MB/s | 106 kB 00:00 2024-03-28T00:12:28.705 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-03-28T00:12:28.709 INFO:teuthology.orchestra.run.smithi165.stdout:(41/174): flex-2.6.4-9.el9.x86_64.rpm 7.2 MB/s | 307 kB 00:00 2024-03-28T00:12:28.743 INFO:teuthology.orchestra.run.smithi165.stdout:(42/174): flexiblas-3.0.4-8.el9.x86_64.rpm 947 kB/s | 32 kB 00:00 2024-03-28T00:12:28.839 INFO:teuthology.orchestra.run.smithi132.stdout:(95/174): postgresql-private-libs-13.14-1.el9.x 271 kB/s | 133 kB 00:00 2024-03-28T00:12:28.843 INFO:teuthology.orchestra.run.smithi165.stdout:(43/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 30 MB/s | 3.0 MB 00:00 2024-03-28T00:12:28.864 INFO:teuthology.orchestra.run.smithi132.stdout:(96/174): postgresql-contrib-13.14-1.el9.x86_64 1.0 MB/s | 911 kB 00:00 2024-03-28T00:12:28.877 INFO:teuthology.orchestra.run.smithi165.stdout:(44/174): flexiblas-openblas-openmp-3.0.4-8.el9 515 kB/s | 17 kB 00:00 2024-03-28T00:12:28.898 INFO:teuthology.orchestra.run.smithi132.stdout:(97/174): postgresql-13.14-1.el9.x86_64.rpm 1.7 MB/s | 1.6 MB 00:00 2024-03-28T00:12:28.898 INFO:teuthology.orchestra.run.smithi132.stdout:[MIRROR] postgresql-server-13.14-1.el9.x86_64.rpm: Status code: 404 for http://ftpmirror.your.org/pub/CentOS-Stream-All/9-stream/AppStream/x86_64/os/Packages/postgresql-server-13.14-1.el9.x86_64.rpm (IP: 163.237.219.7) 2024-03-28T00:12:28.910 INFO:teuthology.orchestra.run.smithi165.stdout:(45/174): libaio-devel-0.3.111-13.el9.x86_64.rp 304 kB/s | 10 kB 00:00 2024-03-28T00:12:28.952 INFO:teuthology.orchestra.run.smithi165.stdout:(46/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.7 MB/s | 159 kB 00:00 2024-03-28T00:12:28.973 INFO:teuthology.orchestra.run.smithi132.stdout:(98/174): python3-devel-3.9.17-2.el9.x86_64.rpm 2.2 MB/s | 245 kB 00:00 2024-03-28T00:12:28.986 INFO:teuthology.orchestra.run.smithi165.stdout:(47/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.3 MB/s | 44 kB 00:00 2024-03-28T00:12:29.015 INFO:teuthology.orchestra.run.smithi132.stdout:(99/174): python3-jinja2-2.11.3-5.el9.noarch.rp 2.1 MB/s | 248 kB 00:00 2024-03-28T00:12:29.036 INFO:teuthology.orchestra.run.smithi165.stdout:(48/174): librdkafka-1.6.1-102.el9.x86_64.rpm 13 MB/s | 661 kB 00:00 2024-03-28T00:12:29.078 INFO:teuthology.orchestra.run.smithi165.stdout:(49/174): libtool-2.4.6-45.el9.x86_64.rpm 14 MB/s | 578 kB 00:00 2024-03-28T00:12:29.083 INFO:teuthology.orchestra.run.smithi132.stdout:(100/174): qatlib-24.02.0-1.el9.x86_64.rpm 2.0 MB/s | 221 kB 00:00 2024-03-28T00:12:29.111 INFO:teuthology.orchestra.run.smithi165.stdout:(50/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 1.4 MB/s | 46 kB 00:00 2024-03-28T00:12:29.117 INFO:teuthology.orchestra.run.smithi132.stdout:(101/174): qatlib-service-24.02.0-1.el9.x86_64. 353 kB/s | 36 kB 00:00 2024-03-28T00:12:29.146 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-03-28T00:12:29.153 INFO:teuthology.orchestra.run.smithi165.stdout:(51/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 6.6 MB/s | 282 kB 00:00 2024-03-28T00:12:29.184 INFO:teuthology.orchestra.run.smithi132.stdout:(102/174): qatzip-libs-1.2.0-1.el9.x86_64.rpm 472 kB/s | 47 kB 00:00 2024-03-28T00:12:29.185 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : librados2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 10/198 2024-03-28T00:12:29.195 INFO:teuthology.orchestra.run.smithi165.stdout:(52/174): lua-5.4.4-4.el9.x86_64.rpm 4.4 MB/s | 187 kB 00:00 2024-03-28T00:12:29.226 INFO:teuthology.orchestra.run.smithi132.stdout:(103/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 863 kB/s | 94 kB 00:00 2024-03-28T00:12:29.237 INFO:teuthology.orchestra.run.smithi165.stdout:(53/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 5.1 MB/s | 217 kB 00:00 2024-03-28T00:12:29.271 INFO:teuthology.orchestra.run.smithi165.stdout:(54/174): openblas-0.3.21-2.el9.x86_64.rpm 1.0 MB/s | 34 kB 00:00 2024-03-28T00:12:29.334 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: librados2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 10/198 2024-03-28T00:12:29.361 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libcephfs2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 11/198 2024-03-28T00:12:29.413 INFO:teuthology.orchestra.run.smithi165.stdout:(55/174): openblas-openmp-0.3.21-2.el9.x86_64.r 33 MB/s | 4.7 MB 00:00 2024-03-28T00:12:29.472 INFO:teuthology.orchestra.run.smithi165.stdout:(56/174): protobuf-3.14.0-13.el9.x86_64.rpm 17 MB/s | 1.0 MB 00:00 2024-03-28T00:12:29.481 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: libcephfs2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 11/198 2024-03-28T00:12:29.505 INFO:teuthology.orchestra.run.smithi165.stdout:(57/174): python3-attrs-20.3.0-7.el9.noarch.rpm 2.3 MB/s | 78 kB 00:00 2024-03-28T00:12:29.528 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-rados-2:19.0.0-2145.g3be9a5c9.el9.x86_64 12/198 2024-03-28T00:12:29.584 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 13/198 2024-03-28T00:12:29.622 INFO:teuthology.orchestra.run.smithi165.stdout:(58/174): ceph-radosgw-19.0.0-2145.g3be9a5c9.el 1.8 MB/s | 9.9 MB 00:05 2024-03-28T00:12:29.739 INFO:teuthology.orchestra.run.smithi165.stdout:(59/174): python3-babel-2.9.1-2.el9.noarch.rpm 25 MB/s | 5.8 MB 00:00 2024-03-28T00:12:29.762 INFO:teuthology.orchestra.run.smithi165.stdout:(60/174): python3-jsonpatch-1.21-16.el9.noarch. 178 kB/s | 24 kB 00:00 2024-03-28T00:12:29.784 INFO:teuthology.orchestra.run.smithi132.stdout:(104/174): lua-devel-5.4.4-4.el9.x86_64.rpm 37 kB/s | 22 kB 00:00 2024-03-28T00:12:29.787 INFO:teuthology.orchestra.run.smithi165.stdout:(61/174): python3-jsonpointer-2.0-4.el9.noarch. 376 kB/s | 18 kB 00:00 2024-03-28T00:12:29.805 INFO:teuthology.orchestra.run.smithi165.stdout:(62/174): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 144 kB 00:00 2024-03-28T00:12:29.830 INFO:teuthology.orchestra.run.smithi165.stdout:(63/174): python3-markupsafe-1.1.1-12.el9.x86_6 758 kB/s | 32 kB 00:00 2024-03-28T00:12:29.868 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-03-28T00:12:29.924 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : postgresql-13.14-1.el9.x86_64 15/198 2024-03-28T00:12:29.926 INFO:teuthology.orchestra.run.smithi132.stdout:(105/174): postgresql-server-13.14-1.el9.x86_64 5.3 MB/s | 5.8 MB 00:01 2024-03-28T00:12:29.950 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libcephsqlite-2:19.0.0-2145.g3be9a5c9.el9.x86_64 16/198 2024-03-28T00:12:29.952 INFO:teuthology.orchestra.run.smithi132.stdout:(106/174): abseil-cpp-20211102.0-3.el9.x86_64.r 3.2 MB/s | 548 kB 00:00 2024-03-28T00:12:29.972 INFO:teuthology.orchestra.run.smithi165.stdout:(64/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 29 MB/s | 4.8 MB 00:00 2024-03-28T00:12:29.985 INFO:teuthology.orchestra.run.smithi132.stdout:(107/174): grpc-data-1.46.7-10.el9.noarch.rpm 592 kB/s | 19 kB 00:00 2024-03-28T00:12:29.997 INFO:teuthology.orchestra.run.smithi165.stdout:(65/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.3 MB/s | 217 kB 00:00 2024-03-28T00:12:30.023 INFO:teuthology.orchestra.run.smithi165.stdout:(66/174): python3-oauthlib-3.1.1-5.el9.noarch.r 3.4 MB/s | 174 kB 00:00 2024-03-28T00:12:30.035 INFO:teuthology.orchestra.run.smithi132.stdout:(108/174): gperftools-libs-2.9.1-2.el9.x86_64.r 2.8 MB/s | 309 kB 00:00 2024-03-28T00:12:30.048 INFO:teuthology.orchestra.run.smithi165.stdout:(67/174): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 69 kB 00:00 2024-03-28T00:12:30.061 INFO:teuthology.orchestra.run.smithi132.stdout:(109/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-03-28T00:12:30.073 INFO:teuthology.orchestra.run.smithi165.stdout:(68/174): python3-prettytable-0.7.2-27.el9.noar 822 kB/s | 41 kB 00:00 2024-03-28T00:12:30.086 INFO:teuthology.orchestra.run.smithi132.stdout:(110/174): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-03-28T00:12:30.098 INFO:teuthology.orchestra.run.smithi165.stdout:(69/174): python3-protobuf-3.14.0-13.el9.noarch 4.6 MB/s | 238 kB 00:00 2024-03-28T00:12:30.107 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2145.g3be9a5c9.el9.x86_64 16/198 2024-03-28T00:12:30.124 INFO:teuthology.orchestra.run.smithi165.stdout:(70/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.6 MB/s | 132 kB 00:00 2024-03-28T00:12:30.143 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libradosstriper1-2:19.0.0-2145.g3be9a5c9.el9.x86 17/198 2024-03-28T00:12:30.149 INFO:teuthology.orchestra.run.smithi165.stdout:(71/174): python3-pyasn1-modules-0.4.8-6.el9.no 4.1 MB/s | 211 kB 00:00 2024-03-28T00:12:30.161 INFO:teuthology.orchestra.run.smithi132.stdout:(111/174): libarrow-9.0.0-10.el9.x86_64.rpm 25 MB/s | 4.4 MB 00:00 2024-03-28T00:12:30.166 INFO:teuthology.orchestra.run.smithi165.stdout:(72/174): python3-pytz-2021.1-5.el9.noarch.rpm 1.1 MB/s | 47 kB 00:00 2024-03-28T00:12:30.183 INFO:teuthology.orchestra.run.smithi165.stdout:(73/174): python3-requests-oauthlib-1.3.0-12.el 1.2 MB/s | 43 kB 00:00 2024-03-28T00:12:30.187 INFO:teuthology.orchestra.run.smithi132.stdout:(112/174): libunwind-1.6.2-1.el9.x86_64.rpm 672 kB/s | 67 kB 00:00 2024-03-28T00:12:30.212 INFO:teuthology.orchestra.run.smithi132.stdout:(113/174): luarocks-3.9.2-1.el9.noarch.rpm 2.9 MB/s | 151 kB 00:00 2024-03-28T00:12:30.232 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2145.g3be9a5c9.el9.x86 17/198 2024-03-28T00:12:30.254 INFO:teuthology.orchestra.run.smithi132.stdout:(114/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 12 MB/s | 838 kB 00:00 2024-03-28T00:12:30.279 INFO:teuthology.orchestra.run.smithi132.stdout:(115/174): python3-asyncssh-2.13.2-5.el9.noarch 8.0 MB/s | 548 kB 00:00 2024-03-28T00:12:30.284 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-03-28T00:12:30.304 INFO:teuthology.orchestra.run.smithi132.stdout:(116/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 868 kB/s | 43 kB 00:00 2024-03-28T00:12:30.319 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-03-28T00:12:30.329 INFO:teuthology.orchestra.run.smithi132.stdout:(117/174): python3-cachetools-4.2.4-1.el9.noarc 640 kB/s | 32 kB 00:00 2024-03-28T00:12:30.342 INFO:teuthology.orchestra.run.smithi165.stdout:(74/174): python3-toml-0.10.2-6.el9.noarch.rpm 245 kB/s | 39 kB 00:00 2024-03-28T00:12:30.355 INFO:teuthology.orchestra.run.smithi132.stdout:(118/174): python3-certifi-2021.10.8-2.el9.noar 300 kB/s | 15 kB 00:00 2024-03-28T00:12:30.381 INFO:teuthology.orchestra.run.smithi132.stdout:(119/174): python3-cheroot-8.6.0-4.el9.noarch.r 3.4 MB/s | 172 kB 00:00 2024-03-28T00:12:30.391 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-03-28T00:12:30.405 INFO:teuthology.orchestra.run.smithi132.stdout:(120/174): python3-cherrypy-18.6.1-2.el9.noarch 7.0 MB/s | 358 kB 00:00 2024-03-28T00:12:30.433 INFO:teuthology.orchestra.run.smithi132.stdout:(121/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.7 MB/s | 303 kB 00:00 2024-03-28T00:12:30.459 INFO:teuthology.orchestra.run.smithi132.stdout:(122/174): python3-colorama-0.4.4-12.el9.noarch 675 kB/s | 35 kB 00:00 2024-03-28T00:12:30.484 INFO:teuthology.orchestra.run.smithi132.stdout:(123/174): python3-google-auth-2.28.2-1.el9.noa 3.9 MB/s | 203 kB 00:00 2024-03-28T00:12:30.517 INFO:teuthology.orchestra.run.smithi132.stdout:(124/174): python3-grpcio-tools-1.46.7-10.el9.x 4.3 MB/s | 144 kB 00:00 2024-03-28T00:12:30.542 INFO:teuthology.orchestra.run.smithi165.stdout:(75/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 42 MB/s | 16 MB 00:00 2024-03-28T00:12:30.559 INFO:teuthology.orchestra.run.smithi165.stdout:(76/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 190 kB/s | 41 kB 00:00 2024-03-28T00:12:30.559 INFO:teuthology.orchestra.run.smithi132.stdout:(125/174): python3-grpcio-1.46.7-10.el9.x86_64. 20 MB/s | 2.0 MB 00:00 2024-03-28T00:12:30.584 INFO:teuthology.orchestra.run.smithi165.stdout:(77/174): socat-1.7.4.1-5.el9.x86_64.rpm 7.1 MB/s | 300 kB 00:00 2024-03-28T00:12:30.584 INFO:teuthology.orchestra.run.smithi132.stdout:(126/174): python3-jaraco-8.2.1-3.el9.noarch.rp 160 kB/s | 11 kB 00:00 2024-03-28T00:12:30.609 INFO:teuthology.orchestra.run.smithi165.stdout:(78/174): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 56 kB 00:00 2024-03-28T00:12:30.610 INFO:teuthology.orchestra.run.smithi132.stdout:(127/174): python3-jaraco-classes-3.2.1-5.el9.n 351 kB/s | 18 kB 00:00 2024-03-28T00:12:30.634 INFO:teuthology.orchestra.run.smithi165.stdout:(79/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 63 kB 00:00 2024-03-28T00:12:30.635 INFO:teuthology.orchestra.run.smithi132.stdout:(128/174): python3-jaraco-collections-3.0.0-8.e 461 kB/s | 23 kB 00:00 2024-03-28T00:12:30.652 INFO:teuthology.orchestra.run.smithi165.stdout:(80/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 1.0 MB/s | 44 kB 00:00 2024-03-28T00:12:30.660 INFO:teuthology.orchestra.run.smithi132.stdout:(129/174): python3-jaraco-functools-3.5.0-2.el9 389 kB/s | 19 kB 00:00 2024-03-28T00:12:30.677 INFO:teuthology.orchestra.run.smithi165.stdout:(81/174): fuse-2.9.9-15.el9.x86_64.rpm 1.8 MB/s | 79 kB 00:00 2024-03-28T00:12:30.685 INFO:teuthology.orchestra.run.smithi132.stdout:(130/174): python3-jaraco-text-3.2.0-6.el9.noar 392 kB/s | 20 kB 00:00 2024-03-28T00:12:30.702 INFO:teuthology.orchestra.run.smithi165.stdout:(82/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.7 MB/s | 241 kB 00:00 2024-03-28T00:12:30.719 INFO:teuthology.orchestra.run.smithi132.stdout:(131/174): python3-logutils-0.3.5-21.el9.noarch 1.4 MB/s | 46 kB 00:00 2024-03-28T00:12:30.736 INFO:teuthology.orchestra.run.smithi165.stdout:(83/174): python3-cryptography-36.0.1-4.el9.x86 20 MB/s | 1.1 MB 00:00 2024-03-28T00:12:30.739 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 20/198 2024-03-28T00:12:30.744 INFO:teuthology.orchestra.run.smithi132.stdout:(132/174): python3-kubernetes-26.1.0-2.el9.noar 12 MB/s | 1.0 MB 00:00 2024-03-28T00:12:30.761 INFO:teuthology.orchestra.run.smithi165.stdout:(84/174): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 103 kB 00:00 2024-03-28T00:12:30.769 INFO:teuthology.orchestra.run.smithi132.stdout:(133/174): python3-more-itertools-8.12.0-2.el9. 1.5 MB/s | 79 kB 00:00 2024-03-28T00:12:30.786 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 20/198 2024-03-28T00:12:30.786 INFO:teuthology.orchestra.run.smithi165.stdout:(85/174): python3-pycparser-2.20-6.el9.noarch.r 2.4 MB/s | 124 kB 00:00 2024-03-28T00:12:30.795 INFO:teuthology.orchestra.run.smithi132.stdout:(134/174): python3-natsort-7.1.1-5.el9.noarch.r 1.1 MB/s | 58 kB 00:00 2024-03-28T00:12:30.812 INFO:teuthology.orchestra.run.smithi165.stdout:(86/174): python3-urllib3-1.26.5-5.el9.noarch.r 3.7 MB/s | 187 kB 00:00 2024-03-28T00:12:30.820 INFO:teuthology.orchestra.run.smithi132.stdout:(135/174): python3-pecan-1.4.2-3.el9.noarch.rpm 5.3 MB/s | 272 kB 00:00 2024-03-28T00:12:30.828 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-03-28T00:12:30.845 INFO:teuthology.orchestra.run.smithi132.stdout:(136/174): python3-portend-3.1.0-2.el9.noarch.r 328 kB/s | 16 kB 00:00 2024-03-28T00:12:30.871 INFO:teuthology.orchestra.run.smithi132.stdout:(137/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.8 MB/s | 90 kB 00:00 2024-03-28T00:12:30.896 INFO:teuthology.orchestra.run.smithi132.stdout:(138/174): python3-pyperclip-1.8.2-2.el9.noarch 562 kB/s | 28 kB 00:00 2024-03-28T00:12:30.921 INFO:teuthology.orchestra.run.smithi132.stdout:(139/174): python3-repoze-lru-0.7-16.el9.noarch 613 kB/s | 31 kB 00:00 2024-03-28T00:12:30.947 INFO:teuthology.orchestra.run.smithi132.stdout:(140/174): python3-routes-2.5.1-5.el9.noarch.rp 3.7 MB/s | 188 kB 00:00 2024-03-28T00:12:30.972 INFO:teuthology.orchestra.run.smithi132.stdout:(141/174): python3-rsa-4.9-2.el9.noarch.rpm 1.2 MB/s | 59 kB 00:00 2024-03-28T00:12:30.989 INFO:teuthology.orchestra.run.smithi132.stdout:(142/174): python3-tempora-5.0.0-2.el9.noarch.r 855 kB/s | 36 kB 00:00 2024-03-28T00:12:31.006 INFO:teuthology.orchestra.run.smithi132.stdout:(143/174): python3-typing-extensions-4.4.0-2.el 1.5 MB/s | 51 kB 00:00 2024-03-28T00:12:31.031 INFO:teuthology.orchestra.run.smithi132.stdout:(144/174): python3-webob-1.8.7-6.el9.noarch.rpm 5.4 MB/s | 230 kB 00:00 2024-03-28T00:12:31.065 INFO:teuthology.orchestra.run.smithi132.stdout:(145/174): python3-websocket-client-1.2.3-2.el9 1.5 MB/s | 90 kB 00:00 2024-03-28T00:12:31.090 INFO:teuthology.orchestra.run.smithi132.stdout:(146/174): python3-werkzeug-2.0.3-3.el9.1.noarc 7.2 MB/s | 427 kB 00:00 2024-03-28T00:12:31.115 INFO:teuthology.orchestra.run.smithi132.stdout:(147/174): python3-zc-lockfile-2.0-10.el9.noarc 398 kB/s | 20 kB 00:00 2024-03-28T00:12:31.152 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-03-28T00:12:31.157 INFO:teuthology.orchestra.run.smithi132.stdout:(148/174): protobuf-compiler-3.14.0-13.el9.x86_ 447 kB/s | 863 kB 00:01 2024-03-28T00:12:31.182 INFO:teuthology.orchestra.run.smithi132.stdout:(149/174): re2-20211101-3.el9.x86_64.rpm 2.0 MB/s | 192 kB 00:00 2024-03-28T00:12:31.258 INFO:teuthology.orchestra.run.smithi132.stdout:(150/174): thrift-0.15.0-2.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-03-28T00:12:31.327 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-28T00:12:31.406 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-28T00:12:31.542 INFO:teuthology.orchestra.run.smithi132.stdout:(151/174): xfsdump-3.1.12-4.el9.x86_64.rpm 1.2 MB/s | 321 kB 00:00 2024-03-28T00:12:31.570 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-28T00:12:31.630 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-28T00:12:31.650 INFO:teuthology.orchestra.run.smithi132.stdout:(152/174): librados2-19.0.0-2145.g3be9a5c9.el9. 6.9 MB/s | 3.4 MB 00:00 2024-03-28T00:12:31.717 INFO:teuthology.orchestra.run.smithi132.stdout:(153/174): librbd1-19.0.0-2145.g3be9a5c9.el9.x8 6.2 MB/s | 3.1 MB 00:00 2024-03-28T00:12:31.793 INFO:teuthology.orchestra.run.smithi132.stdout:(154/174): NetworkManager-team-1.47.2-1.el9.x86 561 kB/s | 42 kB 00:00 2024-03-28T00:12:31.820 INFO:teuthology.orchestra.run.smithi165.stdout:(87/174): jq-1.6-16.el9.x86_64.rpm 181 kB/s | 187 kB 00:01 2024-03-28T00:12:31.901 INFO:teuthology.orchestra.run.smithi132.stdout:(155/174): NetworkManager-tui-1.47.2-1.el9.x86_ 2.2 MB/s | 248 kB 00:00 2024-03-28T00:12:31.943 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-28T00:12:31.968 INFO:teuthology.orchestra.run.smithi132.stdout:(156/174): acl-2.3.1-4.el9.x86_64.rpm 1.0 MB/s | 71 kB 00:00 2024-03-28T00:12:32.035 INFO:teuthology.orchestra.run.smithi132.stdout:(157/174): elfutils-debuginfod-client-0.190-2.e 548 kB/s | 36 kB 00:00 2024-03-28T00:12:32.110 INFO:teuthology.orchestra.run.smithi132.stdout:(158/174): elfutils-libelf-0.190-2.el9.x86_64.r 2.5 MB/s | 192 kB 00:00 2024-03-28T00:12:32.142 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-03-28T00:12:32.160 INFO:teuthology.orchestra.run.smithi132.stdout:(159/174): NetworkManager-libnm-1.47.2-1.el9.x8 3.6 MB/s | 1.8 MB 00:00 2024-03-28T00:12:32.186 INFO:teuthology.orchestra.run.smithi132.stdout:(160/174): elfutils-libs-0.190-2.el9.x86_64.rpm 3.3 MB/s | 256 kB 00:00 2024-03-28T00:12:32.227 INFO:teuthology.orchestra.run.smithi132.stdout:(161/174): libacl-2.3.1-4.el9.x86_64.rpm 346 kB/s | 23 kB 00:00 2024-03-28T00:12:32.253 INFO:teuthology.orchestra.run.smithi132.stdout:(162/174): libblkid-2.37.4-18.el9.x86_64.rpm 1.6 MB/s | 108 kB 00:00 2024-03-28T00:12:32.259 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-03-28T00:12:32.295 INFO:teuthology.orchestra.run.smithi132.stdout:(163/174): libfdisk-2.37.4-18.el9.x86_64.rpm 2.3 MB/s | 155 kB 00:00 2024-03-28T00:12:32.313 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-03-28T00:12:32.320 INFO:teuthology.orchestra.run.smithi132.stdout:(164/174): libmount-2.37.4-18.el9.x86_64.rpm 2.0 MB/s | 136 kB 00:00 2024-03-28T00:12:32.347 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-03-28T00:12:32.362 INFO:teuthology.orchestra.run.smithi132.stdout:(165/174): libsmartcols-2.37.4-18.el9.x86_64.rp 955 kB/s | 64 kB 00:00 2024-03-28T00:12:32.387 INFO:teuthology.orchestra.run.smithi132.stdout:(166/174): libuuid-2.37.4-18.el9.x86_64.rpm 430 kB/s | 29 kB 00:00 2024-03-28T00:12:32.407 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-03-28T00:12:32.523 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-03-28T00:12:32.585 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-03-28T00:12:32.596 INFO:teuthology.orchestra.run.smithi132.stdout:(167/174): NetworkManager-1.47.2-1.el9.x86_64.r 2.2 MB/s | 2.3 MB 00:01 2024-03-28T00:12:32.635 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-03-28T00:12:32.726 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-03-28T00:12:32.746 INFO:teuthology.orchestra.run.smithi132.stdout:(168/174): openssl-3.0.7-27.el9.x86_64.rpm 3.2 MB/s | 1.2 MB 00:00 2024-03-28T00:12:32.770 INFO:teuthology.orchestra.run.smithi165.stdout:(88/174): ceph-test-19.0.0-2145.g3be9a5c9.el9.x 6.2 MB/s | 48 MB 00:07 2024-03-28T00:12:32.774 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-03-28T00:12:32.788 INFO:teuthology.orchestra.run.smithi132.stdout:(169/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 5.4 MB/s | 2.2 MB 00:00 2024-03-28T00:12:32.804 INFO:teuthology.orchestra.run.smithi165.stdout:(89/174): libgfortran-11.4.1-3.el9.x86_64.rpm 403 kB/s | 803 kB 00:01 2024-03-28T00:12:32.836 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-03-28T00:12:32.879 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-03-28T00:12:32.888 INFO:teuthology.orchestra.run.smithi132.stdout:(170/174): util-linux-user-2.37.4-18.el9.x86_64 311 kB/s | 31 kB 00:00 2024-03-28T00:12:32.936 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-03-28T00:12:32.937 INFO:teuthology.orchestra.run.smithi165.stdout:(90/174): elfutils-libelf-devel-0.190-2.el9.x86 162 kB/s | 21 kB 00:00 2024-03-28T00:12:32.964 INFO:teuthology.orchestra.run.smithi132.stdout:(171/174): util-linux-core-2.37.4-18.el9.x86_64 2.1 MB/s | 465 kB 00:00 2024-03-28T00:12:33.038 INFO:teuthology.orchestra.run.smithi165.stdout:(91/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 255 kB/s | 25 kB 00:00 2024-03-28T00:12:33.054 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-03-28T00:12:33.071 INFO:teuthology.orchestra.run.smithi165.stdout:(92/174): libquadmath-11.4.1-3.el9.x86_64.rpm 156 kB/s | 195 kB 00:01 2024-03-28T00:12:33.089 INFO:teuthology.orchestra.run.smithi132.stdout:(172/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 5.4 MB/s | 1.1 MB 00:00 2024-03-28T00:12:33.090 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-ceph-common-2:19.0.0-2145.g3be9a5c9.el9. 39/198 2024-03-28T00:12:33.151 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-ceph-argparse-2:19.0.0-2145.g3be9a5c9.el 40/198 2024-03-28T00:12:33.188 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-cephfs-2:19.0.0-2145.g3be9a5c9.el9.x86_6 41/198 2024-03-28T00:12:33.201 INFO:teuthology.orchestra.run.smithi165.stdout:[MIRROR] postgresql-13.14-1.el9.x86_64.rpm: Status code: 404 for http://ftpmirror.your.org/pub/CentOS-Stream-All/9-stream/AppStream/x86_64/os/Packages/postgresql-13.14-1.el9.x86_64.rpm (IP: 163.237.219.7) 2024-03-28T00:12:33.289 INFO:teuthology.orchestra.run.smithi132.stdout:(173/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 229 kB/s | 74 kB 00:00 2024-03-28T00:12:33.318 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-03-28T00:12:33.373 INFO:teuthology.orchestra.run.smithi165.stdout:[MIRROR] postgresql-13.14-1.el9.x86_64.rpm: Status code: 404 for https://ftpmirror.your.org/pub/CentOS-Stream-All/9-stream/AppStream/x86_64/os/Packages/postgresql-13.14-1.el9.x86_64.rpm (IP: 163.237.219.7) 2024-03-28T00:12:33.385 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-03-28T00:12:33.504 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-03-28T00:12:33.548 INFO:teuthology.orchestra.run.smithi132.stdout:(174/174): util-linux-2.37.4-18.el9.x86_64.rpm 2.4 MB/s | 2.3 MB 00:00 2024-03-28T00:12:33.551 INFO:teuthology.orchestra.run.smithi132.stdout:-------------------------------------------------------------------------------- 2024-03-28T00:12:33.551 INFO:teuthology.orchestra.run.smithi132.stdout:Total 14 MB/s | 225 MB 00:15 2024-03-28T00:12:33.677 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-03-28T00:12:33.730 INFO:teuthology.orchestra.run.smithi165.stdout:(93/174): openssl-devel-3.0.7-27.el9.x86_64.rpm 6.0 MB/s | 4.1 MB 00:00 2024-03-28T00:12:33.747 INFO:teuthology.orchestra.run.smithi165.stdout:(94/174): python3-requests-2.25.1-8.el9.noarch. 128 kB/s | 125 kB 00:00 2024-03-28T00:12:33.751 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-03-28T00:12:33.772 INFO:teuthology.orchestra.run.smithi165.stdout:[MIRROR] postgresql-contrib-13.14-1.el9.x86_64.rpm: Status code: 404 for http://ftpmirror.your.org/pub/CentOS-Stream-All/9-stream/AppStream/x86_64/os/Packages/postgresql-contrib-13.14-1.el9.x86_64.rpm (IP: 163.237.219.7) 2024-03-28T00:12:33.787 INFO:teuthology.orchestra.run.smithi165.stdout:[MIRROR] postgresql-private-libs-13.14-1.el9.x86_64.rpm: Status code: 404 for http://ftpmirror.your.org/pub/CentOS-Stream-All/9-stream/AppStream/x86_64/os/Packages/postgresql-private-libs-13.14-1.el9.x86_64.rpm (IP: 163.237.219.7) 2024-03-28T00:12:33.821 INFO:teuthology.orchestra.run.smithi165.stdout:[MIRROR] postgresql-contrib-13.14-1.el9.x86_64.rpm: Status code: 404 for https://ftpmirror.your.org/pub/CentOS-Stream-All/9-stream/AppStream/x86_64/os/Packages/postgresql-contrib-13.14-1.el9.x86_64.rpm (IP: 163.237.219.7) 2024-03-28T00:12:33.833 INFO:teuthology.orchestra.run.smithi165.stdout:[MIRROR] postgresql-private-libs-13.14-1.el9.x86_64.rpm: Status code: 404 for https://ftpmirror.your.org/pub/CentOS-Stream-All/9-stream/AppStream/x86_64/os/Packages/postgresql-private-libs-13.14-1.el9.x86_64.rpm (IP: 163.237.219.7) 2024-03-28T00:12:34.121 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction check 2024-03-28T00:12:34.392 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction check succeeded. 2024-03-28T00:12:34.392 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction test 2024-03-28T00:12:34.432 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-03-28T00:12:34.532 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-03-28T00:12:34.635 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-03-28T00:12:34.684 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-28T00:12:34.711 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-28T00:12:34.763 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 51/198 2024-03-28T00:12:34.789 INFO:teuthology.orchestra.run.smithi165.stdout:(95/174): postgresql-private-libs-13.14-1.el9.x 128 kB/s | 133 kB 00:01 2024-03-28T00:12:34.830 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-03-28T00:12:34.909 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : librados-devel-2:19.0.0-2145.g3be9a5c9.el9.x86_6 53/198 2024-03-28T00:12:34.958 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-03-28T00:12:35.367 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-03-28T00:12:35.581 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-03-28T00:12:35.688 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-03-28T00:12:35.748 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-03-28T00:12:35.839 INFO:teuthology.orchestra.run.smithi165.stdout:(96/174): postgresql-contrib-13.14-1.el9.x86_64 432 kB/s | 911 kB 00:02 2024-03-28T00:12:35.956 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-03-28T00:12:36.009 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-03-28T00:12:36.077 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-03-28T00:12:36.135 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction test succeeded. 2024-03-28T00:12:36.135 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction 2024-03-28T00:12:36.168 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-03-28T00:12:36.217 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-03-28T00:12:36.256 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-03-28T00:12:36.324 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-03-28T00:12:36.373 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-03-28T00:12:36.405 INFO:teuthology.orchestra.run.smithi165.stdout:(97/174): postgresql-13.14-1.el9.x86_64.rpm 491 kB/s | 1.6 MB 00:03 2024-03-28T00:12:36.436 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-03-28T00:12:36.496 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-03-28T00:12:36.692 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-03-28T00:12:36.745 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-03-28T00:12:36.783 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-03-28T00:12:36.856 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-03-28T00:12:36.939 INFO:teuthology.orchestra.run.smithi165.stdout:(98/174): python3-devel-3.9.17-2.el9.x86_64.rpm 223 kB/s | 245 kB 00:01 2024-03-28T00:12:36.989 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-03-28T00:12:37.024 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 74/198 2024-03-28T00:12:37.069 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-03-28T00:12:37.098 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-03-28T00:12:37.131 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-03-28T00:12:37.278 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-03-28T00:12:37.469 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-03-28T00:12:37.472 INFO:teuthology.orchestra.run.smithi165.stdout:(99/174): python3-jinja2-2.11.3-5.el9.noarch.rp 232 kB/s | 248 kB 00:01 2024-03-28T00:12:37.615 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-03-28T00:12:37.666 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-03-28T00:12:37.729 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-03-28T00:12:37.805 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-03-28T00:12:37.964 INFO:teuthology.orchestra.run.smithi165.stdout:(100/174): qatlib-24.02.0-1.el9.x86_64.rpm 215 kB/s | 221 kB 00:01 2024-03-28T00:12:38.054 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-03-28T00:12:38.115 INFO:teuthology.orchestra.run.smithi165.stdout:(101/174): qatlib-service-24.02.0-1.el9.x86_64. 56 kB/s | 36 kB 00:00 2024-03-28T00:12:38.122 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-03-28T00:12:38.294 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-03-28T00:12:38.342 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 87/198 2024-03-28T00:12:38.391 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-03-28T00:12:38.438 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-03-28T00:12:38.481 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-03-28T00:12:38.528 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-03-28T00:12:38.594 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-03-28T00:12:38.651 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-03-28T00:12:38.657 INFO:teuthology.orchestra.run.smithi165.stdout:(102/174): qatzip-libs-1.2.0-1.el9.x86_64.rpm 68 kB/s | 47 kB 00:00 2024-03-28T00:12:38.692 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-03-28T00:12:38.818 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-03-28T00:12:38.846 INFO:teuthology.orchestra.run.smithi165.stdout:(103/174): lua-devel-5.4.4-4.el9.x86_64.rpm 128 kB/s | 22 kB 00:00 2024-03-28T00:12:38.863 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-03-28T00:12:38.890 INFO:teuthology.orchestra.run.smithi165.stdout:(104/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 121 kB/s | 94 kB 00:00 2024-03-28T00:12:38.898 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-03-28T00:12:38.959 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-03-28T00:12:39.010 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-03-28T00:12:39.082 INFO:teuthology.orchestra.run.smithi165.stdout:(105/174): protobuf-compiler-3.14.0-13.el9.x86_ 3.4 MB/s | 863 kB 00:00 2024-03-28T00:12:39.163 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-03-28T00:12:39.275 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-03-28T00:12:39.330 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-03-28T00:12:39.366 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-03-28T00:12:39.423 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-03-28T00:12:39.561 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-03-28T00:12:39.632 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-03-28T00:12:39.657 INFO:teuthology.orchestra.run.smithi165.stdout:(106/174): abseil-cpp-20211102.0-3.el9.x86_64.r 715 kB/s | 548 kB 00:00 2024-03-28T00:12:39.766 INFO:teuthology.orchestra.run.smithi165.stdout:(107/174): gperftools-libs-2.9.1-2.el9.x86_64.r 452 kB/s | 309 kB 00:00 2024-03-28T00:12:39.783 INFO:teuthology.orchestra.run.smithi165.stdout:(108/174): grpc-data-1.46.7-10.el9.noarch.rpm 155 kB/s | 19 kB 00:00 2024-03-28T00:12:39.900 INFO:teuthology.orchestra.run.smithi165.stdout:(109/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 222 kB/s | 26 kB 00:00 2024-03-28T00:12:40.017 INFO:teuthology.orchestra.run.smithi165.stdout:(110/174): liboath-2.6.7-2.el9.x86_64.rpm 418 kB/s | 49 kB 00:00 2024-03-28T00:12:40.144 INFO:teuthology.orchestra.run.smithi165.stdout:(111/174): libunwind-1.6.2-1.el9.x86_64.rpm 530 kB/s | 67 kB 00:00 2024-03-28T00:12:40.303 INFO:teuthology.orchestra.run.smithi165.stdout:(112/174): libarrow-9.0.0-10.el9.x86_64.rpm 8.2 MB/s | 4.4 MB 00:00 2024-03-28T00:12:40.328 INFO:teuthology.orchestra.run.smithi165.stdout:(113/174): luarocks-3.9.2-1.el9.noarch.rpm 823 kB/s | 151 kB 00:00 2024-03-28T00:12:40.428 INFO:teuthology.orchestra.run.smithi165.stdout:(114/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 6.5 MB/s | 838 kB 00:00 2024-03-28T00:12:40.495 INFO:teuthology.orchestra.run.smithi165.stdout:(115/174): python3-asyncssh-2.13.2-5.el9.noarch 3.2 MB/s | 548 kB 00:00 2024-03-28T00:12:40.537 INFO:teuthology.orchestra.run.smithi165.stdout:(116/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 401 kB/s | 43 kB 00:00 2024-03-28T00:12:40.602 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-03-28T00:12:40.604 INFO:teuthology.orchestra.run.smithi165.stdout:(117/174): python3-cachetools-4.2.4-1.el9.noarc 297 kB/s | 32 kB 00:00 2024-03-28T00:12:40.646 INFO:teuthology.orchestra.run.smithi165.stdout:(118/174): python3-certifi-2021.10.8-2.el9.noar 138 kB/s | 15 kB 00:00 2024-03-28T00:12:40.721 INFO:teuthology.orchestra.run.smithi165.stdout:(119/174): python3-cheroot-8.6.0-4.el9.noarch.r 1.4 MB/s | 172 kB 00:00 2024-03-28T00:12:40.755 INFO:teuthology.orchestra.run.smithi165.stdout:(120/174): python3-cherrypy-18.6.1-2.el9.noarch 3.2 MB/s | 358 kB 00:00 2024-03-28T00:12:40.840 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-03-28T00:12:40.864 INFO:teuthology.orchestra.run.smithi165.stdout:(121/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.1 MB/s | 303 kB 00:00 2024-03-28T00:12:40.885 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-03-28T00:12:40.889 INFO:teuthology.orchestra.run.smithi165.stdout:(122/174): python3-colorama-0.4.4-12.el9.noarch 264 kB/s | 35 kB 00:00 2024-03-28T00:12:40.963 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-03-28T00:12:40.997 INFO:teuthology.orchestra.run.smithi165.stdout:(123/174): python3-google-auth-2.28.2-1.el9.noa 1.5 MB/s | 203 kB 00:00 2024-03-28T00:12:41.048 INFO:teuthology.orchestra.run.smithi165.stdout:(124/174): python3-grpcio-1.46.7-10.el9.x86_64. 13 MB/s | 2.0 MB 00:00 2024-03-28T00:12:41.095 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-03-28T00:12:41.115 INFO:teuthology.orchestra.run.smithi165.stdout:(125/174): python3-grpcio-tools-1.46.7-10.el9.x 1.2 MB/s | 144 kB 00:00 2024-03-28T00:12:41.141 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-03-28T00:12:41.157 INFO:teuthology.orchestra.run.smithi165.stdout:(126/174): python3-jaraco-8.2.1-3.el9.noarch.rp 98 kB/s | 11 kB 00:00 2024-03-28T00:12:41.216 INFO:teuthology.orchestra.run.smithi165.stdout:(127/174): python3-jaraco-classes-3.2.1-5.el9.n 176 kB/s | 18 kB 00:00 2024-03-28T00:12:41.258 INFO:teuthology.orchestra.run.smithi165.stdout:(128/174): python3-jaraco-collections-3.0.0-8.e 231 kB/s | 23 kB 00:00 2024-03-28T00:12:41.316 INFO:teuthology.orchestra.run.smithi165.stdout:(129/174): python3-jaraco-functools-3.5.0-2.el9 194 kB/s | 19 kB 00:00 2024-03-28T00:12:41.358 INFO:teuthology.orchestra.run.smithi165.stdout:(130/174): python3-jaraco-text-3.2.0-6.el9.noar 196 kB/s | 20 kB 00:00 2024-03-28T00:12:41.391 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-03-28T00:12:41.429 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-03-28T00:12:41.458 INFO:teuthology.orchestra.run.smithi165.stdout:(131/174): python3-logutils-0.3.5-21.el9.noarch 463 kB/s | 46 kB 00:00 2024-03-28T00:12:41.525 INFO:teuthology.orchestra.run.smithi165.stdout:(132/174): python3-kubernetes-26.1.0-2.el9.noar 4.9 MB/s | 1.0 MB 00:00 2024-03-28T00:12:41.537 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-03-28T00:12:41.559 INFO:teuthology.orchestra.run.smithi165.stdout:(133/174): python3-more-itertools-8.12.0-2.el9. 786 kB/s | 79 kB 00:00 2024-03-28T00:12:41.635 INFO:teuthology.orchestra.run.smithi165.stdout:(134/174): python3-natsort-7.1.1-5.el9.noarch.r 529 kB/s | 58 kB 00:00 2024-03-28T00:12:41.668 INFO:teuthology.orchestra.run.smithi165.stdout:(135/174): python3-pecan-1.4.2-3.el9.noarch.rpm 2.4 MB/s | 272 kB 00:00 2024-03-28T00:12:41.743 INFO:teuthology.orchestra.run.smithi165.stdout:(136/174): python3-portend-3.1.0-2.el9.noarch.r 151 kB/s | 16 kB 00:00 2024-03-28T00:12:41.777 INFO:teuthology.orchestra.run.smithi165.stdout:(137/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 828 kB/s | 90 kB 00:00 2024-03-28T00:12:41.861 INFO:teuthology.orchestra.run.smithi165.stdout:(138/174): python3-pyperclip-1.8.2-2.el9.noarch 241 kB/s | 28 kB 00:00 2024-03-28T00:12:41.886 INFO:teuthology.orchestra.run.smithi165.stdout:(139/174): python3-repoze-lru-0.7-16.el9.noarch 283 kB/s | 31 kB 00:00 2024-03-28T00:12:41.970 INFO:teuthology.orchestra.run.smithi165.stdout:(140/174): python3-routes-2.5.1-5.el9.noarch.rp 1.7 MB/s | 188 kB 00:00 2024-03-28T00:12:41.995 INFO:teuthology.orchestra.run.smithi165.stdout:(141/174): python3-rsa-4.9-2.el9.noarch.rpm 544 kB/s | 59 kB 00:00 2024-03-28T00:12:42.078 INFO:teuthology.orchestra.run.smithi165.stdout:(142/174): python3-tempora-5.0.0-2.el9.noarch.r 329 kB/s | 36 kB 00:00 2024-03-28T00:12:42.105 INFO:teuthology.orchestra.run.smithi165.stdout:(143/174): python3-typing-extensions-4.4.0-2.el 470 kB/s | 51 kB 00:00 2024-03-28T00:12:42.188 INFO:teuthology.orchestra.run.smithi165.stdout:(144/174): python3-webob-1.8.7-6.el9.noarch.rpm 2.1 MB/s | 230 kB 00:00 2024-03-28T00:12:42.213 INFO:teuthology.orchestra.run.smithi165.stdout:(145/174): python3-websocket-client-1.2.3-2.el9 826 kB/s | 90 kB 00:00 2024-03-28T00:12:42.305 INFO:teuthology.orchestra.run.smithi165.stdout:(146/174): python3-werkzeug-2.0.3-3.el9.1.noarc 3.6 MB/s | 427 kB 00:00 2024-03-28T00:12:42.330 INFO:teuthology.orchestra.run.smithi165.stdout:(147/174): python3-zc-lockfile-2.0-10.el9.noarc 170 kB/s | 20 kB 00:00 2024-03-28T00:12:42.383 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-03-28T00:12:42.413 INFO:teuthology.orchestra.run.smithi165.stdout:(148/174): re2-20211101-3.el9.x86_64.rpm 1.7 MB/s | 192 kB 00:00 2024-03-28T00:12:42.472 INFO:teuthology.orchestra.run.smithi165.stdout:(149/174): thrift-0.15.0-2.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-03-28T00:12:42.535 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-03-28T00:12:42.898 INFO:teuthology.orchestra.run.smithi165.stdout:(150/174): librbd1-19.0.0-2145.g3be9a5c9.el9.x8 7.4 MB/s | 3.1 MB 00:00 2024-03-28T00:12:42.939 INFO:teuthology.orchestra.run.smithi165.stdout:(151/174): xfsdump-3.1.12-4.el9.x86_64.rpm 7.6 MB/s | 321 kB 00:00 2024-03-28T00:12:43.013 INFO:teuthology.orchestra.run.smithi132.stdout: Preparing : 1/1 2024-03-28T00:12:43.144 INFO:teuthology.orchestra.run.smithi132.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-03-28T00:12:43.178 INFO:teuthology.orchestra.run.smithi132.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-03-28T00:12:43.217 INFO:teuthology.orchestra.run.smithi132.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-28T00:12:43.226 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-28T00:12:43.266 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-03-28T00:12:43.322 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-03-28T00:12:43.353 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-03-28T00:12:43.379 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-03-28T00:12:43.421 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-03-28T00:12:43.531 INFO:teuthology.orchestra.run.smithi165.stdout:(152/174): librados2-19.0.0-2145.g3be9a5c9.el9. 3.0 MB/s | 3.4 MB 00:01 2024-03-28T00:12:43.765 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-03-28T00:12:43.782 INFO:teuthology.orchestra.run.smithi165.stdout:(153/174): postgresql-server-13.14-1.el9.x86_64 658 kB/s | 5.8 MB 00:08 2024-03-28T00:12:43.822 INFO:teuthology.orchestra.run.smithi132.stdout: Upgrading : librados2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 10/198 2024-03-28T00:12:43.933 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: librados2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 10/198 2024-03-28T00:12:43.965 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libcephfs2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 11/198 2024-03-28T00:12:44.039 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: libcephfs2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 11/198 2024-03-28T00:12:44.043 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-03-28T00:12:44.069 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-rados-2:19.0.0-2145.g3be9a5c9.el9.x86_64 12/198 2024-03-28T00:12:44.106 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 13/198 2024-03-28T00:12:44.124 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-03-28T00:12:44.162 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-03-28T00:12:44.193 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-03-28T00:12:44.232 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-03-28T00:12:44.283 INFO:teuthology.orchestra.run.smithi132.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-03-28T00:12:44.333 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : postgresql-13.14-1.el9.x86_64 15/198 2024-03-28T00:12:44.361 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libcephsqlite-2:19.0.0-2145.g3be9a5c9.el9.x86_64 16/198 2024-03-28T00:12:44.432 INFO:teuthology.orchestra.run.smithi165.stdout:(154/174): NetworkManager-team-1.47.2-1.el9.x86 67 kB/s | 42 kB 00:00 2024-03-28T00:12:44.448 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2145.g3be9a5c9.el9.x86_64 16/198 2024-03-28T00:12:44.471 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libradosstriper1-2:19.0.0-2145.g3be9a5c9.el9.x86 17/198 2024-03-28T00:12:44.534 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2145.g3be9a5c9.el9.x86 17/198 2024-03-28T00:12:44.583 INFO:teuthology.orchestra.run.smithi132.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-03-28T00:12:44.614 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-03-28T00:12:44.618 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-03-28T00:12:44.629 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-03-28T00:12:44.649 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : librbd1-2:19.0.0-2145.g3be9a5c9.el9.x86_64 124/198 2024-03-28T00:12:44.780 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 20/198 2024-03-28T00:12:44.787 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: librbd1-2:19.0.0-2145.g3be9a5c9.el9.x86_64 124/198 2024-03-28T00:12:44.811 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 20/198 2024-03-28T00:12:44.821 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-rbd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 125/198 2024-03-28T00:12:44.843 INFO:teuthology.orchestra.run.smithi132.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-03-28T00:12:44.878 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-03-28T00:12:44.940 INFO:teuthology.orchestra.run.smithi132.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-03-28T00:12:45.079 INFO:teuthology.orchestra.run.smithi132.stdout: Upgrading : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-28T00:12:45.139 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-28T00:12:45.206 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-28T00:12:45.249 INFO:teuthology.orchestra.run.smithi132.stdout: Upgrading : NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-28T00:12:45.357 INFO:teuthology.orchestra.run.smithi165.stdout:(155/174): NetworkManager-tui-1.47.2-1.el9.x86_ 268 kB/s | 248 kB 00:00 2024-03-28T00:12:45.400 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-03-28T00:12:45.434 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : librgw2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 128/198 2024-03-28T00:12:45.462 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-28T00:12:45.526 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: librgw2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 128/198 2024-03-28T00:12:45.553 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-rgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 129/198 2024-03-28T00:12:45.570 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-03-28T00:12:45.656 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-03-28T00:12:45.687 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-03-28T00:12:45.714 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-03-28T00:12:45.749 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-03-28T00:12:45.831 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-03-28T00:12:45.885 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-03-28T00:12:45.924 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-03-28T00:12:45.987 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-03-28T00:12:46.023 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-03-28T00:12:46.049 INFO:teuthology.orchestra.run.smithi165.stdout:(156/174): acl-2.3.1-4.el9.x86_64.rpm 103 kB/s | 71 kB 00:00 2024-03-28T00:12:46.066 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-03-28T00:12:46.099 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-03-28T00:12:46.127 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-03-28T00:12:46.197 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-03-28T00:12:46.224 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-ceph-common-2:19.0.0-2145.g3be9a5c9.el9. 39/198 2024-03-28T00:12:46.259 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-ceph-argparse-2:19.0.0-2145.g3be9a5c9.el 40/198 2024-03-28T00:12:46.287 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-cephfs-2:19.0.0-2145.g3be9a5c9.el9.x86_6 41/198 2024-03-28T00:12:46.355 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-03-28T00:12:46.402 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-03-28T00:12:46.475 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-03-28T00:12:46.557 INFO:teuthology.orchestra.run.smithi165.stdout:(157/174): NetworkManager-libnm-1.47.2-1.el9.x8 620 kB/s | 1.8 MB 00:03 2024-03-28T00:12:46.609 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-03-28T00:12:46.616 INFO:teuthology.orchestra.run.smithi165.stdout:(158/174): NetworkManager-1.47.2-1.el9.x86_64.r 642 kB/s | 2.3 MB 00:03 2024-03-28T00:12:46.653 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-03-28T00:12:46.683 INFO:teuthology.orchestra.run.smithi165.stdout:(159/174): elfutils-debuginfod-client-0.190-2.e 57 kB/s | 36 kB 00:00 2024-03-28T00:12:47.169 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-03-28T00:12:47.249 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-03-28T00:12:47.292 INFO:teuthology.orchestra.run.smithi165.stdout:(160/174): libacl-2.3.1-4.el9.x86_64.rpm 38 kB/s | 23 kB 00:00 2024-03-28T00:12:47.328 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-03-28T00:12:47.362 INFO:teuthology.orchestra.run.smithi132.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-28T00:12:47.372 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-28T00:12:47.392 INFO:teuthology.orchestra.run.smithi165.stdout:(161/174): elfutils-libelf-0.190-2.el9.x86_64.r 230 kB/s | 192 kB 00:00 2024-03-28T00:12:47.401 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 51/198 2024-03-28T00:12:47.442 INFO:teuthology.orchestra.run.smithi132.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-03-28T00:12:47.480 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : librados-devel-2:19.0.0-2145.g3be9a5c9.el9.x86_6 53/198 2024-03-28T00:12:47.509 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-03-28T00:12:47.517 INFO:teuthology.orchestra.run.smithi165.stdout:(162/174): elfutils-libs-0.190-2.el9.x86_64.rpm 284 kB/s | 256 kB 00:00 2024-03-28T00:12:47.726 INFO:teuthology.orchestra.run.smithi132.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-03-28T00:12:47.904 INFO:teuthology.orchestra.run.smithi132.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-03-28T00:12:47.987 INFO:teuthology.orchestra.run.smithi132.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-03-28T00:12:48.031 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-03-28T00:12:48.091 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-common-2:19.0.0-2145.g3be9a5c9.el9.x86_64 130/198 2024-03-28T00:12:48.092 INFO:teuthology.orchestra.run.smithi165.stdout:(163/174): libblkid-2.37.4-18.el9.x86_64.rpm 135 kB/s | 108 kB 00:00 2024-03-28T00:12:48.133 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-common-2:19.0.0-2145.g3be9a5c9.el9.x86_64 130/198 2024-03-28T00:12:48.169 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-03-28T00:12:48.217 INFO:teuthology.orchestra.run.smithi132.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-03-28T00:12:48.242 INFO:teuthology.orchestra.run.smithi165.stdout:(164/174): libfdisk-2.37.4-18.el9.x86_64.rpm 182 kB/s | 155 kB 00:00 2024-03-28T00:12:48.268 INFO:teuthology.orchestra.run.smithi165.stdout:(165/174): libmount-2.37.4-18.el9.x86_64.rpm 181 kB/s | 136 kB 00:00 2024-03-28T00:12:48.271 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-03-28T00:12:48.336 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-03-28T00:12:48.368 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-03-28T00:12:48.408 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-03-28T00:12:48.534 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-03-28T00:12:48.683 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-03-28T00:12:48.726 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-03-28T00:12:48.751 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-common-2:19.0.0-2145.g3be9a5c9.el9.x86_64 130/198 2024-03-28T00:12:48.769 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-03-28T00:12:48.793 INFO:teuthology.orchestra.run.smithi165.stdout:(166/174): libsmartcols-2.37.4-18.el9.x86_64.rp 91 kB/s | 64 kB 00:00 2024-03-28T00:12:48.818 INFO:teuthology.orchestra.run.smithi165.stdout:(167/174): libuuid-2.37.4-18.el9.x86_64.rpm 50 kB/s | 29 kB 00:00 2024-03-28T00:12:48.870 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-base-2:19.0.0-2145.g3be9a5c9.el9.x86_64 131/198 2024-03-28T00:12:48.901 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-03-28T00:12:48.948 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-03-28T00:12:48.978 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-03-28T00:12:49.019 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-03-28T00:12:49.034 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-base-2:19.0.0-2145.g3be9a5c9.el9.x86_64 131/198 2024-03-28T00:12:49.034 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-28T00:12:49.034 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-28T00:12:49.034 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-28T00:12:49.079 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-selinux-2:19.0.0-2145.g3be9a5c9.el9.x86_64 132/198 2024-03-28T00:12:49.113 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-03-28T00:12:49.153 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 74/198 2024-03-28T00:12:49.193 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-03-28T00:12:49.219 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-03-28T00:12:49.243 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-03-28T00:12:49.315 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-03-28T00:12:49.444 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-03-28T00:12:49.550 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-03-28T00:12:49.594 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-03-28T00:12:49.642 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-03-28T00:12:49.697 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-03-28T00:12:49.879 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-03-28T00:12:49.931 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-03-28T00:12:50.058 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-03-28T00:12:50.096 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 87/198 2024-03-28T00:12:50.128 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-03-28T00:12:50.160 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-03-28T00:12:50.187 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-03-28T00:12:50.214 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-03-28T00:12:50.252 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-03-28T00:12:50.284 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-03-28T00:12:50.314 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-03-28T00:12:50.390 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-03-28T00:12:50.418 INFO:teuthology.orchestra.run.smithi165.stdout:(168/174): openssl-3.0.7-27.el9.x86_64.rpm 576 kB/s | 1.2 MB 00:02 2024-03-28T00:12:50.425 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-03-28T00:12:50.458 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-03-28T00:12:50.499 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-03-28T00:12:50.534 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-03-28T00:12:50.622 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-03-28T00:12:50.704 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-03-28T00:12:50.748 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-03-28T00:12:50.780 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-03-28T00:12:50.827 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-03-28T00:12:50.923 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-03-28T00:12:50.973 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-03-28T00:12:51.669 INFO:teuthology.orchestra.run.smithi165.stdout:(169/174): util-linux-core-2.37.4-18.el9.x86_64 372 kB/s | 465 kB 00:01 2024-03-28T00:12:51.698 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-03-28T00:12:51.871 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-03-28T00:12:51.904 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-03-28T00:12:51.953 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-03-28T00:12:52.039 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-03-28T00:12:52.089 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-03-28T00:12:52.268 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-03-28T00:12:52.277 INFO:teuthology.orchestra.run.smithi165.stdout:(170/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 634 kB/s | 2.2 MB 00:03 2024-03-28T00:12:52.295 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-03-28T00:12:52.302 INFO:teuthology.orchestra.run.smithi165.stdout:(171/174): util-linux-user-2.37.4-18.el9.x86_64 49 kB/s | 31 kB 00:00 2024-03-28T00:12:52.344 INFO:teuthology.orchestra.run.smithi165.stdout:(172/174): util-linux-2.37.4-18.el9.x86_64.rpm 663 kB/s | 2.3 MB 00:03 2024-03-28T00:12:52.372 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-03-28T00:12:53.013 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-03-28T00:12:53.019 INFO:teuthology.orchestra.run.smithi165.stdout:(173/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 104 kB/s | 74 kB 00:00 2024-03-28T00:12:53.124 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-03-28T00:12:54.297 INFO:teuthology.orchestra.run.smithi165.stdout:(174/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 545 kB/s | 1.1 MB 00:02 2024-03-28T00:12:54.303 INFO:teuthology.orchestra.run.smithi165.stdout:-------------------------------------------------------------------------------- 2024-03-28T00:12:54.303 INFO:teuthology.orchestra.run.smithi165.stdout:Total 6.3 MB/s | 225 MB 00:35 2024-03-28T00:12:54.341 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-03-28T00:12:54.421 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-03-28T00:12:54.452 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-03-28T00:12:54.474 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-03-28T00:12:54.494 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-03-28T00:12:54.805 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-03-28T00:12:54.832 INFO:teuthology.orchestra.run.smithi132.stdout: Upgrading : librbd1-2:19.0.0-2145.g3be9a5c9.el9.x86_64 124/198 2024-03-28T00:12:54.861 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction check 2024-03-28T00:12:54.914 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: librbd1-2:19.0.0-2145.g3be9a5c9.el9.x86_64 124/198 2024-03-28T00:12:54.945 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-rbd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 125/198 2024-03-28T00:12:54.982 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-03-28T00:12:55.137 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction check succeeded. 2024-03-28T00:12:55.137 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction test 2024-03-28T00:12:55.441 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-03-28T00:12:55.477 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : librgw2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 128/198 2024-03-28T00:12:55.531 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: librgw2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 128/198 2024-03-28T00:12:55.554 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-rgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 129/198 2024-03-28T00:12:56.894 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction test succeeded. 2024-03-28T00:12:56.895 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction 2024-03-28T00:12:57.696 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-common-2:19.0.0-2145.g3be9a5c9.el9.x86_64 130/198 2024-03-28T00:12:57.726 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-common-2:19.0.0-2145.g3be9a5c9.el9.x86_64 130/198 2024-03-28T00:12:58.261 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-common-2:19.0.0-2145.g3be9a5c9.el9.x86_64 130/198 2024-03-28T00:12:58.296 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-base-2:19.0.0-2145.g3be9a5c9.el9.x86_64 131/198 2024-03-28T00:12:58.360 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-base-2:19.0.0-2145.g3be9a5c9.el9.x86_64 131/198 2024-03-28T00:12:58.360 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-28T00:12:58.361 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-28T00:12:58.361 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-28T00:12:58.389 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-selinux-2:19.0.0-2145.g3be9a5c9.el9.x86_64 132/198 2024-03-28T00:13:02.114 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2145.g3be9a5c9.el9.x86_64 132/198 2024-03-28T00:13:02.114 INFO:teuthology.orchestra.run.smithi119.stdout:skipping the directory /sys 2024-03-28T00:13:02.114 INFO:teuthology.orchestra.run.smithi119.stdout:skipping the directory /proc 2024-03-28T00:13:02.114 INFO:teuthology.orchestra.run.smithi119.stdout:skipping the directory /mnt 2024-03-28T00:13:02.114 INFO:teuthology.orchestra.run.smithi119.stdout:skipping the directory /var/tmp 2024-03-28T00:13:02.114 INFO:teuthology.orchestra.run.smithi119.stdout:skipping the directory /home 2024-03-28T00:13:02.114 INFO:teuthology.orchestra.run.smithi119.stdout:skipping the directory /root 2024-03-28T00:13:02.114 INFO:teuthology.orchestra.run.smithi119.stdout:skipping the directory /tmp 2024-03-28T00:13:02.114 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-28T00:13:02.374 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-volume-2:19.0.0-2145.g3be9a5c9.el9.noarch 133/198 2024-03-28T00:13:02.422 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-volume-2:19.0.0-2145.g3be9a5c9.el9.noarch 133/198 2024-03-28T00:13:02.422 INFO:teuthology.orchestra.run.smithi119.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-28T00:13:02.422 INFO:teuthology.orchestra.run.smithi119.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-28T00:13:02.422 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-28T00:13:03.744 INFO:teuthology.orchestra.run.smithi165.stdout: Preparing : 1/1 2024-03-28T00:13:03.873 INFO:teuthology.orchestra.run.smithi165.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-03-28T00:13:03.898 INFO:teuthology.orchestra.run.smithi165.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-03-28T00:13:03.929 INFO:teuthology.orchestra.run.smithi165.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-28T00:13:03.938 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-28T00:13:03.978 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-03-28T00:13:04.032 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-03-28T00:13:04.066 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-03-28T00:13:04.072 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-osd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 134/198 2024-03-28T00:13:04.092 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-03-28T00:13:04.134 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-03-28T00:13:04.162 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-osd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 134/198 2024-03-28T00:13:04.162 INFO:teuthology.orchestra.run.smithi119.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-28T00:13:04.163 INFO:teuthology.orchestra.run.smithi119.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-28T00:13:04.163 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-28T00:13:04.163 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-28T00:13:04.163 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-28T00:13:04.405 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-mds-2:19.0.0-2145.g3be9a5c9.el9.x86_64 135/198 2024-03-28T00:13:04.455 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-03-28T00:13:04.475 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-mds-2:19.0.0-2145.g3be9a5c9.el9.x86_64 135/198 2024-03-28T00:13:04.475 INFO:teuthology.orchestra.run.smithi119.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-28T00:13:04.475 INFO:teuthology.orchestra.run.smithi119.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-28T00:13:04.475 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-28T00:13:04.475 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-28T00:13:04.475 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-28T00:13:04.518 INFO:teuthology.orchestra.run.smithi165.stdout: Upgrading : librados2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 10/198 2024-03-28T00:13:04.639 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: librados2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 10/198 2024-03-28T00:13:04.678 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libcephfs2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 11/198 2024-03-28T00:13:04.752 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: libcephfs2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 11/198 2024-03-28T00:13:04.782 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-rados-2:19.0.0-2145.g3be9a5c9.el9.x86_64 12/198 2024-03-28T00:13:04.819 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 13/198 2024-03-28T00:13:04.964 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-mon-2:19.0.0-2145.g3be9a5c9.el9.x86_64 136/198 2024-03-28T00:13:04.998 INFO:teuthology.orchestra.run.smithi165.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-03-28T00:13:05.036 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-mon-2:19.0.0-2145.g3be9a5c9.el9.x86_64 136/198 2024-03-28T00:13:05.036 INFO:teuthology.orchestra.run.smithi119.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-28T00:13:05.036 INFO:teuthology.orchestra.run.smithi119.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-28T00:13:05.036 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-28T00:13:05.036 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-28T00:13:05.036 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-28T00:13:05.055 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : postgresql-13.14-1.el9.x86_64 15/198 2024-03-28T00:13:05.083 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libcephsqlite-2:19.0.0-2145.g3be9a5c9.el9.x86_64 16/198 2024-03-28T00:13:05.166 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-03-28T00:13:05.186 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2145.g3be9a5c9.el9.x86_64 16/198 2024-03-28T00:13:05.209 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libradosstriper1-2:19.0.0-2145.g3be9a5c9.el9.x86 17/198 2024-03-28T00:13:05.273 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2145.g3be9a5c9.el9.x86 17/198 2024-03-28T00:13:05.313 INFO:teuthology.orchestra.run.smithi165.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-03-28T00:13:05.336 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-03-28T00:13:05.351 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-03-28T00:13:05.554 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 20/198 2024-03-28T00:13:05.584 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 20/198 2024-03-28T00:13:05.616 INFO:teuthology.orchestra.run.smithi165.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-03-28T00:13:05.722 INFO:teuthology.orchestra.run.smithi165.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-03-28T00:13:05.778 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: cephadm-2:19.0.0-2145.g3be9a5c9.el9.noarch 138/198 2024-03-28T00:13:05.816 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : cephadm-2:19.0.0-2145.g3be9a5c9.el9.noarch 138/198 2024-03-28T00:13:05.827 INFO:teuthology.orchestra.run.smithi165.stdout: Upgrading : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-28T00:13:05.868 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2145.g3be9a5c9.e 139/198 2024-03-28T00:13:05.896 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-28T00:13:05.957 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-28T00:13:05.968 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2145.g3be9a5c9. 140/198 2024-03-28T00:13:05.995 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2145.g3be9a5c9.el9.noa 141/198 2024-03-28T00:13:06.022 INFO:teuthology.orchestra.run.smithi165.stdout: Upgrading : NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-28T00:13:06.230 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-28T00:13:06.329 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2145.g3be9a5c9.el9.noa 141/198 2024-03-28T00:13:06.336 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-03-28T00:13:06.372 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2145.g3be9a5c9.el9.n 142/198 2024-03-28T00:13:06.429 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-03-28T00:13:06.469 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-03-28T00:13:06.495 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-03-28T00:13:06.531 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-03-28T00:13:06.604 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-03-28T00:13:06.662 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-03-28T00:13:06.697 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-03-28T00:13:06.761 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-03-28T00:13:06.805 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-03-28T00:13:06.848 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-03-28T00:13:06.881 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-03-28T00:13:06.909 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-03-28T00:13:06.979 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-03-28T00:13:07.014 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-ceph-common-2:19.0.0-2145.g3be9a5c9.el9. 39/198 2024-03-28T00:13:07.057 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-ceph-argparse-2:19.0.0-2145.g3be9a5c9.el 40/198 2024-03-28T00:13:07.086 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-cephfs-2:19.0.0-2145.g3be9a5c9.el9.x86_6 41/198 2024-03-28T00:13:07.154 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-03-28T00:13:07.201 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-03-28T00:13:07.273 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-03-28T00:13:07.408 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-03-28T00:13:07.461 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-03-28T00:13:07.488 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2145.g3be9a5c9.el9.n 142/198 2024-03-28T00:13:07.522 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2145.g3be 143/198 2024-03-28T00:13:07.662 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2145.g3be 143/198 2024-03-28T00:13:07.859 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2145.g3be9a5c9.el 144/198 2024-03-28T00:13:07.890 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-mgr-2:19.0.0-2145.g3be9a5c9.el9.x86_64 145/198 2024-03-28T00:13:07.964 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2145.g3be9a5c9.el9.x86_64 145/198 2024-03-28T00:13:07.964 INFO:teuthology.orchestra.run.smithi119.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-28T00:13:07.964 INFO:teuthology.orchestra.run.smithi119.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-28T00:13:07.964 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-28T00:13:07.965 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-28T00:13:07.965 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-28T00:13:07.979 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-03-28T00:13:08.016 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-mgr-rook-2:19.0.0-2145.g3be9a5c9.el9.noarch 146/198 2024-03-28T00:13:08.054 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2145.g3be9a5c9.el9.noarch 146/198 2024-03-28T00:13:08.065 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-03-28T00:13:08.140 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-2:19.0.0-2145.g3be9a5c9.el9.x86_64 147/198 2024-03-28T00:13:08.142 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-03-28T00:13:08.186 INFO:teuthology.orchestra.run.smithi165.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-28T00:13:08.195 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-28T00:13:08.226 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 51/198 2024-03-28T00:13:08.275 INFO:teuthology.orchestra.run.smithi165.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-03-28T00:13:08.313 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : librados-devel-2:19.0.0-2145.g3be9a5c9.el9.x86_6 53/198 2024-03-28T00:13:08.342 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-03-28T00:13:08.576 INFO:teuthology.orchestra.run.smithi165.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-03-28T00:13:08.745 INFO:teuthology.orchestra.run.smithi165.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-03-28T00:13:08.820 INFO:teuthology.orchestra.run.smithi165.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-03-28T00:13:08.864 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-03-28T00:13:08.996 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-03-28T00:13:09.043 INFO:teuthology.orchestra.run.smithi165.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-03-28T00:13:09.097 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-03-28T00:13:09.135 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-03-28T00:13:09.179 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-03-28T00:13:09.196 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-radosgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 149/198 2024-03-28T00:13:09.210 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-03-28T00:13:09.247 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-03-28T00:13:09.283 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 149/198 2024-03-28T00:13:09.284 INFO:teuthology.orchestra.run.smithi119.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-28T00:13:09.284 INFO:teuthology.orchestra.run.smithi119.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-28T00:13:09.284 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-28T00:13:09.284 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-28T00:13:09.284 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-28T00:13:09.293 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-03-28T00:13:09.326 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-03-28T00:13:09.327 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2145.g3be9a 150/198 2024-03-28T00:13:09.370 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-03-28T00:13:09.412 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-03-28T00:13:09.416 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2145.g3be9a 150/198 2024-03-28T00:13:09.416 INFO:teuthology.orchestra.run.smithi119.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-28T00:13:09.416 INFO:teuthology.orchestra.run.smithi119.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-28T00:13:09.417 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-28T00:13:09.495 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : cephfs-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 151/198 2024-03-28T00:13:09.544 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-03-28T00:13:09.594 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 151/198 2024-03-28T00:13:09.594 INFO:teuthology.orchestra.run.smithi119.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-28T00:13:09.594 INFO:teuthology.orchestra.run.smithi119.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-28T00:13:09.594 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-28T00:13:09.594 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-28T00:13:09.594 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-28T00:13:09.594 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-03-28T00:13:09.631 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-03-28T00:13:09.673 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-03-28T00:13:09.767 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-03-28T00:13:09.815 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 74/198 2024-03-28T00:13:09.855 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-03-28T00:13:09.880 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-03-28T00:13:09.905 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-03-28T00:13:09.954 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : rbd-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 152/198 2024-03-28T00:13:09.977 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-03-28T00:13:10.021 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 152/198 2024-03-28T00:13:10.021 INFO:teuthology.orchestra.run.smithi119.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-28T00:13:10.021 INFO:teuthology.orchestra.run.smithi119.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-28T00:13:10.021 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-28T00:13:10.022 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-28T00:13:10.022 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-28T00:13:10.107 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-03-28T00:13:10.217 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-03-28T00:13:10.256 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-03-28T00:13:10.304 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-03-28T00:13:10.358 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2145.g3be9a5c9.el9.x86_64 132/198 2024-03-28T00:13:10.359 INFO:teuthology.orchestra.run.smithi132.stdout:skipping the directory /sys 2024-03-28T00:13:10.359 INFO:teuthology.orchestra.run.smithi132.stdout:skipping the directory /proc 2024-03-28T00:13:10.359 INFO:teuthology.orchestra.run.smithi132.stdout:skipping the directory /mnt 2024-03-28T00:13:10.359 INFO:teuthology.orchestra.run.smithi132.stdout:skipping the directory /var/tmp 2024-03-28T00:13:10.359 INFO:teuthology.orchestra.run.smithi132.stdout:skipping the directory /home 2024-03-28T00:13:10.359 INFO:teuthology.orchestra.run.smithi132.stdout:skipping the directory /root 2024-03-28T00:13:10.359 INFO:teuthology.orchestra.run.smithi132.stdout:skipping the directory /tmp 2024-03-28T00:13:10.359 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-28T00:13:10.360 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-03-28T00:13:10.530 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-volume-2:19.0.0-2145.g3be9a5c9.el9.noarch 133/198 2024-03-28T00:13:10.542 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-03-28T00:13:10.545 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-volume-2:19.0.0-2145.g3be9a5c9.el9.noarch 133/198 2024-03-28T00:13:10.546 INFO:teuthology.orchestra.run.smithi132.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-28T00:13:10.546 INFO:teuthology.orchestra.run.smithi132.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-28T00:13:10.546 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-28T00:13:10.585 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-03-28T00:13:10.714 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-03-28T00:13:10.749 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 87/198 2024-03-28T00:13:10.790 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-03-28T00:13:10.821 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-03-28T00:13:10.849 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-03-28T00:13:10.884 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-03-28T00:13:10.922 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-03-28T00:13:10.954 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-03-28T00:13:10.984 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-03-28T00:13:11.061 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-03-28T00:13:11.095 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-03-28T00:13:11.128 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-03-28T00:13:11.169 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-03-28T00:13:11.204 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-03-28T00:13:11.292 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-03-28T00:13:11.383 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-03-28T00:13:11.427 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-03-28T00:13:11.458 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-03-28T00:13:11.505 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-03-28T00:13:11.602 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-03-28T00:13:11.660 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-03-28T00:13:12.033 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-osd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 134/198 2024-03-28T00:13:12.058 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-osd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 134/198 2024-03-28T00:13:12.058 INFO:teuthology.orchestra.run.smithi132.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-28T00:13:12.058 INFO:teuthology.orchestra.run.smithi132.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-28T00:13:12.059 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-28T00:13:12.059 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-28T00:13:12.059 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-28T00:13:12.284 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-mds-2:19.0.0-2145.g3be9a5c9.el9.x86_64 135/198 2024-03-28T00:13:12.306 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-mds-2:19.0.0-2145.g3be9a5c9.el9.x86_64 135/198 2024-03-28T00:13:12.306 INFO:teuthology.orchestra.run.smithi132.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-28T00:13:12.306 INFO:teuthology.orchestra.run.smithi132.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-28T00:13:12.306 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-28T00:13:12.306 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-28T00:13:12.306 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-28T00:13:12.378 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-03-28T00:13:12.424 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-03-28T00:13:12.458 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-03-28T00:13:12.505 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-03-28T00:13:12.584 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-03-28T00:13:12.642 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-03-28T00:13:12.743 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-mon-2:19.0.0-2145.g3be9a5c9.el9.x86_64 136/198 2024-03-28T00:13:12.765 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-mon-2:19.0.0-2145.g3be9a5c9.el9.x86_64 136/198 2024-03-28T00:13:12.765 INFO:teuthology.orchestra.run.smithi132.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-28T00:13:12.765 INFO:teuthology.orchestra.run.smithi132.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-28T00:13:12.765 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-28T00:13:12.765 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-28T00:13:12.765 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-28T00:13:12.818 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-03-28T00:13:12.848 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-03-28T00:13:12.861 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-03-28T00:13:12.925 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-03-28T00:13:13.270 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: cephadm-2:19.0.0-2145.g3be9a5c9.el9.noarch 138/198 2024-03-28T00:13:13.301 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : cephadm-2:19.0.0-2145.g3be9a5c9.el9.noarch 138/198 2024-03-28T00:13:13.334 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2145.g3be9a5c9.e 139/198 2024-03-28T00:13:13.393 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2145.g3be9a5c9. 140/198 2024-03-28T00:13:13.425 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2145.g3be9a5c9.el9.noa 141/198 2024-03-28T00:13:13.557 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-03-28T00:13:13.659 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2145.g3be9a5c9.el9.noa 141/198 2024-03-28T00:13:13.661 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-03-28T00:13:13.692 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2145.g3be9a5c9.el9.n 142/198 2024-03-28T00:13:14.655 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2145.g3be9a5c9.el9.n 142/198 2024-03-28T00:13:14.684 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2145.g3be 143/198 2024-03-28T00:13:14.784 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2145.g3be 143/198 2024-03-28T00:13:14.836 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-test-2:19.0.0-2145.g3be9a5c9.el9.x86_64 153/198 2024-03-28T00:13:14.856 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-03-28T00:13:14.890 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : rbd-fuse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 154/198 2024-03-28T00:13:14.924 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : rbd-nbd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 155/198 2024-03-28T00:13:14.933 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-03-28T00:13:14.947 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2145.g3be9a5c9.el 144/198 2024-03-28T00:13:14.962 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-03-28T00:13:14.964 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-03-28T00:13:14.970 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-mgr-2:19.0.0-2145.g3be9a5c9.el9.x86_64 145/198 2024-03-28T00:13:14.993 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2145.g3be9a5c9.el9.x86_64 145/198 2024-03-28T00:13:14.993 INFO:teuthology.orchestra.run.smithi132.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-28T00:13:14.993 INFO:teuthology.orchestra.run.smithi132.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-28T00:13:14.994 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-28T00:13:14.994 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-28T00:13:14.994 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-28T00:13:14.996 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-03-28T00:13:15.023 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-03-28T00:13:15.037 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-mgr-rook-2:19.0.0-2145.g3be9a5c9.el9.noarch 146/198 2024-03-28T00:13:15.048 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2145.g3be9a5c9.el9.noarch 146/198 2024-03-28T00:13:15.094 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-2:19.0.0-2145.g3be9a5c9.el9.x86_64 147/198 2024-03-28T00:13:15.225 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : cephfs-shell-2:19.0.0-2145.g3be9a5c9.el9.x86_64 157/198 2024-03-28T00:13:15.335 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-03-28T00:13:15.346 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 158/198 2024-03-28T00:13:15.362 INFO:teuthology.orchestra.run.smithi165.stdout: Upgrading : librbd1-2:19.0.0-2145.g3be9a5c9.el9.x86_64 124/198 2024-03-28T00:13:15.391 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-fuse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 159/198 2024-03-28T00:13:15.441 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-03-28T00:13:15.452 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: librbd1-2:19.0.0-2145.g3be9a5c9.el9.x86_64 124/198 2024-03-28T00:13:15.483 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-rbd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 125/198 2024-03-28T00:13:15.507 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-03-28T00:13:15.519 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-03-28T00:13:15.577 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-03-28T00:13:15.612 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libcephfs-devel-2:19.0.0-2145.g3be9a5c9.el9.x86_ 163/198 2024-03-28T00:13:15.658 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-03-28T00:13:15.710 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : NetworkManager-team-1:1.47.2-1.el9.x86_64 165/198 2024-03-28T00:13:15.796 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : NetworkManager-tui-1:1.47.2-1.el9.x86_64 166/198 2024-03-28T00:13:15.905 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-03-28T00:13:15.950 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-03-28T00:13:15.967 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-03-28T00:13:15.980 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-03-28T00:13:16.006 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-radosgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 149/198 2024-03-28T00:13:16.007 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : librgw2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 128/198 2024-03-28T00:13:16.029 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 149/198 2024-03-28T00:13:16.029 INFO:teuthology.orchestra.run.smithi132.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-28T00:13:16.029 INFO:teuthology.orchestra.run.smithi132.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-28T00:13:16.030 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-28T00:13:16.030 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-28T00:13:16.030 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-28T00:13:16.074 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2145.g3be9a 150/198 2024-03-28T00:13:16.077 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: librgw2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 128/198 2024-03-28T00:13:16.096 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2145.g3be9a 150/198 2024-03-28T00:13:16.096 INFO:teuthology.orchestra.run.smithi132.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-28T00:13:16.096 INFO:teuthology.orchestra.run.smithi132.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-28T00:13:16.096 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-28T00:13:16.100 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-rgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 129/198 2024-03-28T00:13:16.141 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : cephfs-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 151/198 2024-03-28T00:13:16.163 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 151/198 2024-03-28T00:13:16.163 INFO:teuthology.orchestra.run.smithi132.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-28T00:13:16.163 INFO:teuthology.orchestra.run.smithi132.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-28T00:13:16.163 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-28T00:13:16.164 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-28T00:13:16.164 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-28T00:13:16.483 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : rbd-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 152/198 2024-03-28T00:13:16.506 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 152/198 2024-03-28T00:13:16.506 INFO:teuthology.orchestra.run.smithi132.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-28T00:13:16.506 INFO:teuthology.orchestra.run.smithi132.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-28T00:13:16.506 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-28T00:13:16.506 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-28T00:13:16.506 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-28T00:13:16.851 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-03-28T00:13:16.897 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 169/198 2024-03-28T00:13:16.950 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-03-28T00:13:18.257 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : cephfs-top-2:19.0.0-2145.g3be9a5c9.el9.noarch 170/198 2024-03-28T00:13:18.325 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-common-2:19.0.0-2145.g3be9a5c9.el9.x86_64 130/198 2024-03-28T00:13:18.344 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-03-28T00:13:18.395 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-03-28T00:13:18.516 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-03-28T00:13:18.528 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-common-2:19.0.0-2145.g3be9a5c9.el9.x86_64 130/198 2024-03-28T00:13:18.554 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-03-28T00:13:18.588 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-03-28T00:13:18.622 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-03-28T00:13:18.662 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-03-28T00:13:18.663 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-28T00:13:18.793 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-28T00:13:18.835 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-03-28T00:13:18.835 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-28T00:13:18.998 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-28T00:13:19.042 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-03-28T00:13:19.063 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-common-2:19.0.0-2145.g3be9a5c9.el9.x86_64 130/198 2024-03-28T00:13:19.088 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-03-28T00:13:19.108 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-base-2:19.0.0-2145.g3be9a5c9.el9.x86_64 131/198 2024-03-28T00:13:19.121 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-03-28T00:13:19.153 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-03-28T00:13:19.190 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-base-2:19.0.0-2145.g3be9a5c9.el9.x86_64 131/198 2024-03-28T00:13:19.190 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-28T00:13:19.190 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-28T00:13:19.190 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-03-28T00:13:19.223 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-03-28T00:13:19.226 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-selinux-2:19.0.0-2145.g3be9a5c9.el9.x86_64 132/198 2024-03-28T00:13:19.293 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-03-28T00:13:19.350 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-03-28T00:13:19.383 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-03-28T00:13:19.417 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-03-28T00:13:19.467 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-03-28T00:13:19.501 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-03-28T00:13:19.536 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-28T00:13:19.537 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-28T00:13:19.833 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-28T00:13:19.833 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-28T00:13:19.973 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-28T00:13:20.003 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-03-28T00:13:20.057 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-03-28T00:13:20.089 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-03-28T00:13:20.129 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-03-28T00:13:20.248 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-28T00:13:20.502 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-28T00:13:20.502 INFO:teuthology.orchestra.run.smithi119.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-03-28T00:13:20.502 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-28T00:13:20.938 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-test-2:19.0.0-2145.g3be9a5c9.el9.x86_64 153/198 2024-03-28T00:13:20.979 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : rbd-fuse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 154/198 2024-03-28T00:13:21.009 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : rbd-nbd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 155/198 2024-03-28T00:13:21.036 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-03-28T00:13:21.215 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : cephfs-shell-2:19.0.0-2145.g3be9a5c9.el9.x86_64 157/198 2024-03-28T00:13:21.336 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 158/198 2024-03-28T00:13:21.393 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-fuse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 159/198 2024-03-28T00:13:21.432 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-03-28T00:13:21.493 INFO:teuthology.orchestra.run.smithi132.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-03-28T00:13:21.536 INFO:teuthology.orchestra.run.smithi132.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-03-28T00:13:21.569 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libcephfs-devel-2:19.0.0-2145.g3be9a5c9.el9.x86_ 163/198 2024-03-28T00:13:21.602 INFO:teuthology.orchestra.run.smithi132.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-03-28T00:13:21.631 INFO:teuthology.orchestra.run.smithi132.stdout: Upgrading : NetworkManager-team-1:1.47.2-1.el9.x86_64 165/198 2024-03-28T00:13:21.685 INFO:teuthology.orchestra.run.smithi132.stdout: Upgrading : NetworkManager-tui-1:1.47.2-1.el9.x86_64 166/198 2024-03-28T00:13:21.747 INFO:teuthology.orchestra.run.smithi132.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-03-28T00:13:21.780 INFO:teuthology.orchestra.run.smithi132.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-03-28T00:13:22.344 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-03-28T00:13:22.378 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 169/198 2024-03-28T00:13:22.397 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-03-28T00:13:23.208 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : cephfs-top-2:19.0.0-2145.g3be9a5c9.el9.noarch 170/198 2024-03-28T00:13:23.282 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-03-28T00:13:23.321 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-03-28T00:13:23.403 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-03-28T00:13:23.454 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-03-28T00:13:23.477 INFO:teuthology.orchestra.run.smithi132.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-03-28T00:13:23.502 INFO:teuthology.orchestra.run.smithi132.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-03-28T00:13:23.528 INFO:teuthology.orchestra.run.smithi132.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-03-28T00:13:23.529 INFO:teuthology.orchestra.run.smithi132.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-28T00:13:23.600 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-28T00:13:23.630 INFO:teuthology.orchestra.run.smithi132.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-03-28T00:13:23.630 INFO:teuthology.orchestra.run.smithi132.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-28T00:13:23.698 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-28T00:13:23.724 INFO:teuthology.orchestra.run.smithi132.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-03-28T00:13:23.757 INFO:teuthology.orchestra.run.smithi132.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-03-28T00:13:23.791 INFO:teuthology.orchestra.run.smithi132.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-03-28T00:13:23.816 INFO:teuthology.orchestra.run.smithi132.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-03-28T00:13:23.849 INFO:teuthology.orchestra.run.smithi132.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-03-28T00:13:23.889 INFO:teuthology.orchestra.run.smithi132.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-03-28T00:13:23.921 INFO:teuthology.orchestra.run.smithi132.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-03-28T00:13:23.946 INFO:teuthology.orchestra.run.smithi132.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-03-28T00:13:23.972 INFO:teuthology.orchestra.run.smithi132.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-03-28T00:13:23.997 INFO:teuthology.orchestra.run.smithi132.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-03-28T00:13:24.023 INFO:teuthology.orchestra.run.smithi132.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-03-28T00:13:24.029 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-28T00:13:24.029 INFO:teuthology.orchestra.run.smithi132.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-28T00:13:24.217 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-28T00:13:24.217 INFO:teuthology.orchestra.run.smithi132.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-28T00:13:24.283 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-28T00:13:24.307 INFO:teuthology.orchestra.run.smithi132.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-03-28T00:13:24.340 INFO:teuthology.orchestra.run.smithi132.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-03-28T00:13:24.367 INFO:teuthology.orchestra.run.smithi132.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-03-28T00:13:24.392 INFO:teuthology.orchestra.run.smithi132.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-03-28T00:13:24.456 INFO:teuthology.orchestra.run.smithi132.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-28T00:13:25.424 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-2:19.0.0-2145.g3be9a5c9.el9.x86_64 1/198 2024-03-28T00:13:25.424 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-base-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2/198 2024-03-28T00:13:25.424 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-common-2:19.0.0-2145.g3be9a5c9.el9.x86_64 3/198 2024-03-28T00:13:25.424 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-fuse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 4/198 2024-03-28T00:13:25.424 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2145.g3be9a 5/198 2024-03-28T00:13:25.425 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-mds-2:19.0.0-2145.g3be9a5c9.el9.x86_64 6/198 2024-03-28T00:13:25.425 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-mgr-2:19.0.0-2145.g3be9a5c9.el9.x86_64 7/198 2024-03-28T00:13:25.425 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-mon-2:19.0.0-2145.g3be9a5c9.el9.x86_64 8/198 2024-03-28T00:13:25.425 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-osd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 9/198 2024-03-28T00:13:25.425 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-radosgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 10/198 2024-03-28T00:13:25.425 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-selinux-2:19.0.0-2145.g3be9a5c9.el9.x86_64 11/198 2024-03-28T00:13:25.425 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-test-2:19.0.0-2145.g3be9a5c9.el9.x86_64 12/198 2024-03-28T00:13:25.425 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : cephfs-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 13/198 2024-03-28T00:13:25.425 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : cephfs-shell-2:19.0.0-2145.g3be9a5c9.el9.x86_64 14/198 2024-03-28T00:13:25.425 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libcephfs-devel-2:19.0.0-2145.g3be9a5c9.el9.x86_ 15/198 2024-03-28T00:13:25.426 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libcephfs2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 16/198 2024-03-28T00:13:25.426 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libcephsqlite-2:19.0.0-2145.g3be9a5c9.el9.x86_64 17/198 2024-03-28T00:13:25.426 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : librados-devel-2:19.0.0-2145.g3be9a5c9.el9.x86_6 18/198 2024-03-28T00:13:25.426 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libradosstriper1-2:19.0.0-2145.g3be9a5c9.el9.x86 19/198 2024-03-28T00:13:25.426 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : librgw2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 20/198 2024-03-28T00:13:25.426 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2145.g3be9a5c9.el 21/198 2024-03-28T00:13:25.426 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-ceph-common-2:19.0.0-2145.g3be9a5c9.el9. 22/198 2024-03-28T00:13:25.426 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-cephfs-2:19.0.0-2145.g3be9a5c9.el9.x86_6 23/198 2024-03-28T00:13:25.426 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-rados-2:19.0.0-2145.g3be9a5c9.el9.x86_64 24/198 2024-03-28T00:13:25.426 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-rbd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 25/198 2024-03-28T00:13:25.426 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-rgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 26/198 2024-03-28T00:13:25.426 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : rbd-fuse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 27/198 2024-03-28T00:13:25.445 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : rbd-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 28/198 2024-03-28T00:13:25.445 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : rbd-nbd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 29/198 2024-03-28T00:13:25.445 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2145.g3be9a5c9. 30/198 2024-03-28T00:13:25.445 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2145.g3be9a5c9.el9.noa 31/198 2024-03-28T00:13:25.445 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2145.g3be9a5c9.el9.n 32/198 2024-03-28T00:13:25.445 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2145.g3be 33/198 2024-03-28T00:13:25.445 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2145.g3be9a5c9.el 34/198 2024-03-28T00:13:25.445 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2145.g3be9a5c9.el9.noarch 35/198 2024-03-28T00:13:25.445 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2145.g3be9a5c9.e 36/198 2024-03-28T00:13:25.446 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-volume-2:19.0.0-2145.g3be9a5c9.el9.noarch 37/198 2024-03-28T00:13:25.446 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : cephadm-2:19.0.0-2145.g3be9a5c9.el9.noarch 38/198 2024-03-28T00:13:25.446 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : cephfs-top-2:19.0.0-2145.g3be9a5c9.el9.noarch 39/198 2024-03-28T00:13:25.446 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-03-28T00:13:25.446 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-03-28T00:13:25.446 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-03-28T00:13:25.446 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-03-28T00:13:25.446 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-03-28T00:13:25.446 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-03-28T00:13:25.446 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-03-28T00:13:25.446 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-03-28T00:13:25.446 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-03-28T00:13:25.446 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-03-28T00:13:25.447 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-03-28T00:13:25.447 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-03-28T00:13:25.447 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-03-28T00:13:25.447 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-03-28T00:13:25.447 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-03-28T00:13:25.447 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-03-28T00:13:25.447 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-03-28T00:13:25.447 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-03-28T00:13:25.447 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 58/198 2024-03-28T00:13:25.447 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 59/198 2024-03-28T00:13:25.447 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 60/198 2024-03-28T00:13:25.448 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 61/198 2024-03-28T00:13:25.448 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 62/198 2024-03-28T00:13:25.448 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 63/198 2024-03-28T00:13:25.448 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 64/198 2024-03-28T00:13:25.448 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 65/198 2024-03-28T00:13:25.448 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 66/198 2024-03-28T00:13:25.448 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 67/198 2024-03-28T00:13:25.448 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 68/198 2024-03-28T00:13:25.448 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 69/198 2024-03-28T00:13:25.448 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 70/198 2024-03-28T00:13:25.448 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 71/198 2024-03-28T00:13:25.448 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 72/198 2024-03-28T00:13:25.448 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 73/198 2024-03-28T00:13:25.449 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 74/198 2024-03-28T00:13:25.449 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 75/198 2024-03-28T00:13:25.449 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 76/198 2024-03-28T00:13:25.449 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 77/198 2024-03-28T00:13:25.449 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 78/198 2024-03-28T00:13:25.449 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 79/198 2024-03-28T00:13:25.449 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 80/198 2024-03-28T00:13:25.449 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 81/198 2024-03-28T00:13:25.449 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 82/198 2024-03-28T00:13:25.449 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 83/198 2024-03-28T00:13:25.449 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 84/198 2024-03-28T00:13:25.449 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-ply-3.11-14.el9.noarch 85/198 2024-03-28T00:13:25.449 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 86/198 2024-03-28T00:13:25.450 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 87/198 2024-03-28T00:13:25.450 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : jq-1.6-16.el9.x86_64 88/198 2024-03-28T00:13:25.450 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 89/198 2024-03-28T00:13:25.450 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 90/198 2024-03-28T00:13:25.450 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 91/198 2024-03-28T00:13:25.450 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 92/198 2024-03-28T00:13:25.450 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 93/198 2024-03-28T00:13:25.450 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 94/198 2024-03-28T00:13:25.450 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : postgresql-13.14-1.el9.x86_64 95/198 2024-03-28T00:13:25.451 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 96/198 2024-03-28T00:13:25.451 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 97/198 2024-03-28T00:13:25.451 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 98/198 2024-03-28T00:13:25.451 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 99/198 2024-03-28T00:13:25.451 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 100/198 2024-03-28T00:13:25.451 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 101/198 2024-03-28T00:13:25.451 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 102/198 2024-03-28T00:13:25.451 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 103/198 2024-03-28T00:13:25.451 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-03-28T00:13:25.451 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-03-28T00:13:25.451 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-03-28T00:13:25.451 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-03-28T00:13:25.451 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-03-28T00:13:25.451 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-03-28T00:13:25.452 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-03-28T00:13:25.452 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-03-28T00:13:25.452 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-03-28T00:13:25.452 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-03-28T00:13:25.452 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-03-28T00:13:25.452 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-03-28T00:13:25.452 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/198 2024-03-28T00:13:25.452 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-03-28T00:13:25.452 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-03-28T00:13:25.452 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-03-28T00:13:25.452 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-03-28T00:13:25.452 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-03-28T00:13:25.452 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-03-28T00:13:25.452 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-03-28T00:13:25.453 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 124/198 2024-03-28T00:13:25.453 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-03-28T00:13:25.453 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-03-28T00:13:25.453 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-03-28T00:13:25.453 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-03-28T00:13:25.453 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-03-28T00:13:25.453 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-03-28T00:13:25.453 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-03-28T00:13:25.453 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-03-28T00:13:25.454 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-03-28T00:13:25.454 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-03-28T00:13:25.454 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-03-28T00:13:25.454 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-03-28T00:13:25.454 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-03-28T00:13:25.454 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-03-28T00:13:25.454 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-03-28T00:13:25.454 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-03-28T00:13:25.455 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-03-28T00:13:25.455 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-03-28T00:13:25.455 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-03-28T00:13:25.455 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-03-28T00:13:25.455 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-03-28T00:13:25.455 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-03-28T00:13:25.455 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-03-28T00:13:25.455 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-03-28T00:13:25.455 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-03-28T00:13:25.455 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-03-28T00:13:25.455 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : librados2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 151/198 2024-03-28T00:13:25.455 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-03-28T00:13:25.456 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : librbd1-2:19.0.0-2145.g3be9a5c9.el9.x86_64 153/198 2024-03-28T00:13:25.456 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-03-28T00:13:25.456 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 155/198 2024-03-28T00:13:25.456 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 156/198 2024-03-28T00:13:25.456 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : NetworkManager-1:1.47.2-1.el9.x86_64 157/198 2024-03-28T00:13:25.456 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 158/198 2024-03-28T00:13:25.456 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 159/198 2024-03-28T00:13:25.456 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/198 2024-03-28T00:13:25.456 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : NetworkManager-team-1:1.47.2-1.el9.x86_64 161/198 2024-03-28T00:13:25.456 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 162/198 2024-03-28T00:13:25.456 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : NetworkManager-tui-1:1.47.2-1.el9.x86_64 163/198 2024-03-28T00:13:25.456 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 164/198 2024-03-28T00:13:25.456 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-03-28T00:13:25.457 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-03-28T00:13:25.457 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-03-28T00:13:25.457 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-03-28T00:13:25.457 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-03-28T00:13:25.457 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-03-28T00:13:25.457 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-03-28T00:13:25.457 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-03-28T00:13:25.457 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-03-28T00:13:25.457 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-03-28T00:13:25.457 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-03-28T00:13:25.457 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-03-28T00:13:25.457 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-03-28T00:13:25.458 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-03-28T00:13:25.458 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-03-28T00:13:25.458 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-03-28T00:13:25.458 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-03-28T00:13:25.458 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-03-28T00:13:25.458 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-03-28T00:13:25.458 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-03-28T00:13:25.458 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-03-28T00:13:25.458 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-03-28T00:13:25.458 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-03-28T00:13:25.458 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-03-28T00:13:25.458 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-03-28T00:13:25.459 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-03-28T00:13:25.459 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-03-28T00:13:25.459 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-03-28T00:13:25.459 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-03-28T00:13:25.459 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-03-28T00:13:25.459 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-03-28T00:13:25.459 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-03-28T00:13:25.459 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-03-28T00:13:28.069 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-03-28T00:13:28.069 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-28T00:13:28.069 INFO:teuthology.orchestra.run.smithi119.stdout:Upgraded: 2024-03-28T00:13:28.069 INFO:teuthology.orchestra.run.smithi119.stdout: NetworkManager-1:1.47.2-1.el9.x86_64 2024-03-28T00:13:28.069 INFO:teuthology.orchestra.run.smithi119.stdout: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 2024-03-28T00:13:28.070 INFO:teuthology.orchestra.run.smithi119.stdout: NetworkManager-team-1:1.47.2-1.el9.x86_64 2024-03-28T00:13:28.070 INFO:teuthology.orchestra.run.smithi119.stdout: NetworkManager-tui-1:1.47.2-1.el9.x86_64 2024-03-28T00:13:28.070 INFO:teuthology.orchestra.run.smithi119.stdout: acl-2.3.1-4.el9.x86_64 2024-03-28T00:13:28.070 INFO:teuthology.orchestra.run.smithi119.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-28T00:13:28.070 INFO:teuthology.orchestra.run.smithi119.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-28T00:13:28.070 INFO:teuthology.orchestra.run.smithi119.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-28T00:13:28.071 INFO:teuthology.orchestra.run.smithi119.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-28T00:13:28.071 INFO:teuthology.orchestra.run.smithi119.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-28T00:13:28.071 INFO:teuthology.orchestra.run.smithi119.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-28T00:13:28.071 INFO:teuthology.orchestra.run.smithi119.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-28T00:13:28.071 INFO:teuthology.orchestra.run.smithi119.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-28T00:13:28.071 INFO:teuthology.orchestra.run.smithi119.stdout: librados2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T00:13:28.071 INFO:teuthology.orchestra.run.smithi119.stdout: librbd1-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T00:13:28.072 INFO:teuthology.orchestra.run.smithi119.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-28T00:13:28.072 INFO:teuthology.orchestra.run.smithi119.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-28T00:13:28.072 INFO:teuthology.orchestra.run.smithi119.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-28T00:13:28.072 INFO:teuthology.orchestra.run.smithi119.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-28T00:13:28.072 INFO:teuthology.orchestra.run.smithi119.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-28T00:13:28.072 INFO:teuthology.orchestra.run.smithi119.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-28T00:13:28.072 INFO:teuthology.orchestra.run.smithi119.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-28T00:13:28.073 INFO:teuthology.orchestra.run.smithi119.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-28T00:13:28.073 INFO:teuthology.orchestra.run.smithi119.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-28T00:13:28.073 INFO:teuthology.orchestra.run.smithi119.stdout:Installed: 2024-03-28T00:13:28.073 INFO:teuthology.orchestra.run.smithi119.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-28T00:13:28.073 INFO:teuthology.orchestra.run.smithi119.stdout: automake-1.16.2-8.el9.noarch 2024-03-28T00:13:28.073 INFO:teuthology.orchestra.run.smithi119.stdout: bison-3.7.4-5.el9.x86_64 2024-03-28T00:13:28.073 INFO:teuthology.orchestra.run.smithi119.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-28T00:13:28.073 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T00:13:28.074 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-base-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T00:13:28.074 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-common-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T00:13:28.074 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-fuse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T00:13:28.074 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-grafana-dashboards-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-28T00:13:28.074 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-immutable-object-cache-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T00:13:28.074 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mds-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T00:13:28.075 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T00:13:28.075 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-cephadm-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-28T00:13:28.075 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-dashboard-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-28T00:13:28.075 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-28T00:13:28.075 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-modules-core-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-28T00:13:28.075 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-rook-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-28T00:13:28.076 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mon-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T00:13:28.076 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-osd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T00:13:28.076 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-prometheus-alerts-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-28T00:13:28.076 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-radosgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T00:13:28.076 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-selinux-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T00:13:28.076 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-test-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T00:13:28.077 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-volume-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-28T00:13:28.077 INFO:teuthology.orchestra.run.smithi119.stdout: cephadm-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-28T00:13:28.077 INFO:teuthology.orchestra.run.smithi119.stdout: cephfs-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T00:13:28.077 INFO:teuthology.orchestra.run.smithi119.stdout: cephfs-shell-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T00:13:28.077 INFO:teuthology.orchestra.run.smithi119.stdout: cephfs-top-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-28T00:13:28.077 INFO:teuthology.orchestra.run.smithi119.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-28T00:13:28.077 INFO:teuthology.orchestra.run.smithi119.stdout: flex-2.6.4-9.el9.x86_64 2024-03-28T00:13:28.077 INFO:teuthology.orchestra.run.smithi119.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-28T00:13:28.078 INFO:teuthology.orchestra.run.smithi119.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-28T00:13:28.078 INFO:teuthology.orchestra.run.smithi119.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-28T00:13:28.078 INFO:teuthology.orchestra.run.smithi119.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-28T00:13:28.078 INFO:teuthology.orchestra.run.smithi119.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-28T00:13:28.078 INFO:teuthology.orchestra.run.smithi119.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-28T00:13:28.078 INFO:teuthology.orchestra.run.smithi119.stdout: jq-1.6-16.el9.x86_64 2024-03-28T00:13:28.078 INFO:teuthology.orchestra.run.smithi119.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-28T00:13:28.078 INFO:teuthology.orchestra.run.smithi119.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-28T00:13:28.078 INFO:teuthology.orchestra.run.smithi119.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-28T00:13:28.079 INFO:teuthology.orchestra.run.smithi119.stdout: libcephfs-devel-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T00:13:28.079 INFO:teuthology.orchestra.run.smithi119.stdout: libcephfs2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T00:13:28.079 INFO:teuthology.orchestra.run.smithi119.stdout: libcephsqlite-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T00:13:28.079 INFO:teuthology.orchestra.run.smithi119.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-28T00:13:28.079 INFO:teuthology.orchestra.run.smithi119.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-28T00:13:28.079 INFO:teuthology.orchestra.run.smithi119.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-28T00:13:28.079 INFO:teuthology.orchestra.run.smithi119.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-28T00:13:28.079 INFO:teuthology.orchestra.run.smithi119.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-28T00:13:28.079 INFO:teuthology.orchestra.run.smithi119.stdout: librados-devel-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T00:13:28.079 INFO:teuthology.orchestra.run.smithi119.stdout: libradosstriper1-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T00:13:28.079 INFO:teuthology.orchestra.run.smithi119.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-28T00:13:28.079 INFO:teuthology.orchestra.run.smithi119.stdout: librgw2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T00:13:28.079 INFO:teuthology.orchestra.run.smithi119.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-28T00:13:28.079 INFO:teuthology.orchestra.run.smithi119.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-28T00:13:28.080 INFO:teuthology.orchestra.run.smithi119.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-28T00:13:28.080 INFO:teuthology.orchestra.run.smithi119.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-28T00:13:28.080 INFO:teuthology.orchestra.run.smithi119.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-28T00:13:28.080 INFO:teuthology.orchestra.run.smithi119.stdout: lua-5.4.4-4.el9.x86_64 2024-03-28T00:13:28.080 INFO:teuthology.orchestra.run.smithi119.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-28T00:13:28.080 INFO:teuthology.orchestra.run.smithi119.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-28T00:13:28.080 INFO:teuthology.orchestra.run.smithi119.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-28T00:13:28.080 INFO:teuthology.orchestra.run.smithi119.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-28T00:13:28.080 INFO:teuthology.orchestra.run.smithi119.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-28T00:13:28.080 INFO:teuthology.orchestra.run.smithi119.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-28T00:13:28.080 INFO:teuthology.orchestra.run.smithi119.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-28T00:13:28.080 INFO:teuthology.orchestra.run.smithi119.stdout: postgresql-13.14-1.el9.x86_64 2024-03-28T00:13:28.080 INFO:teuthology.orchestra.run.smithi119.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-03-28T00:13:28.080 INFO:teuthology.orchestra.run.smithi119.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-03-28T00:13:28.081 INFO:teuthology.orchestra.run.smithi119.stdout: postgresql-server-13.14-1.el9.x86_64 2024-03-28T00:13:28.081 INFO:teuthology.orchestra.run.smithi119.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-28T00:13:28.081 INFO:teuthology.orchestra.run.smithi119.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-28T00:13:28.081 INFO:teuthology.orchestra.run.smithi119.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-03-28T00:13:28.081 INFO:teuthology.orchestra.run.smithi119.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-28T00:13:28.081 INFO:teuthology.orchestra.run.smithi119.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-28T00:13:28.081 INFO:teuthology.orchestra.run.smithi119.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-28T00:13:28.081 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-28T00:13:28.081 INFO:teuthology.orchestra.run.smithi119.stdout: python3-ceph-argparse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T00:13:28.081 INFO:teuthology.orchestra.run.smithi119.stdout: python3-ceph-common-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T00:13:28.081 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cephfs-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T00:13:28.081 INFO:teuthology.orchestra.run.smithi119.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-28T00:13:28.081 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-28T00:13:28.081 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-28T00:13:28.082 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-28T00:13:28.082 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-03-28T00:13:28.082 INFO:teuthology.orchestra.run.smithi119.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-03-28T00:13:28.082 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-28T00:13:28.082 INFO:teuthology.orchestra.run.smithi119.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-28T00:13:28.082 INFO:teuthology.orchestra.run.smithi119.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-03-28T00:13:28.082 INFO:teuthology.orchestra.run.smithi119.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-28T00:13:28.082 INFO:teuthology.orchestra.run.smithi119.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-28T00:13:28.082 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-28T00:13:28.082 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-28T00:13:28.082 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-28T00:13:28.082 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-28T00:13:28.083 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-28T00:13:28.083 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-28T00:13:28.083 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-28T00:13:28.083 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-28T00:13:28.083 INFO:teuthology.orchestra.run.smithi119.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-28T00:13:28.083 INFO:teuthology.orchestra.run.smithi119.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-28T00:13:28.083 INFO:teuthology.orchestra.run.smithi119.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-28T00:13:28.083 INFO:teuthology.orchestra.run.smithi119.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-28T00:13:28.083 INFO:teuthology.orchestra.run.smithi119.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-28T00:13:28.083 INFO:teuthology.orchestra.run.smithi119.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-28T00:13:28.083 INFO:teuthology.orchestra.run.smithi119.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-28T00:13:28.083 INFO:teuthology.orchestra.run.smithi119.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-28T00:13:28.083 INFO:teuthology.orchestra.run.smithi119.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-28T00:13:28.083 INFO:teuthology.orchestra.run.smithi119.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-28T00:13:28.084 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-28T00:13:28.084 INFO:teuthology.orchestra.run.smithi119.stdout: python3-ply-3.11-14.el9.noarch 2024-03-28T00:13:28.084 INFO:teuthology.orchestra.run.smithi119.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-28T00:13:28.084 INFO:teuthology.orchestra.run.smithi119.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-28T00:13:28.084 INFO:teuthology.orchestra.run.smithi119.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-28T00:13:28.084 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-28T00:13:28.084 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-28T00:13:28.084 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-28T00:13:28.084 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-28T00:13:28.084 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-03-28T00:13:28.084 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-28T00:13:28.084 INFO:teuthology.orchestra.run.smithi119.stdout: python3-rados-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T00:13:28.084 INFO:teuthology.orchestra.run.smithi119.stdout: python3-rbd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T00:13:28.085 INFO:teuthology.orchestra.run.smithi119.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-28T00:13:28.085 INFO:teuthology.orchestra.run.smithi119.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-28T00:13:28.085 INFO:teuthology.orchestra.run.smithi119.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-28T00:13:28.085 INFO:teuthology.orchestra.run.smithi119.stdout: python3-rgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T00:13:28.085 INFO:teuthology.orchestra.run.smithi119.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-28T00:13:28.085 INFO:teuthology.orchestra.run.smithi119.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-28T00:13:28.085 INFO:teuthology.orchestra.run.smithi119.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-28T00:13:28.085 INFO:teuthology.orchestra.run.smithi119.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-28T00:13:28.085 INFO:teuthology.orchestra.run.smithi119.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-28T00:13:28.085 INFO:teuthology.orchestra.run.smithi119.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-28T00:13:28.085 INFO:teuthology.orchestra.run.smithi119.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-28T00:13:28.085 INFO:teuthology.orchestra.run.smithi119.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-03-28T00:13:28.085 INFO:teuthology.orchestra.run.smithi119.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-28T00:13:28.086 INFO:teuthology.orchestra.run.smithi119.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-28T00:13:28.086 INFO:teuthology.orchestra.run.smithi119.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-28T00:13:28.086 INFO:teuthology.orchestra.run.smithi119.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-28T00:13:28.086 INFO:teuthology.orchestra.run.smithi119.stdout: qatlib-24.02.0-1.el9.x86_64 2024-03-28T00:13:28.086 INFO:teuthology.orchestra.run.smithi119.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-03-28T00:13:28.086 INFO:teuthology.orchestra.run.smithi119.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-03-28T00:13:28.086 INFO:teuthology.orchestra.run.smithi119.stdout: rbd-fuse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T00:13:28.086 INFO:teuthology.orchestra.run.smithi119.stdout: rbd-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T00:13:28.086 INFO:teuthology.orchestra.run.smithi119.stdout: rbd-nbd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T00:13:28.086 INFO:teuthology.orchestra.run.smithi119.stdout: re2-1:20211101-3.el9.x86_64 2024-03-28T00:13:28.086 INFO:teuthology.orchestra.run.smithi119.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-28T00:13:28.086 INFO:teuthology.orchestra.run.smithi119.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-28T00:13:28.086 INFO:teuthology.orchestra.run.smithi119.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-28T00:13:28.086 INFO:teuthology.orchestra.run.smithi119.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-28T00:13:28.087 INFO:teuthology.orchestra.run.smithi119.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-28T00:13:28.087 INFO:teuthology.orchestra.run.smithi119.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-28T00:13:28.087 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-28T00:13:28.087 INFO:teuthology.orchestra.run.smithi119.stdout:Complete! 2024-03-28T00:13:28.830 DEBUG:teuthology.parallel:result is None 2024-03-28T00:13:29.400 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-28T00:13:29.400 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-2:19.0.0-2145.g3be9a5c9.el9.x86_64 1/198 2024-03-28T00:13:29.400 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-base-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2/198 2024-03-28T00:13:29.400 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-common-2:19.0.0-2145.g3be9a5c9.el9.x86_64 3/198 2024-03-28T00:13:29.400 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-fuse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 4/198 2024-03-28T00:13:29.401 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2145.g3be9a 5/198 2024-03-28T00:13:29.401 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-mds-2:19.0.0-2145.g3be9a5c9.el9.x86_64 6/198 2024-03-28T00:13:29.401 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-mgr-2:19.0.0-2145.g3be9a5c9.el9.x86_64 7/198 2024-03-28T00:13:29.401 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-mon-2:19.0.0-2145.g3be9a5c9.el9.x86_64 8/198 2024-03-28T00:13:29.401 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-osd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 9/198 2024-03-28T00:13:29.401 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-radosgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 10/198 2024-03-28T00:13:29.401 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-selinux-2:19.0.0-2145.g3be9a5c9.el9.x86_64 11/198 2024-03-28T00:13:29.401 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-test-2:19.0.0-2145.g3be9a5c9.el9.x86_64 12/198 2024-03-28T00:13:29.401 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : cephfs-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 13/198 2024-03-28T00:13:29.401 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : cephfs-shell-2:19.0.0-2145.g3be9a5c9.el9.x86_64 14/198 2024-03-28T00:13:29.401 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libcephfs-devel-2:19.0.0-2145.g3be9a5c9.el9.x86_ 15/198 2024-03-28T00:13:29.401 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libcephfs2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 16/198 2024-03-28T00:13:29.401 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libcephsqlite-2:19.0.0-2145.g3be9a5c9.el9.x86_64 17/198 2024-03-28T00:13:29.401 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : librados-devel-2:19.0.0-2145.g3be9a5c9.el9.x86_6 18/198 2024-03-28T00:13:29.402 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libradosstriper1-2:19.0.0-2145.g3be9a5c9.el9.x86 19/198 2024-03-28T00:13:29.402 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : librgw2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 20/198 2024-03-28T00:13:29.402 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2145.g3be9a5c9.el 21/198 2024-03-28T00:13:29.402 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-ceph-common-2:19.0.0-2145.g3be9a5c9.el9. 22/198 2024-03-28T00:13:29.402 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-cephfs-2:19.0.0-2145.g3be9a5c9.el9.x86_6 23/198 2024-03-28T00:13:29.402 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-rados-2:19.0.0-2145.g3be9a5c9.el9.x86_64 24/198 2024-03-28T00:13:29.402 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-rbd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 25/198 2024-03-28T00:13:29.402 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-rgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 26/198 2024-03-28T00:13:29.402 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : rbd-fuse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 27/198 2024-03-28T00:13:29.402 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : rbd-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 28/198 2024-03-28T00:13:29.402 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : rbd-nbd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 29/198 2024-03-28T00:13:29.402 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2145.g3be9a5c9. 30/198 2024-03-28T00:13:29.418 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2145.g3be9a5c9.el9.noa 31/198 2024-03-28T00:13:29.418 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2145.g3be9a5c9.el9.n 32/198 2024-03-28T00:13:29.418 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2145.g3be 33/198 2024-03-28T00:13:29.418 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2145.g3be9a5c9.el 34/198 2024-03-28T00:13:29.418 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2145.g3be9a5c9.el9.noarch 35/198 2024-03-28T00:13:29.418 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2145.g3be9a5c9.e 36/198 2024-03-28T00:13:29.418 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-volume-2:19.0.0-2145.g3be9a5c9.el9.noarch 37/198 2024-03-28T00:13:29.419 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : cephadm-2:19.0.0-2145.g3be9a5c9.el9.noarch 38/198 2024-03-28T00:13:29.419 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : cephfs-top-2:19.0.0-2145.g3be9a5c9.el9.noarch 39/198 2024-03-28T00:13:29.419 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-03-28T00:13:29.419 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-03-28T00:13:29.419 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-03-28T00:13:29.419 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-03-28T00:13:29.419 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-03-28T00:13:29.419 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-03-28T00:13:29.419 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-03-28T00:13:29.419 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-03-28T00:13:29.419 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-03-28T00:13:29.419 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-03-28T00:13:29.419 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-03-28T00:13:29.419 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-03-28T00:13:29.420 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-03-28T00:13:29.420 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-03-28T00:13:29.420 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-03-28T00:13:29.420 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-03-28T00:13:29.420 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-03-28T00:13:29.420 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-03-28T00:13:29.420 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 58/198 2024-03-28T00:13:29.420 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 59/198 2024-03-28T00:13:29.420 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 60/198 2024-03-28T00:13:29.420 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 61/198 2024-03-28T00:13:29.420 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 62/198 2024-03-28T00:13:29.420 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 63/198 2024-03-28T00:13:29.420 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 64/198 2024-03-28T00:13:29.420 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 65/198 2024-03-28T00:13:29.420 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 66/198 2024-03-28T00:13:29.421 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 67/198 2024-03-28T00:13:29.421 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 68/198 2024-03-28T00:13:29.421 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 69/198 2024-03-28T00:13:29.421 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 70/198 2024-03-28T00:13:29.421 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 71/198 2024-03-28T00:13:29.421 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 72/198 2024-03-28T00:13:29.421 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 73/198 2024-03-28T00:13:29.421 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 74/198 2024-03-28T00:13:29.421 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 75/198 2024-03-28T00:13:29.421 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 76/198 2024-03-28T00:13:29.421 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 77/198 2024-03-28T00:13:29.421 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 78/198 2024-03-28T00:13:29.421 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 79/198 2024-03-28T00:13:29.422 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 80/198 2024-03-28T00:13:29.422 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 81/198 2024-03-28T00:13:29.422 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 82/198 2024-03-28T00:13:29.422 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 83/198 2024-03-28T00:13:29.422 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 84/198 2024-03-28T00:13:29.422 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-ply-3.11-14.el9.noarch 85/198 2024-03-28T00:13:29.422 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 86/198 2024-03-28T00:13:29.422 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 87/198 2024-03-28T00:13:29.422 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : jq-1.6-16.el9.x86_64 88/198 2024-03-28T00:13:29.422 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 89/198 2024-03-28T00:13:29.422 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 90/198 2024-03-28T00:13:29.422 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 91/198 2024-03-28T00:13:29.422 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 92/198 2024-03-28T00:13:29.422 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 93/198 2024-03-28T00:13:29.423 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 94/198 2024-03-28T00:13:29.423 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : postgresql-13.14-1.el9.x86_64 95/198 2024-03-28T00:13:29.423 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 96/198 2024-03-28T00:13:29.423 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 97/198 2024-03-28T00:13:29.423 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 98/198 2024-03-28T00:13:29.423 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 99/198 2024-03-28T00:13:29.423 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 100/198 2024-03-28T00:13:29.423 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 101/198 2024-03-28T00:13:29.423 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 102/198 2024-03-28T00:13:29.423 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 103/198 2024-03-28T00:13:29.423 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-03-28T00:13:29.423 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-03-28T00:13:29.423 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-03-28T00:13:29.423 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-03-28T00:13:29.424 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-03-28T00:13:29.424 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-03-28T00:13:29.424 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-03-28T00:13:29.424 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-03-28T00:13:29.424 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-03-28T00:13:29.424 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-03-28T00:13:29.424 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-03-28T00:13:29.424 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-03-28T00:13:29.424 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/198 2024-03-28T00:13:29.424 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-03-28T00:13:29.424 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-03-28T00:13:29.424 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-03-28T00:13:29.424 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-03-28T00:13:29.424 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-03-28T00:13:29.424 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-03-28T00:13:29.425 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-03-28T00:13:29.425 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 124/198 2024-03-28T00:13:29.425 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-03-28T00:13:29.425 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-03-28T00:13:29.425 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-03-28T00:13:29.425 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-03-28T00:13:29.425 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-03-28T00:13:29.425 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-03-28T00:13:29.425 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-03-28T00:13:29.425 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-03-28T00:13:29.425 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-03-28T00:13:29.425 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-03-28T00:13:29.425 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-03-28T00:13:29.425 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-03-28T00:13:29.425 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-03-28T00:13:29.426 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-03-28T00:13:29.426 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-03-28T00:13:29.426 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-03-28T00:13:29.426 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-03-28T00:13:29.426 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-03-28T00:13:29.426 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-03-28T00:13:29.426 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-03-28T00:13:29.426 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-03-28T00:13:29.426 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-03-28T00:13:29.426 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-03-28T00:13:29.426 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-03-28T00:13:29.426 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-03-28T00:13:29.426 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-03-28T00:13:29.426 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : librados2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 151/198 2024-03-28T00:13:29.427 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-03-28T00:13:29.427 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : librbd1-2:19.0.0-2145.g3be9a5c9.el9.x86_64 153/198 2024-03-28T00:13:29.427 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-03-28T00:13:29.427 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 155/198 2024-03-28T00:13:29.427 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 156/198 2024-03-28T00:13:29.427 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : NetworkManager-1:1.47.2-1.el9.x86_64 157/198 2024-03-28T00:13:29.427 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 158/198 2024-03-28T00:13:29.427 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 159/198 2024-03-28T00:13:29.427 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/198 2024-03-28T00:13:29.427 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : NetworkManager-team-1:1.47.2-1.el9.x86_64 161/198 2024-03-28T00:13:29.427 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 162/198 2024-03-28T00:13:29.427 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : NetworkManager-tui-1:1.47.2-1.el9.x86_64 163/198 2024-03-28T00:13:29.427 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 164/198 2024-03-28T00:13:29.427 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-03-28T00:13:29.427 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-03-28T00:13:29.428 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-03-28T00:13:29.428 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-03-28T00:13:29.428 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-03-28T00:13:29.428 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-03-28T00:13:29.428 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-03-28T00:13:29.428 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-03-28T00:13:29.428 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-03-28T00:13:29.428 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-03-28T00:13:29.428 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-03-28T00:13:29.428 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-03-28T00:13:29.428 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-03-28T00:13:29.428 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-03-28T00:13:29.428 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-03-28T00:13:29.428 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-03-28T00:13:29.429 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-03-28T00:13:29.429 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-03-28T00:13:29.429 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-03-28T00:13:29.429 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-03-28T00:13:29.429 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-03-28T00:13:29.429 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-03-28T00:13:29.429 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-03-28T00:13:29.429 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-03-28T00:13:29.429 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-03-28T00:13:29.429 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-03-28T00:13:29.429 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-03-28T00:13:29.429 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-03-28T00:13:29.429 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-03-28T00:13:29.429 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-03-28T00:13:29.429 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-03-28T00:13:29.430 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-03-28T00:13:29.430 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-03-28T00:13:30.891 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2145.g3be9a5c9.el9.x86_64 132/198 2024-03-28T00:13:30.891 INFO:teuthology.orchestra.run.smithi165.stdout:skipping the directory /sys 2024-03-28T00:13:30.892 INFO:teuthology.orchestra.run.smithi165.stdout:skipping the directory /proc 2024-03-28T00:13:30.892 INFO:teuthology.orchestra.run.smithi165.stdout:skipping the directory /mnt 2024-03-28T00:13:30.892 INFO:teuthology.orchestra.run.smithi165.stdout:skipping the directory /var/tmp 2024-03-28T00:13:30.892 INFO:teuthology.orchestra.run.smithi165.stdout:skipping the directory /home 2024-03-28T00:13:30.892 INFO:teuthology.orchestra.run.smithi165.stdout:skipping the directory /root 2024-03-28T00:13:30.892 INFO:teuthology.orchestra.run.smithi165.stdout:skipping the directory /tmp 2024-03-28T00:13:30.892 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-03-28T00:13:31.078 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-volume-2:19.0.0-2145.g3be9a5c9.el9.noarch 133/198 2024-03-28T00:13:31.095 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-volume-2:19.0.0-2145.g3be9a5c9.el9.noarch 133/198 2024-03-28T00:13:31.095 INFO:teuthology.orchestra.run.smithi165.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-28T00:13:31.096 INFO:teuthology.orchestra.run.smithi165.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-28T00:13:31.096 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-03-28T00:13:32.377 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-03-28T00:13:32.377 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-28T00:13:32.377 INFO:teuthology.orchestra.run.smithi132.stdout:Upgraded: 2024-03-28T00:13:32.377 INFO:teuthology.orchestra.run.smithi132.stdout: NetworkManager-1:1.47.2-1.el9.x86_64 2024-03-28T00:13:32.377 INFO:teuthology.orchestra.run.smithi132.stdout: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 2024-03-28T00:13:32.377 INFO:teuthology.orchestra.run.smithi132.stdout: NetworkManager-team-1:1.47.2-1.el9.x86_64 2024-03-28T00:13:32.377 INFO:teuthology.orchestra.run.smithi132.stdout: NetworkManager-tui-1:1.47.2-1.el9.x86_64 2024-03-28T00:13:32.377 INFO:teuthology.orchestra.run.smithi132.stdout: acl-2.3.1-4.el9.x86_64 2024-03-28T00:13:32.377 INFO:teuthology.orchestra.run.smithi132.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-28T00:13:32.378 INFO:teuthology.orchestra.run.smithi132.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-28T00:13:32.378 INFO:teuthology.orchestra.run.smithi132.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-28T00:13:32.378 INFO:teuthology.orchestra.run.smithi132.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-28T00:13:32.378 INFO:teuthology.orchestra.run.smithi132.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-28T00:13:32.378 INFO:teuthology.orchestra.run.smithi132.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-28T00:13:32.378 INFO:teuthology.orchestra.run.smithi132.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-28T00:13:32.378 INFO:teuthology.orchestra.run.smithi132.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-28T00:13:32.378 INFO:teuthology.orchestra.run.smithi132.stdout: librados2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T00:13:32.378 INFO:teuthology.orchestra.run.smithi132.stdout: librbd1-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T00:13:32.378 INFO:teuthology.orchestra.run.smithi132.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-28T00:13:32.378 INFO:teuthology.orchestra.run.smithi132.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-28T00:13:32.378 INFO:teuthology.orchestra.run.smithi132.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-28T00:13:32.378 INFO:teuthology.orchestra.run.smithi132.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-28T00:13:32.379 INFO:teuthology.orchestra.run.smithi132.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-28T00:13:32.379 INFO:teuthology.orchestra.run.smithi132.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-28T00:13:32.379 INFO:teuthology.orchestra.run.smithi132.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-28T00:13:32.379 INFO:teuthology.orchestra.run.smithi132.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-28T00:13:32.379 INFO:teuthology.orchestra.run.smithi132.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-28T00:13:32.379 INFO:teuthology.orchestra.run.smithi132.stdout:Installed: 2024-03-28T00:13:32.379 INFO:teuthology.orchestra.run.smithi132.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-28T00:13:32.379 INFO:teuthology.orchestra.run.smithi132.stdout: automake-1.16.2-8.el9.noarch 2024-03-28T00:13:32.379 INFO:teuthology.orchestra.run.smithi132.stdout: bison-3.7.4-5.el9.x86_64 2024-03-28T00:13:32.379 INFO:teuthology.orchestra.run.smithi132.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-28T00:13:32.379 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T00:13:32.379 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-base-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T00:13:32.379 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-common-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T00:13:32.379 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-fuse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T00:13:32.380 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-grafana-dashboards-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-28T00:13:32.380 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-immutable-object-cache-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T00:13:32.380 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mds-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T00:13:32.380 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mgr-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T00:13:32.380 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mgr-cephadm-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-28T00:13:32.380 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mgr-dashboard-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-28T00:13:32.380 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-28T00:13:32.380 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mgr-modules-core-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-28T00:13:32.380 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mgr-rook-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-28T00:13:32.380 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mon-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T00:13:32.380 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-osd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T00:13:32.380 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-prometheus-alerts-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-28T00:13:32.380 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-radosgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T00:13:32.381 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-selinux-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T00:13:32.381 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-test-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T00:13:32.381 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-volume-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-28T00:13:32.381 INFO:teuthology.orchestra.run.smithi132.stdout: cephadm-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-28T00:13:32.381 INFO:teuthology.orchestra.run.smithi132.stdout: cephfs-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T00:13:32.381 INFO:teuthology.orchestra.run.smithi132.stdout: cephfs-shell-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T00:13:32.381 INFO:teuthology.orchestra.run.smithi132.stdout: cephfs-top-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-28T00:13:32.381 INFO:teuthology.orchestra.run.smithi132.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-28T00:13:32.381 INFO:teuthology.orchestra.run.smithi132.stdout: flex-2.6.4-9.el9.x86_64 2024-03-28T00:13:32.381 INFO:teuthology.orchestra.run.smithi132.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-28T00:13:32.381 INFO:teuthology.orchestra.run.smithi132.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-28T00:13:32.381 INFO:teuthology.orchestra.run.smithi132.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-28T00:13:32.381 INFO:teuthology.orchestra.run.smithi132.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-28T00:13:32.381 INFO:teuthology.orchestra.run.smithi132.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-28T00:13:32.382 INFO:teuthology.orchestra.run.smithi132.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-28T00:13:32.382 INFO:teuthology.orchestra.run.smithi132.stdout: jq-1.6-16.el9.x86_64 2024-03-28T00:13:32.382 INFO:teuthology.orchestra.run.smithi132.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-28T00:13:32.382 INFO:teuthology.orchestra.run.smithi132.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-28T00:13:32.382 INFO:teuthology.orchestra.run.smithi132.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-28T00:13:32.382 INFO:teuthology.orchestra.run.smithi132.stdout: libcephfs-devel-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T00:13:32.382 INFO:teuthology.orchestra.run.smithi132.stdout: libcephfs2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T00:13:32.382 INFO:teuthology.orchestra.run.smithi132.stdout: libcephsqlite-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T00:13:32.382 INFO:teuthology.orchestra.run.smithi132.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-28T00:13:32.382 INFO:teuthology.orchestra.run.smithi132.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-28T00:13:32.382 INFO:teuthology.orchestra.run.smithi132.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-28T00:13:32.382 INFO:teuthology.orchestra.run.smithi132.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-28T00:13:32.382 INFO:teuthology.orchestra.run.smithi132.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-28T00:13:32.382 INFO:teuthology.orchestra.run.smithi132.stdout: librados-devel-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T00:13:32.383 INFO:teuthology.orchestra.run.smithi132.stdout: libradosstriper1-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T00:13:32.383 INFO:teuthology.orchestra.run.smithi132.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-28T00:13:32.383 INFO:teuthology.orchestra.run.smithi132.stdout: librgw2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T00:13:32.383 INFO:teuthology.orchestra.run.smithi132.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-28T00:13:32.383 INFO:teuthology.orchestra.run.smithi132.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-28T00:13:32.383 INFO:teuthology.orchestra.run.smithi132.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-28T00:13:32.383 INFO:teuthology.orchestra.run.smithi132.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-28T00:13:32.383 INFO:teuthology.orchestra.run.smithi132.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-28T00:13:32.383 INFO:teuthology.orchestra.run.smithi132.stdout: lua-5.4.4-4.el9.x86_64 2024-03-28T00:13:32.383 INFO:teuthology.orchestra.run.smithi132.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-28T00:13:32.383 INFO:teuthology.orchestra.run.smithi132.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-28T00:13:32.383 INFO:teuthology.orchestra.run.smithi132.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-28T00:13:32.383 INFO:teuthology.orchestra.run.smithi132.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-28T00:13:32.383 INFO:teuthology.orchestra.run.smithi132.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-28T00:13:32.384 INFO:teuthology.orchestra.run.smithi132.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-28T00:13:32.384 INFO:teuthology.orchestra.run.smithi132.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-28T00:13:32.384 INFO:teuthology.orchestra.run.smithi132.stdout: postgresql-13.14-1.el9.x86_64 2024-03-28T00:13:32.384 INFO:teuthology.orchestra.run.smithi132.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-03-28T00:13:32.384 INFO:teuthology.orchestra.run.smithi132.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-03-28T00:13:32.384 INFO:teuthology.orchestra.run.smithi132.stdout: postgresql-server-13.14-1.el9.x86_64 2024-03-28T00:13:32.384 INFO:teuthology.orchestra.run.smithi132.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-28T00:13:32.384 INFO:teuthology.orchestra.run.smithi132.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-28T00:13:32.384 INFO:teuthology.orchestra.run.smithi132.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-03-28T00:13:32.384 INFO:teuthology.orchestra.run.smithi132.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-28T00:13:32.384 INFO:teuthology.orchestra.run.smithi132.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-28T00:13:32.384 INFO:teuthology.orchestra.run.smithi132.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-28T00:13:32.384 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-28T00:13:32.385 INFO:teuthology.orchestra.run.smithi132.stdout: python3-ceph-argparse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T00:13:32.385 INFO:teuthology.orchestra.run.smithi132.stdout: python3-ceph-common-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T00:13:32.385 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cephfs-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T00:13:32.385 INFO:teuthology.orchestra.run.smithi132.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-28T00:13:32.385 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-28T00:13:32.385 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-28T00:13:32.385 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-28T00:13:32.385 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-03-28T00:13:32.385 INFO:teuthology.orchestra.run.smithi132.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-03-28T00:13:32.385 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-28T00:13:32.385 INFO:teuthology.orchestra.run.smithi132.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-28T00:13:32.385 INFO:teuthology.orchestra.run.smithi132.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-03-28T00:13:32.385 INFO:teuthology.orchestra.run.smithi132.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-28T00:13:32.385 INFO:teuthology.orchestra.run.smithi132.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-28T00:13:32.386 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-28T00:13:32.386 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-28T00:13:32.386 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-28T00:13:32.386 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-28T00:13:32.386 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-28T00:13:32.386 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-28T00:13:32.386 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-28T00:13:32.386 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-28T00:13:32.386 INFO:teuthology.orchestra.run.smithi132.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-28T00:13:32.386 INFO:teuthology.orchestra.run.smithi132.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-28T00:13:32.386 INFO:teuthology.orchestra.run.smithi132.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-28T00:13:32.386 INFO:teuthology.orchestra.run.smithi132.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-28T00:13:32.386 INFO:teuthology.orchestra.run.smithi132.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-28T00:13:32.386 INFO:teuthology.orchestra.run.smithi132.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-28T00:13:32.387 INFO:teuthology.orchestra.run.smithi132.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-28T00:13:32.387 INFO:teuthology.orchestra.run.smithi132.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-28T00:13:32.387 INFO:teuthology.orchestra.run.smithi132.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-28T00:13:32.387 INFO:teuthology.orchestra.run.smithi132.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-28T00:13:32.387 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-28T00:13:32.387 INFO:teuthology.orchestra.run.smithi132.stdout: python3-ply-3.11-14.el9.noarch 2024-03-28T00:13:32.387 INFO:teuthology.orchestra.run.smithi132.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-28T00:13:32.387 INFO:teuthology.orchestra.run.smithi132.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-28T00:13:32.387 INFO:teuthology.orchestra.run.smithi132.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-28T00:13:32.387 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-28T00:13:32.387 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-28T00:13:32.387 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-28T00:13:32.387 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-28T00:13:32.387 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-03-28T00:13:32.388 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-28T00:13:32.388 INFO:teuthology.orchestra.run.smithi132.stdout: python3-rados-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T00:13:32.388 INFO:teuthology.orchestra.run.smithi132.stdout: python3-rbd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T00:13:32.388 INFO:teuthology.orchestra.run.smithi132.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-28T00:13:32.388 INFO:teuthology.orchestra.run.smithi132.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-28T00:13:32.388 INFO:teuthology.orchestra.run.smithi132.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-28T00:13:32.388 INFO:teuthology.orchestra.run.smithi132.stdout: python3-rgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T00:13:32.388 INFO:teuthology.orchestra.run.smithi132.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-28T00:13:32.388 INFO:teuthology.orchestra.run.smithi132.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-28T00:13:32.388 INFO:teuthology.orchestra.run.smithi132.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-28T00:13:32.388 INFO:teuthology.orchestra.run.smithi132.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-28T00:13:32.388 INFO:teuthology.orchestra.run.smithi132.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-28T00:13:32.388 INFO:teuthology.orchestra.run.smithi132.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-28T00:13:32.388 INFO:teuthology.orchestra.run.smithi132.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-28T00:13:32.388 INFO:teuthology.orchestra.run.smithi132.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-03-28T00:13:32.389 INFO:teuthology.orchestra.run.smithi132.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-28T00:13:32.389 INFO:teuthology.orchestra.run.smithi132.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-28T00:13:32.389 INFO:teuthology.orchestra.run.smithi132.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-28T00:13:32.389 INFO:teuthology.orchestra.run.smithi132.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-28T00:13:32.389 INFO:teuthology.orchestra.run.smithi132.stdout: qatlib-24.02.0-1.el9.x86_64 2024-03-28T00:13:32.389 INFO:teuthology.orchestra.run.smithi132.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-03-28T00:13:32.389 INFO:teuthology.orchestra.run.smithi132.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-03-28T00:13:32.389 INFO:teuthology.orchestra.run.smithi132.stdout: rbd-fuse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T00:13:32.389 INFO:teuthology.orchestra.run.smithi132.stdout: rbd-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T00:13:32.389 INFO:teuthology.orchestra.run.smithi132.stdout: rbd-nbd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T00:13:32.389 INFO:teuthology.orchestra.run.smithi132.stdout: re2-1:20211101-3.el9.x86_64 2024-03-28T00:13:32.389 INFO:teuthology.orchestra.run.smithi132.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-28T00:13:32.389 INFO:teuthology.orchestra.run.smithi132.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-28T00:13:32.389 INFO:teuthology.orchestra.run.smithi132.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-28T00:13:32.390 INFO:teuthology.orchestra.run.smithi132.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-28T00:13:32.390 INFO:teuthology.orchestra.run.smithi132.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-28T00:13:32.390 INFO:teuthology.orchestra.run.smithi132.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-28T00:13:32.390 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-28T00:13:32.390 INFO:teuthology.orchestra.run.smithi132.stdout:Complete! 2024-03-28T00:13:32.564 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-osd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 134/198 2024-03-28T00:13:32.589 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-osd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 134/198 2024-03-28T00:13:32.589 INFO:teuthology.orchestra.run.smithi165.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-28T00:13:32.589 INFO:teuthology.orchestra.run.smithi165.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-28T00:13:32.590 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-28T00:13:32.590 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-28T00:13:32.590 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-03-28T00:13:32.815 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-mds-2:19.0.0-2145.g3be9a5c9.el9.x86_64 135/198 2024-03-28T00:13:32.837 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-mds-2:19.0.0-2145.g3be9a5c9.el9.x86_64 135/198 2024-03-28T00:13:32.837 INFO:teuthology.orchestra.run.smithi165.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-28T00:13:32.837 INFO:teuthology.orchestra.run.smithi165.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-28T00:13:32.838 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-28T00:13:32.838 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-28T00:13:32.838 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-03-28T00:13:32.853 DEBUG:teuthology.parallel:result is None 2024-03-28T00:13:33.274 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-mon-2:19.0.0-2145.g3be9a5c9.el9.x86_64 136/198 2024-03-28T00:13:33.295 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-mon-2:19.0.0-2145.g3be9a5c9.el9.x86_64 136/198 2024-03-28T00:13:33.296 INFO:teuthology.orchestra.run.smithi165.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-28T00:13:33.296 INFO:teuthology.orchestra.run.smithi165.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-28T00:13:33.296 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-28T00:13:33.296 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-28T00:13:33.296 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-03-28T00:13:33.392 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-03-28T00:13:33.718 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: cephadm-2:19.0.0-2145.g3be9a5c9.el9.noarch 138/198 2024-03-28T00:13:33.749 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : cephadm-2:19.0.0-2145.g3be9a5c9.el9.noarch 138/198 2024-03-28T00:13:33.781 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2145.g3be9a5c9.e 139/198 2024-03-28T00:13:33.842 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2145.g3be9a5c9. 140/198 2024-03-28T00:13:33.872 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2145.g3be9a5c9.el9.noa 141/198 2024-03-28T00:13:34.107 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2145.g3be9a5c9.el9.noa 141/198 2024-03-28T00:13:34.140 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2145.g3be9a5c9.el9.n 142/198 2024-03-28T00:13:35.101 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2145.g3be9a5c9.el9.n 142/198 2024-03-28T00:13:35.132 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2145.g3be 143/198 2024-03-28T00:13:35.233 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2145.g3be 143/198 2024-03-28T00:13:35.395 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2145.g3be9a5c9.el 144/198 2024-03-28T00:13:35.434 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-mgr-2:19.0.0-2145.g3be9a5c9.el9.x86_64 145/198 2024-03-28T00:13:35.457 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2145.g3be9a5c9.el9.x86_64 145/198 2024-03-28T00:13:35.457 INFO:teuthology.orchestra.run.smithi165.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-28T00:13:35.457 INFO:teuthology.orchestra.run.smithi165.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-28T00:13:35.458 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-28T00:13:35.458 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-28T00:13:35.458 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-03-28T00:13:35.502 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-mgr-rook-2:19.0.0-2145.g3be9a5c9.el9.noarch 146/198 2024-03-28T00:13:35.513 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2145.g3be9a5c9.el9.noarch 146/198 2024-03-28T00:13:35.558 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-2:19.0.0-2145.g3be9a5c9.el9.x86_64 147/198 2024-03-28T00:13:36.436 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-03-28T00:13:36.462 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-radosgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 149/198 2024-03-28T00:13:36.485 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 149/198 2024-03-28T00:13:36.485 INFO:teuthology.orchestra.run.smithi165.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-28T00:13:36.485 INFO:teuthology.orchestra.run.smithi165.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-28T00:13:36.485 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-28T00:13:36.485 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-28T00:13:36.485 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-03-28T00:13:36.530 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2145.g3be9a 150/198 2024-03-28T00:13:36.552 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2145.g3be9a 150/198 2024-03-28T00:13:36.552 INFO:teuthology.orchestra.run.smithi165.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-28T00:13:36.552 INFO:teuthology.orchestra.run.smithi165.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-28T00:13:36.552 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-03-28T00:13:36.597 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : cephfs-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 151/198 2024-03-28T00:13:36.619 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 151/198 2024-03-28T00:13:36.619 INFO:teuthology.orchestra.run.smithi165.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-28T00:13:36.619 INFO:teuthology.orchestra.run.smithi165.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-28T00:13:36.619 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-28T00:13:36.619 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-28T00:13:36.619 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-03-28T00:13:36.931 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : rbd-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 152/198 2024-03-28T00:13:36.954 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 152/198 2024-03-28T00:13:36.954 INFO:teuthology.orchestra.run.smithi165.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-28T00:13:36.954 INFO:teuthology.orchestra.run.smithi165.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-28T00:13:36.954 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-28T00:13:36.955 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-28T00:13:36.955 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-03-28T00:13:41.378 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-test-2:19.0.0-2145.g3be9a5c9.el9.x86_64 153/198 2024-03-28T00:13:41.418 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : rbd-fuse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 154/198 2024-03-28T00:13:41.449 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : rbd-nbd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 155/198 2024-03-28T00:13:41.476 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-03-28T00:13:41.655 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : cephfs-shell-2:19.0.0-2145.g3be9a5c9.el9.x86_64 157/198 2024-03-28T00:13:41.776 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 158/198 2024-03-28T00:13:41.816 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-fuse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 159/198 2024-03-28T00:13:41.857 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-03-28T00:13:41.916 INFO:teuthology.orchestra.run.smithi165.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-03-28T00:13:41.960 INFO:teuthology.orchestra.run.smithi165.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-03-28T00:13:41.984 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libcephfs-devel-2:19.0.0-2145.g3be9a5c9.el9.x86_ 163/198 2024-03-28T00:13:42.025 INFO:teuthology.orchestra.run.smithi165.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-03-28T00:13:42.054 INFO:teuthology.orchestra.run.smithi165.stdout: Upgrading : NetworkManager-team-1:1.47.2-1.el9.x86_64 165/198 2024-03-28T00:13:42.108 INFO:teuthology.orchestra.run.smithi165.stdout: Upgrading : NetworkManager-tui-1:1.47.2-1.el9.x86_64 166/198 2024-03-28T00:13:42.169 INFO:teuthology.orchestra.run.smithi165.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-03-28T00:13:42.212 INFO:teuthology.orchestra.run.smithi165.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-03-28T00:13:42.790 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-03-28T00:13:42.827 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 169/198 2024-03-28T00:13:42.846 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-03-28T00:13:43.663 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : cephfs-top-2:19.0.0-2145.g3be9a5c9.el9.noarch 170/198 2024-03-28T00:13:43.739 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-03-28T00:13:43.771 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-03-28T00:13:43.861 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-03-28T00:13:43.910 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-03-28T00:13:43.935 INFO:teuthology.orchestra.run.smithi165.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-03-28T00:13:43.960 INFO:teuthology.orchestra.run.smithi165.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-03-28T00:13:43.986 INFO:teuthology.orchestra.run.smithi165.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-03-28T00:13:43.986 INFO:teuthology.orchestra.run.smithi165.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-28T00:13:44.075 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-28T00:13:44.104 INFO:teuthology.orchestra.run.smithi165.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-03-28T00:13:44.104 INFO:teuthology.orchestra.run.smithi165.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-28T00:13:44.189 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-28T00:13:44.215 INFO:teuthology.orchestra.run.smithi165.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-03-28T00:13:44.257 INFO:teuthology.orchestra.run.smithi165.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-03-28T00:13:44.282 INFO:teuthology.orchestra.run.smithi165.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-03-28T00:13:44.307 INFO:teuthology.orchestra.run.smithi165.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-03-28T00:13:44.340 INFO:teuthology.orchestra.run.smithi165.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-03-28T00:13:44.388 INFO:teuthology.orchestra.run.smithi165.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-03-28T00:13:44.421 INFO:teuthology.orchestra.run.smithi165.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-03-28T00:13:44.446 INFO:teuthology.orchestra.run.smithi165.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-03-28T00:13:44.472 INFO:teuthology.orchestra.run.smithi165.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-03-28T00:13:44.497 INFO:teuthology.orchestra.run.smithi165.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-03-28T00:13:44.522 INFO:teuthology.orchestra.run.smithi165.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-03-28T00:13:44.528 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-28T00:13:44.528 INFO:teuthology.orchestra.run.smithi165.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-28T00:13:44.716 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-28T00:13:44.717 INFO:teuthology.orchestra.run.smithi165.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-28T00:13:44.799 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-28T00:13:44.824 INFO:teuthology.orchestra.run.smithi165.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-03-28T00:13:44.856 INFO:teuthology.orchestra.run.smithi165.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-03-28T00:13:44.884 INFO:teuthology.orchestra.run.smithi165.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-03-28T00:13:44.909 INFO:teuthology.orchestra.run.smithi165.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-03-28T00:13:44.973 INFO:teuthology.orchestra.run.smithi165.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-28T00:13:49.712 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-28T00:13:49.712 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-2:19.0.0-2145.g3be9a5c9.el9.x86_64 1/198 2024-03-28T00:13:49.712 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-base-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2/198 2024-03-28T00:13:49.712 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-common-2:19.0.0-2145.g3be9a5c9.el9.x86_64 3/198 2024-03-28T00:13:49.712 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-fuse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 4/198 2024-03-28T00:13:49.712 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2145.g3be9a 5/198 2024-03-28T00:13:49.712 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-mds-2:19.0.0-2145.g3be9a5c9.el9.x86_64 6/198 2024-03-28T00:13:49.712 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-mgr-2:19.0.0-2145.g3be9a5c9.el9.x86_64 7/198 2024-03-28T00:13:49.712 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-mon-2:19.0.0-2145.g3be9a5c9.el9.x86_64 8/198 2024-03-28T00:13:49.712 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-osd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 9/198 2024-03-28T00:13:49.713 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-radosgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 10/198 2024-03-28T00:13:49.713 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-selinux-2:19.0.0-2145.g3be9a5c9.el9.x86_64 11/198 2024-03-28T00:13:49.713 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-test-2:19.0.0-2145.g3be9a5c9.el9.x86_64 12/198 2024-03-28T00:13:49.713 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : cephfs-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 13/198 2024-03-28T00:13:49.713 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : cephfs-shell-2:19.0.0-2145.g3be9a5c9.el9.x86_64 14/198 2024-03-28T00:13:49.713 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libcephfs-devel-2:19.0.0-2145.g3be9a5c9.el9.x86_ 15/198 2024-03-28T00:13:49.713 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libcephfs2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 16/198 2024-03-28T00:13:49.713 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libcephsqlite-2:19.0.0-2145.g3be9a5c9.el9.x86_64 17/198 2024-03-28T00:13:49.713 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : librados-devel-2:19.0.0-2145.g3be9a5c9.el9.x86_6 18/198 2024-03-28T00:13:49.713 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libradosstriper1-2:19.0.0-2145.g3be9a5c9.el9.x86 19/198 2024-03-28T00:13:49.713 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : librgw2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 20/198 2024-03-28T00:13:49.713 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2145.g3be9a5c9.el 21/198 2024-03-28T00:13:49.713 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-ceph-common-2:19.0.0-2145.g3be9a5c9.el9. 22/198 2024-03-28T00:13:49.714 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-cephfs-2:19.0.0-2145.g3be9a5c9.el9.x86_6 23/198 2024-03-28T00:13:49.714 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-rados-2:19.0.0-2145.g3be9a5c9.el9.x86_64 24/198 2024-03-28T00:13:49.714 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-rbd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 25/198 2024-03-28T00:13:49.714 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-rgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 26/198 2024-03-28T00:13:49.714 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : rbd-fuse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 27/198 2024-03-28T00:13:49.714 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : rbd-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 28/198 2024-03-28T00:13:49.714 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : rbd-nbd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 29/198 2024-03-28T00:13:49.714 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2145.g3be9a5c9. 30/198 2024-03-28T00:13:49.714 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2145.g3be9a5c9.el9.noa 31/198 2024-03-28T00:13:49.714 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2145.g3be9a5c9.el9.n 32/198 2024-03-28T00:13:49.714 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2145.g3be 33/198 2024-03-28T00:13:49.714 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2145.g3be9a5c9.el 34/198 2024-03-28T00:13:49.715 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2145.g3be9a5c9.el9.noarch 35/198 2024-03-28T00:13:49.715 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2145.g3be9a5c9.e 36/198 2024-03-28T00:13:49.715 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-volume-2:19.0.0-2145.g3be9a5c9.el9.noarch 37/198 2024-03-28T00:13:49.715 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : cephadm-2:19.0.0-2145.g3be9a5c9.el9.noarch 38/198 2024-03-28T00:13:49.715 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : cephfs-top-2:19.0.0-2145.g3be9a5c9.el9.noarch 39/198 2024-03-28T00:13:49.715 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-03-28T00:13:49.715 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-03-28T00:13:49.715 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-03-28T00:13:49.715 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-03-28T00:13:49.715 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-03-28T00:13:49.715 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-03-28T00:13:49.715 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-03-28T00:13:49.715 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-03-28T00:13:49.716 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-03-28T00:13:49.716 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-03-28T00:13:49.716 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-03-28T00:13:49.716 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-03-28T00:13:49.716 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-03-28T00:13:49.716 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-03-28T00:13:49.716 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-03-28T00:13:49.716 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-03-28T00:13:49.716 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-03-28T00:13:49.716 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-03-28T00:13:49.716 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 58/198 2024-03-28T00:13:49.716 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 59/198 2024-03-28T00:13:49.716 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 60/198 2024-03-28T00:13:49.717 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 61/198 2024-03-28T00:13:49.717 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 62/198 2024-03-28T00:13:49.717 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 63/198 2024-03-28T00:13:49.717 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 64/198 2024-03-28T00:13:49.717 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 65/198 2024-03-28T00:13:49.717 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 66/198 2024-03-28T00:13:49.717 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 67/198 2024-03-28T00:13:49.717 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 68/198 2024-03-28T00:13:49.717 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 69/198 2024-03-28T00:13:49.717 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 70/198 2024-03-28T00:13:49.717 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 71/198 2024-03-28T00:13:49.718 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 72/198 2024-03-28T00:13:49.718 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 73/198 2024-03-28T00:13:49.718 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 74/198 2024-03-28T00:13:49.718 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 75/198 2024-03-28T00:13:49.718 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 76/198 2024-03-28T00:13:49.718 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 77/198 2024-03-28T00:13:49.718 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 78/198 2024-03-28T00:13:49.718 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 79/198 2024-03-28T00:13:49.718 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 80/198 2024-03-28T00:13:49.718 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 81/198 2024-03-28T00:13:49.718 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 82/198 2024-03-28T00:13:49.718 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 83/198 2024-03-28T00:13:49.718 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 84/198 2024-03-28T00:13:49.719 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-ply-3.11-14.el9.noarch 85/198 2024-03-28T00:13:49.719 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 86/198 2024-03-28T00:13:49.719 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 87/198 2024-03-28T00:13:49.719 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : jq-1.6-16.el9.x86_64 88/198 2024-03-28T00:13:49.719 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 89/198 2024-03-28T00:13:49.719 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 90/198 2024-03-28T00:13:49.719 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 91/198 2024-03-28T00:13:49.719 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 92/198 2024-03-28T00:13:49.719 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 93/198 2024-03-28T00:13:49.719 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 94/198 2024-03-28T00:13:49.719 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : postgresql-13.14-1.el9.x86_64 95/198 2024-03-28T00:13:49.719 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 96/198 2024-03-28T00:13:49.719 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 97/198 2024-03-28T00:13:49.719 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 98/198 2024-03-28T00:13:49.719 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 99/198 2024-03-28T00:13:49.720 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 100/198 2024-03-28T00:13:49.720 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 101/198 2024-03-28T00:13:49.720 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 102/198 2024-03-28T00:13:49.720 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 103/198 2024-03-28T00:13:49.720 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-03-28T00:13:49.720 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-03-28T00:13:49.720 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-03-28T00:13:49.720 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-03-28T00:13:49.720 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-03-28T00:13:49.720 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-03-28T00:13:49.720 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-03-28T00:13:49.720 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-03-28T00:13:49.720 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-03-28T00:13:49.721 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-03-28T00:13:49.721 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-03-28T00:13:49.721 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-03-28T00:13:49.721 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/198 2024-03-28T00:13:49.721 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-03-28T00:13:49.721 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-03-28T00:13:49.721 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-03-28T00:13:49.721 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-03-28T00:13:49.721 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-03-28T00:13:49.721 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-03-28T00:13:49.721 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-03-28T00:13:49.721 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 124/198 2024-03-28T00:13:49.721 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-03-28T00:13:49.721 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-03-28T00:13:49.722 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-03-28T00:13:49.722 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-03-28T00:13:49.722 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-03-28T00:13:49.722 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-03-28T00:13:49.722 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-03-28T00:13:49.722 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-03-28T00:13:49.722 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-03-28T00:13:49.722 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-03-28T00:13:49.722 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-03-28T00:13:49.722 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-03-28T00:13:49.722 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-03-28T00:13:49.722 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-03-28T00:13:49.722 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-03-28T00:13:49.723 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-03-28T00:13:49.723 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-03-28T00:13:49.723 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-03-28T00:13:49.723 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-03-28T00:13:49.723 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-03-28T00:13:49.723 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-03-28T00:13:49.723 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-03-28T00:13:49.723 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-03-28T00:13:49.723 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-03-28T00:13:49.723 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-03-28T00:13:49.723 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-03-28T00:13:49.723 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : librados2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 151/198 2024-03-28T00:13:49.723 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-03-28T00:13:49.724 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : librbd1-2:19.0.0-2145.g3be9a5c9.el9.x86_64 153/198 2024-03-28T00:13:49.724 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-03-28T00:13:49.724 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 155/198 2024-03-28T00:13:49.724 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 156/198 2024-03-28T00:13:49.724 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : NetworkManager-1:1.47.2-1.el9.x86_64 157/198 2024-03-28T00:13:49.724 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 158/198 2024-03-28T00:13:49.724 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 159/198 2024-03-28T00:13:49.724 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/198 2024-03-28T00:13:49.724 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : NetworkManager-team-1:1.47.2-1.el9.x86_64 161/198 2024-03-28T00:13:49.724 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 162/198 2024-03-28T00:13:49.724 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : NetworkManager-tui-1:1.47.2-1.el9.x86_64 163/198 2024-03-28T00:13:49.724 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 164/198 2024-03-28T00:13:49.724 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-03-28T00:13:49.725 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-03-28T00:13:49.725 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-03-28T00:13:49.725 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-03-28T00:13:49.725 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-03-28T00:13:49.725 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-03-28T00:13:49.725 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-03-28T00:13:49.725 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-03-28T00:13:49.725 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-03-28T00:13:49.725 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-03-28T00:13:49.725 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-03-28T00:13:49.725 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-03-28T00:13:49.725 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-03-28T00:13:49.725 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-03-28T00:13:49.726 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-03-28T00:13:49.726 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-03-28T00:13:49.726 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-03-28T00:13:49.726 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-03-28T00:13:49.726 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-03-28T00:13:49.726 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-03-28T00:13:49.726 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-03-28T00:13:49.726 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-03-28T00:13:49.726 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-03-28T00:13:49.726 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-03-28T00:13:49.726 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-03-28T00:13:49.726 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-03-28T00:13:49.726 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-03-28T00:13:49.727 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-03-28T00:13:49.727 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-03-28T00:13:49.727 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-03-28T00:13:49.727 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-03-28T00:13:49.727 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-03-28T00:13:49.727 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-03-28T00:13:52.723 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-03-28T00:13:52.724 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-03-28T00:13:52.724 INFO:teuthology.orchestra.run.smithi165.stdout:Upgraded: 2024-03-28T00:13:52.724 INFO:teuthology.orchestra.run.smithi165.stdout: NetworkManager-1:1.47.2-1.el9.x86_64 2024-03-28T00:13:52.724 INFO:teuthology.orchestra.run.smithi165.stdout: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 2024-03-28T00:13:52.724 INFO:teuthology.orchestra.run.smithi165.stdout: NetworkManager-team-1:1.47.2-1.el9.x86_64 2024-03-28T00:13:52.724 INFO:teuthology.orchestra.run.smithi165.stdout: NetworkManager-tui-1:1.47.2-1.el9.x86_64 2024-03-28T00:13:52.724 INFO:teuthology.orchestra.run.smithi165.stdout: acl-2.3.1-4.el9.x86_64 2024-03-28T00:13:52.724 INFO:teuthology.orchestra.run.smithi165.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-28T00:13:52.724 INFO:teuthology.orchestra.run.smithi165.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-28T00:13:52.724 INFO:teuthology.orchestra.run.smithi165.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-28T00:13:52.724 INFO:teuthology.orchestra.run.smithi165.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-28T00:13:52.725 INFO:teuthology.orchestra.run.smithi165.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-28T00:13:52.725 INFO:teuthology.orchestra.run.smithi165.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-28T00:13:52.725 INFO:teuthology.orchestra.run.smithi165.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-28T00:13:52.725 INFO:teuthology.orchestra.run.smithi165.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-28T00:13:52.725 INFO:teuthology.orchestra.run.smithi165.stdout: librados2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T00:13:52.725 INFO:teuthology.orchestra.run.smithi165.stdout: librbd1-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T00:13:52.725 INFO:teuthology.orchestra.run.smithi165.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-28T00:13:52.725 INFO:teuthology.orchestra.run.smithi165.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-28T00:13:52.725 INFO:teuthology.orchestra.run.smithi165.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-28T00:13:52.725 INFO:teuthology.orchestra.run.smithi165.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-28T00:13:52.725 INFO:teuthology.orchestra.run.smithi165.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-28T00:13:52.725 INFO:teuthology.orchestra.run.smithi165.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-28T00:13:52.725 INFO:teuthology.orchestra.run.smithi165.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-28T00:13:52.725 INFO:teuthology.orchestra.run.smithi165.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-28T00:13:52.726 INFO:teuthology.orchestra.run.smithi165.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-28T00:13:52.726 INFO:teuthology.orchestra.run.smithi165.stdout:Installed: 2024-03-28T00:13:52.726 INFO:teuthology.orchestra.run.smithi165.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-28T00:13:52.726 INFO:teuthology.orchestra.run.smithi165.stdout: automake-1.16.2-8.el9.noarch 2024-03-28T00:13:52.726 INFO:teuthology.orchestra.run.smithi165.stdout: bison-3.7.4-5.el9.x86_64 2024-03-28T00:13:52.726 INFO:teuthology.orchestra.run.smithi165.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-28T00:13:52.726 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T00:13:52.726 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-base-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T00:13:52.726 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-common-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T00:13:52.726 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-fuse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T00:13:52.726 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-grafana-dashboards-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-28T00:13:52.726 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-immutable-object-cache-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T00:13:52.726 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mds-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T00:13:52.727 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T00:13:52.727 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr-cephadm-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-28T00:13:52.727 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr-dashboard-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-28T00:13:52.727 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-28T00:13:52.727 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr-modules-core-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-28T00:13:52.727 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr-rook-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-28T00:13:52.727 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mon-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T00:13:52.727 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-osd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T00:13:52.727 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-prometheus-alerts-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-28T00:13:52.727 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-radosgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T00:13:52.727 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-selinux-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T00:13:52.727 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-test-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T00:13:52.727 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-volume-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-28T00:13:52.727 INFO:teuthology.orchestra.run.smithi165.stdout: cephadm-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-28T00:13:52.728 INFO:teuthology.orchestra.run.smithi165.stdout: cephfs-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T00:13:52.728 INFO:teuthology.orchestra.run.smithi165.stdout: cephfs-shell-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T00:13:52.728 INFO:teuthology.orchestra.run.smithi165.stdout: cephfs-top-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-28T00:13:52.728 INFO:teuthology.orchestra.run.smithi165.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-28T00:13:52.728 INFO:teuthology.orchestra.run.smithi165.stdout: flex-2.6.4-9.el9.x86_64 2024-03-28T00:13:52.728 INFO:teuthology.orchestra.run.smithi165.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-28T00:13:52.728 INFO:teuthology.orchestra.run.smithi165.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-28T00:13:52.728 INFO:teuthology.orchestra.run.smithi165.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-28T00:13:52.728 INFO:teuthology.orchestra.run.smithi165.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-28T00:13:52.728 INFO:teuthology.orchestra.run.smithi165.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-28T00:13:52.728 INFO:teuthology.orchestra.run.smithi165.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-28T00:13:52.728 INFO:teuthology.orchestra.run.smithi165.stdout: jq-1.6-16.el9.x86_64 2024-03-28T00:13:52.728 INFO:teuthology.orchestra.run.smithi165.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-28T00:13:52.729 INFO:teuthology.orchestra.run.smithi165.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-28T00:13:52.729 INFO:teuthology.orchestra.run.smithi165.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-28T00:13:52.729 INFO:teuthology.orchestra.run.smithi165.stdout: libcephfs-devel-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T00:13:52.729 INFO:teuthology.orchestra.run.smithi165.stdout: libcephfs2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T00:13:52.729 INFO:teuthology.orchestra.run.smithi165.stdout: libcephsqlite-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T00:13:52.729 INFO:teuthology.orchestra.run.smithi165.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-28T00:13:52.729 INFO:teuthology.orchestra.run.smithi165.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-28T00:13:52.729 INFO:teuthology.orchestra.run.smithi165.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-28T00:13:52.729 INFO:teuthology.orchestra.run.smithi165.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-28T00:13:52.729 INFO:teuthology.orchestra.run.smithi165.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-28T00:13:52.729 INFO:teuthology.orchestra.run.smithi165.stdout: librados-devel-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T00:13:52.729 INFO:teuthology.orchestra.run.smithi165.stdout: libradosstriper1-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T00:13:52.729 INFO:teuthology.orchestra.run.smithi165.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-28T00:13:52.729 INFO:teuthology.orchestra.run.smithi165.stdout: librgw2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T00:13:52.730 INFO:teuthology.orchestra.run.smithi165.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-28T00:13:52.730 INFO:teuthology.orchestra.run.smithi165.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-28T00:13:52.730 INFO:teuthology.orchestra.run.smithi165.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-28T00:13:52.730 INFO:teuthology.orchestra.run.smithi165.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-28T00:13:52.730 INFO:teuthology.orchestra.run.smithi165.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-28T00:13:52.730 INFO:teuthology.orchestra.run.smithi165.stdout: lua-5.4.4-4.el9.x86_64 2024-03-28T00:13:52.730 INFO:teuthology.orchestra.run.smithi165.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-28T00:13:52.730 INFO:teuthology.orchestra.run.smithi165.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-28T00:13:52.730 INFO:teuthology.orchestra.run.smithi165.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-28T00:13:52.730 INFO:teuthology.orchestra.run.smithi165.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-28T00:13:52.730 INFO:teuthology.orchestra.run.smithi165.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-28T00:13:52.730 INFO:teuthology.orchestra.run.smithi165.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-28T00:13:52.730 INFO:teuthology.orchestra.run.smithi165.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-28T00:13:52.730 INFO:teuthology.orchestra.run.smithi165.stdout: postgresql-13.14-1.el9.x86_64 2024-03-28T00:13:52.731 INFO:teuthology.orchestra.run.smithi165.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-03-28T00:13:52.731 INFO:teuthology.orchestra.run.smithi165.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-03-28T00:13:52.731 INFO:teuthology.orchestra.run.smithi165.stdout: postgresql-server-13.14-1.el9.x86_64 2024-03-28T00:13:52.731 INFO:teuthology.orchestra.run.smithi165.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-28T00:13:52.731 INFO:teuthology.orchestra.run.smithi165.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-28T00:13:52.731 INFO:teuthology.orchestra.run.smithi165.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-03-28T00:13:52.731 INFO:teuthology.orchestra.run.smithi165.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-28T00:13:52.731 INFO:teuthology.orchestra.run.smithi165.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-28T00:13:52.731 INFO:teuthology.orchestra.run.smithi165.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-28T00:13:52.731 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-28T00:13:52.731 INFO:teuthology.orchestra.run.smithi165.stdout: python3-ceph-argparse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T00:13:52.731 INFO:teuthology.orchestra.run.smithi165.stdout: python3-ceph-common-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T00:13:52.731 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cephfs-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T00:13:52.731 INFO:teuthology.orchestra.run.smithi165.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-28T00:13:52.732 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-28T00:13:52.732 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-28T00:13:52.732 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-28T00:13:52.732 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-03-28T00:13:52.732 INFO:teuthology.orchestra.run.smithi165.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-03-28T00:13:52.732 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-28T00:13:52.732 INFO:teuthology.orchestra.run.smithi165.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-28T00:13:52.732 INFO:teuthology.orchestra.run.smithi165.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-03-28T00:13:52.732 INFO:teuthology.orchestra.run.smithi165.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-28T00:13:52.732 INFO:teuthology.orchestra.run.smithi165.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-28T00:13:52.732 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-28T00:13:52.732 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-28T00:13:52.732 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-28T00:13:52.733 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-28T00:13:52.733 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-28T00:13:52.733 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-28T00:13:52.733 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-28T00:13:52.733 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-28T00:13:52.733 INFO:teuthology.orchestra.run.smithi165.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-28T00:13:52.733 INFO:teuthology.orchestra.run.smithi165.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-28T00:13:52.733 INFO:teuthology.orchestra.run.smithi165.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-28T00:13:52.733 INFO:teuthology.orchestra.run.smithi165.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-28T00:13:52.733 INFO:teuthology.orchestra.run.smithi165.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-28T00:13:52.733 INFO:teuthology.orchestra.run.smithi165.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-28T00:13:52.733 INFO:teuthology.orchestra.run.smithi165.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-28T00:13:52.733 INFO:teuthology.orchestra.run.smithi165.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-28T00:13:52.733 INFO:teuthology.orchestra.run.smithi165.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-28T00:13:52.733 INFO:teuthology.orchestra.run.smithi165.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-28T00:13:52.734 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-28T00:13:52.734 INFO:teuthology.orchestra.run.smithi165.stdout: python3-ply-3.11-14.el9.noarch 2024-03-28T00:13:52.734 INFO:teuthology.orchestra.run.smithi165.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-28T00:13:52.734 INFO:teuthology.orchestra.run.smithi165.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-28T00:13:52.734 INFO:teuthology.orchestra.run.smithi165.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-28T00:13:52.734 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-28T00:13:52.734 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-28T00:13:52.734 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-28T00:13:52.734 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-28T00:13:52.734 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-03-28T00:13:52.734 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-28T00:13:52.735 INFO:teuthology.orchestra.run.smithi165.stdout: python3-rados-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T00:13:52.735 INFO:teuthology.orchestra.run.smithi165.stdout: python3-rbd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T00:13:52.735 INFO:teuthology.orchestra.run.smithi165.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-28T00:13:52.735 INFO:teuthology.orchestra.run.smithi165.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-28T00:13:52.735 INFO:teuthology.orchestra.run.smithi165.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-28T00:13:52.735 INFO:teuthology.orchestra.run.smithi165.stdout: python3-rgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T00:13:52.735 INFO:teuthology.orchestra.run.smithi165.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-28T00:13:52.735 INFO:teuthology.orchestra.run.smithi165.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-28T00:13:52.735 INFO:teuthology.orchestra.run.smithi165.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-28T00:13:52.735 INFO:teuthology.orchestra.run.smithi165.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-28T00:13:52.735 INFO:teuthology.orchestra.run.smithi165.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-28T00:13:52.735 INFO:teuthology.orchestra.run.smithi165.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-28T00:13:52.735 INFO:teuthology.orchestra.run.smithi165.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-28T00:13:52.735 INFO:teuthology.orchestra.run.smithi165.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-03-28T00:13:52.736 INFO:teuthology.orchestra.run.smithi165.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-28T00:13:52.736 INFO:teuthology.orchestra.run.smithi165.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-28T00:13:52.736 INFO:teuthology.orchestra.run.smithi165.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-28T00:13:52.736 INFO:teuthology.orchestra.run.smithi165.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-28T00:13:52.736 INFO:teuthology.orchestra.run.smithi165.stdout: qatlib-24.02.0-1.el9.x86_64 2024-03-28T00:13:52.736 INFO:teuthology.orchestra.run.smithi165.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-03-28T00:13:52.736 INFO:teuthology.orchestra.run.smithi165.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-03-28T00:13:52.736 INFO:teuthology.orchestra.run.smithi165.stdout: rbd-fuse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T00:13:52.736 INFO:teuthology.orchestra.run.smithi165.stdout: rbd-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T00:13:52.736 INFO:teuthology.orchestra.run.smithi165.stdout: rbd-nbd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T00:13:52.736 INFO:teuthology.orchestra.run.smithi165.stdout: re2-1:20211101-3.el9.x86_64 2024-03-28T00:13:52.736 INFO:teuthology.orchestra.run.smithi165.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-28T00:13:52.736 INFO:teuthology.orchestra.run.smithi165.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-28T00:13:52.736 INFO:teuthology.orchestra.run.smithi165.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-28T00:13:52.736 INFO:teuthology.orchestra.run.smithi165.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-28T00:13:52.737 INFO:teuthology.orchestra.run.smithi165.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-28T00:13:52.737 INFO:teuthology.orchestra.run.smithi165.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-28T00:13:52.737 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-03-28T00:13:52.737 INFO:teuthology.orchestra.run.smithi165.stdout:Complete! 2024-03-28T00:13:53.288 DEBUG:teuthology.parallel:result is None 2024-03-28T00:13:53.289 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3be9a5c9bc793e11e2800a8c0c696e8b46742033 2024-03-28T00:13:53.568 DEBUG:teuthology.orchestra.run.smithi119:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-28T00:13:53.639 INFO:teuthology.orchestra.run.smithi119.stdout:19.0.0-2145.g3be9a5c9.el9 2024-03-28T00:13:53.639 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2145.g3be9a5c9.el9 2024-03-28T00:13:53.639 INFO:teuthology.task.install:The correct ceph version 19.0.0-2145.g3be9a5c9 is installed. 2024-03-28T00:13:53.641 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3be9a5c9bc793e11e2800a8c0c696e8b46742033 2024-03-28T00:13:53.935 DEBUG:teuthology.orchestra.run.smithi132:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-28T00:13:53.960 INFO:teuthology.orchestra.run.smithi132.stdout:19.0.0-2145.g3be9a5c9.el9 2024-03-28T00:13:53.960 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2145.g3be9a5c9.el9 2024-03-28T00:13:53.961 INFO:teuthology.task.install:The correct ceph version 19.0.0-2145.g3be9a5c9 is installed. 2024-03-28T00:13:53.963 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3be9a5c9bc793e11e2800a8c0c696e8b46742033 2024-03-28T00:13:54.272 DEBUG:teuthology.orchestra.run.smithi165:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-28T00:13:54.298 INFO:teuthology.orchestra.run.smithi165.stdout:19.0.0-2145.g3be9a5c9.el9 2024-03-28T00:13:54.298 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2145.g3be9a5c9.el9 2024-03-28T00:13:54.298 INFO:teuthology.task.install:The correct ceph version 19.0.0-2145.g3be9a5c9 is installed. 2024-03-28T00:13:54.300 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-03-28T00:13:54.300 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-03-28T00:13:54.300 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-28T00:13:54.404 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-03-28T00:13:54.404 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-28T00:13:54.439 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-03-28T00:13:54.439 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-28T00:13:54.473 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-03-28T00:13:54.473 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-03-28T00:13:54.473 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/usr/bin/daemon-helper 2024-03-28T00:13:54.574 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-28T00:13:54.713 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-03-28T00:13:54.713 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd of=/usr/bin/daemon-helper 2024-03-28T00:13:54.745 DEBUG:teuthology.orchestra.run.smithi132:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-28T00:13:54.815 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-03-28T00:13:54.815 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd of=/usr/bin/daemon-helper 2024-03-28T00:13:54.849 DEBUG:teuthology.orchestra.run.smithi165:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-28T00:13:54.919 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-03-28T00:13:54.920 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-03-28T00:13:54.920 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-28T00:13:55.019 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-28T00:13:55.157 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-03-28T00:13:55.157 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-28T00:13:55.184 DEBUG:teuthology.orchestra.run.smithi132:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-28T00:13:55.252 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-03-28T00:13:55.252 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-28T00:13:55.284 DEBUG:teuthology.orchestra.run.smithi165:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-28T00:13:55.355 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-03-28T00:13:55.355 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-03-28T00:13:55.355 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/usr/bin/stdin-killer 2024-03-28T00:13:55.449 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-28T00:13:55.590 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-03-28T00:13:55.590 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd of=/usr/bin/stdin-killer 2024-03-28T00:13:55.617 DEBUG:teuthology.orchestra.run.smithi132:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-28T00:13:55.685 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-03-28T00:13:55.685 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd of=/usr/bin/stdin-killer 2024-03-28T00:13:55.717 DEBUG:teuthology.orchestra.run.smithi165:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-28T00:13:55.787 INFO:teuthology.run_tasks:Running task cephadm... 2024-03-28T00:13:55.887 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN'], 'sha1': '3be9a5c9bc793e11e2800a8c0c696e8b46742033', 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-03-28T00:13:55.888 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 2024-03-28T00:13:55.888 INFO:tasks.cephadm:Cluster fsid is 10f73b92-ec98-11ee-95d0-87774f69a715 2024-03-28T00:13:55.888 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-03-28T00:13:55.889 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.119', 'mon.b': '172.21.15.132', 'mon.c': '172.21.15.165'} 2024-03-28T00:13:55.889 INFO:tasks.cephadm:First mon is mon.a on smithi119 2024-03-28T00:13:55.889 INFO:tasks.cephadm:First mgr is x 2024-03-28T00:13:55.889 INFO:tasks.cephadm:Normalizing hostnames... 2024-03-28T00:13:55.889 DEBUG:teuthology.orchestra.run.smithi119:> sudo hostname $(hostname -s) 2024-03-28T00:13:55.989 DEBUG:teuthology.orchestra.run.smithi132:> sudo hostname $(hostname -s) 2024-03-28T00:13:56.017 DEBUG:teuthology.orchestra.run.smithi165:> sudo hostname $(hostname -s) 2024-03-28T00:13:56.047 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-03-28T00:13:56.047 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3be9a5c9bc793e11e2800a8c0c696e8b46742033 2024-03-28T00:13:56.184 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '3be9a5c9bc793e11e2800a8c0c696e8b46742033', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/77877/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-2145-g3be9a5c9', 'node_name': '172.21.5.36+adami06', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-2145.g3be9a5c9'}, 'url': 'https://4.chacra.ceph.com/r/ceph/wip-mchangir-testing-main-20240318.032620/3be9a5c9bc793e11e2800a8c0c696e8b46742033/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-03-22 05:51:42.267384', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-mchangir-testing-main-20240318.032620', 'chacra_url': 'https://4.chacra.ceph.com/repos/ceph/wip-mchangir-testing-main-20240318.032620/3be9a5c9bc793e11e2800a8c0c696e8b46742033/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-03-28T00:13:56.324 INFO:tasks.util.chacra:got chacra host 4.chacra.ceph.com, ref wip-mchangir-testing-main-20240318.032620, sha1 3be9a5c9bc793e11e2800a8c0c696e8b46742033 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=3be9a5c9bc793e11e2800a8c0c696e8b46742033 2024-03-28T00:13:56.326 INFO:tasks.cephadm:Discovered cachra url: https://4.chacra.ceph.com/binaries/ceph/wip-mchangir-testing-main-20240318.032620/3be9a5c9bc793e11e2800a8c0c696e8b46742033/centos/9/x86_64/flavors/default/cephadm 2024-03-28T00:13:56.326 DEBUG:teuthology.orchestra.run.smithi119:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-mchangir-testing-main-20240318.032620/3be9a5c9bc793e11e2800a8c0c696e8b46742033/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-28T00:13:57.032 INFO:teuthology.orchestra.run.smithi119.stdout:-rw-r--r--. 1 ubuntu ubuntu 767371 Mar 28 00:13 /home/ubuntu/cephtest/cephadm 2024-03-28T00:13:57.033 DEBUG:teuthology.orchestra.run.smithi132:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-mchangir-testing-main-20240318.032620/3be9a5c9bc793e11e2800a8c0c696e8b46742033/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-28T00:13:57.694 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r--. 1 ubuntu ubuntu 767371 Mar 28 00:13 /home/ubuntu/cephtest/cephadm 2024-03-28T00:13:57.694 DEBUG:teuthology.orchestra.run.smithi165:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-mchangir-testing-main-20240318.032620/3be9a5c9bc793e11e2800a8c0c696e8b46742033/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-28T00:13:58.160 INFO:teuthology.orchestra.run.smithi165.stdout:-rw-r--r--. 1 ubuntu ubuntu 767371 Mar 28 00:13 /home/ubuntu/cephtest/cephadm 2024-03-28T00:13:58.161 DEBUG:teuthology.orchestra.run.smithi119:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-28T00:13:58.223 DEBUG:teuthology.orchestra.run.smithi132:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-28T00:13:58.242 DEBUG:teuthology.orchestra.run.smithi165:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-28T00:13:58.280 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 on all hosts... 2024-03-28T00:13:58.280 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 pull 2024-03-28T00:13:58.287 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 pull 2024-03-28T00:13:58.290 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 pull 2024-03-28T00:13:59.075 INFO:teuthology.orchestra.run.smithi165.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033... 2024-03-28T00:13:59.083 INFO:teuthology.orchestra.run.smithi132.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033... 2024-03-28T00:13:59.421 INFO:teuthology.orchestra.run.smithi119.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033... 2024-03-28T00:14:25.601 INFO:teuthology.orchestra.run.smithi132.stdout:{ 2024-03-28T00:14:25.602 INFO:teuthology.orchestra.run.smithi132.stdout: "ceph_version": "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)", 2024-03-28T00:14:25.602 INFO:teuthology.orchestra.run.smithi132.stdout: "image_id": "c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2", 2024-03-28T00:14:25.602 INFO:teuthology.orchestra.run.smithi132.stdout: "repo_digests": [ 2024-03-28T00:14:25.602 INFO:teuthology.orchestra.run.smithi132.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc" 2024-03-28T00:14:25.602 INFO:teuthology.orchestra.run.smithi132.stdout: ] 2024-03-28T00:14:25.602 INFO:teuthology.orchestra.run.smithi132.stdout:} 2024-03-28T00:14:28.228 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-03-28T00:14:28.228 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_version": "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)", 2024-03-28T00:14:28.228 INFO:teuthology.orchestra.run.smithi165.stdout: "image_id": "c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2", 2024-03-28T00:14:28.228 INFO:teuthology.orchestra.run.smithi165.stdout: "repo_digests": [ 2024-03-28T00:14:28.229 INFO:teuthology.orchestra.run.smithi165.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc" 2024-03-28T00:14:28.229 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-03-28T00:14:28.229 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-03-28T00:14:34.793 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-03-28T00:14:34.794 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph_version": "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)", 2024-03-28T00:14:34.794 INFO:teuthology.orchestra.run.smithi119.stdout: "image_id": "c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2", 2024-03-28T00:14:34.794 INFO:teuthology.orchestra.run.smithi119.stdout: "repo_digests": [ 2024-03-28T00:14:34.794 INFO:teuthology.orchestra.run.smithi119.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc" 2024-03-28T00:14:34.794 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-03-28T00:14:34.794 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-03-28T00:14:34.860 DEBUG:teuthology.orchestra.run.smithi119:> sudo mkdir -p /etc/ceph 2024-03-28T00:14:34.959 DEBUG:teuthology.orchestra.run.smithi132:> sudo mkdir -p /etc/ceph 2024-03-28T00:14:34.993 DEBUG:teuthology.orchestra.run.smithi165:> sudo mkdir -p /etc/ceph 2024-03-28T00:14:35.029 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod 777 /etc/ceph 2024-03-28T00:14:35.122 DEBUG:teuthology.orchestra.run.smithi132:> sudo chmod 777 /etc/ceph 2024-03-28T00:14:35.154 DEBUG:teuthology.orchestra.run.smithi165:> sudo chmod 777 /etc/ceph 2024-03-28T00:14:35.185 INFO:tasks.cephadm:Writing seed config... 2024-03-28T00:14:35.186 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-03-28T00:14:35.186 INFO:tasks.cephadm: override: [client] debug client = 20 2024-03-28T00:14:35.186 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-03-28T00:14:35.187 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-03-28T00:14:35.187 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-03-28T00:14:35.187 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-03-28T00:14:35.187 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-03-28T00:14:35.187 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-03-28T00:14:35.187 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-03-28T00:14:35.187 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-03-28T00:14:35.187 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-03-28T00:14:35.187 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-03-28T00:14:35.187 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-03-28T00:14:35.187 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-03-28T00:14:35.187 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-03-28T00:14:35.187 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-03-28T00:14:35.187 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-03-28T00:14:35.188 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2024-03-28T00:14:35.188 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-03-28T00:14:35.188 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-03-28T00:14:35.188 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-03-28T00:14:35.188 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-03-28T00:14:35.188 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-03-28T00:14:35.188 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-03-28T00:14:35.188 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-03-28T00:14:35.188 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-03-28T00:14:35.188 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-03-28T00:14:35.188 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-03-28T00:14:35.188 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-03-28T00:14:35.188 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-03-28T00:14:35.188 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-03-28T00:14:35.188 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-03-28T00:14:35.189 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-03-28T00:14:35.189 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-03-28T00:14:35.189 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-03-28T00:14:35.189 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-03-28T00:14:35.189 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-03-28T00:14:35.189 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-03-28T00:14:35.189 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-03-28T00:14:35.189 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-03-28T00:14:35.189 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-03-28T00:14:35.189 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-03-28T00:14:35.189 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-03-28T00:14:35.189 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-03-28T00:14:35.189 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-03-28T00:14:35.190 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-03-28T00:14:35.190 DEBUG:teuthology.orchestra.run.smithi119:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-03-28T00:14:35.243 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 10f73b92-ec98-11ee-95d0-87774f69a715 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_bal_replicate_threshold = 1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-03-28T00:14:35.244 DEBUG:teuthology.orchestra.run.smithi119:mon.a> sudo journalctl -f -n 0 -u ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mon.a.service 2024-03-28T00:14:35.292 DEBUG:teuthology.orchestra.run.smithi119:mgr.x> sudo journalctl -f -n 0 -u ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mgr.x.service 2024-03-28T00:14:35.337 INFO:tasks.cephadm:Bootstrapping... 2024-03-28T00:14:35.338 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 -v bootstrap --fsid 10f73b92-ec98-11ee-95d0-87774f69a715 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.119 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-03-28T00:14:35.609 INFO:teuthology.orchestra.run.smithi119.stdout:-------------------------------------------------------------------------------- 2024-03-28T00:14:35.609 INFO:teuthology.orchestra.run.smithi119.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033', '-v', 'bootstrap', '--fsid', '10f73b92-ec98-11ee-95d0-87774f69a715', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.119', '--skip-admin-label'] 2024-03-28T00:14:35.666 INFO:teuthology.orchestra.run.smithi119.stdout:/bin/podman: stdout 4.9.4-dev 2024-03-28T00:14:35.667 INFO:teuthology.orchestra.run.smithi119.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-03-28T00:14:35.667 INFO:teuthology.orchestra.run.smithi119.stdout:Verifying podman|docker is present... 2024-03-28T00:14:35.720 INFO:teuthology.orchestra.run.smithi119.stdout:/bin/podman: stdout 4.9.4-dev 2024-03-28T00:14:35.720 INFO:teuthology.orchestra.run.smithi119.stdout:Verifying lvm2 is present... 2024-03-28T00:14:35.720 INFO:teuthology.orchestra.run.smithi119.stdout:Verifying time synchronization is in place... 2024-03-28T00:14:35.738 INFO:teuthology.orchestra.run.smithi119.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-03-28T00:14:35.738 INFO:teuthology.orchestra.run.smithi119.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-03-28T00:14:35.758 INFO:teuthology.orchestra.run.smithi119.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-03-28T00:14:35.759 INFO:teuthology.orchestra.run.smithi119.stdout:systemctl: stdout inactive 2024-03-28T00:14:35.783 INFO:teuthology.orchestra.run.smithi119.stdout:systemctl: stdout enabled 2024-03-28T00:14:35.805 INFO:teuthology.orchestra.run.smithi119.stdout:systemctl: stdout active 2024-03-28T00:14:35.805 INFO:teuthology.orchestra.run.smithi119.stdout:Unit chronyd.service is enabled and running 2024-03-28T00:14:35.805 INFO:teuthology.orchestra.run.smithi119.stdout:Repeating the final host check... 2024-03-28T00:14:35.862 INFO:teuthology.orchestra.run.smithi119.stdout:/bin/podman: stdout 4.9.4-dev 2024-03-28T00:14:35.862 INFO:teuthology.orchestra.run.smithi119.stdout:podman (/bin/podman) version 4.9.4 is present 2024-03-28T00:14:35.862 INFO:teuthology.orchestra.run.smithi119.stdout:systemctl is present 2024-03-28T00:14:35.862 INFO:teuthology.orchestra.run.smithi119.stdout:lvcreate is present 2024-03-28T00:14:35.879 INFO:teuthology.orchestra.run.smithi119.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-03-28T00:14:35.879 INFO:teuthology.orchestra.run.smithi119.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-03-28T00:14:35.900 INFO:teuthology.orchestra.run.smithi119.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-03-28T00:14:35.900 INFO:teuthology.orchestra.run.smithi119.stdout:systemctl: stdout inactive 2024-03-28T00:14:35.924 INFO:teuthology.orchestra.run.smithi119.stdout:systemctl: stdout enabled 2024-03-28T00:14:35.944 INFO:teuthology.orchestra.run.smithi119.stdout:systemctl: stdout active 2024-03-28T00:14:35.944 INFO:teuthology.orchestra.run.smithi119.stdout:Unit chronyd.service is enabled and running 2024-03-28T00:14:35.945 INFO:teuthology.orchestra.run.smithi119.stdout:Host looks OK 2024-03-28T00:14:35.945 INFO:teuthology.orchestra.run.smithi119.stdout:Cluster fsid: 10f73b92-ec98-11ee-95d0-87774f69a715 2024-03-28T00:14:35.945 INFO:teuthology.orchestra.run.smithi119.stdout:Acquiring lock 139722674372032 on /run/cephadm/10f73b92-ec98-11ee-95d0-87774f69a715.lock 2024-03-28T00:14:35.946 INFO:teuthology.orchestra.run.smithi119.stdout:Lock 139722674372032 acquired on /run/cephadm/10f73b92-ec98-11ee-95d0-87774f69a715.lock 2024-03-28T00:14:35.947 INFO:teuthology.orchestra.run.smithi119.stdout:Verifying IP 172.21.15.119 port 3300 ... 2024-03-28T00:14:35.948 INFO:teuthology.orchestra.run.smithi119.stdout:Verifying IP 172.21.15.119 port 6789 ... 2024-03-28T00:14:35.948 INFO:teuthology.orchestra.run.smithi119.stdout:Base mon IP(s) is [172.21.15.119:3300, 172.21.15.119:6789], mon addrv is [v2:172.21.15.119:3300,v1:172.21.15.119:6789] 2024-03-28T00:14:35.963 INFO:teuthology.orchestra.run.smithi119.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.119 metric 100 2024-03-28T00:14:35.963 INFO:teuthology.orchestra.run.smithi119.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.119 metric 100 2024-03-28T00:14:35.976 INFO:teuthology.orchestra.run.smithi119.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-03-28T00:14:35.977 INFO:teuthology.orchestra.run.smithi119.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 256 pref medium 2024-03-28T00:14:35.977 INFO:teuthology.orchestra.run.smithi119.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 1024 expires 1797sec hoplimit 64 pref medium 2024-03-28T00:14:35.987 INFO:teuthology.orchestra.run.smithi119.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-03-28T00:14:35.988 INFO:teuthology.orchestra.run.smithi119.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-03-28T00:14:35.988 INFO:teuthology.orchestra.run.smithi119.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-03-28T00:14:35.988 INFO:teuthology.orchestra.run.smithi119.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-03-28T00:14:35.988 INFO:teuthology.orchestra.run.smithi119.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe8f:d5ef/64 scope link 2024-03-28T00:14:35.988 INFO:teuthology.orchestra.run.smithi119.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-03-28T00:14:35.991 INFO:teuthology.orchestra.run.smithi119.stdout:Mon IP `172.21.15.119` is in CIDR network `172.21.0.0/20` 2024-03-28T00:14:35.991 INFO:teuthology.orchestra.run.smithi119.stdout:Mon IP `172.21.15.119` is in CIDR network `172.21.0.0/20` 2024-03-28T00:14:35.991 INFO:teuthology.orchestra.run.smithi119.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-03-28T00:14:35.994 INFO:teuthology.orchestra.run.smithi119.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-03-28T00:14:35.996 INFO:teuthology.orchestra.run.smithi119.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033... 2024-03-28T00:14:40.079 INFO:teuthology.orchestra.run.smithi119.stdout:/bin/podman: stdout c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 2024-03-28T00:14:40.080 INFO:teuthology.orchestra.run.smithi119.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033... 2024-03-28T00:14:40.080 INFO:teuthology.orchestra.run.smithi119.stdout:/bin/podman: stderr Getting image source signatures 2024-03-28T00:14:40.080 INFO:teuthology.orchestra.run.smithi119.stdout:/bin/podman: stderr Copying blob sha256:4dd83c6ee8921fe17396d77c31e205985ae94c87d3f752465b5f04aaf9ae340a 2024-03-28T00:14:40.080 INFO:teuthology.orchestra.run.smithi119.stdout:/bin/podman: stderr Copying blob sha256:9516588f908cdb14b5e4062a6063db2d9291b355c3f32e2a99864608a4a7b649 2024-03-28T00:14:40.080 INFO:teuthology.orchestra.run.smithi119.stdout:/bin/podman: stderr Copying config sha256:c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 2024-03-28T00:14:40.080 INFO:teuthology.orchestra.run.smithi119.stdout:/bin/podman: stderr Writing manifest to image destination 2024-03-28T00:14:48.297 INFO:teuthology.orchestra.run.smithi119.stdout:ceph: stdout ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:14:48.297 INFO:teuthology.orchestra.run.smithi119.stdout:Ceph version: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:14:48.298 INFO:teuthology.orchestra.run.smithi119.stdout:Extracting ceph user uid/gid from container image... 2024-03-28T00:14:49.453 INFO:teuthology.orchestra.run.smithi119.stdout:stat: stdout 167 167 2024-03-28T00:14:49.453 INFO:teuthology.orchestra.run.smithi119.stdout:Creating initial keys... 2024-03-28T00:14:50.528 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph-authtool: stdout AQB5tgRmBV1+NhAA932ibTjiRRieTWpaT7wgiw== 2024-03-28T00:14:51.647 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph-authtool: stdout AQB6tgRm4ZxWOhAAM3Osf0yzmglvZz4J3hAqgQ== 2024-03-28T00:14:52.740 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph-authtool: stdout AQB8tgRmvBQeBBAASg/iIc3Agz/sdi69cl9e0Q== 2024-03-28T00:14:52.743 INFO:teuthology.orchestra.run.smithi119.stdout:Creating initial monmap... 2024-03-28T00:14:54.096 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-03-28T00:14:54.096 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-03-28T00:14:54.096 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 10f73b92-ec98-11ee-95d0-87774f69a715 2024-03-28T00:14:54.097 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-03-28T00:14:54.097 INFO:teuthology.orchestra.run.smithi119.stdout:monmaptool for a [v2:172.21.15.119:3300,v1:172.21.15.119:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-03-28T00:14:54.097 INFO:teuthology.orchestra.run.smithi119.stdout:setting min_mon_release = quincy 2024-03-28T00:14:54.097 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/monmaptool: set fsid to 10f73b92-ec98-11ee-95d0-87774f69a715 2024-03-28T00:14:54.097 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-03-28T00:14:54.097 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-28T00:14:54.097 INFO:teuthology.orchestra.run.smithi119.stdout:Creating mon... 2024-03-28T00:14:55.272 INFO:teuthology.orchestra.run.smithi119.stdout:create mon.a on 2024-03-28T00:14:55.622 INFO:teuthology.orchestra.run.smithi119.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-03-28T00:14:55.855 INFO:teuthology.orchestra.run.smithi119.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-03-28T00:14:56.137 INFO:teuthology.orchestra.run.smithi119.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-10f73b92-ec98-11ee-95d0-87774f69a715.target → /etc/systemd/system/ceph-10f73b92-ec98-11ee-95d0-87774f69a715.target. 2024-03-28T00:14:56.138 INFO:teuthology.orchestra.run.smithi119.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-10f73b92-ec98-11ee-95d0-87774f69a715.target → /etc/systemd/system/ceph-10f73b92-ec98-11ee-95d0-87774f69a715.target. 2024-03-28T00:14:56.520 INFO:teuthology.orchestra.run.smithi119.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mon.a 2024-03-28T00:14:56.520 INFO:teuthology.orchestra.run.smithi119.stdout:systemctl: stderr Failed to reset failed state of unit ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mon.a.service: Unit ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mon.a.service not loaded. 2024-03-28T00:14:56.791 INFO:teuthology.orchestra.run.smithi119.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-10f73b92-ec98-11ee-95d0-87774f69a715.target.wants/ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mon.a.service → /etc/systemd/system/ceph-10f73b92-ec98-11ee-95d0-87774f69a715@.service. 2024-03-28T00:14:57.486 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:14:57 smithi119 podman[31127]: 2024-03-28 00:14:57.241994169 +0000 UTC m=+0.203436497 container create fa1f99fa62753d0042c3e8f28132daac3b3e9a6fc6d307fb8d75a3b7f636c775 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mon-a, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-03-28T00:14:57.486 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:14:57 smithi119 podman[31127]: 2024-03-28 00:14:57.47868094 +0000 UTC m=+0.440123274 container init fa1f99fa62753d0042c3e8f28132daac3b3e9a6fc6d307fb8d75a3b7f636c775 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mon-a, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0) 2024-03-28T00:14:57.624 INFO:teuthology.orchestra.run.smithi119.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-28T00:14:57.624 INFO:teuthology.orchestra.run.smithi119.stdout:systemctl: stdout disabled 2024-03-28T00:14:57.639 INFO:teuthology.orchestra.run.smithi119.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-28T00:14:57.639 INFO:teuthology.orchestra.run.smithi119.stdout:systemctl: stdout inactive 2024-03-28T00:14:57.639 INFO:teuthology.orchestra.run.smithi119.stdout:firewalld.service is not enabled 2024-03-28T00:14:57.639 INFO:teuthology.orchestra.run.smithi119.stdout:Not possible to enable service . firewalld.service is not available 2024-03-28T00:14:57.640 INFO:teuthology.orchestra.run.smithi119.stdout:Waiting for mon to start... 2024-03-28T00:14:57.640 INFO:teuthology.orchestra.run.smithi119.stdout:Waiting for mon... 2024-03-28T00:14:57.787 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:14:57 smithi119 podman[31127]: 2024-03-28 00:14:57.48843838 +0000 UTC m=+0.449880676 container start fa1f99fa62753d0042c3e8f28132daac3b3e9a6fc6d307fb8d75a3b7f636c775 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mon-a, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c) 2024-03-28T00:14:57.788 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:14:57 smithi119 ceph-mon[31145]: mkfs 10f73b92-ec98-11ee-95d0-87774f69a715 2024-03-28T00:14:57.788 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:14:57 smithi119 ceph-mon[31145]: mon.a is new leader, mons a in quorum (ranks 0) 2024-03-28T00:14:57.788 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:14:57 smithi119 bash[31127]: fa1f99fa62753d0042c3e8f28132daac3b3e9a6fc6d307fb8d75a3b7f636c775 2024-03-28T00:14:57.788 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:14:57 smithi119 systemd[1]: Started Ceph mon.a for 10f73b92-ec98-11ee-95d0-87774f69a715. 2024-03-28T00:14:58.874 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:14:58 smithi119 ceph-mon[31145]: mon.a is new leader, mons a in quorum (ranks 0) 2024-03-28T00:14:58.874 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:14:58 smithi119 ceph-mon[31145]: monmap e1: 1 mons at {a=[v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-28T00:14:58.874 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:14:58 smithi119 ceph-mon[31145]: fsmap 2024-03-28T00:14:58.874 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:14:58 smithi119 ceph-mon[31145]: osdmap e1: 0 total, 0 up, 0 in 2024-03-28T00:14:58.874 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:14:58 smithi119 ceph-mon[31145]: mgrmap e1: no daemons active 2024-03-28T00:14:58.874 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:14:58 smithi119 ceph-mon[31145]: from='client.? 172.21.15.119:0/2324156289' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-03-28T00:14:58.884 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout cluster: 2024-03-28T00:14:58.890 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout id: 10f73b92-ec98-11ee-95d0-87774f69a715 2024-03-28T00:14:58.890 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-03-28T00:14:58.890 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout 2024-03-28T00:14:58.890 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout services: 2024-03-28T00:14:58.890 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.720281s) 2024-03-28T00:14:58.890 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-03-28T00:14:58.890 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-03-28T00:14:58.890 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout 2024-03-28T00:14:58.891 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout data: 2024-03-28T00:14:58.891 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-03-28T00:14:58.891 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-03-28T00:14:58.891 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-03-28T00:14:58.891 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout pgs: 2024-03-28T00:14:58.891 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout 2024-03-28T00:14:58.891 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:14:58.190+0000 7fb54e600700 1 Processor -- start 2024-03-28T00:14:58.891 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:14:58.190+0000 7fb54e600700 1 -- start start 2024-03-28T00:14:58.891 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:14:58.191+0000 7fb54e600700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb548107af0 0x7fb548107f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:14:58.891 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:14:58.191+0000 7fb54e600700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5481084e0 con 0x7fb548107af0 2024-03-28T00:14:58.891 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:14:58.192+0000 7fb547400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb548107af0 0x7fb548107f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:14:58.891 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:14:58.192+0000 7fb547400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb548107af0 0x7fb548107f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:47298/0 (socket says 172.21.15.119:47298) 2024-03-28T00:14:58.892 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:14:58.192+0000 7fb547400700 1 -- 172.21.15.119:0/515923355 learned_addr learned my addr 172.21.15.119:0/515923355 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:14:58.892 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:14:58.193+0000 7fb547400700 1 -- 172.21.15.119:0/515923355 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb548108cf0 con 0x7fb548107af0 2024-03-28T00:14:58.892 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:14:58.193+0000 7fb547400700 1 --2- 172.21.15.119:0/515923355 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb548107af0 0x7fb548107f10 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fb5380098a0 tx=0x7fb538009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a424f43cdbac448c server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:14:58.892 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:14:58.194+0000 7fb546a00700 1 -- 172.21.15.119:0/515923355 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb538004160 con 0x7fb548107af0 2024-03-28T00:14:58.892 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:14:58.194+0000 7fb546a00700 1 -- 172.21.15.119:0/515923355 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb538004300 con 0x7fb548107af0 2024-03-28T00:14:58.892 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:14:58.194+0000 7fb546a00700 1 -- 172.21.15.119:0/515923355 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb5380036a0 con 0x7fb548107af0 2024-03-28T00:14:58.892 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:14:58.194+0000 7fb54e600700 1 -- 172.21.15.119:0/515923355 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb548107af0 msgr2=0x7fb548107f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:14:58.892 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:14:58.194+0000 7fb54e600700 1 --2- 172.21.15.119:0/515923355 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb548107af0 0x7fb548107f10 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fb5380098a0 tx=0x7fb538009bb0 comp rx=0 tx=0).stop 2024-03-28T00:14:58.892 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:14:58.194+0000 7fb54e600700 1 -- 172.21.15.119:0/515923355 shutdown_connections 2024-03-28T00:14:58.892 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:14:58.194+0000 7fb54e600700 1 --2- 172.21.15.119:0/515923355 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb548107af0 0x7fb548107f10 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:14:58.892 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:14:58.194+0000 7fb54e600700 1 -- 172.21.15.119:0/515923355 >> 172.21.15.119:0/515923355 conn(0x7fb5481030f0 msgr2=0x7fb548105510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:14:58.892 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:14:58.195+0000 7fb54e600700 1 -- 172.21.15.119:0/515923355 shutdown_connections 2024-03-28T00:14:58.892 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:14:58.195+0000 7fb54e600700 1 -- 172.21.15.119:0/515923355 wait complete. 2024-03-28T00:14:58.892 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:14:58.196+0000 7fb54e600700 1 Processor -- start 2024-03-28T00:14:58.893 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:14:58.196+0000 7fb54e600700 1 -- start start 2024-03-28T00:14:58.893 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:14:58.197+0000 7fb54e600700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb548107af0 0x7fb54819d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:14:58.893 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:14:58.197+0000 7fb54e600700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb548109220 con 0x7fb548107af0 2024-03-28T00:14:58.893 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:14:58.197+0000 7fb547400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb548107af0 0x7fb54819d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:14:58.893 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:14:58.197+0000 7fb547400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb548107af0 0x7fb54819d470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:47308/0 (socket says 172.21.15.119:47308) 2024-03-28T00:14:58.893 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:14:58.197+0000 7fb547400700 1 -- 172.21.15.119:0/2324156289 learned_addr learned my addr 172.21.15.119:0/2324156289 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:14:58.893 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:14:58.197+0000 7fb547400700 1 -- 172.21.15.119:0/2324156289 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb54819d9b0 con 0x7fb548107af0 2024-03-28T00:14:58.893 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:14:58.198+0000 7fb547400700 1 --2- 172.21.15.119:0/2324156289 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb548107af0 0x7fb54819d470 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fb538000c00 tx=0x7fb538004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:14:58.893 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:14:58.198+0000 7fb544c00700 1 -- 172.21.15.119:0/2324156289 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb538003db0 con 0x7fb548107af0 2024-03-28T00:14:58.893 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:14:58.198+0000 7fb544c00700 1 -- 172.21.15.119:0/2324156289 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb538011630 con 0x7fb548107af0 2024-03-28T00:14:58.893 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:14:58.198+0000 7fb544c00700 1 -- 172.21.15.119:0/2324156289 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb538011950 con 0x7fb548107af0 2024-03-28T00:14:58.893 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:14:58.198+0000 7fb54e600700 1 -- 172.21.15.119:0/2324156289 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb54819dc40 con 0x7fb548107af0 2024-03-28T00:14:58.893 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:14:58.198+0000 7fb54e600700 1 -- 172.21.15.119:0/2324156289 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5481a0930 con 0x7fb548107af0 2024-03-28T00:14:58.894 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:14:58.199+0000 7fb544c00700 1 -- 172.21.15.119:0/2324156289 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fb53801a460 con 0x7fb548107af0 2024-03-28T00:14:58.894 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:14:58.199+0000 7fb544c00700 1 -- 172.21.15.119:0/2324156289 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fb53801b780 con 0x7fb548107af0 2024-03-28T00:14:58.894 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:14:58.199+0000 7fb54e600700 1 -- 172.21.15.119:0/2324156289 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb548069260 con 0x7fb548107af0 2024-03-28T00:14:58.894 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:14:58.203+0000 7fb544c00700 1 -- 172.21.15.119:0/2324156289 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fb538031020 con 0x7fb548107af0 2024-03-28T00:14:58.894 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:14:58.281+0000 7fb54e600700 1 -- 172.21.15.119:0/2324156289 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7fb5481a0b00 con 0x7fb548107af0 2024-03-28T00:14:58.894 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:14:58.281+0000 7fb544c00700 1 -- 172.21.15.119:0/2324156289 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+317 (secure 0 0 0) 0x7fb53801ba20 con 0x7fb548107af0 2024-03-28T00:14:58.894 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:14:58.283+0000 7fb54e600700 1 -- 172.21.15.119:0/2324156289 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb548107af0 msgr2=0x7fb54819d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:14:58.894 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:14:58.283+0000 7fb54e600700 1 --2- 172.21.15.119:0/2324156289 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb548107af0 0x7fb54819d470 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fb538000c00 tx=0x7fb538004750 comp rx=0 tx=0).stop 2024-03-28T00:14:58.894 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:14:58.283+0000 7fb54e600700 1 -- 172.21.15.119:0/2324156289 shutdown_connections 2024-03-28T00:14:58.894 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:14:58.283+0000 7fb54e600700 1 --2- 172.21.15.119:0/2324156289 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb548107af0 0x7fb54819d470 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:14:58.894 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:14:58.283+0000 7fb54e600700 1 -- 172.21.15.119:0/2324156289 >> 172.21.15.119:0/2324156289 conn(0x7fb5481030f0 msgr2=0x7fb54807d290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:14:58.894 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:14:58.283+0000 7fb54e600700 1 -- 172.21.15.119:0/2324156289 shutdown_connections 2024-03-28T00:14:58.895 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:14:58.283+0000 7fb54e600700 1 -- 172.21.15.119:0/2324156289 wait complete. 2024-03-28T00:14:58.895 INFO:teuthology.orchestra.run.smithi119.stdout:mon is available 2024-03-28T00:14:58.895 INFO:teuthology.orchestra.run.smithi119.stdout:Assimilating anything we can from ceph.conf... 2024-03-28T00:14:59.872 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:14:59 smithi119 ceph-mon[31145]: from='client.? 172.21.15.119:0/3865094174' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-03-28T00:14:59.872 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:14:59 smithi119 ceph-mon[31145]: from='client.? 172.21.15.119:0/3865094174' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-03-28T00:15:00.183 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout 2024-03-28T00:15:00.183 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout [global] 2024-03-28T00:15:00.183 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout fsid = 10f73b92-ec98-11ee-95d0-87774f69a715 2024-03-28T00:15:00.183 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.119:3300,v1:172.21.15.119:6789] 2024-03-28T00:15:00.183 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-03-28T00:15:00.183 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-03-28T00:15:00.183 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-03-28T00:15:00.183 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-03-28T00:15:00.187 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout 2024-03-28T00:15:00.187 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout [mgr] 2024-03-28T00:15:00.187 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-03-28T00:15:00.187 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout 2024-03-28T00:15:00.187 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout [osd] 2024-03-28T00:15:00.187 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-03-28T00:15:00.187 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-03-28T00:15:00.187 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-03-28T00:15:00.187 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-03-28T00:15:00.188 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-03-28T00:15:00.188 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:14:59.461+0000 7fa936a00700 1 Processor -- start 2024-03-28T00:15:00.188 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:14:59.462+0000 7fa936a00700 1 -- start start 2024-03-28T00:15:00.188 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:14:59.462+0000 7fa936a00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa93010ec80 0x7fa930111070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:15:00.188 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:14:59.462+0000 7fa936a00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa930082380 con 0x7fa93010ec80 2024-03-28T00:15:00.188 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:14:59.462+0000 7fa92fe00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa93010ec80 0x7fa930111070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:15:00.188 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:14:59.462+0000 7fa92fe00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa93010ec80 0x7fa930111070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:47316/0 (socket says 172.21.15.119:47316) 2024-03-28T00:15:00.188 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:14:59.462+0000 7fa92fe00700 1 -- 172.21.15.119:0/129929518 learned_addr learned my addr 172.21.15.119:0/129929518 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:15:00.188 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:14:59.463+0000 7fa92fe00700 1 -- 172.21.15.119:0/129929518 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9301115b0 con 0x7fa93010ec80 2024-03-28T00:15:00.188 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:14:59.463+0000 7fa92fe00700 1 --2- 172.21.15.119:0/129929518 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa93010ec80 0x7fa930111070 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fa9200098a0 tx=0x7fa920009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa209272585da84c server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:15:00.188 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:14:59.464+0000 7fa92ea00700 1 -- 172.21.15.119:0/129929518 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa920004160 con 0x7fa93010ec80 2024-03-28T00:15:00.188 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:14:59.464+0000 7fa92ea00700 1 -- 172.21.15.119:0/129929518 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa920004300 con 0x7fa93010ec80 2024-03-28T00:15:00.188 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:14:59.464+0000 7fa92ea00700 1 -- 172.21.15.119:0/129929518 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa9200036a0 con 0x7fa93010ec80 2024-03-28T00:15:00.189 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:14:59.464+0000 7fa936a00700 1 -- 172.21.15.119:0/129929518 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa93010ec80 msgr2=0x7fa930111070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:15:00.189 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:14:59.464+0000 7fa936a00700 1 --2- 172.21.15.119:0/129929518 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa93010ec80 0x7fa930111070 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fa9200098a0 tx=0x7fa920009bb0 comp rx=0 tx=0).stop 2024-03-28T00:15:00.189 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:14:59.464+0000 7fa936a00700 1 -- 172.21.15.119:0/129929518 shutdown_connections 2024-03-28T00:15:00.189 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:14:59.464+0000 7fa936a00700 1 --2- 172.21.15.119:0/129929518 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa93010ec80 0x7fa930111070 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:15:00.189 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:14:59.464+0000 7fa936a00700 1 -- 172.21.15.119:0/129929518 >> 172.21.15.119:0/129929518 conn(0x7fa93007d100 msgr2=0x7fa93007f520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:15:00.189 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:14:59.465+0000 7fa936a00700 1 -- 172.21.15.119:0/129929518 shutdown_connections 2024-03-28T00:15:00.189 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:14:59.465+0000 7fa936a00700 1 -- 172.21.15.119:0/129929518 wait complete. 2024-03-28T00:15:00.189 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:14:59.466+0000 7fa936a00700 1 Processor -- start 2024-03-28T00:15:00.189 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:14:59.466+0000 7fa936a00700 1 -- start start 2024-03-28T00:15:00.189 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:14:59.466+0000 7fa936a00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa93010ec80 0x7fa9301a1820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:15:00.189 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:14:59.466+0000 7fa936a00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa930111960 con 0x7fa93010ec80 2024-03-28T00:15:00.192 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:14:59.466+0000 7fa92fe00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa93010ec80 0x7fa9301a1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:15:00.193 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:14:59.466+0000 7fa92fe00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa93010ec80 0x7fa9301a1820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:47322/0 (socket says 172.21.15.119:47322) 2024-03-28T00:15:00.193 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:14:59.466+0000 7fa92fe00700 1 -- 172.21.15.119:0/3865094174 learned_addr learned my addr 172.21.15.119:0/3865094174 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:15:00.193 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:14:59.467+0000 7fa92fe00700 1 -- 172.21.15.119:0/3865094174 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9301a1d60 con 0x7fa93010ec80 2024-03-28T00:15:00.193 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:14:59.467+0000 7fa92fe00700 1 --2- 172.21.15.119:0/3865094174 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa93010ec80 0x7fa9301a1820 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fa920000c00 tx=0x7fa920004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:15:00.193 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:14:59.467+0000 7fa92cc00700 1 -- 172.21.15.119:0/3865094174 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa920003db0 con 0x7fa93010ec80 2024-03-28T00:15:00.193 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:14:59.467+0000 7fa92cc00700 1 -- 172.21.15.119:0/3865094174 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa920011630 con 0x7fa93010ec80 2024-03-28T00:15:00.193 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:14:59.467+0000 7fa92cc00700 1 -- 172.21.15.119:0/3865094174 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa920011950 con 0x7fa93010ec80 2024-03-28T00:15:00.193 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:14:59.467+0000 7fa936a00700 1 -- 172.21.15.119:0/3865094174 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9301a1ff0 con 0x7fa93010ec80 2024-03-28T00:15:00.193 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:14:59.468+0000 7fa936a00700 1 -- 172.21.15.119:0/3865094174 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9301a4ce0 con 0x7fa93010ec80 2024-03-28T00:15:00.193 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:14:59.468+0000 7fa92cc00700 1 -- 172.21.15.119:0/3865094174 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fa920011af0 con 0x7fa93010ec80 2024-03-28T00:15:00.193 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:14:59.468+0000 7fa92cc00700 1 -- 172.21.15.119:0/3865094174 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fa92001c450 con 0x7fa93010ec80 2024-03-28T00:15:00.193 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:14:59.469+0000 7fa936a00700 1 -- 172.21.15.119:0/3865094174 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa930069260 con 0x7fa93010ec80 2024-03-28T00:15:00.193 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:14:59.471+0000 7fa92cc00700 1 -- 172.21.15.119:0/3865094174 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fa920032020 con 0x7fa93010ec80 2024-03-28T00:15:00.193 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:14:59.547+0000 7fa936a00700 1 -- 172.21.15.119:0/3865094174 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7fa93002dbf0 con 0x7fa93010ec80 2024-03-28T00:15:00.194 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:14:59.555+0000 7fa92cc00700 1 -- 172.21.15.119:0/3865094174 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+431 (secure 0 0 0) 0x7fa920004260 con 0x7fa93010ec80 2024-03-28T00:15:00.194 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:14:59.556+0000 7fa936a00700 1 -- 172.21.15.119:0/3865094174 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa93010ec80 msgr2=0x7fa9301a1820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:15:00.194 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:14:59.556+0000 7fa936a00700 1 --2- 172.21.15.119:0/3865094174 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa93010ec80 0x7fa9301a1820 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fa920000c00 tx=0x7fa920004750 comp rx=0 tx=0).stop 2024-03-28T00:15:00.194 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:14:59.557+0000 7fa936a00700 1 -- 172.21.15.119:0/3865094174 shutdown_connections 2024-03-28T00:15:00.194 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:14:59.557+0000 7fa936a00700 1 --2- 172.21.15.119:0/3865094174 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa93010ec80 0x7fa9301a1820 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:15:00.194 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:14:59.557+0000 7fa936a00700 1 -- 172.21.15.119:0/3865094174 >> 172.21.15.119:0/3865094174 conn(0x7fa93007d100 msgr2=0x7fa93007f3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:15:00.194 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:14:59.557+0000 7fa936a00700 1 -- 172.21.15.119:0/3865094174 shutdown_connections 2024-03-28T00:15:00.194 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:14:59.557+0000 7fa936a00700 1 -- 172.21.15.119:0/3865094174 wait complete. 2024-03-28T00:15:00.194 INFO:teuthology.orchestra.run.smithi119.stdout:Generating new minimal ceph.conf... 2024-03-28T00:15:01.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:00 smithi119 ceph-mon[31145]: from='client.? 172.21.15.119:0/566877883' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:15:01.590 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:00.736+0000 7fa99ca00700 1 Processor -- start 2024-03-28T00:15:01.590 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:00.736+0000 7fa99ca00700 1 -- start start 2024-03-28T00:15:01.590 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:00.737+0000 7fa99ca00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa998109d10 0x7fa99810a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:15:01.590 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:00.737+0000 7fa99ca00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa99810a700 con 0x7fa998109d10 2024-03-28T00:15:01.590 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:00.737+0000 7fa995e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa998109d10 0x7fa99810a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:15:01.590 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:00.737+0000 7fa995e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa998109d10 0x7fa99810a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:47334/0 (socket says 172.21.15.119:47334) 2024-03-28T00:15:01.590 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:00.737+0000 7fa995e00700 1 -- 172.21.15.119:0/1467634150 learned_addr learned my addr 172.21.15.119:0/1467634150 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:15:01.590 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:00.738+0000 7fa995e00700 1 -- 172.21.15.119:0/1467634150 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa99810af10 con 0x7fa998109d10 2024-03-28T00:15:01.591 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:00.738+0000 7fa995e00700 1 --2- 172.21.15.119:0/1467634150 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa998109d10 0x7fa99810a130 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fa9880098a0 tx=0x7fa988009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dbe463bf6f26819 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:15:01.591 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:00.738+0000 7fa994a00700 1 -- 172.21.15.119:0/1467634150 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa988004160 con 0x7fa998109d10 2024-03-28T00:15:01.591 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:00.738+0000 7fa994a00700 1 -- 172.21.15.119:0/1467634150 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa988004300 con 0x7fa998109d10 2024-03-28T00:15:01.591 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:00.738+0000 7fa994a00700 1 -- 172.21.15.119:0/1467634150 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa988011b50 con 0x7fa998109d10 2024-03-28T00:15:01.591 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:00.739+0000 7fa99ca00700 1 -- 172.21.15.119:0/1467634150 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa998109d10 msgr2=0x7fa99810a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:15:01.591 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:00.739+0000 7fa99ca00700 1 --2- 172.21.15.119:0/1467634150 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa998109d10 0x7fa99810a130 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fa9880098a0 tx=0x7fa988009bb0 comp rx=0 tx=0).stop 2024-03-28T00:15:01.591 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:00.739+0000 7fa99ca00700 1 -- 172.21.15.119:0/1467634150 shutdown_connections 2024-03-28T00:15:01.591 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:00.739+0000 7fa99ca00700 1 --2- 172.21.15.119:0/1467634150 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa998109d10 0x7fa99810a130 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:15:01.591 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:00.739+0000 7fa99ca00700 1 -- 172.21.15.119:0/1467634150 >> 172.21.15.119:0/1467634150 conn(0x7fa99807d100 msgr2=0x7fa99807d510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:15:01.591 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:00.739+0000 7fa99ca00700 1 -- 172.21.15.119:0/1467634150 shutdown_connections 2024-03-28T00:15:01.591 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:00.739+0000 7fa99ca00700 1 -- 172.21.15.119:0/1467634150 wait complete. 2024-03-28T00:15:01.591 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:00.740+0000 7fa99ca00700 1 Processor -- start 2024-03-28T00:15:01.591 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:00.740+0000 7fa99ca00700 1 -- start start 2024-03-28T00:15:01.591 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:00.741+0000 7fa99ca00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa998109d10 0x7fa99819dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:15:01.592 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:00.741+0000 7fa99ca00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa99810e2c0 con 0x7fa998109d10 2024-03-28T00:15:01.592 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:00.741+0000 7fa995e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa998109d10 0x7fa99819dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:15:01.592 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:00.741+0000 7fa995e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa998109d10 0x7fa99819dce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:47346/0 (socket says 172.21.15.119:47346) 2024-03-28T00:15:01.592 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:00.741+0000 7fa995e00700 1 -- 172.21.15.119:0/566877883 learned_addr learned my addr 172.21.15.119:0/566877883 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:15:01.592 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:00.741+0000 7fa995e00700 1 -- 172.21.15.119:0/566877883 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa99819e220 con 0x7fa998109d10 2024-03-28T00:15:01.592 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:00.741+0000 7fa995e00700 1 --2- 172.21.15.119:0/566877883 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa998109d10 0x7fa99819dce0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fa988000c00 tx=0x7fa988011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:15:01.592 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:00.742+0000 7fa98ea00700 1 -- 172.21.15.119:0/566877883 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa988003b20 con 0x7fa998109d10 2024-03-28T00:15:01.592 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:00.742+0000 7fa98ea00700 1 -- 172.21.15.119:0/566877883 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa988003cc0 con 0x7fa998109d10 2024-03-28T00:15:01.592 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:00.742+0000 7fa99ca00700 1 -- 172.21.15.119:0/566877883 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa99819e4b0 con 0x7fa998109d10 2024-03-28T00:15:01.592 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:00.742+0000 7fa98ea00700 1 -- 172.21.15.119:0/566877883 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa988024910 con 0x7fa998109d10 2024-03-28T00:15:01.592 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:00.742+0000 7fa99ca00700 1 -- 172.21.15.119:0/566877883 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9981a1190 con 0x7fa998109d10 2024-03-28T00:15:01.592 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:00.742+0000 7fa98ea00700 1 -- 172.21.15.119:0/566877883 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fa988022020 con 0x7fa998109d10 2024-03-28T00:15:01.592 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:00.742+0000 7fa98ea00700 1 -- 172.21.15.119:0/566877883 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fa988011e00 con 0x7fa998109d10 2024-03-28T00:15:01.593 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:00.743+0000 7fa99ca00700 1 -- 172.21.15.119:0/566877883 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa998069260 con 0x7fa998109d10 2024-03-28T00:15:01.593 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:00.746+0000 7fa98ea00700 1 -- 172.21.15.119:0/566877883 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fa988042dd0 con 0x7fa998109d10 2024-03-28T00:15:01.593 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:00.822+0000 7fa99ca00700 1 -- 172.21.15.119:0/566877883 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7fa9981a1360 con 0x7fa998109d10 2024-03-28T00:15:01.593 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:00.822+0000 7fa98ea00700 1 -- 172.21.15.119:0/566877883 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+177 (secure 0 0 0) 0x7fa988031020 con 0x7fa998109d10 2024-03-28T00:15:01.593 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:00.824+0000 7fa99ca00700 1 -- 172.21.15.119:0/566877883 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa998109d10 msgr2=0x7fa99819dce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:15:01.593 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:00.824+0000 7fa99ca00700 1 --2- 172.21.15.119:0/566877883 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa998109d10 0x7fa99819dce0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fa988000c00 tx=0x7fa988011c10 comp rx=0 tx=0).stop 2024-03-28T00:15:01.593 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:00.824+0000 7fa99ca00700 1 -- 172.21.15.119:0/566877883 shutdown_connections 2024-03-28T00:15:01.593 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:00.824+0000 7fa99ca00700 1 --2- 172.21.15.119:0/566877883 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa998109d10 0x7fa99819dce0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:15:01.593 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:00.824+0000 7fa99ca00700 1 -- 172.21.15.119:0/566877883 >> 172.21.15.119:0/566877883 conn(0x7fa99807d100 msgr2=0x7fa998107480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:15:01.593 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:00.824+0000 7fa99ca00700 1 -- 172.21.15.119:0/566877883 shutdown_connections 2024-03-28T00:15:01.593 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:00.824+0000 7fa99ca00700 1 -- 172.21.15.119:0/566877883 wait complete. 2024-03-28T00:15:01.593 INFO:teuthology.orchestra.run.smithi119.stdout:Restarting the monitor... 2024-03-28T00:15:01.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:01 smithi119 systemd[1]: Stopping Ceph mon.a for 10f73b92-ec98-11ee-95d0-87774f69a715... 2024-03-28T00:15:02.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:02 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mon-a[31141]: 2024-03-28T00:15:02.039+0000 7f1753c00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-28T00:15:02.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:02 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mon-a[31141]: 2024-03-28T00:15:02.039+0000 7f1753c00700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-03-28T00:15:02.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:02 smithi119 podman[31391]: 2024-03-28 00:15:02.111715525 +0000 UTC m=+0.238211790 container stop fa1f99fa62753d0042c3e8f28132daac3b3e9a6fc6d307fb8d75a3b7f636c775 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mon-a, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-28T00:15:02.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:02 smithi119 podman[31391]: 2024-03-28 00:15:02.137739337 +0000 UTC m=+0.264235595 container died fa1f99fa62753d0042c3e8f28132daac3b3e9a6fc6d307fb8d75a3b7f636c775 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mon-a, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.license=GPLv2) 2024-03-28T00:15:02.998 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:02 smithi119 podman[31391]: 2024-03-28 00:15:02.782771567 +0000 UTC m=+0.909267873 container cleanup fa1f99fa62753d0042c3e8f28132daac3b3e9a6fc6d307fb8d75a3b7f636c775 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mon-a, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-28T00:15:02.998 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:02 smithi119 bash[31391]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mon-a 2024-03-28T00:15:03.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:02 smithi119 podman[31405]: 2024-03-28 00:15:02.991795477 +0000 UTC m=+0.860879716 container remove fa1f99fa62753d0042c3e8f28132daac3b3e9a6fc6d307fb8d75a3b7f636c775 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mon-a, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.license=GPLv2) 2024-03-28T00:15:03.847 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:03 smithi119 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mon.a.service: Deactivated successfully. 2024-03-28T00:15:03.847 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:03 smithi119 systemd[1]: Stopped Ceph mon.a for 10f73b92-ec98-11ee-95d0-87774f69a715. 2024-03-28T00:15:03.847 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:03 smithi119 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mon.a.service: Consumed 1.823s CPU time. 2024-03-28T00:15:03.847 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:03 smithi119 systemd[1]: Starting Ceph mon.a for 10f73b92-ec98-11ee-95d0-87774f69a715... 2024-03-28T00:15:04.108 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:03 smithi119 podman[31499]: 2024-03-28 00:15:03.745664459 +0000 UTC m=+0.033695523 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 2024-03-28T00:15:04.108 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:03 smithi119 podman[31499]: 2024-03-28 00:15:03.842872829 +0000 UTC m=+0.130903939 container create 746fea8df4c21d73b15a5bde181da80e3c3ae150443b0a6041213fc1c0286b2c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mon-a, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, CEPH_POINT_RELEASE=) 2024-03-28T00:15:04.108 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 podman[31499]: 2024-03-28 00:15:04.089933339 +0000 UTC m=+0.377964466 container init 746fea8df4c21d73b15a5bde181da80e3c3ae150443b0a6041213fc1c0286b2c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mon-a, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-28T00:15:04.108 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 podman[31499]: 2024-03-28 00:15:04.101515201 +0000 UTC m=+0.389546271 container start 746fea8df4c21d73b15a5bde181da80e3c3ae150443b0a6041213fc1c0286b2c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mon-a, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True) 2024-03-28T00:15:04.195 INFO:teuthology.orchestra.run.smithi119.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-03-28T00:15:04.408 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: set uid:gid to 167:167 (ceph:ceph) 2024-03-28T00:15:04.409 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev), process ceph-mon, pid 2 2024-03-28T00:15:04.409 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: pidfile_write: ignore empty --pid-file 2024-03-28T00:15:04.409 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: load: jerasure load: lrc 2024-03-28T00:15:04.409 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: RocksDB version: 7.9.2 2024-03-28T00:15:04.409 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Git sha 0 2024-03-28T00:15:04.409 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Compile date 2024-03-22 04:58:55 2024-03-28T00:15:04.409 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: DB SUMMARY 2024-03-28T00:15:04.409 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: DB Session ID: 9OV4OK2DS0IA6YIXAJ55 2024-03-28T00:15:04.409 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: CURRENT file: CURRENT 2024-03-28T00:15:04.409 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: IDENTITY file: IDENTITY 2024-03-28T00:15:04.409 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-03-28T00:15:04.409 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-03-28T00:15:04.409 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 100652 ; 2024-03-28T00:15:04.409 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.error_if_exists: 0 2024-03-28T00:15:04.410 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.create_if_missing: 0 2024-03-28T00:15:04.410 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.paranoid_checks: 1 2024-03-28T00:15:04.410 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.flush_verify_memtable_count: 1 2024-03-28T00:15:04.410 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-28T00:15:04.410 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-03-28T00:15:04.410 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.env: 0x555e738e7b20 2024-03-28T00:15:04.410 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.fs: PosixFileSystem 2024-03-28T00:15:04.410 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.info_log: 0x555e74f8d960 2024-03-28T00:15:04.410 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.max_file_opening_threads: 16 2024-03-28T00:15:04.410 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.statistics: (nil) 2024-03-28T00:15:04.410 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.use_fsync: 0 2024-03-28T00:15:04.410 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.max_log_file_size: 0 2024-03-28T00:15:04.410 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-28T00:15:04.410 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-28T00:15:04.410 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.keep_log_file_num: 1000 2024-03-28T00:15:04.411 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.recycle_log_file_num: 0 2024-03-28T00:15:04.411 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.allow_fallocate: 1 2024-03-28T00:15:04.411 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.allow_mmap_reads: 0 2024-03-28T00:15:04.411 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.allow_mmap_writes: 0 2024-03-28T00:15:04.411 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.use_direct_reads: 0 2024-03-28T00:15:04.411 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-28T00:15:04.411 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.create_missing_column_families: 0 2024-03-28T00:15:04.411 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.db_log_dir: 2024-03-28T00:15:04.411 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.wal_dir: 2024-03-28T00:15:04.411 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-28T00:15:04.412 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-28T00:15:04.412 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-28T00:15:04.412 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-28T00:15:04.412 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-28T00:15:04.412 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-28T00:15:04.412 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.advise_random_on_open: 1 2024-03-28T00:15:04.412 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.db_write_buffer_size: 0 2024-03-28T00:15:04.412 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.write_buffer_manager: 0x555e74f601e0 2024-03-28T00:15:04.412 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-28T00:15:04.413 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-28T00:15:04.413 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-28T00:15:04.413 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.rate_limiter: (nil) 2024-03-28T00:15:04.413 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-28T00:15:04.413 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.wal_recovery_mode: 2 2024-03-28T00:15:04.413 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.enable_thread_tracking: 0 2024-03-28T00:15:04.413 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.enable_pipelined_write: 0 2024-03-28T00:15:04.413 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.unordered_write: 0 2024-03-28T00:15:04.413 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-28T00:15:04.414 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-28T00:15:04.414 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-28T00:15:04.414 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-28T00:15:04.414 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.row_cache: None 2024-03-28T00:15:04.414 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.wal_filter: None 2024-03-28T00:15:04.414 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-28T00:15:04.414 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.allow_ingest_behind: 0 2024-03-28T00:15:04.414 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.two_write_queues: 0 2024-03-28T00:15:04.414 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.manual_wal_flush: 0 2024-03-28T00:15:04.414 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.wal_compression: 0 2024-03-28T00:15:04.414 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.atomic_flush: 0 2024-03-28T00:15:04.414 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-28T00:15:04.414 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-28T00:15:04.415 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-28T00:15:04.415 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.log_readahead_size: 0 2024-03-28T00:15:04.415 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-28T00:15:04.415 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.best_efforts_recovery: 0 2024-03-28T00:15:04.415 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-28T00:15:04.415 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-28T00:15:04.415 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.allow_data_in_errors: 0 2024-03-28T00:15:04.415 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.db_host_id: __hostname__ 2024-03-28T00:15:04.415 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.enforce_single_del_contracts: true 2024-03-28T00:15:04.415 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.max_background_jobs: 2 2024-03-28T00:15:04.415 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.max_background_compactions: -1 2024-03-28T00:15:04.415 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.max_subcompactions: 1 2024-03-28T00:15:04.416 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-28T00:15:04.416 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-28T00:15:04.416 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-28T00:15:04.416 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.max_total_wal_size: 0 2024-03-28T00:15:04.416 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-28T00:15:04.416 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-28T00:15:04.416 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-28T00:15:04.416 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-28T00:15:04.416 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.max_open_files: -1 2024-03-28T00:15:04.416 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.bytes_per_sync: 0 2024-03-28T00:15:04.416 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-28T00:15:04.416 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-28T00:15:04.416 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.compaction_readahead_size: 0 2024-03-28T00:15:04.416 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.max_background_flushes: -1 2024-03-28T00:15:04.416 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Compression algorithms supported: 2024-03-28T00:15:04.417 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-28T00:15:04.417 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: kZSTD supported: 0 2024-03-28T00:15:04.417 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: kXpressCompression supported: 0 2024-03-28T00:15:04.417 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: kLZ4HCCompression supported: 1 2024-03-28T00:15:04.417 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: kZlibCompression supported: 1 2024-03-28T00:15:04.417 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: kSnappyCompression supported: 1 2024-03-28T00:15:04.417 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: kLZ4Compression supported: 1 2024-03-28T00:15:04.417 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: kBZip2Compression supported: 0 2024-03-28T00:15:04.417 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-28T00:15:04.417 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: DMutex implementation: pthread_mutex_t 2024-03-28T00:15:04.417 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-03-28T00:15:04.417 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-03-28T00:15:04.417 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-28T00:15:04.417 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.merge_operator: 2024-03-28T00:15:04.418 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.compaction_filter: None 2024-03-28T00:15:04.418 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.compaction_filter_factory: None 2024-03-28T00:15:04.418 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.sst_partitioner_factory: None 2024-03-28T00:15:04.418 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-28T00:15:04.418 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-28T00:15:04.418 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x555e74f8da60) 2024-03-28T00:15:04.418 INFO:journalctl@ceph.mon.a.smithi119.stdout: cache_index_and_filter_blocks: 1 2024-03-28T00:15:04.418 INFO:journalctl@ceph.mon.a.smithi119.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-28T00:15:04.418 INFO:journalctl@ceph.mon.a.smithi119.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-28T00:15:04.418 INFO:journalctl@ceph.mon.a.smithi119.stdout: pin_top_level_index_and_filter: 1 2024-03-28T00:15:04.418 INFO:journalctl@ceph.mon.a.smithi119.stdout: index_type: 0 2024-03-28T00:15:04.418 INFO:journalctl@ceph.mon.a.smithi119.stdout: data_block_index_type: 0 2024-03-28T00:15:04.418 INFO:journalctl@ceph.mon.a.smithi119.stdout: index_shortening: 1 2024-03-28T00:15:04.418 INFO:journalctl@ceph.mon.a.smithi119.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-28T00:15:04.419 INFO:journalctl@ceph.mon.a.smithi119.stdout: checksum: 4 2024-03-28T00:15:04.419 INFO:journalctl@ceph.mon.a.smithi119.stdout: no_block_cache: 0 2024-03-28T00:15:04.419 INFO:journalctl@ceph.mon.a.smithi119.stdout: block_cache: 0x555e74f93090 2024-03-28T00:15:04.419 INFO:journalctl@ceph.mon.a.smithi119.stdout: block_cache_name: BinnedLRUCache 2024-03-28T00:15:04.419 INFO:journalctl@ceph.mon.a.smithi119.stdout: block_cache_options: 2024-03-28T00:15:04.419 INFO:journalctl@ceph.mon.a.smithi119.stdout: capacity : 536870912 2024-03-28T00:15:04.419 INFO:journalctl@ceph.mon.a.smithi119.stdout: num_shard_bits : 4 2024-03-28T00:15:04.419 INFO:journalctl@ceph.mon.a.smithi119.stdout: strict_capacity_limit : 0 2024-03-28T00:15:04.419 INFO:journalctl@ceph.mon.a.smithi119.stdout: high_pri_pool_ratio: 0.000 2024-03-28T00:15:04.419 INFO:journalctl@ceph.mon.a.smithi119.stdout: block_cache_compressed: (nil) 2024-03-28T00:15:04.419 INFO:journalctl@ceph.mon.a.smithi119.stdout: persistent_cache: (nil) 2024-03-28T00:15:04.419 INFO:journalctl@ceph.mon.a.smithi119.stdout: block_size: 4096 2024-03-28T00:15:04.419 INFO:journalctl@ceph.mon.a.smithi119.stdout: block_size_deviation: 10 2024-03-28T00:15:04.419 INFO:journalctl@ceph.mon.a.smithi119.stdout: block_restart_interval: 16 2024-03-28T00:15:04.419 INFO:journalctl@ceph.mon.a.smithi119.stdout: index_block_restart_interval: 1 2024-03-28T00:15:04.420 INFO:journalctl@ceph.mon.a.smithi119.stdout: metadata_block_size: 4096 2024-03-28T00:15:04.420 INFO:journalctl@ceph.mon.a.smithi119.stdout: partition_filters: 0 2024-03-28T00:15:04.420 INFO:journalctl@ceph.mon.a.smithi119.stdout: use_delta_encoding: 1 2024-03-28T00:15:04.420 INFO:journalctl@ceph.mon.a.smithi119.stdout: filter_policy: bloomfilter 2024-03-28T00:15:04.420 INFO:journalctl@ceph.mon.a.smithi119.stdout: whole_key_filtering: 1 2024-03-28T00:15:04.420 INFO:journalctl@ceph.mon.a.smithi119.stdout: verify_compression: 0 2024-03-28T00:15:04.420 INFO:journalctl@ceph.mon.a.smithi119.stdout: read_amp_bytes_per_bit: 0 2024-03-28T00:15:04.420 INFO:journalctl@ceph.mon.a.smithi119.stdout: format_version: 5 2024-03-28T00:15:04.420 INFO:journalctl@ceph.mon.a.smithi119.stdout: enable_index_compression: 1 2024-03-28T00:15:04.420 INFO:journalctl@ceph.mon.a.smithi119.stdout: block_align: 0 2024-03-28T00:15:04.420 INFO:journalctl@ceph.mon.a.smithi119.stdout: max_auto_readahead_size: 262144 2024-03-28T00:15:04.420 INFO:journalctl@ceph.mon.a.smithi119.stdout: prepopulate_block_cache: 0 2024-03-28T00:15:04.420 INFO:journalctl@ceph.mon.a.smithi119.stdout: initial_auto_readahead_size: 8192 2024-03-28T00:15:04.420 INFO:journalctl@ceph.mon.a.smithi119.stdout: num_file_reads_for_auto_readahead: 2 2024-03-28T00:15:04.420 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.write_buffer_size: 33554432 2024-03-28T00:15:04.421 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.max_write_buffer_number: 2 2024-03-28T00:15:04.421 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.compression: NoCompression 2024-03-28T00:15:04.421 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.bottommost_compression: Disabled 2024-03-28T00:15:04.421 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.prefix_extractor: nullptr 2024-03-28T00:15:04.421 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-28T00:15:04.421 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.num_levels: 7 2024-03-28T00:15:04.421 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-28T00:15:04.421 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-28T00:15:04.421 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-28T00:15:04.421 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-28T00:15:04.421 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-28T00:15:04.421 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-28T00:15:04.421 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-28T00:15:04.421 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-28T00:15:04.422 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-28T00:15:04.422 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-28T00:15:04.422 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-03-28T00:15:04.423 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-03-28T00:15:04.423 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-28T00:15:04.423 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.compression_opts.level: 32767 2024-03-28T00:15:04.423 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.compression_opts.strategy: 0 2024-03-28T00:15:04.423 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-28T00:15:04.423 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-28T00:15:04.423 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-03-28T00:15:04.423 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-28T00:15:04.423 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.compression_opts.enabled: false 2024-03-28T00:15:04.423 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-03-28T00:15:04.423 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-28T00:15:04.423 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-28T00:15:04.423 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-28T00:15:04.424 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.target_file_size_base: 67108864 2024-03-28T00:15:04.424 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-28T00:15:04.424 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-28T00:15:04.424 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-28T00:15:04.424 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-28T00:15:04.424 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-28T00:15:04.424 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-28T00:15:04.424 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-28T00:15:04.424 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-28T00:15:04.424 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-28T00:15:04.424 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-28T00:15:04.424 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-28T00:15:04.424 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-28T00:15:04.424 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-28T00:15:04.425 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-03-28T00:15:04.425 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.arena_block_size: 1048576 2024-03-28T00:15:04.425 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-28T00:15:04.425 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-28T00:15:04.425 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.disable_auto_compactions: 0 2024-03-28T00:15:04.425 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-28T00:15:04.425 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-28T00:15:04.425 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-28T00:15:04.425 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-28T00:15:04.425 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-28T00:15:04.425 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-28T00:15:04.425 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-28T00:15:04.425 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-28T00:15:04.425 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-28T00:15:04.425 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-28T00:15:04.426 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.table_properties_collectors: 2024-03-28T00:15:04.426 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.inplace_update_support: 0 2024-03-28T00:15:04.426 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-28T00:15:04.426 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-28T00:15:04.426 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-28T00:15:04.426 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-28T00:15:04.426 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.bloom_locality: 0 2024-03-28T00:15:04.426 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.max_successive_merges: 0 2024-03-28T00:15:04.426 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-28T00:15:04.426 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.paranoid_file_checks: 0 2024-03-28T00:15:04.426 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.force_consistency_checks: 1 2024-03-28T00:15:04.426 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.report_bg_io_stats: 0 2024-03-28T00:15:04.426 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.ttl: 2592000 2024-03-28T00:15:04.427 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-28T00:15:04.427 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-03-28T00:15:04.427 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-03-28T00:15:04.427 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.enable_blob_files: false 2024-03-28T00:15:04.427 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.min_blob_size: 0 2024-03-28T00:15:04.427 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.blob_file_size: 268435456 2024-03-28T00:15:04.427 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-28T00:15:04.427 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-28T00:15:04.427 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-28T00:15:04.427 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-03-28T00:15:04.427 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-03-28T00:15:04.427 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.blob_file_starting_level: 0 2024-03-28T00:15:04.427 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-03-28T00:15:04.427 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-03-28T00:15:04.427 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-03-28T00:15:04.428 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 387345ca-5036-4df6-a31e-d8f45465d79f 2024-03-28T00:15:04.428 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711584904161089, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-03-28T00:15:04.428 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-03-28T00:15:04.428 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711584904163807, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 95773, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 318, "table_properties": {"data_size": 93759, "index_size": 296, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13904, "raw_average_key_size": 50, "raw_value_size": 86219, "raw_average_value_size": 311, "num_data_blocks": 11, "num_entries": 277, "num_filter_entries": 277, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1711584904, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "387345ca-5036-4df6-a31e-d8f45465d79f", "db_session_id": "9OV4OK2DS0IA6YIXAJ55", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-03-28T00:15:04.428 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711584904164025, "job": 1, "event": "recovery_finished"} 2024-03-28T00:15:04.428 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-03-28T00:15:04.428 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-28T00:15:04.428 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x555e75086000 2024-03-28T00:15:04.428 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: DB pointer 0x555e75070000 2024-03-28T00:15:04.428 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-28T00:15:04.428 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-28T00:15:04.428 INFO:journalctl@ceph.mon.a.smithi119.stdout: ** DB Stats ** 2024-03-28T00:15:04.428 INFO:journalctl@ceph.mon.a.smithi119.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-28T00:15:04.428 INFO:journalctl@ceph.mon.a.smithi119.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-28T00:15:04.429 INFO:journalctl@ceph.mon.a.smithi119.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-28T00:15:04.429 INFO:journalctl@ceph.mon.a.smithi119.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-28T00:15:04.429 INFO:journalctl@ceph.mon.a.smithi119.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-28T00:15:04.429 INFO:journalctl@ceph.mon.a.smithi119.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-28T00:15:04.429 INFO:journalctl@ceph.mon.a.smithi119.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-28T00:15:04.429 INFO:journalctl@ceph.mon.a.smithi119.stdout: 2024-03-28T00:15:04.429 INFO:journalctl@ceph.mon.a.smithi119.stdout: ** Compaction Stats [default] ** 2024-03-28T00:15:04.429 INFO:journalctl@ceph.mon.a.smithi119.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-28T00:15:04.429 INFO:journalctl@ceph.mon.a.smithi119.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-28T00:15:04.429 INFO:journalctl@ceph.mon.a.smithi119.stdout: L0 2/0 95.36 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 54.2 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-03-28T00:15:04.429 INFO:journalctl@ceph.mon.a.smithi119.stdout: Sum 2/0 95.36 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 54.2 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-03-28T00:15:04.429 INFO:journalctl@ceph.mon.a.smithi119.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 54.2 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-03-28T00:15:04.429 INFO:journalctl@ceph.mon.a.smithi119.stdout: 2024-03-28T00:15:04.429 INFO:journalctl@ceph.mon.a.smithi119.stdout: ** Compaction Stats [default] ** 2024-03-28T00:15:04.430 INFO:journalctl@ceph.mon.a.smithi119.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-28T00:15:04.430 INFO:journalctl@ceph.mon.a.smithi119.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-28T00:15:04.430 INFO:journalctl@ceph.mon.a.smithi119.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 54.2 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-03-28T00:15:04.430 INFO:journalctl@ceph.mon.a.smithi119.stdout: 2024-03-28T00:15:04.430 INFO:journalctl@ceph.mon.a.smithi119.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-28T00:15:04.430 INFO:journalctl@ceph.mon.a.smithi119.stdout: 2024-03-28T00:15:04.430 INFO:journalctl@ceph.mon.a.smithi119.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-28T00:15:04.430 INFO:journalctl@ceph.mon.a.smithi119.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-03-28T00:15:04.430 INFO:journalctl@ceph.mon.a.smithi119.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-28T00:15:04.430 INFO:journalctl@ceph.mon.a.smithi119.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-28T00:15:04.430 INFO:journalctl@ceph.mon.a.smithi119.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-28T00:15:04.430 INFO:journalctl@ceph.mon.a.smithi119.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-28T00:15:04.430 INFO:journalctl@ceph.mon.a.smithi119.stdout: Cumulative compaction: 0.00 GB write, 9.23 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-28T00:15:04.430 INFO:journalctl@ceph.mon.a.smithi119.stdout: Interval compaction: 0.00 GB write, 9.23 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-28T00:15:04.430 INFO:journalctl@ceph.mon.a.smithi119.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-28T00:15:04.431 INFO:journalctl@ceph.mon.a.smithi119.stdout: Block cache BinnedLRUCache@0x555e74f93090#2 capacity: 512.00 MB usage: 1.39 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.2e-05 secs_since: 0 2024-03-28T00:15:04.431 INFO:journalctl@ceph.mon.a.smithi119.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.89 KB,0.000169873%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-03-28T00:15:04.431 INFO:journalctl@ceph.mon.a.smithi119.stdout: 2024-03-28T00:15:04.431 INFO:journalctl@ceph.mon.a.smithi119.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-28T00:15:04.431 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: starting mon.a rank 0 at public addrs [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] at bind addrs [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 10f73b92-ec98-11ee-95d0-87774f69a715 2024-03-28T00:15:04.431 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: mon.a@-1(???) e1 preinit fsid 10f73b92-ec98-11ee-95d0-87774f69a715 2024-03-28T00:15:04.431 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: mon.a@-1(???).mds e1 new map 2024-03-28T00:15:04.431 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: mon.a@-1(???).mds e1 print_map 2024-03-28T00:15:04.431 INFO:journalctl@ceph.mon.a.smithi119.stdout: e1 2024-03-28T00:15:04.431 INFO:journalctl@ceph.mon.a.smithi119.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-28T00:15:04.431 INFO:journalctl@ceph.mon.a.smithi119.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-03-28T00:15:04.431 INFO:journalctl@ceph.mon.a.smithi119.stdout: legacy client fscid: -1 2024-03-28T00:15:04.431 INFO:journalctl@ceph.mon.a.smithi119.stdout: 2024-03-28T00:15:04.431 INFO:journalctl@ceph.mon.a.smithi119.stdout: No filesystems configured 2024-03-28T00:15:04.431 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-03-28T00:15:04.432 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-28T00:15:04.432 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-28T00:15:04.432 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-28T00:15:04.432 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-03-28T00:15:04.432 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 bash[31499]: 746fea8df4c21d73b15a5bde181da80e3c3ae150443b0a6041213fc1c0286b2c 2024-03-28T00:15:04.432 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 systemd[1]: Started Ceph mon.a for 10f73b92-ec98-11ee-95d0-87774f69a715. 2024-03-28T00:15:04.432 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: mon.a is new leader, mons a in quorum (ranks 0) 2024-03-28T00:15:04.432 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: monmap e1: 1 mons at {a=[v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-28T00:15:04.432 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: fsmap 2024-03-28T00:15:04.432 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: osdmap e1: 0 total, 0 up, 0 in 2024-03-28T00:15:04.432 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:04 smithi119 ceph-mon[31518]: mgrmap e1: no daemons active 2024-03-28T00:15:05.583 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:04.807+0000 7f6a9a200700 1 Processor -- start 2024-03-28T00:15:05.588 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:04.807+0000 7f6a9a200700 1 -- start start 2024-03-28T00:15:05.588 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:04.808+0000 7f6a9a200700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6a94107920 0x7f6a94107d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:15:05.588 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:04.808+0000 7f6a9a200700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a94108310 con 0x7f6a94107920 2024-03-28T00:15:05.588 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:04.808+0000 7f6a98e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6a94107920 0x7f6a94107d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:15:05.588 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:04.808+0000 7f6a98e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6a94107920 0x7f6a94107d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:36454/0 (socket says 172.21.15.119:36454) 2024-03-28T00:15:05.588 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:04.808+0000 7f6a98e00700 1 -- 172.21.15.119:0/1252851077 learned_addr learned my addr 172.21.15.119:0/1252851077 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:15:05.588 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:04.809+0000 7f6a98e00700 1 -- 172.21.15.119:0/1252851077 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a94108b20 con 0x7f6a94107920 2024-03-28T00:15:05.588 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:04.809+0000 7f6a98e00700 1 --2- 172.21.15.119:0/1252851077 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6a94107920 0x7f6a94107d40 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f6a880098a0 tx=0x7f6a88009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3108177d508eb9e server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:15:05.588 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:04.810+0000 7f6a93400700 1 -- 172.21.15.119:0/1252851077 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6a88004160 con 0x7f6a94107920 2024-03-28T00:15:05.589 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:04.810+0000 7f6a93400700 1 -- 172.21.15.119:0/1252851077 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6a88004300 con 0x7f6a94107920 2024-03-28T00:15:05.589 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:04.810+0000 7f6a93400700 1 -- 172.21.15.119:0/1252851077 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6a88011b50 con 0x7f6a94107920 2024-03-28T00:15:05.589 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:04.810+0000 7f6a9a200700 1 -- 172.21.15.119:0/1252851077 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6a94107920 msgr2=0x7f6a94107d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:15:05.589 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:04.810+0000 7f6a9a200700 1 --2- 172.21.15.119:0/1252851077 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6a94107920 0x7f6a94107d40 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f6a880098a0 tx=0x7f6a88009bb0 comp rx=0 tx=0).stop 2024-03-28T00:15:05.589 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:04.810+0000 7f6a9a200700 1 -- 172.21.15.119:0/1252851077 shutdown_connections 2024-03-28T00:15:05.589 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:04.810+0000 7f6a9a200700 1 --2- 172.21.15.119:0/1252851077 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6a94107920 0x7f6a94107d40 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:15:05.589 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:04.811+0000 7f6a9a200700 1 -- 172.21.15.119:0/1252851077 >> 172.21.15.119:0/1252851077 conn(0x7f6a94102ea0 msgr2=0x7f6a94105300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:15:05.589 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:04.811+0000 7f6a9a200700 1 -- 172.21.15.119:0/1252851077 shutdown_connections 2024-03-28T00:15:05.589 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:04.811+0000 7f6a9a200700 1 -- 172.21.15.119:0/1252851077 wait complete. 2024-03-28T00:15:05.589 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:04.812+0000 7f6a9a200700 1 Processor -- start 2024-03-28T00:15:05.589 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:04.812+0000 7f6a9a200700 1 -- start start 2024-03-28T00:15:05.589 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:04.812+0000 7f6a9a200700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6a94107920 0x7f6a9419db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:15:05.589 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:04.812+0000 7f6a9a200700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a9410bed0 con 0x7f6a94107920 2024-03-28T00:15:05.590 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:04.812+0000 7f6a98e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6a94107920 0x7f6a9419db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:15:05.590 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:04.812+0000 7f6a98e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6a94107920 0x7f6a9419db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:36456/0 (socket says 172.21.15.119:36456) 2024-03-28T00:15:05.590 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:04.812+0000 7f6a98e00700 1 -- 172.21.15.119:0/2502564597 learned_addr learned my addr 172.21.15.119:0/2502564597 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:15:05.590 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:04.813+0000 7f6a98e00700 1 -- 172.21.15.119:0/2502564597 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a9419e050 con 0x7f6a94107920 2024-03-28T00:15:05.590 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:04.813+0000 7f6a98e00700 1 --2- 172.21.15.119:0/2502564597 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6a94107920 0x7f6a9419db10 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f6a88011670 tx=0x7f6a88011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:15:05.590 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:04.813+0000 7f6a91600700 1 -- 172.21.15.119:0/2502564597 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6a880039c0 con 0x7f6a94107920 2024-03-28T00:15:05.590 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:04.813+0000 7f6a91600700 1 -- 172.21.15.119:0/2502564597 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6a88011e50 con 0x7f6a94107920 2024-03-28T00:15:05.590 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:04.813+0000 7f6a9a200700 1 -- 172.21.15.119:0/2502564597 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a9419e2e0 con 0x7f6a94107920 2024-03-28T00:15:05.590 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:04.814+0000 7f6a91600700 1 -- 172.21.15.119:0/2502564597 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6a8801a5c0 con 0x7f6a94107920 2024-03-28T00:15:05.590 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:04.814+0000 7f6a9a200700 1 -- 172.21.15.119:0/2502564597 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a941a0fd0 con 0x7f6a94107920 2024-03-28T00:15:05.590 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:04.814+0000 7f6a91600700 1 -- 172.21.15.119:0/2502564597 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f6a8801ac50 con 0x7f6a94107920 2024-03-28T00:15:05.590 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:04.814+0000 7f6a91600700 1 -- 172.21.15.119:0/2502564597 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f6a88019700 con 0x7f6a94107920 2024-03-28T00:15:05.591 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:04.815+0000 7f6a9a200700 1 -- 172.21.15.119:0/2502564597 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a94069260 con 0x7f6a94107920 2024-03-28T00:15:05.591 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:04.818+0000 7f6a91600700 1 -- 172.21.15.119:0/2502564597 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f6a88003b60 con 0x7f6a94107920 2024-03-28T00:15:05.591 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:04.893+0000 7f6a9a200700 1 -- 172.21.15.119:0/2502564597 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7f6a941a11a0 con 0x7f6a94107920 2024-03-28T00:15:05.591 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:04.898+0000 7f6a91600700 1 -- 172.21.15.119:0/2502564597 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7f6a88019900 con 0x7f6a94107920 2024-03-28T00:15:05.591 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:04.899+0000 7f6a9a200700 1 -- 172.21.15.119:0/2502564597 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6a94107920 msgr2=0x7f6a9419db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:15:05.591 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:04.899+0000 7f6a9a200700 1 --2- 172.21.15.119:0/2502564597 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6a94107920 0x7f6a9419db10 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f6a88011670 tx=0x7f6a88011b60 comp rx=0 tx=0).stop 2024-03-28T00:15:05.591 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:04.899+0000 7f6a9a200700 1 -- 172.21.15.119:0/2502564597 shutdown_connections 2024-03-28T00:15:05.591 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:04.899+0000 7f6a9a200700 1 --2- 172.21.15.119:0/2502564597 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6a94107920 0x7f6a9419db10 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:15:05.591 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:04.899+0000 7f6a9a200700 1 -- 172.21.15.119:0/2502564597 >> 172.21.15.119:0/2502564597 conn(0x7f6a94102ea0 msgr2=0x7f6a94103970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:15:05.591 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:04.900+0000 7f6a9a200700 1 -- 172.21.15.119:0/2502564597 shutdown_connections 2024-03-28T00:15:05.591 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:04.900+0000 7f6a9a200700 1 -- 172.21.15.119:0/2502564597 wait complete. 2024-03-28T00:15:05.591 INFO:teuthology.orchestra.run.smithi119.stdout:Wrote config to /etc/ceph/ceph.conf 2024-03-28T00:15:05.614 INFO:teuthology.orchestra.run.smithi119.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-03-28T00:15:05.614 INFO:teuthology.orchestra.run.smithi119.stdout:Creating mgr... 2024-03-28T00:15:05.616 INFO:teuthology.orchestra.run.smithi119.stdout:Verifying port 0.0.0.0:9283 ... 2024-03-28T00:15:05.617 INFO:teuthology.orchestra.run.smithi119.stdout:Verifying port 0.0.0.0:8765 ... 2024-03-28T00:15:06.019 INFO:teuthology.orchestra.run.smithi119.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mgr.x 2024-03-28T00:15:06.019 INFO:teuthology.orchestra.run.smithi119.stdout:systemctl: stderr Failed to reset failed state of unit ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mgr.x.service: Unit ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mgr.x.service not loaded. 2024-03-28T00:15:06.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:05 smithi119 ceph-mon[31518]: from='client.? 172.21.15.119:0/2502564597' entity='client.admin' 2024-03-28T00:15:06.288 INFO:teuthology.orchestra.run.smithi119.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-10f73b92-ec98-11ee-95d0-87774f69a715.target.wants/ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mgr.x.service → /etc/systemd/system/ceph-10f73b92-ec98-11ee-95d0-87774f69a715@.service. 2024-03-28T00:15:06.656 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 00:15:06 smithi119 systemd[1]: Starting Ceph mgr.x for 10f73b92-ec98-11ee-95d0-87774f69a715... 2024-03-28T00:15:06.971 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 00:15:06 smithi119 podman[31734]: 2024-03-28 00:15:06.582389394 +0000 UTC m=+0.033261870 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 2024-03-28T00:15:06.971 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 00:15:06 smithi119 podman[31734]: 2024-03-28 00:15:06.703203421 +0000 UTC m=+0.154075915 container create f628950cf3c0735f866da143ab033c0cc9fb1e088fc1b45918c638f1a6fb596b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-x, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.build-date=20240319, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-28T00:15:07.064 INFO:teuthology.orchestra.run.smithi119.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-28T00:15:07.064 INFO:teuthology.orchestra.run.smithi119.stdout:systemctl: stdout disabled 2024-03-28T00:15:07.081 INFO:teuthology.orchestra.run.smithi119.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-28T00:15:07.081 INFO:teuthology.orchestra.run.smithi119.stdout:systemctl: stdout inactive 2024-03-28T00:15:07.081 INFO:teuthology.orchestra.run.smithi119.stdout:firewalld.service is not enabled 2024-03-28T00:15:07.082 INFO:teuthology.orchestra.run.smithi119.stdout:Not possible to enable service . firewalld.service is not available 2024-03-28T00:15:07.099 INFO:teuthology.orchestra.run.smithi119.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-28T00:15:07.099 INFO:teuthology.orchestra.run.smithi119.stdout:systemctl: stdout disabled 2024-03-28T00:15:07.115 INFO:teuthology.orchestra.run.smithi119.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-28T00:15:07.115 INFO:teuthology.orchestra.run.smithi119.stdout:systemctl: stdout inactive 2024-03-28T00:15:07.115 INFO:teuthology.orchestra.run.smithi119.stdout:firewalld.service is not enabled 2024-03-28T00:15:07.115 INFO:teuthology.orchestra.run.smithi119.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-03-28T00:15:07.115 INFO:teuthology.orchestra.run.smithi119.stdout:Waiting for mgr to start... 2024-03-28T00:15:07.115 INFO:teuthology.orchestra.run.smithi119.stdout:Waiting for mgr... 2024-03-28T00:15:07.270 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 00:15:06 smithi119 podman[31734]: 2024-03-28 00:15:06.965569093 +0000 UTC m=+0.416441594 container init f628950cf3c0735f866da143ab033c0cc9fb1e088fc1b45918c638f1a6fb596b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-x, org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-03-28T00:15:07.270 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 00:15:06 smithi119 podman[31734]: 2024-03-28 00:15:06.975253553 +0000 UTC m=+0.426126012 container start f628950cf3c0735f866da143ab033c0cc9fb1e088fc1b45918c638f1a6fb596b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-x, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-28T00:15:07.270 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 00:15:07 smithi119 bash[31734]: f628950cf3c0735f866da143ab033c0cc9fb1e088fc1b45918c638f1a6fb596b 2024-03-28T00:15:07.271 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 00:15:07 smithi119 systemd[1]: Started Ceph mgr.x for 10f73b92-ec98-11ee-95d0-87774f69a715. 2024-03-28T00:15:07.941 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 00:15:07 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-x[31752]: 2024-03-28T00:15:07.855+0000 7f000d48c200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-28T00:15:08.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:08 smithi119 ceph-mon[31518]: from='client.? 172.21.15.119:0/3268542159' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-28T00:15:08.406 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 00:15:07 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-x[31752]: 2024-03-28T00:15:07.988+0000 7f000d48c200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-28T00:15:08.786 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout 2024-03-28T00:15:08.786 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout { 2024-03-28T00:15:08.787 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "fsid": "10f73b92-ec98-11ee-95d0-87774f69a715", 2024-03-28T00:15:08.787 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "health": { 2024-03-28T00:15:08.787 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-28T00:15:08.787 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-28T00:15:08.787 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-28T00:15:08.787 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout }, 2024-03-28T00:15:08.787 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-28T00:15:08.787 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-28T00:15:08.787 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout 0 2024-03-28T00:15:08.787 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout ], 2024-03-28T00:15:08.787 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-28T00:15:08.787 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "a" 2024-03-28T00:15:08.787 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout ], 2024-03-28T00:15:08.787 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2024-03-28T00:15:08.789 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-28T00:15:08.789 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-28T00:15:08.789 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-28T00:15:08.789 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-28T00:15:08.789 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout }, 2024-03-28T00:15:08.789 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-28T00:15:08.789 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-28T00:15:08.789 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-28T00:15:08.789 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-28T00:15:08.789 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-28T00:15:08.798 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-28T00:15:08.798 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-28T00:15:08.798 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-28T00:15:08.798 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout }, 2024-03-28T00:15:08.798 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-28T00:15:08.798 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-28T00:15:08.798 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-28T00:15:08.798 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-28T00:15:08.798 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-28T00:15:08.798 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-28T00:15:08.799 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-28T00:15:08.799 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-28T00:15:08.799 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-28T00:15:08.799 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout }, 2024-03-28T00:15:08.799 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-28T00:15:08.799 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-28T00:15:08.799 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-28T00:15:08.799 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-28T00:15:08.799 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout }, 2024-03-28T00:15:08.799 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-28T00:15:08.799 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-28T00:15:08.799 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-28T00:15:08.799 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-28T00:15:08.799 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-28T00:15:08.799 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-28T00:15:08.800 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "restful" 2024-03-28T00:15:08.800 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout ], 2024-03-28T00:15:08.800 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-28T00:15:08.800 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout }, 2024-03-28T00:15:08.800 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-28T00:15:08.800 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-28T00:15:08.800 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "modified": "2024-03-28T00:14:57.564140+0000", 2024-03-28T00:15:08.800 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-28T00:15:08.800 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout }, 2024-03-28T00:15:08.800 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-28T00:15:08.800 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout } 2024-03-28T00:15:08.800 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:07.974+0000 7fbf97000700 1 Processor -- start 2024-03-28T00:15:08.800 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:07.974+0000 7fbf97000700 1 -- start start 2024-03-28T00:15:08.800 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:07.974+0000 7fbf97000700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbf9007de90 0x7fbf9007c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:15:08.801 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:07.974+0000 7fbf97000700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf9007c8c0 con 0x7fbf9007de90 2024-03-28T00:15:08.801 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:07.975+0000 7fbf95c00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbf9007de90 0x7fbf9007c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:15:08.801 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:07.975+0000 7fbf95c00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbf9007de90 0x7fbf9007c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:36480/0 (socket says 172.21.15.119:36480) 2024-03-28T00:15:08.801 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:07.975+0000 7fbf95c00700 1 -- 172.21.15.119:0/315089487 learned_addr learned my addr 172.21.15.119:0/315089487 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:15:08.801 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:07.975+0000 7fbf95c00700 1 -- 172.21.15.119:0/315089487 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf9007ca40 con 0x7fbf9007de90 2024-03-28T00:15:08.801 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:07.976+0000 7fbf95c00700 1 --2- 172.21.15.119:0/315089487 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbf9007de90 0x7fbf9007c2f0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fbf840098a0 tx=0x7fbf84009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8af31ba4916d851f server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:15:08.801 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:07.976+0000 7fbf8fe00700 1 -- 172.21.15.119:0/315089487 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbf84004160 con 0x7fbf9007de90 2024-03-28T00:15:08.801 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:07.976+0000 7fbf8fe00700 1 -- 172.21.15.119:0/315089487 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbf84004300 con 0x7fbf9007de90 2024-03-28T00:15:08.801 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:07.976+0000 7fbf8fe00700 1 -- 172.21.15.119:0/315089487 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbf84011b50 con 0x7fbf9007de90 2024-03-28T00:15:08.801 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:07.977+0000 7fbf97000700 1 -- 172.21.15.119:0/315089487 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbf9007de90 msgr2=0x7fbf9007c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:15:08.801 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:07.977+0000 7fbf97000700 1 --2- 172.21.15.119:0/315089487 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbf9007de90 0x7fbf9007c2f0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fbf840098a0 tx=0x7fbf84009bb0 comp rx=0 tx=0).stop 2024-03-28T00:15:08.801 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:07.977+0000 7fbf97000700 1 -- 172.21.15.119:0/315089487 shutdown_connections 2024-03-28T00:15:08.801 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:07.977+0000 7fbf97000700 1 --2- 172.21.15.119:0/315089487 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbf9007de90 0x7fbf9007c2f0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:15:08.801 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:07.977+0000 7fbf97000700 1 -- 172.21.15.119:0/315089487 >> 172.21.15.119:0/315089487 conn(0x7fbf90102ec0 msgr2=0x7fbf901052e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:15:08.802 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:07.977+0000 7fbf97000700 1 -- 172.21.15.119:0/315089487 shutdown_connections 2024-03-28T00:15:08.802 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:07.977+0000 7fbf97000700 1 -- 172.21.15.119:0/315089487 wait complete. 2024-03-28T00:15:08.802 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:07.979+0000 7fbf97000700 1 Processor -- start 2024-03-28T00:15:08.802 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:07.979+0000 7fbf97000700 1 -- start start 2024-03-28T00:15:08.802 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:07.979+0000 7fbf97000700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbf9007de90 0x7fbf9019dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:15:08.802 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:07.979+0000 7fbf97000700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf9010df20 con 0x7fbf9007de90 2024-03-28T00:15:08.802 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:07.980+0000 7fbf95c00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbf9007de90 0x7fbf9019dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:15:08.802 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:07.980+0000 7fbf95c00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbf9007de90 0x7fbf9019dad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:36482/0 (socket says 172.21.15.119:36482) 2024-03-28T00:15:08.802 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:07.980+0000 7fbf95c00700 1 -- 172.21.15.119:0/3268542159 learned_addr learned my addr 172.21.15.119:0/3268542159 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:15:08.802 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:07.980+0000 7fbf95c00700 1 -- 172.21.15.119:0/3268542159 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf9019e010 con 0x7fbf9007de90 2024-03-28T00:15:08.802 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:07.980+0000 7fbf95c00700 1 --2- 172.21.15.119:0/3268542159 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbf9007de90 0x7fbf9019dad0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fbf84000c00 tx=0x7fbf84011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:15:08.802 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:07.981+0000 7fbf8e000700 1 -- 172.21.15.119:0/3268542159 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbf84003b20 con 0x7fbf9007de90 2024-03-28T00:15:08.802 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:07.981+0000 7fbf8e000700 1 -- 172.21.15.119:0/3268542159 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbf84003cc0 con 0x7fbf9007de90 2024-03-28T00:15:08.802 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:07.981+0000 7fbf8e000700 1 -- 172.21.15.119:0/3268542159 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbf84024910 con 0x7fbf9007de90 2024-03-28T00:15:08.803 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:07.981+0000 7fbf97000700 1 -- 172.21.15.119:0/3268542159 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf9019e2a0 con 0x7fbf9007de90 2024-03-28T00:15:08.803 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:07.981+0000 7fbf97000700 1 -- 172.21.15.119:0/3268542159 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf901a0f90 con 0x7fbf9007de90 2024-03-28T00:15:08.803 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:07.981+0000 7fbf8e000700 1 -- 172.21.15.119:0/3268542159 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fbf84022020 con 0x7fbf9007de90 2024-03-28T00:15:08.803 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:07.981+0000 7fbf8e000700 1 -- 172.21.15.119:0/3268542159 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fbf84011e00 con 0x7fbf9007de90 2024-03-28T00:15:08.803 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:07.982+0000 7fbf97000700 1 -- 172.21.15.119:0/3268542159 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf90069260 con 0x7fbf9007de90 2024-03-28T00:15:08.803 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:07.984+0000 7fbf8e000700 1 -- 172.21.15.119:0/3268542159 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fbf8401acd0 con 0x7fbf9007de90 2024-03-28T00:15:08.803 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:08.058+0000 7fbf97000700 1 -- 172.21.15.119:0/3268542159 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fbf901a1160 con 0x7fbf9007de90 2024-03-28T00:15:08.803 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:08.059+0000 7fbf8e000700 1 -- 172.21.15.119:0/3268542159 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7fbf84032030 con 0x7fbf9007de90 2024-03-28T00:15:08.803 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:08.060+0000 7fbf97000700 1 -- 172.21.15.119:0/3268542159 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbf9007de90 msgr2=0x7fbf9019dad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:15:08.803 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:08.060+0000 7fbf97000700 1 --2- 172.21.15.119:0/3268542159 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbf9007de90 0x7fbf9019dad0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fbf84000c00 tx=0x7fbf84011c10 comp rx=0 tx=0).stop 2024-03-28T00:15:08.803 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:08.060+0000 7fbf97000700 1 -- 172.21.15.119:0/3268542159 shutdown_connections 2024-03-28T00:15:08.803 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:08.060+0000 7fbf97000700 1 --2- 172.21.15.119:0/3268542159 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbf9007de90 0x7fbf9019dad0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:15:08.803 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:08.060+0000 7fbf97000700 1 -- 172.21.15.119:0/3268542159 >> 172.21.15.119:0/3268542159 conn(0x7fbf90102ec0 msgr2=0x7fbf90103990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:15:08.803 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:08.060+0000 7fbf97000700 1 -- 172.21.15.119:0/3268542159 shutdown_connections 2024-03-28T00:15:08.804 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:08.060+0000 7fbf97000700 1 -- 172.21.15.119:0/3268542159 wait complete. 2024-03-28T00:15:08.804 INFO:teuthology.orchestra.run.smithi119.stdout:mgr not available, waiting (1/15)... 2024-03-28T00:15:09.096 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 00:15:08 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-x[31752]: 2024-03-28T00:15:08.800+0000 7f000d48c200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-28T00:15:09.096 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 00:15:08 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-x[31752]: 2024-03-28T00:15:08.918+0000 7f000d48c200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-28T00:15:09.096 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 00:15:09 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-x[31752]: 2024-03-28T00:15:09.089+0000 7f000d48c200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-28T00:15:09.405 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 00:15:09 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-x[31752]: 2024-03-28T00:15:09.209+0000 7f000d48c200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-28T00:15:09.406 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 00:15:09 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-x[31752]: 2024-03-28T00:15:09.314+0000 7f000d48c200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-28T00:15:09.906 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 00:15:09 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-x[31752]: 2024-03-28T00:15:09.638+0000 7f000d48c200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-28T00:15:09.906 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 00:15:09 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-x[31752]: 2024-03-28T00:15:09.773+0000 7f000d48c200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-28T00:15:09.906 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 00:15:09 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-x[31752]: 2024-03-28T00:15:09.881+0000 7f000d48c200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-28T00:15:10.405 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 00:15:10 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-x[31752]: 2024-03-28T00:15:10.004+0000 7f000d48c200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-28T00:15:10.406 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 00:15:10 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-x[31752]: 2024-03-28T00:15:10.217+0000 7f000d48c200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-28T00:15:11.732 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 00:15:11 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-x[31752]: 2024-03-28T00:15:11.557+0000 7f000d48c200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-28T00:15:11.732 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:11 smithi119 ceph-mon[31518]: from='client.? 172.21.15.119:0/1737513274' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-28T00:15:12.151 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 00:15:11 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-x[31752]: 2024-03-28T00:15:11.725+0000 7f000d48c200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-28T00:15:12.210 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout 2024-03-28T00:15:12.211 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout { 2024-03-28T00:15:12.211 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "fsid": "10f73b92-ec98-11ee-95d0-87774f69a715", 2024-03-28T00:15:12.211 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "health": { 2024-03-28T00:15:12.211 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-28T00:15:12.211 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-28T00:15:12.211 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-28T00:15:12.211 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout }, 2024-03-28T00:15:12.211 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-28T00:15:12.211 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-28T00:15:12.211 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout 0 2024-03-28T00:15:12.211 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout ], 2024-03-28T00:15:12.211 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-28T00:15:12.212 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "a" 2024-03-28T00:15:12.212 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout ], 2024-03-28T00:15:12.212 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "quorum_age": 7, 2024-03-28T00:15:12.212 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-28T00:15:12.212 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-28T00:15:12.212 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-28T00:15:12.212 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-28T00:15:12.212 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout }, 2024-03-28T00:15:12.212 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-28T00:15:12.212 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-28T00:15:12.212 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-28T00:15:12.212 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-28T00:15:12.212 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-28T00:15:12.212 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-28T00:15:12.213 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-28T00:15:12.213 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-28T00:15:12.213 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout }, 2024-03-28T00:15:12.213 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-28T00:15:12.213 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-28T00:15:12.213 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-28T00:15:12.213 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-28T00:15:12.213 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-28T00:15:12.213 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-28T00:15:12.213 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-28T00:15:12.213 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-28T00:15:12.213 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-28T00:15:12.213 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout }, 2024-03-28T00:15:12.213 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-28T00:15:12.213 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-28T00:15:12.214 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-28T00:15:12.214 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-28T00:15:12.214 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout }, 2024-03-28T00:15:12.214 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-28T00:15:12.214 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-28T00:15:12.214 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-28T00:15:12.214 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-28T00:15:12.214 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-28T00:15:12.214 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-28T00:15:12.214 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "restful" 2024-03-28T00:15:12.214 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout ], 2024-03-28T00:15:12.214 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-28T00:15:12.214 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout }, 2024-03-28T00:15:12.214 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-28T00:15:12.215 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-28T00:15:12.215 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "modified": "2024-03-28T00:14:57.564140+0000", 2024-03-28T00:15:12.215 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-28T00:15:12.215 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout }, 2024-03-28T00:15:12.215 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-28T00:15:12.215 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout } 2024-03-28T00:15:12.215 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:11.343+0000 7f144ba00700 1 Processor -- start 2024-03-28T00:15:12.215 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:11.343+0000 7f144ba00700 1 -- start start 2024-03-28T00:15:12.215 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:11.344+0000 7f144ba00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1444107af0 0x7f1444107f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:15:12.215 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:11.344+0000 7f144ba00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14441084e0 con 0x7f1444107af0 2024-03-28T00:15:12.215 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:11.344+0000 7f1449000700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1444107af0 0x7f1444107f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:15:12.215 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:11.344+0000 7f1449000700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1444107af0 0x7f1444107f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:36488/0 (socket says 172.21.15.119:36488) 2024-03-28T00:15:12.215 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:11.344+0000 7f1449000700 1 -- 172.21.15.119:0/811533124 learned_addr learned my addr 172.21.15.119:0/811533124 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:15:12.215 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:11.345+0000 7f1449000700 1 -- 172.21.15.119:0/811533124 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1444108cf0 con 0x7f1444107af0 2024-03-28T00:15:12.216 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:11.345+0000 7f1449000700 1 --2- 172.21.15.119:0/811533124 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1444107af0 0x7f1444107f10 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f14380098a0 tx=0x7f1438009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c60172a67a19b339 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:15:12.216 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:11.345+0000 7f1443400700 1 -- 172.21.15.119:0/811533124 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1438004160 con 0x7f1444107af0 2024-03-28T00:15:12.216 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:11.345+0000 7f1443400700 1 -- 172.21.15.119:0/811533124 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1438004300 con 0x7f1444107af0 2024-03-28T00:15:12.216 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:11.345+0000 7f1443400700 1 -- 172.21.15.119:0/811533124 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1438011b50 con 0x7f1444107af0 2024-03-28T00:15:12.216 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:11.346+0000 7f144ba00700 1 -- 172.21.15.119:0/811533124 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1444107af0 msgr2=0x7f1444107f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:15:12.216 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:11.346+0000 7f144ba00700 1 --2- 172.21.15.119:0/811533124 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1444107af0 0x7f1444107f10 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f14380098a0 tx=0x7f1438009bb0 comp rx=0 tx=0).stop 2024-03-28T00:15:12.216 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:11.346+0000 7f144ba00700 1 -- 172.21.15.119:0/811533124 shutdown_connections 2024-03-28T00:15:12.216 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:11.346+0000 7f144ba00700 1 --2- 172.21.15.119:0/811533124 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1444107af0 0x7f1444107f10 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:15:12.216 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:11.346+0000 7f144ba00700 1 -- 172.21.15.119:0/811533124 >> 172.21.15.119:0/811533124 conn(0x7f14441030f0 msgr2=0x7f1444105510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:15:12.216 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:11.346+0000 7f144ba00700 1 -- 172.21.15.119:0/811533124 shutdown_connections 2024-03-28T00:15:12.216 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:11.346+0000 7f144ba00700 1 -- 172.21.15.119:0/811533124 wait complete. 2024-03-28T00:15:12.216 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:11.347+0000 7f144ba00700 1 Processor -- start 2024-03-28T00:15:12.216 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:11.347+0000 7f144ba00700 1 -- start start 2024-03-28T00:15:12.216 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:11.348+0000 7f144ba00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1444107af0 0x7f144407db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:15:12.217 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:11.348+0000 7f144ba00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f144410c0a0 con 0x7f1444107af0 2024-03-28T00:15:12.217 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:11.348+0000 7f1449000700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1444107af0 0x7f144407db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:15:12.217 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:11.348+0000 7f1449000700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1444107af0 0x7f144407db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:36490/0 (socket says 172.21.15.119:36490) 2024-03-28T00:15:12.217 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:11.348+0000 7f1449000700 1 -- 172.21.15.119:0/1737513274 learned_addr learned my addr 172.21.15.119:0/1737513274 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:15:12.217 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:11.348+0000 7f1449000700 1 -- 172.21.15.119:0/1737513274 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f144407e060 con 0x7f1444107af0 2024-03-28T00:15:12.217 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:11.349+0000 7f1449000700 1 --2- 172.21.15.119:0/1737513274 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1444107af0 0x7f144407db20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1438009fb0 tx=0x7f1438011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:15:12.217 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:11.349+0000 7f1441600700 1 -- 172.21.15.119:0/1737513274 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f14380038b0 con 0x7f1444107af0 2024-03-28T00:15:12.217 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:11.349+0000 7f1441600700 1 -- 172.21.15.119:0/1737513274 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1438011e50 con 0x7f1444107af0 2024-03-28T00:15:12.217 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:11.349+0000 7f144ba00700 1 -- 172.21.15.119:0/1737513274 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f144407c250 con 0x7f1444107af0 2024-03-28T00:15:12.217 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:11.349+0000 7f1441600700 1 -- 172.21.15.119:0/1737513274 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1438019550 con 0x7f1444107af0 2024-03-28T00:15:12.217 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:11.349+0000 7f144ba00700 1 -- 172.21.15.119:0/1737513274 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f144407c790 con 0x7f1444107af0 2024-03-28T00:15:12.217 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:11.350+0000 7f1441600700 1 -- 172.21.15.119:0/1737513274 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f1438019800 con 0x7f1444107af0 2024-03-28T00:15:12.217 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:11.350+0000 7f1441600700 1 -- 172.21.15.119:0/1737513274 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f143802dc30 con 0x7f1444107af0 2024-03-28T00:15:12.217 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:11.350+0000 7f144ba00700 1 -- 172.21.15.119:0/1737513274 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1444069260 con 0x7f1444107af0 2024-03-28T00:15:12.218 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:11.353+0000 7f1441600700 1 -- 172.21.15.119:0/1737513274 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f1438043dd0 con 0x7f1444107af0 2024-03-28T00:15:12.218 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:11.441+0000 7f144ba00700 1 -- 172.21.15.119:0/1737513274 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f1444104390 con 0x7f1444107af0 2024-03-28T00:15:12.218 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:11.442+0000 7f1441600700 1 -- 172.21.15.119:0/1737513274 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7f1438032030 con 0x7f1444107af0 2024-03-28T00:15:12.218 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:11.443+0000 7f144ba00700 1 -- 172.21.15.119:0/1737513274 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1444107af0 msgr2=0x7f144407db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:15:12.218 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:11.443+0000 7f144ba00700 1 --2- 172.21.15.119:0/1737513274 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1444107af0 0x7f144407db20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1438009fb0 tx=0x7f1438011bf0 comp rx=0 tx=0).stop 2024-03-28T00:15:12.218 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:11.443+0000 7f144ba00700 1 -- 172.21.15.119:0/1737513274 shutdown_connections 2024-03-28T00:15:12.218 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:11.443+0000 7f144ba00700 1 --2- 172.21.15.119:0/1737513274 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1444107af0 0x7f144407db20 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:15:12.218 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:11.443+0000 7f144ba00700 1 -- 172.21.15.119:0/1737513274 >> 172.21.15.119:0/1737513274 conn(0x7f14441030f0 msgr2=0x7f1444103c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:15:12.218 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:11.443+0000 7f144ba00700 1 -- 172.21.15.119:0/1737513274 shutdown_connections 2024-03-28T00:15:12.218 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:11.443+0000 7f144ba00700 1 -- 172.21.15.119:0/1737513274 wait complete. 2024-03-28T00:15:12.218 INFO:teuthology.orchestra.run.smithi119.stdout:mgr not available, waiting (2/15)... 2024-03-28T00:15:12.405 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 00:15:12 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-x[31752]: 2024-03-28T00:15:12.144+0000 7f000d48c200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-28T00:15:12.406 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 00:15:12 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-x[31752]: 2024-03-28T00:15:12.260+0000 7f000d48c200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-28T00:15:12.905 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 00:15:12 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-x[31752]: 2024-03-28T00:15:12.506+0000 7f000d48c200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-28T00:15:12.905 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 00:15:12 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-x[31752]: 2024-03-28T00:15:12.739+0000 7f000d48c200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-28T00:15:13.405 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 00:15:13 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-x[31752]: 2024-03-28T00:15:13.099+0000 7f000d48c200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-28T00:15:13.707 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 00:15:13 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-x[31752]: 2024-03-28T00:15:13.408+0000 7f000d48c200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-28T00:15:14.155 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 00:15:13 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-x[31752]: 2024-03-28T00:15:13.700+0000 7f000d48c200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-28T00:15:14.882 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 00:15:14 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-x[31752]: 2024-03-28T00:15:14.717+0000 7f000d48c200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-28T00:15:15.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:14 smithi119 ceph-mon[31518]: from='client.? 172.21.15.119:0/2341623834' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-28T00:15:15.538 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 00:15:15 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-x[31752]: 2024-03-28T00:15:15.421+0000 7f000d48c200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-28T00:15:15.557 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout 2024-03-28T00:15:15.557 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout { 2024-03-28T00:15:15.557 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "fsid": "10f73b92-ec98-11ee-95d0-87774f69a715", 2024-03-28T00:15:15.557 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "health": { 2024-03-28T00:15:15.557 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-28T00:15:15.557 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-28T00:15:15.558 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-28T00:15:15.558 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout }, 2024-03-28T00:15:15.558 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-28T00:15:15.558 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-28T00:15:15.558 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout 0 2024-03-28T00:15:15.558 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout ], 2024-03-28T00:15:15.558 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-28T00:15:15.558 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "a" 2024-03-28T00:15:15.558 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout ], 2024-03-28T00:15:15.558 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "quorum_age": 10, 2024-03-28T00:15:15.558 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-28T00:15:15.559 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-28T00:15:15.559 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-28T00:15:15.559 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-28T00:15:15.559 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout }, 2024-03-28T00:15:15.559 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-28T00:15:15.559 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-28T00:15:15.559 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-28T00:15:15.559 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-28T00:15:15.559 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-28T00:15:15.559 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-28T00:15:15.559 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-28T00:15:15.559 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-28T00:15:15.559 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout }, 2024-03-28T00:15:15.559 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-28T00:15:15.560 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-28T00:15:15.560 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-28T00:15:15.560 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-28T00:15:15.560 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-28T00:15:15.560 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-28T00:15:15.560 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-28T00:15:15.560 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-28T00:15:15.560 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-28T00:15:15.560 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout }, 2024-03-28T00:15:15.560 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-28T00:15:15.560 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-28T00:15:15.560 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-28T00:15:15.560 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-28T00:15:15.560 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout }, 2024-03-28T00:15:15.561 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-28T00:15:15.561 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-28T00:15:15.561 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-28T00:15:15.561 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-28T00:15:15.561 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-28T00:15:15.561 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-28T00:15:15.561 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "restful" 2024-03-28T00:15:15.561 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout ], 2024-03-28T00:15:15.561 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-28T00:15:15.561 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout }, 2024-03-28T00:15:15.561 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-28T00:15:15.561 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-28T00:15:15.561 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "modified": "2024-03-28T00:14:57.564140+0000", 2024-03-28T00:15:15.561 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-28T00:15:15.562 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout }, 2024-03-28T00:15:15.562 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-28T00:15:15.562 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout } 2024-03-28T00:15:15.562 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:14.766+0000 7fef80200700 1 Processor -- start 2024-03-28T00:15:15.562 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:14.766+0000 7fef80200700 1 -- start start 2024-03-28T00:15:15.562 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:14.767+0000 7fef80200700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fef78107af0 0x7fef78107f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:15:15.562 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:14.767+0000 7fef80200700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef781084e0 con 0x7fef78107af0 2024-03-28T00:15:15.562 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:14.767+0000 7fef7d800700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fef78107af0 0x7fef78107f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:15:15.562 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:14.767+0000 7fef7d800700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fef78107af0 0x7fef78107f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:39962/0 (socket says 172.21.15.119:39962) 2024-03-28T00:15:15.562 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:14.767+0000 7fef7d800700 1 -- 172.21.15.119:0/3206213891 learned_addr learned my addr 172.21.15.119:0/3206213891 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:15:15.562 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:14.768+0000 7fef7d800700 1 -- 172.21.15.119:0/3206213891 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef78108cf0 con 0x7fef78107af0 2024-03-28T00:15:15.562 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:14.768+0000 7fef7d800700 1 --2- 172.21.15.119:0/3206213891 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fef78107af0 0x7fef78107f10 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fef6c0098a0 tx=0x7fef6c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=753f0b9a272ae906 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:15:15.562 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:14.768+0000 7fef77e00700 1 -- 172.21.15.119:0/3206213891 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fef6c004160 con 0x7fef78107af0 2024-03-28T00:15:15.563 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:14.768+0000 7fef77e00700 1 -- 172.21.15.119:0/3206213891 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fef6c004300 con 0x7fef78107af0 2024-03-28T00:15:15.563 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:14.768+0000 7fef77e00700 1 -- 172.21.15.119:0/3206213891 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fef6c011b50 con 0x7fef78107af0 2024-03-28T00:15:15.563 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:14.769+0000 7fef80200700 1 -- 172.21.15.119:0/3206213891 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fef78107af0 msgr2=0x7fef78107f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:15:15.563 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:14.769+0000 7fef80200700 1 --2- 172.21.15.119:0/3206213891 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fef78107af0 0x7fef78107f10 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fef6c0098a0 tx=0x7fef6c009bb0 comp rx=0 tx=0).stop 2024-03-28T00:15:15.563 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:14.769+0000 7fef80200700 1 -- 172.21.15.119:0/3206213891 shutdown_connections 2024-03-28T00:15:15.563 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:14.769+0000 7fef80200700 1 --2- 172.21.15.119:0/3206213891 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fef78107af0 0x7fef78107f10 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:15:15.563 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:14.769+0000 7fef80200700 1 -- 172.21.15.119:0/3206213891 >> 172.21.15.119:0/3206213891 conn(0x7fef781030f0 msgr2=0x7fef78105510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:15:15.563 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:14.769+0000 7fef80200700 1 -- 172.21.15.119:0/3206213891 shutdown_connections 2024-03-28T00:15:15.563 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:14.769+0000 7fef80200700 1 -- 172.21.15.119:0/3206213891 wait complete. 2024-03-28T00:15:15.563 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:14.770+0000 7fef80200700 1 Processor -- start 2024-03-28T00:15:15.563 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:14.771+0000 7fef80200700 1 -- start start 2024-03-28T00:15:15.563 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:14.771+0000 7fef80200700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fef78107af0 0x7fef7819dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:15:15.563 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:14.771+0000 7fef80200700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef7810c0a0 con 0x7fef78107af0 2024-03-28T00:15:15.563 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:14.771+0000 7fef7d800700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fef78107af0 0x7fef7819dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:15:15.564 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:14.771+0000 7fef7d800700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fef78107af0 0x7fef7819dd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:39964/0 (socket says 172.21.15.119:39964) 2024-03-28T00:15:15.564 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:14.771+0000 7fef7d800700 1 -- 172.21.15.119:0/2341623834 learned_addr learned my addr 172.21.15.119:0/2341623834 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:15:15.564 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:14.771+0000 7fef7d800700 1 -- 172.21.15.119:0/2341623834 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef7819e290 con 0x7fef78107af0 2024-03-28T00:15:15.564 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:14.772+0000 7fef7d800700 1 --2- 172.21.15.119:0/2341623834 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fef78107af0 0x7fef7819dd50 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fef6c011850 tx=0x7fef6c011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:15:15.564 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:14.772+0000 7fef76000700 1 -- 172.21.15.119:0/2341623834 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fef6c003af0 con 0x7fef78107af0 2024-03-28T00:15:15.564 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:14.772+0000 7fef76000700 1 -- 172.21.15.119:0/2341623834 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fef6c020070 con 0x7fef78107af0 2024-03-28T00:15:15.564 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:14.772+0000 7fef80200700 1 -- 172.21.15.119:0/2341623834 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef7819e520 con 0x7fef78107af0 2024-03-28T00:15:15.564 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:14.772+0000 7fef76000700 1 -- 172.21.15.119:0/2341623834 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fef6c0195a0 con 0x7fef78107af0 2024-03-28T00:15:15.564 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:14.772+0000 7fef80200700 1 -- 172.21.15.119:0/2341623834 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef781a1210 con 0x7fef78107af0 2024-03-28T00:15:15.564 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:14.772+0000 7fef76000700 1 -- 172.21.15.119:0/2341623834 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fef6c019be0 con 0x7fef78107af0 2024-03-28T00:15:15.564 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:14.773+0000 7fef76000700 1 -- 172.21.15.119:0/2341623834 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fef6c011e00 con 0x7fef78107af0 2024-03-28T00:15:15.564 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:14.773+0000 7fef6b400700 1 -- 172.21.15.119:0/2341623834 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef54005330 con 0x7fef78107af0 2024-03-28T00:15:15.564 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:14.776+0000 7fef76000700 1 -- 172.21.15.119:0/2341623834 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fef6c003c90 con 0x7fef78107af0 2024-03-28T00:15:15.565 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:14.855+0000 7fef6b400700 1 -- 172.21.15.119:0/2341623834 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fef540058f0 con 0x7fef78107af0 2024-03-28T00:15:15.565 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:14.855+0000 7fef76000700 1 -- 172.21.15.119:0/2341623834 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1240 (secure 0 0 0) 0x7fef6c00ba80 con 0x7fef78107af0 2024-03-28T00:15:15.565 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:14.857+0000 7fef6b400700 1 -- 172.21.15.119:0/2341623834 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fef78107af0 msgr2=0x7fef7819dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:15:15.565 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:14.857+0000 7fef6b400700 1 --2- 172.21.15.119:0/2341623834 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fef78107af0 0x7fef7819dd50 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fef6c011850 tx=0x7fef6c011c10 comp rx=0 tx=0).stop 2024-03-28T00:15:15.565 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:14.857+0000 7fef6b400700 1 -- 172.21.15.119:0/2341623834 shutdown_connections 2024-03-28T00:15:15.565 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:14.857+0000 7fef6b400700 1 --2- 172.21.15.119:0/2341623834 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fef78107af0 0x7fef7819dd50 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:15:15.565 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:14.857+0000 7fef6b400700 1 -- 172.21.15.119:0/2341623834 >> 172.21.15.119:0/2341623834 conn(0x7fef781030f0 msgr2=0x7fef78103c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:15:15.565 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:14.857+0000 7fef6b400700 1 -- 172.21.15.119:0/2341623834 shutdown_connections 2024-03-28T00:15:15.565 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:14.857+0000 7fef6b400700 1 -- 172.21.15.119:0/2341623834 wait complete. 2024-03-28T00:15:15.565 INFO:teuthology.orchestra.run.smithi119.stdout:mgr not available, waiting (3/15)... 2024-03-28T00:15:15.833 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 00:15:15 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-x[31752]: 2024-03-28T00:15:15.554+0000 7f000d48c200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-28T00:15:16.156 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 00:15:15 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-x[31752]: 2024-03-28T00:15:15.826+0000 7f000d48c200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-28T00:15:16.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:15 smithi119 ceph-mon[31518]: Activating manager daemon x 2024-03-28T00:15:16.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:15 smithi119 ceph-mon[31518]: mgrmap e2: x(active, starting, since 0.0123886s) 2024-03-28T00:15:16.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:15 smithi119 ceph-mon[31518]: from='mgr.14100 172.21.15.119:0/1766400728' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-28T00:15:16.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:15 smithi119 ceph-mon[31518]: from='mgr.14100 172.21.15.119:0/1766400728' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-28T00:15:16.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:15 smithi119 ceph-mon[31518]: from='mgr.14100 172.21.15.119:0/1766400728' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-28T00:15:16.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:15 smithi119 ceph-mon[31518]: from='mgr.14100 172.21.15.119:0/1766400728' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-28T00:15:16.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:15 smithi119 ceph-mon[31518]: from='mgr.14100 172.21.15.119:0/1766400728' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-28T00:15:16.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:15 smithi119 ceph-mon[31518]: Manager daemon x is now available 2024-03-28T00:15:16.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:15 smithi119 ceph-mon[31518]: from='mgr.14100 172.21.15.119:0/1766400728' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-28T00:15:16.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:15 smithi119 ceph-mon[31518]: from='mgr.14100 172.21.15.119:0/1766400728' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-28T00:15:16.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:15 smithi119 ceph-mon[31518]: from='mgr.14100 172.21.15.119:0/1766400728' entity='mgr.x' 2024-03-28T00:15:16.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:15 smithi119 ceph-mon[31518]: from='mgr.14100 172.21.15.119:0/1766400728' entity='mgr.x' 2024-03-28T00:15:16.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:15 smithi119 ceph-mon[31518]: from='mgr.14100 172.21.15.119:0/1766400728' entity='mgr.x' 2024-03-28T00:15:17.987 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:17 smithi119 ceph-mon[31518]: mgrmap e3: x(active, since 1.02225s) 2024-03-28T00:15:19.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:18 smithi119 ceph-mon[31518]: mgrmap e4: x(active, since 2s) 2024-03-28T00:15:19.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:18 smithi119 ceph-mon[31518]: from='client.? 172.21.15.119:0/1101760458' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-28T00:15:19.317 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout 2024-03-28T00:15:19.317 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout { 2024-03-28T00:15:19.317 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "fsid": "10f73b92-ec98-11ee-95d0-87774f69a715", 2024-03-28T00:15:19.317 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "health": { 2024-03-28T00:15:19.318 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-28T00:15:19.318 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-28T00:15:19.318 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-28T00:15:19.318 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout }, 2024-03-28T00:15:19.318 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-28T00:15:19.319 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-28T00:15:19.319 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout 0 2024-03-28T00:15:19.319 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout ], 2024-03-28T00:15:19.319 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-28T00:15:19.320 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "a" 2024-03-28T00:15:19.320 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout ], 2024-03-28T00:15:19.320 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "quorum_age": 14, 2024-03-28T00:15:19.321 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-28T00:15:19.321 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-28T00:15:19.321 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-28T00:15:19.321 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-28T00:15:19.321 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout }, 2024-03-28T00:15:19.322 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-28T00:15:19.322 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-28T00:15:19.322 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-28T00:15:19.322 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-28T00:15:19.323 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-28T00:15:19.323 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-28T00:15:19.323 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-28T00:15:19.323 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-28T00:15:19.323 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout }, 2024-03-28T00:15:19.323 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-28T00:15:19.324 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-28T00:15:19.324 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-28T00:15:19.324 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-28T00:15:19.324 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-28T00:15:19.324 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-28T00:15:19.324 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-28T00:15:19.325 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-28T00:15:19.325 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-28T00:15:19.325 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout }, 2024-03-28T00:15:19.325 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-28T00:15:19.325 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-28T00:15:19.325 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-28T00:15:19.333 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-28T00:15:19.333 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout }, 2024-03-28T00:15:19.333 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-28T00:15:19.333 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-28T00:15:19.333 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-28T00:15:19.334 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-28T00:15:19.334 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-28T00:15:19.334 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-28T00:15:19.334 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "restful" 2024-03-28T00:15:19.334 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout ], 2024-03-28T00:15:19.334 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-28T00:15:19.334 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout }, 2024-03-28T00:15:19.334 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-28T00:15:19.334 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-28T00:15:19.334 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "modified": "2024-03-28T00:14:57.564140+0000", 2024-03-28T00:15:19.334 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-28T00:15:19.334 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout }, 2024-03-28T00:15:19.334 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-28T00:15:19.334 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout } 2024-03-28T00:15:19.335 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:18.135+0000 7f2b45800700 1 Processor -- start 2024-03-28T00:15:19.335 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:18.135+0000 7f2b45800700 1 -- start start 2024-03-28T00:15:19.335 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:18.136+0000 7f2b45800700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2b40107af0 0x7f2b40107f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:15:19.335 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:18.136+0000 7f2b45800700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b401084e0 con 0x7f2b40107af0 2024-03-28T00:15:19.335 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:18.136+0000 7f2b3ea00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2b40107af0 0x7f2b40107f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:15:19.335 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:18.137+0000 7f2b3ea00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2b40107af0 0x7f2b40107f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40030/0 (socket says 172.21.15.119:40030) 2024-03-28T00:15:19.335 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:18.137+0000 7f2b3ea00700 1 -- 172.21.15.119:0/2125172012 learned_addr learned my addr 172.21.15.119:0/2125172012 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:15:19.335 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:18.137+0000 7f2b3ea00700 1 -- 172.21.15.119:0/2125172012 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b40108cf0 con 0x7f2b40107af0 2024-03-28T00:15:19.335 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:18.137+0000 7f2b3ea00700 1 --2- 172.21.15.119:0/2125172012 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2b40107af0 0x7f2b40107f10 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f2b300098a0 tx=0x7f2b30009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=935c5c5b5bf7e7a6 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:15:19.335 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:18.138+0000 7f2b3d600700 1 -- 172.21.15.119:0/2125172012 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2b30004160 con 0x7f2b40107af0 2024-03-28T00:15:19.335 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:18.138+0000 7f2b3d600700 1 -- 172.21.15.119:0/2125172012 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2b30004300 con 0x7f2b40107af0 2024-03-28T00:15:19.335 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:18.138+0000 7f2b3d600700 1 -- 172.21.15.119:0/2125172012 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2b30011b50 con 0x7f2b40107af0 2024-03-28T00:15:19.335 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:18.138+0000 7f2b45800700 1 -- 172.21.15.119:0/2125172012 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2b40107af0 msgr2=0x7f2b40107f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:15:19.335 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:18.138+0000 7f2b45800700 1 --2- 172.21.15.119:0/2125172012 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2b40107af0 0x7f2b40107f10 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f2b300098a0 tx=0x7f2b30009bb0 comp rx=0 tx=0).stop 2024-03-28T00:15:19.335 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:18.139+0000 7f2b45800700 1 -- 172.21.15.119:0/2125172012 shutdown_connections 2024-03-28T00:15:19.336 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:18.139+0000 7f2b45800700 1 --2- 172.21.15.119:0/2125172012 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2b40107af0 0x7f2b40107f10 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:15:19.336 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:18.139+0000 7f2b45800700 1 -- 172.21.15.119:0/2125172012 >> 172.21.15.119:0/2125172012 conn(0x7f2b401030f0 msgr2=0x7f2b40105510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:15:19.336 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:18.139+0000 7f2b45800700 1 -- 172.21.15.119:0/2125172012 shutdown_connections 2024-03-28T00:15:19.336 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:18.139+0000 7f2b45800700 1 -- 172.21.15.119:0/2125172012 wait complete. 2024-03-28T00:15:19.336 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:18.140+0000 7f2b45800700 1 Processor -- start 2024-03-28T00:15:19.339 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:18.140+0000 7f2b45800700 1 -- start start 2024-03-28T00:15:19.339 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:18.141+0000 7f2b45800700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2b40107af0 0x7f2b4019dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:15:19.339 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:18.141+0000 7f2b45800700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b4010c0a0 con 0x7f2b40107af0 2024-03-28T00:15:19.339 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:18.141+0000 7f2b3ea00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2b40107af0 0x7f2b4019dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:15:19.339 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:18.141+0000 7f2b3ea00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2b40107af0 0x7f2b4019dd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40038/0 (socket says 172.21.15.119:40038) 2024-03-28T00:15:19.339 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:18.141+0000 7f2b3ea00700 1 -- 172.21.15.119:0/1101760458 learned_addr learned my addr 172.21.15.119:0/1101760458 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:15:19.339 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:18.141+0000 7f2b3ea00700 1 -- 172.21.15.119:0/1101760458 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b4019e2d0 con 0x7f2b40107af0 2024-03-28T00:15:19.339 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:18.142+0000 7f2b3ea00700 1 --2- 172.21.15.119:0/1101760458 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2b40107af0 0x7f2b4019dd90 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f2b30011850 tx=0x7f2b30011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:15:19.339 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:18.142+0000 7f2b37400700 1 -- 172.21.15.119:0/1101760458 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2b30003af0 con 0x7f2b40107af0 2024-03-28T00:15:19.339 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:18.142+0000 7f2b37400700 1 -- 172.21.15.119:0/1101760458 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2b30020070 con 0x7f2b40107af0 2024-03-28T00:15:19.339 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:18.142+0000 7f2b45800700 1 -- 172.21.15.119:0/1101760458 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b4019e560 con 0x7f2b40107af0 2024-03-28T00:15:19.340 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:18.142+0000 7f2b37400700 1 -- 172.21.15.119:0/1101760458 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2b300195a0 con 0x7f2b40107af0 2024-03-28T00:15:19.340 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:18.142+0000 7f2b45800700 1 -- 172.21.15.119:0/1101760458 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b401a1210 con 0x7f2b40107af0 2024-03-28T00:15:19.340 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:18.143+0000 7f2b37400700 1 -- 172.21.15.119:0/1101760458 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 4) v1 ==== 46095+0+0 (secure 0 0 0) 0x7f2b30019850 con 0x7f2b40107af0 2024-03-28T00:15:19.340 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:18.143+0000 7f2b45800700 1 -- 172.21.15.119:0/1101760458 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b40069260 con 0x7f2b40107af0 2024-03-28T00:15:19.340 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:18.144+0000 7f2b37400700 1 --2- 172.21.15.119:0/1101760458 >> [v2:172.21.15.119:6800/2056426384,v1:172.21.15.119:6801/2056426384] conn(0x7f2b24038f20 0x7f2b2403b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:15:19.340 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:18.144+0000 7f2b37400700 1 -- 172.21.15.119:0/1101760458 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f2b30011e00 con 0x7f2b40107af0 2024-03-28T00:15:19.340 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:18.144+0000 7f2b3e000700 1 --2- 172.21.15.119:0/1101760458 >> [v2:172.21.15.119:6800/2056426384,v1:172.21.15.119:6801/2056426384] conn(0x7f2b24038f20 0x7f2b2403b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:15:19.340 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:18.145+0000 7f2b3e000700 1 --2- 172.21.15.119:0/1101760458 >> [v2:172.21.15.119:6800/2056426384,v1:172.21.15.119:6801/2056426384] conn(0x7f2b24038f20 0x7f2b2403b3e0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c006fd0 tx=0x7f2b2c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:15:19.340 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:18.150+0000 7f2b37400700 1 -- 172.21.15.119:0/1101760458 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f2b30003c90 con 0x7f2b40107af0 2024-03-28T00:15:19.340 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:18.451+0000 7f2b45800700 1 -- 172.21.15.119:0/1101760458 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f2b401a1430 con 0x7f2b40107af0 2024-03-28T00:15:19.340 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:18.451+0000 7f2b37400700 1 -- 172.21.15.119:0/1101760458 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7f2b401a1430 con 0x7f2b40107af0 2024-03-28T00:15:19.340 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:18.454+0000 7f2b45800700 1 -- 172.21.15.119:0/1101760458 >> [v2:172.21.15.119:6800/2056426384,v1:172.21.15.119:6801/2056426384] conn(0x7f2b24038f20 msgr2=0x7f2b2403b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:15:19.340 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:18.454+0000 7f2b45800700 1 --2- 172.21.15.119:0/1101760458 >> [v2:172.21.15.119:6800/2056426384,v1:172.21.15.119:6801/2056426384] conn(0x7f2b24038f20 0x7f2b2403b3e0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c006fd0 tx=0x7f2b2c006e40 comp rx=0 tx=0).stop 2024-03-28T00:15:19.340 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:18.454+0000 7f2b45800700 1 -- 172.21.15.119:0/1101760458 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2b40107af0 msgr2=0x7f2b4019dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:15:19.340 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:18.454+0000 7f2b45800700 1 --2- 172.21.15.119:0/1101760458 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2b40107af0 0x7f2b4019dd90 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f2b30011850 tx=0x7f2b30011c10 comp rx=0 tx=0).stop 2024-03-28T00:15:19.341 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:18.454+0000 7f2b45800700 1 -- 172.21.15.119:0/1101760458 shutdown_connections 2024-03-28T00:15:19.341 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:18.454+0000 7f2b45800700 1 --2- 172.21.15.119:0/1101760458 >> [v2:172.21.15.119:6800/2056426384,v1:172.21.15.119:6801/2056426384] conn(0x7f2b24038f20 0x7f2b2403b3e0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:15:19.341 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:18.454+0000 7f2b45800700 1 --2- 172.21.15.119:0/1101760458 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2b40107af0 0x7f2b4019dd90 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:15:19.341 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:18.454+0000 7f2b45800700 1 -- 172.21.15.119:0/1101760458 >> 172.21.15.119:0/1101760458 conn(0x7f2b401030f0 msgr2=0x7f2b40103c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:15:19.342 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:18.454+0000 7f2b45800700 1 -- 172.21.15.119:0/1101760458 shutdown_connections 2024-03-28T00:15:19.342 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:18.454+0000 7f2b45800700 1 -- 172.21.15.119:0/1101760458 wait complete. 2024-03-28T00:15:19.342 INFO:teuthology.orchestra.run.smithi119.stdout:mgr is available 2024-03-28T00:15:20.529 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:20 smithi119 ceph-mon[31518]: from='client.? 172.21.15.119:0/1618012492' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-03-28T00:15:20.852 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout 2024-03-28T00:15:20.852 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout [global] 2024-03-28T00:15:20.852 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout fsid = 10f73b92-ec98-11ee-95d0-87774f69a715 2024-03-28T00:15:20.852 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-03-28T00:15:20.852 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-03-28T00:15:20.852 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-03-28T00:15:20.852 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-03-28T00:15:20.852 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout 2024-03-28T00:15:20.852 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout [mgr] 2024-03-28T00:15:20.852 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-03-28T00:15:20.852 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout 2024-03-28T00:15:20.852 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout [osd] 2024-03-28T00:15:20.853 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-03-28T00:15:20.853 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-03-28T00:15:20.853 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-03-28T00:15:20.853 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-03-28T00:15:20.853 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-03-28T00:15:20.853 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:19.945+0000 7f7ba6e00700 1 Processor -- start 2024-03-28T00:15:20.853 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:19.945+0000 7f7ba6e00700 1 -- start start 2024-03-28T00:15:20.853 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:19.945+0000 7f7ba6e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ba0109b10 0x7f7ba0109f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:15:20.853 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:19.946+0000 7f7ba6e00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ba010a500 con 0x7f7ba0109b10 2024-03-28T00:15:20.853 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:19.946+0000 7f7ba5a00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ba0109b10 0x7f7ba0109f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:15:20.853 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:19.946+0000 7f7ba5a00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ba0109b10 0x7f7ba0109f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40048/0 (socket says 172.21.15.119:40048) 2024-03-28T00:15:20.858 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:19.946+0000 7f7ba5a00700 1 -- 172.21.15.119:0/2706472085 learned_addr learned my addr 172.21.15.119:0/2706472085 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:15:20.859 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:19.947+0000 7f7ba5a00700 1 -- 172.21.15.119:0/2706472085 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ba010ad10 con 0x7f7ba0109b10 2024-03-28T00:15:20.859 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:19.947+0000 7f7ba5a00700 1 --2- 172.21.15.119:0/2706472085 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ba0109b10 0x7f7ba0109f30 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f7b940098a0 tx=0x7f7b94009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c76b1c417475f90 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:15:20.859 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:19.948+0000 7f7b9fe00700 1 -- 172.21.15.119:0/2706472085 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7b94004160 con 0x7f7ba0109b10 2024-03-28T00:15:20.859 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:19.948+0000 7f7b9fe00700 1 -- 172.21.15.119:0/2706472085 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7b94004300 con 0x7f7ba0109b10 2024-03-28T00:15:20.859 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:19.948+0000 7f7b9fe00700 1 -- 172.21.15.119:0/2706472085 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7b94011b50 con 0x7f7ba0109b10 2024-03-28T00:15:20.859 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:19.948+0000 7f7ba6e00700 1 -- 172.21.15.119:0/2706472085 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ba0109b10 msgr2=0x7f7ba0109f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:15:20.859 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:19.948+0000 7f7ba6e00700 1 --2- 172.21.15.119:0/2706472085 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ba0109b10 0x7f7ba0109f30 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f7b940098a0 tx=0x7f7b94009bb0 comp rx=0 tx=0).stop 2024-03-28T00:15:20.859 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:19.949+0000 7f7ba6e00700 1 -- 172.21.15.119:0/2706472085 shutdown_connections 2024-03-28T00:15:20.859 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:19.949+0000 7f7ba6e00700 1 --2- 172.21.15.119:0/2706472085 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ba0109b10 0x7f7ba0109f30 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:15:20.859 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:19.949+0000 7f7ba6e00700 1 -- 172.21.15.119:0/2706472085 >> 172.21.15.119:0/2706472085 conn(0x7f7ba007cec0 msgr2=0x7f7ba007d310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:15:20.859 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:19.949+0000 7f7ba6e00700 1 -- 172.21.15.119:0/2706472085 shutdown_connections 2024-03-28T00:15:20.859 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:19.949+0000 7f7ba6e00700 1 -- 172.21.15.119:0/2706472085 wait complete. 2024-03-28T00:15:20.859 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:19.950+0000 7f7ba6e00700 1 Processor -- start 2024-03-28T00:15:20.859 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:19.950+0000 7f7ba6e00700 1 -- start start 2024-03-28T00:15:20.860 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:19.951+0000 7f7ba6e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ba0109b10 0x7f7ba019db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:15:20.860 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:19.951+0000 7f7ba6e00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ba010e0c0 con 0x7f7ba0109b10 2024-03-28T00:15:20.860 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:19.951+0000 7f7ba5a00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ba0109b10 0x7f7ba019db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:15:20.860 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:19.951+0000 7f7ba5a00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ba0109b10 0x7f7ba019db00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40056/0 (socket says 172.21.15.119:40056) 2024-03-28T00:15:20.860 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:19.951+0000 7f7ba5a00700 1 -- 172.21.15.119:0/1618012492 learned_addr learned my addr 172.21.15.119:0/1618012492 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:15:20.860 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:19.951+0000 7f7ba5a00700 1 -- 172.21.15.119:0/1618012492 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ba019e040 con 0x7f7ba0109b10 2024-03-28T00:15:20.860 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:19.952+0000 7f7ba5a00700 1 --2- 172.21.15.119:0/1618012492 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ba0109b10 0x7f7ba019db00 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f7b94011670 tx=0x7f7b94011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:15:20.860 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:19.952+0000 7f7b9e000700 1 -- 172.21.15.119:0/1618012492 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7b940039c0 con 0x7f7ba0109b10 2024-03-28T00:15:20.860 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:19.952+0000 7f7b9e000700 1 -- 172.21.15.119:0/1618012492 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7b94011e50 con 0x7f7ba0109b10 2024-03-28T00:15:20.860 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:19.952+0000 7f7b9e000700 1 -- 172.21.15.119:0/1618012492 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7b9401a5c0 con 0x7f7ba0109b10 2024-03-28T00:15:20.860 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:19.952+0000 7f7ba6e00700 1 -- 172.21.15.119:0/1618012492 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ba019e2d0 con 0x7f7ba0109b10 2024-03-28T00:15:20.860 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:19.952+0000 7f7ba6e00700 1 -- 172.21.15.119:0/1618012492 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ba01a0fc0 con 0x7f7ba0109b10 2024-03-28T00:15:20.860 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:19.953+0000 7f7b9e000700 1 -- 172.21.15.119:0/1618012492 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 4) v1 ==== 46095+0+0 (secure 0 0 0) 0x7f7b94024430 con 0x7f7ba0109b10 2024-03-28T00:15:20.860 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:19.953+0000 7f7ba6e00700 1 -- 172.21.15.119:0/1618012492 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ba0069260 con 0x7f7ba0109b10 2024-03-28T00:15:20.860 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:19.954+0000 7f7b9e000700 1 --2- 172.21.15.119:0/1618012492 >> [v2:172.21.15.119:6800/2056426384,v1:172.21.15.119:6801/2056426384] conn(0x7f7b88038ed0 0x7f7b8803b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:15:20.861 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:19.954+0000 7f7b9e000700 1 -- 172.21.15.119:0/1618012492 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f7b9404d1b0 con 0x7f7ba0109b10 2024-03-28T00:15:20.861 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:19.954+0000 7f7ba5000700 1 --2- 172.21.15.119:0/1618012492 >> [v2:172.21.15.119:6800/2056426384,v1:172.21.15.119:6801/2056426384] conn(0x7f7b88038ed0 0x7f7b8803b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:15:20.861 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:19.954+0000 7f7ba5000700 1 --2- 172.21.15.119:0/1618012492 >> [v2:172.21.15.119:6800/2056426384,v1:172.21.15.119:6801/2056426384] conn(0x7f7b88038ed0 0x7f7b8803b390 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c006fd0 tx=0x7f7b8c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:15:20.861 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:19.959+0000 7f7b9e000700 1 -- 172.21.15.119:0/1618012492 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f7b94017ca0 con 0x7f7ba0109b10 2024-03-28T00:15:20.861 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:20.172+0000 7f7ba6e00700 1 -- 172.21.15.119:0/1618012492 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f7ba01a1190 con 0x7f7ba0109b10 2024-03-28T00:15:20.861 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:20.174+0000 7f7b9e000700 1 -- 172.21.15.119:0/1618012492 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+373 (secure 0 0 0) 0x7f7ba01a1190 con 0x7f7ba0109b10 2024-03-28T00:15:20.861 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:20.177+0000 7f7ba6e00700 1 -- 172.21.15.119:0/1618012492 >> [v2:172.21.15.119:6800/2056426384,v1:172.21.15.119:6801/2056426384] conn(0x7f7b88038ed0 msgr2=0x7f7b8803b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:15:20.861 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:20.177+0000 7f7ba6e00700 1 --2- 172.21.15.119:0/1618012492 >> [v2:172.21.15.119:6800/2056426384,v1:172.21.15.119:6801/2056426384] conn(0x7f7b88038ed0 0x7f7b8803b390 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c006fd0 tx=0x7f7b8c006e40 comp rx=0 tx=0).stop 2024-03-28T00:15:20.861 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:20.177+0000 7f7ba6e00700 1 -- 172.21.15.119:0/1618012492 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ba0109b10 msgr2=0x7f7ba019db00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:15:20.861 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:20.177+0000 7f7ba6e00700 1 --2- 172.21.15.119:0/1618012492 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ba0109b10 0x7f7ba019db00 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f7b94011670 tx=0x7f7b94011b60 comp rx=0 tx=0).stop 2024-03-28T00:15:20.861 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:20.177+0000 7f7ba6e00700 1 -- 172.21.15.119:0/1618012492 shutdown_connections 2024-03-28T00:15:20.861 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:20.177+0000 7f7ba6e00700 1 --2- 172.21.15.119:0/1618012492 >> [v2:172.21.15.119:6800/2056426384,v1:172.21.15.119:6801/2056426384] conn(0x7f7b88038ed0 0x7f7b8803b390 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:15:20.861 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:20.177+0000 7f7ba6e00700 1 --2- 172.21.15.119:0/1618012492 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ba0109b10 0x7f7ba019db00 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:15:20.861 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:20.177+0000 7f7ba6e00700 1 -- 172.21.15.119:0/1618012492 >> 172.21.15.119:0/1618012492 conn(0x7f7ba007cec0 msgr2=0x7f7ba01075a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:15:20.862 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:20.177+0000 7f7ba6e00700 1 -- 172.21.15.119:0/1618012492 shutdown_connections 2024-03-28T00:15:20.862 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:20.177+0000 7f7ba6e00700 1 -- 172.21.15.119:0/1618012492 wait complete. 2024-03-28T00:15:20.862 INFO:teuthology.orchestra.run.smithi119.stdout:Enabling cephadm module... 2024-03-28T00:15:22.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:21 smithi119 ceph-mon[31518]: from='client.? 172.21.15.119:0/3722552431' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-03-28T00:15:22.156 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 00:15:21 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-x[31752]: ignoring --setuser ceph since I am not root 2024-03-28T00:15:22.156 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 00:15:21 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-x[31752]: ignoring --setgroup ceph since I am not root 2024-03-28T00:15:22.516 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:21.425+0000 7f98f4c00700 1 Processor -- start 2024-03-28T00:15:22.522 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:21.425+0000 7f98f4c00700 1 -- start start 2024-03-28T00:15:22.522 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:21.426+0000 7f98f4c00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f98f010ec80 0x7f98f0111070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:15:22.523 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:21.426+0000 7f98f4c00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98f0082380 con 0x7f98f010ec80 2024-03-28T00:15:22.523 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:21.426+0000 7f98ede00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f98f010ec80 0x7f98f0111070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:15:22.523 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:21.427+0000 7f98ede00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f98f010ec80 0x7f98f0111070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40058/0 (socket says 172.21.15.119:40058) 2024-03-28T00:15:22.523 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:21.427+0000 7f98ede00700 1 -- 172.21.15.119:0/3497274972 learned_addr learned my addr 172.21.15.119:0/3497274972 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:15:22.523 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:21.427+0000 7f98ede00700 1 -- 172.21.15.119:0/3497274972 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98f01115b0 con 0x7f98f010ec80 2024-03-28T00:15:22.523 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:21.427+0000 7f98ede00700 1 --2- 172.21.15.119:0/3497274972 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f98f010ec80 0x7f98f0111070 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f98e0009e80 tx=0x7f98e000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17d641f687b08edb server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:15:22.523 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:21.427+0000 7f98ed400700 1 -- 172.21.15.119:0/3497274972 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f98e0017070 con 0x7f98f010ec80 2024-03-28T00:15:22.523 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:21.428+0000 7f98ed400700 1 -- 172.21.15.119:0/3497274972 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f98e000b8c0 con 0x7f98f010ec80 2024-03-28T00:15:22.523 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:21.428+0000 7f98ed400700 1 -- 172.21.15.119:0/3497274972 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f98e00045c0 con 0x7f98f010ec80 2024-03-28T00:15:22.523 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:21.428+0000 7f98f4c00700 1 -- 172.21.15.119:0/3497274972 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f98f010ec80 msgr2=0x7f98f0111070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:15:22.523 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:21.428+0000 7f98f4c00700 1 --2- 172.21.15.119:0/3497274972 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f98f010ec80 0x7f98f0111070 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f98e0009e80 tx=0x7f98e000b330 comp rx=0 tx=0).stop 2024-03-28T00:15:22.523 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:21.428+0000 7f98f4c00700 1 -- 172.21.15.119:0/3497274972 shutdown_connections 2024-03-28T00:15:22.523 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:21.428+0000 7f98f4c00700 1 --2- 172.21.15.119:0/3497274972 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f98f010ec80 0x7f98f0111070 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:15:22.523 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:21.428+0000 7f98f4c00700 1 -- 172.21.15.119:0/3497274972 >> 172.21.15.119:0/3497274972 conn(0x7f98f007d100 msgr2=0x7f98f007f520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:15:22.524 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:21.428+0000 7f98f4c00700 1 -- 172.21.15.119:0/3497274972 shutdown_connections 2024-03-28T00:15:22.524 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:21.428+0000 7f98f4c00700 1 -- 172.21.15.119:0/3497274972 wait complete. 2024-03-28T00:15:22.524 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:21.429+0000 7f98f4c00700 1 Processor -- start 2024-03-28T00:15:22.524 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:21.430+0000 7f98f4c00700 1 -- start start 2024-03-28T00:15:22.524 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:21.430+0000 7f98f4c00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f98f010ec80 0x7f98f01a2130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:15:22.524 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:21.430+0000 7f98f4c00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98f0112510 con 0x7f98f010ec80 2024-03-28T00:15:22.524 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:21.430+0000 7f98ede00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f98f010ec80 0x7f98f01a2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:15:22.524 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:21.430+0000 7f98ede00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f98f010ec80 0x7f98f01a2130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40060/0 (socket says 172.21.15.119:40060) 2024-03-28T00:15:22.524 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:21.430+0000 7f98ede00700 1 -- 172.21.15.119:0/3722552431 learned_addr learned my addr 172.21.15.119:0/3722552431 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:15:22.524 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:21.430+0000 7f98ede00700 1 -- 172.21.15.119:0/3722552431 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98f01a2670 con 0x7f98f010ec80 2024-03-28T00:15:22.524 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:21.431+0000 7f98ede00700 1 --2- 172.21.15.119:0/3722552431 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f98f010ec80 0x7f98f01a2130 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f98e0009e50 tx=0x7f98e000bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:15:22.524 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:21.431+0000 7f98e7400700 1 -- 172.21.15.119:0/3722552431 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f98e0020020 con 0x7f98f010ec80 2024-03-28T00:15:22.524 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:21.431+0000 7f98e7400700 1 -- 172.21.15.119:0/3722552431 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f98e0003b50 con 0x7f98f010ec80 2024-03-28T00:15:22.524 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:21.431+0000 7f98f4c00700 1 -- 172.21.15.119:0/3722552431 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98f01a2900 con 0x7f98f010ec80 2024-03-28T00:15:22.525 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:21.431+0000 7f98e7400700 1 -- 172.21.15.119:0/3722552431 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f98e0017040 con 0x7f98f010ec80 2024-03-28T00:15:22.525 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:21.431+0000 7f98f4c00700 1 -- 172.21.15.119:0/3722552431 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98f01a3420 con 0x7f98f010ec80 2024-03-28T00:15:22.525 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:21.432+0000 7f98e7400700 1 -- 172.21.15.119:0/3722552431 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 4) v1 ==== 46095+0+0 (secure 0 0 0) 0x7f98e0023020 con 0x7f98f010ec80 2024-03-28T00:15:22.525 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:21.432+0000 7f98f4c00700 1 -- 172.21.15.119:0/3722552431 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98f00637c0 con 0x7f98f010ec80 2024-03-28T00:15:22.525 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:21.433+0000 7f98e7400700 1 --2- 172.21.15.119:0/3722552431 >> [v2:172.21.15.119:6800/2056426384,v1:172.21.15.119:6801/2056426384] conn(0x7f98d4038ed0 0x7f98d403b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:15:22.525 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:21.433+0000 7f98e7400700 1 -- 172.21.15.119:0/3722552431 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f98e000f050 con 0x7f98f010ec80 2024-03-28T00:15:22.525 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:21.433+0000 7f98e5400700 1 --2- 172.21.15.119:0/3722552431 >> [v2:172.21.15.119:6800/2056426384,v1:172.21.15.119:6801/2056426384] conn(0x7f98d4038ed0 0x7f98d403b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:15:22.525 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:21.434+0000 7f98e5400700 1 --2- 172.21.15.119:0/3722552431 >> [v2:172.21.15.119:6800/2056426384,v1:172.21.15.119:6801/2056426384] conn(0x7f98d4038ed0 0x7f98d403b390 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f98d8006fd0 tx=0x7f98d8006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:15:22.525 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:21.438+0000 7f98e7400700 1 -- 172.21.15.119:0/3722552431 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f98e002a020 con 0x7f98f010ec80 2024-03-28T00:15:22.525 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:21.717+0000 7f98f4c00700 1 -- 172.21.15.119:0/3722552431 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7f98f01a3990 con 0x7f98f010ec80 2024-03-28T00:15:22.525 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:21.771+0000 7f98e7400700 1 -- 172.21.15.119:0/3722552431 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7f98e0011e70 con 0x7f98f010ec80 2024-03-28T00:15:22.525 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:21.774+0000 7f98e7400700 1 -- 172.21.15.119:0/3722552431 <== mon.0 v2:172.21.15.119:3300/0 8 ==== mgrmap(e 5) v1 ==== 46106+0+0 (secure 0 0 0) 0x7f98e00127b0 con 0x7f98f010ec80 2024-03-28T00:15:22.525 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:21.777+0000 7f98f4c00700 1 -- 172.21.15.119:0/3722552431 >> [v2:172.21.15.119:6800/2056426384,v1:172.21.15.119:6801/2056426384] conn(0x7f98d4038ed0 msgr2=0x7f98d403b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:15:22.526 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:21.777+0000 7f98f4c00700 1 --2- 172.21.15.119:0/3722552431 >> [v2:172.21.15.119:6800/2056426384,v1:172.21.15.119:6801/2056426384] conn(0x7f98d4038ed0 0x7f98d403b390 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f98d8006fd0 tx=0x7f98d8006e40 comp rx=0 tx=0).stop 2024-03-28T00:15:22.526 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:21.777+0000 7f98f4c00700 1 -- 172.21.15.119:0/3722552431 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f98f010ec80 msgr2=0x7f98f01a2130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:15:22.526 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:21.777+0000 7f98f4c00700 1 --2- 172.21.15.119:0/3722552431 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f98f010ec80 0x7f98f01a2130 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f98e0009e50 tx=0x7f98e000bd30 comp rx=0 tx=0).stop 2024-03-28T00:15:22.526 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:21.777+0000 7f98f4c00700 1 -- 172.21.15.119:0/3722552431 shutdown_connections 2024-03-28T00:15:22.526 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:21.777+0000 7f98f4c00700 1 --2- 172.21.15.119:0/3722552431 >> [v2:172.21.15.119:6800/2056426384,v1:172.21.15.119:6801/2056426384] conn(0x7f98d4038ed0 0x7f98d403b390 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:15:22.526 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:21.777+0000 7f98f4c00700 1 --2- 172.21.15.119:0/3722552431 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f98f010ec80 0x7f98f01a2130 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:15:22.526 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:21.777+0000 7f98f4c00700 1 -- 172.21.15.119:0/3722552431 >> 172.21.15.119:0/3722552431 conn(0x7f98f007d100 msgr2=0x7f98f007dc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:15:22.526 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:21.777+0000 7f98f4c00700 1 -- 172.21.15.119:0/3722552431 shutdown_connections 2024-03-28T00:15:22.526 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:21.777+0000 7f98f4c00700 1 -- 172.21.15.119:0/3722552431 wait complete. 2024-03-28T00:15:22.786 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 00:15:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-x[31752]: 2024-03-28T00:15:22.538+0000 7f8e1aaa9200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-28T00:15:22.786 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 00:15:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-x[31752]: 2024-03-28T00:15:22.664+0000 7f8e1aaa9200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-28T00:15:23.065 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:22 smithi119 ceph-mon[31518]: from='client.? 172.21.15.119:0/3722552431' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-03-28T00:15:23.065 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:22 smithi119 ceph-mon[31518]: mgrmap e5: x(active, since 5s) 2024-03-28T00:15:23.619 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 00:15:23 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-x[31752]: 2024-03-28T00:15:23.490+0000 7f8e1aaa9200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-28T00:15:23.619 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 00:15:23 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-x[31752]: 2024-03-28T00:15:23.612+0000 7f8e1aaa9200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-28T00:15:23.905 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 00:15:23 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-x[31752]: 2024-03-28T00:15:23.793+0000 7f8e1aaa9200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-28T00:15:23.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:23 smithi119 ceph-mon[31518]: from='client.? 172.21.15.119:0/3965801874' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-03-28T00:15:24.050 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout { 2024-03-28T00:15:24.050 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-03-28T00:15:24.051 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-28T00:15:24.051 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-03-28T00:15:24.051 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-03-28T00:15:24.051 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout } 2024-03-28T00:15:24.051 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:23.057+0000 7f41b2200700 1 Processor -- start 2024-03-28T00:15:24.051 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:23.058+0000 7f41b2200700 1 -- start start 2024-03-28T00:15:24.051 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:23.058+0000 7f41b2200700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f41ac107b40 0x7f41ac107f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:15:24.051 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:23.058+0000 7f41b2200700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41ac108530 con 0x7f41ac107b40 2024-03-28T00:15:24.051 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:23.058+0000 7f41ab400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f41ac107b40 0x7f41ac107f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:15:24.051 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:23.059+0000 7f41ab400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f41ac107b40 0x7f41ac107f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40092/0 (socket says 172.21.15.119:40092) 2024-03-28T00:15:24.051 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:23.059+0000 7f41ab400700 1 -- 172.21.15.119:0/4230116683 learned_addr learned my addr 172.21.15.119:0/4230116683 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:15:24.052 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:23.059+0000 7f41ab400700 1 -- 172.21.15.119:0/4230116683 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41ac108cf0 con 0x7f41ac107b40 2024-03-28T00:15:24.052 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:23.059+0000 7f41ab400700 1 --2- 172.21.15.119:0/4230116683 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f41ac107b40 0x7f41ac107f60 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f419c0098a0 tx=0x7f419c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fefdec59a55c37b6 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:15:24.052 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:23.060+0000 7f41aa000700 1 -- 172.21.15.119:0/4230116683 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f419c004160 con 0x7f41ac107b40 2024-03-28T00:15:24.052 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:23.060+0000 7f41aa000700 1 -- 172.21.15.119:0/4230116683 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f419c004300 con 0x7f41ac107b40 2024-03-28T00:15:24.052 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:23.060+0000 7f41aa000700 1 -- 172.21.15.119:0/4230116683 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f419c011b50 con 0x7f41ac107b40 2024-03-28T00:15:24.052 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:23.061+0000 7f41b2200700 1 -- 172.21.15.119:0/4230116683 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f41ac107b40 msgr2=0x7f41ac107f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:15:24.052 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:23.061+0000 7f41b2200700 1 --2- 172.21.15.119:0/4230116683 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f41ac107b40 0x7f41ac107f60 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f419c0098a0 tx=0x7f419c009bb0 comp rx=0 tx=0).stop 2024-03-28T00:15:24.052 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:23.061+0000 7f41b2200700 1 -- 172.21.15.119:0/4230116683 shutdown_connections 2024-03-28T00:15:24.052 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:23.061+0000 7f41b2200700 1 --2- 172.21.15.119:0/4230116683 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f41ac107b40 0x7f41ac107f60 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:15:24.052 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:23.061+0000 7f41b2200700 1 -- 172.21.15.119:0/4230116683 >> 172.21.15.119:0/4230116683 conn(0x7f41ac103140 msgr2=0x7f41ac105560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:15:24.052 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:23.061+0000 7f41b2200700 1 -- 172.21.15.119:0/4230116683 shutdown_connections 2024-03-28T00:15:24.052 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:23.061+0000 7f41b2200700 1 -- 172.21.15.119:0/4230116683 wait complete. 2024-03-28T00:15:24.052 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:23.062+0000 7f41b2200700 1 Processor -- start 2024-03-28T00:15:24.052 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:23.062+0000 7f41b2200700 1 -- start start 2024-03-28T00:15:24.053 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:23.063+0000 7f41b2200700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f41ac107b40 0x7f41ac19bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:15:24.053 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:23.063+0000 7f41b2200700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41ac10c0a0 con 0x7f41ac107b40 2024-03-28T00:15:24.053 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:23.063+0000 7f41ab400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f41ac107b40 0x7f41ac19bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:15:24.053 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:23.063+0000 7f41ab400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f41ac107b40 0x7f41ac19bae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40100/0 (socket says 172.21.15.119:40100) 2024-03-28T00:15:24.053 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:23.063+0000 7f41ab400700 1 -- 172.21.15.119:0/3965801874 learned_addr learned my addr 172.21.15.119:0/3965801874 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:15:24.053 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:23.064+0000 7f41ab400700 1 -- 172.21.15.119:0/3965801874 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41ac19c020 con 0x7f41ac107b40 2024-03-28T00:15:24.053 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:23.064+0000 7f41ab400700 1 --2- 172.21.15.119:0/3965801874 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f41ac107b40 0x7f41ac19bae0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f419c011850 tx=0x7f419c011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:15:24.053 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:23.064+0000 7f41a3e00700 1 -- 172.21.15.119:0/3965801874 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f419c003af0 con 0x7f41ac107b40 2024-03-28T00:15:24.053 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:23.064+0000 7f41a3e00700 1 -- 172.21.15.119:0/3965801874 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f419c020070 con 0x7f41ac107b40 2024-03-28T00:15:24.053 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:23.064+0000 7f41b2200700 1 -- 172.21.15.119:0/3965801874 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41ac19c2b0 con 0x7f41ac107b40 2024-03-28T00:15:24.053 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:23.064+0000 7f41a3e00700 1 -- 172.21.15.119:0/3965801874 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f419c0195a0 con 0x7f41ac107b40 2024-03-28T00:15:24.053 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:23.064+0000 7f41b2200700 1 -- 172.21.15.119:0/3965801874 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41ac19efa0 con 0x7f41ac107b40 2024-03-28T00:15:24.053 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:23.065+0000 7f41b2200700 1 -- 172.21.15.119:0/3965801874 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41ac069260 con 0x7f41ac107b40 2024-03-28T00:15:24.053 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:23.065+0000 7f41a3e00700 1 -- 172.21.15.119:0/3965801874 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 46106+0+0 (secure 0 0 0) 0x7f419c019850 con 0x7f41ac107b40 2024-03-28T00:15:24.053 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:23.066+0000 7f41a3e00700 1 --2- 172.21.15.119:0/3965801874 >> [v2:172.21.15.119:6800/2056426384,v1:172.21.15.119:6801/2056426384] conn(0x7f4190038f70 0x7f419003b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:15:24.054 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:23.066+0000 7f41a3e00700 1 -- 172.21.15.119:0/3965801874 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f419c011e00 con 0x7f41ac107b40 2024-03-28T00:15:24.054 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:23.066+0000 7f41aaa00700 1 -- 172.21.15.119:0/3965801874 >> [v2:172.21.15.119:6800/2056426384,v1:172.21.15.119:6801/2056426384] conn(0x7f4190038f70 msgr2=0x7f419003b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6800/2056426384 2024-03-28T00:15:24.054 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:23.066+0000 7f41aaa00700 1 --2- 172.21.15.119:0/3965801874 >> [v2:172.21.15.119:6800/2056426384,v1:172.21.15.119:6801/2056426384] conn(0x7f4190038f70 0x7f419003b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-28T00:15:24.054 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:23.071+0000 7f41a3e00700 1 -- 172.21.15.119:0/3965801874 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f419c003c90 con 0x7f41ac107b40 2024-03-28T00:15:24.054 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:23.267+0000 7f41aaa00700 1 -- 172.21.15.119:0/3965801874 >> [v2:172.21.15.119:6800/2056426384,v1:172.21.15.119:6801/2056426384] conn(0x7f4190038f70 msgr2=0x7f419003b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6800/2056426384 2024-03-28T00:15:24.054 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:23.267+0000 7f41aaa00700 1 --2- 172.21.15.119:0/3965801874 >> [v2:172.21.15.119:6800/2056426384,v1:172.21.15.119:6801/2056426384] conn(0x7f4190038f70 0x7f419003b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-28T00:15:24.054 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:23.351+0000 7f41b2200700 1 -- 172.21.15.119:0/3965801874 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f41ac19f170 con 0x7f41ac107b40 2024-03-28T00:15:24.054 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:23.352+0000 7f41a3e00700 1 -- 172.21.15.119:0/3965801874 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+88 (secure 0 0 0) 0x7f41ac19f170 con 0x7f41ac107b40 2024-03-28T00:15:24.054 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:23.356+0000 7f41b2200700 1 -- 172.21.15.119:0/3965801874 >> [v2:172.21.15.119:6800/2056426384,v1:172.21.15.119:6801/2056426384] conn(0x7f4190038f70 msgr2=0x7f419003b430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-28T00:15:24.054 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:23.356+0000 7f41b2200700 1 --2- 172.21.15.119:0/3965801874 >> [v2:172.21.15.119:6800/2056426384,v1:172.21.15.119:6801/2056426384] conn(0x7f4190038f70 0x7f419003b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:15:24.054 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:23.356+0000 7f41b2200700 1 -- 172.21.15.119:0/3965801874 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f41ac107b40 msgr2=0x7f41ac19bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:15:24.054 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:23.356+0000 7f41b2200700 1 --2- 172.21.15.119:0/3965801874 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f41ac107b40 0x7f41ac19bae0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f419c011850 tx=0x7f419c011c10 comp rx=0 tx=0).stop 2024-03-28T00:15:24.054 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:23.356+0000 7f41b2200700 1 -- 172.21.15.119:0/3965801874 shutdown_connections 2024-03-28T00:15:24.054 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:23.356+0000 7f41b2200700 1 --2- 172.21.15.119:0/3965801874 >> [v2:172.21.15.119:6800/2056426384,v1:172.21.15.119:6801/2056426384] conn(0x7f4190038f70 0x7f419003b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:15:24.055 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:23.356+0000 7f41b2200700 1 --2- 172.21.15.119:0/3965801874 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f41ac107b40 0x7f41ac19bae0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:15:24.055 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:23.356+0000 7f41b2200700 1 -- 172.21.15.119:0/3965801874 >> 172.21.15.119:0/3965801874 conn(0x7f41ac103140 msgr2=0x7f41ac103c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:15:24.055 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:23.357+0000 7f41b2200700 1 -- 172.21.15.119:0/3965801874 shutdown_connections 2024-03-28T00:15:24.055 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:23.357+0000 7f41b2200700 1 -- 172.21.15.119:0/3965801874 wait complete. 2024-03-28T00:15:24.055 INFO:teuthology.orchestra.run.smithi119.stdout:Waiting for the mgr to restart... 2024-03-28T00:15:24.055 INFO:teuthology.orchestra.run.smithi119.stdout:Waiting for mgr epoch 5... 2024-03-28T00:15:24.196 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 00:15:23 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-x[31752]: 2024-03-28T00:15:23.917+0000 7f8e1aaa9200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-28T00:15:24.196 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 00:15:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-x[31752]: 2024-03-28T00:15:24.027+0000 7f8e1aaa9200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-28T00:15:24.484 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 00:15:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-x[31752]: 2024-03-28T00:15:24.368+0000 7f8e1aaa9200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-28T00:15:24.760 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 00:15:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-x[31752]: 2024-03-28T00:15:24.512+0000 7f8e1aaa9200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-28T00:15:24.760 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 00:15:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-x[31752]: 2024-03-28T00:15:24.628+0000 7f8e1aaa9200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-28T00:15:24.760 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 00:15:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-x[31752]: 2024-03-28T00:15:24.753+0000 7f8e1aaa9200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-28T00:15:25.155 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 00:15:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-x[31752]: 2024-03-28T00:15:24.970+0000 7f8e1aaa9200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-28T00:15:26.656 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 00:15:26 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-x[31752]: 2024-03-28T00:15:26.215+0000 7f8e1aaa9200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-28T00:15:26.656 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 00:15:26 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-x[31752]: 2024-03-28T00:15:26.383+0000 7f8e1aaa9200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-28T00:15:27.155 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 00:15:26 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-x[31752]: 2024-03-28T00:15:26.793+0000 7f8e1aaa9200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-28T00:15:27.155 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 00:15:26 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-x[31752]: 2024-03-28T00:15:26.911+0000 7f8e1aaa9200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-28T00:15:27.649 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 00:15:27 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-x[31752]: 2024-03-28T00:15:27.162+0000 7f8e1aaa9200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-28T00:15:27.649 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 00:15:27 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-x[31752]: 2024-03-28T00:15:27.404+0000 7f8e1aaa9200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-28T00:15:27.906 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 00:15:27 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-x[31752]: 2024-03-28T00:15:27.775+0000 7f8e1aaa9200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-28T00:15:28.391 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 00:15:28 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-x[31752]: 2024-03-28T00:15:28.089+0000 7f8e1aaa9200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-28T00:15:28.391 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 00:15:28 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-x[31752]: 2024-03-28T00:15:28.385+0000 7f8e1aaa9200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-28T00:15:29.655 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 00:15:29 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-x[31752]: 2024-03-28T00:15:29.383+0000 7f8e1aaa9200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-28T00:15:30.405 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 00:15:30 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-x[31752]: 2024-03-28T00:15:30.067+0000 7f8e1aaa9200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-28T00:15:30.405 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 00:15:30 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-x[31752]: 2024-03-28T00:15:30.194+0000 7f8e1aaa9200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-28T00:15:30.905 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 00:15:30 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-x[31752]: 2024-03-28T00:15:30.468+0000 7f8e1aaa9200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-28T00:15:30.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:30 smithi119 ceph-mon[31518]: Active manager daemon x restarted 2024-03-28T00:15:30.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:30 smithi119 ceph-mon[31518]: Activating manager daemon x 2024-03-28T00:15:30.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:30 smithi119 ceph-mon[31518]: osdmap e2: 0 total, 0 up, 0 in 2024-03-28T00:15:30.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:30 smithi119 ceph-mon[31518]: mgrmap e6: x(active, starting, since 0.00992261s) 2024-03-28T00:15:30.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:30 smithi119 ceph-mon[31518]: from='mgr.14120 172.21.15.119:0/4048015587' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-28T00:15:30.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:30 smithi119 ceph-mon[31518]: from='mgr.14120 172.21.15.119:0/4048015587' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-28T00:15:30.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:30 smithi119 ceph-mon[31518]: from='mgr.14120 172.21.15.119:0/4048015587' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-28T00:15:30.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:30 smithi119 ceph-mon[31518]: from='mgr.14120 172.21.15.119:0/4048015587' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-28T00:15:30.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:30 smithi119 ceph-mon[31518]: from='mgr.14120 172.21.15.119:0/4048015587' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-28T00:15:30.907 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:30 smithi119 ceph-mon[31518]: Manager daemon x is now available 2024-03-28T00:15:30.907 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:30 smithi119 ceph-mon[31518]: from='mgr.14120 172.21.15.119:0/4048015587' entity='mgr.x' 2024-03-28T00:15:30.907 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:30 smithi119 ceph-mon[31518]: from='mgr.14120 172.21.15.119:0/4048015587' entity='mgr.x' 2024-03-28T00:15:31.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:31 smithi119 ceph-mon[31518]: Found migration_current of "None". Setting to last migration. 2024-03-28T00:15:31.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:31 smithi119 ceph-mon[31518]: from='mgr.14120 172.21.15.119:0/4048015587' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:15:31.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:31 smithi119 ceph-mon[31518]: from='mgr.14120 172.21.15.119:0/4048015587' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:15:31.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:31 smithi119 ceph-mon[31518]: from='mgr.14120 172.21.15.119:0/4048015587' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:15:31.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:31 smithi119 ceph-mon[31518]: from='mgr.14120 172.21.15.119:0/4048015587' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-28T00:15:31.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:31 smithi119 ceph-mon[31518]: from='mgr.14120 172.21.15.119:0/4048015587' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-28T00:15:31.907 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:31 smithi119 ceph-mon[31518]: from='mgr.14120 172.21.15.119:0/4048015587' entity='mgr.x' 2024-03-28T00:15:31.907 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:31 smithi119 ceph-mon[31518]: from='mgr.14120 172.21.15.119:0/4048015587' entity='mgr.x' 2024-03-28T00:15:31.907 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:31 smithi119 ceph-mon[31518]: from='mgr.14120 172.21.15.119:0/4048015587' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:15:31.907 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:31 smithi119 ceph-mon[31518]: mgrmap e7: x(active, since 1.0188s) 2024-03-28T00:15:32.237 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout { 2024-03-28T00:15:32.237 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-03-28T00:15:32.237 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "initialized": true 2024-03-28T00:15:32.238 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout } 2024-03-28T00:15:32.238 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:24.629+0000 7f7fe5000700 1 Processor -- start 2024-03-28T00:15:32.238 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:24.629+0000 7f7fe5000700 1 -- start start 2024-03-28T00:15:32.238 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:24.630+0000 7f7fe5000700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7fe0107900 0x7f7fe0107d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:15:32.238 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:24.630+0000 7f7fe5000700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fe01082f0 con 0x7f7fe0107900 2024-03-28T00:15:32.238 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:24.630+0000 7f7fdf400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7fe0107900 0x7f7fe0107d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:15:32.238 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:24.630+0000 7f7fdf400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7fe0107900 0x7f7fe0107d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:56872/0 (socket says 172.21.15.119:56872) 2024-03-28T00:15:32.238 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:24.630+0000 7f7fdf400700 1 -- 172.21.15.119:0/1476862864 learned_addr learned my addr 172.21.15.119:0/1476862864 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:15:32.238 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:24.630+0000 7f7fdf400700 1 -- 172.21.15.119:0/1476862864 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fe0108b00 con 0x7f7fe0107900 2024-03-28T00:15:32.238 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:24.631+0000 7f7fdf400700 1 --2- 172.21.15.119:0/1476862864 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7fe0107900 0x7f7fe0107d20 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f7fd00098a0 tx=0x7f7fd0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2954f1b16a3c5f81 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:15:32.238 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:24.631+0000 7f7fde000700 1 -- 172.21.15.119:0/1476862864 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7fd0004160 con 0x7f7fe0107900 2024-03-28T00:15:32.238 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:24.631+0000 7f7fde000700 1 -- 172.21.15.119:0/1476862864 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7fd0004300 con 0x7f7fe0107900 2024-03-28T00:15:32.238 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:24.631+0000 7f7fde000700 1 -- 172.21.15.119:0/1476862864 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7fd0011b50 con 0x7f7fe0107900 2024-03-28T00:15:32.238 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:24.632+0000 7f7fe5000700 1 -- 172.21.15.119:0/1476862864 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7fe0107900 msgr2=0x7f7fe0107d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:15:32.239 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:24.632+0000 7f7fe5000700 1 --2- 172.21.15.119:0/1476862864 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7fe0107900 0x7f7fe0107d20 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f7fd00098a0 tx=0x7f7fd0009bb0 comp rx=0 tx=0).stop 2024-03-28T00:15:32.239 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:24.632+0000 7f7fe5000700 1 -- 172.21.15.119:0/1476862864 shutdown_connections 2024-03-28T00:15:32.239 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:24.632+0000 7f7fe5000700 1 --2- 172.21.15.119:0/1476862864 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7fe0107900 0x7f7fe0107d20 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:15:32.239 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:24.632+0000 7f7fe5000700 1 -- 172.21.15.119:0/1476862864 >> 172.21.15.119:0/1476862864 conn(0x7f7fe0102ec0 msgr2=0x7f7fe01052e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:15:32.239 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:24.632+0000 7f7fe5000700 1 -- 172.21.15.119:0/1476862864 shutdown_connections 2024-03-28T00:15:32.239 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:24.632+0000 7f7fe5000700 1 -- 172.21.15.119:0/1476862864 wait complete. 2024-03-28T00:15:32.239 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:24.633+0000 7f7fe5000700 1 Processor -- start 2024-03-28T00:15:32.239 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:24.633+0000 7f7fe5000700 1 -- start start 2024-03-28T00:15:32.239 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:24.633+0000 7f7fe5000700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7fe0107900 0x7f7fe019b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:15:32.239 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:24.633+0000 7f7fe5000700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fe010beb0 con 0x7f7fe0107900 2024-03-28T00:15:32.239 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:24.634+0000 7f7fdf400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7fe0107900 0x7f7fe019b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:15:32.241 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:24.634+0000 7f7fdf400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7fe0107900 0x7f7fe019b8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:56888/0 (socket says 172.21.15.119:56888) 2024-03-28T00:15:32.241 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:24.634+0000 7f7fdf400700 1 -- 172.21.15.119:0/1918348786 learned_addr learned my addr 172.21.15.119:0/1918348786 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:15:32.241 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:24.634+0000 7f7fdf400700 1 -- 172.21.15.119:0/1918348786 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fe019be10 con 0x7f7fe0107900 2024-03-28T00:15:32.241 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:24.634+0000 7f7fdf400700 1 --2- 172.21.15.119:0/1918348786 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7fe0107900 0x7f7fe019b8d0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f7fd0011670 tx=0x7f7fd0011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:15:32.241 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:24.635+0000 7f7fd7e00700 1 -- 172.21.15.119:0/1918348786 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7fd00039c0 con 0x7f7fe0107900 2024-03-28T00:15:32.241 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:24.635+0000 7f7fd7e00700 1 -- 172.21.15.119:0/1918348786 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7fd0011e50 con 0x7f7fe0107900 2024-03-28T00:15:32.241 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:24.635+0000 7f7fe5000700 1 -- 172.21.15.119:0/1918348786 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fe019c0a0 con 0x7f7fe0107900 2024-03-28T00:15:32.242 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:24.635+0000 7f7fd7e00700 1 -- 172.21.15.119:0/1918348786 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7fd001a5c0 con 0x7f7fe0107900 2024-03-28T00:15:32.242 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:24.635+0000 7f7fe5000700 1 -- 172.21.15.119:0/1918348786 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fe019ed90 con 0x7f7fe0107900 2024-03-28T00:15:32.242 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:24.636+0000 7f7fd7e00700 1 -- 172.21.15.119:0/1918348786 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 46106+0+0 (secure 0 0 0) 0x7f7fd0024430 con 0x7f7fe0107900 2024-03-28T00:15:32.242 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:24.637+0000 7f7fd7e00700 1 --2- 172.21.15.119:0/1918348786 >> [v2:172.21.15.119:6800/2056426384,v1:172.21.15.119:6801/2056426384] conn(0x7f7fc4038ed0 0x7f7fc403b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:15:32.242 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:24.637+0000 7f7fd7e00700 1 -- 172.21.15.119:0/1918348786 --> [v2:172.21.15.119:6800/2056426384,v1:172.21.15.119:6801/2056426384] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7fc403baa0 con 0x7f7fc4038ed0 2024-03-28T00:15:32.242 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:24.637+0000 7f7fd7e00700 1 -- 172.21.15.119:0/1918348786 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f7fd004d2f0 con 0x7f7fe0107900 2024-03-28T00:15:32.242 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:24.637+0000 7f7fdea00700 1 -- 172.21.15.119:0/1918348786 >> [v2:172.21.15.119:6800/2056426384,v1:172.21.15.119:6801/2056426384] conn(0x7f7fc4038ed0 msgr2=0x7f7fc403b390 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6800/2056426384 2024-03-28T00:15:32.242 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:24.637+0000 7f7fdea00700 1 --2- 172.21.15.119:0/1918348786 >> [v2:172.21.15.119:6800/2056426384,v1:172.21.15.119:6801/2056426384] conn(0x7f7fc4038ed0 0x7f7fc403b390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-28T00:15:32.242 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:24.838+0000 7f7fdea00700 1 -- 172.21.15.119:0/1918348786 >> [v2:172.21.15.119:6800/2056426384,v1:172.21.15.119:6801/2056426384] conn(0x7f7fc4038ed0 msgr2=0x7f7fc403b390 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6800/2056426384 2024-03-28T00:15:32.242 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:24.838+0000 7f7fdea00700 1 --2- 172.21.15.119:0/1918348786 >> [v2:172.21.15.119:6800/2056426384,v1:172.21.15.119:6801/2056426384] conn(0x7f7fc4038ed0 0x7f7fc403b390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-28T00:15:32.242 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:25.239+0000 7f7fdea00700 1 -- 172.21.15.119:0/1918348786 >> [v2:172.21.15.119:6800/2056426384,v1:172.21.15.119:6801/2056426384] conn(0x7f7fc4038ed0 msgr2=0x7f7fc403b390 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6800/2056426384 2024-03-28T00:15:32.242 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:25.239+0000 7f7fdea00700 1 --2- 172.21.15.119:0/1918348786 >> [v2:172.21.15.119:6800/2056426384,v1:172.21.15.119:6801/2056426384] conn(0x7f7fc4038ed0 0x7f7fc403b390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-28T00:15:32.242 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:26.040+0000 7f7fdea00700 1 -- 172.21.15.119:0/1918348786 >> [v2:172.21.15.119:6800/2056426384,v1:172.21.15.119:6801/2056426384] conn(0x7f7fc4038ed0 msgr2=0x7f7fc403b390 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6800/2056426384 2024-03-28T00:15:32.242 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:26.040+0000 7f7fdea00700 1 --2- 172.21.15.119:0/1918348786 >> [v2:172.21.15.119:6800/2056426384,v1:172.21.15.119:6801/2056426384] conn(0x7f7fc4038ed0 0x7f7fc403b390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-28T00:15:32.243 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:27.642+0000 7f7fdea00700 1 -- 172.21.15.119:0/1918348786 >> [v2:172.21.15.119:6800/2056426384,v1:172.21.15.119:6801/2056426384] conn(0x7f7fc4038ed0 msgr2=0x7f7fc403b390 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6800/2056426384 2024-03-28T00:15:32.243 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:27.642+0000 7f7fdea00700 1 --2- 172.21.15.119:0/1918348786 >> [v2:172.21.15.119:6800/2056426384,v1:172.21.15.119:6801/2056426384] conn(0x7f7fc4038ed0 0x7f7fc403b390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-03-28T00:15:32.243 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:30.478+0000 7f7fd7e00700 1 -- 172.21.15.119:0/1918348786 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mgrmap(e 6) v1 ==== 45873+0+0 (secure 0 0 0) 0x7f7fd0024770 con 0x7f7fe0107900 2024-03-28T00:15:32.243 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:30.479+0000 7f7fd7e00700 1 -- 172.21.15.119:0/1918348786 >> [v2:172.21.15.119:6800/2056426384,v1:172.21.15.119:6801/2056426384] conn(0x7f7fc4038ed0 msgr2=0x7f7fc403b390 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-28T00:15:32.243 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:30.479+0000 7f7fd7e00700 1 --2- 172.21.15.119:0/1918348786 >> [v2:172.21.15.119:6800/2056426384,v1:172.21.15.119:6801/2056426384] conn(0x7f7fc4038ed0 0x7f7fc403b390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:15:32.243 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:31.485+0000 7f7fd7e00700 1 -- 172.21.15.119:0/1918348786 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mgrmap(e 7) v1 ==== 46000+0+0 (secure 0 0 0) 0x7f7fd00188a0 con 0x7f7fe0107900 2024-03-28T00:15:32.243 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:31.486+0000 7f7fd7e00700 1 --2- 172.21.15.119:0/1918348786 >> [v2:172.21.15.119:6800/2575246519,v1:172.21.15.119:6801/2575246519] conn(0x7f7fc403c7c0 0x7f7fc403ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:15:32.243 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:31.486+0000 7f7fd7e00700 1 -- 172.21.15.119:0/1918348786 --> [v2:172.21.15.119:6800/2575246519,v1:172.21.15.119:6801/2575246519] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7fd0008ea0 con 0x7f7fc403c7c0 2024-03-28T00:15:32.243 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:31.487+0000 7f7fdea00700 1 --2- 172.21.15.119:0/1918348786 >> [v2:172.21.15.119:6800/2575246519,v1:172.21.15.119:6801/2575246519] conn(0x7f7fc403c7c0 0x7f7fc403ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:15:32.243 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:31.490+0000 7f7fdea00700 1 --2- 172.21.15.119:0/1918348786 >> [v2:172.21.15.119:6800/2575246519,v1:172.21.15.119:6801/2575246519] conn(0x7f7fc403c7c0 0x7f7fc403ebb0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f7fcc003a10 tx=0x7f7fcc009290 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:15:32.243 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:31.491+0000 7f7fd7e00700 1 -- 172.21.15.119:0/1918348786 <== mgr.14120 v2:172.21.15.119:6800/2575246519 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+7774 (secure 0 0 0) 0x7f7fd0008ea0 con 0x7f7fc403c7c0 2024-03-28T00:15:32.243 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:31.500+0000 7f7fe5000700 1 -- 172.21.15.119:0/1918348786 --> [v2:172.21.15.119:6800/2575246519,v1:172.21.15.119:6801/2575246519] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f7fe0069260 con 0x7f7fc403c7c0 2024-03-28T00:15:32.243 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:31.501+0000 7f7fd7e00700 1 -- 172.21.15.119:0/1918348786 <== mgr.14120 v2:172.21.15.119:6800/2575246519 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f7fe0069260 con 0x7f7fc403c7c0 2024-03-28T00:15:32.243 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:31.501+0000 7f7fe5000700 1 -- 172.21.15.119:0/1918348786 >> [v2:172.21.15.119:6800/2575246519,v1:172.21.15.119:6801/2575246519] conn(0x7f7fc403c7c0 msgr2=0x7f7fc403ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:15:32.243 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:31.501+0000 7f7fe5000700 1 --2- 172.21.15.119:0/1918348786 >> [v2:172.21.15.119:6800/2575246519,v1:172.21.15.119:6801/2575246519] conn(0x7f7fc403c7c0 0x7f7fc403ebb0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f7fcc003a10 tx=0x7f7fcc009290 comp rx=0 tx=0).stop 2024-03-28T00:15:32.244 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:31.501+0000 7f7fe5000700 1 -- 172.21.15.119:0/1918348786 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7fe0107900 msgr2=0x7f7fe019b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:15:32.244 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:31.501+0000 7f7fe5000700 1 --2- 172.21.15.119:0/1918348786 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7fe0107900 0x7f7fe019b8d0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f7fd0011670 tx=0x7f7fd0011b60 comp rx=0 tx=0).stop 2024-03-28T00:15:32.244 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:31.501+0000 7f7fe5000700 1 -- 172.21.15.119:0/1918348786 shutdown_connections 2024-03-28T00:15:32.244 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:31.502+0000 7f7fe5000700 1 --2- 172.21.15.119:0/1918348786 >> [v2:172.21.15.119:6800/2575246519,v1:172.21.15.119:6801/2575246519] conn(0x7f7fc403c7c0 0x7f7fc403ebb0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:15:32.244 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:31.502+0000 7f7fe5000700 1 --2- 172.21.15.119:0/1918348786 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7fe0107900 0x7f7fe019b8d0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:15:32.244 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:31.502+0000 7f7fe5000700 1 --2- 172.21.15.119:0/1918348786 >> [v2:172.21.15.119:6800/2056426384,v1:172.21.15.119:6801/2056426384] conn(0x7f7fc4038ed0 0x7f7fc403b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:15:32.244 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:31.502+0000 7f7fe5000700 1 -- 172.21.15.119:0/1918348786 >> 172.21.15.119:0/1918348786 conn(0x7f7fe0102ec0 msgr2=0x7f7fe0103a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:15:32.244 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:31.502+0000 7f7fe5000700 1 -- 172.21.15.119:0/1918348786 shutdown_connections 2024-03-28T00:15:32.244 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:31.502+0000 7f7fe5000700 1 -- 172.21.15.119:0/1918348786 wait complete. 2024-03-28T00:15:32.244 INFO:teuthology.orchestra.run.smithi119.stdout:mgr epoch 5 is available 2024-03-28T00:15:32.244 INFO:teuthology.orchestra.run.smithi119.stdout:Setting orchestrator backend to cephadm... 2024-03-28T00:15:32.839 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:32 smithi119 ceph-mon[31518]: [28/Mar/2024:00:15:31] ENGINE Bus STARTING 2024-03-28T00:15:32.839 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:32 smithi119 ceph-mon[31518]: [28/Mar/2024:00:15:31] ENGINE Serving on http://172.21.15.119:8765 2024-03-28T00:15:32.839 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:32 smithi119 ceph-mon[31518]: [28/Mar/2024:00:15:31] ENGINE Serving on https://172.21.15.119:7150 2024-03-28T00:15:32.839 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:32 smithi119 ceph-mon[31518]: [28/Mar/2024:00:15:31] ENGINE Bus STARTED 2024-03-28T00:15:32.839 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:32 smithi119 ceph-mon[31518]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-03-28T00:15:32.839 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:32 smithi119 ceph-mon[31518]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-03-28T00:15:33.743 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:33 smithi119 ceph-mon[31518]: mgrmap e8: x(active, since 2s) 2024-03-28T00:15:33.744 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:33 smithi119 ceph-mon[31518]: from='mgr.14120 172.21.15.119:0/4048015587' entity='mgr.x' 2024-03-28T00:15:33.744 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:33 smithi119 ceph-mon[31518]: from='mgr.14120 172.21.15.119:0/4048015587' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:15:33.757 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:32.832+0000 7fa7f1000700 1 Processor -- start 2024-03-28T00:15:33.763 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:32.833+0000 7fa7f1000700 1 -- start start 2024-03-28T00:15:33.763 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:32.833+0000 7fa7f1000700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa7ec07de70 0x7fa7ec07c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:15:33.763 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:32.833+0000 7fa7f1000700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7ec07c8a0 con 0x7fa7ec07de70 2024-03-28T00:15:33.763 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:32.833+0000 7fa7e9e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa7ec07de70 0x7fa7ec07c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:15:33.763 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:32.834+0000 7fa7e9e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa7ec07de70 0x7fa7ec07c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:56948/0 (socket says 172.21.15.119:56948) 2024-03-28T00:15:33.763 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:32.834+0000 7fa7e9e00700 1 -- 172.21.15.119:0/1810615035 learned_addr learned my addr 172.21.15.119:0/1810615035 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:15:33.763 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:32.834+0000 7fa7e9e00700 1 -- 172.21.15.119:0/1810615035 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7ec07ca20 con 0x7fa7ec07de70 2024-03-28T00:15:33.764 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:32.834+0000 7fa7e9e00700 1 --2- 172.21.15.119:0/1810615035 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa7ec07de70 0x7fa7ec07c2d0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fa7dc0098a0 tx=0x7fa7dc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f84b12983051cb8 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:15:33.764 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:32.835+0000 7fa7e8a00700 1 -- 172.21.15.119:0/1810615035 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa7dc004160 con 0x7fa7ec07de70 2024-03-28T00:15:33.764 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:32.835+0000 7fa7e8a00700 1 -- 172.21.15.119:0/1810615035 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa7dc004300 con 0x7fa7ec07de70 2024-03-28T00:15:33.764 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:32.835+0000 7fa7e8a00700 1 -- 172.21.15.119:0/1810615035 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa7dc011b50 con 0x7fa7ec07de70 2024-03-28T00:15:33.764 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:32.835+0000 7fa7f1000700 1 -- 172.21.15.119:0/1810615035 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa7ec07de70 msgr2=0x7fa7ec07c2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:15:33.764 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:32.835+0000 7fa7f1000700 1 --2- 172.21.15.119:0/1810615035 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa7ec07de70 0x7fa7ec07c2d0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fa7dc0098a0 tx=0x7fa7dc009bb0 comp rx=0 tx=0).stop 2024-03-28T00:15:33.764 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:32.836+0000 7fa7f1000700 1 -- 172.21.15.119:0/1810615035 shutdown_connections 2024-03-28T00:15:33.764 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:32.836+0000 7fa7f1000700 1 --2- 172.21.15.119:0/1810615035 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa7ec07de70 0x7fa7ec07c2d0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:15:33.764 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:32.836+0000 7fa7f1000700 1 -- 172.21.15.119:0/1810615035 >> 172.21.15.119:0/1810615035 conn(0x7fa7ec103190 msgr2=0x7fa7ec1055b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:15:33.764 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:32.836+0000 7fa7f1000700 1 -- 172.21.15.119:0/1810615035 shutdown_connections 2024-03-28T00:15:33.764 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:32.836+0000 7fa7f1000700 1 -- 172.21.15.119:0/1810615035 wait complete. 2024-03-28T00:15:33.764 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:32.837+0000 7fa7f1000700 1 Processor -- start 2024-03-28T00:15:33.764 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:32.837+0000 7fa7f1000700 1 -- start start 2024-03-28T00:15:33.764 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:32.837+0000 7fa7f1000700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa7ec07de70 0x7fa7ec19deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:15:33.764 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:32.837+0000 7fa7f1000700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7ec10e100 con 0x7fa7ec07de70 2024-03-28T00:15:33.765 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:32.838+0000 7fa7e9e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa7ec07de70 0x7fa7ec19deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:15:33.765 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:32.838+0000 7fa7e9e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa7ec07de70 0x7fa7ec19deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:56958/0 (socket says 172.21.15.119:56958) 2024-03-28T00:15:33.765 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:32.838+0000 7fa7e9e00700 1 -- 172.21.15.119:0/674034673 learned_addr learned my addr 172.21.15.119:0/674034673 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:15:33.765 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:32.838+0000 7fa7e9e00700 1 -- 172.21.15.119:0/674034673 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7ec19e3f0 con 0x7fa7ec07de70 2024-03-28T00:15:33.765 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:32.838+0000 7fa7e9e00700 1 --2- 172.21.15.119:0/674034673 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa7ec07de70 0x7fa7ec19deb0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fa7dc011850 tx=0x7fa7dc011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:15:33.765 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:32.838+0000 7fa7e2a00700 1 -- 172.21.15.119:0/674034673 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa7dc003af0 con 0x7fa7ec07de70 2024-03-28T00:15:33.765 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:32.838+0000 7fa7e2a00700 1 -- 172.21.15.119:0/674034673 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa7dc020070 con 0x7fa7ec07de70 2024-03-28T00:15:33.765 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:32.838+0000 7fa7f1000700 1 -- 172.21.15.119:0/674034673 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7ec19e680 con 0x7fa7ec07de70 2024-03-28T00:15:33.765 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:32.839+0000 7fa7e2a00700 1 -- 172.21.15.119:0/674034673 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa7dc0195a0 con 0x7fa7ec07de70 2024-03-28T00:15:33.765 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:32.839+0000 7fa7f1000700 1 -- 172.21.15.119:0/674034673 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7ec19f060 con 0x7fa7ec07de70 2024-03-28T00:15:33.765 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:32.840+0000 7fa7e2a00700 1 -- 172.21.15.119:0/674034673 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 8) v1 ==== 46106+0+0 (secure 0 0 0) 0x7fa7dc019850 con 0x7fa7ec07de70 2024-03-28T00:15:33.765 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:32.840+0000 7fa7f1000700 1 -- 172.21.15.119:0/674034673 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7ec0637c0 con 0x7fa7ec07de70 2024-03-28T00:15:33.765 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:32.840+0000 7fa7e2a00700 1 --2- 172.21.15.119:0/674034673 >> [v2:172.21.15.119:6800/2575246519,v1:172.21.15.119:6801/2575246519] conn(0x7fa7d0038f70 0x7fa7d003b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:15:33.765 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:32.840+0000 7fa7e2a00700 1 -- 172.21.15.119:0/674034673 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fa7dc011e00 con 0x7fa7ec07de70 2024-03-28T00:15:33.766 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:32.841+0000 7fa7e9400700 1 --2- 172.21.15.119:0/674034673 >> [v2:172.21.15.119:6800/2575246519,v1:172.21.15.119:6801/2575246519] conn(0x7fa7d0038f70 0x7fa7d003b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:15:33.766 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:32.841+0000 7fa7e9400700 1 --2- 172.21.15.119:0/674034673 >> [v2:172.21.15.119:6800/2575246519,v1:172.21.15.119:6801/2575246519] conn(0x7fa7d0038f70 0x7fa7d003b430 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa7d8006fd0 tx=0x7fa7d8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:15:33.766 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:32.847+0000 7fa7e2a00700 1 -- 172.21.15.119:0/674034673 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa7dc019b40 con 0x7fa7ec07de70 2024-03-28T00:15:33.766 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:33.085+0000 7fa7f1000700 1 -- 172.21.15.119:0/674034673 --> [v2:172.21.15.119:6800/2575246519,v1:172.21.15.119:6801/2575246519] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7fa7ec1043f0 con 0x7fa7d0038f70 2024-03-28T00:15:33.766 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:33.094+0000 7fa7e2a00700 1 -- 172.21.15.119:0/674034673 <== mgr.14120 v2:172.21.15.119:6800/2575246519 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fa7ec1043f0 con 0x7fa7d0038f70 2024-03-28T00:15:33.766 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:33.098+0000 7fa7f1000700 1 -- 172.21.15.119:0/674034673 >> [v2:172.21.15.119:6800/2575246519,v1:172.21.15.119:6801/2575246519] conn(0x7fa7d0038f70 msgr2=0x7fa7d003b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:15:33.766 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:33.098+0000 7fa7f1000700 1 --2- 172.21.15.119:0/674034673 >> [v2:172.21.15.119:6800/2575246519,v1:172.21.15.119:6801/2575246519] conn(0x7fa7d0038f70 0x7fa7d003b430 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa7d8006fd0 tx=0x7fa7d8006e40 comp rx=0 tx=0).stop 2024-03-28T00:15:33.766 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:33.098+0000 7fa7f1000700 1 -- 172.21.15.119:0/674034673 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa7ec07de70 msgr2=0x7fa7ec19deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:15:33.766 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:33.098+0000 7fa7f1000700 1 --2- 172.21.15.119:0/674034673 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa7ec07de70 0x7fa7ec19deb0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fa7dc011850 tx=0x7fa7dc011c10 comp rx=0 tx=0).stop 2024-03-28T00:15:33.766 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:33.098+0000 7fa7f1000700 1 -- 172.21.15.119:0/674034673 shutdown_connections 2024-03-28T00:15:33.766 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:33.098+0000 7fa7f1000700 1 --2- 172.21.15.119:0/674034673 >> [v2:172.21.15.119:6800/2575246519,v1:172.21.15.119:6801/2575246519] conn(0x7fa7d0038f70 0x7fa7d003b430 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:15:33.766 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:33.098+0000 7fa7f1000700 1 --2- 172.21.15.119:0/674034673 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa7ec07de70 0x7fa7ec19deb0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:15:33.766 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:33.098+0000 7fa7f1000700 1 -- 172.21.15.119:0/674034673 >> 172.21.15.119:0/674034673 conn(0x7fa7ec103190 msgr2=0x7fa7ec103ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:15:33.767 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:33.098+0000 7fa7f1000700 1 -- 172.21.15.119:0/674034673 shutdown_connections 2024-03-28T00:15:33.767 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:33.098+0000 7fa7f1000700 1 -- 172.21.15.119:0/674034673 wait complete. 2024-03-28T00:15:34.624 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:34 smithi119 ceph-mon[31518]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T00:15:35.272 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout value unchanged 2024-03-28T00:15:35.272 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:34.362+0000 7fe43e400700 1 Processor -- start 2024-03-28T00:15:35.272 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:34.362+0000 7fe43e400700 1 -- start start 2024-03-28T00:15:35.272 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:34.362+0000 7fe43e400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe43810ec80 0x7fe438111070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:15:35.272 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:34.362+0000 7fe43e400700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe438082380 con 0x7fe43810ec80 2024-03-28T00:15:35.272 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:34.363+0000 7fe437400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe43810ec80 0x7fe438111070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:15:35.272 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:34.363+0000 7fe437400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe43810ec80 0x7fe438111070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:47126/0 (socket says 172.21.15.119:47126) 2024-03-28T00:15:35.272 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:34.363+0000 7fe437400700 1 -- 172.21.15.119:0/2618813329 learned_addr learned my addr 172.21.15.119:0/2618813329 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:15:35.272 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:34.363+0000 7fe437400700 1 -- 172.21.15.119:0/2618813329 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4381115b0 con 0x7fe43810ec80 2024-03-28T00:15:35.272 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:34.363+0000 7fe437400700 1 --2- 172.21.15.119:0/2618813329 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe43810ec80 0x7fe438111070 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fe4280098a0 tx=0x7fe428009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cd7742156b67e7c server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:15:35.272 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:34.364+0000 7fe436a00700 1 -- 172.21.15.119:0/2618813329 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe428004160 con 0x7fe43810ec80 2024-03-28T00:15:35.273 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:34.364+0000 7fe436a00700 1 -- 172.21.15.119:0/2618813329 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe428004300 con 0x7fe43810ec80 2024-03-28T00:15:35.273 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:34.364+0000 7fe436a00700 1 -- 172.21.15.119:0/2618813329 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe428011b50 con 0x7fe43810ec80 2024-03-28T00:15:35.273 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:34.365+0000 7fe43e400700 1 -- 172.21.15.119:0/2618813329 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe43810ec80 msgr2=0x7fe438111070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:15:35.273 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:34.365+0000 7fe43e400700 1 --2- 172.21.15.119:0/2618813329 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe43810ec80 0x7fe438111070 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fe4280098a0 tx=0x7fe428009bb0 comp rx=0 tx=0).stop 2024-03-28T00:15:35.273 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:34.365+0000 7fe43e400700 1 -- 172.21.15.119:0/2618813329 shutdown_connections 2024-03-28T00:15:35.273 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:34.365+0000 7fe43e400700 1 --2- 172.21.15.119:0/2618813329 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe43810ec80 0x7fe438111070 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:15:35.273 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:34.365+0000 7fe43e400700 1 -- 172.21.15.119:0/2618813329 >> 172.21.15.119:0/2618813329 conn(0x7fe43807d100 msgr2=0x7fe43807f520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:15:35.273 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:34.365+0000 7fe43e400700 1 -- 172.21.15.119:0/2618813329 shutdown_connections 2024-03-28T00:15:35.273 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:34.365+0000 7fe43e400700 1 -- 172.21.15.119:0/2618813329 wait complete. 2024-03-28T00:15:35.273 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:34.366+0000 7fe43e400700 1 Processor -- start 2024-03-28T00:15:35.273 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:34.366+0000 7fe43e400700 1 -- start start 2024-03-28T00:15:35.273 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:34.366+0000 7fe43e400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe43810ec80 0x7fe4381a2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:15:35.273 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:34.366+0000 7fe43e400700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe438112510 con 0x7fe43810ec80 2024-03-28T00:15:35.274 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:34.367+0000 7fe437400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe43810ec80 0x7fe4381a2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:15:35.274 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:34.367+0000 7fe437400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe43810ec80 0x7fe4381a2080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:47136/0 (socket says 172.21.15.119:47136) 2024-03-28T00:15:35.274 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:34.367+0000 7fe437400700 1 -- 172.21.15.119:0/1538633192 learned_addr learned my addr 172.21.15.119:0/1538633192 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:15:35.274 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:34.367+0000 7fe437400700 1 -- 172.21.15.119:0/1538633192 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4381a25c0 con 0x7fe43810ec80 2024-03-28T00:15:35.274 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:34.367+0000 7fe437400700 1 --2- 172.21.15.119:0/1538633192 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe43810ec80 0x7fe4381a2080 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe428011b60 tx=0x7fe428011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:15:35.274 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:34.367+0000 7fe434c00700 1 -- 172.21.15.119:0/1538633192 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe428003b20 con 0x7fe43810ec80 2024-03-28T00:15:35.274 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:34.368+0000 7fe434c00700 1 -- 172.21.15.119:0/1538633192 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe428003cc0 con 0x7fe43810ec80 2024-03-28T00:15:35.274 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:34.368+0000 7fe434c00700 1 -- 172.21.15.119:0/1538633192 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe428024840 con 0x7fe43810ec80 2024-03-28T00:15:35.274 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:34.368+0000 7fe43e400700 1 -- 172.21.15.119:0/1538633192 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4381a2850 con 0x7fe43810ec80 2024-03-28T00:15:35.274 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:34.368+0000 7fe43e400700 1 -- 172.21.15.119:0/1538633192 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4381a5540 con 0x7fe43810ec80 2024-03-28T00:15:35.274 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:34.369+0000 7fe434c00700 1 -- 172.21.15.119:0/1538633192 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 8) v1 ==== 46106+0+0 (secure 0 0 0) 0x7fe428022020 con 0x7fe43810ec80 2024-03-28T00:15:35.274 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:34.369+0000 7fe43e400700 1 -- 172.21.15.119:0/1538633192 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe438069260 con 0x7fe43810ec80 2024-03-28T00:15:35.274 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:34.369+0000 7fe434c00700 1 --2- 172.21.15.119:0/1538633192 >> [v2:172.21.15.119:6800/2575246519,v1:172.21.15.119:6801/2575246519] conn(0x7fe41c038f20 0x7fe41c03b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:15:35.275 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:34.369+0000 7fe434c00700 1 -- 172.21.15.119:0/1538633192 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fe428011e00 con 0x7fe43810ec80 2024-03-28T00:15:35.275 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:34.369+0000 7fe42fe00700 1 --2- 172.21.15.119:0/1538633192 >> [v2:172.21.15.119:6800/2575246519,v1:172.21.15.119:6801/2575246519] conn(0x7fe41c038f20 0x7fe41c03b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:15:35.275 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:34.370+0000 7fe42fe00700 1 --2- 172.21.15.119:0/1538633192 >> [v2:172.21.15.119:6800/2575246519,v1:172.21.15.119:6801/2575246519] conn(0x7fe41c038f20 0x7fe41c03b3e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe420006fd0 tx=0x7fe420006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:15:35.275 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:34.375+0000 7fe434c00700 1 -- 172.21.15.119:0/1538633192 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fe42804ca20 con 0x7fe43810ec80 2024-03-28T00:15:35.275 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:34.598+0000 7fe43e400700 1 -- 172.21.15.119:0/1538633192 --> [v2:172.21.15.119:6800/2575246519,v1:172.21.15.119:6801/2575246519] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7fe43807e2e0 con 0x7fe41c038f20 2024-03-28T00:15:35.275 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:34.599+0000 7fe434c00700 1 -- 172.21.15.119:0/1538633192 <== mgr.14120 v2:172.21.15.119:6800/2575246519 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7fe43807e2e0 con 0x7fe41c038f20 2024-03-28T00:15:35.275 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:34.602+0000 7fe43e400700 1 -- 172.21.15.119:0/1538633192 >> [v2:172.21.15.119:6800/2575246519,v1:172.21.15.119:6801/2575246519] conn(0x7fe41c038f20 msgr2=0x7fe41c03b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:15:35.275 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:34.602+0000 7fe43e400700 1 --2- 172.21.15.119:0/1538633192 >> [v2:172.21.15.119:6800/2575246519,v1:172.21.15.119:6801/2575246519] conn(0x7fe41c038f20 0x7fe41c03b3e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe420006fd0 tx=0x7fe420006e40 comp rx=0 tx=0).stop 2024-03-28T00:15:35.275 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:34.602+0000 7fe43e400700 1 -- 172.21.15.119:0/1538633192 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe43810ec80 msgr2=0x7fe4381a2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:15:35.275 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:34.602+0000 7fe43e400700 1 --2- 172.21.15.119:0/1538633192 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe43810ec80 0x7fe4381a2080 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe428011b60 tx=0x7fe428011c40 comp rx=0 tx=0).stop 2024-03-28T00:15:35.275 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:34.602+0000 7fe43e400700 1 -- 172.21.15.119:0/1538633192 shutdown_connections 2024-03-28T00:15:35.275 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:34.602+0000 7fe43e400700 1 --2- 172.21.15.119:0/1538633192 >> [v2:172.21.15.119:6800/2575246519,v1:172.21.15.119:6801/2575246519] conn(0x7fe41c038f20 0x7fe41c03b3e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:15:35.275 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:34.602+0000 7fe43e400700 1 --2- 172.21.15.119:0/1538633192 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe43810ec80 0x7fe4381a2080 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:15:35.275 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:34.602+0000 7fe43e400700 1 -- 172.21.15.119:0/1538633192 >> 172.21.15.119:0/1538633192 conn(0x7fe43807d100 msgr2=0x7fe43807dbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:15:35.276 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:34.602+0000 7fe43e400700 1 -- 172.21.15.119:0/1538633192 shutdown_connections 2024-03-28T00:15:35.276 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:34.602+0000 7fe43e400700 1 -- 172.21.15.119:0/1538633192 wait complete. 2024-03-28T00:15:35.276 INFO:teuthology.orchestra.run.smithi119.stdout:Generating ssh key... 2024-03-28T00:15:35.803 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:35 smithi119 ceph-mon[31518]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T00:15:36.369 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 00:15:36 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-x[31752]: Generating public/private rsa key pair. 2024-03-28T00:15:36.369 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 00:15:36 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-x[31752]: Your identification has been saved in /tmp/tmpdwgk9j3d/key. 2024-03-28T00:15:36.369 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 00:15:36 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-x[31752]: Your public key has been saved in /tmp/tmpdwgk9j3d/key.pub. 2024-03-28T00:15:36.369 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 00:15:36 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-x[31752]: The key fingerprint is: 2024-03-28T00:15:36.369 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 00:15:36 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-x[31752]: SHA256:3iDc1sqclmKrhxnNMX/y/2oMmkUrVe4kmq1OoqP5X4Y ceph-10f73b92-ec98-11ee-95d0-87774f69a715 2024-03-28T00:15:36.370 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 00:15:36 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-x[31752]: The key's randomart image is: 2024-03-28T00:15:36.370 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 00:15:36 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-x[31752]: +---[RSA 3072]----+ 2024-03-28T00:15:36.370 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 00:15:36 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-x[31752]: | | 2024-03-28T00:15:36.370 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 00:15:36 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-x[31752]: | . | 2024-03-28T00:15:36.370 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 00:15:36 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-x[31752]: | o | 2024-03-28T00:15:36.370 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 00:15:36 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-x[31752]: | .o. .+ o | 2024-03-28T00:15:36.370 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 00:15:36 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-x[31752]: | oo+S*.= | 2024-03-28T00:15:36.370 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 00:15:36 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-x[31752]: | . oB*** . | 2024-03-28T00:15:36.370 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 00:15:36 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-x[31752]: | +E %X.o | 2024-03-28T00:15:36.370 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 00:15:36 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-x[31752]: | .+o.O+ . o | 2024-03-28T00:15:36.370 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 00:15:36 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-x[31752]: | oo+=o.. ooo. | 2024-03-28T00:15:36.370 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 00:15:36 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-x[31752]: +----[SHA256]-----+ 2024-03-28T00:15:37.033 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:35.859+0000 7ff8fde00700 1 Processor -- start 2024-03-28T00:15:37.034 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:35.859+0000 7ff8fde00700 1 -- start start 2024-03-28T00:15:37.034 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:35.859+0000 7ff8fde00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff8f8109d10 0x7ff8f810a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:15:37.034 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:35.859+0000 7ff8fde00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8f810a700 con 0x7ff8f8109d10 2024-03-28T00:15:37.034 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:35.860+0000 7ff8f6a00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff8f8109d10 0x7ff8f810a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:15:37.034 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:35.860+0000 7ff8f6a00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff8f8109d10 0x7ff8f810a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:47138/0 (socket says 172.21.15.119:47138) 2024-03-28T00:15:37.034 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:35.860+0000 7ff8f6a00700 1 -- 172.21.15.119:0/306548743 learned_addr learned my addr 172.21.15.119:0/306548743 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:15:37.034 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:35.860+0000 7ff8f6a00700 1 -- 172.21.15.119:0/306548743 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8f810af10 con 0x7ff8f8109d10 2024-03-28T00:15:37.034 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:35.861+0000 7ff8f6a00700 1 --2- 172.21.15.119:0/306548743 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff8f8109d10 0x7ff8f810a130 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ff8e80098a0 tx=0x7ff8e8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47f3fc98a6ca5dfc server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:15:37.035 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:35.861+0000 7ff8f6000700 1 -- 172.21.15.119:0/306548743 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff8e8004160 con 0x7ff8f8109d10 2024-03-28T00:15:37.035 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:35.861+0000 7ff8f6000700 1 -- 172.21.15.119:0/306548743 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff8e8004300 con 0x7ff8f8109d10 2024-03-28T00:15:37.035 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:35.861+0000 7ff8f6000700 1 -- 172.21.15.119:0/306548743 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff8e8011b50 con 0x7ff8f8109d10 2024-03-28T00:15:37.035 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:35.861+0000 7ff8fde00700 1 -- 172.21.15.119:0/306548743 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff8f8109d10 msgr2=0x7ff8f810a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:15:37.035 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:35.861+0000 7ff8fde00700 1 --2- 172.21.15.119:0/306548743 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff8f8109d10 0x7ff8f810a130 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ff8e80098a0 tx=0x7ff8e8009bb0 comp rx=0 tx=0).stop 2024-03-28T00:15:37.035 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:35.862+0000 7ff8fde00700 1 -- 172.21.15.119:0/306548743 shutdown_connections 2024-03-28T00:15:37.035 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:35.862+0000 7ff8fde00700 1 --2- 172.21.15.119:0/306548743 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff8f8109d10 0x7ff8f810a130 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:15:37.035 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:35.862+0000 7ff8fde00700 1 -- 172.21.15.119:0/306548743 >> 172.21.15.119:0/306548743 conn(0x7ff8f807d100 msgr2=0x7ff8f807d510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:15:37.035 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:35.862+0000 7ff8fde00700 1 -- 172.21.15.119:0/306548743 shutdown_connections 2024-03-28T00:15:37.035 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:35.862+0000 7ff8fde00700 1 -- 172.21.15.119:0/306548743 wait complete. 2024-03-28T00:15:37.035 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:35.863+0000 7ff8fde00700 1 Processor -- start 2024-03-28T00:15:37.035 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:35.863+0000 7ff8fde00700 1 -- start start 2024-03-28T00:15:37.035 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:35.863+0000 7ff8fde00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff8f8109d10 0x7ff8f819dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:15:37.035 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:35.863+0000 7ff8fde00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8f810e2c0 con 0x7ff8f8109d10 2024-03-28T00:15:37.036 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:35.864+0000 7ff8f6a00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff8f8109d10 0x7ff8f819dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:15:37.036 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:35.864+0000 7ff8f6a00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff8f8109d10 0x7ff8f819dce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:47152/0 (socket says 172.21.15.119:47152) 2024-03-28T00:15:37.036 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:35.864+0000 7ff8f6a00700 1 -- 172.21.15.119:0/1034102978 learned_addr learned my addr 172.21.15.119:0/1034102978 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:15:37.036 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:35.864+0000 7ff8f6a00700 1 -- 172.21.15.119:0/1034102978 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8f819e220 con 0x7ff8f8109d10 2024-03-28T00:15:37.036 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:35.864+0000 7ff8f6a00700 1 --2- 172.21.15.119:0/1034102978 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff8f8109d10 0x7ff8f819dce0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff8e8011670 tx=0x7ff8e8011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:15:37.036 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:35.864+0000 7ff8fca00700 1 -- 172.21.15.119:0/1034102978 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff8e80039c0 con 0x7ff8f8109d10 2024-03-28T00:15:37.036 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:35.864+0000 7ff8fca00700 1 -- 172.21.15.119:0/1034102978 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff8e8011e50 con 0x7ff8f8109d10 2024-03-28T00:15:37.036 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:35.864+0000 7ff8fca00700 1 -- 172.21.15.119:0/1034102978 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff8e801a5c0 con 0x7ff8f8109d10 2024-03-28T00:15:37.036 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:35.864+0000 7ff8fde00700 1 -- 172.21.15.119:0/1034102978 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8f819e4b0 con 0x7ff8f8109d10 2024-03-28T00:15:37.036 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:35.865+0000 7ff8fde00700 1 -- 172.21.15.119:0/1034102978 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8f81a1190 con 0x7ff8f8109d10 2024-03-28T00:15:37.036 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:35.866+0000 7ff8fca00700 1 -- 172.21.15.119:0/1034102978 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 8) v1 ==== 46106+0+0 (secure 0 0 0) 0x7ff8e8024430 con 0x7ff8f8109d10 2024-03-28T00:15:37.036 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:35.866+0000 7ff8ed600700 1 -- 172.21.15.119:0/1034102978 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8f8069260 con 0x7ff8f8109d10 2024-03-28T00:15:37.036 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:35.866+0000 7ff8fca00700 1 --2- 172.21.15.119:0/1034102978 >> [v2:172.21.15.119:6800/2575246519,v1:172.21.15.119:6801/2575246519] conn(0x7ff8dc038f20 0x7ff8dc03b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:15:37.036 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:35.866+0000 7ff8fca00700 1 -- 172.21.15.119:0/1034102978 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7ff8e804d340 con 0x7ff8f8109d10 2024-03-28T00:15:37.037 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:35.867+0000 7ff8ee000700 1 --2- 172.21.15.119:0/1034102978 >> [v2:172.21.15.119:6800/2575246519,v1:172.21.15.119:6801/2575246519] conn(0x7ff8dc038f20 0x7ff8dc03b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:15:37.037 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:35.867+0000 7ff8ee000700 1 --2- 172.21.15.119:0/1034102978 >> [v2:172.21.15.119:6800/2575246519,v1:172.21.15.119:6801/2575246519] conn(0x7ff8dc038f20 0x7ff8dc03b3e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ff8e000ad30 tx=0x7ff8e00093f0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:15:37.037 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:35.872+0000 7ff8fca00700 1 -- 172.21.15.119:0/1034102978 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff8e804cc90 con 0x7ff8f8109d10 2024-03-28T00:15:37.037 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:36.096+0000 7ff8ed600700 1 -- 172.21.15.119:0/1034102978 --> [v2:172.21.15.119:6800/2575246519,v1:172.21.15.119:6801/2575246519] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7ff8f8062f30 con 0x7ff8dc038f20 2024-03-28T00:15:37.037 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:36.314+0000 7ff8fca00700 1 -- 172.21.15.119:0/1034102978 <== mgr.14120 v2:172.21.15.119:6800/2575246519 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7ff8f8062f30 con 0x7ff8dc038f20 2024-03-28T00:15:37.037 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:36.317+0000 7ff8ed600700 1 -- 172.21.15.119:0/1034102978 >> [v2:172.21.15.119:6800/2575246519,v1:172.21.15.119:6801/2575246519] conn(0x7ff8dc038f20 msgr2=0x7ff8dc03b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:15:37.037 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:36.317+0000 7ff8ed600700 1 --2- 172.21.15.119:0/1034102978 >> [v2:172.21.15.119:6800/2575246519,v1:172.21.15.119:6801/2575246519] conn(0x7ff8dc038f20 0x7ff8dc03b3e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ff8e000ad30 tx=0x7ff8e00093f0 comp rx=0 tx=0).stop 2024-03-28T00:15:37.037 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:36.317+0000 7ff8ed600700 1 -- 172.21.15.119:0/1034102978 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff8f8109d10 msgr2=0x7ff8f819dce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:15:37.037 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:36.317+0000 7ff8ed600700 1 --2- 172.21.15.119:0/1034102978 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff8f8109d10 0x7ff8f819dce0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff8e8011670 tx=0x7ff8e8011b60 comp rx=0 tx=0).stop 2024-03-28T00:15:37.037 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:36.317+0000 7ff8ed600700 1 -- 172.21.15.119:0/1034102978 shutdown_connections 2024-03-28T00:15:37.037 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:36.317+0000 7ff8ed600700 1 --2- 172.21.15.119:0/1034102978 >> [v2:172.21.15.119:6800/2575246519,v1:172.21.15.119:6801/2575246519] conn(0x7ff8dc038f20 0x7ff8dc03b3e0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:15:37.037 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:36.317+0000 7ff8ed600700 1 --2- 172.21.15.119:0/1034102978 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff8f8109d10 0x7ff8f819dce0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:15:37.037 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:36.317+0000 7ff8ed600700 1 -- 172.21.15.119:0/1034102978 >> 172.21.15.119:0/1034102978 conn(0x7ff8f807d100 msgr2=0x7ff8f8107480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:15:37.037 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:36.317+0000 7ff8ed600700 1 -- 172.21.15.119:0/1034102978 shutdown_connections 2024-03-28T00:15:37.037 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:36.318+0000 7ff8ed600700 1 -- 172.21.15.119:0/1034102978 wait complete. 2024-03-28T00:15:37.559 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:37 smithi119 ceph-mon[31518]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T00:15:37.560 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:37 smithi119 ceph-mon[31518]: Generating ssh key... 2024-03-28T00:15:37.560 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:37 smithi119 ceph-mon[31518]: from='mgr.14120 172.21.15.119:0/4048015587' entity='mgr.x' 2024-03-28T00:15:37.560 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:37 smithi119 ceph-mon[31518]: from='mgr.14120 172.21.15.119:0/4048015587' entity='mgr.x' 2024-03-28T00:15:38.638 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:38 smithi119 ceph-mon[31518]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T00:15:38.651 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC9AehF0WSVUyKaaVqCSIvTvWY1NZFB5dfJ7NBoWfLQyoi8RP+XNkWCMQRg6gBCuc4+MPehgv8yFJsA+La+EfD+DLtR6MemXndxbUEtHRttqd04w6/Ng41HYZoGP6EYvLSMAl6ykfI5Tq2/r4Uz3tgDAqnb7LcKJz+9Xg94y73VJ8M9B+E6BSVgNQVdz4hvxYcv1MRfl6g2lzmcF2wPhbqP+0SA0uVihKODAiphdQY8po2ugSw0zHmsTH0YWPIFPAelQ2eVT4du8sghGllcXNH66DEkjhMb7TCFG2gLnP+v9/DmGQ9A95ODRYRXxy7H1qvbmyDXJ8VMhW2SFDYgia5DOjPtcfdPRc4Ckhl4YSDgMDwlV0dBHeXoTgE17WvtSsVc9pHOgKyXpeuN42Rw/Zk9C1NW0eKWdOAkU9CD/tIrOHKQjgOEyMSDkFk54AD6yh6gmgtnXG7ZpUgUrHCFwdt3lLxUHjx0zAsTXXrWHC1QLmG7sM4P5HTDbxNXXzOmA88= ceph-10f73b92-ec98-11ee-95d0-87774f69a715 2024-03-28T00:15:38.652 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:37.553+0000 7f93e6e00700 1 Processor -- start 2024-03-28T00:15:38.652 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:37.553+0000 7f93e6e00700 1 -- start start 2024-03-28T00:15:38.652 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:37.553+0000 7f93e6e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93e0108600 0x7f93e010a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:15:38.652 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:37.553+0000 7f93e6e00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93e007bda0 con 0x7f93e0108600 2024-03-28T00:15:38.652 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:37.554+0000 7f93dfe00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93e0108600 0x7f93e010a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:15:38.652 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:37.554+0000 7f93dfe00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93e0108600 0x7f93e010a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:47166/0 (socket says 172.21.15.119:47166) 2024-03-28T00:15:38.652 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:37.554+0000 7f93dfe00700 1 -- 172.21.15.119:0/3979022750 learned_addr learned my addr 172.21.15.119:0/3979022750 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:15:38.652 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:37.554+0000 7f93dfe00700 1 -- 172.21.15.119:0/3979022750 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93e010af30 con 0x7f93e0108600 2024-03-28T00:15:38.652 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:37.555+0000 7f93dfe00700 1 --2- 172.21.15.119:0/3979022750 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93e0108600 0x7f93e010a9f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f93c80098a0 tx=0x7f93c8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d04c931365790c3 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:15:38.652 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:37.555+0000 7f93dea00700 1 -- 172.21.15.119:0/3979022750 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f93c8004160 con 0x7f93e0108600 2024-03-28T00:15:38.652 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:37.555+0000 7f93dea00700 1 -- 172.21.15.119:0/3979022750 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f93c8004300 con 0x7f93e0108600 2024-03-28T00:15:38.652 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:37.555+0000 7f93dea00700 1 -- 172.21.15.119:0/3979022750 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f93c8011b50 con 0x7f93e0108600 2024-03-28T00:15:38.653 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:37.556+0000 7f93e6e00700 1 -- 172.21.15.119:0/3979022750 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93e0108600 msgr2=0x7f93e010a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:15:38.653 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:37.556+0000 7f93e6e00700 1 --2- 172.21.15.119:0/3979022750 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93e0108600 0x7f93e010a9f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f93c80098a0 tx=0x7f93c8009bb0 comp rx=0 tx=0).stop 2024-03-28T00:15:38.653 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:37.556+0000 7f93e6e00700 1 -- 172.21.15.119:0/3979022750 shutdown_connections 2024-03-28T00:15:38.653 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:37.556+0000 7f93e6e00700 1 --2- 172.21.15.119:0/3979022750 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93e0108600 0x7f93e010a9f0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:15:38.653 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:37.556+0000 7f93e6e00700 1 -- 172.21.15.119:0/3979022750 >> 172.21.15.119:0/3979022750 conn(0x7f93e0101f70 msgr2=0x7f93e01043b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:15:38.653 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:37.556+0000 7f93e6e00700 1 -- 172.21.15.119:0/3979022750 shutdown_connections 2024-03-28T00:15:38.653 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:37.556+0000 7f93e6e00700 1 -- 172.21.15.119:0/3979022750 wait complete. 2024-03-28T00:15:38.653 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:37.557+0000 7f93e6e00700 1 Processor -- start 2024-03-28T00:15:38.653 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:37.557+0000 7f93e6e00700 1 -- start start 2024-03-28T00:15:38.653 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:37.557+0000 7f93e6e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93e0108600 0x7f93e0107e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:15:38.653 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:37.557+0000 7f93e6e00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93e010bea0 con 0x7f93e0108600 2024-03-28T00:15:38.653 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:37.557+0000 7f93dfe00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93e0108600 0x7f93e0107e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:15:38.653 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:37.558+0000 7f93dfe00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93e0108600 0x7f93e0107e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:47180/0 (socket says 172.21.15.119:47180) 2024-03-28T00:15:38.653 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:37.558+0000 7f93dfe00700 1 -- 172.21.15.119:0/3984730122 learned_addr learned my addr 172.21.15.119:0/3984730122 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:15:38.653 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:37.558+0000 7f93dfe00700 1 -- 172.21.15.119:0/3984730122 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93e01083c0 con 0x7f93e0108600 2024-03-28T00:15:38.654 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:37.558+0000 7f93dfe00700 1 --2- 172.21.15.119:0/3984730122 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93e0108600 0x7f93e0107e80 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f93c8011670 tx=0x7f93c8011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:15:38.654 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:37.558+0000 7f93dcc00700 1 -- 172.21.15.119:0/3984730122 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f93c80039c0 con 0x7f93e0108600 2024-03-28T00:15:38.654 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:37.558+0000 7f93dcc00700 1 -- 172.21.15.119:0/3984730122 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f93c8011e50 con 0x7f93e0108600 2024-03-28T00:15:38.654 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:37.558+0000 7f93dcc00700 1 -- 172.21.15.119:0/3984730122 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f93c801a5c0 con 0x7f93e0108600 2024-03-28T00:15:38.654 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:37.558+0000 7f93e6e00700 1 -- 172.21.15.119:0/3984730122 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93e01065b0 con 0x7f93e0108600 2024-03-28T00:15:38.654 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:37.559+0000 7f93e6e00700 1 -- 172.21.15.119:0/3984730122 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93e0106af0 con 0x7f93e0108600 2024-03-28T00:15:38.654 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:37.560+0000 7f93dcc00700 1 -- 172.21.15.119:0/3984730122 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 8) v1 ==== 46106+0+0 (secure 0 0 0) 0x7f93c8024430 con 0x7f93e0108600 2024-03-28T00:15:38.654 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:37.560+0000 7f93e6e00700 1 -- 172.21.15.119:0/3984730122 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93e0069260 con 0x7f93e0108600 2024-03-28T00:15:38.654 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:37.560+0000 7f93dcc00700 1 --2- 172.21.15.119:0/3984730122 >> [v2:172.21.15.119:6800/2575246519,v1:172.21.15.119:6801/2575246519] conn(0x7f93c4038ed0 0x7f93c403b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:15:38.654 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:37.560+0000 7f93dcc00700 1 -- 172.21.15.119:0/3984730122 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f93c804d2d0 con 0x7f93e0108600 2024-03-28T00:15:38.654 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:37.560+0000 7f93df400700 1 --2- 172.21.15.119:0/3984730122 >> [v2:172.21.15.119:6800/2575246519,v1:172.21.15.119:6801/2575246519] conn(0x7f93c4038ed0 0x7f93c403b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:15:38.654 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:37.561+0000 7f93df400700 1 --2- 172.21.15.119:0/3984730122 >> [v2:172.21.15.119:6800/2575246519,v1:172.21.15.119:6801/2575246519] conn(0x7f93c4038ed0 0x7f93c403b390 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f93d0006fd0 tx=0x7f93d0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:15:38.654 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:37.566+0000 7f93dcc00700 1 -- 172.21.15.119:0/3984730122 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f93c802d910 con 0x7f93e0108600 2024-03-28T00:15:38.655 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:37.779+0000 7f93e6e00700 1 -- 172.21.15.119:0/3984730122 --> [v2:172.21.15.119:6800/2575246519,v1:172.21.15.119:6801/2575246519] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f93e01031e0 con 0x7f93c4038ed0 2024-03-28T00:15:38.655 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:37.780+0000 7f93dcc00700 1 -- 172.21.15.119:0/3984730122 <== mgr.14120 v2:172.21.15.119:6800/2575246519 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7f93e01031e0 con 0x7f93c4038ed0 2024-03-28T00:15:38.655 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:37.782+0000 7f93e6e00700 1 -- 172.21.15.119:0/3984730122 >> [v2:172.21.15.119:6800/2575246519,v1:172.21.15.119:6801/2575246519] conn(0x7f93c4038ed0 msgr2=0x7f93c403b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:15:38.655 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:37.782+0000 7f93e6e00700 1 --2- 172.21.15.119:0/3984730122 >> [v2:172.21.15.119:6800/2575246519,v1:172.21.15.119:6801/2575246519] conn(0x7f93c4038ed0 0x7f93c403b390 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f93d0006fd0 tx=0x7f93d0006e40 comp rx=0 tx=0).stop 2024-03-28T00:15:38.655 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:37.782+0000 7f93e6e00700 1 -- 172.21.15.119:0/3984730122 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93e0108600 msgr2=0x7f93e0107e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:15:38.655 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:37.782+0000 7f93e6e00700 1 --2- 172.21.15.119:0/3984730122 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93e0108600 0x7f93e0107e80 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f93c8011670 tx=0x7f93c8011b60 comp rx=0 tx=0).stop 2024-03-28T00:15:38.655 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:37.782+0000 7f93e6e00700 1 -- 172.21.15.119:0/3984730122 shutdown_connections 2024-03-28T00:15:38.655 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:37.782+0000 7f93e6e00700 1 --2- 172.21.15.119:0/3984730122 >> [v2:172.21.15.119:6800/2575246519,v1:172.21.15.119:6801/2575246519] conn(0x7f93c4038ed0 0x7f93c403b390 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:15:38.655 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:37.782+0000 7f93e6e00700 1 --2- 172.21.15.119:0/3984730122 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93e0108600 0x7f93e0107e80 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:15:38.655 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:37.782+0000 7f93e6e00700 1 -- 172.21.15.119:0/3984730122 >> 172.21.15.119:0/3984730122 conn(0x7f93e0101f70 msgr2=0x7f93e0102ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:15:38.655 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:37.783+0000 7f93e6e00700 1 -- 172.21.15.119:0/3984730122 shutdown_connections 2024-03-28T00:15:38.655 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:37.783+0000 7f93e6e00700 1 -- 172.21.15.119:0/3984730122 wait complete. 2024-03-28T00:15:38.655 INFO:teuthology.orchestra.run.smithi119.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-03-28T00:15:38.655 INFO:teuthology.orchestra.run.smithi119.stdout:Adding key to root@localhost authorized_keys... 2024-03-28T00:15:38.656 INFO:teuthology.orchestra.run.smithi119.stdout:Adding host smithi119... 2024-03-28T00:15:40.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:39 smithi119 ceph-mon[31518]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi119", "addr": "172.21.15.119", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T00:15:41.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:40 smithi119 ceph-mon[31518]: Deploying cephadm binary to smithi119 2024-03-28T00:15:46.427 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout Added host 'smithi119' with addr '172.21.15.119' 2024-03-28T00:15:46.427 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:39.241+0000 7f8f14a00700 1 Processor -- start 2024-03-28T00:15:46.427 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:39.241+0000 7f8f14a00700 1 -- start start 2024-03-28T00:15:46.427 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:39.242+0000 7f8f14a00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8f101085a0 0x7f8f1010a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:15:46.427 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:39.242+0000 7f8f14a00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f1007be60 con 0x7f8f101085a0 2024-03-28T00:15:46.427 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:39.242+0000 7f8f0de00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8f101085a0 0x7f8f1010a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:15:46.428 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:39.242+0000 7f8f0de00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8f101085a0 0x7f8f1010a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:47188/0 (socket says 172.21.15.119:47188) 2024-03-28T00:15:46.428 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:39.242+0000 7f8f0de00700 1 -- 172.21.15.119:0/1304273057 learned_addr learned my addr 172.21.15.119:0/1304273057 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:15:46.428 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:39.243+0000 7f8f0de00700 1 -- 172.21.15.119:0/1304273057 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f1010aed0 con 0x7f8f101085a0 2024-03-28T00:15:46.428 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:39.243+0000 7f8f0de00700 1 --2- 172.21.15.119:0/1304273057 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8f101085a0 0x7f8f1010a990 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f8f000098a0 tx=0x7f8f00009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee117cb643e2f4f1 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:15:46.428 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:39.243+0000 7f8f0ca00700 1 -- 172.21.15.119:0/1304273057 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8f00004160 con 0x7f8f101085a0 2024-03-28T00:15:46.428 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:39.244+0000 7f8f0ca00700 1 -- 172.21.15.119:0/1304273057 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8f00004300 con 0x7f8f101085a0 2024-03-28T00:15:46.428 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:39.244+0000 7f8f0ca00700 1 -- 172.21.15.119:0/1304273057 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8f00011b50 con 0x7f8f101085a0 2024-03-28T00:15:46.428 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:39.244+0000 7f8f14a00700 1 -- 172.21.15.119:0/1304273057 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8f101085a0 msgr2=0x7f8f1010a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:15:46.428 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:39.244+0000 7f8f14a00700 1 --2- 172.21.15.119:0/1304273057 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8f101085a0 0x7f8f1010a990 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f8f000098a0 tx=0x7f8f00009bb0 comp rx=0 tx=0).stop 2024-03-28T00:15:46.429 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:39.244+0000 7f8f14a00700 1 -- 172.21.15.119:0/1304273057 shutdown_connections 2024-03-28T00:15:46.429 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:39.244+0000 7f8f14a00700 1 --2- 172.21.15.119:0/1304273057 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8f101085a0 0x7f8f1010a990 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:15:46.429 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:39.244+0000 7f8f14a00700 1 -- 172.21.15.119:0/1304273057 >> 172.21.15.119:0/1304273057 conn(0x7f8f10101ff0 msgr2=0x7f8f10104410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:15:46.429 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:39.244+0000 7f8f14a00700 1 -- 172.21.15.119:0/1304273057 shutdown_connections 2024-03-28T00:15:46.429 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:39.245+0000 7f8f14a00700 1 -- 172.21.15.119:0/1304273057 wait complete. 2024-03-28T00:15:46.429 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:39.246+0000 7f8f14a00700 1 Processor -- start 2024-03-28T00:15:46.429 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:39.246+0000 7f8f14a00700 1 -- start start 2024-03-28T00:15:46.429 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:39.246+0000 7f8f14a00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8f101085a0 0x7f8f1019bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:15:46.429 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:39.246+0000 7f8f14a00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f1010bf00 con 0x7f8f101085a0 2024-03-28T00:15:46.430 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:39.246+0000 7f8f0de00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8f101085a0 0x7f8f1019bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:15:46.430 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:39.246+0000 7f8f0de00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8f101085a0 0x7f8f1019bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:47194/0 (socket says 172.21.15.119:47194) 2024-03-28T00:15:46.430 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:39.246+0000 7f8f0de00700 1 -- 172.21.15.119:0/1522953775 learned_addr learned my addr 172.21.15.119:0/1522953775 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:15:46.430 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:39.247+0000 7f8f0de00700 1 -- 172.21.15.119:0/1522953775 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f1019c0c0 con 0x7f8f101085a0 2024-03-28T00:15:46.430 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:39.247+0000 7f8f0de00700 1 --2- 172.21.15.119:0/1522953775 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8f101085a0 0x7f8f1019bb80 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f8f00011670 tx=0x7f8f00011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:15:46.430 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:39.247+0000 7f8f06a00700 1 -- 172.21.15.119:0/1522953775 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8f000039c0 con 0x7f8f101085a0 2024-03-28T00:15:46.430 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:39.247+0000 7f8f06a00700 1 -- 172.21.15.119:0/1522953775 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8f00011e50 con 0x7f8f101085a0 2024-03-28T00:15:46.430 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:39.247+0000 7f8f06a00700 1 -- 172.21.15.119:0/1522953775 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8f0001a5c0 con 0x7f8f101085a0 2024-03-28T00:15:46.431 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:39.247+0000 7f8f14a00700 1 -- 172.21.15.119:0/1522953775 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f1019c350 con 0x7f8f101085a0 2024-03-28T00:15:46.431 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:39.248+0000 7f8f14a00700 1 -- 172.21.15.119:0/1522953775 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f1019ce70 con 0x7f8f101085a0 2024-03-28T00:15:46.431 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:39.249+0000 7f8f14a00700 1 -- 172.21.15.119:0/1522953775 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f100637c0 con 0x7f8f101085a0 2024-03-28T00:15:46.431 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:39.249+0000 7f8f06a00700 1 -- 172.21.15.119:0/1522953775 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 8) v1 ==== 46106+0+0 (secure 0 0 0) 0x7f8f00024430 con 0x7f8f101085a0 2024-03-28T00:15:46.431 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:39.249+0000 7f8f06a00700 1 --2- 172.21.15.119:0/1522953775 >> [v2:172.21.15.119:6800/2575246519,v1:172.21.15.119:6801/2575246519] conn(0x7f8ef4038f20 0x7f8ef403b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:15:46.431 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:39.249+0000 7f8f06a00700 1 -- 172.21.15.119:0/1522953775 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f8f0004d2d0 con 0x7f8f101085a0 2024-03-28T00:15:46.431 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:39.250+0000 7f8f0d400700 1 --2- 172.21.15.119:0/1522953775 >> [v2:172.21.15.119:6800/2575246519,v1:172.21.15.119:6801/2575246519] conn(0x7f8ef4038f20 0x7f8ef403b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:15:46.432 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:39.250+0000 7f8f0d400700 1 --2- 172.21.15.119:0/1522953775 >> [v2:172.21.15.119:6800/2575246519,v1:172.21.15.119:6801/2575246519] conn(0x7f8ef4038f20 0x7f8ef403b3e0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f8efc006fd0 tx=0x7f8efc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:15:46.432 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:39.255+0000 7f8f06a00700 1 -- 172.21.15.119:0/1522953775 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f8f0004cc20 con 0x7f8f101085a0 2024-03-28T00:15:46.432 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:39.478+0000 7f8f14a00700 1 -- 172.21.15.119:0/1522953775 --> [v2:172.21.15.119:6800/2575246519,v1:172.21.15.119:6801/2575246519] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi119", "addr": "172.21.15.119", "target": ["mon-mgr", ""]}) v1 -- 0x7f8f101031c0 con 0x7f8ef4038f20 2024-03-28T00:15:46.432 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:45.689+0000 7f8f06a00700 1 -- 172.21.15.119:0/1522953775 <== mgr.14120 v2:172.21.15.119:6800/2575246519 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f8f101031c0 con 0x7f8ef4038f20 2024-03-28T00:15:46.432 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:45.693+0000 7f8f14a00700 1 -- 172.21.15.119:0/1522953775 >> [v2:172.21.15.119:6800/2575246519,v1:172.21.15.119:6801/2575246519] conn(0x7f8ef4038f20 msgr2=0x7f8ef403b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:15:46.432 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:45.693+0000 7f8f14a00700 1 --2- 172.21.15.119:0/1522953775 >> [v2:172.21.15.119:6800/2575246519,v1:172.21.15.119:6801/2575246519] conn(0x7f8ef4038f20 0x7f8ef403b3e0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f8efc006fd0 tx=0x7f8efc006e40 comp rx=0 tx=0).stop 2024-03-28T00:15:46.432 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:45.693+0000 7f8f14a00700 1 -- 172.21.15.119:0/1522953775 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8f101085a0 msgr2=0x7f8f1019bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:15:46.433 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:45.693+0000 7f8f14a00700 1 --2- 172.21.15.119:0/1522953775 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8f101085a0 0x7f8f1019bb80 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f8f00011670 tx=0x7f8f00011b60 comp rx=0 tx=0).stop 2024-03-28T00:15:46.433 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:45.693+0000 7f8f14a00700 1 -- 172.21.15.119:0/1522953775 shutdown_connections 2024-03-28T00:15:46.433 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:45.693+0000 7f8f14a00700 1 --2- 172.21.15.119:0/1522953775 >> [v2:172.21.15.119:6800/2575246519,v1:172.21.15.119:6801/2575246519] conn(0x7f8ef4038f20 0x7f8ef403b3e0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:15:46.433 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:45.693+0000 7f8f14a00700 1 --2- 172.21.15.119:0/1522953775 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8f101085a0 0x7f8f1019bb80 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:15:46.433 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:45.693+0000 7f8f14a00700 1 -- 172.21.15.119:0/1522953775 >> 172.21.15.119:0/1522953775 conn(0x7f8f10101ff0 msgr2=0x7f8f10102ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:15:46.433 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:45.693+0000 7f8f14a00700 1 -- 172.21.15.119:0/1522953775 shutdown_connections 2024-03-28T00:15:46.433 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:45.693+0000 7f8f14a00700 1 -- 172.21.15.119:0/1522953775 wait complete. 2024-03-28T00:15:46.433 INFO:teuthology.orchestra.run.smithi119.stdout:Deploying unmanaged mon service... 2024-03-28T00:15:46.946 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:46 smithi119 ceph-mon[31518]: from='mgr.14120 172.21.15.119:0/4048015587' entity='mgr.x' 2024-03-28T00:15:46.946 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:46 smithi119 ceph-mon[31518]: Added host smithi119 2024-03-28T00:15:46.946 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:46 smithi119 ceph-mon[31518]: from='mgr.14120 172.21.15.119:0/4048015587' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:15:48.264 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-03-28T00:15:48.264 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:47.096+0000 7fa0c5a00700 1 Processor -- start 2024-03-28T00:15:48.264 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:47.097+0000 7fa0c5a00700 1 -- start start 2024-03-28T00:15:48.264 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:47.097+0000 7fa0c5a00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa0c007de90 0x7fa0c007c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:15:48.265 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:47.097+0000 7fa0c5a00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c007c8c0 con 0x7fa0c007de90 2024-03-28T00:15:48.265 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:47.097+0000 7fa0bea00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa0c007de90 0x7fa0c007c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:15:48.265 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:47.098+0000 7fa0bea00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa0c007de90 0x7fa0c007c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35588/0 (socket says 172.21.15.119:35588) 2024-03-28T00:15:48.265 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:47.098+0000 7fa0bea00700 1 -- 172.21.15.119:0/2941032921 learned_addr learned my addr 172.21.15.119:0/2941032921 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:15:48.265 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:47.098+0000 7fa0bea00700 1 -- 172.21.15.119:0/2941032921 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0c007ca40 con 0x7fa0c007de90 2024-03-28T00:15:48.265 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:47.098+0000 7fa0bea00700 1 --2- 172.21.15.119:0/2941032921 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa0c007de90 0x7fa0c007c2f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa0b00098a0 tx=0x7fa0b0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=319a985672a502af server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:15:48.265 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:47.098+0000 7fa0bd600700 1 -- 172.21.15.119:0/2941032921 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa0b0004160 con 0x7fa0c007de90 2024-03-28T00:15:48.265 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:47.099+0000 7fa0bd600700 1 -- 172.21.15.119:0/2941032921 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa0b0004300 con 0x7fa0c007de90 2024-03-28T00:15:48.265 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:47.099+0000 7fa0bd600700 1 -- 172.21.15.119:0/2941032921 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa0b0011b50 con 0x7fa0c007de90 2024-03-28T00:15:48.265 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:47.099+0000 7fa0c5a00700 1 -- 172.21.15.119:0/2941032921 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa0c007de90 msgr2=0x7fa0c007c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:15:48.265 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:47.099+0000 7fa0c5a00700 1 --2- 172.21.15.119:0/2941032921 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa0c007de90 0x7fa0c007c2f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa0b00098a0 tx=0x7fa0b0009bb0 comp rx=0 tx=0).stop 2024-03-28T00:15:48.265 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:47.099+0000 7fa0c5a00700 1 -- 172.21.15.119:0/2941032921 shutdown_connections 2024-03-28T00:15:48.265 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:47.099+0000 7fa0c5a00700 1 --2- 172.21.15.119:0/2941032921 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa0c007de90 0x7fa0c007c2f0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:15:48.266 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:47.099+0000 7fa0c5a00700 1 -- 172.21.15.119:0/2941032921 >> 172.21.15.119:0/2941032921 conn(0x7fa0c0103190 msgr2=0x7fa0c01055b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:15:48.266 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:47.100+0000 7fa0c5a00700 1 -- 172.21.15.119:0/2941032921 shutdown_connections 2024-03-28T00:15:48.266 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:47.100+0000 7fa0c5a00700 1 -- 172.21.15.119:0/2941032921 wait complete. 2024-03-28T00:15:48.266 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:47.101+0000 7fa0c5a00700 1 Processor -- start 2024-03-28T00:15:48.266 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:47.101+0000 7fa0c5a00700 1 -- start start 2024-03-28T00:15:48.266 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:47.101+0000 7fa0c5a00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa0c007de90 0x7fa0c01a2200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:15:48.266 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:47.101+0000 7fa0c5a00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c0109d20 con 0x7fa0c007de90 2024-03-28T00:15:48.266 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:47.102+0000 7fa0bea00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa0c007de90 0x7fa0c01a2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:15:48.266 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:47.102+0000 7fa0bea00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa0c007de90 0x7fa0c01a2200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35598/0 (socket says 172.21.15.119:35598) 2024-03-28T00:15:48.266 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:47.102+0000 7fa0bea00700 1 -- 172.21.15.119:0/2517502123 learned_addr learned my addr 172.21.15.119:0/2517502123 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:15:48.266 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:47.102+0000 7fa0bea00700 1 -- 172.21.15.119:0/2517502123 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0c01a2740 con 0x7fa0c007de90 2024-03-28T00:15:48.266 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:47.102+0000 7fa0bea00700 1 --2- 172.21.15.119:0/2517502123 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa0c007de90 0x7fa0c01a2200 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa0b0011850 tx=0x7fa0b0011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:15:48.266 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:47.102+0000 7fa0b7400700 1 -- 172.21.15.119:0/2517502123 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa0b0003af0 con 0x7fa0c007de90 2024-03-28T00:15:48.266 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:47.103+0000 7fa0b7400700 1 -- 172.21.15.119:0/2517502123 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa0b0020070 con 0x7fa0c007de90 2024-03-28T00:15:48.267 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:47.103+0000 7fa0c5a00700 1 -- 172.21.15.119:0/2517502123 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0c01a29d0 con 0x7fa0c007de90 2024-03-28T00:15:48.267 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:47.103+0000 7fa0b7400700 1 -- 172.21.15.119:0/2517502123 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa0b00195a0 con 0x7fa0c007de90 2024-03-28T00:15:48.267 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:47.103+0000 7fa0c5a00700 1 -- 172.21.15.119:0/2517502123 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0c01a34f0 con 0x7fa0c007de90 2024-03-28T00:15:48.267 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:47.104+0000 7fa0c5a00700 1 -- 172.21.15.119:0/2517502123 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0c00637c0 con 0x7fa0c007de90 2024-03-28T00:15:48.267 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:47.104+0000 7fa0b7400700 1 -- 172.21.15.119:0/2517502123 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 8) v1 ==== 46106+0+0 (secure 0 0 0) 0x7fa0b0019850 con 0x7fa0c007de90 2024-03-28T00:15:48.267 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:47.104+0000 7fa0b7400700 1 --2- 172.21.15.119:0/2517502123 >> [v2:172.21.15.119:6800/2575246519,v1:172.21.15.119:6801/2575246519] conn(0x7fa0a4038f70 0x7fa0a403b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:15:48.267 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:47.104+0000 7fa0b7400700 1 -- 172.21.15.119:0/2517502123 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fa0b0011e00 con 0x7fa0c007de90 2024-03-28T00:15:48.267 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:47.105+0000 7fa0be000700 1 --2- 172.21.15.119:0/2517502123 >> [v2:172.21.15.119:6800/2575246519,v1:172.21.15.119:6801/2575246519] conn(0x7fa0a4038f70 0x7fa0a403b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:15:48.267 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:47.105+0000 7fa0be000700 1 --2- 172.21.15.119:0/2517502123 >> [v2:172.21.15.119:6800/2575246519,v1:172.21.15.119:6801/2575246519] conn(0x7fa0a4038f70 0x7fa0a403b430 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fa0ac006fd0 tx=0x7fa0ac006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:15:48.267 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:47.110+0000 7fa0b7400700 1 -- 172.21.15.119:0/2517502123 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa0b0019b40 con 0x7fa0c007de90 2024-03-28T00:15:48.267 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:47.364+0000 7fa0c5a00700 1 -- 172.21.15.119:0/2517502123 --> [v2:172.21.15.119:6800/2575246519,v1:172.21.15.119:6801/2575246519] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fa0c01a4240 con 0x7fa0a4038f70 2024-03-28T00:15:48.267 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:47.371+0000 7fa0b7400700 1 -- 172.21.15.119:0/2517502123 <== mgr.14120 v2:172.21.15.119:6800/2575246519 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fa0c01a4240 con 0x7fa0a4038f70 2024-03-28T00:15:48.267 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:47.374+0000 7fa0c5a00700 1 -- 172.21.15.119:0/2517502123 >> [v2:172.21.15.119:6800/2575246519,v1:172.21.15.119:6801/2575246519] conn(0x7fa0a4038f70 msgr2=0x7fa0a403b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:15:48.268 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:47.374+0000 7fa0c5a00700 1 --2- 172.21.15.119:0/2517502123 >> [v2:172.21.15.119:6800/2575246519,v1:172.21.15.119:6801/2575246519] conn(0x7fa0a4038f70 0x7fa0a403b430 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fa0ac006fd0 tx=0x7fa0ac006e40 comp rx=0 tx=0).stop 2024-03-28T00:15:48.268 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:47.374+0000 7fa0c5a00700 1 -- 172.21.15.119:0/2517502123 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa0c007de90 msgr2=0x7fa0c01a2200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:15:48.268 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:47.374+0000 7fa0c5a00700 1 --2- 172.21.15.119:0/2517502123 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa0c007de90 0x7fa0c01a2200 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa0b0011850 tx=0x7fa0b0011c10 comp rx=0 tx=0).stop 2024-03-28T00:15:48.268 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:47.374+0000 7fa0c5a00700 1 -- 172.21.15.119:0/2517502123 shutdown_connections 2024-03-28T00:15:48.268 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:47.374+0000 7fa0c5a00700 1 --2- 172.21.15.119:0/2517502123 >> [v2:172.21.15.119:6800/2575246519,v1:172.21.15.119:6801/2575246519] conn(0x7fa0a4038f70 0x7fa0a403b430 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:15:48.268 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:47.374+0000 7fa0c5a00700 1 --2- 172.21.15.119:0/2517502123 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa0c007de90 0x7fa0c01a2200 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:15:48.268 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:47.374+0000 7fa0c5a00700 1 -- 172.21.15.119:0/2517502123 >> 172.21.15.119:0/2517502123 conn(0x7fa0c0103190 msgr2=0x7fa0c0198e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:15:48.268 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:47.374+0000 7fa0c5a00700 1 -- 172.21.15.119:0/2517502123 shutdown_connections 2024-03-28T00:15:48.268 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:47.374+0000 7fa0c5a00700 1 -- 172.21.15.119:0/2517502123 wait complete. 2024-03-28T00:15:48.268 INFO:teuthology.orchestra.run.smithi119.stdout:Deploying unmanaged mgr service... 2024-03-28T00:15:48.519 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:48 smithi119 ceph-mon[31518]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T00:15:48.519 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:48 smithi119 ceph-mon[31518]: Saving service mon spec with placement count:5 2024-03-28T00:15:48.519 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:48 smithi119 ceph-mon[31518]: from='mgr.14120 172.21.15.119:0/4048015587' entity='mgr.x' 2024-03-28T00:15:50.365 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-03-28T00:15:50.365 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:49.207+0000 7f74b4400700 1 Processor -- start 2024-03-28T00:15:50.366 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:49.207+0000 7f74b4400700 1 -- start start 2024-03-28T00:15:50.366 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:49.208+0000 7f74b4400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f74ac107b30 0x7f74ac107f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:15:50.366 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:49.208+0000 7f74b4400700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74ac108520 con 0x7f74ac107b30 2024-03-28T00:15:50.366 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:49.208+0000 7f74b1a00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f74ac107b30 0x7f74ac107f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:15:50.366 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:49.208+0000 7f74b1a00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f74ac107b30 0x7f74ac107f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35606/0 (socket says 172.21.15.119:35606) 2024-03-28T00:15:50.366 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:49.208+0000 7f74b1a00700 1 -- 172.21.15.119:0/1777526355 learned_addr learned my addr 172.21.15.119:0/1777526355 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:15:50.366 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:49.209+0000 7f74b1a00700 1 -- 172.21.15.119:0/1777526355 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74ac108d30 con 0x7f74ac107b30 2024-03-28T00:15:50.366 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:49.209+0000 7f74b1a00700 1 --2- 172.21.15.119:0/1777526355 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f74ac107b30 0x7f74ac107f50 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f74a0009e80 tx=0x7f74a000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=725b131b291c1459 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:15:50.371 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:49.209+0000 7f74abe00700 1 -- 172.21.15.119:0/1777526355 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f74a0017070 con 0x7f74ac107b30 2024-03-28T00:15:50.371 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:49.209+0000 7f74abe00700 1 -- 172.21.15.119:0/1777526355 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f74a000b8c0 con 0x7f74ac107b30 2024-03-28T00:15:50.371 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:49.209+0000 7f74abe00700 1 -- 172.21.15.119:0/1777526355 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f74a00045c0 con 0x7f74ac107b30 2024-03-28T00:15:50.371 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:49.210+0000 7f74b4400700 1 -- 172.21.15.119:0/1777526355 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f74ac107b30 msgr2=0x7f74ac107f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:15:50.371 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:49.210+0000 7f74b4400700 1 --2- 172.21.15.119:0/1777526355 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f74ac107b30 0x7f74ac107f50 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f74a0009e80 tx=0x7f74a000b330 comp rx=0 tx=0).stop 2024-03-28T00:15:50.371 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:49.210+0000 7f74b4400700 1 -- 172.21.15.119:0/1777526355 shutdown_connections 2024-03-28T00:15:50.371 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:49.210+0000 7f74b4400700 1 --2- 172.21.15.119:0/1777526355 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f74ac107b30 0x7f74ac107f50 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:15:50.371 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:49.210+0000 7f74b4400700 1 -- 172.21.15.119:0/1777526355 >> 172.21.15.119:0/1777526355 conn(0x7f74ac1030f0 msgr2=0x7f74ac105510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:15:50.371 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:49.210+0000 7f74b4400700 1 -- 172.21.15.119:0/1777526355 shutdown_connections 2024-03-28T00:15:50.371 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:49.210+0000 7f74b4400700 1 -- 172.21.15.119:0/1777526355 wait complete. 2024-03-28T00:15:50.371 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:49.211+0000 7f74b4400700 1 Processor -- start 2024-03-28T00:15:50.372 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:49.212+0000 7f74b4400700 1 -- start start 2024-03-28T00:15:50.372 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:49.212+0000 7f74b4400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f74ac107b30 0x7f74ac07dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:15:50.372 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:49.212+0000 7f74b4400700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74ac10c0e0 con 0x7f74ac107b30 2024-03-28T00:15:50.372 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:49.212+0000 7f74b1a00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f74ac107b30 0x7f74ac07dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:15:50.372 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:49.212+0000 7f74b1a00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f74ac107b30 0x7f74ac07dd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35622/0 (socket says 172.21.15.119:35622) 2024-03-28T00:15:50.372 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:49.212+0000 7f74b1a00700 1 -- 172.21.15.119:0/814948819 learned_addr learned my addr 172.21.15.119:0/814948819 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:15:50.372 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:49.213+0000 7f74b1a00700 1 -- 172.21.15.119:0/814948819 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74ac07c250 con 0x7f74ac107b30 2024-03-28T00:15:50.372 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:49.213+0000 7f74b1a00700 1 --2- 172.21.15.119:0/814948819 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f74ac107b30 0x7f74ac07dd00 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f74a0009530 tx=0x7f74a000bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:15:50.372 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:49.213+0000 7f74aa000700 1 -- 172.21.15.119:0/814948819 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f74a0020020 con 0x7f74ac107b30 2024-03-28T00:15:50.372 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:49.213+0000 7f74aa000700 1 -- 172.21.15.119:0/814948819 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f74a0003b50 con 0x7f74ac107b30 2024-03-28T00:15:50.372 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:49.213+0000 7f74b4400700 1 -- 172.21.15.119:0/814948819 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74ac07c4e0 con 0x7f74ac107b30 2024-03-28T00:15:50.372 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:49.213+0000 7f74aa000700 1 -- 172.21.15.119:0/814948819 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f74a0017040 con 0x7f74ac107b30 2024-03-28T00:15:50.372 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:49.213+0000 7f74b4400700 1 -- 172.21.15.119:0/814948819 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74ac07ca20 con 0x7f74ac107b30 2024-03-28T00:15:50.372 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:49.214+0000 7f74aa000700 1 -- 172.21.15.119:0/814948819 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 8) v1 ==== 46106+0+0 (secure 0 0 0) 0x7f74a0023020 con 0x7f74ac107b30 2024-03-28T00:15:50.373 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:49.214+0000 7f74b4400700 1 -- 172.21.15.119:0/814948819 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74ac0637c0 con 0x7f74ac107b30 2024-03-28T00:15:50.373 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:49.215+0000 7f74aa000700 1 --2- 172.21.15.119:0/814948819 >> [v2:172.21.15.119:6800/2575246519,v1:172.21.15.119:6801/2575246519] conn(0x7f7494038f20 0x7f749403b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:15:50.373 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:49.215+0000 7f74aa000700 1 -- 172.21.15.119:0/814948819 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f74a000f050 con 0x7f74ac107b30 2024-03-28T00:15:50.373 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:49.215+0000 7f74b1000700 1 --2- 172.21.15.119:0/814948819 >> [v2:172.21.15.119:6800/2575246519,v1:172.21.15.119:6801/2575246519] conn(0x7f7494038f20 0x7f749403b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:15:50.373 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:49.216+0000 7f74b1000700 1 --2- 172.21.15.119:0/814948819 >> [v2:172.21.15.119:6800/2575246519,v1:172.21.15.119:6801/2575246519] conn(0x7f7494038f20 0x7f749403b3e0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f7498006fd0 tx=0x7f7498006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:15:50.373 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:49.220+0000 7f74aa000700 1 -- 172.21.15.119:0/814948819 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f74a002a020 con 0x7f74ac107b30 2024-03-28T00:15:50.373 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:49.456+0000 7f74b4400700 1 -- 172.21.15.119:0/814948819 --> [v2:172.21.15.119:6800/2575246519,v1:172.21.15.119:6801/2575246519] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f74ac07d040 con 0x7f7494038f20 2024-03-28T00:15:50.373 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:49.464+0000 7f74aa000700 1 -- 172.21.15.119:0/814948819 <== mgr.14120 v2:172.21.15.119:6800/2575246519 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f74ac07d040 con 0x7f7494038f20 2024-03-28T00:15:50.373 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:49.467+0000 7f74b4400700 1 -- 172.21.15.119:0/814948819 >> [v2:172.21.15.119:6800/2575246519,v1:172.21.15.119:6801/2575246519] conn(0x7f7494038f20 msgr2=0x7f749403b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:15:50.373 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:49.467+0000 7f74b4400700 1 --2- 172.21.15.119:0/814948819 >> [v2:172.21.15.119:6800/2575246519,v1:172.21.15.119:6801/2575246519] conn(0x7f7494038f20 0x7f749403b3e0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f7498006fd0 tx=0x7f7498006e40 comp rx=0 tx=0).stop 2024-03-28T00:15:50.373 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:49.467+0000 7f74b4400700 1 -- 172.21.15.119:0/814948819 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f74ac107b30 msgr2=0x7f74ac07dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:15:50.373 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:49.467+0000 7f74b4400700 1 --2- 172.21.15.119:0/814948819 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f74ac107b30 0x7f74ac07dd00 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f74a0009530 tx=0x7f74a000bd50 comp rx=0 tx=0).stop 2024-03-28T00:15:50.373 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:49.467+0000 7f74b4400700 1 -- 172.21.15.119:0/814948819 shutdown_connections 2024-03-28T00:15:50.373 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:49.467+0000 7f74b4400700 1 --2- 172.21.15.119:0/814948819 >> [v2:172.21.15.119:6800/2575246519,v1:172.21.15.119:6801/2575246519] conn(0x7f7494038f20 0x7f749403b3e0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:15:50.374 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:49.467+0000 7f74b4400700 1 --2- 172.21.15.119:0/814948819 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f74ac107b30 0x7f74ac07dd00 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:15:50.374 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:49.467+0000 7f74b4400700 1 -- 172.21.15.119:0/814948819 >> 172.21.15.119:0/814948819 conn(0x7f74ac1030f0 msgr2=0x7f74ac103bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:15:50.374 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:49.467+0000 7f74b4400700 1 -- 172.21.15.119:0/814948819 shutdown_connections 2024-03-28T00:15:50.374 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:49.467+0000 7f74b4400700 1 -- 172.21.15.119:0/814948819 wait complete. 2024-03-28T00:15:50.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:50 smithi119 ceph-mon[31518]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T00:15:50.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:50 smithi119 ceph-mon[31518]: Saving service mgr spec with placement count:2 2024-03-28T00:15:50.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:50 smithi119 ceph-mon[31518]: from='mgr.14120 172.21.15.119:0/4048015587' entity='mgr.x' 2024-03-28T00:15:50.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:50 smithi119 ceph-mon[31518]: from='mgr.14120 172.21.15.119:0/4048015587' entity='mgr.x' 2024-03-28T00:15:51.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:51 smithi119 ceph-mon[31518]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T00:15:51.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:51 smithi119 ceph-mon[31518]: from='mgr.14120 172.21.15.119:0/4048015587' entity='mgr.x' 2024-03-28T00:15:51.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:51 smithi119 ceph-mon[31518]: from='client.? 172.21.15.119:0/3063387328' entity='client.admin' 2024-03-28T00:15:51.957 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:50.949+0000 7f0bda800700 1 Processor -- start 2024-03-28T00:15:51.958 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:50.949+0000 7f0bda800700 1 -- start start 2024-03-28T00:15:51.958 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:50.949+0000 7f0bda800700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0bd407dde0 0x7f0bd407c290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:15:51.958 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:50.949+0000 7f0bda800700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bd407c7d0 con 0x7f0bd407dde0 2024-03-28T00:15:51.958 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:50.950+0000 7f0bd3400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0bd407dde0 0x7f0bd407c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:15:51.960 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:50.950+0000 7f0bd3400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0bd407dde0 0x7f0bd407c290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35632/0 (socket says 172.21.15.119:35632) 2024-03-28T00:15:51.960 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:50.950+0000 7f0bd3400700 1 -- 172.21.15.119:0/707949128 learned_addr learned my addr 172.21.15.119:0/707949128 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:15:51.961 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:50.950+0000 7f0bd3400700 1 -- 172.21.15.119:0/707949128 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bd407c950 con 0x7f0bd407dde0 2024-03-28T00:15:51.961 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:50.950+0000 7f0bd3400700 1 --2- 172.21.15.119:0/707949128 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0bd407dde0 0x7f0bd407c290 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f0bc40098a0 tx=0x7f0bc4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce83a6689038f807 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:15:51.961 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:50.951+0000 7f0bd2000700 1 -- 172.21.15.119:0/707949128 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0bc4004160 con 0x7f0bd407dde0 2024-03-28T00:15:51.961 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:50.951+0000 7f0bd2000700 1 -- 172.21.15.119:0/707949128 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0bc4004300 con 0x7f0bd407dde0 2024-03-28T00:15:51.961 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:50.951+0000 7f0bd2000700 1 -- 172.21.15.119:0/707949128 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0bc4011b20 con 0x7f0bd407dde0 2024-03-28T00:15:51.961 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:50.951+0000 7f0bda800700 1 -- 172.21.15.119:0/707949128 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0bd407dde0 msgr2=0x7f0bd407c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:15:51.961 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:50.951+0000 7f0bda800700 1 --2- 172.21.15.119:0/707949128 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0bd407dde0 0x7f0bd407c290 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f0bc40098a0 tx=0x7f0bc4009bb0 comp rx=0 tx=0).stop 2024-03-28T00:15:51.961 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:50.952+0000 7f0bda800700 1 -- 172.21.15.119:0/707949128 shutdown_connections 2024-03-28T00:15:51.961 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:50.952+0000 7f0bda800700 1 --2- 172.21.15.119:0/707949128 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0bd407dde0 0x7f0bd407c290 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:15:51.961 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:50.952+0000 7f0bda800700 1 -- 172.21.15.119:0/707949128 >> 172.21.15.119:0/707949128 conn(0x7f0bd4103140 msgr2=0x7f0bd4105560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:15:51.961 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:50.952+0000 7f0bda800700 1 -- 172.21.15.119:0/707949128 shutdown_connections 2024-03-28T00:15:51.961 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:50.952+0000 7f0bda800700 1 -- 172.21.15.119:0/707949128 wait complete. 2024-03-28T00:15:51.961 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:50.953+0000 7f0bda800700 1 Processor -- start 2024-03-28T00:15:51.961 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:50.953+0000 7f0bda800700 1 -- start start 2024-03-28T00:15:51.962 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:50.953+0000 7f0bda800700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0bd407dde0 0x7f0bd419dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:15:51.962 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:50.953+0000 7f0bda800700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bd410e100 con 0x7f0bd407dde0 2024-03-28T00:15:51.962 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:50.953+0000 7f0bd3400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0bd407dde0 0x7f0bd419dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:15:51.962 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:50.953+0000 7f0bd3400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0bd407dde0 0x7f0bd419dc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35648/0 (socket says 172.21.15.119:35648) 2024-03-28T00:15:51.962 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:50.953+0000 7f0bd3400700 1 -- 172.21.15.119:0/3063387328 learned_addr learned my addr 172.21.15.119:0/3063387328 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:15:51.965 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:50.954+0000 7f0bd3400700 1 -- 172.21.15.119:0/3063387328 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bd419e1a0 con 0x7f0bd407dde0 2024-03-28T00:15:51.965 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:50.954+0000 7f0bd3400700 1 --2- 172.21.15.119:0/3063387328 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0bd407dde0 0x7f0bd419dc60 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f0bc40116a0 tx=0x7f0bc4011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:15:51.965 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:50.954+0000 7f0bcbe00700 1 -- 172.21.15.119:0/3063387328 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0bc4003ac0 con 0x7f0bd407dde0 2024-03-28T00:15:51.965 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:50.954+0000 7f0bcbe00700 1 -- 172.21.15.119:0/3063387328 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0bc4003c60 con 0x7f0bd407dde0 2024-03-28T00:15:51.965 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:50.955+0000 7f0bda800700 1 -- 172.21.15.119:0/3063387328 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bd419e430 con 0x7f0bd407dde0 2024-03-28T00:15:51.965 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:50.955+0000 7f0bcbe00700 1 -- 172.21.15.119:0/3063387328 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0bc401a7d0 con 0x7f0bd407dde0 2024-03-28T00:15:51.965 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:50.955+0000 7f0bda800700 1 -- 172.21.15.119:0/3063387328 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bd419e890 con 0x7f0bd407dde0 2024-03-28T00:15:51.965 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:50.956+0000 7f0bcbe00700 1 -- 172.21.15.119:0/3063387328 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 8) v1 ==== 46106+0+0 (secure 0 0 0) 0x7f0bc401aac0 con 0x7f0bd407dde0 2024-03-28T00:15:51.965 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:50.956+0000 7f0bda800700 1 -- 172.21.15.119:0/3063387328 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bd4069260 con 0x7f0bd407dde0 2024-03-28T00:15:51.965 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:50.957+0000 7f0bcbe00700 1 --2- 172.21.15.119:0/3063387328 >> [v2:172.21.15.119:6800/2575246519,v1:172.21.15.119:6801/2575246519] conn(0x7f0bb8038f20 0x7f0bb803b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:15:51.966 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:50.957+0000 7f0bcbe00700 1 -- 172.21.15.119:0/3063387328 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f0bc404d500 con 0x7f0bd407dde0 2024-03-28T00:15:51.966 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:50.957+0000 7f0bd2a00700 1 --2- 172.21.15.119:0/3063387328 >> [v2:172.21.15.119:6800/2575246519,v1:172.21.15.119:6801/2575246519] conn(0x7f0bb8038f20 0x7f0bb803b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:15:51.966 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:50.958+0000 7f0bd2a00700 1 --2- 172.21.15.119:0/3063387328 >> [v2:172.21.15.119:6800/2575246519,v1:172.21.15.119:6801/2575246519] conn(0x7f0bb8038f20 0x7f0bb803b3e0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f0bc0006fd0 tx=0x7f0bc0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:15:51.966 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:50.962+0000 7f0bcbe00700 1 -- 172.21.15.119:0/3063387328 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f0bc4011e20 con 0x7f0bd407dde0 2024-03-28T00:15:51.966 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:51.183+0000 7f0bda800700 1 -- 172.21.15.119:0/3063387328 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7f0bd41a1120 con 0x7f0bd407dde0 2024-03-28T00:15:51.966 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:51.189+0000 7f0bcbe00700 1 -- 172.21.15.119:0/3063387328 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7f0bc404ce50 con 0x7f0bd407dde0 2024-03-28T00:15:51.966 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:51.192+0000 7f0bda800700 1 -- 172.21.15.119:0/3063387328 >> [v2:172.21.15.119:6800/2575246519,v1:172.21.15.119:6801/2575246519] conn(0x7f0bb8038f20 msgr2=0x7f0bb803b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:15:51.966 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:51.192+0000 7f0bda800700 1 --2- 172.21.15.119:0/3063387328 >> [v2:172.21.15.119:6800/2575246519,v1:172.21.15.119:6801/2575246519] conn(0x7f0bb8038f20 0x7f0bb803b3e0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f0bc0006fd0 tx=0x7f0bc0006e40 comp rx=0 tx=0).stop 2024-03-28T00:15:51.966 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:51.192+0000 7f0bda800700 1 -- 172.21.15.119:0/3063387328 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0bd407dde0 msgr2=0x7f0bd419dc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:15:51.966 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:51.192+0000 7f0bda800700 1 --2- 172.21.15.119:0/3063387328 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0bd407dde0 0x7f0bd419dc60 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f0bc40116a0 tx=0x7f0bc4011c10 comp rx=0 tx=0).stop 2024-03-28T00:15:51.966 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:51.192+0000 7f0bda800700 1 -- 172.21.15.119:0/3063387328 shutdown_connections 2024-03-28T00:15:51.966 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:51.192+0000 7f0bda800700 1 --2- 172.21.15.119:0/3063387328 >> [v2:172.21.15.119:6800/2575246519,v1:172.21.15.119:6801/2575246519] conn(0x7f0bb8038f20 0x7f0bb803b3e0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:15:51.966 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:51.192+0000 7f0bda800700 1 --2- 172.21.15.119:0/3063387328 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0bd407dde0 0x7f0bd419dc60 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:15:51.966 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:51.192+0000 7f0bda800700 1 -- 172.21.15.119:0/3063387328 >> 172.21.15.119:0/3063387328 conn(0x7f0bd4103140 msgr2=0x7f0bd4103bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:15:51.967 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:51.192+0000 7f0bda800700 1 -- 172.21.15.119:0/3063387328 shutdown_connections 2024-03-28T00:15:51.967 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:51.193+0000 7f0bda800700 1 -- 172.21.15.119:0/3063387328 wait complete. 2024-03-28T00:15:53.831 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:53 smithi119 ceph-mon[31518]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T00:15:53.831 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:53 smithi119 ceph-mon[31518]: from='client.? 172.21.15.119:0/4170352498' entity='client.admin' 2024-03-28T00:15:53.831 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:53 smithi119 ceph-mon[31518]: from='mgr.14120 172.21.15.119:0/4048015587' entity='mgr.x' 2024-03-28T00:15:53.844 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:52.533+0000 7f9b43e00700 1 Processor -- start 2024-03-28T00:15:53.844 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:52.533+0000 7f9b43e00700 1 -- start start 2024-03-28T00:15:53.844 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:52.534+0000 7f9b43e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9b4407e810 0x7f9b4407ec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:15:53.844 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:52.534+0000 7f9b43e00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b4407f200 con 0x7f9b4407e810 2024-03-28T00:15:53.844 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:52.534+0000 7f9b42a00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9b4407e810 0x7f9b4407ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:15:53.844 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:52.534+0000 7f9b42a00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9b4407e810 0x7f9b4407ec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35658/0 (socket says 172.21.15.119:35658) 2024-03-28T00:15:53.844 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:52.534+0000 7f9b42a00700 1 -- 172.21.15.119:0/1071322943 learned_addr learned my addr 172.21.15.119:0/1071322943 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:15:53.844 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:52.534+0000 7f9b42a00700 1 -- 172.21.15.119:0/1071322943 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b4407fa60 con 0x7f9b4407e810 2024-03-28T00:15:53.844 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:52.535+0000 7f9b42a00700 1 --2- 172.21.15.119:0/1071322943 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9b4407e810 0x7f9b4407ec30 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f9b340098a0 tx=0x7f9b34009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdecd150159382c7 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:15:53.844 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:52.535+0000 7f9b41600700 1 -- 172.21.15.119:0/1071322943 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9b34004160 con 0x7f9b4407e810 2024-03-28T00:15:53.844 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:52.535+0000 7f9b41600700 1 -- 172.21.15.119:0/1071322943 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9b34004300 con 0x7f9b4407e810 2024-03-28T00:15:53.845 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:52.535+0000 7f9b41600700 1 -- 172.21.15.119:0/1071322943 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9b34011b20 con 0x7f9b4407e810 2024-03-28T00:15:53.845 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:52.536+0000 7f9b43e00700 1 -- 172.21.15.119:0/1071322943 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9b4407e810 msgr2=0x7f9b4407ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:15:53.845 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:52.536+0000 7f9b43e00700 1 --2- 172.21.15.119:0/1071322943 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9b4407e810 0x7f9b4407ec30 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f9b340098a0 tx=0x7f9b34009bb0 comp rx=0 tx=0).stop 2024-03-28T00:15:53.845 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:52.536+0000 7f9b43e00700 1 -- 172.21.15.119:0/1071322943 shutdown_connections 2024-03-28T00:15:53.845 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:52.536+0000 7f9b43e00700 1 --2- 172.21.15.119:0/1071322943 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9b4407e810 0x7f9b4407ec30 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:15:53.845 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:52.536+0000 7f9b43e00700 1 -- 172.21.15.119:0/1071322943 >> 172.21.15.119:0/1071322943 conn(0x7f9b4407cec0 msgr2=0x7f9b4407d310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:15:53.845 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:52.536+0000 7f9b43e00700 1 -- 172.21.15.119:0/1071322943 shutdown_connections 2024-03-28T00:15:53.845 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:52.536+0000 7f9b43e00700 1 -- 172.21.15.119:0/1071322943 wait complete. 2024-03-28T00:15:53.845 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:52.537+0000 7f9b43e00700 1 Processor -- start 2024-03-28T00:15:53.845 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:52.537+0000 7f9b43e00700 1 -- start start 2024-03-28T00:15:53.845 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:52.537+0000 7f9b43e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9b4407e810 0x7f9b441a1e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:15:53.845 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:52.538+0000 7f9b43e00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b44080e20 con 0x7f9b4407e810 2024-03-28T00:15:53.845 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:52.538+0000 7f9b42a00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9b4407e810 0x7f9b441a1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:15:53.845 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:52.538+0000 7f9b42a00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9b4407e810 0x7f9b441a1e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35662/0 (socket says 172.21.15.119:35662) 2024-03-28T00:15:53.846 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:52.538+0000 7f9b42a00700 1 -- 172.21.15.119:0/4170352498 learned_addr learned my addr 172.21.15.119:0/4170352498 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:15:53.846 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:52.538+0000 7f9b42a00700 1 -- 172.21.15.119:0/4170352498 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b441a2370 con 0x7f9b4407e810 2024-03-28T00:15:53.846 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:52.538+0000 7f9b42a00700 1 --2- 172.21.15.119:0/4170352498 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9b4407e810 0x7f9b441a1e30 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f9b340117e0 tx=0x7f9b34011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:15:53.846 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:52.539+0000 7f9b3b400700 1 -- 172.21.15.119:0/4170352498 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9b34003ba0 con 0x7f9b4407e810 2024-03-28T00:15:53.846 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:52.539+0000 7f9b3b400700 1 -- 172.21.15.119:0/4170352498 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9b3401a910 con 0x7f9b4407e810 2024-03-28T00:15:53.846 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:52.539+0000 7f9b3b400700 1 -- 172.21.15.119:0/4170352498 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9b340248f0 con 0x7f9b4407e810 2024-03-28T00:15:53.846 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:52.539+0000 7f9b43e00700 1 -- 172.21.15.119:0/4170352498 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b441a2600 con 0x7f9b4407e810 2024-03-28T00:15:53.846 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:52.539+0000 7f9b43e00700 1 -- 172.21.15.119:0/4170352498 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b441a2a60 con 0x7f9b4407e810 2024-03-28T00:15:53.846 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:52.540+0000 7f9b3b400700 1 -- 172.21.15.119:0/4170352498 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 8) v1 ==== 46106+0+0 (secure 0 0 0) 0x7f9b34022020 con 0x7f9b4407e810 2024-03-28T00:15:53.846 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:52.540+0000 7f9b43e00700 1 -- 172.21.15.119:0/4170352498 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b44069260 con 0x7f9b4407e810 2024-03-28T00:15:53.846 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:52.541+0000 7f9b3b400700 1 --2- 172.21.15.119:0/4170352498 >> [v2:172.21.15.119:6800/2575246519,v1:172.21.15.119:6801/2575246519] conn(0x7f9b28038f70 0x7f9b2803b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:15:53.846 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:52.541+0000 7f9b3b400700 1 -- 172.21.15.119:0/4170352498 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f9b3404d340 con 0x7f9b4407e810 2024-03-28T00:15:53.847 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:52.541+0000 7f9b42000700 1 --2- 172.21.15.119:0/4170352498 >> [v2:172.21.15.119:6800/2575246519,v1:172.21.15.119:6801/2575246519] conn(0x7f9b28038f70 0x7f9b2803b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:15:53.847 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:52.542+0000 7f9b42000700 1 --2- 172.21.15.119:0/4170352498 >> [v2:172.21.15.119:6800/2575246519,v1:172.21.15.119:6801/2575246519] conn(0x7f9b28038f70 0x7f9b2803b430 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f9b30006fd0 tx=0x7f9b30006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:15:53.847 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:52.546+0000 7f9b3b400700 1 -- 172.21.15.119:0/4170352498 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f9b3401ab70 con 0x7f9b4407e810 2024-03-28T00:15:53.847 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:52.775+0000 7f9b43e00700 1 -- 172.21.15.119:0/4170352498 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7f9b44062da0 con 0x7f9b4407e810 2024-03-28T00:15:53.847 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:52.779+0000 7f9b3b400700 1 -- 172.21.15.119:0/4170352498 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7f9b3404cc90 con 0x7f9b4407e810 2024-03-28T00:15:53.847 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:52.782+0000 7f9b43e00700 1 -- 172.21.15.119:0/4170352498 >> [v2:172.21.15.119:6800/2575246519,v1:172.21.15.119:6801/2575246519] conn(0x7f9b28038f70 msgr2=0x7f9b2803b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:15:53.847 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:52.782+0000 7f9b43e00700 1 --2- 172.21.15.119:0/4170352498 >> [v2:172.21.15.119:6800/2575246519,v1:172.21.15.119:6801/2575246519] conn(0x7f9b28038f70 0x7f9b2803b430 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f9b30006fd0 tx=0x7f9b30006e40 comp rx=0 tx=0).stop 2024-03-28T00:15:53.847 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:52.782+0000 7f9b43e00700 1 -- 172.21.15.119:0/4170352498 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9b4407e810 msgr2=0x7f9b441a1e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:15:53.847 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:52.782+0000 7f9b43e00700 1 --2- 172.21.15.119:0/4170352498 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9b4407e810 0x7f9b441a1e30 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f9b340117e0 tx=0x7f9b34011ba0 comp rx=0 tx=0).stop 2024-03-28T00:15:53.847 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:52.782+0000 7f9b43e00700 1 -- 172.21.15.119:0/4170352498 shutdown_connections 2024-03-28T00:15:53.847 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:52.782+0000 7f9b43e00700 1 --2- 172.21.15.119:0/4170352498 >> [v2:172.21.15.119:6800/2575246519,v1:172.21.15.119:6801/2575246519] conn(0x7f9b28038f70 0x7f9b2803b430 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:15:53.847 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:52.782+0000 7f9b43e00700 1 --2- 172.21.15.119:0/4170352498 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9b4407e810 0x7f9b441a1e30 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:15:53.847 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:52.782+0000 7f9b43e00700 1 -- 172.21.15.119:0/4170352498 >> 172.21.15.119:0/4170352498 conn(0x7f9b4407cec0 msgr2=0x7f9b44198af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:15:53.847 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:52.782+0000 7f9b43e00700 1 -- 172.21.15.119:0/4170352498 shutdown_connections 2024-03-28T00:15:53.848 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:52.783+0000 7f9b43e00700 1 -- 172.21.15.119:0/4170352498 wait complete. 2024-03-28T00:15:53.848 INFO:teuthology.orchestra.run.smithi119.stdout:Enabling the dashboard module... 2024-03-28T00:15:55.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:54 smithi119 ceph-mon[31518]: from='client.? 172.21.15.119:0/2021195529' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-03-28T00:15:55.156 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 00:15:54 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-x[31752]: ignoring --setuser ceph since I am not root 2024-03-28T00:15:55.156 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 00:15:54 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-x[31752]: ignoring --setgroup ceph since I am not root 2024-03-28T00:15:55.577 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:54.436+0000 7f24f7a00700 1 Processor -- start 2024-03-28T00:15:55.577 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:54.436+0000 7f24f7a00700 1 -- start start 2024-03-28T00:15:55.578 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:54.437+0000 7f24f7a00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f24f01085a0 0x7f24f010a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:15:55.578 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:54.437+0000 7f24f7a00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24f007be60 con 0x7f24f01085a0 2024-03-28T00:15:55.578 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:54.437+0000 7f24f5000700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f24f01085a0 0x7f24f010a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:15:55.578 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:54.437+0000 7f24f5000700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f24f01085a0 0x7f24f010a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55968/0 (socket says 172.21.15.119:55968) 2024-03-28T00:15:55.578 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:54.437+0000 7f24f5000700 1 -- 172.21.15.119:0/2623963903 learned_addr learned my addr 172.21.15.119:0/2623963903 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:15:55.578 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:54.438+0000 7f24f5000700 1 -- 172.21.15.119:0/2623963903 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24f010aed0 con 0x7f24f01085a0 2024-03-28T00:15:55.578 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:54.438+0000 7f24f5000700 1 --2- 172.21.15.119:0/2623963903 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f24f01085a0 0x7f24f010a990 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f24e40098a0 tx=0x7f24e4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7cc843648c9eb92 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:15:55.578 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:54.438+0000 7f24ef400700 1 -- 172.21.15.119:0/2623963903 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f24e4004160 con 0x7f24f01085a0 2024-03-28T00:15:55.578 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:54.438+0000 7f24ef400700 1 -- 172.21.15.119:0/2623963903 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f24e4004300 con 0x7f24f01085a0 2024-03-28T00:15:55.578 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:54.438+0000 7f24ef400700 1 -- 172.21.15.119:0/2623963903 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f24e4011b20 con 0x7f24f01085a0 2024-03-28T00:15:55.578 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:54.439+0000 7f24f7a00700 1 -- 172.21.15.119:0/2623963903 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f24f01085a0 msgr2=0x7f24f010a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:15:55.578 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:54.439+0000 7f24f7a00700 1 --2- 172.21.15.119:0/2623963903 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f24f01085a0 0x7f24f010a990 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f24e40098a0 tx=0x7f24e4009bb0 comp rx=0 tx=0).stop 2024-03-28T00:15:55.578 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:54.439+0000 7f24f7a00700 1 -- 172.21.15.119:0/2623963903 shutdown_connections 2024-03-28T00:15:55.579 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:54.439+0000 7f24f7a00700 1 --2- 172.21.15.119:0/2623963903 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f24f01085a0 0x7f24f010a990 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:15:55.579 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:54.439+0000 7f24f7a00700 1 -- 172.21.15.119:0/2623963903 >> 172.21.15.119:0/2623963903 conn(0x7f24f0101ff0 msgr2=0x7f24f0104410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:15:55.579 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:54.439+0000 7f24f7a00700 1 -- 172.21.15.119:0/2623963903 shutdown_connections 2024-03-28T00:15:55.579 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:54.439+0000 7f24f7a00700 1 -- 172.21.15.119:0/2623963903 wait complete. 2024-03-28T00:15:55.579 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:54.440+0000 7f24f7a00700 1 Processor -- start 2024-03-28T00:15:55.579 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:54.441+0000 7f24f7a00700 1 -- start start 2024-03-28T00:15:55.579 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:54.441+0000 7f24f7a00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f24f01085a0 0x7f24f019ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:15:55.579 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:54.441+0000 7f24f7a00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24f010bf40 con 0x7f24f01085a0 2024-03-28T00:15:55.579 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:54.441+0000 7f24f5000700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f24f01085a0 0x7f24f019ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:15:55.580 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:54.441+0000 7f24f5000700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f24f01085a0 0x7f24f019ddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55974/0 (socket says 172.21.15.119:55974) 2024-03-28T00:15:55.580 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:54.441+0000 7f24f5000700 1 -- 172.21.15.119:0/2021195529 learned_addr learned my addr 172.21.15.119:0/2021195529 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:15:55.580 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:54.441+0000 7f24f5000700 1 -- 172.21.15.119:0/2021195529 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24f019e2f0 con 0x7f24f01085a0 2024-03-28T00:15:55.580 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:54.442+0000 7f24f5000700 1 --2- 172.21.15.119:0/2021195529 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f24f01085a0 0x7f24f019ddb0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f24e40117e0 tx=0x7f24e4011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:15:55.580 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:54.442+0000 7f24ed600700 1 -- 172.21.15.119:0/2021195529 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f24e4003ba0 con 0x7f24f01085a0 2024-03-28T00:15:55.580 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:54.442+0000 7f24ed600700 1 -- 172.21.15.119:0/2021195529 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f24e401a910 con 0x7f24f01085a0 2024-03-28T00:15:55.580 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:54.442+0000 7f24ed600700 1 -- 172.21.15.119:0/2021195529 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f24e40248f0 con 0x7f24f01085a0 2024-03-28T00:15:55.580 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:54.442+0000 7f24f7a00700 1 -- 172.21.15.119:0/2021195529 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24f019e580 con 0x7f24f01085a0 2024-03-28T00:15:55.580 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:54.442+0000 7f24f7a00700 1 -- 172.21.15.119:0/2021195529 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24f019ef60 con 0x7f24f01085a0 2024-03-28T00:15:55.580 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:54.443+0000 7f24ed600700 1 -- 172.21.15.119:0/2021195529 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 8) v1 ==== 46106+0+0 (secure 0 0 0) 0x7f24e4022020 con 0x7f24f01085a0 2024-03-28T00:15:55.580 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:54.443+0000 7f24f7a00700 1 -- 172.21.15.119:0/2021195529 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24f00637c0 con 0x7f24f01085a0 2024-03-28T00:15:55.580 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:54.444+0000 7f24ed600700 1 --2- 172.21.15.119:0/2021195529 >> [v2:172.21.15.119:6800/2575246519,v1:172.21.15.119:6801/2575246519] conn(0x7f24d8038f70 0x7f24d803b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:15:55.580 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:54.444+0000 7f24ed600700 1 -- 172.21.15.119:0/2021195529 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f24e404d330 con 0x7f24f01085a0 2024-03-28T00:15:55.580 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:54.444+0000 7f24efe00700 1 --2- 172.21.15.119:0/2021195529 >> [v2:172.21.15.119:6800/2575246519,v1:172.21.15.119:6801/2575246519] conn(0x7f24d8038f70 0x7f24d803b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:15:55.581 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:54.445+0000 7f24efe00700 1 --2- 172.21.15.119:0/2021195529 >> [v2:172.21.15.119:6800/2575246519,v1:172.21.15.119:6801/2575246519] conn(0x7f24d8038f70 0x7f24d803b430 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f24dc006fd0 tx=0x7f24dc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:15:55.581 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:54.449+0000 7f24ed600700 1 -- 172.21.15.119:0/2021195529 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f24e404cc80 con 0x7f24f01085a0 2024-03-28T00:15:55.581 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:54.735+0000 7f24f7a00700 1 -- 172.21.15.119:0/2021195529 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7f24f019f450 con 0x7f24f01085a0 2024-03-28T00:15:55.581 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:54.790+0000 7f24ed600700 1 -- 172.21.15.119:0/2021195529 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7f24e402ad60 con 0x7f24f01085a0 2024-03-28T00:15:55.581 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:54.794+0000 7f24ed600700 1 -- 172.21.15.119:0/2021195529 <== mon.0 v2:172.21.15.119:3300/0 8 ==== mgrmap(e 9) v1 ==== 46119+0+0 (secure 0 0 0) 0x7f24e402d3f0 con 0x7f24f01085a0 2024-03-28T00:15:55.581 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:54.795+0000 7f24f7a00700 1 -- 172.21.15.119:0/2021195529 >> [v2:172.21.15.119:6800/2575246519,v1:172.21.15.119:6801/2575246519] conn(0x7f24d8038f70 msgr2=0x7f24d803b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:15:55.581 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:54.795+0000 7f24f7a00700 1 --2- 172.21.15.119:0/2021195529 >> [v2:172.21.15.119:6800/2575246519,v1:172.21.15.119:6801/2575246519] conn(0x7f24d8038f70 0x7f24d803b430 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f24dc006fd0 tx=0x7f24dc006e40 comp rx=0 tx=0).stop 2024-03-28T00:15:55.581 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:54.795+0000 7f24f7a00700 1 -- 172.21.15.119:0/2021195529 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f24f01085a0 msgr2=0x7f24f019ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:15:55.581 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:54.795+0000 7f24f7a00700 1 --2- 172.21.15.119:0/2021195529 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f24f01085a0 0x7f24f019ddb0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f24e40117e0 tx=0x7f24e4011ba0 comp rx=0 tx=0).stop 2024-03-28T00:15:55.581 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:54.796+0000 7f24f7a00700 1 -- 172.21.15.119:0/2021195529 shutdown_connections 2024-03-28T00:15:55.581 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:54.796+0000 7f24f7a00700 1 --2- 172.21.15.119:0/2021195529 >> [v2:172.21.15.119:6800/2575246519,v1:172.21.15.119:6801/2575246519] conn(0x7f24d8038f70 0x7f24d803b430 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:15:55.581 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:54.796+0000 7f24f7a00700 1 --2- 172.21.15.119:0/2021195529 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f24f01085a0 0x7f24f019ddb0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:15:55.581 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:54.796+0000 7f24f7a00700 1 -- 172.21.15.119:0/2021195529 >> 172.21.15.119:0/2021195529 conn(0x7f24f0101ff0 msgr2=0x7f24f0102a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:15:55.581 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:54.796+0000 7f24f7a00700 1 -- 172.21.15.119:0/2021195529 shutdown_connections 2024-03-28T00:15:55.581 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:54.796+0000 7f24f7a00700 1 -- 172.21.15.119:0/2021195529 wait complete. 2024-03-28T00:15:55.847 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:55 smithi119 ceph-mon[31518]: from='client.? 172.21.15.119:0/2021195529' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-03-28T00:15:55.847 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:55 smithi119 ceph-mon[31518]: mgrmap e9: x(active, since 24s) 2024-03-28T00:15:55.848 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 00:15:55 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-x[31752]: 2024-03-28T00:15:55.581+0000 7f5e88026200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-28T00:15:55.848 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 00:15:55 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-x[31752]: 2024-03-28T00:15:55.702+0000 7f5e88026200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-28T00:15:56.655 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 00:15:56 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-x[31752]: 2024-03-28T00:15:56.515+0000 7f5e88026200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-28T00:15:56.980 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:15:56 smithi119 ceph-mon[31518]: from='client.? 172.21.15.119:0/654823337' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-03-28T00:15:56.980 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 00:15:56 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-x[31752]: 2024-03-28T00:15:56.650+0000 7f5e88026200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-28T00:15:56.981 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 00:15:56 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-x[31752]: 2024-03-28T00:15:56.850+0000 7f5e88026200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-28T00:15:56.981 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 00:15:56 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-x[31752]: 2024-03-28T00:15:56.973+0000 7f5e88026200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-28T00:15:57.225 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout { 2024-03-28T00:15:57.225 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-03-28T00:15:57.225 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-28T00:15:57.226 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-03-28T00:15:57.226 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-03-28T00:15:57.226 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout } 2024-03-28T00:15:57.226 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:56.163+0000 7f287ce00700 1 Processor -- start 2024-03-28T00:15:57.226 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:56.164+0000 7f287ce00700 1 -- start start 2024-03-28T00:15:57.226 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:56.164+0000 7f287ce00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f28781085c0 0x7f287810a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:15:57.232 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:56.164+0000 7f287ce00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f287807be20 con 0x7f28781085c0 2024-03-28T00:15:57.232 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:56.164+0000 7f2875e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f28781085c0 0x7f287810a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:15:57.232 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:56.164+0000 7f2875e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f28781085c0 0x7f287810a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:56004/0 (socket says 172.21.15.119:56004) 2024-03-28T00:15:57.232 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:56.164+0000 7f2875e00700 1 -- 172.21.15.119:0/2575718186 learned_addr learned my addr 172.21.15.119:0/2575718186 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:15:57.232 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:56.165+0000 7f2875e00700 1 -- 172.21.15.119:0/2575718186 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f287810aef0 con 0x7f28781085c0 2024-03-28T00:15:57.232 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:56.165+0000 7f2875e00700 1 --2- 172.21.15.119:0/2575718186 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f28781085c0 0x7f287810a9b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f28680098a0 tx=0x7f2868009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=362591ede089b3a3 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:15:57.232 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:56.165+0000 7f2874a00700 1 -- 172.21.15.119:0/2575718186 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2868004160 con 0x7f28781085c0 2024-03-28T00:15:57.232 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:56.165+0000 7f2874a00700 1 -- 172.21.15.119:0/2575718186 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2868004300 con 0x7f28781085c0 2024-03-28T00:15:57.232 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:56.166+0000 7f2874a00700 1 -- 172.21.15.119:0/2575718186 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2868011b20 con 0x7f28781085c0 2024-03-28T00:15:57.232 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:56.166+0000 7f287ce00700 1 -- 172.21.15.119:0/2575718186 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f28781085c0 msgr2=0x7f287810a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:15:57.232 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:56.166+0000 7f287ce00700 1 --2- 172.21.15.119:0/2575718186 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f28781085c0 0x7f287810a9b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f28680098a0 tx=0x7f2868009bb0 comp rx=0 tx=0).stop 2024-03-28T00:15:57.233 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:56.167+0000 7f287ce00700 1 -- 172.21.15.119:0/2575718186 shutdown_connections 2024-03-28T00:15:57.233 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:56.167+0000 7f287ce00700 1 --2- 172.21.15.119:0/2575718186 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f28781085c0 0x7f287810a9b0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:15:57.233 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:56.167+0000 7f287ce00700 1 -- 172.21.15.119:0/2575718186 >> 172.21.15.119:0/2575718186 conn(0x7f2878101ff0 msgr2=0x7f2878104410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:15:57.233 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:56.167+0000 7f287ce00700 1 -- 172.21.15.119:0/2575718186 shutdown_connections 2024-03-28T00:15:57.233 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:56.167+0000 7f287ce00700 1 -- 172.21.15.119:0/2575718186 wait complete. 2024-03-28T00:15:57.233 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:56.168+0000 7f287ce00700 1 Processor -- start 2024-03-28T00:15:57.233 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:56.168+0000 7f287ce00700 1 -- start start 2024-03-28T00:15:57.233 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:56.169+0000 7f287ce00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f28781085c0 0x7f287819baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:15:57.233 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:56.169+0000 7f287ce00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f287810bf50 con 0x7f28781085c0 2024-03-28T00:15:57.233 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:56.169+0000 7f2875e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f28781085c0 0x7f287819baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:15:57.233 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:56.169+0000 7f2875e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f28781085c0 0x7f287819baa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:56020/0 (socket says 172.21.15.119:56020) 2024-03-28T00:15:57.233 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:56.169+0000 7f2875e00700 1 -- 172.21.15.119:0/654823337 learned_addr learned my addr 172.21.15.119:0/654823337 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:15:57.233 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:56.169+0000 7f2875e00700 1 -- 172.21.15.119:0/654823337 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f287819bfe0 con 0x7f28781085c0 2024-03-28T00:15:57.233 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:56.170+0000 7f2875e00700 1 --2- 172.21.15.119:0/654823337 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f28781085c0 0x7f287819baa0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f28680116a0 tx=0x7f2868011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:15:57.234 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:56.170+0000 7f286ea00700 1 -- 172.21.15.119:0/654823337 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2868003a40 con 0x7f28781085c0 2024-03-28T00:15:57.234 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:56.170+0000 7f286ea00700 1 -- 172.21.15.119:0/654823337 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2868003be0 con 0x7f28781085c0 2024-03-28T00:15:57.234 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:56.170+0000 7f286ea00700 1 -- 172.21.15.119:0/654823337 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f286801a750 con 0x7f28781085c0 2024-03-28T00:15:57.234 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:56.170+0000 7f287ce00700 1 -- 172.21.15.119:0/654823337 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f287819c270 con 0x7f28781085c0 2024-03-28T00:15:57.234 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:56.170+0000 7f287ce00700 1 -- 172.21.15.119:0/654823337 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f287819c6d0 con 0x7f28781085c0 2024-03-28T00:15:57.234 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:56.171+0000 7f286ea00700 1 -- 172.21.15.119:0/654823337 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 9) v1 ==== 46119+0+0 (secure 0 0 0) 0x7f286801aa00 con 0x7f28781085c0 2024-03-28T00:15:57.234 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:56.171+0000 7f287ce00700 1 -- 172.21.15.119:0/654823337 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2878069260 con 0x7f28781085c0 2024-03-28T00:15:57.234 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:56.172+0000 7f286ea00700 1 --2- 172.21.15.119:0/654823337 >> [v2:172.21.15.119:6800/2575246519,v1:172.21.15.119:6801/2575246519] conn(0x7f285c038f80 0x7f285c03b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:15:57.234 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:56.172+0000 7f286ea00700 1 -- 172.21.15.119:0/654823337 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f286804d670 con 0x7f28781085c0 2024-03-28T00:15:57.234 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:56.172+0000 7f2875400700 1 -- 172.21.15.119:0/654823337 >> [v2:172.21.15.119:6800/2575246519,v1:172.21.15.119:6801/2575246519] conn(0x7f285c038f80 msgr2=0x7f285c03b440 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6800/2575246519 2024-03-28T00:15:57.234 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:56.172+0000 7f2875400700 1 --2- 172.21.15.119:0/654823337 >> [v2:172.21.15.119:6800/2575246519,v1:172.21.15.119:6801/2575246519] conn(0x7f285c038f80 0x7f285c03b440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-28T00:15:57.234 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:56.177+0000 7f286ea00700 1 -- 172.21.15.119:0/654823337 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f2868011e20 con 0x7f28781085c0 2024-03-28T00:15:57.234 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:56.373+0000 7f2875400700 1 -- 172.21.15.119:0/654823337 >> [v2:172.21.15.119:6800/2575246519,v1:172.21.15.119:6801/2575246519] conn(0x7f285c038f80 msgr2=0x7f285c03b440 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6800/2575246519 2024-03-28T00:15:57.235 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:56.373+0000 7f2875400700 1 --2- 172.21.15.119:0/654823337 >> [v2:172.21.15.119:6800/2575246519,v1:172.21.15.119:6801/2575246519] conn(0x7f285c038f80 0x7f285c03b440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-28T00:15:57.235 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:56.459+0000 7f287ce00700 1 -- 172.21.15.119:0/654823337 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f287819f080 con 0x7f28781085c0 2024-03-28T00:15:57.235 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:56.460+0000 7f286ea00700 1 -- 172.21.15.119:0/654823337 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+88 (secure 0 0 0) 0x7f286801adf0 con 0x7f28781085c0 2024-03-28T00:15:57.235 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:56.463+0000 7f287ce00700 1 -- 172.21.15.119:0/654823337 >> [v2:172.21.15.119:6800/2575246519,v1:172.21.15.119:6801/2575246519] conn(0x7f285c038f80 msgr2=0x7f285c03b440 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-28T00:15:57.235 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:56.463+0000 7f287ce00700 1 --2- 172.21.15.119:0/654823337 >> [v2:172.21.15.119:6800/2575246519,v1:172.21.15.119:6801/2575246519] conn(0x7f285c038f80 0x7f285c03b440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:15:57.235 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:56.463+0000 7f287ce00700 1 -- 172.21.15.119:0/654823337 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f28781085c0 msgr2=0x7f287819baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:15:57.235 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:56.463+0000 7f287ce00700 1 --2- 172.21.15.119:0/654823337 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f28781085c0 0x7f287819baa0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f28680116a0 tx=0x7f2868011ba0 comp rx=0 tx=0).stop 2024-03-28T00:15:57.235 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:56.463+0000 7f287ce00700 1 -- 172.21.15.119:0/654823337 shutdown_connections 2024-03-28T00:15:57.235 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:56.463+0000 7f287ce00700 1 --2- 172.21.15.119:0/654823337 >> [v2:172.21.15.119:6800/2575246519,v1:172.21.15.119:6801/2575246519] conn(0x7f285c038f80 0x7f285c03b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:15:57.237 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:56.463+0000 7f287ce00700 1 --2- 172.21.15.119:0/654823337 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f28781085c0 0x7f287819baa0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:15:57.237 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:56.463+0000 7f287ce00700 1 -- 172.21.15.119:0/654823337 >> 172.21.15.119:0/654823337 conn(0x7f2878101ff0 msgr2=0x7f2878102a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:15:57.237 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:56.463+0000 7f287ce00700 1 -- 172.21.15.119:0/654823337 shutdown_connections 2024-03-28T00:15:57.237 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:56.463+0000 7f287ce00700 1 -- 172.21.15.119:0/654823337 wait complete. 2024-03-28T00:15:57.237 INFO:teuthology.orchestra.run.smithi119.stdout:Waiting for the mgr to restart... 2024-03-28T00:15:57.237 INFO:teuthology.orchestra.run.smithi119.stdout:Waiting for mgr epoch 9... 2024-03-28T00:15:57.376 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 00:15:57 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-x[31752]: 2024-03-28T00:15:57.083+0000 7f5e88026200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-28T00:15:57.655 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 00:15:57 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-x[31752]: 2024-03-28T00:15:57.421+0000 7f5e88026200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-28T00:15:57.655 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 00:15:57 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-x[31752]: 2024-03-28T00:15:57.565+0000 7f5e88026200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-28T00:15:58.033 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 00:15:57 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-x[31752]: 2024-03-28T00:15:57.678+0000 7f5e88026200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-28T00:15:58.033 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 00:15:57 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-x[31752]: 2024-03-28T00:15:57.807+0000 7f5e88026200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-28T00:15:58.033 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 00:15:58 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-x[31752]: 2024-03-28T00:15:58.026+0000 7f5e88026200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-28T00:15:59.655 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 00:15:59 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-x[31752]: 2024-03-28T00:15:59.274+0000 7f5e88026200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-28T00:15:59.655 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 00:15:59 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-x[31752]: 2024-03-28T00:15:59.441+0000 7f5e88026200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-28T00:16:00.155 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 00:15:59 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-x[31752]: 2024-03-28T00:15:59.850+0000 7f5e88026200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-28T00:16:00.156 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 00:15:59 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-x[31752]: 2024-03-28T00:15:59.967+0000 7f5e88026200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-28T00:16:00.655 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 00:16:00 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-x[31752]: 2024-03-28T00:16:00.217+0000 7f5e88026200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-28T00:16:00.655 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 00:16:00 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-x[31752]: 2024-03-28T00:16:00.460+0000 7f5e88026200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-28T00:16:01.155 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 00:16:00 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-x[31752]: 2024-03-28T00:16:00.832+0000 7f5e88026200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-28T00:16:01.477 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 00:16:01 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-x[31752]: 2024-03-28T00:16:01.171+0000 7f5e88026200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-28T00:16:01.905 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 00:16:01 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-x[31752]: 2024-03-28T00:16:01.470+0000 7f5e88026200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-28T00:16:02.905 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 00:16:02 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-x[31752]: 2024-03-28T00:16:02.477+0000 7f5e88026200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-28T00:16:03.586 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 00:16:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-x[31752]: 2024-03-28T00:16:03.170+0000 7f5e88026200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-28T00:16:03.586 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 00:16:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-x[31752]: 2024-03-28T00:16:03.299+0000 7f5e88026200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-28T00:16:03.905 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 00:16:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-x[31752]: 2024-03-28T00:16:03.578+0000 7f5e88026200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-28T00:16:03.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:16:03 smithi119 ceph-mon[31518]: Active manager daemon x restarted 2024-03-28T00:16:03.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:16:03 smithi119 ceph-mon[31518]: Activating manager daemon x 2024-03-28T00:16:03.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:16:03 smithi119 ceph-mon[31518]: osdmap e3: 0 total, 0 up, 0 in 2024-03-28T00:16:03.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:16:03 smithi119 ceph-mon[31518]: mgrmap e10: x(active, starting, since 0.00836143s) 2024-03-28T00:16:03.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:16:03 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-28T00:16:03.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:16:03 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-28T00:16:03.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:16:03 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-28T00:16:03.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:16:03 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-28T00:16:03.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:16:03 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-28T00:16:03.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:16:03 smithi119 ceph-mon[31518]: Manager daemon x is now available 2024-03-28T00:16:04.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:16:04 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:16:04.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:16:04 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:16:04.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:16:04 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-28T00:16:04.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:16:04 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-28T00:16:04.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:16:04 smithi119 ceph-mon[31518]: [28/Mar/2024:00:16:03] ENGINE Bus STARTING 2024-03-28T00:16:04.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:16:04 smithi119 ceph-mon[31518]: mgrmap e11: x(active, since 1.00963s) 2024-03-28T00:16:05.717 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout { 2024-03-28T00:16:05.717 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-03-28T00:16:05.717 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "initialized": true 2024-03-28T00:16:05.717 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout } 2024-03-28T00:16:05.717 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:57.838+0000 7fa0f7200700 1 Processor -- start 2024-03-28T00:16:05.725 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:57.839+0000 7fa0f7200700 1 -- start start 2024-03-28T00:16:05.726 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:57.839+0000 7fa0f7200700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa0f0105db0 0x7fa0f01061d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:16:05.726 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:57.839+0000 7fa0f7200700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0f01067a0 con 0x7fa0f0105db0 2024-03-28T00:16:05.726 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:57.839+0000 7fa0efe00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa0f0105db0 0x7fa0f01061d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:16:05.726 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:57.839+0000 7fa0efe00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa0f0105db0 0x7fa0f01061d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:56032/0 (socket says 172.21.15.119:56032) 2024-03-28T00:16:05.726 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:57.839+0000 7fa0efe00700 1 -- 172.21.15.119:0/159742198 learned_addr learned my addr 172.21.15.119:0/159742198 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:16:05.726 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:57.840+0000 7fa0efe00700 1 -- 172.21.15.119:0/159742198 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0f0106920 con 0x7fa0f0105db0 2024-03-28T00:16:05.726 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:57.840+0000 7fa0efe00700 1 --2- 172.21.15.119:0/159742198 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa0f0105db0 0x7fa0f01061d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa0e00098a0 tx=0x7fa0e0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac25212dc2b89d6b server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:16:05.726 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:57.840+0000 7fa0eea00700 1 -- 172.21.15.119:0/159742198 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa0e0004160 con 0x7fa0f0105db0 2024-03-28T00:16:05.726 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:57.840+0000 7fa0eea00700 1 -- 172.21.15.119:0/159742198 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa0e0004300 con 0x7fa0f0105db0 2024-03-28T00:16:05.726 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:57.840+0000 7fa0eea00700 1 -- 172.21.15.119:0/159742198 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa0e0011b20 con 0x7fa0f0105db0 2024-03-28T00:16:05.726 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:57.841+0000 7fa0f7200700 1 -- 172.21.15.119:0/159742198 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa0f0105db0 msgr2=0x7fa0f01061d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:16:05.726 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:57.841+0000 7fa0f7200700 1 --2- 172.21.15.119:0/159742198 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa0f0105db0 0x7fa0f01061d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa0e00098a0 tx=0x7fa0e0009bb0 comp rx=0 tx=0).stop 2024-03-28T00:16:05.726 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:57.841+0000 7fa0f7200700 1 -- 172.21.15.119:0/159742198 shutdown_connections 2024-03-28T00:16:05.727 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:57.841+0000 7fa0f7200700 1 --2- 172.21.15.119:0/159742198 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa0f0105db0 0x7fa0f01061d0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:16:05.727 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:57.841+0000 7fa0f7200700 1 -- 172.21.15.119:0/159742198 >> 172.21.15.119:0/159742198 conn(0x7fa0f0101710 msgr2=0x7fa0f0103b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:16:05.727 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:57.841+0000 7fa0f7200700 1 -- 172.21.15.119:0/159742198 shutdown_connections 2024-03-28T00:16:05.727 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:57.841+0000 7fa0f7200700 1 -- 172.21.15.119:0/159742198 wait complete. 2024-03-28T00:16:05.727 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:57.842+0000 7fa0f7200700 1 Processor -- start 2024-03-28T00:16:05.727 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:57.842+0000 7fa0f7200700 1 -- start start 2024-03-28T00:16:05.727 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:57.843+0000 7fa0f7200700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa0f0105db0 0x7fa0f01998e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:16:05.727 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:57.843+0000 7fa0f7200700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0f0106c20 con 0x7fa0f0105db0 2024-03-28T00:16:05.727 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:57.843+0000 7fa0efe00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa0f0105db0 0x7fa0f01998e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:16:05.727 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:57.843+0000 7fa0efe00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa0f0105db0 0x7fa0f01998e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:56048/0 (socket says 172.21.15.119:56048) 2024-03-28T00:16:05.727 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:57.843+0000 7fa0efe00700 1 -- 172.21.15.119:0/349922527 learned_addr learned my addr 172.21.15.119:0/349922527 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:16:05.727 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:57.843+0000 7fa0efe00700 1 -- 172.21.15.119:0/349922527 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0f0199e20 con 0x7fa0f0105db0 2024-03-28T00:16:05.727 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:57.844+0000 7fa0efe00700 1 --2- 172.21.15.119:0/349922527 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa0f0105db0 0x7fa0f01998e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa0e0000c00 tx=0x7fa0e0011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:16:05.727 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:57.844+0000 7fa0ecc00700 1 -- 172.21.15.119:0/349922527 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa0e0003b60 con 0x7fa0f0105db0 2024-03-28T00:16:05.728 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:57.844+0000 7fa0ecc00700 1 -- 172.21.15.119:0/349922527 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa0e001a910 con 0x7fa0f0105db0 2024-03-28T00:16:05.728 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:57.844+0000 7fa0f7200700 1 -- 172.21.15.119:0/349922527 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0f019a0b0 con 0x7fa0f0105db0 2024-03-28T00:16:05.728 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:57.844+0000 7fa0ecc00700 1 -- 172.21.15.119:0/349922527 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa0e0024930 con 0x7fa0f0105db0 2024-03-28T00:16:05.728 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:57.844+0000 7fa0f7200700 1 -- 172.21.15.119:0/349922527 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0f019a510 con 0x7fa0f0105db0 2024-03-28T00:16:05.728 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:57.845+0000 7fa0ecc00700 1 -- 172.21.15.119:0/349922527 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 9) v1 ==== 46119+0+0 (secure 0 0 0) 0x7fa0e0022020 con 0x7fa0f0105db0 2024-03-28T00:16:05.728 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:57.846+0000 7fa0ecc00700 1 --2- 172.21.15.119:0/349922527 >> [v2:172.21.15.119:6800/2575246519,v1:172.21.15.119:6801/2575246519] conn(0x7fa0d403d3d0 0x7fa0d403f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:16:05.728 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:57.846+0000 7fa0ecc00700 1 -- 172.21.15.119:0/349922527 --> [v2:172.21.15.119:6800/2575246519,v1:172.21.15.119:6801/2575246519] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa0d403ffa0 con 0x7fa0d403d3d0 2024-03-28T00:16:05.728 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:57.846+0000 7fa0ecc00700 1 -- 172.21.15.119:0/349922527 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fa0e004d170 con 0x7fa0f0105db0 2024-03-28T00:16:05.728 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:57.846+0000 7fa0ef400700 1 -- 172.21.15.119:0/349922527 >> [v2:172.21.15.119:6800/2575246519,v1:172.21.15.119:6801/2575246519] conn(0x7fa0d403d3d0 msgr2=0x7fa0d403f890 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6800/2575246519 2024-03-28T00:16:05.728 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:57.846+0000 7fa0ef400700 1 --2- 172.21.15.119:0/349922527 >> [v2:172.21.15.119:6800/2575246519,v1:172.21.15.119:6801/2575246519] conn(0x7fa0d403d3d0 0x7fa0d403f890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-28T00:16:05.728 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:58.047+0000 7fa0ef400700 1 -- 172.21.15.119:0/349922527 >> [v2:172.21.15.119:6800/2575246519,v1:172.21.15.119:6801/2575246519] conn(0x7fa0d403d3d0 msgr2=0x7fa0d403f890 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6800/2575246519 2024-03-28T00:16:05.728 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:58.047+0000 7fa0ef400700 1 --2- 172.21.15.119:0/349922527 >> [v2:172.21.15.119:6800/2575246519,v1:172.21.15.119:6801/2575246519] conn(0x7fa0d403d3d0 0x7fa0d403f890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-28T00:16:05.728 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:58.448+0000 7fa0ef400700 1 -- 172.21.15.119:0/349922527 >> [v2:172.21.15.119:6800/2575246519,v1:172.21.15.119:6801/2575246519] conn(0x7fa0d403d3d0 msgr2=0x7fa0d403f890 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6800/2575246519 2024-03-28T00:16:05.729 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:58.448+0000 7fa0ef400700 1 --2- 172.21.15.119:0/349922527 >> [v2:172.21.15.119:6800/2575246519,v1:172.21.15.119:6801/2575246519] conn(0x7fa0d403d3d0 0x7fa0d403f890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-28T00:16:05.729 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:59.249+0000 7fa0ef400700 1 -- 172.21.15.119:0/349922527 >> [v2:172.21.15.119:6800/2575246519,v1:172.21.15.119:6801/2575246519] conn(0x7fa0d403d3d0 msgr2=0x7fa0d403f890 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6800/2575246519 2024-03-28T00:16:05.729 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:15:59.249+0000 7fa0ef400700 1 --2- 172.21.15.119:0/349922527 >> [v2:172.21.15.119:6800/2575246519,v1:172.21.15.119:6801/2575246519] conn(0x7fa0d403d3d0 0x7fa0d403f890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-28T00:16:05.729 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:00.851+0000 7fa0ef400700 1 -- 172.21.15.119:0/349922527 >> [v2:172.21.15.119:6800/2575246519,v1:172.21.15.119:6801/2575246519] conn(0x7fa0d403d3d0 msgr2=0x7fa0d403f890 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6800/2575246519 2024-03-28T00:16:05.729 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:00.851+0000 7fa0ef400700 1 --2- 172.21.15.119:0/349922527 >> [v2:172.21.15.119:6800/2575246519,v1:172.21.15.119:6801/2575246519] conn(0x7fa0d403d3d0 0x7fa0d403f890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-03-28T00:16:05.729 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:03.587+0000 7fa0ecc00700 1 -- 172.21.15.119:0/349922527 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mgrmap(e 10) v1 ==== 45886+0+0 (secure 0 0 0) 0x7fa0e002d810 con 0x7fa0f0105db0 2024-03-28T00:16:05.729 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:03.588+0000 7fa0ecc00700 1 -- 172.21.15.119:0/349922527 >> [v2:172.21.15.119:6800/2575246519,v1:172.21.15.119:6801/2575246519] conn(0x7fa0d403d3d0 msgr2=0x7fa0d403f890 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-28T00:16:05.729 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:03.588+0000 7fa0ecc00700 1 --2- 172.21.15.119:0/349922527 >> [v2:172.21.15.119:6800/2575246519,v1:172.21.15.119:6801/2575246519] conn(0x7fa0d403d3d0 0x7fa0d403f890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:16:05.729 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:04.588+0000 7fa0ecc00700 1 -- 172.21.15.119:0/349922527 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mgrmap(e 11) v1 ==== 46013+0+0 (secure 0 0 0) 0x7fa0e004f0e0 con 0x7fa0f0105db0 2024-03-28T00:16:05.729 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:04.589+0000 7fa0ecc00700 1 --2- 172.21.15.119:0/349922527 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fa0d4040ea0 0x7fa0d4043290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:16:05.729 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:04.589+0000 7fa0ecc00700 1 -- 172.21.15.119:0/349922527 --> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa0d403ffa0 con 0x7fa0d4040ea0 2024-03-28T00:16:05.729 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:04.589+0000 7fa0ef400700 1 --2- 172.21.15.119:0/349922527 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fa0d4040ea0 0x7fa0d4043290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:16:05.729 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:04.590+0000 7fa0ef400700 1 --2- 172.21.15.119:0/349922527 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fa0d4040ea0 0x7fa0d4043290 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fa0dc003a10 tx=0x7fa0dc0092b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:16:05.729 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:04.592+0000 7fa0ecc00700 1 -- 172.21.15.119:0/349922527 <== mgr.14152 v2:172.21.15.119:6800/1401937096 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+7774 (secure 0 0 0) 0x7fa0d403ffa0 con 0x7fa0d4040ea0 2024-03-28T00:16:05.729 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:04.599+0000 7fa0e6000700 1 -- 172.21.15.119:0/349922527 --> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7fa0c8002600 con 0x7fa0d4040ea0 2024-03-28T00:16:05.730 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:04.599+0000 7fa0ecc00700 1 -- 172.21.15.119:0/349922527 <== mgr.14152 v2:172.21.15.119:6800/1401937096 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7fa0c8002600 con 0x7fa0d4040ea0 2024-03-28T00:16:05.730 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:04.600+0000 7fa0e6000700 1 -- 172.21.15.119:0/349922527 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fa0d4040ea0 msgr2=0x7fa0d4043290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:16:05.730 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:04.600+0000 7fa0e6000700 1 --2- 172.21.15.119:0/349922527 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fa0d4040ea0 0x7fa0d4043290 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fa0dc003a10 tx=0x7fa0dc0092b0 comp rx=0 tx=0).stop 2024-03-28T00:16:05.730 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:04.600+0000 7fa0e6000700 1 -- 172.21.15.119:0/349922527 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa0f0105db0 msgr2=0x7fa0f01998e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:16:05.730 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:04.600+0000 7fa0e6000700 1 --2- 172.21.15.119:0/349922527 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa0f0105db0 0x7fa0f01998e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa0e0000c00 tx=0x7fa0e0011c10 comp rx=0 tx=0).stop 2024-03-28T00:16:05.730 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:04.600+0000 7fa0e6000700 1 -- 172.21.15.119:0/349922527 shutdown_connections 2024-03-28T00:16:05.730 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:04.600+0000 7fa0e6000700 1 --2- 172.21.15.119:0/349922527 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fa0d4040ea0 0x7fa0d4043290 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:16:05.730 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:04.600+0000 7fa0e6000700 1 --2- 172.21.15.119:0/349922527 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa0f0105db0 0x7fa0f01998e0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:16:05.730 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:04.600+0000 7fa0e6000700 1 --2- 172.21.15.119:0/349922527 >> [v2:172.21.15.119:6800/2575246519,v1:172.21.15.119:6801/2575246519] conn(0x7fa0d403d3d0 0x7fa0d403f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:16:05.730 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:04.600+0000 7fa0e6000700 1 -- 172.21.15.119:0/349922527 >> 172.21.15.119:0/349922527 conn(0x7fa0f0101710 msgr2=0x7fa0f0103a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:16:05.730 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:04.600+0000 7fa0e6000700 1 -- 172.21.15.119:0/349922527 shutdown_connections 2024-03-28T00:16:05.730 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:04.600+0000 7fa0e6000700 1 -- 172.21.15.119:0/349922527 wait complete. 2024-03-28T00:16:05.730 INFO:teuthology.orchestra.run.smithi119.stdout:mgr epoch 9 is available 2024-03-28T00:16:05.731 INFO:teuthology.orchestra.run.smithi119.stdout:Generating a dashboard self-signed certificate... 2024-03-28T00:16:05.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:16:05 smithi119 ceph-mon[31518]: [28/Mar/2024:00:16:03] ENGINE Serving on https://172.21.15.119:7150 2024-03-28T00:16:05.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:16:05 smithi119 ceph-mon[31518]: [28/Mar/2024:00:16:04] ENGINE Serving on http://172.21.15.119:8765 2024-03-28T00:16:05.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:16:05 smithi119 ceph-mon[31518]: [28/Mar/2024:00:16:04] ENGINE Bus STARTED 2024-03-28T00:16:05.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:16:05 smithi119 ceph-mon[31518]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-03-28T00:16:05.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:16:05 smithi119 ceph-mon[31518]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-03-28T00:16:06.941 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:16:06 smithi119 ceph-mon[31518]: mgrmap e12: x(active, since 2s) 2024-03-28T00:16:06.941 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:16:06 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:16:07.603 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-03-28T00:16:07.603 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:06.607+0000 7faa7b800700 1 Processor -- start 2024-03-28T00:16:07.603 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:06.607+0000 7faa7b800700 1 -- start start 2024-03-28T00:16:07.603 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:06.608+0000 7faa7b800700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa7407de20 0x7faa7407c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:16:07.603 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:06.608+0000 7faa7b800700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa7407c8a0 con 0x7faa7407de20 2024-03-28T00:16:07.603 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:06.608+0000 7faa78e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa7407de20 0x7faa7407c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:16:07.604 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:06.608+0000 7faa78e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa7407de20 0x7faa7407c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34162/0 (socket says 172.21.15.119:34162) 2024-03-28T00:16:07.604 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:06.608+0000 7faa78e00700 1 -- 172.21.15.119:0/4207711707 learned_addr learned my addr 172.21.15.119:0/4207711707 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:16:07.604 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:06.609+0000 7faa78e00700 1 -- 172.21.15.119:0/4207711707 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa7407ca20 con 0x7faa7407de20 2024-03-28T00:16:07.604 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:06.609+0000 7faa78e00700 1 --2- 172.21.15.119:0/4207711707 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa7407de20 0x7faa7407c2d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7faa680098a0 tx=0x7faa68009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3fd38dabb4d982f server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:16:07.604 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:06.609+0000 7faa73400700 1 -- 172.21.15.119:0/4207711707 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7faa68004160 con 0x7faa7407de20 2024-03-28T00:16:07.604 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:06.609+0000 7faa73400700 1 -- 172.21.15.119:0/4207711707 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7faa68004300 con 0x7faa7407de20 2024-03-28T00:16:07.604 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:06.609+0000 7faa73400700 1 -- 172.21.15.119:0/4207711707 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7faa68011b20 con 0x7faa7407de20 2024-03-28T00:16:07.604 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:06.610+0000 7faa7b800700 1 -- 172.21.15.119:0/4207711707 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa7407de20 msgr2=0x7faa7407c2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:16:07.604 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:06.610+0000 7faa7b800700 1 --2- 172.21.15.119:0/4207711707 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa7407de20 0x7faa7407c2d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7faa680098a0 tx=0x7faa68009bb0 comp rx=0 tx=0).stop 2024-03-28T00:16:07.604 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:06.610+0000 7faa7b800700 1 -- 172.21.15.119:0/4207711707 shutdown_connections 2024-03-28T00:16:07.604 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:06.610+0000 7faa7b800700 1 --2- 172.21.15.119:0/4207711707 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa7407de20 0x7faa7407c2d0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:16:07.604 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:06.610+0000 7faa7b800700 1 -- 172.21.15.119:0/4207711707 >> 172.21.15.119:0/4207711707 conn(0x7faa74103140 msgr2=0x7faa74105560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:16:07.604 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:06.610+0000 7faa7b800700 1 -- 172.21.15.119:0/4207711707 shutdown_connections 2024-03-28T00:16:07.604 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:06.611+0000 7faa7b800700 1 -- 172.21.15.119:0/4207711707 wait complete. 2024-03-28T00:16:07.605 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:06.612+0000 7faa7b800700 1 Processor -- start 2024-03-28T00:16:07.605 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:06.612+0000 7faa7b800700 1 -- start start 2024-03-28T00:16:07.605 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:06.612+0000 7faa7b800700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa7407de20 0x7faa7419dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:16:07.605 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:06.612+0000 7faa7b800700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa7410e120 con 0x7faa7407de20 2024-03-28T00:16:07.605 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:06.612+0000 7faa78e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa7407de20 0x7faa7419dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:16:07.605 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:06.613+0000 7faa78e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa7407de20 0x7faa7419dc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34172/0 (socket says 172.21.15.119:34172) 2024-03-28T00:16:07.605 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:06.613+0000 7faa78e00700 1 -- 172.21.15.119:0/2252982990 learned_addr learned my addr 172.21.15.119:0/2252982990 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:16:07.605 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:06.613+0000 7faa78e00700 1 -- 172.21.15.119:0/2252982990 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa7419e1d0 con 0x7faa7407de20 2024-03-28T00:16:07.605 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:06.613+0000 7faa78e00700 1 --2- 172.21.15.119:0/2252982990 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa7407de20 0x7faa7419dc90 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7faa680116a0 tx=0x7faa68011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:16:07.605 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:06.613+0000 7faa71600700 1 -- 172.21.15.119:0/2252982990 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7faa68003a40 con 0x7faa7407de20 2024-03-28T00:16:07.605 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:06.614+0000 7faa71600700 1 -- 172.21.15.119:0/2252982990 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7faa68003be0 con 0x7faa7407de20 2024-03-28T00:16:07.605 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:06.614+0000 7faa7b800700 1 -- 172.21.15.119:0/2252982990 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa7419e460 con 0x7faa7407de20 2024-03-28T00:16:07.605 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:06.614+0000 7faa71600700 1 -- 172.21.15.119:0/2252982990 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7faa6801a750 con 0x7faa7407de20 2024-03-28T00:16:07.605 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:06.614+0000 7faa7b800700 1 -- 172.21.15.119:0/2252982990 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa7419e8c0 con 0x7faa7407de20 2024-03-28T00:16:07.606 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:06.615+0000 7faa71600700 1 -- 172.21.15.119:0/2252982990 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 12) v1 ==== 46119+0+0 (secure 0 0 0) 0x7faa68024aa0 con 0x7faa7407de20 2024-03-28T00:16:07.606 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:06.615+0000 7faa7b800700 1 -- 172.21.15.119:0/2252982990 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa74069260 con 0x7faa7407de20 2024-03-28T00:16:07.606 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:06.616+0000 7faa71600700 1 --2- 172.21.15.119:0/2252982990 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7faa5c038f20 0x7faa5c03b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:16:07.606 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:06.616+0000 7faa71600700 1 -- 172.21.15.119:0/2252982990 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7faa6804d650 con 0x7faa7407de20 2024-03-28T00:16:07.606 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:06.616+0000 7faa73e00700 1 --2- 172.21.15.119:0/2252982990 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7faa5c038f20 0x7faa5c03b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:16:07.606 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:06.617+0000 7faa73e00700 1 --2- 172.21.15.119:0/2252982990 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7faa5c038f20 0x7faa5c03b3e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7faa60006fd0 tx=0x7faa60006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:16:07.606 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:06.625+0000 7faa71600700 1 -- 172.21.15.119:0/2252982990 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7faa68011e20 con 0x7faa7407de20 2024-03-28T00:16:07.606 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:06.855+0000 7faa7b800700 1 -- 172.21.15.119:0/2252982990 --> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7faa741042f0 con 0x7faa5c038f20 2024-03-28T00:16:07.606 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:06.934+0000 7faa71600700 1 -- 172.21.15.119:0/2252982990 <== mgr.14152 v2:172.21.15.119:6800/1401937096 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7faa741042f0 con 0x7faa5c038f20 2024-03-28T00:16:07.606 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:06.938+0000 7faa7b800700 1 -- 172.21.15.119:0/2252982990 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7faa5c038f20 msgr2=0x7faa5c03b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:16:07.606 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:06.938+0000 7faa7b800700 1 --2- 172.21.15.119:0/2252982990 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7faa5c038f20 0x7faa5c03b3e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7faa60006fd0 tx=0x7faa60006e40 comp rx=0 tx=0).stop 2024-03-28T00:16:07.606 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:06.938+0000 7faa7b800700 1 -- 172.21.15.119:0/2252982990 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa7407de20 msgr2=0x7faa7419dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:16:07.606 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:06.938+0000 7faa7b800700 1 --2- 172.21.15.119:0/2252982990 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa7407de20 0x7faa7419dc90 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7faa680116a0 tx=0x7faa68011ba0 comp rx=0 tx=0).stop 2024-03-28T00:16:07.607 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:06.938+0000 7faa7b800700 1 -- 172.21.15.119:0/2252982990 shutdown_connections 2024-03-28T00:16:07.607 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:06.938+0000 7faa7b800700 1 --2- 172.21.15.119:0/2252982990 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7faa5c038f20 0x7faa5c03b3e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:16:07.607 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:06.938+0000 7faa7b800700 1 --2- 172.21.15.119:0/2252982990 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa7407de20 0x7faa7419dc90 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:16:07.607 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:06.938+0000 7faa7b800700 1 -- 172.21.15.119:0/2252982990 >> 172.21.15.119:0/2252982990 conn(0x7faa74103140 msgr2=0x7faa74103be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:16:07.607 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:06.938+0000 7faa7b800700 1 -- 172.21.15.119:0/2252982990 shutdown_connections 2024-03-28T00:16:07.607 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:06.938+0000 7faa7b800700 1 -- 172.21.15.119:0/2252982990 wait complete. 2024-03-28T00:16:07.607 INFO:teuthology.orchestra.run.smithi119.stdout:Creating initial admin user... 2024-03-28T00:16:08.206 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:16:07 smithi119 ceph-mon[31518]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T00:16:08.206 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:16:07 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:16:08.206 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:16:07 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:16:09.433 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$L/jEQJJwOMumMlnaaX52auaUWR73ooU8eYd.B87EImp/qHNDlCDDK", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1711584968, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-03-28T00:16:09.433 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:08.199+0000 7f2ef1200700 1 Processor -- start 2024-03-28T00:16:09.433 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:08.199+0000 7f2ef1200700 1 -- start start 2024-03-28T00:16:09.434 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:08.199+0000 7f2ef1200700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2eec1083b0 0x7f2eec10a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:16:09.434 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:08.199+0000 7f2ef1200700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2eec07be20 con 0x7f2eec1083b0 2024-03-28T00:16:09.434 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:08.200+0000 7f2eeb400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2eec1083b0 0x7f2eec10a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:16:09.434 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:08.200+0000 7f2eeb400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2eec1083b0 0x7f2eec10a7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34180/0 (socket says 172.21.15.119:34180) 2024-03-28T00:16:09.434 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:08.200+0000 7f2eeb400700 1 -- 172.21.15.119:0/3028116067 learned_addr learned my addr 172.21.15.119:0/3028116067 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:16:09.434 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:08.201+0000 7f2eeb400700 1 -- 172.21.15.119:0/3028116067 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2eec1061a0 con 0x7f2eec1083b0 2024-03-28T00:16:09.434 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:08.201+0000 7f2eeb400700 1 --2- 172.21.15.119:0/3028116067 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2eec1083b0 0x7f2eec10a7a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2edc0098a0 tx=0x7f2edc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a9f7608cb801d28 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:16:09.434 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:08.201+0000 7f2eea000700 1 -- 172.21.15.119:0/3028116067 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2edc004160 con 0x7f2eec1083b0 2024-03-28T00:16:09.434 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:08.202+0000 7f2eea000700 1 -- 172.21.15.119:0/3028116067 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2edc004300 con 0x7f2eec1083b0 2024-03-28T00:16:09.434 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:08.202+0000 7f2eea000700 1 -- 172.21.15.119:0/3028116067 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2edc011b20 con 0x7f2eec1083b0 2024-03-28T00:16:09.434 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:08.202+0000 7f2ef1200700 1 -- 172.21.15.119:0/3028116067 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2eec1083b0 msgr2=0x7f2eec10a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:16:09.434 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:08.202+0000 7f2ef1200700 1 --2- 172.21.15.119:0/3028116067 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2eec1083b0 0x7f2eec10a7a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2edc0098a0 tx=0x7f2edc009bb0 comp rx=0 tx=0).stop 2024-03-28T00:16:09.434 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:08.203+0000 7f2ef1200700 1 -- 172.21.15.119:0/3028116067 shutdown_connections 2024-03-28T00:16:09.434 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:08.203+0000 7f2ef1200700 1 --2- 172.21.15.119:0/3028116067 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2eec1083b0 0x7f2eec10a7a0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:16:09.435 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:08.203+0000 7f2ef1200700 1 -- 172.21.15.119:0/3028116067 >> 172.21.15.119:0/3028116067 conn(0x7f2eec101dc0 msgr2=0x7f2eec1041e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:16:09.435 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:08.203+0000 7f2ef1200700 1 -- 172.21.15.119:0/3028116067 shutdown_connections 2024-03-28T00:16:09.435 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:08.203+0000 7f2ef1200700 1 -- 172.21.15.119:0/3028116067 wait complete. 2024-03-28T00:16:09.435 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:08.204+0000 7f2ef1200700 1 Processor -- start 2024-03-28T00:16:09.435 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:08.204+0000 7f2ef1200700 1 -- start start 2024-03-28T00:16:09.435 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:08.205+0000 7f2ef1200700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2eec1083b0 0x7f2eec19da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:16:09.435 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:08.205+0000 7f2ef1200700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2eec1072c0 con 0x7f2eec1083b0 2024-03-28T00:16:09.435 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:08.205+0000 7f2eeb400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2eec1083b0 0x7f2eec19da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:16:09.435 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:08.205+0000 7f2eeb400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2eec1083b0 0x7f2eec19da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34196/0 (socket says 172.21.15.119:34196) 2024-03-28T00:16:09.435 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:08.205+0000 7f2eeb400700 1 -- 172.21.15.119:0/295994721 learned_addr learned my addr 172.21.15.119:0/295994721 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:16:09.435 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:08.206+0000 7f2eeb400700 1 -- 172.21.15.119:0/295994721 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2eec19df70 con 0x7f2eec1083b0 2024-03-28T00:16:09.435 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:08.206+0000 7f2eeb400700 1 --2- 172.21.15.119:0/295994721 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2eec1083b0 0x7f2eec19da30 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2edc0117e0 tx=0x7f2edc011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:16:09.435 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:08.206+0000 7f2ee3e00700 1 -- 172.21.15.119:0/295994721 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2edc003ba0 con 0x7f2eec1083b0 2024-03-28T00:16:09.435 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:08.206+0000 7f2ee3e00700 1 -- 172.21.15.119:0/295994721 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2edc01a910 con 0x7f2eec1083b0 2024-03-28T00:16:09.436 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:08.206+0000 7f2ee3e00700 1 -- 172.21.15.119:0/295994721 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2edc0248f0 con 0x7f2eec1083b0 2024-03-28T00:16:09.436 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:08.206+0000 7f2ef1200700 1 -- 172.21.15.119:0/295994721 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2eec19e200 con 0x7f2eec1083b0 2024-03-28T00:16:09.436 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:08.206+0000 7f2ef1200700 1 -- 172.21.15.119:0/295994721 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2eec19e660 con 0x7f2eec1083b0 2024-03-28T00:16:09.436 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:08.208+0000 7f2ef1200700 1 -- 172.21.15.119:0/295994721 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2eec069260 con 0x7f2eec1083b0 2024-03-28T00:16:09.436 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:08.208+0000 7f2ee3e00700 1 -- 172.21.15.119:0/295994721 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 12) v1 ==== 46119+0+0 (secure 0 0 0) 0x7f2edc022020 con 0x7f2eec1083b0 2024-03-28T00:16:09.436 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:08.208+0000 7f2ee3e00700 1 --2- 172.21.15.119:0/295994721 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f2ed0038fd0 0x7f2ed003b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:16:09.436 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:08.208+0000 7f2ee3e00700 1 -- 172.21.15.119:0/295994721 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f2edc04d4b0 con 0x7f2eec1083b0 2024-03-28T00:16:09.436 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:08.209+0000 7f2eeaa00700 1 --2- 172.21.15.119:0/295994721 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f2ed0038fd0 0x7f2ed003b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:16:09.436 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:08.209+0000 7f2eeaa00700 1 --2- 172.21.15.119:0/295994721 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f2ed0038fd0 0x7f2ed003b490 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2ed8006fd0 tx=0x7f2ed8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:16:09.436 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:08.213+0000 7f2ee3e00700 1 -- 172.21.15.119:0/295994721 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f2edc00b3e0 con 0x7f2eec1083b0 2024-03-28T00:16:09.438 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:08.478+0000 7f2ef1200700 1 -- 172.21.15.119:0/295994721 --> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f2eec1975b0 con 0x7f2ed0038fd0 2024-03-28T00:16:09.438 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:08.710+0000 7f2ee3e00700 1 -- 172.21.15.119:0/295994721 <== mgr.14152 v2:172.21.15.119:6800/1401937096 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7f2eec1975b0 con 0x7f2ed0038fd0 2024-03-28T00:16:09.438 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:08.713+0000 7f2ef1200700 1 -- 172.21.15.119:0/295994721 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f2ed0038fd0 msgr2=0x7f2ed003b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:16:09.438 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:08.713+0000 7f2ef1200700 1 --2- 172.21.15.119:0/295994721 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f2ed0038fd0 0x7f2ed003b490 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2ed8006fd0 tx=0x7f2ed8006e40 comp rx=0 tx=0).stop 2024-03-28T00:16:09.438 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:08.713+0000 7f2ef1200700 1 -- 172.21.15.119:0/295994721 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2eec1083b0 msgr2=0x7f2eec19da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:16:09.438 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:08.713+0000 7f2ef1200700 1 --2- 172.21.15.119:0/295994721 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2eec1083b0 0x7f2eec19da30 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2edc0117e0 tx=0x7f2edc011ba0 comp rx=0 tx=0).stop 2024-03-28T00:16:09.438 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:08.713+0000 7f2ef1200700 1 -- 172.21.15.119:0/295994721 shutdown_connections 2024-03-28T00:16:09.438 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:08.713+0000 7f2ef1200700 1 --2- 172.21.15.119:0/295994721 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f2ed0038fd0 0x7f2ed003b490 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:16:09.438 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:08.713+0000 7f2ef1200700 1 --2- 172.21.15.119:0/295994721 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2eec1083b0 0x7f2eec19da30 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:16:09.438 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:08.713+0000 7f2ef1200700 1 -- 172.21.15.119:0/295994721 >> 172.21.15.119:0/295994721 conn(0x7f2eec101dc0 msgr2=0x7f2eec102900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:16:09.439 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:08.713+0000 7f2ef1200700 1 -- 172.21.15.119:0/295994721 shutdown_connections 2024-03-28T00:16:09.439 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:08.713+0000 7f2ef1200700 1 -- 172.21.15.119:0/295994721 wait complete. 2024-03-28T00:16:09.439 INFO:teuthology.orchestra.run.smithi119.stdout:Fetching dashboard port number... 2024-03-28T00:16:09.594 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:16:09 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:16:09.594 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:16:09 smithi119 ceph-mon[31518]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T00:16:09.594 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:16:09 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:16:11.263 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout 8443 2024-03-28T00:16:11.263 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:10.244+0000 7f607e400700 1 Processor -- start 2024-03-28T00:16:11.263 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:10.245+0000 7f607e400700 1 -- start start 2024-03-28T00:16:11.264 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:10.245+0000 7f607e400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f607807de10 0x7f607807c270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:16:11.264 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:10.245+0000 7f607e400700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f607807c840 con 0x7f607807de10 2024-03-28T00:16:11.264 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:10.246+0000 7f6077400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f607807de10 0x7f607807c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:16:11.264 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:10.246+0000 7f6077400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f607807de10 0x7f607807c270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34208/0 (socket says 172.21.15.119:34208) 2024-03-28T00:16:11.264 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:10.246+0000 7f6077400700 1 -- 172.21.15.119:0/4221492072 learned_addr learned my addr 172.21.15.119:0/4221492072 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:16:11.264 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:10.246+0000 7f6077400700 1 -- 172.21.15.119:0/4221492072 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f607807c9c0 con 0x7f607807de10 2024-03-28T00:16:11.264 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:10.246+0000 7f6077400700 1 --2- 172.21.15.119:0/4221492072 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f607807de10 0x7f607807c270 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f60680098a0 tx=0x7f6068009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=295dfea30e79b1d9 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:16:11.264 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:10.247+0000 7f6076000700 1 -- 172.21.15.119:0/4221492072 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6068004160 con 0x7f607807de10 2024-03-28T00:16:11.264 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:10.247+0000 7f6076000700 1 -- 172.21.15.119:0/4221492072 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6068004300 con 0x7f607807de10 2024-03-28T00:16:11.264 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:10.247+0000 7f6076000700 1 -- 172.21.15.119:0/4221492072 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6068011b20 con 0x7f607807de10 2024-03-28T00:16:11.264 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:10.248+0000 7f607e400700 1 -- 172.21.15.119:0/4221492072 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f607807de10 msgr2=0x7f607807c270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:16:11.264 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:10.248+0000 7f607e400700 1 --2- 172.21.15.119:0/4221492072 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f607807de10 0x7f607807c270 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f60680098a0 tx=0x7f6068009bb0 comp rx=0 tx=0).stop 2024-03-28T00:16:11.264 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:10.248+0000 7f607e400700 1 -- 172.21.15.119:0/4221492072 shutdown_connections 2024-03-28T00:16:11.265 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:10.248+0000 7f607e400700 1 --2- 172.21.15.119:0/4221492072 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f607807de10 0x7f607807c270 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:16:11.265 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:10.248+0000 7f607e400700 1 -- 172.21.15.119:0/4221492072 >> 172.21.15.119:0/4221492072 conn(0x7f60781030f0 msgr2=0x7f6078105510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:16:11.265 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:10.248+0000 7f607e400700 1 -- 172.21.15.119:0/4221492072 shutdown_connections 2024-03-28T00:16:11.265 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:10.248+0000 7f607e400700 1 -- 172.21.15.119:0/4221492072 wait complete. 2024-03-28T00:16:11.265 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:10.249+0000 7f607e400700 1 Processor -- start 2024-03-28T00:16:11.265 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:10.249+0000 7f607e400700 1 -- start start 2024-03-28T00:16:11.265 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:10.249+0000 7f607e400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f607807de10 0x7f607819ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:16:11.265 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:10.249+0000 7f607e400700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f607810e170 con 0x7f607807de10 2024-03-28T00:16:11.265 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:10.250+0000 7f6077400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f607807de10 0x7f607819ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:16:11.265 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:10.250+0000 7f6077400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f607807de10 0x7f607819ddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34210/0 (socket says 172.21.15.119:34210) 2024-03-28T00:16:11.265 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:10.250+0000 7f6077400700 1 -- 172.21.15.119:0/3595030632 learned_addr learned my addr 172.21.15.119:0/3595030632 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:16:11.265 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:10.250+0000 7f6077400700 1 -- 172.21.15.119:0/3595030632 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f607819e330 con 0x7f607807de10 2024-03-28T00:16:11.265 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:10.250+0000 7f6077400700 1 --2- 172.21.15.119:0/3595030632 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f607807de10 0x7f607819ddf0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f60680117e0 tx=0x7f6068011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:16:11.265 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:10.251+0000 7f606fe00700 1 -- 172.21.15.119:0/3595030632 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6068003ba0 con 0x7f607807de10 2024-03-28T00:16:11.265 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:10.251+0000 7f606fe00700 1 -- 172.21.15.119:0/3595030632 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f606801a910 con 0x7f607807de10 2024-03-28T00:16:11.266 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:10.251+0000 7f606fe00700 1 -- 172.21.15.119:0/3595030632 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f60680248f0 con 0x7f607807de10 2024-03-28T00:16:11.266 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:10.251+0000 7f607e400700 1 -- 172.21.15.119:0/3595030632 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f607819e5c0 con 0x7f607807de10 2024-03-28T00:16:11.266 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:10.251+0000 7f607e400700 1 -- 172.21.15.119:0/3595030632 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f607819efa0 con 0x7f607807de10 2024-03-28T00:16:11.266 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:10.252+0000 7f606fe00700 1 -- 172.21.15.119:0/3595030632 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 12) v1 ==== 46119+0+0 (secure 0 0 0) 0x7f6068022020 con 0x7f607807de10 2024-03-28T00:16:11.266 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:10.252+0000 7f607e400700 1 -- 172.21.15.119:0/3595030632 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60780637c0 con 0x7f607807de10 2024-03-28T00:16:11.266 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:10.253+0000 7f606fe00700 1 --2- 172.21.15.119:0/3595030632 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f605c038fd0 0x7f605c03b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:16:11.266 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:10.253+0000 7f606fe00700 1 -- 172.21.15.119:0/3595030632 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f606804d460 con 0x7f607807de10 2024-03-28T00:16:11.266 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:10.253+0000 7f6076a00700 1 --2- 172.21.15.119:0/3595030632 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f605c038fd0 0x7f605c03b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:16:11.266 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:10.254+0000 7f6076a00700 1 --2- 172.21.15.119:0/3595030632 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f605c038fd0 0x7f605c03b490 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f6064006fd0 tx=0x7f6064006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:16:11.266 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:10.258+0000 7f606fe00700 1 -- 172.21.15.119:0/3595030632 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f606801ab70 con 0x7f607807de10 2024-03-28T00:16:11.266 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:10.320+0000 7f606fe00700 1 -- 172.21.15.119:0/3595030632 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mgrmap(e 13) v1 ==== 46163+0+0 (secure 0 0 0) 0x7f606802d3f0 con 0x7f607807de10 2024-03-28T00:16:11.266 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:10.483+0000 7f607e400700 1 -- 172.21.15.119:0/3595030632 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7f607819f490 con 0x7f607807de10 2024-03-28T00:16:11.266 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:10.484+0000 7f606fe00700 1 -- 172.21.15.119:0/3595030632 <== mon.0 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7f6068019700 con 0x7f607807de10 2024-03-28T00:16:11.267 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:10.487+0000 7f607e400700 1 -- 172.21.15.119:0/3595030632 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f605c038fd0 msgr2=0x7f605c03b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:16:11.267 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:10.487+0000 7f607e400700 1 --2- 172.21.15.119:0/3595030632 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f605c038fd0 0x7f605c03b490 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f6064006fd0 tx=0x7f6064006e40 comp rx=0 tx=0).stop 2024-03-28T00:16:11.267 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:10.487+0000 7f607e400700 1 -- 172.21.15.119:0/3595030632 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f607807de10 msgr2=0x7f607819ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:16:11.267 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:10.487+0000 7f607e400700 1 --2- 172.21.15.119:0/3595030632 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f607807de10 0x7f607819ddf0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f60680117e0 tx=0x7f6068011ba0 comp rx=0 tx=0).stop 2024-03-28T00:16:11.267 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:10.488+0000 7f607e400700 1 -- 172.21.15.119:0/3595030632 shutdown_connections 2024-03-28T00:16:11.267 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:10.488+0000 7f607e400700 1 --2- 172.21.15.119:0/3595030632 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f605c038fd0 0x7f605c03b490 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:16:11.267 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:10.488+0000 7f607e400700 1 --2- 172.21.15.119:0/3595030632 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f607807de10 0x7f607819ddf0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:16:11.267 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:10.488+0000 7f607e400700 1 -- 172.21.15.119:0/3595030632 >> 172.21.15.119:0/3595030632 conn(0x7f60781030f0 msgr2=0x7f6078103b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:16:11.267 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:10.488+0000 7f607e400700 1 -- 172.21.15.119:0/3595030632 shutdown_connections 2024-03-28T00:16:11.267 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:10.488+0000 7f607e400700 1 -- 172.21.15.119:0/3595030632 wait complete. 2024-03-28T00:16:11.287 INFO:teuthology.orchestra.run.smithi119.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-28T00:16:11.287 INFO:teuthology.orchestra.run.smithi119.stdout:systemctl: stdout disabled 2024-03-28T00:16:11.305 INFO:teuthology.orchestra.run.smithi119.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-28T00:16:11.305 INFO:teuthology.orchestra.run.smithi119.stdout:systemctl: stdout inactive 2024-03-28T00:16:11.305 INFO:teuthology.orchestra.run.smithi119.stdout:firewalld.service is not enabled 2024-03-28T00:16:11.305 INFO:teuthology.orchestra.run.smithi119.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-03-28T00:16:11.308 INFO:teuthology.orchestra.run.smithi119.stdout:Ceph Dashboard is now available at: 2024-03-28T00:16:11.308 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-28T00:16:11.308 INFO:teuthology.orchestra.run.smithi119.stdout: URL: https://smithi119.front.sepia.ceph.com:8443/ 2024-03-28T00:16:11.308 INFO:teuthology.orchestra.run.smithi119.stdout: User: admin 2024-03-28T00:16:11.308 INFO:teuthology.orchestra.run.smithi119.stdout: Password: jl7c7rdqtq 2024-03-28T00:16:11.308 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-28T00:16:11.309 INFO:teuthology.orchestra.run.smithi119.stdout:Saving cluster configuration to /var/lib/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/config directory 2024-03-28T00:16:11.310 INFO:teuthology.orchestra.run.smithi119.stdout:Enabling autotune for osd_memory_target 2024-03-28T00:16:11.513 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:16:11 smithi119 ceph-mon[31518]: mgrmap e13: x(active, since 6s) 2024-03-28T00:16:11.513 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:16:11 smithi119 ceph-mon[31518]: from='client.? 172.21.15.119:0/3595030632' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-03-28T00:16:13.205 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:12.247+0000 7f9628a00700 1 Processor -- start 2024-03-28T00:16:13.205 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:12.247+0000 7f9628a00700 1 -- start start 2024-03-28T00:16:13.206 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:12.247+0000 7f9628a00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f962410ec80 0x7f9624111070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:16:13.206 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:12.247+0000 7f9628a00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9624082380 con 0x7f962410ec80 2024-03-28T00:16:13.206 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:12.248+0000 7f9621e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f962410ec80 0x7f9624111070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:16:13.206 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:12.248+0000 7f9621e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f962410ec80 0x7f9624111070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34220/0 (socket says 172.21.15.119:34220) 2024-03-28T00:16:13.206 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:12.248+0000 7f9621e00700 1 -- 172.21.15.119:0/618974314 learned_addr learned my addr 172.21.15.119:0/618974314 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:16:13.206 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:12.248+0000 7f9621e00700 1 -- 172.21.15.119:0/618974314 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96241115b0 con 0x7f962410ec80 2024-03-28T00:16:13.206 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:12.249+0000 7f9621e00700 1 --2- 172.21.15.119:0/618974314 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f962410ec80 0x7f9624111070 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f96140098a0 tx=0x7f9614009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe670869431a4216 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:16:13.206 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:12.249+0000 7f9620a00700 1 -- 172.21.15.119:0/618974314 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9614004160 con 0x7f962410ec80 2024-03-28T00:16:13.206 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:12.249+0000 7f9620a00700 1 -- 172.21.15.119:0/618974314 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9614004300 con 0x7f962410ec80 2024-03-28T00:16:13.206 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:12.249+0000 7f9620a00700 1 -- 172.21.15.119:0/618974314 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9614011b20 con 0x7f962410ec80 2024-03-28T00:16:13.206 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:12.250+0000 7f9628a00700 1 -- 172.21.15.119:0/618974314 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f962410ec80 msgr2=0x7f9624111070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:16:13.206 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:12.250+0000 7f9628a00700 1 --2- 172.21.15.119:0/618974314 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f962410ec80 0x7f9624111070 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f96140098a0 tx=0x7f9614009bb0 comp rx=0 tx=0).stop 2024-03-28T00:16:13.206 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:12.250+0000 7f9628a00700 1 -- 172.21.15.119:0/618974314 shutdown_connections 2024-03-28T00:16:13.207 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:12.250+0000 7f9628a00700 1 --2- 172.21.15.119:0/618974314 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f962410ec80 0x7f9624111070 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:16:13.207 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:12.250+0000 7f9628a00700 1 -- 172.21.15.119:0/618974314 >> 172.21.15.119:0/618974314 conn(0x7f962407d100 msgr2=0x7f962407f520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:16:13.207 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:12.250+0000 7f9628a00700 1 -- 172.21.15.119:0/618974314 shutdown_connections 2024-03-28T00:16:13.207 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:12.250+0000 7f9628a00700 1 -- 172.21.15.119:0/618974314 wait complete. 2024-03-28T00:16:13.207 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:12.251+0000 7f9628a00700 1 Processor -- start 2024-03-28T00:16:13.207 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:12.251+0000 7f9628a00700 1 -- start start 2024-03-28T00:16:13.207 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:12.251+0000 7f9628a00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f962410ec80 0x7f96241a2100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:16:13.207 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:12.252+0000 7f9628a00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9624112550 con 0x7f962410ec80 2024-03-28T00:16:13.207 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:12.252+0000 7f9621e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f962410ec80 0x7f96241a2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:16:13.207 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:12.252+0000 7f9621e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f962410ec80 0x7f96241a2100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34236/0 (socket says 172.21.15.119:34236) 2024-03-28T00:16:13.207 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:12.252+0000 7f9621e00700 1 -- 172.21.15.119:0/3951651789 learned_addr learned my addr 172.21.15.119:0/3951651789 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:16:13.207 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:12.252+0000 7f9621e00700 1 -- 172.21.15.119:0/3951651789 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96241a2640 con 0x7f962410ec80 2024-03-28T00:16:13.207 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:12.253+0000 7f9621e00700 1 --2- 172.21.15.119:0/3951651789 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f962410ec80 0x7f96241a2100 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9614011b60 tx=0x7f9614011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:16:13.208 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:12.253+0000 7f961aa00700 1 -- 172.21.15.119:0/3951651789 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9614003b60 con 0x7f962410ec80 2024-03-28T00:16:13.208 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:12.253+0000 7f961aa00700 1 -- 172.21.15.119:0/3951651789 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f961401a910 con 0x7f962410ec80 2024-03-28T00:16:13.208 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:12.253+0000 7f9628a00700 1 -- 172.21.15.119:0/3951651789 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96241a28d0 con 0x7f962410ec80 2024-03-28T00:16:13.208 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:12.253+0000 7f961aa00700 1 -- 172.21.15.119:0/3951651789 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9614024930 con 0x7f962410ec80 2024-03-28T00:16:13.208 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:12.253+0000 7f9628a00700 1 -- 172.21.15.119:0/3951651789 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96241a33f0 con 0x7f962410ec80 2024-03-28T00:16:13.208 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:12.254+0000 7f961aa00700 1 -- 172.21.15.119:0/3951651789 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 13) v1 ==== 46163+0+0 (secure 0 0 0) 0x7f9614022020 con 0x7f962410ec80 2024-03-28T00:16:13.208 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:12.254+0000 7f9628a00700 1 -- 172.21.15.119:0/3951651789 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96240637c0 con 0x7f962410ec80 2024-03-28T00:16:13.208 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:12.255+0000 7f961aa00700 1 --2- 172.21.15.119:0/3951651789 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f9608039070 0x7f960803b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:16:13.208 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:12.255+0000 7f961aa00700 1 -- 172.21.15.119:0/3951651789 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f961404d2e0 con 0x7f962410ec80 2024-03-28T00:16:13.208 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:12.255+0000 7f9621400700 1 --2- 172.21.15.119:0/3951651789 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f9608039070 0x7f960803b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:16:13.208 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:12.256+0000 7f9621400700 1 --2- 172.21.15.119:0/3951651789 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f9608039070 0x7f960803b530 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f9610006fd0 tx=0x7f9610006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:16:13.208 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:12.260+0000 7f961aa00700 1 -- 172.21.15.119:0/3951651789 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f961400b3e0 con 0x7f962410ec80 2024-03-28T00:16:13.208 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:12.489+0000 7f9628a00700 1 -- 172.21.15.119:0/3951651789 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7f96241a3870 con 0x7f962410ec80 2024-03-28T00:16:13.208 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:12.490+0000 7f961aa00700 1 -- 172.21.15.119:0/3951651789 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7f961402dde0 con 0x7f962410ec80 2024-03-28T00:16:13.209 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:12.493+0000 7f9628a00700 1 -- 172.21.15.119:0/3951651789 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f9608039070 msgr2=0x7f960803b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:16:13.209 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:12.493+0000 7f9628a00700 1 --2- 172.21.15.119:0/3951651789 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f9608039070 0x7f960803b530 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f9610006fd0 tx=0x7f9610006e40 comp rx=0 tx=0).stop 2024-03-28T00:16:13.209 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:12.493+0000 7f9628a00700 1 -- 172.21.15.119:0/3951651789 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f962410ec80 msgr2=0x7f96241a2100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:16:13.209 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:12.493+0000 7f9628a00700 1 --2- 172.21.15.119:0/3951651789 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f962410ec80 0x7f96241a2100 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9614011b60 tx=0x7f9614011c40 comp rx=0 tx=0).stop 2024-03-28T00:16:13.209 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:12.493+0000 7f9628a00700 1 -- 172.21.15.119:0/3951651789 shutdown_connections 2024-03-28T00:16:13.209 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:12.493+0000 7f9628a00700 1 --2- 172.21.15.119:0/3951651789 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f9608039070 0x7f960803b530 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:16:13.209 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:12.493+0000 7f9628a00700 1 --2- 172.21.15.119:0/3951651789 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f962410ec80 0x7f96241a2100 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:16:13.209 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:12.493+0000 7f9628a00700 1 -- 172.21.15.119:0/3951651789 >> 172.21.15.119:0/3951651789 conn(0x7f962407d100 msgr2=0x7f962407dbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:16:13.209 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:12.494+0000 7f9628a00700 1 -- 172.21.15.119:0/3951651789 shutdown_connections 2024-03-28T00:16:13.209 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:12.494+0000 7f9628a00700 1 -- 172.21.15.119:0/3951651789 wait complete. 2024-03-28T00:16:14.873 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:13.818+0000 7fe4fd600700 1 Processor -- start 2024-03-28T00:16:14.874 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:13.819+0000 7fe4fd600700 1 -- start start 2024-03-28T00:16:14.874 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:13.819+0000 7fe4fd600700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4f807de70 0x7fe4f807c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:16:14.874 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:13.819+0000 7fe4fd600700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4f807c8a0 con 0x7fe4f807de70 2024-03-28T00:16:14.874 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:13.819+0000 7fe4f6a00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4f807de70 0x7fe4f807c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:16:14.874 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:13.819+0000 7fe4f6a00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4f807de70 0x7fe4f807c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34250/0 (socket says 172.21.15.119:34250) 2024-03-28T00:16:14.874 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:13.819+0000 7fe4f6a00700 1 -- 172.21.15.119:0/2405193696 learned_addr learned my addr 172.21.15.119:0/2405193696 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:16:14.874 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:13.820+0000 7fe4f6a00700 1 -- 172.21.15.119:0/2405193696 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4f807ca20 con 0x7fe4f807de70 2024-03-28T00:16:14.874 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:13.820+0000 7fe4f6a00700 1 --2- 172.21.15.119:0/2405193696 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4f807de70 0x7fe4f807c2d0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe4e8009e80 tx=0x7fe4e800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58a58816547df9f server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:16:14.875 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:13.820+0000 7fe4f5600700 1 -- 172.21.15.119:0/2405193696 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe4e8017070 con 0x7fe4f807de70 2024-03-28T00:16:14.875 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:13.820+0000 7fe4f5600700 1 -- 172.21.15.119:0/2405193696 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe4e800b8c0 con 0x7fe4f807de70 2024-03-28T00:16:14.875 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:13.820+0000 7fe4f5600700 1 -- 172.21.15.119:0/2405193696 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe4e80045c0 con 0x7fe4f807de70 2024-03-28T00:16:14.875 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:13.821+0000 7fe4fd600700 1 -- 172.21.15.119:0/2405193696 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4f807de70 msgr2=0x7fe4f807c2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:16:14.875 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:13.821+0000 7fe4fd600700 1 --2- 172.21.15.119:0/2405193696 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4f807de70 0x7fe4f807c2d0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe4e8009e80 tx=0x7fe4e800b330 comp rx=0 tx=0).stop 2024-03-28T00:16:14.875 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:13.821+0000 7fe4fd600700 1 -- 172.21.15.119:0/2405193696 shutdown_connections 2024-03-28T00:16:14.875 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:13.821+0000 7fe4fd600700 1 --2- 172.21.15.119:0/2405193696 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4f807de70 0x7fe4f807c2d0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:16:14.875 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:13.821+0000 7fe4fd600700 1 -- 172.21.15.119:0/2405193696 >> 172.21.15.119:0/2405193696 conn(0x7fe4f81030f0 msgr2=0x7fe4f8105510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:16:14.875 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:13.821+0000 7fe4fd600700 1 -- 172.21.15.119:0/2405193696 shutdown_connections 2024-03-28T00:16:14.875 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:13.821+0000 7fe4fd600700 1 -- 172.21.15.119:0/2405193696 wait complete. 2024-03-28T00:16:14.875 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:13.822+0000 7fe4fd600700 1 Processor -- start 2024-03-28T00:16:14.876 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:13.822+0000 7fe4fd600700 1 -- start start 2024-03-28T00:16:14.876 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:13.823+0000 7fe4fd600700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4f807de70 0x7fe4f819de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:16:14.876 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:13.823+0000 7fe4fd600700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4f810e170 con 0x7fe4f807de70 2024-03-28T00:16:14.876 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:13.823+0000 7fe4f6a00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4f807de70 0x7fe4f819de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:16:14.876 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:13.823+0000 7fe4f6a00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4f807de70 0x7fe4f819de40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34254/0 (socket says 172.21.15.119:34254) 2024-03-28T00:16:14.876 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:13.823+0000 7fe4f6a00700 1 -- 172.21.15.119:0/2258056682 learned_addr learned my addr 172.21.15.119:0/2258056682 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:16:14.876 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:13.823+0000 7fe4f6a00700 1 -- 172.21.15.119:0/2258056682 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4f819e380 con 0x7fe4f807de70 2024-03-28T00:16:14.876 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:13.824+0000 7fe4f6a00700 1 --2- 172.21.15.119:0/2258056682 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4f807de70 0x7fe4f819de40 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe4e8009fd0 tx=0x7fe4e8003d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:16:14.876 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:13.824+0000 7fe4ef400700 1 -- 172.21.15.119:0/2258056682 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe4e8020020 con 0x7fe4f807de70 2024-03-28T00:16:14.876 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:13.824+0000 7fe4ef400700 1 -- 172.21.15.119:0/2258056682 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe4e800bba0 con 0x7fe4f807de70 2024-03-28T00:16:14.876 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:13.824+0000 7fe4fd600700 1 -- 172.21.15.119:0/2258056682 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4f819e610 con 0x7fe4f807de70 2024-03-28T00:16:14.876 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:13.824+0000 7fe4ef400700 1 -- 172.21.15.119:0/2258056682 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe4e8017040 con 0x7fe4f807de70 2024-03-28T00:16:14.876 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:13.824+0000 7fe4fd600700 1 -- 172.21.15.119:0/2258056682 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4f819f240 con 0x7fe4f807de70 2024-03-28T00:16:14.876 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:13.825+0000 7fe4ef400700 1 -- 172.21.15.119:0/2258056682 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 13) v1 ==== 46163+0+0 (secure 0 0 0) 0x7fe4e8023020 con 0x7fe4f807de70 2024-03-28T00:16:14.876 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:13.825+0000 7fe4fd600700 1 -- 172.21.15.119:0/2258056682 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4f80637c0 con 0x7fe4f807de70 2024-03-28T00:16:14.877 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:13.826+0000 7fe4ef400700 1 --2- 172.21.15.119:0/2258056682 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fe4dc039010 0x7fe4dc03b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:16:14.877 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:13.826+0000 7fe4ef400700 1 -- 172.21.15.119:0/2258056682 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fe4e800f050 con 0x7fe4f807de70 2024-03-28T00:16:14.877 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:13.826+0000 7fe4f6000700 1 --2- 172.21.15.119:0/2258056682 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fe4dc039010 0x7fe4dc03b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:16:14.877 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:13.827+0000 7fe4f6000700 1 --2- 172.21.15.119:0/2258056682 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fe4dc039010 0x7fe4dc03b4d0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fe4e4006fd0 tx=0x7fe4e4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:16:14.877 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:13.831+0000 7fe4ef400700 1 -- 172.21.15.119:0/2258056682 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fe4e802a020 con 0x7fe4f807de70 2024-03-28T00:16:14.877 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:14.167+0000 7fe4fd600700 1 -- 172.21.15.119:0/2258056682 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7fe4f819f500 con 0x7fe4f807de70 2024-03-28T00:16:14.877 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:14.172+0000 7fe4ef400700 1 -- 172.21.15.119:0/2258056682 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v27)=0 set mgr/dashboard/cluster/status v27) v1 ==== 153+0+0 (secure 0 0 0) 0x7fe4e80124e0 con 0x7fe4f807de70 2024-03-28T00:16:14.877 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:14.175+0000 7fe4fd600700 1 -- 172.21.15.119:0/2258056682 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fe4dc039010 msgr2=0x7fe4dc03b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:16:14.877 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:14.175+0000 7fe4fd600700 1 --2- 172.21.15.119:0/2258056682 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fe4dc039010 0x7fe4dc03b4d0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fe4e4006fd0 tx=0x7fe4e4006e40 comp rx=0 tx=0).stop 2024-03-28T00:16:14.877 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:14.175+0000 7fe4fd600700 1 -- 172.21.15.119:0/2258056682 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4f807de70 msgr2=0x7fe4f819de40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:16:14.877 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:14.175+0000 7fe4fd600700 1 --2- 172.21.15.119:0/2258056682 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4f807de70 0x7fe4f819de40 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe4e8009fd0 tx=0x7fe4e8003d00 comp rx=0 tx=0).stop 2024-03-28T00:16:14.877 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:14.175+0000 7fe4fd600700 1 -- 172.21.15.119:0/2258056682 shutdown_connections 2024-03-28T00:16:14.877 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:14.175+0000 7fe4fd600700 1 --2- 172.21.15.119:0/2258056682 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fe4dc039010 0x7fe4dc03b4d0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:16:14.877 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:14.175+0000 7fe4fd600700 1 --2- 172.21.15.119:0/2258056682 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4f807de70 0x7fe4f819de40 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:16:14.878 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:14.175+0000 7fe4fd600700 1 -- 172.21.15.119:0/2258056682 >> 172.21.15.119:0/2258056682 conn(0x7fe4f81030f0 msgr2=0x7fe4f8103bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:16:14.878 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:14.175+0000 7fe4fd600700 1 -- 172.21.15.119:0/2258056682 shutdown_connections 2024-03-28T00:16:14.878 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-03-28T00:16:14.175+0000 7fe4fd600700 1 -- 172.21.15.119:0/2258056682 wait complete. 2024-03-28T00:16:14.878 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-03-28T00:16:14.878 INFO:teuthology.orchestra.run.smithi119.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-03-28T00:16:14.878 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-28T00:16:14.878 INFO:teuthology.orchestra.run.smithi119.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 10f73b92-ec98-11ee-95d0-87774f69a715 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-03-28T00:16:14.878 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-28T00:16:14.879 INFO:teuthology.orchestra.run.smithi119.stdout:Or, if you are only running a single cluster on this host: 2024-03-28T00:16:14.879 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-28T00:16:14.879 INFO:teuthology.orchestra.run.smithi119.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-03-28T00:16:14.879 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-28T00:16:14.879 INFO:teuthology.orchestra.run.smithi119.stdout:Please consider enabling telemetry to help improve Ceph: 2024-03-28T00:16:14.879 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-28T00:16:14.879 INFO:teuthology.orchestra.run.smithi119.stdout: ceph telemetry on 2024-03-28T00:16:14.879 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-28T00:16:14.879 INFO:teuthology.orchestra.run.smithi119.stdout:For more information see: 2024-03-28T00:16:14.879 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-28T00:16:14.879 INFO:teuthology.orchestra.run.smithi119.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-03-28T00:16:14.879 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-28T00:16:14.879 INFO:teuthology.orchestra.run.smithi119.stdout:Bootstrap complete. 2024-03-28T00:16:14.983 INFO:tasks.cephadm:Fetching config... 2024-03-28T00:16:14.983 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-03-28T00:16:14.983 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-03-28T00:16:15.030 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-03-28T00:16:15.030 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-03-28T00:16:15.030 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-03-28T00:16:15.110 INFO:tasks.cephadm:Fetching mon keyring... 2024-03-28T00:16:15.110 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-03-28T00:16:15.110 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/var/lib/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/mon.a/keyring of=/dev/stdout 2024-03-28T00:16:15.234 INFO:tasks.cephadm:Fetching pub ssh key... 2024-03-28T00:16:15.234 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-03-28T00:16:15.234 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-03-28T00:16:15.326 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-03-28T00:16:15.326 DEBUG:teuthology.orchestra.run.smithi119:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC9AehF0WSVUyKaaVqCSIvTvWY1NZFB5dfJ7NBoWfLQyoi8RP+XNkWCMQRg6gBCuc4+MPehgv8yFJsA+La+EfD+DLtR6MemXndxbUEtHRttqd04w6/Ng41HYZoGP6EYvLSMAl6ykfI5Tq2/r4Uz3tgDAqnb7LcKJz+9Xg94y73VJ8M9B+E6BSVgNQVdz4hvxYcv1MRfl6g2lzmcF2wPhbqP+0SA0uVihKODAiphdQY8po2ugSw0zHmsTH0YWPIFPAelQ2eVT4du8sghGllcXNH66DEkjhMb7TCFG2gLnP+v9/DmGQ9A95ODRYRXxy7H1qvbmyDXJ8VMhW2SFDYgia5DOjPtcfdPRc4Ckhl4YSDgMDwlV0dBHeXoTgE17WvtSsVc9pHOgKyXpeuN42Rw/Zk9C1NW0eKWdOAkU9CD/tIrOHKQjgOEyMSDkFk54AD6yh6gmgtnXG7ZpUgUrHCFwdt3lLxUHjx0zAsTXXrWHC1QLmG7sM4P5HTDbxNXXzOmA88= ceph-10f73b92-ec98-11ee-95d0-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-28T00:16:15.428 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:16:15 smithi119 ceph-mon[31518]: from='client.? 172.21.15.119:0/2258056682' entity='client.admin' 2024-03-28T00:16:15.432 INFO:teuthology.orchestra.run.smithi119.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC9AehF0WSVUyKaaVqCSIvTvWY1NZFB5dfJ7NBoWfLQyoi8RP+XNkWCMQRg6gBCuc4+MPehgv8yFJsA+La+EfD+DLtR6MemXndxbUEtHRttqd04w6/Ng41HYZoGP6EYvLSMAl6ykfI5Tq2/r4Uz3tgDAqnb7LcKJz+9Xg94y73VJ8M9B+E6BSVgNQVdz4hvxYcv1MRfl6g2lzmcF2wPhbqP+0SA0uVihKODAiphdQY8po2ugSw0zHmsTH0YWPIFPAelQ2eVT4du8sghGllcXNH66DEkjhMb7TCFG2gLnP+v9/DmGQ9A95ODRYRXxy7H1qvbmyDXJ8VMhW2SFDYgia5DOjPtcfdPRc4Ckhl4YSDgMDwlV0dBHeXoTgE17WvtSsVc9pHOgKyXpeuN42Rw/Zk9C1NW0eKWdOAkU9CD/tIrOHKQjgOEyMSDkFk54AD6yh6gmgtnXG7ZpUgUrHCFwdt3lLxUHjx0zAsTXXrWHC1QLmG7sM4P5HTDbxNXXzOmA88= ceph-10f73b92-ec98-11ee-95d0-87774f69a715 2024-03-28T00:16:15.494 DEBUG:teuthology.orchestra.run.smithi132:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC9AehF0WSVUyKaaVqCSIvTvWY1NZFB5dfJ7NBoWfLQyoi8RP+XNkWCMQRg6gBCuc4+MPehgv8yFJsA+La+EfD+DLtR6MemXndxbUEtHRttqd04w6/Ng41HYZoGP6EYvLSMAl6ykfI5Tq2/r4Uz3tgDAqnb7LcKJz+9Xg94y73VJ8M9B+E6BSVgNQVdz4hvxYcv1MRfl6g2lzmcF2wPhbqP+0SA0uVihKODAiphdQY8po2ugSw0zHmsTH0YWPIFPAelQ2eVT4du8sghGllcXNH66DEkjhMb7TCFG2gLnP+v9/DmGQ9A95ODRYRXxy7H1qvbmyDXJ8VMhW2SFDYgia5DOjPtcfdPRc4Ckhl4YSDgMDwlV0dBHeXoTgE17WvtSsVc9pHOgKyXpeuN42Rw/Zk9C1NW0eKWdOAkU9CD/tIrOHKQjgOEyMSDkFk54AD6yh6gmgtnXG7ZpUgUrHCFwdt3lLxUHjx0zAsTXXrWHC1QLmG7sM4P5HTDbxNXXzOmA88= ceph-10f73b92-ec98-11ee-95d0-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-28T00:16:15.542 INFO:teuthology.orchestra.run.smithi132.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC9AehF0WSVUyKaaVqCSIvTvWY1NZFB5dfJ7NBoWfLQyoi8RP+XNkWCMQRg6gBCuc4+MPehgv8yFJsA+La+EfD+DLtR6MemXndxbUEtHRttqd04w6/Ng41HYZoGP6EYvLSMAl6ykfI5Tq2/r4Uz3tgDAqnb7LcKJz+9Xg94y73VJ8M9B+E6BSVgNQVdz4hvxYcv1MRfl6g2lzmcF2wPhbqP+0SA0uVihKODAiphdQY8po2ugSw0zHmsTH0YWPIFPAelQ2eVT4du8sghGllcXNH66DEkjhMb7TCFG2gLnP+v9/DmGQ9A95ODRYRXxy7H1qvbmyDXJ8VMhW2SFDYgia5DOjPtcfdPRc4Ckhl4YSDgMDwlV0dBHeXoTgE17WvtSsVc9pHOgKyXpeuN42Rw/Zk9C1NW0eKWdOAkU9CD/tIrOHKQjgOEyMSDkFk54AD6yh6gmgtnXG7ZpUgUrHCFwdt3lLxUHjx0zAsTXXrWHC1QLmG7sM4P5HTDbxNXXzOmA88= ceph-10f73b92-ec98-11ee-95d0-87774f69a715 2024-03-28T00:16:15.558 DEBUG:teuthology.orchestra.run.smithi165:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC9AehF0WSVUyKaaVqCSIvTvWY1NZFB5dfJ7NBoWfLQyoi8RP+XNkWCMQRg6gBCuc4+MPehgv8yFJsA+La+EfD+DLtR6MemXndxbUEtHRttqd04w6/Ng41HYZoGP6EYvLSMAl6ykfI5Tq2/r4Uz3tgDAqnb7LcKJz+9Xg94y73VJ8M9B+E6BSVgNQVdz4hvxYcv1MRfl6g2lzmcF2wPhbqP+0SA0uVihKODAiphdQY8po2ugSw0zHmsTH0YWPIFPAelQ2eVT4du8sghGllcXNH66DEkjhMb7TCFG2gLnP+v9/DmGQ9A95ODRYRXxy7H1qvbmyDXJ8VMhW2SFDYgia5DOjPtcfdPRc4Ckhl4YSDgMDwlV0dBHeXoTgE17WvtSsVc9pHOgKyXpeuN42Rw/Zk9C1NW0eKWdOAkU9CD/tIrOHKQjgOEyMSDkFk54AD6yh6gmgtnXG7ZpUgUrHCFwdt3lLxUHjx0zAsTXXrWHC1QLmG7sM4P5HTDbxNXXzOmA88= ceph-10f73b92-ec98-11ee-95d0-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-28T00:16:15.606 INFO:teuthology.orchestra.run.smithi165.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC9AehF0WSVUyKaaVqCSIvTvWY1NZFB5dfJ7NBoWfLQyoi8RP+XNkWCMQRg6gBCuc4+MPehgv8yFJsA+La+EfD+DLtR6MemXndxbUEtHRttqd04w6/Ng41HYZoGP6EYvLSMAl6ykfI5Tq2/r4Uz3tgDAqnb7LcKJz+9Xg94y73VJ8M9B+E6BSVgNQVdz4hvxYcv1MRfl6g2lzmcF2wPhbqP+0SA0uVihKODAiphdQY8po2ugSw0zHmsTH0YWPIFPAelQ2eVT4du8sghGllcXNH66DEkjhMb7TCFG2gLnP+v9/DmGQ9A95ODRYRXxy7H1qvbmyDXJ8VMhW2SFDYgia5DOjPtcfdPRc4Ckhl4YSDgMDwlV0dBHeXoTgE17WvtSsVc9pHOgKyXpeuN42Rw/Zk9C1NW0eKWdOAkU9CD/tIrOHKQjgOEyMSDkFk54AD6yh6gmgtnXG7ZpUgUrHCFwdt3lLxUHjx0zAsTXXrWHC1QLmG7sM4P5HTDbxNXXzOmA88= ceph-10f73b92-ec98-11ee-95d0-87774f69a715 2024-03-28T00:16:15.621 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 10f73b92-ec98-11ee-95d0-87774f69a715 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-03-28T00:16:16.035 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/mon.a/config 2024-03-28T00:16:18.194 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:18.187+0000 7fa72ec00700 1 -- 172.21.15.119:0/3777330424 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa728103dc0 msgr2=0x7fa7281041e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:16:18.194 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:18.187+0000 7fa72ec00700 1 --2- 172.21.15.119:0/3777330424 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa728103dc0 0x7fa7281041e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa718009910 tx=0x7fa718009c20 comp rx=0 tx=0).stop 2024-03-28T00:16:18.194 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:18.187+0000 7fa72ec00700 1 -- 172.21.15.119:0/3777330424 shutdown_connections 2024-03-28T00:16:18.195 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:18.187+0000 7fa72ec00700 1 --2- 172.21.15.119:0/3777330424 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa728103dc0 0x7fa7281041e0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:16:18.195 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:18.187+0000 7fa72ec00700 1 -- 172.21.15.119:0/3777330424 >> 172.21.15.119:0/3777330424 conn(0x7fa7280ff360 msgr2=0x7fa7281017a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:16:18.195 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:18.187+0000 7fa72ec00700 1 -- 172.21.15.119:0/3777330424 shutdown_connections 2024-03-28T00:16:18.195 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:18.187+0000 7fa72ec00700 1 -- 172.21.15.119:0/3777330424 wait complete. 2024-03-28T00:16:18.197 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:18.188+0000 7fa72ec00700 1 Processor -- start 2024-03-28T00:16:18.197 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:18.189+0000 7fa72ec00700 1 -- start start 2024-03-28T00:16:18.197 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:18.189+0000 7fa72ec00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa728103dc0 0x7fa7281994d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:16:18.198 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:18.189+0000 7fa72ec00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa728109e50 con 0x7fa728103dc0 2024-03-28T00:16:18.198 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:18.189+0000 7fa727e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa728103dc0 0x7fa7281994d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:16:18.198 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:18.189+0000 7fa727e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa728103dc0 0x7fa7281994d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:60554/0 (socket says 172.21.15.119:60554) 2024-03-28T00:16:18.198 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:18.189+0000 7fa727e00700 1 -- 172.21.15.119:0/40327372 learned_addr learned my addr 172.21.15.119:0/40327372 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:16:18.198 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:18.190+0000 7fa727e00700 1 -- 172.21.15.119:0/40327372 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa728199a10 con 0x7fa728103dc0 2024-03-28T00:16:18.198 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:18.190+0000 7fa727e00700 1 --2- 172.21.15.119:0/40327372 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa728103dc0 0x7fa7281994d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa718005850 tx=0x7fa718005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:16:18.198 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:18.190+0000 7fa724c00700 1 -- 172.21.15.119:0/40327372 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa718026070 con 0x7fa728103dc0 2024-03-28T00:16:18.198 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:18.190+0000 7fa724c00700 1 -- 172.21.15.119:0/40327372 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa7180044a0 con 0x7fa728103dc0 2024-03-28T00:16:18.199 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:18.190+0000 7fa724c00700 1 -- 172.21.15.119:0/40327372 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa718003ab0 con 0x7fa728103dc0 2024-03-28T00:16:18.199 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:18.190+0000 7fa72ec00700 1 -- 172.21.15.119:0/40327372 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa728199ca0 con 0x7fa728103dc0 2024-03-28T00:16:18.199 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:18.190+0000 7fa72ec00700 1 -- 172.21.15.119:0/40327372 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa72819c990 con 0x7fa728103dc0 2024-03-28T00:16:18.200 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:18.191+0000 7fa724c00700 1 -- 172.21.15.119:0/40327372 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 13) v1 ==== 46163+0+0 (secure 0 0 0) 0x7fa718029070 con 0x7fa728103dc0 2024-03-28T00:16:18.200 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:18.192+0000 7fa72ec00700 1 -- 172.21.15.119:0/40327372 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa72810a5d0 con 0x7fa728103dc0 2024-03-28T00:16:18.200 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:18.192+0000 7fa724c00700 1 --2- 172.21.15.119:0/40327372 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fa70c038fc0 0x7fa70c03b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:16:18.200 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:18.192+0000 7fa724c00700 1 -- 172.21.15.119:0/40327372 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fa71801e070 con 0x7fa728103dc0 2024-03-28T00:16:18.201 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:18.192+0000 7fa727400700 1 --2- 172.21.15.119:0/40327372 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fa70c038fc0 0x7fa70c03b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:16:18.201 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:18.193+0000 7fa727400700 1 --2- 172.21.15.119:0/40327372 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fa70c038fc0 0x7fa70c03b480 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fa714006fd0 tx=0x7fa714006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:16:18.204 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:18.198+0000 7fa724c00700 1 -- 172.21.15.119:0/40327372 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa718013070 con 0x7fa728103dc0 2024-03-28T00:16:18.426 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:18.420+0000 7fa72ec00700 1 -- 172.21.15.119:0/40327372 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7fa72802db40 con 0x7fa728103dc0 2024-03-28T00:16:18.430 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:18.424+0000 7fa724c00700 1 -- 172.21.15.119:0/40327372 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7fa718030020 con 0x7fa728103dc0 2024-03-28T00:16:18.434 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:18.427+0000 7fa72ec00700 1 -- 172.21.15.119:0/40327372 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fa70c038fc0 msgr2=0x7fa70c03b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:16:18.434 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:18.427+0000 7fa72ec00700 1 --2- 172.21.15.119:0/40327372 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fa70c038fc0 0x7fa70c03b480 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fa714006fd0 tx=0x7fa714006e40 comp rx=0 tx=0).stop 2024-03-28T00:16:18.434 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:18.427+0000 7fa72ec00700 1 -- 172.21.15.119:0/40327372 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa728103dc0 msgr2=0x7fa7281994d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:16:18.434 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:18.427+0000 7fa72ec00700 1 --2- 172.21.15.119:0/40327372 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa728103dc0 0x7fa7281994d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa718005850 tx=0x7fa718005030 comp rx=0 tx=0).stop 2024-03-28T00:16:18.434 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:18.427+0000 7fa72ec00700 1 -- 172.21.15.119:0/40327372 shutdown_connections 2024-03-28T00:16:18.434 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:18.427+0000 7fa72ec00700 1 --2- 172.21.15.119:0/40327372 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fa70c038fc0 0x7fa70c03b480 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:16:18.434 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:18.427+0000 7fa72ec00700 1 --2- 172.21.15.119:0/40327372 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa728103dc0 0x7fa7281994d0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:16:18.434 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:18.428+0000 7fa72ec00700 1 -- 172.21.15.119:0/40327372 >> 172.21.15.119:0/40327372 conn(0x7fa7280ff360 msgr2=0x7fa7280ffe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:16:18.435 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:18.428+0000 7fa72ec00700 1 -- 172.21.15.119:0/40327372 shutdown_connections 2024-03-28T00:16:18.435 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:18.428+0000 7fa72ec00700 1 -- 172.21.15.119:0/40327372 wait complete. 2024-03-28T00:16:19.174 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-03-28T00:16:19.174 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 10f73b92-ec98-11ee-95d0-87774f69a715 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-03-28T00:16:19.605 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/mon.a/config 2024-03-28T00:16:19.757 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:16:19 smithi119 ceph-mon[31518]: from='client.? 172.21.15.119:0/40327372' entity='client.admin' 2024-03-28T00:16:19.757 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:16:19 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:16:19.757 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:16:19 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:16:19.757 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:16:19 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi119", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:16:19.757 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:16:19 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:16:19.757 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:16:19 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:16:19.757 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:16:19 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:16:21.515 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:21.508+0000 7f232c800700 1 -- 172.21.15.119:0/1639075744 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f23240761c0 msgr2=0x7f2324074620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:16:21.515 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:21.508+0000 7f232c800700 1 --2- 172.21.15.119:0/1639075744 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f23240761c0 0x7f2324074620 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2318009910 tx=0x7f2318009c20 comp rx=0 tx=0).stop 2024-03-28T00:16:21.516 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:21.508+0000 7f232c800700 1 -- 172.21.15.119:0/1639075744 shutdown_connections 2024-03-28T00:16:21.516 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:21.508+0000 7f232c800700 1 --2- 172.21.15.119:0/1639075744 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f23240761c0 0x7f2324074620 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:16:21.516 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:21.508+0000 7f232c800700 1 -- 172.21.15.119:0/1639075744 >> 172.21.15.119:0/1639075744 conn(0x7f23240fd150 msgr2=0x7f23240ff590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:16:21.516 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:21.508+0000 7f232c800700 1 -- 172.21.15.119:0/1639075744 shutdown_connections 2024-03-28T00:16:21.516 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:21.508+0000 7f232c800700 1 -- 172.21.15.119:0/1639075744 wait complete. 2024-03-28T00:16:21.517 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:21.510+0000 7f232c800700 1 Processor -- start 2024-03-28T00:16:21.517 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:21.510+0000 7f232c800700 1 -- start start 2024-03-28T00:16:21.517 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:21.510+0000 7f232c800700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f23240761c0 0x7f23241994e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:16:21.517 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:21.510+0000 7f232c800700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2324109e40 con 0x7f23240761c0 2024-03-28T00:16:21.517 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:21.511+0000 7f2329e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f23240761c0 0x7f23241994e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:16:21.517 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:21.511+0000 7f2329e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f23240761c0 0x7f23241994e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:60570/0 (socket says 172.21.15.119:60570) 2024-03-28T00:16:21.517 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:21.511+0000 7f2329e00700 1 -- 172.21.15.119:0/1811558099 learned_addr learned my addr 172.21.15.119:0/1811558099 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:16:21.517 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:21.511+0000 7f2329e00700 1 -- 172.21.15.119:0/1811558099 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2324199a20 con 0x7f23240761c0 2024-03-28T00:16:21.518 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:21.511+0000 7f2329e00700 1 --2- 172.21.15.119:0/1811558099 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f23240761c0 0x7f23241994e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f2318005850 tx=0x7f2318005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:16:21.518 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:21.511+0000 7f231ea00700 1 -- 172.21.15.119:0/1811558099 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2318026070 con 0x7f23240761c0 2024-03-28T00:16:21.518 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:21.512+0000 7f231ea00700 1 -- 172.21.15.119:0/1811558099 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f23180044a0 con 0x7f23240761c0 2024-03-28T00:16:21.518 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:21.512+0000 7f232c800700 1 -- 172.21.15.119:0/1811558099 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2324199cb0 con 0x7f23240761c0 2024-03-28T00:16:21.518 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:21.512+0000 7f231ea00700 1 -- 172.21.15.119:0/1811558099 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2318003ab0 con 0x7f23240761c0 2024-03-28T00:16:21.518 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:21.512+0000 7f232c800700 1 -- 172.21.15.119:0/1811558099 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f232419c9a0 con 0x7f23240761c0 2024-03-28T00:16:21.519 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:21.513+0000 7f232c800700 1 -- 172.21.15.119:0/1811558099 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f232410a5c0 con 0x7f23240761c0 2024-03-28T00:16:21.519 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:21.513+0000 7f231ea00700 1 -- 172.21.15.119:0/1811558099 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 13) v1 ==== 46163+0+0 (secure 0 0 0) 0x7f2318029070 con 0x7f23240761c0 2024-03-28T00:16:21.519 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:21.513+0000 7f231ea00700 1 --2- 172.21.15.119:0/1811558099 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f230c039020 0x7f230c03b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:16:21.520 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:21.513+0000 7f231ea00700 1 -- 172.21.15.119:0/1811558099 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f231801e070 con 0x7f23240761c0 2024-03-28T00:16:21.520 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:21.514+0000 7f2329400700 1 --2- 172.21.15.119:0/1811558099 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f230c039020 0x7f230c03b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:16:21.520 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:21.514+0000 7f2329400700 1 --2- 172.21.15.119:0/1811558099 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f230c039020 0x7f230c03b4e0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f2314006fd0 tx=0x7f2314006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:16:21.525 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:21.519+0000 7f231ea00700 1 -- 172.21.15.119:0/1811558099 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f2318013070 con 0x7f23240761c0 2024-03-28T00:16:21.753 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:21.747+0000 7f232c800700 1 -- 172.21.15.119:0/1811558099 --> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7f232402db40 con 0x7f230c039020 2024-03-28T00:16:21.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:21.755+0000 7f231ea00700 1 -- 172.21.15.119:0/1811558099 <== mgr.14152 v2:172.21.15.119:6800/1401937096 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f232402db40 con 0x7f230c039020 2024-03-28T00:16:21.764 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:21.758+0000 7f232c800700 1 -- 172.21.15.119:0/1811558099 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f230c039020 msgr2=0x7f230c03b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:16:21.764 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:21.758+0000 7f232c800700 1 --2- 172.21.15.119:0/1811558099 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f230c039020 0x7f230c03b4e0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f2314006fd0 tx=0x7f2314006e40 comp rx=0 tx=0).stop 2024-03-28T00:16:21.764 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:21.758+0000 7f232c800700 1 -- 172.21.15.119:0/1811558099 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f23240761c0 msgr2=0x7f23241994e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:16:21.764 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:21.758+0000 7f232c800700 1 --2- 172.21.15.119:0/1811558099 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f23240761c0 0x7f23241994e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f2318005850 tx=0x7f2318005030 comp rx=0 tx=0).stop 2024-03-28T00:16:21.765 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:21.758+0000 7f232c800700 1 -- 172.21.15.119:0/1811558099 shutdown_connections 2024-03-28T00:16:21.765 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:21.758+0000 7f232c800700 1 --2- 172.21.15.119:0/1811558099 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f230c039020 0x7f230c03b4e0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:16:21.765 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:21.758+0000 7f232c800700 1 --2- 172.21.15.119:0/1811558099 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f23240761c0 0x7f23241994e0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:16:21.765 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:21.758+0000 7f232c800700 1 -- 172.21.15.119:0/1811558099 >> 172.21.15.119:0/1811558099 conn(0x7f23240fd150 msgr2=0x7f23240fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:16:21.765 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:21.759+0000 7f232c800700 1 -- 172.21.15.119:0/1811558099 shutdown_connections 2024-03-28T00:16:21.765 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:21.759+0000 7f232c800700 1 -- 172.21.15.119:0/1811558099 wait complete. 2024-03-28T00:16:22.602 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi132 2024-03-28T00:16:22.602 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-03-28T00:16:22.602 DEBUG:teuthology.orchestra.run.smithi132:> dd of=/etc/ceph/ceph.conf 2024-03-28T00:16:22.620 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-03-28T00:16:22.620 DEBUG:teuthology.orchestra.run.smithi132:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-28T00:16:22.678 INFO:tasks.cephadm:Adding host smithi132 to orchestrator... 2024-03-28T00:16:22.678 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 10f73b92-ec98-11ee-95d0-87774f69a715 -- ceph orch host add smithi132 2024-03-28T00:16:22.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:16:22 smithi119 ceph-mon[31518]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T00:16:22.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:16:22 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:16:22.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:16:22 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:16:22.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:16:22 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:16:22.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:16:22 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:16:22.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:16:22 smithi119 ceph-mon[31518]: Updating smithi119:/etc/ceph/ceph.conf 2024-03-28T00:16:23.106 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/mon.a/config 2024-03-28T00:16:23.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:16:23 smithi119 ceph-mon[31518]: Updating smithi119:/var/lib/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-28T00:16:24.995 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:16:24 smithi119 ceph-mon[31518]: Updating smithi119:/etc/ceph/ceph.client.admin.keyring 2024-03-28T00:16:24.995 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:16:24 smithi119 ceph-mon[31518]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T00:16:24.995 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:16:24 smithi119 ceph-mon[31518]: Updating smithi119:/var/lib/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/config/ceph.client.admin.keyring 2024-03-28T00:16:24.995 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:16:24 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:16:24.996 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:16:24 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:16:24.996 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:16:24 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:16:25.160 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:25.153+0000 7f363e200700 1 -- 172.21.15.119:0/752971500 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3638103fb0 msgr2=0x7f3638104390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:16:25.161 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:25.153+0000 7f363e200700 1 --2- 172.21.15.119:0/752971500 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3638103fb0 0x7f3638104390 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f3628009910 tx=0x7f3628009c20 comp rx=0 tx=0).stop 2024-03-28T00:16:25.161 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:25.153+0000 7f363e200700 1 -- 172.21.15.119:0/752971500 shutdown_connections 2024-03-28T00:16:25.161 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:25.153+0000 7f363e200700 1 --2- 172.21.15.119:0/752971500 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3638103fb0 0x7f3638104390 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:16:25.161 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:25.153+0000 7f363e200700 1 -- 172.21.15.119:0/752971500 >> 172.21.15.119:0/752971500 conn(0x7f36380ff820 msgr2=0x7f3638101c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:16:25.161 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:25.153+0000 7f363e200700 1 -- 172.21.15.119:0/752971500 shutdown_connections 2024-03-28T00:16:25.161 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:25.153+0000 7f363e200700 1 -- 172.21.15.119:0/752971500 wait complete. 2024-03-28T00:16:25.163 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:25.154+0000 7f363e200700 1 Processor -- start 2024-03-28T00:16:25.163 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:25.154+0000 7f363e200700 1 -- start start 2024-03-28T00:16:25.163 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:25.155+0000 7f363e200700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3638103fb0 0x7f363819f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:16:25.163 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:25.155+0000 7f363e200700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f363810e380 con 0x7f3638103fb0 2024-03-28T00:16:25.163 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:25.155+0000 7f3637400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3638103fb0 0x7f363819f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:16:25.163 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:25.155+0000 7f3637400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3638103fb0 0x7f363819f5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:59280/0 (socket says 172.21.15.119:59280) 2024-03-28T00:16:25.163 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:25.155+0000 7f3637400700 1 -- 172.21.15.119:0/2899031680 learned_addr learned my addr 172.21.15.119:0/2899031680 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:16:25.163 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:25.155+0000 7f3637400700 1 -- 172.21.15.119:0/2899031680 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f363819fb30 con 0x7f3638103fb0 2024-03-28T00:16:25.163 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:25.156+0000 7f3637400700 1 --2- 172.21.15.119:0/2899031680 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3638103fb0 0x7f363819f5f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3628006010 tx=0x7f3628005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:16:25.163 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:25.156+0000 7f362fe00700 1 -- 172.21.15.119:0/2899031680 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3628026070 con 0x7f3638103fb0 2024-03-28T00:16:25.163 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:25.156+0000 7f362fe00700 1 -- 172.21.15.119:0/2899031680 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f362800bbd0 con 0x7f3638103fb0 2024-03-28T00:16:25.164 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:25.156+0000 7f363e200700 1 -- 172.21.15.119:0/2899031680 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f363819fe20 con 0x7f3638103fb0 2024-03-28T00:16:25.164 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:25.156+0000 7f362fe00700 1 -- 172.21.15.119:0/2899031680 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3628003910 con 0x7f3638103fb0 2024-03-28T00:16:25.164 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:25.156+0000 7f363e200700 1 -- 172.21.15.119:0/2899031680 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3638199890 con 0x7f3638103fb0 2024-03-28T00:16:25.165 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:25.157+0000 7f362fe00700 1 -- 172.21.15.119:0/2899031680 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 13) v1 ==== 46163+0+0 (secure 0 0 0) 0x7f3628029070 con 0x7f3638103fb0 2024-03-28T00:16:25.165 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:25.157+0000 7f363e200700 1 -- 172.21.15.119:0/2899031680 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3638105490 con 0x7f3638103fb0 2024-03-28T00:16:25.165 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:25.158+0000 7f362fe00700 1 --2- 172.21.15.119:0/2899031680 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f361c039020 0x7f361c03b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:16:25.165 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:25.158+0000 7f362fe00700 1 -- 172.21.15.119:0/2899031680 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f362801e070 con 0x7f3638103fb0 2024-03-28T00:16:25.165 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:25.158+0000 7f3636a00700 1 --2- 172.21.15.119:0/2899031680 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f361c039020 0x7f361c03b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:16:25.165 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:25.159+0000 7f3636a00700 1 --2- 172.21.15.119:0/2899031680 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f361c039020 0x7f361c03b4e0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f3624006fd0 tx=0x7f3624006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:16:25.169 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:25.163+0000 7f362fe00700 1 -- 172.21.15.119:0/2899031680 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f3628013070 con 0x7f3638103fb0 2024-03-28T00:16:25.393 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:25.387+0000 7f363e200700 1 -- 172.21.15.119:0/2899031680 --> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi132", "target": ["mon-mgr", ""]}) v1 -- 0x7f363819ac20 con 0x7f361c039020 2024-03-28T00:16:27.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:16:26 smithi119 ceph-mon[31518]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi132", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T00:16:27.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:16:26 smithi119 ceph-mon[31518]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T00:16:28.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:16:27 smithi119 ceph-mon[31518]: Deploying cephadm binary to smithi132 2024-03-28T00:16:29.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:16:28 smithi119 ceph-mon[31518]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T00:16:30.097 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:30.091+0000 7f362fe00700 1 -- 172.21.15.119:0/2899031680 <== mgr.14152 v2:172.21.15.119:6800/1401937096 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f363819ac20 con 0x7f361c039020 2024-03-28T00:16:30.098 INFO:teuthology.orchestra.run.smithi119.stdout:Added host 'smithi132' with addr '172.21.15.132' 2024-03-28T00:16:30.101 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:30.095+0000 7f363e200700 1 -- 172.21.15.119:0/2899031680 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f361c039020 msgr2=0x7f361c03b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:16:30.102 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:30.095+0000 7f363e200700 1 --2- 172.21.15.119:0/2899031680 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f361c039020 0x7f361c03b4e0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f3624006fd0 tx=0x7f3624006e40 comp rx=0 tx=0).stop 2024-03-28T00:16:30.102 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:30.095+0000 7f363e200700 1 -- 172.21.15.119:0/2899031680 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3638103fb0 msgr2=0x7f363819f5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:16:30.102 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:30.095+0000 7f363e200700 1 --2- 172.21.15.119:0/2899031680 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3638103fb0 0x7f363819f5f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3628006010 tx=0x7f3628005030 comp rx=0 tx=0).stop 2024-03-28T00:16:30.102 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:30.095+0000 7f363e200700 1 -- 172.21.15.119:0/2899031680 shutdown_connections 2024-03-28T00:16:30.102 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:30.095+0000 7f363e200700 1 --2- 172.21.15.119:0/2899031680 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f361c039020 0x7f361c03b4e0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:16:30.102 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:30.095+0000 7f363e200700 1 --2- 172.21.15.119:0/2899031680 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3638103fb0 0x7f363819f5f0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:16:30.102 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:30.095+0000 7f363e200700 1 -- 172.21.15.119:0/2899031680 >> 172.21.15.119:0/2899031680 conn(0x7f36380ff820 msgr2=0x7f3638105980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:16:30.102 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:30.095+0000 7f363e200700 1 -- 172.21.15.119:0/2899031680 shutdown_connections 2024-03-28T00:16:30.102 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:30.095+0000 7f363e200700 1 -- 172.21.15.119:0/2899031680 wait complete. 2024-03-28T00:16:31.072 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:16:30 smithi119 ceph-mon[31518]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T00:16:31.073 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:16:30 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:16:31.073 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:16:30 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:16:31.073 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:16:30 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:16:31.126 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 10f73b92-ec98-11ee-95d0-87774f69a715 -- ceph orch host ls --format=json 2024-03-28T00:16:31.553 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/mon.a/config 2024-03-28T00:16:31.967 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:16:31 smithi119 ceph-mon[31518]: Added host smithi132 2024-03-28T00:16:31.968 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:16:31 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:16:33.062 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:16:32 smithi119 ceph-mon[31518]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T00:16:33.062 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:16:32 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:16:33.539 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:33.532+0000 7f8fce600700 1 -- 172.21.15.119:0/3636577720 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8fc8101dc0 msgr2=0x7f8fc81021a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:16:33.540 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:33.532+0000 7f8fce600700 1 --2- 172.21.15.119:0/3636577720 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8fc8101dc0 0x7f8fc81021a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f8fb800b0a0 tx=0x7f8fb800b3b0 comp rx=0 tx=0).stop 2024-03-28T00:16:33.540 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:33.533+0000 7f8fce600700 1 -- 172.21.15.119:0/3636577720 shutdown_connections 2024-03-28T00:16:33.540 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:33.533+0000 7f8fce600700 1 --2- 172.21.15.119:0/3636577720 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8fc8101dc0 0x7f8fc81021a0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:16:33.540 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:33.533+0000 7f8fce600700 1 -- 172.21.15.119:0/3636577720 >> 172.21.15.119:0/3636577720 conn(0x7f8fc80fd670 msgr2=0x7f8fc80ffa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:16:33.540 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:33.533+0000 7f8fce600700 1 -- 172.21.15.119:0/3636577720 shutdown_connections 2024-03-28T00:16:33.540 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:33.533+0000 7f8fce600700 1 -- 172.21.15.119:0/3636577720 wait complete. 2024-03-28T00:16:33.541 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:33.534+0000 7f8fce600700 1 Processor -- start 2024-03-28T00:16:33.541 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:33.534+0000 7f8fce600700 1 -- start start 2024-03-28T00:16:33.541 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:33.534+0000 7f8fce600700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8fc8101dc0 0x7f8fc8074490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:16:33.541 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:33.534+0000 7f8fce600700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fc810c190 con 0x7f8fc8101dc0 2024-03-28T00:16:33.541 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:33.534+0000 7f8fc7400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8fc8101dc0 0x7f8fc8074490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:16:33.541 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:33.535+0000 7f8fc7400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8fc8101dc0 0x7f8fc8074490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:59294/0 (socket says 172.21.15.119:59294) 2024-03-28T00:16:33.541 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:33.535+0000 7f8fc7400700 1 -- 172.21.15.119:0/3164960320 learned_addr learned my addr 172.21.15.119:0/3164960320 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:16:33.542 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:33.535+0000 7f8fc7400700 1 -- 172.21.15.119:0/3164960320 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fc8076380 con 0x7f8fc8101dc0 2024-03-28T00:16:33.542 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:33.535+0000 7f8fc7400700 1 --2- 172.21.15.119:0/3164960320 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8fc8101dc0 0x7f8fc8074490 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f8fb80093e0 tx=0x7f8fb8003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:16:33.542 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:33.535+0000 7f8fc4c00700 1 -- 172.21.15.119:0/3164960320 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8fb8017040 con 0x7f8fc8101dc0 2024-03-28T00:16:33.542 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:33.535+0000 7f8fc4c00700 1 -- 172.21.15.119:0/3164960320 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8fb8022070 con 0x7f8fc8101dc0 2024-03-28T00:16:33.543 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:33.535+0000 7f8fce600700 1 -- 172.21.15.119:0/3164960320 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fc8074a30 con 0x7f8fc8101dc0 2024-03-28T00:16:33.543 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:33.535+0000 7f8fc4c00700 1 -- 172.21.15.119:0/3164960320 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8fb801d440 con 0x7f8fc8101dc0 2024-03-28T00:16:33.543 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:33.536+0000 7f8fce600700 1 -- 172.21.15.119:0/3164960320 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fc8074ef0 con 0x7f8fc8101dc0 2024-03-28T00:16:33.543 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:33.537+0000 7f8fc4c00700 1 -- 172.21.15.119:0/3164960320 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 13) v1 ==== 46163+0+0 (secure 0 0 0) 0x7f8fb8004000 con 0x7f8fc8101dc0 2024-03-28T00:16:33.544 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:33.537+0000 7f8fce600700 1 -- 172.21.15.119:0/3164960320 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fc81032a0 con 0x7f8fc8101dc0 2024-03-28T00:16:33.544 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:33.537+0000 7f8fc4c00700 1 --2- 172.21.15.119:0/3164960320 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f8fac039070 0x7f8fac03b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:16:33.544 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:33.538+0000 7f8fc4c00700 1 -- 172.21.15.119:0/3164960320 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f8fb800f050 con 0x7f8fc8101dc0 2024-03-28T00:16:33.544 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:33.538+0000 7f8fbea00700 1 --2- 172.21.15.119:0/3164960320 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f8fac039070 0x7f8fac03b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:16:33.544 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:33.538+0000 7f8fbea00700 1 --2- 172.21.15.119:0/3164960320 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f8fac039070 0x7f8fac03b530 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f8fb0006fd0 tx=0x7f8fb0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:16:33.549 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:33.543+0000 7f8fc4c00700 1 -- 172.21.15.119:0/3164960320 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f8fb8007e20 con 0x7f8fc8101dc0 2024-03-28T00:16:33.778 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:33.771+0000 7f8fce600700 1 -- 172.21.15.119:0/3164960320 --> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8fc80759c0 con 0x7f8fac039070 2024-03-28T00:16:33.780 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:33.774+0000 7f8fc4c00700 1 -- 172.21.15.119:0/3164960320 <== mgr.14152 v2:172.21.15.119:6800/1401937096 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+161 (secure 0 0 0) 0x7f8fc80759c0 con 0x7f8fac039070 2024-03-28T00:16:33.780 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-28T00:16:33.780 INFO:teuthology.orchestra.run.smithi119.stdout:[{"addr": "172.21.15.119", "hostname": "smithi119", "labels": [], "status": ""}, {"addr": "172.21.15.132", "hostname": "smithi132", "labels": [], "status": ""}] 2024-03-28T00:16:33.783 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:33.777+0000 7f8fce600700 1 -- 172.21.15.119:0/3164960320 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f8fac039070 msgr2=0x7f8fac03b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:16:33.783 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:33.777+0000 7f8fce600700 1 --2- 172.21.15.119:0/3164960320 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f8fac039070 0x7f8fac03b530 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f8fb0006fd0 tx=0x7f8fb0006e40 comp rx=0 tx=0).stop 2024-03-28T00:16:33.783 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:33.777+0000 7f8fce600700 1 -- 172.21.15.119:0/3164960320 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8fc8101dc0 msgr2=0x7f8fc8074490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:16:33.783 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:33.777+0000 7f8fce600700 1 --2- 172.21.15.119:0/3164960320 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8fc8101dc0 0x7f8fc8074490 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f8fb80093e0 tx=0x7f8fb8003ce0 comp rx=0 tx=0).stop 2024-03-28T00:16:33.783 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:33.777+0000 7f8fce600700 1 -- 172.21.15.119:0/3164960320 shutdown_connections 2024-03-28T00:16:33.783 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:33.777+0000 7f8fce600700 1 --2- 172.21.15.119:0/3164960320 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f8fac039070 0x7f8fac03b530 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:16:33.783 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:33.777+0000 7f8fce600700 1 --2- 172.21.15.119:0/3164960320 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8fc8101dc0 0x7f8fc8074490 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:16:33.783 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:33.777+0000 7f8fce600700 1 -- 172.21.15.119:0/3164960320 >> 172.21.15.119:0/3164960320 conn(0x7f8fc80fd670 msgr2=0x7f8fc8103790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:16:33.784 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:33.777+0000 7f8fce600700 1 -- 172.21.15.119:0/3164960320 shutdown_connections 2024-03-28T00:16:33.784 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:33.777+0000 7f8fce600700 1 -- 172.21.15.119:0/3164960320 wait complete. 2024-03-28T00:16:34.559 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi165 2024-03-28T00:16:34.559 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-03-28T00:16:34.559 DEBUG:teuthology.orchestra.run.smithi165:> dd of=/etc/ceph/ceph.conf 2024-03-28T00:16:34.577 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-03-28T00:16:34.577 DEBUG:teuthology.orchestra.run.smithi165:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-28T00:16:34.635 INFO:tasks.cephadm:Adding host smithi165 to orchestrator... 2024-03-28T00:16:34.635 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 10f73b92-ec98-11ee-95d0-87774f69a715 -- ceph orch host add smithi165 2024-03-28T00:16:35.059 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/mon.a/config 2024-03-28T00:16:35.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:16:34 smithi119 ceph-mon[31518]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T00:16:35.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:16:34 smithi119 ceph-mon[31518]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-28T00:16:37.001 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:16:36 smithi119 ceph-mon[31518]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T00:16:37.001 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:16:36 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:16:37.001 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:16:36 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:16:37.001 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:16:36 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:16:37.001 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:16:36 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:16:37.001 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:16:36 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi132", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:16:37.001 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:16:36 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:16:37.001 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:16:36 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:16:37.001 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:16:36 smithi119 ceph-mon[31518]: Updating smithi132:/etc/ceph/ceph.conf 2024-03-28T00:16:37.284 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:37.277+0000 7f60bfe00700 1 -- 172.21.15.119:0/1063382555 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f60c0107980 msgr2=0x7f60c0107d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:16:37.284 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:37.277+0000 7f60bfe00700 1 --2- 172.21.15.119:0/1063382555 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f60c0107980 0x7f60c0107d60 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f60b0009910 tx=0x7f60b0009c20 comp rx=0 tx=0).stop 2024-03-28T00:16:37.284 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:37.277+0000 7f60bfe00700 1 -- 172.21.15.119:0/1063382555 shutdown_connections 2024-03-28T00:16:37.284 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:37.277+0000 7f60bfe00700 1 --2- 172.21.15.119:0/1063382555 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f60c0107980 0x7f60c0107d60 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:16:37.284 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:37.277+0000 7f60bfe00700 1 -- 172.21.15.119:0/1063382555 >> 172.21.15.119:0/1063382555 conn(0x7f60c00fd490 msgr2=0x7f60c00ff8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:16:37.284 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:37.277+0000 7f60bfe00700 1 -- 172.21.15.119:0/1063382555 shutdown_connections 2024-03-28T00:16:37.284 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:37.277+0000 7f60bfe00700 1 -- 172.21.15.119:0/1063382555 wait complete. 2024-03-28T00:16:37.285 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:37.278+0000 7f60bfe00700 1 Processor -- start 2024-03-28T00:16:37.285 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:37.278+0000 7f60bfe00700 1 -- start start 2024-03-28T00:16:37.285 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:37.278+0000 7f60bfe00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f60c0107980 0x7f60c01996a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:16:37.285 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:37.278+0000 7f60bfe00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60c010bfc0 con 0x7f60c0107980 2024-03-28T00:16:37.285 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:37.279+0000 7f60bea00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f60c0107980 0x7f60c01996a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:16:37.285 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:37.279+0000 7f60bea00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f60c0107980 0x7f60c01996a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35194/0 (socket says 172.21.15.119:35194) 2024-03-28T00:16:37.286 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:37.279+0000 7f60bea00700 1 -- 172.21.15.119:0/3765124688 learned_addr learned my addr 172.21.15.119:0/3765124688 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:16:37.286 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:37.279+0000 7f60bea00700 1 -- 172.21.15.119:0/3765124688 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60c0199be0 con 0x7f60c0107980 2024-03-28T00:16:37.286 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:37.279+0000 7f60bea00700 1 --2- 172.21.15.119:0/3765124688 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f60c0107980 0x7f60c01996a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f60b0006010 tx=0x7f60b0005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:16:37.286 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:37.280+0000 7f60b7400700 1 -- 172.21.15.119:0/3765124688 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f60b0026070 con 0x7f60c0107980 2024-03-28T00:16:37.286 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:37.280+0000 7f60b7400700 1 -- 172.21.15.119:0/3765124688 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f60b000bbd0 con 0x7f60c0107980 2024-03-28T00:16:37.286 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:37.280+0000 7f60b7400700 1 -- 172.21.15.119:0/3765124688 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f60b0003910 con 0x7f60c0107980 2024-03-28T00:16:37.287 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:37.280+0000 7f60bfe00700 1 -- 172.21.15.119:0/3765124688 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60c0199ed0 con 0x7f60c0107980 2024-03-28T00:16:37.287 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:37.280+0000 7f60bfe00700 1 -- 172.21.15.119:0/3765124688 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60c019df70 con 0x7f60c0107980 2024-03-28T00:16:37.287 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:37.281+0000 7f60bfe00700 1 -- 172.21.15.119:0/3765124688 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60c01030c0 con 0x7f60c0107980 2024-03-28T00:16:37.288 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:37.282+0000 7f60b7400700 1 -- 172.21.15.119:0/3765124688 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 13) v1 ==== 46163+0+0 (secure 0 0 0) 0x7f60b0029070 con 0x7f60c0107980 2024-03-28T00:16:37.288 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:37.282+0000 7f60b7400700 1 --2- 172.21.15.119:0/3765124688 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f60a4039020 0x7f60a403b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:16:37.288 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:37.282+0000 7f60b7400700 1 -- 172.21.15.119:0/3765124688 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f60b001e070 con 0x7f60c0107980 2024-03-28T00:16:37.289 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:37.283+0000 7f60be000700 1 --2- 172.21.15.119:0/3765124688 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f60a4039020 0x7f60a403b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:16:37.289 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:37.283+0000 7f60be000700 1 --2- 172.21.15.119:0/3765124688 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f60a4039020 0x7f60a403b4e0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f60ac006fd0 tx=0x7f60ac006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:16:37.298 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:37.293+0000 7f60b7400700 1 -- 172.21.15.119:0/3765124688 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f60b0013070 con 0x7f60c0107980 2024-03-28T00:16:37.526 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:37.519+0000 7f60bfe00700 1 -- 172.21.15.119:0/3765124688 --> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi165", "target": ["mon-mgr", ""]}) v1 -- 0x7f60c019aa00 con 0x7f60a4039020 2024-03-28T00:16:38.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:16:37 smithi119 ceph-mon[31518]: Updating smithi132:/var/lib/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-28T00:16:38.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:16:37 smithi119 ceph-mon[31518]: Updating smithi132:/etc/ceph/ceph.client.admin.keyring 2024-03-28T00:16:38.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:16:37 smithi119 ceph-mon[31518]: Updating smithi132:/var/lib/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/config/ceph.client.admin.keyring 2024-03-28T00:16:38.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:16:37 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:16:38.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:16:37 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:16:38.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:16:37 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:16:39.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:16:38 smithi119 ceph-mon[31518]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi165", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T00:16:39.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:16:38 smithi119 ceph-mon[31518]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T00:16:40.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:16:39 smithi119 ceph-mon[31518]: Deploying cephadm binary to smithi165 2024-03-28T00:16:41.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:16:40 smithi119 ceph-mon[31518]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T00:16:42.184 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:42.177+0000 7f60b7400700 1 -- 172.21.15.119:0/3765124688 <== mgr.14152 v2:172.21.15.119:6800/1401937096 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f60c019aa00 con 0x7f60a4039020 2024-03-28T00:16:42.184 INFO:teuthology.orchestra.run.smithi119.stdout:Added host 'smithi165' with addr '172.21.15.165' 2024-03-28T00:16:42.188 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:42.181+0000 7f60bfe00700 1 -- 172.21.15.119:0/3765124688 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f60a4039020 msgr2=0x7f60a403b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:16:42.188 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:42.181+0000 7f60bfe00700 1 --2- 172.21.15.119:0/3765124688 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f60a4039020 0x7f60a403b4e0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f60ac006fd0 tx=0x7f60ac006e40 comp rx=0 tx=0).stop 2024-03-28T00:16:42.188 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:42.181+0000 7f60bfe00700 1 -- 172.21.15.119:0/3765124688 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f60c0107980 msgr2=0x7f60c01996a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:16:42.188 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:42.181+0000 7f60bfe00700 1 --2- 172.21.15.119:0/3765124688 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f60c0107980 0x7f60c01996a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f60b0006010 tx=0x7f60b0005030 comp rx=0 tx=0).stop 2024-03-28T00:16:42.188 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:42.181+0000 7f60bfe00700 1 -- 172.21.15.119:0/3765124688 shutdown_connections 2024-03-28T00:16:42.188 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:42.181+0000 7f60bfe00700 1 --2- 172.21.15.119:0/3765124688 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f60a4039020 0x7f60a403b4e0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:16:42.188 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:42.181+0000 7f60bfe00700 1 --2- 172.21.15.119:0/3765124688 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f60c0107980 0x7f60c01996a0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:16:42.188 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:42.181+0000 7f60bfe00700 1 -- 172.21.15.119:0/3765124688 >> 172.21.15.119:0/3765124688 conn(0x7f60c00fd490 msgr2=0x7f60c0109aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:16:42.188 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:42.182+0000 7f60bfe00700 1 -- 172.21.15.119:0/3765124688 shutdown_connections 2024-03-28T00:16:42.188 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:42.182+0000 7f60bfe00700 1 -- 172.21.15.119:0/3765124688 wait complete. 2024-03-28T00:16:42.886 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 10f73b92-ec98-11ee-95d0-87774f69a715 -- ceph orch host ls --format=json 2024-03-28T00:16:43.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:16:42 smithi119 ceph-mon[31518]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T00:16:43.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:16:42 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:16:43.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:16:42 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:16:43.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:16:42 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:16:43.328 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/mon.a/config 2024-03-28T00:16:44.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:16:43 smithi119 ceph-mon[31518]: Added host smithi165 2024-03-28T00:16:44.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:16:43 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:16:45.130 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:16:44 smithi119 ceph-mon[31518]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T00:16:45.293 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:45.286+0000 7f121dc00700 1 -- 172.21.15.119:0/3468255107 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1218103fa0 msgr2=0x7f1218104380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:16:45.293 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:45.286+0000 7f121dc00700 1 --2- 172.21.15.119:0/3468255107 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1218103fa0 0x7f1218104380 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1208009910 tx=0x7f1208009c20 comp rx=0 tx=0).stop 2024-03-28T00:16:45.293 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:45.286+0000 7f121dc00700 1 -- 172.21.15.119:0/3468255107 shutdown_connections 2024-03-28T00:16:45.293 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:45.286+0000 7f121dc00700 1 --2- 172.21.15.119:0/3468255107 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1218103fa0 0x7f1218104380 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:16:45.293 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:45.286+0000 7f121dc00700 1 -- 172.21.15.119:0/3468255107 >> 172.21.15.119:0/3468255107 conn(0x7f12180ff810 msgr2=0x7f1218101c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:16:45.293 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:45.287+0000 7f121dc00700 1 -- 172.21.15.119:0/3468255107 shutdown_connections 2024-03-28T00:16:45.294 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:45.287+0000 7f121dc00700 1 -- 172.21.15.119:0/3468255107 wait complete. 2024-03-28T00:16:45.295 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:45.288+0000 7f121dc00700 1 Processor -- start 2024-03-28T00:16:45.295 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:45.288+0000 7f121dc00700 1 -- start start 2024-03-28T00:16:45.295 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:45.288+0000 7f121dc00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1218103fa0 0x7f121819f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:16:45.295 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:45.288+0000 7f121dc00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f121810e370 con 0x7f1218103fa0 2024-03-28T00:16:45.295 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:45.288+0000 7f1216a00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1218103fa0 0x7f121819f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:16:45.295 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:45.288+0000 7f1216a00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1218103fa0 0x7f121819f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40942/0 (socket says 172.21.15.119:40942) 2024-03-28T00:16:45.295 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:45.288+0000 7f1216a00700 1 -- 172.21.15.119:0/2674099487 learned_addr learned my addr 172.21.15.119:0/2674099487 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:16:45.295 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:45.289+0000 7f1216a00700 1 -- 172.21.15.119:0/2674099487 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f121819fb20 con 0x7f1218103fa0 2024-03-28T00:16:45.296 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:45.289+0000 7f1216a00700 1 --2- 172.21.15.119:0/2674099487 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1218103fa0 0x7f121819f5e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1208006010 tx=0x7f1208005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:16:45.296 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:45.289+0000 7f120f400700 1 -- 172.21.15.119:0/2674099487 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1208026070 con 0x7f1218103fa0 2024-03-28T00:16:45.296 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:45.289+0000 7f120f400700 1 -- 172.21.15.119:0/2674099487 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f120800bbd0 con 0x7f1218103fa0 2024-03-28T00:16:45.296 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:45.289+0000 7f121dc00700 1 -- 172.21.15.119:0/2674099487 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f121819fe10 con 0x7f1218103fa0 2024-03-28T00:16:45.296 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:45.289+0000 7f120f400700 1 -- 172.21.15.119:0/2674099487 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1208003910 con 0x7f1218103fa0 2024-03-28T00:16:45.296 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:45.290+0000 7f121dc00700 1 -- 172.21.15.119:0/2674099487 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1218199880 con 0x7f1218103fa0 2024-03-28T00:16:45.297 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:45.291+0000 7f121dc00700 1 -- 172.21.15.119:0/2674099487 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1218105480 con 0x7f1218103fa0 2024-03-28T00:16:45.297 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:45.291+0000 7f120f400700 1 -- 172.21.15.119:0/2674099487 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 13) v1 ==== 46163+0+0 (secure 0 0 0) 0x7f1208029070 con 0x7f1218103fa0 2024-03-28T00:16:45.297 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:45.291+0000 7f120f400700 1 --2- 172.21.15.119:0/2674099487 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f11fc039020 0x7f11fc03b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:16:45.298 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:45.291+0000 7f120f400700 1 -- 172.21.15.119:0/2674099487 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f120801e070 con 0x7f1218103fa0 2024-03-28T00:16:45.298 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:45.291+0000 7f1216000700 1 --2- 172.21.15.119:0/2674099487 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f11fc039020 0x7f11fc03b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:16:45.298 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:45.292+0000 7f1216000700 1 --2- 172.21.15.119:0/2674099487 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f11fc039020 0x7f11fc03b4e0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f1204006fd0 tx=0x7f1204006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:16:45.303 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:45.297+0000 7f120f400700 1 -- 172.21.15.119:0/2674099487 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f1208013070 con 0x7f1218103fa0 2024-03-28T00:16:45.529 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:45.523+0000 7f121dc00700 1 -- 172.21.15.119:0/2674099487 --> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f121819ac10 con 0x7f11fc039020 2024-03-28T00:16:45.532 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:45.526+0000 7f120f400700 1 -- 172.21.15.119:0/2674099487 <== mgr.14152 v2:172.21.15.119:6800/1401937096 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+241 (secure 0 0 0) 0x7f121819ac10 con 0x7f11fc039020 2024-03-28T00:16:45.532 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-28T00:16:45.532 INFO:teuthology.orchestra.run.smithi119.stdout:[{"addr": "172.21.15.119", "hostname": "smithi119", "labels": [], "status": ""}, {"addr": "172.21.15.132", "hostname": "smithi132", "labels": [], "status": ""}, {"addr": "172.21.15.165", "hostname": "smithi165", "labels": [], "status": ""}] 2024-03-28T00:16:45.535 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:45.528+0000 7f121dc00700 1 -- 172.21.15.119:0/2674099487 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f11fc039020 msgr2=0x7f11fc03b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:16:45.535 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:45.528+0000 7f121dc00700 1 --2- 172.21.15.119:0/2674099487 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f11fc039020 0x7f11fc03b4e0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f1204006fd0 tx=0x7f1204006e40 comp rx=0 tx=0).stop 2024-03-28T00:16:45.535 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:45.529+0000 7f121dc00700 1 -- 172.21.15.119:0/2674099487 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1218103fa0 msgr2=0x7f121819f5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:16:45.535 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:45.529+0000 7f121dc00700 1 --2- 172.21.15.119:0/2674099487 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1218103fa0 0x7f121819f5e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1208006010 tx=0x7f1208005030 comp rx=0 tx=0).stop 2024-03-28T00:16:45.536 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:45.529+0000 7f121dc00700 1 -- 172.21.15.119:0/2674099487 shutdown_connections 2024-03-28T00:16:45.536 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:45.529+0000 7f121dc00700 1 --2- 172.21.15.119:0/2674099487 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f11fc039020 0x7f11fc03b4e0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:16:45.536 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:45.529+0000 7f121dc00700 1 --2- 172.21.15.119:0/2674099487 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1218103fa0 0x7f121819f5e0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:16:45.536 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:45.529+0000 7f121dc00700 1 -- 172.21.15.119:0/2674099487 >> 172.21.15.119:0/2674099487 conn(0x7f12180ff810 msgr2=0x7f1218105970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:16:45.536 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:45.529+0000 7f121dc00700 1 -- 172.21.15.119:0/2674099487 shutdown_connections 2024-03-28T00:16:45.537 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:45.529+0000 7f121dc00700 1 -- 172.21.15.119:0/2674099487 wait complete. 2024-03-28T00:16:46.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:16:45 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:16:46.323 INFO:tasks.cephadm:Setting crush tunables to default 2024-03-28T00:16:46.323 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 10f73b92-ec98-11ee-95d0-87774f69a715 -- ceph osd crush tunables default 2024-03-28T00:16:46.759 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/mon.a/config 2024-03-28T00:16:47.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:16:46 smithi119 ceph-mon[31518]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-28T00:16:47.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:16:46 smithi119 ceph-mon[31518]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T00:16:48.746 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:48.737+0000 7f42e4e00700 1 -- 172.21.15.119:0/238228576 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f42e0100940 msgr2=0x7f42e0100d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:16:48.746 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:48.737+0000 7f42e4e00700 1 --2- 172.21.15.119:0/238228576 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f42e0100940 0x7f42e0100d20 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f42d0009910 tx=0x7f42d0009c20 comp rx=0 tx=0).stop 2024-03-28T00:16:48.746 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:48.737+0000 7f42e4e00700 1 -- 172.21.15.119:0/238228576 shutdown_connections 2024-03-28T00:16:48.746 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:48.737+0000 7f42e4e00700 1 --2- 172.21.15.119:0/238228576 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f42e0100940 0x7f42e0100d20 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:16:48.746 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:48.737+0000 7f42e4e00700 1 -- 172.21.15.119:0/238228576 >> 172.21.15.119:0/238228576 conn(0x7f42e0077830 msgr2=0x7f42e00febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:16:48.746 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:48.737+0000 7f42e4e00700 1 -- 172.21.15.119:0/238228576 shutdown_connections 2024-03-28T00:16:48.747 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:48.737+0000 7f42e4e00700 1 -- 172.21.15.119:0/238228576 wait complete. 2024-03-28T00:16:48.747 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:48.738+0000 7f42e4e00700 1 Processor -- start 2024-03-28T00:16:48.747 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:48.738+0000 7f42e4e00700 1 -- start start 2024-03-28T00:16:48.747 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:48.738+0000 7f42e4e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f42e0100940 0x7f42e019f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:16:48.747 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:48.738+0000 7f42e4e00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42e010e3e0 con 0x7f42e0100940 2024-03-28T00:16:48.747 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:48.739+0000 7f42dde00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f42e0100940 0x7f42e019f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:16:48.747 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:48.739+0000 7f42dde00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f42e0100940 0x7f42e019f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40964/0 (socket says 172.21.15.119:40964) 2024-03-28T00:16:48.747 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:48.739+0000 7f42dde00700 1 -- 172.21.15.119:0/2890662145 learned_addr learned my addr 172.21.15.119:0/2890662145 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:16:48.747 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:48.739+0000 7f42dde00700 1 -- 172.21.15.119:0/2890662145 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42e019fb40 con 0x7f42e0100940 2024-03-28T00:16:48.747 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:48.739+0000 7f42dde00700 1 --2- 172.21.15.119:0/2890662145 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f42e0100940 0x7f42e019f600 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f42d0006010 tx=0x7f42d0005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:16:48.747 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:48.739+0000 7f42d6a00700 1 -- 172.21.15.119:0/2890662145 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f42d0026070 con 0x7f42e0100940 2024-03-28T00:16:48.747 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:48.739+0000 7f42d6a00700 1 -- 172.21.15.119:0/2890662145 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f42d000bc30 con 0x7f42e0100940 2024-03-28T00:16:48.747 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:48.740+0000 7f42e4e00700 1 -- 172.21.15.119:0/2890662145 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42e019fdd0 con 0x7f42e0100940 2024-03-28T00:16:48.748 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:48.740+0000 7f42d6a00700 1 -- 172.21.15.119:0/2890662145 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f42d00039d0 con 0x7f42e0100940 2024-03-28T00:16:48.748 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:48.740+0000 7f42e4e00700 1 -- 172.21.15.119:0/2890662145 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42e01996d0 con 0x7f42e0100940 2024-03-28T00:16:48.749 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:48.741+0000 7f42d6a00700 1 -- 172.21.15.119:0/2890662145 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 13) v1 ==== 46163+0+0 (secure 0 0 0) 0x7f42d0029070 con 0x7f42e0100940 2024-03-28T00:16:48.749 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:48.741+0000 7f42e4e00700 1 -- 172.21.15.119:0/2890662145 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42e0069dc0 con 0x7f42e0100940 2024-03-28T00:16:48.749 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:48.741+0000 7f42d6a00700 1 --2- 172.21.15.119:0/2890662145 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f42c4039020 0x7f42c403b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:16:48.749 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:48.741+0000 7f42d6a00700 1 -- 172.21.15.119:0/2890662145 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f42d001e070 con 0x7f42e0100940 2024-03-28T00:16:48.749 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:48.742+0000 7f42dd400700 1 --2- 172.21.15.119:0/2890662145 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f42c4039020 0x7f42c403b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:16:48.749 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:48.742+0000 7f42dd400700 1 --2- 172.21.15.119:0/2890662145 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f42c4039020 0x7f42c403b4e0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f42cc006fd0 tx=0x7f42cc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:16:48.753 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:48.747+0000 7f42d6a00700 1 -- 172.21.15.119:0/2890662145 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f42d0013070 con 0x7f42e0100940 2024-03-28T00:16:48.985 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:48.979+0000 7f42e4e00700 1 -- 172.21.15.119:0/2890662145 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f42e019ac00 con 0x7f42e0100940 2024-03-28T00:16:49.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:16:48 smithi119 ceph-mon[31518]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T00:16:49.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:16:48 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:16:49.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:16:48 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:16:49.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:16:48 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:16:49.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:16:48 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:16:49.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:16:48 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi165", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:16:49.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:16:48 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:16:49.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:16:48 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:16:49.852 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:49.845+0000 7f42d6a00700 1 -- 172.21.15.119:0/2890662145 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7f42d0013250 con 0x7f42e0100940 2024-03-28T00:16:49.856 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:49.849+0000 7f42e4e00700 1 -- 172.21.15.119:0/2890662145 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f42c4039020 msgr2=0x7f42c403b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:16:49.856 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:49.849+0000 7f42e4e00700 1 --2- 172.21.15.119:0/2890662145 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f42c4039020 0x7f42c403b4e0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f42cc006fd0 tx=0x7f42cc006e40 comp rx=0 tx=0).stop 2024-03-28T00:16:49.856 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:49.850+0000 7f42e4e00700 1 -- 172.21.15.119:0/2890662145 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f42e0100940 msgr2=0x7f42e019f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:16:49.856 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:49.850+0000 7f42e4e00700 1 --2- 172.21.15.119:0/2890662145 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f42e0100940 0x7f42e019f600 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f42d0006010 tx=0x7f42d0005030 comp rx=0 tx=0).stop 2024-03-28T00:16:49.857 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:49.850+0000 7f42e4e00700 1 -- 172.21.15.119:0/2890662145 shutdown_connections 2024-03-28T00:16:49.857 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:49.850+0000 7f42e4e00700 1 --2- 172.21.15.119:0/2890662145 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f42c4039020 0x7f42c403b4e0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:16:49.857 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:49.850+0000 7f42e4e00700 1 --2- 172.21.15.119:0/2890662145 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f42e0100940 0x7f42e019f600 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:16:49.857 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:49.850+0000 7f42e4e00700 1 -- 172.21.15.119:0/2890662145 >> 172.21.15.119:0/2890662145 conn(0x7f42e0077830 msgr2=0x7f42e00fe780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:16:49.857 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:49.850+0000 7f42e4e00700 1 -- 172.21.15.119:0/2890662145 shutdown_connections 2024-03-28T00:16:49.857 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:16:49.850+0000 7f42e4e00700 1 -- 172.21.15.119:0/2890662145 wait complete. 2024-03-28T00:16:49.858 INFO:teuthology.orchestra.run.smithi119.stderr:adjusted tunables profile to default 2024-03-28T00:16:50.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:16:49 smithi119 ceph-mon[31518]: Updating smithi165:/etc/ceph/ceph.conf 2024-03-28T00:16:50.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:16:49 smithi119 ceph-mon[31518]: Updating smithi165:/var/lib/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-28T00:16:50.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:16:49 smithi119 ceph-mon[31518]: Updating smithi165:/etc/ceph/ceph.client.admin.keyring 2024-03-28T00:16:50.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:16:49 smithi119 ceph-mon[31518]: Updating smithi165:/var/lib/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/config/ceph.client.admin.keyring 2024-03-28T00:16:50.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:16:49 smithi119 ceph-mon[31518]: from='client.? 172.21.15.119:0/2890662145' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-03-28T00:16:50.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:16:49 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:16:50.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:16:49 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:16:50.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:16:49 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:16:50.625 INFO:tasks.cephadm:Adding mon.a on smithi119 2024-03-28T00:16:50.625 INFO:tasks.cephadm:Adding mon.b on smithi132 2024-03-28T00:16:50.625 INFO:tasks.cephadm:Adding mon.c on smithi165 2024-03-28T00:16:50.626 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 10f73b92-ec98-11ee-95d0-87774f69a715 -- ceph orch apply mon '3;smithi119:172.21.15.119=a;smithi132:172.21.15.132=b;smithi165:172.21.15.165=c' 2024-03-28T00:16:50.863 INFO:teuthology.orchestra.run.smithi165.stderr:Inferring config /var/lib/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-28T00:16:51.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:16:50 smithi119 ceph-mon[31518]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T00:16:51.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:16:50 smithi119 ceph-mon[31518]: from='client.? 172.21.15.119:0/2890662145' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-03-28T00:16:51.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:16:50 smithi119 ceph-mon[31518]: osdmap e4: 0 total, 0 up, 0 in 2024-03-28T00:16:52.483 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:16:52.475+0000 7f5d507b0700 1 -- 172.21.15.165:0/962654904 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5d48106a70 msgr2=0x7f5d48106e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:16:52.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:16:52.475+0000 7f5d507b0700 1 --2- 172.21.15.165:0/962654904 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5d48106a70 0x7f5d48106e50 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f5d44009910 tx=0x7f5d44009c20 comp rx=0 tx=0).stop 2024-03-28T00:16:52.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:16:52.475+0000 7f5d507b0700 1 -- 172.21.15.165:0/962654904 shutdown_connections 2024-03-28T00:16:52.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:16:52.475+0000 7f5d507b0700 1 --2- 172.21.15.165:0/962654904 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5d48106a70 0x7f5d48106e50 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:16:52.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:16:52.475+0000 7f5d507b0700 1 -- 172.21.15.165:0/962654904 >> 172.21.15.165:0/962654904 conn(0x7f5d480fc5a0 msgr2=0x7f5d480fe9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:16:52.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:16:52.475+0000 7f5d507b0700 1 -- 172.21.15.165:0/962654904 shutdown_connections 2024-03-28T00:16:52.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:16:52.475+0000 7f5d507b0700 1 -- 172.21.15.165:0/962654904 wait complete. 2024-03-28T00:16:52.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:16:52.476+0000 7f5d507b0700 1 Processor -- start 2024-03-28T00:16:52.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:16:52.476+0000 7f5d507b0700 1 -- start start 2024-03-28T00:16:52.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:16:52.476+0000 7f5d507b0700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5d48106a70 0x7f5d48198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:16:52.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:16:52.476+0000 7f5d507b0700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d4810b0b0 con 0x7f5d48106a70 2024-03-28T00:16:52.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:16:52.477+0000 7f5d4e54c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5d48106a70 0x7f5d48198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:16:52.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:16:52.477+0000 7f5d4e54c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5d48106a70 0x7f5d48198500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.165:53522/0 (socket says 172.21.15.165:53522) 2024-03-28T00:16:52.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:16:52.477+0000 7f5d4e54c700 1 -- 172.21.15.165:0/2836985398 learned_addr learned my addr 172.21.15.165:0/2836985398 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-03-28T00:16:52.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:16:52.477+0000 7f5d4e54c700 1 -- 172.21.15.165:0/2836985398 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d48198a40 con 0x7f5d48106a70 2024-03-28T00:16:52.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:16:52.477+0000 7f5d4e54c700 1 --2- 172.21.15.165:0/2836985398 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5d48106a70 0x7f5d48198500 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f5d44005850 tx=0x7f5d44005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:16:52.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:16:52.478+0000 7f5d3f7fe700 1 -- 172.21.15.165:0/2836985398 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5d44026070 con 0x7f5d48106a70 2024-03-28T00:16:52.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:16:52.478+0000 7f5d507b0700 1 -- 172.21.15.165:0/2836985398 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d48198d30 con 0x7f5d48106a70 2024-03-28T00:16:52.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:16:52.478+0000 7f5d507b0700 1 -- 172.21.15.165:0/2836985398 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d481991f0 con 0x7f5d48106a70 2024-03-28T00:16:52.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:16:52.478+0000 7f5d3f7fe700 1 -- 172.21.15.165:0/2836985398 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5d4400b6f0 con 0x7f5d48106a70 2024-03-28T00:16:52.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:16:52.478+0000 7f5d3f7fe700 1 -- 172.21.15.165:0/2836985398 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5d44030060 con 0x7f5d48106a70 2024-03-28T00:16:52.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:16:52.479+0000 7f5d507b0700 1 -- 172.21.15.165:0/2836985398 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d2c005330 con 0x7f5d48106a70 2024-03-28T00:16:52.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:16:52.479+0000 7f5d3f7fe700 1 -- 172.21.15.165:0/2836985398 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 13) v1 ==== 46163+0+0 (secure 0 0 0) 0x7f5d44029070 con 0x7f5d48106a70 2024-03-28T00:16:52.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:16:52.479+0000 7f5d3f7fe700 1 --2- 172.21.15.165:0/2836985398 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f5d34038fc0 0x7f5d3403b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:16:52.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:16:52.479+0000 7f5d3f7fe700 1 -- 172.21.15.165:0/2836985398 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f5d4401e070 con 0x7f5d48106a70 2024-03-28T00:16:52.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:16:52.480+0000 7f5d4dd4b700 1 --2- 172.21.15.165:0/2836985398 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f5d34038fc0 0x7f5d3403b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:16:52.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:16:52.481+0000 7f5d4dd4b700 1 --2- 172.21.15.165:0/2836985398 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f5d34038fc0 0x7f5d3403b480 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f5d38006fd0 tx=0x7f5d38006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:16:52.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:16:52.495+0000 7f5d3f7fe700 1 -- 172.21.15.165:0/2836985398 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f5d44013070 con 0x7f5d48106a70 2024-03-28T00:16:52.708 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:16:52.701+0000 7f5d507b0700 1 -- 172.21.15.165:0/2836985398 --> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi119:172.21.15.119=a;smithi132:172.21.15.132=b;smithi165:172.21.15.165=c", "target": ["mon-mgr", ""]}) v1 -- 0x7f5d2c000c80 con 0x7f5d34038fc0 2024-03-28T00:16:52.722 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:16:52.716+0000 7f5d3f7fe700 1 -- 172.21.15.165:0/2836985398 <== mgr.14152 v2:172.21.15.119:6800/1401937096 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f5d2c000c80 con 0x7f5d34038fc0 2024-03-28T00:16:52.723 INFO:teuthology.orchestra.run.smithi165.stdout:Scheduled mon update... 2024-03-28T00:16:52.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:16:52.718+0000 7f5d507b0700 1 -- 172.21.15.165:0/2836985398 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f5d34038fc0 msgr2=0x7f5d3403b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:16:52.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:16:52.718+0000 7f5d507b0700 1 --2- 172.21.15.165:0/2836985398 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f5d34038fc0 0x7f5d3403b480 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f5d38006fd0 tx=0x7f5d38006e40 comp rx=0 tx=0).stop 2024-03-28T00:16:52.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:16:52.718+0000 7f5d507b0700 1 -- 172.21.15.165:0/2836985398 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5d48106a70 msgr2=0x7f5d48198500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:16:52.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:16:52.718+0000 7f5d507b0700 1 --2- 172.21.15.165:0/2836985398 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5d48106a70 0x7f5d48198500 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f5d44005850 tx=0x7f5d44005030 comp rx=0 tx=0).stop 2024-03-28T00:16:52.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:16:52.718+0000 7f5d507b0700 1 -- 172.21.15.165:0/2836985398 shutdown_connections 2024-03-28T00:16:52.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:16:52.718+0000 7f5d507b0700 1 --2- 172.21.15.165:0/2836985398 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f5d34038fc0 0x7f5d3403b480 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:16:52.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:16:52.718+0000 7f5d507b0700 1 --2- 172.21.15.165:0/2836985398 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5d48106a70 0x7f5d48198500 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:16:52.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:16:52.718+0000 7f5d507b0700 1 -- 172.21.15.165:0/2836985398 >> 172.21.15.165:0/2836985398 conn(0x7f5d480fc5a0 msgr2=0x7f5d480fd2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:16:52.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:16:52.718+0000 7f5d507b0700 1 -- 172.21.15.165:0/2836985398 shutdown_connections 2024-03-28T00:16:52.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:16:52.718+0000 7f5d507b0700 1 -- 172.21.15.165:0/2836985398 wait complete. 2024-03-28T00:16:53.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:16:52 smithi119 ceph-mon[31518]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T00:16:53.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:16:52 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:16:53.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:16:52 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:16:53.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:16:52 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:16:53.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:16:52 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:16:53.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:16:52 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:16:53.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:16:52 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-28T00:16:53.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:16:52 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:16:53.211 DEBUG:teuthology.orchestra.run.smithi132:mon.b> sudo journalctl -f -n 0 -u ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mon.b.service 2024-03-28T00:16:53.214 DEBUG:teuthology.orchestra.run.smithi165:mon.c> sudo journalctl -f -n 0 -u ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mon.c.service 2024-03-28T00:16:53.216 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-28T00:16:53.217 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 10f73b92-ec98-11ee-95d0-87774f69a715 -- ceph mon dump -f json 2024-03-28T00:16:54.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:16:53 smithi119 ceph-mon[31518]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi119:172.21.15.119=a;smithi132:172.21.15.132=b;smithi165:172.21.15.165=c", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T00:16:54.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:16:53 smithi119 ceph-mon[31518]: Saving service mon spec with placement smithi119:172.21.15.119=a;smithi132:172.21.15.132=b;smithi165:172.21.15.165=c;count:3 2024-03-28T00:16:54.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:16:53 smithi119 ceph-mon[31518]: Deploying daemon mon.c on smithi165 2024-03-28T00:16:55.008 INFO:teuthology.orchestra.run.smithi165.stderr:Inferring config /var/lib/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-28T00:16:55.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:16:54 smithi119 ceph-mon[31518]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T00:16:57.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 podman[36274]: 2024-03-28 00:16:57.087774117 +0000 UTC m=+0.326954055 container create e52d038dbab0e33ddba2ad62b67119b0f9fac7e01509e2ba5dd786d2c122f555 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mon-c, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-03-28T00:16:57.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:16:56 smithi119 ceph-mon[31518]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T00:16:57.458 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 podman[36274]: 2024-03-28 00:16:57.391257851 +0000 UTC m=+0.630437811 container init e52d038dbab0e33ddba2ad62b67119b0f9fac7e01509e2ba5dd786d2c122f555 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mon-c, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-03-28T00:16:57.458 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 podman[36274]: 2024-03-28 00:16:57.394548178 +0000 UTC m=+0.633728125 container start e52d038dbab0e33ddba2ad62b67119b0f9fac7e01509e2ba5dd786d2c122f555 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mon-c, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, CEPH_POINT_RELEASE=) 2024-03-28T00:16:57.458 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: set uid:gid to 167:167 (ceph:ceph) 2024-03-28T00:16:57.459 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev), process ceph-mon, pid 2 2024-03-28T00:16:57.459 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: pidfile_write: ignore empty --pid-file 2024-03-28T00:16:57.459 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: load: jerasure load: lrc 2024-03-28T00:16:57.459 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: RocksDB version: 7.9.2 2024-03-28T00:16:57.459 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Git sha 0 2024-03-28T00:16:57.459 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Compile date 2024-03-22 04:58:55 2024-03-28T00:16:57.459 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: DB SUMMARY 2024-03-28T00:16:57.459 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: DB Session ID: W1SQYIK0GN3NIUE5TQ47 2024-03-28T00:16:57.459 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: CURRENT file: CURRENT 2024-03-28T00:16:57.459 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: IDENTITY file: IDENTITY 2024-03-28T00:16:57.459 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-03-28T00:16:57.459 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-03-28T00:16:57.460 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-03-28T00:16:57.460 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.error_if_exists: 0 2024-03-28T00:16:57.460 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.create_if_missing: 0 2024-03-28T00:16:57.460 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.paranoid_checks: 1 2024-03-28T00:16:57.460 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.flush_verify_memtable_count: 1 2024-03-28T00:16:57.460 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-28T00:16:57.460 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-03-28T00:16:57.460 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.env: 0x558b0e568b20 2024-03-28T00:16:57.460 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.fs: PosixFileSystem 2024-03-28T00:16:57.460 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.info_log: 0x558b10d87920 2024-03-28T00:16:57.460 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.max_file_opening_threads: 16 2024-03-28T00:16:57.462 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.statistics: (nil) 2024-03-28T00:16:57.462 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.use_fsync: 0 2024-03-28T00:16:57.462 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.max_log_file_size: 0 2024-03-28T00:16:57.462 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-28T00:16:57.462 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-28T00:16:57.462 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.keep_log_file_num: 1000 2024-03-28T00:16:57.462 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.recycle_log_file_num: 0 2024-03-28T00:16:57.462 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.allow_fallocate: 1 2024-03-28T00:16:57.462 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.allow_mmap_reads: 0 2024-03-28T00:16:57.463 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.allow_mmap_writes: 0 2024-03-28T00:16:57.463 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.use_direct_reads: 0 2024-03-28T00:16:57.463 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-28T00:16:57.463 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.create_missing_column_families: 0 2024-03-28T00:16:57.463 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.db_log_dir: 2024-03-28T00:16:57.463 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.wal_dir: 2024-03-28T00:16:57.463 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-28T00:16:57.463 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-28T00:16:57.463 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-28T00:16:57.463 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-28T00:16:57.463 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-28T00:16:57.464 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-28T00:16:57.464 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.advise_random_on_open: 1 2024-03-28T00:16:57.464 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.db_write_buffer_size: 0 2024-03-28T00:16:57.464 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.write_buffer_manager: 0x558b10d5a1e0 2024-03-28T00:16:57.464 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-28T00:16:57.464 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-28T00:16:57.464 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-28T00:16:57.464 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.rate_limiter: (nil) 2024-03-28T00:16:57.464 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-28T00:16:57.464 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.wal_recovery_mode: 2 2024-03-28T00:16:57.464 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.enable_thread_tracking: 0 2024-03-28T00:16:57.464 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.enable_pipelined_write: 0 2024-03-28T00:16:57.465 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.unordered_write: 0 2024-03-28T00:16:57.465 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-28T00:16:57.465 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-28T00:16:57.465 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-28T00:16:57.465 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-28T00:16:57.465 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.row_cache: None 2024-03-28T00:16:57.465 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.wal_filter: None 2024-03-28T00:16:57.465 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-28T00:16:57.465 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.allow_ingest_behind: 0 2024-03-28T00:16:57.465 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.two_write_queues: 0 2024-03-28T00:16:57.465 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.manual_wal_flush: 0 2024-03-28T00:16:57.465 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.wal_compression: 0 2024-03-28T00:16:57.466 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.atomic_flush: 0 2024-03-28T00:16:57.466 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-28T00:16:57.466 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-28T00:16:57.466 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-28T00:16:57.466 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.log_readahead_size: 0 2024-03-28T00:16:57.466 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-28T00:16:57.466 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.best_efforts_recovery: 0 2024-03-28T00:16:57.466 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-28T00:16:57.466 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-28T00:16:57.466 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.allow_data_in_errors: 0 2024-03-28T00:16:57.466 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.db_host_id: __hostname__ 2024-03-28T00:16:57.466 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.enforce_single_del_contracts: true 2024-03-28T00:16:57.467 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.max_background_jobs: 2 2024-03-28T00:16:57.467 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.max_background_compactions: -1 2024-03-28T00:16:57.467 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.max_subcompactions: 1 2024-03-28T00:16:57.467 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-28T00:16:57.467 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-28T00:16:57.467 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-28T00:16:57.467 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.max_total_wal_size: 0 2024-03-28T00:16:57.467 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-28T00:16:57.467 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-28T00:16:57.467 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-28T00:16:57.467 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-28T00:16:57.467 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.max_open_files: -1 2024-03-28T00:16:57.467 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.bytes_per_sync: 0 2024-03-28T00:16:57.468 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-28T00:16:57.468 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-28T00:16:57.468 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.compaction_readahead_size: 0 2024-03-28T00:16:57.468 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.max_background_flushes: -1 2024-03-28T00:16:57.468 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Compression algorithms supported: 2024-03-28T00:16:57.468 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-28T00:16:57.468 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: kZSTD supported: 0 2024-03-28T00:16:57.468 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: kXpressCompression supported: 0 2024-03-28T00:16:57.468 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: kLZ4HCCompression supported: 1 2024-03-28T00:16:57.468 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: kZlibCompression supported: 1 2024-03-28T00:16:57.468 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: kSnappyCompression supported: 1 2024-03-28T00:16:57.468 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: kLZ4Compression supported: 1 2024-03-28T00:16:57.469 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: kBZip2Compression supported: 0 2024-03-28T00:16:57.469 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-28T00:16:57.469 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: DMutex implementation: pthread_mutex_t 2024-03-28T00:16:57.469 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-03-28T00:16:57.469 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-03-28T00:16:57.469 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-28T00:16:57.469 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.merge_operator: 2024-03-28T00:16:57.469 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.compaction_filter: None 2024-03-28T00:16:57.469 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.compaction_filter_factory: None 2024-03-28T00:16:57.469 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.sst_partitioner_factory: None 2024-03-28T00:16:57.469 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-28T00:16:57.469 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-28T00:16:57.469 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x558b10d87a20) 2024-03-28T00:16:57.470 INFO:journalctl@ceph.mon.c.smithi165.stdout: cache_index_and_filter_blocks: 1 2024-03-28T00:16:57.470 INFO:journalctl@ceph.mon.c.smithi165.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-28T00:16:57.470 INFO:journalctl@ceph.mon.c.smithi165.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-28T00:16:57.470 INFO:journalctl@ceph.mon.c.smithi165.stdout: pin_top_level_index_and_filter: 1 2024-03-28T00:16:57.470 INFO:journalctl@ceph.mon.c.smithi165.stdout: index_type: 0 2024-03-28T00:16:57.470 INFO:journalctl@ceph.mon.c.smithi165.stdout: data_block_index_type: 0 2024-03-28T00:16:57.470 INFO:journalctl@ceph.mon.c.smithi165.stdout: index_shortening: 1 2024-03-28T00:16:57.470 INFO:journalctl@ceph.mon.c.smithi165.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-28T00:16:57.470 INFO:journalctl@ceph.mon.c.smithi165.stdout: checksum: 4 2024-03-28T00:16:57.470 INFO:journalctl@ceph.mon.c.smithi165.stdout: no_block_cache: 0 2024-03-28T00:16:57.470 INFO:journalctl@ceph.mon.c.smithi165.stdout: block_cache: 0x558b10d8d090 2024-03-28T00:16:57.470 INFO:journalctl@ceph.mon.c.smithi165.stdout: block_cache_name: BinnedLRUCache 2024-03-28T00:16:57.471 INFO:journalctl@ceph.mon.c.smithi165.stdout: block_cache_options: 2024-03-28T00:16:57.471 INFO:journalctl@ceph.mon.c.smithi165.stdout: capacity : 536870912 2024-03-28T00:16:57.471 INFO:journalctl@ceph.mon.c.smithi165.stdout: num_shard_bits : 4 2024-03-28T00:16:57.471 INFO:journalctl@ceph.mon.c.smithi165.stdout: strict_capacity_limit : 0 2024-03-28T00:16:57.471 INFO:journalctl@ceph.mon.c.smithi165.stdout: high_pri_pool_ratio: 0.000 2024-03-28T00:16:57.471 INFO:journalctl@ceph.mon.c.smithi165.stdout: block_cache_compressed: (nil) 2024-03-28T00:16:57.471 INFO:journalctl@ceph.mon.c.smithi165.stdout: persistent_cache: (nil) 2024-03-28T00:16:57.471 INFO:journalctl@ceph.mon.c.smithi165.stdout: block_size: 4096 2024-03-28T00:16:57.471 INFO:journalctl@ceph.mon.c.smithi165.stdout: block_size_deviation: 10 2024-03-28T00:16:57.471 INFO:journalctl@ceph.mon.c.smithi165.stdout: block_restart_interval: 16 2024-03-28T00:16:57.471 INFO:journalctl@ceph.mon.c.smithi165.stdout: index_block_restart_interval: 1 2024-03-28T00:16:57.471 INFO:journalctl@ceph.mon.c.smithi165.stdout: metadata_block_size: 4096 2024-03-28T00:16:57.471 INFO:journalctl@ceph.mon.c.smithi165.stdout: partition_filters: 0 2024-03-28T00:16:57.472 INFO:journalctl@ceph.mon.c.smithi165.stdout: use_delta_encoding: 1 2024-03-28T00:16:57.472 INFO:journalctl@ceph.mon.c.smithi165.stdout: filter_policy: bloomfilter 2024-03-28T00:16:57.472 INFO:journalctl@ceph.mon.c.smithi165.stdout: whole_key_filtering: 1 2024-03-28T00:16:57.472 INFO:journalctl@ceph.mon.c.smithi165.stdout: verify_compression: 0 2024-03-28T00:16:57.472 INFO:journalctl@ceph.mon.c.smithi165.stdout: read_amp_bytes_per_bit: 0 2024-03-28T00:16:57.472 INFO:journalctl@ceph.mon.c.smithi165.stdout: format_version: 5 2024-03-28T00:16:57.472 INFO:journalctl@ceph.mon.c.smithi165.stdout: enable_index_compression: 1 2024-03-28T00:16:57.472 INFO:journalctl@ceph.mon.c.smithi165.stdout: block_align: 0 2024-03-28T00:16:57.472 INFO:journalctl@ceph.mon.c.smithi165.stdout: max_auto_readahead_size: 262144 2024-03-28T00:16:57.472 INFO:journalctl@ceph.mon.c.smithi165.stdout: prepopulate_block_cache: 0 2024-03-28T00:16:57.472 INFO:journalctl@ceph.mon.c.smithi165.stdout: initial_auto_readahead_size: 8192 2024-03-28T00:16:57.472 INFO:journalctl@ceph.mon.c.smithi165.stdout: num_file_reads_for_auto_readahead: 2 2024-03-28T00:16:57.473 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.write_buffer_size: 33554432 2024-03-28T00:16:57.473 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.max_write_buffer_number: 2 2024-03-28T00:16:57.473 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.compression: NoCompression 2024-03-28T00:16:57.473 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.bottommost_compression: Disabled 2024-03-28T00:16:57.473 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.prefix_extractor: nullptr 2024-03-28T00:16:57.473 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-28T00:16:57.473 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.num_levels: 7 2024-03-28T00:16:57.473 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-28T00:16:57.473 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-28T00:16:57.473 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-28T00:16:57.473 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-28T00:16:57.473 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-28T00:16:57.474 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-28T00:16:57.474 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-28T00:16:57.474 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-28T00:16:57.474 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-28T00:16:57.474 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-28T00:16:57.474 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-03-28T00:16:57.474 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-03-28T00:16:57.474 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-28T00:16:57.474 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.compression_opts.level: 32767 2024-03-28T00:16:57.474 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.compression_opts.strategy: 0 2024-03-28T00:16:57.474 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-28T00:16:57.474 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-28T00:16:57.475 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-03-28T00:16:57.475 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-28T00:16:57.475 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.compression_opts.enabled: false 2024-03-28T00:16:57.475 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-03-28T00:16:57.475 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-28T00:16:57.475 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-28T00:16:57.475 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-28T00:16:57.475 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.target_file_size_base: 67108864 2024-03-28T00:16:57.475 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-28T00:16:57.475 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-28T00:16:57.475 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-28T00:16:57.475 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-28T00:16:57.475 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-28T00:16:57.476 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-28T00:16:57.476 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-28T00:16:57.476 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-28T00:16:57.476 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-28T00:16:57.476 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-28T00:16:57.476 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-28T00:16:57.476 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-28T00:16:57.476 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-28T00:16:57.476 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-03-28T00:16:57.476 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.arena_block_size: 1048576 2024-03-28T00:16:57.476 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-28T00:16:57.476 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-28T00:16:57.476 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.disable_auto_compactions: 0 2024-03-28T00:16:57.477 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-28T00:16:57.477 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-28T00:16:57.477 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-28T00:16:57.477 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-28T00:16:57.477 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-28T00:16:57.477 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-28T00:16:57.477 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-28T00:16:57.477 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-28T00:16:57.477 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-28T00:16:57.477 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-28T00:16:57.477 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.table_properties_collectors: 2024-03-28T00:16:57.477 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.inplace_update_support: 0 2024-03-28T00:16:57.477 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-28T00:16:57.478 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-28T00:16:57.478 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-28T00:16:57.478 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-28T00:16:57.478 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.bloom_locality: 0 2024-03-28T00:16:57.478 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.max_successive_merges: 0 2024-03-28T00:16:57.478 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-28T00:16:57.478 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.paranoid_file_checks: 0 2024-03-28T00:16:57.478 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.force_consistency_checks: 1 2024-03-28T00:16:57.478 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.report_bg_io_stats: 0 2024-03-28T00:16:57.478 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.ttl: 2592000 2024-03-28T00:16:57.478 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-28T00:16:57.478 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-03-28T00:16:57.478 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-03-28T00:16:57.480 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.enable_blob_files: false 2024-03-28T00:16:57.480 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.min_blob_size: 0 2024-03-28T00:16:57.480 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.blob_file_size: 268435456 2024-03-28T00:16:57.480 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-28T00:16:57.480 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-28T00:16:57.480 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-28T00:16:57.480 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-03-28T00:16:57.480 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-03-28T00:16:57.480 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.blob_file_starting_level: 0 2024-03-28T00:16:57.480 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-03-28T00:16:57.480 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-03-28T00:16:57.481 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-03-28T00:16:57.481 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: bef2a6cf-d7c7-4abb-b6f3-3a6ab7978370 2024-03-28T00:16:57.481 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711585017426096, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-03-28T00:16:57.481 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-03-28T00:16:57.481 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711585017426690, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1622, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1711585017, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "bef2a6cf-d7c7-4abb-b6f3-3a6ab7978370", "db_session_id": "W1SQYIK0GN3NIUE5TQ47", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-03-28T00:16:57.481 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711585017426827, "job": 1, "event": "recovery_finished"} 2024-03-28T00:16:57.481 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-03-28T00:16:57.481 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-28T00:16:57.481 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x558b10e80000 2024-03-28T00:16:57.481 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: DB pointer 0x558b10e6a000 2024-03-28T00:16:57.481 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-28T00:16:57.481 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-28T00:16:57.482 INFO:journalctl@ceph.mon.c.smithi165.stdout: ** DB Stats ** 2024-03-28T00:16:57.482 INFO:journalctl@ceph.mon.c.smithi165.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-28T00:16:57.482 INFO:journalctl@ceph.mon.c.smithi165.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-28T00:16:57.482 INFO:journalctl@ceph.mon.c.smithi165.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-28T00:16:57.482 INFO:journalctl@ceph.mon.c.smithi165.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-28T00:16:57.482 INFO:journalctl@ceph.mon.c.smithi165.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-28T00:16:57.482 INFO:journalctl@ceph.mon.c.smithi165.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-28T00:16:57.482 INFO:journalctl@ceph.mon.c.smithi165.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-28T00:16:57.482 INFO:journalctl@ceph.mon.c.smithi165.stdout: 2024-03-28T00:16:57.482 INFO:journalctl@ceph.mon.c.smithi165.stdout: ** Compaction Stats [default] ** 2024-03-28T00:16:57.482 INFO:journalctl@ceph.mon.c.smithi165.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-28T00:16:57.482 INFO:journalctl@ceph.mon.c.smithi165.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-28T00:16:57.483 INFO:journalctl@ceph.mon.c.smithi165.stdout: L0 1/0 1.58 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-28T00:16:57.483 INFO:journalctl@ceph.mon.c.smithi165.stdout: Sum 1/0 1.58 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-28T00:16:57.483 INFO:journalctl@ceph.mon.c.smithi165.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-28T00:16:57.483 INFO:journalctl@ceph.mon.c.smithi165.stdout: 2024-03-28T00:16:57.483 INFO:journalctl@ceph.mon.c.smithi165.stdout: ** Compaction Stats [default] ** 2024-03-28T00:16:57.483 INFO:journalctl@ceph.mon.c.smithi165.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-28T00:16:57.483 INFO:journalctl@ceph.mon.c.smithi165.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-28T00:16:57.483 INFO:journalctl@ceph.mon.c.smithi165.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 2.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-28T00:16:57.483 INFO:journalctl@ceph.mon.c.smithi165.stdout: 2024-03-28T00:16:57.483 INFO:journalctl@ceph.mon.c.smithi165.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-28T00:16:57.483 INFO:journalctl@ceph.mon.c.smithi165.stdout: 2024-03-28T00:16:57.483 INFO:journalctl@ceph.mon.c.smithi165.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-28T00:16:57.483 INFO:journalctl@ceph.mon.c.smithi165.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-03-28T00:16:57.484 INFO:journalctl@ceph.mon.c.smithi165.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-28T00:16:57.484 INFO:journalctl@ceph.mon.c.smithi165.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-28T00:16:57.484 INFO:journalctl@ceph.mon.c.smithi165.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-28T00:16:57.484 INFO:journalctl@ceph.mon.c.smithi165.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-28T00:16:57.484 INFO:journalctl@ceph.mon.c.smithi165.stdout: Cumulative compaction: 0.00 GB write, 0.45 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-28T00:16:57.484 INFO:journalctl@ceph.mon.c.smithi165.stdout: Interval compaction: 0.00 GB write, 0.45 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-28T00:16:57.484 INFO:journalctl@ceph.mon.c.smithi165.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-28T00:16:57.484 INFO:journalctl@ceph.mon.c.smithi165.stdout: Block cache BinnedLRUCache@0x558b10d8d090#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-03-28T00:16:57.484 INFO:journalctl@ceph.mon.c.smithi165.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-03-28T00:16:57.484 INFO:journalctl@ceph.mon.c.smithi165.stdout: 2024-03-28T00:16:57.484 INFO:journalctl@ceph.mon.c.smithi165.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-28T00:16:57.484 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-03-28T00:16:57.485 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: using public_addr v2:172.21.15.165:0/0 -> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] 2024-03-28T00:16:57.485 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: starting mon.c rank -1 at public addrs [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] at bind addrs [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid 10f73b92-ec98-11ee-95d0-87774f69a715 2024-03-28T00:16:57.485 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: mon.c@-1(???) e0 preinit fsid 10f73b92-ec98-11ee-95d0-87774f69a715 2024-03-28T00:16:57.485 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: mon.c@-1(synchronizing).mds e1 new map 2024-03-28T00:16:57.485 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: mon.c@-1(synchronizing).mds e1 print_map 2024-03-28T00:16:57.485 INFO:journalctl@ceph.mon.c.smithi165.stdout: e1 2024-03-28T00:16:57.485 INFO:journalctl@ceph.mon.c.smithi165.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-28T00:16:57.485 INFO:journalctl@ceph.mon.c.smithi165.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-03-28T00:16:57.485 INFO:journalctl@ceph.mon.c.smithi165.stdout: legacy client fscid: -1 2024-03-28T00:16:57.485 INFO:journalctl@ceph.mon.c.smithi165.stdout: 2024-03-28T00:16:57.485 INFO:journalctl@ceph.mon.c.smithi165.stdout: No filesystems configured 2024-03-28T00:16:57.485 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-03-28T00:16:57.486 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-03-28T00:16:57.486 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-03-28T00:16:57.486 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-03-28T00:16:57.486 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-03-28T00:16:57.486 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-03-28T00:16:57.486 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-03-28T00:16:57.486 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-28T00:16:57.486 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-28T00:16:57.486 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-28T00:16:57.486 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: Added host smithi132 2024-03-28T00:16:57.486 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:16:57.486 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T00:16:57.486 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:16:57.487 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T00:16:57.487 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-28T00:16:57.487 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T00:16:57.487 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:16:57.487 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:16:57.487 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:16:57.487 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:16:57.487 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi132", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:16:57.487 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:16:57.487 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:16:57.487 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: Updating smithi132:/etc/ceph/ceph.conf 2024-03-28T00:16:57.487 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: Updating smithi132:/var/lib/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-28T00:16:57.488 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: Updating smithi132:/etc/ceph/ceph.client.admin.keyring 2024-03-28T00:16:57.488 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: Updating smithi132:/var/lib/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/config/ceph.client.admin.keyring 2024-03-28T00:16:57.488 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:16:57.488 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:16:57.488 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:16:57.488 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi165", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T00:16:57.488 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T00:16:57.488 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: Deploying cephadm binary to smithi165 2024-03-28T00:16:57.488 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T00:16:57.488 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T00:16:57.488 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:16:57.488 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:16:57.488 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:16:57.489 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: Added host smithi165 2024-03-28T00:16:57.489 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:16:57.489 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T00:16:57.489 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:16:57.489 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-28T00:16:57.489 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T00:16:57.489 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T00:16:57.489 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:16:57.489 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:16:57.489 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:16:57.489 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:16:57.489 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi165", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:16:57.489 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:16:57.490 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:16:57.490 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: Updating smithi165:/etc/ceph/ceph.conf 2024-03-28T00:16:57.490 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: Updating smithi165:/var/lib/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-28T00:16:57.490 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: Updating smithi165:/etc/ceph/ceph.client.admin.keyring 2024-03-28T00:16:57.490 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: Updating smithi165:/var/lib/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/config/ceph.client.admin.keyring 2024-03-28T00:16:57.490 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: from='client.? 172.21.15.119:0/2890662145' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-03-28T00:16:57.490 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:16:57.490 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:16:57.490 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:16:57.490 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T00:16:57.490 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: from='client.? 172.21.15.119:0/2890662145' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-03-28T00:16:57.490 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: osdmap e4: 0 total, 0 up, 0 in 2024-03-28T00:16:57.491 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T00:16:57.491 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:16:57.491 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:16:57.491 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:16:57.491 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:16:57.491 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:16:57.491 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-28T00:16:57.491 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:16:57.491 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi119:172.21.15.119=a;smithi132:172.21.15.132=b;smithi165:172.21.15.165=c", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T00:16:57.491 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: Saving service mon spec with placement smithi119:172.21.15.119=a;smithi132:172.21.15.132=b;smithi165:172.21.15.165=c;count:3 2024-03-28T00:16:57.491 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: Deploying daemon mon.c on smithi165 2024-03-28T00:16:57.492 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T00:16:57.492 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T00:16:57.492 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 ceph-mon[36302]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-03-28T00:16:57.608 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:16:57.400+0000 7f598bf97700 1 -- 172.21.15.165:0/4139535549 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5984102d50 msgr2=0x7f5984103130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:16:57.608 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:16:57.400+0000 7f598bf97700 1 --2- 172.21.15.165:0/4139535549 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5984102d50 0x7f5984103130 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f597c009910 tx=0x7f597c009c20 comp rx=0 tx=0).stop 2024-03-28T00:16:57.608 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:16:57.400+0000 7f598bf97700 1 -- 172.21.15.165:0/4139535549 shutdown_connections 2024-03-28T00:16:57.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:16:57.400+0000 7f598bf97700 1 --2- 172.21.15.165:0/4139535549 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5984102d50 0x7f5984103130 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:16:57.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:16:57.400+0000 7f598bf97700 1 -- 172.21.15.165:0/4139535549 >> 172.21.15.165:0/4139535549 conn(0x7f5984077830 msgr2=0x7f59840febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:16:57.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:16:57.400+0000 7f598bf97700 1 -- 172.21.15.165:0/4139535549 shutdown_connections 2024-03-28T00:16:57.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:16:57.400+0000 7f598bf97700 1 -- 172.21.15.165:0/4139535549 wait complete. 2024-03-28T00:16:57.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:16:57.401+0000 7f598bf97700 1 Processor -- start 2024-03-28T00:16:57.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:16:57.401+0000 7f598bf97700 1 -- start start 2024-03-28T00:16:57.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:16:57.401+0000 7f598bf97700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5984102d50 0x7f59841976f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:16:57.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:16:57.401+0000 7f598bf97700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f598410c1f0 con 0x7f5984102d50 2024-03-28T00:16:57.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:16:57.401+0000 7f5989d33700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5984102d50 0x7f59841976f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:16:57.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:16:57.401+0000 7f5989d33700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5984102d50 0x7f59841976f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.165:53550/0 (socket says 172.21.15.165:53550) 2024-03-28T00:16:57.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:16:57.401+0000 7f5989d33700 1 -- 172.21.15.165:0/2614598860 learned_addr learned my addr 172.21.15.165:0/2614598860 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-03-28T00:16:57.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:16:57.402+0000 7f5989d33700 1 -- 172.21.15.165:0/2614598860 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5984197c30 con 0x7f5984102d50 2024-03-28T00:16:57.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:16:57.402+0000 7f5989d33700 1 --2- 172.21.15.165:0/2614598860 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5984102d50 0x7f59841976f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f597c006010 tx=0x7f597c005050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:16:57.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:16:57.403+0000 7f597affd700 1 -- 172.21.15.165:0/2614598860 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f597c026070 con 0x7f5984102d50 2024-03-28T00:16:57.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:16:57.403+0000 7f598bf97700 1 -- 172.21.15.165:0/2614598860 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5984197ec0 con 0x7f5984102d50 2024-03-28T00:16:57.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:16:57.403+0000 7f598bf97700 1 -- 172.21.15.165:0/2614598860 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5984198320 con 0x7f5984102d50 2024-03-28T00:16:57.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:16:57.403+0000 7f597affd700 1 -- 172.21.15.165:0/2614598860 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f597c00bc30 con 0x7f5984102d50 2024-03-28T00:16:57.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:16:57.403+0000 7f597affd700 1 -- 172.21.15.165:0/2614598860 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f597c030060 con 0x7f5984102d50 2024-03-28T00:16:57.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:16:57.404+0000 7f598bf97700 1 -- 172.21.15.165:0/2614598860 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5968005330 con 0x7f5984102d50 2024-03-28T00:16:57.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:16:57.404+0000 7f597affd700 1 -- 172.21.15.165:0/2614598860 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 13) v1 ==== 46163+0+0 (secure 0 0 0) 0x7f597c029070 con 0x7f5984102d50 2024-03-28T00:16:57.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:16:57.404+0000 7f597affd700 1 --2- 172.21.15.165:0/2614598860 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f5970038fc0 0x7f597003b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:16:57.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:16:57.404+0000 7f597affd700 1 -- 172.21.15.165:0/2614598860 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f597c01e070 con 0x7f5984102d50 2024-03-28T00:16:57.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:16:57.405+0000 7f5989532700 1 --2- 172.21.15.165:0/2614598860 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f5970038fc0 0x7f597003b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:16:57.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:16:57.406+0000 7f5989532700 1 --2- 172.21.15.165:0/2614598860 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f5970038fc0 0x7f597003b480 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f5974006fd0 tx=0x7f5974006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:16:57.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:16:57.420+0000 7f597affd700 1 -- 172.21.15.165:0/2614598860 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f597c013070 con 0x7f5984102d50 2024-03-28T00:16:57.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:16:57.461+0000 7f597affd700 1 -- 172.21.15.165:0/2614598860 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7f597c004240 con 0x7f5984102d50 2024-03-28T00:16:57.730 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:16:57.723+0000 7f598bf97700 1 -- 172.21.15.165:0/2614598860 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f5968005620 con 0x7f5984102d50 2024-03-28T00:16:57.731 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 bash[36274]: e52d038dbab0e33ddba2ad62b67119b0f9fac7e01509e2ba5dd786d2c122f555 2024-03-28T00:16:57.731 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:16:57 smithi165 systemd[1]: Started Ceph mon.c for 10f73b92-ec98-11ee-95d0-87774f69a715. 2024-03-28T00:17:02.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:02.496+0000 7f597affd700 1 -- 172.21.15.165:0/2614598860 <== mon.0 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1019 (secure 0 0 0) 0x7f597c017440 con 0x7f5984102d50 2024-03-28T00:17:02.504 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-03-28T00:17:02.504 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":2,"fsid":"10f73b92-ec98-11ee-95d0-87774f69a715","modified":"2024-03-28T00:16:57.461138Z","created":"2024-03-28T00:14:53.446019Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:3300","nonce":0},{"type":"v1","addr":"172.21.15.165:6789","nonce":0}]},"addr":"172.21.15.165:6789/0","public_addr":"172.21.15.165:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-03-28T00:17:02.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:02.499+0000 7f598bf97700 1 -- 172.21.15.165:0/2614598860 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f5970038fc0 msgr2=0x7f597003b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:17:02.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:02.499+0000 7f598bf97700 1 --2- 172.21.15.165:0/2614598860 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f5970038fc0 0x7f597003b480 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f5974006fd0 tx=0x7f5974006e40 comp rx=0 tx=0).stop 2024-03-28T00:17:02.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:02.499+0000 7f598bf97700 1 -- 172.21.15.165:0/2614598860 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5984102d50 msgr2=0x7f59841976f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:17:02.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:02.499+0000 7f598bf97700 1 --2- 172.21.15.165:0/2614598860 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5984102d50 0x7f59841976f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f597c006010 tx=0x7f597c005050 comp rx=0 tx=0).stop 2024-03-28T00:17:02.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:02.499+0000 7f598bf97700 1 -- 172.21.15.165:0/2614598860 shutdown_connections 2024-03-28T00:17:02.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:02.499+0000 7f598bf97700 1 --2- 172.21.15.165:0/2614598860 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f5970038fc0 0x7f597003b480 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:17:02.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:02.499+0000 7f598bf97700 1 --2- 172.21.15.165:0/2614598860 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5984102d50 0x7f59841976f0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:17:02.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:02.499+0000 7f598bf97700 1 -- 172.21.15.165:0/2614598860 >> 172.21.15.165:0/2614598860 conn(0x7f5984077830 msgr2=0x7f5984103b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:17:02.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:02.500+0000 7f598bf97700 1 -- 172.21.15.165:0/2614598860 shutdown_connections 2024-03-28T00:17:02.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:02.500+0000 7f598bf97700 1 -- 172.21.15.165:0/2614598860 wait complete. 2024-03-28T00:17:02.507 INFO:teuthology.orchestra.run.smithi165.stderr:dumped monmap epoch 2 2024-03-28T00:17:02.857 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:02 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-28T00:17:02.858 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:02 smithi165 ceph-mon[36302]: mon.a calling monitor election 2024-03-28T00:17:02.858 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:02 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-28T00:17:02.858 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:02 smithi165 ceph-mon[36302]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T00:17:02.858 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:02 smithi165 ceph-mon[36302]: from='client.? 172.21.15.165:0/2614598860' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-28T00:17:02.858 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:02 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-28T00:17:02.858 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:02 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-28T00:17:02.858 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:02 smithi165 ceph-mon[36302]: mon.c calling monitor election 2024-03-28T00:17:02.858 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:02 smithi165 ceph-mon[36302]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T00:17:02.858 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:02 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-28T00:17:02.858 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:02 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-28T00:17:02.859 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:02 smithi165 ceph-mon[36302]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T00:17:02.859 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:02 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-28T00:17:02.859 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:02 smithi165 ceph-mon[36302]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-28T00:17:02.859 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:02 smithi165 ceph-mon[36302]: monmap e2: 2 mons at {a=[v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0],c=[v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-28T00:17:02.859 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:02 smithi165 ceph-mon[36302]: fsmap 2024-03-28T00:17:02.859 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:02 smithi165 ceph-mon[36302]: osdmap e4: 0 total, 0 up, 0 in 2024-03-28T00:17:02.859 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:02 smithi165 ceph-mon[36302]: mgrmap e13: x(active, since 58s) 2024-03-28T00:17:02.859 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:02 smithi165 ceph-mon[36302]: overall HEALTH_OK 2024-03-28T00:17:02.859 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:02 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:17:02.859 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:02 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:17:02.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:02 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-28T00:17:02.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:02 smithi119 ceph-mon[31518]: mon.a calling monitor election 2024-03-28T00:17:02.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:02 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-28T00:17:02.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:02 smithi119 ceph-mon[31518]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T00:17:02.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:02 smithi119 ceph-mon[31518]: from='client.? 172.21.15.165:0/2614598860' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-28T00:17:02.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:02 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-28T00:17:02.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:02 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-28T00:17:02.907 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:02 smithi119 ceph-mon[31518]: mon.c calling monitor election 2024-03-28T00:17:02.907 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:02 smithi119 ceph-mon[31518]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T00:17:02.907 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:02 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-28T00:17:02.907 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:02 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-28T00:17:02.907 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:02 smithi119 ceph-mon[31518]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T00:17:02.907 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:02 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-28T00:17:02.907 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:02 smithi119 ceph-mon[31518]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-28T00:17:02.907 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:02 smithi119 ceph-mon[31518]: monmap e2: 2 mons at {a=[v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0],c=[v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-28T00:17:02.909 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:02 smithi119 ceph-mon[31518]: fsmap 2024-03-28T00:17:02.910 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:02 smithi119 ceph-mon[31518]: osdmap e4: 0 total, 0 up, 0 in 2024-03-28T00:17:02.910 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:02 smithi119 ceph-mon[31518]: mgrmap e13: x(active, since 58s) 2024-03-28T00:17:02.910 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:02 smithi119 ceph-mon[31518]: overall HEALTH_OK 2024-03-28T00:17:02.910 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:02 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:17:02.910 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:02 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:17:03.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:03 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:17:03.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:03 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-28T00:17:03.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:03 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:17:03.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:03 smithi165 ceph-mon[36302]: Deploying daemon mon.b on smithi132 2024-03-28T00:17:03.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:03 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-28T00:17:03.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:03 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:17:03.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:03 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-28T00:17:03.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:03 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:17:03.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:03 smithi119 ceph-mon[31518]: Deploying daemon mon.b on smithi132 2024-03-28T00:17:03.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:03 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-28T00:17:04.139 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-28T00:17:04.139 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 10f73b92-ec98-11ee-95d0-87774f69a715 -- ceph mon dump -f json 2024-03-28T00:17:04.386 INFO:teuthology.orchestra.run.smithi165.stderr:Inferring config /var/lib/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/mon.c/config 2024-03-28T00:17:04.785 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:04 smithi165 ceph-mon[36302]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T00:17:04.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:04 smithi119 ceph-mon[31518]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T00:17:04.905 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 00:17:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-x[31752]: 2024-03-28T00:17:04.461+0000 7f5e3d200700 -1 mgr.server handle_report got status from non-daemon mon.c 2024-03-28T00:17:05.802 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:05.793+0000 7fe4864ff700 1 -- 172.21.15.165:0/824798959 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe480100df0 msgr2=0x7fe4801011d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:17:05.802 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:05.793+0000 7fe4864ff700 1 --2- 172.21.15.165:0/824798959 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe480100df0 0x7fe4801011d0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fe470009910 tx=0x7fe470009c20 comp rx=0 tx=0).stop 2024-03-28T00:17:05.802 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:05.793+0000 7fe4864ff700 1 -- 172.21.15.165:0/824798959 shutdown_connections 2024-03-28T00:17:05.802 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:05.793+0000 7fe4864ff700 1 --2- 172.21.15.165:0/824798959 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe480100df0 0x7fe4801011d0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:17:05.802 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:05.793+0000 7fe4864ff700 1 -- 172.21.15.165:0/824798959 >> 172.21.15.165:0/824798959 conn(0x7fe4800fc6a0 msgr2=0x7fe4800feac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:17:05.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:05.793+0000 7fe4864ff700 1 -- 172.21.15.165:0/824798959 shutdown_connections 2024-03-28T00:17:05.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:05.793+0000 7fe4864ff700 1 -- 172.21.15.165:0/824798959 wait complete. 2024-03-28T00:17:05.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:05.793+0000 7fe4864ff700 1 Processor -- start 2024-03-28T00:17:05.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:05.794+0000 7fe4864ff700 1 -- start start 2024-03-28T00:17:05.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:05.794+0000 7fe4864ff700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe480100df0 0x7fe48019d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:17:05.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:05.794+0000 7fe4864ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe48019db80 0x7fe480197710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:17:05.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:05.794+0000 7fe4864ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe48010b1c0 con 0x7fe48019db80 2024-03-28T00:17:05.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:05.794+0000 7fe4864ff700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe48010b040 con 0x7fe480100df0 2024-03-28T00:17:05.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:05.794+0000 7fe47ffff700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe480100df0 0x7fe48019d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:17:05.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:05.794+0000 7fe47ffff700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe480100df0 0x7fe48019d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:33694/0 (socket says 172.21.15.165:33694) 2024-03-28T00:17:05.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:05.794+0000 7fe47ffff700 1 -- 172.21.15.165:0/774973391 learned_addr learned my addr 172.21.15.165:0/774973391 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-03-28T00:17:05.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:05.794+0000 7fe47ffff700 1 -- 172.21.15.165:0/774973391 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe480100df0 msgr2=0x7fe48019d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-03-28T00:17:05.804 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:05.794+0000 7fe47ffff700 1 -- 172.21.15.165:0/774973391 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe480100df0 msgr2=0x7fe48019d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-28T00:17:05.804 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:05.794+0000 7fe47ffff700 1 --2- 172.21.15.165:0/774973391 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe480100df0 0x7fe48019d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-28T00:17:05.804 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:05.794+0000 7fe47ffff700 1 --2- 172.21.15.165:0/774973391 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe480100df0 0x7fe48019d640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-28T00:17:05.804 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:05.794+0000 7fe47f7fe700 1 --2- 172.21.15.165:0/774973391 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe48019db80 0x7fe480197710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:17:05.804 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:05.795+0000 7fe47f7fe700 1 -- 172.21.15.165:0/774973391 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe480100df0 msgr2=0x7fe48019d640 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-28T00:17:05.804 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:05.795+0000 7fe47f7fe700 1 --2- 172.21.15.165:0/774973391 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe480100df0 0x7fe48019d640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:17:05.804 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:05.795+0000 7fe47f7fe700 1 -- 172.21.15.165:0/774973391 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe48019e100 con 0x7fe48019db80 2024-03-28T00:17:05.804 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:05.795+0000 7fe47f7fe700 1 --2- 172.21.15.165:0/774973391 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe48019db80 0x7fe480197710 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe47400b4d0 tx=0x7fe47400b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:17:05.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:05.796+0000 7fe47d7fa700 1 -- 172.21.15.165:0/774973391 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7fe474011630 con 0x7fe48019db80 2024-03-28T00:17:05.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:05.796+0000 7fe4864ff700 1 -- 172.21.15.165:0/774973391 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe480197cb0 con 0x7fe48019db80 2024-03-28T00:17:05.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:05.796+0000 7fe4864ff700 1 -- 172.21.15.165:0/774973391 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe480198240 con 0x7fe48019db80 2024-03-28T00:17:05.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:05.796+0000 7fe47d7fa700 1 -- 172.21.15.165:0/774973391 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe474011cb0 con 0x7fe48019db80 2024-03-28T00:17:05.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:05.796+0000 7fe47d7fa700 1 -- 172.21.15.165:0/774973391 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7fe4740193b0 con 0x7fe48019db80 2024-03-28T00:17:05.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:05.797+0000 7fe4864ff700 1 -- 172.21.15.165:0/774973391 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4801022d0 con 0x7fe48019db80 2024-03-28T00:17:05.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:05.797+0000 7fe47d7fa700 1 -- 172.21.15.165:0/774973391 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 13) v1 ==== 46163+0+0 (secure 0 0 0) 0x7fe474018020 con 0x7fe48019db80 2024-03-28T00:17:05.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:05.797+0000 7fe47d7fa700 1 --2- 172.21.15.165:0/774973391 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fe468038d60 0x7fe46803b220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:17:05.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:05.798+0000 7fe47d7fa700 1 -- 172.21.15.165:0/774973391 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fe47404be10 con 0x7fe48019db80 2024-03-28T00:17:05.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:05.798+0000 7fe47ffff700 1 --2- 172.21.15.165:0/774973391 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fe468038d60 0x7fe46803b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:17:05.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:05.799+0000 7fe47ffff700 1 --2- 172.21.15.165:0/774973391 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fe468038d60 0x7fe46803b220 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fe47000b5c0 tx=0x7fe470009650 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:17:05.820 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:05.814+0000 7fe47d7fa700 1 -- 172.21.15.165:0/774973391 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fe47401f020 con 0x7fe48019db80 2024-03-28T00:17:06.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:06.085+0000 7fe4864ff700 1 -- 172.21.15.165:0/774973391 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fe4800637c0 con 0x7fe48019db80 2024-03-28T00:17:06.093 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:06.087+0000 7fe47d7fa700 1 -- 172.21.15.165:0/774973391 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1019 (secure 0 0 0) 0x7fe47404a720 con 0x7fe48019db80 2024-03-28T00:17:06.093 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-03-28T00:17:06.094 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":2,"fsid":"10f73b92-ec98-11ee-95d0-87774f69a715","modified":"2024-03-28T00:16:57.461138Z","created":"2024-03-28T00:14:53.446019Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:3300","nonce":0},{"type":"v1","addr":"172.21.15.165:6789","nonce":0}]},"addr":"172.21.15.165:6789/0","public_addr":"172.21.15.165:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-03-28T00:17:06.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:06.089+0000 7fe4864ff700 1 -- 172.21.15.165:0/774973391 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fe468038d60 msgr2=0x7fe46803b220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:17:06.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:06.089+0000 7fe4864ff700 1 --2- 172.21.15.165:0/774973391 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fe468038d60 0x7fe46803b220 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fe47000b5c0 tx=0x7fe470009650 comp rx=0 tx=0).stop 2024-03-28T00:17:06.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:06.089+0000 7fe4864ff700 1 -- 172.21.15.165:0/774973391 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe48019db80 msgr2=0x7fe480197710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:17:06.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:06.089+0000 7fe4864ff700 1 --2- 172.21.15.165:0/774973391 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe48019db80 0x7fe480197710 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe47400b4d0 tx=0x7fe47400b890 comp rx=0 tx=0).stop 2024-03-28T00:17:06.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:06.089+0000 7fe4864ff700 1 -- 172.21.15.165:0/774973391 shutdown_connections 2024-03-28T00:17:06.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:06.089+0000 7fe4864ff700 1 --2- 172.21.15.165:0/774973391 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fe468038d60 0x7fe46803b220 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:17:06.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:06.089+0000 7fe4864ff700 1 --2- 172.21.15.165:0/774973391 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe480100df0 0x7fe48019d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:17:06.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:06.089+0000 7fe4864ff700 1 --2- 172.21.15.165:0/774973391 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe48019db80 0x7fe480197710 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:17:06.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:06.089+0000 7fe4864ff700 1 -- 172.21.15.165:0/774973391 >> 172.21.15.165:0/774973391 conn(0x7fe4800fc6a0 msgr2=0x7fe48006a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:17:06.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:06.089+0000 7fe4864ff700 1 -- 172.21.15.165:0/774973391 shutdown_connections 2024-03-28T00:17:06.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:06.089+0000 7fe4864ff700 1 -- 172.21.15.165:0/774973391 wait complete. 2024-03-28T00:17:06.096 INFO:teuthology.orchestra.run.smithi165.stderr:dumped monmap epoch 2 2024-03-28T00:17:06.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:06 smithi132 systemd[1]: Started Ceph mon.b for 10f73b92-ec98-11ee-95d0-87774f69a715. 2024-03-28T00:17:07.612 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-28T00:17:07.613 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 10f73b92-ec98-11ee-95d0-87774f69a715 -- ceph mon dump -f json 2024-03-28T00:17:07.859 INFO:teuthology.orchestra.run.smithi165.stderr:Inferring config /var/lib/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/mon.c/config 2024-03-28T00:17:11.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:11 smithi119 ceph-mon[31518]: mon.c calling monitor election 2024-03-28T00:17:11.907 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:11 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-28T00:17:11.907 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:11 smithi119 ceph-mon[31518]: mon.a calling monitor election 2024-03-28T00:17:11.907 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:11 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-28T00:17:11.907 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:11 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-28T00:17:11.907 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:11 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-28T00:17:11.907 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:11 smithi119 ceph-mon[31518]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T00:17:11.907 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:11 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-28T00:17:11.907 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:11 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-28T00:17:11.907 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:11 smithi119 ceph-mon[31518]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T00:17:11.907 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:11 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-28T00:17:11.907 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:11 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-28T00:17:11.907 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:11 smithi119 ceph-mon[31518]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-28T00:17:11.907 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:11 smithi119 ceph-mon[31518]: monmap e3: 3 mons at {a=[v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0],b=[v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0],c=[v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-28T00:17:11.907 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:11 smithi119 ceph-mon[31518]: fsmap 2024-03-28T00:17:11.908 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:11 smithi119 ceph-mon[31518]: osdmap e4: 0 total, 0 up, 0 in 2024-03-28T00:17:11.908 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:11 smithi119 ceph-mon[31518]: mgrmap e13: x(active, since 67s) 2024-03-28T00:17:11.908 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:11 smithi119 ceph-mon[31518]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-03-28T00:17:11.908 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:11 smithi119 ceph-mon[31518]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-03-28T00:17:11.908 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:11 smithi119 ceph-mon[31518]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-03-28T00:17:11.908 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:11 smithi119 ceph-mon[31518]: mon.b (rank 2) addr [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] is down (out of quorum) 2024-03-28T00:17:11.908 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:11 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:17:11.908 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:11 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:17:12.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:11 smithi165 ceph-mon[36302]: mon.c calling monitor election 2024-03-28T00:17:12.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:11 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-28T00:17:12.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:11 smithi165 ceph-mon[36302]: mon.a calling monitor election 2024-03-28T00:17:12.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:11 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-28T00:17:12.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:11 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-28T00:17:12.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:11 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-28T00:17:12.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:11 smithi165 ceph-mon[36302]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T00:17:12.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:11 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-28T00:17:12.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:11 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-28T00:17:12.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:11 smithi165 ceph-mon[36302]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T00:17:12.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:11 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-28T00:17:12.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:11 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-28T00:17:12.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:11 smithi165 ceph-mon[36302]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-28T00:17:12.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:11 smithi165 ceph-mon[36302]: monmap e3: 3 mons at {a=[v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0],b=[v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0],c=[v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-28T00:17:12.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:11 smithi165 ceph-mon[36302]: fsmap 2024-03-28T00:17:12.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:11 smithi165 ceph-mon[36302]: osdmap e4: 0 total, 0 up, 0 in 2024-03-28T00:17:12.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:11 smithi165 ceph-mon[36302]: mgrmap e13: x(active, since 67s) 2024-03-28T00:17:12.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:11 smithi165 ceph-mon[36302]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-03-28T00:17:12.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:11 smithi165 ceph-mon[36302]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-03-28T00:17:12.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:11 smithi165 ceph-mon[36302]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-03-28T00:17:12.135 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:11 smithi165 ceph-mon[36302]: mon.b (rank 2) addr [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] is down (out of quorum) 2024-03-28T00:17:12.135 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:11 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:17:12.135 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:11 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:17:12.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:12.289+0000 7f6323a84700 1 -- 172.21.15.165:0/1625938852 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f631c100db0 msgr2=0x7f6308005610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:17:12.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:12.289+0000 7f6323a84700 1 --2- 172.21.15.165:0/1625938852 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f631c100db0 0x7f6308005610 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f630c005fa0 tx=0x7f630c00ffc0 comp rx=0 tx=0).stop 2024-03-28T00:17:12.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:12.289+0000 7f6323a84700 1 -- 172.21.15.165:0/1625938852 shutdown_connections 2024-03-28T00:17:12.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:12.289+0000 7f6323a84700 1 --2- 172.21.15.165:0/1625938852 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f631c100db0 0x7f6308005610 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:17:12.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:12.289+0000 7f6323a84700 1 -- 172.21.15.165:0/1625938852 >> 172.21.15.165:0/1625938852 conn(0x7f631c0fc640 msgr2=0x7f631c0fea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:17:12.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:12.289+0000 7f6323a84700 1 -- 172.21.15.165:0/1625938852 shutdown_connections 2024-03-28T00:17:12.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:12.289+0000 7f6323a84700 1 -- 172.21.15.165:0/1625938852 wait complete. 2024-03-28T00:17:12.298 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:12.290+0000 7f6323a84700 1 Processor -- start 2024-03-28T00:17:12.298 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:12.290+0000 7f6323a84700 1 -- start start 2024-03-28T00:17:12.298 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:12.290+0000 7f6323a84700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f631c100db0 0x7f631c19c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:17:12.298 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:12.290+0000 7f6323a84700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f631c19cb10 0x7f631c1966a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:17:12.298 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:12.290+0000 7f6323a84700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f631c196be0 0x7f631c197090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:17:12.298 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:12.291+0000 7f6323a84700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f631c108b90 con 0x7f631c196be0 2024-03-28T00:17:12.298 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:12.291+0000 7f6323a84700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f631c108a10 con 0x7f631c100db0 2024-03-28T00:17:12.298 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:12.291+0000 7f6323a84700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f631c108d10 con 0x7f631c19cb10 2024-03-28T00:17:12.298 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:12.291+0000 7f632101f700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f631c19cb10 0x7f631c1966a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:17:12.299 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:12.291+0000 7f632101f700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f631c19cb10 0x7f631c1966a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:52692/0 (socket says 172.21.15.165:52692) 2024-03-28T00:17:12.299 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:12.291+0000 7f632101f700 1 -- 172.21.15.165:0/3342360596 learned_addr learned my addr 172.21.15.165:0/3342360596 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-03-28T00:17:12.299 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:12.291+0000 7f6321820700 1 --2- 172.21.15.165:0/3342360596 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f631c100db0 0x7f631c19c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:17:12.299 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:12.291+0000 7f632101f700 1 -- 172.21.15.165:0/3342360596 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f631c19cb10 msgr2=0x7f631c1966a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-03-28T00:17:12.299 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:12.291+0000 7f632101f700 1 -- 172.21.15.165:0/3342360596 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f631c19cb10 msgr2=0x7f631c1966a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-28T00:17:12.299 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:12.291+0000 7f632101f700 1 --2- 172.21.15.165:0/3342360596 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f631c19cb10 0x7f631c1966a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-28T00:17:12.299 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:12.291+0000 7f6321820700 1 -- 172.21.15.165:0/3342360596 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f631c100db0 msgr2=0x7f631c19c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-03-28T00:17:12.299 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:12.291+0000 7f6321820700 1 -- 172.21.15.165:0/3342360596 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f631c100db0 msgr2=0x7f631c19c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-28T00:17:12.299 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:12.291+0000 7f632101f700 1 --2- 172.21.15.165:0/3342360596 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f631c19cb10 0x7f631c1966a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-28T00:17:12.299 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:12.291+0000 7f6321820700 1 --2- 172.21.15.165:0/3342360596 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f631c100db0 0x7f631c19c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-28T00:17:12.299 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:12.291+0000 7f6321820700 1 --2- 172.21.15.165:0/3342360596 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f631c100db0 0x7f631c19c5d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-28T00:17:12.299 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:12.291+0000 7f6322021700 1 --2- 172.21.15.165:0/3342360596 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f631c196be0 0x7f631c197090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:17:12.299 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:12.291+0000 7f6322021700 1 -- 172.21.15.165:0/3342360596 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f631c19cb10 msgr2=0x7f631c1966a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-28T00:17:12.299 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:12.291+0000 7f6322021700 1 --2- 172.21.15.165:0/3342360596 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f631c19cb10 0x7f631c1966a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:17:12.300 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:12.291+0000 7f6322021700 1 -- 172.21.15.165:0/3342360596 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f631c100db0 msgr2=0x7f631c19c5d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-28T00:17:12.300 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:12.292+0000 7f6322021700 1 --2- 172.21.15.165:0/3342360596 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f631c100db0 0x7f631c19c5d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:17:12.300 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:12.292+0000 7f6322021700 1 -- 172.21.15.165:0/3342360596 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f631c1a3380 con 0x7f631c196be0 2024-03-28T00:17:12.300 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:12.292+0000 7f6322021700 1 --2- 172.21.15.165:0/3342360596 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f631c196be0 0x7f631c197090 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f631400bd50 tx=0x7f631400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:17:12.300 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:12.292+0000 7f6312ffd700 1 -- 172.21.15.165:0/3342360596 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6314019070 con 0x7f631c196be0 2024-03-28T00:17:12.300 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:12.292+0000 7f6312ffd700 1 -- 172.21.15.165:0/3342360596 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f631400cd10 con 0x7f631c196be0 2024-03-28T00:17:12.300 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:12.292+0000 7f6323a84700 1 -- 172.21.15.165:0/3342360596 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f631c1a3610 con 0x7f631c196be0 2024-03-28T00:17:12.300 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:12.292+0000 7f6323a84700 1 -- 172.21.15.165:0/3342360596 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f631c069d10 con 0x7f631c196be0 2024-03-28T00:17:12.300 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:12.292+0000 7f6312ffd700 1 -- 172.21.15.165:0/3342360596 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63140127e0 con 0x7f631c196be0 2024-03-28T00:17:12.301 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:12.293+0000 7f6323a84700 1 -- 172.21.15.165:0/3342360596 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f631c102290 con 0x7f631c196be0 2024-03-28T00:17:12.301 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:12.294+0000 7f6312ffd700 1 -- 172.21.15.165:0/3342360596 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 13) v1 ==== 46163+0+0 (secure 0 0 0) 0x7f631401c070 con 0x7f631c196be0 2024-03-28T00:17:12.301 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:12.294+0000 7f6312ffd700 1 --2- 172.21.15.165:0/3342360596 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f6300039450 0x7f630003b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:17:12.301 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:12.294+0000 7f6312ffd700 1 -- 172.21.15.165:0/3342360596 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f631404b8c0 con 0x7f631c196be0 2024-03-28T00:17:12.301 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:12.294+0000 7f6321820700 1 --2- 172.21.15.165:0/3342360596 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f6300039450 0x7f630003b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:17:12.301 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:12.295+0000 7f6321820700 1 --2- 172.21.15.165:0/3342360596 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f6300039450 0x7f630003b910 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f630c005cd0 tx=0x7f630c005bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:17:12.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:12.304+0000 7f6312ffd700 1 -- 172.21.15.165:0/3342360596 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f631404a2d0 con 0x7f631c196be0 2024-03-28T00:17:12.628 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:12.623+0000 7f6323a84700 1 -- 172.21.15.165:0/3342360596 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f631c06a880 con 0x7f631c196be0 2024-03-28T00:17:12.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:12.624+0000 7f6312ffd700 1 -- 172.21.15.165:0/3342360596 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) v1 ==== 95+0+1287 (secure 0 0 0) 0x7f631404a650 con 0x7f631c196be0 2024-03-28T00:17:12.630 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-03-28T00:17:12.630 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":3,"fsid":"10f73b92-ec98-11ee-95d0-87774f69a715","modified":"2024-03-28T00:17:06.551955Z","created":"2024-03-28T00:14:53.446019Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:3300","nonce":0},{"type":"v1","addr":"172.21.15.165:6789","nonce":0}]},"addr":"172.21.15.165:6789/0","public_addr":"172.21.15.165:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:3300","nonce":0},{"type":"v1","addr":"172.21.15.132:6789","nonce":0}]},"addr":"172.21.15.132:6789/0","public_addr":"172.21.15.132:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-03-28T00:17:12.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:12.626+0000 7f6323a84700 1 -- 172.21.15.165:0/3342360596 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f6300039450 msgr2=0x7f630003b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:17:12.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:12.626+0000 7f6323a84700 1 --2- 172.21.15.165:0/3342360596 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f6300039450 0x7f630003b910 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f630c005cd0 tx=0x7f630c005bc0 comp rx=0 tx=0).stop 2024-03-28T00:17:12.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:12.626+0000 7f6323a84700 1 -- 172.21.15.165:0/3342360596 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f631c196be0 msgr2=0x7f631c197090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:17:12.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:12.626+0000 7f6323a84700 1 --2- 172.21.15.165:0/3342360596 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f631c196be0 0x7f631c197090 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f631400bd50 tx=0x7f631400be30 comp rx=0 tx=0).stop 2024-03-28T00:17:12.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:12.626+0000 7f6323a84700 1 -- 172.21.15.165:0/3342360596 shutdown_connections 2024-03-28T00:17:12.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:12.626+0000 7f6323a84700 1 --2- 172.21.15.165:0/3342360596 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f6300039450 0x7f630003b910 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:17:12.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:12.626+0000 7f6323a84700 1 --2- 172.21.15.165:0/3342360596 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f631c196be0 0x7f631c197090 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:17:12.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:12.626+0000 7f6323a84700 1 --2- 172.21.15.165:0/3342360596 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f631c100db0 0x7f631c19c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:17:12.634 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:12.626+0000 7f6323a84700 1 --2- 172.21.15.165:0/3342360596 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f631c19cb10 0x7f631c1966a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:17:12.634 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:12.626+0000 7f6323a84700 1 -- 172.21.15.165:0/3342360596 >> 172.21.15.165:0/3342360596 conn(0x7f631c0fc640 msgr2=0x7f631c0fd7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:17:12.634 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:12.626+0000 7f6323a84700 1 -- 172.21.15.165:0/3342360596 shutdown_connections 2024-03-28T00:17:12.634 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:12.626+0000 7f6323a84700 1 -- 172.21.15.165:0/3342360596 wait complete. 2024-03-28T00:17:12.634 INFO:teuthology.orchestra.run.smithi165.stderr:dumped monmap epoch 3 2024-03-28T00:17:12.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:12 smithi165 ceph-mon[36302]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T00:17:12.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:12 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:17:12.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:12 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:17:12.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:12 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:17:12.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:12 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-28T00:17:12.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:12 smithi165 ceph-mon[36302]: from='client.? 172.21.15.165:0/3342360596' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-28T00:17:12.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:12 smithi119 ceph-mon[31518]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T00:17:12.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:12 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:17:12.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:12 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:17:12.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:12 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:17:12.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:12 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-28T00:17:12.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:12 smithi119 ceph-mon[31518]: from='client.? 172.21.15.165:0/3342360596' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-28T00:17:13.268 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-03-28T00:17:13.269 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 10f73b92-ec98-11ee-95d0-87774f69a715 -- ceph config generate-minimal-conf 2024-03-28T00:17:13.675 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/mon.a/config 2024-03-28T00:17:13.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:13 smithi132 ceph-mon[35941]: mon.c calling monitor election 2024-03-28T00:17:13.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:13 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-28T00:17:13.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:13 smithi132 ceph-mon[35941]: mon.a calling monitor election 2024-03-28T00:17:13.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:13 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-28T00:17:13.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:13 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-28T00:17:13.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:13 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-28T00:17:13.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:13 smithi132 ceph-mon[35941]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T00:17:13.954 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:13 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-28T00:17:13.954 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:13 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-28T00:17:13.954 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:13 smithi132 ceph-mon[35941]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T00:17:13.954 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:13 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-28T00:17:13.954 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:13 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-28T00:17:13.954 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:13 smithi132 ceph-mon[35941]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-28T00:17:13.954 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:13 smithi132 ceph-mon[35941]: monmap e3: 3 mons at {a=[v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0],b=[v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0],c=[v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-28T00:17:13.954 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:13 smithi132 ceph-mon[35941]: fsmap 2024-03-28T00:17:13.954 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:13 smithi132 ceph-mon[35941]: osdmap e4: 0 total, 0 up, 0 in 2024-03-28T00:17:13.954 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:13 smithi132 ceph-mon[35941]: mgrmap e13: x(active, since 67s) 2024-03-28T00:17:13.954 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:13 smithi132 ceph-mon[35941]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-03-28T00:17:13.954 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:13 smithi132 ceph-mon[35941]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-03-28T00:17:13.955 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:13 smithi132 ceph-mon[35941]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-03-28T00:17:13.955 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:13 smithi132 ceph-mon[35941]: mon.b (rank 2) addr [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] is down (out of quorum) 2024-03-28T00:17:13.955 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:13 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:17:13.955 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:13 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:17:13.955 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:13 smithi132 ceph-mon[35941]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T00:17:13.955 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:13 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:17:13.955 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:13 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:17:13.955 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:13 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:17:13.955 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:13 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-28T00:17:13.955 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:13 smithi132 ceph-mon[35941]: from='client.? 172.21.15.165:0/3342360596' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-28T00:17:14.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:14 smithi119 ceph-mon[31518]: mon.b calling monitor election 2024-03-28T00:17:14.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:14 smithi119 ceph-mon[31518]: mon.b calling monitor election 2024-03-28T00:17:14.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:14 smithi119 ceph-mon[31518]: mon.c calling monitor election 2024-03-28T00:17:14.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:14 smithi119 ceph-mon[31518]: mon.a calling monitor election 2024-03-28T00:17:14.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:14 smithi119 ceph-mon[31518]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-03-28T00:17:14.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:14 smithi119 ceph-mon[31518]: monmap e3: 3 mons at {a=[v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0],b=[v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0],c=[v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-28T00:17:14.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:14 smithi119 ceph-mon[31518]: fsmap 2024-03-28T00:17:14.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:14 smithi119 ceph-mon[31518]: osdmap e4: 0 total, 0 up, 0 in 2024-03-28T00:17:14.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:14 smithi119 ceph-mon[31518]: mgrmap e13: x(active, since 70s) 2024-03-28T00:17:14.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:14 smithi119 ceph-mon[31518]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-03-28T00:17:14.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:14 smithi119 ceph-mon[31518]: Cluster is now healthy 2024-03-28T00:17:14.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:14 smithi119 ceph-mon[31518]: overall HEALTH_OK 2024-03-28T00:17:14.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:14 smithi119 ceph-mon[31518]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T00:17:14.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:14 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:17:14.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:14 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:17:14.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:14 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:17:14.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:14 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:17:14.407 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:14 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:17:14.407 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:14 smithi119 ceph-mon[31518]: Updating smithi119:/etc/ceph/ceph.conf 2024-03-28T00:17:14.407 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:14 smithi119 ceph-mon[31518]: Updating smithi132:/etc/ceph/ceph.conf 2024-03-28T00:17:14.407 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:14 smithi119 ceph-mon[31518]: Updating smithi165:/etc/ceph/ceph.conf 2024-03-28T00:17:14.407 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:14 smithi119 ceph-mon[31518]: Updating smithi132:/var/lib/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-28T00:17:14.407 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:14 smithi119 ceph-mon[31518]: Updating smithi165:/var/lib/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-28T00:17:14.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:14 smithi132 ceph-mon[35941]: mon.b calling monitor election 2024-03-28T00:17:14.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:14 smithi132 ceph-mon[35941]: mon.b calling monitor election 2024-03-28T00:17:14.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:14 smithi132 ceph-mon[35941]: mon.c calling monitor election 2024-03-28T00:17:14.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:14 smithi132 ceph-mon[35941]: mon.a calling monitor election 2024-03-28T00:17:14.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:14 smithi132 ceph-mon[35941]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-03-28T00:17:14.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:14 smithi132 ceph-mon[35941]: monmap e3: 3 mons at {a=[v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0],b=[v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0],c=[v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-28T00:17:14.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:14 smithi132 ceph-mon[35941]: fsmap 2024-03-28T00:17:14.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:14 smithi132 ceph-mon[35941]: osdmap e4: 0 total, 0 up, 0 in 2024-03-28T00:17:14.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:14 smithi132 ceph-mon[35941]: mgrmap e13: x(active, since 70s) 2024-03-28T00:17:14.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:14 smithi132 ceph-mon[35941]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-03-28T00:17:14.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:14 smithi132 ceph-mon[35941]: Cluster is now healthy 2024-03-28T00:17:14.454 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:14 smithi132 ceph-mon[35941]: overall HEALTH_OK 2024-03-28T00:17:14.454 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:14 smithi132 ceph-mon[35941]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T00:17:14.454 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:14 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:17:14.454 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:14 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:17:14.454 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:14 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:17:14.454 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:14 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:17:14.454 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:14 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:17:14.454 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:14 smithi132 ceph-mon[35941]: Updating smithi119:/etc/ceph/ceph.conf 2024-03-28T00:17:14.454 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:14 smithi132 ceph-mon[35941]: Updating smithi132:/etc/ceph/ceph.conf 2024-03-28T00:17:14.454 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:14 smithi132 ceph-mon[35941]: Updating smithi165:/etc/ceph/ceph.conf 2024-03-28T00:17:14.454 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:14 smithi132 ceph-mon[35941]: Updating smithi132:/var/lib/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-28T00:17:14.454 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:14 smithi132 ceph-mon[35941]: Updating smithi165:/var/lib/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-28T00:17:14.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:14 smithi165 ceph-mon[36302]: mon.b calling monitor election 2024-03-28T00:17:14.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:14 smithi165 ceph-mon[36302]: mon.b calling monitor election 2024-03-28T00:17:14.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:14 smithi165 ceph-mon[36302]: mon.c calling monitor election 2024-03-28T00:17:14.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:14 smithi165 ceph-mon[36302]: mon.a calling monitor election 2024-03-28T00:17:14.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:14 smithi165 ceph-mon[36302]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-03-28T00:17:14.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:14 smithi165 ceph-mon[36302]: monmap e3: 3 mons at {a=[v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0],b=[v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0],c=[v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-28T00:17:14.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:14 smithi165 ceph-mon[36302]: fsmap 2024-03-28T00:17:14.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:14 smithi165 ceph-mon[36302]: osdmap e4: 0 total, 0 up, 0 in 2024-03-28T00:17:14.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:14 smithi165 ceph-mon[36302]: mgrmap e13: x(active, since 70s) 2024-03-28T00:17:14.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:14 smithi165 ceph-mon[36302]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-03-28T00:17:14.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:14 smithi165 ceph-mon[36302]: Cluster is now healthy 2024-03-28T00:17:14.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:14 smithi165 ceph-mon[36302]: overall HEALTH_OK 2024-03-28T00:17:14.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:14 smithi165 ceph-mon[36302]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T00:17:14.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:14 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:17:14.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:14 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:17:14.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:14 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:17:14.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:14 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:17:14.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:14 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:17:14.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:14 smithi165 ceph-mon[36302]: Updating smithi119:/etc/ceph/ceph.conf 2024-03-28T00:17:14.635 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:14 smithi165 ceph-mon[36302]: Updating smithi132:/etc/ceph/ceph.conf 2024-03-28T00:17:14.635 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:14 smithi165 ceph-mon[36302]: Updating smithi165:/etc/ceph/ceph.conf 2024-03-28T00:17:14.635 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:14 smithi165 ceph-mon[36302]: Updating smithi132:/var/lib/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-28T00:17:14.635 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:14 smithi165 ceph-mon[36302]: Updating smithi165:/var/lib/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-28T00:17:15.367 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:15 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:17:15.368 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:15 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:17:15.368 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:15 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:17:15.368 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:15 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:17:15.368 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:15 smithi119 ceph-mon[31518]: Updating smithi119:/var/lib/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-28T00:17:15.368 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:15 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-28T00:17:15.368 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:15 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:17:15.368 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:15 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:17:15.368 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:15 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:17:15.368 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:15 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:17:15.368 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:15 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:17:15.368 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:15 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:17:15.368 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:15 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:17:15.368 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:15 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-28T00:17:15.537 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:15 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:17:15.537 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:15 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:17:15.537 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:15 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:17:15.537 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:15 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:17:15.537 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:15 smithi165 ceph-mon[36302]: Updating smithi119:/var/lib/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-28T00:17:15.537 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:15 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-28T00:17:15.537 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:15 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:17:15.537 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:15 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:17:15.537 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:15 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:17:15.537 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:15 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:17:15.538 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:15 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:17:15.538 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:15 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:17:15.538 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:15 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:17:15.538 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:15 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-28T00:17:15.665 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:15 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:17:15.666 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:15 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:17:15.666 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:15 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:17:15.666 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:15 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:17:15.666 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:15 smithi132 ceph-mon[35941]: Updating smithi119:/var/lib/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-28T00:17:15.666 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:15 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-28T00:17:15.666 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:15 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:17:15.666 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:15 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:17:15.666 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:15 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:17:15.666 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:15 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:17:15.666 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:15 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:17:15.666 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:15 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:17:15.666 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:15 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:17:15.666 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:15 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-28T00:17:15.846 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:17:15.839+0000 7fbf59c00700 1 -- 172.21.15.119:0/3284553429 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbf54103fb0 msgr2=0x7fbf54104390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:17:15.846 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:17:15.839+0000 7fbf59c00700 1 --2- 172.21.15.119:0/3284553429 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbf54103fb0 0x7fbf54104390 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fbf44009910 tx=0x7fbf44009c20 comp rx=0 tx=0).stop 2024-03-28T00:17:15.846 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:17:15.839+0000 7fbf59c00700 1 -- 172.21.15.119:0/3284553429 shutdown_connections 2024-03-28T00:17:15.846 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:17:15.839+0000 7fbf59c00700 1 --2- 172.21.15.119:0/3284553429 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbf54103fb0 0x7fbf54104390 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:17:15.846 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:17:15.839+0000 7fbf59c00700 1 -- 172.21.15.119:0/3284553429 >> 172.21.15.119:0/3284553429 conn(0x7fbf540ff880 msgr2=0x7fbf54101ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:17:15.846 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:17:15.840+0000 7fbf59c00700 1 -- 172.21.15.119:0/3284553429 shutdown_connections 2024-03-28T00:17:15.847 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:17:15.840+0000 7fbf59c00700 1 -- 172.21.15.119:0/3284553429 wait complete. 2024-03-28T00:17:15.847 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:17:15.841+0000 7fbf59c00700 1 Processor -- start 2024-03-28T00:17:15.847 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:17:15.841+0000 7fbf59c00700 1 -- start start 2024-03-28T00:17:15.847 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:17:15.841+0000 7fbf59c00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbf54103fb0 0x7fbf5419f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:17:15.847 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:17:15.841+0000 7fbf59c00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbf5419fc70 0x7fbf54199990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:17:15.848 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:17:15.841+0000 7fbf59c00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbf54199ed0 0x7fbf5419a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:17:15.848 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:17:15.841+0000 7fbf59c00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf5410e380 con 0x7fbf54199ed0 2024-03-28T00:17:15.848 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:17:15.841+0000 7fbf59c00700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf5410e200 con 0x7fbf5419fc70 2024-03-28T00:17:15.849 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:17:15.841+0000 7fbf59c00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf5410e500 con 0x7fbf54103fb0 2024-03-28T00:17:15.849 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:17:15.842+0000 7fbf53400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbf54199ed0 0x7fbf5419a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:17:15.849 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:17:15.842+0000 7fbf53400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbf54199ed0 0x7fbf5419a380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:54160/0 (socket says 172.21.15.119:54160) 2024-03-28T00:17:15.849 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:17:15.842+0000 7fbf53400700 1 -- 172.21.15.119:0/4073048578 learned_addr learned my addr 172.21.15.119:0/4073048578 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:17:15.849 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:17:15.842+0000 7fbf52a00700 1 --2- 172.21.15.119:0/4073048578 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbf54103fb0 0x7fbf5419f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:17:15.849 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:17:15.842+0000 7fbf4be00700 1 --2- 172.21.15.119:0/4073048578 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbf5419fc70 0x7fbf54199990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:17:15.849 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:17:15.842+0000 7fbf53400700 1 -- 172.21.15.119:0/4073048578 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbf54103fb0 msgr2=0x7fbf5419f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:17:15.849 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:17:15.842+0000 7fbf53400700 1 --2- 172.21.15.119:0/4073048578 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbf54103fb0 0x7fbf5419f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:17:15.849 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:17:15.842+0000 7fbf53400700 1 -- 172.21.15.119:0/4073048578 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbf5419fc70 msgr2=0x7fbf54199990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:17:15.849 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:17:15.842+0000 7fbf53400700 1 --2- 172.21.15.119:0/4073048578 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbf5419fc70 0x7fbf54199990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:17:15.849 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:17:15.842+0000 7fbf53400700 1 -- 172.21.15.119:0/4073048578 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf541a6600 con 0x7fbf54199ed0 2024-03-28T00:17:15.849 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:17:15.842+0000 7fbf52a00700 1 --2- 172.21.15.119:0/4073048578 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbf54103fb0 0x7fbf5419f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T00:17:15.850 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:17:15.843+0000 7fbf53400700 1 --2- 172.21.15.119:0/4073048578 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbf54199ed0 0x7fbf5419a380 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c00ba50 tx=0x7fbf4c00bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:17:15.850 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:17:15.843+0000 7fbf4b400700 1 -- 172.21.15.119:0/4073048578 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf4c019070 con 0x7fbf54199ed0 2024-03-28T00:17:15.850 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:17:15.843+0000 7fbf4b400700 1 -- 172.21.15.119:0/4073048578 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbf4c00ce10 con 0x7fbf54199ed0 2024-03-28T00:17:15.850 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:17:15.843+0000 7fbf4b400700 1 -- 172.21.15.119:0/4073048578 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf4c007350 con 0x7fbf54199ed0 2024-03-28T00:17:15.850 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:17:15.843+0000 7fbf59c00700 1 -- 172.21.15.119:0/4073048578 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf541a6830 con 0x7fbf54199ed0 2024-03-28T00:17:15.850 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:17:15.843+0000 7fbf59c00700 1 -- 172.21.15.119:0/4073048578 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf541a6d20 con 0x7fbf54199ed0 2024-03-28T00:17:15.851 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:17:15.845+0000 7fbf59c00700 1 -- 172.21.15.119:0/4073048578 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf540637c0 con 0x7fbf54199ed0 2024-03-28T00:17:15.851 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:17:15.845+0000 7fbf4b400700 1 -- 172.21.15.119:0/4073048578 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 13) v1 ==== 46163+0+0 (secure 0 0 0) 0x7fbf4c01c070 con 0x7fbf54199ed0 2024-03-28T00:17:15.851 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:17:15.845+0000 7fbf4b400700 1 --2- 172.21.15.119:0/4073048578 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fbf34039450 0x7fbf3403b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:17:15.851 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:17:15.845+0000 7fbf4b400700 1 -- 172.21.15.119:0/4073048578 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fbf4c04b5e0 con 0x7fbf54199ed0 2024-03-28T00:17:15.852 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:17:15.846+0000 7fbf52a00700 1 --2- 172.21.15.119:0/4073048578 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fbf34039450 0x7fbf3403b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:17:15.852 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:17:15.846+0000 7fbf52a00700 1 --2- 172.21.15.119:0/4073048578 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fbf34039450 0x7fbf3403b910 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fbf44005e00 tx=0x7fbf440058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:17:15.857 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:17:15.851+0000 7fbf4b400700 1 -- 172.21.15.119:0/4073048578 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fbf4c014440 con 0x7fbf54199ed0 2024-03-28T00:17:16.103 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:17:16.097+0000 7fbf59c00700 1 -- 172.21.15.119:0/4073048578 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7fbf541a70d0 con 0x7fbf54199ed0 2024-03-28T00:17:16.104 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:17:16.098+0000 7fbf4b400700 1 -- 172.21.15.119:0/4073048578 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) v1 ==== 76+0+277 (secure 0 0 0) 0x7fbf4c0146f0 con 0x7fbf54199ed0 2024-03-28T00:17:16.104 INFO:teuthology.orchestra.run.smithi119.stdout:# minimal ceph.conf for 10f73b92-ec98-11ee-95d0-87774f69a715 2024-03-28T00:17:16.104 INFO:teuthology.orchestra.run.smithi119.stdout:[global] 2024-03-28T00:17:16.104 INFO:teuthology.orchestra.run.smithi119.stdout: fsid = 10f73b92-ec98-11ee-95d0-87774f69a715 2024-03-28T00:17:16.104 INFO:teuthology.orchestra.run.smithi119.stdout: mon_host = [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] 2024-03-28T00:17:16.107 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:17:16.100+0000 7fbf59c00700 1 -- 172.21.15.119:0/4073048578 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fbf34039450 msgr2=0x7fbf3403b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:17:16.107 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:17:16.100+0000 7fbf59c00700 1 --2- 172.21.15.119:0/4073048578 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fbf34039450 0x7fbf3403b910 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fbf44005e00 tx=0x7fbf440058e0 comp rx=0 tx=0).stop 2024-03-28T00:17:16.107 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:17:16.100+0000 7fbf59c00700 1 -- 172.21.15.119:0/4073048578 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbf54199ed0 msgr2=0x7fbf5419a380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:17:16.107 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:17:16.100+0000 7fbf59c00700 1 --2- 172.21.15.119:0/4073048578 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbf54199ed0 0x7fbf5419a380 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c00ba50 tx=0x7fbf4c00bd60 comp rx=0 tx=0).stop 2024-03-28T00:17:16.107 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:17:16.101+0000 7fbf59c00700 1 -- 172.21.15.119:0/4073048578 shutdown_connections 2024-03-28T00:17:16.107 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:17:16.101+0000 7fbf59c00700 1 --2- 172.21.15.119:0/4073048578 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fbf34039450 0x7fbf3403b910 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:17:16.107 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:17:16.101+0000 7fbf59c00700 1 --2- 172.21.15.119:0/4073048578 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbf54199ed0 0x7fbf5419a380 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:17:16.108 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:17:16.101+0000 7fbf59c00700 1 --2- 172.21.15.119:0/4073048578 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbf54103fb0 0x7fbf5419f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:17:16.108 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:17:16.101+0000 7fbf59c00700 1 --2- 172.21.15.119:0/4073048578 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbf5419fc70 0x7fbf54199990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:17:16.108 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:17:16.101+0000 7fbf59c00700 1 -- 172.21.15.119:0/4073048578 >> 172.21.15.119:0/4073048578 conn(0x7fbf540ff880 msgr2=0x7fbf54105980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:17:16.108 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:17:16.101+0000 7fbf59c00700 1 -- 172.21.15.119:0/4073048578 shutdown_connections 2024-03-28T00:17:16.108 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:17:16.101+0000 7fbf59c00700 1 -- 172.21.15.119:0/4073048578 wait complete. 2024-03-28T00:17:16.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:16 smithi119 ceph-mon[31518]: Reconfiguring mon.a (unknown last config time)... 2024-03-28T00:17:16.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:16 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-28T00:17:16.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:16 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:17:16.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:16 smithi119 ceph-mon[31518]: Reconfiguring daemon mon.a on smithi119 2024-03-28T00:17:16.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:16 smithi119 ceph-mon[31518]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T00:17:16.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:16 smithi119 ceph-mon[31518]: from='client.? 172.21.15.119:0/4073048578' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:17:16.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:16 smithi165 ceph-mon[36302]: Reconfiguring mon.a (unknown last config time)... 2024-03-28T00:17:16.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:16 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-28T00:17:16.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:16 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:17:16.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:16 smithi165 ceph-mon[36302]: Reconfiguring daemon mon.a on smithi119 2024-03-28T00:17:16.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:16 smithi165 ceph-mon[36302]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T00:17:16.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:16 smithi165 ceph-mon[36302]: from='client.? 172.21.15.119:0/4073048578' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:17:16.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:16 smithi132 ceph-mon[35941]: Reconfiguring mon.a (unknown last config time)... 2024-03-28T00:17:16.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:16 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-28T00:17:16.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:16 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:17:16.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:16 smithi132 ceph-mon[35941]: Reconfiguring daemon mon.a on smithi119 2024-03-28T00:17:16.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:16 smithi132 ceph-mon[35941]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T00:17:16.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:16 smithi132 ceph-mon[35941]: from='client.? 172.21.15.119:0/4073048578' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:17:16.895 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-03-28T00:17:16.896 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-03-28T00:17:16.896 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/etc/ceph/ceph.conf 2024-03-28T00:17:16.992 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-03-28T00:17:16.992 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-28T00:17:17.102 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-03-28T00:17:17.102 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd of=/etc/ceph/ceph.conf 2024-03-28T00:17:17.137 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-03-28T00:17:17.137 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-28T00:17:17.209 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-03-28T00:17:17.209 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd of=/etc/ceph/ceph.conf 2024-03-28T00:17:17.241 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-03-28T00:17:17.241 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-28T00:17:17.309 INFO:tasks.cephadm:Adding mgr.x on smithi119 2024-03-28T00:17:17.309 INFO:tasks.cephadm:Adding mgr.y on smithi132 2024-03-28T00:17:17.309 INFO:tasks.cephadm:Adding mgr.z on smithi165 2024-03-28T00:17:17.310 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 10f73b92-ec98-11ee-95d0-87774f69a715 -- ceph orch apply mgr '3;smithi119=x;smithi132=y;smithi165=z' 2024-03-28T00:17:17.580 INFO:teuthology.orchestra.run.smithi165.stderr:Inferring config /var/lib/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/mon.c/config 2024-03-28T00:17:19.130 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:19.122+0000 7fd52fa6e700 1 -- 172.21.15.165:0/3081933713 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd528102170 msgr2=0x7fd528102590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:17:19.130 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:19.122+0000 7fd52fa6e700 1 --2- 172.21.15.165:0/3081933713 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd528102170 0x7fd528102590 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fd518009910 tx=0x7fd518009c20 comp rx=0 tx=0).stop 2024-03-28T00:17:19.130 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:19.123+0000 7fd52fa6e700 1 -- 172.21.15.165:0/3081933713 shutdown_connections 2024-03-28T00:17:19.131 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:19.123+0000 7fd52fa6e700 1 --2- 172.21.15.165:0/3081933713 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd528102170 0x7fd528102590 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:17:19.131 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:19.123+0000 7fd52fa6e700 1 -- 172.21.15.165:0/3081933713 >> 172.21.15.165:0/3081933713 conn(0x7fd5280fb900 msgr2=0x7fd5280fdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:17:19.131 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:19.123+0000 7fd52fa6e700 1 -- 172.21.15.165:0/3081933713 shutdown_connections 2024-03-28T00:17:19.131 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:19.123+0000 7fd52fa6e700 1 -- 172.21.15.165:0/3081933713 wait complete. 2024-03-28T00:17:19.131 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:19.123+0000 7fd52fa6e700 1 Processor -- start 2024-03-28T00:17:19.131 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:19.123+0000 7fd52fa6e700 1 -- start start 2024-03-28T00:17:19.131 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:19.123+0000 7fd52fa6e700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd528102170 0x7fd528100960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:17:19.131 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:19.123+0000 7fd52fa6e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd52819a230 0x7fd52819c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:17:19.131 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:19.123+0000 7fd52fa6e700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd52819cc00 0x7fd52819f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:17:19.131 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:19.123+0000 7fd52fa6e700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd528106c20 con 0x7fd528102170 2024-03-28T00:17:19.131 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:19.123+0000 7fd52fa6e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd528106aa0 con 0x7fd52819a230 2024-03-28T00:17:19.131 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:19.123+0000 7fd52fa6e700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd528106da0 con 0x7fd52819cc00 2024-03-28T00:17:19.131 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:19.123+0000 7fd52e00b700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd52819cc00 0x7fd52819f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:17:19.131 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:19.124+0000 7fd52e00b700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd52819cc00 0x7fd52819f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:52706/0 (socket says 172.21.15.165:52706) 2024-03-28T00:17:19.132 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:19.124+0000 7fd52e00b700 1 -- 172.21.15.165:0/4044659871 learned_addr learned my addr 172.21.15.165:0/4044659871 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-03-28T00:17:19.132 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:19.124+0000 7fd52d009700 1 --2- 172.21.15.165:0/4044659871 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd52819a230 0x7fd52819c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:17:19.132 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:19.124+0000 7fd52e00b700 1 -- 172.21.15.165:0/4044659871 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd52819a230 msgr2=0x7fd52819c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:17:19.132 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:19.124+0000 7fd52e00b700 1 --2- 172.21.15.165:0/4044659871 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd52819a230 0x7fd52819c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:17:19.132 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:19.124+0000 7fd52e00b700 1 -- 172.21.15.165:0/4044659871 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd528102170 msgr2=0x7fd528100960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:17:19.132 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:19.124+0000 7fd52e00b700 1 --2- 172.21.15.165:0/4044659871 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd528102170 0x7fd528100960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:17:19.132 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:19.124+0000 7fd52e00b700 1 -- 172.21.15.165:0/4044659871 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd52819f960 con 0x7fd52819cc00 2024-03-28T00:17:19.132 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:19.124+0000 7fd52e00b700 1 --2- 172.21.15.165:0/4044659871 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd52819cc00 0x7fd52819f090 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fd52000ba50 tx=0x7fd52000be10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:17:19.132 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:19.124+0000 7fd51effd700 1 -- 172.21.15.165:0/4044659871 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd520019070 con 0x7fd52819cc00 2024-03-28T00:17:19.132 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:19.124+0000 7fd52fa6e700 1 -- 172.21.15.165:0/4044659871 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd52819fc50 con 0x7fd52819cc00 2024-03-28T00:17:19.132 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:19.124+0000 7fd51effd700 1 -- 172.21.15.165:0/4044659871 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd520014440 con 0x7fd52819cc00 2024-03-28T00:17:19.132 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:19.124+0000 7fd51effd700 1 -- 172.21.15.165:0/4044659871 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd520007590 con 0x7fd52819cc00 2024-03-28T00:17:19.132 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:19.124+0000 7fd52fa6e700 1 -- 172.21.15.165:0/4044659871 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5281a0140 con 0x7fd52819cc00 2024-03-28T00:17:19.133 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:19.125+0000 7fd51effd700 1 -- 172.21.15.165:0/4044659871 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 13) v1 ==== 46163+0+0 (secure 0 0 0) 0x7fd52001c070 con 0x7fd52819cc00 2024-03-28T00:17:19.133 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:19.125+0000 7fd52fa6e700 1 -- 172.21.15.165:0/4044659871 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5280637c0 con 0x7fd52819cc00 2024-03-28T00:17:19.134 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:19.125+0000 7fd51effd700 1 --2- 172.21.15.165:0/4044659871 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fd514039450 0x7fd51403b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:17:19.134 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:19.125+0000 7fd51effd700 1 -- 172.21.15.165:0/4044659871 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fd52004b7a0 con 0x7fd52819cc00 2024-03-28T00:17:19.134 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:19.126+0000 7fd52d80a700 1 --2- 172.21.15.165:0/4044659871 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fd514039450 0x7fd51403b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:17:19.134 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:19.127+0000 7fd52d80a700 1 --2- 172.21.15.165:0/4044659871 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fd514039450 0x7fd51403b910 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fd518005330 tx=0x7fd518005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:17:19.136 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:19.130+0000 7fd51effd700 1 -- 172.21.15.165:0/4044659871 <== mon.1 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd52001b2c0 con 0x7fd52819cc00 2024-03-28T00:17:19.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:18 smithi132 ceph-mon[35941]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T00:17:19.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:18 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:17:19.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:18 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:17:19.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:18 smithi132 ceph-mon[35941]: Reconfiguring mon.b (monmap changed)... 2024-03-28T00:17:19.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:18 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-28T00:17:19.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:18 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-28T00:17:19.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:18 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:17:19.345 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:19.339+0000 7fd52fa6e700 1 -- 172.21.15.165:0/4044659871 --> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi119=x;smithi132=y;smithi165=z", "target": ["mon-mgr", ""]}) v1 -- 0x7fd5280625d0 con 0x7fd514039450 2024-03-28T00:17:19.345 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:18 smithi165 ceph-mon[36302]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T00:17:19.345 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:18 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:17:19.345 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:18 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:17:19.345 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:18 smithi165 ceph-mon[36302]: Reconfiguring mon.b (monmap changed)... 2024-03-28T00:17:19.345 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:18 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-28T00:17:19.345 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:18 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-28T00:17:19.345 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:18 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:17:19.358 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:19.353+0000 7fd51effd700 1 -- 172.21.15.165:0/4044659871 <== mgr.14152 v2:172.21.15.119:6800/1401937096 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fd5280625d0 con 0x7fd514039450 2024-03-28T00:17:19.359 INFO:teuthology.orchestra.run.smithi165.stdout:Scheduled mgr update... 2024-03-28T00:17:19.361 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:19.355+0000 7fd52fa6e700 1 -- 172.21.15.165:0/4044659871 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fd514039450 msgr2=0x7fd51403b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:17:19.361 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:19.355+0000 7fd52fa6e700 1 --2- 172.21.15.165:0/4044659871 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fd514039450 0x7fd51403b910 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fd518005330 tx=0x7fd518005fb0 comp rx=0 tx=0).stop 2024-03-28T00:17:19.361 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:19.355+0000 7fd52fa6e700 1 -- 172.21.15.165:0/4044659871 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd52819cc00 msgr2=0x7fd52819f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:17:19.361 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:19.355+0000 7fd52fa6e700 1 --2- 172.21.15.165:0/4044659871 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd52819cc00 0x7fd52819f090 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fd52000ba50 tx=0x7fd52000be10 comp rx=0 tx=0).stop 2024-03-28T00:17:19.361 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:19.355+0000 7fd52fa6e700 1 -- 172.21.15.165:0/4044659871 shutdown_connections 2024-03-28T00:17:19.362 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:19.355+0000 7fd52fa6e700 1 --2- 172.21.15.165:0/4044659871 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fd514039450 0x7fd51403b910 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:17:19.362 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:19.355+0000 7fd52fa6e700 1 --2- 172.21.15.165:0/4044659871 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd52819cc00 0x7fd52819f090 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:17:19.362 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:19.355+0000 7fd52fa6e700 1 --2- 172.21.15.165:0/4044659871 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd528102170 0x7fd528100960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:17:19.362 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:19.355+0000 7fd52fa6e700 1 --2- 172.21.15.165:0/4044659871 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd52819a230 0x7fd52819c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:17:19.362 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:19.355+0000 7fd52fa6e700 1 -- 172.21.15.165:0/4044659871 >> 172.21.15.165:0/4044659871 conn(0x7fd5280fb900 msgr2=0x7fd5280fc640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:17:19.362 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:19.355+0000 7fd52fa6e700 1 -- 172.21.15.165:0/4044659871 shutdown_connections 2024-03-28T00:17:19.362 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:17:19.356+0000 7fd52fa6e700 1 -- 172.21.15.165:0/4044659871 wait complete. 2024-03-28T00:17:19.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:18 smithi119 ceph-mon[31518]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T00:17:19.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:18 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:17:19.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:18 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:17:19.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:18 smithi119 ceph-mon[31518]: Reconfiguring mon.b (monmap changed)... 2024-03-28T00:17:19.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:18 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-28T00:17:19.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:18 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-28T00:17:19.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:18 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:17:19.970 DEBUG:teuthology.orchestra.run.smithi132:mgr.y> sudo journalctl -f -n 0 -u ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mgr.y.service 2024-03-28T00:17:19.972 DEBUG:teuthology.orchestra.run.smithi165:mgr.z> sudo journalctl -f -n 0 -u ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mgr.z.service 2024-03-28T00:17:19.974 INFO:tasks.cephadm:Deploying OSDs... 2024-03-28T00:17:19.974 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-03-28T00:17:19.975 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/scratch_devs of=/dev/stdout 2024-03-28T00:17:19.996 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:19 smithi165 ceph-mon[36302]: Reconfiguring daemon mon.b on smithi132 2024-03-28T00:17:19.997 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:19 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:17:19.997 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:19 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:17:19.997 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:19 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-28T00:17:19.997 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:19 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-28T00:17:19.997 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:19 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:17:19.997 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:19 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:17:20.032 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-28T00:17:20.032 DEBUG:teuthology.orchestra.run.smithi119:> stat /dev/vg_nvme/lv_1 2024-03-28T00:17:20.085 INFO:teuthology.orchestra.run.smithi119.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-28T00:17:20.085 INFO:teuthology.orchestra.run.smithi119.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-28T00:17:20.085 INFO:teuthology.orchestra.run.smithi119.stdout:Device: 5h/5d Inode: 633 Links: 1 2024-03-28T00:17:20.085 INFO:teuthology.orchestra.run.smithi119.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-28T00:17:20.086 INFO:teuthology.orchestra.run.smithi119.stdout:Context: system_u:object_r:device_t:s0 2024-03-28T00:17:20.086 INFO:teuthology.orchestra.run.smithi119.stdout:Access: 2024-03-28 00:16:16.468020846 +0000 2024-03-28T00:17:20.086 INFO:teuthology.orchestra.run.smithi119.stdout:Modify: 2024-03-28 00:10:46.966771128 +0000 2024-03-28T00:17:20.086 INFO:teuthology.orchestra.run.smithi119.stdout:Change: 2024-03-28 00:10:46.966771128 +0000 2024-03-28T00:17:20.086 INFO:teuthology.orchestra.run.smithi119.stdout: Birth: 2024-03-28 00:10:46.966771128 +0000 2024-03-28T00:17:20.086 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-28T00:17:20.177 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records in 2024-03-28T00:17:20.177 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records out 2024-03-28T00:17:20.177 INFO:teuthology.orchestra.run.smithi119.stderr:512 bytes copied, 0.000382177 s, 1.3 MB/s 2024-03-28T00:17:20.181 DEBUG:teuthology.orchestra.run.smithi119:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-28T00:17:20.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:19 smithi132 ceph-mon[35941]: Reconfiguring daemon mon.b on smithi132 2024-03-28T00:17:20.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:19 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:17:20.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:19 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:17:20.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:19 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-28T00:17:20.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:19 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-28T00:17:20.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:19 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:17:20.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:19 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:17:20.240 DEBUG:teuthology.orchestra.run.smithi119:> stat /dev/vg_nvme/lv_2 2024-03-28T00:17:20.290 INFO:teuthology.orchestra.run.smithi119.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-28T00:17:20.291 INFO:teuthology.orchestra.run.smithi119.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-28T00:17:20.291 INFO:teuthology.orchestra.run.smithi119.stdout:Device: 5h/5d Inode: 650 Links: 1 2024-03-28T00:17:20.291 INFO:teuthology.orchestra.run.smithi119.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-28T00:17:20.291 INFO:teuthology.orchestra.run.smithi119.stdout:Context: system_u:object_r:device_t:s0 2024-03-28T00:17:20.291 INFO:teuthology.orchestra.run.smithi119.stdout:Access: 2024-03-28 00:16:16.557018682 +0000 2024-03-28T00:17:20.291 INFO:teuthology.orchestra.run.smithi119.stdout:Modify: 2024-03-28 00:10:47.543757190 +0000 2024-03-28T00:17:20.291 INFO:teuthology.orchestra.run.smithi119.stdout:Change: 2024-03-28 00:10:47.543757190 +0000 2024-03-28T00:17:20.291 INFO:teuthology.orchestra.run.smithi119.stdout: Birth: 2024-03-28 00:10:47.543757190 +0000 2024-03-28T00:17:20.291 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-28T00:17:20.371 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:19 smithi119 ceph-mon[31518]: Reconfiguring daemon mon.b on smithi132 2024-03-28T00:17:20.372 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:19 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:17:20.372 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:19 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:17:20.372 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:19 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-28T00:17:20.372 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:19 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-28T00:17:20.372 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:19 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:17:20.372 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:19 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:17:20.382 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records in 2024-03-28T00:17:20.382 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records out 2024-03-28T00:17:20.382 INFO:teuthology.orchestra.run.smithi119.stderr:512 bytes copied, 0.000350969 s, 1.5 MB/s 2024-03-28T00:17:20.387 DEBUG:teuthology.orchestra.run.smithi119:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-28T00:17:20.444 DEBUG:teuthology.orchestra.run.smithi119:> stat /dev/vg_nvme/lv_3 2024-03-28T00:17:20.495 INFO:teuthology.orchestra.run.smithi119.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-28T00:17:20.495 INFO:teuthology.orchestra.run.smithi119.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-28T00:17:20.495 INFO:teuthology.orchestra.run.smithi119.stdout:Device: 5h/5d Inode: 672 Links: 1 2024-03-28T00:17:20.495 INFO:teuthology.orchestra.run.smithi119.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-28T00:17:20.495 INFO:teuthology.orchestra.run.smithi119.stdout:Context: system_u:object_r:device_t:s0 2024-03-28T00:17:20.495 INFO:teuthology.orchestra.run.smithi119.stdout:Access: 2024-03-28 00:16:16.614017297 +0000 2024-03-28T00:17:20.495 INFO:teuthology.orchestra.run.smithi119.stdout:Modify: 2024-03-28 00:10:48.088744025 +0000 2024-03-28T00:17:20.495 INFO:teuthology.orchestra.run.smithi119.stdout:Change: 2024-03-28 00:10:48.088744025 +0000 2024-03-28T00:17:20.495 INFO:teuthology.orchestra.run.smithi119.stdout: Birth: 2024-03-28 00:10:48.088744025 +0000 2024-03-28T00:17:20.496 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-28T00:17:20.581 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records in 2024-03-28T00:17:20.581 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records out 2024-03-28T00:17:20.581 INFO:teuthology.orchestra.run.smithi119.stderr:512 bytes copied, 0.00081677 s, 627 kB/s 2024-03-28T00:17:20.585 DEBUG:teuthology.orchestra.run.smithi119:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-28T00:17:20.638 DEBUG:teuthology.orchestra.run.smithi119:> stat /dev/vg_nvme/lv_4 2024-03-28T00:17:20.687 INFO:teuthology.orchestra.run.smithi119.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-28T00:17:20.687 INFO:teuthology.orchestra.run.smithi119.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-28T00:17:20.687 INFO:teuthology.orchestra.run.smithi119.stdout:Device: 5h/5d Inode: 688 Links: 1 2024-03-28T00:17:20.687 INFO:teuthology.orchestra.run.smithi119.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-28T00:17:20.687 INFO:teuthology.orchestra.run.smithi119.stdout:Context: system_u:object_r:device_t:s0 2024-03-28T00:17:20.688 INFO:teuthology.orchestra.run.smithi119.stdout:Access: 2024-03-28 00:16:16.670015936 +0000 2024-03-28T00:17:20.688 INFO:teuthology.orchestra.run.smithi119.stdout:Modify: 2024-03-28 00:10:48.660730207 +0000 2024-03-28T00:17:20.688 INFO:teuthology.orchestra.run.smithi119.stdout:Change: 2024-03-28 00:10:48.660730207 +0000 2024-03-28T00:17:20.688 INFO:teuthology.orchestra.run.smithi119.stdout: Birth: 2024-03-28 00:10:48.660730207 +0000 2024-03-28T00:17:20.688 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-28T00:17:20.777 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records in 2024-03-28T00:17:20.777 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records out 2024-03-28T00:17:20.777 INFO:teuthology.orchestra.run.smithi119.stderr:512 bytes copied, 0.000456737 s, 1.1 MB/s 2024-03-28T00:17:20.781 DEBUG:teuthology.orchestra.run.smithi119:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-28T00:17:20.837 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-03-28T00:17:20.838 DEBUG:teuthology.orchestra.run.smithi132:> dd if=/scratch_devs of=/dev/stdout 2024-03-28T00:17:20.857 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-28T00:17:20.857 DEBUG:teuthology.orchestra.run.smithi132:> stat /dev/vg_nvme/lv_1 2024-03-28T00:17:20.915 INFO:teuthology.orchestra.run.smithi132.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-28T00:17:20.915 INFO:teuthology.orchestra.run.smithi132.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-28T00:17:20.915 INFO:teuthology.orchestra.run.smithi132.stdout:Device: 5h/5d Inode: 663 Links: 1 2024-03-28T00:17:20.915 INFO:teuthology.orchestra.run.smithi132.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-28T00:17:20.915 INFO:teuthology.orchestra.run.smithi132.stdout:Context: system_u:object_r:device_t:s0 2024-03-28T00:17:20.915 INFO:teuthology.orchestra.run.smithi132.stdout:Access: 2024-03-28 00:17:06.012692123 +0000 2024-03-28T00:17:20.915 INFO:teuthology.orchestra.run.smithi132.stdout:Modify: 2024-03-28 00:09:36.625672083 +0000 2024-03-28T00:17:20.915 INFO:teuthology.orchestra.run.smithi132.stdout:Change: 2024-03-28 00:09:36.625672083 +0000 2024-03-28T00:17:20.915 INFO:teuthology.orchestra.run.smithi132.stdout: Birth: 2024-03-28 00:09:36.625672083 +0000 2024-03-28T00:17:20.916 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-28T00:17:20.985 INFO:teuthology.orchestra.run.smithi132.stderr:1+0 records in 2024-03-28T00:17:20.986 INFO:teuthology.orchestra.run.smithi132.stderr:1+0 records out 2024-03-28T00:17:20.986 INFO:teuthology.orchestra.run.smithi132.stderr:512 bytes copied, 0.000210894 s, 2.4 MB/s 2024-03-28T00:17:20.987 DEBUG:teuthology.orchestra.run.smithi132:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-28T00:17:21.045 DEBUG:teuthology.orchestra.run.smithi132:> stat /dev/vg_nvme/lv_2 2024-03-28T00:17:21.102 INFO:teuthology.orchestra.run.smithi132.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-28T00:17:21.102 INFO:teuthology.orchestra.run.smithi132.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-28T00:17:21.102 INFO:teuthology.orchestra.run.smithi132.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-03-28T00:17:21.102 INFO:teuthology.orchestra.run.smithi132.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-28T00:17:21.102 INFO:teuthology.orchestra.run.smithi132.stdout:Context: system_u:object_r:device_t:s0 2024-03-28T00:17:21.102 INFO:teuthology.orchestra.run.smithi132.stdout:Access: 2024-03-28 00:17:06.012692123 +0000 2024-03-28T00:17:21.102 INFO:teuthology.orchestra.run.smithi132.stdout:Modify: 2024-03-28 00:09:36.977662826 +0000 2024-03-28T00:17:21.102 INFO:teuthology.orchestra.run.smithi132.stdout:Change: 2024-03-28 00:09:36.977662826 +0000 2024-03-28T00:17:21.102 INFO:teuthology.orchestra.run.smithi132.stdout: Birth: 2024-03-28 00:09:36.977662826 +0000 2024-03-28T00:17:21.103 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-28T00:17:21.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:20 smithi165 ceph-mon[36302]: Reconfiguring mon.c (monmap changed)... 2024-03-28T00:17:21.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:20 smithi165 ceph-mon[36302]: Reconfiguring daemon mon.c on smithi165 2024-03-28T00:17:21.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:20 smithi165 ceph-mon[36302]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi119=x;smithi132=y;smithi165=z", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T00:17:21.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:20 smithi165 ceph-mon[36302]: Saving service mgr spec with placement smithi119=x;smithi132=y;smithi165=z;count:3 2024-03-28T00:17:21.135 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:20 smithi165 ceph-mon[36302]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T00:17:21.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:20 smithi119 ceph-mon[31518]: Reconfiguring mon.c (monmap changed)... 2024-03-28T00:17:21.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:20 smithi119 ceph-mon[31518]: Reconfiguring daemon mon.c on smithi165 2024-03-28T00:17:21.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:20 smithi119 ceph-mon[31518]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi119=x;smithi132=y;smithi165=z", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T00:17:21.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:20 smithi119 ceph-mon[31518]: Saving service mgr spec with placement smithi119=x;smithi132=y;smithi165=z;count:3 2024-03-28T00:17:21.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:20 smithi119 ceph-mon[31518]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T00:17:21.170 INFO:teuthology.orchestra.run.smithi132.stderr:1+0 records in 2024-03-28T00:17:21.170 INFO:teuthology.orchestra.run.smithi132.stderr:1+0 records out 2024-03-28T00:17:21.170 INFO:teuthology.orchestra.run.smithi132.stderr:512 bytes copied, 0.000222959 s, 2.3 MB/s 2024-03-28T00:17:21.171 DEBUG:teuthology.orchestra.run.smithi132:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-28T00:17:21.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:20 smithi132 ceph-mon[35941]: Reconfiguring mon.c (monmap changed)... 2024-03-28T00:17:21.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:20 smithi132 ceph-mon[35941]: Reconfiguring daemon mon.c on smithi165 2024-03-28T00:17:21.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:20 smithi132 ceph-mon[35941]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi119=x;smithi132=y;smithi165=z", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T00:17:21.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:20 smithi132 ceph-mon[35941]: Saving service mgr spec with placement smithi119=x;smithi132=y;smithi165=z;count:3 2024-03-28T00:17:21.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:20 smithi132 ceph-mon[35941]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T00:17:21.220 DEBUG:teuthology.orchestra.run.smithi132:> stat /dev/vg_nvme/lv_3 2024-03-28T00:17:21.276 INFO:teuthology.orchestra.run.smithi132.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-28T00:17:21.276 INFO:teuthology.orchestra.run.smithi132.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-28T00:17:21.276 INFO:teuthology.orchestra.run.smithi132.stdout:Device: 5h/5d Inode: 701 Links: 1 2024-03-28T00:17:21.276 INFO:teuthology.orchestra.run.smithi132.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-28T00:17:21.277 INFO:teuthology.orchestra.run.smithi132.stdout:Context: system_u:object_r:device_t:s0 2024-03-28T00:17:21.277 INFO:teuthology.orchestra.run.smithi132.stdout:Access: 2024-03-28 00:17:06.012692123 +0000 2024-03-28T00:17:21.277 INFO:teuthology.orchestra.run.smithi132.stdout:Modify: 2024-03-28 00:09:37.306654173 +0000 2024-03-28T00:17:21.277 INFO:teuthology.orchestra.run.smithi132.stdout:Change: 2024-03-28 00:09:37.306654173 +0000 2024-03-28T00:17:21.277 INFO:teuthology.orchestra.run.smithi132.stdout: Birth: 2024-03-28 00:09:37.306654173 +0000 2024-03-28T00:17:21.277 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-28T00:17:21.343 INFO:teuthology.orchestra.run.smithi132.stderr:1+0 records in 2024-03-28T00:17:21.343 INFO:teuthology.orchestra.run.smithi132.stderr:1+0 records out 2024-03-28T00:17:21.344 INFO:teuthology.orchestra.run.smithi132.stderr:512 bytes copied, 0.000237775 s, 2.2 MB/s 2024-03-28T00:17:21.345 DEBUG:teuthology.orchestra.run.smithi132:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-28T00:17:21.400 DEBUG:teuthology.orchestra.run.smithi132:> stat /dev/vg_nvme/lv_4 2024-03-28T00:17:21.455 INFO:teuthology.orchestra.run.smithi132.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-28T00:17:21.455 INFO:teuthology.orchestra.run.smithi132.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-28T00:17:21.456 INFO:teuthology.orchestra.run.smithi132.stdout:Device: 5h/5d Inode: 719 Links: 1 2024-03-28T00:17:21.456 INFO:teuthology.orchestra.run.smithi132.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-28T00:17:21.456 INFO:teuthology.orchestra.run.smithi132.stdout:Context: system_u:object_r:device_t:s0 2024-03-28T00:17:21.456 INFO:teuthology.orchestra.run.smithi132.stdout:Access: 2024-03-28 00:17:06.013692098 +0000 2024-03-28T00:17:21.456 INFO:teuthology.orchestra.run.smithi132.stdout:Modify: 2024-03-28 00:09:37.645645258 +0000 2024-03-28T00:17:21.456 INFO:teuthology.orchestra.run.smithi132.stdout:Change: 2024-03-28 00:09:37.645645258 +0000 2024-03-28T00:17:21.456 INFO:teuthology.orchestra.run.smithi132.stdout: Birth: 2024-03-28 00:09:37.645645258 +0000 2024-03-28T00:17:21.456 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-28T00:17:21.522 INFO:teuthology.orchestra.run.smithi132.stderr:1+0 records in 2024-03-28T00:17:21.522 INFO:teuthology.orchestra.run.smithi132.stderr:1+0 records out 2024-03-28T00:17:21.522 INFO:teuthology.orchestra.run.smithi132.stderr:512 bytes copied, 0.00021642 s, 2.4 MB/s 2024-03-28T00:17:21.523 DEBUG:teuthology.orchestra.run.smithi132:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-28T00:17:21.579 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-03-28T00:17:21.579 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/scratch_devs of=/dev/stdout 2024-03-28T00:17:21.595 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-28T00:17:21.595 DEBUG:teuthology.orchestra.run.smithi165:> stat /dev/vg_nvme/lv_1 2024-03-28T00:17:21.651 INFO:teuthology.orchestra.run.smithi165.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-28T00:17:21.651 INFO:teuthology.orchestra.run.smithi165.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-28T00:17:21.651 INFO:teuthology.orchestra.run.smithi165.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-03-28T00:17:21.651 INFO:teuthology.orchestra.run.smithi165.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-28T00:17:21.651 INFO:teuthology.orchestra.run.smithi165.stdout:Context: system_u:object_r:device_t:s0 2024-03-28T00:17:21.651 INFO:teuthology.orchestra.run.smithi165.stdout:Access: 2024-03-28 00:16:56.461900352 +0000 2024-03-28T00:17:21.651 INFO:teuthology.orchestra.run.smithi165.stdout:Modify: 2024-03-28 00:09:34.081552680 +0000 2024-03-28T00:17:21.651 INFO:teuthology.orchestra.run.smithi165.stdout:Change: 2024-03-28 00:09:34.081552680 +0000 2024-03-28T00:17:21.651 INFO:teuthology.orchestra.run.smithi165.stdout: Birth: 2024-03-28 00:09:34.081552680 +0000 2024-03-28T00:17:21.652 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-28T00:17:21.718 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-03-28T00:17:21.718 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-03-28T00:17:21.718 INFO:teuthology.orchestra.run.smithi165.stderr:512 bytes copied, 0.000210546 s, 2.4 MB/s 2024-03-28T00:17:21.719 DEBUG:teuthology.orchestra.run.smithi165:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-28T00:17:21.775 DEBUG:teuthology.orchestra.run.smithi165:> stat /dev/vg_nvme/lv_2 2024-03-28T00:17:21.830 INFO:teuthology.orchestra.run.smithi165.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-28T00:17:21.830 INFO:teuthology.orchestra.run.smithi165.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-28T00:17:21.830 INFO:teuthology.orchestra.run.smithi165.stdout:Device: 5h/5d Inode: 681 Links: 1 2024-03-28T00:17:21.830 INFO:teuthology.orchestra.run.smithi165.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-28T00:17:21.831 INFO:teuthology.orchestra.run.smithi165.stdout:Context: system_u:object_r:device_t:s0 2024-03-28T00:17:21.831 INFO:teuthology.orchestra.run.smithi165.stdout:Access: 2024-03-28 00:16:56.461900352 +0000 2024-03-28T00:17:21.831 INFO:teuthology.orchestra.run.smithi165.stdout:Modify: 2024-03-28 00:09:34.435547196 +0000 2024-03-28T00:17:21.831 INFO:teuthology.orchestra.run.smithi165.stdout:Change: 2024-03-28 00:09:34.435547196 +0000 2024-03-28T00:17:21.831 INFO:teuthology.orchestra.run.smithi165.stdout: Birth: 2024-03-28 00:09:34.435547196 +0000 2024-03-28T00:17:21.831 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-28T00:17:21.896 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-03-28T00:17:21.896 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-03-28T00:17:21.896 INFO:teuthology.orchestra.run.smithi165.stderr:512 bytes copied, 0.00019232 s, 2.7 MB/s 2024-03-28T00:17:21.897 DEBUG:teuthology.orchestra.run.smithi165:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-28T00:17:21.953 DEBUG:teuthology.orchestra.run.smithi165:> stat /dev/vg_nvme/lv_3 2024-03-28T00:17:22.008 INFO:teuthology.orchestra.run.smithi165.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-28T00:17:22.009 INFO:teuthology.orchestra.run.smithi165.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-28T00:17:22.009 INFO:teuthology.orchestra.run.smithi165.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-03-28T00:17:22.009 INFO:teuthology.orchestra.run.smithi165.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-28T00:17:22.009 INFO:teuthology.orchestra.run.smithi165.stdout:Context: system_u:object_r:device_t:s0 2024-03-28T00:17:22.009 INFO:teuthology.orchestra.run.smithi165.stdout:Access: 2024-03-28 00:16:56.462900336 +0000 2024-03-28T00:17:22.009 INFO:teuthology.orchestra.run.smithi165.stdout:Modify: 2024-03-28 00:09:34.787541743 +0000 2024-03-28T00:17:22.009 INFO:teuthology.orchestra.run.smithi165.stdout:Change: 2024-03-28 00:09:34.787541743 +0000 2024-03-28T00:17:22.009 INFO:teuthology.orchestra.run.smithi165.stdout: Birth: 2024-03-28 00:09:34.787541743 +0000 2024-03-28T00:17:22.009 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-28T00:17:22.074 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-03-28T00:17:22.075 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-03-28T00:17:22.075 INFO:teuthology.orchestra.run.smithi165.stderr:512 bytes copied, 0.000216652 s, 2.4 MB/s 2024-03-28T00:17:22.076 DEBUG:teuthology.orchestra.run.smithi165:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-28T00:17:22.132 DEBUG:teuthology.orchestra.run.smithi165:> stat /dev/vg_nvme/lv_4 2024-03-28T00:17:22.187 INFO:teuthology.orchestra.run.smithi165.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-28T00:17:22.187 INFO:teuthology.orchestra.run.smithi165.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-28T00:17:22.187 INFO:teuthology.orchestra.run.smithi165.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-03-28T00:17:22.187 INFO:teuthology.orchestra.run.smithi165.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-28T00:17:22.187 INFO:teuthology.orchestra.run.smithi165.stdout:Context: system_u:object_r:device_t:s0 2024-03-28T00:17:22.187 INFO:teuthology.orchestra.run.smithi165.stdout:Access: 2024-03-28 00:16:56.462900336 +0000 2024-03-28T00:17:22.187 INFO:teuthology.orchestra.run.smithi165.stdout:Modify: 2024-03-28 00:09:35.122536553 +0000 2024-03-28T00:17:22.187 INFO:teuthology.orchestra.run.smithi165.stdout:Change: 2024-03-28 00:09:35.122536553 +0000 2024-03-28T00:17:22.187 INFO:teuthology.orchestra.run.smithi165.stdout: Birth: 2024-03-28 00:09:35.122536553 +0000 2024-03-28T00:17:22.188 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-28T00:17:22.252 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-03-28T00:17:22.252 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-03-28T00:17:22.252 INFO:teuthology.orchestra.run.smithi165.stderr:512 bytes copied, 0.000190535 s, 2.7 MB/s 2024-03-28T00:17:22.253 DEBUG:teuthology.orchestra.run.smithi165:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-28T00:17:22.309 INFO:tasks.cephadm:Deploying osd.0 on smithi119 with /dev/vg_nvme/lv_4... 2024-03-28T00:17:22.309 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 10f73b92-ec98-11ee-95d0-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-28T00:17:22.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:22 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:17:22.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:22 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:17:22.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:22 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:17:22.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:22 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:17:22.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:22 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:17:22.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:22 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:17:22.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:22 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-28T00:17:22.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:22 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-28T00:17:22.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:22 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-28T00:17:22.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:22 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:17:22.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:22 smithi165 ceph-mon[36302]: Deploying daemon mgr.z on smithi165 2024-03-28T00:17:22.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:22 smithi165 ceph-mon[36302]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T00:17:22.395 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:22 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:17:22.395 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:22 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:17:22.395 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:22 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:17:22.395 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:22 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:17:22.395 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:22 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:17:22.395 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:22 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:17:22.395 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:22 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-28T00:17:22.395 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:22 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-28T00:17:22.395 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:22 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-28T00:17:22.395 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:22 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:17:22.395 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:22 smithi119 ceph-mon[31518]: Deploying daemon mgr.z on smithi165 2024-03-28T00:17:22.395 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:22 smithi119 ceph-mon[31518]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T00:17:22.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:22 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:17:22.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:22 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:17:22.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:22 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:17:22.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:22 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:17:22.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:22 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:17:22.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:22 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:17:22.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:22 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-28T00:17:22.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:22 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-28T00:17:22.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:22 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-28T00:17:22.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:22 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:17:22.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:22 smithi132 ceph-mon[35941]: Deploying daemon mgr.z on smithi165 2024-03-28T00:17:22.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:22 smithi132 ceph-mon[35941]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T00:17:22.745 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/mon.a/config 2024-03-28T00:17:23.324 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Mar 28 00:17:23 smithi165 systemd[1]: Starting Ceph mgr.z for 10f73b92-ec98-11ee-95d0-87774f69a715... 2024-03-28T00:17:23.586 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Mar 28 00:17:23 smithi165 podman[38191]: 2024-03-28 00:17:23.222090747 +0000 UTC m=+0.017720872 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-28T00:17:23.586 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Mar 28 00:17:23 smithi165 podman[38191]: 2024-03-28 00:17:23.319939396 +0000 UTC m=+0.115569512 container create 3ec565f6ab2b0c98e8ad23d6a00d8cf4111920693418a4ae1410c76760835df4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-z, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319) 2024-03-28T00:17:23.586 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Mar 28 00:17:23 smithi165 podman[38191]: 2024-03-28 00:17:23.513780039 +0000 UTC m=+0.309410161 container init 3ec565f6ab2b0c98e8ad23d6a00d8cf4111920693418a4ae1410c76760835df4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-z, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, ceph=True) 2024-03-28T00:17:23.586 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Mar 28 00:17:23 smithi165 podman[38191]: 2024-03-28 00:17:23.516341505 +0000 UTC m=+0.311971634 container start 3ec565f6ab2b0c98e8ad23d6a00d8cf4111920693418a4ae1410c76760835df4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-z, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-03-28T00:17:23.586 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Mar 28 00:17:23 smithi165 ceph-mgr[38210]: -- 172.21.15.165:0/1982107846 <== mon.1 v2:172.21.15.165:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5642064685a0 con 0x56420643c800 2024-03-28T00:17:23.586 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Mar 28 00:17:23 smithi165 bash[38191]: 3ec565f6ab2b0c98e8ad23d6a00d8cf4111920693418a4ae1410c76760835df4 2024-03-28T00:17:23.882 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Mar 28 00:17:23 smithi165 systemd[1]: Started Ceph mgr.z for 10f73b92-ec98-11ee-95d0-87774f69a715. 2024-03-28T00:17:24.382 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Mar 28 00:17:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-z[38206]: 2024-03-28T00:17:24.067+0000 7f3c8b9ef200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-28T00:17:24.383 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Mar 28 00:17:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-z[38206]: 2024-03-28T00:17:24.159+0000 7f3c8b9ef200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-28T00:17:24.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:24 smithi119 ceph-mon[31518]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T00:17:24.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:24 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:17:24.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:24 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:17:24.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:24 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:17:24.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:24 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-28T00:17:24.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:24 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-28T00:17:24.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:24 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-28T00:17:24.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:24 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:17:24.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:24 smithi119 ceph-mon[31518]: Deploying daemon mgr.y on smithi132 2024-03-28T00:17:24.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:24 smithi132 ceph-mon[35941]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T00:17:24.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:24 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:17:24.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:24 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:17:24.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:24 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:17:24.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:24 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-28T00:17:24.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:24 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-28T00:17:24.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:24 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-28T00:17:24.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:24 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:17:24.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:24 smithi132 ceph-mon[35941]: Deploying daemon mgr.y on smithi132 2024-03-28T00:17:25.038 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Mar 28 00:17:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-z[38206]: 2024-03-28T00:17:24.799+0000 7f3c8b9ef200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-28T00:17:25.038 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Mar 28 00:17:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-z[38206]: 2024-03-28T00:17:24.891+0000 7f3c8b9ef200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-28T00:17:25.038 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Mar 28 00:17:25 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-z[38206]: 2024-03-28T00:17:25.032+0000 7f3c8b9ef200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-28T00:17:25.038 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:24 smithi165 ceph-mon[36302]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T00:17:25.038 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:24 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:17:25.039 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:24 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:17:25.039 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:24 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:17:25.039 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:24 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-28T00:17:25.039 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:24 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-28T00:17:25.039 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:24 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-28T00:17:25.039 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:24 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:17:25.039 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:24 smithi165 ceph-mon[36302]: Deploying daemon mgr.y on smithi132 2024-03-28T00:17:25.382 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Mar 28 00:17:25 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-z[38206]: 2024-03-28T00:17:25.126+0000 7f3c8b9ef200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-28T00:17:25.382 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Mar 28 00:17:25 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-z[38206]: 2024-03-28T00:17:25.207+0000 7f3c8b9ef200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-28T00:17:25.436 INFO:teuthology.orchestra.run.smithi119.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-28T00:17:25.437 INFO:teuthology.orchestra.run.smithi119.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-28T00:17:25.437 INFO:teuthology.orchestra.run.smithi119.stderr: stderr: 10+0 records in 2024-03-28T00:17:25.437 INFO:teuthology.orchestra.run.smithi119.stderr:10+0 records out 2024-03-28T00:17:25.437 INFO:teuthology.orchestra.run.smithi119.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0389432 s, 269 MB/s 2024-03-28T00:17:25.437 INFO:teuthology.orchestra.run.smithi119.stderr:--> Zapping successful for: 2024-03-28T00:17:25.763 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Mar 28 00:17:25 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-z[38206]: 2024-03-28T00:17:25.468+0000 7f3c8b9ef200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-28T00:17:25.764 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Mar 28 00:17:25 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-z[38206]: 2024-03-28T00:17:25.577+0000 7f3c8b9ef200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-28T00:17:25.764 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Mar 28 00:17:25 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-z[38206]: 2024-03-28T00:17:25.662+0000 7f3c8b9ef200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-28T00:17:25.764 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Mar 28 00:17:25 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-z[38206]: 2024-03-28T00:17:25.757+0000 7f3c8b9ef200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-28T00:17:26.132 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Mar 28 00:17:25 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-z[38206]: 2024-03-28T00:17:25.934+0000 7f3c8b9ef200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-28T00:17:26.154 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 10f73b92-ec98-11ee-95d0-87774f69a715 -- ceph orch daemon add osd smithi119:vg_nvme/lv_4 2024-03-28T00:17:26.587 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/mon.a/config 2024-03-28T00:17:26.667 INFO:journalctl@ceph.mgr.y.smithi132.stdout:Mar 28 00:17:26 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-y[37327]: 2024-03-28T00:17:26.620+0000 7f08d443c200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-28T00:17:26.934 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:26 smithi165 ceph-mon[36302]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T00:17:26.934 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:26 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:17:26.934 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:26 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:17:26.934 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:26 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:17:26.934 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:26 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:17:26.934 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:26 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:17:26.944 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:26 smithi119 ceph-mon[31518]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T00:17:26.944 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:26 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:17:26.944 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:26 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:17:26.944 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:26 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:17:26.944 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:26 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:17:26.944 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:26 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:17:26.952 INFO:journalctl@ceph.mgr.y.smithi132.stdout:Mar 28 00:17:26 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-y[37327]: 2024-03-28T00:17:26.719+0000 7f08d443c200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-28T00:17:26.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:26 smithi132 ceph-mon[35941]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T00:17:26.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:26 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:17:26.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:26 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:17:26.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:26 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:17:26.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:26 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:17:26.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:26 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:17:27.292 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Mar 28 00:17:26 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-z[38206]: 2024-03-28T00:17:26.928+0000 7f3c8b9ef200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-28T00:17:27.292 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Mar 28 00:17:27 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-z[38206]: 2024-03-28T00:17:27.075+0000 7f3c8b9ef200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-28T00:17:27.546 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Mar 28 00:17:27 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-z[38206]: 2024-03-28T00:17:27.423+0000 7f3c8b9ef200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-28T00:17:27.546 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Mar 28 00:17:27 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-z[38206]: 2024-03-28T00:17:27.512+0000 7f3c8b9ef200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-28T00:17:27.663 INFO:journalctl@ceph.mgr.y.smithi132.stdout:Mar 28 00:17:27 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-y[37327]: 2024-03-28T00:17:27.414+0000 7f08d443c200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-28T00:17:27.663 INFO:journalctl@ceph.mgr.y.smithi132.stdout:Mar 28 00:17:27 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-y[37327]: 2024-03-28T00:17:27.512+0000 7f08d443c200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-28T00:17:27.882 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Mar 28 00:17:27 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-z[38206]: 2024-03-28T00:17:27.708+0000 7f3c8b9ef200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-28T00:17:27.952 INFO:journalctl@ceph.mgr.y.smithi132.stdout:Mar 28 00:17:27 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-y[37327]: 2024-03-28T00:17:27.657+0000 7f08d443c200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-28T00:17:27.952 INFO:journalctl@ceph.mgr.y.smithi132.stdout:Mar 28 00:17:27 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-y[37327]: 2024-03-28T00:17:27.756+0000 7f08d443c200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-28T00:17:27.953 INFO:journalctl@ceph.mgr.y.smithi132.stdout:Mar 28 00:17:27 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-y[37327]: 2024-03-28T00:17:27.847+0000 7f08d443c200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-28T00:17:28.190 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Mar 28 00:17:27 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-z[38206]: 2024-03-28T00:17:27.900+0000 7f3c8b9ef200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-28T00:17:28.408 INFO:journalctl@ceph.mgr.y.smithi132.stdout:Mar 28 00:17:28 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-y[37327]: 2024-03-28T00:17:28.115+0000 7f08d443c200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-28T00:17:28.409 INFO:journalctl@ceph.mgr.y.smithi132.stdout:Mar 28 00:17:28 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-y[37327]: 2024-03-28T00:17:28.224+0000 7f08d443c200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-28T00:17:28.409 INFO:journalctl@ceph.mgr.y.smithi132.stdout:Mar 28 00:17:28 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-y[37327]: 2024-03-28T00:17:28.309+0000 7f08d443c200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-28T00:17:28.409 INFO:journalctl@ceph.mgr.y.smithi132.stdout:Mar 28 00:17:28 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-y[37327]: 2024-03-28T00:17:28.403+0000 7f08d443c200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-28T00:17:28.450 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Mar 28 00:17:28 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-z[38206]: 2024-03-28T00:17:28.184+0000 7f3c8b9ef200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-28T00:17:28.450 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Mar 28 00:17:28 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-z[38206]: 2024-03-28T00:17:28.444+0000 7f3c8b9ef200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-28T00:17:28.702 INFO:journalctl@ceph.mgr.y.smithi132.stdout:Mar 28 00:17:28 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-y[37327]: 2024-03-28T00:17:28.578+0000 7f08d443c200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-28T00:17:28.742 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:17:28.735+0000 7fac25600700 1 -- 172.21.15.119:0/2871181137 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fac20101400 msgr2=0x7fac20101880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:17:28.742 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:17:28.735+0000 7fac25600700 1 --2- 172.21.15.119:0/2871181137 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fac20101400 0x7fac20101880 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fac0c009990 tx=0x7fac0c009ca0 comp rx=0 tx=0).stop 2024-03-28T00:17:28.742 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:17:28.736+0000 7fac25600700 1 -- 172.21.15.119:0/2871181137 shutdown_connections 2024-03-28T00:17:28.743 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:17:28.736+0000 7fac25600700 1 --2- 172.21.15.119:0/2871181137 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fac20101dc0 0x7fac20102240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:17:28.743 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:17:28.736+0000 7fac25600700 1 --2- 172.21.15.119:0/2871181137 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fac20100aa0 0x7fac20100ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:17:28.743 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:17:28.736+0000 7fac25600700 1 --2- 172.21.15.119:0/2871181137 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fac20101400 0x7fac20101880 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:17:28.743 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:17:28.736+0000 7fac25600700 1 -- 172.21.15.119:0/2871181137 >> 172.21.15.119:0/2871181137 conn(0x7fac200fc660 msgr2=0x7fac200feaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:17:28.743 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:17:28.736+0000 7fac25600700 1 -- 172.21.15.119:0/2871181137 shutdown_connections 2024-03-28T00:17:28.743 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:17:28.736+0000 7fac25600700 1 -- 172.21.15.119:0/2871181137 wait complete. 2024-03-28T00:17:28.745 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:17:28.737+0000 7fac25600700 1 Processor -- start 2024-03-28T00:17:28.745 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:17:28.737+0000 7fac25600700 1 -- start start 2024-03-28T00:17:28.745 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:17:28.737+0000 7fac25600700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fac20100aa0 0x7fac2019a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:17:28.746 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:17:28.737+0000 7fac25600700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fac20101400 0x7fac2019ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:17:28.746 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:17:28.737+0000 7fac25600700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fac20101dc0 0x7fac201a1c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:17:28.746 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:17:28.737+0000 7fac25600700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac2010ffc0 con 0x7fac20101dc0 2024-03-28T00:17:28.746 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:17:28.737+0000 7fac25600700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac2010fe40 con 0x7fac20101400 2024-03-28T00:17:28.747 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:17:28.737+0000 7fac25600700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac20110140 con 0x7fac20100aa0 2024-03-28T00:17:28.747 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:17:28.737+0000 7fac1e000700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fac20101400 0x7fac2019ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:17:28.747 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:17:28.738+0000 7fac1e000700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fac20101400 0x7fac2019ac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.119:42552/0 (socket says 172.21.15.119:42552) 2024-03-28T00:17:28.748 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:17:28.738+0000 7fac1ea00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fac20100aa0 0x7fac2019a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:17:28.748 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:17:28.738+0000 7fac1e000700 1 -- 172.21.15.119:0/2451886429 learned_addr learned my addr 172.21.15.119:0/2451886429 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:17:28.748 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:17:28.738+0000 7fac1f400700 1 --2- 172.21.15.119:0/2451886429 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fac20101dc0 0x7fac201a1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:17:28.748 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:17:28.738+0000 7fac1e000700 1 -- 172.21.15.119:0/2451886429 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fac20100aa0 msgr2=0x7fac2019a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:17:28.749 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:17:28.738+0000 7fac1e000700 1 --2- 172.21.15.119:0/2451886429 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fac20100aa0 0x7fac2019a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:17:28.749 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:17:28.738+0000 7fac1e000700 1 -- 172.21.15.119:0/2451886429 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fac20101dc0 msgr2=0x7fac201a1c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:17:28.749 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:17:28.738+0000 7fac1e000700 1 --2- 172.21.15.119:0/2451886429 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fac20101dc0 0x7fac201a1c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:17:28.749 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:17:28.738+0000 7fac1e000700 1 -- 172.21.15.119:0/2451886429 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac201a2330 con 0x7fac20101400 2024-03-28T00:17:28.749 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:17:28.738+0000 7fac1e000700 1 --2- 172.21.15.119:0/2451886429 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fac20101400 0x7fac2019ac10 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fac0c00b2e0 tx=0x7fac0c00bbb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:17:28.749 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:17:28.739+0000 7fac17400700 1 -- 172.21.15.119:0/2451886429 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac0c026070 con 0x7fac20101400 2024-03-28T00:17:28.749 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:17:28.739+0000 7fac17400700 1 -- 172.21.15.119:0/2451886429 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fac0c00f460 con 0x7fac20101400 2024-03-28T00:17:28.749 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:17:28.739+0000 7fac17400700 1 -- 172.21.15.119:0/2451886429 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac0c021650 con 0x7fac20101400 2024-03-28T00:17:28.749 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:17:28.739+0000 7fac25600700 1 -- 172.21.15.119:0/2451886429 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac201a25c0 con 0x7fac20101400 2024-03-28T00:17:28.749 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:17:28.739+0000 7fac25600700 1 -- 172.21.15.119:0/2451886429 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac201a2a70 con 0x7fac20101400 2024-03-28T00:17:28.750 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:17:28.740+0000 7fac17400700 1 -- 172.21.15.119:0/2451886429 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 13) v1 ==== 46163+0+0 (secure 0 0 0) 0x7fac0c029070 con 0x7fac20101400 2024-03-28T00:17:28.750 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:17:28.740+0000 7fac17400700 1 --2- 172.21.15.119:0/2451886429 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fac04039200 0x7fac0403b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:17:28.750 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:17:28.740+0000 7fac17400700 1 -- 172.21.15.119:0/2451886429 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fac0c04cf40 con 0x7fac20101400 2024-03-28T00:17:28.750 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:17:28.740+0000 7fac25600700 1 -- 172.21.15.119:0/2451886429 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac20068e50 con 0x7fac20101400 2024-03-28T00:17:28.750 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:17:28.741+0000 7fac1ea00700 1 --2- 172.21.15.119:0/2451886429 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fac04039200 0x7fac0403b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:17:28.750 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:17:28.741+0000 7fac1ea00700 1 --2- 172.21.15.119:0/2451886429 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fac04039200 0x7fac0403b6c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fac10009c60 tx=0x7fac10009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:17:28.752 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:17:28.746+0000 7fac17400700 1 -- 172.21.15.119:0/2451886429 <== mon.2 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fac0c030080 con 0x7fac20101400 2024-03-28T00:17:28.835 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Mar 28 00:17:28 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-z[38206]: 2024-03-28T00:17:28.679+0000 7f3c8b9ef200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-28T00:17:28.999 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:17:28.993+0000 7fac25600700 1 -- 172.21.15.119:0/2451886429 --> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi119:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7fac20062690 con 0x7fac04039200 2024-03-28T00:17:29.096 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:28 smithi119 ceph-mon[31518]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T00:17:29.096 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:28 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:17:29.097 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:28 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:17:29.097 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:28 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:17:29.097 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:28 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:17:29.097 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:28 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:17:29.097 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:28 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:17:29.097 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:28 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:17:29.097 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:28 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-28T00:17:29.097 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:28 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-28T00:17:29.097 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:28 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:17:29.097 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:28 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:17:29.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:28 smithi165 ceph-mon[36302]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T00:17:29.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:28 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:17:29.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:28 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:17:29.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:28 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:17:29.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:28 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:17:29.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:28 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:17:29.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:28 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:17:29.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:28 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:17:29.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:28 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-28T00:17:29.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:28 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-28T00:17:29.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:28 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:17:29.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:28 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:17:29.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:28 smithi132 ceph-mon[35941]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T00:17:29.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:28 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:17:29.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:28 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:17:29.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:28 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:17:29.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:28 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:17:29.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:28 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:17:29.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:28 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:17:29.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:28 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:17:29.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:28 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-28T00:17:29.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:28 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-28T00:17:29.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:28 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:17:29.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:28 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:17:29.844 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Mar 28 00:17:29 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-z[38206]: 2024-03-28T00:17:29.475+0000 7f3c8b9ef200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-28T00:17:29.844 INFO:journalctl@ceph.mgr.y.smithi132.stdout:Mar 28 00:17:29 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-y[37327]: 2024-03-28T00:17:29.544+0000 7f08d443c200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-28T00:17:29.844 INFO:journalctl@ceph.mgr.y.smithi132.stdout:Mar 28 00:17:29 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-y[37327]: 2024-03-28T00:17:29.673+0000 7f08d443c200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-28T00:17:29.977 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:29 smithi119 ceph-mon[31518]: Reconfiguring mgr.x (unknown last config time)... 2024-03-28T00:17:29.977 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:29 smithi119 ceph-mon[31518]: Reconfiguring daemon mgr.x on smithi119 2024-03-28T00:17:29.977 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:29 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-28T00:17:29.978 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:29 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-28T00:17:29.978 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:29 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:17:30.107 INFO:journalctl@ceph.mgr.y.smithi132.stdout:Mar 28 00:17:30 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-y[37327]: 2024-03-28T00:17:30.013+0000 7f08d443c200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-28T00:17:30.107 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:29 smithi132 ceph-mon[35941]: Reconfiguring mgr.x (unknown last config time)... 2024-03-28T00:17:30.107 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:29 smithi132 ceph-mon[35941]: Reconfiguring daemon mgr.x on smithi119 2024-03-28T00:17:30.107 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:29 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-28T00:17:30.107 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:29 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-28T00:17:30.107 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:29 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:17:30.132 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Mar 28 00:17:30 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-z[38206]: 2024-03-28T00:17:30.044+0000 7f3c8b9ef200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-28T00:17:30.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:29 smithi165 ceph-mon[36302]: Reconfiguring mgr.x (unknown last config time)... 2024-03-28T00:17:30.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:29 smithi165 ceph-mon[36302]: Reconfiguring daemon mgr.x on smithi119 2024-03-28T00:17:30.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:29 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-28T00:17:30.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:29 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-28T00:17:30.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:29 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:17:30.452 INFO:journalctl@ceph.mgr.y.smithi132.stdout:Mar 28 00:17:30 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-y[37327]: 2024-03-28T00:17:30.101+0000 7f08d443c200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-28T00:17:30.452 INFO:journalctl@ceph.mgr.y.smithi132.stdout:Mar 28 00:17:30 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-y[37327]: 2024-03-28T00:17:30.301+0000 7f08d443c200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-28T00:17:30.632 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Mar 28 00:17:30 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-z[38206]: 2024-03-28T00:17:30.141+0000 7f3c8b9ef200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-28T00:17:30.632 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Mar 28 00:17:30 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-z[38206]: 2024-03-28T00:17:30.357+0000 7f3c8b9ef200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-28T00:17:30.767 INFO:journalctl@ceph.mgr.y.smithi132.stdout:Mar 28 00:17:30 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-y[37327]: 2024-03-28T00:17:30.484+0000 7f08d443c200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-28T00:17:31.028 INFO:journalctl@ceph.mgr.y.smithi132.stdout:Mar 28 00:17:30 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-y[37327]: 2024-03-28T00:17:30.761+0000 7f08d443c200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-28T00:17:31.028 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:30 smithi132 ceph-mon[35941]: from='client.24116 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi119:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T00:17:31.028 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:30 smithi132 ceph-mon[35941]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T00:17:31.028 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:30 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:17:31.028 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:30 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:17:31.028 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:30 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:17:31.028 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:30 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:17:31.028 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:30 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:17:31.029 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:30 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:17:31.029 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:30 smithi132 ceph-mon[35941]: from='mgr.? 172.21.15.165:0/195254930' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-28T00:17:31.029 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:30 smithi132 ceph-mon[35941]: Standby manager daemon z started 2024-03-28T00:17:31.029 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:30 smithi132 ceph-mon[35941]: from='mgr.? 172.21.15.165:0/195254930' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-28T00:17:31.029 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:30 smithi132 ceph-mon[35941]: from='mgr.? 172.21.15.165:0/195254930' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-28T00:17:31.029 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:30 smithi132 ceph-mon[35941]: from='mgr.? 172.21.15.165:0/195254930' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-28T00:17:31.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:30 smithi165 ceph-mon[36302]: from='client.24116 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi119:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T00:17:31.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:30 smithi165 ceph-mon[36302]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T00:17:31.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:30 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:17:31.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:30 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:17:31.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:30 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:17:31.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:30 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:17:31.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:30 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:17:31.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:30 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:17:31.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:30 smithi165 ceph-mon[36302]: from='mgr.? 172.21.15.165:0/195254930' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-28T00:17:31.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:30 smithi165 ceph-mon[36302]: Standby manager daemon z started 2024-03-28T00:17:31.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:30 smithi165 ceph-mon[36302]: from='mgr.? 172.21.15.165:0/195254930' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-28T00:17:31.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:30 smithi165 ceph-mon[36302]: from='mgr.? 172.21.15.165:0/195254930' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-28T00:17:31.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:30 smithi165 ceph-mon[36302]: from='mgr.? 172.21.15.165:0/195254930' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-28T00:17:31.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:30 smithi119 ceph-mon[31518]: from='client.24116 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi119:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T00:17:31.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:30 smithi119 ceph-mon[31518]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T00:17:31.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:30 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:17:31.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:30 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:17:31.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:30 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:17:31.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:30 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:17:31.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:30 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:17:31.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:30 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:17:31.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:30 smithi119 ceph-mon[31518]: from='mgr.? 172.21.15.165:0/195254930' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-28T00:17:31.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:30 smithi119 ceph-mon[31518]: Standby manager daemon z started 2024-03-28T00:17:31.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:30 smithi119 ceph-mon[31518]: from='mgr.? 172.21.15.165:0/195254930' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-28T00:17:31.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:30 smithi119 ceph-mon[31518]: from='mgr.? 172.21.15.165:0/195254930' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-28T00:17:31.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:30 smithi119 ceph-mon[31518]: from='mgr.? 172.21.15.165:0/195254930' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-28T00:17:31.161 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:17:31.155+0000 7fac17400700 1 -- 172.21.15.119:0/2451886429 <== mon.2 v2:172.21.15.132:3300/0 7 ==== mgrmap(e 14) v1 ==== 91699+0+0 (secure 0 0 0) 0x7fac0c009d10 con 0x7fac20101400 2024-03-28T00:17:31.452 INFO:journalctl@ceph.mgr.y.smithi132.stdout:Mar 28 00:17:31 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-y[37327]: 2024-03-28T00:17:31.022+0000 7f08d443c200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-28T00:17:31.453 INFO:journalctl@ceph.mgr.y.smithi132.stdout:Mar 28 00:17:31 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-y[37327]: 2024-03-28T00:17:31.258+0000 7f08d443c200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-28T00:17:32.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:32 smithi119 ceph-mon[31518]: mgrmap e14: x(active, since 87s), standbys: z 2024-03-28T00:17:32.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:32 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-28T00:17:32.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:32 smithi119 ceph-mon[31518]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T00:17:32.452 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:32 smithi132 ceph-mon[35941]: mgrmap e14: x(active, since 87s), standbys: z 2024-03-28T00:17:32.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:32 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-28T00:17:32.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:32 smithi132 ceph-mon[35941]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T00:17:32.453 INFO:journalctl@ceph.mgr.y.smithi132.stdout:Mar 28 00:17:32 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-y[37327]: 2024-03-28T00:17:32.051+0000 7f08d443c200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-28T00:17:32.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:32 smithi165 ceph-mon[36302]: mgrmap e14: x(active, since 87s), standbys: z 2024-03-28T00:17:32.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:32 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-28T00:17:32.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:32 smithi165 ceph-mon[36302]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T00:17:32.932 INFO:journalctl@ceph.mgr.y.smithi132.stdout:Mar 28 00:17:32 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-y[37327]: 2024-03-28T00:17:32.622+0000 7f08d443c200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-28T00:17:32.933 INFO:journalctl@ceph.mgr.y.smithi132.stdout:Mar 28 00:17:32 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-y[37327]: 2024-03-28T00:17:32.717+0000 7f08d443c200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-28T00:17:32.933 INFO:journalctl@ceph.mgr.y.smithi132.stdout:Mar 28 00:17:32 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-y[37327]: 2024-03-28T00:17:32.925+0000 7f08d443c200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-28T00:17:33.191 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:17:33.184+0000 7fac17400700 1 -- 172.21.15.119:0/2451886429 <== mon.2 v2:172.21.15.132:3300/0 8 ==== mgrmap(e 15) v1 ==== 137235+0+0 (secure 0 0 0) 0x7fac0c02bc60 con 0x7fac20101400 2024-03-28T00:17:33.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:33 smithi132 ceph-mon[35941]: Standby manager daemon y started 2024-03-28T00:17:33.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:33 smithi132 ceph-mon[35941]: from='mgr.? 172.21.15.132:0/3179967148' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-28T00:17:33.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:33 smithi132 ceph-mon[35941]: from='mgr.? 172.21.15.132:0/3179967148' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-28T00:17:33.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:33 smithi132 ceph-mon[35941]: from='mgr.? 172.21.15.132:0/3179967148' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-28T00:17:33.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:33 smithi132 ceph-mon[35941]: from='mgr.? 172.21.15.132:0/3179967148' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-28T00:17:33.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:33 smithi132 ceph-mon[35941]: from='client.? 172.21.15.119:0/1701436423' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7c0b875d-a01e-4294-9c9c-5d0b048b8912"}]: dispatch 2024-03-28T00:17:33.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:33 smithi132 ceph-mon[35941]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7c0b875d-a01e-4294-9c9c-5d0b048b8912"}]: dispatch 2024-03-28T00:17:33.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:33 smithi132 ceph-mon[35941]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7c0b875d-a01e-4294-9c9c-5d0b048b8912"}]': finished 2024-03-28T00:17:33.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:33 smithi132 ceph-mon[35941]: osdmap e5: 1 total, 0 up, 1 in 2024-03-28T00:17:33.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:33 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-28T00:17:33.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:33 smithi165 ceph-mon[36302]: Standby manager daemon y started 2024-03-28T00:17:33.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:33 smithi165 ceph-mon[36302]: from='mgr.? 172.21.15.132:0/3179967148' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-28T00:17:33.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:33 smithi165 ceph-mon[36302]: from='mgr.? 172.21.15.132:0/3179967148' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-28T00:17:33.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:33 smithi165 ceph-mon[36302]: from='mgr.? 172.21.15.132:0/3179967148' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-28T00:17:33.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:33 smithi165 ceph-mon[36302]: from='mgr.? 172.21.15.132:0/3179967148' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-28T00:17:33.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:33 smithi165 ceph-mon[36302]: from='client.? 172.21.15.119:0/1701436423' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7c0b875d-a01e-4294-9c9c-5d0b048b8912"}]: dispatch 2024-03-28T00:17:33.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:33 smithi165 ceph-mon[36302]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7c0b875d-a01e-4294-9c9c-5d0b048b8912"}]: dispatch 2024-03-28T00:17:33.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:33 smithi165 ceph-mon[36302]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7c0b875d-a01e-4294-9c9c-5d0b048b8912"}]': finished 2024-03-28T00:17:33.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:33 smithi165 ceph-mon[36302]: osdmap e5: 1 total, 0 up, 1 in 2024-03-28T00:17:33.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:33 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-28T00:17:33.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:33 smithi119 ceph-mon[31518]: Standby manager daemon y started 2024-03-28T00:17:33.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:33 smithi119 ceph-mon[31518]: from='mgr.? 172.21.15.132:0/3179967148' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-28T00:17:33.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:33 smithi119 ceph-mon[31518]: from='mgr.? 172.21.15.132:0/3179967148' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-28T00:17:33.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:33 smithi119 ceph-mon[31518]: from='mgr.? 172.21.15.132:0/3179967148' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-28T00:17:33.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:33 smithi119 ceph-mon[31518]: from='mgr.? 172.21.15.132:0/3179967148' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-28T00:17:33.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:33 smithi119 ceph-mon[31518]: from='client.? 172.21.15.119:0/1701436423' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7c0b875d-a01e-4294-9c9c-5d0b048b8912"}]: dispatch 2024-03-28T00:17:33.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:33 smithi119 ceph-mon[31518]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7c0b875d-a01e-4294-9c9c-5d0b048b8912"}]: dispatch 2024-03-28T00:17:33.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:33 smithi119 ceph-mon[31518]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7c0b875d-a01e-4294-9c9c-5d0b048b8912"}]': finished 2024-03-28T00:17:33.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:33 smithi119 ceph-mon[31518]: osdmap e5: 1 total, 0 up, 1 in 2024-03-28T00:17:33.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:33 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-28T00:17:34.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:34 smithi132 ceph-mon[35941]: mgrmap e15: x(active, since 89s), standbys: z, y 2024-03-28T00:17:34.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:34 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-28T00:17:34.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:34 smithi132 ceph-mon[35941]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T00:17:34.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:34 smithi132 ceph-mon[35941]: from='client.? 172.21.15.119:0/690328985' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-28T00:17:34.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:34 smithi165 ceph-mon[36302]: mgrmap e15: x(active, since 89s), standbys: z, y 2024-03-28T00:17:34.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:34 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-28T00:17:34.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:34 smithi165 ceph-mon[36302]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T00:17:34.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:34 smithi165 ceph-mon[36302]: from='client.? 172.21.15.119:0/690328985' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-28T00:17:34.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:34 smithi119 ceph-mon[31518]: mgrmap e15: x(active, since 89s), standbys: z, y 2024-03-28T00:17:34.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:34 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-28T00:17:34.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:34 smithi119 ceph-mon[31518]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T00:17:34.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:34 smithi119 ceph-mon[31518]: from='client.? 172.21.15.119:0/690328985' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-28T00:17:36.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:36 smithi132 ceph-mon[35941]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T00:17:37.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:36 smithi165 ceph-mon[36302]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T00:17:37.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:36 smithi119 ceph-mon[31518]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T00:17:38.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:38 smithi132 ceph-mon[35941]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T00:17:39.069 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:38 smithi119 ceph-mon[31518]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T00:17:39.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:38 smithi165 ceph-mon[36302]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T00:17:40.795 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:40 smithi119 ceph-mon[31518]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T00:17:40.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:40 smithi132 ceph-mon[35941]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T00:17:41.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:40 smithi165 ceph-mon[36302]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T00:17:42.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:42 smithi132 ceph-mon[35941]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T00:17:42.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:42 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-28T00:17:42.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:42 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:17:42.965 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:42 smithi119 ceph-mon[31518]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T00:17:42.965 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:42 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-28T00:17:42.965 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:42 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:17:43.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:42 smithi165 ceph-mon[36302]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T00:17:43.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:42 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-28T00:17:43.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:42 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:17:43.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:43 smithi132 ceph-mon[35941]: Deploying daemon osd.0 on smithi119 2024-03-28T00:17:43.965 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:43 smithi119 ceph-mon[31518]: Deploying daemon osd.0 on smithi119 2024-03-28T00:17:44.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:43 smithi165 ceph-mon[36302]: Deploying daemon osd.0 on smithi119 2024-03-28T00:17:44.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:44 smithi132 ceph-mon[35941]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T00:17:44.992 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:44 smithi119 ceph-mon[31518]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T00:17:45.033 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:44 smithi165 ceph-mon[36302]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T00:17:46.803 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:46 smithi119 ceph-mon[31518]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T00:17:47.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:46 smithi165 ceph-mon[36302]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T00:17:47.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:46 smithi132 ceph-mon[35941]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T00:17:49.046 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:48 smithi119 ceph-mon[31518]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T00:17:49.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:48 smithi165 ceph-mon[36302]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T00:17:49.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:48 smithi132 ceph-mon[35941]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T00:17:51.045 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:50 smithi119 ceph-mon[31518]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T00:17:51.045 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:50 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:17:51.045 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:50 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:17:51.045 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:50 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:17:51.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:50 smithi165 ceph-mon[36302]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T00:17:51.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:50 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:17:51.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:50 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:17:51.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:50 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:17:51.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:50 smithi132 ceph-mon[35941]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T00:17:51.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:50 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:17:51.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:50 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:17:51.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:50 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:17:53.023 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:52 smithi119 ceph-mon[31518]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T00:17:53.023 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:52 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:17:53.023 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:52 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:17:53.023 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:52 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:17:53.023 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:52 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:17:53.023 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:52 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:17:53.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:52 smithi165 ceph-mon[36302]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T00:17:53.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:52 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:17:53.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:52 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:17:53.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:52 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:17:53.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:52 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:17:53.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:52 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:17:53.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:52 smithi132 ceph-mon[35941]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T00:17:53.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:52 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:17:53.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:52 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:17:53.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:52 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:17:53.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:52 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:17:53.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:52 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:17:54.746 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:17:54.739+0000 7fac17400700 1 -- 172.21.15.119:0/2451886429 <== mgr.14152 v2:172.21.15.119:6800/1401937096 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fac20062690 con 0x7fac04039200 2024-03-28T00:17:54.746 INFO:teuthology.orchestra.run.smithi119.stdout:Created osd(s) 0 on host 'smithi119' 2024-03-28T00:17:54.749 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:17:54.743+0000 7fac25600700 1 -- 172.21.15.119:0/2451886429 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fac04039200 msgr2=0x7fac0403b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:17:54.749 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:17:54.743+0000 7fac25600700 1 --2- 172.21.15.119:0/2451886429 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fac04039200 0x7fac0403b6c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fac10009c60 tx=0x7fac10009260 comp rx=0 tx=0).stop 2024-03-28T00:17:54.749 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:17:54.743+0000 7fac25600700 1 -- 172.21.15.119:0/2451886429 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fac20101400 msgr2=0x7fac2019ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:17:54.749 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:17:54.743+0000 7fac25600700 1 --2- 172.21.15.119:0/2451886429 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fac20101400 0x7fac2019ac10 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fac0c00b2e0 tx=0x7fac0c00bbb0 comp rx=0 tx=0).stop 2024-03-28T00:17:54.750 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:17:54.743+0000 7fac25600700 1 -- 172.21.15.119:0/2451886429 shutdown_connections 2024-03-28T00:17:54.750 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:17:54.743+0000 7fac25600700 1 --2- 172.21.15.119:0/2451886429 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fac04039200 0x7fac0403b6c0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:17:54.750 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:17:54.743+0000 7fac25600700 1 --2- 172.21.15.119:0/2451886429 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fac20101dc0 0x7fac201a1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:17:54.750 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:17:54.743+0000 7fac25600700 1 --2- 172.21.15.119:0/2451886429 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fac20100aa0 0x7fac2019a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:17:54.750 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:17:54.743+0000 7fac25600700 1 --2- 172.21.15.119:0/2451886429 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fac20101400 0x7fac2019ac10 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:17:54.750 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:17:54.743+0000 7fac25600700 1 -- 172.21.15.119:0/2451886429 >> 172.21.15.119:0/2451886429 conn(0x7fac200fc660 msgr2=0x7fac200fe160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:17:54.750 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:17:54.744+0000 7fac25600700 1 -- 172.21.15.119:0/2451886429 shutdown_connections 2024-03-28T00:17:54.750 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:17:54.744+0000 7fac25600700 1 -- 172.21.15.119:0/2451886429 wait complete. 2024-03-28T00:17:54.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:54 smithi119 ceph-mon[31518]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T00:17:54.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:54 smithi119 ceph-mon[31518]: from='osd.0 [v2:172.21.15.119:6802/580396252,v1:172.21.15.119:6803/580396252]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-28T00:17:54.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:54 smithi119 ceph-mon[31518]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-28T00:17:54.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:54 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:17:55.005 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:54 smithi132 ceph-mon[35941]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T00:17:55.005 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:54 smithi132 ceph-mon[35941]: from='osd.0 [v2:172.21.15.119:6802/580396252,v1:172.21.15.119:6803/580396252]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-28T00:17:55.005 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:54 smithi132 ceph-mon[35941]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-28T00:17:55.005 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:54 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:17:55.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:54 smithi165 ceph-mon[36302]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T00:17:55.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:54 smithi165 ceph-mon[36302]: from='osd.0 [v2:172.21.15.119:6802/580396252,v1:172.21.15.119:6803/580396252]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-28T00:17:55.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:54 smithi165 ceph-mon[36302]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-28T00:17:55.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:54 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:17:55.500 DEBUG:teuthology.orchestra.run.smithi119:osd.0> sudo journalctl -f -n 0 -u ceph-10f73b92-ec98-11ee-95d0-87774f69a715@osd.0.service 2024-03-28T00:17:55.506 INFO:tasks.cephadm:Deploying osd.1 on smithi132 with /dev/vg_nvme/lv_4... 2024-03-28T00:17:55.506 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 10f73b92-ec98-11ee-95d0-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-28T00:17:55.743 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/mon.b/config 2024-03-28T00:17:55.853 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:55 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:17:55.853 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:55 smithi132 ceph-mon[35941]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-28T00:17:55.853 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:55 smithi132 ceph-mon[35941]: osdmap e6: 1 total, 0 up, 1 in 2024-03-28T00:17:55.853 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:55 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-28T00:17:55.853 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:55 smithi132 ceph-mon[35941]: from='osd.0 [v2:172.21.15.119:6802/580396252,v1:172.21.15.119:6803/580396252]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]: dispatch 2024-03-28T00:17:55.853 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:55 smithi132 ceph-mon[35941]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]: dispatch 2024-03-28T00:17:55.853 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:55 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:17:56.053 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:55 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:17:56.053 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:55 smithi119 ceph-mon[31518]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-28T00:17:56.053 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:55 smithi119 ceph-mon[31518]: osdmap e6: 1 total, 0 up, 1 in 2024-03-28T00:17:56.053 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:55 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-28T00:17:56.053 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:55 smithi119 ceph-mon[31518]: from='osd.0 [v2:172.21.15.119:6802/580396252,v1:172.21.15.119:6803/580396252]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]: dispatch 2024-03-28T00:17:56.054 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:55 smithi119 ceph-mon[31518]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]: dispatch 2024-03-28T00:17:56.054 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:55 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:17:56.054 INFO:journalctl@ceph.osd.0.smithi119.stdout:Mar 28 00:17:55 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-0[43761]: 2024-03-28T00:17:55.752+0000 7f65aaa00700 -1 osd.0 0 waiting for initial osdmap 2024-03-28T00:17:56.054 INFO:journalctl@ceph.osd.0.smithi119.stdout:Mar 28 00:17:55 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-0[43761]: 2024-03-28T00:17:55.761+0000 7f65a5800700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-28T00:17:56.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:55 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:17:56.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:55 smithi165 ceph-mon[36302]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-28T00:17:56.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:55 smithi165 ceph-mon[36302]: osdmap e6: 1 total, 0 up, 1 in 2024-03-28T00:17:56.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:55 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-28T00:17:56.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:55 smithi165 ceph-mon[36302]: from='osd.0 [v2:172.21.15.119:6802/580396252,v1:172.21.15.119:6803/580396252]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]: dispatch 2024-03-28T00:17:56.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:55 smithi165 ceph-mon[36302]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]: dispatch 2024-03-28T00:17:56.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:55 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:17:56.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:56 smithi119 ceph-mon[31518]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T00:17:56.907 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:56 smithi119 ceph-mon[31518]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]': finished 2024-03-28T00:17:56.907 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:56 smithi119 ceph-mon[31518]: osdmap e7: 1 total, 0 up, 1 in 2024-03-28T00:17:56.907 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:56 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-28T00:17:56.907 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:56 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-28T00:17:56.919 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:56 smithi132 ceph-mon[35941]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T00:17:56.919 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:56 smithi132 ceph-mon[35941]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]': finished 2024-03-28T00:17:56.920 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:56 smithi132 ceph-mon[35941]: osdmap e7: 1 total, 0 up, 1 in 2024-03-28T00:17:56.920 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:56 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-28T00:17:56.920 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:56 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-28T00:17:57.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:56 smithi165 ceph-mon[36302]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T00:17:57.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:56 smithi165 ceph-mon[36302]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]': finished 2024-03-28T00:17:57.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:56 smithi165 ceph-mon[36302]: osdmap e7: 1 total, 0 up, 1 in 2024-03-28T00:17:57.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:56 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-28T00:17:57.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:56 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-28T00:17:57.366 INFO:teuthology.orchestra.run.smithi132.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-28T00:17:57.366 INFO:teuthology.orchestra.run.smithi132.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-28T00:17:57.367 INFO:teuthology.orchestra.run.smithi132.stderr: stderr: 10+0 records in 2024-03-28T00:17:57.367 INFO:teuthology.orchestra.run.smithi132.stderr:10+0 records out 2024-03-28T00:17:57.367 INFO:teuthology.orchestra.run.smithi132.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112438 s, 933 MB/s 2024-03-28T00:17:57.367 INFO:teuthology.orchestra.run.smithi132.stderr:--> Zapping successful for: 2024-03-28T00:17:57.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:57 smithi119 ceph-mon[31518]: purged_snaps scrub starts 2024-03-28T00:17:57.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:57 smithi119 ceph-mon[31518]: purged_snaps scrub ok 2024-03-28T00:17:57.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:57 smithi119 ceph-mon[31518]: osd.0 [v2:172.21.15.119:6802/580396252,v1:172.21.15.119:6803/580396252] boot 2024-03-28T00:17:57.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:57 smithi119 ceph-mon[31518]: osdmap e8: 1 total, 1 up, 1 in 2024-03-28T00:17:57.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:57 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-28T00:17:57.965 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 10f73b92-ec98-11ee-95d0-87774f69a715 -- ceph orch daemon add osd smithi132:vg_nvme/lv_4 2024-03-28T00:17:58.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:57 smithi165 ceph-mon[36302]: purged_snaps scrub starts 2024-03-28T00:17:58.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:57 smithi165 ceph-mon[36302]: purged_snaps scrub ok 2024-03-28T00:17:58.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:57 smithi165 ceph-mon[36302]: osd.0 [v2:172.21.15.119:6802/580396252,v1:172.21.15.119:6803/580396252] boot 2024-03-28T00:17:58.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:57 smithi165 ceph-mon[36302]: osdmap e8: 1 total, 1 up, 1 in 2024-03-28T00:17:58.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:57 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-28T00:17:58.197 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/mon.b/config 2024-03-28T00:17:58.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:57 smithi132 ceph-mon[35941]: purged_snaps scrub starts 2024-03-28T00:17:58.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:57 smithi132 ceph-mon[35941]: purged_snaps scrub ok 2024-03-28T00:17:58.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:57 smithi132 ceph-mon[35941]: osd.0 [v2:172.21.15.119:6802/580396252,v1:172.21.15.119:6803/580396252] boot 2024-03-28T00:17:58.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:57 smithi132 ceph-mon[35941]: osdmap e8: 1 total, 1 up, 1 in 2024-03-28T00:17:58.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:57 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-28T00:17:58.945 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:58 smithi132 ceph-mon[35941]: pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T00:17:58.945 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:17:58 smithi132 ceph-mon[35941]: osdmap e9: 1 total, 1 up, 1 in 2024-03-28T00:17:59.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:58 smithi165 ceph-mon[36302]: pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T00:17:59.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:17:58 smithi165 ceph-mon[36302]: osdmap e9: 1 total, 1 up, 1 in 2024-03-28T00:17:59.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:58 smithi119 ceph-mon[31518]: pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T00:17:59.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:17:58 smithi119 ceph-mon[31518]: osdmap e9: 1 total, 1 up, 1 in 2024-03-28T00:17:59.683 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:17:59.678+0000 7f5a35189700 1 -- 172.21.15.132:0/2865960541 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5a30105580 msgr2=0x7f5a30107970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:17:59.684 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:17:59.678+0000 7f5a35189700 1 --2- 172.21.15.132:0/2865960541 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5a30105580 0x7f5a30107970 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f5a18009990 tx=0x7f5a18009ca0 comp rx=0 tx=0).stop 2024-03-28T00:17:59.684 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:17:59.678+0000 7f5a35189700 1 -- 172.21.15.132:0/2865960541 shutdown_connections 2024-03-28T00:17:59.684 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:17:59.678+0000 7f5a35189700 1 --2- 172.21.15.132:0/2865960541 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5a30107eb0 0x7f5a3010a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:17:59.684 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:17:59.678+0000 7f5a35189700 1 --2- 172.21.15.132:0/2865960541 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5a30102c50 0x7f5a30105040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:17:59.684 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:17:59.678+0000 7f5a35189700 1 --2- 172.21.15.132:0/2865960541 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5a30105580 0x7f5a30107970 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:17:59.684 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:17:59.678+0000 7f5a35189700 1 -- 172.21.15.132:0/2865960541 >> 172.21.15.132:0/2865960541 conn(0x7f5a300fc660 msgr2=0x7f5a300feaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:17:59.684 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:17:59.678+0000 7f5a35189700 1 -- 172.21.15.132:0/2865960541 shutdown_connections 2024-03-28T00:17:59.684 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:17:59.678+0000 7f5a35189700 1 -- 172.21.15.132:0/2865960541 wait complete. 2024-03-28T00:17:59.688 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:17:59.678+0000 7f5a35189700 1 Processor -- start 2024-03-28T00:17:59.688 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:17:59.679+0000 7f5a35189700 1 -- start start 2024-03-28T00:17:59.688 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:17:59.679+0000 7f5a35189700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5a30102c50 0x7f5a301025b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:17:59.688 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:17:59.679+0000 7f5a35189700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5a30105580 0x7f5a30100c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:17:59.689 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:17:59.679+0000 7f5a35189700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5a30107eb0 0x7f5a30101140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:17:59.689 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:17:59.679+0000 7f5a35189700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a3010de10 con 0x7f5a30102c50 2024-03-28T00:17:59.689 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:17:59.679+0000 7f5a35189700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a3010dc90 con 0x7f5a30107eb0 2024-03-28T00:17:59.689 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:17:59.679+0000 7f5a35189700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a3010df90 con 0x7f5a30105580 2024-03-28T00:17:59.689 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:17:59.679+0000 7f5a2f59e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5a30107eb0 0x7f5a30101140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:17:59.689 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:17:59.679+0000 7f5a2f59e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5a30107eb0 0x7f5a30101140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:48290/0 (socket says 172.21.15.132:48290) 2024-03-28T00:17:59.689 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:17:59.679+0000 7f5a2f59e700 1 -- 172.21.15.132:0/1484597281 learned_addr learned my addr 172.21.15.132:0/1484597281 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-28T00:17:59.689 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:17:59.679+0000 7f5a2659c700 1 --2- 172.21.15.132:0/1484597281 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5a30105580 0x7f5a30100c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:17:59.689 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:17:59.679+0000 7f5a2f59e700 1 -- 172.21.15.132:0/1484597281 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5a30105580 msgr2=0x7f5a30100c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:17:59.690 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:17:59.679+0000 7f5a2f59e700 1 --2- 172.21.15.132:0/1484597281 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5a30105580 0x7f5a30100c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:17:59.690 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:17:59.679+0000 7f5a2ed9d700 1 --2- 172.21.15.132:0/1484597281 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5a30102c50 0x7f5a301025b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:17:59.690 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:17:59.679+0000 7f5a2f59e700 1 -- 172.21.15.132:0/1484597281 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5a30102c50 msgr2=0x7f5a301025b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:17:59.690 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:17:59.679+0000 7f5a2f59e700 1 --2- 172.21.15.132:0/1484597281 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5a30102c50 0x7f5a301025b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:17:59.690 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:17:59.679+0000 7f5a2f59e700 1 -- 172.21.15.132:0/1484597281 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a301019f0 con 0x7f5a30107eb0 2024-03-28T00:17:59.690 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:17:59.679+0000 7f5a2659c700 1 --2- 172.21.15.132:0/1484597281 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5a30105580 0x7f5a30100c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T00:17:59.690 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:17:59.680+0000 7f5a2f59e700 1 --2- 172.21.15.132:0/1484597281 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5a30107eb0 0x7f5a30101140 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f5a28007d40 tx=0x7f5a2800a340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:17:59.691 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:17:59.680+0000 7f5a2cd99700 1 -- 172.21.15.132:0/1484597281 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a28016020 con 0x7f5a30107eb0 2024-03-28T00:17:59.691 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:17:59.680+0000 7f5a35189700 1 -- 172.21.15.132:0/1484597281 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a301a25c0 con 0x7f5a30107eb0 2024-03-28T00:17:59.691 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:17:59.680+0000 7f5a2cd99700 1 -- 172.21.15.132:0/1484597281 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5a2800ad40 con 0x7f5a30107eb0 2024-03-28T00:17:59.691 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:17:59.680+0000 7f5a2cd99700 1 -- 172.21.15.132:0/1484597281 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a28011740 con 0x7f5a30107eb0 2024-03-28T00:17:59.691 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:17:59.680+0000 7f5a35189700 1 -- 172.21.15.132:0/1484597281 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a301a2b80 con 0x7f5a30107eb0 2024-03-28T00:17:59.691 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:17:59.681+0000 7f5a35189700 1 -- 172.21.15.132:0/1484597281 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a30068e50 con 0x7f5a30107eb0 2024-03-28T00:17:59.691 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:17:59.682+0000 7f5a2cd99700 1 -- 172.21.15.132:0/1484597281 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 15) v1 ==== 137235+0+0 (secure 0 0 0) 0x7f5a28019020 con 0x7f5a30107eb0 2024-03-28T00:17:59.691 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:17:59.683+0000 7f5a2cd99700 1 --2- 172.21.15.132:0/1484597281 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f5a1c0a2800 0x7f5a1c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:17:59.691 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:17:59.683+0000 7f5a2cd99700 1 -- 172.21.15.132:0/1484597281 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1767+0+0 (secure 0 0 0) 0x7f5a280cd450 con 0x7f5a30107eb0 2024-03-28T00:17:59.692 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:17:59.683+0000 7f5a2ed9d700 1 --2- 172.21.15.132:0/1484597281 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f5a1c0a2800 0x7f5a1c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:17:59.692 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:17:59.684+0000 7f5a2ed9d700 1 --2- 172.21.15.132:0/1484597281 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f5a1c0a2800 0x7f5a1c0a4cc0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5a20006fd0 tx=0x7f5a20008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:17:59.692 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:17:59.687+0000 7f5a2cd99700 1 -- 172.21.15.132:0/1484597281 <== mon.2 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f5a28020020 con 0x7f5a30107eb0 2024-03-28T00:17:59.896 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:17:59.890+0000 7f5a35189700 1 -- 172.21.15.132:0/1484597281 --> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi132:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f5a300625d0 con 0x7f5a1c0a2800 2024-03-28T00:18:01.104 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:00 smithi132 ceph-mon[35941]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-28T00:18:01.104 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:00 smithi132 ceph-mon[35941]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi132:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T00:18:01.104 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:00 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-28T00:18:01.104 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:00 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-28T00:18:01.104 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:00 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:18:01.104 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:00 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:18:01.105 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:00 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:18:01.105 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:00 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:18:01.105 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:00 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:18:01.105 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:00 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:18:01.105 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:00 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:18:01.105 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:00 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:18:01.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:00 smithi165 ceph-mon[36302]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-28T00:18:01.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:00 smithi165 ceph-mon[36302]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi132:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T00:18:01.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:00 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-28T00:18:01.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:00 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-28T00:18:01.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:00 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:18:01.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:00 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:18:01.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:00 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:18:01.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:00 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:18:01.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:00 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:18:01.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:00 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:18:01.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:00 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:18:01.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:00 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:18:01.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:00 smithi119 ceph-mon[31518]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-28T00:18:01.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:00 smithi119 ceph-mon[31518]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi132:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T00:18:01.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:00 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-28T00:18:01.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:00 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-28T00:18:01.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:00 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:18:01.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:00 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:18:01.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:00 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:18:01.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:00 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:18:01.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:00 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:18:01.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:00 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:18:01.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:00 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:18:01.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:00 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:18:02.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:01 smithi165 ceph-mon[36302]: Detected new or changed devices on smithi119 2024-03-28T00:18:02.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:01 smithi165 ceph-mon[36302]: Adjusting osd_memory_target on smithi119 to 17091M 2024-03-28T00:18:02.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:01 smithi119 ceph-mon[31518]: Detected new or changed devices on smithi119 2024-03-28T00:18:02.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:01 smithi119 ceph-mon[31518]: Adjusting osd_memory_target on smithi119 to 17091M 2024-03-28T00:18:02.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:01 smithi132 ceph-mon[35941]: Detected new or changed devices on smithi119 2024-03-28T00:18:02.205 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:01 smithi132 ceph-mon[35941]: Adjusting osd_memory_target on smithi119 to 17091M 2024-03-28T00:18:03.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:02 smithi165 ceph-mon[36302]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-28T00:18:03.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:02 smithi165 ceph-mon[36302]: from='client.? 172.21.15.132:0/309785' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5567f7c2-8ea4-4789-9ea1-467793324fb0"}]: dispatch 2024-03-28T00:18:03.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:02 smithi165 ceph-mon[36302]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5567f7c2-8ea4-4789-9ea1-467793324fb0"}]: dispatch 2024-03-28T00:18:03.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:02 smithi165 ceph-mon[36302]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5567f7c2-8ea4-4789-9ea1-467793324fb0"}]': finished 2024-03-28T00:18:03.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:02 smithi165 ceph-mon[36302]: osdmap e10: 2 total, 1 up, 2 in 2024-03-28T00:18:03.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:02 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-28T00:18:03.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:02 smithi119 ceph-mon[31518]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-28T00:18:03.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:02 smithi119 ceph-mon[31518]: from='client.? 172.21.15.132:0/309785' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5567f7c2-8ea4-4789-9ea1-467793324fb0"}]: dispatch 2024-03-28T00:18:03.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:02 smithi119 ceph-mon[31518]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5567f7c2-8ea4-4789-9ea1-467793324fb0"}]: dispatch 2024-03-28T00:18:03.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:02 smithi119 ceph-mon[31518]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5567f7c2-8ea4-4789-9ea1-467793324fb0"}]': finished 2024-03-28T00:18:03.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:02 smithi119 ceph-mon[31518]: osdmap e10: 2 total, 1 up, 2 in 2024-03-28T00:18:03.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:02 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-28T00:18:03.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:02 smithi132 ceph-mon[35941]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-28T00:18:03.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:02 smithi132 ceph-mon[35941]: from='client.? 172.21.15.132:0/309785' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5567f7c2-8ea4-4789-9ea1-467793324fb0"}]: dispatch 2024-03-28T00:18:03.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:02 smithi132 ceph-mon[35941]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5567f7c2-8ea4-4789-9ea1-467793324fb0"}]: dispatch 2024-03-28T00:18:03.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:02 smithi132 ceph-mon[35941]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5567f7c2-8ea4-4789-9ea1-467793324fb0"}]': finished 2024-03-28T00:18:03.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:02 smithi132 ceph-mon[35941]: osdmap e10: 2 total, 1 up, 2 in 2024-03-28T00:18:03.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:02 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-28T00:18:04.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:03 smithi165 ceph-mon[36302]: from='client.? 172.21.15.132:0/2425935658' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-28T00:18:04.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:03 smithi119 ceph-mon[31518]: from='client.? 172.21.15.132:0/2425935658' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-28T00:18:04.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:03 smithi132 ceph-mon[35941]: from='client.? 172.21.15.132:0/2425935658' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-28T00:18:05.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:04 smithi165 ceph-mon[36302]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-28T00:18:05.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:04 smithi119 ceph-mon[31518]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-28T00:18:05.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:04 smithi132 ceph-mon[35941]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-28T00:18:07.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:06 smithi165 ceph-mon[36302]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-28T00:18:07.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:06 smithi119 ceph-mon[31518]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-28T00:18:07.187 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:06 smithi132 ceph-mon[35941]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-28T00:18:09.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:08 smithi165 ceph-mon[36302]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-28T00:18:09.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:08 smithi119 ceph-mon[31518]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-28T00:18:09.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:08 smithi132 ceph-mon[35941]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-28T00:18:10.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:09 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-28T00:18:10.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:09 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:18:10.152 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:09 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-28T00:18:10.153 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:09 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:18:10.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:09 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-28T00:18:10.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:09 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:18:10.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:10 smithi132 ceph-mon[35941]: Deploying daemon osd.1 on smithi132 2024-03-28T00:18:10.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:10 smithi132 ceph-mon[35941]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-28T00:18:11.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:10 smithi165 ceph-mon[36302]: Deploying daemon osd.1 on smithi132 2024-03-28T00:18:11.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:10 smithi165 ceph-mon[36302]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-28T00:18:11.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:10 smithi119 ceph-mon[31518]: Deploying daemon osd.1 on smithi132 2024-03-28T00:18:11.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:10 smithi119 ceph-mon[31518]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-28T00:18:13.034 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:12 smithi132 ceph-mon[35941]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-28T00:18:13.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:12 smithi165 ceph-mon[36302]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-28T00:18:13.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:12 smithi119 ceph-mon[31518]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-28T00:18:15.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:14 smithi165 ceph-mon[36302]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-28T00:18:15.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:14 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:18:15.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:14 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:18:15.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:14 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:18:15.150 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:14 smithi132 ceph-mon[35941]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-28T00:18:15.151 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:14 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:18:15.151 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:14 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:18:15.151 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:14 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:18:15.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:14 smithi119 ceph-mon[31518]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-28T00:18:15.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:14 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:18:15.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:14 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:18:15.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:14 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:18:17.011 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:16 smithi132 ceph-mon[35941]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-28T00:18:17.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:16 smithi165 ceph-mon[36302]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-28T00:18:17.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:16 smithi119 ceph-mon[31518]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-28T00:18:18.085 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:18:18.079+0000 7f5a2cd99700 1 -- 172.21.15.132:0/1484597281 <== mgr.14152 v2:172.21.15.119:6800/1401937096 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f5a300625d0 con 0x7f5a1c0a2800 2024-03-28T00:18:18.086 INFO:teuthology.orchestra.run.smithi132.stdout:Created osd(s) 1 on host 'smithi132' 2024-03-28T00:18:18.088 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:18:18.081+0000 7f5a35189700 1 -- 172.21.15.132:0/1484597281 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f5a1c0a2800 msgr2=0x7f5a1c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:18:18.088 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:18:18.081+0000 7f5a35189700 1 --2- 172.21.15.132:0/1484597281 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f5a1c0a2800 0x7f5a1c0a4cc0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5a20006fd0 tx=0x7f5a20008040 comp rx=0 tx=0).stop 2024-03-28T00:18:18.088 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:18:18.081+0000 7f5a35189700 1 -- 172.21.15.132:0/1484597281 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5a30107eb0 msgr2=0x7f5a30101140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:18:18.088 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:18:18.081+0000 7f5a35189700 1 --2- 172.21.15.132:0/1484597281 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5a30107eb0 0x7f5a30101140 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f5a28007d40 tx=0x7f5a2800a340 comp rx=0 tx=0).stop 2024-03-28T00:18:18.088 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:18:18.081+0000 7f5a35189700 1 -- 172.21.15.132:0/1484597281 shutdown_connections 2024-03-28T00:18:18.088 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:18:18.081+0000 7f5a35189700 1 --2- 172.21.15.132:0/1484597281 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f5a1c0a2800 0x7f5a1c0a4cc0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:18:18.088 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:18:18.081+0000 7f5a35189700 1 --2- 172.21.15.132:0/1484597281 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5a30107eb0 0x7f5a30101140 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:18:18.088 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:18:18.081+0000 7f5a35189700 1 --2- 172.21.15.132:0/1484597281 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5a30102c50 0x7f5a301025b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:18:18.088 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:18:18.081+0000 7f5a35189700 1 --2- 172.21.15.132:0/1484597281 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5a30105580 0x7f5a30100c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:18:18.088 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:18:18.081+0000 7f5a35189700 1 -- 172.21.15.132:0/1484597281 >> 172.21.15.132:0/1484597281 conn(0x7f5a300fc660 msgr2=0x7f5a30103590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:18:18.089 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:18:18.081+0000 7f5a35189700 1 -- 172.21.15.132:0/1484597281 shutdown_connections 2024-03-28T00:18:18.089 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:18:18.081+0000 7f5a35189700 1 -- 172.21.15.132:0/1484597281 wait complete. 2024-03-28T00:18:18.376 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:18 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:18:18.376 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:18 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:18:18.376 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:18 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:18:18.376 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:18 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:18:18.377 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:18 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:18:18.377 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:18 smithi132 ceph-mon[35941]: pgmap v67: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-28T00:18:18.377 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:18 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:18:18.377 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:18 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:18:18.377 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:18 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:18:18.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:18 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:18:18.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:18 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:18:18.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:18 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:18:18.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:18 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:18:18.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:18 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:18:18.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:18 smithi165 ceph-mon[36302]: pgmap v67: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-28T00:18:18.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:18 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:18:18.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:18 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:18:18.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:18 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:18:18.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:18 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:18:18.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:18 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:18:18.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:18 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:18:18.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:18 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:18:18.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:18 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:18:18.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:18 smithi119 ceph-mon[31518]: pgmap v67: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-28T00:18:18.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:18 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:18:18.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:18 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:18:18.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:18 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:18:18.737 DEBUG:teuthology.orchestra.run.smithi132:osd.1> sudo journalctl -f -n 0 -u ceph-10f73b92-ec98-11ee-95d0-87774f69a715@osd.1.service 2024-03-28T00:18:18.739 INFO:tasks.cephadm:Deploying osd.2 on smithi165 with /dev/vg_nvme/lv_4... 2024-03-28T00:18:18.739 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 10f73b92-ec98-11ee-95d0-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-28T00:18:18.986 INFO:teuthology.orchestra.run.smithi165.stderr:Inferring config /var/lib/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/mon.c/config 2024-03-28T00:18:19.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:19 smithi165 ceph-mon[36302]: from='osd.1 [v2:172.21.15.132:6800/2500318888,v1:172.21.15.132:6801/2500318888]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-28T00:18:19.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:19 smithi165 ceph-mon[36302]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-28T00:18:19.452 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:19 smithi132 ceph-mon[35941]: from='osd.1 [v2:172.21.15.132:6800/2500318888,v1:172.21.15.132:6801/2500318888]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-28T00:18:19.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:19 smithi132 ceph-mon[35941]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-28T00:18:19.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:19 smithi119 ceph-mon[31518]: from='osd.1 [v2:172.21.15.132:6800/2500318888,v1:172.21.15.132:6801/2500318888]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-28T00:18:19.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:19 smithi119 ceph-mon[31518]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-28T00:18:20.271 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:20 smithi165 ceph-mon[36302]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-28T00:18:20.271 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:20 smithi165 ceph-mon[36302]: osdmap e11: 2 total, 1 up, 2 in 2024-03-28T00:18:20.271 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:20 smithi165 ceph-mon[36302]: from='osd.1 [v2:172.21.15.132:6800/2500318888,v1:172.21.15.132:6801/2500318888]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]: dispatch 2024-03-28T00:18:20.272 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:20 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-28T00:18:20.272 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:20 smithi165 ceph-mon[36302]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]: dispatch 2024-03-28T00:18:20.272 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:20 smithi165 ceph-mon[36302]: pgmap v69: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-28T00:18:20.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:20 smithi132 ceph-mon[35941]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-28T00:18:20.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:20 smithi132 ceph-mon[35941]: osdmap e11: 2 total, 1 up, 2 in 2024-03-28T00:18:20.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:20 smithi132 ceph-mon[35941]: from='osd.1 [v2:172.21.15.132:6800/2500318888,v1:172.21.15.132:6801/2500318888]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]: dispatch 2024-03-28T00:18:20.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:20 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-28T00:18:20.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:20 smithi132 ceph-mon[35941]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]: dispatch 2024-03-28T00:18:20.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:20 smithi132 ceph-mon[35941]: pgmap v69: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-28T00:18:20.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:20 smithi119 ceph-mon[31518]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-28T00:18:20.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:20 smithi119 ceph-mon[31518]: osdmap e11: 2 total, 1 up, 2 in 2024-03-28T00:18:20.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:20 smithi119 ceph-mon[31518]: from='osd.1 [v2:172.21.15.132:6800/2500318888,v1:172.21.15.132:6801/2500318888]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]: dispatch 2024-03-28T00:18:20.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:20 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-28T00:18:20.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:20 smithi119 ceph-mon[31518]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]: dispatch 2024-03-28T00:18:20.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:20 smithi119 ceph-mon[31518]: pgmap v69: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-28T00:18:20.841 INFO:teuthology.orchestra.run.smithi165.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-28T00:18:20.841 INFO:teuthology.orchestra.run.smithi165.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-28T00:18:20.842 INFO:teuthology.orchestra.run.smithi165.stderr: stderr: 10+0 records in 2024-03-28T00:18:20.842 INFO:teuthology.orchestra.run.smithi165.stderr:10+0 records out 2024-03-28T00:18:20.842 INFO:teuthology.orchestra.run.smithi165.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.013992 s, 749 MB/s 2024-03-28T00:18:20.842 INFO:teuthology.orchestra.run.smithi165.stderr:--> Zapping successful for: 2024-03-28T00:18:20.952 INFO:journalctl@ceph.osd.1.smithi132.stdout:Mar 28 00:18:20 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-1[40548]: 2024-03-28T00:18:20.646+0000 7f0b3a19f700 -1 osd.1 0 waiting for initial osdmap 2024-03-28T00:18:20.952 INFO:journalctl@ceph.osd.1.smithi132.stdout:Mar 28 00:18:20 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-1[40548]: 2024-03-28T00:18:20.650+0000 7f0b31789700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-28T00:18:21.323 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:21 smithi132 ceph-mon[35941]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]': finished 2024-03-28T00:18:21.323 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:21 smithi132 ceph-mon[35941]: osdmap e12: 2 total, 1 up, 2 in 2024-03-28T00:18:21.323 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:21 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-28T00:18:21.324 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:21 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-28T00:18:21.324 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:21 smithi132 ceph-mon[35941]: from='osd.1 ' entity='osd.1' 2024-03-28T00:18:21.445 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:21 smithi165 ceph-mon[36302]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]': finished 2024-03-28T00:18:21.445 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:21 smithi165 ceph-mon[36302]: osdmap e12: 2 total, 1 up, 2 in 2024-03-28T00:18:21.445 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:21 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-28T00:18:21.445 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:21 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-28T00:18:21.445 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:21 smithi165 ceph-mon[36302]: from='osd.1 ' entity='osd.1' 2024-03-28T00:18:21.475 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 10f73b92-ec98-11ee-95d0-87774f69a715 -- ceph orch daemon add osd smithi165:vg_nvme/lv_4 2024-03-28T00:18:21.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:21 smithi119 ceph-mon[31518]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]': finished 2024-03-28T00:18:21.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:21 smithi119 ceph-mon[31518]: osdmap e12: 2 total, 1 up, 2 in 2024-03-28T00:18:21.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:21 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-28T00:18:21.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:21 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-28T00:18:21.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:21 smithi119 ceph-mon[31518]: from='osd.1 ' entity='osd.1' 2024-03-28T00:18:21.713 INFO:teuthology.orchestra.run.smithi165.stderr:Inferring config /var/lib/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/mon.c/config 2024-03-28T00:18:22.524 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:22 smithi165 ceph-mon[36302]: purged_snaps scrub starts 2024-03-28T00:18:22.524 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:22 smithi165 ceph-mon[36302]: purged_snaps scrub ok 2024-03-28T00:18:22.524 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:22 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-28T00:18:22.524 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:22 smithi165 ceph-mon[36302]: pgmap v71: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-28T00:18:22.524 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:22 smithi165 ceph-mon[36302]: Detected new or changed devices on smithi132 2024-03-28T00:18:22.525 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:22 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:18:22.525 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:22 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:18:22.525 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:22 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:18:22.525 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:22 smithi165 ceph-mon[36302]: Adjusting osd_memory_target on smithi132 to 17080M 2024-03-28T00:18:22.525 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:22 smithi165 ceph-mon[36302]: osd.1 [v2:172.21.15.132:6800/2500318888,v1:172.21.15.132:6801/2500318888] boot 2024-03-28T00:18:22.525 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:22 smithi165 ceph-mon[36302]: osdmap e13: 2 total, 2 up, 2 in 2024-03-28T00:18:22.525 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:22 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-28T00:18:22.525 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:22 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:18:22.525 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:22 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:18:22.525 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:22 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:18:22.525 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:22 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:18:22.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:22 smithi119 ceph-mon[31518]: purged_snaps scrub starts 2024-03-28T00:18:22.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:22 smithi119 ceph-mon[31518]: purged_snaps scrub ok 2024-03-28T00:18:22.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:22 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-28T00:18:22.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:22 smithi119 ceph-mon[31518]: pgmap v71: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-28T00:18:22.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:22 smithi119 ceph-mon[31518]: Detected new or changed devices on smithi132 2024-03-28T00:18:22.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:22 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:18:22.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:22 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:18:22.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:22 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:18:22.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:22 smithi119 ceph-mon[31518]: Adjusting osd_memory_target on smithi132 to 17080M 2024-03-28T00:18:22.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:22 smithi119 ceph-mon[31518]: osd.1 [v2:172.21.15.132:6800/2500318888,v1:172.21.15.132:6801/2500318888] boot 2024-03-28T00:18:22.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:22 smithi119 ceph-mon[31518]: osdmap e13: 2 total, 2 up, 2 in 2024-03-28T00:18:22.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:22 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-28T00:18:22.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:22 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:18:22.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:22 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:18:22.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:22 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:18:22.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:22 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:18:22.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:22 smithi132 ceph-mon[35941]: purged_snaps scrub starts 2024-03-28T00:18:22.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:22 smithi132 ceph-mon[35941]: purged_snaps scrub ok 2024-03-28T00:18:22.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:22 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-28T00:18:22.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:22 smithi132 ceph-mon[35941]: pgmap v71: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-28T00:18:22.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:22 smithi132 ceph-mon[35941]: Detected new or changed devices on smithi132 2024-03-28T00:18:22.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:22 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:18:22.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:22 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:18:22.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:22 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:18:22.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:22 smithi132 ceph-mon[35941]: Adjusting osd_memory_target on smithi132 to 17080M 2024-03-28T00:18:22.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:22 smithi132 ceph-mon[35941]: osd.1 [v2:172.21.15.132:6800/2500318888,v1:172.21.15.132:6801/2500318888] boot 2024-03-28T00:18:22.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:22 smithi132 ceph-mon[35941]: osdmap e13: 2 total, 2 up, 2 in 2024-03-28T00:18:22.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:22 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-28T00:18:22.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:22 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:18:22.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:22 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:18:22.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:22 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:18:22.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:22 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:18:23.263 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:18:23.258+0000 7f68b1127700 1 -- 172.21.15.165:0/387533792 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f68ac102c40 msgr2=0x7f68ac105030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:18:23.264 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:18:23.258+0000 7f68b1127700 1 --2- 172.21.15.165:0/387533792 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f68ac102c40 0x7f68ac105030 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f689c009990 tx=0x7f689c009ca0 comp rx=0 tx=0).stop 2024-03-28T00:18:23.264 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:18:23.258+0000 7f68b1127700 1 -- 172.21.15.165:0/387533792 shutdown_connections 2024-03-28T00:18:23.264 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:18:23.258+0000 7f68b1127700 1 --2- 172.21.15.165:0/387533792 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f68ac107ea0 0x7f68ac10a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:18:23.264 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:18:23.258+0000 7f68b1127700 1 --2- 172.21.15.165:0/387533792 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f68ac102c40 0x7f68ac105030 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:18:23.264 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:18:23.258+0000 7f68b1127700 1 --2- 172.21.15.165:0/387533792 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f68ac105570 0x7f68ac107960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:18:23.264 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:18:23.258+0000 7f68b1127700 1 -- 172.21.15.165:0/387533792 >> 172.21.15.165:0/387533792 conn(0x7f68ac0fc690 msgr2=0x7f68ac0feab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:18:23.264 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:18:23.258+0000 7f68b1127700 1 -- 172.21.15.165:0/387533792 shutdown_connections 2024-03-28T00:18:23.264 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:18:23.258+0000 7f68b1127700 1 -- 172.21.15.165:0/387533792 wait complete. 2024-03-28T00:18:23.266 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:18:23.258+0000 7f68b1127700 1 Processor -- start 2024-03-28T00:18:23.266 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:18:23.259+0000 7f68b1127700 1 -- start start 2024-03-28T00:18:23.266 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:18:23.259+0000 7f68b1127700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f68ac102c40 0x7f68ac198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:18:23.266 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:18:23.259+0000 7f68b1127700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f68ac105570 0x7f68ac1989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:18:23.266 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:18:23.259+0000 7f68b1127700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f68ac107ea0 0x7f68ac19f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:18:23.267 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:18:23.259+0000 7f68b1127700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68ac10de70 con 0x7f68ac102c40 2024-03-28T00:18:23.267 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:18:23.259+0000 7f68b1127700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68ac10dcf0 con 0x7f68ac107ea0 2024-03-28T00:18:23.267 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:18:23.259+0000 7f68b1127700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68ac10dff0 con 0x7f68ac105570 2024-03-28T00:18:23.267 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:18:23.259+0000 7f68a259c700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f68ac105570 0x7f68ac1989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:18:23.267 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:18:23.259+0000 7f68a259c700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f68ac105570 0x7f68ac1989c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:60446/0 (socket says 172.21.15.165:60446) 2024-03-28T00:18:23.267 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:18:23.259+0000 7f68a259c700 1 -- 172.21.15.165:0/1071179928 learned_addr learned my addr 172.21.15.165:0/1071179928 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-03-28T00:18:23.267 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:18:23.259+0000 7f68ab59e700 1 --2- 172.21.15.165:0/1071179928 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f68ac107ea0 0x7f68ac19f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:18:23.267 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:18:23.259+0000 7f68a259c700 1 -- 172.21.15.165:0/1071179928 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f68ac107ea0 msgr2=0x7f68ac19f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:18:23.267 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:18:23.259+0000 7f68a259c700 1 --2- 172.21.15.165:0/1071179928 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f68ac107ea0 0x7f68ac19f9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:18:23.267 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:18:23.259+0000 7f68a259c700 1 -- 172.21.15.165:0/1071179928 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f68ac102c40 msgr2=0x7f68ac198480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:18:23.267 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:18:23.259+0000 7f68a259c700 1 --2- 172.21.15.165:0/1071179928 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f68ac102c40 0x7f68ac198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:18:23.267 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:18:23.259+0000 7f68a259c700 1 -- 172.21.15.165:0/1071179928 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68ac1a0090 con 0x7f68ac105570 2024-03-28T00:18:23.267 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:18:23.260+0000 7f68a259c700 1 --2- 172.21.15.165:0/1071179928 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f68ac105570 0x7f68ac1989c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f689400c9a0 tx=0x7f689400ccb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:18:23.267 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:18:23.260+0000 7f68a8d99700 1 -- 172.21.15.165:0/1071179928 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6894007a60 con 0x7f68ac105570 2024-03-28T00:18:23.268 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:18:23.260+0000 7f68a8d99700 1 -- 172.21.15.165:0/1071179928 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6894004d10 con 0x7f68ac105570 2024-03-28T00:18:23.268 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:18:23.260+0000 7f68b1127700 1 -- 172.21.15.165:0/1071179928 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68ac1a0380 con 0x7f68ac105570 2024-03-28T00:18:23.268 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:18:23.260+0000 7f68a8d99700 1 -- 172.21.15.165:0/1071179928 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6894005690 con 0x7f68ac105570 2024-03-28T00:18:23.268 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:18:23.260+0000 7f68b1127700 1 -- 172.21.15.165:0/1071179928 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68ac1a08f0 con 0x7f68ac105570 2024-03-28T00:18:23.268 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:18:23.261+0000 7f68b1127700 1 -- 172.21.15.165:0/1071179928 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68ac068e50 con 0x7f68ac105570 2024-03-28T00:18:23.268 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:18:23.262+0000 7f68a8d99700 1 -- 172.21.15.165:0/1071179928 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 15) v1 ==== 137235+0+0 (secure 0 0 0) 0x7f6894007c00 con 0x7f68ac105570 2024-03-28T00:18:23.269 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:18:23.262+0000 7f68a8d99700 1 --2- 172.21.15.165:0/1071179928 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f68980a2800 0x7f68980a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:18:23.269 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:18:23.262+0000 7f68a8d99700 1 -- 172.21.15.165:0/1071179928 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2309+0+0 (secure 0 0 0) 0x7f68940ccf90 con 0x7f68ac105570 2024-03-28T00:18:23.269 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:18:23.263+0000 7f68aad9d700 1 --2- 172.21.15.165:0/1071179928 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f68980a2800 0x7f68980a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:18:23.269 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:18:23.264+0000 7f68aad9d700 1 --2- 172.21.15.165:0/1071179928 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f68980a2800 0x7f68980a4cc0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f689c005cb0 tx=0x7f689c005c20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:18:23.272 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:18:23.267+0000 7f68a8d99700 1 -- 172.21.15.165:0/1071179928 <== mon.1 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f689409b910 con 0x7f68ac105570 2024-03-28T00:18:23.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:18:23.470+0000 7f68b1127700 1 -- 172.21.15.165:0/1071179928 --> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi165:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f68ac0625d0 con 0x7f68980a2800 2024-03-28T00:18:24.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:24 smithi165 ceph-mon[36302]: osdmap e14: 2 total, 2 up, 2 in 2024-03-28T00:18:24.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:24 smithi165 ceph-mon[36302]: from='client.24157 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi165:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T00:18:24.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:24 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-28T00:18:24.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:24 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-28T00:18:24.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:24 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:18:24.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:24 smithi165 ceph-mon[36302]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-28T00:18:24.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:24 smithi119 ceph-mon[31518]: osdmap e14: 2 total, 2 up, 2 in 2024-03-28T00:18:24.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:24 smithi119 ceph-mon[31518]: from='client.24157 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi165:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T00:18:24.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:24 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-28T00:18:24.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:24 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-28T00:18:24.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:24 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:18:24.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:24 smithi119 ceph-mon[31518]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-28T00:18:24.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:24 smithi132 ceph-mon[35941]: osdmap e14: 2 total, 2 up, 2 in 2024-03-28T00:18:24.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:24 smithi132 ceph-mon[35941]: from='client.24157 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi165:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T00:18:24.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:24 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-28T00:18:24.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:24 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-28T00:18:24.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:24 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:18:24.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:24 smithi132 ceph-mon[35941]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-28T00:18:26.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:26 smithi132 ceph-mon[35941]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-28T00:18:26.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:26 smithi132 ceph-mon[35941]: from='client.? 172.21.15.165:0/88305083' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ee54dbd0-24b4-4ee6-988c-ce71491199d7"}]: dispatch 2024-03-28T00:18:26.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:26 smithi132 ceph-mon[35941]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ee54dbd0-24b4-4ee6-988c-ce71491199d7"}]: dispatch 2024-03-28T00:18:26.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:26 smithi132 ceph-mon[35941]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ee54dbd0-24b4-4ee6-988c-ce71491199d7"}]': finished 2024-03-28T00:18:26.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:26 smithi132 ceph-mon[35941]: osdmap e15: 3 total, 2 up, 3 in 2024-03-28T00:18:26.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:26 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-28T00:18:27.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:26 smithi165 ceph-mon[36302]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-28T00:18:27.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:26 smithi165 ceph-mon[36302]: from='client.? 172.21.15.165:0/88305083' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ee54dbd0-24b4-4ee6-988c-ce71491199d7"}]: dispatch 2024-03-28T00:18:27.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:26 smithi165 ceph-mon[36302]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ee54dbd0-24b4-4ee6-988c-ce71491199d7"}]: dispatch 2024-03-28T00:18:27.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:26 smithi165 ceph-mon[36302]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ee54dbd0-24b4-4ee6-988c-ce71491199d7"}]': finished 2024-03-28T00:18:27.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:26 smithi165 ceph-mon[36302]: osdmap e15: 3 total, 2 up, 3 in 2024-03-28T00:18:27.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:26 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-28T00:18:27.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:26 smithi119 ceph-mon[31518]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-28T00:18:27.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:26 smithi119 ceph-mon[31518]: from='client.? 172.21.15.165:0/88305083' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ee54dbd0-24b4-4ee6-988c-ce71491199d7"}]: dispatch 2024-03-28T00:18:27.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:26 smithi119 ceph-mon[31518]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ee54dbd0-24b4-4ee6-988c-ce71491199d7"}]: dispatch 2024-03-28T00:18:27.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:26 smithi119 ceph-mon[31518]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ee54dbd0-24b4-4ee6-988c-ce71491199d7"}]': finished 2024-03-28T00:18:27.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:26 smithi119 ceph-mon[31518]: osdmap e15: 3 total, 2 up, 3 in 2024-03-28T00:18:27.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:26 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-28T00:18:27.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:27 smithi132 ceph-mon[35941]: from='client.? 172.21.15.165:0/1414752055' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-28T00:18:28.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:27 smithi165 ceph-mon[36302]: from='client.? 172.21.15.165:0/1414752055' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-28T00:18:28.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:27 smithi119 ceph-mon[31518]: from='client.? 172.21.15.165:0/1414752055' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-28T00:18:28.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:28 smithi132 ceph-mon[35941]: pgmap v77: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-28T00:18:29.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:28 smithi165 ceph-mon[36302]: pgmap v77: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-28T00:18:29.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:28 smithi119 ceph-mon[31518]: pgmap v77: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-28T00:18:31.044 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:30 smithi165 ceph-mon[36302]: pgmap v78: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-28T00:18:31.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:30 smithi119 ceph-mon[31518]: pgmap v78: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-28T00:18:31.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:30 smithi132 ceph-mon[35941]: pgmap v78: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-28T00:18:32.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:32 smithi165 ceph-mon[36302]: pgmap v79: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-28T00:18:33.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:32 smithi119 ceph-mon[31518]: pgmap v79: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-28T00:18:33.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:32 smithi132 ceph-mon[35941]: pgmap v79: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-28T00:18:33.995 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:33 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-28T00:18:33.995 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:33 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:18:34.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:33 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-28T00:18:34.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:33 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:18:34.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:33 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-28T00:18:34.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:33 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:18:34.978 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:34 smithi165 ceph-mon[36302]: Deploying daemon osd.2 on smithi165 2024-03-28T00:18:34.978 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:34 smithi165 ceph-mon[36302]: pgmap v80: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-28T00:18:35.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:34 smithi119 ceph-mon[31518]: Deploying daemon osd.2 on smithi165 2024-03-28T00:18:35.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:34 smithi119 ceph-mon[31518]: pgmap v80: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-28T00:18:35.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:34 smithi132 ceph-mon[35941]: Deploying daemon osd.2 on smithi165 2024-03-28T00:18:35.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:34 smithi132 ceph-mon[35941]: pgmap v80: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-28T00:18:36.927 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:36 smithi165 ceph-mon[36302]: pgmap v81: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-28T00:18:37.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:36 smithi119 ceph-mon[31518]: pgmap v81: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-28T00:18:37.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:36 smithi132 ceph-mon[35941]: pgmap v81: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-28T00:18:38.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:38 smithi132 ceph-mon[35941]: pgmap v82: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-28T00:18:38.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:38 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:18:38.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:38 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:18:38.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:38 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:18:39.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:38 smithi165 ceph-mon[36302]: pgmap v82: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-28T00:18:39.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:38 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:18:39.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:38 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:18:39.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:38 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:18:39.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:38 smithi119 ceph-mon[31518]: pgmap v82: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-28T00:18:39.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:38 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:18:39.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:38 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:18:39.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:38 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:18:41.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:40 smithi165 ceph-mon[36302]: pgmap v83: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-28T00:18:41.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:40 smithi119 ceph-mon[31518]: pgmap v83: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-28T00:18:41.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:40 smithi132 ceph-mon[35941]: pgmap v83: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-28T00:18:41.857 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:18:41.851+0000 7f68a8d99700 1 -- 172.21.15.165:0/1071179928 <== mgr.14152 v2:172.21.15.119:6800/1401937096 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f68ac0625d0 con 0x7f68980a2800 2024-03-28T00:18:41.857 INFO:teuthology.orchestra.run.smithi165.stdout:Created osd(s) 2 on host 'smithi165' 2024-03-28T00:18:41.859 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:18:41.853+0000 7f68b1127700 1 -- 172.21.15.165:0/1071179928 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f68980a2800 msgr2=0x7f68980a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:18:41.859 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:18:41.853+0000 7f68b1127700 1 --2- 172.21.15.165:0/1071179928 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f68980a2800 0x7f68980a4cc0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f689c005cb0 tx=0x7f689c005c20 comp rx=0 tx=0).stop 2024-03-28T00:18:41.859 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:18:41.853+0000 7f68b1127700 1 -- 172.21.15.165:0/1071179928 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f68ac105570 msgr2=0x7f68ac1989c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:18:41.859 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:18:41.853+0000 7f68b1127700 1 --2- 172.21.15.165:0/1071179928 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f68ac105570 0x7f68ac1989c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f689400c9a0 tx=0x7f689400ccb0 comp rx=0 tx=0).stop 2024-03-28T00:18:41.859 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:18:41.853+0000 7f68b1127700 1 -- 172.21.15.165:0/1071179928 shutdown_connections 2024-03-28T00:18:41.859 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:18:41.853+0000 7f68b1127700 1 --2- 172.21.15.165:0/1071179928 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f68980a2800 0x7f68980a4cc0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:18:41.860 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:18:41.854+0000 7f68b1127700 1 --2- 172.21.15.165:0/1071179928 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f68ac107ea0 0x7f68ac19f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:18:41.860 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:18:41.854+0000 7f68b1127700 1 --2- 172.21.15.165:0/1071179928 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f68ac102c40 0x7f68ac198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:18:41.860 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:18:41.854+0000 7f68b1127700 1 --2- 172.21.15.165:0/1071179928 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f68ac105570 0x7f68ac1989c0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:18:41.860 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:18:41.854+0000 7f68b1127700 1 -- 172.21.15.165:0/1071179928 >> 172.21.15.165:0/1071179928 conn(0x7f68ac0fc690 msgr2=0x7f68ac103580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:18:41.860 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:18:41.854+0000 7f68b1127700 1 -- 172.21.15.165:0/1071179928 shutdown_connections 2024-03-28T00:18:41.860 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:18:41.854+0000 7f68b1127700 1 -- 172.21.15.165:0/1071179928 wait complete. 2024-03-28T00:18:42.496 DEBUG:teuthology.orchestra.run.smithi165:osd.2> sudo journalctl -f -n 0 -u ceph-10f73b92-ec98-11ee-95d0-87774f69a715@osd.2.service 2024-03-28T00:18:42.498 INFO:tasks.cephadm:Deploying osd.3 on smithi119 with /dev/vg_nvme/lv_3... 2024-03-28T00:18:42.498 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 10f73b92-ec98-11ee-95d0-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-28T00:18:42.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:42 smithi119 ceph-mon[31518]: pgmap v84: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-28T00:18:42.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:42 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:18:42.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:42 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:18:42.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:42 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:18:42.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:42 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:18:42.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:42 smithi119 ceph-mon[31518]: from='osd.2 [v2:172.21.15.165:6800/2361434832,v1:172.21.15.165:6801/2361434832]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-28T00:18:42.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:42 smithi119 ceph-mon[31518]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-28T00:18:42.935 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/mon.a/config 2024-03-28T00:18:43.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:42 smithi165 ceph-mon[36302]: pgmap v84: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-28T00:18:43.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:42 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:18:43.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:42 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:18:43.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:42 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:18:43.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:42 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:18:43.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:42 smithi165 ceph-mon[36302]: from='osd.2 [v2:172.21.15.165:6800/2361434832,v1:172.21.15.165:6801/2361434832]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-28T00:18:43.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:42 smithi165 ceph-mon[36302]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-28T00:18:43.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:42 smithi132 ceph-mon[35941]: pgmap v84: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-28T00:18:43.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:42 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:18:43.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:42 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:18:43.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:42 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:18:43.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:42 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:18:43.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:42 smithi132 ceph-mon[35941]: from='osd.2 [v2:172.21.15.165:6800/2361434832,v1:172.21.15.165:6801/2361434832]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-28T00:18:43.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:42 smithi132 ceph-mon[35941]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-28T00:18:44.257 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:43 smithi119 ceph-mon[31518]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-28T00:18:44.257 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:43 smithi119 ceph-mon[31518]: from='osd.2 [v2:172.21.15.165:6800/2361434832,v1:172.21.15.165:6801/2361434832]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]: dispatch 2024-03-28T00:18:44.257 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:43 smithi119 ceph-mon[31518]: osdmap e16: 3 total, 2 up, 3 in 2024-03-28T00:18:44.257 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:43 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-28T00:18:44.257 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:43 smithi119 ceph-mon[31518]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]: dispatch 2024-03-28T00:18:44.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:43 smithi165 ceph-mon[36302]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-28T00:18:44.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:43 smithi165 ceph-mon[36302]: from='osd.2 [v2:172.21.15.165:6800/2361434832,v1:172.21.15.165:6801/2361434832]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]: dispatch 2024-03-28T00:18:44.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:43 smithi165 ceph-mon[36302]: osdmap e16: 3 total, 2 up, 3 in 2024-03-28T00:18:44.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:43 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-28T00:18:44.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:43 smithi165 ceph-mon[36302]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]: dispatch 2024-03-28T00:18:44.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:43 smithi132 ceph-mon[35941]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-28T00:18:44.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:43 smithi132 ceph-mon[35941]: from='osd.2 [v2:172.21.15.165:6800/2361434832,v1:172.21.15.165:6801/2361434832]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]: dispatch 2024-03-28T00:18:44.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:43 smithi132 ceph-mon[35941]: osdmap e16: 3 total, 2 up, 3 in 2024-03-28T00:18:44.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:43 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-28T00:18:44.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:43 smithi132 ceph-mon[35941]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]: dispatch 2024-03-28T00:18:44.848 INFO:journalctl@ceph.osd.2.smithi165.stdout:Mar 28 00:18:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-2[41460]: 2024-03-28T00:18:44.421+0000 7fe363da8700 -1 osd.2 0 waiting for initial osdmap 2024-03-28T00:18:44.848 INFO:journalctl@ceph.osd.2.smithi165.stdout:Mar 28 00:18:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-2[41460]: 2024-03-28T00:18:44.424+0000 7fe35b392700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-28T00:18:45.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:44 smithi165 ceph-mon[36302]: pgmap v86: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-28T00:18:45.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:44 smithi165 ceph-mon[36302]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]': finished 2024-03-28T00:18:45.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:44 smithi165 ceph-mon[36302]: osdmap e17: 3 total, 2 up, 3 in 2024-03-28T00:18:45.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:44 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-28T00:18:45.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:44 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-28T00:18:45.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:44 smithi165 ceph-mon[36302]: from='osd.2 ' entity='osd.2' 2024-03-28T00:18:45.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:44 smithi119 ceph-mon[31518]: pgmap v86: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-28T00:18:45.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:44 smithi119 ceph-mon[31518]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]': finished 2024-03-28T00:18:45.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:44 smithi119 ceph-mon[31518]: osdmap e17: 3 total, 2 up, 3 in 2024-03-28T00:18:45.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:44 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-28T00:18:45.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:44 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-28T00:18:45.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:44 smithi119 ceph-mon[31518]: from='osd.2 ' entity='osd.2' 2024-03-28T00:18:45.452 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:44 smithi132 ceph-mon[35941]: pgmap v86: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-28T00:18:45.452 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:44 smithi132 ceph-mon[35941]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]': finished 2024-03-28T00:18:45.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:44 smithi132 ceph-mon[35941]: osdmap e17: 3 total, 2 up, 3 in 2024-03-28T00:18:45.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:44 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-28T00:18:45.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:44 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-28T00:18:45.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:44 smithi132 ceph-mon[35941]: from='osd.2 ' entity='osd.2' 2024-03-28T00:18:45.671 INFO:teuthology.orchestra.run.smithi119.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-28T00:18:45.671 INFO:teuthology.orchestra.run.smithi119.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-28T00:18:45.671 INFO:teuthology.orchestra.run.smithi119.stderr: stderr: 10+0 records in 2024-03-28T00:18:45.671 INFO:teuthology.orchestra.run.smithi119.stderr:10+0 records out 2024-03-28T00:18:45.671 INFO:teuthology.orchestra.run.smithi119.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0352738 s, 297 MB/s 2024-03-28T00:18:45.671 INFO:teuthology.orchestra.run.smithi119.stderr:--> Zapping successful for: 2024-03-28T00:18:46.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:46 smithi119 ceph-mon[31518]: purged_snaps scrub starts 2024-03-28T00:18:46.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:46 smithi119 ceph-mon[31518]: purged_snaps scrub ok 2024-03-28T00:18:46.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:46 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-28T00:18:46.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:46 smithi119 ceph-mon[31518]: osd.2 [v2:172.21.15.165:6800/2361434832,v1:172.21.15.165:6801/2361434832] boot 2024-03-28T00:18:46.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:46 smithi119 ceph-mon[31518]: osdmap e18: 3 total, 3 up, 3 in 2024-03-28T00:18:46.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:46 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-28T00:18:46.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:46 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:18:46.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:46 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:18:46.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:46 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:18:46.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:46 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:18:46.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:46 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:18:46.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:46 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:18:46.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:46 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:18:46.512 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 10f73b92-ec98-11ee-95d0-87774f69a715 -- ceph orch daemon add osd smithi119:vg_nvme/lv_3 2024-03-28T00:18:46.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:46 smithi165 ceph-mon[36302]: purged_snaps scrub starts 2024-03-28T00:18:46.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:46 smithi165 ceph-mon[36302]: purged_snaps scrub ok 2024-03-28T00:18:46.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:46 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-28T00:18:46.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:46 smithi165 ceph-mon[36302]: osd.2 [v2:172.21.15.165:6800/2361434832,v1:172.21.15.165:6801/2361434832] boot 2024-03-28T00:18:46.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:46 smithi165 ceph-mon[36302]: osdmap e18: 3 total, 3 up, 3 in 2024-03-28T00:18:46.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:46 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-28T00:18:46.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:46 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:18:46.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:46 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:18:46.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:46 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:18:46.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:46 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:18:46.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:46 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:18:46.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:46 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:18:46.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:46 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:18:46.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:46 smithi132 ceph-mon[35941]: purged_snaps scrub starts 2024-03-28T00:18:46.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:46 smithi132 ceph-mon[35941]: purged_snaps scrub ok 2024-03-28T00:18:46.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:46 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-28T00:18:46.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:46 smithi132 ceph-mon[35941]: osd.2 [v2:172.21.15.165:6800/2361434832,v1:172.21.15.165:6801/2361434832] boot 2024-03-28T00:18:46.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:46 smithi132 ceph-mon[35941]: osdmap e18: 3 total, 3 up, 3 in 2024-03-28T00:18:46.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:46 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-28T00:18:46.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:46 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:18:46.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:46 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:18:46.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:46 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:18:46.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:46 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:18:46.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:46 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:18:46.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:46 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:18:46.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:46 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:18:46.944 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/mon.a/config 2024-03-28T00:18:47.361 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:47 smithi119 ceph-mon[31518]: Detected new or changed devices on smithi165 2024-03-28T00:18:47.361 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:47 smithi119 ceph-mon[31518]: Adjusting osd_memory_target on smithi165 to 17080M 2024-03-28T00:18:47.361 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:47 smithi119 ceph-mon[31518]: pgmap v89: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-28T00:18:47.361 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:47 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-28T00:18:47.361 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:47 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:18:47.361 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:47 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:18:47.362 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:47 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:18:47.362 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:47 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:18:47.362 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:47 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-28T00:18:47.362 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:47 smithi119 ceph-mon[31518]: osdmap e19: 3 total, 3 up, 3 in 2024-03-28T00:18:47.362 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:47 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-28T00:18:47.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:47 smithi165 ceph-mon[36302]: Detected new or changed devices on smithi165 2024-03-28T00:18:47.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:47 smithi165 ceph-mon[36302]: Adjusting osd_memory_target on smithi165 to 17080M 2024-03-28T00:18:47.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:47 smithi165 ceph-mon[36302]: pgmap v89: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-28T00:18:47.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:47 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-28T00:18:47.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:47 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:18:47.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:47 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:18:47.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:47 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:18:47.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:47 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:18:47.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:47 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-28T00:18:47.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:47 smithi165 ceph-mon[36302]: osdmap e19: 3 total, 3 up, 3 in 2024-03-28T00:18:47.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:47 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-28T00:18:47.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:47 smithi132 ceph-mon[35941]: Detected new or changed devices on smithi165 2024-03-28T00:18:47.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:47 smithi132 ceph-mon[35941]: Adjusting osd_memory_target on smithi165 to 17080M 2024-03-28T00:18:47.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:47 smithi132 ceph-mon[35941]: pgmap v89: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-28T00:18:47.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:47 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-28T00:18:47.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:47 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:18:47.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:47 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:18:47.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:47 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:18:47.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:47 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:18:47.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:47 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-28T00:18:47.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:47 smithi132 ceph-mon[35941]: osdmap e19: 3 total, 3 up, 3 in 2024-03-28T00:18:47.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:47 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-28T00:18:48.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:48 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-28T00:18:48.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:48 smithi165 ceph-mon[36302]: osdmap e20: 3 total, 3 up, 3 in 2024-03-28T00:18:48.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:48 smithi165 ceph-mon[36302]: pgmap v92: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-28T00:18:48.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:48 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-28T00:18:48.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:48 smithi119 ceph-mon[31518]: osdmap e20: 3 total, 3 up, 3 in 2024-03-28T00:18:48.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:48 smithi119 ceph-mon[31518]: pgmap v92: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-28T00:18:48.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:48 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-28T00:18:48.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:48 smithi132 ceph-mon[35941]: osdmap e20: 3 total, 3 up, 3 in 2024-03-28T00:18:48.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:48 smithi132 ceph-mon[35941]: pgmap v92: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-28T00:18:49.111 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:18:49.105+0000 7f1f61a00700 1 -- 172.21.15.119:0/2618564519 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1f5c107f50 msgr2=0x7f1f5c10a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:18:49.112 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:18:49.105+0000 7f1f61a00700 1 --2- 172.21.15.119:0/2618564519 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1f5c107f50 0x7f1f5c10a340 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f1f5400b0a0 tx=0x7f1f5400b3b0 comp rx=0 tx=0).stop 2024-03-28T00:18:49.112 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:18:49.105+0000 7f1f61a00700 1 -- 172.21.15.119:0/2618564519 shutdown_connections 2024-03-28T00:18:49.112 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:18:49.105+0000 7f1f61a00700 1 --2- 172.21.15.119:0/2618564519 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1f5c107f50 0x7f1f5c10a340 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:18:49.112 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:18:49.105+0000 7f1f61a00700 1 --2- 172.21.15.119:0/2618564519 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1f5c102cf0 0x7f1f5c1050e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:18:49.112 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:18:49.105+0000 7f1f61a00700 1 --2- 172.21.15.119:0/2618564519 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1f5c105620 0x7f1f5c107a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:18:49.112 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:18:49.105+0000 7f1f61a00700 1 -- 172.21.15.119:0/2618564519 >> 172.21.15.119:0/2618564519 conn(0x7f1f5c0fc660 msgr2=0x7f1f5c0feaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:18:49.112 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:18:49.105+0000 7f1f61a00700 1 -- 172.21.15.119:0/2618564519 shutdown_connections 2024-03-28T00:18:49.112 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:18:49.105+0000 7f1f61a00700 1 -- 172.21.15.119:0/2618564519 wait complete. 2024-03-28T00:18:49.114 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:18:49.106+0000 7f1f61a00700 1 Processor -- start 2024-03-28T00:18:49.114 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:18:49.106+0000 7f1f61a00700 1 -- start start 2024-03-28T00:18:49.114 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:18:49.107+0000 7f1f61a00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1f5c102cf0 0x7f1f5c19a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:18:49.114 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:18:49.107+0000 7f1f61a00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1f5c105620 0x7f1f5c19ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:18:49.114 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:18:49.107+0000 7f1f61a00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1f5c107f50 0x7f1f5c1a1be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:18:49.114 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:18:49.107+0000 7f1f61a00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f5c10dda0 con 0x7f1f5c107f50 2024-03-28T00:18:49.114 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:18:49.107+0000 7f1f61a00700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f5c10dc20 con 0x7f1f5c102cf0 2024-03-28T00:18:49.115 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:18:49.107+0000 7f1f61a00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f5c10df20 con 0x7f1f5c105620 2024-03-28T00:18:49.115 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:18:49.107+0000 7f1f5b400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1f5c107f50 0x7f1f5c1a1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:18:49.115 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:18:49.107+0000 7f1f5a000700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1f5c105620 0x7f1f5c19ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:18:49.115 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:18:49.107+0000 7f1f5b400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1f5c107f50 0x7f1f5c1a1be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:60978/0 (socket says 172.21.15.119:60978) 2024-03-28T00:18:49.115 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:18:49.107+0000 7f1f5b400700 1 -- 172.21.15.119:0/1623108228 learned_addr learned my addr 172.21.15.119:0/1623108228 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:18:49.115 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:18:49.107+0000 7f1f5aa00700 1 --2- 172.21.15.119:0/1623108228 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1f5c102cf0 0x7f1f5c19a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:18:49.115 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:18:49.107+0000 7f1f5a000700 1 -- 172.21.15.119:0/1623108228 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1f5c102cf0 msgr2=0x7f1f5c19a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:18:49.115 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:18:49.107+0000 7f1f5a000700 1 --2- 172.21.15.119:0/1623108228 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1f5c102cf0 0x7f1f5c19a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:18:49.116 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:18:49.107+0000 7f1f5a000700 1 -- 172.21.15.119:0/1623108228 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1f5c107f50 msgr2=0x7f1f5c1a1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:18:49.116 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:18:49.107+0000 7f1f5a000700 1 --2- 172.21.15.119:0/1623108228 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1f5c107f50 0x7f1f5c1a1be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:18:49.116 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:18:49.107+0000 7f1f5a000700 1 -- 172.21.15.119:0/1623108228 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f5c1a2280 con 0x7f1f5c105620 2024-03-28T00:18:49.116 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:18:49.108+0000 7f1f5a000700 1 --2- 172.21.15.119:0/1623108228 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1f5c105620 0x7f1f5c19ab60 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f1f4800ea20 tx=0x7f1f4800ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:18:49.116 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:18:49.108+0000 7f1f53400700 1 -- 172.21.15.119:0/1623108228 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f4800cbd0 con 0x7f1f5c105620 2024-03-28T00:18:49.116 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:18:49.108+0000 7f1f53400700 1 -- 172.21.15.119:0/1623108228 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1f48004500 con 0x7f1f5c105620 2024-03-28T00:18:49.116 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:18:49.108+0000 7f1f61a00700 1 -- 172.21.15.119:0/1623108228 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f5c1a2570 con 0x7f1f5c105620 2024-03-28T00:18:49.116 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:18:49.108+0000 7f1f53400700 1 -- 172.21.15.119:0/1623108228 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f48005020 con 0x7f1f5c105620 2024-03-28T00:18:49.116 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:18:49.108+0000 7f1f61a00700 1 -- 172.21.15.119:0/1623108228 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f5c1a2ab0 con 0x7f1f5c105620 2024-03-28T00:18:49.117 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:18:49.109+0000 7f1f61a00700 1 -- 172.21.15.119:0/1623108228 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f5c068e50 con 0x7f1f5c105620 2024-03-28T00:18:49.117 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:18:49.111+0000 7f1f53400700 1 -- 172.21.15.119:0/1623108228 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 15) v1 ==== 137235+0+0 (secure 0 0 0) 0x7f1f48005200 con 0x7f1f5c105620 2024-03-28T00:18:49.118 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:18:49.112+0000 7f1f53400700 1 --2- 172.21.15.119:0/1623108228 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f1f400a2880 0x7f1f400a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:18:49.118 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:18:49.112+0000 7f1f53400700 1 -- 172.21.15.119:0/1623108228 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 3237+0+0 (secure 0 0 0) 0x7f1f480cd440 con 0x7f1f5c105620 2024-03-28T00:18:49.118 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:18:49.112+0000 7f1f5aa00700 1 --2- 172.21.15.119:0/1623108228 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f1f400a2880 0x7f1f400a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:18:49.119 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:18:49.113+0000 7f1f5aa00700 1 --2- 172.21.15.119:0/1623108228 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f1f400a2880 0x7f1f400a4d40 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f1f4c009680 tx=0x7f1f4c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:18:49.122 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:18:49.116+0000 7f1f53400700 1 -- 172.21.15.119:0/1623108228 <== mon.1 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f1f4809b8f0 con 0x7f1f5c105620 2024-03-28T00:18:49.203 INFO:journalctl@ceph.osd.0.smithi119.stdout:Mar 28 00:18:49 smithi119 sudo[48174]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-28T00:18:49.203 INFO:journalctl@ceph.osd.0.smithi119.stdout:Mar 28 00:18:49 smithi119 sudo[48174]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-28T00:18:49.203 INFO:journalctl@ceph.osd.0.smithi119.stdout:Mar 28 00:18:49 smithi119 sudo[48174]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-28T00:18:49.203 INFO:journalctl@ceph.osd.0.smithi119.stdout:Mar 28 00:18:49 smithi119 sudo[48174]: pam_unix(sudo:session): session closed for user root 2024-03-28T00:18:49.355 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:18:49.349+0000 7f1f61a00700 1 -- 172.21.15.119:0/1623108228 --> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi119:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f1f5c0625d0 con 0x7f1f400a2880 2024-03-28T00:18:49.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:49 smithi165 ceph-mon[36302]: osdmap e21: 3 total, 3 up, 3 in 2024-03-28T00:18:49.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:49 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-28T00:18:49.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:49 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-28T00:18:49.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:49 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:18:49.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:49 smithi119 ceph-mon[31518]: osdmap e21: 3 total, 3 up, 3 in 2024-03-28T00:18:49.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:49 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-28T00:18:49.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:49 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-28T00:18:49.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:49 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:18:49.906 INFO:journalctl@ceph.osd.0.smithi119.stdout:Mar 28 00:18:49 smithi119 sudo[48244]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-28T00:18:49.906 INFO:journalctl@ceph.osd.0.smithi119.stdout:Mar 28 00:18:49 smithi119 sudo[48244]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-28T00:18:49.906 INFO:journalctl@ceph.osd.0.smithi119.stdout:Mar 28 00:18:49 smithi119 sudo[48244]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-28T00:18:49.906 INFO:journalctl@ceph.osd.0.smithi119.stdout:Mar 28 00:18:49 smithi119 sudo[48244]: pam_unix(sudo:session): session closed for user root 2024-03-28T00:18:49.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:49 smithi132 ceph-mon[35941]: osdmap e21: 3 total, 3 up, 3 in 2024-03-28T00:18:49.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:49 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-28T00:18:49.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:49 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-28T00:18:49.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:49 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:18:50.290 INFO:journalctl@ceph.osd.1.smithi132.stdout:Mar 28 00:18:50 smithi132 sudo[44685]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-28T00:18:50.290 INFO:journalctl@ceph.osd.1.smithi132.stdout:Mar 28 00:18:50 smithi132 sudo[44685]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-28T00:18:50.290 INFO:journalctl@ceph.osd.1.smithi132.stdout:Mar 28 00:18:50 smithi132 sudo[44685]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-28T00:18:50.290 INFO:journalctl@ceph.osd.1.smithi132.stdout:Mar 28 00:18:50 smithi132 sudo[44685]: pam_unix(sudo:session): session closed for user root 2024-03-28T00:18:50.649 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:18:50.641+0000 7f1f53400700 1 -- 172.21.15.119:0/1623108228 <== mon.1 v2:172.21.15.165:3300/0 7 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f1f48097860 con 0x7f1f5c105620 2024-03-28T00:18:50.702 INFO:journalctl@ceph.osd.1.smithi132.stdout:Mar 28 00:18:50 smithi132 sudo[44688]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-28T00:18:50.703 INFO:journalctl@ceph.osd.1.smithi132.stdout:Mar 28 00:18:50 smithi132 sudo[44688]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-28T00:18:50.703 INFO:journalctl@ceph.osd.1.smithi132.stdout:Mar 28 00:18:50 smithi132 sudo[44688]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-28T00:18:50.703 INFO:journalctl@ceph.osd.1.smithi132.stdout:Mar 28 00:18:50 smithi132 sudo[44688]: pam_unix(sudo:session): session closed for user root 2024-03-28T00:18:50.836 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:50 smithi165 ceph-mon[36302]: from='client.24190 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi119:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T00:18:50.836 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:50 smithi165 ceph-mon[36302]: pgmap v94: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-28T00:18:50.836 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:50 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:18:50.836 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:50 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:18:50.836 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:50 smithi165 ceph-mon[36302]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-03-28T00:18:50.836 INFO:journalctl@ceph.osd.2.smithi165.stdout:Mar 28 00:18:50 smithi165 sudo[45539]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-28T00:18:50.836 INFO:journalctl@ceph.osd.2.smithi165.stdout:Mar 28 00:18:50 smithi165 sudo[45539]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-28T00:18:50.836 INFO:journalctl@ceph.osd.2.smithi165.stdout:Mar 28 00:18:50 smithi165 sudo[45539]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-28T00:18:50.837 INFO:journalctl@ceph.osd.2.smithi165.stdout:Mar 28 00:18:50 smithi165 sudo[45539]: pam_unix(sudo:session): session closed for user root 2024-03-28T00:18:50.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:50 smithi119 ceph-mon[31518]: from='client.24190 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi119:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T00:18:50.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:50 smithi119 ceph-mon[31518]: pgmap v94: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-28T00:18:50.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:50 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:18:50.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:50 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:18:50.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:50 smithi119 ceph-mon[31518]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-03-28T00:18:51.132 INFO:journalctl@ceph.osd.2.smithi165.stdout:Mar 28 00:18:50 smithi165 sudo[45542]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-28T00:18:51.132 INFO:journalctl@ceph.osd.2.smithi165.stdout:Mar 28 00:18:50 smithi165 sudo[45542]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-28T00:18:51.132 INFO:journalctl@ceph.osd.2.smithi165.stdout:Mar 28 00:18:50 smithi165 sudo[45542]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-28T00:18:51.132 INFO:journalctl@ceph.osd.2.smithi165.stdout:Mar 28 00:18:50 smithi165 sudo[45542]: pam_unix(sudo:session): session closed for user root 2024-03-28T00:18:51.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:50 smithi132 ceph-mon[35941]: from='client.24190 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi119:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T00:18:51.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:50 smithi132 ceph-mon[35941]: pgmap v94: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-28T00:18:51.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:50 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:18:51.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:50 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:18:51.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:50 smithi132 ceph-mon[35941]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-03-28T00:18:51.775 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:51 smithi119 sudo[48322]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-28T00:18:51.775 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:51 smithi119 sudo[48322]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-28T00:18:51.775 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:51 smithi119 sudo[48322]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-28T00:18:51.775 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:51 smithi119 sudo[48322]: pam_unix(sudo:session): session closed for user root 2024-03-28T00:18:52.028 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:51 smithi119 sudo[48348]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-28T00:18:52.028 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:51 smithi119 sudo[48348]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-28T00:18:52.028 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:51 smithi119 sudo[48348]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-28T00:18:52.028 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:52 smithi119 sudo[48348]: pam_unix(sudo:session): session closed for user root 2024-03-28T00:18:52.028 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:52 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:18:52.028 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:52 smithi119 ceph-mon[31518]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-28T00:18:52.274 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:52 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:18:52.274 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:52 smithi165 ceph-mon[36302]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-28T00:18:52.274 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:52 smithi165 sudo[45545]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-28T00:18:52.274 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:52 smithi165 sudo[45545]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-28T00:18:52.275 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:52 smithi165 sudo[45545]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-28T00:18:52.275 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:52 smithi165 sudo[45545]: pam_unix(sudo:session): session closed for user root 2024-03-28T00:18:52.452 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:52 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:18:52.452 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:52 smithi132 ceph-mon[35941]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-28T00:18:52.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:52 smithi165 sudo[45548]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-28T00:18:52.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:52 smithi165 sudo[45548]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-28T00:18:52.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:52 smithi165 sudo[45548]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-28T00:18:52.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:52 smithi165 sudo[45548]: pam_unix(sudo:session): session closed for user root 2024-03-28T00:18:53.029 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:52 smithi132 sudo[44691]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-28T00:18:53.029 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:52 smithi132 sudo[44691]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-28T00:18:53.029 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:52 smithi132 sudo[44691]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-28T00:18:53.029 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:52 smithi132 sudo[44691]: pam_unix(sudo:session): session closed for user root 2024-03-28T00:18:53.029 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:52 smithi132 sudo[44694]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-28T00:18:53.029 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:52 smithi132 sudo[44694]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-28T00:18:53.030 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:52 smithi132 sudo[44694]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-28T00:18:53.030 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:53 smithi132 sudo[44694]: pam_unix(sudo:session): session closed for user root 2024-03-28T00:18:53.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:53 smithi165 ceph-mon[36302]: pgmap v95: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-28T00:18:53.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:53 smithi165 ceph-mon[36302]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-28T00:18:53.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:53 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-28T00:18:53.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:53 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-28T00:18:53.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:53 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-28T00:18:53.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:53 smithi165 ceph-mon[36302]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-28T00:18:53.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:53 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-28T00:18:53.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:53 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-28T00:18:53.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:53 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-28T00:18:53.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:53 smithi165 ceph-mon[36302]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-28T00:18:53.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:53 smithi165 ceph-mon[36302]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-28T00:18:53.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:53 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-28T00:18:53.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:53 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-28T00:18:53.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:53 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-28T00:18:53.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:53 smithi165 ceph-mon[36302]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-28T00:18:53.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:53 smithi119 ceph-mon[31518]: pgmap v95: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-28T00:18:53.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:53 smithi119 ceph-mon[31518]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-28T00:18:53.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:53 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-28T00:18:53.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:53 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-28T00:18:53.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:53 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-28T00:18:53.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:53 smithi119 ceph-mon[31518]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-28T00:18:53.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:53 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-28T00:18:53.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:53 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-28T00:18:53.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:53 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-28T00:18:53.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:53 smithi119 ceph-mon[31518]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-28T00:18:53.407 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:53 smithi119 ceph-mon[31518]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-28T00:18:53.407 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:53 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-28T00:18:53.407 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:53 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-28T00:18:53.407 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:53 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-28T00:18:53.407 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:53 smithi119 ceph-mon[31518]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-28T00:18:53.452 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:53 smithi132 ceph-mon[35941]: pgmap v95: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-28T00:18:53.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:53 smithi132 ceph-mon[35941]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-28T00:18:53.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:53 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-28T00:18:53.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:53 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-28T00:18:53.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:53 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-28T00:18:53.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:53 smithi132 ceph-mon[35941]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-28T00:18:53.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:53 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-28T00:18:53.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:53 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-28T00:18:53.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:53 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-28T00:18:53.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:53 smithi132 ceph-mon[35941]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-28T00:18:53.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:53 smithi132 ceph-mon[35941]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-28T00:18:53.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:53 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-28T00:18:53.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:53 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-28T00:18:53.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:53 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-28T00:18:53.454 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:53 smithi132 ceph-mon[35941]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-28T00:18:54.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:54 smithi119 ceph-mon[31518]: from='client.? 172.21.15.119:0/1049889082' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0464c40b-828f-4db4-96d9-3f7a6325e6da"}]: dispatch 2024-03-28T00:18:54.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:54 smithi119 ceph-mon[31518]: from='client.? 172.21.15.119:0/1049889082' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0464c40b-828f-4db4-96d9-3f7a6325e6da"}]': finished 2024-03-28T00:18:54.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:54 smithi119 ceph-mon[31518]: osdmap e22: 4 total, 3 up, 4 in 2024-03-28T00:18:54.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:54 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-28T00:18:54.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:54 smithi119 ceph-mon[31518]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-28T00:18:54.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:54 smithi165 ceph-mon[36302]: from='client.? 172.21.15.119:0/1049889082' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0464c40b-828f-4db4-96d9-3f7a6325e6da"}]: dispatch 2024-03-28T00:18:54.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:54 smithi165 ceph-mon[36302]: from='client.? 172.21.15.119:0/1049889082' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0464c40b-828f-4db4-96d9-3f7a6325e6da"}]': finished 2024-03-28T00:18:54.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:54 smithi165 ceph-mon[36302]: osdmap e22: 4 total, 3 up, 4 in 2024-03-28T00:18:54.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:54 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-28T00:18:54.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:54 smithi165 ceph-mon[36302]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-28T00:18:54.452 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:54 smithi132 ceph-mon[35941]: from='client.? 172.21.15.119:0/1049889082' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0464c40b-828f-4db4-96d9-3f7a6325e6da"}]: dispatch 2024-03-28T00:18:54.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:54 smithi132 ceph-mon[35941]: from='client.? 172.21.15.119:0/1049889082' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0464c40b-828f-4db4-96d9-3f7a6325e6da"}]': finished 2024-03-28T00:18:54.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:54 smithi132 ceph-mon[35941]: osdmap e22: 4 total, 3 up, 4 in 2024-03-28T00:18:54.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:54 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-28T00:18:54.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:54 smithi132 ceph-mon[35941]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-28T00:18:55.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:55 smithi165 ceph-mon[36302]: from='client.? 172.21.15.119:0/2313300865' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-28T00:18:55.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:55 smithi119 ceph-mon[31518]: from='client.? 172.21.15.119:0/2313300865' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-28T00:18:55.452 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:55 smithi132 ceph-mon[35941]: from='client.? 172.21.15.119:0/2313300865' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-28T00:18:56.354 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:56 smithi132 ceph-mon[35941]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-28T00:18:56.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:56 smithi165 ceph-mon[36302]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-28T00:18:56.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:56 smithi119 ceph-mon[31518]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-28T00:18:58.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:18:58 smithi132 ceph-mon[35941]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 2.3 KiB/s rd, 60 KiB/s wr, 5 op/s 2024-03-28T00:18:59.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:18:58 smithi165 ceph-mon[36302]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 2.3 KiB/s rd, 60 KiB/s wr, 5 op/s 2024-03-28T00:18:59.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:18:58 smithi119 ceph-mon[31518]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 2.3 KiB/s rd, 60 KiB/s wr, 5 op/s 2024-03-28T00:19:00.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:19:00 smithi132 ceph-mon[35941]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 1.6 KiB/s rd, 41 KiB/s wr, 3 op/s 2024-03-28T00:19:01.012 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:19:00 smithi119 ceph-mon[31518]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 1.6 KiB/s rd, 41 KiB/s wr, 3 op/s 2024-03-28T00:19:01.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:19:00 smithi165 ceph-mon[36302]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 1.6 KiB/s rd, 41 KiB/s wr, 3 op/s 2024-03-28T00:19:03.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:19:02 smithi165 ceph-mon[36302]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 31 KiB/s wr, 2 op/s 2024-03-28T00:19:03.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:19:02 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-28T00:19:03.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:19:02 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:19:03.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:19:02 smithi119 ceph-mon[31518]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 31 KiB/s wr, 2 op/s 2024-03-28T00:19:03.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:19:02 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-28T00:19:03.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:19:02 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:19:03.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:19:02 smithi132 ceph-mon[35941]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 31 KiB/s wr, 2 op/s 2024-03-28T00:19:03.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:19:02 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-28T00:19:03.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:19:02 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:19:03.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:19:03 smithi119 ceph-mon[31518]: Deploying daemon osd.3 on smithi119 2024-03-28T00:19:04.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:19:03 smithi165 ceph-mon[36302]: Deploying daemon osd.3 on smithi119 2024-03-28T00:19:04.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:19:03 smithi132 ceph-mon[35941]: Deploying daemon osd.3 on smithi119 2024-03-28T00:19:04.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:19:04 smithi119 ceph-mon[31518]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 986 B/s rd, 25 KiB/s wr, 2 op/s 2024-03-28T00:19:05.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:19:04 smithi165 ceph-mon[36302]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 986 B/s rd, 25 KiB/s wr, 2 op/s 2024-03-28T00:19:05.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:19:04 smithi132 ceph-mon[35941]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 986 B/s rd, 25 KiB/s wr, 2 op/s 2024-03-28T00:19:07.102 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:19:06 smithi119 ceph-mon[31518]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-03-28T00:19:07.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:19:06 smithi165 ceph-mon[36302]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-03-28T00:19:07.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:19:06 smithi132 ceph-mon[35941]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-03-28T00:19:08.983 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:19:08 smithi119 ceph-mon[31518]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-03-28T00:19:09.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:19:08 smithi165 ceph-mon[36302]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-03-28T00:19:09.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:19:08 smithi132 ceph-mon[35941]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-03-28T00:19:10.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:19:10 smithi119 ceph-mon[31518]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-28T00:19:11.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:19:10 smithi165 ceph-mon[36302]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-28T00:19:11.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:19:10 smithi132 ceph-mon[35941]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-28T00:19:12.030 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:19:11 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:19:12.030 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:19:11 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:19:12.030 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:19:11 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:19:12.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:19:11 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:19:12.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:19:11 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:19:12.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:19:11 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:19:12.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:19:11 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:19:12.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:19:11 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:19:12.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:19:11 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:19:13.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:19:12 smithi165 ceph-mon[36302]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-28T00:19:13.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:19:12 smithi119 ceph-mon[31518]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-28T00:19:13.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:19:12 smithi132 ceph-mon[35941]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-28T00:19:15.052 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:19:14 smithi119 ceph-mon[31518]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-28T00:19:15.053 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:19:14 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:19:15.053 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:19:14 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:19:15.053 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:19:14 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:19:15.053 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:19:14 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:19:15.053 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:19:14 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:19:15.053 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:19:14 smithi119 ceph-mon[31518]: from='osd.3 [v2:172.21.15.119:6810/3732272210,v1:172.21.15.119:6811/3732272210]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-28T00:19:15.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:19:14 smithi165 ceph-mon[36302]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-28T00:19:15.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:19:14 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:19:15.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:19:14 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:19:15.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:19:14 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:19:15.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:19:14 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:19:15.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:19:14 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:19:15.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:19:14 smithi165 ceph-mon[36302]: from='osd.3 [v2:172.21.15.119:6810/3732272210,v1:172.21.15.119:6811/3732272210]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-28T00:19:15.197 INFO:teuthology.orchestra.run.smithi119.stdout:Created osd(s) 3 on host 'smithi119' 2024-03-28T00:19:15.197 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:19:15.190+0000 7f1f53400700 1 -- 172.21.15.119:0/1623108228 <== mgr.14152 v2:172.21.15.119:6800/1401937096 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f1f5c0625d0 con 0x7f1f400a2880 2024-03-28T00:19:15.200 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:19:15.194+0000 7f1f61a00700 1 -- 172.21.15.119:0/1623108228 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f1f400a2880 msgr2=0x7f1f400a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:19:15.200 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:19:15.194+0000 7f1f61a00700 1 --2- 172.21.15.119:0/1623108228 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f1f400a2880 0x7f1f400a4d40 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f1f4c009680 tx=0x7f1f4c009260 comp rx=0 tx=0).stop 2024-03-28T00:19:15.200 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:19:15.194+0000 7f1f61a00700 1 -- 172.21.15.119:0/1623108228 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1f5c105620 msgr2=0x7f1f5c19ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:19:15.200 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:19:15.194+0000 7f1f61a00700 1 --2- 172.21.15.119:0/1623108228 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1f5c105620 0x7f1f5c19ab60 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f1f4800ea20 tx=0x7f1f4800ed30 comp rx=0 tx=0).stop 2024-03-28T00:19:15.201 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:19:15.194+0000 7f1f61a00700 1 -- 172.21.15.119:0/1623108228 shutdown_connections 2024-03-28T00:19:15.201 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:19:15.194+0000 7f1f61a00700 1 --2- 172.21.15.119:0/1623108228 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f1f400a2880 0x7f1f400a4d40 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:19:15.201 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:19:15.194+0000 7f1f61a00700 1 --2- 172.21.15.119:0/1623108228 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1f5c107f50 0x7f1f5c1a1be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:19:15.201 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:19:15.194+0000 7f1f61a00700 1 --2- 172.21.15.119:0/1623108228 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1f5c102cf0 0x7f1f5c19a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:19:15.201 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:19:15.194+0000 7f1f61a00700 1 --2- 172.21.15.119:0/1623108228 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1f5c105620 0x7f1f5c19ab60 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:19:15.201 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:19:15.194+0000 7f1f61a00700 1 -- 172.21.15.119:0/1623108228 >> 172.21.15.119:0/1623108228 conn(0x7f1f5c0fc660 msgr2=0x7f1f5c102520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:19:15.201 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:19:15.195+0000 7f1f61a00700 1 -- 172.21.15.119:0/1623108228 shutdown_connections 2024-03-28T00:19:15.201 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:19:15.195+0000 7f1f61a00700 1 -- 172.21.15.119:0/1623108228 wait complete. 2024-03-28T00:19:15.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:19:14 smithi132 ceph-mon[35941]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-28T00:19:15.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:19:14 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:19:15.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:19:14 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:19:15.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:19:14 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:19:15.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:19:14 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:19:15.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:19:14 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:19:15.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:19:14 smithi132 ceph-mon[35941]: from='osd.3 [v2:172.21.15.119:6810/3732272210,v1:172.21.15.119:6811/3732272210]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-28T00:19:15.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:19:15 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:19:15.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:19:15 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:19:15.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:19:15 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:19:15.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:19:15 smithi119 ceph-mon[31518]: from='osd.3 [v2:172.21.15.119:6810/3732272210,v1:172.21.15.119:6811/3732272210]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-28T00:19:15.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:19:15 smithi119 ceph-mon[31518]: osdmap e23: 4 total, 3 up, 4 in 2024-03-28T00:19:15.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:19:15 smithi119 ceph-mon[31518]: from='osd.3 [v2:172.21.15.119:6810/3732272210,v1:172.21.15.119:6811/3732272210]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]: dispatch 2024-03-28T00:19:15.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:19:15 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-28T00:19:16.049 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:19:15 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:19:16.049 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:19:15 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:19:16.049 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:19:15 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:19:16.050 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:19:15 smithi165 ceph-mon[36302]: from='osd.3 [v2:172.21.15.119:6810/3732272210,v1:172.21.15.119:6811/3732272210]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-28T00:19:16.050 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:19:15 smithi165 ceph-mon[36302]: osdmap e23: 4 total, 3 up, 4 in 2024-03-28T00:19:16.050 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:19:15 smithi165 ceph-mon[36302]: from='osd.3 [v2:172.21.15.119:6810/3732272210,v1:172.21.15.119:6811/3732272210]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]: dispatch 2024-03-28T00:19:16.050 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:19:15 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-28T00:19:16.056 DEBUG:teuthology.orchestra.run.smithi119:osd.3> sudo journalctl -f -n 0 -u ceph-10f73b92-ec98-11ee-95d0-87774f69a715@osd.3.service 2024-03-28T00:19:16.064 INFO:tasks.cephadm:Deploying osd.4 on smithi132 with /dev/vg_nvme/lv_3... 2024-03-28T00:19:16.064 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 10f73b92-ec98-11ee-95d0-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-28T00:19:16.094 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:19:15 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:19:16.094 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:19:15 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:19:16.094 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:19:15 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:19:16.094 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:19:15 smithi132 ceph-mon[35941]: from='osd.3 [v2:172.21.15.119:6810/3732272210,v1:172.21.15.119:6811/3732272210]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-28T00:19:16.094 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:19:15 smithi132 ceph-mon[35941]: osdmap e23: 4 total, 3 up, 4 in 2024-03-28T00:19:16.094 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:19:15 smithi132 ceph-mon[35941]: from='osd.3 [v2:172.21.15.119:6810/3732272210,v1:172.21.15.119:6811/3732272210]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]: dispatch 2024-03-28T00:19:16.094 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:19:15 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-28T00:19:16.308 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/mon.b/config 2024-03-28T00:19:16.656 INFO:journalctl@ceph.osd.3.smithi119.stdout:Mar 28 00:19:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-3[50436]: 2024-03-28T00:19:16.444+0000 7fc069600700 -1 osd.3 0 waiting for initial osdmap 2024-03-28T00:19:16.656 INFO:journalctl@ceph.osd.3.smithi119.stdout:Mar 28 00:19:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-3[50436]: 2024-03-28T00:19:16.460+0000 7fc064400700 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-28T00:19:16.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:19:16 smithi132 ceph-mon[35941]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-28T00:19:16.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:19:16 smithi132 ceph-mon[35941]: from='osd.3 [v2:172.21.15.119:6810/3732272210,v1:172.21.15.119:6811/3732272210]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]': finished 2024-03-28T00:19:16.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:19:16 smithi132 ceph-mon[35941]: osdmap e24: 4 total, 3 up, 4 in 2024-03-28T00:19:16.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:19:16 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-28T00:19:16.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:19:16 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-28T00:19:17.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:19:16 smithi165 ceph-mon[36302]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-28T00:19:17.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:19:16 smithi165 ceph-mon[36302]: from='osd.3 [v2:172.21.15.119:6810/3732272210,v1:172.21.15.119:6811/3732272210]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]': finished 2024-03-28T00:19:17.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:19:16 smithi165 ceph-mon[36302]: osdmap e24: 4 total, 3 up, 4 in 2024-03-28T00:19:17.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:19:16 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-28T00:19:17.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:19:16 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-28T00:19:17.136 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:19:16 smithi119 ceph-mon[31518]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-28T00:19:17.137 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:19:16 smithi119 ceph-mon[31518]: from='osd.3 [v2:172.21.15.119:6810/3732272210,v1:172.21.15.119:6811/3732272210]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]': finished 2024-03-28T00:19:17.137 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:19:16 smithi119 ceph-mon[31518]: osdmap e24: 4 total, 3 up, 4 in 2024-03-28T00:19:17.137 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:19:16 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-28T00:19:17.137 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:19:16 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-28T00:19:18.255 INFO:teuthology.orchestra.run.smithi132.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-28T00:19:18.255 INFO:teuthology.orchestra.run.smithi132.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-28T00:19:18.255 INFO:teuthology.orchestra.run.smithi132.stderr: stderr: 10+0 records in 2024-03-28T00:19:18.255 INFO:teuthology.orchestra.run.smithi132.stderr:10+0 records out 2024-03-28T00:19:18.256 INFO:teuthology.orchestra.run.smithi132.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0114536 s, 915 MB/s 2024-03-28T00:19:18.256 INFO:teuthology.orchestra.run.smithi132.stderr:--> Zapping successful for: 2024-03-28T00:19:18.560 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:19:18 smithi132 ceph-mon[35941]: purged_snaps scrub starts 2024-03-28T00:19:18.560 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:19:18 smithi132 ceph-mon[35941]: purged_snaps scrub ok 2024-03-28T00:19:18.560 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:19:18 smithi132 ceph-mon[35941]: osd.3 [v2:172.21.15.119:6810/3732272210,v1:172.21.15.119:6811/3732272210] boot 2024-03-28T00:19:18.561 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:19:18 smithi132 ceph-mon[35941]: osdmap e25: 4 total, 4 up, 4 in 2024-03-28T00:19:18.561 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:19:18 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-28T00:19:18.561 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:19:18 smithi132 ceph-mon[35941]: pgmap v112: 1 pgs: 1 peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-28T00:19:18.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:19:18 smithi165 ceph-mon[36302]: purged_snaps scrub starts 2024-03-28T00:19:18.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:19:18 smithi165 ceph-mon[36302]: purged_snaps scrub ok 2024-03-28T00:19:18.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:19:18 smithi165 ceph-mon[36302]: osd.3 [v2:172.21.15.119:6810/3732272210,v1:172.21.15.119:6811/3732272210] boot 2024-03-28T00:19:18.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:19:18 smithi165 ceph-mon[36302]: osdmap e25: 4 total, 4 up, 4 in 2024-03-28T00:19:18.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:19:18 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-28T00:19:18.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:19:18 smithi165 ceph-mon[36302]: pgmap v112: 1 pgs: 1 peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-28T00:19:18.904 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 10f73b92-ec98-11ee-95d0-87774f69a715 -- ceph orch daemon add osd smithi132:vg_nvme/lv_3 2024-03-28T00:19:18.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:19:18 smithi119 ceph-mon[31518]: purged_snaps scrub starts 2024-03-28T00:19:18.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:19:18 smithi119 ceph-mon[31518]: purged_snaps scrub ok 2024-03-28T00:19:18.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:19:18 smithi119 ceph-mon[31518]: osd.3 [v2:172.21.15.119:6810/3732272210,v1:172.21.15.119:6811/3732272210] boot 2024-03-28T00:19:18.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:19:18 smithi119 ceph-mon[31518]: osdmap e25: 4 total, 4 up, 4 in 2024-03-28T00:19:18.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:19:18 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-28T00:19:18.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:19:18 smithi119 ceph-mon[31518]: pgmap v112: 1 pgs: 1 peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-28T00:19:19.188 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/mon.b/config 2024-03-28T00:19:19.562 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:19:19 smithi132 ceph-mon[35941]: osdmap e26: 4 total, 4 up, 4 in 2024-03-28T00:19:19.721 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:19:19 smithi119 ceph-mon[31518]: osdmap e26: 4 total, 4 up, 4 in 2024-03-28T00:19:19.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:19:19 smithi165 ceph-mon[36302]: osdmap e26: 4 total, 4 up, 4 in 2024-03-28T00:19:20.599 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:19:20 smithi132 ceph-mon[35941]: osdmap e27: 4 total, 4 up, 4 in 2024-03-28T00:19:20.600 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:19:20 smithi132 ceph-mon[35941]: pgmap v115: 1 pgs: 1 peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-28T00:19:20.738 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:19:20 smithi165 ceph-mon[36302]: osdmap e27: 4 total, 4 up, 4 in 2024-03-28T00:19:20.739 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:19:20 smithi165 ceph-mon[36302]: pgmap v115: 1 pgs: 1 peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-28T00:19:20.798 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:19:20.792+0000 7f964aa32700 1 -- 172.21.15.132:0/2401928086 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9644101d60 msgr2=0x7f9644109620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:19:20.798 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:19:20.792+0000 7f964aa32700 1 --2- 172.21.15.132:0/2401928086 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9644101d60 0x7f9644109620 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f963c00b0a0 tx=0x7f963c00b3b0 comp rx=0 tx=0).stop 2024-03-28T00:19:20.798 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:19:20.792+0000 7f964aa32700 1 -- 172.21.15.132:0/2401928086 shutdown_connections 2024-03-28T00:19:20.798 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:19:20.792+0000 7f964aa32700 1 --2- 172.21.15.132:0/2401928086 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9644101d60 0x7f9644109620 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:19:20.798 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:19:20.792+0000 7f964aa32700 1 --2- 172.21.15.132:0/2401928086 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9644102680 0x7f9644102aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:19:20.798 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:19:20.792+0000 7f964aa32700 1 --2- 172.21.15.132:0/2401928086 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f96441013c0 0x7f9644101820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:19:20.799 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:19:20.792+0000 7f964aa32700 1 -- 172.21.15.132:0/2401928086 >> 172.21.15.132:0/2401928086 conn(0x7f96440fbe30 msgr2=0x7f96440fe250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:19:20.799 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:19:20.792+0000 7f964aa32700 1 -- 172.21.15.132:0/2401928086 shutdown_connections 2024-03-28T00:19:20.799 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:19:20.792+0000 7f964aa32700 1 -- 172.21.15.132:0/2401928086 wait complete. 2024-03-28T00:19:20.799 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:19:20.793+0000 7f964aa32700 1 Processor -- start 2024-03-28T00:19:20.799 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:19:20.793+0000 7f964aa32700 1 -- start start 2024-03-28T00:19:20.801 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:19:20.793+0000 7f964aa32700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f96441013c0 0x7f9644100e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:19:20.801 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:19:20.793+0000 7f964aa32700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9644101d60 0x7f964419d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:19:20.801 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:19:20.793+0000 7f964aa32700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9644102680 0x7f964419f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:19:20.801 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:19:20.793+0000 7f964aa32700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f964410ce20 con 0x7f96441013c0 2024-03-28T00:19:20.801 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:19:20.793+0000 7f964aa32700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f964410cca0 con 0x7f9644102680 2024-03-28T00:19:20.801 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:19:20.793+0000 7f964aa32700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f964410cfa0 con 0x7f9644101d60 2024-03-28T00:19:20.802 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:19:20.793+0000 7f9648fcf700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9644102680 0x7f964419f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:19:20.802 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:19:20.793+0000 7f9648fcf700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9644102680 0x7f964419f8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:57576/0 (socket says 172.21.15.132:57576) 2024-03-28T00:19:20.802 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:19:20.793+0000 7f9648fcf700 1 -- 172.21.15.132:0/2738033038 learned_addr learned my addr 172.21.15.132:0/2738033038 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-28T00:19:20.802 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:19:20.793+0000 7f9648fcf700 1 -- 172.21.15.132:0/2738033038 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9644101d60 msgr2=0x7f964419d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:19:20.802 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:19:20.793+0000 7f96437fe700 1 --2- 172.21.15.132:0/2738033038 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9644101d60 0x7f964419d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:19:20.802 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:19:20.793+0000 7f9648fcf700 1 --2- 172.21.15.132:0/2738033038 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9644101d60 0x7f964419d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:19:20.802 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:19:20.793+0000 7f9648fcf700 1 -- 172.21.15.132:0/2738033038 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f96441013c0 msgr2=0x7f9644100e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:19:20.802 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:19:20.793+0000 7f9648fcf700 1 --2- 172.21.15.132:0/2738033038 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f96441013c0 0x7f9644100e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:19:20.802 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:19:20.793+0000 7f9648fcf700 1 -- 172.21.15.132:0/2738033038 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96441a0170 con 0x7f9644102680 2024-03-28T00:19:20.802 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:19:20.793+0000 7f96437fe700 1 --2- 172.21.15.132:0/2738033038 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9644101d60 0x7f964419d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T00:19:20.802 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:19:20.793+0000 7f9648fcf700 1 --2- 172.21.15.132:0/2738033038 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9644102680 0x7f964419f8a0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f963c009be0 tx=0x7f963c007770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:19:20.802 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:19:20.794+0000 7f96417fa700 1 -- 172.21.15.132:0/2738033038 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f963c021070 con 0x7f9644102680 2024-03-28T00:19:20.802 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:19:20.794+0000 7f96417fa700 1 -- 172.21.15.132:0/2738033038 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f963c004020 con 0x7f9644102680 2024-03-28T00:19:20.802 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:19:20.794+0000 7f964aa32700 1 -- 172.21.15.132:0/2738033038 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96441a0400 con 0x7f9644102680 2024-03-28T00:19:20.803 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:19:20.794+0000 7f96417fa700 1 -- 172.21.15.132:0/2738033038 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f963c0052a0 con 0x7f9644102680 2024-03-28T00:19:20.803 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:19:20.794+0000 7f964aa32700 1 -- 172.21.15.132:0/2738033038 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96441a0930 con 0x7f9644102680 2024-03-28T00:19:20.803 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:19:20.795+0000 7f964aa32700 1 -- 172.21.15.132:0/2738033038 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9644068e50 con 0x7f9644102680 2024-03-28T00:19:20.803 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:19:20.796+0000 7f96417fa700 1 -- 172.21.15.132:0/2738033038 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f963c0041d0 con 0x7f9644102680 2024-03-28T00:19:20.803 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:19:20.797+0000 7f96417fa700 1 --2- 172.21.15.132:0/2738033038 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f962c0a2890 0x7f962c0a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:19:20.803 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:19:20.797+0000 7f96417fa700 1 -- 172.21.15.132:0/2738033038 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 3669+0+0 (secure 0 0 0) 0x7f963c00f020 con 0x7f9644102680 2024-03-28T00:19:20.804 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:19:20.797+0000 7f9643fff700 1 --2- 172.21.15.132:0/2738033038 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f962c0a2890 0x7f962c0a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:19:20.804 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:19:20.798+0000 7f9643fff700 1 --2- 172.21.15.132:0/2738033038 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f962c0a2890 0x7f962c0a4d50 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f964406a5b0 tx=0x7f9634006d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:19:20.806 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:19:20.801+0000 7f96417fa700 1 -- 172.21.15.132:0/2738033038 <== mon.2 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f963c09c990 con 0x7f9644102680 2024-03-28T00:19:20.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:19:20 smithi119 ceph-mon[31518]: osdmap e27: 4 total, 4 up, 4 in 2024-03-28T00:19:20.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:19:20 smithi119 ceph-mon[31518]: pgmap v115: 1 pgs: 1 peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-28T00:19:21.011 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:19:21.005+0000 7f964aa32700 1 -- 172.21.15.132:0/2738033038 --> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi132:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f96440625d0 con 0x7f962c0a2890 2024-03-28T00:19:21.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:19:21 smithi165 ceph-mon[36302]: osdmap e28: 4 total, 4 up, 4 in 2024-03-28T00:19:21.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:19:21 smithi165 ceph-mon[36302]: Detected new or changed devices on smithi119 2024-03-28T00:19:21.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:19:21 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:19:21.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:19:21 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:19:21.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:19:21 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:19:21.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:19:21 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:19:21.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:19:21 smithi165 ceph-mon[36302]: Adjusting osd_memory_target on smithi119 to 8545M 2024-03-28T00:19:21.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:19:21 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:19:21.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:19:21 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:19:21.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:19:21 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:19:21.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:19:21 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:19:21.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:19:21 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-28T00:19:21.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:19:21 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-28T00:19:21.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:19:21 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:19:21.900 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:19:21 smithi132 ceph-mon[35941]: osdmap e28: 4 total, 4 up, 4 in 2024-03-28T00:19:21.900 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:19:21 smithi132 ceph-mon[35941]: Detected new or changed devices on smithi119 2024-03-28T00:19:21.900 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:19:21 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:19:21.900 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:19:21 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:19:21.901 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:19:21 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:19:21.901 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:19:21 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:19:21.901 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:19:21 smithi132 ceph-mon[35941]: Adjusting osd_memory_target on smithi119 to 8545M 2024-03-28T00:19:21.901 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:19:21 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:19:21.901 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:19:21 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:19:21.901 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:19:21 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:19:21.901 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:19:21 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:19:21.901 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:19:21 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-28T00:19:21.901 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:19:21 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-28T00:19:21.901 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:19:21 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:19:21.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:19:21 smithi119 ceph-mon[31518]: osdmap e28: 4 total, 4 up, 4 in 2024-03-28T00:19:21.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:19:21 smithi119 ceph-mon[31518]: Detected new or changed devices on smithi119 2024-03-28T00:19:21.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:19:21 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:19:21.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:19:21 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:19:21.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:19:21 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:19:21.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:19:21 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:19:21.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:19:21 smithi119 ceph-mon[31518]: Adjusting osd_memory_target on smithi119 to 8545M 2024-03-28T00:19:21.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:19:21 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:19:21.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:19:21 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:19:21.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:19:21 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:19:21.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:19:21 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:19:21.907 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:19:21 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-28T00:19:21.907 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:19:21 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-28T00:19:21.907 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:19:21 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:19:22.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:19:22 smithi165 ceph-mon[36302]: from='client.24221 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi132:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T00:19:22.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:19:22 smithi165 ceph-mon[36302]: osdmap e29: 4 total, 4 up, 4 in 2024-03-28T00:19:22.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:19:22 smithi165 ceph-mon[36302]: pgmap v118: 1 pgs: 1 peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-28T00:19:22.885 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:19:22 smithi132 ceph-mon[35941]: from='client.24221 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi132:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T00:19:22.885 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:19:22 smithi132 ceph-mon[35941]: osdmap e29: 4 total, 4 up, 4 in 2024-03-28T00:19:22.885 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:19:22 smithi132 ceph-mon[35941]: pgmap v118: 1 pgs: 1 peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-28T00:19:22.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:19:22 smithi119 ceph-mon[31518]: from='client.24221 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi132:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T00:19:22.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:19:22 smithi119 ceph-mon[31518]: osdmap e29: 4 total, 4 up, 4 in 2024-03-28T00:19:22.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:19:22 smithi119 ceph-mon[31518]: pgmap v118: 1 pgs: 1 peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-28T00:19:24.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:19:24 smithi132 ceph-mon[35941]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 111 KiB/s, 0 objects/s recovering 2024-03-28T00:19:24.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:19:24 smithi132 ceph-mon[35941]: from='client.? 172.21.15.132:0/4084298769' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8fd70b8e-5128-4173-aa0e-e735dfeb995d"}]: dispatch 2024-03-28T00:19:24.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:19:24 smithi132 ceph-mon[35941]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8fd70b8e-5128-4173-aa0e-e735dfeb995d"}]: dispatch 2024-03-28T00:19:24.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:19:24 smithi132 ceph-mon[35941]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8fd70b8e-5128-4173-aa0e-e735dfeb995d"}]': finished 2024-03-28T00:19:24.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:19:24 smithi132 ceph-mon[35941]: osdmap e30: 5 total, 4 up, 5 in 2024-03-28T00:19:24.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:19:24 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-28T00:19:24.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:19:24 smithi132 ceph-mon[35941]: from='client.? 172.21.15.132:0/3448905584' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-28T00:19:25.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:19:24 smithi165 ceph-mon[36302]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 111 KiB/s, 0 objects/s recovering 2024-03-28T00:19:25.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:19:24 smithi165 ceph-mon[36302]: from='client.? 172.21.15.132:0/4084298769' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8fd70b8e-5128-4173-aa0e-e735dfeb995d"}]: dispatch 2024-03-28T00:19:25.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:19:24 smithi165 ceph-mon[36302]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8fd70b8e-5128-4173-aa0e-e735dfeb995d"}]: dispatch 2024-03-28T00:19:25.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:19:24 smithi165 ceph-mon[36302]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8fd70b8e-5128-4173-aa0e-e735dfeb995d"}]': finished 2024-03-28T00:19:25.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:19:24 smithi165 ceph-mon[36302]: osdmap e30: 5 total, 4 up, 5 in 2024-03-28T00:19:25.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:19:24 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-28T00:19:25.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:19:24 smithi165 ceph-mon[36302]: from='client.? 172.21.15.132:0/3448905584' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-28T00:19:25.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:19:24 smithi119 ceph-mon[31518]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 111 KiB/s, 0 objects/s recovering 2024-03-28T00:19:25.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:19:24 smithi119 ceph-mon[31518]: from='client.? 172.21.15.132:0/4084298769' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8fd70b8e-5128-4173-aa0e-e735dfeb995d"}]: dispatch 2024-03-28T00:19:25.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:19:24 smithi119 ceph-mon[31518]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8fd70b8e-5128-4173-aa0e-e735dfeb995d"}]: dispatch 2024-03-28T00:19:25.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:19:24 smithi119 ceph-mon[31518]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8fd70b8e-5128-4173-aa0e-e735dfeb995d"}]': finished 2024-03-28T00:19:25.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:19:24 smithi119 ceph-mon[31518]: osdmap e30: 5 total, 4 up, 5 in 2024-03-28T00:19:25.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:19:24 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-28T00:19:25.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:19:24 smithi119 ceph-mon[31518]: from='client.? 172.21.15.132:0/3448905584' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-28T00:19:27.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:19:26 smithi165 ceph-mon[36302]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-28T00:19:27.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:19:26 smithi119 ceph-mon[31518]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-28T00:19:27.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:19:26 smithi132 ceph-mon[35941]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-28T00:19:29.042 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:19:28 smithi132 ceph-mon[35941]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-03-28T00:19:29.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:19:28 smithi165 ceph-mon[36302]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-03-28T00:19:29.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:19:28 smithi119 ceph-mon[31518]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-03-28T00:19:30.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:19:30 smithi132 ceph-mon[35941]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-03-28T00:19:31.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:19:30 smithi165 ceph-mon[36302]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-03-28T00:19:31.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:19:30 smithi119 ceph-mon[31518]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-03-28T00:19:32.052 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:19:31 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-28T00:19:32.053 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:19:31 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:19:32.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:19:31 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-28T00:19:32.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:19:31 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:19:32.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:19:31 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-28T00:19:32.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:19:31 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:19:33.056 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:19:32 smithi132 ceph-mon[35941]: Deploying daemon osd.4 on smithi132 2024-03-28T00:19:33.056 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:19:32 smithi132 ceph-mon[35941]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-28T00:19:33.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:19:32 smithi165 ceph-mon[36302]: Deploying daemon osd.4 on smithi132 2024-03-28T00:19:33.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:19:32 smithi165 ceph-mon[36302]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-28T00:19:33.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:19:32 smithi119 ceph-mon[31518]: Deploying daemon osd.4 on smithi132 2024-03-28T00:19:33.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:19:32 smithi119 ceph-mon[31518]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-28T00:19:35.001 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:19:34 smithi132 ceph-mon[35941]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-28T00:19:35.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:19:34 smithi165 ceph-mon[36302]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-28T00:19:35.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:19:34 smithi119 ceph-mon[31518]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-28T00:19:37.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:19:36 smithi165 ceph-mon[36302]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-28T00:19:37.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:19:36 smithi119 ceph-mon[31518]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-28T00:19:37.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:19:36 smithi132 ceph-mon[35941]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-28T00:19:38.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:19:37 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:19:38.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:19:37 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:19:38.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:19:37 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:19:38.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:19:37 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:19:38.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:19:37 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:19:38.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:19:37 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:19:38.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:19:37 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:19:38.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:19:37 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:19:38.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:19:37 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:19:39.070 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:19:38 smithi132 ceph-mon[35941]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-28T00:19:39.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:19:38 smithi119 ceph-mon[31518]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-28T00:19:39.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:19:38 smithi165 ceph-mon[36302]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-28T00:19:40.106 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:19:40.101+0000 7f96417fa700 1 -- 172.21.15.132:0/2738033038 <== mgr.14152 v2:172.21.15.119:6800/1401937096 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f96440625d0 con 0x7f962c0a2890 2024-03-28T00:19:40.107 INFO:teuthology.orchestra.run.smithi132.stdout:Created osd(s) 4 on host 'smithi132' 2024-03-28T00:19:40.109 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:19:40.103+0000 7f964aa32700 1 -- 172.21.15.132:0/2738033038 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f962c0a2890 msgr2=0x7f962c0a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:19:40.109 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:19:40.103+0000 7f964aa32700 1 --2- 172.21.15.132:0/2738033038 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f962c0a2890 0x7f962c0a4d50 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f964406a5b0 tx=0x7f9634006d20 comp rx=0 tx=0).stop 2024-03-28T00:19:40.109 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:19:40.103+0000 7f964aa32700 1 -- 172.21.15.132:0/2738033038 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9644102680 msgr2=0x7f964419f8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:19:40.109 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:19:40.103+0000 7f964aa32700 1 --2- 172.21.15.132:0/2738033038 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9644102680 0x7f964419f8a0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f963c009be0 tx=0x7f963c007770 comp rx=0 tx=0).stop 2024-03-28T00:19:40.109 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:19:40.103+0000 7f964aa32700 1 -- 172.21.15.132:0/2738033038 shutdown_connections 2024-03-28T00:19:40.109 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:19:40.103+0000 7f964aa32700 1 --2- 172.21.15.132:0/2738033038 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f962c0a2890 0x7f962c0a4d50 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:19:40.109 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:19:40.103+0000 7f964aa32700 1 --2- 172.21.15.132:0/2738033038 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9644102680 0x7f964419f8a0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:19:40.109 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:19:40.103+0000 7f964aa32700 1 --2- 172.21.15.132:0/2738033038 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f96441013c0 0x7f9644100e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:19:40.109 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:19:40.103+0000 7f964aa32700 1 --2- 172.21.15.132:0/2738033038 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9644101d60 0x7f964419d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:19:40.109 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:19:40.103+0000 7f964aa32700 1 -- 172.21.15.132:0/2738033038 >> 172.21.15.132:0/2738033038 conn(0x7f96440fbe30 msgr2=0x7f96441056d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:19:40.109 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:19:40.103+0000 7f964aa32700 1 -- 172.21.15.132:0/2738033038 shutdown_connections 2024-03-28T00:19:40.109 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:19:40.103+0000 7f964aa32700 1 -- 172.21.15.132:0/2738033038 wait complete. 2024-03-28T00:19:40.785 DEBUG:teuthology.orchestra.run.smithi132:osd.4> sudo journalctl -f -n 0 -u ceph-10f73b92-ec98-11ee-95d0-87774f69a715@osd.4.service 2024-03-28T00:19:40.787 INFO:tasks.cephadm:Deploying osd.5 on smithi165 with /dev/vg_nvme/lv_3... 2024-03-28T00:19:40.788 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 10f73b92-ec98-11ee-95d0-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-28T00:19:41.036 INFO:teuthology.orchestra.run.smithi165.stderr:Inferring config /var/lib/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/mon.c/config 2024-03-28T00:19:41.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:19:40 smithi165 ceph-mon[36302]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-28T00:19:41.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:19:40 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:19:41.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:19:40 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:19:41.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:19:40 smithi165 ceph-mon[36302]: from='osd.4 [v2:172.21.15.132:6808/2803039505,v1:172.21.15.132:6809/2803039505]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-28T00:19:41.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:19:40 smithi165 ceph-mon[36302]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-28T00:19:41.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:19:40 smithi119 ceph-mon[31518]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-28T00:19:41.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:19:40 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:19:41.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:19:40 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:19:41.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:19:40 smithi119 ceph-mon[31518]: from='osd.4 [v2:172.21.15.132:6808/2803039505,v1:172.21.15.132:6809/2803039505]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-28T00:19:41.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:19:40 smithi119 ceph-mon[31518]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-28T00:19:41.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:19:40 smithi132 ceph-mon[35941]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-28T00:19:41.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:19:40 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:19:41.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:19:40 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:19:41.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:19:40 smithi132 ceph-mon[35941]: from='osd.4 [v2:172.21.15.132:6808/2803039505,v1:172.21.15.132:6809/2803039505]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-28T00:19:41.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:19:40 smithi132 ceph-mon[35941]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-28T00:19:42.166 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:19:41 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:19:42.166 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:19:41 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:19:42.166 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:19:41 smithi132 ceph-mon[35941]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-28T00:19:42.166 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:19:41 smithi132 ceph-mon[35941]: from='osd.4 [v2:172.21.15.132:6808/2803039505,v1:172.21.15.132:6809/2803039505]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]: dispatch 2024-03-28T00:19:42.166 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:19:41 smithi132 ceph-mon[35941]: osdmap e31: 5 total, 4 up, 5 in 2024-03-28T00:19:42.166 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:19:41 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-28T00:19:42.166 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:19:41 smithi132 ceph-mon[35941]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]: dispatch 2024-03-28T00:19:42.297 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:19:41 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:19:42.297 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:19:41 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:19:42.297 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:19:41 smithi165 ceph-mon[36302]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-28T00:19:42.297 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:19:41 smithi165 ceph-mon[36302]: from='osd.4 [v2:172.21.15.132:6808/2803039505,v1:172.21.15.132:6809/2803039505]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]: dispatch 2024-03-28T00:19:42.297 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:19:41 smithi165 ceph-mon[36302]: osdmap e31: 5 total, 4 up, 5 in 2024-03-28T00:19:42.297 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:19:41 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-28T00:19:42.297 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:19:41 smithi165 ceph-mon[36302]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]: dispatch 2024-03-28T00:19:42.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:19:41 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:19:42.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:19:41 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:19:42.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:19:41 smithi119 ceph-mon[31518]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-28T00:19:42.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:19:41 smithi119 ceph-mon[31518]: from='osd.4 [v2:172.21.15.132:6808/2803039505,v1:172.21.15.132:6809/2803039505]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]: dispatch 2024-03-28T00:19:42.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:19:41 smithi119 ceph-mon[31518]: osdmap e31: 5 total, 4 up, 5 in 2024-03-28T00:19:42.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:19:41 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-28T00:19:42.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:19:41 smithi119 ceph-mon[31518]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]: dispatch 2024-03-28T00:19:42.789 INFO:journalctl@ceph.osd.4.smithi132.stdout:Mar 28 00:19:42 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-4[47490]: 2024-03-28T00:19:42.498+0000 7f96420b1700 -1 osd.4 0 waiting for initial osdmap 2024-03-28T00:19:42.789 INFO:journalctl@ceph.osd.4.smithi132.stdout:Mar 28 00:19:42 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-4[47490]: 2024-03-28T00:19:42.506+0000 7f9639e9c700 -1 osd.4 32 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-28T00:19:43.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:19:42 smithi119 ceph-mon[31518]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-28T00:19:43.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:19:42 smithi119 ceph-mon[31518]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]': finished 2024-03-28T00:19:43.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:19:42 smithi119 ceph-mon[31518]: osdmap e32: 5 total, 4 up, 5 in 2024-03-28T00:19:43.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:19:42 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-28T00:19:43.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:19:42 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-28T00:19:43.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:19:42 smithi119 ceph-mon[31518]: from='osd.4 ' entity='osd.4' 2024-03-28T00:19:43.180 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:19:42 smithi165 ceph-mon[36302]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-28T00:19:43.181 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:19:42 smithi165 ceph-mon[36302]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]': finished 2024-03-28T00:19:43.181 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:19:42 smithi165 ceph-mon[36302]: osdmap e32: 5 total, 4 up, 5 in 2024-03-28T00:19:43.181 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:19:42 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-28T00:19:43.181 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:19:42 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-28T00:19:43.181 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:19:42 smithi165 ceph-mon[36302]: from='osd.4 ' entity='osd.4' 2024-03-28T00:19:43.181 INFO:teuthology.orchestra.run.smithi165.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-28T00:19:43.181 INFO:teuthology.orchestra.run.smithi165.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-28T00:19:43.181 INFO:teuthology.orchestra.run.smithi165.stderr: stderr: 10+0 records in 2024-03-28T00:19:43.181 INFO:teuthology.orchestra.run.smithi165.stderr:10+0 records out 2024-03-28T00:19:43.182 INFO:teuthology.orchestra.run.smithi165.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113792 s, 921 MB/s 2024-03-28T00:19:43.182 INFO:teuthology.orchestra.run.smithi165.stderr:--> Zapping successful for: 2024-03-28T00:19:43.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:19:42 smithi132 ceph-mon[35941]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-28T00:19:43.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:19:42 smithi132 ceph-mon[35941]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]': finished 2024-03-28T00:19:43.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:19:42 smithi132 ceph-mon[35941]: osdmap e32: 5 total, 4 up, 5 in 2024-03-28T00:19:43.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:19:42 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-28T00:19:43.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:19:42 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-28T00:19:43.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:19:42 smithi132 ceph-mon[35941]: from='osd.4 ' entity='osd.4' 2024-03-28T00:19:43.833 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 10f73b92-ec98-11ee-95d0-87774f69a715 -- ceph orch daemon add osd smithi165:vg_nvme/lv_3 2024-03-28T00:19:44.079 INFO:teuthology.orchestra.run.smithi165.stderr:Inferring config /var/lib/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/mon.c/config 2024-03-28T00:19:44.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:19:43 smithi165 ceph-mon[36302]: purged_snaps scrub starts 2024-03-28T00:19:44.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:19:43 smithi165 ceph-mon[36302]: purged_snaps scrub ok 2024-03-28T00:19:44.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:19:43 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-28T00:19:44.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:19:43 smithi165 ceph-mon[36302]: osd.4 [v2:172.21.15.132:6808/2803039505,v1:172.21.15.132:6809/2803039505] boot 2024-03-28T00:19:44.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:19:43 smithi165 ceph-mon[36302]: osdmap e33: 5 total, 5 up, 5 in 2024-03-28T00:19:44.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:19:43 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-28T00:19:44.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:19:43 smithi132 ceph-mon[35941]: purged_snaps scrub starts 2024-03-28T00:19:44.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:19:43 smithi132 ceph-mon[35941]: purged_snaps scrub ok 2024-03-28T00:19:44.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:19:43 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-28T00:19:44.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:19:43 smithi132 ceph-mon[35941]: osd.4 [v2:172.21.15.132:6808/2803039505,v1:172.21.15.132:6809/2803039505] boot 2024-03-28T00:19:44.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:19:43 smithi132 ceph-mon[35941]: osdmap e33: 5 total, 5 up, 5 in 2024-03-28T00:19:44.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:19:43 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-28T00:19:44.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:19:43 smithi119 ceph-mon[31518]: purged_snaps scrub starts 2024-03-28T00:19:44.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:19:43 smithi119 ceph-mon[31518]: purged_snaps scrub ok 2024-03-28T00:19:44.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:19:43 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-28T00:19:44.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:19:43 smithi119 ceph-mon[31518]: osd.4 [v2:172.21.15.132:6808/2803039505,v1:172.21.15.132:6809/2803039505] boot 2024-03-28T00:19:44.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:19:43 smithi119 ceph-mon[31518]: osdmap e33: 5 total, 5 up, 5 in 2024-03-28T00:19:44.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:19:43 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-28T00:19:45.243 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:19:44 smithi165 ceph-mon[36302]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-28T00:19:45.244 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:19:44 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:19:45.244 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:19:44 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:19:45.244 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:19:44 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:19:45.244 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:19:44 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:19:45.244 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:19:44 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:19:45.244 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:19:44 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:19:45.244 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:19:44 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:19:45.244 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:19:44 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:19:45.244 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:19:44 smithi165 ceph-mon[36302]: osdmap e34: 5 total, 5 up, 5 in 2024-03-28T00:19:45.244 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:19:44 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:19:45.244 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:19:44 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:19:45.244 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:19:44 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:19:45.245 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:19:44 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:19:45.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:19:44 smithi119 ceph-mon[31518]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-28T00:19:45.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:19:44 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:19:45.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:19:44 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:19:45.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:19:44 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:19:45.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:19:44 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:19:45.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:19:44 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:19:45.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:19:44 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:19:45.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:19:44 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:19:45.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:19:44 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:19:45.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:19:44 smithi119 ceph-mon[31518]: osdmap e34: 5 total, 5 up, 5 in 2024-03-28T00:19:45.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:19:44 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:19:45.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:19:44 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:19:45.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:19:44 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:19:45.407 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:19:44 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:19:45.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:19:44 smithi132 ceph-mon[35941]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-28T00:19:45.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:19:44 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:19:45.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:19:44 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:19:45.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:19:44 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:19:45.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:19:44 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:19:45.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:19:44 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:19:45.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:19:44 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:19:45.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:19:44 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:19:45.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:19:44 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:19:45.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:19:44 smithi132 ceph-mon[35941]: osdmap e34: 5 total, 5 up, 5 in 2024-03-28T00:19:45.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:19:44 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:19:45.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:19:44 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:19:45.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:19:44 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:19:45.454 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:19:44 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:19:45.733 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:19:45.727+0000 7f73d4210700 1 -- 172.21.15.165:0/1485329525 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f73cc1021e0 msgr2=0x7f73cc102600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:19:45.733 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:19:45.727+0000 7f73d4210700 1 --2- 172.21.15.165:0/1485329525 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f73cc1021e0 0x7f73cc102600 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f73bc009990 tx=0x7f73bc009ca0 comp rx=0 tx=0).stop 2024-03-28T00:19:45.733 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:19:45.727+0000 7f73d4210700 1 -- 172.21.15.165:0/1485329525 shutdown_connections 2024-03-28T00:19:45.733 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:19:45.727+0000 7f73d4210700 1 --2- 172.21.15.165:0/1485329525 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f73cc103d80 0x7f73cc10a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:19:45.733 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:19:45.727+0000 7f73d4210700 1 --2- 172.21.15.165:0/1485329525 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f73cc1021e0 0x7f73cc102600 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:19:45.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:19:45.727+0000 7f73d4210700 1 --2- 172.21.15.165:0/1485329525 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73cc1033e0 0x7f73cc103840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:19:45.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:19:45.727+0000 7f73d4210700 1 -- 172.21.15.165:0/1485329525 >> 172.21.15.165:0/1485329525 conn(0x7f73cc0fd740 msgr2=0x7f73cc0ffbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:19:45.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:19:45.727+0000 7f73d4210700 1 -- 172.21.15.165:0/1485329525 shutdown_connections 2024-03-28T00:19:45.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:19:45.727+0000 7f73d4210700 1 -- 172.21.15.165:0/1485329525 wait complete. 2024-03-28T00:19:45.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:19:45.728+0000 7f73d4210700 1 Processor -- start 2024-03-28T00:19:45.736 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:19:45.728+0000 7f73d4210700 1 -- start start 2024-03-28T00:19:45.736 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:19:45.728+0000 7f73d4210700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73cc1021e0 0x7f73cc198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:19:45.736 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:19:45.728+0000 7f73d4210700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f73cc1033e0 0x7f73cc1989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:19:45.736 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:19:45.728+0000 7f73d4210700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f73cc103d80 0x7f73cc19fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:19:45.736 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:19:45.728+0000 7f73d4210700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73cc10de50 con 0x7f73cc1021e0 2024-03-28T00:19:45.736 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:19:45.728+0000 7f73d4210700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73cc10dcd0 con 0x7f73cc1033e0 2024-03-28T00:19:45.736 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:19:45.728+0000 7f73d4210700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73cc10dfd0 con 0x7f73cc103d80 2024-03-28T00:19:45.736 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:19:45.728+0000 7f73d27ad700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f73cc103d80 0x7f73cc19fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:19:45.736 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:19:45.728+0000 7f73d27ad700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f73cc103d80 0x7f73cc19fa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:54116/0 (socket says 172.21.15.165:54116) 2024-03-28T00:19:45.736 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:19:45.728+0000 7f73d27ad700 1 -- 172.21.15.165:0/271351018 learned_addr learned my addr 172.21.15.165:0/271351018 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-03-28T00:19:45.736 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:19:45.728+0000 7f73d17ab700 1 --2- 172.21.15.165:0/271351018 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f73cc1033e0 0x7f73cc1989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:19:45.736 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:19:45.728+0000 7f73d27ad700 1 -- 172.21.15.165:0/271351018 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f73cc1033e0 msgr2=0x7f73cc1989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:19:45.736 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:19:45.728+0000 7f73d27ad700 1 --2- 172.21.15.165:0/271351018 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f73cc1033e0 0x7f73cc1989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:19:45.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:19:45.728+0000 7f73d27ad700 1 -- 172.21.15.165:0/271351018 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73cc1021e0 msgr2=0x7f73cc198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:19:45.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:19:45.728+0000 7f73d27ad700 1 --2- 172.21.15.165:0/271351018 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73cc1021e0 0x7f73cc198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:19:45.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:19:45.728+0000 7f73d27ad700 1 -- 172.21.15.165:0/271351018 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73cc1a00d0 con 0x7f73cc103d80 2024-03-28T00:19:45.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:19:45.729+0000 7f73d27ad700 1 --2- 172.21.15.165:0/271351018 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f73cc103d80 0x7f73cc19fa30 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f73c4007f10 tx=0x7f73c400d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:19:45.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:19:45.729+0000 7f73c2ffd700 1 -- 172.21.15.165:0/271351018 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73c4004bb0 con 0x7f73cc103d80 2024-03-28T00:19:45.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:19:45.729+0000 7f73c2ffd700 1 -- 172.21.15.165:0/271351018 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f73c4004d50 con 0x7f73cc103d80 2024-03-28T00:19:45.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:19:45.729+0000 7f73d4210700 1 -- 172.21.15.165:0/271351018 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73cc1a03c0 con 0x7f73cc103d80 2024-03-28T00:19:45.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:19:45.729+0000 7f73c2ffd700 1 -- 172.21.15.165:0/271351018 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73c4005a60 con 0x7f73cc103d80 2024-03-28T00:19:45.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:19:45.729+0000 7f73d4210700 1 -- 172.21.15.165:0/271351018 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73cc1a09d0 con 0x7f73cc103d80 2024-03-28T00:19:45.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:19:45.730+0000 7f73d4210700 1 -- 172.21.15.165:0/271351018 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73cc068e50 con 0x7f73cc103d80 2024-03-28T00:19:45.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:19:45.731+0000 7f73c2ffd700 1 -- 172.21.15.165:0/271351018 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f73c401a020 con 0x7f73cc103d80 2024-03-28T00:19:45.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:19:45.731+0000 7f73c2ffd700 1 --2- 172.21.15.165:0/271351018 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f73b80a2910 0x7f73b80a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:19:45.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:19:45.732+0000 7f73c2ffd700 1 -- 172.21.15.165:0/271351018 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 4101+0+0 (secure 0 0 0) 0x7f73c40cd0f0 con 0x7f73cc103d80 2024-03-28T00:19:45.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:19:45.732+0000 7f73d1fac700 1 --2- 172.21.15.165:0/271351018 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f73b80a2910 0x7f73b80a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:19:45.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:19:45.733+0000 7f73d1fac700 1 --2- 172.21.15.165:0/271351018 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f73b80a2910 0x7f73b80a4dd0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f73bc005f50 tx=0x7f73bc005e80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:19:45.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:19:45.736+0000 7f73c2ffd700 1 -- 172.21.15.165:0/271351018 <== mon.1 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f73c4010070 con 0x7f73cc103d80 2024-03-28T00:19:45.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:19:45.941+0000 7f73d4210700 1 -- 172.21.15.165:0/271351018 --> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi165:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f73cc0625d0 con 0x7f73b80a2910 2024-03-28T00:19:46.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:19:45 smithi165 ceph-mon[36302]: Detected new or changed devices on smithi132 2024-03-28T00:19:46.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:19:45 smithi165 ceph-mon[36302]: Adjusting osd_memory_target on smithi132 to 8540M 2024-03-28T00:19:46.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:19:45 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-28T00:19:46.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:19:45 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-28T00:19:46.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:19:45 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:19:46.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:19:45 smithi119 ceph-mon[31518]: Detected new or changed devices on smithi132 2024-03-28T00:19:46.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:19:45 smithi119 ceph-mon[31518]: Adjusting osd_memory_target on smithi132 to 8540M 2024-03-28T00:19:46.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:19:45 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-28T00:19:46.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:19:45 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-28T00:19:46.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:19:45 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:19:46.452 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:19:45 smithi132 ceph-mon[35941]: Detected new or changed devices on smithi132 2024-03-28T00:19:46.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:19:45 smithi132 ceph-mon[35941]: Adjusting osd_memory_target on smithi132 to 8540M 2024-03-28T00:19:46.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:19:45 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-28T00:19:46.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:19:45 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-28T00:19:46.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:19:45 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:19:47.294 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:19:46 smithi165 ceph-mon[36302]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-28T00:19:47.294 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:19:46 smithi165 ceph-mon[36302]: from='client.24235 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi165:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T00:19:47.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:19:46 smithi119 ceph-mon[31518]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-28T00:19:47.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:19:46 smithi119 ceph-mon[31518]: from='client.24235 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi165:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T00:19:47.452 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:19:46 smithi132 ceph-mon[35941]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-28T00:19:47.452 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:19:46 smithi132 ceph-mon[35941]: from='client.24235 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi165:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T00:19:48.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:19:47 smithi165 ceph-mon[36302]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-28T00:19:48.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:19:47 smithi119 ceph-mon[31518]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-28T00:19:48.452 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:19:47 smithi132 ceph-mon[35941]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-28T00:19:49.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:19:48 smithi165 ceph-mon[36302]: from='client.? 172.21.15.165:0/4131398223' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3224b4c1-733e-4424-a6da-870bbb5f94e5"}]: dispatch 2024-03-28T00:19:49.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:19:48 smithi165 ceph-mon[36302]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3224b4c1-733e-4424-a6da-870bbb5f94e5"}]: dispatch 2024-03-28T00:19:49.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:19:48 smithi165 ceph-mon[36302]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3224b4c1-733e-4424-a6da-870bbb5f94e5"}]': finished 2024-03-28T00:19:49.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:19:48 smithi165 ceph-mon[36302]: osdmap e35: 6 total, 5 up, 6 in 2024-03-28T00:19:49.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:19:48 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-28T00:19:49.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:19:48 smithi119 ceph-mon[31518]: from='client.? 172.21.15.165:0/4131398223' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3224b4c1-733e-4424-a6da-870bbb5f94e5"}]: dispatch 2024-03-28T00:19:49.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:19:48 smithi119 ceph-mon[31518]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3224b4c1-733e-4424-a6da-870bbb5f94e5"}]: dispatch 2024-03-28T00:19:49.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:19:48 smithi119 ceph-mon[31518]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3224b4c1-733e-4424-a6da-870bbb5f94e5"}]': finished 2024-03-28T00:19:49.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:19:48 smithi119 ceph-mon[31518]: osdmap e35: 6 total, 5 up, 6 in 2024-03-28T00:19:49.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:19:48 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-28T00:19:49.452 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:19:48 smithi132 ceph-mon[35941]: from='client.? 172.21.15.165:0/4131398223' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3224b4c1-733e-4424-a6da-870bbb5f94e5"}]: dispatch 2024-03-28T00:19:49.452 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:19:48 smithi132 ceph-mon[35941]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3224b4c1-733e-4424-a6da-870bbb5f94e5"}]: dispatch 2024-03-28T00:19:49.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:19:48 smithi132 ceph-mon[35941]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3224b4c1-733e-4424-a6da-870bbb5f94e5"}]': finished 2024-03-28T00:19:49.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:19:48 smithi132 ceph-mon[35941]: osdmap e35: 6 total, 5 up, 6 in 2024-03-28T00:19:49.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:19:48 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-28T00:19:50.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:19:50 smithi165 ceph-mon[36302]: from='client.? 172.21.15.165:0/2312645900' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-28T00:19:50.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:19:50 smithi165 ceph-mon[36302]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-28T00:19:50.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:19:50 smithi119 ceph-mon[31518]: from='client.? 172.21.15.165:0/2312645900' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-28T00:19:50.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:19:50 smithi119 ceph-mon[31518]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-28T00:19:50.452 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:19:50 smithi132 ceph-mon[35941]: from='client.? 172.21.15.165:0/2312645900' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-28T00:19:50.452 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:19:50 smithi132 ceph-mon[35941]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-28T00:19:53.093 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:19:52 smithi119 ceph-mon[31518]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-28T00:19:53.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:19:52 smithi165 ceph-mon[36302]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-28T00:19:53.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:19:52 smithi132 ceph-mon[35941]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-28T00:19:54.984 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:19:54 smithi165 ceph-mon[36302]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-28T00:19:55.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:19:54 smithi119 ceph-mon[31518]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-28T00:19:55.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:19:54 smithi132 ceph-mon[35941]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-28T00:19:57.005 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:19:56 smithi132 ceph-mon[35941]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-28T00:19:57.005 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:19:56 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-28T00:19:57.005 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:19:56 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:19:57.034 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:19:56 smithi165 ceph-mon[36302]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-28T00:19:57.034 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:19:56 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-28T00:19:57.034 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:19:56 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:19:57.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:19:56 smithi119 ceph-mon[31518]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-28T00:19:57.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:19:56 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-28T00:19:57.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:19:56 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:19:58.018 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:19:57 smithi132 ceph-mon[35941]: Deploying daemon osd.5 on smithi165 2024-03-28T00:19:58.073 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:19:57 smithi165 ceph-mon[36302]: Deploying daemon osd.5 on smithi165 2024-03-28T00:19:58.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:19:57 smithi119 ceph-mon[31518]: Deploying daemon osd.5 on smithi165 2024-03-28T00:19:59.094 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:19:58 smithi165 ceph-mon[36302]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-28T00:19:59.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:19:58 smithi119 ceph-mon[31518]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-28T00:19:59.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:19:58 smithi132 ceph-mon[35941]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-28T00:20:00.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:20:00 smithi165 ceph-mon[36302]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-28T00:20:00.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:20:00 smithi165 ceph-mon[36302]: overall HEALTH_OK 2024-03-28T00:20:01.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:20:00 smithi119 ceph-mon[31518]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-28T00:20:01.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:20:00 smithi119 ceph-mon[31518]: overall HEALTH_OK 2024-03-28T00:20:01.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:20:00 smithi132 ceph-mon[35941]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-28T00:20:01.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:20:00 smithi132 ceph-mon[35941]: overall HEALTH_OK 2024-03-28T00:20:03.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:20:02 smithi165 ceph-mon[36302]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-28T00:20:03.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:20:02 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:20:03.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:20:02 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:20:03.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:20:02 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:20:03.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:20:02 smithi119 ceph-mon[31518]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-28T00:20:03.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:20:02 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:20:03.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:20:02 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:20:03.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:20:02 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:20:03.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:20:02 smithi132 ceph-mon[35941]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-28T00:20:03.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:20:02 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:20:03.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:20:02 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:20:03.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:20:02 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:20:05.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:20:04 smithi165 ceph-mon[36302]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-28T00:20:05.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:20:04 smithi119 ceph-mon[31518]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-28T00:20:05.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:20:04 smithi132 ceph-mon[35941]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-28T00:20:05.336 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:20:05.330+0000 7f73c2ffd700 1 -- 172.21.15.165:0/271351018 <== mgr.14152 v2:172.21.15.119:6800/1401937096 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f73cc0625d0 con 0x7f73b80a2910 2024-03-28T00:20:05.336 INFO:teuthology.orchestra.run.smithi165.stdout:Created osd(s) 5 on host 'smithi165' 2024-03-28T00:20:05.338 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:20:05.332+0000 7f73d4210700 1 -- 172.21.15.165:0/271351018 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f73b80a2910 msgr2=0x7f73b80a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:20:05.338 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:20:05.332+0000 7f73d4210700 1 --2- 172.21.15.165:0/271351018 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f73b80a2910 0x7f73b80a4dd0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f73bc005f50 tx=0x7f73bc005e80 comp rx=0 tx=0).stop 2024-03-28T00:20:05.339 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:20:05.332+0000 7f73d4210700 1 -- 172.21.15.165:0/271351018 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f73cc103d80 msgr2=0x7f73cc19fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:20:05.339 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:20:05.332+0000 7f73d4210700 1 --2- 172.21.15.165:0/271351018 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f73cc103d80 0x7f73cc19fa30 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f73c4007f10 tx=0x7f73c400d3b0 comp rx=0 tx=0).stop 2024-03-28T00:20:05.339 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:20:05.332+0000 7f73d4210700 1 -- 172.21.15.165:0/271351018 shutdown_connections 2024-03-28T00:20:05.339 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:20:05.332+0000 7f73d4210700 1 --2- 172.21.15.165:0/271351018 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f73b80a2910 0x7f73b80a4dd0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:20:05.339 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:20:05.332+0000 7f73d4210700 1 --2- 172.21.15.165:0/271351018 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f73cc103d80 0x7f73cc19fa30 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:20:05.339 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:20:05.332+0000 7f73d4210700 1 --2- 172.21.15.165:0/271351018 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73cc1021e0 0x7f73cc198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:20:05.339 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:20:05.332+0000 7f73d4210700 1 --2- 172.21.15.165:0/271351018 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f73cc1033e0 0x7f73cc1989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:20:05.339 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:20:05.332+0000 7f73d4210700 1 -- 172.21.15.165:0/271351018 >> 172.21.15.165:0/271351018 conn(0x7f73cc0fd740 msgr2=0x7f73cc106760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:20:05.339 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:20:05.333+0000 7f73d4210700 1 -- 172.21.15.165:0/271351018 shutdown_connections 2024-03-28T00:20:05.339 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:20:05.333+0000 7f73d4210700 1 -- 172.21.15.165:0/271351018 wait complete. 2024-03-28T00:20:06.001 DEBUG:teuthology.orchestra.run.smithi165:osd.5> sudo journalctl -f -n 0 -u ceph-10f73b92-ec98-11ee-95d0-87774f69a715@osd.5.service 2024-03-28T00:20:06.004 INFO:tasks.cephadm:Deploying osd.6 on smithi119 with /dev/vg_nvme/lv_2... 2024-03-28T00:20:06.004 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 10f73b92-ec98-11ee-95d0-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-28T00:20:06.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:20:06 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:20:06.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:20:06 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:20:06.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:20:06 smithi119 ceph-mon[31518]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-28T00:20:06.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:20:06 smithi119 ceph-mon[31518]: from='osd.5 [v2:172.21.15.165:6808/2561252206,v1:172.21.15.165:6809/2561252206]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-28T00:20:06.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:20:06 smithi119 ceph-mon[31518]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-28T00:20:06.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:20:06 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:20:06.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:20:06 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:20:06.475 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/mon.a/config 2024-03-28T00:20:06.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:20:06 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:20:06.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:20:06 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:20:06.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:20:06 smithi165 ceph-mon[36302]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-28T00:20:06.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:20:06 smithi165 ceph-mon[36302]: from='osd.5 [v2:172.21.15.165:6808/2561252206,v1:172.21.15.165:6809/2561252206]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-28T00:20:06.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:20:06 smithi165 ceph-mon[36302]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-28T00:20:06.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:20:06 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:20:06.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:20:06 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:20:06.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:20:06 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:20:06.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:20:06 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:20:06.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:20:06 smithi132 ceph-mon[35941]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-28T00:20:06.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:20:06 smithi132 ceph-mon[35941]: from='osd.5 [v2:172.21.15.165:6808/2561252206,v1:172.21.15.165:6809/2561252206]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-28T00:20:06.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:20:06 smithi132 ceph-mon[35941]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-28T00:20:06.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:20:06 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:20:06.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:20:06 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:20:07.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:20:07 smithi165 ceph-mon[36302]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-28T00:20:07.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:20:07 smithi165 ceph-mon[36302]: from='osd.5 [v2:172.21.15.165:6808/2561252206,v1:172.21.15.165:6809/2561252206]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]: dispatch 2024-03-28T00:20:07.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:20:07 smithi165 ceph-mon[36302]: osdmap e36: 6 total, 5 up, 6 in 2024-03-28T00:20:07.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:20:07 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-28T00:20:07.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:20:07 smithi165 ceph-mon[36302]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]: dispatch 2024-03-28T00:20:07.633 INFO:journalctl@ceph.osd.5.smithi165.stdout:Mar 28 00:20:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-5[48453]: 2024-03-28T00:20:07.340+0000 7f5e32841700 -1 osd.5 0 waiting for initial osdmap 2024-03-28T00:20:07.633 INFO:journalctl@ceph.osd.5.smithi165.stdout:Mar 28 00:20:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-5[48453]: 2024-03-28T00:20:07.348+0000 7f5e2e634700 -1 osd.5 37 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-28T00:20:07.634 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:20:07 smithi119 ceph-mon[31518]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-28T00:20:07.634 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:20:07 smithi119 ceph-mon[31518]: from='osd.5 [v2:172.21.15.165:6808/2561252206,v1:172.21.15.165:6809/2561252206]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]: dispatch 2024-03-28T00:20:07.634 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:20:07 smithi119 ceph-mon[31518]: osdmap e36: 6 total, 5 up, 6 in 2024-03-28T00:20:07.635 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:20:07 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-28T00:20:07.635 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:20:07 smithi119 ceph-mon[31518]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]: dispatch 2024-03-28T00:20:07.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:20:07 smithi132 ceph-mon[35941]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-28T00:20:07.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:20:07 smithi132 ceph-mon[35941]: from='osd.5 [v2:172.21.15.165:6808/2561252206,v1:172.21.15.165:6809/2561252206]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]: dispatch 2024-03-28T00:20:07.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:20:07 smithi132 ceph-mon[35941]: osdmap e36: 6 total, 5 up, 6 in 2024-03-28T00:20:07.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:20:07 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-28T00:20:07.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:20:07 smithi132 ceph-mon[35941]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]: dispatch 2024-03-28T00:20:08.611 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:20:08 smithi119 ceph-mon[31518]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]': finished 2024-03-28T00:20:08.611 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:20:08 smithi119 ceph-mon[31518]: osdmap e37: 6 total, 5 up, 6 in 2024-03-28T00:20:08.611 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:20:08 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-28T00:20:08.611 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:20:08 smithi119 ceph-mon[31518]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-28T00:20:08.612 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:20:08 smithi119 ceph-mon[31518]: osd.5 [v2:172.21.15.165:6808/2561252206,v1:172.21.15.165:6809/2561252206] boot 2024-03-28T00:20:08.612 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:20:08 smithi119 ceph-mon[31518]: osdmap e38: 6 total, 6 up, 6 in 2024-03-28T00:20:08.612 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:20:08 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-28T00:20:08.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:20:08 smithi165 ceph-mon[36302]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]': finished 2024-03-28T00:20:08.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:20:08 smithi165 ceph-mon[36302]: osdmap e37: 6 total, 5 up, 6 in 2024-03-28T00:20:08.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:20:08 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-28T00:20:08.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:20:08 smithi165 ceph-mon[36302]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-28T00:20:08.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:20:08 smithi165 ceph-mon[36302]: osd.5 [v2:172.21.15.165:6808/2561252206,v1:172.21.15.165:6809/2561252206] boot 2024-03-28T00:20:08.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:20:08 smithi165 ceph-mon[36302]: osdmap e38: 6 total, 6 up, 6 in 2024-03-28T00:20:08.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:20:08 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-28T00:20:08.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:20:08 smithi132 ceph-mon[35941]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]': finished 2024-03-28T00:20:08.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:20:08 smithi132 ceph-mon[35941]: osdmap e37: 6 total, 5 up, 6 in 2024-03-28T00:20:08.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:20:08 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-28T00:20:08.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:20:08 smithi132 ceph-mon[35941]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-28T00:20:08.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:20:08 smithi132 ceph-mon[35941]: osd.5 [v2:172.21.15.165:6808/2561252206,v1:172.21.15.165:6809/2561252206] boot 2024-03-28T00:20:08.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:20:08 smithi132 ceph-mon[35941]: osdmap e38: 6 total, 6 up, 6 in 2024-03-28T00:20:08.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:20:08 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-28T00:20:09.587 INFO:teuthology.orchestra.run.smithi119.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-28T00:20:09.587 INFO:teuthology.orchestra.run.smithi119.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-28T00:20:09.587 INFO:teuthology.orchestra.run.smithi119.stderr: stderr: 10+0 records in 2024-03-28T00:20:09.587 INFO:teuthology.orchestra.run.smithi119.stderr:10+0 records out 2024-03-28T00:20:09.587 INFO:teuthology.orchestra.run.smithi119.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0317536 s, 330 MB/s 2024-03-28T00:20:09.588 INFO:teuthology.orchestra.run.smithi119.stderr:--> Zapping successful for: 2024-03-28T00:20:09.629 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:20:09 smithi119 ceph-mon[31518]: purged_snaps scrub starts 2024-03-28T00:20:09.629 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:20:09 smithi119 ceph-mon[31518]: purged_snaps scrub ok 2024-03-28T00:20:09.629 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:20:09 smithi119 ceph-mon[31518]: osdmap e39: 6 total, 6 up, 6 in 2024-03-28T00:20:09.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:20:09 smithi165 ceph-mon[36302]: purged_snaps scrub starts 2024-03-28T00:20:09.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:20:09 smithi165 ceph-mon[36302]: purged_snaps scrub ok 2024-03-28T00:20:09.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:20:09 smithi165 ceph-mon[36302]: osdmap e39: 6 total, 6 up, 6 in 2024-03-28T00:20:09.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:20:09 smithi132 ceph-mon[35941]: purged_snaps scrub starts 2024-03-28T00:20:09.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:20:09 smithi132 ceph-mon[35941]: purged_snaps scrub ok 2024-03-28T00:20:09.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:20:09 smithi132 ceph-mon[35941]: osdmap e39: 6 total, 6 up, 6 in 2024-03-28T00:20:10.588 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 10f73b92-ec98-11ee-95d0-87774f69a715 -- ceph orch daemon add osd smithi119:vg_nvme/lv_2 2024-03-28T00:20:10.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:20:10 smithi119 ceph-mon[31518]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 172 MiB used, 536 GiB / 536 GiB avail 2024-03-28T00:20:10.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:20:10 smithi119 ceph-mon[31518]: Detected new or changed devices on smithi165 2024-03-28T00:20:10.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:20:10 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:20:10.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:20:10 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:20:10.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:20:10 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:20:10.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:20:10 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:20:10.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:20:10 smithi119 ceph-mon[31518]: Adjusting osd_memory_target on smithi165 to 8540M 2024-03-28T00:20:10.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:20:10 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:20:10.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:20:10 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:20:10.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:20:10 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:20:10.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:20:10 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:20:10.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:20:10 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:20:10.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:20:10 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:20:10.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:20:10 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:20:10.907 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:20:10 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:20:10.907 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:20:10 smithi119 ceph-mon[31518]: osdmap e40: 6 total, 6 up, 6 in 2024-03-28T00:20:11.012 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/mon.a/config 2024-03-28T00:20:11.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:20:10 smithi165 ceph-mon[36302]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 172 MiB used, 536 GiB / 536 GiB avail 2024-03-28T00:20:11.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:20:10 smithi165 ceph-mon[36302]: Detected new or changed devices on smithi165 2024-03-28T00:20:11.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:20:10 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:20:11.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:20:10 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:20:11.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:20:10 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:20:11.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:20:10 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:20:11.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:20:10 smithi165 ceph-mon[36302]: Adjusting osd_memory_target on smithi165 to 8540M 2024-03-28T00:20:11.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:20:10 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:20:11.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:20:10 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:20:11.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:20:10 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:20:11.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:20:10 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:20:11.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:20:10 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:20:11.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:20:10 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:20:11.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:20:10 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:20:11.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:20:10 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:20:11.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:20:10 smithi165 ceph-mon[36302]: osdmap e40: 6 total, 6 up, 6 in 2024-03-28T00:20:11.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:20:10 smithi132 ceph-mon[35941]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 172 MiB used, 536 GiB / 536 GiB avail 2024-03-28T00:20:11.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:20:10 smithi132 ceph-mon[35941]: Detected new or changed devices on smithi165 2024-03-28T00:20:11.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:20:10 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:20:11.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:20:10 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:20:11.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:20:10 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:20:11.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:20:10 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:20:11.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:20:10 smithi132 ceph-mon[35941]: Adjusting osd_memory_target on smithi165 to 8540M 2024-03-28T00:20:11.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:20:10 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:20:11.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:20:10 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:20:11.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:20:10 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:20:11.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:20:10 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:20:11.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:20:10 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:20:11.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:20:10 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:20:11.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:20:10 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:20:11.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:20:10 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:20:11.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:20:10 smithi132 ceph-mon[35941]: osdmap e40: 6 total, 6 up, 6 in 2024-03-28T00:20:12.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:20:12 smithi165 ceph-mon[36302]: osdmap e41: 6 total, 6 up, 6 in 2024-03-28T00:20:12.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:20:12 smithi165 ceph-mon[36302]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-28T00:20:12.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:20:12 smithi119 ceph-mon[31518]: osdmap e41: 6 total, 6 up, 6 in 2024-03-28T00:20:12.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:20:12 smithi119 ceph-mon[31518]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-28T00:20:12.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:20:12 smithi132 ceph-mon[35941]: osdmap e41: 6 total, 6 up, 6 in 2024-03-28T00:20:12.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:20:12 smithi132 ceph-mon[35941]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-28T00:20:13.195 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:20:13.188+0000 7ff946400700 1 -- 172.21.15.119:0/3217140421 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff940100ab0 msgr2=0x7ff940100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:20:13.195 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:20:13.188+0000 7ff946400700 1 --2- 172.21.15.119:0/3217140421 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff940100ab0 0x7ff940100ed0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff930009990 tx=0x7ff930009ca0 comp rx=0 tx=0).stop 2024-03-28T00:20:13.195 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:20:13.188+0000 7ff946400700 1 -- 172.21.15.119:0/3217140421 shutdown_connections 2024-03-28T00:20:13.195 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:20:13.188+0000 7ff946400700 1 --2- 172.21.15.119:0/3217140421 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff940101dd0 0x7ff940102250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:20:13.195 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:20:13.188+0000 7ff946400700 1 --2- 172.21.15.119:0/3217140421 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff940100ab0 0x7ff940100ed0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:20:13.195 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:20:13.188+0000 7ff946400700 1 --2- 172.21.15.119:0/3217140421 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff940101410 0x7ff940101890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:20:13.195 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:20:13.188+0000 7ff946400700 1 -- 172.21.15.119:0/3217140421 >> 172.21.15.119:0/3217140421 conn(0x7ff9400fc6f0 msgr2=0x7ff9400feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:20:13.195 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:20:13.188+0000 7ff946400700 1 -- 172.21.15.119:0/3217140421 shutdown_connections 2024-03-28T00:20:13.195 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:20:13.188+0000 7ff946400700 1 -- 172.21.15.119:0/3217140421 wait complete. 2024-03-28T00:20:13.196 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:20:13.190+0000 7ff946400700 1 Processor -- start 2024-03-28T00:20:13.196 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:20:13.190+0000 7ff946400700 1 -- start start 2024-03-28T00:20:13.197 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:20:13.191+0000 7ff946400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff940100ab0 0x7ff94019a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:20:13.197 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:20:13.191+0000 7ff946400700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff940101410 0x7ff94019aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:20:13.197 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:20:13.191+0000 7ff946400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff940101dd0 0x7ff9401a1b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:20:13.197 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:20:13.191+0000 7ff946400700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff94010ff60 con 0x7ff940101dd0 2024-03-28T00:20:13.197 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:20:13.191+0000 7ff946400700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff94010fde0 con 0x7ff940100ab0 2024-03-28T00:20:13.197 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:20:13.191+0000 7ff946400700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9401100e0 con 0x7ff940101410 2024-03-28T00:20:13.198 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:20:13.191+0000 7ff93ea00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff940101410 0x7ff94019aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:20:13.199 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:20:13.191+0000 7ff93fe00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff940101dd0 0x7ff9401a1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:20:13.199 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:20:13.191+0000 7ff93fe00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff940101dd0 0x7ff9401a1b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:45190/0 (socket says 172.21.15.119:45190) 2024-03-28T00:20:13.199 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:20:13.191+0000 7ff93fe00700 1 -- 172.21.15.119:0/2291649778 learned_addr learned my addr 172.21.15.119:0/2291649778 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:20:13.199 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:20:13.191+0000 7ff93ea00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff940101410 0x7ff94019aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.119:46928/0 (socket says 172.21.15.119:46928) 2024-03-28T00:20:13.199 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:20:13.191+0000 7ff93f400700 1 --2- 172.21.15.119:0/2291649778 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff940100ab0 0x7ff94019a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:20:13.199 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:20:13.192+0000 7ff93ea00700 1 -- 172.21.15.119:0/2291649778 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff940100ab0 msgr2=0x7ff94019a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:20:13.199 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:20:13.192+0000 7ff93ea00700 1 --2- 172.21.15.119:0/2291649778 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff940100ab0 0x7ff94019a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:20:13.199 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:20:13.192+0000 7ff93ea00700 1 -- 172.21.15.119:0/2291649778 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff940101dd0 msgr2=0x7ff9401a1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:20:13.199 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:20:13.192+0000 7ff93ea00700 1 --2- 172.21.15.119:0/2291649778 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff940101dd0 0x7ff9401a1b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:20:13.199 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:20:13.192+0000 7ff93ea00700 1 -- 172.21.15.119:0/2291649778 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9401a21c0 con 0x7ff940101410 2024-03-28T00:20:13.199 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:20:13.192+0000 7ff93fe00700 1 --2- 172.21.15.119:0/2291649778 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff940101dd0 0x7ff9401a1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T00:20:13.199 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:20:13.192+0000 7ff93ea00700 1 --2- 172.21.15.119:0/2291649778 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff940101410 0x7ff94019aaa0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7ff92c00ea20 tx=0x7ff92c00ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:20:13.199 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:20:13.192+0000 7ff937e00700 1 -- 172.21.15.119:0/2291649778 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff92c004220 con 0x7ff940101410 2024-03-28T00:20:13.199 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:20:13.192+0000 7ff937e00700 1 -- 172.21.15.119:0/2291649778 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff92c00ce50 con 0x7ff940101410 2024-03-28T00:20:13.200 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:20:13.192+0000 7ff946400700 1 -- 172.21.15.119:0/2291649778 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9401a24b0 con 0x7ff940101410 2024-03-28T00:20:13.200 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:20:13.192+0000 7ff937e00700 1 -- 172.21.15.119:0/2291649778 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff92c003dc0 con 0x7ff940101410 2024-03-28T00:20:13.200 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:20:13.193+0000 7ff946400700 1 -- 172.21.15.119:0/2291649778 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9401a29f0 con 0x7ff940101410 2024-03-28T00:20:13.200 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:20:13.194+0000 7ff946400700 1 -- 172.21.15.119:0/2291649778 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff940068e50 con 0x7ff940101410 2024-03-28T00:20:13.202 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:20:13.196+0000 7ff937e00700 1 -- 172.21.15.119:0/2291649778 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7ff92c00c970 con 0x7ff940101410 2024-03-28T00:20:13.204 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:20:13.198+0000 7ff937e00700 1 --2- 172.21.15.119:0/2291649778 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7ff9240a2890 0x7ff9240a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:20:13.204 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:20:13.198+0000 7ff937e00700 1 -- 172.21.15.119:0/2291649778 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4533+0+0 (secure 0 0 0) 0x7ff92c0cde20 con 0x7ff940101410 2024-03-28T00:20:13.204 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:20:13.198+0000 7ff93f400700 1 --2- 172.21.15.119:0/2291649778 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7ff9240a2890 0x7ff9240a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:20:13.204 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:20:13.199+0000 7ff93f400700 1 --2- 172.21.15.119:0/2291649778 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7ff9240a2890 0x7ff9240a4d50 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff93000b2e0 tx=0x7ff93000b880 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:20:13.208 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:20:13.202+0000 7ff937e00700 1 -- 172.21.15.119:0/2291649778 <== mon.1 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff92c09bce0 con 0x7ff940101410 2024-03-28T00:20:13.440 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:20:13.434+0000 7ff946400700 1 -- 172.21.15.119:0/2291649778 --> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi119:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7ff9400625d0 con 0x7ff9240a2890 2024-03-28T00:20:13.465 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:20:13 smithi119 ceph-mon[31518]: osdmap e42: 6 total, 6 up, 6 in 2024-03-28T00:20:13.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:20:13 smithi165 ceph-mon[36302]: osdmap e42: 6 total, 6 up, 6 in 2024-03-28T00:20:13.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:20:13 smithi132 ceph-mon[35941]: osdmap e42: 6 total, 6 up, 6 in 2024-03-28T00:20:14.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:20:14 smithi119 ceph-mon[31518]: from='client.24262 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi119:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T00:20:14.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:20:14 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-28T00:20:14.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:20:14 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-28T00:20:14.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:20:14 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:20:14.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:20:14 smithi119 ceph-mon[31518]: pgmap v157: 1 pgs: 1 peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 133 KiB/s, 0 objects/s recovering 2024-03-28T00:20:14.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:20:14 smithi132 ceph-mon[35941]: from='client.24262 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi119:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T00:20:14.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:20:14 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-28T00:20:14.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:20:14 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-28T00:20:14.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:20:14 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:20:14.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:20:14 smithi132 ceph-mon[35941]: pgmap v157: 1 pgs: 1 peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 133 KiB/s, 0 objects/s recovering 2024-03-28T00:20:14.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:20:14 smithi165 ceph-mon[36302]: from='client.24262 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi119:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T00:20:14.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:20:14 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-28T00:20:14.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:20:14 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-28T00:20:14.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:20:14 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:20:14.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:20:14 smithi165 ceph-mon[36302]: pgmap v157: 1 pgs: 1 peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 133 KiB/s, 0 objects/s recovering 2024-03-28T00:20:17.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:20:16 smithi165 ceph-mon[36302]: pgmap v158: 1 pgs: 1 peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-28T00:20:17.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:20:16 smithi119 ceph-mon[31518]: pgmap v158: 1 pgs: 1 peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-28T00:20:17.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:20:16 smithi132 ceph-mon[35941]: pgmap v158: 1 pgs: 1 peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-28T00:20:18.033 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:20:17 smithi119 ceph-mon[31518]: from='client.? 172.21.15.119:0/2146996913' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e435745d-59b5-439a-9091-0946ca0e9332"}]: dispatch 2024-03-28T00:20:18.033 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:20:17 smithi119 ceph-mon[31518]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e435745d-59b5-439a-9091-0946ca0e9332"}]: dispatch 2024-03-28T00:20:18.034 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:20:17 smithi119 ceph-mon[31518]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e435745d-59b5-439a-9091-0946ca0e9332"}]': finished 2024-03-28T00:20:18.034 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:20:17 smithi119 ceph-mon[31518]: osdmap e43: 7 total, 6 up, 7 in 2024-03-28T00:20:18.034 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:20:17 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-28T00:20:18.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:20:17 smithi165 ceph-mon[36302]: from='client.? 172.21.15.119:0/2146996913' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e435745d-59b5-439a-9091-0946ca0e9332"}]: dispatch 2024-03-28T00:20:18.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:20:17 smithi165 ceph-mon[36302]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e435745d-59b5-439a-9091-0946ca0e9332"}]: dispatch 2024-03-28T00:20:18.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:20:17 smithi165 ceph-mon[36302]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e435745d-59b5-439a-9091-0946ca0e9332"}]': finished 2024-03-28T00:20:18.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:20:17 smithi165 ceph-mon[36302]: osdmap e43: 7 total, 6 up, 7 in 2024-03-28T00:20:18.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:20:17 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-28T00:20:18.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:20:17 smithi132 ceph-mon[35941]: from='client.? 172.21.15.119:0/2146996913' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e435745d-59b5-439a-9091-0946ca0e9332"}]: dispatch 2024-03-28T00:20:18.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:20:17 smithi132 ceph-mon[35941]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e435745d-59b5-439a-9091-0946ca0e9332"}]: dispatch 2024-03-28T00:20:18.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:20:17 smithi132 ceph-mon[35941]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e435745d-59b5-439a-9091-0946ca0e9332"}]': finished 2024-03-28T00:20:18.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:20:17 smithi132 ceph-mon[35941]: osdmap e43: 7 total, 6 up, 7 in 2024-03-28T00:20:18.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:20:17 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-28T00:20:19.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:20:18 smithi165 ceph-mon[36302]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-03-28T00:20:19.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:20:18 smithi165 ceph-mon[36302]: from='client.? 172.21.15.119:0/4039729619' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-28T00:20:19.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:20:18 smithi119 ceph-mon[31518]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-03-28T00:20:19.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:20:18 smithi119 ceph-mon[31518]: from='client.? 172.21.15.119:0/4039729619' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-28T00:20:19.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:20:18 smithi132 ceph-mon[35941]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-03-28T00:20:19.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:20:18 smithi132 ceph-mon[35941]: from='client.? 172.21.15.119:0/4039729619' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-28T00:20:21.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:20:20 smithi165 ceph-mon[36302]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-28T00:20:21.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:20:20 smithi119 ceph-mon[31518]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-28T00:20:21.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:20:20 smithi132 ceph-mon[35941]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-28T00:20:22.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:20:22 smithi119 ceph-mon[31518]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-28T00:20:23.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:20:22 smithi165 ceph-mon[36302]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-28T00:20:23.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:20:22 smithi132 ceph-mon[35941]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-28T00:20:25.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:20:24 smithi165 ceph-mon[36302]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-28T00:20:25.144 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:20:24 smithi119 ceph-mon[31518]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-28T00:20:25.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:20:24 smithi132 ceph-mon[35941]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-28T00:20:27.005 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:20:26 smithi132 ceph-mon[35941]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-28T00:20:27.005 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:20:26 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-28T00:20:27.005 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:20:26 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:20:27.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:20:26 smithi165 ceph-mon[36302]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-28T00:20:27.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:20:26 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-28T00:20:27.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:20:26 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:20:27.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:20:26 smithi119 ceph-mon[31518]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-28T00:20:27.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:20:26 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-28T00:20:27.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:20:26 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:20:27.957 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:20:27 smithi119 ceph-mon[31518]: Deploying daemon osd.6 on smithi119 2024-03-28T00:20:28.018 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:20:27 smithi132 ceph-mon[35941]: Deploying daemon osd.6 on smithi119 2024-03-28T00:20:28.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:20:27 smithi165 ceph-mon[36302]: Deploying daemon osd.6 on smithi119 2024-03-28T00:20:28.810 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:20:28 smithi119 ceph-mon[31518]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-28T00:20:29.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:20:28 smithi165 ceph-mon[36302]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-28T00:20:29.187 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:20:28 smithi132 ceph-mon[35941]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-28T00:20:31.056 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:20:30 smithi119 ceph-mon[31518]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-28T00:20:31.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:20:30 smithi165 ceph-mon[36302]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-28T00:20:31.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:20:30 smithi132 ceph-mon[35941]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-28T00:20:33.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:20:32 smithi165 ceph-mon[36302]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-28T00:20:33.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:20:32 smithi119 ceph-mon[31518]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-28T00:20:33.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:20:32 smithi132 ceph-mon[35941]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-28T00:20:35.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:20:34 smithi165 ceph-mon[36302]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-28T00:20:35.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:20:34 smithi119 ceph-mon[31518]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-28T00:20:35.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:20:34 smithi132 ceph-mon[35941]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-28T00:20:36.237 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:20:35 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:20:36.237 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:20:35 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:20:36.237 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:20:35 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:20:36.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:20:35 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:20:36.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:20:35 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:20:36.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:20:35 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:20:36.452 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:20:35 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:20:36.452 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:20:35 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:20:36.452 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:20:35 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:20:37.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:20:36 smithi119 ceph-mon[31518]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-28T00:20:37.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:20:36 smithi165 ceph-mon[36302]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-28T00:20:37.452 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:20:36 smithi132 ceph-mon[35941]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-28T00:20:39.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:20:38 smithi119 ceph-mon[31518]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-28T00:20:39.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:20:38 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:20:39.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:20:38 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:20:39.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:20:38 smithi119 ceph-mon[31518]: from='osd.6 [v2:172.21.15.119:6818/2545410230,v1:172.21.15.119:6819/2545410230]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-28T00:20:39.257 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:20:38 smithi132 ceph-mon[35941]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-28T00:20:39.258 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:20:38 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:20:39.258 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:20:38 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:20:39.258 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:20:38 smithi132 ceph-mon[35941]: from='osd.6 [v2:172.21.15.119:6818/2545410230,v1:172.21.15.119:6819/2545410230]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-28T00:20:39.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:20:38 smithi165 ceph-mon[36302]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-28T00:20:39.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:20:38 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:20:39.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:20:38 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:20:39.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:20:38 smithi165 ceph-mon[36302]: from='osd.6 [v2:172.21.15.119:6818/2545410230,v1:172.21.15.119:6819/2545410230]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-28T00:20:40.043 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:20:40.035+0000 7ff937e00700 1 -- 172.21.15.119:0/2291649778 <== mgr.14152 v2:172.21.15.119:6800/1401937096 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7ff9400625d0 con 0x7ff9240a2890 2024-03-28T00:20:40.044 INFO:teuthology.orchestra.run.smithi119.stdout:Created osd(s) 6 on host 'smithi119' 2024-03-28T00:20:40.045 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:20:40.039+0000 7ff946400700 1 -- 172.21.15.119:0/2291649778 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7ff9240a2890 msgr2=0x7ff9240a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:20:40.045 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:20:40.039+0000 7ff946400700 1 --2- 172.21.15.119:0/2291649778 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7ff9240a2890 0x7ff9240a4d50 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff93000b2e0 tx=0x7ff93000b880 comp rx=0 tx=0).stop 2024-03-28T00:20:40.045 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:20:40.039+0000 7ff946400700 1 -- 172.21.15.119:0/2291649778 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff940101410 msgr2=0x7ff94019aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:20:40.045 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:20:40.039+0000 7ff946400700 1 --2- 172.21.15.119:0/2291649778 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff940101410 0x7ff94019aaa0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7ff92c00ea20 tx=0x7ff92c00ed30 comp rx=0 tx=0).stop 2024-03-28T00:20:40.046 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:20:40.039+0000 7ff946400700 1 -- 172.21.15.119:0/2291649778 shutdown_connections 2024-03-28T00:20:40.046 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:20:40.039+0000 7ff946400700 1 --2- 172.21.15.119:0/2291649778 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7ff9240a2890 0x7ff9240a4d50 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:20:40.046 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:20:40.039+0000 7ff946400700 1 --2- 172.21.15.119:0/2291649778 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff940101dd0 0x7ff9401a1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:20:40.046 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:20:40.039+0000 7ff946400700 1 --2- 172.21.15.119:0/2291649778 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff940100ab0 0x7ff94019a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:20:40.046 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:20:40.039+0000 7ff946400700 1 --2- 172.21.15.119:0/2291649778 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff940101410 0x7ff94019aaa0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:20:40.046 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:20:40.039+0000 7ff946400700 1 -- 172.21.15.119:0/2291649778 >> 172.21.15.119:0/2291649778 conn(0x7ff9400fc6f0 msgr2=0x7ff9400fe1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:20:40.046 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:20:40.040+0000 7ff946400700 1 -- 172.21.15.119:0/2291649778 shutdown_connections 2024-03-28T00:20:40.047 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:20:40.040+0000 7ff946400700 1 -- 172.21.15.119:0/2291649778 wait complete. 2024-03-28T00:20:40.908 DEBUG:teuthology.orchestra.run.smithi119:osd.6> sudo journalctl -f -n 0 -u ceph-10f73b92-ec98-11ee-95d0-87774f69a715@osd.6.service 2024-03-28T00:20:40.912 INFO:tasks.cephadm:Deploying osd.7 on smithi132 with /dev/vg_nvme/lv_2... 2024-03-28T00:20:40.912 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 10f73b92-ec98-11ee-95d0-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-28T00:20:41.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:20:40 smithi165 ceph-mon[36302]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-28T00:20:41.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:20:40 smithi165 ceph-mon[36302]: from='osd.6 [v2:172.21.15.119:6818/2545410230,v1:172.21.15.119:6819/2545410230]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-28T00:20:41.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:20:40 smithi165 ceph-mon[36302]: osdmap e44: 7 total, 6 up, 7 in 2024-03-28T00:20:41.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:20:40 smithi165 ceph-mon[36302]: from='osd.6 [v2:172.21.15.119:6818/2545410230,v1:172.21.15.119:6819/2545410230]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]: dispatch 2024-03-28T00:20:41.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:20:40 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-28T00:20:41.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:20:40 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:20:41.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:20:40 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:20:41.133 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:20:40 smithi119 ceph-mon[31518]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-28T00:20:41.133 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:20:40 smithi119 ceph-mon[31518]: from='osd.6 [v2:172.21.15.119:6818/2545410230,v1:172.21.15.119:6819/2545410230]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-28T00:20:41.133 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:20:40 smithi119 ceph-mon[31518]: osdmap e44: 7 total, 6 up, 7 in 2024-03-28T00:20:41.133 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:20:40 smithi119 ceph-mon[31518]: from='osd.6 [v2:172.21.15.119:6818/2545410230,v1:172.21.15.119:6819/2545410230]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]: dispatch 2024-03-28T00:20:41.134 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:20:40 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-28T00:20:41.134 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:20:40 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:20:41.134 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:20:40 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:20:41.157 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/mon.b/config 2024-03-28T00:20:41.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:20:40 smithi132 ceph-mon[35941]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-28T00:20:41.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:20:40 smithi132 ceph-mon[35941]: from='osd.6 [v2:172.21.15.119:6818/2545410230,v1:172.21.15.119:6819/2545410230]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-28T00:20:41.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:20:40 smithi132 ceph-mon[35941]: osdmap e44: 7 total, 6 up, 7 in 2024-03-28T00:20:41.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:20:40 smithi132 ceph-mon[35941]: from='osd.6 [v2:172.21.15.119:6818/2545410230,v1:172.21.15.119:6819/2545410230]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]: dispatch 2024-03-28T00:20:41.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:20:40 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-28T00:20:41.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:20:40 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:20:41.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:20:40 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:20:41.878 INFO:journalctl@ceph.osd.6.smithi119.stdout:Mar 28 00:20:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-6[57000]: 2024-03-28T00:20:41.426+0000 7f9273400700 -1 osd.6 0 waiting for initial osdmap 2024-03-28T00:20:41.879 INFO:journalctl@ceph.osd.6.smithi119.stdout:Mar 28 00:20:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-6[57000]: 2024-03-28T00:20:41.453+0000 7f9268800700 -1 osd.6 45 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-28T00:20:42.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:20:41 smithi165 ceph-mon[36302]: from='osd.6 [v2:172.21.15.119:6818/2545410230,v1:172.21.15.119:6819/2545410230]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]': finished 2024-03-28T00:20:42.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:20:41 smithi165 ceph-mon[36302]: osdmap e45: 7 total, 6 up, 7 in 2024-03-28T00:20:42.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:20:41 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-28T00:20:42.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:20:41 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-28T00:20:42.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:20:41 smithi165 ceph-mon[36302]: from='osd.6 [v2:172.21.15.119:6818/2545410230,v1:172.21.15.119:6819/2545410230]' entity='osd.6' 2024-03-28T00:20:42.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:20:41 smithi119 ceph-mon[31518]: from='osd.6 [v2:172.21.15.119:6818/2545410230,v1:172.21.15.119:6819/2545410230]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]': finished 2024-03-28T00:20:42.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:20:41 smithi119 ceph-mon[31518]: osdmap e45: 7 total, 6 up, 7 in 2024-03-28T00:20:42.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:20:41 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-28T00:20:42.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:20:41 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-28T00:20:42.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:20:41 smithi119 ceph-mon[31518]: from='osd.6 [v2:172.21.15.119:6818/2545410230,v1:172.21.15.119:6819/2545410230]' entity='osd.6' 2024-03-28T00:20:42.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:20:41 smithi132 ceph-mon[35941]: from='osd.6 [v2:172.21.15.119:6818/2545410230,v1:172.21.15.119:6819/2545410230]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]': finished 2024-03-28T00:20:42.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:20:41 smithi132 ceph-mon[35941]: osdmap e45: 7 total, 6 up, 7 in 2024-03-28T00:20:42.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:20:41 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-28T00:20:42.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:20:41 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-28T00:20:42.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:20:41 smithi132 ceph-mon[35941]: from='osd.6 [v2:172.21.15.119:6818/2545410230,v1:172.21.15.119:6819/2545410230]' entity='osd.6' 2024-03-28T00:20:42.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:20:42 smithi119 ceph-mon[31518]: purged_snaps scrub starts 2024-03-28T00:20:42.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:20:42 smithi119 ceph-mon[31518]: purged_snaps scrub ok 2024-03-28T00:20:42.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:20:42 smithi119 ceph-mon[31518]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-28T00:20:42.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:20:42 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-28T00:20:42.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:20:42 smithi119 ceph-mon[31518]: osd.6 [v2:172.21.15.119:6818/2545410230,v1:172.21.15.119:6819/2545410230] boot 2024-03-28T00:20:42.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:20:42 smithi119 ceph-mon[31518]: osdmap e46: 7 total, 7 up, 7 in 2024-03-28T00:20:42.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:20:42 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-28T00:20:43.061 INFO:teuthology.orchestra.run.smithi132.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-28T00:20:43.061 INFO:teuthology.orchestra.run.smithi132.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-28T00:20:43.061 INFO:teuthology.orchestra.run.smithi132.stderr: stderr: 10+0 records in 2024-03-28T00:20:43.061 INFO:teuthology.orchestra.run.smithi132.stderr:10+0 records out 2024-03-28T00:20:43.061 INFO:teuthology.orchestra.run.smithi132.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.013201 s, 794 MB/s 2024-03-28T00:20:43.061 INFO:teuthology.orchestra.run.smithi132.stderr: stderr: 2024-03-28T00:20:43.062 INFO:teuthology.orchestra.run.smithi132.stderr:--> Zapping successful for: 2024-03-28T00:20:43.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:20:42 smithi165 ceph-mon[36302]: purged_snaps scrub starts 2024-03-28T00:20:43.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:20:42 smithi165 ceph-mon[36302]: purged_snaps scrub ok 2024-03-28T00:20:43.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:20:42 smithi165 ceph-mon[36302]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-28T00:20:43.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:20:42 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-28T00:20:43.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:20:42 smithi165 ceph-mon[36302]: osd.6 [v2:172.21.15.119:6818/2545410230,v1:172.21.15.119:6819/2545410230] boot 2024-03-28T00:20:43.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:20:42 smithi165 ceph-mon[36302]: osdmap e46: 7 total, 7 up, 7 in 2024-03-28T00:20:43.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:20:42 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-28T00:20:43.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:20:42 smithi132 ceph-mon[35941]: purged_snaps scrub starts 2024-03-28T00:20:43.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:20:42 smithi132 ceph-mon[35941]: purged_snaps scrub ok 2024-03-28T00:20:43.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:20:42 smithi132 ceph-mon[35941]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-28T00:20:43.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:20:42 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-28T00:20:43.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:20:42 smithi132 ceph-mon[35941]: osd.6 [v2:172.21.15.119:6818/2545410230,v1:172.21.15.119:6819/2545410230] boot 2024-03-28T00:20:43.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:20:42 smithi132 ceph-mon[35941]: osdmap e46: 7 total, 7 up, 7 in 2024-03-28T00:20:43.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:20:42 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-28T00:20:43.697 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 10f73b92-ec98-11ee-95d0-87774f69a715 -- ceph orch daemon add osd smithi132:vg_nvme/lv_2 2024-03-28T00:20:43.937 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/mon.b/config 2024-03-28T00:20:44.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:20:44 smithi119 ceph-mon[31518]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-28T00:20:44.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:20:44 smithi119 ceph-mon[31518]: osdmap e47: 7 total, 7 up, 7 in 2024-03-28T00:20:45.188 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:20:44 smithi132 ceph-mon[35941]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-28T00:20:45.188 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:20:44 smithi132 ceph-mon[35941]: osdmap e47: 7 total, 7 up, 7 in 2024-03-28T00:20:45.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:20:44 smithi165 ceph-mon[36302]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-28T00:20:45.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:20:44 smithi165 ceph-mon[36302]: osdmap e47: 7 total, 7 up, 7 in 2024-03-28T00:20:45.576 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:20:45.563+0000 7f20f0780700 1 -- 172.21.15.132:0/4046446966 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f20e81033b0 msgr2=0x7f20e8103810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:20:45.577 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:20:45.563+0000 7f20f0780700 1 --2- 172.21.15.132:0/4046446966 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f20e81033b0 0x7f20e8103810 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f20e40099c0 tx=0x7f20e4009cd0 comp rx=0 tx=0).stop 2024-03-28T00:20:45.577 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:20:45.563+0000 7f20f0780700 1 -- 172.21.15.132:0/4046446966 shutdown_connections 2024-03-28T00:20:45.577 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:20:45.563+0000 7f20f0780700 1 --2- 172.21.15.132:0/4046446966 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f20e8103d50 0x7f20e810a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:20:45.577 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:20:45.563+0000 7f20f0780700 1 --2- 172.21.15.132:0/4046446966 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f20e81021b0 0x7f20e81025d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:20:45.577 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:20:45.563+0000 7f20f0780700 1 --2- 172.21.15.132:0/4046446966 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f20e81033b0 0x7f20e8103810 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:20:45.577 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:20:45.563+0000 7f20f0780700 1 -- 172.21.15.132:0/4046446966 >> 172.21.15.132:0/4046446966 conn(0x7f20e80fd770 msgr2=0x7f20e80ffb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:20:45.577 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:20:45.564+0000 7f20f0780700 1 -- 172.21.15.132:0/4046446966 shutdown_connections 2024-03-28T00:20:45.577 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:20:45.564+0000 7f20f0780700 1 -- 172.21.15.132:0/4046446966 wait complete. 2024-03-28T00:20:45.577 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:20:45.564+0000 7f20f0780700 1 Processor -- start 2024-03-28T00:20:45.578 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:20:45.564+0000 7f20f0780700 1 -- start start 2024-03-28T00:20:45.578 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:20:45.564+0000 7f20f0780700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f20e81021b0 0x7f20e8075f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:20:45.578 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:20:45.564+0000 7f20f0780700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f20e81033b0 0x7f20e8074580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:20:45.578 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:20:45.564+0000 7f20f0780700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f20e8103d50 0x7f20e8074ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:20:45.578 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:20:45.564+0000 7f20f0780700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20e810de20 con 0x7f20e81033b0 2024-03-28T00:20:45.578 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:20:45.564+0000 7f20f0780700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20e810dca0 con 0x7f20e8103d50 2024-03-28T00:20:45.578 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:20:45.564+0000 7f20f0780700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20e810dfa0 con 0x7f20e81021b0 2024-03-28T00:20:45.578 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:20:45.564+0000 7f20eed1d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f20e8103d50 0x7f20e8074ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:20:45.578 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:20:45.564+0000 7f20eed1d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f20e8103d50 0x7f20e8074ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:51842/0 (socket says 172.21.15.132:51842) 2024-03-28T00:20:45.578 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:20:45.564+0000 7f20eed1d700 1 -- 172.21.15.132:0/478306234 learned_addr learned my addr 172.21.15.132:0/478306234 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-28T00:20:45.578 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:20:45.564+0000 7f20ee51c700 1 --2- 172.21.15.132:0/478306234 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f20e81021b0 0x7f20e8075f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:20:45.579 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:20:45.565+0000 7f20edd1b700 1 --2- 172.21.15.132:0/478306234 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f20e81033b0 0x7f20e8074580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:20:45.579 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:20:45.565+0000 7f20eed1d700 1 -- 172.21.15.132:0/478306234 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f20e81021b0 msgr2=0x7f20e8075f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:20:45.579 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:20:45.565+0000 7f20eed1d700 1 --2- 172.21.15.132:0/478306234 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f20e81021b0 0x7f20e8075f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:20:45.579 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:20:45.565+0000 7f20eed1d700 1 -- 172.21.15.132:0/478306234 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f20e81033b0 msgr2=0x7f20e8074580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:20:45.579 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:20:45.565+0000 7f20eed1d700 1 --2- 172.21.15.132:0/478306234 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f20e81033b0 0x7f20e8074580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:20:45.579 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:20:45.565+0000 7f20eed1d700 1 -- 172.21.15.132:0/478306234 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20e8075370 con 0x7f20e8103d50 2024-03-28T00:20:45.579 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:20:45.565+0000 7f20eed1d700 1 --2- 172.21.15.132:0/478306234 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f20e8103d50 0x7f20e8074ac0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f20e0007d70 tx=0x7f20e000a340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:20:45.579 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:20:45.565+0000 7f20df7fe700 1 -- 172.21.15.132:0/478306234 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20e0016020 con 0x7f20e8103d50 2024-03-28T00:20:45.579 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:20:45.565+0000 7f20df7fe700 1 -- 172.21.15.132:0/478306234 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f20e000ad40 con 0x7f20e8103d50 2024-03-28T00:20:45.579 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:20:45.565+0000 7f20df7fe700 1 -- 172.21.15.132:0/478306234 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20e0011600 con 0x7f20e8103d50 2024-03-28T00:20:45.579 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:20:45.565+0000 7f20f0780700 1 -- 172.21.15.132:0/478306234 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20e81a2560 con 0x7f20e8103d50 2024-03-28T00:20:45.579 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:20:45.565+0000 7f20f0780700 1 -- 172.21.15.132:0/478306234 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20e81a2af0 con 0x7f20e8103d50 2024-03-28T00:20:45.580 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:20:45.566+0000 7f20f0780700 1 -- 172.21.15.132:0/478306234 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20e8068e50 con 0x7f20e8103d50 2024-03-28T00:20:45.580 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:20:45.567+0000 7f20df7fe700 1 -- 172.21.15.132:0/478306234 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f20e0019020 con 0x7f20e8103d50 2024-03-28T00:20:45.580 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:20:45.568+0000 7f20df7fe700 1 --2- 172.21.15.132:0/478306234 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f20d40a2910 0x7f20d40a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:20:45.580 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:20:45.568+0000 7f20df7fe700 1 -- 172.21.15.132:0/478306234 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 4965+0+0 (secure 0 0 0) 0x7f20e00cdc90 con 0x7f20e8103d50 2024-03-28T00:20:45.580 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:20:45.568+0000 7f20ee51c700 1 --2- 172.21.15.132:0/478306234 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f20d40a2910 0x7f20d40a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:20:45.580 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:20:45.569+0000 7f20ee51c700 1 --2- 172.21.15.132:0/478306234 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f20d40a2910 0x7f20d40a4dd0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f20e8103210 tx=0x7f20d8009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:20:45.580 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:20:45.572+0000 7f20df7fe700 1 -- 172.21.15.132:0/478306234 <== mon.2 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f20e0020020 con 0x7f20e8103d50 2024-03-28T00:20:45.784 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:20:45.779+0000 7f20f0780700 1 -- 172.21.15.132:0/478306234 --> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi132:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f20e80625d0 con 0x7f20d40a2910 2024-03-28T00:20:47.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:20:46 smithi165 ceph-mon[36302]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-28T00:20:47.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:20:46 smithi165 ceph-mon[36302]: Detected new or changed devices on smithi119 2024-03-28T00:20:47.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:20:46 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:20:47.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:20:46 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:20:47.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:20:46 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:20:47.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:20:46 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:20:47.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:20:46 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:20:47.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:20:46 smithi165 ceph-mon[36302]: Adjusting osd_memory_target on smithi119 to 5697M 2024-03-28T00:20:47.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:20:46 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:20:47.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:20:46 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:20:47.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:20:46 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:20:47.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:20:46 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:20:47.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:20:46 smithi165 ceph-mon[36302]: from='client.24284 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi132:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T00:20:47.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:20:46 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-28T00:20:47.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:20:46 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-28T00:20:47.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:20:46 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:20:47.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:20:46 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:20:47.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:20:46 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:20:47.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:20:46 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:20:47.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:20:46 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:20:47.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:20:46 smithi119 ceph-mon[31518]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-28T00:20:47.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:20:46 smithi119 ceph-mon[31518]: Detected new or changed devices on smithi119 2024-03-28T00:20:47.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:20:46 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:20:47.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:20:46 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:20:47.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:20:46 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:20:47.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:20:46 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:20:47.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:20:46 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:20:47.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:20:46 smithi119 ceph-mon[31518]: Adjusting osd_memory_target on smithi119 to 5697M 2024-03-28T00:20:47.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:20:46 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:20:47.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:20:46 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:20:47.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:20:46 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:20:47.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:20:46 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:20:47.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:20:46 smithi119 ceph-mon[31518]: from='client.24284 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi132:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T00:20:47.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:20:46 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-28T00:20:47.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:20:46 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-28T00:20:47.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:20:46 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:20:47.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:20:46 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:20:47.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:20:46 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:20:47.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:20:46 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:20:47.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:20:46 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:20:47.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:20:46 smithi132 ceph-mon[35941]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-28T00:20:47.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:20:46 smithi132 ceph-mon[35941]: Detected new or changed devices on smithi119 2024-03-28T00:20:47.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:20:46 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:20:47.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:20:46 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:20:47.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:20:46 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:20:47.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:20:46 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:20:47.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:20:46 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:20:47.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:20:46 smithi132 ceph-mon[35941]: Adjusting osd_memory_target on smithi119 to 5697M 2024-03-28T00:20:47.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:20:46 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:20:47.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:20:46 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:20:47.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:20:46 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:20:47.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:20:46 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:20:47.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:20:46 smithi132 ceph-mon[35941]: from='client.24284 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi132:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T00:20:47.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:20:46 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-28T00:20:47.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:20:46 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-28T00:20:47.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:20:46 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:20:47.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:20:46 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:20:47.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:20:46 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:20:47.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:20:46 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:20:47.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:20:46 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:20:49.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:20:48 smithi165 ceph-mon[36302]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-28T00:20:49.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:20:48 smithi165 ceph-mon[36302]: from='client.? 172.21.15.132:0/1713976359' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "79b97a12-f6b4-48c3-b517-2dd6b6e7c85f"}]: dispatch 2024-03-28T00:20:49.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:20:48 smithi165 ceph-mon[36302]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "79b97a12-f6b4-48c3-b517-2dd6b6e7c85f"}]: dispatch 2024-03-28T00:20:49.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:20:48 smithi165 ceph-mon[36302]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "79b97a12-f6b4-48c3-b517-2dd6b6e7c85f"}]': finished 2024-03-28T00:20:49.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:20:48 smithi165 ceph-mon[36302]: osdmap e48: 8 total, 7 up, 8 in 2024-03-28T00:20:49.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:20:48 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-28T00:20:49.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:20:48 smithi119 ceph-mon[31518]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-28T00:20:49.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:20:48 smithi119 ceph-mon[31518]: from='client.? 172.21.15.132:0/1713976359' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "79b97a12-f6b4-48c3-b517-2dd6b6e7c85f"}]: dispatch 2024-03-28T00:20:49.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:20:48 smithi119 ceph-mon[31518]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "79b97a12-f6b4-48c3-b517-2dd6b6e7c85f"}]: dispatch 2024-03-28T00:20:49.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:20:48 smithi119 ceph-mon[31518]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "79b97a12-f6b4-48c3-b517-2dd6b6e7c85f"}]': finished 2024-03-28T00:20:49.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:20:48 smithi119 ceph-mon[31518]: osdmap e48: 8 total, 7 up, 8 in 2024-03-28T00:20:49.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:20:48 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-28T00:20:49.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:20:48 smithi132 ceph-mon[35941]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-28T00:20:49.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:20:48 smithi132 ceph-mon[35941]: from='client.? 172.21.15.132:0/1713976359' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "79b97a12-f6b4-48c3-b517-2dd6b6e7c85f"}]: dispatch 2024-03-28T00:20:49.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:20:48 smithi132 ceph-mon[35941]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "79b97a12-f6b4-48c3-b517-2dd6b6e7c85f"}]: dispatch 2024-03-28T00:20:49.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:20:48 smithi132 ceph-mon[35941]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "79b97a12-f6b4-48c3-b517-2dd6b6e7c85f"}]': finished 2024-03-28T00:20:49.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:20:48 smithi132 ceph-mon[35941]: osdmap e48: 8 total, 7 up, 8 in 2024-03-28T00:20:49.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:20:48 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-28T00:20:50.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:20:49 smithi165 ceph-mon[36302]: from='client.? 172.21.15.132:0/3582097917' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-28T00:20:50.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:20:49 smithi119 ceph-mon[31518]: from='client.? 172.21.15.132:0/3582097917' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-28T00:20:50.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:20:49 smithi132 ceph-mon[35941]: from='client.? 172.21.15.132:0/3582097917' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-28T00:20:51.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:20:50 smithi165 ceph-mon[36302]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-28T00:20:51.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:20:50 smithi119 ceph-mon[31518]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-28T00:20:51.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:20:50 smithi132 ceph-mon[35941]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-28T00:20:53.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:20:52 smithi165 ceph-mon[36302]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-28T00:20:53.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:20:52 smithi119 ceph-mon[31518]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-28T00:20:53.191 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:20:52 smithi132 ceph-mon[35941]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-28T00:20:55.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:20:54 smithi165 ceph-mon[36302]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-28T00:20:55.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:20:54 smithi119 ceph-mon[31518]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-28T00:20:55.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:20:54 smithi132 ceph-mon[35941]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-28T00:20:56.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:20:55 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-28T00:20:56.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:20:55 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:20:56.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:20:55 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-28T00:20:56.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:20:55 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:20:56.165 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:20:55 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-28T00:20:56.165 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:20:55 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:20:57.080 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:20:56 smithi132 ceph-mon[35941]: Deploying daemon osd.7 on smithi132 2024-03-28T00:20:57.080 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:20:56 smithi132 ceph-mon[35941]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-28T00:20:57.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:20:56 smithi165 ceph-mon[36302]: Deploying daemon osd.7 on smithi132 2024-03-28T00:20:57.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:20:56 smithi165 ceph-mon[36302]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-28T00:20:57.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:20:56 smithi119 ceph-mon[31518]: Deploying daemon osd.7 on smithi132 2024-03-28T00:20:57.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:20:56 smithi119 ceph-mon[31518]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-28T00:20:58.914 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:20:58 smithi132 ceph-mon[35941]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-28T00:20:59.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:20:58 smithi165 ceph-mon[36302]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-28T00:20:59.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:20:58 smithi119 ceph-mon[31518]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-28T00:21:01.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:21:00 smithi165 ceph-mon[36302]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-28T00:21:01.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:21:00 smithi119 ceph-mon[31518]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-28T00:21:01.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:21:00 smithi132 ceph-mon[35941]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-28T00:21:02.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:21:01 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:21:02.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:21:01 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:21:02.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:21:01 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:21:02.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:21:01 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:21:02.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:21:01 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:21:02.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:21:01 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:21:02.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:21:01 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:21:02.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:21:01 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:21:02.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:21:01 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:21:03.138 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:21:02 smithi132 ceph-mon[35941]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-28T00:21:03.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:21:02 smithi165 ceph-mon[36302]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-28T00:21:03.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:21:02 smithi119 ceph-mon[31518]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-28T00:21:04.319 INFO:teuthology.orchestra.run.smithi132.stdout:Created osd(s) 7 on host 'smithi132' 2024-03-28T00:21:04.319 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:21:04.313+0000 7f20df7fe700 1 -- 172.21.15.132:0/478306234 <== mgr.14152 v2:172.21.15.119:6800/1401937096 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f20e80625d0 con 0x7f20d40a2910 2024-03-28T00:21:04.322 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:21:04.316+0000 7f20f0780700 1 -- 172.21.15.132:0/478306234 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f20d40a2910 msgr2=0x7f20d40a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:21:04.322 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:21:04.316+0000 7f20f0780700 1 --2- 172.21.15.132:0/478306234 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f20d40a2910 0x7f20d40a4dd0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f20e8103210 tx=0x7f20d8009190 comp rx=0 tx=0).stop 2024-03-28T00:21:04.322 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:21:04.316+0000 7f20f0780700 1 -- 172.21.15.132:0/478306234 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f20e8103d50 msgr2=0x7f20e8074ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:21:04.322 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:21:04.316+0000 7f20f0780700 1 --2- 172.21.15.132:0/478306234 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f20e8103d50 0x7f20e8074ac0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f20e0007d70 tx=0x7f20e000a340 comp rx=0 tx=0).stop 2024-03-28T00:21:04.322 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:21:04.316+0000 7f20f0780700 1 -- 172.21.15.132:0/478306234 shutdown_connections 2024-03-28T00:21:04.322 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:21:04.316+0000 7f20f0780700 1 --2- 172.21.15.132:0/478306234 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f20d40a2910 0x7f20d40a4dd0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:21:04.322 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:21:04.316+0000 7f20f0780700 1 --2- 172.21.15.132:0/478306234 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f20e8103d50 0x7f20e8074ac0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:21:04.322 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:21:04.316+0000 7f20f0780700 1 --2- 172.21.15.132:0/478306234 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f20e81021b0 0x7f20e8075f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:21:04.322 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:21:04.316+0000 7f20f0780700 1 --2- 172.21.15.132:0/478306234 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f20e81033b0 0x7f20e8074580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:21:04.323 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:21:04.316+0000 7f20f0780700 1 -- 172.21.15.132:0/478306234 >> 172.21.15.132:0/478306234 conn(0x7f20e80fd770 msgr2=0x7f20e81066b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:21:04.323 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:21:04.316+0000 7f20f0780700 1 -- 172.21.15.132:0/478306234 shutdown_connections 2024-03-28T00:21:04.323 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:21:04.316+0000 7f20f0780700 1 -- 172.21.15.132:0/478306234 wait complete. 2024-03-28T00:21:04.946 DEBUG:teuthology.orchestra.run.smithi132:osd.7> sudo journalctl -f -n 0 -u ceph-10f73b92-ec98-11ee-95d0-87774f69a715@osd.7.service 2024-03-28T00:21:04.949 INFO:tasks.cephadm:Deploying osd.8 on smithi165 with /dev/vg_nvme/lv_2... 2024-03-28T00:21:04.949 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 10f73b92-ec98-11ee-95d0-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-28T00:21:05.198 INFO:teuthology.orchestra.run.smithi165.stderr:Inferring config /var/lib/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/mon.c/config 2024-03-28T00:21:05.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:21:04 smithi132 ceph-mon[35941]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-28T00:21:05.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:21:04 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:21:05.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:21:04 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:21:05.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:21:04 smithi132 ceph-mon[35941]: from='osd.7 [v2:172.21.15.132:6816/2877892376,v1:172.21.15.132:6817/2877892376]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-28T00:21:05.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:21:04 smithi132 ceph-mon[35941]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-28T00:21:05.320 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:21:04 smithi165 ceph-mon[36302]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-28T00:21:05.320 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:21:04 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:21:05.320 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:21:04 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:21:05.320 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:21:04 smithi165 ceph-mon[36302]: from='osd.7 [v2:172.21.15.132:6816/2877892376,v1:172.21.15.132:6817/2877892376]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-28T00:21:05.320 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:21:04 smithi165 ceph-mon[36302]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-28T00:21:05.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:21:04 smithi119 ceph-mon[31518]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-28T00:21:05.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:21:04 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:21:05.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:21:04 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:21:05.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:21:04 smithi119 ceph-mon[31518]: from='osd.7 [v2:172.21.15.132:6816/2877892376,v1:172.21.15.132:6817/2877892376]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-28T00:21:05.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:21:04 smithi119 ceph-mon[31518]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-28T00:21:06.362 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:21:06 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:21:06.362 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:21:06 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:21:06.362 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:21:06 smithi165 ceph-mon[36302]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-28T00:21:06.362 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:21:06 smithi165 ceph-mon[36302]: from='osd.7 [v2:172.21.15.132:6816/2877892376,v1:172.21.15.132:6817/2877892376]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]: dispatch 2024-03-28T00:21:06.363 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:21:06 smithi165 ceph-mon[36302]: osdmap e49: 8 total, 7 up, 8 in 2024-03-28T00:21:06.363 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:21:06 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-28T00:21:06.363 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:21:06 smithi165 ceph-mon[36302]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]: dispatch 2024-03-28T00:21:06.363 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:21:06 smithi165 ceph-mon[36302]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-28T00:21:06.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:21:06 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:21:06.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:21:06 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:21:06.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:21:06 smithi119 ceph-mon[31518]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-28T00:21:06.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:21:06 smithi119 ceph-mon[31518]: from='osd.7 [v2:172.21.15.132:6816/2877892376,v1:172.21.15.132:6817/2877892376]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]: dispatch 2024-03-28T00:21:06.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:21:06 smithi119 ceph-mon[31518]: osdmap e49: 8 total, 7 up, 8 in 2024-03-28T00:21:06.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:21:06 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-28T00:21:06.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:21:06 smithi119 ceph-mon[31518]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]: dispatch 2024-03-28T00:21:06.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:21:06 smithi119 ceph-mon[31518]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-28T00:21:06.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:21:06 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:21:06.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:21:06 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:21:06.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:21:06 smithi132 ceph-mon[35941]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-28T00:21:06.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:21:06 smithi132 ceph-mon[35941]: from='osd.7 [v2:172.21.15.132:6816/2877892376,v1:172.21.15.132:6817/2877892376]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]: dispatch 2024-03-28T00:21:06.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:21:06 smithi132 ceph-mon[35941]: osdmap e49: 8 total, 7 up, 8 in 2024-03-28T00:21:06.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:21:06 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-28T00:21:06.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:21:06 smithi132 ceph-mon[35941]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]: dispatch 2024-03-28T00:21:06.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:21:06 smithi132 ceph-mon[35941]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-28T00:21:06.991 INFO:journalctl@ceph.osd.7.smithi132.stdout:Mar 28 00:21:06 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-7[54273]: 2024-03-28T00:21:06.706+0000 7f76012c3700 -1 osd.7 0 waiting for initial osdmap 2024-03-28T00:21:06.992 INFO:journalctl@ceph.osd.7.smithi132.stdout:Mar 28 00:21:06 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-7[54273]: 2024-03-28T00:21:06.717+0000 7f75f90ae700 -1 osd.7 50 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-28T00:21:07.138 INFO:teuthology.orchestra.run.smithi165.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-28T00:21:07.139 INFO:teuthology.orchestra.run.smithi165.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-28T00:21:07.139 INFO:teuthology.orchestra.run.smithi165.stderr: stderr: 10+0 records in 2024-03-28T00:21:07.139 INFO:teuthology.orchestra.run.smithi165.stderr:10+0 records out 2024-03-28T00:21:07.139 INFO:teuthology.orchestra.run.smithi165.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0133804 s, 784 MB/s 2024-03-28T00:21:07.140 INFO:teuthology.orchestra.run.smithi165.stderr:--> Zapping successful for: 2024-03-28T00:21:07.437 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:21:07 smithi165 ceph-mon[36302]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]': finished 2024-03-28T00:21:07.437 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:21:07 smithi165 ceph-mon[36302]: osdmap e50: 8 total, 7 up, 8 in 2024-03-28T00:21:07.437 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:21:07 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-28T00:21:07.437 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:21:07 smithi165 ceph-mon[36302]: from='osd.7 ' entity='osd.7' 2024-03-28T00:21:07.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:21:07 smithi119 ceph-mon[31518]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]': finished 2024-03-28T00:21:07.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:21:07 smithi119 ceph-mon[31518]: osdmap e50: 8 total, 7 up, 8 in 2024-03-28T00:21:07.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:21:07 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-28T00:21:07.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:21:07 smithi119 ceph-mon[31518]: from='osd.7 ' entity='osd.7' 2024-03-28T00:21:07.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:21:07 smithi132 ceph-mon[35941]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]': finished 2024-03-28T00:21:07.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:21:07 smithi132 ceph-mon[35941]: osdmap e50: 8 total, 7 up, 8 in 2024-03-28T00:21:07.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:21:07 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-28T00:21:07.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:21:07 smithi132 ceph-mon[35941]: from='osd.7 ' entity='osd.7' 2024-03-28T00:21:07.775 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 10f73b92-ec98-11ee-95d0-87774f69a715 -- ceph orch daemon add osd smithi165:vg_nvme/lv_2 2024-03-28T00:21:08.018 INFO:teuthology.orchestra.run.smithi165.stderr:Inferring config /var/lib/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/mon.c/config 2024-03-28T00:21:08.398 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:21:08 smithi165 ceph-mon[36302]: purged_snaps scrub starts 2024-03-28T00:21:08.398 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:21:08 smithi165 ceph-mon[36302]: purged_snaps scrub ok 2024-03-28T00:21:08.398 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:21:08 smithi165 ceph-mon[36302]: osd.7 [v2:172.21.15.132:6816/2877892376,v1:172.21.15.132:6817/2877892376] boot 2024-03-28T00:21:08.398 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:21:08 smithi165 ceph-mon[36302]: osdmap e51: 8 total, 8 up, 8 in 2024-03-28T00:21:08.398 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:21:08 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-28T00:21:08.398 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:21:08 smithi165 ceph-mon[36302]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-03-28T00:21:08.452 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:21:08 smithi132 ceph-mon[35941]: purged_snaps scrub starts 2024-03-28T00:21:08.452 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:21:08 smithi132 ceph-mon[35941]: purged_snaps scrub ok 2024-03-28T00:21:08.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:21:08 smithi132 ceph-mon[35941]: osd.7 [v2:172.21.15.132:6816/2877892376,v1:172.21.15.132:6817/2877892376] boot 2024-03-28T00:21:08.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:21:08 smithi132 ceph-mon[35941]: osdmap e51: 8 total, 8 up, 8 in 2024-03-28T00:21:08.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:21:08 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-28T00:21:08.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:21:08 smithi132 ceph-mon[35941]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-03-28T00:21:08.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:21:08 smithi119 ceph-mon[31518]: purged_snaps scrub starts 2024-03-28T00:21:08.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:21:08 smithi119 ceph-mon[31518]: purged_snaps scrub ok 2024-03-28T00:21:08.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:21:08 smithi119 ceph-mon[31518]: osd.7 [v2:172.21.15.132:6816/2877892376,v1:172.21.15.132:6817/2877892376] boot 2024-03-28T00:21:08.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:21:08 smithi119 ceph-mon[31518]: osdmap e51: 8 total, 8 up, 8 in 2024-03-28T00:21:08.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:21:08 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-28T00:21:08.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:21:08 smithi119 ceph-mon[31518]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-03-28T00:21:09.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:21:09 smithi165 ceph-mon[36302]: osdmap e52: 8 total, 8 up, 8 in 2024-03-28T00:21:09.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:21:09 smithi165 ceph-mon[36302]: Detected new or changed devices on smithi132 2024-03-28T00:21:09.480 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:21:09 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:21:09.480 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:21:09 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:21:09.480 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:21:09 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:21:09.480 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:21:09 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:21:09.480 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:21:09 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:21:09.480 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:21:09 smithi165 ceph-mon[36302]: Adjusting osd_memory_target on smithi132 to 5693M 2024-03-28T00:21:09.480 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:21:09 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:21:09.480 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:21:09 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:21:09.480 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:21:09 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:21:09.480 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:21:09 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:21:09.480 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:21:09 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:21:09.480 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:21:09 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:21:09.480 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:21:09 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:21:09.481 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:21:09 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:21:09.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:21:09 smithi119 ceph-mon[31518]: osdmap e52: 8 total, 8 up, 8 in 2024-03-28T00:21:09.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:21:09 smithi119 ceph-mon[31518]: Detected new or changed devices on smithi132 2024-03-28T00:21:09.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:21:09 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:21:09.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:21:09 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:21:09.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:21:09 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:21:09.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:21:09 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:21:09.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:21:09 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:21:09.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:21:09 smithi119 ceph-mon[31518]: Adjusting osd_memory_target on smithi132 to 5693M 2024-03-28T00:21:09.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:21:09 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:21:09.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:21:09 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:21:09.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:21:09 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:21:09.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:21:09 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:21:09.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:21:09 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:21:09.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:21:09 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:21:09.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:21:09 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:21:09.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:21:09 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:21:09.693 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:21:09.686+0000 7fb622728700 1 -- 172.21.15.165:0/3754796144 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb61c1022c0 msgr2=0x7fb61c102720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:21:09.694 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:21:09.686+0000 7fb622728700 1 --2- 172.21.15.165:0/3754796144 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb61c1022c0 0x7fb61c102720 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fb610009990 tx=0x7fb610009ca0 comp rx=0 tx=0).stop 2024-03-28T00:21:09.694 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:21:09.686+0000 7fb622728700 1 -- 172.21.15.165:0/3754796144 shutdown_connections 2024-03-28T00:21:09.694 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:21:09.686+0000 7fb622728700 1 --2- 172.21.15.165:0/3754796144 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb61c102c60 0x7fb61c109510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:21:09.694 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:21:09.686+0000 7fb622728700 1 --2- 172.21.15.165:0/3754796144 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb61c1010c0 0x7fb61c1014e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:21:09.694 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:21:09.686+0000 7fb622728700 1 --2- 172.21.15.165:0/3754796144 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb61c1022c0 0x7fb61c102720 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:21:09.694 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:21:09.686+0000 7fb622728700 1 -- 172.21.15.165:0/3754796144 >> 172.21.15.165:0/3754796144 conn(0x7fb61c0fc660 msgr2=0x7fb61c0feaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:21:09.694 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:21:09.686+0000 7fb622728700 1 -- 172.21.15.165:0/3754796144 shutdown_connections 2024-03-28T00:21:09.694 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:21:09.686+0000 7fb622728700 1 -- 172.21.15.165:0/3754796144 wait complete. 2024-03-28T00:21:09.694 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:21:09.687+0000 7fb622728700 1 Processor -- start 2024-03-28T00:21:09.694 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:21:09.687+0000 7fb622728700 1 -- start start 2024-03-28T00:21:09.694 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:21:09.687+0000 7fb622728700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb61c1010c0 0x7fb61c06a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:21:09.694 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:21:09.687+0000 7fb622728700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb61c1022c0 0x7fb61c19d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:21:09.695 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:21:09.687+0000 7fb622728700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb61c102c60 0x7fb61c19f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:21:09.695 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:21:09.687+0000 7fb622728700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb61c10cd50 con 0x7fb61c1010c0 2024-03-28T00:21:09.695 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:21:09.687+0000 7fb622728700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb61c10cbd0 con 0x7fb61c1022c0 2024-03-28T00:21:09.695 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:21:09.687+0000 7fb622728700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb61c10ced0 con 0x7fb61c102c60 2024-03-28T00:21:09.695 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:21:09.687+0000 7fb620cc5700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb61c102c60 0x7fb61c19f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:21:09.695 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:21:09.687+0000 7fb620cc5700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb61c102c60 0x7fb61c19f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:52756/0 (socket says 172.21.15.165:52756) 2024-03-28T00:21:09.695 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:21:09.687+0000 7fb620cc5700 1 -- 172.21.15.165:0/3978091365 learned_addr learned my addr 172.21.15.165:0/3978091365 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-03-28T00:21:09.695 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:21:09.687+0000 7fb61b7fe700 1 --2- 172.21.15.165:0/3978091365 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb61c1022c0 0x7fb61c19d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:21:09.695 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:21:09.687+0000 7fb620cc5700 1 -- 172.21.15.165:0/3978091365 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb61c1022c0 msgr2=0x7fb61c19d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:21:09.695 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:21:09.687+0000 7fb620cc5700 1 --2- 172.21.15.165:0/3978091365 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb61c1022c0 0x7fb61c19d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:21:09.695 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:21:09.687+0000 7fb620cc5700 1 -- 172.21.15.165:0/3978091365 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb61c1010c0 msgr2=0x7fb61c06a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:21:09.695 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:21:09.687+0000 7fb61bfff700 1 --2- 172.21.15.165:0/3978091365 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb61c1010c0 0x7fb61c06a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:21:09.695 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:21:09.687+0000 7fb620cc5700 1 --2- 172.21.15.165:0/3978091365 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb61c1010c0 0x7fb61c06a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:21:09.695 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:21:09.687+0000 7fb620cc5700 1 -- 172.21.15.165:0/3978091365 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb61c1a0100 con 0x7fb61c102c60 2024-03-28T00:21:09.695 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:21:09.688+0000 7fb620cc5700 1 --2- 172.21.15.165:0/3978091365 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb61c102c60 0x7fb61c19f830 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fb61400ef30 tx=0x7fb61400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:21:09.696 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:21:09.688+0000 7fb6197fa700 1 -- 172.21.15.165:0/3978091365 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6140092e0 con 0x7fb61c102c60 2024-03-28T00:21:09.696 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:21:09.688+0000 7fb622728700 1 -- 172.21.15.165:0/3978091365 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb61c1a0390 con 0x7fb61c102c60 2024-03-28T00:21:09.696 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:21:09.688+0000 7fb6197fa700 1 -- 172.21.15.165:0/3978091365 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb614004bb0 con 0x7fb61c102c60 2024-03-28T00:21:09.696 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:21:09.688+0000 7fb6197fa700 1 -- 172.21.15.165:0/3978091365 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb614004280 con 0x7fb61c102c60 2024-03-28T00:21:09.696 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:21:09.688+0000 7fb622728700 1 -- 172.21.15.165:0/3978091365 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb61c1a0950 con 0x7fb61c102c60 2024-03-28T00:21:09.697 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:21:09.689+0000 7fb622728700 1 -- 172.21.15.165:0/3978091365 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb61c068e50 con 0x7fb61c102c60 2024-03-28T00:21:09.697 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:21:09.690+0000 7fb6197fa700 1 -- 172.21.15.165:0/3978091365 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fb61401b030 con 0x7fb61c102c60 2024-03-28T00:21:09.697 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:21:09.691+0000 7fb6197fa700 1 --2- 172.21.15.165:0/3978091365 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fb6040a2890 0x7fb6040a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:21:09.697 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:21:09.691+0000 7fb6197fa700 1 -- 172.21.15.165:0/3978091365 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 5426+0+0 (secure 0 0 0) 0x7fb6140ce470 con 0x7fb61c102c60 2024-03-28T00:21:09.697 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:21:09.691+0000 7fb61bfff700 1 --2- 172.21.15.165:0/3978091365 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fb6040a2890 0x7fb6040a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:21:09.697 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:21:09.692+0000 7fb61bfff700 1 --2- 172.21.15.165:0/3978091365 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fb6040a2890 0x7fb6040a4d50 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb60c006fd0 tx=0x7fb60c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:21:09.700 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:21:09.694+0000 7fb6197fa700 1 -- 172.21.15.165:0/3978091365 <== mon.1 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fb614022020 con 0x7fb61c102c60 2024-03-28T00:21:09.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:21:09 smithi132 ceph-mon[35941]: osdmap e52: 8 total, 8 up, 8 in 2024-03-28T00:21:09.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:21:09 smithi132 ceph-mon[35941]: Detected new or changed devices on smithi132 2024-03-28T00:21:09.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:21:09 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:21:09.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:21:09 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:21:09.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:21:09 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:21:09.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:21:09 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:21:09.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:21:09 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:21:09.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:21:09 smithi132 ceph-mon[35941]: Adjusting osd_memory_target on smithi132 to 5693M 2024-03-28T00:21:09.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:21:09 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:21:09.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:21:09 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:21:09.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:21:09 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:21:09.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:21:09 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:21:09.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:21:09 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:21:09.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:21:09 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:21:09.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:21:09 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:21:09.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:21:09 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:21:09.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:21:09.901+0000 7fb622728700 1 -- 172.21.15.165:0/3978091365 --> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi165:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7fb61c0625d0 con 0x7fb6040a2890 2024-03-28T00:21:10.624 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:21:10 smithi165 ceph-mon[36302]: osdmap e53: 8 total, 8 up, 8 in 2024-03-28T00:21:10.624 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:21:10 smithi165 ceph-mon[36302]: pgmap v196: 1 pgs: 1 remapped+peering; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-03-28T00:21:10.624 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:21:10 smithi165 ceph-mon[36302]: from='client.24307 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi165:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T00:21:10.624 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:21:10 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-28T00:21:10.624 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:21:10 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-28T00:21:10.624 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:21:10 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:21:10.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:21:10 smithi119 ceph-mon[31518]: osdmap e53: 8 total, 8 up, 8 in 2024-03-28T00:21:10.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:21:10 smithi119 ceph-mon[31518]: pgmap v196: 1 pgs: 1 remapped+peering; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-03-28T00:21:10.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:21:10 smithi119 ceph-mon[31518]: from='client.24307 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi165:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T00:21:10.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:21:10 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-28T00:21:10.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:21:10 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-28T00:21:10.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:21:10 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:21:10.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:21:10 smithi132 ceph-mon[35941]: osdmap e53: 8 total, 8 up, 8 in 2024-03-28T00:21:10.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:21:10 smithi132 ceph-mon[35941]: pgmap v196: 1 pgs: 1 remapped+peering; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-03-28T00:21:10.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:21:10 smithi132 ceph-mon[35941]: from='client.24307 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi165:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T00:21:10.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:21:10 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-28T00:21:10.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:21:10 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-28T00:21:10.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:21:10 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:21:11.596 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:21:11 smithi165 ceph-mon[36302]: Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-03-28T00:21:11.596 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:21:11 smithi165 ceph-mon[36302]: osdmap e54: 8 total, 8 up, 8 in 2024-03-28T00:21:11.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:21:11 smithi119 ceph-mon[31518]: Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-03-28T00:21:11.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:21:11 smithi119 ceph-mon[31518]: osdmap e54: 8 total, 8 up, 8 in 2024-03-28T00:21:11.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:21:11 smithi132 ceph-mon[35941]: Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-03-28T00:21:11.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:21:11 smithi132 ceph-mon[35941]: osdmap e54: 8 total, 8 up, 8 in 2024-03-28T00:21:12.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:21:12 smithi165 ceph-mon[36302]: osdmap e55: 8 total, 8 up, 8 in 2024-03-28T00:21:12.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:21:12 smithi165 ceph-mon[36302]: pgmap v199: 1 pgs: 1 remapped+peering; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-03-28T00:21:12.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:21:12 smithi119 ceph-mon[31518]: osdmap e55: 8 total, 8 up, 8 in 2024-03-28T00:21:12.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:21:12 smithi119 ceph-mon[31518]: pgmap v199: 1 pgs: 1 remapped+peering; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-03-28T00:21:12.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:21:12 smithi132 ceph-mon[35941]: osdmap e55: 8 total, 8 up, 8 in 2024-03-28T00:21:12.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:21:12 smithi132 ceph-mon[35941]: pgmap v199: 1 pgs: 1 remapped+peering; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-03-28T00:21:13.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:21:13 smithi165 ceph-mon[36302]: from='client.? 172.21.15.165:0/4144827844' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2970524c-b99a-412d-85c2-6c29b54035e0"}]: dispatch 2024-03-28T00:21:13.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:21:13 smithi165 ceph-mon[36302]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2970524c-b99a-412d-85c2-6c29b54035e0"}]: dispatch 2024-03-28T00:21:13.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:21:13 smithi165 ceph-mon[36302]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2970524c-b99a-412d-85c2-6c29b54035e0"}]': finished 2024-03-28T00:21:13.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:21:13 smithi165 ceph-mon[36302]: osdmap e56: 9 total, 8 up, 9 in 2024-03-28T00:21:13.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:21:13 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-28T00:21:13.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:21:13 smithi119 ceph-mon[31518]: from='client.? 172.21.15.165:0/4144827844' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2970524c-b99a-412d-85c2-6c29b54035e0"}]: dispatch 2024-03-28T00:21:13.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:21:13 smithi119 ceph-mon[31518]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2970524c-b99a-412d-85c2-6c29b54035e0"}]: dispatch 2024-03-28T00:21:13.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:21:13 smithi119 ceph-mon[31518]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2970524c-b99a-412d-85c2-6c29b54035e0"}]': finished 2024-03-28T00:21:13.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:21:13 smithi119 ceph-mon[31518]: osdmap e56: 9 total, 8 up, 9 in 2024-03-28T00:21:13.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:21:13 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-28T00:21:13.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:21:13 smithi132 ceph-mon[35941]: from='client.? 172.21.15.165:0/4144827844' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2970524c-b99a-412d-85c2-6c29b54035e0"}]: dispatch 2024-03-28T00:21:13.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:21:13 smithi132 ceph-mon[35941]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2970524c-b99a-412d-85c2-6c29b54035e0"}]: dispatch 2024-03-28T00:21:13.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:21:13 smithi132 ceph-mon[35941]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2970524c-b99a-412d-85c2-6c29b54035e0"}]': finished 2024-03-28T00:21:13.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:21:13 smithi132 ceph-mon[35941]: osdmap e56: 9 total, 8 up, 9 in 2024-03-28T00:21:13.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:21:13 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-28T00:21:14.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:21:14 smithi119 ceph-mon[31518]: from='client.? 172.21.15.165:0/3209871794' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-28T00:21:14.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:21:14 smithi119 ceph-mon[31518]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-03-28T00:21:14.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:21:14 smithi165 ceph-mon[36302]: from='client.? 172.21.15.165:0/3209871794' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-28T00:21:14.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:21:14 smithi165 ceph-mon[36302]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-03-28T00:21:14.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:21:14 smithi132 ceph-mon[35941]: from='client.? 172.21.15.165:0/3209871794' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-28T00:21:14.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:21:14 smithi132 ceph-mon[35941]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-03-28T00:21:15.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:21:15 smithi165 ceph-mon[36302]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-03-28T00:21:15.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:21:15 smithi165 ceph-mon[36302]: Cluster is now healthy 2024-03-28T00:21:15.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:21:15 smithi119 ceph-mon[31518]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-03-28T00:21:15.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:21:15 smithi119 ceph-mon[31518]: Cluster is now healthy 2024-03-28T00:21:15.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:21:15 smithi132 ceph-mon[35941]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-03-28T00:21:15.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:21:15 smithi132 ceph-mon[35941]: Cluster is now healthy 2024-03-28T00:21:16.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:21:16 smithi165 ceph-mon[36302]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-03-28T00:21:16.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:21:16 smithi119 ceph-mon[31518]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-03-28T00:21:16.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:21:16 smithi132 ceph-mon[35941]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-03-28T00:21:18.820 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:21:18 smithi165 ceph-mon[36302]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-03-28T00:21:19.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:21:18 smithi119 ceph-mon[31518]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-03-28T00:21:19.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:21:18 smithi132 ceph-mon[35941]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-03-28T00:21:21.070 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:21:20 smithi165 ceph-mon[36302]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-03-28T00:21:21.071 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:21:20 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-03-28T00:21:21.071 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:21:20 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:21:21.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:21:20 smithi119 ceph-mon[31518]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-03-28T00:21:21.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:21:20 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-03-28T00:21:21.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:21:20 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:21:21.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:21:20 smithi132 ceph-mon[35941]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-03-28T00:21:21.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:21:20 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-03-28T00:21:21.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:21:20 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:21:21.849 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:21:21 smithi165 ceph-mon[36302]: Deploying daemon osd.8 on smithi165 2024-03-28T00:21:22.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:21:21 smithi119 ceph-mon[31518]: Deploying daemon osd.8 on smithi165 2024-03-28T00:21:22.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:21:21 smithi132 ceph-mon[35941]: Deploying daemon osd.8 on smithi165 2024-03-28T00:21:22.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:21:22 smithi165 ceph-mon[36302]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-03-28T00:21:23.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:21:22 smithi119 ceph-mon[31518]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-03-28T00:21:23.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:21:22 smithi132 ceph-mon[35941]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-03-28T00:21:24.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:21:24 smithi165 ceph-mon[36302]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-03-28T00:21:25.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:21:24 smithi119 ceph-mon[31518]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-03-28T00:21:25.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:21:24 smithi132 ceph-mon[35941]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-03-28T00:21:27.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:21:26 smithi165 ceph-mon[36302]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-03-28T00:21:27.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:21:26 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:21:27.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:21:26 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:21:27.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:21:26 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:21:27.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:21:26 smithi119 ceph-mon[31518]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-03-28T00:21:27.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:21:26 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:21:27.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:21:26 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:21:27.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:21:26 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:21:27.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:21:26 smithi132 ceph-mon[35941]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-03-28T00:21:27.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:21:26 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:21:27.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:21:26 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:21:27.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:21:26 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:21:29.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:21:28 smithi165 ceph-mon[36302]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-03-28T00:21:29.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:21:28 smithi119 ceph-mon[31518]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-03-28T00:21:29.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:21:28 smithi132 ceph-mon[35941]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-03-28T00:21:29.287 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:21:29.280+0000 7fb6197fa700 1 -- 172.21.15.165:0/3978091365 <== mgr.14152 v2:172.21.15.119:6800/1401937096 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fb61c0625d0 con 0x7fb6040a2890 2024-03-28T00:21:29.287 INFO:teuthology.orchestra.run.smithi165.stdout:Created osd(s) 8 on host 'smithi165' 2024-03-28T00:21:29.289 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:21:29.282+0000 7fb622728700 1 -- 172.21.15.165:0/3978091365 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fb6040a2890 msgr2=0x7fb6040a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:21:29.289 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:21:29.282+0000 7fb622728700 1 --2- 172.21.15.165:0/3978091365 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fb6040a2890 0x7fb6040a4d50 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb60c006fd0 tx=0x7fb60c008040 comp rx=0 tx=0).stop 2024-03-28T00:21:29.289 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:21:29.282+0000 7fb622728700 1 -- 172.21.15.165:0/3978091365 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb61c102c60 msgr2=0x7fb61c19f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:21:29.289 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:21:29.282+0000 7fb622728700 1 --2- 172.21.15.165:0/3978091365 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb61c102c60 0x7fb61c19f830 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fb61400ef30 tx=0x7fb61400c3a0 comp rx=0 tx=0).stop 2024-03-28T00:21:29.289 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:21:29.282+0000 7fb622728700 1 -- 172.21.15.165:0/3978091365 shutdown_connections 2024-03-28T00:21:29.289 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:21:29.282+0000 7fb622728700 1 --2- 172.21.15.165:0/3978091365 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fb6040a2890 0x7fb6040a4d50 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:21:29.289 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:21:29.282+0000 7fb622728700 1 --2- 172.21.15.165:0/3978091365 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb61c102c60 0x7fb61c19f830 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:21:29.289 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:21:29.282+0000 7fb622728700 1 --2- 172.21.15.165:0/3978091365 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb61c1010c0 0x7fb61c06a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:21:29.289 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:21:29.282+0000 7fb622728700 1 --2- 172.21.15.165:0/3978091365 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb61c1022c0 0x7fb61c19d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:21:29.289 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:21:29.282+0000 7fb622728700 1 -- 172.21.15.165:0/3978091365 >> 172.21.15.165:0/3978091365 conn(0x7fb61c0fc660 msgr2=0x7fb61c105640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:21:29.289 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:21:29.282+0000 7fb622728700 1 -- 172.21.15.165:0/3978091365 shutdown_connections 2024-03-28T00:21:29.290 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:21:29.282+0000 7fb622728700 1 -- 172.21.15.165:0/3978091365 wait complete. 2024-03-28T00:21:29.975 DEBUG:teuthology.orchestra.run.smithi165:osd.8> sudo journalctl -f -n 0 -u ceph-10f73b92-ec98-11ee-95d0-87774f69a715@osd.8.service 2024-03-28T00:21:29.978 INFO:tasks.cephadm:Deploying osd.9 on smithi119 with /dev/vg_nvme/lv_1... 2024-03-28T00:21:29.978 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 10f73b92-ec98-11ee-95d0-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-03-28T00:21:30.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:21:30 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:21:30.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:21:30 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:21:30.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:21:30 smithi119 ceph-mon[31518]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-03-28T00:21:30.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:21:30 smithi119 ceph-mon[31518]: from='osd.8 [v2:172.21.15.165:6816/4115226150,v1:172.21.15.165:6817/4115226150]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-28T00:21:30.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:21:30 smithi119 ceph-mon[31518]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-28T00:21:30.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:21:30 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:21:30.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:21:30 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:21:30.424 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/mon.a/config 2024-03-28T00:21:30.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:21:30 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:21:30.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:21:30 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:21:30.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:21:30 smithi165 ceph-mon[36302]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-03-28T00:21:30.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:21:30 smithi165 ceph-mon[36302]: from='osd.8 [v2:172.21.15.165:6816/4115226150,v1:172.21.15.165:6817/4115226150]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-28T00:21:30.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:21:30 smithi165 ceph-mon[36302]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-28T00:21:30.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:21:30 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:21:30.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:21:30 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:21:30.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:21:30 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:21:30.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:21:30 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:21:30.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:21:30 smithi132 ceph-mon[35941]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-03-28T00:21:30.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:21:30 smithi132 ceph-mon[35941]: from='osd.8 [v2:172.21.15.165:6816/4115226150,v1:172.21.15.165:6817/4115226150]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-28T00:21:30.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:21:30 smithi132 ceph-mon[35941]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-28T00:21:30.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:21:30 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:21:30.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:21:30 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:21:31.620 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:21:31 smithi119 ceph-mon[31518]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-03-28T00:21:31.620 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:21:31 smithi119 ceph-mon[31518]: from='osd.8 [v2:172.21.15.165:6816/4115226150,v1:172.21.15.165:6817/4115226150]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]: dispatch 2024-03-28T00:21:31.620 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:21:31 smithi119 ceph-mon[31518]: osdmap e57: 9 total, 8 up, 9 in 2024-03-28T00:21:31.620 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:21:31 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-28T00:21:31.620 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:21:31 smithi119 ceph-mon[31518]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]: dispatch 2024-03-28T00:21:31.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:21:31 smithi165 ceph-mon[36302]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-03-28T00:21:31.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:21:31 smithi165 ceph-mon[36302]: from='osd.8 [v2:172.21.15.165:6816/4115226150,v1:172.21.15.165:6817/4115226150]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]: dispatch 2024-03-28T00:21:31.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:21:31 smithi165 ceph-mon[36302]: osdmap e57: 9 total, 8 up, 9 in 2024-03-28T00:21:31.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:21:31 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-28T00:21:31.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:21:31 smithi165 ceph-mon[36302]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]: dispatch 2024-03-28T00:21:31.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:21:31 smithi132 ceph-mon[35941]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-03-28T00:21:31.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:21:31 smithi132 ceph-mon[35941]: from='osd.8 [v2:172.21.15.165:6816/4115226150,v1:172.21.15.165:6817/4115226150]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]: dispatch 2024-03-28T00:21:31.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:21:31 smithi132 ceph-mon[35941]: osdmap e57: 9 total, 8 up, 9 in 2024-03-28T00:21:31.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:21:31 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-28T00:21:31.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:21:31 smithi132 ceph-mon[35941]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]: dispatch 2024-03-28T00:21:31.966 INFO:journalctl@ceph.osd.8.smithi165.stdout:Mar 28 00:21:31 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-8[55184]: 2024-03-28T00:21:31.699+0000 7fd54a666700 -1 osd.8 0 waiting for initial osdmap 2024-03-28T00:21:31.966 INFO:journalctl@ceph.osd.8.smithi165.stdout:Mar 28 00:21:31 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-8[55184]: 2024-03-28T00:21:31.710+0000 7fd542451700 -1 osd.8 58 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-28T00:21:32.539 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:21:32 smithi119 ceph-mon[31518]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]': finished 2024-03-28T00:21:32.539 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:21:32 smithi119 ceph-mon[31518]: osdmap e58: 9 total, 8 up, 9 in 2024-03-28T00:21:32.539 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:21:32 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-28T00:21:32.539 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:21:32 smithi119 ceph-mon[31518]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-03-28T00:21:32.539 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:21:32 smithi119 ceph-mon[31518]: from='osd.8 ' entity='osd.8' 2024-03-28T00:21:32.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:21:32 smithi165 ceph-mon[36302]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]': finished 2024-03-28T00:21:32.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:21:32 smithi165 ceph-mon[36302]: osdmap e58: 9 total, 8 up, 9 in 2024-03-28T00:21:32.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:21:32 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-28T00:21:32.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:21:32 smithi165 ceph-mon[36302]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-03-28T00:21:32.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:21:32 smithi165 ceph-mon[36302]: from='osd.8 ' entity='osd.8' 2024-03-28T00:21:32.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:21:32 smithi132 ceph-mon[35941]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]': finished 2024-03-28T00:21:32.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:21:32 smithi132 ceph-mon[35941]: osdmap e58: 9 total, 8 up, 9 in 2024-03-28T00:21:32.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:21:32 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-28T00:21:32.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:21:32 smithi132 ceph-mon[35941]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-03-28T00:21:32.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:21:32 smithi132 ceph-mon[35941]: from='osd.8 ' entity='osd.8' 2024-03-28T00:21:33.378 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:21:33 smithi119 ceph-mon[31518]: purged_snaps scrub starts 2024-03-28T00:21:33.379 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:21:33 smithi119 ceph-mon[31518]: purged_snaps scrub ok 2024-03-28T00:21:33.379 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:21:33 smithi119 ceph-mon[31518]: osd.8 [v2:172.21.15.165:6816/4115226150,v1:172.21.15.165:6817/4115226150] boot 2024-03-28T00:21:33.379 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:21:33 smithi119 ceph-mon[31518]: osdmap e59: 9 total, 9 up, 9 in 2024-03-28T00:21:33.379 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:21:33 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-28T00:21:33.379 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:21:33 smithi119 ceph-mon[31518]: osdmap e60: 9 total, 9 up, 9 in 2024-03-28T00:21:33.379 INFO:teuthology.orchestra.run.smithi119.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-03-28T00:21:33.379 INFO:teuthology.orchestra.run.smithi119.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-03-28T00:21:33.379 INFO:teuthology.orchestra.run.smithi119.stderr: stderr: 10+0 records in 2024-03-28T00:21:33.379 INFO:teuthology.orchestra.run.smithi119.stderr:10+0 records out 2024-03-28T00:21:33.379 INFO:teuthology.orchestra.run.smithi119.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0326909 s, 321 MB/s 2024-03-28T00:21:33.379 INFO:teuthology.orchestra.run.smithi119.stderr:--> Zapping successful for: 2024-03-28T00:21:33.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:21:33 smithi165 ceph-mon[36302]: purged_snaps scrub starts 2024-03-28T00:21:33.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:21:33 smithi165 ceph-mon[36302]: purged_snaps scrub ok 2024-03-28T00:21:33.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:21:33 smithi165 ceph-mon[36302]: osd.8 [v2:172.21.15.165:6816/4115226150,v1:172.21.15.165:6817/4115226150] boot 2024-03-28T00:21:33.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:21:33 smithi165 ceph-mon[36302]: osdmap e59: 9 total, 9 up, 9 in 2024-03-28T00:21:33.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:21:33 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-28T00:21:33.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:21:33 smithi165 ceph-mon[36302]: osdmap e60: 9 total, 9 up, 9 in 2024-03-28T00:21:33.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:21:33 smithi132 ceph-mon[35941]: purged_snaps scrub starts 2024-03-28T00:21:33.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:21:33 smithi132 ceph-mon[35941]: purged_snaps scrub ok 2024-03-28T00:21:33.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:21:33 smithi132 ceph-mon[35941]: osd.8 [v2:172.21.15.165:6816/4115226150,v1:172.21.15.165:6817/4115226150] boot 2024-03-28T00:21:33.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:21:33 smithi132 ceph-mon[35941]: osdmap e59: 9 total, 9 up, 9 in 2024-03-28T00:21:33.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:21:33 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-28T00:21:33.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:21:33 smithi132 ceph-mon[35941]: osdmap e60: 9 total, 9 up, 9 in 2024-03-28T00:21:34.275 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 10f73b92-ec98-11ee-95d0-87774f69a715 -- ceph orch daemon add osd smithi119:vg_nvme/lv_1 2024-03-28T00:21:34.723 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/mon.a/config 2024-03-28T00:21:35.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:21:34 smithi165 ceph-mon[36302]: pgmap v215: 1 pgs: 1 remapped+peering; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-03-28T00:21:35.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:21:34 smithi165 ceph-mon[36302]: Detected new or changed devices on smithi165 2024-03-28T00:21:35.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:21:34 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:21:35.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:21:34 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:21:35.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:21:34 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:21:35.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:21:34 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:21:35.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:21:34 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:21:35.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:21:34 smithi165 ceph-mon[36302]: Adjusting osd_memory_target on smithi165 to 5693M 2024-03-28T00:21:35.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:21:34 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:21:35.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:21:34 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:21:35.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:21:34 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:21:35.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:21:34 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:21:35.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:21:34 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:21:35.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:21:34 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:21:35.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:21:34 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:21:35.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:21:34 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:21:35.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:21:34 smithi165 ceph-mon[36302]: osdmap e61: 9 total, 9 up, 9 in 2024-03-28T00:21:35.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:21:34 smithi165 ceph-mon[36302]: osdmap e62: 9 total, 9 up, 9 in 2024-03-28T00:21:35.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:21:34 smithi119 ceph-mon[31518]: pgmap v215: 1 pgs: 1 remapped+peering; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-03-28T00:21:35.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:21:34 smithi119 ceph-mon[31518]: Detected new or changed devices on smithi165 2024-03-28T00:21:35.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:21:34 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:21:35.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:21:34 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:21:35.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:21:34 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:21:35.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:21:34 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:21:35.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:21:34 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:21:35.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:21:34 smithi119 ceph-mon[31518]: Adjusting osd_memory_target on smithi165 to 5693M 2024-03-28T00:21:35.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:21:34 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:21:35.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:21:34 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:21:35.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:21:34 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:21:35.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:21:34 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:21:35.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:21:34 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:21:35.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:21:34 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:21:35.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:21:34 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:21:35.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:21:34 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:21:35.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:21:34 smithi119 ceph-mon[31518]: osdmap e61: 9 total, 9 up, 9 in 2024-03-28T00:21:35.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:21:34 smithi119 ceph-mon[31518]: osdmap e62: 9 total, 9 up, 9 in 2024-03-28T00:21:35.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:21:34 smithi132 ceph-mon[35941]: pgmap v215: 1 pgs: 1 remapped+peering; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-03-28T00:21:35.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:21:34 smithi132 ceph-mon[35941]: Detected new or changed devices on smithi165 2024-03-28T00:21:35.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:21:34 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:21:35.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:21:34 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:21:35.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:21:34 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:21:35.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:21:34 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:21:35.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:21:34 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:21:35.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:21:34 smithi132 ceph-mon[35941]: Adjusting osd_memory_target on smithi165 to 5693M 2024-03-28T00:21:35.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:21:34 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:21:35.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:21:34 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:21:35.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:21:34 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:21:35.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:21:34 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:21:35.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:21:34 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:21:35.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:21:34 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:21:35.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:21:34 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:21:35.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:21:34 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:21:35.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:21:34 smithi132 ceph-mon[35941]: osdmap e61: 9 total, 9 up, 9 in 2024-03-28T00:21:35.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:21:34 smithi132 ceph-mon[35941]: osdmap e62: 9 total, 9 up, 9 in 2024-03-28T00:21:36.569 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:21:36 smithi119 ceph-mon[31518]: osdmap e63: 9 total, 9 up, 9 in 2024-03-28T00:21:36.569 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:21:36 smithi119 ceph-mon[31518]: pgmap v219: 1 pgs: 1 remapped+peering; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-03-28T00:21:36.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:21:36 smithi132 ceph-mon[35941]: osdmap e63: 9 total, 9 up, 9 in 2024-03-28T00:21:36.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:21:36 smithi132 ceph-mon[35941]: pgmap v219: 1 pgs: 1 remapped+peering; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-03-28T00:21:36.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:21:36 smithi165 ceph-mon[36302]: osdmap e63: 9 total, 9 up, 9 in 2024-03-28T00:21:36.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:21:36 smithi165 ceph-mon[36302]: pgmap v219: 1 pgs: 1 remapped+peering; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-03-28T00:21:37.070 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:21:37.064+0000 7f2b73000700 1 -- 172.21.15.119:0/1426597300 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2b6c10a2a0 msgr2=0x7f2b6c110b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:21:37.070 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:21:37.064+0000 7f2b73000700 1 --2- 172.21.15.119:0/1426597300 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2b6c10a2a0 0x7f2b6c110b50 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f2b64009f90 tx=0x7f2b6400b330 comp rx=0 tx=0).stop 2024-03-28T00:21:37.070 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:21:37.064+0000 7f2b73000700 1 -- 172.21.15.119:0/1426597300 shutdown_connections 2024-03-28T00:21:37.071 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:21:37.064+0000 7f2b73000700 1 --2- 172.21.15.119:0/1426597300 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2b6c10a2a0 0x7f2b6c110b50 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:21:37.071 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:21:37.064+0000 7f2b73000700 1 --2- 172.21.15.119:0/1426597300 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2b6c108700 0x7f2b6c108b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:21:37.071 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:21:37.064+0000 7f2b73000700 1 --2- 172.21.15.119:0/1426597300 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2b6c109900 0x7f2b6c109d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:21:37.071 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:21:37.064+0000 7f2b73000700 1 -- 172.21.15.119:0/1426597300 >> 172.21.15.119:0/1426597300 conn(0x7f2b6c0776d0 msgr2=0x7f2b6c079b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:21:37.071 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:21:37.064+0000 7f2b73000700 1 -- 172.21.15.119:0/1426597300 shutdown_connections 2024-03-28T00:21:37.071 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:21:37.064+0000 7f2b73000700 1 -- 172.21.15.119:0/1426597300 wait complete. 2024-03-28T00:21:37.073 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:21:37.065+0000 7f2b73000700 1 Processor -- start 2024-03-28T00:21:37.073 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:21:37.065+0000 7f2b73000700 1 -- start start 2024-03-28T00:21:37.073 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:21:37.065+0000 7f2b73000700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2b6c108700 0x7f2b6c19e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:21:37.073 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:21:37.065+0000 7f2b73000700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2b6c109900 0x7f2b6c19ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:21:37.073 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:21:37.066+0000 7f2b73000700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2b6c10a2a0 0x7f2b6c1a5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:21:37.073 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:21:37.066+0000 7f2b73000700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b6c114340 con 0x7f2b6c109900 2024-03-28T00:21:37.073 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:21:37.066+0000 7f2b73000700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b6c1141c0 con 0x7f2b6c10a2a0 2024-03-28T00:21:37.073 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:21:37.066+0000 7f2b73000700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b6c1144c0 con 0x7f2b6c108700 2024-03-28T00:21:37.073 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:21:37.066+0000 7f2b71200700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2b6c109900 0x7f2b6c19ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:21:37.073 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:21:37.066+0000 7f2b71200700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2b6c109900 0x7f2b6c19ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:33190/0 (socket says 172.21.15.119:33190) 2024-03-28T00:21:37.073 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:21:37.066+0000 7f2b71200700 1 -- 172.21.15.119:0/375776660 learned_addr learned my addr 172.21.15.119:0/375776660 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:21:37.074 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:21:37.066+0000 7f2b71c00700 1 --2- 172.21.15.119:0/375776660 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2b6c108700 0x7f2b6c19e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:21:37.074 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:21:37.066+0000 7f2b72600700 1 --2- 172.21.15.119:0/375776660 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2b6c10a2a0 0x7f2b6c1a5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:21:37.074 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:21:37.066+0000 7f2b71c00700 1 -- 172.21.15.119:0/375776660 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2b6c10a2a0 msgr2=0x7f2b6c1a5ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:21:37.074 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:21:37.066+0000 7f2b71c00700 1 --2- 172.21.15.119:0/375776660 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2b6c10a2a0 0x7f2b6c1a5ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:21:37.074 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:21:37.066+0000 7f2b71c00700 1 -- 172.21.15.119:0/375776660 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2b6c109900 msgr2=0x7f2b6c19ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:21:37.074 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:21:37.066+0000 7f2b71c00700 1 --2- 172.21.15.119:0/375776660 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2b6c109900 0x7f2b6c19ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:21:37.074 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:21:37.066+0000 7f2b71c00700 1 -- 172.21.15.119:0/375776660 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b6c1a6560 con 0x7f2b6c108700 2024-03-28T00:21:37.074 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:21:37.066+0000 7f2b71200700 1 --2- 172.21.15.119:0/375776660 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2b6c109900 0x7f2b6c19ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T00:21:37.074 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:21:37.067+0000 7f2b71c00700 1 --2- 172.21.15.119:0/375776660 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2b6c108700 0x7f2b6c19e900 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f2b6000e9f0 tx=0x7f2b6000ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:21:37.074 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:21:37.067+0000 7f2b6a000700 1 -- 172.21.15.119:0/375776660 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b60004230 con 0x7f2b6c108700 2024-03-28T00:21:37.074 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:21:37.067+0000 7f2b73000700 1 -- 172.21.15.119:0/375776660 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b6c1a6850 con 0x7f2b6c108700 2024-03-28T00:21:37.074 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:21:37.067+0000 7f2b6a000700 1 -- 172.21.15.119:0/375776660 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2b600043d0 con 0x7f2b6c108700 2024-03-28T00:21:37.074 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:21:37.067+0000 7f2b6a000700 1 -- 172.21.15.119:0/375776660 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b60010430 con 0x7f2b6c108700 2024-03-28T00:21:37.074 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:21:37.067+0000 7f2b73000700 1 -- 172.21.15.119:0/375776660 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b6c1a6d90 con 0x7f2b6c108700 2024-03-28T00:21:37.075 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:21:37.068+0000 7f2b73000700 1 -- 172.21.15.119:0/375776660 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b6c068e50 con 0x7f2b6c108700 2024-03-28T00:21:37.076 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:21:37.070+0000 7f2b6a000700 1 -- 172.21.15.119:0/375776660 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f2b6000c7c0 con 0x7f2b6c108700 2024-03-28T00:21:37.077 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:21:37.071+0000 7f2b6a000700 1 --2- 172.21.15.119:0/375776660 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f2b540a2910 0x7f2b540a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:21:37.077 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:21:37.071+0000 7f2b6a000700 1 -- 172.21.15.119:0/375776660 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5829+0+0 (secure 0 0 0) 0x7f2b600ce3b0 con 0x7f2b6c108700 2024-03-28T00:21:37.077 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:21:37.071+0000 7f2b71200700 1 --2- 172.21.15.119:0/375776660 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f2b540a2910 0x7f2b540a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:21:37.078 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:21:37.072+0000 7f2b71200700 1 --2- 172.21.15.119:0/375776660 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f2b540a2910 0x7f2b540a4dd0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f2b58005950 tx=0x7f2b58009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:21:37.081 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:21:37.076+0000 7f2b6a000700 1 -- 172.21.15.119:0/375776660 <== mon.1 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f2b6009bd60 con 0x7f2b6c108700 2024-03-28T00:21:37.312 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:21:37.306+0000 7f2b73000700 1 -- 172.21.15.119:0/375776660 --> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi119:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f2b6c0625d0 con 0x7f2b540a2910 2024-03-28T00:21:37.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:21:37 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-28T00:21:37.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:21:37 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-28T00:21:37.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:21:37 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:21:37.701 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:21:37 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-28T00:21:37.701 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:21:37 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-28T00:21:37.701 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:21:37 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:21:37.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:21:37 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-28T00:21:37.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:21:37 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-28T00:21:37.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:21:37 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:21:38.472 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:21:38 smithi119 ceph-mon[31518]: from='client.24334 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi119:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T00:21:38.472 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:21:38 smithi119 ceph-mon[31518]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 131 KiB/s, 0 objects/s recovering 2024-03-28T00:21:38.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:21:38 smithi132 ceph-mon[35941]: from='client.24334 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi119:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T00:21:38.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:21:38 smithi132 ceph-mon[35941]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 131 KiB/s, 0 objects/s recovering 2024-03-28T00:21:38.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:21:38 smithi165 ceph-mon[36302]: from='client.24334 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi119:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T00:21:38.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:21:38 smithi165 ceph-mon[36302]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 131 KiB/s, 0 objects/s recovering 2024-03-28T00:21:41.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:21:40 smithi165 ceph-mon[36302]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-28T00:21:41.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:21:40 smithi119 ceph-mon[31518]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-28T00:21:41.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:21:40 smithi132 ceph-mon[35941]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-28T00:21:41.763 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:21:41 smithi119 ceph-mon[31518]: from='client.? 172.21.15.119:0/4216859734' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2b1a08ed-cde9-4def-b3e8-838fff22709c"}]: dispatch 2024-03-28T00:21:41.763 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:21:41 smithi119 ceph-mon[31518]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2b1a08ed-cde9-4def-b3e8-838fff22709c"}]: dispatch 2024-03-28T00:21:41.763 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:21:41 smithi119 ceph-mon[31518]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2b1a08ed-cde9-4def-b3e8-838fff22709c"}]': finished 2024-03-28T00:21:41.763 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:21:41 smithi119 ceph-mon[31518]: osdmap e64: 10 total, 9 up, 10 in 2024-03-28T00:21:41.763 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:21:41 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-28T00:21:42.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:21:41 smithi165 ceph-mon[36302]: from='client.? 172.21.15.119:0/4216859734' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2b1a08ed-cde9-4def-b3e8-838fff22709c"}]: dispatch 2024-03-28T00:21:42.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:21:41 smithi165 ceph-mon[36302]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2b1a08ed-cde9-4def-b3e8-838fff22709c"}]: dispatch 2024-03-28T00:21:42.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:21:41 smithi165 ceph-mon[36302]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2b1a08ed-cde9-4def-b3e8-838fff22709c"}]': finished 2024-03-28T00:21:42.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:21:41 smithi165 ceph-mon[36302]: osdmap e64: 10 total, 9 up, 10 in 2024-03-28T00:21:42.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:21:41 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-28T00:21:42.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:21:41 smithi132 ceph-mon[35941]: from='client.? 172.21.15.119:0/4216859734' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2b1a08ed-cde9-4def-b3e8-838fff22709c"}]: dispatch 2024-03-28T00:21:42.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:21:41 smithi132 ceph-mon[35941]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2b1a08ed-cde9-4def-b3e8-838fff22709c"}]: dispatch 2024-03-28T00:21:42.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:21:41 smithi132 ceph-mon[35941]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2b1a08ed-cde9-4def-b3e8-838fff22709c"}]': finished 2024-03-28T00:21:42.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:21:41 smithi132 ceph-mon[35941]: osdmap e64: 10 total, 9 up, 10 in 2024-03-28T00:21:42.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:21:41 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-28T00:21:43.057 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:21:42 smithi119 ceph-mon[31518]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-03-28T00:21:43.057 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:21:42 smithi119 ceph-mon[31518]: from='client.? 172.21.15.119:0/3190498149' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-28T00:21:43.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:21:42 smithi165 ceph-mon[36302]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-03-28T00:21:43.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:21:42 smithi165 ceph-mon[36302]: from='client.? 172.21.15.119:0/3190498149' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-28T00:21:43.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:21:42 smithi132 ceph-mon[35941]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-03-28T00:21:43.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:21:42 smithi132 ceph-mon[35941]: from='client.? 172.21.15.119:0/3190498149' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-28T00:21:45.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:21:44 smithi165 ceph-mon[36302]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-03-28T00:21:45.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:21:44 smithi119 ceph-mon[31518]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-03-28T00:21:45.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:21:44 smithi132 ceph-mon[35941]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-03-28T00:21:47.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:21:46 smithi165 ceph-mon[36302]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-28T00:21:47.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:21:46 smithi119 ceph-mon[31518]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-28T00:21:47.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:21:46 smithi132 ceph-mon[35941]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-28T00:21:49.107 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:21:48 smithi119 ceph-mon[31518]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-03-28T00:21:49.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:21:48 smithi165 ceph-mon[36302]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-03-28T00:21:49.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:21:48 smithi132 ceph-mon[35941]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-03-28T00:21:51.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:21:50 smithi165 ceph-mon[36302]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-03-28T00:21:51.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:21:50 smithi119 ceph-mon[31518]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-03-28T00:21:51.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:21:50 smithi132 ceph-mon[35941]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-03-28T00:21:52.066 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:21:51 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-03-28T00:21:52.067 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:21:51 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:21:52.067 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:21:51 smithi119 ceph-mon[31518]: Deploying daemon osd.9 on smithi119 2024-03-28T00:21:52.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:21:51 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-03-28T00:21:52.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:21:51 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:21:52.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:21:51 smithi165 ceph-mon[36302]: Deploying daemon osd.9 on smithi119 2024-03-28T00:21:52.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:21:51 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-03-28T00:21:52.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:21:51 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:21:52.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:21:51 smithi132 ceph-mon[35941]: Deploying daemon osd.9 on smithi119 2024-03-28T00:21:53.034 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:21:52 smithi119 ceph-mon[31518]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-03-28T00:21:53.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:21:52 smithi165 ceph-mon[36302]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-03-28T00:21:53.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:21:52 smithi132 ceph-mon[35941]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-03-28T00:21:55.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:21:54 smithi165 ceph-mon[36302]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-03-28T00:21:55.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:21:54 smithi119 ceph-mon[31518]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-03-28T00:21:55.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:21:54 smithi132 ceph-mon[35941]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-03-28T00:21:56.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:21:56 smithi119 ceph-mon[31518]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-03-28T00:21:57.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:21:56 smithi165 ceph-mon[36302]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-03-28T00:21:57.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:21:56 smithi132 ceph-mon[35941]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-03-28T00:21:59.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:21:58 smithi165 ceph-mon[36302]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-03-28T00:21:59.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:21:58 smithi119 ceph-mon[31518]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-03-28T00:21:59.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:21:58 smithi132 ceph-mon[35941]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-03-28T00:22:00.358 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:00 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:22:00.358 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:00 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:22:00.358 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:00 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:22:00.358 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:00 smithi119 ceph-mon[31518]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-03-28T00:22:00.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:00 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:22:00.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:00 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:22:00.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:00 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:22:00.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:00 smithi165 ceph-mon[36302]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-03-28T00:22:00.452 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:00 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:22:00.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:00 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:22:00.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:00 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:22:00.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:00 smithi132 ceph-mon[35941]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-03-28T00:22:02.875 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:02 smithi119 ceph-mon[31518]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-03-28T00:22:03.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:02 smithi165 ceph-mon[36302]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-03-28T00:22:03.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:02 smithi132 ceph-mon[35941]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-03-28T00:22:04.262 INFO:teuthology.orchestra.run.smithi119.stdout:Created osd(s) 9 on host 'smithi119' 2024-03-28T00:22:04.263 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:22:04.255+0000 7f2b6a000700 1 -- 172.21.15.119:0/375776660 <== mgr.14152 v2:172.21.15.119:6800/1401937096 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f2b6c0625d0 con 0x7f2b540a2910 2024-03-28T00:22:04.264 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:22:04.259+0000 7f2b73000700 1 -- 172.21.15.119:0/375776660 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f2b540a2910 msgr2=0x7f2b540a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:22:04.265 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:22:04.259+0000 7f2b73000700 1 --2- 172.21.15.119:0/375776660 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f2b540a2910 0x7f2b540a4dd0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f2b58005950 tx=0x7f2b58009310 comp rx=0 tx=0).stop 2024-03-28T00:22:04.265 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:22:04.259+0000 7f2b73000700 1 -- 172.21.15.119:0/375776660 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2b6c108700 msgr2=0x7f2b6c19e900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:22:04.265 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:22:04.259+0000 7f2b73000700 1 --2- 172.21.15.119:0/375776660 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2b6c108700 0x7f2b6c19e900 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f2b6000e9f0 tx=0x7f2b6000ed00 comp rx=0 tx=0).stop 2024-03-28T00:22:04.266 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:22:04.259+0000 7f2b73000700 1 -- 172.21.15.119:0/375776660 shutdown_connections 2024-03-28T00:22:04.266 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:22:04.259+0000 7f2b73000700 1 --2- 172.21.15.119:0/375776660 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f2b540a2910 0x7f2b540a4dd0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:22:04.266 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:22:04.259+0000 7f2b73000700 1 --2- 172.21.15.119:0/375776660 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2b6c10a2a0 0x7f2b6c1a5ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:22:04.266 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:22:04.259+0000 7f2b73000700 1 --2- 172.21.15.119:0/375776660 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2b6c108700 0x7f2b6c19e900 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:22:04.266 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:22:04.259+0000 7f2b73000700 1 --2- 172.21.15.119:0/375776660 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2b6c109900 0x7f2b6c19ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:22:04.266 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:22:04.259+0000 7f2b73000700 1 -- 172.21.15.119:0/375776660 >> 172.21.15.119:0/375776660 conn(0x7f2b6c0776d0 msgr2=0x7f2b6c10cba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:22:04.266 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:22:04.259+0000 7f2b73000700 1 -- 172.21.15.119:0/375776660 shutdown_connections 2024-03-28T00:22:04.266 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:22:04.259+0000 7f2b73000700 1 -- 172.21.15.119:0/375776660 wait complete. 2024-03-28T00:22:04.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:04 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:22:04.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:04 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:22:04.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:04 smithi165 ceph-mon[36302]: from='osd.9 [v2:172.21.15.119:6826/2532245222,v1:172.21.15.119:6827/2532245222]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-03-28T00:22:04.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:04 smithi165 ceph-mon[36302]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-03-28T00:22:04.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:04 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:22:04.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:04 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:22:04.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:04 smithi119 ceph-mon[31518]: from='osd.9 [v2:172.21.15.119:6826/2532245222,v1:172.21.15.119:6827/2532245222]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-03-28T00:22:04.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:04 smithi119 ceph-mon[31518]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-03-28T00:22:04.452 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:04 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:22:04.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:04 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:22:04.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:04 smithi132 ceph-mon[35941]: from='osd.9 [v2:172.21.15.119:6826/2532245222,v1:172.21.15.119:6827/2532245222]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-03-28T00:22:04.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:04 smithi132 ceph-mon[35941]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-03-28T00:22:05.118 DEBUG:teuthology.orchestra.run.smithi119:osd.9> sudo journalctl -f -n 0 -u ceph-10f73b92-ec98-11ee-95d0-87774f69a715@osd.9.service 2024-03-28T00:22:05.124 INFO:tasks.cephadm:Deploying osd.10 on smithi132 with /dev/vg_nvme/lv_1... 2024-03-28T00:22:05.124 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 10f73b92-ec98-11ee-95d0-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-03-28T00:22:05.339 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:05 smithi119 ceph-mon[31518]: from='osd.9 [v2:172.21.15.119:6826/2532245222,v1:172.21.15.119:6827/2532245222]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-03-28T00:22:05.339 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:05 smithi119 ceph-mon[31518]: osdmap e65: 10 total, 9 up, 10 in 2024-03-28T00:22:05.339 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:05 smithi119 ceph-mon[31518]: from='osd.9 [v2:172.21.15.119:6826/2532245222,v1:172.21.15.119:6827/2532245222]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]: dispatch 2024-03-28T00:22:05.339 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:05 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-28T00:22:05.339 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:05 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:22:05.339 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:05 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:22:05.369 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/mon.b/config 2024-03-28T00:22:05.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:05 smithi165 ceph-mon[36302]: from='osd.9 [v2:172.21.15.119:6826/2532245222,v1:172.21.15.119:6827/2532245222]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-03-28T00:22:05.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:05 smithi165 ceph-mon[36302]: osdmap e65: 10 total, 9 up, 10 in 2024-03-28T00:22:05.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:05 smithi165 ceph-mon[36302]: from='osd.9 [v2:172.21.15.119:6826/2532245222,v1:172.21.15.119:6827/2532245222]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]: dispatch 2024-03-28T00:22:05.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:05 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-28T00:22:05.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:05 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:22:05.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:05 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:22:05.452 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:05 smithi132 ceph-mon[35941]: from='osd.9 [v2:172.21.15.119:6826/2532245222,v1:172.21.15.119:6827/2532245222]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-03-28T00:22:05.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:05 smithi132 ceph-mon[35941]: osdmap e65: 10 total, 9 up, 10 in 2024-03-28T00:22:05.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:05 smithi132 ceph-mon[35941]: from='osd.9 [v2:172.21.15.119:6826/2532245222,v1:172.21.15.119:6827/2532245222]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]: dispatch 2024-03-28T00:22:05.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:05 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-28T00:22:05.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:05 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:22:05.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:05 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:22:06.091 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 00:22:05 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T00:22:05.695+0000 7f6397400700 -1 osd.9 0 waiting for initial osdmap 2024-03-28T00:22:06.092 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 00:22:05 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T00:22:05.735+0000 7f638c800700 -1 osd.9 66 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-28T00:22:06.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:06 smithi165 ceph-mon[36302]: purged_snaps scrub starts 2024-03-28T00:22:06.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:06 smithi165 ceph-mon[36302]: purged_snaps scrub ok 2024-03-28T00:22:06.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:06 smithi165 ceph-mon[36302]: from='osd.9 [v2:172.21.15.119:6826/2532245222,v1:172.21.15.119:6827/2532245222]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]': finished 2024-03-28T00:22:06.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:06 smithi165 ceph-mon[36302]: osdmap e66: 10 total, 9 up, 10 in 2024-03-28T00:22:06.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:06 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-28T00:22:06.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:06 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-28T00:22:06.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:06 smithi165 ceph-mon[36302]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-03-28T00:22:06.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:06 smithi165 ceph-mon[36302]: from='osd.9 [v2:172.21.15.119:6826/2532245222,v1:172.21.15.119:6827/2532245222]' entity='osd.9' 2024-03-28T00:22:06.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:06 smithi119 ceph-mon[31518]: purged_snaps scrub starts 2024-03-28T00:22:06.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:06 smithi119 ceph-mon[31518]: purged_snaps scrub ok 2024-03-28T00:22:06.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:06 smithi119 ceph-mon[31518]: from='osd.9 [v2:172.21.15.119:6826/2532245222,v1:172.21.15.119:6827/2532245222]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]': finished 2024-03-28T00:22:06.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:06 smithi119 ceph-mon[31518]: osdmap e66: 10 total, 9 up, 10 in 2024-03-28T00:22:06.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:06 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-28T00:22:06.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:06 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-28T00:22:06.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:06 smithi119 ceph-mon[31518]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-03-28T00:22:06.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:06 smithi119 ceph-mon[31518]: from='osd.9 [v2:172.21.15.119:6826/2532245222,v1:172.21.15.119:6827/2532245222]' entity='osd.9' 2024-03-28T00:22:06.452 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:06 smithi132 ceph-mon[35941]: purged_snaps scrub starts 2024-03-28T00:22:06.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:06 smithi132 ceph-mon[35941]: purged_snaps scrub ok 2024-03-28T00:22:06.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:06 smithi132 ceph-mon[35941]: from='osd.9 [v2:172.21.15.119:6826/2532245222,v1:172.21.15.119:6827/2532245222]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]': finished 2024-03-28T00:22:06.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:06 smithi132 ceph-mon[35941]: osdmap e66: 10 total, 9 up, 10 in 2024-03-28T00:22:06.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:06 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-28T00:22:06.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:06 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-28T00:22:06.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:06 smithi132 ceph-mon[35941]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-03-28T00:22:06.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:06 smithi132 ceph-mon[35941]: from='osd.9 [v2:172.21.15.119:6826/2532245222,v1:172.21.15.119:6827/2532245222]' entity='osd.9' 2024-03-28T00:22:07.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:07 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-28T00:22:07.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:07 smithi165 ceph-mon[36302]: osd.9 [v2:172.21.15.119:6826/2532245222,v1:172.21.15.119:6827/2532245222] boot 2024-03-28T00:22:07.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:07 smithi165 ceph-mon[36302]: osdmap e67: 10 total, 10 up, 10 in 2024-03-28T00:22:07.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:07 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-28T00:22:07.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:07 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-28T00:22:07.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:07 smithi119 ceph-mon[31518]: osd.9 [v2:172.21.15.119:6826/2532245222,v1:172.21.15.119:6827/2532245222] boot 2024-03-28T00:22:07.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:07 smithi119 ceph-mon[31518]: osdmap e67: 10 total, 10 up, 10 in 2024-03-28T00:22:07.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:07 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-28T00:22:07.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:07 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-28T00:22:07.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:07 smithi132 ceph-mon[35941]: osd.9 [v2:172.21.15.119:6826/2532245222,v1:172.21.15.119:6827/2532245222] boot 2024-03-28T00:22:07.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:07 smithi132 ceph-mon[35941]: osdmap e67: 10 total, 10 up, 10 in 2024-03-28T00:22:07.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:07 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-28T00:22:07.464 INFO:teuthology.orchestra.run.smithi132.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-03-28T00:22:07.464 INFO:teuthology.orchestra.run.smithi132.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-03-28T00:22:07.464 INFO:teuthology.orchestra.run.smithi132.stderr: stderr: 10+0 records in 2024-03-28T00:22:07.464 INFO:teuthology.orchestra.run.smithi132.stderr:10+0 records out 2024-03-28T00:22:07.464 INFO:teuthology.orchestra.run.smithi132.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.011585 s, 905 MB/s 2024-03-28T00:22:07.464 INFO:teuthology.orchestra.run.smithi132.stderr:--> Zapping successful for: 2024-03-28T00:22:08.131 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 10f73b92-ec98-11ee-95d0-87774f69a715 -- ceph orch daemon add osd smithi132:vg_nvme/lv_1 2024-03-28T00:22:08.364 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/mon.b/config 2024-03-28T00:22:08.801 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:08 smithi132 ceph-mon[35941]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-28T00:22:08.801 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:08 smithi132 ceph-mon[35941]: osdmap e68: 10 total, 10 up, 10 in 2024-03-28T00:22:09.092 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:08 smithi119 ceph-mon[31518]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-28T00:22:09.092 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:08 smithi119 ceph-mon[31518]: osdmap e68: 10 total, 10 up, 10 in 2024-03-28T00:22:09.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:08 smithi165 ceph-mon[36302]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-28T00:22:09.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:08 smithi165 ceph-mon[36302]: osdmap e68: 10 total, 10 up, 10 in 2024-03-28T00:22:09.963 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:22:09.956+0000 7fd44ca30700 1 -- 172.21.15.132:0/1046529053 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd448102c50 msgr2=0x7fd448109500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:22:09.963 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:22:09.956+0000 7fd44ca30700 1 --2- 172.21.15.132:0/1046529053 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd448102c50 0x7fd448109500 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd44000b0d0 tx=0x7fd44000b3e0 comp rx=0 tx=0).stop 2024-03-28T00:22:09.963 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:22:09.956+0000 7fd44ca30700 1 -- 172.21.15.132:0/1046529053 shutdown_connections 2024-03-28T00:22:09.963 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:22:09.956+0000 7fd44ca30700 1 --2- 172.21.15.132:0/1046529053 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd448102c50 0x7fd448109500 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:22:09.963 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:22:09.956+0000 7fd44ca30700 1 --2- 172.21.15.132:0/1046529053 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4481010b0 0x7fd4481014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:22:09.963 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:22:09.956+0000 7fd44ca30700 1 --2- 172.21.15.132:0/1046529053 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd4481022b0 0x7fd448102710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:22:09.963 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:22:09.956+0000 7fd44ca30700 1 -- 172.21.15.132:0/1046529053 >> 172.21.15.132:0/1046529053 conn(0x7fd4480fc650 msgr2=0x7fd4480fea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:22:09.963 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:22:09.956+0000 7fd44ca30700 1 -- 172.21.15.132:0/1046529053 shutdown_connections 2024-03-28T00:22:09.964 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:22:09.956+0000 7fd44ca30700 1 -- 172.21.15.132:0/1046529053 wait complete. 2024-03-28T00:22:09.965 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:22:09.957+0000 7fd44ca30700 1 Processor -- start 2024-03-28T00:22:09.966 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:22:09.957+0000 7fd44ca30700 1 -- start start 2024-03-28T00:22:09.966 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:22:09.957+0000 7fd44ca30700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4481010b0 0x7fd4481973b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:22:09.966 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:22:09.957+0000 7fd44ca30700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd4481022b0 0x7fd4481978f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:22:09.966 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:22:09.957+0000 7fd44ca30700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd448102c50 0x7fd44819e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:22:09.966 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:22:09.957+0000 7fd44ca30700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd44810cd90 con 0x7fd4481010b0 2024-03-28T00:22:09.966 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:22:09.957+0000 7fd44ca30700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd44810cc10 con 0x7fd4481022b0 2024-03-28T00:22:09.966 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:22:09.957+0000 7fd44ca30700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd44810cf10 con 0x7fd448102c50 2024-03-28T00:22:09.966 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:22:09.957+0000 7fd445d9b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd4481022b0 0x7fd4481978f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:22:09.966 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:22:09.957+0000 7fd445d9b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd4481022b0 0x7fd4481978f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:44372/0 (socket says 172.21.15.132:44372) 2024-03-28T00:22:09.966 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:22:09.957+0000 7fd445d9b700 1 -- 172.21.15.132:0/2103765824 learned_addr learned my addr 172.21.15.132:0/2103765824 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-28T00:22:09.966 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:22:09.957+0000 7fd445d9b700 1 -- 172.21.15.132:0/2103765824 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd448102c50 msgr2=0x7fd44819e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:22:09.967 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:22:09.957+0000 7fd445d9b700 1 --2- 172.21.15.132:0/2103765824 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd448102c50 0x7fd44819e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:22:09.967 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:22:09.957+0000 7fd445d9b700 1 -- 172.21.15.132:0/2103765824 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4481010b0 msgr2=0x7fd4481973b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:22:09.967 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:22:09.957+0000 7fd445d9b700 1 --2- 172.21.15.132:0/2103765824 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4481010b0 0x7fd4481973b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:22:09.967 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:22:09.957+0000 7fd445d9b700 1 -- 172.21.15.132:0/2103765824 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd44819f010 con 0x7fd4481022b0 2024-03-28T00:22:09.967 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:22:09.957+0000 7fd445d9b700 1 --2- 172.21.15.132:0/2103765824 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd4481022b0 0x7fd4481978f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fd43c00b530 tx=0x7fd43c00b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:22:09.967 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:22:09.958+0000 7fd4377fe700 1 -- 172.21.15.132:0/2103765824 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd43c015020 con 0x7fd4481022b0 2024-03-28T00:22:09.967 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:22:09.958+0000 7fd4377fe700 1 -- 172.21.15.132:0/2103765824 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd43c011d10 con 0x7fd4481022b0 2024-03-28T00:22:09.967 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:22:09.958+0000 7fd44ca30700 1 -- 172.21.15.132:0/2103765824 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd44819f300 con 0x7fd4481022b0 2024-03-28T00:22:09.967 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:22:09.958+0000 7fd4377fe700 1 -- 172.21.15.132:0/2103765824 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd43c00f2e0 con 0x7fd4481022b0 2024-03-28T00:22:09.967 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:22:09.958+0000 7fd44ca30700 1 -- 172.21.15.132:0/2103765824 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd44819f8c0 con 0x7fd4481022b0 2024-03-28T00:22:09.967 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:22:09.959+0000 7fd44ca30700 1 -- 172.21.15.132:0/2103765824 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd448068e50 con 0x7fd4481022b0 2024-03-28T00:22:09.968 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:22:09.960+0000 7fd4377fe700 1 -- 172.21.15.132:0/2103765824 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fd43c018020 con 0x7fd4481022b0 2024-03-28T00:22:09.968 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:22:09.960+0000 7fd4377fe700 1 --2- 172.21.15.132:0/2103765824 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fd4300a2890 0x7fd4300a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:22:09.968 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:22:09.961+0000 7fd4377fe700 1 -- 172.21.15.132:0/2103765824 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6261+0+0 (secure 0 0 0) 0x7fd43c0cda60 con 0x7fd4481022b0 2024-03-28T00:22:09.969 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:22:09.961+0000 7fd44659c700 1 --2- 172.21.15.132:0/2103765824 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fd4300a2890 0x7fd4300a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:22:09.969 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:22:09.962+0000 7fd44659c700 1 --2- 172.21.15.132:0/2103765824 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fd4300a2890 0x7fd4300a4d50 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd448102110 tx=0x7fd438006d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:22:09.971 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:22:09.966+0000 7fd4377fe700 1 -- 172.21.15.132:0/2103765824 <== mon.2 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd43c01f020 con 0x7fd4481022b0 2024-03-28T00:22:10.180 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:22:10.173+0000 7fd44ca30700 1 -- 172.21.15.132:0/2103765824 --> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi132:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7fd4480625d0 con 0x7fd4300a2890 2024-03-28T00:22:11.172 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:11 smithi132 ceph-mon[35941]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-28T00:22:11.172 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:11 smithi132 ceph-mon[35941]: Detected new or changed devices on smithi119 2024-03-28T00:22:11.172 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:11 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:22:11.173 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:11 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:22:11.173 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:11 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:22:11.173 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:11 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:22:11.173 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:11 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:22:11.173 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:11 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:22:11.173 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:11 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:22:11.173 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:11 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:22:11.173 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:11 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:22:11.173 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:11 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:22:11.173 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:11 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-28T00:22:11.173 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:11 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-28T00:22:11.173 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:11 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:22:11.173 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:11 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:22:11.173 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:11 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:22:11.174 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:11 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:22:11.174 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:11 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:22:11.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:11 smithi165 ceph-mon[36302]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-28T00:22:11.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:11 smithi165 ceph-mon[36302]: Detected new or changed devices on smithi119 2024-03-28T00:22:11.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:11 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:22:11.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:11 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:22:11.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:11 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:22:11.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:11 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:22:11.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:11 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:22:11.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:11 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:22:11.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:11 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:22:11.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:11 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:22:11.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:11 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:22:11.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:11 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:22:11.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:11 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-28T00:22:11.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:11 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-28T00:22:11.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:11 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:22:11.384 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:11 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:22:11.384 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:11 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:22:11.384 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:11 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:22:11.384 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:11 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:22:11.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:11 smithi119 ceph-mon[31518]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-28T00:22:11.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:11 smithi119 ceph-mon[31518]: Detected new or changed devices on smithi119 2024-03-28T00:22:11.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:11 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:22:11.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:11 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:22:11.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:11 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:22:11.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:11 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:22:11.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:11 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:22:11.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:11 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:22:11.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:11 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:22:11.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:11 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:22:11.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:11 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:22:11.407 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:11 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:22:11.407 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:11 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-28T00:22:11.407 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:11 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-28T00:22:11.407 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:11 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:22:11.407 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:11 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:22:11.407 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:11 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:22:11.407 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:11 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:22:11.407 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:11 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:22:12.193 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:12 smithi132 ceph-mon[35941]: Adjusting osd_memory_target on smithi119 to 4272M 2024-03-28T00:22:12.193 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:12 smithi132 ceph-mon[35941]: from='client.24359 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi132:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T00:22:12.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:12 smithi165 ceph-mon[36302]: Adjusting osd_memory_target on smithi119 to 4272M 2024-03-28T00:22:12.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:12 smithi165 ceph-mon[36302]: from='client.24359 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi132:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T00:22:12.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:12 smithi119 ceph-mon[31518]: Adjusting osd_memory_target on smithi119 to 4272M 2024-03-28T00:22:12.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:12 smithi119 ceph-mon[31518]: from='client.24359 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi132:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T00:22:13.307 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:13 smithi119 ceph-mon[31518]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-28T00:22:13.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:13 smithi165 ceph-mon[36302]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-28T00:22:13.392 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:13 smithi132 ceph-mon[35941]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-28T00:22:14.285 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:14 smithi119 ceph-mon[31518]: from='client.? 172.21.15.132:0/887743034' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f019acd0-915a-4094-b12c-b53672740a5e"}]: dispatch 2024-03-28T00:22:14.286 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:14 smithi119 ceph-mon[31518]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f019acd0-915a-4094-b12c-b53672740a5e"}]: dispatch 2024-03-28T00:22:14.286 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:14 smithi119 ceph-mon[31518]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f019acd0-915a-4094-b12c-b53672740a5e"}]': finished 2024-03-28T00:22:14.286 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:14 smithi119 ceph-mon[31518]: osdmap e69: 11 total, 10 up, 11 in 2024-03-28T00:22:14.286 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:14 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-28T00:22:14.286 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:14 smithi119 ceph-mon[31518]: from='client.? 172.21.15.132:0/2761742502' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-28T00:22:14.286 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:14 smithi119 ceph-mon[31518]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-28T00:22:14.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:14 smithi165 ceph-mon[36302]: from='client.? 172.21.15.132:0/887743034' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f019acd0-915a-4094-b12c-b53672740a5e"}]: dispatch 2024-03-28T00:22:14.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:14 smithi165 ceph-mon[36302]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f019acd0-915a-4094-b12c-b53672740a5e"}]: dispatch 2024-03-28T00:22:14.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:14 smithi165 ceph-mon[36302]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f019acd0-915a-4094-b12c-b53672740a5e"}]': finished 2024-03-28T00:22:14.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:14 smithi165 ceph-mon[36302]: osdmap e69: 11 total, 10 up, 11 in 2024-03-28T00:22:14.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:14 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-28T00:22:14.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:14 smithi165 ceph-mon[36302]: from='client.? 172.21.15.132:0/2761742502' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-28T00:22:14.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:14 smithi165 ceph-mon[36302]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-28T00:22:14.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:14 smithi132 ceph-mon[35941]: from='client.? 172.21.15.132:0/887743034' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f019acd0-915a-4094-b12c-b53672740a5e"}]: dispatch 2024-03-28T00:22:14.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:14 smithi132 ceph-mon[35941]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f019acd0-915a-4094-b12c-b53672740a5e"}]: dispatch 2024-03-28T00:22:14.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:14 smithi132 ceph-mon[35941]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f019acd0-915a-4094-b12c-b53672740a5e"}]': finished 2024-03-28T00:22:14.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:14 smithi132 ceph-mon[35941]: osdmap e69: 11 total, 10 up, 11 in 2024-03-28T00:22:14.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:14 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-28T00:22:14.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:14 smithi132 ceph-mon[35941]: from='client.? 172.21.15.132:0/2761742502' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-28T00:22:14.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:14 smithi132 ceph-mon[35941]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-28T00:22:16.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:16 smithi132 ceph-mon[35941]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-28T00:22:17.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:16 smithi165 ceph-mon[36302]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-28T00:22:17.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:16 smithi119 ceph-mon[31518]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-28T00:22:19.018 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:18 smithi132 ceph-mon[35941]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-28T00:22:19.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:18 smithi165 ceph-mon[36302]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-28T00:22:19.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:18 smithi119 ceph-mon[31518]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-28T00:22:21.069 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:20 smithi132 ceph-mon[35941]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-28T00:22:21.069 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:20 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-03-28T00:22:21.070 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:20 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:22:21.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:20 smithi165 ceph-mon[36302]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-28T00:22:21.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:20 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-03-28T00:22:21.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:20 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:22:21.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:20 smithi119 ceph-mon[31518]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-28T00:22:21.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:20 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-03-28T00:22:21.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:20 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:22:21.930 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:21 smithi132 ceph-mon[35941]: Deploying daemon osd.10 on smithi132 2024-03-28T00:22:22.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:21 smithi165 ceph-mon[36302]: Deploying daemon osd.10 on smithi132 2024-03-28T00:22:22.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:21 smithi119 ceph-mon[31518]: Deploying daemon osd.10 on smithi132 2024-03-28T00:22:22.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:22 smithi132 ceph-mon[35941]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-28T00:22:23.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:22 smithi165 ceph-mon[36302]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-28T00:22:23.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:22 smithi119 ceph-mon[31518]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-28T00:22:24.989 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:24 smithi132 ceph-mon[35941]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-28T00:22:25.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:24 smithi165 ceph-mon[36302]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-28T00:22:25.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:24 smithi119 ceph-mon[31518]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-28T00:22:27.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:26 smithi132 ceph-mon[35941]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-28T00:22:27.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:26 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:22:27.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:26 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:22:27.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:26 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:22:27.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:26 smithi165 ceph-mon[36302]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-28T00:22:27.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:26 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:22:27.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:26 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:22:27.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:26 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:22:27.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:26 smithi119 ceph-mon[31518]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-28T00:22:27.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:26 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:22:27.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:26 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:22:27.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:26 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:22:29.152 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:28 smithi132 ceph-mon[35941]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-28T00:22:29.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:28 smithi165 ceph-mon[36302]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-28T00:22:29.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:28 smithi119 ceph-mon[31518]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-28T00:22:29.964 INFO:teuthology.orchestra.run.smithi132.stdout:Created osd(s) 10 on host 'smithi132' 2024-03-28T00:22:29.964 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:22:29.958+0000 7fd4377fe700 1 -- 172.21.15.132:0/2103765824 <== mgr.14152 v2:172.21.15.119:6800/1401937096 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7fd4480625d0 con 0x7fd4300a2890 2024-03-28T00:22:29.967 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:22:29.961+0000 7fd44ca30700 1 -- 172.21.15.132:0/2103765824 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fd4300a2890 msgr2=0x7fd4300a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:22:29.967 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:22:29.961+0000 7fd44ca30700 1 --2- 172.21.15.132:0/2103765824 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fd4300a2890 0x7fd4300a4d50 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd448102110 tx=0x7fd438006d20 comp rx=0 tx=0).stop 2024-03-28T00:22:29.967 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:22:29.961+0000 7fd44ca30700 1 -- 172.21.15.132:0/2103765824 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd4481022b0 msgr2=0x7fd4481978f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:22:29.967 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:22:29.961+0000 7fd44ca30700 1 --2- 172.21.15.132:0/2103765824 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd4481022b0 0x7fd4481978f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fd43c00b530 tx=0x7fd43c00b8f0 comp rx=0 tx=0).stop 2024-03-28T00:22:29.967 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:22:29.961+0000 7fd44ca30700 1 -- 172.21.15.132:0/2103765824 shutdown_connections 2024-03-28T00:22:29.967 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:22:29.961+0000 7fd44ca30700 1 --2- 172.21.15.132:0/2103765824 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fd4300a2890 0x7fd4300a4d50 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:22:29.967 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:22:29.961+0000 7fd44ca30700 1 --2- 172.21.15.132:0/2103765824 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd448102c50 0x7fd44819e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:22:29.967 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:22:29.961+0000 7fd44ca30700 1 --2- 172.21.15.132:0/2103765824 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4481010b0 0x7fd4481973b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:22:29.968 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:22:29.961+0000 7fd44ca30700 1 --2- 172.21.15.132:0/2103765824 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd4481022b0 0x7fd4481978f0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:22:29.968 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:22:29.961+0000 7fd44ca30700 1 -- 172.21.15.132:0/2103765824 >> 172.21.15.132:0/2103765824 conn(0x7fd4480fc650 msgr2=0x7fd448105630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:22:29.968 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:22:29.961+0000 7fd44ca30700 1 -- 172.21.15.132:0/2103765824 shutdown_connections 2024-03-28T00:22:29.968 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-28T00:22:29.961+0000 7fd44ca30700 1 -- 172.21.15.132:0/2103765824 wait complete. 2024-03-28T00:22:30.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:30 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:22:30.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:30 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:22:30.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:30 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:22:30.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:30 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:22:30.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:30 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:22:30.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:30 smithi132 ceph-mon[35941]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-28T00:22:30.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:30 smithi132 ceph-mon[35941]: from='osd.10 [v2:172.21.15.132:6824/3454298492,v1:172.21.15.132:6825/3454298492]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-28T00:22:30.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:30 smithi132 ceph-mon[35941]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-28T00:22:30.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:30 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:22:30.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:30 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:22:30.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:30 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:22:30.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:30 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:22:30.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:30 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:22:30.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:30 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:22:30.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:30 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:22:30.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:30 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:22:30.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:30 smithi165 ceph-mon[36302]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-28T00:22:30.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:30 smithi165 ceph-mon[36302]: from='osd.10 [v2:172.21.15.132:6824/3454298492,v1:172.21.15.132:6825/3454298492]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-28T00:22:30.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:30 smithi165 ceph-mon[36302]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-28T00:22:30.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:30 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:22:30.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:30 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:22:30.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:30 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:22:30.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:30 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:22:30.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:30 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:22:30.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:30 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:22:30.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:30 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:22:30.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:30 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:22:30.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:30 smithi119 ceph-mon[31518]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-28T00:22:30.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:30 smithi119 ceph-mon[31518]: from='osd.10 [v2:172.21.15.132:6824/3454298492,v1:172.21.15.132:6825/3454298492]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-28T00:22:30.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:30 smithi119 ceph-mon[31518]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-28T00:22:30.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:30 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:22:30.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:30 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:22:30.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:30 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:22:30.573 DEBUG:teuthology.orchestra.run.smithi132:osd.10> sudo journalctl -f -n 0 -u ceph-10f73b92-ec98-11ee-95d0-87774f69a715@osd.10.service 2024-03-28T00:22:30.576 INFO:tasks.cephadm:Deploying osd.11 on smithi165 with /dev/vg_nvme/lv_1... 2024-03-28T00:22:30.576 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 10f73b92-ec98-11ee-95d0-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-03-28T00:22:30.813 INFO:teuthology.orchestra.run.smithi165.stderr:Inferring config /var/lib/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/mon.c/config 2024-03-28T00:22:31.270 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:31 smithi165 ceph-mon[36302]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-03-28T00:22:31.271 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:31 smithi165 ceph-mon[36302]: osdmap e70: 11 total, 10 up, 11 in 2024-03-28T00:22:31.271 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:31 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-28T00:22:31.271 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:31 smithi165 ceph-mon[36302]: from='osd.10 [v2:172.21.15.132:6824/3454298492,v1:172.21.15.132:6825/3454298492]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]: dispatch 2024-03-28T00:22:31.271 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:31 smithi165 ceph-mon[36302]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]: dispatch 2024-03-28T00:22:31.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:31 smithi119 ceph-mon[31518]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-03-28T00:22:31.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:31 smithi119 ceph-mon[31518]: osdmap e70: 11 total, 10 up, 11 in 2024-03-28T00:22:31.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:31 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-28T00:22:31.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:31 smithi119 ceph-mon[31518]: from='osd.10 [v2:172.21.15.132:6824/3454298492,v1:172.21.15.132:6825/3454298492]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]: dispatch 2024-03-28T00:22:31.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:31 smithi119 ceph-mon[31518]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]: dispatch 2024-03-28T00:22:31.427 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:31 smithi132 ceph-mon[35941]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-03-28T00:22:31.427 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:31 smithi132 ceph-mon[35941]: osdmap e70: 11 total, 10 up, 11 in 2024-03-28T00:22:31.427 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:31 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-28T00:22:31.427 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:31 smithi132 ceph-mon[35941]: from='osd.10 [v2:172.21.15.132:6824/3454298492,v1:172.21.15.132:6825/3454298492]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]: dispatch 2024-03-28T00:22:31.428 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:31 smithi132 ceph-mon[35941]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]: dispatch 2024-03-28T00:22:31.428 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 00:22:31 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T00:22:31.136+0000 7f0ea2f57700 -1 osd.10 0 waiting for initial osdmap 2024-03-28T00:22:31.428 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 00:22:31 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T00:22:31.149+0000 7f0e9ed4a700 -1 osd.10 71 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-28T00:22:32.205 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:32 smithi165 ceph-mon[36302]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]': finished 2024-03-28T00:22:32.205 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:32 smithi165 ceph-mon[36302]: osdmap e71: 11 total, 10 up, 11 in 2024-03-28T00:22:32.205 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:32 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-28T00:22:32.206 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:32 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-28T00:22:32.206 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:32 smithi165 ceph-mon[36302]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-28T00:22:32.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:32 smithi119 ceph-mon[31518]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]': finished 2024-03-28T00:22:32.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:32 smithi119 ceph-mon[31518]: osdmap e71: 11 total, 10 up, 11 in 2024-03-28T00:22:32.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:32 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-28T00:22:32.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:32 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-28T00:22:32.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:32 smithi119 ceph-mon[31518]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-28T00:22:32.452 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:32 smithi132 ceph-mon[35941]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]': finished 2024-03-28T00:22:32.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:32 smithi132 ceph-mon[35941]: osdmap e71: 11 total, 10 up, 11 in 2024-03-28T00:22:32.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:32 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-28T00:22:32.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:32 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-28T00:22:32.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:32 smithi132 ceph-mon[35941]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-28T00:22:32.889 INFO:teuthology.orchestra.run.smithi165.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-03-28T00:22:32.890 INFO:teuthology.orchestra.run.smithi165.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-03-28T00:22:32.890 INFO:teuthology.orchestra.run.smithi165.stderr: stderr: 10+0 records in 2024-03-28T00:22:32.890 INFO:teuthology.orchestra.run.smithi165.stderr:10+0 records out 2024-03-28T00:22:32.890 INFO:teuthology.orchestra.run.smithi165.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0115318 s, 909 MB/s 2024-03-28T00:22:32.890 INFO:teuthology.orchestra.run.smithi165.stderr:--> Zapping successful for: 2024-03-28T00:22:33.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:33 smithi119 ceph-mon[31518]: purged_snaps scrub starts 2024-03-28T00:22:33.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:33 smithi119 ceph-mon[31518]: purged_snaps scrub ok 2024-03-28T00:22:33.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:33 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-28T00:22:33.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:33 smithi119 ceph-mon[31518]: osd.10 [v2:172.21.15.132:6824/3454298492,v1:172.21.15.132:6825/3454298492] boot 2024-03-28T00:22:33.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:33 smithi119 ceph-mon[31518]: osdmap e72: 11 total, 11 up, 11 in 2024-03-28T00:22:33.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:33 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-28T00:22:33.452 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:33 smithi132 ceph-mon[35941]: purged_snaps scrub starts 2024-03-28T00:22:33.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:33 smithi132 ceph-mon[35941]: purged_snaps scrub ok 2024-03-28T00:22:33.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:33 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-28T00:22:33.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:33 smithi132 ceph-mon[35941]: osd.10 [v2:172.21.15.132:6824/3454298492,v1:172.21.15.132:6825/3454298492] boot 2024-03-28T00:22:33.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:33 smithi132 ceph-mon[35941]: osdmap e72: 11 total, 11 up, 11 in 2024-03-28T00:22:33.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:33 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-28T00:22:33.590 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:33 smithi165 ceph-mon[36302]: purged_snaps scrub starts 2024-03-28T00:22:33.590 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:33 smithi165 ceph-mon[36302]: purged_snaps scrub ok 2024-03-28T00:22:33.590 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:33 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-28T00:22:33.590 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:33 smithi165 ceph-mon[36302]: osd.10 [v2:172.21.15.132:6824/3454298492,v1:172.21.15.132:6825/3454298492] boot 2024-03-28T00:22:33.591 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:33 smithi165 ceph-mon[36302]: osdmap e72: 11 total, 11 up, 11 in 2024-03-28T00:22:33.591 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:33 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-28T00:22:33.617 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 10f73b92-ec98-11ee-95d0-87774f69a715 -- ceph orch daemon add osd smithi165:vg_nvme/lv_1 2024-03-28T00:22:33.865 INFO:teuthology.orchestra.run.smithi165.stderr:Inferring config /var/lib/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/mon.c/config 2024-03-28T00:22:34.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:34 smithi132 ceph-mon[35941]: osdmap e73: 11 total, 11 up, 11 in 2024-03-28T00:22:34.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:34 smithi132 ceph-mon[35941]: Detected new or changed devices on smithi132 2024-03-28T00:22:34.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:34 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:22:34.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:34 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:22:34.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:34 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:22:34.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:34 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:22:34.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:34 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:22:34.454 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:34 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:22:34.454 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:34 smithi132 ceph-mon[35941]: Adjusting osd_memory_target on smithi132 to 4270M 2024-03-28T00:22:34.454 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:34 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:22:34.454 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:34 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:22:34.454 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:34 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:22:34.454 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:34 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:22:34.454 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:34 smithi132 ceph-mon[35941]: pgmap v258: 1 pgs: 1 peering; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-03-28T00:22:34.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:34 smithi165 ceph-mon[36302]: osdmap e73: 11 total, 11 up, 11 in 2024-03-28T00:22:34.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:34 smithi165 ceph-mon[36302]: Detected new or changed devices on smithi132 2024-03-28T00:22:34.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:34 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:22:34.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:34 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:22:34.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:34 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:22:34.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:34 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:22:34.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:34 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:22:34.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:34 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:22:34.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:34 smithi165 ceph-mon[36302]: Adjusting osd_memory_target on smithi132 to 4270M 2024-03-28T00:22:34.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:34 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:22:34.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:34 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:22:34.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:34 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:22:34.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:34 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:22:34.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:34 smithi165 ceph-mon[36302]: pgmap v258: 1 pgs: 1 peering; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-03-28T00:22:34.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:34 smithi119 ceph-mon[31518]: osdmap e73: 11 total, 11 up, 11 in 2024-03-28T00:22:34.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:34 smithi119 ceph-mon[31518]: Detected new or changed devices on smithi132 2024-03-28T00:22:34.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:34 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:22:34.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:34 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:22:34.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:34 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:22:34.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:34 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:22:34.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:34 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:22:34.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:34 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:22:34.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:34 smithi119 ceph-mon[31518]: Adjusting osd_memory_target on smithi132 to 4270M 2024-03-28T00:22:34.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:34 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:22:34.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:34 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:22:34.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:34 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:22:34.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:34 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:22:34.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:34 smithi119 ceph-mon[31518]: pgmap v258: 1 pgs: 1 peering; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-03-28T00:22:35.276 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:35 smithi165 ceph-mon[36302]: osdmap e74: 11 total, 11 up, 11 in 2024-03-28T00:22:35.276 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:35 smithi165 ceph-mon[36302]: osdmap e75: 11 total, 11 up, 11 in 2024-03-28T00:22:35.452 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:35 smithi132 ceph-mon[35941]: osdmap e74: 11 total, 11 up, 11 in 2024-03-28T00:22:35.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:35 smithi132 ceph-mon[35941]: osdmap e75: 11 total, 11 up, 11 in 2024-03-28T00:22:35.533 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:22:35.524+0000 7fc8eeb16700 1 -- 172.21.15.165:0/3165789796 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc8e81010d0 msgr2=0x7fc8e81014f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:22:35.533 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:22:35.524+0000 7fc8eeb16700 1 --2- 172.21.15.165:0/3165789796 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc8e81010d0 0x7fc8e81014f0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc009990 tx=0x7fc8dc009ca0 comp rx=0 tx=0).stop 2024-03-28T00:22:35.533 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:22:35.524+0000 7fc8eeb16700 1 -- 172.21.15.165:0/3165789796 shutdown_connections 2024-03-28T00:22:35.534 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:22:35.524+0000 7fc8eeb16700 1 --2- 172.21.15.165:0/3165789796 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc8e8102c70 0x7fc8e8109520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:22:35.534 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:22:35.524+0000 7fc8eeb16700 1 --2- 172.21.15.165:0/3165789796 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc8e81010d0 0x7fc8e81014f0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:22:35.534 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:22:35.525+0000 7fc8eeb16700 1 --2- 172.21.15.165:0/3165789796 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc8e81022d0 0x7fc8e8102730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:22:35.534 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:22:35.525+0000 7fc8eeb16700 1 -- 172.21.15.165:0/3165789796 >> 172.21.15.165:0/3165789796 conn(0x7fc8e80fc690 msgr2=0x7fc8e80feab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:22:35.534 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:22:35.525+0000 7fc8eeb16700 1 -- 172.21.15.165:0/3165789796 shutdown_connections 2024-03-28T00:22:35.534 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:22:35.525+0000 7fc8eeb16700 1 -- 172.21.15.165:0/3165789796 wait complete. 2024-03-28T00:22:35.534 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:22:35.525+0000 7fc8eeb16700 1 Processor -- start 2024-03-28T00:22:35.534 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:22:35.525+0000 7fc8eeb16700 1 -- start start 2024-03-28T00:22:35.534 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:22:35.526+0000 7fc8eeb16700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc8e81010d0 0x7fc8e806b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:22:35.534 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:22:35.526+0000 7fc8eeb16700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc8e81022d0 0x7fc8e819d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:22:35.534 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:22:35.526+0000 7fc8eeb16700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc8e8102c70 0x7fc8e819f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:22:35.535 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:22:35.526+0000 7fc8eeb16700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8e810cd40 con 0x7fc8e81010d0 2024-03-28T00:22:35.535 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:22:35.526+0000 7fc8eeb16700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8e810cbc0 con 0x7fc8e81022d0 2024-03-28T00:22:35.535 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:22:35.526+0000 7fc8eeb16700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8e810cec0 con 0x7fc8e8102c70 2024-03-28T00:22:35.535 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:22:35.526+0000 7fc8ed0b3700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc8e8102c70 0x7fc8e819f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:22:35.535 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:22:35.526+0000 7fc8e7fff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc8e81022d0 0x7fc8e819d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:22:35.535 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:22:35.526+0000 7fc8ed0b3700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc8e8102c70 0x7fc8e819f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:46988/0 (socket says 172.21.15.165:46988) 2024-03-28T00:22:35.535 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:22:35.526+0000 7fc8ed0b3700 1 -- 172.21.15.165:0/2785539420 learned_addr learned my addr 172.21.15.165:0/2785539420 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-03-28T00:22:35.535 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:22:35.526+0000 7fc8ed0b3700 1 -- 172.21.15.165:0/2785539420 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc8e81022d0 msgr2=0x7fc8e819d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:22:35.535 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:22:35.526+0000 7fc8ed0b3700 1 --2- 172.21.15.165:0/2785539420 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc8e81022d0 0x7fc8e819d1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:22:35.535 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:22:35.526+0000 7fc8ed0b3700 1 -- 172.21.15.165:0/2785539420 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc8e81010d0 msgr2=0x7fc8e806b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:22:35.535 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:22:35.526+0000 7fc8ed0b3700 1 --2- 172.21.15.165:0/2785539420 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc8e81010d0 0x7fc8e806b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:22:35.536 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:22:35.526+0000 7fc8ed0b3700 1 -- 172.21.15.165:0/2785539420 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8e81a00e0 con 0x7fc8e8102c70 2024-03-28T00:22:35.536 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:22:35.526+0000 7fc8ed0b3700 1 --2- 172.21.15.165:0/2785539420 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc8e8102c70 0x7fc8e819f810 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fc8e000b970 tx=0x7fc8e000bc80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:22:35.536 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:22:35.526+0000 7fc8e5ffb700 1 -- 172.21.15.165:0/2785539420 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8e0019070 con 0x7fc8e8102c70 2024-03-28T00:22:35.536 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:22:35.526+0000 7fc8e5ffb700 1 -- 172.21.15.165:0/2785539420 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc8e000ce10 con 0x7fc8e8102c70 2024-03-28T00:22:35.536 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:22:35.526+0000 7fc8eeb16700 1 -- 172.21.15.165:0/2785539420 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8e81a03d0 con 0x7fc8e8102c70 2024-03-28T00:22:35.536 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:22:35.526+0000 7fc8e5ffb700 1 -- 172.21.15.165:0/2785539420 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8e0007370 con 0x7fc8e8102c70 2024-03-28T00:22:35.536 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:22:35.527+0000 7fc8eeb16700 1 -- 172.21.15.165:0/2785539420 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8e81a0990 con 0x7fc8e8102c70 2024-03-28T00:22:35.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:22:35.528+0000 7fc8eeb16700 1 -- 172.21.15.165:0/2785539420 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8e8068e50 con 0x7fc8e8102c70 2024-03-28T00:22:35.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:22:35.529+0000 7fc8e5ffb700 1 -- 172.21.15.165:0/2785539420 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fc8e001c070 con 0x7fc8e8102c70 2024-03-28T00:22:35.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:22:35.529+0000 7fc8e5ffb700 1 --2- 172.21.15.165:0/2785539420 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fc8d80a2890 0x7fc8d80a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:22:35.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:22:35.529+0000 7fc8e5ffb700 1 -- 172.21.15.165:0/2785539420 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6694+0+0 (secure 0 0 0) 0x7fc8e00cd780 con 0x7fc8e8102c70 2024-03-28T00:22:35.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:22:35.530+0000 7fc8ec8b2700 1 --2- 172.21.15.165:0/2785539420 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fc8d80a2890 0x7fc8d80a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:22:35.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:22:35.531+0000 7fc8ec8b2700 1 --2- 172.21.15.165:0/2785539420 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fc8d80a2890 0x7fc8d80a4d50 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc005f50 tx=0x7fc8dc005e80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:22:35.540 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:22:35.534+0000 7fc8e5ffb700 1 -- 172.21.15.165:0/2785539420 <== mon.1 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fc8e009ae70 con 0x7fc8e8102c70 2024-03-28T00:22:35.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:35 smithi119 ceph-mon[31518]: osdmap e74: 11 total, 11 up, 11 in 2024-03-28T00:22:35.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:35 smithi119 ceph-mon[31518]: osdmap e75: 11 total, 11 up, 11 in 2024-03-28T00:22:35.745 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:22:35.739+0000 7fc8eeb16700 1 -- 172.21.15.165:0/2785539420 --> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi165:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7fc8e80625d0 con 0x7fc8d80a2890 2024-03-28T00:22:36.688 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:36 smithi165 ceph-mon[36302]: osdmap e76: 11 total, 11 up, 11 in 2024-03-28T00:22:36.688 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:36 smithi165 ceph-mon[36302]: pgmap v262: 1 pgs: 1 peering; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-03-28T00:22:36.688 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:36 smithi165 ceph-mon[36302]: from='client.24376 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi165:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T00:22:36.688 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:36 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-28T00:22:36.688 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:36 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-28T00:22:36.688 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:36 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:22:36.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:36 smithi132 ceph-mon[35941]: osdmap e76: 11 total, 11 up, 11 in 2024-03-28T00:22:36.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:36 smithi132 ceph-mon[35941]: pgmap v262: 1 pgs: 1 peering; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-03-28T00:22:36.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:36 smithi132 ceph-mon[35941]: from='client.24376 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi165:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T00:22:36.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:36 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-28T00:22:36.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:36 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-28T00:22:36.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:36 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:22:36.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:36 smithi119 ceph-mon[31518]: osdmap e76: 11 total, 11 up, 11 in 2024-03-28T00:22:36.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:36 smithi119 ceph-mon[31518]: pgmap v262: 1 pgs: 1 peering; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-03-28T00:22:36.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:36 smithi119 ceph-mon[31518]: from='client.24376 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi165:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T00:22:36.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:36 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-28T00:22:36.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:36 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-28T00:22:36.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:36 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:22:39.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:38 smithi165 ceph-mon[36302]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 127 KiB/s, 0 objects/s recovering 2024-03-28T00:22:39.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:38 smithi165 ceph-mon[36302]: from='client.? 172.21.15.165:0/4249078313' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2c1e59a8-8f21-45c0-8568-4b632ce020f2"}]: dispatch 2024-03-28T00:22:39.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:38 smithi165 ceph-mon[36302]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2c1e59a8-8f21-45c0-8568-4b632ce020f2"}]: dispatch 2024-03-28T00:22:39.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:38 smithi165 ceph-mon[36302]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2c1e59a8-8f21-45c0-8568-4b632ce020f2"}]': finished 2024-03-28T00:22:39.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:38 smithi165 ceph-mon[36302]: osdmap e77: 12 total, 11 up, 12 in 2024-03-28T00:22:39.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:38 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-28T00:22:39.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:38 smithi119 ceph-mon[31518]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 127 KiB/s, 0 objects/s recovering 2024-03-28T00:22:39.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:38 smithi119 ceph-mon[31518]: from='client.? 172.21.15.165:0/4249078313' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2c1e59a8-8f21-45c0-8568-4b632ce020f2"}]: dispatch 2024-03-28T00:22:39.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:38 smithi119 ceph-mon[31518]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2c1e59a8-8f21-45c0-8568-4b632ce020f2"}]: dispatch 2024-03-28T00:22:39.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:38 smithi119 ceph-mon[31518]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2c1e59a8-8f21-45c0-8568-4b632ce020f2"}]': finished 2024-03-28T00:22:39.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:38 smithi119 ceph-mon[31518]: osdmap e77: 12 total, 11 up, 12 in 2024-03-28T00:22:39.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:38 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-28T00:22:39.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:38 smithi132 ceph-mon[35941]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 127 KiB/s, 0 objects/s recovering 2024-03-28T00:22:39.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:38 smithi132 ceph-mon[35941]: from='client.? 172.21.15.165:0/4249078313' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2c1e59a8-8f21-45c0-8568-4b632ce020f2"}]: dispatch 2024-03-28T00:22:39.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:38 smithi132 ceph-mon[35941]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2c1e59a8-8f21-45c0-8568-4b632ce020f2"}]: dispatch 2024-03-28T00:22:39.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:38 smithi132 ceph-mon[35941]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2c1e59a8-8f21-45c0-8568-4b632ce020f2"}]': finished 2024-03-28T00:22:39.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:38 smithi132 ceph-mon[35941]: osdmap e77: 12 total, 11 up, 12 in 2024-03-28T00:22:39.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:38 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-28T00:22:39.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:39 smithi132 ceph-mon[35941]: from='client.? 172.21.15.165:0/283890532' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-28T00:22:40.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:39 smithi165 ceph-mon[36302]: from='client.? 172.21.15.165:0/283890532' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-28T00:22:40.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:39 smithi119 ceph-mon[31518]: from='client.? 172.21.15.165:0/283890532' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-28T00:22:41.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:40 smithi165 ceph-mon[36302]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 104 KiB/s, 0 objects/s recovering 2024-03-28T00:22:41.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:40 smithi119 ceph-mon[31518]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 104 KiB/s, 0 objects/s recovering 2024-03-28T00:22:41.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:40 smithi132 ceph-mon[35941]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 104 KiB/s, 0 objects/s recovering 2024-03-28T00:22:43.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:42 smithi165 ceph-mon[36302]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-03-28T00:22:43.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:42 smithi119 ceph-mon[31518]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-03-28T00:22:43.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:42 smithi132 ceph-mon[35941]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-03-28T00:22:45.052 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:44 smithi165 ceph-mon[36302]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-03-28T00:22:45.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:44 smithi119 ceph-mon[31518]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-03-28T00:22:45.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:44 smithi132 ceph-mon[35941]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-03-28T00:22:46.835 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:46 smithi165 ceph-mon[36302]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-28T00:22:46.835 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:46 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-03-28T00:22:46.835 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:46 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:22:46.835 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:46 smithi165 ceph-mon[36302]: Deploying daemon osd.11 on smithi165 2024-03-28T00:22:47.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:46 smithi119 ceph-mon[31518]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-28T00:22:47.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:46 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-03-28T00:22:47.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:46 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:22:47.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:46 smithi119 ceph-mon[31518]: Deploying daemon osd.11 on smithi165 2024-03-28T00:22:47.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:46 smithi132 ceph-mon[35941]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-28T00:22:47.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:46 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-03-28T00:22:47.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:46 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:22:47.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:46 smithi132 ceph-mon[35941]: Deploying daemon osd.11 on smithi165 2024-03-28T00:22:48.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:48 smithi165 ceph-mon[36302]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-03-28T00:22:49.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:48 smithi119 ceph-mon[31518]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-03-28T00:22:49.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:48 smithi132 ceph-mon[35941]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-03-28T00:22:51.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:50 smithi165 ceph-mon[36302]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-03-28T00:22:51.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:50 smithi119 ceph-mon[31518]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-03-28T00:22:51.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:50 smithi132 ceph-mon[35941]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-03-28T00:22:53.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:52 smithi165 ceph-mon[36302]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-03-28T00:22:53.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:52 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:22:53.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:52 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:22:53.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:52 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:22:53.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:52 smithi119 ceph-mon[31518]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-03-28T00:22:53.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:52 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:22:53.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:52 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:22:53.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:52 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:22:53.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:52 smithi132 ceph-mon[35941]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-03-28T00:22:53.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:52 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:22:53.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:52 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:22:53.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:52 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:22:55.034 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:54 smithi165 ceph-mon[36302]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-03-28T00:22:55.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:54 smithi119 ceph-mon[31518]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-03-28T00:22:55.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:54 smithi132 ceph-mon[35941]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-03-28T00:22:55.431 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:22:55.425+0000 7fc8e5ffb700 1 -- 172.21.15.165:0/2785539420 <== mgr.14152 v2:172.21.15.119:6800/1401937096 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7fc8e80625d0 con 0x7fc8d80a2890 2024-03-28T00:22:55.431 INFO:teuthology.orchestra.run.smithi165.stdout:Created osd(s) 11 on host 'smithi165' 2024-03-28T00:22:55.433 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:22:55.428+0000 7fc8eeb16700 1 -- 172.21.15.165:0/2785539420 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fc8d80a2890 msgr2=0x7fc8d80a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:22:55.433 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:22:55.428+0000 7fc8eeb16700 1 --2- 172.21.15.165:0/2785539420 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fc8d80a2890 0x7fc8d80a4d50 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc005f50 tx=0x7fc8dc005e80 comp rx=0 tx=0).stop 2024-03-28T00:22:55.433 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:22:55.428+0000 7fc8eeb16700 1 -- 172.21.15.165:0/2785539420 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc8e8102c70 msgr2=0x7fc8e819f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:22:55.434 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:22:55.428+0000 7fc8eeb16700 1 --2- 172.21.15.165:0/2785539420 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc8e8102c70 0x7fc8e819f810 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fc8e000b970 tx=0x7fc8e000bc80 comp rx=0 tx=0).stop 2024-03-28T00:22:55.434 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:22:55.428+0000 7fc8eeb16700 1 -- 172.21.15.165:0/2785539420 shutdown_connections 2024-03-28T00:22:55.434 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:22:55.428+0000 7fc8eeb16700 1 --2- 172.21.15.165:0/2785539420 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fc8d80a2890 0x7fc8d80a4d50 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:22:55.434 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:22:55.428+0000 7fc8eeb16700 1 --2- 172.21.15.165:0/2785539420 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc8e8102c70 0x7fc8e819f810 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:22:55.434 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:22:55.428+0000 7fc8eeb16700 1 --2- 172.21.15.165:0/2785539420 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc8e81010d0 0x7fc8e806b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:22:55.434 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:22:55.428+0000 7fc8eeb16700 1 --2- 172.21.15.165:0/2785539420 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc8e81022d0 0x7fc8e819d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:22:55.434 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:22:55.428+0000 7fc8eeb16700 1 -- 172.21.15.165:0/2785539420 >> 172.21.15.165:0/2785539420 conn(0x7fc8e80fc690 msgr2=0x7fc8e8105650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:22:55.434 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:22:55.428+0000 7fc8eeb16700 1 -- 172.21.15.165:0/2785539420 shutdown_connections 2024-03-28T00:22:55.434 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:22:55.428+0000 7fc8eeb16700 1 -- 172.21.15.165:0/2785539420 wait complete. 2024-03-28T00:22:56.144 DEBUG:teuthology.orchestra.run.smithi165:osd.11> sudo journalctl -f -n 0 -u ceph-10f73b92-ec98-11ee-95d0-87774f69a715@osd.11.service 2024-03-28T00:22:56.146 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-03-28T00:22:56.147 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 10f73b92-ec98-11ee-95d0-87774f69a715 -- ceph osd stat -f json 2024-03-28T00:22:56.600 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/mon.a/config 2024-03-28T00:22:56.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:56 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:22:56.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:56 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:22:56.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:56 smithi119 ceph-mon[31518]: from='osd.11 [v2:172.21.15.165:6824/2138771601,v1:172.21.15.165:6825/2138771601]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-28T00:22:56.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:56 smithi119 ceph-mon[31518]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-28T00:22:56.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:56 smithi119 ceph-mon[31518]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-03-28T00:22:56.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:56 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:22:56.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:56 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:22:56.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:56 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:22:56.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:56 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:22:56.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:56 smithi132 ceph-mon[35941]: from='osd.11 [v2:172.21.15.165:6824/2138771601,v1:172.21.15.165:6825/2138771601]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-28T00:22:56.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:56 smithi132 ceph-mon[35941]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-28T00:22:56.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:56 smithi132 ceph-mon[35941]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-03-28T00:22:56.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:56 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:22:56.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:56 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:22:56.865 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:56 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:22:56.865 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:56 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:22:56.866 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:56 smithi165 ceph-mon[36302]: from='osd.11 [v2:172.21.15.165:6824/2138771601,v1:172.21.15.165:6825/2138771601]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-28T00:22:56.866 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:56 smithi165 ceph-mon[36302]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-28T00:22:56.866 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:56 smithi165 ceph-mon[36302]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-03-28T00:22:56.866 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:56 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:22:56.866 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:56 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:22:57.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:57 smithi165 ceph-mon[36302]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-03-28T00:22:57.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:57 smithi165 ceph-mon[36302]: osdmap e78: 12 total, 11 up, 12 in 2024-03-28T00:22:57.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:57 smithi165 ceph-mon[36302]: from='osd.11 [v2:172.21.15.165:6824/2138771601,v1:172.21.15.165:6825/2138771601]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]: dispatch 2024-03-28T00:22:57.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:57 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-28T00:22:57.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:57 smithi165 ceph-mon[36302]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]: dispatch 2024-03-28T00:22:57.668 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:57 smithi119 ceph-mon[31518]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-03-28T00:22:57.668 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:57 smithi119 ceph-mon[31518]: osdmap e78: 12 total, 11 up, 12 in 2024-03-28T00:22:57.668 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:57 smithi119 ceph-mon[31518]: from='osd.11 [v2:172.21.15.165:6824/2138771601,v1:172.21.15.165:6825/2138771601]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]: dispatch 2024-03-28T00:22:57.668 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:57 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-28T00:22:57.668 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:57 smithi119 ceph-mon[31518]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]: dispatch 2024-03-28T00:22:57.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:57 smithi132 ceph-mon[35941]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-03-28T00:22:57.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:57 smithi132 ceph-mon[35941]: osdmap e78: 12 total, 11 up, 12 in 2024-03-28T00:22:57.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:57 smithi132 ceph-mon[35941]: from='osd.11 [v2:172.21.15.165:6824/2138771601,v1:172.21.15.165:6825/2138771601]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]: dispatch 2024-03-28T00:22:57.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:57 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-28T00:22:57.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:57 smithi132 ceph-mon[35941]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]: dispatch 2024-03-28T00:22:58.082 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 00:22:57 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T00:22:57.817+0000 7f38ad517700 -1 osd.11 0 waiting for initial osdmap 2024-03-28T00:22:58.082 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 00:22:57 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T00:22:57.831+0000 7f38a5302700 -1 osd.11 79 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-28T00:22:58.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:58 smithi132 ceph-mon[35941]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]': finished 2024-03-28T00:22:58.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:58 smithi132 ceph-mon[35941]: osdmap e79: 12 total, 11 up, 12 in 2024-03-28T00:22:58.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:58 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-28T00:22:58.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:58 smithi132 ceph-mon[35941]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-03-28T00:22:58.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:58 smithi132 ceph-mon[35941]: from='osd.11 ' entity='osd.11' 2024-03-28T00:22:58.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:58 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-28T00:22:58.796 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:58 smithi119 ceph-mon[31518]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]': finished 2024-03-28T00:22:58.796 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:58 smithi119 ceph-mon[31518]: osdmap e79: 12 total, 11 up, 12 in 2024-03-28T00:22:58.796 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:58 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-28T00:22:58.796 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:58 smithi119 ceph-mon[31518]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-03-28T00:22:58.796 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:58 smithi119 ceph-mon[31518]: from='osd.11 ' entity='osd.11' 2024-03-28T00:22:58.796 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:58 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-28T00:22:58.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:58 smithi165 ceph-mon[36302]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]': finished 2024-03-28T00:22:58.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:58 smithi165 ceph-mon[36302]: osdmap e79: 12 total, 11 up, 12 in 2024-03-28T00:22:58.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:58 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-28T00:22:58.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:58 smithi165 ceph-mon[36302]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-03-28T00:22:58.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:58 smithi165 ceph-mon[36302]: from='osd.11 ' entity='osd.11' 2024-03-28T00:22:58.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:58 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-28T00:22:58.967 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:22:58.956+0000 7ff112e00700 1 -- 172.21.15.119:0/2791427017 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff10c1033c0 msgr2=0x7ff10c103820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:22:58.967 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:22:58.956+0000 7ff112e00700 1 --2- 172.21.15.119:0/2791427017 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff10c1033c0 0x7ff10c103820 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ff0f40099c0 tx=0x7ff0f4009cd0 comp rx=0 tx=0).stop 2024-03-28T00:22:58.967 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:22:58.956+0000 7ff112e00700 1 -- 172.21.15.119:0/2791427017 shutdown_connections 2024-03-28T00:22:58.967 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:22:58.956+0000 7ff112e00700 1 --2- 172.21.15.119:0/2791427017 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff10c103d60 0x7ff10c10a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:22:58.967 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:22:58.956+0000 7ff112e00700 1 --2- 172.21.15.119:0/2791427017 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff10c1021c0 0x7ff10c1025e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:22:58.967 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:22:58.956+0000 7ff112e00700 1 --2- 172.21.15.119:0/2791427017 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff10c1033c0 0x7ff10c103820 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:22:58.967 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:22:58.956+0000 7ff112e00700 1 -- 172.21.15.119:0/2791427017 >> 172.21.15.119:0/2791427017 conn(0x7ff10c0fd7a0 msgr2=0x7ff10c0ffbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:22:58.967 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:22:58.956+0000 7ff112e00700 1 -- 172.21.15.119:0/2791427017 shutdown_connections 2024-03-28T00:22:58.967 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:22:58.956+0000 7ff112e00700 1 -- 172.21.15.119:0/2791427017 wait complete. 2024-03-28T00:22:58.967 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:22:58.958+0000 7ff112e00700 1 Processor -- start 2024-03-28T00:22:58.967 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:22:58.958+0000 7ff112e00700 1 -- start start 2024-03-28T00:22:58.967 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:22:58.958+0000 7ff112e00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff10c1021c0 0x7ff10c1983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:22:58.968 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:22:58.958+0000 7ff112e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff10c1033c0 0x7ff10c198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:22:58.968 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:22:58.958+0000 7ff112e00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff10c103d60 0x7ff10c19f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:22:58.968 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:22:58.958+0000 7ff112e00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff10c10ddd0 con 0x7ff10c1033c0 2024-03-28T00:22:58.968 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:22:58.958+0000 7ff112e00700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff10c10dc50 con 0x7ff10c103d60 2024-03-28T00:22:58.968 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:22:58.958+0000 7ff112e00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff10c10df50 con 0x7ff10c1021c0 2024-03-28T00:22:58.968 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:22:58.958+0000 7ff10be00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff10c1021c0 0x7ff10c1983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:22:58.968 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:22:58.959+0000 7ff10be00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff10c1021c0 0x7ff10c1983e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.119:38888/0 (socket says 172.21.15.119:38888) 2024-03-28T00:22:58.968 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:22:58.959+0000 7ff10be00700 1 -- 172.21.15.119:0/2164990524 learned_addr learned my addr 172.21.15.119:0/2164990524 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:22:58.968 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:22:58.959+0000 7ff110e00700 1 --2- 172.21.15.119:0/2164990524 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff10c103d60 0x7ff10c19f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:22:58.968 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:22:58.959+0000 7ff103400700 1 --2- 172.21.15.119:0/2164990524 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff10c1033c0 0x7ff10c198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:22:58.968 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:22:58.959+0000 7ff10be00700 1 -- 172.21.15.119:0/2164990524 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff10c103d60 msgr2=0x7ff10c19f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:22:58.968 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:22:58.959+0000 7ff10be00700 1 --2- 172.21.15.119:0/2164990524 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff10c103d60 0x7ff10c19f9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:22:58.968 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:22:58.959+0000 7ff10be00700 1 -- 172.21.15.119:0/2164990524 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff10c1033c0 msgr2=0x7ff10c198920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:22:58.969 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:22:58.959+0000 7ff10be00700 1 --2- 172.21.15.119:0/2164990524 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff10c1033c0 0x7ff10c198920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:22:58.969 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:22:58.959+0000 7ff10be00700 1 -- 172.21.15.119:0/2164990524 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff10c1a0040 con 0x7ff10c1021c0 2024-03-28T00:22:58.969 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:22:58.959+0000 7ff103400700 1 --2- 172.21.15.119:0/2164990524 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff10c1033c0 0x7ff10c198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T00:22:58.969 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:22:58.959+0000 7ff10be00700 1 --2- 172.21.15.119:0/2164990524 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff10c1021c0 0x7ff10c1983e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff0fc009f40 tx=0x7ff0fc00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:22:58.969 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:22:58.959+0000 7ff109600700 1 -- 172.21.15.119:0/2164990524 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0fc0092e0 con 0x7ff10c1021c0 2024-03-28T00:22:58.969 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:22:58.959+0000 7ff109600700 1 -- 172.21.15.119:0/2164990524 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff0fc003a00 con 0x7ff10c1021c0 2024-03-28T00:22:58.969 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:22:58.960+0000 7ff112e00700 1 -- 172.21.15.119:0/2164990524 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff10c1a02d0 con 0x7ff10c1021c0 2024-03-28T00:22:58.969 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:22:58.960+0000 7ff109600700 1 -- 172.21.15.119:0/2164990524 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0fc004680 con 0x7ff10c1021c0 2024-03-28T00:22:58.969 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:22:58.960+0000 7ff112e00700 1 -- 172.21.15.119:0/2164990524 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff10c1a0860 con 0x7ff10c1021c0 2024-03-28T00:22:58.969 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:22:58.961+0000 7ff112e00700 1 -- 172.21.15.119:0/2164990524 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff10c068e50 con 0x7ff10c1021c0 2024-03-28T00:22:58.970 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:22:58.963+0000 7ff109600700 1 -- 172.21.15.119:0/2164990524 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7ff0fc01b030 con 0x7ff10c1021c0 2024-03-28T00:22:58.970 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:22:58.964+0000 7ff109600700 1 --2- 172.21.15.119:0/2164990524 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7ff0f00a2910 0x7ff0f00a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:22:58.970 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:22:58.964+0000 7ff103400700 1 --2- 172.21.15.119:0/2164990524 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7ff0f00a2910 0x7ff0f00a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:22:58.970 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:22:58.964+0000 7ff109600700 1 -- 172.21.15.119:0/2164990524 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7ff0fc0ce660 con 0x7ff10c1021c0 2024-03-28T00:22:58.970 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:22:58.965+0000 7ff103400700 1 --2- 172.21.15.119:0/2164990524 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7ff0f00a2910 0x7ff0f00a4dd0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7ff0f4000c00 tx=0x7ff0f4005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:22:58.972 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:22:58.966+0000 7ff109600700 1 -- 172.21.15.119:0/2164990524 <== mon.1 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff0fc022020 con 0x7ff10c1021c0 2024-03-28T00:22:59.199 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:22:59.194+0000 7ff112e00700 1 -- 172.21.15.119:0/2164990524 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7ff10c062da0 con 0x7ff10c1021c0 2024-03-28T00:22:59.200 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:22:59.194+0000 7ff109600700 1 -- 172.21.15.119:0/2164990524 <== mon.1 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v80) v1 ==== 74+0+133 (secure 0 0 0) 0x7ff0fc010040 con 0x7ff10c1021c0 2024-03-28T00:22:59.200 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-28T00:22:59.202 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:22:59.197+0000 7ff112e00700 1 -- 172.21.15.119:0/2164990524 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7ff0f00a2910 msgr2=0x7ff0f00a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:22:59.202 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:22:59.197+0000 7ff112e00700 1 --2- 172.21.15.119:0/2164990524 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7ff0f00a2910 0x7ff0f00a4dd0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7ff0f4000c00 tx=0x7ff0f4005fd0 comp rx=0 tx=0).stop 2024-03-28T00:22:59.203 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:22:59.197+0000 7ff112e00700 1 -- 172.21.15.119:0/2164990524 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff10c1021c0 msgr2=0x7ff10c1983e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:22:59.203 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:22:59.197+0000 7ff112e00700 1 --2- 172.21.15.119:0/2164990524 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff10c1021c0 0x7ff10c1983e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff0fc009f40 tx=0x7ff0fc00c3a0 comp rx=0 tx=0).stop 2024-03-28T00:22:59.203 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:22:59.198+0000 7ff112e00700 1 -- 172.21.15.119:0/2164990524 shutdown_connections 2024-03-28T00:22:59.203 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:22:59.198+0000 7ff112e00700 1 --2- 172.21.15.119:0/2164990524 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7ff0f00a2910 0x7ff0f00a4dd0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:22:59.203 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:22:59.198+0000 7ff112e00700 1 --2- 172.21.15.119:0/2164990524 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff10c103d60 0x7ff10c19f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:22:59.203 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:22:59.198+0000 7ff112e00700 1 --2- 172.21.15.119:0/2164990524 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff10c1021c0 0x7ff10c1983e0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:22:59.203 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:22:59.198+0000 7ff112e00700 1 --2- 172.21.15.119:0/2164990524 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff10c1033c0 0x7ff10c198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:22:59.203 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:22:59.198+0000 7ff112e00700 1 -- 172.21.15.119:0/2164990524 >> 172.21.15.119:0/2164990524 conn(0x7ff10c0fd7a0 msgr2=0x7ff10c1066c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:22:59.204 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:22:59.198+0000 7ff112e00700 1 -- 172.21.15.119:0/2164990524 shutdown_connections 2024-03-28T00:22:59.204 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:22:59.198+0000 7ff112e00700 1 -- 172.21.15.119:0/2164990524 wait complete. 2024-03-28T00:22:59.548 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:59 smithi119 ceph-mon[31518]: purged_snaps scrub starts 2024-03-28T00:22:59.548 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:59 smithi119 ceph-mon[31518]: purged_snaps scrub ok 2024-03-28T00:22:59.548 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:59 smithi119 ceph-mon[31518]: osd.11 [v2:172.21.15.165:6824/2138771601,v1:172.21.15.165:6825/2138771601] boot 2024-03-28T00:22:59.549 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:59 smithi119 ceph-mon[31518]: osdmap e80: 12 total, 12 up, 12 in 2024-03-28T00:22:59.549 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:59 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-28T00:22:59.549 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:22:59 smithi119 ceph-mon[31518]: from='client.? 172.21.15.119:0/2164990524' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-28T00:22:59.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:59 smithi132 ceph-mon[35941]: purged_snaps scrub starts 2024-03-28T00:22:59.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:59 smithi132 ceph-mon[35941]: purged_snaps scrub ok 2024-03-28T00:22:59.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:59 smithi132 ceph-mon[35941]: osd.11 [v2:172.21.15.165:6824/2138771601,v1:172.21.15.165:6825/2138771601] boot 2024-03-28T00:22:59.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:59 smithi132 ceph-mon[35941]: osdmap e80: 12 total, 12 up, 12 in 2024-03-28T00:22:59.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:59 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-28T00:22:59.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:22:59 smithi132 ceph-mon[35941]: from='client.? 172.21.15.119:0/2164990524' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-28T00:22:59.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:59 smithi165 ceph-mon[36302]: purged_snaps scrub starts 2024-03-28T00:22:59.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:59 smithi165 ceph-mon[36302]: purged_snaps scrub ok 2024-03-28T00:22:59.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:59 smithi165 ceph-mon[36302]: osd.11 [v2:172.21.15.165:6824/2138771601,v1:172.21.15.165:6825/2138771601] boot 2024-03-28T00:22:59.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:59 smithi165 ceph-mon[36302]: osdmap e80: 12 total, 12 up, 12 in 2024-03-28T00:22:59.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:59 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-28T00:22:59.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:22:59 smithi165 ceph-mon[36302]: from='client.? 172.21.15.119:0/2164990524' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-28T00:23:00.084 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":80,"num_osds":12,"num_up_osds":12,"osd_up_since":1711585378,"num_in_osds":12,"osd_in_since":1711585358,"num_remapped_pgs":0} 2024-03-28T00:23:00.085 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 10f73b92-ec98-11ee-95d0-87774f69a715 -- ceph osd dump --format=json 2024-03-28T00:23:00.494 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/mon.a/config 2024-03-28T00:23:00.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:00 smithi132 ceph-mon[35941]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T00:23:00.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:00 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:00.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:00 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:00.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:00 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:23:00.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:00 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:23:00.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:00 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:23:00.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:00 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:23:00.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:00 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:00.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:00 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:23:00.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:00 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:23:00.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:00 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:00.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:00 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:23:00.824 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:00 smithi119 ceph-mon[31518]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T00:23:00.824 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:00 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:00.824 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:00 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:00.824 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:00 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:23:00.824 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:00 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:23:00.824 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:00 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:23:00.825 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:00 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:23:00.825 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:00 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:00.825 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:00 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:23:00.825 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:00 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:23:00.825 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:00 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:00.825 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:00 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:23:00.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:00 smithi165 ceph-mon[36302]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T00:23:00.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:00 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:00.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:00 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:00.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:00 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:23:00.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:00 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:23:00.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:00 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:23:00.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:00 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:23:00.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:00 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:00.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:00 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:23:00.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:00 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:23:00.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:00 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:00.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:00 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:23:01.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:01 smithi132 ceph-mon[35941]: Detected new or changed devices on smithi165 2024-03-28T00:23:01.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:01 smithi132 ceph-mon[35941]: Adjusting osd_memory_target on smithi165 to 4270M 2024-03-28T00:23:01.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:01 smithi132 ceph-mon[35941]: osdmap e81: 12 total, 12 up, 12 in 2024-03-28T00:23:01.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:01 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:23:01.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:01 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:23:01.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:01 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:01.796 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:01 smithi119 ceph-mon[31518]: Detected new or changed devices on smithi165 2024-03-28T00:23:01.796 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:01 smithi119 ceph-mon[31518]: Adjusting osd_memory_target on smithi165 to 4270M 2024-03-28T00:23:01.796 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:01 smithi119 ceph-mon[31518]: osdmap e81: 12 total, 12 up, 12 in 2024-03-28T00:23:01.796 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:01 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:23:01.796 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:01 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:23:01.796 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:01 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:01.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:01 smithi165 ceph-mon[36302]: Detected new or changed devices on smithi165 2024-03-28T00:23:01.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:01 smithi165 ceph-mon[36302]: Adjusting osd_memory_target on smithi165 to 4270M 2024-03-28T00:23:01.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:01 smithi165 ceph-mon[36302]: osdmap e81: 12 total, 12 up, 12 in 2024-03-28T00:23:01.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:01 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:23:01.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:01 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:23:01.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:01 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:02.626 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:02.620+0000 7f0ff9000700 1 -- 172.21.15.119:0/4197690497 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0ff4102440 msgr2=0x7f0ff4102820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:02.626 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:02.620+0000 7f0ff9000700 1 --2- 172.21.15.119:0/4197690497 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0ff4102440 0x7f0ff4102820 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f0fe4009990 tx=0x7f0fe4009ca0 comp rx=0 tx=0).stop 2024-03-28T00:23:02.626 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:02.620+0000 7f0ff9000700 1 -- 172.21.15.119:0/4197690497 shutdown_connections 2024-03-28T00:23:02.626 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:02.620+0000 7f0ff9000700 1 --2- 172.21.15.119:0/4197690497 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ff4107510 0x7f0ff410f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:02.626 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:02.620+0000 7f0ff9000700 1 --2- 172.21.15.119:0/4197690497 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0ff4102440 0x7f0ff4102820 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:02.626 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:02.620+0000 7f0ff9000700 1 --2- 172.21.15.119:0/4197690497 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0ff4102df0 0x7f0ff4106e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:02.626 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:02.620+0000 7f0ff9000700 1 -- 172.21.15.119:0/4197690497 >> 172.21.15.119:0/4197690497 conn(0x7f0ff40792f0 msgr2=0x7f0ff4079700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:23:02.626 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:02.620+0000 7f0ff9000700 1 -- 172.21.15.119:0/4197690497 shutdown_connections 2024-03-28T00:23:02.626 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:02.620+0000 7f0ff9000700 1 -- 172.21.15.119:0/4197690497 wait complete. 2024-03-28T00:23:02.627 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:02.622+0000 7f0ff9000700 1 Processor -- start 2024-03-28T00:23:02.627 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:02.622+0000 7f0ff9000700 1 -- start start 2024-03-28T00:23:02.628 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:02.623+0000 7f0ff9000700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ff4102440 0x7f0ff41a0880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:23:02.628 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:02.623+0000 7f0ff9000700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0ff4102df0 0x7f0ff41a0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:23:02.628 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:02.623+0000 7f0ff9000700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0ff4107510 0x7f0ff419aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:23:02.628 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:02.623+0000 7f0ff9000700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ff4112540 con 0x7f0ff4102440 2024-03-28T00:23:02.628 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:02.623+0000 7f0ff9000700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ff41123c0 con 0x7f0ff4102df0 2024-03-28T00:23:02.629 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:02.623+0000 7f0ff9000700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ff41126c0 con 0x7f0ff4107510 2024-03-28T00:23:02.629 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:02.623+0000 7f0ff1e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ff4102440 0x7f0ff41a0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:23:02.629 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:02.623+0000 7f0ff1e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ff4102440 0x7f0ff41a0880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43182/0 (socket says 172.21.15.119:43182) 2024-03-28T00:23:02.629 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:02.623+0000 7f0ff1e00700 1 -- 172.21.15.119:0/2563876857 learned_addr learned my addr 172.21.15.119:0/2563876857 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:23:02.629 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:02.623+0000 7f0febe00700 1 --2- 172.21.15.119:0/2563876857 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0ff4102df0 0x7f0ff41a0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:23:02.629 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:02.623+0000 7f0ff2800700 1 --2- 172.21.15.119:0/2563876857 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0ff4107510 0x7f0ff419aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:23:02.630 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:02.624+0000 7f0ff1e00700 1 -- 172.21.15.119:0/2563876857 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0ff4107510 msgr2=0x7f0ff419aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:02.630 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:02.624+0000 7f0ff1e00700 1 --2- 172.21.15.119:0/2563876857 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0ff4107510 0x7f0ff419aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:02.630 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:02.624+0000 7f0ff1e00700 1 -- 172.21.15.119:0/2563876857 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0ff4102df0 msgr2=0x7f0ff41a0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:02.630 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:02.624+0000 7f0ff1e00700 1 --2- 172.21.15.119:0/2563876857 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0ff4102df0 0x7f0ff41a0dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:02.630 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:02.624+0000 7f0ff1e00700 1 -- 172.21.15.119:0/2563876857 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ff419b1a0 con 0x7f0ff4102440 2024-03-28T00:23:02.630 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:02.624+0000 7f0febe00700 1 --2- 172.21.15.119:0/2563876857 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0ff4102df0 0x7f0ff41a0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T00:23:02.630 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:02.624+0000 7f0ff1e00700 1 --2- 172.21.15.119:0/2563876857 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ff4102440 0x7f0ff41a0880 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f0fe4000c00 tx=0x7f0fe4005590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:23:02.631 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:02.624+0000 7f0feaa00700 1 -- 172.21.15.119:0/2563876857 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fe4027070 con 0x7f0ff4102440 2024-03-28T00:23:02.631 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:02.624+0000 7f0feaa00700 1 -- 172.21.15.119:0/2563876857 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0fe400fda0 con 0x7f0ff4102440 2024-03-28T00:23:02.631 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:02.624+0000 7f0ff9000700 1 -- 172.21.15.119:0/2563876857 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ff419b430 con 0x7f0ff4102440 2024-03-28T00:23:02.631 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:02.624+0000 7f0feaa00700 1 -- 172.21.15.119:0/2563876857 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fe40173f0 con 0x7f0ff4102440 2024-03-28T00:23:02.631 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:02.625+0000 7f0ff9000700 1 -- 172.21.15.119:0/2563876857 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ff41a7750 con 0x7f0ff4102440 2024-03-28T00:23:02.631 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:02.626+0000 7f0ff9000700 1 -- 172.21.15.119:0/2563876857 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ff4103910 con 0x7f0ff4102440 2024-03-28T00:23:02.633 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:02.628+0000 7f0feaa00700 1 -- 172.21.15.119:0/2563876857 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f0fe402a070 con 0x7f0ff4102440 2024-03-28T00:23:02.634 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:02.629+0000 7f0feaa00700 1 --2- 172.21.15.119:0/2563876857 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f0fd80a2890 0x7f0fd80a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:23:02.634 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:02.629+0000 7f0feaa00700 1 -- 172.21.15.119:0/2563876857 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f0fe40cf5f0 con 0x7f0ff4102440 2024-03-28T00:23:02.635 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:02.629+0000 7f0febe00700 1 --2- 172.21.15.119:0/2563876857 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f0fd80a2890 0x7f0fd80a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:23:02.635 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:02.630+0000 7f0febe00700 1 --2- 172.21.15.119:0/2563876857 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f0fd80a2890 0x7f0fd80a4d50 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f0ff4101010 tx=0x7f0fdc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:23:02.637 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:02.632+0000 7f0feaa00700 1 -- 172.21.15.119:0/2563876857 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f0fe4031080 con 0x7f0ff4102440 2024-03-28T00:23:02.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:02 smithi132 ceph-mon[35941]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T00:23:02.857 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:02.852+0000 7f0ff9000700 1 -- 172.21.15.119:0/2563876857 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0ff4068e50 con 0x7f0ff4102440 2024-03-28T00:23:02.858 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:02 smithi119 ceph-mon[31518]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T00:23:02.862 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:02.857+0000 7f0feaa00700 1 -- 172.21.15.119:0/2563876857 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+18537 (secure 0 0 0) 0x7f0fe4013070 con 0x7f0ff4102440 2024-03-28T00:23:02.862 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-28T00:23:02.863 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":81,"fsid":"10f73b92-ec98-11ee-95d0-87774f69a715","created":"2024-03-28T00:14:57.563604+0000","modified":"2024-03-28T00:23:00.426768+0000","last_up_change":"2024-03-28T00:22:58.821662+0000","last_in_change":"2024-03-28T00:22:38.467270+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T00:18:46.253414+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c0b875d-a01e-4294-9c9c-5d0b048b8912","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":580396252},{"type":"v1","addr":"172.21.15.119:6803","nonce":580396252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":580396252},{"type":"v1","addr":"172.21.15.119:6805","nonce":580396252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":580396252},{"type":"v1","addr":"172.21.15.119:6809","nonce":580396252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":580396252},{"type":"v1","addr":"172.21.15.119:6807","nonce":580396252}]},"public_addr":"172.21.15.119:6803/580396252","cluster_addr":"172.21.15.119:6805/580396252","heartbeat_back_addr":"172.21.15.119:6809/580396252","heartbeat_front_addr":"172.21.15.119:6807/580396252","state":["exists","up"]},{"osd":1,"uuid":"5567f7c2-8ea4-4789-9ea1-467793324fb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":2500318888},{"type":"v1","addr":"172.21.15.132:6801","nonce":2500318888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":2500318888},{"type":"v1","addr":"172.21.15.132:6803","nonce":2500318888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":2500318888},{"type":"v1","addr":"172.21.15.132:6807","nonce":2500318888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":2500318888},{"type":"v1","addr":"172.21.15.132:6805","nonce":2500318888}]},"public_addr":"172.21.15.132:6801/2500318888","cluster_addr":"172.21.15.132:6803/2500318888","heartbeat_back_addr":"172.21.15.132:6807/2500318888","heartbeat_front_addr":"172.21.15.132:6805/2500318888","state":["exists","up"]},{"osd":2,"uuid":"ee54dbd0-24b4-4ee6-988c-ce71491199d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":2361434832},{"type":"v1","addr":"172.21.15.165:6801","nonce":2361434832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":2361434832},{"type":"v1","addr":"172.21.15.165:6803","nonce":2361434832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":2361434832},{"type":"v1","addr":"172.21.15.165:6807","nonce":2361434832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":2361434832},{"type":"v1","addr":"172.21.15.165:6805","nonce":2361434832}]},"public_addr":"172.21.15.165:6801/2361434832","cluster_addr":"172.21.15.165:6803/2361434832","heartbeat_back_addr":"172.21.15.165:6807/2361434832","heartbeat_front_addr":"172.21.15.165:6805/2361434832","state":["exists","up"]},{"osd":3,"uuid":"0464c40b-828f-4db4-96d9-3f7a6325e6da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":3732272210},{"type":"v1","addr":"172.21.15.119:6811","nonce":3732272210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":3732272210},{"type":"v1","addr":"172.21.15.119:6813","nonce":3732272210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":3732272210},{"type":"v1","addr":"172.21.15.119:6817","nonce":3732272210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":3732272210},{"type":"v1","addr":"172.21.15.119:6815","nonce":3732272210}]},"public_addr":"172.21.15.119:6811/3732272210","cluster_addr":"172.21.15.119:6813/3732272210","heartbeat_back_addr":"172.21.15.119:6817/3732272210","heartbeat_front_addr":"172.21.15.119:6815/3732272210","state":["exists","up"]},{"osd":4,"uuid":"8fd70b8e-5128-4173-aa0e-e735dfeb995d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":2803039505},{"type":"v1","addr":"172.21.15.132:6809","nonce":2803039505}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":2803039505},{"type":"v1","addr":"172.21.15.132:6811","nonce":2803039505}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":2803039505},{"type":"v1","addr":"172.21.15.132:6815","nonce":2803039505}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":2803039505},{"type":"v1","addr":"172.21.15.132:6813","nonce":2803039505}]},"public_addr":"172.21.15.132:6809/2803039505","cluster_addr":"172.21.15.132:6811/2803039505","heartbeat_back_addr":"172.21.15.132:6815/2803039505","heartbeat_front_addr":"172.21.15.132:6813/2803039505","state":["exists","up"]},{"osd":5,"uuid":"3224b4c1-733e-4424-a6da-870bbb5f94e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":2561252206},{"type":"v1","addr":"172.21.15.165:6809","nonce":2561252206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":2561252206},{"type":"v1","addr":"172.21.15.165:6811","nonce":2561252206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":2561252206},{"type":"v1","addr":"172.21.15.165:6815","nonce":2561252206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":2561252206},{"type":"v1","addr":"172.21.15.165:6813","nonce":2561252206}]},"public_addr":"172.21.15.165:6809/2561252206","cluster_addr":"172.21.15.165:6811/2561252206","heartbeat_back_addr":"172.21.15.165:6815/2561252206","heartbeat_front_addr":"172.21.15.165:6813/2561252206","state":["exists","up"]},{"osd":6,"uuid":"e435745d-59b5-439a-9091-0946ca0e9332","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":2545410230},{"type":"v1","addr":"172.21.15.119:6819","nonce":2545410230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":2545410230},{"type":"v1","addr":"172.21.15.119:6821","nonce":2545410230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":2545410230},{"type":"v1","addr":"172.21.15.119:6825","nonce":2545410230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":2545410230},{"type":"v1","addr":"172.21.15.119:6823","nonce":2545410230}]},"public_addr":"172.21.15.119:6819/2545410230","cluster_addr":"172.21.15.119:6821/2545410230","heartbeat_back_addr":"172.21.15.119:6825/2545410230","heartbeat_front_addr":"172.21.15.119:6823/2545410230","state":["exists","up"]},{"osd":7,"uuid":"79b97a12-f6b4-48c3-b517-2dd6b6e7c85f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":2877892376},{"type":"v1","addr":"172.21.15.132:6817","nonce":2877892376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":2877892376},{"type":"v1","addr":"172.21.15.132:6819","nonce":2877892376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":2877892376},{"type":"v1","addr":"172.21.15.132:6823","nonce":2877892376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":2877892376},{"type":"v1","addr":"172.21.15.132:6821","nonce":2877892376}]},"public_addr":"172.21.15.132:6817/2877892376","cluster_addr":"172.21.15.132:6819/2877892376","heartbeat_back_addr":"172.21.15.132:6823/2877892376","heartbeat_front_addr":"172.21.15.132:6821/2877892376","state":["exists","up"]},{"osd":8,"uuid":"2970524c-b99a-412d-85c2-6c29b54035e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":4115226150},{"type":"v1","addr":"172.21.15.165:6817","nonce":4115226150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":4115226150},{"type":"v1","addr":"172.21.15.165:6819","nonce":4115226150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":4115226150},{"type":"v1","addr":"172.21.15.165:6823","nonce":4115226150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":4115226150},{"type":"v1","addr":"172.21.15.165:6821","nonce":4115226150}]},"public_addr":"172.21.15.165:6817/4115226150","cluster_addr":"172.21.15.165:6819/4115226150","heartbeat_back_addr":"172.21.15.165:6823/4115226150","heartbeat_front_addr":"172.21.15.165:6821/4115226150","state":["exists","up"]},{"osd":9,"uuid":"2b1a08ed-cde9-4def-b3e8-838fff22709c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":2532245222},{"type":"v1","addr":"172.21.15.119:6827","nonce":2532245222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":2532245222},{"type":"v1","addr":"172.21.15.119:6829","nonce":2532245222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":2532245222},{"type":"v1","addr":"172.21.15.119:6833","nonce":2532245222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":2532245222},{"type":"v1","addr":"172.21.15.119:6831","nonce":2532245222}]},"public_addr":"172.21.15.119:6827/2532245222","cluster_addr":"172.21.15.119:6829/2532245222","heartbeat_back_addr":"172.21.15.119:6833/2532245222","heartbeat_front_addr":"172.21.15.119:6831/2532245222","state":["exists","up"]},{"osd":10,"uuid":"f019acd0-915a-4094-b12c-b53672740a5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":3454298492},{"type":"v1","addr":"172.21.15.132:6825","nonce":3454298492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":3454298492},{"type":"v1","addr":"172.21.15.132:6827","nonce":3454298492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":3454298492},{"type":"v1","addr":"172.21.15.132:6831","nonce":3454298492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":3454298492},{"type":"v1","addr":"172.21.15.132:6829","nonce":3454298492}]},"public_addr":"172.21.15.132:6825/3454298492","cluster_addr":"172.21.15.132:6827/3454298492","heartbeat_back_addr":"172.21.15.132:6831/3454298492","heartbeat_front_addr":"172.21.15.132:6829/3454298492","state":["exists","up"]},{"osd":11,"uuid":"2c1e59a8-8f21-45c0-8568-4b632ce020f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":2138771601},{"type":"v1","addr":"172.21.15.165:6825","nonce":2138771601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":2138771601},{"type":"v1","addr":"172.21.15.165:6827","nonce":2138771601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":2138771601},{"type":"v1","addr":"172.21.15.165:6831","nonce":2138771601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":2138771601},{"type":"v1","addr":"172.21.15.165:6829","nonce":2138771601}]},"public_addr":"172.21.15.165:6825/2138771601","cluster_addr":"172.21.15.165:6827/2138771601","heartbeat_back_addr":"172.21.15.165:6831/2138771601","heartbeat_front_addr":"172.21.15.165:6829/2138771601","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:17:54.791531+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:18:19.466638+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:18:43.435536+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:19:15.702685+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:19:41.767370+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:20:06.793003+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:20:39.952744+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:21:05.727262+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:21:30.735829+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:22:04.072142+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:22:30.894047+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:22:56.637728+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.119:6801/2575246519":"2024-03-29T00:16:03.582545+0000","172.21.15.119:6800/2575246519":"2024-03-29T00:16:03.582545+0000","172.21.15.119:0/87565965":"2024-03-29T00:16:03.582545+0000","172.21.15.119:6801/2056426384":"2024-03-29T00:15:30.472283+0000","172.21.15.119:0/1639109126":"2024-03-29T00:15:30.472283+0000","172.21.15.119:0/468478300":"2024-03-29T00:15:30.472283+0000","172.21.15.119:0/2891240357":"2024-03-29T00:15:30.472283+0000","172.21.15.119:0/4201699272":"2024-03-29T00:16:03.582545+0000","172.21.15.119:6800/2056426384":"2024-03-29T00:15:30.472283+0000","172.21.15.119:0/2635982178":"2024-03-29T00:16:03.582545+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T00:23:02.865 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:02.860+0000 7f0ff9000700 1 -- 172.21.15.119:0/2563876857 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f0fd80a2890 msgr2=0x7f0fd80a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:02.866 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:02.860+0000 7f0ff9000700 1 --2- 172.21.15.119:0/2563876857 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f0fd80a2890 0x7f0fd80a4d50 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f0ff4101010 tx=0x7f0fdc009260 comp rx=0 tx=0).stop 2024-03-28T00:23:02.866 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:02.860+0000 7f0ff9000700 1 -- 172.21.15.119:0/2563876857 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ff4102440 msgr2=0x7f0ff41a0880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:02.866 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:02.860+0000 7f0ff9000700 1 --2- 172.21.15.119:0/2563876857 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ff4102440 0x7f0ff41a0880 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f0fe4000c00 tx=0x7f0fe4005590 comp rx=0 tx=0).stop 2024-03-28T00:23:02.866 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:02.860+0000 7f0ff9000700 1 -- 172.21.15.119:0/2563876857 shutdown_connections 2024-03-28T00:23:02.866 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:02.860+0000 7f0ff9000700 1 --2- 172.21.15.119:0/2563876857 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f0fd80a2890 0x7f0fd80a4d50 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:02.866 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:02.860+0000 7f0ff9000700 1 --2- 172.21.15.119:0/2563876857 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0ff4107510 0x7f0ff419aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:02.866 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:02.860+0000 7f0ff9000700 1 --2- 172.21.15.119:0/2563876857 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ff4102440 0x7f0ff41a0880 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:02.866 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:02.860+0000 7f0ff9000700 1 --2- 172.21.15.119:0/2563876857 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0ff4102df0 0x7f0ff41a0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:02.866 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:02.860+0000 7f0ff9000700 1 -- 172.21.15.119:0/2563876857 >> 172.21.15.119:0/2563876857 conn(0x7f0ff40792f0 msgr2=0x7f0ff41019a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:23:02.866 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:02.861+0000 7f0ff9000700 1 -- 172.21.15.119:0/2563876857 shutdown_connections 2024-03-28T00:23:02.866 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:02.861+0000 7f0ff9000700 1 -- 172.21.15.119:0/2563876857 wait complete. 2024-03-28T00:23:02.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:02 smithi165 ceph-mon[36302]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T00:23:03.604 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:03 smithi119 ceph-mon[31518]: from='client.? 172.21.15.119:0/2563876857' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-28T00:23:03.661 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-28T00:18:46.253414+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-28T00:23:03.662 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 10f73b92-ec98-11ee-95d0-87774f69a715 -- ceph osd pool get .mgr pg_num 2024-03-28T00:23:03.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:03 smithi165 ceph-mon[36302]: from='client.? 172.21.15.119:0/2563876857' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-28T00:23:03.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:03 smithi132 ceph-mon[35941]: from='client.? 172.21.15.119:0/2563876857' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-28T00:23:04.112 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/mon.a/config 2024-03-28T00:23:04.575 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:04 smithi119 ceph-mon[31518]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T00:23:04.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:04 smithi165 ceph-mon[36302]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T00:23:04.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:04 smithi132 ceph-mon[35941]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T00:23:06.253 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:06.247+0000 7f4464400700 1 -- 172.21.15.119:0/3149223868 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f445c104bc0 msgr2=0x7f445c105040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:06.253 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:06.247+0000 7f4464400700 1 --2- 172.21.15.119:0/3149223868 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f445c104bc0 0x7f445c105040 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f445400b0a0 tx=0x7f445400b3b0 comp rx=0 tx=0).stop 2024-03-28T00:23:06.254 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:06.248+0000 7f4464400700 1 -- 172.21.15.119:0/3149223868 shutdown_connections 2024-03-28T00:23:06.254 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:06.248+0000 7f4464400700 1 --2- 172.21.15.119:0/3149223868 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f445c104bc0 0x7f445c105040 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:06.254 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:06.248+0000 7f4464400700 1 --2- 172.21.15.119:0/3149223868 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f445c107150 0x7f445c06a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:06.254 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:06.248+0000 7f4464400700 1 --2- 172.21.15.119:0/3149223868 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f445c06ac10 0x7f445c06b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:06.254 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:06.248+0000 7f4464400700 1 -- 172.21.15.119:0/3149223868 >> 172.21.15.119:0/3149223868 conn(0x7f445c0770e0 msgr2=0x7f445c0774f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:23:06.254 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:06.248+0000 7f4464400700 1 -- 172.21.15.119:0/3149223868 shutdown_connections 2024-03-28T00:23:06.254 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:06.248+0000 7f4464400700 1 -- 172.21.15.119:0/3149223868 wait complete. 2024-03-28T00:23:06.254 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:06.249+0000 7f4464400700 1 Processor -- start 2024-03-28T00:23:06.256 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:06.249+0000 7f4464400700 1 -- start start 2024-03-28T00:23:06.256 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:06.249+0000 7f4464400700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f445c06ac10 0x7f445c19ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:23:06.256 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:06.249+0000 7f4464400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f445c104bc0 0x7f445c19f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:23:06.256 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:06.249+0000 7f4464400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f445c107150 0x7f445c1a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:23:06.256 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:06.249+0000 7f4464400700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f445c118aa0 con 0x7f445c104bc0 2024-03-28T00:23:06.256 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:06.249+0000 7f4464400700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f445c118920 con 0x7f445c107150 2024-03-28T00:23:06.256 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:06.249+0000 7f4464400700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f445c118c20 con 0x7f445c06ac10 2024-03-28T00:23:06.256 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:06.250+0000 7f4461000700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f445c104bc0 0x7f445c19f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:23:06.256 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:06.250+0000 7f4461a00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f445c06ac10 0x7f445c19ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:23:06.256 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:06.250+0000 7f4461000700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f445c104bc0 0x7f445c19f3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:38930/0 (socket says 172.21.15.119:38930) 2024-03-28T00:23:06.257 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:06.250+0000 7f4461a00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f445c06ac10 0x7f445c19ee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.119:36806/0 (socket says 172.21.15.119:36806) 2024-03-28T00:23:06.257 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:06.250+0000 7f4461000700 1 -- 172.21.15.119:0/333541376 learned_addr learned my addr 172.21.15.119:0/333541376 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:23:06.257 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:06.250+0000 7f4462400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f445c107150 0x7f445c1a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:23:06.257 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:06.250+0000 7f4461000700 1 -- 172.21.15.119:0/333541376 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f445c06ac10 msgr2=0x7f445c19ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:06.257 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:06.250+0000 7f4461000700 1 --2- 172.21.15.119:0/333541376 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f445c06ac10 0x7f445c19ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:06.257 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:06.250+0000 7f4461000700 1 -- 172.21.15.119:0/333541376 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f445c107150 msgr2=0x7f445c1a3740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:06.257 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:06.250+0000 7f4461000700 1 --2- 172.21.15.119:0/333541376 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f445c107150 0x7f445c1a3740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:06.257 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:06.250+0000 7f4461000700 1 -- 172.21.15.119:0/333541376 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f445c1a3e80 con 0x7f445c104bc0 2024-03-28T00:23:06.257 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:06.250+0000 7f4462400700 1 --2- 172.21.15.119:0/333541376 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f445c107150 0x7f445c1a3740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T00:23:06.257 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:06.250+0000 7f4461000700 1 --2- 172.21.15.119:0/333541376 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f445c104bc0 0x7f445c19f3b0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f444800e9c0 tx=0x7f444800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:23:06.257 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:06.251+0000 7f445a000700 1 -- 172.21.15.119:0/333541376 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f444800cbd0 con 0x7f445c104bc0 2024-03-28T00:23:06.257 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:06.251+0000 7f445a000700 1 -- 172.21.15.119:0/333541376 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f444800ee60 con 0x7f445c104bc0 2024-03-28T00:23:06.257 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:06.251+0000 7f4464400700 1 -- 172.21.15.119:0/333541376 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f445c1a4170 con 0x7f445c104bc0 2024-03-28T00:23:06.257 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:06.251+0000 7f445a000700 1 -- 172.21.15.119:0/333541376 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4448005630 con 0x7f445c104bc0 2024-03-28T00:23:06.258 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:06.251+0000 7f4464400700 1 -- 172.21.15.119:0/333541376 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f445c1ab9f0 con 0x7f445c104bc0 2024-03-28T00:23:06.258 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:06.252+0000 7f4464400700 1 -- 172.21.15.119:0/333541376 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f445c068e50 con 0x7f445c104bc0 2024-03-28T00:23:06.259 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:06.254+0000 7f445a000700 1 -- 172.21.15.119:0/333541376 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f4448004750 con 0x7f445c104bc0 2024-03-28T00:23:06.260 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:06.255+0000 7f445a000700 1 --2- 172.21.15.119:0/333541376 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f44440a2910 0x7f44440a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:23:06.260 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:06.256+0000 7f445a000700 1 -- 172.21.15.119:0/333541376 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f44480cea70 con 0x7f445c104bc0 2024-03-28T00:23:06.261 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:06.256+0000 7f4461a00700 1 --2- 172.21.15.119:0/333541376 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f44440a2910 0x7f44440a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:23:06.261 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:06.256+0000 7f4461a00700 1 --2- 172.21.15.119:0/333541376 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f44440a2910 0x7f44440a4dd0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f4450009680 tx=0x7f4450009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:23:06.263 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:06.258+0000 7f445a000700 1 -- 172.21.15.119:0/333541376 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f4448053740 con 0x7f445c104bc0 2024-03-28T00:23:06.485 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:06.480+0000 7f4464400700 1 -- 172.21.15.119:0/333541376 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f445c1abc50 con 0x7f445c104bc0 2024-03-28T00:23:06.485 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:06.480+0000 7f445a000700 1 -- 172.21.15.119:0/333541376 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) v1 ==== 93+0+10 (secure 0 0 0) 0x7f444809bf10 con 0x7f445c104bc0 2024-03-28T00:23:06.486 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 1 2024-03-28T00:23:06.489 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:06.484+0000 7f4464400700 1 -- 172.21.15.119:0/333541376 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f44440a2910 msgr2=0x7f44440a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:06.489 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:06.484+0000 7f4464400700 1 --2- 172.21.15.119:0/333541376 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f44440a2910 0x7f44440a4dd0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f4450009680 tx=0x7f4450009260 comp rx=0 tx=0).stop 2024-03-28T00:23:06.489 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:06.484+0000 7f4464400700 1 -- 172.21.15.119:0/333541376 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f445c104bc0 msgr2=0x7f445c19f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:06.489 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:06.484+0000 7f4464400700 1 --2- 172.21.15.119:0/333541376 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f445c104bc0 0x7f445c19f3b0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f444800e9c0 tx=0x7f444800ecd0 comp rx=0 tx=0).stop 2024-03-28T00:23:06.490 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:06.484+0000 7f4464400700 1 -- 172.21.15.119:0/333541376 shutdown_connections 2024-03-28T00:23:06.490 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:06.484+0000 7f4464400700 1 --2- 172.21.15.119:0/333541376 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f44440a2910 0x7f44440a4dd0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:06.490 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:06.484+0000 7f4464400700 1 --2- 172.21.15.119:0/333541376 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f445c107150 0x7f445c1a3740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:06.490 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:06.484+0000 7f4464400700 1 --2- 172.21.15.119:0/333541376 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f445c06ac10 0x7f445c19ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:06.490 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:06.484+0000 7f4464400700 1 --2- 172.21.15.119:0/333541376 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f445c104bc0 0x7f445c19f3b0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:06.490 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:06.484+0000 7f4464400700 1 -- 172.21.15.119:0/333541376 >> 172.21.15.119:0/333541376 conn(0x7f445c0770e0 msgr2=0x7f445c1000b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:23:06.490 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:06.484+0000 7f4464400700 1 -- 172.21.15.119:0/333541376 shutdown_connections 2024-03-28T00:23:06.490 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:06.484+0000 7f4464400700 1 -- 172.21.15.119:0/333541376 wait complete. 2024-03-28T00:23:07.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:06 smithi165 ceph-mon[36302]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T00:23:07.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:06 smithi165 ceph-mon[36302]: from='client.? 172.21.15.119:0/333541376' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-28T00:23:07.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:06 smithi119 ceph-mon[31518]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T00:23:07.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:06 smithi119 ceph-mon[31518]: from='client.? 172.21.15.119:0/333541376' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-28T00:23:07.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:06 smithi132 ceph-mon[35941]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T00:23:07.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:06 smithi132 ceph-mon[35941]: from='client.? 172.21.15.119:0/333541376' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-28T00:23:07.297 INFO:tasks.cephadm:Adding mds.a on smithi119 2024-03-28T00:23:07.297 INFO:tasks.cephadm:Adding mds.d on smithi119 2024-03-28T00:23:07.297 INFO:tasks.cephadm:Adding mds.g on smithi119 2024-03-28T00:23:07.297 INFO:tasks.cephadm:Adding mds.j on smithi119 2024-03-28T00:23:07.297 INFO:tasks.cephadm:Adding mds.b on smithi132 2024-03-28T00:23:07.297 INFO:tasks.cephadm:Adding mds.e on smithi132 2024-03-28T00:23:07.297 INFO:tasks.cephadm:Adding mds.h on smithi132 2024-03-28T00:23:07.297 INFO:tasks.cephadm:Adding mds.k on smithi132 2024-03-28T00:23:07.297 INFO:tasks.cephadm:Adding mds.c on smithi165 2024-03-28T00:23:07.297 INFO:tasks.cephadm:Adding mds.f on smithi165 2024-03-28T00:23:07.297 INFO:tasks.cephadm:Adding mds.i on smithi165 2024-03-28T00:23:07.298 INFO:tasks.cephadm:Adding mds.l on smithi165 2024-03-28T00:23:07.298 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 10f73b92-ec98-11ee-95d0-87774f69a715 -- ceph orch apply mds all '12;smithi119=a;smithi119=d;smithi119=g;smithi119=j;smithi132=b;smithi132=e;smithi132=h;smithi132=k;smithi165=c;smithi165=f;smithi165=i;smithi165=l' 2024-03-28T00:23:07.548 INFO:teuthology.orchestra.run.smithi165.stderr:Inferring config /var/lib/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/mon.c/config 2024-03-28T00:23:09.084 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:08 smithi165 ceph-mon[36302]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T00:23:09.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:08 smithi119 ceph-mon[31518]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T00:23:09.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:08 smithi132 ceph-mon[35941]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T00:23:09.322 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:23:09.314+0000 7f027ab90700 1 -- 172.21.15.165:0/1362347384 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f02741010c0 msgr2=0x7f02741014e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:09.322 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:23:09.314+0000 7f027ab90700 1 --2- 172.21.15.165:0/1362347384 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f02741010c0 0x7f02741014e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f0268009990 tx=0x7f0268009ca0 comp rx=0 tx=0).stop 2024-03-28T00:23:09.322 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:23:09.314+0000 7f027ab90700 1 -- 172.21.15.165:0/1362347384 shutdown_connections 2024-03-28T00:23:09.322 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:23:09.314+0000 7f027ab90700 1 --2- 172.21.15.165:0/1362347384 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0274102c60 0x7f0274109510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:09.322 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:23:09.314+0000 7f027ab90700 1 --2- 172.21.15.165:0/1362347384 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f02741010c0 0x7f02741014e0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:09.322 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:23:09.314+0000 7f027ab90700 1 --2- 172.21.15.165:0/1362347384 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f02741022c0 0x7f0274102720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:09.322 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:23:09.314+0000 7f027ab90700 1 -- 172.21.15.165:0/1362347384 >> 172.21.15.165:0/1362347384 conn(0x7f02740fc660 msgr2=0x7f02740feaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:23:09.323 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:23:09.314+0000 7f027ab90700 1 -- 172.21.15.165:0/1362347384 shutdown_connections 2024-03-28T00:23:09.323 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:23:09.314+0000 7f027ab90700 1 -- 172.21.15.165:0/1362347384 wait complete. 2024-03-28T00:23:09.323 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:23:09.315+0000 7f027ab90700 1 Processor -- start 2024-03-28T00:23:09.323 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:23:09.315+0000 7f027ab90700 1 -- start start 2024-03-28T00:23:09.323 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:23:09.315+0000 7f027ab90700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f02741010c0 0x7f027406a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:23:09.323 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:23:09.315+0000 7f027ab90700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f02741022c0 0x7f027419d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:23:09.323 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:23:09.315+0000 7f027ab90700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0274102c60 0x7f027419f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:23:09.323 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:23:09.315+0000 7f027ab90700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f027410cd10 con 0x7f0274102c60 2024-03-28T00:23:09.323 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:23:09.315+0000 7f027ab90700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f027410cb90 con 0x7f02741022c0 2024-03-28T00:23:09.323 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:23:09.315+0000 7f027ab90700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f027410ce90 con 0x7f02741010c0 2024-03-28T00:23:09.323 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:23:09.315+0000 7f027892c700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f02741010c0 0x7f027406a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:23:09.323 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:23:09.315+0000 7f027892c700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f02741010c0 0x7f027406a9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:51790/0 (socket says 172.21.15.165:51790) 2024-03-28T00:23:09.324 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:23:09.315+0000 7f027892c700 1 -- 172.21.15.165:0/967239835 learned_addr learned my addr 172.21.15.165:0/967239835 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-03-28T00:23:09.324 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:23:09.315+0000 7f0273fff700 1 --2- 172.21.15.165:0/967239835 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f02741022c0 0x7f027419d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:23:09.324 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:23:09.315+0000 7f027892c700 1 -- 172.21.15.165:0/967239835 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f02741022c0 msgr2=0x7f027419d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:09.324 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:23:09.315+0000 7f027892c700 1 --2- 172.21.15.165:0/967239835 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f02741022c0 0x7f027419d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:09.324 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:23:09.315+0000 7f027892c700 1 -- 172.21.15.165:0/967239835 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0274102c60 msgr2=0x7f027419f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:09.324 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:23:09.315+0000 7f027892c700 1 --2- 172.21.15.165:0/967239835 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0274102c60 0x7f027419f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:09.324 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:23:09.315+0000 7f027892c700 1 -- 172.21.15.165:0/967239835 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02741a0120 con 0x7f02741010c0 2024-03-28T00:23:09.324 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:23:09.316+0000 7f027892c700 1 --2- 172.21.15.165:0/967239835 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f02741010c0 0x7f027406a9a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f02680098e0 tx=0x7f0268009ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:23:09.324 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:23:09.316+0000 7f0271ffb700 1 -- 172.21.15.165:0/967239835 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0268027070 con 0x7f02741010c0 2024-03-28T00:23:09.324 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:23:09.316+0000 7f0271ffb700 1 -- 172.21.15.165:0/967239835 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0268004150 con 0x7f02741010c0 2024-03-28T00:23:09.324 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:23:09.316+0000 7f027ab90700 1 -- 172.21.15.165:0/967239835 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02741a03b0 con 0x7f02741010c0 2024-03-28T00:23:09.324 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:23:09.316+0000 7f0271ffb700 1 -- 172.21.15.165:0/967239835 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f026800f460 con 0x7f02741010c0 2024-03-28T00:23:09.324 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:23:09.316+0000 7f027ab90700 1 -- 172.21.15.165:0/967239835 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02741a08e0 con 0x7f02741010c0 2024-03-28T00:23:09.326 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:23:09.317+0000 7f027ab90700 1 -- 172.21.15.165:0/967239835 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0274068e50 con 0x7f02741010c0 2024-03-28T00:23:09.326 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:23:09.318+0000 7f0271ffb700 1 -- 172.21.15.165:0/967239835 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f026802a070 con 0x7f02741010c0 2024-03-28T00:23:09.326 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:23:09.318+0000 7f0271ffb700 1 --2- 172.21.15.165:0/967239835 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f02640a2910 0x7f02640a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:23:09.326 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:23:09.319+0000 7f0271ffb700 1 -- 172.21.15.165:0/967239835 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f02680cf2e0 con 0x7f02741010c0 2024-03-28T00:23:09.326 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:23:09.319+0000 7f0273fff700 1 --2- 172.21.15.165:0/967239835 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f02640a2910 0x7f02640a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:23:09.326 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:23:09.320+0000 7f0273fff700 1 --2- 172.21.15.165:0/967239835 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f02640a2910 0x7f02640a4dd0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f0260005950 tx=0x7f02600058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:23:09.328 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:23:09.323+0000 7f0271ffb700 1 -- 172.21.15.165:0/967239835 <== mon.1 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f0268013070 con 0x7f02741010c0 2024-03-28T00:23:09.539 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:23:09.533+0000 7f027ab90700 1 -- 172.21.15.165:0/967239835 --> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi119=a;smithi119=d;smithi119=g;smithi119=j;smithi132=b;smithi132=e;smithi132=h;smithi132=k;smithi165=c;smithi165=f;smithi165=i;smithi165=l", "target": ["mon-mgr", ""]}) v1 -- 0x7f02740627d0 con 0x7f02640a2910 2024-03-28T00:23:09.554 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:23:09.549+0000 7f0271ffb700 1 -- 172.21.15.165:0/967239835 <== mgr.14152 v2:172.21.15.119:6800/1401937096 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7f02740627d0 con 0x7f02640a2910 2024-03-28T00:23:09.554 INFO:teuthology.orchestra.run.smithi165.stdout:Scheduled mds.all update... 2024-03-28T00:23:09.557 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:23:09.551+0000 7f027ab90700 1 -- 172.21.15.165:0/967239835 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f02640a2910 msgr2=0x7f02640a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:09.557 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:23:09.551+0000 7f027ab90700 1 --2- 172.21.15.165:0/967239835 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f02640a2910 0x7f02640a4dd0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f0260005950 tx=0x7f02600058e0 comp rx=0 tx=0).stop 2024-03-28T00:23:09.557 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:23:09.551+0000 7f027ab90700 1 -- 172.21.15.165:0/967239835 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f02741010c0 msgr2=0x7f027406a9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:09.557 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:23:09.551+0000 7f027ab90700 1 --2- 172.21.15.165:0/967239835 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f02741010c0 0x7f027406a9a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f02680098e0 tx=0x7f0268009ee0 comp rx=0 tx=0).stop 2024-03-28T00:23:09.557 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:23:09.551+0000 7f027ab90700 1 -- 172.21.15.165:0/967239835 shutdown_connections 2024-03-28T00:23:09.557 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:23:09.551+0000 7f027ab90700 1 --2- 172.21.15.165:0/967239835 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f02640a2910 0x7f02640a4dd0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:09.557 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:23:09.551+0000 7f027ab90700 1 --2- 172.21.15.165:0/967239835 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0274102c60 0x7f027419f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:09.557 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:23:09.551+0000 7f027ab90700 1 --2- 172.21.15.165:0/967239835 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f02741010c0 0x7f027406a9a0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:09.557 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:23:09.551+0000 7f027ab90700 1 --2- 172.21.15.165:0/967239835 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f02741022c0 0x7f027419d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:09.557 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:23:09.551+0000 7f027ab90700 1 -- 172.21.15.165:0/967239835 >> 172.21.15.165:0/967239835 conn(0x7f02740fc660 msgr2=0x7f0274105690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:23:09.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:23:09.551+0000 7f027ab90700 1 -- 172.21.15.165:0/967239835 shutdown_connections 2024-03-28T00:23:09.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-03-28T00:23:09.551+0000 7f027ab90700 1 -- 172.21.15.165:0/967239835 wait complete. 2024-03-28T00:23:10.174 DEBUG:teuthology.orchestra.run.smithi119:mds.a> sudo journalctl -f -n 0 -u ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.a.service 2024-03-28T00:23:10.182 DEBUG:teuthology.orchestra.run.smithi119:mds.d> sudo journalctl -f -n 0 -u ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.d.service 2024-03-28T00:23:10.228 DEBUG:teuthology.orchestra.run.smithi119:mds.g> sudo journalctl -f -n 0 -u ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.g.service 2024-03-28T00:23:10.274 DEBUG:teuthology.orchestra.run.smithi119:mds.j> sudo journalctl -f -n 0 -u ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.j.service 2024-03-28T00:23:10.321 DEBUG:teuthology.orchestra.run.smithi132:mds.b> sudo journalctl -f -n 0 -u ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.b.service 2024-03-28T00:23:10.324 DEBUG:teuthology.orchestra.run.smithi132:mds.e> sudo journalctl -f -n 0 -u ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.e.service 2024-03-28T00:23:10.367 DEBUG:teuthology.orchestra.run.smithi132:mds.h> sudo journalctl -f -n 0 -u ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.h.service 2024-03-28T00:23:10.411 DEBUG:teuthology.orchestra.run.smithi132:mds.k> sudo journalctl -f -n 0 -u ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.k.service 2024-03-28T00:23:10.453 DEBUG:teuthology.orchestra.run.smithi165:mds.c> sudo journalctl -f -n 0 -u ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.c.service 2024-03-28T00:23:10.456 DEBUG:teuthology.orchestra.run.smithi165:mds.f> sudo journalctl -f -n 0 -u ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.f.service 2024-03-28T00:23:10.499 DEBUG:teuthology.orchestra.run.smithi165:mds.i> sudo journalctl -f -n 0 -u ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.i.service 2024-03-28T00:23:10.542 DEBUG:teuthology.orchestra.run.smithi165:mds.l> sudo journalctl -f -n 0 -u ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.l.service 2024-03-28T00:23:10.585 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-03-28T00:23:10.586 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T00:23:10.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:10 smithi132 ceph-mon[35941]: from='client.24421 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi119=a;smithi119=d;smithi119=g;smithi119=j;smithi132=b;smithi132=e;smithi132=h;smithi132=k;smithi165=c;smithi165=f;smithi165=i;smithi165=l", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T00:23:10.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:10 smithi132 ceph-mon[35941]: Saving service mds.all spec with placement smithi119=a;smithi119=d;smithi119=g;smithi119=j;smithi132=b;smithi132=e;smithi132=h;smithi132=k;smithi165=c;smithi165=f;smithi165=i;smithi165=l;count:12 2024-03-28T00:23:10.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:10 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:10.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:10 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:23:10.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:10 smithi132 ceph-mon[35941]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T00:23:10.810 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:10.803+0000 7fc907c00640 1 -- 172.21.15.119:0/3225635031 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc900104ad0 msgr2=0x7fc900104f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:10.810 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:10.803+0000 7fc907c00640 1 --2- 172.21.15.119:0/3225635031 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc900104ad0 0x7fc900104f50 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fc8e8009900 tx=0x7fc8e802bc50 comp rx=0 tx=0).stop 2024-03-28T00:23:10.811 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:10.803+0000 7fc907c00640 1 -- 172.21.15.119:0/3225635031 shutdown_connections 2024-03-28T00:23:10.811 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:10.803+0000 7fc907c00640 1 --2- 172.21.15.119:0/3225635031 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc900105490 0x7fc90010bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:10.811 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:10.803+0000 7fc907c00640 1 --2- 172.21.15.119:0/3225635031 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc900104ad0 0x7fc900104f50 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:10.811 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:10.803+0000 7fc907c00640 1 --2- 172.21.15.119:0/3225635031 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc9001038e0 0x7fc900103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:10.811 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:10.803+0000 7fc907c00640 1 -- 172.21.15.119:0/3225635031 >> 172.21.15.119:0/3225635031 conn(0x7fc9000fee50 msgr2=0x7fc900101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:23:10.811 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:10.803+0000 7fc907c00640 1 -- 172.21.15.119:0/3225635031 shutdown_connections 2024-03-28T00:23:10.811 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:10.803+0000 7fc907c00640 1 -- 172.21.15.119:0/3225635031 wait complete. 2024-03-28T00:23:10.811 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:10.804+0000 7fc907c00640 1 Processor -- start 2024-03-28T00:23:10.811 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:10.805+0000 7fc907c00640 1 -- start start 2024-03-28T00:23:10.813 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:10.805+0000 7fc907c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc9001038e0 0x7fc900199bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:23:10.813 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:10.805+0000 7fc907c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc900104ad0 0x7fc90019a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:23:10.813 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:10.805+0000 7fc907c00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc900105490 0x7fc9001a1170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:23:10.813 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:10.805+0000 7fc907c00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc90010f5b0 con 0x7fc9001038e0 2024-03-28T00:23:10.813 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:10.805+0000 7fc907c00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc90010f430 con 0x7fc900105490 2024-03-28T00:23:10.813 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:10.805+0000 7fc907c00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc90010f730 con 0x7fc900104ad0 2024-03-28T00:23:10.813 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:10.805+0000 7fc8f7e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc900104ad0 0x7fc90019a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:23:10.814 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:10.805+0000 7fc8f7e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc900104ad0 0x7fc90019a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.119:36828/0 (socket says 172.21.15.119:36828) 2024-03-28T00:23:10.814 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:10.805+0000 7fc8f7e00640 1 -- 172.21.15.119:0/2088599605 learned_addr learned my addr 172.21.15.119:0/2088599605 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:23:10.814 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:10.805+0000 7fc905200640 1 --2- 172.21.15.119:0/2088599605 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc9001038e0 0x7fc900199bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:23:10.814 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:10.805+0000 7fc905c00640 1 --2- 172.21.15.119:0/2088599605 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc900105490 0x7fc9001a1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:23:10.814 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:10.805+0000 7fc8f7e00640 1 -- 172.21.15.119:0/2088599605 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc900105490 msgr2=0x7fc9001a1170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:10.814 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:10.805+0000 7fc8f7e00640 1 --2- 172.21.15.119:0/2088599605 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc900105490 0x7fc9001a1170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:10.814 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:10.805+0000 7fc8f7e00640 1 -- 172.21.15.119:0/2088599605 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc9001038e0 msgr2=0x7fc900199bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:10.814 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:10.805+0000 7fc8f7e00640 1 --2- 172.21.15.119:0/2088599605 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc9001038e0 0x7fc900199bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:10.814 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:10.805+0000 7fc8f7e00640 1 -- 172.21.15.119:0/2088599605 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9001a17e0 con 0x7fc900104ad0 2024-03-28T00:23:10.814 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:10.806+0000 7fc8f7e00640 1 --2- 172.21.15.119:0/2088599605 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc900104ad0 0x7fc90019a100 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc8e802c260 tx=0x7fc8e802c670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:23:10.814 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:10.806+0000 7fc8fe000640 1 -- 172.21.15.119:0/2088599605 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8e8041070 con 0x7fc900104ad0 2024-03-28T00:23:10.814 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:10.806+0000 7fc8fe000640 1 -- 172.21.15.119:0/2088599605 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc8e8004450 con 0x7fc900104ad0 2024-03-28T00:23:10.814 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:10.806+0000 7fc8fe000640 1 -- 172.21.15.119:0/2088599605 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8e8034440 con 0x7fc900104ad0 2024-03-28T00:23:10.814 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:10.806+0000 7fc907c00640 1 -- 172.21.15.119:0/2088599605 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9001a1a70 con 0x7fc900104ad0 2024-03-28T00:23:10.814 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:10.806+0000 7fc907c00640 1 -- 172.21.15.119:0/2088599605 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9001a2010 con 0x7fc900104ad0 2024-03-28T00:23:10.815 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:10.807+0000 7fc907c00640 1 -- 172.21.15.119:0/2088599605 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc90010ac00 con 0x7fc900104ad0 2024-03-28T00:23:10.815 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:10.809+0000 7fc8fe000640 1 -- 172.21.15.119:0/2088599605 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fc8e8044070 con 0x7fc900104ad0 2024-03-28T00:23:10.816 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:10.810+0000 7fc8fe000640 1 --2- 172.21.15.119:0/2088599605 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fc8e40a2880 0x7fc8e40a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:23:10.816 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:10.810+0000 7fc8fe000640 1 -- 172.21.15.119:0/2088599605 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fc8e80f0150 con 0x7fc900104ad0 2024-03-28T00:23:10.816 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:10.811+0000 7fc905200640 1 --2- 172.21.15.119:0/2088599605 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fc8e40a2880 0x7fc8e40a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:23:10.817 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:10.811+0000 7fc905200640 1 --2- 172.21.15.119:0/2088599605 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fc8e40a2880 0x7fc8e40a4d40 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fc900104930 tx=0x7fc8f0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:23:10.820 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:10.814+0000 7fc8fe000640 1 -- 172.21.15.119:0/2088599605 <== mon.1 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fc8e80bd5f0 con 0x7fc900104ad0 2024-03-28T00:23:10.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:10 smithi165 ceph-mon[36302]: from='client.24421 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi119=a;smithi119=d;smithi119=g;smithi119=j;smithi132=b;smithi132=e;smithi132=h;smithi132=k;smithi165=c;smithi165=f;smithi165=i;smithi165=l", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T00:23:10.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:10 smithi165 ceph-mon[36302]: Saving service mds.all spec with placement smithi119=a;smithi119=d;smithi119=g;smithi119=j;smithi132=b;smithi132=e;smithi132=h;smithi132=k;smithi165=c;smithi165=f;smithi165=i;smithi165=l;count:12 2024-03-28T00:23:10.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:10 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:10.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:10 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:23:10.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:10 smithi165 ceph-mon[36302]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T00:23:10.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:10 smithi119 ceph-mon[31518]: from='client.24421 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi119=a;smithi119=d;smithi119=g;smithi119=j;smithi132=b;smithi132=e;smithi132=h;smithi132=k;smithi165=c;smithi165=f;smithi165=i;smithi165=l", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T00:23:10.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:10 smithi119 ceph-mon[31518]: Saving service mds.all spec with placement smithi119=a;smithi119=d;smithi119=g;smithi119=j;smithi132=b;smithi132=e;smithi132=h;smithi132=k;smithi165=c;smithi165=f;smithi165=i;smithi165=l;count:12 2024-03-28T00:23:10.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:10 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:10.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:10 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:23:10.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:10 smithi119 ceph-mon[31518]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T00:23:11.023 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:11.017+0000 7fc907c00640 1 -- 172.21.15.119:0/2088599605 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc90002d070 con 0x7fc900104ad0 2024-03-28T00:23:11.025 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:11.019+0000 7fc8fe000640 1 -- 172.21.15.119:0/2088599605 <== mon.1 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+18537 (secure 0 0 0) 0x7fc8e80bcef0 con 0x7fc900104ad0 2024-03-28T00:23:11.026 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-28T00:23:11.026 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":81,"fsid":"10f73b92-ec98-11ee-95d0-87774f69a715","created":"2024-03-28T00:14:57.563604+0000","modified":"2024-03-28T00:23:00.426768+0000","last_up_change":"2024-03-28T00:22:58.821662+0000","last_in_change":"2024-03-28T00:22:38.467270+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T00:18:46.253414+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c0b875d-a01e-4294-9c9c-5d0b048b8912","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":580396252},{"type":"v1","addr":"172.21.15.119:6803","nonce":580396252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":580396252},{"type":"v1","addr":"172.21.15.119:6805","nonce":580396252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":580396252},{"type":"v1","addr":"172.21.15.119:6809","nonce":580396252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":580396252},{"type":"v1","addr":"172.21.15.119:6807","nonce":580396252}]},"public_addr":"172.21.15.119:6803/580396252","cluster_addr":"172.21.15.119:6805/580396252","heartbeat_back_addr":"172.21.15.119:6809/580396252","heartbeat_front_addr":"172.21.15.119:6807/580396252","state":["exists","up"]},{"osd":1,"uuid":"5567f7c2-8ea4-4789-9ea1-467793324fb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":2500318888},{"type":"v1","addr":"172.21.15.132:6801","nonce":2500318888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":2500318888},{"type":"v1","addr":"172.21.15.132:6803","nonce":2500318888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":2500318888},{"type":"v1","addr":"172.21.15.132:6807","nonce":2500318888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":2500318888},{"type":"v1","addr":"172.21.15.132:6805","nonce":2500318888}]},"public_addr":"172.21.15.132:6801/2500318888","cluster_addr":"172.21.15.132:6803/2500318888","heartbeat_back_addr":"172.21.15.132:6807/2500318888","heartbeat_front_addr":"172.21.15.132:6805/2500318888","state":["exists","up"]},{"osd":2,"uuid":"ee54dbd0-24b4-4ee6-988c-ce71491199d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":2361434832},{"type":"v1","addr":"172.21.15.165:6801","nonce":2361434832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":2361434832},{"type":"v1","addr":"172.21.15.165:6803","nonce":2361434832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":2361434832},{"type":"v1","addr":"172.21.15.165:6807","nonce":2361434832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":2361434832},{"type":"v1","addr":"172.21.15.165:6805","nonce":2361434832}]},"public_addr":"172.21.15.165:6801/2361434832","cluster_addr":"172.21.15.165:6803/2361434832","heartbeat_back_addr":"172.21.15.165:6807/2361434832","heartbeat_front_addr":"172.21.15.165:6805/2361434832","state":["exists","up"]},{"osd":3,"uuid":"0464c40b-828f-4db4-96d9-3f7a6325e6da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":3732272210},{"type":"v1","addr":"172.21.15.119:6811","nonce":3732272210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":3732272210},{"type":"v1","addr":"172.21.15.119:6813","nonce":3732272210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":3732272210},{"type":"v1","addr":"172.21.15.119:6817","nonce":3732272210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":3732272210},{"type":"v1","addr":"172.21.15.119:6815","nonce":3732272210}]},"public_addr":"172.21.15.119:6811/3732272210","cluster_addr":"172.21.15.119:6813/3732272210","heartbeat_back_addr":"172.21.15.119:6817/3732272210","heartbeat_front_addr":"172.21.15.119:6815/3732272210","state":["exists","up"]},{"osd":4,"uuid":"8fd70b8e-5128-4173-aa0e-e735dfeb995d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":2803039505},{"type":"v1","addr":"172.21.15.132:6809","nonce":2803039505}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":2803039505},{"type":"v1","addr":"172.21.15.132:6811","nonce":2803039505}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":2803039505},{"type":"v1","addr":"172.21.15.132:6815","nonce":2803039505}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":2803039505},{"type":"v1","addr":"172.21.15.132:6813","nonce":2803039505}]},"public_addr":"172.21.15.132:6809/2803039505","cluster_addr":"172.21.15.132:6811/2803039505","heartbeat_back_addr":"172.21.15.132:6815/2803039505","heartbeat_front_addr":"172.21.15.132:6813/2803039505","state":["exists","up"]},{"osd":5,"uuid":"3224b4c1-733e-4424-a6da-870bbb5f94e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":2561252206},{"type":"v1","addr":"172.21.15.165:6809","nonce":2561252206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":2561252206},{"type":"v1","addr":"172.21.15.165:6811","nonce":2561252206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":2561252206},{"type":"v1","addr":"172.21.15.165:6815","nonce":2561252206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":2561252206},{"type":"v1","addr":"172.21.15.165:6813","nonce":2561252206}]},"public_addr":"172.21.15.165:6809/2561252206","cluster_addr":"172.21.15.165:6811/2561252206","heartbeat_back_addr":"172.21.15.165:6815/2561252206","heartbeat_front_addr":"172.21.15.165:6813/2561252206","state":["exists","up"]},{"osd":6,"uuid":"e435745d-59b5-439a-9091-0946ca0e9332","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":2545410230},{"type":"v1","addr":"172.21.15.119:6819","nonce":2545410230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":2545410230},{"type":"v1","addr":"172.21.15.119:6821","nonce":2545410230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":2545410230},{"type":"v1","addr":"172.21.15.119:6825","nonce":2545410230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":2545410230},{"type":"v1","addr":"172.21.15.119:6823","nonce":2545410230}]},"public_addr":"172.21.15.119:6819/2545410230","cluster_addr":"172.21.15.119:6821/2545410230","heartbeat_back_addr":"172.21.15.119:6825/2545410230","heartbeat_front_addr":"172.21.15.119:6823/2545410230","state":["exists","up"]},{"osd":7,"uuid":"79b97a12-f6b4-48c3-b517-2dd6b6e7c85f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":2877892376},{"type":"v1","addr":"172.21.15.132:6817","nonce":2877892376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":2877892376},{"type":"v1","addr":"172.21.15.132:6819","nonce":2877892376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":2877892376},{"type":"v1","addr":"172.21.15.132:6823","nonce":2877892376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":2877892376},{"type":"v1","addr":"172.21.15.132:6821","nonce":2877892376}]},"public_addr":"172.21.15.132:6817/2877892376","cluster_addr":"172.21.15.132:6819/2877892376","heartbeat_back_addr":"172.21.15.132:6823/2877892376","heartbeat_front_addr":"172.21.15.132:6821/2877892376","state":["exists","up"]},{"osd":8,"uuid":"2970524c-b99a-412d-85c2-6c29b54035e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":4115226150},{"type":"v1","addr":"172.21.15.165:6817","nonce":4115226150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":4115226150},{"type":"v1","addr":"172.21.15.165:6819","nonce":4115226150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":4115226150},{"type":"v1","addr":"172.21.15.165:6823","nonce":4115226150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":4115226150},{"type":"v1","addr":"172.21.15.165:6821","nonce":4115226150}]},"public_addr":"172.21.15.165:6817/4115226150","cluster_addr":"172.21.15.165:6819/4115226150","heartbeat_back_addr":"172.21.15.165:6823/4115226150","heartbeat_front_addr":"172.21.15.165:6821/4115226150","state":["exists","up"]},{"osd":9,"uuid":"2b1a08ed-cde9-4def-b3e8-838fff22709c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":2532245222},{"type":"v1","addr":"172.21.15.119:6827","nonce":2532245222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":2532245222},{"type":"v1","addr":"172.21.15.119:6829","nonce":2532245222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":2532245222},{"type":"v1","addr":"172.21.15.119:6833","nonce":2532245222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":2532245222},{"type":"v1","addr":"172.21.15.119:6831","nonce":2532245222}]},"public_addr":"172.21.15.119:6827/2532245222","cluster_addr":"172.21.15.119:6829/2532245222","heartbeat_back_addr":"172.21.15.119:6833/2532245222","heartbeat_front_addr":"172.21.15.119:6831/2532245222","state":["exists","up"]},{"osd":10,"uuid":"f019acd0-915a-4094-b12c-b53672740a5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":3454298492},{"type":"v1","addr":"172.21.15.132:6825","nonce":3454298492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":3454298492},{"type":"v1","addr":"172.21.15.132:6827","nonce":3454298492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":3454298492},{"type":"v1","addr":"172.21.15.132:6831","nonce":3454298492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":3454298492},{"type":"v1","addr":"172.21.15.132:6829","nonce":3454298492}]},"public_addr":"172.21.15.132:6825/3454298492","cluster_addr":"172.21.15.132:6827/3454298492","heartbeat_back_addr":"172.21.15.132:6831/3454298492","heartbeat_front_addr":"172.21.15.132:6829/3454298492","state":["exists","up"]},{"osd":11,"uuid":"2c1e59a8-8f21-45c0-8568-4b632ce020f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":2138771601},{"type":"v1","addr":"172.21.15.165:6825","nonce":2138771601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":2138771601},{"type":"v1","addr":"172.21.15.165:6827","nonce":2138771601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":2138771601},{"type":"v1","addr":"172.21.15.165:6831","nonce":2138771601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":2138771601},{"type":"v1","addr":"172.21.15.165:6829","nonce":2138771601}]},"public_addr":"172.21.15.165:6825/2138771601","cluster_addr":"172.21.15.165:6827/2138771601","heartbeat_back_addr":"172.21.15.165:6831/2138771601","heartbeat_front_addr":"172.21.15.165:6829/2138771601","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:17:54.791531+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:18:19.466638+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:18:43.435536+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:19:15.702685+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:19:41.767370+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:20:06.793003+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:20:39.952744+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:21:05.727262+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:21:30.735829+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:22:04.072142+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:22:30.894047+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:22:56.637728+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.119:6801/2575246519":"2024-03-29T00:16:03.582545+0000","172.21.15.119:6800/2575246519":"2024-03-29T00:16:03.582545+0000","172.21.15.119:0/87565965":"2024-03-29T00:16:03.582545+0000","172.21.15.119:6801/2056426384":"2024-03-29T00:15:30.472283+0000","172.21.15.119:0/1639109126":"2024-03-29T00:15:30.472283+0000","172.21.15.119:0/468478300":"2024-03-29T00:15:30.472283+0000","172.21.15.119:0/2891240357":"2024-03-29T00:15:30.472283+0000","172.21.15.119:0/4201699272":"2024-03-29T00:16:03.582545+0000","172.21.15.119:6800/2056426384":"2024-03-29T00:15:30.472283+0000","172.21.15.119:0/2635982178":"2024-03-29T00:16:03.582545+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T00:23:11.028 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:11.022+0000 7fc907c00640 1 -- 172.21.15.119:0/2088599605 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fc8e40a2880 msgr2=0x7fc8e40a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:11.028 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:11.022+0000 7fc907c00640 1 --2- 172.21.15.119:0/2088599605 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fc8e40a2880 0x7fc8e40a4d40 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fc900104930 tx=0x7fc8f0005f90 comp rx=0 tx=0).stop 2024-03-28T00:23:11.028 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:11.022+0000 7fc907c00640 1 -- 172.21.15.119:0/2088599605 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc900104ad0 msgr2=0x7fc90019a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:11.028 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:11.022+0000 7fc907c00640 1 --2- 172.21.15.119:0/2088599605 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc900104ad0 0x7fc90019a100 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc8e802c260 tx=0x7fc8e802c670 comp rx=0 tx=0).stop 2024-03-28T00:23:11.028 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:11.022+0000 7fc907c00640 1 -- 172.21.15.119:0/2088599605 shutdown_connections 2024-03-28T00:23:11.029 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:11.022+0000 7fc907c00640 1 --2- 172.21.15.119:0/2088599605 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fc8e40a2880 0x7fc8e40a4d40 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:11.029 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:11.022+0000 7fc907c00640 1 --2- 172.21.15.119:0/2088599605 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc900105490 0x7fc9001a1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:11.029 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:11.022+0000 7fc907c00640 1 --2- 172.21.15.119:0/2088599605 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc900104ad0 0x7fc90019a100 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:11.029 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:11.022+0000 7fc907c00640 1 --2- 172.21.15.119:0/2088599605 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc9001038e0 0x7fc900199bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:11.029 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:11.022+0000 7fc907c00640 1 -- 172.21.15.119:0/2088599605 >> 172.21.15.119:0/2088599605 conn(0x7fc9000fee50 msgr2=0x7fc900107dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:23:11.029 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:11.022+0000 7fc907c00640 1 -- 172.21.15.119:0/2088599605 shutdown_connections 2024-03-28T00:23:11.029 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:11.022+0000 7fc907c00640 1 -- 172.21.15.119:0/2088599605 wait complete. 2024-03-28T00:23:11.049 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-28T00:18:46.253414+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-28T00:23:11.050 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-28T00:23:11.265 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:11.258+0000 7fb6c1400640 1 -- 172.21.15.119:0/3610504078 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb6bc104990 msgr2=0x7fb6bc104e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:11.265 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:11.258+0000 7fb6c1400640 1 --2- 172.21.15.119:0/3610504078 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb6bc104990 0x7fb6bc104e10 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb6a8009900 tx=0x7fb6a802bc30 comp rx=0 tx=0).stop 2024-03-28T00:23:11.265 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:11.259+0000 7fb6c1400640 1 -- 172.21.15.119:0/3610504078 shutdown_connections 2024-03-28T00:23:11.265 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:11.259+0000 7fb6c1400640 1 --2- 172.21.15.119:0/3610504078 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb6bc105350 0x7fb6bc10bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:11.265 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:11.259+0000 7fb6c1400640 1 --2- 172.21.15.119:0/3610504078 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb6bc104990 0x7fb6bc104e10 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:11.265 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:11.259+0000 7fb6c1400640 1 --2- 172.21.15.119:0/3610504078 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb6bc1037a0 0x7fb6bc103bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:11.265 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:11.259+0000 7fb6c1400640 1 -- 172.21.15.119:0/3610504078 >> 172.21.15.119:0/3610504078 conn(0x7fb6bc0fed10 msgr2=0x7fb6bc101150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:23:11.265 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:11.259+0000 7fb6c1400640 1 -- 172.21.15.119:0/3610504078 shutdown_connections 2024-03-28T00:23:11.266 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:11.259+0000 7fb6c1400640 1 -- 172.21.15.119:0/3610504078 wait complete. 2024-03-28T00:23:11.268 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:11.260+0000 7fb6c1400640 1 Processor -- start 2024-03-28T00:23:11.268 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:11.260+0000 7fb6c1400640 1 -- start start 2024-03-28T00:23:11.268 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:11.260+0000 7fb6c1400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb6bc1037a0 0x7fb6bc199a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:23:11.268 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:11.260+0000 7fb6c1400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb6bc104990 0x7fb6bc199fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:23:11.268 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:11.260+0000 7fb6c1400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb6bc105350 0x7fb6bc1a1020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:23:11.268 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:11.260+0000 7fb6c1400640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6bc10f400 con 0x7fb6bc104990 2024-03-28T00:23:11.268 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:11.260+0000 7fb6c1400640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6bc10f280 con 0x7fb6bc1037a0 2024-03-28T00:23:11.268 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:11.260+0000 7fb6c1400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6bc10f580 con 0x7fb6bc105350 2024-03-28T00:23:11.268 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:11.260+0000 7fb6bb400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb6bc104990 0x7fb6bc199fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:23:11.268 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:11.260+0000 7fb6bbe00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb6bc1037a0 0x7fb6bc199a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:23:11.268 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:11.260+0000 7fb6bbe00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb6bc1037a0 0x7fb6bc199a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.119:47640/0 (socket says 172.21.15.119:47640) 2024-03-28T00:23:11.269 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:11.260+0000 7fb6bbe00640 1 -- 172.21.15.119:0/2231865240 learned_addr learned my addr 172.21.15.119:0/2231865240 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:23:11.269 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:11.260+0000 7fb6bb400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb6bc104990 0x7fb6bc199fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:38952/0 (socket says 172.21.15.119:38952) 2024-03-28T00:23:11.269 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:11.260+0000 7fb6c0a00640 1 --2- 172.21.15.119:0/2231865240 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb6bc105350 0x7fb6bc1a1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:23:11.269 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:11.261+0000 7fb6c0a00640 1 -- 172.21.15.119:0/2231865240 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb6bc1037a0 msgr2=0x7fb6bc199a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:11.269 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:11.261+0000 7fb6c0a00640 1 --2- 172.21.15.119:0/2231865240 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb6bc1037a0 0x7fb6bc199a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:11.269 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:11.261+0000 7fb6c0a00640 1 -- 172.21.15.119:0/2231865240 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb6bc104990 msgr2=0x7fb6bc199fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:11.269 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:11.261+0000 7fb6c0a00640 1 --2- 172.21.15.119:0/2231865240 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb6bc104990 0x7fb6bc199fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:11.269 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:11.261+0000 7fb6c0a00640 1 -- 172.21.15.119:0/2231865240 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6bc1a1690 con 0x7fb6bc105350 2024-03-28T00:23:11.269 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:11.261+0000 7fb6bbe00640 1 --2- 172.21.15.119:0/2231865240 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb6bc1037a0 0x7fb6bc199a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T00:23:11.269 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:11.261+0000 7fb6bb400640 1 --2- 172.21.15.119:0/2231865240 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb6bc104990 0x7fb6bc199fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T00:23:11.269 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:11.261+0000 7fb6c0a00640 1 --2- 172.21.15.119:0/2231865240 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb6bc105350 0x7fb6bc1a1020 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb6b4009fd0 tx=0x7fb6b400ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:23:11.269 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:11.261+0000 7fb6b8c00640 1 -- 172.21.15.119:0/2231865240 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6b400ce20 con 0x7fb6bc105350 2024-03-28T00:23:11.269 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:11.261+0000 7fb6b8c00640 1 -- 172.21.15.119:0/2231865240 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb6b4005020 con 0x7fb6bc105350 2024-03-28T00:23:11.270 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:11.261+0000 7fb6b8c00640 1 -- 172.21.15.119:0/2231865240 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6b4010720 con 0x7fb6bc105350 2024-03-28T00:23:11.270 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:11.261+0000 7fb6c1400640 1 -- 172.21.15.119:0/2231865240 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6bc1a1980 con 0x7fb6bc105350 2024-03-28T00:23:11.270 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:11.262+0000 7fb6c1400640 1 -- 172.21.15.119:0/2231865240 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6bc1a1e50 con 0x7fb6bc105350 2024-03-28T00:23:11.270 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:11.263+0000 7fb6c1400640 1 -- 172.21.15.119:0/2231865240 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6bc068800 con 0x7fb6bc105350 2024-03-28T00:23:11.270 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:11.265+0000 7fb6b8c00640 1 -- 172.21.15.119:0/2231865240 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fb6b4005540 con 0x7fb6bc105350 2024-03-28T00:23:11.272 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:11.266+0000 7fb6b8c00640 1 --2- 172.21.15.119:0/2231865240 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fb6a00a2880 0x7fb6a00a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:23:11.272 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:11.266+0000 7fb6b8c00640 1 -- 172.21.15.119:0/2231865240 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fb6b40cf000 con 0x7fb6bc105350 2024-03-28T00:23:11.272 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:11.266+0000 7fb6bbe00640 1 --2- 172.21.15.119:0/2231865240 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fb6a00a2880 0x7fb6a00a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:23:11.273 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:11.267+0000 7fb6bbe00640 1 --2- 172.21.15.119:0/2231865240 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fb6a00a2880 0x7fb6a00a4d40 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fb6ac0058c0 tx=0x7fb6ac005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:23:11.274 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:11.268+0000 7fb6b8c00640 1 -- 172.21.15.119:0/2231865240 <== mon.1 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fb6b409c4a0 con 0x7fb6bc105350 2024-03-28T00:23:11.472 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:11.466+0000 7fb6c1400640 1 -- 172.21.15.119:0/2231865240 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb6bc02d070 con 0x7fb6bc105350 2024-03-28T00:23:11.472 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:11.466+0000 7fb6b8c00640 1 -- 172.21.15.119:0/2231865240 <== mon.1 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb6b409bda0 con 0x7fb6bc105350 2024-03-28T00:23:11.473 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 1 2024-03-28T00:23:11.476 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:11.469+0000 7fb6c1400640 1 -- 172.21.15.119:0/2231865240 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fb6a00a2880 msgr2=0x7fb6a00a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:11.476 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:11.469+0000 7fb6c1400640 1 --2- 172.21.15.119:0/2231865240 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fb6a00a2880 0x7fb6a00a4d40 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fb6ac0058c0 tx=0x7fb6ac005f90 comp rx=0 tx=0).stop 2024-03-28T00:23:11.477 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:11.469+0000 7fb6c1400640 1 -- 172.21.15.119:0/2231865240 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb6bc105350 msgr2=0x7fb6bc1a1020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:11.477 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:11.469+0000 7fb6c1400640 1 --2- 172.21.15.119:0/2231865240 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb6bc105350 0x7fb6bc1a1020 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb6b4009fd0 tx=0x7fb6b400ef10 comp rx=0 tx=0).stop 2024-03-28T00:23:11.477 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:11.470+0000 7fb6c1400640 1 -- 172.21.15.119:0/2231865240 shutdown_connections 2024-03-28T00:23:11.477 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:11.470+0000 7fb6c1400640 1 --2- 172.21.15.119:0/2231865240 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fb6a00a2880 0x7fb6a00a4d40 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:11.477 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:11.470+0000 7fb6c1400640 1 --2- 172.21.15.119:0/2231865240 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb6bc105350 0x7fb6bc1a1020 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:11.477 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:11.470+0000 7fb6c1400640 1 --2- 172.21.15.119:0/2231865240 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb6bc104990 0x7fb6bc199fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:11.477 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:11.470+0000 7fb6c1400640 1 --2- 172.21.15.119:0/2231865240 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb6bc1037a0 0x7fb6bc199a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:11.477 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:11.470+0000 7fb6c1400640 1 -- 172.21.15.119:0/2231865240 >> 172.21.15.119:0/2231865240 conn(0x7fb6bc0fed10 msgr2=0x7fb6bc107c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:23:11.477 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:11.470+0000 7fb6c1400640 1 -- 172.21.15.119:0/2231865240 shutdown_connections 2024-03-28T00:23:11.477 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:11.470+0000 7fb6c1400640 1 -- 172.21.15.119:0/2231865240 wait complete. 2024-03-28T00:23:11.497 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T00:23:11.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:11 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:23:11.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:11 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:23:11.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:11 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:11.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:11 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-28T00:23:11.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:11 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-28T00:23:11.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:11 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:23:11.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:11 smithi165 ceph-mon[36302]: Deploying daemon mds.l on smithi165 2024-03-28T00:23:11.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:11 smithi165 ceph-mon[36302]: from='client.? 172.21.15.119:0/2088599605' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-28T00:23:11.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:11 smithi165 ceph-mon[36302]: from='client.? 172.21.15.119:0/2231865240' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-28T00:23:11.709 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:11.703+0000 7f27bbc00640 1 -- 172.21.15.119:0/3591374698 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f27b41028b0 msgr2=0x7f27b4102d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:11.709 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:11.703+0000 7f27bbc00640 1 --2- 172.21.15.119:0/3591374698 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f27b41028b0 0x7f27b4102d30 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f27a0009900 tx=0x7f27a002bc30 comp rx=0 tx=0).stop 2024-03-28T00:23:11.709 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:11.703+0000 7f27bbc00640 1 -- 172.21.15.119:0/3591374698 shutdown_connections 2024-03-28T00:23:11.710 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:11.703+0000 7f27bbc00640 1 --2- 172.21.15.119:0/3591374698 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f27b4103270 0x7f27b4109b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:11.710 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:11.703+0000 7f27bbc00640 1 --2- 172.21.15.119:0/3591374698 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f27b41028b0 0x7f27b4102d30 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:11.710 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:11.703+0000 7f27bbc00640 1 --2- 172.21.15.119:0/3591374698 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f27b41016c0 0x7f27b4101ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:11.710 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:11.703+0000 7f27bbc00640 1 -- 172.21.15.119:0/3591374698 >> 172.21.15.119:0/3591374698 conn(0x7f27b40fcc50 msgr2=0x7f27b40ff070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:23:11.710 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:11.703+0000 7f27bbc00640 1 -- 172.21.15.119:0/3591374698 shutdown_connections 2024-03-28T00:23:11.710 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:11.703+0000 7f27bbc00640 1 -- 172.21.15.119:0/3591374698 wait complete. 2024-03-28T00:23:11.712 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:11.704+0000 7f27bbc00640 1 Processor -- start 2024-03-28T00:23:11.712 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:11.704+0000 7f27bbc00640 1 -- start start 2024-03-28T00:23:11.712 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:11.705+0000 7f27bbc00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f27b41016c0 0x7f27b4199bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:23:11.713 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:11.705+0000 7f27bbc00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f27b41028b0 0x7f27b419a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:23:11.713 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:11.705+0000 7f27bbc00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f27b4103270 0x7f27b41a1180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:23:11.713 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:11.705+0000 7f27bbc00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27b410d370 con 0x7f27b41028b0 2024-03-28T00:23:11.713 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:11.705+0000 7f27bbc00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27b410d1f0 con 0x7f27b4103270 2024-03-28T00:23:11.713 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:11.705+0000 7f27bbc00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27b410d4f0 con 0x7f27b41016c0 2024-03-28T00:23:11.713 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:11.705+0000 7f27b3e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f27b41028b0 0x7f27b419a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:23:11.713 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:11.705+0000 7f27b3e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f27b41028b0 0x7f27b419a110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:38970/0 (socket says 172.21.15.119:38970) 2024-03-28T00:23:11.713 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:11.705+0000 7f27b3e00640 1 -- 172.21.15.119:0/1012109738 learned_addr learned my addr 172.21.15.119:0/1012109738 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:23:11.713 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:11.705+0000 7f27b9200640 1 --2- 172.21.15.119:0/1012109738 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f27b41016c0 0x7f27b4199bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:23:11.713 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:11.705+0000 7f27b9c00640 1 --2- 172.21.15.119:0/1012109738 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f27b4103270 0x7f27b41a1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:23:11.713 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:11.705+0000 7f27b3e00640 1 -- 172.21.15.119:0/1012109738 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f27b41016c0 msgr2=0x7f27b4199bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:11.713 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:11.705+0000 7f27b3e00640 1 --2- 172.21.15.119:0/1012109738 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f27b41016c0 0x7f27b4199bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:11.713 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:11.705+0000 7f27b3e00640 1 -- 172.21.15.119:0/1012109738 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f27b4103270 msgr2=0x7f27b41a1180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:11.713 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:11.705+0000 7f27b3e00640 1 --2- 172.21.15.119:0/1012109738 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f27b4103270 0x7f27b41a1180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:11.713 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:11.705+0000 7f27b3e00640 1 -- 172.21.15.119:0/1012109738 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27b41a17f0 con 0x7f27b41028b0 2024-03-28T00:23:11.714 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:11.705+0000 7f27b9200640 1 --2- 172.21.15.119:0/1012109738 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f27b41016c0 0x7f27b4199bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T00:23:11.714 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:11.706+0000 7f27b3e00640 1 --2- 172.21.15.119:0/1012109738 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f27b41028b0 0x7f27b419a110 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f27a0000c00 tx=0x7f27a002c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:23:11.714 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:11.706+0000 7f27b1600640 1 -- 172.21.15.119:0/1012109738 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27a0043020 con 0x7f27b41028b0 2024-03-28T00:23:11.714 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:11.706+0000 7f27b1600640 1 -- 172.21.15.119:0/1012109738 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f27a00054e0 con 0x7f27b41028b0 2024-03-28T00:23:11.714 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:11.706+0000 7f27b1600640 1 -- 172.21.15.119:0/1012109738 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27a0003c40 con 0x7f27b41028b0 2024-03-28T00:23:11.714 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:11.706+0000 7f27bbc00640 1 -- 172.21.15.119:0/1012109738 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27b41a1a80 con 0x7f27b41028b0 2024-03-28T00:23:11.714 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:11.706+0000 7f27bbc00640 1 -- 172.21.15.119:0/1012109738 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27b41a2020 con 0x7f27b41028b0 2024-03-28T00:23:11.714 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:11.707+0000 7f27bbc00640 1 -- 172.21.15.119:0/1012109738 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27b41a21f0 con 0x7f27b41028b0 2024-03-28T00:23:11.715 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:11.709+0000 7f27b1600640 1 -- 172.21.15.119:0/1012109738 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f27a0046020 con 0x7f27b41028b0 2024-03-28T00:23:11.717 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:11.711+0000 7f27b1600640 1 --2- 172.21.15.119:0/1012109738 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f279c0a2880 0x7f279c0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:23:11.717 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:11.711+0000 7f27b1600640 1 -- 172.21.15.119:0/1012109738 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f27a00f0620 con 0x7f27b41028b0 2024-03-28T00:23:11.717 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:11.711+0000 7f27b9200640 1 --2- 172.21.15.119:0/1012109738 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f279c0a2880 0x7f279c0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:23:11.718 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:11.712+0000 7f27b9200640 1 --2- 172.21.15.119:0/1012109738 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f279c0a2880 0x7f279c0a4d40 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f27b4102710 tx=0x7f27a8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:23:11.722 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:11.716+0000 7f27b1600640 1 -- 172.21.15.119:0/1012109738 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f27a004d020 con 0x7f27b41028b0 2024-03-28T00:23:11.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:11 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:23:11.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:11 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:23:11.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:11 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:11.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:11 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-28T00:23:11.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:11 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-28T00:23:11.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:11 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:23:11.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:11 smithi119 ceph-mon[31518]: Deploying daemon mds.l on smithi165 2024-03-28T00:23:11.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:11 smithi119 ceph-mon[31518]: from='client.? 172.21.15.119:0/2088599605' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-28T00:23:11.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:11 smithi119 ceph-mon[31518]: from='client.? 172.21.15.119:0/2231865240' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-28T00:23:11.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:11 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:23:11.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:11 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:23:11.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:11 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:11.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:11 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-28T00:23:11.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:11 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-28T00:23:11.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:11 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:23:11.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:11 smithi132 ceph-mon[35941]: Deploying daemon mds.l on smithi165 2024-03-28T00:23:11.954 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:11 smithi132 ceph-mon[35941]: from='client.? 172.21.15.119:0/2088599605' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-28T00:23:11.954 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:11 smithi132 ceph-mon[35941]: from='client.? 172.21.15.119:0/2231865240' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-28T00:23:11.961 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:11.955+0000 7f27bbc00640 1 -- 172.21.15.119:0/1012109738 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f27b41a23c0 con 0x7f27b41028b0 2024-03-28T00:23:11.962 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:11.956+0000 7f27b1600640 1 -- 172.21.15.119:0/1012109738 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) v1 ==== 93+0+535 (secure 0 0 0) 0x7f27a00bdac0 con 0x7f27b41028b0 2024-03-28T00:23:11.962 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-28T00:23:11.962 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-03-28T00:23:11.962 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 1 2024-03-28T00:23:11.965 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:11.959+0000 7f27bbc00640 1 -- 172.21.15.119:0/1012109738 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f279c0a2880 msgr2=0x7f279c0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:11.965 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:11.959+0000 7f27bbc00640 1 --2- 172.21.15.119:0/1012109738 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f279c0a2880 0x7f279c0a4d40 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f27b4102710 tx=0x7f27a8005f90 comp rx=0 tx=0).stop 2024-03-28T00:23:11.965 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:11.959+0000 7f27bbc00640 1 -- 172.21.15.119:0/1012109738 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f27b41028b0 msgr2=0x7f27b419a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:11.966 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:11.959+0000 7f27bbc00640 1 --2- 172.21.15.119:0/1012109738 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f27b41028b0 0x7f27b419a110 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f27a0000c00 tx=0x7f27a002c650 comp rx=0 tx=0).stop 2024-03-28T00:23:11.966 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:11.959+0000 7f27bbc00640 1 -- 172.21.15.119:0/1012109738 shutdown_connections 2024-03-28T00:23:11.966 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:11.959+0000 7f27bbc00640 1 --2- 172.21.15.119:0/1012109738 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f279c0a2880 0x7f279c0a4d40 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:11.966 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:11.959+0000 7f27bbc00640 1 --2- 172.21.15.119:0/1012109738 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f27b4103270 0x7f27b41a1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:11.966 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:11.959+0000 7f27bbc00640 1 --2- 172.21.15.119:0/1012109738 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f27b41028b0 0x7f27b419a110 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:11.966 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:11.959+0000 7f27bbc00640 1 --2- 172.21.15.119:0/1012109738 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f27b41016c0 0x7f27b4199bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:11.966 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:11.959+0000 7f27bbc00640 1 -- 172.21.15.119:0/1012109738 >> 172.21.15.119:0/1012109738 conn(0x7f27b40fcc50 msgr2=0x7f27b40738d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:23:11.966 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:11.959+0000 7f27bbc00640 1 -- 172.21.15.119:0/1012109738 shutdown_connections 2024-03-28T00:23:11.966 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:11.959+0000 7f27bbc00640 1 -- 172.21.15.119:0/1012109738 wait complete. 2024-03-28T00:23:12.713 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:12 smithi165 ceph-mon[36302]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T00:23:12.713 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:12 smithi165 ceph-mon[36302]: from='client.? 172.21.15.119:0/1012109738' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T00:23:12.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:12 smithi119 ceph-mon[31518]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T00:23:12.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:12 smithi119 ceph-mon[31518]: from='client.? 172.21.15.119:0/1012109738' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T00:23:12.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:12 smithi132 ceph-mon[35941]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T00:23:12.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:12 smithi132 ceph-mon[35941]: from='client.? 172.21.15.119:0/1012109738' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T00:23:13.108 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:23:12 smithi165 systemd[1]: Starting Ceph mds.l for 10f73b92-ec98-11ee-95d0-87774f69a715... 2024-03-28T00:23:13.375 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:23:13 smithi165 podman[67130]: 2024-03-28 00:23:13.005912362 +0000 UTC m=+0.017561496 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-28T00:23:13.375 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:23:13 smithi165 podman[67130]: 2024-03-28 00:23:13.125958023 +0000 UTC m=+0.137607151 container create fa62f3785e42c16fb08c1a22e89922df56595b0015bf38d1fc63b909bde70e78 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-03-28T00:23:13.632 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:23:13 smithi165 podman[67130]: 2024-03-28 00:23:13.370234528 +0000 UTC m=+0.381883656 container init fa62f3785e42c16fb08c1a22e89922df56595b0015bf38d1fc63b909bde70e78 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-28T00:23:13.633 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:23:13 smithi165 podman[67130]: 2024-03-28 00:23:13.373088967 +0000 UTC m=+0.384738097 container start fa62f3785e42c16fb08c1a22e89922df56595b0015bf38d1fc63b909bde70e78 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-03-28T00:23:13.633 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:23:13 smithi165 ceph-mds[67148]: -- 172.21.15.165:0/636831699 <== mon.1 v2:172.21.15.165:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55a2c446ba40 con 0x55a2c443ec00 2024-03-28T00:23:13.633 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:23:13 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: starting mds.l at 2024-03-28T00:23:13.633 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:23:13 smithi165 bash[67130]: fa62f3785e42c16fb08c1a22e89922df56595b0015bf38d1fc63b909bde70e78 2024-03-28T00:23:13.633 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:23:13 smithi165 systemd[1]: Started Ceph mds.l for 10f73b92-ec98-11ee-95d0-87774f69a715. 2024-03-28T00:23:13.986 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T00:23:14.210 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:14.204+0000 7f0a9e600640 1 -- 172.21.15.119:0/3580416539 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0a98075660 msgr2=0x7f0a98073ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:14.211 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:14.204+0000 7f0a9e600640 1 --2- 172.21.15.119:0/3580416539 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0a98075660 0x7f0a98073ab0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f0a88009900 tx=0x7f0a8802bcd0 comp rx=0 tx=0).stop 2024-03-28T00:23:14.211 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:14.204+0000 7f0a9e600640 1 -- 172.21.15.119:0/3580416539 shutdown_connections 2024-03-28T00:23:14.211 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:14.204+0000 7f0a9e600640 1 --2- 172.21.15.119:0/3580416539 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0a98109570 0x7f0a9810ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:14.211 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:14.204+0000 7f0a9e600640 1 --2- 172.21.15.119:0/3580416539 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0a98074080 0x7f0a980744e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:14.211 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:14.204+0000 7f0a9e600640 1 --2- 172.21.15.119:0/3580416539 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0a98075660 0x7f0a98073ab0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:14.211 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:14.204+0000 7f0a9e600640 1 -- 172.21.15.119:0/3580416539 >> 172.21.15.119:0/3580416539 conn(0x7f0a980fcc50 msgr2=0x7f0a980ff070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:23:14.211 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:14.204+0000 7f0a9e600640 1 -- 172.21.15.119:0/3580416539 shutdown_connections 2024-03-28T00:23:14.211 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:14.205+0000 7f0a9e600640 1 -- 172.21.15.119:0/3580416539 wait complete. 2024-03-28T00:23:14.214 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:14.205+0000 7f0a9e600640 1 Processor -- start 2024-03-28T00:23:14.214 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:14.206+0000 7f0a9e600640 1 -- start start 2024-03-28T00:23:14.214 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:14.206+0000 7f0a9e600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0a98074080 0x7f0a98199ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:23:14.214 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:14.206+0000 7f0a9e600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0a98075660 0x7f0a9819a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:23:14.214 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:14.206+0000 7f0a9e600640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0a98109570 0x7f0a981a1150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:23:14.214 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:14.206+0000 7f0a9e600640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a9810f590 con 0x7f0a98075660 2024-03-28T00:23:14.214 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:14.206+0000 7f0a9e600640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a9810f410 con 0x7f0a98109570 2024-03-28T00:23:14.214 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:14.206+0000 7f0a9e600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a9810f710 con 0x7f0a98074080 2024-03-28T00:23:14.214 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:14.206+0000 7f0a96a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0a98075660 0x7f0a9819a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:23:14.214 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:14.206+0000 7f0a97e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0a98109570 0x7f0a981a1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:23:14.214 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:14.206+0000 7f0a97e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0a98109570 0x7f0a981a1150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.119:60722/0 (socket says 172.21.15.119:60722) 2024-03-28T00:23:14.214 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:14.206+0000 7f0a97e00640 1 -- 172.21.15.119:0/546069661 learned_addr learned my addr 172.21.15.119:0/546069661 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:23:14.214 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:14.206+0000 7f0a97400640 1 --2- 172.21.15.119:0/546069661 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0a98074080 0x7f0a98199ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:23:14.215 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:14.207+0000 7f0a97e00640 1 -- 172.21.15.119:0/546069661 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0a98074080 msgr2=0x7f0a98199ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:14.215 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:14.207+0000 7f0a97e00640 1 --2- 172.21.15.119:0/546069661 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0a98074080 0x7f0a98199ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:14.215 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:14.207+0000 7f0a97e00640 1 -- 172.21.15.119:0/546069661 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0a98075660 msgr2=0x7f0a9819a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:14.215 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:14.207+0000 7f0a97e00640 1 --2- 172.21.15.119:0/546069661 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0a98075660 0x7f0a9819a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:14.215 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:14.207+0000 7f0a97e00640 1 -- 172.21.15.119:0/546069661 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a981a17c0 con 0x7f0a98109570 2024-03-28T00:23:14.215 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:14.207+0000 7f0a96a00640 1 --2- 172.21.15.119:0/546069661 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0a98075660 0x7f0a9819a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T00:23:14.215 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:14.207+0000 7f0a97e00640 1 --2- 172.21.15.119:0/546069661 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0a98109570 0x7f0a981a1150 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f0a9000e970 tx=0x7f0a9000ed70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:23:14.215 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:14.207+0000 7f0a8fe00640 1 -- 172.21.15.119:0/546069661 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a9000ccc0 con 0x7f0a98109570 2024-03-28T00:23:14.215 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:14.207+0000 7f0a9e600640 1 -- 172.21.15.119:0/546069661 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a981a1ab0 con 0x7f0a98109570 2024-03-28T00:23:14.215 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:14.207+0000 7f0a8fe00640 1 -- 172.21.15.119:0/546069661 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0a9000ce60 con 0x7f0a98109570 2024-03-28T00:23:14.215 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:14.207+0000 7f0a8fe00640 1 -- 172.21.15.119:0/546069661 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a90010450 con 0x7f0a98109570 2024-03-28T00:23:14.215 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:14.207+0000 7f0a9e600640 1 -- 172.21.15.119:0/546069661 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a981a2050 con 0x7f0a98109570 2024-03-28T00:23:14.216 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:14.208+0000 7f0a9e600640 1 -- 172.21.15.119:0/546069661 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a98108330 con 0x7f0a98109570 2024-03-28T00:23:14.216 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:14.210+0000 7f0a8fe00640 1 -- 172.21.15.119:0/546069661 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f0a90005020 con 0x7f0a98109570 2024-03-28T00:23:14.217 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:14.211+0000 7f0a8fe00640 1 --2- 172.21.15.119:0/546069661 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f0a7c0a2800 0x7f0a7c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:23:14.218 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:14.212+0000 7f0a8fe00640 1 -- 172.21.15.119:0/546069661 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f0a900cea70 con 0x7f0a98109570 2024-03-28T00:23:14.218 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:14.212+0000 7f0a97400640 1 --2- 172.21.15.119:0/546069661 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f0a7c0a2800 0x7f0a7c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:23:14.218 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:14.212+0000 7f0a97400640 1 --2- 172.21.15.119:0/546069661 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f0a7c0a2800 0x7f0a7c0a4cc0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f0a8802c2e0 tx=0x7f0a8802c920 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:23:14.219 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:14.214+0000 7f0a8fe00640 1 -- 172.21.15.119:0/546069661 <== mon.2 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f0a9009bf10 con 0x7f0a98109570 2024-03-28T00:23:14.461 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:14.455+0000 7f0a9e600640 1 -- 172.21.15.119:0/546069661 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0a9802d070 con 0x7f0a98109570 2024-03-28T00:23:14.462 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:14.456+0000 7f0a8fe00640 1 -- 172.21.15.119:0/546069661 <== mon.2 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1325 (secure 0 0 0) 0x7f0a9009b810 con 0x7f0a98109570 2024-03-28T00:23:14.462 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-28T00:23:14.463 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/609963168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":609963168},{"type":"v1","addr":"172.21.15.165:6833","nonce":609963168}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-03-28T00:23:14.463 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 2 2024-03-28T00:23:14.466 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:14.459+0000 7f0a9e600640 1 -- 172.21.15.119:0/546069661 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f0a7c0a2800 msgr2=0x7f0a7c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:14.466 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:14.459+0000 7f0a9e600640 1 --2- 172.21.15.119:0/546069661 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f0a7c0a2800 0x7f0a7c0a4cc0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f0a8802c2e0 tx=0x7f0a8802c920 comp rx=0 tx=0).stop 2024-03-28T00:23:14.466 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:14.459+0000 7f0a9e600640 1 -- 172.21.15.119:0/546069661 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0a98109570 msgr2=0x7f0a981a1150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:14.466 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:14.459+0000 7f0a9e600640 1 --2- 172.21.15.119:0/546069661 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0a98109570 0x7f0a981a1150 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f0a9000e970 tx=0x7f0a9000ed70 comp rx=0 tx=0).stop 2024-03-28T00:23:14.466 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:14.460+0000 7f0a9e600640 1 -- 172.21.15.119:0/546069661 shutdown_connections 2024-03-28T00:23:14.466 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:14.460+0000 7f0a9e600640 1 --2- 172.21.15.119:0/546069661 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f0a7c0a2800 0x7f0a7c0a4cc0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:14.466 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:14.460+0000 7f0a9e600640 1 --2- 172.21.15.119:0/546069661 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0a98109570 0x7f0a981a1150 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:14.466 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:14.460+0000 7f0a9e600640 1 --2- 172.21.15.119:0/546069661 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0a98075660 0x7f0a9819a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:14.466 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:14.460+0000 7f0a9e600640 1 --2- 172.21.15.119:0/546069661 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0a98074080 0x7f0a98199ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:14.466 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:14.460+0000 7f0a9e600640 1 -- 172.21.15.119:0/546069661 >> 172.21.15.119:0/546069661 conn(0x7f0a980fcc50 msgr2=0x7f0a98105500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:23:14.467 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:14.460+0000 7f0a9e600640 1 -- 172.21.15.119:0/546069661 shutdown_connections 2024-03-28T00:23:14.467 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:14.460+0000 7f0a9e600640 1 -- 172.21.15.119:0/546069661 wait complete. 2024-03-28T00:23:14.565 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:14 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.165:6832/609963168,v1:172.21.15.165:6833/609963168] up:boot 2024-03-28T00:23:14.565 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:14 smithi132 ceph-mon[35941]: fsmap 1 up:standby 2024-03-28T00:23:14.565 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:14 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-28T00:23:14.565 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:14 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:14.565 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:14 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:14.565 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:14 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:14.565 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:14 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-28T00:23:14.565 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:14 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-28T00:23:14.565 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:14 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:23:14.565 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:14 smithi132 ceph-mon[35941]: Deploying daemon mds.k on smithi132 2024-03-28T00:23:14.566 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:14 smithi132 ceph-mon[35941]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T00:23:14.775 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:14 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.165:6832/609963168,v1:172.21.15.165:6833/609963168] up:boot 2024-03-28T00:23:14.776 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:14 smithi119 ceph-mon[31518]: fsmap 1 up:standby 2024-03-28T00:23:14.776 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:14 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-28T00:23:14.776 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:14 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:14.776 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:14 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:14.776 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:14 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:14.776 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:14 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-28T00:23:14.776 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:14 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-28T00:23:14.777 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:14 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:23:14.777 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:14 smithi119 ceph-mon[31518]: Deploying daemon mds.k on smithi132 2024-03-28T00:23:14.777 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:14 smithi119 ceph-mon[31518]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T00:23:14.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:14 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.165:6832/609963168,v1:172.21.15.165:6833/609963168] up:boot 2024-03-28T00:23:14.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:14 smithi165 ceph-mon[36302]: fsmap 1 up:standby 2024-03-28T00:23:14.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:14 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-28T00:23:14.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:14 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:14.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:14 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:14.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:14 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:14.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:14 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-28T00:23:14.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:14 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-28T00:23:14.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:14 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:23:14.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:14 smithi165 ceph-mon[36302]: Deploying daemon mds.k on smithi132 2024-03-28T00:23:14.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:14 smithi165 ceph-mon[36302]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T00:23:15.635 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:15 smithi132 ceph-mon[35941]: from='client.? 172.21.15.119:0/546069661' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T00:23:15.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:15 smithi165 ceph-mon[36302]: from='client.? 172.21.15.119:0/546069661' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T00:23:15.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:15 smithi119 ceph-mon[31518]: from='client.? 172.21.15.119:0/546069661' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T00:23:15.952 INFO:journalctl@ceph.mds.k.smithi132.stdout:Mar 28 00:23:15 smithi132 systemd[1]: Starting Ceph mds.k for 10f73b92-ec98-11ee-95d0-87774f69a715... 2024-03-28T00:23:16.300 INFO:journalctl@ceph.mds.k.smithi132.stdout:Mar 28 00:23:16 smithi132 podman[65483]: 2024-03-28 00:23:15.929473839 +0000 UTC m=+0.017625045 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-28T00:23:16.300 INFO:journalctl@ceph.mds.k.smithi132.stdout:Mar 28 00:23:16 smithi132 podman[65483]: 2024-03-28 00:23:16.050841336 +0000 UTC m=+0.138992582 container create 90f8d0de7d8a42a54de9c0acf392e4eeaeaa3c3694c55603e96b844552c3d82e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-k, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-03-28T00:23:16.300 INFO:journalctl@ceph.mds.k.smithi132.stdout:Mar 28 00:23:16 smithi132 podman[65483]: 2024-03-28 00:23:16.295204607 +0000 UTC m=+0.383355790 container init 90f8d0de7d8a42a54de9c0acf392e4eeaeaa3c3694c55603e96b844552c3d82e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-k, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True, io.buildah.version=1.33.5) 2024-03-28T00:23:16.486 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T00:23:16.681 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:16.675+0000 7f7629800640 1 -- 172.21.15.119:0/2083066942 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f76241030c0 msgr2=0x7f7624109950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:16.681 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:16.675+0000 7f7629800640 1 --2- 172.21.15.119:0/2083066942 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f76241030c0 0x7f7624109950 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f761c009be0 tx=0x7f761c02bef0 comp rx=0 tx=0).stop 2024-03-28T00:23:16.681 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:16.675+0000 7f7629800640 1 -- 172.21.15.119:0/2083066942 shutdown_connections 2024-03-28T00:23:16.681 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:16.675+0000 7f7629800640 1 --2- 172.21.15.119:0/2083066942 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f76241030c0 0x7f7624109950 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:16.682 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:16.675+0000 7f7629800640 1 --2- 172.21.15.119:0/2083066942 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7624102700 0x7f7624102b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:16.682 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:16.675+0000 7f7629800640 1 --2- 172.21.15.119:0/2083066942 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7624101510 0x7f7624101930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:16.682 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:16.675+0000 7f7629800640 1 -- 172.21.15.119:0/2083066942 >> 172.21.15.119:0/2083066942 conn(0x7f76240fca80 msgr2=0x7f76240feec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:23:16.682 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:16.675+0000 7f7629800640 1 -- 172.21.15.119:0/2083066942 shutdown_connections 2024-03-28T00:23:16.682 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:16.675+0000 7f7629800640 1 -- 172.21.15.119:0/2083066942 wait complete. 2024-03-28T00:23:16.684 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:16.676+0000 7f7629800640 1 Processor -- start 2024-03-28T00:23:16.684 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:16.676+0000 7f7629800640 1 -- start start 2024-03-28T00:23:16.684 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:16.676+0000 7f7629800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7624101510 0x7f76241977b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:23:16.684 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:16.676+0000 7f7629800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7624102700 0x7f7624197cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:23:16.684 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:16.676+0000 7f7629800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f76241030c0 0x7f762419ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:23:16.684 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:16.676+0000 7f7629800640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f762410d1f0 con 0x7f7624102700 2024-03-28T00:23:16.684 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:16.676+0000 7f7629800640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f762410d070 con 0x7f76241030c0 2024-03-28T00:23:16.684 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:16.676+0000 7f7629800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f762410d370 con 0x7f7624101510 2024-03-28T00:23:16.684 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:16.677+0000 7f7628e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f76241030c0 0x7f762419ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:23:16.684 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:16.677+0000 7f7628e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f76241030c0 0x7f762419ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.119:60742/0 (socket says 172.21.15.119:60742) 2024-03-28T00:23:16.685 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:16.677+0000 7f7628e00640 1 -- 172.21.15.119:0/3381202996 learned_addr learned my addr 172.21.15.119:0/3381202996 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:23:16.685 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:16.677+0000 7f7623e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7624101510 0x7f76241977b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:23:16.685 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:16.677+0000 7f7623400640 1 --2- 172.21.15.119:0/3381202996 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7624102700 0x7f7624197cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:23:16.685 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:16.677+0000 7f7628e00640 1 -- 172.21.15.119:0/3381202996 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7624101510 msgr2=0x7f76241977b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:16.685 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:16.677+0000 7f7628e00640 1 --2- 172.21.15.119:0/3381202996 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7624101510 0x7f76241977b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:16.685 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:16.677+0000 7f7628e00640 1 -- 172.21.15.119:0/3381202996 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7624102700 msgr2=0x7f7624197cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:16.685 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:16.677+0000 7f7628e00640 1 --2- 172.21.15.119:0/3381202996 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7624102700 0x7f7624197cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:16.685 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:16.677+0000 7f7628e00640 1 -- 172.21.15.119:0/3381202996 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f762419f3d0 con 0x7f76241030c0 2024-03-28T00:23:16.685 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:16.677+0000 7f7628e00640 1 --2- 172.21.15.119:0/3381202996 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f76241030c0 0x7f762419ed60 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f761c036040 tx=0x7f761c0337b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:23:16.685 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:16.677+0000 7f7620c00640 1 -- 172.21.15.119:0/3381202996 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f761c043020 con 0x7f76241030c0 2024-03-28T00:23:16.685 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:16.678+0000 7f7620c00640 1 -- 172.21.15.119:0/3381202996 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f761c02ca60 con 0x7f76241030c0 2024-03-28T00:23:16.685 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:16.678+0000 7f7629800640 1 -- 172.21.15.119:0/3381202996 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f762419f660 con 0x7f76241030c0 2024-03-28T00:23:16.685 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:16.678+0000 7f7620c00640 1 -- 172.21.15.119:0/3381202996 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f761c034690 con 0x7f76241030c0 2024-03-28T00:23:16.685 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:16.678+0000 7f7629800640 1 -- 172.21.15.119:0/3381202996 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f762419fc00 con 0x7f76241030c0 2024-03-28T00:23:16.686 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:16.679+0000 7f7629800640 1 -- 172.21.15.119:0/3381202996 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7624108830 con 0x7f76241030c0 2024-03-28T00:23:16.686 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:16.680+0000 7f7620c00640 1 -- 172.21.15.119:0/3381202996 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f761c046020 con 0x7f76241030c0 2024-03-28T00:23:16.688 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:16.682+0000 7f7620c00640 1 --2- 172.21.15.119:0/3381202996 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f76080a2880 0x7f76080a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:23:16.688 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:16.682+0000 7f7620c00640 1 -- 172.21.15.119:0/3381202996 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f761c0f0c30 con 0x7f76241030c0 2024-03-28T00:23:16.688 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:16.682+0000 7f7623e00640 1 --2- 172.21.15.119:0/3381202996 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f76080a2880 0x7f76080a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:23:16.688 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:16.683+0000 7f7623e00640 1 --2- 172.21.15.119:0/3381202996 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f76080a2880 0x7f76080a4d40 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f7624102560 tx=0x7f7614005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:23:16.692 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:16.686+0000 7f7620c00640 1 -- 172.21.15.119:0/3381202996 <== mon.2 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f761c04d020 con 0x7f76241030c0 2024-03-28T00:23:16.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:16 smithi132 ceph-mon[35941]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T00:23:16.702 INFO:journalctl@ceph.mds.k.smithi132.stdout:Mar 28 00:23:16 smithi132 podman[65483]: 2024-03-28 00:23:16.29803293 +0000 UTC m=+0.386184120 container start 90f8d0de7d8a42a54de9c0acf392e4eeaeaa3c3694c55603e96b844552c3d82e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-k, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-03-28T00:23:16.703 INFO:journalctl@ceph.mds.k.smithi132.stdout:Mar 28 00:23:16 smithi132 ceph-mds[65502]: -- 172.21.15.132:0/241060515 <== mon.2 v2:172.21.15.132:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55c27507ba40 con 0x55c27504ec00 2024-03-28T00:23:16.703 INFO:journalctl@ceph.mds.k.smithi132.stdout:Mar 28 00:23:16 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-k[65498]: starting mds.k at 2024-03-28T00:23:16.703 INFO:journalctl@ceph.mds.k.smithi132.stdout:Mar 28 00:23:16 smithi132 bash[65483]: 90f8d0de7d8a42a54de9c0acf392e4eeaeaa3c3694c55603e96b844552c3d82e 2024-03-28T00:23:16.703 INFO:journalctl@ceph.mds.k.smithi132.stdout:Mar 28 00:23:16 smithi132 systemd[1]: Started Ceph mds.k for 10f73b92-ec98-11ee-95d0-87774f69a715. 2024-03-28T00:23:16.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:16 smithi165 ceph-mon[36302]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T00:23:16.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:16 smithi119 ceph-mon[31518]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T00:23:16.933 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:16.927+0000 7f7629800640 1 -- 172.21.15.119:0/3381202996 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f76240622a0 con 0x7f76241030c0 2024-03-28T00:23:16.934 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:16.928+0000 7f7620c00640 1 -- 172.21.15.119:0/3381202996 <== mon.2 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+2119 (secure 0 0 0) 0x7f761c0be0d0 con 0x7f76241030c0 2024-03-28T00:23:16.934 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-28T00:23:16.934 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/609963168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":609963168},{"type":"v1","addr":"172.21.15.165:6833","nonce":609963168}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/1179016670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":1179016670},{"type":"v1","addr":"172.21.15.132:6833","nonce":1179016670}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2024-03-28T00:23:16.934 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 3 2024-03-28T00:23:16.937 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:16.931+0000 7f7629800640 1 -- 172.21.15.119:0/3381202996 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f76080a2880 msgr2=0x7f76080a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:16.937 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:16.931+0000 7f7629800640 1 --2- 172.21.15.119:0/3381202996 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f76080a2880 0x7f76080a4d40 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f7624102560 tx=0x7f7614005f90 comp rx=0 tx=0).stop 2024-03-28T00:23:16.937 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:16.931+0000 7f7629800640 1 -- 172.21.15.119:0/3381202996 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f76241030c0 msgr2=0x7f762419ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:16.938 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:16.931+0000 7f7629800640 1 --2- 172.21.15.119:0/3381202996 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f76241030c0 0x7f762419ed60 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f761c036040 tx=0x7f761c0337b0 comp rx=0 tx=0).stop 2024-03-28T00:23:16.938 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:16.931+0000 7f7629800640 1 -- 172.21.15.119:0/3381202996 shutdown_connections 2024-03-28T00:23:16.938 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:16.931+0000 7f7629800640 1 --2- 172.21.15.119:0/3381202996 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f76080a2880 0x7f76080a4d40 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:16.938 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:16.931+0000 7f7629800640 1 --2- 172.21.15.119:0/3381202996 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f76241030c0 0x7f762419ed60 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:16.938 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:16.931+0000 7f7629800640 1 --2- 172.21.15.119:0/3381202996 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7624102700 0x7f7624197cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:16.938 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:16.931+0000 7f7629800640 1 --2- 172.21.15.119:0/3381202996 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7624101510 0x7f76241977b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:16.938 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:16.931+0000 7f7629800640 1 -- 172.21.15.119:0/3381202996 >> 172.21.15.119:0/3381202996 conn(0x7f76240fca80 msgr2=0x7f7624105a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:23:16.938 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:16.931+0000 7f7629800640 1 -- 172.21.15.119:0/3381202996 shutdown_connections 2024-03-28T00:23:16.938 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:16.931+0000 7f7629800640 1 -- 172.21.15.119:0/3381202996 wait complete. 2024-03-28T00:23:17.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:17 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.132:6832/1179016670,v1:172.21.15.132:6833/1179016670] up:boot 2024-03-28T00:23:17.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:17 smithi132 ceph-mon[35941]: fsmap 2 up:standby 2024-03-28T00:23:17.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:17 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-28T00:23:17.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:17 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:17.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:17 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:17.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:17 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:17.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:17 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-28T00:23:17.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:17 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-28T00:23:17.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:17 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:23:17.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:17 smithi132 ceph-mon[35941]: Deploying daemon mds.j on smithi119 2024-03-28T00:23:17.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:17 smithi132 ceph-mon[35941]: from='client.? 172.21.15.119:0/3381202996' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T00:23:17.718 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:17 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.132:6832/1179016670,v1:172.21.15.132:6833/1179016670] up:boot 2024-03-28T00:23:17.719 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:17 smithi119 ceph-mon[31518]: fsmap 2 up:standby 2024-03-28T00:23:17.719 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:17 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-28T00:23:17.719 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:17 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:17.719 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:17 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:17.719 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:17 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:17.719 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:17 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-28T00:23:17.719 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:17 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-28T00:23:17.719 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:17 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:23:17.719 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:17 smithi119 ceph-mon[31518]: Deploying daemon mds.j on smithi119 2024-03-28T00:23:17.719 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:17 smithi119 ceph-mon[31518]: from='client.? 172.21.15.119:0/3381202996' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T00:23:17.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:17 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.132:6832/1179016670,v1:172.21.15.132:6833/1179016670] up:boot 2024-03-28T00:23:17.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:17 smithi165 ceph-mon[36302]: fsmap 2 up:standby 2024-03-28T00:23:17.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:17 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-28T00:23:17.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:17 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:17.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:17 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:17.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:17 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:17.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:17 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-28T00:23:17.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:17 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-28T00:23:17.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:17 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:23:17.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:17 smithi165 ceph-mon[36302]: Deploying daemon mds.j on smithi119 2024-03-28T00:23:17.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:17 smithi165 ceph-mon[36302]: from='client.? 172.21.15.119:0/3381202996' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T00:23:18.638 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:18 smithi119 ceph-mon[31518]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T00:23:18.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:18 smithi132 ceph-mon[35941]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T00:23:18.784 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:18 smithi165 ceph-mon[36302]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T00:23:18.956 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T00:23:19.239 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:19.232+0000 7fc5b0600640 1 -- 172.21.15.119:0/3185248272 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc5a80772a0 msgr2=0x7fc5a8077700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:19.239 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:19.232+0000 7fc5b0600640 1 --2- 172.21.15.119:0/3185248272 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc5a80772a0 0x7fc5a8077700 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fc594009900 tx=0x7fc59402bc30 comp rx=0 tx=0).stop 2024-03-28T00:23:19.239 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:19.233+0000 7fc5b0600640 1 -- 172.21.15.119:0/3185248272 shutdown_connections 2024-03-28T00:23:19.239 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:19.233+0000 7fc5b0600640 1 --2- 172.21.15.119:0/3185248272 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc5a8077c40 0x7fc5a80784e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:19.239 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:19.233+0000 7fc5b0600640 1 --2- 172.21.15.119:0/3185248272 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc5a80772a0 0x7fc5a8077700 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:19.239 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:19.233+0000 7fc5b0600640 1 --2- 172.21.15.119:0/3185248272 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc5a8076060 0x7fc5a8076480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:19.240 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:19.233+0000 7fc5b0600640 1 -- 172.21.15.119:0/3185248272 >> 172.21.15.119:0/3185248272 conn(0x7fc5a80fee90 msgr2=0x7fc5a81012b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:23:19.240 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:19.233+0000 7fc5b0600640 1 -- 172.21.15.119:0/3185248272 shutdown_connections 2024-03-28T00:23:19.240 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:19.233+0000 7fc5b0600640 1 -- 172.21.15.119:0/3185248272 wait complete. 2024-03-28T00:23:19.241 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:19.234+0000 7fc5b0600640 1 Processor -- start 2024-03-28T00:23:19.241 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:19.234+0000 7fc5b0600640 1 -- start start 2024-03-28T00:23:19.241 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:19.234+0000 7fc5b0600640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc5a8076060 0x7fc5a819dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:23:19.241 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:19.234+0000 7fc5b0600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc5a80772a0 0x7fc5a819e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:23:19.241 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:19.234+0000 7fc5b0600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc5a8077c40 0x7fc5a81a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:23:19.241 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:19.234+0000 7fc5b0600640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a81139b0 con 0x7fc5a80772a0 2024-03-28T00:23:19.241 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:19.234+0000 7fc5b0600640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a8113830 con 0x7fc5a8076060 2024-03-28T00:23:19.241 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:19.234+0000 7fc5b0600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a8113b30 con 0x7fc5a8077c40 2024-03-28T00:23:19.241 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:19.235+0000 7fc5adc00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc5a8076060 0x7fc5a819dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:23:19.242 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:19.235+0000 7fc5ad200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc5a80772a0 0x7fc5a819e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:23:19.242 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:19.235+0000 7fc5adc00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc5a8076060 0x7fc5a819dfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.119:60756/0 (socket says 172.21.15.119:60756) 2024-03-28T00:23:19.242 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:19.235+0000 7fc5adc00640 1 -- 172.21.15.119:0/3272924629 learned_addr learned my addr 172.21.15.119:0/3272924629 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:23:19.242 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:19.235+0000 7fc5ae600640 1 --2- 172.21.15.119:0/3272924629 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc5a8077c40 0x7fc5a81a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:23:19.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:19.235+0000 7fc5adc00640 1 -- 172.21.15.119:0/3272924629 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc5a8077c40 msgr2=0x7fc5a81a5550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:19.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:19.235+0000 7fc5adc00640 1 --2- 172.21.15.119:0/3272924629 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc5a8077c40 0x7fc5a81a5550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:19.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:19.235+0000 7fc5adc00640 1 -- 172.21.15.119:0/3272924629 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc5a80772a0 msgr2=0x7fc5a819e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:19.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:19.235+0000 7fc5adc00640 1 --2- 172.21.15.119:0/3272924629 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc5a80772a0 0x7fc5a819e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:19.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:19.235+0000 7fc5adc00640 1 -- 172.21.15.119:0/3272924629 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5a81a5bc0 con 0x7fc5a8076060 2024-03-28T00:23:19.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:19.235+0000 7fc5ad200640 1 --2- 172.21.15.119:0/3272924629 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc5a80772a0 0x7fc5a819e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T00:23:19.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:19.236+0000 7fc5adc00640 1 --2- 172.21.15.119:0/3272924629 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc5a8076060 0x7fc5a819dfa0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc59c00e9f0 tx=0x7fc59c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:23:19.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:19.236+0000 7fc5a6000640 1 -- 172.21.15.119:0/3272924629 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc59c00cd80 con 0x7fc5a8076060 2024-03-28T00:23:19.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:19.236+0000 7fc5b0600640 1 -- 172.21.15.119:0/3272924629 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5a81a5eb0 con 0x7fc5a8076060 2024-03-28T00:23:19.244 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:19.236+0000 7fc5a6000640 1 -- 172.21.15.119:0/3272924629 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc59c010910 con 0x7fc5a8076060 2024-03-28T00:23:19.244 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:19.236+0000 7fc5a6000640 1 -- 172.21.15.119:0/3272924629 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc59c0188f0 con 0x7fc5a8076060 2024-03-28T00:23:19.244 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:19.237+0000 7fc5b0600640 1 -- 172.21.15.119:0/3272924629 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5a81a6320 con 0x7fc5a8076060 2024-03-28T00:23:19.244 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:19.238+0000 7fc5b0600640 1 -- 172.21.15.119:0/3272924629 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5a8110390 con 0x7fc5a8076060 2024-03-28T00:23:19.246 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:19.241+0000 7fc5a6000640 1 -- 172.21.15.119:0/3272924629 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fc59c010ac0 con 0x7fc5a8076060 2024-03-28T00:23:19.248 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:19.242+0000 7fc5a6000640 1 --2- 172.21.15.119:0/3272924629 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fc5900a2880 0x7fc5900a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:23:19.248 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:19.242+0000 7fc5a6000640 1 -- 172.21.15.119:0/3272924629 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fc59c0ce950 con 0x7fc5a8076060 2024-03-28T00:23:19.248 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:19.242+0000 7fc5ad200640 1 --2- 172.21.15.119:0/3272924629 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fc5900a2880 0x7fc5900a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:23:19.249 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:19.243+0000 7fc5ad200640 1 --2- 172.21.15.119:0/3272924629 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fc5900a2880 0x7fc5900a4d40 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fc59402c240 tx=0x7fc59402df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:23:19.252 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:19.246+0000 7fc5a6000640 1 -- 172.21.15.119:0/3272924629 <== mon.2 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fc59c09b7a0 con 0x7fc5a8076060 2024-03-28T00:23:19.503 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:19.497+0000 7fc5b0600640 1 -- 172.21.15.119:0/3272924629 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc5a80623a0 con 0x7fc5a8076060 2024-03-28T00:23:19.503 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:19.498+0000 7fc5a6000640 1 -- 172.21.15.119:0/3272924629 <== mon.2 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+2119 (secure 0 0 0) 0x7fc59c09f840 con 0x7fc5a8076060 2024-03-28T00:23:19.504 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-28T00:23:19.504 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/609963168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":609963168},{"type":"v1","addr":"172.21.15.165:6833","nonce":609963168}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/1179016670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":1179016670},{"type":"v1","addr":"172.21.15.132:6833","nonce":1179016670}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2024-03-28T00:23:19.504 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 3 2024-03-28T00:23:19.506 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:19.500+0000 7fc5b0600640 1 -- 172.21.15.119:0/3272924629 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fc5900a2880 msgr2=0x7fc5900a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:19.506 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:19.500+0000 7fc5b0600640 1 --2- 172.21.15.119:0/3272924629 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fc5900a2880 0x7fc5900a4d40 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fc59402c240 tx=0x7fc59402df90 comp rx=0 tx=0).stop 2024-03-28T00:23:19.507 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:19.500+0000 7fc5b0600640 1 -- 172.21.15.119:0/3272924629 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc5a8076060 msgr2=0x7fc5a819dfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:19.507 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:19.500+0000 7fc5b0600640 1 --2- 172.21.15.119:0/3272924629 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc5a8076060 0x7fc5a819dfa0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc59c00e9f0 tx=0x7fc59c00eeb0 comp rx=0 tx=0).stop 2024-03-28T00:23:19.507 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:19.500+0000 7fc5b0600640 1 -- 172.21.15.119:0/3272924629 shutdown_connections 2024-03-28T00:23:19.507 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:19.500+0000 7fc5b0600640 1 --2- 172.21.15.119:0/3272924629 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fc5900a2880 0x7fc5900a4d40 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:19.507 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:19.500+0000 7fc5b0600640 1 --2- 172.21.15.119:0/3272924629 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc5a8077c40 0x7fc5a81a5550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:19.507 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:19.500+0000 7fc5b0600640 1 --2- 172.21.15.119:0/3272924629 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc5a80772a0 0x7fc5a819e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:19.507 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:19.500+0000 7fc5b0600640 1 --2- 172.21.15.119:0/3272924629 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc5a8076060 0x7fc5a819dfa0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:19.507 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:19.500+0000 7fc5b0600640 1 -- 172.21.15.119:0/3272924629 >> 172.21.15.119:0/3272924629 conn(0x7fc5a80fee90 msgr2=0x7fc5a8100490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:23:19.507 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:19.500+0000 7fc5b0600640 1 -- 172.21.15.119:0/3272924629 shutdown_connections 2024-03-28T00:23:19.507 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:19.501+0000 7fc5b0600640 1 -- 172.21.15.119:0/3272924629 wait complete. 2024-03-28T00:23:19.824 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:19 smithi119 ceph-mon[31518]: from='client.? 172.21.15.119:0/3272924629' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T00:23:19.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:19 smithi165 ceph-mon[36302]: from='client.? 172.21.15.119:0/3272924629' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T00:23:19.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:19 smithi132 ceph-mon[35941]: from='client.? 172.21.15.119:0/3272924629' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T00:23:20.155 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:23:19 smithi119 podman[68767]: 2024-03-28 00:23:19.729100619 +0000 UTC m=+0.034293861 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-28T00:23:20.155 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:23:20 smithi119 podman[68767]: 2024-03-28 00:23:20.147526189 +0000 UTC m=+0.452719475 container init 91c6d8f59d030cc4ae168abd47e8a10ab49251520429c439228e62e74fac9f42 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-03-28T00:23:20.406 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:23:20 smithi119 podman[68767]: 2024-03-28 00:23:20.158584869 +0000 UTC m=+0.463778126 container start 91c6d8f59d030cc4ae168abd47e8a10ab49251520429c439228e62e74fac9f42 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c) 2024-03-28T00:23:20.406 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:23:20 smithi119 ceph-mds[68786]: -- 172.21.15.119:0/1652265208 <== mon.0 v2:172.21.15.119:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55f97a677a40 con 0x55f97a64b000 2024-03-28T00:23:20.406 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:23:20 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: starting mds.j at 2024-03-28T00:23:20.406 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:23:20 smithi119 bash[68767]: 91c6d8f59d030cc4ae168abd47e8a10ab49251520429c439228e62e74fac9f42 2024-03-28T00:23:20.406 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:23:20 smithi119 systemd[1]: Started Ceph mds.j for 10f73b92-ec98-11ee-95d0-87774f69a715. 2024-03-28T00:23:20.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:20 smithi165 ceph-mon[36302]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T00:23:20.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:20 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:20.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:20 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:20.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:20 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:20.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:20 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-28T00:23:20.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:20 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-28T00:23:20.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:20 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:23:20.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:20 smithi119 ceph-mon[31518]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T00:23:20.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:20 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:20.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:20 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:20.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:20 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:20.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:20 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-28T00:23:20.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:20 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-28T00:23:20.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:20 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:23:20.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:20 smithi132 ceph-mon[35941]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T00:23:20.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:20 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:20.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:20 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:20.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:20 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:20.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:20 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-28T00:23:20.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:20 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-28T00:23:20.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:20 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:23:21.527 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T00:23:21.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:21 smithi119 ceph-mon[31518]: Deploying daemon mds.d on smithi119 2024-03-28T00:23:21.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:21 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.119:6834/744950487,v1:172.21.15.119:6835/744950487] up:boot 2024-03-28T00:23:21.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:21 smithi119 ceph-mon[31518]: fsmap 3 up:standby 2024-03-28T00:23:21.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:21 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-28T00:23:21.751 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:21.745+0000 7f3f08000640 1 -- 172.21.15.119:0/2002858056 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3f00104ad0 msgr2=0x7f3f00104f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:21.751 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:21.745+0000 7f3f08000640 1 --2- 172.21.15.119:0/2002858056 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3f00104ad0 0x7f3f00104f50 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f3eec009900 tx=0x7f3eec02bc30 comp rx=0 tx=0).stop 2024-03-28T00:23:21.751 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:21.745+0000 7f3f08000640 1 -- 172.21.15.119:0/2002858056 shutdown_connections 2024-03-28T00:23:21.752 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:21.745+0000 7f3f08000640 1 --2- 172.21.15.119:0/2002858056 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3f00105490 0x7f3f0010bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:21.752 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:21.745+0000 7f3f08000640 1 --2- 172.21.15.119:0/2002858056 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3f00104ad0 0x7f3f00104f50 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:21.752 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:21.745+0000 7f3f08000640 1 --2- 172.21.15.119:0/2002858056 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3f001038e0 0x7f3f00103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:21.752 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:21.745+0000 7f3f08000640 1 -- 172.21.15.119:0/2002858056 >> 172.21.15.119:0/2002858056 conn(0x7f3f000fee90 msgr2=0x7f3f001012b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:23:21.752 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:21.745+0000 7f3f08000640 1 -- 172.21.15.119:0/2002858056 shutdown_connections 2024-03-28T00:23:21.752 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:21.745+0000 7f3f08000640 1 -- 172.21.15.119:0/2002858056 wait complete. 2024-03-28T00:23:21.754 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:21.746+0000 7f3f08000640 1 Processor -- start 2024-03-28T00:23:21.754 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:21.746+0000 7f3f08000640 1 -- start start 2024-03-28T00:23:21.754 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:21.747+0000 7f3f08000640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3f001038e0 0x7f3f00199c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:23:21.754 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:21.747+0000 7f3f08000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3f00104ad0 0x7f3f0019a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:23:21.754 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:21.747+0000 7f3f08000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3f00105490 0x7f3f001a1190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:23:21.754 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:21.747+0000 7f3f08000640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f0010f5b0 con 0x7f3f00104ad0 2024-03-28T00:23:21.754 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:21.747+0000 7f3f08000640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f0010f430 con 0x7f3f001038e0 2024-03-28T00:23:21.754 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:21.747+0000 7f3f08000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f0010f730 con 0x7f3f00105490 2024-03-28T00:23:21.754 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:21.747+0000 7f3f04c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3f00104ad0 0x7f3f0019a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:23:21.754 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:21.747+0000 7f3f04c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3f00104ad0 0x7f3f0019a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40748/0 (socket says 172.21.15.119:40748) 2024-03-28T00:23:21.754 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:21.747+0000 7f3f04c00640 1 -- 172.21.15.119:0/2488385255 learned_addr learned my addr 172.21.15.119:0/2488385255 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:23:21.755 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:21.747+0000 7f3f05600640 1 --2- 172.21.15.119:0/2488385255 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3f001038e0 0x7f3f00199c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:23:21.755 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:21.747+0000 7f3f06000640 1 --2- 172.21.15.119:0/2488385255 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3f00105490 0x7f3f001a1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:23:21.755 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:21.748+0000 7f3f04c00640 1 -- 172.21.15.119:0/2488385255 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3f00105490 msgr2=0x7f3f001a1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:21.755 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:21.748+0000 7f3f04c00640 1 --2- 172.21.15.119:0/2488385255 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3f00105490 0x7f3f001a1190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:21.755 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:21.748+0000 7f3f04c00640 1 -- 172.21.15.119:0/2488385255 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3f001038e0 msgr2=0x7f3f00199c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:21.755 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:21.748+0000 7f3f04c00640 1 --2- 172.21.15.119:0/2488385255 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3f001038e0 0x7f3f00199c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:21.755 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:21.748+0000 7f3f04c00640 1 -- 172.21.15.119:0/2488385255 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f001a1830 con 0x7f3f00104ad0 2024-03-28T00:23:21.755 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:21.748+0000 7f3f06000640 1 --2- 172.21.15.119:0/2488385255 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3f00105490 0x7f3f001a1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T00:23:21.755 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:21.748+0000 7f3f04c00640 1 --2- 172.21.15.119:0/2488385255 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3f00104ad0 0x7f3f0019a160 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f3eec000c00 tx=0x7f3eec02c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:23:21.756 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:21.748+0000 7f3efe000640 1 -- 172.21.15.119:0/2488385255 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eec043020 con 0x7f3f00104ad0 2024-03-28T00:23:21.756 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:21.748+0000 7f3efe000640 1 -- 172.21.15.119:0/2488385255 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3eec0054e0 con 0x7f3f00104ad0 2024-03-28T00:23:21.756 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:21.748+0000 7f3efe000640 1 -- 172.21.15.119:0/2488385255 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eec003c40 con 0x7f3f00104ad0 2024-03-28T00:23:21.756 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:21.748+0000 7f3f08000640 1 -- 172.21.15.119:0/2488385255 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f001a1ac0 con 0x7f3f00104ad0 2024-03-28T00:23:21.756 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:21.749+0000 7f3f08000640 1 -- 172.21.15.119:0/2488385255 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f001a2010 con 0x7f3f00104ad0 2024-03-28T00:23:21.756 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:21.750+0000 7f3f08000640 1 -- 172.21.15.119:0/2488385255 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f0010ac00 con 0x7f3f00104ad0 2024-03-28T00:23:21.757 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:21.752+0000 7f3efe000640 1 -- 172.21.15.119:0/2488385255 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f3eec046020 con 0x7f3f00104ad0 2024-03-28T00:23:21.760 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:21.754+0000 7f3efe000640 1 --2- 172.21.15.119:0/2488385255 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f3ee40a2800 0x7f3ee40a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:23:21.760 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:21.754+0000 7f3efe000640 1 -- 172.21.15.119:0/2488385255 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f3eec0f0b50 con 0x7f3f00104ad0 2024-03-28T00:23:21.760 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:21.754+0000 7f3f05600640 1 --2- 172.21.15.119:0/2488385255 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f3ee40a2800 0x7f3ee40a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:23:21.760 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:21.755+0000 7f3f05600640 1 --2- 172.21.15.119:0/2488385255 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f3ee40a2800 0x7f3ee40a4cc0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f3f0019b0f0 tx=0x7f3ef0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:23:21.763 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:21.757+0000 7f3efe000640 1 -- 172.21.15.119:0/2488385255 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f3eec04d020 con 0x7f3f00104ad0 2024-03-28T00:23:21.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:21 smithi165 ceph-mon[36302]: Deploying daemon mds.d on smithi119 2024-03-28T00:23:21.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:21 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.119:6834/744950487,v1:172.21.15.119:6835/744950487] up:boot 2024-03-28T00:23:21.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:21 smithi165 ceph-mon[36302]: fsmap 3 up:standby 2024-03-28T00:23:21.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:21 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-28T00:23:21.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:21 smithi132 ceph-mon[35941]: Deploying daemon mds.d on smithi119 2024-03-28T00:23:21.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:21 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.119:6834/744950487,v1:172.21.15.119:6835/744950487] up:boot 2024-03-28T00:23:21.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:21 smithi132 ceph-mon[35941]: fsmap 3 up:standby 2024-03-28T00:23:21.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:21 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-28T00:23:22.009 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:22.003+0000 7f3f08000640 1 -- 172.21.15.119:0/2488385255 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3f0002d070 con 0x7f3f00104ad0 2024-03-28T00:23:22.010 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:22.004+0000 7f3efe000640 1 -- 172.21.15.119:0/2488385255 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) v1 ==== 93+0+2910 (secure 0 0 0) 0x7f3eec0bdff0 con 0x7f3f00104ad0 2024-03-28T00:23:22.010 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-28T00:23:22.010 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/609963168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":609963168},{"type":"v1","addr":"172.21.15.165:6833","nonce":609963168}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/1179016670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":1179016670},{"type":"v1","addr":"172.21.15.132:6833","nonce":1179016670}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24473,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/744950487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":744950487},{"type":"v1","addr":"172.21.15.119:6835","nonce":744950487}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4}],"filesystems":[]} 2024-03-28T00:23:22.010 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 4 2024-03-28T00:23:22.013 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:22.006+0000 7f3f08000640 1 -- 172.21.15.119:0/2488385255 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f3ee40a2800 msgr2=0x7f3ee40a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:22.013 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:22.006+0000 7f3f08000640 1 --2- 172.21.15.119:0/2488385255 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f3ee40a2800 0x7f3ee40a4cc0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f3f0019b0f0 tx=0x7f3ef0005f90 comp rx=0 tx=0).stop 2024-03-28T00:23:22.013 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:22.006+0000 7f3f08000640 1 -- 172.21.15.119:0/2488385255 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3f00104ad0 msgr2=0x7f3f0019a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:22.013 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:22.006+0000 7f3f08000640 1 --2- 172.21.15.119:0/2488385255 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3f00104ad0 0x7f3f0019a160 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f3eec000c00 tx=0x7f3eec02c650 comp rx=0 tx=0).stop 2024-03-28T00:23:22.013 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:22.006+0000 7f3f08000640 1 -- 172.21.15.119:0/2488385255 shutdown_connections 2024-03-28T00:23:22.013 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:22.006+0000 7f3f08000640 1 --2- 172.21.15.119:0/2488385255 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f3ee40a2800 0x7f3ee40a4cc0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:22.013 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:22.006+0000 7f3f08000640 1 --2- 172.21.15.119:0/2488385255 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3f00105490 0x7f3f001a1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:22.013 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:22.006+0000 7f3f08000640 1 --2- 172.21.15.119:0/2488385255 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3f00104ad0 0x7f3f0019a160 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:22.013 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:22.006+0000 7f3f08000640 1 --2- 172.21.15.119:0/2488385255 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3f001038e0 0x7f3f00199c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:22.013 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:22.006+0000 7f3f08000640 1 -- 172.21.15.119:0/2488385255 >> 172.21.15.119:0/2488385255 conn(0x7f3f000fee90 msgr2=0x7f3f00107dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:23:22.014 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:22.007+0000 7f3f08000640 1 -- 172.21.15.119:0/2488385255 shutdown_connections 2024-03-28T00:23:22.014 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:22.007+0000 7f3f08000640 1 -- 172.21.15.119:0/2488385255 wait complete. 2024-03-28T00:23:22.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:22 smithi165 ceph-mon[36302]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T00:23:22.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:22 smithi165 ceph-mon[36302]: from='client.? 172.21.15.119:0/2488385255' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T00:23:22.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:22 smithi119 ceph-mon[31518]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T00:23:22.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:22 smithi119 ceph-mon[31518]: from='client.? 172.21.15.119:0/2488385255' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T00:23:22.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:22 smithi132 ceph-mon[35941]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T00:23:22.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:22 smithi132 ceph-mon[35941]: from='client.? 172.21.15.119:0/2488385255' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T00:23:23.836 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:23:23 smithi119 systemd[1]: Starting Ceph mds.d for 10f73b92-ec98-11ee-95d0-87774f69a715... 2024-03-28T00:23:24.031 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T00:23:24.109 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:23:23 smithi119 podman[69105]: 2024-03-28 00:23:23.737475537 +0000 UTC m=+0.036673499 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-28T00:23:24.109 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:23:23 smithi119 podman[69105]: 2024-03-28 00:23:23.88269772 +0000 UTC m=+0.181895681 container create b6a9b8c07b0db4f6ddf2798322d3605dad93cedc394640dbb4bc3b63a167cec7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-03-28T00:23:24.240 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:24.234+0000 7fa066800640 1 -- 172.21.15.119:0/3701218552 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa060105490 msgr2=0x7fa06010bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:24.241 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:24.234+0000 7fa066800640 1 --2- 172.21.15.119:0/3701218552 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa060105490 0x7fa06010bd20 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fa058009ca0 tx=0x7fa05802c0f0 comp rx=0 tx=0).stop 2024-03-28T00:23:24.241 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:24.234+0000 7fa066800640 1 -- 172.21.15.119:0/3701218552 shutdown_connections 2024-03-28T00:23:24.241 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:24.234+0000 7fa066800640 1 --2- 172.21.15.119:0/3701218552 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa060105490 0x7fa06010bd20 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:24.241 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:24.234+0000 7fa066800640 1 --2- 172.21.15.119:0/3701218552 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa060104ad0 0x7fa060104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:24.241 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:24.234+0000 7fa066800640 1 --2- 172.21.15.119:0/3701218552 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa0601038e0 0x7fa060103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:24.241 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:24.234+0000 7fa066800640 1 -- 172.21.15.119:0/3701218552 >> 172.21.15.119:0/3701218552 conn(0x7fa0600fee90 msgr2=0x7fa0601012b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:23:24.241 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:24.234+0000 7fa066800640 1 -- 172.21.15.119:0/3701218552 shutdown_connections 2024-03-28T00:23:24.241 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:24.234+0000 7fa066800640 1 -- 172.21.15.119:0/3701218552 wait complete. 2024-03-28T00:23:24.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:24.235+0000 7fa066800640 1 Processor -- start 2024-03-28T00:23:24.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:24.235+0000 7fa066800640 1 -- start start 2024-03-28T00:23:24.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:24.236+0000 7fa066800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa0601038e0 0x7fa060199b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:23:24.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:24.236+0000 7fa066800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa060104ad0 0x7fa06019a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:23:24.244 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:24.236+0000 7fa066800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa060105490 0x7fa0601a1130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:23:24.244 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:24.236+0000 7fa066800640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa06010f560 con 0x7fa0601038e0 2024-03-28T00:23:24.244 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:24.236+0000 7fa066800640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa06010f3e0 con 0x7fa060105490 2024-03-28T00:23:24.244 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:24.236+0000 7fa066800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa06010f6e0 con 0x7fa060104ad0 2024-03-28T00:23:24.244 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:24.236+0000 7fa05f400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa0601038e0 0x7fa060199b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:23:24.244 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:24.236+0000 7fa05f400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa0601038e0 0x7fa060199b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41692/0 (socket says 172.21.15.119:41692) 2024-03-28T00:23:24.244 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:24.236+0000 7fa05f400640 1 -- 172.21.15.119:0/925416157 learned_addr learned my addr 172.21.15.119:0/925416157 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:23:24.244 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:24.236+0000 7fa05fe00640 1 --2- 172.21.15.119:0/925416157 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa060105490 0x7fa0601a1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:23:24.244 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:24.236+0000 7fa05ea00640 1 --2- 172.21.15.119:0/925416157 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa060104ad0 0x7fa06019a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:23:24.244 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:24.236+0000 7fa05f400640 1 -- 172.21.15.119:0/925416157 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa060104ad0 msgr2=0x7fa06019a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:24.244 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:24.236+0000 7fa05f400640 1 --2- 172.21.15.119:0/925416157 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa060104ad0 0x7fa06019a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:24.244 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:24.236+0000 7fa05f400640 1 -- 172.21.15.119:0/925416157 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa060105490 msgr2=0x7fa0601a1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:24.244 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:24.236+0000 7fa05f400640 1 --2- 172.21.15.119:0/925416157 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa060105490 0x7fa0601a1130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:24.245 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:24.237+0000 7fa05f400640 1 -- 172.21.15.119:0/925416157 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0601a17a0 con 0x7fa0601038e0 2024-03-28T00:23:24.245 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:24.237+0000 7fa05f400640 1 --2- 172.21.15.119:0/925416157 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa0601038e0 0x7fa060199b80 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa05000e9c0 tx=0x7fa05000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:23:24.245 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:24.237+0000 7fa057e00640 1 -- 172.21.15.119:0/925416157 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa05000cd80 con 0x7fa0601038e0 2024-03-28T00:23:24.245 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:24.237+0000 7fa057e00640 1 -- 172.21.15.119:0/925416157 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa0500035f0 con 0x7fa0601038e0 2024-03-28T00:23:24.245 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:24.237+0000 7fa057e00640 1 -- 172.21.15.119:0/925416157 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa050010690 con 0x7fa0601038e0 2024-03-28T00:23:24.245 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:24.237+0000 7fa066800640 1 -- 172.21.15.119:0/925416157 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0601a1a90 con 0x7fa0601038e0 2024-03-28T00:23:24.245 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:24.237+0000 7fa066800640 1 -- 172.21.15.119:0/925416157 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0601a1f90 con 0x7fa0601038e0 2024-03-28T00:23:24.245 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:24.239+0000 7fa066800640 1 -- 172.21.15.119:0/925416157 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa06010ac00 con 0x7fa0601038e0 2024-03-28T00:23:24.247 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:24.241+0000 7fa057e00640 1 -- 172.21.15.119:0/925416157 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fa0500037a0 con 0x7fa0601038e0 2024-03-28T00:23:24.249 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:24.243+0000 7fa057e00640 1 --2- 172.21.15.119:0/925416157 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fa0440a2880 0x7fa0440a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:23:24.249 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:24.243+0000 7fa057e00640 1 -- 172.21.15.119:0/925416157 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fa0500ce930 con 0x7fa0601038e0 2024-03-28T00:23:24.249 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:24.243+0000 7fa05ea00640 1 --2- 172.21.15.119:0/925416157 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fa0440a2880 0x7fa0440a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:23:24.249 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:24.244+0000 7fa05ea00640 1 --2- 172.21.15.119:0/925416157 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fa0440a2880 0x7fa0440a4d40 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa04c0095f0 tx=0x7fa04c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:23:24.253 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:24.247+0000 7fa057e00640 1 -- 172.21.15.119:0/925416157 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa05009bdd0 con 0x7fa0601038e0 2024-03-28T00:23:24.405 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:23:24 smithi119 podman[69105]: 2024-03-28 00:23:24.187048944 +0000 UTC m=+0.486246915 container init b6a9b8c07b0db4f6ddf2798322d3605dad93cedc394640dbb4bc3b63a167cec7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c) 2024-03-28T00:23:24.405 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:23:24 smithi119 podman[69105]: 2024-03-28 00:23:24.198558922 +0000 UTC m=+0.497756861 container start b6a9b8c07b0db4f6ddf2798322d3605dad93cedc394640dbb4bc3b63a167cec7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-03-28T00:23:24.405 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:23:24 smithi119 ceph-mds[69144]: -- 172.21.15.119:0/3926107801 <== mon.0 v2:172.21.15.119:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x556a467c1a40 con 0x556a46794c00 2024-03-28T00:23:24.405 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:23:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: starting mds.d at 2024-03-28T00:23:24.405 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:23:24 smithi119 bash[69105]: b6a9b8c07b0db4f6ddf2798322d3605dad93cedc394640dbb4bc3b63a167cec7 2024-03-28T00:23:24.405 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:23:24 smithi119 systemd[1]: Started Ceph mds.d for 10f73b92-ec98-11ee-95d0-87774f69a715. 2024-03-28T00:23:24.510 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:24.504+0000 7fa066800640 1 -- 172.21.15.119:0/925416157 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa06002d2b0 con 0x7fa0601038e0 2024-03-28T00:23:24.511 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:24.505+0000 7fa057e00640 1 -- 172.21.15.119:0/925416157 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) v1 ==== 93+0+2910 (secure 0 0 0) 0x7fa0500860e0 con 0x7fa0601038e0 2024-03-28T00:23:24.511 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-28T00:23:24.511 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/609963168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":609963168},{"type":"v1","addr":"172.21.15.165:6833","nonce":609963168}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/1179016670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":1179016670},{"type":"v1","addr":"172.21.15.132:6833","nonce":1179016670}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24473,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/744950487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":744950487},{"type":"v1","addr":"172.21.15.119:6835","nonce":744950487}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4}],"filesystems":[]} 2024-03-28T00:23:24.512 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 4 2024-03-28T00:23:24.514 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:24.507+0000 7fa066800640 1 -- 172.21.15.119:0/925416157 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fa0440a2880 msgr2=0x7fa0440a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:24.514 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:24.507+0000 7fa066800640 1 --2- 172.21.15.119:0/925416157 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fa0440a2880 0x7fa0440a4d40 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa04c0095f0 tx=0x7fa04c005f90 comp rx=0 tx=0).stop 2024-03-28T00:23:24.514 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:24.507+0000 7fa066800640 1 -- 172.21.15.119:0/925416157 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa0601038e0 msgr2=0x7fa060199b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:24.514 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:24.507+0000 7fa066800640 1 --2- 172.21.15.119:0/925416157 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa0601038e0 0x7fa060199b80 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa05000e9c0 tx=0x7fa05000ee80 comp rx=0 tx=0).stop 2024-03-28T00:23:24.514 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:24.508+0000 7fa066800640 1 -- 172.21.15.119:0/925416157 shutdown_connections 2024-03-28T00:23:24.514 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:24.508+0000 7fa066800640 1 --2- 172.21.15.119:0/925416157 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fa0440a2880 0x7fa0440a4d40 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:24.514 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:24.508+0000 7fa066800640 1 --2- 172.21.15.119:0/925416157 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa060105490 0x7fa0601a1130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:24.514 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:24.508+0000 7fa066800640 1 --2- 172.21.15.119:0/925416157 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa060104ad0 0x7fa06019a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:24.514 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:24.508+0000 7fa066800640 1 --2- 172.21.15.119:0/925416157 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa0601038e0 0x7fa060199b80 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:24.514 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:24.508+0000 7fa066800640 1 -- 172.21.15.119:0/925416157 >> 172.21.15.119:0/925416157 conn(0x7fa0600fee90 msgr2=0x7fa060107dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:23:24.514 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:24.508+0000 7fa066800640 1 -- 172.21.15.119:0/925416157 shutdown_connections 2024-03-28T00:23:24.515 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:24.508+0000 7fa066800640 1 -- 172.21.15.119:0/925416157 wait complete. 2024-03-28T00:23:24.780 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:24 smithi119 ceph-mon[31518]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T00:23:24.781 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:24 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:24.781 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:24 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:24.781 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:24 smithi119 ceph-mon[31518]: from='client.? 172.21.15.119:0/925416157' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T00:23:25.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:24 smithi165 ceph-mon[36302]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T00:23:25.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:24 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:25.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:24 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:25.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:24 smithi165 ceph-mon[36302]: from='client.? 172.21.15.119:0/925416157' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T00:23:25.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:24 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:25.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:24 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-28T00:23:25.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:24 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-28T00:23:25.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:24 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:23:25.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:24 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:25.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:24 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-28T00:23:25.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:24 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-28T00:23:25.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:24 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:23:25.171 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:24 smithi132 ceph-mon[35941]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T00:23:25.171 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:24 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:25.171 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:24 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:25.171 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:24 smithi132 ceph-mon[35941]: from='client.? 172.21.15.119:0/925416157' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T00:23:25.171 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:24 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:25.171 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:24 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-28T00:23:25.171 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:24 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-28T00:23:25.171 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:24 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:23:26.092 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:25 smithi119 ceph-mon[31518]: Deploying daemon mds.e on smithi132 2024-03-28T00:23:26.093 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:25 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.119:6836/3988635716,v1:172.21.15.119:6837/3988635716] up:boot 2024-03-28T00:23:26.093 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:25 smithi119 ceph-mon[31518]: fsmap 4 up:standby 2024-03-28T00:23:26.093 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:25 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-28T00:23:26.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:25 smithi165 ceph-mon[36302]: Deploying daemon mds.e on smithi132 2024-03-28T00:23:26.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:25 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.119:6836/3988635716,v1:172.21.15.119:6837/3988635716] up:boot 2024-03-28T00:23:26.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:25 smithi165 ceph-mon[36302]: fsmap 4 up:standby 2024-03-28T00:23:26.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:25 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-28T00:23:26.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:25 smithi132 ceph-mon[35941]: Deploying daemon mds.e on smithi132 2024-03-28T00:23:26.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:25 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.119:6836/3988635716,v1:172.21.15.119:6837/3988635716] up:boot 2024-03-28T00:23:26.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:25 smithi132 ceph-mon[35941]: fsmap 4 up:standby 2024-03-28T00:23:26.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:25 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-28T00:23:26.533 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T00:23:26.755 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:26.747+0000 7f8e2f200640 1 -- 172.21.15.119:0/387270443 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8e301014e0 msgr2=0x7f8e30101900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:26.755 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:26.747+0000 7f8e2f200640 1 --2- 172.21.15.119:0/387270443 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8e301014e0 0x7f8e30101900 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8e20009900 tx=0x7f8e2002bc30 comp rx=0 tx=0).stop 2024-03-28T00:23:26.755 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:26.747+0000 7f8e2f200640 1 -- 172.21.15.119:0/387270443 shutdown_connections 2024-03-28T00:23:26.755 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:26.747+0000 7f8e2f200640 1 --2- 172.21.15.119:0/387270443 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8e30103090 0x7f8e30109920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:26.755 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:26.747+0000 7f8e2f200640 1 --2- 172.21.15.119:0/387270443 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8e301026d0 0x7f8e30102b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:26.755 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:26.747+0000 7f8e2f200640 1 --2- 172.21.15.119:0/387270443 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8e301014e0 0x7f8e30101900 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:26.755 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:26.747+0000 7f8e2f200640 1 -- 172.21.15.119:0/387270443 >> 172.21.15.119:0/387270443 conn(0x7f8e300fca30 msgr2=0x7f8e300fee90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:23:26.755 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:26.748+0000 7f8e2f200640 1 -- 172.21.15.119:0/387270443 shutdown_connections 2024-03-28T00:23:26.755 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:26.748+0000 7f8e2f200640 1 -- 172.21.15.119:0/387270443 wait complete. 2024-03-28T00:23:26.755 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:26.749+0000 7f8e2f200640 1 Processor -- start 2024-03-28T00:23:26.755 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:26.749+0000 7f8e2f200640 1 -- start start 2024-03-28T00:23:26.757 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:26.749+0000 7f8e2f200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8e301014e0 0x7f8e30199ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:23:26.757 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:26.749+0000 7f8e2f200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8e301026d0 0x7f8e3019a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:23:26.757 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:26.749+0000 7f8e2f200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8e30103090 0x7f8e301a1020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:23:26.757 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:26.749+0000 7f8e2f200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e3010d1c0 con 0x7f8e301026d0 2024-03-28T00:23:26.758 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:26.749+0000 7f8e2f200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e3010d040 con 0x7f8e30103090 2024-03-28T00:23:26.758 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:26.749+0000 7f8e2f200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e3010d340 con 0x7f8e301014e0 2024-03-28T00:23:26.758 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:26.749+0000 7f8e2d400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8e301026d0 0x7f8e3019a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:23:26.758 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:26.750+0000 7f8e2e800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8e30103090 0x7f8e301a1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:23:26.758 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:26.750+0000 7f8e2d400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8e301026d0 0x7f8e3019a000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41738/0 (socket says 172.21.15.119:41738) 2024-03-28T00:23:26.758 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:26.750+0000 7f8e2de00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8e301014e0 0x7f8e30199ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:23:26.758 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:26.750+0000 7f8e2d400640 1 -- 172.21.15.119:0/2799406059 learned_addr learned my addr 172.21.15.119:0/2799406059 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:23:26.758 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:26.750+0000 7f8e2d400640 1 -- 172.21.15.119:0/2799406059 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8e301014e0 msgr2=0x7f8e30199ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:26.758 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:26.750+0000 7f8e2d400640 1 --2- 172.21.15.119:0/2799406059 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8e301014e0 0x7f8e30199ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:26.758 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:26.750+0000 7f8e2d400640 1 -- 172.21.15.119:0/2799406059 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8e30103090 msgr2=0x7f8e301a1020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:26.758 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:26.750+0000 7f8e2d400640 1 --2- 172.21.15.119:0/2799406059 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8e30103090 0x7f8e301a1020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:26.758 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:26.750+0000 7f8e2d400640 1 -- 172.21.15.119:0/2799406059 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e301a1690 con 0x7f8e301026d0 2024-03-28T00:23:26.758 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:26.750+0000 7f8e2e800640 1 --2- 172.21.15.119:0/2799406059 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8e30103090 0x7f8e301a1020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T00:23:26.759 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:26.750+0000 7f8e2d400640 1 --2- 172.21.15.119:0/2799406059 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8e301026d0 0x7f8e3019a000 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f8e1800e9f0 tx=0x7f8e1800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:23:26.759 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:26.750+0000 7f8e26a00640 1 -- 172.21.15.119:0/2799406059 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e1800cd80 con 0x7f8e301026d0 2024-03-28T00:23:26.759 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:26.751+0000 7f8e2f200640 1 -- 172.21.15.119:0/2799406059 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e301a1980 con 0x7f8e301026d0 2024-03-28T00:23:26.759 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:26.751+0000 7f8e26a00640 1 -- 172.21.15.119:0/2799406059 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8e18005500 con 0x7f8e301026d0 2024-03-28T00:23:26.759 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:26.751+0000 7f8e26a00640 1 -- 172.21.15.119:0/2799406059 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e18010690 con 0x7f8e301026d0 2024-03-28T00:23:26.759 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:26.751+0000 7f8e2f200640 1 -- 172.21.15.119:0/2799406059 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e301a1ed0 con 0x7f8e301026d0 2024-03-28T00:23:26.759 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:26.752+0000 7f8e2f200640 1 -- 172.21.15.119:0/2799406059 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e301a2160 con 0x7f8e301026d0 2024-03-28T00:23:26.760 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:26.754+0000 7f8e26a00640 1 -- 172.21.15.119:0/2799406059 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f8e180046c0 con 0x7f8e301026d0 2024-03-28T00:23:26.762 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:26.756+0000 7f8e26a00640 1 --2- 172.21.15.119:0/2799406059 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f8e140a2880 0x7f8e140a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:23:26.762 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:26.756+0000 7f8e26a00640 1 -- 172.21.15.119:0/2799406059 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f8e180ce9c0 con 0x7f8e301026d0 2024-03-28T00:23:26.762 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:26.756+0000 7f8e2de00640 1 --2- 172.21.15.119:0/2799406059 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f8e140a2880 0x7f8e140a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:23:26.763 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:26.757+0000 7f8e2de00640 1 --2- 172.21.15.119:0/2799406059 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f8e140a2880 0x7f8e140a4d40 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f8e2002c170 tx=0x7f8e2002df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:23:26.768 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:26.762+0000 7f8e26a00640 1 -- 172.21.15.119:0/2799406059 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f8e1809be60 con 0x7f8e301026d0 2024-03-28T00:23:26.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:26 smithi119 ceph-mon[31518]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T00:23:26.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:26 smithi132 ceph-mon[35941]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T00:23:27.001 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:26.995+0000 7f8e2f200640 1 -- 172.21.15.119:0/2799406059 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8e301a2330 con 0x7f8e301026d0 2024-03-28T00:23:27.002 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:26.996+0000 7f8e26a00640 1 -- 172.21.15.119:0/2799406059 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+3704 (secure 0 0 0) 0x7f8e1809b760 con 0x7f8e301026d0 2024-03-28T00:23:27.002 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-28T00:23:27.003 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14400,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/3988635716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":3988635716},{"type":"v1","addr":"172.21.15.119:6837","nonce":3988635716}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/609963168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":609963168},{"type":"v1","addr":"172.21.15.165:6833","nonce":609963168}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/1179016670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":1179016670},{"type":"v1","addr":"172.21.15.132:6833","nonce":1179016670}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24473,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/744950487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":744950487},{"type":"v1","addr":"172.21.15.119:6835","nonce":744950487}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4}],"filesystems":[]} 2024-03-28T00:23:27.003 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 5 2024-03-28T00:23:27.005 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:26.998+0000 7f8e2f200640 1 -- 172.21.15.119:0/2799406059 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f8e140a2880 msgr2=0x7f8e140a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:27.005 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:26.998+0000 7f8e2f200640 1 --2- 172.21.15.119:0/2799406059 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f8e140a2880 0x7f8e140a4d40 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f8e2002c170 tx=0x7f8e2002df90 comp rx=0 tx=0).stop 2024-03-28T00:23:27.005 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:26.999+0000 7f8e2f200640 1 -- 172.21.15.119:0/2799406059 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8e301026d0 msgr2=0x7f8e3019a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:27.005 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:26.999+0000 7f8e2f200640 1 --2- 172.21.15.119:0/2799406059 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8e301026d0 0x7f8e3019a000 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f8e1800e9f0 tx=0x7f8e1800edf0 comp rx=0 tx=0).stop 2024-03-28T00:23:27.005 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:26.999+0000 7f8e2f200640 1 -- 172.21.15.119:0/2799406059 shutdown_connections 2024-03-28T00:23:27.005 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:26.999+0000 7f8e2f200640 1 --2- 172.21.15.119:0/2799406059 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f8e140a2880 0x7f8e140a4d40 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:27.005 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:26.999+0000 7f8e2f200640 1 --2- 172.21.15.119:0/2799406059 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8e30103090 0x7f8e301a1020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:27.005 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:26.999+0000 7f8e2f200640 1 --2- 172.21.15.119:0/2799406059 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8e301026d0 0x7f8e3019a000 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:27.006 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:26.999+0000 7f8e2f200640 1 --2- 172.21.15.119:0/2799406059 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8e301014e0 0x7f8e30199ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:27.006 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:26.999+0000 7f8e2f200640 1 -- 172.21.15.119:0/2799406059 >> 172.21.15.119:0/2799406059 conn(0x7f8e300fca30 msgr2=0x7f8e300738d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:23:27.006 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:26.999+0000 7f8e2f200640 1 -- 172.21.15.119:0/2799406059 shutdown_connections 2024-03-28T00:23:27.006 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:26.999+0000 7f8e2f200640 1 -- 172.21.15.119:0/2799406059 wait complete. 2024-03-28T00:23:27.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:26 smithi165 ceph-mon[36302]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T00:23:27.702 INFO:journalctl@ceph.mds.e.smithi132.stdout:Mar 28 00:23:27 smithi132 systemd[1]: Started Ceph mds.e for 10f73b92-ec98-11ee-95d0-87774f69a715. 2024-03-28T00:23:28.121 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:27 smithi132 ceph-mon[35941]: from='client.? 172.21.15.119:0/2799406059' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T00:23:28.122 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:27 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:28.122 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:27 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:28.122 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:27 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:28.122 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:27 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-28T00:23:28.122 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:27 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-28T00:23:28.122 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:27 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:23:28.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:27 smithi165 ceph-mon[36302]: from='client.? 172.21.15.119:0/2799406059' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T00:23:28.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:27 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:28.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:27 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:28.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:27 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:28.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:27 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-28T00:23:28.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:27 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-28T00:23:28.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:27 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:23:28.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:27 smithi119 ceph-mon[31518]: from='client.? 172.21.15.119:0/2799406059' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T00:23:28.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:27 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:28.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:27 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:28.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:27 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:28.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:27 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-28T00:23:28.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:27 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-28T00:23:28.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:27 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:23:29.024 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T00:23:29.110 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:28 smithi119 ceph-mon[31518]: Deploying daemon mds.f on smithi165 2024-03-28T00:23:29.110 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:28 smithi119 ceph-mon[31518]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T00:23:29.110 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:28 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.132:6834/356981946,v1:172.21.15.132:6835/356981946] up:boot 2024-03-28T00:23:29.110 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:28 smithi119 ceph-mon[31518]: fsmap 5 up:standby 2024-03-28T00:23:29.110 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:28 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-28T00:23:29.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:28 smithi165 ceph-mon[36302]: Deploying daemon mds.f on smithi165 2024-03-28T00:23:29.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:28 smithi165 ceph-mon[36302]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T00:23:29.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:28 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.132:6834/356981946,v1:172.21.15.132:6835/356981946] up:boot 2024-03-28T00:23:29.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:28 smithi165 ceph-mon[36302]: fsmap 5 up:standby 2024-03-28T00:23:29.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:28 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-28T00:23:29.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:28 smithi132 ceph-mon[35941]: Deploying daemon mds.f on smithi165 2024-03-28T00:23:29.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:28 smithi132 ceph-mon[35941]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T00:23:29.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:28 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.132:6834/356981946,v1:172.21.15.132:6835/356981946] up:boot 2024-03-28T00:23:29.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:28 smithi132 ceph-mon[35941]: fsmap 5 up:standby 2024-03-28T00:23:29.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:28 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-28T00:23:29.245 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:29.238+0000 7fc146800640 1 -- 172.21.15.119:0/1908540934 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc140104ad0 msgr2=0x7fc140104f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:29.245 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:29.238+0000 7fc146800640 1 --2- 172.21.15.119:0/1908540934 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc140104ad0 0x7fc140104f50 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fc128009900 tx=0x7fc12802bc30 comp rx=0 tx=0).stop 2024-03-28T00:23:29.245 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:29.238+0000 7fc146800640 1 -- 172.21.15.119:0/1908540934 shutdown_connections 2024-03-28T00:23:29.245 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:29.238+0000 7fc146800640 1 --2- 172.21.15.119:0/1908540934 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc140105490 0x7fc14010bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:29.245 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:29.238+0000 7fc146800640 1 --2- 172.21.15.119:0/1908540934 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc140104ad0 0x7fc140104f50 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:29.245 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:29.238+0000 7fc146800640 1 --2- 172.21.15.119:0/1908540934 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc1401038e0 0x7fc140103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:29.245 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:29.238+0000 7fc146800640 1 -- 172.21.15.119:0/1908540934 >> 172.21.15.119:0/1908540934 conn(0x7fc1400fee50 msgr2=0x7fc140101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:23:29.246 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:29.238+0000 7fc146800640 1 -- 172.21.15.119:0/1908540934 shutdown_connections 2024-03-28T00:23:29.246 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:29.238+0000 7fc146800640 1 -- 172.21.15.119:0/1908540934 wait complete. 2024-03-28T00:23:29.248 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:29.239+0000 7fc146800640 1 Processor -- start 2024-03-28T00:23:29.248 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:29.239+0000 7fc146800640 1 -- start start 2024-03-28T00:23:29.248 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:29.240+0000 7fc146800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc1401038e0 0x7fc140199c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:23:29.248 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:29.240+0000 7fc146800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc140104ad0 0x7fc14019a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:23:29.248 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:29.240+0000 7fc146800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc140105490 0x7fc1401a11c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:23:29.248 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:29.240+0000 7fc146800640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc14010f590 con 0x7fc1401038e0 2024-03-28T00:23:29.248 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:29.240+0000 7fc146800640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc14010f410 con 0x7fc140105490 2024-03-28T00:23:29.248 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:29.240+0000 7fc146800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc14010f710 con 0x7fc140104ad0 2024-03-28T00:23:29.249 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:29.240+0000 7fc13f400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc1401038e0 0x7fc140199c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:23:29.249 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:29.240+0000 7fc13f400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc1401038e0 0x7fc140199c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41746/0 (socket says 172.21.15.119:41746) 2024-03-28T00:23:29.249 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:29.240+0000 7fc13f400640 1 -- 172.21.15.119:0/1940708088 learned_addr learned my addr 172.21.15.119:0/1940708088 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:23:29.249 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:29.240+0000 7fc13ea00640 1 --2- 172.21.15.119:0/1940708088 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc140104ad0 0x7fc14019a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:23:29.249 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:29.240+0000 7fc13fe00640 1 --2- 172.21.15.119:0/1940708088 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc140105490 0x7fc1401a11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:23:29.249 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:29.240+0000 7fc13f400640 1 -- 172.21.15.119:0/1940708088 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc140104ad0 msgr2=0x7fc14019a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:29.249 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:29.240+0000 7fc13f400640 1 --2- 172.21.15.119:0/1940708088 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc140104ad0 0x7fc14019a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:29.249 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:29.240+0000 7fc13f400640 1 -- 172.21.15.119:0/1940708088 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc140105490 msgr2=0x7fc1401a11c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:29.249 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:29.240+0000 7fc13f400640 1 --2- 172.21.15.119:0/1940708088 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc140105490 0x7fc1401a11c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:29.250 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:29.240+0000 7fc13f400640 1 -- 172.21.15.119:0/1940708088 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1401a1860 con 0x7fc1401038e0 2024-03-28T00:23:29.250 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:29.241+0000 7fc13f400640 1 --2- 172.21.15.119:0/1940708088 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc1401038e0 0x7fc140199c50 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fc13000e9c0 tx=0x7fc13000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:23:29.250 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:29.241+0000 7fc137e00640 1 -- 172.21.15.119:0/1940708088 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc13000cd80 con 0x7fc1401038e0 2024-03-28T00:23:29.250 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:29.241+0000 7fc137e00640 1 -- 172.21.15.119:0/1940708088 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc1300035f0 con 0x7fc1401038e0 2024-03-28T00:23:29.250 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:29.241+0000 7fc146800640 1 -- 172.21.15.119:0/1940708088 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1401a1b50 con 0x7fc1401038e0 2024-03-28T00:23:29.250 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:29.241+0000 7fc137e00640 1 -- 172.21.15.119:0/1940708088 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc130010690 con 0x7fc1401038e0 2024-03-28T00:23:29.250 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:29.241+0000 7fc146800640 1 -- 172.21.15.119:0/1940708088 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1401a20a0 con 0x7fc1401038e0 2024-03-28T00:23:29.250 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:29.242+0000 7fc146800640 1 -- 172.21.15.119:0/1940708088 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc14010ac80 con 0x7fc1401038e0 2024-03-28T00:23:29.251 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:29.244+0000 7fc137e00640 1 -- 172.21.15.119:0/1940708088 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fc1300037a0 con 0x7fc1401038e0 2024-03-28T00:23:29.251 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:29.245+0000 7fc137e00640 1 --2- 172.21.15.119:0/1940708088 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fc1240a2800 0x7fc1240a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:23:29.252 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:29.246+0000 7fc137e00640 1 -- 172.21.15.119:0/1940708088 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fc1300cea80 con 0x7fc1401038e0 2024-03-28T00:23:29.252 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:29.246+0000 7fc13ea00640 1 --2- 172.21.15.119:0/1940708088 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fc1240a2800 0x7fc1240a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:23:29.253 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:29.247+0000 7fc13ea00640 1 --2- 172.21.15.119:0/1940708088 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fc1240a2800 0x7fc1240a4cc0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fc12802c240 tx=0x7fc12802df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:23:29.256 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:29.250+0000 7fc137e00640 1 -- 172.21.15.119:0/1940708088 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fc1300d0e20 con 0x7fc1401038e0 2024-03-28T00:23:29.496 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:29.490+0000 7fc146800640 1 -- 172.21.15.119:0/1940708088 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc14002d070 con 0x7fc1401038e0 2024-03-28T00:23:29.497 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:29.491+0000 7fc137e00640 1 -- 172.21.15.119:0/1940708088 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4495 (secure 0 0 0) 0x7fc13009bf20 con 0x7fc1401038e0 2024-03-28T00:23:29.498 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-28T00:23:29.498 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14400,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/3988635716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":3988635716},{"type":"v1","addr":"172.21.15.119:6837","nonce":3988635716}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/609963168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":609963168},{"type":"v1","addr":"172.21.15.165:6833","nonce":609963168}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/1179016670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":1179016670},{"type":"v1","addr":"172.21.15.132:6833","nonce":1179016670}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24473,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/744950487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":744950487},{"type":"v1","addr":"172.21.15.119:6835","nonce":744950487}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24500,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/356981946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":356981946},{"type":"v1","addr":"172.21.15.132:6835","nonce":356981946}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[]} 2024-03-28T00:23:29.498 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 6 2024-03-28T00:23:29.500 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:29.494+0000 7fc146800640 1 -- 172.21.15.119:0/1940708088 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fc1240a2800 msgr2=0x7fc1240a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:29.501 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:29.494+0000 7fc146800640 1 --2- 172.21.15.119:0/1940708088 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fc1240a2800 0x7fc1240a4cc0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fc12802c240 tx=0x7fc12802df90 comp rx=0 tx=0).stop 2024-03-28T00:23:29.501 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:29.494+0000 7fc146800640 1 -- 172.21.15.119:0/1940708088 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc1401038e0 msgr2=0x7fc140199c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:29.501 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:29.494+0000 7fc146800640 1 --2- 172.21.15.119:0/1940708088 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc1401038e0 0x7fc140199c50 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fc13000e9c0 tx=0x7fc13000ee80 comp rx=0 tx=0).stop 2024-03-28T00:23:29.501 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:29.494+0000 7fc146800640 1 -- 172.21.15.119:0/1940708088 shutdown_connections 2024-03-28T00:23:29.501 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:29.494+0000 7fc146800640 1 --2- 172.21.15.119:0/1940708088 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fc1240a2800 0x7fc1240a4cc0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:29.501 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:29.494+0000 7fc146800640 1 --2- 172.21.15.119:0/1940708088 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc140105490 0x7fc1401a11c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:29.501 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:29.494+0000 7fc146800640 1 --2- 172.21.15.119:0/1940708088 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc140104ad0 0x7fc14019a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:29.501 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:29.494+0000 7fc146800640 1 --2- 172.21.15.119:0/1940708088 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc1401038e0 0x7fc140199c50 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:29.501 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:29.494+0000 7fc146800640 1 -- 172.21.15.119:0/1940708088 >> 172.21.15.119:0/1940708088 conn(0x7fc1400fee50 msgr2=0x7fc140107e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:23:29.501 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:29.495+0000 7fc146800640 1 -- 172.21.15.119:0/1940708088 shutdown_connections 2024-03-28T00:23:29.501 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:29.495+0000 7fc146800640 1 -- 172.21.15.119:0/1940708088 wait complete. 2024-03-28T00:23:30.088 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:29 smithi165 ceph-mon[36302]: from='client.? 172.21.15.119:0/1940708088' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T00:23:30.088 INFO:journalctl@ceph.mds.f.smithi165.stdout:Mar 28 00:23:29 smithi165 systemd[1]: Starting Ceph mds.f for 10f73b92-ec98-11ee-95d0-87774f69a715... 2024-03-28T00:23:30.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:29 smithi119 ceph-mon[31518]: from='client.? 172.21.15.119:0/1940708088' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T00:23:30.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:29 smithi132 ceph-mon[35941]: from='client.? 172.21.15.119:0/1940708088' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T00:23:30.355 INFO:journalctl@ceph.mds.f.smithi165.stdout:Mar 28 00:23:30 smithi165 podman[67441]: 2024-03-28 00:23:29.985277043 +0000 UTC m=+0.017750392 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-28T00:23:30.355 INFO:journalctl@ceph.mds.f.smithi165.stdout:Mar 28 00:23:30 smithi165 podman[67441]: 2024-03-28 00:23:30.087956683 +0000 UTC m=+0.120430077 container create 935245452bf88b28981fcd8b5cd9517d05d9147ac88ba8b31b6af1887d61e3f4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-f, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True) 2024-03-28T00:23:30.355 INFO:journalctl@ceph.mds.f.smithi165.stdout:Mar 28 00:23:30 smithi165 podman[67441]: 2024-03-28 00:23:30.349209515 +0000 UTC m=+0.381682863 container init 935245452bf88b28981fcd8b5cd9517d05d9147ac88ba8b31b6af1887d61e3f4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-f, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_BRANCH=HEAD) 2024-03-28T00:23:30.633 INFO:journalctl@ceph.mds.f.smithi165.stdout:Mar 28 00:23:30 smithi165 podman[67441]: 2024-03-28 00:23:30.352304057 +0000 UTC m=+0.384777410 container start 935245452bf88b28981fcd8b5cd9517d05d9147ac88ba8b31b6af1887d61e3f4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-f, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-03-28T00:23:30.633 INFO:journalctl@ceph.mds.f.smithi165.stdout:Mar 28 00:23:30 smithi165 ceph-mds[67460]: -- 172.21.15.165:0/3640636440 <== mon.1 v2:172.21.15.165:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56283dc17a40 con 0x56283dbea800 2024-03-28T00:23:30.633 INFO:journalctl@ceph.mds.f.smithi165.stdout:Mar 28 00:23:30 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-f[67456]: starting mds.f at 2024-03-28T00:23:30.633 INFO:journalctl@ceph.mds.f.smithi165.stdout:Mar 28 00:23:30 smithi165 bash[67441]: 935245452bf88b28981fcd8b5cd9517d05d9147ac88ba8b31b6af1887d61e3f4 2024-03-28T00:23:30.633 INFO:journalctl@ceph.mds.f.smithi165.stdout:Mar 28 00:23:30 smithi165 systemd[1]: Started Ceph mds.f for 10f73b92-ec98-11ee-95d0-87774f69a715. 2024-03-28T00:23:31.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:30 smithi165 ceph-mon[36302]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T00:23:31.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:30 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:31.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:30 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:31.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:30 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:31.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:30 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-28T00:23:31.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:30 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-28T00:23:31.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:30 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:23:31.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:30 smithi119 ceph-mon[31518]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T00:23:31.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:30 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:31.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:30 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:31.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:30 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:31.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:30 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-28T00:23:31.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:30 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-28T00:23:31.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:30 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:23:31.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:30 smithi132 ceph-mon[35941]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T00:23:31.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:30 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:31.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:30 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:31.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:30 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:31.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:30 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-28T00:23:31.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:30 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-28T00:23:31.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:30 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:23:31.518 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T00:23:31.736 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:31.729+0000 7fec77800640 1 -- 172.21.15.119:0/893387128 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fec70104ad0 msgr2=0x7fec70104f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:31.736 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:31.729+0000 7fec77800640 1 --2- 172.21.15.119:0/893387128 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fec70104ad0 0x7fec70104f50 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fec58009900 tx=0x7fec5802bc30 comp rx=0 tx=0).stop 2024-03-28T00:23:31.736 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:31.730+0000 7fec77800640 1 -- 172.21.15.119:0/893387128 shutdown_connections 2024-03-28T00:23:31.736 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:31.730+0000 7fec77800640 1 --2- 172.21.15.119:0/893387128 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fec70105490 0x7fec7010bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:31.736 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:31.730+0000 7fec77800640 1 --2- 172.21.15.119:0/893387128 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fec70104ad0 0x7fec70104f50 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:31.736 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:31.730+0000 7fec77800640 1 --2- 172.21.15.119:0/893387128 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fec701038e0 0x7fec70103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:31.736 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:31.730+0000 7fec77800640 1 -- 172.21.15.119:0/893387128 >> 172.21.15.119:0/893387128 conn(0x7fec700fee50 msgr2=0x7fec70101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:23:31.736 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:31.730+0000 7fec77800640 1 -- 172.21.15.119:0/893387128 shutdown_connections 2024-03-28T00:23:31.736 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:31.730+0000 7fec77800640 1 -- 172.21.15.119:0/893387128 wait complete. 2024-03-28T00:23:31.739 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:31.731+0000 7fec77800640 1 Processor -- start 2024-03-28T00:23:31.739 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:31.731+0000 7fec77800640 1 -- start start 2024-03-28T00:23:31.739 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:31.731+0000 7fec77800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fec701038e0 0x7fec70199c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:23:31.739 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:31.731+0000 7fec77800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fec70104ad0 0x7fec7019a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:23:31.739 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:31.731+0000 7fec77800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fec70105490 0x7fec701a1190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:23:31.739 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:31.731+0000 7fec77800640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec7010f5b0 con 0x7fec70104ad0 2024-03-28T00:23:31.739 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:31.731+0000 7fec77800640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec7010f430 con 0x7fec70105490 2024-03-28T00:23:31.739 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:31.731+0000 7fec77800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec7010f730 con 0x7fec701038e0 2024-03-28T00:23:31.739 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:31.732+0000 7fec67e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fec70104ad0 0x7fec7019a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:23:31.739 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:31.732+0000 7fec74e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fec701038e0 0x7fec70199c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:23:31.739 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:31.732+0000 7fec74e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fec701038e0 0x7fec70199c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.119:43924/0 (socket says 172.21.15.119:43924) 2024-03-28T00:23:31.739 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:31.732+0000 7fec74e00640 1 -- 172.21.15.119:0/2547183507 learned_addr learned my addr 172.21.15.119:0/2547183507 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:23:31.739 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:31.732+0000 7fec75800640 1 --2- 172.21.15.119:0/2547183507 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fec70105490 0x7fec701a1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:23:31.739 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:31.732+0000 7fec75800640 1 -- 172.21.15.119:0/2547183507 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fec701038e0 msgr2=0x7fec70199c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:31.740 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:31.732+0000 7fec75800640 1 --2- 172.21.15.119:0/2547183507 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fec701038e0 0x7fec70199c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:31.740 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:31.732+0000 7fec75800640 1 -- 172.21.15.119:0/2547183507 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fec70104ad0 msgr2=0x7fec7019a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:31.740 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:31.732+0000 7fec75800640 1 --2- 172.21.15.119:0/2547183507 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fec70104ad0 0x7fec7019a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:31.740 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:31.732+0000 7fec75800640 1 -- 172.21.15.119:0/2547183507 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec701a1830 con 0x7fec70105490 2024-03-28T00:23:31.740 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:31.732+0000 7fec67e00640 1 --2- 172.21.15.119:0/2547183507 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fec70104ad0 0x7fec7019a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T00:23:31.740 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:31.732+0000 7fec75800640 1 --2- 172.21.15.119:0/2547183507 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fec70105490 0x7fec701a1190 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fec68009fd0 tx=0x7fec6800ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:23:31.740 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:31.733+0000 7fec6e000640 1 -- 172.21.15.119:0/2547183507 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec6800ce20 con 0x7fec70105490 2024-03-28T00:23:31.740 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:31.733+0000 7fec6e000640 1 -- 172.21.15.119:0/2547183507 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fec68005020 con 0x7fec70105490 2024-03-28T00:23:31.740 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:31.733+0000 7fec6e000640 1 -- 172.21.15.119:0/2547183507 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec68010720 con 0x7fec70105490 2024-03-28T00:23:31.740 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:31.733+0000 7fec77800640 1 -- 172.21.15.119:0/2547183507 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec701a1b20 con 0x7fec70105490 2024-03-28T00:23:31.740 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:31.733+0000 7fec77800640 1 -- 172.21.15.119:0/2547183507 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec701a2020 con 0x7fec70105490 2024-03-28T00:23:31.741 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:31.734+0000 7fec77800640 1 -- 172.21.15.119:0/2547183507 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec7010aba0 con 0x7fec70105490 2024-03-28T00:23:31.742 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:31.736+0000 7fec6e000640 1 -- 172.21.15.119:0/2547183507 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fec68005540 con 0x7fec70105490 2024-03-28T00:23:31.743 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:31.737+0000 7fec6e000640 1 --2- 172.21.15.119:0/2547183507 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fec540a2880 0x7fec540a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:23:31.744 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:31.737+0000 7fec6e000640 1 -- 172.21.15.119:0/2547183507 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fec680ced30 con 0x7fec70105490 2024-03-28T00:23:31.744 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:31.737+0000 7fec74e00640 1 --2- 172.21.15.119:0/2547183507 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fec540a2880 0x7fec540a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:23:31.744 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:31.738+0000 7fec74e00640 1 --2- 172.21.15.119:0/2547183507 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fec540a2880 0x7fec540a4d40 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fec7019b0f0 tx=0x7fec60005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:23:31.748 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:31.742+0000 7fec6e000640 1 -- 172.21.15.119:0/2547183507 <== mon.2 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fec6809c1d0 con 0x7fec70105490 2024-03-28T00:23:31.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:31 smithi119 ceph-mon[31518]: Deploying daemon mds.c on smithi165 2024-03-28T00:23:31.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:31 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.165:6834/1947980102,v1:172.21.15.165:6835/1947980102] up:boot 2024-03-28T00:23:31.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:31 smithi119 ceph-mon[31518]: fsmap 6 up:standby 2024-03-28T00:23:31.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:31 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-28T00:23:31.991 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:31.985+0000 7fec77800640 1 -- 172.21.15.119:0/2547183507 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fec7002d070 con 0x7fec70105490 2024-03-28T00:23:31.992 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:31.986+0000 7fec6e000640 1 -- 172.21.15.119:0/2547183507 <== mon.2 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+5289 (secure 0 0 0) 0x7fec6809bad0 con 0x7fec70105490 2024-03-28T00:23:31.992 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-28T00:23:31.992 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14400,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/3988635716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":3988635716},{"type":"v1","addr":"172.21.15.119:6837","nonce":3988635716}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/609963168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":609963168},{"type":"v1","addr":"172.21.15.165:6833","nonce":609963168}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/1179016670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":1179016670},{"type":"v1","addr":"172.21.15.132:6833","nonce":1179016670}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24473,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/744950487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":744950487},{"type":"v1","addr":"172.21.15.119:6835","nonce":744950487}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24500,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/356981946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":356981946},{"type":"v1","addr":"172.21.15.132:6835","nonce":356981946}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24511,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6835/1947980102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":1947980102},{"type":"v1","addr":"172.21.15.165:6835","nonce":1947980102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7}],"filesystems":[]} 2024-03-28T00:23:31.992 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 7 2024-03-28T00:23:31.995 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:31.988+0000 7fec77800640 1 -- 172.21.15.119:0/2547183507 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fec540a2880 msgr2=0x7fec540a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:31.995 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:31.988+0000 7fec77800640 1 --2- 172.21.15.119:0/2547183507 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fec540a2880 0x7fec540a4d40 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fec7019b0f0 tx=0x7fec60005f90 comp rx=0 tx=0).stop 2024-03-28T00:23:31.995 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:31.988+0000 7fec77800640 1 -- 172.21.15.119:0/2547183507 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fec70105490 msgr2=0x7fec701a1190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:31.995 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:31.988+0000 7fec77800640 1 --2- 172.21.15.119:0/2547183507 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fec70105490 0x7fec701a1190 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fec68009fd0 tx=0x7fec6800ef10 comp rx=0 tx=0).stop 2024-03-28T00:23:31.995 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:31.989+0000 7fec77800640 1 -- 172.21.15.119:0/2547183507 shutdown_connections 2024-03-28T00:23:31.995 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:31.989+0000 7fec77800640 1 --2- 172.21.15.119:0/2547183507 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fec540a2880 0x7fec540a4d40 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:31.995 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:31.989+0000 7fec77800640 1 --2- 172.21.15.119:0/2547183507 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fec70105490 0x7fec701a1190 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:31.995 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:31.989+0000 7fec77800640 1 --2- 172.21.15.119:0/2547183507 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fec70104ad0 0x7fec7019a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:31.996 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:31.989+0000 7fec77800640 1 --2- 172.21.15.119:0/2547183507 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fec701038e0 0x7fec70199c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:31.996 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:31.989+0000 7fec77800640 1 -- 172.21.15.119:0/2547183507 >> 172.21.15.119:0/2547183507 conn(0x7fec700fee50 msgr2=0x7fec70107d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:23:31.996 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:31.989+0000 7fec77800640 1 -- 172.21.15.119:0/2547183507 shutdown_connections 2024-03-28T00:23:31.996 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:31.989+0000 7fec77800640 1 -- 172.21.15.119:0/2547183507 wait complete. 2024-03-28T00:23:32.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:31 smithi165 ceph-mon[36302]: Deploying daemon mds.c on smithi165 2024-03-28T00:23:32.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:31 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.165:6834/1947980102,v1:172.21.15.165:6835/1947980102] up:boot 2024-03-28T00:23:32.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:31 smithi165 ceph-mon[36302]: fsmap 6 up:standby 2024-03-28T00:23:32.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:31 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-28T00:23:32.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:31 smithi132 ceph-mon[35941]: Deploying daemon mds.c on smithi165 2024-03-28T00:23:32.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:31 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.165:6834/1947980102,v1:172.21.15.165:6835/1947980102] up:boot 2024-03-28T00:23:32.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:31 smithi132 ceph-mon[35941]: fsmap 6 up:standby 2024-03-28T00:23:32.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:31 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-28T00:23:33.097 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:32 smithi165 ceph-mon[36302]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T00:23:33.097 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:32 smithi165 ceph-mon[36302]: from='client.? 172.21.15.119:0/2547183507' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T00:23:33.097 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:23:32 smithi165 systemd[1]: Starting Ceph mds.c for 10f73b92-ec98-11ee-95d0-87774f69a715... 2024-03-28T00:23:33.098 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:23:33 smithi165 podman[67749]: 2024-03-28 00:23:32.994421346 +0000 UTC m=+0.017941104 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-28T00:23:33.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:32 smithi119 ceph-mon[31518]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T00:23:33.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:32 smithi119 ceph-mon[31518]: from='client.? 172.21.15.119:0/2547183507' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T00:23:33.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:32 smithi132 ceph-mon[35941]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T00:23:33.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:32 smithi132 ceph-mon[35941]: from='client.? 172.21.15.119:0/2547183507' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T00:23:33.382 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:23:33 smithi165 podman[67749]: 2024-03-28 00:23:33.126016277 +0000 UTC m=+0.149536024 container create 1af8232810b86e2c9f926f5ce25fa81bd44baaeb9883d5ea6068a59d7732e4fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-03-28T00:23:33.882 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:23:33 smithi165 podman[67749]: 2024-03-28 00:23:33.403727855 +0000 UTC m=+0.427247602 container init 1af8232810b86e2c9f926f5ce25fa81bd44baaeb9883d5ea6068a59d7732e4fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c) 2024-03-28T00:23:33.883 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:23:33 smithi165 podman[67749]: 2024-03-28 00:23:33.406464774 +0000 UTC m=+0.429984523 container start 1af8232810b86e2c9f926f5ce25fa81bd44baaeb9883d5ea6068a59d7732e4fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, ceph=True, io.buildah.version=1.33.5) 2024-03-28T00:23:33.883 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:23:33 smithi165 ceph-mds[67768]: -- 172.21.15.165:0/3446292625 <== mon.1 v2:172.21.15.165:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x555d892e7a40 con 0x555d892bac00 2024-03-28T00:23:33.883 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:23:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: starting mds.c at 2024-03-28T00:23:33.883 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:23:33 smithi165 bash[67749]: 1af8232810b86e2c9f926f5ce25fa81bd44baaeb9883d5ea6068a59d7732e4fe 2024-03-28T00:23:33.883 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:23:33 smithi165 systemd[1]: Started Ceph mds.c for 10f73b92-ec98-11ee-95d0-87774f69a715. 2024-03-28T00:23:34.016 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T00:23:34.239 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:34.232+0000 7f490be00640 1 -- 172.21.15.119:0/1651045025 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4904077c40 msgr2=0x7f49040784e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:34.239 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:34.232+0000 7f490be00640 1 --2- 172.21.15.119:0/1651045025 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4904077c40 0x7f49040784e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f48fc009ca0 tx=0x7f48fc02c100 comp rx=0 tx=0).stop 2024-03-28T00:23:34.239 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:34.232+0000 7f490be00640 1 -- 172.21.15.119:0/1651045025 shutdown_connections 2024-03-28T00:23:34.239 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:34.232+0000 7f490be00640 1 --2- 172.21.15.119:0/1651045025 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4904077c40 0x7f49040784e0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:34.239 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:34.232+0000 7f490be00640 1 --2- 172.21.15.119:0/1651045025 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f49040772a0 0x7f4904077700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:34.239 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:34.232+0000 7f490be00640 1 --2- 172.21.15.119:0/1651045025 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4904076060 0x7f4904076480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:34.239 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:34.232+0000 7f490be00640 1 -- 172.21.15.119:0/1651045025 >> 172.21.15.119:0/1651045025 conn(0x7f49040fee50 msgr2=0x7f4904101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:23:34.240 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:34.233+0000 7f490be00640 1 -- 172.21.15.119:0/1651045025 shutdown_connections 2024-03-28T00:23:34.240 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:34.233+0000 7f490be00640 1 -- 172.21.15.119:0/1651045025 wait complete. 2024-03-28T00:23:34.242 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:34.233+0000 7f490be00640 1 Processor -- start 2024-03-28T00:23:34.242 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:34.234+0000 7f490be00640 1 -- start start 2024-03-28T00:23:34.242 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:34.234+0000 7f490be00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4904076060 0x7f490419df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:23:34.242 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:34.234+0000 7f490be00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f49040772a0 0x7f490419e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:23:34.242 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:34.234+0000 7f490be00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4904077c40 0x7f49041a5540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:23:34.242 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:34.234+0000 7f490be00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4904113970 con 0x7f4904077c40 2024-03-28T00:23:34.242 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:34.234+0000 7f490be00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49041137f0 con 0x7f49040772a0 2024-03-28T00:23:34.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:34.234+0000 7f490be00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4904113af0 con 0x7f4904076060 2024-03-28T00:23:34.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:34.234+0000 7f4909e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4904077c40 0x7f49041a5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:23:34.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:34.234+0000 7f4909e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4904077c40 0x7f49041a5540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:60740/0 (socket says 172.21.15.119:60740) 2024-03-28T00:23:34.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:34.234+0000 7f4909e00640 1 -- 172.21.15.119:0/1207790007 learned_addr learned my addr 172.21.15.119:0/1207790007 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:23:34.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:34.234+0000 7f4908a00640 1 --2- 172.21.15.119:0/1207790007 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f49040772a0 0x7f490419e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:23:34.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:34.234+0000 7f4909400640 1 --2- 172.21.15.119:0/1207790007 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4904076060 0x7f490419df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:23:34.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:34.235+0000 7f4909e00640 1 -- 172.21.15.119:0/1207790007 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4904076060 msgr2=0x7f490419df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:34.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:34.235+0000 7f4909e00640 1 --2- 172.21.15.119:0/1207790007 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4904076060 0x7f490419df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:34.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:34.235+0000 7f4909e00640 1 -- 172.21.15.119:0/1207790007 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f49040772a0 msgr2=0x7f490419e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:34.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:34.235+0000 7f4909e00640 1 --2- 172.21.15.119:0/1207790007 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f49040772a0 0x7f490419e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:34.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:34.235+0000 7f4909e00640 1 -- 172.21.15.119:0/1207790007 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49041a5bb0 con 0x7f4904077c40 2024-03-28T00:23:34.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:34.235+0000 7f4908a00640 1 --2- 172.21.15.119:0/1207790007 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f49040772a0 0x7f490419e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T00:23:34.244 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:34.235+0000 7f4909e00640 1 --2- 172.21.15.119:0/1207790007 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4904077c40 0x7f49041a5540 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f48fc02c640 tx=0x7f48fc035d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:23:34.244 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:34.235+0000 7f4902000640 1 -- 172.21.15.119:0/1207790007 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48fc039040 con 0x7f4904077c40 2024-03-28T00:23:34.244 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:34.235+0000 7f4902000640 1 -- 172.21.15.119:0/1207790007 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f48fc044070 con 0x7f4904077c40 2024-03-28T00:23:34.244 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:34.235+0000 7f490be00640 1 -- 172.21.15.119:0/1207790007 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49041a5e40 con 0x7f4904077c40 2024-03-28T00:23:34.244 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:34.235+0000 7f4902000640 1 -- 172.21.15.119:0/1207790007 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48fc0043c0 con 0x7f4904077c40 2024-03-28T00:23:34.244 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:34.235+0000 7f490be00640 1 -- 172.21.15.119:0/1207790007 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49041a62a0 con 0x7f4904077c40 2024-03-28T00:23:34.245 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:34.236+0000 7f490be00640 1 -- 172.21.15.119:0/1207790007 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49041103e0 con 0x7f4904077c40 2024-03-28T00:23:34.245 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:34.238+0000 7f4902000640 1 -- 172.21.15.119:0/1207790007 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f48fc004560 con 0x7f4904077c40 2024-03-28T00:23:34.246 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:34.240+0000 7f4902000640 1 --2- 172.21.15.119:0/1207790007 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f48ec0a2800 0x7f48ec0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:23:34.247 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:34.241+0000 7f4902000640 1 -- 172.21.15.119:0/1207790007 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f48fc0f0c10 con 0x7f4904077c40 2024-03-28T00:23:34.247 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:34.241+0000 7f4909400640 1 --2- 172.21.15.119:0/1207790007 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f48ec0a2800 0x7f48ec0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:23:34.247 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:34.241+0000 7f4909400640 1 --2- 172.21.15.119:0/1207790007 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f48ec0a2800 0x7f48ec0a4cc0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f48f8009bd0 tx=0x7f48f8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:23:34.252 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:34.245+0000 7f4902000640 1 -- 172.21.15.119:0/1207790007 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f48fc0be0b0 con 0x7f4904077c40 2024-03-28T00:23:34.490 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:34.484+0000 7f490be00640 1 -- 172.21.15.119:0/1207790007 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f490402d070 con 0x7f4904077c40 2024-03-28T00:23:34.491 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:34.485+0000 7f4902000640 1 -- 172.21.15.119:0/1207790007 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+6083 (secure 0 0 0) 0x7f48fc0098b0 con 0x7f4904077c40 2024-03-28T00:23:34.492 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-28T00:23:34.492 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14400,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/3988635716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":3988635716},{"type":"v1","addr":"172.21.15.119:6837","nonce":3988635716}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/609963168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":609963168},{"type":"v1","addr":"172.21.15.165:6833","nonce":609963168}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/1179016670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":1179016670},{"type":"v1","addr":"172.21.15.132:6833","nonce":1179016670}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24473,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/744950487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":744950487},{"type":"v1","addr":"172.21.15.119:6835","nonce":744950487}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24500,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/356981946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":356981946},{"type":"v1","addr":"172.21.15.132:6835","nonce":356981946}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24511,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6835/1947980102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":1947980102},{"type":"v1","addr":"172.21.15.165:6835","nonce":1947980102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24523,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/1531193032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":1531193032},{"type":"v1","addr":"172.21.15.165:6837","nonce":1531193032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8}],"filesystems":[]} 2024-03-28T00:23:34.492 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 8 2024-03-28T00:23:34.494 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:34.488+0000 7f490be00640 1 -- 172.21.15.119:0/1207790007 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f48ec0a2800 msgr2=0x7f48ec0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:34.494 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:34.488+0000 7f490be00640 1 --2- 172.21.15.119:0/1207790007 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f48ec0a2800 0x7f48ec0a4cc0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f48f8009bd0 tx=0x7f48f8005f90 comp rx=0 tx=0).stop 2024-03-28T00:23:34.494 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:34.488+0000 7f490be00640 1 -- 172.21.15.119:0/1207790007 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4904077c40 msgr2=0x7f49041a5540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:34.494 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:34.488+0000 7f490be00640 1 --2- 172.21.15.119:0/1207790007 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4904077c40 0x7f49041a5540 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f48fc02c640 tx=0x7f48fc035d10 comp rx=0 tx=0).stop 2024-03-28T00:23:34.494 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:34.488+0000 7f490be00640 1 -- 172.21.15.119:0/1207790007 shutdown_connections 2024-03-28T00:23:34.495 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:34.488+0000 7f490be00640 1 --2- 172.21.15.119:0/1207790007 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f48ec0a2800 0x7f48ec0a4cc0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:34.495 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:34.488+0000 7f490be00640 1 --2- 172.21.15.119:0/1207790007 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4904077c40 0x7f49041a5540 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:34.495 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:34.488+0000 7f490be00640 1 --2- 172.21.15.119:0/1207790007 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f49040772a0 0x7f490419e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:34.495 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:34.488+0000 7f490be00640 1 --2- 172.21.15.119:0/1207790007 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4904076060 0x7f490419df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:34.495 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:34.488+0000 7f490be00640 1 -- 172.21.15.119:0/1207790007 >> 172.21.15.119:0/1207790007 conn(0x7f49040fee50 msgr2=0x7f4904100550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:23:34.495 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:34.488+0000 7f490be00640 1 -- 172.21.15.119:0/1207790007 shutdown_connections 2024-03-28T00:23:34.495 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:34.488+0000 7f490be00640 1 -- 172.21.15.119:0/1207790007 wait complete. 2024-03-28T00:23:34.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:34 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:34.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:34 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:34.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:34 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:34.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:34 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-28T00:23:34.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:34 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-28T00:23:34.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:34 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:23:34.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:34 smithi165 ceph-mon[36302]: Deploying daemon mds.h on smithi132 2024-03-28T00:23:34.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:34 smithi165 ceph-mon[36302]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T00:23:34.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:34 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.165:6836/1531193032,v1:172.21.15.165:6837/1531193032] up:boot 2024-03-28T00:23:34.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:34 smithi165 ceph-mon[36302]: fsmap 7 up:standby 2024-03-28T00:23:34.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:34 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-28T00:23:34.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:34 smithi165 ceph-mon[36302]: from='client.? 172.21.15.119:0/1207790007' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T00:23:34.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:34 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:34.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:34 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:34.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:34 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:34.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:34 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-28T00:23:34.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:34 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-28T00:23:34.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:34 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:23:34.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:34 smithi119 ceph-mon[31518]: Deploying daemon mds.h on smithi132 2024-03-28T00:23:34.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:34 smithi119 ceph-mon[31518]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T00:23:34.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:34 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.165:6836/1531193032,v1:172.21.15.165:6837/1531193032] up:boot 2024-03-28T00:23:34.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:34 smithi119 ceph-mon[31518]: fsmap 7 up:standby 2024-03-28T00:23:34.907 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:34 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-28T00:23:34.907 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:34 smithi119 ceph-mon[31518]: from='client.? 172.21.15.119:0/1207790007' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T00:23:34.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:34 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:34.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:34 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:34.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:34 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:34.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:34 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-28T00:23:34.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:34 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-28T00:23:34.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:34 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:23:34.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:34 smithi132 ceph-mon[35941]: Deploying daemon mds.h on smithi132 2024-03-28T00:23:34.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:34 smithi132 ceph-mon[35941]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T00:23:34.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:34 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.165:6836/1531193032,v1:172.21.15.165:6837/1531193032] up:boot 2024-03-28T00:23:34.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:34 smithi132 ceph-mon[35941]: fsmap 7 up:standby 2024-03-28T00:23:34.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:34 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-28T00:23:34.954 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:34 smithi132 ceph-mon[35941]: from='client.? 172.21.15.119:0/1207790007' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T00:23:35.954 INFO:journalctl@ceph.mds.h.smithi132.stdout:Mar 28 00:23:35 smithi132 systemd[1]: Starting Ceph mds.h for 10f73b92-ec98-11ee-95d0-87774f69a715... 2024-03-28T00:23:36.290 INFO:journalctl@ceph.mds.h.smithi132.stdout:Mar 28 00:23:36 smithi132 podman[66114]: 2024-03-28 00:23:35.936972723 +0000 UTC m=+0.017712797 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-28T00:23:36.291 INFO:journalctl@ceph.mds.h.smithi132.stdout:Mar 28 00:23:36 smithi132 podman[66114]: 2024-03-28 00:23:36.054894598 +0000 UTC m=+0.135634672 container create 2b9188b0f91b2effcad70e5d7bf2773db9ef9f79c6227489c54f214032e800ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-h, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319) 2024-03-28T00:23:36.291 INFO:journalctl@ceph.mds.h.smithi132.stdout:Mar 28 00:23:36 smithi132 podman[66114]: 2024-03-28 00:23:36.283015386 +0000 UTC m=+0.363755461 container init 2b9188b0f91b2effcad70e5d7bf2773db9ef9f79c6227489c54f214032e800ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-h, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c) 2024-03-28T00:23:36.513 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T00:23:36.702 INFO:journalctl@ceph.mds.h.smithi132.stdout:Mar 28 00:23:36 smithi132 podman[66114]: 2024-03-28 00:23:36.285520851 +0000 UTC m=+0.366260925 container start 2b9188b0f91b2effcad70e5d7bf2773db9ef9f79c6227489c54f214032e800ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-h, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-28T00:23:36.703 INFO:journalctl@ceph.mds.h.smithi132.stdout:Mar 28 00:23:36 smithi132 ceph-mds[66133]: -- 172.21.15.132:0/1077881770 <== mon.2 v2:172.21.15.132:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x558d88c1da40 con 0x558d88bf0800 2024-03-28T00:23:36.703 INFO:journalctl@ceph.mds.h.smithi132.stdout:Mar 28 00:23:36 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-h[66129]: starting mds.h at 2024-03-28T00:23:36.703 INFO:journalctl@ceph.mds.h.smithi132.stdout:Mar 28 00:23:36 smithi132 bash[66114]: 2b9188b0f91b2effcad70e5d7bf2773db9ef9f79c6227489c54f214032e800ee 2024-03-28T00:23:36.703 INFO:journalctl@ceph.mds.h.smithi132.stdout:Mar 28 00:23:36 smithi132 systemd[1]: Started Ceph mds.h for 10f73b92-ec98-11ee-95d0-87774f69a715. 2024-03-28T00:23:36.723 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:36.717+0000 7fb5f7e00640 1 -- 172.21.15.119:0/2761118106 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb5f81052a0 msgr2=0x7fb5f810bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:36.723 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:36.717+0000 7fb5f7e00640 1 --2- 172.21.15.119:0/2761118106 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb5f81052a0 0x7fb5f810bb30 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fb5f0009ca0 tx=0x7fb5f002c100 comp rx=0 tx=0).stop 2024-03-28T00:23:36.723 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:36.717+0000 7fb5f7e00640 1 -- 172.21.15.119:0/2761118106 shutdown_connections 2024-03-28T00:23:36.723 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:36.717+0000 7fb5f7e00640 1 --2- 172.21.15.119:0/2761118106 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb5f81052a0 0x7fb5f810bb30 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:36.724 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:36.717+0000 7fb5f7e00640 1 --2- 172.21.15.119:0/2761118106 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb5f81048e0 0x7fb5f8104d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:36.724 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:36.717+0000 7fb5f7e00640 1 --2- 172.21.15.119:0/2761118106 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb5f81036f0 0x7fb5f8103b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:36.724 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:36.717+0000 7fb5f7e00640 1 -- 172.21.15.119:0/2761118106 >> 172.21.15.119:0/2761118106 conn(0x7fb5f80fec80 msgr2=0x7fb5f81010a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:23:36.724 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:36.717+0000 7fb5f7e00640 1 -- 172.21.15.119:0/2761118106 shutdown_connections 2024-03-28T00:23:36.724 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:36.717+0000 7fb5f7e00640 1 -- 172.21.15.119:0/2761118106 wait complete. 2024-03-28T00:23:36.726 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:36.718+0000 7fb5f7e00640 1 Processor -- start 2024-03-28T00:23:36.726 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:36.718+0000 7fb5f7e00640 1 -- start start 2024-03-28T00:23:36.726 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:36.718+0000 7fb5f7e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb5f81036f0 0x7fb5f8199950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:23:36.726 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:36.718+0000 7fb5f7e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb5f81048e0 0x7fb5f8199e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:23:36.726 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:36.718+0000 7fb5f7e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb5f81052a0 0x7fb5f81a0f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:23:36.726 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:36.718+0000 7fb5f7e00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5f810f370 con 0x7fb5f81048e0 2024-03-28T00:23:36.726 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:36.718+0000 7fb5f7e00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5f810f1f0 con 0x7fb5f81052a0 2024-03-28T00:23:36.726 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:36.718+0000 7fb5f7e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5f810f4f0 con 0x7fb5f81036f0 2024-03-28T00:23:36.727 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:36.718+0000 7fb5f6a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb5f81036f0 0x7fb5f8199950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:23:36.727 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:36.719+0000 7fb5f6a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb5f81036f0 0x7fb5f8199950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.119:52724/0 (socket says 172.21.15.119:52724) 2024-03-28T00:23:36.727 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:36.719+0000 7fb5f6a00640 1 -- 172.21.15.119:0/1650534618 learned_addr learned my addr 172.21.15.119:0/1650534618 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:23:36.727 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:36.719+0000 7fb5efe00640 1 --2- 172.21.15.119:0/1650534618 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb5f81048e0 0x7fb5f8199e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:23:36.727 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:36.719+0000 7fb5f7400640 1 --2- 172.21.15.119:0/1650534618 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb5f81052a0 0x7fb5f81a0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:23:36.727 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:36.719+0000 7fb5f6a00640 1 -- 172.21.15.119:0/1650534618 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb5f81052a0 msgr2=0x7fb5f81a0f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:36.727 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:36.719+0000 7fb5f6a00640 1 --2- 172.21.15.119:0/1650534618 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb5f81052a0 0x7fb5f81a0f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:36.727 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:36.719+0000 7fb5f6a00640 1 -- 172.21.15.119:0/1650534618 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb5f81048e0 msgr2=0x7fb5f8199e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:36.727 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:36.719+0000 7fb5f6a00640 1 --2- 172.21.15.119:0/1650534618 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb5f81048e0 0x7fb5f8199e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:36.727 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:36.719+0000 7fb5f6a00640 1 -- 172.21.15.119:0/1650534618 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5f81a1570 con 0x7fb5f81036f0 2024-03-28T00:23:36.727 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:36.719+0000 7fb5f6a00640 1 --2- 172.21.15.119:0/1650534618 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb5f81036f0 0x7fb5f8199950 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fb5e800e9c0 tx=0x7fb5e800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:23:36.727 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:36.719+0000 7fb5ef400640 1 -- 172.21.15.119:0/1650534618 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5e800cd80 con 0x7fb5f81036f0 2024-03-28T00:23:36.728 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:36.720+0000 7fb5ef400640 1 -- 172.21.15.119:0/1650534618 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb5e80035f0 con 0x7fb5f81036f0 2024-03-28T00:23:36.728 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:36.720+0000 7fb5ef400640 1 -- 172.21.15.119:0/1650534618 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5e8010690 con 0x7fb5f81036f0 2024-03-28T00:23:36.728 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:36.720+0000 7fb5f7e00640 1 -- 172.21.15.119:0/1650534618 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5f81a1860 con 0x7fb5f81036f0 2024-03-28T00:23:36.728 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:36.720+0000 7fb5f7e00640 1 -- 172.21.15.119:0/1650534618 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5f81a1d60 con 0x7fb5f81036f0 2024-03-28T00:23:36.728 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:36.721+0000 7fb5f7e00640 1 -- 172.21.15.119:0/1650534618 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5f810a9b0 con 0x7fb5f81036f0 2024-03-28T00:23:36.729 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:36.723+0000 7fb5ef400640 1 -- 172.21.15.119:0/1650534618 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fb5e80037a0 con 0x7fb5f81036f0 2024-03-28T00:23:36.730 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:36.724+0000 7fb5ef400640 1 --2- 172.21.15.119:0/1650534618 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fb5dc0a2880 0x7fb5dc0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:23:36.730 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:36.724+0000 7fb5ef400640 1 -- 172.21.15.119:0/1650534618 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fb5e80ceb10 con 0x7fb5f81036f0 2024-03-28T00:23:36.730 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:36.725+0000 7fb5efe00640 1 --2- 172.21.15.119:0/1650534618 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fb5dc0a2880 0x7fb5dc0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:23:36.731 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:36.725+0000 7fb5efe00640 1 --2- 172.21.15.119:0/1650534618 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fb5dc0a2880 0x7fb5dc0a4d40 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb5f819ae60 tx=0x7fb5e0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:23:36.732 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:36.726+0000 7fb5ef400640 1 -- 172.21.15.119:0/1650534618 <== mon.1 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fb5e809c040 con 0x7fb5f81036f0 2024-03-28T00:23:36.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:36 smithi119 ceph-mon[31518]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T00:23:36.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:36 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:36.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:36 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:36.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:36 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:36.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:36 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-28T00:23:36.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:36 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-28T00:23:36.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:36 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:23:36.974 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:36.968+0000 7fb5f7e00640 1 -- 172.21.15.119:0/1650534618 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb5f802d320 con 0x7fb5f81036f0 2024-03-28T00:23:36.975 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:36.969+0000 7fb5ef400640 1 -- 172.21.15.119:0/1650534618 <== mon.1 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+6877 (secure 0 0 0) 0x7fb5e809b940 con 0x7fb5f81036f0 2024-03-28T00:23:36.975 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-28T00:23:36.975 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14400,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/3988635716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":3988635716},{"type":"v1","addr":"172.21.15.119:6837","nonce":3988635716}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/609963168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":609963168},{"type":"v1","addr":"172.21.15.165:6833","nonce":609963168}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/1179016670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":1179016670},{"type":"v1","addr":"172.21.15.132:6833","nonce":1179016670}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24473,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/744950487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":744950487},{"type":"v1","addr":"172.21.15.119:6835","nonce":744950487}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24500,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/356981946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":356981946},{"type":"v1","addr":"172.21.15.132:6835","nonce":356981946}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24511,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6835/1947980102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":1947980102},{"type":"v1","addr":"172.21.15.165:6835","nonce":1947980102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24523,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/1531193032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":1531193032},{"type":"v1","addr":"172.21.15.165:6837","nonce":1531193032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24530,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6837/3383082967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6836","nonce":3383082967},{"type":"v1","addr":"172.21.15.132:6837","nonce":3383082967}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[]} 2024-03-28T00:23:36.976 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 9 2024-03-28T00:23:36.978 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:36.971+0000 7fb5f7e00640 1 -- 172.21.15.119:0/1650534618 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fb5dc0a2880 msgr2=0x7fb5dc0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:36.978 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:36.971+0000 7fb5f7e00640 1 --2- 172.21.15.119:0/1650534618 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fb5dc0a2880 0x7fb5dc0a4d40 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb5f819ae60 tx=0x7fb5e0005f90 comp rx=0 tx=0).stop 2024-03-28T00:23:36.978 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:36.971+0000 7fb5f7e00640 1 -- 172.21.15.119:0/1650534618 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb5f81036f0 msgr2=0x7fb5f8199950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:36.978 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:36.971+0000 7fb5f7e00640 1 --2- 172.21.15.119:0/1650534618 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb5f81036f0 0x7fb5f8199950 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fb5e800e9c0 tx=0x7fb5e800ee80 comp rx=0 tx=0).stop 2024-03-28T00:23:36.978 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:36.972+0000 7fb5f7e00640 1 -- 172.21.15.119:0/1650534618 shutdown_connections 2024-03-28T00:23:36.978 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:36.972+0000 7fb5f7e00640 1 --2- 172.21.15.119:0/1650534618 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fb5dc0a2880 0x7fb5dc0a4d40 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:36.978 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:36.972+0000 7fb5f7e00640 1 --2- 172.21.15.119:0/1650534618 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb5f81052a0 0x7fb5f81a0f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:36.978 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:36.972+0000 7fb5f7e00640 1 --2- 172.21.15.119:0/1650534618 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb5f81048e0 0x7fb5f8199e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:36.978 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:36.972+0000 7fb5f7e00640 1 --2- 172.21.15.119:0/1650534618 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb5f81036f0 0x7fb5f8199950 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:36.979 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:36.972+0000 7fb5f7e00640 1 -- 172.21.15.119:0/1650534618 >> 172.21.15.119:0/1650534618 conn(0x7fb5f80fec80 msgr2=0x7fb5f8107b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:23:36.981 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:36.972+0000 7fb5f7e00640 1 -- 172.21.15.119:0/1650534618 shutdown_connections 2024-03-28T00:23:36.981 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:36.972+0000 7fb5f7e00640 1 -- 172.21.15.119:0/1650534618 wait complete. 2024-03-28T00:23:37.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:36 smithi165 ceph-mon[36302]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T00:23:37.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:36 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:37.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:36 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:37.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:36 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:37.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:36 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-28T00:23:37.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:36 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-28T00:23:37.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:36 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:23:37.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:36 smithi132 ceph-mon[35941]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T00:23:37.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:36 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:37.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:36 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:37.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:36 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:37.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:36 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-28T00:23:37.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:36 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-28T00:23:37.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:36 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:23:38.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:37 smithi165 ceph-mon[36302]: Deploying daemon mds.i on smithi165 2024-03-28T00:23:38.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:37 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.132:6836/3383082967,v1:172.21.15.132:6837/3383082967] up:boot 2024-03-28T00:23:38.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:37 smithi165 ceph-mon[36302]: fsmap 8 up:standby 2024-03-28T00:23:38.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:37 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-28T00:23:38.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:37 smithi165 ceph-mon[36302]: from='client.? 172.21.15.119:0/1650534618' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T00:23:38.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:37 smithi119 ceph-mon[31518]: Deploying daemon mds.i on smithi165 2024-03-28T00:23:38.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:37 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.132:6836/3383082967,v1:172.21.15.132:6837/3383082967] up:boot 2024-03-28T00:23:38.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:37 smithi119 ceph-mon[31518]: fsmap 8 up:standby 2024-03-28T00:23:38.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:37 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-28T00:23:38.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:37 smithi119 ceph-mon[31518]: from='client.? 172.21.15.119:0/1650534618' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T00:23:38.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:37 smithi132 ceph-mon[35941]: Deploying daemon mds.i on smithi165 2024-03-28T00:23:38.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:37 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.132:6836/3383082967,v1:172.21.15.132:6837/3383082967] up:boot 2024-03-28T00:23:38.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:37 smithi132 ceph-mon[35941]: fsmap 8 up:standby 2024-03-28T00:23:38.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:37 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-28T00:23:38.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:37 smithi132 ceph-mon[35941]: from='client.? 172.21.15.119:0/1650534618' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T00:23:38.997 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T00:23:39.041 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:38 smithi165 ceph-mon[36302]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T00:23:39.041 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:23:38 smithi165 systemd[1]: Starting Ceph mds.i for 10f73b92-ec98-11ee-95d0-87774f69a715... 2024-03-28T00:23:39.084 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:38 smithi119 ceph-mon[31518]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T00:23:39.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:38 smithi132 ceph-mon[35941]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T00:23:39.219 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:39.213+0000 7fa565a00640 1 -- 172.21.15.119:0/2057700744 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa5601030b0 msgr2=0x7fa560109940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:39.220 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:39.213+0000 7fa565a00640 1 --2- 172.21.15.119:0/2057700744 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa5601030b0 0x7fa560109940 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa558009860 tx=0x7fa55802bb70 comp rx=0 tx=0).stop 2024-03-28T00:23:39.220 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:39.213+0000 7fa565a00640 1 -- 172.21.15.119:0/2057700744 shutdown_connections 2024-03-28T00:23:39.220 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:39.213+0000 7fa565a00640 1 --2- 172.21.15.119:0/2057700744 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa5601030b0 0x7fa560109940 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:39.220 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:39.213+0000 7fa565a00640 1 --2- 172.21.15.119:0/2057700744 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa5601026f0 0x7fa560102b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:39.220 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:39.213+0000 7fa565a00640 1 --2- 172.21.15.119:0/2057700744 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa560101500 0x7fa560101920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:39.220 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:39.213+0000 7fa565a00640 1 -- 172.21.15.119:0/2057700744 >> 172.21.15.119:0/2057700744 conn(0x7fa5600fca70 msgr2=0x7fa5600feeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:23:39.220 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:39.213+0000 7fa565a00640 1 -- 172.21.15.119:0/2057700744 shutdown_connections 2024-03-28T00:23:39.220 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:39.213+0000 7fa565a00640 1 -- 172.21.15.119:0/2057700744 wait complete. 2024-03-28T00:23:39.222 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:39.214+0000 7fa565a00640 1 Processor -- start 2024-03-28T00:23:39.222 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:39.214+0000 7fa565a00640 1 -- start start 2024-03-28T00:23:39.222 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:39.214+0000 7fa565a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa560101500 0x7fa560075450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:23:39.223 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:39.214+0000 7fa565a00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa5601026f0 0x7fa560073a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:23:39.223 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:39.214+0000 7fa565a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa5601030b0 0x7fa560073fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:23:39.223 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:39.214+0000 7fa565a00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa56010d160 con 0x7fa560101500 2024-03-28T00:23:39.223 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:39.214+0000 7fa565a00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa56010cfe0 con 0x7fa5601026f0 2024-03-28T00:23:39.223 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:39.214+0000 7fa565a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa56010d2e0 con 0x7fa5601030b0 2024-03-28T00:23:39.223 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:39.215+0000 7fa565000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa5601030b0 0x7fa560073fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:23:39.223 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:39.215+0000 7fa565000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa5601030b0 0x7fa560073fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.119:52746/0 (socket says 172.21.15.119:52746) 2024-03-28T00:23:39.223 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:39.215+0000 7fa565000640 1 -- 172.21.15.119:0/2924534139 learned_addr learned my addr 172.21.15.119:0/2924534139 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:23:39.223 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:39.215+0000 7fa55fe00640 1 --2- 172.21.15.119:0/2924534139 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa560101500 0x7fa560075450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:23:39.223 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:39.215+0000 7fa55f400640 1 --2- 172.21.15.119:0/2924534139 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa5601026f0 0x7fa560073a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:23:39.223 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:39.215+0000 7fa565000640 1 -- 172.21.15.119:0/2924534139 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa5601026f0 msgr2=0x7fa560073a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:39.223 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:39.215+0000 7fa565000640 1 --2- 172.21.15.119:0/2924534139 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa5601026f0 0x7fa560073a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:39.223 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:39.215+0000 7fa565000640 1 -- 172.21.15.119:0/2924534139 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa560101500 msgr2=0x7fa560075450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:39.223 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:39.215+0000 7fa565000640 1 --2- 172.21.15.119:0/2924534139 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa560101500 0x7fa560075450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:39.224 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:39.215+0000 7fa565000640 1 -- 172.21.15.119:0/2924534139 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa560074860 con 0x7fa5601030b0 2024-03-28T00:23:39.224 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:39.215+0000 7fa565000640 1 --2- 172.21.15.119:0/2924534139 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa5601030b0 0x7fa560073fd0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa55802c180 tx=0x7fa558004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:23:39.224 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:39.216+0000 7fa55cc00640 1 -- 172.21.15.119:0/2924534139 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa558041020 con 0x7fa5601030b0 2024-03-28T00:23:39.224 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:39.216+0000 7fa55cc00640 1 -- 172.21.15.119:0/2924534139 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa5580347b0 con 0x7fa5601030b0 2024-03-28T00:23:39.224 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:39.216+0000 7fa565a00640 1 -- 172.21.15.119:0/2924534139 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5601a1860 con 0x7fa5601030b0 2024-03-28T00:23:39.224 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:39.216+0000 7fa55cc00640 1 -- 172.21.15.119:0/2924534139 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa558039040 con 0x7fa5601030b0 2024-03-28T00:23:39.224 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:39.216+0000 7fa565a00640 1 -- 172.21.15.119:0/2924534139 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5601a1e00 con 0x7fa5601030b0 2024-03-28T00:23:39.225 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:39.217+0000 7fa565a00640 1 -- 172.21.15.119:0/2924534139 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa560108820 con 0x7fa5601030b0 2024-03-28T00:23:39.225 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:39.219+0000 7fa55cc00640 1 -- 172.21.15.119:0/2924534139 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fa558044020 con 0x7fa5601030b0 2024-03-28T00:23:39.226 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:39.220+0000 7fa55cc00640 1 --2- 172.21.15.119:0/2924534139 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fa5400a2880 0x7fa5400a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:23:39.227 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:39.221+0000 7fa55cc00640 1 -- 172.21.15.119:0/2924534139 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fa5580f0340 con 0x7fa5601030b0 2024-03-28T00:23:39.227 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:39.221+0000 7fa55fe00640 1 --2- 172.21.15.119:0/2924534139 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fa5400a2880 0x7fa5400a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:23:39.227 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:39.221+0000 7fa55fe00640 1 --2- 172.21.15.119:0/2924534139 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fa5400a2880 0x7fa5400a4d40 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa560102550 tx=0x7fa550009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:23:39.228 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:39.222+0000 7fa55cc00640 1 -- 172.21.15.119:0/2924534139 <== mon.1 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa55804b020 con 0x7fa5601030b0 2024-03-28T00:23:39.322 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:23:39 smithi165 podman[68059]: 2024-03-28 00:23:38.938554693 +0000 UTC m=+0.017765470 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-28T00:23:39.322 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:23:39 smithi165 podman[68059]: 2024-03-28 00:23:39.072056264 +0000 UTC m=+0.151267020 container create 19dc1822c951d8dfb1a7533592550913b638bcad6b225e1d337341dc7a36af1f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-28T00:23:39.322 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:23:39 smithi165 podman[68059]: 2024-03-28 00:23:39.316557163 +0000 UTC m=+0.395767930 container init 19dc1822c951d8dfb1a7533592550913b638bcad6b225e1d337341dc7a36af1f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-03-28T00:23:39.470 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:39.464+0000 7fa565a00640 1 -- 172.21.15.119:0/2924534139 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa56002d070 con 0x7fa5601030b0 2024-03-28T00:23:39.471 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:39.465+0000 7fa55cc00640 1 -- 172.21.15.119:0/2924534139 <== mon.1 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+6877 (secure 0 0 0) 0x7fa5580bd860 con 0x7fa5601030b0 2024-03-28T00:23:39.471 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-28T00:23:39.472 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14400,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/3988635716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":3988635716},{"type":"v1","addr":"172.21.15.119:6837","nonce":3988635716}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/609963168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":609963168},{"type":"v1","addr":"172.21.15.165:6833","nonce":609963168}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/1179016670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":1179016670},{"type":"v1","addr":"172.21.15.132:6833","nonce":1179016670}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24473,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/744950487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":744950487},{"type":"v1","addr":"172.21.15.119:6835","nonce":744950487}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24500,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/356981946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":356981946},{"type":"v1","addr":"172.21.15.132:6835","nonce":356981946}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24511,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6835/1947980102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":1947980102},{"type":"v1","addr":"172.21.15.165:6835","nonce":1947980102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24523,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/1531193032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":1531193032},{"type":"v1","addr":"172.21.15.165:6837","nonce":1531193032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24530,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6837/3383082967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6836","nonce":3383082967},{"type":"v1","addr":"172.21.15.132:6837","nonce":3383082967}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[]} 2024-03-28T00:23:39.472 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 9 2024-03-28T00:23:39.475 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:39.469+0000 7fa565a00640 1 -- 172.21.15.119:0/2924534139 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fa5400a2880 msgr2=0x7fa5400a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:39.475 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:39.469+0000 7fa565a00640 1 --2- 172.21.15.119:0/2924534139 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fa5400a2880 0x7fa5400a4d40 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa560102550 tx=0x7fa550009260 comp rx=0 tx=0).stop 2024-03-28T00:23:39.475 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:39.469+0000 7fa565a00640 1 -- 172.21.15.119:0/2924534139 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa5601030b0 msgr2=0x7fa560073fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:39.475 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:39.469+0000 7fa565a00640 1 --2- 172.21.15.119:0/2924534139 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa5601030b0 0x7fa560073fd0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa55802c180 tx=0x7fa558004270 comp rx=0 tx=0).stop 2024-03-28T00:23:39.475 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:39.469+0000 7fa565a00640 1 -- 172.21.15.119:0/2924534139 shutdown_connections 2024-03-28T00:23:39.475 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:39.469+0000 7fa565a00640 1 --2- 172.21.15.119:0/2924534139 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fa5400a2880 0x7fa5400a4d40 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:39.476 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:39.469+0000 7fa565a00640 1 --2- 172.21.15.119:0/2924534139 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa5601030b0 0x7fa560073fd0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:39.476 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:39.469+0000 7fa565a00640 1 --2- 172.21.15.119:0/2924534139 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa5601026f0 0x7fa560073a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:39.476 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:39.469+0000 7fa565a00640 1 --2- 172.21.15.119:0/2924534139 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa560101500 0x7fa560075450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:39.476 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:39.469+0000 7fa565a00640 1 -- 172.21.15.119:0/2924534139 >> 172.21.15.119:0/2924534139 conn(0x7fa5600fca70 msgr2=0x7fa5601059f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:23:39.476 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:39.469+0000 7fa565a00640 1 -- 172.21.15.119:0/2924534139 shutdown_connections 2024-03-28T00:23:39.476 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:39.469+0000 7fa565a00640 1 -- 172.21.15.119:0/2924534139 wait complete. 2024-03-28T00:23:39.632 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:23:39 smithi165 podman[68059]: 2024-03-28 00:23:39.31941444 +0000 UTC m=+0.398625202 container start 19dc1822c951d8dfb1a7533592550913b638bcad6b225e1d337341dc7a36af1f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-28T00:23:39.633 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:23:39 smithi165 ceph-mds[68078]: -- 172.21.15.165:0/798295133 <== mon.1 v2:172.21.15.165:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56103a4bda40 con 0x56103a490800 2024-03-28T00:23:39.633 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:23:39 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: starting mds.i at 2024-03-28T00:23:39.633 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:23:39 smithi165 bash[68059]: 19dc1822c951d8dfb1a7533592550913b638bcad6b225e1d337341dc7a36af1f 2024-03-28T00:23:39.633 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:23:39 smithi165 systemd[1]: Started Ceph mds.i for 10f73b92-ec98-11ee-95d0-87774f69a715. 2024-03-28T00:23:40.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:39 smithi165 ceph-mon[36302]: from='client.? 172.21.15.119:0/2924534139' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T00:23:40.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:39 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:40.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:39 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:40.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:39 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:40.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:39 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-28T00:23:40.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:39 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-28T00:23:40.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:39 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:23:40.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:39 smithi119 ceph-mon[31518]: from='client.? 172.21.15.119:0/2924534139' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T00:23:40.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:39 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:40.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:39 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:40.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:39 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:40.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:39 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-28T00:23:40.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:39 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-28T00:23:40.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:39 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:23:40.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:39 smithi132 ceph-mon[35941]: from='client.? 172.21.15.119:0/2924534139' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T00:23:40.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:39 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:40.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:39 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:40.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:39 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:40.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:39 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-28T00:23:40.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:39 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-28T00:23:40.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:39 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:23:41.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:40 smithi165 ceph-mon[36302]: Deploying daemon mds.g on smithi119 2024-03-28T00:23:41.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:40 smithi165 ceph-mon[36302]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T00:23:41.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:40 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.165:6838/3004684592,v1:172.21.15.165:6839/3004684592] up:boot 2024-03-28T00:23:41.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:40 smithi165 ceph-mon[36302]: fsmap 9 up:standby 2024-03-28T00:23:41.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:40 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-28T00:23:41.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:40 smithi119 ceph-mon[31518]: Deploying daemon mds.g on smithi119 2024-03-28T00:23:41.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:40 smithi119 ceph-mon[31518]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T00:23:41.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:40 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.165:6838/3004684592,v1:172.21.15.165:6839/3004684592] up:boot 2024-03-28T00:23:41.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:40 smithi119 ceph-mon[31518]: fsmap 9 up:standby 2024-03-28T00:23:41.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:40 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-28T00:23:41.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:40 smithi132 ceph-mon[35941]: Deploying daemon mds.g on smithi119 2024-03-28T00:23:41.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:40 smithi132 ceph-mon[35941]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T00:23:41.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:40 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.165:6838/3004684592,v1:172.21.15.165:6839/3004684592] up:boot 2024-03-28T00:23:41.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:40 smithi132 ceph-mon[35941]: fsmap 9 up:standby 2024-03-28T00:23:41.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:40 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-28T00:23:41.494 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T00:23:41.706 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:41.701+0000 7f12d2800640 1 -- 172.21.15.119:0/1752147913 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f12cc1071d0 msgr2=0x7f12cc109620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:41.707 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:41.701+0000 7f12d2800640 1 --2- 172.21.15.119:0/1752147913 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f12cc1071d0 0x7f12cc109620 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f12c4009be0 tx=0x7f12c402bef0 comp rx=0 tx=0).stop 2024-03-28T00:23:41.707 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:41.701+0000 7f12d2800640 1 -- 172.21.15.119:0/1752147913 shutdown_connections 2024-03-28T00:23:41.707 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:41.701+0000 7f12d2800640 1 --2- 172.21.15.119:0/1752147913 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f12cc1071d0 0x7f12cc109620 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:41.707 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:41.701+0000 7f12d2800640 1 --2- 172.21.15.119:0/1752147913 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f12cc1048a0 0x7f12cc106c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:41.707 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:41.701+0000 7f12d2800640 1 --2- 172.21.15.119:0/1752147913 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f12cc101f70 0x7f12cc104360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:41.707 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:41.701+0000 7f12d2800640 1 -- 172.21.15.119:0/1752147913 >> 172.21.15.119:0/1752147913 conn(0x7f12cc0fb980 msgr2=0x7f12cc0fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:23:41.707 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:41.701+0000 7f12d2800640 1 -- 172.21.15.119:0/1752147913 shutdown_connections 2024-03-28T00:23:41.707 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:41.701+0000 7f12d2800640 1 -- 172.21.15.119:0/1752147913 wait complete. 2024-03-28T00:23:41.709 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:41.702+0000 7f12d2800640 1 Processor -- start 2024-03-28T00:23:41.709 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:41.702+0000 7f12d2800640 1 -- start start 2024-03-28T00:23:41.709 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:41.703+0000 7f12d2800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f12cc101f70 0x7f12cc1999a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:23:41.709 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:41.703+0000 7f12d2800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f12cc1048a0 0x7f12cc199ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:23:41.709 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:41.703+0000 7f12d2800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f12cc1071d0 0x7f12cc1a0f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:23:41.709 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:41.703+0000 7f12d2800640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12cc10d1d0 con 0x7f12cc1071d0 2024-03-28T00:23:41.709 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:41.703+0000 7f12d2800640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12cc10d050 con 0x7f12cc101f70 2024-03-28T00:23:41.709 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:41.703+0000 7f12d2800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12cc10d350 con 0x7f12cc1048a0 2024-03-28T00:23:41.709 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:41.703+0000 7f12d1400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f12cc101f70 0x7f12cc1999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:23:41.709 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:41.703+0000 7f12d1400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f12cc101f70 0x7f12cc1999a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.119:43080/0 (socket says 172.21.15.119:43080) 2024-03-28T00:23:41.710 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:41.703+0000 7f12d1400640 1 -- 172.21.15.119:0/3821805413 learned_addr learned my addr 172.21.15.119:0/3821805413 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:23:41.710 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:41.703+0000 7f12d0a00640 1 --2- 172.21.15.119:0/3821805413 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f12cc1048a0 0x7f12cc199ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:23:41.710 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:41.703+0000 7f12d1e00640 1 --2- 172.21.15.119:0/3821805413 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f12cc1071d0 0x7f12cc1a0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:23:41.710 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:41.704+0000 7f12d1400640 1 -- 172.21.15.119:0/3821805413 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f12cc1048a0 msgr2=0x7f12cc199ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:41.710 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:41.704+0000 7f12d1400640 1 --2- 172.21.15.119:0/3821805413 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f12cc1048a0 0x7f12cc199ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:41.710 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:41.704+0000 7f12d1400640 1 -- 172.21.15.119:0/3821805413 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f12cc1071d0 msgr2=0x7f12cc1a0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:41.710 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:41.704+0000 7f12d1400640 1 --2- 172.21.15.119:0/3821805413 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f12cc1071d0 0x7f12cc1a0f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:41.710 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:41.704+0000 7f12d1400640 1 -- 172.21.15.119:0/3821805413 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12cc1a15c0 con 0x7f12cc101f70 2024-03-28T00:23:41.710 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:41.704+0000 7f12d1e00640 1 --2- 172.21.15.119:0/3821805413 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f12cc1071d0 0x7f12cc1a0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T00:23:41.710 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:41.704+0000 7f12d1400640 1 --2- 172.21.15.119:0/3821805413 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f12cc101f70 0x7f12cc1999a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f12c000e9f0 tx=0x7f12c000edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:23:41.711 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:41.704+0000 7f12ca000640 1 -- 172.21.15.119:0/3821805413 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12c000cd80 con 0x7f12cc101f70 2024-03-28T00:23:41.711 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:41.704+0000 7f12d2800640 1 -- 172.21.15.119:0/3821805413 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12cc1a18b0 con 0x7f12cc101f70 2024-03-28T00:23:41.711 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:41.705+0000 7f12ca000640 1 -- 172.21.15.119:0/3821805413 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f12c0005500 con 0x7f12cc101f70 2024-03-28T00:23:41.711 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:41.705+0000 7f12ca000640 1 -- 172.21.15.119:0/3821805413 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12c0010690 con 0x7f12cc101f70 2024-03-28T00:23:41.711 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:41.705+0000 7f12d2800640 1 -- 172.21.15.119:0/3821805413 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12cc1a1e00 con 0x7f12cc101f70 2024-03-28T00:23:41.712 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:41.706+0000 7f12d2800640 1 -- 172.21.15.119:0/3821805413 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12cc108cc0 con 0x7f12cc101f70 2024-03-28T00:23:41.715 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:41.709+0000 7f12ca000640 1 -- 172.21.15.119:0/3821805413 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f12c00046c0 con 0x7f12cc101f70 2024-03-28T00:23:41.717 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:41.711+0000 7f12ca000640 1 --2- 172.21.15.119:0/3821805413 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f12b40a2880 0x7f12b40a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:23:41.717 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:41.712+0000 7f12ca000640 1 -- 172.21.15.119:0/3821805413 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f12c00cea90 con 0x7f12cc101f70 2024-03-28T00:23:41.717 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:41.713+0000 7f12d0a00640 1 --2- 172.21.15.119:0/3821805413 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f12b40a2880 0x7f12b40a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:23:41.719 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:41.714+0000 7f12d0a00640 1 --2- 172.21.15.119:0/3821805413 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f12b40a2880 0x7f12b40a4d40 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f12cc19aeb0 tx=0x7f12b8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:23:41.720 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:41.715+0000 7f12ca000640 1 -- 172.21.15.119:0/3821805413 <== mon.2 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f12c009bf30 con 0x7f12cc101f70 2024-03-28T00:23:41.979 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:41.974+0000 7f12d2800640 1 -- 172.21.15.119:0/3821805413 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f12cc0622a0 con 0x7f12cc101f70 2024-03-28T00:23:41.980 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:41.975+0000 7f12ca000640 1 -- 172.21.15.119:0/3821805413 <== mon.2 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+7673 (secure 0 0 0) 0x7f12c009b830 con 0x7f12cc101f70 2024-03-28T00:23:41.980 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-28T00:23:41.981 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14400,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/3988635716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":3988635716},{"type":"v1","addr":"172.21.15.119:6837","nonce":3988635716}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/609963168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":609963168},{"type":"v1","addr":"172.21.15.165:6833","nonce":609963168}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/1179016670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":1179016670},{"type":"v1","addr":"172.21.15.132:6833","nonce":1179016670}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24473,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/744950487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":744950487},{"type":"v1","addr":"172.21.15.119:6835","nonce":744950487}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24500,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/356981946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":356981946},{"type":"v1","addr":"172.21.15.132:6835","nonce":356981946}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24511,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6835/1947980102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":1947980102},{"type":"v1","addr":"172.21.15.165:6835","nonce":1947980102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24523,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/1531193032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":1531193032},{"type":"v1","addr":"172.21.15.165:6837","nonce":1531193032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24530,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6837/3383082967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6836","nonce":3383082967},{"type":"v1","addr":"172.21.15.132:6837","nonce":3383082967}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24550,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6839/3004684592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":3004684592},{"type":"v1","addr":"172.21.15.165:6839","nonce":3004684592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2024-03-28T00:23:41.981 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 10 2024-03-28T00:23:41.984 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:41.978+0000 7f12d2800640 1 -- 172.21.15.119:0/3821805413 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f12b40a2880 msgr2=0x7f12b40a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:41.984 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:41.978+0000 7f12d2800640 1 --2- 172.21.15.119:0/3821805413 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f12b40a2880 0x7f12b40a4d40 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f12cc19aeb0 tx=0x7f12b8009260 comp rx=0 tx=0).stop 2024-03-28T00:23:41.984 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:41.978+0000 7f12d2800640 1 -- 172.21.15.119:0/3821805413 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f12cc101f70 msgr2=0x7f12cc1999a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:41.984 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:41.978+0000 7f12d2800640 1 --2- 172.21.15.119:0/3821805413 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f12cc101f70 0x7f12cc1999a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f12c000e9f0 tx=0x7f12c000edf0 comp rx=0 tx=0).stop 2024-03-28T00:23:41.984 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:41.979+0000 7f12d2800640 1 -- 172.21.15.119:0/3821805413 shutdown_connections 2024-03-28T00:23:41.984 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:41.979+0000 7f12d2800640 1 --2- 172.21.15.119:0/3821805413 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f12b40a2880 0x7f12b40a4d40 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:41.984 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:41.979+0000 7f12d2800640 1 --2- 172.21.15.119:0/3821805413 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f12cc1071d0 0x7f12cc1a0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:41.984 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:41.979+0000 7f12d2800640 1 --2- 172.21.15.119:0/3821805413 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f12cc1048a0 0x7f12cc199ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:41.985 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:41.979+0000 7f12d2800640 1 --2- 172.21.15.119:0/3821805413 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f12cc101f70 0x7f12cc1999a0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:41.985 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:41.979+0000 7f12d2800640 1 -- 172.21.15.119:0/3821805413 >> 172.21.15.119:0/3821805413 conn(0x7f12cc0fb980 msgr2=0x7f12cc1015f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:23:41.985 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:41.979+0000 7f12d2800640 1 -- 172.21.15.119:0/3821805413 shutdown_connections 2024-03-28T00:23:41.985 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:41.979+0000 7f12d2800640 1 -- 172.21.15.119:0/3821805413 wait complete. 2024-03-28T00:23:42.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:42 smithi119 ceph-mon[31518]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T00:23:42.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:42 smithi119 ceph-mon[31518]: from='client.? 172.21.15.119:0/3821805413' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T00:23:43.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:42 smithi165 ceph-mon[36302]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T00:23:43.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:42 smithi165 ceph-mon[36302]: from='client.? 172.21.15.119:0/3821805413' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T00:23:43.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:42 smithi132 ceph-mon[35941]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T00:23:43.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:42 smithi132 ceph-mon[35941]: from='client.? 172.21.15.119:0/3821805413' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T00:23:43.405 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:23:43 smithi119 systemd[1]: Started Ceph mds.g for 10f73b92-ec98-11ee-95d0-87774f69a715. 2024-03-28T00:23:44.004 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T00:23:44.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:44 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:44.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:44 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:44.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:44 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:44.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:44 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-28T00:23:44.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:44 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-28T00:23:44.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:44 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:23:44.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:44 smithi119 ceph-mon[31518]: Deploying daemon mds.a on smithi119 2024-03-28T00:23:44.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:44 smithi119 ceph-mon[31518]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T00:23:44.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:44 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.119:6838/1595343240,v1:172.21.15.119:6839/1595343240] up:boot 2024-03-28T00:23:44.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:44 smithi119 ceph-mon[31518]: fsmap 10 up:standby 2024-03-28T00:23:44.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:44 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-28T00:23:44.196 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:44.191+0000 7ff8ed400640 1 -- 172.21.15.119:0/2342858023 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff8e8105490 msgr2=0x7ff8e810bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:44.196 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:44.191+0000 7ff8ed400640 1 --2- 172.21.15.119:0/2342858023 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff8e8105490 0x7ff8e810bd20 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff8e0009ca0 tx=0x7ff8e002c100 comp rx=0 tx=0).stop 2024-03-28T00:23:44.196 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:44.191+0000 7ff8ed400640 1 -- 172.21.15.119:0/2342858023 shutdown_connections 2024-03-28T00:23:44.196 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:44.191+0000 7ff8ed400640 1 --2- 172.21.15.119:0/2342858023 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff8e8105490 0x7ff8e810bd20 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:44.196 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:44.191+0000 7ff8ed400640 1 --2- 172.21.15.119:0/2342858023 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff8e8104ad0 0x7ff8e8104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:44.196 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:44.191+0000 7ff8ed400640 1 --2- 172.21.15.119:0/2342858023 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff8e81038e0 0x7ff8e8103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:44.197 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:44.191+0000 7ff8ed400640 1 -- 172.21.15.119:0/2342858023 >> 172.21.15.119:0/2342858023 conn(0x7ff8e80fee50 msgr2=0x7ff8e8101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:23:44.197 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:44.191+0000 7ff8ed400640 1 -- 172.21.15.119:0/2342858023 shutdown_connections 2024-03-28T00:23:44.197 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:44.191+0000 7ff8ed400640 1 -- 172.21.15.119:0/2342858023 wait complete. 2024-03-28T00:23:44.199 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:44.192+0000 7ff8ed400640 1 Processor -- start 2024-03-28T00:23:44.199 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:44.192+0000 7ff8ed400640 1 -- start start 2024-03-28T00:23:44.199 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:44.192+0000 7ff8ed400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff8e81038e0 0x7ff8e8199b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:23:44.199 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:44.192+0000 7ff8ed400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff8e8104ad0 0x7ff8e819a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:23:44.199 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:44.192+0000 7ff8ed400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff8e8105490 0x7ff8e81a1130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:23:44.199 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:44.192+0000 7ff8ed400640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8e810f560 con 0x7ff8e8105490 2024-03-28T00:23:44.199 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:44.193+0000 7ff8ed400640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8e810f3e0 con 0x7ff8e8104ad0 2024-03-28T00:23:44.199 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:44.193+0000 7ff8ed400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8e810f6e0 con 0x7ff8e81038e0 2024-03-28T00:23:44.199 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:44.193+0000 7ff8e7200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff8e8105490 0x7ff8e81a1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:23:44.199 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:44.193+0000 7ff8e7200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff8e8105490 0x7ff8e81a1130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:36660/0 (socket says 172.21.15.119:36660) 2024-03-28T00:23:44.199 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:44.193+0000 7ff8e7200640 1 -- 172.21.15.119:0/1750033664 learned_addr learned my addr 172.21.15.119:0/1750033664 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:23:44.199 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:44.193+0000 7ff8e6800640 1 --2- 172.21.15.119:0/1750033664 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff8e81038e0 0x7ff8e8199b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:23:44.199 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:44.193+0000 7ff8dfe00640 1 --2- 172.21.15.119:0/1750033664 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff8e8104ad0 0x7ff8e819a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:23:44.199 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:44.193+0000 7ff8e7200640 1 -- 172.21.15.119:0/1750033664 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff8e81038e0 msgr2=0x7ff8e8199b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:44.200 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:44.193+0000 7ff8e7200640 1 --2- 172.21.15.119:0/1750033664 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff8e81038e0 0x7ff8e8199b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:44.200 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:44.193+0000 7ff8e7200640 1 -- 172.21.15.119:0/1750033664 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff8e8104ad0 msgr2=0x7ff8e819a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:44.200 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:44.193+0000 7ff8e7200640 1 --2- 172.21.15.119:0/1750033664 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff8e8104ad0 0x7ff8e819a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:44.200 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:44.193+0000 7ff8e7200640 1 -- 172.21.15.119:0/1750033664 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8e81a17a0 con 0x7ff8e8105490 2024-03-28T00:23:44.200 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:44.193+0000 7ff8e7200640 1 --2- 172.21.15.119:0/1750033664 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff8e8105490 0x7ff8e81a1130 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ff8e002c640 tx=0x7ff8e0035d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:23:44.200 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:44.194+0000 7ff8df400640 1 -- 172.21.15.119:0/1750033664 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8e0039040 con 0x7ff8e8105490 2024-03-28T00:23:44.200 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:44.194+0000 7ff8df400640 1 -- 172.21.15.119:0/1750033664 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff8e0044070 con 0x7ff8e8105490 2024-03-28T00:23:44.200 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:44.194+0000 7ff8df400640 1 -- 172.21.15.119:0/1750033664 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8e00043c0 con 0x7ff8e8105490 2024-03-28T00:23:44.200 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:44.194+0000 7ff8ed400640 1 -- 172.21.15.119:0/1750033664 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8e81a1a30 con 0x7ff8e8105490 2024-03-28T00:23:44.200 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:44.194+0000 7ff8ed400640 1 -- 172.21.15.119:0/1750033664 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8e81a1ed0 con 0x7ff8e8105490 2024-03-28T00:23:44.200 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:44.195+0000 7ff8ed400640 1 -- 172.21.15.119:0/1750033664 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8e810ac20 con 0x7ff8e8105490 2024-03-28T00:23:44.201 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:44.197+0000 7ff8df400640 1 -- 172.21.15.119:0/1750033664 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7ff8e0004560 con 0x7ff8e8105490 2024-03-28T00:23:44.203 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:44.199+0000 7ff8df400640 1 --2- 172.21.15.119:0/1750033664 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7ff8cc0a2800 0x7ff8cc0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:23:44.204 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:44.199+0000 7ff8df400640 1 -- 172.21.15.119:0/1750033664 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7ff8e00f0c40 con 0x7ff8e8105490 2024-03-28T00:23:44.204 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:44.199+0000 7ff8e6800640 1 --2- 172.21.15.119:0/1750033664 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7ff8cc0a2800 0x7ff8cc0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:23:44.204 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:44.199+0000 7ff8e6800640 1 --2- 172.21.15.119:0/1750033664 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7ff8cc0a2800 0x7ff8cc0a4cc0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ff8e8104930 tx=0x7ff8d8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:23:44.208 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:44.203+0000 7ff8df400640 1 -- 172.21.15.119:0/1750033664 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff8e00be0e0 con 0x7ff8e8105490 2024-03-28T00:23:44.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:44 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:44.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:44 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:44.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:44 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:44.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:44 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-28T00:23:44.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:44 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-28T00:23:44.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:44 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:23:44.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:44 smithi165 ceph-mon[36302]: Deploying daemon mds.a on smithi119 2024-03-28T00:23:44.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:44 smithi165 ceph-mon[36302]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T00:23:44.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:44 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.119:6838/1595343240,v1:172.21.15.119:6839/1595343240] up:boot 2024-03-28T00:23:44.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:44 smithi165 ceph-mon[36302]: fsmap 10 up:standby 2024-03-28T00:23:44.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:44 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-28T00:23:44.453 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:44.447+0000 7ff8ed400640 1 -- 172.21.15.119:0/1750033664 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff8e802d070 con 0x7ff8e8105490 2024-03-28T00:23:44.508 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:44 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:44.508 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:44 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:44.509 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:44 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:44.509 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:44 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-28T00:23:44.509 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:44 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-28T00:23:44.509 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:44 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:23:44.509 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:44 smithi132 ceph-mon[35941]: Deploying daemon mds.a on smithi119 2024-03-28T00:23:44.509 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:44 smithi132 ceph-mon[35941]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T00:23:44.509 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:44 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.119:6838/1595343240,v1:172.21.15.119:6839/1595343240] up:boot 2024-03-28T00:23:44.509 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:44 smithi132 ceph-mon[35941]: fsmap 10 up:standby 2024-03-28T00:23:44.509 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:44 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-28T00:23:44.510 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:44.449+0000 7ff8df400640 1 -- 172.21.15.119:0/1750033664 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+8468 (secure 0 0 0) 0x7ff8e00098b0 con 0x7ff8e8105490 2024-03-28T00:23:44.510 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-28T00:23:44.510 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14400,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/3988635716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":3988635716},{"type":"v1","addr":"172.21.15.119:6837","nonce":3988635716}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6839/1595343240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1595343240},{"type":"v1","addr":"172.21.15.119:6839","nonce":1595343240}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/609963168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":609963168},{"type":"v1","addr":"172.21.15.165:6833","nonce":609963168}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/1179016670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":1179016670},{"type":"v1","addr":"172.21.15.132:6833","nonce":1179016670}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24473,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/744950487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":744950487},{"type":"v1","addr":"172.21.15.119:6835","nonce":744950487}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24500,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/356981946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":356981946},{"type":"v1","addr":"172.21.15.132:6835","nonce":356981946}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24511,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6835/1947980102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":1947980102},{"type":"v1","addr":"172.21.15.165:6835","nonce":1947980102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24523,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/1531193032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":1531193032},{"type":"v1","addr":"172.21.15.165:6837","nonce":1531193032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24530,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6837/3383082967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6836","nonce":3383082967},{"type":"v1","addr":"172.21.15.132:6837","nonce":3383082967}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24550,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6839/3004684592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":3004684592},{"type":"v1","addr":"172.21.15.165:6839","nonce":3004684592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2024-03-28T00:23:44.512 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 11 2024-03-28T00:23:44.512 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:44.451+0000 7ff8ed400640 1 -- 172.21.15.119:0/1750033664 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7ff8cc0a2800 msgr2=0x7ff8cc0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:44.512 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:44.451+0000 7ff8ed400640 1 --2- 172.21.15.119:0/1750033664 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7ff8cc0a2800 0x7ff8cc0a4cc0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ff8e8104930 tx=0x7ff8d8005f90 comp rx=0 tx=0).stop 2024-03-28T00:23:44.512 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:44.451+0000 7ff8ed400640 1 -- 172.21.15.119:0/1750033664 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff8e8105490 msgr2=0x7ff8e81a1130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:44.512 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:44.451+0000 7ff8ed400640 1 --2- 172.21.15.119:0/1750033664 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff8e8105490 0x7ff8e81a1130 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ff8e002c640 tx=0x7ff8e0035d10 comp rx=0 tx=0).stop 2024-03-28T00:23:44.512 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:44.452+0000 7ff8ed400640 1 -- 172.21.15.119:0/1750033664 shutdown_connections 2024-03-28T00:23:44.512 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:44.452+0000 7ff8ed400640 1 --2- 172.21.15.119:0/1750033664 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7ff8cc0a2800 0x7ff8cc0a4cc0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:44.512 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:44.452+0000 7ff8ed400640 1 --2- 172.21.15.119:0/1750033664 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff8e8105490 0x7ff8e81a1130 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:44.512 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:44.452+0000 7ff8ed400640 1 --2- 172.21.15.119:0/1750033664 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff8e8104ad0 0x7ff8e819a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:44.512 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:44.452+0000 7ff8ed400640 1 --2- 172.21.15.119:0/1750033664 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff8e81038e0 0x7ff8e8199b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:44.512 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:44.452+0000 7ff8ed400640 1 -- 172.21.15.119:0/1750033664 >> 172.21.15.119:0/1750033664 conn(0x7ff8e80fee50 msgr2=0x7ff8e8107df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:23:44.512 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:44.452+0000 7ff8ed400640 1 -- 172.21.15.119:0/1750033664 shutdown_connections 2024-03-28T00:23:44.512 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:44.452+0000 7ff8ed400640 1 -- 172.21.15.119:0/1750033664 wait complete. 2024-03-28T00:23:45.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:45 smithi119 ceph-mon[31518]: from='client.? 172.21.15.119:0/1750033664' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T00:23:45.452 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:45 smithi132 ceph-mon[35941]: from='client.? 172.21.15.119:0/1750033664' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T00:23:45.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:45 smithi165 ceph-mon[36302]: from='client.? 172.21.15.119:0/1750033664' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T00:23:46.357 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:46 smithi119 ceph-mon[31518]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T00:23:46.514 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T00:23:46.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:46 smithi165 ceph-mon[36302]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T00:23:46.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:46 smithi132 ceph-mon[35941]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T00:23:46.709 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:46.704+0000 7fe115600640 1 -- 172.21.15.119:0/819819010 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe110108ed0 msgr2=0x7fe110109350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:46.709 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:46.704+0000 7fe115600640 1 --2- 172.21.15.119:0/819819010 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe110108ed0 0x7fe110109350 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe0f8009900 tx=0x7fe0f802bc30 comp rx=0 tx=0).stop 2024-03-28T00:23:46.709 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:46.704+0000 7fe115600640 1 -- 172.21.15.119:0/819819010 shutdown_connections 2024-03-28T00:23:46.710 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:46.704+0000 7fe115600640 1 --2- 172.21.15.119:0/819819010 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe110109890 0x7fe110110120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:46.710 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:46.704+0000 7fe115600640 1 --2- 172.21.15.119:0/819819010 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe110108ed0 0x7fe110109350 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:46.710 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:46.704+0000 7fe115600640 1 --2- 172.21.15.119:0/819819010 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe110107ce0 0x7fe110108100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:46.710 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:46.704+0000 7fe115600640 1 -- 172.21.15.119:0/819819010 >> 172.21.15.119:0/819819010 conn(0x7fe110076cb0 msgr2=0x7fe1100790d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:23:46.710 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:46.704+0000 7fe115600640 1 -- 172.21.15.119:0/819819010 shutdown_connections 2024-03-28T00:23:46.710 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:46.704+0000 7fe115600640 1 -- 172.21.15.119:0/819819010 wait complete. 2024-03-28T00:23:46.712 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:46.705+0000 7fe115600640 1 Processor -- start 2024-03-28T00:23:46.712 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:46.705+0000 7fe115600640 1 -- start start 2024-03-28T00:23:46.713 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:46.705+0000 7fe115600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe110107ce0 0x7fe11019dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:23:46.713 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:46.705+0000 7fe115600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe110108ed0 0x7fe11019e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:23:46.713 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:46.705+0000 7fe115600640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe110109890 0x7fe1101a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:23:46.713 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:46.705+0000 7fe115600640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1101139b0 con 0x7fe110108ed0 2024-03-28T00:23:46.713 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:46.705+0000 7fe115600640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe110113830 con 0x7fe110109890 2024-03-28T00:23:46.713 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:46.705+0000 7fe115600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe110113b30 con 0x7fe110107ce0 2024-03-28T00:23:46.713 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:46.705+0000 7fe107e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe110108ed0 0x7fe11019e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:23:46.713 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:46.706+0000 7fe107e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe110108ed0 0x7fe11019e4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:36672/0 (socket says 172.21.15.119:36672) 2024-03-28T00:23:46.713 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:46.706+0000 7fe107e00640 1 -- 172.21.15.119:0/842275880 learned_addr learned my addr 172.21.15.119:0/842275880 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:23:46.713 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:46.706+0000 7fe10ea00640 1 --2- 172.21.15.119:0/842275880 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe110107ce0 0x7fe11019dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:23:46.713 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:46.706+0000 7fe10f400640 1 --2- 172.21.15.119:0/842275880 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe110109890 0x7fe1101a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:23:46.713 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:46.706+0000 7fe107e00640 1 -- 172.21.15.119:0/842275880 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe110107ce0 msgr2=0x7fe11019dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:46.713 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:46.706+0000 7fe107e00640 1 --2- 172.21.15.119:0/842275880 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe110107ce0 0x7fe11019dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:46.714 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:46.706+0000 7fe107e00640 1 -- 172.21.15.119:0/842275880 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe110109890 msgr2=0x7fe1101a5550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:46.714 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:46.706+0000 7fe107e00640 1 --2- 172.21.15.119:0/842275880 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe110109890 0x7fe1101a5550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:46.714 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:46.706+0000 7fe107e00640 1 -- 172.21.15.119:0/842275880 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1101a5bc0 con 0x7fe110108ed0 2024-03-28T00:23:46.714 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:46.706+0000 7fe10ea00640 1 --2- 172.21.15.119:0/842275880 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe110107ce0 0x7fe11019dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T00:23:46.714 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:46.706+0000 7fe10f400640 1 --2- 172.21.15.119:0/842275880 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe110109890 0x7fe1101a5550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T00:23:46.714 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:46.706+0000 7fe107e00640 1 --2- 172.21.15.119:0/842275880 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe110108ed0 0x7fe11019e4e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe0f8000c00 tx=0x7fe0f802c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:23:46.714 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:46.707+0000 7fe107400640 1 -- 172.21.15.119:0/842275880 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0f8043020 con 0x7fe110108ed0 2024-03-28T00:23:46.714 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:46.707+0000 7fe115600640 1 -- 172.21.15.119:0/842275880 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1101a5e50 con 0x7fe110108ed0 2024-03-28T00:23:46.714 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:46.707+0000 7fe107400640 1 -- 172.21.15.119:0/842275880 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe0f80054e0 con 0x7fe110108ed0 2024-03-28T00:23:46.714 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:46.707+0000 7fe107400640 1 -- 172.21.15.119:0/842275880 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0f8003c40 con 0x7fe110108ed0 2024-03-28T00:23:46.714 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:46.707+0000 7fe115600640 1 -- 172.21.15.119:0/842275880 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1101a6310 con 0x7fe110108ed0 2024-03-28T00:23:46.715 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:46.708+0000 7fe115600640 1 -- 172.21.15.119:0/842275880 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe11010efa0 con 0x7fe110108ed0 2024-03-28T00:23:46.715 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:46.710+0000 7fe107400640 1 -- 172.21.15.119:0/842275880 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fe0f8046020 con 0x7fe110108ed0 2024-03-28T00:23:46.717 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:46.712+0000 7fe107400640 1 --2- 172.21.15.119:0/842275880 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fe0f40a2800 0x7fe0f40a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:23:46.717 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:46.712+0000 7fe107400640 1 -- 172.21.15.119:0/842275880 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fe0f80f0a70 con 0x7fe110108ed0 2024-03-28T00:23:46.717 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:46.712+0000 7fe10ea00640 1 --2- 172.21.15.119:0/842275880 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fe0f40a2800 0x7fe0f40a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:23:46.718 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:46.713+0000 7fe10ea00640 1 --2- 172.21.15.119:0/842275880 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fe0f40a2800 0x7fe0f40a4cc0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe110108d30 tx=0x7fe100005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:23:46.720 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:46.715+0000 7fe107400640 1 -- 172.21.15.119:0/842275880 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fe0f804d020 con 0x7fe110108ed0 2024-03-28T00:23:46.962 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:46.957+0000 7fe115600640 1 -- 172.21.15.119:0/842275880 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe11002d110 con 0x7fe110108ed0 2024-03-28T00:23:46.963 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:46.958+0000 7fe107400640 1 -- 172.21.15.119:0/842275880 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+8468 (secure 0 0 0) 0x7fe0f80bdf10 con 0x7fe110108ed0 2024-03-28T00:23:46.964 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-28T00:23:46.964 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14400,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/3988635716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":3988635716},{"type":"v1","addr":"172.21.15.119:6837","nonce":3988635716}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6839/1595343240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1595343240},{"type":"v1","addr":"172.21.15.119:6839","nonce":1595343240}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/609963168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":609963168},{"type":"v1","addr":"172.21.15.165:6833","nonce":609963168}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/1179016670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":1179016670},{"type":"v1","addr":"172.21.15.132:6833","nonce":1179016670}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24473,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/744950487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":744950487},{"type":"v1","addr":"172.21.15.119:6835","nonce":744950487}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24500,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/356981946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":356981946},{"type":"v1","addr":"172.21.15.132:6835","nonce":356981946}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24511,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6835/1947980102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":1947980102},{"type":"v1","addr":"172.21.15.165:6835","nonce":1947980102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24523,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/1531193032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":1531193032},{"type":"v1","addr":"172.21.15.165:6837","nonce":1531193032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24530,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6837/3383082967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6836","nonce":3383082967},{"type":"v1","addr":"172.21.15.132:6837","nonce":3383082967}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24550,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6839/3004684592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":3004684592},{"type":"v1","addr":"172.21.15.165:6839","nonce":3004684592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2024-03-28T00:23:46.964 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 11 2024-03-28T00:23:46.967 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:46.961+0000 7fe115600640 1 -- 172.21.15.119:0/842275880 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fe0f40a2800 msgr2=0x7fe0f40a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:46.967 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:46.961+0000 7fe115600640 1 --2- 172.21.15.119:0/842275880 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fe0f40a2800 0x7fe0f40a4cc0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe110108d30 tx=0x7fe100005f90 comp rx=0 tx=0).stop 2024-03-28T00:23:46.967 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:46.961+0000 7fe115600640 1 -- 172.21.15.119:0/842275880 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe110108ed0 msgr2=0x7fe11019e4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:46.967 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:46.961+0000 7fe115600640 1 --2- 172.21.15.119:0/842275880 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe110108ed0 0x7fe11019e4e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe0f8000c00 tx=0x7fe0f802c650 comp rx=0 tx=0).stop 2024-03-28T00:23:46.967 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:46.961+0000 7fe115600640 1 -- 172.21.15.119:0/842275880 shutdown_connections 2024-03-28T00:23:46.967 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:46.961+0000 7fe115600640 1 --2- 172.21.15.119:0/842275880 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fe0f40a2800 0x7fe0f40a4cc0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:46.967 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:46.961+0000 7fe115600640 1 --2- 172.21.15.119:0/842275880 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe110109890 0x7fe1101a5550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:46.967 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:46.961+0000 7fe115600640 1 --2- 172.21.15.119:0/842275880 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe110108ed0 0x7fe11019e4e0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:46.967 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:46.961+0000 7fe115600640 1 --2- 172.21.15.119:0/842275880 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe110107ce0 0x7fe11019dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:46.967 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:46.961+0000 7fe115600640 1 -- 172.21.15.119:0/842275880 >> 172.21.15.119:0/842275880 conn(0x7fe110076cb0 msgr2=0x7fe11010c170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:23:46.967 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:46.962+0000 7fe115600640 1 -- 172.21.15.119:0/842275880 shutdown_connections 2024-03-28T00:23:46.967 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:46.962+0000 7fe115600640 1 -- 172.21.15.119:0/842275880 wait complete. 2024-03-28T00:23:46.982 INFO:journalctl@ceph.mds.a.smithi119.stdout:Mar 28 00:23:46 smithi119 podman[70110]: 2024-03-28 00:23:46.753086663 +0000 UTC m=+0.174737483 container create a8cf73f63bdb7aff54cf7961135851c82b6e20cdd59f7781d68590c388401d4f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-a, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240319, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c) 2024-03-28T00:23:47.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:47 smithi119 ceph-mon[31518]: from='client.? 172.21.15.119:0/842275880' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T00:23:47.406 INFO:journalctl@ceph.mds.a.smithi119.stdout:Mar 28 00:23:47 smithi119 podman[70110]: 2024-03-28 00:23:47.091349434 +0000 UTC m=+0.513000281 container init a8cf73f63bdb7aff54cf7961135851c82b6e20cdd59f7781d68590c388401d4f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-a, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-03-28T00:23:47.406 INFO:journalctl@ceph.mds.a.smithi119.stdout:Mar 28 00:23:47 smithi119 podman[70110]: 2024-03-28 00:23:47.101077347 +0000 UTC m=+0.522728167 container start a8cf73f63bdb7aff54cf7961135851c82b6e20cdd59f7781d68590c388401d4f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-a, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-28T00:23:47.406 INFO:journalctl@ceph.mds.a.smithi119.stdout:Mar 28 00:23:47 smithi119 ceph-mds[70153]: -- 172.21.15.119:0/611436903 <== mon.0 v2:172.21.15.119:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x555635425a40 con 0x5556353f8800 2024-03-28T00:23:47.406 INFO:journalctl@ceph.mds.a.smithi119.stdout:Mar 28 00:23:47 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-a[70147]: starting mds.a at 2024-03-28T00:23:47.406 INFO:journalctl@ceph.mds.a.smithi119.stdout:Mar 28 00:23:47 smithi119 bash[70110]: a8cf73f63bdb7aff54cf7961135851c82b6e20cdd59f7781d68590c388401d4f 2024-03-28T00:23:47.406 INFO:journalctl@ceph.mds.a.smithi119.stdout:Mar 28 00:23:47 smithi119 systemd[1]: Started Ceph mds.a for 10f73b92-ec98-11ee-95d0-87774f69a715. 2024-03-28T00:23:47.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:47 smithi165 ceph-mon[36302]: from='client.? 172.21.15.119:0/842275880' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T00:23:47.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:47 smithi132 ceph-mon[35941]: from='client.? 172.21.15.119:0/842275880' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T00:23:48.350 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:48 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.119:6840/1678451188,v1:172.21.15.119:6841/1678451188] up:boot 2024-03-28T00:23:48.350 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:48 smithi132 ceph-mon[35941]: fsmap 11 up:standby 2024-03-28T00:23:48.350 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:48 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-28T00:23:48.350 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:48 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:48.350 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:48 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:48.350 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:48 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:48.350 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:48 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-28T00:23:48.350 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:48 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-28T00:23:48.350 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:48 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:23:48.350 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:48 smithi132 ceph-mon[35941]: Deploying daemon mds.b on smithi132 2024-03-28T00:23:48.350 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:48 smithi132 ceph-mon[35941]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T00:23:48.534 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:48 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.119:6840/1678451188,v1:172.21.15.119:6841/1678451188] up:boot 2024-03-28T00:23:48.534 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:48 smithi165 ceph-mon[36302]: fsmap 11 up:standby 2024-03-28T00:23:48.534 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:48 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-28T00:23:48.534 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:48 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:48.534 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:48 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:48.534 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:48 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:48.535 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:48 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-28T00:23:48.535 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:48 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-28T00:23:48.535 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:48 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:23:48.535 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:48 smithi165 ceph-mon[36302]: Deploying daemon mds.b on smithi132 2024-03-28T00:23:48.535 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:48 smithi165 ceph-mon[36302]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T00:23:48.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:48 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.119:6840/1678451188,v1:172.21.15.119:6841/1678451188] up:boot 2024-03-28T00:23:48.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:48 smithi119 ceph-mon[31518]: fsmap 11 up:standby 2024-03-28T00:23:48.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:48 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-28T00:23:48.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:48 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:48.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:48 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:48.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:48 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:48.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:48 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-28T00:23:48.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:48 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-28T00:23:48.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:48 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:23:48.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:48 smithi119 ceph-mon[31518]: Deploying daemon mds.b on smithi132 2024-03-28T00:23:48.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:48 smithi119 ceph-mon[31518]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T00:23:48.985 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T00:23:49.209 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:49.202+0000 7ffae1a00640 1 -- 172.21.15.119:0/385473551 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffadc074100 msgr2=0x7ffadc074580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:49.209 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:49.202+0000 7ffae1a00640 1 --2- 172.21.15.119:0/385473551 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffadc074100 0x7ffadc074580 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ffac4009890 tx=0x7ffac402bb70 comp rx=0 tx=0).stop 2024-03-28T00:23:49.209 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:49.202+0000 7ffae1a00640 1 -- 172.21.15.119:0/385473551 shutdown_connections 2024-03-28T00:23:49.209 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:49.202+0000 7ffae1a00640 1 --2- 172.21.15.119:0/385473551 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffadc10dab0 0x7ffadc10fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:49.209 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:49.202+0000 7ffae1a00640 1 --2- 172.21.15.119:0/385473551 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffadc074100 0x7ffadc074580 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:49.209 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:49.202+0000 7ffae1a00640 1 --2- 172.21.15.119:0/385473551 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffadc0756e0 0x7ffadc073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:49.209 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:49.202+0000 7ffae1a00640 1 -- 172.21.15.119:0/385473551 >> 172.21.15.119:0/385473551 conn(0x7ffadc0fca30 msgr2=0x7ffadc0fee90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:23:49.209 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:49.203+0000 7ffae1a00640 1 -- 172.21.15.119:0/385473551 shutdown_connections 2024-03-28T00:23:49.209 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:49.203+0000 7ffae1a00640 1 -- 172.21.15.119:0/385473551 wait complete. 2024-03-28T00:23:49.211 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:49.204+0000 7ffae1a00640 1 Processor -- start 2024-03-28T00:23:49.211 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:49.204+0000 7ffae1a00640 1 -- start start 2024-03-28T00:23:49.211 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:49.204+0000 7ffae1a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffadc074100 0x7ffadc19de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:23:49.211 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:49.204+0000 7ffae1a00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffadc0756e0 0x7ffadc19e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:23:49.211 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:49.204+0000 7ffae1a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffadc10dab0 0x7ffadc1a5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:23:49.211 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:49.204+0000 7ffae1a00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffadc113750 con 0x7ffadc10dab0 2024-03-28T00:23:49.211 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:49.204+0000 7ffae1a00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffadc1135d0 con 0x7ffadc0756e0 2024-03-28T00:23:49.211 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:49.204+0000 7ffae1a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffadc1138d0 con 0x7ffadc074100 2024-03-28T00:23:49.211 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:49.204+0000 7ffae1000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffadc10dab0 0x7ffadc1a5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:23:49.212 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:49.205+0000 7ffae1000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffadc10dab0 0x7ffadc1a5400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:36686/0 (socket says 172.21.15.119:36686) 2024-03-28T00:23:49.212 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:49.205+0000 7ffae1000640 1 -- 172.21.15.119:0/4033295100 learned_addr learned my addr 172.21.15.119:0/4033295100 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:23:49.212 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:49.205+0000 7ffad3e00640 1 --2- 172.21.15.119:0/4033295100 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffadc0756e0 0x7ffadc19e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:23:49.212 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:49.205+0000 7ffadbe00640 1 --2- 172.21.15.119:0/4033295100 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffadc074100 0x7ffadc19de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:23:49.212 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:49.205+0000 7ffae1000640 1 -- 172.21.15.119:0/4033295100 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffadc074100 msgr2=0x7ffadc19de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:49.212 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:49.205+0000 7ffae1000640 1 --2- 172.21.15.119:0/4033295100 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffadc074100 0x7ffadc19de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:49.212 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:49.205+0000 7ffae1000640 1 -- 172.21.15.119:0/4033295100 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffadc0756e0 msgr2=0x7ffadc19e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:49.212 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:49.205+0000 7ffae1000640 1 --2- 172.21.15.119:0/4033295100 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffadc0756e0 0x7ffadc19e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:49.212 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:49.205+0000 7ffae1000640 1 -- 172.21.15.119:0/4033295100 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffadc1a5a70 con 0x7ffadc10dab0 2024-03-28T00:23:49.212 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:49.205+0000 7ffad3e00640 1 --2- 172.21.15.119:0/4033295100 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffadc0756e0 0x7ffadc19e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T00:23:49.212 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:49.205+0000 7ffadbe00640 1 --2- 172.21.15.119:0/4033295100 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffadc074100 0x7ffadc19de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T00:23:49.212 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:49.205+0000 7ffae1000640 1 --2- 172.21.15.119:0/4033295100 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffadc10dab0 0x7ffadc1a5400 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ffad4009fd0 tx=0x7ffad400c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:23:49.213 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:49.205+0000 7ffad9600640 1 -- 172.21.15.119:0/4033295100 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffad4004b20 con 0x7ffadc10dab0 2024-03-28T00:23:49.213 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:49.206+0000 7ffad9600640 1 -- 172.21.15.119:0/4033295100 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ffad4003e50 con 0x7ffadc10dab0 2024-03-28T00:23:49.213 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:49.206+0000 7ffae1a00640 1 -- 172.21.15.119:0/4033295100 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffadc1a5d60 con 0x7ffadc10dab0 2024-03-28T00:23:49.213 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:49.206+0000 7ffad9600640 1 -- 172.21.15.119:0/4033295100 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffad400b630 con 0x7ffadc10dab0 2024-03-28T00:23:49.213 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:49.206+0000 7ffae1a00640 1 -- 172.21.15.119:0/4033295100 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffadc1a6250 con 0x7ffadc10dab0 2024-03-28T00:23:49.213 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:49.207+0000 7ffae1a00640 1 -- 172.21.15.119:0/4033295100 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffadc103c80 con 0x7ffadc10dab0 2024-03-28T00:23:49.214 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:49.209+0000 7ffad9600640 1 -- 172.21.15.119:0/4033295100 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7ffad401d020 con 0x7ffadc10dab0 2024-03-28T00:23:49.216 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:49.210+0000 7ffad9600640 1 --2- 172.21.15.119:0/4033295100 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7ffac00a2800 0x7ffac00a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:23:49.216 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:49.210+0000 7ffad9600640 1 -- 172.21.15.119:0/4033295100 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7ffad40ce700 con 0x7ffadc10dab0 2024-03-28T00:23:49.216 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:49.210+0000 7ffadbe00640 1 --2- 172.21.15.119:0/4033295100 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7ffac00a2800 0x7ffac00a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:23:49.216 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:49.211+0000 7ffadbe00640 1 --2- 172.21.15.119:0/4033295100 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7ffac00a2800 0x7ffac00a4cc0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ffacc009b50 tx=0x7ffacc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:23:49.220 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:49.214+0000 7ffad9600640 1 -- 172.21.15.119:0/4033295100 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ffad409bba0 con 0x7ffadc10dab0 2024-03-28T00:23:49.461 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:49.455+0000 7ffae1a00640 1 -- 172.21.15.119:0/4033295100 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffadc02d070 con 0x7ffadc10dab0 2024-03-28T00:23:49.462 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:49.456+0000 7ffad9600640 1 -- 172.21.15.119:0/4033295100 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+9263 (secure 0 0 0) 0x7ffad409b4a0 con 0x7ffadc10dab0 2024-03-28T00:23:49.463 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-28T00:23:49.463 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14400,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/3988635716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":3988635716},{"type":"v1","addr":"172.21.15.119:6837","nonce":3988635716}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6839/1595343240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1595343240},{"type":"v1","addr":"172.21.15.119:6839","nonce":1595343240}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14466,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6841/1678451188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6840","nonce":1678451188},{"type":"v1","addr":"172.21.15.119:6841","nonce":1678451188}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/609963168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":609963168},{"type":"v1","addr":"172.21.15.165:6833","nonce":609963168}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/1179016670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":1179016670},{"type":"v1","addr":"172.21.15.132:6833","nonce":1179016670}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24473,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/744950487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":744950487},{"type":"v1","addr":"172.21.15.119:6835","nonce":744950487}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24500,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/356981946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":356981946},{"type":"v1","addr":"172.21.15.132:6835","nonce":356981946}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24511,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6835/1947980102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":1947980102},{"type":"v1","addr":"172.21.15.165:6835","nonce":1947980102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24523,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/1531193032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":1531193032},{"type":"v1","addr":"172.21.15.165:6837","nonce":1531193032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24530,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6837/3383082967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6836","nonce":3383082967},{"type":"v1","addr":"172.21.15.132:6837","nonce":3383082967}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24550,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6839/3004684592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":3004684592},{"type":"v1","addr":"172.21.15.165:6839","nonce":3004684592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2024-03-28T00:23:49.463 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 12 2024-03-28T00:23:49.466 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:49.459+0000 7ffae1a00640 1 -- 172.21.15.119:0/4033295100 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7ffac00a2800 msgr2=0x7ffac00a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:49.466 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:49.459+0000 7ffae1a00640 1 --2- 172.21.15.119:0/4033295100 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7ffac00a2800 0x7ffac00a4cc0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ffacc009b50 tx=0x7ffacc005f90 comp rx=0 tx=0).stop 2024-03-28T00:23:49.466 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:49.459+0000 7ffae1a00640 1 -- 172.21.15.119:0/4033295100 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffadc10dab0 msgr2=0x7ffadc1a5400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:49.466 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:49.459+0000 7ffae1a00640 1 --2- 172.21.15.119:0/4033295100 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffadc10dab0 0x7ffadc1a5400 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ffad4009fd0 tx=0x7ffad400c800 comp rx=0 tx=0).stop 2024-03-28T00:23:49.466 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:49.460+0000 7ffae1a00640 1 -- 172.21.15.119:0/4033295100 shutdown_connections 2024-03-28T00:23:49.466 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:49.460+0000 7ffae1a00640 1 --2- 172.21.15.119:0/4033295100 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7ffac00a2800 0x7ffac00a4cc0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:49.466 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:49.460+0000 7ffae1a00640 1 --2- 172.21.15.119:0/4033295100 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffadc10dab0 0x7ffadc1a5400 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:49.466 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:49.460+0000 7ffae1a00640 1 --2- 172.21.15.119:0/4033295100 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffadc0756e0 0x7ffadc19e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:49.466 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:49.460+0000 7ffae1a00640 1 --2- 172.21.15.119:0/4033295100 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffadc074100 0x7ffadc19de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:49.466 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:49.460+0000 7ffae1a00640 1 -- 172.21.15.119:0/4033295100 >> 172.21.15.119:0/4033295100 conn(0x7ffadc0fca30 msgr2=0x7ffadc100e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:23:49.466 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:49.460+0000 7ffae1a00640 1 -- 172.21.15.119:0/4033295100 shutdown_connections 2024-03-28T00:23:49.466 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:49.460+0000 7ffae1a00640 1 -- 172.21.15.119:0/4033295100 wait complete. 2024-03-28T00:23:49.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:49 smithi132 ceph-mon[35941]: from='client.? 172.21.15.119:0/4033295100' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T00:23:49.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:49 smithi165 ceph-mon[36302]: from='client.? 172.21.15.119:0/4033295100' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T00:23:49.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:49 smithi119 ceph-mon[31518]: from='client.? 172.21.15.119:0/4033295100' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T00:23:49.953 INFO:journalctl@ceph.mds.b.smithi132.stdout:Mar 28 00:23:49 smithi132 podman[66428]: 2024-03-28 00:23:49.941073428 +0000 UTC m=+0.362041103 container start fe3fee15a0f94a3d3be8b3da321cffc54b745c71edf1b0c7685ea583d6dd0aa1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-b, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-28T00:23:50.452 INFO:journalctl@ceph.mds.b.smithi132.stdout:Mar 28 00:23:49 smithi132 ceph-mds[66447]: -- 172.21.15.132:0/1892833044 <== mon.2 v2:172.21.15.132:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55850e075a40 con 0x55850e049000 2024-03-28T00:23:50.453 INFO:journalctl@ceph.mds.b.smithi132.stdout:Mar 28 00:23:49 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-b[66443]: starting mds.b at 2024-03-28T00:23:50.453 INFO:journalctl@ceph.mds.b.smithi132.stdout:Mar 28 00:23:50 smithi132 bash[66428]: fe3fee15a0f94a3d3be8b3da321cffc54b745c71edf1b0c7685ea583d6dd0aa1 2024-03-28T00:23:50.453 INFO:journalctl@ceph.mds.b.smithi132.stdout:Mar 28 00:23:50 smithi132 systemd[1]: Started Ceph mds.b for 10f73b92-ec98-11ee-95d0-87774f69a715. 2024-03-28T00:23:50.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:50 smithi165 ceph-mon[36302]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T00:23:50.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:50 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:50.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:50 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:50.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:50 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:50.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:50 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:50.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:50 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:50.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:50 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:23:50.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:50 smithi119 ceph-mon[31518]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T00:23:50.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:50 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:50.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:50 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:50.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:50 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:50.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:50 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:50.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:50 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:50.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:50 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:23:50.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:50 smithi132 ceph-mon[35941]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T00:23:50.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:50 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:50.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:50 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:50.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:50 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:50.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:50 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:50.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:50 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:50.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:50 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:23:51.486 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T00:23:51.703 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:51.697+0000 7f73fb200640 1 -- 172.21.15.119:0/1679487802 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73fc074100 msgr2=0x7f73fc074580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:51.703 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:51.697+0000 7f73fb200640 1 --2- 172.21.15.119:0/1679487802 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73fc074100 0x7f73fc074580 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f73e8009930 tx=0x7f73e802bc30 comp rx=0 tx=0).stop 2024-03-28T00:23:51.703 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:51.697+0000 7f73fb200640 1 -- 172.21.15.119:0/1679487802 shutdown_connections 2024-03-28T00:23:51.703 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:51.697+0000 7f73fb200640 1 --2- 172.21.15.119:0/1679487802 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f73fc10dab0 0x7f73fc10fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:51.704 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:51.697+0000 7f73fb200640 1 --2- 172.21.15.119:0/1679487802 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73fc074100 0x7f73fc074580 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:51.704 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:51.697+0000 7f73fb200640 1 --2- 172.21.15.119:0/1679487802 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f73fc0756e0 0x7f73fc073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:51.704 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:51.697+0000 7f73fb200640 1 -- 172.21.15.119:0/1679487802 >> 172.21.15.119:0/1679487802 conn(0x7f73fc0fca30 msgr2=0x7f73fc0fee90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:23:51.704 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:51.697+0000 7f73fb200640 1 -- 172.21.15.119:0/1679487802 shutdown_connections 2024-03-28T00:23:51.704 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:51.697+0000 7f73fb200640 1 -- 172.21.15.119:0/1679487802 wait complete. 2024-03-28T00:23:51.707 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:51.698+0000 7f73fb200640 1 Processor -- start 2024-03-28T00:23:51.707 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:51.698+0000 7f73fb200640 1 -- start start 2024-03-28T00:23:51.707 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:51.699+0000 7f73fb200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f73fc074100 0x7f73fc19de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:23:51.707 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:51.699+0000 7f73fb200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f73fc0756e0 0x7f73fc19e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:23:51.707 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:51.699+0000 7f73fb200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73fc10dab0 0x7f73fc1a53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:23:51.707 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:51.699+0000 7f73fb200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73fc113780 con 0x7f73fc10dab0 2024-03-28T00:23:51.707 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:51.699+0000 7f73fb200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73fc113600 con 0x7f73fc074100 2024-03-28T00:23:51.707 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:51.699+0000 7f73fb200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73fc113900 con 0x7f73fc0756e0 2024-03-28T00:23:51.707 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:51.699+0000 7f73f9e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f73fc074100 0x7f73fc19de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:23:51.707 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:51.699+0000 7f73fa800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73fc10dab0 0x7f73fc1a53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:23:51.708 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:51.699+0000 7f73f9e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f73fc074100 0x7f73fc19de20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.119:51664/0 (socket says 172.21.15.119:51664) 2024-03-28T00:23:51.708 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:51.699+0000 7f73f9e00640 1 -- 172.21.15.119:0/3164192129 learned_addr learned my addr 172.21.15.119:0/3164192129 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:23:51.708 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:51.699+0000 7f73f9400640 1 --2- 172.21.15.119:0/3164192129 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f73fc0756e0 0x7f73fc19e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:23:51.708 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:51.699+0000 7f73f9e00640 1 -- 172.21.15.119:0/3164192129 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f73fc0756e0 msgr2=0x7f73fc19e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:51.708 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:51.699+0000 7f73f9e00640 1 --2- 172.21.15.119:0/3164192129 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f73fc0756e0 0x7f73fc19e360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:51.708 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:51.699+0000 7f73f9e00640 1 -- 172.21.15.119:0/3164192129 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73fc10dab0 msgr2=0x7f73fc1a53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:51.708 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:51.699+0000 7f73f9e00640 1 --2- 172.21.15.119:0/3164192129 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73fc10dab0 0x7f73fc1a53d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:51.708 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:51.699+0000 7f73f9e00640 1 -- 172.21.15.119:0/3164192129 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73fc1a5a40 con 0x7f73fc074100 2024-03-28T00:23:51.708 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:51.700+0000 7f73f9e00640 1 --2- 172.21.15.119:0/3164192129 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f73fc074100 0x7f73fc19de20 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f73ec00ea60 tx=0x7f73ec00ee60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:23:51.708 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:51.700+0000 7f73f2a00640 1 -- 172.21.15.119:0/3164192129 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73ec00cd20 con 0x7f73fc074100 2024-03-28T00:23:51.708 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:51.700+0000 7f73f2a00640 1 -- 172.21.15.119:0/3164192129 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f73ec0035f0 con 0x7f73fc074100 2024-03-28T00:23:51.708 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:51.700+0000 7f73fb200640 1 -- 172.21.15.119:0/3164192129 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73fc1a5d30 con 0x7f73fc074100 2024-03-28T00:23:51.708 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:51.700+0000 7f73f2a00640 1 -- 172.21.15.119:0/3164192129 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73ec010690 con 0x7f73fc074100 2024-03-28T00:23:51.709 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:51.700+0000 7f73fb200640 1 -- 172.21.15.119:0/3164192129 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73fc1a61a0 con 0x7f73fc074100 2024-03-28T00:23:51.709 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:51.701+0000 7f73fb200640 1 -- 172.21.15.119:0/3164192129 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73fc103c80 con 0x7f73fc074100 2024-03-28T00:23:51.709 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:51.703+0000 7f73f2a00640 1 -- 172.21.15.119:0/3164192129 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f73ec0037a0 con 0x7f73fc074100 2024-03-28T00:23:51.710 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:51.704+0000 7f73f2a00640 1 --2- 172.21.15.119:0/3164192129 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f73e00a2800 0x7f73e00a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:23:51.710 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:51.704+0000 7f73f2a00640 1 -- 172.21.15.119:0/3164192129 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f73ec0ce950 con 0x7f73fc074100 2024-03-28T00:23:51.711 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:51.705+0000 7f73f9400640 1 --2- 172.21.15.119:0/3164192129 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f73e00a2800 0x7f73e00a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:23:51.711 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:51.705+0000 7f73f9400640 1 --2- 172.21.15.119:0/3164192129 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f73e00a2800 0x7f73e00a4cc0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f73e8036040 tx=0x7f73e802df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:23:51.719 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:51.713+0000 7f73f2a00640 1 -- 172.21.15.119:0/3164192129 <== mon.2 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f73ec09bdf0 con 0x7f73fc074100 2024-03-28T00:23:51.758 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:51 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.132:6838/3305822525,v1:172.21.15.132:6839/3305822525] up:boot 2024-03-28T00:23:51.758 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:51 smithi132 ceph-mon[35941]: fsmap 12 up:standby 2024-03-28T00:23:51.759 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:51 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-28T00:23:51.764 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:51 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.132:6838/3305822525,v1:172.21.15.132:6839/3305822525] up:boot 2024-03-28T00:23:51.764 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:51 smithi165 ceph-mon[36302]: fsmap 12 up:standby 2024-03-28T00:23:51.764 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:51 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-28T00:23:51.850 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:51 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.132:6838/3305822525,v1:172.21.15.132:6839/3305822525] up:boot 2024-03-28T00:23:51.850 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:51 smithi119 ceph-mon[31518]: fsmap 12 up:standby 2024-03-28T00:23:51.850 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:51 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-28T00:23:51.964 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:51.958+0000 7f73fb200640 1 -- 172.21.15.119:0/3164192129 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f73fc02d070 con 0x7f73fc074100 2024-03-28T00:23:51.965 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:51.959+0000 7f73f2a00640 1 -- 172.21.15.119:0/3164192129 <== mon.2 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+10058 (secure 0 0 0) 0x7f73ec09b6f0 con 0x7f73fc074100 2024-03-28T00:23:51.966 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-28T00:23:51.966 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14400,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/3988635716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":3988635716},{"type":"v1","addr":"172.21.15.119:6837","nonce":3988635716}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6839/1595343240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1595343240},{"type":"v1","addr":"172.21.15.119:6839","nonce":1595343240}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14466,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6841/1678451188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6840","nonce":1678451188},{"type":"v1","addr":"172.21.15.119:6841","nonce":1678451188}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/609963168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":609963168},{"type":"v1","addr":"172.21.15.165:6833","nonce":609963168}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/1179016670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":1179016670},{"type":"v1","addr":"172.21.15.132:6833","nonce":1179016670}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24473,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/744950487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":744950487},{"type":"v1","addr":"172.21.15.119:6835","nonce":744950487}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24500,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/356981946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":356981946},{"type":"v1","addr":"172.21.15.132:6835","nonce":356981946}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24511,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6835/1947980102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":1947980102},{"type":"v1","addr":"172.21.15.165:6835","nonce":1947980102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24523,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/1531193032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":1531193032},{"type":"v1","addr":"172.21.15.165:6837","nonce":1531193032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24530,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6837/3383082967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6836","nonce":3383082967},{"type":"v1","addr":"172.21.15.132:6837","nonce":3383082967}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24550,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6839/3004684592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":3004684592},{"type":"v1","addr":"172.21.15.165:6839","nonce":3004684592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":24581,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6839/3305822525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6838","nonce":3305822525},{"type":"v1","addr":"172.21.15.132:6839","nonce":3305822525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[]} 2024-03-28T00:23:51.966 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 13 2024-03-28T00:23:51.969 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:51.962+0000 7f73fb200640 1 -- 172.21.15.119:0/3164192129 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f73e00a2800 msgr2=0x7f73e00a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:51.969 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:51.962+0000 7f73fb200640 1 --2- 172.21.15.119:0/3164192129 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f73e00a2800 0x7f73e00a4cc0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f73e8036040 tx=0x7f73e802df90 comp rx=0 tx=0).stop 2024-03-28T00:23:51.969 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:51.962+0000 7f73fb200640 1 -- 172.21.15.119:0/3164192129 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f73fc074100 msgr2=0x7f73fc19de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:51.969 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:51.962+0000 7f73fb200640 1 --2- 172.21.15.119:0/3164192129 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f73fc074100 0x7f73fc19de20 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f73ec00ea60 tx=0x7f73ec00ee60 comp rx=0 tx=0).stop 2024-03-28T00:23:51.969 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:51.962+0000 7f73fb200640 1 -- 172.21.15.119:0/3164192129 shutdown_connections 2024-03-28T00:23:51.969 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:51.962+0000 7f73fb200640 1 --2- 172.21.15.119:0/3164192129 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f73e00a2800 0x7f73e00a4cc0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:51.969 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:51.962+0000 7f73fb200640 1 --2- 172.21.15.119:0/3164192129 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73fc10dab0 0x7f73fc1a53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:51.969 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:51.962+0000 7f73fb200640 1 --2- 172.21.15.119:0/3164192129 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f73fc0756e0 0x7f73fc19e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:51.969 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:51.962+0000 7f73fb200640 1 --2- 172.21.15.119:0/3164192129 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f73fc074100 0x7f73fc19de20 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:51.969 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:51.962+0000 7f73fb200640 1 -- 172.21.15.119:0/3164192129 >> 172.21.15.119:0/3164192129 conn(0x7f73fc0fca30 msgr2=0x7f73fc100e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:23:51.969 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:51.962+0000 7f73fb200640 1 -- 172.21.15.119:0/3164192129 shutdown_connections 2024-03-28T00:23:51.969 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:51.963+0000 7f73fb200640 1 -- 172.21.15.119:0/3164192129 wait complete. 2024-03-28T00:23:51.988 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T00:23:52.183 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:52.177+0000 7fa7f5600640 1 -- 172.21.15.119:0/1291800653 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa7f01016c0 msgr2=0x7fa7f0101ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:52.184 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:52.177+0000 7fa7f5600640 1 --2- 172.21.15.119:0/1291800653 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa7f01016c0 0x7fa7f0101ae0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa7e0009900 tx=0x7fa7e002bc30 comp rx=0 tx=0).stop 2024-03-28T00:23:52.184 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:52.177+0000 7fa7f5600640 1 -- 172.21.15.119:0/1291800653 shutdown_connections 2024-03-28T00:23:52.184 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:52.177+0000 7fa7f5600640 1 --2- 172.21.15.119:0/1291800653 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa7f0103270 0x7fa7f0109b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:52.184 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:52.177+0000 7fa7f5600640 1 --2- 172.21.15.119:0/1291800653 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa7f01028b0 0x7fa7f0102d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:52.184 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:52.177+0000 7fa7f5600640 1 --2- 172.21.15.119:0/1291800653 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa7f01016c0 0x7fa7f0101ae0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:52.184 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:52.177+0000 7fa7f5600640 1 -- 172.21.15.119:0/1291800653 >> 172.21.15.119:0/1291800653 conn(0x7fa7f00fcc50 msgr2=0x7fa7f00ff070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:23:52.184 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:52.177+0000 7fa7f5600640 1 -- 172.21.15.119:0/1291800653 shutdown_connections 2024-03-28T00:23:52.184 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:52.177+0000 7fa7f5600640 1 -- 172.21.15.119:0/1291800653 wait complete. 2024-03-28T00:23:52.187 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:52.178+0000 7fa7f5600640 1 Processor -- start 2024-03-28T00:23:52.187 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:52.178+0000 7fa7f5600640 1 -- start start 2024-03-28T00:23:52.187 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:52.179+0000 7fa7f5600640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa7f01016c0 0x7fa7f0075450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:23:52.187 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:52.179+0000 7fa7f5600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa7f01028b0 0x7fa7f0073a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:23:52.187 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:52.179+0000 7fa7f5600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa7f0103270 0x7fa7f0073fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:23:52.187 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:52.179+0000 7fa7f5600640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7f010d310 con 0x7fa7f0103270 2024-03-28T00:23:52.187 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:52.179+0000 7fa7f5600640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7f010d190 con 0x7fa7f01016c0 2024-03-28T00:23:52.187 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:52.179+0000 7fa7f5600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7f010d490 con 0x7fa7f01028b0 2024-03-28T00:23:52.187 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:52.179+0000 7fa7eea00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa7f01016c0 0x7fa7f0075450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:23:52.188 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:52.179+0000 7fa7eea00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa7f01016c0 0x7fa7f0075450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.119:51690/0 (socket says 172.21.15.119:51690) 2024-03-28T00:23:52.188 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:52.179+0000 7fa7eea00640 1 -- 172.21.15.119:0/2813548843 learned_addr learned my addr 172.21.15.119:0/2813548843 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:23:52.188 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:52.179+0000 7fa7ee000640 1 --2- 172.21.15.119:0/2813548843 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa7f01028b0 0x7fa7f0073a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:23:52.188 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:52.179+0000 7fa7ef400640 1 --2- 172.21.15.119:0/2813548843 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa7f0103270 0x7fa7f0073fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:23:52.188 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:52.179+0000 7fa7eea00640 1 -- 172.21.15.119:0/2813548843 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa7f01028b0 msgr2=0x7fa7f0073a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:52.188 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:52.179+0000 7fa7eea00640 1 --2- 172.21.15.119:0/2813548843 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa7f01028b0 0x7fa7f0073a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:52.188 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:52.179+0000 7fa7eea00640 1 -- 172.21.15.119:0/2813548843 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa7f0103270 msgr2=0x7fa7f0073fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:52.188 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:52.179+0000 7fa7eea00640 1 --2- 172.21.15.119:0/2813548843 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa7f0103270 0x7fa7f0073fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:52.188 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:52.179+0000 7fa7eea00640 1 -- 172.21.15.119:0/2813548843 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7f0074860 con 0x7fa7f01016c0 2024-03-28T00:23:52.188 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:52.179+0000 7fa7ef400640 1 --2- 172.21.15.119:0/2813548843 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa7f0103270 0x7fa7f0073fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T00:23:52.188 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:52.180+0000 7fa7eea00640 1 --2- 172.21.15.119:0/2813548843 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa7f01016c0 0x7fa7f0075450 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa7e0000c00 tx=0x7fa7e0005d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:23:52.188 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:52.180+0000 7fa7e7400640 1 -- 172.21.15.119:0/2813548843 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7e0041070 con 0x7fa7f01016c0 2024-03-28T00:23:52.188 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:52.180+0000 7fa7e7400640 1 -- 172.21.15.119:0/2813548843 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa7e00044d0 con 0x7fa7f01016c0 2024-03-28T00:23:52.188 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:52.180+0000 7fa7e7400640 1 -- 172.21.15.119:0/2813548843 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7e0034440 con 0x7fa7f01016c0 2024-03-28T00:23:52.189 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:52.180+0000 7fa7f5600640 1 -- 172.21.15.119:0/2813548843 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7f01a1b00 con 0x7fa7f01016c0 2024-03-28T00:23:52.189 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:52.180+0000 7fa7f5600640 1 -- 172.21.15.119:0/2813548843 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7f01a20a0 con 0x7fa7f01016c0 2024-03-28T00:23:52.189 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:52.181+0000 7fa7f5600640 1 -- 172.21.15.119:0/2813548843 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7f0108a00 con 0x7fa7f01016c0 2024-03-28T00:23:52.189 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:52.183+0000 7fa7e7400640 1 -- 172.21.15.119:0/2813548843 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fa7e0044070 con 0x7fa7f01016c0 2024-03-28T00:23:52.190 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:52.184+0000 7fa7e7400640 1 --2- 172.21.15.119:0/2813548843 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fa7d40a2880 0x7fa7d40a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:23:52.190 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:52.184+0000 7fa7e7400640 1 -- 172.21.15.119:0/2813548843 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fa7e00f00e0 con 0x7fa7f01016c0 2024-03-28T00:23:52.191 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:52.185+0000 7fa7ee000640 1 --2- 172.21.15.119:0/2813548843 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fa7d40a2880 0x7fa7d40a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:23:52.191 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:52.185+0000 7fa7ee000640 1 --2- 172.21.15.119:0/2813548843 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fa7d40a2880 0x7fa7d40a4d40 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa7f00751c0 tx=0x7fa7d8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:23:52.194 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:52.188+0000 7fa7e7400640 1 -- 172.21.15.119:0/2813548843 <== mon.2 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa7e00bd620 con 0x7fa7f01016c0 2024-03-28T00:23:52.402 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:52.396+0000 7fa7f5600640 1 -- 172.21.15.119:0/2813548843 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa7f002d290 con 0x7fa7f01016c0 2024-03-28T00:23:52.403 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:52.397+0000 7fa7e7400640 1 -- 172.21.15.119:0/2813548843 <== mon.2 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+18537 (secure 0 0 0) 0x7fa7e00bcf20 con 0x7fa7f01016c0 2024-03-28T00:23:52.404 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-28T00:23:52.404 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":81,"fsid":"10f73b92-ec98-11ee-95d0-87774f69a715","created":"2024-03-28T00:14:57.563604+0000","modified":"2024-03-28T00:23:00.426768+0000","last_up_change":"2024-03-28T00:22:58.821662+0000","last_in_change":"2024-03-28T00:22:38.467270+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T00:18:46.253414+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c0b875d-a01e-4294-9c9c-5d0b048b8912","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":580396252},{"type":"v1","addr":"172.21.15.119:6803","nonce":580396252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":580396252},{"type":"v1","addr":"172.21.15.119:6805","nonce":580396252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":580396252},{"type":"v1","addr":"172.21.15.119:6809","nonce":580396252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":580396252},{"type":"v1","addr":"172.21.15.119:6807","nonce":580396252}]},"public_addr":"172.21.15.119:6803/580396252","cluster_addr":"172.21.15.119:6805/580396252","heartbeat_back_addr":"172.21.15.119:6809/580396252","heartbeat_front_addr":"172.21.15.119:6807/580396252","state":["exists","up"]},{"osd":1,"uuid":"5567f7c2-8ea4-4789-9ea1-467793324fb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":2500318888},{"type":"v1","addr":"172.21.15.132:6801","nonce":2500318888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":2500318888},{"type":"v1","addr":"172.21.15.132:6803","nonce":2500318888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":2500318888},{"type":"v1","addr":"172.21.15.132:6807","nonce":2500318888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":2500318888},{"type":"v1","addr":"172.21.15.132:6805","nonce":2500318888}]},"public_addr":"172.21.15.132:6801/2500318888","cluster_addr":"172.21.15.132:6803/2500318888","heartbeat_back_addr":"172.21.15.132:6807/2500318888","heartbeat_front_addr":"172.21.15.132:6805/2500318888","state":["exists","up"]},{"osd":2,"uuid":"ee54dbd0-24b4-4ee6-988c-ce71491199d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":2361434832},{"type":"v1","addr":"172.21.15.165:6801","nonce":2361434832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":2361434832},{"type":"v1","addr":"172.21.15.165:6803","nonce":2361434832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":2361434832},{"type":"v1","addr":"172.21.15.165:6807","nonce":2361434832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":2361434832},{"type":"v1","addr":"172.21.15.165:6805","nonce":2361434832}]},"public_addr":"172.21.15.165:6801/2361434832","cluster_addr":"172.21.15.165:6803/2361434832","heartbeat_back_addr":"172.21.15.165:6807/2361434832","heartbeat_front_addr":"172.21.15.165:6805/2361434832","state":["exists","up"]},{"osd":3,"uuid":"0464c40b-828f-4db4-96d9-3f7a6325e6da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":3732272210},{"type":"v1","addr":"172.21.15.119:6811","nonce":3732272210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":3732272210},{"type":"v1","addr":"172.21.15.119:6813","nonce":3732272210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":3732272210},{"type":"v1","addr":"172.21.15.119:6817","nonce":3732272210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":3732272210},{"type":"v1","addr":"172.21.15.119:6815","nonce":3732272210}]},"public_addr":"172.21.15.119:6811/3732272210","cluster_addr":"172.21.15.119:6813/3732272210","heartbeat_back_addr":"172.21.15.119:6817/3732272210","heartbeat_front_addr":"172.21.15.119:6815/3732272210","state":["exists","up"]},{"osd":4,"uuid":"8fd70b8e-5128-4173-aa0e-e735dfeb995d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":2803039505},{"type":"v1","addr":"172.21.15.132:6809","nonce":2803039505}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":2803039505},{"type":"v1","addr":"172.21.15.132:6811","nonce":2803039505}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":2803039505},{"type":"v1","addr":"172.21.15.132:6815","nonce":2803039505}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":2803039505},{"type":"v1","addr":"172.21.15.132:6813","nonce":2803039505}]},"public_addr":"172.21.15.132:6809/2803039505","cluster_addr":"172.21.15.132:6811/2803039505","heartbeat_back_addr":"172.21.15.132:6815/2803039505","heartbeat_front_addr":"172.21.15.132:6813/2803039505","state":["exists","up"]},{"osd":5,"uuid":"3224b4c1-733e-4424-a6da-870bbb5f94e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":2561252206},{"type":"v1","addr":"172.21.15.165:6809","nonce":2561252206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":2561252206},{"type":"v1","addr":"172.21.15.165:6811","nonce":2561252206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":2561252206},{"type":"v1","addr":"172.21.15.165:6815","nonce":2561252206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":2561252206},{"type":"v1","addr":"172.21.15.165:6813","nonce":2561252206}]},"public_addr":"172.21.15.165:6809/2561252206","cluster_addr":"172.21.15.165:6811/2561252206","heartbeat_back_addr":"172.21.15.165:6815/2561252206","heartbeat_front_addr":"172.21.15.165:6813/2561252206","state":["exists","up"]},{"osd":6,"uuid":"e435745d-59b5-439a-9091-0946ca0e9332","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":2545410230},{"type":"v1","addr":"172.21.15.119:6819","nonce":2545410230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":2545410230},{"type":"v1","addr":"172.21.15.119:6821","nonce":2545410230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":2545410230},{"type":"v1","addr":"172.21.15.119:6825","nonce":2545410230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":2545410230},{"type":"v1","addr":"172.21.15.119:6823","nonce":2545410230}]},"public_addr":"172.21.15.119:6819/2545410230","cluster_addr":"172.21.15.119:6821/2545410230","heartbeat_back_addr":"172.21.15.119:6825/2545410230","heartbeat_front_addr":"172.21.15.119:6823/2545410230","state":["exists","up"]},{"osd":7,"uuid":"79b97a12-f6b4-48c3-b517-2dd6b6e7c85f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":2877892376},{"type":"v1","addr":"172.21.15.132:6817","nonce":2877892376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":2877892376},{"type":"v1","addr":"172.21.15.132:6819","nonce":2877892376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":2877892376},{"type":"v1","addr":"172.21.15.132:6823","nonce":2877892376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":2877892376},{"type":"v1","addr":"172.21.15.132:6821","nonce":2877892376}]},"public_addr":"172.21.15.132:6817/2877892376","cluster_addr":"172.21.15.132:6819/2877892376","heartbeat_back_addr":"172.21.15.132:6823/2877892376","heartbeat_front_addr":"172.21.15.132:6821/2877892376","state":["exists","up"]},{"osd":8,"uuid":"2970524c-b99a-412d-85c2-6c29b54035e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":4115226150},{"type":"v1","addr":"172.21.15.165:6817","nonce":4115226150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":4115226150},{"type":"v1","addr":"172.21.15.165:6819","nonce":4115226150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":4115226150},{"type":"v1","addr":"172.21.15.165:6823","nonce":4115226150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":4115226150},{"type":"v1","addr":"172.21.15.165:6821","nonce":4115226150}]},"public_addr":"172.21.15.165:6817/4115226150","cluster_addr":"172.21.15.165:6819/4115226150","heartbeat_back_addr":"172.21.15.165:6823/4115226150","heartbeat_front_addr":"172.21.15.165:6821/4115226150","state":["exists","up"]},{"osd":9,"uuid":"2b1a08ed-cde9-4def-b3e8-838fff22709c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":2532245222},{"type":"v1","addr":"172.21.15.119:6827","nonce":2532245222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":2532245222},{"type":"v1","addr":"172.21.15.119:6829","nonce":2532245222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":2532245222},{"type":"v1","addr":"172.21.15.119:6833","nonce":2532245222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":2532245222},{"type":"v1","addr":"172.21.15.119:6831","nonce":2532245222}]},"public_addr":"172.21.15.119:6827/2532245222","cluster_addr":"172.21.15.119:6829/2532245222","heartbeat_back_addr":"172.21.15.119:6833/2532245222","heartbeat_front_addr":"172.21.15.119:6831/2532245222","state":["exists","up"]},{"osd":10,"uuid":"f019acd0-915a-4094-b12c-b53672740a5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":3454298492},{"type":"v1","addr":"172.21.15.132:6825","nonce":3454298492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":3454298492},{"type":"v1","addr":"172.21.15.132:6827","nonce":3454298492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":3454298492},{"type":"v1","addr":"172.21.15.132:6831","nonce":3454298492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":3454298492},{"type":"v1","addr":"172.21.15.132:6829","nonce":3454298492}]},"public_addr":"172.21.15.132:6825/3454298492","cluster_addr":"172.21.15.132:6827/3454298492","heartbeat_back_addr":"172.21.15.132:6831/3454298492","heartbeat_front_addr":"172.21.15.132:6829/3454298492","state":["exists","up"]},{"osd":11,"uuid":"2c1e59a8-8f21-45c0-8568-4b632ce020f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":2138771601},{"type":"v1","addr":"172.21.15.165:6825","nonce":2138771601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":2138771601},{"type":"v1","addr":"172.21.15.165:6827","nonce":2138771601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":2138771601},{"type":"v1","addr":"172.21.15.165:6831","nonce":2138771601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":2138771601},{"type":"v1","addr":"172.21.15.165:6829","nonce":2138771601}]},"public_addr":"172.21.15.165:6825/2138771601","cluster_addr":"172.21.15.165:6827/2138771601","heartbeat_back_addr":"172.21.15.165:6831/2138771601","heartbeat_front_addr":"172.21.15.165:6829/2138771601","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:17:54.791531+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:18:19.466638+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:18:43.435536+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:19:15.702685+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:19:41.767370+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:20:06.793003+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:20:39.952744+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:21:05.727262+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:21:30.735829+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:22:04.072142+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:22:30.894047+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:22:56.637728+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.119:6801/2575246519":"2024-03-29T00:16:03.582545+0000","172.21.15.119:6800/2575246519":"2024-03-29T00:16:03.582545+0000","172.21.15.119:0/87565965":"2024-03-29T00:16:03.582545+0000","172.21.15.119:6801/2056426384":"2024-03-29T00:15:30.472283+0000","172.21.15.119:0/1639109126":"2024-03-29T00:15:30.472283+0000","172.21.15.119:0/468478300":"2024-03-29T00:15:30.472283+0000","172.21.15.119:0/2891240357":"2024-03-29T00:15:30.472283+0000","172.21.15.119:0/4201699272":"2024-03-29T00:16:03.582545+0000","172.21.15.119:6800/2056426384":"2024-03-29T00:15:30.472283+0000","172.21.15.119:0/2635982178":"2024-03-29T00:16:03.582545+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T00:23:52.406 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:52.400+0000 7fa7f5600640 1 -- 172.21.15.119:0/2813548843 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fa7d40a2880 msgr2=0x7fa7d40a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:52.407 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:52.400+0000 7fa7f5600640 1 --2- 172.21.15.119:0/2813548843 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fa7d40a2880 0x7fa7d40a4d40 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa7f00751c0 tx=0x7fa7d8005f90 comp rx=0 tx=0).stop 2024-03-28T00:23:52.407 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:52.400+0000 7fa7f5600640 1 -- 172.21.15.119:0/2813548843 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa7f01016c0 msgr2=0x7fa7f0075450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:52.407 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:52.400+0000 7fa7f5600640 1 --2- 172.21.15.119:0/2813548843 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa7f01016c0 0x7fa7f0075450 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa7e0000c00 tx=0x7fa7e0005d50 comp rx=0 tx=0).stop 2024-03-28T00:23:52.407 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:52.400+0000 7fa7f5600640 1 -- 172.21.15.119:0/2813548843 shutdown_connections 2024-03-28T00:23:52.407 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:52.400+0000 7fa7f5600640 1 --2- 172.21.15.119:0/2813548843 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fa7d40a2880 0x7fa7d40a4d40 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:52.407 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:52.400+0000 7fa7f5600640 1 --2- 172.21.15.119:0/2813548843 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa7f0103270 0x7fa7f0073fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:52.407 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:52.400+0000 7fa7f5600640 1 --2- 172.21.15.119:0/2813548843 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa7f01028b0 0x7fa7f0073a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:52.407 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:52.400+0000 7fa7f5600640 1 --2- 172.21.15.119:0/2813548843 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa7f01016c0 0x7fa7f0075450 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:52.407 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:52.400+0000 7fa7f5600640 1 -- 172.21.15.119:0/2813548843 >> 172.21.15.119:0/2813548843 conn(0x7fa7f00fcc50 msgr2=0x7fa7f0105bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:23:52.407 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:52.400+0000 7fa7f5600640 1 -- 172.21.15.119:0/2813548843 shutdown_connections 2024-03-28T00:23:52.407 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:52.401+0000 7fa7f5600640 1 -- 172.21.15.119:0/2813548843 wait complete. 2024-03-28T00:23:52.426 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-28T00:18:46.253414+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-28T00:23:52.427 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-28T00:23:52.624 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:52.617+0000 7f45a3200640 1 -- 172.21.15.119:0/1944209861 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f45a41037a0 msgr2=0x7f45a4103bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:52.624 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:52.617+0000 7f45a3200640 1 --2- 172.21.15.119:0/1944209861 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f45a41037a0 0x7f45a4103bc0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f4594009900 tx=0x7f459402bc30 comp rx=0 tx=0).stop 2024-03-28T00:23:52.624 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:52.618+0000 7f45a3200640 1 -- 172.21.15.119:0/1944209861 shutdown_connections 2024-03-28T00:23:52.624 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:52.618+0000 7f45a3200640 1 --2- 172.21.15.119:0/1944209861 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f45a4105350 0x7f45a410bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:52.624 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:52.618+0000 7f45a3200640 1 --2- 172.21.15.119:0/1944209861 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f45a4104990 0x7f45a4104e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:52.624 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:52.618+0000 7f45a3200640 1 --2- 172.21.15.119:0/1944209861 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f45a41037a0 0x7f45a4103bc0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:52.625 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:52.618+0000 7f45a3200640 1 -- 172.21.15.119:0/1944209861 >> 172.21.15.119:0/1944209861 conn(0x7f45a40fed10 msgr2=0x7f45a4101150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:23:52.625 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:52.618+0000 7f45a3200640 1 -- 172.21.15.119:0/1944209861 shutdown_connections 2024-03-28T00:23:52.625 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:52.618+0000 7f45a3200640 1 -- 172.21.15.119:0/1944209861 wait complete. 2024-03-28T00:23:52.627 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:52.619+0000 7f45a3200640 1 Processor -- start 2024-03-28T00:23:52.627 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:52.619+0000 7f45a3200640 1 -- start start 2024-03-28T00:23:52.627 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:52.619+0000 7f45a3200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f45a41037a0 0x7f45a4199a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:23:52.627 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:52.619+0000 7f45a3200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f45a4104990 0x7f45a4199fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:23:52.627 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:52.619+0000 7f45a3200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f45a4105350 0x7f45a41a1040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:23:52.627 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:52.619+0000 7f45a3200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45a410f440 con 0x7f45a4105350 2024-03-28T00:23:52.627 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:52.619+0000 7f45a3200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45a410f2c0 con 0x7f45a41037a0 2024-03-28T00:23:52.627 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:52.619+0000 7f45a3200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45a410f5c0 con 0x7f45a4104990 2024-03-28T00:23:52.627 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:52.619+0000 7f45a2800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f45a4105350 0x7f45a41a1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:23:52.627 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:52.619+0000 7f45a2800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f45a4105350 0x7f45a41a1040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:36740/0 (socket says 172.21.15.119:36740) 2024-03-28T00:23:52.627 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:52.619+0000 7f45a2800640 1 -- 172.21.15.119:0/1576312932 learned_addr learned my addr 172.21.15.119:0/1576312932 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:23:52.627 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:52.620+0000 7f45a2800640 1 -- 172.21.15.119:0/1576312932 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f45a4104990 msgr2=0x7f45a4199fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:52.627 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:52.620+0000 7f45a2800640 1 --2- 172.21.15.119:0/1576312932 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f45a4104990 0x7f45a4199fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:52.628 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:52.620+0000 7f45a2800640 1 -- 172.21.15.119:0/1576312932 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f45a41037a0 msgr2=0x7f45a4199a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:52.628 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:52.620+0000 7f45a2800640 1 --2- 172.21.15.119:0/1576312932 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f45a41037a0 0x7f45a4199a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:52.628 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:52.620+0000 7f45a2800640 1 -- 172.21.15.119:0/1576312932 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45a41a16b0 con 0x7f45a4105350 2024-03-28T00:23:52.628 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:52.620+0000 7f45a2800640 1 --2- 172.21.15.119:0/1576312932 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f45a4105350 0x7f45a41a1040 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f459c00eb10 tx=0x7f459c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:23:52.628 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:52.620+0000 7f459aa00640 1 -- 172.21.15.119:0/1576312932 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f459c004b20 con 0x7f45a4105350 2024-03-28T00:23:52.628 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:52.620+0000 7f459aa00640 1 -- 172.21.15.119:0/1576312932 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f459c004cc0 con 0x7f45a4105350 2024-03-28T00:23:52.628 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:52.620+0000 7f45a3200640 1 -- 172.21.15.119:0/1576312932 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45a41a19a0 con 0x7f45a4105350 2024-03-28T00:23:52.628 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:52.620+0000 7f459aa00640 1 -- 172.21.15.119:0/1576312932 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f459c010760 con 0x7f45a4105350 2024-03-28T00:23:52.628 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:52.621+0000 7f45a3200640 1 -- 172.21.15.119:0/1576312932 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45a41a1ef0 con 0x7f45a4105350 2024-03-28T00:23:52.628 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:52.622+0000 7f45a3200640 1 -- 172.21.15.119:0/1576312932 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45a410aa60 con 0x7f45a4105350 2024-03-28T00:23:52.629 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:52.624+0000 7f459aa00640 1 -- 172.21.15.119:0/1576312932 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f459c005020 con 0x7f45a4105350 2024-03-28T00:23:52.631 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:52.625+0000 7f459aa00640 1 --2- 172.21.15.119:0/1576312932 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f45880a2880 0x7f45880a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:23:52.631 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:52.625+0000 7f459aa00640 1 -- 172.21.15.119:0/1576312932 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f459c0cec20 con 0x7f45a4105350 2024-03-28T00:23:52.631 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:52.625+0000 7f45a1e00640 1 --2- 172.21.15.119:0/1576312932 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f45880a2880 0x7f45880a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:23:52.632 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:52.626+0000 7f45a1e00640 1 --2- 172.21.15.119:0/1576312932 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f45880a2880 0x7f45880a4d40 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f4594036040 tx=0x7f4594000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:23:52.636 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:52.630+0000 7f459aa00640 1 -- 172.21.15.119:0/1576312932 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f459c09c160 con 0x7f45a4105350 2024-03-28T00:23:52.839 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:52.833+0000 7f45a3200640 1 -- 172.21.15.119:0/1576312932 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f45a402d2e0 con 0x7f45a4105350 2024-03-28T00:23:52.840 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:52.834+0000 7f459aa00640 1 -- 172.21.15.119:0/1576312932 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) v1 ==== 93+0+10 (secure 0 0 0) 0x7f459c09ba60 con 0x7f45a4105350 2024-03-28T00:23:52.840 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 1 2024-03-28T00:23:52.843 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:52.837+0000 7f45a3200640 1 -- 172.21.15.119:0/1576312932 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f45880a2880 msgr2=0x7f45880a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:52.843 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:52.837+0000 7f45a3200640 1 --2- 172.21.15.119:0/1576312932 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f45880a2880 0x7f45880a4d40 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f4594036040 tx=0x7f4594000bc0 comp rx=0 tx=0).stop 2024-03-28T00:23:52.844 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:52.837+0000 7f45a3200640 1 -- 172.21.15.119:0/1576312932 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f45a4105350 msgr2=0x7f45a41a1040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:52.844 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:52.837+0000 7f45a3200640 1 --2- 172.21.15.119:0/1576312932 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f45a4105350 0x7f45a41a1040 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f459c00eb10 tx=0x7f459c00c2d0 comp rx=0 tx=0).stop 2024-03-28T00:23:52.844 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:52.837+0000 7f45a3200640 1 -- 172.21.15.119:0/1576312932 shutdown_connections 2024-03-28T00:23:52.844 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:52.837+0000 7f45a3200640 1 --2- 172.21.15.119:0/1576312932 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f45880a2880 0x7f45880a4d40 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:52.844 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:52.837+0000 7f45a3200640 1 --2- 172.21.15.119:0/1576312932 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f45a4105350 0x7f45a41a1040 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:52.844 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:52.837+0000 7f45a3200640 1 --2- 172.21.15.119:0/1576312932 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f45a4104990 0x7f45a4199fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:52.844 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:52.837+0000 7f45a3200640 1 --2- 172.21.15.119:0/1576312932 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f45a41037a0 0x7f45a4199a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:52.844 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:52.837+0000 7f45a3200640 1 -- 172.21.15.119:0/1576312932 >> 172.21.15.119:0/1576312932 conn(0x7f45a40fed10 msgr2=0x7f45a4107c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:23:52.844 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:52.837+0000 7f45a3200640 1 -- 172.21.15.119:0/1576312932 shutdown_connections 2024-03-28T00:23:52.844 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:52.837+0000 7f45a3200640 1 -- 172.21.15.119:0/1576312932 wait complete. 2024-03-28T00:23:52.858 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:52 smithi119 ceph-mon[31518]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T00:23:52.858 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:52 smithi119 ceph-mon[31518]: from='client.? 172.21.15.119:0/3164192129' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T00:23:52.858 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:52 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:52.858 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:52 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:52.858 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:52 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:52.858 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:52 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:52.858 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:52 smithi119 ceph-mon[31518]: from='client.? 172.21.15.119:0/2813548843' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-28T00:23:52.861 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-28T00:23:52.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:52 smithi165 ceph-mon[36302]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T00:23:52.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:52 smithi165 ceph-mon[36302]: from='client.? 172.21.15.119:0/3164192129' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T00:23:52.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:52 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:52.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:52 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:52.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:52 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:52.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:52 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:52.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:52 smithi165 ceph-mon[36302]: from='client.? 172.21.15.119:0/2813548843' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-28T00:23:52.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:52 smithi132 ceph-mon[35941]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T00:23:52.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:52 smithi132 ceph-mon[35941]: from='client.? 172.21.15.119:0/3164192129' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T00:23:52.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:52 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:52.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:52 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:52.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:52 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:52.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:52 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:52.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:52 smithi132 ceph-mon[35941]: from='client.? 172.21.15.119:0/2813548843' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-28T00:23:53.054 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:53.048+0000 7fb72b200640 1 -- 172.21.15.119:0/3468260461 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb7241003f0 msgr2=0x7fb724100810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:53.054 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:53.048+0000 7fb72b200640 1 --2- 172.21.15.119:0/3468260461 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb7241003f0 0x7fb724100810 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb718009900 tx=0x7fb71802bc30 comp rx=0 tx=0).stop 2024-03-28T00:23:53.054 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:53.048+0000 7fb72b200640 1 -- 172.21.15.119:0/3468260461 shutdown_connections 2024-03-28T00:23:53.055 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:53.048+0000 7fb72b200640 1 --2- 172.21.15.119:0/3468260461 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb724101fa0 0x7fb724108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:53.055 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:53.048+0000 7fb72b200640 1 --2- 172.21.15.119:0/3468260461 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb7241015e0 0x7fb724101a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:53.055 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:53.048+0000 7fb72b200640 1 --2- 172.21.15.119:0/3468260461 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb7241003f0 0x7fb724100810 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:53.055 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:53.048+0000 7fb72b200640 1 -- 172.21.15.119:0/3468260461 >> 172.21.15.119:0/3468260461 conn(0x7fb7240fb980 msgr2=0x7fb7240fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:23:53.055 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:53.048+0000 7fb72b200640 1 -- 172.21.15.119:0/3468260461 shutdown_connections 2024-03-28T00:23:53.055 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:53.048+0000 7fb72b200640 1 -- 172.21.15.119:0/3468260461 wait complete. 2024-03-28T00:23:53.057 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:53.049+0000 7fb72b200640 1 Processor -- start 2024-03-28T00:23:53.057 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:53.049+0000 7fb72b200640 1 -- start start 2024-03-28T00:23:53.057 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:53.049+0000 7fb72b200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb7241003f0 0x7fb724199a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:23:53.057 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:53.049+0000 7fb72b200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb7241015e0 0x7fb724199f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:23:53.057 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:53.049+0000 7fb72b200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb724101fa0 0x7fb7241a0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:23:53.058 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:53.049+0000 7fb72b200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb72410c0d0 con 0x7fb7241003f0 2024-03-28T00:23:53.058 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:53.049+0000 7fb72b200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb72410bf50 con 0x7fb7241015e0 2024-03-28T00:23:53.058 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:53.049+0000 7fb72b200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb72410c250 con 0x7fb724101fa0 2024-03-28T00:23:53.058 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:53.050+0000 7fb729400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb7241015e0 0x7fb724199f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:23:53.058 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:53.050+0000 7fb729e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb7241003f0 0x7fb724199a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:23:53.058 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:53.050+0000 7fb72a800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb724101fa0 0x7fb7241a0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:23:53.058 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:53.050+0000 7fb729400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb7241015e0 0x7fb724199f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.119:51714/0 (socket says 172.21.15.119:51714) 2024-03-28T00:23:53.058 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:53.050+0000 7fb729400640 1 -- 172.21.15.119:0/145552554 learned_addr learned my addr 172.21.15.119:0/145552554 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:23:53.058 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:53.050+0000 7fb72a800640 1 --2- 172.21.15.119:0/145552554 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb724101fa0 0x7fb7241a0fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.119:38728/0 (socket says 172.21.15.119:38728) 2024-03-28T00:23:53.058 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:53.050+0000 7fb729400640 1 -- 172.21.15.119:0/145552554 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb724101fa0 msgr2=0x7fb7241a0fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:53.058 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:53.050+0000 7fb729400640 1 --2- 172.21.15.119:0/145552554 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb724101fa0 0x7fb7241a0fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:53.058 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:53.050+0000 7fb729400640 1 -- 172.21.15.119:0/145552554 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb7241003f0 msgr2=0x7fb724199a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:53.058 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:53.050+0000 7fb729400640 1 --2- 172.21.15.119:0/145552554 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb7241003f0 0x7fb724199a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:53.059 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:53.050+0000 7fb729400640 1 -- 172.21.15.119:0/145552554 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7241a1650 con 0x7fb7241015e0 2024-03-28T00:23:53.059 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:53.050+0000 7fb729400640 1 --2- 172.21.15.119:0/145552554 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb7241015e0 0x7fb724199f70 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb71400e9f0 tx=0x7fb71400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:23:53.059 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:53.051+0000 7fb71ea00640 1 -- 172.21.15.119:0/145552554 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb71400cd80 con 0x7fb7241015e0 2024-03-28T00:23:53.059 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:53.051+0000 7fb71ea00640 1 -- 172.21.15.119:0/145552554 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb714010910 con 0x7fb7241015e0 2024-03-28T00:23:53.059 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:53.051+0000 7fb71ea00640 1 -- 172.21.15.119:0/145552554 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb714018990 con 0x7fb7241015e0 2024-03-28T00:23:53.059 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:53.051+0000 7fb72b200640 1 -- 172.21.15.119:0/145552554 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7241a1940 con 0x7fb7241015e0 2024-03-28T00:23:53.059 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:53.051+0000 7fb72b200640 1 -- 172.21.15.119:0/145552554 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7241a1ee0 con 0x7fb7241015e0 2024-03-28T00:23:53.059 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:53.052+0000 7fb72b200640 1 -- 172.21.15.119:0/145552554 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7241a2110 con 0x7fb7241015e0 2024-03-28T00:23:53.060 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:53.055+0000 7fb71ea00640 1 -- 172.21.15.119:0/145552554 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fb714010ac0 con 0x7fb7241015e0 2024-03-28T00:23:53.062 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:53.056+0000 7fb71ea00640 1 --2- 172.21.15.119:0/145552554 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fb70c0a2880 0x7fb70c0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:23:53.062 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:53.056+0000 7fb71ea00640 1 -- 172.21.15.119:0/145552554 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fb7140ce870 con 0x7fb7241015e0 2024-03-28T00:23:53.062 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:53.056+0000 7fb729e00640 1 --2- 172.21.15.119:0/145552554 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fb70c0a2880 0x7fb70c0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:23:53.063 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:53.057+0000 7fb729e00640 1 --2- 172.21.15.119:0/145552554 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fb70c0a2880 0x7fb70c0a4d40 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb71802c170 tx=0x7fb71802df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:23:53.066 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:53.060+0000 7fb71ea00640 1 -- 172.21.15.119:0/145552554 <== mon.2 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fb71409b760 con 0x7fb7241015e0 2024-03-28T00:23:53.306 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:53.300+0000 7fb72b200640 1 -- 172.21.15.119:0/145552554 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fb7241a2380 con 0x7fb7241015e0 2024-03-28T00:23:53.306 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:53.301+0000 7fb71ea00640 1 -- 172.21.15.119:0/145552554 <== mon.2 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v81) v1 ==== 84+0+65 (secure 0 0 0) 0x7fb7140bb020 con 0x7fb7241015e0 2024-03-28T00:23:53.307 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-28T00:23:53.307 INFO:teuthology.orchestra.run.smithi119.stdout:[ 2024-03-28T00:23:53.307 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-03-28T00:23:53.307 INFO:teuthology.orchestra.run.smithi119.stdout: "poolnum": 1, 2024-03-28T00:23:53.307 INFO:teuthology.orchestra.run.smithi119.stdout: "poolname": ".mgr" 2024-03-28T00:23:53.307 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T00:23:53.307 INFO:teuthology.orchestra.run.smithi119.stdout:] 2024-03-28T00:23:53.309 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:53.303+0000 7fb72b200640 1 -- 172.21.15.119:0/145552554 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fb70c0a2880 msgr2=0x7fb70c0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:53.309 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:53.303+0000 7fb72b200640 1 --2- 172.21.15.119:0/145552554 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fb70c0a2880 0x7fb70c0a4d40 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb71802c170 tx=0x7fb71802df90 comp rx=0 tx=0).stop 2024-03-28T00:23:53.309 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:53.303+0000 7fb72b200640 1 -- 172.21.15.119:0/145552554 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb7241015e0 msgr2=0x7fb724199f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:53.310 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:53.303+0000 7fb72b200640 1 --2- 172.21.15.119:0/145552554 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb7241015e0 0x7fb724199f70 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb71400e9f0 tx=0x7fb71400eeb0 comp rx=0 tx=0).stop 2024-03-28T00:23:53.310 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:53.303+0000 7fb72b200640 1 -- 172.21.15.119:0/145552554 shutdown_connections 2024-03-28T00:23:53.310 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:53.303+0000 7fb72b200640 1 --2- 172.21.15.119:0/145552554 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fb70c0a2880 0x7fb70c0a4d40 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:53.310 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:53.303+0000 7fb72b200640 1 --2- 172.21.15.119:0/145552554 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb724101fa0 0x7fb7241a0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:53.310 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:53.303+0000 7fb72b200640 1 --2- 172.21.15.119:0/145552554 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb7241015e0 0x7fb724199f70 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:53.310 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:53.303+0000 7fb72b200640 1 --2- 172.21.15.119:0/145552554 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb7241003f0 0x7fb724199a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:53.310 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:53.303+0000 7fb72b200640 1 -- 172.21.15.119:0/145552554 >> 172.21.15.119:0/145552554 conn(0x7fb7240fb980 msgr2=0x7fb7240fd420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:23:53.310 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:53.303+0000 7fb72b200640 1 -- 172.21.15.119:0/145552554 shutdown_connections 2024-03-28T00:23:53.310 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:53.304+0000 7fb72b200640 1 -- 172.21.15.119:0/145552554 wait complete. 2024-03-28T00:23:53.326 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-28T00:23:53.327 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-28T00:23:53.512 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:53.505+0000 7f56e2400640 1 -- 172.21.15.119:0/942705210 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f56dc102000 msgr2=0x7f56dc1043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:53.513 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:53.505+0000 7f56e2400640 1 --2- 172.21.15.119:0/942705210 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f56dc102000 0x7f56dc1043f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f56d0009900 tx=0x7f56d002bc30 comp rx=0 tx=0).stop 2024-03-28T00:23:53.513 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:53.506+0000 7f56e2400640 1 -- 172.21.15.119:0/942705210 shutdown_connections 2024-03-28T00:23:53.513 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:53.506+0000 7f56e2400640 1 --2- 172.21.15.119:0/942705210 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f56dc107260 0x7f56dc1096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:53.513 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:53.506+0000 7f56e2400640 1 --2- 172.21.15.119:0/942705210 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f56dc104930 0x7f56dc106d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:53.513 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:53.506+0000 7f56e2400640 1 --2- 172.21.15.119:0/942705210 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f56dc102000 0x7f56dc1043f0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:53.513 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:53.506+0000 7f56e2400640 1 -- 172.21.15.119:0/942705210 >> 172.21.15.119:0/942705210 conn(0x7f56dc0fba10 msgr2=0x7f56dc0fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:23:53.513 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:53.506+0000 7f56e2400640 1 -- 172.21.15.119:0/942705210 shutdown_connections 2024-03-28T00:23:53.513 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:53.506+0000 7f56e2400640 1 -- 172.21.15.119:0/942705210 wait complete. 2024-03-28T00:23:53.515 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:53.507+0000 7f56e2400640 1 Processor -- start 2024-03-28T00:23:53.515 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:53.507+0000 7f56e2400640 1 -- start start 2024-03-28T00:23:53.515 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:53.507+0000 7f56e2400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f56dc102000 0x7f56dc199aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:23:53.515 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:53.507+0000 7f56e2400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f56dc104930 0x7f56dc199fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:23:53.515 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:53.507+0000 7f56e2400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f56dc107260 0x7f56dc1a1050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:23:53.515 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:53.507+0000 7f56e2400640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56dc10d210 con 0x7f56dc102000 2024-03-28T00:23:53.515 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:53.507+0000 7f56e2400640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56dc10d090 con 0x7f56dc104930 2024-03-28T00:23:53.516 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:53.507+0000 7f56e2400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56dc10d390 con 0x7f56dc107260 2024-03-28T00:23:53.516 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:53.507+0000 7f56e1000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f56dc102000 0x7f56dc199aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:23:53.516 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:53.507+0000 7f56e1000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f56dc102000 0x7f56dc199aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:36764/0 (socket says 172.21.15.119:36764) 2024-03-28T00:23:53.516 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:53.507+0000 7f56e1000640 1 -- 172.21.15.119:0/3220763246 learned_addr learned my addr 172.21.15.119:0/3220763246 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:23:53.516 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:53.507+0000 7f56e1a00640 1 --2- 172.21.15.119:0/3220763246 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f56dc107260 0x7f56dc1a1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:23:53.516 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:53.507+0000 7f56dbe00640 1 --2- 172.21.15.119:0/3220763246 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f56dc104930 0x7f56dc199fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:23:53.516 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:53.508+0000 7f56e1000640 1 -- 172.21.15.119:0/3220763246 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f56dc107260 msgr2=0x7f56dc1a1050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:53.516 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:53.508+0000 7f56e1000640 1 --2- 172.21.15.119:0/3220763246 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f56dc107260 0x7f56dc1a1050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:53.516 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:53.508+0000 7f56e1000640 1 -- 172.21.15.119:0/3220763246 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f56dc104930 msgr2=0x7f56dc199fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:53.516 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:53.508+0000 7f56e1000640 1 --2- 172.21.15.119:0/3220763246 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f56dc104930 0x7f56dc199fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:53.516 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:53.508+0000 7f56e1000640 1 -- 172.21.15.119:0/3220763246 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56dc1a16c0 con 0x7f56dc102000 2024-03-28T00:23:53.516 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:53.508+0000 7f56e1000640 1 --2- 172.21.15.119:0/3220763246 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f56dc102000 0x7f56dc199aa0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f56d0009230 tx=0x7f56d0005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:23:53.516 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:53.508+0000 7f56d9600640 1 -- 172.21.15.119:0/3220763246 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56d0041070 con 0x7f56dc102000 2024-03-28T00:23:53.516 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:53.508+0000 7f56d9600640 1 -- 172.21.15.119:0/3220763246 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f56d00054e0 con 0x7f56dc102000 2024-03-28T00:23:53.517 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:53.508+0000 7f56e2400640 1 -- 172.21.15.119:0/3220763246 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56dc1a1950 con 0x7f56dc102000 2024-03-28T00:23:53.517 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:53.508+0000 7f56d9600640 1 -- 172.21.15.119:0/3220763246 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56d0034440 con 0x7f56dc102000 2024-03-28T00:23:53.517 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:53.509+0000 7f56e2400640 1 -- 172.21.15.119:0/3220763246 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56dc1a1e40 con 0x7f56dc102000 2024-03-28T00:23:53.517 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:53.510+0000 7f56e2400640 1 -- 172.21.15.119:0/3220763246 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56dc108db0 con 0x7f56dc102000 2024-03-28T00:23:53.517 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:53.511+0000 7f56d9600640 1 -- 172.21.15.119:0/3220763246 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f56d0044070 con 0x7f56dc102000 2024-03-28T00:23:53.519 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:53.513+0000 7f56d9600640 1 --2- 172.21.15.119:0/3220763246 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f56c40a2800 0x7f56c40a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:23:53.519 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:53.513+0000 7f56d9600640 1 -- 172.21.15.119:0/3220763246 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f56d00f02c0 con 0x7f56dc102000 2024-03-28T00:23:53.519 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:53.513+0000 7f56dbe00640 1 --2- 172.21.15.119:0/3220763246 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f56c40a2800 0x7f56c40a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:23:53.519 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:53.513+0000 7f56dbe00640 1 --2- 172.21.15.119:0/3220763246 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f56c40a2800 0x7f56c40a4cc0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f56dc19afb0 tx=0x7f56c800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:23:53.522 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:53.517+0000 7f56d9600640 1 -- 172.21.15.119:0/3220763246 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f56d00bd110 con 0x7f56dc102000 2024-03-28T00:23:53.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:53 smithi119 ceph-mon[31518]: from='client.? 172.21.15.119:0/1576312932' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-28T00:23:53.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:53 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:53.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:53 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:53.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:53 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:23:53.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:53 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:23:53.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:53 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:53.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:53 smithi119 ceph-mon[31518]: from='client.? 172.21.15.119:0/145552554' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-03-28T00:23:53.727 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:53.721+0000 7f56e2400640 1 -- 172.21.15.119:0/3220763246 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f56dc0622a0 con 0x7f56dc102000 2024-03-28T00:23:53.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:53 smithi165 ceph-mon[36302]: from='client.? 172.21.15.119:0/1576312932' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-28T00:23:53.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:53 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:53.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:53 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:53.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:53 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:23:53.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:53 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:23:53.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:53 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:53.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:53 smithi165 ceph-mon[36302]: from='client.? 172.21.15.119:0/145552554' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-03-28T00:23:53.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:53 smithi132 ceph-mon[35941]: from='client.? 172.21.15.119:0/1576312932' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-28T00:23:53.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:53 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:53.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:53 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:53.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:53 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:23:53.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:53 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:23:53.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:53 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:53.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:53 smithi132 ceph-mon[35941]: from='client.? 172.21.15.119:0/145552554' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-03-28T00:23:54.627 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:54 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:23:54.627 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:54 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:54.627 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:54 smithi132 ceph-mon[35941]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T00:23:54.627 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:54 smithi132 ceph-mon[35941]: from='client.? 172.21.15.119:0/3220763246' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-28T00:23:54.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:54 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:23:54.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:54 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:54.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:54 smithi165 ceph-mon[36302]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T00:23:54.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:54 smithi165 ceph-mon[36302]: from='client.? 172.21.15.119:0/3220763246' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-28T00:23:54.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:54 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:23:54.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:54 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:54.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:54 smithi119 ceph-mon[31518]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T00:23:54.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:54 smithi119 ceph-mon[31518]: from='client.? 172.21.15.119:0/3220763246' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-28T00:23:54.709 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:54.703+0000 7f56d9600640 1 -- 172.21.15.119:0/3220763246 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v82) v1 ==== 138+0+0 (secure 0 0 0) 0x7f56d00c11b0 con 0x7f56dc102000 2024-03-28T00:23:54.709 INFO:teuthology.orchestra.run.smithi119.stderr:pool 'cephfs_metadata' created 2024-03-28T00:23:54.713 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:54.707+0000 7f56e2400640 1 -- 172.21.15.119:0/3220763246 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f56c40a2800 msgr2=0x7f56c40a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:54.713 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:54.707+0000 7f56e2400640 1 --2- 172.21.15.119:0/3220763246 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f56c40a2800 0x7f56c40a4cc0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f56dc19afb0 tx=0x7f56c800b390 comp rx=0 tx=0).stop 2024-03-28T00:23:54.713 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:54.707+0000 7f56e2400640 1 -- 172.21.15.119:0/3220763246 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f56dc102000 msgr2=0x7f56dc199aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:54.713 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:54.707+0000 7f56e2400640 1 --2- 172.21.15.119:0/3220763246 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f56dc102000 0x7f56dc199aa0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f56d0009230 tx=0x7f56d0005dd0 comp rx=0 tx=0).stop 2024-03-28T00:23:54.714 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:54.708+0000 7f56e2400640 1 -- 172.21.15.119:0/3220763246 shutdown_connections 2024-03-28T00:23:54.714 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:54.708+0000 7f56e2400640 1 --2- 172.21.15.119:0/3220763246 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f56c40a2800 0x7f56c40a4cc0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:54.714 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:54.708+0000 7f56e2400640 1 --2- 172.21.15.119:0/3220763246 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f56dc107260 0x7f56dc1a1050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:54.714 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:54.708+0000 7f56e2400640 1 --2- 172.21.15.119:0/3220763246 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f56dc104930 0x7f56dc199fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:54.714 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:54.708+0000 7f56e2400640 1 --2- 172.21.15.119:0/3220763246 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f56dc102000 0x7f56dc199aa0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:54.714 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:54.708+0000 7f56e2400640 1 -- 172.21.15.119:0/3220763246 >> 172.21.15.119:0/3220763246 conn(0x7f56dc0fba10 msgr2=0x7f56dc1017b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:23:54.714 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:54.708+0000 7f56e2400640 1 -- 172.21.15.119:0/3220763246 shutdown_connections 2024-03-28T00:23:54.715 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:54.709+0000 7f56e2400640 1 -- 172.21.15.119:0/3220763246 wait complete. 2024-03-28T00:23:54.738 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-03-28T00:23:54.930 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:54.924+0000 7f3ba5e00640 1 -- 172.21.15.119:0/1793793776 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ba0108eb0 msgr2=0x7f3ba0109330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:54.930 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:54.924+0000 7f3ba5e00640 1 --2- 172.21.15.119:0/1793793776 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ba0108eb0 0x7f3ba0109330 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f3b88009ca0 tx=0x7f3b8802c0f0 comp rx=0 tx=0).stop 2024-03-28T00:23:54.930 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:54.924+0000 7f3ba5e00640 1 -- 172.21.15.119:0/1793793776 shutdown_connections 2024-03-28T00:23:54.930 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:54.924+0000 7f3ba5e00640 1 --2- 172.21.15.119:0/1793793776 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3ba0109870 0x7f3ba0110100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:54.930 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:54.924+0000 7f3ba5e00640 1 --2- 172.21.15.119:0/1793793776 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ba0108eb0 0x7f3ba0109330 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:54.930 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:54.924+0000 7f3ba5e00640 1 --2- 172.21.15.119:0/1793793776 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3ba0107cc0 0x7f3ba01080e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:54.931 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:54.924+0000 7f3ba5e00640 1 -- 172.21.15.119:0/1793793776 >> 172.21.15.119:0/1793793776 conn(0x7f3ba0076c60 msgr2=0x7f3ba00790a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:23:54.931 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:54.924+0000 7f3ba5e00640 1 -- 172.21.15.119:0/1793793776 shutdown_connections 2024-03-28T00:23:54.931 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:54.924+0000 7f3ba5e00640 1 -- 172.21.15.119:0/1793793776 wait complete. 2024-03-28T00:23:54.933 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:54.925+0000 7f3ba5e00640 1 Processor -- start 2024-03-28T00:23:54.933 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:54.925+0000 7f3ba5e00640 1 -- start start 2024-03-28T00:23:54.933 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:54.925+0000 7f3ba5e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3ba0107cc0 0x7f3ba019dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:23:54.933 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:54.925+0000 7f3ba5e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ba0108eb0 0x7f3ba019e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:23:54.933 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:54.925+0000 7f3ba5e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3ba0109870 0x7f3ba01a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:23:54.933 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:54.925+0000 7f3ba5e00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ba0113920 con 0x7f3ba0108eb0 2024-03-28T00:23:54.933 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:54.925+0000 7f3ba5e00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ba01137a0 con 0x7f3ba0107cc0 2024-03-28T00:23:54.933 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:54.925+0000 7f3ba5e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ba0113aa0 con 0x7f3ba0109870 2024-03-28T00:23:54.933 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:54.925+0000 7f3b9ea00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3ba0107cc0 0x7f3ba019dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:23:54.933 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:54.925+0000 7f3b9f400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3ba0109870 0x7f3ba01a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:23:54.933 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:54.925+0000 7f3b9e000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ba0108eb0 0x7f3ba019e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:23:54.933 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:54.925+0000 7f3b9ea00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3ba0107cc0 0x7f3ba019dfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.119:36218/0 (socket says 172.21.15.119:36218) 2024-03-28T00:23:54.933 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:54.925+0000 7f3b9ea00640 1 -- 172.21.15.119:0/3290574878 learned_addr learned my addr 172.21.15.119:0/3290574878 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:23:54.934 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:54.925+0000 7f3b9f400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3ba0109870 0x7f3ba01a5550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.119:42482/0 (socket says 172.21.15.119:42482) 2024-03-28T00:23:54.934 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:54.926+0000 7f3b9e000640 1 -- 172.21.15.119:0/3290574878 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3ba0109870 msgr2=0x7f3ba01a5550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:54.934 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:54.926+0000 7f3b9e000640 1 --2- 172.21.15.119:0/3290574878 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3ba0109870 0x7f3ba01a5550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:54.934 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:54.926+0000 7f3b9e000640 1 -- 172.21.15.119:0/3290574878 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3ba0107cc0 msgr2=0x7f3ba019dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:54.934 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:54.926+0000 7f3b9e000640 1 --2- 172.21.15.119:0/3290574878 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3ba0107cc0 0x7f3ba019dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:54.934 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:54.926+0000 7f3b9e000640 1 -- 172.21.15.119:0/3290574878 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ba01a5bc0 con 0x7f3ba0108eb0 2024-03-28T00:23:54.934 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:54.926+0000 7f3b9e000640 1 --2- 172.21.15.119:0/3290574878 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ba0108eb0 0x7f3ba019e4e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f3b88009bf0 tx=0x7f3b88032ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:23:54.934 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:54.926+0000 7f3ba4a00640 1 -- 172.21.15.119:0/3290574878 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b88041020 con 0x7f3ba0108eb0 2024-03-28T00:23:54.934 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:54.926+0000 7f3ba4a00640 1 -- 172.21.15.119:0/3290574878 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3b88034cd0 con 0x7f3ba0108eb0 2024-03-28T00:23:54.934 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:54.927+0000 7f3ba5e00640 1 -- 172.21.15.119:0/3290574878 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ba01a5e50 con 0x7f3ba0108eb0 2024-03-28T00:23:54.934 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:54.927+0000 7f3ba4a00640 1 -- 172.21.15.119:0/3290574878 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b88039040 con 0x7f3ba0108eb0 2024-03-28T00:23:54.934 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:54.927+0000 7f3ba5e00640 1 -- 172.21.15.119:0/3290574878 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ba01a62b0 con 0x7f3ba0108eb0 2024-03-28T00:23:54.935 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:54.928+0000 7f3ba5e00640 1 -- 172.21.15.119:0/3290574878 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ba010efe0 con 0x7f3ba0108eb0 2024-03-28T00:23:54.936 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:54.931+0000 7f3ba4a00640 1 -- 172.21.15.119:0/3290574878 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f3b88044020 con 0x7f3ba0108eb0 2024-03-28T00:23:54.938 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:54.932+0000 7f3ba4a00640 1 --2- 172.21.15.119:0/3290574878 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f3b800a2800 0x7f3b800a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:23:54.938 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:54.932+0000 7f3ba4a00640 1 -- 172.21.15.119:0/3290574878 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7497+0+0 (secure 0 0 0) 0x7f3b880f0840 con 0x7f3ba0108eb0 2024-03-28T00:23:54.938 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:54.932+0000 7f3b9ea00640 1 --2- 172.21.15.119:0/3290574878 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f3b800a2800 0x7f3b800a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:23:54.938 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:54.933+0000 7f3b9ea00640 1 --2- 172.21.15.119:0/3290574878 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f3b800a2800 0x7f3b800a4cc0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f3b900095f0 tx=0x7f3b90005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:23:54.940 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:54.935+0000 7f3ba4a00640 1 -- 172.21.15.119:0/3290574878 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f3b8804b020 con 0x7f3ba0108eb0 2024-03-28T00:23:55.127 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:55.121+0000 7f3ba5e00640 1 -- 172.21.15.119:0/3290574878 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f3ba002d070 con 0x7f3ba0108eb0 2024-03-28T00:23:55.717 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:55.710+0000 7f3ba4a00640 1 -- 172.21.15.119:0/3290574878 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v83) v1 ==== 170+0+0 (secure 0 0 0) 0x7f3b880bdc00 con 0x7f3ba0108eb0 2024-03-28T00:23:55.717 INFO:teuthology.orchestra.run.smithi119.stderr:pool 'cephfs_data' created 2024-03-28T00:23:55.722 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:55.715+0000 7f3ba5e00640 1 -- 172.21.15.119:0/3290574878 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f3b800a2800 msgr2=0x7f3b800a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:55.722 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:55.715+0000 7f3ba5e00640 1 --2- 172.21.15.119:0/3290574878 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f3b800a2800 0x7f3b800a4cc0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f3b900095f0 tx=0x7f3b90005f90 comp rx=0 tx=0).stop 2024-03-28T00:23:55.722 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:55.716+0000 7f3ba5e00640 1 -- 172.21.15.119:0/3290574878 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ba0108eb0 msgr2=0x7f3ba019e4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:55.722 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:55.716+0000 7f3ba5e00640 1 --2- 172.21.15.119:0/3290574878 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ba0108eb0 0x7f3ba019e4e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f3b88009bf0 tx=0x7f3b88032ec0 comp rx=0 tx=0).stop 2024-03-28T00:23:55.724 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:55.716+0000 7f3ba5e00640 1 -- 172.21.15.119:0/3290574878 shutdown_connections 2024-03-28T00:23:55.724 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:55.716+0000 7f3ba5e00640 1 --2- 172.21.15.119:0/3290574878 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f3b800a2800 0x7f3b800a4cc0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:55.724 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:55.716+0000 7f3ba5e00640 1 --2- 172.21.15.119:0/3290574878 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3ba0109870 0x7f3ba01a5550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:55.724 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:55.716+0000 7f3ba5e00640 1 --2- 172.21.15.119:0/3290574878 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ba0108eb0 0x7f3ba019e4e0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:55.724 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:55.716+0000 7f3ba5e00640 1 --2- 172.21.15.119:0/3290574878 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3ba0107cc0 0x7f3ba019dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:55.724 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:55.716+0000 7f3ba5e00640 1 -- 172.21.15.119:0/3290574878 >> 172.21.15.119:0/3290574878 conn(0x7f3ba0076c60 msgr2=0x7f3ba010c1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:23:55.724 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:55.717+0000 7f3ba5e00640 1 -- 172.21.15.119:0/3290574878 shutdown_connections 2024-03-28T00:23:55.724 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:55.717+0000 7f3ba5e00640 1 -- 172.21.15.119:0/3290574878 wait complete. 2024-03-28T00:23:55.757 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-28T00:23:55.949 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:55.943+0000 7f74eca00640 1 -- 172.21.15.119:0/791057160 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f74e81037a0 msgr2=0x7f74e8103bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:55.949 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:55.943+0000 7f74eca00640 1 --2- 172.21.15.119:0/791057160 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f74e81037a0 0x7f74e8103bc0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f74d8009900 tx=0x7f74d802bc30 comp rx=0 tx=0).stop 2024-03-28T00:23:55.949 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:55.943+0000 7f74eca00640 1 -- 172.21.15.119:0/791057160 shutdown_connections 2024-03-28T00:23:55.949 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:55.943+0000 7f74eca00640 1 --2- 172.21.15.119:0/791057160 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f74e8105350 0x7f74e810bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:55.950 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:55.943+0000 7f74eca00640 1 --2- 172.21.15.119:0/791057160 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f74e8104990 0x7f74e8104e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:55.950 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:55.943+0000 7f74eca00640 1 --2- 172.21.15.119:0/791057160 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f74e81037a0 0x7f74e8103bc0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:55.950 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:55.943+0000 7f74eca00640 1 -- 172.21.15.119:0/791057160 >> 172.21.15.119:0/791057160 conn(0x7f74e80fed10 msgr2=0x7f74e8101150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:23:55.950 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:55.943+0000 7f74eca00640 1 -- 172.21.15.119:0/791057160 shutdown_connections 2024-03-28T00:23:55.950 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:55.943+0000 7f74eca00640 1 -- 172.21.15.119:0/791057160 wait complete. 2024-03-28T00:23:55.952 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:55.944+0000 7f74eca00640 1 Processor -- start 2024-03-28T00:23:55.952 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:55.944+0000 7f74eca00640 1 -- start start 2024-03-28T00:23:55.952 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:55.944+0000 7f74eca00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f74e81037a0 0x7f74e8199b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:23:55.952 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:55.944+0000 7f74eca00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f74e8104990 0x7f74e819a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:23:55.953 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:55.944+0000 7f74eca00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f74e8105350 0x7f74e81a10d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:23:55.953 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:55.944+0000 7f74eca00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74e810f470 con 0x7f74e8104990 2024-03-28T00:23:55.953 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:55.944+0000 7f74eca00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74e810f2f0 con 0x7f74e81037a0 2024-03-28T00:23:55.953 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:55.944+0000 7f74eca00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74e810f5f0 con 0x7f74e8105350 2024-03-28T00:23:55.953 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:55.945+0000 7f74e6a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f74e8104990 0x7f74e819a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:23:55.953 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:55.945+0000 7f74e7400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f74e81037a0 0x7f74e8199b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:23:55.953 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:55.945+0000 7f74e6a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f74e8104990 0x7f74e819a060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:56266/0 (socket says 172.21.15.119:56266) 2024-03-28T00:23:55.953 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:55.945+0000 7f74e6a00640 1 -- 172.21.15.119:0/3223120916 learned_addr learned my addr 172.21.15.119:0/3223120916 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:23:55.953 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:55.945+0000 7f74e7400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f74e81037a0 0x7f74e8199b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.119:36238/0 (socket says 172.21.15.119:36238) 2024-03-28T00:23:55.953 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:55.945+0000 7f74e7e00640 1 --2- 172.21.15.119:0/3223120916 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f74e8105350 0x7f74e81a10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:23:55.953 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:55.945+0000 7f74e7400640 1 -- 172.21.15.119:0/3223120916 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f74e8105350 msgr2=0x7f74e81a10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:55.954 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:55.945+0000 7f74e7400640 1 --2- 172.21.15.119:0/3223120916 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f74e8105350 0x7f74e81a10d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:55.954 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:55.945+0000 7f74e7400640 1 -- 172.21.15.119:0/3223120916 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f74e8104990 msgr2=0x7f74e819a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:55.954 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:55.945+0000 7f74e7400640 1 --2- 172.21.15.119:0/3223120916 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f74e8104990 0x7f74e819a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:55.954 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:55.945+0000 7f74e7400640 1 -- 172.21.15.119:0/3223120916 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74e81a1740 con 0x7f74e81037a0 2024-03-28T00:23:55.954 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:55.945+0000 7f74e6a00640 1 --2- 172.21.15.119:0/3223120916 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f74e8104990 0x7f74e819a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T00:23:55.954 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:55.945+0000 7f74e7400640 1 --2- 172.21.15.119:0/3223120916 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f74e81037a0 0x7f74e8199b20 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f74d8036040 tx=0x7f74d802c650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:23:55.954 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:55.946+0000 7f74dfe00640 1 -- 172.21.15.119:0/3223120916 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74d8043020 con 0x7f74e81037a0 2024-03-28T00:23:55.954 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:55.946+0000 7f74eca00640 1 -- 172.21.15.119:0/3223120916 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74e81a19d0 con 0x7f74e81037a0 2024-03-28T00:23:55.954 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:55.946+0000 7f74dfe00640 1 -- 172.21.15.119:0/3223120916 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f74d80054e0 con 0x7f74e81037a0 2024-03-28T00:23:55.954 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:55.946+0000 7f74dfe00640 1 -- 172.21.15.119:0/3223120916 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74d8003c10 con 0x7f74e81037a0 2024-03-28T00:23:55.954 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:55.946+0000 7f74eca00640 1 -- 172.21.15.119:0/3223120916 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74e81a1f40 con 0x7f74e81037a0 2024-03-28T00:23:55.955 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:55.947+0000 7f74eca00640 1 -- 172.21.15.119:0/3223120916 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74e8068800 con 0x7f74e81037a0 2024-03-28T00:23:55.955 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:55.949+0000 7f74dfe00640 1 -- 172.21.15.119:0/3223120916 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f74d8046020 con 0x7f74e81037a0 2024-03-28T00:23:55.956 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:55.950+0000 7f74dfe00640 1 --2- 172.21.15.119:0/3223120916 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f74cc0a2800 0x7f74cc0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:23:55.956 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:55.950+0000 7f74dfe00640 1 -- 172.21.15.119:0/3223120916 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7879+0+0 (secure 0 0 0) 0x7f74d80f0f00 con 0x7f74e81037a0 2024-03-28T00:23:55.956 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:55.950+0000 7f74e6a00640 1 --2- 172.21.15.119:0/3223120916 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f74cc0a2800 0x7f74cc0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:23:55.957 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:55.951+0000 7f74e6a00640 1 --2- 172.21.15.119:0/3223120916 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f74cc0a2800 0x7f74cc0a4cc0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f74e819b030 tx=0x7f74d4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:23:55.958 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:55.952+0000 7f74dfe00640 1 -- 172.21.15.119:0/3223120916 <== mon.2 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f74d804d020 con 0x7f74e81037a0 2024-03-28T00:23:56.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:55 smithi165 ceph-mon[36302]: from='client.? 172.21.15.119:0/3220763246' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-03-28T00:23:56.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:55 smithi165 ceph-mon[36302]: osdmap e82: 12 total, 12 up, 12 in 2024-03-28T00:23:56.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:55 smithi165 ceph-mon[36302]: from='client.? 172.21.15.119:0/3290574878' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-28T00:23:56.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:55 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:56.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:55 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:56.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:55 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:56.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:55 smithi119 ceph-mon[31518]: from='client.? 172.21.15.119:0/3220763246' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-03-28T00:23:56.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:55 smithi119 ceph-mon[31518]: osdmap e82: 12 total, 12 up, 12 in 2024-03-28T00:23:56.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:55 smithi119 ceph-mon[31518]: from='client.? 172.21.15.119:0/3290574878' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-28T00:23:56.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:55 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:56.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:55 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:56.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:55 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:56.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:55 smithi132 ceph-mon[35941]: from='client.? 172.21.15.119:0/3220763246' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-03-28T00:23:56.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:55 smithi132 ceph-mon[35941]: osdmap e82: 12 total, 12 up, 12 in 2024-03-28T00:23:56.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:55 smithi132 ceph-mon[35941]: from='client.? 172.21.15.119:0/3290574878' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-28T00:23:56.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:55 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:56.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:55 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:56.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:55 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:56.218 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:56.212+0000 7f74eca00640 1 -- 172.21.15.119:0/3223120916 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f74e810aa60 con 0x7f74e81037a0 2024-03-28T00:23:57.081 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:57.074+0000 7f74dfe00640 1 -- 172.21.15.119:0/3223120916 <== mon.2 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) v1 ==== 172+0+0 (secure 0 0 0) 0x7f74d80be0b0 con 0x7f74e81037a0 2024-03-28T00:23:57.081 INFO:teuthology.orchestra.run.smithi119.stderr:new fs with metadata pool 2 and data pool 3 2024-03-28T00:23:57.088 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:57.081+0000 7f74eca00640 1 -- 172.21.15.119:0/3223120916 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f74cc0a2800 msgr2=0x7f74cc0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:57.088 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:57.082+0000 7f74eca00640 1 --2- 172.21.15.119:0/3223120916 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f74cc0a2800 0x7f74cc0a4cc0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f74e819b030 tx=0x7f74d4005f90 comp rx=0 tx=0).stop 2024-03-28T00:23:57.088 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:57.082+0000 7f74eca00640 1 -- 172.21.15.119:0/3223120916 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f74e81037a0 msgr2=0x7f74e8199b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:57.088 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:57.082+0000 7f74eca00640 1 --2- 172.21.15.119:0/3223120916 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f74e81037a0 0x7f74e8199b20 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f74d8036040 tx=0x7f74d802c650 comp rx=0 tx=0).stop 2024-03-28T00:23:57.088 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:57.082+0000 7f74eca00640 1 -- 172.21.15.119:0/3223120916 shutdown_connections 2024-03-28T00:23:57.088 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:57.082+0000 7f74eca00640 1 --2- 172.21.15.119:0/3223120916 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f74cc0a2800 0x7f74cc0a4cc0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:57.088 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:57.082+0000 7f74eca00640 1 --2- 172.21.15.119:0/3223120916 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f74e8105350 0x7f74e81a10d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:57.088 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:57.082+0000 7f74eca00640 1 --2- 172.21.15.119:0/3223120916 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f74e8104990 0x7f74e819a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:57.088 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:57.082+0000 7f74eca00640 1 --2- 172.21.15.119:0/3223120916 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f74e81037a0 0x7f74e8199b20 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:57.089 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:57.082+0000 7f74eca00640 1 -- 172.21.15.119:0/3223120916 >> 172.21.15.119:0/3223120916 conn(0x7f74e80fed10 msgr2=0x7f74e8107c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:23:57.089 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:57.083+0000 7f74eca00640 1 -- 172.21.15.119:0/3223120916 shutdown_connections 2024-03-28T00:23:57.089 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:57.083+0000 7f74eca00640 1 -- 172.21.15.119:0/3223120916 wait complete. 2024-03-28T00:23:57.104 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:56 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:57.104 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:56 smithi132 ceph-mon[35941]: from='client.? 172.21.15.119:0/3290574878' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-03-28T00:23:57.104 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:56 smithi132 ceph-mon[35941]: osdmap e83: 12 total, 12 up, 12 in 2024-03-28T00:23:57.104 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:56 smithi132 ceph-mon[35941]: pgmap v309: 97 pgs: 96 unknown, 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T00:23:57.104 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:56 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:57.104 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:56 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:57.104 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:56 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:23:57.104 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:56 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:23:57.105 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:56 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:57.105 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:56 smithi132 ceph-mon[35941]: from='client.? 172.21.15.119:0/3223120916' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-28T00:23:57.105 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:56 smithi132 ceph-mon[35941]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-28T00:23:57.105 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:56 smithi132 ceph-mon[35941]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-03-28T00:23:57.108 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:56 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:57.108 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:56 smithi119 ceph-mon[31518]: from='client.? 172.21.15.119:0/3290574878' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-03-28T00:23:57.109 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:56 smithi119 ceph-mon[31518]: osdmap e83: 12 total, 12 up, 12 in 2024-03-28T00:23:57.109 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:56 smithi119 ceph-mon[31518]: pgmap v309: 97 pgs: 96 unknown, 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T00:23:57.109 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:56 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:57.109 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:56 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:57.109 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:56 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:23:57.109 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:56 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:23:57.109 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:56 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:57.109 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:56 smithi119 ceph-mon[31518]: from='client.? 172.21.15.119:0/3223120916' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-28T00:23:57.109 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:56 smithi119 ceph-mon[31518]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-28T00:23:57.109 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:56 smithi119 ceph-mon[31518]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-03-28T00:23:57.111 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T00:23:57.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:56 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:57.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:56 smithi165 ceph-mon[36302]: from='client.? 172.21.15.119:0/3290574878' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-03-28T00:23:57.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:56 smithi165 ceph-mon[36302]: osdmap e83: 12 total, 12 up, 12 in 2024-03-28T00:23:57.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:56 smithi165 ceph-mon[36302]: pgmap v309: 97 pgs: 96 unknown, 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T00:23:57.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:56 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:57.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:56 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:57.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:56 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:23:57.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:56 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:23:57.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:56 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:23:57.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:56 smithi165 ceph-mon[36302]: from='client.? 172.21.15.119:0/3223120916' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-28T00:23:57.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:56 smithi165 ceph-mon[36302]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-28T00:23:57.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:56 smithi165 ceph-mon[36302]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-03-28T00:23:57.308 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:57.301+0000 7fb1b3200640 1 -- 172.21.15.119:0/3550930990 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb1b40772a0 msgr2=0x7fb1b4077700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:57.308 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:57.301+0000 7fb1b3200640 1 --2- 172.21.15.119:0/3550930990 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb1b40772a0 0x7fb1b4077700 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb1a4009900 tx=0x7fb1a402bc30 comp rx=0 tx=0).stop 2024-03-28T00:23:57.308 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:57.301+0000 7fb1b3200640 1 -- 172.21.15.119:0/3550930990 shutdown_connections 2024-03-28T00:23:57.308 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:57.301+0000 7fb1b3200640 1 --2- 172.21.15.119:0/3550930990 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb1b4077c40 0x7fb1b40784e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:57.308 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:57.301+0000 7fb1b3200640 1 --2- 172.21.15.119:0/3550930990 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb1b40772a0 0x7fb1b4077700 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:57.308 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:57.301+0000 7fb1b3200640 1 --2- 172.21.15.119:0/3550930990 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb1b4076060 0x7fb1b4076480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:57.308 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:57.301+0000 7fb1b3200640 1 -- 172.21.15.119:0/3550930990 >> 172.21.15.119:0/3550930990 conn(0x7fb1b40fec80 msgr2=0x7fb1b41010a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:23:57.308 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:57.302+0000 7fb1b3200640 1 -- 172.21.15.119:0/3550930990 shutdown_connections 2024-03-28T00:23:57.308 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:57.302+0000 7fb1b3200640 1 -- 172.21.15.119:0/3550930990 wait complete. 2024-03-28T00:23:57.311 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:57.302+0000 7fb1b3200640 1 Processor -- start 2024-03-28T00:23:57.311 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:57.303+0000 7fb1b3200640 1 -- start start 2024-03-28T00:23:57.311 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:57.303+0000 7fb1b3200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb1b4076060 0x7fb1b419ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:23:57.311 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:57.303+0000 7fb1b3200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb1b40772a0 0x7fb1b419e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:23:57.311 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:57.303+0000 7fb1b3200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb1b4077c40 0x7fb1b41a5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:23:57.311 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:57.303+0000 7fb1b3200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1b41137c0 con 0x7fb1b4076060 2024-03-28T00:23:57.311 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:57.303+0000 7fb1b3200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1b4113640 con 0x7fb1b4077c40 2024-03-28T00:23:57.311 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:57.303+0000 7fb1b3200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1b4113940 con 0x7fb1b40772a0 2024-03-28T00:23:57.311 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:57.303+0000 7fb1b1e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb1b4076060 0x7fb1b419ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:23:57.311 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:57.303+0000 7fb1b2800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb1b4077c40 0x7fb1b41a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:23:57.311 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:57.303+0000 7fb1b1e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb1b4076060 0x7fb1b419ddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:56290/0 (socket says 172.21.15.119:56290) 2024-03-28T00:23:57.312 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:57.303+0000 7fb1b1e00640 1 -- 172.21.15.119:0/3129309904 learned_addr learned my addr 172.21.15.119:0/3129309904 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:23:57.312 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:57.303+0000 7fb1b1400640 1 --2- 172.21.15.119:0/3129309904 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb1b40772a0 0x7fb1b419e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:23:57.312 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:57.304+0000 7fb1b2800640 1 -- 172.21.15.119:0/3129309904 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb1b40772a0 msgr2=0x7fb1b419e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:57.312 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:57.304+0000 7fb1b2800640 1 --2- 172.21.15.119:0/3129309904 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb1b40772a0 0x7fb1b419e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:57.312 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:57.304+0000 7fb1b2800640 1 -- 172.21.15.119:0/3129309904 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb1b4076060 msgr2=0x7fb1b419ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:57.312 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:57.304+0000 7fb1b2800640 1 --2- 172.21.15.119:0/3129309904 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb1b4076060 0x7fb1b419ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:57.312 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:57.304+0000 7fb1b2800640 1 -- 172.21.15.119:0/3129309904 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1b41a59f0 con 0x7fb1b4077c40 2024-03-28T00:23:57.312 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:57.304+0000 7fb1b1e00640 1 --2- 172.21.15.119:0/3129309904 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb1b4076060 0x7fb1b419ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T00:23:57.312 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:57.304+0000 7fb1b2800640 1 --2- 172.21.15.119:0/3129309904 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb1b4077c40 0x7fb1b41a5380 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb1ac009fd0 tx=0x7fb1ac00ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:23:57.312 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:57.304+0000 7fb1aaa00640 1 -- 172.21.15.119:0/3129309904 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1ac00ce20 con 0x7fb1b4077c40 2024-03-28T00:23:57.312 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:57.304+0000 7fb1b3200640 1 -- 172.21.15.119:0/3129309904 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1b41a5ce0 con 0x7fb1b4077c40 2024-03-28T00:23:57.312 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:57.304+0000 7fb1aaa00640 1 -- 172.21.15.119:0/3129309904 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb1ac005020 con 0x7fb1b4077c40 2024-03-28T00:23:57.312 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:57.304+0000 7fb1aaa00640 1 -- 172.21.15.119:0/3129309904 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1ac010720 con 0x7fb1b4077c40 2024-03-28T00:23:57.312 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:57.304+0000 7fb1b3200640 1 -- 172.21.15.119:0/3129309904 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1b41a61b0 con 0x7fb1b4077c40 2024-03-28T00:23:57.313 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:57.305+0000 7fb1b3200640 1 -- 172.21.15.119:0/3129309904 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1b4110060 con 0x7fb1b4077c40 2024-03-28T00:23:57.313 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:57.307+0000 7fb1aaa00640 1 -- 172.21.15.119:0/3129309904 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fb1ac005540 con 0x7fb1b4077c40 2024-03-28T00:23:57.315 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:57.309+0000 7fb1aaa00640 1 --2- 172.21.15.119:0/3129309904 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fb1980a2880 0x7fb1980a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:23:57.315 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:57.309+0000 7fb1aaa00640 1 -- 172.21.15.119:0/3129309904 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fb1ac0cf370 con 0x7fb1b4077c40 2024-03-28T00:23:57.315 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:57.309+0000 7fb1b1e00640 1 --2- 172.21.15.119:0/3129309904 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fb1980a2880 0x7fb1980a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:23:57.315 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:57.310+0000 7fb1b1e00640 1 --2- 172.21.15.119:0/3129309904 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fb1980a2880 0x7fb1980a4d40 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb19c011550 tx=0x7fb19c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:23:57.317 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:57.311+0000 7fb1aaa00640 1 -- 172.21.15.119:0/3129309904 <== mon.2 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fb1ac09c4c0 con 0x7fb1b4077c40 2024-03-28T00:23:57.519 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:57.513+0000 7fb1b3200640 1 -- 172.21.15.119:0/3129309904 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb1b402d2e0 con 0x7fb1b4077c40 2024-03-28T00:23:57.521 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:57.515+0000 7fb1aaa00640 1 -- 172.21.15.119:0/3129309904 <== mon.2 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+22313 (secure 0 0 0) 0x7fb1ac09bdc0 con 0x7fb1b4077c40 2024-03-28T00:23:57.522 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-28T00:23:57.522 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":84,"fsid":"10f73b92-ec98-11ee-95d0-87774f69a715","created":"2024-03-28T00:14:57.563604+0000","modified":"2024-03-28T00:23:56.214836+0000","last_up_change":"2024-03-28T00:22:58.821662+0000","last_in_change":"2024-03-28T00:22:38.467270+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T00:18:46.253414+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-28T00:23:54.261981+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-28T00:23:55.685836+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c0b875d-a01e-4294-9c9c-5d0b048b8912","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":580396252},{"type":"v1","addr":"172.21.15.119:6803","nonce":580396252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":580396252},{"type":"v1","addr":"172.21.15.119:6805","nonce":580396252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":580396252},{"type":"v1","addr":"172.21.15.119:6809","nonce":580396252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":580396252},{"type":"v1","addr":"172.21.15.119:6807","nonce":580396252}]},"public_addr":"172.21.15.119:6803/580396252","cluster_addr":"172.21.15.119:6805/580396252","heartbeat_back_addr":"172.21.15.119:6809/580396252","heartbeat_front_addr":"172.21.15.119:6807/580396252","state":["exists","up"]},{"osd":1,"uuid":"5567f7c2-8ea4-4789-9ea1-467793324fb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":2500318888},{"type":"v1","addr":"172.21.15.132:6801","nonce":2500318888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":2500318888},{"type":"v1","addr":"172.21.15.132:6803","nonce":2500318888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":2500318888},{"type":"v1","addr":"172.21.15.132:6807","nonce":2500318888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":2500318888},{"type":"v1","addr":"172.21.15.132:6805","nonce":2500318888}]},"public_addr":"172.21.15.132:6801/2500318888","cluster_addr":"172.21.15.132:6803/2500318888","heartbeat_back_addr":"172.21.15.132:6807/2500318888","heartbeat_front_addr":"172.21.15.132:6805/2500318888","state":["exists","up"]},{"osd":2,"uuid":"ee54dbd0-24b4-4ee6-988c-ce71491199d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":2361434832},{"type":"v1","addr":"172.21.15.165:6801","nonce":2361434832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":2361434832},{"type":"v1","addr":"172.21.15.165:6803","nonce":2361434832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":2361434832},{"type":"v1","addr":"172.21.15.165:6807","nonce":2361434832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":2361434832},{"type":"v1","addr":"172.21.15.165:6805","nonce":2361434832}]},"public_addr":"172.21.15.165:6801/2361434832","cluster_addr":"172.21.15.165:6803/2361434832","heartbeat_back_addr":"172.21.15.165:6807/2361434832","heartbeat_front_addr":"172.21.15.165:6805/2361434832","state":["exists","up"]},{"osd":3,"uuid":"0464c40b-828f-4db4-96d9-3f7a6325e6da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":3732272210},{"type":"v1","addr":"172.21.15.119:6811","nonce":3732272210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":3732272210},{"type":"v1","addr":"172.21.15.119:6813","nonce":3732272210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":3732272210},{"type":"v1","addr":"172.21.15.119:6817","nonce":3732272210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":3732272210},{"type":"v1","addr":"172.21.15.119:6815","nonce":3732272210}]},"public_addr":"172.21.15.119:6811/3732272210","cluster_addr":"172.21.15.119:6813/3732272210","heartbeat_back_addr":"172.21.15.119:6817/3732272210","heartbeat_front_addr":"172.21.15.119:6815/3732272210","state":["exists","up"]},{"osd":4,"uuid":"8fd70b8e-5128-4173-aa0e-e735dfeb995d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":2803039505},{"type":"v1","addr":"172.21.15.132:6809","nonce":2803039505}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":2803039505},{"type":"v1","addr":"172.21.15.132:6811","nonce":2803039505}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":2803039505},{"type":"v1","addr":"172.21.15.132:6815","nonce":2803039505}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":2803039505},{"type":"v1","addr":"172.21.15.132:6813","nonce":2803039505}]},"public_addr":"172.21.15.132:6809/2803039505","cluster_addr":"172.21.15.132:6811/2803039505","heartbeat_back_addr":"172.21.15.132:6815/2803039505","heartbeat_front_addr":"172.21.15.132:6813/2803039505","state":["exists","up"]},{"osd":5,"uuid":"3224b4c1-733e-4424-a6da-870bbb5f94e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":2561252206},{"type":"v1","addr":"172.21.15.165:6809","nonce":2561252206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":2561252206},{"type":"v1","addr":"172.21.15.165:6811","nonce":2561252206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":2561252206},{"type":"v1","addr":"172.21.15.165:6815","nonce":2561252206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":2561252206},{"type":"v1","addr":"172.21.15.165:6813","nonce":2561252206}]},"public_addr":"172.21.15.165:6809/2561252206","cluster_addr":"172.21.15.165:6811/2561252206","heartbeat_back_addr":"172.21.15.165:6815/2561252206","heartbeat_front_addr":"172.21.15.165:6813/2561252206","state":["exists","up"]},{"osd":6,"uuid":"e435745d-59b5-439a-9091-0946ca0e9332","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":2545410230},{"type":"v1","addr":"172.21.15.119:6819","nonce":2545410230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":2545410230},{"type":"v1","addr":"172.21.15.119:6821","nonce":2545410230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":2545410230},{"type":"v1","addr":"172.21.15.119:6825","nonce":2545410230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":2545410230},{"type":"v1","addr":"172.21.15.119:6823","nonce":2545410230}]},"public_addr":"172.21.15.119:6819/2545410230","cluster_addr":"172.21.15.119:6821/2545410230","heartbeat_back_addr":"172.21.15.119:6825/2545410230","heartbeat_front_addr":"172.21.15.119:6823/2545410230","state":["exists","up"]},{"osd":7,"uuid":"79b97a12-f6b4-48c3-b517-2dd6b6e7c85f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":2877892376},{"type":"v1","addr":"172.21.15.132:6817","nonce":2877892376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":2877892376},{"type":"v1","addr":"172.21.15.132:6819","nonce":2877892376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":2877892376},{"type":"v1","addr":"172.21.15.132:6823","nonce":2877892376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":2877892376},{"type":"v1","addr":"172.21.15.132:6821","nonce":2877892376}]},"public_addr":"172.21.15.132:6817/2877892376","cluster_addr":"172.21.15.132:6819/2877892376","heartbeat_back_addr":"172.21.15.132:6823/2877892376","heartbeat_front_addr":"172.21.15.132:6821/2877892376","state":["exists","up"]},{"osd":8,"uuid":"2970524c-b99a-412d-85c2-6c29b54035e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":4115226150},{"type":"v1","addr":"172.21.15.165:6817","nonce":4115226150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":4115226150},{"type":"v1","addr":"172.21.15.165:6819","nonce":4115226150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":4115226150},{"type":"v1","addr":"172.21.15.165:6823","nonce":4115226150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":4115226150},{"type":"v1","addr":"172.21.15.165:6821","nonce":4115226150}]},"public_addr":"172.21.15.165:6817/4115226150","cluster_addr":"172.21.15.165:6819/4115226150","heartbeat_back_addr":"172.21.15.165:6823/4115226150","heartbeat_front_addr":"172.21.15.165:6821/4115226150","state":["exists","up"]},{"osd":9,"uuid":"2b1a08ed-cde9-4def-b3e8-838fff22709c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":2532245222},{"type":"v1","addr":"172.21.15.119:6827","nonce":2532245222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":2532245222},{"type":"v1","addr":"172.21.15.119:6829","nonce":2532245222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":2532245222},{"type":"v1","addr":"172.21.15.119:6833","nonce":2532245222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":2532245222},{"type":"v1","addr":"172.21.15.119:6831","nonce":2532245222}]},"public_addr":"172.21.15.119:6827/2532245222","cluster_addr":"172.21.15.119:6829/2532245222","heartbeat_back_addr":"172.21.15.119:6833/2532245222","heartbeat_front_addr":"172.21.15.119:6831/2532245222","state":["exists","up"]},{"osd":10,"uuid":"f019acd0-915a-4094-b12c-b53672740a5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":3454298492},{"type":"v1","addr":"172.21.15.132:6825","nonce":3454298492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":3454298492},{"type":"v1","addr":"172.21.15.132:6827","nonce":3454298492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":3454298492},{"type":"v1","addr":"172.21.15.132:6831","nonce":3454298492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":3454298492},{"type":"v1","addr":"172.21.15.132:6829","nonce":3454298492}]},"public_addr":"172.21.15.132:6825/3454298492","cluster_addr":"172.21.15.132:6827/3454298492","heartbeat_back_addr":"172.21.15.132:6831/3454298492","heartbeat_front_addr":"172.21.15.132:6829/3454298492","state":["exists","up"]},{"osd":11,"uuid":"2c1e59a8-8f21-45c0-8568-4b632ce020f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":2138771601},{"type":"v1","addr":"172.21.15.165:6825","nonce":2138771601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":2138771601},{"type":"v1","addr":"172.21.15.165:6827","nonce":2138771601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":2138771601},{"type":"v1","addr":"172.21.15.165:6831","nonce":2138771601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":2138771601},{"type":"v1","addr":"172.21.15.165:6829","nonce":2138771601}]},"public_addr":"172.21.15.165:6825/2138771601","cluster_addr":"172.21.15.165:6827/2138771601","heartbeat_back_addr":"172.21.15.165:6831/2138771601","heartbeat_front_addr":"172.21.15.165:6829/2138771601","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:17:54.791531+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:18:19.466638+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:18:43.435536+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:19:15.702685+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:19:41.767370+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:20:06.793003+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:20:39.952744+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:21:05.727262+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:21:30.735829+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:22:04.072142+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:22:30.894047+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:22:56.637728+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.119:6801/2575246519":"2024-03-29T00:16:03.582545+0000","172.21.15.119:6800/2575246519":"2024-03-29T00:16:03.582545+0000","172.21.15.119:0/87565965":"2024-03-29T00:16:03.582545+0000","172.21.15.119:6801/2056426384":"2024-03-29T00:15:30.472283+0000","172.21.15.119:0/1639109126":"2024-03-29T00:15:30.472283+0000","172.21.15.119:0/468478300":"2024-03-29T00:15:30.472283+0000","172.21.15.119:0/2891240357":"2024-03-29T00:15:30.472283+0000","172.21.15.119:0/4201699272":"2024-03-29T00:16:03.582545+0000","172.21.15.119:6800/2056426384":"2024-03-29T00:15:30.472283+0000","172.21.15.119:0/2635982178":"2024-03-29T00:16:03.582545+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T00:23:57.524 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:57.518+0000 7fb1b3200640 1 -- 172.21.15.119:0/3129309904 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fb1980a2880 msgr2=0x7fb1980a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:57.524 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:57.518+0000 7fb1b3200640 1 --2- 172.21.15.119:0/3129309904 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fb1980a2880 0x7fb1980a4d40 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb19c011550 tx=0x7fb19c005f90 comp rx=0 tx=0).stop 2024-03-28T00:23:57.524 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:57.518+0000 7fb1b3200640 1 -- 172.21.15.119:0/3129309904 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb1b4077c40 msgr2=0x7fb1b41a5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:57.524 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:57.518+0000 7fb1b3200640 1 --2- 172.21.15.119:0/3129309904 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb1b4077c40 0x7fb1b41a5380 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb1ac009fd0 tx=0x7fb1ac00ef10 comp rx=0 tx=0).stop 2024-03-28T00:23:57.524 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:57.518+0000 7fb1b3200640 1 -- 172.21.15.119:0/3129309904 shutdown_connections 2024-03-28T00:23:57.524 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:57.518+0000 7fb1b3200640 1 --2- 172.21.15.119:0/3129309904 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fb1980a2880 0x7fb1980a4d40 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:57.524 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:57.518+0000 7fb1b3200640 1 --2- 172.21.15.119:0/3129309904 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb1b4077c40 0x7fb1b41a5380 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:57.525 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:57.518+0000 7fb1b3200640 1 --2- 172.21.15.119:0/3129309904 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb1b40772a0 0x7fb1b419e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:57.525 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:57.518+0000 7fb1b3200640 1 --2- 172.21.15.119:0/3129309904 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb1b4076060 0x7fb1b419ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:57.525 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:57.518+0000 7fb1b3200640 1 -- 172.21.15.119:0/3129309904 >> 172.21.15.119:0/3129309904 conn(0x7fb1b40fec80 msgr2=0x7fb1b4100780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:23:57.525 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:57.518+0000 7fb1b3200640 1 -- 172.21.15.119:0/3129309904 shutdown_connections 2024-03-28T00:23:57.525 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:57.518+0000 7fb1b3200640 1 -- 172.21.15.119:0/3129309904 wait complete. 2024-03-28T00:23:57.543 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T00:23:57.755 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:57.749+0000 7f0c7e600640 1 -- 172.21.15.119:0/2639683133 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0c78101f70 msgr2=0x7f0c78104360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:57.755 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:57.749+0000 7f0c7e600640 1 --2- 172.21.15.119:0/2639683133 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0c78101f70 0x7f0c78104360 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f0c6c009900 tx=0x7f0c6c02bc30 comp rx=0 tx=0).stop 2024-03-28T00:23:57.755 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:57.749+0000 7f0c7e600640 1 -- 172.21.15.119:0/2639683133 shutdown_connections 2024-03-28T00:23:57.756 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:57.749+0000 7f0c7e600640 1 --2- 172.21.15.119:0/2639683133 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0c781071d0 0x7f0c78109620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:57.756 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:57.749+0000 7f0c7e600640 1 --2- 172.21.15.119:0/2639683133 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0c781048a0 0x7f0c78106c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:57.756 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:57.749+0000 7f0c7e600640 1 --2- 172.21.15.119:0/2639683133 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0c78101f70 0x7f0c78104360 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:57.756 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:57.749+0000 7f0c7e600640 1 -- 172.21.15.119:0/2639683133 >> 172.21.15.119:0/2639683133 conn(0x7f0c780fb980 msgr2=0x7f0c780fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:23:57.756 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:57.749+0000 7f0c7e600640 1 -- 172.21.15.119:0/2639683133 shutdown_connections 2024-03-28T00:23:57.756 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:57.749+0000 7f0c7e600640 1 -- 172.21.15.119:0/2639683133 wait complete. 2024-03-28T00:23:57.758 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:57.750+0000 7f0c7e600640 1 Processor -- start 2024-03-28T00:23:57.758 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:57.750+0000 7f0c7e600640 1 -- start start 2024-03-28T00:23:57.758 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:57.751+0000 7f0c7e600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0c78101f70 0x7f0c78199a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:23:57.758 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:57.751+0000 7f0c7e600640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0c781048a0 0x7f0c78199f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:23:57.758 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:57.751+0000 7f0c7e600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0c781071d0 0x7f0c781a0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:23:57.758 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:57.751+0000 7f0c7e600640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c7810d1d0 con 0x7f0c781071d0 2024-03-28T00:23:57.758 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:57.751+0000 7f0c7e600640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c7810d050 con 0x7f0c781048a0 2024-03-28T00:23:57.758 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:57.751+0000 7f0c7e600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c7810d350 con 0x7f0c78101f70 2024-03-28T00:23:57.758 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:57.751+0000 7f0c7dc00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0c781071d0 0x7f0c781a0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:23:57.759 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:57.751+0000 7f0c77e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0c781048a0 0x7f0c78199f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:23:57.759 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:57.751+0000 7f0c7dc00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0c781071d0 0x7f0c781a0fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:56306/0 (socket says 172.21.15.119:56306) 2024-03-28T00:23:57.759 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:57.751+0000 7f0c7dc00640 1 -- 172.21.15.119:0/1866549661 learned_addr learned my addr 172.21.15.119:0/1866549661 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:23:57.759 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:57.751+0000 7f0c77e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0c781048a0 0x7f0c78199f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.119:36286/0 (socket says 172.21.15.119:36286) 2024-03-28T00:23:57.759 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:57.751+0000 7f0c7d200640 1 --2- 172.21.15.119:0/1866549661 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0c78101f70 0x7f0c78199a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:23:57.759 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:57.751+0000 7f0c7dc00640 1 -- 172.21.15.119:0/1866549661 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0c78101f70 msgr2=0x7f0c78199a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:57.759 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:57.751+0000 7f0c7dc00640 1 --2- 172.21.15.119:0/1866549661 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0c78101f70 0x7f0c78199a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:57.759 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:57.751+0000 7f0c7dc00640 1 -- 172.21.15.119:0/1866549661 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0c781048a0 msgr2=0x7f0c78199f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:57.759 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:57.751+0000 7f0c7dc00640 1 --2- 172.21.15.119:0/1866549661 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0c781048a0 0x7f0c78199f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:57.759 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:57.751+0000 7f0c7dc00640 1 -- 172.21.15.119:0/1866549661 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c781a1640 con 0x7f0c781071d0 2024-03-28T00:23:57.759 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:57.751+0000 7f0c77e00640 1 --2- 172.21.15.119:0/1866549661 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0c781048a0 0x7f0c78199f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T00:23:57.759 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:57.752+0000 7f0c7dc00640 1 --2- 172.21.15.119:0/1866549661 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0c781071d0 0x7f0c781a0fd0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f0c7000eb10 tx=0x7f0c7000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:23:57.759 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:57.752+0000 7f0c75600640 1 -- 172.21.15.119:0/1866549661 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c70004b20 con 0x7f0c781071d0 2024-03-28T00:23:57.759 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:57.752+0000 7f0c75600640 1 -- 172.21.15.119:0/1866549661 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0c70004cc0 con 0x7f0c781071d0 2024-03-28T00:23:57.759 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:57.752+0000 7f0c7e600640 1 -- 172.21.15.119:0/1866549661 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c781a1930 con 0x7f0c781071d0 2024-03-28T00:23:57.760 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:57.752+0000 7f0c75600640 1 -- 172.21.15.119:0/1866549661 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c70010760 con 0x7f0c781071d0 2024-03-28T00:23:57.760 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:57.752+0000 7f0c7e600640 1 -- 172.21.15.119:0/1866549661 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c781a1e80 con 0x7f0c781071d0 2024-03-28T00:23:57.760 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:57.753+0000 7f0c7e600640 1 -- 172.21.15.119:0/1866549661 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c78108d20 con 0x7f0c781071d0 2024-03-28T00:23:57.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:57.755+0000 7f0c75600640 1 -- 172.21.15.119:0/1866549661 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f0c70005020 con 0x7f0c781071d0 2024-03-28T00:23:57.762 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:57.756+0000 7f0c75600640 1 --2- 172.21.15.119:0/1866549661 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f0c600a2880 0x7f0c600a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:23:57.763 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:57.757+0000 7f0c75600640 1 -- 172.21.15.119:0/1866549661 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f0c700cee90 con 0x7f0c781071d0 2024-03-28T00:23:57.763 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:57.757+0000 7f0c7d200640 1 --2- 172.21.15.119:0/1866549661 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f0c600a2880 0x7f0c600a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:23:57.763 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:57.757+0000 7f0c7d200640 1 --2- 172.21.15.119:0/1866549661 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f0c600a2880 0x7f0c600a4d40 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f0c6c02c240 tx=0x7f0c6c02df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:23:57.767 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:57.761+0000 7f0c75600640 1 -- 172.21.15.119:0/1866549661 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f0c7009bfe0 con 0x7f0c781071d0 2024-03-28T00:23:57.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:57 smithi119 ceph-mon[31518]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-28T00:23:57.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:57 smithi119 ceph-mon[31518]: osdmap e84: 12 total, 12 up, 12 in 2024-03-28T00:23:57.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:57 smithi119 ceph-mon[31518]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-03-28T00:23:57.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:57 smithi119 ceph-mon[31518]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-03-28T00:23:57.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:57 smithi119 ceph-mon[31518]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-03-28T00:23:57.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:57 smithi119 ceph-mon[31518]: from='client.? 172.21.15.119:0/3129309904' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-28T00:23:57.907 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:57 smithi119 ceph-mon[31518]: osdmap e85: 12 total, 12 up, 12 in 2024-03-28T00:23:57.971 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:57.965+0000 7f0c7e600640 1 -- 172.21.15.119:0/1866549661 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0c780622a0 con 0x7f0c781071d0 2024-03-28T00:23:57.974 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:57.968+0000 7f0c75600640 1 -- 172.21.15.119:0/1866549661 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+22300 (secure 0 0 0) 0x7f0c7009b8e0 con 0x7f0c781071d0 2024-03-28T00:23:57.975 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-28T00:23:57.975 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":85,"fsid":"10f73b92-ec98-11ee-95d0-87774f69a715","created":"2024-03-28T00:14:57.563604+0000","modified":"2024-03-28T00:23:57.715342+0000","last_up_change":"2024-03-28T00:22:58.821662+0000","last_in_change":"2024-03-28T00:22:38.467270+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T00:18:46.253414+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-28T00:23:54.261981+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-28T00:23:55.685836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c0b875d-a01e-4294-9c9c-5d0b048b8912","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":580396252},{"type":"v1","addr":"172.21.15.119:6803","nonce":580396252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":580396252},{"type":"v1","addr":"172.21.15.119:6805","nonce":580396252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":580396252},{"type":"v1","addr":"172.21.15.119:6809","nonce":580396252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":580396252},{"type":"v1","addr":"172.21.15.119:6807","nonce":580396252}]},"public_addr":"172.21.15.119:6803/580396252","cluster_addr":"172.21.15.119:6805/580396252","heartbeat_back_addr":"172.21.15.119:6809/580396252","heartbeat_front_addr":"172.21.15.119:6807/580396252","state":["exists","up"]},{"osd":1,"uuid":"5567f7c2-8ea4-4789-9ea1-467793324fb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":2500318888},{"type":"v1","addr":"172.21.15.132:6801","nonce":2500318888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":2500318888},{"type":"v1","addr":"172.21.15.132:6803","nonce":2500318888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":2500318888},{"type":"v1","addr":"172.21.15.132:6807","nonce":2500318888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":2500318888},{"type":"v1","addr":"172.21.15.132:6805","nonce":2500318888}]},"public_addr":"172.21.15.132:6801/2500318888","cluster_addr":"172.21.15.132:6803/2500318888","heartbeat_back_addr":"172.21.15.132:6807/2500318888","heartbeat_front_addr":"172.21.15.132:6805/2500318888","state":["exists","up"]},{"osd":2,"uuid":"ee54dbd0-24b4-4ee6-988c-ce71491199d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":2361434832},{"type":"v1","addr":"172.21.15.165:6801","nonce":2361434832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":2361434832},{"type":"v1","addr":"172.21.15.165:6803","nonce":2361434832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":2361434832},{"type":"v1","addr":"172.21.15.165:6807","nonce":2361434832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":2361434832},{"type":"v1","addr":"172.21.15.165:6805","nonce":2361434832}]},"public_addr":"172.21.15.165:6801/2361434832","cluster_addr":"172.21.15.165:6803/2361434832","heartbeat_back_addr":"172.21.15.165:6807/2361434832","heartbeat_front_addr":"172.21.15.165:6805/2361434832","state":["exists","up"]},{"osd":3,"uuid":"0464c40b-828f-4db4-96d9-3f7a6325e6da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":3732272210},{"type":"v1","addr":"172.21.15.119:6811","nonce":3732272210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":3732272210},{"type":"v1","addr":"172.21.15.119:6813","nonce":3732272210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":3732272210},{"type":"v1","addr":"172.21.15.119:6817","nonce":3732272210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":3732272210},{"type":"v1","addr":"172.21.15.119:6815","nonce":3732272210}]},"public_addr":"172.21.15.119:6811/3732272210","cluster_addr":"172.21.15.119:6813/3732272210","heartbeat_back_addr":"172.21.15.119:6817/3732272210","heartbeat_front_addr":"172.21.15.119:6815/3732272210","state":["exists","up"]},{"osd":4,"uuid":"8fd70b8e-5128-4173-aa0e-e735dfeb995d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":2803039505},{"type":"v1","addr":"172.21.15.132:6809","nonce":2803039505}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":2803039505},{"type":"v1","addr":"172.21.15.132:6811","nonce":2803039505}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":2803039505},{"type":"v1","addr":"172.21.15.132:6815","nonce":2803039505}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":2803039505},{"type":"v1","addr":"172.21.15.132:6813","nonce":2803039505}]},"public_addr":"172.21.15.132:6809/2803039505","cluster_addr":"172.21.15.132:6811/2803039505","heartbeat_back_addr":"172.21.15.132:6815/2803039505","heartbeat_front_addr":"172.21.15.132:6813/2803039505","state":["exists","up"]},{"osd":5,"uuid":"3224b4c1-733e-4424-a6da-870bbb5f94e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":2561252206},{"type":"v1","addr":"172.21.15.165:6809","nonce":2561252206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":2561252206},{"type":"v1","addr":"172.21.15.165:6811","nonce":2561252206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":2561252206},{"type":"v1","addr":"172.21.15.165:6815","nonce":2561252206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":2561252206},{"type":"v1","addr":"172.21.15.165:6813","nonce":2561252206}]},"public_addr":"172.21.15.165:6809/2561252206","cluster_addr":"172.21.15.165:6811/2561252206","heartbeat_back_addr":"172.21.15.165:6815/2561252206","heartbeat_front_addr":"172.21.15.165:6813/2561252206","state":["exists","up"]},{"osd":6,"uuid":"e435745d-59b5-439a-9091-0946ca0e9332","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":2545410230},{"type":"v1","addr":"172.21.15.119:6819","nonce":2545410230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":2545410230},{"type":"v1","addr":"172.21.15.119:6821","nonce":2545410230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":2545410230},{"type":"v1","addr":"172.21.15.119:6825","nonce":2545410230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":2545410230},{"type":"v1","addr":"172.21.15.119:6823","nonce":2545410230}]},"public_addr":"172.21.15.119:6819/2545410230","cluster_addr":"172.21.15.119:6821/2545410230","heartbeat_back_addr":"172.21.15.119:6825/2545410230","heartbeat_front_addr":"172.21.15.119:6823/2545410230","state":["exists","up"]},{"osd":7,"uuid":"79b97a12-f6b4-48c3-b517-2dd6b6e7c85f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":2877892376},{"type":"v1","addr":"172.21.15.132:6817","nonce":2877892376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":2877892376},{"type":"v1","addr":"172.21.15.132:6819","nonce":2877892376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":2877892376},{"type":"v1","addr":"172.21.15.132:6823","nonce":2877892376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":2877892376},{"type":"v1","addr":"172.21.15.132:6821","nonce":2877892376}]},"public_addr":"172.21.15.132:6817/2877892376","cluster_addr":"172.21.15.132:6819/2877892376","heartbeat_back_addr":"172.21.15.132:6823/2877892376","heartbeat_front_addr":"172.21.15.132:6821/2877892376","state":["exists","up"]},{"osd":8,"uuid":"2970524c-b99a-412d-85c2-6c29b54035e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":4115226150},{"type":"v1","addr":"172.21.15.165:6817","nonce":4115226150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":4115226150},{"type":"v1","addr":"172.21.15.165:6819","nonce":4115226150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":4115226150},{"type":"v1","addr":"172.21.15.165:6823","nonce":4115226150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":4115226150},{"type":"v1","addr":"172.21.15.165:6821","nonce":4115226150}]},"public_addr":"172.21.15.165:6817/4115226150","cluster_addr":"172.21.15.165:6819/4115226150","heartbeat_back_addr":"172.21.15.165:6823/4115226150","heartbeat_front_addr":"172.21.15.165:6821/4115226150","state":["exists","up"]},{"osd":9,"uuid":"2b1a08ed-cde9-4def-b3e8-838fff22709c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":2532245222},{"type":"v1","addr":"172.21.15.119:6827","nonce":2532245222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":2532245222},{"type":"v1","addr":"172.21.15.119:6829","nonce":2532245222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":2532245222},{"type":"v1","addr":"172.21.15.119:6833","nonce":2532245222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":2532245222},{"type":"v1","addr":"172.21.15.119:6831","nonce":2532245222}]},"public_addr":"172.21.15.119:6827/2532245222","cluster_addr":"172.21.15.119:6829/2532245222","heartbeat_back_addr":"172.21.15.119:6833/2532245222","heartbeat_front_addr":"172.21.15.119:6831/2532245222","state":["exists","up"]},{"osd":10,"uuid":"f019acd0-915a-4094-b12c-b53672740a5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":3454298492},{"type":"v1","addr":"172.21.15.132:6825","nonce":3454298492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":3454298492},{"type":"v1","addr":"172.21.15.132:6827","nonce":3454298492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":3454298492},{"type":"v1","addr":"172.21.15.132:6831","nonce":3454298492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":3454298492},{"type":"v1","addr":"172.21.15.132:6829","nonce":3454298492}]},"public_addr":"172.21.15.132:6825/3454298492","cluster_addr":"172.21.15.132:6827/3454298492","heartbeat_back_addr":"172.21.15.132:6831/3454298492","heartbeat_front_addr":"172.21.15.132:6829/3454298492","state":["exists","up"]},{"osd":11,"uuid":"2c1e59a8-8f21-45c0-8568-4b632ce020f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":2138771601},{"type":"v1","addr":"172.21.15.165:6825","nonce":2138771601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":2138771601},{"type":"v1","addr":"172.21.15.165:6827","nonce":2138771601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":2138771601},{"type":"v1","addr":"172.21.15.165:6831","nonce":2138771601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":2138771601},{"type":"v1","addr":"172.21.15.165:6829","nonce":2138771601}]},"public_addr":"172.21.15.165:6825/2138771601","cluster_addr":"172.21.15.165:6827/2138771601","heartbeat_back_addr":"172.21.15.165:6831/2138771601","heartbeat_front_addr":"172.21.15.165:6829/2138771601","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:17:54.791531+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:18:19.466638+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:18:43.435536+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:19:15.702685+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:19:41.767370+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:20:06.793003+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:20:39.952744+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:21:05.727262+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:21:30.735829+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:22:04.072142+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:22:30.894047+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:22:56.637728+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.119:6801/2575246519":"2024-03-29T00:16:03.582545+0000","172.21.15.119:6800/2575246519":"2024-03-29T00:16:03.582545+0000","172.21.15.119:0/87565965":"2024-03-29T00:16:03.582545+0000","172.21.15.119:6801/2056426384":"2024-03-29T00:15:30.472283+0000","172.21.15.119:0/1639109126":"2024-03-29T00:15:30.472283+0000","172.21.15.119:0/468478300":"2024-03-29T00:15:30.472283+0000","172.21.15.119:0/2891240357":"2024-03-29T00:15:30.472283+0000","172.21.15.119:0/4201699272":"2024-03-29T00:16:03.582545+0000","172.21.15.119:6800/2056426384":"2024-03-29T00:15:30.472283+0000","172.21.15.119:0/2635982178":"2024-03-29T00:16:03.582545+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T00:23:57.977 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:57.971+0000 7f0c7e600640 1 -- 172.21.15.119:0/1866549661 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f0c600a2880 msgr2=0x7f0c600a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:57.977 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:57.971+0000 7f0c7e600640 1 --2- 172.21.15.119:0/1866549661 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f0c600a2880 0x7f0c600a4d40 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f0c6c02c240 tx=0x7f0c6c02df90 comp rx=0 tx=0).stop 2024-03-28T00:23:57.977 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:57.971+0000 7f0c7e600640 1 -- 172.21.15.119:0/1866549661 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0c781071d0 msgr2=0x7f0c781a0fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:57.978 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:57.971+0000 7f0c7e600640 1 --2- 172.21.15.119:0/1866549661 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0c781071d0 0x7f0c781a0fd0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f0c7000eb10 tx=0x7f0c7000c2d0 comp rx=0 tx=0).stop 2024-03-28T00:23:57.978 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:57.971+0000 7f0c7e600640 1 -- 172.21.15.119:0/1866549661 shutdown_connections 2024-03-28T00:23:57.978 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:57.971+0000 7f0c7e600640 1 --2- 172.21.15.119:0/1866549661 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f0c600a2880 0x7f0c600a4d40 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:57.978 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:57.971+0000 7f0c7e600640 1 --2- 172.21.15.119:0/1866549661 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0c781071d0 0x7f0c781a0fd0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:57.978 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:57.971+0000 7f0c7e600640 1 --2- 172.21.15.119:0/1866549661 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0c781048a0 0x7f0c78199f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:57.978 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:57.971+0000 7f0c7e600640 1 --2- 172.21.15.119:0/1866549661 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0c78101f70 0x7f0c78199a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:57.978 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:57.971+0000 7f0c7e600640 1 -- 172.21.15.119:0/1866549661 >> 172.21.15.119:0/1866549661 conn(0x7f0c780fb980 msgr2=0x7f0c78101700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:23:57.978 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:57.971+0000 7f0c7e600640 1 -- 172.21.15.119:0/1866549661 shutdown_connections 2024-03-28T00:23:57.978 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:57.971+0000 7f0c7e600640 1 -- 172.21.15.119:0/1866549661 wait complete. 2024-03-28T00:23:57.997 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-28T00:23:58.114 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:57 smithi132 ceph-mon[35941]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-28T00:23:58.114 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:57 smithi132 ceph-mon[35941]: osdmap e84: 12 total, 12 up, 12 in 2024-03-28T00:23:58.114 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:57 smithi132 ceph-mon[35941]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-03-28T00:23:58.114 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:57 smithi132 ceph-mon[35941]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-03-28T00:23:58.114 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:57 smithi132 ceph-mon[35941]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-03-28T00:23:58.115 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:57 smithi132 ceph-mon[35941]: from='client.? 172.21.15.119:0/3129309904' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-28T00:23:58.115 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:57 smithi132 ceph-mon[35941]: osdmap e85: 12 total, 12 up, 12 in 2024-03-28T00:23:58.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:57 smithi165 ceph-mon[36302]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-28T00:23:58.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:57 smithi165 ceph-mon[36302]: osdmap e84: 12 total, 12 up, 12 in 2024-03-28T00:23:58.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:57 smithi165 ceph-mon[36302]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-03-28T00:23:58.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:57 smithi165 ceph-mon[36302]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-03-28T00:23:58.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:57 smithi165 ceph-mon[36302]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-03-28T00:23:58.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:57 smithi165 ceph-mon[36302]: from='client.? 172.21.15.119:0/3129309904' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-28T00:23:58.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:57 smithi165 ceph-mon[36302]: osdmap e85: 12 total, 12 up, 12 in 2024-03-28T00:23:58.206 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:58.199+0000 7f55d1a00640 1 -- 172.21.15.119:0/3017654857 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f55cc1038e0 msgr2=0x7f55cc103d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:58.206 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:58.199+0000 7f55d1a00640 1 --2- 172.21.15.119:0/3017654857 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f55cc1038e0 0x7f55cc103d00 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f55bc009900 tx=0x7f55bc02bc30 comp rx=0 tx=0).stop 2024-03-28T00:23:58.206 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:58.199+0000 7f55d1a00640 1 -- 172.21.15.119:0/3017654857 shutdown_connections 2024-03-28T00:23:58.206 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:58.200+0000 7f55d1a00640 1 --2- 172.21.15.119:0/3017654857 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f55cc105490 0x7f55cc10bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:58.206 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:58.200+0000 7f55d1a00640 1 --2- 172.21.15.119:0/3017654857 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f55cc104ad0 0x7f55cc104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:58.206 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:58.200+0000 7f55d1a00640 1 --2- 172.21.15.119:0/3017654857 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f55cc1038e0 0x7f55cc103d00 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:58.206 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:58.200+0000 7f55d1a00640 1 -- 172.21.15.119:0/3017654857 >> 172.21.15.119:0/3017654857 conn(0x7f55cc0fee50 msgr2=0x7f55cc101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:23:58.206 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:58.200+0000 7f55d1a00640 1 -- 172.21.15.119:0/3017654857 shutdown_connections 2024-03-28T00:23:58.206 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:58.200+0000 7f55d1a00640 1 -- 172.21.15.119:0/3017654857 wait complete. 2024-03-28T00:23:58.209 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:58.201+0000 7f55d1a00640 1 Processor -- start 2024-03-28T00:23:58.209 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:58.201+0000 7f55d1a00640 1 -- start start 2024-03-28T00:23:58.209 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:58.201+0000 7f55d1a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f55cc1038e0 0x7f55cc199ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:23:58.209 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:58.201+0000 7f55d1a00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f55cc104ad0 0x7f55cc19a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:23:58.209 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:58.201+0000 7f55d1a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f55cc105490 0x7f55cc1a1240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:23:58.209 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:58.201+0000 7f55d1a00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55cc10f5a0 con 0x7f55cc105490 2024-03-28T00:23:58.209 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:58.201+0000 7f55d1a00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55cc10f420 con 0x7f55cc104ad0 2024-03-28T00:23:58.209 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:58.201+0000 7f55d1a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55cc10f720 con 0x7f55cc1038e0 2024-03-28T00:23:58.209 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:58.201+0000 7f55c2000640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f55cc104ad0 0x7f55cc19a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:23:58.209 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:58.201+0000 7f55caa00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f55cc1038e0 0x7f55cc199ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:23:58.209 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:58.202+0000 7f55cb400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f55cc105490 0x7f55cc1a1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:23:58.209 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:58.202+0000 7f55c2000640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f55cc104ad0 0x7f55cc19a1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.119:36306/0 (socket says 172.21.15.119:36306) 2024-03-28T00:23:58.209 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:58.202+0000 7f55c2000640 1 -- 172.21.15.119:0/1646829942 learned_addr learned my addr 172.21.15.119:0/1646829942 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:23:58.210 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:58.202+0000 7f55caa00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f55cc1038e0 0x7f55cc199ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.119:42554/0 (socket says 172.21.15.119:42554) 2024-03-28T00:23:58.210 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:58.202+0000 7f55c2000640 1 -- 172.21.15.119:0/1646829942 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f55cc1038e0 msgr2=0x7f55cc199ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:58.210 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:58.202+0000 7f55c2000640 1 --2- 172.21.15.119:0/1646829942 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f55cc1038e0 0x7f55cc199ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:58.210 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:58.202+0000 7f55c2000640 1 -- 172.21.15.119:0/1646829942 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f55cc105490 msgr2=0x7f55cc1a1240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:58.210 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:58.202+0000 7f55c2000640 1 --2- 172.21.15.119:0/1646829942 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f55cc105490 0x7f55cc1a1240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:58.210 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:58.202+0000 7f55c2000640 1 -- 172.21.15.119:0/1646829942 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55cc1a18e0 con 0x7f55cc104ad0 2024-03-28T00:23:58.210 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:58.202+0000 7f55c2000640 1 --2- 172.21.15.119:0/1646829942 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f55cc104ad0 0x7f55cc19a1e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f55b400e9f0 tx=0x7f55b400edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:23:58.210 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:58.202+0000 7f55c3e00640 1 -- 172.21.15.119:0/1646829942 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55b400ccc0 con 0x7f55cc104ad0 2024-03-28T00:23:58.210 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:58.202+0000 7f55c3e00640 1 -- 172.21.15.119:0/1646829942 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f55b400ce60 con 0x7f55cc104ad0 2024-03-28T00:23:58.211 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:58.202+0000 7f55d1a00640 1 -- 172.21.15.119:0/1646829942 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55cc1a1bd0 con 0x7f55cc104ad0 2024-03-28T00:23:58.211 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:58.202+0000 7f55c3e00640 1 -- 172.21.15.119:0/1646829942 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55b4010690 con 0x7f55cc104ad0 2024-03-28T00:23:58.211 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:58.203+0000 7f55d1a00640 1 -- 172.21.15.119:0/1646829942 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55cc1a20f0 con 0x7f55cc104ad0 2024-03-28T00:23:58.211 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:58.204+0000 7f55d1a00640 1 -- 172.21.15.119:0/1646829942 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55cc10ad20 con 0x7f55cc104ad0 2024-03-28T00:23:58.211 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:58.205+0000 7f55c3e00640 1 -- 172.21.15.119:0/1646829942 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f55b40046c0 con 0x7f55cc104ad0 2024-03-28T00:23:58.213 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:58.207+0000 7f55c3e00640 1 --2- 172.21.15.119:0/1646829942 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f55b00a2880 0x7f55b00a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:23:58.213 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:58.207+0000 7f55c3e00640 1 -- 172.21.15.119:0/1646829942 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f55b40ced00 con 0x7f55cc104ad0 2024-03-28T00:23:58.213 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:58.207+0000 7f55caa00640 1 --2- 172.21.15.119:0/1646829942 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f55b00a2880 0x7f55b00a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:23:58.214 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:58.208+0000 7f55caa00640 1 --2- 172.21.15.119:0/1646829942 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f55b00a2880 0x7f55b00a4d40 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f55bc02c170 tx=0x7f55bc02df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:23:58.217 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:58.211+0000 7f55c3e00640 1 -- 172.21.15.119:0/1646829942 <== mon.2 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f55b409be50 con 0x7f55cc104ad0 2024-03-28T00:23:58.456 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:58.450+0000 7f55d1a00640 1 -- 172.21.15.119:0/1646829942 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f55cc062340 con 0x7f55cc104ad0 2024-03-28T00:23:59.092 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:59.086+0000 7f55c3e00640 1 -- 172.21.15.119:0/1646829942 <== mon.2 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) v1 ==== 118+0+0 (secure 0 0 0) 0x7f55b409b750 con 0x7f55cc104ad0 2024-03-28T00:23:59.097 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:59.091+0000 7f55d1a00640 1 -- 172.21.15.119:0/1646829942 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f55b00a2880 msgr2=0x7f55b00a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:59.097 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:59.091+0000 7f55d1a00640 1 --2- 172.21.15.119:0/1646829942 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f55b00a2880 0x7f55b00a4d40 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f55bc02c170 tx=0x7f55bc02df90 comp rx=0 tx=0).stop 2024-03-28T00:23:59.097 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:59.091+0000 7f55d1a00640 1 -- 172.21.15.119:0/1646829942 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f55cc104ad0 msgr2=0x7f55cc19a1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:59.097 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:59.091+0000 7f55d1a00640 1 --2- 172.21.15.119:0/1646829942 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f55cc104ad0 0x7f55cc19a1e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f55b400e9f0 tx=0x7f55b400edf0 comp rx=0 tx=0).stop 2024-03-28T00:23:59.097 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:59.091+0000 7f55d1a00640 1 -- 172.21.15.119:0/1646829942 shutdown_connections 2024-03-28T00:23:59.097 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:59.091+0000 7f55d1a00640 1 --2- 172.21.15.119:0/1646829942 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f55b00a2880 0x7f55b00a4d40 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:59.098 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:59.091+0000 7f55d1a00640 1 --2- 172.21.15.119:0/1646829942 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f55cc105490 0x7f55cc1a1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:59.098 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:59.091+0000 7f55d1a00640 1 --2- 172.21.15.119:0/1646829942 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f55cc104ad0 0x7f55cc19a1e0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:59.098 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:59.091+0000 7f55d1a00640 1 --2- 172.21.15.119:0/1646829942 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f55cc1038e0 0x7f55cc199ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:59.098 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:59.091+0000 7f55d1a00640 1 -- 172.21.15.119:0/1646829942 >> 172.21.15.119:0/1646829942 conn(0x7f55cc0fee50 msgr2=0x7f55cc107ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:23:59.098 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:59.091+0000 7f55d1a00640 1 -- 172.21.15.119:0/1646829942 shutdown_connections 2024-03-28T00:23:59.098 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:59.091+0000 7f55d1a00640 1 -- 172.21.15.119:0/1646829942 wait complete. 2024-03-28T00:23:59.115 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:58 smithi119 ceph-mon[31518]: pgmap v311: 97 pgs: 4 creating+peering, 37 active+clean, 56 unknown; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-03-28T00:23:59.115 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:58 smithi119 ceph-mon[31518]: from='client.? 172.21.15.119:0/1866549661' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-28T00:23:59.115 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:58 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.132:6838/3305822525,v1:172.21.15.132:6839/3305822525] up:active 2024-03-28T00:23:59.115 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:58 smithi119 ceph-mon[31518]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-28T00:23:59.115 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:58 smithi119 ceph-mon[31518]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-28T00:23:59.115 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:58 smithi119 ceph-mon[31518]: from='client.? 172.21.15.119:0/1646829942' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-28T00:23:59.115 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:58 smithi119 ceph-mon[31518]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-28T00:23:59.116 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:58 smithi119 ceph-mon[31518]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-03-28T00:23:59.116 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:23:58 smithi119 ceph-mon[31518]: Cluster is now healthy 2024-03-28T00:23:59.117 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-03-28T00:23:59.117 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 5 2024-03-28T00:23:59.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:58 smithi165 ceph-mon[36302]: pgmap v311: 97 pgs: 4 creating+peering, 37 active+clean, 56 unknown; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-03-28T00:23:59.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:58 smithi165 ceph-mon[36302]: from='client.? 172.21.15.119:0/1866549661' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-28T00:23:59.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:58 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.132:6838/3305822525,v1:172.21.15.132:6839/3305822525] up:active 2024-03-28T00:23:59.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:58 smithi165 ceph-mon[36302]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-28T00:23:59.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:58 smithi165 ceph-mon[36302]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-28T00:23:59.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:58 smithi165 ceph-mon[36302]: from='client.? 172.21.15.119:0/1646829942' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-28T00:23:59.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:58 smithi165 ceph-mon[36302]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-28T00:23:59.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:58 smithi165 ceph-mon[36302]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-03-28T00:23:59.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:23:58 smithi165 ceph-mon[36302]: Cluster is now healthy 2024-03-28T00:23:59.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:58 smithi132 ceph-mon[35941]: pgmap v311: 97 pgs: 4 creating+peering, 37 active+clean, 56 unknown; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-03-28T00:23:59.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:58 smithi132 ceph-mon[35941]: from='client.? 172.21.15.119:0/1866549661' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-28T00:23:59.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:58 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.132:6838/3305822525,v1:172.21.15.132:6839/3305822525] up:active 2024-03-28T00:23:59.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:58 smithi132 ceph-mon[35941]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-28T00:23:59.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:58 smithi132 ceph-mon[35941]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-28T00:23:59.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:58 smithi132 ceph-mon[35941]: from='client.? 172.21.15.119:0/1646829942' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-28T00:23:59.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:58 smithi132 ceph-mon[35941]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-28T00:23:59.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:58 smithi132 ceph-mon[35941]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-03-28T00:23:59.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:23:58 smithi132 ceph-mon[35941]: Cluster is now healthy 2024-03-28T00:23:59.326 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:59.320+0000 7f842de00640 1 -- 172.21.15.119:0/2941376343 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8428105d20 msgr2=0x7f8428108110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:59.327 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:59.320+0000 7f842de00640 1 --2- 172.21.15.119:0/2941376343 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8428105d20 0x7f8428108110 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8420009ca0 tx=0x7f842002c100 comp rx=0 tx=0).stop 2024-03-28T00:23:59.327 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:59.320+0000 7f842de00640 1 -- 172.21.15.119:0/2941376343 shutdown_connections 2024-03-28T00:23:59.327 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:59.320+0000 7f842de00640 1 --2- 172.21.15.119:0/2941376343 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8428105d20 0x7f8428108110 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:59.327 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:59.320+0000 7f842de00640 1 --2- 172.21.15.119:0/2941376343 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8428101020 0x7f84281014a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:59.327 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:59.320+0000 7f842de00640 1 --2- 172.21.15.119:0/2941376343 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8428067c00 0x7f8428100ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:59.327 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:59.320+0000 7f842de00640 1 -- 172.21.15.119:0/2941376343 >> 172.21.15.119:0/2941376343 conn(0x7f84280fae60 msgr2=0x7f84280fd2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:23:59.327 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:59.320+0000 7f842de00640 1 -- 172.21.15.119:0/2941376343 shutdown_connections 2024-03-28T00:23:59.327 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:59.320+0000 7f842de00640 1 -- 172.21.15.119:0/2941376343 wait complete. 2024-03-28T00:23:59.329 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:59.321+0000 7f842de00640 1 Processor -- start 2024-03-28T00:23:59.329 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:59.321+0000 7f842de00640 1 -- start start 2024-03-28T00:23:59.330 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:59.322+0000 7f842de00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8428067c00 0x7f842810b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:23:59.330 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:59.322+0000 7f842de00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8428101020 0x7f8428109aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:23:59.330 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:59.322+0000 7f842de00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8428105d20 0x7f8428109fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:23:59.330 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:59.322+0000 7f842de00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f842810d380 con 0x7f8428105d20 2024-03-28T00:23:59.330 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:59.322+0000 7f842de00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f842810d200 con 0x7f8428067c00 2024-03-28T00:23:59.330 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:59.322+0000 7f842de00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f842810d500 con 0x7f8428101020 2024-03-28T00:23:59.330 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:59.322+0000 7f8426a00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8428067c00 0x7f842810b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:23:59.330 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:59.322+0000 7f8426a00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8428067c00 0x7f842810b460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.119:36312/0 (socket says 172.21.15.119:36312) 2024-03-28T00:23:59.330 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:59.322+0000 7f8426a00640 1 -- 172.21.15.119:0/1546721714 learned_addr learned my addr 172.21.15.119:0/1546721714 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:23:59.330 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:59.322+0000 7f8426000640 1 --2- 172.21.15.119:0/1546721714 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8428101020 0x7f8428109aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:23:59.330 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:59.322+0000 7f8427400640 1 --2- 172.21.15.119:0/1546721714 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8428105d20 0x7f8428109fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:23:59.330 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:59.322+0000 7f8426a00640 1 -- 172.21.15.119:0/1546721714 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8428101020 msgr2=0x7f8428109aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:59.331 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:59.322+0000 7f8426a00640 1 --2- 172.21.15.119:0/1546721714 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8428101020 0x7f8428109aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:59.331 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:59.322+0000 7f8426a00640 1 -- 172.21.15.119:0/1546721714 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8428105d20 msgr2=0x7f8428109fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:23:59.331 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:59.322+0000 7f8426a00640 1 --2- 172.21.15.119:0/1546721714 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8428105d20 0x7f8428109fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:23:59.331 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:59.322+0000 7f8426a00640 1 -- 172.21.15.119:0/1546721714 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f842810a870 con 0x7f8428067c00 2024-03-28T00:23:59.331 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:59.323+0000 7f8426a00640 1 --2- 172.21.15.119:0/1546721714 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8428067c00 0x7f842810b460 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f841800e9c0 tx=0x7f841800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:23:59.331 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:59.323+0000 7f842ca00640 1 -- 172.21.15.119:0/1546721714 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f841800cd80 con 0x7f8428067c00 2024-03-28T00:23:59.331 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:59.323+0000 7f842ca00640 1 -- 172.21.15.119:0/1546721714 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f84180035f0 con 0x7f8428067c00 2024-03-28T00:23:59.331 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:59.323+0000 7f842de00640 1 -- 172.21.15.119:0/1546721714 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84281a1a90 con 0x7f8428067c00 2024-03-28T00:23:59.331 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:59.323+0000 7f842ca00640 1 -- 172.21.15.119:0/1546721714 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8418010690 con 0x7f8428067c00 2024-03-28T00:23:59.331 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:59.323+0000 7f842de00640 1 -- 172.21.15.119:0/1546721714 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84281a2030 con 0x7f8428067c00 2024-03-28T00:23:59.332 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:59.324+0000 7f842de00640 1 -- 172.21.15.119:0/1546721714 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8428104b30 con 0x7f8428067c00 2024-03-28T00:23:59.332 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:59.326+0000 7f842ca00640 1 -- 172.21.15.119:0/1546721714 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f84180037a0 con 0x7f8428067c00 2024-03-28T00:23:59.333 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:59.327+0000 7f842ca00640 1 --2- 172.21.15.119:0/1546721714 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f840c0a2880 0x7f840c0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:23:59.333 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:59.327+0000 7f842ca00640 1 -- 172.21.15.119:0/1546721714 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f84180ced40 con 0x7f8428067c00 2024-03-28T00:23:59.333 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:59.328+0000 7f8426000640 1 --2- 172.21.15.119:0/1546721714 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f840c0a2880 0x7f840c0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:23:59.334 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:59.328+0000 7f8426000640 1 --2- 172.21.15.119:0/1546721714 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f840c0a2880 0x7f840c0a4d40 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f842810b1d0 tx=0x7f8414005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:23:59.337 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:59.331+0000 7f842ca00640 1 -- 172.21.15.119:0/1546721714 <== mon.2 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f841809be90 con 0x7f8428067c00 2024-03-28T00:23:59.577 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:23:59.570+0000 7f842de00640 1 -- 172.21.15.119:0/1546721714 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) v1 -- 0x7f842802d070 con 0x7f8428067c00 2024-03-28T00:24:00.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:24:00 smithi165 ceph-mon[36302]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-03-28T00:24:00.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:24:00 smithi165 ceph-mon[36302]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-28T00:24:00.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:24:00 smithi165 ceph-mon[36302]: from='client.? 172.21.15.119:0/1546721714' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-03-28T00:24:00.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:24:00 smithi165 ceph-mon[36302]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-03-28T00:24:00.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:24:00 smithi165 ceph-mon[36302]: pgmap v313: 97 pgs: 4 creating+peering, 93 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 8 op/s 2024-03-28T00:24:00.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:24:00 smithi119 ceph-mon[31518]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-03-28T00:24:00.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:24:00 smithi119 ceph-mon[31518]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-28T00:24:00.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:24:00 smithi119 ceph-mon[31518]: from='client.? 172.21.15.119:0/1546721714' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-03-28T00:24:00.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:24:00 smithi119 ceph-mon[31518]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-03-28T00:24:00.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:24:00 smithi119 ceph-mon[31518]: pgmap v313: 97 pgs: 4 creating+peering, 93 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 8 op/s 2024-03-28T00:24:00.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:24:00 smithi132 ceph-mon[35941]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-03-28T00:24:00.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:24:00 smithi132 ceph-mon[35941]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-28T00:24:00.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:24:00 smithi132 ceph-mon[35941]: from='client.? 172.21.15.119:0/1546721714' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-03-28T00:24:00.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:24:00 smithi132 ceph-mon[35941]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-03-28T00:24:00.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:24:00 smithi132 ceph-mon[35941]: pgmap v313: 97 pgs: 4 creating+peering, 93 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 8 op/s 2024-03-28T00:24:00.472 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:00.465+0000 7f842ca00640 1 -- 172.21.15.119:0/1546721714 <== mon.2 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v18) v1 ==== 105+0+0 (secure 0 0 0) 0x7f842802d070 con 0x7f8428067c00 2024-03-28T00:24:00.475 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:00.469+0000 7f842de00640 1 -- 172.21.15.119:0/1546721714 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f840c0a2880 msgr2=0x7f840c0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:24:00.475 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:00.469+0000 7f842de00640 1 --2- 172.21.15.119:0/1546721714 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f840c0a2880 0x7f840c0a4d40 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f842810b1d0 tx=0x7f8414005f90 comp rx=0 tx=0).stop 2024-03-28T00:24:00.475 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:00.469+0000 7f842de00640 1 -- 172.21.15.119:0/1546721714 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8428067c00 msgr2=0x7f842810b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:24:00.475 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:00.469+0000 7f842de00640 1 --2- 172.21.15.119:0/1546721714 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8428067c00 0x7f842810b460 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f841800e9c0 tx=0x7f841800ee80 comp rx=0 tx=0).stop 2024-03-28T00:24:00.475 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:00.469+0000 7f842de00640 1 -- 172.21.15.119:0/1546721714 shutdown_connections 2024-03-28T00:24:00.475 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:00.469+0000 7f842de00640 1 --2- 172.21.15.119:0/1546721714 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f840c0a2880 0x7f840c0a4d40 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:00.476 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:00.469+0000 7f842de00640 1 --2- 172.21.15.119:0/1546721714 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8428105d20 0x7f8428109fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:00.476 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:00.469+0000 7f842de00640 1 --2- 172.21.15.119:0/1546721714 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8428101020 0x7f8428109aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:00.476 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:00.469+0000 7f842de00640 1 --2- 172.21.15.119:0/1546721714 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8428067c00 0x7f842810b460 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:00.476 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:00.469+0000 7f842de00640 1 -- 172.21.15.119:0/1546721714 >> 172.21.15.119:0/1546721714 conn(0x7f84280fae60 msgr2=0x7f8428101d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:24:00.476 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:00.469+0000 7f842de00640 1 -- 172.21.15.119:0/1546721714 shutdown_connections 2024-03-28T00:24:00.476 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:00.469+0000 7f842de00640 1 -- 172.21.15.119:0/1546721714 wait complete. 2024-03-28T00:24:00.503 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-03-28T00:24:00.700 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:00.693+0000 7f0a0bc00640 1 -- 172.21.15.119:0/2879274040 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0a04108eb0 msgr2=0x7f0a04109330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:24:00.700 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:00.693+0000 7f0a0bc00640 1 --2- 172.21.15.119:0/2879274040 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0a04108eb0 0x7f0a04109330 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f09ec009900 tx=0x7f09ec02bc40 comp rx=0 tx=0).stop 2024-03-28T00:24:00.700 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:00.693+0000 7f0a0bc00640 1 -- 172.21.15.119:0/2879274040 shutdown_connections 2024-03-28T00:24:00.700 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:00.693+0000 7f0a0bc00640 1 --2- 172.21.15.119:0/2879274040 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0a04109870 0x7f0a04110100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:00.700 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:00.693+0000 7f0a0bc00640 1 --2- 172.21.15.119:0/2879274040 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0a04108eb0 0x7f0a04109330 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:00.701 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:00.693+0000 7f0a0bc00640 1 --2- 172.21.15.119:0/2879274040 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0a04107cc0 0x7f0a041080e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:00.701 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:00.693+0000 7f0a0bc00640 1 -- 172.21.15.119:0/2879274040 >> 172.21.15.119:0/2879274040 conn(0x7f0a04076c60 msgr2=0x7f0a040790a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:24:00.701 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:00.694+0000 7f0a0bc00640 1 -- 172.21.15.119:0/2879274040 shutdown_connections 2024-03-28T00:24:00.701 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:00.694+0000 7f0a0bc00640 1 -- 172.21.15.119:0/2879274040 wait complete. 2024-03-28T00:24:00.704 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:00.695+0000 7f0a0bc00640 1 Processor -- start 2024-03-28T00:24:00.704 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:00.695+0000 7f0a0bc00640 1 -- start start 2024-03-28T00:24:00.705 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:00.695+0000 7f0a0bc00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0a04107cc0 0x7f0a0419dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:24:00.705 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:00.695+0000 7f0a0bc00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0a04108eb0 0x7f0a0419e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:24:00.705 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:00.695+0000 7f0a0bc00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0a04109870 0x7f0a041a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:24:00.705 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:00.695+0000 7f0a0bc00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a04113990 con 0x7f0a04108eb0 2024-03-28T00:24:00.705 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:00.695+0000 7f0a0bc00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a04113810 con 0x7f0a04107cc0 2024-03-28T00:24:00.705 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:00.695+0000 7f0a0bc00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a04113b10 con 0x7f0a04109870 2024-03-28T00:24:00.705 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:00.695+0000 7f0a09200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0a04107cc0 0x7f0a0419dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:24:00.705 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:00.695+0000 7f0a09200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0a04107cc0 0x7f0a0419dfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.119:36320/0 (socket says 172.21.15.119:36320) 2024-03-28T00:24:00.705 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:00.695+0000 7f0a09200640 1 -- 172.21.15.119:0/3103272582 learned_addr learned my addr 172.21.15.119:0/3103272582 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:24:00.705 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:00.695+0000 7f09fbe00640 1 --2- 172.21.15.119:0/3103272582 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0a04108eb0 0x7f0a0419e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:24:00.705 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:00.696+0000 7f0a09c00640 1 --2- 172.21.15.119:0/3103272582 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0a04109870 0x7f0a041a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:24:00.706 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:00.696+0000 7f0a09200640 1 -- 172.21.15.119:0/3103272582 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0a04109870 msgr2=0x7f0a041a5550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:24:00.706 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:00.696+0000 7f0a09200640 1 --2- 172.21.15.119:0/3103272582 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0a04109870 0x7f0a041a5550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:00.706 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:00.696+0000 7f0a09200640 1 -- 172.21.15.119:0/3103272582 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0a04108eb0 msgr2=0x7f0a0419e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:24:00.706 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:00.696+0000 7f0a09200640 1 --2- 172.21.15.119:0/3103272582 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0a04108eb0 0x7f0a0419e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:00.706 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:00.696+0000 7f0a09200640 1 -- 172.21.15.119:0/3103272582 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a041a5bc0 con 0x7f0a04107cc0 2024-03-28T00:24:00.706 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:00.696+0000 7f0a09c00640 1 --2- 172.21.15.119:0/3103272582 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0a04109870 0x7f0a041a5550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T00:24:00.706 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:00.696+0000 7f0a09200640 1 --2- 172.21.15.119:0/3103272582 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0a04107cc0 0x7f0a0419dfa0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f09f400e9c0 tx=0x7f09f400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:24:00.706 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:00.696+0000 7f0a02000640 1 -- 172.21.15.119:0/3103272582 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09f400cd80 con 0x7f0a04107cc0 2024-03-28T00:24:00.707 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:00.696+0000 7f0a02000640 1 -- 172.21.15.119:0/3103272582 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f09f40035f0 con 0x7f0a04107cc0 2024-03-28T00:24:00.707 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:00.696+0000 7f0a0bc00640 1 -- 172.21.15.119:0/3103272582 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a041a5eb0 con 0x7f0a04107cc0 2024-03-28T00:24:00.707 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:00.696+0000 7f0a02000640 1 -- 172.21.15.119:0/3103272582 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09f4010690 con 0x7f0a04107cc0 2024-03-28T00:24:00.707 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:00.697+0000 7f0a0bc00640 1 -- 172.21.15.119:0/3103272582 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a041a6320 con 0x7f0a04107cc0 2024-03-28T00:24:00.707 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:00.698+0000 7f0a0bc00640 1 -- 172.21.15.119:0/3103272582 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a0410efe0 con 0x7f0a04107cc0 2024-03-28T00:24:00.708 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:00.699+0000 7f0a02000640 1 -- 172.21.15.119:0/3103272582 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f09f40037a0 con 0x7f0a04107cc0 2024-03-28T00:24:00.708 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:00.701+0000 7f0a02000640 1 --2- 172.21.15.119:0/3103272582 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f09e80a2880 0x7f09e80a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:24:00.708 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:00.701+0000 7f0a02000640 1 -- 172.21.15.119:0/3103272582 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f09f40ced70 con 0x7f0a04107cc0 2024-03-28T00:24:00.709 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:00.701+0000 7f09fbe00640 1 --2- 172.21.15.119:0/3103272582 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f09e80a2880 0x7f09e80a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:24:00.709 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:00.702+0000 7f09fbe00640 1 --2- 172.21.15.119:0/3103272582 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f09e80a2880 0x7f09e80a4d40 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f09ec02c250 tx=0x7f09ec02cf90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:24:00.710 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:00.705+0000 7f0a02000640 1 -- 172.21.15.119:0/3103272582 <== mon.2 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f09f409bec0 con 0x7f0a04107cc0 2024-03-28T00:24:00.949 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:00.943+0000 7f0a0bc00640 1 -- 172.21.15.119:0/3103272582 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) v1 -- 0x7f0a04062370 con 0x7f0a04107cc0 2024-03-28T00:24:00.952 INFO:journalctl@ceph.mds.b.smithi132.stdout:Mar 28 00:24:00 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-b[66443]: 2024-03-28T00:24:00.524+0000 7f857cecc700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-28T00:24:01.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:24:01 smithi165 ceph-mon[36302]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-28T00:24:01.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:24:01 smithi165 ceph-mon[36302]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-03-28T00:24:01.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:24:01 smithi165 ceph-mon[36302]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-03-28T00:24:01.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:24:01 smithi165 ceph-mon[36302]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-28T00:24:01.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:24:01 smithi165 ceph-mon[36302]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-03-28T00:24:01.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:24:01 smithi165 ceph-mon[36302]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-03-28T00:24:01.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:24:01 smithi165 ceph-mon[36302]: from='client.? 172.21.15.119:0/3103272582' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-28T00:24:01.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:24:01 smithi165 ceph-mon[36302]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-28T00:24:01.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:24:01 smithi119 ceph-mon[31518]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-28T00:24:01.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:24:01 smithi119 ceph-mon[31518]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-03-28T00:24:01.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:24:01 smithi119 ceph-mon[31518]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-03-28T00:24:01.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:24:01 smithi119 ceph-mon[31518]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-28T00:24:01.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:24:01 smithi119 ceph-mon[31518]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-03-28T00:24:01.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:24:01 smithi119 ceph-mon[31518]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-03-28T00:24:01.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:24:01 smithi119 ceph-mon[31518]: from='client.? 172.21.15.119:0/3103272582' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-28T00:24:01.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:24:01 smithi119 ceph-mon[31518]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-28T00:24:01.452 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:24:01 smithi132 ceph-mon[35941]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-28T00:24:01.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:24:01 smithi132 ceph-mon[35941]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-03-28T00:24:01.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:24:01 smithi132 ceph-mon[35941]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-03-28T00:24:01.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:24:01 smithi132 ceph-mon[35941]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-28T00:24:01.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:24:01 smithi132 ceph-mon[35941]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-03-28T00:24:01.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:24:01 smithi132 ceph-mon[35941]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-03-28T00:24:01.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:24:01 smithi132 ceph-mon[35941]: from='client.? 172.21.15.119:0/3103272582' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-28T00:24:01.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:24:01 smithi132 ceph-mon[35941]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-28T00:24:01.485 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:01.479+0000 7f0a02000640 1 -- 172.21.15.119:0/3103272582 <== mon.2 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v20) v1 ==== 121+0+0 (secure 0 0 0) 0x7f09f409b7c0 con 0x7f0a04107cc0 2024-03-28T00:24:01.489 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:01.482+0000 7f0a0bc00640 1 -- 172.21.15.119:0/3103272582 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f09e80a2880 msgr2=0x7f09e80a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:24:01.490 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:01.482+0000 7f0a0bc00640 1 --2- 172.21.15.119:0/3103272582 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f09e80a2880 0x7f09e80a4d40 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f09ec02c250 tx=0x7f09ec02cf90 comp rx=0 tx=0).stop 2024-03-28T00:24:01.490 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:01.482+0000 7f0a0bc00640 1 -- 172.21.15.119:0/3103272582 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0a04107cc0 msgr2=0x7f0a0419dfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:24:01.490 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:01.482+0000 7f0a0bc00640 1 --2- 172.21.15.119:0/3103272582 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0a04107cc0 0x7f0a0419dfa0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f09f400e9c0 tx=0x7f09f400ee80 comp rx=0 tx=0).stop 2024-03-28T00:24:01.490 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:01.483+0000 7f0a0bc00640 1 -- 172.21.15.119:0/3103272582 shutdown_connections 2024-03-28T00:24:01.490 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:01.483+0000 7f0a0bc00640 1 --2- 172.21.15.119:0/3103272582 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f09e80a2880 0x7f09e80a4d40 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:01.490 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:01.483+0000 7f0a0bc00640 1 --2- 172.21.15.119:0/3103272582 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0a04109870 0x7f0a041a5550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:01.490 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:01.483+0000 7f0a0bc00640 1 --2- 172.21.15.119:0/3103272582 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0a04108eb0 0x7f0a0419e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:01.490 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:01.483+0000 7f0a0bc00640 1 --2- 172.21.15.119:0/3103272582 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0a04107cc0 0x7f0a0419dfa0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:01.490 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:01.483+0000 7f0a0bc00640 1 -- 172.21.15.119:0/3103272582 >> 172.21.15.119:0/3103272582 conn(0x7f0a04076c60 msgr2=0x7f0a0410c1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:24:01.490 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:01.483+0000 7f0a0bc00640 1 -- 172.21.15.119:0/3103272582 shutdown_connections 2024-03-28T00:24:01.490 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:01.483+0000 7f0a0bc00640 1 -- 172.21.15.119:0/3103272582 wait complete. 2024-03-28T00:24:01.516 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-03-28T00:24:01.714 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:01.707+0000 7f286b400640 1 -- 172.21.15.119:0/2838938986 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2864104ad0 msgr2=0x7f2864104f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:24:01.714 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:01.707+0000 7f286b400640 1 --2- 172.21.15.119:0/2838938986 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2864104ad0 0x7f2864104f50 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f2854009900 tx=0x7f285402bc30 comp rx=0 tx=0).stop 2024-03-28T00:24:01.714 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:01.707+0000 7f286b400640 1 -- 172.21.15.119:0/2838938986 shutdown_connections 2024-03-28T00:24:01.714 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:01.707+0000 7f286b400640 1 --2- 172.21.15.119:0/2838938986 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2864105490 0x7f286410bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:01.714 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:01.707+0000 7f286b400640 1 --2- 172.21.15.119:0/2838938986 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2864104ad0 0x7f2864104f50 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:01.714 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:01.707+0000 7f286b400640 1 --2- 172.21.15.119:0/2838938986 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f28641038e0 0x7f2864103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:01.714 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:01.707+0000 7f286b400640 1 -- 172.21.15.119:0/2838938986 >> 172.21.15.119:0/2838938986 conn(0x7f28640fee50 msgr2=0x7f2864101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:24:01.714 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:01.707+0000 7f286b400640 1 -- 172.21.15.119:0/2838938986 shutdown_connections 2024-03-28T00:24:01.714 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:01.707+0000 7f286b400640 1 -- 172.21.15.119:0/2838938986 wait complete. 2024-03-28T00:24:01.716 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:01.708+0000 7f286b400640 1 Processor -- start 2024-03-28T00:24:01.717 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:01.708+0000 7f286b400640 1 -- start start 2024-03-28T00:24:01.717 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:01.709+0000 7f286b400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f28641038e0 0x7f2864199c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:24:01.717 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:01.709+0000 7f286b400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2864104ad0 0x7f286419a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:24:01.717 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:01.709+0000 7f286b400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2864105490 0x7f28641a11e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:24:01.717 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:01.709+0000 7f286b400640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f286410f5b0 con 0x7f28641038e0 2024-03-28T00:24:01.717 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:01.709+0000 7f286b400640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f286410f430 con 0x7f2864105490 2024-03-28T00:24:01.717 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:01.709+0000 7f286b400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f286410f730 con 0x7f2864104ad0 2024-03-28T00:24:01.717 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:01.709+0000 7f2863e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2864104ad0 0x7f286419a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:24:01.717 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:01.709+0000 7f2869400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2864105490 0x7f28641a11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:24:01.717 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:01.709+0000 7f2868a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f28641038e0 0x7f2864199c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:24:01.717 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:01.709+0000 7f2863e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2864104ad0 0x7f286419a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.119:42604/0 (socket says 172.21.15.119:42604) 2024-03-28T00:24:01.717 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:01.709+0000 7f2863e00640 1 -- 172.21.15.119:0/4073588327 learned_addr learned my addr 172.21.15.119:0/4073588327 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:24:01.718 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:01.709+0000 7f2863e00640 1 -- 172.21.15.119:0/4073588327 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2864105490 msgr2=0x7f28641a11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:24:01.718 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:01.709+0000 7f2863e00640 1 --2- 172.21.15.119:0/4073588327 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2864105490 0x7f28641a11e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:01.718 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:01.709+0000 7f2863e00640 1 -- 172.21.15.119:0/4073588327 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f28641038e0 msgr2=0x7f2864199c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:24:01.718 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:01.709+0000 7f2863e00640 1 --2- 172.21.15.119:0/4073588327 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f28641038e0 0x7f2864199c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:01.718 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:01.710+0000 7f2863e00640 1 -- 172.21.15.119:0/4073588327 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28641a1880 con 0x7f2864104ad0 2024-03-28T00:24:01.718 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:01.710+0000 7f2869400640 1 --2- 172.21.15.119:0/4073588327 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2864105490 0x7f28641a11e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T00:24:01.718 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:01.710+0000 7f2863e00640 1 --2- 172.21.15.119:0/4073588327 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2864104ad0 0x7f286419a1b0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f2854000c00 tx=0x7f285402c650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:24:01.718 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:01.710+0000 7f2861600640 1 -- 172.21.15.119:0/4073588327 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2854043020 con 0x7f2864104ad0 2024-03-28T00:24:01.718 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:01.710+0000 7f286b400640 1 -- 172.21.15.119:0/4073588327 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28641a1b10 con 0x7f2864104ad0 2024-03-28T00:24:01.718 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:01.710+0000 7f2861600640 1 -- 172.21.15.119:0/4073588327 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f28540054e0 con 0x7f2864104ad0 2024-03-28T00:24:01.718 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:01.710+0000 7f2861600640 1 -- 172.21.15.119:0/4073588327 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2854003c40 con 0x7f2864104ad0 2024-03-28T00:24:01.719 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:01.710+0000 7f286b400640 1 -- 172.21.15.119:0/4073588327 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28641a20b0 con 0x7f2864104ad0 2024-03-28T00:24:01.719 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:01.711+0000 7f286b400640 1 -- 172.21.15.119:0/4073588327 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f286410ac20 con 0x7f2864104ad0 2024-03-28T00:24:01.720 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:01.713+0000 7f2861600640 1 -- 172.21.15.119:0/4073588327 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f2854046020 con 0x7f2864104ad0 2024-03-28T00:24:01.720 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:01.714+0000 7f2861600640 1 --2- 172.21.15.119:0/4073588327 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f284c0a2800 0x7f284c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:24:01.721 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:01.715+0000 7f2861600640 1 -- 172.21.15.119:0/4073588327 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f28540f0a80 con 0x7f2864104ad0 2024-03-28T00:24:01.721 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:01.715+0000 7f2868a00640 1 --2- 172.21.15.119:0/4073588327 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f284c0a2800 0x7f284c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:24:01.721 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:01.715+0000 7f2868a00640 1 --2- 172.21.15.119:0/4073588327 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f284c0a2800 0x7f284c0a4cc0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f286419b140 tx=0x7f2858005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:24:01.728 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:01.723+0000 7f2861600640 1 -- 172.21.15.119:0/4073588327 <== mon.1 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f28540f2e20 con 0x7f2864104ad0 2024-03-28T00:24:01.967 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:01.961+0000 7f286b400640 1 -- 172.21.15.119:0/4073588327 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) v1 -- 0x7f28640622a0 con 0x7f2864104ad0 2024-03-28T00:24:02.506 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:02.500+0000 7f2861600640 1 -- 172.21.15.119:0/4073588327 <== mon.1 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v22) v1 ==== 115+0+0 (secure 0 0 0) 0x7f285404d020 con 0x7f2864104ad0 2024-03-28T00:24:02.511 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:02.504+0000 7f286b400640 1 -- 172.21.15.119:0/4073588327 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f284c0a2800 msgr2=0x7f284c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:24:02.511 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:02.504+0000 7f286b400640 1 --2- 172.21.15.119:0/4073588327 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f284c0a2800 0x7f284c0a4cc0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f286419b140 tx=0x7f2858005f90 comp rx=0 tx=0).stop 2024-03-28T00:24:02.511 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:02.504+0000 7f286b400640 1 -- 172.21.15.119:0/4073588327 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2864104ad0 msgr2=0x7f286419a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:24:02.511 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:02.504+0000 7f286b400640 1 --2- 172.21.15.119:0/4073588327 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2864104ad0 0x7f286419a1b0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f2854000c00 tx=0x7f285402c650 comp rx=0 tx=0).stop 2024-03-28T00:24:02.511 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:02.504+0000 7f286b400640 1 -- 172.21.15.119:0/4073588327 shutdown_connections 2024-03-28T00:24:02.511 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:02.504+0000 7f286b400640 1 --2- 172.21.15.119:0/4073588327 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f284c0a2800 0x7f284c0a4cc0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:02.511 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:02.504+0000 7f286b400640 1 --2- 172.21.15.119:0/4073588327 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2864105490 0x7f28641a11e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:02.511 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:02.504+0000 7f286b400640 1 --2- 172.21.15.119:0/4073588327 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2864104ad0 0x7f286419a1b0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:02.511 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:02.504+0000 7f286b400640 1 --2- 172.21.15.119:0/4073588327 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f28641038e0 0x7f2864199c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:02.511 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:02.504+0000 7f286b400640 1 -- 172.21.15.119:0/4073588327 >> 172.21.15.119:0/4073588327 conn(0x7f28640fee50 msgr2=0x7f2864107df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:24:02.511 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:02.505+0000 7f286b400640 1 -- 172.21.15.119:0/4073588327 shutdown_connections 2024-03-28T00:24:02.511 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:02.505+0000 7f286b400640 1 -- 172.21.15.119:0/4073588327 wait complete. 2024-03-28T00:24:02.537 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--size 25000000000'} subvols for filesystem 'cephfs' 2024-03-28T00:24:02.537 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs qa 2024-03-28T00:24:02.736 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:02.726+0000 7fd153200640 1 -- 172.21.15.119:0/811106201 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd154103050 msgr2=0x7fd1541098e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:24:02.736 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:02.726+0000 7fd153200640 1 --2- 172.21.15.119:0/811106201 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd154103050 0x7fd1541098e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd14c009ca0 tx=0x7fd14c02c0f0 comp rx=0 tx=0).stop 2024-03-28T00:24:02.736 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:02.726+0000 7fd153200640 1 -- 172.21.15.119:0/811106201 shutdown_connections 2024-03-28T00:24:02.736 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:02.726+0000 7fd153200640 1 --2- 172.21.15.119:0/811106201 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd154103050 0x7fd1541098e0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:02.737 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:02.726+0000 7fd153200640 1 --2- 172.21.15.119:0/811106201 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd154102690 0x7fd154102b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:02.737 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:02.726+0000 7fd153200640 1 --2- 172.21.15.119:0/811106201 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd1541014a0 0x7fd1541018c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:02.737 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:02.726+0000 7fd153200640 1 -- 172.21.15.119:0/811106201 >> 172.21.15.119:0/811106201 conn(0x7fd1540fca50 msgr2=0x7fd1540fee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:24:02.737 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:02.726+0000 7fd153200640 1 -- 172.21.15.119:0/811106201 shutdown_connections 2024-03-28T00:24:02.737 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:02.726+0000 7fd153200640 1 -- 172.21.15.119:0/811106201 wait complete. 2024-03-28T00:24:02.737 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:02.727+0000 7fd153200640 1 Processor -- start 2024-03-28T00:24:02.737 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:02.727+0000 7fd153200640 1 -- start start 2024-03-28T00:24:02.737 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:02.727+0000 7fd153200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd1541014a0 0x7fd154199900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:24:02.738 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:02.727+0000 7fd153200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd154102690 0x7fd154199e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:24:02.738 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:02.727+0000 7fd153200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd154103050 0x7fd1541a0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:24:02.738 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:02.727+0000 7fd153200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd15410d130 con 0x7fd1541014a0 2024-03-28T00:24:02.738 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:02.727+0000 7fd153200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd15410cfb0 con 0x7fd154102690 2024-03-28T00:24:02.738 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:02.727+0000 7fd153200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd15410d2b0 con 0x7fd154103050 2024-03-28T00:24:02.738 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:02.727+0000 7fd14be00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd154102690 0x7fd154199e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:24:02.738 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:02.727+0000 7fd151e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd1541014a0 0x7fd154199900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:24:02.738 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:02.727+0000 7fd14be00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd154102690 0x7fd154199e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.119:36356/0 (socket says 172.21.15.119:36356) 2024-03-28T00:24:02.739 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:02.727+0000 7fd14be00640 1 -- 172.21.15.119:0/1685648312 learned_addr learned my addr 172.21.15.119:0/1685648312 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:24:02.739 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:02.727+0000 7fd152800640 1 --2- 172.21.15.119:0/1685648312 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd154103050 0x7fd1541a0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:24:02.739 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:02.728+0000 7fd14be00640 1 -- 172.21.15.119:0/1685648312 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd154103050 msgr2=0x7fd1541a0eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:24:02.739 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:02.728+0000 7fd14be00640 1 --2- 172.21.15.119:0/1685648312 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd154103050 0x7fd1541a0eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:02.739 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:02.728+0000 7fd14be00640 1 -- 172.21.15.119:0/1685648312 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd1541014a0 msgr2=0x7fd154199900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:24:02.739 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:02.728+0000 7fd14be00640 1 --2- 172.21.15.119:0/1685648312 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd1541014a0 0x7fd154199900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:02.739 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:02.728+0000 7fd14be00640 1 -- 172.21.15.119:0/1685648312 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1541a1520 con 0x7fd154102690 2024-03-28T00:24:02.739 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:02.728+0000 7fd152800640 1 --2- 172.21.15.119:0/1685648312 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd154103050 0x7fd1541a0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T00:24:02.739 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:02.728+0000 7fd14be00640 1 --2- 172.21.15.119:0/1685648312 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd154102690 0x7fd154199e40 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd13c00ea60 tx=0x7fd13c00ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:24:02.739 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:02.728+0000 7fd14aa00640 1 -- 172.21.15.119:0/1685648312 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd13c00cde0 con 0x7fd154102690 2024-03-28T00:24:02.740 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:02.728+0000 7fd14aa00640 1 -- 172.21.15.119:0/1685648312 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd13c0035f0 con 0x7fd154102690 2024-03-28T00:24:02.740 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:02.728+0000 7fd153200640 1 -- 172.21.15.119:0/1685648312 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1541a1810 con 0x7fd154102690 2024-03-28T00:24:02.740 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:02.728+0000 7fd14aa00640 1 -- 172.21.15.119:0/1685648312 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd13c010670 con 0x7fd154102690 2024-03-28T00:24:02.740 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:02.728+0000 7fd153200640 1 -- 172.21.15.119:0/1685648312 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1541a1d60 con 0x7fd154102690 2024-03-28T00:24:02.740 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:02.729+0000 7fd153200640 1 -- 172.21.15.119:0/1685648312 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1541a1fc0 con 0x7fd154102690 2024-03-28T00:24:02.741 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:02.732+0000 7fd14aa00640 1 -- 172.21.15.119:0/1685648312 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fd13c0037a0 con 0x7fd154102690 2024-03-28T00:24:02.742 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:02.733+0000 7fd14aa00640 1 --2- 172.21.15.119:0/1685648312 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fd1340a2800 0x7fd1340a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:24:02.742 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:02.734+0000 7fd14aa00640 1 -- 172.21.15.119:0/1685648312 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fd13c0cec60 con 0x7fd154102690 2024-03-28T00:24:02.742 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:02.734+0000 7fd151e00640 1 --2- 172.21.15.119:0/1685648312 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fd1340a2800 0x7fd1340a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:24:02.742 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:02.734+0000 7fd151e00640 1 --2- 172.21.15.119:0/1685648312 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fd1340a2800 0x7fd1340a4cc0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fd1541024f0 tx=0x7fd144005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:24:02.742 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:02.735+0000 7fd14aa00640 1 -- 172.21.15.119:0/1685648312 <== mon.2 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd13c09bdb0 con 0x7fd154102690 2024-03-28T00:24:02.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:24:02 smithi165 ceph-mon[36302]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-03-28T00:24:02.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:24:02 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.165:6838/3004684592,v1:172.21.15.165:6839/3004684592] up:active 2024-03-28T00:24:02.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:24:02 smithi165 ceph-mon[36302]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-03-28T00:24:02.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:24:02 smithi165 ceph-mon[36302]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-03-28T00:24:02.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:24:02 smithi165 ceph-mon[36302]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-03-28T00:24:02.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:24:02 smithi165 ceph-mon[36302]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-03-28T00:24:02.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:24:02 smithi165 ceph-mon[36302]: pgmap v314: 97 pgs: 97 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 7 op/s 2024-03-28T00:24:02.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:24:02 smithi165 ceph-mon[36302]: from='client.? 172.21.15.119:0/4073588327' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-28T00:24:02.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:24:02 smithi165 ceph-mon[36302]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-28T00:24:02.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:24:02 smithi119 ceph-mon[31518]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-03-28T00:24:02.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:24:02 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.165:6838/3004684592,v1:172.21.15.165:6839/3004684592] up:active 2024-03-28T00:24:02.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:24:02 smithi119 ceph-mon[31518]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-03-28T00:24:02.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:24:02 smithi119 ceph-mon[31518]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-03-28T00:24:02.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:24:02 smithi119 ceph-mon[31518]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-03-28T00:24:02.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:24:02 smithi119 ceph-mon[31518]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-03-28T00:24:02.907 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:24:02 smithi119 ceph-mon[31518]: pgmap v314: 97 pgs: 97 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 7 op/s 2024-03-28T00:24:02.907 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:24:02 smithi119 ceph-mon[31518]: from='client.? 172.21.15.119:0/4073588327' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-28T00:24:02.907 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:24:02 smithi119 ceph-mon[31518]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-28T00:24:02.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:24:02 smithi132 ceph-mon[35941]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-03-28T00:24:02.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:24:02 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.165:6838/3004684592,v1:172.21.15.165:6839/3004684592] up:active 2024-03-28T00:24:02.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:24:02 smithi132 ceph-mon[35941]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-03-28T00:24:02.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:24:02 smithi132 ceph-mon[35941]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-03-28T00:24:02.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:24:02 smithi132 ceph-mon[35941]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-03-28T00:24:02.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:24:02 smithi132 ceph-mon[35941]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-03-28T00:24:02.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:24:02 smithi132 ceph-mon[35941]: pgmap v314: 97 pgs: 97 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 7 op/s 2024-03-28T00:24:02.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:24:02 smithi132 ceph-mon[35941]: from='client.? 172.21.15.119:0/4073588327' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-28T00:24:02.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:24:02 smithi132 ceph-mon[35941]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-28T00:24:02.966 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:02.960+0000 7fd153200640 1 -- 172.21.15.119:0/1685648312 --> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7fd1541a2190 con 0x7fd1340a2800 2024-03-28T00:24:03.406 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 00:24:02 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-x[31752]: 2024-03-28T00:24:02.973+0000 7f5ddda00700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-28T00:24:03.406 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 00:24:02 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-x[31752]: 2024-03-28T00:24:02.973+0000 7f5ddda00700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-28T00:24:03.406 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 00:24:02 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-x[31752]: 2024-03-28T00:24:02.973+0000 7f5ddda00700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-28T00:24:03.406 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 00:24:02 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-x[31752]: 2024-03-28T00:24:02.973+0000 7f5ddda00700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-28T00:24:03.406 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 00:24:02 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-x[31752]: 2024-03-28T00:24:02.973+0000 7f5ddda00700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-28T00:24:03.558 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:03.551+0000 7fd14aa00640 1 -- 172.21.15.119:0/1685648312 <== mgr.14152 v2:172.21.15.119:6800/1401937096 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fd1541a2190 con 0x7fd1340a2800 2024-03-28T00:24:03.562 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:03.555+0000 7fd153200640 1 -- 172.21.15.119:0/1685648312 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fd1340a2800 msgr2=0x7fd1340a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:24:03.562 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:03.555+0000 7fd153200640 1 --2- 172.21.15.119:0/1685648312 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fd1340a2800 0x7fd1340a4cc0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fd1541024f0 tx=0x7fd144005f90 comp rx=0 tx=0).stop 2024-03-28T00:24:03.562 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:03.555+0000 7fd153200640 1 -- 172.21.15.119:0/1685648312 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd154102690 msgr2=0x7fd154199e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:24:03.562 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:03.555+0000 7fd153200640 1 --2- 172.21.15.119:0/1685648312 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd154102690 0x7fd154199e40 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd13c00ea60 tx=0x7fd13c00ef20 comp rx=0 tx=0).stop 2024-03-28T00:24:03.562 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:03.555+0000 7fd153200640 1 -- 172.21.15.119:0/1685648312 shutdown_connections 2024-03-28T00:24:03.562 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:03.555+0000 7fd153200640 1 --2- 172.21.15.119:0/1685648312 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fd1340a2800 0x7fd1340a4cc0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:03.562 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:03.555+0000 7fd153200640 1 --2- 172.21.15.119:0/1685648312 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd154103050 0x7fd1541a0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:03.562 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:03.555+0000 7fd153200640 1 --2- 172.21.15.119:0/1685648312 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd154102690 0x7fd154199e40 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:03.562 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:03.555+0000 7fd153200640 1 --2- 172.21.15.119:0/1685648312 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd1541014a0 0x7fd154199900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:03.562 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:03.555+0000 7fd153200640 1 -- 172.21.15.119:0/1685648312 >> 172.21.15.119:0/1685648312 conn(0x7fd1540fca50 msgr2=0x7fd1540738d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:24:03.562 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:03.556+0000 7fd153200640 1 -- 172.21.15.119:0/1685648312 shutdown_connections 2024-03-28T00:24:03.562 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:03.556+0000 7fd153200640 1 -- 172.21.15.119:0/1685648312 wait complete. 2024-03-28T00:24:03.583 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 --group_name qa 2024-03-28T00:24:03.806 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:03.799+0000 7fdf4c400640 1 -- 172.21.15.119:0/3408089385 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdf441038e0 msgr2=0x7fdf44103d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:24:03.806 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:03.799+0000 7fdf4c400640 1 --2- 172.21.15.119:0/3408089385 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdf441038e0 0x7fdf44103d00 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fdf38009900 tx=0x7fdf3802bc30 comp rx=0 tx=0).stop 2024-03-28T00:24:03.806 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:03.799+0000 7fdf4c400640 1 -- 172.21.15.119:0/3408089385 shutdown_connections 2024-03-28T00:24:03.806 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:03.799+0000 7fdf4c400640 1 --2- 172.21.15.119:0/3408089385 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdf44105490 0x7fdf4410bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:03.806 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:03.799+0000 7fdf4c400640 1 --2- 172.21.15.119:0/3408089385 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdf44104ad0 0x7fdf44104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:03.806 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:03.799+0000 7fdf4c400640 1 --2- 172.21.15.119:0/3408089385 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdf441038e0 0x7fdf44103d00 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:03.806 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:03.799+0000 7fdf4c400640 1 -- 172.21.15.119:0/3408089385 >> 172.21.15.119:0/3408089385 conn(0x7fdf440fee50 msgr2=0x7fdf44101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:24:03.806 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:03.799+0000 7fdf4c400640 1 -- 172.21.15.119:0/3408089385 shutdown_connections 2024-03-28T00:24:03.807 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:03.799+0000 7fdf4c400640 1 -- 172.21.15.119:0/3408089385 wait complete. 2024-03-28T00:24:03.809 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:03.800+0000 7fdf4c400640 1 Processor -- start 2024-03-28T00:24:03.809 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:03.800+0000 7fdf4c400640 1 -- start start 2024-03-28T00:24:03.809 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:03.801+0000 7fdf4c400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdf441038e0 0x7fdf44199c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:24:03.809 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:03.801+0000 7fdf4c400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdf44104ad0 0x7fdf4419a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:24:03.809 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:03.801+0000 7fdf4c400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdf44105490 0x7fdf441a11f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:24:03.809 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:03.801+0000 7fdf4c400640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf4410f530 con 0x7fdf441038e0 2024-03-28T00:24:03.809 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:03.801+0000 7fdf4c400640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf4410f3b0 con 0x7fdf44104ad0 2024-03-28T00:24:03.809 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:03.801+0000 7fdf4c400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf4410f6b0 con 0x7fdf44105490 2024-03-28T00:24:03.809 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:03.801+0000 7fdf49a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdf441038e0 0x7fdf44199c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:24:03.809 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:03.801+0000 7fdf4a400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdf44105490 0x7fdf441a11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:24:03.809 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:03.801+0000 7fdf4a400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdf44105490 0x7fdf441a11f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.119:42656/0 (socket says 172.21.15.119:42656) 2024-03-28T00:24:03.810 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:03.801+0000 7fdf49000640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdf44104ad0 0x7fdf4419a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:24:03.810 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:03.801+0000 7fdf4a400640 1 -- 172.21.15.119:0/2719453942 learned_addr learned my addr 172.21.15.119:0/2719453942 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:24:03.810 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:03.801+0000 7fdf4a400640 1 -- 172.21.15.119:0/2719453942 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdf44104ad0 msgr2=0x7fdf4419a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:24:03.810 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:03.801+0000 7fdf4a400640 1 --2- 172.21.15.119:0/2719453942 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdf44104ad0 0x7fdf4419a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:03.810 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:03.801+0000 7fdf4a400640 1 -- 172.21.15.119:0/2719453942 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdf441038e0 msgr2=0x7fdf44199c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:24:03.810 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:03.801+0000 7fdf4a400640 1 --2- 172.21.15.119:0/2719453942 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdf441038e0 0x7fdf44199c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:03.810 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:03.801+0000 7fdf4a400640 1 -- 172.21.15.119:0/2719453942 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf441a1890 con 0x7fdf44105490 2024-03-28T00:24:03.810 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:03.802+0000 7fdf4a400640 1 --2- 172.21.15.119:0/2719453942 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdf44105490 0x7fdf441a11f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fdf3c00efd0 tx=0x7fdf3c00c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:24:03.810 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:03.802+0000 7fdf42000640 1 -- 172.21.15.119:0/2719453942 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf3c003b70 con 0x7fdf44105490 2024-03-28T00:24:03.810 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:03.802+0000 7fdf42000640 1 -- 172.21.15.119:0/2719453942 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdf3c003d10 con 0x7fdf44105490 2024-03-28T00:24:03.810 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:03.802+0000 7fdf42000640 1 -- 172.21.15.119:0/2719453942 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf3c0109f0 con 0x7fdf44105490 2024-03-28T00:24:03.810 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:03.802+0000 7fdf4c400640 1 -- 172.21.15.119:0/2719453942 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf441a1b80 con 0x7fdf44105490 2024-03-28T00:24:03.811 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:03.802+0000 7fdf4c400640 1 -- 172.21.15.119:0/2719453942 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf441a20a0 con 0x7fdf44105490 2024-03-28T00:24:03.811 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:03.804+0000 7fdf4c400640 1 -- 172.21.15.119:0/2719453942 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf4410ad70 con 0x7fdf44105490 2024-03-28T00:24:03.811 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:03.806+0000 7fdf42000640 1 -- 172.21.15.119:0/2719453942 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fdf3c004020 con 0x7fdf44105490 2024-03-28T00:24:03.813 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:03.807+0000 7fdf42000640 1 --2- 172.21.15.119:0/2719453942 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fdf2c0a2880 0x7fdf2c0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:24:03.813 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:03.807+0000 7fdf42000640 1 -- 172.21.15.119:0/2719453942 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fdf3c0cee30 con 0x7fdf44105490 2024-03-28T00:24:03.813 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:03.807+0000 7fdf49a00640 1 --2- 172.21.15.119:0/2719453942 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fdf2c0a2880 0x7fdf2c0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:24:03.814 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:03.808+0000 7fdf49a00640 1 --2- 172.21.15.119:0/2719453942 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fdf2c0a2880 0x7fdf2c0a4d40 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fdf38000c00 tx=0x7fdf38005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:24:03.817 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:03.811+0000 7fdf42000640 1 -- 172.21.15.119:0/2719453942 <== mon.1 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fdf3c09bf80 con 0x7fdf44105490 2024-03-28T00:24:03.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:24:03 smithi165 ceph-mon[36302]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-03-28T00:24:03.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:24:03 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.132:6836/3383082967,v1:172.21.15.132:6837/3383082967] up:active 2024-03-28T00:24:03.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:24:03 smithi165 ceph-mon[36302]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-03-28T00:24:03.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:24:03 smithi165 ceph-mon[36302]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-03-28T00:24:03.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:24:03 smithi165 ceph-mon[36302]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:creating} 8 up:standby 2024-03-28T00:24:03.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:24:03 smithi165 ceph-mon[36302]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-03-28T00:24:03.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:24:03 smithi165 ceph-mon[36302]: from='client.24662 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T00:24:03.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:24:03 smithi119 ceph-mon[31518]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-03-28T00:24:03.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:24:03 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.132:6836/3383082967,v1:172.21.15.132:6837/3383082967] up:active 2024-03-28T00:24:03.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:24:03 smithi119 ceph-mon[31518]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-03-28T00:24:03.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:24:03 smithi119 ceph-mon[31518]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-03-28T00:24:03.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:24:03 smithi119 ceph-mon[31518]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:creating} 8 up:standby 2024-03-28T00:24:03.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:24:03 smithi119 ceph-mon[31518]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-03-28T00:24:03.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:24:03 smithi119 ceph-mon[31518]: from='client.24662 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T00:24:03.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:24:03 smithi132 ceph-mon[35941]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-03-28T00:24:03.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:24:03 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.132:6836/3383082967,v1:172.21.15.132:6837/3383082967] up:active 2024-03-28T00:24:03.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:24:03 smithi132 ceph-mon[35941]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-03-28T00:24:03.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:24:03 smithi132 ceph-mon[35941]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-03-28T00:24:03.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:24:03 smithi132 ceph-mon[35941]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:creating} 8 up:standby 2024-03-28T00:24:03.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:24:03 smithi132 ceph-mon[35941]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-03-28T00:24:03.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:24:03 smithi132 ceph-mon[35941]: from='client.24662 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T00:24:04.045 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:04.039+0000 7fdf4c400640 1 -- 172.21.15.119:0/2719453942 --> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7fdf44061ae0 con 0x7fdf2c0a2880 2024-03-28T00:24:04.080 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:04.074+0000 7fdf42000640 1 -- 172.21.15.119:0/2719453942 <== mgr.14152 v2:172.21.15.119:6800/1401937096 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fdf44061ae0 con 0x7fdf2c0a2880 2024-03-28T00:24:04.084 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:04.077+0000 7fdf4c400640 1 -- 172.21.15.119:0/2719453942 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fdf2c0a2880 msgr2=0x7fdf2c0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:24:04.084 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:04.077+0000 7fdf4c400640 1 --2- 172.21.15.119:0/2719453942 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fdf2c0a2880 0x7fdf2c0a4d40 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fdf38000c00 tx=0x7fdf38005fd0 comp rx=0 tx=0).stop 2024-03-28T00:24:04.084 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:04.077+0000 7fdf4c400640 1 -- 172.21.15.119:0/2719453942 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdf44105490 msgr2=0x7fdf441a11f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:24:04.084 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:04.077+0000 7fdf4c400640 1 --2- 172.21.15.119:0/2719453942 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdf44105490 0x7fdf441a11f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fdf3c00efd0 tx=0x7fdf3c00c6a0 comp rx=0 tx=0).stop 2024-03-28T00:24:04.084 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:04.077+0000 7fdf4c400640 1 -- 172.21.15.119:0/2719453942 shutdown_connections 2024-03-28T00:24:04.085 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:04.078+0000 7fdf4c400640 1 --2- 172.21.15.119:0/2719453942 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fdf2c0a2880 0x7fdf2c0a4d40 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:04.085 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:04.078+0000 7fdf4c400640 1 --2- 172.21.15.119:0/2719453942 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdf44105490 0x7fdf441a11f0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:04.085 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:04.078+0000 7fdf4c400640 1 --2- 172.21.15.119:0/2719453942 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdf44104ad0 0x7fdf4419a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:04.085 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:04.078+0000 7fdf4c400640 1 --2- 172.21.15.119:0/2719453942 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdf441038e0 0x7fdf44199c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:04.085 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:04.078+0000 7fdf4c400640 1 -- 172.21.15.119:0/2719453942 >> 172.21.15.119:0/2719453942 conn(0x7fdf440fee50 msgr2=0x7fdf44107f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:24:04.085 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:04.078+0000 7fdf4c400640 1 -- 172.21.15.119:0/2719453942 shutdown_connections 2024-03-28T00:24:04.085 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:04.078+0000 7fdf4c400640 1 -- 172.21.15.119:0/2719453942 wait complete. 2024-03-28T00:24:04.106 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-03-28T00:24:04.325 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:04.318+0000 7fc7ad800640 1 -- 172.21.15.119:0/1828884196 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc7a8109870 msgr2=0x7fc7a8110100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:24:04.325 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:04.318+0000 7fc7ad800640 1 --2- 172.21.15.119:0/1828884196 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc7a8109870 0x7fc7a8110100 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc7a0009ca0 tx=0x7fc7a002c0f0 comp rx=0 tx=0).stop 2024-03-28T00:24:04.325 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:04.318+0000 7fc7ad800640 1 -- 172.21.15.119:0/1828884196 shutdown_connections 2024-03-28T00:24:04.325 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:04.318+0000 7fc7ad800640 1 --2- 172.21.15.119:0/1828884196 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc7a8109870 0x7fc7a8110100 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:04.325 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:04.318+0000 7fc7ad800640 1 --2- 172.21.15.119:0/1828884196 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc7a8108eb0 0x7fc7a8109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:04.325 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:04.318+0000 7fc7ad800640 1 --2- 172.21.15.119:0/1828884196 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc7a8107cc0 0x7fc7a81080e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:04.325 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:04.318+0000 7fc7ad800640 1 -- 172.21.15.119:0/1828884196 >> 172.21.15.119:0/1828884196 conn(0x7fc7a8076c60 msgr2=0x7fc7a80790a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:24:04.325 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:04.318+0000 7fc7ad800640 1 -- 172.21.15.119:0/1828884196 shutdown_connections 2024-03-28T00:24:04.325 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:04.318+0000 7fc7ad800640 1 -- 172.21.15.119:0/1828884196 wait complete. 2024-03-28T00:24:04.328 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:04.320+0000 7fc7ad800640 1 Processor -- start 2024-03-28T00:24:04.328 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:04.320+0000 7fc7ad800640 1 -- start start 2024-03-28T00:24:04.328 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:04.320+0000 7fc7ad800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc7a8107cc0 0x7fc7a819df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:24:04.328 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:04.320+0000 7fc7ad800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc7a8108eb0 0x7fc7a819e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:24:04.328 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:04.320+0000 7fc7ad800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc7a8109870 0x7fc7a81a5510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:24:04.328 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:04.320+0000 7fc7ad800640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7a8113920 con 0x7fc7a8108eb0 2024-03-28T00:24:04.328 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:04.320+0000 7fc7ad800640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7a81137a0 con 0x7fc7a8109870 2024-03-28T00:24:04.328 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:04.320+0000 7fc7ad800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7a8113aa0 con 0x7fc7a8107cc0 2024-03-28T00:24:04.328 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:04.320+0000 7fc7a6a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc7a8107cc0 0x7fc7a819df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:24:04.328 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:04.320+0000 7fc7a7400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc7a8109870 0x7fc7a81a5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:24:04.329 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:04.320+0000 7fc7a6a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc7a8107cc0 0x7fc7a819df60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.119:36884/0 (socket says 172.21.15.119:36884) 2024-03-28T00:24:04.329 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:04.320+0000 7fc7a6a00640 1 -- 172.21.15.119:0/1331291861 learned_addr learned my addr 172.21.15.119:0/1331291861 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:24:04.329 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:04.320+0000 7fc7a7400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc7a8109870 0x7fc7a81a5510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.119:53950/0 (socket says 172.21.15.119:53950) 2024-03-28T00:24:04.329 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:04.320+0000 7fc79fe00640 1 --2- 172.21.15.119:0/1331291861 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc7a8108eb0 0x7fc7a819e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:24:04.329 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:04.321+0000 7fc7a7400640 1 -- 172.21.15.119:0/1331291861 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc7a8107cc0 msgr2=0x7fc7a819df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:24:04.329 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:04.321+0000 7fc7a7400640 1 --2- 172.21.15.119:0/1331291861 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc7a8107cc0 0x7fc7a819df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:04.329 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:04.321+0000 7fc7a7400640 1 -- 172.21.15.119:0/1331291861 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc7a8108eb0 msgr2=0x7fc7a819e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:24:04.329 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:04.321+0000 7fc7a7400640 1 --2- 172.21.15.119:0/1331291861 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc7a8108eb0 0x7fc7a819e4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:04.329 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:04.321+0000 7fc7a7400640 1 -- 172.21.15.119:0/1331291861 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7a81a5b80 con 0x7fc7a8109870 2024-03-28T00:24:04.329 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:04.321+0000 7fc7a6a00640 1 --2- 172.21.15.119:0/1331291861 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc7a8107cc0 0x7fc7a819df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T00:24:04.329 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:04.321+0000 7fc7a7400640 1 --2- 172.21.15.119:0/1331291861 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc7a8109870 0x7fc7a81a5510 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc7a0037040 tx=0x7fc7a0032ea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:24:04.329 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:04.321+0000 7fc79f400640 1 -- 172.21.15.119:0/1331291861 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7a0041020 con 0x7fc7a8109870 2024-03-28T00:24:04.329 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:04.321+0000 7fc79f400640 1 -- 172.21.15.119:0/1331291861 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc7a0034cd0 con 0x7fc7a8109870 2024-03-28T00:24:04.329 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:04.321+0000 7fc7ad800640 1 -- 172.21.15.119:0/1331291861 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7a81a5e10 con 0x7fc7a8109870 2024-03-28T00:24:04.330 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:04.321+0000 7fc79f400640 1 -- 172.21.15.119:0/1331291861 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7a0039040 con 0x7fc7a8109870 2024-03-28T00:24:04.330 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:04.322+0000 7fc7ad800640 1 -- 172.21.15.119:0/1331291861 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7a81a6270 con 0x7fc7a8109870 2024-03-28T00:24:04.330 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:04.323+0000 7fc7ad800640 1 -- 172.21.15.119:0/1331291861 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7a810efe0 con 0x7fc7a8109870 2024-03-28T00:24:04.330 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:04.324+0000 7fc79f400640 1 -- 172.21.15.119:0/1331291861 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fc7a0044020 con 0x7fc7a8109870 2024-03-28T00:24:04.332 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:04.326+0000 7fc79f400640 1 --2- 172.21.15.119:0/1331291861 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fc78c0a2800 0x7fc78c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:24:04.332 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:04.326+0000 7fc79f400640 1 -- 172.21.15.119:0/1331291861 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fc7a00f07d0 con 0x7fc7a8109870 2024-03-28T00:24:04.332 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:04.326+0000 7fc7a6a00640 1 --2- 172.21.15.119:0/1331291861 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fc78c0a2800 0x7fc78c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:24:04.333 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:04.327+0000 7fc7a6a00640 1 --2- 172.21.15.119:0/1331291861 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fc78c0a2800 0x7fc78c0a4cc0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc7980058c0 tx=0x7fc798005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:24:04.340 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:04.334+0000 7fc79f400640 1 -- 172.21.15.119:0/1331291861 <== mon.2 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fc7a004b020 con 0x7fc7a8109870 2024-03-28T00:24:04.553 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:04.547+0000 7fc79f400640 1 -- 172.21.15.119:0/1331291861 <== mon.2 v2:172.21.15.132:3300/0 7 ==== mgrmap(e 17) v1 ==== 137342+0+0 (secure 0 0 0) 0x7fc7a0003970 con 0x7fc7a8109870 2024-03-28T00:24:04.591 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:04.585+0000 7fc7ad800640 1 -- 172.21.15.119:0/1331291861 --> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) v1 -- 0x7fc7a8061b70 con 0x7fc78c0a2800 2024-03-28T00:24:04.603 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:04.598+0000 7fc79f400640 1 -- 172.21.15.119:0/1331291861 <== mgr.14152 v2:172.21.15.119:6800/1401937096 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+54 (secure 0 0 0) 0x7fc7a8061b70 con 0x7fc78c0a2800 2024-03-28T00:24:04.604 INFO:teuthology.orchestra.run.smithi119.stdout:/volumes/qa/sv_0/6ee05ea5-1aed-4c3a-8887-bb89ba7a3b36 2024-03-28T00:24:04.606 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:04.600+0000 7fc7ad800640 1 -- 172.21.15.119:0/1331291861 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fc78c0a2800 msgr2=0x7fc78c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:24:04.607 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:04.600+0000 7fc7ad800640 1 --2- 172.21.15.119:0/1331291861 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fc78c0a2800 0x7fc78c0a4cc0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc7980058c0 tx=0x7fc798005f90 comp rx=0 tx=0).stop 2024-03-28T00:24:04.607 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:04.600+0000 7fc7ad800640 1 -- 172.21.15.119:0/1331291861 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc7a8109870 msgr2=0x7fc7a81a5510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:24:04.607 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:04.600+0000 7fc7ad800640 1 --2- 172.21.15.119:0/1331291861 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc7a8109870 0x7fc7a81a5510 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc7a0037040 tx=0x7fc7a0032ea0 comp rx=0 tx=0).stop 2024-03-28T00:24:04.607 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:04.600+0000 7fc7ad800640 1 -- 172.21.15.119:0/1331291861 shutdown_connections 2024-03-28T00:24:04.607 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:04.600+0000 7fc7ad800640 1 --2- 172.21.15.119:0/1331291861 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fc78c0a2800 0x7fc78c0a4cc0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:04.607 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:04.600+0000 7fc7ad800640 1 --2- 172.21.15.119:0/1331291861 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc7a8109870 0x7fc7a81a5510 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:04.607 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:04.600+0000 7fc7ad800640 1 --2- 172.21.15.119:0/1331291861 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc7a8108eb0 0x7fc7a819e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:04.607 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:04.600+0000 7fc7ad800640 1 --2- 172.21.15.119:0/1331291861 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc7a8107cc0 0x7fc7a819df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:04.607 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:04.600+0000 7fc7ad800640 1 -- 172.21.15.119:0/1331291861 >> 172.21.15.119:0/1331291861 conn(0x7fc7a8076c60 msgr2=0x7fc7a810c1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:24:04.607 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:04.600+0000 7fc7ad800640 1 -- 172.21.15.119:0/1331291861 shutdown_connections 2024-03-28T00:24:04.607 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:04.601+0000 7fc7ad800640 1 -- 172.21.15.119:0/1331291861 wait complete. 2024-03-28T00:24:04.624 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 --group_name qa 2024-03-28T00:24:04.828 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:04.822+0000 7f0f57200640 1 -- 172.21.15.119:0/1823687630 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0f581049b0 msgr2=0x7f0f58104e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:24:04.829 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:04.822+0000 7f0f57200640 1 --2- 172.21.15.119:0/1823687630 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0f581049b0 0x7f0f58104e30 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f0f40009900 tx=0x7f0f4002bc30 comp rx=0 tx=0).stop 2024-03-28T00:24:04.829 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:04.822+0000 7f0f57200640 1 -- 172.21.15.119:0/1823687630 shutdown_connections 2024-03-28T00:24:04.829 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:04.822+0000 7f0f57200640 1 --2- 172.21.15.119:0/1823687630 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f58105370 0x7f0f5810bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:04.829 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:04.822+0000 7f0f57200640 1 --2- 172.21.15.119:0/1823687630 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0f581049b0 0x7f0f58104e30 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:04.829 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:04.822+0000 7f0f57200640 1 --2- 172.21.15.119:0/1823687630 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0f581037c0 0x7f0f58103be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:04.829 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:04.822+0000 7f0f57200640 1 -- 172.21.15.119:0/1823687630 >> 172.21.15.119:0/1823687630 conn(0x7f0f580fed50 msgr2=0x7f0f58101170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:24:04.829 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:04.822+0000 7f0f57200640 1 -- 172.21.15.119:0/1823687630 shutdown_connections 2024-03-28T00:24:04.829 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:04.822+0000 7f0f57200640 1 -- 172.21.15.119:0/1823687630 wait complete. 2024-03-28T00:24:04.831 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:04.823+0000 7f0f57200640 1 Processor -- start 2024-03-28T00:24:04.832 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:04.823+0000 7f0f57200640 1 -- start start 2024-03-28T00:24:04.832 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:04.823+0000 7f0f57200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f581037c0 0x7f0f58199b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:24:04.832 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:04.823+0000 7f0f57200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0f581049b0 0x7f0f5819a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:24:04.832 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:04.823+0000 7f0f57200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0f58105370 0x7f0f581a10b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:24:04.832 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:04.823+0000 7f0f57200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f5810f490 con 0x7f0f581037c0 2024-03-28T00:24:04.832 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:04.823+0000 7f0f57200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f5810f310 con 0x7f0f58105370 2024-03-28T00:24:04.832 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:04.823+0000 7f0f57200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f5810f610 con 0x7f0f581049b0 2024-03-28T00:24:04.832 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:04.824+0000 7f0f55e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f581037c0 0x7f0f58199b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:24:04.832 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:04.824+0000 7f0f55400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0f581049b0 0x7f0f5819a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:24:04.832 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:04.824+0000 7f0f56800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0f58105370 0x7f0f581a10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:24:04.832 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:04.824+0000 7f0f55400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0f581049b0 0x7f0f5819a040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.119:36890/0 (socket says 172.21.15.119:36890) 2024-03-28T00:24:04.832 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:04.824+0000 7f0f55400640 1 -- 172.21.15.119:0/1584920026 learned_addr learned my addr 172.21.15.119:0/1584920026 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:24:04.833 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:04.824+0000 7f0f55400640 1 -- 172.21.15.119:0/1584920026 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0f58105370 msgr2=0x7f0f581a10b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:24:04.833 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:04.824+0000 7f0f55400640 1 --2- 172.21.15.119:0/1584920026 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0f58105370 0x7f0f581a10b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:04.833 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:04.824+0000 7f0f55400640 1 -- 172.21.15.119:0/1584920026 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f581037c0 msgr2=0x7f0f58199b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:24:04.833 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:04.824+0000 7f0f55400640 1 --2- 172.21.15.119:0/1584920026 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f581037c0 0x7f0f58199b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:04.833 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:04.824+0000 7f0f55400640 1 -- 172.21.15.119:0/1584920026 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f581a1720 con 0x7f0f581049b0 2024-03-28T00:24:04.833 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:04.824+0000 7f0f55e00640 1 --2- 172.21.15.119:0/1584920026 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f581037c0 0x7f0f58199b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T00:24:04.833 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:04.824+0000 7f0f55400640 1 --2- 172.21.15.119:0/1584920026 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0f581049b0 0x7f0f5819a040 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f0f40000c00 tx=0x7f0f4002c650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:24:04.833 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:04.825+0000 7f0f4ea00640 1 -- 172.21.15.119:0/1584920026 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f40043020 con 0x7f0f581049b0 2024-03-28T00:24:04.833 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:04.825+0000 7f0f57200640 1 -- 172.21.15.119:0/1584920026 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f581a19b0 con 0x7f0f581049b0 2024-03-28T00:24:04.833 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:04.825+0000 7f0f4ea00640 1 -- 172.21.15.119:0/1584920026 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0f400054e0 con 0x7f0f581049b0 2024-03-28T00:24:04.833 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:04.825+0000 7f0f4ea00640 1 -- 172.21.15.119:0/1584920026 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f40003bd0 con 0x7f0f581049b0 2024-03-28T00:24:04.833 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:04.825+0000 7f0f57200640 1 -- 172.21.15.119:0/1584920026 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f581a1f00 con 0x7f0f581049b0 2024-03-28T00:24:04.833 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:04.826+0000 7f0f57200640 1 -- 172.21.15.119:0/1584920026 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f5810ab60 con 0x7f0f581049b0 2024-03-28T00:24:04.834 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:04.828+0000 7f0f4ea00640 1 -- 172.21.15.119:0/1584920026 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 17) v1 ==== 137342+0+0 (secure 0 0 0) 0x7f0f40046020 con 0x7f0f581049b0 2024-03-28T00:24:04.835 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:04.829+0000 7f0f4ea00640 1 --2- 172.21.15.119:0/1584920026 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f0f3c0a2890 0x7f0f3c0a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:24:04.835 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:04.829+0000 7f0f4ea00640 1 -- 172.21.15.119:0/1584920026 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f0f400f0e70 con 0x7f0f581049b0 2024-03-28T00:24:04.835 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:04.829+0000 7f0f55e00640 1 --2- 172.21.15.119:0/1584920026 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f0f3c0a2890 0x7f0f3c0a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:24:04.835 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:04.830+0000 7f0f55e00640 1 --2- 172.21.15.119:0/1584920026 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f0f3c0a2890 0x7f0f3c0a4d50 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f0f58104810 tx=0x7f0f48005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:24:04.841 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:04.835+0000 7f0f4ea00640 1 -- 172.21.15.119:0/1584920026 <== mon.1 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f0f4004d020 con 0x7f0f581049b0 2024-03-28T00:24:04.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:24:04 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.165:6836/1531193032,v1:172.21.15.165:6837/1531193032] up:active 2024-03-28T00:24:04.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:24:04 smithi165 ceph-mon[36302]: daemon mds.f assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-03-28T00:24:04.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:24:04 smithi165 ceph-mon[36302]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-28T00:24:04.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:24:04 smithi165 ceph-mon[36302]: Cluster is now healthy 2024-03-28T00:24:04.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:24:04 smithi165 ceph-mon[36302]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:active} 8 up:standby 2024-03-28T00:24:04.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:24:04 smithi165 ceph-mon[36302]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-03-28T00:24:04.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:24:04 smithi165 ceph-mon[36302]: daemon mds.f is now active in filesystem cephfs as rank 4 2024-03-28T00:24:04.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:24:04 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]: dispatch 2024-03-28T00:24:04.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:24:04 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2024-03-28T00:24:04.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:24:04 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8]}]: dispatch 2024-03-28T00:24:04.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:24:04 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-03-28T00:24:04.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:24:04 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]: dispatch 2024-03-28T00:24:04.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:24:04 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2024-03-28T00:24:04.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:24:04 smithi165 ceph-mon[36302]: pgmap v315: 97 pgs: 97 active+clean; 581 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s wr, 15 op/s 2024-03-28T00:24:04.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:24:04 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.165:6834/1947980102,v1:172.21.15.165:6835/1947980102] up:active 2024-03-28T00:24:04.884 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:24:04 smithi165 ceph-mon[36302]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-28T00:24:04.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:24:04 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.165:6836/1531193032,v1:172.21.15.165:6837/1531193032] up:active 2024-03-28T00:24:04.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:24:04 smithi119 ceph-mon[31518]: daemon mds.f assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-03-28T00:24:04.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:24:04 smithi119 ceph-mon[31518]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-28T00:24:04.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:24:04 smithi119 ceph-mon[31518]: Cluster is now healthy 2024-03-28T00:24:04.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:24:04 smithi119 ceph-mon[31518]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:active} 8 up:standby 2024-03-28T00:24:04.907 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:24:04 smithi119 ceph-mon[31518]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-03-28T00:24:04.907 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:24:04 smithi119 ceph-mon[31518]: daemon mds.f is now active in filesystem cephfs as rank 4 2024-03-28T00:24:04.907 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:24:04 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]: dispatch 2024-03-28T00:24:04.907 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:24:04 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2024-03-28T00:24:04.907 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:24:04 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8]}]: dispatch 2024-03-28T00:24:04.907 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:24:04 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-03-28T00:24:04.907 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:24:04 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]: dispatch 2024-03-28T00:24:04.907 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:24:04 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2024-03-28T00:24:04.907 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:24:04 smithi119 ceph-mon[31518]: pgmap v315: 97 pgs: 97 active+clean; 581 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s wr, 15 op/s 2024-03-28T00:24:04.908 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:24:04 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.165:6834/1947980102,v1:172.21.15.165:6835/1947980102] up:active 2024-03-28T00:24:04.908 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:24:04 smithi119 ceph-mon[31518]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-28T00:24:04.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:24:04 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.165:6836/1531193032,v1:172.21.15.165:6837/1531193032] up:active 2024-03-28T00:24:04.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:24:04 smithi132 ceph-mon[35941]: daemon mds.f assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-03-28T00:24:04.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:24:04 smithi132 ceph-mon[35941]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-28T00:24:04.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:24:04 smithi132 ceph-mon[35941]: Cluster is now healthy 2024-03-28T00:24:04.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:24:04 smithi132 ceph-mon[35941]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:active} 8 up:standby 2024-03-28T00:24:04.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:24:04 smithi132 ceph-mon[35941]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-03-28T00:24:04.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:24:04 smithi132 ceph-mon[35941]: daemon mds.f is now active in filesystem cephfs as rank 4 2024-03-28T00:24:04.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:24:04 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]: dispatch 2024-03-28T00:24:04.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:24:04 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2024-03-28T00:24:04.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:24:04 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8]}]: dispatch 2024-03-28T00:24:04.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:24:04 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-03-28T00:24:04.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:24:04 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]: dispatch 2024-03-28T00:24:04.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:24:04 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2024-03-28T00:24:04.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:24:04 smithi132 ceph-mon[35941]: pgmap v315: 97 pgs: 97 active+clean; 581 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s wr, 15 op/s 2024-03-28T00:24:04.954 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:24:04 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.165:6834/1947980102,v1:172.21.15.165:6835/1947980102] up:active 2024-03-28T00:24:04.954 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:24:04 smithi132 ceph-mon[35941]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-28T00:24:05.070 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:05.063+0000 7f0f57200640 1 -- 172.21.15.119:0/1584920026 --> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7f0f58061ae0 con 0x7f0f3c0a2890 2024-03-28T00:24:05.099 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:05.093+0000 7f0f4ea00640 1 -- 172.21.15.119:0/1584920026 <== mgr.14152 v2:172.21.15.119:6800/1401937096 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f0f58061ae0 con 0x7f0f3c0a2890 2024-03-28T00:24:05.103 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:05.097+0000 7f0f57200640 1 -- 172.21.15.119:0/1584920026 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f0f3c0a2890 msgr2=0x7f0f3c0a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:24:05.103 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:05.097+0000 7f0f57200640 1 --2- 172.21.15.119:0/1584920026 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f0f3c0a2890 0x7f0f3c0a4d50 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f0f58104810 tx=0x7f0f48005f90 comp rx=0 tx=0).stop 2024-03-28T00:24:05.103 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:05.097+0000 7f0f57200640 1 -- 172.21.15.119:0/1584920026 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0f581049b0 msgr2=0x7f0f5819a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:24:05.103 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:05.097+0000 7f0f57200640 1 --2- 172.21.15.119:0/1584920026 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0f581049b0 0x7f0f5819a040 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f0f40000c00 tx=0x7f0f4002c650 comp rx=0 tx=0).stop 2024-03-28T00:24:05.103 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:05.097+0000 7f0f57200640 1 -- 172.21.15.119:0/1584920026 shutdown_connections 2024-03-28T00:24:05.103 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:05.097+0000 7f0f57200640 1 --2- 172.21.15.119:0/1584920026 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f0f3c0a2890 0x7f0f3c0a4d50 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:05.103 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:05.097+0000 7f0f57200640 1 --2- 172.21.15.119:0/1584920026 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0f58105370 0x7f0f581a10b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:05.103 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:05.097+0000 7f0f57200640 1 --2- 172.21.15.119:0/1584920026 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0f581049b0 0x7f0f5819a040 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:05.104 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:05.097+0000 7f0f57200640 1 --2- 172.21.15.119:0/1584920026 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f581037c0 0x7f0f58199b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:05.104 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:05.097+0000 7f0f57200640 1 -- 172.21.15.119:0/1584920026 >> 172.21.15.119:0/1584920026 conn(0x7f0f580fed50 msgr2=0x7f0f58107d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:24:05.104 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:05.097+0000 7f0f57200640 1 -- 172.21.15.119:0/1584920026 shutdown_connections 2024-03-28T00:24:05.104 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:05.097+0000 7f0f57200640 1 -- 172.21.15.119:0/1584920026 wait complete. 2024-03-28T00:24:05.121 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-03-28T00:24:05.331 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:05.324+0000 7fb162400640 1 -- 172.21.15.119:0/1301669161 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb15c104ad0 msgr2=0x7fb15c104f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:24:05.331 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:05.324+0000 7fb162400640 1 --2- 172.21.15.119:0/1301669161 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb15c104ad0 0x7fb15c104f50 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb144009860 tx=0x7fb14402bb70 comp rx=0 tx=0).stop 2024-03-28T00:24:05.331 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:05.324+0000 7fb162400640 1 -- 172.21.15.119:0/1301669161 shutdown_connections 2024-03-28T00:24:05.331 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:05.324+0000 7fb162400640 1 --2- 172.21.15.119:0/1301669161 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb15c105490 0x7fb15c10bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:05.331 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:05.324+0000 7fb162400640 1 --2- 172.21.15.119:0/1301669161 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb15c104ad0 0x7fb15c104f50 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:05.331 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:05.324+0000 7fb162400640 1 --2- 172.21.15.119:0/1301669161 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb15c1038e0 0x7fb15c103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:05.331 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:05.324+0000 7fb162400640 1 -- 172.21.15.119:0/1301669161 >> 172.21.15.119:0/1301669161 conn(0x7fb15c0fee50 msgr2=0x7fb15c101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:24:05.331 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:05.324+0000 7fb162400640 1 -- 172.21.15.119:0/1301669161 shutdown_connections 2024-03-28T00:24:05.331 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:05.325+0000 7fb162400640 1 -- 172.21.15.119:0/1301669161 wait complete. 2024-03-28T00:24:05.333 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:05.325+0000 7fb162400640 1 Processor -- start 2024-03-28T00:24:05.333 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:05.325+0000 7fb162400640 1 -- start start 2024-03-28T00:24:05.333 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:05.326+0000 7fb162400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb15c1038e0 0x7fb15c199c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:24:05.334 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:05.326+0000 7fb162400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb15c104ad0 0x7fb15c19a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:24:05.334 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:05.326+0000 7fb162400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb15c105490 0x7fb15c1a1190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:24:05.334 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:05.326+0000 7fb162400640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb15c10f560 con 0x7fb15c104ad0 2024-03-28T00:24:05.334 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:05.326+0000 7fb162400640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb15c10f3e0 con 0x7fb15c1038e0 2024-03-28T00:24:05.334 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:05.326+0000 7fb162400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb15c10f6e0 con 0x7fb15c105490 2024-03-28T00:24:05.334 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:05.326+0000 7fb15b400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb15c1038e0 0x7fb15c199c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:24:05.334 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:05.326+0000 7fb152a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb15c104ad0 0x7fb15c19a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:24:05.334 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:05.326+0000 7fb15b400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb15c1038e0 0x7fb15c199c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.119:54006/0 (socket says 172.21.15.119:54006) 2024-03-28T00:24:05.334 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:05.326+0000 7fb15b400640 1 -- 172.21.15.119:0/595997453 learned_addr learned my addr 172.21.15.119:0/595997453 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:24:05.334 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:05.326+0000 7fb15be00640 1 --2- 172.21.15.119:0/595997453 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb15c105490 0x7fb15c1a1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:24:05.334 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:05.326+0000 7fb15b400640 1 -- 172.21.15.119:0/595997453 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb15c105490 msgr2=0x7fb15c1a1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:24:05.334 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:05.326+0000 7fb15b400640 1 --2- 172.21.15.119:0/595997453 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb15c105490 0x7fb15c1a1190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:05.334 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:05.326+0000 7fb15b400640 1 -- 172.21.15.119:0/595997453 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb15c104ad0 msgr2=0x7fb15c19a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:24:05.335 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:05.326+0000 7fb15b400640 1 --2- 172.21.15.119:0/595997453 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb15c104ad0 0x7fb15c19a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:05.335 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:05.326+0000 7fb15b400640 1 -- 172.21.15.119:0/595997453 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb15c1a1830 con 0x7fb15c1038e0 2024-03-28T00:24:05.335 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:05.326+0000 7fb152a00640 1 --2- 172.21.15.119:0/595997453 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb15c104ad0 0x7fb15c19a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T00:24:05.335 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:05.327+0000 7fb15b400640 1 --2- 172.21.15.119:0/595997453 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb15c1038e0 0x7fb15c199c20 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fb14c00d770 tx=0x7fb14c00dc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:24:05.335 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:05.327+0000 7fb158c00640 1 -- 172.21.15.119:0/595997453 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb14c017070 con 0x7fb15c1038e0 2024-03-28T00:24:05.335 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:05.327+0000 7fb158c00640 1 -- 172.21.15.119:0/595997453 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb14c009c30 con 0x7fb15c1038e0 2024-03-28T00:24:05.335 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:05.327+0000 7fb158c00640 1 -- 172.21.15.119:0/595997453 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb14c003dd0 con 0x7fb15c1038e0 2024-03-28T00:24:05.335 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:05.327+0000 7fb162400640 1 -- 172.21.15.119:0/595997453 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb15c1a1b20 con 0x7fb15c1038e0 2024-03-28T00:24:05.335 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:05.327+0000 7fb162400640 1 -- 172.21.15.119:0/595997453 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb15c1a2020 con 0x7fb15c1038e0 2024-03-28T00:24:05.336 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:05.328+0000 7fb162400640 1 -- 172.21.15.119:0/595997453 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb15c10ac70 con 0x7fb15c1038e0 2024-03-28T00:24:05.336 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:05.330+0000 7fb158c00640 1 -- 172.21.15.119:0/595997453 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 17) v1 ==== 137342+0+0 (secure 0 0 0) 0x7fb14c003f70 con 0x7fb15c1038e0 2024-03-28T00:24:05.337 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:05.331+0000 7fb158c00640 1 --2- 172.21.15.119:0/595997453 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fb1400a2890 0x7fb1400a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:24:05.337 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:05.331+0000 7fb158c00640 1 -- 172.21.15.119:0/595997453 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fb14c0ce8e0 con 0x7fb15c1038e0 2024-03-28T00:24:05.338 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:05.331+0000 7fb152a00640 1 --2- 172.21.15.119:0/595997453 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fb1400a2890 0x7fb1400a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:24:05.338 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:05.332+0000 7fb152a00640 1 --2- 172.21.15.119:0/595997453 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fb1400a2890 0x7fb1400a4d50 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fb14402c180 tx=0x7fb14402e040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:24:05.339 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:05.333+0000 7fb158c00640 1 -- 172.21.15.119:0/595997453 <== mon.2 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fb14c009de0 con 0x7fb15c1038e0 2024-03-28T00:24:05.579 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:05.572+0000 7fb162400640 1 -- 172.21.15.119:0/595997453 --> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7fb15c061ae0 con 0x7fb1400a2890 2024-03-28T00:24:05.591 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:05.585+0000 7fb158c00640 1 -- 172.21.15.119:0/595997453 <== mgr.14152 v2:172.21.15.119:6800/1401937096 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+54 (secure 0 0 0) 0x7fb15c061ae0 con 0x7fb1400a2890 2024-03-28T00:24:05.592 INFO:teuthology.orchestra.run.smithi119.stdout:/volumes/qa/sv_1/65f09c1d-44b8-4ce5-9bb1-ad370f1bb828 2024-03-28T00:24:05.594 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:05.588+0000 7fb162400640 1 -- 172.21.15.119:0/595997453 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fb1400a2890 msgr2=0x7fb1400a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:24:05.594 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:05.588+0000 7fb162400640 1 --2- 172.21.15.119:0/595997453 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fb1400a2890 0x7fb1400a4d50 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fb14402c180 tx=0x7fb14402e040 comp rx=0 tx=0).stop 2024-03-28T00:24:05.595 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:05.588+0000 7fb162400640 1 -- 172.21.15.119:0/595997453 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb15c1038e0 msgr2=0x7fb15c199c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:24:05.595 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:05.588+0000 7fb162400640 1 --2- 172.21.15.119:0/595997453 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb15c1038e0 0x7fb15c199c20 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fb14c00d770 tx=0x7fb14c00dc30 comp rx=0 tx=0).stop 2024-03-28T00:24:05.595 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:05.588+0000 7fb162400640 1 -- 172.21.15.119:0/595997453 shutdown_connections 2024-03-28T00:24:05.595 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:05.588+0000 7fb162400640 1 --2- 172.21.15.119:0/595997453 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fb1400a2890 0x7fb1400a4d50 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:05.595 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:05.588+0000 7fb162400640 1 --2- 172.21.15.119:0/595997453 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb15c105490 0x7fb15c1a1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:05.595 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:05.588+0000 7fb162400640 1 --2- 172.21.15.119:0/595997453 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb15c104ad0 0x7fb15c19a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:05.595 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:05.588+0000 7fb162400640 1 --2- 172.21.15.119:0/595997453 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb15c1038e0 0x7fb15c199c20 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:05.595 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:05.588+0000 7fb162400640 1 -- 172.21.15.119:0/595997453 >> 172.21.15.119:0/595997453 conn(0x7fb15c0fee50 msgr2=0x7fb15c107e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:24:05.595 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:05.588+0000 7fb162400640 1 -- 172.21.15.119:0/595997453 shutdown_connections 2024-03-28T00:24:05.595 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:05.588+0000 7fb162400640 1 -- 172.21.15.119:0/595997453 wait complete. 2024-03-28T00:24:05.614 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T00:24:05.821 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:05.814+0000 7fb453200640 1 -- 172.21.15.119:0/1200665447 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb44c0ffd50 msgr2=0x7fb44c102180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:24:05.821 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:05.814+0000 7fb453200640 1 --2- 172.21.15.119:0/1200665447 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb44c0ffd50 0x7fb44c102180 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb440009900 tx=0x7fb44002bc50 comp rx=0 tx=0).stop 2024-03-28T00:24:05.821 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:05.814+0000 7fb453200640 1 -- 172.21.15.119:0/1200665447 shutdown_connections 2024-03-28T00:24:05.821 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:05.814+0000 7fb453200640 1 --2- 172.21.15.119:0/1200665447 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb44c105070 0x7fb44c1054f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:05.821 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:05.814+0000 7fb453200640 1 --2- 172.21.15.119:0/1200665447 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb44c1026c0 0x7fb44c104b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:05.821 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:05.814+0000 7fb453200640 1 --2- 172.21.15.119:0/1200665447 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb44c0ffd50 0x7fb44c102180 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:05.821 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:05.814+0000 7fb453200640 1 -- 172.21.15.119:0/1200665447 >> 172.21.15.119:0/1200665447 conn(0x7fb44c0fb950 msgr2=0x7fb44c0fdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:24:05.821 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:05.814+0000 7fb453200640 1 -- 172.21.15.119:0/1200665447 shutdown_connections 2024-03-28T00:24:05.821 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:05.814+0000 7fb453200640 1 -- 172.21.15.119:0/1200665447 wait complete. 2024-03-28T00:24:05.824 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:05.815+0000 7fb453200640 1 Processor -- start 2024-03-28T00:24:05.824 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:05.816+0000 7fb453200640 1 -- start start 2024-03-28T00:24:05.824 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:05.816+0000 7fb453200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb44c0ffd50 0x7fb44c19ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:24:05.824 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:05.816+0000 7fb453200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb44c1026c0 0x7fb44c19e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:24:05.824 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:05.816+0000 7fb453200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb44c105070 0x7fb44c1a5370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:24:05.824 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:05.816+0000 7fb453200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb44c113780 con 0x7fb44c1026c0 2024-03-28T00:24:05.824 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:05.816+0000 7fb453200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb44c113600 con 0x7fb44c0ffd50 2024-03-28T00:24:05.824 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:05.816+0000 7fb453200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb44c113900 con 0x7fb44c105070 2024-03-28T00:24:05.824 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:05.816+0000 7fb451e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb44c0ffd50 0x7fb44c19ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:24:05.824 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:05.816+0000 7fb451e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb44c0ffd50 0x7fb44c19ddc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.119:54032/0 (socket says 172.21.15.119:54032) 2024-03-28T00:24:05.825 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:05.816+0000 7fb451e00640 1 -- 172.21.15.119:0/2375137111 learned_addr learned my addr 172.21.15.119:0/2375137111 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:24:05.825 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:05.816+0000 7fb451400640 1 --2- 172.21.15.119:0/2375137111 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb44c1026c0 0x7fb44c19e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:24:05.825 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:05.816+0000 7fb452800640 1 --2- 172.21.15.119:0/2375137111 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb44c105070 0x7fb44c1a5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:24:05.825 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:05.817+0000 7fb452800640 1 -- 172.21.15.119:0/2375137111 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb44c0ffd50 msgr2=0x7fb44c19ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:24:05.825 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:05.817+0000 7fb452800640 1 --2- 172.21.15.119:0/2375137111 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb44c0ffd50 0x7fb44c19ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:05.825 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:05.817+0000 7fb452800640 1 -- 172.21.15.119:0/2375137111 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb44c1026c0 msgr2=0x7fb44c19e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:24:05.825 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:05.817+0000 7fb452800640 1 --2- 172.21.15.119:0/2375137111 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb44c1026c0 0x7fb44c19e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:05.825 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:05.817+0000 7fb452800640 1 -- 172.21.15.119:0/2375137111 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb44c1a59e0 con 0x7fb44c105070 2024-03-28T00:24:05.825 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:05.817+0000 7fb451400640 1 --2- 172.21.15.119:0/2375137111 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb44c1026c0 0x7fb44c19e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T00:24:05.825 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:05.817+0000 7fb451e00640 1 --2- 172.21.15.119:0/2375137111 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb44c0ffd50 0x7fb44c19ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T00:24:05.825 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:05.817+0000 7fb452800640 1 --2- 172.21.15.119:0/2375137111 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb44c105070 0x7fb44c1a5370 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb448009fd0 tx=0x7fb44800ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:24:05.825 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:05.817+0000 7fb446a00640 1 -- 172.21.15.119:0/2375137111 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb44800ce20 con 0x7fb44c105070 2024-03-28T00:24:05.826 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:05.817+0000 7fb446a00640 1 -- 172.21.15.119:0/2375137111 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb448005020 con 0x7fb44c105070 2024-03-28T00:24:05.826 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:05.817+0000 7fb453200640 1 -- 172.21.15.119:0/2375137111 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb44c1a5cd0 con 0x7fb44c105070 2024-03-28T00:24:05.826 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:05.817+0000 7fb446a00640 1 -- 172.21.15.119:0/2375137111 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb448010720 con 0x7fb44c105070 2024-03-28T00:24:05.826 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:05.817+0000 7fb453200640 1 -- 172.21.15.119:0/2375137111 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb44c1a61d0 con 0x7fb44c105070 2024-03-28T00:24:05.827 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:05.818+0000 7fb453200640 1 -- 172.21.15.119:0/2375137111 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb44c0fd440 con 0x7fb44c105070 2024-03-28T00:24:05.827 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:05.821+0000 7fb446a00640 1 -- 172.21.15.119:0/2375137111 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 17) v1 ==== 137342+0+0 (secure 0 0 0) 0x7fb448005540 con 0x7fb44c105070 2024-03-28T00:24:05.828 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:05.822+0000 7fb446a00640 1 --2- 172.21.15.119:0/2375137111 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fb4340a2890 0x7fb4340a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:24:05.828 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:05.822+0000 7fb446a00640 1 -- 172.21.15.119:0/2375137111 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fb4480cf2a0 con 0x7fb44c105070 2024-03-28T00:24:05.828 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:05.822+0000 7fb451e00640 1 --2- 172.21.15.119:0/2375137111 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fb4340a2890 0x7fb4340a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:24:05.828 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:05.823+0000 7fb451e00640 1 --2- 172.21.15.119:0/2375137111 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fb4340a2890 0x7fb4340a4d50 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb4400096c0 tx=0x7fb440000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:24:05.834 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:05.828+0000 7fb446a00640 1 -- 172.21.15.119:0/2375137111 <== mon.1 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fb44809c1e0 con 0x7fb44c105070 2024-03-28T00:24:05.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:24:05 smithi165 ceph-mon[36302]: from='client.24676 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T00:24:05.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:24:05 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]': finished 2024-03-28T00:24:05.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:24:05 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-03-28T00:24:05.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:24:05 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8]}]': finished 2024-03-28T00:24:05.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:24:05 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-03-28T00:24:05.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:24:05 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]': finished 2024-03-28T00:24:05.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:24:05 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2024-03-28T00:24:05.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:24:05 smithi165 ceph-mon[36302]: osdmap e86: 12 total, 12 up, 12 in 2024-03-28T00:24:05.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:24:05 smithi165 ceph-mon[36302]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-03-28T00:24:05.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:24:05 smithi165 ceph-mon[36302]: from='client.24677 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T00:24:05.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:24:05 smithi119 ceph-mon[31518]: from='client.24676 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T00:24:05.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:24:05 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]': finished 2024-03-28T00:24:05.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:24:05 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-03-28T00:24:05.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:24:05 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8]}]': finished 2024-03-28T00:24:05.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:24:05 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-03-28T00:24:05.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:24:05 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]': finished 2024-03-28T00:24:05.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:24:05 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2024-03-28T00:24:05.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:24:05 smithi119 ceph-mon[31518]: osdmap e86: 12 total, 12 up, 12 in 2024-03-28T00:24:05.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:24:05 smithi119 ceph-mon[31518]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-03-28T00:24:05.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:24:05 smithi119 ceph-mon[31518]: from='client.24677 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T00:24:05.952 INFO:journalctl@ceph.mds.b.smithi132.stdout:Mar 28 00:24:05 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-b[66443]: 2024-03-28T00:24:05.526+0000 7f857cecc700 -1 mds.pinger is_rank_lagging: rank=3 was never sent ping request. 2024-03-28T00:24:05.952 INFO:journalctl@ceph.mds.b.smithi132.stdout:Mar 28 00:24:05 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-b[66443]: 2024-03-28T00:24:05.526+0000 7f857cecc700 -1 mds.pinger is_rank_lagging: rank=4 was never sent ping request. 2024-03-28T00:24:05.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:24:05 smithi132 ceph-mon[35941]: from='client.24676 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T00:24:05.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:24:05 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]': finished 2024-03-28T00:24:05.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:24:05 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-03-28T00:24:05.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:24:05 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8]}]': finished 2024-03-28T00:24:05.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:24:05 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-03-28T00:24:05.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:24:05 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]': finished 2024-03-28T00:24:05.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:24:05 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2024-03-28T00:24:05.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:24:05 smithi132 ceph-mon[35941]: osdmap e86: 12 total, 12 up, 12 in 2024-03-28T00:24:05.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:24:05 smithi132 ceph-mon[35941]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-03-28T00:24:05.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:24:05 smithi132 ceph-mon[35941]: from='client.24677 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T00:24:06.077 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:06.071+0000 7fb453200640 1 -- 172.21.15.119:0/2375137111 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb44c02d070 con 0x7fb44c105070 2024-03-28T00:24:06.078 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:06.072+0000 7fb446a00640 1 -- 172.21.15.119:0/2375137111 <== mon.1 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+11473 (secure 0 0 0) 0x7fb44809bae0 con 0x7fb44c105070 2024-03-28T00:24:06.079 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-28T00:24:06.079 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14400,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/3988635716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":3988635716},{"type":"v1","addr":"172.21.15.119:6837","nonce":3988635716}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6839/1595343240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1595343240},{"type":"v1","addr":"172.21.15.119:6839","nonce":1595343240}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-28T00:23:56.215355+0000","modified":"2024-03-28T00:24:04.459985+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24581,"mds_1":24550,"mds_2":24530,"mds_3":24523,"mds_4":24511},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14466":{"gid":14466,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6841/1678451188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6840","nonce":1678451188},{"type":"v1","addr":"172.21.15.119:6841","nonce":1678451188}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24445":{"gid":24445,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/609963168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":609963168},{"type":"v1","addr":"172.21.15.165:6833","nonce":609963168}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24455":{"gid":24455,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.132:6833/1179016670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":1179016670},{"type":"v1","addr":"172.21.15.132:6833","nonce":1179016670}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24473":{"gid":24473,"name":"j","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6835/744950487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":744950487},{"type":"v1","addr":"172.21.15.119:6835","nonce":744950487}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24500":{"gid":24500,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.132:6835/356981946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":356981946},{"type":"v1","addr":"172.21.15.132:6835","nonce":356981946}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24511":{"gid":24511,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.165:6835/1947980102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":1947980102},{"type":"v1","addr":"172.21.15.165:6835","nonce":1947980102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24523":{"gid":24523,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.165:6837/1531193032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":1531193032},{"type":"v1","addr":"172.21.15.165:6837","nonce":1531193032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24530":{"gid":24530,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.132:6837/3383082967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6836","nonce":3383082967},{"type":"v1","addr":"172.21.15.132:6837","nonce":3383082967}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24550":{"gid":24550,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.165:6839/3004684592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":3004684592},{"type":"v1","addr":"172.21.15.165:6839","nonce":3004684592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.132:6839/3305822525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6838","nonce":3305822525},{"type":"v1","addr":"172.21.15.132:6839","nonce":3305822525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24523,24530,24550,24511,24581]},"id":1}]} 2024-03-28T00:24:06.079 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 26 2024-03-28T00:24:06.081 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:06.075+0000 7fb453200640 1 -- 172.21.15.119:0/2375137111 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fb4340a2890 msgr2=0x7fb4340a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:24:06.081 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:06.075+0000 7fb453200640 1 --2- 172.21.15.119:0/2375137111 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fb4340a2890 0x7fb4340a4d50 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb4400096c0 tx=0x7fb440000bc0 comp rx=0 tx=0).stop 2024-03-28T00:24:06.081 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:06.075+0000 7fb453200640 1 -- 172.21.15.119:0/2375137111 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb44c105070 msgr2=0x7fb44c1a5370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:24:06.082 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:06.075+0000 7fb453200640 1 --2- 172.21.15.119:0/2375137111 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb44c105070 0x7fb44c1a5370 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb448009fd0 tx=0x7fb44800ef10 comp rx=0 tx=0).stop 2024-03-28T00:24:06.082 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:06.075+0000 7fb453200640 1 -- 172.21.15.119:0/2375137111 shutdown_connections 2024-03-28T00:24:06.082 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:06.075+0000 7fb453200640 1 --2- 172.21.15.119:0/2375137111 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fb4340a2890 0x7fb4340a4d50 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:06.082 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:06.075+0000 7fb453200640 1 --2- 172.21.15.119:0/2375137111 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb44c105070 0x7fb44c1a5370 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:06.082 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:06.075+0000 7fb453200640 1 --2- 172.21.15.119:0/2375137111 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb44c1026c0 0x7fb44c19e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:06.082 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:06.075+0000 7fb453200640 1 --2- 172.21.15.119:0/2375137111 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb44c0ffd50 0x7fb44c19ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:06.082 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:06.075+0000 7fb453200640 1 -- 172.21.15.119:0/2375137111 >> 172.21.15.119:0/2375137111 conn(0x7fb44c0fb950 msgr2=0x7fb44c103050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:24:06.082 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:06.075+0000 7fb453200640 1 -- 172.21.15.119:0/2375137111 shutdown_connections 2024-03-28T00:24:06.082 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:06.075+0000 7fb453200640 1 -- 172.21.15.119:0/2375137111 wait complete. 2024-03-28T00:24:06.099 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T00:24:06.311 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:06.305+0000 7f61bf200640 1 -- 172.21.15.119:0/1313241198 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f61c01048e0 msgr2=0x7f61c0104d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:24:06.311 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:06.305+0000 7f61bf200640 1 --2- 172.21.15.119:0/1313241198 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f61c01048e0 0x7f61c0104d60 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f61ac009900 tx=0x7f61ac02bc50 comp rx=0 tx=0).stop 2024-03-28T00:24:06.311 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:06.305+0000 7f61bf200640 1 -- 172.21.15.119:0/1313241198 shutdown_connections 2024-03-28T00:24:06.311 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:06.305+0000 7f61bf200640 1 --2- 172.21.15.119:0/1313241198 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f61c01052a0 0x7f61c010bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:06.312 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:06.305+0000 7f61bf200640 1 --2- 172.21.15.119:0/1313241198 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f61c01048e0 0x7f61c0104d60 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:06.312 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:06.305+0000 7f61bf200640 1 --2- 172.21.15.119:0/1313241198 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f61c01036f0 0x7f61c0103b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:06.312 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:06.305+0000 7f61bf200640 1 -- 172.21.15.119:0/1313241198 >> 172.21.15.119:0/1313241198 conn(0x7f61c00fec80 msgr2=0x7f61c01010a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:24:06.312 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:06.305+0000 7f61bf200640 1 -- 172.21.15.119:0/1313241198 shutdown_connections 2024-03-28T00:24:06.312 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:06.305+0000 7f61bf200640 1 -- 172.21.15.119:0/1313241198 wait complete. 2024-03-28T00:24:06.314 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:06.306+0000 7f61bf200640 1 Processor -- start 2024-03-28T00:24:06.314 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:06.306+0000 7f61bf200640 1 -- start start 2024-03-28T00:24:06.314 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:06.307+0000 7f61bf200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f61c01036f0 0x7f61c01999d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:24:06.314 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:06.307+0000 7f61bf200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f61c01048e0 0x7f61c0199f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:24:06.314 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:06.307+0000 7f61bf200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f61c01052a0 0x7f61c01a0f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:24:06.314 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:06.307+0000 7f61bf200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61c010f3c0 con 0x7f61c01036f0 2024-03-28T00:24:06.315 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:06.307+0000 7f61bf200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61c010f240 con 0x7f61c01052a0 2024-03-28T00:24:06.315 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:06.307+0000 7f61bf200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61c010f540 con 0x7f61c01048e0 2024-03-28T00:24:06.315 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:06.307+0000 7f61bde00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f61c01036f0 0x7f61c01999d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:24:06.315 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:06.307+0000 7f61bd400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f61c01048e0 0x7f61c0199f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:24:06.315 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:06.307+0000 7f61bd400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f61c01048e0 0x7f61c0199f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.119:36926/0 (socket says 172.21.15.119:36926) 2024-03-28T00:24:06.315 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:06.307+0000 7f61bd400640 1 -- 172.21.15.119:0/4118238208 learned_addr learned my addr 172.21.15.119:0/4118238208 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:24:06.315 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:06.307+0000 7f61be800640 1 --2- 172.21.15.119:0/4118238208 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f61c01052a0 0x7f61c01a0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:24:06.315 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:06.307+0000 7f61bd400640 1 -- 172.21.15.119:0/4118238208 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f61c01052a0 msgr2=0x7f61c01a0f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:24:06.315 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:06.307+0000 7f61bd400640 1 --2- 172.21.15.119:0/4118238208 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f61c01052a0 0x7f61c01a0f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:06.315 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:06.307+0000 7f61bd400640 1 -- 172.21.15.119:0/4118238208 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f61c01036f0 msgr2=0x7f61c01999d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:24:06.315 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:06.307+0000 7f61bd400640 1 --2- 172.21.15.119:0/4118238208 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f61c01036f0 0x7f61c01999d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:06.315 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:06.307+0000 7f61bd400640 1 -- 172.21.15.119:0/4118238208 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61c01a15f0 con 0x7f61c01048e0 2024-03-28T00:24:06.315 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:06.307+0000 7f61be800640 1 --2- 172.21.15.119:0/4118238208 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f61c01052a0 0x7f61c01a0f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T00:24:06.316 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:06.308+0000 7f61bd400640 1 --2- 172.21.15.119:0/4118238208 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f61c01048e0 0x7f61c0199f10 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f61ac02c260 tx=0x7f61ac02c670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:24:06.316 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:06.308+0000 7f61b6a00640 1 -- 172.21.15.119:0/4118238208 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61ac041070 con 0x7f61c01048e0 2024-03-28T00:24:06.316 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:06.308+0000 7f61b6a00640 1 -- 172.21.15.119:0/4118238208 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f61ac004450 con 0x7f61c01048e0 2024-03-28T00:24:06.316 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:06.308+0000 7f61bf200640 1 -- 172.21.15.119:0/4118238208 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61c01a1880 con 0x7f61c01048e0 2024-03-28T00:24:06.316 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:06.308+0000 7f61b6a00640 1 -- 172.21.15.119:0/4118238208 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61ac034440 con 0x7f61c01048e0 2024-03-28T00:24:06.316 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:06.308+0000 7f61bf200640 1 -- 172.21.15.119:0/4118238208 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61c01a1dd0 con 0x7f61c01048e0 2024-03-28T00:24:06.316 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:06.309+0000 7f61bf200640 1 -- 172.21.15.119:0/4118238208 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61c010aa10 con 0x7f61c01048e0 2024-03-28T00:24:06.317 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:06.311+0000 7f61b6a00640 1 -- 172.21.15.119:0/4118238208 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 17) v1 ==== 137342+0+0 (secure 0 0 0) 0x7f61ac044070 con 0x7f61c01048e0 2024-03-28T00:24:06.318 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:06.312+0000 7f61b6a00640 1 --2- 172.21.15.119:0/4118238208 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f61a40a2890 0x7f61a40a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:24:06.319 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:06.313+0000 7f61b6a00640 1 -- 172.21.15.119:0/4118238208 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f61ac0f0990 con 0x7f61c01048e0 2024-03-28T00:24:06.319 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:06.313+0000 7f61bde00640 1 --2- 172.21.15.119:0/4118238208 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f61a40a2890 0x7f61a40a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:24:06.319 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:06.313+0000 7f61bde00640 1 --2- 172.21.15.119:0/4118238208 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f61a40a2890 0x7f61a40a4d50 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f61c0104740 tx=0x7f61b0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:24:06.325 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:06.319+0000 7f61b6a00640 1 -- 172.21.15.119:0/4118238208 <== mon.1 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f61ac0bd390 con 0x7f61c01048e0 2024-03-28T00:24:06.524 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:06.518+0000 7f61bf200640 1 -- 172.21.15.119:0/4118238208 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f61c002d240 con 0x7f61c01048e0 2024-03-28T00:24:06.525 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:06.519+0000 7f61b6a00640 1 -- 172.21.15.119:0/4118238208 <== mon.1 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+22638 (secure 0 0 0) 0x7f61ac0c1430 con 0x7f61c01048e0 2024-03-28T00:24:06.526 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-28T00:24:06.526 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":87,"fsid":"10f73b92-ec98-11ee-95d0-87774f69a715","created":"2024-03-28T00:14:57.563604+0000","modified":"2024-03-28T00:24:05.529756+0000","last_up_change":"2024-03-28T00:22:58.821662+0000","last_in_change":"2024-03-28T00:22:38.467270+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T00:18:46.253414+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-28T00:23:54.261981+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-28T00:23:55.685836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c0b875d-a01e-4294-9c9c-5d0b048b8912","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":580396252},{"type":"v1","addr":"172.21.15.119:6803","nonce":580396252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":580396252},{"type":"v1","addr":"172.21.15.119:6805","nonce":580396252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":580396252},{"type":"v1","addr":"172.21.15.119:6809","nonce":580396252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":580396252},{"type":"v1","addr":"172.21.15.119:6807","nonce":580396252}]},"public_addr":"172.21.15.119:6803/580396252","cluster_addr":"172.21.15.119:6805/580396252","heartbeat_back_addr":"172.21.15.119:6809/580396252","heartbeat_front_addr":"172.21.15.119:6807/580396252","state":["exists","up"]},{"osd":1,"uuid":"5567f7c2-8ea4-4789-9ea1-467793324fb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":2500318888},{"type":"v1","addr":"172.21.15.132:6801","nonce":2500318888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":2500318888},{"type":"v1","addr":"172.21.15.132:6803","nonce":2500318888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":2500318888},{"type":"v1","addr":"172.21.15.132:6807","nonce":2500318888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":2500318888},{"type":"v1","addr":"172.21.15.132:6805","nonce":2500318888}]},"public_addr":"172.21.15.132:6801/2500318888","cluster_addr":"172.21.15.132:6803/2500318888","heartbeat_back_addr":"172.21.15.132:6807/2500318888","heartbeat_front_addr":"172.21.15.132:6805/2500318888","state":["exists","up"]},{"osd":2,"uuid":"ee54dbd0-24b4-4ee6-988c-ce71491199d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":2361434832},{"type":"v1","addr":"172.21.15.165:6801","nonce":2361434832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":2361434832},{"type":"v1","addr":"172.21.15.165:6803","nonce":2361434832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":2361434832},{"type":"v1","addr":"172.21.15.165:6807","nonce":2361434832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":2361434832},{"type":"v1","addr":"172.21.15.165:6805","nonce":2361434832}]},"public_addr":"172.21.15.165:6801/2361434832","cluster_addr":"172.21.15.165:6803/2361434832","heartbeat_back_addr":"172.21.15.165:6807/2361434832","heartbeat_front_addr":"172.21.15.165:6805/2361434832","state":["exists","up"]},{"osd":3,"uuid":"0464c40b-828f-4db4-96d9-3f7a6325e6da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":3732272210},{"type":"v1","addr":"172.21.15.119:6811","nonce":3732272210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":3732272210},{"type":"v1","addr":"172.21.15.119:6813","nonce":3732272210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":3732272210},{"type":"v1","addr":"172.21.15.119:6817","nonce":3732272210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":3732272210},{"type":"v1","addr":"172.21.15.119:6815","nonce":3732272210}]},"public_addr":"172.21.15.119:6811/3732272210","cluster_addr":"172.21.15.119:6813/3732272210","heartbeat_back_addr":"172.21.15.119:6817/3732272210","heartbeat_front_addr":"172.21.15.119:6815/3732272210","state":["exists","up"]},{"osd":4,"uuid":"8fd70b8e-5128-4173-aa0e-e735dfeb995d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":2803039505},{"type":"v1","addr":"172.21.15.132:6809","nonce":2803039505}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":2803039505},{"type":"v1","addr":"172.21.15.132:6811","nonce":2803039505}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":2803039505},{"type":"v1","addr":"172.21.15.132:6815","nonce":2803039505}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":2803039505},{"type":"v1","addr":"172.21.15.132:6813","nonce":2803039505}]},"public_addr":"172.21.15.132:6809/2803039505","cluster_addr":"172.21.15.132:6811/2803039505","heartbeat_back_addr":"172.21.15.132:6815/2803039505","heartbeat_front_addr":"172.21.15.132:6813/2803039505","state":["exists","up"]},{"osd":5,"uuid":"3224b4c1-733e-4424-a6da-870bbb5f94e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":2561252206},{"type":"v1","addr":"172.21.15.165:6809","nonce":2561252206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":2561252206},{"type":"v1","addr":"172.21.15.165:6811","nonce":2561252206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":2561252206},{"type":"v1","addr":"172.21.15.165:6815","nonce":2561252206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":2561252206},{"type":"v1","addr":"172.21.15.165:6813","nonce":2561252206}]},"public_addr":"172.21.15.165:6809/2561252206","cluster_addr":"172.21.15.165:6811/2561252206","heartbeat_back_addr":"172.21.15.165:6815/2561252206","heartbeat_front_addr":"172.21.15.165:6813/2561252206","state":["exists","up"]},{"osd":6,"uuid":"e435745d-59b5-439a-9091-0946ca0e9332","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":2545410230},{"type":"v1","addr":"172.21.15.119:6819","nonce":2545410230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":2545410230},{"type":"v1","addr":"172.21.15.119:6821","nonce":2545410230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":2545410230},{"type":"v1","addr":"172.21.15.119:6825","nonce":2545410230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":2545410230},{"type":"v1","addr":"172.21.15.119:6823","nonce":2545410230}]},"public_addr":"172.21.15.119:6819/2545410230","cluster_addr":"172.21.15.119:6821/2545410230","heartbeat_back_addr":"172.21.15.119:6825/2545410230","heartbeat_front_addr":"172.21.15.119:6823/2545410230","state":["exists","up"]},{"osd":7,"uuid":"79b97a12-f6b4-48c3-b517-2dd6b6e7c85f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":2877892376},{"type":"v1","addr":"172.21.15.132:6817","nonce":2877892376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":2877892376},{"type":"v1","addr":"172.21.15.132:6819","nonce":2877892376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":2877892376},{"type":"v1","addr":"172.21.15.132:6823","nonce":2877892376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":2877892376},{"type":"v1","addr":"172.21.15.132:6821","nonce":2877892376}]},"public_addr":"172.21.15.132:6817/2877892376","cluster_addr":"172.21.15.132:6819/2877892376","heartbeat_back_addr":"172.21.15.132:6823/2877892376","heartbeat_front_addr":"172.21.15.132:6821/2877892376","state":["exists","up"]},{"osd":8,"uuid":"2970524c-b99a-412d-85c2-6c29b54035e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":4115226150},{"type":"v1","addr":"172.21.15.165:6817","nonce":4115226150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":4115226150},{"type":"v1","addr":"172.21.15.165:6819","nonce":4115226150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":4115226150},{"type":"v1","addr":"172.21.15.165:6823","nonce":4115226150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":4115226150},{"type":"v1","addr":"172.21.15.165:6821","nonce":4115226150}]},"public_addr":"172.21.15.165:6817/4115226150","cluster_addr":"172.21.15.165:6819/4115226150","heartbeat_back_addr":"172.21.15.165:6823/4115226150","heartbeat_front_addr":"172.21.15.165:6821/4115226150","state":["exists","up"]},{"osd":9,"uuid":"2b1a08ed-cde9-4def-b3e8-838fff22709c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":2532245222},{"type":"v1","addr":"172.21.15.119:6827","nonce":2532245222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":2532245222},{"type":"v1","addr":"172.21.15.119:6829","nonce":2532245222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":2532245222},{"type":"v1","addr":"172.21.15.119:6833","nonce":2532245222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":2532245222},{"type":"v1","addr":"172.21.15.119:6831","nonce":2532245222}]},"public_addr":"172.21.15.119:6827/2532245222","cluster_addr":"172.21.15.119:6829/2532245222","heartbeat_back_addr":"172.21.15.119:6833/2532245222","heartbeat_front_addr":"172.21.15.119:6831/2532245222","state":["exists","up"]},{"osd":10,"uuid":"f019acd0-915a-4094-b12c-b53672740a5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":3454298492},{"type":"v1","addr":"172.21.15.132:6825","nonce":3454298492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":3454298492},{"type":"v1","addr":"172.21.15.132:6827","nonce":3454298492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":3454298492},{"type":"v1","addr":"172.21.15.132:6831","nonce":3454298492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":3454298492},{"type":"v1","addr":"172.21.15.132:6829","nonce":3454298492}]},"public_addr":"172.21.15.132:6825/3454298492","cluster_addr":"172.21.15.132:6827/3454298492","heartbeat_back_addr":"172.21.15.132:6831/3454298492","heartbeat_front_addr":"172.21.15.132:6829/3454298492","state":["exists","up"]},{"osd":11,"uuid":"2c1e59a8-8f21-45c0-8568-4b632ce020f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":2138771601},{"type":"v1","addr":"172.21.15.165:6825","nonce":2138771601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":2138771601},{"type":"v1","addr":"172.21.15.165:6827","nonce":2138771601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":2138771601},{"type":"v1","addr":"172.21.15.165:6831","nonce":2138771601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":2138771601},{"type":"v1","addr":"172.21.15.165:6829","nonce":2138771601}]},"public_addr":"172.21.15.165:6825/2138771601","cluster_addr":"172.21.15.165:6827/2138771601","heartbeat_back_addr":"172.21.15.165:6831/2138771601","heartbeat_front_addr":"172.21.15.165:6829/2138771601","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:17:54.791531+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:18:19.466638+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:18:43.435536+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:19:15.702685+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:19:41.767370+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:20:06.793003+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:20:39.952744+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:21:05.727262+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:21:30.735829+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:22:04.072142+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:22:30.894047+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:22:56.637728+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.119:6801/2575246519":"2024-03-29T00:16:03.582545+0000","172.21.15.119:6800/2575246519":"2024-03-29T00:16:03.582545+0000","172.21.15.119:0/87565965":"2024-03-29T00:16:03.582545+0000","172.21.15.119:6801/2056426384":"2024-03-29T00:15:30.472283+0000","172.21.15.119:0/1639109126":"2024-03-29T00:15:30.472283+0000","172.21.15.119:0/468478300":"2024-03-29T00:15:30.472283+0000","172.21.15.119:0/2891240357":"2024-03-29T00:15:30.472283+0000","172.21.15.119:0/4201699272":"2024-03-29T00:16:03.582545+0000","172.21.15.119:6800/2056426384":"2024-03-29T00:15:30.472283+0000","172.21.15.119:0/2635982178":"2024-03-29T00:16:03.582545+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T00:24:06.528 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:06.522+0000 7f61bf200640 1 -- 172.21.15.119:0/4118238208 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f61a40a2890 msgr2=0x7f61a40a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:24:06.528 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:06.522+0000 7f61bf200640 1 --2- 172.21.15.119:0/4118238208 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f61a40a2890 0x7f61a40a4d50 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f61c0104740 tx=0x7f61b0005f90 comp rx=0 tx=0).stop 2024-03-28T00:24:06.528 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:06.522+0000 7f61bf200640 1 -- 172.21.15.119:0/4118238208 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f61c01048e0 msgr2=0x7f61c0199f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:24:06.528 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:06.522+0000 7f61bf200640 1 --2- 172.21.15.119:0/4118238208 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f61c01048e0 0x7f61c0199f10 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f61ac02c260 tx=0x7f61ac02c670 comp rx=0 tx=0).stop 2024-03-28T00:24:06.528 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:06.522+0000 7f61bf200640 1 -- 172.21.15.119:0/4118238208 shutdown_connections 2024-03-28T00:24:06.529 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:06.522+0000 7f61bf200640 1 --2- 172.21.15.119:0/4118238208 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f61a40a2890 0x7f61a40a4d50 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:06.529 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:06.522+0000 7f61bf200640 1 --2- 172.21.15.119:0/4118238208 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f61c01052a0 0x7f61c01a0f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:06.529 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:06.522+0000 7f61bf200640 1 --2- 172.21.15.119:0/4118238208 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f61c01048e0 0x7f61c0199f10 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:06.529 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:06.522+0000 7f61bf200640 1 --2- 172.21.15.119:0/4118238208 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f61c01036f0 0x7f61c01999d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:06.529 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:06.522+0000 7f61bf200640 1 -- 172.21.15.119:0/4118238208 >> 172.21.15.119:0/4118238208 conn(0x7f61c00fec80 msgr2=0x7f61c0107be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:24:06.529 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:06.522+0000 7f61bf200640 1 -- 172.21.15.119:0/4118238208 shutdown_connections 2024-03-28T00:24:06.529 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:06.522+0000 7f61bf200640 1 -- 172.21.15.119:0/4118238208 wait complete. 2024-03-28T00:24:06.548 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-28T00:24:06.548 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-28T00:24:06.746 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:06.739+0000 7f6d17800640 1 -- 172.21.15.119:0/56754113 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d101038e0 msgr2=0x7f6d10103d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:24:06.746 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:06.739+0000 7f6d17800640 1 --2- 172.21.15.119:0/56754113 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d101038e0 0x7f6d10103d00 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f6d00009900 tx=0x7f6d0002bc30 comp rx=0 tx=0).stop 2024-03-28T00:24:06.746 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:06.739+0000 7f6d17800640 1 -- 172.21.15.119:0/56754113 shutdown_connections 2024-03-28T00:24:06.746 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:06.739+0000 7f6d17800640 1 --2- 172.21.15.119:0/56754113 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6d10105490 0x7f6d1010bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:06.746 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:06.739+0000 7f6d17800640 1 --2- 172.21.15.119:0/56754113 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6d10104ad0 0x7f6d10104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:06.746 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:06.739+0000 7f6d17800640 1 --2- 172.21.15.119:0/56754113 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d101038e0 0x7f6d10103d00 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:06.746 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:06.739+0000 7f6d17800640 1 -- 172.21.15.119:0/56754113 >> 172.21.15.119:0/56754113 conn(0x7f6d100fee50 msgr2=0x7f6d10101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:24:06.746 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:06.739+0000 7f6d17800640 1 -- 172.21.15.119:0/56754113 shutdown_connections 2024-03-28T00:24:06.746 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:06.739+0000 7f6d17800640 1 -- 172.21.15.119:0/56754113 wait complete. 2024-03-28T00:24:06.748 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:06.740+0000 7f6d17800640 1 Processor -- start 2024-03-28T00:24:06.748 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:06.741+0000 7f6d17800640 1 -- start start 2024-03-28T00:24:06.749 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:06.741+0000 7f6d17800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d101038e0 0x7f6d10199c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:24:06.749 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:06.741+0000 7f6d17800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6d10104ad0 0x7f6d1019a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:24:06.749 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:06.741+0000 7f6d17800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6d10105490 0x7f6d101a11d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:24:06.749 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:06.741+0000 7f6d17800640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d1010f580 con 0x7f6d101038e0 2024-03-28T00:24:06.749 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:06.741+0000 7f6d17800640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d1010f400 con 0x7f6d10104ad0 2024-03-28T00:24:06.749 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:06.741+0000 7f6d17800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d1010f700 con 0x7f6d10105490 2024-03-28T00:24:06.749 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:06.741+0000 7f6d14e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d101038e0 0x7f6d10199c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:24:06.749 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:06.741+0000 7f6d07e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6d10104ad0 0x7f6d1019a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:24:06.749 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:06.741+0000 7f6d14e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d101038e0 0x7f6d10199c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49112/0 (socket says 172.21.15.119:49112) 2024-03-28T00:24:06.749 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:06.741+0000 7f6d07e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6d10104ad0 0x7f6d1019a1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.119:54058/0 (socket says 172.21.15.119:54058) 2024-03-28T00:24:06.749 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:06.741+0000 7f6d15800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6d10105490 0x7f6d101a11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:24:06.749 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:06.741+0000 7f6d07e00640 1 -- 172.21.15.119:0/131494115 learned_addr learned my addr 172.21.15.119:0/131494115 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:24:06.749 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:06.742+0000 7f6d07e00640 1 -- 172.21.15.119:0/131494115 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6d10105490 msgr2=0x7f6d101a11d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:24:06.749 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:06.742+0000 7f6d07e00640 1 --2- 172.21.15.119:0/131494115 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6d10105490 0x7f6d101a11d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:06.750 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:06.742+0000 7f6d07e00640 1 -- 172.21.15.119:0/131494115 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d101038e0 msgr2=0x7f6d10199c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:24:06.750 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:06.742+0000 7f6d07e00640 1 --2- 172.21.15.119:0/131494115 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d101038e0 0x7f6d10199c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:06.750 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:06.742+0000 7f6d07e00640 1 -- 172.21.15.119:0/131494115 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d101a1870 con 0x7f6d10104ad0 2024-03-28T00:24:06.750 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:06.742+0000 7f6d14e00640 1 --2- 172.21.15.119:0/131494115 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d101038e0 0x7f6d10199c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T00:24:06.750 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:06.742+0000 7f6d07e00640 1 --2- 172.21.15.119:0/131494115 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6d10104ad0 0x7f6d1019a1a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6cf800e9f0 tx=0x7f6cf800edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:24:06.750 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:06.742+0000 7f6d0e000640 1 -- 172.21.15.119:0/131494115 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cf800ccc0 con 0x7f6d10104ad0 2024-03-28T00:24:06.750 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:06.742+0000 7f6d0e000640 1 -- 172.21.15.119:0/131494115 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6cf8010910 con 0x7f6d10104ad0 2024-03-28T00:24:06.750 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:06.742+0000 7f6d0e000640 1 -- 172.21.15.119:0/131494115 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cf80188a0 con 0x7f6d10104ad0 2024-03-28T00:24:06.750 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:06.742+0000 7f6d17800640 1 -- 172.21.15.119:0/131494115 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d101a1b60 con 0x7f6d10104ad0 2024-03-28T00:24:06.750 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:06.742+0000 7f6d17800640 1 -- 172.21.15.119:0/131494115 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d101a2030 con 0x7f6d10104ad0 2024-03-28T00:24:06.750 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:06.743+0000 7f6d17800640 1 -- 172.21.15.119:0/131494115 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d1010ac00 con 0x7f6d10104ad0 2024-03-28T00:24:06.751 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:06.745+0000 7f6d0e000640 1 -- 172.21.15.119:0/131494115 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 17) v1 ==== 137342+0+0 (secure 0 0 0) 0x7f6cf8010ac0 con 0x7f6d10104ad0 2024-03-28T00:24:06.752 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:06.746+0000 7f6d0e000640 1 --2- 172.21.15.119:0/131494115 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f6cf40a2890 0x7f6cf40a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:24:06.752 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:06.746+0000 7f6d0e000640 1 -- 172.21.15.119:0/131494115 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f6cf80cf0c0 con 0x7f6d10104ad0 2024-03-28T00:24:06.753 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:06.747+0000 7f6d14e00640 1 --2- 172.21.15.119:0/131494115 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f6cf40a2890 0x7f6cf40a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:24:06.753 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:06.747+0000 7f6d14e00640 1 --2- 172.21.15.119:0/131494115 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f6cf40a2890 0x7f6cf40a4d50 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f6d0002c170 tx=0x7f6d0002df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:24:06.756 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:06.750+0000 7f6d0e000640 1 -- 172.21.15.119:0/131494115 <== mon.2 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f6cf809c000 con 0x7f6d10104ad0 2024-03-28T00:24:06.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:24:06 smithi165 ceph-mon[36302]: from='client.24688 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T00:24:06.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:24:06 smithi165 ceph-mon[36302]: osdmap e87: 12 total, 12 up, 12 in 2024-03-28T00:24:06.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:24:06 smithi165 ceph-mon[36302]: from='client.24689 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T00:24:06.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:24:06 smithi165 ceph-mon[36302]: pgmap v318: 97 pgs: 97 active+clean; 582 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s wr, 17 op/s 2024-03-28T00:24:06.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:24:06 smithi165 ceph-mon[36302]: from='client.? 172.21.15.119:0/2375137111' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T00:24:06.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:24:06 smithi165 ceph-mon[36302]: from='client.? 172.21.15.119:0/4118238208' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-28T00:24:06.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:24:06 smithi119 ceph-mon[31518]: from='client.24688 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T00:24:06.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:24:06 smithi119 ceph-mon[31518]: osdmap e87: 12 total, 12 up, 12 in 2024-03-28T00:24:06.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:24:06 smithi119 ceph-mon[31518]: from='client.24689 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T00:24:06.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:24:06 smithi119 ceph-mon[31518]: pgmap v318: 97 pgs: 97 active+clean; 582 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s wr, 17 op/s 2024-03-28T00:24:06.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:24:06 smithi119 ceph-mon[31518]: from='client.? 172.21.15.119:0/2375137111' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T00:24:06.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:24:06 smithi119 ceph-mon[31518]: from='client.? 172.21.15.119:0/4118238208' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-28T00:24:06.950 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:06.944+0000 7f6d17800640 1 -- 172.21.15.119:0/131494115 --> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6d10061ae0 con 0x7f6cf40a2890 2024-03-28T00:24:06.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:24:06 smithi132 ceph-mon[35941]: from='client.24688 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T00:24:06.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:24:06 smithi132 ceph-mon[35941]: osdmap e87: 12 total, 12 up, 12 in 2024-03-28T00:24:06.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:24:06 smithi132 ceph-mon[35941]: from='client.24689 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T00:24:06.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:24:06 smithi132 ceph-mon[35941]: pgmap v318: 97 pgs: 97 active+clean; 582 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s wr, 17 op/s 2024-03-28T00:24:06.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:24:06 smithi132 ceph-mon[35941]: from='client.? 172.21.15.119:0/2375137111' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T00:24:06.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:24:06 smithi132 ceph-mon[35941]: from='client.? 172.21.15.119:0/4118238208' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-28T00:24:06.975 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:06.969+0000 7f6d0e000640 1 -- 172.21.15.119:0/131494115 <== mgr.14152 v2:172.21.15.119:6800/1401937096 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+258887 (secure 0 0 0) 0x7f6d10061ae0 con 0x7f6cf40a2890 2024-03-28T00:24:06.981 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-28T00:24:06.983 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2024-03-28T00:24:06.985 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:06.976+0000 7f6d17800640 1 -- 172.21.15.119:0/131494115 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f6cf40a2890 msgr2=0x7f6cf40a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:24:06.985 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:06.976+0000 7f6d17800640 1 --2- 172.21.15.119:0/131494115 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f6cf40a2890 0x7f6cf40a4d50 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f6d0002c170 tx=0x7f6d0002df90 comp rx=0 tx=0).stop 2024-03-28T00:24:06.985 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:06.976+0000 7f6d17800640 1 -- 172.21.15.119:0/131494115 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6d10104ad0 msgr2=0x7f6d1019a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:24:06.985 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:06.976+0000 7f6d17800640 1 --2- 172.21.15.119:0/131494115 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6d10104ad0 0x7f6d1019a1a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6cf800e9f0 tx=0x7f6cf800edf0 comp rx=0 tx=0).stop 2024-03-28T00:24:06.985 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:06.976+0000 7f6d17800640 1 -- 172.21.15.119:0/131494115 shutdown_connections 2024-03-28T00:24:06.985 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:06.976+0000 7f6d17800640 1 --2- 172.21.15.119:0/131494115 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f6cf40a2890 0x7f6cf40a4d50 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:06.985 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:06.976+0000 7f6d17800640 1 --2- 172.21.15.119:0/131494115 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6d10105490 0x7f6d101a11d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:06.985 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:06.976+0000 7f6d17800640 1 --2- 172.21.15.119:0/131494115 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6d10104ad0 0x7f6d1019a1a0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:06.985 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:06.976+0000 7f6d17800640 1 --2- 172.21.15.119:0/131494115 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d101038e0 0x7f6d10199c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:06.985 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:06.976+0000 7f6d17800640 1 -- 172.21.15.119:0/131494115 >> 172.21.15.119:0/131494115 conn(0x7f6d100fee50 msgr2=0x7f6d10107dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:24:06.986 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:06.976+0000 7f6d17800640 1 -- 172.21.15.119:0/131494115 shutdown_connections 2024-03-28T00:24:06.986 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:06.976+0000 7f6d17800640 1 -- 172.21.15.119:0/131494115 wait complete. 2024-03-28T00:24:07.000 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":318,"stamp":"2024-03-28T00:24:05.722770+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596217,"num_objects":71,"num_object_clones":0,"num_object_copies":213,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":327,"num_write_kb":2987,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":45,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":264,"ondisk_log_size":264,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":368576,"kb_used_data":11104,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124573248,"statfs":{"total":1151940427776,"available":1151563005952,"internally_reserved":0,"allocated":11370496,"data_stored":7049567,"data_compressed":107535,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19143,"internal_metadata":365933881},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5267,"num_objects":67,"num_object_clones":0,"num_object_copies":201,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":142,"num_write_kb":48,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":44,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001276"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:23:57.970251+0000","last_change":"2024-03-28T00:23:56.763630+0000","last_active":"2024-03-28T00:23:57.970251+0000","last_peered":"2024-03-28T00:23:57.970251+0000","last_clean":"2024-03-28T00:23:57.970251+0000","last_became_active":"2024-03-28T00:23:56.763296+0000","last_became_peered":"2024-03-28T00:23:56.763296+0000","last_unstale":"2024-03-28T00:23:57.970251+0000","last_undegraded":"2024-03-28T00:23:57.970251+0000","last_fullsized":"2024-03-28T00:23:57.970251+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T11:40:36.071225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:23:57.880797+0000","last_change":"2024-03-28T00:23:56.763628+0000","last_active":"2024-03-28T00:23:57.880797+0000","last_peered":"2024-03-28T00:23:57.880797+0000","last_clean":"2024-03-28T00:23:57.880797+0000","last_became_active":"2024-03-28T00:23:56.761753+0000","last_became_peered":"2024-03-28T00:23:56.761753+0000","last_unstale":"2024-03-28T00:23:57.880797+0000","last_undegraded":"2024-03-28T00:23:57.880797+0000","last_fullsized":"2024-03-28T00:23:57.880797+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T11:48:50.804269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:23:57.880032+0000","last_change":"2024-03-28T00:23:56.759198+0000","last_active":"2024-03-28T00:23:57.880032+0000","last_peered":"2024-03-28T00:23:57.880032+0000","last_clean":"2024-03-28T00:23:57.880032+0000","last_became_active":"2024-03-28T00:23:56.758780+0000","last_became_peered":"2024-03-28T00:23:56.758780+0000","last_unstale":"2024-03-28T00:23:57.880032+0000","last_undegraded":"2024-03-28T00:23:57.880032+0000","last_fullsized":"2024-03-28T00:23:57.880032+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T10:46:15.731193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:23:57.878082+0000","last_change":"2024-03-28T00:23:56.736651+0000","last_active":"2024-03-28T00:23:57.878082+0000","last_peered":"2024-03-28T00:23:57.878082+0000","last_clean":"2024-03-28T00:23:57.878082+0000","last_became_active":"2024-03-28T00:23:56.736063+0000","last_became_peered":"2024-03-28T00:23:56.736063+0000","last_unstale":"2024-03-28T00:23:57.878082+0000","last_undegraded":"2024-03-28T00:23:57.878082+0000","last_fullsized":"2024-03-28T00:23:57.878082+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T09:34:28.221202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:23:57.877088+0000","last_change":"2024-03-28T00:23:56.751413+0000","last_active":"2024-03-28T00:23:57.877088+0000","last_peered":"2024-03-28T00:23:57.877088+0000","last_clean":"2024-03-28T00:23:57.877088+0000","last_became_active":"2024-03-28T00:23:56.751038+0000","last_became_peered":"2024-03-28T00:23:56.751038+0000","last_unstale":"2024-03-28T00:23:57.877088+0000","last_undegraded":"2024-03-28T00:23:57.877088+0000","last_fullsized":"2024-03-28T00:23:57.877088+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T10:45:08.645799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:23:57.877718+0000","last_change":"2024-03-28T00:23:56.750970+0000","last_active":"2024-03-28T00:23:57.877718+0000","last_peered":"2024-03-28T00:23:57.877718+0000","last_clean":"2024-03-28T00:23:57.877718+0000","last_became_active":"2024-03-28T00:23:56.750608+0000","last_became_peered":"2024-03-28T00:23:56.750608+0000","last_unstale":"2024-03-28T00:23:57.877718+0000","last_undegraded":"2024-03-28T00:23:57.877718+0000","last_fullsized":"2024-03-28T00:23:57.877718+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T00:32:07.448685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:23:57.877399+0000","last_change":"2024-03-28T00:23:56.751489+0000","last_active":"2024-03-28T00:23:57.877399+0000","last_peered":"2024-03-28T00:23:57.877399+0000","last_clean":"2024-03-28T00:23:57.877399+0000","last_became_active":"2024-03-28T00:23:56.750531+0000","last_became_peered":"2024-03-28T00:23:56.750531+0000","last_unstale":"2024-03-28T00:23:57.877399+0000","last_undegraded":"2024-03-28T00:23:57.877399+0000","last_fullsized":"2024-03-28T00:23:57.877399+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T11:18:22.604126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:23:57.878853+0000","last_change":"2024-03-28T00:23:56.751537+0000","last_active":"2024-03-28T00:23:57.878853+0000","last_peered":"2024-03-28T00:23:57.878853+0000","last_clean":"2024-03-28T00:23:57.878853+0000","last_became_active":"2024-03-28T00:23:56.751185+0000","last_became_peered":"2024-03-28T00:23:56.751185+0000","last_unstale":"2024-03-28T00:23:57.878853+0000","last_undegraded":"2024-03-28T00:23:57.878853+0000","last_fullsized":"2024-03-28T00:23:57.878853+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T03:21:30.334886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:23:57.726349+0000","last_change":"2024-03-28T00:23:56.739405+0000","last_active":"2024-03-28T00:23:57.726349+0000","last_peered":"2024-03-28T00:23:57.726349+0000","last_clean":"2024-03-28T00:23:57.726349+0000","last_became_active":"2024-03-28T00:23:56.739085+0000","last_became_peered":"2024-03-28T00:23:56.739085+0000","last_unstale":"2024-03-28T00:23:57.726349+0000","last_undegraded":"2024-03-28T00:23:57.726349+0000","last_fullsized":"2024-03-28T00:23:57.726349+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T10:57:08.892446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:23:57.878318+0000","last_change":"2024-03-28T00:23:56.736880+0000","last_active":"2024-03-28T00:23:57.878318+0000","last_peered":"2024-03-28T00:23:57.878318+0000","last_clean":"2024-03-28T00:23:57.878318+0000","last_became_active":"2024-03-28T00:23:56.736524+0000","last_became_peered":"2024-03-28T00:23:56.736524+0000","last_unstale":"2024-03-28T00:23:57.878318+0000","last_undegraded":"2024-03-28T00:23:57.878318+0000","last_fullsized":"2024-03-28T00:23:57.878318+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T09:55:19.552415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:23:57.878048+0000","last_change":"2024-03-28T00:23:56.736717+0000","last_active":"2024-03-28T00:23:57.878048+0000","last_peered":"2024-03-28T00:23:57.878048+0000","last_clean":"2024-03-28T00:23:57.878048+0000","last_became_active":"2024-03-28T00:23:56.735981+0000","last_became_peered":"2024-03-28T00:23:56.735981+0000","last_unstale":"2024-03-28T00:23:57.878048+0000","last_undegraded":"2024-03-28T00:23:57.878048+0000","last_fullsized":"2024-03-28T00:23:57.878048+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T12:02:54.293418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:23:57.877915+0000","last_change":"2024-03-28T00:23:56.740591+0000","last_active":"2024-03-28T00:23:57.877915+0000","last_peered":"2024-03-28T00:23:57.877915+0000","last_clean":"2024-03-28T00:23:57.877915+0000","last_became_active":"2024-03-28T00:23:56.739965+0000","last_became_peered":"2024-03-28T00:23:56.739965+0000","last_unstale":"2024-03-28T00:23:57.877915+0000","last_undegraded":"2024-03-28T00:23:57.877915+0000","last_fullsized":"2024-03-28T00:23:57.877915+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T07:10:35.426260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:23:57.725043+0000","last_change":"2024-03-28T00:23:56.744686+0000","last_active":"2024-03-28T00:23:57.725043+0000","last_peered":"2024-03-28T00:23:57.725043+0000","last_clean":"2024-03-28T00:23:57.725043+0000","last_became_active":"2024-03-28T00:23:56.744355+0000","last_became_peered":"2024-03-28T00:23:56.744355+0000","last_unstale":"2024-03-28T00:23:57.725043+0000","last_undegraded":"2024-03-28T00:23:57.725043+0000","last_fullsized":"2024-03-28T00:23:57.725043+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T07:52:27.360917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:23:57.878787+0000","last_change":"2024-03-28T00:23:56.762465+0000","last_active":"2024-03-28T00:23:57.878787+0000","last_peered":"2024-03-28T00:23:57.878787+0000","last_clean":"2024-03-28T00:23:57.878787+0000","last_became_active":"2024-03-28T00:23:56.751467+0000","last_became_peered":"2024-03-28T00:23:56.751467+0000","last_unstale":"2024-03-28T00:23:57.878787+0000","last_undegraded":"2024-03-28T00:23:57.878787+0000","last_fullsized":"2024-03-28T00:23:57.878787+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T10:55:33.880872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:23:57.878882+0000","last_change":"2024-03-28T00:23:56.759344+0000","last_active":"2024-03-28T00:23:57.878882+0000","last_peered":"2024-03-28T00:23:57.878882+0000","last_clean":"2024-03-28T00:23:57.878882+0000","last_became_active":"2024-03-28T00:23:56.758694+0000","last_became_peered":"2024-03-28T00:23:56.758694+0000","last_unstale":"2024-03-28T00:23:57.878882+0000","last_undegraded":"2024-03-28T00:23:57.878882+0000","last_fullsized":"2024-03-28T00:23:57.878882+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T02:43:42.586070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:23:57.725824+0000","last_change":"2024-03-28T00:23:56.759298+0000","last_active":"2024-03-28T00:23:57.725824+0000","last_peered":"2024-03-28T00:23:57.725824+0000","last_clean":"2024-03-28T00:23:57.725824+0000","last_became_active":"2024-03-28T00:23:56.759006+0000","last_became_peered":"2024-03-28T00:23:56.759006+0000","last_unstale":"2024-03-28T00:23:57.725824+0000","last_undegraded":"2024-03-28T00:23:57.725824+0000","last_fullsized":"2024-03-28T00:23:57.725824+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T00:43:09.433789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:23:57.970302+0000","last_change":"2024-03-28T00:23:56.760324+0000","last_active":"2024-03-28T00:23:57.970302+0000","last_peered":"2024-03-28T00:23:57.970302+0000","last_clean":"2024-03-28T00:23:57.970302+0000","last_became_active":"2024-03-28T00:23:56.759866+0000","last_became_peered":"2024-03-28T00:23:56.759866+0000","last_unstale":"2024-03-28T00:23:57.970302+0000","last_undegraded":"2024-03-28T00:23:57.970302+0000","last_fullsized":"2024-03-28T00:23:57.970302+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T01:11:15.923114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:23:57.882402+0000","last_change":"2024-03-28T00:23:56.755227+0000","last_active":"2024-03-28T00:23:57.882402+0000","last_peered":"2024-03-28T00:23:57.882402+0000","last_clean":"2024-03-28T00:23:57.882402+0000","last_became_active":"2024-03-28T00:23:56.754767+0000","last_became_peered":"2024-03-28T00:23:56.754767+0000","last_unstale":"2024-03-28T00:23:57.882402+0000","last_undegraded":"2024-03-28T00:23:57.882402+0000","last_fullsized":"2024-03-28T00:23:57.882402+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T07:26:04.797904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:23:57.877936+0000","last_change":"2024-03-28T00:23:56.744199+0000","last_active":"2024-03-28T00:23:57.877936+0000","last_peered":"2024-03-28T00:23:57.877936+0000","last_clean":"2024-03-28T00:23:57.877936+0000","last_became_active":"2024-03-28T00:23:56.743848+0000","last_became_peered":"2024-03-28T00:23:56.743848+0000","last_unstale":"2024-03-28T00:23:57.877936+0000","last_undegraded":"2024-03-28T00:23:57.877936+0000","last_fullsized":"2024-03-28T00:23:57.877936+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T10:08:01.426115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'9","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:24:01.500088+0000","last_change":"2024-03-28T00:23:55.731996+0000","last_active":"2024-03-28T00:24:01.500088+0000","last_peered":"2024-03-28T00:24:01.500088+0000","last_clean":"2024-03-28T00:24:01.500088+0000","last_became_active":"2024-03-28T00:23:55.731626+0000","last_became_peered":"2024-03-28T00:23:55.731626+0000","last_unstale":"2024-03-28T00:24:01.500088+0000","last_undegraded":"2024-03-28T00:24:01.500088+0000","last_fullsized":"2024-03-28T00:24:01.500088+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_clean_scrub_stamp":"2024-03-28T00:23:54.693629+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T08:56:00.420387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:23:57.881708+0000","last_change":"2024-03-28T00:23:56.757048+0000","last_active":"2024-03-28T00:23:57.881708+0000","last_peered":"2024-03-28T00:23:57.881708+0000","last_clean":"2024-03-28T00:23:57.881708+0000","last_became_active":"2024-03-28T00:23:56.752710+0000","last_became_peered":"2024-03-28T00:23:56.752710+0000","last_unstale":"2024-03-28T00:23:57.881708+0000","last_undegraded":"2024-03-28T00:23:57.881708+0000","last_fullsized":"2024-03-28T00:23:57.881708+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T06:26:39.723255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:23:57.882574+0000","last_change":"2024-03-28T00:23:55.748191+0000","last_active":"2024-03-28T00:23:57.882574+0000","last_peered":"2024-03-28T00:23:57.882574+0000","last_clean":"2024-03-28T00:23:57.882574+0000","last_became_active":"2024-03-28T00:23:55.747154+0000","last_became_peered":"2024-03-28T00:23:55.747154+0000","last_unstale":"2024-03-28T00:23:57.882574+0000","last_undegraded":"2024-03-28T00:23:57.882574+0000","last_fullsized":"2024-03-28T00:23:57.882574+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_clean_scrub_stamp":"2024-03-28T00:23:54.693629+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T07:17:12.296494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:23:57.881926+0000","last_change":"2024-03-28T00:23:56.758979+0000","last_active":"2024-03-28T00:23:57.881926+0000","last_peered":"2024-03-28T00:23:57.881926+0000","last_clean":"2024-03-28T00:23:57.881926+0000","last_became_active":"2024-03-28T00:23:56.752628+0000","last_became_peered":"2024-03-28T00:23:56.752628+0000","last_unstale":"2024-03-28T00:23:57.881926+0000","last_undegraded":"2024-03-28T00:23:57.881926+0000","last_fullsized":"2024-03-28T00:23:57.881926+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T10:11:57.245938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"85'7","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:24:02.517073+0000","last_change":"2024-03-28T00:23:55.728972+0000","last_active":"2024-03-28T00:24:02.517073+0000","last_peered":"2024-03-28T00:24:02.517073+0000","last_clean":"2024-03-28T00:24:02.517073+0000","last_became_active":"2024-03-28T00:23:55.728053+0000","last_became_peered":"2024-03-28T00:23:55.728053+0000","last_unstale":"2024-03-28T00:24:02.517073+0000","last_undegraded":"2024-03-28T00:24:02.517073+0000","last_fullsized":"2024-03-28T00:24:02.517073+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_clean_scrub_stamp":"2024-03-28T00:23:54.693629+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T04:12:27.129196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:23:57.725920+0000","last_change":"2024-03-28T00:23:56.752585+0000","last_active":"2024-03-28T00:23:57.725920+0000","last_peered":"2024-03-28T00:23:57.725920+0000","last_clean":"2024-03-28T00:23:57.725920+0000","last_became_active":"2024-03-28T00:23:56.752252+0000","last_became_peered":"2024-03-28T00:23:56.752252+0000","last_unstale":"2024-03-28T00:23:57.725920+0000","last_undegraded":"2024-03-28T00:23:57.725920+0000","last_fullsized":"2024-03-28T00:23:57.725920+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T11:51:02.864486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:23:57.877516+0000","last_change":"2024-03-28T00:23:55.726859+0000","last_active":"2024-03-28T00:23:57.877516+0000","last_peered":"2024-03-28T00:23:57.877516+0000","last_clean":"2024-03-28T00:23:57.877516+0000","last_became_active":"2024-03-28T00:23:55.725372+0000","last_became_peered":"2024-03-28T00:23:55.725372+0000","last_unstale":"2024-03-28T00:23:57.877516+0000","last_undegraded":"2024-03-28T00:23:57.877516+0000","last_fullsized":"2024-03-28T00:23:57.877516+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_clean_scrub_stamp":"2024-03-28T00:23:54.693629+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T00:52:35.278544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:23:57.721317+0000","last_change":"2024-03-28T00:23:56.742076+0000","last_active":"2024-03-28T00:23:57.721317+0000","last_peered":"2024-03-28T00:23:57.721317+0000","last_clean":"2024-03-28T00:23:57.721317+0000","last_became_active":"2024-03-28T00:23:56.741761+0000","last_became_peered":"2024-03-28T00:23:56.741761+0000","last_unstale":"2024-03-28T00:23:57.721317+0000","last_undegraded":"2024-03-28T00:23:57.721317+0000","last_fullsized":"2024-03-28T00:23:57.721317+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T08:17:02.024132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"85'5","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:24:00.485020+0000","last_change":"2024-03-28T00:23:55.728587+0000","last_active":"2024-03-28T00:24:00.485020+0000","last_peered":"2024-03-28T00:24:00.485020+0000","last_clean":"2024-03-28T00:24:00.485020+0000","last_became_active":"2024-03-28T00:23:55.728309+0000","last_became_peered":"2024-03-28T00:23:55.728309+0000","last_unstale":"2024-03-28T00:24:00.485020+0000","last_undegraded":"2024-03-28T00:24:00.485020+0000","last_fullsized":"2024-03-28T00:24:00.485020+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_clean_scrub_stamp":"2024-03-28T00:23:54.693629+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T04:15:12.060649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:23:57.877122+0000","last_change":"2024-03-28T00:23:56.747351+0000","last_active":"2024-03-28T00:23:57.877122+0000","last_peered":"2024-03-28T00:23:57.877122+0000","last_clean":"2024-03-28T00:23:57.877122+0000","last_became_active":"2024-03-28T00:23:56.746995+0000","last_became_peered":"2024-03-28T00:23:56.746995+0000","last_unstale":"2024-03-28T00:23:57.877122+0000","last_undegraded":"2024-03-28T00:23:57.877122+0000","last_fullsized":"2024-03-28T00:23:57.877122+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T00:43:53.521420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"85'7","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:24:02.522127+0000","last_change":"2024-03-28T00:23:55.730157+0000","last_active":"2024-03-28T00:24:02.522127+0000","last_peered":"2024-03-28T00:24:02.522127+0000","last_clean":"2024-03-28T00:24:02.522127+0000","last_became_active":"2024-03-28T00:23:55.729841+0000","last_became_peered":"2024-03-28T00:23:55.729841+0000","last_unstale":"2024-03-28T00:24:02.522127+0000","last_undegraded":"2024-03-28T00:24:02.522127+0000","last_fullsized":"2024-03-28T00:24:02.522127+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_clean_scrub_stamp":"2024-03-28T00:23:54.693629+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T09:57:31.865444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:23:57.721426+0000","last_change":"2024-03-28T00:23:56.760077+0000","last_active":"2024-03-28T00:23:57.721426+0000","last_peered":"2024-03-28T00:23:57.721426+0000","last_clean":"2024-03-28T00:23:57.721426+0000","last_became_active":"2024-03-28T00:23:56.759735+0000","last_became_peered":"2024-03-28T00:23:56.759735+0000","last_unstale":"2024-03-28T00:23:57.721426+0000","last_undegraded":"2024-03-28T00:23:57.721426+0000","last_fullsized":"2024-03-28T00:23:57.721426+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T11:36:17.224248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"85'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:24:01.500378+0000","last_change":"2024-03-28T00:23:55.738216+0000","last_active":"2024-03-28T00:24:01.500378+0000","last_peered":"2024-03-28T00:24:01.500378+0000","last_clean":"2024-03-28T00:24:01.500378+0000","last_became_active":"2024-03-28T00:23:55.737878+0000","last_became_peered":"2024-03-28T00:23:55.737878+0000","last_unstale":"2024-03-28T00:24:01.500378+0000","last_undegraded":"2024-03-28T00:24:01.500378+0000","last_fullsized":"2024-03-28T00:24:01.500378+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_clean_scrub_stamp":"2024-03-28T00:23:54.693629+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T06:30:00.465906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:23:57.883052+0000","last_change":"2024-03-28T00:23:56.760362+0000","last_active":"2024-03-28T00:23:57.883052+0000","last_peered":"2024-03-28T00:23:57.883052+0000","last_clean":"2024-03-28T00:23:57.883052+0000","last_became_active":"2024-03-28T00:23:56.759918+0000","last_became_peered":"2024-03-28T00:23:56.759918+0000","last_unstale":"2024-03-28T00:23:57.883052+0000","last_undegraded":"2024-03-28T00:23:57.883052+0000","last_fullsized":"2024-03-28T00:23:57.883052+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T04:59:01.038539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"85'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:24:00.483263+0000","last_change":"2024-03-28T00:23:55.727557+0000","last_active":"2024-03-28T00:24:00.483263+0000","last_peered":"2024-03-28T00:24:00.483263+0000","last_clean":"2024-03-28T00:24:00.483263+0000","last_became_active":"2024-03-28T00:23:55.727235+0000","last_became_peered":"2024-03-28T00:23:55.727235+0000","last_unstale":"2024-03-28T00:24:00.483263+0000","last_undegraded":"2024-03-28T00:24:00.483263+0000","last_fullsized":"2024-03-28T00:24:00.483263+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_clean_scrub_stamp":"2024-03-28T00:23:54.693629+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T08:14:57.268850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:23:57.881530+0000","last_change":"2024-03-28T00:23:56.766812+0000","last_active":"2024-03-28T00:23:57.881530+0000","last_peered":"2024-03-28T00:23:57.881530+0000","last_clean":"2024-03-28T00:23:57.881530+0000","last_became_active":"2024-03-28T00:23:56.766386+0000","last_became_peered":"2024-03-28T00:23:56.766386+0000","last_unstale":"2024-03-28T00:23:57.881530+0000","last_undegraded":"2024-03-28T00:23:57.881530+0000","last_fullsized":"2024-03-28T00:23:57.881530+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T05:58:07.349158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"85'4","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:24:00.486130+0000","last_change":"2024-03-28T00:23:55.727617+0000","last_active":"2024-03-28T00:24:00.486130+0000","last_peered":"2024-03-28T00:24:00.486130+0000","last_clean":"2024-03-28T00:24:00.486130+0000","last_became_active":"2024-03-28T00:23:55.727276+0000","last_became_peered":"2024-03-28T00:23:55.727276+0000","last_unstale":"2024-03-28T00:24:00.486130+0000","last_undegraded":"2024-03-28T00:24:00.486130+0000","last_fullsized":"2024-03-28T00:24:00.486130+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_clean_scrub_stamp":"2024-03-28T00:23:54.693629+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T02:27:04.376342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:23:57.726354+0000","last_change":"2024-03-28T00:23:56.747261+0000","last_active":"2024-03-28T00:23:57.726354+0000","last_peered":"2024-03-28T00:23:57.726354+0000","last_clean":"2024-03-28T00:23:57.726354+0000","last_became_active":"2024-03-28T00:23:56.746862+0000","last_became_peered":"2024-03-28T00:23:56.746862+0000","last_unstale":"2024-03-28T00:23:57.726354+0000","last_undegraded":"2024-03-28T00:23:57.726354+0000","last_fullsized":"2024-03-28T00:23:57.726354+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T07:33:35.747722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"85'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:24:00.484222+0000","last_change":"2024-03-28T00:23:55.725568+0000","last_active":"2024-03-28T00:24:00.484222+0000","last_peered":"2024-03-28T00:24:00.484222+0000","last_clean":"2024-03-28T00:24:00.484222+0000","last_became_active":"2024-03-28T00:23:55.724967+0000","last_became_peered":"2024-03-28T00:23:55.724967+0000","last_unstale":"2024-03-28T00:24:00.484222+0000","last_undegraded":"2024-03-28T00:24:00.484222+0000","last_fullsized":"2024-03-28T00:24:00.484222+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_clean_scrub_stamp":"2024-03-28T00:23:54.693629+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T10:40:00.731885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:23:57.881704+0000","last_change":"2024-03-28T00:23:56.763176+0000","last_active":"2024-03-28T00:23:57.881704+0000","last_peered":"2024-03-28T00:23:57.881704+0000","last_clean":"2024-03-28T00:23:57.881704+0000","last_became_active":"2024-03-28T00:23:56.762709+0000","last_became_peered":"2024-03-28T00:23:56.762709+0000","last_unstale":"2024-03-28T00:23:57.881704+0000","last_undegraded":"2024-03-28T00:23:57.881704+0000","last_fullsized":"2024-03-28T00:23:57.881704+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T05:58:52.003151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'5","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:24:00.484309+0000","last_change":"2024-03-28T00:23:55.745579+0000","last_active":"2024-03-28T00:24:00.484309+0000","last_peered":"2024-03-28T00:24:00.484309+0000","last_clean":"2024-03-28T00:24:00.484309+0000","last_became_active":"2024-03-28T00:23:55.745262+0000","last_became_peered":"2024-03-28T00:23:55.745262+0000","last_unstale":"2024-03-28T00:24:00.484309+0000","last_undegraded":"2024-03-28T00:24:00.484309+0000","last_fullsized":"2024-03-28T00:24:00.484309+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_clean_scrub_stamp":"2024-03-28T00:23:54.693629+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T00:48:36.212168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:23:57.726092+0000","last_change":"2024-03-28T00:23:56.736393+0000","last_active":"2024-03-28T00:23:57.726092+0000","last_peered":"2024-03-28T00:23:57.726092+0000","last_clean":"2024-03-28T00:23:57.726092+0000","last_became_active":"2024-03-28T00:23:56.736054+0000","last_became_peered":"2024-03-28T00:23:56.736054+0000","last_unstale":"2024-03-28T00:23:57.726092+0000","last_undegraded":"2024-03-28T00:23:57.726092+0000","last_fullsized":"2024-03-28T00:23:57.726092+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T11:56:00.039257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"85'8","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:24:00.483690+0000","last_change":"2024-03-28T00:23:55.722192+0000","last_active":"2024-03-28T00:24:00.483690+0000","last_peered":"2024-03-28T00:24:00.483690+0000","last_clean":"2024-03-28T00:24:00.483690+0000","last_became_active":"2024-03-28T00:23:55.721881+0000","last_became_peered":"2024-03-28T00:23:55.721881+0000","last_unstale":"2024-03-28T00:24:00.483690+0000","last_undegraded":"2024-03-28T00:24:00.483690+0000","last_fullsized":"2024-03-28T00:24:00.483690+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_clean_scrub_stamp":"2024-03-28T00:23:54.693629+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T03:47:34.950264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:23:57.877521+0000","last_change":"2024-03-28T00:23:56.738361+0000","last_active":"2024-03-28T00:23:57.877521+0000","last_peered":"2024-03-28T00:23:57.877521+0000","last_clean":"2024-03-28T00:23:57.877521+0000","last_became_active":"2024-03-28T00:23:56.737040+0000","last_became_peered":"2024-03-28T00:23:56.737040+0000","last_unstale":"2024-03-28T00:23:57.877521+0000","last_undegraded":"2024-03-28T00:23:57.877521+0000","last_fullsized":"2024-03-28T00:23:57.877521+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T11:39:40.653421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'4","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:24:00.487700+0000","last_change":"2024-03-28T00:23:55.733943+0000","last_active":"2024-03-28T00:24:00.487700+0000","last_peered":"2024-03-28T00:24:00.487700+0000","last_clean":"2024-03-28T00:24:00.487700+0000","last_became_active":"2024-03-28T00:23:55.733236+0000","last_became_peered":"2024-03-28T00:23:55.733236+0000","last_unstale":"2024-03-28T00:24:00.487700+0000","last_undegraded":"2024-03-28T00:24:00.487700+0000","last_fullsized":"2024-03-28T00:24:00.487700+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_clean_scrub_stamp":"2024-03-28T00:23:54.693629+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T09:45:33.070408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:23:57.724548+0000","last_change":"2024-03-28T00:23:56.758751+0000","last_active":"2024-03-28T00:23:57.724548+0000","last_peered":"2024-03-28T00:23:57.724548+0000","last_clean":"2024-03-28T00:23:57.724548+0000","last_became_active":"2024-03-28T00:23:56.758066+0000","last_became_peered":"2024-03-28T00:23:56.758066+0000","last_unstale":"2024-03-28T00:23:57.724548+0000","last_undegraded":"2024-03-28T00:23:57.724548+0000","last_fullsized":"2024-03-28T00:23:57.724548+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T02:21:42.514212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"85'4","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:24:00.487908+0000","last_change":"2024-03-28T00:23:55.737167+0000","last_active":"2024-03-28T00:24:00.487908+0000","last_peered":"2024-03-28T00:24:00.487908+0000","last_clean":"2024-03-28T00:24:00.487908+0000","last_became_active":"2024-03-28T00:23:55.736925+0000","last_became_peered":"2024-03-28T00:23:55.736925+0000","last_unstale":"2024-03-28T00:24:00.487908+0000","last_undegraded":"2024-03-28T00:24:00.487908+0000","last_fullsized":"2024-03-28T00:24:00.487908+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_clean_scrub_stamp":"2024-03-28T00:23:54.693629+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T09:13:24.820375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:23:57.877871+0000","last_change":"2024-03-28T00:23:56.745939+0000","last_active":"2024-03-28T00:23:57.877871+0000","last_peered":"2024-03-28T00:23:57.877871+0000","last_clean":"2024-03-28T00:23:57.877871+0000","last_became_active":"2024-03-28T00:23:56.745504+0000","last_became_peered":"2024-03-28T00:23:56.745504+0000","last_unstale":"2024-03-28T00:23:57.877871+0000","last_undegraded":"2024-03-28T00:23:57.877871+0000","last_fullsized":"2024-03-28T00:23:57.877871+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T06:02:34.199029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"85'6","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:24:02.516249+0000","last_change":"2024-03-28T00:23:55.726916+0000","last_active":"2024-03-28T00:24:02.516249+0000","last_peered":"2024-03-28T00:24:02.516249+0000","last_clean":"2024-03-28T00:24:02.516249+0000","last_became_active":"2024-03-28T00:23:55.726570+0000","last_became_peered":"2024-03-28T00:23:55.726570+0000","last_unstale":"2024-03-28T00:24:02.516249+0000","last_undegraded":"2024-03-28T00:24:02.516249+0000","last_fullsized":"2024-03-28T00:24:02.516249+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_clean_scrub_stamp":"2024-03-28T00:23:54.693629+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T06:28:54.581129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:23:57.877115+0000","last_change":"2024-03-28T00:23:56.738303+0000","last_active":"2024-03-28T00:23:57.877115+0000","last_peered":"2024-03-28T00:23:57.877115+0000","last_clean":"2024-03-28T00:23:57.877115+0000","last_became_active":"2024-03-28T00:23:56.737926+0000","last_became_peered":"2024-03-28T00:23:56.737926+0000","last_unstale":"2024-03-28T00:23:57.877115+0000","last_undegraded":"2024-03-28T00:23:57.877115+0000","last_fullsized":"2024-03-28T00:23:57.877115+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T01:56:53.985793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'16","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:24:01.503051+0000","last_change":"2024-03-28T00:23:55.728708+0000","last_active":"2024-03-28T00:24:01.503051+0000","last_peered":"2024-03-28T00:24:01.503051+0000","last_clean":"2024-03-28T00:24:01.503051+0000","last_became_active":"2024-03-28T00:23:55.728415+0000","last_became_peered":"2024-03-28T00:23:55.728415+0000","last_unstale":"2024-03-28T00:24:01.503051+0000","last_undegraded":"2024-03-28T00:24:01.503051+0000","last_fullsized":"2024-03-28T00:24:01.503051+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_clean_scrub_stamp":"2024-03-28T00:23:54.693629+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T05:36:14.405953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:23:57.721157+0000","last_change":"2024-03-28T00:23:56.741982+0000","last_active":"2024-03-28T00:23:57.721157+0000","last_peered":"2024-03-28T00:23:57.721157+0000","last_clean":"2024-03-28T00:23:57.721157+0000","last_became_active":"2024-03-28T00:23:56.741642+0000","last_became_peered":"2024-03-28T00:23:56.741642+0000","last_unstale":"2024-03-28T00:23:57.721157+0000","last_undegraded":"2024-03-28T00:23:57.721157+0000","last_fullsized":"2024-03-28T00:23:57.721157+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T04:27:49.168164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"84'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:23:57.877547+0000","last_change":"2024-03-28T00:23:55.729168+0000","last_active":"2024-03-28T00:23:57.877547+0000","last_peered":"2024-03-28T00:23:57.877547+0000","last_clean":"2024-03-28T00:23:57.877547+0000","last_became_active":"2024-03-28T00:23:55.728686+0000","last_became_peered":"2024-03-28T00:23:55.728686+0000","last_unstale":"2024-03-28T00:23:57.877547+0000","last_undegraded":"2024-03-28T00:23:57.877547+0000","last_fullsized":"2024-03-28T00:23:57.877547+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_clean_scrub_stamp":"2024-03-28T00:23:54.693629+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T12:12:43.807889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:23:57.970140+0000","last_change":"2024-03-28T00:23:56.739241+0000","last_active":"2024-03-28T00:23:57.970140+0000","last_peered":"2024-03-28T00:23:57.970140+0000","last_clean":"2024-03-28T00:23:57.970140+0000","last_became_active":"2024-03-28T00:23:56.738122+0000","last_became_peered":"2024-03-28T00:23:56.738122+0000","last_unstale":"2024-03-28T00:23:57.970140+0000","last_undegraded":"2024-03-28T00:23:57.970140+0000","last_fullsized":"2024-03-28T00:23:57.970140+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T02:52:50.375314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:23:57.881802+0000","last_change":"2024-03-28T00:23:55.745860+0000","last_active":"2024-03-28T00:23:57.881802+0000","last_peered":"2024-03-28T00:23:57.881802+0000","last_clean":"2024-03-28T00:23:57.881802+0000","last_became_active":"2024-03-28T00:23:55.745518+0000","last_became_peered":"2024-03-28T00:23:55.745518+0000","last_unstale":"2024-03-28T00:23:57.881802+0000","last_undegraded":"2024-03-28T00:23:57.881802+0000","last_fullsized":"2024-03-28T00:23:57.881802+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_clean_scrub_stamp":"2024-03-28T00:23:54.693629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T10:48:24.356150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:23:57.878455+0000","last_change":"2024-03-28T00:23:56.752780+0000","last_active":"2024-03-28T00:23:57.878455+0000","last_peered":"2024-03-28T00:23:57.878455+0000","last_clean":"2024-03-28T00:23:57.878455+0000","last_became_active":"2024-03-28T00:23:56.751608+0000","last_became_peered":"2024-03-28T00:23:56.751608+0000","last_unstale":"2024-03-28T00:23:57.878455+0000","last_undegraded":"2024-03-28T00:23:57.878455+0000","last_fullsized":"2024-03-28T00:23:57.878455+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T01:20:45.485846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"85'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:23:57.878084+0000","last_change":"2024-03-28T00:23:55.728123+0000","last_active":"2024-03-28T00:23:57.878084+0000","last_peered":"2024-03-28T00:23:57.878084+0000","last_clean":"2024-03-28T00:23:57.878084+0000","last_became_active":"2024-03-28T00:23:55.727839+0000","last_became_peered":"2024-03-28T00:23:55.727839+0000","last_unstale":"2024-03-28T00:23:57.878084+0000","last_undegraded":"2024-03-28T00:23:57.878084+0000","last_fullsized":"2024-03-28T00:23:57.878084+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_clean_scrub_stamp":"2024-03-28T00:23:54.693629+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T11:39:30.315824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:23:57.878226+0000","last_change":"2024-03-28T00:23:56.752362+0000","last_active":"2024-03-28T00:23:57.878226+0000","last_peered":"2024-03-28T00:23:57.878226+0000","last_clean":"2024-03-28T00:23:57.878226+0000","last_became_active":"2024-03-28T00:23:56.751129+0000","last_became_peered":"2024-03-28T00:23:56.751129+0000","last_unstale":"2024-03-28T00:23:57.878226+0000","last_undegraded":"2024-03-28T00:23:57.878226+0000","last_fullsized":"2024-03-28T00:23:57.878226+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T07:24:15.850437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:23:57.721321+0000","last_change":"2024-03-28T00:23:56.742573+0000","last_active":"2024-03-28T00:23:57.721321+0000","last_peered":"2024-03-28T00:23:57.721321+0000","last_clean":"2024-03-28T00:23:57.721321+0000","last_became_active":"2024-03-28T00:23:56.742236+0000","last_became_peered":"2024-03-28T00:23:56.742236+0000","last_unstale":"2024-03-28T00:23:57.721321+0000","last_undegraded":"2024-03-28T00:23:57.721321+0000","last_fullsized":"2024-03-28T00:23:57.721321+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T11:46:36.357845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"85'4","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:24:01.503895+0000","last_change":"2024-03-28T00:23:55.747124+0000","last_active":"2024-03-28T00:24:01.503895+0000","last_peered":"2024-03-28T00:24:01.503895+0000","last_clean":"2024-03-28T00:24:01.503895+0000","last_became_active":"2024-03-28T00:23:55.745721+0000","last_became_peered":"2024-03-28T00:23:55.745721+0000","last_unstale":"2024-03-28T00:24:01.503895+0000","last_undegraded":"2024-03-28T00:24:01.503895+0000","last_fullsized":"2024-03-28T00:24:01.503895+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_clean_scrub_stamp":"2024-03-28T00:23:54.693629+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T10:37:15.898923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:23:57.881005+0000","last_change":"2024-03-28T00:23:56.757060+0000","last_active":"2024-03-28T00:23:57.881005+0000","last_peered":"2024-03-28T00:23:57.881005+0000","last_clean":"2024-03-28T00:23:57.881005+0000","last_became_active":"2024-03-28T00:23:56.756754+0000","last_became_peered":"2024-03-28T00:23:56.756754+0000","last_unstale":"2024-03-28T00:23:57.881005+0000","last_undegraded":"2024-03-28T00:23:57.881005+0000","last_fullsized":"2024-03-28T00:23:57.881005+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T11:59:32.877471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'6","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:24:02.519052+0000","last_change":"2024-03-28T00:23:55.728039+0000","last_active":"2024-03-28T00:24:02.519052+0000","last_peered":"2024-03-28T00:24:02.519052+0000","last_clean":"2024-03-28T00:24:02.519052+0000","last_became_active":"2024-03-28T00:23:55.727705+0000","last_became_peered":"2024-03-28T00:23:55.727705+0000","last_unstale":"2024-03-28T00:24:02.519052+0000","last_undegraded":"2024-03-28T00:24:02.519052+0000","last_fullsized":"2024-03-28T00:24:02.519052+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_clean_scrub_stamp":"2024-03-28T00:23:54.693629+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T03:48:39.028754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":350,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:23:57.877080+0000","last_change":"2024-03-28T00:22:35.416127+0000","last_active":"2024-03-28T00:23:57.877080+0000","last_peered":"2024-03-28T00:23:57.877080+0000","last_clean":"2024-03-28T00:23:57.877080+0000","last_became_active":"2024-03-28T00:22:35.415710+0000","last_became_peered":"2024-03-28T00:22:35.415710+0000","last_unstale":"2024-03-28T00:23:57.877080+0000","last_undegraded":"2024-03-28T00:23:57.877080+0000","last_fullsized":"2024-03-28T00:23:57.877080+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:18:46.602205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:18:46.602205+0000","last_clean_scrub_stamp":"2024-03-28T00:18:46.602205+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T11:31:44.851848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:23:57.723892+0000","last_change":"2024-03-28T00:23:56.741550+0000","last_active":"2024-03-28T00:23:57.723892+0000","last_peered":"2024-03-28T00:23:57.723892+0000","last_clean":"2024-03-28T00:23:57.723892+0000","last_became_active":"2024-03-28T00:23:56.740750+0000","last_became_peered":"2024-03-28T00:23:56.740750+0000","last_unstale":"2024-03-28T00:23:57.723892+0000","last_undegraded":"2024-03-28T00:23:57.723892+0000","last_fullsized":"2024-03-28T00:23:57.723892+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T09:14:35.259597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:24:01.496602+0000","last_change":"2024-03-28T00:23:55.731623+0000","last_active":"2024-03-28T00:24:01.496602+0000","last_peered":"2024-03-28T00:24:01.496602+0000","last_clean":"2024-03-28T00:24:01.496602+0000","last_became_active":"2024-03-28T00:23:55.731292+0000","last_became_peered":"2024-03-28T00:23:55.731292+0000","last_unstale":"2024-03-28T00:24:01.496602+0000","last_undegraded":"2024-03-28T00:24:01.496602+0000","last_fullsized":"2024-03-28T00:24:01.496602+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_clean_scrub_stamp":"2024-03-28T00:23:54.693629+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T06:16:23.001339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:24:02.520938+0000","last_change":"2024-03-28T00:23:55.745553+0000","last_active":"2024-03-28T00:24:02.520938+0000","last_peered":"2024-03-28T00:24:02.520938+0000","last_clean":"2024-03-28T00:24:02.520938+0000","last_became_active":"2024-03-28T00:23:55.745204+0000","last_became_peered":"2024-03-28T00:23:55.745204+0000","last_unstale":"2024-03-28T00:24:02.520938+0000","last_undegraded":"2024-03-28T00:24:02.520938+0000","last_fullsized":"2024-03-28T00:24:02.520938+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_clean_scrub_stamp":"2024-03-28T00:23:54.693629+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T03:23:50.584362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:23:57.725106+0000","last_change":"2024-03-28T00:23:56.751394+0000","last_active":"2024-03-28T00:23:57.725106+0000","last_peered":"2024-03-28T00:23:57.725106+0000","last_clean":"2024-03-28T00:23:57.725106+0000","last_became_active":"2024-03-28T00:23:56.751025+0000","last_became_peered":"2024-03-28T00:23:56.751025+0000","last_unstale":"2024-03-28T00:23:57.725106+0000","last_undegraded":"2024-03-28T00:23:57.725106+0000","last_fullsized":"2024-03-28T00:23:57.725106+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T03:59:28.738018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:24:01.504330+0000","last_change":"2024-03-28T00:23:55.735055+0000","last_active":"2024-03-28T00:24:01.504330+0000","last_peered":"2024-03-28T00:24:01.504330+0000","last_clean":"2024-03-28T00:24:01.504330+0000","last_became_active":"2024-03-28T00:23:55.734747+0000","last_became_peered":"2024-03-28T00:23:55.734747+0000","last_unstale":"2024-03-28T00:24:01.504330+0000","last_undegraded":"2024-03-28T00:24:01.504330+0000","last_fullsized":"2024-03-28T00:24:01.504330+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_clean_scrub_stamp":"2024-03-28T00:23:54.693629+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T03:02:13.533883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:23:57.883285+0000","last_change":"2024-03-28T00:23:56.760581+0000","last_active":"2024-03-28T00:23:57.883285+0000","last_peered":"2024-03-28T00:23:57.883285+0000","last_clean":"2024-03-28T00:23:57.883285+0000","last_became_active":"2024-03-28T00:23:56.756807+0000","last_became_peered":"2024-03-28T00:23:56.756807+0000","last_unstale":"2024-03-28T00:23:57.883285+0000","last_undegraded":"2024-03-28T00:23:57.883285+0000","last_fullsized":"2024-03-28T00:23:57.883285+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T07:13:34.097671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'7","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:24:03.534835+0000","last_change":"2024-03-28T00:23:55.727864+0000","last_active":"2024-03-28T00:24:03.534835+0000","last_peered":"2024-03-28T00:24:03.534835+0000","last_clean":"2024-03-28T00:24:03.534835+0000","last_became_active":"2024-03-28T00:23:55.727507+0000","last_became_peered":"2024-03-28T00:23:55.727507+0000","last_unstale":"2024-03-28T00:24:03.534835+0000","last_undegraded":"2024-03-28T00:24:03.534835+0000","last_fullsized":"2024-03-28T00:24:03.534835+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_clean_scrub_stamp":"2024-03-28T00:23:54.693629+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T09:01:45.401356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:23:57.880964+0000","last_change":"2024-03-28T00:23:56.766973+0000","last_active":"2024-03-28T00:23:57.880964+0000","last_peered":"2024-03-28T00:23:57.880964+0000","last_clean":"2024-03-28T00:23:57.880964+0000","last_became_active":"2024-03-28T00:23:56.766549+0000","last_became_peered":"2024-03-28T00:23:56.766549+0000","last_unstale":"2024-03-28T00:23:57.880964+0000","last_undegraded":"2024-03-28T00:23:57.880964+0000","last_fullsized":"2024-03-28T00:23:57.880964+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T02:37:52.330879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:24:00.486330+0000","last_change":"2024-03-28T00:23:55.728936+0000","last_active":"2024-03-28T00:24:00.486330+0000","last_peered":"2024-03-28T00:24:00.486330+0000","last_clean":"2024-03-28T00:24:00.486330+0000","last_became_active":"2024-03-28T00:23:55.728517+0000","last_became_peered":"2024-03-28T00:23:55.728517+0000","last_unstale":"2024-03-28T00:24:00.486330+0000","last_undegraded":"2024-03-28T00:24:00.486330+0000","last_fullsized":"2024-03-28T00:24:00.486330+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_clean_scrub_stamp":"2024-03-28T00:23:54.693629+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T10:13:20.841286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:23:57.877099+0000","last_change":"2024-03-28T00:23:56.738438+0000","last_active":"2024-03-28T00:23:57.877099+0000","last_peered":"2024-03-28T00:23:57.877099+0000","last_clean":"2024-03-28T00:23:57.877099+0000","last_became_active":"2024-03-28T00:23:56.738092+0000","last_became_peered":"2024-03-28T00:23:56.738092+0000","last_unstale":"2024-03-28T00:23:57.877099+0000","last_undegraded":"2024-03-28T00:23:57.877099+0000","last_fullsized":"2024-03-28T00:23:57.877099+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T02:54:57.424857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:24:00.487877+0000","last_change":"2024-03-28T00:23:55.741042+0000","last_active":"2024-03-28T00:24:00.487877+0000","last_peered":"2024-03-28T00:24:00.487877+0000","last_clean":"2024-03-28T00:24:00.487877+0000","last_became_active":"2024-03-28T00:23:55.740165+0000","last_became_peered":"2024-03-28T00:23:55.740165+0000","last_unstale":"2024-03-28T00:24:00.487877+0000","last_undegraded":"2024-03-28T00:24:00.487877+0000","last_fullsized":"2024-03-28T00:24:00.487877+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_clean_scrub_stamp":"2024-03-28T00:23:54.693629+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T04:47:58.519326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:23:57.970053+0000","last_change":"2024-03-28T00:23:56.760261+0000","last_active":"2024-03-28T00:23:57.970053+0000","last_peered":"2024-03-28T00:23:57.970053+0000","last_clean":"2024-03-28T00:23:57.970053+0000","last_became_active":"2024-03-28T00:23:56.759925+0000","last_became_peered":"2024-03-28T00:23:56.759925+0000","last_unstale":"2024-03-28T00:23:57.970053+0000","last_undegraded":"2024-03-28T00:23:57.970053+0000","last_fullsized":"2024-03-28T00:23:57.970053+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T00:58:03.546582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:23:57.877653+0000","last_change":"2024-03-28T00:23:55.733455+0000","last_active":"2024-03-28T00:23:57.877653+0000","last_peered":"2024-03-28T00:23:57.877653+0000","last_clean":"2024-03-28T00:23:57.877653+0000","last_became_active":"2024-03-28T00:23:55.733116+0000","last_became_peered":"2024-03-28T00:23:55.733116+0000","last_unstale":"2024-03-28T00:23:57.877653+0000","last_undegraded":"2024-03-28T00:23:57.877653+0000","last_fullsized":"2024-03-28T00:23:57.877653+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_clean_scrub_stamp":"2024-03-28T00:23:54.693629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T07:43:41.301213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:23:57.880176+0000","last_change":"2024-03-28T00:23:56.766884+0000","last_active":"2024-03-28T00:23:57.880176+0000","last_peered":"2024-03-28T00:23:57.880176+0000","last_clean":"2024-03-28T00:23:57.880176+0000","last_became_active":"2024-03-28T00:23:56.766476+0000","last_became_peered":"2024-03-28T00:23:56.766476+0000","last_unstale":"2024-03-28T00:23:57.880176+0000","last_undegraded":"2024-03-28T00:23:57.880176+0000","last_fullsized":"2024-03-28T00:23:57.880176+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T08:01:13.280796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:24:00.484149+0000","last_change":"2024-03-28T00:23:55.728001+0000","last_active":"2024-03-28T00:24:00.484149+0000","last_peered":"2024-03-28T00:24:00.484149+0000","last_clean":"2024-03-28T00:24:00.484149+0000","last_became_active":"2024-03-28T00:23:55.727659+0000","last_became_peered":"2024-03-28T00:23:55.727659+0000","last_unstale":"2024-03-28T00:24:00.484149+0000","last_undegraded":"2024-03-28T00:24:00.484149+0000","last_fullsized":"2024-03-28T00:24:00.484149+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_clean_scrub_stamp":"2024-03-28T00:23:54.693629+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T05:59:09.165928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:23:57.878274+0000","last_change":"2024-03-28T00:23:56.740843+0000","last_active":"2024-03-28T00:23:57.878274+0000","last_peered":"2024-03-28T00:23:57.878274+0000","last_clean":"2024-03-28T00:23:57.878274+0000","last_became_active":"2024-03-28T00:23:56.740364+0000","last_became_peered":"2024-03-28T00:23:56.740364+0000","last_unstale":"2024-03-28T00:23:57.878274+0000","last_undegraded":"2024-03-28T00:23:57.878274+0000","last_fullsized":"2024-03-28T00:23:57.878274+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T10:44:10.040424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:23:57.878041+0000","last_change":"2024-03-28T00:23:55.745772+0000","last_active":"2024-03-28T00:23:57.878041+0000","last_peered":"2024-03-28T00:23:57.878041+0000","last_clean":"2024-03-28T00:23:57.878041+0000","last_became_active":"2024-03-28T00:23:55.745457+0000","last_became_peered":"2024-03-28T00:23:55.745457+0000","last_unstale":"2024-03-28T00:23:57.878041+0000","last_undegraded":"2024-03-28T00:23:57.878041+0000","last_fullsized":"2024-03-28T00:23:57.878041+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_clean_scrub_stamp":"2024-03-28T00:23:54.693629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T01:27:44.632523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:23:57.878100+0000","last_change":"2024-03-28T00:23:56.737008+0000","last_active":"2024-03-28T00:23:57.878100+0000","last_peered":"2024-03-28T00:23:57.878100+0000","last_clean":"2024-03-28T00:23:57.878100+0000","last_became_active":"2024-03-28T00:23:56.736399+0000","last_became_peered":"2024-03-28T00:23:56.736399+0000","last_unstale":"2024-03-28T00:23:57.878100+0000","last_undegraded":"2024-03-28T00:23:57.878100+0000","last_fullsized":"2024-03-28T00:23:57.878100+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T07:16:05.968362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:23:58.374815+0000","last_change":"2024-03-28T00:23:55.734518+0000","last_active":"2024-03-28T00:23:58.374815+0000","last_peered":"2024-03-28T00:23:58.374815+0000","last_clean":"2024-03-28T00:23:58.374815+0000","last_became_active":"2024-03-28T00:23:55.734200+0000","last_became_peered":"2024-03-28T00:23:55.734200+0000","last_unstale":"2024-03-28T00:23:58.374815+0000","last_undegraded":"2024-03-28T00:23:58.374815+0000","last_fullsized":"2024-03-28T00:23:58.374815+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_clean_scrub_stamp":"2024-03-28T00:23:54.693629+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T05:00:56.602654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:23:57.877913+0000","last_change":"2024-03-28T00:23:56.757249+0000","last_active":"2024-03-28T00:23:57.877913+0000","last_peered":"2024-03-28T00:23:57.877913+0000","last_clean":"2024-03-28T00:23:57.877913+0000","last_became_active":"2024-03-28T00:23:56.756957+0000","last_became_peered":"2024-03-28T00:23:56.756957+0000","last_unstale":"2024-03-28T00:23:57.877913+0000","last_undegraded":"2024-03-28T00:23:57.877913+0000","last_fullsized":"2024-03-28T00:23:57.877913+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T03:55:30.166369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:23:57.901757+0000","last_change":"2024-03-28T00:23:55.741463+0000","last_active":"2024-03-28T00:23:57.901757+0000","last_peered":"2024-03-28T00:23:57.901757+0000","last_clean":"2024-03-28T00:23:57.901757+0000","last_became_active":"2024-03-28T00:23:55.741095+0000","last_became_peered":"2024-03-28T00:23:55.741095+0000","last_unstale":"2024-03-28T00:23:57.901757+0000","last_undegraded":"2024-03-28T00:23:57.901757+0000","last_fullsized":"2024-03-28T00:23:57.901757+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_clean_scrub_stamp":"2024-03-28T00:23:54.693629+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T02:58:49.587896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:23:57.877529+0000","last_change":"2024-03-28T00:23:56.736382+0000","last_active":"2024-03-28T00:23:57.877529+0000","last_peered":"2024-03-28T00:23:57.877529+0000","last_clean":"2024-03-28T00:23:57.877529+0000","last_became_active":"2024-03-28T00:23:56.736036+0000","last_became_peered":"2024-03-28T00:23:56.736036+0000","last_unstale":"2024-03-28T00:23:57.877529+0000","last_undegraded":"2024-03-28T00:23:57.877529+0000","last_fullsized":"2024-03-28T00:23:57.877529+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T04:26:01.792967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:23:57.970103+0000","last_change":"2024-03-28T00:23:56.739530+0000","last_active":"2024-03-28T00:23:57.970103+0000","last_peered":"2024-03-28T00:23:57.970103+0000","last_clean":"2024-03-28T00:23:57.970103+0000","last_became_active":"2024-03-28T00:23:56.738166+0000","last_became_peered":"2024-03-28T00:23:56.738166+0000","last_unstale":"2024-03-28T00:23:57.970103+0000","last_undegraded":"2024-03-28T00:23:57.970103+0000","last_fullsized":"2024-03-28T00:23:57.970103+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T01:11:44.872866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:23:57.877499+0000","last_change":"2024-03-28T00:23:56.736044+0000","last_active":"2024-03-28T00:23:57.877499+0000","last_peered":"2024-03-28T00:23:57.877499+0000","last_clean":"2024-03-28T00:23:57.877499+0000","last_became_active":"2024-03-28T00:23:56.735688+0000","last_became_peered":"2024-03-28T00:23:56.735688+0000","last_unstale":"2024-03-28T00:23:57.877499+0000","last_undegraded":"2024-03-28T00:23:57.877499+0000","last_fullsized":"2024-03-28T00:23:57.877499+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T08:57:16.687620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:23:57.877840+0000","last_change":"2024-03-28T00:23:56.760935+0000","last_active":"2024-03-28T00:23:57.877840+0000","last_peered":"2024-03-28T00:23:57.877840+0000","last_clean":"2024-03-28T00:23:57.877840+0000","last_became_active":"2024-03-28T00:23:56.760600+0000","last_became_peered":"2024-03-28T00:23:56.760600+0000","last_unstale":"2024-03-28T00:23:57.877840+0000","last_undegraded":"2024-03-28T00:23:57.877840+0000","last_fullsized":"2024-03-28T00:23:57.877840+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T03:44:30.047881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:23:57.879254+0000","last_change":"2024-03-28T00:23:56.760767+0000","last_active":"2024-03-28T00:23:57.879254+0000","last_peered":"2024-03-28T00:23:57.879254+0000","last_clean":"2024-03-28T00:23:57.879254+0000","last_became_active":"2024-03-28T00:23:56.760431+0000","last_became_peered":"2024-03-28T00:23:56.760431+0000","last_unstale":"2024-03-28T00:23:57.879254+0000","last_undegraded":"2024-03-28T00:23:57.879254+0000","last_fullsized":"2024-03-28T00:23:57.879254+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T01:53:43.479645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:23:57.724996+0000","last_change":"2024-03-28T00:23:56.760772+0000","last_active":"2024-03-28T00:23:57.724996+0000","last_peered":"2024-03-28T00:23:57.724996+0000","last_clean":"2024-03-28T00:23:57.724996+0000","last_became_active":"2024-03-28T00:23:56.760439+0000","last_became_peered":"2024-03-28T00:23:56.760439+0000","last_unstale":"2024-03-28T00:23:57.724996+0000","last_undegraded":"2024-03-28T00:23:57.724996+0000","last_fullsized":"2024-03-28T00:23:57.724996+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T05:46:09.592560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:23:57.970111+0000","last_change":"2024-03-28T00:23:56.740004+0000","last_active":"2024-03-28T00:23:57.970111+0000","last_peered":"2024-03-28T00:23:57.970111+0000","last_clean":"2024-03-28T00:23:57.970111+0000","last_became_active":"2024-03-28T00:23:56.739630+0000","last_became_peered":"2024-03-28T00:23:56.739630+0000","last_unstale":"2024-03-28T00:23:57.970111+0000","last_undegraded":"2024-03-28T00:23:57.970111+0000","last_fullsized":"2024-03-28T00:23:57.970111+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T01:04:39.297383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:23:57.878074+0000","last_change":"2024-03-28T00:23:56.740716+0000","last_active":"2024-03-28T00:23:57.878074+0000","last_peered":"2024-03-28T00:23:57.878074+0000","last_clean":"2024-03-28T00:23:57.878074+0000","last_became_active":"2024-03-28T00:23:56.739587+0000","last_became_peered":"2024-03-28T00:23:56.739587+0000","last_unstale":"2024-03-28T00:23:57.878074+0000","last_undegraded":"2024-03-28T00:23:57.878074+0000","last_fullsized":"2024-03-28T00:23:57.878074+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T11:17:37.115216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:23:57.883180+0000","last_change":"2024-03-28T00:23:56.761001+0000","last_active":"2024-03-28T00:23:57.883180+0000","last_peered":"2024-03-28T00:23:57.883180+0000","last_clean":"2024-03-28T00:23:57.883180+0000","last_became_active":"2024-03-28T00:23:56.757373+0000","last_became_peered":"2024-03-28T00:23:56.757373+0000","last_unstale":"2024-03-28T00:23:57.883180+0000","last_undegraded":"2024-03-28T00:23:57.883180+0000","last_fullsized":"2024-03-28T00:23:57.883180+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T02:55:47.106100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:23:57.721043+0000","last_change":"2024-03-28T00:23:56.750715+0000","last_active":"2024-03-28T00:23:57.721043+0000","last_peered":"2024-03-28T00:23:57.721043+0000","last_clean":"2024-03-28T00:23:57.721043+0000","last_became_active":"2024-03-28T00:23:56.750446+0000","last_became_peered":"2024-03-28T00:23:56.750446+0000","last_unstale":"2024-03-28T00:23:57.721043+0000","last_undegraded":"2024-03-28T00:23:57.721043+0000","last_fullsized":"2024-03-28T00:23:57.721043+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T11:51:27.935181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:23:57.880443+0000","last_change":"2024-03-28T00:23:56.752571+0000","last_active":"2024-03-28T00:23:57.880443+0000","last_peered":"2024-03-28T00:23:57.880443+0000","last_clean":"2024-03-28T00:23:57.880443+0000","last_became_active":"2024-03-28T00:23:56.751853+0000","last_became_peered":"2024-03-28T00:23:56.751853+0000","last_unstale":"2024-03-28T00:23:57.880443+0000","last_undegraded":"2024-03-28T00:23:57.880443+0000","last_fullsized":"2024-03-28T00:23:57.880443+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T05:30:02.897139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:23:57.726163+0000","last_change":"2024-03-28T00:23:56.750006+0000","last_active":"2024-03-28T00:23:57.726163+0000","last_peered":"2024-03-28T00:23:57.726163+0000","last_clean":"2024-03-28T00:23:57.726163+0000","last_became_active":"2024-03-28T00:23:56.749639+0000","last_became_peered":"2024-03-28T00:23:56.749639+0000","last_unstale":"2024-03-28T00:23:57.726163+0000","last_undegraded":"2024-03-28T00:23:57.726163+0000","last_fullsized":"2024-03-28T00:23:57.726163+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T02:26:51.076107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:23:57.724872+0000","last_change":"2024-03-28T00:23:56.764982+0000","last_active":"2024-03-28T00:23:57.724872+0000","last_peered":"2024-03-28T00:23:57.724872+0000","last_clean":"2024-03-28T00:23:57.724872+0000","last_became_active":"2024-03-28T00:23:56.764596+0000","last_became_peered":"2024-03-28T00:23:56.764596+0000","last_unstale":"2024-03-28T00:23:57.724872+0000","last_undegraded":"2024-03-28T00:23:57.724872+0000","last_fullsized":"2024-03-28T00:23:57.724872+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T09:05:30.948689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:23:57.970015+0000","last_change":"2024-03-28T00:23:56.736439+0000","last_active":"2024-03-28T00:23:57.970015+0000","last_peered":"2024-03-28T00:23:57.970015+0000","last_clean":"2024-03-28T00:23:57.970015+0000","last_became_active":"2024-03-28T00:23:56.736099+0000","last_became_peered":"2024-03-28T00:23:56.736099+0000","last_unstale":"2024-03-28T00:23:57.970015+0000","last_undegraded":"2024-03-28T00:23:57.970015+0000","last_fullsized":"2024-03-28T00:23:57.970015+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T02:57:42.821566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":4132576,"data_compressed":245983,"data_compressed_allocated":315392,"data_compressed_original":4079616,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":5849,"num_objects":69,"num_object_clones":0,"num_object_copies":207,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":146,"num_write_kb":49,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":45,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":278528,"data_stored":17916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":127,"ondisk_log_size":127,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383696,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32080,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713072,"statfs":{"total":95995035648,"available":95962185728,"internally_reserved":0,"allocated":925696,"data_stored":440051,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645332,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1015808,"data_stored":1032150,"data_compressed":35845,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808857,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32084,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713068,"statfs":{"total":95995035648,"available":95962181632,"internally_reserved":0,"allocated":929792,"data_stored":439252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070497,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32088,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713064,"statfs":{"total":95995035648,"available":95962177536,"internally_reserved":0,"allocated":933888,"data_stored":439206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332132,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":978944,"data_stored":1030628,"data_compressed":35845,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495659,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32072,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713080,"statfs":{"total":95995035648,"available":95962193920,"internally_reserved":0,"allocated":917504,"data_stored":438558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574919,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32080,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713072,"statfs":{"total":95995035648,"available":95962185728,"internally_reserved":0,"allocated":925696,"data_stored":440080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738444,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":942080,"data_stored":439946,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411394,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":937984,"data_stored":440736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182458,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":929792,"data_stored":438752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920823,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32084,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713068,"statfs":{"total":95995035648,"available":95962181632,"internally_reserved":0,"allocated":929792,"data_stored":439134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757296,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":1003520,"data_stored":1031074,"data_compressed":35845,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34201,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34201,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34201,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35845,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35845,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35845,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35845,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":3314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-28T00:24:07.008 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-28T00:24:07.249 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:07.242+0000 7f779ca00640 1 -- 172.21.15.119:0/4222078301 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7798105490 msgr2=0x7f779810bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:24:07.249 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:07.242+0000 7f779ca00640 1 --2- 172.21.15.119:0/4222078301 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7798105490 0x7f779810bd20 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f7790009ca0 tx=0x7f779002c0f0 comp rx=0 tx=0).stop 2024-03-28T00:24:07.249 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:07.243+0000 7f779ca00640 1 -- 172.21.15.119:0/4222078301 shutdown_connections 2024-03-28T00:24:07.249 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:07.243+0000 7f779ca00640 1 --2- 172.21.15.119:0/4222078301 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7798105490 0x7f779810bd20 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:07.249 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:07.243+0000 7f779ca00640 1 --2- 172.21.15.119:0/4222078301 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7798104ad0 0x7f7798104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:07.249 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:07.243+0000 7f779ca00640 1 --2- 172.21.15.119:0/4222078301 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f77981038e0 0x7f7798103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:07.250 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:07.243+0000 7f779ca00640 1 -- 172.21.15.119:0/4222078301 >> 172.21.15.119:0/4222078301 conn(0x7f77980fee50 msgr2=0x7f7798101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:24:07.250 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:07.243+0000 7f779ca00640 1 -- 172.21.15.119:0/4222078301 shutdown_connections 2024-03-28T00:24:07.250 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:07.243+0000 7f779ca00640 1 -- 172.21.15.119:0/4222078301 wait complete. 2024-03-28T00:24:07.252 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:07.244+0000 7f779ca00640 1 Processor -- start 2024-03-28T00:24:07.252 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:07.244+0000 7f779ca00640 1 -- start start 2024-03-28T00:24:07.252 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:07.244+0000 7f779ca00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f77981038e0 0x7f7798199ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:24:07.252 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:07.244+0000 7f779ca00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7798104ad0 0x7f779819a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:24:07.252 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:07.244+0000 7f779ca00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7798105490 0x7f77981a1150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:24:07.252 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:07.244+0000 7f779ca00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f779810f560 con 0x7f7798105490 2024-03-28T00:24:07.252 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:07.244+0000 7f779ca00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f779810f3e0 con 0x7f7798104ad0 2024-03-28T00:24:07.252 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:07.244+0000 7f779ca00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f779810f6e0 con 0x7f77981038e0 2024-03-28T00:24:07.252 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:07.244+0000 7f7795e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f77981038e0 0x7f7798199ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:24:07.252 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:07.244+0000 7f7795e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f77981038e0 0x7f7798199ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.119:36960/0 (socket says 172.21.15.119:36960) 2024-03-28T00:24:07.253 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:07.244+0000 7f7795400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7798104ad0 0x7f779819a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:24:07.253 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:07.244+0000 7f7795e00640 1 -- 172.21.15.119:0/2068742615 learned_addr learned my addr 172.21.15.119:0/2068742615 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:24:07.253 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:07.244+0000 7f7796800640 1 --2- 172.21.15.119:0/2068742615 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7798105490 0x7f77981a1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:24:07.253 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:07.245+0000 7f7795400640 1 -- 172.21.15.119:0/2068742615 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f77981038e0 msgr2=0x7f7798199ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:24:07.253 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:07.245+0000 7f7795400640 1 --2- 172.21.15.119:0/2068742615 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f77981038e0 0x7f7798199ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:07.253 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:07.245+0000 7f7795400640 1 -- 172.21.15.119:0/2068742615 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7798105490 msgr2=0x7f77981a1150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:24:07.253 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:07.245+0000 7f7795400640 1 --2- 172.21.15.119:0/2068742615 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7798105490 0x7f77981a1150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:07.253 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:07.245+0000 7f7795400640 1 -- 172.21.15.119:0/2068742615 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77981a17c0 con 0x7f7798104ad0 2024-03-28T00:24:07.253 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:07.245+0000 7f7795e00640 1 --2- 172.21.15.119:0/2068742615 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f77981038e0 0x7f7798199ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T00:24:07.253 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:07.245+0000 7f7795400640 1 --2- 172.21.15.119:0/2068742615 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7798104ad0 0x7f779819a0e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f778400e9f0 tx=0x7f778400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:24:07.253 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:07.245+0000 7f778ea00640 1 -- 172.21.15.119:0/2068742615 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f778400ccc0 con 0x7f7798104ad0 2024-03-28T00:24:07.253 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:07.245+0000 7f778ea00640 1 -- 172.21.15.119:0/2068742615 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7784005020 con 0x7f7798104ad0 2024-03-28T00:24:07.253 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:07.245+0000 7f779ca00640 1 -- 172.21.15.119:0/2068742615 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77981a1ab0 con 0x7f7798104ad0 2024-03-28T00:24:07.253 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:07.245+0000 7f778ea00640 1 -- 172.21.15.119:0/2068742615 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77840049e0 con 0x7f7798104ad0 2024-03-28T00:24:07.253 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:07.245+0000 7f779ca00640 1 -- 172.21.15.119:0/2068742615 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77981a2000 con 0x7f7798104ad0 2024-03-28T00:24:07.254 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:07.246+0000 7f779ca00640 1 -- 172.21.15.119:0/2068742615 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f779810ac00 con 0x7f7798104ad0 2024-03-28T00:24:07.255 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:07.249+0000 7f778ea00640 1 -- 172.21.15.119:0/2068742615 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 17) v1 ==== 137342+0+0 (secure 0 0 0) 0x7f7784005560 con 0x7f7798104ad0 2024-03-28T00:24:07.256 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:07.250+0000 7f778ea00640 1 --2- 172.21.15.119:0/2068742615 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f777c0a2910 0x7f777c0a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:24:07.256 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:07.250+0000 7f778ea00640 1 -- 172.21.15.119:0/2068742615 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f77840cea60 con 0x7f7798104ad0 2024-03-28T00:24:07.256 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:07.250+0000 7f7795e00640 1 --2- 172.21.15.119:0/2068742615 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f777c0a2910 0x7f777c0a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:24:07.257 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:07.251+0000 7f7795e00640 1 --2- 172.21.15.119:0/2068742615 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f777c0a2910 0x7f777c0a4dd0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f77880058c0 tx=0x7f7788005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:24:07.260 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:07.254+0000 7f778ea00640 1 -- 172.21.15.119:0/2068742615 <== mon.2 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f77840051d0 con 0x7f7798104ad0 2024-03-28T00:24:07.456 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:07.450+0000 7f779ca00640 1 -- 172.21.15.119:0/2068742615 --> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7798061ae0 con 0x7f777c0a2910 2024-03-28T00:24:07.467 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:07.461+0000 7f778ea00640 1 -- 172.21.15.119:0/2068742615 <== mgr.14152 v2:172.21.15.119:6800/1401937096 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+258887 (secure 0 0 0) 0x7f7798061ae0 con 0x7f777c0a2910 2024-03-28T00:24:07.471 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-28T00:24:07.472 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2024-03-28T00:24:07.473 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:07.466+0000 7f779ca00640 1 -- 172.21.15.119:0/2068742615 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f777c0a2910 msgr2=0x7f777c0a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:24:07.474 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:07.466+0000 7f779ca00640 1 --2- 172.21.15.119:0/2068742615 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f777c0a2910 0x7f777c0a4dd0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f77880058c0 tx=0x7f7788005f90 comp rx=0 tx=0).stop 2024-03-28T00:24:07.474 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:07.466+0000 7f779ca00640 1 -- 172.21.15.119:0/2068742615 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7798104ad0 msgr2=0x7f779819a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:24:07.474 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:07.466+0000 7f779ca00640 1 --2- 172.21.15.119:0/2068742615 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7798104ad0 0x7f779819a0e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f778400e9f0 tx=0x7f778400eeb0 comp rx=0 tx=0).stop 2024-03-28T00:24:07.474 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:07.466+0000 7f779ca00640 1 -- 172.21.15.119:0/2068742615 shutdown_connections 2024-03-28T00:24:07.474 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:07.466+0000 7f779ca00640 1 --2- 172.21.15.119:0/2068742615 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f777c0a2910 0x7f777c0a4dd0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:07.474 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:07.466+0000 7f779ca00640 1 --2- 172.21.15.119:0/2068742615 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7798105490 0x7f77981a1150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:07.474 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:07.466+0000 7f779ca00640 1 --2- 172.21.15.119:0/2068742615 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7798104ad0 0x7f779819a0e0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:07.474 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:07.466+0000 7f779ca00640 1 --2- 172.21.15.119:0/2068742615 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f77981038e0 0x7f7798199ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:07.474 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:07.466+0000 7f779ca00640 1 -- 172.21.15.119:0/2068742615 >> 172.21.15.119:0/2068742615 conn(0x7f77980fee50 msgr2=0x7f7798107dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:24:07.474 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:07.466+0000 7f779ca00640 1 -- 172.21.15.119:0/2068742615 shutdown_connections 2024-03-28T00:24:07.474 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:07.466+0000 7f779ca00640 1 -- 172.21.15.119:0/2068742615 wait complete. 2024-03-28T00:24:07.493 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":318,"stamp":"2024-03-28T00:24:05.722770+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596217,"num_objects":71,"num_object_clones":0,"num_object_copies":213,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":327,"num_write_kb":2987,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":45,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":264,"ondisk_log_size":264,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":368576,"kb_used_data":11104,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124573248,"statfs":{"total":1151940427776,"available":1151563005952,"internally_reserved":0,"allocated":11370496,"data_stored":7049567,"data_compressed":107535,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19143,"internal_metadata":365933881},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5267,"num_objects":67,"num_object_clones":0,"num_object_copies":201,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":142,"num_write_kb":48,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":44,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001276"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:23:57.970251+0000","last_change":"2024-03-28T00:23:56.763630+0000","last_active":"2024-03-28T00:23:57.970251+0000","last_peered":"2024-03-28T00:23:57.970251+0000","last_clean":"2024-03-28T00:23:57.970251+0000","last_became_active":"2024-03-28T00:23:56.763296+0000","last_became_peered":"2024-03-28T00:23:56.763296+0000","last_unstale":"2024-03-28T00:23:57.970251+0000","last_undegraded":"2024-03-28T00:23:57.970251+0000","last_fullsized":"2024-03-28T00:23:57.970251+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T11:40:36.071225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:23:57.880797+0000","last_change":"2024-03-28T00:23:56.763628+0000","last_active":"2024-03-28T00:23:57.880797+0000","last_peered":"2024-03-28T00:23:57.880797+0000","last_clean":"2024-03-28T00:23:57.880797+0000","last_became_active":"2024-03-28T00:23:56.761753+0000","last_became_peered":"2024-03-28T00:23:56.761753+0000","last_unstale":"2024-03-28T00:23:57.880797+0000","last_undegraded":"2024-03-28T00:23:57.880797+0000","last_fullsized":"2024-03-28T00:23:57.880797+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T11:48:50.804269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:23:57.880032+0000","last_change":"2024-03-28T00:23:56.759198+0000","last_active":"2024-03-28T00:23:57.880032+0000","last_peered":"2024-03-28T00:23:57.880032+0000","last_clean":"2024-03-28T00:23:57.880032+0000","last_became_active":"2024-03-28T00:23:56.758780+0000","last_became_peered":"2024-03-28T00:23:56.758780+0000","last_unstale":"2024-03-28T00:23:57.880032+0000","last_undegraded":"2024-03-28T00:23:57.880032+0000","last_fullsized":"2024-03-28T00:23:57.880032+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T10:46:15.731193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:23:57.878082+0000","last_change":"2024-03-28T00:23:56.736651+0000","last_active":"2024-03-28T00:23:57.878082+0000","last_peered":"2024-03-28T00:23:57.878082+0000","last_clean":"2024-03-28T00:23:57.878082+0000","last_became_active":"2024-03-28T00:23:56.736063+0000","last_became_peered":"2024-03-28T00:23:56.736063+0000","last_unstale":"2024-03-28T00:23:57.878082+0000","last_undegraded":"2024-03-28T00:23:57.878082+0000","last_fullsized":"2024-03-28T00:23:57.878082+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T09:34:28.221202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:23:57.877088+0000","last_change":"2024-03-28T00:23:56.751413+0000","last_active":"2024-03-28T00:23:57.877088+0000","last_peered":"2024-03-28T00:23:57.877088+0000","last_clean":"2024-03-28T00:23:57.877088+0000","last_became_active":"2024-03-28T00:23:56.751038+0000","last_became_peered":"2024-03-28T00:23:56.751038+0000","last_unstale":"2024-03-28T00:23:57.877088+0000","last_undegraded":"2024-03-28T00:23:57.877088+0000","last_fullsized":"2024-03-28T00:23:57.877088+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T10:45:08.645799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:23:57.877718+0000","last_change":"2024-03-28T00:23:56.750970+0000","last_active":"2024-03-28T00:23:57.877718+0000","last_peered":"2024-03-28T00:23:57.877718+0000","last_clean":"2024-03-28T00:23:57.877718+0000","last_became_active":"2024-03-28T00:23:56.750608+0000","last_became_peered":"2024-03-28T00:23:56.750608+0000","last_unstale":"2024-03-28T00:23:57.877718+0000","last_undegraded":"2024-03-28T00:23:57.877718+0000","last_fullsized":"2024-03-28T00:23:57.877718+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T00:32:07.448685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:23:57.877399+0000","last_change":"2024-03-28T00:23:56.751489+0000","last_active":"2024-03-28T00:23:57.877399+0000","last_peered":"2024-03-28T00:23:57.877399+0000","last_clean":"2024-03-28T00:23:57.877399+0000","last_became_active":"2024-03-28T00:23:56.750531+0000","last_became_peered":"2024-03-28T00:23:56.750531+0000","last_unstale":"2024-03-28T00:23:57.877399+0000","last_undegraded":"2024-03-28T00:23:57.877399+0000","last_fullsized":"2024-03-28T00:23:57.877399+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T11:18:22.604126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:23:57.878853+0000","last_change":"2024-03-28T00:23:56.751537+0000","last_active":"2024-03-28T00:23:57.878853+0000","last_peered":"2024-03-28T00:23:57.878853+0000","last_clean":"2024-03-28T00:23:57.878853+0000","last_became_active":"2024-03-28T00:23:56.751185+0000","last_became_peered":"2024-03-28T00:23:56.751185+0000","last_unstale":"2024-03-28T00:23:57.878853+0000","last_undegraded":"2024-03-28T00:23:57.878853+0000","last_fullsized":"2024-03-28T00:23:57.878853+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T03:21:30.334886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:23:57.726349+0000","last_change":"2024-03-28T00:23:56.739405+0000","last_active":"2024-03-28T00:23:57.726349+0000","last_peered":"2024-03-28T00:23:57.726349+0000","last_clean":"2024-03-28T00:23:57.726349+0000","last_became_active":"2024-03-28T00:23:56.739085+0000","last_became_peered":"2024-03-28T00:23:56.739085+0000","last_unstale":"2024-03-28T00:23:57.726349+0000","last_undegraded":"2024-03-28T00:23:57.726349+0000","last_fullsized":"2024-03-28T00:23:57.726349+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T10:57:08.892446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:23:57.878318+0000","last_change":"2024-03-28T00:23:56.736880+0000","last_active":"2024-03-28T00:23:57.878318+0000","last_peered":"2024-03-28T00:23:57.878318+0000","last_clean":"2024-03-28T00:23:57.878318+0000","last_became_active":"2024-03-28T00:23:56.736524+0000","last_became_peered":"2024-03-28T00:23:56.736524+0000","last_unstale":"2024-03-28T00:23:57.878318+0000","last_undegraded":"2024-03-28T00:23:57.878318+0000","last_fullsized":"2024-03-28T00:23:57.878318+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T09:55:19.552415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:23:57.878048+0000","last_change":"2024-03-28T00:23:56.736717+0000","last_active":"2024-03-28T00:23:57.878048+0000","last_peered":"2024-03-28T00:23:57.878048+0000","last_clean":"2024-03-28T00:23:57.878048+0000","last_became_active":"2024-03-28T00:23:56.735981+0000","last_became_peered":"2024-03-28T00:23:56.735981+0000","last_unstale":"2024-03-28T00:23:57.878048+0000","last_undegraded":"2024-03-28T00:23:57.878048+0000","last_fullsized":"2024-03-28T00:23:57.878048+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T12:02:54.293418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:23:57.877915+0000","last_change":"2024-03-28T00:23:56.740591+0000","last_active":"2024-03-28T00:23:57.877915+0000","last_peered":"2024-03-28T00:23:57.877915+0000","last_clean":"2024-03-28T00:23:57.877915+0000","last_became_active":"2024-03-28T00:23:56.739965+0000","last_became_peered":"2024-03-28T00:23:56.739965+0000","last_unstale":"2024-03-28T00:23:57.877915+0000","last_undegraded":"2024-03-28T00:23:57.877915+0000","last_fullsized":"2024-03-28T00:23:57.877915+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T07:10:35.426260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:23:57.725043+0000","last_change":"2024-03-28T00:23:56.744686+0000","last_active":"2024-03-28T00:23:57.725043+0000","last_peered":"2024-03-28T00:23:57.725043+0000","last_clean":"2024-03-28T00:23:57.725043+0000","last_became_active":"2024-03-28T00:23:56.744355+0000","last_became_peered":"2024-03-28T00:23:56.744355+0000","last_unstale":"2024-03-28T00:23:57.725043+0000","last_undegraded":"2024-03-28T00:23:57.725043+0000","last_fullsized":"2024-03-28T00:23:57.725043+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T07:52:27.360917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:23:57.878787+0000","last_change":"2024-03-28T00:23:56.762465+0000","last_active":"2024-03-28T00:23:57.878787+0000","last_peered":"2024-03-28T00:23:57.878787+0000","last_clean":"2024-03-28T00:23:57.878787+0000","last_became_active":"2024-03-28T00:23:56.751467+0000","last_became_peered":"2024-03-28T00:23:56.751467+0000","last_unstale":"2024-03-28T00:23:57.878787+0000","last_undegraded":"2024-03-28T00:23:57.878787+0000","last_fullsized":"2024-03-28T00:23:57.878787+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T10:55:33.880872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:23:57.878882+0000","last_change":"2024-03-28T00:23:56.759344+0000","last_active":"2024-03-28T00:23:57.878882+0000","last_peered":"2024-03-28T00:23:57.878882+0000","last_clean":"2024-03-28T00:23:57.878882+0000","last_became_active":"2024-03-28T00:23:56.758694+0000","last_became_peered":"2024-03-28T00:23:56.758694+0000","last_unstale":"2024-03-28T00:23:57.878882+0000","last_undegraded":"2024-03-28T00:23:57.878882+0000","last_fullsized":"2024-03-28T00:23:57.878882+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T02:43:42.586070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:23:57.725824+0000","last_change":"2024-03-28T00:23:56.759298+0000","last_active":"2024-03-28T00:23:57.725824+0000","last_peered":"2024-03-28T00:23:57.725824+0000","last_clean":"2024-03-28T00:23:57.725824+0000","last_became_active":"2024-03-28T00:23:56.759006+0000","last_became_peered":"2024-03-28T00:23:56.759006+0000","last_unstale":"2024-03-28T00:23:57.725824+0000","last_undegraded":"2024-03-28T00:23:57.725824+0000","last_fullsized":"2024-03-28T00:23:57.725824+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T00:43:09.433789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:23:57.970302+0000","last_change":"2024-03-28T00:23:56.760324+0000","last_active":"2024-03-28T00:23:57.970302+0000","last_peered":"2024-03-28T00:23:57.970302+0000","last_clean":"2024-03-28T00:23:57.970302+0000","last_became_active":"2024-03-28T00:23:56.759866+0000","last_became_peered":"2024-03-28T00:23:56.759866+0000","last_unstale":"2024-03-28T00:23:57.970302+0000","last_undegraded":"2024-03-28T00:23:57.970302+0000","last_fullsized":"2024-03-28T00:23:57.970302+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T01:11:15.923114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:23:57.882402+0000","last_change":"2024-03-28T00:23:56.755227+0000","last_active":"2024-03-28T00:23:57.882402+0000","last_peered":"2024-03-28T00:23:57.882402+0000","last_clean":"2024-03-28T00:23:57.882402+0000","last_became_active":"2024-03-28T00:23:56.754767+0000","last_became_peered":"2024-03-28T00:23:56.754767+0000","last_unstale":"2024-03-28T00:23:57.882402+0000","last_undegraded":"2024-03-28T00:23:57.882402+0000","last_fullsized":"2024-03-28T00:23:57.882402+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T07:26:04.797904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:23:57.877936+0000","last_change":"2024-03-28T00:23:56.744199+0000","last_active":"2024-03-28T00:23:57.877936+0000","last_peered":"2024-03-28T00:23:57.877936+0000","last_clean":"2024-03-28T00:23:57.877936+0000","last_became_active":"2024-03-28T00:23:56.743848+0000","last_became_peered":"2024-03-28T00:23:56.743848+0000","last_unstale":"2024-03-28T00:23:57.877936+0000","last_undegraded":"2024-03-28T00:23:57.877936+0000","last_fullsized":"2024-03-28T00:23:57.877936+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T10:08:01.426115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'9","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:24:01.500088+0000","last_change":"2024-03-28T00:23:55.731996+0000","last_active":"2024-03-28T00:24:01.500088+0000","last_peered":"2024-03-28T00:24:01.500088+0000","last_clean":"2024-03-28T00:24:01.500088+0000","last_became_active":"2024-03-28T00:23:55.731626+0000","last_became_peered":"2024-03-28T00:23:55.731626+0000","last_unstale":"2024-03-28T00:24:01.500088+0000","last_undegraded":"2024-03-28T00:24:01.500088+0000","last_fullsized":"2024-03-28T00:24:01.500088+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_clean_scrub_stamp":"2024-03-28T00:23:54.693629+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T08:56:00.420387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:23:57.881708+0000","last_change":"2024-03-28T00:23:56.757048+0000","last_active":"2024-03-28T00:23:57.881708+0000","last_peered":"2024-03-28T00:23:57.881708+0000","last_clean":"2024-03-28T00:23:57.881708+0000","last_became_active":"2024-03-28T00:23:56.752710+0000","last_became_peered":"2024-03-28T00:23:56.752710+0000","last_unstale":"2024-03-28T00:23:57.881708+0000","last_undegraded":"2024-03-28T00:23:57.881708+0000","last_fullsized":"2024-03-28T00:23:57.881708+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T06:26:39.723255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:23:57.882574+0000","last_change":"2024-03-28T00:23:55.748191+0000","last_active":"2024-03-28T00:23:57.882574+0000","last_peered":"2024-03-28T00:23:57.882574+0000","last_clean":"2024-03-28T00:23:57.882574+0000","last_became_active":"2024-03-28T00:23:55.747154+0000","last_became_peered":"2024-03-28T00:23:55.747154+0000","last_unstale":"2024-03-28T00:23:57.882574+0000","last_undegraded":"2024-03-28T00:23:57.882574+0000","last_fullsized":"2024-03-28T00:23:57.882574+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_clean_scrub_stamp":"2024-03-28T00:23:54.693629+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T07:17:12.296494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:23:57.881926+0000","last_change":"2024-03-28T00:23:56.758979+0000","last_active":"2024-03-28T00:23:57.881926+0000","last_peered":"2024-03-28T00:23:57.881926+0000","last_clean":"2024-03-28T00:23:57.881926+0000","last_became_active":"2024-03-28T00:23:56.752628+0000","last_became_peered":"2024-03-28T00:23:56.752628+0000","last_unstale":"2024-03-28T00:23:57.881926+0000","last_undegraded":"2024-03-28T00:23:57.881926+0000","last_fullsized":"2024-03-28T00:23:57.881926+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T10:11:57.245938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"85'7","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:24:02.517073+0000","last_change":"2024-03-28T00:23:55.728972+0000","last_active":"2024-03-28T00:24:02.517073+0000","last_peered":"2024-03-28T00:24:02.517073+0000","last_clean":"2024-03-28T00:24:02.517073+0000","last_became_active":"2024-03-28T00:23:55.728053+0000","last_became_peered":"2024-03-28T00:23:55.728053+0000","last_unstale":"2024-03-28T00:24:02.517073+0000","last_undegraded":"2024-03-28T00:24:02.517073+0000","last_fullsized":"2024-03-28T00:24:02.517073+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_clean_scrub_stamp":"2024-03-28T00:23:54.693629+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T04:12:27.129196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:23:57.725920+0000","last_change":"2024-03-28T00:23:56.752585+0000","last_active":"2024-03-28T00:23:57.725920+0000","last_peered":"2024-03-28T00:23:57.725920+0000","last_clean":"2024-03-28T00:23:57.725920+0000","last_became_active":"2024-03-28T00:23:56.752252+0000","last_became_peered":"2024-03-28T00:23:56.752252+0000","last_unstale":"2024-03-28T00:23:57.725920+0000","last_undegraded":"2024-03-28T00:23:57.725920+0000","last_fullsized":"2024-03-28T00:23:57.725920+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T11:51:02.864486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:23:57.877516+0000","last_change":"2024-03-28T00:23:55.726859+0000","last_active":"2024-03-28T00:23:57.877516+0000","last_peered":"2024-03-28T00:23:57.877516+0000","last_clean":"2024-03-28T00:23:57.877516+0000","last_became_active":"2024-03-28T00:23:55.725372+0000","last_became_peered":"2024-03-28T00:23:55.725372+0000","last_unstale":"2024-03-28T00:23:57.877516+0000","last_undegraded":"2024-03-28T00:23:57.877516+0000","last_fullsized":"2024-03-28T00:23:57.877516+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_clean_scrub_stamp":"2024-03-28T00:23:54.693629+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T00:52:35.278544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:23:57.721317+0000","last_change":"2024-03-28T00:23:56.742076+0000","last_active":"2024-03-28T00:23:57.721317+0000","last_peered":"2024-03-28T00:23:57.721317+0000","last_clean":"2024-03-28T00:23:57.721317+0000","last_became_active":"2024-03-28T00:23:56.741761+0000","last_became_peered":"2024-03-28T00:23:56.741761+0000","last_unstale":"2024-03-28T00:23:57.721317+0000","last_undegraded":"2024-03-28T00:23:57.721317+0000","last_fullsized":"2024-03-28T00:23:57.721317+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T08:17:02.024132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"85'5","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:24:00.485020+0000","last_change":"2024-03-28T00:23:55.728587+0000","last_active":"2024-03-28T00:24:00.485020+0000","last_peered":"2024-03-28T00:24:00.485020+0000","last_clean":"2024-03-28T00:24:00.485020+0000","last_became_active":"2024-03-28T00:23:55.728309+0000","last_became_peered":"2024-03-28T00:23:55.728309+0000","last_unstale":"2024-03-28T00:24:00.485020+0000","last_undegraded":"2024-03-28T00:24:00.485020+0000","last_fullsized":"2024-03-28T00:24:00.485020+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_clean_scrub_stamp":"2024-03-28T00:23:54.693629+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T04:15:12.060649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:23:57.877122+0000","last_change":"2024-03-28T00:23:56.747351+0000","last_active":"2024-03-28T00:23:57.877122+0000","last_peered":"2024-03-28T00:23:57.877122+0000","last_clean":"2024-03-28T00:23:57.877122+0000","last_became_active":"2024-03-28T00:23:56.746995+0000","last_became_peered":"2024-03-28T00:23:56.746995+0000","last_unstale":"2024-03-28T00:23:57.877122+0000","last_undegraded":"2024-03-28T00:23:57.877122+0000","last_fullsized":"2024-03-28T00:23:57.877122+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T00:43:53.521420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"85'7","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:24:02.522127+0000","last_change":"2024-03-28T00:23:55.730157+0000","last_active":"2024-03-28T00:24:02.522127+0000","last_peered":"2024-03-28T00:24:02.522127+0000","last_clean":"2024-03-28T00:24:02.522127+0000","last_became_active":"2024-03-28T00:23:55.729841+0000","last_became_peered":"2024-03-28T00:23:55.729841+0000","last_unstale":"2024-03-28T00:24:02.522127+0000","last_undegraded":"2024-03-28T00:24:02.522127+0000","last_fullsized":"2024-03-28T00:24:02.522127+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_clean_scrub_stamp":"2024-03-28T00:23:54.693629+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T09:57:31.865444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:23:57.721426+0000","last_change":"2024-03-28T00:23:56.760077+0000","last_active":"2024-03-28T00:23:57.721426+0000","last_peered":"2024-03-28T00:23:57.721426+0000","last_clean":"2024-03-28T00:23:57.721426+0000","last_became_active":"2024-03-28T00:23:56.759735+0000","last_became_peered":"2024-03-28T00:23:56.759735+0000","last_unstale":"2024-03-28T00:23:57.721426+0000","last_undegraded":"2024-03-28T00:23:57.721426+0000","last_fullsized":"2024-03-28T00:23:57.721426+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T11:36:17.224248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"85'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:24:01.500378+0000","last_change":"2024-03-28T00:23:55.738216+0000","last_active":"2024-03-28T00:24:01.500378+0000","last_peered":"2024-03-28T00:24:01.500378+0000","last_clean":"2024-03-28T00:24:01.500378+0000","last_became_active":"2024-03-28T00:23:55.737878+0000","last_became_peered":"2024-03-28T00:23:55.737878+0000","last_unstale":"2024-03-28T00:24:01.500378+0000","last_undegraded":"2024-03-28T00:24:01.500378+0000","last_fullsized":"2024-03-28T00:24:01.500378+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_clean_scrub_stamp":"2024-03-28T00:23:54.693629+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T06:30:00.465906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:23:57.883052+0000","last_change":"2024-03-28T00:23:56.760362+0000","last_active":"2024-03-28T00:23:57.883052+0000","last_peered":"2024-03-28T00:23:57.883052+0000","last_clean":"2024-03-28T00:23:57.883052+0000","last_became_active":"2024-03-28T00:23:56.759918+0000","last_became_peered":"2024-03-28T00:23:56.759918+0000","last_unstale":"2024-03-28T00:23:57.883052+0000","last_undegraded":"2024-03-28T00:23:57.883052+0000","last_fullsized":"2024-03-28T00:23:57.883052+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T04:59:01.038539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"85'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:24:00.483263+0000","last_change":"2024-03-28T00:23:55.727557+0000","last_active":"2024-03-28T00:24:00.483263+0000","last_peered":"2024-03-28T00:24:00.483263+0000","last_clean":"2024-03-28T00:24:00.483263+0000","last_became_active":"2024-03-28T00:23:55.727235+0000","last_became_peered":"2024-03-28T00:23:55.727235+0000","last_unstale":"2024-03-28T00:24:00.483263+0000","last_undegraded":"2024-03-28T00:24:00.483263+0000","last_fullsized":"2024-03-28T00:24:00.483263+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_clean_scrub_stamp":"2024-03-28T00:23:54.693629+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T08:14:57.268850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:23:57.881530+0000","last_change":"2024-03-28T00:23:56.766812+0000","last_active":"2024-03-28T00:23:57.881530+0000","last_peered":"2024-03-28T00:23:57.881530+0000","last_clean":"2024-03-28T00:23:57.881530+0000","last_became_active":"2024-03-28T00:23:56.766386+0000","last_became_peered":"2024-03-28T00:23:56.766386+0000","last_unstale":"2024-03-28T00:23:57.881530+0000","last_undegraded":"2024-03-28T00:23:57.881530+0000","last_fullsized":"2024-03-28T00:23:57.881530+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T05:58:07.349158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"85'4","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:24:00.486130+0000","last_change":"2024-03-28T00:23:55.727617+0000","last_active":"2024-03-28T00:24:00.486130+0000","last_peered":"2024-03-28T00:24:00.486130+0000","last_clean":"2024-03-28T00:24:00.486130+0000","last_became_active":"2024-03-28T00:23:55.727276+0000","last_became_peered":"2024-03-28T00:23:55.727276+0000","last_unstale":"2024-03-28T00:24:00.486130+0000","last_undegraded":"2024-03-28T00:24:00.486130+0000","last_fullsized":"2024-03-28T00:24:00.486130+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_clean_scrub_stamp":"2024-03-28T00:23:54.693629+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T02:27:04.376342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:23:57.726354+0000","last_change":"2024-03-28T00:23:56.747261+0000","last_active":"2024-03-28T00:23:57.726354+0000","last_peered":"2024-03-28T00:23:57.726354+0000","last_clean":"2024-03-28T00:23:57.726354+0000","last_became_active":"2024-03-28T00:23:56.746862+0000","last_became_peered":"2024-03-28T00:23:56.746862+0000","last_unstale":"2024-03-28T00:23:57.726354+0000","last_undegraded":"2024-03-28T00:23:57.726354+0000","last_fullsized":"2024-03-28T00:23:57.726354+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T07:33:35.747722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"85'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:24:00.484222+0000","last_change":"2024-03-28T00:23:55.725568+0000","last_active":"2024-03-28T00:24:00.484222+0000","last_peered":"2024-03-28T00:24:00.484222+0000","last_clean":"2024-03-28T00:24:00.484222+0000","last_became_active":"2024-03-28T00:23:55.724967+0000","last_became_peered":"2024-03-28T00:23:55.724967+0000","last_unstale":"2024-03-28T00:24:00.484222+0000","last_undegraded":"2024-03-28T00:24:00.484222+0000","last_fullsized":"2024-03-28T00:24:00.484222+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_clean_scrub_stamp":"2024-03-28T00:23:54.693629+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T10:40:00.731885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:23:57.881704+0000","last_change":"2024-03-28T00:23:56.763176+0000","last_active":"2024-03-28T00:23:57.881704+0000","last_peered":"2024-03-28T00:23:57.881704+0000","last_clean":"2024-03-28T00:23:57.881704+0000","last_became_active":"2024-03-28T00:23:56.762709+0000","last_became_peered":"2024-03-28T00:23:56.762709+0000","last_unstale":"2024-03-28T00:23:57.881704+0000","last_undegraded":"2024-03-28T00:23:57.881704+0000","last_fullsized":"2024-03-28T00:23:57.881704+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T05:58:52.003151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'5","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:24:00.484309+0000","last_change":"2024-03-28T00:23:55.745579+0000","last_active":"2024-03-28T00:24:00.484309+0000","last_peered":"2024-03-28T00:24:00.484309+0000","last_clean":"2024-03-28T00:24:00.484309+0000","last_became_active":"2024-03-28T00:23:55.745262+0000","last_became_peered":"2024-03-28T00:23:55.745262+0000","last_unstale":"2024-03-28T00:24:00.484309+0000","last_undegraded":"2024-03-28T00:24:00.484309+0000","last_fullsized":"2024-03-28T00:24:00.484309+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_clean_scrub_stamp":"2024-03-28T00:23:54.693629+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T00:48:36.212168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:23:57.726092+0000","last_change":"2024-03-28T00:23:56.736393+0000","last_active":"2024-03-28T00:23:57.726092+0000","last_peered":"2024-03-28T00:23:57.726092+0000","last_clean":"2024-03-28T00:23:57.726092+0000","last_became_active":"2024-03-28T00:23:56.736054+0000","last_became_peered":"2024-03-28T00:23:56.736054+0000","last_unstale":"2024-03-28T00:23:57.726092+0000","last_undegraded":"2024-03-28T00:23:57.726092+0000","last_fullsized":"2024-03-28T00:23:57.726092+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T11:56:00.039257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"85'8","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:24:00.483690+0000","last_change":"2024-03-28T00:23:55.722192+0000","last_active":"2024-03-28T00:24:00.483690+0000","last_peered":"2024-03-28T00:24:00.483690+0000","last_clean":"2024-03-28T00:24:00.483690+0000","last_became_active":"2024-03-28T00:23:55.721881+0000","last_became_peered":"2024-03-28T00:23:55.721881+0000","last_unstale":"2024-03-28T00:24:00.483690+0000","last_undegraded":"2024-03-28T00:24:00.483690+0000","last_fullsized":"2024-03-28T00:24:00.483690+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_clean_scrub_stamp":"2024-03-28T00:23:54.693629+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T03:47:34.950264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:23:57.877521+0000","last_change":"2024-03-28T00:23:56.738361+0000","last_active":"2024-03-28T00:23:57.877521+0000","last_peered":"2024-03-28T00:23:57.877521+0000","last_clean":"2024-03-28T00:23:57.877521+0000","last_became_active":"2024-03-28T00:23:56.737040+0000","last_became_peered":"2024-03-28T00:23:56.737040+0000","last_unstale":"2024-03-28T00:23:57.877521+0000","last_undegraded":"2024-03-28T00:23:57.877521+0000","last_fullsized":"2024-03-28T00:23:57.877521+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T11:39:40.653421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'4","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:24:00.487700+0000","last_change":"2024-03-28T00:23:55.733943+0000","last_active":"2024-03-28T00:24:00.487700+0000","last_peered":"2024-03-28T00:24:00.487700+0000","last_clean":"2024-03-28T00:24:00.487700+0000","last_became_active":"2024-03-28T00:23:55.733236+0000","last_became_peered":"2024-03-28T00:23:55.733236+0000","last_unstale":"2024-03-28T00:24:00.487700+0000","last_undegraded":"2024-03-28T00:24:00.487700+0000","last_fullsized":"2024-03-28T00:24:00.487700+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_clean_scrub_stamp":"2024-03-28T00:23:54.693629+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T09:45:33.070408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:23:57.724548+0000","last_change":"2024-03-28T00:23:56.758751+0000","last_active":"2024-03-28T00:23:57.724548+0000","last_peered":"2024-03-28T00:23:57.724548+0000","last_clean":"2024-03-28T00:23:57.724548+0000","last_became_active":"2024-03-28T00:23:56.758066+0000","last_became_peered":"2024-03-28T00:23:56.758066+0000","last_unstale":"2024-03-28T00:23:57.724548+0000","last_undegraded":"2024-03-28T00:23:57.724548+0000","last_fullsized":"2024-03-28T00:23:57.724548+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T02:21:42.514212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"85'4","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:24:00.487908+0000","last_change":"2024-03-28T00:23:55.737167+0000","last_active":"2024-03-28T00:24:00.487908+0000","last_peered":"2024-03-28T00:24:00.487908+0000","last_clean":"2024-03-28T00:24:00.487908+0000","last_became_active":"2024-03-28T00:23:55.736925+0000","last_became_peered":"2024-03-28T00:23:55.736925+0000","last_unstale":"2024-03-28T00:24:00.487908+0000","last_undegraded":"2024-03-28T00:24:00.487908+0000","last_fullsized":"2024-03-28T00:24:00.487908+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_clean_scrub_stamp":"2024-03-28T00:23:54.693629+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T09:13:24.820375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:23:57.877871+0000","last_change":"2024-03-28T00:23:56.745939+0000","last_active":"2024-03-28T00:23:57.877871+0000","last_peered":"2024-03-28T00:23:57.877871+0000","last_clean":"2024-03-28T00:23:57.877871+0000","last_became_active":"2024-03-28T00:23:56.745504+0000","last_became_peered":"2024-03-28T00:23:56.745504+0000","last_unstale":"2024-03-28T00:23:57.877871+0000","last_undegraded":"2024-03-28T00:23:57.877871+0000","last_fullsized":"2024-03-28T00:23:57.877871+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T06:02:34.199029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"85'6","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:24:02.516249+0000","last_change":"2024-03-28T00:23:55.726916+0000","last_active":"2024-03-28T00:24:02.516249+0000","last_peered":"2024-03-28T00:24:02.516249+0000","last_clean":"2024-03-28T00:24:02.516249+0000","last_became_active":"2024-03-28T00:23:55.726570+0000","last_became_peered":"2024-03-28T00:23:55.726570+0000","last_unstale":"2024-03-28T00:24:02.516249+0000","last_undegraded":"2024-03-28T00:24:02.516249+0000","last_fullsized":"2024-03-28T00:24:02.516249+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_clean_scrub_stamp":"2024-03-28T00:23:54.693629+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T06:28:54.581129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:23:57.877115+0000","last_change":"2024-03-28T00:23:56.738303+0000","last_active":"2024-03-28T00:23:57.877115+0000","last_peered":"2024-03-28T00:23:57.877115+0000","last_clean":"2024-03-28T00:23:57.877115+0000","last_became_active":"2024-03-28T00:23:56.737926+0000","last_became_peered":"2024-03-28T00:23:56.737926+0000","last_unstale":"2024-03-28T00:23:57.877115+0000","last_undegraded":"2024-03-28T00:23:57.877115+0000","last_fullsized":"2024-03-28T00:23:57.877115+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T01:56:53.985793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'16","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:24:01.503051+0000","last_change":"2024-03-28T00:23:55.728708+0000","last_active":"2024-03-28T00:24:01.503051+0000","last_peered":"2024-03-28T00:24:01.503051+0000","last_clean":"2024-03-28T00:24:01.503051+0000","last_became_active":"2024-03-28T00:23:55.728415+0000","last_became_peered":"2024-03-28T00:23:55.728415+0000","last_unstale":"2024-03-28T00:24:01.503051+0000","last_undegraded":"2024-03-28T00:24:01.503051+0000","last_fullsized":"2024-03-28T00:24:01.503051+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_clean_scrub_stamp":"2024-03-28T00:23:54.693629+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T05:36:14.405953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:23:57.721157+0000","last_change":"2024-03-28T00:23:56.741982+0000","last_active":"2024-03-28T00:23:57.721157+0000","last_peered":"2024-03-28T00:23:57.721157+0000","last_clean":"2024-03-28T00:23:57.721157+0000","last_became_active":"2024-03-28T00:23:56.741642+0000","last_became_peered":"2024-03-28T00:23:56.741642+0000","last_unstale":"2024-03-28T00:23:57.721157+0000","last_undegraded":"2024-03-28T00:23:57.721157+0000","last_fullsized":"2024-03-28T00:23:57.721157+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T04:27:49.168164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"84'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:23:57.877547+0000","last_change":"2024-03-28T00:23:55.729168+0000","last_active":"2024-03-28T00:23:57.877547+0000","last_peered":"2024-03-28T00:23:57.877547+0000","last_clean":"2024-03-28T00:23:57.877547+0000","last_became_active":"2024-03-28T00:23:55.728686+0000","last_became_peered":"2024-03-28T00:23:55.728686+0000","last_unstale":"2024-03-28T00:23:57.877547+0000","last_undegraded":"2024-03-28T00:23:57.877547+0000","last_fullsized":"2024-03-28T00:23:57.877547+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_clean_scrub_stamp":"2024-03-28T00:23:54.693629+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T12:12:43.807889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:23:57.970140+0000","last_change":"2024-03-28T00:23:56.739241+0000","last_active":"2024-03-28T00:23:57.970140+0000","last_peered":"2024-03-28T00:23:57.970140+0000","last_clean":"2024-03-28T00:23:57.970140+0000","last_became_active":"2024-03-28T00:23:56.738122+0000","last_became_peered":"2024-03-28T00:23:56.738122+0000","last_unstale":"2024-03-28T00:23:57.970140+0000","last_undegraded":"2024-03-28T00:23:57.970140+0000","last_fullsized":"2024-03-28T00:23:57.970140+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T02:52:50.375314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:23:57.881802+0000","last_change":"2024-03-28T00:23:55.745860+0000","last_active":"2024-03-28T00:23:57.881802+0000","last_peered":"2024-03-28T00:23:57.881802+0000","last_clean":"2024-03-28T00:23:57.881802+0000","last_became_active":"2024-03-28T00:23:55.745518+0000","last_became_peered":"2024-03-28T00:23:55.745518+0000","last_unstale":"2024-03-28T00:23:57.881802+0000","last_undegraded":"2024-03-28T00:23:57.881802+0000","last_fullsized":"2024-03-28T00:23:57.881802+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_clean_scrub_stamp":"2024-03-28T00:23:54.693629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T10:48:24.356150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:23:57.878455+0000","last_change":"2024-03-28T00:23:56.752780+0000","last_active":"2024-03-28T00:23:57.878455+0000","last_peered":"2024-03-28T00:23:57.878455+0000","last_clean":"2024-03-28T00:23:57.878455+0000","last_became_active":"2024-03-28T00:23:56.751608+0000","last_became_peered":"2024-03-28T00:23:56.751608+0000","last_unstale":"2024-03-28T00:23:57.878455+0000","last_undegraded":"2024-03-28T00:23:57.878455+0000","last_fullsized":"2024-03-28T00:23:57.878455+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T01:20:45.485846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"85'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:23:57.878084+0000","last_change":"2024-03-28T00:23:55.728123+0000","last_active":"2024-03-28T00:23:57.878084+0000","last_peered":"2024-03-28T00:23:57.878084+0000","last_clean":"2024-03-28T00:23:57.878084+0000","last_became_active":"2024-03-28T00:23:55.727839+0000","last_became_peered":"2024-03-28T00:23:55.727839+0000","last_unstale":"2024-03-28T00:23:57.878084+0000","last_undegraded":"2024-03-28T00:23:57.878084+0000","last_fullsized":"2024-03-28T00:23:57.878084+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_clean_scrub_stamp":"2024-03-28T00:23:54.693629+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T11:39:30.315824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:23:57.878226+0000","last_change":"2024-03-28T00:23:56.752362+0000","last_active":"2024-03-28T00:23:57.878226+0000","last_peered":"2024-03-28T00:23:57.878226+0000","last_clean":"2024-03-28T00:23:57.878226+0000","last_became_active":"2024-03-28T00:23:56.751129+0000","last_became_peered":"2024-03-28T00:23:56.751129+0000","last_unstale":"2024-03-28T00:23:57.878226+0000","last_undegraded":"2024-03-28T00:23:57.878226+0000","last_fullsized":"2024-03-28T00:23:57.878226+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T07:24:15.850437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:23:57.721321+0000","last_change":"2024-03-28T00:23:56.742573+0000","last_active":"2024-03-28T00:23:57.721321+0000","last_peered":"2024-03-28T00:23:57.721321+0000","last_clean":"2024-03-28T00:23:57.721321+0000","last_became_active":"2024-03-28T00:23:56.742236+0000","last_became_peered":"2024-03-28T00:23:56.742236+0000","last_unstale":"2024-03-28T00:23:57.721321+0000","last_undegraded":"2024-03-28T00:23:57.721321+0000","last_fullsized":"2024-03-28T00:23:57.721321+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T11:46:36.357845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"85'4","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:24:01.503895+0000","last_change":"2024-03-28T00:23:55.747124+0000","last_active":"2024-03-28T00:24:01.503895+0000","last_peered":"2024-03-28T00:24:01.503895+0000","last_clean":"2024-03-28T00:24:01.503895+0000","last_became_active":"2024-03-28T00:23:55.745721+0000","last_became_peered":"2024-03-28T00:23:55.745721+0000","last_unstale":"2024-03-28T00:24:01.503895+0000","last_undegraded":"2024-03-28T00:24:01.503895+0000","last_fullsized":"2024-03-28T00:24:01.503895+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_clean_scrub_stamp":"2024-03-28T00:23:54.693629+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T10:37:15.898923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:23:57.881005+0000","last_change":"2024-03-28T00:23:56.757060+0000","last_active":"2024-03-28T00:23:57.881005+0000","last_peered":"2024-03-28T00:23:57.881005+0000","last_clean":"2024-03-28T00:23:57.881005+0000","last_became_active":"2024-03-28T00:23:56.756754+0000","last_became_peered":"2024-03-28T00:23:56.756754+0000","last_unstale":"2024-03-28T00:23:57.881005+0000","last_undegraded":"2024-03-28T00:23:57.881005+0000","last_fullsized":"2024-03-28T00:23:57.881005+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T11:59:32.877471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'6","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:24:02.519052+0000","last_change":"2024-03-28T00:23:55.728039+0000","last_active":"2024-03-28T00:24:02.519052+0000","last_peered":"2024-03-28T00:24:02.519052+0000","last_clean":"2024-03-28T00:24:02.519052+0000","last_became_active":"2024-03-28T00:23:55.727705+0000","last_became_peered":"2024-03-28T00:23:55.727705+0000","last_unstale":"2024-03-28T00:24:02.519052+0000","last_undegraded":"2024-03-28T00:24:02.519052+0000","last_fullsized":"2024-03-28T00:24:02.519052+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_clean_scrub_stamp":"2024-03-28T00:23:54.693629+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T03:48:39.028754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":350,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:23:57.877080+0000","last_change":"2024-03-28T00:22:35.416127+0000","last_active":"2024-03-28T00:23:57.877080+0000","last_peered":"2024-03-28T00:23:57.877080+0000","last_clean":"2024-03-28T00:23:57.877080+0000","last_became_active":"2024-03-28T00:22:35.415710+0000","last_became_peered":"2024-03-28T00:22:35.415710+0000","last_unstale":"2024-03-28T00:23:57.877080+0000","last_undegraded":"2024-03-28T00:23:57.877080+0000","last_fullsized":"2024-03-28T00:23:57.877080+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:18:46.602205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:18:46.602205+0000","last_clean_scrub_stamp":"2024-03-28T00:18:46.602205+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T11:31:44.851848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:23:57.723892+0000","last_change":"2024-03-28T00:23:56.741550+0000","last_active":"2024-03-28T00:23:57.723892+0000","last_peered":"2024-03-28T00:23:57.723892+0000","last_clean":"2024-03-28T00:23:57.723892+0000","last_became_active":"2024-03-28T00:23:56.740750+0000","last_became_peered":"2024-03-28T00:23:56.740750+0000","last_unstale":"2024-03-28T00:23:57.723892+0000","last_undegraded":"2024-03-28T00:23:57.723892+0000","last_fullsized":"2024-03-28T00:23:57.723892+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T09:14:35.259597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:24:01.496602+0000","last_change":"2024-03-28T00:23:55.731623+0000","last_active":"2024-03-28T00:24:01.496602+0000","last_peered":"2024-03-28T00:24:01.496602+0000","last_clean":"2024-03-28T00:24:01.496602+0000","last_became_active":"2024-03-28T00:23:55.731292+0000","last_became_peered":"2024-03-28T00:23:55.731292+0000","last_unstale":"2024-03-28T00:24:01.496602+0000","last_undegraded":"2024-03-28T00:24:01.496602+0000","last_fullsized":"2024-03-28T00:24:01.496602+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_clean_scrub_stamp":"2024-03-28T00:23:54.693629+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T06:16:23.001339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:24:02.520938+0000","last_change":"2024-03-28T00:23:55.745553+0000","last_active":"2024-03-28T00:24:02.520938+0000","last_peered":"2024-03-28T00:24:02.520938+0000","last_clean":"2024-03-28T00:24:02.520938+0000","last_became_active":"2024-03-28T00:23:55.745204+0000","last_became_peered":"2024-03-28T00:23:55.745204+0000","last_unstale":"2024-03-28T00:24:02.520938+0000","last_undegraded":"2024-03-28T00:24:02.520938+0000","last_fullsized":"2024-03-28T00:24:02.520938+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_clean_scrub_stamp":"2024-03-28T00:23:54.693629+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T03:23:50.584362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:23:57.725106+0000","last_change":"2024-03-28T00:23:56.751394+0000","last_active":"2024-03-28T00:23:57.725106+0000","last_peered":"2024-03-28T00:23:57.725106+0000","last_clean":"2024-03-28T00:23:57.725106+0000","last_became_active":"2024-03-28T00:23:56.751025+0000","last_became_peered":"2024-03-28T00:23:56.751025+0000","last_unstale":"2024-03-28T00:23:57.725106+0000","last_undegraded":"2024-03-28T00:23:57.725106+0000","last_fullsized":"2024-03-28T00:23:57.725106+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T03:59:28.738018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:24:01.504330+0000","last_change":"2024-03-28T00:23:55.735055+0000","last_active":"2024-03-28T00:24:01.504330+0000","last_peered":"2024-03-28T00:24:01.504330+0000","last_clean":"2024-03-28T00:24:01.504330+0000","last_became_active":"2024-03-28T00:23:55.734747+0000","last_became_peered":"2024-03-28T00:23:55.734747+0000","last_unstale":"2024-03-28T00:24:01.504330+0000","last_undegraded":"2024-03-28T00:24:01.504330+0000","last_fullsized":"2024-03-28T00:24:01.504330+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_clean_scrub_stamp":"2024-03-28T00:23:54.693629+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T03:02:13.533883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:23:57.883285+0000","last_change":"2024-03-28T00:23:56.760581+0000","last_active":"2024-03-28T00:23:57.883285+0000","last_peered":"2024-03-28T00:23:57.883285+0000","last_clean":"2024-03-28T00:23:57.883285+0000","last_became_active":"2024-03-28T00:23:56.756807+0000","last_became_peered":"2024-03-28T00:23:56.756807+0000","last_unstale":"2024-03-28T00:23:57.883285+0000","last_undegraded":"2024-03-28T00:23:57.883285+0000","last_fullsized":"2024-03-28T00:23:57.883285+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T07:13:34.097671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'7","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:24:03.534835+0000","last_change":"2024-03-28T00:23:55.727864+0000","last_active":"2024-03-28T00:24:03.534835+0000","last_peered":"2024-03-28T00:24:03.534835+0000","last_clean":"2024-03-28T00:24:03.534835+0000","last_became_active":"2024-03-28T00:23:55.727507+0000","last_became_peered":"2024-03-28T00:23:55.727507+0000","last_unstale":"2024-03-28T00:24:03.534835+0000","last_undegraded":"2024-03-28T00:24:03.534835+0000","last_fullsized":"2024-03-28T00:24:03.534835+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_clean_scrub_stamp":"2024-03-28T00:23:54.693629+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T09:01:45.401356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:23:57.880964+0000","last_change":"2024-03-28T00:23:56.766973+0000","last_active":"2024-03-28T00:23:57.880964+0000","last_peered":"2024-03-28T00:23:57.880964+0000","last_clean":"2024-03-28T00:23:57.880964+0000","last_became_active":"2024-03-28T00:23:56.766549+0000","last_became_peered":"2024-03-28T00:23:56.766549+0000","last_unstale":"2024-03-28T00:23:57.880964+0000","last_undegraded":"2024-03-28T00:23:57.880964+0000","last_fullsized":"2024-03-28T00:23:57.880964+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T02:37:52.330879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:24:00.486330+0000","last_change":"2024-03-28T00:23:55.728936+0000","last_active":"2024-03-28T00:24:00.486330+0000","last_peered":"2024-03-28T00:24:00.486330+0000","last_clean":"2024-03-28T00:24:00.486330+0000","last_became_active":"2024-03-28T00:23:55.728517+0000","last_became_peered":"2024-03-28T00:23:55.728517+0000","last_unstale":"2024-03-28T00:24:00.486330+0000","last_undegraded":"2024-03-28T00:24:00.486330+0000","last_fullsized":"2024-03-28T00:24:00.486330+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_clean_scrub_stamp":"2024-03-28T00:23:54.693629+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T10:13:20.841286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:23:57.877099+0000","last_change":"2024-03-28T00:23:56.738438+0000","last_active":"2024-03-28T00:23:57.877099+0000","last_peered":"2024-03-28T00:23:57.877099+0000","last_clean":"2024-03-28T00:23:57.877099+0000","last_became_active":"2024-03-28T00:23:56.738092+0000","last_became_peered":"2024-03-28T00:23:56.738092+0000","last_unstale":"2024-03-28T00:23:57.877099+0000","last_undegraded":"2024-03-28T00:23:57.877099+0000","last_fullsized":"2024-03-28T00:23:57.877099+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T02:54:57.424857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:24:00.487877+0000","last_change":"2024-03-28T00:23:55.741042+0000","last_active":"2024-03-28T00:24:00.487877+0000","last_peered":"2024-03-28T00:24:00.487877+0000","last_clean":"2024-03-28T00:24:00.487877+0000","last_became_active":"2024-03-28T00:23:55.740165+0000","last_became_peered":"2024-03-28T00:23:55.740165+0000","last_unstale":"2024-03-28T00:24:00.487877+0000","last_undegraded":"2024-03-28T00:24:00.487877+0000","last_fullsized":"2024-03-28T00:24:00.487877+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_clean_scrub_stamp":"2024-03-28T00:23:54.693629+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T04:47:58.519326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:23:57.970053+0000","last_change":"2024-03-28T00:23:56.760261+0000","last_active":"2024-03-28T00:23:57.970053+0000","last_peered":"2024-03-28T00:23:57.970053+0000","last_clean":"2024-03-28T00:23:57.970053+0000","last_became_active":"2024-03-28T00:23:56.759925+0000","last_became_peered":"2024-03-28T00:23:56.759925+0000","last_unstale":"2024-03-28T00:23:57.970053+0000","last_undegraded":"2024-03-28T00:23:57.970053+0000","last_fullsized":"2024-03-28T00:23:57.970053+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T00:58:03.546582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:23:57.877653+0000","last_change":"2024-03-28T00:23:55.733455+0000","last_active":"2024-03-28T00:23:57.877653+0000","last_peered":"2024-03-28T00:23:57.877653+0000","last_clean":"2024-03-28T00:23:57.877653+0000","last_became_active":"2024-03-28T00:23:55.733116+0000","last_became_peered":"2024-03-28T00:23:55.733116+0000","last_unstale":"2024-03-28T00:23:57.877653+0000","last_undegraded":"2024-03-28T00:23:57.877653+0000","last_fullsized":"2024-03-28T00:23:57.877653+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_clean_scrub_stamp":"2024-03-28T00:23:54.693629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T07:43:41.301213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:23:57.880176+0000","last_change":"2024-03-28T00:23:56.766884+0000","last_active":"2024-03-28T00:23:57.880176+0000","last_peered":"2024-03-28T00:23:57.880176+0000","last_clean":"2024-03-28T00:23:57.880176+0000","last_became_active":"2024-03-28T00:23:56.766476+0000","last_became_peered":"2024-03-28T00:23:56.766476+0000","last_unstale":"2024-03-28T00:23:57.880176+0000","last_undegraded":"2024-03-28T00:23:57.880176+0000","last_fullsized":"2024-03-28T00:23:57.880176+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T08:01:13.280796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:24:00.484149+0000","last_change":"2024-03-28T00:23:55.728001+0000","last_active":"2024-03-28T00:24:00.484149+0000","last_peered":"2024-03-28T00:24:00.484149+0000","last_clean":"2024-03-28T00:24:00.484149+0000","last_became_active":"2024-03-28T00:23:55.727659+0000","last_became_peered":"2024-03-28T00:23:55.727659+0000","last_unstale":"2024-03-28T00:24:00.484149+0000","last_undegraded":"2024-03-28T00:24:00.484149+0000","last_fullsized":"2024-03-28T00:24:00.484149+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_clean_scrub_stamp":"2024-03-28T00:23:54.693629+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T05:59:09.165928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:23:57.878274+0000","last_change":"2024-03-28T00:23:56.740843+0000","last_active":"2024-03-28T00:23:57.878274+0000","last_peered":"2024-03-28T00:23:57.878274+0000","last_clean":"2024-03-28T00:23:57.878274+0000","last_became_active":"2024-03-28T00:23:56.740364+0000","last_became_peered":"2024-03-28T00:23:56.740364+0000","last_unstale":"2024-03-28T00:23:57.878274+0000","last_undegraded":"2024-03-28T00:23:57.878274+0000","last_fullsized":"2024-03-28T00:23:57.878274+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T10:44:10.040424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:23:57.878041+0000","last_change":"2024-03-28T00:23:55.745772+0000","last_active":"2024-03-28T00:23:57.878041+0000","last_peered":"2024-03-28T00:23:57.878041+0000","last_clean":"2024-03-28T00:23:57.878041+0000","last_became_active":"2024-03-28T00:23:55.745457+0000","last_became_peered":"2024-03-28T00:23:55.745457+0000","last_unstale":"2024-03-28T00:23:57.878041+0000","last_undegraded":"2024-03-28T00:23:57.878041+0000","last_fullsized":"2024-03-28T00:23:57.878041+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_clean_scrub_stamp":"2024-03-28T00:23:54.693629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T01:27:44.632523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:23:57.878100+0000","last_change":"2024-03-28T00:23:56.737008+0000","last_active":"2024-03-28T00:23:57.878100+0000","last_peered":"2024-03-28T00:23:57.878100+0000","last_clean":"2024-03-28T00:23:57.878100+0000","last_became_active":"2024-03-28T00:23:56.736399+0000","last_became_peered":"2024-03-28T00:23:56.736399+0000","last_unstale":"2024-03-28T00:23:57.878100+0000","last_undegraded":"2024-03-28T00:23:57.878100+0000","last_fullsized":"2024-03-28T00:23:57.878100+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T07:16:05.968362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:23:58.374815+0000","last_change":"2024-03-28T00:23:55.734518+0000","last_active":"2024-03-28T00:23:58.374815+0000","last_peered":"2024-03-28T00:23:58.374815+0000","last_clean":"2024-03-28T00:23:58.374815+0000","last_became_active":"2024-03-28T00:23:55.734200+0000","last_became_peered":"2024-03-28T00:23:55.734200+0000","last_unstale":"2024-03-28T00:23:58.374815+0000","last_undegraded":"2024-03-28T00:23:58.374815+0000","last_fullsized":"2024-03-28T00:23:58.374815+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_clean_scrub_stamp":"2024-03-28T00:23:54.693629+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T05:00:56.602654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:23:57.877913+0000","last_change":"2024-03-28T00:23:56.757249+0000","last_active":"2024-03-28T00:23:57.877913+0000","last_peered":"2024-03-28T00:23:57.877913+0000","last_clean":"2024-03-28T00:23:57.877913+0000","last_became_active":"2024-03-28T00:23:56.756957+0000","last_became_peered":"2024-03-28T00:23:56.756957+0000","last_unstale":"2024-03-28T00:23:57.877913+0000","last_undegraded":"2024-03-28T00:23:57.877913+0000","last_fullsized":"2024-03-28T00:23:57.877913+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T03:55:30.166369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:23:57.901757+0000","last_change":"2024-03-28T00:23:55.741463+0000","last_active":"2024-03-28T00:23:57.901757+0000","last_peered":"2024-03-28T00:23:57.901757+0000","last_clean":"2024-03-28T00:23:57.901757+0000","last_became_active":"2024-03-28T00:23:55.741095+0000","last_became_peered":"2024-03-28T00:23:55.741095+0000","last_unstale":"2024-03-28T00:23:57.901757+0000","last_undegraded":"2024-03-28T00:23:57.901757+0000","last_fullsized":"2024-03-28T00:23:57.901757+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_clean_scrub_stamp":"2024-03-28T00:23:54.693629+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T02:58:49.587896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:23:57.877529+0000","last_change":"2024-03-28T00:23:56.736382+0000","last_active":"2024-03-28T00:23:57.877529+0000","last_peered":"2024-03-28T00:23:57.877529+0000","last_clean":"2024-03-28T00:23:57.877529+0000","last_became_active":"2024-03-28T00:23:56.736036+0000","last_became_peered":"2024-03-28T00:23:56.736036+0000","last_unstale":"2024-03-28T00:23:57.877529+0000","last_undegraded":"2024-03-28T00:23:57.877529+0000","last_fullsized":"2024-03-28T00:23:57.877529+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T04:26:01.792967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:23:57.970103+0000","last_change":"2024-03-28T00:23:56.739530+0000","last_active":"2024-03-28T00:23:57.970103+0000","last_peered":"2024-03-28T00:23:57.970103+0000","last_clean":"2024-03-28T00:23:57.970103+0000","last_became_active":"2024-03-28T00:23:56.738166+0000","last_became_peered":"2024-03-28T00:23:56.738166+0000","last_unstale":"2024-03-28T00:23:57.970103+0000","last_undegraded":"2024-03-28T00:23:57.970103+0000","last_fullsized":"2024-03-28T00:23:57.970103+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T01:11:44.872866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:23:57.877499+0000","last_change":"2024-03-28T00:23:56.736044+0000","last_active":"2024-03-28T00:23:57.877499+0000","last_peered":"2024-03-28T00:23:57.877499+0000","last_clean":"2024-03-28T00:23:57.877499+0000","last_became_active":"2024-03-28T00:23:56.735688+0000","last_became_peered":"2024-03-28T00:23:56.735688+0000","last_unstale":"2024-03-28T00:23:57.877499+0000","last_undegraded":"2024-03-28T00:23:57.877499+0000","last_fullsized":"2024-03-28T00:23:57.877499+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T08:57:16.687620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:23:57.877840+0000","last_change":"2024-03-28T00:23:56.760935+0000","last_active":"2024-03-28T00:23:57.877840+0000","last_peered":"2024-03-28T00:23:57.877840+0000","last_clean":"2024-03-28T00:23:57.877840+0000","last_became_active":"2024-03-28T00:23:56.760600+0000","last_became_peered":"2024-03-28T00:23:56.760600+0000","last_unstale":"2024-03-28T00:23:57.877840+0000","last_undegraded":"2024-03-28T00:23:57.877840+0000","last_fullsized":"2024-03-28T00:23:57.877840+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T03:44:30.047881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:23:57.879254+0000","last_change":"2024-03-28T00:23:56.760767+0000","last_active":"2024-03-28T00:23:57.879254+0000","last_peered":"2024-03-28T00:23:57.879254+0000","last_clean":"2024-03-28T00:23:57.879254+0000","last_became_active":"2024-03-28T00:23:56.760431+0000","last_became_peered":"2024-03-28T00:23:56.760431+0000","last_unstale":"2024-03-28T00:23:57.879254+0000","last_undegraded":"2024-03-28T00:23:57.879254+0000","last_fullsized":"2024-03-28T00:23:57.879254+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T01:53:43.479645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:23:57.724996+0000","last_change":"2024-03-28T00:23:56.760772+0000","last_active":"2024-03-28T00:23:57.724996+0000","last_peered":"2024-03-28T00:23:57.724996+0000","last_clean":"2024-03-28T00:23:57.724996+0000","last_became_active":"2024-03-28T00:23:56.760439+0000","last_became_peered":"2024-03-28T00:23:56.760439+0000","last_unstale":"2024-03-28T00:23:57.724996+0000","last_undegraded":"2024-03-28T00:23:57.724996+0000","last_fullsized":"2024-03-28T00:23:57.724996+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T05:46:09.592560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:23:57.970111+0000","last_change":"2024-03-28T00:23:56.740004+0000","last_active":"2024-03-28T00:23:57.970111+0000","last_peered":"2024-03-28T00:23:57.970111+0000","last_clean":"2024-03-28T00:23:57.970111+0000","last_became_active":"2024-03-28T00:23:56.739630+0000","last_became_peered":"2024-03-28T00:23:56.739630+0000","last_unstale":"2024-03-28T00:23:57.970111+0000","last_undegraded":"2024-03-28T00:23:57.970111+0000","last_fullsized":"2024-03-28T00:23:57.970111+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T01:04:39.297383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:23:57.878074+0000","last_change":"2024-03-28T00:23:56.740716+0000","last_active":"2024-03-28T00:23:57.878074+0000","last_peered":"2024-03-28T00:23:57.878074+0000","last_clean":"2024-03-28T00:23:57.878074+0000","last_became_active":"2024-03-28T00:23:56.739587+0000","last_became_peered":"2024-03-28T00:23:56.739587+0000","last_unstale":"2024-03-28T00:23:57.878074+0000","last_undegraded":"2024-03-28T00:23:57.878074+0000","last_fullsized":"2024-03-28T00:23:57.878074+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T11:17:37.115216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:23:57.883180+0000","last_change":"2024-03-28T00:23:56.761001+0000","last_active":"2024-03-28T00:23:57.883180+0000","last_peered":"2024-03-28T00:23:57.883180+0000","last_clean":"2024-03-28T00:23:57.883180+0000","last_became_active":"2024-03-28T00:23:56.757373+0000","last_became_peered":"2024-03-28T00:23:56.757373+0000","last_unstale":"2024-03-28T00:23:57.883180+0000","last_undegraded":"2024-03-28T00:23:57.883180+0000","last_fullsized":"2024-03-28T00:23:57.883180+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T02:55:47.106100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:23:57.721043+0000","last_change":"2024-03-28T00:23:56.750715+0000","last_active":"2024-03-28T00:23:57.721043+0000","last_peered":"2024-03-28T00:23:57.721043+0000","last_clean":"2024-03-28T00:23:57.721043+0000","last_became_active":"2024-03-28T00:23:56.750446+0000","last_became_peered":"2024-03-28T00:23:56.750446+0000","last_unstale":"2024-03-28T00:23:57.721043+0000","last_undegraded":"2024-03-28T00:23:57.721043+0000","last_fullsized":"2024-03-28T00:23:57.721043+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T11:51:27.935181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:23:57.880443+0000","last_change":"2024-03-28T00:23:56.752571+0000","last_active":"2024-03-28T00:23:57.880443+0000","last_peered":"2024-03-28T00:23:57.880443+0000","last_clean":"2024-03-28T00:23:57.880443+0000","last_became_active":"2024-03-28T00:23:56.751853+0000","last_became_peered":"2024-03-28T00:23:56.751853+0000","last_unstale":"2024-03-28T00:23:57.880443+0000","last_undegraded":"2024-03-28T00:23:57.880443+0000","last_fullsized":"2024-03-28T00:23:57.880443+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T05:30:02.897139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:23:57.726163+0000","last_change":"2024-03-28T00:23:56.750006+0000","last_active":"2024-03-28T00:23:57.726163+0000","last_peered":"2024-03-28T00:23:57.726163+0000","last_clean":"2024-03-28T00:23:57.726163+0000","last_became_active":"2024-03-28T00:23:56.749639+0000","last_became_peered":"2024-03-28T00:23:56.749639+0000","last_unstale":"2024-03-28T00:23:57.726163+0000","last_undegraded":"2024-03-28T00:23:57.726163+0000","last_fullsized":"2024-03-28T00:23:57.726163+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T02:26:51.076107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:23:57.724872+0000","last_change":"2024-03-28T00:23:56.764982+0000","last_active":"2024-03-28T00:23:57.724872+0000","last_peered":"2024-03-28T00:23:57.724872+0000","last_clean":"2024-03-28T00:23:57.724872+0000","last_became_active":"2024-03-28T00:23:56.764596+0000","last_became_peered":"2024-03-28T00:23:56.764596+0000","last_unstale":"2024-03-28T00:23:57.724872+0000","last_undegraded":"2024-03-28T00:23:57.724872+0000","last_fullsized":"2024-03-28T00:23:57.724872+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T09:05:30.948689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-28T00:23:57.970015+0000","last_change":"2024-03-28T00:23:56.736439+0000","last_active":"2024-03-28T00:23:57.970015+0000","last_peered":"2024-03-28T00:23:57.970015+0000","last_clean":"2024-03-28T00:23:57.970015+0000","last_became_active":"2024-03-28T00:23:56.736099+0000","last_became_peered":"2024-03-28T00:23:56.736099+0000","last_unstale":"2024-03-28T00:23:57.970015+0000","last_undegraded":"2024-03-28T00:23:57.970015+0000","last_fullsized":"2024-03-28T00:23:57.970015+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T02:57:42.821566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":4132576,"data_compressed":245983,"data_compressed_allocated":315392,"data_compressed_original":4079616,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":5849,"num_objects":69,"num_object_clones":0,"num_object_copies":207,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":146,"num_write_kb":49,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":45,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":278528,"data_stored":17916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":127,"ondisk_log_size":127,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383696,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32080,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713072,"statfs":{"total":95995035648,"available":95962185728,"internally_reserved":0,"allocated":925696,"data_stored":440051,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645332,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1015808,"data_stored":1032150,"data_compressed":35845,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808857,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32084,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713068,"statfs":{"total":95995035648,"available":95962181632,"internally_reserved":0,"allocated":929792,"data_stored":439252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070497,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32088,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713064,"statfs":{"total":95995035648,"available":95962177536,"internally_reserved":0,"allocated":933888,"data_stored":439206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332132,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":978944,"data_stored":1030628,"data_compressed":35845,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495659,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32072,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713080,"statfs":{"total":95995035648,"available":95962193920,"internally_reserved":0,"allocated":917504,"data_stored":438558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574919,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32080,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713072,"statfs":{"total":95995035648,"available":95962185728,"internally_reserved":0,"allocated":925696,"data_stored":440080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738444,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":942080,"data_stored":439946,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411394,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":937984,"data_stored":440736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182458,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":929792,"data_stored":438752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920823,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32084,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713068,"statfs":{"total":95995035648,"available":95962181632,"internally_reserved":0,"allocated":929792,"data_stored":439134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757296,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":1003520,"data_stored":1031074,"data_compressed":35845,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34201,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34201,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34201,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35845,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35845,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35845,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35845,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":3314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-28T00:24:07.498 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-28T00:24:07.499 INFO:tasks.cephadm:Setting up client nodes... 2024-03-28T00:24:07.499 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 10f73b92-ec98-11ee-95d0-87774f69a715 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-03-28T00:24:07.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:24:07 smithi165 ceph-mon[36302]: from='client.24707 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-28T00:24:07.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:24:07 smithi119 ceph-mon[31518]: from='client.24707 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-28T00:24:07.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:24:07 smithi132 ceph-mon[35941]: from='client.24707 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-28T00:24:07.972 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/mon.a/config 2024-03-28T00:24:08.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:24:08 smithi165 ceph-mon[36302]: from='client.24713 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-28T00:24:08.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:24:08 smithi165 ceph-mon[36302]: pgmap v319: 97 pgs: 97 active+clean; 585 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 6.5 KiB/s wr, 20 op/s 2024-03-28T00:24:08.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:24:08 smithi119 ceph-mon[31518]: from='client.24713 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-28T00:24:08.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:24:08 smithi119 ceph-mon[31518]: pgmap v319: 97 pgs: 97 active+clean; 585 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 6.5 KiB/s wr, 20 op/s 2024-03-28T00:24:08.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:24:08 smithi132 ceph-mon[35941]: from='client.24713 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-28T00:24:08.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:24:08 smithi132 ceph-mon[35941]: pgmap v319: 97 pgs: 97 active+clean; 585 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 6.5 KiB/s wr, 20 op/s 2024-03-28T00:24:10.326 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:10.319+0000 7f2760c00700 1 -- 172.21.15.119:0/1961471032 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f275c107d10 msgr2=0x7f275c10a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:24:10.326 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:10.319+0000 7f2760c00700 1 --2- 172.21.15.119:0/1961471032 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f275c107d10 0x7f275c10a180 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f275400b0a0 tx=0x7f275400b3b0 comp rx=0 tx=0).stop 2024-03-28T00:24:10.326 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:10.319+0000 7f2760c00700 1 -- 172.21.15.119:0/1961471032 shutdown_connections 2024-03-28T00:24:10.326 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:10.319+0000 7f2760c00700 1 --2- 172.21.15.119:0/1961471032 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f275c107d10 0x7f275c10a180 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:10.326 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:10.319+0000 7f2760c00700 1 --2- 172.21.15.119:0/1961471032 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f275c102ab0 0x7f275c104ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:10.326 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:10.319+0000 7f2760c00700 1 --2- 172.21.15.119:0/1961471032 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f275c1053e0 0x7f275c1077d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:10.326 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:10.319+0000 7f2760c00700 1 -- 172.21.15.119:0/1961471032 >> 172.21.15.119:0/1961471032 conn(0x7f275c0fc4c0 msgr2=0x7f275c0fe900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:24:10.326 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:10.319+0000 7f2760c00700 1 -- 172.21.15.119:0/1961471032 shutdown_connections 2024-03-28T00:24:10.326 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:10.319+0000 7f2760c00700 1 -- 172.21.15.119:0/1961471032 wait complete. 2024-03-28T00:24:10.327 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:10.320+0000 7f2760c00700 1 Processor -- start 2024-03-28T00:24:10.327 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:10.321+0000 7f2760c00700 1 -- start start 2024-03-28T00:24:10.327 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:10.321+0000 7f2760c00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f275c102ab0 0x7f275c19a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:24:10.328 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:10.321+0000 7f2760c00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f275c1053e0 0x7f275c19a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:24:10.328 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:10.321+0000 7f2760c00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f275c107d10 0x7f275c1a18d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:24:10.328 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:10.321+0000 7f2760c00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f275c10dc80 con 0x7f275c102ab0 2024-03-28T00:24:10.328 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:10.321+0000 7f2760c00700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f275c10db00 con 0x7f275c1053e0 2024-03-28T00:24:10.328 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:10.321+0000 7f2760c00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f275c10de00 con 0x7f275c107d10 2024-03-28T00:24:10.328 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:10.321+0000 7f275b400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f275c102ab0 0x7f275c19a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:24:10.328 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:10.321+0000 7f275aa00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f275c1053e0 0x7f275c19a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:24:10.328 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:10.321+0000 7f275be00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f275c107d10 0x7f275c1a18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:24:10.328 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:10.321+0000 7f275aa00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f275c1053e0 0x7f275c19a850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.119:54082/0 (socket says 172.21.15.119:54082) 2024-03-28T00:24:10.328 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:10.321+0000 7f275aa00700 1 -- 172.21.15.119:0/1238995842 learned_addr learned my addr 172.21.15.119:0/1238995842 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:24:10.329 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:10.322+0000 7f275be00700 1 -- 172.21.15.119:0/1238995842 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f275c1053e0 msgr2=0x7f275c19a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:24:10.329 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:10.322+0000 7f275be00700 1 --2- 172.21.15.119:0/1238995842 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f275c1053e0 0x7f275c19a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:10.329 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:10.322+0000 7f275be00700 1 -- 172.21.15.119:0/1238995842 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f275c102ab0 msgr2=0x7f275c19a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:24:10.329 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:10.322+0000 7f275be00700 1 --2- 172.21.15.119:0/1238995842 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f275c102ab0 0x7f275c19a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:10.329 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:10.322+0000 7f275be00700 1 -- 172.21.15.119:0/1238995842 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f275c1a1f70 con 0x7f275c107d10 2024-03-28T00:24:10.329 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:10.322+0000 7f275be00700 1 --2- 172.21.15.119:0/1238995842 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f275c107d10 0x7f275c1a18d0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f2754007fa0 tx=0x7f2754004c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:24:10.330 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:10.322+0000 7f2753e00700 1 -- 172.21.15.119:0/1238995842 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2754021070 con 0x7f275c107d10 2024-03-28T00:24:10.330 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:10.322+0000 7f2753e00700 1 -- 172.21.15.119:0/1238995842 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2754007b80 con 0x7f275c107d10 2024-03-28T00:24:10.330 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:10.322+0000 7f2753e00700 1 -- 172.21.15.119:0/1238995842 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2754004750 con 0x7f275c107d10 2024-03-28T00:24:10.330 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:10.322+0000 7f2760c00700 1 -- 172.21.15.119:0/1238995842 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f275c1a2200 con 0x7f275c107d10 2024-03-28T00:24:10.330 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:10.322+0000 7f2760c00700 1 -- 172.21.15.119:0/1238995842 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f275c1a2660 con 0x7f275c107d10 2024-03-28T00:24:10.330 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:10.324+0000 7f2760c00700 1 -- 172.21.15.119:0/1238995842 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f275c0637c0 con 0x7f275c107d10 2024-03-28T00:24:10.331 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:10.325+0000 7f2753e00700 1 -- 172.21.15.119:0/1238995842 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 17) v1 ==== 137342+0+0 (secure 0 0 0) 0x7f2754003a00 con 0x7f275c107d10 2024-03-28T00:24:10.332 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:10.326+0000 7f2753e00700 1 --2- 172.21.15.119:0/1238995842 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f27400a2a50 0x7f27400a4f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:24:10.333 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:10.327+0000 7f2753e00700 1 -- 172.21.15.119:0/1238995842 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f275400f020 con 0x7f275c107d10 2024-03-28T00:24:10.333 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:10.327+0000 7f275b400700 1 --2- 172.21.15.119:0/1238995842 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f27400a2a50 0x7f27400a4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:24:10.333 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:10.327+0000 7f275b400700 1 --2- 172.21.15.119:0/1238995842 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f27400a2a50 0x7f27400a4f10 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f274c009c60 tx=0x7f274c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:24:10.338 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:10.333+0000 7f2753e00700 1 -- 172.21.15.119:0/1238995842 <== mon.1 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f275402a020 con 0x7f275c107d10 2024-03-28T00:24:10.644 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:10.637+0000 7f2760c00700 1 -- 172.21.15.119:0/1238995842 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f275c062da0 con 0x7f275c107d10 2024-03-28T00:24:10.652 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:10.646+0000 7f2753e00700 1 -- 172.21.15.119:0/1238995842 <== mon.1 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) v1 ==== 170+0+59 (secure 0 0 0) 0x7f275409cfb0 con 0x7f275c107d10 2024-03-28T00:24:10.652 INFO:teuthology.orchestra.run.smithi119.stdout:[client.0] 2024-03-28T00:24:10.652 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQCquARmrbEqJhAABEHBy3mxpo+CWcXU+kB6KA== 2024-03-28T00:24:10.655 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:10.649+0000 7f2760c00700 1 -- 172.21.15.119:0/1238995842 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f27400a2a50 msgr2=0x7f27400a4f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:24:10.655 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:10.649+0000 7f2760c00700 1 --2- 172.21.15.119:0/1238995842 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f27400a2a50 0x7f27400a4f10 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f274c009c60 tx=0x7f274c009260 comp rx=0 tx=0).stop 2024-03-28T00:24:10.655 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:10.649+0000 7f2760c00700 1 -- 172.21.15.119:0/1238995842 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f275c107d10 msgr2=0x7f275c1a18d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:24:10.656 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:10.649+0000 7f2760c00700 1 --2- 172.21.15.119:0/1238995842 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f275c107d10 0x7f275c1a18d0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f2754007fa0 tx=0x7f2754004c80 comp rx=0 tx=0).stop 2024-03-28T00:24:10.656 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:10.649+0000 7f2760c00700 1 -- 172.21.15.119:0/1238995842 shutdown_connections 2024-03-28T00:24:10.656 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:10.649+0000 7f2760c00700 1 --2- 172.21.15.119:0/1238995842 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f27400a2a50 0x7f27400a4f10 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:10.656 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:10.649+0000 7f2760c00700 1 --2- 172.21.15.119:0/1238995842 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f275c107d10 0x7f275c1a18d0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:10.656 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:10.649+0000 7f2760c00700 1 --2- 172.21.15.119:0/1238995842 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f275c102ab0 0x7f275c19a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:10.656 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:10.649+0000 7f2760c00700 1 --2- 172.21.15.119:0/1238995842 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f275c1053e0 0x7f275c19a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:10.656 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:10.649+0000 7f2760c00700 1 -- 172.21.15.119:0/1238995842 >> 172.21.15.119:0/1238995842 conn(0x7f275c0fc4c0 msgr2=0x7f275c101fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:24:10.656 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:10.649+0000 7f2760c00700 1 -- 172.21.15.119:0/1238995842 shutdown_connections 2024-03-28T00:24:10.656 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:10.649+0000 7f2760c00700 1 -- 172.21.15.119:0/1238995842 wait complete. 2024-03-28T00:24:10.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:24:10 smithi119 ceph-mon[31518]: pgmap v320: 97 pgs: 97 active+clean; 636 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 31 op/s; 2 B/s, 0 objects/s recovering 2024-03-28T00:24:10.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:24:10 smithi119 ceph-mon[31518]: from='client.? 172.21.15.119:0/1238995842' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-28T00:24:10.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:24:10 smithi119 ceph-mon[31518]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-28T00:24:10.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:24:10 smithi119 ceph-mon[31518]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-28T00:24:11.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:24:10 smithi165 ceph-mon[36302]: pgmap v320: 97 pgs: 97 active+clean; 636 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 31 op/s; 2 B/s, 0 objects/s recovering 2024-03-28T00:24:11.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:24:10 smithi165 ceph-mon[36302]: from='client.? 172.21.15.119:0/1238995842' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-28T00:24:11.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:24:10 smithi165 ceph-mon[36302]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-28T00:24:11.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:24:10 smithi165 ceph-mon[36302]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-28T00:24:11.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:24:10 smithi132 ceph-mon[35941]: pgmap v320: 97 pgs: 97 active+clean; 636 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 31 op/s; 2 B/s, 0 objects/s recovering 2024-03-28T00:24:11.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:24:10 smithi132 ceph-mon[35941]: from='client.? 172.21.15.119:0/1238995842' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-28T00:24:11.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:24:10 smithi132 ceph-mon[35941]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-28T00:24:11.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:24:10 smithi132 ceph-mon[35941]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-28T00:24:11.515 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-03-28T00:24:11.515 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-03-28T00:24:11.516 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-03-28T00:24:11.671 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-03-28T00:24:11.671 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-03-28T00:24:11.672 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 10f73b92-ec98-11ee-95d0-87774f69a715 -- ceph mgr dump --format=json 2024-03-28T00:24:12.174 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/mon.a/config 2024-03-28T00:24:13.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:24:12 smithi165 ceph-mon[36302]: pgmap v321: 97 pgs: 97 active+clean; 636 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 22 op/s; 2 B/s, 0 objects/s recovering 2024-03-28T00:24:13.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:24:12 smithi119 ceph-mon[31518]: pgmap v321: 97 pgs: 97 active+clean; 636 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 22 op/s; 2 B/s, 0 objects/s recovering 2024-03-28T00:24:13.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:24:12 smithi132 ceph-mon[35941]: pgmap v321: 97 pgs: 97 active+clean; 636 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 22 op/s; 2 B/s, 0 objects/s recovering 2024-03-28T00:24:14.525 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:14.518+0000 7f1858e00700 1 -- 172.21.15.119:0/1515421759 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1854107510 msgr2=0x7f185410f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:24:14.525 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:14.518+0000 7f1858e00700 1 --2- 172.21.15.119:0/1515421759 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1854107510 0x7f185410f0e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f184c00b0d0 tx=0x7f184c00b3e0 comp rx=0 tx=0).stop 2024-03-28T00:24:14.525 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:14.518+0000 7f1858e00700 1 -- 172.21.15.119:0/1515421759 shutdown_connections 2024-03-28T00:24:14.525 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:14.518+0000 7f1858e00700 1 --2- 172.21.15.119:0/1515421759 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1854107510 0x7f185410f0e0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:14.525 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:14.518+0000 7f1858e00700 1 --2- 172.21.15.119:0/1515421759 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1854102440 0x7f1854102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:14.525 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:14.518+0000 7f1858e00700 1 --2- 172.21.15.119:0/1515421759 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1854102df0 0x7f1854106e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:14.525 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:14.518+0000 7f1858e00700 1 -- 172.21.15.119:0/1515421759 >> 172.21.15.119:0/1515421759 conn(0x7f18540792f0 msgr2=0x7f1854079700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:24:14.525 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:14.518+0000 7f1858e00700 1 -- 172.21.15.119:0/1515421759 shutdown_connections 2024-03-28T00:24:14.525 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:14.518+0000 7f1858e00700 1 -- 172.21.15.119:0/1515421759 wait complete. 2024-03-28T00:24:14.527 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:14.519+0000 7f1858e00700 1 Processor -- start 2024-03-28T00:24:14.527 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:14.519+0000 7f1858e00700 1 -- start start 2024-03-28T00:24:14.527 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:14.520+0000 7f1858e00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1854102440 0x7f185419e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:24:14.528 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:14.520+0000 7f1858e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1854102df0 0x7f185419eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:24:14.528 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:14.520+0000 7f1858e00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1854107510 0x7f18541987b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:24:14.528 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:14.520+0000 7f1858e00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1854112510 con 0x7f1854102df0 2024-03-28T00:24:14.528 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:14.520+0000 7f1858e00700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1854112390 con 0x7f1854107510 2024-03-28T00:24:14.528 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:14.520+0000 7f1858e00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1854112690 con 0x7f1854102440 2024-03-28T00:24:14.528 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:14.520+0000 7f1851e00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1854102440 0x7f185419e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:24:14.528 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:14.520+0000 7f1851e00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1854102440 0x7f185419e630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.119:46340/0 (socket says 172.21.15.119:46340) 2024-03-28T00:24:14.528 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:14.520+0000 7f1851e00700 1 -- 172.21.15.119:0/2536869123 learned_addr learned my addr 172.21.15.119:0/2536869123 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:24:14.528 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:14.520+0000 7f184be00700 1 --2- 172.21.15.119:0/2536869123 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1854102df0 0x7f185419eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:24:14.528 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:14.521+0000 7f1851e00700 1 -- 172.21.15.119:0/2536869123 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1854107510 msgr2=0x7f18541987b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:24:14.528 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:14.521+0000 7f1852800700 1 --2- 172.21.15.119:0/2536869123 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1854107510 0x7f18541987b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:24:14.528 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:14.521+0000 7f1851e00700 1 --2- 172.21.15.119:0/2536869123 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1854107510 0x7f18541987b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:14.528 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:14.521+0000 7f1851e00700 1 -- 172.21.15.119:0/2536869123 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1854102df0 msgr2=0x7f185419eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:24:14.529 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:14.521+0000 7f1851e00700 1 --2- 172.21.15.119:0/2536869123 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1854102df0 0x7f185419eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:14.529 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:14.521+0000 7f1851e00700 1 -- 172.21.15.119:0/2536869123 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1854198f90 con 0x7f1854102440 2024-03-28T00:24:14.529 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:14.521+0000 7f1852800700 1 --2- 172.21.15.119:0/2536869123 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1854107510 0x7f18541987b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T00:24:14.529 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:14.521+0000 7f1851e00700 1 --2- 172.21.15.119:0/2536869123 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1854102440 0x7f185419e630 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f184400e9a0 tx=0x7f184400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:24:14.529 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:14.521+0000 7f184aa00700 1 -- 172.21.15.119:0/2536869123 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f184400cbd0 con 0x7f1854102440 2024-03-28T00:24:14.529 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:14.521+0000 7f184aa00700 1 -- 172.21.15.119:0/2536869123 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f184400cd70 con 0x7f1854102440 2024-03-28T00:24:14.529 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:14.521+0000 7f1858e00700 1 -- 172.21.15.119:0/2536869123 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1854199280 con 0x7f1854102440 2024-03-28T00:24:14.529 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:14.521+0000 7f184aa00700 1 -- 172.21.15.119:0/2536869123 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1844018750 con 0x7f1854102440 2024-03-28T00:24:14.529 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:14.521+0000 7f1858e00700 1 -- 172.21.15.119:0/2536869123 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18541a5490 con 0x7f1854102440 2024-03-28T00:24:14.529 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:14.522+0000 7f1858e00700 1 -- 172.21.15.119:0/2536869123 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1854103910 con 0x7f1854102440 2024-03-28T00:24:14.530 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:14.524+0000 7f184aa00700 1 -- 172.21.15.119:0/2536869123 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 17) v1 ==== 137342+0+0 (secure 0 0 0) 0x7f1844010430 con 0x7f1854102440 2024-03-28T00:24:14.531 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:14.525+0000 7f184aa00700 1 --2- 172.21.15.119:0/2536869123 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f18380a2920 0x7f18380a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:24:14.531 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:14.526+0000 7f184aa00700 1 -- 172.21.15.119:0/2536869123 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f18440ceeb0 con 0x7f1854102440 2024-03-28T00:24:14.532 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:14.526+0000 7f184be00700 1 --2- 172.21.15.119:0/2536869123 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f18380a2920 0x7f18380a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:24:14.532 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:14.526+0000 7f184be00700 1 --2- 172.21.15.119:0/2536869123 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f18380a2920 0x7f18380a4de0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f183c009680 tx=0x7f183c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:24:14.540 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:14.534+0000 7f184aa00700 1 -- 172.21.15.119:0/2536869123 <== mon.1 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f184409bdf0 con 0x7f1854102440 2024-03-28T00:24:14.821 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:14.814+0000 7f1858e00700 1 -- 172.21.15.119:0/2536869123 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f185402da30 con 0x7f1854102440 2024-03-28T00:24:14.828 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:14.822+0000 7f184aa00700 1 -- 172.21.15.119:0/2536869123 <== mon.1 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) v1 ==== 74+0+262666 (secure 0 0 0) 0x7f184409b6f0 con 0x7f1854102440 2024-03-28T00:24:14.835 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-28T00:24:14.836 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:14.828+0000 7f1858e00700 1 -- 172.21.15.119:0/2536869123 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f18380a2920 msgr2=0x7f18380a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:24:14.837 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:14.828+0000 7f1858e00700 1 --2- 172.21.15.119:0/2536869123 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f18380a2920 0x7f18380a4de0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f183c009680 tx=0x7f183c009260 comp rx=0 tx=0).stop 2024-03-28T00:24:14.837 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:14.828+0000 7f1858e00700 1 -- 172.21.15.119:0/2536869123 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1854102440 msgr2=0x7f185419e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:24:14.837 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:14.828+0000 7f1858e00700 1 --2- 172.21.15.119:0/2536869123 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1854102440 0x7f185419e630 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f184400e9a0 tx=0x7f184400ecb0 comp rx=0 tx=0).stop 2024-03-28T00:24:14.837 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:14.828+0000 7f1858e00700 1 -- 172.21.15.119:0/2536869123 shutdown_connections 2024-03-28T00:24:14.837 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:14.828+0000 7f1858e00700 1 --2- 172.21.15.119:0/2536869123 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f18380a2920 0x7f18380a4de0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:14.837 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:14.828+0000 7f1858e00700 1 --2- 172.21.15.119:0/2536869123 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1854107510 0x7f18541987b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:14.837 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:14.828+0000 7f1858e00700 1 --2- 172.21.15.119:0/2536869123 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1854102440 0x7f185419e630 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:14.837 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:14.828+0000 7f1858e00700 1 --2- 172.21.15.119:0/2536869123 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1854102df0 0x7f185419eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:14.837 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:14.828+0000 7f1858e00700 1 -- 172.21.15.119:0/2536869123 >> 172.21.15.119:0/2536869123 conn(0x7f18540792f0 msgr2=0x7f1854100190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:24:14.837 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:14.828+0000 7f1858e00700 1 -- 172.21.15.119:0/2536869123 shutdown_connections 2024-03-28T00:24:14.837 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:14.829+0000 7f1858e00700 1 -- 172.21.15.119:0/2536869123 wait complete. 2024-03-28T00:24:15.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:24:14 smithi165 ceph-mon[36302]: pgmap v322: 97 pgs: 97 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.0 KiB/s wr, 22 op/s; 2 B/s, 0 objects/s recovering 2024-03-28T00:24:15.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:24:14 smithi119 ceph-mon[31518]: pgmap v322: 97 pgs: 97 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.0 KiB/s wr, 22 op/s; 2 B/s, 0 objects/s recovering 2024-03-28T00:24:15.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:24:14 smithi132 ceph-mon[35941]: pgmap v322: 97 pgs: 97 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.0 KiB/s wr, 22 op/s; 2 B/s, 0 objects/s recovering 2024-03-28T00:24:15.897 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":1401937096},{"type":"v1","addr":"172.21.15.119:6801","nonce":1401937096}]},"active_addr":"172.21.15.119:6801/1401937096","active_change":"2024-03-28T00:16:03.582759+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24110,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.119:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.119:0","nonce":524303043}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.119:0","nonce":357445067}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.119:0","nonce":2430401731}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.119:0","nonce":4119730435}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.119:0","nonce":88094031}]}]} 2024-03-28T00:24:15.904 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-03-28T00:24:15.904 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-03-28T00:24:15.904 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 10f73b92-ec98-11ee-95d0-87774f69a715 -- ceph osd dump --format=json 2024-03-28T00:24:16.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:24:15 smithi165 ceph-mon[36302]: from='client.? 172.21.15.119:0/2536869123' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-28T00:24:16.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:24:15 smithi119 ceph-mon[31518]: from='client.? 172.21.15.119:0/2536869123' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-28T00:24:16.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:24:15 smithi132 ceph-mon[35941]: from='client.? 172.21.15.119:0/2536869123' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-28T00:24:16.359 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/mon.a/config 2024-03-28T00:24:17.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:24:16 smithi165 ceph-mon[36302]: pgmap v323: 97 pgs: 97 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.1 KiB/s wr, 20 op/s; 2 B/s, 0 objects/s recovering 2024-03-28T00:24:17.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:24:16 smithi119 ceph-mon[31518]: pgmap v323: 97 pgs: 97 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.1 KiB/s wr, 20 op/s; 2 B/s, 0 objects/s recovering 2024-03-28T00:24:17.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:24:16 smithi132 ceph-mon[35941]: pgmap v323: 97 pgs: 97 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.1 KiB/s wr, 20 op/s; 2 B/s, 0 objects/s recovering 2024-03-28T00:24:18.647 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:18.640+0000 7fa492400700 1 -- 172.21.15.119:0/33030138 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa48c0743c0 msgr2=0x7fa48c0747a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:24:18.647 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:18.640+0000 7fa492400700 1 --2- 172.21.15.119:0/33030138 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa48c0743c0 0x7fa48c0747a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa47c0098f0 tx=0x7fa47c009c00 comp rx=0 tx=0).stop 2024-03-28T00:24:18.647 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:18.640+0000 7fa492400700 1 -- 172.21.15.119:0/33030138 shutdown_connections 2024-03-28T00:24:18.647 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:18.640+0000 7fa492400700 1 --2- 172.21.15.119:0/33030138 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa48c10eb90 0x7fa48c110f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:18.647 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:18.640+0000 7fa492400700 1 --2- 172.21.15.119:0/33030138 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa48c0743c0 0x7fa48c0747a0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:18.647 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:18.640+0000 7fa492400700 1 --2- 172.21.15.119:0/33030138 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa48c074ce0 0x7fa48c10e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:18.647 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:18.640+0000 7fa492400700 1 -- 172.21.15.119:0/33030138 >> 172.21.15.119:0/33030138 conn(0x7fa48c0792f0 msgr2=0x7fa48c079700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:24:18.647 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:18.640+0000 7fa492400700 1 -- 172.21.15.119:0/33030138 shutdown_connections 2024-03-28T00:24:18.647 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:18.640+0000 7fa492400700 1 -- 172.21.15.119:0/33030138 wait complete. 2024-03-28T00:24:18.649 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:18.642+0000 7fa492400700 1 Processor -- start 2024-03-28T00:24:18.649 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:18.642+0000 7fa492400700 1 -- start start 2024-03-28T00:24:18.649 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:18.642+0000 7fa492400700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa48c0743c0 0x7fa48c1a08e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:24:18.649 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:18.642+0000 7fa492400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa48c074ce0 0x7fa48c1a0e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:24:18.649 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:18.642+0000 7fa492400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa48c10eb90 0x7fa48c19aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:24:18.649 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:18.642+0000 7fa492400700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa48c114780 con 0x7fa48c10eb90 2024-03-28T00:24:18.649 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:18.642+0000 7fa492400700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa48c114600 con 0x7fa48c074ce0 2024-03-28T00:24:18.649 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:18.642+0000 7fa492400700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa48c114900 con 0x7fa48c0743c0 2024-03-28T00:24:18.649 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:18.642+0000 7fa48b400700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa48c0743c0 0x7fa48c1a08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:24:18.650 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:18.642+0000 7fa48b400700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa48c0743c0 0x7fa48c1a08e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.119:46356/0 (socket says 172.21.15.119:46356) 2024-03-28T00:24:18.650 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:18.642+0000 7fa482a00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa48c074ce0 0x7fa48c1a0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:24:18.650 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:18.642+0000 7fa48b400700 1 -- 172.21.15.119:0/2668822297 learned_addr learned my addr 172.21.15.119:0/2668822297 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:24:18.650 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:18.643+0000 7fa48be00700 1 --2- 172.21.15.119:0/2668822297 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa48c10eb90 0x7fa48c19aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:24:18.650 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:18.643+0000 7fa48b400700 1 -- 172.21.15.119:0/2668822297 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa48c074ce0 msgr2=0x7fa48c1a0e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:24:18.650 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:18.643+0000 7fa48b400700 1 --2- 172.21.15.119:0/2668822297 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa48c074ce0 0x7fa48c1a0e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:18.650 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:18.643+0000 7fa48b400700 1 -- 172.21.15.119:0/2668822297 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa48c10eb90 msgr2=0x7fa48c19aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:24:18.650 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:18.643+0000 7fa48b400700 1 --2- 172.21.15.119:0/2668822297 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa48c10eb90 0x7fa48c19aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:18.650 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:18.643+0000 7fa48b400700 1 -- 172.21.15.119:0/2668822297 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa48c19b200 con 0x7fa48c0743c0 2024-03-28T00:24:18.650 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:18.643+0000 7fa48b400700 1 --2- 172.21.15.119:0/2668822297 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa48c0743c0 0x7fa48c1a08e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa47c005950 tx=0x7fa47c005040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:24:18.651 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:18.643+0000 7fa488c00700 1 -- 172.21.15.119:0/2668822297 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa47c0052e0 con 0x7fa48c0743c0 2024-03-28T00:24:18.651 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:18.643+0000 7fa488c00700 1 -- 172.21.15.119:0/2668822297 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa47c00ba40 con 0x7fa48c0743c0 2024-03-28T00:24:18.651 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:18.643+0000 7fa492400700 1 -- 172.21.15.119:0/2668822297 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa48c19b490 con 0x7fa48c0743c0 2024-03-28T00:24:18.651 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:18.643+0000 7fa488c00700 1 -- 172.21.15.119:0/2668822297 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa47c003cb0 con 0x7fa48c0743c0 2024-03-28T00:24:18.651 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:18.644+0000 7fa492400700 1 -- 172.21.15.119:0/2668822297 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa48c1a77b0 con 0x7fa48c0743c0 2024-03-28T00:24:18.651 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:18.645+0000 7fa492400700 1 -- 172.21.15.119:0/2668822297 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa48c068e50 con 0x7fa48c0743c0 2024-03-28T00:24:18.652 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:18.646+0000 7fa488c00700 1 -- 172.21.15.119:0/2668822297 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 17) v1 ==== 137342+0+0 (secure 0 0 0) 0x7fa47c005510 con 0x7fa48c0743c0 2024-03-28T00:24:18.654 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:18.648+0000 7fa488c00700 1 --2- 172.21.15.119:0/2668822297 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fa4700a29a0 0x7fa4700a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:24:18.654 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:18.648+0000 7fa488c00700 1 -- 172.21.15.119:0/2668822297 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fa47c0cfb70 con 0x7fa48c0743c0 2024-03-28T00:24:18.654 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:18.648+0000 7fa482a00700 1 --2- 172.21.15.119:0/2668822297 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fa4700a29a0 0x7fa4700a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:24:18.654 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:18.648+0000 7fa482a00700 1 --2- 172.21.15.119:0/2668822297 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fa4700a29a0 0x7fa4700a4e60 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa474009750 tx=0x7fa474009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:24:18.658 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:18.652+0000 7fa488c00700 1 -- 172.21.15.119:0/2668822297 <== mon.1 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa47c013030 con 0x7fa48c0743c0 2024-03-28T00:24:18.881 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:18.875+0000 7fa492400700 1 -- 172.21.15.119:0/2668822297 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa48c19bfd0 con 0x7fa48c0743c0 2024-03-28T00:24:18.883 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:18.877+0000 7fa488c00700 1 -- 172.21.15.119:0/2668822297 <== mon.1 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+22638 (secure 0 0 0) 0x7fa47c09cb40 con 0x7fa48c0743c0 2024-03-28T00:24:18.883 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-28T00:24:18.884 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":87,"fsid":"10f73b92-ec98-11ee-95d0-87774f69a715","created":"2024-03-28T00:14:57.563604+0000","modified":"2024-03-28T00:24:05.529756+0000","last_up_change":"2024-03-28T00:22:58.821662+0000","last_in_change":"2024-03-28T00:22:38.467270+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T00:18:46.253414+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-28T00:23:54.261981+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-28T00:23:55.685836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c0b875d-a01e-4294-9c9c-5d0b048b8912","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":580396252},{"type":"v1","addr":"172.21.15.119:6803","nonce":580396252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":580396252},{"type":"v1","addr":"172.21.15.119:6805","nonce":580396252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":580396252},{"type":"v1","addr":"172.21.15.119:6809","nonce":580396252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":580396252},{"type":"v1","addr":"172.21.15.119:6807","nonce":580396252}]},"public_addr":"172.21.15.119:6803/580396252","cluster_addr":"172.21.15.119:6805/580396252","heartbeat_back_addr":"172.21.15.119:6809/580396252","heartbeat_front_addr":"172.21.15.119:6807/580396252","state":["exists","up"]},{"osd":1,"uuid":"5567f7c2-8ea4-4789-9ea1-467793324fb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":2500318888},{"type":"v1","addr":"172.21.15.132:6801","nonce":2500318888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":2500318888},{"type":"v1","addr":"172.21.15.132:6803","nonce":2500318888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":2500318888},{"type":"v1","addr":"172.21.15.132:6807","nonce":2500318888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":2500318888},{"type":"v1","addr":"172.21.15.132:6805","nonce":2500318888}]},"public_addr":"172.21.15.132:6801/2500318888","cluster_addr":"172.21.15.132:6803/2500318888","heartbeat_back_addr":"172.21.15.132:6807/2500318888","heartbeat_front_addr":"172.21.15.132:6805/2500318888","state":["exists","up"]},{"osd":2,"uuid":"ee54dbd0-24b4-4ee6-988c-ce71491199d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":2361434832},{"type":"v1","addr":"172.21.15.165:6801","nonce":2361434832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":2361434832},{"type":"v1","addr":"172.21.15.165:6803","nonce":2361434832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":2361434832},{"type":"v1","addr":"172.21.15.165:6807","nonce":2361434832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":2361434832},{"type":"v1","addr":"172.21.15.165:6805","nonce":2361434832}]},"public_addr":"172.21.15.165:6801/2361434832","cluster_addr":"172.21.15.165:6803/2361434832","heartbeat_back_addr":"172.21.15.165:6807/2361434832","heartbeat_front_addr":"172.21.15.165:6805/2361434832","state":["exists","up"]},{"osd":3,"uuid":"0464c40b-828f-4db4-96d9-3f7a6325e6da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":3732272210},{"type":"v1","addr":"172.21.15.119:6811","nonce":3732272210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":3732272210},{"type":"v1","addr":"172.21.15.119:6813","nonce":3732272210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":3732272210},{"type":"v1","addr":"172.21.15.119:6817","nonce":3732272210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":3732272210},{"type":"v1","addr":"172.21.15.119:6815","nonce":3732272210}]},"public_addr":"172.21.15.119:6811/3732272210","cluster_addr":"172.21.15.119:6813/3732272210","heartbeat_back_addr":"172.21.15.119:6817/3732272210","heartbeat_front_addr":"172.21.15.119:6815/3732272210","state":["exists","up"]},{"osd":4,"uuid":"8fd70b8e-5128-4173-aa0e-e735dfeb995d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":2803039505},{"type":"v1","addr":"172.21.15.132:6809","nonce":2803039505}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":2803039505},{"type":"v1","addr":"172.21.15.132:6811","nonce":2803039505}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":2803039505},{"type":"v1","addr":"172.21.15.132:6815","nonce":2803039505}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":2803039505},{"type":"v1","addr":"172.21.15.132:6813","nonce":2803039505}]},"public_addr":"172.21.15.132:6809/2803039505","cluster_addr":"172.21.15.132:6811/2803039505","heartbeat_back_addr":"172.21.15.132:6815/2803039505","heartbeat_front_addr":"172.21.15.132:6813/2803039505","state":["exists","up"]},{"osd":5,"uuid":"3224b4c1-733e-4424-a6da-870bbb5f94e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":2561252206},{"type":"v1","addr":"172.21.15.165:6809","nonce":2561252206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":2561252206},{"type":"v1","addr":"172.21.15.165:6811","nonce":2561252206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":2561252206},{"type":"v1","addr":"172.21.15.165:6815","nonce":2561252206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":2561252206},{"type":"v1","addr":"172.21.15.165:6813","nonce":2561252206}]},"public_addr":"172.21.15.165:6809/2561252206","cluster_addr":"172.21.15.165:6811/2561252206","heartbeat_back_addr":"172.21.15.165:6815/2561252206","heartbeat_front_addr":"172.21.15.165:6813/2561252206","state":["exists","up"]},{"osd":6,"uuid":"e435745d-59b5-439a-9091-0946ca0e9332","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":2545410230},{"type":"v1","addr":"172.21.15.119:6819","nonce":2545410230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":2545410230},{"type":"v1","addr":"172.21.15.119:6821","nonce":2545410230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":2545410230},{"type":"v1","addr":"172.21.15.119:6825","nonce":2545410230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":2545410230},{"type":"v1","addr":"172.21.15.119:6823","nonce":2545410230}]},"public_addr":"172.21.15.119:6819/2545410230","cluster_addr":"172.21.15.119:6821/2545410230","heartbeat_back_addr":"172.21.15.119:6825/2545410230","heartbeat_front_addr":"172.21.15.119:6823/2545410230","state":["exists","up"]},{"osd":7,"uuid":"79b97a12-f6b4-48c3-b517-2dd6b6e7c85f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":2877892376},{"type":"v1","addr":"172.21.15.132:6817","nonce":2877892376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":2877892376},{"type":"v1","addr":"172.21.15.132:6819","nonce":2877892376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":2877892376},{"type":"v1","addr":"172.21.15.132:6823","nonce":2877892376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":2877892376},{"type":"v1","addr":"172.21.15.132:6821","nonce":2877892376}]},"public_addr":"172.21.15.132:6817/2877892376","cluster_addr":"172.21.15.132:6819/2877892376","heartbeat_back_addr":"172.21.15.132:6823/2877892376","heartbeat_front_addr":"172.21.15.132:6821/2877892376","state":["exists","up"]},{"osd":8,"uuid":"2970524c-b99a-412d-85c2-6c29b54035e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":4115226150},{"type":"v1","addr":"172.21.15.165:6817","nonce":4115226150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":4115226150},{"type":"v1","addr":"172.21.15.165:6819","nonce":4115226150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":4115226150},{"type":"v1","addr":"172.21.15.165:6823","nonce":4115226150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":4115226150},{"type":"v1","addr":"172.21.15.165:6821","nonce":4115226150}]},"public_addr":"172.21.15.165:6817/4115226150","cluster_addr":"172.21.15.165:6819/4115226150","heartbeat_back_addr":"172.21.15.165:6823/4115226150","heartbeat_front_addr":"172.21.15.165:6821/4115226150","state":["exists","up"]},{"osd":9,"uuid":"2b1a08ed-cde9-4def-b3e8-838fff22709c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":2532245222},{"type":"v1","addr":"172.21.15.119:6827","nonce":2532245222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":2532245222},{"type":"v1","addr":"172.21.15.119:6829","nonce":2532245222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":2532245222},{"type":"v1","addr":"172.21.15.119:6833","nonce":2532245222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":2532245222},{"type":"v1","addr":"172.21.15.119:6831","nonce":2532245222}]},"public_addr":"172.21.15.119:6827/2532245222","cluster_addr":"172.21.15.119:6829/2532245222","heartbeat_back_addr":"172.21.15.119:6833/2532245222","heartbeat_front_addr":"172.21.15.119:6831/2532245222","state":["exists","up"]},{"osd":10,"uuid":"f019acd0-915a-4094-b12c-b53672740a5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":3454298492},{"type":"v1","addr":"172.21.15.132:6825","nonce":3454298492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":3454298492},{"type":"v1","addr":"172.21.15.132:6827","nonce":3454298492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":3454298492},{"type":"v1","addr":"172.21.15.132:6831","nonce":3454298492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":3454298492},{"type":"v1","addr":"172.21.15.132:6829","nonce":3454298492}]},"public_addr":"172.21.15.132:6825/3454298492","cluster_addr":"172.21.15.132:6827/3454298492","heartbeat_back_addr":"172.21.15.132:6831/3454298492","heartbeat_front_addr":"172.21.15.132:6829/3454298492","state":["exists","up"]},{"osd":11,"uuid":"2c1e59a8-8f21-45c0-8568-4b632ce020f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":2138771601},{"type":"v1","addr":"172.21.15.165:6825","nonce":2138771601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":2138771601},{"type":"v1","addr":"172.21.15.165:6827","nonce":2138771601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":2138771601},{"type":"v1","addr":"172.21.15.165:6831","nonce":2138771601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":2138771601},{"type":"v1","addr":"172.21.15.165:6829","nonce":2138771601}]},"public_addr":"172.21.15.165:6825/2138771601","cluster_addr":"172.21.15.165:6827/2138771601","heartbeat_back_addr":"172.21.15.165:6831/2138771601","heartbeat_front_addr":"172.21.15.165:6829/2138771601","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:17:54.791531+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:18:19.466638+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:18:43.435536+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:19:15.702685+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:19:41.767370+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:20:06.793003+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:20:39.952744+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:21:05.727262+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:21:30.735829+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:22:04.072142+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:22:30.894047+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:22:56.637728+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.119:6801/2575246519":"2024-03-29T00:16:03.582545+0000","172.21.15.119:6800/2575246519":"2024-03-29T00:16:03.582545+0000","172.21.15.119:0/87565965":"2024-03-29T00:16:03.582545+0000","172.21.15.119:6801/2056426384":"2024-03-29T00:15:30.472283+0000","172.21.15.119:0/1639109126":"2024-03-29T00:15:30.472283+0000","172.21.15.119:0/468478300":"2024-03-29T00:15:30.472283+0000","172.21.15.119:0/2891240357":"2024-03-29T00:15:30.472283+0000","172.21.15.119:0/4201699272":"2024-03-29T00:16:03.582545+0000","172.21.15.119:6800/2056426384":"2024-03-29T00:15:30.472283+0000","172.21.15.119:0/2635982178":"2024-03-29T00:16:03.582545+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T00:24:18.886 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:18.880+0000 7fa492400700 1 -- 172.21.15.119:0/2668822297 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fa4700a29a0 msgr2=0x7fa4700a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:24:18.886 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:18.880+0000 7fa492400700 1 --2- 172.21.15.119:0/2668822297 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fa4700a29a0 0x7fa4700a4e60 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa474009750 tx=0x7fa474009310 comp rx=0 tx=0).stop 2024-03-28T00:24:18.886 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:18.880+0000 7fa492400700 1 -- 172.21.15.119:0/2668822297 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa48c0743c0 msgr2=0x7fa48c1a08e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:24:18.886 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:18.880+0000 7fa492400700 1 --2- 172.21.15.119:0/2668822297 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa48c0743c0 0x7fa48c1a08e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa47c005950 tx=0x7fa47c005040 comp rx=0 tx=0).stop 2024-03-28T00:24:18.886 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:18.880+0000 7fa492400700 1 -- 172.21.15.119:0/2668822297 shutdown_connections 2024-03-28T00:24:18.887 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:18.880+0000 7fa492400700 1 --2- 172.21.15.119:0/2668822297 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fa4700a29a0 0x7fa4700a4e60 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:18.887 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:18.880+0000 7fa492400700 1 --2- 172.21.15.119:0/2668822297 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa48c10eb90 0x7fa48c19aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:18.887 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:18.880+0000 7fa492400700 1 --2- 172.21.15.119:0/2668822297 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa48c0743c0 0x7fa48c1a08e0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:18.887 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:18.880+0000 7fa492400700 1 --2- 172.21.15.119:0/2668822297 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa48c074ce0 0x7fa48c1a0e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:18.887 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:18.880+0000 7fa492400700 1 -- 172.21.15.119:0/2668822297 >> 172.21.15.119:0/2668822297 conn(0x7fa48c0792f0 msgr2=0x7fa48c100570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:24:18.887 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:18.880+0000 7fa492400700 1 -- 172.21.15.119:0/2668822297 shutdown_connections 2024-03-28T00:24:18.887 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:18.880+0000 7fa492400700 1 -- 172.21.15.119:0/2668822297 wait complete. 2024-03-28T00:24:18.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:24:18 smithi119 ceph-mon[31518]: pgmap v324: 97 pgs: 97 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.9 KiB/s wr, 18 op/s; 1 B/s, 0 objects/s recovering 2024-03-28T00:24:19.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:24:18 smithi165 ceph-mon[36302]: pgmap v324: 97 pgs: 97 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.9 KiB/s wr, 18 op/s; 1 B/s, 0 objects/s recovering 2024-03-28T00:24:19.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:24:18 smithi132 ceph-mon[35941]: pgmap v324: 97 pgs: 97 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.9 KiB/s wr, 18 op/s; 1 B/s, 0 objects/s recovering 2024-03-28T00:24:19.859 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-03-28T00:24:19.859 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 10f73b92-ec98-11ee-95d0-87774f69a715 -- ceph osd dump --format=json 2024-03-28T00:24:20.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:24:19 smithi165 ceph-mon[36302]: from='client.? 172.21.15.119:0/2668822297' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-28T00:24:20.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:24:19 smithi119 ceph-mon[31518]: from='client.? 172.21.15.119:0/2668822297' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-28T00:24:20.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:24:19 smithi132 ceph-mon[35941]: from='client.? 172.21.15.119:0/2668822297' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-28T00:24:20.294 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/mon.a/config 2024-03-28T00:24:21.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:24:20 smithi165 ceph-mon[36302]: pgmap v325: 97 pgs: 97 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.1 KiB/s wr, 16 op/s; 1 B/s, 0 objects/s recovering 2024-03-28T00:24:21.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:24:20 smithi119 ceph-mon[31518]: pgmap v325: 97 pgs: 97 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.1 KiB/s wr, 16 op/s; 1 B/s, 0 objects/s recovering 2024-03-28T00:24:21.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:24:20 smithi132 ceph-mon[35941]: pgmap v325: 97 pgs: 97 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.1 KiB/s wr, 16 op/s; 1 B/s, 0 objects/s recovering 2024-03-28T00:24:22.411 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:22.392+0000 7ff35a400700 1 -- 172.21.15.119:0/2772221311 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff3541081e0 msgr2=0x7ff3541085c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:24:22.411 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:22.392+0000 7ff35a400700 1 --2- 172.21.15.119:0/2772221311 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff3541081e0 0x7ff3541085c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff34400b0a0 tx=0x7ff34400b3b0 comp rx=0 tx=0).stop 2024-03-28T00:24:22.411 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:22.392+0000 7ff35a400700 1 -- 172.21.15.119:0/2772221311 shutdown_connections 2024-03-28T00:24:22.411 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:22.392+0000 7ff35a400700 1 --2- 172.21.15.119:0/2772221311 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff35406a7e0 0x7ff35410f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:22.411 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:22.392+0000 7ff35a400700 1 --2- 172.21.15.119:0/2772221311 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff3541081e0 0x7ff3541085c0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:22.411 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:22.392+0000 7ff35a400700 1 --2- 172.21.15.119:0/2772221311 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff354069e40 0x7ff35406a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:22.411 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:22.392+0000 7ff35a400700 1 -- 172.21.15.119:0/2772221311 >> 172.21.15.119:0/2772221311 conn(0x7ff354077fc0 msgr2=0x7ff3540783d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:24:22.411 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:22.392+0000 7ff35a400700 1 -- 172.21.15.119:0/2772221311 shutdown_connections 2024-03-28T00:24:22.411 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:22.392+0000 7ff35a400700 1 -- 172.21.15.119:0/2772221311 wait complete. 2024-03-28T00:24:22.412 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:22.393+0000 7ff35a400700 1 Processor -- start 2024-03-28T00:24:22.412 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:22.394+0000 7ff35a400700 1 -- start start 2024-03-28T00:24:22.412 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:22.394+0000 7ff35a400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff354069e40 0x7ff354101160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:24:22.412 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:22.394+0000 7ff35a400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff35406a7e0 0x7ff3541016a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:24:22.412 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:22.394+0000 7ff35a400700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff3541081e0 0x7ff354101be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:24:22.412 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:22.394+0000 7ff35a400700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff354112490 con 0x7ff354069e40 2024-03-28T00:24:22.412 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:22.394+0000 7ff35a400700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff354112310 con 0x7ff35406a7e0 2024-03-28T00:24:22.412 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:22.394+0000 7ff35a400700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff354112610 con 0x7ff3541081e0 2024-03-28T00:24:22.412 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:22.394+0000 7ff34aa00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff35406a7e0 0x7ff3541016a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:24:22.412 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:22.394+0000 7ff353400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff354069e40 0x7ff354101160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:24:22.412 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:22.394+0000 7ff353e00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff3541081e0 0x7ff354101be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:24:22.412 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:22.394+0000 7ff353e00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff3541081e0 0x7ff354101be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.119:46364/0 (socket says 172.21.15.119:46364) 2024-03-28T00:24:22.412 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:22.394+0000 7ff34aa00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff35406a7e0 0x7ff3541016a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.119:45798/0 (socket says 172.21.15.119:45798) 2024-03-28T00:24:22.412 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:22.394+0000 7ff353e00700 1 -- 172.21.15.119:0/2825267937 learned_addr learned my addr 172.21.15.119:0/2825267937 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:24:22.412 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:22.395+0000 7ff353e00700 1 -- 172.21.15.119:0/2825267937 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff35406a7e0 msgr2=0x7ff3541016a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:24:22.413 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:22.395+0000 7ff353e00700 1 --2- 172.21.15.119:0/2825267937 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff35406a7e0 0x7ff3541016a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:22.413 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:22.395+0000 7ff353e00700 1 -- 172.21.15.119:0/2825267937 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff354069e40 msgr2=0x7ff354101160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:24:22.413 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:22.395+0000 7ff353e00700 1 --2- 172.21.15.119:0/2825267937 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff354069e40 0x7ff354101160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:22.413 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:22.395+0000 7ff353e00700 1 -- 172.21.15.119:0/2825267937 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3541a6e30 con 0x7ff3541081e0 2024-03-28T00:24:22.413 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:22.395+0000 7ff34aa00700 1 --2- 172.21.15.119:0/2825267937 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff35406a7e0 0x7ff3541016a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T00:24:22.413 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:22.395+0000 7ff353e00700 1 --2- 172.21.15.119:0/2825267937 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff3541081e0 0x7ff354101be0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ff34c00e940 tx=0x7ff34c00ec50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:24:22.413 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:22.395+0000 7ff350c00700 1 -- 172.21.15.119:0/2825267937 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff34c004a30 con 0x7ff3541081e0 2024-03-28T00:24:22.413 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:22.395+0000 7ff35a400700 1 -- 172.21.15.119:0/2825267937 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3541a7120 con 0x7ff3541081e0 2024-03-28T00:24:22.413 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:22.395+0000 7ff350c00700 1 -- 172.21.15.119:0/2825267937 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff34c005500 con 0x7ff3541081e0 2024-03-28T00:24:22.413 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:22.395+0000 7ff350c00700 1 -- 172.21.15.119:0/2825267937 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff34c00ce20 con 0x7ff3541081e0 2024-03-28T00:24:22.413 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:22.396+0000 7ff35a400700 1 -- 172.21.15.119:0/2825267937 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3541a7660 con 0x7ff3541081e0 2024-03-28T00:24:22.413 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:22.396+0000 7ff35a400700 1 -- 172.21.15.119:0/2825267937 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff354068e50 con 0x7ff3541081e0 2024-03-28T00:24:22.413 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:22.399+0000 7ff350c00700 1 -- 172.21.15.119:0/2825267937 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 17) v1 ==== 137342+0+0 (secure 0 0 0) 0x7ff34c005000 con 0x7ff3541081e0 2024-03-28T00:24:22.413 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:22.400+0000 7ff350c00700 1 --2- 172.21.15.119:0/2825267937 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7ff3380a2920 0x7ff3380a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:24:22.413 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:22.400+0000 7ff350c00700 1 -- 172.21.15.119:0/2825267937 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8153+0+0 (secure 0 0 0) 0x7ff34c0ceec0 con 0x7ff3541081e0 2024-03-28T00:24:22.414 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:22.400+0000 7ff353400700 1 --2- 172.21.15.119:0/2825267937 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7ff3380a2920 0x7ff3380a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:24:22.414 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:22.401+0000 7ff353400700 1 --2- 172.21.15.119:0/2825267937 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7ff3380a2920 0x7ff3380a4de0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ff344009e60 tx=0x7ff34400b740 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:24:22.414 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:22.406+0000 7ff350c00700 1 -- 172.21.15.119:0/2825267937 <== mon.1 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff34c053740 con 0x7ff3541081e0 2024-03-28T00:24:22.637 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:22.631+0000 7ff35a400700 1 -- 172.21.15.119:0/2825267937 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff35402da30 con 0x7ff3541081e0 2024-03-28T00:24:22.638 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:22.633+0000 7ff350c00700 1 -- 172.21.15.119:0/2825267937 <== mon.1 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+22638 (secure 0 0 0) 0x7ff34c09be00 con 0x7ff3541081e0 2024-03-28T00:24:22.639 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-28T00:24:22.639 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":87,"fsid":"10f73b92-ec98-11ee-95d0-87774f69a715","created":"2024-03-28T00:14:57.563604+0000","modified":"2024-03-28T00:24:05.529756+0000","last_up_change":"2024-03-28T00:22:58.821662+0000","last_in_change":"2024-03-28T00:22:38.467270+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T00:18:46.253414+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-28T00:23:54.261981+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-28T00:23:55.685836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c0b875d-a01e-4294-9c9c-5d0b048b8912","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":580396252},{"type":"v1","addr":"172.21.15.119:6803","nonce":580396252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":580396252},{"type":"v1","addr":"172.21.15.119:6805","nonce":580396252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":580396252},{"type":"v1","addr":"172.21.15.119:6809","nonce":580396252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":580396252},{"type":"v1","addr":"172.21.15.119:6807","nonce":580396252}]},"public_addr":"172.21.15.119:6803/580396252","cluster_addr":"172.21.15.119:6805/580396252","heartbeat_back_addr":"172.21.15.119:6809/580396252","heartbeat_front_addr":"172.21.15.119:6807/580396252","state":["exists","up"]},{"osd":1,"uuid":"5567f7c2-8ea4-4789-9ea1-467793324fb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":2500318888},{"type":"v1","addr":"172.21.15.132:6801","nonce":2500318888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":2500318888},{"type":"v1","addr":"172.21.15.132:6803","nonce":2500318888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":2500318888},{"type":"v1","addr":"172.21.15.132:6807","nonce":2500318888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":2500318888},{"type":"v1","addr":"172.21.15.132:6805","nonce":2500318888}]},"public_addr":"172.21.15.132:6801/2500318888","cluster_addr":"172.21.15.132:6803/2500318888","heartbeat_back_addr":"172.21.15.132:6807/2500318888","heartbeat_front_addr":"172.21.15.132:6805/2500318888","state":["exists","up"]},{"osd":2,"uuid":"ee54dbd0-24b4-4ee6-988c-ce71491199d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":2361434832},{"type":"v1","addr":"172.21.15.165:6801","nonce":2361434832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":2361434832},{"type":"v1","addr":"172.21.15.165:6803","nonce":2361434832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":2361434832},{"type":"v1","addr":"172.21.15.165:6807","nonce":2361434832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":2361434832},{"type":"v1","addr":"172.21.15.165:6805","nonce":2361434832}]},"public_addr":"172.21.15.165:6801/2361434832","cluster_addr":"172.21.15.165:6803/2361434832","heartbeat_back_addr":"172.21.15.165:6807/2361434832","heartbeat_front_addr":"172.21.15.165:6805/2361434832","state":["exists","up"]},{"osd":3,"uuid":"0464c40b-828f-4db4-96d9-3f7a6325e6da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":3732272210},{"type":"v1","addr":"172.21.15.119:6811","nonce":3732272210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":3732272210},{"type":"v1","addr":"172.21.15.119:6813","nonce":3732272210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":3732272210},{"type":"v1","addr":"172.21.15.119:6817","nonce":3732272210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":3732272210},{"type":"v1","addr":"172.21.15.119:6815","nonce":3732272210}]},"public_addr":"172.21.15.119:6811/3732272210","cluster_addr":"172.21.15.119:6813/3732272210","heartbeat_back_addr":"172.21.15.119:6817/3732272210","heartbeat_front_addr":"172.21.15.119:6815/3732272210","state":["exists","up"]},{"osd":4,"uuid":"8fd70b8e-5128-4173-aa0e-e735dfeb995d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":2803039505},{"type":"v1","addr":"172.21.15.132:6809","nonce":2803039505}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":2803039505},{"type":"v1","addr":"172.21.15.132:6811","nonce":2803039505}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":2803039505},{"type":"v1","addr":"172.21.15.132:6815","nonce":2803039505}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":2803039505},{"type":"v1","addr":"172.21.15.132:6813","nonce":2803039505}]},"public_addr":"172.21.15.132:6809/2803039505","cluster_addr":"172.21.15.132:6811/2803039505","heartbeat_back_addr":"172.21.15.132:6815/2803039505","heartbeat_front_addr":"172.21.15.132:6813/2803039505","state":["exists","up"]},{"osd":5,"uuid":"3224b4c1-733e-4424-a6da-870bbb5f94e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":2561252206},{"type":"v1","addr":"172.21.15.165:6809","nonce":2561252206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":2561252206},{"type":"v1","addr":"172.21.15.165:6811","nonce":2561252206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":2561252206},{"type":"v1","addr":"172.21.15.165:6815","nonce":2561252206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":2561252206},{"type":"v1","addr":"172.21.15.165:6813","nonce":2561252206}]},"public_addr":"172.21.15.165:6809/2561252206","cluster_addr":"172.21.15.165:6811/2561252206","heartbeat_back_addr":"172.21.15.165:6815/2561252206","heartbeat_front_addr":"172.21.15.165:6813/2561252206","state":["exists","up"]},{"osd":6,"uuid":"e435745d-59b5-439a-9091-0946ca0e9332","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":2545410230},{"type":"v1","addr":"172.21.15.119:6819","nonce":2545410230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":2545410230},{"type":"v1","addr":"172.21.15.119:6821","nonce":2545410230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":2545410230},{"type":"v1","addr":"172.21.15.119:6825","nonce":2545410230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":2545410230},{"type":"v1","addr":"172.21.15.119:6823","nonce":2545410230}]},"public_addr":"172.21.15.119:6819/2545410230","cluster_addr":"172.21.15.119:6821/2545410230","heartbeat_back_addr":"172.21.15.119:6825/2545410230","heartbeat_front_addr":"172.21.15.119:6823/2545410230","state":["exists","up"]},{"osd":7,"uuid":"79b97a12-f6b4-48c3-b517-2dd6b6e7c85f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":2877892376},{"type":"v1","addr":"172.21.15.132:6817","nonce":2877892376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":2877892376},{"type":"v1","addr":"172.21.15.132:6819","nonce":2877892376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":2877892376},{"type":"v1","addr":"172.21.15.132:6823","nonce":2877892376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":2877892376},{"type":"v1","addr":"172.21.15.132:6821","nonce":2877892376}]},"public_addr":"172.21.15.132:6817/2877892376","cluster_addr":"172.21.15.132:6819/2877892376","heartbeat_back_addr":"172.21.15.132:6823/2877892376","heartbeat_front_addr":"172.21.15.132:6821/2877892376","state":["exists","up"]},{"osd":8,"uuid":"2970524c-b99a-412d-85c2-6c29b54035e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":4115226150},{"type":"v1","addr":"172.21.15.165:6817","nonce":4115226150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":4115226150},{"type":"v1","addr":"172.21.15.165:6819","nonce":4115226150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":4115226150},{"type":"v1","addr":"172.21.15.165:6823","nonce":4115226150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":4115226150},{"type":"v1","addr":"172.21.15.165:6821","nonce":4115226150}]},"public_addr":"172.21.15.165:6817/4115226150","cluster_addr":"172.21.15.165:6819/4115226150","heartbeat_back_addr":"172.21.15.165:6823/4115226150","heartbeat_front_addr":"172.21.15.165:6821/4115226150","state":["exists","up"]},{"osd":9,"uuid":"2b1a08ed-cde9-4def-b3e8-838fff22709c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":2532245222},{"type":"v1","addr":"172.21.15.119:6827","nonce":2532245222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":2532245222},{"type":"v1","addr":"172.21.15.119:6829","nonce":2532245222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":2532245222},{"type":"v1","addr":"172.21.15.119:6833","nonce":2532245222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":2532245222},{"type":"v1","addr":"172.21.15.119:6831","nonce":2532245222}]},"public_addr":"172.21.15.119:6827/2532245222","cluster_addr":"172.21.15.119:6829/2532245222","heartbeat_back_addr":"172.21.15.119:6833/2532245222","heartbeat_front_addr":"172.21.15.119:6831/2532245222","state":["exists","up"]},{"osd":10,"uuid":"f019acd0-915a-4094-b12c-b53672740a5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":3454298492},{"type":"v1","addr":"172.21.15.132:6825","nonce":3454298492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":3454298492},{"type":"v1","addr":"172.21.15.132:6827","nonce":3454298492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":3454298492},{"type":"v1","addr":"172.21.15.132:6831","nonce":3454298492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":3454298492},{"type":"v1","addr":"172.21.15.132:6829","nonce":3454298492}]},"public_addr":"172.21.15.132:6825/3454298492","cluster_addr":"172.21.15.132:6827/3454298492","heartbeat_back_addr":"172.21.15.132:6831/3454298492","heartbeat_front_addr":"172.21.15.132:6829/3454298492","state":["exists","up"]},{"osd":11,"uuid":"2c1e59a8-8f21-45c0-8568-4b632ce020f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":2138771601},{"type":"v1","addr":"172.21.15.165:6825","nonce":2138771601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":2138771601},{"type":"v1","addr":"172.21.15.165:6827","nonce":2138771601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":2138771601},{"type":"v1","addr":"172.21.15.165:6831","nonce":2138771601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":2138771601},{"type":"v1","addr":"172.21.15.165:6829","nonce":2138771601}]},"public_addr":"172.21.15.165:6825/2138771601","cluster_addr":"172.21.15.165:6827/2138771601","heartbeat_back_addr":"172.21.15.165:6831/2138771601","heartbeat_front_addr":"172.21.15.165:6829/2138771601","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:17:54.791531+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:18:19.466638+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:18:43.435536+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:19:15.702685+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:19:41.767370+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:20:06.793003+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:20:39.952744+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:21:05.727262+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:21:30.735829+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:22:04.072142+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:22:30.894047+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:22:56.637728+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.119:6801/2575246519":"2024-03-29T00:16:03.582545+0000","172.21.15.119:6800/2575246519":"2024-03-29T00:16:03.582545+0000","172.21.15.119:0/87565965":"2024-03-29T00:16:03.582545+0000","172.21.15.119:6801/2056426384":"2024-03-29T00:15:30.472283+0000","172.21.15.119:0/1639109126":"2024-03-29T00:15:30.472283+0000","172.21.15.119:0/468478300":"2024-03-29T00:15:30.472283+0000","172.21.15.119:0/2891240357":"2024-03-29T00:15:30.472283+0000","172.21.15.119:0/4201699272":"2024-03-29T00:16:03.582545+0000","172.21.15.119:6800/2056426384":"2024-03-29T00:15:30.472283+0000","172.21.15.119:0/2635982178":"2024-03-29T00:16:03.582545+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T00:24:22.641 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:22.636+0000 7ff35a400700 1 -- 172.21.15.119:0/2825267937 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7ff3380a2920 msgr2=0x7ff3380a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:24:22.641 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:22.636+0000 7ff35a400700 1 --2- 172.21.15.119:0/2825267937 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7ff3380a2920 0x7ff3380a4de0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ff344009e60 tx=0x7ff34400b740 comp rx=0 tx=0).stop 2024-03-28T00:24:22.642 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:22.636+0000 7ff35a400700 1 -- 172.21.15.119:0/2825267937 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff3541081e0 msgr2=0x7ff354101be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:24:22.642 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:22.636+0000 7ff35a400700 1 --2- 172.21.15.119:0/2825267937 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff3541081e0 0x7ff354101be0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ff34c00e940 tx=0x7ff34c00ec50 comp rx=0 tx=0).stop 2024-03-28T00:24:22.642 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:22.636+0000 7ff35a400700 1 -- 172.21.15.119:0/2825267937 shutdown_connections 2024-03-28T00:24:22.642 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:22.636+0000 7ff35a400700 1 --2- 172.21.15.119:0/2825267937 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7ff3380a2920 0x7ff3380a4de0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:22.642 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:22.636+0000 7ff35a400700 1 --2- 172.21.15.119:0/2825267937 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff3541081e0 0x7ff354101be0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:22.642 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:22.636+0000 7ff35a400700 1 --2- 172.21.15.119:0/2825267937 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff354069e40 0x7ff354101160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:22.642 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:22.636+0000 7ff35a400700 1 --2- 172.21.15.119:0/2825267937 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff35406a7e0 0x7ff3541016a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:22.642 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:22.636+0000 7ff35a400700 1 -- 172.21.15.119:0/2825267937 >> 172.21.15.119:0/2825267937 conn(0x7ff354077fc0 msgr2=0x7ff3540ffbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:24:22.642 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:22.636+0000 7ff35a400700 1 -- 172.21.15.119:0/2825267937 shutdown_connections 2024-03-28T00:24:22.642 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:22.636+0000 7ff35a400700 1 -- 172.21.15.119:0/2825267937 wait complete. 2024-03-28T00:24:23.121 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:24:22 smithi119 ceph-mon[31518]: pgmap v326: 97 pgs: 97 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 9 op/s 2024-03-28T00:24:23.121 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:24:22 smithi119 ceph-mon[31518]: from='client.? 172.21.15.119:0/2825267937' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-28T00:24:23.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:24:22 smithi165 ceph-mon[36302]: pgmap v326: 97 pgs: 97 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 9 op/s 2024-03-28T00:24:23.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:24:22 smithi165 ceph-mon[36302]: from='client.? 172.21.15.119:0/2825267937' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-28T00:24:23.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:24:22 smithi132 ceph-mon[35941]: pgmap v326: 97 pgs: 97 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 9 op/s 2024-03-28T00:24:23.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:24:22 smithi132 ceph-mon[35941]: from='client.? 172.21.15.119:0/2825267937' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-28T00:24:23.559 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 10f73b92-ec98-11ee-95d0-87774f69a715 -- ceph tell osd.0 flush_pg_stats 2024-03-28T00:24:23.559 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 10f73b92-ec98-11ee-95d0-87774f69a715 -- ceph tell osd.1 flush_pg_stats 2024-03-28T00:24:23.560 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 10f73b92-ec98-11ee-95d0-87774f69a715 -- ceph tell osd.2 flush_pg_stats 2024-03-28T00:24:23.560 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 10f73b92-ec98-11ee-95d0-87774f69a715 -- ceph tell osd.3 flush_pg_stats 2024-03-28T00:24:23.561 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 10f73b92-ec98-11ee-95d0-87774f69a715 -- ceph tell osd.4 flush_pg_stats 2024-03-28T00:24:23.561 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 10f73b92-ec98-11ee-95d0-87774f69a715 -- ceph tell osd.5 flush_pg_stats 2024-03-28T00:24:23.561 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 10f73b92-ec98-11ee-95d0-87774f69a715 -- ceph tell osd.6 flush_pg_stats 2024-03-28T00:24:23.562 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 10f73b92-ec98-11ee-95d0-87774f69a715 -- ceph tell osd.7 flush_pg_stats 2024-03-28T00:24:23.562 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 10f73b92-ec98-11ee-95d0-87774f69a715 -- ceph tell osd.8 flush_pg_stats 2024-03-28T00:24:23.563 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 10f73b92-ec98-11ee-95d0-87774f69a715 -- ceph tell osd.9 flush_pg_stats 2024-03-28T00:24:23.563 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 10f73b92-ec98-11ee-95d0-87774f69a715 -- ceph tell osd.10 flush_pg_stats 2024-03-28T00:24:23.564 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 10f73b92-ec98-11ee-95d0-87774f69a715 -- ceph tell osd.11 flush_pg_stats 2024-03-28T00:24:24.580 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/mon.a/config 2024-03-28T00:24:24.604 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/mon.a/config 2024-03-28T00:24:24.610 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/mon.a/config 2024-03-28T00:24:24.611 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/mon.a/config 2024-03-28T00:24:25.008 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/mon.a/config 2024-03-28T00:24:25.011 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/mon.a/config 2024-03-28T00:24:25.019 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/mon.a/config 2024-03-28T00:24:25.019 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/mon.a/config 2024-03-28T00:24:25.021 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/mon.a/config 2024-03-28T00:24:25.022 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/mon.a/config 2024-03-28T00:24:25.030 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/mon.a/config 2024-03-28T00:24:25.092 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:24:24 smithi119 ceph-mon[31518]: pgmap v327: 97 pgs: 97 active+clean; 645 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1.1 KiB/s wr, 12 op/s 2024-03-28T00:24:25.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:24:24 smithi165 ceph-mon[36302]: pgmap v327: 97 pgs: 97 active+clean; 645 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1.1 KiB/s wr, 12 op/s 2024-03-28T00:24:25.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:24:24 smithi132 ceph-mon[35941]: pgmap v327: 97 pgs: 97 active+clean; 645 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1.1 KiB/s wr, 12 op/s 2024-03-28T00:24:26.857 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/mon.a/config 2024-03-28T00:24:27.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:24:26 smithi165 ceph-mon[36302]: pgmap v328: 97 pgs: 97 active+clean; 645 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 426 B/s wr, 9 op/s 2024-03-28T00:24:27.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:24:26 smithi119 ceph-mon[31518]: pgmap v328: 97 pgs: 97 active+clean; 645 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 426 B/s wr, 9 op/s 2024-03-28T00:24:27.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:24:26 smithi132 ceph-mon[35941]: pgmap v328: 97 pgs: 97 active+clean; 645 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 426 B/s wr, 9 op/s 2024-03-28T00:24:29.098 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:24:28 smithi119 ceph-mon[31518]: pgmap v329: 97 pgs: 97 active+clean; 645 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 426 B/s wr, 9 op/s 2024-03-28T00:24:29.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:24:28 smithi165 ceph-mon[36302]: pgmap v329: 97 pgs: 97 active+clean; 645 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 426 B/s wr, 9 op/s 2024-03-28T00:24:29.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:24:28 smithi132 ceph-mon[35941]: pgmap v329: 97 pgs: 97 active+clean; 645 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 426 B/s wr, 9 op/s 2024-03-28T00:24:30.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:24:30 smithi165 ceph-mon[36302]: pgmap v330: 97 pgs: 97 active+clean; 645 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 426 B/s wr, 11 op/s 2024-03-28T00:24:30.899 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:30.773+0000 7f18b0a00700 1 -- 172.21.15.119:0/410187810 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f18ac0745c0 msgr2=0x7f18ac0749a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:24:30.899 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:30.773+0000 7f18b0a00700 1 --2- 172.21.15.119:0/410187810 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f18ac0745c0 0x7f18ac0749a0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f189c009990 tx=0x7f189c009ca0 comp rx=0 tx=0).stop 2024-03-28T00:24:30.899 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:30.773+0000 7f18b0a00700 1 -- 172.21.15.119:0/410187810 shutdown_connections 2024-03-28T00:24:30.899 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:30.773+0000 7f18b0a00700 1 --2- 172.21.15.119:0/410187810 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f18ac10e8c0 0x7f18ac110d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:30.899 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:30.773+0000 7f18b0a00700 1 --2- 172.21.15.119:0/410187810 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f18ac0745c0 0x7f18ac0749a0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:30.899 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:30.773+0000 7f18b0a00700 1 --2- 172.21.15.119:0/410187810 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f18ac074ee0 0x7f18ac10e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:30.899 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:30.773+0000 7f18b0a00700 1 -- 172.21.15.119:0/410187810 >> 172.21.15.119:0/410187810 conn(0x7f18ac0fdb90 msgr2=0x7f18ac0fffb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:24:30.899 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:30.773+0000 7f18b0a00700 1 -- 172.21.15.119:0/410187810 shutdown_connections 2024-03-28T00:24:30.899 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:30.773+0000 7f18b0a00700 1 -- 172.21.15.119:0/410187810 wait complete. 2024-03-28T00:24:30.899 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:30.774+0000 7f18b0a00700 1 Processor -- start 2024-03-28T00:24:30.899 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:30.775+0000 7f18b0a00700 1 -- start start 2024-03-28T00:24:30.899 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:30.775+0000 7f18b0a00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f18ac0745c0 0x7f18ac1a06e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:24:30.900 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:30.775+0000 7f18b0a00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f18ac074ee0 0x7f18ac1a0c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:24:30.900 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:30.775+0000 7f18b0a00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f18ac10e8c0 0x7f18ac19a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:24:30.900 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:30.775+0000 7f18b0a00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18ac114600 con 0x7f18ac0745c0 2024-03-28T00:24:30.900 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:30.775+0000 7f18b0a00700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18ac114480 con 0x7f18ac074ee0 2024-03-28T00:24:30.900 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:30.775+0000 7f18b0a00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18ac114780 con 0x7f18ac10e8c0 2024-03-28T00:24:30.900 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:30.775+0000 7f18ab200700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f18ac10e8c0 0x7f18ac19a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:24:30.900 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:30.775+0000 7f18aa800700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f18ac0745c0 0x7f18ac1a06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:24:30.900 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:30.775+0000 7f18a3e00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f18ac074ee0 0x7f18ac1a0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:24:30.900 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:30.775+0000 7f18ab200700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f18ac10e8c0 0x7f18ac19a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.119:54524/0 (socket says 172.21.15.119:54524) 2024-03-28T00:24:30.900 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:30.775+0000 7f18ab200700 1 -- 172.21.15.119:0/56211215 learned_addr learned my addr 172.21.15.119:0/56211215 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:24:30.900 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:30.776+0000 7f18ab200700 1 -- 172.21.15.119:0/56211215 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f18ac074ee0 msgr2=0x7f18ac1a0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:24:30.900 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:30.776+0000 7f18ab200700 1 --2- 172.21.15.119:0/56211215 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f18ac074ee0 0x7f18ac1a0c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:30.900 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:30.776+0000 7f18ab200700 1 -- 172.21.15.119:0/56211215 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f18ac0745c0 msgr2=0x7f18ac1a06e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:24:30.901 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:30.776+0000 7f18ab200700 1 --2- 172.21.15.119:0/56211215 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f18ac0745c0 0x7f18ac1a06e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:30.901 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:30.776+0000 7f18ab200700 1 -- 172.21.15.119:0/56211215 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18ac19b000 con 0x7f18ac10e8c0 2024-03-28T00:24:30.901 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:30.776+0000 7f18a3e00700 1 --2- 172.21.15.119:0/56211215 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f18ac074ee0 0x7f18ac1a0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T00:24:30.901 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:30.776+0000 7f18ab200700 1 --2- 172.21.15.119:0/56211215 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f18ac10e8c0 0x7f18ac19a8a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f18a400ef30 tx=0x7f18a400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:24:30.901 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:30.776+0000 7f18a3400700 1 -- 172.21.15.119:0/56211215 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18a40092e0 con 0x7f18ac10e8c0 2024-03-28T00:24:30.901 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:30.776+0000 7f18a3400700 1 -- 172.21.15.119:0/56211215 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f18a4004bb0 con 0x7f18ac10e8c0 2024-03-28T00:24:30.901 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:30.776+0000 7f18b0a00700 1 -- 172.21.15.119:0/56211215 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18ac19b290 con 0x7f18ac10e8c0 2024-03-28T00:24:30.901 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:30.776+0000 7f18a3400700 1 -- 172.21.15.119:0/56211215 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18a4004280 con 0x7f18ac10e8c0 2024-03-28T00:24:30.901 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:30.777+0000 7f18b0a00700 1 -- 172.21.15.119:0/56211215 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18ac1a75b0 con 0x7f18ac10e8c0 2024-03-28T00:24:30.901 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:30.777+0000 7f18b0a00700 1 -- 172.21.15.119:0/56211215 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f18ac062920 con 0x7f18ac10e8c0 2024-03-28T00:24:30.901 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:30.780+0000 7f18a3400700 1 -- 172.21.15.119:0/56211215 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 17) v1 ==== 137342+0+0 (secure 0 0 0) 0x7f18a401b030 con 0x7f18ac10e8c0 2024-03-28T00:24:30.901 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:30.781+0000 7f18a3400700 1 --2- 172.21.15.119:0/56211215 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f18900a2920 0x7f18900a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:24:30.901 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:30.781+0000 7f18a3400700 1 -- 172.21.15.119:0/56211215 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f18a40cefa0 con 0x7f18ac10e8c0 2024-03-28T00:24:30.901 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:30.781+0000 7f18aa800700 1 --2- 172.21.15.119:0/56211215 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f18900a2920 0x7f18900a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:24:30.902 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:30.781+0000 7f18a3400700 1 --2- 172.21.15.119:0/56211215 >> [v2:172.21.15.165:6800/2361434832,v1:172.21.15.165:6801/2361434832] conn(0x7f18900ac360 0x7f18900ae800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:24:30.902 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:30.781+0000 7f18a3400700 1 -- 172.21.15.119:0/56211215 --> [v2:172.21.15.165:6800/2361434832,v1:172.21.15.165:6801/2361434832] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f18a401fe50 con 0x7f18900ac360 2024-03-28T00:24:30.902 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:30.781+0000 7f18a3400700 1 -- 172.21.15.119:0/56211215 <== mon.1 v2:172.21.15.165:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) v2 ==== 24+0+0 (secure 0 0 0) 0x7f18a40cf350 con 0x7f18ac10e8c0 2024-03-28T00:24:30.902 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:30.782+0000 7f18a3e00700 1 --2- 172.21.15.119:0/56211215 >> [v2:172.21.15.165:6800/2361434832,v1:172.21.15.165:6801/2361434832] conn(0x7f18900ac360 0x7f18900ae800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:24:30.902 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:30.782+0000 7f18aa800700 1 --2- 172.21.15.119:0/56211215 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f18900a2920 0x7f18900a4de0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f18ac19b9e0 tx=0x7f189c00b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:24:30.902 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:30.782+0000 7f18a3e00700 1 --2- 172.21.15.119:0/56211215 >> [v2:172.21.15.165:6800/2361434832,v1:172.21.15.165:6801/2361434832] conn(0x7f18900ac360 0x7f18900ae800 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:24:30.903 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:30.783+0000 7f18a3400700 1 -- 172.21.15.119:0/56211215 <== osd.2 v2:172.21.15.165:6800/2361434832 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f18a401fe50 con 0x7f18900ac360 2024-03-28T00:24:30.903 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:30.802+0000 7f18b0a00700 1 -- 172.21.15.119:0/56211215 --> [v2:172.21.15.165:6800/2361434832,v1:172.21.15.165:6801/2361434832] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f18ac062da0 con 0x7f18900ac360 2024-03-28T00:24:30.903 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:30.802+0000 7f18a3400700 1 -- 172.21.15.119:0/56211215 <== osd.2 v2:172.21.15.165:6800/2361434832 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f18ac062da0 con 0x7f18900ac360 2024-03-28T00:24:30.903 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:30.803+0000 7f18b0a00700 1 -- 172.21.15.119:0/56211215 >> [v2:172.21.15.165:6800/2361434832,v1:172.21.15.165:6801/2361434832] conn(0x7f18900ac360 msgr2=0x7f18900ae800 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:24:30.903 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:30.803+0000 7f18b0a00700 1 --2- 172.21.15.119:0/56211215 >> [v2:172.21.15.165:6800/2361434832,v1:172.21.15.165:6801/2361434832] conn(0x7f18900ac360 0x7f18900ae800 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:30.903 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:30.803+0000 7f18b0a00700 1 -- 172.21.15.119:0/56211215 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f18900a2920 msgr2=0x7f18900a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:24:30.903 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:30.803+0000 7f18b0a00700 1 --2- 172.21.15.119:0/56211215 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f18900a2920 0x7f18900a4de0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f18ac19b9e0 tx=0x7f189c00b260 comp rx=0 tx=0).stop 2024-03-28T00:24:30.903 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:30.803+0000 7f18b0a00700 1 -- 172.21.15.119:0/56211215 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f18ac10e8c0 msgr2=0x7f18ac19a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:24:30.903 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:30.803+0000 7f18b0a00700 1 --2- 172.21.15.119:0/56211215 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f18ac10e8c0 0x7f18ac19a8a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f18a400ef30 tx=0x7f18a400c3a0 comp rx=0 tx=0).stop 2024-03-28T00:24:30.903 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:30.803+0000 7f18ab200700 1 -- 172.21.15.119:0/56211215 reap_dead start 2024-03-28T00:24:30.903 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:30.803+0000 7f18b0a00700 1 -- 172.21.15.119:0/56211215 shutdown_connections 2024-03-28T00:24:30.903 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:30.803+0000 7f18b0a00700 1 -- 172.21.15.119:0/56211215 >> 172.21.15.119:0/56211215 conn(0x7f18ac0fdb90 msgr2=0x7f18ac0fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:24:30.903 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:30.804+0000 7f18b0a00700 1 -- 172.21.15.119:0/56211215 shutdown_connections 2024-03-28T00:24:30.904 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:30.804+0000 7f18b0a00700 1 -- 172.21.15.119:0/56211215 wait complete. 2024-03-28T00:24:31.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:24:30 smithi119 ceph-mon[31518]: pgmap v330: 97 pgs: 97 active+clean; 645 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 426 B/s wr, 11 op/s 2024-03-28T00:24:31.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:24:30 smithi132 ceph-mon[35941]: pgmap v330: 97 pgs: 97 active+clean; 645 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 426 B/s wr, 11 op/s 2024-03-28T00:24:33.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:24:32 smithi165 ceph-mon[36302]: pgmap v331: 97 pgs: 97 active+clean; 645 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 426 B/s wr, 9 op/s 2024-03-28T00:24:33.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:24:32 smithi119 ceph-mon[31518]: pgmap v331: 97 pgs: 97 active+clean; 645 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 426 B/s wr, 9 op/s 2024-03-28T00:24:33.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:24:32 smithi132 ceph-mon[35941]: pgmap v331: 97 pgs: 97 active+clean; 645 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 426 B/s wr, 9 op/s 2024-03-28T00:24:34.412 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:33.714+0000 7f0415a00700 1 -- 172.21.15.119:0/2455619585 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0410074ce0 msgr2=0x7f041010e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:24:34.412 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:33.714+0000 7f0415a00700 1 --2- 172.21.15.119:0/2455619585 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0410074ce0 0x7f041010e650 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f03f80098f0 tx=0x7f03f8009c00 comp rx=0 tx=0).stop 2024-03-28T00:24:34.412 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:33.714+0000 7f0415a00700 1 -- 172.21.15.119:0/2455619585 shutdown_connections 2024-03-28T00:24:34.412 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:33.714+0000 7f0415a00700 1 --2- 172.21.15.119:0/2455619585 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f041010eb90 0x7f0410110f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:34.412 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:33.714+0000 7f0415a00700 1 --2- 172.21.15.119:0/2455619585 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f04100743c0 0x7f04100747a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:34.413 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:33.714+0000 7f0415a00700 1 --2- 172.21.15.119:0/2455619585 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0410074ce0 0x7f041010e650 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:34.413 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:33.714+0000 7f0415a00700 1 -- 172.21.15.119:0/2455619585 >> 172.21.15.119:0/2455619585 conn(0x7f04100792f0 msgr2=0x7f0410079700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:24:34.413 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:33.714+0000 7f0415a00700 1 -- 172.21.15.119:0/2455619585 shutdown_connections 2024-03-28T00:24:34.413 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:33.714+0000 7f0415a00700 1 -- 172.21.15.119:0/2455619585 wait complete. 2024-03-28T00:24:34.413 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:33.715+0000 7f0415a00700 1 Processor -- start 2024-03-28T00:24:34.413 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:33.715+0000 7f0415a00700 1 -- start start 2024-03-28T00:24:34.413 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:33.715+0000 7f0415a00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f04100743c0 0x7f04101a0860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:24:34.413 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:33.715+0000 7f0415a00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0410074ce0 0x7f04101a0da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:24:34.413 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:33.716+0000 7f0415a00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f041010eb90 0x7f041019aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:24:34.413 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:33.716+0000 7f0415a00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04101147a0 con 0x7f04100743c0 2024-03-28T00:24:34.413 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:33.716+0000 7f0415a00700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0410114620 con 0x7f041010eb90 2024-03-28T00:24:34.413 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:33.716+0000 7f0415a00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0410114920 con 0x7f0410074ce0 2024-03-28T00:24:34.413 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:33.716+0000 7f0407e00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0410074ce0 0x7f04101a0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:24:34.413 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:33.716+0000 7f0407e00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0410074ce0 0x7f04101a0da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.119:54546/0 (socket says 172.21.15.119:54546) 2024-03-28T00:24:34.414 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:33.716+0000 7f0407e00700 1 -- 172.21.15.119:0/4104682329 learned_addr learned my addr 172.21.15.119:0/4104682329 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:24:34.414 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:33.716+0000 7f040ea00700 1 --2- 172.21.15.119:0/4104682329 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f04100743c0 0x7f04101a0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:24:34.414 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:33.716+0000 7f040f400700 1 --2- 172.21.15.119:0/4104682329 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f041010eb90 0x7f041019aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:24:34.414 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:33.716+0000 7f0407e00700 1 -- 172.21.15.119:0/4104682329 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f041010eb90 msgr2=0x7f041019aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:24:34.414 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:33.716+0000 7f0407e00700 1 --2- 172.21.15.119:0/4104682329 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f041010eb90 0x7f041019aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:34.414 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:33.716+0000 7f0407e00700 1 -- 172.21.15.119:0/4104682329 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f04100743c0 msgr2=0x7f04101a0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:24:34.414 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:33.716+0000 7f0407e00700 1 --2- 172.21.15.119:0/4104682329 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f04100743c0 0x7f04101a0860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:34.414 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:33.716+0000 7f0407e00700 1 -- 172.21.15.119:0/4104682329 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f041019b180 con 0x7f0410074ce0 2024-03-28T00:24:34.414 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:33.716+0000 7f0407e00700 1 --2- 172.21.15.119:0/4104682329 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0410074ce0 0x7f04101a0da0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f03f8005950 tx=0x7f03f800fb20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:24:34.414 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:33.717+0000 7f0407400700 1 -- 172.21.15.119:0/4104682329 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03f8005140 con 0x7f0410074ce0 2024-03-28T00:24:34.414 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:33.717+0000 7f0407400700 1 -- 172.21.15.119:0/4104682329 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f03f800b560 con 0x7f0410074ce0 2024-03-28T00:24:34.414 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:33.717+0000 7f0407400700 1 -- 172.21.15.119:0/4104682329 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03f8003ae0 con 0x7f0410074ce0 2024-03-28T00:24:34.414 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:33.717+0000 7f0415a00700 1 -- 172.21.15.119:0/4104682329 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f041019b410 con 0x7f0410074ce0 2024-03-28T00:24:34.414 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:33.717+0000 7f0415a00700 1 -- 172.21.15.119:0/4104682329 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04101a7730 con 0x7f0410074ce0 2024-03-28T00:24:34.415 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:33.718+0000 7f0415a00700 1 -- 172.21.15.119:0/4104682329 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f04100628a0 con 0x7f0410074ce0 2024-03-28T00:24:34.415 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:33.720+0000 7f0407400700 1 -- 172.21.15.119:0/4104682329 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 17) v1 ==== 137342+0+0 (secure 0 0 0) 0x7f03f800ba80 con 0x7f0410074ce0 2024-03-28T00:24:34.415 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:33.721+0000 7f0407400700 1 --2- 172.21.15.119:0/4104682329 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f03f40a2920 0x7f03f40a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:24:34.415 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:33.721+0000 7f0407400700 1 -- 172.21.15.119:0/4104682329 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f03f80cff20 con 0x7f0410074ce0 2024-03-28T00:24:34.415 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:33.721+0000 7f040ea00700 1 --2- 172.21.15.119:0/4104682329 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f03f40a2920 0x7f03f40a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:24:34.415 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:33.722+0000 7f0407400700 1 --2- 172.21.15.119:0/4104682329 >> [v2:172.21.15.165:6816/4115226150,v1:172.21.15.165:6817/4115226150] conn(0x7f03f40ac230 0x7f03f40ae6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:24:34.415 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:33.722+0000 7f0407400700 1 -- 172.21.15.119:0/4104682329 --> [v2:172.21.15.165:6816/4115226150,v1:172.21.15.165:6817/4115226150] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f03f800b0b0 con 0x7f03f40ac230 2024-03-28T00:24:34.415 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:33.722+0000 7f0407400700 1 -- 172.21.15.119:0/4104682329 <== mon.1 v2:172.21.15.165:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) v2 ==== 24+0+0 (secure 0 0 0) 0x7f03f80d02d0 con 0x7f0410074ce0 2024-03-28T00:24:34.415 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:33.722+0000 7f040f400700 1 --2- 172.21.15.119:0/4104682329 >> [v2:172.21.15.165:6816/4115226150,v1:172.21.15.165:6817/4115226150] conn(0x7f03f40ac230 0x7f03f40ae6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:24:34.415 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:33.722+0000 7f040ea00700 1 --2- 172.21.15.119:0/4104682329 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f03f40a2920 0x7f03f40a4de0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f041019bb60 tx=0x7f0400009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:24:34.415 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:33.722+0000 7f040f400700 1 --2- 172.21.15.119:0/4104682329 >> [v2:172.21.15.165:6816/4115226150,v1:172.21.15.165:6817/4115226150] conn(0x7f03f40ac230 0x7f03f40ae6d0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:24:34.415 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:33.723+0000 7f0407400700 1 -- 172.21.15.119:0/4104682329 <== osd.8 v2:172.21.15.165:6816/4115226150 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f03f800b0b0 con 0x7f03f40ac230 2024-03-28T00:24:34.416 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:33.742+0000 7f0415a00700 1 -- 172.21.15.119:0/4104682329 --> [v2:172.21.15.165:6816/4115226150,v1:172.21.15.165:6817/4115226150] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f0410062ca0 con 0x7f03f40ac230 2024-03-28T00:24:34.416 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:33.742+0000 7f0407400700 1 -- 172.21.15.119:0/4104682329 <== osd.8 v2:172.21.15.165:6816/4115226150 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f0410062ca0 con 0x7f03f40ac230 2024-03-28T00:24:34.416 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:33.743+0000 7f0415a00700 1 -- 172.21.15.119:0/4104682329 >> [v2:172.21.15.165:6816/4115226150,v1:172.21.15.165:6817/4115226150] conn(0x7f03f40ac230 msgr2=0x7f03f40ae6d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:24:34.416 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:33.743+0000 7f0415a00700 1 --2- 172.21.15.119:0/4104682329 >> [v2:172.21.15.165:6816/4115226150,v1:172.21.15.165:6817/4115226150] conn(0x7f03f40ac230 0x7f03f40ae6d0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:34.416 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:33.743+0000 7f0415a00700 1 -- 172.21.15.119:0/4104682329 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f03f40a2920 msgr2=0x7f03f40a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:24:34.416 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:33.743+0000 7f0415a00700 1 --2- 172.21.15.119:0/4104682329 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f03f40a2920 0x7f03f40a4de0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f041019bb60 tx=0x7f0400009310 comp rx=0 tx=0).stop 2024-03-28T00:24:34.417 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:33.743+0000 7f0415a00700 1 -- 172.21.15.119:0/4104682329 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0410074ce0 msgr2=0x7f04101a0da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:24:34.417 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:33.743+0000 7f0415a00700 1 --2- 172.21.15.119:0/4104682329 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0410074ce0 0x7f04101a0da0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f03f8005950 tx=0x7f03f800fb20 comp rx=0 tx=0).stop 2024-03-28T00:24:34.417 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:33.743+0000 7f040f400700 1 -- 172.21.15.119:0/4104682329 reap_dead start 2024-03-28T00:24:34.417 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:33.743+0000 7f0415a00700 1 -- 172.21.15.119:0/4104682329 shutdown_connections 2024-03-28T00:24:34.417 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:33.743+0000 7f0415a00700 1 -- 172.21.15.119:0/4104682329 >> 172.21.15.119:0/4104682329 conn(0x7f04100792f0 msgr2=0x7f04100ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:24:34.417 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:33.743+0000 7f0415a00700 1 -- 172.21.15.119:0/4104682329 shutdown_connections 2024-03-28T00:24:34.417 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:33.743+0000 7f0415a00700 1 -- 172.21.15.119:0/4104682329 wait complete. 2024-03-28T00:24:34.999 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:24:34 smithi119 ceph-mon[31518]: pgmap v332: 97 pgs: 97 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 597 B/s wr, 12 op/s 2024-03-28T00:24:35.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:24:34 smithi165 ceph-mon[36302]: pgmap v332: 97 pgs: 97 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 597 B/s wr, 12 op/s 2024-03-28T00:24:35.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:24:34 smithi132 ceph-mon[35941]: pgmap v332: 97 pgs: 97 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 597 B/s wr, 12 op/s 2024-03-28T00:24:35.794 INFO:teuthology.orchestra.run.smithi119.stdout:77309411400 2024-03-28T00:24:35.794 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 10f73b92-ec98-11ee-95d0-87774f69a715 -- ceph osd last-stat-seq osd.2 2024-03-28T00:24:36.523 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:36.516+0000 7f7664a00700 1 -- 172.21.15.119:0/2568101291 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f76601095c0 msgr2=0x7f7660111190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:24:36.523 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:36.516+0000 7f7664a00700 1 --2- 172.21.15.119:0/2568101291 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f76601095c0 0x7f7660111190 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f7658009f90 tx=0x7f765800b330 comp rx=0 tx=0).stop 2024-03-28T00:24:36.523 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:36.516+0000 7f7664a00700 1 -- 172.21.15.119:0/2568101291 shutdown_connections 2024-03-28T00:24:36.524 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:36.516+0000 7f7664a00700 1 --2- 172.21.15.119:0/2568101291 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f76601095c0 0x7f7660111190 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:36.524 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:36.516+0000 7f7664a00700 1 --2- 172.21.15.119:0/2568101291 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f76601044e0 0x7f76601048c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:36.524 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:36.516+0000 7f7664a00700 1 --2- 172.21.15.119:0/2568101291 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7660104e90 0x7f7660108ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:36.524 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:36.516+0000 7f7664a00700 1 -- 172.21.15.119:0/2568101291 >> 172.21.15.119:0/2568101291 conn(0x7f76600ffdb0 msgr2=0x7f76601021d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:24:36.524 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:36.517+0000 7f7664a00700 1 -- 172.21.15.119:0/2568101291 shutdown_connections 2024-03-28T00:24:36.524 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:36.517+0000 7f7664a00700 1 -- 172.21.15.119:0/2568101291 wait complete. 2024-03-28T00:24:36.526 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:36.518+0000 7f7664a00700 1 Processor -- start 2024-03-28T00:24:36.526 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:36.518+0000 7f7664a00700 1 -- start start 2024-03-28T00:24:36.526 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:36.518+0000 7f7664a00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f76601044e0 0x7f76601a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:24:36.526 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:36.518+0000 7f7664a00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7660104e90 0x7f76601a0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:24:36.526 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:36.518+0000 7f7664a00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f76601095c0 0x7f766019a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:24:36.526 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:36.518+0000 7f7664a00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7660114600 con 0x7f76601095c0 2024-03-28T00:24:36.526 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:36.518+0000 7f7664a00700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7660114480 con 0x7f7660104e90 2024-03-28T00:24:36.526 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:36.518+0000 7f7664a00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7660114780 con 0x7f76601044e0 2024-03-28T00:24:36.527 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:36.518+0000 7f765e800700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f76601044e0 0x7f76601a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:24:36.527 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:36.518+0000 7f765e800700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f76601044e0 0x7f76601a0650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.119:49972/0 (socket says 172.21.15.119:49972) 2024-03-28T00:24:36.527 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:36.518+0000 7f765e800700 1 -- 172.21.15.119:0/319373391 learned_addr learned my addr 172.21.15.119:0/319373391 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:24:36.527 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:36.518+0000 7f765de00700 1 --2- 172.21.15.119:0/319373391 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7660104e90 0x7f76601a0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:24:36.527 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:36.518+0000 7f765f200700 1 --2- 172.21.15.119:0/319373391 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f76601095c0 0x7f766019a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:24:36.527 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:36.519+0000 7f765e800700 1 -- 172.21.15.119:0/319373391 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7660104e90 msgr2=0x7f76601a0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:24:36.527 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:36.519+0000 7f765e800700 1 --2- 172.21.15.119:0/319373391 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7660104e90 0x7f76601a0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:36.527 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:36.519+0000 7f765e800700 1 -- 172.21.15.119:0/319373391 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f76601095c0 msgr2=0x7f766019a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:24:36.527 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:36.519+0000 7f765e800700 1 --2- 172.21.15.119:0/319373391 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f76601095c0 0x7f766019a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:36.527 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:36.519+0000 7f765e800700 1 -- 172.21.15.119:0/319373391 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f766019b000 con 0x7f76601044e0 2024-03-28T00:24:36.527 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:36.519+0000 7f765de00700 1 --2- 172.21.15.119:0/319373391 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7660104e90 0x7f76601a0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T00:24:36.527 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:36.519+0000 7f765e800700 1 --2- 172.21.15.119:0/319373391 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f76601044e0 0x7f76601a0650 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f765000ec10 tx=0x7f765000ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:24:36.527 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:36.519+0000 7f7657400700 1 -- 172.21.15.119:0/319373391 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f765000cc90 con 0x7f76601044e0 2024-03-28T00:24:36.528 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:36.519+0000 7f7657400700 1 -- 172.21.15.119:0/319373391 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f765000ce30 con 0x7f76601044e0 2024-03-28T00:24:36.528 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:36.520+0000 7f7657400700 1 -- 172.21.15.119:0/319373391 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7650010690 con 0x7f76601044e0 2024-03-28T00:24:36.528 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:36.520+0000 7f7664a00700 1 -- 172.21.15.119:0/319373391 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f766019b2f0 con 0x7f76601044e0 2024-03-28T00:24:36.528 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:36.520+0000 7f7664a00700 1 -- 172.21.15.119:0/319373391 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76601a74b0 con 0x7f76601044e0 2024-03-28T00:24:36.528 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:36.521+0000 7f7664a00700 1 -- 172.21.15.119:0/319373391 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f7660101250 con 0x7f76601044e0 2024-03-28T00:24:36.529 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:36.523+0000 7f7657400700 1 -- 172.21.15.119:0/319373391 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 17) v1 ==== 137342+0+0 (secure 0 0 0) 0x7f7650003680 con 0x7f76601044e0 2024-03-28T00:24:36.530 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:36.524+0000 7f7657400700 1 --2- 172.21.15.119:0/319373391 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f76440a29a0 0x7f76440a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:24:36.530 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:36.524+0000 7f7657400700 1 -- 172.21.15.119:0/319373391 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f76500cf190 con 0x7f76601044e0 2024-03-28T00:24:36.531 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:36.524+0000 7f765de00700 1 --2- 172.21.15.119:0/319373391 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f76440a29a0 0x7f76440a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:24:36.531 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:36.525+0000 7f7657400700 1 --2- 172.21.15.119:0/319373391 >> [v2:172.21.15.119:6802/580396252,v1:172.21.15.119:6803/580396252] conn(0x7f76440ac3e0 0x7f76440ae880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:24:36.531 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:36.525+0000 7f7657400700 1 -- 172.21.15.119:0/319373391 --> [v2:172.21.15.119:6802/580396252,v1:172.21.15.119:6803/580396252] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7650012ec0 con 0x7f76440ac3e0 2024-03-28T00:24:36.532 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:36.525+0000 7f7657400700 1 -- 172.21.15.119:0/319373391 <== mon.1 v2:172.21.15.165:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) v2 ==== 24+0+0 (secure 0 0 0) 0x7f76500cf540 con 0x7f76601044e0 2024-03-28T00:24:36.532 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:36.525+0000 7f765f200700 1 --2- 172.21.15.119:0/319373391 >> [v2:172.21.15.119:6802/580396252,v1:172.21.15.119:6803/580396252] conn(0x7f76440ac3e0 0x7f76440ae880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:24:36.532 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:36.525+0000 7f765de00700 1 --2- 172.21.15.119:0/319373391 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f76440a29a0 0x7f76440a4e60 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f7660103450 tx=0x7f764c0073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:24:36.532 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:36.526+0000 7f765f200700 1 --2- 172.21.15.119:0/319373391 >> [v2:172.21.15.119:6802/580396252,v1:172.21.15.119:6803/580396252] conn(0x7f76440ac3e0 0x7f76440ae880 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:24:36.532 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:36.527+0000 7f7657400700 1 -- 172.21.15.119:0/319373391 <== osd.0 v2:172.21.15.119:6802/580396252 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f7650012ec0 con 0x7f76440ac3e0 2024-03-28T00:24:36.550 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:36.545+0000 7f7664a00700 1 -- 172.21.15.119:0/319373391 --> [v2:172.21.15.119:6802/580396252,v1:172.21.15.119:6803/580396252] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f76601059c0 con 0x7f76440ac3e0 2024-03-28T00:24:36.551 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:36.545+0000 7f7657400700 1 -- 172.21.15.119:0/319373391 <== osd.0 v2:172.21.15.119:6802/580396252 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f76601059c0 con 0x7f76440ac3e0 2024-03-28T00:24:36.552 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:36.545+0000 7f7664a00700 1 -- 172.21.15.119:0/319373391 >> [v2:172.21.15.119:6802/580396252,v1:172.21.15.119:6803/580396252] conn(0x7f76440ac3e0 msgr2=0x7f76440ae880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:24:36.552 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:36.546+0000 7f7664a00700 1 --2- 172.21.15.119:0/319373391 >> [v2:172.21.15.119:6802/580396252,v1:172.21.15.119:6803/580396252] conn(0x7f76440ac3e0 0x7f76440ae880 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:36.552 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:36.546+0000 7f7664a00700 1 -- 172.21.15.119:0/319373391 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f76440a29a0 msgr2=0x7f76440a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:24:36.552 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:36.546+0000 7f7664a00700 1 --2- 172.21.15.119:0/319373391 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f76440a29a0 0x7f76440a4e60 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f7660103450 tx=0x7f764c0073d0 comp rx=0 tx=0).stop 2024-03-28T00:24:36.552 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:36.546+0000 7f7664a00700 1 -- 172.21.15.119:0/319373391 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f76601044e0 msgr2=0x7f76601a0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:24:36.553 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:36.546+0000 7f7664a00700 1 --2- 172.21.15.119:0/319373391 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f76601044e0 0x7f76601a0650 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f765000ec10 tx=0x7f765000ef20 comp rx=0 tx=0).stop 2024-03-28T00:24:36.553 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:36.546+0000 7f765f200700 1 -- 172.21.15.119:0/319373391 reap_dead start 2024-03-28T00:24:36.553 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:36.546+0000 7f7664a00700 1 -- 172.21.15.119:0/319373391 shutdown_connections 2024-03-28T00:24:36.553 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:36.546+0000 7f7664a00700 1 -- 172.21.15.119:0/319373391 >> 172.21.15.119:0/319373391 conn(0x7f76600ffdb0 msgr2=0x7f76601007f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:24:36.553 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:36.546+0000 7f7664a00700 1 -- 172.21.15.119:0/319373391 shutdown_connections 2024-03-28T00:24:36.553 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:36.546+0000 7f7664a00700 1 -- 172.21.15.119:0/319373391 wait complete. 2024-03-28T00:24:37.030 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:24:36 smithi119 ceph-mon[31518]: pgmap v333: 97 pgs: 97 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 8 op/s 2024-03-28T00:24:37.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:24:36 smithi165 ceph-mon[36302]: pgmap v333: 97 pgs: 97 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 8 op/s 2024-03-28T00:24:37.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:24:36 smithi132 ceph-mon[35941]: pgmap v333: 97 pgs: 97 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 8 op/s 2024-03-28T00:24:38.835 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:38.828+0000 7eff93200700 1 -- 172.21.15.119:0/2803240124 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7eff9410e920 msgr2=0x7eff94110d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:24:38.835 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:38.828+0000 7eff93200700 1 --2- 172.21.15.119:0/2803240124 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7eff9410e920 0x7eff94110d30 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7eff8c00b0a0 tx=0x7eff8c00b3b0 comp rx=0 tx=0).stop 2024-03-28T00:24:38.835 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:38.828+0000 7eff93200700 1 -- 172.21.15.119:0/2803240124 shutdown_connections 2024-03-28T00:24:38.835 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:38.828+0000 7eff93200700 1 --2- 172.21.15.119:0/2803240124 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7eff9410e920 0x7eff94110d30 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:38.835 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:38.828+0000 7eff93200700 1 --2- 172.21.15.119:0/2803240124 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7eff94074560 0x7eff94074940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:38.835 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:38.828+0000 7eff93200700 1 --2- 172.21.15.119:0/2803240124 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7eff94074e80 0x7eff9410e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:38.835 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:38.828+0000 7eff93200700 1 -- 172.21.15.119:0/2803240124 >> 172.21.15.119:0/2803240124 conn(0x7eff940fdb90 msgr2=0x7eff940fffb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:24:38.835 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:38.828+0000 7eff93200700 1 -- 172.21.15.119:0/2803240124 shutdown_connections 2024-03-28T00:24:38.835 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:38.828+0000 7eff93200700 1 -- 172.21.15.119:0/2803240124 wait complete. 2024-03-28T00:24:38.836 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:38.829+0000 7eff93200700 1 Processor -- start 2024-03-28T00:24:38.836 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:38.830+0000 7eff93200700 1 -- start start 2024-03-28T00:24:38.836 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:38.830+0000 7eff93200700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7eff94074560 0x7eff941a06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:24:38.836 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:38.830+0000 7eff93200700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7eff94074e80 0x7eff941a0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:24:38.836 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:38.830+0000 7eff93200700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7eff9410e920 0x7eff9419a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:24:38.836 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:38.830+0000 7eff93200700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff941145e0 con 0x7eff94074e80 2024-03-28T00:24:38.836 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:38.830+0000 7eff93200700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff94114460 con 0x7eff9410e920 2024-03-28T00:24:38.837 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:38.830+0000 7eff93200700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff94114760 con 0x7eff94074560 2024-03-28T00:24:38.838 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:38.830+0000 7eff91e00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7eff94074560 0x7eff941a06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:24:38.838 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:38.830+0000 7eff91e00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7eff94074560 0x7eff941a06a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.119:49988/0 (socket says 172.21.15.119:49988) 2024-03-28T00:24:38.838 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:38.830+0000 7eff91e00700 1 -- 172.21.15.119:0/3331676931 learned_addr learned my addr 172.21.15.119:0/3331676931 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:24:38.838 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:38.830+0000 7eff92800700 1 --2- 172.21.15.119:0/3331676931 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7eff9410e920 0x7eff9419a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:24:38.838 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:38.830+0000 7eff91400700 1 --2- 172.21.15.119:0/3331676931 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7eff94074e80 0x7eff941a0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:24:38.838 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:38.831+0000 7eff92800700 1 -- 172.21.15.119:0/3331676931 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7eff94074560 msgr2=0x7eff941a06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:24:38.838 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:38.831+0000 7eff92800700 1 --2- 172.21.15.119:0/3331676931 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7eff94074560 0x7eff941a06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:38.838 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:38.831+0000 7eff92800700 1 -- 172.21.15.119:0/3331676931 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7eff94074e80 msgr2=0x7eff941a0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:24:38.838 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:38.831+0000 7eff92800700 1 --2- 172.21.15.119:0/3331676931 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7eff94074e80 0x7eff941a0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:38.838 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:38.831+0000 7eff92800700 1 -- 172.21.15.119:0/3331676931 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff9419afc0 con 0x7eff9410e920 2024-03-28T00:24:38.838 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:38.831+0000 7eff92800700 1 --2- 172.21.15.119:0/3331676931 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7eff9410e920 0x7eff9419a860 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7eff8c00b070 tx=0x7eff8c004c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:24:38.838 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:38.831+0000 7eff8aa00700 1 -- 172.21.15.119:0/3331676931 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff8c021070 con 0x7eff9410e920 2024-03-28T00:24:38.839 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:38.831+0000 7eff8aa00700 1 -- 172.21.15.119:0/3331676931 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7eff8c007b80 con 0x7eff9410e920 2024-03-28T00:24:38.839 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:38.831+0000 7eff8aa00700 1 -- 172.21.15.119:0/3331676931 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff8c005040 con 0x7eff9410e920 2024-03-28T00:24:38.839 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:38.832+0000 7eff93200700 1 -- 172.21.15.119:0/3331676931 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff9419b250 con 0x7eff9410e920 2024-03-28T00:24:38.839 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:38.832+0000 7eff93200700 1 -- 172.21.15.119:0/3331676931 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff941a7570 con 0x7eff9410e920 2024-03-28T00:24:38.839 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:38.833+0000 7eff93200700 1 -- 172.21.15.119:0/3331676931 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7eff94062920 con 0x7eff9410e920 2024-03-28T00:24:38.841 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:38.835+0000 7eff8aa00700 1 -- 172.21.15.119:0/3331676931 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 17) v1 ==== 137342+0+0 (secure 0 0 0) 0x7eff8c024070 con 0x7eff9410e920 2024-03-28T00:24:38.842 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:38.836+0000 7eff8aa00700 1 --2- 172.21.15.119:0/3331676931 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7eff780a29a0 0x7eff780a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:24:38.843 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:38.836+0000 7eff8aa00700 1 -- 172.21.15.119:0/3331676931 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8153+0+0 (secure 0 0 0) 0x7eff8c00f020 con 0x7eff9410e920 2024-03-28T00:24:38.843 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:38.837+0000 7eff91e00700 1 --2- 172.21.15.119:0/3331676931 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7eff780a29a0 0x7eff780a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:24:38.843 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:38.837+0000 7eff8aa00700 1 --2- 172.21.15.119:0/3331676931 >> [v2:172.21.15.165:6824/2138771601,v1:172.21.15.165:6825/2138771601] conn(0x7eff780ac3e0 0x7eff780ae880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:24:38.843 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:38.837+0000 7eff8aa00700 1 -- 172.21.15.119:0/3331676931 --> [v2:172.21.15.165:6824/2138771601,v1:172.21.15.165:6825/2138771601] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7eff8c0020b0 con 0x7eff780ac3e0 2024-03-28T00:24:38.843 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:38.837+0000 7eff8aa00700 1 -- 172.21.15.119:0/3331676931 <== mon.2 v2:172.21.15.132:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) v2 ==== 24+0+0 (secure 0 0 0) 0x7eff8c0d0490 con 0x7eff9410e920 2024-03-28T00:24:38.843 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:38.837+0000 7eff91400700 1 --2- 172.21.15.119:0/3331676931 >> [v2:172.21.15.165:6824/2138771601,v1:172.21.15.165:6825/2138771601] conn(0x7eff780ac3e0 0x7eff780ae880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:24:38.844 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:38.837+0000 7eff91e00700 1 --2- 172.21.15.119:0/3331676931 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7eff780a29a0 0x7eff780a4e60 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7eff9419b9a0 tx=0x7eff84009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:24:38.844 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:38.837+0000 7eff91400700 1 --2- 172.21.15.119:0/3331676931 >> [v2:172.21.15.165:6824/2138771601,v1:172.21.15.165:6825/2138771601] conn(0x7eff780ac3e0 0x7eff780ae880 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:24:38.844 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:38.838+0000 7eff8aa00700 1 -- 172.21.15.119:0/3331676931 <== osd.11 v2:172.21.15.165:6824/2138771601 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7eff8c0020b0 con 0x7eff780ac3e0 2024-03-28T00:24:38.862 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:38.856+0000 7eff93200700 1 -- 172.21.15.119:0/3331676931 --> [v2:172.21.15.165:6824/2138771601,v1:172.21.15.165:6825/2138771601] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7eff94062df0 con 0x7eff780ac3e0 2024-03-28T00:24:38.863 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:38.857+0000 7eff8aa00700 1 -- 172.21.15.119:0/3331676931 <== osd.11 v2:172.21.15.165:6824/2138771601 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7eff94062df0 con 0x7eff780ac3e0 2024-03-28T00:24:38.864 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:38.857+0000 7eff93200700 1 -- 172.21.15.119:0/3331676931 >> [v2:172.21.15.165:6824/2138771601,v1:172.21.15.165:6825/2138771601] conn(0x7eff780ac3e0 msgr2=0x7eff780ae880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:24:38.864 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:38.857+0000 7eff93200700 1 --2- 172.21.15.119:0/3331676931 >> [v2:172.21.15.165:6824/2138771601,v1:172.21.15.165:6825/2138771601] conn(0x7eff780ac3e0 0x7eff780ae880 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:38.864 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:38.858+0000 7eff93200700 1 -- 172.21.15.119:0/3331676931 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7eff780a29a0 msgr2=0x7eff780a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:24:38.864 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:38.858+0000 7eff93200700 1 --2- 172.21.15.119:0/3331676931 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7eff780a29a0 0x7eff780a4e60 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7eff9419b9a0 tx=0x7eff84009260 comp rx=0 tx=0).stop 2024-03-28T00:24:38.864 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:38.858+0000 7eff93200700 1 -- 172.21.15.119:0/3331676931 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7eff9410e920 msgr2=0x7eff9419a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:24:38.864 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:38.858+0000 7eff93200700 1 --2- 172.21.15.119:0/3331676931 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7eff9410e920 0x7eff9419a860 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7eff8c00b070 tx=0x7eff8c004c80 comp rx=0 tx=0).stop 2024-03-28T00:24:38.864 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:38.858+0000 7eff92800700 1 -- 172.21.15.119:0/3331676931 reap_dead start 2024-03-28T00:24:38.864 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:38.858+0000 7eff93200700 1 -- 172.21.15.119:0/3331676931 shutdown_connections 2024-03-28T00:24:38.865 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:38.858+0000 7eff93200700 1 -- 172.21.15.119:0/3331676931 >> 172.21.15.119:0/3331676931 conn(0x7eff940fdb90 msgr2=0x7eff940fe800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:24:38.865 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:38.858+0000 7eff93200700 1 -- 172.21.15.119:0/3331676931 shutdown_connections 2024-03-28T00:24:38.865 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:38.858+0000 7eff93200700 1 -- 172.21.15.119:0/3331676931 wait complete. 2024-03-28T00:24:38.922 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:24:38 smithi119 ceph-mon[31518]: pgmap v334: 97 pgs: 97 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s 2024-03-28T00:24:39.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:24:38 smithi165 ceph-mon[36302]: pgmap v334: 97 pgs: 97 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s 2024-03-28T00:24:39.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:24:38 smithi132 ceph-mon[35941]: pgmap v334: 97 pgs: 97 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s 2024-03-28T00:24:40.193 INFO:teuthology.orchestra.run.smithi119.stdout:253403070504 2024-03-28T00:24:40.194 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 10f73b92-ec98-11ee-95d0-87774f69a715 -- ceph osd last-stat-seq osd.8 2024-03-28T00:24:41.077 INFO:teuthology.orchestra.run.smithi119.stdout:343597383703 2024-03-28T00:24:41.078 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 10f73b92-ec98-11ee-95d0-87774f69a715 -- ceph osd last-stat-seq osd.11 2024-03-28T00:24:41.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:24:40 smithi165 ceph-mon[36302]: pgmap v335: 97 pgs: 97 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 170 B/s wr, 11 op/s 2024-03-28T00:24:41.139 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:24:40 smithi119 ceph-mon[31518]: pgmap v335: 97 pgs: 97 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 170 B/s wr, 11 op/s 2024-03-28T00:24:41.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:24:40 smithi132 ceph-mon[35941]: pgmap v335: 97 pgs: 97 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 170 B/s wr, 11 op/s 2024-03-28T00:24:41.302 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:41.296+0000 7f1a33200700 1 -- 172.21.15.119:0/1634074721 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1a2c06a930 msgr2=0x7f1a2c06ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:24:41.303 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:41.296+0000 7f1a33200700 1 --2- 172.21.15.119:0/1634074721 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1a2c06a930 0x7f1a2c06ad10 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f1a1c009990 tx=0x7f1a1c009ca0 comp rx=0 tx=0).stop 2024-03-28T00:24:41.303 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:41.296+0000 7f1a33200700 1 -- 172.21.15.119:0/1634074721 shutdown_connections 2024-03-28T00:24:41.303 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:41.296+0000 7f1a33200700 1 --2- 172.21.15.119:0/1634074721 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a2c107470 0x7f1a2c10f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:41.303 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:41.296+0000 7f1a33200700 1 --2- 172.21.15.119:0/1634074721 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1a2c06a930 0x7f1a2c06ad10 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:41.303 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:41.296+0000 7f1a33200700 1 --2- 172.21.15.119:0/1634074721 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1a2c06b2e0 0x7f1a2c106d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:41.303 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:41.296+0000 7f1a33200700 1 -- 172.21.15.119:0/1634074721 >> 172.21.15.119:0/1634074721 conn(0x7f1a2c0770e0 msgr2=0x7f1a2c0774f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:24:41.303 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:41.296+0000 7f1a33200700 1 -- 172.21.15.119:0/1634074721 shutdown_connections 2024-03-28T00:24:41.303 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:41.296+0000 7f1a33200700 1 -- 172.21.15.119:0/1634074721 wait complete. 2024-03-28T00:24:41.304 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:41.297+0000 7f1a33200700 1 Processor -- start 2024-03-28T00:24:41.304 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:41.298+0000 7f1a33200700 1 -- start start 2024-03-28T00:24:41.304 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:41.298+0000 7f1a33200700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a2c06a930 0x7f1a2c19e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:24:41.304 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:41.298+0000 7f1a33200700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1a2c06b2e0 0x7f1a2c19ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:24:41.304 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:41.298+0000 7f1a33200700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1a2c107470 0x7f1a2c198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:24:41.305 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:41.298+0000 7f1a33200700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a2c1124a0 con 0x7f1a2c06a930 2024-03-28T00:24:41.305 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:41.298+0000 7f1a33200700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a2c112320 con 0x7f1a2c107470 2024-03-28T00:24:41.305 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:41.298+0000 7f1a33200700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a2c112620 con 0x7f1a2c06b2e0 2024-03-28T00:24:41.306 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:41.298+0000 7f1a2b400700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1a2c06b2e0 0x7f1a2c19ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:24:41.306 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:41.298+0000 7f1a2b400700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1a2c06b2e0 0x7f1a2c19ebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.119:50014/0 (socket says 172.21.15.119:50014) 2024-03-28T00:24:41.306 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:41.298+0000 7f1a2b400700 1 -- 172.21.15.119:0/1637943275 learned_addr learned my addr 172.21.15.119:0/1637943275 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:24:41.306 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:41.298+0000 7f1a2be00700 1 --2- 172.21.15.119:0/1637943275 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a2c06a930 0x7f1a2c19e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:24:41.306 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:41.299+0000 7f1a31200700 1 --2- 172.21.15.119:0/1637943275 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1a2c107470 0x7f1a2c198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:24:41.306 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:41.299+0000 7f1a2b400700 1 -- 172.21.15.119:0/1637943275 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1a2c107470 msgr2=0x7f1a2c198740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:24:41.306 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:41.299+0000 7f1a2b400700 1 --2- 172.21.15.119:0/1637943275 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1a2c107470 0x7f1a2c198740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:41.306 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:41.299+0000 7f1a2b400700 1 -- 172.21.15.119:0/1637943275 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a2c06a930 msgr2=0x7f1a2c19e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:24:41.307 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:41.299+0000 7f1a2b400700 1 --2- 172.21.15.119:0/1637943275 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a2c06a930 0x7f1a2c19e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:41.307 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:41.299+0000 7f1a2b400700 1 -- 172.21.15.119:0/1637943275 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a2c199020 con 0x7f1a2c06b2e0 2024-03-28T00:24:41.307 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:41.299+0000 7f1a31200700 1 --2- 172.21.15.119:0/1637943275 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1a2c107470 0x7f1a2c198740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T00:24:41.307 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:41.299+0000 7f1a2b400700 1 --2- 172.21.15.119:0/1637943275 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1a2c06b2e0 0x7f1a2c19ebb0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f1a1800e970 tx=0x7f1a1800ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:24:41.307 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:41.299+0000 7f1a28c00700 1 -- 172.21.15.119:0/1637943275 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a1800cbd0 con 0x7f1a2c06b2e0 2024-03-28T00:24:41.307 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:41.299+0000 7f1a28c00700 1 -- 172.21.15.119:0/1637943275 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1a1800cd70 con 0x7f1a2c06b2e0 2024-03-28T00:24:41.307 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:41.299+0000 7f1a33200700 1 -- 172.21.15.119:0/1637943275 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a2c199310 con 0x7f1a2c06b2e0 2024-03-28T00:24:41.307 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:41.300+0000 7f1a28c00700 1 -- 172.21.15.119:0/1637943275 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a18010430 con 0x7f1a2c06b2e0 2024-03-28T00:24:41.307 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:41.300+0000 7f1a33200700 1 -- 172.21.15.119:0/1637943275 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a2c1a55f0 con 0x7f1a2c06b2e0 2024-03-28T00:24:41.308 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:41.300+0000 7f1a33200700 1 -- 172.21.15.119:0/1637943275 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1a2c0628e0 con 0x7f1a2c06b2e0 2024-03-28T00:24:41.310 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:41.304+0000 7f1a28c00700 1 -- 172.21.15.119:0/1637943275 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 17) v1 ==== 137342+0+0 (secure 0 0 0) 0x7f1a18004750 con 0x7f1a2c06b2e0 2024-03-28T00:24:41.311 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:41.305+0000 7f1a28c00700 1 --2- 172.21.15.119:0/1637943275 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f1a100a29a0 0x7f1a100a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:24:41.312 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:41.306+0000 7f1a28c00700 1 -- 172.21.15.119:0/1637943275 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f1a180ceed0 con 0x7f1a2c06b2e0 2024-03-28T00:24:41.312 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:41.306+0000 7f1a2be00700 1 --2- 172.21.15.119:0/1637943275 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f1a100a29a0 0x7f1a100a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:24:41.312 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:41.306+0000 7f1a28c00700 1 --2- 172.21.15.119:0/1637943275 >> [v2:172.21.15.132:6800/2500318888,v1:172.21.15.132:6801/2500318888] conn(0x7f1a100ac3e0 0x7f1a100ae880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:24:41.313 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:41.306+0000 7f1a28c00700 1 -- 172.21.15.119:0/1637943275 --> [v2:172.21.15.132:6800/2500318888,v1:172.21.15.132:6801/2500318888] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1a100aef50 con 0x7f1a100ac3e0 2024-03-28T00:24:41.313 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:41.306+0000 7f1a28c00700 1 -- 172.21.15.119:0/1637943275 <== mon.1 v2:172.21.15.165:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1a180d14f0 con 0x7f1a2c06b2e0 2024-03-28T00:24:41.313 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:41.306+0000 7f1a31200700 1 --2- 172.21.15.119:0/1637943275 >> [v2:172.21.15.132:6800/2500318888,v1:172.21.15.132:6801/2500318888] conn(0x7f1a100ac3e0 0x7f1a100ae880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:24:41.313 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:41.307+0000 7f1a2be00700 1 --2- 172.21.15.119:0/1637943275 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f1a100a29a0 0x7f1a100a4e60 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f1a1c00b2e0 tx=0x7f1a1c01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:24:41.313 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:41.307+0000 7f1a31200700 1 --2- 172.21.15.119:0/1637943275 >> [v2:172.21.15.132:6800/2500318888,v1:172.21.15.132:6801/2500318888] conn(0x7f1a100ac3e0 0x7f1a100ae880 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:24:41.314 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:41.308+0000 7f1a28c00700 1 -- 172.21.15.119:0/1637943275 <== osd.1 v2:172.21.15.132:6800/2500318888 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f1a100aef50 con 0x7f1a100ac3e0 2024-03-28T00:24:41.332 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:41.326+0000 7f1a33200700 1 -- 172.21.15.119:0/1637943275 --> [v2:172.21.15.132:6800/2500318888,v1:172.21.15.132:6801/2500318888] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f1a2c062ce0 con 0x7f1a100ac3e0 2024-03-28T00:24:41.332 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:41.326+0000 7f1a28c00700 1 -- 172.21.15.119:0/1637943275 <== osd.1 v2:172.21.15.132:6800/2500318888 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f1a2c062ce0 con 0x7f1a100ac3e0 2024-03-28T00:24:41.333 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:41.327+0000 7f1a33200700 1 -- 172.21.15.119:0/1637943275 >> [v2:172.21.15.132:6800/2500318888,v1:172.21.15.132:6801/2500318888] conn(0x7f1a100ac3e0 msgr2=0x7f1a100ae880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:24:41.334 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:41.327+0000 7f1a33200700 1 --2- 172.21.15.119:0/1637943275 >> [v2:172.21.15.132:6800/2500318888,v1:172.21.15.132:6801/2500318888] conn(0x7f1a100ac3e0 0x7f1a100ae880 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:41.334 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:41.327+0000 7f1a33200700 1 -- 172.21.15.119:0/1637943275 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f1a100a29a0 msgr2=0x7f1a100a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:24:41.334 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:41.327+0000 7f1a33200700 1 --2- 172.21.15.119:0/1637943275 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f1a100a29a0 0x7f1a100a4e60 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f1a1c00b2e0 tx=0x7f1a1c01a040 comp rx=0 tx=0).stop 2024-03-28T00:24:41.334 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:41.327+0000 7f1a33200700 1 -- 172.21.15.119:0/1637943275 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1a2c06b2e0 msgr2=0x7f1a2c19ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:24:41.334 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:41.327+0000 7f1a33200700 1 --2- 172.21.15.119:0/1637943275 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1a2c06b2e0 0x7f1a2c19ebb0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f1a1800e970 tx=0x7f1a1800ed30 comp rx=0 tx=0).stop 2024-03-28T00:24:41.334 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:41.327+0000 7f1a31200700 1 -- 172.21.15.119:0/1637943275 reap_dead start 2024-03-28T00:24:41.334 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:41.327+0000 7f1a33200700 1 -- 172.21.15.119:0/1637943275 shutdown_connections 2024-03-28T00:24:41.334 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:41.327+0000 7f1a33200700 1 -- 172.21.15.119:0/1637943275 >> 172.21.15.119:0/1637943275 conn(0x7f1a2c0770e0 msgr2=0x7f1a2c0ff110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:24:41.334 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:41.327+0000 7f1a33200700 1 -- 172.21.15.119:0/1637943275 shutdown_connections 2024-03-28T00:24:41.334 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:41.327+0000 7f1a33200700 1 -- 172.21.15.119:0/1637943275 wait complete. 2024-03-28T00:24:42.681 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:41.004+0000 7f8dfec00700 1 -- 172.21.15.119:0/3833158232 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8df8106950 msgr2=0x7f8df8111330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:24:42.681 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:41.004+0000 7f8dfec00700 1 --2- 172.21.15.119:0/3833158232 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8df8106950 0x7f8df8111330 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f8df000b0d0 tx=0x7f8df000b3e0 comp rx=0 tx=0).stop 2024-03-28T00:24:42.682 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:41.004+0000 7f8dfec00700 1 -- 172.21.15.119:0/3833158232 shutdown_connections 2024-03-28T00:24:42.682 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:41.004+0000 7f8dfec00700 1 --2- 172.21.15.119:0/3833158232 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8df8106950 0x7f8df8111330 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:42.682 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:41.004+0000 7f8dfec00700 1 --2- 172.21.15.119:0/3833158232 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8df81046a0 0x7f8df8104a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:42.682 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:41.004+0000 7f8dfec00700 1 --2- 172.21.15.119:0/3833158232 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8df8105f90 0x7f8df8106410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:42.682 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:41.004+0000 7f8dfec00700 1 -- 172.21.15.119:0/3833158232 >> 172.21.15.119:0/3833158232 conn(0x7f8df80ffe30 msgr2=0x7f8df8102250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:24:42.682 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:41.004+0000 7f8dfec00700 1 -- 172.21.15.119:0/3833158232 shutdown_connections 2024-03-28T00:24:42.682 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:41.004+0000 7f8dfec00700 1 -- 172.21.15.119:0/3833158232 wait complete. 2024-03-28T00:24:42.682 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:41.005+0000 7f8dfec00700 1 Processor -- start 2024-03-28T00:24:42.682 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:41.006+0000 7f8dfec00700 1 -- start start 2024-03-28T00:24:42.682 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:41.006+0000 7f8dfec00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8df81046a0 0x7f8df81a07c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:24:42.682 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:41.006+0000 7f8dfec00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8df8105f90 0x7f8df81a0d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:24:42.682 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:41.006+0000 7f8dfec00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8df8106950 0x7f8df819a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:24:42.682 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:41.006+0000 7f8dfec00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8df81147a0 con 0x7f8df8106950 2024-03-28T00:24:42.683 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:41.006+0000 7f8dfec00700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8df8114620 con 0x7f8df81046a0 2024-03-28T00:24:42.683 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:41.006+0000 7f8dfec00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8df8114920 con 0x7f8df8105f90 2024-03-28T00:24:42.683 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:41.006+0000 7f8df7400700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8df8105f90 0x7f8df81a0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:24:42.683 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:41.006+0000 7f8df7e00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8df81046a0 0x7f8df81a07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:24:42.683 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:41.006+0000 7f8df7400700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8df8105f90 0x7f8df81a0d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.119:49998/0 (socket says 172.21.15.119:49998) 2024-03-28T00:24:42.683 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:41.006+0000 7f8df7400700 1 -- 172.21.15.119:0/722476674 learned_addr learned my addr 172.21.15.119:0/722476674 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:24:42.683 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:41.006+0000 7f8dfcc00700 1 --2- 172.21.15.119:0/722476674 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8df8106950 0x7f8df819a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:24:42.683 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:41.007+0000 7f8df7400700 1 -- 172.21.15.119:0/722476674 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8df81046a0 msgr2=0x7f8df81a07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:24:42.684 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:41.007+0000 7f8df7400700 1 --2- 172.21.15.119:0/722476674 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8df81046a0 0x7f8df81a07c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:42.684 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:41.007+0000 7f8df7400700 1 -- 172.21.15.119:0/722476674 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8df8106950 msgr2=0x7f8df819a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:24:42.684 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:41.007+0000 7f8df7400700 1 --2- 172.21.15.119:0/722476674 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8df8106950 0x7f8df819a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:42.684 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:41.007+0000 7f8df7400700 1 -- 172.21.15.119:0/722476674 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8df819b0e0 con 0x7f8df8105f90 2024-03-28T00:24:42.684 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:41.007+0000 7f8df7e00700 1 --2- 172.21.15.119:0/722476674 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8df81046a0 0x7f8df81a07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T00:24:42.684 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:41.007+0000 7f8df7400700 1 --2- 172.21.15.119:0/722476674 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8df8105f90 0x7f8df81a0d00 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f8de400eb30 tx=0x7f8de400ee40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:24:42.684 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:41.007+0000 7f8df4c00700 1 -- 172.21.15.119:0/722476674 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8de400cc90 con 0x7f8df8105f90 2024-03-28T00:24:42.684 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:41.007+0000 7f8df4c00700 1 -- 172.21.15.119:0/722476674 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8de4004500 con 0x7f8df8105f90 2024-03-28T00:24:42.684 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:41.007+0000 7f8df4c00700 1 -- 172.21.15.119:0/722476674 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8de4010430 con 0x7f8df8105f90 2024-03-28T00:24:42.684 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:41.007+0000 7f8dfec00700 1 -- 172.21.15.119:0/722476674 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8df819b3d0 con 0x7f8df8105f90 2024-03-28T00:24:42.685 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:41.007+0000 7f8dfec00700 1 -- 172.21.15.119:0/722476674 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8df81a7690 con 0x7f8df8105f90 2024-03-28T00:24:42.685 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:41.008+0000 7f8dfec00700 1 -- 172.21.15.119:0/722476674 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8df8100d30 con 0x7f8df8105f90 2024-03-28T00:24:42.685 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:41.010+0000 7f8df4c00700 1 -- 172.21.15.119:0/722476674 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 17) v1 ==== 137342+0+0 (secure 0 0 0) 0x7f8de4003f70 con 0x7f8df8105f90 2024-03-28T00:24:42.685 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:41.011+0000 7f8df4c00700 1 --2- 172.21.15.119:0/722476674 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f8ddc0a29a0 0x7f8ddc0a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:24:42.685 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:41.012+0000 7f8df4c00700 1 -- 172.21.15.119:0/722476674 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f8de40cf0a0 con 0x7f8df8105f90 2024-03-28T00:24:42.685 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:41.012+0000 7f8df7e00700 1 --2- 172.21.15.119:0/722476674 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f8ddc0a29a0 0x7f8ddc0a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:24:42.685 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:41.012+0000 7f8df4c00700 1 --2- 172.21.15.119:0/722476674 >> [v2:172.21.15.132:6816/2877892376,v1:172.21.15.132:6817/2877892376] conn(0x7f8ddc0ac3e0 0x7f8ddc0ae880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:24:42.685 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:41.012+0000 7f8df4c00700 1 -- 172.21.15.119:0/722476674 --> [v2:172.21.15.132:6816/2877892376,v1:172.21.15.132:6817/2877892376] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8de4001d30 con 0x7f8ddc0ac3e0 2024-03-28T00:24:42.685 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:41.012+0000 7f8df4c00700 1 -- 172.21.15.119:0/722476674 <== mon.1 v2:172.21.15.165:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8de40cf490 con 0x7f8df8105f90 2024-03-28T00:24:42.685 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:41.012+0000 7f8dfcc00700 1 --2- 172.21.15.119:0/722476674 >> [v2:172.21.15.132:6816/2877892376,v1:172.21.15.132:6817/2877892376] conn(0x7f8ddc0ac3e0 0x7f8ddc0ae880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:24:42.686 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:41.013+0000 7f8df7e00700 1 --2- 172.21.15.119:0/722476674 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f8ddc0a29a0 0x7f8ddc0a4e60 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f8df819bac0 tx=0x7f8de8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:24:42.686 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:41.013+0000 7f8dfcc00700 1 --2- 172.21.15.119:0/722476674 >> [v2:172.21.15.132:6816/2877892376,v1:172.21.15.132:6817/2877892376] conn(0x7f8ddc0ac3e0 0x7f8ddc0ae880 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:24:42.686 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:41.014+0000 7f8df4c00700 1 -- 172.21.15.119:0/722476674 <== osd.7 v2:172.21.15.132:6816/2877892376 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f8de4001d30 con 0x7f8ddc0ac3e0 2024-03-28T00:24:42.686 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:41.032+0000 7f8dfec00700 1 -- 172.21.15.119:0/722476674 --> [v2:172.21.15.132:6816/2877892376,v1:172.21.15.132:6817/2877892376] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f8df8105aa0 con 0x7f8ddc0ac3e0 2024-03-28T00:24:42.686 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:41.033+0000 7f8df4c00700 1 -- 172.21.15.119:0/722476674 <== osd.7 v2:172.21.15.132:6816/2877892376 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f8df8105aa0 con 0x7f8ddc0ac3e0 2024-03-28T00:24:42.686 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:41.033+0000 7f8dfec00700 1 -- 172.21.15.119:0/722476674 >> [v2:172.21.15.132:6816/2877892376,v1:172.21.15.132:6817/2877892376] conn(0x7f8ddc0ac3e0 msgr2=0x7f8ddc0ae880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:24:42.686 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:41.033+0000 7f8dfec00700 1 --2- 172.21.15.119:0/722476674 >> [v2:172.21.15.132:6816/2877892376,v1:172.21.15.132:6817/2877892376] conn(0x7f8ddc0ac3e0 0x7f8ddc0ae880 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:42.687 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:41.033+0000 7f8dfec00700 1 -- 172.21.15.119:0/722476674 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f8ddc0a29a0 msgr2=0x7f8ddc0a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:24:42.687 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:41.034+0000 7f8dfec00700 1 --2- 172.21.15.119:0/722476674 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f8ddc0a29a0 0x7f8ddc0a4e60 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f8df819bac0 tx=0x7f8de8009260 comp rx=0 tx=0).stop 2024-03-28T00:24:42.687 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:41.034+0000 7f8dfec00700 1 -- 172.21.15.119:0/722476674 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8df8105f90 msgr2=0x7f8df81a0d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:24:42.687 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:41.034+0000 7f8dfec00700 1 --2- 172.21.15.119:0/722476674 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8df8105f90 0x7f8df81a0d00 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f8de400eb30 tx=0x7f8de400ee40 comp rx=0 tx=0).stop 2024-03-28T00:24:42.687 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:41.034+0000 7f8dfcc00700 1 -- 172.21.15.119:0/722476674 reap_dead start 2024-03-28T00:24:42.687 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:41.034+0000 7f8dfec00700 1 -- 172.21.15.119:0/722476674 shutdown_connections 2024-03-28T00:24:42.687 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:41.034+0000 7f8dfec00700 1 -- 172.21.15.119:0/722476674 >> 172.21.15.119:0/722476674 conn(0x7f8df80ffe30 msgr2=0x7f8df8100310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:24:42.687 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:41.034+0000 7f8dfec00700 1 -- 172.21.15.119:0/722476674 shutdown_connections 2024-03-28T00:24:42.687 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:41.034+0000 7f8dfec00700 1 -- 172.21.15.119:0/722476674 wait complete. 2024-03-28T00:24:43.008 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:24:42 smithi119 ceph-mon[31518]: pgmap v336: 97 pgs: 97 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 9 op/s 2024-03-28T00:24:43.083 INFO:teuthology.orchestra.run.smithi119.stdout:34359738451 2024-03-28T00:24:43.084 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 10f73b92-ec98-11ee-95d0-87774f69a715 -- ceph osd last-stat-seq osd.0 2024-03-28T00:24:43.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:24:42 smithi165 ceph-mon[36302]: pgmap v336: 97 pgs: 97 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 9 op/s 2024-03-28T00:24:43.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:24:42 smithi132 ceph-mon[35941]: pgmap v336: 97 pgs: 97 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 9 op/s 2024-03-28T00:24:44.214 INFO:teuthology.orchestra.run.smithi119.stdout:55834574927 2024-03-28T00:24:44.215 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 10f73b92-ec98-11ee-95d0-87774f69a715 -- ceph osd last-stat-seq osd.1 2024-03-28T00:24:44.982 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:24:44 smithi119 ceph-mon[31518]: pgmap v337: 97 pgs: 97 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 170 B/s wr, 11 op/s 2024-03-28T00:24:45.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:24:44 smithi165 ceph-mon[36302]: pgmap v337: 97 pgs: 97 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 170 B/s wr, 11 op/s 2024-03-28T00:24:45.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:24:44 smithi132 ceph-mon[35941]: pgmap v337: 97 pgs: 97 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 170 B/s wr, 11 op/s 2024-03-28T00:24:46.315 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:45.039+0000 7fa54f200700 1 -- 172.21.15.119:0/3035832266 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa550103bb0 msgr2=0x7fa55010c0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:24:46.315 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:45.040+0000 7fa54f200700 1 --2- 172.21.15.119:0/3035832266 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa550103bb0 0x7fa55010c0a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fa53c009990 tx=0x7fa53c009ca0 comp rx=0 tx=0).stop 2024-03-28T00:24:46.315 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:45.040+0000 7fa54f200700 1 -- 172.21.15.119:0/3035832266 shutdown_connections 2024-03-28T00:24:46.315 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:45.040+0000 7fa54f200700 1 --2- 172.21.15.119:0/3035832266 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa55010c670 0x7fa55010eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:46.315 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:45.040+0000 7fa54f200700 1 --2- 172.21.15.119:0/3035832266 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa550103290 0x7fa550103670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:46.315 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:45.040+0000 7fa54f200700 1 --2- 172.21.15.119:0/3035832266 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa550103bb0 0x7fa55010c0a0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:46.315 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:45.040+0000 7fa54f200700 1 -- 172.21.15.119:0/3035832266 >> 172.21.15.119:0/3035832266 conn(0x7fa5500fca60 msgr2=0x7fa5500fee80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:24:46.315 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:45.040+0000 7fa54f200700 1 -- 172.21.15.119:0/3035832266 shutdown_connections 2024-03-28T00:24:46.316 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:45.040+0000 7fa54f200700 1 -- 172.21.15.119:0/3035832266 wait complete. 2024-03-28T00:24:46.316 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:45.041+0000 7fa54f200700 1 Processor -- start 2024-03-28T00:24:46.316 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:45.041+0000 7fa54f200700 1 -- start start 2024-03-28T00:24:46.316 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:45.041+0000 7fa54f200700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa550103290 0x7fa550100ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:24:46.316 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:45.041+0000 7fa54f200700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa550103bb0 0x7fa550101430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:24:46.316 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:45.041+0000 7fa54f200700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa55010c670 0x7fa550101970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:24:46.316 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:45.041+0000 7fa54f200700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5501123d0 con 0x7fa55010c670 2024-03-28T00:24:46.316 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:45.041+0000 7fa54f200700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa550112250 con 0x7fa550103bb0 2024-03-28T00:24:46.316 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:45.041+0000 7fa54f200700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa550112550 con 0x7fa550103290 2024-03-28T00:24:46.316 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:45.041+0000 7fa54de00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa550103290 0x7fa550100ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:24:46.316 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:45.041+0000 7fa54de00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa550103290 0x7fa550100ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.119:39818/0 (socket says 172.21.15.119:39818) 2024-03-28T00:24:46.317 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:45.041+0000 7fa54de00700 1 -- 172.21.15.119:0/2511158446 learned_addr learned my addr 172.21.15.119:0/2511158446 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:24:46.317 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:45.042+0000 7fa54d400700 1 --2- 172.21.15.119:0/2511158446 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa550103bb0 0x7fa550101430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:24:46.317 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:45.042+0000 7fa54e800700 1 --2- 172.21.15.119:0/2511158446 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa55010c670 0x7fa550101970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:24:46.317 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:45.042+0000 7fa54de00700 1 -- 172.21.15.119:0/2511158446 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa550103bb0 msgr2=0x7fa550101430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:24:46.317 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:45.042+0000 7fa54de00700 1 --2- 172.21.15.119:0/2511158446 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa550103bb0 0x7fa550101430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:46.317 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:45.042+0000 7fa54de00700 1 -- 172.21.15.119:0/2511158446 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa55010c670 msgr2=0x7fa550101970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:24:46.317 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:45.042+0000 7fa54de00700 1 --2- 172.21.15.119:0/2511158446 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa55010c670 0x7fa550101970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:46.317 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:45.042+0000 7fa54de00700 1 -- 172.21.15.119:0/2511158446 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5501a6dc0 con 0x7fa550103290 2024-03-28T00:24:46.317 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:45.042+0000 7fa54de00700 1 --2- 172.21.15.119:0/2511158446 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa550103290 0x7fa550100ef0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fa54000e9a0 tx=0x7fa54000ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:24:46.317 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:45.043+0000 7fa546a00700 1 -- 172.21.15.119:0/2511158446 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa54000ca10 con 0x7fa550103290 2024-03-28T00:24:46.317 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:45.043+0000 7fa546a00700 1 -- 172.21.15.119:0/2511158446 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa54000cbb0 con 0x7fa550103290 2024-03-28T00:24:46.317 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:45.043+0000 7fa54f200700 1 -- 172.21.15.119:0/2511158446 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5501a70b0 con 0x7fa550103290 2024-03-28T00:24:46.317 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:45.043+0000 7fa546a00700 1 -- 172.21.15.119:0/2511158446 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa540018680 con 0x7fa550103290 2024-03-28T00:24:46.318 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:45.043+0000 7fa54f200700 1 -- 172.21.15.119:0/2511158446 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5501a75c0 con 0x7fa550103290 2024-03-28T00:24:46.318 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:45.043+0000 7fa54f200700 1 -- 172.21.15.119:0/2511158446 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa550062860 con 0x7fa550103290 2024-03-28T00:24:46.318 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:45.046+0000 7fa546a00700 1 -- 172.21.15.119:0/2511158446 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 17) v1 ==== 137342+0+0 (secure 0 0 0) 0x7fa540010430 con 0x7fa550103290 2024-03-28T00:24:46.318 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:45.047+0000 7fa546a00700 1 --2- 172.21.15.119:0/2511158446 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fa5340a2920 0x7fa5340a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:24:46.318 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:45.047+0000 7fa546a00700 1 -- 172.21.15.119:0/2511158446 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fa5400cf640 con 0x7fa550103290 2024-03-28T00:24:46.318 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:45.047+0000 7fa54d400700 1 --2- 172.21.15.119:0/2511158446 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fa5340a2920 0x7fa5340a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:24:46.318 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:45.047+0000 7fa546a00700 1 --2- 172.21.15.119:0/2511158446 >> [v2:172.21.15.119:6818/2545410230,v1:172.21.15.119:6819/2545410230] conn(0x7fa5340ac360 0x7fa5340ae800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:24:46.318 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:45.047+0000 7fa546a00700 1 -- 172.21.15.119:0/2511158446 --> [v2:172.21.15.119:6818/2545410230,v1:172.21.15.119:6819/2545410230] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa54001bec0 con 0x7fa5340ac360 2024-03-28T00:24:46.318 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:45.047+0000 7fa546a00700 1 -- 172.21.15.119:0/2511158446 <== mon.1 v2:172.21.15.165:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa5400cf9f0 con 0x7fa550103290 2024-03-28T00:24:46.318 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:45.048+0000 7fa54e800700 1 --2- 172.21.15.119:0/2511158446 >> [v2:172.21.15.119:6818/2545410230,v1:172.21.15.119:6819/2545410230] conn(0x7fa5340ac360 0x7fa5340ae800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:24:46.318 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:45.048+0000 7fa54d400700 1 --2- 172.21.15.119:0/2511158446 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fa5340a2920 0x7fa5340a4de0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa550102930 tx=0x7fa53c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:24:46.318 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:45.048+0000 7fa54e800700 1 --2- 172.21.15.119:0/2511158446 >> [v2:172.21.15.119:6818/2545410230,v1:172.21.15.119:6819/2545410230] conn(0x7fa5340ac360 0x7fa5340ae800 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:24:46.318 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:45.049+0000 7fa546a00700 1 -- 172.21.15.119:0/2511158446 <== osd.6 v2:172.21.15.119:6818/2545410230 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fa54001bec0 con 0x7fa5340ac360 2024-03-28T00:24:46.319 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:45.067+0000 7fa54f200700 1 -- 172.21.15.119:0/2511158446 --> [v2:172.21.15.119:6818/2545410230,v1:172.21.15.119:6819/2545410230] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fa550062c60 con 0x7fa5340ac360 2024-03-28T00:24:46.319 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:45.067+0000 7fa546a00700 1 -- 172.21.15.119:0/2511158446 <== osd.6 v2:172.21.15.119:6818/2545410230 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fa550062c60 con 0x7fa5340ac360 2024-03-28T00:24:46.319 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:45.068+0000 7fa54f200700 1 -- 172.21.15.119:0/2511158446 >> [v2:172.21.15.119:6818/2545410230,v1:172.21.15.119:6819/2545410230] conn(0x7fa5340ac360 msgr2=0x7fa5340ae800 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:24:46.319 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:45.068+0000 7fa54f200700 1 --2- 172.21.15.119:0/2511158446 >> [v2:172.21.15.119:6818/2545410230,v1:172.21.15.119:6819/2545410230] conn(0x7fa5340ac360 0x7fa5340ae800 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:46.320 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:45.068+0000 7fa54f200700 1 -- 172.21.15.119:0/2511158446 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fa5340a2920 msgr2=0x7fa5340a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:24:46.320 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:45.068+0000 7fa54f200700 1 --2- 172.21.15.119:0/2511158446 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fa5340a2920 0x7fa5340a4de0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa550102930 tx=0x7fa53c005fd0 comp rx=0 tx=0).stop 2024-03-28T00:24:46.320 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:45.068+0000 7fa54f200700 1 -- 172.21.15.119:0/2511158446 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa550103290 msgr2=0x7fa550100ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:24:46.320 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:45.068+0000 7fa54f200700 1 --2- 172.21.15.119:0/2511158446 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa550103290 0x7fa550100ef0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fa54000e9a0 tx=0x7fa54000ecb0 comp rx=0 tx=0).stop 2024-03-28T00:24:46.320 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:45.068+0000 7fa54e800700 1 -- 172.21.15.119:0/2511158446 reap_dead start 2024-03-28T00:24:46.320 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:45.068+0000 7fa54f200700 1 -- 172.21.15.119:0/2511158446 shutdown_connections 2024-03-28T00:24:46.320 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:45.068+0000 7fa54f200700 1 -- 172.21.15.119:0/2511158446 >> 172.21.15.119:0/2511158446 conn(0x7fa5500fca60 msgr2=0x7fa5500fd770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:24:46.320 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:45.068+0000 7fa54f200700 1 -- 172.21.15.119:0/2511158446 shutdown_connections 2024-03-28T00:24:46.320 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:45.068+0000 7fa54f200700 1 -- 172.21.15.119:0/2511158446 wait complete. 2024-03-28T00:24:46.630 INFO:teuthology.orchestra.run.smithi119.stdout:219043332140 2024-03-28T00:24:46.630 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 10f73b92-ec98-11ee-95d0-87774f69a715 -- ceph osd last-stat-seq osd.7 2024-03-28T00:24:47.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:24:46 smithi119 ceph-mon[31518]: pgmap v338: 97 pgs: 97 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-28T00:24:47.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:24:46 smithi132 ceph-mon[35941]: pgmap v338: 97 pgs: 97 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-28T00:24:47.263 INFO:teuthology.orchestra.run.smithi119.stdout:197568495668 2024-03-28T00:24:47.263 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 10f73b92-ec98-11ee-95d0-87774f69a715 -- ceph osd last-stat-seq osd.6 2024-03-28T00:24:47.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:24:46 smithi165 ceph-mon[36302]: pgmap v338: 97 pgs: 97 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-28T00:24:48.872 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:48.862+0000 7f1742200700 1 -- 172.21.15.119:0/947786452 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f173c1021e0 msgr2=0x7f173c102660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:24:48.872 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:48.862+0000 7f1742200700 1 --2- 172.21.15.119:0/947786452 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f173c1021e0 0x7f173c102660 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f1724009990 tx=0x7f1724009ca0 comp rx=0 tx=0).stop 2024-03-28T00:24:48.872 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:48.862+0000 7f1742200700 1 -- 172.21.15.119:0/947786452 shutdown_connections 2024-03-28T00:24:48.872 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:48.862+0000 7f1742200700 1 --2- 172.21.15.119:0/947786452 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f173c102ba0 0x7f173c10efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:48.873 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:48.862+0000 7f1742200700 1 --2- 172.21.15.119:0/947786452 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f173c1081c0 0x7f173c1085a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:48.873 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:48.862+0000 7f1742200700 1 --2- 172.21.15.119:0/947786452 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f173c1021e0 0x7f173c102660 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:48.873 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:48.862+0000 7f1742200700 1 -- 172.21.15.119:0/947786452 >> 172.21.15.119:0/947786452 conn(0x7f173c0792c0 msgr2=0x7f173c0796d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:24:48.873 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:48.863+0000 7f1742200700 1 -- 172.21.15.119:0/947786452 shutdown_connections 2024-03-28T00:24:48.873 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:48.863+0000 7f1742200700 1 -- 172.21.15.119:0/947786452 wait complete. 2024-03-28T00:24:48.873 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:48.864+0000 7f1742200700 1 Processor -- start 2024-03-28T00:24:48.873 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:48.864+0000 7f1742200700 1 -- start start 2024-03-28T00:24:48.873 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:48.864+0000 7f1742200700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f173c1021e0 0x7f173c19e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:24:48.873 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:48.865+0000 7f1742200700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f173c102ba0 0x7f173c19ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:24:48.873 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:48.865+0000 7f1742200700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f173c1081c0 0x7f173c198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:24:48.873 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:48.865+0000 7f1742200700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f173c1124c0 con 0x7f173c1081c0 2024-03-28T00:24:48.873 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:48.865+0000 7f1742200700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f173c112340 con 0x7f173c102ba0 2024-03-28T00:24:48.874 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:48.865+0000 7f1742200700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f173c112640 con 0x7f173c1021e0 2024-03-28T00:24:48.874 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:48.865+0000 7f173b400700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f173c1021e0 0x7f173c19e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:24:48.874 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:48.865+0000 7f173b400700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f173c1021e0 0x7f173c19e670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.119:39856/0 (socket says 172.21.15.119:39856) 2024-03-28T00:24:48.874 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:48.865+0000 7f173b400700 1 -- 172.21.15.119:0/4066939732 learned_addr learned my addr 172.21.15.119:0/4066939732 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:24:48.874 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:48.865+0000 7f1733e00700 1 --2- 172.21.15.119:0/4066939732 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f173c102ba0 0x7f173c19ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:24:48.874 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:48.865+0000 7f173be00700 1 --2- 172.21.15.119:0/4066939732 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f173c1081c0 0x7f173c198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:24:48.874 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:48.865+0000 7f173b400700 1 -- 172.21.15.119:0/4066939732 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f173c102ba0 msgr2=0x7f173c19ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:24:48.874 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:48.865+0000 7f173b400700 1 --2- 172.21.15.119:0/4066939732 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f173c102ba0 0x7f173c19ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:48.874 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:48.865+0000 7f173b400700 1 -- 172.21.15.119:0/4066939732 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f173c1081c0 msgr2=0x7f173c198740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:24:48.877 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:48.865+0000 7f173b400700 1 --2- 172.21.15.119:0/4066939732 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f173c1081c0 0x7f173c198740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:48.877 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:48.865+0000 7f173b400700 1 -- 172.21.15.119:0/4066939732 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f173c199020 con 0x7f173c1021e0 2024-03-28T00:24:48.877 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:48.865+0000 7f173b400700 1 --2- 172.21.15.119:0/4066939732 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f173c1021e0 0x7f173c19e670 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f172c00e9a0 tx=0x7f172c00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:24:48.878 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:48.866+0000 7f1738c00700 1 -- 172.21.15.119:0/4066939732 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f172c00cbd0 con 0x7f173c1021e0 2024-03-28T00:24:48.878 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:48.866+0000 7f1738c00700 1 -- 172.21.15.119:0/4066939732 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f172c00cd70 con 0x7f173c1021e0 2024-03-28T00:24:48.878 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:48.866+0000 7f1742200700 1 -- 172.21.15.119:0/4066939732 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f173c199310 con 0x7f173c1021e0 2024-03-28T00:24:48.878 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:48.866+0000 7f1738c00700 1 -- 172.21.15.119:0/4066939732 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f172c018750 con 0x7f173c1021e0 2024-03-28T00:24:48.878 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:48.866+0000 7f1742200700 1 -- 172.21.15.119:0/4066939732 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f173c1a55f0 con 0x7f173c1021e0 2024-03-28T00:24:48.879 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:48.867+0000 7f1742200700 1 -- 172.21.15.119:0/4066939732 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f173c0628a0 con 0x7f173c1021e0 2024-03-28T00:24:48.879 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:48.869+0000 7f1738c00700 1 -- 172.21.15.119:0/4066939732 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 17) v1 ==== 137342+0+0 (secure 0 0 0) 0x7f172c010430 con 0x7f173c1021e0 2024-03-28T00:24:48.879 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:48.870+0000 7f1738c00700 1 --2- 172.21.15.119:0/4066939732 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f17200a29a0 0x7f17200a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:24:48.879 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:48.870+0000 7f1738c00700 1 -- 172.21.15.119:0/4066939732 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f172c0ced70 con 0x7f173c1021e0 2024-03-28T00:24:48.879 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:48.870+0000 7f1733e00700 1 --2- 172.21.15.119:0/4066939732 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f17200a29a0 0x7f17200a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:24:48.880 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:48.605+0000 7fc396c00700 1 -- 172.21.15.119:0/3050126844 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc390112e20 msgr2=0x7fc390115210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:24:48.880 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:48.605+0000 7fc396c00700 1 --2- 172.21.15.119:0/3050126844 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc390112e20 0x7fc390115210 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fc38800b0a0 tx=0x7fc38800b3b0 comp rx=0 tx=0).stop 2024-03-28T00:24:48.880 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:48.605+0000 7fc396c00700 1 -- 172.21.15.119:0/3050126844 shutdown_connections 2024-03-28T00:24:48.880 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:48.605+0000 7fc396c00700 1 --2- 172.21.15.119:0/3050126844 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc390112e20 0x7fc390115210 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:48.880 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:48.605+0000 7fc396c00700 1 --2- 172.21.15.119:0/3050126844 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc3900761c0 0x7fc3900743c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:48.881 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:48.605+0000 7fc396c00700 1 --2- 172.21.15.119:0/3050126844 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc390074900 0x7fc390074d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:48.881 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:48.605+0000 7fc396c00700 1 -- 172.21.15.119:0/3050126844 >> 172.21.15.119:0/3050126844 conn(0x7fc3900792c0 msgr2=0x7fc3900796d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:24:48.881 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:48.605+0000 7fc396c00700 1 -- 172.21.15.119:0/3050126844 shutdown_connections 2024-03-28T00:24:48.881 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:48.605+0000 7fc396c00700 1 -- 172.21.15.119:0/3050126844 wait complete. 2024-03-28T00:24:48.881 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:48.607+0000 7fc396c00700 1 Processor -- start 2024-03-28T00:24:48.881 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:48.607+0000 7fc396c00700 1 -- start start 2024-03-28T00:24:48.881 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:48.607+0000 7fc396c00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc390074900 0x7fc39019eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:24:48.881 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:48.607+0000 7fc396c00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc3900761c0 0x7fc39019f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:24:48.881 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:48.607+0000 7fc396c00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc390112e20 0x7fc3901a37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:24:48.881 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:48.607+0000 7fc396c00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc390118ab0 con 0x7fc390074900 2024-03-28T00:24:48.881 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:48.607+0000 7fc396c00700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc390118930 con 0x7fc3900761c0 2024-03-28T00:24:48.881 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:48.607+0000 7fc396c00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc390118c30 con 0x7fc390112e20 2024-03-28T00:24:48.881 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:48.607+0000 7fc394c00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc390112e20 0x7fc3901a37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:24:48.881 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:48.607+0000 7fc38f400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc3900761c0 0x7fc39019f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:24:48.882 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:48.607+0000 7fc394c00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc390112e20 0x7fc3901a37b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.119:39834/0 (socket says 172.21.15.119:39834) 2024-03-28T00:24:48.882 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:48.607+0000 7fc38fe00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc390074900 0x7fc39019eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:24:48.882 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:48.607+0000 7fc394c00700 1 -- 172.21.15.119:0/922395529 learned_addr learned my addr 172.21.15.119:0/922395529 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:24:48.882 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:48.608+0000 7fc394c00700 1 -- 172.21.15.119:0/922395529 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc3900761c0 msgr2=0x7fc39019f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:24:48.882 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:48.608+0000 7fc394c00700 1 --2- 172.21.15.119:0/922395529 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc3900761c0 0x7fc39019f420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:48.882 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:48.608+0000 7fc394c00700 1 -- 172.21.15.119:0/922395529 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc390074900 msgr2=0x7fc39019eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:24:48.882 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:48.608+0000 7fc394c00700 1 --2- 172.21.15.119:0/922395529 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc390074900 0x7fc39019eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:48.882 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:48.608+0000 7fc394c00700 1 -- 172.21.15.119:0/922395529 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3901a3ef0 con 0x7fc390112e20 2024-03-28T00:24:48.882 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:48.608+0000 7fc394c00700 1 --2- 172.21.15.119:0/922395529 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc390112e20 0x7fc3901a37b0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fc388006070 tx=0x7fc388004c60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:24:48.882 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:48.608+0000 7fc38cc00700 1 -- 172.21.15.119:0/922395529 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc388021070 con 0x7fc390112e20 2024-03-28T00:24:48.882 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:48.608+0000 7fc38cc00700 1 -- 172.21.15.119:0/922395529 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc388003c00 con 0x7fc390112e20 2024-03-28T00:24:48.883 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:48.608+0000 7fc396c00700 1 -- 172.21.15.119:0/922395529 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3901a4180 con 0x7fc390112e20 2024-03-28T00:24:48.883 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:48.608+0000 7fc38cc00700 1 -- 172.21.15.119:0/922395529 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3880046c0 con 0x7fc390112e20 2024-03-28T00:24:48.883 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:48.609+0000 7fc396c00700 1 -- 172.21.15.119:0/922395529 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3901aba60 con 0x7fc390112e20 2024-03-28T00:24:48.883 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:48.609+0000 7fc396c00700 1 -- 172.21.15.119:0/922395529 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc390112520 con 0x7fc390112e20 2024-03-28T00:24:48.883 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:48.611+0000 7fc38cc00700 1 -- 172.21.15.119:0/922395529 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 17) v1 ==== 137342+0+0 (secure 0 0 0) 0x7fc388003db0 con 0x7fc390112e20 2024-03-28T00:24:48.883 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:48.613+0000 7fc38cc00700 1 --2- 172.21.15.119:0/922395529 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fc3740a29a0 0x7fc3740a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:24:48.883 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:48.613+0000 7fc38cc00700 1 -- 172.21.15.119:0/922395529 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fc38800f020 con 0x7fc390112e20 2024-03-28T00:24:48.883 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:48.613+0000 7fc38fe00700 1 --2- 172.21.15.119:0/922395529 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fc3740a29a0 0x7fc3740a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:24:48.883 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:48.613+0000 7fc38cc00700 1 --2- 172.21.15.119:0/922395529 >> [v2:172.21.15.132:6808/2803039505,v1:172.21.15.132:6809/2803039505] conn(0x7fc3740ac3e0 0x7fc3740ae880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:24:48.884 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:48.613+0000 7fc38cc00700 1 -- 172.21.15.119:0/922395529 --> [v2:172.21.15.132:6808/2803039505,v1:172.21.15.132:6809/2803039505] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc38801cea0 con 0x7fc3740ac3e0 2024-03-28T00:24:48.884 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:48.613+0000 7fc38cc00700 1 -- 172.21.15.119:0/922395529 <== mon.1 v2:172.21.15.165:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc3880d03f0 con 0x7fc390112e20 2024-03-28T00:24:48.884 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:48.613+0000 7fc38f400700 1 --2- 172.21.15.119:0/922395529 >> [v2:172.21.15.132:6808/2803039505,v1:172.21.15.132:6809/2803039505] conn(0x7fc3740ac3e0 0x7fc3740ae880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:24:48.884 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:48.614+0000 7fc38fe00700 1 --2- 172.21.15.119:0/922395529 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fc3740a29a0 0x7fc3740a4e60 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fc380009be0 tx=0x7fc380009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:24:48.885 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:48.614+0000 7fc38f400700 1 --2- 172.21.15.119:0/922395529 >> [v2:172.21.15.132:6808/2803039505,v1:172.21.15.132:6809/2803039505] conn(0x7fc3740ac3e0 0x7fc3740ae880 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:24:48.885 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:48.615+0000 7fc38cc00700 1 -- 172.21.15.119:0/922395529 <== osd.4 v2:172.21.15.132:6808/2803039505 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fc38801cea0 con 0x7fc3740ac3e0 2024-03-28T00:24:48.885 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:48.633+0000 7fc396c00700 1 -- 172.21.15.119:0/922395529 --> [v2:172.21.15.132:6808/2803039505,v1:172.21.15.132:6809/2803039505] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fc390075a70 con 0x7fc3740ac3e0 2024-03-28T00:24:48.885 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:48.633+0000 7fc38cc00700 1 -- 172.21.15.119:0/922395529 <== osd.4 v2:172.21.15.132:6808/2803039505 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fc390075a70 con 0x7fc3740ac3e0 2024-03-28T00:24:48.885 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:48.634+0000 7fc396c00700 1 -- 172.21.15.119:0/922395529 >> [v2:172.21.15.132:6808/2803039505,v1:172.21.15.132:6809/2803039505] conn(0x7fc3740ac3e0 msgr2=0x7fc3740ae880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:24:48.885 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:48.634+0000 7fc396c00700 1 --2- 172.21.15.119:0/922395529 >> [v2:172.21.15.132:6808/2803039505,v1:172.21.15.132:6809/2803039505] conn(0x7fc3740ac3e0 0x7fc3740ae880 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:48.886 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:48.634+0000 7fc396c00700 1 -- 172.21.15.119:0/922395529 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fc3740a29a0 msgr2=0x7fc3740a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:24:48.886 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:48.634+0000 7fc396c00700 1 --2- 172.21.15.119:0/922395529 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fc3740a29a0 0x7fc3740a4e60 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fc380009be0 tx=0x7fc380009260 comp rx=0 tx=0).stop 2024-03-28T00:24:48.886 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:48.634+0000 7fc396c00700 1 -- 172.21.15.119:0/922395529 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc390112e20 msgr2=0x7fc3901a37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:24:48.886 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:48.634+0000 7fc396c00700 1 --2- 172.21.15.119:0/922395529 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc390112e20 0x7fc3901a37b0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fc388006070 tx=0x7fc388004c60 comp rx=0 tx=0).stop 2024-03-28T00:24:48.886 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:48.634+0000 7fc394c00700 1 -- 172.21.15.119:0/922395529 reap_dead start 2024-03-28T00:24:48.886 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:48.634+0000 7fc396c00700 1 -- 172.21.15.119:0/922395529 shutdown_connections 2024-03-28T00:24:48.886 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:48.634+0000 7fc396c00700 1 -- 172.21.15.119:0/922395529 >> 172.21.15.119:0/922395529 conn(0x7fc3900792c0 msgr2=0x7fc390111b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:24:48.886 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:48.634+0000 7fc396c00700 1 -- 172.21.15.119:0/922395529 shutdown_connections 2024-03-28T00:24:48.886 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:48.634+0000 7fc396c00700 1 -- 172.21.15.119:0/922395529 wait complete. 2024-03-28T00:24:48.887 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:48.871+0000 7f1738c00700 1 --2- 172.21.15.119:0/4066939732 >> [v2:172.21.15.165:6808/2561252206,v1:172.21.15.165:6809/2561252206] conn(0x7f17200ac3e0 0x7f17200ae880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:24:48.887 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:48.871+0000 7f1738c00700 1 -- 172.21.15.119:0/4066939732 --> [v2:172.21.15.165:6808/2561252206,v1:172.21.15.165:6809/2561252206] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f172c01ee50 con 0x7f17200ac3e0 2024-03-28T00:24:48.887 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:48.871+0000 7f1738c00700 1 -- 172.21.15.119:0/4066939732 <== mon.1 v2:172.21.15.165:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) v2 ==== 24+0+0 (secure 0 0 0) 0x7f172c0d1390 con 0x7f173c1021e0 2024-03-28T00:24:48.887 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:48.871+0000 7f173be00700 1 --2- 172.21.15.119:0/4066939732 >> [v2:172.21.15.165:6808/2561252206,v1:172.21.15.165:6809/2561252206] conn(0x7f17200ac3e0 0x7f17200ae880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:24:48.887 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:48.871+0000 7f1733e00700 1 --2- 172.21.15.119:0/4066939732 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f17200a29a0 0x7f17200a4e60 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f173c199f00 tx=0x7f1724005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:24:48.887 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:48.872+0000 7f173be00700 1 --2- 172.21.15.119:0/4066939732 >> [v2:172.21.15.165:6808/2561252206,v1:172.21.15.165:6809/2561252206] conn(0x7f17200ac3e0 0x7f17200ae880 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:24:48.888 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:48.873+0000 7f1738c00700 1 -- 172.21.15.119:0/4066939732 <== osd.5 v2:172.21.15.165:6808/2561252206 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f172c01ee50 con 0x7f17200ac3e0 2024-03-28T00:24:48.898 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:48.891+0000 7f1742200700 1 -- 172.21.15.119:0/4066939732 --> [v2:172.21.15.165:6808/2561252206,v1:172.21.15.165:6809/2561252206] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f173c062cd0 con 0x7f17200ac3e0 2024-03-28T00:24:48.898 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:48.891+0000 7f1738c00700 1 -- 172.21.15.119:0/4066939732 <== osd.5 v2:172.21.15.165:6808/2561252206 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f173c062cd0 con 0x7f17200ac3e0 2024-03-28T00:24:48.899 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:48.892+0000 7f1742200700 1 -- 172.21.15.119:0/4066939732 >> [v2:172.21.15.165:6808/2561252206,v1:172.21.15.165:6809/2561252206] conn(0x7f17200ac3e0 msgr2=0x7f17200ae880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:24:48.899 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:48.892+0000 7f1742200700 1 --2- 172.21.15.119:0/4066939732 >> [v2:172.21.15.165:6808/2561252206,v1:172.21.15.165:6809/2561252206] conn(0x7f17200ac3e0 0x7f17200ae880 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:48.899 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:48.892+0000 7f1742200700 1 -- 172.21.15.119:0/4066939732 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f17200a29a0 msgr2=0x7f17200a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:24:48.899 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:48.892+0000 7f1742200700 1 --2- 172.21.15.119:0/4066939732 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f17200a29a0 0x7f17200a4e60 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f173c199f00 tx=0x7f1724005fd0 comp rx=0 tx=0).stop 2024-03-28T00:24:48.899 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:48.892+0000 7f1742200700 1 -- 172.21.15.119:0/4066939732 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f173c1021e0 msgr2=0x7f173c19e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:24:48.899 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:48.892+0000 7f1742200700 1 --2- 172.21.15.119:0/4066939732 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f173c1021e0 0x7f173c19e670 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f172c00e9a0 tx=0x7f172c00ecb0 comp rx=0 tx=0).stop 2024-03-28T00:24:48.899 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:48.892+0000 7f173be00700 1 -- 172.21.15.119:0/4066939732 reap_dead start 2024-03-28T00:24:48.899 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:48.892+0000 7f1742200700 1 -- 172.21.15.119:0/4066939732 shutdown_connections 2024-03-28T00:24:48.899 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:48.892+0000 7f1742200700 1 -- 172.21.15.119:0/4066939732 >> 172.21.15.119:0/4066939732 conn(0x7f173c0792c0 msgr2=0x7f173c100da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:24:48.899 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:48.892+0000 7f1742200700 1 -- 172.21.15.119:0/4066939732 shutdown_connections 2024-03-28T00:24:48.899 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:48.892+0000 7f1742200700 1 -- 172.21.15.119:0/4066939732 wait complete. 2024-03-28T00:24:49.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:24:48 smithi119 ceph-mon[31518]: pgmap v339: 97 pgs: 97 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-03-28T00:24:49.184 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:24:48 smithi165 ceph-mon[36302]: pgmap v339: 97 pgs: 97 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-03-28T00:24:49.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:24:48 smithi132 ceph-mon[35941]: pgmap v339: 97 pgs: 97 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-03-28T00:24:49.775 INFO:teuthology.orchestra.run.smithi119.stdout:141733920832 2024-03-28T00:24:49.775 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 10f73b92-ec98-11ee-95d0-87774f69a715 -- ceph osd last-stat-seq osd.4 2024-03-28T00:24:50.389 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:49.525+0000 7f5d0e000700 1 -- 172.21.15.119:0/3642074278 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5d0806ac10 msgr2=0x7f5d0806b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:24:50.389 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:49.525+0000 7f5d0e000700 1 --2- 172.21.15.119:0/3642074278 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5d0806ac10 0x7f5d0806b090 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f5cf0009990 tx=0x7f5cf0009ca0 comp rx=0 tx=0).stop 2024-03-28T00:24:50.389 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:49.525+0000 7f5d0e000700 1 -- 172.21.15.119:0/3642074278 shutdown_connections 2024-03-28T00:24:50.389 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:49.525+0000 7f5d0e000700 1 --2- 172.21.15.119:0/3642074278 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5d08104bc0 0x7f5d08105040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:50.389 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:49.525+0000 7f5d0e000700 1 --2- 172.21.15.119:0/3642074278 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5d08107150 0x7f5d0806a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:50.389 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:49.525+0000 7f5d0e000700 1 --2- 172.21.15.119:0/3642074278 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5d0806ac10 0x7f5d0806b090 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:50.389 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:49.525+0000 7f5d0e000700 1 -- 172.21.15.119:0/3642074278 >> 172.21.15.119:0/3642074278 conn(0x7f5d080770e0 msgr2=0x7f5d080774f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:24:50.389 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:49.525+0000 7f5d0e000700 1 -- 172.21.15.119:0/3642074278 shutdown_connections 2024-03-28T00:24:50.389 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:49.525+0000 7f5d0e000700 1 -- 172.21.15.119:0/3642074278 wait complete. 2024-03-28T00:24:50.389 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:49.526+0000 7f5d0e000700 1 Processor -- start 2024-03-28T00:24:50.390 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:49.527+0000 7f5d0e000700 1 -- start start 2024-03-28T00:24:50.390 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:49.527+0000 7f5d0e000700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5d0806ac10 0x7f5d0819ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:24:50.390 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:49.527+0000 7f5d0e000700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5d08104bc0 0x7f5d0819f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:24:50.390 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:49.527+0000 7f5d0e000700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5d08107150 0x7f5d081a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:24:50.390 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:49.527+0000 7f5d0e000700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d08118aa0 con 0x7f5d08107150 2024-03-28T00:24:50.390 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:49.527+0000 7f5d0e000700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d08118920 con 0x7f5d08104bc0 2024-03-28T00:24:50.390 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:49.527+0000 7f5d0e000700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d08118c20 con 0x7f5d0806ac10 2024-03-28T00:24:50.390 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:49.527+0000 7f5cffe00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5d08104bc0 0x7f5d0819f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:24:50.390 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:49.527+0000 7f5d07400700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5d0806ac10 0x7f5d0819ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:24:50.390 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:49.527+0000 7f5cffe00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5d08104bc0 0x7f5d0819f3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.119:39588/0 (socket says 172.21.15.119:39588) 2024-03-28T00:24:50.390 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:49.527+0000 7f5cffe00700 1 -- 172.21.15.119:0/566960876 learned_addr learned my addr 172.21.15.119:0/566960876 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:24:50.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:49.528+0000 7f5d07e00700 1 --2- 172.21.15.119:0/566960876 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5d08107150 0x7f5d081a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:24:50.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:49.528+0000 7f5cffe00700 1 -- 172.21.15.119:0/566960876 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5d0806ac10 msgr2=0x7f5d0819ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:24:50.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:49.528+0000 7f5cffe00700 1 --2- 172.21.15.119:0/566960876 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5d0806ac10 0x7f5d0819ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:50.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:49.528+0000 7f5cffe00700 1 -- 172.21.15.119:0/566960876 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5d08107150 msgr2=0x7f5d081a3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:24:50.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:49.528+0000 7f5cffe00700 1 --2- 172.21.15.119:0/566960876 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5d08107150 0x7f5d081a3710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:50.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:49.528+0000 7f5cffe00700 1 -- 172.21.15.119:0/566960876 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d081a3e50 con 0x7f5d08104bc0 2024-03-28T00:24:50.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:49.528+0000 7f5d07e00700 1 --2- 172.21.15.119:0/566960876 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5d08107150 0x7f5d081a3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T00:24:50.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:49.528+0000 7f5cffe00700 1 --2- 172.21.15.119:0/566960876 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5d08104bc0 0x7f5d0819f3d0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f5cf0000c00 tx=0x7f5cf000ba50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:24:50.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:49.528+0000 7f5d04c00700 1 -- 172.21.15.119:0/566960876 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cf0027070 con 0x7f5d08104bc0 2024-03-28T00:24:50.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:49.528+0000 7f5d04c00700 1 -- 172.21.15.119:0/566960876 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5cf00056d0 con 0x7f5d08104bc0 2024-03-28T00:24:50.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:49.528+0000 7f5d04c00700 1 -- 172.21.15.119:0/566960876 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cf0022e60 con 0x7f5d08104bc0 2024-03-28T00:24:50.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:49.528+0000 7f5d0e000700 1 -- 172.21.15.119:0/566960876 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d081a40e0 con 0x7f5d08104bc0 2024-03-28T00:24:50.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:49.529+0000 7f5d0e000700 1 -- 172.21.15.119:0/566960876 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d081ab9c0 con 0x7f5d08104bc0 2024-03-28T00:24:50.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:49.529+0000 7f5d0e000700 1 -- 172.21.15.119:0/566960876 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f5d08100b10 con 0x7f5d08104bc0 2024-03-28T00:24:50.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:49.531+0000 7f5d04c00700 1 -- 172.21.15.119:0/566960876 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 17) v1 ==== 137342+0+0 (secure 0 0 0) 0x7f5cf002a070 con 0x7f5d08104bc0 2024-03-28T00:24:50.392 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:49.533+0000 7f5d04c00700 1 --2- 172.21.15.119:0/566960876 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f5cec0a2920 0x7f5cec0a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:24:50.392 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:49.533+0000 7f5d04c00700 1 -- 172.21.15.119:0/566960876 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f5cf00d0190 con 0x7f5d08104bc0 2024-03-28T00:24:50.392 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:49.533+0000 7f5d07400700 1 --2- 172.21.15.119:0/566960876 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f5cec0a2920 0x7f5cec0a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:24:50.392 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:49.533+0000 7f5d04c00700 1 --2- 172.21.15.119:0/566960876 >> [v2:172.21.15.132:6824/3454298492,v1:172.21.15.132:6825/3454298492] conn(0x7f5cec0ac360 0x7f5cec0ae800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:24:50.392 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:49.533+0000 7f5d04c00700 1 -- 172.21.15.119:0/566960876 --> [v2:172.21.15.132:6824/3454298492,v1:172.21.15.132:6825/3454298492] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5cf000b0d0 con 0x7f5cec0ac360 2024-03-28T00:24:50.392 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:49.533+0000 7f5d04c00700 1 -- 172.21.15.119:0/566960876 <== mon.2 v2:172.21.15.132:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5cf00d0540 con 0x7f5d08104bc0 2024-03-28T00:24:50.392 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:49.533+0000 7f5d07e00700 1 --2- 172.21.15.119:0/566960876 >> [v2:172.21.15.132:6824/3454298492,v1:172.21.15.132:6825/3454298492] conn(0x7f5cec0ac360 0x7f5cec0ae800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:24:50.392 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:49.534+0000 7f5d07400700 1 --2- 172.21.15.119:0/566960876 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f5cec0a2920 0x7f5cec0a4de0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f5cf8009c60 tx=0x7f5cf8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:24:50.392 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:49.534+0000 7f5d07e00700 1 --2- 172.21.15.119:0/566960876 >> [v2:172.21.15.132:6824/3454298492,v1:172.21.15.132:6825/3454298492] conn(0x7f5cec0ac360 0x7f5cec0ae800 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:24:50.392 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:49.535+0000 7f5d04c00700 1 -- 172.21.15.119:0/566960876 <== osd.10 v2:172.21.15.132:6824/3454298492 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f5cf000b0d0 con 0x7f5cec0ac360 2024-03-28T00:24:50.392 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:49.553+0000 7f5d0e000700 1 -- 172.21.15.119:0/566960876 --> [v2:172.21.15.132:6824/3454298492,v1:172.21.15.132:6825/3454298492] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f5d081016a0 con 0x7f5cec0ac360 2024-03-28T00:24:50.392 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:49.553+0000 7f5d04c00700 1 -- 172.21.15.119:0/566960876 <== osd.10 v2:172.21.15.132:6824/3454298492 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f5d081016a0 con 0x7f5cec0ac360 2024-03-28T00:24:50.393 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:49.554+0000 7f5d0e000700 1 -- 172.21.15.119:0/566960876 >> [v2:172.21.15.132:6824/3454298492,v1:172.21.15.132:6825/3454298492] conn(0x7f5cec0ac360 msgr2=0x7f5cec0ae800 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:24:50.393 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:49.554+0000 7f5d0e000700 1 --2- 172.21.15.119:0/566960876 >> [v2:172.21.15.132:6824/3454298492,v1:172.21.15.132:6825/3454298492] conn(0x7f5cec0ac360 0x7f5cec0ae800 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:50.393 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:49.554+0000 7f5d0e000700 1 -- 172.21.15.119:0/566960876 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f5cec0a2920 msgr2=0x7f5cec0a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:24:50.393 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:49.554+0000 7f5d0e000700 1 --2- 172.21.15.119:0/566960876 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f5cec0a2920 0x7f5cec0a4de0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f5cf8009c60 tx=0x7f5cf8009260 comp rx=0 tx=0).stop 2024-03-28T00:24:50.393 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:49.554+0000 7f5d0e000700 1 -- 172.21.15.119:0/566960876 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5d08104bc0 msgr2=0x7f5d0819f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:24:50.393 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:49.554+0000 7f5d0e000700 1 --2- 172.21.15.119:0/566960876 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5d08104bc0 0x7f5d0819f3d0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f5cf0000c00 tx=0x7f5cf000ba50 comp rx=0 tx=0).stop 2024-03-28T00:24:50.393 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:49.554+0000 7f5d07e00700 1 -- 172.21.15.119:0/566960876 reap_dead start 2024-03-28T00:24:50.394 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:49.554+0000 7f5d0e000700 1 -- 172.21.15.119:0/566960876 shutdown_connections 2024-03-28T00:24:50.394 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:49.554+0000 7f5d0e000700 1 -- 172.21.15.119:0/566960876 >> 172.21.15.119:0/566960876 conn(0x7f5d080770e0 msgr2=0x7f5d081000b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:24:50.394 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:49.554+0000 7f5d0e000700 1 -- 172.21.15.119:0/566960876 shutdown_connections 2024-03-28T00:24:50.394 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:49.555+0000 7f5d0e000700 1 -- 172.21.15.119:0/566960876 wait complete. 2024-03-28T00:24:51.152 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:51.145+0000 7fa051400700 1 -- 172.21.15.119:0/993806749 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa04c10ec40 msgr2=0x7fa04c111030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:24:51.152 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:51.145+0000 7fa051400700 1 --2- 172.21.15.119:0/993806749 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa04c10ec40 0x7fa04c111030 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa04400b0a0 tx=0x7fa04400b3b0 comp rx=0 tx=0).stop 2024-03-28T00:24:51.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:51.145+0000 7fa051400700 1 -- 172.21.15.119:0/993806749 shutdown_connections 2024-03-28T00:24:51.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:51.145+0000 7fa051400700 1 --2- 172.21.15.119:0/993806749 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa04c10ec40 0x7fa04c111030 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:51.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:51.145+0000 7fa051400700 1 --2- 172.21.15.119:0/993806749 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa04c069e40 0x7fa04c06a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:51.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:51.145+0000 7fa051400700 1 --2- 172.21.15.119:0/993806749 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa04c06a760 0x7fa04c10e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:51.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:51.145+0000 7fa051400700 1 -- 172.21.15.119:0/993806749 >> 172.21.15.119:0/993806749 conn(0x7fa04c077fc0 msgr2=0x7fa04c0783d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:24:51.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:51.146+0000 7fa051400700 1 -- 172.21.15.119:0/993806749 shutdown_connections 2024-03-28T00:24:51.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:51.146+0000 7fa051400700 1 -- 172.21.15.119:0/993806749 wait complete. 2024-03-28T00:24:51.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:24:50 smithi119 ceph-mon[31518]: pgmap v340: 97 pgs: 97 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 11 op/s 2024-03-28T00:24:51.155 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:51.147+0000 7fa051400700 1 Processor -- start 2024-03-28T00:24:51.155 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:51.147+0000 7fa051400700 1 -- start start 2024-03-28T00:24:51.155 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:51.148+0000 7fa051400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa04c069e40 0x7fa04c1a0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:24:51.156 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:51.148+0000 7fa051400700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa04c06a760 0x7fa04c1a0d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:24:51.156 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:51.148+0000 7fa051400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa04c10ec40 0x7fa04c19aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:24:51.156 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:51.148+0000 7fa051400700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa04c114690 con 0x7fa04c10ec40 2024-03-28T00:24:51.156 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:51.148+0000 7fa051400700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa04c114510 con 0x7fa04c069e40 2024-03-28T00:24:51.156 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:51.148+0000 7fa051400700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa04c114810 con 0x7fa04c06a760 2024-03-28T00:24:51.156 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:51.148+0000 7fa04a800700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa04c069e40 0x7fa04c1a0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:24:51.156 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:51.148+0000 7fa04a800700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa04c069e40 0x7fa04c1a0820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.119:39632/0 (socket says 172.21.15.119:39632) 2024-03-28T00:24:51.156 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:51.148+0000 7fa04a800700 1 -- 172.21.15.119:0/699777863 learned_addr learned my addr 172.21.15.119:0/699777863 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:24:51.156 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:51.148+0000 7fa049e00700 1 --2- 172.21.15.119:0/699777863 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa04c06a760 0x7fa04c1a0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:24:51.156 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:51.148+0000 7fa04b200700 1 --2- 172.21.15.119:0/699777863 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa04c10ec40 0x7fa04c19aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:24:51.156 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:51.148+0000 7fa04a800700 1 -- 172.21.15.119:0/699777863 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa04c06a760 msgr2=0x7fa04c1a0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:24:51.156 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:51.148+0000 7fa04a800700 1 --2- 172.21.15.119:0/699777863 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa04c06a760 0x7fa04c1a0d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:51.157 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:51.148+0000 7fa04a800700 1 -- 172.21.15.119:0/699777863 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa04c10ec40 msgr2=0x7fa04c19aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:24:51.157 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:51.148+0000 7fa04a800700 1 --2- 172.21.15.119:0/699777863 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa04c10ec40 0x7fa04c19aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:51.157 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:51.148+0000 7fa04a800700 1 -- 172.21.15.119:0/699777863 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa04c19b190 con 0x7fa04c069e40 2024-03-28T00:24:51.157 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:51.149+0000 7fa04a800700 1 --2- 172.21.15.119:0/699777863 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa04c069e40 0x7fa04c1a0820 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa03c00e9a0 tx=0x7fa03c00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:24:51.157 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:51.149+0000 7fa043400700 1 -- 172.21.15.119:0/699777863 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa03c00ca10 con 0x7fa04c069e40 2024-03-28T00:24:51.157 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:51.149+0000 7fa043400700 1 -- 172.21.15.119:0/699777863 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa03c00cbb0 con 0x7fa04c069e40 2024-03-28T00:24:51.157 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:51.149+0000 7fa051400700 1 -- 172.21.15.119:0/699777863 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa04c19b480 con 0x7fa04c069e40 2024-03-28T00:24:51.157 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:51.149+0000 7fa043400700 1 -- 172.21.15.119:0/699777863 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa03c018680 con 0x7fa04c069e40 2024-03-28T00:24:51.157 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:51.149+0000 7fa051400700 1 -- 172.21.15.119:0/699777863 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa04c1a76f0 con 0x7fa04c069e40 2024-03-28T00:24:51.158 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:51.150+0000 7fa051400700 1 -- 172.21.15.119:0/699777863 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa04c0628a0 con 0x7fa04c069e40 2024-03-28T00:24:51.159 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:51.152+0000 7fa043400700 1 -- 172.21.15.119:0/699777863 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 17) v1 ==== 137342+0+0 (secure 0 0 0) 0x7fa03c010430 con 0x7fa04c069e40 2024-03-28T00:24:51.160 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:51.153+0000 7fa043400700 1 --2- 172.21.15.119:0/699777863 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fa0300a29a0 0x7fa0300a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:24:51.160 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:51.153+0000 7fa043400700 1 -- 172.21.15.119:0/699777863 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fa03c0ceda0 con 0x7fa04c069e40 2024-03-28T00:24:51.160 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:51.153+0000 7fa049e00700 1 --2- 172.21.15.119:0/699777863 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fa0300a29a0 0x7fa0300a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:24:51.160 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:51.154+0000 7fa043400700 1 --2- 172.21.15.119:0/699777863 >> [v2:172.21.15.119:6810/3732272210,v1:172.21.15.119:6811/3732272210] conn(0x7fa0300ac3e0 0x7fa0300ae880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:24:51.160 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:51.154+0000 7fa043400700 1 -- 172.21.15.119:0/699777863 --> [v2:172.21.15.119:6810/3732272210,v1:172.21.15.119:6811/3732272210] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa03c015e80 con 0x7fa0300ac3e0 2024-03-28T00:24:51.160 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:51.154+0000 7fa043400700 1 -- 172.21.15.119:0/699777863 <== mon.2 v2:172.21.15.132:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa03c0d13c0 con 0x7fa04c069e40 2024-03-28T00:24:51.161 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:51.154+0000 7fa04b200700 1 --2- 172.21.15.119:0/699777863 >> [v2:172.21.15.119:6810/3732272210,v1:172.21.15.119:6811/3732272210] conn(0x7fa0300ac3e0 0x7fa0300ae880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:24:51.161 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:51.154+0000 7fa049e00700 1 --2- 172.21.15.119:0/699777863 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fa0300a29a0 0x7fa0300a4e60 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fa04c102610 tx=0x7fa038009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:24:51.161 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:51.155+0000 7fa04b200700 1 --2- 172.21.15.119:0/699777863 >> [v2:172.21.15.119:6810/3732272210,v1:172.21.15.119:6811/3732272210] conn(0x7fa0300ac3e0 0x7fa0300ae880 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:24:51.162 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:51.156+0000 7fa043400700 1 -- 172.21.15.119:0/699777863 <== osd.3 v2:172.21.15.119:6810/3732272210 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fa03c015e80 con 0x7fa0300ac3e0 2024-03-28T00:24:51.180 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:51.174+0000 7fa051400700 1 -- 172.21.15.119:0/699777863 --> [v2:172.21.15.119:6810/3732272210,v1:172.21.15.119:6811/3732272210] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fa04c062cd0 con 0x7fa0300ac3e0 2024-03-28T00:24:51.181 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:51.175+0000 7fa043400700 1 -- 172.21.15.119:0/699777863 <== osd.3 v2:172.21.15.119:6810/3732272210 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fa04c062cd0 con 0x7fa0300ac3e0 2024-03-28T00:24:51.182 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:51.175+0000 7fa051400700 1 -- 172.21.15.119:0/699777863 >> [v2:172.21.15.119:6810/3732272210,v1:172.21.15.119:6811/3732272210] conn(0x7fa0300ac3e0 msgr2=0x7fa0300ae880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:24:51.182 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:51.175+0000 7fa051400700 1 --2- 172.21.15.119:0/699777863 >> [v2:172.21.15.119:6810/3732272210,v1:172.21.15.119:6811/3732272210] conn(0x7fa0300ac3e0 0x7fa0300ae880 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:51.182 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:51.175+0000 7fa051400700 1 -- 172.21.15.119:0/699777863 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fa0300a29a0 msgr2=0x7fa0300a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:24:51.182 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:51.175+0000 7fa051400700 1 --2- 172.21.15.119:0/699777863 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fa0300a29a0 0x7fa0300a4e60 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fa04c102610 tx=0x7fa038009260 comp rx=0 tx=0).stop 2024-03-28T00:24:51.182 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:51.175+0000 7fa051400700 1 -- 172.21.15.119:0/699777863 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa04c069e40 msgr2=0x7fa04c1a0820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:24:51.182 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:51.175+0000 7fa051400700 1 --2- 172.21.15.119:0/699777863 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa04c069e40 0x7fa04c1a0820 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa03c00e9a0 tx=0x7fa03c00ecb0 comp rx=0 tx=0).stop 2024-03-28T00:24:51.182 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:51.175+0000 7fa04b200700 1 -- 172.21.15.119:0/699777863 reap_dead start 2024-03-28T00:24:51.182 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:51.175+0000 7fa051400700 1 -- 172.21.15.119:0/699777863 shutdown_connections 2024-03-28T00:24:51.182 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:51.175+0000 7fa051400700 1 -- 172.21.15.119:0/699777863 >> 172.21.15.119:0/699777863 conn(0x7fa04c077fc0 msgr2=0x7fa04c110130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:24:51.182 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:51.175+0000 7fa051400700 1 -- 172.21.15.119:0/699777863 shutdown_connections 2024-03-28T00:24:51.182 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:51.176+0000 7fa051400700 1 -- 172.21.15.119:0/699777863 wait complete. 2024-03-28T00:24:51.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:24:50 smithi132 ceph-mon[35941]: pgmap v340: 97 pgs: 97 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 11 op/s 2024-03-28T00:24:51.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:24:50 smithi165 ceph-mon[36302]: pgmap v340: 97 pgs: 97 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 11 op/s 2024-03-28T00:24:51.458 INFO:teuthology.orchestra.run.smithi119.stdout:309237645342 2024-03-28T00:24:51.458 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 10f73b92-ec98-11ee-95d0-87774f69a715 -- ceph osd last-stat-seq osd.10 2024-03-28T00:24:52.234 INFO:teuthology.orchestra.run.smithi119.stdout:107374182468 2024-03-28T00:24:52.235 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 10f73b92-ec98-11ee-95d0-87774f69a715 -- ceph osd last-stat-seq osd.3 2024-03-28T00:24:52.317 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:50.643+0000 7fb0d6000700 1 -- 172.21.15.119:0/518123367 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb0d006a7e0 msgr2=0x7fb0d010f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:24:52.317 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:50.643+0000 7fb0d6000700 1 --2- 172.21.15.119:0/518123367 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb0d006a7e0 0x7fb0d010f020 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fb0c800b0d0 tx=0x7fb0c800b3e0 comp rx=0 tx=0).stop 2024-03-28T00:24:52.317 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:50.643+0000 7fb0d6000700 1 -- 172.21.15.119:0/518123367 shutdown_connections 2024-03-28T00:24:52.317 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:50.643+0000 7fb0d6000700 1 --2- 172.21.15.119:0/518123367 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb0d006a7e0 0x7fb0d010f020 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:52.317 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:50.643+0000 7fb0d6000700 1 --2- 172.21.15.119:0/518123367 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb0d01081e0 0x7fb0d01085c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:52.317 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:50.643+0000 7fb0d6000700 1 --2- 172.21.15.119:0/518123367 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb0d0069e40 0x7fb0d006a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:52.317 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:50.643+0000 7fb0d6000700 1 -- 172.21.15.119:0/518123367 >> 172.21.15.119:0/518123367 conn(0x7fb0d0077fc0 msgr2=0x7fb0d00783d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:24:52.318 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:50.643+0000 7fb0d6000700 1 -- 172.21.15.119:0/518123367 shutdown_connections 2024-03-28T00:24:52.318 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:50.643+0000 7fb0d6000700 1 -- 172.21.15.119:0/518123367 wait complete. 2024-03-28T00:24:52.318 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:50.644+0000 7fb0d6000700 1 Processor -- start 2024-03-28T00:24:52.318 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:50.645+0000 7fb0d6000700 1 -- start start 2024-03-28T00:24:52.318 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:50.645+0000 7fb0d6000700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb0d0069e40 0x7fb0d01a07f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:24:52.318 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:50.645+0000 7fb0d6000700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb0d006a7e0 0x7fb0d01a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:24:52.318 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:50.645+0000 7fb0d6000700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb0d01081e0 0x7fb0d019a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:24:52.318 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:50.645+0000 7fb0d6000700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0d0112480 con 0x7fb0d0069e40 2024-03-28T00:24:52.318 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:50.645+0000 7fb0d6000700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0d0112300 con 0x7fb0d01081e0 2024-03-28T00:24:52.318 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:50.645+0000 7fb0d6000700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0d0112600 con 0x7fb0d006a7e0 2024-03-28T00:24:52.318 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:50.645+0000 7fb0c7e00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb0d006a7e0 0x7fb0d01a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:24:52.318 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:50.645+0000 7fb0cf400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb0d0069e40 0x7fb0d01a07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:24:52.318 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:50.645+0000 7fb0c7e00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb0d006a7e0 0x7fb0d01a0d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.119:39908/0 (socket says 172.21.15.119:39908) 2024-03-28T00:24:52.319 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:50.645+0000 7fb0c7e00700 1 -- 172.21.15.119:0/1891468897 learned_addr learned my addr 172.21.15.119:0/1891468897 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:24:52.319 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:50.645+0000 7fb0cfe00700 1 --2- 172.21.15.119:0/1891468897 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb0d01081e0 0x7fb0d019a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:24:52.319 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:50.646+0000 7fb0c7e00700 1 -- 172.21.15.119:0/1891468897 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb0d01081e0 msgr2=0x7fb0d019a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:24:52.319 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:50.646+0000 7fb0c7e00700 1 --2- 172.21.15.119:0/1891468897 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb0d01081e0 0x7fb0d019a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:52.319 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:50.646+0000 7fb0c7e00700 1 -- 172.21.15.119:0/1891468897 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb0d0069e40 msgr2=0x7fb0d01a07f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:24:52.319 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:50.646+0000 7fb0c7e00700 1 --2- 172.21.15.119:0/1891468897 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb0d0069e40 0x7fb0d01a07f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:52.319 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:50.646+0000 7fb0c7e00700 1 -- 172.21.15.119:0/1891468897 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0d019b110 con 0x7fb0d006a7e0 2024-03-28T00:24:52.319 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:50.646+0000 7fb0c7e00700 1 --2- 172.21.15.119:0/1891468897 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb0d006a7e0 0x7fb0d01a0d30 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fb0b800e9a0 tx=0x7fb0b800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:24:52.319 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:50.646+0000 7fb0ccc00700 1 -- 172.21.15.119:0/1891468897 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0b800ca10 con 0x7fb0d006a7e0 2024-03-28T00:24:52.319 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:50.646+0000 7fb0ccc00700 1 -- 172.21.15.119:0/1891468897 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb0b800cbb0 con 0x7fb0d006a7e0 2024-03-28T00:24:52.319 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:50.646+0000 7fb0d6000700 1 -- 172.21.15.119:0/1891468897 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0d019b400 con 0x7fb0d006a7e0 2024-03-28T00:24:52.319 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:50.646+0000 7fb0ccc00700 1 -- 172.21.15.119:0/1891468897 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0b8018680 con 0x7fb0d006a7e0 2024-03-28T00:24:52.319 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:50.646+0000 7fb0d6000700 1 -- 172.21.15.119:0/1891468897 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0d01a76c0 con 0x7fb0d006a7e0 2024-03-28T00:24:52.319 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:50.647+0000 7fb0d6000700 1 -- 172.21.15.119:0/1891468897 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb0d010dd10 con 0x7fb0d006a7e0 2024-03-28T00:24:52.320 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:50.649+0000 7fb0ccc00700 1 -- 172.21.15.119:0/1891468897 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 17) v1 ==== 137342+0+0 (secure 0 0 0) 0x7fb0b8010430 con 0x7fb0d006a7e0 2024-03-28T00:24:52.320 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:50.651+0000 7fb0ccc00700 1 --2- 172.21.15.119:0/1891468897 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fb0b40a29a0 0x7fb0b40a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:24:52.320 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:50.651+0000 7fb0ccc00700 1 -- 172.21.15.119:0/1891468897 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fb0b80cf600 con 0x7fb0d006a7e0 2024-03-28T00:24:52.320 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:50.651+0000 7fb0cf400700 1 --2- 172.21.15.119:0/1891468897 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fb0b40a29a0 0x7fb0b40a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:24:52.320 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:50.651+0000 7fb0ccc00700 1 --2- 172.21.15.119:0/1891468897 >> [v2:172.21.15.119:6826/2532245222,v1:172.21.15.119:6827/2532245222] conn(0x7fb0b40ac3e0 0x7fb0b40ae880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:24:52.320 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:50.651+0000 7fb0ccc00700 1 -- 172.21.15.119:0/1891468897 --> [v2:172.21.15.119:6826/2532245222,v1:172.21.15.119:6827/2532245222] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb0b801bec0 con 0x7fb0b40ac3e0 2024-03-28T00:24:52.320 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:50.651+0000 7fb0ccc00700 1 -- 172.21.15.119:0/1891468897 <== mon.1 v2:172.21.15.165:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb0b80cf9b0 con 0x7fb0d006a7e0 2024-03-28T00:24:52.321 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:50.651+0000 7fb0cfe00700 1 --2- 172.21.15.119:0/1891468897 >> [v2:172.21.15.119:6826/2532245222,v1:172.21.15.119:6827/2532245222] conn(0x7fb0b40ac3e0 0x7fb0b40ae880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:24:52.321 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:50.651+0000 7fb0cf400700 1 --2- 172.21.15.119:0/1891468897 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fb0b40a29a0 0x7fb0b40a4e60 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fb0c0009680 tx=0x7fb0c0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:24:52.321 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:50.652+0000 7fb0cfe00700 1 --2- 172.21.15.119:0/1891468897 >> [v2:172.21.15.119:6826/2532245222,v1:172.21.15.119:6827/2532245222] conn(0x7fb0b40ac3e0 0x7fb0b40ae880 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:24:52.321 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:50.653+0000 7fb0ccc00700 1 -- 172.21.15.119:0/1891468897 <== osd.9 v2:172.21.15.119:6826/2532245222 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fb0b801bec0 con 0x7fb0b40ac3e0 2024-03-28T00:24:52.321 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:50.671+0000 7fb0d6000700 1 -- 172.21.15.119:0/1891468897 --> [v2:172.21.15.119:6826/2532245222,v1:172.21.15.119:6827/2532245222] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb0d0103890 con 0x7fb0b40ac3e0 2024-03-28T00:24:52.321 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:50.671+0000 7fb0ccc00700 1 -- 172.21.15.119:0/1891468897 <== osd.9 v2:172.21.15.119:6826/2532245222 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fb0d0103890 con 0x7fb0b40ac3e0 2024-03-28T00:24:52.321 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:50.672+0000 7fb0d6000700 1 -- 172.21.15.119:0/1891468897 >> [v2:172.21.15.119:6826/2532245222,v1:172.21.15.119:6827/2532245222] conn(0x7fb0b40ac3e0 msgr2=0x7fb0b40ae880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:24:52.321 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:50.672+0000 7fb0d6000700 1 --2- 172.21.15.119:0/1891468897 >> [v2:172.21.15.119:6826/2532245222,v1:172.21.15.119:6827/2532245222] conn(0x7fb0b40ac3e0 0x7fb0b40ae880 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:52.322 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:50.672+0000 7fb0d6000700 1 -- 172.21.15.119:0/1891468897 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fb0b40a29a0 msgr2=0x7fb0b40a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:24:52.322 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:50.672+0000 7fb0d6000700 1 --2- 172.21.15.119:0/1891468897 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fb0b40a29a0 0x7fb0b40a4e60 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fb0c0009680 tx=0x7fb0c0009260 comp rx=0 tx=0).stop 2024-03-28T00:24:52.322 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:50.672+0000 7fb0d6000700 1 -- 172.21.15.119:0/1891468897 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb0d006a7e0 msgr2=0x7fb0d01a0d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:24:52.322 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:50.672+0000 7fb0d6000700 1 --2- 172.21.15.119:0/1891468897 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb0d006a7e0 0x7fb0d01a0d30 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fb0b800e9a0 tx=0x7fb0b800ecb0 comp rx=0 tx=0).stop 2024-03-28T00:24:52.322 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:50.672+0000 7fb0cfe00700 1 -- 172.21.15.119:0/1891468897 reap_dead start 2024-03-28T00:24:52.322 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:50.672+0000 7fb0d6000700 1 -- 172.21.15.119:0/1891468897 shutdown_connections 2024-03-28T00:24:52.322 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:50.672+0000 7fb0d6000700 1 -- 172.21.15.119:0/1891468897 >> 172.21.15.119:0/1891468897 conn(0x7fb0d0077fc0 msgr2=0x7fb0d010d370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:24:52.322 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:50.672+0000 7fb0d6000700 1 -- 172.21.15.119:0/1891468897 shutdown_connections 2024-03-28T00:24:52.322 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:50.672+0000 7fb0d6000700 1 -- 172.21.15.119:0/1891468897 wait complete. 2024-03-28T00:24:53.020 INFO:teuthology.orchestra.run.smithi119.stdout:163208757306 2024-03-28T00:24:53.021 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 10f73b92-ec98-11ee-95d0-87774f69a715 -- ceph osd last-stat-seq osd.5 2024-03-28T00:24:53.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:24:52 smithi119 ceph-mon[31518]: pgmap v341: 97 pgs: 97 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T00:24:53.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:24:52 smithi132 ceph-mon[35941]: pgmap v341: 97 pgs: 97 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T00:24:53.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:24:52 smithi165 ceph-mon[36302]: pgmap v341: 97 pgs: 97 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T00:24:53.538 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/mon.a/config 2024-03-28T00:24:53.546 INFO:teuthology.orchestra.run.smithi119.stdout:287762808867 2024-03-28T00:24:53.547 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 10f73b92-ec98-11ee-95d0-87774f69a715 -- ceph osd last-stat-seq osd.9 2024-03-28T00:24:53.566 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/mon.a/config 2024-03-28T00:24:53.566 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/mon.a/config 2024-03-28T00:24:53.566 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/mon.a/config 2024-03-28T00:24:53.567 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/mon.a/config 2024-03-28T00:24:53.567 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/mon.a/config 2024-03-28T00:24:53.716 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/mon.a/config 2024-03-28T00:24:54.670 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/mon.a/config 2024-03-28T00:24:55.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:24:54 smithi132 ceph-mon[35941]: pgmap v342: 97 pgs: 97 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-28T00:24:55.372 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:24:54 smithi119 ceph-mon[31518]: pgmap v342: 97 pgs: 97 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-28T00:24:55.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:24:54 smithi165 ceph-mon[36302]: pgmap v342: 97 pgs: 97 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-28T00:24:57.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:24:56 smithi119 ceph-mon[31518]: pgmap v343: 97 pgs: 97 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T00:24:57.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:24:56 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:24:57.177 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:24:56 smithi132 ceph-mon[35941]: pgmap v343: 97 pgs: 97 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T00:24:57.177 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:24:56 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:24:57.206 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:56.898+0000 7f0e71e00700 1 -- 172.21.15.119:0/250218037 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0e6c108260 msgr2=0x7f0e6c108640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:24:57.206 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:56.898+0000 7f0e71e00700 1 --2- 172.21.15.119:0/250218037 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0e6c108260 0x7f0e6c108640 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f0e5c0099c0 tx=0x7f0e5c009cd0 comp rx=0 tx=0).stop 2024-03-28T00:24:57.206 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:56.898+0000 7f0e71e00700 1 -- 172.21.15.119:0/250218037 shutdown_connections 2024-03-28T00:24:57.206 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:56.898+0000 7f0e71e00700 1 --2- 172.21.15.119:0/250218037 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e6c102bd0 0x7f0e6c10f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:57.206 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:56.898+0000 7f0e71e00700 1 --2- 172.21.15.119:0/250218037 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0e6c108260 0x7f0e6c108640 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:57.206 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:56.898+0000 7f0e71e00700 1 --2- 172.21.15.119:0/250218037 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0e6c102210 0x7f0e6c102690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:57.206 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:56.898+0000 7f0e71e00700 1 -- 172.21.15.119:0/250218037 >> 172.21.15.119:0/250218037 conn(0x7f0e6c0792e0 msgr2=0x7f0e6c0796f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:24:57.206 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:56.898+0000 7f0e71e00700 1 -- 172.21.15.119:0/250218037 shutdown_connections 2024-03-28T00:24:57.206 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:56.898+0000 7f0e71e00700 1 -- 172.21.15.119:0/250218037 wait complete. 2024-03-28T00:24:57.207 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:56.900+0000 7f0e71e00700 1 Processor -- start 2024-03-28T00:24:57.207 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:56.900+0000 7f0e71e00700 1 -- start start 2024-03-28T00:24:57.207 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:56.900+0000 7f0e71e00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0e6c102210 0x7f0e6c1a0760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:24:57.207 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:56.900+0000 7f0e71e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e6c102bd0 0x7f0e6c1a0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:24:57.207 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:56.900+0000 7f0e71e00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0e6c108260 0x7f0e6c19a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:24:57.207 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:56.900+0000 7f0e71e00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e6c112480 con 0x7f0e6c102bd0 2024-03-28T00:24:57.207 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:56.900+0000 7f0e71e00700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e6c112300 con 0x7f0e6c102210 2024-03-28T00:24:57.207 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:56.900+0000 7f0e71e00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e6c112600 con 0x7f0e6c108260 2024-03-28T00:24:57.207 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:56.901+0000 7f0e6aa00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0e6c102210 0x7f0e6c1a0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:24:57.207 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:56.901+0000 7f0e6aa00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0e6c102210 0x7f0e6c1a0760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.119:56166/0 (socket says 172.21.15.119:56166) 2024-03-28T00:24:57.207 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:56.901+0000 7f0e6aa00700 1 -- 172.21.15.119:0/853005978 learned_addr learned my addr 172.21.15.119:0/853005978 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:24:57.207 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:56.901+0000 7f0e6a000700 1 --2- 172.21.15.119:0/853005978 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e6c102bd0 0x7f0e6c1a0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:24:57.207 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:56.901+0000 7f0e6b400700 1 --2- 172.21.15.119:0/853005978 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0e6c108260 0x7f0e6c19a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:24:57.207 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:56.901+0000 7f0e6aa00700 1 -- 172.21.15.119:0/853005978 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0e6c108260 msgr2=0x7f0e6c19a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:24:57.208 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:56.901+0000 7f0e6aa00700 1 --2- 172.21.15.119:0/853005978 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0e6c108260 0x7f0e6c19a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:57.208 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:56.901+0000 7f0e6aa00700 1 -- 172.21.15.119:0/853005978 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e6c102bd0 msgr2=0x7f0e6c1a0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:24:57.208 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:56.901+0000 7f0e6aa00700 1 --2- 172.21.15.119:0/853005978 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e6c102bd0 0x7f0e6c1a0ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:57.208 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:56.901+0000 7f0e6aa00700 1 -- 172.21.15.119:0/853005978 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e6c19b080 con 0x7f0e6c102210 2024-03-28T00:24:57.208 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:56.901+0000 7f0e6aa00700 1 --2- 172.21.15.119:0/853005978 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0e6c102210 0x7f0e6c1a0760 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f0e5c006010 tx=0x7f0e5c003b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:24:57.208 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:56.902+0000 7f0e70a00700 1 -- 172.21.15.119:0/853005978 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e5c003680 con 0x7f0e6c102210 2024-03-28T00:24:57.208 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:56.902+0000 7f0e71e00700 1 -- 172.21.15.119:0/853005978 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e6c19b310 con 0x7f0e6c102210 2024-03-28T00:24:57.208 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:56.902+0000 7f0e70a00700 1 -- 172.21.15.119:0/853005978 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0e5c0055a0 con 0x7f0e6c102210 2024-03-28T00:24:57.208 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:56.902+0000 7f0e70a00700 1 -- 172.21.15.119:0/853005978 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e5c0173f0 con 0x7f0e6c102210 2024-03-28T00:24:57.208 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:56.902+0000 7f0e71e00700 1 -- 172.21.15.119:0/853005978 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e6c1a7630 con 0x7f0e6c102210 2024-03-28T00:24:57.208 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:56.903+0000 7f0e71e00700 1 -- 172.21.15.119:0/853005978 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e6c1038d0 con 0x7f0e6c102210 2024-03-28T00:24:57.208 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:56.905+0000 7f0e70a00700 1 -- 172.21.15.119:0/853005978 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 17) v1 ==== 137342+0+0 (secure 0 0 0) 0x7f0e5c00f710 con 0x7f0e6c102210 2024-03-28T00:24:57.209 INFO:teuthology.orchestra.run.smithi119.stdout:34359738455 2024-03-28T00:24:57.209 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:56.906+0000 7f0e70a00700 1 --2- 172.21.15.119:0/853005978 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f0e500a2ac0 0x7f0e500a4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:24:57.209 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:56.907+0000 7f0e6a000700 1 --2- 172.21.15.119:0/853005978 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f0e500a2ac0 0x7f0e500a4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:24:57.209 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:56.907+0000 7f0e6a000700 1 --2- 172.21.15.119:0/853005978 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f0e500a2ac0 0x7f0e500a4f80 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c079b90 tx=0x7f0e58009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:24:57.209 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:56.908+0000 7f0e70a00700 1 -- 172.21.15.119:0/853005978 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f0e5c09a350 con 0x7f0e6c102210 2024-03-28T00:24:57.209 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:56.914+0000 7f0e70a00700 1 -- 172.21.15.119:0/853005978 <== mon.2 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f0e5c013070 con 0x7f0e6c102210 2024-03-28T00:24:57.209 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:57.147+0000 7f0e71e00700 1 -- 172.21.15.119:0/853005978 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f0e6c1a7800 con 0x7f0e6c102210 2024-03-28T00:24:57.210 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:57.147+0000 7f0e70a00700 1 -- 172.21.15.119:0/853005978 <== mon.2 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f0e5c096e20 con 0x7f0e6c102210 2024-03-28T00:24:57.210 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:57.150+0000 7f0e71e00700 1 -- 172.21.15.119:0/853005978 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f0e500a2ac0 msgr2=0x7f0e500a4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:24:57.210 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:57.150+0000 7f0e71e00700 1 --2- 172.21.15.119:0/853005978 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f0e500a2ac0 0x7f0e500a4f80 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c079b90 tx=0x7f0e58009260 comp rx=0 tx=0).stop 2024-03-28T00:24:57.210 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:57.150+0000 7f0e71e00700 1 -- 172.21.15.119:0/853005978 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0e6c102210 msgr2=0x7f0e6c1a0760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:24:57.210 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:57.150+0000 7f0e71e00700 1 --2- 172.21.15.119:0/853005978 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0e6c102210 0x7f0e6c1a0760 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f0e5c006010 tx=0x7f0e5c003b40 comp rx=0 tx=0).stop 2024-03-28T00:24:57.210 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:57.150+0000 7f0e71e00700 1 -- 172.21.15.119:0/853005978 shutdown_connections 2024-03-28T00:24:57.210 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:57.150+0000 7f0e71e00700 1 --2- 172.21.15.119:0/853005978 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f0e500a2ac0 0x7f0e500a4f80 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:57.210 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:57.150+0000 7f0e71e00700 1 --2- 172.21.15.119:0/853005978 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0e6c108260 0x7f0e6c19a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:57.210 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:57.150+0000 7f0e71e00700 1 --2- 172.21.15.119:0/853005978 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0e6c102210 0x7f0e6c1a0760 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:57.210 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:57.150+0000 7f0e71e00700 1 --2- 172.21.15.119:0/853005978 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e6c102bd0 0x7f0e6c1a0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:24:57.210 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:57.150+0000 7f0e71e00700 1 -- 172.21.15.119:0/853005978 >> 172.21.15.119:0/853005978 conn(0x7f0e6c0792e0 msgr2=0x7f0e6c10d640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:24:57.210 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:57.150+0000 7f0e71e00700 1 -- 172.21.15.119:0/853005978 shutdown_connections 2024-03-28T00:24:57.210 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:24:57.150+0000 7f0e71e00700 1 -- 172.21.15.119:0/853005978 wait complete. 2024-03-28T00:24:57.303 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:24:56 smithi165 ceph-mon[36302]: pgmap v343: 97 pgs: 97 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T00:24:57.304 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:24:56 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:24:57.426 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/mon.a/config 2024-03-28T00:24:57.431 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/mon.a/config 2024-03-28T00:24:58.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:24:57 smithi119 ceph-mon[31518]: from='client.? 172.21.15.119:0/853005978' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-28T00:24:58.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:24:57 smithi132 ceph-mon[35941]: from='client.? 172.21.15.119:0/853005978' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-28T00:24:58.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:24:57 smithi165 ceph-mon[36302]: from='client.? 172.21.15.119:0/853005978' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-28T00:24:59.084 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:24:58 smithi119 ceph-mon[31518]: pgmap v344: 97 pgs: 97 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2024-03-28T00:24:59.084 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:24:58 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:24:59.084 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:24:58 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:24:59.084 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:24:58 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:24:59.084 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:24:58 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:24:59.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:24:58 smithi132 ceph-mon[35941]: pgmap v344: 97 pgs: 97 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2024-03-28T00:24:59.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:24:58 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:24:59.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:24:58 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:24:59.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:24:58 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:24:59.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:24:58 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:24:59.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:24:58 smithi165 ceph-mon[36302]: pgmap v344: 97 pgs: 97 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2024-03-28T00:24:59.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:24:58 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:24:59.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:24:58 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:24:59.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:24:58 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:24:59.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:24:58 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:25:01.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:25:00 smithi119 ceph-mon[31518]: pgmap v345: 97 pgs: 97 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-03-28T00:25:01.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:25:00 smithi132 ceph-mon[35941]: pgmap v345: 97 pgs: 97 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-03-28T00:25:01.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:25:00 smithi165 ceph-mon[36302]: pgmap v345: 97 pgs: 97 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-03-28T00:25:02.159 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/mon.a/config 2024-03-28T00:25:03.022 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:25:02 smithi119 ceph-mon[31518]: pgmap v346: 97 pgs: 97 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:25:03.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:25:02 smithi132 ceph-mon[35941]: pgmap v346: 97 pgs: 97 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:25:03.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:25:02 smithi165 ceph-mon[36302]: pgmap v346: 97 pgs: 97 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:25:04.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:25:03 smithi119 ceph-mon[31518]: from='client.? 172.21.15.119:0/2577463414' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-28T00:25:04.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:25:03 smithi132 ceph-mon[35941]: from='client.? 172.21.15.119:0/2577463414' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-28T00:25:04.272 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/mon.a/config 2024-03-28T00:25:04.317 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738451 got 34359738455 for osd.0 2024-03-28T00:25:04.317 DEBUG:teuthology.parallel:result is None 2024-03-28T00:25:04.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:25:03 smithi165 ceph-mon[36302]: from='client.? 172.21.15.119:0/2577463414' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-28T00:25:04.404 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:02.777+0000 7fd310a00700 1 -- 172.21.15.119:0/999127622 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd30c104e80 msgr2=0x7fd30c108ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:04.405 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:02.777+0000 7fd310a00700 1 --2- 172.21.15.119:0/999127622 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd30c104e80 0x7fd30c108ed0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd2f80099c0 tx=0x7fd2f8009cd0 comp rx=0 tx=0).stop 2024-03-28T00:25:04.405 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:02.777+0000 7fd310a00700 1 -- 172.21.15.119:0/999127622 shutdown_connections 2024-03-28T00:25:04.405 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:02.777+0000 7fd310a00700 1 --2- 172.21.15.119:0/999127622 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd30c1095b0 0x7fd30c111180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:04.405 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:02.777+0000 7fd310a00700 1 --2- 172.21.15.119:0/999127622 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd30c1044d0 0x7fd30c1048b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:04.405 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:02.777+0000 7fd310a00700 1 --2- 172.21.15.119:0/999127622 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd30c104e80 0x7fd30c108ed0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:04.405 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:02.777+0000 7fd310a00700 1 -- 172.21.15.119:0/999127622 >> 172.21.15.119:0/999127622 conn(0x7fd30c0ffd40 msgr2=0x7fd30c102160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:25:04.405 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:02.777+0000 7fd310a00700 1 -- 172.21.15.119:0/999127622 shutdown_connections 2024-03-28T00:25:04.405 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:02.777+0000 7fd310a00700 1 -- 172.21.15.119:0/999127622 wait complete. 2024-03-28T00:25:04.405 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:02.779+0000 7fd310a00700 1 Processor -- start 2024-03-28T00:25:04.405 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:02.779+0000 7fd310a00700 1 -- start start 2024-03-28T00:25:04.405 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:02.779+0000 7fd310a00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd30c1044d0 0x7fd30c1a0750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:04.405 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:02.779+0000 7fd310a00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd30c104e80 0x7fd30c1a0c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:04.405 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:02.779+0000 7fd310a00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd30c1095b0 0x7fd30c19a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:04.406 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:02.779+0000 7fd310a00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd30c1145f0 con 0x7fd30c104e80 2024-03-28T00:25:04.406 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:02.779+0000 7fd310a00700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd30c114470 con 0x7fd30c1044d0 2024-03-28T00:25:04.406 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:02.780+0000 7fd310a00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd30c114770 con 0x7fd30c1095b0 2024-03-28T00:25:04.406 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:02.780+0000 7fd30aa00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd30c104e80 0x7fd30c1a0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:04.406 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:02.780+0000 7fd30aa00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd30c104e80 0x7fd30c1a0c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:56172/0 (socket says 172.21.15.119:56172) 2024-03-28T00:25:04.406 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:02.780+0000 7fd30aa00700 1 -- 172.21.15.119:0/2577463414 learned_addr learned my addr 172.21.15.119:0/2577463414 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:25:04.406 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:02.780+0000 7fd30be00700 1 --2- 172.21.15.119:0/2577463414 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd30c1095b0 0x7fd30c19a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:04.406 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:02.780+0000 7fd30b400700 1 --2- 172.21.15.119:0/2577463414 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd30c1044d0 0x7fd30c1a0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:04.406 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:02.780+0000 7fd30aa00700 1 -- 172.21.15.119:0/2577463414 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd30c1095b0 msgr2=0x7fd30c19a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:04.406 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:02.780+0000 7fd30aa00700 1 --2- 172.21.15.119:0/2577463414 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd30c1095b0 0x7fd30c19a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:04.406 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:02.780+0000 7fd30aa00700 1 -- 172.21.15.119:0/2577463414 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd30c1044d0 msgr2=0x7fd30c1a0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:04.406 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:02.780+0000 7fd30aa00700 1 --2- 172.21.15.119:0/2577463414 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd30c1044d0 0x7fd30c1a0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:04.407 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:02.780+0000 7fd30aa00700 1 -- 172.21.15.119:0/2577463414 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd30c19b070 con 0x7fd30c104e80 2024-03-28T00:25:04.407 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:02.781+0000 7fd30aa00700 1 --2- 172.21.15.119:0/2577463414 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd30c104e80 0x7fd30c1a0c90 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fd2f800fa00 tx=0x7fd2f800fa30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:25:04.407 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:02.781+0000 7fd303e00700 1 -- 172.21.15.119:0/2577463414 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2f8027070 con 0x7fd30c104e80 2024-03-28T00:25:04.407 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:02.781+0000 7fd310a00700 1 -- 172.21.15.119:0/2577463414 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd30c19b300 con 0x7fd30c104e80 2024-03-28T00:25:04.407 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:02.781+0000 7fd303e00700 1 -- 172.21.15.119:0/2577463414 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd2f8003680 con 0x7fd30c104e80 2024-03-28T00:25:04.407 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:02.781+0000 7fd303e00700 1 -- 172.21.15.119:0/2577463414 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2f80173f0 con 0x7fd30c104e80 2024-03-28T00:25:04.407 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:02.781+0000 7fd310a00700 1 -- 172.21.15.119:0/2577463414 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd30c1a7620 con 0x7fd30c104e80 2024-03-28T00:25:04.407 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:02.782+0000 7fd310a00700 1 -- 172.21.15.119:0/2577463414 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd30c1059b0 con 0x7fd30c104e80 2024-03-28T00:25:04.407 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:02.784+0000 7fd303e00700 1 -- 172.21.15.119:0/2577463414 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 17) v1 ==== 137342+0+0 (secure 0 0 0) 0x7fd2f802a070 con 0x7fd30c104e80 2024-03-28T00:25:04.407 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:02.786+0000 7fd303e00700 1 --2- 172.21.15.119:0/2577463414 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fd2f00a2920 0x7fd2f00a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:04.407 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:02.786+0000 7fd303e00700 1 -- 172.21.15.119:0/2577463414 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fd2f80cf7f0 con 0x7fd30c104e80 2024-03-28T00:25:04.407 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:02.786+0000 7fd30b400700 1 --2- 172.21.15.119:0/2577463414 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fd2f00a2920 0x7fd2f00a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:04.407 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:02.786+0000 7fd30b400700 1 --2- 172.21.15.119:0/2577463414 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fd2f00a2920 0x7fd2f00a4de0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fd30c19ba50 tx=0x7fd2fc009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:25:04.407 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:02.790+0000 7fd303e00700 1 -- 172.21.15.119:0/2577463414 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd2f805e740 con 0x7fd30c104e80 2024-03-28T00:25:04.408 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:03.017+0000 7fd310a00700 1 -- 172.21.15.119:0/2577463414 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fd30c02da30 con 0x7fd30c104e80 2024-03-28T00:25:04.408 INFO:teuthology.orchestra.run.smithi119.stdout:55834574931 2024-03-28T00:25:04.408 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:03.018+0000 7fd303e00700 1 -- 172.21.15.119:0/2577463414 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fd2f8013070 con 0x7fd30c104e80 2024-03-28T00:25:04.408 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:03.022+0000 7fd310a00700 1 -- 172.21.15.119:0/2577463414 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fd2f00a2920 msgr2=0x7fd2f00a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:04.408 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:03.022+0000 7fd310a00700 1 --2- 172.21.15.119:0/2577463414 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fd2f00a2920 0x7fd2f00a4de0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fd30c19ba50 tx=0x7fd2fc009190 comp rx=0 tx=0).stop 2024-03-28T00:25:04.408 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:03.022+0000 7fd310a00700 1 -- 172.21.15.119:0/2577463414 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd30c104e80 msgr2=0x7fd30c1a0c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:04.408 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:03.022+0000 7fd310a00700 1 --2- 172.21.15.119:0/2577463414 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd30c104e80 0x7fd30c1a0c90 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fd2f800fa00 tx=0x7fd2f800fa30 comp rx=0 tx=0).stop 2024-03-28T00:25:04.409 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:03.022+0000 7fd310a00700 1 -- 172.21.15.119:0/2577463414 shutdown_connections 2024-03-28T00:25:04.409 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:03.022+0000 7fd310a00700 1 --2- 172.21.15.119:0/2577463414 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fd2f00a2920 0x7fd2f00a4de0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:04.409 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:03.022+0000 7fd310a00700 1 --2- 172.21.15.119:0/2577463414 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd30c1095b0 0x7fd30c19a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:04.409 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:03.022+0000 7fd310a00700 1 --2- 172.21.15.119:0/2577463414 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd30c1044d0 0x7fd30c1a0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:04.409 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:03.022+0000 7fd310a00700 1 --2- 172.21.15.119:0/2577463414 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd30c104e80 0x7fd30c1a0c90 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:04.409 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:03.022+0000 7fd310a00700 1 -- 172.21.15.119:0/2577463414 >> 172.21.15.119:0/2577463414 conn(0x7fd30c0ffd40 msgr2=0x7fd30c102060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:25:04.409 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:03.022+0000 7fd310a00700 1 -- 172.21.15.119:0/2577463414 shutdown_connections 2024-03-28T00:25:04.409 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:03.022+0000 7fd310a00700 1 -- 172.21.15.119:0/2577463414 wait complete. 2024-03-28T00:25:05.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:25:04 smithi132 ceph-mon[35941]: pgmap v347: 97 pgs: 97 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-28T00:25:05.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:25:04 smithi165 ceph-mon[36302]: pgmap v347: 97 pgs: 97 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-28T00:25:05.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:25:04 smithi119 ceph-mon[31518]: pgmap v347: 97 pgs: 97 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-28T00:25:07.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:25:06 smithi119 ceph-mon[31518]: pgmap v348: 97 pgs: 97 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T00:25:07.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:25:06 smithi132 ceph-mon[35941]: pgmap v348: 97 pgs: 97 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T00:25:07.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:25:06 smithi165 ceph-mon[36302]: pgmap v348: 97 pgs: 97 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T00:25:08.281 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:07.871+0000 7f04a6c00700 1 -- 172.21.15.119:0/803138628 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f04a01081e0 msgr2=0x7f04a01085c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:08.281 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:07.871+0000 7f04a6c00700 1 --2- 172.21.15.119:0/803138628 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f04a01081e0 0x7f04a01085c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f04900099c0 tx=0x7f0490009cd0 comp rx=0 tx=0).stop 2024-03-28T00:25:08.281 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:07.872+0000 7f04a6c00700 1 -- 172.21.15.119:0/803138628 shutdown_connections 2024-03-28T00:25:08.281 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:07.872+0000 7f04a6c00700 1 --2- 172.21.15.119:0/803138628 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f04a006a7e0 0x7f04a010f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:08.282 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:07.872+0000 7f04a6c00700 1 --2- 172.21.15.119:0/803138628 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f04a01081e0 0x7f04a01085c0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:08.282 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:07.872+0000 7f04a6c00700 1 --2- 172.21.15.119:0/803138628 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f04a0069e40 0x7f04a006a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:08.282 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:07.872+0000 7f04a6c00700 1 -- 172.21.15.119:0/803138628 >> 172.21.15.119:0/803138628 conn(0x7f04a0077fc0 msgr2=0x7f04a00783d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:25:08.282 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:07.872+0000 7f04a6c00700 1 -- 172.21.15.119:0/803138628 shutdown_connections 2024-03-28T00:25:08.282 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:07.872+0000 7f04a6c00700 1 -- 172.21.15.119:0/803138628 wait complete. 2024-03-28T00:25:08.282 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:07.873+0000 7f04a6c00700 1 Processor -- start 2024-03-28T00:25:08.282 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:07.873+0000 7f04a6c00700 1 -- start start 2024-03-28T00:25:08.282 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:07.873+0000 7f04a6c00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f04a0069e40 0x7f04a0101160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:08.282 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:07.873+0000 7f04a6c00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f04a006a7e0 0x7f04a01016a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:08.282 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:07.873+0000 7f04a6c00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f04a01081e0 0x7f04a0101be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:08.282 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:07.873+0000 7f04a6c00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04a0112470 con 0x7f04a0069e40 2024-03-28T00:25:08.282 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:07.873+0000 7f04a6c00700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04a01122f0 con 0x7f04a01081e0 2024-03-28T00:25:08.283 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:07.873+0000 7f04a6c00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04a01125f0 con 0x7f04a006a7e0 2024-03-28T00:25:08.283 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:07.874+0000 7f049f400700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f04a006a7e0 0x7f04a01016a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:08.283 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:07.874+0000 7f049fe00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f04a0069e40 0x7f04a0101160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:08.283 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:07.874+0000 7f049f400700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f04a006a7e0 0x7f04a01016a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.119:56122/0 (socket says 172.21.15.119:56122) 2024-03-28T00:25:08.283 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:07.874+0000 7f049f400700 1 -- 172.21.15.119:0/2622429113 learned_addr learned my addr 172.21.15.119:0/2622429113 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:25:08.283 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:07.874+0000 7f04a4c00700 1 --2- 172.21.15.119:0/2622429113 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f04a01081e0 0x7f04a0101be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:08.283 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:07.874+0000 7f049f400700 1 -- 172.21.15.119:0/2622429113 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f04a01081e0 msgr2=0x7f04a0101be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:08.283 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:07.874+0000 7f049f400700 1 --2- 172.21.15.119:0/2622429113 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f04a01081e0 0x7f04a0101be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:08.283 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:07.874+0000 7f049f400700 1 -- 172.21.15.119:0/2622429113 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f04a0069e40 msgr2=0x7f04a0101160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:08.283 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:07.874+0000 7f049f400700 1 --2- 172.21.15.119:0/2622429113 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f04a0069e40 0x7f04a0101160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:08.283 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:07.874+0000 7f049f400700 1 -- 172.21.15.119:0/2622429113 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04a01a6e50 con 0x7f04a006a7e0 2024-03-28T00:25:08.283 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:07.874+0000 7f049f400700 1 --2- 172.21.15.119:0/2622429113 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f04a006a7e0 0x7f04a01016a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f048c00eb30 tx=0x7f048c00ee40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:25:08.283 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:07.875+0000 7f049cc00700 1 -- 172.21.15.119:0/2622429113 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f048c00cbd0 con 0x7f04a006a7e0 2024-03-28T00:25:08.284 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:07.875+0000 7f049cc00700 1 -- 172.21.15.119:0/2622429113 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f048c004500 con 0x7f04a006a7e0 2024-03-28T00:25:08.284 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:07.875+0000 7f04a6c00700 1 -- 172.21.15.119:0/2622429113 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04a01a7140 con 0x7f04a006a7e0 2024-03-28T00:25:08.284 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:07.875+0000 7f049cc00700 1 -- 172.21.15.119:0/2622429113 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f048c010430 con 0x7f04a006a7e0 2024-03-28T00:25:08.284 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:07.875+0000 7f04a6c00700 1 -- 172.21.15.119:0/2622429113 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04a01a7680 con 0x7f04a006a7e0 2024-03-28T00:25:08.284 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:07.876+0000 7f04a6c00700 1 -- 172.21.15.119:0/2622429113 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04a0068e50 con 0x7f04a006a7e0 2024-03-28T00:25:08.284 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:07.878+0000 7f049cc00700 1 -- 172.21.15.119:0/2622429113 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 17) v1 ==== 137342+0+0 (secure 0 0 0) 0x7f048c003ed0 con 0x7f04a006a7e0 2024-03-28T00:25:08.284 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:07.879+0000 7f049cc00700 1 --2- 172.21.15.119:0/2622429113 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f04840a2920 0x7f04840a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:08.284 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:07.879+0000 7f049cc00700 1 -- 172.21.15.119:0/2622429113 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f048c0cef70 con 0x7f04a006a7e0 2024-03-28T00:25:08.284 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:07.879+0000 7f049fe00700 1 --2- 172.21.15.119:0/2622429113 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f04840a2920 0x7f04840a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:08.284 INFO:teuthology.orchestra.run.smithi119.stdout:219043332146 2024-03-28T00:25:08.285 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:07.880+0000 7f049fe00700 1 --2- 172.21.15.119:0/2622429113 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f04840a2920 0x7f04840a4de0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f0490000c00 tx=0x7f0490005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:25:08.285 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:07.886+0000 7f049cc00700 1 -- 172.21.15.119:0/2622429113 <== mon.1 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f048c09beb0 con 0x7f04a006a7e0 2024-03-28T00:25:08.285 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:08.113+0000 7f04a6c00700 1 -- 172.21.15.119:0/2622429113 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f04a002da30 con 0x7f04a006a7e0 2024-03-28T00:25:08.285 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:08.114+0000 7f049cc00700 1 -- 172.21.15.119:0/2622429113 <== mon.1 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f048c09b7b0 con 0x7f04a006a7e0 2024-03-28T00:25:08.285 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:08.117+0000 7f04a6c00700 1 -- 172.21.15.119:0/2622429113 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f04840a2920 msgr2=0x7f04840a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:08.285 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:08.117+0000 7f04a6c00700 1 --2- 172.21.15.119:0/2622429113 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f04840a2920 0x7f04840a4de0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f0490000c00 tx=0x7f0490005fb0 comp rx=0 tx=0).stop 2024-03-28T00:25:08.286 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:08.117+0000 7f04a6c00700 1 -- 172.21.15.119:0/2622429113 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f04a006a7e0 msgr2=0x7f04a01016a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:08.286 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:08.117+0000 7f04a6c00700 1 --2- 172.21.15.119:0/2622429113 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f04a006a7e0 0x7f04a01016a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f048c00eb30 tx=0x7f048c00ee40 comp rx=0 tx=0).stop 2024-03-28T00:25:08.286 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:08.117+0000 7f04a6c00700 1 -- 172.21.15.119:0/2622429113 shutdown_connections 2024-03-28T00:25:08.286 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:08.117+0000 7f04a6c00700 1 --2- 172.21.15.119:0/2622429113 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f04840a2920 0x7f04840a4de0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:08.286 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:08.117+0000 7f04a6c00700 1 --2- 172.21.15.119:0/2622429113 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f04a01081e0 0x7f04a0101be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:08.286 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:08.117+0000 7f04a6c00700 1 --2- 172.21.15.119:0/2622429113 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f04a0069e40 0x7f04a0101160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:08.286 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:08.117+0000 7f04a6c00700 1 --2- 172.21.15.119:0/2622429113 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f04a006a7e0 0x7f04a01016a0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:08.286 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:08.117+0000 7f04a6c00700 1 -- 172.21.15.119:0/2622429113 >> 172.21.15.119:0/2622429113 conn(0x7f04a0077fc0 msgr2=0x7f04a00ffc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:25:08.286 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:08.117+0000 7f04a6c00700 1 -- 172.21.15.119:0/2622429113 shutdown_connections 2024-03-28T00:25:08.286 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:08.117+0000 7f04a6c00700 1 -- 172.21.15.119:0/2622429113 wait complete. 2024-03-28T00:25:09.016 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:09.010+0000 7f45d1800700 1 -- 172.21.15.119:0/3296112687 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f45cc074ce0 msgr2=0x7f45cc10e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:09.016 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:09.010+0000 7f45d1800700 1 --2- 172.21.15.119:0/3296112687 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f45cc074ce0 0x7f45cc10e650 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f45b8009990 tx=0x7f45b8009ca0 comp rx=0 tx=0).stop 2024-03-28T00:25:09.016 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:09.010+0000 7f45d1800700 1 -- 172.21.15.119:0/3296112687 shutdown_connections 2024-03-28T00:25:09.016 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:09.010+0000 7f45d1800700 1 --2- 172.21.15.119:0/3296112687 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f45cc10eb90 0x7f45cc110f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:09.016 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:09.010+0000 7f45d1800700 1 --2- 172.21.15.119:0/3296112687 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f45cc0743c0 0x7f45cc0747a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:09.016 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:09.010+0000 7f45d1800700 1 --2- 172.21.15.119:0/3296112687 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f45cc074ce0 0x7f45cc10e650 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:09.016 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:09.010+0000 7f45d1800700 1 -- 172.21.15.119:0/3296112687 >> 172.21.15.119:0/3296112687 conn(0x7f45cc0792c0 msgr2=0x7f45cc0796d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:25:09.016 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:09.010+0000 7f45d1800700 1 -- 172.21.15.119:0/3296112687 shutdown_connections 2024-03-28T00:25:09.016 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:09.010+0000 7f45d1800700 1 -- 172.21.15.119:0/3296112687 wait complete. 2024-03-28T00:25:09.017 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:09.011+0000 7f45d1800700 1 Processor -- start 2024-03-28T00:25:09.017 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:09.012+0000 7f45d1800700 1 -- start start 2024-03-28T00:25:09.017 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:09.012+0000 7f45d1800700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f45cc0743c0 0x7f45cc1a07b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:09.017 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:09.012+0000 7f45d1800700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f45cc074ce0 0x7f45cc1a0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:09.017 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:09.012+0000 7f45d1800700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f45cc10eb90 0x7f45cc19a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:09.017 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:09.012+0000 7f45d1800700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45cc1146b0 con 0x7f45cc0743c0 2024-03-28T00:25:09.017 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:09.012+0000 7f45d1800700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45cc114530 con 0x7f45cc10eb90 2024-03-28T00:25:09.017 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:09.012+0000 7f45d1800700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45cc114830 con 0x7f45cc074ce0 2024-03-28T00:25:09.018 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:09.012+0000 7f45cb400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f45cc10eb90 0x7f45cc19a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:09.019 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:09.012+0000 7f45caa00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f45cc0743c0 0x7f45cc1a07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:09.019 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:09.012+0000 7f45cb400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f45cc10eb90 0x7f45cc19a970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.119:53020/0 (socket says 172.21.15.119:53020) 2024-03-28T00:25:09.019 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:09.012+0000 7f45cb400700 1 -- 172.21.15.119:0/1159884202 learned_addr learned my addr 172.21.15.119:0/1159884202 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:25:09.019 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:09.012+0000 7f45ca000700 1 --2- 172.21.15.119:0/1159884202 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f45cc074ce0 0x7f45cc1a0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:09.019 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:09.013+0000 7f45cb400700 1 -- 172.21.15.119:0/1159884202 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f45cc074ce0 msgr2=0x7f45cc1a0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:09.019 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:09.013+0000 7f45cb400700 1 --2- 172.21.15.119:0/1159884202 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f45cc074ce0 0x7f45cc1a0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:09.019 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:09.013+0000 7f45cb400700 1 -- 172.21.15.119:0/1159884202 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f45cc0743c0 msgr2=0x7f45cc1a07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:09.019 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:09.013+0000 7f45cb400700 1 --2- 172.21.15.119:0/1159884202 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f45cc0743c0 0x7f45cc1a07b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:09.019 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:09.013+0000 7f45cb400700 1 -- 172.21.15.119:0/1159884202 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45cc19b0d0 con 0x7f45cc10eb90 2024-03-28T00:25:09.019 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:09.013+0000 7f45caa00700 1 --2- 172.21.15.119:0/1159884202 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f45cc0743c0 0x7f45cc1a07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T00:25:09.019 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:09.013+0000 7f45cb400700 1 --2- 172.21.15.119:0/1159884202 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f45cc10eb90 0x7f45cc19a970 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f45c400ef30 tx=0x7f45c400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:25:09.019 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:09.013+0000 7f45c3400700 1 -- 172.21.15.119:0/1159884202 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45c40092e0 con 0x7f45cc10eb90 2024-03-28T00:25:09.019 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:09.013+0000 7f45c3400700 1 -- 172.21.15.119:0/1159884202 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f45c4004bb0 con 0x7f45cc10eb90 2024-03-28T00:25:09.019 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:09.013+0000 7f45c3400700 1 -- 172.21.15.119:0/1159884202 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45c4004280 con 0x7f45cc10eb90 2024-03-28T00:25:09.020 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:09.013+0000 7f45d1800700 1 -- 172.21.15.119:0/1159884202 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45cc19b360 con 0x7f45cc10eb90 2024-03-28T00:25:09.020 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:09.014+0000 7f45d1800700 1 -- 172.21.15.119:0/1159884202 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45cc1a7680 con 0x7f45cc10eb90 2024-03-28T00:25:09.020 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:09.015+0000 7f45d1800700 1 -- 172.21.15.119:0/1159884202 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45cc068e50 con 0x7f45cc10eb90 2024-03-28T00:25:09.022 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:09.017+0000 7f45c3400700 1 -- 172.21.15.119:0/1159884202 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 17) v1 ==== 137342+0+0 (secure 0 0 0) 0x7f45c4012050 con 0x7f45cc10eb90 2024-03-28T00:25:09.023 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:09.018+0000 7f45c3400700 1 --2- 172.21.15.119:0/1159884202 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f45b00a29a0 0x7f45b00a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:09.023 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:09.018+0000 7f45c3400700 1 -- 172.21.15.119:0/1159884202 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f45c40ce640 con 0x7f45cc10eb90 2024-03-28T00:25:09.024 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:09.018+0000 7f45caa00700 1 --2- 172.21.15.119:0/1159884202 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f45b00a29a0 0x7f45b00a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:09.024 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:09.019+0000 7f45caa00700 1 --2- 172.21.15.119:0/1159884202 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f45b00a29a0 0x7f45b00a4e60 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f45cc19bab0 tx=0x7f45bc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:25:09.025 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:09.020+0000 7f45c3400700 1 -- 172.21.15.119:0/1159884202 <== mon.2 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f45c4010040 con 0x7f45cc10eb90 2024-03-28T00:25:09.115 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:25:08 smithi119 ceph-mon[31518]: pgmap v349: 97 pgs: 97 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-28T00:25:09.115 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:25:08 smithi119 ceph-mon[31518]: from='client.? 172.21.15.119:0/2622429113' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-28T00:25:09.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:25:08 smithi132 ceph-mon[35941]: pgmap v349: 97 pgs: 97 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-28T00:25:09.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:25:08 smithi132 ceph-mon[35941]: from='client.? 172.21.15.119:0/2622429113' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-28T00:25:09.259 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:09.253+0000 7f45d1800700 1 -- 172.21.15.119:0/1159884202 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f45cc19bea0 con 0x7f45cc10eb90 2024-03-28T00:25:09.260 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:09.253+0000 7f45c3400700 1 -- 172.21.15.119:0/1159884202 <== mon.2 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f45c409b580 con 0x7f45cc10eb90 2024-03-28T00:25:09.260 INFO:teuthology.orchestra.run.smithi119.stdout:197568495672 2024-03-28T00:25:09.264 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:09.257+0000 7f45d1800700 1 -- 172.21.15.119:0/1159884202 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f45b00a29a0 msgr2=0x7f45b00a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:09.264 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:09.257+0000 7f45d1800700 1 --2- 172.21.15.119:0/1159884202 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f45b00a29a0 0x7f45b00a4e60 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f45cc19bab0 tx=0x7f45bc009260 comp rx=0 tx=0).stop 2024-03-28T00:25:09.264 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:09.257+0000 7f45d1800700 1 -- 172.21.15.119:0/1159884202 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f45cc10eb90 msgr2=0x7f45cc19a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:09.264 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:09.257+0000 7f45d1800700 1 --2- 172.21.15.119:0/1159884202 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f45cc10eb90 0x7f45cc19a970 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f45c400ef30 tx=0x7f45c400c3a0 comp rx=0 tx=0).stop 2024-03-28T00:25:09.264 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:09.258+0000 7f45d1800700 1 -- 172.21.15.119:0/1159884202 shutdown_connections 2024-03-28T00:25:09.265 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:09.258+0000 7f45d1800700 1 --2- 172.21.15.119:0/1159884202 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f45b00a29a0 0x7f45b00a4e60 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:09.265 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:09.258+0000 7f45d1800700 1 --2- 172.21.15.119:0/1159884202 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f45cc10eb90 0x7f45cc19a970 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:09.265 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:09.258+0000 7f45d1800700 1 --2- 172.21.15.119:0/1159884202 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f45cc0743c0 0x7f45cc1a07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:09.265 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:09.258+0000 7f45d1800700 1 --2- 172.21.15.119:0/1159884202 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f45cc074ce0 0x7f45cc1a0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:09.265 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:09.258+0000 7f45d1800700 1 -- 172.21.15.119:0/1159884202 >> 172.21.15.119:0/1159884202 conn(0x7f45cc0792c0 msgr2=0x7f45cc100450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:25:09.265 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:09.258+0000 7f45d1800700 1 -- 172.21.15.119:0/1159884202 shutdown_connections 2024-03-28T00:25:09.265 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:09.258+0000 7f45d1800700 1 -- 172.21.15.119:0/1159884202 wait complete. 2024-03-28T00:25:09.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:25:08 smithi165 ceph-mon[36302]: pgmap v349: 97 pgs: 97 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-28T00:25:09.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:25:08 smithi165 ceph-mon[36302]: from='client.? 172.21.15.119:0/2622429113' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-28T00:25:10.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:25:09 smithi132 ceph-mon[35941]: from='client.? 172.21.15.119:0/1159884202' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-28T00:25:10.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:25:09 smithi165 ceph-mon[36302]: from='client.? 172.21.15.119:0/1159884202' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-28T00:25:10.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:25:09 smithi119 ceph-mon[31518]: from='client.? 172.21.15.119:0/1159884202' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-28T00:25:11.182 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574927 got 55834574931 for osd.1 2024-03-28T00:25:11.182 DEBUG:teuthology.parallel:result is None 2024-03-28T00:25:11.294 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:25:10 smithi119 ceph-mon[31518]: pgmap v350: 97 pgs: 97 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-28T00:25:11.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:25:10 smithi165 ceph-mon[36302]: pgmap v350: 97 pgs: 97 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-28T00:25:11.452 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:25:10 smithi132 ceph-mon[35941]: pgmap v350: 97 pgs: 97 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-28T00:25:11.637 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:11.454+0000 7fea47400700 1 -- 172.21.15.119:0/1932521085 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fea40109160 msgr2=0x7fea40115670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:11.637 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:11.454+0000 7fea47400700 1 --2- 172.21.15.119:0/1932521085 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fea40109160 0x7fea40115670 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fea3800b0a0 tx=0x7fea3800b3b0 comp rx=0 tx=0).stop 2024-03-28T00:25:11.638 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:11.455+0000 7fea47400700 1 -- 172.21.15.119:0/1932521085 shutdown_connections 2024-03-28T00:25:11.638 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:11.455+0000 7fea47400700 1 --2- 172.21.15.119:0/1932521085 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fea40109160 0x7fea40115670 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:11.638 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:11.455+0000 7fea47400700 1 --2- 172.21.15.119:0/1932521085 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fea4010e7a0 0x7fea4010eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:11.638 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:11.455+0000 7fea47400700 1 --2- 172.21.15.119:0/1932521085 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fea401087a0 0x7fea40108c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:11.638 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:11.455+0000 7fea47400700 1 -- 172.21.15.119:0/1932521085 >> 172.21.15.119:0/1932521085 conn(0x7fea40075960 msgr2=0x7fea40077d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:25:11.638 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:11.455+0000 7fea47400700 1 -- 172.21.15.119:0/1932521085 shutdown_connections 2024-03-28T00:25:11.638 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:11.455+0000 7fea47400700 1 -- 172.21.15.119:0/1932521085 wait complete. 2024-03-28T00:25:11.638 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:11.456+0000 7fea47400700 1 Processor -- start 2024-03-28T00:25:11.638 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:11.456+0000 7fea47400700 1 -- start start 2024-03-28T00:25:11.638 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:11.457+0000 7fea47400700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fea401087a0 0x7fea4019ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:11.638 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:11.457+0000 7fea47400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fea40109160 0x7fea4019f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:11.638 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:11.457+0000 7fea47400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fea4010e7a0 0x7fea401a37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:11.638 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:11.457+0000 7fea47400700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea40118b80 con 0x7fea40109160 2024-03-28T00:25:11.639 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:11.457+0000 7fea47400700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea40118a00 con 0x7fea4010e7a0 2024-03-28T00:25:11.639 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:11.457+0000 7fea47400700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea40118d00 con 0x7fea401087a0 2024-03-28T00:25:11.639 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:11.457+0000 7fea44a00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fea401087a0 0x7fea4019ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:11.639 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:11.457+0000 7fea44a00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fea401087a0 0x7fea4019ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.119:56152/0 (socket says 172.21.15.119:56152) 2024-03-28T00:25:11.639 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:11.457+0000 7fea44a00700 1 -- 172.21.15.119:0/1746557529 learned_addr learned my addr 172.21.15.119:0/1746557529 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:25:11.639 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:11.457+0000 7fea3fe00700 1 --2- 172.21.15.119:0/1746557529 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fea40109160 0x7fea4019f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:11.639 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:11.457+0000 7fea45400700 1 --2- 172.21.15.119:0/1746557529 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fea4010e7a0 0x7fea401a37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:11.639 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:11.457+0000 7fea44a00700 1 -- 172.21.15.119:0/1746557529 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fea4010e7a0 msgr2=0x7fea401a37e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:11.639 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:11.457+0000 7fea44a00700 1 --2- 172.21.15.119:0/1746557529 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fea4010e7a0 0x7fea401a37e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:11.639 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:11.457+0000 7fea44a00700 1 -- 172.21.15.119:0/1746557529 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fea40109160 msgr2=0x7fea4019f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:11.639 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:11.457+0000 7fea44a00700 1 --2- 172.21.15.119:0/1746557529 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fea40109160 0x7fea4019f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:11.639 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:11.457+0000 7fea44a00700 1 -- 172.21.15.119:0/1746557529 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea401a3f20 con 0x7fea401087a0 2024-03-28T00:25:11.639 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:11.458+0000 7fea44a00700 1 --2- 172.21.15.119:0/1746557529 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fea401087a0 0x7fea4019ef10 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fea3400e9c0 tx=0x7fea3400ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:25:11.640 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:11.458+0000 7fea3d600700 1 -- 172.21.15.119:0/1746557529 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea3400cbd0 con 0x7fea401087a0 2024-03-28T00:25:11.640 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:11.458+0000 7fea3d600700 1 -- 172.21.15.119:0/1746557529 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fea3400ee60 con 0x7fea401087a0 2024-03-28T00:25:11.640 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:11.458+0000 7fea47400700 1 -- 172.21.15.119:0/1746557529 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea401a4210 con 0x7fea401087a0 2024-03-28T00:25:11.640 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:11.458+0000 7fea3d600700 1 -- 172.21.15.119:0/1746557529 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea34005630 con 0x7fea401087a0 2024-03-28T00:25:11.640 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:11.458+0000 7fea47400700 1 -- 172.21.15.119:0/1746557529 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea401aba90 con 0x7fea401087a0 2024-03-28T00:25:11.640 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:11.459+0000 7fea47400700 1 -- 172.21.15.119:0/1746557529 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea40068e50 con 0x7fea401087a0 2024-03-28T00:25:11.640 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:11.461+0000 7fea3d600700 1 -- 172.21.15.119:0/1746557529 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 17) v1 ==== 137342+0+0 (secure 0 0 0) 0x7fea34004750 con 0x7fea401087a0 2024-03-28T00:25:11.640 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:11.462+0000 7fea3d600700 1 --2- 172.21.15.119:0/1746557529 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fea280a2920 0x7fea280a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:11.640 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:11.463+0000 7fea3d600700 1 -- 172.21.15.119:0/1746557529 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fea340cee00 con 0x7fea401087a0 2024-03-28T00:25:11.640 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:11.463+0000 7fea3fe00700 1 --2- 172.21.15.119:0/1746557529 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fea280a2920 0x7fea280a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:11.640 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:11.463+0000 7fea3fe00700 1 --2- 172.21.15.119:0/1746557529 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fea280a2920 0x7fea280a4de0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fea4010dc80 tx=0x7fea2c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:25:11.640 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:11.466+0000 7fea3d600700 1 -- 172.21.15.119:0/1746557529 <== mon.1 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fea3409bd40 con 0x7fea401087a0 2024-03-28T00:25:11.710 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:11.704+0000 7fea47400700 1 -- 172.21.15.119:0/1746557529 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) v1 -- 0x7fea401abc60 con 0x7fea401087a0 2024-03-28T00:25:11.711 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:11.704+0000 7fea3d600700 1 -- 172.21.15.119:0/1746557529 <== mon.1 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fea34097020 con 0x7fea401087a0 2024-03-28T00:25:11.711 INFO:teuthology.orchestra.run.smithi119.stdout:253403070511 2024-03-28T00:25:11.714 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:11.707+0000 7fea47400700 1 -- 172.21.15.119:0/1746557529 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fea280a2920 msgr2=0x7fea280a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:11.714 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:11.707+0000 7fea47400700 1 --2- 172.21.15.119:0/1746557529 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fea280a2920 0x7fea280a4de0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fea4010dc80 tx=0x7fea2c009260 comp rx=0 tx=0).stop 2024-03-28T00:25:11.714 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:11.707+0000 7fea47400700 1 -- 172.21.15.119:0/1746557529 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fea401087a0 msgr2=0x7fea4019ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:11.714 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:11.707+0000 7fea47400700 1 --2- 172.21.15.119:0/1746557529 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fea401087a0 0x7fea4019ef10 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fea3400e9c0 tx=0x7fea3400ecd0 comp rx=0 tx=0).stop 2024-03-28T00:25:11.714 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:11.707+0000 7fea47400700 1 -- 172.21.15.119:0/1746557529 shutdown_connections 2024-03-28T00:25:11.714 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:11.707+0000 7fea47400700 1 --2- 172.21.15.119:0/1746557529 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fea280a2920 0x7fea280a4de0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:11.714 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:11.707+0000 7fea47400700 1 --2- 172.21.15.119:0/1746557529 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fea4010e7a0 0x7fea401a37e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:11.714 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:11.707+0000 7fea47400700 1 --2- 172.21.15.119:0/1746557529 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fea401087a0 0x7fea4019ef10 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:11.714 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:11.707+0000 7fea47400700 1 --2- 172.21.15.119:0/1746557529 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fea40109160 0x7fea4019f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:11.714 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:11.707+0000 7fea47400700 1 -- 172.21.15.119:0/1746557529 >> 172.21.15.119:0/1746557529 conn(0x7fea40075960 msgr2=0x7fea40075e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:25:11.714 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:11.708+0000 7fea47400700 1 -- 172.21.15.119:0/1746557529 shutdown_connections 2024-03-28T00:25:11.714 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:11.708+0000 7fea47400700 1 -- 172.21.15.119:0/1746557529 wait complete. 2024-03-28T00:25:12.232 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:25:11 smithi119 ceph-mon[31518]: from='client.? 172.21.15.119:0/1746557529' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-03-28T00:25:12.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:25:11 smithi165 ceph-mon[36302]: from='client.? 172.21.15.119:0/1746557529' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-03-28T00:25:12.452 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:25:11 smithi132 ceph-mon[35941]: from='client.? 172.21.15.119:0/1746557529' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-03-28T00:25:12.616 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:12.609+0000 7f6838a00700 1 -- 172.21.15.119:0/2487972484 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6834100940 msgr2=0x7f6834100dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:12.616 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:12.609+0000 7f6838a00700 1 --2- 172.21.15.119:0/2487972484 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6834100940 0x7f6834100dc0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f681c009990 tx=0x7f681c009ca0 comp rx=0 tx=0).stop 2024-03-28T00:25:12.617 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:12.609+0000 7f6838a00700 1 -- 172.21.15.119:0/2487972484 shutdown_connections 2024-03-28T00:25:12.617 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:12.609+0000 7f6838a00700 1 --2- 172.21.15.119:0/2487972484 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6834101300 0x7f68341110d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:12.617 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:12.609+0000 7f6838a00700 1 --2- 172.21.15.119:0/2487972484 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6834102ed0 0x7f68341032b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:12.617 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:12.609+0000 7f6838a00700 1 --2- 172.21.15.119:0/2487972484 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6834100940 0x7f6834100dc0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:12.617 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:12.609+0000 7f6838a00700 1 -- 172.21.15.119:0/2487972484 >> 172.21.15.119:0/2487972484 conn(0x7f68340762d0 msgr2=0x7f68340766e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:25:12.617 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:12.610+0000 7f6838a00700 1 -- 172.21.15.119:0/2487972484 shutdown_connections 2024-03-28T00:25:12.617 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:12.610+0000 7f6838a00700 1 -- 172.21.15.119:0/2487972484 wait complete. 2024-03-28T00:25:12.617 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:12.611+0000 7f6838a00700 1 Processor -- start 2024-03-28T00:25:12.617 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:12.611+0000 7f6838a00700 1 -- start start 2024-03-28T00:25:12.619 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:12.611+0000 7f6838a00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6834100940 0x7f68341a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:12.619 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:12.611+0000 7f6838a00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6834101300 0x7f68341a0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:12.619 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:12.611+0000 7f6838a00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6834102ed0 0x7f683419a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:12.619 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:12.611+0000 7f6838a00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6834114580 con 0x7f6834100940 2024-03-28T00:25:12.619 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:12.611+0000 7f6838a00700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6834114400 con 0x7f6834102ed0 2024-03-28T00:25:12.619 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:12.611+0000 7f6838a00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6834114700 con 0x7f6834101300 2024-03-28T00:25:12.619 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:12.612+0000 7f6832800700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6834100940 0x7f68341a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:12.619 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:12.612+0000 7f6832800700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6834100940 0x7f68341a0670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49606/0 (socket says 172.21.15.119:49606) 2024-03-28T00:25:12.619 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:12.612+0000 7f6832800700 1 -- 172.21.15.119:0/206072320 learned_addr learned my addr 172.21.15.119:0/206072320 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:25:12.619 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:12.612+0000 7f6833200700 1 --2- 172.21.15.119:0/206072320 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6834102ed0 0x7f683419a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:12.619 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:12.612+0000 7f682be00700 1 --2- 172.21.15.119:0/206072320 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6834101300 0x7f68341a0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:12.620 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:12.612+0000 7f6833200700 1 -- 172.21.15.119:0/206072320 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6834101300 msgr2=0x7f68341a0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:12.620 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:12.612+0000 7f6833200700 1 --2- 172.21.15.119:0/206072320 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6834101300 0x7f68341a0bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:12.620 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:12.612+0000 7f6833200700 1 -- 172.21.15.119:0/206072320 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6834100940 msgr2=0x7f68341a0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:12.620 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:12.612+0000 7f6833200700 1 --2- 172.21.15.119:0/206072320 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6834100940 0x7f68341a0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:12.620 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:12.612+0000 7f6833200700 1 -- 172.21.15.119:0/206072320 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f683419b020 con 0x7f6834102ed0 2024-03-28T00:25:12.620 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:12.612+0000 7f6832800700 1 --2- 172.21.15.119:0/206072320 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6834100940 0x7f68341a0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T00:25:12.620 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:12.612+0000 7f6833200700 1 --2- 172.21.15.119:0/206072320 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6834102ed0 0x7f683419a840 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f682c00efc0 tx=0x7f682c00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:25:12.620 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:12.613+0000 7f682b400700 1 -- 172.21.15.119:0/206072320 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f682c0092e0 con 0x7f6834102ed0 2024-03-28T00:25:12.620 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:12.613+0000 7f6838a00700 1 -- 172.21.15.119:0/206072320 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f683419b2b0 con 0x7f6834102ed0 2024-03-28T00:25:12.620 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:12.613+0000 7f682b400700 1 -- 172.21.15.119:0/206072320 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f682c004bb0 con 0x7f6834102ed0 2024-03-28T00:25:12.620 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:12.613+0000 7f682b400700 1 -- 172.21.15.119:0/206072320 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f682c004280 con 0x7f6834102ed0 2024-03-28T00:25:12.621 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:12.613+0000 7f6838a00700 1 -- 172.21.15.119:0/206072320 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68341a74d0 con 0x7f6834102ed0 2024-03-28T00:25:12.621 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:12.614+0000 7f6838a00700 1 -- 172.21.15.119:0/206072320 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6834069ab0 con 0x7f6834102ed0 2024-03-28T00:25:12.622 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:12.616+0000 7f682b400700 1 -- 172.21.15.119:0/206072320 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 17) v1 ==== 137342+0+0 (secure 0 0 0) 0x7f682c01b030 con 0x7f6834102ed0 2024-03-28T00:25:12.623 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:12.617+0000 7f682b400700 1 --2- 172.21.15.119:0/206072320 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f68180a2920 0x7f68180a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:12.623 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:12.617+0000 7f682b400700 1 -- 172.21.15.119:0/206072320 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f682c0ce750 con 0x7f6834102ed0 2024-03-28T00:25:12.624 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:12.617+0000 7f6832800700 1 --2- 172.21.15.119:0/206072320 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f68180a2920 0x7f68180a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:12.624 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:12.618+0000 7f6832800700 1 --2- 172.21.15.119:0/206072320 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f68180a2920 0x7f68180a4de0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f6824011550 tx=0x7f6824009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:25:12.629 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:12.623+0000 7f682b400700 1 -- 172.21.15.119:0/206072320 <== mon.2 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f682c022020 con 0x7f6834102ed0 2024-03-28T00:25:12.878 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:12.872+0000 7f6838a00700 1 -- 172.21.15.119:0/206072320 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f683419bd20 con 0x7f6834102ed0 2024-03-28T00:25:12.879 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:12.872+0000 7f682b400700 1 -- 172.21.15.119:0/206072320 <== mon.2 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f682c010040 con 0x7f6834102ed0 2024-03-28T00:25:12.879 INFO:teuthology.orchestra.run.smithi119.stdout:141733920837 2024-03-28T00:25:12.882 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:12.875+0000 7f6838a00700 1 -- 172.21.15.119:0/206072320 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f68180a2920 msgr2=0x7f68180a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:12.882 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:12.875+0000 7f6838a00700 1 --2- 172.21.15.119:0/206072320 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f68180a2920 0x7f68180a4de0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f6824011550 tx=0x7f6824009260 comp rx=0 tx=0).stop 2024-03-28T00:25:12.882 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:12.875+0000 7f6838a00700 1 -- 172.21.15.119:0/206072320 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6834102ed0 msgr2=0x7f683419a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:12.882 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:12.875+0000 7f6838a00700 1 --2- 172.21.15.119:0/206072320 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6834102ed0 0x7f683419a840 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f682c00efc0 tx=0x7f682c00c3a0 comp rx=0 tx=0).stop 2024-03-28T00:25:12.882 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:12.875+0000 7f6838a00700 1 -- 172.21.15.119:0/206072320 shutdown_connections 2024-03-28T00:25:12.882 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:12.875+0000 7f6838a00700 1 --2- 172.21.15.119:0/206072320 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f68180a2920 0x7f68180a4de0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:12.882 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:12.875+0000 7f6838a00700 1 --2- 172.21.15.119:0/206072320 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6834102ed0 0x7f683419a840 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:12.882 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:12.875+0000 7f6838a00700 1 --2- 172.21.15.119:0/206072320 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6834100940 0x7f68341a0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:12.882 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:12.875+0000 7f6838a00700 1 --2- 172.21.15.119:0/206072320 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6834101300 0x7f68341a0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:12.882 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:12.875+0000 7f6838a00700 1 -- 172.21.15.119:0/206072320 >> 172.21.15.119:0/206072320 conn(0x7f68340762d0 msgr2=0x7f68340fe990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:25:12.882 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:12.875+0000 7f6838a00700 1 -- 172.21.15.119:0/206072320 shutdown_connections 2024-03-28T00:25:12.883 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:12.875+0000 7f6838a00700 1 -- 172.21.15.119:0/206072320 wait complete. 2024-03-28T00:25:13.234 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:25:12 smithi119 ceph-mon[31518]: pgmap v351: 97 pgs: 97 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T00:25:13.234 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:25:12 smithi119 ceph-mon[31518]: from='client.? 172.21.15.119:0/206072320' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-28T00:25:13.258 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:13.251+0000 7ff904800700 1 -- 172.21.15.119:0/903944658 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff8fc1087a0 msgr2=0x7ff8fc108c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:13.258 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:13.251+0000 7ff904800700 1 --2- 172.21.15.119:0/903944658 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff8fc1087a0 0x7ff8fc108c20 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ff8ec009990 tx=0x7ff8ec009ca0 comp rx=0 tx=0).stop 2024-03-28T00:25:13.258 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:13.251+0000 7ff904800700 1 -- 172.21.15.119:0/903944658 shutdown_connections 2024-03-28T00:25:13.258 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:13.251+0000 7ff904800700 1 --2- 172.21.15.119:0/903944658 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff8fc109160 0x7ff8fc115670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:13.258 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:13.251+0000 7ff904800700 1 --2- 172.21.15.119:0/903944658 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff8fc10e7a0 0x7ff8fc10eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:13.258 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:13.251+0000 7ff904800700 1 --2- 172.21.15.119:0/903944658 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff8fc1087a0 0x7ff8fc108c20 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:13.258 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:13.251+0000 7ff904800700 1 -- 172.21.15.119:0/903944658 >> 172.21.15.119:0/903944658 conn(0x7ff8fc075960 msgr2=0x7ff8fc077d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:25:13.258 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:13.251+0000 7ff904800700 1 -- 172.21.15.119:0/903944658 shutdown_connections 2024-03-28T00:25:13.258 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:13.251+0000 7ff904800700 1 -- 172.21.15.119:0/903944658 wait complete. 2024-03-28T00:25:13.259 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:13.252+0000 7ff904800700 1 Processor -- start 2024-03-28T00:25:13.259 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:13.252+0000 7ff904800700 1 -- start start 2024-03-28T00:25:13.260 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:13.253+0000 7ff904800700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff8fc1087a0 0x7ff8fc19efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:13.260 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:13.253+0000 7ff904800700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff8fc109160 0x7ff8fc19f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:13.260 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:13.253+0000 7ff904800700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff8fc10e7a0 0x7ff8fc1a3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:13.260 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:13.253+0000 7ff904800700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8fc118b80 con 0x7ff8fc109160 2024-03-28T00:25:13.260 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:13.253+0000 7ff904800700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8fc118a00 con 0x7ff8fc10e7a0 2024-03-28T00:25:13.260 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:13.253+0000 7ff904800700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8fc118d00 con 0x7ff8fc1087a0 2024-03-28T00:25:13.260 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:13.253+0000 7ff901e00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff8fc1087a0 0x7ff8fc19efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:13.260 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:13.253+0000 7ff901e00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff8fc1087a0 0x7ff8fc19efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.119:56186/0 (socket says 172.21.15.119:56186) 2024-03-28T00:25:13.260 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:13.253+0000 7ff901e00700 1 -- 172.21.15.119:0/2144296568 learned_addr learned my addr 172.21.15.119:0/2144296568 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:25:13.260 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:13.253+0000 7ff901400700 1 --2- 172.21.15.119:0/2144296568 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff8fc109160 0x7ff8fc19f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:13.260 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:13.253+0000 7ff902800700 1 --2- 172.21.15.119:0/2144296568 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff8fc10e7a0 0x7ff8fc1a3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:13.260 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:13.253+0000 7ff901e00700 1 -- 172.21.15.119:0/2144296568 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff8fc10e7a0 msgr2=0x7ff8fc1a3830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:13.261 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:13.253+0000 7ff901e00700 1 --2- 172.21.15.119:0/2144296568 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff8fc10e7a0 0x7ff8fc1a3830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:13.261 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:13.253+0000 7ff901e00700 1 -- 172.21.15.119:0/2144296568 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff8fc109160 msgr2=0x7ff8fc19f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:13.261 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:13.253+0000 7ff901e00700 1 --2- 172.21.15.119:0/2144296568 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff8fc109160 0x7ff8fc19f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:13.261 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:13.253+0000 7ff901e00700 1 -- 172.21.15.119:0/2144296568 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8fc1a3f70 con 0x7ff8fc1087a0 2024-03-28T00:25:13.261 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:13.253+0000 7ff901400700 1 --2- 172.21.15.119:0/2144296568 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff8fc109160 0x7ff8fc19f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T00:25:13.262 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:13.254+0000 7ff901e00700 1 --2- 172.21.15.119:0/2144296568 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff8fc1087a0 0x7ff8fc19efb0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff8f000e9c0 tx=0x7ff8f000ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:25:13.262 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:13.254+0000 7ff8f6a00700 1 -- 172.21.15.119:0/2144296568 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8f000cbd0 con 0x7ff8fc1087a0 2024-03-28T00:25:13.262 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:13.254+0000 7ff8f6a00700 1 -- 172.21.15.119:0/2144296568 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff8f000cd70 con 0x7ff8fc1087a0 2024-03-28T00:25:13.262 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:13.254+0000 7ff904800700 1 -- 172.21.15.119:0/2144296568 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8fc1a4260 con 0x7ff8fc1087a0 2024-03-28T00:25:13.262 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:13.254+0000 7ff8f6a00700 1 -- 172.21.15.119:0/2144296568 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8f0010430 con 0x7ff8fc1087a0 2024-03-28T00:25:13.262 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:13.254+0000 7ff904800700 1 -- 172.21.15.119:0/2144296568 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8fc1abae0 con 0x7ff8fc1087a0 2024-03-28T00:25:13.262 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:13.255+0000 7ff904800700 1 -- 172.21.15.119:0/2144296568 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8fc110d30 con 0x7ff8fc1087a0 2024-03-28T00:25:13.263 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:13.257+0000 7ff8f6a00700 1 -- 172.21.15.119:0/2144296568 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 17) v1 ==== 137342+0+0 (secure 0 0 0) 0x7ff8f0004750 con 0x7ff8fc1087a0 2024-03-28T00:25:13.265 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:13.258+0000 7ff8f6a00700 1 --2- 172.21.15.119:0/2144296568 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7ff8e40a29a0 0x7ff8e40a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:13.265 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:13.259+0000 7ff8f6a00700 1 -- 172.21.15.119:0/2144296568 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8153+0+0 (secure 0 0 0) 0x7ff8f00ced90 con 0x7ff8fc1087a0 2024-03-28T00:25:13.265 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:13.259+0000 7ff901400700 1 --2- 172.21.15.119:0/2144296568 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7ff8e40a29a0 0x7ff8e40a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:13.265 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:13.259+0000 7ff901400700 1 --2- 172.21.15.119:0/2144296568 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7ff8e40a29a0 0x7ff8e40a4e60 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ff8ec00b2e0 tx=0x7ff8ec01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:25:13.268 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:13.262+0000 7ff8f6a00700 1 -- 172.21.15.119:0/2144296568 <== mon.1 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff8f009bcd0 con 0x7ff8fc1087a0 2024-03-28T00:25:13.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:25:12 smithi165 ceph-mon[36302]: pgmap v351: 97 pgs: 97 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T00:25:13.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:25:12 smithi165 ceph-mon[36302]: from='client.? 172.21.15.119:0/206072320' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-28T00:25:13.452 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:25:12 smithi132 ceph-mon[35941]: pgmap v351: 97 pgs: 97 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T00:25:13.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:25:12 smithi132 ceph-mon[35941]: from='client.? 172.21.15.119:0/206072320' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-28T00:25:13.502 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:13.495+0000 7ff904800700 1 -- 172.21.15.119:0/2144296568 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7ff8fc1a02f0 con 0x7ff8fc1087a0 2024-03-28T00:25:13.502 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:13.496+0000 7ff8f6a00700 1 -- 172.21.15.119:0/2144296568 <== mon.1 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ff8fc1a02f0 con 0x7ff8fc1087a0 2024-03-28T00:25:13.502 INFO:teuthology.orchestra.run.smithi119.stdout:77309411409 2024-03-28T00:25:13.505 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:13.499+0000 7ff904800700 1 -- 172.21.15.119:0/2144296568 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7ff8e40a29a0 msgr2=0x7ff8e40a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:13.505 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:13.499+0000 7ff904800700 1 --2- 172.21.15.119:0/2144296568 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7ff8e40a29a0 0x7ff8e40a4e60 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ff8ec00b2e0 tx=0x7ff8ec01a040 comp rx=0 tx=0).stop 2024-03-28T00:25:13.505 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:13.499+0000 7ff904800700 1 -- 172.21.15.119:0/2144296568 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff8fc1087a0 msgr2=0x7ff8fc19efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:13.505 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:13.499+0000 7ff904800700 1 --2- 172.21.15.119:0/2144296568 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff8fc1087a0 0x7ff8fc19efb0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff8f000e9c0 tx=0x7ff8f000ecd0 comp rx=0 tx=0).stop 2024-03-28T00:25:13.506 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:13.499+0000 7ff904800700 1 -- 172.21.15.119:0/2144296568 shutdown_connections 2024-03-28T00:25:13.506 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:13.499+0000 7ff904800700 1 --2- 172.21.15.119:0/2144296568 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7ff8e40a29a0 0x7ff8e40a4e60 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:13.506 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:13.499+0000 7ff904800700 1 --2- 172.21.15.119:0/2144296568 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff8fc10e7a0 0x7ff8fc1a3830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:13.506 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:13.499+0000 7ff904800700 1 --2- 172.21.15.119:0/2144296568 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff8fc1087a0 0x7ff8fc19efb0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:13.506 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:13.499+0000 7ff904800700 1 --2- 172.21.15.119:0/2144296568 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff8fc109160 0x7ff8fc19f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:13.506 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:13.499+0000 7ff904800700 1 -- 172.21.15.119:0/2144296568 >> 172.21.15.119:0/2144296568 conn(0x7ff8fc075960 msgr2=0x7ff8fc075f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:25:13.506 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:13.499+0000 7ff904800700 1 -- 172.21.15.119:0/2144296568 shutdown_connections 2024-03-28T00:25:13.506 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:13.499+0000 7ff904800700 1 -- 172.21.15.119:0/2144296568 wait complete. 2024-03-28T00:25:13.844 INFO:tasks.cephadm.ceph_manager.ceph:need seq 219043332140 got 219043332146 for osd.7 2024-03-28T00:25:13.844 DEBUG:teuthology.parallel:result is None 2024-03-28T00:25:14.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:25:13 smithi119 ceph-mon[31518]: from='client.? 172.21.15.119:0/2144296568' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-28T00:25:14.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:25:13 smithi165 ceph-mon[36302]: from='client.? 172.21.15.119:0/2144296568' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-28T00:25:14.452 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:25:13 smithi132 ceph-mon[35941]: from='client.? 172.21.15.119:0/2144296568' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-28T00:25:15.347 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:25:14 smithi119 ceph-mon[31518]: pgmap v352: 97 pgs: 97 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-28T00:25:15.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:25:14 smithi165 ceph-mon[36302]: pgmap v352: 97 pgs: 97 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-28T00:25:15.426 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:15.131+0000 7fb96f200700 1 -- 172.21.15.119:0/931218283 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb970103270 msgr2=0x7fb970103650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:15.426 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:15.131+0000 7fb96f200700 1 --2- 172.21.15.119:0/931218283 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb970103270 0x7fb970103650 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb960009f90 tx=0x7fb96000b330 comp rx=0 tx=0).stop 2024-03-28T00:25:15.426 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:15.131+0000 7fb96f200700 1 -- 172.21.15.119:0/931218283 shutdown_connections 2024-03-28T00:25:15.426 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:15.131+0000 7fb96f200700 1 --2- 172.21.15.119:0/931218283 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb97010c6a0 0x7fb97010eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:15.426 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:15.131+0000 7fb96f200700 1 --2- 172.21.15.119:0/931218283 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb970103270 0x7fb970103650 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:15.426 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:15.131+0000 7fb96f200700 1 --2- 172.21.15.119:0/931218283 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb970103b90 0x7fb97010c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:15.426 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:15.131+0000 7fb96f200700 1 -- 172.21.15.119:0/931218283 >> 172.21.15.119:0/931218283 conn(0x7fb9700fca60 msgr2=0x7fb9700fee80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:25:15.427 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:15.131+0000 7fb96f200700 1 -- 172.21.15.119:0/931218283 shutdown_connections 2024-03-28T00:25:15.427 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:15.131+0000 7fb96f200700 1 -- 172.21.15.119:0/931218283 wait complete. 2024-03-28T00:25:15.427 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:15.132+0000 7fb96f200700 1 Processor -- start 2024-03-28T00:25:15.427 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:15.132+0000 7fb96f200700 1 -- start start 2024-03-28T00:25:15.427 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:15.133+0000 7fb96f200700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb970103270 0x7fb9701a06d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:15.427 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:15.133+0000 7fb96f200700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb970103b90 0x7fb9701a0c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:15.427 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:15.133+0000 7fb96f200700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb97010c6a0 0x7fb97019a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:15.427 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:15.133+0000 7fb96f200700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb970112390 con 0x7fb970103b90 2024-03-28T00:25:15.427 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:15.133+0000 7fb96f200700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb970112210 con 0x7fb97010c6a0 2024-03-28T00:25:15.427 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:15.133+0000 7fb96f200700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb970112510 con 0x7fb970103270 2024-03-28T00:25:15.427 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:15.133+0000 7fb96de00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb970103270 0x7fb9701a06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:15.427 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:15.133+0000 7fb96de00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb970103270 0x7fb9701a06d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.119:35554/0 (socket says 172.21.15.119:35554) 2024-03-28T00:25:15.427 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:15.133+0000 7fb96de00700 1 -- 172.21.15.119:0/534530143 learned_addr learned my addr 172.21.15.119:0/534530143 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:25:15.428 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:15.133+0000 7fb965400700 1 --2- 172.21.15.119:0/534530143 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb970103b90 0x7fb9701a0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:15.428 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:15.133+0000 7fb96e800700 1 --2- 172.21.15.119:0/534530143 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb97010c6a0 0x7fb97019a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:15.428 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:15.134+0000 7fb96de00700 1 -- 172.21.15.119:0/534530143 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb97010c6a0 msgr2=0x7fb97019a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:15.428 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:15.134+0000 7fb96de00700 1 --2- 172.21.15.119:0/534530143 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb97010c6a0 0x7fb97019a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:15.428 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:15.134+0000 7fb96de00700 1 -- 172.21.15.119:0/534530143 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb970103b90 msgr2=0x7fb9701a0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:15.428 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:15.134+0000 7fb96de00700 1 --2- 172.21.15.119:0/534530143 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb970103b90 0x7fb9701a0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:15.428 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:15.134+0000 7fb96de00700 1 -- 172.21.15.119:0/534530143 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb97019b030 con 0x7fb970103270 2024-03-28T00:25:15.428 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:15.134+0000 7fb96de00700 1 --2- 172.21.15.119:0/534530143 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb970103270 0x7fb9701a06d0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb960004f20 tx=0x7fb960004f50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:25:15.428 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:15.134+0000 7fb967400700 1 -- 172.21.15.119:0/534530143 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb960021070 con 0x7fb970103270 2024-03-28T00:25:15.428 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:15.134+0000 7fb967400700 1 -- 172.21.15.119:0/534530143 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb960007ac0 con 0x7fb970103270 2024-03-28T00:25:15.428 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:15.134+0000 7fb96f200700 1 -- 172.21.15.119:0/534530143 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb97019b2c0 con 0x7fb970103270 2024-03-28T00:25:15.428 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:15.134+0000 7fb967400700 1 -- 172.21.15.119:0/534530143 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9600052a0 con 0x7fb970103270 2024-03-28T00:25:15.429 INFO:teuthology.orchestra.run.smithi119.stdout:107374182473 2024-03-28T00:25:15.429 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:15.134+0000 7fb96f200700 1 -- 172.21.15.119:0/534530143 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9701a7530 con 0x7fb970103270 2024-03-28T00:25:15.429 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:15.135+0000 7fb96f200700 1 -- 172.21.15.119:0/534530143 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb97006a6e0 con 0x7fb970103270 2024-03-28T00:25:15.429 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:15.137+0000 7fb967400700 1 -- 172.21.15.119:0/534530143 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 17) v1 ==== 137342+0+0 (secure 0 0 0) 0x7fb960024070 con 0x7fb970103270 2024-03-28T00:25:15.429 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:15.139+0000 7fb967400700 1 --2- 172.21.15.119:0/534530143 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fb9540a29a0 0x7fb9540a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:15.429 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:15.139+0000 7fb967400700 1 -- 172.21.15.119:0/534530143 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fb96000f020 con 0x7fb970103270 2024-03-28T00:25:15.429 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:15.139+0000 7fb965400700 1 --2- 172.21.15.119:0/534530143 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fb9540a29a0 0x7fb9540a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:15.430 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:15.139+0000 7fb965400700 1 --2- 172.21.15.119:0/534530143 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fb9540a29a0 0x7fb9540a4e60 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb97010ee90 tx=0x7fb958009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:25:15.430 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:15.141+0000 7fb967400700 1 -- 172.21.15.119:0/534530143 <== mon.1 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fb96002b020 con 0x7fb970103270 2024-03-28T00:25:15.430 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:15.387+0000 7fb96f200700 1 -- 172.21.15.119:0/534530143 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fb9701a7700 con 0x7fb970103270 2024-03-28T00:25:15.430 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:15.388+0000 7fb967400700 1 -- 172.21.15.119:0/534530143 <== mon.1 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb960011e50 con 0x7fb970103270 2024-03-28T00:25:15.430 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:15.390+0000 7fb96f200700 1 -- 172.21.15.119:0/534530143 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fb9540a29a0 msgr2=0x7fb9540a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:15.430 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:15.390+0000 7fb96f200700 1 --2- 172.21.15.119:0/534530143 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fb9540a29a0 0x7fb9540a4e60 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb97010ee90 tx=0x7fb958009310 comp rx=0 tx=0).stop 2024-03-28T00:25:15.430 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:15.390+0000 7fb96f200700 1 -- 172.21.15.119:0/534530143 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb970103270 msgr2=0x7fb9701a06d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:15.430 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:15.390+0000 7fb96f200700 1 --2- 172.21.15.119:0/534530143 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb970103270 0x7fb9701a06d0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb960004f20 tx=0x7fb960004f50 comp rx=0 tx=0).stop 2024-03-28T00:25:15.430 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:15.390+0000 7fb96f200700 1 -- 172.21.15.119:0/534530143 shutdown_connections 2024-03-28T00:25:15.430 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:15.390+0000 7fb96f200700 1 --2- 172.21.15.119:0/534530143 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fb9540a29a0 0x7fb9540a4e60 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:15.430 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:15.390+0000 7fb96f200700 1 --2- 172.21.15.119:0/534530143 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb97010c6a0 0x7fb97019a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:15.430 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:15.390+0000 7fb96f200700 1 --2- 172.21.15.119:0/534530143 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb970103270 0x7fb9701a06d0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:15.430 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:15.390+0000 7fb96f200700 1 --2- 172.21.15.119:0/534530143 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb970103b90 0x7fb9701a0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:15.430 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:15.390+0000 7fb96f200700 1 -- 172.21.15.119:0/534530143 >> 172.21.15.119:0/534530143 conn(0x7fb9700fca60 msgr2=0x7fb9700fee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:25:15.431 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:15.391+0000 7fb96f200700 1 -- 172.21.15.119:0/534530143 shutdown_connections 2024-03-28T00:25:15.431 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:15.391+0000 7fb96f200700 1 -- 172.21.15.119:0/534530143 wait complete. 2024-03-28T00:25:15.452 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:25:14 smithi132 ceph-mon[35941]: pgmap v352: 97 pgs: 97 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-28T00:25:16.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:25:15 smithi119 ceph-mon[31518]: from='client.? 172.21.15.119:0/534530143' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-28T00:25:16.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:25:15 smithi165 ceph-mon[36302]: from='client.? 172.21.15.119:0/534530143' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-28T00:25:16.452 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:25:15 smithi132 ceph-mon[35941]: from='client.? 172.21.15.119:0/534530143' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-28T00:25:16.468 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411400 got 77309411409 for osd.2 2024-03-28T00:25:16.468 DEBUG:teuthology.parallel:result is None 2024-03-28T00:25:16.683 INFO:tasks.cephadm.ceph_manager.ceph:need seq 253403070504 got 253403070511 for osd.8 2024-03-28T00:25:16.683 DEBUG:teuthology.parallel:result is None 2024-03-28T00:25:16.928 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:16.922+0000 7f9f15000700 1 -- 172.21.15.119:0/510061841 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9f101087a0 msgr2=0x7f9f10108c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:16.929 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:16.922+0000 7f9f15000700 1 --2- 172.21.15.119:0/510061841 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9f101087a0 0x7f9f10108c20 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f9efc009990 tx=0x7f9efc009ca0 comp rx=0 tx=0).stop 2024-03-28T00:25:16.929 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:16.922+0000 7f9f15000700 1 -- 172.21.15.119:0/510061841 shutdown_connections 2024-03-28T00:25:16.929 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:16.922+0000 7f9f15000700 1 --2- 172.21.15.119:0/510061841 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9f10109160 0x7f9f10115670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:16.929 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:16.922+0000 7f9f15000700 1 --2- 172.21.15.119:0/510061841 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9f1010e7a0 0x7f9f1010eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:16.929 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:16.922+0000 7f9f15000700 1 --2- 172.21.15.119:0/510061841 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9f101087a0 0x7f9f10108c20 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:16.930 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:16.922+0000 7f9f15000700 1 -- 172.21.15.119:0/510061841 >> 172.21.15.119:0/510061841 conn(0x7f9f10075960 msgr2=0x7f9f10077d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:25:16.930 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:16.922+0000 7f9f15000700 1 -- 172.21.15.119:0/510061841 shutdown_connections 2024-03-28T00:25:16.930 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:16.922+0000 7f9f15000700 1 -- 172.21.15.119:0/510061841 wait complete. 2024-03-28T00:25:16.932 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:16.923+0000 7f9f15000700 1 Processor -- start 2024-03-28T00:25:16.932 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:16.923+0000 7f9f15000700 1 -- start start 2024-03-28T00:25:16.933 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:16.923+0000 7f9f15000700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9f101087a0 0x7f9f1019efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:16.933 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:16.923+0000 7f9f15000700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9f10109160 0x7f9f1019f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:16.933 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:16.923+0000 7f9f15000700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9f1010e7a0 0x7f9f101a3820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:16.933 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:16.923+0000 7f9f15000700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f10118b30 con 0x7f9f1010e7a0 2024-03-28T00:25:16.933 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:16.923+0000 7f9f15000700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f101189b0 con 0x7f9f101087a0 2024-03-28T00:25:16.934 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:16.923+0000 7f9f15000700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f10118cb0 con 0x7f9f10109160 2024-03-28T00:25:16.934 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:16.924+0000 7f9f0de00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9f101087a0 0x7f9f1019efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:16.934 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:16.924+0000 7f9f0de00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9f101087a0 0x7f9f1019efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.119:42354/0 (socket says 172.21.15.119:42354) 2024-03-28T00:25:16.934 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:16.924+0000 7f9f0de00700 1 -- 172.21.15.119:0/1997525292 learned_addr learned my addr 172.21.15.119:0/1997525292 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:25:16.934 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:16.924+0000 7f9f0d400700 1 --2- 172.21.15.119:0/1997525292 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9f10109160 0x7f9f1019f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:16.934 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:16.924+0000 7f9f0e800700 1 --2- 172.21.15.119:0/1997525292 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9f1010e7a0 0x7f9f101a3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:16.934 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:16.924+0000 7f9f0de00700 1 -- 172.21.15.119:0/1997525292 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9f10109160 msgr2=0x7f9f1019f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:16.934 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:16.924+0000 7f9f0de00700 1 --2- 172.21.15.119:0/1997525292 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9f10109160 0x7f9f1019f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:16.934 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:16.924+0000 7f9f0de00700 1 -- 172.21.15.119:0/1997525292 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9f1010e7a0 msgr2=0x7f9f101a3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:16.934 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:16.924+0000 7f9f0de00700 1 --2- 172.21.15.119:0/1997525292 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9f1010e7a0 0x7f9f101a3820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:16.934 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:16.924+0000 7f9f0de00700 1 -- 172.21.15.119:0/1997525292 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f101a3f60 con 0x7f9f101087a0 2024-03-28T00:25:16.934 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:16.924+0000 7f9f0de00700 1 --2- 172.21.15.119:0/1997525292 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9f101087a0 0x7f9f1019efa0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f9f0000e9a0 tx=0x7f9f0000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:25:16.935 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:16.925+0000 7f9f06a00700 1 -- 172.21.15.119:0/1997525292 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f0000cbd0 con 0x7f9f101087a0 2024-03-28T00:25:16.935 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:16.925+0000 7f9f06a00700 1 -- 172.21.15.119:0/1997525292 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9f0000cd70 con 0x7f9f101087a0 2024-03-28T00:25:16.935 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:16.925+0000 7f9f15000700 1 -- 172.21.15.119:0/1997525292 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f101a4250 con 0x7f9f101087a0 2024-03-28T00:25:16.935 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:16.925+0000 7f9f15000700 1 -- 172.21.15.119:0/1997525292 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f101abad0 con 0x7f9f101087a0 2024-03-28T00:25:16.935 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:16.925+0000 7f9f06a00700 1 -- 172.21.15.119:0/1997525292 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f00018750 con 0x7f9f101087a0 2024-03-28T00:25:16.935 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:16.926+0000 7f9f15000700 1 -- 172.21.15.119:0/1997525292 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ef0005330 con 0x7f9f101087a0 2024-03-28T00:25:16.936 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:16.928+0000 7f9f06a00700 1 -- 172.21.15.119:0/1997525292 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 17) v1 ==== 137342+0+0 (secure 0 0 0) 0x7f9f000109a0 con 0x7f9f101087a0 2024-03-28T00:25:16.936 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:16.929+0000 7f9f06a00700 1 --2- 172.21.15.119:0/1997525292 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f9ef40a29a0 0x7f9ef40a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:16.936 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:16.929+0000 7f9f06a00700 1 -- 172.21.15.119:0/1997525292 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f9f000ced30 con 0x7f9f101087a0 2024-03-28T00:25:16.936 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:16.930+0000 7f9f0d400700 1 --2- 172.21.15.119:0/1997525292 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f9ef40a29a0 0x7f9ef40a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:16.936 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:16.930+0000 7f9f0d400700 1 --2- 172.21.15.119:0/1997525292 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f9ef40a29a0 0x7f9ef40a4e60 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f9efc00b2e0 tx=0x7f9efc005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:25:16.941 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:16.935+0000 7f9f06a00700 1 -- 172.21.15.119:0/1997525292 <== mon.2 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f9f0009bc70 con 0x7f9f101087a0 2024-03-28T00:25:17.174 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:17.167+0000 7f9f15000700 1 -- 172.21.15.119:0/1997525292 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) v1 -- 0x7f9ef00058f0 con 0x7f9f101087a0 2024-03-28T00:25:17.174 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:17.168+0000 7f9f06a00700 1 -- 172.21.15.119:0/1997525292 <== mon.2 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7f9f0009b570 con 0x7f9f101087a0 2024-03-28T00:25:17.174 INFO:teuthology.orchestra.run.smithi119.stdout:343597383711 2024-03-28T00:25:17.178 INFO:tasks.cephadm.ceph_manager.ceph:need seq 197568495668 got 197568495672 for osd.6 2024-03-28T00:25:17.179 DEBUG:teuthology.parallel:result is None 2024-03-28T00:25:17.179 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:17.172+0000 7f9f15000700 1 -- 172.21.15.119:0/1997525292 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f9ef40a29a0 msgr2=0x7f9ef40a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:17.179 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:17.172+0000 7f9f15000700 1 --2- 172.21.15.119:0/1997525292 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f9ef40a29a0 0x7f9ef40a4e60 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f9efc00b2e0 tx=0x7f9efc005960 comp rx=0 tx=0).stop 2024-03-28T00:25:17.179 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:17.172+0000 7f9f15000700 1 -- 172.21.15.119:0/1997525292 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9f101087a0 msgr2=0x7f9f1019efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:17.179 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:17.172+0000 7f9f15000700 1 --2- 172.21.15.119:0/1997525292 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9f101087a0 0x7f9f1019efa0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f9f0000e9a0 tx=0x7f9f0000ecb0 comp rx=0 tx=0).stop 2024-03-28T00:25:17.179 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:17.172+0000 7f9f15000700 1 -- 172.21.15.119:0/1997525292 shutdown_connections 2024-03-28T00:25:17.179 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:17.172+0000 7f9f15000700 1 --2- 172.21.15.119:0/1997525292 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f9ef40a29a0 0x7f9ef40a4e60 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:17.180 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:17.172+0000 7f9f15000700 1 --2- 172.21.15.119:0/1997525292 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9f1010e7a0 0x7f9f101a3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:17.180 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:17.172+0000 7f9f15000700 1 --2- 172.21.15.119:0/1997525292 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9f101087a0 0x7f9f1019efa0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:17.180 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:17.172+0000 7f9f15000700 1 --2- 172.21.15.119:0/1997525292 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9f10109160 0x7f9f1019f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:17.180 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:17.172+0000 7f9f15000700 1 -- 172.21.15.119:0/1997525292 >> 172.21.15.119:0/1997525292 conn(0x7f9f10075960 msgr2=0x7f9f10075f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:25:17.180 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:17.172+0000 7f9f15000700 1 -- 172.21.15.119:0/1997525292 shutdown_connections 2024-03-28T00:25:17.180 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:17.172+0000 7f9f15000700 1 -- 172.21.15.119:0/1997525292 wait complete. 2024-03-28T00:25:17.198 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:25:16 smithi119 ceph-mon[31518]: pgmap v353: 97 pgs: 97 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T00:25:17.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:25:16 smithi165 ceph-mon[36302]: pgmap v353: 97 pgs: 97 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T00:25:17.452 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:25:16 smithi132 ceph-mon[35941]: pgmap v353: 97 pgs: 97 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T00:25:18.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:25:18 smithi119 ceph-mon[31518]: from='client.? 172.21.15.119:0/1997525292' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-03-28T00:25:18.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:25:18 smithi165 ceph-mon[36302]: from='client.? 172.21.15.119:0/1997525292' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-03-28T00:25:18.452 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:25:18 smithi132 ceph-mon[35941]: from='client.? 172.21.15.119:0/1997525292' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-03-28T00:25:18.539 INFO:tasks.cephadm.ceph_manager.ceph:need seq 141733920832 got 141733920837 for osd.4 2024-03-28T00:25:18.539 DEBUG:teuthology.parallel:result is None 2024-03-28T00:25:18.877 INFO:tasks.cephadm.ceph_manager.ceph:need seq 343597383703 got 343597383711 for osd.11 2024-03-28T00:25:18.877 DEBUG:teuthology.parallel:result is None 2024-03-28T00:25:19.065 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:25:19 smithi119 ceph-mon[31518]: pgmap v354: 97 pgs: 97 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-28T00:25:19.283 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:25:19 smithi165 ceph-mon[36302]: pgmap v354: 97 pgs: 97 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-28T00:25:19.452 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:25:19 smithi132 ceph-mon[35941]: pgmap v354: 97 pgs: 97 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-28T00:25:19.482 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:19.476+0000 7fd296e00700 1 -- 172.21.15.119:0/3163079185 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd290107510 msgr2=0x7fd29010f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:19.482 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:19.476+0000 7fd296e00700 1 --2- 172.21.15.119:0/3163079185 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd290107510 0x7fd29010f0e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fd288009fc0 tx=0x7fd28800b330 comp rx=0 tx=0).stop 2024-03-28T00:25:19.483 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:19.476+0000 7fd296e00700 1 -- 172.21.15.119:0/3163079185 shutdown_connections 2024-03-28T00:25:19.483 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:19.476+0000 7fd296e00700 1 --2- 172.21.15.119:0/3163079185 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd290107510 0x7fd29010f0e0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:19.483 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:19.476+0000 7fd296e00700 1 --2- 172.21.15.119:0/3163079185 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd290102440 0x7fd290102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:19.483 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:19.476+0000 7fd296e00700 1 --2- 172.21.15.119:0/3163079185 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd290102df0 0x7fd290106e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:19.483 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:19.476+0000 7fd296e00700 1 -- 172.21.15.119:0/3163079185 >> 172.21.15.119:0/3163079185 conn(0x7fd2900792f0 msgr2=0x7fd290079700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:25:19.483 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:19.476+0000 7fd296e00700 1 -- 172.21.15.119:0/3163079185 shutdown_connections 2024-03-28T00:25:19.483 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:19.476+0000 7fd296e00700 1 -- 172.21.15.119:0/3163079185 wait complete. 2024-03-28T00:25:19.484 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:19.477+0000 7fd296e00700 1 Processor -- start 2024-03-28T00:25:19.484 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:19.477+0000 7fd296e00700 1 -- start start 2024-03-28T00:25:19.484 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:19.477+0000 7fd296e00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd290102440 0x7fd290074490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:19.484 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:19.477+0000 7fd296e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd290102df0 0x7fd2900749d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:19.484 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:19.477+0000 7fd296e00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd290107510 0x7fd290074f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:19.484 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:19.477+0000 7fd296e00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd290112560 con 0x7fd290102df0 2024-03-28T00:25:19.484 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:19.477+0000 7fd296e00700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2901123e0 con 0x7fd290102440 2024-03-28T00:25:19.484 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:19.477+0000 7fd296e00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2901126e0 con 0x7fd290107510 2024-03-28T00:25:19.484 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:19.478+0000 7fd28fe00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd290102440 0x7fd290074490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:19.485 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:19.478+0000 7fd294e00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd290107510 0x7fd290074f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:19.485 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:19.478+0000 7fd28fe00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd290102440 0x7fd290074490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.119:42402/0 (socket says 172.21.15.119:42402) 2024-03-28T00:25:19.485 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:19.478+0000 7fd294e00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd290107510 0x7fd290074f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.119:35602/0 (socket says 172.21.15.119:35602) 2024-03-28T00:25:19.485 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:19.478+0000 7fd28fe00700 1 -- 172.21.15.119:0/807987031 learned_addr learned my addr 172.21.15.119:0/807987031 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:25:19.485 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:19.478+0000 7fd28f400700 1 --2- 172.21.15.119:0/807987031 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd290102df0 0x7fd2900749d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:19.486 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:19.478+0000 7fd28fe00700 1 -- 172.21.15.119:0/807987031 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd290107510 msgr2=0x7fd290074f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:19.486 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:19.478+0000 7fd28fe00700 1 --2- 172.21.15.119:0/807987031 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd290107510 0x7fd290074f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:19.486 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:19.478+0000 7fd28fe00700 1 -- 172.21.15.119:0/807987031 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd290102df0 msgr2=0x7fd2900749d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:19.486 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:19.478+0000 7fd28fe00700 1 --2- 172.21.15.119:0/807987031 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd290102df0 0x7fd2900749d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:19.486 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:19.478+0000 7fd28fe00700 1 -- 172.21.15.119:0/807987031 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2901a6ec0 con 0x7fd290102440 2024-03-28T00:25:19.486 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:19.478+0000 7fd294e00700 1 --2- 172.21.15.119:0/807987031 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd290107510 0x7fd290074f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T00:25:19.487 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:19.478+0000 7fd28fe00700 1 --2- 172.21.15.119:0/807987031 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd290102440 0x7fd290074490 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fd28000e9c0 tx=0x7fd28000ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:25:19.487 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:19.479+0000 7fd28cc00700 1 -- 172.21.15.119:0/807987031 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd28000cbd0 con 0x7fd290102440 2024-03-28T00:25:19.487 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:19.479+0000 7fd28cc00700 1 -- 172.21.15.119:0/807987031 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd28000cd70 con 0x7fd290102440 2024-03-28T00:25:19.487 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:19.479+0000 7fd28cc00700 1 -- 172.21.15.119:0/807987031 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd280005660 con 0x7fd290102440 2024-03-28T00:25:19.487 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:19.479+0000 7fd296e00700 1 -- 172.21.15.119:0/807987031 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2901a71b0 con 0x7fd290102440 2024-03-28T00:25:19.487 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:19.479+0000 7fd296e00700 1 -- 172.21.15.119:0/807987031 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2901a76a0 con 0x7fd290102440 2024-03-28T00:25:19.487 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:19.480+0000 7fd296e00700 1 -- 172.21.15.119:0/807987031 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd290068e50 con 0x7fd290102440 2024-03-28T00:25:19.488 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:19.482+0000 7fd28cc00700 1 -- 172.21.15.119:0/807987031 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 17) v1 ==== 137342+0+0 (secure 0 0 0) 0x7fd280004750 con 0x7fd290102440 2024-03-28T00:25:19.490 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:19.483+0000 7fd28cc00700 1 --2- 172.21.15.119:0/807987031 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fd2740a29a0 0x7fd2740a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:19.490 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:19.484+0000 7fd28cc00700 1 -- 172.21.15.119:0/807987031 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fd2800cef00 con 0x7fd290102440 2024-03-28T00:25:19.490 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:19.484+0000 7fd28f400700 1 --2- 172.21.15.119:0/807987031 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fd2740a29a0 0x7fd2740a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:19.490 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:19.485+0000 7fd28f400700 1 --2- 172.21.15.119:0/807987031 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fd2740a29a0 0x7fd2740a4e60 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fd290108970 tx=0x7fd27c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:25:19.491 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:19.485+0000 7fd28cc00700 1 -- 172.21.15.119:0/807987031 <== mon.2 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd280053740 con 0x7fd290102440 2024-03-28T00:25:19.537 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:19.357+0000 7fef53200700 1 -- 172.21.15.119:0/2392281716 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fef541073b0 msgr2=0x7fef5410ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:19.537 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:19.357+0000 7fef53200700 1 --2- 172.21.15.119:0/2392281716 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fef541073b0 0x7fef5410ef80 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fef4c00b0a0 tx=0x7fef4c00b3b0 comp rx=0 tx=0).stop 2024-03-28T00:25:19.537 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:19.358+0000 7fef53200700 1 -- 172.21.15.119:0/2392281716 shutdown_connections 2024-03-28T00:25:19.537 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:19.358+0000 7fef53200700 1 --2- 172.21.15.119:0/2392281716 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fef541073b0 0x7fef5410ef80 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:19.537 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:19.358+0000 7fef53200700 1 --2- 172.21.15.119:0/2392281716 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fef541022d0 0x7fef541026b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:19.537 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:19.358+0000 7fef53200700 1 --2- 172.21.15.119:0/2392281716 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fef54102c80 0x7fef54106cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:19.537 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:19.358+0000 7fef53200700 1 -- 172.21.15.119:0/2392281716 >> 172.21.15.119:0/2392281716 conn(0x7fef540fdb60 msgr2=0x7fef540fff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:25:19.537 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:19.358+0000 7fef53200700 1 -- 172.21.15.119:0/2392281716 shutdown_connections 2024-03-28T00:25:19.537 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:19.358+0000 7fef53200700 1 -- 172.21.15.119:0/2392281716 wait complete. 2024-03-28T00:25:19.537 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:19.359+0000 7fef53200700 1 Processor -- start 2024-03-28T00:25:19.537 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:19.359+0000 7fef53200700 1 -- start start 2024-03-28T00:25:19.537 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:19.360+0000 7fef53200700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fef541022d0 0x7fef5419e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:19.538 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:19.360+0000 7fef53200700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fef54102c80 0x7fef5419eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:19.538 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:19.360+0000 7fef53200700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fef541073b0 0x7fef541986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:19.538 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:19.360+0000 7fef53200700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef541123f0 con 0x7fef54102c80 2024-03-28T00:25:19.538 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:19.360+0000 7fef53200700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef54112270 con 0x7fef541022d0 2024-03-28T00:25:19.538 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:19.360+0000 7fef53200700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef54112570 con 0x7fef541073b0 2024-03-28T00:25:19.538 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:19.360+0000 7fef51e00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fef541022d0 0x7fef5419e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:19.538 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:19.360+0000 7fef51e00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fef541022d0 0x7fef5419e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.119:42382/0 (socket says 172.21.15.119:42382) 2024-03-28T00:25:19.538 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:19.360+0000 7fef51e00700 1 -- 172.21.15.119:0/3195044279 learned_addr learned my addr 172.21.15.119:0/3195044279 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:25:19.538 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:19.360+0000 7fef51400700 1 --2- 172.21.15.119:0/3195044279 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fef54102c80 0x7fef5419eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:19.538 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:19.361+0000 7fef52800700 1 --2- 172.21.15.119:0/3195044279 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fef541073b0 0x7fef541986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:19.538 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:19.361+0000 7fef51e00700 1 -- 172.21.15.119:0/3195044279 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fef541073b0 msgr2=0x7fef541986f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:19.538 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:19.361+0000 7fef51e00700 1 --2- 172.21.15.119:0/3195044279 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fef541073b0 0x7fef541986f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:19.538 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:19.361+0000 7fef51e00700 1 -- 172.21.15.119:0/3195044279 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fef54102c80 msgr2=0x7fef5419eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:19.538 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:19.361+0000 7fef51e00700 1 --2- 172.21.15.119:0/3195044279 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fef54102c80 0x7fef5419eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:19.539 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:19.361+0000 7fef51e00700 1 -- 172.21.15.119:0/3195044279 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef54198fd0 con 0x7fef541022d0 2024-03-28T00:25:19.539 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:19.361+0000 7fef51400700 1 --2- 172.21.15.119:0/3195044279 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fef54102c80 0x7fef5419eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T00:25:19.539 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:19.361+0000 7fef52800700 1 --2- 172.21.15.119:0/3195044279 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fef541073b0 0x7fef541986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T00:25:19.539 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:19.361+0000 7fef51e00700 1 --2- 172.21.15.119:0/3195044279 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fef541022d0 0x7fef5419e620 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fef4400eb30 tx=0x7fef4400eef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:25:19.539 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:19.361+0000 7fef4aa00700 1 -- 172.21.15.119:0/3195044279 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef4400cd40 con 0x7fef541022d0 2024-03-28T00:25:19.539 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:19.361+0000 7fef4aa00700 1 -- 172.21.15.119:0/3195044279 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fef44004500 con 0x7fef541022d0 2024-03-28T00:25:19.539 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:19.361+0000 7fef4aa00700 1 -- 172.21.15.119:0/3195044279 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef44010690 con 0x7fef541022d0 2024-03-28T00:25:19.539 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:19.361+0000 7fef53200700 1 -- 172.21.15.119:0/3195044279 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef541992c0 con 0x7fef541022d0 2024-03-28T00:25:19.539 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:19.361+0000 7fef53200700 1 -- 172.21.15.119:0/3195044279 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef541a55a0 con 0x7fef541022d0 2024-03-28T00:25:19.539 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:19.363+0000 7fef53200700 1 -- 172.21.15.119:0/3195044279 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef541037b0 con 0x7fef541022d0 2024-03-28T00:25:19.539 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:19.364+0000 7fef4aa00700 1 -- 172.21.15.119:0/3195044279 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 17) v1 ==== 137342+0+0 (secure 0 0 0) 0x7fef44003e10 con 0x7fef541022d0 2024-03-28T00:25:19.539 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:19.366+0000 7fef4aa00700 1 --2- 172.21.15.119:0/3195044279 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fef380a29a0 0x7fef380a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:19.539 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:19.366+0000 7fef4aa00700 1 -- 172.21.15.119:0/3195044279 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fef440cef30 con 0x7fef541022d0 2024-03-28T00:25:19.539 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:19.366+0000 7fef51400700 1 --2- 172.21.15.119:0/3195044279 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fef380a29a0 0x7fef380a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:19.540 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:19.367+0000 7fef51400700 1 --2- 172.21.15.119:0/3195044279 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fef380a29a0 0x7fef380a4e60 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fef54199eb0 tx=0x7fef40009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:25:19.540 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:19.369+0000 7fef4aa00700 1 -- 172.21.15.119:0/3195044279 <== mon.2 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fef4409be70 con 0x7fef541022d0 2024-03-28T00:25:19.638 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:19.632+0000 7fef53200700 1 -- 172.21.15.119:0/3195044279 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) v1 -- 0x7fef54199c20 con 0x7fef541022d0 2024-03-28T00:25:19.638 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:19.632+0000 7fef4aa00700 1 -- 172.21.15.119:0/3195044279 <== mon.2 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7fef4409b770 con 0x7fef541022d0 2024-03-28T00:25:19.639 INFO:teuthology.orchestra.run.smithi119.stdout:309237645347 2024-03-28T00:25:19.641 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:19.635+0000 7fef53200700 1 -- 172.21.15.119:0/3195044279 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fef380a29a0 msgr2=0x7fef380a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:19.641 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:19.635+0000 7fef53200700 1 --2- 172.21.15.119:0/3195044279 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fef380a29a0 0x7fef380a4e60 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fef54199eb0 tx=0x7fef40009260 comp rx=0 tx=0).stop 2024-03-28T00:25:19.641 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:19.635+0000 7fef53200700 1 -- 172.21.15.119:0/3195044279 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fef541022d0 msgr2=0x7fef5419e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:19.642 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:19.635+0000 7fef53200700 1 --2- 172.21.15.119:0/3195044279 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fef541022d0 0x7fef5419e620 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fef4400eb30 tx=0x7fef4400eef0 comp rx=0 tx=0).stop 2024-03-28T00:25:19.642 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:19.635+0000 7fef53200700 1 -- 172.21.15.119:0/3195044279 shutdown_connections 2024-03-28T00:25:19.642 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:19.635+0000 7fef53200700 1 --2- 172.21.15.119:0/3195044279 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fef380a29a0 0x7fef380a4e60 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:19.642 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:19.635+0000 7fef53200700 1 --2- 172.21.15.119:0/3195044279 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fef541073b0 0x7fef541986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:19.642 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:19.635+0000 7fef53200700 1 --2- 172.21.15.119:0/3195044279 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fef541022d0 0x7fef5419e620 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:19.642 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:19.635+0000 7fef53200700 1 --2- 172.21.15.119:0/3195044279 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fef54102c80 0x7fef5419eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:19.642 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:19.635+0000 7fef53200700 1 -- 172.21.15.119:0/3195044279 >> 172.21.15.119:0/3195044279 conn(0x7fef540fdb60 msgr2=0x7fef540fe770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:25:19.642 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:19.635+0000 7fef53200700 1 -- 172.21.15.119:0/3195044279 shutdown_connections 2024-03-28T00:25:19.642 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:19.635+0000 7fef53200700 1 -- 172.21.15.119:0/3195044279 wait complete. 2024-03-28T00:25:19.739 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182468 got 107374182473 for osd.3 2024-03-28T00:25:19.739 DEBUG:teuthology.parallel:result is None 2024-03-28T00:25:19.745 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:19.739+0000 7fd296e00700 1 -- 172.21.15.119:0/807987031 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) v1 -- 0x7fd29002da30 con 0x7fd290102440 2024-03-28T00:25:19.746 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:19.739+0000 7fd28cc00700 1 -- 172.21.15.119:0/807987031 <== mon.2 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd28009be40 con 0x7fd290102440 2024-03-28T00:25:19.746 INFO:teuthology.orchestra.run.smithi119.stdout:287762808873 2024-03-28T00:25:19.749 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:19.742+0000 7fd296e00700 1 -- 172.21.15.119:0/807987031 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fd2740a29a0 msgr2=0x7fd2740a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:19.749 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:19.742+0000 7fd296e00700 1 --2- 172.21.15.119:0/807987031 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fd2740a29a0 0x7fd2740a4e60 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fd290108970 tx=0x7fd27c009260 comp rx=0 tx=0).stop 2024-03-28T00:25:19.749 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:19.742+0000 7fd296e00700 1 -- 172.21.15.119:0/807987031 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd290102440 msgr2=0x7fd290074490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:19.749 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:19.742+0000 7fd296e00700 1 --2- 172.21.15.119:0/807987031 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd290102440 0x7fd290074490 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fd28000e9c0 tx=0x7fd28000ecd0 comp rx=0 tx=0).stop 2024-03-28T00:25:19.749 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:19.742+0000 7fd296e00700 1 -- 172.21.15.119:0/807987031 shutdown_connections 2024-03-28T00:25:19.749 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:19.742+0000 7fd296e00700 1 --2- 172.21.15.119:0/807987031 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fd2740a29a0 0x7fd2740a4e60 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:19.749 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:19.742+0000 7fd296e00700 1 --2- 172.21.15.119:0/807987031 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd290107510 0x7fd290074f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:19.749 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:19.742+0000 7fd296e00700 1 --2- 172.21.15.119:0/807987031 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd290102440 0x7fd290074490 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:19.750 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:19.742+0000 7fd296e00700 1 --2- 172.21.15.119:0/807987031 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd290102df0 0x7fd2900749d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:19.750 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:19.742+0000 7fd296e00700 1 -- 172.21.15.119:0/807987031 >> 172.21.15.119:0/807987031 conn(0x7fd2900792f0 msgr2=0x7fd2901001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:25:19.750 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:19.742+0000 7fd296e00700 1 -- 172.21.15.119:0/807987031 shutdown_connections 2024-03-28T00:25:19.750 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:19.742+0000 7fd296e00700 1 -- 172.21.15.119:0/807987031 wait complete. 2024-03-28T00:25:20.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:25:20 smithi119 ceph-mon[31518]: from='client.? 172.21.15.119:0/3195044279' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-03-28T00:25:20.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:25:20 smithi119 ceph-mon[31518]: from='client.? 172.21.15.119:0/807987031' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-03-28T00:25:20.298 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:25:20 smithi165 ceph-mon[36302]: from='client.? 172.21.15.119:0/3195044279' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-03-28T00:25:20.298 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:25:20 smithi165 ceph-mon[36302]: from='client.? 172.21.15.119:0/807987031' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-03-28T00:25:20.452 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:25:20 smithi132 ceph-mon[35941]: from='client.? 172.21.15.119:0/3195044279' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-03-28T00:25:20.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:25:20 smithi132 ceph-mon[35941]: from='client.? 172.21.15.119:0/807987031' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-03-28T00:25:21.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:25:21 smithi119 ceph-mon[31518]: pgmap v355: 97 pgs: 97 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-28T00:25:21.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:25:21 smithi165 ceph-mon[36302]: pgmap v355: 97 pgs: 97 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-28T00:25:21.452 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:25:21 smithi132 ceph-mon[35941]: pgmap v355: 97 pgs: 97 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-28T00:25:21.604 INFO:tasks.cephadm.ceph_manager.ceph:need seq 287762808867 got 287762808873 for osd.9 2024-03-28T00:25:21.605 DEBUG:teuthology.parallel:result is None 2024-03-28T00:25:22.223 INFO:tasks.cephadm.ceph_manager.ceph:need seq 309237645342 got 309237645347 for osd.10 2024-03-28T00:25:22.224 DEBUG:teuthology.parallel:result is None 2024-03-28T00:25:22.325 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:21.153+0000 7f98c7000700 1 -- 172.21.15.119:0/1352246667 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f98c0104e70 msgr2=0x7f98c0108ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:22.325 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:21.153+0000 7f98c7000700 1 --2- 172.21.15.119:0/1352246667 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f98c0104e70 0x7f98c0108ec0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f98ac009990 tx=0x7f98ac009ca0 comp rx=0 tx=0).stop 2024-03-28T00:25:22.325 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:21.153+0000 7f98c7000700 1 -- 172.21.15.119:0/1352246667 shutdown_connections 2024-03-28T00:25:22.325 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:21.153+0000 7f98c7000700 1 --2- 172.21.15.119:0/1352246667 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f98c01095a0 0x7f98c0111170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:22.325 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:21.153+0000 7f98c7000700 1 --2- 172.21.15.119:0/1352246667 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f98c01044c0 0x7f98c01048a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:22.325 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:21.153+0000 7f98c7000700 1 --2- 172.21.15.119:0/1352246667 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f98c0104e70 0x7f98c0108ec0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:22.325 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:21.153+0000 7f98c7000700 1 -- 172.21.15.119:0/1352246667 >> 172.21.15.119:0/1352246667 conn(0x7f98c00ffd70 msgr2=0x7f98c0102190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:25:22.325 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:21.153+0000 7f98c7000700 1 -- 172.21.15.119:0/1352246667 shutdown_connections 2024-03-28T00:25:22.325 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:21.154+0000 7f98c7000700 1 -- 172.21.15.119:0/1352246667 wait complete. 2024-03-28T00:25:22.325 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:21.154+0000 7f98c7000700 1 Processor -- start 2024-03-28T00:25:22.326 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:21.155+0000 7f98c7000700 1 -- start start 2024-03-28T00:25:22.326 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:21.155+0000 7f98c7000700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f98c01044c0 0x7f98c01a06f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:22.326 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:21.155+0000 7f98c7000700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f98c0104e70 0x7f98c01a0c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:22.326 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:21.155+0000 7f98c7000700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f98c01095a0 0x7f98c019a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:22.326 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:21.155+0000 7f98c7000700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98c01145b0 con 0x7f98c0104e70 2024-03-28T00:25:22.326 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:21.155+0000 7f98c7000700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98c0114430 con 0x7f98c01044c0 2024-03-28T00:25:22.326 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:21.155+0000 7f98c7000700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98c0114730 con 0x7f98c01095a0 2024-03-28T00:25:22.326 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:21.155+0000 7f98c5c00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f98c01044c0 0x7f98c01a06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:22.326 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:21.155+0000 7f98c6600700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f98c01095a0 0x7f98c019a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:22.326 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:21.155+0000 7f98c5c00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f98c01044c0 0x7f98c01a06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.119:42432/0 (socket says 172.21.15.119:42432) 2024-03-28T00:25:22.326 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:21.155+0000 7f98c5c00700 1 -- 172.21.15.119:0/4234972784 learned_addr learned my addr 172.21.15.119:0/4234972784 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:25:22.326 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:21.155+0000 7f98c5200700 1 --2- 172.21.15.119:0/4234972784 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f98c0104e70 0x7f98c01a0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:22.326 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:21.156+0000 7f98c6600700 1 -- 172.21.15.119:0/4234972784 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f98c01044c0 msgr2=0x7f98c01a06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:22.327 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:21.156+0000 7f98c6600700 1 --2- 172.21.15.119:0/4234972784 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f98c01044c0 0x7f98c01a06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:22.327 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:21.156+0000 7f98c6600700 1 -- 172.21.15.119:0/4234972784 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f98c0104e70 msgr2=0x7f98c01a0c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:22.327 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:21.156+0000 7f98c6600700 1 --2- 172.21.15.119:0/4234972784 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f98c0104e70 0x7f98c01a0c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:22.327 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:21.156+0000 7f98c6600700 1 -- 172.21.15.119:0/4234972784 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98c019b010 con 0x7f98c01095a0 2024-03-28T00:25:22.327 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:21.156+0000 7f98c5c00700 1 --2- 172.21.15.119:0/4234972784 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f98c01044c0 0x7f98c01a06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T00:25:22.327 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:21.156+0000 7f98c6600700 1 --2- 172.21.15.119:0/4234972784 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f98c01095a0 0x7f98c019a8b0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f98b800e900 tx=0x7f98b800ec10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:25:22.327 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:21.156+0000 7f98be000700 1 -- 172.21.15.119:0/4234972784 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98b8004220 con 0x7f98c01095a0 2024-03-28T00:25:22.327 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:21.156+0000 7f98c7000700 1 -- 172.21.15.119:0/4234972784 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98c019b300 con 0x7f98c01095a0 2024-03-28T00:25:22.327 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:21.156+0000 7f98be000700 1 -- 172.21.15.119:0/4234972784 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f98b80043c0 con 0x7f98c01095a0 2024-03-28T00:25:22.327 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:21.156+0000 7f98be000700 1 -- 172.21.15.119:0/4234972784 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98b8010430 con 0x7f98c01095a0 2024-03-28T00:25:22.327 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:21.156+0000 7f98c7000700 1 -- 172.21.15.119:0/4234972784 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98c01a75c0 con 0x7f98c01095a0 2024-03-28T00:25:22.327 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:21.157+0000 7f98c7000700 1 -- 172.21.15.119:0/4234972784 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98c0068e50 con 0x7f98c01095a0 2024-03-28T00:25:22.327 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:21.159+0000 7f98be000700 1 -- 172.21.15.119:0/4234972784 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 17) v1 ==== 137342+0+0 (secure 0 0 0) 0x7f98b800c780 con 0x7f98c01095a0 2024-03-28T00:25:22.327 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:21.161+0000 7f98be000700 1 --2- 172.21.15.119:0/4234972784 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f98a80a2920 0x7f98a80a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:22.328 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:21.161+0000 7f98be000700 1 -- 172.21.15.119:0/4234972784 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f98b80ced20 con 0x7f98c01095a0 2024-03-28T00:25:22.328 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:21.161+0000 7f98c5c00700 1 --2- 172.21.15.119:0/4234972784 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f98a80a2920 0x7f98a80a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:22.328 INFO:teuthology.orchestra.run.smithi119.stdout:163208757312 2024-03-28T00:25:22.328 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:21.162+0000 7f98c5c00700 1 --2- 172.21.15.119:0/4234972784 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f98a80a2920 0x7f98a80a4de0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f98b4009c10 tx=0x7f98b400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:25:22.328 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:21.163+0000 7f98be000700 1 -- 172.21.15.119:0/4234972784 <== mon.1 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f98b800f3c0 con 0x7f98c01095a0 2024-03-28T00:25:22.329 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:21.385+0000 7f98c7000700 1 -- 172.21.15.119:0/4234972784 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f98c002da30 con 0x7f98c01095a0 2024-03-28T00:25:22.329 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:21.385+0000 7f98be000700 1 -- 172.21.15.119:0/4234972784 <== mon.1 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f98b809bc60 con 0x7f98c01095a0 2024-03-28T00:25:22.329 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:21.388+0000 7f98c7000700 1 -- 172.21.15.119:0/4234972784 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f98a80a2920 msgr2=0x7f98a80a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:22.329 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:21.388+0000 7f98c7000700 1 --2- 172.21.15.119:0/4234972784 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f98a80a2920 0x7f98a80a4de0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f98b4009c10 tx=0x7f98b400b040 comp rx=0 tx=0).stop 2024-03-28T00:25:22.329 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:21.388+0000 7f98c7000700 1 -- 172.21.15.119:0/4234972784 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f98c01095a0 msgr2=0x7f98c019a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:22.329 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:21.388+0000 7f98c7000700 1 --2- 172.21.15.119:0/4234972784 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f98c01095a0 0x7f98c019a8b0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f98b800e900 tx=0x7f98b800ec10 comp rx=0 tx=0).stop 2024-03-28T00:25:22.329 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:21.388+0000 7f98c7000700 1 -- 172.21.15.119:0/4234972784 shutdown_connections 2024-03-28T00:25:22.329 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:21.388+0000 7f98c7000700 1 --2- 172.21.15.119:0/4234972784 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f98a80a2920 0x7f98a80a4de0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:22.329 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:21.388+0000 7f98c7000700 1 --2- 172.21.15.119:0/4234972784 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f98c01095a0 0x7f98c019a8b0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:22.329 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:21.388+0000 7f98c7000700 1 --2- 172.21.15.119:0/4234972784 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f98c01044c0 0x7f98c01a06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:22.329 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:21.388+0000 7f98c7000700 1 --2- 172.21.15.119:0/4234972784 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f98c0104e70 0x7f98c01a0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:22.329 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:21.388+0000 7f98c7000700 1 -- 172.21.15.119:0/4234972784 >> 172.21.15.119:0/4234972784 conn(0x7f98c00ffd70 msgr2=0x7f98c0102120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:25:22.329 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:21.389+0000 7f98c7000700 1 -- 172.21.15.119:0/4234972784 shutdown_connections 2024-03-28T00:25:22.330 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:21.389+0000 7f98c7000700 1 -- 172.21.15.119:0/4234972784 wait complete. 2024-03-28T00:25:22.330 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:25:22 smithi119 ceph-mon[31518]: from='client.? 172.21.15.119:0/4234972784' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-28T00:25:22.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:25:22 smithi165 ceph-mon[36302]: from='client.? 172.21.15.119:0/4234972784' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-28T00:25:22.452 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:25:22 smithi132 ceph-mon[35941]: from='client.? 172.21.15.119:0/4234972784' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-28T00:25:23.056 INFO:tasks.cephadm.ceph_manager.ceph:need seq 163208757306 got 163208757312 for osd.5 2024-03-28T00:25:23.056 DEBUG:teuthology.parallel:result is None 2024-03-28T00:25:23.056 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-03-28T00:25:23.057 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 10f73b92-ec98-11ee-95d0-87774f69a715 -- ceph pg dump --format=json 2024-03-28T00:25:23.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:25:23 smithi165 ceph-mon[36302]: pgmap v356: 97 pgs: 97 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T00:25:23.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:25:23 smithi119 ceph-mon[31518]: pgmap v356: 97 pgs: 97 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T00:25:23.452 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:25:23 smithi132 ceph-mon[35941]: pgmap v356: 97 pgs: 97 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T00:25:23.486 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/mon.a/config 2024-03-28T00:25:25.291 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:25:25 smithi119 ceph-mon[31518]: pgmap v357: 97 pgs: 97 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-28T00:25:25.291 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:25:25 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:25:25.291 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:25:25 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:25:25.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:25:25 smithi165 ceph-mon[36302]: pgmap v357: 97 pgs: 97 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-28T00:25:25.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:25:25 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:25:25.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:25:25 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:25:25.452 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:25:25 smithi132 ceph-mon[35941]: pgmap v357: 97 pgs: 97 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-28T00:25:25.452 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:25:25 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:25:25.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:25:25 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:25:25.906 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:25.898+0000 7fb6d1a00700 1 -- 172.21.15.119:0/3868928684 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb6cc06a7e0 msgr2=0x7fb6cc10f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:25.906 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:25.898+0000 7fb6d1a00700 1 --2- 172.21.15.119:0/3868928684 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb6cc06a7e0 0x7fb6cc10f020 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fb6c400b0d0 tx=0x7fb6c400b3e0 comp rx=0 tx=0).stop 2024-03-28T00:25:25.906 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:25.899+0000 7fb6d1a00700 1 -- 172.21.15.119:0/3868928684 shutdown_connections 2024-03-28T00:25:25.906 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:25.899+0000 7fb6d1a00700 1 --2- 172.21.15.119:0/3868928684 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb6cc06a7e0 0x7fb6cc10f020 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:25.906 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:25.899+0000 7fb6d1a00700 1 --2- 172.21.15.119:0/3868928684 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb6cc1081e0 0x7fb6cc1085c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:25.906 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:25.899+0000 7fb6d1a00700 1 --2- 172.21.15.119:0/3868928684 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb6cc069e40 0x7fb6cc06a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:25.906 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:25.899+0000 7fb6d1a00700 1 -- 172.21.15.119:0/3868928684 >> 172.21.15.119:0/3868928684 conn(0x7fb6cc077fc0 msgr2=0x7fb6cc0783d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:25:25.906 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:25.899+0000 7fb6d1a00700 1 -- 172.21.15.119:0/3868928684 shutdown_connections 2024-03-28T00:25:25.906 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:25.899+0000 7fb6d1a00700 1 -- 172.21.15.119:0/3868928684 wait complete. 2024-03-28T00:25:25.908 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:25.900+0000 7fb6d1a00700 1 Processor -- start 2024-03-28T00:25:25.908 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:25.900+0000 7fb6d1a00700 1 -- start start 2024-03-28T00:25:25.908 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:25.901+0000 7fb6d1a00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb6cc069e40 0x7fb6cc1a06e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:25.908 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:25.901+0000 7fb6d1a00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb6cc06a7e0 0x7fb6cc1a0c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:25.908 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:25.901+0000 7fb6d1a00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb6cc1081e0 0x7fb6cc19a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:25.908 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:25.901+0000 7fb6d1a00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6cc112460 con 0x7fb6cc1081e0 2024-03-28T00:25:25.908 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:25.901+0000 7fb6d1a00700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6cc1122e0 con 0x7fb6cc06a7e0 2024-03-28T00:25:25.908 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:25.901+0000 7fb6d1a00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6cc1125e0 con 0x7fb6cc069e40 2024-03-28T00:25:25.908 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:25.901+0000 7fb6ca000700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb6cc06a7e0 0x7fb6cc1a0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:25.909 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:25.901+0000 7fb6ca000700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb6cc06a7e0 0x7fb6cc1a0c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.119:51172/0 (socket says 172.21.15.119:51172) 2024-03-28T00:25:25.909 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:25.901+0000 7fb6caa00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb6cc069e40 0x7fb6cc1a06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:25.909 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:25.901+0000 7fb6ca000700 1 -- 172.21.15.119:0/1575696243 learned_addr learned my addr 172.21.15.119:0/1575696243 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:25:25.909 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:25.901+0000 7fb6cb400700 1 --2- 172.21.15.119:0/1575696243 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb6cc1081e0 0x7fb6cc19a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:25.909 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:25.901+0000 7fb6ca000700 1 -- 172.21.15.119:0/1575696243 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb6cc069e40 msgr2=0x7fb6cc1a06e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:25.909 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:25.901+0000 7fb6ca000700 1 --2- 172.21.15.119:0/1575696243 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb6cc069e40 0x7fb6cc1a06e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:25.909 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:25.901+0000 7fb6ca000700 1 -- 172.21.15.119:0/1575696243 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb6cc1081e0 msgr2=0x7fb6cc19a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:25.909 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:25.901+0000 7fb6ca000700 1 --2- 172.21.15.119:0/1575696243 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb6cc1081e0 0x7fb6cc19a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:25.909 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:25.901+0000 7fb6ca000700 1 -- 172.21.15.119:0/1575696243 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6cc19b000 con 0x7fb6cc06a7e0 2024-03-28T00:25:25.909 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:25.901+0000 7fb6caa00700 1 --2- 172.21.15.119:0/1575696243 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb6cc069e40 0x7fb6cc1a06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T00:25:25.909 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:25.901+0000 7fb6cb400700 1 --2- 172.21.15.119:0/1575696243 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb6cc1081e0 0x7fb6cc19a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T00:25:25.909 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:25.902+0000 7fb6ca000700 1 --2- 172.21.15.119:0/1575696243 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb6cc06a7e0 0x7fb6cc1a0c20 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fb6b800d980 tx=0x7fb6b800dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:25:25.910 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:25.902+0000 7fb6c3400700 1 -- 172.21.15.119:0/1575696243 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6b8016070 con 0x7fb6cc06a7e0 2024-03-28T00:25:25.910 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:25.902+0000 7fb6d1a00700 1 -- 172.21.15.119:0/1575696243 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6cc19b2f0 con 0x7fb6cc06a7e0 2024-03-28T00:25:25.910 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:25.902+0000 7fb6c3400700 1 -- 172.21.15.119:0/1575696243 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb6b8004d10 con 0x7fb6cc06a7e0 2024-03-28T00:25:25.910 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:25.902+0000 7fb6c3400700 1 -- 172.21.15.119:0/1575696243 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6b80055f0 con 0x7fb6cc06a7e0 2024-03-28T00:25:25.911 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:25.902+0000 7fb6d1a00700 1 -- 172.21.15.119:0/1575696243 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6cc1a75b0 con 0x7fb6cc06a7e0 2024-03-28T00:25:25.911 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:25.903+0000 7fb6d1a00700 1 -- 172.21.15.119:0/1575696243 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6cc068e50 con 0x7fb6cc06a7e0 2024-03-28T00:25:25.911 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:25.905+0000 7fb6c3400700 1 -- 172.21.15.119:0/1575696243 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 17) v1 ==== 137342+0+0 (secure 0 0 0) 0x7fb6b8004830 con 0x7fb6cc06a7e0 2024-03-28T00:25:25.913 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:25.906+0000 7fb6c3400700 1 --2- 172.21.15.119:0/1575696243 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fb6b00a2920 0x7fb6b00a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:25.913 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:25.907+0000 7fb6c3400700 1 -- 172.21.15.119:0/1575696243 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fb6b80cea90 con 0x7fb6cc06a7e0 2024-03-28T00:25:25.913 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:25.907+0000 7fb6caa00700 1 --2- 172.21.15.119:0/1575696243 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fb6b00a2920 0x7fb6b00a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:25.914 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:25.908+0000 7fb6caa00700 1 --2- 172.21.15.119:0/1575696243 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fb6b00a2920 0x7fb6b00a4de0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fb6cc19b9e0 tx=0x7fb6bc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:25:25.916 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:25.910+0000 7fb6c3400700 1 -- 172.21.15.119:0/1575696243 <== mon.2 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fb6b801f020 con 0x7fb6cc06a7e0 2024-03-28T00:25:26.139 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:26.132+0000 7fb6d1a00700 1 -- 172.21.15.119:0/1575696243 --> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb6cc10da80 con 0x7fb6b00a2920 2024-03-28T00:25:26.151 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:26.145+0000 7fb6c3400700 1 -- 172.21.15.119:0/1575696243 <== mgr.14152 v2:172.21.15.119:6800/1401937096 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+258965 (secure 0 0 0) 0x7fb6cc10da80 con 0x7fb6b00a2920 2024-03-28T00:25:26.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:25:26 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:25:26.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:25:26 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:25:26.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:25:26 smithi119 ceph-mon[31518]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:25:26.153 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-28T00:25:26.159 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:26.152+0000 7fb6d1a00700 1 -- 172.21.15.119:0/1575696243 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fb6b00a2920 msgr2=0x7fb6b00a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:26.159 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:26.152+0000 7fb6d1a00700 1 --2- 172.21.15.119:0/1575696243 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fb6b00a2920 0x7fb6b00a4de0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fb6cc19b9e0 tx=0x7fb6bc009260 comp rx=0 tx=0).stop 2024-03-28T00:25:26.160 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:26.152+0000 7fb6d1a00700 1 -- 172.21.15.119:0/1575696243 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb6cc06a7e0 msgr2=0x7fb6cc1a0c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:26.160 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:26.152+0000 7fb6d1a00700 1 --2- 172.21.15.119:0/1575696243 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb6cc06a7e0 0x7fb6cc1a0c20 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fb6b800d980 tx=0x7fb6b800dc90 comp rx=0 tx=0).stop 2024-03-28T00:25:26.160 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:26.152+0000 7fb6d1a00700 1 -- 172.21.15.119:0/1575696243 shutdown_connections 2024-03-28T00:25:26.160 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:26.152+0000 7fb6d1a00700 1 --2- 172.21.15.119:0/1575696243 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fb6b00a2920 0x7fb6b00a4de0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:26.160 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:26.152+0000 7fb6d1a00700 1 --2- 172.21.15.119:0/1575696243 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb6cc1081e0 0x7fb6cc19a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:26.160 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:26.152+0000 7fb6d1a00700 1 --2- 172.21.15.119:0/1575696243 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb6cc069e40 0x7fb6cc1a06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:26.160 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:26.152+0000 7fb6d1a00700 1 --2- 172.21.15.119:0/1575696243 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb6cc06a7e0 0x7fb6cc1a0c20 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:26.160 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:26.152+0000 7fb6d1a00700 1 -- 172.21.15.119:0/1575696243 >> 172.21.15.119:0/1575696243 conn(0x7fb6cc077fc0 msgr2=0x7fb6cc10d370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:25:26.160 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:26.152+0000 7fb6d1a00700 1 -- 172.21.15.119:0/1575696243 shutdown_connections 2024-03-28T00:25:26.160 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:26.152+0000 7fb6d1a00700 1 -- 172.21.15.119:0/1575696243 wait complete. 2024-03-28T00:25:26.160 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2024-03-28T00:25:26.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:25:26 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:25:26.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:25:26 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:25:26.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:25:26 smithi165 ceph-mon[36302]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:25:26.452 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:25:26 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:25:26.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:25:26 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:25:26.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:25:26 smithi132 ceph-mon[35941]: from='mgr.14152 172.21.15.119:0/2710417582' entity='mgr.x' 2024-03-28T00:25:27.077 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":358,"stamp":"2024-03-28T00:25:25.757125+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":662599,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":959,"num_read_kb":607,"num_write":423,"num_write_kb":3077,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1180758,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":348,"ondisk_log_size":348,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":373392,"kb_used_data":11712,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124568432,"statfs":{"total":1151940427776,"available":1151558074368,"internally_reserved":0,"allocated":11993088,"data_stored":7452453,"data_compressed":112659,"data_compressed_allocated":147456,"data_compressed_original":1794048,"omap_allocated":19143,"internal_metadata":370193721},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":105,"num_read_kb":50,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005036"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.536838+0000","last_change":"2024-03-28T00:23:56.763630+0000","last_active":"2024-03-28T00:24:05.536838+0000","last_peered":"2024-03-28T00:24:05.536838+0000","last_clean":"2024-03-28T00:24:05.536838+0000","last_became_active":"2024-03-28T00:23:56.763296+0000","last_became_peered":"2024-03-28T00:23:56.763296+0000","last_unstale":"2024-03-28T00:24:05.536838+0000","last_undegraded":"2024-03-28T00:24:05.536838+0000","last_fullsized":"2024-03-28T00:24:05.536838+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T11:40:36.071225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.554106+0000","last_change":"2024-03-28T00:23:56.763628+0000","last_active":"2024-03-28T00:24:05.554106+0000","last_peered":"2024-03-28T00:24:05.554106+0000","last_clean":"2024-03-28T00:24:05.554106+0000","last_became_active":"2024-03-28T00:23:56.761753+0000","last_became_peered":"2024-03-28T00:23:56.761753+0000","last_unstale":"2024-03-28T00:24:05.554106+0000","last_undegraded":"2024-03-28T00:24:05.554106+0000","last_fullsized":"2024-03-28T00:24:05.554106+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T11:48:50.804269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.553822+0000","last_change":"2024-03-28T00:23:56.759198+0000","last_active":"2024-03-28T00:24:05.553822+0000","last_peered":"2024-03-28T00:24:05.553822+0000","last_clean":"2024-03-28T00:24:05.553822+0000","last_became_active":"2024-03-28T00:23:56.758780+0000","last_became_peered":"2024-03-28T00:23:56.758780+0000","last_unstale":"2024-03-28T00:24:05.553822+0000","last_undegraded":"2024-03-28T00:24:05.553822+0000","last_fullsized":"2024-03-28T00:24:05.553822+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T10:46:15.731193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.538176+0000","last_change":"2024-03-28T00:23:56.736651+0000","last_active":"2024-03-28T00:24:05.538176+0000","last_peered":"2024-03-28T00:24:05.538176+0000","last_clean":"2024-03-28T00:24:05.538176+0000","last_became_active":"2024-03-28T00:23:56.736063+0000","last_became_peered":"2024-03-28T00:23:56.736063+0000","last_unstale":"2024-03-28T00:24:05.538176+0000","last_undegraded":"2024-03-28T00:24:05.538176+0000","last_fullsized":"2024-03-28T00:24:05.538176+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T09:34:28.221202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.534628+0000","last_change":"2024-03-28T00:23:56.751413+0000","last_active":"2024-03-28T00:24:05.534628+0000","last_peered":"2024-03-28T00:24:05.534628+0000","last_clean":"2024-03-28T00:24:05.534628+0000","last_became_active":"2024-03-28T00:23:56.751038+0000","last_became_peered":"2024-03-28T00:23:56.751038+0000","last_unstale":"2024-03-28T00:24:05.534628+0000","last_undegraded":"2024-03-28T00:24:05.534628+0000","last_fullsized":"2024-03-28T00:24:05.534628+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T10:45:08.645799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.539576+0000","last_change":"2024-03-28T00:23:56.750970+0000","last_active":"2024-03-28T00:24:05.539576+0000","last_peered":"2024-03-28T00:24:05.539576+0000","last_clean":"2024-03-28T00:24:05.539576+0000","last_became_active":"2024-03-28T00:23:56.750608+0000","last_became_peered":"2024-03-28T00:23:56.750608+0000","last_unstale":"2024-03-28T00:24:05.539576+0000","last_undegraded":"2024-03-28T00:24:05.539576+0000","last_fullsized":"2024-03-28T00:24:05.539576+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T00:32:07.448685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.534811+0000","last_change":"2024-03-28T00:23:56.751489+0000","last_active":"2024-03-28T00:24:05.534811+0000","last_peered":"2024-03-28T00:24:05.534811+0000","last_clean":"2024-03-28T00:24:05.534811+0000","last_became_active":"2024-03-28T00:23:56.750531+0000","last_became_peered":"2024-03-28T00:23:56.750531+0000","last_unstale":"2024-03-28T00:24:05.534811+0000","last_undegraded":"2024-03-28T00:24:05.534811+0000","last_fullsized":"2024-03-28T00:24:05.534811+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T11:18:22.604126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.973011+0000","last_change":"2024-03-28T00:23:56.751537+0000","last_active":"2024-03-28T00:24:05.973011+0000","last_peered":"2024-03-28T00:24:05.973011+0000","last_clean":"2024-03-28T00:24:05.973011+0000","last_became_active":"2024-03-28T00:23:56.751185+0000","last_became_peered":"2024-03-28T00:23:56.751185+0000","last_unstale":"2024-03-28T00:24:05.973011+0000","last_undegraded":"2024-03-28T00:24:05.973011+0000","last_fullsized":"2024-03-28T00:24:05.973011+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T03:21:30.334886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.583110+0000","last_change":"2024-03-28T00:23:56.739405+0000","last_active":"2024-03-28T00:24:05.583110+0000","last_peered":"2024-03-28T00:24:05.583110+0000","last_clean":"2024-03-28T00:24:05.583110+0000","last_became_active":"2024-03-28T00:23:56.739085+0000","last_became_peered":"2024-03-28T00:23:56.739085+0000","last_unstale":"2024-03-28T00:24:05.583110+0000","last_undegraded":"2024-03-28T00:24:05.583110+0000","last_fullsized":"2024-03-28T00:24:05.583110+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T10:57:08.892446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.539895+0000","last_change":"2024-03-28T00:23:56.736880+0000","last_active":"2024-03-28T00:24:05.539895+0000","last_peered":"2024-03-28T00:24:05.539895+0000","last_clean":"2024-03-28T00:24:05.539895+0000","last_became_active":"2024-03-28T00:23:56.736524+0000","last_became_peered":"2024-03-28T00:23:56.736524+0000","last_unstale":"2024-03-28T00:24:05.539895+0000","last_undegraded":"2024-03-28T00:24:05.539895+0000","last_fullsized":"2024-03-28T00:24:05.539895+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T09:55:19.552415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.535208+0000","last_change":"2024-03-28T00:23:56.736717+0000","last_active":"2024-03-28T00:24:05.535208+0000","last_peered":"2024-03-28T00:24:05.535208+0000","last_clean":"2024-03-28T00:24:05.535208+0000","last_became_active":"2024-03-28T00:23:56.735981+0000","last_became_peered":"2024-03-28T00:23:56.735981+0000","last_unstale":"2024-03-28T00:24:05.535208+0000","last_undegraded":"2024-03-28T00:24:05.535208+0000","last_fullsized":"2024-03-28T00:24:05.535208+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T12:02:54.293418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.538045+0000","last_change":"2024-03-28T00:23:56.740591+0000","last_active":"2024-03-28T00:24:05.538045+0000","last_peered":"2024-03-28T00:24:05.538045+0000","last_clean":"2024-03-28T00:24:05.538045+0000","last_became_active":"2024-03-28T00:23:56.739965+0000","last_became_peered":"2024-03-28T00:23:56.739965+0000","last_unstale":"2024-03-28T00:24:05.538045+0000","last_undegraded":"2024-03-28T00:24:05.538045+0000","last_fullsized":"2024-03-28T00:24:05.538045+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T07:10:35.426260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.538408+0000","last_change":"2024-03-28T00:23:56.744686+0000","last_active":"2024-03-28T00:24:05.538408+0000","last_peered":"2024-03-28T00:24:05.538408+0000","last_clean":"2024-03-28T00:24:05.538408+0000","last_became_active":"2024-03-28T00:23:56.744355+0000","last_became_peered":"2024-03-28T00:23:56.744355+0000","last_unstale":"2024-03-28T00:24:05.538408+0000","last_undegraded":"2024-03-28T00:24:05.538408+0000","last_fullsized":"2024-03-28T00:24:05.538408+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T07:52:27.360917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.553265+0000","last_change":"2024-03-28T00:23:56.762465+0000","last_active":"2024-03-28T00:24:05.553265+0000","last_peered":"2024-03-28T00:24:05.553265+0000","last_clean":"2024-03-28T00:24:05.553265+0000","last_became_active":"2024-03-28T00:23:56.751467+0000","last_became_peered":"2024-03-28T00:23:56.751467+0000","last_unstale":"2024-03-28T00:24:05.553265+0000","last_undegraded":"2024-03-28T00:24:05.553265+0000","last_fullsized":"2024-03-28T00:24:05.553265+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T10:55:33.880872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.553329+0000","last_change":"2024-03-28T00:23:56.759344+0000","last_active":"2024-03-28T00:24:05.553329+0000","last_peered":"2024-03-28T00:24:05.553329+0000","last_clean":"2024-03-28T00:24:05.553329+0000","last_became_active":"2024-03-28T00:23:56.758694+0000","last_became_peered":"2024-03-28T00:23:56.758694+0000","last_unstale":"2024-03-28T00:24:05.553329+0000","last_undegraded":"2024-03-28T00:24:05.553329+0000","last_fullsized":"2024-03-28T00:24:05.553329+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T02:43:42.586070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.582511+0000","last_change":"2024-03-28T00:23:56.759298+0000","last_active":"2024-03-28T00:24:05.582511+0000","last_peered":"2024-03-28T00:24:05.582511+0000","last_clean":"2024-03-28T00:24:05.582511+0000","last_became_active":"2024-03-28T00:23:56.759006+0000","last_became_peered":"2024-03-28T00:23:56.759006+0000","last_unstale":"2024-03-28T00:24:05.582511+0000","last_undegraded":"2024-03-28T00:24:05.582511+0000","last_fullsized":"2024-03-28T00:24:05.582511+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T00:43:09.433789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.536834+0000","last_change":"2024-03-28T00:23:56.760324+0000","last_active":"2024-03-28T00:24:05.536834+0000","last_peered":"2024-03-28T00:24:05.536834+0000","last_clean":"2024-03-28T00:24:05.536834+0000","last_became_active":"2024-03-28T00:23:56.759866+0000","last_became_peered":"2024-03-28T00:23:56.759866+0000","last_unstale":"2024-03-28T00:24:05.536834+0000","last_undegraded":"2024-03-28T00:24:05.536834+0000","last_fullsized":"2024-03-28T00:24:05.536834+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T01:11:15.923114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.547070+0000","last_change":"2024-03-28T00:23:56.755227+0000","last_active":"2024-03-28T00:24:05.547070+0000","last_peered":"2024-03-28T00:24:05.547070+0000","last_clean":"2024-03-28T00:24:05.547070+0000","last_became_active":"2024-03-28T00:23:56.754767+0000","last_became_peered":"2024-03-28T00:23:56.754767+0000","last_unstale":"2024-03-28T00:24:05.547070+0000","last_undegraded":"2024-03-28T00:24:05.547070+0000","last_fullsized":"2024-03-28T00:24:05.547070+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T07:26:04.797904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.582571+0000","last_change":"2024-03-28T00:23:56.744199+0000","last_active":"2024-03-28T00:24:05.582571+0000","last_peered":"2024-03-28T00:24:05.582571+0000","last_clean":"2024-03-28T00:24:05.582571+0000","last_became_active":"2024-03-28T00:23:56.743848+0000","last_became_peered":"2024-03-28T00:23:56.743848+0000","last_unstale":"2024-03-28T00:24:05.582571+0000","last_undegraded":"2024-03-28T00:24:05.582571+0000","last_fullsized":"2024-03-28T00:24:05.582571+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T10:08:01.426115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'13","reported_seq":111,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:25:20.627067+0000","last_change":"2024-03-28T00:23:55.731996+0000","last_active":"2024-03-28T00:25:20.627067+0000","last_peered":"2024-03-28T00:25:20.627067+0000","last_clean":"2024-03-28T00:25:20.627067+0000","last_became_active":"2024-03-28T00:23:55.731626+0000","last_became_peered":"2024-03-28T00:23:55.731626+0000","last_unstale":"2024-03-28T00:25:20.627067+0000","last_undegraded":"2024-03-28T00:25:20.627067+0000","last_fullsized":"2024-03-28T00:25:20.627067+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_clean_scrub_stamp":"2024-03-28T00:23:54.693629+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T08:56:00.420387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":79,"num_read_kb":79,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.973978+0000","last_change":"2024-03-28T00:23:56.757048+0000","last_active":"2024-03-28T00:24:05.973978+0000","last_peered":"2024-03-28T00:24:05.973978+0000","last_clean":"2024-03-28T00:24:05.973978+0000","last_became_active":"2024-03-28T00:23:56.752710+0000","last_became_peered":"2024-03-28T00:23:56.752710+0000","last_unstale":"2024-03-28T00:24:05.973978+0000","last_undegraded":"2024-03-28T00:24:05.973978+0000","last_fullsized":"2024-03-28T00:24:05.973978+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T06:26:39.723255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.974155+0000","last_change":"2024-03-28T00:23:55.748191+0000","last_active":"2024-03-28T00:24:05.974155+0000","last_peered":"2024-03-28T00:24:05.974155+0000","last_clean":"2024-03-28T00:24:05.974155+0000","last_became_active":"2024-03-28T00:23:55.747154+0000","last_became_peered":"2024-03-28T00:23:55.747154+0000","last_unstale":"2024-03-28T00:24:05.974155+0000","last_undegraded":"2024-03-28T00:24:05.974155+0000","last_fullsized":"2024-03-28T00:24:05.974155+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_clean_scrub_stamp":"2024-03-28T00:23:54.693629+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T07:17:12.296494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.553663+0000","last_change":"2024-03-28T00:23:56.758979+0000","last_active":"2024-03-28T00:24:05.553663+0000","last_peered":"2024-03-28T00:24:05.553663+0000","last_clean":"2024-03-28T00:24:05.553663+0000","last_became_active":"2024-03-28T00:23:56.752628+0000","last_became_peered":"2024-03-28T00:23:56.752628+0000","last_unstale":"2024-03-28T00:24:05.553663+0000","last_undegraded":"2024-03-28T00:24:05.553663+0000","last_fullsized":"2024-03-28T00:24:05.553663+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T10:11:57.245938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"85'12","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.535173+0000","last_change":"2024-03-28T00:23:55.728972+0000","last_active":"2024-03-28T00:24:05.535173+0000","last_peered":"2024-03-28T00:24:05.535173+0000","last_clean":"2024-03-28T00:24:05.535173+0000","last_became_active":"2024-03-28T00:23:55.728053+0000","last_became_peered":"2024-03-28T00:23:55.728053+0000","last_unstale":"2024-03-28T00:24:05.535173+0000","last_undegraded":"2024-03-28T00:24:05.535173+0000","last_fullsized":"2024-03-28T00:24:05.535173+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_clean_scrub_stamp":"2024-03-28T00:23:54.693629+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T04:12:27.129196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.582740+0000","last_change":"2024-03-28T00:23:56.752585+0000","last_active":"2024-03-28T00:24:05.582740+0000","last_peered":"2024-03-28T00:24:05.582740+0000","last_clean":"2024-03-28T00:24:05.582740+0000","last_became_active":"2024-03-28T00:23:56.752252+0000","last_became_peered":"2024-03-28T00:23:56.752252+0000","last_unstale":"2024-03-28T00:24:05.582740+0000","last_undegraded":"2024-03-28T00:24:05.582740+0000","last_fullsized":"2024-03-28T00:24:05.582740+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T11:51:02.864486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'5","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.534887+0000","last_change":"2024-03-28T00:23:55.726859+0000","last_active":"2024-03-28T00:24:05.534887+0000","last_peered":"2024-03-28T00:24:05.534887+0000","last_clean":"2024-03-28T00:24:05.534887+0000","last_became_active":"2024-03-28T00:23:55.725372+0000","last_became_peered":"2024-03-28T00:23:55.725372+0000","last_unstale":"2024-03-28T00:24:05.534887+0000","last_undegraded":"2024-03-28T00:24:05.534887+0000","last_fullsized":"2024-03-28T00:24:05.534887+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_clean_scrub_stamp":"2024-03-28T00:23:54.693629+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T00:52:35.278544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.538784+0000","last_change":"2024-03-28T00:23:56.742076+0000","last_active":"2024-03-28T00:24:05.538784+0000","last_peered":"2024-03-28T00:24:05.538784+0000","last_clean":"2024-03-28T00:24:05.538784+0000","last_became_active":"2024-03-28T00:23:56.741761+0000","last_became_peered":"2024-03-28T00:23:56.741761+0000","last_unstale":"2024-03-28T00:24:05.538784+0000","last_undegraded":"2024-03-28T00:24:05.538784+0000","last_fullsized":"2024-03-28T00:24:05.538784+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T08:17:02.024132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"85'6","reported_seq":106,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:25:21.596862+0000","last_change":"2024-03-28T00:23:55.728587+0000","last_active":"2024-03-28T00:25:21.596862+0000","last_peered":"2024-03-28T00:25:21.596862+0000","last_clean":"2024-03-28T00:25:21.596862+0000","last_became_active":"2024-03-28T00:23:55.728309+0000","last_became_peered":"2024-03-28T00:23:55.728309+0000","last_unstale":"2024-03-28T00:25:21.596862+0000","last_undegraded":"2024-03-28T00:25:21.596862+0000","last_fullsized":"2024-03-28T00:25:21.596862+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_clean_scrub_stamp":"2024-03-28T00:23:54.693629+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T04:15:12.060649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":2,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.534609+0000","last_change":"2024-03-28T00:23:56.747351+0000","last_active":"2024-03-28T00:24:05.534609+0000","last_peered":"2024-03-28T00:24:05.534609+0000","last_clean":"2024-03-28T00:24:05.534609+0000","last_became_active":"2024-03-28T00:23:56.746995+0000","last_became_peered":"2024-03-28T00:23:56.746995+0000","last_unstale":"2024-03-28T00:24:05.534609+0000","last_undegraded":"2024-03-28T00:24:05.534609+0000","last_fullsized":"2024-03-28T00:24:05.534609+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T00:43:53.521420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"85'7","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.539564+0000","last_change":"2024-03-28T00:23:55.730157+0000","last_active":"2024-03-28T00:24:05.539564+0000","last_peered":"2024-03-28T00:24:05.539564+0000","last_clean":"2024-03-28T00:24:05.539564+0000","last_became_active":"2024-03-28T00:23:55.729841+0000","last_became_peered":"2024-03-28T00:23:55.729841+0000","last_unstale":"2024-03-28T00:24:05.539564+0000","last_undegraded":"2024-03-28T00:24:05.539564+0000","last_fullsized":"2024-03-28T00:24:05.539564+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_clean_scrub_stamp":"2024-03-28T00:23:54.693629+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T09:57:31.865444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.538154+0000","last_change":"2024-03-28T00:23:56.760077+0000","last_active":"2024-03-28T00:24:05.538154+0000","last_peered":"2024-03-28T00:24:05.538154+0000","last_clean":"2024-03-28T00:24:05.538154+0000","last_became_active":"2024-03-28T00:23:56.759735+0000","last_became_peered":"2024-03-28T00:23:56.759735+0000","last_unstale":"2024-03-28T00:24:05.538154+0000","last_undegraded":"2024-03-28T00:24:05.538154+0000","last_fullsized":"2024-03-28T00:24:05.538154+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T11:36:17.224248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.537959+0000","last_change":"2024-03-28T00:23:55.738216+0000","last_active":"2024-03-28T00:24:05.537959+0000","last_peered":"2024-03-28T00:24:05.537959+0000","last_clean":"2024-03-28T00:24:05.537959+0000","last_became_active":"2024-03-28T00:23:55.737878+0000","last_became_peered":"2024-03-28T00:23:55.737878+0000","last_unstale":"2024-03-28T00:24:05.537959+0000","last_undegraded":"2024-03-28T00:24:05.537959+0000","last_fullsized":"2024-03-28T00:24:05.537959+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_clean_scrub_stamp":"2024-03-28T00:23:54.693629+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T06:30:00.465906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.973060+0000","last_change":"2024-03-28T00:23:56.760362+0000","last_active":"2024-03-28T00:24:05.973060+0000","last_peered":"2024-03-28T00:24:05.973060+0000","last_clean":"2024-03-28T00:24:05.973060+0000","last_became_active":"2024-03-28T00:23:56.759918+0000","last_became_peered":"2024-03-28T00:23:56.759918+0000","last_unstale":"2024-03-28T00:24:05.973060+0000","last_undegraded":"2024-03-28T00:24:05.973060+0000","last_fullsized":"2024-03-28T00:24:05.973060+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T04:59:01.038539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"85'3","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.537941+0000","last_change":"2024-03-28T00:23:55.727557+0000","last_active":"2024-03-28T00:24:05.537941+0000","last_peered":"2024-03-28T00:24:05.537941+0000","last_clean":"2024-03-28T00:24:05.537941+0000","last_became_active":"2024-03-28T00:23:55.727235+0000","last_became_peered":"2024-03-28T00:23:55.727235+0000","last_unstale":"2024-03-28T00:24:05.537941+0000","last_undegraded":"2024-03-28T00:24:05.537941+0000","last_fullsized":"2024-03-28T00:24:05.537941+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_clean_scrub_stamp":"2024-03-28T00:23:54.693629+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T08:14:57.268850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.546468+0000","last_change":"2024-03-28T00:23:56.766812+0000","last_active":"2024-03-28T00:24:05.546468+0000","last_peered":"2024-03-28T00:24:05.546468+0000","last_clean":"2024-03-28T00:24:05.546468+0000","last_became_active":"2024-03-28T00:23:56.766386+0000","last_became_peered":"2024-03-28T00:23:56.766386+0000","last_unstale":"2024-03-28T00:24:05.546468+0000","last_undegraded":"2024-03-28T00:24:05.546468+0000","last_fullsized":"2024-03-28T00:24:05.546468+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T05:58:07.349158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"85'4","reported_seq":181,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:25:22.551539+0000","last_change":"2024-03-28T00:23:55.727617+0000","last_active":"2024-03-28T00:25:22.551539+0000","last_peered":"2024-03-28T00:25:22.551539+0000","last_clean":"2024-03-28T00:25:22.551539+0000","last_became_active":"2024-03-28T00:23:55.727276+0000","last_became_peered":"2024-03-28T00:23:55.727276+0000","last_unstale":"2024-03-28T00:25:22.551539+0000","last_undegraded":"2024-03-28T00:25:22.551539+0000","last_fullsized":"2024-03-28T00:25:22.551539+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_clean_scrub_stamp":"2024-03-28T00:23:54.693629+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T02:27:04.376342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":158,"num_read_kb":158,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.582928+0000","last_change":"2024-03-28T00:23:56.747261+0000","last_active":"2024-03-28T00:24:05.582928+0000","last_peered":"2024-03-28T00:24:05.582928+0000","last_clean":"2024-03-28T00:24:05.582928+0000","last_became_active":"2024-03-28T00:23:56.746862+0000","last_became_peered":"2024-03-28T00:23:56.746862+0000","last_unstale":"2024-03-28T00:24:05.582928+0000","last_undegraded":"2024-03-28T00:24:05.582928+0000","last_fullsized":"2024-03-28T00:24:05.582928+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T07:33:35.747722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"85'4","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.535084+0000","last_change":"2024-03-28T00:23:55.725568+0000","last_active":"2024-03-28T00:24:05.535084+0000","last_peered":"2024-03-28T00:24:05.535084+0000","last_clean":"2024-03-28T00:24:05.535084+0000","last_became_active":"2024-03-28T00:23:55.724967+0000","last_became_peered":"2024-03-28T00:23:55.724967+0000","last_unstale":"2024-03-28T00:24:05.535084+0000","last_undegraded":"2024-03-28T00:24:05.535084+0000","last_fullsized":"2024-03-28T00:24:05.535084+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_clean_scrub_stamp":"2024-03-28T00:23:54.693629+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T10:40:00.731885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.553554+0000","last_change":"2024-03-28T00:23:56.763176+0000","last_active":"2024-03-28T00:24:05.553554+0000","last_peered":"2024-03-28T00:24:05.553554+0000","last_clean":"2024-03-28T00:24:05.553554+0000","last_became_active":"2024-03-28T00:23:56.762709+0000","last_became_peered":"2024-03-28T00:23:56.762709+0000","last_unstale":"2024-03-28T00:24:05.553554+0000","last_undegraded":"2024-03-28T00:24:05.553554+0000","last_fullsized":"2024-03-28T00:24:05.553554+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T05:58:52.003151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'5","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.538869+0000","last_change":"2024-03-28T00:23:55.745579+0000","last_active":"2024-03-28T00:24:05.538869+0000","last_peered":"2024-03-28T00:24:05.538869+0000","last_clean":"2024-03-28T00:24:05.538869+0000","last_became_active":"2024-03-28T00:23:55.745262+0000","last_became_peered":"2024-03-28T00:23:55.745262+0000","last_unstale":"2024-03-28T00:24:05.538869+0000","last_undegraded":"2024-03-28T00:24:05.538869+0000","last_fullsized":"2024-03-28T00:24:05.538869+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_clean_scrub_stamp":"2024-03-28T00:23:54.693629+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T00:48:36.212168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.582850+0000","last_change":"2024-03-28T00:23:56.736393+0000","last_active":"2024-03-28T00:24:05.582850+0000","last_peered":"2024-03-28T00:24:05.582850+0000","last_clean":"2024-03-28T00:24:05.582850+0000","last_became_active":"2024-03-28T00:23:56.736054+0000","last_became_peered":"2024-03-28T00:23:56.736054+0000","last_unstale":"2024-03-28T00:24:05.582850+0000","last_undegraded":"2024-03-28T00:24:05.582850+0000","last_fullsized":"2024-03-28T00:24:05.582850+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T11:56:00.039257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"87'32","reported_seq":216,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:25:22.551443+0000","last_change":"2024-03-28T00:23:55.722192+0000","last_active":"2024-03-28T00:25:22.551443+0000","last_peered":"2024-03-28T00:25:22.551443+0000","last_clean":"2024-03-28T00:25:22.551443+0000","last_became_active":"2024-03-28T00:23:55.721881+0000","last_became_peered":"2024-03-28T00:23:55.721881+0000","last_unstale":"2024-03-28T00:25:22.551443+0000","last_undegraded":"2024-03-28T00:25:22.551443+0000","last_fullsized":"2024-03-28T00:25:22.551443+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_clean_scrub_stamp":"2024-03-28T00:23:54.693629+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T03:47:34.950264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64402,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":165,"num_read_kb":144,"num_write":34,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.534978+0000","last_change":"2024-03-28T00:23:56.738361+0000","last_active":"2024-03-28T00:24:05.534978+0000","last_peered":"2024-03-28T00:24:05.534978+0000","last_clean":"2024-03-28T00:24:05.534978+0000","last_became_active":"2024-03-28T00:23:56.737040+0000","last_became_peered":"2024-03-28T00:23:56.737040+0000","last_unstale":"2024-03-28T00:24:05.534978+0000","last_undegraded":"2024-03-28T00:24:05.534978+0000","last_fullsized":"2024-03-28T00:24:05.534978+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T11:39:40.653421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'9","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.582640+0000","last_change":"2024-03-28T00:23:55.733943+0000","last_active":"2024-03-28T00:24:05.582640+0000","last_peered":"2024-03-28T00:24:05.582640+0000","last_clean":"2024-03-28T00:24:05.582640+0000","last_became_active":"2024-03-28T00:23:55.733236+0000","last_became_peered":"2024-03-28T00:23:55.733236+0000","last_unstale":"2024-03-28T00:24:05.582640+0000","last_undegraded":"2024-03-28T00:24:05.582640+0000","last_fullsized":"2024-03-28T00:24:05.582640+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_clean_scrub_stamp":"2024-03-28T00:23:54.693629+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T09:45:33.070408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.538244+0000","last_change":"2024-03-28T00:23:56.758751+0000","last_active":"2024-03-28T00:24:05.538244+0000","last_peered":"2024-03-28T00:24:05.538244+0000","last_clean":"2024-03-28T00:24:05.538244+0000","last_became_active":"2024-03-28T00:23:56.758066+0000","last_became_peered":"2024-03-28T00:23:56.758066+0000","last_unstale":"2024-03-28T00:24:05.538244+0000","last_undegraded":"2024-03-28T00:24:05.538244+0000","last_fullsized":"2024-03-28T00:24:05.538244+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T02:21:42.514212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"85'5","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.538122+0000","last_change":"2024-03-28T00:23:55.737167+0000","last_active":"2024-03-28T00:24:05.538122+0000","last_peered":"2024-03-28T00:24:05.538122+0000","last_clean":"2024-03-28T00:24:05.538122+0000","last_became_active":"2024-03-28T00:23:55.736925+0000","last_became_peered":"2024-03-28T00:23:55.736925+0000","last_unstale":"2024-03-28T00:24:05.538122+0000","last_undegraded":"2024-03-28T00:24:05.538122+0000","last_fullsized":"2024-03-28T00:24:05.538122+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_clean_scrub_stamp":"2024-03-28T00:23:54.693629+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T09:13:24.820375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.535057+0000","last_change":"2024-03-28T00:23:56.745939+0000","last_active":"2024-03-28T00:24:05.535057+0000","last_peered":"2024-03-28T00:24:05.535057+0000","last_clean":"2024-03-28T00:24:05.535057+0000","last_became_active":"2024-03-28T00:23:56.745504+0000","last_became_peered":"2024-03-28T00:23:56.745504+0000","last_unstale":"2024-03-28T00:24:05.535057+0000","last_undegraded":"2024-03-28T00:24:05.535057+0000","last_fullsized":"2024-03-28T00:24:05.535057+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T06:02:34.199029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"85'6","reported_seq":107,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:25:22.590921+0000","last_change":"2024-03-28T00:23:55.726916+0000","last_active":"2024-03-28T00:25:22.590921+0000","last_peered":"2024-03-28T00:25:22.590921+0000","last_clean":"2024-03-28T00:25:22.590921+0000","last_became_active":"2024-03-28T00:23:55.726570+0000","last_became_peered":"2024-03-28T00:23:55.726570+0000","last_unstale":"2024-03-28T00:25:22.590921+0000","last_undegraded":"2024-03-28T00:25:22.590921+0000","last_fullsized":"2024-03-28T00:25:22.590921+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_clean_scrub_stamp":"2024-03-28T00:23:54.693629+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T06:28:54.581129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":83,"num_read_kb":3,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.534720+0000","last_change":"2024-03-28T00:23:56.738303+0000","last_active":"2024-03-28T00:24:05.534720+0000","last_peered":"2024-03-28T00:24:05.534720+0000","last_clean":"2024-03-28T00:24:05.534720+0000","last_became_active":"2024-03-28T00:23:56.737926+0000","last_became_peered":"2024-03-28T00:23:56.737926+0000","last_unstale":"2024-03-28T00:24:05.534720+0000","last_undegraded":"2024-03-28T00:24:05.534720+0000","last_fullsized":"2024-03-28T00:24:05.534720+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T01:56:53.985793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'17","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.534473+0000","last_change":"2024-03-28T00:23:55.728708+0000","last_active":"2024-03-28T00:24:05.534473+0000","last_peered":"2024-03-28T00:24:05.534473+0000","last_clean":"2024-03-28T00:24:05.534473+0000","last_became_active":"2024-03-28T00:23:55.728415+0000","last_became_peered":"2024-03-28T00:23:55.728415+0000","last_unstale":"2024-03-28T00:24:05.534473+0000","last_undegraded":"2024-03-28T00:24:05.534473+0000","last_fullsized":"2024-03-28T00:24:05.534473+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_clean_scrub_stamp":"2024-03-28T00:23:54.693629+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T05:36:14.405953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.538144+0000","last_change":"2024-03-28T00:23:56.741982+0000","last_active":"2024-03-28T00:24:05.538144+0000","last_peered":"2024-03-28T00:24:05.538144+0000","last_clean":"2024-03-28T00:24:05.538144+0000","last_became_active":"2024-03-28T00:23:56.741642+0000","last_became_peered":"2024-03-28T00:23:56.741642+0000","last_unstale":"2024-03-28T00:24:05.538144+0000","last_undegraded":"2024-03-28T00:24:05.538144+0000","last_fullsized":"2024-03-28T00:24:05.538144+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T04:27:49.168164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.544593+0000","last_change":"2024-03-28T00:24:05.544144+0000","last_active":"2024-03-28T00:24:05.544593+0000","last_peered":"2024-03-28T00:24:05.544593+0000","last_clean":"2024-03-28T00:24:05.544593+0000","last_became_active":"2024-03-28T00:24:05.540061+0000","last_became_peered":"2024-03-28T00:24:05.540061+0000","last_unstale":"2024-03-28T00:24:05.544593+0000","last_undegraded":"2024-03-28T00:24:05.544593+0000","last_fullsized":"2024-03-28T00:24:05.544593+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_clean_scrub_stamp":"2024-03-28T00:23:54.693629+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T11:24:24.644313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.536666+0000","last_change":"2024-03-28T00:23:56.739241+0000","last_active":"2024-03-28T00:24:05.536666+0000","last_peered":"2024-03-28T00:24:05.536666+0000","last_clean":"2024-03-28T00:24:05.536666+0000","last_became_active":"2024-03-28T00:23:56.738122+0000","last_became_peered":"2024-03-28T00:23:56.738122+0000","last_unstale":"2024-03-28T00:24:05.536666+0000","last_undegraded":"2024-03-28T00:24:05.536666+0000","last_fullsized":"2024-03-28T00:24:05.536666+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T02:52:50.375314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"85'1","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.558013+0000","last_change":"2024-03-28T00:24:05.558013+0000","last_active":"2024-03-28T00:24:05.558013+0000","last_peered":"2024-03-28T00:24:05.558013+0000","last_clean":"2024-03-28T00:24:05.558013+0000","last_became_active":"2024-03-28T00:24:05.555882+0000","last_became_peered":"2024-03-28T00:24:05.555882+0000","last_unstale":"2024-03-28T00:24:05.558013+0000","last_undegraded":"2024-03-28T00:24:05.558013+0000","last_fullsized":"2024-03-28T00:24:05.558013+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_clean_scrub_stamp":"2024-03-28T00:23:54.693629+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T01:51:23.576990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"85'1","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.539849+0000","last_change":"2024-03-28T00:23:56.752780+0000","last_active":"2024-03-28T00:24:05.539849+0000","last_peered":"2024-03-28T00:24:05.539849+0000","last_clean":"2024-03-28T00:24:05.539849+0000","last_became_active":"2024-03-28T00:23:56.751608+0000","last_became_peered":"2024-03-28T00:23:56.751608+0000","last_unstale":"2024-03-28T00:24:05.539849+0000","last_undegraded":"2024-03-28T00:24:05.539849+0000","last_fullsized":"2024-03-28T00:24:05.539849+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T01:20:45.485846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"85'5","reported_seq":103,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:25:20.627972+0000","last_change":"2024-03-28T00:23:55.728123+0000","last_active":"2024-03-28T00:25:20.627972+0000","last_peered":"2024-03-28T00:25:20.627972+0000","last_clean":"2024-03-28T00:25:20.627972+0000","last_became_active":"2024-03-28T00:23:55.727839+0000","last_became_peered":"2024-03-28T00:23:55.727839+0000","last_unstale":"2024-03-28T00:25:20.627972+0000","last_undegraded":"2024-03-28T00:25:20.627972+0000","last_fullsized":"2024-03-28T00:25:20.627972+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_clean_scrub_stamp":"2024-03-28T00:23:54.693629+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T11:39:30.315824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":79,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.555736+0000","last_change":"2024-03-28T00:24:05.555736+0000","last_active":"2024-03-28T00:24:05.555736+0000","last_peered":"2024-03-28T00:24:05.555736+0000","last_clean":"2024-03-28T00:24:05.555736+0000","last_became_active":"2024-03-28T00:24:05.555429+0000","last_became_peered":"2024-03-28T00:24:05.555429+0000","last_unstale":"2024-03-28T00:24:05.555736+0000","last_undegraded":"2024-03-28T00:24:05.555736+0000","last_fullsized":"2024-03-28T00:24:05.555736+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T03:57:58.828548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.583672+0000","last_change":"2024-03-28T00:24:05.583672+0000","last_active":"2024-03-28T00:24:05.583672+0000","last_peered":"2024-03-28T00:24:05.583672+0000","last_clean":"2024-03-28T00:24:05.583672+0000","last_became_active":"2024-03-28T00:24:05.583301+0000","last_became_peered":"2024-03-28T00:24:05.583301+0000","last_unstale":"2024-03-28T00:24:05.583672+0000","last_undegraded":"2024-03-28T00:24:05.583672+0000","last_fullsized":"2024-03-28T00:24:05.583672+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T08:20:17.464773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,8],"acting":[5,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"85'5","reported_seq":104,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:25:21.588698+0000","last_change":"2024-03-28T00:23:55.747124+0000","last_active":"2024-03-28T00:25:21.588698+0000","last_peered":"2024-03-28T00:25:21.588698+0000","last_clean":"2024-03-28T00:25:21.588698+0000","last_became_active":"2024-03-28T00:23:55.745721+0000","last_became_peered":"2024-03-28T00:23:55.745721+0000","last_unstale":"2024-03-28T00:25:21.588698+0000","last_undegraded":"2024-03-28T00:25:21.588698+0000","last_fullsized":"2024-03-28T00:25:21.588698+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_clean_scrub_stamp":"2024-03-28T00:23:54.693629+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T10:37:15.898923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":80,"num_read_kb":80,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.973656+0000","last_change":"2024-03-28T00:23:56.757060+0000","last_active":"2024-03-28T00:24:05.973656+0000","last_peered":"2024-03-28T00:24:05.973656+0000","last_clean":"2024-03-28T00:24:05.973656+0000","last_became_active":"2024-03-28T00:23:56.756754+0000","last_became_peered":"2024-03-28T00:23:56.756754+0000","last_unstale":"2024-03-28T00:24:05.973656+0000","last_undegraded":"2024-03-28T00:24:05.973656+0000","last_fullsized":"2024-03-28T00:24:05.973656+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T11:59:32.877471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'6","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.537983+0000","last_change":"2024-03-28T00:23:55.728039+0000","last_active":"2024-03-28T00:24:05.537983+0000","last_peered":"2024-03-28T00:24:05.537983+0000","last_clean":"2024-03-28T00:24:05.537983+0000","last_became_active":"2024-03-28T00:23:55.727705+0000","last_became_peered":"2024-03-28T00:23:55.727705+0000","last_unstale":"2024-03-28T00:24:05.537983+0000","last_undegraded":"2024-03-28T00:24:05.537983+0000","last_fullsized":"2024-03-28T00:24:05.537983+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_clean_scrub_stamp":"2024-03-28T00:23:54.693629+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T03:48:39.028754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":355,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.534852+0000","last_change":"2024-03-28T00:22:35.416127+0000","last_active":"2024-03-28T00:24:05.534852+0000","last_peered":"2024-03-28T00:24:05.534852+0000","last_clean":"2024-03-28T00:24:05.534852+0000","last_became_active":"2024-03-28T00:22:35.415710+0000","last_became_peered":"2024-03-28T00:22:35.415710+0000","last_unstale":"2024-03-28T00:24:05.534852+0000","last_undegraded":"2024-03-28T00:24:05.534852+0000","last_fullsized":"2024-03-28T00:24:05.534852+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:18:46.602205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:18:46.602205+0000","last_clean_scrub_stamp":"2024-03-28T00:18:46.602205+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T11:31:44.851848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.537824+0000","last_change":"2024-03-28T00:23:56.741550+0000","last_active":"2024-03-28T00:24:05.537824+0000","last_peered":"2024-03-28T00:24:05.537824+0000","last_clean":"2024-03-28T00:24:05.537824+0000","last_became_active":"2024-03-28T00:23:56.740750+0000","last_became_peered":"2024-03-28T00:23:56.740750+0000","last_unstale":"2024-03-28T00:24:05.537824+0000","last_undegraded":"2024-03-28T00:24:05.537824+0000","last_fullsized":"2024-03-28T00:24:05.537824+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T09:14:35.259597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'4","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.534882+0000","last_change":"2024-03-28T00:23:55.731623+0000","last_active":"2024-03-28T00:24:05.534882+0000","last_peered":"2024-03-28T00:24:05.534882+0000","last_clean":"2024-03-28T00:24:05.534882+0000","last_became_active":"2024-03-28T00:23:55.731292+0000","last_became_peered":"2024-03-28T00:23:55.731292+0000","last_unstale":"2024-03-28T00:24:05.534882+0000","last_undegraded":"2024-03-28T00:24:05.534882+0000","last_fullsized":"2024-03-28T00:24:05.534882+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_clean_scrub_stamp":"2024-03-28T00:23:54.693629+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T06:16:23.001339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"85'4","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.535069+0000","last_change":"2024-03-28T00:23:55.745553+0000","last_active":"2024-03-28T00:24:05.535069+0000","last_peered":"2024-03-28T00:24:05.535069+0000","last_clean":"2024-03-28T00:24:05.535069+0000","last_became_active":"2024-03-28T00:23:55.745204+0000","last_became_peered":"2024-03-28T00:23:55.745204+0000","last_unstale":"2024-03-28T00:24:05.535069+0000","last_undegraded":"2024-03-28T00:24:05.535069+0000","last_fullsized":"2024-03-28T00:24:05.535069+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_clean_scrub_stamp":"2024-03-28T00:23:54.693629+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T03:23:50.584362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.538733+0000","last_change":"2024-03-28T00:23:56.751394+0000","last_active":"2024-03-28T00:24:05.538733+0000","last_peered":"2024-03-28T00:24:05.538733+0000","last_clean":"2024-03-28T00:24:05.538733+0000","last_became_active":"2024-03-28T00:23:56.751025+0000","last_became_peered":"2024-03-28T00:23:56.751025+0000","last_unstale":"2024-03-28T00:24:05.538733+0000","last_undegraded":"2024-03-28T00:24:05.538733+0000","last_fullsized":"2024-03-28T00:24:05.538733+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T03:59:28.738018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"87'12","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:10.080892+0000","last_change":"2024-03-28T00:23:55.735055+0000","last_active":"2024-03-28T00:24:10.080892+0000","last_peered":"2024-03-28T00:24:10.080892+0000","last_clean":"2024-03-28T00:24:10.080892+0000","last_became_active":"2024-03-28T00:23:55.734747+0000","last_became_peered":"2024-03-28T00:23:55.734747+0000","last_unstale":"2024-03-28T00:24:10.080892+0000","last_undegraded":"2024-03-28T00:24:10.080892+0000","last_fullsized":"2024-03-28T00:24:10.080892+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_clean_scrub_stamp":"2024-03-28T00:23:54.693629+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T03:02:13.533883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.974285+0000","last_change":"2024-03-28T00:23:56.760581+0000","last_active":"2024-03-28T00:24:05.974285+0000","last_peered":"2024-03-28T00:24:05.974285+0000","last_clean":"2024-03-28T00:24:05.974285+0000","last_became_active":"2024-03-28T00:23:56.756807+0000","last_became_peered":"2024-03-28T00:23:56.756807+0000","last_unstale":"2024-03-28T00:24:05.974285+0000","last_undegraded":"2024-03-28T00:24:05.974285+0000","last_fullsized":"2024-03-28T00:24:05.974285+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T07:13:34.097671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'7","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.538170+0000","last_change":"2024-03-28T00:23:55.727864+0000","last_active":"2024-03-28T00:24:05.538170+0000","last_peered":"2024-03-28T00:24:05.538170+0000","last_clean":"2024-03-28T00:24:05.538170+0000","last_became_active":"2024-03-28T00:23:55.727507+0000","last_became_peered":"2024-03-28T00:23:55.727507+0000","last_unstale":"2024-03-28T00:24:05.538170+0000","last_undegraded":"2024-03-28T00:24:05.538170+0000","last_fullsized":"2024-03-28T00:24:05.538170+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_clean_scrub_stamp":"2024-03-28T00:23:54.693629+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T09:01:45.401356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.554158+0000","last_change":"2024-03-28T00:23:56.766973+0000","last_active":"2024-03-28T00:24:05.554158+0000","last_peered":"2024-03-28T00:24:05.554158+0000","last_clean":"2024-03-28T00:24:05.554158+0000","last_became_active":"2024-03-28T00:23:56.766549+0000","last_became_peered":"2024-03-28T00:23:56.766549+0000","last_unstale":"2024-03-28T00:24:05.554158+0000","last_undegraded":"2024-03-28T00:24:05.554158+0000","last_fullsized":"2024-03-28T00:24:05.554158+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T02:37:52.330879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.582622+0000","last_change":"2024-03-28T00:23:55.728936+0000","last_active":"2024-03-28T00:24:05.582622+0000","last_peered":"2024-03-28T00:24:05.582622+0000","last_clean":"2024-03-28T00:24:05.582622+0000","last_became_active":"2024-03-28T00:23:55.728517+0000","last_became_peered":"2024-03-28T00:23:55.728517+0000","last_unstale":"2024-03-28T00:24:05.582622+0000","last_undegraded":"2024-03-28T00:24:05.582622+0000","last_fullsized":"2024-03-28T00:24:05.582622+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_clean_scrub_stamp":"2024-03-28T00:23:54.693629+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T10:13:20.841286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.534504+0000","last_change":"2024-03-28T00:23:56.738438+0000","last_active":"2024-03-28T00:24:05.534504+0000","last_peered":"2024-03-28T00:24:05.534504+0000","last_clean":"2024-03-28T00:24:05.534504+0000","last_became_active":"2024-03-28T00:23:56.738092+0000","last_became_peered":"2024-03-28T00:23:56.738092+0000","last_unstale":"2024-03-28T00:24:05.534504+0000","last_undegraded":"2024-03-28T00:24:05.534504+0000","last_fullsized":"2024-03-28T00:24:05.534504+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T02:54:57.424857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"85'7","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.973138+0000","last_change":"2024-03-28T00:23:55.741042+0000","last_active":"2024-03-28T00:24:05.973138+0000","last_peered":"2024-03-28T00:24:05.973138+0000","last_clean":"2024-03-28T00:24:05.973138+0000","last_became_active":"2024-03-28T00:23:55.740165+0000","last_became_peered":"2024-03-28T00:23:55.740165+0000","last_unstale":"2024-03-28T00:24:05.973138+0000","last_undegraded":"2024-03-28T00:24:05.973138+0000","last_fullsized":"2024-03-28T00:24:05.973138+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_clean_scrub_stamp":"2024-03-28T00:23:54.693629+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T04:47:58.519326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.536221+0000","last_change":"2024-03-28T00:23:56.760261+0000","last_active":"2024-03-28T00:24:05.536221+0000","last_peered":"2024-03-28T00:24:05.536221+0000","last_clean":"2024-03-28T00:24:05.536221+0000","last_became_active":"2024-03-28T00:23:56.759925+0000","last_became_peered":"2024-03-28T00:23:56.759925+0000","last_unstale":"2024-03-28T00:24:05.536221+0000","last_undegraded":"2024-03-28T00:24:05.536221+0000","last_fullsized":"2024-03-28T00:24:05.536221+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T00:58:03.546582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.582758+0000","last_change":"2024-03-28T00:23:55.733455+0000","last_active":"2024-03-28T00:24:05.582758+0000","last_peered":"2024-03-28T00:24:05.582758+0000","last_clean":"2024-03-28T00:24:05.582758+0000","last_became_active":"2024-03-28T00:23:55.733116+0000","last_became_peered":"2024-03-28T00:23:55.733116+0000","last_unstale":"2024-03-28T00:24:05.582758+0000","last_undegraded":"2024-03-28T00:24:05.582758+0000","last_fullsized":"2024-03-28T00:24:05.582758+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_clean_scrub_stamp":"2024-03-28T00:23:54.693629+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T07:43:41.301213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.973417+0000","last_change":"2024-03-28T00:23:56.766884+0000","last_active":"2024-03-28T00:24:05.973417+0000","last_peered":"2024-03-28T00:24:05.973417+0000","last_clean":"2024-03-28T00:24:05.973417+0000","last_became_active":"2024-03-28T00:23:56.766476+0000","last_became_peered":"2024-03-28T00:23:56.766476+0000","last_unstale":"2024-03-28T00:24:05.973417+0000","last_undegraded":"2024-03-28T00:24:05.973417+0000","last_fullsized":"2024-03-28T00:24:05.973417+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T08:01:13.280796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.534658+0000","last_change":"2024-03-28T00:23:55.728001+0000","last_active":"2024-03-28T00:24:05.534658+0000","last_peered":"2024-03-28T00:24:05.534658+0000","last_clean":"2024-03-28T00:24:05.534658+0000","last_became_active":"2024-03-28T00:23:55.727659+0000","last_became_peered":"2024-03-28T00:23:55.727659+0000","last_unstale":"2024-03-28T00:24:05.534658+0000","last_undegraded":"2024-03-28T00:24:05.534658+0000","last_fullsized":"2024-03-28T00:24:05.534658+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_clean_scrub_stamp":"2024-03-28T00:23:54.693629+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T05:59:09.165928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.535045+0000","last_change":"2024-03-28T00:23:56.740843+0000","last_active":"2024-03-28T00:24:05.535045+0000","last_peered":"2024-03-28T00:24:05.535045+0000","last_clean":"2024-03-28T00:24:05.535045+0000","last_became_active":"2024-03-28T00:23:56.740364+0000","last_became_peered":"2024-03-28T00:23:56.740364+0000","last_unstale":"2024-03-28T00:24:05.535045+0000","last_undegraded":"2024-03-28T00:24:05.535045+0000","last_fullsized":"2024-03-28T00:24:05.535045+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T10:44:10.040424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'6","reported_seq":104,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:25:19.549025+0000","last_change":"2024-03-28T00:23:55.745772+0000","last_active":"2024-03-28T00:25:19.549025+0000","last_peered":"2024-03-28T00:25:19.549025+0000","last_clean":"2024-03-28T00:25:19.549025+0000","last_became_active":"2024-03-28T00:23:55.745457+0000","last_became_peered":"2024-03-28T00:23:55.745457+0000","last_unstale":"2024-03-28T00:25:19.549025+0000","last_undegraded":"2024-03-28T00:25:19.549025+0000","last_fullsized":"2024-03-28T00:25:19.549025+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_clean_scrub_stamp":"2024-03-28T00:23:54.693629+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T01:27:44.632523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1561,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":79,"num_read_kb":2,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.535251+0000","last_change":"2024-03-28T00:23:56.737008+0000","last_active":"2024-03-28T00:24:05.535251+0000","last_peered":"2024-03-28T00:24:05.535251+0000","last_clean":"2024-03-28T00:24:05.535251+0000","last_became_active":"2024-03-28T00:23:56.736399+0000","last_became_peered":"2024-03-28T00:23:56.736399+0000","last_unstale":"2024-03-28T00:24:05.535251+0000","last_undegraded":"2024-03-28T00:24:05.535251+0000","last_fullsized":"2024-03-28T00:24:05.535251+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T07:16:05.968362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'3","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.538765+0000","last_change":"2024-03-28T00:23:55.734518+0000","last_active":"2024-03-28T00:24:05.538765+0000","last_peered":"2024-03-28T00:24:05.538765+0000","last_clean":"2024-03-28T00:24:05.538765+0000","last_became_active":"2024-03-28T00:23:55.734200+0000","last_became_peered":"2024-03-28T00:23:55.734200+0000","last_unstale":"2024-03-28T00:24:05.538765+0000","last_undegraded":"2024-03-28T00:24:05.538765+0000","last_fullsized":"2024-03-28T00:24:05.538765+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_clean_scrub_stamp":"2024-03-28T00:23:54.693629+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T05:00:56.602654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"86'1","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.535023+0000","last_change":"2024-03-28T00:23:56.757249+0000","last_active":"2024-03-28T00:24:05.535023+0000","last_peered":"2024-03-28T00:24:05.535023+0000","last_clean":"2024-03-28T00:24:05.535023+0000","last_became_active":"2024-03-28T00:23:56.756957+0000","last_became_peered":"2024-03-28T00:23:56.756957+0000","last_unstale":"2024-03-28T00:24:05.535023+0000","last_undegraded":"2024-03-28T00:24:05.535023+0000","last_fullsized":"2024-03-28T00:24:05.535023+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T03:55:30.166369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'5","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.538430+0000","last_change":"2024-03-28T00:23:55.741463+0000","last_active":"2024-03-28T00:24:05.538430+0000","last_peered":"2024-03-28T00:24:05.538430+0000","last_clean":"2024-03-28T00:24:05.538430+0000","last_became_active":"2024-03-28T00:23:55.741095+0000","last_became_peered":"2024-03-28T00:23:55.741095+0000","last_unstale":"2024-03-28T00:24:05.538430+0000","last_undegraded":"2024-03-28T00:24:05.538430+0000","last_fullsized":"2024-03-28T00:24:05.538430+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_clean_scrub_stamp":"2024-03-28T00:23:54.693629+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T02:58:49.587896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.535208+0000","last_change":"2024-03-28T00:23:56.736382+0000","last_active":"2024-03-28T00:24:05.535208+0000","last_peered":"2024-03-28T00:24:05.535208+0000","last_clean":"2024-03-28T00:24:05.535208+0000","last_became_active":"2024-03-28T00:23:56.736036+0000","last_became_peered":"2024-03-28T00:23:56.736036+0000","last_unstale":"2024-03-28T00:24:05.535208+0000","last_undegraded":"2024-03-28T00:24:05.535208+0000","last_fullsized":"2024-03-28T00:24:05.535208+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T04:26:01.792967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.583909+0000","last_change":"2024-03-28T00:24:05.583909+0000","last_active":"2024-03-28T00:24:05.583909+0000","last_peered":"2024-03-28T00:24:05.583909+0000","last_clean":"2024-03-28T00:24:05.583909+0000","last_became_active":"2024-03-28T00:24:05.583502+0000","last_became_peered":"2024-03-28T00:24:05.583502+0000","last_unstale":"2024-03-28T00:24:05.583909+0000","last_undegraded":"2024-03-28T00:24:05.583909+0000","last_fullsized":"2024-03-28T00:24:05.583909+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T12:15:24.378306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,9],"acting":[11,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.582428+0000","last_change":"2024-03-28T00:23:56.736044+0000","last_active":"2024-03-28T00:24:05.582428+0000","last_peered":"2024-03-28T00:24:05.582428+0000","last_clean":"2024-03-28T00:24:05.582428+0000","last_became_active":"2024-03-28T00:23:56.735688+0000","last_became_peered":"2024-03-28T00:23:56.735688+0000","last_unstale":"2024-03-28T00:24:05.582428+0000","last_undegraded":"2024-03-28T00:24:05.582428+0000","last_fullsized":"2024-03-28T00:24:05.582428+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T08:57:16.687620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.582399+0000","last_change":"2024-03-28T00:23:56.760935+0000","last_active":"2024-03-28T00:24:05.582399+0000","last_peered":"2024-03-28T00:24:05.582399+0000","last_clean":"2024-03-28T00:24:05.582399+0000","last_became_active":"2024-03-28T00:23:56.760600+0000","last_became_peered":"2024-03-28T00:23:56.760600+0000","last_unstale":"2024-03-28T00:24:05.582399+0000","last_undegraded":"2024-03-28T00:24:05.582399+0000","last_fullsized":"2024-03-28T00:24:05.582399+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T03:44:30.047881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.553794+0000","last_change":"2024-03-28T00:23:56.760767+0000","last_active":"2024-03-28T00:24:05.553794+0000","last_peered":"2024-03-28T00:24:05.553794+0000","last_clean":"2024-03-28T00:24:05.553794+0000","last_became_active":"2024-03-28T00:23:56.760431+0000","last_became_peered":"2024-03-28T00:23:56.760431+0000","last_unstale":"2024-03-28T00:24:05.553794+0000","last_undegraded":"2024-03-28T00:24:05.553794+0000","last_fullsized":"2024-03-28T00:24:05.553794+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T01:53:43.479645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.560249+0000","last_change":"2024-03-28T00:24:05.560249+0000","last_active":"2024-03-28T00:24:05.560249+0000","last_peered":"2024-03-28T00:24:05.560249+0000","last_clean":"2024-03-28T00:24:05.560249+0000","last_became_active":"2024-03-28T00:24:05.558901+0000","last_became_peered":"2024-03-28T00:24:05.558901+0000","last_unstale":"2024-03-28T00:24:05.560249+0000","last_undegraded":"2024-03-28T00:24:05.560249+0000","last_fullsized":"2024-03-28T00:24:05.560249+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T05:31:16.782355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.536385+0000","last_change":"2024-03-28T00:23:56.740004+0000","last_active":"2024-03-28T00:24:05.536385+0000","last_peered":"2024-03-28T00:24:05.536385+0000","last_clean":"2024-03-28T00:24:05.536385+0000","last_became_active":"2024-03-28T00:23:56.739630+0000","last_became_peered":"2024-03-28T00:23:56.739630+0000","last_unstale":"2024-03-28T00:24:05.536385+0000","last_undegraded":"2024-03-28T00:24:05.536385+0000","last_fullsized":"2024-03-28T00:24:05.536385+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T01:04:39.297383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.583124+0000","last_change":"2024-03-28T00:23:56.740716+0000","last_active":"2024-03-28T00:24:05.583124+0000","last_peered":"2024-03-28T00:24:05.583124+0000","last_clean":"2024-03-28T00:24:05.583124+0000","last_became_active":"2024-03-28T00:23:56.739587+0000","last_became_peered":"2024-03-28T00:23:56.739587+0000","last_unstale":"2024-03-28T00:24:05.583124+0000","last_undegraded":"2024-03-28T00:24:05.583124+0000","last_fullsized":"2024-03-28T00:24:05.583124+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T11:17:37.115216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.974165+0000","last_change":"2024-03-28T00:23:56.761001+0000","last_active":"2024-03-28T00:24:05.974165+0000","last_peered":"2024-03-28T00:24:05.974165+0000","last_clean":"2024-03-28T00:24:05.974165+0000","last_became_active":"2024-03-28T00:23:56.757373+0000","last_became_peered":"2024-03-28T00:23:56.757373+0000","last_unstale":"2024-03-28T00:24:05.974165+0000","last_undegraded":"2024-03-28T00:24:05.974165+0000","last_fullsized":"2024-03-28T00:24:05.974165+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T02:55:47.106100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.538064+0000","last_change":"2024-03-28T00:23:56.750715+0000","last_active":"2024-03-28T00:24:05.538064+0000","last_peered":"2024-03-28T00:24:05.538064+0000","last_clean":"2024-03-28T00:24:05.538064+0000","last_became_active":"2024-03-28T00:23:56.750446+0000","last_became_peered":"2024-03-28T00:23:56.750446+0000","last_unstale":"2024-03-28T00:24:05.538064+0000","last_undegraded":"2024-03-28T00:24:05.538064+0000","last_fullsized":"2024-03-28T00:24:05.538064+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T11:51:27.935181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.973070+0000","last_change":"2024-03-28T00:23:56.752571+0000","last_active":"2024-03-28T00:24:05.973070+0000","last_peered":"2024-03-28T00:24:05.973070+0000","last_clean":"2024-03-28T00:24:05.973070+0000","last_became_active":"2024-03-28T00:23:56.751853+0000","last_became_peered":"2024-03-28T00:23:56.751853+0000","last_unstale":"2024-03-28T00:24:05.973070+0000","last_undegraded":"2024-03-28T00:24:05.973070+0000","last_fullsized":"2024-03-28T00:24:05.973070+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T05:30:02.897139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.582681+0000","last_change":"2024-03-28T00:23:56.750006+0000","last_active":"2024-03-28T00:24:05.582681+0000","last_peered":"2024-03-28T00:24:05.582681+0000","last_clean":"2024-03-28T00:24:05.582681+0000","last_became_active":"2024-03-28T00:23:56.749639+0000","last_became_peered":"2024-03-28T00:23:56.749639+0000","last_unstale":"2024-03-28T00:24:05.582681+0000","last_undegraded":"2024-03-28T00:24:05.582681+0000","last_fullsized":"2024-03-28T00:24:05.582681+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T02:26:51.076107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.538700+0000","last_change":"2024-03-28T00:23:56.764982+0000","last_active":"2024-03-28T00:24:05.538700+0000","last_peered":"2024-03-28T00:24:05.538700+0000","last_clean":"2024-03-28T00:24:05.538700+0000","last_became_active":"2024-03-28T00:23:56.764596+0000","last_became_peered":"2024-03-28T00:23:56.764596+0000","last_unstale":"2024-03-28T00:24:05.538700+0000","last_undegraded":"2024-03-28T00:24:05.538700+0000","last_fullsized":"2024-03-28T00:24:05.538700+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T09:05:30.948689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.536382+0000","last_change":"2024-03-28T00:23:56.736439+0000","last_active":"2024-03-28T00:24:05.536382+0000","last_peered":"2024-03-28T00:24:05.536382+0000","last_clean":"2024-03-28T00:24:05.536382+0000","last_became_active":"2024-03-28T00:23:56.736099+0000","last_became_peered":"2024-03-28T00:23:56.736099+0000","last_unstale":"2024-03-28T00:24:05.536382+0000","last_undegraded":"2024-03-28T00:24:05.536382+0000","last_fullsized":"2024-03-28T00:24:05.536382+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T02:57:42.821566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":4132576,"data_compressed":245983,"data_compressed_allocated":315392,"data_compressed_original":4079616,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":71997,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":823,"num_read_kb":488,"num_write":240,"num_write_kb":137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":215991,"data_compressed":5124,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":209,"ondisk_log_size":209,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383713,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32116,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713036,"statfs":{"total":95995035648,"available":95962148864,"internally_reserved":0,"allocated":954368,"data_stored":456993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645349,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28096,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717056,"statfs":{"total":95995035648,"available":95966265344,"internally_reserved":0,"allocated":1105920,"data_stored":1112402,"data_compressed":37553,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808874,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32128,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713024,"statfs":{"total":95995035648,"available":95962136576,"internally_reserved":0,"allocated":974848,"data_stored":456510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070514,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32120,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713032,"statfs":{"total":95995035648,"available":95962144768,"internally_reserved":0,"allocated":966656,"data_stored":456866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332149,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1024000,"data_stored":1049797,"data_compressed":35845,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495676,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32120,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713032,"statfs":{"total":95995035648,"available":95962144768,"internally_reserved":0,"allocated":958464,"data_stored":456227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574936,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32172,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712980,"statfs":{"total":95995035648,"available":95962091520,"internally_reserved":0,"allocated":1019904,"data_stored":521542,"data_compressed":1708,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738461,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":978944,"data_stored":457957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411411,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36352,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708800,"statfs":{"total":95995035648,"available":95957811200,"internally_reserved":0,"allocated":1040384,"data_stored":522544,"data_compressed":1708,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182475,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":974848,"data_stored":456052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920840,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32120,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713032,"statfs":{"total":95995035648,"available":95962144768,"internally_reserved":0,"allocated":958464,"data_stored":456479,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757313,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1036288,"data_stored":1049084,"data_compressed":35845,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34201,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34201,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34201,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35845,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35845,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35845,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35845,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":66154,"data_compressed":1708,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":67156,"data_compressed":1708,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":66529,"data_compressed":1708,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-28T00:25:27.082 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 10f73b92-ec98-11ee-95d0-87774f69a715 -- ceph pg dump --format=json 2024-03-28T00:25:27.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:25:27 smithi165 ceph-mon[36302]: pgmap v358: 97 pgs: 97 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T00:25:27.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:25:27 smithi119 ceph-mon[31518]: pgmap v358: 97 pgs: 97 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T00:25:27.452 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:25:27 smithi132 ceph-mon[35941]: pgmap v358: 97 pgs: 97 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T00:25:27.565 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/mon.a/config 2024-03-28T00:25:28.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:25:28 smithi165 ceph-mon[36302]: from='client.24878 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-28T00:25:28.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:25:28 smithi119 ceph-mon[31518]: from='client.24878 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-28T00:25:28.452 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:25:28 smithi132 ceph-mon[35941]: from='client.24878 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-28T00:25:29.354 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:25:29 smithi132 ceph-mon[35941]: pgmap v359: 97 pgs: 97 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-28T00:25:29.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:25:29 smithi165 ceph-mon[36302]: pgmap v359: 97 pgs: 97 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-28T00:25:29.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:25:29 smithi119 ceph-mon[31518]: pgmap v359: 97 pgs: 97 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-28T00:25:29.725 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:29.718+0000 7fb9e6a00700 1 -- 172.21.15.119:0/3041157353 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb9e0102bb0 msgr2=0x7fb9e010f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:29.726 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:29.718+0000 7fb9e6a00700 1 --2- 172.21.15.119:0/3041157353 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb9e0102bb0 0x7fb9e010f060 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fb9d800b0a0 tx=0x7fb9d800b3b0 comp rx=0 tx=0).stop 2024-03-28T00:25:29.726 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:29.718+0000 7fb9e6a00700 1 -- 172.21.15.119:0/3041157353 shutdown_connections 2024-03-28T00:25:29.726 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:29.718+0000 7fb9e6a00700 1 --2- 172.21.15.119:0/3041157353 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb9e0102bb0 0x7fb9e010f060 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:29.726 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:29.718+0000 7fb9e6a00700 1 --2- 172.21.15.119:0/3041157353 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb9e0108220 0x7fb9e0108600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:29.726 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:29.718+0000 7fb9e6a00700 1 --2- 172.21.15.119:0/3041157353 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb9e0102210 0x7fb9e0102670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:29.726 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:29.718+0000 7fb9e6a00700 1 -- 172.21.15.119:0/3041157353 >> 172.21.15.119:0/3041157353 conn(0x7fb9e0079260 msgr2=0x7fb9e0079670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:25:29.726 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:29.718+0000 7fb9e6a00700 1 -- 172.21.15.119:0/3041157353 shutdown_connections 2024-03-28T00:25:29.726 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:29.719+0000 7fb9e6a00700 1 -- 172.21.15.119:0/3041157353 wait complete. 2024-03-28T00:25:29.726 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:29.719+0000 7fb9e6a00700 1 Processor -- start 2024-03-28T00:25:29.728 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:29.720+0000 7fb9e6a00700 1 -- start start 2024-03-28T00:25:29.728 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:29.720+0000 7fb9e6a00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb9e0102210 0x7fb9e0074490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:29.728 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:29.720+0000 7fb9e6a00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb9e0102bb0 0x7fb9e00749d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:29.728 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:29.720+0000 7fb9e6a00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb9e0108220 0x7fb9e0074f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:29.728 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:29.720+0000 7fb9e6a00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9e01124c0 con 0x7fb9e0102bb0 2024-03-28T00:25:29.728 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:29.720+0000 7fb9e6a00700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9e0112340 con 0x7fb9e0108220 2024-03-28T00:25:29.728 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:29.720+0000 7fb9e6a00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9e0112640 con 0x7fb9e0102210 2024-03-28T00:25:29.728 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:29.720+0000 7fb9dfe00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb9e0102210 0x7fb9e0074490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:29.728 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:29.720+0000 7fb9dfe00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb9e0102210 0x7fb9e0074490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.119:51394/0 (socket says 172.21.15.119:51394) 2024-03-28T00:25:29.728 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:29.720+0000 7fb9dfe00700 1 -- 172.21.15.119:0/2679417190 learned_addr learned my addr 172.21.15.119:0/2679417190 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:25:29.728 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:29.720+0000 7fb9e4a00700 1 --2- 172.21.15.119:0/2679417190 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb9e0108220 0x7fb9e0074f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:29.728 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:29.720+0000 7fb9d7400700 1 --2- 172.21.15.119:0/2679417190 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb9e0102bb0 0x7fb9e00749d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:29.729 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:29.721+0000 7fb9dfe00700 1 -- 172.21.15.119:0/2679417190 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb9e0108220 msgr2=0x7fb9e0074f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:29.729 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:29.721+0000 7fb9dfe00700 1 --2- 172.21.15.119:0/2679417190 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb9e0108220 0x7fb9e0074f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:29.729 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:29.721+0000 7fb9dfe00700 1 -- 172.21.15.119:0/2679417190 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb9e0102bb0 msgr2=0x7fb9e00749d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:29.729 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:29.721+0000 7fb9dfe00700 1 --2- 172.21.15.119:0/2679417190 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb9e0102bb0 0x7fb9e00749d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:29.729 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:29.721+0000 7fb9dfe00700 1 -- 172.21.15.119:0/2679417190 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9e01a6ea0 con 0x7fb9e0102210 2024-03-28T00:25:29.729 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:29.721+0000 7fb9e4a00700 1 --2- 172.21.15.119:0/2679417190 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb9e0108220 0x7fb9e0074f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T00:25:29.729 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:29.721+0000 7fb9d7400700 1 --2- 172.21.15.119:0/2679417190 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb9e0102bb0 0x7fb9e00749d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T00:25:29.729 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:29.721+0000 7fb9dfe00700 1 --2- 172.21.15.119:0/2679417190 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb9e0102210 0x7fb9e0074490 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fb9d000e9a0 tx=0x7fb9d000ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:25:29.729 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:29.721+0000 7fb9dd600700 1 -- 172.21.15.119:0/2679417190 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9d000ca10 con 0x7fb9e0102210 2024-03-28T00:25:29.729 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:29.721+0000 7fb9dd600700 1 -- 172.21.15.119:0/2679417190 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb9d000cbb0 con 0x7fb9e0102210 2024-03-28T00:25:29.729 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:29.721+0000 7fb9e6a00700 1 -- 172.21.15.119:0/2679417190 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9e01a7190 con 0x7fb9e0102210 2024-03-28T00:25:29.729 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:29.721+0000 7fb9dd600700 1 -- 172.21.15.119:0/2679417190 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9d0018680 con 0x7fb9e0102210 2024-03-28T00:25:29.729 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:29.721+0000 7fb9e6a00700 1 -- 172.21.15.119:0/2679417190 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9e01a7720 con 0x7fb9e0102210 2024-03-28T00:25:29.730 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:29.723+0000 7fb9e6a00700 1 -- 172.21.15.119:0/2679417190 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9e0068e50 con 0x7fb9e0102210 2024-03-28T00:25:29.730 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:29.724+0000 7fb9dd600700 1 -- 172.21.15.119:0/2679417190 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 17) v1 ==== 137342+0+0 (secure 0 0 0) 0x7fb9d0010430 con 0x7fb9e0102210 2024-03-28T00:25:29.731 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:29.725+0000 7fb9dd600700 1 --2- 172.21.15.119:0/2679417190 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fb9c40a29a0 0x7fb9c40a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:29.732 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:29.726+0000 7fb9dd600700 1 -- 172.21.15.119:0/2679417190 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fb9d00ced40 con 0x7fb9e0102210 2024-03-28T00:25:29.732 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:29.726+0000 7fb9d7400700 1 --2- 172.21.15.119:0/2679417190 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fb9c40a29a0 0x7fb9c40a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:29.732 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:29.726+0000 7fb9d7400700 1 --2- 172.21.15.119:0/2679417190 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fb9c40a29a0 0x7fb9c40a4e60 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fb9e0079840 tx=0x7fb9c8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:25:29.735 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:29.729+0000 7fb9dd600700 1 -- 172.21.15.119:0/2679417190 <== mon.1 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fb9d009bc80 con 0x7fb9e0102210 2024-03-28T00:25:29.956 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:29.949+0000 7fb9e6a00700 1 -- 172.21.15.119:0/2679417190 --> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb9e00759c0 con 0x7fb9c40a29a0 2024-03-28T00:25:29.977 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:29.971+0000 7fb9dd600700 1 -- 172.21.15.119:0/2679417190 <== mgr.14152 v2:172.21.15.119:6800/1401937096 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+258966 (secure 0 0 0) 0x7fb9e00759c0 con 0x7fb9c40a29a0 2024-03-28T00:25:29.982 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-28T00:25:29.984 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:29.977+0000 7fb9e6a00700 1 -- 172.21.15.119:0/2679417190 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fb9c40a29a0 msgr2=0x7fb9c40a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:29.984 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:29.977+0000 7fb9e6a00700 1 --2- 172.21.15.119:0/2679417190 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fb9c40a29a0 0x7fb9c40a4e60 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fb9e0079840 tx=0x7fb9c8009260 comp rx=0 tx=0).stop 2024-03-28T00:25:29.984 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:29.977+0000 7fb9e6a00700 1 -- 172.21.15.119:0/2679417190 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb9e0102210 msgr2=0x7fb9e0074490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:29.984 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:29.977+0000 7fb9e6a00700 1 --2- 172.21.15.119:0/2679417190 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb9e0102210 0x7fb9e0074490 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fb9d000e9a0 tx=0x7fb9d000ecb0 comp rx=0 tx=0).stop 2024-03-28T00:25:29.984 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:29.977+0000 7fb9e6a00700 1 -- 172.21.15.119:0/2679417190 shutdown_connections 2024-03-28T00:25:29.985 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:29.977+0000 7fb9e6a00700 1 --2- 172.21.15.119:0/2679417190 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fb9c40a29a0 0x7fb9c40a4e60 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:29.985 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:29.977+0000 7fb9e6a00700 1 --2- 172.21.15.119:0/2679417190 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb9e0108220 0x7fb9e0074f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:29.985 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:29.977+0000 7fb9e6a00700 1 --2- 172.21.15.119:0/2679417190 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb9e0102210 0x7fb9e0074490 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:29.985 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:29.977+0000 7fb9e6a00700 1 --2- 172.21.15.119:0/2679417190 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb9e0102bb0 0x7fb9e00749d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:29.985 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:29.977+0000 7fb9e6a00700 1 -- 172.21.15.119:0/2679417190 >> 172.21.15.119:0/2679417190 conn(0x7fb9e0079260 msgr2=0x7fb9e0100e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:25:29.985 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:29.977+0000 7fb9e6a00700 1 -- 172.21.15.119:0/2679417190 shutdown_connections 2024-03-28T00:25:29.985 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:29.977+0000 7fb9e6a00700 1 -- 172.21.15.119:0/2679417190 wait complete. 2024-03-28T00:25:29.985 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2024-03-28T00:25:30.904 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":360,"stamp":"2024-03-28T00:25:29.758618+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":662599,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1009,"num_read_kb":632,"num_write":423,"num_write_kb":3077,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1180758,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":348,"ondisk_log_size":348,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":373392,"kb_used_data":11712,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124568432,"statfs":{"total":1151940427776,"available":1151558074368,"internally_reserved":0,"allocated":11993088,"data_stored":7452453,"data_compressed":112659,"data_compressed_allocated":147456,"data_compressed_original":1794048,"omap_allocated":19143,"internal_metadata":370193721},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":135,"num_read_kb":70,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004747"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.536838+0000","last_change":"2024-03-28T00:23:56.763630+0000","last_active":"2024-03-28T00:24:05.536838+0000","last_peered":"2024-03-28T00:24:05.536838+0000","last_clean":"2024-03-28T00:24:05.536838+0000","last_became_active":"2024-03-28T00:23:56.763296+0000","last_became_peered":"2024-03-28T00:23:56.763296+0000","last_unstale":"2024-03-28T00:24:05.536838+0000","last_undegraded":"2024-03-28T00:24:05.536838+0000","last_fullsized":"2024-03-28T00:24:05.536838+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T11:40:36.071225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.554106+0000","last_change":"2024-03-28T00:23:56.763628+0000","last_active":"2024-03-28T00:24:05.554106+0000","last_peered":"2024-03-28T00:24:05.554106+0000","last_clean":"2024-03-28T00:24:05.554106+0000","last_became_active":"2024-03-28T00:23:56.761753+0000","last_became_peered":"2024-03-28T00:23:56.761753+0000","last_unstale":"2024-03-28T00:24:05.554106+0000","last_undegraded":"2024-03-28T00:24:05.554106+0000","last_fullsized":"2024-03-28T00:24:05.554106+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T11:48:50.804269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.553822+0000","last_change":"2024-03-28T00:23:56.759198+0000","last_active":"2024-03-28T00:24:05.553822+0000","last_peered":"2024-03-28T00:24:05.553822+0000","last_clean":"2024-03-28T00:24:05.553822+0000","last_became_active":"2024-03-28T00:23:56.758780+0000","last_became_peered":"2024-03-28T00:23:56.758780+0000","last_unstale":"2024-03-28T00:24:05.553822+0000","last_undegraded":"2024-03-28T00:24:05.553822+0000","last_fullsized":"2024-03-28T00:24:05.553822+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T10:46:15.731193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.538176+0000","last_change":"2024-03-28T00:23:56.736651+0000","last_active":"2024-03-28T00:24:05.538176+0000","last_peered":"2024-03-28T00:24:05.538176+0000","last_clean":"2024-03-28T00:24:05.538176+0000","last_became_active":"2024-03-28T00:23:56.736063+0000","last_became_peered":"2024-03-28T00:23:56.736063+0000","last_unstale":"2024-03-28T00:24:05.538176+0000","last_undegraded":"2024-03-28T00:24:05.538176+0000","last_fullsized":"2024-03-28T00:24:05.538176+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T09:34:28.221202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.534628+0000","last_change":"2024-03-28T00:23:56.751413+0000","last_active":"2024-03-28T00:24:05.534628+0000","last_peered":"2024-03-28T00:24:05.534628+0000","last_clean":"2024-03-28T00:24:05.534628+0000","last_became_active":"2024-03-28T00:23:56.751038+0000","last_became_peered":"2024-03-28T00:23:56.751038+0000","last_unstale":"2024-03-28T00:24:05.534628+0000","last_undegraded":"2024-03-28T00:24:05.534628+0000","last_fullsized":"2024-03-28T00:24:05.534628+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T10:45:08.645799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.539576+0000","last_change":"2024-03-28T00:23:56.750970+0000","last_active":"2024-03-28T00:24:05.539576+0000","last_peered":"2024-03-28T00:24:05.539576+0000","last_clean":"2024-03-28T00:24:05.539576+0000","last_became_active":"2024-03-28T00:23:56.750608+0000","last_became_peered":"2024-03-28T00:23:56.750608+0000","last_unstale":"2024-03-28T00:24:05.539576+0000","last_undegraded":"2024-03-28T00:24:05.539576+0000","last_fullsized":"2024-03-28T00:24:05.539576+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T00:32:07.448685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.534811+0000","last_change":"2024-03-28T00:23:56.751489+0000","last_active":"2024-03-28T00:24:05.534811+0000","last_peered":"2024-03-28T00:24:05.534811+0000","last_clean":"2024-03-28T00:24:05.534811+0000","last_became_active":"2024-03-28T00:23:56.750531+0000","last_became_peered":"2024-03-28T00:23:56.750531+0000","last_unstale":"2024-03-28T00:24:05.534811+0000","last_undegraded":"2024-03-28T00:24:05.534811+0000","last_fullsized":"2024-03-28T00:24:05.534811+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T11:18:22.604126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.973011+0000","last_change":"2024-03-28T00:23:56.751537+0000","last_active":"2024-03-28T00:24:05.973011+0000","last_peered":"2024-03-28T00:24:05.973011+0000","last_clean":"2024-03-28T00:24:05.973011+0000","last_became_active":"2024-03-28T00:23:56.751185+0000","last_became_peered":"2024-03-28T00:23:56.751185+0000","last_unstale":"2024-03-28T00:24:05.973011+0000","last_undegraded":"2024-03-28T00:24:05.973011+0000","last_fullsized":"2024-03-28T00:24:05.973011+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T03:21:30.334886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.583110+0000","last_change":"2024-03-28T00:23:56.739405+0000","last_active":"2024-03-28T00:24:05.583110+0000","last_peered":"2024-03-28T00:24:05.583110+0000","last_clean":"2024-03-28T00:24:05.583110+0000","last_became_active":"2024-03-28T00:23:56.739085+0000","last_became_peered":"2024-03-28T00:23:56.739085+0000","last_unstale":"2024-03-28T00:24:05.583110+0000","last_undegraded":"2024-03-28T00:24:05.583110+0000","last_fullsized":"2024-03-28T00:24:05.583110+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T10:57:08.892446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.539895+0000","last_change":"2024-03-28T00:23:56.736880+0000","last_active":"2024-03-28T00:24:05.539895+0000","last_peered":"2024-03-28T00:24:05.539895+0000","last_clean":"2024-03-28T00:24:05.539895+0000","last_became_active":"2024-03-28T00:23:56.736524+0000","last_became_peered":"2024-03-28T00:23:56.736524+0000","last_unstale":"2024-03-28T00:24:05.539895+0000","last_undegraded":"2024-03-28T00:24:05.539895+0000","last_fullsized":"2024-03-28T00:24:05.539895+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T09:55:19.552415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.535208+0000","last_change":"2024-03-28T00:23:56.736717+0000","last_active":"2024-03-28T00:24:05.535208+0000","last_peered":"2024-03-28T00:24:05.535208+0000","last_clean":"2024-03-28T00:24:05.535208+0000","last_became_active":"2024-03-28T00:23:56.735981+0000","last_became_peered":"2024-03-28T00:23:56.735981+0000","last_unstale":"2024-03-28T00:24:05.535208+0000","last_undegraded":"2024-03-28T00:24:05.535208+0000","last_fullsized":"2024-03-28T00:24:05.535208+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T12:02:54.293418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.538045+0000","last_change":"2024-03-28T00:23:56.740591+0000","last_active":"2024-03-28T00:24:05.538045+0000","last_peered":"2024-03-28T00:24:05.538045+0000","last_clean":"2024-03-28T00:24:05.538045+0000","last_became_active":"2024-03-28T00:23:56.739965+0000","last_became_peered":"2024-03-28T00:23:56.739965+0000","last_unstale":"2024-03-28T00:24:05.538045+0000","last_undegraded":"2024-03-28T00:24:05.538045+0000","last_fullsized":"2024-03-28T00:24:05.538045+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T07:10:35.426260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.538408+0000","last_change":"2024-03-28T00:23:56.744686+0000","last_active":"2024-03-28T00:24:05.538408+0000","last_peered":"2024-03-28T00:24:05.538408+0000","last_clean":"2024-03-28T00:24:05.538408+0000","last_became_active":"2024-03-28T00:23:56.744355+0000","last_became_peered":"2024-03-28T00:23:56.744355+0000","last_unstale":"2024-03-28T00:24:05.538408+0000","last_undegraded":"2024-03-28T00:24:05.538408+0000","last_fullsized":"2024-03-28T00:24:05.538408+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T07:52:27.360917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.553265+0000","last_change":"2024-03-28T00:23:56.762465+0000","last_active":"2024-03-28T00:24:05.553265+0000","last_peered":"2024-03-28T00:24:05.553265+0000","last_clean":"2024-03-28T00:24:05.553265+0000","last_became_active":"2024-03-28T00:23:56.751467+0000","last_became_peered":"2024-03-28T00:23:56.751467+0000","last_unstale":"2024-03-28T00:24:05.553265+0000","last_undegraded":"2024-03-28T00:24:05.553265+0000","last_fullsized":"2024-03-28T00:24:05.553265+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T10:55:33.880872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.553329+0000","last_change":"2024-03-28T00:23:56.759344+0000","last_active":"2024-03-28T00:24:05.553329+0000","last_peered":"2024-03-28T00:24:05.553329+0000","last_clean":"2024-03-28T00:24:05.553329+0000","last_became_active":"2024-03-28T00:23:56.758694+0000","last_became_peered":"2024-03-28T00:23:56.758694+0000","last_unstale":"2024-03-28T00:24:05.553329+0000","last_undegraded":"2024-03-28T00:24:05.553329+0000","last_fullsized":"2024-03-28T00:24:05.553329+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T02:43:42.586070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.582511+0000","last_change":"2024-03-28T00:23:56.759298+0000","last_active":"2024-03-28T00:24:05.582511+0000","last_peered":"2024-03-28T00:24:05.582511+0000","last_clean":"2024-03-28T00:24:05.582511+0000","last_became_active":"2024-03-28T00:23:56.759006+0000","last_became_peered":"2024-03-28T00:23:56.759006+0000","last_unstale":"2024-03-28T00:24:05.582511+0000","last_undegraded":"2024-03-28T00:24:05.582511+0000","last_fullsized":"2024-03-28T00:24:05.582511+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T00:43:09.433789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.536834+0000","last_change":"2024-03-28T00:23:56.760324+0000","last_active":"2024-03-28T00:24:05.536834+0000","last_peered":"2024-03-28T00:24:05.536834+0000","last_clean":"2024-03-28T00:24:05.536834+0000","last_became_active":"2024-03-28T00:23:56.759866+0000","last_became_peered":"2024-03-28T00:23:56.759866+0000","last_unstale":"2024-03-28T00:24:05.536834+0000","last_undegraded":"2024-03-28T00:24:05.536834+0000","last_fullsized":"2024-03-28T00:24:05.536834+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T01:11:15.923114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.547070+0000","last_change":"2024-03-28T00:23:56.755227+0000","last_active":"2024-03-28T00:24:05.547070+0000","last_peered":"2024-03-28T00:24:05.547070+0000","last_clean":"2024-03-28T00:24:05.547070+0000","last_became_active":"2024-03-28T00:23:56.754767+0000","last_became_peered":"2024-03-28T00:23:56.754767+0000","last_unstale":"2024-03-28T00:24:05.547070+0000","last_undegraded":"2024-03-28T00:24:05.547070+0000","last_fullsized":"2024-03-28T00:24:05.547070+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T07:26:04.797904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.582571+0000","last_change":"2024-03-28T00:23:56.744199+0000","last_active":"2024-03-28T00:24:05.582571+0000","last_peered":"2024-03-28T00:24:05.582571+0000","last_clean":"2024-03-28T00:24:05.582571+0000","last_became_active":"2024-03-28T00:23:56.743848+0000","last_became_peered":"2024-03-28T00:23:56.743848+0000","last_unstale":"2024-03-28T00:24:05.582571+0000","last_undegraded":"2024-03-28T00:24:05.582571+0000","last_fullsized":"2024-03-28T00:24:05.582571+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T10:08:01.426115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'13","reported_seq":116,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:25:25.637344+0000","last_change":"2024-03-28T00:23:55.731996+0000","last_active":"2024-03-28T00:25:25.637344+0000","last_peered":"2024-03-28T00:25:25.637344+0000","last_clean":"2024-03-28T00:25:25.637344+0000","last_became_active":"2024-03-28T00:23:55.731626+0000","last_became_peered":"2024-03-28T00:23:55.731626+0000","last_unstale":"2024-03-28T00:25:25.637344+0000","last_undegraded":"2024-03-28T00:25:25.637344+0000","last_fullsized":"2024-03-28T00:25:25.637344+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_clean_scrub_stamp":"2024-03-28T00:23:54.693629+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T08:56:00.420387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":84,"num_read_kb":84,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.973978+0000","last_change":"2024-03-28T00:23:56.757048+0000","last_active":"2024-03-28T00:24:05.973978+0000","last_peered":"2024-03-28T00:24:05.973978+0000","last_clean":"2024-03-28T00:24:05.973978+0000","last_became_active":"2024-03-28T00:23:56.752710+0000","last_became_peered":"2024-03-28T00:23:56.752710+0000","last_unstale":"2024-03-28T00:24:05.973978+0000","last_undegraded":"2024-03-28T00:24:05.973978+0000","last_fullsized":"2024-03-28T00:24:05.973978+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T06:26:39.723255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.974155+0000","last_change":"2024-03-28T00:23:55.748191+0000","last_active":"2024-03-28T00:24:05.974155+0000","last_peered":"2024-03-28T00:24:05.974155+0000","last_clean":"2024-03-28T00:24:05.974155+0000","last_became_active":"2024-03-28T00:23:55.747154+0000","last_became_peered":"2024-03-28T00:23:55.747154+0000","last_unstale":"2024-03-28T00:24:05.974155+0000","last_undegraded":"2024-03-28T00:24:05.974155+0000","last_fullsized":"2024-03-28T00:24:05.974155+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_clean_scrub_stamp":"2024-03-28T00:23:54.693629+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T07:17:12.296494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.553663+0000","last_change":"2024-03-28T00:23:56.758979+0000","last_active":"2024-03-28T00:24:05.553663+0000","last_peered":"2024-03-28T00:24:05.553663+0000","last_clean":"2024-03-28T00:24:05.553663+0000","last_became_active":"2024-03-28T00:23:56.752628+0000","last_became_peered":"2024-03-28T00:23:56.752628+0000","last_unstale":"2024-03-28T00:24:05.553663+0000","last_undegraded":"2024-03-28T00:24:05.553663+0000","last_fullsized":"2024-03-28T00:24:05.553663+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T10:11:57.245938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"85'12","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.535173+0000","last_change":"2024-03-28T00:23:55.728972+0000","last_active":"2024-03-28T00:24:05.535173+0000","last_peered":"2024-03-28T00:24:05.535173+0000","last_clean":"2024-03-28T00:24:05.535173+0000","last_became_active":"2024-03-28T00:23:55.728053+0000","last_became_peered":"2024-03-28T00:23:55.728053+0000","last_unstale":"2024-03-28T00:24:05.535173+0000","last_undegraded":"2024-03-28T00:24:05.535173+0000","last_fullsized":"2024-03-28T00:24:05.535173+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_clean_scrub_stamp":"2024-03-28T00:23:54.693629+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T04:12:27.129196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.582740+0000","last_change":"2024-03-28T00:23:56.752585+0000","last_active":"2024-03-28T00:24:05.582740+0000","last_peered":"2024-03-28T00:24:05.582740+0000","last_clean":"2024-03-28T00:24:05.582740+0000","last_became_active":"2024-03-28T00:23:56.752252+0000","last_became_peered":"2024-03-28T00:23:56.752252+0000","last_unstale":"2024-03-28T00:24:05.582740+0000","last_undegraded":"2024-03-28T00:24:05.582740+0000","last_fullsized":"2024-03-28T00:24:05.582740+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T11:51:02.864486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'5","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.534887+0000","last_change":"2024-03-28T00:23:55.726859+0000","last_active":"2024-03-28T00:24:05.534887+0000","last_peered":"2024-03-28T00:24:05.534887+0000","last_clean":"2024-03-28T00:24:05.534887+0000","last_became_active":"2024-03-28T00:23:55.725372+0000","last_became_peered":"2024-03-28T00:23:55.725372+0000","last_unstale":"2024-03-28T00:24:05.534887+0000","last_undegraded":"2024-03-28T00:24:05.534887+0000","last_fullsized":"2024-03-28T00:24:05.534887+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_clean_scrub_stamp":"2024-03-28T00:23:54.693629+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T00:52:35.278544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.538784+0000","last_change":"2024-03-28T00:23:56.742076+0000","last_active":"2024-03-28T00:24:05.538784+0000","last_peered":"2024-03-28T00:24:05.538784+0000","last_clean":"2024-03-28T00:24:05.538784+0000","last_became_active":"2024-03-28T00:23:56.741761+0000","last_became_peered":"2024-03-28T00:23:56.741761+0000","last_unstale":"2024-03-28T00:24:05.538784+0000","last_undegraded":"2024-03-28T00:24:05.538784+0000","last_fullsized":"2024-03-28T00:24:05.538784+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T08:17:02.024132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"85'6","reported_seq":111,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:25:26.604840+0000","last_change":"2024-03-28T00:23:55.728587+0000","last_active":"2024-03-28T00:25:26.604840+0000","last_peered":"2024-03-28T00:25:26.604840+0000","last_clean":"2024-03-28T00:25:26.604840+0000","last_became_active":"2024-03-28T00:23:55.728309+0000","last_became_peered":"2024-03-28T00:23:55.728309+0000","last_unstale":"2024-03-28T00:25:26.604840+0000","last_undegraded":"2024-03-28T00:25:26.604840+0000","last_fullsized":"2024-03-28T00:25:26.604840+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_clean_scrub_stamp":"2024-03-28T00:23:54.693629+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T04:15:12.060649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":2,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.534609+0000","last_change":"2024-03-28T00:23:56.747351+0000","last_active":"2024-03-28T00:24:05.534609+0000","last_peered":"2024-03-28T00:24:05.534609+0000","last_clean":"2024-03-28T00:24:05.534609+0000","last_became_active":"2024-03-28T00:23:56.746995+0000","last_became_peered":"2024-03-28T00:23:56.746995+0000","last_unstale":"2024-03-28T00:24:05.534609+0000","last_undegraded":"2024-03-28T00:24:05.534609+0000","last_fullsized":"2024-03-28T00:24:05.534609+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T00:43:53.521420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"85'7","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.539564+0000","last_change":"2024-03-28T00:23:55.730157+0000","last_active":"2024-03-28T00:24:05.539564+0000","last_peered":"2024-03-28T00:24:05.539564+0000","last_clean":"2024-03-28T00:24:05.539564+0000","last_became_active":"2024-03-28T00:23:55.729841+0000","last_became_peered":"2024-03-28T00:23:55.729841+0000","last_unstale":"2024-03-28T00:24:05.539564+0000","last_undegraded":"2024-03-28T00:24:05.539564+0000","last_fullsized":"2024-03-28T00:24:05.539564+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_clean_scrub_stamp":"2024-03-28T00:23:54.693629+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T09:57:31.865444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.538154+0000","last_change":"2024-03-28T00:23:56.760077+0000","last_active":"2024-03-28T00:24:05.538154+0000","last_peered":"2024-03-28T00:24:05.538154+0000","last_clean":"2024-03-28T00:24:05.538154+0000","last_became_active":"2024-03-28T00:23:56.759735+0000","last_became_peered":"2024-03-28T00:23:56.759735+0000","last_unstale":"2024-03-28T00:24:05.538154+0000","last_undegraded":"2024-03-28T00:24:05.538154+0000","last_fullsized":"2024-03-28T00:24:05.538154+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T11:36:17.224248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.537959+0000","last_change":"2024-03-28T00:23:55.738216+0000","last_active":"2024-03-28T00:24:05.537959+0000","last_peered":"2024-03-28T00:24:05.537959+0000","last_clean":"2024-03-28T00:24:05.537959+0000","last_became_active":"2024-03-28T00:23:55.737878+0000","last_became_peered":"2024-03-28T00:23:55.737878+0000","last_unstale":"2024-03-28T00:24:05.537959+0000","last_undegraded":"2024-03-28T00:24:05.537959+0000","last_fullsized":"2024-03-28T00:24:05.537959+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_clean_scrub_stamp":"2024-03-28T00:23:54.693629+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T06:30:00.465906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.973060+0000","last_change":"2024-03-28T00:23:56.760362+0000","last_active":"2024-03-28T00:24:05.973060+0000","last_peered":"2024-03-28T00:24:05.973060+0000","last_clean":"2024-03-28T00:24:05.973060+0000","last_became_active":"2024-03-28T00:23:56.759918+0000","last_became_peered":"2024-03-28T00:23:56.759918+0000","last_unstale":"2024-03-28T00:24:05.973060+0000","last_undegraded":"2024-03-28T00:24:05.973060+0000","last_fullsized":"2024-03-28T00:24:05.973060+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T04:59:01.038539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"85'3","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.537941+0000","last_change":"2024-03-28T00:23:55.727557+0000","last_active":"2024-03-28T00:24:05.537941+0000","last_peered":"2024-03-28T00:24:05.537941+0000","last_clean":"2024-03-28T00:24:05.537941+0000","last_became_active":"2024-03-28T00:23:55.727235+0000","last_became_peered":"2024-03-28T00:23:55.727235+0000","last_unstale":"2024-03-28T00:24:05.537941+0000","last_undegraded":"2024-03-28T00:24:05.537941+0000","last_fullsized":"2024-03-28T00:24:05.537941+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_clean_scrub_stamp":"2024-03-28T00:23:54.693629+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T08:14:57.268850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.546468+0000","last_change":"2024-03-28T00:23:56.766812+0000","last_active":"2024-03-28T00:24:05.546468+0000","last_peered":"2024-03-28T00:24:05.546468+0000","last_clean":"2024-03-28T00:24:05.546468+0000","last_became_active":"2024-03-28T00:23:56.766386+0000","last_became_peered":"2024-03-28T00:23:56.766386+0000","last_unstale":"2024-03-28T00:24:05.546468+0000","last_undegraded":"2024-03-28T00:24:05.546468+0000","last_fullsized":"2024-03-28T00:24:05.546468+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T05:58:07.349158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"85'4","reported_seq":191,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:25:27.556291+0000","last_change":"2024-03-28T00:23:55.727617+0000","last_active":"2024-03-28T00:25:27.556291+0000","last_peered":"2024-03-28T00:25:27.556291+0000","last_clean":"2024-03-28T00:25:27.556291+0000","last_became_active":"2024-03-28T00:23:55.727276+0000","last_became_peered":"2024-03-28T00:23:55.727276+0000","last_unstale":"2024-03-28T00:25:27.556291+0000","last_undegraded":"2024-03-28T00:25:27.556291+0000","last_fullsized":"2024-03-28T00:25:27.556291+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_clean_scrub_stamp":"2024-03-28T00:23:54.693629+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T02:27:04.376342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":168,"num_read_kb":168,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.582928+0000","last_change":"2024-03-28T00:23:56.747261+0000","last_active":"2024-03-28T00:24:05.582928+0000","last_peered":"2024-03-28T00:24:05.582928+0000","last_clean":"2024-03-28T00:24:05.582928+0000","last_became_active":"2024-03-28T00:23:56.746862+0000","last_became_peered":"2024-03-28T00:23:56.746862+0000","last_unstale":"2024-03-28T00:24:05.582928+0000","last_undegraded":"2024-03-28T00:24:05.582928+0000","last_fullsized":"2024-03-28T00:24:05.582928+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T07:33:35.747722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"85'4","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.535084+0000","last_change":"2024-03-28T00:23:55.725568+0000","last_active":"2024-03-28T00:24:05.535084+0000","last_peered":"2024-03-28T00:24:05.535084+0000","last_clean":"2024-03-28T00:24:05.535084+0000","last_became_active":"2024-03-28T00:23:55.724967+0000","last_became_peered":"2024-03-28T00:23:55.724967+0000","last_unstale":"2024-03-28T00:24:05.535084+0000","last_undegraded":"2024-03-28T00:24:05.535084+0000","last_fullsized":"2024-03-28T00:24:05.535084+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_clean_scrub_stamp":"2024-03-28T00:23:54.693629+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T10:40:00.731885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.553554+0000","last_change":"2024-03-28T00:23:56.763176+0000","last_active":"2024-03-28T00:24:05.553554+0000","last_peered":"2024-03-28T00:24:05.553554+0000","last_clean":"2024-03-28T00:24:05.553554+0000","last_became_active":"2024-03-28T00:23:56.762709+0000","last_became_peered":"2024-03-28T00:23:56.762709+0000","last_unstale":"2024-03-28T00:24:05.553554+0000","last_undegraded":"2024-03-28T00:24:05.553554+0000","last_fullsized":"2024-03-28T00:24:05.553554+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T05:58:52.003151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'5","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.538869+0000","last_change":"2024-03-28T00:23:55.745579+0000","last_active":"2024-03-28T00:24:05.538869+0000","last_peered":"2024-03-28T00:24:05.538869+0000","last_clean":"2024-03-28T00:24:05.538869+0000","last_became_active":"2024-03-28T00:23:55.745262+0000","last_became_peered":"2024-03-28T00:23:55.745262+0000","last_unstale":"2024-03-28T00:24:05.538869+0000","last_undegraded":"2024-03-28T00:24:05.538869+0000","last_fullsized":"2024-03-28T00:24:05.538869+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_clean_scrub_stamp":"2024-03-28T00:23:54.693629+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T00:48:36.212168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.582850+0000","last_change":"2024-03-28T00:23:56.736393+0000","last_active":"2024-03-28T00:24:05.582850+0000","last_peered":"2024-03-28T00:24:05.582850+0000","last_clean":"2024-03-28T00:24:05.582850+0000","last_became_active":"2024-03-28T00:23:56.736054+0000","last_became_peered":"2024-03-28T00:23:56.736054+0000","last_unstale":"2024-03-28T00:24:05.582850+0000","last_undegraded":"2024-03-28T00:24:05.582850+0000","last_fullsized":"2024-03-28T00:24:05.582850+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T11:56:00.039257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"87'32","reported_seq":226,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:25:27.556247+0000","last_change":"2024-03-28T00:23:55.722192+0000","last_active":"2024-03-28T00:25:27.556247+0000","last_peered":"2024-03-28T00:25:27.556247+0000","last_clean":"2024-03-28T00:25:27.556247+0000","last_became_active":"2024-03-28T00:23:55.721881+0000","last_became_peered":"2024-03-28T00:23:55.721881+0000","last_unstale":"2024-03-28T00:25:27.556247+0000","last_undegraded":"2024-03-28T00:25:27.556247+0000","last_fullsized":"2024-03-28T00:25:27.556247+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_clean_scrub_stamp":"2024-03-28T00:23:54.693629+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T03:47:34.950264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64402,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":175,"num_read_kb":149,"num_write":34,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.534978+0000","last_change":"2024-03-28T00:23:56.738361+0000","last_active":"2024-03-28T00:24:05.534978+0000","last_peered":"2024-03-28T00:24:05.534978+0000","last_clean":"2024-03-28T00:24:05.534978+0000","last_became_active":"2024-03-28T00:23:56.737040+0000","last_became_peered":"2024-03-28T00:23:56.737040+0000","last_unstale":"2024-03-28T00:24:05.534978+0000","last_undegraded":"2024-03-28T00:24:05.534978+0000","last_fullsized":"2024-03-28T00:24:05.534978+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T11:39:40.653421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'9","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.582640+0000","last_change":"2024-03-28T00:23:55.733943+0000","last_active":"2024-03-28T00:24:05.582640+0000","last_peered":"2024-03-28T00:24:05.582640+0000","last_clean":"2024-03-28T00:24:05.582640+0000","last_became_active":"2024-03-28T00:23:55.733236+0000","last_became_peered":"2024-03-28T00:23:55.733236+0000","last_unstale":"2024-03-28T00:24:05.582640+0000","last_undegraded":"2024-03-28T00:24:05.582640+0000","last_fullsized":"2024-03-28T00:24:05.582640+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_clean_scrub_stamp":"2024-03-28T00:23:54.693629+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T09:45:33.070408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.538244+0000","last_change":"2024-03-28T00:23:56.758751+0000","last_active":"2024-03-28T00:24:05.538244+0000","last_peered":"2024-03-28T00:24:05.538244+0000","last_clean":"2024-03-28T00:24:05.538244+0000","last_became_active":"2024-03-28T00:23:56.758066+0000","last_became_peered":"2024-03-28T00:23:56.758066+0000","last_unstale":"2024-03-28T00:24:05.538244+0000","last_undegraded":"2024-03-28T00:24:05.538244+0000","last_fullsized":"2024-03-28T00:24:05.538244+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T02:21:42.514212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"85'5","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.538122+0000","last_change":"2024-03-28T00:23:55.737167+0000","last_active":"2024-03-28T00:24:05.538122+0000","last_peered":"2024-03-28T00:24:05.538122+0000","last_clean":"2024-03-28T00:24:05.538122+0000","last_became_active":"2024-03-28T00:23:55.736925+0000","last_became_peered":"2024-03-28T00:23:55.736925+0000","last_unstale":"2024-03-28T00:24:05.538122+0000","last_undegraded":"2024-03-28T00:24:05.538122+0000","last_fullsized":"2024-03-28T00:24:05.538122+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_clean_scrub_stamp":"2024-03-28T00:23:54.693629+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T09:13:24.820375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.535057+0000","last_change":"2024-03-28T00:23:56.745939+0000","last_active":"2024-03-28T00:24:05.535057+0000","last_peered":"2024-03-28T00:24:05.535057+0000","last_clean":"2024-03-28T00:24:05.535057+0000","last_became_active":"2024-03-28T00:23:56.745504+0000","last_became_peered":"2024-03-28T00:23:56.745504+0000","last_unstale":"2024-03-28T00:24:05.535057+0000","last_undegraded":"2024-03-28T00:24:05.535057+0000","last_fullsized":"2024-03-28T00:24:05.535057+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T06:02:34.199029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"85'6","reported_seq":112,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:25:27.597961+0000","last_change":"2024-03-28T00:23:55.726916+0000","last_active":"2024-03-28T00:25:27.597961+0000","last_peered":"2024-03-28T00:25:27.597961+0000","last_clean":"2024-03-28T00:25:27.597961+0000","last_became_active":"2024-03-28T00:23:55.726570+0000","last_became_peered":"2024-03-28T00:23:55.726570+0000","last_unstale":"2024-03-28T00:25:27.597961+0000","last_undegraded":"2024-03-28T00:25:27.597961+0000","last_fullsized":"2024-03-28T00:25:27.597961+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_clean_scrub_stamp":"2024-03-28T00:23:54.693629+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T06:28:54.581129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":88,"num_read_kb":3,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.534720+0000","last_change":"2024-03-28T00:23:56.738303+0000","last_active":"2024-03-28T00:24:05.534720+0000","last_peered":"2024-03-28T00:24:05.534720+0000","last_clean":"2024-03-28T00:24:05.534720+0000","last_became_active":"2024-03-28T00:23:56.737926+0000","last_became_peered":"2024-03-28T00:23:56.737926+0000","last_unstale":"2024-03-28T00:24:05.534720+0000","last_undegraded":"2024-03-28T00:24:05.534720+0000","last_fullsized":"2024-03-28T00:24:05.534720+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T01:56:53.985793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'17","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.534473+0000","last_change":"2024-03-28T00:23:55.728708+0000","last_active":"2024-03-28T00:24:05.534473+0000","last_peered":"2024-03-28T00:24:05.534473+0000","last_clean":"2024-03-28T00:24:05.534473+0000","last_became_active":"2024-03-28T00:23:55.728415+0000","last_became_peered":"2024-03-28T00:23:55.728415+0000","last_unstale":"2024-03-28T00:24:05.534473+0000","last_undegraded":"2024-03-28T00:24:05.534473+0000","last_fullsized":"2024-03-28T00:24:05.534473+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_clean_scrub_stamp":"2024-03-28T00:23:54.693629+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T05:36:14.405953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.538144+0000","last_change":"2024-03-28T00:23:56.741982+0000","last_active":"2024-03-28T00:24:05.538144+0000","last_peered":"2024-03-28T00:24:05.538144+0000","last_clean":"2024-03-28T00:24:05.538144+0000","last_became_active":"2024-03-28T00:23:56.741642+0000","last_became_peered":"2024-03-28T00:23:56.741642+0000","last_unstale":"2024-03-28T00:24:05.538144+0000","last_undegraded":"2024-03-28T00:24:05.538144+0000","last_fullsized":"2024-03-28T00:24:05.538144+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T04:27:49.168164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.544593+0000","last_change":"2024-03-28T00:24:05.544144+0000","last_active":"2024-03-28T00:24:05.544593+0000","last_peered":"2024-03-28T00:24:05.544593+0000","last_clean":"2024-03-28T00:24:05.544593+0000","last_became_active":"2024-03-28T00:24:05.540061+0000","last_became_peered":"2024-03-28T00:24:05.540061+0000","last_unstale":"2024-03-28T00:24:05.544593+0000","last_undegraded":"2024-03-28T00:24:05.544593+0000","last_fullsized":"2024-03-28T00:24:05.544593+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_clean_scrub_stamp":"2024-03-28T00:23:54.693629+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T11:24:24.644313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.536666+0000","last_change":"2024-03-28T00:23:56.739241+0000","last_active":"2024-03-28T00:24:05.536666+0000","last_peered":"2024-03-28T00:24:05.536666+0000","last_clean":"2024-03-28T00:24:05.536666+0000","last_became_active":"2024-03-28T00:23:56.738122+0000","last_became_peered":"2024-03-28T00:23:56.738122+0000","last_unstale":"2024-03-28T00:24:05.536666+0000","last_undegraded":"2024-03-28T00:24:05.536666+0000","last_fullsized":"2024-03-28T00:24:05.536666+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T02:52:50.375314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"85'1","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.558013+0000","last_change":"2024-03-28T00:24:05.558013+0000","last_active":"2024-03-28T00:24:05.558013+0000","last_peered":"2024-03-28T00:24:05.558013+0000","last_clean":"2024-03-28T00:24:05.558013+0000","last_became_active":"2024-03-28T00:24:05.555882+0000","last_became_peered":"2024-03-28T00:24:05.555882+0000","last_unstale":"2024-03-28T00:24:05.558013+0000","last_undegraded":"2024-03-28T00:24:05.558013+0000","last_fullsized":"2024-03-28T00:24:05.558013+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_clean_scrub_stamp":"2024-03-28T00:23:54.693629+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T01:51:23.576990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"85'1","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.539849+0000","last_change":"2024-03-28T00:23:56.752780+0000","last_active":"2024-03-28T00:24:05.539849+0000","last_peered":"2024-03-28T00:24:05.539849+0000","last_clean":"2024-03-28T00:24:05.539849+0000","last_became_active":"2024-03-28T00:23:56.751608+0000","last_became_peered":"2024-03-28T00:23:56.751608+0000","last_unstale":"2024-03-28T00:24:05.539849+0000","last_undegraded":"2024-03-28T00:24:05.539849+0000","last_fullsized":"2024-03-28T00:24:05.539849+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T01:20:45.485846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"85'5","reported_seq":108,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:25:25.637891+0000","last_change":"2024-03-28T00:23:55.728123+0000","last_active":"2024-03-28T00:25:25.637891+0000","last_peered":"2024-03-28T00:25:25.637891+0000","last_clean":"2024-03-28T00:25:25.637891+0000","last_became_active":"2024-03-28T00:23:55.727839+0000","last_became_peered":"2024-03-28T00:23:55.727839+0000","last_unstale":"2024-03-28T00:25:25.637891+0000","last_undegraded":"2024-03-28T00:25:25.637891+0000","last_fullsized":"2024-03-28T00:25:25.637891+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_clean_scrub_stamp":"2024-03-28T00:23:54.693629+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T11:39:30.315824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":84,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.555736+0000","last_change":"2024-03-28T00:24:05.555736+0000","last_active":"2024-03-28T00:24:05.555736+0000","last_peered":"2024-03-28T00:24:05.555736+0000","last_clean":"2024-03-28T00:24:05.555736+0000","last_became_active":"2024-03-28T00:24:05.555429+0000","last_became_peered":"2024-03-28T00:24:05.555429+0000","last_unstale":"2024-03-28T00:24:05.555736+0000","last_undegraded":"2024-03-28T00:24:05.555736+0000","last_fullsized":"2024-03-28T00:24:05.555736+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T03:57:58.828548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.583672+0000","last_change":"2024-03-28T00:24:05.583672+0000","last_active":"2024-03-28T00:24:05.583672+0000","last_peered":"2024-03-28T00:24:05.583672+0000","last_clean":"2024-03-28T00:24:05.583672+0000","last_became_active":"2024-03-28T00:24:05.583301+0000","last_became_peered":"2024-03-28T00:24:05.583301+0000","last_unstale":"2024-03-28T00:24:05.583672+0000","last_undegraded":"2024-03-28T00:24:05.583672+0000","last_fullsized":"2024-03-28T00:24:05.583672+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T08:20:17.464773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,8],"acting":[5,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"85'5","reported_seq":109,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:25:26.596094+0000","last_change":"2024-03-28T00:23:55.747124+0000","last_active":"2024-03-28T00:25:26.596094+0000","last_peered":"2024-03-28T00:25:26.596094+0000","last_clean":"2024-03-28T00:25:26.596094+0000","last_became_active":"2024-03-28T00:23:55.745721+0000","last_became_peered":"2024-03-28T00:23:55.745721+0000","last_unstale":"2024-03-28T00:25:26.596094+0000","last_undegraded":"2024-03-28T00:25:26.596094+0000","last_fullsized":"2024-03-28T00:25:26.596094+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_clean_scrub_stamp":"2024-03-28T00:23:54.693629+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T10:37:15.898923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":85,"num_read_kb":85,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.973656+0000","last_change":"2024-03-28T00:23:56.757060+0000","last_active":"2024-03-28T00:24:05.973656+0000","last_peered":"2024-03-28T00:24:05.973656+0000","last_clean":"2024-03-28T00:24:05.973656+0000","last_became_active":"2024-03-28T00:23:56.756754+0000","last_became_peered":"2024-03-28T00:23:56.756754+0000","last_unstale":"2024-03-28T00:24:05.973656+0000","last_undegraded":"2024-03-28T00:24:05.973656+0000","last_fullsized":"2024-03-28T00:24:05.973656+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T11:59:32.877471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'6","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.537983+0000","last_change":"2024-03-28T00:23:55.728039+0000","last_active":"2024-03-28T00:24:05.537983+0000","last_peered":"2024-03-28T00:24:05.537983+0000","last_clean":"2024-03-28T00:24:05.537983+0000","last_became_active":"2024-03-28T00:23:55.727705+0000","last_became_peered":"2024-03-28T00:23:55.727705+0000","last_unstale":"2024-03-28T00:24:05.537983+0000","last_undegraded":"2024-03-28T00:24:05.537983+0000","last_fullsized":"2024-03-28T00:24:05.537983+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_clean_scrub_stamp":"2024-03-28T00:23:54.693629+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T03:48:39.028754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":355,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.534852+0000","last_change":"2024-03-28T00:22:35.416127+0000","last_active":"2024-03-28T00:24:05.534852+0000","last_peered":"2024-03-28T00:24:05.534852+0000","last_clean":"2024-03-28T00:24:05.534852+0000","last_became_active":"2024-03-28T00:22:35.415710+0000","last_became_peered":"2024-03-28T00:22:35.415710+0000","last_unstale":"2024-03-28T00:24:05.534852+0000","last_undegraded":"2024-03-28T00:24:05.534852+0000","last_fullsized":"2024-03-28T00:24:05.534852+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:18:46.602205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:18:46.602205+0000","last_clean_scrub_stamp":"2024-03-28T00:18:46.602205+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T11:31:44.851848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.537824+0000","last_change":"2024-03-28T00:23:56.741550+0000","last_active":"2024-03-28T00:24:05.537824+0000","last_peered":"2024-03-28T00:24:05.537824+0000","last_clean":"2024-03-28T00:24:05.537824+0000","last_became_active":"2024-03-28T00:23:56.740750+0000","last_became_peered":"2024-03-28T00:23:56.740750+0000","last_unstale":"2024-03-28T00:24:05.537824+0000","last_undegraded":"2024-03-28T00:24:05.537824+0000","last_fullsized":"2024-03-28T00:24:05.537824+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T09:14:35.259597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'4","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.534882+0000","last_change":"2024-03-28T00:23:55.731623+0000","last_active":"2024-03-28T00:24:05.534882+0000","last_peered":"2024-03-28T00:24:05.534882+0000","last_clean":"2024-03-28T00:24:05.534882+0000","last_became_active":"2024-03-28T00:23:55.731292+0000","last_became_peered":"2024-03-28T00:23:55.731292+0000","last_unstale":"2024-03-28T00:24:05.534882+0000","last_undegraded":"2024-03-28T00:24:05.534882+0000","last_fullsized":"2024-03-28T00:24:05.534882+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_clean_scrub_stamp":"2024-03-28T00:23:54.693629+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T06:16:23.001339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"85'4","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.535069+0000","last_change":"2024-03-28T00:23:55.745553+0000","last_active":"2024-03-28T00:24:05.535069+0000","last_peered":"2024-03-28T00:24:05.535069+0000","last_clean":"2024-03-28T00:24:05.535069+0000","last_became_active":"2024-03-28T00:23:55.745204+0000","last_became_peered":"2024-03-28T00:23:55.745204+0000","last_unstale":"2024-03-28T00:24:05.535069+0000","last_undegraded":"2024-03-28T00:24:05.535069+0000","last_fullsized":"2024-03-28T00:24:05.535069+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_clean_scrub_stamp":"2024-03-28T00:23:54.693629+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T03:23:50.584362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.538733+0000","last_change":"2024-03-28T00:23:56.751394+0000","last_active":"2024-03-28T00:24:05.538733+0000","last_peered":"2024-03-28T00:24:05.538733+0000","last_clean":"2024-03-28T00:24:05.538733+0000","last_became_active":"2024-03-28T00:23:56.751025+0000","last_became_peered":"2024-03-28T00:23:56.751025+0000","last_unstale":"2024-03-28T00:24:05.538733+0000","last_undegraded":"2024-03-28T00:24:05.538733+0000","last_fullsized":"2024-03-28T00:24:05.538733+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T03:59:28.738018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"87'12","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:10.080892+0000","last_change":"2024-03-28T00:23:55.735055+0000","last_active":"2024-03-28T00:24:10.080892+0000","last_peered":"2024-03-28T00:24:10.080892+0000","last_clean":"2024-03-28T00:24:10.080892+0000","last_became_active":"2024-03-28T00:23:55.734747+0000","last_became_peered":"2024-03-28T00:23:55.734747+0000","last_unstale":"2024-03-28T00:24:10.080892+0000","last_undegraded":"2024-03-28T00:24:10.080892+0000","last_fullsized":"2024-03-28T00:24:10.080892+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_clean_scrub_stamp":"2024-03-28T00:23:54.693629+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T03:02:13.533883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.974285+0000","last_change":"2024-03-28T00:23:56.760581+0000","last_active":"2024-03-28T00:24:05.974285+0000","last_peered":"2024-03-28T00:24:05.974285+0000","last_clean":"2024-03-28T00:24:05.974285+0000","last_became_active":"2024-03-28T00:23:56.756807+0000","last_became_peered":"2024-03-28T00:23:56.756807+0000","last_unstale":"2024-03-28T00:24:05.974285+0000","last_undegraded":"2024-03-28T00:24:05.974285+0000","last_fullsized":"2024-03-28T00:24:05.974285+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T07:13:34.097671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'7","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.538170+0000","last_change":"2024-03-28T00:23:55.727864+0000","last_active":"2024-03-28T00:24:05.538170+0000","last_peered":"2024-03-28T00:24:05.538170+0000","last_clean":"2024-03-28T00:24:05.538170+0000","last_became_active":"2024-03-28T00:23:55.727507+0000","last_became_peered":"2024-03-28T00:23:55.727507+0000","last_unstale":"2024-03-28T00:24:05.538170+0000","last_undegraded":"2024-03-28T00:24:05.538170+0000","last_fullsized":"2024-03-28T00:24:05.538170+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_clean_scrub_stamp":"2024-03-28T00:23:54.693629+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T09:01:45.401356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.554158+0000","last_change":"2024-03-28T00:23:56.766973+0000","last_active":"2024-03-28T00:24:05.554158+0000","last_peered":"2024-03-28T00:24:05.554158+0000","last_clean":"2024-03-28T00:24:05.554158+0000","last_became_active":"2024-03-28T00:23:56.766549+0000","last_became_peered":"2024-03-28T00:23:56.766549+0000","last_unstale":"2024-03-28T00:24:05.554158+0000","last_undegraded":"2024-03-28T00:24:05.554158+0000","last_fullsized":"2024-03-28T00:24:05.554158+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T02:37:52.330879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.582622+0000","last_change":"2024-03-28T00:23:55.728936+0000","last_active":"2024-03-28T00:24:05.582622+0000","last_peered":"2024-03-28T00:24:05.582622+0000","last_clean":"2024-03-28T00:24:05.582622+0000","last_became_active":"2024-03-28T00:23:55.728517+0000","last_became_peered":"2024-03-28T00:23:55.728517+0000","last_unstale":"2024-03-28T00:24:05.582622+0000","last_undegraded":"2024-03-28T00:24:05.582622+0000","last_fullsized":"2024-03-28T00:24:05.582622+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_clean_scrub_stamp":"2024-03-28T00:23:54.693629+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T10:13:20.841286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.534504+0000","last_change":"2024-03-28T00:23:56.738438+0000","last_active":"2024-03-28T00:24:05.534504+0000","last_peered":"2024-03-28T00:24:05.534504+0000","last_clean":"2024-03-28T00:24:05.534504+0000","last_became_active":"2024-03-28T00:23:56.738092+0000","last_became_peered":"2024-03-28T00:23:56.738092+0000","last_unstale":"2024-03-28T00:24:05.534504+0000","last_undegraded":"2024-03-28T00:24:05.534504+0000","last_fullsized":"2024-03-28T00:24:05.534504+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T02:54:57.424857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"85'7","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.973138+0000","last_change":"2024-03-28T00:23:55.741042+0000","last_active":"2024-03-28T00:24:05.973138+0000","last_peered":"2024-03-28T00:24:05.973138+0000","last_clean":"2024-03-28T00:24:05.973138+0000","last_became_active":"2024-03-28T00:23:55.740165+0000","last_became_peered":"2024-03-28T00:23:55.740165+0000","last_unstale":"2024-03-28T00:24:05.973138+0000","last_undegraded":"2024-03-28T00:24:05.973138+0000","last_fullsized":"2024-03-28T00:24:05.973138+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_clean_scrub_stamp":"2024-03-28T00:23:54.693629+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T04:47:58.519326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.536221+0000","last_change":"2024-03-28T00:23:56.760261+0000","last_active":"2024-03-28T00:24:05.536221+0000","last_peered":"2024-03-28T00:24:05.536221+0000","last_clean":"2024-03-28T00:24:05.536221+0000","last_became_active":"2024-03-28T00:23:56.759925+0000","last_became_peered":"2024-03-28T00:23:56.759925+0000","last_unstale":"2024-03-28T00:24:05.536221+0000","last_undegraded":"2024-03-28T00:24:05.536221+0000","last_fullsized":"2024-03-28T00:24:05.536221+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T00:58:03.546582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.582758+0000","last_change":"2024-03-28T00:23:55.733455+0000","last_active":"2024-03-28T00:24:05.582758+0000","last_peered":"2024-03-28T00:24:05.582758+0000","last_clean":"2024-03-28T00:24:05.582758+0000","last_became_active":"2024-03-28T00:23:55.733116+0000","last_became_peered":"2024-03-28T00:23:55.733116+0000","last_unstale":"2024-03-28T00:24:05.582758+0000","last_undegraded":"2024-03-28T00:24:05.582758+0000","last_fullsized":"2024-03-28T00:24:05.582758+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_clean_scrub_stamp":"2024-03-28T00:23:54.693629+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T07:43:41.301213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.973417+0000","last_change":"2024-03-28T00:23:56.766884+0000","last_active":"2024-03-28T00:24:05.973417+0000","last_peered":"2024-03-28T00:24:05.973417+0000","last_clean":"2024-03-28T00:24:05.973417+0000","last_became_active":"2024-03-28T00:23:56.766476+0000","last_became_peered":"2024-03-28T00:23:56.766476+0000","last_unstale":"2024-03-28T00:24:05.973417+0000","last_undegraded":"2024-03-28T00:24:05.973417+0000","last_fullsized":"2024-03-28T00:24:05.973417+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T08:01:13.280796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.534658+0000","last_change":"2024-03-28T00:23:55.728001+0000","last_active":"2024-03-28T00:24:05.534658+0000","last_peered":"2024-03-28T00:24:05.534658+0000","last_clean":"2024-03-28T00:24:05.534658+0000","last_became_active":"2024-03-28T00:23:55.727659+0000","last_became_peered":"2024-03-28T00:23:55.727659+0000","last_unstale":"2024-03-28T00:24:05.534658+0000","last_undegraded":"2024-03-28T00:24:05.534658+0000","last_fullsized":"2024-03-28T00:24:05.534658+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_clean_scrub_stamp":"2024-03-28T00:23:54.693629+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T05:59:09.165928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.535045+0000","last_change":"2024-03-28T00:23:56.740843+0000","last_active":"2024-03-28T00:24:05.535045+0000","last_peered":"2024-03-28T00:24:05.535045+0000","last_clean":"2024-03-28T00:24:05.535045+0000","last_became_active":"2024-03-28T00:23:56.740364+0000","last_became_peered":"2024-03-28T00:23:56.740364+0000","last_unstale":"2024-03-28T00:24:05.535045+0000","last_undegraded":"2024-03-28T00:24:05.535045+0000","last_fullsized":"2024-03-28T00:24:05.535045+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T10:44:10.040424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'6","reported_seq":109,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:25:24.553885+0000","last_change":"2024-03-28T00:23:55.745772+0000","last_active":"2024-03-28T00:25:24.553885+0000","last_peered":"2024-03-28T00:25:24.553885+0000","last_clean":"2024-03-28T00:25:24.553885+0000","last_became_active":"2024-03-28T00:23:55.745457+0000","last_became_peered":"2024-03-28T00:23:55.745457+0000","last_unstale":"2024-03-28T00:25:24.553885+0000","last_undegraded":"2024-03-28T00:25:24.553885+0000","last_fullsized":"2024-03-28T00:25:24.553885+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_clean_scrub_stamp":"2024-03-28T00:23:54.693629+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T01:27:44.632523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1561,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":84,"num_read_kb":2,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.535251+0000","last_change":"2024-03-28T00:23:56.737008+0000","last_active":"2024-03-28T00:24:05.535251+0000","last_peered":"2024-03-28T00:24:05.535251+0000","last_clean":"2024-03-28T00:24:05.535251+0000","last_became_active":"2024-03-28T00:23:56.736399+0000","last_became_peered":"2024-03-28T00:23:56.736399+0000","last_unstale":"2024-03-28T00:24:05.535251+0000","last_undegraded":"2024-03-28T00:24:05.535251+0000","last_fullsized":"2024-03-28T00:24:05.535251+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T07:16:05.968362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'3","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.538765+0000","last_change":"2024-03-28T00:23:55.734518+0000","last_active":"2024-03-28T00:24:05.538765+0000","last_peered":"2024-03-28T00:24:05.538765+0000","last_clean":"2024-03-28T00:24:05.538765+0000","last_became_active":"2024-03-28T00:23:55.734200+0000","last_became_peered":"2024-03-28T00:23:55.734200+0000","last_unstale":"2024-03-28T00:24:05.538765+0000","last_undegraded":"2024-03-28T00:24:05.538765+0000","last_fullsized":"2024-03-28T00:24:05.538765+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_clean_scrub_stamp":"2024-03-28T00:23:54.693629+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T05:00:56.602654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"86'1","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.535023+0000","last_change":"2024-03-28T00:23:56.757249+0000","last_active":"2024-03-28T00:24:05.535023+0000","last_peered":"2024-03-28T00:24:05.535023+0000","last_clean":"2024-03-28T00:24:05.535023+0000","last_became_active":"2024-03-28T00:23:56.756957+0000","last_became_peered":"2024-03-28T00:23:56.756957+0000","last_unstale":"2024-03-28T00:24:05.535023+0000","last_undegraded":"2024-03-28T00:24:05.535023+0000","last_fullsized":"2024-03-28T00:24:05.535023+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T03:55:30.166369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'5","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.538430+0000","last_change":"2024-03-28T00:23:55.741463+0000","last_active":"2024-03-28T00:24:05.538430+0000","last_peered":"2024-03-28T00:24:05.538430+0000","last_clean":"2024-03-28T00:24:05.538430+0000","last_became_active":"2024-03-28T00:23:55.741095+0000","last_became_peered":"2024-03-28T00:23:55.741095+0000","last_unstale":"2024-03-28T00:24:05.538430+0000","last_undegraded":"2024-03-28T00:24:05.538430+0000","last_fullsized":"2024-03-28T00:24:05.538430+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:54.693629+0000","last_clean_scrub_stamp":"2024-03-28T00:23:54.693629+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T02:58:49.587896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.535208+0000","last_change":"2024-03-28T00:23:56.736382+0000","last_active":"2024-03-28T00:24:05.535208+0000","last_peered":"2024-03-28T00:24:05.535208+0000","last_clean":"2024-03-28T00:24:05.535208+0000","last_became_active":"2024-03-28T00:23:56.736036+0000","last_became_peered":"2024-03-28T00:23:56.736036+0000","last_unstale":"2024-03-28T00:24:05.535208+0000","last_undegraded":"2024-03-28T00:24:05.535208+0000","last_fullsized":"2024-03-28T00:24:05.535208+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T04:26:01.792967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.583909+0000","last_change":"2024-03-28T00:24:05.583909+0000","last_active":"2024-03-28T00:24:05.583909+0000","last_peered":"2024-03-28T00:24:05.583909+0000","last_clean":"2024-03-28T00:24:05.583909+0000","last_became_active":"2024-03-28T00:24:05.583502+0000","last_became_peered":"2024-03-28T00:24:05.583502+0000","last_unstale":"2024-03-28T00:24:05.583909+0000","last_undegraded":"2024-03-28T00:24:05.583909+0000","last_fullsized":"2024-03-28T00:24:05.583909+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T12:15:24.378306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,9],"acting":[11,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.582428+0000","last_change":"2024-03-28T00:23:56.736044+0000","last_active":"2024-03-28T00:24:05.582428+0000","last_peered":"2024-03-28T00:24:05.582428+0000","last_clean":"2024-03-28T00:24:05.582428+0000","last_became_active":"2024-03-28T00:23:56.735688+0000","last_became_peered":"2024-03-28T00:23:56.735688+0000","last_unstale":"2024-03-28T00:24:05.582428+0000","last_undegraded":"2024-03-28T00:24:05.582428+0000","last_fullsized":"2024-03-28T00:24:05.582428+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T08:57:16.687620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.582399+0000","last_change":"2024-03-28T00:23:56.760935+0000","last_active":"2024-03-28T00:24:05.582399+0000","last_peered":"2024-03-28T00:24:05.582399+0000","last_clean":"2024-03-28T00:24:05.582399+0000","last_became_active":"2024-03-28T00:23:56.760600+0000","last_became_peered":"2024-03-28T00:23:56.760600+0000","last_unstale":"2024-03-28T00:24:05.582399+0000","last_undegraded":"2024-03-28T00:24:05.582399+0000","last_fullsized":"2024-03-28T00:24:05.582399+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T03:44:30.047881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.553794+0000","last_change":"2024-03-28T00:23:56.760767+0000","last_active":"2024-03-28T00:24:05.553794+0000","last_peered":"2024-03-28T00:24:05.553794+0000","last_clean":"2024-03-28T00:24:05.553794+0000","last_became_active":"2024-03-28T00:23:56.760431+0000","last_became_peered":"2024-03-28T00:23:56.760431+0000","last_unstale":"2024-03-28T00:24:05.553794+0000","last_undegraded":"2024-03-28T00:24:05.553794+0000","last_fullsized":"2024-03-28T00:24:05.553794+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T01:53:43.479645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.560249+0000","last_change":"2024-03-28T00:24:05.560249+0000","last_active":"2024-03-28T00:24:05.560249+0000","last_peered":"2024-03-28T00:24:05.560249+0000","last_clean":"2024-03-28T00:24:05.560249+0000","last_became_active":"2024-03-28T00:24:05.558901+0000","last_became_peered":"2024-03-28T00:24:05.558901+0000","last_unstale":"2024-03-28T00:24:05.560249+0000","last_undegraded":"2024-03-28T00:24:05.560249+0000","last_fullsized":"2024-03-28T00:24:05.560249+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T05:31:16.782355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.536385+0000","last_change":"2024-03-28T00:23:56.740004+0000","last_active":"2024-03-28T00:24:05.536385+0000","last_peered":"2024-03-28T00:24:05.536385+0000","last_clean":"2024-03-28T00:24:05.536385+0000","last_became_active":"2024-03-28T00:23:56.739630+0000","last_became_peered":"2024-03-28T00:23:56.739630+0000","last_unstale":"2024-03-28T00:24:05.536385+0000","last_undegraded":"2024-03-28T00:24:05.536385+0000","last_fullsized":"2024-03-28T00:24:05.536385+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T01:04:39.297383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.583124+0000","last_change":"2024-03-28T00:23:56.740716+0000","last_active":"2024-03-28T00:24:05.583124+0000","last_peered":"2024-03-28T00:24:05.583124+0000","last_clean":"2024-03-28T00:24:05.583124+0000","last_became_active":"2024-03-28T00:23:56.739587+0000","last_became_peered":"2024-03-28T00:23:56.739587+0000","last_unstale":"2024-03-28T00:24:05.583124+0000","last_undegraded":"2024-03-28T00:24:05.583124+0000","last_fullsized":"2024-03-28T00:24:05.583124+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T11:17:37.115216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.974165+0000","last_change":"2024-03-28T00:23:56.761001+0000","last_active":"2024-03-28T00:24:05.974165+0000","last_peered":"2024-03-28T00:24:05.974165+0000","last_clean":"2024-03-28T00:24:05.974165+0000","last_became_active":"2024-03-28T00:23:56.757373+0000","last_became_peered":"2024-03-28T00:23:56.757373+0000","last_unstale":"2024-03-28T00:24:05.974165+0000","last_undegraded":"2024-03-28T00:24:05.974165+0000","last_fullsized":"2024-03-28T00:24:05.974165+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T02:55:47.106100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.538064+0000","last_change":"2024-03-28T00:23:56.750715+0000","last_active":"2024-03-28T00:24:05.538064+0000","last_peered":"2024-03-28T00:24:05.538064+0000","last_clean":"2024-03-28T00:24:05.538064+0000","last_became_active":"2024-03-28T00:23:56.750446+0000","last_became_peered":"2024-03-28T00:23:56.750446+0000","last_unstale":"2024-03-28T00:24:05.538064+0000","last_undegraded":"2024-03-28T00:24:05.538064+0000","last_fullsized":"2024-03-28T00:24:05.538064+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T11:51:27.935181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.973070+0000","last_change":"2024-03-28T00:23:56.752571+0000","last_active":"2024-03-28T00:24:05.973070+0000","last_peered":"2024-03-28T00:24:05.973070+0000","last_clean":"2024-03-28T00:24:05.973070+0000","last_became_active":"2024-03-28T00:23:56.751853+0000","last_became_peered":"2024-03-28T00:23:56.751853+0000","last_unstale":"2024-03-28T00:24:05.973070+0000","last_undegraded":"2024-03-28T00:24:05.973070+0000","last_fullsized":"2024-03-28T00:24:05.973070+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T05:30:02.897139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.582681+0000","last_change":"2024-03-28T00:23:56.750006+0000","last_active":"2024-03-28T00:24:05.582681+0000","last_peered":"2024-03-28T00:24:05.582681+0000","last_clean":"2024-03-28T00:24:05.582681+0000","last_became_active":"2024-03-28T00:23:56.749639+0000","last_became_peered":"2024-03-28T00:23:56.749639+0000","last_unstale":"2024-03-28T00:24:05.582681+0000","last_undegraded":"2024-03-28T00:24:05.582681+0000","last_fullsized":"2024-03-28T00:24:05.582681+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T02:26:51.076107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.538700+0000","last_change":"2024-03-28T00:23:56.764982+0000","last_active":"2024-03-28T00:24:05.538700+0000","last_peered":"2024-03-28T00:24:05.538700+0000","last_clean":"2024-03-28T00:24:05.538700+0000","last_became_active":"2024-03-28T00:23:56.764596+0000","last_became_peered":"2024-03-28T00:23:56.764596+0000","last_unstale":"2024-03-28T00:24:05.538700+0000","last_undegraded":"2024-03-28T00:24:05.538700+0000","last_fullsized":"2024-03-28T00:24:05.538700+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T09:05:30.948689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-28T00:24:05.536382+0000","last_change":"2024-03-28T00:23:56.736439+0000","last_active":"2024-03-28T00:24:05.536382+0000","last_peered":"2024-03-28T00:24:05.536382+0000","last_clean":"2024-03-28T00:24:05.536382+0000","last_became_active":"2024-03-28T00:23:56.736099+0000","last_became_peered":"2024-03-28T00:23:56.736099+0000","last_unstale":"2024-03-28T00:24:05.536382+0000","last_undegraded":"2024-03-28T00:24:05.536382+0000","last_fullsized":"2024-03-28T00:24:05.536382+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T00:23:55.703575+0000","last_clean_scrub_stamp":"2024-03-28T00:23:55.703575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T02:57:42.821566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":4132576,"data_compressed":245983,"data_compressed_allocated":315392,"data_compressed_original":4079616,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":71997,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":873,"num_read_kb":513,"num_write":240,"num_write_kb":137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":215991,"data_compressed":5124,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":209,"ondisk_log_size":209,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383714,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32116,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713036,"statfs":{"total":95995035648,"available":95962148864,"internally_reserved":0,"allocated":954368,"data_stored":456993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645350,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28096,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717056,"statfs":{"total":95995035648,"available":95966265344,"internally_reserved":0,"allocated":1105920,"data_stored":1112402,"data_compressed":37553,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808875,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32128,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713024,"statfs":{"total":95995035648,"available":95962136576,"internally_reserved":0,"allocated":974848,"data_stored":456510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070515,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32120,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713032,"statfs":{"total":95995035648,"available":95962144768,"internally_reserved":0,"allocated":966656,"data_stored":456866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332150,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1024000,"data_stored":1049797,"data_compressed":35845,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495677,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32120,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713032,"statfs":{"total":95995035648,"available":95962144768,"internally_reserved":0,"allocated":958464,"data_stored":456227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574937,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32172,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712980,"statfs":{"total":95995035648,"available":95962091520,"internally_reserved":0,"allocated":1019904,"data_stored":521542,"data_compressed":1708,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738462,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":978944,"data_stored":457957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411412,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36352,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708800,"statfs":{"total":95995035648,"available":95957811200,"internally_reserved":0,"allocated":1040384,"data_stored":522544,"data_compressed":1708,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182476,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":974848,"data_stored":456052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920841,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32120,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713032,"statfs":{"total":95995035648,"available":95962144768,"internally_reserved":0,"allocated":958464,"data_stored":456479,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757314,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1036288,"data_stored":1049084,"data_compressed":35845,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34201,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34201,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34201,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35845,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35845,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35845,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35845,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":66154,"data_compressed":1708,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":67156,"data_compressed":1708,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":66529,"data_compressed":1708,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-28T00:25:30.908 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-03-28T00:25:30.908 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-03-28T00:25:30.908 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-03-28T00:25:30.908 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 10f73b92-ec98-11ee-95d0-87774f69a715 -- ceph health --format=json 2024-03-28T00:25:31.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:25:31 smithi119 ceph-mon[31518]: pgmap v360: 97 pgs: 97 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-28T00:25:31.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:25:31 smithi119 ceph-mon[31518]: from='client.24895 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-28T00:25:31.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:25:31 smithi165 ceph-mon[36302]: pgmap v360: 97 pgs: 97 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-28T00:25:31.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:25:31 smithi165 ceph-mon[36302]: from='client.24895 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-28T00:25:31.417 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/mon.a/config 2024-03-28T00:25:31.452 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:25:31 smithi132 ceph-mon[35941]: pgmap v360: 97 pgs: 97 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-28T00:25:31.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:25:31 smithi132 ceph-mon[35941]: from='client.24895 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-28T00:25:32.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:25:32 smithi165 ceph-mon[36302]: pgmap v361: 97 pgs: 97 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T00:25:32.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:25:32 smithi119 ceph-mon[31518]: pgmap v361: 97 pgs: 97 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T00:25:32.452 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:25:32 smithi132 ceph-mon[35941]: pgmap v361: 97 pgs: 97 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T00:25:33.575 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:33.568+0000 7fa62cc00700 1 -- 172.21.15.119:0/2020925033 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa628074ee0 msgr2=0x7fa62810e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:33.575 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:33.568+0000 7fa62cc00700 1 --2- 172.21.15.119:0/2020925033 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa628074ee0 0x7fa62810e370 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fa610009990 tx=0x7fa610009ca0 comp rx=0 tx=0).stop 2024-03-28T00:25:33.575 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:33.569+0000 7fa62cc00700 1 -- 172.21.15.119:0/2020925033 shutdown_connections 2024-03-28T00:25:33.575 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:33.569+0000 7fa62cc00700 1 --2- 172.21.15.119:0/2020925033 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa62810e8b0 0x7fa628110d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:33.576 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:33.569+0000 7fa62cc00700 1 --2- 172.21.15.119:0/2020925033 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa6280745c0 0x7fa6280749a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:33.576 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:33.569+0000 7fa62cc00700 1 --2- 172.21.15.119:0/2020925033 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa628074ee0 0x7fa62810e370 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:33.576 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:33.569+0000 7fa62cc00700 1 -- 172.21.15.119:0/2020925033 >> 172.21.15.119:0/2020925033 conn(0x7fa6280fdb60 msgr2=0x7fa6280fff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:25:33.576 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:33.569+0000 7fa62cc00700 1 -- 172.21.15.119:0/2020925033 shutdown_connections 2024-03-28T00:25:33.576 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:33.569+0000 7fa62cc00700 1 -- 172.21.15.119:0/2020925033 wait complete. 2024-03-28T00:25:33.576 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:33.570+0000 7fa62cc00700 1 Processor -- start 2024-03-28T00:25:33.577 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:33.571+0000 7fa62cc00700 1 -- start start 2024-03-28T00:25:33.578 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:33.571+0000 7fa62cc00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa6280745c0 0x7fa6281a06f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:33.578 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:33.571+0000 7fa62cc00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa628074ee0 0x7fa6281a0c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:33.578 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:33.571+0000 7fa62cc00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa62810e8b0 0x7fa62819a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:33.578 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:33.571+0000 7fa62cc00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6281145e0 con 0x7fa628074ee0 2024-03-28T00:25:33.578 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:33.571+0000 7fa62cc00700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa628114460 con 0x7fa62810e8b0 2024-03-28T00:25:33.578 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:33.571+0000 7fa62cc00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa628114760 con 0x7fa6280745c0 2024-03-28T00:25:33.578 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:33.571+0000 7fa627400700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa6280745c0 0x7fa6281a06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:33.578 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:33.571+0000 7fa627400700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa6280745c0 0x7fa6281a06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.119:51410/0 (socket says 172.21.15.119:51410) 2024-03-28T00:25:33.578 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:33.571+0000 7fa627400700 1 -- 172.21.15.119:0/291885840 learned_addr learned my addr 172.21.15.119:0/291885840 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:25:33.578 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:33.572+0000 7fa61ea00700 1 --2- 172.21.15.119:0/291885840 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa628074ee0 0x7fa6281a0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:33.578 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:33.572+0000 7fa627e00700 1 --2- 172.21.15.119:0/291885840 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa62810e8b0 0x7fa62819a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:33.579 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:33.572+0000 7fa627400700 1 -- 172.21.15.119:0/291885840 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa62810e8b0 msgr2=0x7fa62819a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:33.579 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:33.572+0000 7fa627400700 1 --2- 172.21.15.119:0/291885840 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa62810e8b0 0x7fa62819a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:33.580 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:33.572+0000 7fa627400700 1 -- 172.21.15.119:0/291885840 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa628074ee0 msgr2=0x7fa6281a0c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:33.580 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:33.572+0000 7fa627400700 1 --2- 172.21.15.119:0/291885840 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa628074ee0 0x7fa6281a0c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:33.580 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:33.572+0000 7fa627400700 1 -- 172.21.15.119:0/291885840 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa62819b060 con 0x7fa6280745c0 2024-03-28T00:25:33.580 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:33.572+0000 7fa627e00700 1 --2- 172.21.15.119:0/291885840 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa62810e8b0 0x7fa62819a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T00:25:33.580 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:33.572+0000 7fa627400700 1 --2- 172.21.15.119:0/291885840 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa6280745c0 0x7fa6281a06f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa61800ef10 tx=0x7fa61800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:25:33.580 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:33.572+0000 7fa624c00700 1 -- 172.21.15.119:0/291885840 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa61800c9c0 con 0x7fa6280745c0 2024-03-28T00:25:33.580 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:33.573+0000 7fa624c00700 1 -- 172.21.15.119:0/291885840 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa618019070 con 0x7fa6280745c0 2024-03-28T00:25:33.580 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:33.573+0000 7fa62cc00700 1 -- 172.21.15.119:0/291885840 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa62819b2f0 con 0x7fa6280745c0 2024-03-28T00:25:33.580 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:33.573+0000 7fa624c00700 1 -- 172.21.15.119:0/291885840 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa618007e60 con 0x7fa6280745c0 2024-03-28T00:25:33.580 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:33.573+0000 7fa62cc00700 1 -- 172.21.15.119:0/291885840 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6281a75c0 con 0x7fa6280745c0 2024-03-28T00:25:33.580 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:33.574+0000 7fa62cc00700 1 -- 172.21.15.119:0/291885840 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa628068e50 con 0x7fa6280745c0 2024-03-28T00:25:33.582 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:33.576+0000 7fa624c00700 1 -- 172.21.15.119:0/291885840 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 17) v1 ==== 137342+0+0 (secure 0 0 0) 0x7fa618012050 con 0x7fa6280745c0 2024-03-28T00:25:33.583 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:33.577+0000 7fa624c00700 1 --2- 172.21.15.119:0/291885840 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fa60c0a2920 0x7fa60c0a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:33.584 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:33.578+0000 7fa624c00700 1 -- 172.21.15.119:0/291885840 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fa6180cf200 con 0x7fa6280745c0 2024-03-28T00:25:33.584 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:33.578+0000 7fa61ea00700 1 --2- 172.21.15.119:0/291885840 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fa60c0a2920 0x7fa60c0a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:33.584 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:33.578+0000 7fa61ea00700 1 --2- 172.21.15.119:0/291885840 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fa60c0a2920 0x7fa60c0a4de0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fa62810fbb0 tx=0x7fa610005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:25:33.590 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:33.584+0000 7fa624c00700 1 -- 172.21.15.119:0/291885840 <== mon.1 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa618010040 con 0x7fa6280745c0 2024-03-28T00:25:33.895 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:33.888+0000 7fa62cc00700 1 -- 172.21.15.119:0/291885840 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fa62802da30 con 0x7fa6280745c0 2024-03-28T00:25:33.895 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:33.889+0000 7fa624c00700 1 -- 172.21.15.119:0/291885840 <== mon.1 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fa61809c140 con 0x7fa6280745c0 2024-03-28T00:25:33.896 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-28T00:25:33.896 INFO:teuthology.orchestra.run.smithi119.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-03-28T00:25:33.899 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:33.893+0000 7fa62cc00700 1 -- 172.21.15.119:0/291885840 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fa60c0a2920 msgr2=0x7fa60c0a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:33.899 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:33.893+0000 7fa62cc00700 1 --2- 172.21.15.119:0/291885840 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fa60c0a2920 0x7fa60c0a4de0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fa62810fbb0 tx=0x7fa610005fd0 comp rx=0 tx=0).stop 2024-03-28T00:25:33.899 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:33.893+0000 7fa62cc00700 1 -- 172.21.15.119:0/291885840 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa6280745c0 msgr2=0x7fa6281a06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:33.899 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:33.893+0000 7fa62cc00700 1 --2- 172.21.15.119:0/291885840 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa6280745c0 0x7fa6281a06f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa61800ef10 tx=0x7fa61800c3a0 comp rx=0 tx=0).stop 2024-03-28T00:25:33.900 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:33.893+0000 7fa62cc00700 1 -- 172.21.15.119:0/291885840 shutdown_connections 2024-03-28T00:25:33.900 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:33.893+0000 7fa62cc00700 1 --2- 172.21.15.119:0/291885840 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fa60c0a2920 0x7fa60c0a4de0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:33.900 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:33.893+0000 7fa62cc00700 1 --2- 172.21.15.119:0/291885840 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa62810e8b0 0x7fa62819a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:33.900 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:33.893+0000 7fa62cc00700 1 --2- 172.21.15.119:0/291885840 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa6280745c0 0x7fa6281a06f0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:33.900 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:33.893+0000 7fa62cc00700 1 --2- 172.21.15.119:0/291885840 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa628074ee0 0x7fa6281a0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:33.900 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:33.893+0000 7fa62cc00700 1 -- 172.21.15.119:0/291885840 >> 172.21.15.119:0/291885840 conn(0x7fa6280fdb60 msgr2=0x7fa6280fe870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:25:33.900 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:33.893+0000 7fa62cc00700 1 -- 172.21.15.119:0/291885840 shutdown_connections 2024-03-28T00:25:33.900 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:33.893+0000 7fa62cc00700 1 -- 172.21.15.119:0/291885840 wait complete. 2024-03-28T00:25:34.837 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-03-28T00:25:34.838 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T00:25:35.041 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:35.034+0000 7f58d7800640 1 -- 172.21.15.119:0/1646019905 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f58d0103260 msgr2=0x7f58d0109af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:35.041 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:35.034+0000 7f58d7800640 1 --2- 172.21.15.119:0/1646019905 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f58d0103260 0x7f58d0109af0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f58c8009ca0 tx=0x7f58c802c100 comp rx=0 tx=0).stop 2024-03-28T00:25:35.041 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:35.034+0000 7f58d7800640 1 -- 172.21.15.119:0/1646019905 shutdown_connections 2024-03-28T00:25:35.041 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:35.034+0000 7f58d7800640 1 --2- 172.21.15.119:0/1646019905 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f58d0103260 0x7f58d0109af0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:35.041 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:35.034+0000 7f58d7800640 1 --2- 172.21.15.119:0/1646019905 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f58d01028a0 0x7f58d0102d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:35.041 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:35.034+0000 7f58d7800640 1 --2- 172.21.15.119:0/1646019905 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58d01016b0 0x7f58d0101ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:35.042 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:35.034+0000 7f58d7800640 1 -- 172.21.15.119:0/1646019905 >> 172.21.15.119:0/1646019905 conn(0x7f58d00fcc80 msgr2=0x7f58d00ff0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:25:35.042 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:35.034+0000 7f58d7800640 1 -- 172.21.15.119:0/1646019905 shutdown_connections 2024-03-28T00:25:35.042 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:35.034+0000 7f58d7800640 1 -- 172.21.15.119:0/1646019905 wait complete. 2024-03-28T00:25:35.045 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:35.035+0000 7f58d7800640 1 Processor -- start 2024-03-28T00:25:35.045 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:35.035+0000 7f58d7800640 1 -- start start 2024-03-28T00:25:35.046 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:35.035+0000 7f58d7800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f58d01016b0 0x7f58d0199b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:35.046 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:35.035+0000 7f58d7800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f58d01028a0 0x7f58d019a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:35.046 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:35.035+0000 7f58d7800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58d0103260 0x7f58d01a1120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:35.046 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:35.035+0000 7f58d7800640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58d010d310 con 0x7f58d01016b0 2024-03-28T00:25:35.046 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:35.035+0000 7f58d7800640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58d010d190 con 0x7f58d0103260 2024-03-28T00:25:35.046 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:35.035+0000 7f58d7800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58d010d490 con 0x7f58d01028a0 2024-03-28T00:25:35.046 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:35.036+0000 7f58cfe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f58d01028a0 0x7f58d019a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:35.046 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:35.036+0000 7f58d5800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58d0103260 0x7f58d01a1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:35.046 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:35.036+0000 7f58cfe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f58d01028a0 0x7f58d019a0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.119:55352/0 (socket says 172.21.15.119:55352) 2024-03-28T00:25:35.046 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:35.036+0000 7f58cfe00640 1 -- 172.21.15.119:0/741535644 learned_addr learned my addr 172.21.15.119:0/741535644 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:25:35.046 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:35.036+0000 7f58d4e00640 1 --2- 172.21.15.119:0/741535644 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f58d01016b0 0x7f58d0199b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:35.046 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:35.036+0000 7f58d5800640 1 -- 172.21.15.119:0/741535644 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f58d01028a0 msgr2=0x7f58d019a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:35.046 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:35.036+0000 7f58d5800640 1 --2- 172.21.15.119:0/741535644 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f58d01028a0 0x7f58d019a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:35.047 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:35.036+0000 7f58d5800640 1 -- 172.21.15.119:0/741535644 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f58d01016b0 msgr2=0x7f58d0199b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:35.047 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:35.036+0000 7f58d5800640 1 --2- 172.21.15.119:0/741535644 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f58d01016b0 0x7f58d0199b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:35.047 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:35.036+0000 7f58d5800640 1 -- 172.21.15.119:0/741535644 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58d01a1790 con 0x7f58d0103260 2024-03-28T00:25:35.047 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:35.036+0000 7f58cfe00640 1 --2- 172.21.15.119:0/741535644 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f58d01028a0 0x7f58d019a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T00:25:35.047 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:35.036+0000 7f58d5800640 1 --2- 172.21.15.119:0/741535644 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58d0103260 0x7f58d01a1120 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f58c8037040 tx=0x7f58c8035d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:25:35.047 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:35.037+0000 7f58cd600640 1 -- 172.21.15.119:0/741535644 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58c8039040 con 0x7f58d0103260 2024-03-28T00:25:35.047 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:35.037+0000 7f58cd600640 1 -- 172.21.15.119:0/741535644 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f58c8044070 con 0x7f58d0103260 2024-03-28T00:25:35.047 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:35.037+0000 7f58d7800640 1 -- 172.21.15.119:0/741535644 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58d01a1a20 con 0x7f58d0103260 2024-03-28T00:25:35.047 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:35.037+0000 7f58cd600640 1 -- 172.21.15.119:0/741535644 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58c8004350 con 0x7f58d0103260 2024-03-28T00:25:35.047 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:35.037+0000 7f58d7800640 1 -- 172.21.15.119:0/741535644 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58d01a1ee0 con 0x7f58d0103260 2024-03-28T00:25:35.047 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:35.038+0000 7f58d7800640 1 -- 172.21.15.119:0/741535644 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58d01a2110 con 0x7f58d0103260 2024-03-28T00:25:35.047 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:35.039+0000 7f58cd600640 1 -- 172.21.15.119:0/741535644 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 18) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f58c80044f0 con 0x7f58d0103260 2024-03-28T00:25:35.048 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:35.041+0000 7f58cd600640 1 --2- 172.21.15.119:0/741535644 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f58b80a2800 0x7f58b80a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:35.048 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:35.041+0000 7f58cd600640 1 -- 172.21.15.119:0/741535644 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f58c80f0f70 con 0x7f58d0103260 2024-03-28T00:25:35.048 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:35.041+0000 7f58d4e00640 1 --2- 172.21.15.119:0/741535644 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f58b80a2800 0x7f58b80a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:35.048 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:35.041+0000 7f58d4e00640 1 --2- 172.21.15.119:0/741535644 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f58b80a2800 0x7f58b80a4cc0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f58d0102700 tx=0x7f58c4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:25:35.055 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:35.049+0000 7f58cd600640 1 -- 172.21.15.119:0/741535644 <== mon.2 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f58c80be010 con 0x7f58d0103260 2024-03-28T00:25:35.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:25:34 smithi165 ceph-mon[36302]: pgmap v362: 97 pgs: 97 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-28T00:25:35.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:25:34 smithi165 ceph-mon[36302]: from='client.? 172.21.15.119:0/291885840' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-28T00:25:35.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:25:34 smithi119 ceph-mon[31518]: pgmap v362: 97 pgs: 97 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-28T00:25:35.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:25:34 smithi119 ceph-mon[31518]: from='client.? 172.21.15.119:0/291885840' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-28T00:25:35.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:25:34 smithi132 ceph-mon[35941]: pgmap v362: 97 pgs: 97 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-28T00:25:35.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:25:34 smithi132 ceph-mon[35941]: from='client.? 172.21.15.119:0/291885840' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-28T00:25:35.256 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:35.250+0000 7f58d7800640 1 -- 172.21.15.119:0/741535644 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f58d01a2890 con 0x7f58d0103260 2024-03-28T00:25:35.258 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:35.252+0000 7f58cd600640 1 -- 172.21.15.119:0/741535644 <== mon.2 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+22638 (secure 0 0 0) 0x7f58c80098b0 con 0x7f58d0103260 2024-03-28T00:25:35.259 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-28T00:25:35.259 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":87,"fsid":"10f73b92-ec98-11ee-95d0-87774f69a715","created":"2024-03-28T00:14:57.563604+0000","modified":"2024-03-28T00:24:05.529756+0000","last_up_change":"2024-03-28T00:22:58.821662+0000","last_in_change":"2024-03-28T00:22:38.467270+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T00:18:46.253414+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-28T00:23:54.261981+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-28T00:23:55.685836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c0b875d-a01e-4294-9c9c-5d0b048b8912","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":580396252},{"type":"v1","addr":"172.21.15.119:6803","nonce":580396252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":580396252},{"type":"v1","addr":"172.21.15.119:6805","nonce":580396252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":580396252},{"type":"v1","addr":"172.21.15.119:6809","nonce":580396252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":580396252},{"type":"v1","addr":"172.21.15.119:6807","nonce":580396252}]},"public_addr":"172.21.15.119:6803/580396252","cluster_addr":"172.21.15.119:6805/580396252","heartbeat_back_addr":"172.21.15.119:6809/580396252","heartbeat_front_addr":"172.21.15.119:6807/580396252","state":["exists","up"]},{"osd":1,"uuid":"5567f7c2-8ea4-4789-9ea1-467793324fb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":2500318888},{"type":"v1","addr":"172.21.15.132:6801","nonce":2500318888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":2500318888},{"type":"v1","addr":"172.21.15.132:6803","nonce":2500318888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":2500318888},{"type":"v1","addr":"172.21.15.132:6807","nonce":2500318888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":2500318888},{"type":"v1","addr":"172.21.15.132:6805","nonce":2500318888}]},"public_addr":"172.21.15.132:6801/2500318888","cluster_addr":"172.21.15.132:6803/2500318888","heartbeat_back_addr":"172.21.15.132:6807/2500318888","heartbeat_front_addr":"172.21.15.132:6805/2500318888","state":["exists","up"]},{"osd":2,"uuid":"ee54dbd0-24b4-4ee6-988c-ce71491199d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":2361434832},{"type":"v1","addr":"172.21.15.165:6801","nonce":2361434832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":2361434832},{"type":"v1","addr":"172.21.15.165:6803","nonce":2361434832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":2361434832},{"type":"v1","addr":"172.21.15.165:6807","nonce":2361434832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":2361434832},{"type":"v1","addr":"172.21.15.165:6805","nonce":2361434832}]},"public_addr":"172.21.15.165:6801/2361434832","cluster_addr":"172.21.15.165:6803/2361434832","heartbeat_back_addr":"172.21.15.165:6807/2361434832","heartbeat_front_addr":"172.21.15.165:6805/2361434832","state":["exists","up"]},{"osd":3,"uuid":"0464c40b-828f-4db4-96d9-3f7a6325e6da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":3732272210},{"type":"v1","addr":"172.21.15.119:6811","nonce":3732272210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":3732272210},{"type":"v1","addr":"172.21.15.119:6813","nonce":3732272210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":3732272210},{"type":"v1","addr":"172.21.15.119:6817","nonce":3732272210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":3732272210},{"type":"v1","addr":"172.21.15.119:6815","nonce":3732272210}]},"public_addr":"172.21.15.119:6811/3732272210","cluster_addr":"172.21.15.119:6813/3732272210","heartbeat_back_addr":"172.21.15.119:6817/3732272210","heartbeat_front_addr":"172.21.15.119:6815/3732272210","state":["exists","up"]},{"osd":4,"uuid":"8fd70b8e-5128-4173-aa0e-e735dfeb995d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":2803039505},{"type":"v1","addr":"172.21.15.132:6809","nonce":2803039505}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":2803039505},{"type":"v1","addr":"172.21.15.132:6811","nonce":2803039505}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":2803039505},{"type":"v1","addr":"172.21.15.132:6815","nonce":2803039505}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":2803039505},{"type":"v1","addr":"172.21.15.132:6813","nonce":2803039505}]},"public_addr":"172.21.15.132:6809/2803039505","cluster_addr":"172.21.15.132:6811/2803039505","heartbeat_back_addr":"172.21.15.132:6815/2803039505","heartbeat_front_addr":"172.21.15.132:6813/2803039505","state":["exists","up"]},{"osd":5,"uuid":"3224b4c1-733e-4424-a6da-870bbb5f94e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":2561252206},{"type":"v1","addr":"172.21.15.165:6809","nonce":2561252206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":2561252206},{"type":"v1","addr":"172.21.15.165:6811","nonce":2561252206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":2561252206},{"type":"v1","addr":"172.21.15.165:6815","nonce":2561252206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":2561252206},{"type":"v1","addr":"172.21.15.165:6813","nonce":2561252206}]},"public_addr":"172.21.15.165:6809/2561252206","cluster_addr":"172.21.15.165:6811/2561252206","heartbeat_back_addr":"172.21.15.165:6815/2561252206","heartbeat_front_addr":"172.21.15.165:6813/2561252206","state":["exists","up"]},{"osd":6,"uuid":"e435745d-59b5-439a-9091-0946ca0e9332","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":2545410230},{"type":"v1","addr":"172.21.15.119:6819","nonce":2545410230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":2545410230},{"type":"v1","addr":"172.21.15.119:6821","nonce":2545410230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":2545410230},{"type":"v1","addr":"172.21.15.119:6825","nonce":2545410230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":2545410230},{"type":"v1","addr":"172.21.15.119:6823","nonce":2545410230}]},"public_addr":"172.21.15.119:6819/2545410230","cluster_addr":"172.21.15.119:6821/2545410230","heartbeat_back_addr":"172.21.15.119:6825/2545410230","heartbeat_front_addr":"172.21.15.119:6823/2545410230","state":["exists","up"]},{"osd":7,"uuid":"79b97a12-f6b4-48c3-b517-2dd6b6e7c85f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":2877892376},{"type":"v1","addr":"172.21.15.132:6817","nonce":2877892376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":2877892376},{"type":"v1","addr":"172.21.15.132:6819","nonce":2877892376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":2877892376},{"type":"v1","addr":"172.21.15.132:6823","nonce":2877892376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":2877892376},{"type":"v1","addr":"172.21.15.132:6821","nonce":2877892376}]},"public_addr":"172.21.15.132:6817/2877892376","cluster_addr":"172.21.15.132:6819/2877892376","heartbeat_back_addr":"172.21.15.132:6823/2877892376","heartbeat_front_addr":"172.21.15.132:6821/2877892376","state":["exists","up"]},{"osd":8,"uuid":"2970524c-b99a-412d-85c2-6c29b54035e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":4115226150},{"type":"v1","addr":"172.21.15.165:6817","nonce":4115226150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":4115226150},{"type":"v1","addr":"172.21.15.165:6819","nonce":4115226150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":4115226150},{"type":"v1","addr":"172.21.15.165:6823","nonce":4115226150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":4115226150},{"type":"v1","addr":"172.21.15.165:6821","nonce":4115226150}]},"public_addr":"172.21.15.165:6817/4115226150","cluster_addr":"172.21.15.165:6819/4115226150","heartbeat_back_addr":"172.21.15.165:6823/4115226150","heartbeat_front_addr":"172.21.15.165:6821/4115226150","state":["exists","up"]},{"osd":9,"uuid":"2b1a08ed-cde9-4def-b3e8-838fff22709c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":2532245222},{"type":"v1","addr":"172.21.15.119:6827","nonce":2532245222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":2532245222},{"type":"v1","addr":"172.21.15.119:6829","nonce":2532245222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":2532245222},{"type":"v1","addr":"172.21.15.119:6833","nonce":2532245222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":2532245222},{"type":"v1","addr":"172.21.15.119:6831","nonce":2532245222}]},"public_addr":"172.21.15.119:6827/2532245222","cluster_addr":"172.21.15.119:6829/2532245222","heartbeat_back_addr":"172.21.15.119:6833/2532245222","heartbeat_front_addr":"172.21.15.119:6831/2532245222","state":["exists","up"]},{"osd":10,"uuid":"f019acd0-915a-4094-b12c-b53672740a5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":3454298492},{"type":"v1","addr":"172.21.15.132:6825","nonce":3454298492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":3454298492},{"type":"v1","addr":"172.21.15.132:6827","nonce":3454298492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":3454298492},{"type":"v1","addr":"172.21.15.132:6831","nonce":3454298492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":3454298492},{"type":"v1","addr":"172.21.15.132:6829","nonce":3454298492}]},"public_addr":"172.21.15.132:6825/3454298492","cluster_addr":"172.21.15.132:6827/3454298492","heartbeat_back_addr":"172.21.15.132:6831/3454298492","heartbeat_front_addr":"172.21.15.132:6829/3454298492","state":["exists","up"]},{"osd":11,"uuid":"2c1e59a8-8f21-45c0-8568-4b632ce020f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":2138771601},{"type":"v1","addr":"172.21.15.165:6825","nonce":2138771601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":2138771601},{"type":"v1","addr":"172.21.15.165:6827","nonce":2138771601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":2138771601},{"type":"v1","addr":"172.21.15.165:6831","nonce":2138771601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":2138771601},{"type":"v1","addr":"172.21.15.165:6829","nonce":2138771601}]},"public_addr":"172.21.15.165:6825/2138771601","cluster_addr":"172.21.15.165:6827/2138771601","heartbeat_back_addr":"172.21.15.165:6831/2138771601","heartbeat_front_addr":"172.21.15.165:6829/2138771601","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:17:54.791531+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:18:19.466638+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:18:43.435536+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:19:15.702685+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:19:41.767370+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:20:06.793003+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:20:39.952744+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:21:05.727262+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:21:30.735829+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:22:04.072142+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:22:30.894047+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:22:56.637728+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.119:6801/2575246519":"2024-03-29T00:16:03.582545+0000","172.21.15.119:6800/2575246519":"2024-03-29T00:16:03.582545+0000","172.21.15.119:0/87565965":"2024-03-29T00:16:03.582545+0000","172.21.15.119:6801/2056426384":"2024-03-29T00:15:30.472283+0000","172.21.15.119:0/1639109126":"2024-03-29T00:15:30.472283+0000","172.21.15.119:0/468478300":"2024-03-29T00:15:30.472283+0000","172.21.15.119:0/2891240357":"2024-03-29T00:15:30.472283+0000","172.21.15.119:0/4201699272":"2024-03-29T00:16:03.582545+0000","172.21.15.119:6800/2056426384":"2024-03-29T00:15:30.472283+0000","172.21.15.119:0/2635982178":"2024-03-29T00:16:03.582545+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T00:25:35.262 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:35.255+0000 7f58d7800640 1 -- 172.21.15.119:0/741535644 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f58b80a2800 msgr2=0x7f58b80a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:35.262 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:35.255+0000 7f58d7800640 1 --2- 172.21.15.119:0/741535644 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f58b80a2800 0x7f58b80a4cc0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f58d0102700 tx=0x7f58c4005f90 comp rx=0 tx=0).stop 2024-03-28T00:25:35.263 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:35.255+0000 7f58d7800640 1 -- 172.21.15.119:0/741535644 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58d0103260 msgr2=0x7f58d01a1120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:35.263 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:35.255+0000 7f58d7800640 1 --2- 172.21.15.119:0/741535644 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58d0103260 0x7f58d01a1120 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f58c8037040 tx=0x7f58c8035d10 comp rx=0 tx=0).stop 2024-03-28T00:25:35.263 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:35.256+0000 7f58d7800640 1 -- 172.21.15.119:0/741535644 shutdown_connections 2024-03-28T00:25:35.263 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:35.256+0000 7f58d7800640 1 --2- 172.21.15.119:0/741535644 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f58b80a2800 0x7f58b80a4cc0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:35.263 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:35.256+0000 7f58d7800640 1 --2- 172.21.15.119:0/741535644 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58d0103260 0x7f58d01a1120 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:35.263 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:35.256+0000 7f58d7800640 1 --2- 172.21.15.119:0/741535644 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f58d01028a0 0x7f58d019a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:35.263 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:35.256+0000 7f58d7800640 1 --2- 172.21.15.119:0/741535644 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f58d01016b0 0x7f58d0199b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:35.263 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:35.256+0000 7f58d7800640 1 -- 172.21.15.119:0/741535644 >> 172.21.15.119:0/741535644 conn(0x7f58d00fcc80 msgr2=0x7f58d00738d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:25:35.263 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:35.256+0000 7f58d7800640 1 -- 172.21.15.119:0/741535644 shutdown_connections 2024-03-28T00:25:35.263 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:35.256+0000 7f58d7800640 1 -- 172.21.15.119:0/741535644 wait complete. 2024-03-28T00:25:35.280 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-28T00:18:46.253414+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-28T00:23:54.261981+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-28T00:23:55.685836+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-28T00:25:35.281 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-28T00:25:35.494 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:35.488+0000 7ff9cde00640 1 -- 172.21.15.119:0/1726727345 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff9c8105240 msgr2=0x7ff9c81056c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:35.495 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:35.488+0000 7ff9cde00640 1 --2- 172.21.15.119:0/1726727345 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff9c8105240 0x7ff9c81056c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ff9c0009900 tx=0x7ff9c002bc30 comp rx=0 tx=0).stop 2024-03-28T00:25:35.495 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:35.488+0000 7ff9cde00640 1 -- 172.21.15.119:0/1726727345 shutdown_connections 2024-03-28T00:25:35.495 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:35.488+0000 7ff9cde00640 1 --2- 172.21.15.119:0/1726727345 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff9c8105240 0x7ff9c81056c0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:35.495 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:35.488+0000 7ff9cde00640 1 --2- 172.21.15.119:0/1726727345 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff9c8102890 0x7ff9c8104d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:35.495 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:35.488+0000 7ff9cde00640 1 --2- 172.21.15.119:0/1726727345 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff9c80fff20 0x7ff9c8102350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:35.495 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:35.488+0000 7ff9cde00640 1 -- 172.21.15.119:0/1726727345 >> 172.21.15.119:0/1726727345 conn(0x7ff9c80fbb40 msgr2=0x7ff9c80fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:25:35.495 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:35.488+0000 7ff9cde00640 1 -- 172.21.15.119:0/1726727345 shutdown_connections 2024-03-28T00:25:35.495 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:35.488+0000 7ff9cde00640 1 -- 172.21.15.119:0/1726727345 wait complete. 2024-03-28T00:25:35.497 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:35.489+0000 7ff9cde00640 1 Processor -- start 2024-03-28T00:25:35.497 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:35.489+0000 7ff9cde00640 1 -- start start 2024-03-28T00:25:35.497 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:35.489+0000 7ff9cde00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff9c80fff20 0x7ff9c819df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:35.497 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:35.489+0000 7ff9cde00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff9c8102890 0x7ff9c819e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:35.497 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:35.489+0000 7ff9cde00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff9c8105240 0x7ff9c81a5540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:35.498 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:35.489+0000 7ff9cde00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9c8113970 con 0x7ff9c8105240 2024-03-28T00:25:35.498 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:35.489+0000 7ff9cde00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9c81137f0 con 0x7ff9c80fff20 2024-03-28T00:25:35.498 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:35.489+0000 7ff9cde00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9c8113af0 con 0x7ff9c8102890 2024-03-28T00:25:35.498 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:35.490+0000 7ff9be000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff9c8102890 0x7ff9c819e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:35.498 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:35.490+0000 7ff9c6a00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff9c80fff20 0x7ff9c819df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:35.498 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:35.490+0000 7ff9be000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff9c8102890 0x7ff9c819e4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.119:55378/0 (socket says 172.21.15.119:55378) 2024-03-28T00:25:35.498 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:35.490+0000 7ff9be000640 1 -- 172.21.15.119:0/3280537819 learned_addr learned my addr 172.21.15.119:0/3280537819 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:25:35.498 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:35.490+0000 7ff9c7400640 1 --2- 172.21.15.119:0/3280537819 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff9c8105240 0x7ff9c81a5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:35.498 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:35.490+0000 7ff9be000640 1 -- 172.21.15.119:0/3280537819 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff9c80fff20 msgr2=0x7ff9c819df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:35.498 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:35.490+0000 7ff9be000640 1 --2- 172.21.15.119:0/3280537819 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff9c80fff20 0x7ff9c819df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:35.498 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:35.490+0000 7ff9be000640 1 -- 172.21.15.119:0/3280537819 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff9c8105240 msgr2=0x7ff9c81a5540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:35.498 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:35.490+0000 7ff9be000640 1 --2- 172.21.15.119:0/3280537819 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff9c8105240 0x7ff9c81a5540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:35.498 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:35.490+0000 7ff9be000640 1 -- 172.21.15.119:0/3280537819 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9c81a5bb0 con 0x7ff9c8102890 2024-03-28T00:25:35.499 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:35.490+0000 7ff9be000640 1 --2- 172.21.15.119:0/3280537819 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff9c8102890 0x7ff9c819e4d0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ff9b000e9f0 tx=0x7ff9b000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:25:35.499 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:35.490+0000 7ff9cca00640 1 -- 172.21.15.119:0/3280537819 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9b000cd80 con 0x7ff9c8102890 2024-03-28T00:25:35.499 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:35.491+0000 7ff9cca00640 1 -- 172.21.15.119:0/3280537819 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff9b0010910 con 0x7ff9c8102890 2024-03-28T00:25:35.499 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:35.491+0000 7ff9cde00640 1 -- 172.21.15.119:0/3280537819 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9c81a5ea0 con 0x7ff9c8102890 2024-03-28T00:25:35.499 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:35.491+0000 7ff9cca00640 1 -- 172.21.15.119:0/3280537819 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9b00188f0 con 0x7ff9c8102890 2024-03-28T00:25:35.499 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:35.491+0000 7ff9cde00640 1 -- 172.21.15.119:0/3280537819 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9c81a6360 con 0x7ff9c8102890 2024-03-28T00:25:35.499 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:35.492+0000 7ff9cde00640 1 -- 172.21.15.119:0/3280537819 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9c80fd630 con 0x7ff9c8102890 2024-03-28T00:25:35.500 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:35.494+0000 7ff9cca00640 1 -- 172.21.15.119:0/3280537819 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 18) v1 ==== 137291+0+0 (secure 0 0 0) 0x7ff9b0010ac0 con 0x7ff9c8102890 2024-03-28T00:25:35.501 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:35.495+0000 7ff9cca00640 1 --2- 172.21.15.119:0/3280537819 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7ff9ac0a2800 0x7ff9ac0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:35.501 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:35.495+0000 7ff9cca00640 1 -- 172.21.15.119:0/3280537819 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8153+0+0 (secure 0 0 0) 0x7ff9b00cef40 con 0x7ff9c8102890 2024-03-28T00:25:35.501 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:35.495+0000 7ff9c6a00640 1 --2- 172.21.15.119:0/3280537819 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7ff9ac0a2800 0x7ff9ac0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:35.502 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:35.496+0000 7ff9c6a00640 1 --2- 172.21.15.119:0/3280537819 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7ff9ac0a2800 0x7ff9ac0a4cc0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ff9b8005fd0 tx=0x7ff9b8009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:25:35.503 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:35.497+0000 7ff9cca00640 1 -- 172.21.15.119:0/3280537819 <== mon.1 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff9b009b990 con 0x7ff9c8102890 2024-03-28T00:25:35.705 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:35.699+0000 7ff9cde00640 1 -- 172.21.15.119:0/3280537819 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff9c802d110 con 0x7ff9c8102890 2024-03-28T00:25:35.706 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:35.700+0000 7ff9cca00640 1 -- 172.21.15.119:0/3280537819 <== mon.1 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff9b009fa30 con 0x7ff9c8102890 2024-03-28T00:25:35.706 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 1 2024-03-28T00:25:35.709 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:35.703+0000 7ff9cde00640 1 -- 172.21.15.119:0/3280537819 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7ff9ac0a2800 msgr2=0x7ff9ac0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:35.709 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:35.703+0000 7ff9cde00640 1 --2- 172.21.15.119:0/3280537819 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7ff9ac0a2800 0x7ff9ac0a4cc0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ff9b8005fd0 tx=0x7ff9b8009500 comp rx=0 tx=0).stop 2024-03-28T00:25:35.709 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:35.703+0000 7ff9cde00640 1 -- 172.21.15.119:0/3280537819 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff9c8102890 msgr2=0x7ff9c819e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:35.709 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:35.703+0000 7ff9cde00640 1 --2- 172.21.15.119:0/3280537819 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff9c8102890 0x7ff9c819e4d0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ff9b000e9f0 tx=0x7ff9b000eeb0 comp rx=0 tx=0).stop 2024-03-28T00:25:35.709 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:35.703+0000 7ff9cde00640 1 -- 172.21.15.119:0/3280537819 shutdown_connections 2024-03-28T00:25:35.709 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:35.703+0000 7ff9cde00640 1 --2- 172.21.15.119:0/3280537819 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7ff9ac0a2800 0x7ff9ac0a4cc0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:35.710 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:35.703+0000 7ff9cde00640 1 --2- 172.21.15.119:0/3280537819 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff9c8105240 0x7ff9c81a5540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:35.710 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:35.703+0000 7ff9cde00640 1 --2- 172.21.15.119:0/3280537819 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff9c8102890 0x7ff9c819e4d0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:35.710 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:35.703+0000 7ff9cde00640 1 --2- 172.21.15.119:0/3280537819 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff9c80fff20 0x7ff9c819df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:35.710 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:35.703+0000 7ff9cde00640 1 -- 172.21.15.119:0/3280537819 >> 172.21.15.119:0/3280537819 conn(0x7ff9c80fbb40 msgr2=0x7ff9c81008c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:25:35.710 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:35.703+0000 7ff9cde00640 1 -- 172.21.15.119:0/3280537819 shutdown_connections 2024-03-28T00:25:35.710 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:35.703+0000 7ff9cde00640 1 -- 172.21.15.119:0/3280537819 wait complete. 2024-03-28T00:25:35.730 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-28T00:25:35.949 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:35.942+0000 7f9092400640 1 -- 172.21.15.119:0/2591911437 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f908c104ad0 msgr2=0x7f908c104f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:35.949 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:35.942+0000 7f9092400640 1 --2- 172.21.15.119:0/2591911437 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f908c104ad0 0x7f908c104f50 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f9078009900 tx=0x7f907802bc30 comp rx=0 tx=0).stop 2024-03-28T00:25:35.950 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:35.942+0000 7f9092400640 1 -- 172.21.15.119:0/2591911437 shutdown_connections 2024-03-28T00:25:35.950 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:35.943+0000 7f9092400640 1 --2- 172.21.15.119:0/2591911437 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f908c105490 0x7f908c10bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:35.950 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:35.943+0000 7f9092400640 1 --2- 172.21.15.119:0/2591911437 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f908c104ad0 0x7f908c104f50 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:35.950 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:35.943+0000 7f9092400640 1 --2- 172.21.15.119:0/2591911437 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f908c1038e0 0x7f908c103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:35.950 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:35.943+0000 7f9092400640 1 -- 172.21.15.119:0/2591911437 >> 172.21.15.119:0/2591911437 conn(0x7f908c0fee50 msgr2=0x7f908c101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:25:35.950 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:35.943+0000 7f9092400640 1 -- 172.21.15.119:0/2591911437 shutdown_connections 2024-03-28T00:25:35.950 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:35.943+0000 7f9092400640 1 -- 172.21.15.119:0/2591911437 wait complete. 2024-03-28T00:25:35.952 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:35.943+0000 7f9092400640 1 Processor -- start 2024-03-28T00:25:35.952 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:35.944+0000 7f9092400640 1 -- start start 2024-03-28T00:25:35.952 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:35.944+0000 7f9092400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f908c1038e0 0x7f908c199c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:35.952 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:35.944+0000 7f9092400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f908c104ad0 0x7f908c19a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:35.953 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:35.944+0000 7f9092400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f908c105490 0x7f908c1a11b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:35.953 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:35.944+0000 7f9092400640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f908c10f5b0 con 0x7f908c105490 2024-03-28T00:25:35.953 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:35.944+0000 7f9092400640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f908c10f430 con 0x7f908c104ad0 2024-03-28T00:25:35.953 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:35.944+0000 7f9092400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f908c10f730 con 0x7f908c1038e0 2024-03-28T00:25:35.953 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:35.944+0000 7f908b400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f908c1038e0 0x7f908c199c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:35.953 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:35.944+0000 7f908aa00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f908c104ad0 0x7f908c19a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:35.953 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:35.944+0000 7f908aa00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f908c104ad0 0x7f908c19a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.119:45240/0 (socket says 172.21.15.119:45240) 2024-03-28T00:25:35.953 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:35.944+0000 7f908aa00640 1 -- 172.21.15.119:0/2407966401 learned_addr learned my addr 172.21.15.119:0/2407966401 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:25:35.953 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:35.944+0000 7f908b400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f908c1038e0 0x7f908c199c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.119:55390/0 (socket says 172.21.15.119:55390) 2024-03-28T00:25:35.953 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:35.944+0000 7f908be00640 1 --2- 172.21.15.119:0/2407966401 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f908c105490 0x7f908c1a11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:35.953 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:35.945+0000 7f908b400640 1 -- 172.21.15.119:0/2407966401 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f908c104ad0 msgr2=0x7f908c19a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:35.953 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:35.945+0000 7f908b400640 1 --2- 172.21.15.119:0/2407966401 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f908c104ad0 0x7f908c19a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:35.953 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:35.945+0000 7f908b400640 1 -- 172.21.15.119:0/2407966401 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f908c105490 msgr2=0x7f908c1a11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:35.953 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:35.945+0000 7f908b400640 1 --2- 172.21.15.119:0/2407966401 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f908c105490 0x7f908c1a11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:35.954 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:35.945+0000 7f908b400640 1 -- 172.21.15.119:0/2407966401 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f908c1a1850 con 0x7f908c1038e0 2024-03-28T00:25:35.954 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:35.945+0000 7f908aa00640 1 --2- 172.21.15.119:0/2407966401 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f908c104ad0 0x7f908c19a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T00:25:35.954 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:35.945+0000 7f908be00640 1 --2- 172.21.15.119:0/2407966401 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f908c105490 0x7f908c1a11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T00:25:35.954 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:35.945+0000 7f908b400640 1 --2- 172.21.15.119:0/2407966401 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f908c1038e0 0x7f908c199c40 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f907c00e9c0 tx=0x7f907c00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:25:35.954 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:35.945+0000 7f9083e00640 1 -- 172.21.15.119:0/2407966401 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f907c00ccc0 con 0x7f908c1038e0 2024-03-28T00:25:35.954 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:35.945+0000 7f9083e00640 1 -- 172.21.15.119:0/2407966401 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f907c0035f0 con 0x7f908c1038e0 2024-03-28T00:25:35.954 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:35.945+0000 7f9092400640 1 -- 172.21.15.119:0/2407966401 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f908c1a1b40 con 0x7f908c1038e0 2024-03-28T00:25:35.954 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:35.945+0000 7f9083e00640 1 -- 172.21.15.119:0/2407966401 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f907c00ce60 con 0x7f908c1038e0 2024-03-28T00:25:35.954 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:35.945+0000 7f9092400640 1 -- 172.21.15.119:0/2407966401 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f908c1a2090 con 0x7f908c1038e0 2024-03-28T00:25:35.954 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:35.946+0000 7f9092400640 1 -- 172.21.15.119:0/2407966401 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f908c10ac70 con 0x7f908c1038e0 2024-03-28T00:25:35.955 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:35.948+0000 7f9083e00640 1 -- 172.21.15.119:0/2407966401 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 18) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f907c0037a0 con 0x7f908c1038e0 2024-03-28T00:25:35.955 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:35.949+0000 7f9083e00640 1 --2- 172.21.15.119:0/2407966401 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f90700a2800 0x7f90700a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:35.955 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:35.949+0000 7f9083e00640 1 -- 172.21.15.119:0/2407966401 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f907c0cee30 con 0x7f908c1038e0 2024-03-28T00:25:35.955 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:35.949+0000 7f908aa00640 1 --2- 172.21.15.119:0/2407966401 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f90700a2800 0x7f90700a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:35.956 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:35.950+0000 7f908aa00640 1 --2- 172.21.15.119:0/2407966401 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f90700a2800 0x7f90700a4cc0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f907802c8e0 tx=0x7f907802df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:25:35.964 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:35.958+0000 7f9083e00640 1 -- 172.21.15.119:0/2407966401 <== mon.1 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f907c09bed0 con 0x7f908c1038e0 2024-03-28T00:25:36.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:25:35 smithi165 ceph-mon[36302]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-03-28T00:25:36.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:25:35 smithi165 ceph-mon[36302]: from='client.? 172.21.15.119:0/741535644' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-28T00:25:36.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:25:35 smithi165 ceph-mon[36302]: from='client.? 172.21.15.119:0/3280537819' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-28T00:25:36.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:25:35 smithi119 ceph-mon[31518]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-03-28T00:25:36.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:25:35 smithi119 ceph-mon[31518]: from='client.? 172.21.15.119:0/741535644' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-28T00:25:36.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:25:35 smithi119 ceph-mon[31518]: from='client.? 172.21.15.119:0/3280537819' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-28T00:25:36.166 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:36.160+0000 7f9092400640 1 -- 172.21.15.119:0/2407966401 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f908c0622a0 con 0x7f908c1038e0 2024-03-28T00:25:36.167 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:36.161+0000 7f9083e00640 1 -- 172.21.15.119:0/2407966401 <== mon.1 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) v1 ==== 104+0+11 (secure 0 0 0) 0x7f907c09b7d0 con 0x7f908c1038e0 2024-03-28T00:25:36.167 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 32 2024-03-28T00:25:36.170 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:36.164+0000 7f9092400640 1 -- 172.21.15.119:0/2407966401 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f90700a2800 msgr2=0x7f90700a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:36.171 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:36.164+0000 7f9092400640 1 --2- 172.21.15.119:0/2407966401 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f90700a2800 0x7f90700a4cc0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f907802c8e0 tx=0x7f907802df90 comp rx=0 tx=0).stop 2024-03-28T00:25:36.171 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:36.164+0000 7f9092400640 1 -- 172.21.15.119:0/2407966401 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f908c1038e0 msgr2=0x7f908c199c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:36.171 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:36.164+0000 7f9092400640 1 --2- 172.21.15.119:0/2407966401 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f908c1038e0 0x7f908c199c40 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f907c00e9c0 tx=0x7f907c00ee80 comp rx=0 tx=0).stop 2024-03-28T00:25:36.171 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:36.164+0000 7f9092400640 1 -- 172.21.15.119:0/2407966401 shutdown_connections 2024-03-28T00:25:36.171 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:36.164+0000 7f9092400640 1 --2- 172.21.15.119:0/2407966401 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f90700a2800 0x7f90700a4cc0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:36.171 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:36.164+0000 7f9092400640 1 --2- 172.21.15.119:0/2407966401 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f908c105490 0x7f908c1a11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:36.171 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:36.164+0000 7f9092400640 1 --2- 172.21.15.119:0/2407966401 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f908c104ad0 0x7f908c19a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:36.171 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:36.164+0000 7f9092400640 1 --2- 172.21.15.119:0/2407966401 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f908c1038e0 0x7f908c199c40 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:36.171 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:36.164+0000 7f9092400640 1 -- 172.21.15.119:0/2407966401 >> 172.21.15.119:0/2407966401 conn(0x7f908c0fee50 msgr2=0x7f908c107e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:25:36.171 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:36.164+0000 7f9092400640 1 -- 172.21.15.119:0/2407966401 shutdown_connections 2024-03-28T00:25:36.172 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:36.164+0000 7f9092400640 1 -- 172.21.15.119:0/2407966401 wait complete. 2024-03-28T00:25:36.190 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-28T00:25:36.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:25:35 smithi132 ceph-mon[35941]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-03-28T00:25:36.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:25:35 smithi132 ceph-mon[35941]: from='client.? 172.21.15.119:0/741535644' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-28T00:25:36.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:25:35 smithi132 ceph-mon[35941]: from='client.? 172.21.15.119:0/3280537819' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-28T00:25:36.399 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:36.393+0000 7f91c9c00640 1 -- 172.21.15.119:0/1943284144 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f91c40740c0 msgr2=0x7f91c4074540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:36.400 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:36.393+0000 7f91c9c00640 1 --2- 172.21.15.119:0/1943284144 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f91c40740c0 0x7f91c4074540 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f91b0009900 tx=0x7f91b002bc50 comp rx=0 tx=0).stop 2024-03-28T00:25:36.400 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:36.393+0000 7f91c9c00640 1 -- 172.21.15.119:0/1943284144 shutdown_connections 2024-03-28T00:25:36.400 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:36.393+0000 7f91c9c00640 1 --2- 172.21.15.119:0/1943284144 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f91c4109560 0x7f91c410b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:36.400 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:36.393+0000 7f91c9c00640 1 --2- 172.21.15.119:0/1943284144 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f91c40740c0 0x7f91c4074540 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:36.400 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:36.393+0000 7f91c9c00640 1 --2- 172.21.15.119:0/1943284144 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f91c40756a0 0x7f91c4073af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:36.400 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:36.393+0000 7f91c9c00640 1 -- 172.21.15.119:0/1943284144 >> 172.21.15.119:0/1943284144 conn(0x7f91c40fcc40 msgr2=0x7f91c40ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:25:36.400 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:36.393+0000 7f91c9c00640 1 -- 172.21.15.119:0/1943284144 shutdown_connections 2024-03-28T00:25:36.400 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:36.393+0000 7f91c9c00640 1 -- 172.21.15.119:0/1943284144 wait complete. 2024-03-28T00:25:36.403 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:36.394+0000 7f91c9c00640 1 Processor -- start 2024-03-28T00:25:36.403 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:36.394+0000 7f91c9c00640 1 -- start start 2024-03-28T00:25:36.403 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:36.395+0000 7f91c9c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f91c40740c0 0x7f91c4199bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:36.403 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:36.395+0000 7f91c9c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f91c40756a0 0x7f91c419a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:36.403 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:36.395+0000 7f91c9c00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f91c4109560 0x7f91c41a1180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:36.403 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:36.395+0000 7f91c9c00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91c410f540 con 0x7f91c40740c0 2024-03-28T00:25:36.403 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:36.395+0000 7f91c9c00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91c410f3c0 con 0x7f91c4109560 2024-03-28T00:25:36.403 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:36.395+0000 7f91c9c00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91c410f6c0 con 0x7f91c40756a0 2024-03-28T00:25:36.403 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:36.395+0000 7f91c2a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f91c40740c0 0x7f91c4199bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:36.403 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:36.395+0000 7f91c2a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f91c40740c0 0x7f91c4199bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:60044/0 (socket says 172.21.15.119:60044) 2024-03-28T00:25:36.404 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:36.395+0000 7f91c2a00640 1 -- 172.21.15.119:0/1963322927 learned_addr learned my addr 172.21.15.119:0/1963322927 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:25:36.404 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:36.395+0000 7f91c3400640 1 --2- 172.21.15.119:0/1963322927 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f91c4109560 0x7f91c41a1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:36.404 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:36.395+0000 7f91c2000640 1 --2- 172.21.15.119:0/1963322927 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f91c40756a0 0x7f91c419a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:36.404 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:36.395+0000 7f91c2a00640 1 -- 172.21.15.119:0/1963322927 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f91c40756a0 msgr2=0x7f91c419a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:36.404 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:36.395+0000 7f91c2a00640 1 --2- 172.21.15.119:0/1963322927 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f91c40756a0 0x7f91c419a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:36.404 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:36.395+0000 7f91c2a00640 1 -- 172.21.15.119:0/1963322927 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f91c4109560 msgr2=0x7f91c41a1180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:36.404 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:36.395+0000 7f91c2a00640 1 --2- 172.21.15.119:0/1963322927 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f91c4109560 0x7f91c41a1180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:36.404 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:36.395+0000 7f91c2a00640 1 -- 172.21.15.119:0/1963322927 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91c41a17f0 con 0x7f91c40740c0 2024-03-28T00:25:36.404 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:36.395+0000 7f91c3400640 1 --2- 172.21.15.119:0/1963322927 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f91c4109560 0x7f91c41a1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T00:25:36.404 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:36.396+0000 7f91c2a00640 1 --2- 172.21.15.119:0/1963322927 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f91c40740c0 0x7f91c4199bd0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f91bc009d50 tx=0x7f91bc00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:25:36.404 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:36.396+0000 7f91bb400640 1 -- 172.21.15.119:0/1963322927 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91bc003b70 con 0x7f91c40740c0 2024-03-28T00:25:36.404 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:36.396+0000 7f91bb400640 1 -- 172.21.15.119:0/1963322927 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f91bc003d10 con 0x7f91c40740c0 2024-03-28T00:25:36.404 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:36.396+0000 7f91c9c00640 1 -- 172.21.15.119:0/1963322927 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91c41a1ae0 con 0x7f91c40740c0 2024-03-28T00:25:36.404 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:36.396+0000 7f91bb400640 1 -- 172.21.15.119:0/1963322927 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91bc010a20 con 0x7f91c40740c0 2024-03-28T00:25:36.405 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:36.396+0000 7f91c9c00640 1 -- 172.21.15.119:0/1963322927 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91c41a2000 con 0x7f91c40740c0 2024-03-28T00:25:36.405 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:36.397+0000 7f91c9c00640 1 -- 172.21.15.119:0/1963322927 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91c41082e0 con 0x7f91c40740c0 2024-03-28T00:25:36.405 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:36.399+0000 7f91bb400640 1 -- 172.21.15.119:0/1963322927 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 18) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f91bc004000 con 0x7f91c40740c0 2024-03-28T00:25:36.406 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:36.400+0000 7f91bb400640 1 --2- 172.21.15.119:0/1963322927 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f91a80a2800 0x7f91a80a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:36.407 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:36.401+0000 7f91bb400640 1 -- 172.21.15.119:0/1963322927 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f91bc0cf0d0 con 0x7f91c40740c0 2024-03-28T00:25:36.407 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:36.401+0000 7f91c2000640 1 --2- 172.21.15.119:0/1963322927 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f91a80a2800 0x7f91a80a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:36.407 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:36.402+0000 7f91c2000640 1 --2- 172.21.15.119:0/1963322927 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f91a80a2800 0x7f91a80a4cc0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f91b002c260 tx=0x7f91b0000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:25:36.411 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:36.405+0000 7f91bb400640 1 -- 172.21.15.119:0/1963322927 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f91bc09c170 con 0x7f91c40740c0 2024-03-28T00:25:36.610 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:36.604+0000 7f91c9c00640 1 -- 172.21.15.119:0/1963322927 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f91c402d070 con 0x7f91c40740c0 2024-03-28T00:25:36.611 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:36.605+0000 7f91bb400640 1 -- 172.21.15.119:0/1963322927 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) v1 ==== 100+0+11 (secure 0 0 0) 0x7f91c402d070 con 0x7f91c40740c0 2024-03-28T00:25:36.612 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 64 2024-03-28T00:25:36.614 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:36.608+0000 7f91c9c00640 1 -- 172.21.15.119:0/1963322927 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f91a80a2800 msgr2=0x7f91a80a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:36.615 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:36.608+0000 7f91c9c00640 1 --2- 172.21.15.119:0/1963322927 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f91a80a2800 0x7f91a80a4cc0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f91b002c260 tx=0x7f91b0000bc0 comp rx=0 tx=0).stop 2024-03-28T00:25:36.615 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:36.608+0000 7f91c9c00640 1 -- 172.21.15.119:0/1963322927 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f91c40740c0 msgr2=0x7f91c4199bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:36.615 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:36.608+0000 7f91c9c00640 1 --2- 172.21.15.119:0/1963322927 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f91c40740c0 0x7f91c4199bd0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f91bc009d50 tx=0x7f91bc00c6a0 comp rx=0 tx=0).stop 2024-03-28T00:25:36.615 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:36.608+0000 7f91c9c00640 1 -- 172.21.15.119:0/1963322927 shutdown_connections 2024-03-28T00:25:36.615 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:36.608+0000 7f91c9c00640 1 --2- 172.21.15.119:0/1963322927 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f91a80a2800 0x7f91a80a4cc0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:36.615 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:36.608+0000 7f91c9c00640 1 --2- 172.21.15.119:0/1963322927 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f91c4109560 0x7f91c41a1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:36.615 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:36.608+0000 7f91c9c00640 1 --2- 172.21.15.119:0/1963322927 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f91c40756a0 0x7f91c419a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:36.615 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:36.608+0000 7f91c9c00640 1 --2- 172.21.15.119:0/1963322927 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f91c40740c0 0x7f91c4199bd0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:36.615 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:36.608+0000 7f91c9c00640 1 -- 172.21.15.119:0/1963322927 >> 172.21.15.119:0/1963322927 conn(0x7f91c40fcc40 msgr2=0x7f91c41054b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:25:36.615 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:36.608+0000 7f91c9c00640 1 -- 172.21.15.119:0/1963322927 shutdown_connections 2024-03-28T00:25:36.615 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:36.608+0000 7f91c9c00640 1 -- 172.21.15.119:0/1963322927 wait complete. 2024-03-28T00:25:36.633 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T00:25:36.846 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:36.839+0000 7f3947c00640 1 -- 172.21.15.119:0/1840329807 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f39401038e0 msgr2=0x7f3940103d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:36.846 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:36.839+0000 7f3947c00640 1 --2- 172.21.15.119:0/1840329807 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f39401038e0 0x7f3940103d00 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f3934009900 tx=0x7f393402bc50 comp rx=0 tx=0).stop 2024-03-28T00:25:36.846 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:36.840+0000 7f3947c00640 1 -- 172.21.15.119:0/1840329807 shutdown_connections 2024-03-28T00:25:36.846 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:36.840+0000 7f3947c00640 1 --2- 172.21.15.119:0/1840329807 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3940105490 0x7f394010bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:36.846 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:36.840+0000 7f3947c00640 1 --2- 172.21.15.119:0/1840329807 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3940104ad0 0x7f3940104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:36.846 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:36.840+0000 7f3947c00640 1 --2- 172.21.15.119:0/1840329807 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f39401038e0 0x7f3940103d00 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:36.846 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:36.840+0000 7f3947c00640 1 -- 172.21.15.119:0/1840329807 >> 172.21.15.119:0/1840329807 conn(0x7f39400fee50 msgr2=0x7f3940101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:25:36.846 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:36.840+0000 7f3947c00640 1 -- 172.21.15.119:0/1840329807 shutdown_connections 2024-03-28T00:25:36.847 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:36.840+0000 7f3947c00640 1 -- 172.21.15.119:0/1840329807 wait complete. 2024-03-28T00:25:36.848 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:36.841+0000 7f3947c00640 1 Processor -- start 2024-03-28T00:25:36.849 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:36.841+0000 7f3947c00640 1 -- start start 2024-03-28T00:25:36.849 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:36.841+0000 7f3947c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f39401038e0 0x7f3940199c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:36.849 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:36.841+0000 7f3947c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3940104ad0 0x7f394019a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:36.849 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:36.841+0000 7f3947c00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3940105490 0x7f39401a11d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:36.849 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:36.841+0000 7f3947c00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f394010f5a0 con 0x7f3940104ad0 2024-03-28T00:25:36.849 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:36.841+0000 7f3947c00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f394010f420 con 0x7f3940105490 2024-03-28T00:25:36.849 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:36.841+0000 7f3947c00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f394010f720 con 0x7f39401038e0 2024-03-28T00:25:36.849 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:36.841+0000 7f3945200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f39401038e0 0x7f3940199c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:36.849 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:36.841+0000 7f3945200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f39401038e0 0x7f3940199c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.119:55426/0 (socket says 172.21.15.119:55426) 2024-03-28T00:25:36.849 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:36.841+0000 7f3945200640 1 -- 172.21.15.119:0/2441959677 learned_addr learned my addr 172.21.15.119:0/2441959677 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:25:36.849 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:36.842+0000 7f3945c00640 1 --2- 172.21.15.119:0/2441959677 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3940105490 0x7f39401a11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:36.849 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:36.842+0000 7f393fe00640 1 --2- 172.21.15.119:0/2441959677 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3940104ad0 0x7f394019a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:36.849 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:36.842+0000 7f3945200640 1 -- 172.21.15.119:0/2441959677 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3940105490 msgr2=0x7f39401a11d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:36.849 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:36.842+0000 7f3945200640 1 --2- 172.21.15.119:0/2441959677 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3940105490 0x7f39401a11d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:36.850 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:36.842+0000 7f3945200640 1 -- 172.21.15.119:0/2441959677 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3940104ad0 msgr2=0x7f394019a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:36.850 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:36.842+0000 7f3945200640 1 --2- 172.21.15.119:0/2441959677 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3940104ad0 0x7f394019a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:36.850 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:36.842+0000 7f3945200640 1 -- 172.21.15.119:0/2441959677 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39401a1870 con 0x7f39401038e0 2024-03-28T00:25:36.850 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:36.842+0000 7f393fe00640 1 --2- 172.21.15.119:0/2441959677 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3940104ad0 0x7f394019a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T00:25:36.850 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:36.842+0000 7f3945200640 1 --2- 172.21.15.119:0/2441959677 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f39401038e0 0x7f3940199c60 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f3934005ab0 tx=0x7f393402c6e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:25:36.850 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:36.842+0000 7f393d600640 1 -- 172.21.15.119:0/2441959677 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3934041070 con 0x7f39401038e0 2024-03-28T00:25:36.850 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:36.842+0000 7f393d600640 1 -- 172.21.15.119:0/2441959677 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3934004450 con 0x7f39401038e0 2024-03-28T00:25:36.850 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:36.842+0000 7f393d600640 1 -- 172.21.15.119:0/2441959677 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3934034440 con 0x7f39401038e0 2024-03-28T00:25:36.850 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:36.842+0000 7f3947c00640 1 -- 172.21.15.119:0/2441959677 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39401a1b00 con 0x7f39401038e0 2024-03-28T00:25:36.850 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:36.843+0000 7f3947c00640 1 -- 172.21.15.119:0/2441959677 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39401a20a0 con 0x7f39401038e0 2024-03-28T00:25:36.851 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:36.844+0000 7f3947c00640 1 -- 172.21.15.119:0/2441959677 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f394010ac80 con 0x7f39401038e0 2024-03-28T00:25:36.851 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:36.845+0000 7f393d600640 1 -- 172.21.15.119:0/2441959677 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 18) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f3934044070 con 0x7f39401038e0 2024-03-28T00:25:36.853 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:36.846+0000 7f393d600640 1 --2- 172.21.15.119:0/2441959677 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f39280a2880 0x7f39280a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:36.853 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:36.847+0000 7f393d600640 1 -- 172.21.15.119:0/2441959677 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f39340f06b0 con 0x7f39401038e0 2024-03-28T00:25:36.853 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:36.847+0000 7f393fe00640 1 --2- 172.21.15.119:0/2441959677 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f39280a2880 0x7f39280a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:36.853 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:36.847+0000 7f393fe00640 1 --2- 172.21.15.119:0/2441959677 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f39280a2880 0x7f39280a4d40 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f392c009c20 tx=0x7f392c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:25:36.856 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:36.851+0000 7f393d600640 1 -- 172.21.15.119:0/2441959677 <== mon.1 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f39340bd750 con 0x7f39401038e0 2024-03-28T00:25:36.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:25:36 smithi119 ceph-mon[31518]: pgmap v363: 97 pgs: 97 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-03-28T00:25:36.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:25:36 smithi119 ceph-mon[31518]: from='client.? 172.21.15.119:0/2407966401' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-28T00:25:36.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:25:36 smithi119 ceph-mon[31518]: from='client.? 172.21.15.119:0/1963322927' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-28T00:25:37.099 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:37.093+0000 7f3947c00640 1 -- 172.21.15.119:0/2441959677 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f394002d070 con 0x7f39401038e0 2024-03-28T00:25:37.100 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:37.094+0000 7f393d600640 1 -- 172.21.15.119:0/2441959677 <== mon.1 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+11473 (secure 0 0 0) 0x7f39340bd050 con 0x7f39401038e0 2024-03-28T00:25:37.100 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-28T00:25:37.101 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14400,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/3988635716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":3988635716},{"type":"v1","addr":"172.21.15.119:6837","nonce":3988635716}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6839/1595343240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1595343240},{"type":"v1","addr":"172.21.15.119:6839","nonce":1595343240}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-28T00:23:56.215355+0000","modified":"2024-03-28T00:24:04.459985+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24581,"mds_1":24550,"mds_2":24530,"mds_3":24523,"mds_4":24511},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14466":{"gid":14466,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6841/1678451188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6840","nonce":1678451188},{"type":"v1","addr":"172.21.15.119:6841","nonce":1678451188}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24445":{"gid":24445,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/609963168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":609963168},{"type":"v1","addr":"172.21.15.165:6833","nonce":609963168}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24455":{"gid":24455,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.132:6833/1179016670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":1179016670},{"type":"v1","addr":"172.21.15.132:6833","nonce":1179016670}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24473":{"gid":24473,"name":"j","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6835/744950487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":744950487},{"type":"v1","addr":"172.21.15.119:6835","nonce":744950487}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24500":{"gid":24500,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.132:6835/356981946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":356981946},{"type":"v1","addr":"172.21.15.132:6835","nonce":356981946}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24511":{"gid":24511,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.165:6835/1947980102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":1947980102},{"type":"v1","addr":"172.21.15.165:6835","nonce":1947980102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24523":{"gid":24523,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.165:6837/1531193032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":1531193032},{"type":"v1","addr":"172.21.15.165:6837","nonce":1531193032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24530":{"gid":24530,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.132:6837/3383082967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6836","nonce":3383082967},{"type":"v1","addr":"172.21.15.132:6837","nonce":3383082967}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24550":{"gid":24550,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.165:6839/3004684592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":3004684592},{"type":"v1","addr":"172.21.15.165:6839","nonce":3004684592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.132:6839/3305822525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6838","nonce":3305822525},{"type":"v1","addr":"172.21.15.132:6839","nonce":3305822525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24523,24530,24550,24511,24581]},"id":1}]} 2024-03-28T00:25:37.101 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 26 2024-03-28T00:25:37.103 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:37.097+0000 7f3947c00640 1 -- 172.21.15.119:0/2441959677 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f39280a2880 msgr2=0x7f39280a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:37.104 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:37.097+0000 7f3947c00640 1 --2- 172.21.15.119:0/2441959677 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f39280a2880 0x7f39280a4d40 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f392c009c20 tx=0x7f392c005f90 comp rx=0 tx=0).stop 2024-03-28T00:25:37.104 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:37.097+0000 7f3947c00640 1 -- 172.21.15.119:0/2441959677 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f39401038e0 msgr2=0x7f3940199c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:37.104 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:37.097+0000 7f3947c00640 1 --2- 172.21.15.119:0/2441959677 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f39401038e0 0x7f3940199c60 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f3934005ab0 tx=0x7f393402c6e0 comp rx=0 tx=0).stop 2024-03-28T00:25:37.104 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:37.097+0000 7f3947c00640 1 -- 172.21.15.119:0/2441959677 shutdown_connections 2024-03-28T00:25:37.104 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:37.097+0000 7f3947c00640 1 --2- 172.21.15.119:0/2441959677 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f39280a2880 0x7f39280a4d40 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:37.104 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:37.097+0000 7f3947c00640 1 --2- 172.21.15.119:0/2441959677 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3940105490 0x7f39401a11d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:37.104 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:37.097+0000 7f3947c00640 1 --2- 172.21.15.119:0/2441959677 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3940104ad0 0x7f394019a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:37.104 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:37.097+0000 7f3947c00640 1 --2- 172.21.15.119:0/2441959677 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f39401038e0 0x7f3940199c60 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:37.104 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:37.097+0000 7f3947c00640 1 -- 172.21.15.119:0/2441959677 >> 172.21.15.119:0/2441959677 conn(0x7f39400fee50 msgr2=0x7f3940107e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:25:37.104 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:37.097+0000 7f3947c00640 1 -- 172.21.15.119:0/2441959677 shutdown_connections 2024-03-28T00:25:37.104 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:37.097+0000 7f3947c00640 1 -- 172.21.15.119:0/2441959677 wait complete. 2024-03-28T00:25:37.123 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T00:25:37.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:25:36 smithi165 ceph-mon[36302]: pgmap v363: 97 pgs: 97 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-03-28T00:25:37.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:25:36 smithi165 ceph-mon[36302]: from='client.? 172.21.15.119:0/2407966401' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-28T00:25:37.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:25:36 smithi165 ceph-mon[36302]: from='client.? 172.21.15.119:0/1963322927' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-28T00:25:37.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:25:36 smithi132 ceph-mon[35941]: pgmap v363: 97 pgs: 97 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-03-28T00:25:37.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:25:36 smithi132 ceph-mon[35941]: from='client.? 172.21.15.119:0/2407966401' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-28T00:25:37.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:25:36 smithi132 ceph-mon[35941]: from='client.? 172.21.15.119:0/1963322927' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-28T00:25:37.334 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:37.327+0000 7f58e4800640 1 -- 172.21.15.119:0/4284185133 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f58dc0772a0 msgr2=0x7f58dc077700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:37.334 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:37.327+0000 7f58e4800640 1 --2- 172.21.15.119:0/4284185133 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f58dc0772a0 0x7f58dc077700 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f58c8009900 tx=0x7f58c802bc50 comp rx=0 tx=0).stop 2024-03-28T00:25:37.334 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:37.327+0000 7f58e4800640 1 -- 172.21.15.119:0/4284185133 shutdown_connections 2024-03-28T00:25:37.334 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:37.327+0000 7f58e4800640 1 --2- 172.21.15.119:0/4284185133 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f58dc077c40 0x7f58dc0784e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:37.334 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:37.327+0000 7f58e4800640 1 --2- 172.21.15.119:0/4284185133 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f58dc0772a0 0x7f58dc077700 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:37.334 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:37.327+0000 7f58e4800640 1 --2- 172.21.15.119:0/4284185133 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58dc076060 0x7f58dc076480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:37.334 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:37.327+0000 7f58e4800640 1 -- 172.21.15.119:0/4284185133 >> 172.21.15.119:0/4284185133 conn(0x7f58dc0fee50 msgr2=0x7f58dc101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:25:37.334 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:37.327+0000 7f58e4800640 1 -- 172.21.15.119:0/4284185133 shutdown_connections 2024-03-28T00:25:37.334 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:37.327+0000 7f58e4800640 1 -- 172.21.15.119:0/4284185133 wait complete. 2024-03-28T00:25:37.336 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:37.328+0000 7f58e4800640 1 Processor -- start 2024-03-28T00:25:37.336 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:37.328+0000 7f58e4800640 1 -- start start 2024-03-28T00:25:37.337 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:37.329+0000 7f58e4800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f58dc076060 0x7f58dc19dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:37.337 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:37.329+0000 7f58e4800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f58dc0772a0 0x7f58dc19e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:37.337 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:37.329+0000 7f58e4800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58dc077c40 0x7f58dc1a5590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:37.337 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:37.329+0000 7f58e4800640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58dc113970 con 0x7f58dc076060 2024-03-28T00:25:37.337 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:37.329+0000 7f58e4800640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58dc1137f0 con 0x7f58dc077c40 2024-03-28T00:25:37.337 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:37.329+0000 7f58e4800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58dc113af0 con 0x7f58dc0772a0 2024-03-28T00:25:37.337 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:37.329+0000 7f58e1e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f58dc076060 0x7f58dc19dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:37.337 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:37.329+0000 7f58e1e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f58dc076060 0x7f58dc19dfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:60080/0 (socket says 172.21.15.119:60080) 2024-03-28T00:25:37.337 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:37.329+0000 7f58e1e00640 1 -- 172.21.15.119:0/3594782666 learned_addr learned my addr 172.21.15.119:0/3594782666 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:25:37.337 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:37.329+0000 7f58e2800640 1 --2- 172.21.15.119:0/3594782666 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58dc077c40 0x7f58dc1a5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:37.337 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:37.329+0000 7f58e1400640 1 --2- 172.21.15.119:0/3594782666 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f58dc0772a0 0x7f58dc19e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:37.337 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:37.329+0000 7f58e1e00640 1 -- 172.21.15.119:0/3594782666 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f58dc0772a0 msgr2=0x7f58dc19e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:37.337 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:37.329+0000 7f58e1e00640 1 --2- 172.21.15.119:0/3594782666 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f58dc0772a0 0x7f58dc19e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:37.338 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:37.329+0000 7f58e1e00640 1 -- 172.21.15.119:0/3594782666 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58dc077c40 msgr2=0x7f58dc1a5590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:37.338 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:37.329+0000 7f58e1e00640 1 --2- 172.21.15.119:0/3594782666 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58dc077c40 0x7f58dc1a5590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:37.338 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:37.329+0000 7f58e1e00640 1 -- 172.21.15.119:0/3594782666 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58dc1a5c00 con 0x7f58dc076060 2024-03-28T00:25:37.338 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:37.329+0000 7f58e2800640 1 --2- 172.21.15.119:0/3594782666 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58dc077c40 0x7f58dc1a5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T00:25:37.338 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:37.330+0000 7f58e1e00640 1 --2- 172.21.15.119:0/3594782666 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f58dc076060 0x7f58dc19dfe0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f58d000e9f0 tx=0x7f58d000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:25:37.338 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:37.330+0000 7f58d6a00640 1 -- 172.21.15.119:0/3594782666 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58d000cd80 con 0x7f58dc076060 2024-03-28T00:25:37.338 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:37.330+0000 7f58d6a00640 1 -- 172.21.15.119:0/3594782666 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f58d0010910 con 0x7f58dc076060 2024-03-28T00:25:37.338 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:37.330+0000 7f58e4800640 1 -- 172.21.15.119:0/3594782666 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58dc1a5ef0 con 0x7f58dc076060 2024-03-28T00:25:37.338 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:37.330+0000 7f58d6a00640 1 -- 172.21.15.119:0/3594782666 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58d0018990 con 0x7f58dc076060 2024-03-28T00:25:37.338 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:37.330+0000 7f58e4800640 1 -- 172.21.15.119:0/3594782666 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58dc1a6350 con 0x7f58dc076060 2024-03-28T00:25:37.338 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:37.331+0000 7f58e4800640 1 -- 172.21.15.119:0/3594782666 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58dc1103e0 con 0x7f58dc076060 2024-03-28T00:25:37.339 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:37.333+0000 7f58d6a00640 1 -- 172.21.15.119:0/3594782666 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 18) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f58d0010ac0 con 0x7f58dc076060 2024-03-28T00:25:37.340 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:37.334+0000 7f58d6a00640 1 --2- 172.21.15.119:0/3594782666 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f58c40a2800 0x7f58c40a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:37.341 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:37.334+0000 7f58d6a00640 1 -- 172.21.15.119:0/3594782666 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f58d00cedb0 con 0x7f58dc076060 2024-03-28T00:25:37.341 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:37.335+0000 7f58e1400640 1 --2- 172.21.15.119:0/3594782666 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f58c40a2800 0x7f58c40a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:37.341 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:37.335+0000 7f58e1400640 1 --2- 172.21.15.119:0/3594782666 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f58c40a2800 0x7f58c40a4cc0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f58c802c260 tx=0x7f58c8000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:25:37.345 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:37.339+0000 7f58d6a00640 1 -- 172.21.15.119:0/3594782666 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f58d00d3070 con 0x7f58dc076060 2024-03-28T00:25:37.549 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:37.542+0000 7f58e4800640 1 -- 172.21.15.119:0/3594782666 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f58dc0622d0 con 0x7f58dc076060 2024-03-28T00:25:37.551 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:37.545+0000 7f58d6a00640 1 -- 172.21.15.119:0/3594782666 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+22638 (secure 0 0 0) 0x7f58d009b800 con 0x7f58dc076060 2024-03-28T00:25:37.551 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-28T00:25:37.552 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":87,"fsid":"10f73b92-ec98-11ee-95d0-87774f69a715","created":"2024-03-28T00:14:57.563604+0000","modified":"2024-03-28T00:24:05.529756+0000","last_up_change":"2024-03-28T00:22:58.821662+0000","last_in_change":"2024-03-28T00:22:38.467270+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T00:18:46.253414+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-28T00:23:54.261981+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-28T00:23:55.685836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c0b875d-a01e-4294-9c9c-5d0b048b8912","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":580396252},{"type":"v1","addr":"172.21.15.119:6803","nonce":580396252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":580396252},{"type":"v1","addr":"172.21.15.119:6805","nonce":580396252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":580396252},{"type":"v1","addr":"172.21.15.119:6809","nonce":580396252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":580396252},{"type":"v1","addr":"172.21.15.119:6807","nonce":580396252}]},"public_addr":"172.21.15.119:6803/580396252","cluster_addr":"172.21.15.119:6805/580396252","heartbeat_back_addr":"172.21.15.119:6809/580396252","heartbeat_front_addr":"172.21.15.119:6807/580396252","state":["exists","up"]},{"osd":1,"uuid":"5567f7c2-8ea4-4789-9ea1-467793324fb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":2500318888},{"type":"v1","addr":"172.21.15.132:6801","nonce":2500318888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":2500318888},{"type":"v1","addr":"172.21.15.132:6803","nonce":2500318888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":2500318888},{"type":"v1","addr":"172.21.15.132:6807","nonce":2500318888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":2500318888},{"type":"v1","addr":"172.21.15.132:6805","nonce":2500318888}]},"public_addr":"172.21.15.132:6801/2500318888","cluster_addr":"172.21.15.132:6803/2500318888","heartbeat_back_addr":"172.21.15.132:6807/2500318888","heartbeat_front_addr":"172.21.15.132:6805/2500318888","state":["exists","up"]},{"osd":2,"uuid":"ee54dbd0-24b4-4ee6-988c-ce71491199d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":2361434832},{"type":"v1","addr":"172.21.15.165:6801","nonce":2361434832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":2361434832},{"type":"v1","addr":"172.21.15.165:6803","nonce":2361434832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":2361434832},{"type":"v1","addr":"172.21.15.165:6807","nonce":2361434832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":2361434832},{"type":"v1","addr":"172.21.15.165:6805","nonce":2361434832}]},"public_addr":"172.21.15.165:6801/2361434832","cluster_addr":"172.21.15.165:6803/2361434832","heartbeat_back_addr":"172.21.15.165:6807/2361434832","heartbeat_front_addr":"172.21.15.165:6805/2361434832","state":["exists","up"]},{"osd":3,"uuid":"0464c40b-828f-4db4-96d9-3f7a6325e6da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":3732272210},{"type":"v1","addr":"172.21.15.119:6811","nonce":3732272210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":3732272210},{"type":"v1","addr":"172.21.15.119:6813","nonce":3732272210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":3732272210},{"type":"v1","addr":"172.21.15.119:6817","nonce":3732272210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":3732272210},{"type":"v1","addr":"172.21.15.119:6815","nonce":3732272210}]},"public_addr":"172.21.15.119:6811/3732272210","cluster_addr":"172.21.15.119:6813/3732272210","heartbeat_back_addr":"172.21.15.119:6817/3732272210","heartbeat_front_addr":"172.21.15.119:6815/3732272210","state":["exists","up"]},{"osd":4,"uuid":"8fd70b8e-5128-4173-aa0e-e735dfeb995d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":2803039505},{"type":"v1","addr":"172.21.15.132:6809","nonce":2803039505}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":2803039505},{"type":"v1","addr":"172.21.15.132:6811","nonce":2803039505}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":2803039505},{"type":"v1","addr":"172.21.15.132:6815","nonce":2803039505}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":2803039505},{"type":"v1","addr":"172.21.15.132:6813","nonce":2803039505}]},"public_addr":"172.21.15.132:6809/2803039505","cluster_addr":"172.21.15.132:6811/2803039505","heartbeat_back_addr":"172.21.15.132:6815/2803039505","heartbeat_front_addr":"172.21.15.132:6813/2803039505","state":["exists","up"]},{"osd":5,"uuid":"3224b4c1-733e-4424-a6da-870bbb5f94e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":2561252206},{"type":"v1","addr":"172.21.15.165:6809","nonce":2561252206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":2561252206},{"type":"v1","addr":"172.21.15.165:6811","nonce":2561252206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":2561252206},{"type":"v1","addr":"172.21.15.165:6815","nonce":2561252206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":2561252206},{"type":"v1","addr":"172.21.15.165:6813","nonce":2561252206}]},"public_addr":"172.21.15.165:6809/2561252206","cluster_addr":"172.21.15.165:6811/2561252206","heartbeat_back_addr":"172.21.15.165:6815/2561252206","heartbeat_front_addr":"172.21.15.165:6813/2561252206","state":["exists","up"]},{"osd":6,"uuid":"e435745d-59b5-439a-9091-0946ca0e9332","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":2545410230},{"type":"v1","addr":"172.21.15.119:6819","nonce":2545410230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":2545410230},{"type":"v1","addr":"172.21.15.119:6821","nonce":2545410230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":2545410230},{"type":"v1","addr":"172.21.15.119:6825","nonce":2545410230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":2545410230},{"type":"v1","addr":"172.21.15.119:6823","nonce":2545410230}]},"public_addr":"172.21.15.119:6819/2545410230","cluster_addr":"172.21.15.119:6821/2545410230","heartbeat_back_addr":"172.21.15.119:6825/2545410230","heartbeat_front_addr":"172.21.15.119:6823/2545410230","state":["exists","up"]},{"osd":7,"uuid":"79b97a12-f6b4-48c3-b517-2dd6b6e7c85f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":2877892376},{"type":"v1","addr":"172.21.15.132:6817","nonce":2877892376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":2877892376},{"type":"v1","addr":"172.21.15.132:6819","nonce":2877892376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":2877892376},{"type":"v1","addr":"172.21.15.132:6823","nonce":2877892376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":2877892376},{"type":"v1","addr":"172.21.15.132:6821","nonce":2877892376}]},"public_addr":"172.21.15.132:6817/2877892376","cluster_addr":"172.21.15.132:6819/2877892376","heartbeat_back_addr":"172.21.15.132:6823/2877892376","heartbeat_front_addr":"172.21.15.132:6821/2877892376","state":["exists","up"]},{"osd":8,"uuid":"2970524c-b99a-412d-85c2-6c29b54035e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":4115226150},{"type":"v1","addr":"172.21.15.165:6817","nonce":4115226150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":4115226150},{"type":"v1","addr":"172.21.15.165:6819","nonce":4115226150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":4115226150},{"type":"v1","addr":"172.21.15.165:6823","nonce":4115226150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":4115226150},{"type":"v1","addr":"172.21.15.165:6821","nonce":4115226150}]},"public_addr":"172.21.15.165:6817/4115226150","cluster_addr":"172.21.15.165:6819/4115226150","heartbeat_back_addr":"172.21.15.165:6823/4115226150","heartbeat_front_addr":"172.21.15.165:6821/4115226150","state":["exists","up"]},{"osd":9,"uuid":"2b1a08ed-cde9-4def-b3e8-838fff22709c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":2532245222},{"type":"v1","addr":"172.21.15.119:6827","nonce":2532245222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":2532245222},{"type":"v1","addr":"172.21.15.119:6829","nonce":2532245222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":2532245222},{"type":"v1","addr":"172.21.15.119:6833","nonce":2532245222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":2532245222},{"type":"v1","addr":"172.21.15.119:6831","nonce":2532245222}]},"public_addr":"172.21.15.119:6827/2532245222","cluster_addr":"172.21.15.119:6829/2532245222","heartbeat_back_addr":"172.21.15.119:6833/2532245222","heartbeat_front_addr":"172.21.15.119:6831/2532245222","state":["exists","up"]},{"osd":10,"uuid":"f019acd0-915a-4094-b12c-b53672740a5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":3454298492},{"type":"v1","addr":"172.21.15.132:6825","nonce":3454298492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":3454298492},{"type":"v1","addr":"172.21.15.132:6827","nonce":3454298492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":3454298492},{"type":"v1","addr":"172.21.15.132:6831","nonce":3454298492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":3454298492},{"type":"v1","addr":"172.21.15.132:6829","nonce":3454298492}]},"public_addr":"172.21.15.132:6825/3454298492","cluster_addr":"172.21.15.132:6827/3454298492","heartbeat_back_addr":"172.21.15.132:6831/3454298492","heartbeat_front_addr":"172.21.15.132:6829/3454298492","state":["exists","up"]},{"osd":11,"uuid":"2c1e59a8-8f21-45c0-8568-4b632ce020f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":2138771601},{"type":"v1","addr":"172.21.15.165:6825","nonce":2138771601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":2138771601},{"type":"v1","addr":"172.21.15.165:6827","nonce":2138771601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":2138771601},{"type":"v1","addr":"172.21.15.165:6831","nonce":2138771601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":2138771601},{"type":"v1","addr":"172.21.15.165:6829","nonce":2138771601}]},"public_addr":"172.21.15.165:6825/2138771601","cluster_addr":"172.21.15.165:6827/2138771601","heartbeat_back_addr":"172.21.15.165:6831/2138771601","heartbeat_front_addr":"172.21.15.165:6829/2138771601","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:17:54.791531+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:18:19.466638+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:18:43.435536+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:19:15.702685+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:19:41.767370+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:20:06.793003+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:20:39.952744+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:21:05.727262+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:21:30.735829+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:22:04.072142+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:22:30.894047+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:22:56.637728+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.119:6801/2575246519":"2024-03-29T00:16:03.582545+0000","172.21.15.119:6800/2575246519":"2024-03-29T00:16:03.582545+0000","172.21.15.119:0/87565965":"2024-03-29T00:16:03.582545+0000","172.21.15.119:6801/2056426384":"2024-03-29T00:15:30.472283+0000","172.21.15.119:0/1639109126":"2024-03-29T00:15:30.472283+0000","172.21.15.119:0/468478300":"2024-03-29T00:15:30.472283+0000","172.21.15.119:0/2891240357":"2024-03-29T00:15:30.472283+0000","172.21.15.119:0/4201699272":"2024-03-29T00:16:03.582545+0000","172.21.15.119:6800/2056426384":"2024-03-29T00:15:30.472283+0000","172.21.15.119:0/2635982178":"2024-03-29T00:16:03.582545+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T00:25:37.554 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:37.548+0000 7f58e4800640 1 -- 172.21.15.119:0/3594782666 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f58c40a2800 msgr2=0x7f58c40a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:37.555 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:37.548+0000 7f58e4800640 1 --2- 172.21.15.119:0/3594782666 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f58c40a2800 0x7f58c40a4cc0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f58c802c260 tx=0x7f58c8000bc0 comp rx=0 tx=0).stop 2024-03-28T00:25:37.555 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:37.548+0000 7f58e4800640 1 -- 172.21.15.119:0/3594782666 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f58dc076060 msgr2=0x7f58dc19dfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:37.555 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:37.548+0000 7f58e4800640 1 --2- 172.21.15.119:0/3594782666 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f58dc076060 0x7f58dc19dfe0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f58d000e9f0 tx=0x7f58d000eeb0 comp rx=0 tx=0).stop 2024-03-28T00:25:37.555 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:37.548+0000 7f58e4800640 1 -- 172.21.15.119:0/3594782666 shutdown_connections 2024-03-28T00:25:37.555 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:37.548+0000 7f58e4800640 1 --2- 172.21.15.119:0/3594782666 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f58c40a2800 0x7f58c40a4cc0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:37.555 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:37.548+0000 7f58e4800640 1 --2- 172.21.15.119:0/3594782666 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58dc077c40 0x7f58dc1a5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:37.555 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:37.548+0000 7f58e4800640 1 --2- 172.21.15.119:0/3594782666 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f58dc0772a0 0x7f58dc19e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:37.555 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:37.548+0000 7f58e4800640 1 --2- 172.21.15.119:0/3594782666 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f58dc076060 0x7f58dc19dfe0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:37.555 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:37.548+0000 7f58e4800640 1 -- 172.21.15.119:0/3594782666 >> 172.21.15.119:0/3594782666 conn(0x7f58dc0fee50 msgr2=0x7f58dc100570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:25:37.555 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:37.548+0000 7f58e4800640 1 -- 172.21.15.119:0/3594782666 shutdown_connections 2024-03-28T00:25:37.555 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:37.548+0000 7f58e4800640 1 -- 172.21.15.119:0/3594782666 wait complete. 2024-03-28T00:25:37.574 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-28T00:18:46.253414+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-28T00:23:54.261981+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-28T00:23:55.685836+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-28T00:25:37.574 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-28T00:25:37.784 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:37.777+0000 7f8bd3800640 1 -- 172.21.15.119:0/2942557620 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8bcc102150 msgr2=0x7f8bcc1089e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:37.784 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:37.777+0000 7f8bd3800640 1 --2- 172.21.15.119:0/2942557620 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8bcc102150 0x7f8bcc1089e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f8bc4009ca0 tx=0x7f8bc402c0f0 comp rx=0 tx=0).stop 2024-03-28T00:25:37.784 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:37.777+0000 7f8bd3800640 1 -- 172.21.15.119:0/2942557620 shutdown_connections 2024-03-28T00:25:37.784 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:37.777+0000 7f8bd3800640 1 --2- 172.21.15.119:0/2942557620 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8bcc102150 0x7f8bcc1089e0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:37.784 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:37.777+0000 7f8bd3800640 1 --2- 172.21.15.119:0/2942557620 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8bcc101790 0x7f8bcc101c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:37.784 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:37.777+0000 7f8bd3800640 1 --2- 172.21.15.119:0/2942557620 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8bcc1005a0 0x7f8bcc1009c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:37.784 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:37.777+0000 7f8bd3800640 1 -- 172.21.15.119:0/2942557620 >> 172.21.15.119:0/2942557620 conn(0x7f8bcc0fbb30 msgr2=0x7f8bcc0fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:25:37.784 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:37.777+0000 7f8bd3800640 1 -- 172.21.15.119:0/2942557620 shutdown_connections 2024-03-28T00:25:37.784 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:37.777+0000 7f8bd3800640 1 -- 172.21.15.119:0/2942557620 wait complete. 2024-03-28T00:25:37.786 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:37.778+0000 7f8bd3800640 1 Processor -- start 2024-03-28T00:25:37.786 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:37.778+0000 7f8bd3800640 1 -- start start 2024-03-28T00:25:37.786 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:37.779+0000 7f8bd3800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8bcc1005a0 0x7f8bcc199b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:37.786 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:37.779+0000 7f8bd3800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8bcc101790 0x7f8bcc19a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:37.786 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:37.779+0000 7f8bd3800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8bcc102150 0x7f8bcc1a1110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:37.786 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:37.779+0000 7f8bd3800640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bcc10c230 con 0x7f8bcc102150 2024-03-28T00:25:37.787 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:37.779+0000 7f8bd3800640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bcc10c0b0 con 0x7f8bcc101790 2024-03-28T00:25:37.787 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:37.779+0000 7f8bd3800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bcc10c3b0 con 0x7f8bcc1005a0 2024-03-28T00:25:37.787 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:37.779+0000 7f8bd0e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8bcc1005a0 0x7f8bcc199b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:37.787 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:37.779+0000 7f8bc3e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8bcc101790 0x7f8bcc19a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:37.787 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:37.779+0000 7f8bc3e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8bcc101790 0x7f8bcc19a0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.119:45322/0 (socket says 172.21.15.119:45322) 2024-03-28T00:25:37.787 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:37.779+0000 7f8bd0e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8bcc1005a0 0x7f8bcc199b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.119:55456/0 (socket says 172.21.15.119:55456) 2024-03-28T00:25:37.787 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:37.779+0000 7f8bc3e00640 1 -- 172.21.15.119:0/3253714458 learned_addr learned my addr 172.21.15.119:0/3253714458 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:25:37.787 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:37.779+0000 7f8bd1800640 1 --2- 172.21.15.119:0/3253714458 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8bcc102150 0x7f8bcc1a1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:37.787 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:37.779+0000 7f8bd0e00640 1 -- 172.21.15.119:0/3253714458 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8bcc101790 msgr2=0x7f8bcc19a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:37.787 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:37.779+0000 7f8bd0e00640 1 --2- 172.21.15.119:0/3253714458 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8bcc101790 0x7f8bcc19a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:37.787 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:37.779+0000 7f8bd0e00640 1 -- 172.21.15.119:0/3253714458 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8bcc102150 msgr2=0x7f8bcc1a1110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:37.787 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:37.779+0000 7f8bd0e00640 1 --2- 172.21.15.119:0/3253714458 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8bcc102150 0x7f8bcc1a1110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:37.787 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:37.779+0000 7f8bd0e00640 1 -- 172.21.15.119:0/3253714458 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bcc1a1780 con 0x7f8bcc1005a0 2024-03-28T00:25:37.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:37.779+0000 7f8bc3e00640 1 --2- 172.21.15.119:0/3253714458 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8bcc101790 0x7f8bcc19a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T00:25:37.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:37.780+0000 7f8bd0e00640 1 --2- 172.21.15.119:0/3253714458 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8bcc1005a0 0x7f8bcc199b60 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f8bb400e9c0 tx=0x7f8bb400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:25:37.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:37.780+0000 7f8bca000640 1 -- 172.21.15.119:0/3253714458 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bb400ccc0 con 0x7f8bcc1005a0 2024-03-28T00:25:37.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:37.780+0000 7f8bca000640 1 -- 172.21.15.119:0/3253714458 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8bb40035f0 con 0x7f8bcc1005a0 2024-03-28T00:25:37.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:37.780+0000 7f8bca000640 1 -- 172.21.15.119:0/3253714458 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bb400ce60 con 0x7f8bcc1005a0 2024-03-28T00:25:37.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:37.780+0000 7f8bd3800640 1 -- 172.21.15.119:0/3253714458 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bcc1a1a70 con 0x7f8bcc1005a0 2024-03-28T00:25:37.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:37.780+0000 7f8bd3800640 1 -- 172.21.15.119:0/3253714458 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bcc1a1f70 con 0x7f8bcc1005a0 2024-03-28T00:25:37.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:37.781+0000 7f8bd3800640 1 -- 172.21.15.119:0/3253714458 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bcc1a2200 con 0x7f8bcc1005a0 2024-03-28T00:25:37.789 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:37.783+0000 7f8bca000640 1 -- 172.21.15.119:0/3253714458 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 18) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f8bb40037a0 con 0x7f8bcc1005a0 2024-03-28T00:25:37.790 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:37.784+0000 7f8bca000640 1 --2- 172.21.15.119:0/3253714458 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f8bac0a2880 0x7f8bac0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:37.791 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:37.784+0000 7f8bca000640 1 -- 172.21.15.119:0/3253714458 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f8bb40ceca0 con 0x7f8bcc1005a0 2024-03-28T00:25:37.791 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:37.785+0000 7f8bc3e00640 1 --2- 172.21.15.119:0/3253714458 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f8bac0a2880 0x7f8bac0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:37.791 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:37.785+0000 7f8bc3e00640 1 --2- 172.21.15.119:0/3253714458 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f8bac0a2880 0x7f8bac0a4d40 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc19b070 tx=0x7f8bbc00b480 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:25:37.794 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:37.788+0000 7f8bca000640 1 -- 172.21.15.119:0/3253714458 <== mon.1 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f8bb409bde0 con 0x7f8bcc1005a0 2024-03-28T00:25:37.998 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:37.992+0000 7f8bd3800640 1 -- 172.21.15.119:0/3253714458 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8bcc1a23d0 con 0x7f8bcc1005a0 2024-03-28T00:25:37.999 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:37.992+0000 7f8bca000640 1 -- 172.21.15.119:0/3253714458 <== mon.1 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8bb40860f0 con 0x7f8bcc1005a0 2024-03-28T00:25:37.999 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 1 2024-03-28T00:25:38.001 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:37.995+0000 7f8bd3800640 1 -- 172.21.15.119:0/3253714458 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f8bac0a2880 msgr2=0x7f8bac0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:38.001 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:37.995+0000 7f8bd3800640 1 --2- 172.21.15.119:0/3253714458 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f8bac0a2880 0x7f8bac0a4d40 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc19b070 tx=0x7f8bbc00b480 comp rx=0 tx=0).stop 2024-03-28T00:25:38.002 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:37.995+0000 7f8bd3800640 1 -- 172.21.15.119:0/3253714458 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8bcc1005a0 msgr2=0x7f8bcc199b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:38.002 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:37.995+0000 7f8bd3800640 1 --2- 172.21.15.119:0/3253714458 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8bcc1005a0 0x7f8bcc199b60 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f8bb400e9c0 tx=0x7f8bb400ee80 comp rx=0 tx=0).stop 2024-03-28T00:25:38.002 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:37.995+0000 7f8bd3800640 1 -- 172.21.15.119:0/3253714458 shutdown_connections 2024-03-28T00:25:38.002 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:37.995+0000 7f8bd3800640 1 --2- 172.21.15.119:0/3253714458 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f8bac0a2880 0x7f8bac0a4d40 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:38.003 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:37.995+0000 7f8bd3800640 1 --2- 172.21.15.119:0/3253714458 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8bcc102150 0x7f8bcc1a1110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:38.003 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:37.996+0000 7f8bd3800640 1 --2- 172.21.15.119:0/3253714458 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8bcc101790 0x7f8bcc19a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:38.003 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:37.996+0000 7f8bd3800640 1 --2- 172.21.15.119:0/3253714458 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8bcc1005a0 0x7f8bcc199b60 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:38.003 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:37.996+0000 7f8bd3800640 1 -- 172.21.15.119:0/3253714458 >> 172.21.15.119:0/3253714458 conn(0x7f8bcc0fbb30 msgr2=0x7f8bcc0fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:25:38.003 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:37.996+0000 7f8bd3800640 1 -- 172.21.15.119:0/3253714458 shutdown_connections 2024-03-28T00:25:38.003 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:37.996+0000 7f8bd3800640 1 -- 172.21.15.119:0/3253714458 wait complete. 2024-03-28T00:25:38.022 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-28T00:25:38.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:25:37 smithi165 ceph-mon[36302]: from='client.? 172.21.15.119:0/2441959677' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T00:25:38.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:25:37 smithi165 ceph-mon[36302]: from='client.? 172.21.15.119:0/3594782666' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-28T00:25:38.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:25:37 smithi119 ceph-mon[31518]: from='client.? 172.21.15.119:0/2441959677' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T00:25:38.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:25:37 smithi119 ceph-mon[31518]: from='client.? 172.21.15.119:0/3594782666' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-28T00:25:38.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:25:37 smithi132 ceph-mon[35941]: from='client.? 172.21.15.119:0/2441959677' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T00:25:38.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:25:37 smithi132 ceph-mon[35941]: from='client.? 172.21.15.119:0/3594782666' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-28T00:25:38.234 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:38.227+0000 7f0122400640 1 -- 172.21.15.119:0/3306120460 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f011c1038e0 msgr2=0x7f011c103d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:38.234 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:38.227+0000 7f0122400640 1 --2- 172.21.15.119:0/3306120460 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f011c1038e0 0x7f011c103d00 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f010c009ca0 tx=0x7f010c02c100 comp rx=0 tx=0).stop 2024-03-28T00:25:38.234 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:38.228+0000 7f0122400640 1 -- 172.21.15.119:0/3306120460 shutdown_connections 2024-03-28T00:25:38.234 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:38.228+0000 7f0122400640 1 --2- 172.21.15.119:0/3306120460 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f011c105490 0x7f011c10bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:38.234 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:38.228+0000 7f0122400640 1 --2- 172.21.15.119:0/3306120460 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f011c104ad0 0x7f011c104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:38.234 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:38.228+0000 7f0122400640 1 --2- 172.21.15.119:0/3306120460 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f011c1038e0 0x7f011c103d00 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:38.235 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:38.228+0000 7f0122400640 1 -- 172.21.15.119:0/3306120460 >> 172.21.15.119:0/3306120460 conn(0x7f011c0fee50 msgr2=0x7f011c101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:25:38.235 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:38.228+0000 7f0122400640 1 -- 172.21.15.119:0/3306120460 shutdown_connections 2024-03-28T00:25:38.235 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:38.228+0000 7f0122400640 1 -- 172.21.15.119:0/3306120460 wait complete. 2024-03-28T00:25:38.237 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:38.229+0000 7f0122400640 1 Processor -- start 2024-03-28T00:25:38.237 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:38.229+0000 7f0122400640 1 -- start start 2024-03-28T00:25:38.237 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:38.229+0000 7f0122400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f011c1038e0 0x7f011c199c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:38.237 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:38.229+0000 7f0122400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f011c104ad0 0x7f011c19a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:38.237 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:38.229+0000 7f0122400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f011c105490 0x7f011c1a1190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:38.237 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:38.229+0000 7f0122400640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f011c10f560 con 0x7f011c104ad0 2024-03-28T00:25:38.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:38.229+0000 7f0122400640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f011c10f3e0 con 0x7f011c1038e0 2024-03-28T00:25:38.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:38.229+0000 7f0122400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f011c10f6e0 con 0x7f011c105490 2024-03-28T00:25:38.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:38.229+0000 7f011b400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f011c1038e0 0x7f011c199c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:38.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:38.229+0000 7f0112a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f011c104ad0 0x7f011c19a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:38.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:38.229+0000 7f011b400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f011c1038e0 0x7f011c199c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.119:45336/0 (socket says 172.21.15.119:45336) 2024-03-28T00:25:38.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:38.229+0000 7f011b400640 1 -- 172.21.15.119:0/653278076 learned_addr learned my addr 172.21.15.119:0/653278076 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:25:38.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:38.229+0000 7f011be00640 1 --2- 172.21.15.119:0/653278076 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f011c105490 0x7f011c1a1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:38.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:38.230+0000 7f011b400640 1 -- 172.21.15.119:0/653278076 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f011c105490 msgr2=0x7f011c1a1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:38.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:38.230+0000 7f011b400640 1 --2- 172.21.15.119:0/653278076 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f011c105490 0x7f011c1a1190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:38.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:38.230+0000 7f011b400640 1 -- 172.21.15.119:0/653278076 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f011c104ad0 msgr2=0x7f011c19a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:38.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:38.230+0000 7f011b400640 1 --2- 172.21.15.119:0/653278076 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f011c104ad0 0x7f011c19a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:38.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:38.230+0000 7f011b400640 1 -- 172.21.15.119:0/653278076 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f011c1a1830 con 0x7f011c1038e0 2024-03-28T00:25:38.239 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:38.230+0000 7f011b400640 1 --2- 172.21.15.119:0/653278076 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f011c1038e0 0x7f011c199c20 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f010c000f80 tx=0x7f010c035d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:25:38.239 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:38.230+0000 7f0118c00640 1 -- 172.21.15.119:0/653278076 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f010c039040 con 0x7f011c1038e0 2024-03-28T00:25:38.239 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:38.230+0000 7f0118c00640 1 -- 172.21.15.119:0/653278076 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f010c044070 con 0x7f011c1038e0 2024-03-28T00:25:38.239 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:38.230+0000 7f0118c00640 1 -- 172.21.15.119:0/653278076 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f010c004430 con 0x7f011c1038e0 2024-03-28T00:25:38.239 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:38.230+0000 7f0122400640 1 -- 172.21.15.119:0/653278076 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f011c1a1ac0 con 0x7f011c1038e0 2024-03-28T00:25:38.239 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:38.230+0000 7f0122400640 1 -- 172.21.15.119:0/653278076 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f011c1a1f80 con 0x7f011c1038e0 2024-03-28T00:25:38.239 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:38.231+0000 7f0122400640 1 -- 172.21.15.119:0/653278076 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f011c10ac00 con 0x7f011c1038e0 2024-03-28T00:25:38.240 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:38.233+0000 7f0118c00640 1 -- 172.21.15.119:0/653278076 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 18) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f010c0045d0 con 0x7f011c1038e0 2024-03-28T00:25:38.240 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:38.234+0000 7f0118c00640 1 --2- 172.21.15.119:0/653278076 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f01000a2800 0x7f01000a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:38.240 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:38.234+0000 7f0118c00640 1 -- 172.21.15.119:0/653278076 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f010c0f0fa0 con 0x7f011c1038e0 2024-03-28T00:25:38.241 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:38.234+0000 7f0112a00640 1 --2- 172.21.15.119:0/653278076 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f01000a2800 0x7f01000a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:38.241 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:38.235+0000 7f0112a00640 1 --2- 172.21.15.119:0/653278076 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f01000a2800 0x7f01000a4cc0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f0104009bd0 tx=0x7f0104005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:25:38.244 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:38.238+0000 7f0118c00640 1 -- 172.21.15.119:0/653278076 <== mon.2 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f010c0be040 con 0x7f011c1038e0 2024-03-28T00:25:38.444 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:38.438+0000 7f0122400640 1 -- 172.21.15.119:0/653278076 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f011c0622a0 con 0x7f011c1038e0 2024-03-28T00:25:38.445 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:38.439+0000 7f0118c00640 1 -- 172.21.15.119:0/653278076 <== mon.2 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) v1 ==== 104+0+11 (secure 0 0 0) 0x7f010c0bd940 con 0x7f011c1038e0 2024-03-28T00:25:38.445 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 32 2024-03-28T00:25:38.448 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:38.442+0000 7f0122400640 1 -- 172.21.15.119:0/653278076 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f01000a2800 msgr2=0x7f01000a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:38.448 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:38.442+0000 7f0122400640 1 --2- 172.21.15.119:0/653278076 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f01000a2800 0x7f01000a4cc0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f0104009bd0 tx=0x7f0104005f90 comp rx=0 tx=0).stop 2024-03-28T00:25:38.449 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:38.442+0000 7f0122400640 1 -- 172.21.15.119:0/653278076 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f011c1038e0 msgr2=0x7f011c199c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:38.449 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:38.442+0000 7f0122400640 1 --2- 172.21.15.119:0/653278076 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f011c1038e0 0x7f011c199c20 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f010c000f80 tx=0x7f010c035d10 comp rx=0 tx=0).stop 2024-03-28T00:25:38.449 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:38.442+0000 7f0122400640 1 -- 172.21.15.119:0/653278076 shutdown_connections 2024-03-28T00:25:38.449 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:38.442+0000 7f0122400640 1 --2- 172.21.15.119:0/653278076 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f01000a2800 0x7f01000a4cc0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:38.449 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:38.442+0000 7f0122400640 1 --2- 172.21.15.119:0/653278076 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f011c105490 0x7f011c1a1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:38.449 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:38.442+0000 7f0122400640 1 --2- 172.21.15.119:0/653278076 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f011c104ad0 0x7f011c19a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:38.449 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:38.442+0000 7f0122400640 1 --2- 172.21.15.119:0/653278076 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f011c1038e0 0x7f011c199c20 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:38.449 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:38.442+0000 7f0122400640 1 -- 172.21.15.119:0/653278076 >> 172.21.15.119:0/653278076 conn(0x7f011c0fee50 msgr2=0x7f011c107dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:25:38.449 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:38.442+0000 7f0122400640 1 -- 172.21.15.119:0/653278076 shutdown_connections 2024-03-28T00:25:38.449 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:38.442+0000 7f0122400640 1 -- 172.21.15.119:0/653278076 wait complete. 2024-03-28T00:25:38.467 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-28T00:25:38.682 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:38.676+0000 7f090da00640 1 -- 172.21.15.119:0/2961767071 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0908102770 msgr2=0x7f0908102bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:38.683 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:38.676+0000 7f090da00640 1 --2- 172.21.15.119:0/2961767071 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0908102770 0x7f0908102bf0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f08f4009860 tx=0x7f08f402bb70 comp rx=0 tx=0).stop 2024-03-28T00:25:38.683 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:38.676+0000 7f090da00640 1 -- 172.21.15.119:0/2961767071 shutdown_connections 2024-03-28T00:25:38.683 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:38.676+0000 7f090da00640 1 --2- 172.21.15.119:0/2961767071 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0908103130 0x7f09081099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:38.683 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:38.676+0000 7f090da00640 1 --2- 172.21.15.119:0/2961767071 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0908102770 0x7f0908102bf0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:38.683 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:38.676+0000 7f090da00640 1 --2- 172.21.15.119:0/2961767071 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0908101580 0x7f09081019a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:38.683 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:38.676+0000 7f090da00640 1 -- 172.21.15.119:0/2961767071 >> 172.21.15.119:0/2961767071 conn(0x7f09080fcb10 msgr2=0x7f09080fef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:25:38.683 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:38.676+0000 7f090da00640 1 -- 172.21.15.119:0/2961767071 shutdown_connections 2024-03-28T00:25:38.683 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:38.676+0000 7f090da00640 1 -- 172.21.15.119:0/2961767071 wait complete. 2024-03-28T00:25:38.686 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:38.677+0000 7f090da00640 1 Processor -- start 2024-03-28T00:25:38.686 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:38.677+0000 7f090da00640 1 -- start start 2024-03-28T00:25:38.686 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:38.677+0000 7f090da00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0908101580 0x7f0908199aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:38.686 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:38.677+0000 7f090da00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0908102770 0x7f0908199fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:38.686 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:38.677+0000 7f090da00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0908103130 0x7f09081a1050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:38.687 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:38.677+0000 7f090da00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f090810d210 con 0x7f0908103130 2024-03-28T00:25:38.687 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:38.677+0000 7f090da00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f090810d090 con 0x7f0908102770 2024-03-28T00:25:38.687 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:38.677+0000 7f090da00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f090810d390 con 0x7f0908101580 2024-03-28T00:25:38.687 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:38.677+0000 7f0907e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0908101580 0x7f0908199aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:38.687 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:38.677+0000 7f0907e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0908101580 0x7f0908199aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.119:55482/0 (socket says 172.21.15.119:55482) 2024-03-28T00:25:38.687 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:38.677+0000 7f0907e00640 1 -- 172.21.15.119:0/4181858655 learned_addr learned my addr 172.21.15.119:0/4181858655 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:25:38.687 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:38.678+0000 7f090d000640 1 --2- 172.21.15.119:0/4181858655 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0908103130 0x7f09081a1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:38.687 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:38.678+0000 7f0907400640 1 --2- 172.21.15.119:0/4181858655 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0908102770 0x7f0908199fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:38.687 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:38.678+0000 7f0907400640 1 -- 172.21.15.119:0/4181858655 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0908101580 msgr2=0x7f0908199aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:38.687 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:38.678+0000 7f0907400640 1 --2- 172.21.15.119:0/4181858655 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0908101580 0x7f0908199aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:38.687 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:38.678+0000 7f0907400640 1 -- 172.21.15.119:0/4181858655 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0908103130 msgr2=0x7f09081a1050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:38.687 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:38.678+0000 7f0907400640 1 --2- 172.21.15.119:0/4181858655 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0908103130 0x7f09081a1050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:38.687 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:38.678+0000 7f0907400640 1 -- 172.21.15.119:0/4181858655 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09081a16c0 con 0x7f0908102770 2024-03-28T00:25:38.687 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:38.678+0000 7f0907e00640 1 --2- 172.21.15.119:0/4181858655 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0908101580 0x7f0908199aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T00:25:38.688 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:38.678+0000 7f0907400640 1 --2- 172.21.15.119:0/4181858655 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0908102770 0x7f0908199fe0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f08f402c0b0 tx=0x7f08f40042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:25:38.688 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:38.678+0000 7f0904c00640 1 -- 172.21.15.119:0/4181858655 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08f4041020 con 0x7f0908102770 2024-03-28T00:25:38.688 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:38.679+0000 7f0904c00640 1 -- 172.21.15.119:0/4181858655 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f08f40323b0 con 0x7f0908102770 2024-03-28T00:25:38.688 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:38.679+0000 7f0904c00640 1 -- 172.21.15.119:0/4181858655 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08f4039040 con 0x7f0908102770 2024-03-28T00:25:38.688 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:38.679+0000 7f090da00640 1 -- 172.21.15.119:0/4181858655 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09081a1950 con 0x7f0908102770 2024-03-28T00:25:38.688 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:38.679+0000 7f090da00640 1 -- 172.21.15.119:0/4181858655 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09081a1e60 con 0x7f0908102770 2024-03-28T00:25:38.688 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:38.680+0000 7f090da00640 1 -- 172.21.15.119:0/4181858655 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09081a2030 con 0x7f0908102770 2024-03-28T00:25:38.689 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:38.681+0000 7f0904c00640 1 -- 172.21.15.119:0/4181858655 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 18) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f08f4044020 con 0x7f0908102770 2024-03-28T00:25:38.689 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:38.683+0000 7f0904c00640 1 --2- 172.21.15.119:0/4181858655 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f08ec0a2880 0x7f08ec0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:38.689 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:38.683+0000 7f0904c00640 1 -- 172.21.15.119:0/4181858655 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f08f40f02b0 con 0x7f0908102770 2024-03-28T00:25:38.689 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:38.683+0000 7f0907e00640 1 --2- 172.21.15.119:0/4181858655 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f08ec0a2880 0x7f08ec0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:38.690 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:38.683+0000 7f0907e00640 1 --2- 172.21.15.119:0/4181858655 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f08ec0a2880 0x7f08ec0a4d40 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f08f8011550 tx=0x7f08f8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:25:38.692 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:38.686+0000 7f0904c00640 1 -- 172.21.15.119:0/4181858655 <== mon.2 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f08f404b020 con 0x7f0908102770 2024-03-28T00:25:38.890 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:38.884+0000 7f090da00640 1 -- 172.21.15.119:0/4181858655 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f09081a2200 con 0x7f0908102770 2024-03-28T00:25:38.891 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:38.884+0000 7f0904c00640 1 -- 172.21.15.119:0/4181858655 <== mon.2 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) v1 ==== 100+0+11 (secure 0 0 0) 0x7f08f40bd350 con 0x7f0908102770 2024-03-28T00:25:38.891 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 64 2024-03-28T00:25:38.894 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:38.887+0000 7f090da00640 1 -- 172.21.15.119:0/4181858655 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f08ec0a2880 msgr2=0x7f08ec0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:38.894 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:38.887+0000 7f090da00640 1 --2- 172.21.15.119:0/4181858655 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f08ec0a2880 0x7f08ec0a4d40 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f08f8011550 tx=0x7f08f8009260 comp rx=0 tx=0).stop 2024-03-28T00:25:38.894 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:38.887+0000 7f090da00640 1 -- 172.21.15.119:0/4181858655 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0908102770 msgr2=0x7f0908199fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:38.894 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:38.887+0000 7f090da00640 1 --2- 172.21.15.119:0/4181858655 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0908102770 0x7f0908199fe0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f08f402c0b0 tx=0x7f08f40042f0 comp rx=0 tx=0).stop 2024-03-28T00:25:38.894 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:38.888+0000 7f090da00640 1 -- 172.21.15.119:0/4181858655 shutdown_connections 2024-03-28T00:25:38.894 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:38.888+0000 7f090da00640 1 --2- 172.21.15.119:0/4181858655 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f08ec0a2880 0x7f08ec0a4d40 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:38.895 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:38.888+0000 7f090da00640 1 --2- 172.21.15.119:0/4181858655 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0908103130 0x7f09081a1050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:38.895 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:38.888+0000 7f090da00640 1 --2- 172.21.15.119:0/4181858655 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0908102770 0x7f0908199fe0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:38.895 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:38.888+0000 7f090da00640 1 --2- 172.21.15.119:0/4181858655 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0908101580 0x7f0908199aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:38.895 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:38.888+0000 7f090da00640 1 -- 172.21.15.119:0/4181858655 >> 172.21.15.119:0/4181858655 conn(0x7f09080fcb10 msgr2=0x7f09080738d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:25:38.895 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:38.888+0000 7f090da00640 1 -- 172.21.15.119:0/4181858655 shutdown_connections 2024-03-28T00:25:38.895 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:38.888+0000 7f090da00640 1 -- 172.21.15.119:0/4181858655 wait complete. 2024-03-28T00:25:38.912 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T00:25:39.106 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:39.099+0000 7fd19f000640 1 -- 172.21.15.119:0/1243567571 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd198104ad0 msgr2=0x7fd198104f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:39.107 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:39.099+0000 7fd19f000640 1 --2- 172.21.15.119:0/1243567571 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd198104ad0 0x7fd198104f50 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fd180009900 tx=0x7fd18002bc40 comp rx=0 tx=0).stop 2024-03-28T00:25:39.107 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:39.099+0000 7fd19f000640 1 -- 172.21.15.119:0/1243567571 shutdown_connections 2024-03-28T00:25:39.107 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:39.099+0000 7fd19f000640 1 --2- 172.21.15.119:0/1243567571 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd198105490 0x7fd19810bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:39.107 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:39.099+0000 7fd19f000640 1 --2- 172.21.15.119:0/1243567571 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd198104ad0 0x7fd198104f50 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:39.107 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:39.099+0000 7fd19f000640 1 --2- 172.21.15.119:0/1243567571 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd1981038e0 0x7fd198103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:39.107 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:39.099+0000 7fd19f000640 1 -- 172.21.15.119:0/1243567571 >> 172.21.15.119:0/1243567571 conn(0x7fd1980fee90 msgr2=0x7fd1981012b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:25:39.107 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:39.099+0000 7fd19f000640 1 -- 172.21.15.119:0/1243567571 shutdown_connections 2024-03-28T00:25:39.107 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:39.099+0000 7fd19f000640 1 -- 172.21.15.119:0/1243567571 wait complete. 2024-03-28T00:25:39.107 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:39.100+0000 7fd19f000640 1 Processor -- start 2024-03-28T00:25:39.110 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:39.101+0000 7fd19f000640 1 -- start start 2024-03-28T00:25:39.110 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:39.101+0000 7fd19f000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd1981038e0 0x7fd198199bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:39.110 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:39.101+0000 7fd19f000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd198104ad0 0x7fd19819a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:39.110 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:39.101+0000 7fd19f000640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd198105490 0x7fd1981a1170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:39.110 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:39.101+0000 7fd19f000640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd19810f5b0 con 0x7fd1981038e0 2024-03-28T00:25:39.110 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:39.101+0000 7fd19f000640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd19810f430 con 0x7fd198105490 2024-03-28T00:25:39.110 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:39.101+0000 7fd19f000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd19810f730 con 0x7fd198104ad0 2024-03-28T00:25:39.110 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:39.101+0000 7fd197e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd1981038e0 0x7fd198199bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:39.111 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:39.101+0000 7fd197e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd1981038e0 0x7fd198199bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:60160/0 (socket says 172.21.15.119:60160) 2024-03-28T00:25:39.111 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:39.101+0000 7fd197e00640 1 -- 172.21.15.119:0/4058602143 learned_addr learned my addr 172.21.15.119:0/4058602143 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:25:39.111 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:39.101+0000 7fd19d000640 1 --2- 172.21.15.119:0/4058602143 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd198105490 0x7fd1981a1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:39.111 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:39.101+0000 7fd18f400640 1 --2- 172.21.15.119:0/4058602143 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd198104ad0 0x7fd19819a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:39.111 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:39.101+0000 7fd197e00640 1 -- 172.21.15.119:0/4058602143 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd198104ad0 msgr2=0x7fd19819a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:39.111 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:39.101+0000 7fd197e00640 1 --2- 172.21.15.119:0/4058602143 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd198104ad0 0x7fd19819a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:39.111 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:39.101+0000 7fd197e00640 1 -- 172.21.15.119:0/4058602143 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd198105490 msgr2=0x7fd1981a1170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:39.111 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:39.101+0000 7fd197e00640 1 --2- 172.21.15.119:0/4058602143 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd198105490 0x7fd1981a1170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:39.111 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:39.101+0000 7fd197e00640 1 -- 172.21.15.119:0/4058602143 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1981a17e0 con 0x7fd1981038e0 2024-03-28T00:25:39.111 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:39.101+0000 7fd19d000640 1 --2- 172.21.15.119:0/4058602143 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd198105490 0x7fd1981a1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T00:25:39.112 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:39.102+0000 7fd197e00640 1 --2- 172.21.15.119:0/4058602143 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd1981038e0 0x7fd198199bc0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd18800e9c0 tx=0x7fd18800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:25:39.112 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:39.102+0000 7fd195600640 1 -- 172.21.15.119:0/4058602143 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd18800cd80 con 0x7fd1981038e0 2024-03-28T00:25:39.112 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:39.102+0000 7fd19f000640 1 -- 172.21.15.119:0/4058602143 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1981a1ad0 con 0x7fd1981038e0 2024-03-28T00:25:39.112 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:39.102+0000 7fd195600640 1 -- 172.21.15.119:0/4058602143 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd1880035f0 con 0x7fd1981038e0 2024-03-28T00:25:39.112 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:39.102+0000 7fd195600640 1 -- 172.21.15.119:0/4058602143 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd188010690 con 0x7fd1981038e0 2024-03-28T00:25:39.112 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:39.102+0000 7fd19f000640 1 -- 172.21.15.119:0/4058602143 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1981a1ff0 con 0x7fd1981038e0 2024-03-28T00:25:39.112 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:39.103+0000 7fd19f000640 1 -- 172.21.15.119:0/4058602143 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd19810ac00 con 0x7fd1981038e0 2024-03-28T00:25:39.113 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:39.105+0000 7fd195600640 1 -- 172.21.15.119:0/4058602143 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 18) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fd1880037a0 con 0x7fd1981038e0 2024-03-28T00:25:39.113 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:39.107+0000 7fd195600640 1 --2- 172.21.15.119:0/4058602143 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fd17c0a2800 0x7fd17c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:39.114 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:39.107+0000 7fd195600640 1 -- 172.21.15.119:0/4058602143 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fd1880cee10 con 0x7fd1981038e0 2024-03-28T00:25:39.114 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:39.107+0000 7fd18f400640 1 --2- 172.21.15.119:0/4058602143 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fd17c0a2800 0x7fd17c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:39.114 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:39.108+0000 7fd18f400640 1 --2- 172.21.15.119:0/4058602143 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fd17c0a2800 0x7fd17c0a4cc0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fd18002c250 tx=0x7fd18002cf90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:25:39.117 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:39.111+0000 7fd195600640 1 -- 172.21.15.119:0/4058602143 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd1880d2030 con 0x7fd1981038e0 2024-03-28T00:25:39.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:25:38 smithi165 ceph-mon[36302]: pgmap v364: 97 pgs: 97 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s 2024-03-28T00:25:39.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:25:38 smithi165 ceph-mon[36302]: from='client.? 172.21.15.119:0/3253714458' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-28T00:25:39.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:25:38 smithi165 ceph-mon[36302]: from='client.? 172.21.15.119:0/653278076' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-28T00:25:39.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:25:38 smithi119 ceph-mon[31518]: pgmap v364: 97 pgs: 97 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s 2024-03-28T00:25:39.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:25:38 smithi119 ceph-mon[31518]: from='client.? 172.21.15.119:0/3253714458' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-28T00:25:39.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:25:38 smithi119 ceph-mon[31518]: from='client.? 172.21.15.119:0/653278076' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-28T00:25:39.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:25:38 smithi132 ceph-mon[35941]: pgmap v364: 97 pgs: 97 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s 2024-03-28T00:25:39.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:25:38 smithi132 ceph-mon[35941]: from='client.? 172.21.15.119:0/3253714458' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-28T00:25:39.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:25:38 smithi132 ceph-mon[35941]: from='client.? 172.21.15.119:0/653278076' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-28T00:25:39.359 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:39.353+0000 7fd19f000640 1 -- 172.21.15.119:0/4058602143 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd1980622a0 con 0x7fd1981038e0 2024-03-28T00:25:39.361 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:39.354+0000 7fd195600640 1 -- 172.21.15.119:0/4058602143 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+11473 (secure 0 0 0) 0x7fd18809beb0 con 0x7fd1981038e0 2024-03-28T00:25:39.361 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-28T00:25:39.361 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14400,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/3988635716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":3988635716},{"type":"v1","addr":"172.21.15.119:6837","nonce":3988635716}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6839/1595343240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1595343240},{"type":"v1","addr":"172.21.15.119:6839","nonce":1595343240}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-28T00:23:56.215355+0000","modified":"2024-03-28T00:24:04.459985+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24581,"mds_1":24550,"mds_2":24530,"mds_3":24523,"mds_4":24511},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14466":{"gid":14466,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6841/1678451188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6840","nonce":1678451188},{"type":"v1","addr":"172.21.15.119:6841","nonce":1678451188}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24445":{"gid":24445,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/609963168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":609963168},{"type":"v1","addr":"172.21.15.165:6833","nonce":609963168}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24455":{"gid":24455,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.132:6833/1179016670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":1179016670},{"type":"v1","addr":"172.21.15.132:6833","nonce":1179016670}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24473":{"gid":24473,"name":"j","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6835/744950487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":744950487},{"type":"v1","addr":"172.21.15.119:6835","nonce":744950487}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24500":{"gid":24500,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.132:6835/356981946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":356981946},{"type":"v1","addr":"172.21.15.132:6835","nonce":356981946}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24511":{"gid":24511,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.165:6835/1947980102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":1947980102},{"type":"v1","addr":"172.21.15.165:6835","nonce":1947980102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24523":{"gid":24523,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.165:6837/1531193032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":1531193032},{"type":"v1","addr":"172.21.15.165:6837","nonce":1531193032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24530":{"gid":24530,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.132:6837/3383082967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6836","nonce":3383082967},{"type":"v1","addr":"172.21.15.132:6837","nonce":3383082967}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24550":{"gid":24550,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.165:6839/3004684592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":3004684592},{"type":"v1","addr":"172.21.15.165:6839","nonce":3004684592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.132:6839/3305822525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6838","nonce":3305822525},{"type":"v1","addr":"172.21.15.132:6839","nonce":3305822525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24523,24530,24550,24511,24581]},"id":1}]} 2024-03-28T00:25:39.361 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 26 2024-03-28T00:25:39.363 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:39.357+0000 7fd19f000640 1 -- 172.21.15.119:0/4058602143 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fd17c0a2800 msgr2=0x7fd17c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:39.364 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:39.357+0000 7fd19f000640 1 --2- 172.21.15.119:0/4058602143 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fd17c0a2800 0x7fd17c0a4cc0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fd18002c250 tx=0x7fd18002cf90 comp rx=0 tx=0).stop 2024-03-28T00:25:39.364 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:39.357+0000 7fd19f000640 1 -- 172.21.15.119:0/4058602143 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd1981038e0 msgr2=0x7fd198199bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:39.364 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:39.357+0000 7fd19f000640 1 --2- 172.21.15.119:0/4058602143 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd1981038e0 0x7fd198199bc0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd18800e9c0 tx=0x7fd18800ee80 comp rx=0 tx=0).stop 2024-03-28T00:25:39.364 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:39.357+0000 7fd19f000640 1 -- 172.21.15.119:0/4058602143 shutdown_connections 2024-03-28T00:25:39.364 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:39.357+0000 7fd19f000640 1 --2- 172.21.15.119:0/4058602143 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fd17c0a2800 0x7fd17c0a4cc0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:39.364 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:39.357+0000 7fd19f000640 1 --2- 172.21.15.119:0/4058602143 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd198105490 0x7fd1981a1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:39.364 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:39.357+0000 7fd19f000640 1 --2- 172.21.15.119:0/4058602143 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd198104ad0 0x7fd19819a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:39.364 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:39.357+0000 7fd19f000640 1 --2- 172.21.15.119:0/4058602143 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd1981038e0 0x7fd198199bc0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:39.364 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:39.357+0000 7fd19f000640 1 -- 172.21.15.119:0/4058602143 >> 172.21.15.119:0/4058602143 conn(0x7fd1980fee90 msgr2=0x7fd198107dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:25:39.364 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:39.357+0000 7fd19f000640 1 -- 172.21.15.119:0/4058602143 shutdown_connections 2024-03-28T00:25:39.364 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:39.357+0000 7fd19f000640 1 -- 172.21.15.119:0/4058602143 wait complete. 2024-03-28T00:25:39.382 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T00:25:39.589 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:39.583+0000 7ff02de00640 1 -- 172.21.15.119:0/3894950945 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff028107b00 msgr2=0x7ff028107f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:39.590 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:39.583+0000 7ff02de00640 1 --2- 172.21.15.119:0/3894950945 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff028107b00 0x7ff028107f20 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff018009900 tx=0x7ff01802bc50 comp rx=0 tx=0).stop 2024-03-28T00:25:39.590 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:39.583+0000 7ff02de00640 1 -- 172.21.15.119:0/3894950945 shutdown_connections 2024-03-28T00:25:39.590 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:39.583+0000 7ff02de00640 1 --2- 172.21.15.119:0/3894950945 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff0281096b0 0x7ff02810ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:39.590 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:39.583+0000 7ff02de00640 1 --2- 172.21.15.119:0/3894950945 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff028108cf0 0x7ff028109170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:39.590 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:39.583+0000 7ff02de00640 1 --2- 172.21.15.119:0/3894950945 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff028107b00 0x7ff028107f20 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:39.590 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:39.583+0000 7ff02de00640 1 -- 172.21.15.119:0/3894950945 >> 172.21.15.119:0/3894950945 conn(0x7ff028076ab0 msgr2=0x7ff028078ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:25:39.590 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:39.583+0000 7ff02de00640 1 -- 172.21.15.119:0/3894950945 shutdown_connections 2024-03-28T00:25:39.590 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:39.583+0000 7ff02de00640 1 -- 172.21.15.119:0/3894950945 wait complete. 2024-03-28T00:25:39.593 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:39.584+0000 7ff02de00640 1 Processor -- start 2024-03-28T00:25:39.593 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:39.584+0000 7ff02de00640 1 -- start start 2024-03-28T00:25:39.593 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:39.584+0000 7ff02de00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff028107b00 0x7ff02819dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:39.594 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:39.584+0000 7ff02de00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff028108cf0 0x7ff02819e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:39.594 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:39.584+0000 7ff02de00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff0281096b0 0x7ff0281a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:39.594 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:39.584+0000 7ff02de00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff028113750 con 0x7ff028107b00 2024-03-28T00:25:39.594 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:39.584+0000 7ff02de00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0281135d0 con 0x7ff0281096b0 2024-03-28T00:25:39.594 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:39.584+0000 7ff02de00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0281138d0 con 0x7ff028108cf0 2024-03-28T00:25:39.594 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:39.585+0000 7ff01fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff028108cf0 0x7ff02819e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:39.594 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:39.585+0000 7ff01fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff028108cf0 0x7ff02819e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.119:55536/0 (socket says 172.21.15.119:55536) 2024-03-28T00:25:39.594 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:39.585+0000 7ff01fe00640 1 -- 172.21.15.119:0/3095247401 learned_addr learned my addr 172.21.15.119:0/3095247401 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:25:39.594 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:39.585+0000 7ff02d400640 1 --2- 172.21.15.119:0/3095247401 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff0281096b0 0x7ff0281a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:39.594 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:39.585+0000 7ff02ca00640 1 --2- 172.21.15.119:0/3095247401 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff028107b00 0x7ff02819dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:39.595 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:39.585+0000 7ff01fe00640 1 -- 172.21.15.119:0/3095247401 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff0281096b0 msgr2=0x7ff0281a5470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:39.595 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:39.585+0000 7ff01fe00640 1 --2- 172.21.15.119:0/3095247401 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff0281096b0 0x7ff0281a5470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:39.595 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:39.585+0000 7ff01fe00640 1 -- 172.21.15.119:0/3095247401 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff028107b00 msgr2=0x7ff02819dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:39.595 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:39.585+0000 7ff01fe00640 1 --2- 172.21.15.119:0/3095247401 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff028107b00 0x7ff02819dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:39.595 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:39.585+0000 7ff01fe00640 1 -- 172.21.15.119:0/3095247401 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0281a5ae0 con 0x7ff028108cf0 2024-03-28T00:25:39.595 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:39.585+0000 7ff02d400640 1 --2- 172.21.15.119:0/3095247401 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff0281096b0 0x7ff0281a5470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T00:25:39.595 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:39.585+0000 7ff01fe00640 1 --2- 172.21.15.119:0/3095247401 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff028108cf0 0x7ff02819e400 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ff01000e9f0 tx=0x7ff01000edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:25:39.595 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:39.586+0000 7ff026000640 1 -- 172.21.15.119:0/3095247401 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff01000ccc0 con 0x7ff028108cf0 2024-03-28T00:25:39.595 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:39.586+0000 7ff026000640 1 -- 172.21.15.119:0/3095247401 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff010010910 con 0x7ff028108cf0 2024-03-28T00:25:39.595 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:39.586+0000 7ff026000640 1 -- 172.21.15.119:0/3095247401 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0100188a0 con 0x7ff028108cf0 2024-03-28T00:25:39.595 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:39.586+0000 7ff02de00640 1 -- 172.21.15.119:0/3095247401 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0281a5dd0 con 0x7ff028108cf0 2024-03-28T00:25:39.595 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:39.586+0000 7ff02de00640 1 -- 172.21.15.119:0/3095247401 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0281a6240 con 0x7ff028108cf0 2024-03-28T00:25:39.596 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:39.587+0000 7ff02de00640 1 -- 172.21.15.119:0/3095247401 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff02810ee70 con 0x7ff028108cf0 2024-03-28T00:25:39.596 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:39.589+0000 7ff026000640 1 -- 172.21.15.119:0/3095247401 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 18) v1 ==== 137291+0+0 (secure 0 0 0) 0x7ff010010ac0 con 0x7ff028108cf0 2024-03-28T00:25:39.596 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:39.590+0000 7ff026000640 1 --2- 172.21.15.119:0/3095247401 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7ff0080a2880 0x7ff0080a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:39.597 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:39.590+0000 7ff026000640 1 -- 172.21.15.119:0/3095247401 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8153+0+0 (secure 0 0 0) 0x7ff0100cf270 con 0x7ff028108cf0 2024-03-28T00:25:39.597 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:39.590+0000 7ff02ca00640 1 --2- 172.21.15.119:0/3095247401 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7ff0080a2880 0x7ff0080a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:39.597 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:39.591+0000 7ff02ca00640 1 --2- 172.21.15.119:0/3095247401 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7ff0080a2880 0x7ff0080a4d40 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7ff018000c00 tx=0x7ff018005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:25:39.602 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:39.597+0000 7ff026000640 1 -- 172.21.15.119:0/3095247401 <== mon.1 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff01009c310 con 0x7ff028108cf0 2024-03-28T00:25:39.794 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:39.788+0000 7ff02de00640 1 -- 172.21.15.119:0/3095247401 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff02802d2c0 con 0x7ff028108cf0 2024-03-28T00:25:39.795 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:39.789+0000 7ff026000640 1 -- 172.21.15.119:0/3095247401 <== mon.1 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+22638 (secure 0 0 0) 0x7ff01009bc10 con 0x7ff028108cf0 2024-03-28T00:25:39.796 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-28T00:25:39.796 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":87,"fsid":"10f73b92-ec98-11ee-95d0-87774f69a715","created":"2024-03-28T00:14:57.563604+0000","modified":"2024-03-28T00:24:05.529756+0000","last_up_change":"2024-03-28T00:22:58.821662+0000","last_in_change":"2024-03-28T00:22:38.467270+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T00:18:46.253414+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-28T00:23:54.261981+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-28T00:23:55.685836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c0b875d-a01e-4294-9c9c-5d0b048b8912","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":580396252},{"type":"v1","addr":"172.21.15.119:6803","nonce":580396252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":580396252},{"type":"v1","addr":"172.21.15.119:6805","nonce":580396252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":580396252},{"type":"v1","addr":"172.21.15.119:6809","nonce":580396252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":580396252},{"type":"v1","addr":"172.21.15.119:6807","nonce":580396252}]},"public_addr":"172.21.15.119:6803/580396252","cluster_addr":"172.21.15.119:6805/580396252","heartbeat_back_addr":"172.21.15.119:6809/580396252","heartbeat_front_addr":"172.21.15.119:6807/580396252","state":["exists","up"]},{"osd":1,"uuid":"5567f7c2-8ea4-4789-9ea1-467793324fb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":2500318888},{"type":"v1","addr":"172.21.15.132:6801","nonce":2500318888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":2500318888},{"type":"v1","addr":"172.21.15.132:6803","nonce":2500318888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":2500318888},{"type":"v1","addr":"172.21.15.132:6807","nonce":2500318888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":2500318888},{"type":"v1","addr":"172.21.15.132:6805","nonce":2500318888}]},"public_addr":"172.21.15.132:6801/2500318888","cluster_addr":"172.21.15.132:6803/2500318888","heartbeat_back_addr":"172.21.15.132:6807/2500318888","heartbeat_front_addr":"172.21.15.132:6805/2500318888","state":["exists","up"]},{"osd":2,"uuid":"ee54dbd0-24b4-4ee6-988c-ce71491199d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":2361434832},{"type":"v1","addr":"172.21.15.165:6801","nonce":2361434832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":2361434832},{"type":"v1","addr":"172.21.15.165:6803","nonce":2361434832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":2361434832},{"type":"v1","addr":"172.21.15.165:6807","nonce":2361434832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":2361434832},{"type":"v1","addr":"172.21.15.165:6805","nonce":2361434832}]},"public_addr":"172.21.15.165:6801/2361434832","cluster_addr":"172.21.15.165:6803/2361434832","heartbeat_back_addr":"172.21.15.165:6807/2361434832","heartbeat_front_addr":"172.21.15.165:6805/2361434832","state":["exists","up"]},{"osd":3,"uuid":"0464c40b-828f-4db4-96d9-3f7a6325e6da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":3732272210},{"type":"v1","addr":"172.21.15.119:6811","nonce":3732272210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":3732272210},{"type":"v1","addr":"172.21.15.119:6813","nonce":3732272210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":3732272210},{"type":"v1","addr":"172.21.15.119:6817","nonce":3732272210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":3732272210},{"type":"v1","addr":"172.21.15.119:6815","nonce":3732272210}]},"public_addr":"172.21.15.119:6811/3732272210","cluster_addr":"172.21.15.119:6813/3732272210","heartbeat_back_addr":"172.21.15.119:6817/3732272210","heartbeat_front_addr":"172.21.15.119:6815/3732272210","state":["exists","up"]},{"osd":4,"uuid":"8fd70b8e-5128-4173-aa0e-e735dfeb995d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":2803039505},{"type":"v1","addr":"172.21.15.132:6809","nonce":2803039505}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":2803039505},{"type":"v1","addr":"172.21.15.132:6811","nonce":2803039505}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":2803039505},{"type":"v1","addr":"172.21.15.132:6815","nonce":2803039505}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":2803039505},{"type":"v1","addr":"172.21.15.132:6813","nonce":2803039505}]},"public_addr":"172.21.15.132:6809/2803039505","cluster_addr":"172.21.15.132:6811/2803039505","heartbeat_back_addr":"172.21.15.132:6815/2803039505","heartbeat_front_addr":"172.21.15.132:6813/2803039505","state":["exists","up"]},{"osd":5,"uuid":"3224b4c1-733e-4424-a6da-870bbb5f94e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":2561252206},{"type":"v1","addr":"172.21.15.165:6809","nonce":2561252206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":2561252206},{"type":"v1","addr":"172.21.15.165:6811","nonce":2561252206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":2561252206},{"type":"v1","addr":"172.21.15.165:6815","nonce":2561252206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":2561252206},{"type":"v1","addr":"172.21.15.165:6813","nonce":2561252206}]},"public_addr":"172.21.15.165:6809/2561252206","cluster_addr":"172.21.15.165:6811/2561252206","heartbeat_back_addr":"172.21.15.165:6815/2561252206","heartbeat_front_addr":"172.21.15.165:6813/2561252206","state":["exists","up"]},{"osd":6,"uuid":"e435745d-59b5-439a-9091-0946ca0e9332","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":2545410230},{"type":"v1","addr":"172.21.15.119:6819","nonce":2545410230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":2545410230},{"type":"v1","addr":"172.21.15.119:6821","nonce":2545410230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":2545410230},{"type":"v1","addr":"172.21.15.119:6825","nonce":2545410230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":2545410230},{"type":"v1","addr":"172.21.15.119:6823","nonce":2545410230}]},"public_addr":"172.21.15.119:6819/2545410230","cluster_addr":"172.21.15.119:6821/2545410230","heartbeat_back_addr":"172.21.15.119:6825/2545410230","heartbeat_front_addr":"172.21.15.119:6823/2545410230","state":["exists","up"]},{"osd":7,"uuid":"79b97a12-f6b4-48c3-b517-2dd6b6e7c85f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":2877892376},{"type":"v1","addr":"172.21.15.132:6817","nonce":2877892376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":2877892376},{"type":"v1","addr":"172.21.15.132:6819","nonce":2877892376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":2877892376},{"type":"v1","addr":"172.21.15.132:6823","nonce":2877892376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":2877892376},{"type":"v1","addr":"172.21.15.132:6821","nonce":2877892376}]},"public_addr":"172.21.15.132:6817/2877892376","cluster_addr":"172.21.15.132:6819/2877892376","heartbeat_back_addr":"172.21.15.132:6823/2877892376","heartbeat_front_addr":"172.21.15.132:6821/2877892376","state":["exists","up"]},{"osd":8,"uuid":"2970524c-b99a-412d-85c2-6c29b54035e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":4115226150},{"type":"v1","addr":"172.21.15.165:6817","nonce":4115226150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":4115226150},{"type":"v1","addr":"172.21.15.165:6819","nonce":4115226150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":4115226150},{"type":"v1","addr":"172.21.15.165:6823","nonce":4115226150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":4115226150},{"type":"v1","addr":"172.21.15.165:6821","nonce":4115226150}]},"public_addr":"172.21.15.165:6817/4115226150","cluster_addr":"172.21.15.165:6819/4115226150","heartbeat_back_addr":"172.21.15.165:6823/4115226150","heartbeat_front_addr":"172.21.15.165:6821/4115226150","state":["exists","up"]},{"osd":9,"uuid":"2b1a08ed-cde9-4def-b3e8-838fff22709c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":2532245222},{"type":"v1","addr":"172.21.15.119:6827","nonce":2532245222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":2532245222},{"type":"v1","addr":"172.21.15.119:6829","nonce":2532245222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":2532245222},{"type":"v1","addr":"172.21.15.119:6833","nonce":2532245222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":2532245222},{"type":"v1","addr":"172.21.15.119:6831","nonce":2532245222}]},"public_addr":"172.21.15.119:6827/2532245222","cluster_addr":"172.21.15.119:6829/2532245222","heartbeat_back_addr":"172.21.15.119:6833/2532245222","heartbeat_front_addr":"172.21.15.119:6831/2532245222","state":["exists","up"]},{"osd":10,"uuid":"f019acd0-915a-4094-b12c-b53672740a5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":3454298492},{"type":"v1","addr":"172.21.15.132:6825","nonce":3454298492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":3454298492},{"type":"v1","addr":"172.21.15.132:6827","nonce":3454298492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":3454298492},{"type":"v1","addr":"172.21.15.132:6831","nonce":3454298492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":3454298492},{"type":"v1","addr":"172.21.15.132:6829","nonce":3454298492}]},"public_addr":"172.21.15.132:6825/3454298492","cluster_addr":"172.21.15.132:6827/3454298492","heartbeat_back_addr":"172.21.15.132:6831/3454298492","heartbeat_front_addr":"172.21.15.132:6829/3454298492","state":["exists","up"]},{"osd":11,"uuid":"2c1e59a8-8f21-45c0-8568-4b632ce020f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":2138771601},{"type":"v1","addr":"172.21.15.165:6825","nonce":2138771601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":2138771601},{"type":"v1","addr":"172.21.15.165:6827","nonce":2138771601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":2138771601},{"type":"v1","addr":"172.21.15.165:6831","nonce":2138771601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":2138771601},{"type":"v1","addr":"172.21.15.165:6829","nonce":2138771601}]},"public_addr":"172.21.15.165:6825/2138771601","cluster_addr":"172.21.15.165:6827/2138771601","heartbeat_back_addr":"172.21.15.165:6831/2138771601","heartbeat_front_addr":"172.21.15.165:6829/2138771601","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:17:54.791531+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:18:19.466638+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:18:43.435536+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:19:15.702685+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:19:41.767370+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:20:06.793003+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:20:39.952744+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:21:05.727262+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:21:30.735829+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:22:04.072142+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:22:30.894047+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:22:56.637728+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.119:6801/2575246519":"2024-03-29T00:16:03.582545+0000","172.21.15.119:6800/2575246519":"2024-03-29T00:16:03.582545+0000","172.21.15.119:0/87565965":"2024-03-29T00:16:03.582545+0000","172.21.15.119:6801/2056426384":"2024-03-29T00:15:30.472283+0000","172.21.15.119:0/1639109126":"2024-03-29T00:15:30.472283+0000","172.21.15.119:0/468478300":"2024-03-29T00:15:30.472283+0000","172.21.15.119:0/2891240357":"2024-03-29T00:15:30.472283+0000","172.21.15.119:0/4201699272":"2024-03-29T00:16:03.582545+0000","172.21.15.119:6800/2056426384":"2024-03-29T00:15:30.472283+0000","172.21.15.119:0/2635982178":"2024-03-29T00:16:03.582545+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T00:25:39.799 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:39.792+0000 7ff02de00640 1 -- 172.21.15.119:0/3095247401 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7ff0080a2880 msgr2=0x7ff0080a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:39.799 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:39.792+0000 7ff02de00640 1 --2- 172.21.15.119:0/3095247401 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7ff0080a2880 0x7ff0080a4d40 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7ff018000c00 tx=0x7ff018005fd0 comp rx=0 tx=0).stop 2024-03-28T00:25:39.799 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:39.792+0000 7ff02de00640 1 -- 172.21.15.119:0/3095247401 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff028108cf0 msgr2=0x7ff02819e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:39.799 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:39.792+0000 7ff02de00640 1 --2- 172.21.15.119:0/3095247401 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff028108cf0 0x7ff02819e400 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ff01000e9f0 tx=0x7ff01000edf0 comp rx=0 tx=0).stop 2024-03-28T00:25:39.799 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:39.792+0000 7ff02de00640 1 -- 172.21.15.119:0/3095247401 shutdown_connections 2024-03-28T00:25:39.799 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:39.792+0000 7ff02de00640 1 --2- 172.21.15.119:0/3095247401 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7ff0080a2880 0x7ff0080a4d40 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:39.799 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:39.792+0000 7ff02de00640 1 --2- 172.21.15.119:0/3095247401 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff0281096b0 0x7ff0281a5470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:39.799 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:39.792+0000 7ff02de00640 1 --2- 172.21.15.119:0/3095247401 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff028108cf0 0x7ff02819e400 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:39.800 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:39.792+0000 7ff02de00640 1 --2- 172.21.15.119:0/3095247401 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff028107b00 0x7ff02819dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:39.800 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:39.792+0000 7ff02de00640 1 -- 172.21.15.119:0/3095247401 >> 172.21.15.119:0/3095247401 conn(0x7ff028076ab0 msgr2=0x7ff02810c040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:25:39.800 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:39.792+0000 7ff02de00640 1 -- 172.21.15.119:0/3095247401 shutdown_connections 2024-03-28T00:25:39.800 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:39.793+0000 7ff02de00640 1 -- 172.21.15.119:0/3095247401 wait complete. 2024-03-28T00:25:39.818 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T00:25:40.016 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.009+0000 7f8860800640 1 -- 172.21.15.119:0/3879710915 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8858108ed0 msgr2=0x7f8858109350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:40.016 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.009+0000 7f8860800640 1 --2- 172.21.15.119:0/3879710915 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8858108ed0 0x7f8858109350 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f884c009900 tx=0x7f884c02bc40 comp rx=0 tx=0).stop 2024-03-28T00:25:40.016 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.009+0000 7f8860800640 1 -- 172.21.15.119:0/3879710915 shutdown_connections 2024-03-28T00:25:40.016 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.009+0000 7f8860800640 1 --2- 172.21.15.119:0/3879710915 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8858109890 0x7f8858110120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:40.016 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.009+0000 7f8860800640 1 --2- 172.21.15.119:0/3879710915 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8858108ed0 0x7f8858109350 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:40.016 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.009+0000 7f8860800640 1 --2- 172.21.15.119:0/3879710915 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8858107ce0 0x7f8858108100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:40.016 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.009+0000 7f8860800640 1 -- 172.21.15.119:0/3879710915 >> 172.21.15.119:0/3879710915 conn(0x7f8858076cb0 msgr2=0x7f88580790d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:25:40.016 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.009+0000 7f8860800640 1 -- 172.21.15.119:0/3879710915 shutdown_connections 2024-03-28T00:25:40.016 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.009+0000 7f8860800640 1 -- 172.21.15.119:0/3879710915 wait complete. 2024-03-28T00:25:40.019 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.010+0000 7f8860800640 1 Processor -- start 2024-03-28T00:25:40.019 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.010+0000 7f8860800640 1 -- start start 2024-03-28T00:25:40.019 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.010+0000 7f8860800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8858107ce0 0x7f885819dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:40.019 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.011+0000 7f8860800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8858108ed0 0x7f885819e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:40.019 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.011+0000 7f8860800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8858109890 0x7f88581a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:40.020 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.011+0000 7f8860800640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8858113990 con 0x7f8858108ed0 2024-03-28T00:25:40.020 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.011+0000 7f8860800640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8858113810 con 0x7f8858107ce0 2024-03-28T00:25:40.020 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.011+0000 7f8860800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8858113b10 con 0x7f8858109890 2024-03-28T00:25:40.020 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.011+0000 7f885de00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8858107ce0 0x7f885819dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:40.020 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.011+0000 7f885de00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8858107ce0 0x7f885819dfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.119:45412/0 (socket says 172.21.15.119:45412) 2024-03-28T00:25:40.020 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.011+0000 7f885e800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8858109890 0x7f88581a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:40.020 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.011+0000 7f885de00640 1 -- 172.21.15.119:0/4095937639 learned_addr learned my addr 172.21.15.119:0/4095937639 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:25:40.020 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.011+0000 7f885d400640 1 --2- 172.21.15.119:0/4095937639 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8858108ed0 0x7f885819e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:40.020 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.011+0000 7f885de00640 1 -- 172.21.15.119:0/4095937639 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8858109890 msgr2=0x7f88581a5550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:40.020 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.011+0000 7f885de00640 1 --2- 172.21.15.119:0/4095937639 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8858109890 0x7f88581a5550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:40.020 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.011+0000 7f885de00640 1 -- 172.21.15.119:0/4095937639 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8858108ed0 msgr2=0x7f885819e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:40.020 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.011+0000 7f885de00640 1 --2- 172.21.15.119:0/4095937639 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8858108ed0 0x7f885819e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:40.021 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.011+0000 7f885de00640 1 -- 172.21.15.119:0/4095937639 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88581a5bc0 con 0x7f8858107ce0 2024-03-28T00:25:40.021 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.011+0000 7f885de00640 1 --2- 172.21.15.119:0/4095937639 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8858107ce0 0x7f885819dfa0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f884400e9c0 tx=0x7f884400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:25:40.021 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.012+0000 7f8852a00640 1 -- 172.21.15.119:0/4095937639 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f884400cd80 con 0x7f8858107ce0 2024-03-28T00:25:40.021 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.012+0000 7f8852a00640 1 -- 172.21.15.119:0/4095937639 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f88440035f0 con 0x7f8858107ce0 2024-03-28T00:25:40.021 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.012+0000 7f8852a00640 1 -- 172.21.15.119:0/4095937639 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8844010690 con 0x7f8858107ce0 2024-03-28T00:25:40.021 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.012+0000 7f8860800640 1 -- 172.21.15.119:0/4095937639 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88581a5eb0 con 0x7f8858107ce0 2024-03-28T00:25:40.021 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.012+0000 7f8860800640 1 -- 172.21.15.119:0/4095937639 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88581a62c0 con 0x7f8858107ce0 2024-03-28T00:25:40.021 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.013+0000 7f8860800640 1 -- 172.21.15.119:0/4095937639 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f885810efa0 con 0x7f8858107ce0 2024-03-28T00:25:40.022 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.014+0000 7f8852a00640 1 -- 172.21.15.119:0/4095937639 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 18) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f88440037a0 con 0x7f8858107ce0 2024-03-28T00:25:40.022 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.017+0000 7f8852a00640 1 --2- 172.21.15.119:0/4095937639 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f88400a2800 0x7f88400a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:40.022 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.017+0000 7f8852a00640 1 -- 172.21.15.119:0/4095937639 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f88440ced70 con 0x7f8858107ce0 2024-03-28T00:25:40.022 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.017+0000 7f885d400640 1 --2- 172.21.15.119:0/4095937639 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f88400a2800 0x7f88400a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:40.023 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.017+0000 7f885d400640 1 --2- 172.21.15.119:0/4095937639 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f88400a2800 0x7f88400a4cc0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f884c02c250 tx=0x7f884c02cf90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:25:40.026 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.020+0000 7f8852a00640 1 -- 172.21.15.119:0/4095937639 <== mon.2 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f884409be90 con 0x7f8858107ce0 2024-03-28T00:25:40.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:25:39 smithi165 ceph-mon[36302]: from='client.? 172.21.15.119:0/4181858655' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-28T00:25:40.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:25:39 smithi165 ceph-mon[36302]: from='client.? 172.21.15.119:0/4058602143' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T00:25:40.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:25:39 smithi165 ceph-mon[36302]: from='client.? 172.21.15.119:0/3095247401' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-28T00:25:40.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:25:39 smithi119 ceph-mon[31518]: from='client.? 172.21.15.119:0/4181858655' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-28T00:25:40.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:25:39 smithi119 ceph-mon[31518]: from='client.? 172.21.15.119:0/4058602143' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T00:25:40.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:25:39 smithi119 ceph-mon[31518]: from='client.? 172.21.15.119:0/3095247401' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-28T00:25:40.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:25:39 smithi132 ceph-mon[35941]: from='client.? 172.21.15.119:0/4181858655' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-28T00:25:40.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:25:39 smithi132 ceph-mon[35941]: from='client.? 172.21.15.119:0/4058602143' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T00:25:40.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:25:39 smithi132 ceph-mon[35941]: from='client.? 172.21.15.119:0/3095247401' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-28T00:25:40.260 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.254+0000 7f8860800640 1 -- 172.21.15.119:0/4095937639 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f885802d070 con 0x7f8858107ce0 2024-03-28T00:25:40.261 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.255+0000 7f8852a00640 1 -- 172.21.15.119:0/4095937639 <== mon.2 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+11473 (secure 0 0 0) 0x7f885802d070 con 0x7f8858107ce0 2024-03-28T00:25:40.261 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-28T00:25:40.261 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14400,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/3988635716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":3988635716},{"type":"v1","addr":"172.21.15.119:6837","nonce":3988635716}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6839/1595343240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1595343240},{"type":"v1","addr":"172.21.15.119:6839","nonce":1595343240}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-28T00:23:56.215355+0000","modified":"2024-03-28T00:24:04.459985+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24581,"mds_1":24550,"mds_2":24530,"mds_3":24523,"mds_4":24511},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14466":{"gid":14466,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6841/1678451188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6840","nonce":1678451188},{"type":"v1","addr":"172.21.15.119:6841","nonce":1678451188}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24445":{"gid":24445,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/609963168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":609963168},{"type":"v1","addr":"172.21.15.165:6833","nonce":609963168}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24455":{"gid":24455,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.132:6833/1179016670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":1179016670},{"type":"v1","addr":"172.21.15.132:6833","nonce":1179016670}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24473":{"gid":24473,"name":"j","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6835/744950487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":744950487},{"type":"v1","addr":"172.21.15.119:6835","nonce":744950487}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24500":{"gid":24500,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.132:6835/356981946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":356981946},{"type":"v1","addr":"172.21.15.132:6835","nonce":356981946}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24511":{"gid":24511,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.165:6835/1947980102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":1947980102},{"type":"v1","addr":"172.21.15.165:6835","nonce":1947980102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24523":{"gid":24523,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.165:6837/1531193032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":1531193032},{"type":"v1","addr":"172.21.15.165:6837","nonce":1531193032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24530":{"gid":24530,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.132:6837/3383082967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6836","nonce":3383082967},{"type":"v1","addr":"172.21.15.132:6837","nonce":3383082967}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24550":{"gid":24550,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.165:6839/3004684592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":3004684592},{"type":"v1","addr":"172.21.15.165:6839","nonce":3004684592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.132:6839/3305822525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6838","nonce":3305822525},{"type":"v1","addr":"172.21.15.132:6839","nonce":3305822525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24523,24530,24550,24511,24581]},"id":1}]} 2024-03-28T00:25:40.261 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 26 2024-03-28T00:25:40.264 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.257+0000 7f8860800640 1 -- 172.21.15.119:0/4095937639 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f88400a2800 msgr2=0x7f88400a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:40.264 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.257+0000 7f8860800640 1 --2- 172.21.15.119:0/4095937639 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f88400a2800 0x7f88400a4cc0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f884c02c250 tx=0x7f884c02cf90 comp rx=0 tx=0).stop 2024-03-28T00:25:40.264 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.257+0000 7f8860800640 1 -- 172.21.15.119:0/4095937639 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8858107ce0 msgr2=0x7f885819dfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:40.264 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.257+0000 7f8860800640 1 --2- 172.21.15.119:0/4095937639 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8858107ce0 0x7f885819dfa0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f884400e9c0 tx=0x7f884400ee80 comp rx=0 tx=0).stop 2024-03-28T00:25:40.264 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.257+0000 7f8860800640 1 -- 172.21.15.119:0/4095937639 shutdown_connections 2024-03-28T00:25:40.264 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.257+0000 7f8860800640 1 --2- 172.21.15.119:0/4095937639 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f88400a2800 0x7f88400a4cc0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:40.264 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.257+0000 7f8860800640 1 --2- 172.21.15.119:0/4095937639 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8858109890 0x7f88581a5550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:40.264 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.257+0000 7f8860800640 1 --2- 172.21.15.119:0/4095937639 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8858108ed0 0x7f885819e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:40.264 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.257+0000 7f8860800640 1 --2- 172.21.15.119:0/4095937639 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8858107ce0 0x7f885819dfa0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:40.264 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.257+0000 7f8860800640 1 -- 172.21.15.119:0/4095937639 >> 172.21.15.119:0/4095937639 conn(0x7f8858076cb0 msgr2=0x7f885810c170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:25:40.264 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.257+0000 7f8860800640 1 -- 172.21.15.119:0/4095937639 shutdown_connections 2024-03-28T00:25:40.264 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.257+0000 7f8860800640 1 -- 172.21.15.119:0/4095937639 wait complete. 2024-03-28T00:25:40.283 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-03-28T00:23:56.215355+0000', 'modified': '2024-03-28T00:24:04.459985+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24581, 'mds_1': 24550, 'mds_2': 24530, 'mds_3': 24523, 'mds_4': 24511}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14466': {'gid': 14466, 'name': 'a', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.119:6841/1678451188', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6840', 'nonce': 1678451188}, {'type': 'v1', 'addr': '172.21.15.119:6841', 'nonce': 1678451188}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24445': {'gid': 24445, 'name': 'l', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.165:6833/609963168', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6832', 'nonce': 609963168}, {'type': 'v1', 'addr': '172.21.15.165:6833', 'nonce': 609963168}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24455': {'gid': 24455, 'name': 'k', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.132:6833/1179016670', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.132:6832', 'nonce': 1179016670}, {'type': 'v1', 'addr': '172.21.15.132:6833', 'nonce': 1179016670}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24473': {'gid': 24473, 'name': 'j', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.119:6835/744950487', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6834', 'nonce': 744950487}, {'type': 'v1', 'addr': '172.21.15.119:6835', 'nonce': 744950487}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24500': {'gid': 24500, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.132:6835/356981946', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.132:6834', 'nonce': 356981946}, {'type': 'v1', 'addr': '172.21.15.132:6835', 'nonce': 356981946}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24511': {'gid': 24511, 'name': 'f', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.165:6835/1947980102', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6834', 'nonce': 1947980102}, {'type': 'v1', 'addr': '172.21.15.165:6835', 'nonce': 1947980102}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24523': {'gid': 24523, 'name': 'c', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.165:6837/1531193032', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6836', 'nonce': 1531193032}, {'type': 'v1', 'addr': '172.21.15.165:6837', 'nonce': 1531193032}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24530': {'gid': 24530, 'name': 'h', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.132:6837/3383082967', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.132:6836', 'nonce': 3383082967}, {'type': 'v1', 'addr': '172.21.15.132:6837', 'nonce': 3383082967}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24550': {'gid': 24550, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.165:6839/3004684592', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6838', 'nonce': 3004684592}, {'type': 'v1', 'addr': '172.21.15.165:6839', 'nonce': 3004684592}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24581': {'gid': 24581, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.132:6839/3305822525', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.132:6838', 'nonce': 3305822525}, {'type': 'v1', 'addr': '172.21.15.132:6839', 'nonce': 3305822525}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24581, 'qdb_cluster': [24523, 24530, 24550, 24511, 24581]} 2024-03-28T00:25:40.283 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-03-28T00:25:40.284 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.f status 2024-03-28T00:25:40.492 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.486+0000 7f5c8c200640 1 -- 172.21.15.119:0/3287790309 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5c84104ad0 msgr2=0x7f5c84104f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:40.492 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.486+0000 7f5c8c200640 1 --2- 172.21.15.119:0/3287790309 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5c84104ad0 0x7f5c84104f50 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f5c70009900 tx=0x7f5c7002bc40 comp rx=0 tx=0).stop 2024-03-28T00:25:40.493 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.486+0000 7f5c8c200640 1 -- 172.21.15.119:0/3287790309 shutdown_connections 2024-03-28T00:25:40.493 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.486+0000 7f5c8c200640 1 --2- 172.21.15.119:0/3287790309 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c84105490 0x7f5c8410bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:40.493 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.486+0000 7f5c8c200640 1 --2- 172.21.15.119:0/3287790309 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5c84104ad0 0x7f5c84104f50 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:40.493 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.486+0000 7f5c8c200640 1 --2- 172.21.15.119:0/3287790309 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5c841038e0 0x7f5c84103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:40.493 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.486+0000 7f5c8c200640 1 -- 172.21.15.119:0/3287790309 >> 172.21.15.119:0/3287790309 conn(0x7f5c840fee50 msgr2=0x7f5c84101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:25:40.493 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.486+0000 7f5c8c200640 1 -- 172.21.15.119:0/3287790309 shutdown_connections 2024-03-28T00:25:40.493 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.486+0000 7f5c8c200640 1 -- 172.21.15.119:0/3287790309 wait complete. 2024-03-28T00:25:40.495 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.487+0000 7f5c8c200640 1 Processor -- start 2024-03-28T00:25:40.496 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.487+0000 7f5c8c200640 1 -- start start 2024-03-28T00:25:40.496 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.487+0000 7f5c8c200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c841038e0 0x7f5c84199be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:40.496 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.488+0000 7f5c8c200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5c84104ad0 0x7f5c8419a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:40.496 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.488+0000 7f5c8c200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5c84105490 0x7f5c841a1190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:40.496 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.488+0000 7f5c8c200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c8410f590 con 0x7f5c841038e0 2024-03-28T00:25:40.496 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.488+0000 7f5c8c200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c8410f410 con 0x7f5c84105490 2024-03-28T00:25:40.496 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.488+0000 7f5c8c200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c8410f710 con 0x7f5c84104ad0 2024-03-28T00:25:40.496 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.488+0000 7f5c89800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c841038e0 0x7f5c84199be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:40.496 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.488+0000 7f5c88e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5c84104ad0 0x7f5c8419a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:40.496 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.488+0000 7f5c89800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c841038e0 0x7f5c84199be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:60220/0 (socket says 172.21.15.119:60220) 2024-03-28T00:25:40.497 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.488+0000 7f5c88e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5c84104ad0 0x7f5c8419a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.119:55580/0 (socket says 172.21.15.119:55580) 2024-03-28T00:25:40.497 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.488+0000 7f5c89800640 1 -- 172.21.15.119:0/3167188108 learned_addr learned my addr 172.21.15.119:0/3167188108 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:25:40.497 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.488+0000 7f5c8a200640 1 --2- 172.21.15.119:0/3167188108 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5c84105490 0x7f5c841a1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:40.497 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.488+0000 7f5c89800640 1 -- 172.21.15.119:0/3167188108 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5c84104ad0 msgr2=0x7f5c8419a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:40.497 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.488+0000 7f5c89800640 1 --2- 172.21.15.119:0/3167188108 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5c84104ad0 0x7f5c8419a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:40.497 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.488+0000 7f5c89800640 1 -- 172.21.15.119:0/3167188108 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5c84105490 msgr2=0x7f5c841a1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:40.497 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.488+0000 7f5c89800640 1 --2- 172.21.15.119:0/3167188108 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5c84105490 0x7f5c841a1190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:40.497 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.488+0000 7f5c89800640 1 -- 172.21.15.119:0/3167188108 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c841a1800 con 0x7f5c841038e0 2024-03-28T00:25:40.497 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.488+0000 7f5c88e00640 1 --2- 172.21.15.119:0/3167188108 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5c84104ad0 0x7f5c8419a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T00:25:40.497 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.488+0000 7f5c8a200640 1 --2- 172.21.15.119:0/3167188108 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5c84105490 0x7f5c841a1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T00:25:40.497 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.489+0000 7f5c89800640 1 --2- 172.21.15.119:0/3167188108 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c841038e0 0x7f5c84199be0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f5c7800e9c0 tx=0x7f5c7800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:25:40.497 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.489+0000 7f5c82000640 1 -- 172.21.15.119:0/3167188108 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c7800cd80 con 0x7f5c841038e0 2024-03-28T00:25:40.497 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.489+0000 7f5c82000640 1 -- 172.21.15.119:0/3167188108 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5c780035f0 con 0x7f5c841038e0 2024-03-28T00:25:40.497 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.489+0000 7f5c82000640 1 -- 172.21.15.119:0/3167188108 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c78010690 con 0x7f5c841038e0 2024-03-28T00:25:40.498 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.489+0000 7f5c8c200640 1 -- 172.21.15.119:0/3167188108 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c841a1af0 con 0x7f5c841038e0 2024-03-28T00:25:40.498 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.489+0000 7f5c8c200640 1 -- 172.21.15.119:0/3167188108 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c841a2060 con 0x7f5c841038e0 2024-03-28T00:25:40.505 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.492+0000 7f5c82000640 1 -- 172.21.15.119:0/3167188108 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 18) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f5c780037a0 con 0x7f5c841038e0 2024-03-28T00:25:40.505 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.492+0000 7f5c8efb6740 1 Processor -- start 2024-03-28T00:25:40.505 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.492+0000 7f5c8efb6740 1 -- start start 2024-03-28T00:25:40.505 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.493+0000 7f5c8efb6740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558976c54ba0 0x558976c54fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:40.505 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.493+0000 7f5c8efb6740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558976c55590 0x558976c5a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:40.505 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.493+0000 7f5c8efb6740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558976c5a700 0x558976c5cb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:40.505 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.493+0000 7f5c8efb6740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x558976bb9c90 con 0x558976c5a700 2024-03-28T00:25:40.505 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.493+0000 7f5c8efb6740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x558976b62980 con 0x558976c55590 2024-03-28T00:25:40.506 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.493+0000 7f5c8efb6740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x558976bb5d30 con 0x558976c54ba0 2024-03-28T00:25:40.506 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.493+0000 7f5c8a200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558976c54ba0 0x558976c54fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:40.506 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.493+0000 7f5c8a200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558976c54ba0 0x558976c54fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.119:55582/0 (socket says 172.21.15.119:55582) 2024-03-28T00:25:40.506 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.493+0000 7f5c8a200640 1 -- 172.21.15.119:0/4091717183 learned_addr learned my addr 172.21.15.119:0/4091717183 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:25:40.506 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.493+0000 7f5c89800640 1 --2- 172.21.15.119:0/4091717183 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558976c55590 0x558976c5a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:40.506 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.493+0000 7f5c88e00640 1 --2- 172.21.15.119:0/4091717183 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558976c5a700 0x558976c5cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:40.506 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.493+0000 7f5c8a200640 1 -- 172.21.15.119:0/4091717183 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558976c55590 msgr2=0x558976c5a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T00:25:40.506 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.493+0000 7f5c8a200640 1 --2- 172.21.15.119:0/4091717183 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558976c55590 0x558976c5a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:40.506 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.493+0000 7f5c8a200640 1 -- 172.21.15.119:0/4091717183 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558976c5a700 msgr2=0x558976c5cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T00:25:40.506 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.493+0000 7f5c8a200640 1 --2- 172.21.15.119:0/4091717183 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558976c5a700 0x558976c5cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:40.506 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.493+0000 7f5c8a200640 1 -- 172.21.15.119:0/4091717183 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558976b7a510 con 0x558976c54ba0 2024-03-28T00:25:40.506 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.493+0000 7f5c88e00640 1 --2- 172.21.15.119:0/4091717183 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558976c5a700 0x558976c5cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T00:25:40.507 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.494+0000 7f5c8a200640 1 --2- 172.21.15.119:0/4091717183 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558976c54ba0 0x558976c54fc0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f5c7c0117e0 tx=0x7f5c7c011ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e5ffdee36b089652 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:25:40.507 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.494+0000 7f5c82000640 1 --2- 172.21.15.119:0/3167188108 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f5c6c0a2880 0x7f5c6c0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:40.507 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.494+0000 7f5c76a00640 1 -- 172.21.15.119:0/4091717183 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c7c011310 con 0x558976c54ba0 2024-03-28T00:25:40.507 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.494+0000 7f5c76a00640 1 -- 172.21.15.119:0/4091717183 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5c7c003e50 con 0x558976c54ba0 2024-03-28T00:25:40.507 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.494+0000 7f5c76a00640 1 -- 172.21.15.119:0/4091717183 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c7c004570 con 0x558976c54ba0 2024-03-28T00:25:40.507 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.494+0000 7f5c82000640 1 -- 172.21.15.119:0/3167188108 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f5c780ced70 con 0x7f5c841038e0 2024-03-28T00:25:40.507 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.494+0000 7f5c89800640 1 --2- 172.21.15.119:0/3167188108 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f5c6c0a2880 0x7f5c6c0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:40.507 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.495+0000 7f5c8efb6740 1 -- 172.21.15.119:0/4091717183 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558976c54ba0 msgr2=0x558976c54fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:40.507 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.495+0000 7f5c89800640 1 --2- 172.21.15.119:0/3167188108 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f5c6c0a2880 0x7f5c6c0a4d40 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f5c780980e0 tx=0x7f5c780d27b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:25:40.507 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.495+0000 7f5c8efb6740 1 --2- 172.21.15.119:0/4091717183 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558976c54ba0 0x558976c54fc0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f5c7c0117e0 tx=0x7f5c7c011ca0 comp rx=0 tx=0).stop 2024-03-28T00:25:40.507 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.495+0000 7f5c8efb6740 1 -- 172.21.15.119:0/4091717183 shutdown_connections 2024-03-28T00:25:40.507 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.495+0000 7f5c8efb6740 1 --2- 172.21.15.119:0/4091717183 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558976c5a700 0x558976c5cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:40.508 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.495+0000 7f5c8efb6740 1 --2- 172.21.15.119:0/4091717183 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558976c55590 0x558976c5a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:40.508 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.495+0000 7f5c8efb6740 1 --2- 172.21.15.119:0/4091717183 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558976c54ba0 0x558976c54fc0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:40.508 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.495+0000 7f5c8efb6740 1 -- 172.21.15.119:0/4091717183 >> 172.21.15.119:0/4091717183 conn(0x558976c50110 msgr2=0x558976c50510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:25:40.508 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.495+0000 7f5c8efb6740 1 -- 172.21.15.119:0/4091717183 shutdown_connections 2024-03-28T00:25:40.508 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.495+0000 7f5c8efb6740 1 -- 172.21.15.119:0/4091717183 wait complete. 2024-03-28T00:25:40.508 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.495+0000 7f5c8efb6740 1 -- start start 2024-03-28T00:25:40.508 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.495+0000 7f5c8efb6740 1 Processor -- start 2024-03-28T00:25:40.508 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.496+0000 7f5c8efb6740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558976c54ba0 0x558976c5f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:40.508 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.496+0000 7f5c8efb6740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558976c55590 0x558976c5f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:40.508 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.496+0000 7f5c8efb6740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558976c5a700 0x558976c66640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:40.508 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.496+0000 7f5c8efb6740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x558976c5d6b0 con 0x558976c5a700 2024-03-28T00:25:40.508 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.496+0000 7f5c8efb6740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x558976b75dd0 con 0x558976c54ba0 2024-03-28T00:25:40.508 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.496+0000 7f5c8efb6740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x558976c5d830 con 0x558976c55590 2024-03-28T00:25:40.508 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.496+0000 7f5c8efb6740 10 client.0 _ll_register_callbacks cb 0x5589768cdac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T00:25:40.509 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.496+0000 7f5c8a200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558976c54ba0 0x558976c5f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:40.509 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.496+0000 7f5c8a200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558976c54ba0 0x558976c5f0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.119:45452/0 (socket says 172.21.15.119:45452) 2024-03-28T00:25:40.509 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.496+0000 7f5c8a200640 1 -- 172.21.15.119:0/2367453577 learned_addr learned my addr 172.21.15.119:0/2367453577 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:25:40.509 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.496+0000 7f5c88e00640 1 --2- 172.21.15.119:0/2367453577 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558976c55590 0x558976c5f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:40.509 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.496+0000 7f5c8a200640 1 --2- 172.21.15.119:0/2367453577 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558976c5a700 0x558976c66640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:40.509 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.496+0000 7f5c88e00640 1 -- 172.21.15.119:0/2367453577 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558976c54ba0 msgr2=0x558976c5f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T00:25:40.509 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.497+0000 7f5c88e00640 1 --2- 172.21.15.119:0/2367453577 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558976c54ba0 0x558976c5f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:40.509 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.497+0000 7f5c88e00640 1 -- 172.21.15.119:0/2367453577 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558976c5a700 msgr2=0x558976c66640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T00:25:40.509 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.497+0000 7f5c88e00640 1 --2- 172.21.15.119:0/2367453577 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558976c5a700 0x558976c66640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:40.509 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.497+0000 7f5c88e00640 1 -- 172.21.15.119:0/2367453577 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5589768ce2d0 con 0x558976c55590 2024-03-28T00:25:40.509 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.497+0000 7f5c88e00640 1 --2- 172.21.15.119:0/2367453577 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558976c55590 0x558976c5f600 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f5c70036040 tx=0x7f5c70004970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=478ea6d3ae9bc3a2 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:25:40.509 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.497+0000 7f5c6b400640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-03-28T00:25:40.509 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.497+0000 7f5c6b400640 1 -- 172.21.15.119:0/2367453577 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c70005d50 con 0x558976c55590 2024-03-28T00:25:40.510 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.497+0000 7f5c6b400640 1 -- 172.21.15.119:0/2367453577 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f5c70004ef0 con 0x558976c55590 2024-03-28T00:25:40.510 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.497+0000 7f5c8efb6740 1 -- 172.21.15.119:0/2367453577 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5c7c003e50 con 0x558976c55590 2024-03-28T00:25:40.510 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.497+0000 7f5c6b400640 1 -- 172.21.15.119:0/2367453577 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5c7002cd40 con 0x558976c55590 2024-03-28T00:25:40.510 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.497+0000 7f5c8efb6740 10 client.24985 fetch_fsmap learned FSMap version 26 2024-03-28T00:25:40.510 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.497+0000 7f5c8efb6740 1 -- 172.21.15.119:0/2367453577 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x5589769986b0 con 0x558976c55590 2024-03-28T00:25:40.510 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.498+0000 7f5c6b400640 1 -- 172.21.15.119:0/2367453577 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5c7c003e50 con 0x558976c55590 2024-03-28T00:25:40.510 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.498+0000 7f5c6b400640 1 -- 172.21.15.119:0/2367453577 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 26) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f5c70003780 con 0x558976c55590 2024-03-28T00:25:40.510 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.498+0000 7f5c8efb6740 10 client.24985 fetch_fsmap finished waiting for FSMap version 26 2024-03-28T00:25:40.510 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.498+0000 7f5c8efb6740 10 client.24985 resolve_mds: resolved name 'f' to daemon mds.f 2024-03-28T00:25:40.510 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.498+0000 7f5c8efb6740 20 client.24985 populate_metadata read hostname 'smithi119' 2024-03-28T00:25:40.510 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.498+0000 7f5c8efb6740 1 --2- 172.21.15.119:0/2367453577 >> [v2:172.21.15.165:6834/1947980102,v1:172.21.15.165:6835/1947980102] conn(0x558976c6b880 0x558976c6a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:40.510 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.498+0000 7f5c8efb6740 4 client.24985 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-28T00:25:40.510 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.498+0000 7f5c8efb6740 1 -- 172.21.15.119:0/2367453577 --> [v2:172.21.15.165:6834/1947980102,v1:172.21.15.165:6835/1947980102] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558976b5b520 con 0x558976c6b880 2024-03-28T00:25:40.510 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.498+0000 7f5c8a200640 1 --2- 172.21.15.119:0/2367453577 >> [v2:172.21.15.165:6834/1947980102,v1:172.21.15.165:6835/1947980102] conn(0x558976c6b880 0x558976c6a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:40.511 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.499+0000 7f5c6b400640 10 client.24985 ms_handle_connect on v2:172.21.15.165:6834/1947980102 2024-03-28T00:25:40.511 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.499+0000 7f5c8a200640 1 --2- 172.21.15.119:0/2367453577 >> [v2:172.21.15.165:6834/1947980102,v1:172.21.15.165:6835/1947980102] conn(0x558976c6b880 0x558976c6a930 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=95813fd45b0895d3 server_cookie=7fcb5f05826b5511 in_seq=0 out_seq=0 2024-03-28T00:25:40.512 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.499+0000 7f5c6b400640 1 -- 172.21.15.119:0/2367453577 <== mds.4 v2:172.21.15.165:6834/1947980102 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x558976b5b520 con 0x558976c6b880 2024-03-28T00:25:40.512 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.499+0000 7f5c6b400640 10 client.24985 handle_command_reply: tid=0 2024-03-28T00:25:40.512 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.499+0000 7f5c8efb6740 1 client.24985 shutdown 2024-03-28T00:25:40.512 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.500+0000 7f5c8efb6740 1 -- 172.21.15.119:0/2367453577 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558976c55590 msgr2=0x558976c5f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:40.512 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.500+0000 7f5c8efb6740 1 --2- 172.21.15.119:0/2367453577 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558976c55590 0x558976c5f600 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f5c70036040 tx=0x7f5c70004970 comp rx=0 tx=0).stop 2024-03-28T00:25:40.512 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.500+0000 7f5c8efb6740 1 -- 172.21.15.119:0/2367453577 shutdown_connections 2024-03-28T00:25:40.512 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.500+0000 7f5c8efb6740 1 --2- 172.21.15.119:0/2367453577 >> [v2:172.21.15.165:6834/1947980102,v1:172.21.15.165:6835/1947980102] conn(0x558976c6b880 0x558976c6a930 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:40.513 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.500+0000 7f5c8efb6740 1 --2- 172.21.15.119:0/2367453577 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558976c5a700 0x558976c66640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:40.513 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.500+0000 7f5c8efb6740 1 --2- 172.21.15.119:0/2367453577 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558976c55590 0x558976c5f600 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:40.513 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.500+0000 7f5c8efb6740 1 --2- 172.21.15.119:0/2367453577 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558976c54ba0 0x558976c5f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:40.513 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.500+0000 7f5c6b400640 0 client.24985 ms_handle_reset on v2:172.21.15.165:6834/1947980102 2024-03-28T00:25:40.513 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.500+0000 7f5c8efb6740 1 -- 172.21.15.119:0/2367453577 >> 172.21.15.119:0/2367453577 conn(0x558976c50110 msgr2=0x558976c4f310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:25:40.513 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.500+0000 7f5c8efb6740 1 -- 172.21.15.119:0/2367453577 shutdown_connections 2024-03-28T00:25:40.513 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.500+0000 7f5c8efb6740 1 -- 172.21.15.119:0/2367453577 wait complete. 2024-03-28T00:25:40.513 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.500+0000 7f5c8efb6740 20 client.24985 trim_cache size 0 max 16384 2024-03-28T00:25:40.525 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.518+0000 7f5c8efb6740 1 Processor -- start 2024-03-28T00:25:40.526 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.518+0000 7f5c8efb6740 1 -- start start 2024-03-28T00:25:40.526 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.518+0000 7f5c8efb6740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558976c54ba0 0x558976c56030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:40.526 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.518+0000 7f5c8efb6740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558976c55590 0x558976c66600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:40.526 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.518+0000 7f5c8efb6740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558976c5a700 0x558976c6a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:40.526 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.518+0000 7f5c8efb6740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x558976c6b0a0 con 0x558976c55590 2024-03-28T00:25:40.526 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.518+0000 7f5c8efb6740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x558976c6b700 con 0x558976c54ba0 2024-03-28T00:25:40.526 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.518+0000 7f5c8efb6740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x558976c6bd60 con 0x558976c5a700 2024-03-28T00:25:40.526 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.518+0000 7f5c88e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558976c55590 0x558976c66600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:40.526 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.518+0000 7f5c88e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558976c55590 0x558976c66600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:60256/0 (socket says 172.21.15.119:60256) 2024-03-28T00:25:40.526 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.518+0000 7f5c88e00640 1 -- 172.21.15.119:0/1163145117 learned_addr learned my addr 172.21.15.119:0/1163145117 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:25:40.526 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.519+0000 7f5c8a200640 1 --2- 172.21.15.119:0/1163145117 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558976c54ba0 0x558976c56030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:40.526 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.519+0000 7f5c88e00640 1 -- 172.21.15.119:0/1163145117 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558976c5a700 msgr2=0x558976c6a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T00:25:40.526 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.519+0000 7f5c8a200640 1 --2- 172.21.15.119:0/1163145117 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558976c5a700 0x558976c6a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:40.526 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.519+0000 7f5c88e00640 1 --2- 172.21.15.119:0/1163145117 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558976c5a700 0x558976c6a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:40.527 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.519+0000 7f5c88e00640 1 -- 172.21.15.119:0/1163145117 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558976c54ba0 msgr2=0x558976c56030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T00:25:40.527 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.519+0000 7f5c88e00640 1 --2- 172.21.15.119:0/1163145117 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558976c54ba0 0x558976c56030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:40.527 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.519+0000 7f5c88e00640 1 -- 172.21.15.119:0/1163145117 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558976c56570 con 0x558976c55590 2024-03-28T00:25:40.527 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.519+0000 7f5c8a200640 1 --2- 172.21.15.119:0/1163145117 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558976c54ba0 0x558976c56030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T00:25:40.527 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.519+0000 7f5c88e00640 1 --2- 172.21.15.119:0/1163145117 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558976c55590 0x558976c66600 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f5c7002e010 tx=0x7f5c7002bc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4302f43be203bad server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:25:40.527 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.519+0000 7f5c76000640 1 -- 172.21.15.119:0/1163145117 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c70000e60 con 0x558976c55590 2024-03-28T00:25:40.527 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.519+0000 7f5c76000640 1 -- 172.21.15.119:0/1163145117 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5c7002cd40 con 0x558976c55590 2024-03-28T00:25:40.527 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.519+0000 7f5c76000640 1 -- 172.21.15.119:0/1163145117 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c70005240 con 0x558976c55590 2024-03-28T00:25:40.532 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.521+0000 7f5c8efb6740 1 -- 172.21.15.119:0/1163145117 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558976c55590 msgr2=0x558976c66600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:40.532 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.521+0000 7f5c8efb6740 1 --2- 172.21.15.119:0/1163145117 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558976c55590 0x558976c66600 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f5c7002e010 tx=0x7f5c7002bc40 comp rx=0 tx=0).stop 2024-03-28T00:25:40.532 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.521+0000 7f5c8efb6740 1 -- 172.21.15.119:0/1163145117 shutdown_connections 2024-03-28T00:25:40.532 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.521+0000 7f5c8efb6740 1 --2- 172.21.15.119:0/1163145117 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558976c5a700 0x558976c6a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:40.532 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.521+0000 7f5c8efb6740 1 --2- 172.21.15.119:0/1163145117 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558976c55590 0x558976c66600 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:40.532 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.521+0000 7f5c8efb6740 1 --2- 172.21.15.119:0/1163145117 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558976c54ba0 0x558976c56030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:40.532 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.521+0000 7f5c8efb6740 1 -- 172.21.15.119:0/1163145117 >> 172.21.15.119:0/1163145117 conn(0x558976c50110 msgr2=0x558976c4f310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:25:40.532 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.521+0000 7f5c8efb6740 1 -- 172.21.15.119:0/1163145117 shutdown_connections 2024-03-28T00:25:40.532 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.521+0000 7f5c8efb6740 1 -- 172.21.15.119:0/1163145117 wait complete. 2024-03-28T00:25:40.532 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.521+0000 7f5c8efb6740 1 -- start start 2024-03-28T00:25:40.533 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.521+0000 7f5c8efb6740 1 Processor -- start 2024-03-28T00:25:40.533 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.521+0000 7f5c8efb6740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558976c54ba0 0x558976c62790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:40.533 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.521+0000 7f5c8efb6740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558976c55590 0x558976c62cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:40.533 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.521+0000 7f5c8efb6740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558976c5a700 0x558976c65b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:40.533 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.521+0000 7f5c8efb6740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x558976c6ca20 con 0x558976c54ba0 2024-03-28T00:25:40.533 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.521+0000 7f5c8efb6740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x558976c6c3c0 con 0x558976c5a700 2024-03-28T00:25:40.533 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.521+0000 7f5c8efb6740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x558976c6d080 con 0x558976c55590 2024-03-28T00:25:40.533 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.521+0000 7f5c8efb6740 10 client.0 _ll_register_callbacks cb 0x7f5c78005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T00:25:40.533 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.521+0000 7f5c88e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558976c55590 0x558976c62cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:40.533 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.521+0000 7f5c8a200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558976c54ba0 0x558976c62790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:40.533 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.521+0000 7f5c88e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558976c55590 0x558976c62cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.119:55600/0 (socket says 172.21.15.119:55600) 2024-03-28T00:25:40.533 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.521+0000 7f5c88e00640 1 -- 172.21.15.119:0/3214656159 learned_addr learned my addr 172.21.15.119:0/3214656159 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:25:40.534 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.522+0000 7f5c8a200640 1 --2- 172.21.15.119:0/3214656159 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558976c5a700 0x558976c65b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:40.534 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.522+0000 7f5c88e00640 1 -- 172.21.15.119:0/3214656159 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558976c5a700 msgr2=0x558976c65b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T00:25:40.534 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.522+0000 7f5c88e00640 1 --2- 172.21.15.119:0/3214656159 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558976c5a700 0x558976c65b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:40.534 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.522+0000 7f5c88e00640 1 -- 172.21.15.119:0/3214656159 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558976c54ba0 msgr2=0x558976c62790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T00:25:40.534 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.522+0000 7f5c88e00640 1 --2- 172.21.15.119:0/3214656159 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558976c54ba0 0x558976c62790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:40.534 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.522+0000 7f5c88e00640 1 -- 172.21.15.119:0/3214656159 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558976c66170 con 0x558976c55590 2024-03-28T00:25:40.534 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.522+0000 7f5c8a200640 1 --2- 172.21.15.119:0/3214656159 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558976c5a700 0x558976c65b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T00:25:40.534 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.522+0000 7f5c88e00640 1 --2- 172.21.15.119:0/3214656159 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558976c55590 0x558976c62cd0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f5c70003c20 tx=0x7f5c70009520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ef333d735d45a562 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:25:40.534 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.522+0000 7f5c6b400640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-03-28T00:25:40.534 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.522+0000 7f5c6b400640 1 -- 172.21.15.119:0/3214656159 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c700037f0 con 0x558976c55590 2024-03-28T00:25:40.534 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.522+0000 7f5c6b400640 1 -- 172.21.15.119:0/3214656159 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f5c700051e0 con 0x558976c55590 2024-03-28T00:25:40.534 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.522+0000 7f5c8efb6740 1 -- 172.21.15.119:0/3214656159 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5c7002cd40 con 0x558976c55590 2024-03-28T00:25:40.534 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.522+0000 7f5c6b400640 1 -- 172.21.15.119:0/3214656159 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5c7006a050 con 0x558976c55590 2024-03-28T00:25:40.535 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.522+0000 7f5c8efb6740 10 client.24988 fetch_fsmap learned FSMap version 26 2024-03-28T00:25:40.535 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.522+0000 7f5c8efb6740 1 -- 172.21.15.119:0/3214656159 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x558976cc7180 con 0x558976c55590 2024-03-28T00:25:40.535 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.523+0000 7f5c6b400640 1 -- 172.21.15.119:0/3214656159 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5c7002cd40 con 0x558976c55590 2024-03-28T00:25:40.535 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.523+0000 7f5c6b400640 1 -- 172.21.15.119:0/3214656159 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 26) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f5c700349a0 con 0x558976c55590 2024-03-28T00:25:40.535 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.523+0000 7f5c8efb6740 10 client.24988 fetch_fsmap finished waiting for FSMap version 26 2024-03-28T00:25:40.535 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.523+0000 7f5c8efb6740 10 client.24988 resolve_mds: resolved name 'f' to daemon mds.f 2024-03-28T00:25:40.535 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.523+0000 7f5c8efb6740 20 client.24988 populate_metadata read hostname 'smithi119' 2024-03-28T00:25:40.535 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.523+0000 7f5c8efb6740 1 --2- 172.21.15.119:0/3214656159 >> [v2:172.21.15.165:6834/1947980102,v1:172.21.15.165:6835/1947980102] conn(0x558976ccb690 0x558976cc7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:40.535 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.523+0000 7f5c8efb6740 4 client.24988 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-28T00:25:40.535 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.523+0000 7f5c8efb6740 1 -- 172.21.15.119:0/3214656159 --> [v2:172.21.15.165:6834/1947980102,v1:172.21.15.165:6835/1947980102] -- command(tid 0: {"prefix": "status"}) v1 -- 0x558976bc71d0 con 0x558976ccb690 2024-03-28T00:25:40.535 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.523+0000 7f5c8a200640 1 --2- 172.21.15.119:0/3214656159 >> [v2:172.21.15.165:6834/1947980102,v1:172.21.15.165:6835/1947980102] conn(0x558976ccb690 0x558976cc7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:40.535 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.524+0000 7f5c8a200640 1 --2- 172.21.15.119:0/3214656159 >> [v2:172.21.15.165:6834/1947980102,v1:172.21.15.165:6835/1947980102] conn(0x558976ccb690 0x558976cc7400 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f3324c16897d4d99 server_cookie=f6e0acc7d560192f in_seq=0 out_seq=0 2024-03-28T00:25:40.535 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.524+0000 7f5c6b400640 10 client.24988 ms_handle_connect on v2:172.21.15.165:6834/1947980102 2024-03-28T00:25:40.535 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.524+0000 7f5c6b400640 1 -- 172.21.15.119:0/3214656159 <== mds.4 v2:172.21.15.165:6834/1947980102 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x558976bc71d0 con 0x558976ccb690 2024-03-28T00:25:40.536 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.524+0000 7f5c6b400640 10 client.24988 handle_command_reply: tid=0 2024-03-28T00:25:40.536 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.524+0000 7f5c8efb6740 1 client.24988 shutdown 2024-03-28T00:25:40.536 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.524+0000 7f5c8efb6740 1 -- 172.21.15.119:0/3214656159 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558976c55590 msgr2=0x558976c62cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:40.536 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.524+0000 7f5c8efb6740 1 --2- 172.21.15.119:0/3214656159 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558976c55590 0x558976c62cd0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f5c70003c20 tx=0x7f5c70009520 comp rx=0 tx=0).stop 2024-03-28T00:25:40.536 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.524+0000 7f5c8efb6740 1 -- 172.21.15.119:0/3214656159 shutdown_connections 2024-03-28T00:25:40.536 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.524+0000 7f5c8efb6740 1 --2- 172.21.15.119:0/3214656159 >> [v2:172.21.15.165:6834/1947980102,v1:172.21.15.165:6835/1947980102] conn(0x558976ccb690 0x558976cc7400 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:40.536 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.524+0000 7f5c8efb6740 1 --2- 172.21.15.119:0/3214656159 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558976c5a700 0x558976c65b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:40.536 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.524+0000 7f5c8efb6740 1 --2- 172.21.15.119:0/3214656159 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558976c55590 0x558976c62cd0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:40.536 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.524+0000 7f5c8efb6740 1 --2- 172.21.15.119:0/3214656159 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558976c54ba0 0x558976c62790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:40.536 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.524+0000 7f5c6b400640 0 client.24988 ms_handle_reset on v2:172.21.15.165:6834/1947980102 2024-03-28T00:25:40.536 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.524+0000 7f5c8efb6740 1 -- 172.21.15.119:0/3214656159 >> 172.21.15.119:0/3214656159 conn(0x558976c50110 msgr2=0x558976c4f310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:25:40.536 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.524+0000 7f5c8efb6740 1 -- 172.21.15.119:0/3214656159 shutdown_connections 2024-03-28T00:25:40.536 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.524+0000 7f5c8efb6740 1 -- 172.21.15.119:0/3214656159 wait complete. 2024-03-28T00:25:40.536 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.525+0000 7f5c8efb6740 20 client.24988 trim_cache size 0 max 16384 2024-03-28T00:25:40.537 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-03-28T00:25:40.537 INFO:teuthology.orchestra.run.smithi119.stdout: "cluster_fsid": "10f73b92-ec98-11ee-95d0-87774f69a715", 2024-03-28T00:25:40.537 INFO:teuthology.orchestra.run.smithi119.stdout: "whoami": 4, 2024-03-28T00:25:40.537 INFO:teuthology.orchestra.run.smithi119.stdout: "id": 24511, 2024-03-28T00:25:40.537 INFO:teuthology.orchestra.run.smithi119.stdout: "want_state": "up:active", 2024-03-28T00:25:40.537 INFO:teuthology.orchestra.run.smithi119.stdout: "state": "up:active", 2024-03-28T00:25:40.537 INFO:teuthology.orchestra.run.smithi119.stdout: "fs_name": "cephfs", 2024-03-28T00:25:40.538 INFO:teuthology.orchestra.run.smithi119.stdout: "rank_uptime": 97.003606219000005, 2024-03-28T00:25:40.538 INFO:teuthology.orchestra.run.smithi119.stdout: "mdsmap_epoch": 26, 2024-03-28T00:25:40.538 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_epoch": 87, 2024-03-28T00:25:40.538 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_epoch_barrier": 85, 2024-03-28T00:25:40.538 INFO:teuthology.orchestra.run.smithi119.stdout: "uptime": 130.14804675900001 2024-03-28T00:25:40.538 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-03-28T00:25:40.539 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.525+0000 7f5c8c200640 1 -- 172.21.15.119:0/3167188108 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f5c6c0a2880 msgr2=0x7f5c6c0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:40.539 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.525+0000 7f5c8c200640 1 --2- 172.21.15.119:0/3167188108 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f5c6c0a2880 0x7f5c6c0a4d40 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f5c780980e0 tx=0x7f5c780d27b0 comp rx=0 tx=0).stop 2024-03-28T00:25:40.539 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.525+0000 7f5c8c200640 1 -- 172.21.15.119:0/3167188108 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c841038e0 msgr2=0x7f5c84199be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:40.539 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.525+0000 7f5c8c200640 1 --2- 172.21.15.119:0/3167188108 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c841038e0 0x7f5c84199be0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f5c7800e9c0 tx=0x7f5c7800ee80 comp rx=0 tx=0).stop 2024-03-28T00:25:40.539 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.525+0000 7f5c8c200640 1 -- 172.21.15.119:0/3167188108 shutdown_connections 2024-03-28T00:25:40.539 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.525+0000 7f5c8c200640 1 --2- 172.21.15.119:0/3167188108 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f5c6c0a2880 0x7f5c6c0a4d40 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:40.539 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.525+0000 7f5c8c200640 1 --2- 172.21.15.119:0/3167188108 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5c84105490 0x7f5c841a1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:40.539 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.525+0000 7f5c8c200640 1 --2- 172.21.15.119:0/3167188108 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5c84104ad0 0x7f5c8419a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:40.539 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.525+0000 7f5c8c200640 1 --2- 172.21.15.119:0/3167188108 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c841038e0 0x7f5c84199be0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:40.540 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.525+0000 7f5c8c200640 1 -- 172.21.15.119:0/3167188108 >> 172.21.15.119:0/3167188108 conn(0x7f5c840fee50 msgr2=0x7f5c84107e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:25:40.540 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.525+0000 7f5c8c200640 1 -- 172.21.15.119:0/3167188108 shutdown_connections 2024-03-28T00:25:40.540 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.525+0000 7f5c8c200640 1 -- 172.21.15.119:0/3167188108 wait complete. 2024-03-28T00:25:40.548 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-03-28T00:25:40.749 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.743+0000 7fc4c6c00640 1 -- 172.21.15.119:0/1204103374 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc4c0108db0 msgr2=0x7fc4c0109230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:40.749 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.743+0000 7fc4c6c00640 1 --2- 172.21.15.119:0/1204103374 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc4c0108db0 0x7fc4c0109230 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fc4ac009900 tx=0x7fc4ac02bc40 comp rx=0 tx=0).stop 2024-03-28T00:25:40.749 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.743+0000 7fc4c6c00640 1 -- 172.21.15.119:0/1204103374 shutdown_connections 2024-03-28T00:25:40.750 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.743+0000 7fc4c6c00640 1 --2- 172.21.15.119:0/1204103374 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc4c0109770 0x7fc4c0110000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:40.750 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.743+0000 7fc4c6c00640 1 --2- 172.21.15.119:0/1204103374 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc4c0108db0 0x7fc4c0109230 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:40.750 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.743+0000 7fc4c6c00640 1 --2- 172.21.15.119:0/1204103374 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc4c0107bc0 0x7fc4c0107fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:40.750 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.743+0000 7fc4c6c00640 1 -- 172.21.15.119:0/1204103374 >> 172.21.15.119:0/1204103374 conn(0x7fc4c0076b70 msgr2=0x7fc4c0078f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:25:40.750 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.743+0000 7fc4c6c00640 1 -- 172.21.15.119:0/1204103374 shutdown_connections 2024-03-28T00:25:40.750 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.743+0000 7fc4c6c00640 1 -- 172.21.15.119:0/1204103374 wait complete. 2024-03-28T00:25:40.750 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.744+0000 7fc4c6c00640 1 Processor -- start 2024-03-28T00:25:40.751 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.744+0000 7fc4c6c00640 1 -- start start 2024-03-28T00:25:40.752 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.745+0000 7fc4c6c00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc4c0107bc0 0x7fc4c019de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:40.752 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.745+0000 7fc4c6c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc4c0108db0 0x7fc4c019e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:40.753 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.745+0000 7fc4c6c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc4c0109770 0x7fc4c01a5440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:40.753 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.745+0000 7fc4c6c00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4c0113870 con 0x7fc4c0109770 2024-03-28T00:25:40.753 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.745+0000 7fc4c6c00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4c01136f0 con 0x7fc4c0107bc0 2024-03-28T00:25:40.753 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.745+0000 7fc4c6c00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4c01139f0 con 0x7fc4c0108db0 2024-03-28T00:25:40.753 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.745+0000 7fc4c5800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc4c0107bc0 0x7fc4c019de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:40.753 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.745+0000 7fc4c5800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc4c0107bc0 0x7fc4c019de90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.119:45482/0 (socket says 172.21.15.119:45482) 2024-03-28T00:25:40.753 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.745+0000 7fc4c5800640 1 -- 172.21.15.119:0/2912387153 learned_addr learned my addr 172.21.15.119:0/2912387153 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:25:40.753 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.745+0000 7fc4c4e00640 1 --2- 172.21.15.119:0/2912387153 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc4c0108db0 0x7fc4c019e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:40.753 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.745+0000 7fc4c6200640 1 --2- 172.21.15.119:0/2912387153 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc4c0109770 0x7fc4c01a5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:40.753 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.745+0000 7fc4c5800640 1 -- 172.21.15.119:0/2912387153 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc4c0108db0 msgr2=0x7fc4c019e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:40.753 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.745+0000 7fc4c5800640 1 --2- 172.21.15.119:0/2912387153 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc4c0108db0 0x7fc4c019e3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:40.753 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.745+0000 7fc4c5800640 1 -- 172.21.15.119:0/2912387153 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc4c0109770 msgr2=0x7fc4c01a5440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:40.753 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.745+0000 7fc4c5800640 1 --2- 172.21.15.119:0/2912387153 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc4c0109770 0x7fc4c01a5440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:40.753 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.745+0000 7fc4c5800640 1 -- 172.21.15.119:0/2912387153 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4c01a5ab0 con 0x7fc4c0107bc0 2024-03-28T00:25:40.754 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.746+0000 7fc4c5800640 1 --2- 172.21.15.119:0/2912387153 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc4c0107bc0 0x7fc4c019de90 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fc4b400e9c0 tx=0x7fc4b400edc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:25:40.754 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.746+0000 7fc4be000640 1 -- 172.21.15.119:0/2912387153 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4b400ccc0 con 0x7fc4c0107bc0 2024-03-28T00:25:40.754 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.746+0000 7fc4be000640 1 -- 172.21.15.119:0/2912387153 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc4b40035f0 con 0x7fc4c0107bc0 2024-03-28T00:25:40.754 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.746+0000 7fc4c6c00640 1 -- 172.21.15.119:0/2912387153 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4c01a5da0 con 0x7fc4c0107bc0 2024-03-28T00:25:40.754 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.746+0000 7fc4be000640 1 -- 172.21.15.119:0/2912387153 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4b400ce60 con 0x7fc4c0107bc0 2024-03-28T00:25:40.754 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.746+0000 7fc4c6c00640 1 -- 172.21.15.119:0/2912387153 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4c01a6260 con 0x7fc4c0107bc0 2024-03-28T00:25:40.756 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.749+0000 7fc4be000640 1 -- 172.21.15.119:0/2912387153 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 18) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fc4b40037a0 con 0x7fc4c0107bc0 2024-03-28T00:25:40.756 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.749+0000 7fc4cafa0740 1 Processor -- start 2024-03-28T00:25:40.756 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.750+0000 7fc4cafa0740 1 -- start start 2024-03-28T00:25:40.757 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.750+0000 7fc4cafa0740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5579a086ed80 0x5579a086f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:40.757 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.750+0000 7fc4cafa0740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5579a086f770 0x5579a0874270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:40.757 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.750+0000 7fc4cafa0740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5579a08748e0 0x5579a0876d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:40.757 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.750+0000 7fc4cafa0740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579a07d3c90 con 0x5579a086f770 2024-03-28T00:25:40.757 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.750+0000 7fc4cafa0740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579a077c980 con 0x5579a08748e0 2024-03-28T00:25:40.758 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.750+0000 7fc4cafa0740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579a07cfd30 con 0x5579a086ed80 2024-03-28T00:25:40.758 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.750+0000 7fc4c6200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5579a086ed80 0x5579a086f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:40.758 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.750+0000 7fc4c5800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5579a086f770 0x5579a0874270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:40.758 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.750+0000 7fc4c6200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5579a086ed80 0x5579a086f1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.119:55622/0 (socket says 172.21.15.119:55622) 2024-03-28T00:25:40.758 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.750+0000 7fc4c6200640 1 -- 172.21.15.119:0/1408391094 learned_addr learned my addr 172.21.15.119:0/1408391094 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:25:40.758 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.750+0000 7fc4c4e00640 1 --2- 172.21.15.119:0/1408391094 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5579a08748e0 0x5579a0876d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:40.758 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.751+0000 7fc4c6200640 1 -- 172.21.15.119:0/1408391094 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5579a08748e0 msgr2=0x5579a0876d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T00:25:40.758 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.751+0000 7fc4c6200640 1 --2- 172.21.15.119:0/1408391094 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5579a08748e0 0x5579a0876d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:40.758 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.751+0000 7fc4c6200640 1 -- 172.21.15.119:0/1408391094 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5579a086f770 msgr2=0x5579a0874270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T00:25:40.758 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.751+0000 7fc4c6200640 1 --2- 172.21.15.119:0/1408391094 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5579a086f770 0x5579a0874270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:40.758 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.751+0000 7fc4c6200640 1 -- 172.21.15.119:0/1408391094 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5579a0794510 con 0x5579a086ed80 2024-03-28T00:25:40.760 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.751+0000 7fc4be000640 1 --2- 172.21.15.119:0/2912387153 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fc4a80a2880 0x7fc4a80a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:40.760 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.751+0000 7fc4c6200640 1 --2- 172.21.15.119:0/1408391094 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5579a086ed80 0x5579a086f1a0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fc4b8011440 tx=0x7fc4b8011900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca5f352f37bfcca8 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:25:40.760 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.751+0000 7fc4be000640 1 -- 172.21.15.119:0/2912387153 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fc4b40ced40 con 0x7fc4c0107bc0 2024-03-28T00:25:40.760 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.751+0000 7fc4b2a00640 1 -- 172.21.15.119:0/1408391094 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4b8003c30 con 0x5579a086ed80 2024-03-28T00:25:40.760 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.752+0000 7fc4b2a00640 1 -- 172.21.15.119:0/1408391094 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc4b8003dd0 con 0x5579a086ed80 2024-03-28T00:25:40.760 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.752+0000 7fc4c5800640 1 --2- 172.21.15.119:0/2912387153 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fc4a80a2880 0x7fc4a80a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:40.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.752+0000 7fc4b2a00640 1 -- 172.21.15.119:0/1408391094 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4b800fb50 con 0x5579a086ed80 2024-03-28T00:25:40.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.752+0000 7fc4cafa0740 1 -- 172.21.15.119:0/1408391094 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5579a086ed80 msgr2=0x5579a086f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:40.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.752+0000 7fc4c5800640 1 --2- 172.21.15.119:0/2912387153 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fc4a80a2880 0x7fc4a80a4d40 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fc4b40980b0 tx=0x7fc4b40d2780 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:25:40.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.752+0000 7fc4cafa0740 1 --2- 172.21.15.119:0/1408391094 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5579a086ed80 0x5579a086f1a0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fc4b8011440 tx=0x7fc4b8011900 comp rx=0 tx=0).stop 2024-03-28T00:25:40.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.753+0000 7fc4cafa0740 1 -- 172.21.15.119:0/1408391094 shutdown_connections 2024-03-28T00:25:40.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.753+0000 7fc4cafa0740 1 --2- 172.21.15.119:0/1408391094 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5579a08748e0 0x5579a0876d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:40.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.753+0000 7fc4cafa0740 1 --2- 172.21.15.119:0/1408391094 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5579a086f770 0x5579a0874270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:40.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.753+0000 7fc4cafa0740 1 --2- 172.21.15.119:0/1408391094 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5579a086ed80 0x5579a086f1a0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:40.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.753+0000 7fc4cafa0740 1 -- 172.21.15.119:0/1408391094 >> 172.21.15.119:0/1408391094 conn(0x5579a086a2f0 msgr2=0x5579a086a6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:25:40.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.753+0000 7fc4cafa0740 1 -- 172.21.15.119:0/1408391094 shutdown_connections 2024-03-28T00:25:40.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.753+0000 7fc4cafa0740 1 -- 172.21.15.119:0/1408391094 wait complete. 2024-03-28T00:25:40.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.753+0000 7fc4cafa0740 1 -- start start 2024-03-28T00:25:40.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.753+0000 7fc4cafa0740 1 Processor -- start 2024-03-28T00:25:40.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.754+0000 7fc4cafa0740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5579a086ed80 0x5579a0879270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:40.762 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.754+0000 7fc4cafa0740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5579a086f770 0x5579a08797b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:40.762 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.754+0000 7fc4cafa0740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5579a08748e0 0x5579a08807f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:40.762 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.754+0000 7fc4cafa0740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579a0877890 con 0x5579a086ed80 2024-03-28T00:25:40.762 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.754+0000 7fc4cafa0740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579a078fdd0 con 0x5579a08748e0 2024-03-28T00:25:40.762 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.754+0000 7fc4cafa0740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579a0877a10 con 0x5579a086f770 2024-03-28T00:25:40.762 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.754+0000 7fc4cafa0740 10 client.0 _ll_register_callbacks cb 0x5579a04e7ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T00:25:40.762 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.754+0000 7fc4c6200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5579a086ed80 0x5579a0879270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:40.762 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.754+0000 7fc4c6200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5579a086ed80 0x5579a0879270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:60308/0 (socket says 172.21.15.119:60308) 2024-03-28T00:25:40.762 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.754+0000 7fc4c6200640 1 -- 172.21.15.119:0/3870693481 learned_addr learned my addr 172.21.15.119:0/3870693481 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:25:40.762 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.754+0000 7fc4c6200640 1 --2- 172.21.15.119:0/3870693481 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5579a08748e0 0x5579a08807f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:40.762 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.754+0000 7fc4c4e00640 1 --2- 172.21.15.119:0/3870693481 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5579a086f770 0x5579a08797b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:40.765 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.755+0000 7fc4c6200640 1 -- 172.21.15.119:0/3870693481 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5579a086f770 msgr2=0x5579a08797b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T00:25:40.765 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.755+0000 7fc4c6200640 1 --2- 172.21.15.119:0/3870693481 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5579a086f770 0x5579a08797b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:40.765 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.755+0000 7fc4c6200640 1 -- 172.21.15.119:0/3870693481 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5579a08748e0 msgr2=0x5579a08807f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T00:25:40.765 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.755+0000 7fc4c6200640 1 --2- 172.21.15.119:0/3870693481 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5579a08748e0 0x5579a08807f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:40.765 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.755+0000 7fc4c6200640 1 -- 172.21.15.119:0/3870693481 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5579a04e82d0 con 0x5579a086ed80 2024-03-28T00:25:40.765 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.755+0000 7fc4c4e00640 1 --2- 172.21.15.119:0/3870693481 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5579a086f770 0x5579a08797b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T00:25:40.765 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.755+0000 7fc4c6200640 1 --2- 172.21.15.119:0/3870693481 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5579a086ed80 0x5579a0879270 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fc4b8009c30 tx=0x7fc4b8009fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8432a2d89ffa2f3a server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:25:40.765 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.755+0000 7fc4a7400640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-03-28T00:25:40.765 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.755+0000 7fc4a7400640 1 -- 172.21.15.119:0/3870693481 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4b800fcd0 con 0x5579a086ed80 2024-03-28T00:25:40.765 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.755+0000 7fc4a7400640 1 -- 172.21.15.119:0/3870693481 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fc4b8004050 con 0x5579a086ed80 2024-03-28T00:25:40.765 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.755+0000 7fc4cafa0740 1 -- 172.21.15.119:0/3870693481 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc4b8003dd0 con 0x5579a086ed80 2024-03-28T00:25:40.765 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.756+0000 7fc4a7400640 1 -- 172.21.15.119:0/3870693481 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc4b800fe70 con 0x5579a086ed80 2024-03-28T00:25:40.766 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.756+0000 7fc4cafa0740 10 client.14895 fetch_fsmap learned FSMap version 26 2024-03-28T00:25:40.766 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.756+0000 7fc4cafa0740 1 -- 172.21.15.119:0/3870693481 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x5579a05b26b0 con 0x5579a086ed80 2024-03-28T00:25:40.766 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.756+0000 7fc4a7400640 1 -- 172.21.15.119:0/3870693481 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc4b8003dd0 con 0x5579a086ed80 2024-03-28T00:25:40.766 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.756+0000 7fc4a7400640 1 -- 172.21.15.119:0/3870693481 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 26) v1 ==== 7934+0+0 (secure 0 0 0) 0x7fc4b8019800 con 0x5579a086ed80 2024-03-28T00:25:40.766 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.756+0000 7fc4cafa0740 10 client.14895 fetch_fsmap finished waiting for FSMap version 26 2024-03-28T00:25:40.766 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.756+0000 7fc4cafa0740 10 client.14895 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-28T00:25:40.766 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.757+0000 7fc4cafa0740 20 client.14895 populate_metadata read hostname 'smithi119' 2024-03-28T00:25:40.766 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.757+0000 7fc4cafa0740 1 --2- 172.21.15.119:0/3870693481 >> [v2:172.21.15.165:6836/1531193032,v1:172.21.15.165:6837/1531193032] conn(0x5579a0885730 0x5579a0889bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:40.766 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.757+0000 7fc4cafa0740 4 client.14895 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-28T00:25:40.766 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.757+0000 7fc4cafa0740 1 -- 172.21.15.119:0/3870693481 --> [v2:172.21.15.165:6836/1531193032,v1:172.21.15.165:6837/1531193032] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5579a0775520 con 0x5579a0885730 2024-03-28T00:25:40.766 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.757+0000 7fc4c4e00640 1 --2- 172.21.15.119:0/3870693481 >> [v2:172.21.15.165:6836/1531193032,v1:172.21.15.165:6837/1531193032] conn(0x5579a0885730 0x5579a0889bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:40.766 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.757+0000 7fc4a7400640 10 client.14895 ms_handle_connect on v2:172.21.15.165:6836/1531193032 2024-03-28T00:25:40.766 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.757+0000 7fc4c4e00640 1 --2- 172.21.15.119:0/3870693481 >> [v2:172.21.15.165:6836/1531193032,v1:172.21.15.165:6837/1531193032] conn(0x5579a0885730 0x5579a0889bb0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=acf3d9cdc6f3dc7f server_cookie=e340504419adbae5 in_seq=0 out_seq=0 2024-03-28T00:25:40.766 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.758+0000 7fc4a7400640 1 -- 172.21.15.119:0/3870693481 <== mds.3 v2:172.21.15.165:6836/1531193032 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5579a0775520 con 0x5579a0885730 2024-03-28T00:25:40.766 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.758+0000 7fc4a7400640 10 client.14895 handle_command_reply: tid=0 2024-03-28T00:25:40.767 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.758+0000 7fc4cafa0740 1 client.14895 shutdown 2024-03-28T00:25:40.767 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.758+0000 7fc4cafa0740 1 -- 172.21.15.119:0/3870693481 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5579a086ed80 msgr2=0x5579a0879270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:40.767 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.758+0000 7fc4cafa0740 1 --2- 172.21.15.119:0/3870693481 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5579a086ed80 0x5579a0879270 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fc4b8009c30 tx=0x7fc4b8009fb0 comp rx=0 tx=0).stop 2024-03-28T00:25:40.767 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.759+0000 7fc4cafa0740 1 -- 172.21.15.119:0/3870693481 shutdown_connections 2024-03-28T00:25:40.767 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.759+0000 7fc4cafa0740 1 --2- 172.21.15.119:0/3870693481 >> [v2:172.21.15.165:6836/1531193032,v1:172.21.15.165:6837/1531193032] conn(0x5579a0885730 0x5579a0889bb0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:40.767 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.759+0000 7fc4cafa0740 1 --2- 172.21.15.119:0/3870693481 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5579a08748e0 0x5579a08807f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:40.767 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.759+0000 7fc4cafa0740 1 --2- 172.21.15.119:0/3870693481 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5579a086f770 0x5579a08797b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:40.767 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.759+0000 7fc4a7400640 0 client.14895 ms_handle_reset on v2:172.21.15.165:6836/1531193032 2024-03-28T00:25:40.767 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.759+0000 7fc4cafa0740 1 --2- 172.21.15.119:0/3870693481 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5579a086ed80 0x5579a0879270 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:40.767 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.759+0000 7fc4cafa0740 1 -- 172.21.15.119:0/3870693481 >> 172.21.15.119:0/3870693481 conn(0x5579a086a2f0 msgr2=0x5579a08694f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:25:40.768 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.759+0000 7fc4cafa0740 1 -- 172.21.15.119:0/3870693481 shutdown_connections 2024-03-28T00:25:40.768 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.759+0000 7fc4cafa0740 1 -- 172.21.15.119:0/3870693481 wait complete. 2024-03-28T00:25:40.768 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.759+0000 7fc4cafa0740 20 client.14895 trim_cache size 0 max 16384 2024-03-28T00:25:40.785 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.778+0000 7fc4cafa0740 1 Processor -- start 2024-03-28T00:25:40.785 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.778+0000 7fc4cafa0740 1 -- start start 2024-03-28T00:25:40.785 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.779+0000 7fc4cafa0740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5579a086ed80 0x5579a0878890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:40.785 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.779+0000 7fc4cafa0740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5579a086f770 0x5579a08807b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:40.786 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.779+0000 7fc4cafa0740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5579a08748e0 0x5579a0877de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:40.786 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.779+0000 7fc4cafa0740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579a0884f50 con 0x5579a08748e0 2024-03-28T00:25:40.786 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.779+0000 7fc4cafa0740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579a08855b0 con 0x5579a086f770 2024-03-28T00:25:40.786 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.779+0000 7fc4cafa0740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579a0885c10 con 0x5579a086ed80 2024-03-28T00:25:40.786 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.779+0000 7fc4c6200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5579a086ed80 0x5579a0878890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:40.786 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.779+0000 7fc4c6200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5579a086ed80 0x5579a0878890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.119:55654/0 (socket says 172.21.15.119:55654) 2024-03-28T00:25:40.786 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.779+0000 7fc4c6200640 1 -- 172.21.15.119:0/3831920747 learned_addr learned my addr 172.21.15.119:0/3831920747 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:25:40.786 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.779+0000 7fc4c4e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5579a086f770 0x5579a08807b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:40.786 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.779+0000 7fc4c6200640 1 --2- 172.21.15.119:0/3831920747 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5579a08748e0 0x5579a0877de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:40.791 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.779+0000 7fc4c4e00640 1 -- 172.21.15.119:0/3831920747 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5579a086ed80 msgr2=0x5579a0878890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T00:25:40.791 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.779+0000 7fc4c4e00640 1 --2- 172.21.15.119:0/3831920747 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5579a086ed80 0x5579a0878890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:40.792 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.779+0000 7fc4c4e00640 1 -- 172.21.15.119:0/3831920747 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5579a08748e0 msgr2=0x5579a0877de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T00:25:40.792 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.779+0000 7fc4c4e00640 1 --2- 172.21.15.119:0/3831920747 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5579a08748e0 0x5579a0877de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:40.792 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.779+0000 7fc4c4e00640 1 -- 172.21.15.119:0/3831920747 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5579a085c170 con 0x5579a086f770 2024-03-28T00:25:40.792 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.780+0000 7fc4c4e00640 1 --2- 172.21.15.119:0/3831920747 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5579a086f770 0x5579a08807b0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fc4ac036040 tx=0x7fc4ac02cf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=70c62d5a82802795 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:25:40.792 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.780+0000 7fc4b2000640 1 -- 172.21.15.119:0/3831920747 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4ac004240 con 0x5579a086f770 2024-03-28T00:25:40.792 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.780+0000 7fc4b2000640 1 -- 172.21.15.119:0/3831920747 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc4ac0043e0 con 0x5579a086f770 2024-03-28T00:25:40.792 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.780+0000 7fc4b2000640 1 -- 172.21.15.119:0/3831920747 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4ac04ad60 con 0x5579a086f770 2024-03-28T00:25:40.792 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.781+0000 7fc4cafa0740 1 -- 172.21.15.119:0/3831920747 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5579a086f770 msgr2=0x5579a08807b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:40.792 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.781+0000 7fc4cafa0740 1 --2- 172.21.15.119:0/3831920747 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5579a086f770 0x5579a08807b0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fc4ac036040 tx=0x7fc4ac02cf90 comp rx=0 tx=0).stop 2024-03-28T00:25:40.793 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.781+0000 7fc4cafa0740 1 -- 172.21.15.119:0/3831920747 shutdown_connections 2024-03-28T00:25:40.793 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.781+0000 7fc4cafa0740 1 --2- 172.21.15.119:0/3831920747 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5579a08748e0 0x5579a0877de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:40.793 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.781+0000 7fc4cafa0740 1 --2- 172.21.15.119:0/3831920747 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5579a086f770 0x5579a08807b0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:40.793 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.781+0000 7fc4cafa0740 1 --2- 172.21.15.119:0/3831920747 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5579a086ed80 0x5579a0878890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:40.793 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.781+0000 7fc4cafa0740 1 -- 172.21.15.119:0/3831920747 >> 172.21.15.119:0/3831920747 conn(0x5579a086a2f0 msgr2=0x5579a0870210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:25:40.793 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.781+0000 7fc4cafa0740 1 -- 172.21.15.119:0/3831920747 shutdown_connections 2024-03-28T00:25:40.793 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.781+0000 7fc4cafa0740 1 -- 172.21.15.119:0/3831920747 wait complete. 2024-03-28T00:25:40.793 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.781+0000 7fc4cafa0740 1 -- start start 2024-03-28T00:25:40.793 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.781+0000 7fc4cafa0740 1 Processor -- start 2024-03-28T00:25:40.793 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.782+0000 7fc4cafa0740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5579a086ed80 0x5579a087e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:40.793 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.782+0000 7fc4cafa0740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5579a086f770 0x5579a08dccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:40.793 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.782+0000 7fc4cafa0740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5579a08748e0 0x5579a08df200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:40.793 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.782+0000 7fc4cafa0740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579a08868d0 con 0x5579a086ed80 2024-03-28T00:25:40.793 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.782+0000 7fc4cafa0740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579a0886270 con 0x5579a08748e0 2024-03-28T00:25:40.794 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.782+0000 7fc4cafa0740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579a0886f30 con 0x5579a086f770 2024-03-28T00:25:40.794 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.782+0000 7fc4cafa0740 10 client.0 _ll_register_callbacks cb 0x7fc4b40d2650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T00:25:40.794 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.782+0000 7fc4c4e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5579a086f770 0x5579a08dccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:40.794 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.782+0000 7fc4c4e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5579a086f770 0x5579a08dccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.119:55660/0 (socket says 172.21.15.119:55660) 2024-03-28T00:25:40.794 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.782+0000 7fc4c4e00640 1 -- 172.21.15.119:0/802149890 learned_addr learned my addr 172.21.15.119:0/802149890 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:25:40.794 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.782+0000 7fc4c6200640 1 --2- 172.21.15.119:0/802149890 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5579a086ed80 0x5579a087e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:40.794 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.782+0000 7fc4c6200640 1 --2- 172.21.15.119:0/802149890 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5579a08748e0 0x5579a08df200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:40.794 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.782+0000 7fc4c4e00640 1 -- 172.21.15.119:0/802149890 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5579a08748e0 msgr2=0x5579a08df200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T00:25:40.794 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.782+0000 7fc4c4e00640 1 --2- 172.21.15.119:0/802149890 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5579a08748e0 0x5579a08df200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:40.794 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.782+0000 7fc4c4e00640 1 -- 172.21.15.119:0/802149890 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5579a086ed80 msgr2=0x5579a087e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T00:25:40.794 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.782+0000 7fc4c4e00640 1 --2- 172.21.15.119:0/802149890 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5579a086ed80 0x5579a087e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:40.794 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.782+0000 7fc4c4e00640 1 -- 172.21.15.119:0/802149890 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5579a08df8f0 con 0x5579a086f770 2024-03-28T00:25:40.794 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.782+0000 7fc4c4e00640 1 --2- 172.21.15.119:0/802149890 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5579a086f770 0x5579a08dccb0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fc4ac009850 tx=0x7fc4ac02cdc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7254069d0badfde6 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:25:40.794 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.782+0000 7fc4a7400640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-03-28T00:25:40.795 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.784+0000 7fc4a7400640 1 -- 172.21.15.119:0/802149890 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4ac04ace0 con 0x5579a086f770 2024-03-28T00:25:40.795 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.784+0000 7fc4a7400640 1 -- 172.21.15.119:0/802149890 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fc4ac0046c0 con 0x5579a086f770 2024-03-28T00:25:40.795 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.784+0000 7fc4cafa0740 1 -- 172.21.15.119:0/802149890 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc4ac0043e0 con 0x5579a086f770 2024-03-28T00:25:40.795 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.784+0000 7fc4a7400640 1 -- 172.21.15.119:0/802149890 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc4ac0048e0 con 0x5579a086f770 2024-03-28T00:25:40.795 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.784+0000 7fc4cafa0740 10 client.25006 fetch_fsmap learned FSMap version 26 2024-03-28T00:25:40.795 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.784+0000 7fc4cafa0740 1 -- 172.21.15.119:0/802149890 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x5579a08e0bf0 con 0x5579a086f770 2024-03-28T00:25:40.795 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.784+0000 7fc4a7400640 1 -- 172.21.15.119:0/802149890 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc4ac0043e0 con 0x5579a086f770 2024-03-28T00:25:40.795 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.784+0000 7fc4a7400640 1 -- 172.21.15.119:0/802149890 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 26) v1 ==== 7934+0+0 (secure 0 0 0) 0x7fc4ac047070 con 0x5579a086f770 2024-03-28T00:25:40.795 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.784+0000 7fc4cafa0740 10 client.25006 fetch_fsmap finished waiting for FSMap version 26 2024-03-28T00:25:40.795 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.784+0000 7fc4cafa0740 10 client.25006 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-28T00:25:40.795 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.784+0000 7fc4cafa0740 20 client.25006 populate_metadata read hostname 'smithi119' 2024-03-28T00:25:40.795 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.784+0000 7fc4cafa0740 1 --2- 172.21.15.119:0/802149890 >> [v2:172.21.15.165:6836/1531193032,v1:172.21.15.165:6837/1531193032] conn(0x5579a08e4e60 0x5579a08e0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:40.795 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.784+0000 7fc4cafa0740 4 client.25006 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-28T00:25:40.795 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.784+0000 7fc4cafa0740 1 -- 172.21.15.119:0/802149890 --> [v2:172.21.15.165:6836/1531193032,v1:172.21.15.165:6837/1531193032] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5579a07e11d0 con 0x5579a08e4e60 2024-03-28T00:25:40.796 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.785+0000 7fc4c6200640 1 --2- 172.21.15.119:0/802149890 >> [v2:172.21.15.165:6836/1531193032,v1:172.21.15.165:6837/1531193032] conn(0x5579a08e4e60 0x5579a08e0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:40.796 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.785+0000 7fc4c6200640 1 --2- 172.21.15.119:0/802149890 >> [v2:172.21.15.165:6836/1531193032,v1:172.21.15.165:6837/1531193032] conn(0x5579a08e4e60 0x5579a08e0dc0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b44b337a99175709 server_cookie=77b64554e02c4c9c in_seq=0 out_seq=0 2024-03-28T00:25:40.796 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.785+0000 7fc4a7400640 10 client.25006 ms_handle_connect on v2:172.21.15.165:6836/1531193032 2024-03-28T00:25:40.796 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.785+0000 7fc4a7400640 1 -- 172.21.15.119:0/802149890 <== mds.3 v2:172.21.15.165:6836/1531193032 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x5579a07e11d0 con 0x5579a08e4e60 2024-03-28T00:25:40.796 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.785+0000 7fc4a7400640 10 client.25006 handle_command_reply: tid=0 2024-03-28T00:25:40.796 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.785+0000 7fc4cafa0740 1 client.25006 shutdown 2024-03-28T00:25:40.796 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.785+0000 7fc4cafa0740 1 -- 172.21.15.119:0/802149890 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5579a086f770 msgr2=0x5579a08dccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:40.796 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.785+0000 7fc4cafa0740 1 --2- 172.21.15.119:0/802149890 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5579a086f770 0x5579a08dccb0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fc4ac009850 tx=0x7fc4ac02cdc0 comp rx=0 tx=0).stop 2024-03-28T00:25:40.796 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.785+0000 7fc4cafa0740 1 -- 172.21.15.119:0/802149890 shutdown_connections 2024-03-28T00:25:40.796 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.785+0000 7fc4cafa0740 1 --2- 172.21.15.119:0/802149890 >> [v2:172.21.15.165:6836/1531193032,v1:172.21.15.165:6837/1531193032] conn(0x5579a08e4e60 0x5579a08e0dc0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:40.796 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.785+0000 7fc4a7400640 0 client.25006 ms_handle_reset on v2:172.21.15.165:6836/1531193032 2024-03-28T00:25:40.796 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.785+0000 7fc4cafa0740 1 --2- 172.21.15.119:0/802149890 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5579a08748e0 0x5579a08df200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:40.796 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.785+0000 7fc4cafa0740 1 --2- 172.21.15.119:0/802149890 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5579a086f770 0x5579a08dccb0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:40.796 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.785+0000 7fc4cafa0740 1 --2- 172.21.15.119:0/802149890 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5579a086ed80 0x5579a087e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:40.797 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.785+0000 7fc4cafa0740 1 -- 172.21.15.119:0/802149890 >> 172.21.15.119:0/802149890 conn(0x5579a086a2f0 msgr2=0x5579a0877de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:25:40.797 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.785+0000 7fc4cafa0740 1 -- 172.21.15.119:0/802149890 shutdown_connections 2024-03-28T00:25:40.797 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.785+0000 7fc4cafa0740 1 -- 172.21.15.119:0/802149890 wait complete. 2024-03-28T00:25:40.797 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.785+0000 7fc4cafa0740 20 client.25006 trim_cache size 0 max 16384 2024-03-28T00:25:40.797 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-03-28T00:25:40.797 INFO:teuthology.orchestra.run.smithi119.stdout: "cluster_fsid": "10f73b92-ec98-11ee-95d0-87774f69a715", 2024-03-28T00:25:40.797 INFO:teuthology.orchestra.run.smithi119.stdout: "whoami": 3, 2024-03-28T00:25:40.797 INFO:teuthology.orchestra.run.smithi119.stdout: "id": 24523, 2024-03-28T00:25:40.797 INFO:teuthology.orchestra.run.smithi119.stdout: "want_state": "up:active", 2024-03-28T00:25:40.797 INFO:teuthology.orchestra.run.smithi119.stdout: "state": "up:active", 2024-03-28T00:25:40.798 INFO:teuthology.orchestra.run.smithi119.stdout: "fs_name": "cephfs", 2024-03-28T00:25:40.798 INFO:teuthology.orchestra.run.smithi119.stdout: "rank_uptime": 98.275948037000006, 2024-03-28T00:25:40.798 INFO:teuthology.orchestra.run.smithi119.stdout: "mdsmap_epoch": 26, 2024-03-28T00:25:40.798 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_epoch": 87, 2024-03-28T00:25:40.798 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_epoch_barrier": 85, 2024-03-28T00:25:40.798 INFO:teuthology.orchestra.run.smithi119.stdout: "uptime": 127.353757218 2024-03-28T00:25:40.798 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-03-28T00:25:40.799 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.785+0000 7fc4c6c00640 1 -- 172.21.15.119:0/2912387153 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fc4a80a2880 msgr2=0x7fc4a80a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:40.799 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.785+0000 7fc4c6c00640 1 --2- 172.21.15.119:0/2912387153 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fc4a80a2880 0x7fc4a80a4d40 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fc4b40980b0 tx=0x7fc4b40d2780 comp rx=0 tx=0).stop 2024-03-28T00:25:40.799 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.785+0000 7fc4c6c00640 1 -- 172.21.15.119:0/2912387153 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc4c0107bc0 msgr2=0x7fc4c019de90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:40.799 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.785+0000 7fc4c6c00640 1 --2- 172.21.15.119:0/2912387153 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc4c0107bc0 0x7fc4c019de90 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fc4b400e9c0 tx=0x7fc4b400edc0 comp rx=0 tx=0).stop 2024-03-28T00:25:40.799 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.787+0000 7fc4c6c00640 1 -- 172.21.15.119:0/2912387153 shutdown_connections 2024-03-28T00:25:40.799 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.787+0000 7fc4c6c00640 1 --2- 172.21.15.119:0/2912387153 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fc4a80a2880 0x7fc4a80a4d40 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:40.799 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.787+0000 7fc4c6c00640 1 --2- 172.21.15.119:0/2912387153 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc4c0109770 0x7fc4c01a5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:40.799 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.787+0000 7fc4c6c00640 1 --2- 172.21.15.119:0/2912387153 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc4c0108db0 0x7fc4c019e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:40.799 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.787+0000 7fc4c6c00640 1 --2- 172.21.15.119:0/2912387153 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc4c0107bc0 0x7fc4c019de90 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:40.799 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.787+0000 7fc4c6c00640 1 -- 172.21.15.119:0/2912387153 >> 172.21.15.119:0/2912387153 conn(0x7fc4c0076b70 msgr2=0x7fc4c010c0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:25:40.799 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.787+0000 7fc4c6c00640 1 -- 172.21.15.119:0/2912387153 shutdown_connections 2024-03-28T00:25:40.800 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:40.787+0000 7fc4c6c00640 1 -- 172.21.15.119:0/2912387153 wait complete. 2024-03-28T00:25:40.808 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-03-28T00:25:41.006 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.000+0000 7f0df4c00640 1 -- 172.21.15.119:0/69525877 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0df0105470 msgr2=0x7f0df010bd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:41.006 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.000+0000 7f0df4c00640 1 --2- 172.21.15.119:0/69525877 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0df0105470 0x7f0df010bd00 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f0de8009ca0 tx=0x7f0de802c100 comp rx=0 tx=0).stop 2024-03-28T00:25:41.006 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.000+0000 7f0df4c00640 1 -- 172.21.15.119:0/69525877 shutdown_connections 2024-03-28T00:25:41.006 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.000+0000 7f0df4c00640 1 --2- 172.21.15.119:0/69525877 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0df0105470 0x7f0df010bd00 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:41.006 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.000+0000 7f0df4c00640 1 --2- 172.21.15.119:0/69525877 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0df0104ab0 0x7f0df0104f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:41.006 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.000+0000 7f0df4c00640 1 --2- 172.21.15.119:0/69525877 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0df01038c0 0x7f0df0103ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:41.007 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.000+0000 7f0df4c00640 1 -- 172.21.15.119:0/69525877 >> 172.21.15.119:0/69525877 conn(0x7f0df00fee50 msgr2=0x7f0df0101270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:25:41.007 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.000+0000 7f0df4c00640 1 -- 172.21.15.119:0/69525877 shutdown_connections 2024-03-28T00:25:41.007 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.000+0000 7f0df4c00640 1 -- 172.21.15.119:0/69525877 wait complete. 2024-03-28T00:25:41.009 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.001+0000 7f0df4c00640 1 Processor -- start 2024-03-28T00:25:41.009 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.001+0000 7f0df4c00640 1 -- start start 2024-03-28T00:25:41.009 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.002+0000 7f0df4c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0df01038c0 0x7f0df0199ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:41.009 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.002+0000 7f0df4c00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0df0104ab0 0x7f0df019a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:41.009 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.002+0000 7f0df4c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0df0105470 0x7f0df01a1090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:41.009 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.002+0000 7f0df4c00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0df010f520 con 0x7f0df01038c0 2024-03-28T00:25:41.010 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.002+0000 7f0df4c00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0df010f3a0 con 0x7f0df0104ab0 2024-03-28T00:25:41.010 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.002+0000 7f0df4c00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0df010f6a0 con 0x7f0df0105470 2024-03-28T00:25:41.010 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.002+0000 7f0ded400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0df0104ab0 0x7f0df019a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:41.010 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.002+0000 7f0dede00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0df01038c0 0x7f0df0199ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:41.010 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.002+0000 7f0ded400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0df0104ab0 0x7f0df019a020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.119:45540/0 (socket says 172.21.15.119:45540) 2024-03-28T00:25:41.010 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.002+0000 7f0ded400640 1 -- 172.21.15.119:0/3766927036 learned_addr learned my addr 172.21.15.119:0/3766927036 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:25:41.010 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.002+0000 7f0dee800640 1 --2- 172.21.15.119:0/3766927036 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0df0105470 0x7f0df01a1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:41.010 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.002+0000 7f0ded400640 1 -- 172.21.15.119:0/3766927036 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0df0105470 msgr2=0x7f0df01a1090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:41.010 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.002+0000 7f0ded400640 1 --2- 172.21.15.119:0/3766927036 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0df0105470 0x7f0df01a1090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:41.010 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.002+0000 7f0ded400640 1 -- 172.21.15.119:0/3766927036 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0df01038c0 msgr2=0x7f0df0199ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:41.010 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.002+0000 7f0ded400640 1 --2- 172.21.15.119:0/3766927036 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0df01038c0 0x7f0df0199ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:41.011 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.002+0000 7f0ded400640 1 -- 172.21.15.119:0/3766927036 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0df01a1700 con 0x7f0df0104ab0 2024-03-28T00:25:41.011 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.002+0000 7f0ded400640 1 --2- 172.21.15.119:0/3766927036 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0df0104ab0 0x7f0df019a020 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f0ddc00c920 tx=0x7f0ddc00cde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:25:41.011 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.003+0000 7f0de6a00640 1 -- 172.21.15.119:0/3766927036 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ddc007c70 con 0x7f0df0104ab0 2024-03-28T00:25:41.011 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.003+0000 7f0df4c00640 1 -- 172.21.15.119:0/3766927036 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0df01a19f0 con 0x7f0df0104ab0 2024-03-28T00:25:41.011 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.003+0000 7f0de6a00640 1 -- 172.21.15.119:0/3766927036 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0ddc007e10 con 0x7f0df0104ab0 2024-03-28T00:25:41.011 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.003+0000 7f0de6a00640 1 -- 172.21.15.119:0/3766927036 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ddc018890 con 0x7f0df0104ab0 2024-03-28T00:25:41.011 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.003+0000 7f0df4c00640 1 -- 172.21.15.119:0/3766927036 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0df01a1f40 con 0x7f0df0104ab0 2024-03-28T00:25:41.013 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.006+0000 7f0de6a00640 1 -- 172.21.15.119:0/3766927036 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 18) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f0ddc018a70 con 0x7f0df0104ab0 2024-03-28T00:25:41.013 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.006+0000 7f0df784b740 1 Processor -- start 2024-03-28T00:25:41.013 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.006+0000 7f0df784b740 1 -- start start 2024-03-28T00:25:41.013 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.006+0000 7f0df784b740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556fd353eba0 0x556fd353efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:41.013 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.006+0000 7f0df784b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556fd353f590 0x556fd3544090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:41.013 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.006+0000 7f0df784b740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556fd3544700 0x556fd3546b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:41.013 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.006+0000 7f0df784b740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x556fd34a3c90 con 0x556fd353eba0 2024-03-28T00:25:41.014 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.006+0000 7f0df784b740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x556fd344c980 con 0x556fd3544700 2024-03-28T00:25:41.014 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.006+0000 7f0df784b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x556fd349fd30 con 0x556fd353f590 2024-03-28T00:25:41.014 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.007+0000 7f0dee800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556fd353eba0 0x556fd353efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:41.014 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.007+0000 7f0dee800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556fd353eba0 0x556fd353efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:60370/0 (socket says 172.21.15.119:60370) 2024-03-28T00:25:41.014 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.007+0000 7f0ded400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556fd3544700 0x556fd3546b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:41.014 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.007+0000 7f0dede00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556fd353f590 0x556fd3544090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:41.014 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.007+0000 7f0dee800640 1 -- 172.21.15.119:0/237018209 learned_addr learned my addr 172.21.15.119:0/237018209 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:25:41.014 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.007+0000 7f0dee800640 1 -- 172.21.15.119:0/237018209 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556fd353f590 msgr2=0x556fd3544090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T00:25:41.014 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.007+0000 7f0dee800640 1 --2- 172.21.15.119:0/237018209 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556fd353f590 0x556fd3544090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:41.014 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.007+0000 7f0dee800640 1 -- 172.21.15.119:0/237018209 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556fd3544700 msgr2=0x556fd3546b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T00:25:41.014 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.007+0000 7f0dee800640 1 --2- 172.21.15.119:0/237018209 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556fd3544700 0x556fd3546b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:41.015 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.007+0000 7f0dee800640 1 -- 172.21.15.119:0/237018209 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556fd3464510 con 0x556fd353eba0 2024-03-28T00:25:41.015 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.007+0000 7f0de6a00640 1 --2- 172.21.15.119:0/3766927036 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f0dd40a2800 0x7f0dd40a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:41.015 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.007+0000 7f0dee800640 1 --2- 172.21.15.119:0/237018209 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556fd353eba0 0x556fd353efc0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f0de8009ca0 tx=0x7f0de8005fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7b1d3ddf8d597f8 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:25:41.015 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.008+0000 7f0de6a00640 1 -- 172.21.15.119:0/3766927036 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f0ddc0ceb00 con 0x7f0df0104ab0 2024-03-28T00:25:41.015 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.008+0000 7f0dede00640 1 --2- 172.21.15.119:0/3766927036 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f0dd40a2800 0x7f0dd40a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:41.015 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.008+0000 7f0ddb000640 1 -- 172.21.15.119:0/237018209 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0de8043070 con 0x556fd353eba0 2024-03-28T00:25:41.015 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.008+0000 7f0ddb000640 1 -- 172.21.15.119:0/237018209 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0de8003970 con 0x556fd353eba0 2024-03-28T00:25:41.015 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.008+0000 7f0ddb000640 1 -- 172.21.15.119:0/237018209 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0de8039040 con 0x556fd353eba0 2024-03-28T00:25:41.020 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.008+0000 7f0df784b740 1 -- 172.21.15.119:0/237018209 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556fd353eba0 msgr2=0x556fd353efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:41.020 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.008+0000 7f0dede00640 1 --2- 172.21.15.119:0/3766927036 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f0dd40a2800 0x7f0dd40a4cc0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x556fd353f990 tx=0x7f0de000a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:25:41.021 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.008+0000 7f0df784b740 1 --2- 172.21.15.119:0/237018209 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556fd353eba0 0x556fd353efc0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f0de8009ca0 tx=0x7f0de8005fa0 comp rx=0 tx=0).stop 2024-03-28T00:25:41.021 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.009+0000 7f0df784b740 1 -- 172.21.15.119:0/237018209 shutdown_connections 2024-03-28T00:25:41.021 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.009+0000 7f0df784b740 1 --2- 172.21.15.119:0/237018209 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556fd3544700 0x556fd3546b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:41.021 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.009+0000 7f0df784b740 1 --2- 172.21.15.119:0/237018209 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556fd353f590 0x556fd3544090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:41.021 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.009+0000 7f0df784b740 1 --2- 172.21.15.119:0/237018209 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556fd353eba0 0x556fd353efc0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:41.021 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.009+0000 7f0df784b740 1 -- 172.21.15.119:0/237018209 >> 172.21.15.119:0/237018209 conn(0x556fd353a110 msgr2=0x556fd353a510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:25:41.021 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.009+0000 7f0df784b740 1 -- 172.21.15.119:0/237018209 shutdown_connections 2024-03-28T00:25:41.021 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.009+0000 7f0df784b740 1 -- 172.21.15.119:0/237018209 wait complete. 2024-03-28T00:25:41.021 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.009+0000 7f0df784b740 1 -- start start 2024-03-28T00:25:41.021 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.009+0000 7f0df784b740 1 Processor -- start 2024-03-28T00:25:41.021 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.010+0000 7f0df784b740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556fd353eba0 0x556fd3549120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:41.021 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.010+0000 7f0df784b740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556fd353f590 0x556fd3549660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:41.022 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.010+0000 7f0df784b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556fd3544700 0x556fd3550630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:41.022 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.010+0000 7f0df784b740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x556fd3547650 con 0x556fd353eba0 2024-03-28T00:25:41.022 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.010+0000 7f0df784b740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x556fd345fdd0 con 0x556fd353f590 2024-03-28T00:25:41.022 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.010+0000 7f0df784b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x556fd35477d0 con 0x556fd3544700 2024-03-28T00:25:41.022 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.010+0000 7f0df784b740 10 client.0 _ll_register_callbacks cb 0x556fd31b7ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T00:25:41.022 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.010+0000 7f0ded400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556fd353f590 0x556fd3549660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:41.022 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.010+0000 7f0dee800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556fd3544700 0x556fd3550630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:41.022 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.010+0000 7f0ded400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556fd353f590 0x556fd3549660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.119:45562/0 (socket says 172.21.15.119:45562) 2024-03-28T00:25:41.022 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.010+0000 7f0dee800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556fd3544700 0x556fd3550630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.119:55702/0 (socket says 172.21.15.119:55702) 2024-03-28T00:25:41.022 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.010+0000 7f0ded400640 1 -- 172.21.15.119:0/1334621365 learned_addr learned my addr 172.21.15.119:0/1334621365 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:25:41.022 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.010+0000 7f0dede00640 1 --2- 172.21.15.119:0/1334621365 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556fd353eba0 0x556fd3549120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:41.023 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.010+0000 7f0ded400640 1 -- 172.21.15.119:0/1334621365 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556fd3544700 msgr2=0x556fd3550630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T00:25:41.023 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.010+0000 7f0ded400640 1 --2- 172.21.15.119:0/1334621365 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556fd3544700 0x556fd3550630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:41.023 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.010+0000 7f0ded400640 1 -- 172.21.15.119:0/1334621365 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556fd353eba0 msgr2=0x556fd3549120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T00:25:41.023 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.010+0000 7f0ded400640 1 --2- 172.21.15.119:0/1334621365 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556fd353eba0 0x556fd3549120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:41.023 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.010+0000 7f0ded400640 1 -- 172.21.15.119:0/1334621365 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556fd31b82d0 con 0x556fd353f590 2024-03-28T00:25:41.023 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.010+0000 7f0dee800640 1 --2- 172.21.15.119:0/1334621365 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556fd3544700 0x556fd3550630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T00:25:41.023 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.010+0000 7f0ded400640 1 --2- 172.21.15.119:0/1334621365 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556fd353f590 0x556fd3549660 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f0ddc097900 tx=0x7f0ddc097740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=40f9d4bea01643d0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:25:41.023 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.010+0000 7f0dcf400640 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-03-28T00:25:41.023 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.011+0000 7f0dcf400640 1 -- 172.21.15.119:0/1334621365 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ddc0d7070 con 0x556fd353f590 2024-03-28T00:25:41.023 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.011+0000 7f0dcf400640 1 -- 172.21.15.119:0/1334621365 <== mon.2 v2:172.21.15.132:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f0ddc00fb30 con 0x556fd353f590 2024-03-28T00:25:41.023 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.011+0000 7f0df784b740 1 -- 172.21.15.119:0/1334621365 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0de8003970 con 0x556fd353f590 2024-03-28T00:25:41.023 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.011+0000 7f0dcf400640 1 -- 172.21.15.119:0/1334621365 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0ddc09ae00 con 0x556fd353f590 2024-03-28T00:25:41.023 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.011+0000 7f0df784b740 10 client.25004 fetch_fsmap learned FSMap version 26 2024-03-28T00:25:41.024 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.011+0000 7f0df784b740 1 -- 172.21.15.119:0/1334621365 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x556fd32826b0 con 0x556fd353f590 2024-03-28T00:25:41.024 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.011+0000 7f0dcf400640 1 -- 172.21.15.119:0/1334621365 <== mon.2 v2:172.21.15.132:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0de8003970 con 0x556fd353f590 2024-03-28T00:25:41.024 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.011+0000 7f0dcf400640 1 -- 172.21.15.119:0/1334621365 <== mon.2 v2:172.21.15.132:3300/0 5 ==== fsmap(e 26) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f0ddc0a37b0 con 0x556fd353f590 2024-03-28T00:25:41.024 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.011+0000 7f0df784b740 10 client.25004 fetch_fsmap finished waiting for FSMap version 26 2024-03-28T00:25:41.024 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.011+0000 7f0df784b740 10 client.25004 resolve_mds: resolved name 'h' to daemon mds.h 2024-03-28T00:25:41.024 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.012+0000 7f0df784b740 20 client.25004 populate_metadata read hostname 'smithi119' 2024-03-28T00:25:41.024 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.012+0000 7f0df784b740 1 --2- 172.21.15.119:0/1334621365 >> [v2:172.21.15.132:6836/3383082967,v1:172.21.15.132:6837/3383082967] conn(0x556fd3555830 0x556fd3554910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:41.024 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.012+0000 7f0df784b740 4 client.25004 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-28T00:25:41.024 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.012+0000 7f0df784b740 1 -- 172.21.15.119:0/1334621365 --> [v2:172.21.15.132:6836/3383082967,v1:172.21.15.132:6837/3383082967] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556fd3445520 con 0x556fd3555830 2024-03-28T00:25:41.024 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.012+0000 7f0dede00640 1 --2- 172.21.15.119:0/1334621365 >> [v2:172.21.15.132:6836/3383082967,v1:172.21.15.132:6837/3383082967] conn(0x556fd3555830 0x556fd3554910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:41.024 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.012+0000 7f0dede00640 1 --2- 172.21.15.119:0/1334621365 >> [v2:172.21.15.132:6836/3383082967,v1:172.21.15.132:6837/3383082967] conn(0x556fd3555830 0x556fd3554910 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=18515248a4a0d43 server_cookie=ac8b26d3951a31d0 in_seq=0 out_seq=0 2024-03-28T00:25:41.024 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.012+0000 7f0dcf400640 10 client.25004 ms_handle_connect on v2:172.21.15.132:6836/3383082967 2024-03-28T00:25:41.025 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.013+0000 7f0dcf400640 1 -- 172.21.15.119:0/1334621365 <== mds.2 v2:172.21.15.132:6836/3383082967 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x556fd3445520 con 0x556fd3555830 2024-03-28T00:25:41.025 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.013+0000 7f0dcf400640 10 client.25004 handle_command_reply: tid=0 2024-03-28T00:25:41.025 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.013+0000 7f0df784b740 1 client.25004 shutdown 2024-03-28T00:25:41.025 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.013+0000 7f0df784b740 1 -- 172.21.15.119:0/1334621365 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556fd353f590 msgr2=0x556fd3549660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:41.025 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.013+0000 7f0df784b740 1 --2- 172.21.15.119:0/1334621365 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556fd353f590 0x556fd3549660 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f0ddc097900 tx=0x7f0ddc097740 comp rx=0 tx=0).stop 2024-03-28T00:25:41.025 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.014+0000 7f0df784b740 1 -- 172.21.15.119:0/1334621365 shutdown_connections 2024-03-28T00:25:41.025 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.014+0000 7f0df784b740 1 --2- 172.21.15.119:0/1334621365 >> [v2:172.21.15.132:6836/3383082967,v1:172.21.15.132:6837/3383082967] conn(0x556fd3555830 0x556fd3554910 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:41.025 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.014+0000 7f0df784b740 1 --2- 172.21.15.119:0/1334621365 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556fd3544700 0x556fd3550630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:41.025 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.014+0000 7f0df784b740 1 --2- 172.21.15.119:0/1334621365 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556fd353f590 0x556fd3549660 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:41.025 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.014+0000 7f0df784b740 1 --2- 172.21.15.119:0/1334621365 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556fd353eba0 0x556fd3549120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:41.025 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.014+0000 7f0df784b740 1 -- 172.21.15.119:0/1334621365 >> 172.21.15.119:0/1334621365 conn(0x556fd353a110 msgr2=0x556fd3539310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:25:41.025 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.014+0000 7f0dcf400640 0 client.25004 ms_handle_reset on v2:172.21.15.132:6836/3383082967 2024-03-28T00:25:41.025 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.014+0000 7f0df784b740 1 -- 172.21.15.119:0/1334621365 shutdown_connections 2024-03-28T00:25:41.025 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.014+0000 7f0df784b740 1 -- 172.21.15.119:0/1334621365 wait complete. 2024-03-28T00:25:41.026 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.014+0000 7f0df784b740 20 client.25004 trim_cache size 0 max 16384 2024-03-28T00:25:41.038 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.032+0000 7f0df784b740 1 Processor -- start 2024-03-28T00:25:41.038 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.033+0000 7f0df784b740 1 -- start start 2024-03-28T00:25:41.039 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.033+0000 7f0df784b740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556fd353eba0 0x556fd3534e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:41.039 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.033+0000 7f0df784b740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556fd353f590 0x556fd353a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:41.039 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.033+0000 7f0df784b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556fd3544700 0x556fd3538a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:41.039 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.033+0000 7f0df784b740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x556fd3555050 con 0x556fd353f590 2024-03-28T00:25:41.039 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.033+0000 7f0df784b740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x556fd35556b0 con 0x556fd353eba0 2024-03-28T00:25:41.039 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.033+0000 7f0df784b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x556fd3555d10 con 0x556fd3544700 2024-03-28T00:25:41.039 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.033+0000 7f0dede00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556fd353eba0 0x556fd3534e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:41.039 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.033+0000 7f0dee800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556fd3544700 0x556fd3538a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:41.039 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.033+0000 7f0dede00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556fd353eba0 0x556fd3534e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.119:45572/0 (socket says 172.21.15.119:45572) 2024-03-28T00:25:41.039 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.033+0000 7f0dede00640 1 -- 172.21.15.119:0/2143165918 learned_addr learned my addr 172.21.15.119:0/2143165918 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:25:41.039 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.033+0000 7f0ded400640 1 --2- 172.21.15.119:0/2143165918 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556fd353f590 0x556fd353a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:41.039 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.033+0000 7f0dee800640 1 -- 172.21.15.119:0/2143165918 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556fd353eba0 msgr2=0x556fd3534e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T00:25:41.040 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.033+0000 7f0dee800640 1 --2- 172.21.15.119:0/2143165918 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556fd353eba0 0x556fd3534e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:41.040 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.033+0000 7f0dee800640 1 -- 172.21.15.119:0/2143165918 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556fd353f590 msgr2=0x556fd353a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T00:25:41.040 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.033+0000 7f0dee800640 1 --2- 172.21.15.119:0/2143165918 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556fd353f590 0x556fd353a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:41.040 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.033+0000 7f0dee800640 1 -- 172.21.15.119:0/2143165918 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556fd3543da0 con 0x556fd3544700 2024-03-28T00:25:41.044 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.034+0000 7f0dee800640 1 --2- 172.21.15.119:0/2143165918 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556fd3544700 0x556fd3538a80 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f0de802c640 tx=0x7f0de8005d80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=414cdcf6354b0ab0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:25:41.045 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.034+0000 7f0dda600640 1 -- 172.21.15.119:0/2143165918 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0de8043040 con 0x556fd3544700 2024-03-28T00:25:41.045 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.034+0000 7f0dda600640 1 -- 172.21.15.119:0/2143165918 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0de8003970 con 0x556fd3544700 2024-03-28T00:25:41.045 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.034+0000 7f0dda600640 1 -- 172.21.15.119:0/2143165918 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0de8039040 con 0x556fd3544700 2024-03-28T00:25:41.045 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.035+0000 7f0df784b740 1 -- 172.21.15.119:0/2143165918 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556fd3544700 msgr2=0x556fd3538a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:41.045 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.035+0000 7f0df784b740 1 --2- 172.21.15.119:0/2143165918 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556fd3544700 0x556fd3538a80 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f0de802c640 tx=0x7f0de8005d80 comp rx=0 tx=0).stop 2024-03-28T00:25:41.045 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.035+0000 7f0df784b740 1 -- 172.21.15.119:0/2143165918 shutdown_connections 2024-03-28T00:25:41.045 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.035+0000 7f0df784b740 1 --2- 172.21.15.119:0/2143165918 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556fd3544700 0x556fd3538a80 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:41.045 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.035+0000 7f0df784b740 1 --2- 172.21.15.119:0/2143165918 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556fd353f590 0x556fd353a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:41.045 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.035+0000 7f0df784b740 1 --2- 172.21.15.119:0/2143165918 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556fd353eba0 0x556fd3534e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:41.045 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.035+0000 7f0df784b740 1 -- 172.21.15.119:0/2143165918 >> 172.21.15.119:0/2143165918 conn(0x556fd353a110 msgr2=0x556fd3547c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:25:41.045 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.035+0000 7f0df784b740 1 -- 172.21.15.119:0/2143165918 shutdown_connections 2024-03-28T00:25:41.045 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.035+0000 7f0df784b740 1 -- 172.21.15.119:0/2143165918 wait complete. 2024-03-28T00:25:41.046 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.035+0000 7f0df784b740 1 -- start start 2024-03-28T00:25:41.046 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.035+0000 7f0df784b740 1 Processor -- start 2024-03-28T00:25:41.046 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.036+0000 7f0df784b740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556fd353eba0 0x556fd354e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:41.046 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.036+0000 7f0df784b740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556fd353f590 0x556fd354eb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:41.046 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.036+0000 7f0df784b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556fd3544700 0x556fd354f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:41.046 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.036+0000 7f0df784b740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x556fd35569d0 con 0x556fd353eba0 2024-03-28T00:25:41.046 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.036+0000 7f0df784b740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x556fd3556370 con 0x556fd353f590 2024-03-28T00:25:41.046 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.036+0000 7f0df784b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x556fd3557030 con 0x556fd3544700 2024-03-28T00:25:41.046 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.036+0000 7f0df784b740 10 client.0 _ll_register_callbacks cb 0x7f0ddc0d2230 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T00:25:41.046 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.036+0000 7f0dede00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556fd353eba0 0x556fd354e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:41.046 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.036+0000 7f0ded400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556fd353f590 0x556fd354eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:41.046 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.036+0000 7f0dede00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556fd353eba0 0x556fd354e5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:60404/0 (socket says 172.21.15.119:60404) 2024-03-28T00:25:41.046 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.036+0000 7f0dede00640 1 -- 172.21.15.119:0/3647644661 learned_addr learned my addr 172.21.15.119:0/3647644661 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:25:41.047 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.036+0000 7f0dee800640 1 --2- 172.21.15.119:0/3647644661 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556fd3544700 0x556fd354f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:41.047 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.036+0000 7f0dede00640 1 -- 172.21.15.119:0/3647644661 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556fd3544700 msgr2=0x556fd354f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T00:25:41.047 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.036+0000 7f0dede00640 1 --2- 172.21.15.119:0/3647644661 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556fd3544700 0x556fd354f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:41.047 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.036+0000 7f0dede00640 1 -- 172.21.15.119:0/3647644661 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556fd353f590 msgr2=0x556fd354eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T00:25:41.047 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.036+0000 7f0dede00640 1 --2- 172.21.15.119:0/3647644661 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556fd353f590 0x556fd354eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:41.047 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.036+0000 7f0dede00640 1 -- 172.21.15.119:0/3647644661 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556fd354f980 con 0x556fd353eba0 2024-03-28T00:25:41.047 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.036+0000 7f0ded400640 1 --2- 172.21.15.119:0/3647644661 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556fd353f590 0x556fd354eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T00:25:41.047 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.037+0000 7f0dede00640 1 --2- 172.21.15.119:0/3647644661 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556fd353eba0 0x556fd354e5e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f0df0104910 tx=0x7f0de0015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15dda118bb80b44f server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:25:41.047 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.037+0000 7f0dcf400640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-03-28T00:25:41.047 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.037+0000 7f0dcf400640 1 -- 172.21.15.119:0/3647644661 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0de00299d0 con 0x556fd353eba0 2024-03-28T00:25:41.047 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.037+0000 7f0dcf400640 1 -- 172.21.15.119:0/3647644661 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f0de0029b70 con 0x556fd353eba0 2024-03-28T00:25:41.047 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.037+0000 7f0df784b740 1 -- 172.21.15.119:0/3647644661 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0de8003970 con 0x556fd353eba0 2024-03-28T00:25:41.047 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.037+0000 7f0dcf400640 1 -- 172.21.15.119:0/3647644661 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0de0039050 con 0x556fd353eba0 2024-03-28T00:25:41.047 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.037+0000 7f0df784b740 10 client.14916 fetch_fsmap learned FSMap version 26 2024-03-28T00:25:41.048 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.037+0000 7f0df784b740 1 -- 172.21.15.119:0/3647644661 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x556fd35b1260 con 0x556fd353eba0 2024-03-28T00:25:41.048 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.038+0000 7f0dcf400640 1 -- 172.21.15.119:0/3647644661 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0de8003970 con 0x556fd353eba0 2024-03-28T00:25:41.048 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.038+0000 7f0dcf400640 1 -- 172.21.15.119:0/3647644661 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 26) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f0de001b5d0 con 0x556fd353eba0 2024-03-28T00:25:41.048 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.038+0000 7f0df784b740 10 client.14916 fetch_fsmap finished waiting for FSMap version 26 2024-03-28T00:25:41.048 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.038+0000 7f0df784b740 10 client.14916 resolve_mds: resolved name 'h' to daemon mds.h 2024-03-28T00:25:41.048 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.038+0000 7f0df784b740 20 client.14916 populate_metadata read hostname 'smithi119' 2024-03-28T00:25:41.048 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.038+0000 7f0df784b740 1 --2- 172.21.15.119:0/3647644661 >> [v2:172.21.15.132:6836/3383082967,v1:172.21.15.132:6837/3383082967] conn(0x556fd35b5930 0x556fd35b1550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:41.048 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.038+0000 7f0df784b740 4 client.14916 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-28T00:25:41.048 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.038+0000 7f0df784b740 1 -- 172.21.15.119:0/3647644661 --> [v2:172.21.15.132:6836/3383082967,v1:172.21.15.132:6837/3383082967] -- command(tid 0: {"prefix": "status"}) v1 -- 0x556fd34b11d0 con 0x556fd35b5930 2024-03-28T00:25:41.048 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.038+0000 7f0ded400640 1 --2- 172.21.15.119:0/3647644661 >> [v2:172.21.15.132:6836/3383082967,v1:172.21.15.132:6837/3383082967] conn(0x556fd35b5930 0x556fd35b1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:41.048 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.039+0000 7f0ded400640 1 --2- 172.21.15.119:0/3647644661 >> [v2:172.21.15.132:6836/3383082967,v1:172.21.15.132:6837/3383082967] conn(0x556fd35b5930 0x556fd35b1550 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e94ed313c6cfbf96 server_cookie=e09a372e56d435d1 in_seq=0 out_seq=0 2024-03-28T00:25:41.048 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.039+0000 7f0dcf400640 10 client.14916 ms_handle_connect on v2:172.21.15.132:6836/3383082967 2024-03-28T00:25:41.048 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.039+0000 7f0dcf400640 1 -- 172.21.15.119:0/3647644661 <== mds.2 v2:172.21.15.132:6836/3383082967 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x556fd34b11d0 con 0x556fd35b5930 2024-03-28T00:25:41.048 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.039+0000 7f0dcf400640 10 client.14916 handle_command_reply: tid=0 2024-03-28T00:25:41.049 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.039+0000 7f0df784b740 1 client.14916 shutdown 2024-03-28T00:25:41.049 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.039+0000 7f0df784b740 1 -- 172.21.15.119:0/3647644661 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556fd353eba0 msgr2=0x556fd354e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:41.049 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.039+0000 7f0df784b740 1 --2- 172.21.15.119:0/3647644661 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556fd353eba0 0x556fd354e5e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f0df0104910 tx=0x7f0de0015000 comp rx=0 tx=0).stop 2024-03-28T00:25:41.049 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.039+0000 7f0df784b740 1 -- 172.21.15.119:0/3647644661 shutdown_connections 2024-03-28T00:25:41.049 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.039+0000 7f0df784b740 1 --2- 172.21.15.119:0/3647644661 >> [v2:172.21.15.132:6836/3383082967,v1:172.21.15.132:6837/3383082967] conn(0x556fd35b5930 0x556fd35b1550 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:41.049 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.039+0000 7f0df784b740 1 --2- 172.21.15.119:0/3647644661 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556fd3544700 0x556fd354f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:41.049 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.039+0000 7f0df784b740 1 --2- 172.21.15.119:0/3647644661 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556fd353f590 0x556fd354eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:41.049 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.039+0000 7f0df784b740 1 --2- 172.21.15.119:0/3647644661 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556fd353eba0 0x556fd354e5e0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:41.049 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.039+0000 7f0dcf400640 0 client.14916 ms_handle_reset on v2:172.21.15.132:6836/3383082967 2024-03-28T00:25:41.049 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.039+0000 7f0df784b740 1 -- 172.21.15.119:0/3647644661 >> 172.21.15.119:0/3647644661 conn(0x556fd353a110 msgr2=0x556fd3534e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:25:41.049 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.039+0000 7f0df784b740 1 -- 172.21.15.119:0/3647644661 shutdown_connections 2024-03-28T00:25:41.049 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.039+0000 7f0df784b740 1 -- 172.21.15.119:0/3647644661 wait complete. 2024-03-28T00:25:41.049 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.039+0000 7f0df784b740 20 client.14916 trim_cache size 0 max 16384 2024-03-28T00:25:41.050 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-03-28T00:25:41.050 INFO:teuthology.orchestra.run.smithi119.stdout: "cluster_fsid": "10f73b92-ec98-11ee-95d0-87774f69a715", 2024-03-28T00:25:41.050 INFO:teuthology.orchestra.run.smithi119.stdout: "whoami": 2, 2024-03-28T00:25:41.050 INFO:teuthology.orchestra.run.smithi119.stdout: "id": 24530, 2024-03-28T00:25:41.050 INFO:teuthology.orchestra.run.smithi119.stdout: "want_state": "up:active", 2024-03-28T00:25:41.050 INFO:teuthology.orchestra.run.smithi119.stdout: "state": "up:active", 2024-03-28T00:25:41.050 INFO:teuthology.orchestra.run.smithi119.stdout: "fs_name": "cephfs", 2024-03-28T00:25:41.050 INFO:teuthology.orchestra.run.smithi119.stdout: "rank_uptime": 99.551311474000002, 2024-03-28T00:25:41.051 INFO:teuthology.orchestra.run.smithi119.stdout: "mdsmap_epoch": 26, 2024-03-28T00:25:41.051 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_epoch": 87, 2024-03-28T00:25:41.051 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_epoch_barrier": 85, 2024-03-28T00:25:41.051 INFO:teuthology.orchestra.run.smithi119.stdout: "uptime": 124.728259419 2024-03-28T00:25:41.051 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-03-28T00:25:41.052 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.040+0000 7f0df4c00640 1 -- 172.21.15.119:0/3766927036 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f0dd40a2800 msgr2=0x7f0dd40a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:41.052 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.040+0000 7f0df4c00640 1 --2- 172.21.15.119:0/3766927036 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f0dd40a2800 0x7f0dd40a4cc0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x556fd353f990 tx=0x7f0de000a2d0 comp rx=0 tx=0).stop 2024-03-28T00:25:41.052 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.040+0000 7f0df4c00640 1 -- 172.21.15.119:0/3766927036 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0df0104ab0 msgr2=0x7f0df019a020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:41.052 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.040+0000 7f0df4c00640 1 --2- 172.21.15.119:0/3766927036 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0df0104ab0 0x7f0df019a020 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f0ddc00c920 tx=0x7f0ddc00cde0 comp rx=0 tx=0).stop 2024-03-28T00:25:41.052 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.040+0000 7f0df4c00640 1 -- 172.21.15.119:0/3766927036 shutdown_connections 2024-03-28T00:25:41.052 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.040+0000 7f0df4c00640 1 --2- 172.21.15.119:0/3766927036 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f0dd40a2800 0x7f0dd40a4cc0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:41.052 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.040+0000 7f0df4c00640 1 --2- 172.21.15.119:0/3766927036 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0df0105470 0x7f0df01a1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:41.052 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.040+0000 7f0df4c00640 1 --2- 172.21.15.119:0/3766927036 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0df0104ab0 0x7f0df019a020 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:41.052 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.040+0000 7f0df4c00640 1 --2- 172.21.15.119:0/3766927036 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0df01038c0 0x7f0df0199ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:41.052 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.040+0000 7f0df4c00640 1 -- 172.21.15.119:0/3766927036 >> 172.21.15.119:0/3766927036 conn(0x7f0df00fee50 msgr2=0x7f0df0107d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:25:41.052 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.040+0000 7f0df4c00640 1 -- 172.21.15.119:0/3766927036 shutdown_connections 2024-03-28T00:25:41.052 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.040+0000 7f0df4c00640 1 -- 172.21.15.119:0/3766927036 wait complete. 2024-03-28T00:25:41.060 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:25:40 smithi119 ceph-mon[31518]: pgmap v365: 97 pgs: 97 active+clean; 648 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 767 B/s wr, 12 op/s 2024-03-28T00:25:41.060 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:25:40 smithi119 ceph-mon[31518]: from='client.? 172.21.15.119:0/4095937639' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T00:25:41.063 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-03-28T00:25:41.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:25:40 smithi165 ceph-mon[36302]: pgmap v365: 97 pgs: 97 active+clean; 648 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 767 B/s wr, 12 op/s 2024-03-28T00:25:41.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:25:40 smithi165 ceph-mon[36302]: from='client.? 172.21.15.119:0/4095937639' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T00:25:41.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:25:40 smithi132 ceph-mon[35941]: pgmap v365: 97 pgs: 97 active+clean; 648 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 767 B/s wr, 12 op/s 2024-03-28T00:25:41.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:25:40 smithi132 ceph-mon[35941]: from='client.? 172.21.15.119:0/4095937639' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T00:25:41.274 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.268+0000 7fcd26c00640 1 -- 172.21.15.119:0/4003179280 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcd20109770 msgr2=0x7fcd20110000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:41.274 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.268+0000 7fcd26c00640 1 --2- 172.21.15.119:0/4003179280 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcd20109770 0x7fcd20110000 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fcd18009ca0 tx=0x7fcd1802c100 comp rx=0 tx=0).stop 2024-03-28T00:25:41.274 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.268+0000 7fcd26c00640 1 -- 172.21.15.119:0/4003179280 shutdown_connections 2024-03-28T00:25:41.274 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.268+0000 7fcd26c00640 1 --2- 172.21.15.119:0/4003179280 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcd20109770 0x7fcd20110000 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:41.274 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.268+0000 7fcd26c00640 1 --2- 172.21.15.119:0/4003179280 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcd20108db0 0x7fcd20109230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:41.275 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.268+0000 7fcd26c00640 1 --2- 172.21.15.119:0/4003179280 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcd20107bc0 0x7fcd20107fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:41.275 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.268+0000 7fcd26c00640 1 -- 172.21.15.119:0/4003179280 >> 172.21.15.119:0/4003179280 conn(0x7fcd20076b70 msgr2=0x7fcd20078f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:25:41.275 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.268+0000 7fcd26c00640 1 -- 172.21.15.119:0/4003179280 shutdown_connections 2024-03-28T00:25:41.275 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.268+0000 7fcd26c00640 1 -- 172.21.15.119:0/4003179280 wait complete. 2024-03-28T00:25:41.277 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.269+0000 7fcd26c00640 1 Processor -- start 2024-03-28T00:25:41.277 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.269+0000 7fcd26c00640 1 -- start start 2024-03-28T00:25:41.278 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.270+0000 7fcd26c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcd20107bc0 0x7fcd2019de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:41.278 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.270+0000 7fcd26c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcd20108db0 0x7fcd2019e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:41.278 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.270+0000 7fcd26c00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcd20109770 0x7fcd201a53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:41.278 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.270+0000 7fcd26c00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd20113840 con 0x7fcd20107bc0 2024-03-28T00:25:41.278 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.270+0000 7fcd26c00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd201136c0 con 0x7fcd20109770 2024-03-28T00:25:41.278 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.270+0000 7fcd26c00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd201139c0 con 0x7fcd20108db0 2024-03-28T00:25:41.278 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.270+0000 7fcd26200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcd20109770 0x7fcd201a53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:41.278 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.270+0000 7fcd25800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcd20107bc0 0x7fcd2019de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:41.278 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.270+0000 7fcd26200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcd20109770 0x7fcd201a53d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.119:45596/0 (socket says 172.21.15.119:45596) 2024-03-28T00:25:41.278 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.270+0000 7fcd26200640 1 -- 172.21.15.119:0/1388261956 learned_addr learned my addr 172.21.15.119:0/1388261956 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:25:41.278 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.270+0000 7fcd24e00640 1 --2- 172.21.15.119:0/1388261956 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcd20108db0 0x7fcd2019e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:41.278 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.270+0000 7fcd26200640 1 -- 172.21.15.119:0/1388261956 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcd20108db0 msgr2=0x7fcd2019e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:41.279 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.270+0000 7fcd26200640 1 --2- 172.21.15.119:0/1388261956 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcd20108db0 0x7fcd2019e360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:41.279 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.270+0000 7fcd26200640 1 -- 172.21.15.119:0/1388261956 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcd20107bc0 msgr2=0x7fcd2019de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:41.279 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.270+0000 7fcd26200640 1 --2- 172.21.15.119:0/1388261956 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcd20107bc0 0x7fcd2019de20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:41.279 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.270+0000 7fcd26200640 1 -- 172.21.15.119:0/1388261956 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd201a5a40 con 0x7fcd20109770 2024-03-28T00:25:41.279 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.271+0000 7fcd26200640 1 --2- 172.21.15.119:0/1388261956 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcd20109770 0x7fcd201a53d0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fcd1802c640 tx=0x7fcd18035d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:25:41.279 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.271+0000 7fcd1e000640 1 -- 172.21.15.119:0/1388261956 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd18039040 con 0x7fcd20109770 2024-03-28T00:25:41.279 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.271+0000 7fcd1e000640 1 -- 172.21.15.119:0/1388261956 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcd18044070 con 0x7fcd20109770 2024-03-28T00:25:41.279 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.271+0000 7fcd26c00640 1 -- 172.21.15.119:0/1388261956 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd201a5cd0 con 0x7fcd20109770 2024-03-28T00:25:41.279 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.271+0000 7fcd1e000640 1 -- 172.21.15.119:0/1388261956 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd180043c0 con 0x7fcd20109770 2024-03-28T00:25:41.279 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.271+0000 7fcd26c00640 1 -- 172.21.15.119:0/1388261956 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd201a61c0 con 0x7fcd20109770 2024-03-28T00:25:41.282 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.274+0000 7fcd2af6a740 1 Processor -- start 2024-03-28T00:25:41.283 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.274+0000 7fcd1e000640 1 -- 172.21.15.119:0/1388261956 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 18) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fcd18004560 con 0x7fcd20109770 2024-03-28T00:25:41.283 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.274+0000 7fcd2af6a740 1 -- start start 2024-03-28T00:25:41.283 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.274+0000 7fcd2af6a740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5608ab871d80 0x5608ab8721a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:41.283 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.274+0000 7fcd2af6a740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5608ab872770 0x5608ab877270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:41.283 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.275+0000 7fcd2af6a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5608ab8778e0 0x5608ab879d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:41.283 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.275+0000 7fcd2af6a740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608ab7d6c90 con 0x5608ab872770 2024-03-28T00:25:41.283 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.275+0000 7fcd2af6a740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608ab77f980 con 0x5608ab871d80 2024-03-28T00:25:41.283 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.275+0000 7fcd2af6a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608ab7d2d30 con 0x5608ab8778e0 2024-03-28T00:25:41.283 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.275+0000 7fcd24e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5608ab871d80 0x5608ab8721a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:41.283 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.275+0000 7fcd25800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5608ab872770 0x5608ab877270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:41.283 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.275+0000 7fcd24e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5608ab871d80 0x5608ab8721a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.119:45604/0 (socket says 172.21.15.119:45604) 2024-03-28T00:25:41.283 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.275+0000 7fcd24e00640 1 -- 172.21.15.119:0/1942480819 learned_addr learned my addr 172.21.15.119:0/1942480819 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:25:41.284 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.275+0000 7fcd24e00640 1 --2- 172.21.15.119:0/1942480819 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5608ab8778e0 0x5608ab879d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:41.284 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.275+0000 7fcd25800640 1 -- 172.21.15.119:0/1942480819 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5608ab8778e0 msgr2=0x5608ab879d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T00:25:41.284 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.275+0000 7fcd25800640 1 --2- 172.21.15.119:0/1942480819 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5608ab8778e0 0x5608ab879d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:41.284 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.275+0000 7fcd25800640 1 -- 172.21.15.119:0/1942480819 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5608ab871d80 msgr2=0x5608ab8721a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T00:25:41.284 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.275+0000 7fcd25800640 1 --2- 172.21.15.119:0/1942480819 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5608ab871d80 0x5608ab8721a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:41.284 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.275+0000 7fcd25800640 1 -- 172.21.15.119:0/1942480819 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5608ab797510 con 0x5608ab872770 2024-03-28T00:25:41.284 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.275+0000 7fcd24e00640 1 --2- 172.21.15.119:0/1942480819 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5608ab871d80 0x5608ab8721a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T00:25:41.284 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.276+0000 7fcd25800640 1 --2- 172.21.15.119:0/1942480819 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5608ab872770 0x5608ab877270 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fcd14009bf0 tx=0x7fcd14011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3476f0c607d4acb server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:25:41.284 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.276+0000 7fcd1e000640 1 --2- 172.21.15.119:0/1388261956 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fcd080a2800 0x7fcd080a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:41.284 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.276+0000 7fcd12a00640 1 -- 172.21.15.119:0/1942480819 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd14009da0 con 0x5608ab872770 2024-03-28T00:25:41.285 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.276+0000 7fcd12a00640 1 -- 172.21.15.119:0/1942480819 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcd14005020 con 0x5608ab872770 2024-03-28T00:25:41.285 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.276+0000 7fcd12a00640 1 -- 172.21.15.119:0/1942480819 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd14004a50 con 0x5608ab872770 2024-03-28T00:25:41.285 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.276+0000 7fcd1e000640 1 -- 172.21.15.119:0/1388261956 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fcd180f0fe0 con 0x7fcd20109770 2024-03-28T00:25:41.285 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.276+0000 7fcd24e00640 1 --2- 172.21.15.119:0/1388261956 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fcd080a2800 0x7fcd080a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:41.285 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.277+0000 7fcd2af6a740 1 -- 172.21.15.119:0/1942480819 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5608ab872770 msgr2=0x5608ab877270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:41.285 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.277+0000 7fcd24e00640 1 --2- 172.21.15.119:0/1388261956 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fcd080a2800 0x7fcd080a4cc0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x5608ab872fd0 tx=0x7fcd0c005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:25:41.285 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.277+0000 7fcd2af6a740 1 --2- 172.21.15.119:0/1942480819 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5608ab872770 0x5608ab877270 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fcd14009bf0 tx=0x7fcd14011810 comp rx=0 tx=0).stop 2024-03-28T00:25:41.285 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.277+0000 7fcd2af6a740 1 -- 172.21.15.119:0/1942480819 shutdown_connections 2024-03-28T00:25:41.285 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.277+0000 7fcd2af6a740 1 --2- 172.21.15.119:0/1942480819 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5608ab8778e0 0x5608ab879d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:41.285 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.277+0000 7fcd2af6a740 1 --2- 172.21.15.119:0/1942480819 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5608ab872770 0x5608ab877270 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:41.285 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.277+0000 7fcd2af6a740 1 --2- 172.21.15.119:0/1942480819 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5608ab871d80 0x5608ab8721a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:41.285 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.277+0000 7fcd2af6a740 1 -- 172.21.15.119:0/1942480819 >> 172.21.15.119:0/1942480819 conn(0x5608ab86d2f0 msgr2=0x5608ab86d6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:25:41.285 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.277+0000 7fcd2af6a740 1 -- 172.21.15.119:0/1942480819 shutdown_connections 2024-03-28T00:25:41.286 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.277+0000 7fcd2af6a740 1 -- 172.21.15.119:0/1942480819 wait complete. 2024-03-28T00:25:41.286 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.277+0000 7fcd2af6a740 1 -- start start 2024-03-28T00:25:41.286 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.278+0000 7fcd2af6a740 1 Processor -- start 2024-03-28T00:25:41.290 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.278+0000 7fcd2af6a740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5608ab871d80 0x5608ab87c230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:41.290 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.278+0000 7fcd2af6a740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5608ab872770 0x5608ab87c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:41.290 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.278+0000 7fcd2af6a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5608ab8778e0 0x5608ab8837b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:41.291 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.278+0000 7fcd2af6a740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608ab87a890 con 0x5608ab871d80 2024-03-28T00:25:41.291 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.278+0000 7fcd2af6a740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608ab792dd0 con 0x5608ab872770 2024-03-28T00:25:41.291 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.278+0000 7fcd2af6a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608ab87aa10 con 0x5608ab8778e0 2024-03-28T00:25:41.291 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.278+0000 7fcd2af6a740 10 client.0 _ll_register_callbacks cb 0x5608ab4eaac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T00:25:41.291 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.278+0000 7fcd26200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5608ab8778e0 0x5608ab8837b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:41.291 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.278+0000 7fcd24e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5608ab872770 0x5608ab87c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:41.291 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.278+0000 7fcd25800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5608ab871d80 0x5608ab87c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:41.291 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.278+0000 7fcd24e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5608ab872770 0x5608ab87c770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.119:45610/0 (socket says 172.21.15.119:45610) 2024-03-28T00:25:41.291 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.278+0000 7fcd24e00640 1 -- 172.21.15.119:0/3884647093 learned_addr learned my addr 172.21.15.119:0/3884647093 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:25:41.291 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.278+0000 7fcd26200640 1 --2- 172.21.15.119:0/3884647093 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5608ab8778e0 0x5608ab8837b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.119:55730/0 (socket says 172.21.15.119:55730) 2024-03-28T00:25:41.291 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.278+0000 7fcd24e00640 1 -- 172.21.15.119:0/3884647093 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5608ab8778e0 msgr2=0x5608ab8837b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T00:25:41.291 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.278+0000 7fcd24e00640 1 --2- 172.21.15.119:0/3884647093 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5608ab8778e0 0x5608ab8837b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:41.291 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.279+0000 7fcd24e00640 1 -- 172.21.15.119:0/3884647093 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5608ab871d80 msgr2=0x5608ab87c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T00:25:41.292 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.279+0000 7fcd24e00640 1 --2- 172.21.15.119:0/3884647093 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5608ab871d80 0x5608ab87c230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:41.292 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.279+0000 7fcd24e00640 1 -- 172.21.15.119:0/3884647093 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5608ab4eb2d0 con 0x5608ab872770 2024-03-28T00:25:41.292 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.279+0000 7fcd24e00640 1 --2- 172.21.15.119:0/3884647093 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5608ab872770 0x5608ab87c770 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fcd0c00ff90 tx=0x7fcd0c005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9cf16b21e79dca5b server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:25:41.292 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.279+0000 7fcd07400640 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-03-28T00:25:41.292 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.279+0000 7fcd07400640 1 -- 172.21.15.119:0/3884647093 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd0c010d00 con 0x5608ab872770 2024-03-28T00:25:41.292 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.279+0000 7fcd07400640 1 -- 172.21.15.119:0/3884647093 <== mon.2 v2:172.21.15.132:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fcd0c009040 con 0x5608ab872770 2024-03-28T00:25:41.292 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.279+0000 7fcd2af6a740 1 -- 172.21.15.119:0/3884647093 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcd14005020 con 0x5608ab872770 2024-03-28T00:25:41.292 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.279+0000 7fcd07400640 1 -- 172.21.15.119:0/3884647093 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcd0c004050 con 0x5608ab872770 2024-03-28T00:25:41.292 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.279+0000 7fcd2af6a740 10 client.25022 fetch_fsmap learned FSMap version 26 2024-03-28T00:25:41.292 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.279+0000 7fcd2af6a740 1 -- 172.21.15.119:0/3884647093 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x5608ab5b56b0 con 0x5608ab872770 2024-03-28T00:25:41.292 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.279+0000 7fcd07400640 1 -- 172.21.15.119:0/3884647093 <== mon.2 v2:172.21.15.132:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcd14005020 con 0x5608ab872770 2024-03-28T00:25:41.293 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.280+0000 7fcd07400640 1 -- 172.21.15.119:0/3884647093 <== mon.2 v2:172.21.15.132:3300/0 5 ==== fsmap(e 26) v1 ==== 7934+0+0 (secure 0 0 0) 0x7fcd0c029020 con 0x5608ab872770 2024-03-28T00:25:41.293 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.280+0000 7fcd2af6a740 10 client.25022 fetch_fsmap finished waiting for FSMap version 26 2024-03-28T00:25:41.293 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.280+0000 7fcd2af6a740 10 client.25022 resolve_mds: resolved name 'i' to daemon mds.i 2024-03-28T00:25:41.293 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.280+0000 7fcd2af6a740 20 client.25022 populate_metadata read hostname 'smithi119' 2024-03-28T00:25:41.293 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.280+0000 7fcd2af6a740 1 --2- 172.21.15.119:0/3884647093 >> [v2:172.21.15.165:6838/3004684592,v1:172.21.15.165:6839/3004684592] conn(0x5608ab888a10 0x5608ab887a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:41.293 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.280+0000 7fcd2af6a740 4 client.25022 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-28T00:25:41.293 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.280+0000 7fcd2af6a740 1 -- 172.21.15.119:0/3884647093 --> [v2:172.21.15.165:6838/3004684592,v1:172.21.15.165:6839/3004684592] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5608ab778520 con 0x5608ab888a10 2024-03-28T00:25:41.293 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.280+0000 7fcd25800640 1 --2- 172.21.15.119:0/3884647093 >> [v2:172.21.15.165:6838/3004684592,v1:172.21.15.165:6839/3004684592] conn(0x5608ab888a10 0x5608ab887a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:41.293 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.281+0000 7fcd07400640 10 client.25022 ms_handle_connect on v2:172.21.15.165:6838/3004684592 2024-03-28T00:25:41.293 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.281+0000 7fcd25800640 1 --2- 172.21.15.119:0/3884647093 >> [v2:172.21.15.165:6838/3004684592,v1:172.21.15.165:6839/3004684592] conn(0x5608ab888a10 0x5608ab887a90 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ec14c17fdd08334c server_cookie=9a5007f38f36d41 in_seq=0 out_seq=0 2024-03-28T00:25:41.293 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.281+0000 7fcd07400640 1 -- 172.21.15.119:0/3884647093 <== mds.1 v2:172.21.15.165:6838/3004684592 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5608ab778520 con 0x5608ab888a10 2024-03-28T00:25:41.293 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.281+0000 7fcd07400640 10 client.25022 handle_command_reply: tid=0 2024-03-28T00:25:41.294 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.281+0000 7fcd2af6a740 1 client.25022 shutdown 2024-03-28T00:25:41.294 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.282+0000 7fcd2af6a740 1 -- 172.21.15.119:0/3884647093 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5608ab872770 msgr2=0x5608ab87c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:41.294 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.282+0000 7fcd2af6a740 1 --2- 172.21.15.119:0/3884647093 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5608ab872770 0x5608ab87c770 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fcd0c00ff90 tx=0x7fcd0c005f90 comp rx=0 tx=0).stop 2024-03-28T00:25:41.294 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.282+0000 7fcd2af6a740 1 -- 172.21.15.119:0/3884647093 shutdown_connections 2024-03-28T00:25:41.294 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.282+0000 7fcd2af6a740 1 --2- 172.21.15.119:0/3884647093 >> [v2:172.21.15.165:6838/3004684592,v1:172.21.15.165:6839/3004684592] conn(0x5608ab888a10 0x5608ab887a90 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:41.294 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.282+0000 7fcd2af6a740 1 --2- 172.21.15.119:0/3884647093 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5608ab8778e0 0x5608ab8837b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:41.294 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.282+0000 7fcd2af6a740 1 --2- 172.21.15.119:0/3884647093 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5608ab872770 0x5608ab87c770 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:41.294 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.282+0000 7fcd2af6a740 1 --2- 172.21.15.119:0/3884647093 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5608ab871d80 0x5608ab87c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:41.294 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.282+0000 7fcd07400640 0 client.25022 ms_handle_reset on v2:172.21.15.165:6838/3004684592 2024-03-28T00:25:41.294 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.282+0000 7fcd2af6a740 1 -- 172.21.15.119:0/3884647093 >> 172.21.15.119:0/3884647093 conn(0x5608ab86d2f0 msgr2=0x5608ab86c4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:25:41.294 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.282+0000 7fcd2af6a740 1 -- 172.21.15.119:0/3884647093 shutdown_connections 2024-03-28T00:25:41.295 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.282+0000 7fcd2af6a740 1 -- 172.21.15.119:0/3884647093 wait complete. 2024-03-28T00:25:41.295 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.282+0000 7fcd2af6a740 20 client.25022 trim_cache size 0 max 16384 2024-03-28T00:25:41.307 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.299+0000 7fcd2af6a740 1 Processor -- start 2024-03-28T00:25:41.307 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.300+0000 7fcd2af6a740 1 -- start start 2024-03-28T00:25:41.307 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.300+0000 7fcd2af6a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5608ab871d80 0x5608ab87c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:41.307 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.300+0000 7fcd2af6a740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5608ab872770 0x5608ab883770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:41.307 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.300+0000 7fcd2af6a740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5608ab8778e0 0x5608ab86d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:41.307 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.300+0000 7fcd2af6a740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608ab888230 con 0x5608ab872770 2024-03-28T00:25:41.308 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.300+0000 7fcd2af6a740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608ab888890 con 0x5608ab8778e0 2024-03-28T00:25:41.308 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.300+0000 7fcd2af6a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608ab888ef0 con 0x5608ab871d80 2024-03-28T00:25:41.308 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.300+0000 7fcd25800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5608ab871d80 0x5608ab87c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:41.308 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.300+0000 7fcd24e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5608ab872770 0x5608ab883770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:41.308 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.300+0000 7fcd25800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5608ab871d80 0x5608ab87c6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.119:55742/0 (socket says 172.21.15.119:55742) 2024-03-28T00:25:41.308 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.300+0000 7fcd25800640 1 -- 172.21.15.119:0/114779227 learned_addr learned my addr 172.21.15.119:0/114779227 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:25:41.308 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.300+0000 7fcd26200640 1 --2- 172.21.15.119:0/114779227 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5608ab8778e0 0x5608ab86d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:41.308 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.300+0000 7fcd25800640 1 -- 172.21.15.119:0/114779227 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5608ab8778e0 msgr2=0x5608ab86d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T00:25:41.308 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.300+0000 7fcd25800640 1 --2- 172.21.15.119:0/114779227 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5608ab8778e0 0x5608ab86d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:41.308 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.300+0000 7fcd25800640 1 -- 172.21.15.119:0/114779227 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5608ab872770 msgr2=0x5608ab883770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T00:25:41.309 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.300+0000 7fcd25800640 1 --2- 172.21.15.119:0/114779227 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5608ab872770 0x5608ab883770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:41.309 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.300+0000 7fcd25800640 1 -- 172.21.15.119:0/114779227 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5608ab85f170 con 0x5608ab871d80 2024-03-28T00:25:41.309 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.300+0000 7fcd26200640 1 --2- 172.21.15.119:0/114779227 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5608ab8778e0 0x5608ab86d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T00:25:41.309 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.301+0000 7fcd25800640 1 --2- 172.21.15.119:0/114779227 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5608ab871d80 0x5608ab87c6d0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fcd14009380 tx=0x7fcd14009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd15958553c3ebf2 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:25:41.309 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.301+0000 7fcd12000640 1 -- 172.21.15.119:0/114779227 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd1402b2a0 con 0x5608ab871d80 2024-03-28T00:25:41.309 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.301+0000 7fcd12000640 1 -- 172.21.15.119:0/114779227 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcd140035f0 con 0x5608ab871d80 2024-03-28T00:25:41.309 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.301+0000 7fcd12000640 1 -- 172.21.15.119:0/114779227 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd1402b780 con 0x5608ab871d80 2024-03-28T00:25:41.309 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.301+0000 7fcd2af6a740 1 -- 172.21.15.119:0/114779227 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5608ab871d80 msgr2=0x5608ab87c6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:41.309 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.301+0000 7fcd2af6a740 1 --2- 172.21.15.119:0/114779227 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5608ab871d80 0x5608ab87c6d0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fcd14009380 tx=0x7fcd14009310 comp rx=0 tx=0).stop 2024-03-28T00:25:41.309 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.301+0000 7fcd2af6a740 1 -- 172.21.15.119:0/114779227 shutdown_connections 2024-03-28T00:25:41.309 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.301+0000 7fcd2af6a740 1 --2- 172.21.15.119:0/114779227 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5608ab8778e0 0x5608ab86d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:41.310 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.301+0000 7fcd2af6a740 1 --2- 172.21.15.119:0/114779227 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5608ab872770 0x5608ab883770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:41.310 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.301+0000 7fcd2af6a740 1 --2- 172.21.15.119:0/114779227 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5608ab871d80 0x5608ab87c6d0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:41.310 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.301+0000 7fcd2af6a740 1 -- 172.21.15.119:0/114779227 >> 172.21.15.119:0/114779227 conn(0x5608ab86d2f0 msgr2=0x5608ab887a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:25:41.310 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.301+0000 7fcd2af6a740 1 -- 172.21.15.119:0/114779227 shutdown_connections 2024-03-28T00:25:41.310 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.301+0000 7fcd2af6a740 1 -- 172.21.15.119:0/114779227 wait complete. 2024-03-28T00:25:41.310 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.302+0000 7fcd2af6a740 1 -- start start 2024-03-28T00:25:41.310 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.302+0000 7fcd2af6a740 1 Processor -- start 2024-03-28T00:25:41.315 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.302+0000 7fcd2af6a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5608ab871d80 0x5608ab87fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:41.315 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.302+0000 7fcd2af6a740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5608ab872770 0x5608ab87ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:41.315 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.302+0000 7fcd2af6a740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5608ab8778e0 0x5608ab882f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:41.315 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.302+0000 7fcd2af6a740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608ab889bb0 con 0x5608ab8778e0 2024-03-28T00:25:41.315 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.302+0000 7fcd2af6a740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608ab889550 con 0x5608ab872770 2024-03-28T00:25:41.315 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.302+0000 7fcd2af6a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608ab88a210 con 0x5608ab871d80 2024-03-28T00:25:41.316 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.302+0000 7fcd2af6a740 10 client.0 _ll_register_callbacks cb 0x7fcd0c00fc10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T00:25:41.316 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.303+0000 7fcd24e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5608ab872770 0x5608ab87ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:41.316 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.303+0000 7fcd25800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5608ab871d80 0x5608ab87fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:41.316 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.303+0000 7fcd24e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5608ab872770 0x5608ab87ff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.119:45622/0 (socket says 172.21.15.119:45622) 2024-03-28T00:25:41.316 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.303+0000 7fcd24e00640 1 -- 172.21.15.119:0/1903076664 learned_addr learned my addr 172.21.15.119:0/1903076664 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:25:41.316 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.303+0000 7fcd26200640 1 --2- 172.21.15.119:0/1903076664 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5608ab8778e0 0x5608ab882f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:41.316 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.303+0000 7fcd24e00640 1 -- 172.21.15.119:0/1903076664 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5608ab871d80 msgr2=0x5608ab87fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T00:25:41.316 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.303+0000 7fcd24e00640 1 --2- 172.21.15.119:0/1903076664 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5608ab871d80 0x5608ab87fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:41.316 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.303+0000 7fcd24e00640 1 -- 172.21.15.119:0/1903076664 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5608ab8778e0 msgr2=0x5608ab882f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T00:25:41.316 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.303+0000 7fcd24e00640 1 --2- 172.21.15.119:0/1903076664 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5608ab8778e0 0x5608ab882f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:41.316 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.303+0000 7fcd24e00640 1 -- 172.21.15.119:0/1903076664 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5608ab883510 con 0x5608ab872770 2024-03-28T00:25:41.316 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.303+0000 7fcd25800640 1 --2- 172.21.15.119:0/1903076664 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5608ab871d80 0x5608ab87fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T00:25:41.316 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.303+0000 7fcd24e00640 1 --2- 172.21.15.119:0/1903076664 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5608ab872770 0x5608ab87ff50 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fcd0c00f800 tx=0x7fcd0c010e60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e0fcd8fc6c382b5 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:25:41.317 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.303+0000 7fcd07400640 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-03-28T00:25:41.317 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.303+0000 7fcd07400640 1 -- 172.21.15.119:0/1903076664 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd0c03a030 con 0x5608ab872770 2024-03-28T00:25:41.317 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.303+0000 7fcd07400640 1 -- 172.21.15.119:0/1903076664 <== mon.2 v2:172.21.15.132:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fcd0c005500 con 0x5608ab872770 2024-03-28T00:25:41.317 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.303+0000 7fcd2af6a740 1 -- 172.21.15.119:0/1903076664 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcd140035f0 con 0x5608ab872770 2024-03-28T00:25:41.317 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.304+0000 7fcd07400640 1 -- 172.21.15.119:0/1903076664 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcd0c0208a0 con 0x5608ab872770 2024-03-28T00:25:41.317 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.304+0000 7fcd2af6a740 10 client.25028 fetch_fsmap learned FSMap version 26 2024-03-28T00:25:41.317 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.304+0000 7fcd2af6a740 1 -- 172.21.15.119:0/1903076664 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x5608ab8e45b0 con 0x5608ab872770 2024-03-28T00:25:41.317 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.304+0000 7fcd07400640 1 -- 172.21.15.119:0/1903076664 <== mon.2 v2:172.21.15.132:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcd140035f0 con 0x5608ab872770 2024-03-28T00:25:41.317 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.304+0000 7fcd07400640 1 -- 172.21.15.119:0/1903076664 <== mon.2 v2:172.21.15.132:3300/0 5 ==== fsmap(e 26) v1 ==== 7934+0+0 (secure 0 0 0) 0x7fcd0c029040 con 0x5608ab872770 2024-03-28T00:25:41.317 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.304+0000 7fcd2af6a740 10 client.25028 fetch_fsmap finished waiting for FSMap version 26 2024-03-28T00:25:41.317 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.304+0000 7fcd2af6a740 10 client.25028 resolve_mds: resolved name 'i' to daemon mds.i 2024-03-28T00:25:41.317 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.304+0000 7fcd2af6a740 20 client.25028 populate_metadata read hostname 'smithi119' 2024-03-28T00:25:41.318 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.304+0000 7fcd2af6a740 1 --2- 172.21.15.119:0/1903076664 >> [v2:172.21.15.165:6838/3004684592,v1:172.21.15.165:6839/3004684592] conn(0x5608ab8e8c40 0x5608ab8e4840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:41.318 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.304+0000 7fcd2af6a740 4 client.25028 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-28T00:25:41.318 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.304+0000 7fcd2af6a740 1 -- 172.21.15.119:0/1903076664 --> [v2:172.21.15.165:6838/3004684592,v1:172.21.15.165:6839/3004684592] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5608ab7e41d0 con 0x5608ab8e8c40 2024-03-28T00:25:41.318 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.304+0000 7fcd25800640 1 --2- 172.21.15.119:0/1903076664 >> [v2:172.21.15.165:6838/3004684592,v1:172.21.15.165:6839/3004684592] conn(0x5608ab8e8c40 0x5608ab8e4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:41.318 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.305+0000 7fcd25800640 1 --2- 172.21.15.119:0/1903076664 >> [v2:172.21.15.165:6838/3004684592,v1:172.21.15.165:6839/3004684592] conn(0x5608ab8e8c40 0x5608ab8e4840 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f3bf0034b254fa7b server_cookie=bba12d5a36cf3950 in_seq=0 out_seq=0 2024-03-28T00:25:41.318 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.305+0000 7fcd07400640 10 client.25028 ms_handle_connect on v2:172.21.15.165:6838/3004684592 2024-03-28T00:25:41.318 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.305+0000 7fcd07400640 1 -- 172.21.15.119:0/1903076664 <== mds.1 v2:172.21.15.165:6838/3004684592 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+322 (crc 0 0 0) 0x5608ab7e41d0 con 0x5608ab8e8c40 2024-03-28T00:25:41.318 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.305+0000 7fcd07400640 10 client.25028 handle_command_reply: tid=0 2024-03-28T00:25:41.318 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.305+0000 7fcd2af6a740 1 client.25028 shutdown 2024-03-28T00:25:41.318 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.305+0000 7fcd2af6a740 1 -- 172.21.15.119:0/1903076664 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5608ab872770 msgr2=0x5608ab87ff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:41.318 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.305+0000 7fcd2af6a740 1 --2- 172.21.15.119:0/1903076664 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5608ab872770 0x5608ab87ff50 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fcd0c00f800 tx=0x7fcd0c010e60 comp rx=0 tx=0).stop 2024-03-28T00:25:41.319 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.305+0000 7fcd2af6a740 1 -- 172.21.15.119:0/1903076664 shutdown_connections 2024-03-28T00:25:41.319 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.305+0000 7fcd2af6a740 1 --2- 172.21.15.119:0/1903076664 >> [v2:172.21.15.165:6838/3004684592,v1:172.21.15.165:6839/3004684592] conn(0x5608ab8e8c40 0x5608ab8e4840 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:41.319 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.305+0000 7fcd2af6a740 1 --2- 172.21.15.119:0/1903076664 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5608ab8778e0 0x5608ab882f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:41.319 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.305+0000 7fcd2af6a740 1 --2- 172.21.15.119:0/1903076664 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5608ab872770 0x5608ab87ff50 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:41.319 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.305+0000 7fcd2af6a740 1 --2- 172.21.15.119:0/1903076664 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5608ab871d80 0x5608ab87fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:41.319 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.305+0000 7fcd07400640 0 client.25028 ms_handle_reset on v2:172.21.15.165:6838/3004684592 2024-03-28T00:25:41.319 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.305+0000 7fcd2af6a740 1 -- 172.21.15.119:0/1903076664 >> 172.21.15.119:0/1903076664 conn(0x5608ab86d2f0 msgr2=0x5608ab86d6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:25:41.319 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.305+0000 7fcd2af6a740 1 -- 172.21.15.119:0/1903076664 shutdown_connections 2024-03-28T00:25:41.319 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.305+0000 7fcd2af6a740 1 -- 172.21.15.119:0/1903076664 wait complete. 2024-03-28T00:25:41.319 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.306+0000 7fcd2af6a740 20 client.25028 trim_cache size 0 max 16384 2024-03-28T00:25:41.319 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-03-28T00:25:41.320 INFO:teuthology.orchestra.run.smithi119.stdout: "cluster_fsid": "10f73b92-ec98-11ee-95d0-87774f69a715", 2024-03-28T00:25:41.320 INFO:teuthology.orchestra.run.smithi119.stdout: "whoami": 1, 2024-03-28T00:25:41.320 INFO:teuthology.orchestra.run.smithi119.stdout: "id": 24550, 2024-03-28T00:25:41.320 INFO:teuthology.orchestra.run.smithi119.stdout: "want_state": "up:active", 2024-03-28T00:25:41.320 INFO:teuthology.orchestra.run.smithi119.stdout: "state": "up:active", 2024-03-28T00:25:41.320 INFO:teuthology.orchestra.run.smithi119.stdout: "fs_name": "cephfs", 2024-03-28T00:25:41.320 INFO:teuthology.orchestra.run.smithi119.stdout: "rank_uptime": 100.830002271, 2024-03-28T00:25:41.320 INFO:teuthology.orchestra.run.smithi119.stdout: "mdsmap_epoch": 26, 2024-03-28T00:25:41.320 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_epoch": 87, 2024-03-28T00:25:41.320 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_epoch_barrier": 85, 2024-03-28T00:25:41.320 INFO:teuthology.orchestra.run.smithi119.stdout: "uptime": 121.960433686 2024-03-28T00:25:41.320 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-03-28T00:25:41.321 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.306+0000 7fcd26c00640 1 -- 172.21.15.119:0/1388261956 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fcd080a2800 msgr2=0x7fcd080a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:41.322 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.306+0000 7fcd26c00640 1 --2- 172.21.15.119:0/1388261956 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fcd080a2800 0x7fcd080a4cc0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x5608ab872fd0 tx=0x7fcd0c005850 comp rx=0 tx=0).stop 2024-03-28T00:25:41.322 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.306+0000 7fcd26c00640 1 -- 172.21.15.119:0/1388261956 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcd20109770 msgr2=0x7fcd201a53d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:41.322 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.306+0000 7fcd26c00640 1 --2- 172.21.15.119:0/1388261956 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcd20109770 0x7fcd201a53d0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fcd1802c640 tx=0x7fcd18035d10 comp rx=0 tx=0).stop 2024-03-28T00:25:41.322 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.306+0000 7fcd26c00640 1 -- 172.21.15.119:0/1388261956 shutdown_connections 2024-03-28T00:25:41.322 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.306+0000 7fcd26c00640 1 --2- 172.21.15.119:0/1388261956 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fcd080a2800 0x7fcd080a4cc0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:41.322 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.306+0000 7fcd26c00640 1 --2- 172.21.15.119:0/1388261956 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcd20109770 0x7fcd201a53d0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:41.322 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.306+0000 7fcd26c00640 1 --2- 172.21.15.119:0/1388261956 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcd20108db0 0x7fcd2019e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:41.322 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.306+0000 7fcd26c00640 1 --2- 172.21.15.119:0/1388261956 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcd20107bc0 0x7fcd2019de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:41.322 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.306+0000 7fcd26c00640 1 -- 172.21.15.119:0/1388261956 >> 172.21.15.119:0/1388261956 conn(0x7fcd20076b70 msgr2=0x7fcd2010c0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:25:41.322 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.306+0000 7fcd26c00640 1 -- 172.21.15.119:0/1388261956 shutdown_connections 2024-03-28T00:25:41.322 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.306+0000 7fcd26c00640 1 -- 172.21.15.119:0/1388261956 wait complete. 2024-03-28T00:25:41.328 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-28T00:25:41.520 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.514+0000 7f007be00640 1 -- 172.21.15.119:0/1092237618 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f00741038e0 msgr2=0x7f0074103d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:41.521 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.514+0000 7f007be00640 1 --2- 172.21.15.119:0/1092237618 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f00741038e0 0x7f0074103d00 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f0068009900 tx=0x7f006802bc30 comp rx=0 tx=0).stop 2024-03-28T00:25:41.521 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.515+0000 7f007be00640 1 -- 172.21.15.119:0/1092237618 shutdown_connections 2024-03-28T00:25:41.521 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.515+0000 7f007be00640 1 --2- 172.21.15.119:0/1092237618 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0074105490 0x7f007410bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:41.521 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.515+0000 7f007be00640 1 --2- 172.21.15.119:0/1092237618 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0074104ad0 0x7f0074104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:41.521 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.515+0000 7f007be00640 1 --2- 172.21.15.119:0/1092237618 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f00741038e0 0x7f0074103d00 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:41.521 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.515+0000 7f007be00640 1 -- 172.21.15.119:0/1092237618 >> 172.21.15.119:0/1092237618 conn(0x7f00740fee50 msgr2=0x7f0074101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:25:41.522 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.515+0000 7f007be00640 1 -- 172.21.15.119:0/1092237618 shutdown_connections 2024-03-28T00:25:41.522 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.515+0000 7f007be00640 1 -- 172.21.15.119:0/1092237618 wait complete. 2024-03-28T00:25:41.524 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.516+0000 7f007be00640 1 Processor -- start 2024-03-28T00:25:41.525 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.516+0000 7f007be00640 1 -- start start 2024-03-28T00:25:41.525 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.516+0000 7f007be00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f00741038e0 0x7f0074199c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:41.525 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.516+0000 7f007be00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0074104ad0 0x7f007419a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:41.525 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.516+0000 7f007be00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0074105490 0x7f00741a11e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:41.525 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.516+0000 7f007be00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f007410f5a0 con 0x7f0074104ad0 2024-03-28T00:25:41.525 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.516+0000 7f007be00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f007410f420 con 0x7f0074105490 2024-03-28T00:25:41.525 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.516+0000 7f007be00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f007410f720 con 0x7f00741038e0 2024-03-28T00:25:41.525 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.516+0000 7f0079400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f00741038e0 0x7f0074199c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:41.525 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.517+0000 7f0079400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f00741038e0 0x7f0074199c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.119:55770/0 (socket says 172.21.15.119:55770) 2024-03-28T00:25:41.525 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.517+0000 7f0079400640 1 -- 172.21.15.119:0/1310260387 learned_addr learned my addr 172.21.15.119:0/1310260387 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:25:41.525 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.517+0000 7f0079e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0074105490 0x7f00741a11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:41.525 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.517+0000 7f0078a00640 1 --2- 172.21.15.119:0/1310260387 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0074104ad0 0x7f007419a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:41.525 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.517+0000 7f0079400640 1 -- 172.21.15.119:0/1310260387 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0074105490 msgr2=0x7f00741a11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:41.526 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.517+0000 7f0079400640 1 --2- 172.21.15.119:0/1310260387 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0074105490 0x7f00741a11e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:41.526 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.517+0000 7f0079400640 1 -- 172.21.15.119:0/1310260387 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0074104ad0 msgr2=0x7f007419a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:41.526 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.517+0000 7f0079400640 1 --2- 172.21.15.119:0/1310260387 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0074104ad0 0x7f007419a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:41.526 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.517+0000 7f0079400640 1 -- 172.21.15.119:0/1310260387 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00741a1880 con 0x7f00741038e0 2024-03-28T00:25:41.526 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.517+0000 7f0078a00640 1 --2- 172.21.15.119:0/1310260387 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0074104ad0 0x7f007419a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T00:25:41.526 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.517+0000 7f0079400640 1 --2- 172.21.15.119:0/1310260387 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f00741038e0 0x7f0074199c70 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f0068000c00 tx=0x7f006802c6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:25:41.526 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.517+0000 7f0072000640 1 -- 172.21.15.119:0/1310260387 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0068043020 con 0x7f00741038e0 2024-03-28T00:25:41.526 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.517+0000 7f0072000640 1 -- 172.21.15.119:0/1310260387 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f00680054e0 con 0x7f00741038e0 2024-03-28T00:25:41.526 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.517+0000 7f007be00640 1 -- 172.21.15.119:0/1310260387 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00741a1b10 con 0x7f00741038e0 2024-03-28T00:25:41.526 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.518+0000 7f0072000640 1 -- 172.21.15.119:0/1310260387 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0068003ce0 con 0x7f00741038e0 2024-03-28T00:25:41.526 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.518+0000 7f007be00640 1 -- 172.21.15.119:0/1310260387 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00741a2030 con 0x7f00741038e0 2024-03-28T00:25:41.529 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.521+0000 7f007ea3f740 1 Processor -- start 2024-03-28T00:25:41.529 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.521+0000 7f0072000640 1 -- 172.21.15.119:0/1310260387 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 18) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f0068046020 con 0x7f00741038e0 2024-03-28T00:25:41.529 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.521+0000 7f007ea3f740 1 -- start start 2024-03-28T00:25:41.529 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.521+0000 7f007ea3f740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564ef4bc7ba0 0x564ef4bc7fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:41.529 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.521+0000 7f007ea3f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564ef4bc8590 0x564ef4bcd090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:41.529 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.521+0000 7f007ea3f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564ef4bcd700 0x564ef4bcfb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:41.530 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.521+0000 7f007ea3f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ef4b2cc90 con 0x564ef4bc8590 2024-03-28T00:25:41.530 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.521+0000 7f007ea3f740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ef4ad5980 con 0x564ef4bc7ba0 2024-03-28T00:25:41.530 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.521+0000 7f007ea3f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ef4b28d30 con 0x564ef4bcd700 2024-03-28T00:25:41.530 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.522+0000 7f0078a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564ef4bcd700 0x564ef4bcfb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:41.530 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.522+0000 7f0078a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564ef4bcd700 0x564ef4bcfb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.119:55782/0 (socket says 172.21.15.119:55782) 2024-03-28T00:25:41.530 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.522+0000 7f0078a00640 1 -- 172.21.15.119:0/171735913 learned_addr learned my addr 172.21.15.119:0/171735913 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:25:41.530 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.522+0000 7f0079e00640 1 --2- 172.21.15.119:0/171735913 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564ef4bc7ba0 0x564ef4bc7fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:41.530 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.522+0000 7f0078a00640 1 -- 172.21.15.119:0/171735913 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564ef4bc7ba0 msgr2=0x564ef4bc7fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T00:25:41.530 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.522+0000 7f0078a00640 1 --2- 172.21.15.119:0/171735913 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564ef4bc7ba0 0x564ef4bc7fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:41.530 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.522+0000 7f0078a00640 1 -- 172.21.15.119:0/171735913 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564ef4bc8590 msgr2=0x564ef4bcd090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T00:25:41.530 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.522+0000 7f0078a00640 1 --2- 172.21.15.119:0/171735913 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564ef4bc8590 0x564ef4bcd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:41.530 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.522+0000 7f0078a00640 1 -- 172.21.15.119:0/171735913 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564ef4aed510 con 0x564ef4bcd700 2024-03-28T00:25:41.530 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.522+0000 7f0078a00640 1 --2- 172.21.15.119:0/171735913 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564ef4bcd700 0x564ef4bcfb90 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f0060009d40 tx=0x7f00600117b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4432185258ee4a40 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:25:41.531 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.522+0000 7f0072000640 1 --2- 172.21.15.119:0/1310260387 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f005c0a2800 0x7f005c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:41.531 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.522+0000 7f0066600640 1 -- 172.21.15.119:0/171735913 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0060012650 con 0x564ef4bcd700 2024-03-28T00:25:41.531 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.522+0000 7f0072000640 1 -- 172.21.15.119:0/1310260387 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f00680f0c80 con 0x7f00741038e0 2024-03-28T00:25:41.531 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.522+0000 7f0066600640 1 -- 172.21.15.119:0/171735913 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0060009e30 con 0x564ef4bcd700 2024-03-28T00:25:41.531 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.523+0000 7f0066600640 1 -- 172.21.15.119:0/171735913 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f006000fc70 con 0x564ef4bcd700 2024-03-28T00:25:41.531 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.523+0000 7f0079e00640 1 --2- 172.21.15.119:0/1310260387 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f005c0a2800 0x7f005c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:41.531 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.523+0000 7f007ea3f740 1 -- 172.21.15.119:0/171735913 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564ef4bcd700 msgr2=0x564ef4bcfb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:41.531 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.523+0000 7f0079e00640 1 --2- 172.21.15.119:0/1310260387 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f005c0a2800 0x7f005c0a4cc0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x564ef4bc8df0 tx=0x7f006c009600 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:25:41.531 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.523+0000 7f007ea3f740 1 --2- 172.21.15.119:0/171735913 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564ef4bcd700 0x564ef4bcfb90 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f0060009d40 tx=0x7f00600117b0 comp rx=0 tx=0).stop 2024-03-28T00:25:41.531 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.523+0000 7f007ea3f740 1 -- 172.21.15.119:0/171735913 shutdown_connections 2024-03-28T00:25:41.531 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.523+0000 7f007ea3f740 1 --2- 172.21.15.119:0/171735913 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564ef4bcd700 0x564ef4bcfb90 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:41.531 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.523+0000 7f007ea3f740 1 --2- 172.21.15.119:0/171735913 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564ef4bc8590 0x564ef4bcd090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:41.531 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.523+0000 7f007ea3f740 1 --2- 172.21.15.119:0/171735913 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564ef4bc7ba0 0x564ef4bc7fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:41.531 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.523+0000 7f007ea3f740 1 -- 172.21.15.119:0/171735913 >> 172.21.15.119:0/171735913 conn(0x564ef4bc3110 msgr2=0x564ef4bc3510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:25:41.532 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.524+0000 7f007ea3f740 1 -- 172.21.15.119:0/171735913 shutdown_connections 2024-03-28T00:25:41.532 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.524+0000 7f007ea3f740 1 -- 172.21.15.119:0/171735913 wait complete. 2024-03-28T00:25:41.532 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.524+0000 7f007ea3f740 1 -- start start 2024-03-28T00:25:41.532 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.524+0000 7f007ea3f740 1 Processor -- start 2024-03-28T00:25:41.535 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.525+0000 7f007ea3f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564ef4bc7ba0 0x564ef4bd20b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:41.535 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.525+0000 7f007ea3f740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564ef4bc8590 0x564ef4bd25f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:41.535 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.525+0000 7f007ea3f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564ef4bcd700 0x564ef4bd9630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:41.535 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.525+0000 7f007ea3f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ef4bd06b0 con 0x564ef4bc7ba0 2024-03-28T00:25:41.535 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.525+0000 7f007ea3f740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ef4ae8dd0 con 0x564ef4bc8590 2024-03-28T00:25:41.535 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.525+0000 7f007ea3f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ef4bd0830 con 0x564ef4bcd700 2024-03-28T00:25:41.535 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.525+0000 7f007ea3f740 10 client.0 _ll_register_callbacks cb 0x564ef4840ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T00:25:41.535 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.525+0000 7f0079400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564ef4bc7ba0 0x564ef4bd20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:41.536 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.525+0000 7f0079400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564ef4bc7ba0 0x564ef4bd20b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:60526/0 (socket says 172.21.15.119:60526) 2024-03-28T00:25:41.536 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.525+0000 7f0079e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564ef4bcd700 0x564ef4bd9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:41.536 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.525+0000 7f0079400640 1 -- 172.21.15.119:0/111570382 learned_addr learned my addr 172.21.15.119:0/111570382 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:25:41.536 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.525+0000 7f0078a00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564ef4bc8590 0x564ef4bd25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:41.536 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.525+0000 7f0078a00640 1 -- 172.21.15.119:0/111570382 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564ef4bcd700 msgr2=0x564ef4bd9630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T00:25:41.536 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.525+0000 7f0078a00640 1 --2- 172.21.15.119:0/111570382 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564ef4bcd700 0x564ef4bd9630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:41.536 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.525+0000 7f0078a00640 1 -- 172.21.15.119:0/111570382 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564ef4bc7ba0 msgr2=0x564ef4bd20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T00:25:41.536 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.525+0000 7f0078a00640 1 --2- 172.21.15.119:0/111570382 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564ef4bc7ba0 0x564ef4bd20b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:41.536 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.525+0000 7f0078a00640 1 -- 172.21.15.119:0/111570382 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564ef48412d0 con 0x564ef4bc8590 2024-03-28T00:25:41.536 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.525+0000 7f0079400640 1 --2- 172.21.15.119:0/111570382 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564ef4bc7ba0 0x564ef4bd20b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T00:25:41.536 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.526+0000 7f0078a00640 1 --2- 172.21.15.119:0/111570382 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564ef4bc8590 0x564ef4bd25f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f006001c040 tx=0x7f00600052f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=40c125d90e3444bf server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:25:41.536 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.526+0000 7f005aa00640 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-03-28T00:25:41.536 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.526+0000 7f005aa00640 1 -- 172.21.15.119:0/111570382 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0060028020 con 0x564ef4bc8590 2024-03-28T00:25:41.536 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.526+0000 7f005aa00640 1 -- 172.21.15.119:0/111570382 <== mon.2 v2:172.21.15.132:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f00600048c0 con 0x564ef4bc8590 2024-03-28T00:25:41.536 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.526+0000 7f007ea3f740 1 -- 172.21.15.119:0/111570382 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0060009e30 con 0x564ef4bc8590 2024-03-28T00:25:41.537 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.526+0000 7f005aa00640 1 -- 172.21.15.119:0/111570382 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f006000f440 con 0x564ef4bc8590 2024-03-28T00:25:41.537 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.526+0000 7f007ea3f740 10 client.25040 fetch_fsmap learned FSMap version 26 2024-03-28T00:25:41.537 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.526+0000 7f007ea3f740 1 -- 172.21.15.119:0/111570382 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x564ef490b6b0 con 0x564ef4bc8590 2024-03-28T00:25:41.537 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.526+0000 7f005aa00640 1 -- 172.21.15.119:0/111570382 <== mon.2 v2:172.21.15.132:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0060009e30 con 0x564ef4bc8590 2024-03-28T00:25:41.537 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.526+0000 7f005aa00640 1 -- 172.21.15.119:0/111570382 <== mon.2 v2:172.21.15.132:3300/0 5 ==== fsmap(e 26) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f00600126f0 con 0x564ef4bc8590 2024-03-28T00:25:41.537 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.526+0000 7f007ea3f740 10 client.25040 fetch_fsmap finished waiting for FSMap version 26 2024-03-28T00:25:41.537 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.527+0000 7f007ea3f740 10 client.25040 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-28T00:25:41.537 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.527+0000 7f007ea3f740 20 client.25040 populate_metadata read hostname 'smithi119' 2024-03-28T00:25:41.537 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.527+0000 7f007ea3f740 1 --2- 172.21.15.119:0/111570382 >> [v2:172.21.15.132:6838/3305822525,v1:172.21.15.132:6839/3305822525] conn(0x564ef4bdd8a0 0x564ef4be28f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:41.537 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.527+0000 7f007ea3f740 4 client.25040 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-28T00:25:41.537 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.527+0000 7f007ea3f740 1 -- 172.21.15.119:0/111570382 --> [v2:172.21.15.132:6838/3305822525,v1:172.21.15.132:6839/3305822525] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564ef4ace520 con 0x564ef4bdd8a0 2024-03-28T00:25:41.537 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.527+0000 7f0079400640 1 --2- 172.21.15.119:0/111570382 >> [v2:172.21.15.132:6838/3305822525,v1:172.21.15.132:6839/3305822525] conn(0x564ef4bdd8a0 0x564ef4be28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:41.537 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.527+0000 7f0079400640 1 --2- 172.21.15.119:0/111570382 >> [v2:172.21.15.132:6838/3305822525,v1:172.21.15.132:6839/3305822525] conn(0x564ef4bdd8a0 0x564ef4be28f0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=29822d899cf2065 server_cookie=57dcf80878cd6cae in_seq=0 out_seq=0 2024-03-28T00:25:41.537 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.527+0000 7f005aa00640 10 client.25040 ms_handle_connect on v2:172.21.15.132:6838/3305822525 2024-03-28T00:25:41.537 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.528+0000 7f005aa00640 1 -- 172.21.15.119:0/111570382 <== mds.0 v2:172.21.15.132:6838/3305822525 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x564ef4ace520 con 0x564ef4bdd8a0 2024-03-28T00:25:41.537 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.528+0000 7f005aa00640 10 client.25040 handle_command_reply: tid=0 2024-03-28T00:25:41.538 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.528+0000 7f007ea3f740 1 client.25040 shutdown 2024-03-28T00:25:41.538 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.529+0000 7f007ea3f740 1 -- 172.21.15.119:0/111570382 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564ef4bc8590 msgr2=0x564ef4bd25f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:41.538 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.529+0000 7f007ea3f740 1 --2- 172.21.15.119:0/111570382 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564ef4bc8590 0x564ef4bd25f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f006001c040 tx=0x7f00600052f0 comp rx=0 tx=0).stop 2024-03-28T00:25:41.538 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.529+0000 7f007ea3f740 1 -- 172.21.15.119:0/111570382 shutdown_connections 2024-03-28T00:25:41.538 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.529+0000 7f007ea3f740 1 --2- 172.21.15.119:0/111570382 >> [v2:172.21.15.132:6838/3305822525,v1:172.21.15.132:6839/3305822525] conn(0x564ef4bdd8a0 0x564ef4be28f0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:41.538 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.529+0000 7f007ea3f740 1 --2- 172.21.15.119:0/111570382 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564ef4bcd700 0x564ef4bd9630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:41.538 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.529+0000 7f007ea3f740 1 --2- 172.21.15.119:0/111570382 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564ef4bc8590 0x564ef4bd25f0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:41.538 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.529+0000 7f005aa00640 0 client.25040 ms_handle_reset on v2:172.21.15.132:6838/3305822525 2024-03-28T00:25:41.538 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.529+0000 7f007ea3f740 1 --2- 172.21.15.119:0/111570382 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564ef4bc7ba0 0x564ef4bd20b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:41.538 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.529+0000 7f007ea3f740 1 -- 172.21.15.119:0/111570382 >> 172.21.15.119:0/111570382 conn(0x564ef4bc3110 msgr2=0x564ef4bc2310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:25:41.538 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.529+0000 7f007ea3f740 1 -- 172.21.15.119:0/111570382 shutdown_connections 2024-03-28T00:25:41.538 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.529+0000 7f007ea3f740 1 -- 172.21.15.119:0/111570382 wait complete. 2024-03-28T00:25:41.538 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.529+0000 7f007ea3f740 20 client.25040 trim_cache size 0 max 16384 2024-03-28T00:25:41.552 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.547+0000 7f007ea3f740 1 Processor -- start 2024-03-28T00:25:41.552 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.547+0000 7f007ea3f740 1 -- start start 2024-03-28T00:25:41.552 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.547+0000 7f007ea3f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564ef4bc7ba0 0x564ef4bd2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:41.552 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.547+0000 7f007ea3f740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564ef4bc8590 0x564ef4bd95f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:41.552 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.547+0000 7f007ea3f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564ef4bcd700 0x564ef4bbde00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:41.552 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.547+0000 7f007ea3f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ef4bddc90 con 0x564ef4bc7ba0 2024-03-28T00:25:41.552 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.547+0000 7f007ea3f740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ef4bde2f0 con 0x564ef4bc8590 2024-03-28T00:25:41.552 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.547+0000 7f007ea3f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ef4bde950 con 0x564ef4bcd700 2024-03-28T00:25:41.555 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.547+0000 7f0079e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564ef4bcd700 0x564ef4bbde00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:41.555 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.547+0000 7f0078a00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564ef4bc8590 0x564ef4bd95f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:41.555 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.547+0000 7f0078a00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564ef4bc8590 0x564ef4bd95f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.119:45666/0 (socket says 172.21.15.119:45666) 2024-03-28T00:25:41.555 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.547+0000 7f0078a00640 1 -- 172.21.15.119:0/1991409463 learned_addr learned my addr 172.21.15.119:0/1991409463 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:25:41.555 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.547+0000 7f0079e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564ef4bcd700 0x564ef4bbde00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.119:55800/0 (socket says 172.21.15.119:55800) 2024-03-28T00:25:41.555 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.547+0000 7f0079400640 1 --2- 172.21.15.119:0/1991409463 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564ef4bc7ba0 0x564ef4bd2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:41.555 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.548+0000 7f0078a00640 1 -- 172.21.15.119:0/1991409463 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564ef4bcd700 msgr2=0x564ef4bbde00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T00:25:41.555 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.548+0000 7f0078a00640 1 --2- 172.21.15.119:0/1991409463 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564ef4bcd700 0x564ef4bbde00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:41.555 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.548+0000 7f0078a00640 1 -- 172.21.15.119:0/1991409463 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564ef4bc7ba0 msgr2=0x564ef4bd2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T00:25:41.555 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.548+0000 7f0078a00640 1 --2- 172.21.15.119:0/1991409463 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564ef4bc7ba0 0x564ef4bd2550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:41.555 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.548+0000 7f0078a00640 1 -- 172.21.15.119:0/1991409463 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564ef4bc3e80 con 0x564ef4bc8590 2024-03-28T00:25:41.555 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.548+0000 7f0078a00640 1 --2- 172.21.15.119:0/1991409463 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564ef4bc8590 0x564ef4bd95f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f0060012be0 tx=0x7f006000fd70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3671c91a76cd6c4e server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:25:41.555 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.548+0000 7f0065c00640 1 -- 172.21.15.119:0/1991409463 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f006003c070 con 0x564ef4bc8590 2024-03-28T00:25:41.556 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.548+0000 7f0065c00640 1 -- 172.21.15.119:0/1991409463 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f00600095a0 con 0x564ef4bc8590 2024-03-28T00:25:41.556 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.548+0000 7f0065c00640 1 -- 172.21.15.119:0/1991409463 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0060028040 con 0x564ef4bc8590 2024-03-28T00:25:41.556 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.549+0000 7f007ea3f740 1 -- 172.21.15.119:0/1991409463 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564ef4bc8590 msgr2=0x564ef4bd95f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:41.556 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.549+0000 7f007ea3f740 1 --2- 172.21.15.119:0/1991409463 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564ef4bc8590 0x564ef4bd95f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f0060012be0 tx=0x7f006000fd70 comp rx=0 tx=0).stop 2024-03-28T00:25:41.556 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.549+0000 7f007ea3f740 1 -- 172.21.15.119:0/1991409463 shutdown_connections 2024-03-28T00:25:41.556 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.549+0000 7f007ea3f740 1 --2- 172.21.15.119:0/1991409463 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564ef4bcd700 0x564ef4bbde00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:41.556 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.549+0000 7f007ea3f740 1 --2- 172.21.15.119:0/1991409463 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564ef4bc8590 0x564ef4bd95f0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:41.556 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.549+0000 7f007ea3f740 1 --2- 172.21.15.119:0/1991409463 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564ef4bc7ba0 0x564ef4bd2550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:41.556 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.549+0000 7f007ea3f740 1 -- 172.21.15.119:0/1991409463 >> 172.21.15.119:0/1991409463 conn(0x564ef4bc3110 msgr2=0x564ef4bc8f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:25:41.556 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.549+0000 7f007ea3f740 1 -- 172.21.15.119:0/1991409463 shutdown_connections 2024-03-28T00:25:41.556 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.549+0000 7f007ea3f740 1 -- 172.21.15.119:0/1991409463 wait complete. 2024-03-28T00:25:41.557 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.549+0000 7f007ea3f740 1 -- start start 2024-03-28T00:25:41.557 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.550+0000 7f007ea3f740 1 Processor -- start 2024-03-28T00:25:41.560 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.550+0000 7f007ea3f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564ef4bc7ba0 0x564ef4bd5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:41.561 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.550+0000 7f007ea3f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564ef4bc8590 0x564ef4bd5990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:41.561 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.550+0000 7f007ea3f740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564ef4bcd700 0x564ef4bd8910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:41.561 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.550+0000 7f007ea3f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ef4bdf610 con 0x564ef4bc7ba0 2024-03-28T00:25:41.561 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.550+0000 7f007ea3f740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ef4bdefb0 con 0x564ef4bcd700 2024-03-28T00:25:41.561 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.550+0000 7f007ea3f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ef4bdfc70 con 0x564ef4bc8590 2024-03-28T00:25:41.561 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.550+0000 7f007ea3f740 10 client.0 _ll_register_callbacks cb 0x7f006c009a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T00:25:41.561 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.550+0000 7f0078a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564ef4bc8590 0x564ef4bd5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:41.561 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.550+0000 7f0079e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564ef4bcd700 0x564ef4bd8910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:41.561 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.550+0000 7f0078a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564ef4bc8590 0x564ef4bd5990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.119:55808/0 (socket says 172.21.15.119:55808) 2024-03-28T00:25:41.561 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.550+0000 7f0078a00640 1 -- 172.21.15.119:0/3485614737 learned_addr learned my addr 172.21.15.119:0/3485614737 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:25:41.561 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.550+0000 7f0079400640 1 --2- 172.21.15.119:0/3485614737 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564ef4bc7ba0 0x564ef4bd5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:41.561 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.550+0000 7f0078a00640 1 -- 172.21.15.119:0/3485614737 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564ef4bcd700 msgr2=0x564ef4bd8910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T00:25:41.561 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.550+0000 7f0078a00640 1 --2- 172.21.15.119:0/3485614737 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564ef4bcd700 0x564ef4bd8910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:41.561 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.550+0000 7f0078a00640 1 -- 172.21.15.119:0/3485614737 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564ef4bc7ba0 msgr2=0x564ef4bd5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T00:25:41.562 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.550+0000 7f0078a00640 1 --2- 172.21.15.119:0/3485614737 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564ef4bc7ba0 0x564ef4bd5450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:41.562 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.551+0000 7f0078a00640 1 -- 172.21.15.119:0/3485614737 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564ef4bd8f00 con 0x564ef4bc8590 2024-03-28T00:25:41.562 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.551+0000 7f0078a00640 1 --2- 172.21.15.119:0/3485614737 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564ef4bc8590 0x564ef4bd5990 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f0060011cb0 tx=0x7f0060012870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b2b756d8fa72ea3c server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:25:41.562 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.551+0000 7f005aa00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-03-28T00:25:41.562 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.551+0000 7f005aa00640 1 -- 172.21.15.119:0/3485614737 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f006003c070 con 0x564ef4bc8590 2024-03-28T00:25:41.562 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.551+0000 7f007ea3f740 1 -- 172.21.15.119:0/3485614737 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f00600095a0 con 0x564ef4bc8590 2024-03-28T00:25:41.562 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.551+0000 7f005aa00640 1 -- 172.21.15.119:0/3485614737 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f0060005250 con 0x564ef4bc8590 2024-03-28T00:25:41.562 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.551+0000 7f005aa00640 1 -- 172.21.15.119:0/3485614737 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0060028070 con 0x564ef4bc8590 2024-03-28T00:25:41.562 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.551+0000 7f007ea3f740 10 client.25057 fetch_fsmap learned FSMap version 26 2024-03-28T00:25:41.562 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.551+0000 7f007ea3f740 1 -- 172.21.15.119:0/3485614737 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x564ef4c42860 con 0x564ef4bc8590 2024-03-28T00:25:41.562 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.552+0000 7f005aa00640 1 -- 172.21.15.119:0/3485614737 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f00600095a0 con 0x564ef4bc8590 2024-03-28T00:25:41.562 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.552+0000 7f005aa00640 1 -- 172.21.15.119:0/3485614737 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 26) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f0060049050 con 0x564ef4bc8590 2024-03-28T00:25:41.562 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.552+0000 7f007ea3f740 10 client.25057 fetch_fsmap finished waiting for FSMap version 26 2024-03-28T00:25:41.562 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.552+0000 7f007ea3f740 10 client.25057 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-28T00:25:41.563 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.552+0000 7f007ea3f740 20 client.25057 populate_metadata read hostname 'smithi119' 2024-03-28T00:25:41.563 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.552+0000 7f007ea3f740 1 --2- 172.21.15.119:0/3485614737 >> [v2:172.21.15.132:6838/3305822525,v1:172.21.15.132:6839/3305822525] conn(0x564ef4c46c40 0x564ef4c42a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:41.563 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.552+0000 7f007ea3f740 4 client.25057 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-28T00:25:41.563 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.552+0000 7f007ea3f740 1 -- 172.21.15.119:0/3485614737 --> [v2:172.21.15.132:6838/3305822525,v1:172.21.15.132:6839/3305822525] -- command(tid 0: {"prefix": "status"}) v1 -- 0x564ef4b3a1d0 con 0x564ef4c46c40 2024-03-28T00:25:41.563 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.552+0000 7f0079400640 1 --2- 172.21.15.119:0/3485614737 >> [v2:172.21.15.132:6838/3305822525,v1:172.21.15.132:6839/3305822525] conn(0x564ef4c46c40 0x564ef4c42a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:41.563 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.553+0000 7f0079400640 1 --2- 172.21.15.119:0/3485614737 >> [v2:172.21.15.132:6838/3305822525,v1:172.21.15.132:6839/3305822525] conn(0x564ef4c46c40 0x564ef4c42a30 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b5742910e49a593b server_cookie=72da9d7c11c9b7e6 in_seq=0 out_seq=0 2024-03-28T00:25:41.563 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.553+0000 7f005aa00640 10 client.25057 ms_handle_connect on v2:172.21.15.132:6838/3305822525 2024-03-28T00:25:41.563 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.553+0000 7f005aa00640 1 -- 172.21.15.119:0/3485614737 <== mds.0 v2:172.21.15.132:6838/3305822525 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x564ef4b3a1d0 con 0x564ef4c46c40 2024-03-28T00:25:41.563 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.553+0000 7f005aa00640 10 client.25057 handle_command_reply: tid=0 2024-03-28T00:25:41.563 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.553+0000 7f007ea3f740 1 client.25057 shutdown 2024-03-28T00:25:41.563 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.553+0000 7f007ea3f740 1 -- 172.21.15.119:0/3485614737 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564ef4bc8590 msgr2=0x564ef4bd5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:41.563 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.553+0000 7f007ea3f740 1 --2- 172.21.15.119:0/3485614737 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564ef4bc8590 0x564ef4bd5990 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f0060011cb0 tx=0x7f0060012870 comp rx=0 tx=0).stop 2024-03-28T00:25:41.563 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.553+0000 7f007ea3f740 1 -- 172.21.15.119:0/3485614737 shutdown_connections 2024-03-28T00:25:41.563 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.553+0000 7f007ea3f740 1 --2- 172.21.15.119:0/3485614737 >> [v2:172.21.15.132:6838/3305822525,v1:172.21.15.132:6839/3305822525] conn(0x564ef4c46c40 0x564ef4c42a30 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:41.563 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.553+0000 7f007ea3f740 1 --2- 172.21.15.119:0/3485614737 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564ef4bcd700 0x564ef4bd8910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:41.564 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.553+0000 7f007ea3f740 1 --2- 172.21.15.119:0/3485614737 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564ef4bc8590 0x564ef4bd5990 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:41.564 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.553+0000 7f005aa00640 0 client.25057 ms_handle_reset on v2:172.21.15.132:6838/3305822525 2024-03-28T00:25:41.564 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.553+0000 7f007ea3f740 1 --2- 172.21.15.119:0/3485614737 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564ef4bc7ba0 0x564ef4bd5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:41.564 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.553+0000 7f007ea3f740 1 -- 172.21.15.119:0/3485614737 >> 172.21.15.119:0/3485614737 conn(0x564ef4bc3110 msgr2=0x564ef4bc34f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:25:41.564 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.553+0000 7f007ea3f740 1 -- 172.21.15.119:0/3485614737 shutdown_connections 2024-03-28T00:25:41.564 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.553+0000 7f007ea3f740 1 -- 172.21.15.119:0/3485614737 wait complete. 2024-03-28T00:25:41.564 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.553+0000 7f007ea3f740 20 client.25057 trim_cache size 0 max 16384 2024-03-28T00:25:41.564 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-03-28T00:25:41.564 INFO:teuthology.orchestra.run.smithi119.stdout: "cluster_fsid": "10f73b92-ec98-11ee-95d0-87774f69a715", 2024-03-28T00:25:41.564 INFO:teuthology.orchestra.run.smithi119.stdout: "whoami": 0, 2024-03-28T00:25:41.564 INFO:teuthology.orchestra.run.smithi119.stdout: "id": 24581, 2024-03-28T00:25:41.564 INFO:teuthology.orchestra.run.smithi119.stdout: "want_state": "up:active", 2024-03-28T00:25:41.565 INFO:teuthology.orchestra.run.smithi119.stdout: "state": "up:active", 2024-03-28T00:25:41.565 INFO:teuthology.orchestra.run.smithi119.stdout: "fs_name": "cephfs", 2024-03-28T00:25:41.565 INFO:teuthology.orchestra.run.smithi119.stdout: "rank_uptime": 104.479551781, 2024-03-28T00:25:41.565 INFO:teuthology.orchestra.run.smithi119.stdout: "mdsmap_epoch": 26, 2024-03-28T00:25:41.565 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_epoch": 87, 2024-03-28T00:25:41.565 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_epoch_barrier": 85, 2024-03-28T00:25:41.565 INFO:teuthology.orchestra.run.smithi119.stdout: "uptime": 111.58658122200001 2024-03-28T00:25:41.565 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-03-28T00:25:41.566 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.554+0000 7f007be00640 1 -- 172.21.15.119:0/1310260387 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f005c0a2800 msgr2=0x7f005c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:41.566 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.554+0000 7f007be00640 1 --2- 172.21.15.119:0/1310260387 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f005c0a2800 0x7f005c0a4cc0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x564ef4bc8df0 tx=0x7f006c009600 comp rx=0 tx=0).stop 2024-03-28T00:25:41.566 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.554+0000 7f007be00640 1 -- 172.21.15.119:0/1310260387 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f00741038e0 msgr2=0x7f0074199c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:41.566 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.554+0000 7f007be00640 1 --2- 172.21.15.119:0/1310260387 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f00741038e0 0x7f0074199c70 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f0068000c00 tx=0x7f006802c6c0 comp rx=0 tx=0).stop 2024-03-28T00:25:41.566 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.554+0000 7f007be00640 1 -- 172.21.15.119:0/1310260387 shutdown_connections 2024-03-28T00:25:41.566 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.554+0000 7f007be00640 1 --2- 172.21.15.119:0/1310260387 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f005c0a2800 0x7f005c0a4cc0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:41.566 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.554+0000 7f007be00640 1 --2- 172.21.15.119:0/1310260387 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0074105490 0x7f00741a11e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:41.566 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.554+0000 7f007be00640 1 --2- 172.21.15.119:0/1310260387 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0074104ad0 0x7f007419a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:41.566 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.554+0000 7f007be00640 1 --2- 172.21.15.119:0/1310260387 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f00741038e0 0x7f0074199c70 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:41.566 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.554+0000 7f007be00640 1 -- 172.21.15.119:0/1310260387 >> 172.21.15.119:0/1310260387 conn(0x7f00740fee50 msgr2=0x7f0074107ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:25:41.566 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.554+0000 7f007be00640 1 -- 172.21.15.119:0/1310260387 shutdown_connections 2024-03-28T00:25:41.566 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:41.554+0000 7f007be00640 1 -- 172.21.15.119:0/1310260387 wait complete. 2024-03-28T00:25:41.578 INFO:tasks.cephadm:Setup complete, yielding 2024-03-28T00:25:41.578 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-28T00:25:41.590 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi119.front.sepia.ceph.com 2024-03-28T00:25:41.593 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 10f73b92-ec98-11ee-95d0-87774f69a715 -- bash -c 'ceph orch status' 2024-03-28T00:25:42.031 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/mon.a/config 2024-03-28T00:25:43.090 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:25:42 smithi119 ceph-mon[31518]: pgmap v366: 97 pgs: 97 active+clean; 648 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 767 B/s wr, 9 op/s 2024-03-28T00:25:43.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:25:42 smithi165 ceph-mon[36302]: pgmap v366: 97 pgs: 97 active+clean; 648 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 767 B/s wr, 9 op/s 2024-03-28T00:25:43.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:25:42 smithi132 ceph-mon[35941]: pgmap v366: 97 pgs: 97 active+clean; 648 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 767 B/s wr, 9 op/s 2024-03-28T00:25:44.356 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:44.350+0000 7f7a8b400700 1 -- 172.21.15.119:0/1556125384 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7a84100a40 msgr2=0x7f7a84100e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:44.356 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:44.350+0000 7f7a8b400700 1 --2- 172.21.15.119:0/1556125384 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7a84100a40 0x7f7a84100e60 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f7a78009990 tx=0x7f7a78009ca0 comp rx=0 tx=0).stop 2024-03-28T00:25:44.357 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:44.350+0000 7f7a8b400700 1 -- 172.21.15.119:0/1556125384 shutdown_connections 2024-03-28T00:25:44.357 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:44.350+0000 7f7a8b400700 1 --2- 172.21.15.119:0/1556125384 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7a84101d60 0x7f7a841021e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:44.357 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:44.350+0000 7f7a8b400700 1 --2- 172.21.15.119:0/1556125384 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7a84100a40 0x7f7a84100e60 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:44.357 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:44.350+0000 7f7a8b400700 1 --2- 172.21.15.119:0/1556125384 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7a841013a0 0x7f7a84101820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:44.357 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:44.350+0000 7f7a8b400700 1 -- 172.21.15.119:0/1556125384 >> 172.21.15.119:0/1556125384 conn(0x7f7a840fc660 msgr2=0x7f7a840feaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:25:44.357 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:44.350+0000 7f7a8b400700 1 -- 172.21.15.119:0/1556125384 shutdown_connections 2024-03-28T00:25:44.357 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:44.351+0000 7f7a8b400700 1 -- 172.21.15.119:0/1556125384 wait complete. 2024-03-28T00:25:44.358 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:44.352+0000 7f7a8b400700 1 Processor -- start 2024-03-28T00:25:44.358 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:44.352+0000 7f7a8b400700 1 -- start start 2024-03-28T00:25:44.358 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:44.352+0000 7f7a8b400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7a84100a40 0x7f7a8419a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:44.358 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:44.353+0000 7f7a8b400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7a841013a0 0x7f7a8419aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:44.359 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:44.353+0000 7f7a8b400700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7a84101d60 0x7f7a841a1c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:44.359 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:44.353+0000 7f7a8b400700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a8410ffd0 con 0x7f7a841013a0 2024-03-28T00:25:44.359 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:44.353+0000 7f7a8b400700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a8410fe50 con 0x7f7a84100a40 2024-03-28T00:25:44.359 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:44.353+0000 7f7a8b400700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a84110150 con 0x7f7a84101d60 2024-03-28T00:25:44.359 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:44.353+0000 7f7a88a00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7a84100a40 0x7f7a8419a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:44.359 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:44.353+0000 7f7a89400700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7a84101d60 0x7f7a841a1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:44.359 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:44.353+0000 7f7a88a00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7a84100a40 0x7f7a8419a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.119:49772/0 (socket says 172.21.15.119:49772) 2024-03-28T00:25:44.359 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:44.353+0000 7f7a88a00700 1 -- 172.21.15.119:0/4099809214 learned_addr learned my addr 172.21.15.119:0/4099809214 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:25:44.359 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:44.353+0000 7f7a89400700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7a84101d60 0x7f7a841a1c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.119:42996/0 (socket says 172.21.15.119:42996) 2024-03-28T00:25:44.359 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:44.353+0000 7f7a83e00700 1 --2- 172.21.15.119:0/4099809214 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7a841013a0 0x7f7a8419aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:44.359 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:44.353+0000 7f7a88a00700 1 -- 172.21.15.119:0/4099809214 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7a84101d60 msgr2=0x7f7a841a1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:44.359 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:44.353+0000 7f7a88a00700 1 --2- 172.21.15.119:0/4099809214 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7a84101d60 0x7f7a841a1c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:44.359 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:44.353+0000 7f7a88a00700 1 -- 172.21.15.119:0/4099809214 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7a841013a0 msgr2=0x7f7a8419aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:44.359 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:44.353+0000 7f7a88a00700 1 --2- 172.21.15.119:0/4099809214 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7a841013a0 0x7f7a8419aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:44.360 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:44.353+0000 7f7a88a00700 1 -- 172.21.15.119:0/4099809214 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a841a22c0 con 0x7f7a84100a40 2024-03-28T00:25:44.360 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:44.353+0000 7f7a89400700 1 --2- 172.21.15.119:0/4099809214 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7a84101d60 0x7f7a841a1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T00:25:44.360 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:44.354+0000 7f7a88a00700 1 --2- 172.21.15.119:0/4099809214 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7a84100a40 0x7f7a8419a660 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f7a78005910 tx=0x7f7a7800bbb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:25:44.360 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:44.354+0000 7f7a81600700 1 -- 172.21.15.119:0/4099809214 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a78026070 con 0x7f7a84100a40 2024-03-28T00:25:44.360 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:44.354+0000 7f7a81600700 1 -- 172.21.15.119:0/4099809214 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7a78003d70 con 0x7f7a84100a40 2024-03-28T00:25:44.360 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:44.354+0000 7f7a8b400700 1 -- 172.21.15.119:0/4099809214 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a841a2550 con 0x7f7a84100a40 2024-03-28T00:25:44.360 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:44.354+0000 7f7a81600700 1 -- 172.21.15.119:0/4099809214 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a78021410 con 0x7f7a84100a40 2024-03-28T00:25:44.361 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:44.354+0000 7f7a8b400700 1 -- 172.21.15.119:0/4099809214 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a841a2a00 con 0x7f7a84100a40 2024-03-28T00:25:44.361 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:44.356+0000 7f7a8b400700 1 -- 172.21.15.119:0/4099809214 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a84068e50 con 0x7f7a84100a40 2024-03-28T00:25:44.362 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:44.357+0000 7f7a81600700 1 -- 172.21.15.119:0/4099809214 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 18) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f7a78029070 con 0x7f7a84100a40 2024-03-28T00:25:44.363 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:44.358+0000 7f7a81600700 1 --2- 172.21.15.119:0/4099809214 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f7a6c0a2910 0x7f7a6c0a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:44.364 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:44.358+0000 7f7a81600700 1 -- 172.21.15.119:0/4099809214 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f7a780cf770 con 0x7f7a84100a40 2024-03-28T00:25:44.364 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:44.358+0000 7f7a83e00700 1 --2- 172.21.15.119:0/4099809214 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f7a6c0a2910 0x7f7a6c0a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:44.364 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:44.359+0000 7f7a83e00700 1 --2- 172.21.15.119:0/4099809214 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f7a6c0a2910 0x7f7a6c0a4dd0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f7a70009c60 tx=0x7f7a70009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:25:44.367 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:44.362+0000 7f7a81600700 1 -- 172.21.15.119:0/4099809214 <== mon.2 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f7a7805d740 con 0x7f7a84100a40 2024-03-28T00:25:44.613 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:44.608+0000 7f7a8b400700 1 -- 172.21.15.119:0/4099809214 --> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7a840625d0 con 0x7f7a6c0a2910 2024-03-28T00:25:44.616 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:44.611+0000 7f7a81600700 1 -- 172.21.15.119:0/4099809214 <== mgr.14152 v2:172.21.15.119:6800/1401937096 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f7a840625d0 con 0x7f7a6c0a2910 2024-03-28T00:25:44.616 INFO:teuthology.orchestra.run.smithi119.stdout:Backend: cephadm 2024-03-28T00:25:44.616 INFO:teuthology.orchestra.run.smithi119.stdout:Available: Yes 2024-03-28T00:25:44.616 INFO:teuthology.orchestra.run.smithi119.stdout:Paused: No 2024-03-28T00:25:44.619 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:44.613+0000 7f7a8b400700 1 -- 172.21.15.119:0/4099809214 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f7a6c0a2910 msgr2=0x7f7a6c0a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:44.619 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:44.613+0000 7f7a8b400700 1 --2- 172.21.15.119:0/4099809214 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f7a6c0a2910 0x7f7a6c0a4dd0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f7a70009c60 tx=0x7f7a70009260 comp rx=0 tx=0).stop 2024-03-28T00:25:44.619 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:44.613+0000 7f7a8b400700 1 -- 172.21.15.119:0/4099809214 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7a84100a40 msgr2=0x7f7a8419a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:44.619 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:44.613+0000 7f7a8b400700 1 --2- 172.21.15.119:0/4099809214 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7a84100a40 0x7f7a8419a660 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f7a78005910 tx=0x7f7a7800bbb0 comp rx=0 tx=0).stop 2024-03-28T00:25:44.619 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:44.614+0000 7f7a8b400700 1 -- 172.21.15.119:0/4099809214 shutdown_connections 2024-03-28T00:25:44.619 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:44.614+0000 7f7a8b400700 1 --2- 172.21.15.119:0/4099809214 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f7a6c0a2910 0x7f7a6c0a4dd0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:44.619 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:44.614+0000 7f7a8b400700 1 --2- 172.21.15.119:0/4099809214 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7a84101d60 0x7f7a841a1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:44.619 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:44.614+0000 7f7a8b400700 1 --2- 172.21.15.119:0/4099809214 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7a84100a40 0x7f7a8419a660 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:44.620 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:44.614+0000 7f7a8b400700 1 --2- 172.21.15.119:0/4099809214 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7a841013a0 0x7f7a8419aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:44.620 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:44.614+0000 7f7a8b400700 1 -- 172.21.15.119:0/4099809214 >> 172.21.15.119:0/4099809214 conn(0x7f7a840fc660 msgr2=0x7f7a840fe160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:25:44.620 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:44.614+0000 7f7a8b400700 1 -- 172.21.15.119:0/4099809214 shutdown_connections 2024-03-28T00:25:44.620 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:44.614+0000 7f7a8b400700 1 -- 172.21.15.119:0/4099809214 wait complete. 2024-03-28T00:25:44.971 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:25:44 smithi119 ceph-mon[31518]: pgmap v367: 97 pgs: 97 active+clean; 648 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 767 B/s wr, 12 op/s 2024-03-28T00:25:45.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:25:44 smithi132 ceph-mon[35941]: pgmap v367: 97 pgs: 97 active+clean; 648 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 767 B/s wr, 12 op/s 2024-03-28T00:25:45.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:25:44 smithi165 ceph-mon[36302]: pgmap v367: 97 pgs: 97 active+clean; 648 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 767 B/s wr, 12 op/s 2024-03-28T00:25:45.491 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 10f73b92-ec98-11ee-95d0-87774f69a715 -- bash -c 'ceph orch ps' 2024-03-28T00:25:45.928 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/mon.a/config 2024-03-28T00:25:46.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:25:45 smithi119 ceph-mon[31518]: from='client.25052 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T00:25:46.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:25:45 smithi132 ceph-mon[35941]: from='client.25052 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T00:25:46.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:25:45 smithi165 ceph-mon[36302]: from='client.25052 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T00:25:47.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:25:46 smithi119 ceph-mon[31518]: pgmap v368: 97 pgs: 97 active+clean; 648 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 767 B/s wr, 9 op/s 2024-03-28T00:25:47.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:25:46 smithi132 ceph-mon[35941]: pgmap v368: 97 pgs: 97 active+clean; 648 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 767 B/s wr, 9 op/s 2024-03-28T00:25:47.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:25:46 smithi165 ceph-mon[36302]: pgmap v368: 97 pgs: 97 active+clean; 648 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 767 B/s wr, 9 op/s 2024-03-28T00:25:48.093 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:48.087+0000 7f3532600700 1 -- 172.21.15.119:0/3143995863 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f352c100970 msgr2=0x7f352c100d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:48.093 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:48.087+0000 7f3532600700 1 --2- 172.21.15.119:0/3143995863 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f352c100970 0x7f352c100d90 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f35200098f0 tx=0x7f3520009c00 comp rx=0 tx=0).stop 2024-03-28T00:25:48.093 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:48.087+0000 7f3532600700 1 -- 172.21.15.119:0/3143995863 shutdown_connections 2024-03-28T00:25:48.094 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:48.087+0000 7f3532600700 1 --2- 172.21.15.119:0/3143995863 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f352c101c90 0x7f352c102110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:48.094 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:48.087+0000 7f3532600700 1 --2- 172.21.15.119:0/3143995863 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f352c100970 0x7f352c100d90 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:48.094 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:48.087+0000 7f3532600700 1 --2- 172.21.15.119:0/3143995863 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f352c1012d0 0x7f352c101750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:48.094 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:48.087+0000 7f3532600700 1 -- 172.21.15.119:0/3143995863 >> 172.21.15.119:0/3143995863 conn(0x7f352c0fc570 msgr2=0x7f352c0fe9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:25:48.094 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:48.087+0000 7f3532600700 1 -- 172.21.15.119:0/3143995863 shutdown_connections 2024-03-28T00:25:48.094 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:48.087+0000 7f3532600700 1 -- 172.21.15.119:0/3143995863 wait complete. 2024-03-28T00:25:48.096 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:48.089+0000 7f3532600700 1 Processor -- start 2024-03-28T00:25:48.096 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:48.089+0000 7f3532600700 1 -- start start 2024-03-28T00:25:48.096 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:48.089+0000 7f3532600700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f352c100970 0x7f352c19a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:48.097 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:48.089+0000 7f3532600700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f352c1012d0 0x7f352c19aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:48.097 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:48.089+0000 7f3532600700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f352c101c90 0x7f352c1a1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:48.097 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:48.089+0000 7f3532600700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f352c10fea0 con 0x7f352c101c90 2024-03-28T00:25:48.097 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:48.089+0000 7f3532600700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f352c10fd20 con 0x7f352c1012d0 2024-03-28T00:25:48.097 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:48.089+0000 7f3532600700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f352c110020 con 0x7f352c100970 2024-03-28T00:25:48.097 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:48.089+0000 7f3531200700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f352c100970 0x7f352c19a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:48.097 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:48.089+0000 7f352be00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f352c1012d0 0x7f352c19aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:48.097 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:48.089+0000 7f3531200700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f352c100970 0x7f352c19a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.119:43012/0 (socket says 172.21.15.119:43012) 2024-03-28T00:25:48.097 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:48.089+0000 7f3531200700 1 -- 172.21.15.119:0/3715808148 learned_addr learned my addr 172.21.15.119:0/3715808148 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:25:48.097 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:48.090+0000 7f3531c00700 1 --2- 172.21.15.119:0/3715808148 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f352c101c90 0x7f352c1a1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:48.097 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:48.090+0000 7f3531200700 1 -- 172.21.15.119:0/3715808148 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f352c1012d0 msgr2=0x7f352c19aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:48.097 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:48.090+0000 7f3531200700 1 --2- 172.21.15.119:0/3715808148 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f352c1012d0 0x7f352c19aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:48.098 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:48.090+0000 7f3531200700 1 -- 172.21.15.119:0/3715808148 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f352c101c90 msgr2=0x7f352c1a1aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:48.098 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:48.090+0000 7f3531200700 1 --2- 172.21.15.119:0/3715808148 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f352c101c90 0x7f352c1a1aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:48.098 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:48.090+0000 7f3531200700 1 -- 172.21.15.119:0/3715808148 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f352c1a2140 con 0x7f352c100970 2024-03-28T00:25:48.098 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:48.090+0000 7f3531c00700 1 --2- 172.21.15.119:0/3715808148 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f352c101c90 0x7f352c1a1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T00:25:48.098 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:48.090+0000 7f352be00700 1 --2- 172.21.15.119:0/3715808148 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f352c1012d0 0x7f352c19aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T00:25:48.098 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:48.090+0000 7f3531200700 1 --2- 172.21.15.119:0/3715808148 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f352c100970 0x7f352c19a4e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f352000fca0 tx=0x7f3520005250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:25:48.098 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:48.090+0000 7f3529600700 1 -- 172.21.15.119:0/3715808148 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3520025070 con 0x7f352c100970 2024-03-28T00:25:48.098 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:48.090+0000 7f3529600700 1 -- 172.21.15.119:0/3715808148 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f352000ba40 con 0x7f352c100970 2024-03-28T00:25:48.098 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:48.090+0000 7f3532600700 1 -- 172.21.15.119:0/3715808148 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f352c1a23d0 con 0x7f352c100970 2024-03-28T00:25:48.098 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:48.091+0000 7f3529600700 1 -- 172.21.15.119:0/3715808148 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3520003df0 con 0x7f352c100970 2024-03-28T00:25:48.098 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:48.091+0000 7f3532600700 1 -- 172.21.15.119:0/3715808148 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f352c1a2880 con 0x7f352c100970 2024-03-28T00:25:48.099 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:48.092+0000 7f3532600700 1 -- 172.21.15.119:0/3715808148 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f352c068e50 con 0x7f352c100970 2024-03-28T00:25:48.099 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:48.094+0000 7f3529600700 1 -- 172.21.15.119:0/3715808148 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 18) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f3520028070 con 0x7f352c100970 2024-03-28T00:25:48.100 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:48.095+0000 7f3529600700 1 --2- 172.21.15.119:0/3715808148 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f35140a2910 0x7f35140a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:48.100 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:48.095+0000 7f3529600700 1 -- 172.21.15.119:0/3715808148 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f35200cf290 con 0x7f352c100970 2024-03-28T00:25:48.100 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:48.095+0000 7f352be00700 1 --2- 172.21.15.119:0/3715808148 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f35140a2910 0x7f35140a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:48.101 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:48.096+0000 7f352be00700 1 --2- 172.21.15.119:0/3715808148 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f35140a2910 0x7f35140a4dd0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f3518011490 tx=0x7f3518009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:25:48.104 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:48.099+0000 7f3529600700 1 -- 172.21.15.119:0/3715808148 <== mon.1 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f3520013030 con 0x7f352c100970 2024-03-28T00:25:48.335 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:48.330+0000 7f3532600700 1 -- 172.21.15.119:0/3715808148 --> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f352c0625d0 con 0x7f35140a2910 2024-03-28T00:25:48.351 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:48.346+0000 7f3529600700 1 -- 172.21.15.119:0/3715808148 <== mgr.14152 v2:172.21.15.119:6800/1401937096 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+4185 (secure 0 0 0) 0x7f352c0625d0 con 0x7f35140a2910 2024-03-28T00:25:48.352 INFO:teuthology.orchestra.run.smithi119.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-28T00:25:48.352 INFO:teuthology.orchestra.run.smithi119.stdout:mds.a smithi119 running (2m) 23s ago 2m 15.5M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 a8cf73f63bdb 2024-03-28T00:25:48.352 INFO:teuthology.orchestra.run.smithi119.stdout:mds.b smithi132 running (118s) 50s ago 118s 25.9M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 fe3fee15a0f9 2024-03-28T00:25:48.352 INFO:teuthology.orchestra.run.smithi119.stdout:mds.c smithi165 running (2m) 50s ago 2m 19.2M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 1af8232810b8 2024-03-28T00:25:48.352 INFO:teuthology.orchestra.run.smithi119.stdout:mds.d smithi119 running (2m) 23s ago 2m 12.3M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 b6a9b8c07b0d 2024-03-28T00:25:48.352 INFO:teuthology.orchestra.run.smithi119.stdout:mds.e smithi132 running (2m) 50s ago 2m 20.0M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 6ff65ee54ee3 2024-03-28T00:25:48.352 INFO:teuthology.orchestra.run.smithi119.stdout:mds.f smithi165 running (2m) 50s ago 2m 20.2M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 935245452bf8 2024-03-28T00:25:48.352 INFO:teuthology.orchestra.run.smithi119.stdout:mds.g smithi119 running (2m) 23s ago 2m 12.2M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 68ec75c44d59 2024-03-28T00:25:48.353 INFO:teuthology.orchestra.run.smithi119.stdout:mds.h smithi132 running (2m) 50s ago 2m 20.7M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 2b9188b0f91b 2024-03-28T00:25:48.353 INFO:teuthology.orchestra.run.smithi119.stdout:mds.i smithi165 running (2m) 50s ago 2m 19.7M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 19dc1822c951 2024-03-28T00:25:48.353 INFO:teuthology.orchestra.run.smithi119.stdout:mds.j smithi119 running (2m) 23s ago 2m 15.6M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 91c6d8f59d03 2024-03-28T00:25:48.353 INFO:teuthology.orchestra.run.smithi119.stdout:mds.k smithi132 running (2m) 50s ago 2m 16.5M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 90f8d0de7d8a 2024-03-28T00:25:48.353 INFO:teuthology.orchestra.run.smithi119.stdout:mds.l smithi165 running (2m) 50s ago 2m 16.7M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 fa62f3785e42 2024-03-28T00:25:48.353 INFO:teuthology.orchestra.run.smithi119.stdout:mgr.x smithi119 *:9283,8765 running (10m) 23s ago 10m 514M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 f628950cf3c0 2024-03-28T00:25:48.353 INFO:teuthology.orchestra.run.smithi119.stdout:mgr.y smithi132 *:8443,8765 running (8m) 50s ago 8m 453M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 eba27e1c6ada 2024-03-28T00:25:48.353 INFO:teuthology.orchestra.run.smithi119.stdout:mgr.z smithi165 *:8443,8765 running (8m) 50s ago 8m 453M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 3ec565f6ab2b 2024-03-28T00:25:48.353 INFO:teuthology.orchestra.run.smithi119.stdout:mon.a smithi119 running (10m) 23s ago 10m 48.5M 2048M 19.0.0-2145-g3be9a5c9 c664afe56bb0 746fea8df4c2 2024-03-28T00:25:48.353 INFO:teuthology.orchestra.run.smithi119.stdout:mon.b smithi132 running (8m) 50s ago 8m 40.5M 2048M 19.0.0-2145-g3be9a5c9 c664afe56bb0 5652a2d5e3de 2024-03-28T00:25:48.353 INFO:teuthology.orchestra.run.smithi119.stdout:mon.c smithi165 running (8m) 50s ago 8m 45.0M 2048M 19.0.0-2145-g3be9a5c9 c664afe56bb0 e52d038dbab0 2024-03-28T00:25:48.353 INFO:teuthology.orchestra.run.smithi119.stdout:osd.0 smithi119 running (7m) 23s ago 7m 44.2M 4272M 19.0.0-2145-g3be9a5c9 c664afe56bb0 23820adadea1 2024-03-28T00:25:48.353 INFO:teuthology.orchestra.run.smithi119.stdout:osd.1 smithi132 running (7m) 50s ago 7m 57.8M 4270M 19.0.0-2145-g3be9a5c9 c664afe56bb0 b006dc96dd4c 2024-03-28T00:25:48.353 INFO:teuthology.orchestra.run.smithi119.stdout:osd.2 smithi165 running (7m) 50s ago 7m 56.8M 4270M 19.0.0-2145-g3be9a5c9 c664afe56bb0 e706fd3899de 2024-03-28T00:25:48.354 INFO:teuthology.orchestra.run.smithi119.stdout:osd.3 smithi119 running (6m) 23s ago 6m 42.2M 4272M 19.0.0-2145-g3be9a5c9 c664afe56bb0 9bcbcf8e146d 2024-03-28T00:25:48.354 INFO:teuthology.orchestra.run.smithi119.stdout:osd.4 smithi132 running (6m) 50s ago 6m 53.0M 4270M 19.0.0-2145-g3be9a5c9 c664afe56bb0 a3ea4b5b9d08 2024-03-28T00:25:48.354 INFO:teuthology.orchestra.run.smithi119.stdout:osd.5 smithi165 running (5m) 50s ago 5m 46.3M 4270M 19.0.0-2145-g3be9a5c9 c664afe56bb0 9ef3682a3653 2024-03-28T00:25:48.354 INFO:teuthology.orchestra.run.smithi119.stdout:osd.6 smithi119 running (5m) 23s ago 5m 52.2M 4272M 19.0.0-2145-g3be9a5c9 c664afe56bb0 ae3bdf70ec38 2024-03-28T00:25:48.354 INFO:teuthology.orchestra.run.smithi119.stdout:osd.7 smithi132 running (4m) 50s ago 4m 57.0M 4270M 19.0.0-2145-g3be9a5c9 c664afe56bb0 0e1b3834426e 2024-03-28T00:25:48.354 INFO:teuthology.orchestra.run.smithi119.stdout:osd.8 smithi165 running (4m) 50s ago 4m 53.4M 4270M 19.0.0-2145-g3be9a5c9 c664afe56bb0 345cf3a76e05 2024-03-28T00:25:48.354 INFO:teuthology.orchestra.run.smithi119.stdout:osd.9 smithi119 running (3m) 23s ago 3m 51.7M 4272M 19.0.0-2145-g3be9a5c9 c664afe56bb0 f3cd3d247249 2024-03-28T00:25:48.354 INFO:teuthology.orchestra.run.smithi119.stdout:osd.10 smithi132 running (3m) 50s ago 3m 41.8M 4270M 19.0.0-2145-g3be9a5c9 c664afe56bb0 c7cc65528546 2024-03-28T00:25:48.354 INFO:teuthology.orchestra.run.smithi119.stdout:osd.11 smithi165 running (2m) 50s ago 2m 51.0M 4270M 19.0.0-2145-g3be9a5c9 c664afe56bb0 ba1686891db3 2024-03-28T00:25:48.355 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:48.349+0000 7f3532600700 1 -- 172.21.15.119:0/3715808148 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f35140a2910 msgr2=0x7f35140a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:48.355 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:48.349+0000 7f3532600700 1 --2- 172.21.15.119:0/3715808148 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f35140a2910 0x7f35140a4dd0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f3518011490 tx=0x7f3518009260 comp rx=0 tx=0).stop 2024-03-28T00:25:48.355 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:48.349+0000 7f3532600700 1 -- 172.21.15.119:0/3715808148 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f352c100970 msgr2=0x7f352c19a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:48.355 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:48.349+0000 7f3532600700 1 --2- 172.21.15.119:0/3715808148 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f352c100970 0x7f352c19a4e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f352000fca0 tx=0x7f3520005250 comp rx=0 tx=0).stop 2024-03-28T00:25:48.355 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:48.349+0000 7f3532600700 1 -- 172.21.15.119:0/3715808148 shutdown_connections 2024-03-28T00:25:48.355 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:48.349+0000 7f3532600700 1 --2- 172.21.15.119:0/3715808148 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f35140a2910 0x7f35140a4dd0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:48.355 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:48.349+0000 7f3532600700 1 --2- 172.21.15.119:0/3715808148 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f352c101c90 0x7f352c1a1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:48.355 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:48.349+0000 7f3532600700 1 --2- 172.21.15.119:0/3715808148 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f352c100970 0x7f352c19a4e0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:48.356 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:48.349+0000 7f3532600700 1 --2- 172.21.15.119:0/3715808148 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f352c1012d0 0x7f352c19aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:48.356 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:48.350+0000 7f3532600700 1 -- 172.21.15.119:0/3715808148 >> 172.21.15.119:0/3715808148 conn(0x7f352c0fc570 msgr2=0x7f352c0fe070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:25:48.356 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:48.350+0000 7f3532600700 1 -- 172.21.15.119:0/3715808148 shutdown_connections 2024-03-28T00:25:48.356 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:48.350+0000 7f3532600700 1 -- 172.21.15.119:0/3715808148 wait complete. 2024-03-28T00:25:49.137 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:25:48 smithi119 ceph-mon[31518]: pgmap v369: 97 pgs: 97 active+clean; 648 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 682 B/s wr, 10 op/s 2024-03-28T00:25:49.183 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:25:48 smithi165 ceph-mon[36302]: pgmap v369: 97 pgs: 97 active+clean; 648 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 682 B/s wr, 10 op/s 2024-03-28T00:25:49.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:25:48 smithi132 ceph-mon[35941]: pgmap v369: 97 pgs: 97 active+clean; 648 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 682 B/s wr, 10 op/s 2024-03-28T00:25:49.213 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 10f73b92-ec98-11ee-95d0-87774f69a715 -- bash -c 'ceph orch ls' 2024-03-28T00:25:49.665 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/mon.a/config 2024-03-28T00:25:50.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:25:49 smithi119 ceph-mon[31518]: from='client.25069 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T00:25:50.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:25:49 smithi132 ceph-mon[35941]: from='client.25069 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T00:25:50.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:25:49 smithi165 ceph-mon[36302]: from='client.25069 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T00:25:51.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:25:50 smithi132 ceph-mon[35941]: pgmap v370: 97 pgs: 97 active+clean; 648 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 597 B/s wr, 12 op/s 2024-03-28T00:25:51.272 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:25:50 smithi119 ceph-mon[31518]: pgmap v370: 97 pgs: 97 active+clean; 648 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 597 B/s wr, 12 op/s 2024-03-28T00:25:51.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:25:50 smithi165 ceph-mon[36302]: pgmap v370: 97 pgs: 97 active+clean; 648 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 597 B/s wr, 12 op/s 2024-03-28T00:25:51.759 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:51.752+0000 7f5723e00700 1 -- 172.21.15.119:0/3402877250 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f571c105580 msgr2=0x7f571c107970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:51.759 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:51.752+0000 7f5723e00700 1 --2- 172.21.15.119:0/3402877250 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f571c105580 0x7f571c107970 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f5708009990 tx=0x7f5708009ca0 comp rx=0 tx=0).stop 2024-03-28T00:25:51.759 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:51.752+0000 7f5723e00700 1 -- 172.21.15.119:0/3402877250 shutdown_connections 2024-03-28T00:25:51.759 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:51.752+0000 7f5723e00700 1 --2- 172.21.15.119:0/3402877250 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f571c107eb0 0x7f571c10a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:51.759 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:51.752+0000 7f5723e00700 1 --2- 172.21.15.119:0/3402877250 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f571c102c50 0x7f571c105040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:51.760 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:51.752+0000 7f5723e00700 1 --2- 172.21.15.119:0/3402877250 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f571c105580 0x7f571c107970 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:51.760 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:51.752+0000 7f5723e00700 1 -- 172.21.15.119:0/3402877250 >> 172.21.15.119:0/3402877250 conn(0x7f571c0fc660 msgr2=0x7f571c0feaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:25:51.760 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:51.752+0000 7f5723e00700 1 -- 172.21.15.119:0/3402877250 shutdown_connections 2024-03-28T00:25:51.760 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:51.753+0000 7f5723e00700 1 -- 172.21.15.119:0/3402877250 wait complete. 2024-03-28T00:25:51.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:51.754+0000 7f5723e00700 1 Processor -- start 2024-03-28T00:25:51.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:51.754+0000 7f5723e00700 1 -- start start 2024-03-28T00:25:51.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:51.754+0000 7f5723e00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f571c102c50 0x7f571c198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:51.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:51.754+0000 7f5723e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f571c105580 0x7f571c198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:51.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:51.754+0000 7f5723e00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f571c107eb0 0x7f571c19f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:51.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:51.754+0000 7f5723e00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f571c10ddd0 con 0x7f571c105580 2024-03-28T00:25:51.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:51.754+0000 7f5723e00700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f571c10dc50 con 0x7f571c102c50 2024-03-28T00:25:51.762 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:51.754+0000 7f5723e00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f571c10df50 con 0x7f571c107eb0 2024-03-28T00:25:51.762 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:51.755+0000 7f5721400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f571c102c50 0x7f571c198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:51.762 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:51.755+0000 7f5721400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f571c102c50 0x7f571c198400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.119:49804/0 (socket says 172.21.15.119:49804) 2024-03-28T00:25:51.762 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:51.755+0000 7f5721400700 1 -- 172.21.15.119:0/1260919455 learned_addr learned my addr 172.21.15.119:0/1260919455 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:25:51.762 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:51.755+0000 7f5721e00700 1 --2- 172.21.15.119:0/1260919455 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f571c107eb0 0x7f571c19f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:51.762 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:51.755+0000 7f5720a00700 1 --2- 172.21.15.119:0/1260919455 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f571c105580 0x7f571c198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:51.762 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:51.755+0000 7f5721400700 1 -- 172.21.15.119:0/1260919455 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f571c107eb0 msgr2=0x7f571c19f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:51.762 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:51.755+0000 7f5721400700 1 --2- 172.21.15.119:0/1260919455 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f571c107eb0 0x7f571c19f9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:51.762 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:51.755+0000 7f5721400700 1 -- 172.21.15.119:0/1260919455 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f571c105580 msgr2=0x7f571c198940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:51.762 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:51.755+0000 7f5721400700 1 --2- 172.21.15.119:0/1260919455 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f571c105580 0x7f571c198940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:51.762 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:51.755+0000 7f5721400700 1 -- 172.21.15.119:0/1260919455 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f571c1a0060 con 0x7f571c102c50 2024-03-28T00:25:51.762 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:51.755+0000 7f5720a00700 1 --2- 172.21.15.119:0/1260919455 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f571c105580 0x7f571c198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T00:25:51.763 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:51.755+0000 7f5721400700 1 --2- 172.21.15.119:0/1260919455 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f571c102c50 0x7f571c198400 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f571000eb30 tx=0x7f571000ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:25:51.763 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:51.756+0000 7f571a000700 1 -- 172.21.15.119:0/1260919455 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f571000cc90 con 0x7f571c102c50 2024-03-28T00:25:51.763 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:51.756+0000 7f571a000700 1 -- 172.21.15.119:0/1260919455 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5710004500 con 0x7f571c102c50 2024-03-28T00:25:51.763 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:51.756+0000 7f5723e00700 1 -- 172.21.15.119:0/1260919455 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f571c1a0350 con 0x7f571c102c50 2024-03-28T00:25:51.763 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:51.756+0000 7f571a000700 1 -- 172.21.15.119:0/1260919455 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5710010430 con 0x7f571c102c50 2024-03-28T00:25:51.764 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:51.756+0000 7f5723e00700 1 -- 172.21.15.119:0/1260919455 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f571c1a0890 con 0x7f571c102c50 2024-03-28T00:25:51.764 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:51.757+0000 7f5723e00700 1 -- 172.21.15.119:0/1260919455 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f571c068e50 con 0x7f571c102c50 2024-03-28T00:25:51.765 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:51.759+0000 7f571a000700 1 -- 172.21.15.119:0/1260919455 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 18) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f5710003fa0 con 0x7f571c102c50 2024-03-28T00:25:51.766 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:51.760+0000 7f571a000700 1 --2- 172.21.15.119:0/1260919455 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f57040a2890 0x7f57040a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:51.767 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:51.760+0000 7f571a000700 1 -- 172.21.15.119:0/1260919455 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f57100ced60 con 0x7f571c102c50 2024-03-28T00:25:51.767 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:51.760+0000 7f5720a00700 1 --2- 172.21.15.119:0/1260919455 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f57040a2890 0x7f57040a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:51.767 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:51.761+0000 7f5720a00700 1 --2- 172.21.15.119:0/1260919455 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f57040a2890 0x7f57040a4d50 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f570800b7b0 tx=0x7f570801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:25:51.772 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:51.766+0000 7f571a000700 1 -- 172.21.15.119:0/1260919455 <== mon.2 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f571009be00 con 0x7f571c102c50 2024-03-28T00:25:52.004 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:51.997+0000 7f5723e00700 1 -- 172.21.15.119:0/1260919455 --> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f571c0625d0 con 0x7f57040a2890 2024-03-28T00:25:52.020 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:52.014+0000 7f571a000700 1 -- 172.21.15.119:0/1260919455 <== mgr.14152 v2:172.21.15.119:6800/1401937096 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+980 (secure 0 0 0) 0x7f571c0625d0 con 0x7f57040a2890 2024-03-28T00:25:52.021 INFO:teuthology.orchestra.run.smithi119.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-03-28T00:25:52.021 INFO:teuthology.orchestra.run.smithi119.stdout:mds.all 12/12 53s ago 2m smithi119=a;smithi119=d;smithi119=g;smithi119=j;smithi132=b;smithi132=e;smithi132=h;smithi132=k;smithi165=c;smithi165=f;smithi165=i;smithi165=l;count:12 2024-03-28T00:25:52.021 INFO:teuthology.orchestra.run.smithi119.stdout:mgr 3/3 53s ago 8m smithi119=x;smithi132=y;smithi165=z;count:3 2024-03-28T00:25:52.021 INFO:teuthology.orchestra.run.smithi119.stdout:mon 3/3 53s ago 8m smithi119:172.21.15.119=a;smithi132:172.21.15.132=b;smithi165:172.21.15.165=c;count:3 2024-03-28T00:25:52.021 INFO:teuthology.orchestra.run.smithi119.stdout:osd 12 53s ago - 2024-03-28T00:25:52.023 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:52.017+0000 7f5723e00700 1 -- 172.21.15.119:0/1260919455 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f57040a2890 msgr2=0x7f57040a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:52.024 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:52.017+0000 7f5723e00700 1 --2- 172.21.15.119:0/1260919455 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f57040a2890 0x7f57040a4d50 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f570800b7b0 tx=0x7f570801a040 comp rx=0 tx=0).stop 2024-03-28T00:25:52.024 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:52.017+0000 7f5723e00700 1 -- 172.21.15.119:0/1260919455 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f571c102c50 msgr2=0x7f571c198400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:52.024 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:52.017+0000 7f5723e00700 1 --2- 172.21.15.119:0/1260919455 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f571c102c50 0x7f571c198400 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f571000eb30 tx=0x7f571000ee40 comp rx=0 tx=0).stop 2024-03-28T00:25:52.024 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:52.017+0000 7f5723e00700 1 -- 172.21.15.119:0/1260919455 shutdown_connections 2024-03-28T00:25:52.024 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:52.017+0000 7f5723e00700 1 --2- 172.21.15.119:0/1260919455 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f57040a2890 0x7f57040a4d50 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:52.024 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:52.017+0000 7f5723e00700 1 --2- 172.21.15.119:0/1260919455 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f571c107eb0 0x7f571c19f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:52.024 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:52.017+0000 7f5723e00700 1 --2- 172.21.15.119:0/1260919455 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f571c102c50 0x7f571c198400 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:52.024 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:52.017+0000 7f5723e00700 1 --2- 172.21.15.119:0/1260919455 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f571c105580 0x7f571c198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:52.024 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:52.017+0000 7f5723e00700 1 -- 172.21.15.119:0/1260919455 >> 172.21.15.119:0/1260919455 conn(0x7f571c0fc660 msgr2=0x7f571c1036d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:25:52.024 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:52.017+0000 7f5723e00700 1 -- 172.21.15.119:0/1260919455 shutdown_connections 2024-03-28T00:25:52.024 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:52.017+0000 7f5723e00700 1 -- 172.21.15.119:0/1260919455 wait complete. 2024-03-28T00:25:52.804 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 10f73b92-ec98-11ee-95d0-87774f69a715 -- bash -c 'ceph orch host ls' 2024-03-28T00:25:53.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:25:52 smithi119 ceph-mon[31518]: pgmap v371: 97 pgs: 97 active+clean; 648 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T00:25:53.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:25:52 smithi119 ceph-mon[31518]: from='client.25064 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T00:25:53.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:25:52 smithi132 ceph-mon[35941]: pgmap v371: 97 pgs: 97 active+clean; 648 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T00:25:53.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:25:52 smithi132 ceph-mon[35941]: from='client.25064 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T00:25:53.237 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/mon.a/config 2024-03-28T00:25:53.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:25:52 smithi165 ceph-mon[36302]: pgmap v371: 97 pgs: 97 active+clean; 648 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T00:25:53.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:25:52 smithi165 ceph-mon[36302]: from='client.25064 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T00:25:55.045 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:25:54 smithi119 ceph-mon[31518]: pgmap v372: 97 pgs: 97 active+clean; 648 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-28T00:25:55.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:25:54 smithi132 ceph-mon[35941]: pgmap v372: 97 pgs: 97 active+clean; 648 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-28T00:25:55.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:25:54 smithi165 ceph-mon[36302]: pgmap v372: 97 pgs: 97 active+clean; 648 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-28T00:25:55.622 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:55.615+0000 7fb09aa00700 1 -- 172.21.15.119:0/991205303 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb094105610 msgr2=0x7fb094105a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:55.622 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:55.615+0000 7fb09aa00700 1 --2- 172.21.15.119:0/991205303 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb094105610 0x7fb094105a70 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb080009990 tx=0x7fb080009ca0 comp rx=0 tx=0).stop 2024-03-28T00:25:55.622 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:55.615+0000 7fb09aa00700 1 -- 172.21.15.119:0/991205303 shutdown_connections 2024-03-28T00:25:55.622 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:55.615+0000 7fb09aa00700 1 --2- 172.21.15.119:0/991205303 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb094105fb0 0x7fb09410c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:55.622 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:55.615+0000 7fb09aa00700 1 --2- 172.21.15.119:0/991205303 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb094104410 0x7fb094104830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:55.622 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:55.615+0000 7fb09aa00700 1 --2- 172.21.15.119:0/991205303 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb094105610 0x7fb094105a70 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:55.622 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:55.615+0000 7fb09aa00700 1 -- 172.21.15.119:0/991205303 >> 172.21.15.119:0/991205303 conn(0x7fb0940ff9b0 msgr2=0x7fb094101df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:25:55.622 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:55.615+0000 7fb09aa00700 1 -- 172.21.15.119:0/991205303 shutdown_connections 2024-03-28T00:25:55.622 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:55.615+0000 7fb09aa00700 1 -- 172.21.15.119:0/991205303 wait complete. 2024-03-28T00:25:55.624 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:55.616+0000 7fb09aa00700 1 Processor -- start 2024-03-28T00:25:55.624 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:55.616+0000 7fb09aa00700 1 -- start start 2024-03-28T00:25:55.624 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:55.617+0000 7fb09aa00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb094104410 0x7fb09419a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:55.624 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:55.617+0000 7fb09aa00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb094105610 0x7fb09419aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:55.625 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:55.617+0000 7fb09aa00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb094105fb0 0x7fb0941a1af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:55.625 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:55.617+0000 7fb09aa00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb09410ffe0 con 0x7fb094105610 2024-03-28T00:25:55.625 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:55.617+0000 7fb09aa00700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb09410fe60 con 0x7fb094104410 2024-03-28T00:25:55.625 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:55.617+0000 7fb09aa00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb094110160 con 0x7fb094105fb0 2024-03-28T00:25:55.625 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:55.617+0000 7fb093e00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb094104410 0x7fb09419a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:55.625 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:55.617+0000 7fb093400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb094105610 0x7fb09419aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:55.625 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:55.617+0000 7fb093e00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb094104410 0x7fb09419a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.119:43484/0 (socket says 172.21.15.119:43484) 2024-03-28T00:25:55.625 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:55.617+0000 7fb093e00700 1 -- 172.21.15.119:0/132345213 learned_addr learned my addr 172.21.15.119:0/132345213 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:25:55.625 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:55.617+0000 7fb098a00700 1 --2- 172.21.15.119:0/132345213 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb094105fb0 0x7fb0941a1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:55.625 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:55.617+0000 7fb093e00700 1 -- 172.21.15.119:0/132345213 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb094105fb0 msgr2=0x7fb0941a1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:55.625 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:55.617+0000 7fb093e00700 1 --2- 172.21.15.119:0/132345213 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb094105fb0 0x7fb0941a1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:55.625 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:55.617+0000 7fb093e00700 1 -- 172.21.15.119:0/132345213 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb094105610 msgr2=0x7fb09419aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:55.625 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:55.617+0000 7fb093e00700 1 --2- 172.21.15.119:0/132345213 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb094105610 0x7fb09419aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:55.626 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:55.617+0000 7fb093e00700 1 -- 172.21.15.119:0/132345213 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0941a2190 con 0x7fb094104410 2024-03-28T00:25:55.626 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:55.618+0000 7fb093e00700 1 --2- 172.21.15.119:0/132345213 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb094104410 0x7fb09419a530 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fb08400e970 tx=0x7fb08400ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:25:55.626 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:55.618+0000 7fb090c00700 1 -- 172.21.15.119:0/132345213 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb084004220 con 0x7fb094104410 2024-03-28T00:25:55.626 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:55.618+0000 7fb090c00700 1 -- 172.21.15.119:0/132345213 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb0840043c0 con 0x7fb094104410 2024-03-28T00:25:55.626 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:55.618+0000 7fb090c00700 1 -- 172.21.15.119:0/132345213 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb084003df0 con 0x7fb094104410 2024-03-28T00:25:55.626 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:55.618+0000 7fb09aa00700 1 -- 172.21.15.119:0/132345213 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0941a2480 con 0x7fb094104410 2024-03-28T00:25:55.626 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:55.618+0000 7fb09aa00700 1 -- 172.21.15.119:0/132345213 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0941a2970 con 0x7fb094104410 2024-03-28T00:25:55.626 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:55.619+0000 7fb09aa00700 1 -- 172.21.15.119:0/132345213 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb094068e50 con 0x7fb094104410 2024-03-28T00:25:55.627 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:55.621+0000 7fb090c00700 1 -- 172.21.15.119:0/132345213 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 18) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fb08400c7e0 con 0x7fb094104410 2024-03-28T00:25:55.629 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:55.622+0000 7fb090c00700 1 --2- 172.21.15.119:0/132345213 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fb0780a2910 0x7fb0780a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:55.629 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:55.623+0000 7fb090c00700 1 -- 172.21.15.119:0/132345213 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fb0840ceb30 con 0x7fb094104410 2024-03-28T00:25:55.629 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:55.623+0000 7fb093400700 1 --2- 172.21.15.119:0/132345213 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fb0780a2910 0x7fb0780a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:55.629 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:55.623+0000 7fb093400700 1 --2- 172.21.15.119:0/132345213 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fb0780a2910 0x7fb0780a4dd0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fb08000b2e0 tx=0x7fb08000b8f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:25:55.637 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:55.631+0000 7fb090c00700 1 -- 172.21.15.119:0/132345213 <== mon.2 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fb08409bbd0 con 0x7fb094104410 2024-03-28T00:25:55.871 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:55.865+0000 7fb09aa00700 1 -- 172.21.15.119:0/132345213 --> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fb0940625d0 con 0x7fb0780a2910 2024-03-28T00:25:55.874 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:55.868+0000 7fb090c00700 1 -- 172.21.15.119:0/132345213 <== mgr.14152 v2:172.21.15.119:6800/1401937096 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+191 (secure 0 0 0) 0x7fb0940625d0 con 0x7fb0780a2910 2024-03-28T00:25:55.874 INFO:teuthology.orchestra.run.smithi119.stdout:HOST ADDR LABELS STATUS 2024-03-28T00:25:55.874 INFO:teuthology.orchestra.run.smithi119.stdout:smithi119 172.21.15.119 2024-03-28T00:25:55.874 INFO:teuthology.orchestra.run.smithi119.stdout:smithi132 172.21.15.132 2024-03-28T00:25:55.874 INFO:teuthology.orchestra.run.smithi119.stdout:smithi165 172.21.15.165 2024-03-28T00:25:55.874 INFO:teuthology.orchestra.run.smithi119.stdout:3 hosts in cluster 2024-03-28T00:25:55.877 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:55.870+0000 7fb09aa00700 1 -- 172.21.15.119:0/132345213 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fb0780a2910 msgr2=0x7fb0780a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:55.877 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:55.870+0000 7fb09aa00700 1 --2- 172.21.15.119:0/132345213 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fb0780a2910 0x7fb0780a4dd0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fb08000b2e0 tx=0x7fb08000b8f0 comp rx=0 tx=0).stop 2024-03-28T00:25:55.877 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:55.870+0000 7fb09aa00700 1 -- 172.21.15.119:0/132345213 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb094104410 msgr2=0x7fb09419a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:55.877 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:55.870+0000 7fb09aa00700 1 --2- 172.21.15.119:0/132345213 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb094104410 0x7fb09419a530 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fb08400e970 tx=0x7fb08400ec80 comp rx=0 tx=0).stop 2024-03-28T00:25:55.877 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:55.871+0000 7fb09aa00700 1 -- 172.21.15.119:0/132345213 shutdown_connections 2024-03-28T00:25:55.877 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:55.871+0000 7fb09aa00700 1 --2- 172.21.15.119:0/132345213 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fb0780a2910 0x7fb0780a4dd0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:55.877 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:55.871+0000 7fb09aa00700 1 --2- 172.21.15.119:0/132345213 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb094105fb0 0x7fb0941a1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:55.878 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:55.871+0000 7fb09aa00700 1 --2- 172.21.15.119:0/132345213 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb094104410 0x7fb09419a530 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:55.878 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:55.871+0000 7fb09aa00700 1 --2- 172.21.15.119:0/132345213 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb094105610 0x7fb09419aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:55.878 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:55.871+0000 7fb09aa00700 1 -- 172.21.15.119:0/132345213 >> 172.21.15.119:0/132345213 conn(0x7fb0940ff9b0 msgr2=0x7fb094108840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:25:55.878 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:55.871+0000 7fb09aa00700 1 -- 172.21.15.119:0/132345213 shutdown_connections 2024-03-28T00:25:55.878 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:55.871+0000 7fb09aa00700 1 -- 172.21.15.119:0/132345213 wait complete. 2024-03-28T00:25:56.693 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 10f73b92-ec98-11ee-95d0-87774f69a715 -- bash -c 'ceph orch device ls' 2024-03-28T00:25:57.138 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/mon.a/config 2024-03-28T00:25:57.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:25:56 smithi132 ceph-mon[35941]: pgmap v373: 97 pgs: 97 active+clean; 648 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T00:25:57.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:25:56 smithi132 ceph-mon[35941]: from='client.25070 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T00:25:57.282 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:25:56 smithi119 ceph-mon[31518]: pgmap v373: 97 pgs: 97 active+clean; 648 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T00:25:57.283 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:25:56 smithi119 ceph-mon[31518]: from='client.25070 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T00:25:57.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:25:56 smithi165 ceph-mon[36302]: pgmap v373: 97 pgs: 97 active+clean; 648 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T00:25:57.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:25:56 smithi165 ceph-mon[36302]: from='client.25070 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T00:25:59.034 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:25:58 smithi119 ceph-mon[31518]: pgmap v374: 97 pgs: 97 active+clean; 648 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-28T00:25:59.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:25:58 smithi132 ceph-mon[35941]: pgmap v374: 97 pgs: 97 active+clean; 648 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-28T00:25:59.322 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:59.315+0000 7fa01a000700 1 -- 172.21.15.119:0/3049057229 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa014105510 msgr2=0x7fa014105970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:59.322 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:59.315+0000 7fa01a000700 1 --2- 172.21.15.119:0/3049057229 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa014105510 0x7fa014105970 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa000009920 tx=0x7fa000009c30 comp rx=0 tx=0).stop 2024-03-28T00:25:59.322 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:59.315+0000 7fa01a000700 1 -- 172.21.15.119:0/3049057229 shutdown_connections 2024-03-28T00:25:59.322 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:59.315+0000 7fa01a000700 1 --2- 172.21.15.119:0/3049057229 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa014105eb0 0x7fa01410c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:59.322 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:59.315+0000 7fa01a000700 1 --2- 172.21.15.119:0/3049057229 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa014104310 0x7fa014104730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:59.322 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:59.315+0000 7fa01a000700 1 --2- 172.21.15.119:0/3049057229 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa014105510 0x7fa014105970 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:59.322 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:59.315+0000 7fa01a000700 1 -- 172.21.15.119:0/3049057229 >> 172.21.15.119:0/3049057229 conn(0x7fa0140ff8b0 msgr2=0x7fa014101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:25:59.322 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:59.315+0000 7fa01a000700 1 -- 172.21.15.119:0/3049057229 shutdown_connections 2024-03-28T00:25:59.322 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:59.315+0000 7fa01a000700 1 -- 172.21.15.119:0/3049057229 wait complete. 2024-03-28T00:25:59.324 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:59.316+0000 7fa01a000700 1 Processor -- start 2024-03-28T00:25:59.324 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:59.317+0000 7fa01a000700 1 -- start start 2024-03-28T00:25:59.324 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:59.317+0000 7fa01a000700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa014104310 0x7fa01419a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:59.325 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:59.317+0000 7fa01a000700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa014105510 0x7fa01419aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:59.325 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:59.317+0000 7fa01a000700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa014105eb0 0x7fa0141a1b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:59.325 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:59.317+0000 7fa01a000700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa01410fee0 con 0x7fa014104310 2024-03-28T00:25:59.325 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:59.317+0000 7fa01a000700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa01410fd60 con 0x7fa014105eb0 2024-03-28T00:25:59.325 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:59.317+0000 7fa01a000700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa014110060 con 0x7fa014105510 2024-03-28T00:25:59.325 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:59.317+0000 7fa013e00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa014105510 0x7fa01419aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:59.325 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:59.318+0000 7fa013e00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa014105510 0x7fa01419aa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.119:44330/0 (socket says 172.21.15.119:44330) 2024-03-28T00:25:59.325 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:59.318+0000 7fa013e00700 1 -- 172.21.15.119:0/759322283 learned_addr learned my addr 172.21.15.119:0/759322283 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:25:59.325 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:59.318+0000 7fa019600700 1 --2- 172.21.15.119:0/759322283 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa014105eb0 0x7fa0141a1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:59.325 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:59.318+0000 7fa018c00700 1 --2- 172.21.15.119:0/759322283 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa014104310 0x7fa01419a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:59.325 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:59.318+0000 7fa019600700 1 -- 172.21.15.119:0/759322283 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa014105510 msgr2=0x7fa01419aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:59.325 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:59.318+0000 7fa019600700 1 --2- 172.21.15.119:0/759322283 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa014105510 0x7fa01419aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:59.325 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:59.318+0000 7fa019600700 1 -- 172.21.15.119:0/759322283 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa014104310 msgr2=0x7fa01419a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:59.325 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:59.318+0000 7fa019600700 1 --2- 172.21.15.119:0/759322283 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa014104310 0x7fa01419a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:59.325 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:59.318+0000 7fa019600700 1 -- 172.21.15.119:0/759322283 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0141a21b0 con 0x7fa014105eb0 2024-03-28T00:25:59.326 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:59.318+0000 7fa019600700 1 --2- 172.21.15.119:0/759322283 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa014105eb0 0x7fa0141a1b10 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa00c00efc0 tx=0x7fa00c00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:25:59.326 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:59.319+0000 7fa011600700 1 -- 172.21.15.119:0/759322283 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa00c0092e0 con 0x7fa014105eb0 2024-03-28T00:25:59.326 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:59.319+0000 7fa011600700 1 -- 172.21.15.119:0/759322283 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa00c004bb0 con 0x7fa014105eb0 2024-03-28T00:25:59.327 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:59.319+0000 7fa01a000700 1 -- 172.21.15.119:0/759322283 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0140765d0 con 0x7fa014105eb0 2024-03-28T00:25:59.327 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:59.319+0000 7fa011600700 1 -- 172.21.15.119:0/759322283 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa00c004280 con 0x7fa014105eb0 2024-03-28T00:25:59.327 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:59.319+0000 7fa01a000700 1 -- 172.21.15.119:0/759322283 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa014076ac0 con 0x7fa014105eb0 2024-03-28T00:25:59.327 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:59.320+0000 7fa01a000700 1 -- 172.21.15.119:0/759322283 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ff4005360 con 0x7fa014105eb0 2024-03-28T00:25:59.328 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:59.322+0000 7fa011600700 1 -- 172.21.15.119:0/759322283 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 18) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fa00c01b030 con 0x7fa014105eb0 2024-03-28T00:25:59.330 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:59.323+0000 7fa011600700 1 --2- 172.21.15.119:0/759322283 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f9ffc0a2890 0x7f9ffc0a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:25:59.330 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:59.323+0000 7fa011600700 1 -- 172.21.15.119:0/759322283 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fa00c0ce3a0 con 0x7fa014105eb0 2024-03-28T00:25:59.330 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:59.324+0000 7fa018c00700 1 --2- 172.21.15.119:0/759322283 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f9ffc0a2890 0x7f9ffc0a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:25:59.330 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:59.324+0000 7fa018c00700 1 --2- 172.21.15.119:0/759322283 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f9ffc0a2890 0x7f9ffc0a4d50 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fa014105370 tx=0x7fa008016040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:25:59.334 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:59.327+0000 7fa011600700 1 -- 172.21.15.119:0/759322283 <== mon.2 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa00c022020 con 0x7fa014105eb0 2024-03-28T00:25:59.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:25:58 smithi165 ceph-mon[36302]: pgmap v374: 97 pgs: 97 active+clean; 648 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-28T00:25:59.567 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:59.560+0000 7fa01a000700 1 -- 172.21.15.119:0/759322283 --> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f9ff4000bf0 con 0x7f9ffc0a2890 2024-03-28T00:25:59.570 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:59.564+0000 7fa011600700 1 -- 172.21.15.119:0/759322283 <== mgr.14152 v2:172.21.15.119:6800/1401937096 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+532 (secure 0 0 0) 0x7f9ff4000bf0 con 0x7f9ffc0a2890 2024-03-28T00:25:59.571 INFO:teuthology.orchestra.run.smithi119.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-03-28T00:25:59.571 INFO:teuthology.orchestra.run.smithi119.stdout:smithi119 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400Y1400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-03-28T00:25:59.571 INFO:teuthology.orchestra.run.smithi119.stdout:smithi132 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6232000B400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-03-28T00:25:59.571 INFO:teuthology.orchestra.run.smithi119.stdout:smithi165 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300CE400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-03-28T00:25:59.574 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:59.567+0000 7fa01a000700 1 -- 172.21.15.119:0/759322283 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f9ffc0a2890 msgr2=0x7f9ffc0a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:59.574 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:59.567+0000 7fa01a000700 1 --2- 172.21.15.119:0/759322283 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f9ffc0a2890 0x7f9ffc0a4d50 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fa014105370 tx=0x7fa008016040 comp rx=0 tx=0).stop 2024-03-28T00:25:59.574 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:59.567+0000 7fa01a000700 1 -- 172.21.15.119:0/759322283 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa014105eb0 msgr2=0x7fa0141a1b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:25:59.574 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:59.567+0000 7fa01a000700 1 --2- 172.21.15.119:0/759322283 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa014105eb0 0x7fa0141a1b10 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa00c00efc0 tx=0x7fa00c00c3a0 comp rx=0 tx=0).stop 2024-03-28T00:25:59.574 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:59.567+0000 7fa01a000700 1 -- 172.21.15.119:0/759322283 shutdown_connections 2024-03-28T00:25:59.574 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:59.567+0000 7fa01a000700 1 --2- 172.21.15.119:0/759322283 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f9ffc0a2890 0x7f9ffc0a4d50 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:59.574 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:59.567+0000 7fa01a000700 1 --2- 172.21.15.119:0/759322283 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa014105eb0 0x7fa0141a1b10 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:59.574 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:59.567+0000 7fa01a000700 1 --2- 172.21.15.119:0/759322283 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa014104310 0x7fa01419a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:59.574 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:59.567+0000 7fa01a000700 1 --2- 172.21.15.119:0/759322283 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa014105510 0x7fa01419aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:25:59.575 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:59.567+0000 7fa01a000700 1 -- 172.21.15.119:0/759322283 >> 172.21.15.119:0/759322283 conn(0x7fa0140ff8b0 msgr2=0x7fa014108880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:25:59.575 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:59.568+0000 7fa01a000700 1 -- 172.21.15.119:0/759322283 shutdown_connections 2024-03-28T00:25:59.575 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:25:59.568+0000 7fa01a000700 1 -- 172.21.15.119:0/759322283 wait complete. 2024-03-28T00:26:00.393 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-28T00:26:00.404 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi119.front.sepia.ceph.com 2024-03-28T00:26:00.404 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 10f73b92-ec98-11ee-95d0-87774f69a715 -- bash -c 'ceph fs dump' 2024-03-28T00:26:00.851 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/mon.a/config 2024-03-28T00:26:01.191 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:00 smithi119 ceph-mon[31518]: from='client.25076 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T00:26:01.191 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:00 smithi119 ceph-mon[31518]: pgmap v375: 97 pgs: 97 active+clean; 648 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-28T00:26:01.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:00 smithi132 ceph-mon[35941]: from='client.25076 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T00:26:01.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:00 smithi132 ceph-mon[35941]: pgmap v375: 97 pgs: 97 active+clean; 648 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-28T00:26:01.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:00 smithi165 ceph-mon[36302]: from='client.25076 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T00:26:01.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:00 smithi165 ceph-mon[36302]: pgmap v375: 97 pgs: 97 active+clean; 648 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-28T00:26:02.996 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:02.989+0000 7f1093400700 1 -- 172.21.15.119:0/890333584 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f108c103d70 msgr2=0x7f108c10a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:26:02.997 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:02.989+0000 7f1093400700 1 --2- 172.21.15.119:0/890333584 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f108c103d70 0x7f108c10a620 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f108400b0d0 tx=0x7f108400b3e0 comp rx=0 tx=0).stop 2024-03-28T00:26:02.997 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:02.989+0000 7f1093400700 1 -- 172.21.15.119:0/890333584 shutdown_connections 2024-03-28T00:26:02.997 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:02.989+0000 7f1093400700 1 --2- 172.21.15.119:0/890333584 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f108c103d70 0x7f108c10a620 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:02.997 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:02.989+0000 7f1093400700 1 --2- 172.21.15.119:0/890333584 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f108c1021d0 0x7f108c1025f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:02.997 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:02.989+0000 7f1093400700 1 --2- 172.21.15.119:0/890333584 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f108c1033d0 0x7f108c103830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:02.997 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:02.989+0000 7f1093400700 1 -- 172.21.15.119:0/890333584 >> 172.21.15.119:0/890333584 conn(0x7f108c0fd730 msgr2=0x7f108c0ffbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:26:02.997 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:02.989+0000 7f1093400700 1 -- 172.21.15.119:0/890333584 shutdown_connections 2024-03-28T00:26:02.997 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:02.989+0000 7f1093400700 1 -- 172.21.15.119:0/890333584 wait complete. 2024-03-28T00:26:02.999 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:02.990+0000 7f1093400700 1 Processor -- start 2024-03-28T00:26:02.999 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:02.991+0000 7f1093400700 1 -- start start 2024-03-28T00:26:02.999 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:02.991+0000 7f1093400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f108c1021d0 0x7f108c075f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:02.999 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:02.991+0000 7f1093400700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f108c1033d0 0x7f108c074580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:02.999 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:02.991+0000 7f1093400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f108c103d70 0x7f108c074ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:02.999 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:02.991+0000 7f1093400700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f108c10de20 con 0x7f108c1021d0 2024-03-28T00:26:02.999 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:02.991+0000 7f1093400700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f108c10dca0 con 0x7f108c103d70 2024-03-28T00:26:03.000 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:02.991+0000 7f1093400700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f108c10dfa0 con 0x7f108c1033d0 2024-03-28T00:26:03.000 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:02.991+0000 7f1083e00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f108c1033d0 0x7f108c074580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:03.000 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:02.991+0000 7f1083e00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f108c1033d0 0x7f108c074580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.119:44338/0 (socket says 172.21.15.119:44338) 2024-03-28T00:26:03.000 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:02.991+0000 7f1083e00700 1 -- 172.21.15.119:0/2861656643 learned_addr learned my addr 172.21.15.119:0/2861656643 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:26:03.000 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:02.991+0000 7f1091400700 1 --2- 172.21.15.119:0/2861656643 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f108c103d70 0x7f108c074ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:03.000 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:02.991+0000 7f1090a00700 1 --2- 172.21.15.119:0/2861656643 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f108c1021d0 0x7f108c075f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:03.000 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:02.992+0000 7f1083e00700 1 -- 172.21.15.119:0/2861656643 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f108c103d70 msgr2=0x7f108c074ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:26:03.000 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:02.992+0000 7f1083e00700 1 --2- 172.21.15.119:0/2861656643 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f108c103d70 0x7f108c074ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:03.000 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:02.992+0000 7f1083e00700 1 -- 172.21.15.119:0/2861656643 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f108c1021d0 msgr2=0x7f108c075f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:26:03.000 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:02.992+0000 7f1083e00700 1 --2- 172.21.15.119:0/2861656643 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f108c1021d0 0x7f108c075f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:03.000 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:02.992+0000 7f1083e00700 1 -- 172.21.15.119:0/2861656643 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f108c075370 con 0x7f108c1033d0 2024-03-28T00:26:03.000 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:02.992+0000 7f1090a00700 1 --2- 172.21.15.119:0/2861656643 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f108c1021d0 0x7f108c075f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T00:26:03.000 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:02.992+0000 7f1083e00700 1 --2- 172.21.15.119:0/2861656643 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f108c1033d0 0x7f108c074580 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f107c00e9a0 tx=0x7f107c00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:26:03.001 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:02.992+0000 7f108a000700 1 -- 172.21.15.119:0/2861656643 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f107c00cbd0 con 0x7f108c1033d0 2024-03-28T00:26:03.001 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:02.992+0000 7f108a000700 1 -- 172.21.15.119:0/2861656643 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f107c00cd70 con 0x7f108c1033d0 2024-03-28T00:26:03.001 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:02.992+0000 7f1093400700 1 -- 172.21.15.119:0/2861656643 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f108c1a2560 con 0x7f108c1033d0 2024-03-28T00:26:03.001 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:02.992+0000 7f108a000700 1 -- 172.21.15.119:0/2861656643 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f107c018750 con 0x7f108c1033d0 2024-03-28T00:26:03.001 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:02.992+0000 7f1093400700 1 -- 172.21.15.119:0/2861656643 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f108c1a2a70 con 0x7f108c1033d0 2024-03-28T00:26:03.001 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:02.994+0000 7f1093400700 1 -- 172.21.15.119:0/2861656643 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f108c068e50 con 0x7f108c1033d0 2024-03-28T00:26:03.002 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:02.995+0000 7f108a000700 1 -- 172.21.15.119:0/2861656643 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 18) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f107c010430 con 0x7f108c1033d0 2024-03-28T00:26:03.003 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:02.997+0000 7f108a000700 1 --2- 172.21.15.119:0/2861656643 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f106c0a2890 0x7f106c0a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:03.003 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:02.997+0000 7f108a000700 1 -- 172.21.15.119:0/2861656643 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f107c0ced20 con 0x7f108c1033d0 2024-03-28T00:26:03.003 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:02.997+0000 7f1090a00700 1 --2- 172.21.15.119:0/2861656643 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f106c0a2890 0x7f106c0a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:03.003 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:02.998+0000 7f1090a00700 1 --2- 172.21.15.119:0/2861656643 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f106c0a2890 0x7f106c0a4d50 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f108c103230 tx=0x7f1074009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:26:03.011 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:03.005+0000 7f108a000700 1 -- 172.21.15.119:0/2861656643 <== mon.1 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f107c09bdc0 con 0x7f108c1033d0 2024-03-28T00:26:03.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:02 smithi119 ceph-mon[31518]: pgmap v376: 97 pgs: 97 active+clean; 648 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T00:26:03.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:02 smithi132 ceph-mon[35941]: pgmap v376: 97 pgs: 97 active+clean; 648 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T00:26:03.283 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:03.277+0000 7f1093400700 1 -- 172.21.15.119:0/2861656643 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f108c062da0 con 0x7f108c1033d0 2024-03-28T00:26:03.283 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:03.277+0000 7f108a000700 1 -- 172.21.15.119:0/2861656643 <== mon.1 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 26 v26) v1 ==== 76+0+3045 (secure 0 0 0) 0x7f107c09b6c0 con 0x7f108c1033d0 2024-03-28T00:26:03.284 INFO:teuthology.orchestra.run.smithi119.stdout:e26 2024-03-28T00:26:03.284 INFO:teuthology.orchestra.run.smithi119.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-28T00:26:03.284 INFO:teuthology.orchestra.run.smithi119.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-03-28T00:26:03.285 INFO:teuthology.orchestra.run.smithi119.stdout:legacy client fscid: 1 2024-03-28T00:26:03.285 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-28T00:26:03.285 INFO:teuthology.orchestra.run.smithi119.stdout:Filesystem 'cephfs' (1) 2024-03-28T00:26:03.285 INFO:teuthology.orchestra.run.smithi119.stdout:fs_name cephfs 2024-03-28T00:26:03.285 INFO:teuthology.orchestra.run.smithi119.stdout:epoch 26 2024-03-28T00:26:03.285 INFO:teuthology.orchestra.run.smithi119.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-28T00:26:03.285 INFO:teuthology.orchestra.run.smithi119.stdout:created 2024-03-28T00:23:56.215355+0000 2024-03-28T00:26:03.285 INFO:teuthology.orchestra.run.smithi119.stdout:modified 2024-03-28T00:24:04.459985+0000 2024-03-28T00:26:03.285 INFO:teuthology.orchestra.run.smithi119.stdout:tableserver 0 2024-03-28T00:26:03.285 INFO:teuthology.orchestra.run.smithi119.stdout:root 0 2024-03-28T00:26:03.285 INFO:teuthology.orchestra.run.smithi119.stdout:session_timeout 300 2024-03-28T00:26:03.285 INFO:teuthology.orchestra.run.smithi119.stdout:session_autoclose 300 2024-03-28T00:26:03.285 INFO:teuthology.orchestra.run.smithi119.stdout:max_file_size 1099511627776 2024-03-28T00:26:03.285 INFO:teuthology.orchestra.run.smithi119.stdout:max_xattr_size 65536 2024-03-28T00:26:03.286 INFO:teuthology.orchestra.run.smithi119.stdout:required_client_features {} 2024-03-28T00:26:03.286 INFO:teuthology.orchestra.run.smithi119.stdout:last_failure 0 2024-03-28T00:26:03.286 INFO:teuthology.orchestra.run.smithi119.stdout:last_failure_osd_epoch 0 2024-03-28T00:26:03.286 INFO:teuthology.orchestra.run.smithi119.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-03-28T00:26:03.286 INFO:teuthology.orchestra.run.smithi119.stdout:max_mds 5 2024-03-28T00:26:03.286 INFO:teuthology.orchestra.run.smithi119.stdout:in 0,1,2,3,4 2024-03-28T00:26:03.286 INFO:teuthology.orchestra.run.smithi119.stdout:up {0=24581,1=24550,2=24530,3=24523,4=24511} 2024-03-28T00:26:03.286 INFO:teuthology.orchestra.run.smithi119.stdout:failed 2024-03-28T00:26:03.286 INFO:teuthology.orchestra.run.smithi119.stdout:damaged 2024-03-28T00:26:03.286 INFO:teuthology.orchestra.run.smithi119.stdout:stopped 2024-03-28T00:26:03.286 INFO:teuthology.orchestra.run.smithi119.stdout:data_pools [3] 2024-03-28T00:26:03.286 INFO:teuthology.orchestra.run.smithi119.stdout:metadata_pool 2 2024-03-28T00:26:03.286 INFO:teuthology.orchestra.run.smithi119.stdout:inline_data disabled 2024-03-28T00:26:03.286 INFO:teuthology.orchestra.run.smithi119.stdout:balancer 2024-03-28T00:26:03.287 INFO:teuthology.orchestra.run.smithi119.stdout:bal_rank_mask -1 2024-03-28T00:26:03.287 INFO:teuthology.orchestra.run.smithi119.stdout:standby_count_wanted 0 2024-03-28T00:26:03.287 INFO:teuthology.orchestra.run.smithi119.stdout:qdb_cluster leader: 24581 members: 24523,24530,24550,24511,24581 2024-03-28T00:26:03.287 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.b{0:24581} state up:active seq 3 addr [v2:172.21.15.132:6838/3305822525,v1:172.21.15.132:6839/3305822525] compat {c=[1],r=[1],i=[fff]}] 2024-03-28T00:26:03.287 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.e{0:24500} state up:standby-replay seq 1 addr [v2:172.21.15.132:6834/356981946,v1:172.21.15.132:6835/356981946] compat {c=[1],r=[1],i=[fff]}] 2024-03-28T00:26:03.289 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.i{1:24550} state up:active seq 7 addr [v2:172.21.15.165:6838/3004684592,v1:172.21.15.165:6839/3004684592] compat {c=[1],r=[1],i=[fff]}] 2024-03-28T00:26:03.290 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.j{1:24473} state up:standby-replay seq 1 addr [v2:172.21.15.119:6834/744950487,v1:172.21.15.119:6835/744950487] compat {c=[1],r=[1],i=[fff]}] 2024-03-28T00:26:03.290 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.h{2:24530} state up:active seq 8 addr [v2:172.21.15.132:6836/3383082967,v1:172.21.15.132:6837/3383082967] compat {c=[1],r=[1],i=[fff]}] 2024-03-28T00:26:03.290 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.k{2:24455} state up:standby-replay seq 1 addr [v2:172.21.15.132:6832/1179016670,v1:172.21.15.132:6833/1179016670] compat {c=[1],r=[1],i=[fff]}] 2024-03-28T00:26:03.290 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.c{3:24523} state up:active seq 9 addr [v2:172.21.15.165:6836/1531193032,v1:172.21.15.165:6837/1531193032] compat {c=[1],r=[1],i=[fff]}] 2024-03-28T00:26:03.290 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.l{3:24445} state up:standby-replay seq 1 addr [v2:172.21.15.165:6832/609963168,v1:172.21.15.165:6833/609963168] compat {c=[1],r=[1],i=[fff]}] 2024-03-28T00:26:03.290 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.f{4:24511} state up:active seq 10 addr [v2:172.21.15.165:6834/1947980102,v1:172.21.15.165:6835/1947980102] compat {c=[1],r=[1],i=[fff]}] 2024-03-28T00:26:03.290 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.a{4:14466} state up:standby-replay seq 1 addr [v2:172.21.15.119:6840/1678451188,v1:172.21.15.119:6841/1678451188] compat {c=[1],r=[1],i=[fff]}] 2024-03-28T00:26:03.290 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-28T00:26:03.290 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-28T00:26:03.290 INFO:teuthology.orchestra.run.smithi119.stdout:Standby daemons: 2024-03-28T00:26:03.290 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-28T00:26:03.290 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.d{-1:14400} state up:standby seq 1 addr [v2:172.21.15.119:6836/3988635716,v1:172.21.15.119:6837/3988635716] compat {c=[1],r=[1],i=[fff]}] 2024-03-28T00:26:03.290 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.g{-1:14448} state up:standby seq 1 addr [v2:172.21.15.119:6838/1595343240,v1:172.21.15.119:6839/1595343240] compat {c=[1],r=[1],i=[fff]}] 2024-03-28T00:26:03.291 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:03.280+0000 7f1093400700 1 -- 172.21.15.119:0/2861656643 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f106c0a2890 msgr2=0x7f106c0a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:26:03.291 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:03.280+0000 7f1093400700 1 --2- 172.21.15.119:0/2861656643 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f106c0a2890 0x7f106c0a4d50 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f108c103230 tx=0x7f1074009260 comp rx=0 tx=0).stop 2024-03-28T00:26:03.292 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:03.280+0000 7f1093400700 1 -- 172.21.15.119:0/2861656643 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f108c1033d0 msgr2=0x7f108c074580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:26:03.292 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:03.280+0000 7f1093400700 1 --2- 172.21.15.119:0/2861656643 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f108c1033d0 0x7f108c074580 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f107c00e9a0 tx=0x7f107c00ecb0 comp rx=0 tx=0).stop 2024-03-28T00:26:03.292 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:03.280+0000 7f1093400700 1 -- 172.21.15.119:0/2861656643 shutdown_connections 2024-03-28T00:26:03.292 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:03.280+0000 7f1093400700 1 --2- 172.21.15.119:0/2861656643 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f106c0a2890 0x7f106c0a4d50 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:03.292 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:03.280+0000 7f1093400700 1 --2- 172.21.15.119:0/2861656643 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f108c103d70 0x7f108c074ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:03.292 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:03.280+0000 7f1093400700 1 --2- 172.21.15.119:0/2861656643 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f108c1021d0 0x7f108c075f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:03.292 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:03.280+0000 7f1093400700 1 --2- 172.21.15.119:0/2861656643 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f108c1033d0 0x7f108c074580 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:03.292 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:03.280+0000 7f1093400700 1 -- 172.21.15.119:0/2861656643 >> 172.21.15.119:0/2861656643 conn(0x7f108c0fd730 msgr2=0x7f108c106670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:26:03.292 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:03.280+0000 7f1093400700 1 -- 172.21.15.119:0/2861656643 shutdown_connections 2024-03-28T00:26:03.292 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:03.280+0000 7f1093400700 1 -- 172.21.15.119:0/2861656643 wait complete. 2024-03-28T00:26:03.292 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 26 2024-03-28T00:26:03.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:02 smithi165 ceph-mon[36302]: pgmap v376: 97 pgs: 97 active+clean; 648 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T00:26:04.088 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 10f73b92-ec98-11ee-95d0-87774f69a715 -- bash -c 'ceph osd dump' 2024-03-28T00:26:04.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:03 smithi132 ceph-mon[35941]: from='client.? 172.21.15.119:0/2861656643' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-28T00:26:04.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:03 smithi165 ceph-mon[36302]: from='client.? 172.21.15.119:0/2861656643' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-28T00:26:04.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:03 smithi119 ceph-mon[31518]: from='client.? 172.21.15.119:0/2861656643' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-28T00:26:04.549 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/mon.a/config 2024-03-28T00:26:04.960 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:04 smithi119 ceph-mon[31518]: pgmap v377: 97 pgs: 97 active+clean; 648 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-28T00:26:05.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:04 smithi132 ceph-mon[35941]: pgmap v377: 97 pgs: 97 active+clean; 648 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-28T00:26:05.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:04 smithi165 ceph-mon[36302]: pgmap v377: 97 pgs: 97 active+clean; 648 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-28T00:26:06.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:06.754+0000 7f4f54000700 1 -- 172.21.15.119:0/2238918181 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4f4c105580 msgr2=0x7f4f4c107970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:26:06.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:06.754+0000 7f4f54000700 1 --2- 172.21.15.119:0/2238918181 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4f4c105580 0x7f4f4c107970 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f4f38009990 tx=0x7f4f38009ca0 comp rx=0 tx=0).stop 2024-03-28T00:26:06.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:06.754+0000 7f4f54000700 1 -- 172.21.15.119:0/2238918181 shutdown_connections 2024-03-28T00:26:06.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:06.754+0000 7f4f54000700 1 --2- 172.21.15.119:0/2238918181 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4f4c107eb0 0x7f4f4c10a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:06.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:06.754+0000 7f4f54000700 1 --2- 172.21.15.119:0/2238918181 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4f4c102c50 0x7f4f4c105040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:06.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:06.754+0000 7f4f54000700 1 --2- 172.21.15.119:0/2238918181 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4f4c105580 0x7f4f4c107970 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:06.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:06.754+0000 7f4f54000700 1 -- 172.21.15.119:0/2238918181 >> 172.21.15.119:0/2238918181 conn(0x7f4f4c0fc660 msgr2=0x7f4f4c0feaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:26:06.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:06.754+0000 7f4f54000700 1 -- 172.21.15.119:0/2238918181 shutdown_connections 2024-03-28T00:26:06.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:06.754+0000 7f4f54000700 1 -- 172.21.15.119:0/2238918181 wait complete. 2024-03-28T00:26:06.763 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:06.755+0000 7f4f54000700 1 Processor -- start 2024-03-28T00:26:06.763 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:06.755+0000 7f4f54000700 1 -- start start 2024-03-28T00:26:06.763 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:06.756+0000 7f4f54000700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4f4c102c50 0x7f4f4c198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:06.763 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:06.756+0000 7f4f54000700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4f4c105580 0x7f4f4c1989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:06.763 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:06.756+0000 7f4f54000700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4f4c107eb0 0x7f4f4c19fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:06.763 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:06.756+0000 7f4f54000700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f4c10ddf0 con 0x7f4f4c107eb0 2024-03-28T00:26:06.763 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:06.756+0000 7f4f54000700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f4c10dc70 con 0x7f4f4c102c50 2024-03-28T00:26:06.763 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:06.756+0000 7f4f54000700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f4c10df70 con 0x7f4f4c105580 2024-03-28T00:26:06.763 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:06.756+0000 7f4f50c00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4f4c105580 0x7f4f4c1989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:06.763 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:06.756+0000 7f4f50c00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4f4c105580 0x7f4f4c1989b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.119:33244/0 (socket says 172.21.15.119:33244) 2024-03-28T00:26:06.763 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:06.756+0000 7f4f50c00700 1 -- 172.21.15.119:0/2076121698 learned_addr learned my addr 172.21.15.119:0/2076121698 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:26:06.763 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:06.756+0000 7f4f52000700 1 --2- 172.21.15.119:0/2076121698 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4f4c107eb0 0x7f4f4c19fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:06.764 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:06.756+0000 7f4f51600700 1 --2- 172.21.15.119:0/2076121698 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4f4c102c50 0x7f4f4c198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:06.764 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:06.757+0000 7f4f50c00700 1 -- 172.21.15.119:0/2076121698 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4f4c102c50 msgr2=0x7f4f4c198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:26:06.764 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:06.757+0000 7f4f50c00700 1 --2- 172.21.15.119:0/2076121698 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4f4c102c50 0x7f4f4c198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:06.764 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:06.757+0000 7f4f50c00700 1 -- 172.21.15.119:0/2076121698 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4f4c107eb0 msgr2=0x7f4f4c19fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:26:06.764 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:06.757+0000 7f4f50c00700 1 --2- 172.21.15.119:0/2076121698 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4f4c107eb0 0x7f4f4c19fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:06.764 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:06.757+0000 7f4f50c00700 1 -- 172.21.15.119:0/2076121698 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f4c1a00d0 con 0x7f4f4c105580 2024-03-28T00:26:06.765 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:06.757+0000 7f4f50c00700 1 --2- 172.21.15.119:0/2076121698 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4f4c105580 0x7f4f4c1989b0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f4f38005f50 tx=0x7f4f3800bbb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:26:06.765 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:06.757+0000 7f4f4a000700 1 -- 172.21.15.119:0/2076121698 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f38026070 con 0x7f4f4c105580 2024-03-28T00:26:06.765 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:06.757+0000 7f4f54000700 1 -- 172.21.15.119:0/2076121698 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f4c1a0360 con 0x7f4f4c105580 2024-03-28T00:26:06.765 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:06.757+0000 7f4f4a000700 1 -- 172.21.15.119:0/2076121698 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4f38003d70 con 0x7f4f4c105580 2024-03-28T00:26:06.765 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:06.757+0000 7f4f4a000700 1 -- 172.21.15.119:0/2076121698 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f38021410 con 0x7f4f4c105580 2024-03-28T00:26:06.765 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:06.757+0000 7f4f54000700 1 -- 172.21.15.119:0/2076121698 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f4c1a0810 con 0x7f4f4c105580 2024-03-28T00:26:06.765 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:06.758+0000 7f4f54000700 1 -- 172.21.15.119:0/2076121698 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f4c068e50 con 0x7f4f4c105580 2024-03-28T00:26:06.767 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:06.760+0000 7f4f4a000700 1 -- 172.21.15.119:0/2076121698 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 18) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f4f38029070 con 0x7f4f4c105580 2024-03-28T00:26:06.768 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:06.762+0000 7f4f4a000700 1 --2- 172.21.15.119:0/2076121698 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f4f340a2890 0x7f4f340a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:06.768 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:06.762+0000 7f4f4a000700 1 -- 172.21.15.119:0/2076121698 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f4f380cfa50 con 0x7f4f4c105580 2024-03-28T00:26:06.768 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:06.762+0000 7f4f51600700 1 --2- 172.21.15.119:0/2076121698 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f4f340a2890 0x7f4f340a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:06.769 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:06.763+0000 7f4f51600700 1 --2- 172.21.15.119:0/2076121698 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f4f340a2890 0x7f4f340a4d50 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f4f40009c60 tx=0x7f4f40009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:26:06.775 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:06.768+0000 7f4f4a000700 1 -- 172.21.15.119:0/2076121698 <== mon.1 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f4f38013070 con 0x7f4f4c105580 2024-03-28T00:26:06.997 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:06.990+0000 7f4f54000700 1 -- 172.21.15.119:0/2076121698 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump"} v 0) v1 -- 0x7f4f4c02da30 con 0x7f4f4c105580 2024-03-28T00:26:06.998 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:06.991+0000 7f4f4a000700 1 -- 172.21.15.119:0/2076121698 <== mon.1 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v87) v1 ==== 56+0+5263 (secure 0 0 0) 0x7f4f3809cb80 con 0x7f4f4c105580 2024-03-28T00:26:06.998 INFO:teuthology.orchestra.run.smithi119.stdout:epoch 87 2024-03-28T00:26:06.998 INFO:teuthology.orchestra.run.smithi119.stdout:fsid 10f73b92-ec98-11ee-95d0-87774f69a715 2024-03-28T00:26:06.998 INFO:teuthology.orchestra.run.smithi119.stdout:created 2024-03-28T00:14:57.563604+0000 2024-03-28T00:26:06.998 INFO:teuthology.orchestra.run.smithi119.stdout:modified 2024-03-28T00:24:05.529756+0000 2024-03-28T00:26:06.998 INFO:teuthology.orchestra.run.smithi119.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-03-28T00:26:06.999 INFO:teuthology.orchestra.run.smithi119.stdout:crush_version 26 2024-03-28T00:26:06.999 INFO:teuthology.orchestra.run.smithi119.stdout:full_ratio 0.95 2024-03-28T00:26:06.999 INFO:teuthology.orchestra.run.smithi119.stdout:backfillfull_ratio 0.9 2024-03-28T00:26:06.999 INFO:teuthology.orchestra.run.smithi119.stdout:nearfull_ratio 0.85 2024-03-28T00:26:06.999 INFO:teuthology.orchestra.run.smithi119.stdout:require_min_compat_client luminous 2024-03-28T00:26:06.999 INFO:teuthology.orchestra.run.smithi119.stdout:min_compat_client luminous 2024-03-28T00:26:06.999 INFO:teuthology.orchestra.run.smithi119.stdout:require_osd_release squid 2024-03-28T00:26:06.999 INFO:teuthology.orchestra.run.smithi119.stdout:stretch_mode_enabled false 2024-03-28T00:26:06.999 INFO:teuthology.orchestra.run.smithi119.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-03-28T00:26:06.999 INFO:teuthology.orchestra.run.smithi119.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 84 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-03-28T00:26:06.999 INFO:teuthology.orchestra.run.smithi119.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 85 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-03-28T00:26:06.999 INFO:teuthology.orchestra.run.smithi119.stdout:max_osd 12 2024-03-28T00:26:06.999 INFO:teuthology.orchestra.run.smithi119.stdout:osd.0 up in weight 1 up_from 8 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.119:6802/580396252,v1:172.21.15.119:6803/580396252] [v2:172.21.15.119:6804/580396252,v1:172.21.15.119:6805/580396252] exists,up 7c0b875d-a01e-4294-9c9c-5d0b048b8912 2024-03-28T00:26:06.999 INFO:teuthology.orchestra.run.smithi119.stdout:osd.1 up in weight 1 up_from 13 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.132:6800/2500318888,v1:172.21.15.132:6801/2500318888] [v2:172.21.15.132:6802/2500318888,v1:172.21.15.132:6803/2500318888] exists,up 5567f7c2-8ea4-4789-9ea1-467793324fb0 2024-03-28T00:26:06.999 INFO:teuthology.orchestra.run.smithi119.stdout:osd.2 up in weight 1 up_from 18 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.165:6800/2361434832,v1:172.21.15.165:6801/2361434832] [v2:172.21.15.165:6802/2361434832,v1:172.21.15.165:6803/2361434832] exists,up ee54dbd0-24b4-4ee6-988c-ce71491199d7 2024-03-28T00:26:07.000 INFO:teuthology.orchestra.run.smithi119.stdout:osd.3 up in weight 1 up_from 25 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.119:6810/3732272210,v1:172.21.15.119:6811/3732272210] [v2:172.21.15.119:6812/3732272210,v1:172.21.15.119:6813/3732272210] exists,up 0464c40b-828f-4db4-96d9-3f7a6325e6da 2024-03-28T00:26:07.000 INFO:teuthology.orchestra.run.smithi119.stdout:osd.4 up in weight 1 up_from 33 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.132:6808/2803039505,v1:172.21.15.132:6809/2803039505] [v2:172.21.15.132:6810/2803039505,v1:172.21.15.132:6811/2803039505] exists,up 8fd70b8e-5128-4173-aa0e-e735dfeb995d 2024-03-28T00:26:07.000 INFO:teuthology.orchestra.run.smithi119.stdout:osd.5 up in weight 1 up_from 38 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.165:6808/2561252206,v1:172.21.15.165:6809/2561252206] [v2:172.21.15.165:6810/2561252206,v1:172.21.15.165:6811/2561252206] exists,up 3224b4c1-733e-4424-a6da-870bbb5f94e5 2024-03-28T00:26:07.000 INFO:teuthology.orchestra.run.smithi119.stdout:osd.6 up in weight 1 up_from 46 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.119:6818/2545410230,v1:172.21.15.119:6819/2545410230] [v2:172.21.15.119:6820/2545410230,v1:172.21.15.119:6821/2545410230] exists,up e435745d-59b5-439a-9091-0946ca0e9332 2024-03-28T00:26:07.000 INFO:teuthology.orchestra.run.smithi119.stdout:osd.7 up in weight 1 up_from 51 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.132:6816/2877892376,v1:172.21.15.132:6817/2877892376] [v2:172.21.15.132:6818/2877892376,v1:172.21.15.132:6819/2877892376] exists,up 79b97a12-f6b4-48c3-b517-2dd6b6e7c85f 2024-03-28T00:26:07.000 INFO:teuthology.orchestra.run.smithi119.stdout:osd.8 up in weight 1 up_from 59 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.165:6816/4115226150,v1:172.21.15.165:6817/4115226150] [v2:172.21.15.165:6818/4115226150,v1:172.21.15.165:6819/4115226150] exists,up 2970524c-b99a-412d-85c2-6c29b54035e0 2024-03-28T00:26:07.000 INFO:teuthology.orchestra.run.smithi119.stdout:osd.9 up in weight 1 up_from 67 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.119:6826/2532245222,v1:172.21.15.119:6827/2532245222] [v2:172.21.15.119:6828/2532245222,v1:172.21.15.119:6829/2532245222] exists,up 2b1a08ed-cde9-4def-b3e8-838fff22709c 2024-03-28T00:26:07.000 INFO:teuthology.orchestra.run.smithi119.stdout:osd.10 up in weight 1 up_from 72 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.132:6824/3454298492,v1:172.21.15.132:6825/3454298492] [v2:172.21.15.132:6826/3454298492,v1:172.21.15.132:6827/3454298492] exists,up f019acd0-915a-4094-b12c-b53672740a5e 2024-03-28T00:26:07.000 INFO:teuthology.orchestra.run.smithi119.stdout:osd.11 up in weight 1 up_from 80 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.165:6824/2138771601,v1:172.21.15.165:6825/2138771601] [v2:172.21.15.165:6826/2138771601,v1:172.21.15.165:6827/2138771601] exists,up 2c1e59a8-8f21-45c0-8568-4b632ce020f2 2024-03-28T00:26:07.000 INFO:teuthology.orchestra.run.smithi119.stdout:pg_upmap_items 2.4 [2,11] 2024-03-28T00:26:07.000 INFO:teuthology.orchestra.run.smithi119.stdout:pg_upmap_items 2.f [8,11] 2024-03-28T00:26:07.000 INFO:teuthology.orchestra.run.smithi119.stdout:pg_upmap_items 3.0 [4,8] 2024-03-28T00:26:07.000 INFO:teuthology.orchestra.run.smithi119.stdout:pg_upmap_items 3.3 [4,9] 2024-03-28T00:26:07.000 INFO:teuthology.orchestra.run.smithi119.stdout:pg_upmap_items 3.20 [5,9] 2024-03-28T00:26:07.000 INFO:teuthology.orchestra.run.smithi119.stdout:pg_upmap_items 3.24 [4,9] 2024-03-28T00:26:07.001 INFO:teuthology.orchestra.run.smithi119.stdout:blocklist 172.21.15.119:6801/2575246519 expires 2024-03-29T00:16:03.582545+0000 2024-03-28T00:26:07.001 INFO:teuthology.orchestra.run.smithi119.stdout:blocklist 172.21.15.119:6800/2575246519 expires 2024-03-29T00:16:03.582545+0000 2024-03-28T00:26:07.001 INFO:teuthology.orchestra.run.smithi119.stdout:blocklist 172.21.15.119:0/87565965 expires 2024-03-29T00:16:03.582545+0000 2024-03-28T00:26:07.001 INFO:teuthology.orchestra.run.smithi119.stdout:blocklist 172.21.15.119:6801/2056426384 expires 2024-03-29T00:15:30.472283+0000 2024-03-28T00:26:07.001 INFO:teuthology.orchestra.run.smithi119.stdout:blocklist 172.21.15.119:0/1639109126 expires 2024-03-29T00:15:30.472283+0000 2024-03-28T00:26:07.001 INFO:teuthology.orchestra.run.smithi119.stdout:blocklist 172.21.15.119:0/468478300 expires 2024-03-29T00:15:30.472283+0000 2024-03-28T00:26:07.001 INFO:teuthology.orchestra.run.smithi119.stdout:blocklist 172.21.15.119:0/2891240357 expires 2024-03-29T00:15:30.472283+0000 2024-03-28T00:26:07.001 INFO:teuthology.orchestra.run.smithi119.stdout:blocklist 172.21.15.119:0/4201699272 expires 2024-03-29T00:16:03.582545+0000 2024-03-28T00:26:07.001 INFO:teuthology.orchestra.run.smithi119.stdout:blocklist 172.21.15.119:6800/2056426384 expires 2024-03-29T00:15:30.472283+0000 2024-03-28T00:26:07.001 INFO:teuthology.orchestra.run.smithi119.stdout:blocklist 172.21.15.119:0/2635982178 expires 2024-03-29T00:16:03.582545+0000 2024-03-28T00:26:07.004 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:06.997+0000 7f4f54000700 1 -- 172.21.15.119:0/2076121698 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f4f340a2890 msgr2=0x7f4f340a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:26:07.004 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:06.997+0000 7f4f54000700 1 --2- 172.21.15.119:0/2076121698 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f4f340a2890 0x7f4f340a4d50 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f4f40009c60 tx=0x7f4f40009260 comp rx=0 tx=0).stop 2024-03-28T00:26:07.004 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:06.997+0000 7f4f54000700 1 -- 172.21.15.119:0/2076121698 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4f4c105580 msgr2=0x7f4f4c1989b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:26:07.004 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:06.997+0000 7f4f54000700 1 --2- 172.21.15.119:0/2076121698 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4f4c105580 0x7f4f4c1989b0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f4f38005f50 tx=0x7f4f3800bbb0 comp rx=0 tx=0).stop 2024-03-28T00:26:07.004 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:06.997+0000 7f4f54000700 1 -- 172.21.15.119:0/2076121698 shutdown_connections 2024-03-28T00:26:07.004 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:06.997+0000 7f4f54000700 1 --2- 172.21.15.119:0/2076121698 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f4f340a2890 0x7f4f340a4d50 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:07.004 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:06.997+0000 7f4f54000700 1 --2- 172.21.15.119:0/2076121698 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4f4c107eb0 0x7f4f4c19fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:07.004 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:06.997+0000 7f4f54000700 1 --2- 172.21.15.119:0/2076121698 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4f4c102c50 0x7f4f4c198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:07.004 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:06.997+0000 7f4f54000700 1 --2- 172.21.15.119:0/2076121698 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4f4c105580 0x7f4f4c1989b0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:07.004 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:06.997+0000 7f4f54000700 1 -- 172.21.15.119:0/2076121698 >> 172.21.15.119:0/2076121698 conn(0x7f4f4c0fc660 msgr2=0x7f4f4c104aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:26:07.004 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:06.997+0000 7f4f54000700 1 -- 172.21.15.119:0/2076121698 shutdown_connections 2024-03-28T00:26:07.004 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:06.997+0000 7f4f54000700 1 -- 172.21.15.119:0/2076121698 wait complete. 2024-03-28T00:26:07.372 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:06 smithi119 ceph-mon[31518]: pgmap v378: 97 pgs: 97 active+clean; 648 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T00:26:07.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:06 smithi165 ceph-mon[36302]: pgmap v378: 97 pgs: 97 active+clean; 648 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T00:26:07.452 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:06 smithi132 ceph-mon[35941]: pgmap v378: 97 pgs: 97 active+clean; 648 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T00:26:07.799 INFO:teuthology.run_tasks:Running task fs.ready... 2024-03-28T00:26:07.817 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T00:26:08.044 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:08.037+0000 7fbddca00640 1 -- 172.21.15.119:0/598186720 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbdd8077c40 msgr2=0x7fbdd80784e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:26:08.044 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:08.037+0000 7fbddca00640 1 --2- 172.21.15.119:0/598186720 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbdd8077c40 0x7fbdd80784e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fbdd0009ca0 tx=0x7fbdd002c0f0 comp rx=0 tx=0).stop 2024-03-28T00:26:08.044 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:08.038+0000 7fbddca00640 1 -- 172.21.15.119:0/598186720 shutdown_connections 2024-03-28T00:26:08.044 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:08.038+0000 7fbddca00640 1 --2- 172.21.15.119:0/598186720 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbdd8077c40 0x7fbdd80784e0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:08.045 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:08.038+0000 7fbddca00640 1 --2- 172.21.15.119:0/598186720 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbdd80772a0 0x7fbdd8077700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:08.045 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:08.038+0000 7fbddca00640 1 --2- 172.21.15.119:0/598186720 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbdd8076060 0x7fbdd8076480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:08.045 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:08.038+0000 7fbddca00640 1 -- 172.21.15.119:0/598186720 >> 172.21.15.119:0/598186720 conn(0x7fbdd80fee50 msgr2=0x7fbdd8101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:26:08.045 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:08.038+0000 7fbddca00640 1 -- 172.21.15.119:0/598186720 shutdown_connections 2024-03-28T00:26:08.045 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:08.038+0000 7fbddca00640 1 -- 172.21.15.119:0/598186720 wait complete. 2024-03-28T00:26:08.047 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:08.039+0000 7fbddca00640 1 Processor -- start 2024-03-28T00:26:08.047 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:08.039+0000 7fbddca00640 1 -- start start 2024-03-28T00:26:08.047 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:08.039+0000 7fbddca00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbdd8076060 0x7fbdd819df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:08.047 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:08.039+0000 7fbddca00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbdd80772a0 0x7fbdd819e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:08.047 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:08.039+0000 7fbddca00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbdd8077c40 0x7fbdd81a5520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:08.047 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:08.039+0000 7fbddca00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdd8113970 con 0x7fbdd80772a0 2024-03-28T00:26:08.048 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:08.039+0000 7fbddca00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdd81137f0 con 0x7fbdd8077c40 2024-03-28T00:26:08.048 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:08.039+0000 7fbddca00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdd8113af0 con 0x7fbdd8076060 2024-03-28T00:26:08.048 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:08.039+0000 7fbdd5e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbdd8076060 0x7fbdd819df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:08.048 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:08.039+0000 7fbdd5e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbdd8076060 0x7fbdd819df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.119:33258/0 (socket says 172.21.15.119:33258) 2024-03-28T00:26:08.048 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:08.039+0000 7fbdd6800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbdd8077c40 0x7fbdd81a5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:08.048 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:08.039+0000 7fbdd5e00640 1 -- 172.21.15.119:0/2266502058 learned_addr learned my addr 172.21.15.119:0/2266502058 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:26:08.048 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:08.039+0000 7fbdcd400640 1 --2- 172.21.15.119:0/2266502058 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbdd80772a0 0x7fbdd819e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:08.048 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:08.040+0000 7fbdd5e00640 1 -- 172.21.15.119:0/2266502058 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbdd8077c40 msgr2=0x7fbdd81a5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:26:08.048 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:08.040+0000 7fbdd5e00640 1 --2- 172.21.15.119:0/2266502058 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbdd8077c40 0x7fbdd81a5520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:08.048 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:08.040+0000 7fbdd5e00640 1 -- 172.21.15.119:0/2266502058 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbdd80772a0 msgr2=0x7fbdd819e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:26:08.048 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:08.040+0000 7fbdd5e00640 1 --2- 172.21.15.119:0/2266502058 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbdd80772a0 0x7fbdd819e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:08.048 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:08.040+0000 7fbdd5e00640 1 -- 172.21.15.119:0/2266502058 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdd81a5b90 con 0x7fbdd8076060 2024-03-28T00:26:08.048 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:08.040+0000 7fbdd6800640 1 --2- 172.21.15.119:0/2266502058 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbdd8077c40 0x7fbdd81a5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T00:26:08.048 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:08.040+0000 7fbdcd400640 1 --2- 172.21.15.119:0/2266502058 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbdd80772a0 0x7fbdd819e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T00:26:08.049 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:08.040+0000 7fbdd5e00640 1 --2- 172.21.15.119:0/2266502058 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbdd8076060 0x7fbdd819df70 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fbdc800e9f0 tx=0x7fbdc800edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:26:08.049 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:08.040+0000 7fbdcf400640 1 -- 172.21.15.119:0/2266502058 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdc800cd80 con 0x7fbdd8076060 2024-03-28T00:26:08.049 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:08.040+0000 7fbdcf400640 1 -- 172.21.15.119:0/2266502058 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbdc8005500 con 0x7fbdd8076060 2024-03-28T00:26:08.049 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:08.040+0000 7fbdcf400640 1 -- 172.21.15.119:0/2266502058 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdc8010690 con 0x7fbdd8076060 2024-03-28T00:26:08.049 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:08.040+0000 7fbddca00640 1 -- 172.21.15.119:0/2266502058 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdd81a5e80 con 0x7fbdd8076060 2024-03-28T00:26:08.049 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:08.040+0000 7fbddca00640 1 -- 172.21.15.119:0/2266502058 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdd81a6290 con 0x7fbdd8076060 2024-03-28T00:26:08.049 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:08.041+0000 7fbddca00640 1 -- 172.21.15.119:0/2266502058 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbdd81103a0 con 0x7fbdd8076060 2024-03-28T00:26:08.049 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:08.043+0000 7fbdcf400640 1 -- 172.21.15.119:0/2266502058 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 18) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fbdc80046c0 con 0x7fbdd8076060 2024-03-28T00:26:08.051 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:08.045+0000 7fbdcf400640 1 --2- 172.21.15.119:0/2266502058 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fbdbc0a2880 0x7fbdbc0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:08.051 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:08.045+0000 7fbdcf400640 1 -- 172.21.15.119:0/2266502058 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fbdc80cede0 con 0x7fbdd8076060 2024-03-28T00:26:08.051 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:08.045+0000 7fbdcd400640 1 --2- 172.21.15.119:0/2266502058 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fbdbc0a2880 0x7fbdbc0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:08.051 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:08.045+0000 7fbdcd400640 1 --2- 172.21.15.119:0/2266502058 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fbdbc0a2880 0x7fbdbc0a4d40 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fbdd819f480 tx=0x7fbdc0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:26:08.055 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:08.049+0000 7fbdcf400640 1 -- 172.21.15.119:0/2266502058 <== mon.1 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fbdc809be80 con 0x7fbdd8076060 2024-03-28T00:26:08.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:07 smithi119 ceph-mon[31518]: from='client.? 172.21.15.119:0/2076121698' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-03-28T00:26:08.255 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:08.248+0000 7fbddca00640 1 -- 172.21.15.119:0/2266502058 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbdd80623e0 con 0x7fbdd8076060 2024-03-28T00:26:08.257 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:08.251+0000 7fbdcf400640 1 -- 172.21.15.119:0/2266502058 <== mon.1 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+22638 (secure 0 0 0) 0x7fbdc809b780 con 0x7fbdd8076060 2024-03-28T00:26:08.258 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-28T00:26:08.258 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":87,"fsid":"10f73b92-ec98-11ee-95d0-87774f69a715","created":"2024-03-28T00:14:57.563604+0000","modified":"2024-03-28T00:24:05.529756+0000","last_up_change":"2024-03-28T00:22:58.821662+0000","last_in_change":"2024-03-28T00:22:38.467270+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T00:18:46.253414+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-28T00:23:54.261981+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-28T00:23:55.685836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c0b875d-a01e-4294-9c9c-5d0b048b8912","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":580396252},{"type":"v1","addr":"172.21.15.119:6803","nonce":580396252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":580396252},{"type":"v1","addr":"172.21.15.119:6805","nonce":580396252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":580396252},{"type":"v1","addr":"172.21.15.119:6809","nonce":580396252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":580396252},{"type":"v1","addr":"172.21.15.119:6807","nonce":580396252}]},"public_addr":"172.21.15.119:6803/580396252","cluster_addr":"172.21.15.119:6805/580396252","heartbeat_back_addr":"172.21.15.119:6809/580396252","heartbeat_front_addr":"172.21.15.119:6807/580396252","state":["exists","up"]},{"osd":1,"uuid":"5567f7c2-8ea4-4789-9ea1-467793324fb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":2500318888},{"type":"v1","addr":"172.21.15.132:6801","nonce":2500318888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":2500318888},{"type":"v1","addr":"172.21.15.132:6803","nonce":2500318888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":2500318888},{"type":"v1","addr":"172.21.15.132:6807","nonce":2500318888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":2500318888},{"type":"v1","addr":"172.21.15.132:6805","nonce":2500318888}]},"public_addr":"172.21.15.132:6801/2500318888","cluster_addr":"172.21.15.132:6803/2500318888","heartbeat_back_addr":"172.21.15.132:6807/2500318888","heartbeat_front_addr":"172.21.15.132:6805/2500318888","state":["exists","up"]},{"osd":2,"uuid":"ee54dbd0-24b4-4ee6-988c-ce71491199d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":2361434832},{"type":"v1","addr":"172.21.15.165:6801","nonce":2361434832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":2361434832},{"type":"v1","addr":"172.21.15.165:6803","nonce":2361434832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":2361434832},{"type":"v1","addr":"172.21.15.165:6807","nonce":2361434832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":2361434832},{"type":"v1","addr":"172.21.15.165:6805","nonce":2361434832}]},"public_addr":"172.21.15.165:6801/2361434832","cluster_addr":"172.21.15.165:6803/2361434832","heartbeat_back_addr":"172.21.15.165:6807/2361434832","heartbeat_front_addr":"172.21.15.165:6805/2361434832","state":["exists","up"]},{"osd":3,"uuid":"0464c40b-828f-4db4-96d9-3f7a6325e6da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":3732272210},{"type":"v1","addr":"172.21.15.119:6811","nonce":3732272210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":3732272210},{"type":"v1","addr":"172.21.15.119:6813","nonce":3732272210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":3732272210},{"type":"v1","addr":"172.21.15.119:6817","nonce":3732272210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":3732272210},{"type":"v1","addr":"172.21.15.119:6815","nonce":3732272210}]},"public_addr":"172.21.15.119:6811/3732272210","cluster_addr":"172.21.15.119:6813/3732272210","heartbeat_back_addr":"172.21.15.119:6817/3732272210","heartbeat_front_addr":"172.21.15.119:6815/3732272210","state":["exists","up"]},{"osd":4,"uuid":"8fd70b8e-5128-4173-aa0e-e735dfeb995d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":2803039505},{"type":"v1","addr":"172.21.15.132:6809","nonce":2803039505}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":2803039505},{"type":"v1","addr":"172.21.15.132:6811","nonce":2803039505}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":2803039505},{"type":"v1","addr":"172.21.15.132:6815","nonce":2803039505}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":2803039505},{"type":"v1","addr":"172.21.15.132:6813","nonce":2803039505}]},"public_addr":"172.21.15.132:6809/2803039505","cluster_addr":"172.21.15.132:6811/2803039505","heartbeat_back_addr":"172.21.15.132:6815/2803039505","heartbeat_front_addr":"172.21.15.132:6813/2803039505","state":["exists","up"]},{"osd":5,"uuid":"3224b4c1-733e-4424-a6da-870bbb5f94e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":2561252206},{"type":"v1","addr":"172.21.15.165:6809","nonce":2561252206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":2561252206},{"type":"v1","addr":"172.21.15.165:6811","nonce":2561252206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":2561252206},{"type":"v1","addr":"172.21.15.165:6815","nonce":2561252206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":2561252206},{"type":"v1","addr":"172.21.15.165:6813","nonce":2561252206}]},"public_addr":"172.21.15.165:6809/2561252206","cluster_addr":"172.21.15.165:6811/2561252206","heartbeat_back_addr":"172.21.15.165:6815/2561252206","heartbeat_front_addr":"172.21.15.165:6813/2561252206","state":["exists","up"]},{"osd":6,"uuid":"e435745d-59b5-439a-9091-0946ca0e9332","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":2545410230},{"type":"v1","addr":"172.21.15.119:6819","nonce":2545410230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":2545410230},{"type":"v1","addr":"172.21.15.119:6821","nonce":2545410230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":2545410230},{"type":"v1","addr":"172.21.15.119:6825","nonce":2545410230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":2545410230},{"type":"v1","addr":"172.21.15.119:6823","nonce":2545410230}]},"public_addr":"172.21.15.119:6819/2545410230","cluster_addr":"172.21.15.119:6821/2545410230","heartbeat_back_addr":"172.21.15.119:6825/2545410230","heartbeat_front_addr":"172.21.15.119:6823/2545410230","state":["exists","up"]},{"osd":7,"uuid":"79b97a12-f6b4-48c3-b517-2dd6b6e7c85f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":2877892376},{"type":"v1","addr":"172.21.15.132:6817","nonce":2877892376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":2877892376},{"type":"v1","addr":"172.21.15.132:6819","nonce":2877892376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":2877892376},{"type":"v1","addr":"172.21.15.132:6823","nonce":2877892376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":2877892376},{"type":"v1","addr":"172.21.15.132:6821","nonce":2877892376}]},"public_addr":"172.21.15.132:6817/2877892376","cluster_addr":"172.21.15.132:6819/2877892376","heartbeat_back_addr":"172.21.15.132:6823/2877892376","heartbeat_front_addr":"172.21.15.132:6821/2877892376","state":["exists","up"]},{"osd":8,"uuid":"2970524c-b99a-412d-85c2-6c29b54035e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":4115226150},{"type":"v1","addr":"172.21.15.165:6817","nonce":4115226150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":4115226150},{"type":"v1","addr":"172.21.15.165:6819","nonce":4115226150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":4115226150},{"type":"v1","addr":"172.21.15.165:6823","nonce":4115226150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":4115226150},{"type":"v1","addr":"172.21.15.165:6821","nonce":4115226150}]},"public_addr":"172.21.15.165:6817/4115226150","cluster_addr":"172.21.15.165:6819/4115226150","heartbeat_back_addr":"172.21.15.165:6823/4115226150","heartbeat_front_addr":"172.21.15.165:6821/4115226150","state":["exists","up"]},{"osd":9,"uuid":"2b1a08ed-cde9-4def-b3e8-838fff22709c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":2532245222},{"type":"v1","addr":"172.21.15.119:6827","nonce":2532245222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":2532245222},{"type":"v1","addr":"172.21.15.119:6829","nonce":2532245222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":2532245222},{"type":"v1","addr":"172.21.15.119:6833","nonce":2532245222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":2532245222},{"type":"v1","addr":"172.21.15.119:6831","nonce":2532245222}]},"public_addr":"172.21.15.119:6827/2532245222","cluster_addr":"172.21.15.119:6829/2532245222","heartbeat_back_addr":"172.21.15.119:6833/2532245222","heartbeat_front_addr":"172.21.15.119:6831/2532245222","state":["exists","up"]},{"osd":10,"uuid":"f019acd0-915a-4094-b12c-b53672740a5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":3454298492},{"type":"v1","addr":"172.21.15.132:6825","nonce":3454298492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":3454298492},{"type":"v1","addr":"172.21.15.132:6827","nonce":3454298492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":3454298492},{"type":"v1","addr":"172.21.15.132:6831","nonce":3454298492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":3454298492},{"type":"v1","addr":"172.21.15.132:6829","nonce":3454298492}]},"public_addr":"172.21.15.132:6825/3454298492","cluster_addr":"172.21.15.132:6827/3454298492","heartbeat_back_addr":"172.21.15.132:6831/3454298492","heartbeat_front_addr":"172.21.15.132:6829/3454298492","state":["exists","up"]},{"osd":11,"uuid":"2c1e59a8-8f21-45c0-8568-4b632ce020f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":2138771601},{"type":"v1","addr":"172.21.15.165:6825","nonce":2138771601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":2138771601},{"type":"v1","addr":"172.21.15.165:6827","nonce":2138771601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":2138771601},{"type":"v1","addr":"172.21.15.165:6831","nonce":2138771601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":2138771601},{"type":"v1","addr":"172.21.15.165:6829","nonce":2138771601}]},"public_addr":"172.21.15.165:6825/2138771601","cluster_addr":"172.21.15.165:6827/2138771601","heartbeat_back_addr":"172.21.15.165:6831/2138771601","heartbeat_front_addr":"172.21.15.165:6829/2138771601","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:17:54.791531+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:18:19.466638+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:18:43.435536+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:19:15.702685+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:19:41.767370+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:20:06.793003+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:20:39.952744+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:21:05.727262+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:21:30.735829+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:22:04.072142+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:22:30.894047+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:22:56.637728+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.119:6801/2575246519":"2024-03-29T00:16:03.582545+0000","172.21.15.119:6800/2575246519":"2024-03-29T00:16:03.582545+0000","172.21.15.119:0/87565965":"2024-03-29T00:16:03.582545+0000","172.21.15.119:6801/2056426384":"2024-03-29T00:15:30.472283+0000","172.21.15.119:0/1639109126":"2024-03-29T00:15:30.472283+0000","172.21.15.119:0/468478300":"2024-03-29T00:15:30.472283+0000","172.21.15.119:0/2891240357":"2024-03-29T00:15:30.472283+0000","172.21.15.119:0/4201699272":"2024-03-29T00:16:03.582545+0000","172.21.15.119:6800/2056426384":"2024-03-29T00:15:30.472283+0000","172.21.15.119:0/2635982178":"2024-03-29T00:16:03.582545+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T00:26:08.261 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:08.254+0000 7fbddca00640 1 -- 172.21.15.119:0/2266502058 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fbdbc0a2880 msgr2=0x7fbdbc0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:26:08.261 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:08.254+0000 7fbddca00640 1 --2- 172.21.15.119:0/2266502058 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fbdbc0a2880 0x7fbdbc0a4d40 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fbdd819f480 tx=0x7fbdc0005f90 comp rx=0 tx=0).stop 2024-03-28T00:26:08.261 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:08.254+0000 7fbddca00640 1 -- 172.21.15.119:0/2266502058 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbdd8076060 msgr2=0x7fbdd819df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:26:08.261 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:08.254+0000 7fbddca00640 1 --2- 172.21.15.119:0/2266502058 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbdd8076060 0x7fbdd819df70 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fbdc800e9f0 tx=0x7fbdc800edf0 comp rx=0 tx=0).stop 2024-03-28T00:26:08.261 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:08.254+0000 7fbddca00640 1 -- 172.21.15.119:0/2266502058 shutdown_connections 2024-03-28T00:26:08.261 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:08.254+0000 7fbddca00640 1 --2- 172.21.15.119:0/2266502058 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fbdbc0a2880 0x7fbdbc0a4d40 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:08.261 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:08.254+0000 7fbddca00640 1 --2- 172.21.15.119:0/2266502058 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbdd8077c40 0x7fbdd81a5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:08.261 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:08.254+0000 7fbddca00640 1 --2- 172.21.15.119:0/2266502058 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbdd80772a0 0x7fbdd819e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:08.261 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:08.254+0000 7fbddca00640 1 --2- 172.21.15.119:0/2266502058 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbdd8076060 0x7fbdd819df70 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:08.261 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:08.254+0000 7fbddca00640 1 -- 172.21.15.119:0/2266502058 >> 172.21.15.119:0/2266502058 conn(0x7fbdd80fee50 msgr2=0x7fbdd8100530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:26:08.261 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:08.254+0000 7fbddca00640 1 -- 172.21.15.119:0/2266502058 shutdown_connections 2024-03-28T00:26:08.261 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:08.254+0000 7fbddca00640 1 -- 172.21.15.119:0/2266502058 wait complete. 2024-03-28T00:26:08.281 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-28T00:18:46.253414+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-28T00:23:54.261981+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-28T00:23:55.685836+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-28T00:26:08.282 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-28T00:26:08.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:07 smithi165 ceph-mon[36302]: from='client.? 172.21.15.119:0/2076121698' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-03-28T00:26:08.452 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:07 smithi132 ceph-mon[35941]: from='client.? 172.21.15.119:0/2076121698' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-03-28T00:26:08.500 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:08.493+0000 7f212e400640 1 -- 172.21.15.119:0/3699892653 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f21281038e0 msgr2=0x7f2128103d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:26:08.500 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:08.493+0000 7f212e400640 1 --2- 172.21.15.119:0/3699892653 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f21281038e0 0x7f2128103d00 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f2118009900 tx=0x7f211802bc40 comp rx=0 tx=0).stop 2024-03-28T00:26:08.500 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:08.493+0000 7f212e400640 1 -- 172.21.15.119:0/3699892653 shutdown_connections 2024-03-28T00:26:08.500 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:08.493+0000 7f212e400640 1 --2- 172.21.15.119:0/3699892653 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2128105490 0x7f212810bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:08.500 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:08.493+0000 7f212e400640 1 --2- 172.21.15.119:0/3699892653 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2128104ad0 0x7f2128104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:08.500 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:08.493+0000 7f212e400640 1 --2- 172.21.15.119:0/3699892653 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f21281038e0 0x7f2128103d00 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:08.500 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:08.493+0000 7f212e400640 1 -- 172.21.15.119:0/3699892653 >> 172.21.15.119:0/3699892653 conn(0x7f21280fee50 msgr2=0x7f2128101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:26:08.500 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:08.493+0000 7f212e400640 1 -- 172.21.15.119:0/3699892653 shutdown_connections 2024-03-28T00:26:08.500 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:08.493+0000 7f212e400640 1 -- 172.21.15.119:0/3699892653 wait complete. 2024-03-28T00:26:08.502 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:08.494+0000 7f212e400640 1 Processor -- start 2024-03-28T00:26:08.502 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:08.494+0000 7f212e400640 1 -- start start 2024-03-28T00:26:08.502 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:08.495+0000 7f212e400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f21281038e0 0x7f2128199c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:08.503 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:08.495+0000 7f212e400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2128104ad0 0x7f212819a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:08.503 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:08.495+0000 7f212e400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2128105490 0x7f21281a11d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:08.503 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:08.495+0000 7f212e400640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f212810f5a0 con 0x7f2128105490 2024-03-28T00:26:08.503 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:08.495+0000 7f212e400640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f212810f420 con 0x7f2128104ad0 2024-03-28T00:26:08.503 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:08.495+0000 7f212e400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f212810f720 con 0x7f21281038e0 2024-03-28T00:26:08.503 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:08.495+0000 7f2127400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f21281038e0 0x7f2128199c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:08.503 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:08.495+0000 7f2127400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f21281038e0 0x7f2128199c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.119:33274/0 (socket says 172.21.15.119:33274) 2024-03-28T00:26:08.503 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:08.495+0000 7f2127400640 1 -- 172.21.15.119:0/3914908740 learned_addr learned my addr 172.21.15.119:0/3914908740 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:26:08.503 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:08.495+0000 7f2126a00640 1 --2- 172.21.15.119:0/3914908740 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2128104ad0 0x7f212819a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:08.503 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:08.495+0000 7f2127e00640 1 --2- 172.21.15.119:0/3914908740 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2128105490 0x7f21281a11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:08.503 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:08.495+0000 7f2127400640 1 -- 172.21.15.119:0/3914908740 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2128104ad0 msgr2=0x7f212819a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:26:08.503 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:08.495+0000 7f2127400640 1 --2- 172.21.15.119:0/3914908740 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2128104ad0 0x7f212819a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:08.503 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:08.495+0000 7f2127400640 1 -- 172.21.15.119:0/3914908740 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2128105490 msgr2=0x7f21281a11d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:26:08.503 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:08.495+0000 7f2127400640 1 --2- 172.21.15.119:0/3914908740 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2128105490 0x7f21281a11d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:08.503 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:08.495+0000 7f2127400640 1 -- 172.21.15.119:0/3914908740 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21281a1870 con 0x7f21281038e0 2024-03-28T00:26:08.503 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:08.495+0000 7f2126a00640 1 --2- 172.21.15.119:0/3914908740 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2128104ad0 0x7f212819a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T00:26:08.504 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:08.496+0000 7f2127400640 1 --2- 172.21.15.119:0/3914908740 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f21281038e0 0x7f2128199c60 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f211802efd0 tx=0x7f211802c660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:26:08.504 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:08.496+0000 7f211fe00640 1 -- 172.21.15.119:0/3914908740 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2118041070 con 0x7f21281038e0 2024-03-28T00:26:08.504 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:08.496+0000 7f212e400640 1 -- 172.21.15.119:0/3914908740 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21281a1b00 con 0x7f21281038e0 2024-03-28T00:26:08.504 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:08.496+0000 7f211fe00640 1 -- 172.21.15.119:0/3914908740 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2118004450 con 0x7f21281038e0 2024-03-28T00:26:08.504 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:08.496+0000 7f211fe00640 1 -- 172.21.15.119:0/3914908740 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2118003e50 con 0x7f21281038e0 2024-03-28T00:26:08.504 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:08.496+0000 7f212e400640 1 -- 172.21.15.119:0/3914908740 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21281a1fc0 con 0x7f21281038e0 2024-03-28T00:26:08.504 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:08.497+0000 7f212e400640 1 -- 172.21.15.119:0/3914908740 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f212810ac80 con 0x7f21281038e0 2024-03-28T00:26:08.505 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:08.499+0000 7f211fe00640 1 -- 172.21.15.119:0/3914908740 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 18) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f2118044070 con 0x7f21281038e0 2024-03-28T00:26:08.506 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:08.500+0000 7f211fe00640 1 --2- 172.21.15.119:0/3914908740 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f210c0a29a0 0x7f210c0a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:08.507 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:08.501+0000 7f211fe00640 1 -- 172.21.15.119:0/3914908740 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f21180f0630 con 0x7f21281038e0 2024-03-28T00:26:08.507 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:08.501+0000 7f2126a00640 1 --2- 172.21.15.119:0/3914908740 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f210c0a29a0 0x7f210c0a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:08.507 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:08.501+0000 7f2126a00640 1 --2- 172.21.15.119:0/3914908740 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f210c0a29a0 0x7f210c0a4e60 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f2114009fc0 tx=0x7f2114005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:26:08.510 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:08.504+0000 7f211fe00640 1 -- 172.21.15.119:0/3914908740 <== mon.1 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f21180bd080 con 0x7f21281038e0 2024-03-28T00:26:08.709 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:08.703+0000 7f212e400640 1 -- 172.21.15.119:0/3914908740 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f21280622d0 con 0x7f21281038e0 2024-03-28T00:26:08.710 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:08.704+0000 7f211fe00640 1 -- 172.21.15.119:0/3914908740 <== mon.1 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7f21180c1120 con 0x7f21281038e0 2024-03-28T00:26:08.710 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 1 2024-03-28T00:26:08.713 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:08.706+0000 7f212e400640 1 -- 172.21.15.119:0/3914908740 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f210c0a29a0 msgr2=0x7f210c0a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:26:08.713 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:08.706+0000 7f212e400640 1 --2- 172.21.15.119:0/3914908740 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f210c0a29a0 0x7f210c0a4e60 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f2114009fc0 tx=0x7f2114005f90 comp rx=0 tx=0).stop 2024-03-28T00:26:08.713 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:08.706+0000 7f212e400640 1 -- 172.21.15.119:0/3914908740 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f21281038e0 msgr2=0x7f2128199c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:26:08.713 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:08.706+0000 7f212e400640 1 --2- 172.21.15.119:0/3914908740 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f21281038e0 0x7f2128199c60 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f211802efd0 tx=0x7f211802c660 comp rx=0 tx=0).stop 2024-03-28T00:26:08.713 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:08.706+0000 7f212e400640 1 -- 172.21.15.119:0/3914908740 shutdown_connections 2024-03-28T00:26:08.713 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:08.706+0000 7f212e400640 1 --2- 172.21.15.119:0/3914908740 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f210c0a29a0 0x7f210c0a4e60 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:08.713 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:08.706+0000 7f212e400640 1 --2- 172.21.15.119:0/3914908740 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2128105490 0x7f21281a11d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:08.713 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:08.706+0000 7f212e400640 1 --2- 172.21.15.119:0/3914908740 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2128104ad0 0x7f212819a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:08.713 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:08.706+0000 7f212e400640 1 --2- 172.21.15.119:0/3914908740 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f21281038e0 0x7f2128199c60 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:08.713 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:08.706+0000 7f212e400640 1 -- 172.21.15.119:0/3914908740 >> 172.21.15.119:0/3914908740 conn(0x7f21280fee50 msgr2=0x7f2128107e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:26:08.713 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:08.706+0000 7f212e400640 1 -- 172.21.15.119:0/3914908740 shutdown_connections 2024-03-28T00:26:08.713 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:08.706+0000 7f212e400640 1 -- 172.21.15.119:0/3914908740 wait complete. 2024-03-28T00:26:08.730 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-28T00:26:08.939 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:08.933+0000 7f8f5d400640 1 -- 172.21.15.119:0/2717703217 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8f580772a0 msgr2=0x7f8f58077700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:26:08.940 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:08.933+0000 7f8f5d400640 1 --2- 172.21.15.119:0/2717703217 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8f580772a0 0x7f8f58077700 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f8f44009900 tx=0x7f8f4402bc30 comp rx=0 tx=0).stop 2024-03-28T00:26:08.940 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:08.933+0000 7f8f5d400640 1 -- 172.21.15.119:0/2717703217 shutdown_connections 2024-03-28T00:26:08.940 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:08.933+0000 7f8f5d400640 1 --2- 172.21.15.119:0/2717703217 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8f58077c40 0x7f8f580784e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:08.940 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:08.933+0000 7f8f5d400640 1 --2- 172.21.15.119:0/2717703217 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8f580772a0 0x7f8f58077700 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:08.940 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:08.933+0000 7f8f5d400640 1 --2- 172.21.15.119:0/2717703217 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8f58076060 0x7f8f58076480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:08.940 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:08.933+0000 7f8f5d400640 1 -- 172.21.15.119:0/2717703217 >> 172.21.15.119:0/2717703217 conn(0x7f8f580fee50 msgr2=0x7f8f58101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:26:08.940 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:08.933+0000 7f8f5d400640 1 -- 172.21.15.119:0/2717703217 shutdown_connections 2024-03-28T00:26:08.940 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:08.933+0000 7f8f5d400640 1 -- 172.21.15.119:0/2717703217 wait complete. 2024-03-28T00:26:08.942 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:08.934+0000 7f8f5d400640 1 Processor -- start 2024-03-28T00:26:08.942 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:08.934+0000 7f8f5d400640 1 -- start start 2024-03-28T00:26:08.942 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:08.935+0000 7f8f5d400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8f58076060 0x7f8f5819dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:08.942 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:08.935+0000 7f8f5d400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8f580772a0 0x7f8f5819e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:08.942 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:08.935+0000 7f8f5d400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8f58077c40 0x7f8f581a5590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:08.943 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:08.935+0000 7f8f5d400640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f581139b0 con 0x7f8f580772a0 2024-03-28T00:26:08.943 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:08.935+0000 7f8f5d400640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f58113830 con 0x7f8f58077c40 2024-03-28T00:26:08.943 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:08.935+0000 7f8f5d400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f58113b30 con 0x7f8f58076060 2024-03-28T00:26:08.943 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:08.935+0000 7f8f56800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8f58076060 0x7f8f5819dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:08.943 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:08.935+0000 7f8f57200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8f58077c40 0x7f8f581a5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:08.943 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:08.935+0000 7f8f56800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8f58076060 0x7f8f5819dfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.119:33290/0 (socket says 172.21.15.119:33290) 2024-03-28T00:26:08.943 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:08.935+0000 7f8f56800640 1 -- 172.21.15.119:0/715732245 learned_addr learned my addr 172.21.15.119:0/715732245 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:26:08.943 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:08.935+0000 7f8f55e00640 1 --2- 172.21.15.119:0/715732245 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8f580772a0 0x7f8f5819e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:08.943 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:08.935+0000 7f8f57200640 1 -- 172.21.15.119:0/715732245 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8f58076060 msgr2=0x7f8f5819dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:26:08.943 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:08.935+0000 7f8f57200640 1 --2- 172.21.15.119:0/715732245 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8f58076060 0x7f8f5819dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:08.943 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:08.935+0000 7f8f57200640 1 -- 172.21.15.119:0/715732245 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8f580772a0 msgr2=0x7f8f5819e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:26:08.943 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:08.935+0000 7f8f57200640 1 --2- 172.21.15.119:0/715732245 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8f580772a0 0x7f8f5819e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:08.943 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:08.935+0000 7f8f57200640 1 -- 172.21.15.119:0/715732245 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f581a5c00 con 0x7f8f58077c40 2024-03-28T00:26:08.943 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:08.936+0000 7f8f57200640 1 --2- 172.21.15.119:0/715732245 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8f58077c40 0x7f8f581a5590 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f8f50009fd0 tx=0x7f8f5000ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:26:08.944 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:08.936+0000 7f8f4f400640 1 -- 172.21.15.119:0/715732245 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f5000ce20 con 0x7f8f58077c40 2024-03-28T00:26:08.944 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:08.936+0000 7f8f4f400640 1 -- 172.21.15.119:0/715732245 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8f50005020 con 0x7f8f58077c40 2024-03-28T00:26:08.944 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:08.936+0000 7f8f4f400640 1 -- 172.21.15.119:0/715732245 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f50010720 con 0x7f8f58077c40 2024-03-28T00:26:08.944 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:08.936+0000 7f8f5d400640 1 -- 172.21.15.119:0/715732245 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f581a5ef0 con 0x7f8f58077c40 2024-03-28T00:26:08.944 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:08.936+0000 7f8f5d400640 1 -- 172.21.15.119:0/715732245 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f581a63b0 con 0x7f8f58077c40 2024-03-28T00:26:08.944 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:08.937+0000 7f8f5d400640 1 -- 172.21.15.119:0/715732245 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f58110250 con 0x7f8f58077c40 2024-03-28T00:26:08.945 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:08.939+0000 7f8f4f400640 1 -- 172.21.15.119:0/715732245 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 18) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f8f50005540 con 0x7f8f58077c40 2024-03-28T00:26:08.946 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:08.940+0000 7f8f4f400640 1 --2- 172.21.15.119:0/715732245 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f8f3c0a2800 0x7f8f3c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:08.947 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:08.940+0000 7f8f4f400640 1 -- 172.21.15.119:0/715732245 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f8f500cf060 con 0x7f8f58077c40 2024-03-28T00:26:08.947 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:08.941+0000 7f8f56800640 1 --2- 172.21.15.119:0/715732245 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f8f3c0a2800 0x7f8f3c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:08.947 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:08.941+0000 7f8f56800640 1 --2- 172.21.15.119:0/715732245 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f8f3c0a2800 0x7f8f3c0a4cc0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f8f48009c90 tx=0x7f8f48005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:26:08.953 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:08.947+0000 7f8f4f400640 1 -- 172.21.15.119:0/715732245 <== mon.2 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f8f5009c100 con 0x7f8f58077c40 2024-03-28T00:26:09.152 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:09.145+0000 7f8f5d400640 1 -- 172.21.15.119:0/715732245 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8f5802d070 con 0x7f8f58077c40 2024-03-28T00:26:09.152 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:09.146+0000 7f8f4f400640 1 -- 172.21.15.119:0/715732245 <== mon.2 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8f5009ba00 con 0x7f8f58077c40 2024-03-28T00:26:09.153 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 32 2024-03-28T00:26:09.155 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:09.149+0000 7f8f5d400640 1 -- 172.21.15.119:0/715732245 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f8f3c0a2800 msgr2=0x7f8f3c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:26:09.155 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:09.149+0000 7f8f5d400640 1 --2- 172.21.15.119:0/715732245 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f8f3c0a2800 0x7f8f3c0a4cc0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f8f48009c90 tx=0x7f8f48005f90 comp rx=0 tx=0).stop 2024-03-28T00:26:09.155 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:09.149+0000 7f8f5d400640 1 -- 172.21.15.119:0/715732245 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8f58077c40 msgr2=0x7f8f581a5590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:26:09.155 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:09.149+0000 7f8f5d400640 1 --2- 172.21.15.119:0/715732245 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8f58077c40 0x7f8f581a5590 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f8f50009fd0 tx=0x7f8f5000ef10 comp rx=0 tx=0).stop 2024-03-28T00:26:09.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:08 smithi119 ceph-mon[31518]: pgmap v379: 97 pgs: 97 active+clean; 648 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-28T00:26:09.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:08 smithi119 ceph-mon[31518]: from='client.? 172.21.15.119:0/2266502058' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-28T00:26:09.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:08 smithi119 ceph-mon[31518]: from='client.? 172.21.15.119:0/3914908740' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-28T00:26:09.156 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:09.149+0000 7f8f5d400640 1 -- 172.21.15.119:0/715732245 shutdown_connections 2024-03-28T00:26:09.156 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:09.149+0000 7f8f5d400640 1 --2- 172.21.15.119:0/715732245 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f8f3c0a2800 0x7f8f3c0a4cc0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:09.156 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:09.149+0000 7f8f5d400640 1 --2- 172.21.15.119:0/715732245 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8f58077c40 0x7f8f581a5590 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:09.156 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:09.149+0000 7f8f5d400640 1 --2- 172.21.15.119:0/715732245 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8f580772a0 0x7f8f5819e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:09.157 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:09.149+0000 7f8f5d400640 1 --2- 172.21.15.119:0/715732245 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8f58076060 0x7f8f5819dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:09.157 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:09.149+0000 7f8f5d400640 1 -- 172.21.15.119:0/715732245 >> 172.21.15.119:0/715732245 conn(0x7f8f580fee50 msgr2=0x7f8f58100950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:26:09.157 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:09.149+0000 7f8f5d400640 1 -- 172.21.15.119:0/715732245 shutdown_connections 2024-03-28T00:26:09.157 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:09.149+0000 7f8f5d400640 1 -- 172.21.15.119:0/715732245 wait complete. 2024-03-28T00:26:09.175 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-28T00:26:09.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:08 smithi165 ceph-mon[36302]: pgmap v379: 97 pgs: 97 active+clean; 648 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-28T00:26:09.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:08 smithi165 ceph-mon[36302]: from='client.? 172.21.15.119:0/2266502058' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-28T00:26:09.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:08 smithi165 ceph-mon[36302]: from='client.? 172.21.15.119:0/3914908740' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-28T00:26:09.387 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:09.381+0000 7fb1a3200640 1 -- 172.21.15.119:0/1466015031 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb1a4108db0 msgr2=0x7fb1a4109230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:26:09.388 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:09.381+0000 7fb1a3200640 1 --2- 172.21.15.119:0/1466015031 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb1a4108db0 0x7fb1a4109230 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb18c009860 tx=0x7fb18c02bb70 comp rx=0 tx=0).stop 2024-03-28T00:26:09.388 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:09.381+0000 7fb1a3200640 1 -- 172.21.15.119:0/1466015031 shutdown_connections 2024-03-28T00:26:09.388 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:09.381+0000 7fb1a3200640 1 --2- 172.21.15.119:0/1466015031 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb1a4109770 0x7fb1a4110000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:09.388 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:09.381+0000 7fb1a3200640 1 --2- 172.21.15.119:0/1466015031 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb1a4108db0 0x7fb1a4109230 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:09.388 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:09.381+0000 7fb1a3200640 1 --2- 172.21.15.119:0/1466015031 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb1a4107bc0 0x7fb1a4107fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:09.388 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:09.381+0000 7fb1a3200640 1 -- 172.21.15.119:0/1466015031 >> 172.21.15.119:0/1466015031 conn(0x7fb1a4076b70 msgr2=0x7fb1a4078f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:26:09.388 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:09.381+0000 7fb1a3200640 1 -- 172.21.15.119:0/1466015031 shutdown_connections 2024-03-28T00:26:09.388 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:09.381+0000 7fb1a3200640 1 -- 172.21.15.119:0/1466015031 wait complete. 2024-03-28T00:26:09.390 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:09.382+0000 7fb1a3200640 1 Processor -- start 2024-03-28T00:26:09.390 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:09.382+0000 7fb1a3200640 1 -- start start 2024-03-28T00:26:09.390 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:09.383+0000 7fb1a3200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb1a4107bc0 0x7fb1a419df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:09.390 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:09.383+0000 7fb1a3200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb1a4108db0 0x7fb1a419e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:09.390 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:09.383+0000 7fb1a3200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb1a4109770 0x7fb1a41a5420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:09.390 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:09.383+0000 7fb1a3200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1a4113840 con 0x7fb1a4109770 2024-03-28T00:26:09.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:09.383+0000 7fb1a3200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1a41136c0 con 0x7fb1a4108db0 2024-03-28T00:26:09.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:09.383+0000 7fb1a3200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1a41139c0 con 0x7fb1a4107bc0 2024-03-28T00:26:09.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:09.383+0000 7fb1a1e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb1a4107bc0 0x7fb1a419df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:09.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:09.383+0000 7fb1a1e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb1a4107bc0 0x7fb1a419df10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.119:33306/0 (socket says 172.21.15.119:33306) 2024-03-28T00:26:09.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:09.383+0000 7fb1a1e00640 1 -- 172.21.15.119:0/1089712227 learned_addr learned my addr 172.21.15.119:0/1089712227 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:26:09.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:09.383+0000 7fb199400640 1 --2- 172.21.15.119:0/1089712227 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb1a4108db0 0x7fb1a419e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:09.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:09.383+0000 7fb1a2800640 1 --2- 172.21.15.119:0/1089712227 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb1a4109770 0x7fb1a41a5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:09.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:09.383+0000 7fb199400640 1 -- 172.21.15.119:0/1089712227 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb1a4107bc0 msgr2=0x7fb1a419df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:26:09.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:09.383+0000 7fb199400640 1 --2- 172.21.15.119:0/1089712227 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb1a4107bc0 0x7fb1a419df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:09.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:09.383+0000 7fb199400640 1 -- 172.21.15.119:0/1089712227 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb1a4109770 msgr2=0x7fb1a41a5420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:26:09.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:09.383+0000 7fb199400640 1 --2- 172.21.15.119:0/1089712227 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb1a4109770 0x7fb1a41a5420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:09.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:09.383+0000 7fb199400640 1 -- 172.21.15.119:0/1089712227 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1a41a5a90 con 0x7fb1a4108db0 2024-03-28T00:26:09.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:09.383+0000 7fb1a1e00640 1 --2- 172.21.15.119:0/1089712227 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb1a4107bc0 0x7fb1a419df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T00:26:09.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:09.384+0000 7fb199400640 1 --2- 172.21.15.119:0/1089712227 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb1a4108db0 0x7fb1a419e450 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fb18c02c0b0 tx=0x7fb18c032430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:26:09.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:09.384+0000 7fb19b400640 1 -- 172.21.15.119:0/1089712227 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb18c041020 con 0x7fb1a4108db0 2024-03-28T00:26:09.392 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:09.384+0000 7fb19b400640 1 -- 172.21.15.119:0/1089712227 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb18c02c9f0 con 0x7fb1a4108db0 2024-03-28T00:26:09.392 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:09.384+0000 7fb19b400640 1 -- 172.21.15.119:0/1089712227 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb18c039040 con 0x7fb1a4108db0 2024-03-28T00:26:09.392 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:09.384+0000 7fb1a3200640 1 -- 172.21.15.119:0/1089712227 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1a41a5d20 con 0x7fb1a4108db0 2024-03-28T00:26:09.392 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:09.384+0000 7fb1a3200640 1 -- 172.21.15.119:0/1089712227 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1a41a6190 con 0x7fb1a4108db0 2024-03-28T00:26:09.392 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:09.385+0000 7fb1a3200640 1 -- 172.21.15.119:0/1089712227 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1a410eee0 con 0x7fb1a4108db0 2024-03-28T00:26:09.393 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:09.387+0000 7fb19b400640 1 -- 172.21.15.119:0/1089712227 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 18) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fb18c044020 con 0x7fb1a4108db0 2024-03-28T00:26:09.394 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:09.388+0000 7fb19b400640 1 --2- 172.21.15.119:0/1089712227 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fb1880a2800 0x7fb1880a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:09.394 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:09.388+0000 7fb19b400640 1 -- 172.21.15.119:0/1089712227 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fb18c0f0240 con 0x7fb1a4108db0 2024-03-28T00:26:09.394 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:09.388+0000 7fb1a1e00640 1 --2- 172.21.15.119:0/1089712227 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fb1880a2800 0x7fb1880a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:09.395 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:09.389+0000 7fb1a1e00640 1 --2- 172.21.15.119:0/1089712227 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fb1880a2800 0x7fb1880a4cc0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fb194011550 tx=0x7fb194009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:26:09.398 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:09.392+0000 7fb19b400640 1 -- 172.21.15.119:0/1089712227 <== mon.2 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fb18c04b020 con 0x7fb1a4108db0 2024-03-28T00:26:09.452 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:08 smithi132 ceph-mon[35941]: pgmap v379: 97 pgs: 97 active+clean; 648 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-28T00:26:09.452 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:08 smithi132 ceph-mon[35941]: from='client.? 172.21.15.119:0/2266502058' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-28T00:26:09.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:08 smithi132 ceph-mon[35941]: from='client.? 172.21.15.119:0/3914908740' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-28T00:26:09.597 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:09.590+0000 7fb1a3200640 1 -- 172.21.15.119:0/1089712227 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb1a40622a0 con 0x7fb1a4108db0 2024-03-28T00:26:09.597 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:09.591+0000 7fb19b400640 1 -- 172.21.15.119:0/1089712227 <== mon.2 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb18c0bd2e0 con 0x7fb1a4108db0 2024-03-28T00:26:09.597 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 64 2024-03-28T00:26:09.600 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:09.593+0000 7fb1a3200640 1 -- 172.21.15.119:0/1089712227 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fb1880a2800 msgr2=0x7fb1880a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:26:09.600 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:09.593+0000 7fb1a3200640 1 --2- 172.21.15.119:0/1089712227 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fb1880a2800 0x7fb1880a4cc0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fb194011550 tx=0x7fb194009260 comp rx=0 tx=0).stop 2024-03-28T00:26:09.600 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:09.593+0000 7fb1a3200640 1 -- 172.21.15.119:0/1089712227 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb1a4108db0 msgr2=0x7fb1a419e450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:26:09.601 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:09.593+0000 7fb1a3200640 1 --2- 172.21.15.119:0/1089712227 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb1a4108db0 0x7fb1a419e450 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fb18c02c0b0 tx=0x7fb18c032430 comp rx=0 tx=0).stop 2024-03-28T00:26:09.601 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:09.594+0000 7fb1a3200640 1 -- 172.21.15.119:0/1089712227 shutdown_connections 2024-03-28T00:26:09.601 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:09.594+0000 7fb1a3200640 1 --2- 172.21.15.119:0/1089712227 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fb1880a2800 0x7fb1880a4cc0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:09.601 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:09.594+0000 7fb1a3200640 1 --2- 172.21.15.119:0/1089712227 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb1a4109770 0x7fb1a41a5420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:09.601 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:09.594+0000 7fb1a3200640 1 --2- 172.21.15.119:0/1089712227 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb1a4108db0 0x7fb1a419e450 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:09.601 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:09.594+0000 7fb1a3200640 1 --2- 172.21.15.119:0/1089712227 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb1a4107bc0 0x7fb1a419df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:09.601 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:09.594+0000 7fb1a3200640 1 -- 172.21.15.119:0/1089712227 >> 172.21.15.119:0/1089712227 conn(0x7fb1a4076b70 msgr2=0x7fb1a410c0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:26:09.601 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:09.594+0000 7fb1a3200640 1 -- 172.21.15.119:0/1089712227 shutdown_connections 2024-03-28T00:26:09.601 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:09.594+0000 7fb1a3200640 1 -- 172.21.15.119:0/1089712227 wait complete. 2024-03-28T00:26:09.619 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T00:26:09.824 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:09.818+0000 7fa724a00640 1 -- 172.21.15.119:0/32856661 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa7201036f0 msgr2=0x7fa720103b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:26:09.824 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:09.818+0000 7fa724a00640 1 --2- 172.21.15.119:0/32856661 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa7201036f0 0x7fa720103b10 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa710009900 tx=0x7fa71002bc50 comp rx=0 tx=0).stop 2024-03-28T00:26:09.824 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:09.818+0000 7fa724a00640 1 -- 172.21.15.119:0/32856661 shutdown_connections 2024-03-28T00:26:09.824 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:09.818+0000 7fa724a00640 1 --2- 172.21.15.119:0/32856661 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa7201052a0 0x7fa72010bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:09.825 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:09.818+0000 7fa724a00640 1 --2- 172.21.15.119:0/32856661 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa7201048e0 0x7fa720104d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:09.825 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:09.818+0000 7fa724a00640 1 --2- 172.21.15.119:0/32856661 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa7201036f0 0x7fa720103b10 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:09.825 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:09.818+0000 7fa724a00640 1 -- 172.21.15.119:0/32856661 >> 172.21.15.119:0/32856661 conn(0x7fa7200fec80 msgr2=0x7fa7201010a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:26:09.825 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:09.818+0000 7fa724a00640 1 -- 172.21.15.119:0/32856661 shutdown_connections 2024-03-28T00:26:09.825 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:09.818+0000 7fa724a00640 1 -- 172.21.15.119:0/32856661 wait complete. 2024-03-28T00:26:09.826 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:09.819+0000 7fa724a00640 1 Processor -- start 2024-03-28T00:26:09.826 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:09.819+0000 7fa724a00640 1 -- start start 2024-03-28T00:26:09.826 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:09.819+0000 7fa724a00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa7201036f0 0x7fa7201999d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:09.826 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:09.819+0000 7fa724a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa7201048e0 0x7fa720199f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:09.826 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:09.819+0000 7fa724a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa7201052a0 0x7fa7201a0f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:09.826 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:09.820+0000 7fa724a00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa72010f390 con 0x7fa7201052a0 2024-03-28T00:26:09.826 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:09.820+0000 7fa724a00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa72010f210 con 0x7fa7201036f0 2024-03-28T00:26:09.826 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:09.820+0000 7fa724a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa72010f510 con 0x7fa7201048e0 2024-03-28T00:26:09.826 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:09.820+0000 7fa71f400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa7201036f0 0x7fa7201999d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:09.827 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:09.820+0000 7fa71f400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa7201036f0 0x7fa7201999d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.119:43986/0 (socket says 172.21.15.119:43986) 2024-03-28T00:26:09.827 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:09.820+0000 7fa71f400640 1 -- 172.21.15.119:0/3027838862 learned_addr learned my addr 172.21.15.119:0/3027838862 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:26:09.827 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:09.820+0000 7fa71ea00640 1 --2- 172.21.15.119:0/3027838862 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa7201048e0 0x7fa720199f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:09.827 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:09.820+0000 7fa71fe00640 1 --2- 172.21.15.119:0/3027838862 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa7201052a0 0x7fa7201a0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:09.828 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:09.820+0000 7fa71f400640 1 -- 172.21.15.119:0/3027838862 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa7201048e0 msgr2=0x7fa720199f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:26:09.828 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:09.820+0000 7fa71f400640 1 --2- 172.21.15.119:0/3027838862 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa7201048e0 0x7fa720199f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:09.828 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:09.820+0000 7fa71f400640 1 -- 172.21.15.119:0/3027838862 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa7201052a0 msgr2=0x7fa7201a0f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:26:09.828 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:09.820+0000 7fa71f400640 1 --2- 172.21.15.119:0/3027838862 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa7201052a0 0x7fa7201a0f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:09.828 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:09.820+0000 7fa71f400640 1 -- 172.21.15.119:0/3027838862 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7201a15f0 con 0x7fa7201036f0 2024-03-28T00:26:09.828 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:09.820+0000 7fa71f400640 1 --2- 172.21.15.119:0/3027838862 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa7201036f0 0x7fa7201999d0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fa710005ab0 tx=0x7fa71002c6e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:26:09.828 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:09.821+0000 7fa717e00640 1 -- 172.21.15.119:0/3027838862 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa710041070 con 0x7fa7201036f0 2024-03-28T00:26:09.828 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:09.821+0000 7fa717e00640 1 -- 172.21.15.119:0/3027838862 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa710004450 con 0x7fa7201036f0 2024-03-28T00:26:09.828 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:09.821+0000 7fa724a00640 1 -- 172.21.15.119:0/3027838862 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7201a1880 con 0x7fa7201036f0 2024-03-28T00:26:09.828 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:09.821+0000 7fa717e00640 1 -- 172.21.15.119:0/3027838862 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa710034440 con 0x7fa7201036f0 2024-03-28T00:26:09.828 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:09.821+0000 7fa724a00640 1 -- 172.21.15.119:0/3027838862 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7201a1e20 con 0x7fa7201036f0 2024-03-28T00:26:09.829 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:09.822+0000 7fa724a00640 1 -- 172.21.15.119:0/3027838862 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa72010aa10 con 0x7fa7201036f0 2024-03-28T00:26:09.830 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:09.824+0000 7fa717e00640 1 -- 172.21.15.119:0/3027838862 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 18) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fa710044070 con 0x7fa7201036f0 2024-03-28T00:26:09.831 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:09.825+0000 7fa717e00640 1 --2- 172.21.15.119:0/3027838862 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fa7040a2880 0x7fa7040a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:09.832 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:09.825+0000 7fa717e00640 1 -- 172.21.15.119:0/3027838862 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fa7100f0570 con 0x7fa7201036f0 2024-03-28T00:26:09.832 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:09.825+0000 7fa71ea00640 1 --2- 172.21.15.119:0/3027838862 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fa7040a2880 0x7fa7040a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:09.832 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:09.826+0000 7fa71ea00640 1 --2- 172.21.15.119:0/3027838862 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fa7040a2880 0x7fa7040a4d40 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fa72019aee0 tx=0x7fa70c009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:26:09.836 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:09.829+0000 7fa717e00640 1 -- 172.21.15.119:0/3027838862 <== mon.2 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa7100bcfc0 con 0x7fa7201036f0 2024-03-28T00:26:10.078 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:10.072+0000 7fa724a00640 1 -- 172.21.15.119:0/3027838862 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa72002d190 con 0x7fa7201036f0 2024-03-28T00:26:10.079 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:10.073+0000 7fa717e00640 1 -- 172.21.15.119:0/3027838862 <== mon.2 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+11473 (secure 0 0 0) 0x7fa7100c1060 con 0x7fa7201036f0 2024-03-28T00:26:10.080 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-28T00:26:10.080 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14400,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/3988635716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":3988635716},{"type":"v1","addr":"172.21.15.119:6837","nonce":3988635716}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6839/1595343240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1595343240},{"type":"v1","addr":"172.21.15.119:6839","nonce":1595343240}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-28T00:23:56.215355+0000","modified":"2024-03-28T00:24:04.459985+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24581,"mds_1":24550,"mds_2":24530,"mds_3":24523,"mds_4":24511},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14466":{"gid":14466,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6841/1678451188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6840","nonce":1678451188},{"type":"v1","addr":"172.21.15.119:6841","nonce":1678451188}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24445":{"gid":24445,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/609963168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":609963168},{"type":"v1","addr":"172.21.15.165:6833","nonce":609963168}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24455":{"gid":24455,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.132:6833/1179016670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":1179016670},{"type":"v1","addr":"172.21.15.132:6833","nonce":1179016670}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24473":{"gid":24473,"name":"j","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6835/744950487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":744950487},{"type":"v1","addr":"172.21.15.119:6835","nonce":744950487}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24500":{"gid":24500,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.132:6835/356981946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":356981946},{"type":"v1","addr":"172.21.15.132:6835","nonce":356981946}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24511":{"gid":24511,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.165:6835/1947980102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":1947980102},{"type":"v1","addr":"172.21.15.165:6835","nonce":1947980102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24523":{"gid":24523,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.165:6837/1531193032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":1531193032},{"type":"v1","addr":"172.21.15.165:6837","nonce":1531193032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24530":{"gid":24530,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.132:6837/3383082967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6836","nonce":3383082967},{"type":"v1","addr":"172.21.15.132:6837","nonce":3383082967}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24550":{"gid":24550,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.165:6839/3004684592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":3004684592},{"type":"v1","addr":"172.21.15.165:6839","nonce":3004684592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.132:6839/3305822525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6838","nonce":3305822525},{"type":"v1","addr":"172.21.15.132:6839","nonce":3305822525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24523,24530,24550,24511,24581]},"id":1}]} 2024-03-28T00:26:10.080 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 26 2024-03-28T00:26:10.082 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:10.076+0000 7fa724a00640 1 -- 172.21.15.119:0/3027838862 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fa7040a2880 msgr2=0x7fa7040a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:26:10.082 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:10.076+0000 7fa724a00640 1 --2- 172.21.15.119:0/3027838862 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fa7040a2880 0x7fa7040a4d40 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fa72019aee0 tx=0x7fa70c009500 comp rx=0 tx=0).stop 2024-03-28T00:26:10.082 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:10.076+0000 7fa724a00640 1 -- 172.21.15.119:0/3027838862 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa7201036f0 msgr2=0x7fa7201999d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:26:10.083 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:10.076+0000 7fa724a00640 1 --2- 172.21.15.119:0/3027838862 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa7201036f0 0x7fa7201999d0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fa710005ab0 tx=0x7fa71002c6e0 comp rx=0 tx=0).stop 2024-03-28T00:26:10.083 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:10.076+0000 7fa724a00640 1 -- 172.21.15.119:0/3027838862 shutdown_connections 2024-03-28T00:26:10.083 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:10.076+0000 7fa724a00640 1 --2- 172.21.15.119:0/3027838862 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fa7040a2880 0x7fa7040a4d40 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:10.083 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:10.076+0000 7fa724a00640 1 --2- 172.21.15.119:0/3027838862 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa7201052a0 0x7fa7201a0f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:10.083 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:10.076+0000 7fa724a00640 1 --2- 172.21.15.119:0/3027838862 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa7201048e0 0x7fa720199f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:10.083 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:10.076+0000 7fa724a00640 1 --2- 172.21.15.119:0/3027838862 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa7201036f0 0x7fa7201999d0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:10.083 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:10.076+0000 7fa724a00640 1 -- 172.21.15.119:0/3027838862 >> 172.21.15.119:0/3027838862 conn(0x7fa7200fec80 msgr2=0x7fa720107be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:26:10.083 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:10.076+0000 7fa724a00640 1 -- 172.21.15.119:0/3027838862 shutdown_connections 2024-03-28T00:26:10.083 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:10.076+0000 7fa724a00640 1 -- 172.21.15.119:0/3027838862 wait complete. 2024-03-28T00:26:10.103 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T00:26:10.316 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:10.310+0000 7f7e31600640 1 -- 172.21.15.119:0/608840573 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7e2c0ffde0 msgr2=0x7f7e2c100200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:26:10.316 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:10.310+0000 7f7e31600640 1 --2- 172.21.15.119:0/608840573 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7e2c0ffde0 0x7f7e2c100200 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f7e24009900 tx=0x7f7e2402bcc0 comp rx=0 tx=0).stop 2024-03-28T00:26:10.316 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:10.310+0000 7f7e31600640 1 -- 172.21.15.119:0/608840573 shutdown_connections 2024-03-28T00:26:10.316 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:10.310+0000 7f7e31600640 1 --2- 172.21.15.119:0/608840573 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e2c1010e0 0x7f7e2c101560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:10.316 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:10.310+0000 7f7e31600640 1 --2- 172.21.15.119:0/608840573 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7e2c100740 0x7f7e2c100ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:10.316 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:10.310+0000 7f7e31600640 1 --2- 172.21.15.119:0/608840573 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7e2c0ffde0 0x7f7e2c100200 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:10.317 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:10.310+0000 7f7e31600640 1 -- 172.21.15.119:0/608840573 >> 172.21.15.119:0/608840573 conn(0x7f7e2c0fb980 msgr2=0x7f7e2c0fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:26:10.317 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:10.310+0000 7f7e31600640 1 -- 172.21.15.119:0/608840573 shutdown_connections 2024-03-28T00:26:10.317 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:10.310+0000 7f7e31600640 1 -- 172.21.15.119:0/608840573 wait complete. 2024-03-28T00:26:10.319 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:10.311+0000 7f7e31600640 1 Processor -- start 2024-03-28T00:26:10.319 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:10.311+0000 7f7e31600640 1 -- start start 2024-03-28T00:26:10.319 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:10.311+0000 7f7e31600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e2c0ffde0 0x7f7e2c1999e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:10.319 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:10.311+0000 7f7e31600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7e2c100740 0x7f7e2c199f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:10.319 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:10.311+0000 7f7e31600640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7e2c1010e0 0x7f7e2c1a0f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:10.319 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:10.311+0000 7f7e31600640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e2c10f340 con 0x7f7e2c0ffde0 2024-03-28T00:26:10.319 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:10.311+0000 7f7e31600640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e2c10f1c0 con 0x7f7e2c1010e0 2024-03-28T00:26:10.319 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:10.311+0000 7f7e31600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e2c10f4c0 con 0x7f7e2c100740 2024-03-28T00:26:10.319 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:10.311+0000 7f7e30c00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7e2c1010e0 0x7f7e2c1a0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:10.319 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:10.311+0000 7f7e2be00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e2c0ffde0 0x7f7e2c1999e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:10.320 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:10.311+0000 7f7e2b400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7e2c100740 0x7f7e2c199f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:10.320 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:10.311+0000 7f7e2b400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7e2c100740 0x7f7e2c199f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.119:33340/0 (socket says 172.21.15.119:33340) 2024-03-28T00:26:10.320 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:10.311+0000 7f7e30c00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7e2c1010e0 0x7f7e2c1a0f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.119:43996/0 (socket says 172.21.15.119:43996) 2024-03-28T00:26:10.320 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:10.311+0000 7f7e2b400640 1 -- 172.21.15.119:0/3755088055 learned_addr learned my addr 172.21.15.119:0/3755088055 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:26:10.320 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:10.312+0000 7f7e2b400640 1 -- 172.21.15.119:0/3755088055 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7e2c1010e0 msgr2=0x7f7e2c1a0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:26:10.320 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:10.312+0000 7f7e2b400640 1 --2- 172.21.15.119:0/3755088055 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7e2c1010e0 0x7f7e2c1a0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:10.320 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:10.312+0000 7f7e2b400640 1 -- 172.21.15.119:0/3755088055 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e2c0ffde0 msgr2=0x7f7e2c1999e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:26:10.320 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:10.312+0000 7f7e2b400640 1 --2- 172.21.15.119:0/3755088055 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e2c0ffde0 0x7f7e2c1999e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:10.320 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:10.312+0000 7f7e2b400640 1 -- 172.21.15.119:0/3755088055 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e2c1a1600 con 0x7f7e2c100740 2024-03-28T00:26:10.320 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:10.312+0000 7f7e2b400640 1 --2- 172.21.15.119:0/3755088055 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7e2c100740 0x7f7e2c199f20 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f7e14009d50 tx=0x7f7e1400c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:26:10.320 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:10.312+0000 7f7e28c00640 1 -- 172.21.15.119:0/3755088055 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e14003b70 con 0x7f7e2c100740 2024-03-28T00:26:10.320 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:10.312+0000 7f7e28c00640 1 -- 172.21.15.119:0/3755088055 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7e14003d10 con 0x7f7e2c100740 2024-03-28T00:26:10.320 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:10.312+0000 7f7e31600640 1 -- 172.21.15.119:0/3755088055 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e2c1a18f0 con 0x7f7e2c100740 2024-03-28T00:26:10.320 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:10.312+0000 7f7e28c00640 1 -- 172.21.15.119:0/3755088055 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e140108f0 con 0x7f7e2c100740 2024-03-28T00:26:10.320 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:10.313+0000 7f7e31600640 1 -- 172.21.15.119:0/3755088055 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e2c1a1e90 con 0x7f7e2c100740 2024-03-28T00:26:10.321 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:10.314+0000 7f7e31600640 1 -- 172.21.15.119:0/3755088055 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e2c10ab30 con 0x7f7e2c100740 2024-03-28T00:26:10.321 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:10.315+0000 7f7e28c00640 1 -- 172.21.15.119:0/3755088055 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 18) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f7e14004000 con 0x7f7e2c100740 2024-03-28T00:26:10.323 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:10.317+0000 7f7e28c00640 1 --2- 172.21.15.119:0/3755088055 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f7e100a2880 0x7f7e100a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:10.323 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:10.317+0000 7f7e28c00640 1 -- 172.21.15.119:0/3755088055 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f7e140cefd0 con 0x7f7e2c100740 2024-03-28T00:26:10.323 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:10.317+0000 7f7e2be00640 1 --2- 172.21.15.119:0/3755088055 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f7e100a2880 0x7f7e100a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:10.324 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:10.318+0000 7f7e2be00640 1 --2- 172.21.15.119:0/3755088055 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f7e100a2880 0x7f7e100a4d40 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f7e24005ab0 tx=0x7f7e2402df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:26:10.327 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:10.321+0000 7f7e28c00640 1 -- 172.21.15.119:0/3755088055 <== mon.1 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f7e1409c070 con 0x7f7e2c100740 2024-03-28T00:26:10.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:09 smithi165 ceph-mon[36302]: from='client.? 172.21.15.119:0/715732245' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-28T00:26:10.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:09 smithi165 ceph-mon[36302]: from='client.? 172.21.15.119:0/1089712227' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-28T00:26:10.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:09 smithi119 ceph-mon[31518]: from='client.? 172.21.15.119:0/715732245' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-28T00:26:10.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:09 smithi119 ceph-mon[31518]: from='client.? 172.21.15.119:0/1089712227' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-28T00:26:10.452 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:09 smithi132 ceph-mon[35941]: from='client.? 172.21.15.119:0/715732245' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-28T00:26:10.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:09 smithi132 ceph-mon[35941]: from='client.? 172.21.15.119:0/1089712227' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-28T00:26:10.526 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:10.520+0000 7f7e31600640 1 -- 172.21.15.119:0/3755088055 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7e2c0622a0 con 0x7f7e2c100740 2024-03-28T00:26:10.529 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:10.523+0000 7f7e28c00640 1 -- 172.21.15.119:0/3755088055 <== mon.1 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+22638 (secure 0 0 0) 0x7f7e1409b970 con 0x7f7e2c100740 2024-03-28T00:26:10.529 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-28T00:26:10.529 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":87,"fsid":"10f73b92-ec98-11ee-95d0-87774f69a715","created":"2024-03-28T00:14:57.563604+0000","modified":"2024-03-28T00:24:05.529756+0000","last_up_change":"2024-03-28T00:22:58.821662+0000","last_in_change":"2024-03-28T00:22:38.467270+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T00:18:46.253414+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-28T00:23:54.261981+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-28T00:23:55.685836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c0b875d-a01e-4294-9c9c-5d0b048b8912","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":580396252},{"type":"v1","addr":"172.21.15.119:6803","nonce":580396252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":580396252},{"type":"v1","addr":"172.21.15.119:6805","nonce":580396252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":580396252},{"type":"v1","addr":"172.21.15.119:6809","nonce":580396252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":580396252},{"type":"v1","addr":"172.21.15.119:6807","nonce":580396252}]},"public_addr":"172.21.15.119:6803/580396252","cluster_addr":"172.21.15.119:6805/580396252","heartbeat_back_addr":"172.21.15.119:6809/580396252","heartbeat_front_addr":"172.21.15.119:6807/580396252","state":["exists","up"]},{"osd":1,"uuid":"5567f7c2-8ea4-4789-9ea1-467793324fb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":2500318888},{"type":"v1","addr":"172.21.15.132:6801","nonce":2500318888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":2500318888},{"type":"v1","addr":"172.21.15.132:6803","nonce":2500318888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":2500318888},{"type":"v1","addr":"172.21.15.132:6807","nonce":2500318888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":2500318888},{"type":"v1","addr":"172.21.15.132:6805","nonce":2500318888}]},"public_addr":"172.21.15.132:6801/2500318888","cluster_addr":"172.21.15.132:6803/2500318888","heartbeat_back_addr":"172.21.15.132:6807/2500318888","heartbeat_front_addr":"172.21.15.132:6805/2500318888","state":["exists","up"]},{"osd":2,"uuid":"ee54dbd0-24b4-4ee6-988c-ce71491199d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":2361434832},{"type":"v1","addr":"172.21.15.165:6801","nonce":2361434832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":2361434832},{"type":"v1","addr":"172.21.15.165:6803","nonce":2361434832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":2361434832},{"type":"v1","addr":"172.21.15.165:6807","nonce":2361434832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":2361434832},{"type":"v1","addr":"172.21.15.165:6805","nonce":2361434832}]},"public_addr":"172.21.15.165:6801/2361434832","cluster_addr":"172.21.15.165:6803/2361434832","heartbeat_back_addr":"172.21.15.165:6807/2361434832","heartbeat_front_addr":"172.21.15.165:6805/2361434832","state":["exists","up"]},{"osd":3,"uuid":"0464c40b-828f-4db4-96d9-3f7a6325e6da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":3732272210},{"type":"v1","addr":"172.21.15.119:6811","nonce":3732272210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":3732272210},{"type":"v1","addr":"172.21.15.119:6813","nonce":3732272210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":3732272210},{"type":"v1","addr":"172.21.15.119:6817","nonce":3732272210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":3732272210},{"type":"v1","addr":"172.21.15.119:6815","nonce":3732272210}]},"public_addr":"172.21.15.119:6811/3732272210","cluster_addr":"172.21.15.119:6813/3732272210","heartbeat_back_addr":"172.21.15.119:6817/3732272210","heartbeat_front_addr":"172.21.15.119:6815/3732272210","state":["exists","up"]},{"osd":4,"uuid":"8fd70b8e-5128-4173-aa0e-e735dfeb995d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":2803039505},{"type":"v1","addr":"172.21.15.132:6809","nonce":2803039505}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":2803039505},{"type":"v1","addr":"172.21.15.132:6811","nonce":2803039505}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":2803039505},{"type":"v1","addr":"172.21.15.132:6815","nonce":2803039505}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":2803039505},{"type":"v1","addr":"172.21.15.132:6813","nonce":2803039505}]},"public_addr":"172.21.15.132:6809/2803039505","cluster_addr":"172.21.15.132:6811/2803039505","heartbeat_back_addr":"172.21.15.132:6815/2803039505","heartbeat_front_addr":"172.21.15.132:6813/2803039505","state":["exists","up"]},{"osd":5,"uuid":"3224b4c1-733e-4424-a6da-870bbb5f94e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":2561252206},{"type":"v1","addr":"172.21.15.165:6809","nonce":2561252206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":2561252206},{"type":"v1","addr":"172.21.15.165:6811","nonce":2561252206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":2561252206},{"type":"v1","addr":"172.21.15.165:6815","nonce":2561252206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":2561252206},{"type":"v1","addr":"172.21.15.165:6813","nonce":2561252206}]},"public_addr":"172.21.15.165:6809/2561252206","cluster_addr":"172.21.15.165:6811/2561252206","heartbeat_back_addr":"172.21.15.165:6815/2561252206","heartbeat_front_addr":"172.21.15.165:6813/2561252206","state":["exists","up"]},{"osd":6,"uuid":"e435745d-59b5-439a-9091-0946ca0e9332","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":2545410230},{"type":"v1","addr":"172.21.15.119:6819","nonce":2545410230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":2545410230},{"type":"v1","addr":"172.21.15.119:6821","nonce":2545410230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":2545410230},{"type":"v1","addr":"172.21.15.119:6825","nonce":2545410230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":2545410230},{"type":"v1","addr":"172.21.15.119:6823","nonce":2545410230}]},"public_addr":"172.21.15.119:6819/2545410230","cluster_addr":"172.21.15.119:6821/2545410230","heartbeat_back_addr":"172.21.15.119:6825/2545410230","heartbeat_front_addr":"172.21.15.119:6823/2545410230","state":["exists","up"]},{"osd":7,"uuid":"79b97a12-f6b4-48c3-b517-2dd6b6e7c85f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":2877892376},{"type":"v1","addr":"172.21.15.132:6817","nonce":2877892376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":2877892376},{"type":"v1","addr":"172.21.15.132:6819","nonce":2877892376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":2877892376},{"type":"v1","addr":"172.21.15.132:6823","nonce":2877892376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":2877892376},{"type":"v1","addr":"172.21.15.132:6821","nonce":2877892376}]},"public_addr":"172.21.15.132:6817/2877892376","cluster_addr":"172.21.15.132:6819/2877892376","heartbeat_back_addr":"172.21.15.132:6823/2877892376","heartbeat_front_addr":"172.21.15.132:6821/2877892376","state":["exists","up"]},{"osd":8,"uuid":"2970524c-b99a-412d-85c2-6c29b54035e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":4115226150},{"type":"v1","addr":"172.21.15.165:6817","nonce":4115226150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":4115226150},{"type":"v1","addr":"172.21.15.165:6819","nonce":4115226150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":4115226150},{"type":"v1","addr":"172.21.15.165:6823","nonce":4115226150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":4115226150},{"type":"v1","addr":"172.21.15.165:6821","nonce":4115226150}]},"public_addr":"172.21.15.165:6817/4115226150","cluster_addr":"172.21.15.165:6819/4115226150","heartbeat_back_addr":"172.21.15.165:6823/4115226150","heartbeat_front_addr":"172.21.15.165:6821/4115226150","state":["exists","up"]},{"osd":9,"uuid":"2b1a08ed-cde9-4def-b3e8-838fff22709c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":2532245222},{"type":"v1","addr":"172.21.15.119:6827","nonce":2532245222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":2532245222},{"type":"v1","addr":"172.21.15.119:6829","nonce":2532245222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":2532245222},{"type":"v1","addr":"172.21.15.119:6833","nonce":2532245222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":2532245222},{"type":"v1","addr":"172.21.15.119:6831","nonce":2532245222}]},"public_addr":"172.21.15.119:6827/2532245222","cluster_addr":"172.21.15.119:6829/2532245222","heartbeat_back_addr":"172.21.15.119:6833/2532245222","heartbeat_front_addr":"172.21.15.119:6831/2532245222","state":["exists","up"]},{"osd":10,"uuid":"f019acd0-915a-4094-b12c-b53672740a5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":3454298492},{"type":"v1","addr":"172.21.15.132:6825","nonce":3454298492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":3454298492},{"type":"v1","addr":"172.21.15.132:6827","nonce":3454298492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":3454298492},{"type":"v1","addr":"172.21.15.132:6831","nonce":3454298492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":3454298492},{"type":"v1","addr":"172.21.15.132:6829","nonce":3454298492}]},"public_addr":"172.21.15.132:6825/3454298492","cluster_addr":"172.21.15.132:6827/3454298492","heartbeat_back_addr":"172.21.15.132:6831/3454298492","heartbeat_front_addr":"172.21.15.132:6829/3454298492","state":["exists","up"]},{"osd":11,"uuid":"2c1e59a8-8f21-45c0-8568-4b632ce020f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":2138771601},{"type":"v1","addr":"172.21.15.165:6825","nonce":2138771601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":2138771601},{"type":"v1","addr":"172.21.15.165:6827","nonce":2138771601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":2138771601},{"type":"v1","addr":"172.21.15.165:6831","nonce":2138771601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":2138771601},{"type":"v1","addr":"172.21.15.165:6829","nonce":2138771601}]},"public_addr":"172.21.15.165:6825/2138771601","cluster_addr":"172.21.15.165:6827/2138771601","heartbeat_back_addr":"172.21.15.165:6831/2138771601","heartbeat_front_addr":"172.21.15.165:6829/2138771601","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:17:54.791531+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:18:19.466638+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:18:43.435536+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:19:15.702685+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:19:41.767370+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:20:06.793003+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:20:39.952744+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:21:05.727262+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:21:30.735829+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:22:04.072142+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:22:30.894047+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:22:56.637728+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.119:6801/2575246519":"2024-03-29T00:16:03.582545+0000","172.21.15.119:6800/2575246519":"2024-03-29T00:16:03.582545+0000","172.21.15.119:0/87565965":"2024-03-29T00:16:03.582545+0000","172.21.15.119:6801/2056426384":"2024-03-29T00:15:30.472283+0000","172.21.15.119:0/1639109126":"2024-03-29T00:15:30.472283+0000","172.21.15.119:0/468478300":"2024-03-29T00:15:30.472283+0000","172.21.15.119:0/2891240357":"2024-03-29T00:15:30.472283+0000","172.21.15.119:0/4201699272":"2024-03-29T00:16:03.582545+0000","172.21.15.119:6800/2056426384":"2024-03-29T00:15:30.472283+0000","172.21.15.119:0/2635982178":"2024-03-29T00:16:03.582545+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T00:26:10.531 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:10.525+0000 7f7e31600640 1 -- 172.21.15.119:0/3755088055 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f7e100a2880 msgr2=0x7f7e100a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:26:10.531 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:10.525+0000 7f7e31600640 1 --2- 172.21.15.119:0/3755088055 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f7e100a2880 0x7f7e100a4d40 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f7e24005ab0 tx=0x7f7e2402df90 comp rx=0 tx=0).stop 2024-03-28T00:26:10.532 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:10.525+0000 7f7e31600640 1 -- 172.21.15.119:0/3755088055 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7e2c100740 msgr2=0x7f7e2c199f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:26:10.532 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:10.525+0000 7f7e31600640 1 --2- 172.21.15.119:0/3755088055 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7e2c100740 0x7f7e2c199f20 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f7e14009d50 tx=0x7f7e1400c6a0 comp rx=0 tx=0).stop 2024-03-28T00:26:10.532 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:10.525+0000 7f7e31600640 1 -- 172.21.15.119:0/3755088055 shutdown_connections 2024-03-28T00:26:10.532 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:10.525+0000 7f7e31600640 1 --2- 172.21.15.119:0/3755088055 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f7e100a2880 0x7f7e100a4d40 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:10.532 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:10.525+0000 7f7e31600640 1 --2- 172.21.15.119:0/3755088055 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7e2c1010e0 0x7f7e2c1a0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:10.532 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:10.525+0000 7f7e31600640 1 --2- 172.21.15.119:0/3755088055 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7e2c100740 0x7f7e2c199f20 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:10.532 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:10.525+0000 7f7e31600640 1 --2- 172.21.15.119:0/3755088055 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e2c0ffde0 0x7f7e2c1999e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:10.532 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:10.525+0000 7f7e31600640 1 -- 172.21.15.119:0/3755088055 >> 172.21.15.119:0/3755088055 conn(0x7f7e2c0fb980 msgr2=0x7f7e2c0fd480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:26:10.532 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:10.525+0000 7f7e31600640 1 -- 172.21.15.119:0/3755088055 shutdown_connections 2024-03-28T00:26:10.532 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:10.525+0000 7f7e31600640 1 -- 172.21.15.119:0/3755088055 wait complete. 2024-03-28T00:26:10.552 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-28T00:18:46.253414+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-28T00:23:54.261981+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-28T00:23:55.685836+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-28T00:26:10.552 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-28T00:26:10.767 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:10.761+0000 7f4494400640 1 -- 172.21.15.119:0/1324932195 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f448c104ad0 msgr2=0x7f448c104f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:26:10.767 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:10.761+0000 7f4494400640 1 --2- 172.21.15.119:0/1324932195 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f448c104ad0 0x7f448c104f50 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f4478009900 tx=0x7f447802bc50 comp rx=0 tx=0).stop 2024-03-28T00:26:10.767 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:10.761+0000 7f4494400640 1 -- 172.21.15.119:0/1324932195 shutdown_connections 2024-03-28T00:26:10.767 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:10.761+0000 7f4494400640 1 --2- 172.21.15.119:0/1324932195 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f448c105490 0x7f448c10bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:10.767 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:10.761+0000 7f4494400640 1 --2- 172.21.15.119:0/1324932195 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f448c104ad0 0x7f448c104f50 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:10.768 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:10.761+0000 7f4494400640 1 --2- 172.21.15.119:0/1324932195 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f448c1038e0 0x7f448c103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:10.768 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:10.761+0000 7f4494400640 1 -- 172.21.15.119:0/1324932195 >> 172.21.15.119:0/1324932195 conn(0x7f448c0fee50 msgr2=0x7f448c101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:26:10.768 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:10.761+0000 7f4494400640 1 -- 172.21.15.119:0/1324932195 shutdown_connections 2024-03-28T00:26:10.768 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:10.761+0000 7f4494400640 1 -- 172.21.15.119:0/1324932195 wait complete. 2024-03-28T00:26:10.768 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:10.762+0000 7f4494400640 1 Processor -- start 2024-03-28T00:26:10.769 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:10.763+0000 7f4494400640 1 -- start start 2024-03-28T00:26:10.769 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:10.763+0000 7f4494400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f448c1038e0 0x7f448c199be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:10.770 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:10.763+0000 7f4494400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f448c104ad0 0x7f448c19a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:10.770 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:10.763+0000 7f4494400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f448c105490 0x7f448c1a1190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:10.770 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:10.763+0000 7f4494400640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f448c10f590 con 0x7f448c1038e0 2024-03-28T00:26:10.770 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:10.763+0000 7f4494400640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f448c10f410 con 0x7f448c105490 2024-03-28T00:26:10.770 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:10.763+0000 7f4494400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f448c10f710 con 0x7f448c104ad0 2024-03-28T00:26:10.770 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:10.763+0000 7f4491000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f448c104ad0 0x7f448c19a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:10.770 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:10.763+0000 7f4491000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f448c104ad0 0x7f448c19a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.119:33358/0 (socket says 172.21.15.119:33358) 2024-03-28T00:26:10.770 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:10.763+0000 7f4491000640 1 -- 172.21.15.119:0/1622354660 learned_addr learned my addr 172.21.15.119:0/1622354660 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:26:10.770 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:10.763+0000 7f4491a00640 1 --2- 172.21.15.119:0/1622354660 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f448c1038e0 0x7f448c199be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:10.770 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:10.763+0000 7f4492400640 1 --2- 172.21.15.119:0/1622354660 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f448c105490 0x7f448c1a1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:10.771 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:10.764+0000 7f4491000640 1 -- 172.21.15.119:0/1622354660 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f448c105490 msgr2=0x7f448c1a1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:26:10.771 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:10.764+0000 7f4491000640 1 --2- 172.21.15.119:0/1622354660 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f448c105490 0x7f448c1a1190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:10.771 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:10.764+0000 7f4491000640 1 -- 172.21.15.119:0/1622354660 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f448c1038e0 msgr2=0x7f448c199be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:26:10.771 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:10.764+0000 7f4491000640 1 --2- 172.21.15.119:0/1622354660 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f448c1038e0 0x7f448c199be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:10.772 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:10.764+0000 7f4491000640 1 -- 172.21.15.119:0/1622354660 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f448c1a1800 con 0x7f448c104ad0 2024-03-28T00:26:10.772 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:10.764+0000 7f4492400640 1 --2- 172.21.15.119:0/1622354660 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f448c105490 0x7f448c1a1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T00:26:10.772 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:10.764+0000 7f4491000640 1 --2- 172.21.15.119:0/1622354660 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f448c104ad0 0x7f448c19a120 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f447802c260 tx=0x7f447802c670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:26:10.772 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:10.764+0000 7f448a000640 1 -- 172.21.15.119:0/1622354660 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4478041070 con 0x7f448c104ad0 2024-03-28T00:26:10.772 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:10.764+0000 7f448a000640 1 -- 172.21.15.119:0/1622354660 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4478004450 con 0x7f448c104ad0 2024-03-28T00:26:10.772 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:10.764+0000 7f4494400640 1 -- 172.21.15.119:0/1622354660 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f448c1a1a90 con 0x7f448c104ad0 2024-03-28T00:26:10.772 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:10.764+0000 7f448a000640 1 -- 172.21.15.119:0/1622354660 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4478034440 con 0x7f448c104ad0 2024-03-28T00:26:10.772 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:10.765+0000 7f4494400640 1 -- 172.21.15.119:0/1622354660 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f448c1a1fe0 con 0x7f448c104ad0 2024-03-28T00:26:10.772 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:10.766+0000 7f4494400640 1 -- 172.21.15.119:0/1622354660 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f448c10ac00 con 0x7f448c104ad0 2024-03-28T00:26:10.774 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:10.767+0000 7f448a000640 1 -- 172.21.15.119:0/1622354660 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 18) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f4478044070 con 0x7f448c104ad0 2024-03-28T00:26:10.775 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:10.769+0000 7f448a000640 1 --2- 172.21.15.119:0/1622354660 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f44740a2800 0x7f44740a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:10.775 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:10.769+0000 7f448a000640 1 -- 172.21.15.119:0/1622354660 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f44780f0570 con 0x7f448c104ad0 2024-03-28T00:26:10.775 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:10.769+0000 7f4491a00640 1 --2- 172.21.15.119:0/1622354660 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f44740a2800 0x7f44740a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:10.776 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:10.770+0000 7f4491a00640 1 --2- 172.21.15.119:0/1622354660 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f44740a2800 0x7f44740a4cc0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f448c104930 tx=0x7f4480005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:26:10.779 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:10.773+0000 7f448a000640 1 -- 172.21.15.119:0/1622354660 <== mon.1 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f44780bcfc0 con 0x7f448c104ad0 2024-03-28T00:26:10.980 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:10.973+0000 7f4494400640 1 -- 172.21.15.119:0/1622354660 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f448c0622a0 con 0x7f448c104ad0 2024-03-28T00:26:10.980 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:10.974+0000 7f448a000640 1 -- 172.21.15.119:0/1622354660 <== mon.1 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7f44780c1060 con 0x7f448c104ad0 2024-03-28T00:26:10.980 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 1 2024-03-28T00:26:10.983 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:10.977+0000 7f4494400640 1 -- 172.21.15.119:0/1622354660 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f44740a2800 msgr2=0x7f44740a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:26:10.984 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:10.977+0000 7f4494400640 1 --2- 172.21.15.119:0/1622354660 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f44740a2800 0x7f44740a4cc0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f448c104930 tx=0x7f4480005f90 comp rx=0 tx=0).stop 2024-03-28T00:26:10.984 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:10.977+0000 7f4494400640 1 -- 172.21.15.119:0/1622354660 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f448c104ad0 msgr2=0x7f448c19a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:26:10.984 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:10.977+0000 7f4494400640 1 --2- 172.21.15.119:0/1622354660 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f448c104ad0 0x7f448c19a120 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f447802c260 tx=0x7f447802c670 comp rx=0 tx=0).stop 2024-03-28T00:26:10.984 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:10.977+0000 7f4494400640 1 -- 172.21.15.119:0/1622354660 shutdown_connections 2024-03-28T00:26:10.984 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:10.977+0000 7f4494400640 1 --2- 172.21.15.119:0/1622354660 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f44740a2800 0x7f44740a4cc0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:10.984 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:10.977+0000 7f4494400640 1 --2- 172.21.15.119:0/1622354660 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f448c105490 0x7f448c1a1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:10.984 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:10.977+0000 7f4494400640 1 --2- 172.21.15.119:0/1622354660 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f448c104ad0 0x7f448c19a120 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:10.984 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:10.977+0000 7f4494400640 1 --2- 172.21.15.119:0/1622354660 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f448c1038e0 0x7f448c199be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:10.984 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:10.977+0000 7f4494400640 1 -- 172.21.15.119:0/1622354660 >> 172.21.15.119:0/1622354660 conn(0x7f448c0fee50 msgr2=0x7f448c107dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:26:10.984 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:10.977+0000 7f4494400640 1 -- 172.21.15.119:0/1622354660 shutdown_connections 2024-03-28T00:26:10.984 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:10.977+0000 7f4494400640 1 -- 172.21.15.119:0/1622354660 wait complete. 2024-03-28T00:26:11.004 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-28T00:26:11.212 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:11.205+0000 7f751f800640 1 -- 172.21.15.119:0/1341091509 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7518104ad0 msgr2=0x7f7518104f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:26:11.212 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:11.205+0000 7f751f800640 1 --2- 172.21.15.119:0/1341091509 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7518104ad0 0x7f7518104f50 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f7500009900 tx=0x7f750002bc30 comp rx=0 tx=0).stop 2024-03-28T00:26:11.212 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:11.205+0000 7f751f800640 1 -- 172.21.15.119:0/1341091509 shutdown_connections 2024-03-28T00:26:11.212 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:11.205+0000 7f751f800640 1 --2- 172.21.15.119:0/1341091509 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7518105490 0x7f751810bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:11.212 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:11.205+0000 7f751f800640 1 --2- 172.21.15.119:0/1341091509 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7518104ad0 0x7f7518104f50 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:11.212 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:11.205+0000 7f751f800640 1 --2- 172.21.15.119:0/1341091509 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f75181038e0 0x7f7518103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:11.212 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:11.205+0000 7f751f800640 1 -- 172.21.15.119:0/1341091509 >> 172.21.15.119:0/1341091509 conn(0x7f75180fee50 msgr2=0x7f7518101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:26:11.212 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:11.206+0000 7f751f800640 1 -- 172.21.15.119:0/1341091509 shutdown_connections 2024-03-28T00:26:11.213 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:11.206+0000 7f751f800640 1 -- 172.21.15.119:0/1341091509 wait complete. 2024-03-28T00:26:11.213 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:11.207+0000 7f751f800640 1 Processor -- start 2024-03-28T00:26:11.213 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:11.207+0000 7f751f800640 1 -- start start 2024-03-28T00:26:11.215 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:11.207+0000 7f751f800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f75181038e0 0x7f7518199c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:11.215 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:11.207+0000 7f751f800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7518104ad0 0x7f751819a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:11.215 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:11.207+0000 7f751f800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7518105490 0x7f75181a1190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:11.215 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:11.207+0000 7f751f800640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f751810f590 con 0x7f7518105490 2024-03-28T00:26:11.215 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:11.207+0000 7f751f800640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f751810f410 con 0x7f7518104ad0 2024-03-28T00:26:11.215 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:11.207+0000 7f751f800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f751810f710 con 0x7f75181038e0 2024-03-28T00:26:11.215 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:11.207+0000 7f7517e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7518104ad0 0x7f751819a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:11.215 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:11.208+0000 7f7517e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7518104ad0 0x7f751819a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.119:44034/0 (socket says 172.21.15.119:44034) 2024-03-28T00:26:11.215 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:11.208+0000 7f7517e00640 1 -- 172.21.15.119:0/991559877 learned_addr learned my addr 172.21.15.119:0/991559877 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:26:11.215 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:11.208+0000 7f751d800640 1 --2- 172.21.15.119:0/991559877 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7518105490 0x7f75181a1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:11.215 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:11.208+0000 7f751ce00640 1 --2- 172.21.15.119:0/991559877 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f75181038e0 0x7f7518199c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:11.215 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:11.208+0000 7f7517e00640 1 -- 172.21.15.119:0/991559877 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f75181038e0 msgr2=0x7f7518199c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:26:11.216 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:11.208+0000 7f7517e00640 1 --2- 172.21.15.119:0/991559877 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f75181038e0 0x7f7518199c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:11.216 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:11.208+0000 7f7517e00640 1 -- 172.21.15.119:0/991559877 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7518105490 msgr2=0x7f75181a1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:26:11.216 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:11.208+0000 7f7517e00640 1 --2- 172.21.15.119:0/991559877 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7518105490 0x7f75181a1190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:11.216 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:11.208+0000 7f7517e00640 1 -- 172.21.15.119:0/991559877 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75181a1830 con 0x7f7518104ad0 2024-03-28T00:26:11.216 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:11.208+0000 7f7517e00640 1 --2- 172.21.15.119:0/991559877 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7518104ad0 0x7f751819a160 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f7500000c00 tx=0x7f750002c650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:26:11.216 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:11.208+0000 7f7515600640 1 -- 172.21.15.119:0/991559877 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7500043020 con 0x7f7518104ad0 2024-03-28T00:26:11.216 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:11.209+0000 7f7515600640 1 -- 172.21.15.119:0/991559877 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f75000054e0 con 0x7f7518104ad0 2024-03-28T00:26:11.216 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:11.209+0000 7f7515600640 1 -- 172.21.15.119:0/991559877 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7500003bd0 con 0x7f7518104ad0 2024-03-28T00:26:11.216 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:11.209+0000 7f751f800640 1 -- 172.21.15.119:0/991559877 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75181a1ac0 con 0x7f7518104ad0 2024-03-28T00:26:11.216 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:11.209+0000 7f751f800640 1 -- 172.21.15.119:0/991559877 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75181a2060 con 0x7f7518104ad0 2024-03-28T00:26:11.216 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:11.210+0000 7f751f800640 1 -- 172.21.15.119:0/991559877 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f751810ac00 con 0x7f7518104ad0 2024-03-28T00:26:11.217 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:11.211+0000 7f7515600640 1 -- 172.21.15.119:0/991559877 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 18) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f7500046020 con 0x7f7518104ad0 2024-03-28T00:26:11.219 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:11.213+0000 7f7515600640 1 --2- 172.21.15.119:0/991559877 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f74fc0a2880 0x7f74fc0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:11.219 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:11.213+0000 7f7515600640 1 -- 172.21.15.119:0/991559877 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f75000f0ad0 con 0x7f7518104ad0 2024-03-28T00:26:11.219 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:11.213+0000 7f751ce00640 1 --2- 172.21.15.119:0/991559877 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f74fc0a2880 0x7f74fc0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:11.219 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:11.213+0000 7f751ce00640 1 --2- 172.21.15.119:0/991559877 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f74fc0a2880 0x7f74fc0a4d40 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f751819b0f0 tx=0x7f7508009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:26:11.222 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:11.216+0000 7f7515600640 1 -- 172.21.15.119:0/991559877 <== mon.2 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f750004d020 con 0x7f7518104ad0 2024-03-28T00:26:11.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:10 smithi165 ceph-mon[36302]: pgmap v380: 97 pgs: 97 active+clean; 648 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-28T00:26:11.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:10 smithi165 ceph-mon[36302]: from='client.? 172.21.15.119:0/3027838862' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T00:26:11.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:10 smithi165 ceph-mon[36302]: from='client.? 172.21.15.119:0/3755088055' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-28T00:26:11.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:10 smithi119 ceph-mon[31518]: pgmap v380: 97 pgs: 97 active+clean; 648 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-28T00:26:11.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:10 smithi119 ceph-mon[31518]: from='client.? 172.21.15.119:0/3027838862' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T00:26:11.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:10 smithi119 ceph-mon[31518]: from='client.? 172.21.15.119:0/3755088055' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-28T00:26:11.422 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:11.415+0000 7f751f800640 1 -- 172.21.15.119:0/991559877 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f75180622d0 con 0x7f7518104ad0 2024-03-28T00:26:11.423 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:11.416+0000 7f7515600640 1 -- 172.21.15.119:0/991559877 <== mon.2 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) v1 ==== 104+0+11 (secure 0 0 0) 0x7f75000bdb70 con 0x7f7518104ad0 2024-03-28T00:26:11.423 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 32 2024-03-28T00:26:11.426 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:11.419+0000 7f751f800640 1 -- 172.21.15.119:0/991559877 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f74fc0a2880 msgr2=0x7f74fc0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:26:11.426 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:11.419+0000 7f751f800640 1 --2- 172.21.15.119:0/991559877 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f74fc0a2880 0x7f74fc0a4d40 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f751819b0f0 tx=0x7f7508009500 comp rx=0 tx=0).stop 2024-03-28T00:26:11.426 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:11.419+0000 7f751f800640 1 -- 172.21.15.119:0/991559877 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7518104ad0 msgr2=0x7f751819a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:26:11.426 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:11.419+0000 7f751f800640 1 --2- 172.21.15.119:0/991559877 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7518104ad0 0x7f751819a160 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f7500000c00 tx=0x7f750002c650 comp rx=0 tx=0).stop 2024-03-28T00:26:11.426 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:11.419+0000 7f751f800640 1 -- 172.21.15.119:0/991559877 shutdown_connections 2024-03-28T00:26:11.426 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:11.419+0000 7f751f800640 1 --2- 172.21.15.119:0/991559877 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f74fc0a2880 0x7f74fc0a4d40 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:11.426 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:11.419+0000 7f751f800640 1 --2- 172.21.15.119:0/991559877 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7518105490 0x7f75181a1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:11.427 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:11.419+0000 7f751f800640 1 --2- 172.21.15.119:0/991559877 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7518104ad0 0x7f751819a160 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:11.427 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:11.419+0000 7f751f800640 1 --2- 172.21.15.119:0/991559877 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f75181038e0 0x7f7518199c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:11.427 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:11.419+0000 7f751f800640 1 -- 172.21.15.119:0/991559877 >> 172.21.15.119:0/991559877 conn(0x7f75180fee50 msgr2=0x7f7518107dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:26:11.427 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:11.419+0000 7f751f800640 1 -- 172.21.15.119:0/991559877 shutdown_connections 2024-03-28T00:26:11.427 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:11.419+0000 7f751f800640 1 -- 172.21.15.119:0/991559877 wait complete. 2024-03-28T00:26:11.445 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-28T00:26:11.452 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:10 smithi132 ceph-mon[35941]: pgmap v380: 97 pgs: 97 active+clean; 648 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-28T00:26:11.452 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:10 smithi132 ceph-mon[35941]: from='client.? 172.21.15.119:0/3027838862' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T00:26:11.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:10 smithi132 ceph-mon[35941]: from='client.? 172.21.15.119:0/3755088055' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-28T00:26:11.659 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:11.652+0000 7fdb2be00640 1 -- 172.21.15.119:0/1087467764 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdb2c108db0 msgr2=0x7fdb2c109230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:26:11.659 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:11.652+0000 7fdb2be00640 1 --2- 172.21.15.119:0/1087467764 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdb2c108db0 0x7fdb2c109230 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fdb18009900 tx=0x7fdb1802bc40 comp rx=0 tx=0).stop 2024-03-28T00:26:11.659 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:11.652+0000 7fdb2be00640 1 -- 172.21.15.119:0/1087467764 shutdown_connections 2024-03-28T00:26:11.659 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:11.652+0000 7fdb2be00640 1 --2- 172.21.15.119:0/1087467764 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdb2c109770 0x7fdb2c110000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:11.659 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:11.652+0000 7fdb2be00640 1 --2- 172.21.15.119:0/1087467764 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdb2c108db0 0x7fdb2c109230 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:11.659 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:11.652+0000 7fdb2be00640 1 --2- 172.21.15.119:0/1087467764 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdb2c107bc0 0x7fdb2c107fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:11.659 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:11.652+0000 7fdb2be00640 1 -- 172.21.15.119:0/1087467764 >> 172.21.15.119:0/1087467764 conn(0x7fdb2c076b70 msgr2=0x7fdb2c078f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:26:11.659 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:11.652+0000 7fdb2be00640 1 -- 172.21.15.119:0/1087467764 shutdown_connections 2024-03-28T00:26:11.659 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:11.652+0000 7fdb2be00640 1 -- 172.21.15.119:0/1087467764 wait complete. 2024-03-28T00:26:11.661 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:11.653+0000 7fdb2be00640 1 Processor -- start 2024-03-28T00:26:11.661 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:11.653+0000 7fdb2be00640 1 -- start start 2024-03-28T00:26:11.661 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:11.654+0000 7fdb2be00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdb2c107bc0 0x7fdb2c19deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:11.661 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:11.654+0000 7fdb2be00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdb2c108db0 0x7fdb2c19e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:11.661 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:11.654+0000 7fdb2be00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdb2c109770 0x7fdb2c1a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:11.662 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:11.654+0000 7fdb2be00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb2c113890 con 0x7fdb2c109770 2024-03-28T00:26:11.662 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:11.654+0000 7fdb2be00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb2c113710 con 0x7fdb2c107bc0 2024-03-28T00:26:11.662 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:11.654+0000 7fdb2be00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb2c113a10 con 0x7fdb2c108db0 2024-03-28T00:26:11.662 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:11.654+0000 7fdb2aa00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdb2c107bc0 0x7fdb2c19deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:11.662 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:11.654+0000 7fdb2aa00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdb2c107bc0 0x7fdb2c19deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.119:44048/0 (socket says 172.21.15.119:44048) 2024-03-28T00:26:11.662 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:11.654+0000 7fdb2aa00640 1 -- 172.21.15.119:0/3613617365 learned_addr learned my addr 172.21.15.119:0/3613617365 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:26:11.662 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:11.654+0000 7fdb2a000640 1 --2- 172.21.15.119:0/3613617365 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdb2c108db0 0x7fdb2c19e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:11.662 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:11.654+0000 7fdb2b400640 1 --2- 172.21.15.119:0/3613617365 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdb2c109770 0x7fdb2c1a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:11.662 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:11.654+0000 7fdb2aa00640 1 -- 172.21.15.119:0/3613617365 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdb2c108db0 msgr2=0x7fdb2c19e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:26:11.662 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:11.654+0000 7fdb2aa00640 1 --2- 172.21.15.119:0/3613617365 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdb2c108db0 0x7fdb2c19e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:11.662 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:11.654+0000 7fdb2aa00640 1 -- 172.21.15.119:0/3613617365 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdb2c109770 msgr2=0x7fdb2c1a5460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:26:11.662 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:11.654+0000 7fdb2aa00640 1 --2- 172.21.15.119:0/3613617365 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdb2c109770 0x7fdb2c1a5460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:11.662 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:11.654+0000 7fdb2aa00640 1 -- 172.21.15.119:0/3613617365 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb2c1a5ad0 con 0x7fdb2c107bc0 2024-03-28T00:26:11.663 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:11.655+0000 7fdb2aa00640 1 --2- 172.21.15.119:0/3613617365 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdb2c107bc0 0x7fdb2c19deb0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fdb1c00d950 tx=0x7fdb1c00de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:26:11.663 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:11.655+0000 7fdb30a00640 1 -- 172.21.15.119:0/3613617365 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb1c017020 con 0x7fdb2c107bc0 2024-03-28T00:26:11.663 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:11.655+0000 7fdb30a00640 1 -- 172.21.15.119:0/3613617365 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdb1c009c30 con 0x7fdb2c107bc0 2024-03-28T00:26:11.663 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:11.655+0000 7fdb30a00640 1 -- 172.21.15.119:0/3613617365 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb1c00f720 con 0x7fdb2c107bc0 2024-03-28T00:26:11.663 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:11.655+0000 7fdb2be00640 1 -- 172.21.15.119:0/3613617365 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb2c1a5dc0 con 0x7fdb2c107bc0 2024-03-28T00:26:11.663 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:11.655+0000 7fdb2be00640 1 -- 172.21.15.119:0/3613617365 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb2c1a6280 con 0x7fdb2c107bc0 2024-03-28T00:26:11.663 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:11.656+0000 7fdb2be00640 1 -- 172.21.15.119:0/3613617365 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb2c10eee0 con 0x7fdb2c107bc0 2024-03-28T00:26:11.664 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:11.658+0000 7fdb30a00640 1 -- 172.21.15.119:0/3613617365 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 18) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fdb1c01a020 con 0x7fdb2c107bc0 2024-03-28T00:26:11.665 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:11.659+0000 7fdb30a00640 1 --2- 172.21.15.119:0/3613617365 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fdb100a2880 0x7fdb100a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:11.666 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:11.659+0000 7fdb30a00640 1 -- 172.21.15.119:0/3613617365 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fdb1c0ce820 con 0x7fdb2c107bc0 2024-03-28T00:26:11.666 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:11.659+0000 7fdb2a000640 1 --2- 172.21.15.119:0/3613617365 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fdb100a2880 0x7fdb100a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:11.666 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:11.660+0000 7fdb2a000640 1 --2- 172.21.15.119:0/3613617365 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fdb100a2880 0x7fdb100a4d40 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fdb1802c250 tx=0x7fdb1802cf90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:26:11.670 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:11.663+0000 7fdb30a00640 1 -- 172.21.15.119:0/3613617365 <== mon.2 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fdb1c021020 con 0x7fdb2c107bc0 2024-03-28T00:26:11.872 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:11.866+0000 7fdb2be00640 1 -- 172.21.15.119:0/3613617365 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fdb2c02d320 con 0x7fdb2c107bc0 2024-03-28T00:26:11.873 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:11.867+0000 7fdb30a00640 1 -- 172.21.15.119:0/3613617365 <== mon.2 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) v1 ==== 100+0+11 (secure 0 0 0) 0x7fdb1c009de0 con 0x7fdb2c107bc0 2024-03-28T00:26:11.873 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 64 2024-03-28T00:26:11.876 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:11.869+0000 7fdb2be00640 1 -- 172.21.15.119:0/3613617365 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fdb100a2880 msgr2=0x7fdb100a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:26:11.877 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:11.869+0000 7fdb2be00640 1 --2- 172.21.15.119:0/3613617365 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fdb100a2880 0x7fdb100a4d40 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fdb1802c250 tx=0x7fdb1802cf90 comp rx=0 tx=0).stop 2024-03-28T00:26:11.877 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:11.869+0000 7fdb2be00640 1 -- 172.21.15.119:0/3613617365 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdb2c107bc0 msgr2=0x7fdb2c19deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:26:11.877 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:11.869+0000 7fdb2be00640 1 --2- 172.21.15.119:0/3613617365 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdb2c107bc0 0x7fdb2c19deb0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fdb1c00d950 tx=0x7fdb1c00de10 comp rx=0 tx=0).stop 2024-03-28T00:26:11.877 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:11.870+0000 7fdb2be00640 1 -- 172.21.15.119:0/3613617365 shutdown_connections 2024-03-28T00:26:11.877 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:11.870+0000 7fdb2be00640 1 --2- 172.21.15.119:0/3613617365 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fdb100a2880 0x7fdb100a4d40 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:11.877 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:11.870+0000 7fdb2be00640 1 --2- 172.21.15.119:0/3613617365 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdb2c109770 0x7fdb2c1a5460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:11.877 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:11.870+0000 7fdb2be00640 1 --2- 172.21.15.119:0/3613617365 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdb2c108db0 0x7fdb2c19e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:11.877 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:11.870+0000 7fdb2be00640 1 --2- 172.21.15.119:0/3613617365 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdb2c107bc0 0x7fdb2c19deb0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:11.877 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:11.870+0000 7fdb2be00640 1 -- 172.21.15.119:0/3613617365 >> 172.21.15.119:0/3613617365 conn(0x7fdb2c076b70 msgr2=0x7fdb2c10c0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:26:11.877 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:11.870+0000 7fdb2be00640 1 -- 172.21.15.119:0/3613617365 shutdown_connections 2024-03-28T00:26:11.878 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:11.870+0000 7fdb2be00640 1 -- 172.21.15.119:0/3613617365 wait complete. 2024-03-28T00:26:11.896 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T00:26:12.108 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:12.101+0000 7f9f0ee00640 1 -- 172.21.15.119:0/4264528676 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9f0810dae0 msgr2=0x7f9f0810fed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:26:12.108 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:12.101+0000 7f9f0ee00640 1 --2- 172.21.15.119:0/4264528676 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9f0810dae0 0x7f9f0810fed0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f9f00009ca0 tx=0x7f9f0002c0f0 comp rx=0 tx=0).stop 2024-03-28T00:26:12.108 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:12.101+0000 7f9f0ee00640 1 -- 172.21.15.119:0/4264528676 shutdown_connections 2024-03-28T00:26:12.108 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:12.101+0000 7f9f0ee00640 1 --2- 172.21.15.119:0/4264528676 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9f0810dae0 0x7f9f0810fed0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:12.108 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:12.101+0000 7f9f0ee00640 1 --2- 172.21.15.119:0/4264528676 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9f08074100 0x7f9f08074580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:12.108 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:12.101+0000 7f9f0ee00640 1 --2- 172.21.15.119:0/4264528676 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9f080756e0 0x7f9f08073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:12.108 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:12.101+0000 7f9f0ee00640 1 -- 172.21.15.119:0/4264528676 >> 172.21.15.119:0/4264528676 conn(0x7f9f080fca80 msgr2=0x7f9f080feec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:26:12.108 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:12.101+0000 7f9f0ee00640 1 -- 172.21.15.119:0/4264528676 shutdown_connections 2024-03-28T00:26:12.108 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:12.101+0000 7f9f0ee00640 1 -- 172.21.15.119:0/4264528676 wait complete. 2024-03-28T00:26:12.110 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:12.102+0000 7f9f0ee00640 1 Processor -- start 2024-03-28T00:26:12.110 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:12.103+0000 7f9f0ee00640 1 -- start start 2024-03-28T00:26:12.110 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:12.103+0000 7f9f0ee00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9f08074100 0x7f9f0819ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:12.111 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:12.103+0000 7f9f0ee00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9f080756e0 0x7f9f0819e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:12.111 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:12.103+0000 7f9f0ee00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9f0810dae0 0x7f9f081a5330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:12.111 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:12.103+0000 7f9f0ee00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f08113780 con 0x7f9f0810dae0 2024-03-28T00:26:12.111 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:12.103+0000 7f9f0ee00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f08113600 con 0x7f9f080756e0 2024-03-28T00:26:12.111 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:12.103+0000 7f9f0ee00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f08113900 con 0x7f9f08074100 2024-03-28T00:26:12.111 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:12.103+0000 7f9f0da00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9f08074100 0x7f9f0819ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:12.111 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:12.103+0000 7f9f0d000640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9f080756e0 0x7f9f0819e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:12.111 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:12.103+0000 7f9f0da00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9f08074100 0x7f9f0819ddc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.119:33418/0 (socket says 172.21.15.119:33418) 2024-03-28T00:26:12.111 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:12.103+0000 7f9f0e400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9f0810dae0 0x7f9f081a5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:12.111 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:12.103+0000 7f9f0da00640 1 -- 172.21.15.119:0/4186951470 learned_addr learned my addr 172.21.15.119:0/4186951470 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:26:12.111 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:12.104+0000 7f9f0d000640 1 -- 172.21.15.119:0/4186951470 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9f08074100 msgr2=0x7f9f0819ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:26:12.111 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:12.104+0000 7f9f0d000640 1 --2- 172.21.15.119:0/4186951470 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9f08074100 0x7f9f0819ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:12.111 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:12.104+0000 7f9f0d000640 1 -- 172.21.15.119:0/4186951470 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9f0810dae0 msgr2=0x7f9f081a5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:26:12.111 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:12.104+0000 7f9f0d000640 1 --2- 172.21.15.119:0/4186951470 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9f0810dae0 0x7f9f081a5330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:12.112 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:12.104+0000 7f9f0d000640 1 -- 172.21.15.119:0/4186951470 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f081a59d0 con 0x7f9f080756e0 2024-03-28T00:26:12.112 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:12.104+0000 7f9f0e400640 1 --2- 172.21.15.119:0/4186951470 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9f0810dae0 0x7f9f081a5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T00:26:12.112 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:12.104+0000 7f9f0d000640 1 --2- 172.21.15.119:0/4186951470 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9f080756e0 0x7f9f0819e300 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f9ef400e9f0 tx=0x7f9ef400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:26:12.112 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:12.104+0000 7f9f06000640 1 -- 172.21.15.119:0/4186951470 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ef400ccc0 con 0x7f9f080756e0 2024-03-28T00:26:12.112 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:12.104+0000 7f9f06000640 1 -- 172.21.15.119:0/4186951470 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9ef4010910 con 0x7f9f080756e0 2024-03-28T00:26:12.112 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:12.104+0000 7f9f06000640 1 -- 172.21.15.119:0/4186951470 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ef4018960 con 0x7f9f080756e0 2024-03-28T00:26:12.112 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:12.104+0000 7f9f0ee00640 1 -- 172.21.15.119:0/4186951470 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f081a5cc0 con 0x7f9f080756e0 2024-03-28T00:26:12.112 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:12.104+0000 7f9f0ee00640 1 -- 172.21.15.119:0/4186951470 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f081a61c0 con 0x7f9f080756e0 2024-03-28T00:26:12.112 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:12.105+0000 7f9f0ee00640 1 -- 172.21.15.119:0/4186951470 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f08103ba0 con 0x7f9f080756e0 2024-03-28T00:26:12.114 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:12.108+0000 7f9f06000640 1 -- 172.21.15.119:0/4186951470 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 18) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f9ef4010ac0 con 0x7f9f080756e0 2024-03-28T00:26:12.115 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:12.109+0000 7f9f06000640 1 --2- 172.21.15.119:0/4186951470 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f9eec0a2800 0x7f9eec0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:12.115 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:12.109+0000 7f9f06000640 1 -- 172.21.15.119:0/4186951470 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f9ef40ceea0 con 0x7f9f080756e0 2024-03-28T00:26:12.116 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:12.109+0000 7f9f0da00640 1 --2- 172.21.15.119:0/4186951470 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f9eec0a2800 0x7f9eec0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:12.116 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:12.110+0000 7f9f0da00640 1 --2- 172.21.15.119:0/4186951470 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f9eec0a2800 0x7f9eec0a4cc0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f9f0819f290 tx=0x7f9ef8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:26:12.119 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:12.113+0000 7f9f06000640 1 -- 172.21.15.119:0/4186951470 <== mon.2 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f9ef409b8f0 con 0x7f9f080756e0 2024-03-28T00:26:12.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:11 smithi119 ceph-mon[31518]: from='client.? 172.21.15.119:0/1622354660' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-28T00:26:12.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:11 smithi119 ceph-mon[31518]: from='client.? 172.21.15.119:0/991559877' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-28T00:26:12.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:11 smithi119 ceph-mon[31518]: from='client.? 172.21.15.119:0/3613617365' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-28T00:26:12.360 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:12.353+0000 7f9f0ee00640 1 -- 172.21.15.119:0/4186951470 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9f08062340 con 0x7f9f080756e0 2024-03-28T00:26:12.361 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:12.354+0000 7f9f06000640 1 -- 172.21.15.119:0/4186951470 <== mon.2 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+11473 (secure 0 0 0) 0x7f9ef409f990 con 0x7f9f080756e0 2024-03-28T00:26:12.361 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-28T00:26:12.361 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14400,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/3988635716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":3988635716},{"type":"v1","addr":"172.21.15.119:6837","nonce":3988635716}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6839/1595343240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1595343240},{"type":"v1","addr":"172.21.15.119:6839","nonce":1595343240}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-28T00:23:56.215355+0000","modified":"2024-03-28T00:24:04.459985+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24581,"mds_1":24550,"mds_2":24530,"mds_3":24523,"mds_4":24511},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14466":{"gid":14466,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6841/1678451188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6840","nonce":1678451188},{"type":"v1","addr":"172.21.15.119:6841","nonce":1678451188}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24445":{"gid":24445,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/609963168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":609963168},{"type":"v1","addr":"172.21.15.165:6833","nonce":609963168}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24455":{"gid":24455,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.132:6833/1179016670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":1179016670},{"type":"v1","addr":"172.21.15.132:6833","nonce":1179016670}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24473":{"gid":24473,"name":"j","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6835/744950487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":744950487},{"type":"v1","addr":"172.21.15.119:6835","nonce":744950487}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24500":{"gid":24500,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.132:6835/356981946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":356981946},{"type":"v1","addr":"172.21.15.132:6835","nonce":356981946}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24511":{"gid":24511,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.165:6835/1947980102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":1947980102},{"type":"v1","addr":"172.21.15.165:6835","nonce":1947980102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24523":{"gid":24523,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.165:6837/1531193032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":1531193032},{"type":"v1","addr":"172.21.15.165:6837","nonce":1531193032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24530":{"gid":24530,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.132:6837/3383082967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6836","nonce":3383082967},{"type":"v1","addr":"172.21.15.132:6837","nonce":3383082967}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24550":{"gid":24550,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.165:6839/3004684592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":3004684592},{"type":"v1","addr":"172.21.15.165:6839","nonce":3004684592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.132:6839/3305822525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6838","nonce":3305822525},{"type":"v1","addr":"172.21.15.132:6839","nonce":3305822525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24523,24530,24550,24511,24581]},"id":1}]} 2024-03-28T00:26:12.362 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 26 2024-03-28T00:26:12.364 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:12.357+0000 7f9f0ee00640 1 -- 172.21.15.119:0/4186951470 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f9eec0a2800 msgr2=0x7f9eec0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:26:12.364 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:12.357+0000 7f9f0ee00640 1 --2- 172.21.15.119:0/4186951470 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f9eec0a2800 0x7f9eec0a4cc0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f9f0819f290 tx=0x7f9ef8005f90 comp rx=0 tx=0).stop 2024-03-28T00:26:12.364 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:12.357+0000 7f9f0ee00640 1 -- 172.21.15.119:0/4186951470 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9f080756e0 msgr2=0x7f9f0819e300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:26:12.364 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:12.357+0000 7f9f0ee00640 1 --2- 172.21.15.119:0/4186951470 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9f080756e0 0x7f9f0819e300 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f9ef400e9f0 tx=0x7f9ef400eeb0 comp rx=0 tx=0).stop 2024-03-28T00:26:12.364 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:12.357+0000 7f9f0ee00640 1 -- 172.21.15.119:0/4186951470 shutdown_connections 2024-03-28T00:26:12.364 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:12.357+0000 7f9f0ee00640 1 --2- 172.21.15.119:0/4186951470 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f9eec0a2800 0x7f9eec0a4cc0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:12.364 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:12.357+0000 7f9f0ee00640 1 --2- 172.21.15.119:0/4186951470 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9f0810dae0 0x7f9f081a5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:12.364 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:12.357+0000 7f9f0ee00640 1 --2- 172.21.15.119:0/4186951470 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9f080756e0 0x7f9f0819e300 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:12.364 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:12.357+0000 7f9f0ee00640 1 --2- 172.21.15.119:0/4186951470 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9f08074100 0x7f9f0819ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:12.364 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:12.357+0000 7f9f0ee00640 1 -- 172.21.15.119:0/4186951470 >> 172.21.15.119:0/4186951470 conn(0x7f9f080fca80 msgr2=0x7f9f08100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:26:12.364 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:12.357+0000 7f9f0ee00640 1 -- 172.21.15.119:0/4186951470 shutdown_connections 2024-03-28T00:26:12.364 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:12.358+0000 7f9f0ee00640 1 -- 172.21.15.119:0/4186951470 wait complete. 2024-03-28T00:26:12.382 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T00:26:12.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:11 smithi165 ceph-mon[36302]: from='client.? 172.21.15.119:0/1622354660' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-28T00:26:12.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:11 smithi165 ceph-mon[36302]: from='client.? 172.21.15.119:0/991559877' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-28T00:26:12.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:11 smithi165 ceph-mon[36302]: from='client.? 172.21.15.119:0/3613617365' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-28T00:26:12.452 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:11 smithi132 ceph-mon[35941]: from='client.? 172.21.15.119:0/1622354660' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-28T00:26:12.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:11 smithi132 ceph-mon[35941]: from='client.? 172.21.15.119:0/991559877' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-28T00:26:12.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:11 smithi132 ceph-mon[35941]: from='client.? 172.21.15.119:0/3613617365' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-28T00:26:12.588 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:12.581+0000 7fc1a1600640 1 -- 172.21.15.119:0/2910408324 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc19c108cf0 msgr2=0x7fc19c109170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:26:12.588 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:12.581+0000 7fc1a1600640 1 --2- 172.21.15.119:0/2910408324 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc19c108cf0 0x7fc19c109170 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc184009900 tx=0x7fc18402bc30 comp rx=0 tx=0).stop 2024-03-28T00:26:12.588 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:12.581+0000 7fc1a1600640 1 -- 172.21.15.119:0/2910408324 shutdown_connections 2024-03-28T00:26:12.588 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:12.581+0000 7fc1a1600640 1 --2- 172.21.15.119:0/2910408324 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc19c1096b0 0x7fc19c10ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:12.588 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:12.581+0000 7fc1a1600640 1 --2- 172.21.15.119:0/2910408324 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc19c108cf0 0x7fc19c109170 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:12.588 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:12.581+0000 7fc1a1600640 1 --2- 172.21.15.119:0/2910408324 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc19c107b00 0x7fc19c107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:12.588 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:12.581+0000 7fc1a1600640 1 -- 172.21.15.119:0/2910408324 >> 172.21.15.119:0/2910408324 conn(0x7fc19c076ab0 msgr2=0x7fc19c078ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:26:12.588 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:12.582+0000 7fc1a1600640 1 -- 172.21.15.119:0/2910408324 shutdown_connections 2024-03-28T00:26:12.589 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:12.582+0000 7fc1a1600640 1 -- 172.21.15.119:0/2910408324 wait complete. 2024-03-28T00:26:12.591 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:12.583+0000 7fc1a1600640 1 Processor -- start 2024-03-28T00:26:12.591 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:12.583+0000 7fc1a1600640 1 -- start start 2024-03-28T00:26:12.591 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:12.583+0000 7fc1a1600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc19c107b00 0x7fc19c19dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:12.591 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:12.583+0000 7fc1a1600640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc19c108cf0 0x7fc19c19e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:12.591 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:12.583+0000 7fc1a1600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc19c1096b0 0x7fc19c1a5390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:12.591 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:12.583+0000 7fc1a1600640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc19c1137d0 con 0x7fc19c107b00 2024-03-28T00:26:12.591 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:12.583+0000 7fc1a1600640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc19c113650 con 0x7fc19c108cf0 2024-03-28T00:26:12.591 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:12.583+0000 7fc1a1600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc19c113950 con 0x7fc19c1096b0 2024-03-28T00:26:12.591 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:12.583+0000 7fc193e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc19c108cf0 0x7fc19c19e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:12.591 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:12.583+0000 7fc1a0c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc19c1096b0 0x7fc19c1a5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:12.591 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:12.583+0000 7fc19be00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc19c107b00 0x7fc19c19dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:12.591 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:12.583+0000 7fc193e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc19c108cf0 0x7fc19c19e320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.119:44082/0 (socket says 172.21.15.119:44082) 2024-03-28T00:26:12.592 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:12.583+0000 7fc193e00640 1 -- 172.21.15.119:0/2808613955 learned_addr learned my addr 172.21.15.119:0/2808613955 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:26:12.592 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:12.584+0000 7fc193e00640 1 -- 172.21.15.119:0/2808613955 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc19c1096b0 msgr2=0x7fc19c1a5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:26:12.592 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:12.584+0000 7fc193e00640 1 --2- 172.21.15.119:0/2808613955 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc19c1096b0 0x7fc19c1a5390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:12.592 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:12.584+0000 7fc193e00640 1 -- 172.21.15.119:0/2808613955 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc19c107b00 msgr2=0x7fc19c19dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:26:12.592 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:12.584+0000 7fc193e00640 1 --2- 172.21.15.119:0/2808613955 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc19c107b00 0x7fc19c19dde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:12.592 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:12.584+0000 7fc193e00640 1 -- 172.21.15.119:0/2808613955 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc19c1a5a00 con 0x7fc19c108cf0 2024-03-28T00:26:12.592 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:12.584+0000 7fc1a0c00640 1 --2- 172.21.15.119:0/2808613955 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc19c1096b0 0x7fc19c1a5390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T00:26:12.592 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:12.584+0000 7fc193e00640 1 --2- 172.21.15.119:0/2808613955 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc19c108cf0 0x7fc19c19e320 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc184005ab0 tx=0x7fc18402c6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:26:12.592 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:12.584+0000 7fc199600640 1 -- 172.21.15.119:0/2808613955 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc184043020 con 0x7fc19c108cf0 2024-03-28T00:26:12.592 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:12.584+0000 7fc199600640 1 -- 172.21.15.119:0/2808613955 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc1840054e0 con 0x7fc19c108cf0 2024-03-28T00:26:12.592 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:12.584+0000 7fc199600640 1 -- 172.21.15.119:0/2808613955 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc184003ce0 con 0x7fc19c108cf0 2024-03-28T00:26:12.592 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:12.584+0000 7fc1a1600640 1 -- 172.21.15.119:0/2808613955 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc19c1a5c90 con 0x7fc19c108cf0 2024-03-28T00:26:12.592 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:12.585+0000 7fc1a1600640 1 -- 172.21.15.119:0/2808613955 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc19c1a6190 con 0x7fc19c108cf0 2024-03-28T00:26:12.593 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:12.586+0000 7fc1a1600640 1 -- 172.21.15.119:0/2808613955 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc19c10ee20 con 0x7fc19c108cf0 2024-03-28T00:26:12.594 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:12.588+0000 7fc199600640 1 -- 172.21.15.119:0/2808613955 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 18) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fc184046020 con 0x7fc19c108cf0 2024-03-28T00:26:12.595 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:12.589+0000 7fc199600640 1 --2- 172.21.15.119:0/2808613955 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fc1800a2880 0x7fc1800a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:12.595 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:12.589+0000 7fc199600640 1 -- 172.21.15.119:0/2808613955 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fc1840f0fe0 con 0x7fc19c108cf0 2024-03-28T00:26:12.595 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:12.589+0000 7fc19be00640 1 --2- 172.21.15.119:0/2808613955 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fc1800a2880 0x7fc1800a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:12.596 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:12.590+0000 7fc19be00640 1 --2- 172.21.15.119:0/2808613955 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fc1800a2880 0x7fc1800a4d40 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fc19c108b50 tx=0x7fc18c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:26:12.597 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:12.591+0000 7fc199600640 1 -- 172.21.15.119:0/2808613955 <== mon.2 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fc18404d020 con 0x7fc19c108cf0 2024-03-28T00:26:12.807 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:12.800+0000 7fc1a1600640 1 -- 172.21.15.119:0/2808613955 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc19c02d240 con 0x7fc19c108cf0 2024-03-28T00:26:12.809 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:12.803+0000 7fc199600640 1 -- 172.21.15.119:0/2808613955 <== mon.2 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+22638 (secure 0 0 0) 0x7fc1840be080 con 0x7fc19c108cf0 2024-03-28T00:26:12.810 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-28T00:26:12.810 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":87,"fsid":"10f73b92-ec98-11ee-95d0-87774f69a715","created":"2024-03-28T00:14:57.563604+0000","modified":"2024-03-28T00:24:05.529756+0000","last_up_change":"2024-03-28T00:22:58.821662+0000","last_in_change":"2024-03-28T00:22:38.467270+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T00:18:46.253414+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-28T00:23:54.261981+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-28T00:23:55.685836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c0b875d-a01e-4294-9c9c-5d0b048b8912","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":580396252},{"type":"v1","addr":"172.21.15.119:6803","nonce":580396252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":580396252},{"type":"v1","addr":"172.21.15.119:6805","nonce":580396252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":580396252},{"type":"v1","addr":"172.21.15.119:6809","nonce":580396252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":580396252},{"type":"v1","addr":"172.21.15.119:6807","nonce":580396252}]},"public_addr":"172.21.15.119:6803/580396252","cluster_addr":"172.21.15.119:6805/580396252","heartbeat_back_addr":"172.21.15.119:6809/580396252","heartbeat_front_addr":"172.21.15.119:6807/580396252","state":["exists","up"]},{"osd":1,"uuid":"5567f7c2-8ea4-4789-9ea1-467793324fb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":2500318888},{"type":"v1","addr":"172.21.15.132:6801","nonce":2500318888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":2500318888},{"type":"v1","addr":"172.21.15.132:6803","nonce":2500318888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":2500318888},{"type":"v1","addr":"172.21.15.132:6807","nonce":2500318888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":2500318888},{"type":"v1","addr":"172.21.15.132:6805","nonce":2500318888}]},"public_addr":"172.21.15.132:6801/2500318888","cluster_addr":"172.21.15.132:6803/2500318888","heartbeat_back_addr":"172.21.15.132:6807/2500318888","heartbeat_front_addr":"172.21.15.132:6805/2500318888","state":["exists","up"]},{"osd":2,"uuid":"ee54dbd0-24b4-4ee6-988c-ce71491199d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":2361434832},{"type":"v1","addr":"172.21.15.165:6801","nonce":2361434832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":2361434832},{"type":"v1","addr":"172.21.15.165:6803","nonce":2361434832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":2361434832},{"type":"v1","addr":"172.21.15.165:6807","nonce":2361434832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":2361434832},{"type":"v1","addr":"172.21.15.165:6805","nonce":2361434832}]},"public_addr":"172.21.15.165:6801/2361434832","cluster_addr":"172.21.15.165:6803/2361434832","heartbeat_back_addr":"172.21.15.165:6807/2361434832","heartbeat_front_addr":"172.21.15.165:6805/2361434832","state":["exists","up"]},{"osd":3,"uuid":"0464c40b-828f-4db4-96d9-3f7a6325e6da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":3732272210},{"type":"v1","addr":"172.21.15.119:6811","nonce":3732272210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":3732272210},{"type":"v1","addr":"172.21.15.119:6813","nonce":3732272210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":3732272210},{"type":"v1","addr":"172.21.15.119:6817","nonce":3732272210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":3732272210},{"type":"v1","addr":"172.21.15.119:6815","nonce":3732272210}]},"public_addr":"172.21.15.119:6811/3732272210","cluster_addr":"172.21.15.119:6813/3732272210","heartbeat_back_addr":"172.21.15.119:6817/3732272210","heartbeat_front_addr":"172.21.15.119:6815/3732272210","state":["exists","up"]},{"osd":4,"uuid":"8fd70b8e-5128-4173-aa0e-e735dfeb995d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":2803039505},{"type":"v1","addr":"172.21.15.132:6809","nonce":2803039505}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":2803039505},{"type":"v1","addr":"172.21.15.132:6811","nonce":2803039505}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":2803039505},{"type":"v1","addr":"172.21.15.132:6815","nonce":2803039505}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":2803039505},{"type":"v1","addr":"172.21.15.132:6813","nonce":2803039505}]},"public_addr":"172.21.15.132:6809/2803039505","cluster_addr":"172.21.15.132:6811/2803039505","heartbeat_back_addr":"172.21.15.132:6815/2803039505","heartbeat_front_addr":"172.21.15.132:6813/2803039505","state":["exists","up"]},{"osd":5,"uuid":"3224b4c1-733e-4424-a6da-870bbb5f94e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":2561252206},{"type":"v1","addr":"172.21.15.165:6809","nonce":2561252206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":2561252206},{"type":"v1","addr":"172.21.15.165:6811","nonce":2561252206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":2561252206},{"type":"v1","addr":"172.21.15.165:6815","nonce":2561252206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":2561252206},{"type":"v1","addr":"172.21.15.165:6813","nonce":2561252206}]},"public_addr":"172.21.15.165:6809/2561252206","cluster_addr":"172.21.15.165:6811/2561252206","heartbeat_back_addr":"172.21.15.165:6815/2561252206","heartbeat_front_addr":"172.21.15.165:6813/2561252206","state":["exists","up"]},{"osd":6,"uuid":"e435745d-59b5-439a-9091-0946ca0e9332","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":2545410230},{"type":"v1","addr":"172.21.15.119:6819","nonce":2545410230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":2545410230},{"type":"v1","addr":"172.21.15.119:6821","nonce":2545410230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":2545410230},{"type":"v1","addr":"172.21.15.119:6825","nonce":2545410230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":2545410230},{"type":"v1","addr":"172.21.15.119:6823","nonce":2545410230}]},"public_addr":"172.21.15.119:6819/2545410230","cluster_addr":"172.21.15.119:6821/2545410230","heartbeat_back_addr":"172.21.15.119:6825/2545410230","heartbeat_front_addr":"172.21.15.119:6823/2545410230","state":["exists","up"]},{"osd":7,"uuid":"79b97a12-f6b4-48c3-b517-2dd6b6e7c85f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":2877892376},{"type":"v1","addr":"172.21.15.132:6817","nonce":2877892376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":2877892376},{"type":"v1","addr":"172.21.15.132:6819","nonce":2877892376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":2877892376},{"type":"v1","addr":"172.21.15.132:6823","nonce":2877892376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":2877892376},{"type":"v1","addr":"172.21.15.132:6821","nonce":2877892376}]},"public_addr":"172.21.15.132:6817/2877892376","cluster_addr":"172.21.15.132:6819/2877892376","heartbeat_back_addr":"172.21.15.132:6823/2877892376","heartbeat_front_addr":"172.21.15.132:6821/2877892376","state":["exists","up"]},{"osd":8,"uuid":"2970524c-b99a-412d-85c2-6c29b54035e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":4115226150},{"type":"v1","addr":"172.21.15.165:6817","nonce":4115226150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":4115226150},{"type":"v1","addr":"172.21.15.165:6819","nonce":4115226150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":4115226150},{"type":"v1","addr":"172.21.15.165:6823","nonce":4115226150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":4115226150},{"type":"v1","addr":"172.21.15.165:6821","nonce":4115226150}]},"public_addr":"172.21.15.165:6817/4115226150","cluster_addr":"172.21.15.165:6819/4115226150","heartbeat_back_addr":"172.21.15.165:6823/4115226150","heartbeat_front_addr":"172.21.15.165:6821/4115226150","state":["exists","up"]},{"osd":9,"uuid":"2b1a08ed-cde9-4def-b3e8-838fff22709c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":2532245222},{"type":"v1","addr":"172.21.15.119:6827","nonce":2532245222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":2532245222},{"type":"v1","addr":"172.21.15.119:6829","nonce":2532245222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":2532245222},{"type":"v1","addr":"172.21.15.119:6833","nonce":2532245222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":2532245222},{"type":"v1","addr":"172.21.15.119:6831","nonce":2532245222}]},"public_addr":"172.21.15.119:6827/2532245222","cluster_addr":"172.21.15.119:6829/2532245222","heartbeat_back_addr":"172.21.15.119:6833/2532245222","heartbeat_front_addr":"172.21.15.119:6831/2532245222","state":["exists","up"]},{"osd":10,"uuid":"f019acd0-915a-4094-b12c-b53672740a5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":3454298492},{"type":"v1","addr":"172.21.15.132:6825","nonce":3454298492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":3454298492},{"type":"v1","addr":"172.21.15.132:6827","nonce":3454298492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":3454298492},{"type":"v1","addr":"172.21.15.132:6831","nonce":3454298492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":3454298492},{"type":"v1","addr":"172.21.15.132:6829","nonce":3454298492}]},"public_addr":"172.21.15.132:6825/3454298492","cluster_addr":"172.21.15.132:6827/3454298492","heartbeat_back_addr":"172.21.15.132:6831/3454298492","heartbeat_front_addr":"172.21.15.132:6829/3454298492","state":["exists","up"]},{"osd":11,"uuid":"2c1e59a8-8f21-45c0-8568-4b632ce020f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":2138771601},{"type":"v1","addr":"172.21.15.165:6825","nonce":2138771601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":2138771601},{"type":"v1","addr":"172.21.15.165:6827","nonce":2138771601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":2138771601},{"type":"v1","addr":"172.21.15.165:6831","nonce":2138771601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":2138771601},{"type":"v1","addr":"172.21.15.165:6829","nonce":2138771601}]},"public_addr":"172.21.15.165:6825/2138771601","cluster_addr":"172.21.15.165:6827/2138771601","heartbeat_back_addr":"172.21.15.165:6831/2138771601","heartbeat_front_addr":"172.21.15.165:6829/2138771601","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:17:54.791531+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:18:19.466638+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:18:43.435536+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:19:15.702685+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:19:41.767370+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:20:06.793003+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:20:39.952744+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:21:05.727262+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:21:30.735829+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:22:04.072142+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:22:30.894047+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:22:56.637728+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.119:6801/2575246519":"2024-03-29T00:16:03.582545+0000","172.21.15.119:6800/2575246519":"2024-03-29T00:16:03.582545+0000","172.21.15.119:0/87565965":"2024-03-29T00:16:03.582545+0000","172.21.15.119:6801/2056426384":"2024-03-29T00:15:30.472283+0000","172.21.15.119:0/1639109126":"2024-03-29T00:15:30.472283+0000","172.21.15.119:0/468478300":"2024-03-29T00:15:30.472283+0000","172.21.15.119:0/2891240357":"2024-03-29T00:15:30.472283+0000","172.21.15.119:0/4201699272":"2024-03-29T00:16:03.582545+0000","172.21.15.119:6800/2056426384":"2024-03-29T00:15:30.472283+0000","172.21.15.119:0/2635982178":"2024-03-29T00:16:03.582545+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T00:26:12.813 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:12.806+0000 7fc1a1600640 1 -- 172.21.15.119:0/2808613955 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fc1800a2880 msgr2=0x7fc1800a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:26:12.813 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:12.806+0000 7fc1a1600640 1 --2- 172.21.15.119:0/2808613955 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fc1800a2880 0x7fc1800a4d40 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fc19c108b50 tx=0x7fc18c005f90 comp rx=0 tx=0).stop 2024-03-28T00:26:12.813 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:12.806+0000 7fc1a1600640 1 -- 172.21.15.119:0/2808613955 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc19c108cf0 msgr2=0x7fc19c19e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:26:12.813 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:12.806+0000 7fc1a1600640 1 --2- 172.21.15.119:0/2808613955 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc19c108cf0 0x7fc19c19e320 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc184005ab0 tx=0x7fc18402c6c0 comp rx=0 tx=0).stop 2024-03-28T00:26:12.813 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:12.806+0000 7fc1a1600640 1 -- 172.21.15.119:0/2808613955 shutdown_connections 2024-03-28T00:26:12.813 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:12.806+0000 7fc1a1600640 1 --2- 172.21.15.119:0/2808613955 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fc1800a2880 0x7fc1800a4d40 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:12.813 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:12.806+0000 7fc1a1600640 1 --2- 172.21.15.119:0/2808613955 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc19c1096b0 0x7fc19c1a5390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:12.813 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:12.806+0000 7fc1a1600640 1 --2- 172.21.15.119:0/2808613955 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc19c108cf0 0x7fc19c19e320 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:12.813 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:12.806+0000 7fc1a1600640 1 --2- 172.21.15.119:0/2808613955 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc19c107b00 0x7fc19c19dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:12.813 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:12.806+0000 7fc1a1600640 1 -- 172.21.15.119:0/2808613955 >> 172.21.15.119:0/2808613955 conn(0x7fc19c076ab0 msgr2=0x7fc19c10bff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:26:12.813 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:12.806+0000 7fc1a1600640 1 -- 172.21.15.119:0/2808613955 shutdown_connections 2024-03-28T00:26:12.813 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:12.807+0000 7fc1a1600640 1 -- 172.21.15.119:0/2808613955 wait complete. 2024-03-28T00:26:12.833 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-03-28T00:23:56.215355+0000', 'modified': '2024-03-28T00:24:04.459985+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24581, 'mds_1': 24550, 'mds_2': 24530, 'mds_3': 24523, 'mds_4': 24511}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14466': {'gid': 14466, 'name': 'a', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.119:6841/1678451188', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6840', 'nonce': 1678451188}, {'type': 'v1', 'addr': '172.21.15.119:6841', 'nonce': 1678451188}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24445': {'gid': 24445, 'name': 'l', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.165:6833/609963168', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6832', 'nonce': 609963168}, {'type': 'v1', 'addr': '172.21.15.165:6833', 'nonce': 609963168}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24455': {'gid': 24455, 'name': 'k', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.132:6833/1179016670', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.132:6832', 'nonce': 1179016670}, {'type': 'v1', 'addr': '172.21.15.132:6833', 'nonce': 1179016670}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24473': {'gid': 24473, 'name': 'j', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.119:6835/744950487', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6834', 'nonce': 744950487}, {'type': 'v1', 'addr': '172.21.15.119:6835', 'nonce': 744950487}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24500': {'gid': 24500, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.132:6835/356981946', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.132:6834', 'nonce': 356981946}, {'type': 'v1', 'addr': '172.21.15.132:6835', 'nonce': 356981946}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24511': {'gid': 24511, 'name': 'f', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.165:6835/1947980102', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6834', 'nonce': 1947980102}, {'type': 'v1', 'addr': '172.21.15.165:6835', 'nonce': 1947980102}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24523': {'gid': 24523, 'name': 'c', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.165:6837/1531193032', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6836', 'nonce': 1531193032}, {'type': 'v1', 'addr': '172.21.15.165:6837', 'nonce': 1531193032}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24530': {'gid': 24530, 'name': 'h', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.132:6837/3383082967', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.132:6836', 'nonce': 3383082967}, {'type': 'v1', 'addr': '172.21.15.132:6837', 'nonce': 3383082967}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24550': {'gid': 24550, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.165:6839/3004684592', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6838', 'nonce': 3004684592}, {'type': 'v1', 'addr': '172.21.15.165:6839', 'nonce': 3004684592}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24581': {'gid': 24581, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.132:6839/3305822525', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.132:6838', 'nonce': 3305822525}, {'type': 'v1', 'addr': '172.21.15.132:6839', 'nonce': 3305822525}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24581, 'qdb_cluster': [24523, 24530, 24550, 24511, 24581]} 2024-03-28T00:26:12.833 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-03-28T00:26:12.833 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.f status 2024-03-28T00:26:13.045 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.039+0000 7f57bf200640 1 -- 172.21.15.119:0/2614394305 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f57b81016b0 msgr2=0x7f57b8101ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:26:13.046 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.039+0000 7f57bf200640 1 --2- 172.21.15.119:0/2614394305 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f57b81016b0 0x7f57b8101ad0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f57a8009900 tx=0x7f57a802bc30 comp rx=0 tx=0).stop 2024-03-28T00:26:13.046 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.039+0000 7f57bf200640 1 -- 172.21.15.119:0/2614394305 shutdown_connections 2024-03-28T00:26:13.046 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.039+0000 7f57bf200640 1 --2- 172.21.15.119:0/2614394305 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f57b8103260 0x7f57b8109af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:13.046 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.039+0000 7f57bf200640 1 --2- 172.21.15.119:0/2614394305 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57b81028a0 0x7f57b8102d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:13.046 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.039+0000 7f57bf200640 1 --2- 172.21.15.119:0/2614394305 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f57b81016b0 0x7f57b8101ad0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:13.046 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.039+0000 7f57bf200640 1 -- 172.21.15.119:0/2614394305 >> 172.21.15.119:0/2614394305 conn(0x7f57b80fcc40 msgr2=0x7f57b80ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:26:13.046 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.039+0000 7f57bf200640 1 -- 172.21.15.119:0/2614394305 shutdown_connections 2024-03-28T00:26:13.046 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.039+0000 7f57bf200640 1 -- 172.21.15.119:0/2614394305 wait complete. 2024-03-28T00:26:13.048 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.040+0000 7f57bf200640 1 Processor -- start 2024-03-28T00:26:13.048 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.040+0000 7f57bf200640 1 -- start start 2024-03-28T00:26:13.048 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.041+0000 7f57bf200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57b81016b0 0x7f57b81979f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:13.048 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.041+0000 7f57bf200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f57b81028a0 0x7f57b8197f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:13.048 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.041+0000 7f57bf200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f57b8103260 0x7f57b819efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:13.048 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.041+0000 7f57bf200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57b810d380 con 0x7f57b81016b0 2024-03-28T00:26:13.048 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.041+0000 7f57bf200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57b810d200 con 0x7f57b81028a0 2024-03-28T00:26:13.048 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.041+0000 7f57bf200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57b810d500 con 0x7f57b8103260 2024-03-28T00:26:13.048 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.041+0000 7f57b7400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f57b81028a0 0x7f57b8197f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:13.048 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.041+0000 7f57b7400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f57b81028a0 0x7f57b8197f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.119:44088/0 (socket says 172.21.15.119:44088) 2024-03-28T00:26:13.048 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.041+0000 7f57b7400640 1 -- 172.21.15.119:0/1900840242 learned_addr learned my addr 172.21.15.119:0/1900840242 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:26:13.048 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.041+0000 7f57bd200640 1 --2- 172.21.15.119:0/1900840242 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f57b8103260 0x7f57b819efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:13.049 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.041+0000 7f57b7e00640 1 --2- 172.21.15.119:0/1900840242 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57b81016b0 0x7f57b81979f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:13.049 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.041+0000 7f57b7400640 1 -- 172.21.15.119:0/1900840242 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f57b8103260 msgr2=0x7f57b819efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:26:13.049 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.041+0000 7f57b7400640 1 --2- 172.21.15.119:0/1900840242 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f57b8103260 0x7f57b819efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:13.049 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.041+0000 7f57b7400640 1 -- 172.21.15.119:0/1900840242 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57b81016b0 msgr2=0x7f57b81979f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:26:13.049 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.041+0000 7f57b7400640 1 --2- 172.21.15.119:0/1900840242 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57b81016b0 0x7f57b81979f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:13.049 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.041+0000 7f57b7400640 1 -- 172.21.15.119:0/1900840242 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57b819f610 con 0x7f57b81028a0 2024-03-28T00:26:13.049 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.042+0000 7f57b7400640 1 --2- 172.21.15.119:0/1900840242 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f57b81028a0 0x7f57b8197f30 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f57a400d950 tx=0x7f57a400de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:26:13.049 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.042+0000 7f57b4c00640 1 -- 172.21.15.119:0/1900840242 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57a4017030 con 0x7f57b81028a0 2024-03-28T00:26:13.050 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.042+0000 7f57b4c00640 1 -- 172.21.15.119:0/1900840242 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f57a4004020 con 0x7f57b81028a0 2024-03-28T00:26:13.050 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.042+0000 7f57b4c00640 1 -- 172.21.15.119:0/1900840242 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57a40047a0 con 0x7f57b81028a0 2024-03-28T00:26:13.050 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.042+0000 7f57bf200640 1 -- 172.21.15.119:0/1900840242 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57b819f900 con 0x7f57b81028a0 2024-03-28T00:26:13.050 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.042+0000 7f57bf200640 1 -- 172.21.15.119:0/1900840242 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57b819fe00 con 0x7f57b81028a0 2024-03-28T00:26:13.053 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.045+0000 7f57c1f70740 1 Processor -- start 2024-03-28T00:26:13.053 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.045+0000 7f57c1f70740 1 -- start start 2024-03-28T00:26:13.053 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.045+0000 7f57b4c00640 1 -- 172.21.15.119:0/1900840242 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 18) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f57a40041d0 con 0x7f57b81028a0 2024-03-28T00:26:13.053 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.046+0000 7f57c1f70740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5572e01c3ba0 0x5572e01c3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:13.053 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.046+0000 7f57c1f70740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5572e01c4590 0x5572e01c9090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:13.053 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.046+0000 7f57c1f70740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5572e01c9700 0x5572e01cbb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:13.054 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.046+0000 7f57c1f70740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572e0128c90 con 0x5572e01c9700 2024-03-28T00:26:13.054 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.046+0000 7f57c1f70740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572e00d1980 con 0x5572e01c4590 2024-03-28T00:26:13.054 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.046+0000 7f57c1f70740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572e0124d30 con 0x5572e01c3ba0 2024-03-28T00:26:13.054 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.046+0000 7f57bd200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5572e01c3ba0 0x5572e01c3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:13.054 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.046+0000 7f57b7e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5572e01c4590 0x5572e01c9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:13.054 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.046+0000 7f57bd200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5572e01c3ba0 0x5572e01c3fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.119:33468/0 (socket says 172.21.15.119:33468) 2024-03-28T00:26:13.054 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.046+0000 7f57b7e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5572e01c4590 0x5572e01c9090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.119:44104/0 (socket says 172.21.15.119:44104) 2024-03-28T00:26:13.054 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.046+0000 7f57bd200640 1 -- 172.21.15.119:0/1215359744 learned_addr learned my addr 172.21.15.119:0/1215359744 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:26:13.054 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.046+0000 7f57b7400640 1 --2- 172.21.15.119:0/1215359744 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5572e01c9700 0x5572e01cbb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:13.054 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.046+0000 7f57bd200640 1 -- 172.21.15.119:0/1215359744 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5572e01c4590 msgr2=0x5572e01c9090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T00:26:13.054 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.046+0000 7f57bd200640 1 --2- 172.21.15.119:0/1215359744 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5572e01c4590 0x5572e01c9090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:13.054 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.046+0000 7f57bd200640 1 -- 172.21.15.119:0/1215359744 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5572e01c9700 msgr2=0x5572e01cbb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T00:26:13.054 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.046+0000 7f57bd200640 1 --2- 172.21.15.119:0/1215359744 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5572e01c9700 0x5572e01cbb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:13.055 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.046+0000 7f57bd200640 1 -- 172.21.15.119:0/1215359744 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5572e00e9510 con 0x5572e01c3ba0 2024-03-28T00:26:13.055 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.047+0000 7f57bd200640 1 --2- 172.21.15.119:0/1215359744 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5572e01c3ba0 0x5572e01c3fc0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f57b0009f50 tx=0x7f57b00118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9304afcf655462d4 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:26:13.055 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.047+0000 7f57ad600640 1 -- 172.21.15.119:0/1215359744 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57b0004d20 con 0x5572e01c3ba0 2024-03-28T00:26:13.055 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.047+0000 7f57ad600640 1 -- 172.21.15.119:0/1215359744 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f57b0004050 con 0x5572e01c3ba0 2024-03-28T00:26:13.055 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.047+0000 7f57ad600640 1 -- 172.21.15.119:0/1215359744 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57b0005280 con 0x5572e01c3ba0 2024-03-28T00:26:13.055 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.047+0000 7f57b4c00640 1 --2- 172.21.15.119:0/1900840242 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f579c0a2800 0x7f579c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:13.059 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.047+0000 7f57b4c00640 1 -- 172.21.15.119:0/1900840242 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f57a40ce400 con 0x7f57b81028a0 2024-03-28T00:26:13.059 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.048+0000 7f57b7e00640 1 --2- 172.21.15.119:0/1900840242 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f579c0a2800 0x7f579c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:13.059 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.048+0000 7f57c1f70740 1 -- 172.21.15.119:0/1215359744 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5572e01c3ba0 msgr2=0x5572e01c3fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:26:13.059 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.048+0000 7f57c1f70740 1 --2- 172.21.15.119:0/1215359744 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5572e01c3ba0 0x5572e01c3fc0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f57b0009f50 tx=0x7f57b00118d0 comp rx=0 tx=0).stop 2024-03-28T00:26:13.059 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.048+0000 7f57c1f70740 1 -- 172.21.15.119:0/1215359744 shutdown_connections 2024-03-28T00:26:13.060 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.048+0000 7f57c1f70740 1 --2- 172.21.15.119:0/1215359744 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5572e01c9700 0x5572e01cbb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:13.060 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.048+0000 7f57c1f70740 1 --2- 172.21.15.119:0/1215359744 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5572e01c4590 0x5572e01c9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:13.060 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.048+0000 7f57c1f70740 1 --2- 172.21.15.119:0/1215359744 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5572e01c3ba0 0x5572e01c3fc0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:13.060 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.048+0000 7f57c1f70740 1 -- 172.21.15.119:0/1215359744 >> 172.21.15.119:0/1215359744 conn(0x5572e01bf110 msgr2=0x5572e01bf510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:26:13.060 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.048+0000 7f57c1f70740 1 -- 172.21.15.119:0/1215359744 shutdown_connections 2024-03-28T00:26:13.060 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.048+0000 7f57c1f70740 1 -- 172.21.15.119:0/1215359744 wait complete. 2024-03-28T00:26:13.060 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.048+0000 7f57b7e00640 1 --2- 172.21.15.119:0/1900840242 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f579c0a2800 0x7f579c0a4cc0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f57a802c170 tx=0x7f57a802df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:26:13.060 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.049+0000 7f57c1f70740 1 -- start start 2024-03-28T00:26:13.060 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.049+0000 7f57c1f70740 1 Processor -- start 2024-03-28T00:26:13.060 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.049+0000 7f57c1f70740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5572e01c3ba0 0x5572e01d24c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:13.060 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.049+0000 7f57c1f70740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5572e01c4590 0x5572e01d2a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:13.060 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.049+0000 7f57c1f70740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5572e01c9700 0x5572e01d99d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:13.060 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.049+0000 7f57c1f70740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572e01cc6b0 con 0x5572e01c9700 2024-03-28T00:26:13.060 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.049+0000 7f57c1f70740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572e00e4dd0 con 0x5572e01c4590 2024-03-28T00:26:13.060 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.049+0000 7f57c1f70740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572e01cc830 con 0x5572e01c3ba0 2024-03-28T00:26:13.061 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.049+0000 7f57c1f70740 10 client.0 _ll_register_callbacks cb 0x5572dfe3cac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T00:26:13.061 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.049+0000 7f57b7e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5572e01c3ba0 0x5572e01d24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:13.061 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.049+0000 7f57b7400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5572e01c4590 0x5572e01d2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:13.061 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.049+0000 7f57b7e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5572e01c3ba0 0x5572e01d24c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.119:33478/0 (socket says 172.21.15.119:33478) 2024-03-28T00:26:13.061 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.049+0000 7f57b7e00640 1 -- 172.21.15.119:0/2989168138 learned_addr learned my addr 172.21.15.119:0/2989168138 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:26:13.061 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.050+0000 7f57bd200640 1 --2- 172.21.15.119:0/2989168138 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5572e01c9700 0x5572e01d99d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:13.061 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.050+0000 7f57b7e00640 1 -- 172.21.15.119:0/2989168138 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5572e01c4590 msgr2=0x5572e01d2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T00:26:13.061 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.050+0000 7f57b7e00640 1 --2- 172.21.15.119:0/2989168138 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5572e01c4590 0x5572e01d2a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:13.061 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.050+0000 7f57b7e00640 1 -- 172.21.15.119:0/2989168138 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5572e01c9700 msgr2=0x5572e01d99d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T00:26:13.061 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.050+0000 7f57b7e00640 1 --2- 172.21.15.119:0/2989168138 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5572e01c9700 0x5572e01d99d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:13.061 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.050+0000 7f57b7e00640 1 -- 172.21.15.119:0/2989168138 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5572dfe3d2d0 con 0x5572e01c3ba0 2024-03-28T00:26:13.061 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.050+0000 7f57bd200640 1 --2- 172.21.15.119:0/2989168138 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5572e01c9700 0x5572e01d99d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T00:26:13.061 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.050+0000 7f57b7e00640 1 --2- 172.21.15.119:0/2989168138 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5572e01c3ba0 0x5572e01d24c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f57a802c240 tx=0x7f57a8009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b1221e75a975b159 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:26:13.061 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.050+0000 7f57a2000640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-03-28T00:26:13.061 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.050+0000 7f57a2000640 1 -- 172.21.15.119:0/2989168138 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57a8045070 con 0x5572e01c3ba0 2024-03-28T00:26:13.062 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.050+0000 7f57a2000640 1 -- 172.21.15.119:0/2989168138 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f57a80035f0 con 0x5572e01c3ba0 2024-03-28T00:26:13.062 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.050+0000 7f57c1f70740 1 -- 172.21.15.119:0/2989168138 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f57b0004050 con 0x5572e01c3ba0 2024-03-28T00:26:13.062 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.051+0000 7f57a2000640 1 -- 172.21.15.119:0/2989168138 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f57a80037f0 con 0x5572e01c3ba0 2024-03-28T00:26:13.062 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.051+0000 7f57c1f70740 10 client.25177 fetch_fsmap learned FSMap version 26 2024-03-28T00:26:13.062 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.051+0000 7f57c1f70740 1 -- 172.21.15.119:0/2989168138 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x5572dff076b0 con 0x5572e01c3ba0 2024-03-28T00:26:13.062 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.051+0000 7f57a2000640 1 -- 172.21.15.119:0/2989168138 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f57b0004050 con 0x5572e01c3ba0 2024-03-28T00:26:13.062 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.051+0000 7f57a2000640 1 -- 172.21.15.119:0/2989168138 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 26) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f57a8034bc0 con 0x5572e01c3ba0 2024-03-28T00:26:13.062 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.051+0000 7f57c1f70740 10 client.25177 fetch_fsmap finished waiting for FSMap version 26 2024-03-28T00:26:13.062 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.051+0000 7f57c1f70740 10 client.25177 resolve_mds: resolved name 'f' to daemon mds.f 2024-03-28T00:26:13.062 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.051+0000 7f57c1f70740 20 client.25177 populate_metadata read hostname 'smithi119' 2024-03-28T00:26:13.062 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.051+0000 7f57c1f70740 1 --2- 172.21.15.119:0/2989168138 >> [v2:172.21.15.165:6834/1947980102,v1:172.21.15.165:6835/1947980102] conn(0x5572e01dec80 0x5572e01ddcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:13.062 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.051+0000 7f57c1f70740 4 client.25177 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-28T00:26:13.062 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.051+0000 7f57c1f70740 1 -- 172.21.15.119:0/2989168138 --> [v2:172.21.15.165:6834/1947980102,v1:172.21.15.165:6835/1947980102] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5572e00ca520 con 0x5572e01dec80 2024-03-28T00:26:13.062 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.051+0000 7f57b7400640 1 --2- 172.21.15.119:0/2989168138 >> [v2:172.21.15.165:6834/1947980102,v1:172.21.15.165:6835/1947980102] conn(0x5572e01dec80 0x5572e01ddcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:13.063 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.052+0000 7f57b7400640 1 --2- 172.21.15.119:0/2989168138 >> [v2:172.21.15.165:6834/1947980102,v1:172.21.15.165:6835/1947980102] conn(0x5572e01dec80 0x5572e01ddcc0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=9d9e98a9b7f98c5d server_cookie=ea0c8abf12a39a6c in_seq=0 out_seq=0 2024-03-28T00:26:13.063 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.052+0000 7f57a2000640 10 client.25177 ms_handle_connect on v2:172.21.15.165:6834/1947980102 2024-03-28T00:26:13.063 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.053+0000 7f57a2000640 1 -- 172.21.15.119:0/2989168138 <== mds.4 v2:172.21.15.165:6834/1947980102 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5572e00ca520 con 0x5572e01dec80 2024-03-28T00:26:13.063 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.053+0000 7f57a2000640 10 client.25177 handle_command_reply: tid=0 2024-03-28T00:26:13.063 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.053+0000 7f57c1f70740 1 client.25177 shutdown 2024-03-28T00:26:13.063 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.053+0000 7f57c1f70740 1 -- 172.21.15.119:0/2989168138 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5572e01c3ba0 msgr2=0x5572e01d24c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:26:13.063 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.053+0000 7f57c1f70740 1 --2- 172.21.15.119:0/2989168138 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5572e01c3ba0 0x5572e01d24c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f57a802c240 tx=0x7f57a8009260 comp rx=0 tx=0).stop 2024-03-28T00:26:13.063 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.053+0000 7f57c1f70740 1 -- 172.21.15.119:0/2989168138 shutdown_connections 2024-03-28T00:26:13.063 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.053+0000 7f57c1f70740 1 --2- 172.21.15.119:0/2989168138 >> [v2:172.21.15.165:6834/1947980102,v1:172.21.15.165:6835/1947980102] conn(0x5572e01dec80 0x5572e01ddcc0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:13.063 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.053+0000 7f57c1f70740 1 --2- 172.21.15.119:0/2989168138 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5572e01c9700 0x5572e01d99d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:13.063 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.053+0000 7f57c1f70740 1 --2- 172.21.15.119:0/2989168138 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5572e01c4590 0x5572e01d2a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:13.063 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.053+0000 7f57c1f70740 1 --2- 172.21.15.119:0/2989168138 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5572e01c3ba0 0x5572e01d24c0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:13.063 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.053+0000 7f57a2000640 0 client.25177 ms_handle_reset on v2:172.21.15.165:6834/1947980102 2024-03-28T00:26:13.063 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.053+0000 7f57c1f70740 1 -- 172.21.15.119:0/2989168138 >> 172.21.15.119:0/2989168138 conn(0x5572e01bf110 msgr2=0x5572e01be310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:26:13.064 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.053+0000 7f57c1f70740 1 -- 172.21.15.119:0/2989168138 shutdown_connections 2024-03-28T00:26:13.064 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.053+0000 7f57c1f70740 1 -- 172.21.15.119:0/2989168138 wait complete. 2024-03-28T00:26:13.064 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.053+0000 7f57c1f70740 20 client.25177 trim_cache size 0 max 16384 2024-03-28T00:26:13.078 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.071+0000 7f57c1f70740 1 Processor -- start 2024-03-28T00:26:13.079 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.071+0000 7f57c1f70740 1 -- start start 2024-03-28T00:26:13.079 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.072+0000 7f57c1f70740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5572e01c3ba0 0x5572e01b9e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:13.079 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.072+0000 7f57c1f70740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5572e01c4590 0x5572e01d9960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:13.079 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.072+0000 7f57c1f70740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5572e01c9700 0x5572e01d20d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:13.079 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.072+0000 7f57c1f70740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572e01de4a0 con 0x5572e01c3ba0 2024-03-28T00:26:13.079 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.072+0000 7f57c1f70740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572e01deb00 con 0x5572e01c9700 2024-03-28T00:26:13.079 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.072+0000 7f57c1f70740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572e01df160 con 0x5572e01c4590 2024-03-28T00:26:13.079 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.072+0000 7f57b7400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5572e01c4590 0x5572e01d9960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:13.079 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.072+0000 7f57bd200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5572e01c9700 0x5572e01d20d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:13.079 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.072+0000 7f57b7e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5572e01c3ba0 0x5572e01b9e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:13.079 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.072+0000 7f57b7400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5572e01c4590 0x5572e01d9960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.119:33488/0 (socket says 172.21.15.119:33488) 2024-03-28T00:26:13.079 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.072+0000 7f57b7400640 1 -- 172.21.15.119:0/2968539533 learned_addr learned my addr 172.21.15.119:0/2968539533 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:26:13.079 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.072+0000 7f57bd200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5572e01c9700 0x5572e01d20d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.119:44110/0 (socket says 172.21.15.119:44110) 2024-03-28T00:26:13.079 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.072+0000 7f57b7400640 1 -- 172.21.15.119:0/2968539533 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5572e01c9700 msgr2=0x5572e01d20d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T00:26:13.080 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.072+0000 7f57b7400640 1 --2- 172.21.15.119:0/2968539533 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5572e01c9700 0x5572e01d20d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:13.080 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.072+0000 7f57b7400640 1 -- 172.21.15.119:0/2968539533 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5572e01c3ba0 msgr2=0x5572e01b9e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T00:26:13.080 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.072+0000 7f57b7400640 1 --2- 172.21.15.119:0/2968539533 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5572e01c3ba0 0x5572e01b9e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:13.080 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.072+0000 7f57b7400640 1 -- 172.21.15.119:0/2968539533 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5572e01c4970 con 0x5572e01c4590 2024-03-28T00:26:13.085 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.073+0000 7f57b7400640 1 --2- 172.21.15.119:0/2968539533 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5572e01c4590 0x5572e01d9960 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f57a4005250 tx=0x7f57a4000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=627f28c68aa10a28 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:26:13.085 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.073+0000 7f57acc00640 1 -- 172.21.15.119:0/2968539533 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57a40b3e60 con 0x5572e01c4590 2024-03-28T00:26:13.085 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.073+0000 7f57acc00640 1 -- 172.21.15.119:0/2968539533 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f57a40b0030 con 0x5572e01c4590 2024-03-28T00:26:13.085 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.073+0000 7f57acc00640 1 -- 172.21.15.119:0/2968539533 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57a409e070 con 0x5572e01c4590 2024-03-28T00:26:13.085 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.073+0000 7f57c1f70740 1 -- 172.21.15.119:0/2968539533 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5572e01c4590 msgr2=0x5572e01d9960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:26:13.086 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.073+0000 7f57c1f70740 1 --2- 172.21.15.119:0/2968539533 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5572e01c4590 0x5572e01d9960 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f57a4005250 tx=0x7f57a4000bc0 comp rx=0 tx=0).stop 2024-03-28T00:26:13.086 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.073+0000 7f57c1f70740 1 -- 172.21.15.119:0/2968539533 shutdown_connections 2024-03-28T00:26:13.086 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.073+0000 7f57c1f70740 1 --2- 172.21.15.119:0/2968539533 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5572e01c9700 0x5572e01d20d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:13.086 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.073+0000 7f57c1f70740 1 --2- 172.21.15.119:0/2968539533 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5572e01c4590 0x5572e01d9960 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:13.086 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.073+0000 7f57c1f70740 1 --2- 172.21.15.119:0/2968539533 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5572e01c3ba0 0x5572e01b9e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:13.086 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.073+0000 7f57c1f70740 1 -- 172.21.15.119:0/2968539533 >> 172.21.15.119:0/2968539533 conn(0x5572e01bf110 msgr2=0x5572e01ddcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:26:13.086 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.074+0000 7f57c1f70740 1 -- 172.21.15.119:0/2968539533 shutdown_connections 2024-03-28T00:26:13.086 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.074+0000 7f57c1f70740 1 -- 172.21.15.119:0/2968539533 wait complete. 2024-03-28T00:26:13.086 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.074+0000 7f57c1f70740 1 -- start start 2024-03-28T00:26:13.086 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.074+0000 7f57c1f70740 1 Processor -- start 2024-03-28T00:26:13.086 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.074+0000 7f57c1f70740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5572e01c3ba0 0x5572e01d9800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:13.086 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.074+0000 7f57c1f70740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5572e01c4590 0x5572e01d62f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:13.087 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.075+0000 7f57c1f70740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5572e01c9700 0x5572e01d6830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:13.087 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.075+0000 7f57c1f70740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572e01dfe20 con 0x5572e01c9700 2024-03-28T00:26:13.087 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.075+0000 7f57c1f70740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572e01df7c0 con 0x5572e01c3ba0 2024-03-28T00:26:13.087 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.075+0000 7f57c1f70740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572e01e0480 con 0x5572e01c4590 2024-03-28T00:26:13.087 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.075+0000 7f57c1f70740 10 client.0 _ll_register_callbacks cb 0x7f57a802c5d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T00:26:13.087 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.075+0000 7f57b7400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5572e01c4590 0x5572e01d62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:13.087 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.075+0000 7f57b7e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5572e01c3ba0 0x5572e01d9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:13.087 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.075+0000 7f57b7e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5572e01c3ba0 0x5572e01d9800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.119:44114/0 (socket says 172.21.15.119:44114) 2024-03-28T00:26:13.087 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.075+0000 7f57b7400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5572e01c4590 0x5572e01d62f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.119:33498/0 (socket says 172.21.15.119:33498) 2024-03-28T00:26:13.087 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.075+0000 7f57b7e00640 1 -- 172.21.15.119:0/2178283943 learned_addr learned my addr 172.21.15.119:0/2178283943 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:26:13.087 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.075+0000 7f57bd200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5572e01c9700 0x5572e01d6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:13.087 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.075+0000 7f57b7e00640 1 -- 172.21.15.119:0/2178283943 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5572e01c4590 msgr2=0x5572e01d62f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T00:26:13.089 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.075+0000 7f57b7e00640 1 --2- 172.21.15.119:0/2178283943 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5572e01c4590 0x5572e01d62f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:13.089 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.075+0000 7f57b7e00640 1 -- 172.21.15.119:0/2178283943 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5572e01c9700 msgr2=0x5572e01d6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T00:26:13.089 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.075+0000 7f57b7e00640 1 --2- 172.21.15.119:0/2178283943 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5572e01c9700 0x5572e01d6830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:13.089 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.075+0000 7f57b7e00640 1 -- 172.21.15.119:0/2178283943 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5572e0239ce0 con 0x5572e01c3ba0 2024-03-28T00:26:13.089 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.075+0000 7f57b7400640 1 --2- 172.21.15.119:0/2178283943 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5572e01c4590 0x5572e01d62f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T00:26:13.089 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.075+0000 7f57bd200640 1 --2- 172.21.15.119:0/2178283943 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5572e01c9700 0x5572e01d6830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T00:26:13.089 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.075+0000 7f57b7e00640 1 --2- 172.21.15.119:0/2178283943 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5572e01c3ba0 0x5572e01d9800 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f57a802ca50 tx=0x7f57a8054650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3921c322ca9e0f05 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:26:13.090 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.075+0000 7f57a2000640 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-03-28T00:26:13.090 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.076+0000 7f57a2000640 1 -- 172.21.15.119:0/2178283943 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57a806c060 con 0x5572e01c3ba0 2024-03-28T00:26:13.090 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.076+0000 7f57a2000640 1 -- 172.21.15.119:0/2178283943 <== mon.2 v2:172.21.15.132:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f57a8054d10 con 0x5572e01c3ba0 2024-03-28T00:26:13.090 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.076+0000 7f57c1f70740 1 -- 172.21.15.119:0/2178283943 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f57a40b0030 con 0x5572e01c3ba0 2024-03-28T00:26:13.090 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.076+0000 7f57a2000640 1 -- 172.21.15.119:0/2178283943 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f57a8003990 con 0x5572e01c3ba0 2024-03-28T00:26:13.090 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.076+0000 7f57c1f70740 10 client.25172 fetch_fsmap learned FSMap version 26 2024-03-28T00:26:13.090 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.076+0000 7f57c1f70740 1 -- 172.21.15.119:0/2178283943 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x5572e023af60 con 0x5572e01c3ba0 2024-03-28T00:26:13.090 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.076+0000 7f57a2000640 1 -- 172.21.15.119:0/2178283943 <== mon.2 v2:172.21.15.132:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f57a40b0030 con 0x5572e01c3ba0 2024-03-28T00:26:13.090 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.076+0000 7f57a2000640 1 -- 172.21.15.119:0/2178283943 <== mon.2 v2:172.21.15.132:3300/0 5 ==== fsmap(e 26) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f57a8071050 con 0x5572e01c3ba0 2024-03-28T00:26:13.090 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.077+0000 7f57c1f70740 10 client.25172 fetch_fsmap finished waiting for FSMap version 26 2024-03-28T00:26:13.091 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.077+0000 7f57c1f70740 10 client.25172 resolve_mds: resolved name 'f' to daemon mds.f 2024-03-28T00:26:13.091 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.077+0000 7f57c1f70740 20 client.25172 populate_metadata read hostname 'smithi119' 2024-03-28T00:26:13.091 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.077+0000 7f57c1f70740 1 --2- 172.21.15.119:0/2178283943 >> [v2:172.21.15.165:6834/1947980102,v1:172.21.15.165:6835/1947980102] conn(0x5572e023f5f0 0x5572e023b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:13.091 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.077+0000 7f57c1f70740 4 client.25172 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-28T00:26:13.091 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.077+0000 7f57c1f70740 1 -- 172.21.15.119:0/2178283943 --> [v2:172.21.15.165:6834/1947980102,v1:172.21.15.165:6835/1947980102] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5572e01361d0 con 0x5572e023f5f0 2024-03-28T00:26:13.091 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.077+0000 7f57b7400640 1 --2- 172.21.15.119:0/2178283943 >> [v2:172.21.15.165:6834/1947980102,v1:172.21.15.165:6835/1947980102] conn(0x5572e023f5f0 0x5572e023b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:13.091 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.077+0000 7f57b7400640 1 --2- 172.21.15.119:0/2178283943 >> [v2:172.21.15.165:6834/1947980102,v1:172.21.15.165:6835/1947980102] conn(0x5572e023f5f0 0x5572e023b1f0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f74436fd61df6d8c server_cookie=652e689dfef4c7b4 in_seq=0 out_seq=0 2024-03-28T00:26:13.091 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.077+0000 7f57a2000640 10 client.25172 ms_handle_connect on v2:172.21.15.165:6834/1947980102 2024-03-28T00:26:13.092 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.077+0000 7f57a2000640 1 -- 172.21.15.119:0/2178283943 <== mds.4 v2:172.21.15.165:6834/1947980102 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+322 (crc 0 0 0) 0x5572e01361d0 con 0x5572e023f5f0 2024-03-28T00:26:13.092 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.077+0000 7f57a2000640 10 client.25172 handle_command_reply: tid=0 2024-03-28T00:26:13.092 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.077+0000 7f57c1f70740 1 client.25172 shutdown 2024-03-28T00:26:13.092 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.078+0000 7f57c1f70740 1 -- 172.21.15.119:0/2178283943 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5572e01c3ba0 msgr2=0x5572e01d9800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:26:13.092 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.078+0000 7f57c1f70740 1 --2- 172.21.15.119:0/2178283943 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5572e01c3ba0 0x5572e01d9800 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f57a802ca50 tx=0x7f57a8054650 comp rx=0 tx=0).stop 2024-03-28T00:26:13.092 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.078+0000 7f57c1f70740 1 -- 172.21.15.119:0/2178283943 shutdown_connections 2024-03-28T00:26:13.092 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.078+0000 7f57c1f70740 1 --2- 172.21.15.119:0/2178283943 >> [v2:172.21.15.165:6834/1947980102,v1:172.21.15.165:6835/1947980102] conn(0x5572e023f5f0 0x5572e023b1f0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:13.092 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.078+0000 7f57c1f70740 1 --2- 172.21.15.119:0/2178283943 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5572e01c9700 0x5572e01d6830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:13.092 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.078+0000 7f57c1f70740 1 --2- 172.21.15.119:0/2178283943 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5572e01c4590 0x5572e01d62f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:13.092 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.078+0000 7f57c1f70740 1 --2- 172.21.15.119:0/2178283943 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5572e01c3ba0 0x5572e01d9800 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:13.092 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.078+0000 7f57a2000640 0 client.25172 ms_handle_reset on v2:172.21.15.165:6834/1947980102 2024-03-28T00:26:13.092 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.078+0000 7f57c1f70740 1 -- 172.21.15.119:0/2178283943 >> 172.21.15.119:0/2178283943 conn(0x5572e01bf110 msgr2=0x5572e01b9e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:26:13.092 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.078+0000 7f57c1f70740 1 -- 172.21.15.119:0/2178283943 shutdown_connections 2024-03-28T00:26:13.093 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.078+0000 7f57c1f70740 1 -- 172.21.15.119:0/2178283943 wait complete. 2024-03-28T00:26:13.093 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.078+0000 7f57c1f70740 20 client.25172 trim_cache size 0 max 16384 2024-03-28T00:26:13.093 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-03-28T00:26:13.093 INFO:teuthology.orchestra.run.smithi119.stdout: "cluster_fsid": "10f73b92-ec98-11ee-95d0-87774f69a715", 2024-03-28T00:26:13.093 INFO:teuthology.orchestra.run.smithi119.stdout: "whoami": 4, 2024-03-28T00:26:13.093 INFO:teuthology.orchestra.run.smithi119.stdout: "id": 24511, 2024-03-28T00:26:13.093 INFO:teuthology.orchestra.run.smithi119.stdout: "want_state": "up:active", 2024-03-28T00:26:13.094 INFO:teuthology.orchestra.run.smithi119.stdout: "state": "up:active", 2024-03-28T00:26:13.094 INFO:teuthology.orchestra.run.smithi119.stdout: "fs_name": "cephfs", 2024-03-28T00:26:13.094 INFO:teuthology.orchestra.run.smithi119.stdout: "rank_uptime": 129.557039995, 2024-03-28T00:26:13.094 INFO:teuthology.orchestra.run.smithi119.stdout: "mdsmap_epoch": 26, 2024-03-28T00:26:13.094 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_epoch": 87, 2024-03-28T00:26:13.094 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_epoch_barrier": 85, 2024-03-28T00:26:13.094 INFO:teuthology.orchestra.run.smithi119.stdout: "uptime": 162.701480815 2024-03-28T00:26:13.094 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-03-28T00:26:13.095 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.078+0000 7f57bf200640 1 -- 172.21.15.119:0/1900840242 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f579c0a2800 msgr2=0x7f579c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:26:13.095 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.079+0000 7f57bf200640 1 --2- 172.21.15.119:0/1900840242 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f579c0a2800 0x7f579c0a4cc0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f57a802c170 tx=0x7f57a802df90 comp rx=0 tx=0).stop 2024-03-28T00:26:13.095 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.079+0000 7f57bf200640 1 -- 172.21.15.119:0/1900840242 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f57b81028a0 msgr2=0x7f57b8197f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:26:13.095 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.079+0000 7f57bf200640 1 --2- 172.21.15.119:0/1900840242 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f57b81028a0 0x7f57b8197f30 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f57a400d950 tx=0x7f57a400de10 comp rx=0 tx=0).stop 2024-03-28T00:26:13.095 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.079+0000 7f57bf200640 1 -- 172.21.15.119:0/1900840242 shutdown_connections 2024-03-28T00:26:13.096 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.079+0000 7f57bf200640 1 --2- 172.21.15.119:0/1900840242 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f579c0a2800 0x7f579c0a4cc0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:13.096 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.079+0000 7f57bf200640 1 --2- 172.21.15.119:0/1900840242 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f57b8103260 0x7f57b819efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:13.096 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.079+0000 7f57bf200640 1 --2- 172.21.15.119:0/1900840242 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f57b81028a0 0x7f57b8197f30 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:13.096 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.079+0000 7f57bf200640 1 --2- 172.21.15.119:0/1900840242 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57b81016b0 0x7f57b81979f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:13.096 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.079+0000 7f57bf200640 1 -- 172.21.15.119:0/1900840242 >> 172.21.15.119:0/1900840242 conn(0x7f57b80fcc40 msgr2=0x7f57b8105bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:26:13.096 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.079+0000 7f57bf200640 1 -- 172.21.15.119:0/1900840242 shutdown_connections 2024-03-28T00:26:13.096 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.079+0000 7f57bf200640 1 -- 172.21.15.119:0/1900840242 wait complete. 2024-03-28T00:26:13.099 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:12 smithi119 ceph-mon[31518]: pgmap v381: 97 pgs: 97 active+clean; 648 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T00:26:13.099 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:12 smithi119 ceph-mon[31518]: from='client.? 172.21.15.119:0/4186951470' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T00:26:13.099 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:12 smithi119 ceph-mon[31518]: from='client.? 172.21.15.119:0/2808613955' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-28T00:26:13.102 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-03-28T00:26:13.306 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.300+0000 7fbdfce00640 1 -- 172.21.15.119:0/2071560910 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbdf8103260 msgr2=0x7fbdf8109af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:26:13.306 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.300+0000 7fbdfce00640 1 --2- 172.21.15.119:0/2071560910 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbdf8103260 0x7fbdf8109af0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fbdf0009ca0 tx=0x7fbdf002c0f0 comp rx=0 tx=0).stop 2024-03-28T00:26:13.307 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.300+0000 7fbdfce00640 1 -- 172.21.15.119:0/2071560910 shutdown_connections 2024-03-28T00:26:13.307 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.300+0000 7fbdfce00640 1 --2- 172.21.15.119:0/2071560910 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbdf8103260 0x7fbdf8109af0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:13.307 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.300+0000 7fbdfce00640 1 --2- 172.21.15.119:0/2071560910 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbdf81028a0 0x7fbdf8102d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:13.307 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.300+0000 7fbdfce00640 1 --2- 172.21.15.119:0/2071560910 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbdf81016b0 0x7fbdf8101ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:13.307 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.300+0000 7fbdfce00640 1 -- 172.21.15.119:0/2071560910 >> 172.21.15.119:0/2071560910 conn(0x7fbdf80fcc40 msgr2=0x7fbdf80ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:26:13.307 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.300+0000 7fbdfce00640 1 -- 172.21.15.119:0/2071560910 shutdown_connections 2024-03-28T00:26:13.307 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.300+0000 7fbdfce00640 1 -- 172.21.15.119:0/2071560910 wait complete. 2024-03-28T00:26:13.309 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.301+0000 7fbdfce00640 1 Processor -- start 2024-03-28T00:26:13.309 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.301+0000 7fbdfce00640 1 -- start start 2024-03-28T00:26:13.309 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.301+0000 7fbdfce00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbdf81016b0 0x7fbdf8197930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:13.309 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.301+0000 7fbdfce00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbdf81028a0 0x7fbdf8197e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:13.309 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.301+0000 7fbdfce00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbdf8103260 0x7fbdf819eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:13.309 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.301+0000 7fbdfce00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdf810d310 con 0x7fbdf81028a0 2024-03-28T00:26:13.309 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.301+0000 7fbdfce00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdf810d190 con 0x7fbdf8103260 2024-03-28T00:26:13.310 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.301+0000 7fbdfce00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdf810d490 con 0x7fbdf81016b0 2024-03-28T00:26:13.310 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.301+0000 7fbdf5e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbdf81016b0 0x7fbdf8197930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:13.310 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.301+0000 7fbdf6800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbdf8103260 0x7fbdf819eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:13.310 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.301+0000 7fbded400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbdf81028a0 0x7fbdf8197e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:13.310 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.301+0000 7fbdf5e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbdf81016b0 0x7fbdf8197930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.119:33524/0 (socket says 172.21.15.119:33524) 2024-03-28T00:26:13.310 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.301+0000 7fbdf6800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbdf8103260 0x7fbdf819eee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.119:44130/0 (socket says 172.21.15.119:44130) 2024-03-28T00:26:13.310 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.301+0000 7fbdf5e00640 1 -- 172.21.15.119:0/673352738 learned_addr learned my addr 172.21.15.119:0/673352738 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:26:13.310 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.302+0000 7fbdf5e00640 1 -- 172.21.15.119:0/673352738 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbdf8103260 msgr2=0x7fbdf819eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:26:13.310 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.302+0000 7fbdf5e00640 1 --2- 172.21.15.119:0/673352738 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbdf8103260 0x7fbdf819eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:13.310 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.302+0000 7fbdf5e00640 1 -- 172.21.15.119:0/673352738 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbdf81028a0 msgr2=0x7fbdf8197e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:26:13.310 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.302+0000 7fbdf5e00640 1 --2- 172.21.15.119:0/673352738 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbdf81028a0 0x7fbdf8197e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:13.311 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.302+0000 7fbdf5e00640 1 -- 172.21.15.119:0/673352738 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdf819f550 con 0x7fbdf81016b0 2024-03-28T00:26:13.311 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.302+0000 7fbdf6800640 1 --2- 172.21.15.119:0/673352738 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbdf8103260 0x7fbdf819eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T00:26:13.311 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.302+0000 7fbdf5e00640 1 --2- 172.21.15.119:0/673352738 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbdf81016b0 0x7fbdf8197930 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fbde800e9c0 tx=0x7fbde800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:26:13.311 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.302+0000 7fbdef400640 1 -- 172.21.15.119:0/673352738 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbde800cd80 con 0x7fbdf81016b0 2024-03-28T00:26:13.311 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.302+0000 7fbdef400640 1 -- 172.21.15.119:0/673352738 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbde80035f0 con 0x7fbdf81016b0 2024-03-28T00:26:13.311 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.302+0000 7fbdef400640 1 -- 172.21.15.119:0/673352738 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbde8010690 con 0x7fbdf81016b0 2024-03-28T00:26:13.311 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.302+0000 7fbdfce00640 1 -- 172.21.15.119:0/673352738 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdf819f840 con 0x7fbdf81016b0 2024-03-28T00:26:13.311 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.302+0000 7fbdfce00640 1 -- 172.21.15.119:0/673352738 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdf819fde0 con 0x7fbdf81016b0 2024-03-28T00:26:13.312 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.305+0000 7fbdef400640 1 -- 172.21.15.119:0/673352738 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 18) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fbde80037a0 con 0x7fbdf81016b0 2024-03-28T00:26:13.312 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.305+0000 7fbdff9f8740 1 Processor -- start 2024-03-28T00:26:13.312 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.306+0000 7fbdff9f8740 1 -- start start 2024-03-28T00:26:13.314 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.306+0000 7fbdff9f8740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b90ef75ba0 0x55b90ef75fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:13.314 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.306+0000 7fbdff9f8740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b90ef76590 0x55b90ef7b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:13.314 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.306+0000 7fbdff9f8740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b90ef7b700 0x55b90ef7db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:13.315 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.306+0000 7fbdff9f8740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b90eedac90 con 0x55b90ef75ba0 2024-03-28T00:26:13.315 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.306+0000 7fbdff9f8740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b90ee83980 con 0x55b90ef76590 2024-03-28T00:26:13.315 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.306+0000 7fbdff9f8740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b90eed6d30 con 0x55b90ef7b700 2024-03-28T00:26:13.315 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.306+0000 7fbdf5e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b90ef76590 0x55b90ef7b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:13.315 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.306+0000 7fbdf6800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b90ef75ba0 0x55b90ef75fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:13.315 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.306+0000 7fbdf5e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b90ef76590 0x55b90ef7b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.119:44142/0 (socket says 172.21.15.119:44142) 2024-03-28T00:26:13.315 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.306+0000 7fbdf6800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b90ef75ba0 0x55b90ef75fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41090/0 (socket says 172.21.15.119:41090) 2024-03-28T00:26:13.315 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.306+0000 7fbdf5e00640 1 -- 172.21.15.119:0/1550337027 learned_addr learned my addr 172.21.15.119:0/1550337027 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:26:13.315 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.306+0000 7fbded400640 1 --2- 172.21.15.119:0/1550337027 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b90ef7b700 0x55b90ef7db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:13.315 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.307+0000 7fbdf5e00640 1 -- 172.21.15.119:0/1550337027 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b90ef7b700 msgr2=0x55b90ef7db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T00:26:13.315 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.307+0000 7fbdf5e00640 1 --2- 172.21.15.119:0/1550337027 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b90ef7b700 0x55b90ef7db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:13.315 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.307+0000 7fbdf5e00640 1 -- 172.21.15.119:0/1550337027 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b90ef75ba0 msgr2=0x55b90ef75fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T00:26:13.315 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.307+0000 7fbdf5e00640 1 --2- 172.21.15.119:0/1550337027 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b90ef75ba0 0x55b90ef75fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:13.315 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.307+0000 7fbdf5e00640 1 -- 172.21.15.119:0/1550337027 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b90ee9b510 con 0x55b90ef76590 2024-03-28T00:26:13.316 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.307+0000 7fbdf5e00640 1 --2- 172.21.15.119:0/1550337027 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b90ef76590 0x55b90ef7b090 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fbde80d2d30 tx=0x7fbde80d77e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=23f7c418aef7e0c6 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:26:13.316 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.307+0000 7fbdef400640 1 --2- 172.21.15.119:0/673352738 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fbddc0a2880 0x7fbddc0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:13.316 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.307+0000 7fbde7400640 1 -- 172.21.15.119:0/1550337027 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbde80979d0 con 0x55b90ef76590 2024-03-28T00:26:13.316 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.307+0000 7fbdf6800640 1 --2- 172.21.15.119:0/673352738 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fbddc0a2880 0x7fbddc0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:13.316 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.307+0000 7fbde7400640 1 -- 172.21.15.119:0/1550337027 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbde80d0030 con 0x55b90ef76590 2024-03-28T00:26:13.316 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.307+0000 7fbdef400640 1 -- 172.21.15.119:0/673352738 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fbde80ced90 con 0x7fbdf81016b0 2024-03-28T00:26:13.316 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.307+0000 7fbde7400640 1 -- 172.21.15.119:0/1550337027 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbde809a260 con 0x55b90ef76590 2024-03-28T00:26:13.316 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.308+0000 7fbdff9f8740 1 -- 172.21.15.119:0/1550337027 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b90ef76590 msgr2=0x55b90ef7b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:26:13.316 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.308+0000 7fbdff9f8740 1 --2- 172.21.15.119:0/1550337027 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b90ef76590 0x55b90ef7b090 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fbde80d2d30 tx=0x7fbde80d77e0 comp rx=0 tx=0).stop 2024-03-28T00:26:13.316 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.308+0000 7fbdf6800640 1 --2- 172.21.15.119:0/673352738 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fbddc0a2880 0x7fbddc0a4d40 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fbdf002c630 tx=0x7fbdf0032590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:26:13.316 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.308+0000 7fbdff9f8740 1 -- 172.21.15.119:0/1550337027 shutdown_connections 2024-03-28T00:26:13.316 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.308+0000 7fbdff9f8740 1 --2- 172.21.15.119:0/1550337027 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b90ef7b700 0x55b90ef7db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:13.316 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.308+0000 7fbdff9f8740 1 --2- 172.21.15.119:0/1550337027 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b90ef76590 0x55b90ef7b090 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:13.316 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.308+0000 7fbdff9f8740 1 --2- 172.21.15.119:0/1550337027 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b90ef75ba0 0x55b90ef75fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:13.317 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.308+0000 7fbdff9f8740 1 -- 172.21.15.119:0/1550337027 >> 172.21.15.119:0/1550337027 conn(0x55b90ef71110 msgr2=0x55b90ef71510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:26:13.317 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.308+0000 7fbdff9f8740 1 -- 172.21.15.119:0/1550337027 shutdown_connections 2024-03-28T00:26:13.317 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.308+0000 7fbdff9f8740 1 -- 172.21.15.119:0/1550337027 wait complete. 2024-03-28T00:26:13.321 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.309+0000 7fbdff9f8740 1 -- start start 2024-03-28T00:26:13.321 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.309+0000 7fbdff9f8740 1 Processor -- start 2024-03-28T00:26:13.321 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.309+0000 7fbdff9f8740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b90ef75ba0 0x55b90ef80090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:13.321 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.309+0000 7fbdff9f8740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b90ef76590 0x55b90ef805d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:13.321 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.309+0000 7fbdff9f8740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b90ef7b700 0x55b90ef87610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:13.321 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.309+0000 7fbdff9f8740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b90ef7e6b0 con 0x55b90ef7b700 2024-03-28T00:26:13.321 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.309+0000 7fbdff9f8740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b90ee96dd0 con 0x55b90ef76590 2024-03-28T00:26:13.321 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.309+0000 7fbdff9f8740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b90ef7e830 con 0x55b90ef75ba0 2024-03-28T00:26:13.321 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.309+0000 7fbdff9f8740 10 client.0 _ll_register_callbacks cb 0x55b90ebeeac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T00:26:13.321 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.309+0000 7fbded400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b90ef76590 0x55b90ef805d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:13.321 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.309+0000 7fbdf5e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b90ef75ba0 0x55b90ef80090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:13.322 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.309+0000 7fbded400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b90ef76590 0x55b90ef805d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.119:44146/0 (socket says 172.21.15.119:44146) 2024-03-28T00:26:13.322 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.309+0000 7fbded400640 1 -- 172.21.15.119:0/4168431212 learned_addr learned my addr 172.21.15.119:0/4168431212 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:26:13.322 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.309+0000 7fbdf6800640 1 --2- 172.21.15.119:0/4168431212 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b90ef7b700 0x55b90ef87610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:13.322 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.310+0000 7fbdf5e00640 1 -- 172.21.15.119:0/4168431212 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b90ef76590 msgr2=0x55b90ef805d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T00:26:13.322 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.310+0000 7fbdf5e00640 1 --2- 172.21.15.119:0/4168431212 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b90ef76590 0x55b90ef805d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:13.322 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.310+0000 7fbdf5e00640 1 -- 172.21.15.119:0/4168431212 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b90ef7b700 msgr2=0x55b90ef87610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T00:26:13.322 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.310+0000 7fbdf5e00640 1 --2- 172.21.15.119:0/4168431212 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b90ef7b700 0x55b90ef87610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:13.322 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.310+0000 7fbdf5e00640 1 -- 172.21.15.119:0/4168431212 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b90ebef2d0 con 0x55b90ef75ba0 2024-03-28T00:26:13.322 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.310+0000 7fbded400640 1 --2- 172.21.15.119:0/4168431212 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b90ef76590 0x55b90ef805d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T00:26:13.322 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.310+0000 7fbdf5e00640 1 --2- 172.21.15.119:0/4168431212 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b90ef75ba0 0x55b90ef80090 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fbde80058c0 tx=0x7fbde8003e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1080ae8e9e8636a server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:26:13.322 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.310+0000 7fbddbe00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-03-28T00:26:13.322 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.310+0000 7fbddbe00640 1 -- 172.21.15.119:0/4168431212 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbde809a430 con 0x55b90ef75ba0 2024-03-28T00:26:13.322 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.310+0000 7fbddbe00640 1 -- 172.21.15.119:0/4168431212 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fbde80cc9a0 con 0x55b90ef75ba0 2024-03-28T00:26:13.322 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.310+0000 7fbdff9f8740 1 -- 172.21.15.119:0/4168431212 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbde80d0030 con 0x55b90ef75ba0 2024-03-28T00:26:13.323 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.310+0000 7fbddbe00640 1 -- 172.21.15.119:0/4168431212 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbde809ab80 con 0x55b90ef75ba0 2024-03-28T00:26:13.323 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.310+0000 7fbdff9f8740 10 client.25195 fetch_fsmap learned FSMap version 26 2024-03-28T00:26:13.323 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.310+0000 7fbdff9f8740 1 -- 172.21.15.119:0/4168431212 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55b90ecb96b0 con 0x55b90ef75ba0 2024-03-28T00:26:13.323 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.311+0000 7fbddbe00640 1 -- 172.21.15.119:0/4168431212 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbde80d0030 con 0x55b90ef75ba0 2024-03-28T00:26:13.323 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.311+0000 7fbddbe00640 1 -- 172.21.15.119:0/4168431212 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 26) v1 ==== 7934+0+0 (secure 0 0 0) 0x7fbde80de070 con 0x55b90ef75ba0 2024-03-28T00:26:13.323 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.311+0000 7fbdff9f8740 10 client.25195 fetch_fsmap finished waiting for FSMap version 26 2024-03-28T00:26:13.323 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.311+0000 7fbdff9f8740 10 client.25195 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-28T00:26:13.323 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.311+0000 7fbdff9f8740 20 client.25195 populate_metadata read hostname 'smithi119' 2024-03-28T00:26:13.323 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.311+0000 7fbdff9f8740 1 --2- 172.21.15.119:0/4168431212 >> [v2:172.21.15.165:6836/1531193032,v1:172.21.15.165:6837/1531193032] conn(0x55b90ef8c260 0x55b90ef906e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:13.323 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.311+0000 7fbdff9f8740 4 client.25195 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-28T00:26:13.323 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.311+0000 7fbdff9f8740 1 -- 172.21.15.119:0/4168431212 --> [v2:172.21.15.165:6836/1531193032,v1:172.21.15.165:6837/1531193032] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b90ee7c520 con 0x55b90ef8c260 2024-03-28T00:26:13.323 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.311+0000 7fbded400640 1 --2- 172.21.15.119:0/4168431212 >> [v2:172.21.15.165:6836/1531193032,v1:172.21.15.165:6837/1531193032] conn(0x55b90ef8c260 0x55b90ef906e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:13.323 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.312+0000 7fbded400640 1 --2- 172.21.15.119:0/4168431212 >> [v2:172.21.15.165:6836/1531193032,v1:172.21.15.165:6837/1531193032] conn(0x55b90ef8c260 0x55b90ef906e0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=69d679f8586ade7e server_cookie=99848cab344f587 in_seq=0 out_seq=0 2024-03-28T00:26:13.323 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.312+0000 7fbddbe00640 10 client.25195 ms_handle_connect on v2:172.21.15.165:6836/1531193032 2024-03-28T00:26:13.323 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.313+0000 7fbddbe00640 1 -- 172.21.15.119:0/4168431212 <== mds.3 v2:172.21.15.165:6836/1531193032 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55b90ee7c520 con 0x55b90ef8c260 2024-03-28T00:26:13.324 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.313+0000 7fbddbe00640 10 client.25195 handle_command_reply: tid=0 2024-03-28T00:26:13.324 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.313+0000 7fbdff9f8740 1 client.25195 shutdown 2024-03-28T00:26:13.324 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.313+0000 7fbdff9f8740 1 -- 172.21.15.119:0/4168431212 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b90ef75ba0 msgr2=0x55b90ef80090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:26:13.324 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.313+0000 7fbdff9f8740 1 --2- 172.21.15.119:0/4168431212 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b90ef75ba0 0x55b90ef80090 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fbde80058c0 tx=0x7fbde8003e30 comp rx=0 tx=0).stop 2024-03-28T00:26:13.324 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.313+0000 7fbdff9f8740 1 -- 172.21.15.119:0/4168431212 shutdown_connections 2024-03-28T00:26:13.324 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.313+0000 7fbdff9f8740 1 --2- 172.21.15.119:0/4168431212 >> [v2:172.21.15.165:6836/1531193032,v1:172.21.15.165:6837/1531193032] conn(0x55b90ef8c260 0x55b90ef906e0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:13.324 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.313+0000 7fbdff9f8740 1 --2- 172.21.15.119:0/4168431212 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b90ef7b700 0x55b90ef87610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:13.324 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.313+0000 7fbdff9f8740 1 --2- 172.21.15.119:0/4168431212 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b90ef76590 0x55b90ef805d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:13.324 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.313+0000 7fbdff9f8740 1 --2- 172.21.15.119:0/4168431212 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b90ef75ba0 0x55b90ef80090 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:13.324 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.313+0000 7fbddbe00640 0 client.25195 ms_handle_reset on v2:172.21.15.165:6836/1531193032 2024-03-28T00:26:13.324 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.313+0000 7fbdff9f8740 1 -- 172.21.15.119:0/4168431212 >> 172.21.15.119:0/4168431212 conn(0x55b90ef71110 msgr2=0x55b90ef70310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:26:13.324 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.313+0000 7fbdff9f8740 1 -- 172.21.15.119:0/4168431212 shutdown_connections 2024-03-28T00:26:13.324 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.313+0000 7fbdff9f8740 1 -- 172.21.15.119:0/4168431212 wait complete. 2024-03-28T00:26:13.324 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.313+0000 7fbdff9f8740 20 client.25195 trim_cache size 0 max 16384 2024-03-28T00:26:13.339 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.331+0000 7fbdff9f8740 1 Processor -- start 2024-03-28T00:26:13.339 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.331+0000 7fbdff9f8740 1 -- start start 2024-03-28T00:26:13.339 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.331+0000 7fbdff9f8740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b90ef75ba0 0x55b90ef714f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:13.339 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.331+0000 7fbdff9f8740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b90ef76590 0x55b90ef76f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:13.340 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.332+0000 7fbdff9f8740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b90ef7b700 0x55b90ef7fca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:13.340 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.332+0000 7fbdff9f8740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b90ef8ba80 con 0x55b90ef7b700 2024-03-28T00:26:13.340 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.332+0000 7fbdff9f8740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b90ef8c0e0 con 0x55b90ef75ba0 2024-03-28T00:26:13.340 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.332+0000 7fbdff9f8740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b90ef8c740 con 0x55b90ef76590 2024-03-28T00:26:13.340 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.332+0000 7fbdf5e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b90ef75ba0 0x55b90ef714f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:13.340 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.332+0000 7fbdf5e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b90ef75ba0 0x55b90ef714f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.119:44154/0 (socket says 172.21.15.119:44154) 2024-03-28T00:26:13.340 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.332+0000 7fbdf5e00640 1 -- 172.21.15.119:0/1507223437 learned_addr learned my addr 172.21.15.119:0/1507223437 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:26:13.340 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.332+0000 7fbdf6800640 1 --2- 172.21.15.119:0/1507223437 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b90ef7b700 0x55b90ef7fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:13.340 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.332+0000 7fbded400640 1 --2- 172.21.15.119:0/1507223437 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b90ef76590 0x55b90ef76f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:13.340 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.332+0000 7fbdf5e00640 1 -- 172.21.15.119:0/1507223437 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b90ef76590 msgr2=0x55b90ef76f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T00:26:13.340 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.332+0000 7fbdf5e00640 1 --2- 172.21.15.119:0/1507223437 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b90ef76590 0x55b90ef76f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:13.340 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.332+0000 7fbdf5e00640 1 -- 172.21.15.119:0/1507223437 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b90ef7b700 msgr2=0x55b90ef7fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T00:26:13.340 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.332+0000 7fbdf5e00640 1 --2- 172.21.15.119:0/1507223437 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b90ef7b700 0x55b90ef7fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:13.341 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.332+0000 7fbdf5e00640 1 -- 172.21.15.119:0/1507223437 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b90ef7ad40 con 0x55b90ef75ba0 2024-03-28T00:26:13.341 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.332+0000 7fbded400640 1 --2- 172.21.15.119:0/1507223437 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b90ef76590 0x55b90ef76f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T00:26:13.341 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.332+0000 7fbdf5e00640 1 --2- 172.21.15.119:0/1507223437 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b90ef75ba0 0x55b90ef714f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fbde809b430 tx=0x7fbde8003ea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=177d43f9e3feee03 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:26:13.341 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.333+0000 7fbde6a00640 1 -- 172.21.15.119:0/1507223437 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbde809aeb0 con 0x55b90ef75ba0 2024-03-28T00:26:13.341 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.333+0000 7fbde6a00640 1 -- 172.21.15.119:0/1507223437 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbde809be30 con 0x55b90ef75ba0 2024-03-28T00:26:13.341 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.333+0000 7fbde6a00640 1 -- 172.21.15.119:0/1507223437 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbde809b090 con 0x55b90ef75ba0 2024-03-28T00:26:13.345 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.333+0000 7fbdff9f8740 1 -- 172.21.15.119:0/1507223437 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b90ef75ba0 msgr2=0x55b90ef714f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:26:13.345 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.333+0000 7fbdff9f8740 1 --2- 172.21.15.119:0/1507223437 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b90ef75ba0 0x55b90ef714f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fbde809b430 tx=0x7fbde8003ea0 comp rx=0 tx=0).stop 2024-03-28T00:26:13.345 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.333+0000 7fbdff9f8740 1 -- 172.21.15.119:0/1507223437 shutdown_connections 2024-03-28T00:26:13.346 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.333+0000 7fbdff9f8740 1 --2- 172.21.15.119:0/1507223437 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b90ef7b700 0x55b90ef7fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:13.346 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.333+0000 7fbdff9f8740 1 --2- 172.21.15.119:0/1507223437 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b90ef76590 0x55b90ef76f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:13.346 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.333+0000 7fbdff9f8740 1 --2- 172.21.15.119:0/1507223437 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b90ef75ba0 0x55b90ef714f0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:13.346 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.333+0000 7fbdff9f8740 1 -- 172.21.15.119:0/1507223437 >> 172.21.15.119:0/1507223437 conn(0x55b90ef71110 msgr2=0x55b90ef6be00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:26:13.346 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.333+0000 7fbdff9f8740 1 -- 172.21.15.119:0/1507223437 shutdown_connections 2024-03-28T00:26:13.346 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.334+0000 7fbdff9f8740 1 -- 172.21.15.119:0/1507223437 wait complete. 2024-03-28T00:26:13.346 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.334+0000 7fbdff9f8740 1 -- start start 2024-03-28T00:26:13.346 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.334+0000 7fbdff9f8740 1 Processor -- start 2024-03-28T00:26:13.346 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.334+0000 7fbdff9f8740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b90ef75ba0 0x55b90efe57f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:13.346 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.334+0000 7fbdff9f8740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b90ef76590 0x55b90efe7d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:13.346 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.334+0000 7fbdff9f8740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b90ef7b700 0x55b90efea290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:13.346 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.334+0000 7fbdff9f8740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b90ef8d400 con 0x55b90ef76590 2024-03-28T00:26:13.346 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.334+0000 7fbdff9f8740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b90ef8cda0 con 0x55b90ef75ba0 2024-03-28T00:26:13.346 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.334+0000 7fbdff9f8740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b90ef8da60 con 0x55b90ef7b700 2024-03-28T00:26:13.347 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.335+0000 7fbdff9f8740 10 client.0 _ll_register_callbacks cb 0x7fbde8005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T00:26:13.347 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.335+0000 7fbdf5e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b90ef75ba0 0x55b90efe57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:13.347 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.335+0000 7fbdf5e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b90ef75ba0 0x55b90efe57f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.119:44168/0 (socket says 172.21.15.119:44168) 2024-03-28T00:26:13.347 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.335+0000 7fbdf5e00640 1 -- 172.21.15.119:0/4104745731 learned_addr learned my addr 172.21.15.119:0/4104745731 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:26:13.347 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.335+0000 7fbded400640 1 --2- 172.21.15.119:0/4104745731 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b90ef76590 0x55b90efe7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:13.347 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.335+0000 7fbdf6800640 1 --2- 172.21.15.119:0/4104745731 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b90ef7b700 0x55b90efea290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:13.347 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.335+0000 7fbdf5e00640 1 -- 172.21.15.119:0/4104745731 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b90ef7b700 msgr2=0x55b90efea290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T00:26:13.347 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.335+0000 7fbdf5e00640 1 --2- 172.21.15.119:0/4104745731 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b90ef7b700 0x55b90efea290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:13.347 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.335+0000 7fbdf5e00640 1 -- 172.21.15.119:0/4104745731 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b90ef76590 msgr2=0x55b90efe7d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T00:26:13.347 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.335+0000 7fbdf5e00640 1 --2- 172.21.15.119:0/4104745731 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b90ef76590 0x55b90efe7d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:13.347 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.335+0000 7fbdf5e00640 1 -- 172.21.15.119:0/4104745731 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b90efea850 con 0x55b90ef75ba0 2024-03-28T00:26:13.347 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.335+0000 7fbdf6800640 1 --2- 172.21.15.119:0/4104745731 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b90ef7b700 0x55b90efea290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T00:26:13.347 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.335+0000 7fbdf5e00640 1 --2- 172.21.15.119:0/4104745731 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b90ef75ba0 0x55b90efe57f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fbde80b1040 tx=0x7fbde809ad90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3bb3f2a6d73059d0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:26:13.348 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.335+0000 7fbddbe00640 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-03-28T00:26:13.348 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.335+0000 7fbddbe00640 1 -- 172.21.15.119:0/4104745731 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbde809b270 con 0x55b90ef75ba0 2024-03-28T00:26:13.348 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.335+0000 7fbddbe00640 1 -- 172.21.15.119:0/4104745731 <== mon.2 v2:172.21.15.132:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fbde809bb90 con 0x55b90ef75ba0 2024-03-28T00:26:13.348 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.335+0000 7fbdff9f8740 1 -- 172.21.15.119:0/4104745731 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbde809be30 con 0x55b90ef75ba0 2024-03-28T00:26:13.348 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.336+0000 7fbddbe00640 1 -- 172.21.15.119:0/4104745731 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbde809b500 con 0x55b90ef75ba0 2024-03-28T00:26:13.348 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.336+0000 7fbdff9f8740 10 client.25190 fetch_fsmap learned FSMap version 26 2024-03-28T00:26:13.348 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.336+0000 7fbdff9f8740 1 -- 172.21.15.119:0/4104745731 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55b90efea850 con 0x55b90ef75ba0 2024-03-28T00:26:13.348 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.336+0000 7fbddbe00640 1 -- 172.21.15.119:0/4104745731 <== mon.2 v2:172.21.15.132:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbde809be30 con 0x55b90ef75ba0 2024-03-28T00:26:13.348 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.336+0000 7fbddbe00640 1 -- 172.21.15.119:0/4104745731 <== mon.2 v2:172.21.15.132:3300/0 5 ==== fsmap(e 26) v1 ==== 7934+0+0 (secure 0 0 0) 0x7fbde80de070 con 0x55b90ef75ba0 2024-03-28T00:26:13.348 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.336+0000 7fbdff9f8740 10 client.25190 fetch_fsmap finished waiting for FSMap version 26 2024-03-28T00:26:13.348 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.336+0000 7fbdff9f8740 10 client.25190 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-28T00:26:13.348 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.336+0000 7fbdff9f8740 20 client.25190 populate_metadata read hostname 'smithi119' 2024-03-28T00:26:13.348 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.336+0000 7fbdff9f8740 1 --2- 172.21.15.119:0/4104745731 >> [v2:172.21.15.165:6836/1531193032,v1:172.21.15.165:6837/1531193032] conn(0x55b90efefbe0 0x55b90efebbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:13.348 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.336+0000 7fbdff9f8740 4 client.25190 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-28T00:26:13.348 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.336+0000 7fbdff9f8740 1 -- 172.21.15.119:0/4104745731 --> [v2:172.21.15.165:6836/1531193032,v1:172.21.15.165:6837/1531193032] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55b90eee81d0 con 0x55b90efefbe0 2024-03-28T00:26:13.349 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.336+0000 7fbded400640 1 --2- 172.21.15.119:0/4104745731 >> [v2:172.21.15.165:6836/1531193032,v1:172.21.15.165:6837/1531193032] conn(0x55b90efefbe0 0x55b90efebbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:13.349 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.337+0000 7fbded400640 1 --2- 172.21.15.119:0/4104745731 >> [v2:172.21.15.165:6836/1531193032,v1:172.21.15.165:6837/1531193032] conn(0x55b90efefbe0 0x55b90efebbb0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ec7a324fe3153fd4 server_cookie=635b13555339c3ba in_seq=0 out_seq=0 2024-03-28T00:26:13.349 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.337+0000 7fbddbe00640 10 client.25190 ms_handle_connect on v2:172.21.15.165:6836/1531193032 2024-03-28T00:26:13.349 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.337+0000 7fbddbe00640 1 -- 172.21.15.119:0/4104745731 <== mds.3 v2:172.21.15.165:6836/1531193032 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+322 (crc 0 0 0) 0x55b90eee81d0 con 0x55b90efefbe0 2024-03-28T00:26:13.349 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.337+0000 7fbddbe00640 10 client.25190 handle_command_reply: tid=0 2024-03-28T00:26:13.349 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.337+0000 7fbdff9f8740 1 client.25190 shutdown 2024-03-28T00:26:13.349 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.337+0000 7fbdff9f8740 1 -- 172.21.15.119:0/4104745731 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b90ef75ba0 msgr2=0x55b90efe57f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:26:13.349 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.337+0000 7fbdff9f8740 1 --2- 172.21.15.119:0/4104745731 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b90ef75ba0 0x55b90efe57f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fbde80b1040 tx=0x7fbde809ad90 comp rx=0 tx=0).stop 2024-03-28T00:26:13.349 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.337+0000 7fbdff9f8740 1 -- 172.21.15.119:0/4104745731 shutdown_connections 2024-03-28T00:26:13.349 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.337+0000 7fbdff9f8740 1 --2- 172.21.15.119:0/4104745731 >> [v2:172.21.15.165:6836/1531193032,v1:172.21.15.165:6837/1531193032] conn(0x55b90efefbe0 0x55b90efebbb0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:13.349 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.337+0000 7fbdff9f8740 1 --2- 172.21.15.119:0/4104745731 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b90ef7b700 0x55b90efea290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:13.349 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.337+0000 7fbdff9f8740 1 --2- 172.21.15.119:0/4104745731 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b90ef76590 0x55b90efe7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:13.349 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.337+0000 7fbddbe00640 0 client.25190 ms_handle_reset on v2:172.21.15.165:6836/1531193032 2024-03-28T00:26:13.349 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.337+0000 7fbdff9f8740 1 --2- 172.21.15.119:0/4104745731 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b90ef75ba0 0x55b90efe57f0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:13.350 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.337+0000 7fbdff9f8740 1 -- 172.21.15.119:0/4104745731 >> 172.21.15.119:0/4104745731 conn(0x55b90ef71110 msgr2=0x55b90ef714f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:26:13.350 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.338+0000 7fbdff9f8740 1 -- 172.21.15.119:0/4104745731 shutdown_connections 2024-03-28T00:26:13.350 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.338+0000 7fbdff9f8740 1 -- 172.21.15.119:0/4104745731 wait complete. 2024-03-28T00:26:13.350 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.338+0000 7fbdff9f8740 20 client.25190 trim_cache size 0 max 16384 2024-03-28T00:26:13.350 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-03-28T00:26:13.350 INFO:teuthology.orchestra.run.smithi119.stdout: "cluster_fsid": "10f73b92-ec98-11ee-95d0-87774f69a715", 2024-03-28T00:26:13.350 INFO:teuthology.orchestra.run.smithi119.stdout: "whoami": 3, 2024-03-28T00:26:13.350 INFO:teuthology.orchestra.run.smithi119.stdout: "id": 24523, 2024-03-28T00:26:13.350 INFO:teuthology.orchestra.run.smithi119.stdout: "want_state": "up:active", 2024-03-28T00:26:13.350 INFO:teuthology.orchestra.run.smithi119.stdout: "state": "up:active", 2024-03-28T00:26:13.351 INFO:teuthology.orchestra.run.smithi119.stdout: "fs_name": "cephfs", 2024-03-28T00:26:13.351 INFO:teuthology.orchestra.run.smithi119.stdout: "rank_uptime": 130.828776076, 2024-03-28T00:26:13.351 INFO:teuthology.orchestra.run.smithi119.stdout: "mdsmap_epoch": 26, 2024-03-28T00:26:13.351 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_epoch": 87, 2024-03-28T00:26:13.351 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_epoch_barrier": 85, 2024-03-28T00:26:13.351 INFO:teuthology.orchestra.run.smithi119.stdout: "uptime": 159.906583071 2024-03-28T00:26:13.351 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-03-28T00:26:13.352 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.338+0000 7fbdfce00640 1 -- 172.21.15.119:0/673352738 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fbddc0a2880 msgr2=0x7fbddc0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:26:13.352 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.338+0000 7fbdfce00640 1 --2- 172.21.15.119:0/673352738 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fbddc0a2880 0x7fbddc0a4d40 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fbdf002c630 tx=0x7fbdf0032590 comp rx=0 tx=0).stop 2024-03-28T00:26:13.352 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.338+0000 7fbdfce00640 1 -- 172.21.15.119:0/673352738 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbdf81016b0 msgr2=0x7fbdf8197930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:26:13.352 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.338+0000 7fbdfce00640 1 --2- 172.21.15.119:0/673352738 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbdf81016b0 0x7fbdf8197930 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fbde800e9c0 tx=0x7fbde800ee80 comp rx=0 tx=0).stop 2024-03-28T00:26:13.352 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.338+0000 7fbdfce00640 1 -- 172.21.15.119:0/673352738 shutdown_connections 2024-03-28T00:26:13.352 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.339+0000 7fbdfce00640 1 --2- 172.21.15.119:0/673352738 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fbddc0a2880 0x7fbddc0a4d40 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:13.352 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.339+0000 7fbdfce00640 1 --2- 172.21.15.119:0/673352738 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbdf8103260 0x7fbdf819eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:13.352 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.339+0000 7fbdfce00640 1 --2- 172.21.15.119:0/673352738 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbdf81028a0 0x7fbdf8197e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:13.352 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.339+0000 7fbdfce00640 1 --2- 172.21.15.119:0/673352738 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbdf81016b0 0x7fbdf8197930 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:13.353 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.339+0000 7fbdfce00640 1 -- 172.21.15.119:0/673352738 >> 172.21.15.119:0/673352738 conn(0x7fbdf80fcc40 msgr2=0x7fbdf8105bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:26:13.353 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.339+0000 7fbdfce00640 1 -- 172.21.15.119:0/673352738 shutdown_connections 2024-03-28T00:26:13.353 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.339+0000 7fbdfce00640 1 -- 172.21.15.119:0/673352738 wait complete. 2024-03-28T00:26:13.364 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-03-28T00:26:13.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:12 smithi165 ceph-mon[36302]: pgmap v381: 97 pgs: 97 active+clean; 648 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T00:26:13.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:12 smithi165 ceph-mon[36302]: from='client.? 172.21.15.119:0/4186951470' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T00:26:13.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:12 smithi165 ceph-mon[36302]: from='client.? 172.21.15.119:0/2808613955' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-28T00:26:13.452 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:12 smithi132 ceph-mon[35941]: pgmap v381: 97 pgs: 97 active+clean; 648 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T00:26:13.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:12 smithi132 ceph-mon[35941]: from='client.? 172.21.15.119:0/4186951470' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T00:26:13.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:12 smithi132 ceph-mon[35941]: from='client.? 172.21.15.119:0/2808613955' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-28T00:26:13.575 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.568+0000 7f0151200640 1 -- 172.21.15.119:0/277874435 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f014c075660 msgr2=0x7f014c073ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:26:13.575 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.568+0000 7f0151200640 1 --2- 172.21.15.119:0/277874435 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f014c075660 0x7f014c073ab0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f013c009900 tx=0x7f013c02bcd0 comp rx=0 tx=0).stop 2024-03-28T00:26:13.575 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.568+0000 7f0151200640 1 -- 172.21.15.119:0/277874435 shutdown_connections 2024-03-28T00:26:13.575 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.568+0000 7f0151200640 1 --2- 172.21.15.119:0/277874435 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f014c109430 0x7f014c10b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:13.575 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.568+0000 7f0151200640 1 --2- 172.21.15.119:0/277874435 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f014c074080 0x7f014c0744e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:13.575 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.568+0000 7f0151200640 1 --2- 172.21.15.119:0/277874435 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f014c075660 0x7f014c073ab0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:13.575 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.568+0000 7f0151200640 1 -- 172.21.15.119:0/277874435 >> 172.21.15.119:0/277874435 conn(0x7f014c0fcb10 msgr2=0x7f014c0fef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:26:13.575 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.568+0000 7f0151200640 1 -- 172.21.15.119:0/277874435 shutdown_connections 2024-03-28T00:26:13.575 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.568+0000 7f0151200640 1 -- 172.21.15.119:0/277874435 wait complete. 2024-03-28T00:26:13.576 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.570+0000 7f0151200640 1 Processor -- start 2024-03-28T00:26:13.576 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.570+0000 7f0151200640 1 -- start start 2024-03-28T00:26:13.578 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.570+0000 7f0151200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f014c074080 0x7f014c199a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:13.578 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.570+0000 7f0151200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f014c075660 0x7f014c199fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:13.578 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.570+0000 7f0151200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f014c109430 0x7f014c1a1010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:13.578 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.570+0000 7f0151200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f014c10f430 con 0x7f014c109430 2024-03-28T00:26:13.578 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.570+0000 7f0151200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f014c10f2b0 con 0x7f014c075660 2024-03-28T00:26:13.578 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.570+0000 7f0151200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f014c10f5b0 con 0x7f014c074080 2024-03-28T00:26:13.578 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.570+0000 7f014b400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f014c074080 0x7f014c199a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:13.578 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.570+0000 7f014b400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f014c074080 0x7f014c199a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.119:33574/0 (socket says 172.21.15.119:33574) 2024-03-28T00:26:13.578 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.570+0000 7f014b400640 1 -- 172.21.15.119:0/3123098116 learned_addr learned my addr 172.21.15.119:0/3123098116 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:26:13.578 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.570+0000 7f0143e00640 1 --2- 172.21.15.119:0/3123098116 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f014c075660 0x7f014c199fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:13.579 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.570+0000 7f014be00640 1 --2- 172.21.15.119:0/3123098116 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f014c109430 0x7f014c1a1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:13.579 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.571+0000 7f014b400640 1 -- 172.21.15.119:0/3123098116 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f014c075660 msgr2=0x7f014c199fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:26:13.579 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.571+0000 7f014b400640 1 --2- 172.21.15.119:0/3123098116 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f014c075660 0x7f014c199fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:13.579 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.571+0000 7f014b400640 1 -- 172.21.15.119:0/3123098116 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f014c109430 msgr2=0x7f014c1a1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:26:13.579 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.571+0000 7f014b400640 1 --2- 172.21.15.119:0/3123098116 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f014c109430 0x7f014c1a1010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:13.579 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.571+0000 7f014b400640 1 -- 172.21.15.119:0/3123098116 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f014c1a1680 con 0x7f014c074080 2024-03-28T00:26:13.579 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.571+0000 7f0143e00640 1 --2- 172.21.15.119:0/3123098116 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f014c075660 0x7f014c199fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T00:26:13.579 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.571+0000 7f014b400640 1 --2- 172.21.15.119:0/3123098116 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f014c074080 0x7f014c199a60 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f013c000c00 tx=0x7f013c02c6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:26:13.579 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.571+0000 7f0148c00640 1 -- 172.21.15.119:0/3123098116 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f013c043020 con 0x7f014c074080 2024-03-28T00:26:13.579 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.571+0000 7f0148c00640 1 -- 172.21.15.119:0/3123098116 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f013c0054e0 con 0x7f014c074080 2024-03-28T00:26:13.579 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.571+0000 7f0151200640 1 -- 172.21.15.119:0/3123098116 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f014c1a1910 con 0x7f014c074080 2024-03-28T00:26:13.579 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.571+0000 7f0148c00640 1 -- 172.21.15.119:0/3123098116 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f013c003c20 con 0x7f014c074080 2024-03-28T00:26:13.579 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.572+0000 7f0151200640 1 -- 172.21.15.119:0/3123098116 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f014c1a1eb0 con 0x7f014c074080 2024-03-28T00:26:13.582 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.575+0000 7f0155592740 1 Processor -- start 2024-03-28T00:26:13.583 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.575+0000 7f0155592740 1 -- start start 2024-03-28T00:26:13.583 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.575+0000 7f0148c00640 1 -- 172.21.15.119:0/3123098116 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 18) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f013c046020 con 0x7f014c074080 2024-03-28T00:26:13.583 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.575+0000 7f0155592740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560970ce6d80 0x560970ce71a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:13.583 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.575+0000 7f0155592740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560970ce7770 0x560970cec270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:13.583 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.575+0000 7f0155592740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560970cec8e0 0x560970ceed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:13.583 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.575+0000 7f0155592740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x560970c4bc90 con 0x560970ce7770 2024-03-28T00:26:13.583 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.575+0000 7f0155592740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x560970bf4980 con 0x560970cec8e0 2024-03-28T00:26:13.583 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.575+0000 7f0155592740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x560970c47d30 con 0x560970ce6d80 2024-03-28T00:26:13.583 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.575+0000 7f014be00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560970ce6d80 0x560970ce71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:13.583 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.575+0000 7f014be00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560970ce6d80 0x560970ce71a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.119:33576/0 (socket says 172.21.15.119:33576) 2024-03-28T00:26:13.583 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.575+0000 7f014be00640 1 -- 172.21.15.119:0/753657581 learned_addr learned my addr 172.21.15.119:0/753657581 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:26:13.583 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.575+0000 7f0143e00640 1 --2- 172.21.15.119:0/753657581 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560970cec8e0 0x560970ceed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:13.583 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.575+0000 7f014b400640 1 --2- 172.21.15.119:0/753657581 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560970ce7770 0x560970cec270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:13.584 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.576+0000 7f014be00640 1 -- 172.21.15.119:0/753657581 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560970cec8e0 msgr2=0x560970ceed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T00:26:13.584 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.576+0000 7f014be00640 1 --2- 172.21.15.119:0/753657581 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560970cec8e0 0x560970ceed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:13.584 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.576+0000 7f014be00640 1 -- 172.21.15.119:0/753657581 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560970ce7770 msgr2=0x560970cec270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T00:26:13.584 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.576+0000 7f014be00640 1 --2- 172.21.15.119:0/753657581 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560970ce7770 0x560970cec270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:13.584 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.576+0000 7f014be00640 1 -- 172.21.15.119:0/753657581 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560970c0c510 con 0x560970ce6d80 2024-03-28T00:26:13.584 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.576+0000 7f014b400640 1 --2- 172.21.15.119:0/753657581 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560970ce7770 0x560970cec270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T00:26:13.584 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.576+0000 7f014be00640 1 --2- 172.21.15.119:0/753657581 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560970ce6d80 0x560970ce71a0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f0144009fd0 tx=0x7f01440118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6963297634e31274 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:26:13.584 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.576+0000 7f0140c00640 1 -- 172.21.15.119:0/753657581 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0144004d20 con 0x560970ce6d80 2024-03-28T00:26:13.584 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.576+0000 7f0140c00640 1 -- 172.21.15.119:0/753657581 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0144004050 con 0x560970ce6d80 2024-03-28T00:26:13.584 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.576+0000 7f0140c00640 1 -- 172.21.15.119:0/753657581 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0144005220 con 0x560970ce6d80 2024-03-28T00:26:13.584 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.576+0000 7f0148c00640 1 --2- 172.21.15.119:0/3123098116 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f01300a2880 0x7f01300a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:13.589 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.577+0000 7f0148c00640 1 -- 172.21.15.119:0/3123098116 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f013c0f0a40 con 0x7f014c074080 2024-03-28T00:26:13.589 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.577+0000 7f014b400640 1 --2- 172.21.15.119:0/3123098116 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f01300a2880 0x7f01300a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:13.589 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.577+0000 7f0155592740 1 -- 172.21.15.119:0/753657581 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560970ce6d80 msgr2=0x560970ce71a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:26:13.589 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.577+0000 7f0155592740 1 --2- 172.21.15.119:0/753657581 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560970ce6d80 0x560970ce71a0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f0144009fd0 tx=0x7f01440118d0 comp rx=0 tx=0).stop 2024-03-28T00:26:13.590 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.577+0000 7f0155592740 1 -- 172.21.15.119:0/753657581 shutdown_connections 2024-03-28T00:26:13.590 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.577+0000 7f0155592740 1 --2- 172.21.15.119:0/753657581 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560970cec8e0 0x560970ceed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:13.590 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.577+0000 7f0155592740 1 --2- 172.21.15.119:0/753657581 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560970ce7770 0x560970cec270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:13.590 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.577+0000 7f0155592740 1 --2- 172.21.15.119:0/753657581 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560970ce6d80 0x560970ce71a0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:13.590 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.577+0000 7f0155592740 1 -- 172.21.15.119:0/753657581 >> 172.21.15.119:0/753657581 conn(0x560970ce22f0 msgr2=0x560970ce26f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:26:13.590 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.577+0000 7f0155592740 1 -- 172.21.15.119:0/753657581 shutdown_connections 2024-03-28T00:26:13.590 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.577+0000 7f0155592740 1 -- 172.21.15.119:0/753657581 wait complete. 2024-03-28T00:26:13.590 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.577+0000 7f014b400640 1 --2- 172.21.15.119:0/3123098116 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f01300a2880 0x7f01300a4d40 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f013c0bd370 tx=0x7f013c0b9040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:26:13.590 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.578+0000 7f0155592740 1 -- start start 2024-03-28T00:26:13.590 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.578+0000 7f0155592740 1 Processor -- start 2024-03-28T00:26:13.590 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.578+0000 7f0155592740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560970ce6d80 0x560970cf1230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:13.590 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.578+0000 7f0155592740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560970ce7770 0x560970cf1770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:13.590 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.578+0000 7f0155592740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560970cec8e0 0x560970cf87b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:13.590 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.578+0000 7f0155592740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x560970cef890 con 0x560970ce6d80 2024-03-28T00:26:13.591 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.578+0000 7f0155592740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x560970c07dd0 con 0x560970ce7770 2024-03-28T00:26:13.591 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.578+0000 7f0155592740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x560970cefa10 con 0x560970cec8e0 2024-03-28T00:26:13.591 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.578+0000 7f0155592740 10 client.0 _ll_register_callbacks cb 0x56097095fac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T00:26:13.591 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.579+0000 7f0143e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560970ce7770 0x560970cf1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:13.591 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.579+0000 7f014be00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560970ce6d80 0x560970cf1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:13.591 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.579+0000 7f0143e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560970ce7770 0x560970cf1770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.119:44200/0 (socket says 172.21.15.119:44200) 2024-03-28T00:26:13.591 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.579+0000 7f0143e00640 1 -- 172.21.15.119:0/1059493631 learned_addr learned my addr 172.21.15.119:0/1059493631 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:26:13.591 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.579+0000 7f014be00640 1 --2- 172.21.15.119:0/1059493631 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560970cec8e0 0x560970cf87b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:13.591 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.579+0000 7f0143e00640 1 -- 172.21.15.119:0/1059493631 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560970cec8e0 msgr2=0x560970cf87b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T00:26:13.591 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.579+0000 7f0143e00640 1 --2- 172.21.15.119:0/1059493631 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560970cec8e0 0x560970cf87b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:13.591 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.579+0000 7f0143e00640 1 -- 172.21.15.119:0/1059493631 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560970ce6d80 msgr2=0x560970cf1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T00:26:13.591 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.579+0000 7f0143e00640 1 --2- 172.21.15.119:0/1059493631 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560970ce6d80 0x560970cf1230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:13.591 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.579+0000 7f0143e00640 1 -- 172.21.15.119:0/1059493631 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5609709602d0 con 0x560970ce7770 2024-03-28T00:26:13.591 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.579+0000 7f014be00640 1 --2- 172.21.15.119:0/1059493631 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560970cec8e0 0x560970cf87b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T00:26:13.592 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.579+0000 7f0143e00640 1 --2- 172.21.15.119:0/1059493631 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560970ce7770 0x560970cf1770 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f0134012d00 tx=0x7f013400f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a875d1e9fc7c25a1 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:26:13.592 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.579+0000 7f0139600640 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-03-28T00:26:13.592 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.579+0000 7f0139600640 1 -- 172.21.15.119:0/1059493631 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0134009d90 con 0x560970ce7770 2024-03-28T00:26:13.592 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.580+0000 7f0139600640 1 -- 172.21.15.119:0/1059493631 <== mon.2 v2:172.21.15.132:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f0134012110 con 0x560970ce7770 2024-03-28T00:26:13.592 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.580+0000 7f0155592740 1 -- 172.21.15.119:0/1059493631 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0144004050 con 0x560970ce7770 2024-03-28T00:26:13.592 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.580+0000 7f0139600640 1 -- 172.21.15.119:0/1059493631 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0134004050 con 0x560970ce7770 2024-03-28T00:26:13.592 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.580+0000 7f0155592740 10 client.25202 fetch_fsmap learned FSMap version 26 2024-03-28T00:26:13.592 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.580+0000 7f0155592740 1 -- 172.21.15.119:0/1059493631 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x560970a2a6b0 con 0x560970ce7770 2024-03-28T00:26:13.592 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.580+0000 7f0139600640 1 -- 172.21.15.119:0/1059493631 <== mon.2 v2:172.21.15.132:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0144004050 con 0x560970ce7770 2024-03-28T00:26:13.592 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.580+0000 7f0139600640 1 -- 172.21.15.119:0/1059493631 <== mon.2 v2:172.21.15.132:3300/0 5 ==== fsmap(e 26) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f0134010430 con 0x560970ce7770 2024-03-28T00:26:13.592 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.580+0000 7f0155592740 10 client.25202 fetch_fsmap finished waiting for FSMap version 26 2024-03-28T00:26:13.592 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.580+0000 7f0155592740 10 client.25202 resolve_mds: resolved name 'h' to daemon mds.h 2024-03-28T00:26:13.592 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.580+0000 7f0155592740 20 client.25202 populate_metadata read hostname 'smithi119' 2024-03-28T00:26:13.592 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.580+0000 7f0155592740 1 --2- 172.21.15.119:0/1059493631 >> [v2:172.21.15.132:6836/3383082967,v1:172.21.15.132:6837/3383082967] conn(0x560970cfda10 0x560970cfcaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:13.593 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.580+0000 7f0155592740 4 client.25202 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-28T00:26:13.593 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.580+0000 7f0155592740 1 -- 172.21.15.119:0/1059493631 --> [v2:172.21.15.132:6836/3383082967,v1:172.21.15.132:6837/3383082967] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560970bed520 con 0x560970cfda10 2024-03-28T00:26:13.593 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.581+0000 7f014be00640 1 --2- 172.21.15.119:0/1059493631 >> [v2:172.21.15.132:6836/3383082967,v1:172.21.15.132:6837/3383082967] conn(0x560970cfda10 0x560970cfcaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:13.593 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.581+0000 7f014be00640 1 --2- 172.21.15.119:0/1059493631 >> [v2:172.21.15.132:6836/3383082967,v1:172.21.15.132:6837/3383082967] conn(0x560970cfda10 0x560970cfcaf0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=44af58fb9539b840 server_cookie=af55d9d11d49aa74 in_seq=0 out_seq=0 2024-03-28T00:26:13.593 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.581+0000 7f0139600640 10 client.25202 ms_handle_connect on v2:172.21.15.132:6836/3383082967 2024-03-28T00:26:13.593 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.582+0000 7f0139600640 1 -- 172.21.15.119:0/1059493631 <== mds.2 v2:172.21.15.132:6836/3383082967 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x560970bed520 con 0x560970cfda10 2024-03-28T00:26:13.593 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.582+0000 7f0139600640 10 client.25202 handle_command_reply: tid=0 2024-03-28T00:26:13.593 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.582+0000 7f0155592740 1 client.25202 shutdown 2024-03-28T00:26:13.593 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.582+0000 7f0155592740 1 -- 172.21.15.119:0/1059493631 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560970ce7770 msgr2=0x560970cf1770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:26:13.593 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.582+0000 7f0155592740 1 --2- 172.21.15.119:0/1059493631 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560970ce7770 0x560970cf1770 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f0134012d00 tx=0x7f013400f790 comp rx=0 tx=0).stop 2024-03-28T00:26:13.593 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.582+0000 7f0155592740 1 -- 172.21.15.119:0/1059493631 shutdown_connections 2024-03-28T00:26:13.593 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.582+0000 7f0155592740 1 --2- 172.21.15.119:0/1059493631 >> [v2:172.21.15.132:6836/3383082967,v1:172.21.15.132:6837/3383082967] conn(0x560970cfda10 0x560970cfcaf0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:13.593 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.582+0000 7f0155592740 1 --2- 172.21.15.119:0/1059493631 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560970cec8e0 0x560970cf87b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:13.593 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.582+0000 7f0155592740 1 --2- 172.21.15.119:0/1059493631 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560970ce7770 0x560970cf1770 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:13.594 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.582+0000 7f0155592740 1 --2- 172.21.15.119:0/1059493631 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560970ce6d80 0x560970cf1230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:13.594 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.582+0000 7f0139600640 0 client.25202 ms_handle_reset on v2:172.21.15.132:6836/3383082967 2024-03-28T00:26:13.594 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.582+0000 7f0155592740 1 -- 172.21.15.119:0/1059493631 >> 172.21.15.119:0/1059493631 conn(0x560970ce22f0 msgr2=0x560970ce14f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:26:13.594 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.582+0000 7f0155592740 1 -- 172.21.15.119:0/1059493631 shutdown_connections 2024-03-28T00:26:13.594 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.582+0000 7f0155592740 1 -- 172.21.15.119:0/1059493631 wait complete. 2024-03-28T00:26:13.594 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.582+0000 7f0155592740 20 client.25202 trim_cache size 0 max 16384 2024-03-28T00:26:13.608 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.600+0000 7f0155592740 1 Processor -- start 2024-03-28T00:26:13.608 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.601+0000 7f0155592740 1 -- start start 2024-03-28T00:26:13.608 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.601+0000 7f0155592740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560970ce6d80 0x560970cef770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:13.608 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.601+0000 7f0155592740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560970ce7770 0x560970cf8770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:13.608 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.601+0000 7f0155592740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560970cec8e0 0x560970cf16d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:13.608 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.601+0000 7f0155592740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x560970cfd230 con 0x560970ce7770 2024-03-28T00:26:13.608 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.601+0000 7f0155592740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x560970cfd890 con 0x560970cec8e0 2024-03-28T00:26:13.609 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.601+0000 7f0155592740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x560970cfdef0 con 0x560970ce6d80 2024-03-28T00:26:13.609 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.601+0000 7f014be00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560970ce6d80 0x560970cef770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:13.609 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.601+0000 7f014be00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560970ce6d80 0x560970cef770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.119:33600/0 (socket says 172.21.15.119:33600) 2024-03-28T00:26:13.609 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.601+0000 7f014be00640 1 -- 172.21.15.119:0/3992789341 learned_addr learned my addr 172.21.15.119:0/3992789341 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:26:13.609 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.601+0000 7f014be00640 1 --2- 172.21.15.119:0/3992789341 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560970cec8e0 0x560970cf16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:13.609 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.601+0000 7f0143e00640 1 --2- 172.21.15.119:0/3992789341 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560970ce7770 0x560970cf8770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:13.609 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.601+0000 7f014be00640 1 -- 172.21.15.119:0/3992789341 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560970cec8e0 msgr2=0x560970cf16d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T00:26:13.609 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.601+0000 7f014be00640 1 --2- 172.21.15.119:0/3992789341 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560970cec8e0 0x560970cf16d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:13.609 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.602+0000 7f014be00640 1 -- 172.21.15.119:0/3992789341 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560970ce7770 msgr2=0x560970cf8770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T00:26:13.609 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.602+0000 7f014be00640 1 --2- 172.21.15.119:0/3992789341 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560970ce7770 0x560970cf8770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:13.609 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.602+0000 7f014be00640 1 -- 172.21.15.119:0/3992789341 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560970cd4170 con 0x560970ce6d80 2024-03-28T00:26:13.609 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.602+0000 7f0143e00640 1 --2- 172.21.15.119:0/3992789341 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560970ce7770 0x560970cf8770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T00:26:13.609 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.602+0000 7f014be00640 1 --2- 172.21.15.119:0/3992789341 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560970ce6d80 0x560970cef770 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f0144009560 tx=0x7f0144009a50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=285124adad3a0f4e server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:26:13.609 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.602+0000 7f013be00640 1 -- 172.21.15.119:0/3992789341 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0144009be0 con 0x560970ce6d80 2024-03-28T00:26:13.610 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.602+0000 7f013be00640 1 -- 172.21.15.119:0/3992789341 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f014402cd30 con 0x560970ce6d80 2024-03-28T00:26:13.610 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.602+0000 7f013be00640 1 -- 172.21.15.119:0/3992789341 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0144004660 con 0x560970ce6d80 2024-03-28T00:26:13.614 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.603+0000 7f0155592740 1 -- 172.21.15.119:0/3992789341 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560970ce6d80 msgr2=0x560970cef770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:26:13.614 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.603+0000 7f0155592740 1 --2- 172.21.15.119:0/3992789341 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560970ce6d80 0x560970cef770 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f0144009560 tx=0x7f0144009a50 comp rx=0 tx=0).stop 2024-03-28T00:26:13.614 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.603+0000 7f0155592740 1 -- 172.21.15.119:0/3992789341 shutdown_connections 2024-03-28T00:26:13.614 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.603+0000 7f0155592740 1 --2- 172.21.15.119:0/3992789341 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560970cec8e0 0x560970cf16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:13.614 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.603+0000 7f0155592740 1 --2- 172.21.15.119:0/3992789341 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560970ce7770 0x560970cf8770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:13.614 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.603+0000 7f0155592740 1 --2- 172.21.15.119:0/3992789341 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560970ce6d80 0x560970cef770 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:13.614 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.603+0000 7f0155592740 1 -- 172.21.15.119:0/3992789341 >> 172.21.15.119:0/3992789341 conn(0x560970ce22f0 msgr2=0x560970cfcaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:26:13.614 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.603+0000 7f0155592740 1 -- 172.21.15.119:0/3992789341 shutdown_connections 2024-03-28T00:26:13.614 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.603+0000 7f0155592740 1 -- 172.21.15.119:0/3992789341 wait complete. 2024-03-28T00:26:13.615 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.603+0000 7f0155592740 1 -- start start 2024-03-28T00:26:13.615 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.603+0000 7f0155592740 1 Processor -- start 2024-03-28T00:26:13.615 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.604+0000 7f0155592740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560970ce6d80 0x560970cf69b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:13.615 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.604+0000 7f0155592740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560970ce7770 0x560970cf6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:13.615 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.604+0000 7f0155592740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560970cec8e0 0x560970cf7520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:13.615 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.604+0000 7f0155592740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x560970cfebb0 con 0x560970ce7770 2024-03-28T00:26:13.615 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.604+0000 7f0155592740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x560970cfe550 con 0x560970cec8e0 2024-03-28T00:26:13.615 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.604+0000 7f0155592740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x560970cff210 con 0x560970ce6d80 2024-03-28T00:26:13.615 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.604+0000 7f0155592740 10 client.0 _ll_register_callbacks cb 0x7f0134009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T00:26:13.615 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.604+0000 7f0143e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560970ce7770 0x560970cf6ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:13.615 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.604+0000 7f014be00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560970ce6d80 0x560970cf69b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:13.615 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.604+0000 7f014be00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560970ce6d80 0x560970cf69b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.119:33616/0 (socket says 172.21.15.119:33616) 2024-03-28T00:26:13.615 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.604+0000 7f0143e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560970ce7770 0x560970cf6ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41162/0 (socket says 172.21.15.119:41162) 2024-03-28T00:26:13.615 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.604+0000 7f014be00640 1 -- 172.21.15.119:0/373580855 learned_addr learned my addr 172.21.15.119:0/373580855 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:26:13.616 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.604+0000 7f014be00640 1 --2- 172.21.15.119:0/373580855 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560970cec8e0 0x560970cf7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:13.616 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.604+0000 7f0143e00640 1 -- 172.21.15.119:0/373580855 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560970ce6d80 msgr2=0x560970cf69b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T00:26:13.616 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.604+0000 7f0143e00640 1 --2- 172.21.15.119:0/373580855 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560970ce6d80 0x560970cf69b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:13.616 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.604+0000 7f0143e00640 1 -- 172.21.15.119:0/373580855 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560970cec8e0 msgr2=0x560970cf7520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T00:26:13.616 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.604+0000 7f0143e00640 1 --2- 172.21.15.119:0/373580855 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560970cec8e0 0x560970cf7520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:13.616 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.604+0000 7f0143e00640 1 -- 172.21.15.119:0/373580855 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560970cf7d50 con 0x560970ce7770 2024-03-28T00:26:13.616 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.605+0000 7f0143e00640 1 --2- 172.21.15.119:0/373580855 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560970ce7770 0x560970cf6ef0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f0134010820 tx=0x7f0134012870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bea5425aacbc900d server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:26:13.616 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.605+0000 7f0139600640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-03-28T00:26:13.616 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.605+0000 7f0139600640 1 -- 172.21.15.119:0/373580855 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0134012110 con 0x560970ce7770 2024-03-28T00:26:13.616 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.605+0000 7f0139600640 1 -- 172.21.15.119:0/373580855 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f0134010430 con 0x560970ce7770 2024-03-28T00:26:13.616 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.605+0000 7f0155592740 1 -- 172.21.15.119:0/373580855 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f014402cd30 con 0x560970ce7770 2024-03-28T00:26:13.616 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.605+0000 7f0139600640 1 -- 172.21.15.119:0/373580855 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f01340122b0 con 0x560970ce7770 2024-03-28T00:26:13.616 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.605+0000 7f0155592740 10 client.15108 fetch_fsmap learned FSMap version 26 2024-03-28T00:26:13.616 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.605+0000 7f0155592740 1 -- 172.21.15.119:0/373580855 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x560970d58e80 con 0x560970ce7770 2024-03-28T00:26:13.617 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.606+0000 7f0139600640 1 -- 172.21.15.119:0/373580855 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f014402cd30 con 0x560970ce7770 2024-03-28T00:26:13.617 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.606+0000 7f0139600640 1 -- 172.21.15.119:0/373580855 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 26) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f013403b070 con 0x560970ce7770 2024-03-28T00:26:13.617 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.606+0000 7f0155592740 10 client.15108 fetch_fsmap finished waiting for FSMap version 26 2024-03-28T00:26:13.617 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.606+0000 7f0155592740 10 client.15108 resolve_mds: resolved name 'h' to daemon mds.h 2024-03-28T00:26:13.617 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.606+0000 7f0155592740 20 client.15108 populate_metadata read hostname 'smithi119' 2024-03-28T00:26:13.617 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.606+0000 7f0155592740 1 --2- 172.21.15.119:0/373580855 >> [v2:172.21.15.132:6836/3383082967,v1:172.21.15.132:6837/3383082967] conn(0x560970d5d4d0 0x560970d59110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:13.617 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.606+0000 7f0155592740 4 client.15108 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-28T00:26:13.617 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.606+0000 7f0155592740 1 -- 172.21.15.119:0/373580855 --> [v2:172.21.15.132:6836/3383082967,v1:172.21.15.132:6837/3383082967] -- command(tid 0: {"prefix": "status"}) v1 -- 0x560970c591d0 con 0x560970d5d4d0 2024-03-28T00:26:13.617 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.606+0000 7f014be00640 1 --2- 172.21.15.119:0/373580855 >> [v2:172.21.15.132:6836/3383082967,v1:172.21.15.132:6837/3383082967] conn(0x560970d5d4d0 0x560970d59110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:13.617 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.607+0000 7f014be00640 1 --2- 172.21.15.119:0/373580855 >> [v2:172.21.15.132:6836/3383082967,v1:172.21.15.132:6837/3383082967] conn(0x560970d5d4d0 0x560970d59110 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=beff40b9d3eab4df server_cookie=29dfbe6e992045b1 in_seq=0 out_seq=0 2024-03-28T00:26:13.617 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.607+0000 7f0139600640 10 client.15108 ms_handle_connect on v2:172.21.15.132:6836/3383082967 2024-03-28T00:26:13.617 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.607+0000 7f0139600640 1 -- 172.21.15.119:0/373580855 <== mds.2 v2:172.21.15.132:6836/3383082967 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x560970c591d0 con 0x560970d5d4d0 2024-03-28T00:26:13.617 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.607+0000 7f0139600640 10 client.15108 handle_command_reply: tid=0 2024-03-28T00:26:13.617 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.607+0000 7f0155592740 1 client.15108 shutdown 2024-03-28T00:26:13.618 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.607+0000 7f0155592740 1 -- 172.21.15.119:0/373580855 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560970ce7770 msgr2=0x560970cf6ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:26:13.618 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.607+0000 7f0155592740 1 --2- 172.21.15.119:0/373580855 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560970ce7770 0x560970cf6ef0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f0134010820 tx=0x7f0134012870 comp rx=0 tx=0).stop 2024-03-28T00:26:13.618 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.607+0000 7f0155592740 1 -- 172.21.15.119:0/373580855 shutdown_connections 2024-03-28T00:26:13.618 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.607+0000 7f0155592740 1 --2- 172.21.15.119:0/373580855 >> [v2:172.21.15.132:6836/3383082967,v1:172.21.15.132:6837/3383082967] conn(0x560970d5d4d0 0x560970d59110 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:13.618 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.607+0000 7f0155592740 1 --2- 172.21.15.119:0/373580855 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560970cec8e0 0x560970cf7520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:13.618 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.607+0000 7f0155592740 1 --2- 172.21.15.119:0/373580855 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560970ce7770 0x560970cf6ef0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:13.618 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.607+0000 7f0155592740 1 --2- 172.21.15.119:0/373580855 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560970ce6d80 0x560970cf69b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:13.618 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.607+0000 7f0155592740 1 -- 172.21.15.119:0/373580855 >> 172.21.15.119:0/373580855 conn(0x560970ce22f0 msgr2=0x560970ce26d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:26:13.618 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.607+0000 7f0139600640 0 client.15108 ms_handle_reset on v2:172.21.15.132:6836/3383082967 2024-03-28T00:26:13.618 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.607+0000 7f0155592740 1 -- 172.21.15.119:0/373580855 shutdown_connections 2024-03-28T00:26:13.618 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.607+0000 7f0155592740 1 -- 172.21.15.119:0/373580855 wait complete. 2024-03-28T00:26:13.618 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.607+0000 7f0155592740 20 client.15108 trim_cache size 0 max 16384 2024-03-28T00:26:13.619 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-03-28T00:26:13.619 INFO:teuthology.orchestra.run.smithi119.stdout: "cluster_fsid": "10f73b92-ec98-11ee-95d0-87774f69a715", 2024-03-28T00:26:13.619 INFO:teuthology.orchestra.run.smithi119.stdout: "whoami": 2, 2024-03-28T00:26:13.619 INFO:teuthology.orchestra.run.smithi119.stdout: "id": 24530, 2024-03-28T00:26:13.619 INFO:teuthology.orchestra.run.smithi119.stdout: "want_state": "up:active", 2024-03-28T00:26:13.619 INFO:teuthology.orchestra.run.smithi119.stdout: "state": "up:active", 2024-03-28T00:26:13.619 INFO:teuthology.orchestra.run.smithi119.stdout: "fs_name": "cephfs", 2024-03-28T00:26:13.619 INFO:teuthology.orchestra.run.smithi119.stdout: "rank_uptime": 132.12022887399999, 2024-03-28T00:26:13.619 INFO:teuthology.orchestra.run.smithi119.stdout: "mdsmap_epoch": 26, 2024-03-28T00:26:13.619 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_epoch": 87, 2024-03-28T00:26:13.619 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_epoch_barrier": 85, 2024-03-28T00:26:13.619 INFO:teuthology.orchestra.run.smithi119.stdout: "uptime": 157.29717519299999 2024-03-28T00:26:13.620 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-03-28T00:26:13.620 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.608+0000 7f0151200640 1 -- 172.21.15.119:0/3123098116 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f01300a2880 msgr2=0x7f01300a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:26:13.620 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.608+0000 7f0151200640 1 --2- 172.21.15.119:0/3123098116 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f01300a2880 0x7f01300a4d40 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f013c0bd370 tx=0x7f013c0b9040 comp rx=0 tx=0).stop 2024-03-28T00:26:13.620 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.608+0000 7f0151200640 1 -- 172.21.15.119:0/3123098116 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f014c074080 msgr2=0x7f014c199a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:26:13.620 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.608+0000 7f0151200640 1 --2- 172.21.15.119:0/3123098116 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f014c074080 0x7f014c199a60 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f013c000c00 tx=0x7f013c02c6f0 comp rx=0 tx=0).stop 2024-03-28T00:26:13.621 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.608+0000 7f0151200640 1 -- 172.21.15.119:0/3123098116 shutdown_connections 2024-03-28T00:26:13.621 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.608+0000 7f0151200640 1 --2- 172.21.15.119:0/3123098116 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f01300a2880 0x7f01300a4d40 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:13.621 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.608+0000 7f0151200640 1 --2- 172.21.15.119:0/3123098116 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f014c109430 0x7f014c1a1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:13.621 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.608+0000 7f0151200640 1 --2- 172.21.15.119:0/3123098116 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f014c075660 0x7f014c199fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:13.621 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.608+0000 7f0151200640 1 --2- 172.21.15.119:0/3123098116 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f014c074080 0x7f014c199a60 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:13.621 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.608+0000 7f0151200640 1 -- 172.21.15.119:0/3123098116 >> 172.21.15.119:0/3123098116 conn(0x7f014c0fcb10 msgr2=0x7f014c1053c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:26:13.621 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.608+0000 7f0151200640 1 -- 172.21.15.119:0/3123098116 shutdown_connections 2024-03-28T00:26:13.621 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.608+0000 7f0151200640 1 -- 172.21.15.119:0/3123098116 wait complete. 2024-03-28T00:26:13.632 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-03-28T00:26:13.845 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.839+0000 7f08a2800640 1 -- 172.21.15.119:0/3261076638 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f089c105490 msgr2=0x7f089c10bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:26:13.846 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.839+0000 7f08a2800640 1 --2- 172.21.15.119:0/3261076638 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f089c105490 0x7f089c10bd20 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f0894009ca0 tx=0x7f089402c100 comp rx=0 tx=0).stop 2024-03-28T00:26:13.846 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.839+0000 7f08a2800640 1 -- 172.21.15.119:0/3261076638 shutdown_connections 2024-03-28T00:26:13.846 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.839+0000 7f08a2800640 1 --2- 172.21.15.119:0/3261076638 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f089c105490 0x7f089c10bd20 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:13.846 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.839+0000 7f08a2800640 1 --2- 172.21.15.119:0/3261076638 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f089c104ad0 0x7f089c104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:13.846 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.839+0000 7f08a2800640 1 --2- 172.21.15.119:0/3261076638 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f089c1038e0 0x7f089c103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:13.846 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.839+0000 7f08a2800640 1 -- 172.21.15.119:0/3261076638 >> 172.21.15.119:0/3261076638 conn(0x7f089c0fee50 msgr2=0x7f089c101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:26:13.846 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.839+0000 7f08a2800640 1 -- 172.21.15.119:0/3261076638 shutdown_connections 2024-03-28T00:26:13.846 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.839+0000 7f08a2800640 1 -- 172.21.15.119:0/3261076638 wait complete. 2024-03-28T00:26:13.848 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.840+0000 7f08a2800640 1 Processor -- start 2024-03-28T00:26:13.848 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.840+0000 7f08a2800640 1 -- start start 2024-03-28T00:26:13.848 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.840+0000 7f08a2800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f089c1038e0 0x7f089c199b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:13.848 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.840+0000 7f08a2800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f089c104ad0 0x7f089c19a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:13.848 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.840+0000 7f08a2800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f089c105490 0x7f089c1a1130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:13.848 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.840+0000 7f08a2800640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f089c10f540 con 0x7f089c105490 2024-03-28T00:26:13.848 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.840+0000 7f08a2800640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f089c10f3c0 con 0x7f089c104ad0 2024-03-28T00:26:13.848 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.840+0000 7f08a2800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f089c10f6c0 con 0x7f089c1038e0 2024-03-28T00:26:13.848 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.841+0000 7f089b400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f089c1038e0 0x7f089c199b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:13.849 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.841+0000 7f089b400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f089c1038e0 0x7f089c199b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.119:43932/0 (socket says 172.21.15.119:43932) 2024-03-28T00:26:13.849 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.841+0000 7f089b400640 1 -- 172.21.15.119:0/3680450168 learned_addr learned my addr 172.21.15.119:0/3680450168 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:26:13.849 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.841+0000 7f0892a00640 1 --2- 172.21.15.119:0/3680450168 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f089c104ad0 0x7f089c19a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:13.849 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.841+0000 7f089be00640 1 --2- 172.21.15.119:0/3680450168 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f089c105490 0x7f089c1a1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:13.849 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.841+0000 7f0892a00640 1 -- 172.21.15.119:0/3680450168 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f089c1038e0 msgr2=0x7f089c199b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:26:13.849 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.841+0000 7f0892a00640 1 --2- 172.21.15.119:0/3680450168 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f089c1038e0 0x7f089c199b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:13.849 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.841+0000 7f0892a00640 1 -- 172.21.15.119:0/3680450168 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f089c105490 msgr2=0x7f089c1a1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:26:13.849 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.841+0000 7f0892a00640 1 --2- 172.21.15.119:0/3680450168 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f089c105490 0x7f089c1a1130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:13.849 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.841+0000 7f0892a00640 1 -- 172.21.15.119:0/3680450168 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f089c1a17a0 con 0x7f089c104ad0 2024-03-28T00:26:13.849 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.841+0000 7f089b400640 1 --2- 172.21.15.119:0/3680450168 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f089c1038e0 0x7f089c199b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T00:26:13.849 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.841+0000 7f089be00640 1 --2- 172.21.15.119:0/3680450168 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f089c105490 0x7f089c1a1130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T00:26:13.849 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.841+0000 7f0892a00640 1 --2- 172.21.15.119:0/3680450168 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f089c104ad0 0x7f089c19a0c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f088400e9f0 tx=0x7f088400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:26:13.849 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.841+0000 7f0898c00640 1 -- 172.21.15.119:0/3680450168 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f088400ccc0 con 0x7f089c104ad0 2024-03-28T00:26:13.849 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.842+0000 7f0898c00640 1 -- 172.21.15.119:0/3680450168 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0884010910 con 0x7f089c104ad0 2024-03-28T00:26:13.849 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.842+0000 7f0898c00640 1 -- 172.21.15.119:0/3680450168 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0884018960 con 0x7f089c104ad0 2024-03-28T00:26:13.850 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.842+0000 7f08a2800640 1 -- 172.21.15.119:0/3680450168 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f089c1a1a90 con 0x7f089c104ad0 2024-03-28T00:26:13.850 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.842+0000 7f08a2800640 1 -- 172.21.15.119:0/3680450168 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f089c1a1f90 con 0x7f089c104ad0 2024-03-28T00:26:13.851 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.845+0000 7f08a560b740 1 Processor -- start 2024-03-28T00:26:13.851 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.845+0000 7f0898c00640 1 -- 172.21.15.119:0/3680450168 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 18) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f0884010ac0 con 0x7f089c104ad0 2024-03-28T00:26:13.851 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.845+0000 7f08a560b740 1 -- start start 2024-03-28T00:26:13.853 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.845+0000 7f08a560b740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558c0626eba0 0x558c0626efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:13.853 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.845+0000 7f08a560b740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558c0626f590 0x558c06274090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:13.853 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.845+0000 7f08a560b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558c06274700 0x558c06276b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:13.853 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.845+0000 7f08a560b740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c061d3c90 con 0x558c0626eba0 2024-03-28T00:26:13.853 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.845+0000 7f08a560b740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c0617c980 con 0x558c0626f590 2024-03-28T00:26:13.853 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.845+0000 7f08a560b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c061cfd30 con 0x558c06274700 2024-03-28T00:26:13.853 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.845+0000 7f0892a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558c06274700 0x558c06276b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:13.853 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.845+0000 7f089be00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558c0626eba0 0x558c0626efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:13.853 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.845+0000 7f089b400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558c0626f590 0x558c06274090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:13.854 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.846+0000 7f0892a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558c06274700 0x558c06276b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.119:43944/0 (socket says 172.21.15.119:43944) 2024-03-28T00:26:13.854 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.846+0000 7f089be00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558c0626eba0 0x558c0626efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:48258/0 (socket says 172.21.15.119:48258) 2024-03-28T00:26:13.854 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.846+0000 7f0892a00640 1 -- 172.21.15.119:0/1069640760 learned_addr learned my addr 172.21.15.119:0/1069640760 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:26:13.854 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.846+0000 7f0892a00640 1 -- 172.21.15.119:0/1069640760 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558c0626f590 msgr2=0x558c06274090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T00:26:13.854 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.846+0000 7f0892a00640 1 --2- 172.21.15.119:0/1069640760 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558c0626f590 0x558c06274090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:13.854 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.846+0000 7f0892a00640 1 -- 172.21.15.119:0/1069640760 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558c0626eba0 msgr2=0x558c0626efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T00:26:13.854 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.846+0000 7f0892a00640 1 --2- 172.21.15.119:0/1069640760 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558c0626eba0 0x558c0626efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:13.854 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.846+0000 7f0892a00640 1 -- 172.21.15.119:0/1069640760 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558c06194510 con 0x558c06274700 2024-03-28T00:26:13.854 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.846+0000 7f0892a00640 1 --2- 172.21.15.119:0/1069640760 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558c06274700 0x558c06276b90 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f08840d2ea0 tx=0x7f08840d2f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d86e5ff40b050706 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:26:13.854 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.846+0000 7f0898c00640 1 --2- 172.21.15.119:0/3680450168 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f08800a2880 0x7f08800a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:13.854 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.846+0000 7f0890c00640 1 -- 172.21.15.119:0/1069640760 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0884097a60 con 0x558c06274700 2024-03-28T00:26:13.854 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.847+0000 7f0890c00640 1 -- 172.21.15.119:0/1069640760 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f08840ba210 con 0x558c06274700 2024-03-28T00:26:13.854 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.847+0000 7f0890c00640 1 -- 172.21.15.119:0/1069640760 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f088409af40 con 0x558c06274700 2024-03-28T00:26:13.854 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.847+0000 7f089be00640 1 --2- 172.21.15.119:0/3680450168 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f08800a2880 0x7f08800a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:13.854 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.847+0000 7f0898c00640 1 -- 172.21.15.119:0/3680450168 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f08840ced70 con 0x7f089c104ad0 2024-03-28T00:26:13.859 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.847+0000 7f08a560b740 1 -- 172.21.15.119:0/1069640760 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558c06274700 msgr2=0x558c06276b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:26:13.859 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.847+0000 7f08a560b740 1 --2- 172.21.15.119:0/1069640760 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558c06274700 0x558c06276b90 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f08840d2ea0 tx=0x7f08840d2f90 comp rx=0 tx=0).stop 2024-03-28T00:26:13.859 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.847+0000 7f089be00640 1 --2- 172.21.15.119:0/3680450168 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f08800a2880 0x7f08800a4d40 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f089402c640 tx=0x7f0894032590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:26:13.859 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.847+0000 7f08a560b740 1 -- 172.21.15.119:0/1069640760 shutdown_connections 2024-03-28T00:26:13.859 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.847+0000 7f08a560b740 1 --2- 172.21.15.119:0/1069640760 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558c06274700 0x558c06276b90 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:13.859 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.847+0000 7f08a560b740 1 --2- 172.21.15.119:0/1069640760 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558c0626f590 0x558c06274090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:13.860 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.847+0000 7f08a560b740 1 --2- 172.21.15.119:0/1069640760 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558c0626eba0 0x558c0626efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:13.860 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.847+0000 7f08a560b740 1 -- 172.21.15.119:0/1069640760 >> 172.21.15.119:0/1069640760 conn(0x558c0626a110 msgr2=0x558c0626a510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:26:13.860 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.847+0000 7f08a560b740 1 -- 172.21.15.119:0/1069640760 shutdown_connections 2024-03-28T00:26:13.860 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.847+0000 7f08a560b740 1 -- 172.21.15.119:0/1069640760 wait complete. 2024-03-28T00:26:13.860 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.848+0000 7f08a560b740 1 -- start start 2024-03-28T00:26:13.860 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.848+0000 7f08a560b740 1 Processor -- start 2024-03-28T00:26:13.860 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.848+0000 7f08a560b740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558c0626eba0 0x558c06279070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:13.860 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.848+0000 7f08a560b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558c0626f590 0x558c062795b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:13.860 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.848+0000 7f08a560b740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558c06274700 0x558c062805f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:13.860 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.848+0000 7f08a560b740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c062776b0 con 0x558c06274700 2024-03-28T00:26:13.860 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.848+0000 7f08a560b740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c0618fdd0 con 0x558c0626eba0 2024-03-28T00:26:13.860 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.848+0000 7f08a560b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c06277830 con 0x558c0626f590 2024-03-28T00:26:13.860 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.849+0000 7f08a560b740 10 client.0 _ll_register_callbacks cb 0x558c05ee7ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T00:26:13.860 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.849+0000 7f0892a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558c0626f590 0x558c062795b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:13.860 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.849+0000 7f089b400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558c0626eba0 0x558c06279070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:13.860 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.849+0000 7f0892a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558c0626f590 0x558c062795b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.119:43950/0 (socket says 172.21.15.119:43950) 2024-03-28T00:26:13.861 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.849+0000 7f0892a00640 1 -- 172.21.15.119:0/675842312 learned_addr learned my addr 172.21.15.119:0/675842312 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:26:13.861 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.849+0000 7f089b400640 1 --2- 172.21.15.119:0/675842312 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558c0626eba0 0x558c06279070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.119:45584/0 (socket says 172.21.15.119:45584) 2024-03-28T00:26:13.861 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.849+0000 7f089be00640 1 --2- 172.21.15.119:0/675842312 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558c06274700 0x558c062805f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:13.861 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.849+0000 7f0892a00640 1 -- 172.21.15.119:0/675842312 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558c0626eba0 msgr2=0x558c06279070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T00:26:13.861 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.849+0000 7f0892a00640 1 --2- 172.21.15.119:0/675842312 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558c0626eba0 0x558c06279070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:13.861 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.849+0000 7f0892a00640 1 -- 172.21.15.119:0/675842312 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558c06274700 msgr2=0x558c062805f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T00:26:13.861 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.849+0000 7f0892a00640 1 --2- 172.21.15.119:0/675842312 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558c06274700 0x558c062805f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:13.861 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.849+0000 7f0892a00640 1 -- 172.21.15.119:0/675842312 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558c05ee82d0 con 0x558c0626f590 2024-03-28T00:26:13.861 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.849+0000 7f089be00640 1 --2- 172.21.15.119:0/675842312 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558c06274700 0x558c062805f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T00:26:13.861 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.849+0000 7f089b400640 1 --2- 172.21.15.119:0/675842312 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558c0626eba0 0x558c06279070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T00:26:13.861 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.849+0000 7f0892a00640 1 --2- 172.21.15.119:0/675842312 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558c0626f590 0x558c062795b0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f088400e9c0 tx=0x7f08840ceff0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f2ec09fb016525c server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:26:13.861 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.849+0000 7f0889600640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-03-28T00:26:13.861 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.849+0000 7f0889600640 1 -- 172.21.15.119:0/675842312 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f088409b120 con 0x558c0626f590 2024-03-28T00:26:13.861 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.850+0000 7f0889600640 1 -- 172.21.15.119:0/675842312 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f08840860e0 con 0x558c0626f590 2024-03-28T00:26:13.862 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.850+0000 7f08a560b740 1 -- 172.21.15.119:0/675842312 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f08840ba210 con 0x558c0626f590 2024-03-28T00:26:13.862 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.850+0000 7f0889600640 1 -- 172.21.15.119:0/675842312 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f088409b300 con 0x558c0626f590 2024-03-28T00:26:13.862 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.850+0000 7f08a560b740 10 client.25231 fetch_fsmap learned FSMap version 26 2024-03-28T00:26:13.862 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.850+0000 7f08a560b740 1 -- 172.21.15.119:0/675842312 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x558c05fb26b0 con 0x558c0626f590 2024-03-28T00:26:13.862 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.850+0000 7f0889600640 1 -- 172.21.15.119:0/675842312 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f08840ba210 con 0x558c0626f590 2024-03-28T00:26:13.862 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.850+0000 7f0889600640 1 -- 172.21.15.119:0/675842312 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 26) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f08840de070 con 0x558c0626f590 2024-03-28T00:26:13.862 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.850+0000 7f08a560b740 10 client.25231 fetch_fsmap finished waiting for FSMap version 26 2024-03-28T00:26:13.862 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.850+0000 7f08a560b740 10 client.25231 resolve_mds: resolved name 'i' to daemon mds.i 2024-03-28T00:26:13.862 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.850+0000 7f08a560b740 20 client.25231 populate_metadata read hostname 'smithi119' 2024-03-28T00:26:13.862 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.850+0000 7f08a560b740 1 --2- 172.21.15.119:0/675842312 >> [v2:172.21.15.165:6838/3004684592,v1:172.21.15.165:6839/3004684592] conn(0x558c06285240 0x558c062896c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:13.862 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.850+0000 7f08a560b740 4 client.25231 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-28T00:26:13.862 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.850+0000 7f08a560b740 1 -- 172.21.15.119:0/675842312 --> [v2:172.21.15.165:6838/3004684592,v1:172.21.15.165:6839/3004684592] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558c06175520 con 0x558c06285240 2024-03-28T00:26:13.862 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.851+0000 7f089b400640 1 --2- 172.21.15.119:0/675842312 >> [v2:172.21.15.165:6838/3004684592,v1:172.21.15.165:6839/3004684592] conn(0x558c06285240 0x558c062896c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:13.862 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.851+0000 7f089b400640 1 --2- 172.21.15.119:0/675842312 >> [v2:172.21.15.165:6838/3004684592,v1:172.21.15.165:6839/3004684592] conn(0x558c06285240 0x558c062896c0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=625afa86fc812d28 server_cookie=426e92edc9274be4 in_seq=0 out_seq=0 2024-03-28T00:26:13.863 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.851+0000 7f0889600640 10 client.25231 ms_handle_connect on v2:172.21.15.165:6838/3004684592 2024-03-28T00:26:13.863 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.852+0000 7f0889600640 1 -- 172.21.15.119:0/675842312 <== mds.1 v2:172.21.15.165:6838/3004684592 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x558c06175520 con 0x558c06285240 2024-03-28T00:26:13.863 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.852+0000 7f0889600640 10 client.25231 handle_command_reply: tid=0 2024-03-28T00:26:13.863 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.852+0000 7f08a560b740 1 client.25231 shutdown 2024-03-28T00:26:13.863 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.853+0000 7f08a560b740 1 -- 172.21.15.119:0/675842312 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558c0626f590 msgr2=0x558c062795b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:26:13.863 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.853+0000 7f08a560b740 1 --2- 172.21.15.119:0/675842312 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558c0626f590 0x558c062795b0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f088400e9c0 tx=0x7f08840ceff0 comp rx=0 tx=0).stop 2024-03-28T00:26:13.863 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.853+0000 7f08a560b740 1 -- 172.21.15.119:0/675842312 shutdown_connections 2024-03-28T00:26:13.863 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.853+0000 7f08a560b740 1 --2- 172.21.15.119:0/675842312 >> [v2:172.21.15.165:6838/3004684592,v1:172.21.15.165:6839/3004684592] conn(0x558c06285240 0x558c062896c0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:13.863 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.853+0000 7f08a560b740 1 --2- 172.21.15.119:0/675842312 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558c06274700 0x558c062805f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:13.863 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.853+0000 7f08a560b740 1 --2- 172.21.15.119:0/675842312 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558c0626f590 0x558c062795b0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:13.863 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.853+0000 7f08a560b740 1 --2- 172.21.15.119:0/675842312 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558c0626eba0 0x558c06279070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:13.863 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.853+0000 7f0889600640 0 client.25231 ms_handle_reset on v2:172.21.15.165:6838/3004684592 2024-03-28T00:26:13.863 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.853+0000 7f08a560b740 1 -- 172.21.15.119:0/675842312 >> 172.21.15.119:0/675842312 conn(0x558c0626a110 msgr2=0x558c06269310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:26:13.863 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.853+0000 7f08a560b740 1 -- 172.21.15.119:0/675842312 shutdown_connections 2024-03-28T00:26:13.863 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.853+0000 7f08a560b740 1 -- 172.21.15.119:0/675842312 wait complete. 2024-03-28T00:26:13.863 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.853+0000 7f08a560b740 20 client.25231 trim_cache size 0 max 16384 2024-03-28T00:26:13.878 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.871+0000 7f08a560b740 1 Processor -- start 2024-03-28T00:26:13.878 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.871+0000 7f08a560b740 1 -- start start 2024-03-28T00:26:13.878 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.871+0000 7f08a560b740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558c0626eba0 0x558c06270030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:13.878 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.871+0000 7f08a560b740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558c0626f590 0x558c062805b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:13.879 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.871+0000 7f08a560b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558c06274700 0x558c06277c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:13.879 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.871+0000 7f08a560b740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c06284a60 con 0x558c0626f590 2024-03-28T00:26:13.879 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.871+0000 7f08a560b740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c062850c0 con 0x558c0626eba0 2024-03-28T00:26:13.879 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.871+0000 7f08a560b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c06285720 con 0x558c06274700 2024-03-28T00:26:13.879 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.871+0000 7f089b400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558c0626eba0 0x558c06270030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:13.879 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.871+0000 7f089b400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558c0626eba0 0x558c06270030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.119:45588/0 (socket says 172.21.15.119:45588) 2024-03-28T00:26:13.879 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.871+0000 7f089b400640 1 -- 172.21.15.119:0/812101519 learned_addr learned my addr 172.21.15.119:0/812101519 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:26:13.879 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.871+0000 7f089be00640 1 --2- 172.21.15.119:0/812101519 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558c06274700 0x558c06277c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:13.879 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.872+0000 7f0892a00640 1 --2- 172.21.15.119:0/812101519 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558c0626f590 0x558c062805b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:13.879 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.872+0000 7f089b400640 1 -- 172.21.15.119:0/812101519 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558c06274700 msgr2=0x558c06277c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T00:26:13.879 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.872+0000 7f089b400640 1 --2- 172.21.15.119:0/812101519 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558c06274700 0x558c06277c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:13.879 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.872+0000 7f089b400640 1 -- 172.21.15.119:0/812101519 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558c0626f590 msgr2=0x558c062805b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T00:26:13.879 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.872+0000 7f089b400640 1 --2- 172.21.15.119:0/812101519 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558c0626f590 0x558c062805b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:13.879 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.872+0000 7f089b400640 1 -- 172.21.15.119:0/812101519 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558c06270570 con 0x558c0626eba0 2024-03-28T00:26:13.879 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.872+0000 7f0892a00640 1 --2- 172.21.15.119:0/812101519 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558c0626f590 0x558c062805b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T00:26:13.880 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.872+0000 7f089b400640 1 --2- 172.21.15.119:0/812101519 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558c0626eba0 0x558c06270030 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x558c06280b30 tx=0x7f088c00b410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf2efd93ef5b6dca server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:26:13.880 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.872+0000 7f088be00640 1 -- 172.21.15.119:0/812101519 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f088c005020 con 0x558c0626eba0 2024-03-28T00:26:13.880 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.872+0000 7f088be00640 1 -- 172.21.15.119:0/812101519 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f088c024030 con 0x558c0626eba0 2024-03-28T00:26:13.880 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.872+0000 7f088be00640 1 -- 172.21.15.119:0/812101519 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f088c01d650 con 0x558c0626eba0 2024-03-28T00:26:13.884 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.873+0000 7f08a560b740 1 -- 172.21.15.119:0/812101519 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558c0626eba0 msgr2=0x558c06270030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:26:13.884 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.873+0000 7f08a560b740 1 --2- 172.21.15.119:0/812101519 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558c0626eba0 0x558c06270030 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x558c06280b30 tx=0x7f088c00b410 comp rx=0 tx=0).stop 2024-03-28T00:26:13.884 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.873+0000 7f08a560b740 1 -- 172.21.15.119:0/812101519 shutdown_connections 2024-03-28T00:26:13.884 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.873+0000 7f08a560b740 1 --2- 172.21.15.119:0/812101519 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558c06274700 0x558c06277c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:13.884 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.873+0000 7f08a560b740 1 --2- 172.21.15.119:0/812101519 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558c0626f590 0x558c062805b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:13.884 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.873+0000 7f08a560b740 1 --2- 172.21.15.119:0/812101519 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558c0626eba0 0x558c06270030 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:13.884 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.873+0000 7f08a560b740 1 -- 172.21.15.119:0/812101519 >> 172.21.15.119:0/812101519 conn(0x558c0626a110 msgr2=0x558c06269310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:26:13.884 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.873+0000 7f08a560b740 1 -- 172.21.15.119:0/812101519 shutdown_connections 2024-03-28T00:26:13.884 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.873+0000 7f08a560b740 1 -- 172.21.15.119:0/812101519 wait complete. 2024-03-28T00:26:13.884 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.873+0000 7f08a560b740 1 -- start start 2024-03-28T00:26:13.884 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.874+0000 7f08a560b740 1 Processor -- start 2024-03-28T00:26:13.884 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.874+0000 7f08a560b740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558c0626eba0 0x558c0627c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:13.885 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.874+0000 7f08a560b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558c0626f590 0x558c0627cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:13.885 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.874+0000 7f08a560b740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558c06274700 0x558c0627fb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:13.885 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.874+0000 7f08a560b740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c062863e0 con 0x558c0626eba0 2024-03-28T00:26:13.885 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.874+0000 7f08a560b740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c06285d80 con 0x558c06274700 2024-03-28T00:26:13.885 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.874+0000 7f08a560b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c06286a40 con 0x558c0626f590 2024-03-28T00:26:13.885 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.874+0000 7f08a560b740 10 client.0 _ll_register_callbacks cb 0x7f0884005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T00:26:13.885 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.874+0000 7f089b400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558c0626eba0 0x558c0627c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:13.885 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.874+0000 7f0892a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558c0626f590 0x558c0627cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:13.885 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.874+0000 7f089b400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558c0626eba0 0x558c0627c7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:48288/0 (socket says 172.21.15.119:48288) 2024-03-28T00:26:13.885 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.874+0000 7f089b400640 1 -- 172.21.15.119:0/875873231 learned_addr learned my addr 172.21.15.119:0/875873231 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:26:13.885 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.874+0000 7f089be00640 1 --2- 172.21.15.119:0/875873231 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558c06274700 0x558c0627fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:13.885 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.875+0000 7f089b400640 1 -- 172.21.15.119:0/875873231 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558c0626f590 msgr2=0x558c0627cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T00:26:13.885 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.875+0000 7f089b400640 1 --2- 172.21.15.119:0/875873231 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558c0626f590 0x558c0627cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:13.885 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.875+0000 7f089b400640 1 -- 172.21.15.119:0/875873231 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558c06274700 msgr2=0x558c0627fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T00:26:13.885 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.875+0000 7f089b400640 1 --2- 172.21.15.119:0/875873231 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558c06274700 0x558c0627fb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:13.886 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.875+0000 7f089b400640 1 -- 172.21.15.119:0/875873231 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558c06280110 con 0x558c0626eba0 2024-03-28T00:26:13.886 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.875+0000 7f0892a00640 1 --2- 172.21.15.119:0/875873231 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558c0626f590 0x558c0627cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T00:26:13.886 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.875+0000 7f089b400640 1 --2- 172.21.15.119:0/875873231 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558c0626eba0 0x558c0627c7e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f088c000c00 tx=0x7f088c005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb9dc94df9a2eec7 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:26:13.886 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.875+0000 7f0889600640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-03-28T00:26:13.886 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.875+0000 7f0889600640 1 -- 172.21.15.119:0/875873231 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f088c02d020 con 0x558c0626eba0 2024-03-28T00:26:13.886 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.875+0000 7f0889600640 1 -- 172.21.15.119:0/875873231 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f088c004730 con 0x558c0626eba0 2024-03-28T00:26:13.886 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.875+0000 7f08a560b740 1 -- 172.21.15.119:0/875873231 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f088c024030 con 0x558c0626eba0 2024-03-28T00:26:13.886 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.875+0000 7f0889600640 1 -- 172.21.15.119:0/875873231 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f088c004930 con 0x558c0626eba0 2024-03-28T00:26:13.886 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.875+0000 7f08a560b740 10 client.15123 fetch_fsmap learned FSMap version 26 2024-03-28T00:26:13.886 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.875+0000 7f08a560b740 1 -- 172.21.15.119:0/875873231 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x558c062e1070 con 0x558c0626eba0 2024-03-28T00:26:13.886 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.876+0000 7f0889600640 1 -- 172.21.15.119:0/875873231 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f088c024030 con 0x558c0626eba0 2024-03-28T00:26:13.886 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.876+0000 7f0889600640 1 -- 172.21.15.119:0/875873231 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 26) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f088c029970 con 0x558c0626eba0 2024-03-28T00:26:13.886 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.876+0000 7f08a560b740 10 client.15123 fetch_fsmap finished waiting for FSMap version 26 2024-03-28T00:26:13.886 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.876+0000 7f08a560b740 10 client.15123 resolve_mds: resolved name 'i' to daemon mds.i 2024-03-28T00:26:13.886 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.876+0000 7f08a560b740 20 client.15123 populate_metadata read hostname 'smithi119' 2024-03-28T00:26:13.887 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.876+0000 7f08a560b740 1 --2- 172.21.15.119:0/875873231 >> [v2:172.21.15.165:6838/3004684592,v1:172.21.15.165:6839/3004684592] conn(0x558c062e53f0 0x558c062e12f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:13.887 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.876+0000 7f08a560b740 4 client.15123 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-28T00:26:13.887 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.876+0000 7f08a560b740 1 -- 172.21.15.119:0/875873231 --> [v2:172.21.15.165:6838/3004684592,v1:172.21.15.165:6839/3004684592] -- command(tid 0: {"prefix": "status"}) v1 -- 0x558c061e11d0 con 0x558c062e53f0 2024-03-28T00:26:13.887 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.876+0000 7f0892a00640 1 --2- 172.21.15.119:0/875873231 >> [v2:172.21.15.165:6838/3004684592,v1:172.21.15.165:6839/3004684592] conn(0x558c062e53f0 0x558c062e12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:13.887 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.877+0000 7f0889600640 10 client.15123 ms_handle_connect on v2:172.21.15.165:6838/3004684592 2024-03-28T00:26:13.887 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.877+0000 7f0892a00640 1 --2- 172.21.15.119:0/875873231 >> [v2:172.21.15.165:6838/3004684592,v1:172.21.15.165:6839/3004684592] conn(0x558c062e53f0 0x558c062e12f0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1a2b3419ac26ba03 server_cookie=e5da38fb9c40e91b in_seq=0 out_seq=0 2024-03-28T00:26:13.887 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.877+0000 7f0889600640 1 -- 172.21.15.119:0/875873231 <== mds.1 v2:172.21.15.165:6838/3004684592 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x558c061e11d0 con 0x558c062e53f0 2024-03-28T00:26:13.887 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.877+0000 7f0889600640 10 client.15123 handle_command_reply: tid=0 2024-03-28T00:26:13.887 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.877+0000 7f08a560b740 1 client.15123 shutdown 2024-03-28T00:26:13.887 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.877+0000 7f08a560b740 1 -- 172.21.15.119:0/875873231 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558c0626eba0 msgr2=0x558c0627c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:26:13.887 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.877+0000 7f08a560b740 1 --2- 172.21.15.119:0/875873231 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558c0626eba0 0x558c0627c7e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f088c000c00 tx=0x7f088c005c00 comp rx=0 tx=0).stop 2024-03-28T00:26:13.887 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.877+0000 7f08a560b740 1 -- 172.21.15.119:0/875873231 shutdown_connections 2024-03-28T00:26:13.887 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.877+0000 7f08a560b740 1 --2- 172.21.15.119:0/875873231 >> [v2:172.21.15.165:6838/3004684592,v1:172.21.15.165:6839/3004684592] conn(0x558c062e53f0 0x558c062e12f0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:13.887 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.877+0000 7f08a560b740 1 --2- 172.21.15.119:0/875873231 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558c06274700 0x558c0627fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:13.887 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.877+0000 7f08a560b740 1 --2- 172.21.15.119:0/875873231 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558c0626f590 0x558c0627cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:13.888 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.877+0000 7f08a560b740 1 --2- 172.21.15.119:0/875873231 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558c0626eba0 0x558c0627c7e0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:13.888 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.877+0000 7f0889600640 0 client.15123 ms_handle_reset on v2:172.21.15.165:6838/3004684592 2024-03-28T00:26:13.888 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.877+0000 7f08a560b740 1 -- 172.21.15.119:0/875873231 >> 172.21.15.119:0/875873231 conn(0x558c0626a110 msgr2=0x558c06269310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:26:13.888 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.877+0000 7f08a560b740 1 -- 172.21.15.119:0/875873231 shutdown_connections 2024-03-28T00:26:13.888 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.877+0000 7f08a560b740 1 -- 172.21.15.119:0/875873231 wait complete. 2024-03-28T00:26:13.888 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.877+0000 7f08a560b740 20 client.15123 trim_cache size 0 max 16384 2024-03-28T00:26:13.888 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-03-28T00:26:13.888 INFO:teuthology.orchestra.run.smithi119.stdout: "cluster_fsid": "10f73b92-ec98-11ee-95d0-87774f69a715", 2024-03-28T00:26:13.888 INFO:teuthology.orchestra.run.smithi119.stdout: "whoami": 1, 2024-03-28T00:26:13.888 INFO:teuthology.orchestra.run.smithi119.stdout: "id": 24550, 2024-03-28T00:26:13.888 INFO:teuthology.orchestra.run.smithi119.stdout: "want_state": "up:active", 2024-03-28T00:26:13.888 INFO:teuthology.orchestra.run.smithi119.stdout: "state": "up:active", 2024-03-28T00:26:13.889 INFO:teuthology.orchestra.run.smithi119.stdout: "fs_name": "cephfs", 2024-03-28T00:26:13.889 INFO:teuthology.orchestra.run.smithi119.stdout: "rank_uptime": 133.402828236, 2024-03-28T00:26:13.889 INFO:teuthology.orchestra.run.smithi119.stdout: "mdsmap_epoch": 26, 2024-03-28T00:26:13.889 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_epoch": 87, 2024-03-28T00:26:13.889 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_epoch_barrier": 85, 2024-03-28T00:26:13.889 INFO:teuthology.orchestra.run.smithi119.stdout: "uptime": 154.53325825799999 2024-03-28T00:26:13.889 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-03-28T00:26:13.890 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.878+0000 7f08a2800640 1 -- 172.21.15.119:0/3680450168 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f08800a2880 msgr2=0x7f08800a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:26:13.890 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.878+0000 7f08a2800640 1 --2- 172.21.15.119:0/3680450168 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f08800a2880 0x7f08800a4d40 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f089402c640 tx=0x7f0894032590 comp rx=0 tx=0).stop 2024-03-28T00:26:13.890 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.878+0000 7f08a2800640 1 -- 172.21.15.119:0/3680450168 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f089c104ad0 msgr2=0x7f089c19a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:26:13.890 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.878+0000 7f08a2800640 1 --2- 172.21.15.119:0/3680450168 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f089c104ad0 0x7f089c19a0c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f088400e9f0 tx=0x7f088400eeb0 comp rx=0 tx=0).stop 2024-03-28T00:26:13.890 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.878+0000 7f08a2800640 1 -- 172.21.15.119:0/3680450168 shutdown_connections 2024-03-28T00:26:13.890 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.878+0000 7f08a2800640 1 --2- 172.21.15.119:0/3680450168 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f08800a2880 0x7f08800a4d40 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:13.890 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.878+0000 7f08a2800640 1 --2- 172.21.15.119:0/3680450168 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f089c105490 0x7f089c1a1130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:13.890 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.878+0000 7f08a2800640 1 --2- 172.21.15.119:0/3680450168 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f089c104ad0 0x7f089c19a0c0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:13.890 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.878+0000 7f08a2800640 1 --2- 172.21.15.119:0/3680450168 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f089c1038e0 0x7f089c199b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:13.890 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.878+0000 7f08a2800640 1 -- 172.21.15.119:0/3680450168 >> 172.21.15.119:0/3680450168 conn(0x7f089c0fee50 msgr2=0x7f089c107dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:26:13.890 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.878+0000 7f08a2800640 1 -- 172.21.15.119:0/3680450168 shutdown_connections 2024-03-28T00:26:13.890 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:13.878+0000 7f08a2800640 1 -- 172.21.15.119:0/3680450168 wait complete. 2024-03-28T00:26:13.902 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-28T00:26:14.118 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.112+0000 7f435d200640 1 -- 172.21.15.119:0/2847903486 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4358105490 msgr2=0x7f435810bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:26:14.119 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.112+0000 7f435d200640 1 --2- 172.21.15.119:0/2847903486 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4358105490 0x7f435810bd20 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f4350009ca0 tx=0x7f435002c100 comp rx=0 tx=0).stop 2024-03-28T00:26:14.119 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.112+0000 7f435d200640 1 -- 172.21.15.119:0/2847903486 shutdown_connections 2024-03-28T00:26:14.119 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.112+0000 7f435d200640 1 --2- 172.21.15.119:0/2847903486 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4358105490 0x7f435810bd20 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:14.119 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.112+0000 7f435d200640 1 --2- 172.21.15.119:0/2847903486 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4358104ad0 0x7f4358104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:14.119 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.112+0000 7f435d200640 1 --2- 172.21.15.119:0/2847903486 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f43581038e0 0x7f4358103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:14.119 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.112+0000 7f435d200640 1 -- 172.21.15.119:0/2847903486 >> 172.21.15.119:0/2847903486 conn(0x7f43580fee50 msgr2=0x7f4358101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:26:14.119 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.112+0000 7f435d200640 1 -- 172.21.15.119:0/2847903486 shutdown_connections 2024-03-28T00:26:14.119 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.112+0000 7f435d200640 1 -- 172.21.15.119:0/2847903486 wait complete. 2024-03-28T00:26:14.122 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.113+0000 7f435d200640 1 Processor -- start 2024-03-28T00:26:14.122 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.113+0000 7f435d200640 1 -- start start 2024-03-28T00:26:14.122 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.113+0000 7f435d200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f43581038e0 0x7f4358199b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:14.122 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.114+0000 7f435d200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4358104ad0 0x7f435819a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:14.122 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.114+0000 7f435d200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4358105490 0x7f43581a1130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:14.122 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.114+0000 7f435d200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f435810f540 con 0x7f4358104ad0 2024-03-28T00:26:14.122 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.114+0000 7f435d200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f435810f3c0 con 0x7f4358105490 2024-03-28T00:26:14.122 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.114+0000 7f435d200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f435810f6c0 con 0x7f43581038e0 2024-03-28T00:26:14.123 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.114+0000 7f4355e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f43581038e0 0x7f4358199b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:14.123 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.114+0000 7f4355e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f43581038e0 0x7f4358199b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.119:43986/0 (socket says 172.21.15.119:43986) 2024-03-28T00:26:14.123 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.114+0000 7f4355e00640 1 -- 172.21.15.119:0/3930219455 learned_addr learned my addr 172.21.15.119:0/3930219455 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:26:14.123 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.114+0000 7f434fe00640 1 --2- 172.21.15.119:0/3930219455 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4358104ad0 0x7f435819a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:14.123 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.114+0000 7f4356800640 1 --2- 172.21.15.119:0/3930219455 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4358105490 0x7f43581a1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:14.123 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.114+0000 7f4355e00640 1 -- 172.21.15.119:0/3930219455 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4358105490 msgr2=0x7f43581a1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:26:14.123 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.114+0000 7f4355e00640 1 --2- 172.21.15.119:0/3930219455 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4358105490 0x7f43581a1130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:14.123 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.114+0000 7f4355e00640 1 -- 172.21.15.119:0/3930219455 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4358104ad0 msgr2=0x7f435819a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:26:14.123 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.114+0000 7f4355e00640 1 --2- 172.21.15.119:0/3930219455 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4358104ad0 0x7f435819a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:14.123 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.114+0000 7f4355e00640 1 -- 172.21.15.119:0/3930219455 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43581a17a0 con 0x7f43581038e0 2024-03-28T00:26:14.123 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.114+0000 7f4356800640 1 --2- 172.21.15.119:0/3930219455 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4358105490 0x7f43581a1130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T00:26:14.123 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.114+0000 7f4355e00640 1 --2- 172.21.15.119:0/3930219455 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f43581038e0 0x7f4358199b80 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f434800d770 tx=0x7f434800dc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:26:14.123 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.115+0000 7f434ea00640 1 -- 172.21.15.119:0/3930219455 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4348017070 con 0x7f43581038e0 2024-03-28T00:26:14.124 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.115+0000 7f434ea00640 1 -- 172.21.15.119:0/3930219455 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4348009c30 con 0x7f43581038e0 2024-03-28T00:26:14.124 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.115+0000 7f434ea00640 1 -- 172.21.15.119:0/3930219455 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4348003dd0 con 0x7f43581038e0 2024-03-28T00:26:14.124 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.115+0000 7f435d200640 1 -- 172.21.15.119:0/3930219455 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43581a1a90 con 0x7f43581038e0 2024-03-28T00:26:14.124 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.115+0000 7f435d200640 1 -- 172.21.15.119:0/3930219455 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43581a1f90 con 0x7f43581038e0 2024-03-28T00:26:14.125 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.119+0000 7f434ea00640 1 -- 172.21.15.119:0/3930219455 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 18) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f4348003f70 con 0x7f43581038e0 2024-03-28T00:26:14.125 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.119+0000 7f435fe4b740 1 Processor -- start 2024-03-28T00:26:14.126 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.119+0000 7f435fe4b740 1 -- start start 2024-03-28T00:26:14.127 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.120+0000 7f435fe4b740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56143f380ba0 0x56143f380fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:14.127 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.120+0000 7f435fe4b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56143f381590 0x56143f386090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:14.127 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.120+0000 7f435fe4b740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56143f386700 0x56143f388b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:14.128 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.120+0000 7f435fe4b740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56143f2e5c90 con 0x56143f386700 2024-03-28T00:26:14.128 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.120+0000 7f435fe4b740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x56143f28e980 con 0x56143f380ba0 2024-03-28T00:26:14.128 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.120+0000 7f435fe4b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x56143f2e1d30 con 0x56143f381590 2024-03-28T00:26:14.128 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.120+0000 7f4356800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56143f380ba0 0x56143f380fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:14.128 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.120+0000 7f4355e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56143f381590 0x56143f386090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:14.128 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.120+0000 7f4356800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56143f380ba0 0x56143f380fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.119:45628/0 (socket says 172.21.15.119:45628) 2024-03-28T00:26:14.128 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.120+0000 7f4356800640 1 -- 172.21.15.119:0/3152702472 learned_addr learned my addr 172.21.15.119:0/3152702472 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:26:14.128 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.120+0000 7f434fe00640 1 --2- 172.21.15.119:0/3152702472 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56143f386700 0x56143f388b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:14.128 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.120+0000 7f4356800640 1 -- 172.21.15.119:0/3152702472 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56143f381590 msgr2=0x56143f386090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T00:26:14.128 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.120+0000 7f4356800640 1 --2- 172.21.15.119:0/3152702472 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56143f381590 0x56143f386090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:14.128 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.120+0000 7f4356800640 1 -- 172.21.15.119:0/3152702472 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56143f386700 msgr2=0x56143f388b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T00:26:14.128 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.120+0000 7f4356800640 1 --2- 172.21.15.119:0/3152702472 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56143f386700 0x56143f388b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:14.128 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.120+0000 7f4356800640 1 -- 172.21.15.119:0/3152702472 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56143f2a6510 con 0x56143f380ba0 2024-03-28T00:26:14.128 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.121+0000 7f4356800640 1 --2- 172.21.15.119:0/3152702472 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56143f380ba0 0x56143f380fc0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f435002c710 tx=0x7f43500323d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f313f3417942a0e8 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:26:14.129 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.121+0000 7f434ea00640 1 --2- 172.21.15.119:0/3930219455 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f433c0a2880 0x7f433c0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:14.129 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.121+0000 7f4347400640 1 -- 172.21.15.119:0/3152702472 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4350043070 con 0x56143f380ba0 2024-03-28T00:26:14.129 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.121+0000 7f4347400640 1 -- 172.21.15.119:0/3152702472 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4350032c90 con 0x56143f380ba0 2024-03-28T00:26:14.129 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.121+0000 7f4347400640 1 -- 172.21.15.119:0/3152702472 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4350039040 con 0x56143f380ba0 2024-03-28T00:26:14.129 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.121+0000 7f4355e00640 1 --2- 172.21.15.119:0/3930219455 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f433c0a2880 0x7f433c0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:14.129 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.121+0000 7f434ea00640 1 -- 172.21.15.119:0/3930219455 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f43480ce210 con 0x7f43581038e0 2024-03-28T00:26:14.133 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.122+0000 7f435fe4b740 1 -- 172.21.15.119:0/3152702472 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56143f380ba0 msgr2=0x56143f380fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:26:14.133 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.122+0000 7f435fe4b740 1 --2- 172.21.15.119:0/3152702472 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56143f380ba0 0x56143f380fc0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f435002c710 tx=0x7f43500323d0 comp rx=0 tx=0).stop 2024-03-28T00:26:14.133 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.122+0000 7f4355e00640 1 --2- 172.21.15.119:0/3930219455 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f433c0a2880 0x7f433c0a4d40 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x56143f381990 tx=0x7f43480d1d30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:26:14.134 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.122+0000 7f435fe4b740 1 -- 172.21.15.119:0/3152702472 shutdown_connections 2024-03-28T00:26:14.134 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.122+0000 7f435fe4b740 1 --2- 172.21.15.119:0/3152702472 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56143f386700 0x56143f388b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:14.134 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.122+0000 7f435fe4b740 1 --2- 172.21.15.119:0/3152702472 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56143f381590 0x56143f386090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:14.134 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.122+0000 7f435fe4b740 1 --2- 172.21.15.119:0/3152702472 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56143f380ba0 0x56143f380fc0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:14.134 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.122+0000 7f435fe4b740 1 -- 172.21.15.119:0/3152702472 >> 172.21.15.119:0/3152702472 conn(0x56143f37c110 msgr2=0x56143f37c510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:26:14.134 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.122+0000 7f435fe4b740 1 -- 172.21.15.119:0/3152702472 shutdown_connections 2024-03-28T00:26:14.134 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.122+0000 7f435fe4b740 1 -- 172.21.15.119:0/3152702472 wait complete. 2024-03-28T00:26:14.134 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.122+0000 7f435fe4b740 1 -- start start 2024-03-28T00:26:14.134 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.122+0000 7f435fe4b740 1 Processor -- start 2024-03-28T00:26:14.134 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.123+0000 7f435fe4b740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56143f380ba0 0x56143f38b060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:14.134 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.123+0000 7f435fe4b740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56143f381590 0x56143f38b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:14.134 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.123+0000 7f435fe4b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56143f386700 0x56143f3925e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:14.134 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.123+0000 7f435fe4b740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56143f389650 con 0x56143f381590 2024-03-28T00:26:14.135 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.123+0000 7f435fe4b740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x56143f2a1dd0 con 0x56143f380ba0 2024-03-28T00:26:14.135 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.123+0000 7f435fe4b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x56143f3897d0 con 0x56143f386700 2024-03-28T00:26:14.135 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.123+0000 7f435fe4b740 10 client.0 _ll_register_callbacks cb 0x56143eff9ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T00:26:14.135 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.123+0000 7f434fe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56143f381590 0x56143f38b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:14.135 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.123+0000 7f4356800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56143f380ba0 0x56143f38b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:14.135 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.123+0000 7f434fe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56143f381590 0x56143f38b5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:48320/0 (socket says 172.21.15.119:48320) 2024-03-28T00:26:14.135 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.123+0000 7f434fe00640 1 -- 172.21.15.119:0/1732069887 learned_addr learned my addr 172.21.15.119:0/1732069887 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:26:14.135 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.123+0000 7f4356800640 1 --2- 172.21.15.119:0/1732069887 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56143f386700 0x56143f3925e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:14.135 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.124+0000 7f434fe00640 1 -- 172.21.15.119:0/1732069887 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56143f386700 msgr2=0x56143f3925e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T00:26:14.135 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.124+0000 7f434fe00640 1 --2- 172.21.15.119:0/1732069887 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56143f386700 0x56143f3925e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:14.135 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.124+0000 7f434fe00640 1 -- 172.21.15.119:0/1732069887 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56143f380ba0 msgr2=0x56143f38b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T00:26:14.135 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.124+0000 7f434fe00640 1 --2- 172.21.15.119:0/1732069887 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56143f380ba0 0x56143f38b060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:14.135 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.124+0000 7f434fe00640 1 -- 172.21.15.119:0/1732069887 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56143effa2d0 con 0x56143f381590 2024-03-28T00:26:14.135 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.124+0000 7f4356800640 1 --2- 172.21.15.119:0/1732069887 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56143f386700 0x56143f3925e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T00:26:14.136 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.124+0000 7f434fe00640 1 --2- 172.21.15.119:0/1732069887 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56143f381590 0x56143f38b5a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f4340012700 tx=0x7f4340012b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac14bf34dbda2206 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:26:14.136 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.124+0000 7f433be00640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-03-28T00:26:14.136 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.124+0000 7f433be00640 1 -- 172.21.15.119:0/1732069887 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f434000f610 con 0x56143f381590 2024-03-28T00:26:14.136 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.124+0000 7f433be00640 1 -- 172.21.15.119:0/1732069887 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f434000f7b0 con 0x56143f381590 2024-03-28T00:26:14.136 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.124+0000 7f435fe4b740 1 -- 172.21.15.119:0/1732069887 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4350032c90 con 0x56143f381590 2024-03-28T00:26:14.136 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.124+0000 7f433be00640 1 -- 172.21.15.119:0/1732069887 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f434000f9b0 con 0x56143f381590 2024-03-28T00:26:14.136 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.124+0000 7f435fe4b740 10 client.15135 fetch_fsmap learned FSMap version 26 2024-03-28T00:26:14.136 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.124+0000 7f435fe4b740 1 -- 172.21.15.119:0/1732069887 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x56143f0c46b0 con 0x56143f381590 2024-03-28T00:26:14.136 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.125+0000 7f433be00640 1 -- 172.21.15.119:0/1732069887 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4350032c90 con 0x56143f381590 2024-03-28T00:26:14.136 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.125+0000 7f433be00640 1 -- 172.21.15.119:0/1732069887 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 26) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f4340010c40 con 0x56143f381590 2024-03-28T00:26:14.136 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.125+0000 7f435fe4b740 10 client.15135 fetch_fsmap finished waiting for FSMap version 26 2024-03-28T00:26:14.136 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.125+0000 7f435fe4b740 10 client.15135 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-28T00:26:14.136 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.125+0000 7f435fe4b740 20 client.15135 populate_metadata read hostname 'smithi119' 2024-03-28T00:26:14.136 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.125+0000 7f435fe4b740 1 --2- 172.21.15.119:0/1732069887 >> [v2:172.21.15.132:6838/3305822525,v1:172.21.15.132:6839/3305822525] conn(0x56143f3978a0 0x56143f396930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:14.137 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.125+0000 7f435fe4b740 4 client.15135 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-28T00:26:14.137 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.125+0000 7f435fe4b740 1 -- 172.21.15.119:0/1732069887 --> [v2:172.21.15.132:6838/3305822525,v1:172.21.15.132:6839/3305822525] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56143f287520 con 0x56143f3978a0 2024-03-28T00:26:14.137 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.125+0000 7f4356800640 1 --2- 172.21.15.119:0/1732069887 >> [v2:172.21.15.132:6838/3305822525,v1:172.21.15.132:6839/3305822525] conn(0x56143f3978a0 0x56143f396930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:14.137 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.126+0000 7f4356800640 1 --2- 172.21.15.119:0/1732069887 >> [v2:172.21.15.132:6838/3305822525,v1:172.21.15.132:6839/3305822525] conn(0x56143f3978a0 0x56143f396930 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91a3e71478a2aa5c server_cookie=1956c3feb960805b in_seq=0 out_seq=0 2024-03-28T00:26:14.137 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.126+0000 7f433be00640 10 client.15135 ms_handle_connect on v2:172.21.15.132:6838/3305822525 2024-03-28T00:26:14.137 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.126+0000 7f433be00640 1 -- 172.21.15.119:0/1732069887 <== mds.0 v2:172.21.15.132:6838/3305822525 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x56143f287520 con 0x56143f3978a0 2024-03-28T00:26:14.137 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.126+0000 7f433be00640 10 client.15135 handle_command_reply: tid=0 2024-03-28T00:26:14.137 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.126+0000 7f435fe4b740 1 client.15135 shutdown 2024-03-28T00:26:14.137 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.127+0000 7f435fe4b740 1 -- 172.21.15.119:0/1732069887 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56143f381590 msgr2=0x56143f38b5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:26:14.137 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.127+0000 7f435fe4b740 1 --2- 172.21.15.119:0/1732069887 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56143f381590 0x56143f38b5a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f4340012700 tx=0x7f4340012b00 comp rx=0 tx=0).stop 2024-03-28T00:26:14.137 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.127+0000 7f435fe4b740 1 -- 172.21.15.119:0/1732069887 shutdown_connections 2024-03-28T00:26:14.137 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.127+0000 7f435fe4b740 1 --2- 172.21.15.119:0/1732069887 >> [v2:172.21.15.132:6838/3305822525,v1:172.21.15.132:6839/3305822525] conn(0x56143f3978a0 0x56143f396930 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:14.137 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.127+0000 7f435fe4b740 1 --2- 172.21.15.119:0/1732069887 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56143f386700 0x56143f3925e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:14.137 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.127+0000 7f435fe4b740 1 --2- 172.21.15.119:0/1732069887 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56143f381590 0x56143f38b5a0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:14.137 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.127+0000 7f433be00640 0 client.15135 ms_handle_reset on v2:172.21.15.132:6838/3305822525 2024-03-28T00:26:14.138 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.127+0000 7f435fe4b740 1 --2- 172.21.15.119:0/1732069887 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56143f380ba0 0x56143f38b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:14.138 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.127+0000 7f435fe4b740 1 -- 172.21.15.119:0/1732069887 >> 172.21.15.119:0/1732069887 conn(0x56143f37c110 msgr2=0x56143f37b310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:26:14.138 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.127+0000 7f435fe4b740 1 -- 172.21.15.119:0/1732069887 shutdown_connections 2024-03-28T00:26:14.138 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.127+0000 7f435fe4b740 1 -- 172.21.15.119:0/1732069887 wait complete. 2024-03-28T00:26:14.138 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.127+0000 7f435fe4b740 20 client.15135 trim_cache size 0 max 16384 2024-03-28T00:26:14.152 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.145+0000 7f435fe4b740 1 Processor -- start 2024-03-28T00:26:14.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.145+0000 7f435fe4b740 1 -- start start 2024-03-28T00:26:14.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.145+0000 7f435fe4b740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56143f380ba0 0x56143f376e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:14.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.145+0000 7f435fe4b740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56143f381590 0x56143f3925a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:14.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.145+0000 7f435fe4b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56143f386700 0x56143f37c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:14.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.145+0000 7f435fe4b740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56143f3970c0 con 0x56143f380ba0 2024-03-28T00:26:14.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.145+0000 7f435fe4b740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x56143f397720 con 0x56143f381590 2024-03-28T00:26:14.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.145+0000 7f435fe4b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x56143f397d80 con 0x56143f386700 2024-03-28T00:26:14.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.146+0000 7f434fe00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56143f381590 0x56143f3925a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:14.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.146+0000 7f4356800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56143f380ba0 0x56143f376e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:14.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.146+0000 7f434fe00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56143f381590 0x56143f3925a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.119:45650/0 (socket says 172.21.15.119:45650) 2024-03-28T00:26:14.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.146+0000 7f434fe00640 1 -- 172.21.15.119:0/3378190581 learned_addr learned my addr 172.21.15.119:0/3378190581 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:26:14.154 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.146+0000 7f4356800640 1 --2- 172.21.15.119:0/3378190581 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56143f386700 0x56143f37c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:14.154 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.146+0000 7f434fe00640 1 -- 172.21.15.119:0/3378190581 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56143f386700 msgr2=0x56143f37c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T00:26:14.154 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.146+0000 7f434fe00640 1 --2- 172.21.15.119:0/3378190581 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56143f386700 0x56143f37c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:14.154 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.146+0000 7f434fe00640 1 -- 172.21.15.119:0/3378190581 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56143f380ba0 msgr2=0x56143f376e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T00:26:14.154 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.146+0000 7f434fe00640 1 --2- 172.21.15.119:0/3378190581 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56143f380ba0 0x56143f376e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:14.154 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.146+0000 7f434fe00640 1 -- 172.21.15.119:0/3378190581 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56143f37ce80 con 0x56143f381590 2024-03-28T00:26:14.154 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.146+0000 7f434fe00640 1 --2- 172.21.15.119:0/3378190581 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56143f381590 0x56143f3925a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f4340012420 tx=0x7f434000f8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=780faf723b5211ff server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:26:14.154 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.146+0000 7f4346a00640 1 -- 172.21.15.119:0/3378190581 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f434000fb50 con 0x56143f381590 2024-03-28T00:26:14.154 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.146+0000 7f4346a00640 1 -- 172.21.15.119:0/3378190581 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f434002b020 con 0x56143f381590 2024-03-28T00:26:14.154 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.146+0000 7f4346a00640 1 -- 172.21.15.119:0/3378190581 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f434001c940 con 0x56143f381590 2024-03-28T00:26:14.159 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.147+0000 7f435fe4b740 1 -- 172.21.15.119:0/3378190581 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56143f381590 msgr2=0x56143f3925a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:26:14.159 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.147+0000 7f435fe4b740 1 --2- 172.21.15.119:0/3378190581 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56143f381590 0x56143f3925a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f4340012420 tx=0x7f434000f8f0 comp rx=0 tx=0).stop 2024-03-28T00:26:14.159 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.147+0000 7f435fe4b740 1 -- 172.21.15.119:0/3378190581 shutdown_connections 2024-03-28T00:26:14.159 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.147+0000 7f435fe4b740 1 --2- 172.21.15.119:0/3378190581 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56143f386700 0x56143f37c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:14.159 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.147+0000 7f435fe4b740 1 --2- 172.21.15.119:0/3378190581 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56143f381590 0x56143f3925a0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:14.159 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.147+0000 7f435fe4b740 1 --2- 172.21.15.119:0/3378190581 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56143f380ba0 0x56143f376e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:14.159 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.147+0000 7f435fe4b740 1 -- 172.21.15.119:0/3378190581 >> 172.21.15.119:0/3378190581 conn(0x56143f37c110 msgr2=0x56143f396930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:26:14.159 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.147+0000 7f435fe4b740 1 -- 172.21.15.119:0/3378190581 shutdown_connections 2024-03-28T00:26:14.159 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.147+0000 7f435fe4b740 1 -- 172.21.15.119:0/3378190581 wait complete. 2024-03-28T00:26:14.160 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.147+0000 7f435fe4b740 1 -- start start 2024-03-28T00:26:14.160 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.148+0000 7f435fe4b740 1 Processor -- start 2024-03-28T00:26:14.160 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.148+0000 7f435fe4b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56143f380ba0 0x56143f3922c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:14.160 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.148+0000 7f435fe4b740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56143f381590 0x56143f38ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:14.160 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.148+0000 7f435fe4b740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56143f386700 0x56143f38f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:14.160 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.148+0000 7f435fe4b740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56143f398a40 con 0x56143f381590 2024-03-28T00:26:14.160 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.148+0000 7f435fe4b740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x56143f3983e0 con 0x56143f386700 2024-03-28T00:26:14.160 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.148+0000 7f435fe4b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x56143f3990a0 con 0x56143f380ba0 2024-03-28T00:26:14.160 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.148+0000 7f435fe4b740 10 client.0 _ll_register_callbacks cb 0x7f4340009850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T00:26:14.160 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.148+0000 7f4356800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56143f380ba0 0x56143f3922c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:14.160 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.148+0000 7f434fe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56143f381590 0x56143f38ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:14.160 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.148+0000 7f4356800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56143f380ba0 0x56143f3922c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.119:44012/0 (socket says 172.21.15.119:44012) 2024-03-28T00:26:14.160 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.148+0000 7f4356800640 1 -- 172.21.15.119:0/2942718761 learned_addr learned my addr 172.21.15.119:0/2942718761 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:26:14.160 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.148+0000 7f434fe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56143f381590 0x56143f38ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:48338/0 (socket says 172.21.15.119:48338) 2024-03-28T00:26:14.161 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.148+0000 7f4356800640 1 --2- 172.21.15.119:0/2942718761 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56143f386700 0x56143f38f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:14.161 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.149+0000 7f4356800640 1 -- 172.21.15.119:0/2942718761 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56143f386700 msgr2=0x56143f38f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T00:26:14.161 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.149+0000 7f4356800640 1 --2- 172.21.15.119:0/2942718761 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56143f386700 0x56143f38f2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:14.161 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.149+0000 7f4356800640 1 -- 172.21.15.119:0/2942718761 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56143f381590 msgr2=0x56143f38ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T00:26:14.161 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.149+0000 7f4356800640 1 --2- 172.21.15.119:0/2942718761 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56143f381590 0x56143f38ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:14.161 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.149+0000 7f4356800640 1 -- 172.21.15.119:0/2942718761 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56143f38f810 con 0x56143f380ba0 2024-03-28T00:26:14.161 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.149+0000 7f4356800640 1 --2- 172.21.15.119:0/2942718761 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56143f380ba0 0x56143f3922c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f43500324c0 tx=0x7f435004a040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f7b9985b94b8c9 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:26:14.161 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.149+0000 7f433be00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-03-28T00:26:14.161 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.149+0000 7f433be00640 1 -- 172.21.15.119:0/2942718761 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4350039040 con 0x56143f380ba0 2024-03-28T00:26:14.161 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.149+0000 7f433be00640 1 -- 172.21.15.119:0/2942718761 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f43500552b0 con 0x56143f380ba0 2024-03-28T00:26:14.161 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.149+0000 7f435fe4b740 1 -- 172.21.15.119:0/2942718761 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f434002b020 con 0x56143f380ba0 2024-03-28T00:26:14.161 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.149+0000 7f433be00640 1 -- 172.21.15.119:0/2942718761 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4350043040 con 0x56143f380ba0 2024-03-28T00:26:14.161 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.149+0000 7f435fe4b740 10 client.25255 fetch_fsmap learned FSMap version 26 2024-03-28T00:26:14.161 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.149+0000 7f435fe4b740 1 -- 172.21.15.119:0/2942718761 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x56143f3f38d0 con 0x56143f380ba0 2024-03-28T00:26:14.162 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.150+0000 7f433be00640 1 -- 172.21.15.119:0/2942718761 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f434002b020 con 0x56143f380ba0 2024-03-28T00:26:14.162 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.150+0000 7f433be00640 1 -- 172.21.15.119:0/2942718761 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 26) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f4350004040 con 0x56143f380ba0 2024-03-28T00:26:14.162 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.150+0000 7f435fe4b740 10 client.25255 fetch_fsmap finished waiting for FSMap version 26 2024-03-28T00:26:14.162 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.150+0000 7f435fe4b740 10 client.25255 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-28T00:26:14.162 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.150+0000 7f435fe4b740 20 client.25255 populate_metadata read hostname 'smithi119' 2024-03-28T00:26:14.162 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.150+0000 7f435fe4b740 1 --2- 172.21.15.119:0/2942718761 >> [v2:172.21.15.132:6838/3305822525,v1:172.21.15.132:6839/3305822525] conn(0x56143f3f7ae0 0x56143f3f3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:14.162 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.150+0000 7f435fe4b740 4 client.25255 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-28T00:26:14.162 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.150+0000 7f435fe4b740 1 -- 172.21.15.119:0/2942718761 --> [v2:172.21.15.132:6838/3305822525,v1:172.21.15.132:6839/3305822525] -- command(tid 0: {"prefix": "status"}) v1 -- 0x56143f2f31d0 con 0x56143f3f7ae0 2024-03-28T00:26:14.162 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.150+0000 7f434fe00640 1 --2- 172.21.15.119:0/2942718761 >> [v2:172.21.15.132:6838/3305822525,v1:172.21.15.132:6839/3305822525] conn(0x56143f3f7ae0 0x56143f3f3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:14.162 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.151+0000 7f434fe00640 1 --2- 172.21.15.119:0/2942718761 >> [v2:172.21.15.132:6838/3305822525,v1:172.21.15.132:6839/3305822525] conn(0x56143f3f7ae0 0x56143f3f3aa0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ab291b93831802a server_cookie=4505bcd4e34aa052 in_seq=0 out_seq=0 2024-03-28T00:26:14.162 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.151+0000 7f433be00640 10 client.25255 ms_handle_connect on v2:172.21.15.132:6838/3305822525 2024-03-28T00:26:14.162 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.151+0000 7f433be00640 1 -- 172.21.15.119:0/2942718761 <== mds.0 v2:172.21.15.132:6838/3305822525 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+322 (crc 0 0 0) 0x56143f2f31d0 con 0x56143f3f7ae0 2024-03-28T00:26:14.162 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.151+0000 7f433be00640 10 client.25255 handle_command_reply: tid=0 2024-03-28T00:26:14.162 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.151+0000 7f435fe4b740 1 client.25255 shutdown 2024-03-28T00:26:14.163 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.151+0000 7f435fe4b740 1 -- 172.21.15.119:0/2942718761 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56143f380ba0 msgr2=0x56143f3922c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:26:14.163 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.151+0000 7f435fe4b740 1 --2- 172.21.15.119:0/2942718761 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56143f380ba0 0x56143f3922c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f43500324c0 tx=0x7f435004a040 comp rx=0 tx=0).stop 2024-03-28T00:26:14.163 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.151+0000 7f435fe4b740 1 -- 172.21.15.119:0/2942718761 shutdown_connections 2024-03-28T00:26:14.163 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.151+0000 7f435fe4b740 1 --2- 172.21.15.119:0/2942718761 >> [v2:172.21.15.132:6838/3305822525,v1:172.21.15.132:6839/3305822525] conn(0x56143f3f7ae0 0x56143f3f3aa0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:14.163 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.151+0000 7f435fe4b740 1 --2- 172.21.15.119:0/2942718761 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56143f386700 0x56143f38f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:14.163 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.151+0000 7f435fe4b740 1 --2- 172.21.15.119:0/2942718761 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56143f381590 0x56143f38ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:14.163 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.151+0000 7f433be00640 0 client.25255 ms_handle_reset on v2:172.21.15.132:6838/3305822525 2024-03-28T00:26:14.163 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.151+0000 7f435fe4b740 1 --2- 172.21.15.119:0/2942718761 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56143f380ba0 0x56143f3922c0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:14.163 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.152+0000 7f435fe4b740 1 -- 172.21.15.119:0/2942718761 >> 172.21.15.119:0/2942718761 conn(0x56143f37c110 msgr2=0x56143f376e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:26:14.163 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.152+0000 7f435fe4b740 1 -- 172.21.15.119:0/2942718761 shutdown_connections 2024-03-28T00:26:14.163 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.152+0000 7f435fe4b740 1 -- 172.21.15.119:0/2942718761 wait complete. 2024-03-28T00:26:14.163 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.152+0000 7f435fe4b740 20 client.25255 trim_cache size 0 max 16384 2024-03-28T00:26:14.164 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-03-28T00:26:14.164 INFO:teuthology.orchestra.run.smithi119.stdout: "cluster_fsid": "10f73b92-ec98-11ee-95d0-87774f69a715", 2024-03-28T00:26:14.164 INFO:teuthology.orchestra.run.smithi119.stdout: "whoami": 0, 2024-03-28T00:26:14.164 INFO:teuthology.orchestra.run.smithi119.stdout: "id": 24581, 2024-03-28T00:26:14.164 INFO:teuthology.orchestra.run.smithi119.stdout: "want_state": "up:active", 2024-03-28T00:26:14.164 INFO:teuthology.orchestra.run.smithi119.stdout: "state": "up:active", 2024-03-28T00:26:14.164 INFO:teuthology.orchestra.run.smithi119.stdout: "fs_name": "cephfs", 2024-03-28T00:26:14.164 INFO:teuthology.orchestra.run.smithi119.stdout: "rank_uptime": 137.078992763, 2024-03-28T00:26:14.164 INFO:teuthology.orchestra.run.smithi119.stdout: "mdsmap_epoch": 26, 2024-03-28T00:26:14.164 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_epoch": 87, 2024-03-28T00:26:14.164 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_epoch_barrier": 85, 2024-03-28T00:26:14.164 INFO:teuthology.orchestra.run.smithi119.stdout: "uptime": 144.186018545 2024-03-28T00:26:14.164 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-03-28T00:26:14.165 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.152+0000 7f435d200640 1 -- 172.21.15.119:0/3930219455 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f433c0a2880 msgr2=0x7f433c0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:26:14.165 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.152+0000 7f435d200640 1 --2- 172.21.15.119:0/3930219455 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f433c0a2880 0x7f433c0a4d40 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x56143f381990 tx=0x7f43480d1d30 comp rx=0 tx=0).stop 2024-03-28T00:26:14.165 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.152+0000 7f435d200640 1 -- 172.21.15.119:0/3930219455 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f43581038e0 msgr2=0x7f4358199b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:26:14.165 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.152+0000 7f435d200640 1 --2- 172.21.15.119:0/3930219455 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f43581038e0 0x7f4358199b80 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f434800d770 tx=0x7f434800dc30 comp rx=0 tx=0).stop 2024-03-28T00:26:14.165 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.152+0000 7f435d200640 1 -- 172.21.15.119:0/3930219455 shutdown_connections 2024-03-28T00:26:14.166 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.152+0000 7f435d200640 1 --2- 172.21.15.119:0/3930219455 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f433c0a2880 0x7f433c0a4d40 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:14.166 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.152+0000 7f435d200640 1 --2- 172.21.15.119:0/3930219455 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4358105490 0x7f43581a1130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:14.166 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.152+0000 7f435d200640 1 --2- 172.21.15.119:0/3930219455 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4358104ad0 0x7f435819a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:14.166 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.152+0000 7f435d200640 1 --2- 172.21.15.119:0/3930219455 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f43581038e0 0x7f4358199b80 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:14.166 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.152+0000 7f435d200640 1 -- 172.21.15.119:0/3930219455 >> 172.21.15.119:0/3930219455 conn(0x7f43580fee50 msgr2=0x7f4358107dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:26:14.166 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.153+0000 7f435d200640 1 -- 172.21.15.119:0/3930219455 shutdown_connections 2024-03-28T00:26:14.166 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:14.153+0000 7f435d200640 1 -- 172.21.15.119:0/3930219455 wait complete. 2024-03-28T00:26:14.178 INFO:teuthology.run_tasks:Running task kclient... 2024-03-28T00:26:14.205 INFO:tasks.kclient:Mounting kernel clients... 2024-03-28T00:26:14.205 INFO:tasks.kclient:config is {'client.0': {}} 2024-03-28T00:26:14.205 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T00:26:14.205 DEBUG:teuthology.orchestra.run.smithi119:> ip netns list 2024-03-28T00:26:14.254 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T00:26:14.254 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip link delete ceph-brx 2024-03-28T00:26:14.392 INFO:teuthology.orchestra.run.smithi119.stderr:Cannot find device "ceph-brx" 2024-03-28T00:26:14.396 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-28T00:26:14.396 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=legacy', 'nowsync'], 'syntax': 'v2', 'mount_subvol_num': 1} 2024-03-28T00:26:14.396 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-28T00:26:14.397 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-28T00:26:14.397 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 10f73b92-ec98-11ee-95d0-87774f69a715 -- ceph fs ls 2024-03-28T00:26:14.857 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/mon.a/config 2024-03-28T00:26:15.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:14 smithi119 ceph-mon[31518]: pgmap v382: 97 pgs: 97 active+clean; 648 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-28T00:26:15.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:14 smithi165 ceph-mon[36302]: pgmap v382: 97 pgs: 97 active+clean; 648 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-28T00:26:15.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:14 smithi132 ceph-mon[35941]: pgmap v382: 97 pgs: 97 active+clean; 648 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-28T00:26:16.976 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:16.963+0000 7f855ee00700 1 -- 172.21.15.119:0/367576394 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f85581096c0 msgr2=0x7f8558111290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:26:16.976 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:16.963+0000 7f855ee00700 1 --2- 172.21.15.119:0/367576394 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f85581096c0 0x7f8558111290 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f855000b0d0 tx=0x7f855000b3e0 comp rx=0 tx=0).stop 2024-03-28T00:26:16.976 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:16.964+0000 7f855ee00700 1 -- 172.21.15.119:0/367576394 shutdown_connections 2024-03-28T00:26:16.976 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:16.964+0000 7f855ee00700 1 --2- 172.21.15.119:0/367576394 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f85581096c0 0x7f8558111290 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:16.976 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:16.964+0000 7f855ee00700 1 --2- 172.21.15.119:0/367576394 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f85581045e0 0x7f85581049c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:16.976 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:16.964+0000 7f855ee00700 1 --2- 172.21.15.119:0/367576394 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8558104f90 0x7f8558108fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:16.976 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:16.964+0000 7f855ee00700 1 -- 172.21.15.119:0/367576394 >> 172.21.15.119:0/367576394 conn(0x7f85580ffe70 msgr2=0x7f8558102290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:26:16.976 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:16.964+0000 7f855ee00700 1 -- 172.21.15.119:0/367576394 shutdown_connections 2024-03-28T00:26:16.976 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:16.964+0000 7f855ee00700 1 -- 172.21.15.119:0/367576394 wait complete. 2024-03-28T00:26:16.976 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:16.965+0000 7f855ee00700 1 Processor -- start 2024-03-28T00:26:16.976 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:16.965+0000 7f855ee00700 1 -- start start 2024-03-28T00:26:16.977 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:16.966+0000 7f855ee00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f85581045e0 0x7f85581a0720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:16.977 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:16.966+0000 7f855ee00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8558104f90 0x7f85581a0c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:16.977 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:16.966+0000 7f855ee00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f85581096c0 0x7f855819a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:16.977 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:16.966+0000 7f855ee00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8558114700 con 0x7f85581096c0 2024-03-28T00:26:16.977 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:16.966+0000 7f855ee00700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8558114580 con 0x7f8558104f90 2024-03-28T00:26:16.977 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:16.966+0000 7f855ee00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8558114880 con 0x7f85581045e0 2024-03-28T00:26:16.977 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:16.966+0000 7f8557400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8558104f90 0x7f85581a0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:16.977 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:16.966+0000 7f8557e00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f85581045e0 0x7f85581a0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:16.977 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:16.966+0000 7f8557400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8558104f90 0x7f85581a0c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.119:45682/0 (socket says 172.21.15.119:45682) 2024-03-28T00:26:16.977 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:16.966+0000 7f8557e00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f85581045e0 0x7f85581a0720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.119:44026/0 (socket says 172.21.15.119:44026) 2024-03-28T00:26:16.977 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:16.966+0000 7f8557400700 1 -- 172.21.15.119:0/184740600 learned_addr learned my addr 172.21.15.119:0/184740600 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:26:16.977 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:16.966+0000 7f855ce00700 1 --2- 172.21.15.119:0/184740600 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f85581096c0 0x7f855819a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:16.977 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:16.966+0000 7f8557e00700 1 -- 172.21.15.119:0/184740600 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8558104f90 msgr2=0x7f85581a0c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:26:16.977 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:16.966+0000 7f8557e00700 1 --2- 172.21.15.119:0/184740600 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8558104f90 0x7f85581a0c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:16.978 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:16.966+0000 7f8557e00700 1 -- 172.21.15.119:0/184740600 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f85581096c0 msgr2=0x7f855819a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:26:16.978 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:16.966+0000 7f8557e00700 1 --2- 172.21.15.119:0/184740600 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f85581096c0 0x7f855819a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:16.978 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:16.966+0000 7f8557e00700 1 -- 172.21.15.119:0/184740600 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f855819b040 con 0x7f85581045e0 2024-03-28T00:26:16.978 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:16.966+0000 7f8557400700 1 --2- 172.21.15.119:0/184740600 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8558104f90 0x7f85581a0c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T00:26:16.978 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:16.967+0000 7f8557e00700 1 --2- 172.21.15.119:0/184740600 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f85581045e0 0x7f85581a0720 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f854800e9f0 tx=0x7f854800ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:26:16.978 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:16.967+0000 7f8554c00700 1 -- 172.21.15.119:0/184740600 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f854800c9c0 con 0x7f85581045e0 2024-03-28T00:26:16.978 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:16.967+0000 7f8554c00700 1 -- 172.21.15.119:0/184740600 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f854800cb60 con 0x7f85581045e0 2024-03-28T00:26:16.978 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:16.967+0000 7f855ee00700 1 -- 172.21.15.119:0/184740600 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f855819b330 con 0x7f85581045e0 2024-03-28T00:26:16.978 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:16.967+0000 7f8554c00700 1 -- 172.21.15.119:0/184740600 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8548018680 con 0x7f85581045e0 2024-03-28T00:26:16.978 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:16.967+0000 7f855ee00700 1 -- 172.21.15.119:0/184740600 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85581a75f0 con 0x7f85581045e0 2024-03-28T00:26:16.978 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:16.968+0000 7f855ee00700 1 -- 172.21.15.119:0/184740600 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8558068e50 con 0x7f85581045e0 2024-03-28T00:26:16.979 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:16.970+0000 7f8554c00700 1 -- 172.21.15.119:0/184740600 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 18) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f8548010430 con 0x7f85581045e0 2024-03-28T00:26:16.979 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:16.971+0000 7f8554c00700 1 --2- 172.21.15.119:0/184740600 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f853c0a2910 0x7f853c0a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:16.979 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:16.971+0000 7f8554c00700 1 -- 172.21.15.119:0/184740600 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f85480ceb20 con 0x7f85581045e0 2024-03-28T00:26:16.979 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:16.971+0000 7f8557400700 1 --2- 172.21.15.119:0/184740600 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f853c0a2910 0x7f853c0a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:16.979 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:16.972+0000 7f8557400700 1 --2- 172.21.15.119:0/184740600 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f853c0a2910 0x7f853c0a4dd0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f85581034c0 tx=0x7f8544009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:26:16.981 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:16.975+0000 7f8554c00700 1 -- 172.21.15.119:0/184740600 <== mon.1 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f854809bbc0 con 0x7f85581045e0 2024-03-28T00:26:17.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:16 smithi119 ceph-mon[31518]: pgmap v383: 97 pgs: 97 active+clean; 648 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T00:26:17.253 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:17.247+0000 7f855ee00700 1 -- 172.21.15.119:0/184740600 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f85581a7850 con 0x7f85581045e0 2024-03-28T00:26:17.254 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:17.247+0000 7f8554c00700 1 -- 172.21.15.119:0/184740600 <== mon.1 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v26) v1 ==== 53+0+73 (secure 0 0 0) 0x7f854809b4c0 con 0x7f85581045e0 2024-03-28T00:26:17.254 INFO:teuthology.orchestra.run.smithi119.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-28T00:26:17.257 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:17.250+0000 7f855ee00700 1 -- 172.21.15.119:0/184740600 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f853c0a2910 msgr2=0x7f853c0a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:26:17.257 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:17.250+0000 7f855ee00700 1 --2- 172.21.15.119:0/184740600 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f853c0a2910 0x7f853c0a4dd0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f85581034c0 tx=0x7f8544009260 comp rx=0 tx=0).stop 2024-03-28T00:26:17.257 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:17.250+0000 7f855ee00700 1 -- 172.21.15.119:0/184740600 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f85581045e0 msgr2=0x7f85581a0720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:26:17.257 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:17.250+0000 7f855ee00700 1 --2- 172.21.15.119:0/184740600 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f85581045e0 0x7f85581a0720 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f854800e9f0 tx=0x7f854800ed00 comp rx=0 tx=0).stop 2024-03-28T00:26:17.257 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:17.250+0000 7f855ee00700 1 -- 172.21.15.119:0/184740600 shutdown_connections 2024-03-28T00:26:17.257 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:17.250+0000 7f855ee00700 1 --2- 172.21.15.119:0/184740600 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f853c0a2910 0x7f853c0a4dd0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:17.257 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:17.250+0000 7f855ee00700 1 --2- 172.21.15.119:0/184740600 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f85581096c0 0x7f855819a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:17.257 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:17.250+0000 7f855ee00700 1 --2- 172.21.15.119:0/184740600 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f85581045e0 0x7f85581a0720 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:17.257 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:17.250+0000 7f855ee00700 1 --2- 172.21.15.119:0/184740600 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8558104f90 0x7f85581a0c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:17.258 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:17.250+0000 7f855ee00700 1 -- 172.21.15.119:0/184740600 >> 172.21.15.119:0/184740600 conn(0x7f85580ffe70 msgr2=0x7f8558100810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:26:17.258 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:17.251+0000 7f855ee00700 1 -- 172.21.15.119:0/184740600 shutdown_connections 2024-03-28T00:26:17.258 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:17.251+0000 7f855ee00700 1 -- 172.21.15.119:0/184740600 wait complete. 2024-03-28T00:26:17.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:16 smithi165 ceph-mon[36302]: pgmap v383: 97 pgs: 97 active+clean; 648 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T00:26:17.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:16 smithi132 ceph-mon[35941]: pgmap v383: 97 pgs: 97 active+clean; 648 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T00:26:18.227 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-28T00:26:18.227 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-28T00:26:18.227 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi119.front.sepia.ceph.com 2024-03-28T00:26:18.227 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-28T00:26:18.227 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-28T00:26:18.227 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-28T00:26:18.227 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-28T00:26:18.227 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-28T00:26:18.227 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-28T00:26:18.227 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T00:26:18.227 DEBUG:teuthology.orchestra.run.smithi119:> ip addr 2024-03-28T00:26:18.285 INFO:teuthology.orchestra.run.smithi119.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-28T00:26:18.285 INFO:teuthology.orchestra.run.smithi119.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-28T00:26:18.285 INFO:teuthology.orchestra.run.smithi119.stdout: inet 127.0.0.1/8 scope host lo 2024-03-28T00:26:18.285 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2024-03-28T00:26:18.285 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 ::1/128 scope host 2024-03-28T00:26:18.285 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2024-03-28T00:26:18.285 INFO:teuthology.orchestra.run.smithi119.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-28T00:26:18.286 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:d8:da:62 brd ff:ff:ff:ff:ff:ff 2024-03-28T00:26:18.286 INFO:teuthology.orchestra.run.smithi119.stdout: altname enp7s0f0 2024-03-28T00:26:18.286 INFO:teuthology.orchestra.run.smithi119.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-28T00:26:18.286 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:d8:da:63 brd ff:ff:ff:ff:ff:ff 2024-03-28T00:26:18.286 INFO:teuthology.orchestra.run.smithi119.stdout: altname enp7s0f1 2024-03-28T00:26:18.286 INFO:teuthology.orchestra.run.smithi119.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-28T00:26:18.286 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:8f:d5:ee brd ff:ff:ff:ff:ff:ff 2024-03-28T00:26:18.286 INFO:teuthology.orchestra.run.smithi119.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-28T00:26:18.286 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:8f:d5:ef brd ff:ff:ff:ff:ff:ff 2024-03-28T00:26:18.286 INFO:teuthology.orchestra.run.smithi119.stdout: inet 172.21.15.119/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-28T00:26:18.286 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft 41928sec preferred_lft 41928sec 2024-03-28T00:26:18.286 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::ec4:7aff:fe8f:d5ef/64 scope link 2024-03-28T00:26:18.286 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2024-03-28T00:26:18.287 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-28T00:26:18.288 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T00:26:18.288 DEBUG:teuthology.orchestra.run.smithi119:> (cd / && exec bash -c ' 2024-03-28T00:26:18.288 DEBUG:teuthology.orchestra.run.smithi119:> set -e 2024-03-28T00:26:18.288 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip link add name ceph-brx type bridge 2024-03-28T00:26:18.288 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip addr flush dev ceph-brx 2024-03-28T00:26:18.288 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip link set ceph-brx up 2024-03-28T00:26:18.288 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-28T00:26:18.288 DEBUG:teuthology.orchestra.run.smithi119:> ') 2024-03-28T00:26:18.374 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:17 smithi119 ceph-mon[31518]: from='client.? 172.21.15.119:0/184740600' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-28T00:26:18.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:17 smithi165 ceph-mon[36302]: from='client.? 172.21.15.119:0/184740600' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-28T00:26:18.452 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:17 smithi132 ceph-mon[35941]: from='client.? 172.21.15.119:0/184740600' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-28T00:26:18.735 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T00:26:18.735 DEBUG:teuthology.orchestra.run.smithi119:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-28T00:26:18.811 INFO:teuthology.orchestra.run.smithi119.stdout:1 2024-03-28T00:26:18.815 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T00:26:18.815 DEBUG:teuthology.orchestra.run.smithi119:> route 2024-03-28T00:26:18.879 INFO:teuthology.orchestra.run.smithi119.stdout:Kernel IP routing table 2024-03-28T00:26:18.879 INFO:teuthology.orchestra.run.smithi119.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-28T00:26:18.879 INFO:teuthology.orchestra.run.smithi119.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-28T00:26:18.879 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-28T00:26:18.879 INFO:teuthology.orchestra.run.smithi119.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-28T00:26:18.880 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T00:26:18.880 DEBUG:teuthology.orchestra.run.smithi119:> (cd / && exec bash -c ' 2024-03-28T00:26:18.880 DEBUG:teuthology.orchestra.run.smithi119:> set -e 2024-03-28T00:26:18.880 DEBUG:teuthology.orchestra.run.smithi119:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-28T00:26:18.880 DEBUG:teuthology.orchestra.run.smithi119:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-28T00:26:18.880 DEBUG:teuthology.orchestra.run.smithi119:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-28T00:26:18.880 DEBUG:teuthology.orchestra.run.smithi119:> ') 2024-03-28T00:26:19.217 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:18 smithi119 ceph-mon[31518]: pgmap v384: 97 pgs: 97 active+clean; 648 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-28T00:26:19.283 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:18 smithi165 ceph-mon[36302]: pgmap v384: 97 pgs: 97 active+clean; 648 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-28T00:26:19.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:18 smithi132 ceph-mon[35941]: pgmap v384: 97 pgs: 97 active+clean; 648 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-28T00:26:19.633 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T00:26:19.633 DEBUG:teuthology.orchestra.run.smithi119:> ip netns list 2024-03-28T00:26:19.690 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T00:26:19.691 DEBUG:teuthology.orchestra.run.smithi119:> ip netns list-id 2024-03-28T00:26:19.744 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T00:26:19.745 DEBUG:teuthology.orchestra.run.smithi119:> (cd / && exec bash -c ' 2024-03-28T00:26:19.745 DEBUG:teuthology.orchestra.run.smithi119:> set -e 2024-03-28T00:26:19.745 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-28T00:26:19.745 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-03-28T00:26:19.745 DEBUG:teuthology.orchestra.run.smithi119:> ') 2024-03-28T00:26:19.883 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-03-28T00:26:19.883 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T00:26:19.884 DEBUG:teuthology.orchestra.run.smithi119:> (cd / && exec bash -c ' 2024-03-28T00:26:19.884 DEBUG:teuthology.orchestra.run.smithi119:> set -e 2024-03-28T00:26:19.884 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-03-28T00:26:19.884 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-28T00:26:19.884 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-03-28T00:26:19.884 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-03-28T00:26:19.884 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-03-28T00:26:19.884 DEBUG:teuthology.orchestra.run.smithi119:> ') 2024-03-28T00:26:20.167 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T00:26:20.167 DEBUG:teuthology.orchestra.run.smithi119:> (cd / && exec bash -c ' 2024-03-28T00:26:20.167 DEBUG:teuthology.orchestra.run.smithi119:> set -e 2024-03-28T00:26:20.167 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip link set brx.0 up 2024-03-28T00:26:20.167 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip link set dev brx.0 master ceph-brx 2024-03-28T00:26:20.167 DEBUG:teuthology.orchestra.run.smithi119:> ') 2024-03-28T00:26:20.329 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-28T00:26:20.329 DEBUG:teuthology.orchestra.run.smithi119:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-28T00:26:20.373 INFO:teuthology.orchestra.run.smithi119.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-28T00:26:20.374 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-28T00:26:20.374 DEBUG:teuthology.orchestra.run.smithi119:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-28T00:26:20.468 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-03-28T00:26:20.469 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-03-28T00:26:20.469 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/volumes/qa/sv_1/65f09c1d-44b8-4ce5-9bb1-ad370f1bb828 2024-03-28T00:26:20.469 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T00:26:20.469 DEBUG:teuthology.orchestra.run.smithi119:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/volumes/qa/sv_1/65f09c1d-44b8-4ce5-9bb1-ad370f1bb828 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=legacy,nowsync,nofallback 2024-03-28T00:26:20.606 INFO:teuthology.orchestra.run.smithi119.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=legacy,nowsync,nofallback 2024-03-28T00:26:20.606 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=legacy,nowsync". 2024-03-28T00:26:21.249 INFO:teuthology.orchestra.run.smithi119.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=legacy,nowsync,nofallback 2024-03-28T00:26:21.250 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=legacy,nowsync". 2024-03-28T00:26:21.250 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: resolved to: "172.21.15.119:6789,172.21.15.132:6789,172.21.15.165:6789" 2024-03-28T00:26:21.250 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: trying mount with new device syntax: 0@10f73b92-ec98-11ee-95d0-87774f69a715.cephfs=/volumes/qa/sv_1/65f09c1d-44b8-4ce5-9bb1-ad370f1bb828 2024-03-28T00:26:21.250 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=legacy,nowsync,name=0,key=0,mon_addr=172.21.15.119:6789/172.21.15.132:6789/172.21.15.165:6789" will pass to kernel 2024-03-28T00:26:21.256 INFO:teuthology.orchestra.run.smithi119.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-03-28T00:26:21.256 INFO:teuthology.orchestra.run.smithi119.stdout: You just mounted a file system that supports labels which does not 2024-03-28T00:26:21.256 INFO:teuthology.orchestra.run.smithi119.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-03-28T00:26:21.256 INFO:teuthology.orchestra.run.smithi119.stdout: applications will generate AVC messages and not be allowed access to 2024-03-28T00:26:21.256 INFO:teuthology.orchestra.run.smithi119.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-03-28T00:26:21.261 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-28T00:26:21.261 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T00:26:21.262 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-28T00:26:21.340 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:21 smithi119 ceph-mon[31518]: pgmap v385: 97 pgs: 97 active+clean; 648 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-28T00:26:21.355 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T00:26:21.355 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-28T00:26:21.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:21 smithi165 ceph-mon[36302]: pgmap v385: 97 pgs: 97 active+clean; 648 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-28T00:26:21.412 INFO:teuthology.orchestra.run.smithi119.stdout:client25264 2024-03-28T00:26:21.412 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-03-28T00:26:21.413 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-28T00:26:21.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:21 smithi132 ceph-mon[35941]: pgmap v385: 97 pgs: 97 active+clean; 648 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-28T00:26:21.463 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T00:26:21.463 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-28T00:26:21.563 INFO:teuthology.orchestra.run.smithi119.stdout:client25264 2024-03-28T00:26:21.564 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T00:26:21.564 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-28T00:26:21.623 INFO:teuthology.orchestra.run.smithi119.stdout:client25264 2024-03-28T00:26:21.623 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T00:26:21.623 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/10f73b92-ec98-11ee-95d0-87774f69a715.client25264/status') 2024-03-28T00:26:21.712 INFO:teuthology.orchestra.run.smithi119.stdout:instance: client.25264 (3)172.21.15.119:0/3863511511 2024-03-28T00:26:21.712 INFO:teuthology.orchestra.run.smithi119.stdout:blocklisted: false 2024-03-28T00:26:21.713 INFO:teuthology.orchestra.run.smithi119.stderr:0+1 records in 2024-03-28T00:26:21.713 INFO:teuthology.orchestra.run.smithi119.stderr:0+1 records out 2024-03-28T00:26:21.713 INFO:teuthology.orchestra.run.smithi119.stderr:72 bytes copied, 6.6786e-05 s, 1.1 MB/s 2024-03-28T00:26:21.717 INFO:teuthology.run_tasks:Running task exec... 2024-03-28T00:26:21.727 INFO:teuthology.task.exec:Executing custom commands... 2024-03-28T00:26:21.727 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi119.front.sepia.ceph.com 2024-03-28T00:26:21.727 DEBUG:teuthology.orchestra.run.smithi119:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate true' 2024-03-28T00:26:21.933 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:21.927+0000 7fda4e600640 1 -- 172.21.15.119:0/2449533817 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fda481054a0 msgr2=0x7fda4810bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:26:21.934 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:21.927+0000 7fda4e600640 1 --2- 172.21.15.119:0/2449533817 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fda481054a0 0x7fda4810bd30 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fda40009900 tx=0x7fda4002bc40 comp rx=0 tx=0).stop 2024-03-28T00:26:21.934 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:21.928+0000 7fda4e600640 1 -- 172.21.15.119:0/2449533817 shutdown_connections 2024-03-28T00:26:21.934 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:21.928+0000 7fda4e600640 1 --2- 172.21.15.119:0/2449533817 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fda481054a0 0x7fda4810bd30 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:21.934 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:21.928+0000 7fda4e600640 1 --2- 172.21.15.119:0/2449533817 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fda48104ae0 0x7fda48104f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:21.934 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:21.928+0000 7fda4e600640 1 --2- 172.21.15.119:0/2449533817 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fda481038f0 0x7fda48103d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:21.934 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:21.928+0000 7fda4e600640 1 -- 172.21.15.119:0/2449533817 >> 172.21.15.119:0/2449533817 conn(0x7fda480fee60 msgr2=0x7fda481012a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:26:21.934 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:21.928+0000 7fda4e600640 1 -- 172.21.15.119:0/2449533817 shutdown_connections 2024-03-28T00:26:21.934 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:21.928+0000 7fda4e600640 1 -- 172.21.15.119:0/2449533817 wait complete. 2024-03-28T00:26:21.936 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:21.929+0000 7fda4e600640 1 Processor -- start 2024-03-28T00:26:21.937 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:21.929+0000 7fda4e600640 1 -- start start 2024-03-28T00:26:21.937 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:21.929+0000 7fda4e600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fda481038f0 0x7fda48199b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:21.937 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:21.929+0000 7fda4e600640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fda48104ae0 0x7fda4819a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:21.937 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:21.929+0000 7fda4e600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fda481054a0 0x7fda481a10c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:21.937 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:21.929+0000 7fda4e600640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda4810f5c0 con 0x7fda481054a0 2024-03-28T00:26:21.937 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:21.929+0000 7fda4e600640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda4810f440 con 0x7fda48104ae0 2024-03-28T00:26:21.937 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:21.929+0000 7fda4e600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda4810f740 con 0x7fda481038f0 2024-03-28T00:26:21.937 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:21.929+0000 7fda47400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fda481038f0 0x7fda48199b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:21.937 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:21.929+0000 7fda47400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fda481038f0 0x7fda48199b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.119:44044/0 (socket says 172.21.15.119:44044) 2024-03-28T00:26:21.937 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:21.930+0000 7fda47400640 1 -- 172.21.15.119:0/3244211963 learned_addr learned my addr 172.21.15.119:0/3244211963 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:26:21.937 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:21.930+0000 7fda47e00640 1 --2- 172.21.15.119:0/3244211963 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fda481054a0 0x7fda481a10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:21.938 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:21.930+0000 7fda46a00640 1 --2- 172.21.15.119:0/3244211963 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fda48104ae0 0x7fda4819a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:21.938 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:21.930+0000 7fda47400640 1 -- 172.21.15.119:0/3244211963 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fda48104ae0 msgr2=0x7fda4819a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:26:21.938 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:21.930+0000 7fda47400640 1 --2- 172.21.15.119:0/3244211963 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fda48104ae0 0x7fda4819a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:21.938 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:21.930+0000 7fda47400640 1 -- 172.21.15.119:0/3244211963 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fda481054a0 msgr2=0x7fda481a10c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:26:21.938 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:21.930+0000 7fda47400640 1 --2- 172.21.15.119:0/3244211963 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fda481054a0 0x7fda481a10c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:21.938 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:21.930+0000 7fda47400640 1 -- 172.21.15.119:0/3244211963 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda481a1730 con 0x7fda481038f0 2024-03-28T00:26:21.938 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:21.930+0000 7fda47400640 1 --2- 172.21.15.119:0/3244211963 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fda481038f0 0x7fda48199b10 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fda3800d740 tx=0x7fda3800dc00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:26:21.938 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:21.930+0000 7fda3fe00640 1 -- 172.21.15.119:0/3244211963 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda38017070 con 0x7fda481038f0 2024-03-28T00:26:21.938 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:21.930+0000 7fda3fe00640 1 -- 172.21.15.119:0/3244211963 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fda3800de50 con 0x7fda481038f0 2024-03-28T00:26:21.938 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:21.930+0000 7fda3fe00640 1 -- 172.21.15.119:0/3244211963 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda38004db0 con 0x7fda481038f0 2024-03-28T00:26:21.938 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:21.930+0000 7fda4e600640 1 -- 172.21.15.119:0/3244211963 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda481a1a20 con 0x7fda481038f0 2024-03-28T00:26:21.938 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:21.931+0000 7fda4e600640 1 -- 172.21.15.119:0/3244211963 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda481a1f20 con 0x7fda481038f0 2024-03-28T00:26:21.939 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:21.932+0000 7fda4e600640 1 -- 172.21.15.119:0/3244211963 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda4810ab40 con 0x7fda481038f0 2024-03-28T00:26:21.939 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:21.934+0000 7fda3fe00640 1 -- 172.21.15.119:0/3244211963 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 18) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fda38004f50 con 0x7fda481038f0 2024-03-28T00:26:21.940 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:21.935+0000 7fda3fe00640 1 --2- 172.21.15.119:0/3244211963 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fda2c0a28a0 0x7fda2c0a4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:21.940 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:21.935+0000 7fda3fe00640 1 -- 172.21.15.119:0/3244211963 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fda380ce840 con 0x7fda481038f0 2024-03-28T00:26:21.941 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:21.935+0000 7fda46a00640 1 --2- 172.21.15.119:0/3244211963 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fda2c0a28a0 0x7fda2c0a4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:21.941 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:21.936+0000 7fda46a00640 1 --2- 172.21.15.119:0/3244211963 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fda2c0a28a0 0x7fda2c0a4d60 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fda4819b020 tx=0x7fda30009680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:26:21.942 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:21.937+0000 7fda3fe00640 1 -- 172.21.15.119:0/3244211963 <== mon.1 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fda3809b290 con 0x7fda481038f0 2024-03-28T00:26:22.187 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:22.182+0000 7fda4e600640 1 -- 172.21.15.119:0/3244211963 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} v 0) v1 -- 0x7fda4802d070 con 0x7fda481038f0 2024-03-28T00:26:23.025 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:23.020+0000 7fda3fe00640 1 -- 172.21.15.119:0/3244211963 <== mon.1 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]=0 v27) v1 ==== 117+0+0 (secure 0 0 0) 0x7fda3809f330 con 0x7fda481038f0 2024-03-28T00:26:23.030 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:23.023+0000 7fda4e600640 1 -- 172.21.15.119:0/3244211963 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fda2c0a28a0 msgr2=0x7fda2c0a4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:26:23.030 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:23.023+0000 7fda4e600640 1 --2- 172.21.15.119:0/3244211963 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fda2c0a28a0 0x7fda2c0a4d60 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fda4819b020 tx=0x7fda30009680 comp rx=0 tx=0).stop 2024-03-28T00:26:23.030 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:23.023+0000 7fda4e600640 1 -- 172.21.15.119:0/3244211963 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fda481038f0 msgr2=0x7fda48199b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:26:23.030 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:23.023+0000 7fda4e600640 1 --2- 172.21.15.119:0/3244211963 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fda481038f0 0x7fda48199b10 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fda3800d740 tx=0x7fda3800dc00 comp rx=0 tx=0).stop 2024-03-28T00:26:23.030 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:23.023+0000 7fda4e600640 1 -- 172.21.15.119:0/3244211963 shutdown_connections 2024-03-28T00:26:23.031 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:23.023+0000 7fda4e600640 1 --2- 172.21.15.119:0/3244211963 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7fda2c0a28a0 0x7fda2c0a4d60 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:23.031 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:23.023+0000 7fda4e600640 1 --2- 172.21.15.119:0/3244211963 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fda481054a0 0x7fda481a10c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:23.031 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:23.023+0000 7fda4e600640 1 --2- 172.21.15.119:0/3244211963 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fda48104ae0 0x7fda4819a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:23.031 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:23.023+0000 7fda4e600640 1 --2- 172.21.15.119:0/3244211963 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fda481038f0 0x7fda48199b10 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:23.031 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:23.023+0000 7fda4e600640 1 -- 172.21.15.119:0/3244211963 >> 172.21.15.119:0/3244211963 conn(0x7fda480fee60 msgr2=0x7fda48107d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:26:23.031 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:23.024+0000 7fda4e600640 1 -- 172.21.15.119:0/3244211963 shutdown_connections 2024-03-28T00:26:23.032 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:23.024+0000 7fda4e600640 1 -- 172.21.15.119:0/3244211963 wait complete. 2024-03-28T00:26:23.050 INFO:teuthology.run_tasks:Running task check-counter... 2024-03-28T00:26:23.060 INFO:teuthology.run_tasks:Running task exec... 2024-03-28T00:26:23.079 INFO:teuthology.task.exec:Executing custom commands... 2024-03-28T00:26:23.079 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi119.front.sepia.ceph.com 2024-03-28T00:26:23.079 DEBUG:teuthology.orchestra.run.smithi119:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-03-28T00:26:23.280 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:23.274+0000 7f11f6c00640 1 -- 172.21.15.119:0/3133807797 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f11f0103720 msgr2=0x7f11f0103b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:26:23.280 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:23.274+0000 7f11f6c00640 1 --2- 172.21.15.119:0/3133807797 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f11f0103720 0x7f11f0103b40 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f11e4009900 tx=0x7f11e402bc30 comp rx=0 tx=0).stop 2024-03-28T00:26:23.280 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:23.274+0000 7f11f6c00640 1 -- 172.21.15.119:0/3133807797 shutdown_connections 2024-03-28T00:26:23.280 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:23.274+0000 7f11f6c00640 1 --2- 172.21.15.119:0/3133807797 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f11f01052d0 0x7f11f010bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:23.280 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:23.274+0000 7f11f6c00640 1 --2- 172.21.15.119:0/3133807797 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f11f0104910 0x7f11f0104d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:23.280 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:23.274+0000 7f11f6c00640 1 --2- 172.21.15.119:0/3133807797 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f11f0103720 0x7f11f0103b40 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:23.280 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:23.274+0000 7f11f6c00640 1 -- 172.21.15.119:0/3133807797 >> 172.21.15.119:0/3133807797 conn(0x7f11f00fecb0 msgr2=0x7f11f01010d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:26:23.280 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:23.274+0000 7f11f6c00640 1 -- 172.21.15.119:0/3133807797 shutdown_connections 2024-03-28T00:26:23.280 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:23.274+0000 7f11f6c00640 1 -- 172.21.15.119:0/3133807797 wait complete. 2024-03-28T00:26:23.282 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:23.275+0000 7f11f6c00640 1 Processor -- start 2024-03-28T00:26:23.282 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:23.275+0000 7f11f6c00640 1 -- start start 2024-03-28T00:26:23.282 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:23.276+0000 7f11f6c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f11f0103720 0x7f11f0199a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:23.283 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:23.276+0000 7f11f6c00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f11f0104910 0x7f11f0199f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:23.283 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:23.276+0000 7f11f6c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f11f01052d0 0x7f11f01a1000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:23.283 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:23.276+0000 7f11f6c00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11f010f3e0 con 0x7f11f01052d0 2024-03-28T00:26:23.283 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:23.276+0000 7f11f6c00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11f010f260 con 0x7f11f0104910 2024-03-28T00:26:23.283 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:23.276+0000 7f11f6c00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11f010f560 con 0x7f11f0103720 2024-03-28T00:26:23.283 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:23.276+0000 7f11f5800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f11f0103720 0x7f11f0199a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:23.283 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:23.276+0000 7f11f5800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f11f0103720 0x7f11f0199a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.119:44074/0 (socket says 172.21.15.119:44074) 2024-03-28T00:26:23.283 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:23.276+0000 7f11f5800640 1 -- 172.21.15.119:0/3994867265 learned_addr learned my addr 172.21.15.119:0/3994867265 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:26:23.283 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:23.276+0000 7f11f4e00640 1 --2- 172.21.15.119:0/3994867265 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f11f0104910 0x7f11f0199f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:23.283 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:23.276+0000 7f11f6200640 1 --2- 172.21.15.119:0/3994867265 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f11f01052d0 0x7f11f01a1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:23.283 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:23.276+0000 7f11f4e00640 1 -- 172.21.15.119:0/3994867265 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f11f0103720 msgr2=0x7f11f0199a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:26:23.283 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:23.276+0000 7f11f4e00640 1 --2- 172.21.15.119:0/3994867265 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f11f0103720 0x7f11f0199a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:23.283 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:23.276+0000 7f11f4e00640 1 -- 172.21.15.119:0/3994867265 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f11f01052d0 msgr2=0x7f11f01a1000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:26:23.283 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:23.276+0000 7f11f4e00640 1 --2- 172.21.15.119:0/3994867265 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f11f01052d0 0x7f11f01a1000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:23.284 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:23.276+0000 7f11f4e00640 1 -- 172.21.15.119:0/3994867265 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11f01a1670 con 0x7f11f0104910 2024-03-28T00:26:23.284 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:23.276+0000 7f11f5800640 1 --2- 172.21.15.119:0/3994867265 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f11f0103720 0x7f11f0199a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T00:26:23.284 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:23.276+0000 7f11f6200640 1 --2- 172.21.15.119:0/3994867265 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f11f01052d0 0x7f11f01a1000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T00:26:23.284 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:23.277+0000 7f11f4e00640 1 --2- 172.21.15.119:0/3994867265 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f11f0104910 0x7f11f0199f90 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f11e000e9f0 tx=0x7f11e000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:26:23.284 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:23.277+0000 7f11ee000640 1 -- 172.21.15.119:0/3994867265 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11e000ccc0 con 0x7f11f0104910 2024-03-28T00:26:23.284 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:23.277+0000 7f11f6c00640 1 -- 172.21.15.119:0/3994867265 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11f01a1960 con 0x7f11f0104910 2024-03-28T00:26:23.284 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:23.277+0000 7f11ee000640 1 -- 172.21.15.119:0/3994867265 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f11e0005020 con 0x7f11f0104910 2024-03-28T00:26:23.284 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:23.277+0000 7f11ee000640 1 -- 172.21.15.119:0/3994867265 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11e00049e0 con 0x7f11f0104910 2024-03-28T00:26:23.284 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:23.277+0000 7f11f6c00640 1 -- 172.21.15.119:0/3994867265 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11f01a1e60 con 0x7f11f0104910 2024-03-28T00:26:23.285 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:23.278+0000 7f11f6c00640 1 -- 172.21.15.119:0/3994867265 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11f010aa40 con 0x7f11f0104910 2024-03-28T00:26:23.285 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:23.280+0000 7f11ee000640 1 -- 172.21.15.119:0/3994867265 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 18) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f11e0005560 con 0x7f11f0104910 2024-03-28T00:26:23.286 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:23.281+0000 7f11ee000640 1 --2- 172.21.15.119:0/3994867265 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f11d80a27f0 0x7f11d80a4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:23.286 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:23.281+0000 7f11ee000640 1 -- 172.21.15.119:0/3994867265 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f11e00cea30 con 0x7f11f0104910 2024-03-28T00:26:23.286 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:23.281+0000 7f11f5800640 1 --2- 172.21.15.119:0/3994867265 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f11d80a27f0 0x7f11d80a4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:23.287 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:23.282+0000 7f11f5800640 1 --2- 172.21.15.119:0/3994867265 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f11d80a27f0 0x7f11d80a4cb0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f11e4006010 tx=0x7f11e402df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:26:23.290 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:23.285+0000 7f11ee000640 1 -- 172.21.15.119:0/3994867265 <== mon.2 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f11e00051d0 con 0x7f11f0104910 2024-03-28T00:26:23.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:23 smithi165 ceph-mon[36302]: pgmap v386: 97 pgs: 97 active+clean; 648 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T00:26:23.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:23 smithi165 ceph-mon[36302]: from='client.? 172.21.15.119:0/3244211963' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-03-28T00:26:23.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:23 smithi165 ceph-mon[36302]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-03-28T00:26:23.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:23 smithi119 ceph-mon[31518]: pgmap v386: 97 pgs: 97 active+clean; 648 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T00:26:23.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:23 smithi119 ceph-mon[31518]: from='client.? 172.21.15.119:0/3244211963' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-03-28T00:26:23.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:23 smithi119 ceph-mon[31518]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-03-28T00:26:23.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:23 smithi132 ceph-mon[35941]: pgmap v386: 97 pgs: 97 active+clean; 648 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T00:26:23.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:23 smithi132 ceph-mon[35941]: from='client.? 172.21.15.119:0/3244211963' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-03-28T00:26:23.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:23 smithi132 ceph-mon[35941]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-03-28T00:26:23.542 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:23.536+0000 7f11f6c00640 1 -- 172.21.15.119:0/3994867265 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f11f002d270 con 0x7f11f0104910 2024-03-28T00:26:24.051 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:24.045+0000 7f11ee000640 1 -- 172.21.15.119:0/3994867265 <== mon.2 v2:172.21.15.132:3300/0 7 ==== mgrmap(e 19) v1 ==== 137308+0+0 (secure 0 0 0) 0x7f11e0097ac0 con 0x7f11f0104910 2024-03-28T00:26:24.053 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:24.047+0000 7f11ee000640 1 -- 172.21.15.119:0/3994867265 <== mon.2 v2:172.21.15.132:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) v1 ==== 92+0+0 (secure 0 0 0) 0x7f11e009bb50 con 0x7f11f0104910 2024-03-28T00:26:24.059 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:24.053+0000 7f11f6c00640 1 -- 172.21.15.119:0/3994867265 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f11d80a27f0 msgr2=0x7f11d80a4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:26:24.059 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:24.053+0000 7f11f6c00640 1 --2- 172.21.15.119:0/3994867265 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f11d80a27f0 0x7f11d80a4cb0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f11e4006010 tx=0x7f11e402df90 comp rx=0 tx=0).stop 2024-03-28T00:26:24.059 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:24.053+0000 7f11f6c00640 1 -- 172.21.15.119:0/3994867265 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f11f0104910 msgr2=0x7f11f0199f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:26:24.059 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:24.053+0000 7f11f6c00640 1 --2- 172.21.15.119:0/3994867265 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f11f0104910 0x7f11f0199f90 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f11e000e9f0 tx=0x7f11e000eeb0 comp rx=0 tx=0).stop 2024-03-28T00:26:24.059 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:24.054+0000 7f11f6c00640 1 -- 172.21.15.119:0/3994867265 shutdown_connections 2024-03-28T00:26:24.059 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:24.054+0000 7f11f6c00640 1 --2- 172.21.15.119:0/3994867265 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f11d80a27f0 0x7f11d80a4cb0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:24.060 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:24.054+0000 7f11f6c00640 1 --2- 172.21.15.119:0/3994867265 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f11f01052d0 0x7f11f01a1000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:24.060 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:24.054+0000 7f11f6c00640 1 --2- 172.21.15.119:0/3994867265 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f11f0104910 0x7f11f0199f90 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:24.060 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:24.054+0000 7f11f6c00640 1 --2- 172.21.15.119:0/3994867265 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f11f0103720 0x7f11f0199a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:24.060 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:24.054+0000 7f11f6c00640 1 -- 172.21.15.119:0/3994867265 >> 172.21.15.119:0/3994867265 conn(0x7f11f00fecb0 msgr2=0x7f11f0107c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:26:24.060 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:24.054+0000 7f11f6c00640 1 -- 172.21.15.119:0/3994867265 shutdown_connections 2024-03-28T00:26:24.060 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:24.054+0000 7f11f6c00640 1 -- 172.21.15.119:0/3994867265 wait complete. 2024-03-28T00:26:24.080 DEBUG:teuthology.orchestra.run.smithi119:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-03-28T00:26:24.293 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:24.288+0000 7f75c2600640 1 -- 172.21.15.119:0/1214264234 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f75bc103900 msgr2=0x7f75bc103d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:26:24.293 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:24.288+0000 7f75c2600640 1 --2- 172.21.15.119:0/1214264234 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f75bc103900 0x7f75bc103d20 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f75ac009900 tx=0x7f75ac02bc50 comp rx=0 tx=0).stop 2024-03-28T00:26:24.293 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:24.288+0000 7f75c2600640 1 -- 172.21.15.119:0/1214264234 shutdown_connections 2024-03-28T00:26:24.294 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:24.288+0000 7f75c2600640 1 --2- 172.21.15.119:0/1214264234 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f75bc1054b0 0x7f75bc10bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:24.294 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:24.288+0000 7f75c2600640 1 --2- 172.21.15.119:0/1214264234 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f75bc104af0 0x7f75bc104f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:24.294 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:24.288+0000 7f75c2600640 1 --2- 172.21.15.119:0/1214264234 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f75bc103900 0x7f75bc103d20 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:24.294 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:24.288+0000 7f75c2600640 1 -- 172.21.15.119:0/1214264234 >> 172.21.15.119:0/1214264234 conn(0x7f75bc0fee90 msgr2=0x7f75bc1012b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:26:24.294 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:24.288+0000 7f75c2600640 1 -- 172.21.15.119:0/1214264234 shutdown_connections 2024-03-28T00:26:24.294 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:24.288+0000 7f75c2600640 1 -- 172.21.15.119:0/1214264234 wait complete. 2024-03-28T00:26:24.295 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:24.289+0000 7f75c2600640 1 Processor -- start 2024-03-28T00:26:24.295 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:24.289+0000 7f75c2600640 1 -- start start 2024-03-28T00:26:24.296 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:24.289+0000 7f75c2600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f75bc103900 0x7f75bc199be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:24.296 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:24.289+0000 7f75c2600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f75bc104af0 0x7f75bc19a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:24.296 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:24.289+0000 7f75c2600640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f75bc1054b0 0x7f75bc1a1190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:24.296 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:24.289+0000 7f75c2600640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75bc10f5a0 con 0x7f75bc104af0 2024-03-28T00:26:24.296 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:24.289+0000 7f75c2600640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75bc10f420 con 0x7f75bc1054b0 2024-03-28T00:26:24.296 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:24.289+0000 7f75c2600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75bc10f720 con 0x7f75bc103900 2024-03-28T00:26:24.296 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:24.290+0000 7f75bb400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f75bc103900 0x7f75bc199be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:24.296 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:24.290+0000 7f75baa00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f75bc104af0 0x7f75bc19a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:24.296 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:24.290+0000 7f75bbe00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f75bc1054b0 0x7f75bc1a1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:24.296 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:24.290+0000 7f75bb400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f75bc103900 0x7f75bc199be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.119:44124/0 (socket says 172.21.15.119:44124) 2024-03-28T00:26:24.296 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:24.290+0000 7f75baa00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f75bc104af0 0x7f75bc19a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:48452/0 (socket says 172.21.15.119:48452) 2024-03-28T00:26:24.296 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:24.290+0000 7f75bb400640 1 -- 172.21.15.119:0/441140388 learned_addr learned my addr 172.21.15.119:0/441140388 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:26:24.296 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:24.290+0000 7f75bb400640 1 -- 172.21.15.119:0/441140388 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f75bc1054b0 msgr2=0x7f75bc1a1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:26:24.296 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:24.290+0000 7f75bb400640 1 --2- 172.21.15.119:0/441140388 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f75bc1054b0 0x7f75bc1a1190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:24.296 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:24.290+0000 7f75bb400640 1 -- 172.21.15.119:0/441140388 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f75bc104af0 msgr2=0x7f75bc19a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:26:24.297 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:24.290+0000 7f75bb400640 1 --2- 172.21.15.119:0/441140388 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f75bc104af0 0x7f75bc19a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:24.297 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:24.290+0000 7f75bb400640 1 -- 172.21.15.119:0/441140388 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75bc1a1800 con 0x7f75bc103900 2024-03-28T00:26:24.297 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:24.290+0000 7f75bbe00640 1 --2- 172.21.15.119:0/441140388 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f75bc1054b0 0x7f75bc1a1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T00:26:24.297 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:24.290+0000 7f75baa00640 1 --2- 172.21.15.119:0/441140388 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f75bc104af0 0x7f75bc19a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T00:26:24.297 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:24.291+0000 7f75bb400640 1 --2- 172.21.15.119:0/441140388 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f75bc103900 0x7f75bc199be0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f75ac02c260 tx=0x7f75ac02c670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:26:24.298 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:24.291+0000 7f75b3e00640 1 -- 172.21.15.119:0/441140388 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75ac041070 con 0x7f75bc103900 2024-03-28T00:26:24.298 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:24.291+0000 7f75b3e00640 1 -- 172.21.15.119:0/441140388 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f75ac004450 con 0x7f75bc103900 2024-03-28T00:26:24.298 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:24.291+0000 7f75c2600640 1 -- 172.21.15.119:0/441140388 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75bc1a1a90 con 0x7f75bc103900 2024-03-28T00:26:24.298 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:24.291+0000 7f75b3e00640 1 -- 172.21.15.119:0/441140388 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75ac034440 con 0x7f75bc103900 2024-03-28T00:26:24.298 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:24.291+0000 7f75c2600640 1 -- 172.21.15.119:0/441140388 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75bc1a2030 con 0x7f75bc103900 2024-03-28T00:26:24.298 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:24.292+0000 7f75c2600640 1 -- 172.21.15.119:0/441140388 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75bc10ab50 con 0x7f75bc103900 2024-03-28T00:26:24.301 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:24.296+0000 7f75b3e00640 1 -- 172.21.15.119:0/441140388 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 19) v1 ==== 137308+0+0 (secure 0 0 0) 0x7f75ac044070 con 0x7f75bc103900 2024-03-28T00:26:24.302 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:24.297+0000 7f75b3e00640 1 --2- 172.21.15.119:0/441140388 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f75a00a2970 0x7f75a00a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:24.302 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:24.297+0000 7f75baa00640 1 -- 172.21.15.119:0/441140388 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f75a00a2970 msgr2=0x7f75a00a4e30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6800/1401937096 2024-03-28T00:26:24.303 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:24.297+0000 7f75b3e00640 1 -- 172.21.15.119:0/441140388 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f75ac0f0420 con 0x7f75bc103900 2024-03-28T00:26:24.303 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:24.297+0000 7f75baa00640 1 --2- 172.21.15.119:0/441140388 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f75a00a2970 0x7f75a00a4e30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-28T00:26:24.306 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:24.301+0000 7f75b3e00640 1 -- 172.21.15.119:0/441140388 <== mon.1 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f75ac0bcf10 con 0x7f75bc103900 2024-03-28T00:26:24.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:24 smithi165 ceph-mon[36302]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-03-28T00:26:24.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:24 smithi165 ceph-mon[36302]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-28T00:26:24.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:24 smithi165 ceph-mon[36302]: from='client.? 172.21.15.119:0/3994867265' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-03-28T00:26:24.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:24 smithi165 ceph-mon[36302]: from='client.? ' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-03-28T00:26:24.383 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Mar 28 00:26:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-z[38206]: ignoring --setuser ceph since I am not root 2024-03-28T00:26:24.383 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Mar 28 00:26:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-z[38206]: ignoring --setgroup ceph since I am not root 2024-03-28T00:26:24.383 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Mar 28 00:26:24 smithi165 ceph-mgr[38210]: -- 172.21.15.165:0/2381723317 <== mon.1 v2:172.21.15.165:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x560bc09ae5a0 con 0x560bc0983000 2024-03-28T00:26:24.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:24 smithi119 ceph-mon[31518]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-03-28T00:26:24.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:24 smithi119 ceph-mon[31518]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-28T00:26:24.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:24 smithi119 ceph-mon[31518]: from='client.? 172.21.15.119:0/3994867265' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-03-28T00:26:24.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:24 smithi119 ceph-mon[31518]: from='client.? ' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-03-28T00:26:24.406 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 00:26:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-x[31752]: ignoring --setuser ceph since I am not root 2024-03-28T00:26:24.406 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 00:26:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-x[31752]: ignoring --setgroup ceph since I am not root 2024-03-28T00:26:24.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:24 smithi132 ceph-mon[35941]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-03-28T00:26:24.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:24 smithi132 ceph-mon[35941]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-28T00:26:24.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:24 smithi132 ceph-mon[35941]: from='client.? 172.21.15.119:0/3994867265' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-03-28T00:26:24.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:24 smithi132 ceph-mon[35941]: from='client.? ' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-03-28T00:26:24.453 INFO:journalctl@ceph.mgr.y.smithi132.stdout:Mar 28 00:26:24 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-y[37327]: ignoring --setuser ceph since I am not root 2024-03-28T00:26:24.453 INFO:journalctl@ceph.mgr.y.smithi132.stdout:Mar 28 00:26:24 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-y[37327]: ignoring --setgroup ceph since I am not root 2024-03-28T00:26:24.454 INFO:journalctl@ceph.mgr.y.smithi132.stdout:Mar 28 00:26:24 smithi132 ceph-mgr[37331]: -- 172.21.15.132:0/2437758050 <== mon.2 v2:172.21.15.132:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x562812f6c5a0 con 0x562812f40800 2024-03-28T00:26:24.503 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:24.498+0000 7f75baa00640 1 -- 172.21.15.119:0/441140388 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f75a00a2970 msgr2=0x7f75a00a4e30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6800/1401937096 2024-03-28T00:26:24.503 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:24.498+0000 7f75baa00640 1 --2- 172.21.15.119:0/441140388 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f75a00a2970 0x7f75a00a4e30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-28T00:26:24.510 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:24.505+0000 7f75c2600640 1 -- 172.21.15.119:0/441140388 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f75bc0622a0 con 0x7f75bc103900 2024-03-28T00:26:24.517 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:24.511+0000 7f75b3e00640 1 -- 172.21.15.119:0/441140388 <== mon.1 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v31)=0 v31) v1 ==== 138+0+0 (secure 0 0 0) 0x7f75ac0c0fb0 con 0x7f75bc103900 2024-03-28T00:26:24.520 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:24.514+0000 7f75c2600640 1 -- 172.21.15.119:0/441140388 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f75a00a2970 msgr2=0x7f75a00a4e30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-28T00:26:24.520 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:24.514+0000 7f75c2600640 1 --2- 172.21.15.119:0/441140388 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f75a00a2970 0x7f75a00a4e30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:24.520 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:24.514+0000 7f75c2600640 1 -- 172.21.15.119:0/441140388 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f75bc103900 msgr2=0x7f75bc199be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:26:24.520 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:24.514+0000 7f75c2600640 1 --2- 172.21.15.119:0/441140388 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f75bc103900 0x7f75bc199be0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f75ac02c260 tx=0x7f75ac02c670 comp rx=0 tx=0).stop 2024-03-28T00:26:24.520 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:24.514+0000 7f75c2600640 1 -- 172.21.15.119:0/441140388 shutdown_connections 2024-03-28T00:26:24.520 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:24.514+0000 7f75c2600640 1 --2- 172.21.15.119:0/441140388 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f75a00a2970 0x7f75a00a4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:24.520 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:24.514+0000 7f75c2600640 1 --2- 172.21.15.119:0/441140388 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f75bc1054b0 0x7f75bc1a1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:24.521 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:24.514+0000 7f75c2600640 1 --2- 172.21.15.119:0/441140388 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f75bc104af0 0x7f75bc19a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:24.521 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:24.514+0000 7f75c2600640 1 --2- 172.21.15.119:0/441140388 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f75bc103900 0x7f75bc199be0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:24.521 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:24.514+0000 7f75c2600640 1 -- 172.21.15.119:0/441140388 >> 172.21.15.119:0/441140388 conn(0x7f75bc0fee90 msgr2=0x7f75bc107d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:26:24.521 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:24.514+0000 7f75c2600640 1 -- 172.21.15.119:0/441140388 shutdown_connections 2024-03-28T00:26:24.521 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:24.514+0000 7f75c2600640 1 -- 172.21.15.119:0/441140388 wait complete. 2024-03-28T00:26:24.538 DEBUG:teuthology.orchestra.run.smithi119:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-03-28T00:26:24.731 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:24.726+0000 7f14bbe00640 1 -- 172.21.15.119:0/3916090481 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f14b41028d0 msgr2=0x7f14b4102d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:26:24.732 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:24.726+0000 7f14bbe00640 1 --2- 172.21.15.119:0/3916090481 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f14b41028d0 0x7f14b4102d50 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f14a0009900 tx=0x7f14a002bc40 comp rx=0 tx=0).stop 2024-03-28T00:26:24.732 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:24.726+0000 7f14bbe00640 1 -- 172.21.15.119:0/3916090481 shutdown_connections 2024-03-28T00:26:24.732 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:24.726+0000 7f14bbe00640 1 --2- 172.21.15.119:0/3916090481 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14b4103290 0x7f14b4109b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:24.732 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:24.726+0000 7f14bbe00640 1 --2- 172.21.15.119:0/3916090481 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f14b41028d0 0x7f14b4102d50 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:24.732 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:24.726+0000 7f14bbe00640 1 --2- 172.21.15.119:0/3916090481 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f14b41016e0 0x7f14b4101b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:24.732 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:24.726+0000 7f14bbe00640 1 -- 172.21.15.119:0/3916090481 >> 172.21.15.119:0/3916090481 conn(0x7f14b40fcc30 msgr2=0x7f14b40ff090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:26:24.732 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:24.726+0000 7f14bbe00640 1 -- 172.21.15.119:0/3916090481 shutdown_connections 2024-03-28T00:26:24.732 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:24.726+0000 7f14bbe00640 1 -- 172.21.15.119:0/3916090481 wait complete. 2024-03-28T00:26:24.734 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:24.727+0000 7f14bbe00640 1 Processor -- start 2024-03-28T00:26:24.734 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:24.727+0000 7f14bbe00640 1 -- start start 2024-03-28T00:26:24.734 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:24.727+0000 7f14bbe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f14b41016e0 0x7f14b4199b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:24.734 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:24.727+0000 7f14bbe00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f14b41028d0 0x7f14b419a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:24.734 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:24.727+0000 7f14bbe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14b4103290 0x7f14b41a1120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:24.734 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:24.727+0000 7f14bbe00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14b410d3b0 con 0x7f14b4103290 2024-03-28T00:26:24.735 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:24.727+0000 7f14bbe00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14b410d230 con 0x7f14b41028d0 2024-03-28T00:26:24.735 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:24.727+0000 7f14bbe00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14b410d530 con 0x7f14b41016e0 2024-03-28T00:26:24.735 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:24.727+0000 7f14b9400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f14b41016e0 0x7f14b4199b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:24.735 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:24.727+0000 7f14b9400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f14b41016e0 0x7f14b4199b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.119:44146/0 (socket says 172.21.15.119:44146) 2024-03-28T00:26:24.735 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:24.727+0000 7f14b9400640 1 -- 172.21.15.119:0/3331224940 learned_addr learned my addr 172.21.15.119:0/3331224940 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:26:24.735 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:24.727+0000 7f14b9e00640 1 --2- 172.21.15.119:0/3331224940 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14b4103290 0x7f14b41a1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:24.735 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:24.727+0000 7f14b8a00640 1 --2- 172.21.15.119:0/3331224940 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f14b41028d0 0x7f14b419a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:24.735 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:24.728+0000 7f14b9400640 1 -- 172.21.15.119:0/3331224940 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f14b41028d0 msgr2=0x7f14b419a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:26:24.735 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:24.728+0000 7f14b9400640 1 --2- 172.21.15.119:0/3331224940 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f14b41028d0 0x7f14b419a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:24.735 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:24.728+0000 7f14b9400640 1 -- 172.21.15.119:0/3331224940 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14b4103290 msgr2=0x7f14b41a1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:26:24.735 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:24.728+0000 7f14b9400640 1 --2- 172.21.15.119:0/3331224940 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14b4103290 0x7f14b41a1120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:24.735 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:24.728+0000 7f14b9400640 1 -- 172.21.15.119:0/3331224940 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14b41a1790 con 0x7f14b41016e0 2024-03-28T00:26:24.735 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:24.728+0000 7f14b9400640 1 --2- 172.21.15.119:0/3331224940 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f14b41016e0 0x7f14b4199b70 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f14a800e9c0 tx=0x7f14a800edc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:26:24.735 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:24.728+0000 7f14b2000640 1 -- 172.21.15.119:0/3331224940 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a800ccc0 con 0x7f14b41016e0 2024-03-28T00:26:24.735 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:24.728+0000 7f14b2000640 1 -- 172.21.15.119:0/3331224940 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f14a80035f0 con 0x7f14b41016e0 2024-03-28T00:26:24.736 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:24.728+0000 7f14b2000640 1 -- 172.21.15.119:0/3331224940 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a800ce60 con 0x7f14b41016e0 2024-03-28T00:26:24.736 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:24.728+0000 7f14bbe00640 1 -- 172.21.15.119:0/3331224940 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14b41a1a80 con 0x7f14b41016e0 2024-03-28T00:26:24.736 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:24.728+0000 7f14bbe00640 1 -- 172.21.15.119:0/3331224940 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14b41a1fa0 con 0x7f14b41016e0 2024-03-28T00:26:24.736 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:24.730+0000 7f14bbe00640 1 -- 172.21.15.119:0/3331224940 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14b41a21d0 con 0x7f14b41016e0 2024-03-28T00:26:24.737 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:24.732+0000 7f14b2000640 1 -- 172.21.15.119:0/3331224940 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 19) v1 ==== 137308+0+0 (secure 0 0 0) 0x7f14a80037a0 con 0x7f14b41016e0 2024-03-28T00:26:24.738 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:24.733+0000 7f14b2000640 1 --2- 172.21.15.119:0/3331224940 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f149c0a2970 0x7f149c0a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:24.738 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:24.733+0000 7f14b2000640 1 -- 172.21.15.119:0/3331224940 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f14a80ced90 con 0x7f14b41016e0 2024-03-28T00:26:24.738 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:24.733+0000 7f14b8a00640 1 -- 172.21.15.119:0/3331224940 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f149c0a2970 msgr2=0x7f149c0a4e30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6800/1401937096 2024-03-28T00:26:24.738 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:24.733+0000 7f14b8a00640 1 --2- 172.21.15.119:0/3331224940 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f149c0a2970 0x7f149c0a4e30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-28T00:26:24.742 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:24.737+0000 7f14b2000640 1 -- 172.21.15.119:0/3331224940 <== mon.1 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f14a809be30 con 0x7f14b41016e0 2024-03-28T00:26:24.939 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:24.934+0000 7f14b8a00640 1 -- 172.21.15.119:0/3331224940 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f149c0a2970 msgr2=0x7f149c0a4e30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6800/1401937096 2024-03-28T00:26:24.939 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:24.934+0000 7f14b8a00640 1 --2- 172.21.15.119:0/3331224940 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f149c0a2970 0x7f149c0a4e30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-28T00:26:24.946 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:24.940+0000 7f14bbe00640 1 -- 172.21.15.119:0/3331224940 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f14b41a23a0 con 0x7f14b41016e0 2024-03-28T00:26:24.952 INFO:journalctl@ceph.mgr.y.smithi132.stdout:Mar 28 00:26:24 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-y[37327]: 2024-03-28T00:26:24.609+0000 7f12f9aec200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-28T00:26:24.953 INFO:journalctl@ceph.mgr.y.smithi132.stdout:Mar 28 00:26:24 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-y[37327]: 2024-03-28T00:26:24.703+0000 7f12f9aec200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-28T00:26:24.954 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:24.949+0000 7f14b2000640 1 -- 172.21.15.119:0/3331224940 <== mon.1 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v32)=0 v32) v1 ==== 133+0+0 (secure 0 0 0) 0x7f14a809b730 con 0x7f14b41016e0 2024-03-28T00:26:24.957 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:24.951+0000 7f14bbe00640 1 -- 172.21.15.119:0/3331224940 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f149c0a2970 msgr2=0x7f149c0a4e30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-28T00:26:24.957 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:24.951+0000 7f14bbe00640 1 --2- 172.21.15.119:0/3331224940 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f149c0a2970 0x7f149c0a4e30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:24.957 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:24.951+0000 7f14bbe00640 1 -- 172.21.15.119:0/3331224940 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f14b41016e0 msgr2=0x7f14b4199b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:26:24.957 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:24.951+0000 7f14bbe00640 1 --2- 172.21.15.119:0/3331224940 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f14b41016e0 0x7f14b4199b70 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f14a800e9c0 tx=0x7f14a800edc0 comp rx=0 tx=0).stop 2024-03-28T00:26:24.957 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:24.952+0000 7f14bbe00640 1 -- 172.21.15.119:0/3331224940 shutdown_connections 2024-03-28T00:26:24.958 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:24.952+0000 7f14bbe00640 1 --2- 172.21.15.119:0/3331224940 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f149c0a2970 0x7f149c0a4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:24.958 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:24.952+0000 7f14bbe00640 1 --2- 172.21.15.119:0/3331224940 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14b4103290 0x7f14b41a1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:24.958 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:24.952+0000 7f14bbe00640 1 --2- 172.21.15.119:0/3331224940 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f14b41028d0 0x7f14b419a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:24.958 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:24.952+0000 7f14bbe00640 1 --2- 172.21.15.119:0/3331224940 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f14b41016e0 0x7f14b4199b70 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:24.958 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:24.952+0000 7f14bbe00640 1 -- 172.21.15.119:0/3331224940 >> 172.21.15.119:0/3331224940 conn(0x7f14b40fcc30 msgr2=0x7f14b40738d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:26:24.958 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:24.952+0000 7f14bbe00640 1 -- 172.21.15.119:0/3331224940 shutdown_connections 2024-03-28T00:26:24.958 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:24.952+0000 7f14bbe00640 1 -- 172.21.15.119:0/3331224940 wait complete. 2024-03-28T00:26:24.973 DEBUG:teuthology.orchestra.run.smithi119:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-03-28T00:26:25.042 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Mar 28 00:26:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-z[38206]: 2024-03-28T00:26:24.772+0000 7f62e6636200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-28T00:26:25.042 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Mar 28 00:26:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-z[38206]: 2024-03-28T00:26:24.888+0000 7f62e6636200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-28T00:26:25.158 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 00:26:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-x[31752]: 2024-03-28T00:26:24.895+0000 7fac9e46e200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-28T00:26:25.158 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 00:26:25 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-x[31752]: 2024-03-28T00:26:25.020+0000 7fac9e46e200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-28T00:26:25.158 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:25 smithi119 ceph-mon[31518]: from='client.? ' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-03-28T00:26:25.158 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:25 smithi119 ceph-mon[31518]: mgrmap e19: x(active, since 10m), standbys: z, y 2024-03-28T00:26:25.158 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:25 smithi119 ceph-mon[31518]: from='client.? ' entity='client.admin' 2024-03-28T00:26:25.158 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:25 smithi119 ceph-mon[31518]: from='client.? ' entity='client.admin' 2024-03-28T00:26:25.159 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:25.153+0000 7f13a0a00640 1 -- 172.21.15.119:0/1447895550 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f139c1054b0 msgr2=0x7f139c10bd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:26:25.159 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:25.153+0000 7f13a0a00640 1 --2- 172.21.15.119:0/1447895550 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f139c1054b0 0x7f139c10bd40 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f1394009be0 tx=0x7f139402bef0 comp rx=0 tx=0).stop 2024-03-28T00:26:25.161 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:25.155+0000 7f13a0a00640 1 -- 172.21.15.119:0/1447895550 shutdown_connections 2024-03-28T00:26:25.161 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:25.155+0000 7f13a0a00640 1 --2- 172.21.15.119:0/1447895550 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f139c1054b0 0x7f139c10bd40 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:25.161 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:25.155+0000 7f13a0a00640 1 --2- 172.21.15.119:0/1447895550 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f139c104af0 0x7f139c104f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:25.162 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:25.155+0000 7f13a0a00640 1 --2- 172.21.15.119:0/1447895550 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f139c103900 0x7f139c103d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:25.162 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:25.155+0000 7f13a0a00640 1 -- 172.21.15.119:0/1447895550 >> 172.21.15.119:0/1447895550 conn(0x7f139c0fee90 msgr2=0x7f139c1012b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:26:25.162 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:25.156+0000 7f13a0a00640 1 -- 172.21.15.119:0/1447895550 shutdown_connections 2024-03-28T00:26:25.162 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:25.156+0000 7f13a0a00640 1 -- 172.21.15.119:0/1447895550 wait complete. 2024-03-28T00:26:25.164 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:25.157+0000 7f13a0a00640 1 Processor -- start 2024-03-28T00:26:25.164 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:25.157+0000 7f13a0a00640 1 -- start start 2024-03-28T00:26:25.164 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:25.157+0000 7f13a0a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f139c103900 0x7f139c199ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:25.164 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:25.157+0000 7f13a0a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f139c104af0 0x7f139c19a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:25.164 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:25.157+0000 7f13a0a00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f139c1054b0 0x7f139c1a1090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:25.164 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:25.157+0000 7f13a0a00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f139c10f5b0 con 0x7f139c104af0 2024-03-28T00:26:25.164 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:25.157+0000 7f13a0a00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f139c10f430 con 0x7f139c1054b0 2024-03-28T00:26:25.164 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:25.157+0000 7f13a0a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f139c10f730 con 0x7f139c103900 2024-03-28T00:26:25.165 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:25.158+0000 7f1399e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f139c103900 0x7f139c199ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:25.165 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:25.158+0000 7f1399e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f139c103900 0x7f139c199ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.119:44174/0 (socket says 172.21.15.119:44174) 2024-03-28T00:26:25.165 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:25.158+0000 7f1399e00640 1 -- 172.21.15.119:0/363652436 learned_addr learned my addr 172.21.15.119:0/363652436 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:26:25.165 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:25.158+0000 7f1399400640 1 --2- 172.21.15.119:0/363652436 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f139c104af0 0x7f139c19a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:25.165 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:25.158+0000 7f139a800640 1 --2- 172.21.15.119:0/363652436 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f139c1054b0 0x7f139c1a1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:25.165 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:25.158+0000 7f1399e00640 1 -- 172.21.15.119:0/363652436 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f139c1054b0 msgr2=0x7f139c1a1090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:26:25.165 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:25.158+0000 7f1399e00640 1 --2- 172.21.15.119:0/363652436 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f139c1054b0 0x7f139c1a1090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:25.165 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:25.158+0000 7f1399e00640 1 -- 172.21.15.119:0/363652436 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f139c104af0 msgr2=0x7f139c19a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:26:25.165 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:25.158+0000 7f1399e00640 1 --2- 172.21.15.119:0/363652436 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f139c104af0 0x7f139c19a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:25.165 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:25.158+0000 7f1399e00640 1 -- 172.21.15.119:0/363652436 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f139c1a1700 con 0x7f139c103900 2024-03-28T00:26:25.165 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:25.158+0000 7f139a800640 1 --2- 172.21.15.119:0/363652436 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f139c1054b0 0x7f139c1a1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T00:26:25.165 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:25.158+0000 7f1399e00640 1 --2- 172.21.15.119:0/363652436 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f139c103900 0x7f139c199ae0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f138c00e9a0 tx=0x7f138c00ee60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:26:25.165 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:25.159+0000 7f1392a00640 1 -- 172.21.15.119:0/363652436 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f138c00cd80 con 0x7f139c103900 2024-03-28T00:26:25.165 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:25.159+0000 7f13a0a00640 1 -- 172.21.15.119:0/363652436 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f139c1a19f0 con 0x7f139c103900 2024-03-28T00:26:25.166 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:25.159+0000 7f1392a00640 1 -- 172.21.15.119:0/363652436 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f138c010910 con 0x7f139c103900 2024-03-28T00:26:25.166 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:25.159+0000 7f1392a00640 1 -- 172.21.15.119:0/363652436 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f138c018a40 con 0x7f139c103900 2024-03-28T00:26:25.166 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:25.159+0000 7f13a0a00640 1 -- 172.21.15.119:0/363652436 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f139c1a1f90 con 0x7f139c103900 2024-03-28T00:26:25.166 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:25.160+0000 7f13a0a00640 1 -- 172.21.15.119:0/363652436 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f139c10ab50 con 0x7f139c103900 2024-03-28T00:26:25.167 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:25.162+0000 7f1392a00640 1 -- 172.21.15.119:0/363652436 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 19) v1 ==== 137308+0+0 (secure 0 0 0) 0x7f138c010ac0 con 0x7f139c103900 2024-03-28T00:26:25.168 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:25.163+0000 7f1392a00640 1 --2- 172.21.15.119:0/363652436 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f13800a28f0 0x7f13800a4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:25.168 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:25.163+0000 7f1399400640 1 -- 172.21.15.119:0/363652436 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f13800a28f0 msgr2=0x7f13800a4db0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6800/1401937096 2024-03-28T00:26:25.169 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:25.163+0000 7f1392a00640 1 -- 172.21.15.119:0/363652436 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f138c0cee80 con 0x7f139c103900 2024-03-28T00:26:25.169 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:25.163+0000 7f1399400640 1 --2- 172.21.15.119:0/363652436 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f13800a28f0 0x7f13800a4db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-28T00:26:25.176 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:25.171+0000 7f1392a00640 1 -- 172.21.15.119:0/363652436 <== mon.1 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f138c053740 con 0x7f139c103900 2024-03-28T00:26:25.369 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:25.364+0000 7f1399400640 1 -- 172.21.15.119:0/363652436 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f13800a28f0 msgr2=0x7f13800a4db0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6800/1401937096 2024-03-28T00:26:25.369 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:25.364+0000 7f1399400640 1 --2- 172.21.15.119:0/363652436 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f13800a28f0 0x7f13800a4db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-28T00:26:25.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:25 smithi165 ceph-mon[36302]: from='client.? ' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-03-28T00:26:25.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:25 smithi165 ceph-mon[36302]: mgrmap e19: x(active, since 10m), standbys: z, y 2024-03-28T00:26:25.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:25 smithi165 ceph-mon[36302]: from='client.? ' entity='client.admin' 2024-03-28T00:26:25.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:25 smithi165 ceph-mon[36302]: from='client.? ' entity='client.admin' 2024-03-28T00:26:25.396 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:25.391+0000 7f13a0a00640 1 -- 172.21.15.119:0/363652436 --> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f139c061ae0 con 0x7f13800a28f0 2024-03-28T00:26:25.444 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:25 smithi132 ceph-mon[35941]: from='client.? ' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-03-28T00:26:25.444 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:25 smithi132 ceph-mon[35941]: mgrmap e19: x(active, since 10m), standbys: z, y 2024-03-28T00:26:25.444 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:25 smithi132 ceph-mon[35941]: from='client.? ' entity='client.admin' 2024-03-28T00:26:25.444 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:25 smithi132 ceph-mon[35941]: from='client.? ' entity='client.admin' 2024-03-28T00:26:25.703 INFO:journalctl@ceph.mgr.y.smithi132.stdout:Mar 28 00:26:25 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-y[37327]: 2024-03-28T00:26:25.438+0000 7f12f9aec200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-28T00:26:25.703 INFO:journalctl@ceph.mgr.y.smithi132.stdout:Mar 28 00:26:25 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-y[37327]: 2024-03-28T00:26:25.564+0000 7f12f9aec200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-28T00:26:25.770 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:25.765+0000 7f1399400640 1 -- 172.21.15.119:0/363652436 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f13800a28f0 msgr2=0x7f13800a4db0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6800/1401937096 2024-03-28T00:26:25.771 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:25.765+0000 7f1399400640 1 --2- 172.21.15.119:0/363652436 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f13800a28f0 0x7f13800a4db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-28T00:26:25.880 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Mar 28 00:26:25 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-z[38206]: 2024-03-28T00:26:25.603+0000 7f62e6636200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-28T00:26:25.880 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Mar 28 00:26:25 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-z[38206]: 2024-03-28T00:26:25.716+0000 7f62e6636200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-28T00:26:26.119 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 00:26:25 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-x[31752]: 2024-03-28T00:26:25.813+0000 7fac9e46e200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-28T00:26:26.119 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 00:26:25 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-x[31752]: 2024-03-28T00:26:25.936+0000 7fac9e46e200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-28T00:26:26.132 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Mar 28 00:26:25 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-z[38206]: 2024-03-28T00:26:25.874+0000 7f62e6636200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-28T00:26:26.132 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Mar 28 00:26:26 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-z[38206]: 2024-03-28T00:26:26.012+0000 7f62e6636200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-28T00:26:26.164 INFO:journalctl@ceph.mgr.y.smithi132.stdout:Mar 28 00:26:25 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-y[37327]: 2024-03-28T00:26:25.708+0000 7f12f9aec200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-28T00:26:26.165 INFO:journalctl@ceph.mgr.y.smithi132.stdout:Mar 28 00:26:25 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-y[37327]: 2024-03-28T00:26:25.805+0000 7f12f9aec200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-28T00:26:26.165 INFO:journalctl@ceph.mgr.y.smithi132.stdout:Mar 28 00:26:25 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-y[37327]: 2024-03-28T00:26:25.889+0000 7f12f9aec200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-28T00:26:26.165 INFO:journalctl@ceph.mgr.y.smithi132.stdout:Mar 28 00:26:26 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-y[37327]: 2024-03-28T00:26:26.158+0000 7f12f9aec200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-28T00:26:26.406 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 00:26:26 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-x[31752]: 2024-03-28T00:26:26.113+0000 7fac9e46e200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-28T00:26:26.406 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 00:26:26 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-x[31752]: 2024-03-28T00:26:26.238+0000 7fac9e46e200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-28T00:26:26.406 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 00:26:26 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-x[31752]: 2024-03-28T00:26:26.355+0000 7fac9e46e200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-28T00:26:26.453 INFO:journalctl@ceph.mgr.y.smithi132.stdout:Mar 28 00:26:26 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-y[37327]: 2024-03-28T00:26:26.269+0000 7f12f9aec200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-28T00:26:26.453 INFO:journalctl@ceph.mgr.y.smithi132.stdout:Mar 28 00:26:26 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-y[37327]: 2024-03-28T00:26:26.357+0000 7f12f9aec200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-28T00:26:26.467 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Mar 28 00:26:26 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-z[38206]: 2024-03-28T00:26:26.126+0000 7f62e6636200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-28T00:26:26.467 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Mar 28 00:26:26 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-z[38206]: 2024-03-28T00:26:26.461+0000 7f62e6636200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-28T00:26:26.572 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:26.566+0000 7f1399400640 1 -- 172.21.15.119:0/363652436 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f13800a28f0 msgr2=0x7f13800a4db0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6800/1401937096 2024-03-28T00:26:26.572 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:26.566+0000 7f1399400640 1 --2- 172.21.15.119:0/363652436 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f13800a28f0 0x7f13800a4db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-28T00:26:26.822 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Mar 28 00:26:26 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-z[38206]: 2024-03-28T00:26:26.586+0000 7f62e6636200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-28T00:26:26.822 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Mar 28 00:26:26 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-z[38206]: 2024-03-28T00:26:26.689+0000 7f62e6636200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-28T00:26:26.822 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Mar 28 00:26:26 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-z[38206]: 2024-03-28T00:26:26.816+0000 7f62e6636200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-28T00:26:26.946 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 00:26:26 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-x[31752]: 2024-03-28T00:26:26.689+0000 7fac9e46e200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-28T00:26:26.947 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 00:26:26 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-x[31752]: 2024-03-28T00:26:26.829+0000 7fac9e46e200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-28T00:26:26.952 INFO:journalctl@ceph.mgr.y.smithi132.stdout:Mar 28 00:26:26 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-y[37327]: 2024-03-28T00:26:26.453+0000 7f12f9aec200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-28T00:26:26.953 INFO:journalctl@ceph.mgr.y.smithi132.stdout:Mar 28 00:26:26 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-y[37327]: 2024-03-28T00:26:26.633+0000 7f12f9aec200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-28T00:26:27.132 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Mar 28 00:26:27 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-z[38206]: 2024-03-28T00:26:27.017+0000 7f62e6636200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-28T00:26:27.287 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 00:26:26 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-x[31752]: 2024-03-28T00:26:26.940+0000 7fac9e46e200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-28T00:26:27.287 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 00:26:27 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-x[31752]: 2024-03-28T00:26:27.063+0000 7fac9e46e200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-28T00:26:27.287 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 00:26:27 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-x[31752]: 2024-03-28T00:26:27.281+0000 7fac9e46e200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-28T00:26:28.174 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:28.168+0000 7f1399400640 1 -- 172.21.15.119:0/363652436 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f13800a28f0 msgr2=0x7f13800a4db0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6800/1401937096 2024-03-28T00:26:28.174 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:28.168+0000 7f1399400640 1 --2- 172.21.15.119:0/363652436 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f13800a28f0 0x7f13800a4db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-03-28T00:26:28.203 INFO:journalctl@ceph.mgr.y.smithi132.stdout:Mar 28 00:26:27 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-y[37327]: 2024-03-28T00:26:27.699+0000 7f12f9aec200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-28T00:26:28.203 INFO:journalctl@ceph.mgr.y.smithi132.stdout:Mar 28 00:26:27 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-y[37327]: 2024-03-28T00:26:27.843+0000 7f12f9aec200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-28T00:26:28.549 INFO:journalctl@ceph.mgr.y.smithi132.stdout:Mar 28 00:26:28 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-y[37327]: 2024-03-28T00:26:28.216+0000 7f12f9aec200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-28T00:26:28.549 INFO:journalctl@ceph.mgr.y.smithi132.stdout:Mar 28 00:26:28 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-y[37327]: 2024-03-28T00:26:28.306+0000 7f12f9aec200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-28T00:26:28.633 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Mar 28 00:26:28 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-z[38206]: 2024-03-28T00:26:28.239+0000 7f62e6636200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-28T00:26:28.634 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Mar 28 00:26:28 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-z[38206]: 2024-03-28T00:26:28.420+0000 7f62e6636200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-28T00:26:28.906 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 00:26:28 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-x[31752]: 2024-03-28T00:26:28.581+0000 7fac9e46e200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-28T00:26:28.906 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 00:26:28 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-x[31752]: 2024-03-28T00:26:28.754+0000 7fac9e46e200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-28T00:26:28.953 INFO:journalctl@ceph.mgr.y.smithi132.stdout:Mar 28 00:26:28 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-y[37327]: 2024-03-28T00:26:28.543+0000 7f12f9aec200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-28T00:26:28.953 INFO:journalctl@ceph.mgr.y.smithi132.stdout:Mar 28 00:26:28 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-y[37327]: 2024-03-28T00:26:28.744+0000 7f12f9aec200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-28T00:26:29.133 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Mar 28 00:26:28 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-z[38206]: 2024-03-28T00:26:28.853+0000 7f62e6636200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-28T00:26:29.133 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Mar 28 00:26:28 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-z[38206]: 2024-03-28T00:26:28.977+0000 7f62e6636200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-28T00:26:29.379 INFO:journalctl@ceph.mgr.y.smithi132.stdout:Mar 28 00:26:29 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-y[37327]: 2024-03-28T00:26:29.070+0000 7f12f9aec200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-28T00:26:29.380 INFO:journalctl@ceph.mgr.y.smithi132.stdout:Mar 28 00:26:29 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-y[37327]: 2024-03-28T00:26:29.372+0000 7f12f9aec200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-28T00:26:29.529 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Mar 28 00:26:29 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-z[38206]: 2024-03-28T00:26:29.263+0000 7f62e6636200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-28T00:26:29.563 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 00:26:29 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-x[31752]: 2024-03-28T00:26:29.180+0000 7fac9e46e200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-28T00:26:29.564 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 00:26:29 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-x[31752]: 2024-03-28T00:26:29.298+0000 7fac9e46e200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-28T00:26:29.564 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 00:26:29 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-x[31752]: 2024-03-28T00:26:29.556+0000 7fac9e46e200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-28T00:26:29.883 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Mar 28 00:26:29 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-z[38206]: 2024-03-28T00:26:29.523+0000 7f62e6636200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-28T00:26:29.905 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 00:26:29 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-x[31752]: 2024-03-28T00:26:29.804+0000 7fac9e46e200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-28T00:26:29.952 INFO:journalctl@ceph.mgr.y.smithi132.stdout:Mar 28 00:26:29 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-y[37327]: 2024-03-28T00:26:29.628+0000 7f12f9aec200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-28T00:26:30.198 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Mar 28 00:26:29 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-z[38206]: 2024-03-28T00:26:29.896+0000 7f62e6636200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-28T00:26:30.502 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 00:26:30 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-x[31752]: 2024-03-28T00:26:30.179+0000 7fac9e46e200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-28T00:26:30.503 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 00:26:30 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-x[31752]: 2024-03-28T00:26:30.495+0000 7fac9e46e200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-28T00:26:30.633 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Mar 28 00:26:30 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-z[38206]: 2024-03-28T00:26:30.192+0000 7f62e6636200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-28T00:26:30.633 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Mar 28 00:26:30 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-z[38206]: 2024-03-28T00:26:30.435+0000 7f62e6636200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-28T00:26:30.953 INFO:journalctl@ceph.mgr.y.smithi132.stdout:Mar 28 00:26:30 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-y[37327]: 2024-03-28T00:26:30.533+0000 7f12f9aec200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-28T00:26:31.155 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 00:26:30 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-x[31752]: 2024-03-28T00:26:30.799+0000 7fac9e46e200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-28T00:26:31.377 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:31.371+0000 7f1399400640 1 -- 172.21.15.119:0/363652436 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f13800a28f0 msgr2=0x7f13800a4db0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6800/1401937096 2024-03-28T00:26:31.378 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:31.371+0000 7f1399400640 1 --2- 172.21.15.119:0/363652436 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f13800a28f0 0x7f13800a4db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2024-03-28T00:26:31.606 INFO:journalctl@ceph.mgr.y.smithi132.stdout:Mar 28 00:26:31 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-y[37327]: 2024-03-28T00:26:31.230+0000 7f12f9aec200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-28T00:26:31.606 INFO:journalctl@ceph.mgr.y.smithi132.stdout:Mar 28 00:26:31 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-y[37327]: 2024-03-28T00:26:31.355+0000 7f12f9aec200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-28T00:26:31.693 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:31.686+0000 7f1392a00640 1 -- 172.21.15.119:0/363652436 <== mon.1 v2:172.21.15.165:3300/0 7 ==== mgrmap(e 20) v1 ==== 137308+0+0 (secure 0 0 0) 0x7f138c097e90 con 0x7f139c103900 2024-03-28T00:26:31.697 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Mar 28 00:26:31 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-z[38206]: 2024-03-28T00:26:31.388+0000 7f62e6636200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-28T00:26:31.953 INFO:journalctl@ceph.mgr.y.smithi132.stdout:Mar 28 00:26:31 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-y[37327]: 2024-03-28T00:26:31.599+0000 7f12f9aec200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-28T00:26:31.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:31 smithi132 ceph-mon[35941]: Standby manager daemon y restarted 2024-03-28T00:26:31.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:31 smithi132 ceph-mon[35941]: Standby manager daemon y started 2024-03-28T00:26:31.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:31 smithi132 ceph-mon[35941]: from='mgr.? 172.21.15.132:0/3252468504' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-28T00:26:31.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:31 smithi132 ceph-mon[35941]: from='mgr.? 172.21.15.132:0/3252468504' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-28T00:26:31.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:31 smithi132 ceph-mon[35941]: from='mgr.? 172.21.15.132:0/3252468504' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-28T00:26:31.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:31 smithi132 ceph-mon[35941]: from='mgr.? 172.21.15.132:0/3252468504' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-28T00:26:32.072 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:31 smithi165 ceph-mon[36302]: Standby manager daemon y restarted 2024-03-28T00:26:32.072 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:31 smithi165 ceph-mon[36302]: Standby manager daemon y started 2024-03-28T00:26:32.072 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:31 smithi165 ceph-mon[36302]: from='mgr.? 172.21.15.132:0/3252468504' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-28T00:26:32.072 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:31 smithi165 ceph-mon[36302]: from='mgr.? 172.21.15.132:0/3252468504' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-28T00:26:32.073 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:31 smithi165 ceph-mon[36302]: from='mgr.? 172.21.15.132:0/3252468504' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-28T00:26:32.073 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:31 smithi165 ceph-mon[36302]: from='mgr.? 172.21.15.132:0/3252468504' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-28T00:26:32.155 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 00:26:31 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-x[31752]: 2024-03-28T00:26:31.840+0000 7fac9e46e200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-28T00:26:32.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:31 smithi119 ceph-mon[31518]: Standby manager daemon y restarted 2024-03-28T00:26:32.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:31 smithi119 ceph-mon[31518]: Standby manager daemon y started 2024-03-28T00:26:32.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:31 smithi119 ceph-mon[31518]: from='mgr.? 172.21.15.132:0/3252468504' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-28T00:26:32.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:31 smithi119 ceph-mon[31518]: from='mgr.? 172.21.15.132:0/3252468504' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-28T00:26:32.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:31 smithi119 ceph-mon[31518]: from='mgr.? 172.21.15.132:0/3252468504' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-28T00:26:32.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:31 smithi119 ceph-mon[31518]: from='mgr.? 172.21.15.132:0/3252468504' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-28T00:26:32.383 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Mar 28 00:26:32 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-z[38206]: 2024-03-28T00:26:32.066+0000 7f62e6636200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-28T00:26:32.383 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Mar 28 00:26:32 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-z[38206]: 2024-03-28T00:26:32.167+0000 7f62e6636200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-28T00:26:32.700 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Mar 28 00:26:32 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-z[38206]: 2024-03-28T00:26:32.414+0000 7f62e6636200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-28T00:26:32.721 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:32.714+0000 7f1392a00640 1 -- 172.21.15.119:0/363652436 <== mon.1 v2:172.21.15.165:3300/0 8 ==== mgrmap(e 21) v1 ==== 137308+0+0 (secure 0 0 0) 0x7f138c0cca00 con 0x7f139c103900 2024-03-28T00:26:32.906 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 00:26:32 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-x[31752]: 2024-03-28T00:26:32.536+0000 7fac9e46e200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-28T00:26:32.906 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 00:26:32 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-x[31752]: 2024-03-28T00:26:32.668+0000 7fac9e46e200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-28T00:26:32.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:32 smithi119 ceph-mon[31518]: mgrmap e20: x(active, since 10m), standbys: z, y 2024-03-28T00:26:32.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:32 smithi119 ceph-mon[31518]: Standby manager daemon z restarted 2024-03-28T00:26:32.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:32 smithi119 ceph-mon[31518]: Standby manager daemon z started 2024-03-28T00:26:32.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:32 smithi119 ceph-mon[31518]: from='mgr.? 172.21.15.165:0/4095061745' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-28T00:26:32.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:32 smithi119 ceph-mon[31518]: from='mgr.? 172.21.15.165:0/4095061745' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-28T00:26:32.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:32 smithi119 ceph-mon[31518]: from='mgr.? 172.21.15.165:0/4095061745' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-28T00:26:32.907 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:32 smithi119 ceph-mon[31518]: from='mgr.? 172.21.15.165:0/4095061745' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-28T00:26:32.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:32 smithi132 ceph-mon[35941]: mgrmap e20: x(active, since 10m), standbys: z, y 2024-03-28T00:26:32.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:32 smithi132 ceph-mon[35941]: Standby manager daemon z restarted 2024-03-28T00:26:32.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:32 smithi132 ceph-mon[35941]: Standby manager daemon z started 2024-03-28T00:26:32.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:32 smithi132 ceph-mon[35941]: from='mgr.? 172.21.15.165:0/4095061745' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-28T00:26:32.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:32 smithi132 ceph-mon[35941]: from='mgr.? 172.21.15.165:0/4095061745' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-28T00:26:32.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:32 smithi132 ceph-mon[35941]: from='mgr.? 172.21.15.165:0/4095061745' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-28T00:26:32.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:32 smithi132 ceph-mon[35941]: from='mgr.? 172.21.15.165:0/4095061745' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-28T00:26:32.989 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:32.983+0000 7f1392a00640 1 -- 172.21.15.119:0/363652436 <== mon.1 v2:172.21.15.165:3300/0 9 ==== mgrmap(e 22) v1 ==== 136975+0+0 (secure 0 0 0) 0x7f138c01bc40 con 0x7f139c103900 2024-03-28T00:26:32.990 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:32.983+0000 7f1392a00640 1 -- 172.21.15.119:0/363652436 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f13800a28f0 msgr2=0x7f13800a4db0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-28T00:26:32.990 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:32.983+0000 7f1392a00640 1 --2- 172.21.15.119:0/363652436 >> [v2:172.21.15.119:6800/1401937096,v1:172.21.15.119:6801/1401937096] conn(0x7f13800a28f0 0x7f13800a4db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:33.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:32 smithi165 ceph-mon[36302]: mgrmap e20: x(active, since 10m), standbys: z, y 2024-03-28T00:26:33.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:32 smithi165 ceph-mon[36302]: Standby manager daemon z restarted 2024-03-28T00:26:33.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:32 smithi165 ceph-mon[36302]: Standby manager daemon z started 2024-03-28T00:26:33.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:32 smithi165 ceph-mon[36302]: from='mgr.? 172.21.15.165:0/4095061745' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-28T00:26:33.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:32 smithi165 ceph-mon[36302]: from='mgr.? 172.21.15.165:0/4095061745' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-28T00:26:33.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:32 smithi165 ceph-mon[36302]: from='mgr.? 172.21.15.165:0/4095061745' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-28T00:26:33.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:32 smithi165 ceph-mon[36302]: from='mgr.? 172.21.15.165:0/4095061745' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-28T00:26:33.271 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 00:26:32 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-x[31752]: 2024-03-28T00:26:32.960+0000 7fac9e46e200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-28T00:26:33.271 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 00:26:33 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-x[31752]: 2024-03-28T00:26:33.263+0000 7fabefa00700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-28T00:26:33.656 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 00:26:33 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-x[31752]: 2024-03-28T00:26:33.263+0000 7fabefa00700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-28T00:26:33.657 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 00:26:33 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-x[31752]: 2024-03-28T00:26:33.263+0000 7fabefa00700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-28T00:26:33.657 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 00:26:33 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-x[31752]: 2024-03-28T00:26:33.263+0000 7fabefa00700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-28T00:26:33.657 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 00:26:33 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-x[31752]: 2024-03-28T00:26:33.263+0000 7fabefa00700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-28T00:26:33.657 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 00:26:33 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-x[31752]: 2024-03-28T00:26:33.264+0000 7fabf3600700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-28T00:26:33.657 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 00:26:33 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-x[31752]: 2024-03-28T00:26:33.264+0000 7fabf3600700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-28T00:26:33.657 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 00:26:33 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-x[31752]: 2024-03-28T00:26:33.264+0000 7fabf3600700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-28T00:26:33.657 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 00:26:33 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-x[31752]: 2024-03-28T00:26:33.264+0000 7fabf3600700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-28T00:26:33.657 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 00:26:33 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-x[31752]: 2024-03-28T00:26:33.264+0000 7fabf3600700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-28T00:26:34.003 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:33.997+0000 7f1392a00640 1 -- 172.21.15.119:0/363652436 <== mon.1 v2:172.21.15.165:3300/0 10 ==== mgrmap(e 23) v1 ==== 137102+0+0 (secure 0 0 0) 0x7f138c098760 con 0x7f139c103900 2024-03-28T00:26:34.005 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:33.998+0000 7f1392a00640 1 --2- 172.21.15.119:0/363652436 >> [v2:172.21.15.119:6800/4273482239,v1:172.21.15.119:6801/4273482239] conn(0x7f1380067f30 0x7f1380068310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:34.005 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:33.998+0000 7f1392a00640 1 -- 172.21.15.119:0/363652436 --> [v2:172.21.15.119:6800/4273482239,v1:172.21.15.119:6801/4273482239] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f138c023cc0 con 0x7f1380067f30 2024-03-28T00:26:34.016 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:34.010+0000 7f1399400640 1 --2- 172.21.15.119:0/363652436 >> [v2:172.21.15.119:6800/4273482239,v1:172.21.15.119:6801/4273482239] conn(0x7f1380067f30 0x7f1380068310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:34.022 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:34.016+0000 7f1399400640 1 --2- 172.21.15.119:0/363652436 >> [v2:172.21.15.119:6800/4273482239,v1:172.21.15.119:6801/4273482239] conn(0x7f1380067f30 0x7f1380068310 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f139c19aff0 tx=0x7f1388005f90 comp rx=0 tx=0).ready entity=mgr.25277 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:26:34.111 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:34.105+0000 7f1392a00640 1 -- 172.21.15.119:0/363652436 <== mgr.25277 v2:172.21.15.119:6800/4273482239 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f138c023cc0 con 0x7f1380067f30 2024-03-28T00:26:34.111 INFO:teuthology.orchestra.run.smithi119.stdout:Schedule set for path / 2024-03-28T00:26:34.115 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:34.108+0000 7f13a0a00640 1 -- 172.21.15.119:0/363652436 >> [v2:172.21.15.119:6800/4273482239,v1:172.21.15.119:6801/4273482239] conn(0x7f1380067f30 msgr2=0x7f1380068310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:26:34.115 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:34.108+0000 7f13a0a00640 1 --2- 172.21.15.119:0/363652436 >> [v2:172.21.15.119:6800/4273482239,v1:172.21.15.119:6801/4273482239] conn(0x7f1380067f30 0x7f1380068310 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f139c19aff0 tx=0x7f1388005f90 comp rx=0 tx=0).stop 2024-03-28T00:26:34.115 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:34.108+0000 7f13a0a00640 1 -- 172.21.15.119:0/363652436 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f139c103900 msgr2=0x7f139c199ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:26:34.115 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:34.108+0000 7f13a0a00640 1 --2- 172.21.15.119:0/363652436 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f139c103900 0x7f139c199ae0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f138c00e9a0 tx=0x7f138c00ee60 comp rx=0 tx=0).stop 2024-03-28T00:26:34.115 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:34.108+0000 7f139a800640 1 -- 172.21.15.119:0/363652436 reap_dead start 2024-03-28T00:26:34.115 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:34.108+0000 7f13a0a00640 1 -- 172.21.15.119:0/363652436 shutdown_connections 2024-03-28T00:26:34.115 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:34.108+0000 7f13a0a00640 1 -- 172.21.15.119:0/363652436 >> 172.21.15.119:0/363652436 conn(0x7f139c0fee90 msgr2=0x7f139c107d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:26:34.115 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:34.108+0000 7f13a0a00640 1 -- 172.21.15.119:0/363652436 shutdown_connections 2024-03-28T00:26:34.115 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:34.108+0000 7f13a0a00640 1 -- 172.21.15.119:0/363652436 wait complete. 2024-03-28T00:26:34.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:33 smithi165 ceph-mon[36302]: mgrmap e21: x(active, since 10m), standbys: y, z 2024-03-28T00:26:34.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:33 smithi165 ceph-mon[36302]: Active manager daemon x restarted 2024-03-28T00:26:34.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:33 smithi165 ceph-mon[36302]: Activating manager daemon x 2024-03-28T00:26:34.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:33 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-28T00:26:34.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:33 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-28T00:26:34.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:33 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-28T00:26:34.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:33 smithi165 ceph-mon[36302]: osdmap e88: 12 total, 12 up, 12 in 2024-03-28T00:26:34.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:33 smithi165 ceph-mon[36302]: mgrmap e22: x(active, starting, since 0.0281657s), standbys: y, z 2024-03-28T00:26:34.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:33 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-28T00:26:34.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:33 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-28T00:26:34.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:33 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-28T00:26:34.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:33 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-28T00:26:34.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:33 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-28T00:26:34.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:33 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-28T00:26:34.135 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:33 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-28T00:26:34.135 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:33 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-28T00:26:34.135 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:33 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-28T00:26:34.135 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:33 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-28T00:26:34.135 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:33 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-28T00:26:34.135 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:33 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-28T00:26:34.135 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:33 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-28T00:26:34.135 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:33 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-28T00:26:34.135 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:33 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-28T00:26:34.135 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:33 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-28T00:26:34.135 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:33 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-28T00:26:34.135 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:33 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-28T00:26:34.135 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:33 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-28T00:26:34.135 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:33 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-28T00:26:34.135 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:33 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-28T00:26:34.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:33 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-28T00:26:34.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:33 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-28T00:26:34.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:33 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-28T00:26:34.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:33 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-28T00:26:34.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:33 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-28T00:26:34.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:33 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-28T00:26:34.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:33 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-28T00:26:34.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:33 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-28T00:26:34.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:33 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-28T00:26:34.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:33 smithi165 ceph-mon[36302]: Manager daemon x is now available 2024-03-28T00:26:34.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:33 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:26:34.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:33 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:26:34.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:33 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-28T00:26:34.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:33 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-28T00:26:34.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:33 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-28T00:26:34.137 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:33 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-28T00:26:34.138 DEBUG:teuthology.orchestra.run.smithi119:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-03-28T00:26:34.138 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:33 smithi119 ceph-mon[31518]: mgrmap e21: x(active, since 10m), standbys: y, z 2024-03-28T00:26:34.138 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:33 smithi119 ceph-mon[31518]: Active manager daemon x restarted 2024-03-28T00:26:34.139 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:33 smithi119 ceph-mon[31518]: Activating manager daemon x 2024-03-28T00:26:34.139 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:33 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-28T00:26:34.139 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:33 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-28T00:26:34.139 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:33 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-28T00:26:34.139 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:33 smithi119 ceph-mon[31518]: osdmap e88: 12 total, 12 up, 12 in 2024-03-28T00:26:34.139 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:33 smithi119 ceph-mon[31518]: mgrmap e22: x(active, starting, since 0.0281657s), standbys: y, z 2024-03-28T00:26:34.139 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:33 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-28T00:26:34.139 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:33 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-28T00:26:34.139 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:33 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-28T00:26:34.139 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:33 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-28T00:26:34.139 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:33 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-28T00:26:34.139 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:33 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-28T00:26:34.139 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:33 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-28T00:26:34.140 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:33 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-28T00:26:34.140 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:33 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-28T00:26:34.140 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:33 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-28T00:26:34.140 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:33 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-28T00:26:34.140 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:33 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-28T00:26:34.140 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:33 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-28T00:26:34.140 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:33 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-28T00:26:34.140 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:33 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-28T00:26:34.140 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:33 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-28T00:26:34.140 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:33 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-28T00:26:34.140 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:33 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-28T00:26:34.140 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:33 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-28T00:26:34.140 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:33 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-28T00:26:34.141 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:33 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-28T00:26:34.141 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:33 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-28T00:26:34.141 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:33 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-28T00:26:34.141 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:33 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-28T00:26:34.141 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:33 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-28T00:26:34.141 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:33 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-28T00:26:34.141 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:33 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-28T00:26:34.141 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:33 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-28T00:26:34.141 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:33 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-28T00:26:34.141 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:33 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-28T00:26:34.141 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:33 smithi119 ceph-mon[31518]: Manager daemon x is now available 2024-03-28T00:26:34.141 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:33 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:26:34.141 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:33 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:26:34.141 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:33 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-28T00:26:34.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:33 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-28T00:26:34.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:33 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-28T00:26:34.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:33 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-28T00:26:34.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:33 smithi132 ceph-mon[35941]: mgrmap e21: x(active, since 10m), standbys: y, z 2024-03-28T00:26:34.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:33 smithi132 ceph-mon[35941]: Active manager daemon x restarted 2024-03-28T00:26:34.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:33 smithi132 ceph-mon[35941]: Activating manager daemon x 2024-03-28T00:26:34.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:33 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-28T00:26:34.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:33 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-28T00:26:34.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:33 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-28T00:26:34.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:33 smithi132 ceph-mon[35941]: osdmap e88: 12 total, 12 up, 12 in 2024-03-28T00:26:34.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:33 smithi132 ceph-mon[35941]: mgrmap e22: x(active, starting, since 0.0281657s), standbys: y, z 2024-03-28T00:26:34.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:33 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-28T00:26:34.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:33 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-28T00:26:34.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:33 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-28T00:26:34.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:33 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-28T00:26:34.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:33 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-28T00:26:34.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:33 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-28T00:26:34.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:33 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-28T00:26:34.205 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:33 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-28T00:26:34.205 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:33 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-28T00:26:34.205 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:33 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-28T00:26:34.205 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:33 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-28T00:26:34.205 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:33 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-28T00:26:34.205 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:33 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-28T00:26:34.205 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:33 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-28T00:26:34.205 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:33 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-28T00:26:34.205 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:33 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-28T00:26:34.205 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:33 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-28T00:26:34.205 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:33 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-28T00:26:34.206 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:33 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-28T00:26:34.206 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:33 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-28T00:26:34.206 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:33 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-28T00:26:34.206 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:33 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-28T00:26:34.206 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:33 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-28T00:26:34.206 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:33 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-28T00:26:34.206 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:33 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-28T00:26:34.206 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:33 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-28T00:26:34.206 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:33 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-28T00:26:34.206 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:33 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-28T00:26:34.206 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:33 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-28T00:26:34.207 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:33 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-28T00:26:34.207 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:33 smithi132 ceph-mon[35941]: Manager daemon x is now available 2024-03-28T00:26:34.207 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:33 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:26:34.207 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:33 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:26:34.207 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:33 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-28T00:26:34.207 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:33 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-28T00:26:34.207 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:33 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-28T00:26:34.207 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:33 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-28T00:26:34.324 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:34.317+0000 7f5be3400640 1 -- 172.21.15.119:0/2794108412 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5bdc103900 msgr2=0x7f5bdc103d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:26:34.324 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:34.317+0000 7f5be3400640 1 --2- 172.21.15.119:0/2794108412 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5bdc103900 0x7f5bdc103d20 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f5bd0009900 tx=0x7f5bd002bc30 comp rx=0 tx=0).stop 2024-03-28T00:26:34.324 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:34.317+0000 7f5be3400640 1 -- 172.21.15.119:0/2794108412 shutdown_connections 2024-03-28T00:26:34.324 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:34.317+0000 7f5be3400640 1 --2- 172.21.15.119:0/2794108412 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5bdc1054b0 0x7f5bdc10bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:34.324 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:34.317+0000 7f5be3400640 1 --2- 172.21.15.119:0/2794108412 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5bdc104af0 0x7f5bdc104f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:34.324 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:34.317+0000 7f5be3400640 1 --2- 172.21.15.119:0/2794108412 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5bdc103900 0x7f5bdc103d20 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:34.324 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:34.317+0000 7f5be3400640 1 -- 172.21.15.119:0/2794108412 >> 172.21.15.119:0/2794108412 conn(0x7f5bdc0fee90 msgr2=0x7f5bdc1012b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:26:34.324 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:34.317+0000 7f5be3400640 1 -- 172.21.15.119:0/2794108412 shutdown_connections 2024-03-28T00:26:34.324 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:34.317+0000 7f5be3400640 1 -- 172.21.15.119:0/2794108412 wait complete. 2024-03-28T00:26:34.326 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:34.318+0000 7f5be3400640 1 Processor -- start 2024-03-28T00:26:34.326 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:34.318+0000 7f5be3400640 1 -- start start 2024-03-28T00:26:34.326 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:34.318+0000 7f5be3400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5bdc103900 0x7f5bdc199ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:34.326 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:34.318+0000 7f5be3400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5bdc104af0 0x7f5bdc19a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:34.326 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:34.318+0000 7f5be3400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5bdc1054b0 0x7f5bdc1a1150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:34.326 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:34.318+0000 7f5be3400640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bdc10f5c0 con 0x7f5bdc1054b0 2024-03-28T00:26:34.326 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:34.318+0000 7f5be3400640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bdc10f440 con 0x7f5bdc104af0 2024-03-28T00:26:34.327 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:34.318+0000 7f5be3400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bdc10f740 con 0x7f5bdc103900 2024-03-28T00:26:34.327 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:34.319+0000 7f5be0a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5bdc103900 0x7f5bdc199ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:34.327 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:34.319+0000 7f5be0a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5bdc103900 0x7f5bdc199ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.119:59942/0 (socket says 172.21.15.119:59942) 2024-03-28T00:26:34.327 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:34.319+0000 7f5be0a00640 1 -- 172.21.15.119:0/1257253113 learned_addr learned my addr 172.21.15.119:0/1257253113 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:26:34.327 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:34.319+0000 7f5bdbe00640 1 --2- 172.21.15.119:0/1257253113 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5bdc104af0 0x7f5bdc19a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:34.327 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:34.319+0000 7f5be1400640 1 --2- 172.21.15.119:0/1257253113 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5bdc1054b0 0x7f5bdc1a1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:34.327 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:34.319+0000 7f5be0a00640 1 -- 172.21.15.119:0/1257253113 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5bdc104af0 msgr2=0x7f5bdc19a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:26:34.327 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:34.319+0000 7f5be0a00640 1 --2- 172.21.15.119:0/1257253113 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5bdc104af0 0x7f5bdc19a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:34.327 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:34.319+0000 7f5be0a00640 1 -- 172.21.15.119:0/1257253113 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5bdc1054b0 msgr2=0x7f5bdc1a1150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:26:34.327 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:34.319+0000 7f5be0a00640 1 --2- 172.21.15.119:0/1257253113 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5bdc1054b0 0x7f5bdc1a1150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:34.327 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:34.319+0000 7f5be0a00640 1 -- 172.21.15.119:0/1257253113 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bdc1a17c0 con 0x7f5bdc103900 2024-03-28T00:26:34.327 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:34.319+0000 7f5be1400640 1 --2- 172.21.15.119:0/1257253113 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5bdc1054b0 0x7f5bdc1a1150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T00:26:34.327 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:34.319+0000 7f5be0a00640 1 --2- 172.21.15.119:0/1257253113 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5bdc103900 0x7f5bdc199ba0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f5bd0000c00 tx=0x7f5bd002c650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:26:34.327 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:34.319+0000 7f5bd9600640 1 -- 172.21.15.119:0/1257253113 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bd0043020 con 0x7f5bdc103900 2024-03-28T00:26:34.327 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:34.320+0000 7f5be3400640 1 -- 172.21.15.119:0/1257253113 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bdc1a1a50 con 0x7f5bdc103900 2024-03-28T00:26:34.328 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:34.320+0000 7f5bd9600640 1 -- 172.21.15.119:0/1257253113 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5bd00054e0 con 0x7f5bdc103900 2024-03-28T00:26:34.328 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:34.320+0000 7f5bd9600640 1 -- 172.21.15.119:0/1257253113 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bd0003c40 con 0x7f5bdc103900 2024-03-28T00:26:34.328 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:34.320+0000 7f5be3400640 1 -- 172.21.15.119:0/1257253113 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bdc1a1fa0 con 0x7f5bdc103900 2024-03-28T00:26:34.328 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:34.321+0000 7f5be3400640 1 -- 172.21.15.119:0/1257253113 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bdc10abc0 con 0x7f5bdc103900 2024-03-28T00:26:34.329 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:34.323+0000 7f5bd9600640 1 -- 172.21.15.119:0/1257253113 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 23) v1 ==== 137102+0+0 (secure 0 0 0) 0x7f5bd0046020 con 0x7f5bdc103900 2024-03-28T00:26:34.330 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:34.324+0000 7f5bd9600640 1 --2- 172.21.15.119:0/1257253113 >> [v2:172.21.15.119:6800/4273482239,v1:172.21.15.119:6801/4273482239] conn(0x7f5bc40a26b0 0x7f5bc40a4b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:34.330 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:34.324+0000 7f5bd9600640 1 -- 172.21.15.119:0/1257253113 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8411+0+0 (secure 0 0 0) 0x7f5bd00f07a0 con 0x7f5bdc103900 2024-03-28T00:26:34.330 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:34.324+0000 7f5bdbe00640 1 --2- 172.21.15.119:0/1257253113 >> [v2:172.21.15.119:6800/4273482239,v1:172.21.15.119:6801/4273482239] conn(0x7f5bc40a26b0 0x7f5bc40a4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:34.331 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:34.325+0000 7f5bdbe00640 1 --2- 172.21.15.119:0/1257253113 >> [v2:172.21.15.119:6800/4273482239,v1:172.21.15.119:6801/4273482239] conn(0x7f5bc40a26b0 0x7f5bc40a4b70 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f5bdc19b0b0 tx=0x7f5bc8005f90 comp rx=0 tx=0).ready entity=mgr.25277 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:26:34.338 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:34.332+0000 7f5bd9600640 1 -- 172.21.15.119:0/1257253113 <== mon.1 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f5bd004d020 con 0x7f5bdc103900 2024-03-28T00:26:34.562 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:34.555+0000 7f5be3400640 1 -- 172.21.15.119:0/1257253113 --> [v2:172.21.15.119:6800/4273482239,v1:172.21.15.119:6801/4273482239] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) v1 -- 0x7f5bdc061b80 con 0x7f5bc40a26b0 2024-03-28T00:26:34.662 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:34.656+0000 7f5bd9600640 1 -- 172.21.15.119:0/1257253113 <== mgr.25277 v2:172.21.15.119:6800/4273482239 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+26 (secure 0 0 0) 0x7f5bdc061b80 con 0x7f5bc40a26b0 2024-03-28T00:26:34.663 INFO:teuthology.orchestra.run.smithi119.stdout:Retention added to path / 2024-03-28T00:26:34.666 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:34.659+0000 7f5be3400640 1 -- 172.21.15.119:0/1257253113 >> [v2:172.21.15.119:6800/4273482239,v1:172.21.15.119:6801/4273482239] conn(0x7f5bc40a26b0 msgr2=0x7f5bc40a4b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:26:34.666 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:34.659+0000 7f5be3400640 1 --2- 172.21.15.119:0/1257253113 >> [v2:172.21.15.119:6800/4273482239,v1:172.21.15.119:6801/4273482239] conn(0x7f5bc40a26b0 0x7f5bc40a4b70 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f5bdc19b0b0 tx=0x7f5bc8005f90 comp rx=0 tx=0).stop 2024-03-28T00:26:34.666 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:34.659+0000 7f5be3400640 1 -- 172.21.15.119:0/1257253113 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5bdc103900 msgr2=0x7f5bdc199ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:26:34.666 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:34.659+0000 7f5be3400640 1 --2- 172.21.15.119:0/1257253113 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5bdc103900 0x7f5bdc199ba0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f5bd0000c00 tx=0x7f5bd002c650 comp rx=0 tx=0).stop 2024-03-28T00:26:34.666 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:34.659+0000 7f5be3400640 1 -- 172.21.15.119:0/1257253113 shutdown_connections 2024-03-28T00:26:34.666 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:34.659+0000 7f5be3400640 1 --2- 172.21.15.119:0/1257253113 >> [v2:172.21.15.119:6800/4273482239,v1:172.21.15.119:6801/4273482239] conn(0x7f5bc40a26b0 0x7f5bc40a4b70 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:34.666 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:34.659+0000 7f5be3400640 1 --2- 172.21.15.119:0/1257253113 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5bdc1054b0 0x7f5bdc1a1150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:34.667 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:34.659+0000 7f5be3400640 1 --2- 172.21.15.119:0/1257253113 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5bdc104af0 0x7f5bdc19a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:34.667 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:34.659+0000 7f5be3400640 1 --2- 172.21.15.119:0/1257253113 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5bdc103900 0x7f5bdc199ba0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:34.667 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:34.659+0000 7f5be3400640 1 -- 172.21.15.119:0/1257253113 >> 172.21.15.119:0/1257253113 conn(0x7f5bdc0fee90 msgr2=0x7f5bdc107d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:26:34.667 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:34.659+0000 7f5be3400640 1 -- 172.21.15.119:0/1257253113 shutdown_connections 2024-03-28T00:26:34.667 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:34.659+0000 7f5be3400640 1 -- 172.21.15.119:0/1257253113 wait complete. 2024-03-28T00:26:34.683 DEBUG:teuthology.orchestra.run.smithi119:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-03-28T00:26:34.863 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:34.856+0000 7fe8abe00640 1 -- 172.21.15.119:0/3932407705 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe8ac102700 msgr2=0x7fe8ac102b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:26:34.863 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:34.856+0000 7fe8abe00640 1 --2- 172.21.15.119:0/3932407705 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe8ac102700 0x7fe8ac102b80 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fe898009900 tx=0x7fe89802bc30 comp rx=0 tx=0).stop 2024-03-28T00:26:34.863 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:34.856+0000 7fe8abe00640 1 -- 172.21.15.119:0/3932407705 shutdown_connections 2024-03-28T00:26:34.864 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:34.856+0000 7fe8abe00640 1 --2- 172.21.15.119:0/3932407705 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe8ac1030c0 0x7fe8ac109950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:34.864 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:34.857+0000 7fe8abe00640 1 --2- 172.21.15.119:0/3932407705 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe8ac102700 0x7fe8ac102b80 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:34.864 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:34.857+0000 7fe8abe00640 1 --2- 172.21.15.119:0/3932407705 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe8ac101510 0x7fe8ac101930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:34.864 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:34.857+0000 7fe8abe00640 1 -- 172.21.15.119:0/3932407705 >> 172.21.15.119:0/3932407705 conn(0x7fe8ac0fca80 msgr2=0x7fe8ac0feec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:26:34.864 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:34.857+0000 7fe8abe00640 1 -- 172.21.15.119:0/3932407705 shutdown_connections 2024-03-28T00:26:34.864 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:34.857+0000 7fe8abe00640 1 -- 172.21.15.119:0/3932407705 wait complete. 2024-03-28T00:26:34.866 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:34.857+0000 7fe8abe00640 1 Processor -- start 2024-03-28T00:26:34.866 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:34.858+0000 7fe8abe00640 1 -- start start 2024-03-28T00:26:34.866 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:34.858+0000 7fe8abe00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe8ac101510 0x7fe8ac075450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:34.866 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:34.858+0000 7fe8abe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe8ac102700 0x7fe8ac073a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:34.866 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:34.858+0000 7fe8abe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe8ac1030c0 0x7fe8ac073fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:34.866 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:34.858+0000 7fe8abe00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8ac10d1f0 con 0x7fe8ac102700 2024-03-28T00:26:34.866 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:34.858+0000 7fe8abe00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8ac10d070 con 0x7fe8ac101510 2024-03-28T00:26:34.866 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:34.858+0000 7fe8abe00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8ac10d370 con 0x7fe8ac1030c0 2024-03-28T00:26:34.866 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:34.858+0000 7fe8aaa00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe8ac101510 0x7fe8ac075450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:34.866 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:34.858+0000 7fe8ab400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe8ac1030c0 0x7fe8ac073fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:34.866 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:34.858+0000 7fe8aa000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe8ac102700 0x7fe8ac073a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:34.866 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:34.858+0000 7fe8aaa00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe8ac101510 0x7fe8ac075450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.119:47546/0 (socket says 172.21.15.119:47546) 2024-03-28T00:26:34.866 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:34.858+0000 7fe8aaa00640 1 -- 172.21.15.119:0/3542314910 learned_addr learned my addr 172.21.15.119:0/3542314910 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:26:34.867 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:34.859+0000 7fe8ab400640 1 -- 172.21.15.119:0/3542314910 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe8ac101510 msgr2=0x7fe8ac075450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:26:34.867 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:34.859+0000 7fe8ab400640 1 --2- 172.21.15.119:0/3542314910 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe8ac101510 0x7fe8ac075450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:34.867 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:34.859+0000 7fe8ab400640 1 -- 172.21.15.119:0/3542314910 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe8ac102700 msgr2=0x7fe8ac073a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:26:34.867 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:34.859+0000 7fe8ab400640 1 --2- 172.21.15.119:0/3542314910 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe8ac102700 0x7fe8ac073a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:34.867 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:34.859+0000 7fe8ab400640 1 -- 172.21.15.119:0/3542314910 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8ac074860 con 0x7fe8ac1030c0 2024-03-28T00:26:34.867 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:34.859+0000 7fe8aa000640 1 --2- 172.21.15.119:0/3542314910 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe8ac102700 0x7fe8ac073a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T00:26:34.867 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:34.859+0000 7fe8ab400640 1 --2- 172.21.15.119:0/3542314910 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe8ac1030c0 0x7fe8ac073fd0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fe8a4009fd0 tx=0x7fe8a400ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:26:34.867 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:34.859+0000 7fe8b0a00640 1 -- 172.21.15.119:0/3542314910 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a400ce20 con 0x7fe8ac1030c0 2024-03-28T00:26:34.867 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:34.859+0000 7fe8abe00640 1 -- 172.21.15.119:0/3542314910 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8ac1a1900 con 0x7fe8ac1030c0 2024-03-28T00:26:34.867 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:34.859+0000 7fe8b0a00640 1 -- 172.21.15.119:0/3542314910 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe8a4005020 con 0x7fe8ac1030c0 2024-03-28T00:26:34.867 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:34.859+0000 7fe8b0a00640 1 -- 172.21.15.119:0/3542314910 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a4010720 con 0x7fe8ac1030c0 2024-03-28T00:26:34.867 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:34.859+0000 7fe8abe00640 1 -- 172.21.15.119:0/3542314910 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8ac1a1e70 con 0x7fe8ac1030c0 2024-03-28T00:26:34.868 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:34.860+0000 7fe8abe00640 1 -- 172.21.15.119:0/3542314910 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8ac1087d0 con 0x7fe8ac1030c0 2024-03-28T00:26:34.868 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:34.862+0000 7fe8b0a00640 1 -- 172.21.15.119:0/3542314910 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 23) v1 ==== 137102+0+0 (secure 0 0 0) 0x7fe8a4005540 con 0x7fe8ac1030c0 2024-03-28T00:26:34.870 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:34.863+0000 7fe8b0a00640 1 --2- 172.21.15.119:0/3542314910 >> [v2:172.21.15.119:6800/4273482239,v1:172.21.15.119:6801/4273482239] conn(0x7fe8900a25e0 0x7fe8900a4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:34.870 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:34.864+0000 7fe8b0a00640 1 -- 172.21.15.119:0/3542314910 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8411+0+0 (secure 0 0 0) 0x7fe8a40ced80 con 0x7fe8ac1030c0 2024-03-28T00:26:34.870 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:34.864+0000 7fe8aaa00640 1 --2- 172.21.15.119:0/3542314910 >> [v2:172.21.15.119:6800/4273482239,v1:172.21.15.119:6801/4273482239] conn(0x7fe8900a25e0 0x7fe8900a4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:34.871 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:34.864+0000 7fe8aaa00640 1 --2- 172.21.15.119:0/3542314910 >> [v2:172.21.15.119:6800/4273482239,v1:172.21.15.119:6801/4273482239] conn(0x7fe8900a25e0 0x7fe8900a4aa0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe89c0058c0 tx=0x7fe89c005f90 comp rx=0 tx=0).ready entity=mgr.25277 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:26:34.877 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:34.871+0000 7fe8b0a00640 1 -- 172.21.15.119:0/3542314910 <== mon.1 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fe8a409c020 con 0x7fe8ac1030c0 2024-03-28T00:26:35.127 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:35.120+0000 7fe8abe00640 1 -- 172.21.15.119:0/3542314910 --> [v2:172.21.15.119:6800/4273482239,v1:172.21.15.119:6801/4273482239] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) v1 -- 0x7fe8ac061ae0 con 0x7fe8900a25e0 2024-03-28T00:26:35.140 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:35.133+0000 7fe8b0a00640 1 -- 172.21.15.119:0/3542314910 <== mgr.25277 v2:172.21.15.119:6800/4273482239 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+298 (secure 0 0 0) 0x7fe8ac061ae0 con 0x7fe8900a25e0 2024-03-28T00:26:35.140 INFO:teuthology.orchestra.run.smithi119.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-03-28T00:00:00", "created": "2024-03-28T00:26:34", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-03-28T00:26:35.142 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:35.136+0000 7fe8abe00640 1 -- 172.21.15.119:0/3542314910 >> [v2:172.21.15.119:6800/4273482239,v1:172.21.15.119:6801/4273482239] conn(0x7fe8900a25e0 msgr2=0x7fe8900a4aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:26:35.142 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:35.136+0000 7fe8abe00640 1 --2- 172.21.15.119:0/3542314910 >> [v2:172.21.15.119:6800/4273482239,v1:172.21.15.119:6801/4273482239] conn(0x7fe8900a25e0 0x7fe8900a4aa0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe89c0058c0 tx=0x7fe89c005f90 comp rx=0 tx=0).stop 2024-03-28T00:26:35.142 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:35.136+0000 7fe8abe00640 1 -- 172.21.15.119:0/3542314910 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe8ac1030c0 msgr2=0x7fe8ac073fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:26:35.142 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:35.136+0000 7fe8abe00640 1 --2- 172.21.15.119:0/3542314910 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe8ac1030c0 0x7fe8ac073fd0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fe8a4009fd0 tx=0x7fe8a400ef10 comp rx=0 tx=0).stop 2024-03-28T00:26:35.143 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:35.136+0000 7fe8abe00640 1 -- 172.21.15.119:0/3542314910 shutdown_connections 2024-03-28T00:26:35.143 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:35.136+0000 7fe8abe00640 1 --2- 172.21.15.119:0/3542314910 >> [v2:172.21.15.119:6800/4273482239,v1:172.21.15.119:6801/4273482239] conn(0x7fe8900a25e0 0x7fe8900a4aa0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:35.143 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:35.136+0000 7fe8abe00640 1 --2- 172.21.15.119:0/3542314910 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe8ac1030c0 0x7fe8ac073fd0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:35.143 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:35.136+0000 7fe8abe00640 1 --2- 172.21.15.119:0/3542314910 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe8ac102700 0x7fe8ac073a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:35.143 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:35.136+0000 7fe8abe00640 1 --2- 172.21.15.119:0/3542314910 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe8ac101510 0x7fe8ac075450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:35.143 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:35.136+0000 7fe8abe00640 1 -- 172.21.15.119:0/3542314910 >> 172.21.15.119:0/3542314910 conn(0x7fe8ac0fca80 msgr2=0x7fe8ac1059a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:26:35.143 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:35.136+0000 7fe8abe00640 1 -- 172.21.15.119:0/3542314910 shutdown_connections 2024-03-28T00:26:35.143 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:35.136+0000 7fe8abe00640 1 -- 172.21.15.119:0/3542314910 wait complete. 2024-03-28T00:26:35.163 DEBUG:teuthology.orchestra.run.smithi119:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-03-28T00:26:35.353 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:35.344+0000 7f94a8c00640 1 -- 172.21.15.119:0/2712990336 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f94a4102200 msgr2=0x7f94a41045f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:26:35.353 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:35.344+0000 7f94a8c00640 1 --2- 172.21.15.119:0/2712990336 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f94a4102200 0x7f94a41045f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f9494009900 tx=0x7f949402bc50 comp rx=0 tx=0).stop 2024-03-28T00:26:35.353 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:35.344+0000 7f94a8c00640 1 -- 172.21.15.119:0/2712990336 shutdown_connections 2024-03-28T00:26:35.353 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:35.344+0000 7f94a8c00640 1 --2- 172.21.15.119:0/2712990336 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f94a4107460 0x7f94a4109850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:35.354 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:35.344+0000 7f94a8c00640 1 --2- 172.21.15.119:0/2712990336 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f94a4104b30 0x7f94a4106f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:35.354 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:35.344+0000 7f94a8c00640 1 --2- 172.21.15.119:0/2712990336 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f94a4102200 0x7f94a41045f0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:35.354 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:35.344+0000 7f94a8c00640 1 -- 172.21.15.119:0/2712990336 >> 172.21.15.119:0/2712990336 conn(0x7f94a40fbb30 msgr2=0x7f94a40fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:26:35.354 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:35.344+0000 7f94a8c00640 1 -- 172.21.15.119:0/2712990336 shutdown_connections 2024-03-28T00:26:35.354 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:35.344+0000 7f94a8c00640 1 -- 172.21.15.119:0/2712990336 wait complete. 2024-03-28T00:26:35.354 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:35.345+0000 7f94a8c00640 1 Processor -- start 2024-03-28T00:26:35.354 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:35.345+0000 7f94a8c00640 1 -- start start 2024-03-28T00:26:35.355 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:35.345+0000 7f94a8c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f94a4102200 0x7f94a4199b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:35.355 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:35.345+0000 7f94a8c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f94a4104b30 0x7f94a419a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:35.355 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:35.345+0000 7f94a8c00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f94a4107460 0x7f94a41a1100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:35.355 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:35.345+0000 7f94a8c00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94a410d3c0 con 0x7f94a4102200 2024-03-28T00:26:35.355 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:35.345+0000 7f94a8c00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94a410d240 con 0x7f94a4107460 2024-03-28T00:26:35.355 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:35.345+0000 7f94a8c00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94a410d540 con 0x7f94a4104b30 2024-03-28T00:26:35.355 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:35.345+0000 7f94a1400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f94a4104b30 0x7f94a419a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:35.355 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:35.346+0000 7f94a1400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f94a4104b30 0x7f94a419a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.119:59990/0 (socket says 172.21.15.119:59990) 2024-03-28T00:26:35.355 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:35.346+0000 7f94a1400640 1 -- 172.21.15.119:0/1360716312 learned_addr learned my addr 172.21.15.119:0/1360716312 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T00:26:35.355 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:35.346+0000 7f94a2800640 1 --2- 172.21.15.119:0/1360716312 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f94a4107460 0x7f94a41a1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:35.355 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:35.346+0000 7f94a1e00640 1 --2- 172.21.15.119:0/1360716312 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f94a4102200 0x7f94a4199b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:35.355 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:35.346+0000 7f94a1400640 1 -- 172.21.15.119:0/1360716312 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f94a4107460 msgr2=0x7f94a41a1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:26:35.355 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:35.346+0000 7f94a1400640 1 --2- 172.21.15.119:0/1360716312 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f94a4107460 0x7f94a41a1100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:35.356 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:35.346+0000 7f94a1400640 1 -- 172.21.15.119:0/1360716312 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f94a4102200 msgr2=0x7f94a4199b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:26:35.356 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:35.346+0000 7f94a1400640 1 --2- 172.21.15.119:0/1360716312 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f94a4102200 0x7f94a4199b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:35.356 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:35.346+0000 7f94a1400640 1 -- 172.21.15.119:0/1360716312 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94a41a1770 con 0x7f94a4104b30 2024-03-28T00:26:35.356 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:35.346+0000 7f94a2800640 1 --2- 172.21.15.119:0/1360716312 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f94a4107460 0x7f94a41a1100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T00:26:35.356 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:35.346+0000 7f94a1400640 1 --2- 172.21.15.119:0/1360716312 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f94a4104b30 0x7f94a419a090 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f949000e9a0 tx=0x7f949000ee60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:26:35.356 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:35.347+0000 7f949aa00640 1 -- 172.21.15.119:0/1360716312 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f949000cd80 con 0x7f94a4104b30 2024-03-28T00:26:35.360 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:35.347+0000 7f94a8c00640 1 -- 172.21.15.119:0/1360716312 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94a41a1a60 con 0x7f94a4104b30 2024-03-28T00:26:35.360 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:35.347+0000 7f949aa00640 1 -- 172.21.15.119:0/1360716312 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f94900035f0 con 0x7f94a4104b30 2024-03-28T00:26:35.360 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:35.347+0000 7f949aa00640 1 -- 172.21.15.119:0/1360716312 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9490010670 con 0x7f94a4104b30 2024-03-28T00:26:35.360 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:35.347+0000 7f94a8c00640 1 -- 172.21.15.119:0/1360716312 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94a41a1fb0 con 0x7f94a4104b30 2024-03-28T00:26:35.361 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:35.348+0000 7f94a8c00640 1 -- 172.21.15.119:0/1360716312 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94a4108f50 con 0x7f94a4104b30 2024-03-28T00:26:35.361 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:35.351+0000 7f949aa00640 1 -- 172.21.15.119:0/1360716312 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 24) v1 ==== 137411+0+0 (secure 0 0 0) 0x7f94900037a0 con 0x7f94a4104b30 2024-03-28T00:26:35.361 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:35.352+0000 7f949aa00640 1 --2- 172.21.15.119:0/1360716312 >> [v2:172.21.15.119:6800/4273482239,v1:172.21.15.119:6801/4273482239] conn(0x7f94880a2a10 0x7f94880a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T00:26:35.361 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:35.352+0000 7f949aa00640 1 -- 172.21.15.119:0/1360716312 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8411+0+0 (secure 0 0 0) 0x7f94900cf110 con 0x7f94a4104b30 2024-03-28T00:26:35.361 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:35.352+0000 7f94a1e00640 1 --2- 172.21.15.119:0/1360716312 >> [v2:172.21.15.119:6800/4273482239,v1:172.21.15.119:6801/4273482239] conn(0x7f94880a2a10 0x7f94880a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T00:26:35.361 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:35.353+0000 7f94a1e00640 1 --2- 172.21.15.119:0/1360716312 >> [v2:172.21.15.119:6800/4273482239,v1:172.21.15.119:6801/4273482239] conn(0x7f94880a2a10 0x7f94880a4ed0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f949402c260 tx=0x7f9494000bc0 comp rx=0 tx=0).ready entity=mgr.25277 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T00:26:35.364 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:35.358+0000 7f949aa00640 1 -- 172.21.15.119:0/1360716312 <== mon.1 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f949009bf70 con 0x7f94a4104b30 2024-03-28T00:26:35.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:34 smithi165 ceph-mon[36302]: [28/Mar/2024:00:26:33] ENGINE Bus STARTING 2024-03-28T00:26:35.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:34 smithi165 ceph-mon[36302]: [28/Mar/2024:00:26:33] ENGINE Serving on https://172.21.15.119:7150 2024-03-28T00:26:35.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:34 smithi165 ceph-mon[36302]: [28/Mar/2024:00:26:33] ENGINE Serving on http://172.21.15.119:8765 2024-03-28T00:26:35.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:34 smithi165 ceph-mon[36302]: [28/Mar/2024:00:26:33] ENGINE Bus STARTED 2024-03-28T00:26:35.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:34 smithi165 ceph-mon[36302]: mgrmap e23: x(active, since 1.03541s), standbys: y, z 2024-03-28T00:26:35.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:34 smithi165 ceph-mon[36302]: from='client.25309 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T00:26:35.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:34 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:26:35.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:34 smithi165 ceph-mon[36302]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T00:26:35.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:34 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:26:35.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:34 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:26:35.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:34 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:26:35.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:34 smithi119 ceph-mon[31518]: [28/Mar/2024:00:26:33] ENGINE Bus STARTING 2024-03-28T00:26:35.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:34 smithi119 ceph-mon[31518]: [28/Mar/2024:00:26:33] ENGINE Serving on https://172.21.15.119:7150 2024-03-28T00:26:35.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:34 smithi119 ceph-mon[31518]: [28/Mar/2024:00:26:33] ENGINE Serving on http://172.21.15.119:8765 2024-03-28T00:26:35.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:34 smithi119 ceph-mon[31518]: [28/Mar/2024:00:26:33] ENGINE Bus STARTED 2024-03-28T00:26:35.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:34 smithi119 ceph-mon[31518]: mgrmap e23: x(active, since 1.03541s), standbys: y, z 2024-03-28T00:26:35.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:34 smithi119 ceph-mon[31518]: from='client.25309 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T00:26:35.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:34 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:26:35.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:34 smithi119 ceph-mon[31518]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T00:26:35.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:34 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:26:35.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:34 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:26:35.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:34 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:26:35.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:34 smithi132 ceph-mon[35941]: [28/Mar/2024:00:26:33] ENGINE Bus STARTING 2024-03-28T00:26:35.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:34 smithi132 ceph-mon[35941]: [28/Mar/2024:00:26:33] ENGINE Serving on https://172.21.15.119:7150 2024-03-28T00:26:35.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:34 smithi132 ceph-mon[35941]: [28/Mar/2024:00:26:33] ENGINE Serving on http://172.21.15.119:8765 2024-03-28T00:26:35.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:34 smithi132 ceph-mon[35941]: [28/Mar/2024:00:26:33] ENGINE Bus STARTED 2024-03-28T00:26:35.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:34 smithi132 ceph-mon[35941]: mgrmap e23: x(active, since 1.03541s), standbys: y, z 2024-03-28T00:26:35.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:34 smithi132 ceph-mon[35941]: from='client.25309 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T00:26:35.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:34 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:26:35.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:34 smithi132 ceph-mon[35941]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T00:26:35.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:34 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:26:35.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:34 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:26:35.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:34 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:26:35.595 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:35.589+0000 7f94a8c00640 1 -- 172.21.15.119:0/1360716312 --> [v2:172.21.15.119:6800/4273482239,v1:172.21.15.119:6801/4273482239] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f94a4061ae0 con 0x7f94880a2a10 2024-03-28T00:26:35.605 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:35.599+0000 7f949aa00640 1 -- 172.21.15.119:0/1360716312 <== mgr.25277 v2:172.21.15.119:6800/4273482239 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+10 (secure 0 0 0) 0x7f94a4061ae0 con 0x7f94880a2a10 2024-03-28T00:26:35.605 INFO:teuthology.orchestra.run.smithi119.stdout:/ 1m 6m3h 2024-03-28T00:26:35.607 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:35.601+0000 7f94a8c00640 1 -- 172.21.15.119:0/1360716312 >> [v2:172.21.15.119:6800/4273482239,v1:172.21.15.119:6801/4273482239] conn(0x7f94880a2a10 msgr2=0x7f94880a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:26:35.608 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:35.601+0000 7f94a8c00640 1 --2- 172.21.15.119:0/1360716312 >> [v2:172.21.15.119:6800/4273482239,v1:172.21.15.119:6801/4273482239] conn(0x7f94880a2a10 0x7f94880a4ed0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f949402c260 tx=0x7f9494000bc0 comp rx=0 tx=0).stop 2024-03-28T00:26:35.608 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:35.601+0000 7f94a8c00640 1 -- 172.21.15.119:0/1360716312 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f94a4104b30 msgr2=0x7f94a419a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T00:26:35.608 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:35.601+0000 7f94a8c00640 1 --2- 172.21.15.119:0/1360716312 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f94a4104b30 0x7f94a419a090 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f949000e9a0 tx=0x7f949000ee60 comp rx=0 tx=0).stop 2024-03-28T00:26:35.608 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:35.601+0000 7f94a8c00640 1 -- 172.21.15.119:0/1360716312 shutdown_connections 2024-03-28T00:26:35.608 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:35.601+0000 7f94a8c00640 1 --2- 172.21.15.119:0/1360716312 >> [v2:172.21.15.119:6800/4273482239,v1:172.21.15.119:6801/4273482239] conn(0x7f94880a2a10 0x7f94880a4ed0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:35.608 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:35.601+0000 7f94a8c00640 1 --2- 172.21.15.119:0/1360716312 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f94a4107460 0x7f94a41a1100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:35.608 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:35.601+0000 7f94a8c00640 1 --2- 172.21.15.119:0/1360716312 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f94a4104b30 0x7f94a419a090 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:35.608 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:35.601+0000 7f94a8c00640 1 --2- 172.21.15.119:0/1360716312 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f94a4102200 0x7f94a4199b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T00:26:35.608 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:35.601+0000 7f94a8c00640 1 -- 172.21.15.119:0/1360716312 >> 172.21.15.119:0/1360716312 conn(0x7f94a40fbb30 msgr2=0x7f94a4101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T00:26:35.608 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:35.601+0000 7f94a8c00640 1 -- 172.21.15.119:0/1360716312 shutdown_connections 2024-03-28T00:26:35.608 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T00:26:35.601+0000 7f94a8c00640 1 -- 172.21.15.119:0/1360716312 wait complete. 2024-03-28T00:26:35.625 DEBUG:teuthology.orchestra.run.smithi119:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-03-28T00:26:35.708 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-03-28T00:26:35.719 INFO:teuthology.run_tasks:Running task workunit... 2024-03-28T00:26:35.741 INFO:tasks.workunit:Pulling workunits from ref 3be9a5c9bc793e11e2800a8c0c696e8b46742033 2024-03-28T00:26:35.741 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-03-28T00:26:35.741 INFO:tasks.workunit:timeout=3h 2024-03-28T00:26:35.741 INFO:tasks.workunit:cleanup=True 2024-03-28T00:26:35.742 DEBUG:teuthology.orchestra.run.smithi119:> stat -- /home/ubuntu/cephtest/mnt.0 2024-03-28T00:26:35.791 INFO:teuthology.orchestra.run.smithi119.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-03-28T00:26:35.791 INFO:teuthology.orchestra.run.smithi119.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-03-28T00:26:35.792 INFO:teuthology.orchestra.run.smithi119.stdout:Device: 76h/118d Inode: 1099511628283 Links: 2 2024-03-28T00:26:35.792 INFO:teuthology.orchestra.run.smithi119.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-28T00:26:35.792 INFO:teuthology.orchestra.run.smithi119.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-03-28T00:26:35.792 INFO:teuthology.orchestra.run.smithi119.stdout:Access: 2024-03-28 00:24:05.071867619 +0000 2024-03-28T00:26:35.792 INFO:teuthology.orchestra.run.smithi119.stdout:Modify: 2024-03-28 00:24:05.071867619 +0000 2024-03-28T00:26:35.792 INFO:teuthology.orchestra.run.smithi119.stdout:Change: 2024-03-28 00:26:21.343181949 +0000 2024-03-28T00:26:35.792 INFO:teuthology.orchestra.run.smithi119.stdout: Birth: 2024-03-28 00:24:05.071867619 +0000 2024-03-28T00:26:35.792 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-03-28T00:26:35.792 DEBUG:teuthology.orchestra.run.smithi119:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-03-28T00:26:35.889 DEBUG:teuthology.orchestra.run.smithi119:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 3be9a5c9bc793e11e2800a8c0c696e8b46742033 2024-03-28T00:26:35.936 INFO:tasks.workunit.client.0.smithi119.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-03-28T00:26:36.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:36 smithi165 ceph-mon[36302]: from='client.25351 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T00:26:36.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:36 smithi165 ceph-mon[36302]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T00:26:36.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:36 smithi165 ceph-mon[36302]: from='client.25357 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T00:26:36.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:36 smithi165 ceph-mon[36302]: mgrmap e24: x(active, since 2s), standbys: y, z 2024-03-28T00:26:36.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:36 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:26:36.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:36 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:26:36.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:36 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:26:36.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:36 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:26:36.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:36 smithi119 ceph-mon[31518]: from='client.25351 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T00:26:36.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:36 smithi119 ceph-mon[31518]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T00:26:36.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:36 smithi119 ceph-mon[31518]: from='client.25357 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T00:26:36.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:36 smithi119 ceph-mon[31518]: mgrmap e24: x(active, since 2s), standbys: y, z 2024-03-28T00:26:36.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:36 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:26:36.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:36 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:26:36.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:36 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:26:36.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:36 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:26:36.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:36 smithi132 ceph-mon[35941]: from='client.25351 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T00:26:36.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:36 smithi132 ceph-mon[35941]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T00:26:36.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:36 smithi132 ceph-mon[35941]: from='client.25357 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T00:26:36.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:36 smithi132 ceph-mon[35941]: mgrmap e24: x(active, since 2s), standbys: y, z 2024-03-28T00:26:36.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:36 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:26:36.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:36 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:26:36.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:36 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:26:36.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:36 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:26:37.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:37 smithi165 ceph-mon[36302]: from='client.25369 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T00:26:37.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:37 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:26:37.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:37 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:26:37.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:37 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:26:37.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:37 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:26:37.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:37 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:26:37.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:37 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:26:37.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:37 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:26:37.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:37 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:26:37.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:37 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:26:37.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:37 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:26:37.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:37 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:26:37.384 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:37 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:26:37.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:37 smithi119 ceph-mon[31518]: from='client.25369 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T00:26:37.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:37 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:26:37.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:37 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:26:37.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:37 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:26:37.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:37 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:26:37.407 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:37 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:26:37.407 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:37 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:26:37.407 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:37 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:26:37.407 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:37 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:26:37.407 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:37 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:26:37.407 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:37 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:26:37.407 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:37 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:26:37.407 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:37 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:26:37.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:37 smithi132 ceph-mon[35941]: from='client.25369 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T00:26:37.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:37 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:26:37.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:37 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:26:37.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:37 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:26:37.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:37 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:26:37.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:37 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:26:37.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:37 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:26:37.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:37 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:26:37.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:37 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:26:37.454 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:37 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:26:37.454 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:37 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:26:37.454 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:37 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:26:37.454 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:37 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:26:38.407 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:38 smithi119 ceph-mon[31518]: Adjusting osd_memory_target on smithi165 to 174.1M 2024-03-28T00:26:38.407 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:38 smithi119 ceph-mon[31518]: Unable to set osd_memory_target on smithi165 to 182635622: error parsing value: Value '182635622' is below minimum 939524096 2024-03-28T00:26:38.407 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:38 smithi119 ceph-mon[31518]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T00:26:38.407 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:38 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:26:38.407 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:38 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:26:38.407 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:38 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:26:38.407 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:38 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:26:38.408 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:38 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:26:38.408 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:38 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:26:38.408 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:38 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:26:38.408 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:38 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:26:38.408 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:38 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:26:38.408 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:38 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:26:38.408 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:38 smithi119 ceph-mon[31518]: Adjusting osd_memory_target on smithi132 to 174.1M 2024-03-28T00:26:38.408 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:38 smithi119 ceph-mon[31518]: Unable to set osd_memory_target on smithi132 to 182640640: error parsing value: Value '182640640' is below minimum 939524096 2024-03-28T00:26:38.408 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:38 smithi119 ceph-mon[31518]: mgrmap e25: x(active, since 4s), standbys: y, z 2024-03-28T00:26:38.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:38 smithi132 ceph-mon[35941]: Adjusting osd_memory_target on smithi165 to 174.1M 2024-03-28T00:26:38.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:38 smithi132 ceph-mon[35941]: Unable to set osd_memory_target on smithi165 to 182635622: error parsing value: Value '182635622' is below minimum 939524096 2024-03-28T00:26:38.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:38 smithi132 ceph-mon[35941]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T00:26:38.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:38 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:26:38.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:38 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:26:38.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:38 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:26:38.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:38 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:26:38.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:38 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:26:38.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:38 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:26:38.454 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:38 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:26:38.454 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:38 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:26:38.454 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:38 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:26:38.454 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:38 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:26:38.454 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:38 smithi132 ceph-mon[35941]: Adjusting osd_memory_target on smithi132 to 174.1M 2024-03-28T00:26:38.454 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:38 smithi132 ceph-mon[35941]: Unable to set osd_memory_target on smithi132 to 182640640: error parsing value: Value '182640640' is below minimum 939524096 2024-03-28T00:26:38.454 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:38 smithi132 ceph-mon[35941]: mgrmap e25: x(active, since 4s), standbys: y, z 2024-03-28T00:26:38.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:38 smithi165 ceph-mon[36302]: Adjusting osd_memory_target on smithi165 to 174.1M 2024-03-28T00:26:38.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:38 smithi165 ceph-mon[36302]: Unable to set osd_memory_target on smithi165 to 182635622: error parsing value: Value '182635622' is below minimum 939524096 2024-03-28T00:26:38.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:38 smithi165 ceph-mon[36302]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T00:26:38.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:38 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:26:38.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:38 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:26:38.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:38 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:26:38.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:38 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:26:38.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:38 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:26:38.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:38 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:26:38.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:38 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:26:38.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:38 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:26:38.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:38 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:26:38.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:38 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:26:38.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:38 smithi165 ceph-mon[36302]: Adjusting osd_memory_target on smithi132 to 174.1M 2024-03-28T00:26:38.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:38 smithi165 ceph-mon[36302]: Unable to set osd_memory_target on smithi132 to 182640640: error parsing value: Value '182640640' is below minimum 939524096 2024-03-28T00:26:38.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:38 smithi165 ceph-mon[36302]: mgrmap e25: x(active, since 4s), standbys: y, z 2024-03-28T00:26:40.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:39 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:26:40.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:39 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:26:40.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:39 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:26:40.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:39 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:26:40.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:39 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:26:40.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:39 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:26:40.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:39 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:26:40.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:39 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:26:40.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:39 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:26:40.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:39 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:26:40.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:39 smithi165 ceph-mon[36302]: Adjusting osd_memory_target on smithi119 to 176.7M 2024-03-28T00:26:40.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:39 smithi165 ceph-mon[36302]: Unable to set osd_memory_target on smithi119 to 185372364: error parsing value: Value '185372364' is below minimum 939524096 2024-03-28T00:26:40.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:39 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:26:40.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:39 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:26:40.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:39 smithi165 ceph-mon[36302]: Updating smithi119:/etc/ceph/ceph.conf 2024-03-28T00:26:40.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:39 smithi165 ceph-mon[36302]: Updating smithi132:/etc/ceph/ceph.conf 2024-03-28T00:26:40.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:39 smithi165 ceph-mon[36302]: Updating smithi165:/etc/ceph/ceph.conf 2024-03-28T00:26:40.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:39 smithi165 ceph-mon[36302]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T00:26:40.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:39 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:26:40.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:39 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:26:40.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:39 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:26:40.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:39 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:26:40.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:39 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:26:40.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:39 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:26:40.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:39 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:26:40.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:39 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:26:40.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:39 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:26:40.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:39 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:26:40.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:39 smithi119 ceph-mon[31518]: Adjusting osd_memory_target on smithi119 to 176.7M 2024-03-28T00:26:40.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:39 smithi119 ceph-mon[31518]: Unable to set osd_memory_target on smithi119 to 185372364: error parsing value: Value '185372364' is below minimum 939524096 2024-03-28T00:26:40.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:39 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:26:40.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:39 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:26:40.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:39 smithi119 ceph-mon[31518]: Updating smithi119:/etc/ceph/ceph.conf 2024-03-28T00:26:40.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:39 smithi119 ceph-mon[31518]: Updating smithi132:/etc/ceph/ceph.conf 2024-03-28T00:26:40.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:39 smithi119 ceph-mon[31518]: Updating smithi165:/etc/ceph/ceph.conf 2024-03-28T00:26:40.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:39 smithi119 ceph-mon[31518]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T00:26:40.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:39 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:26:40.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:39 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:26:40.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:39 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:26:40.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:39 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:26:40.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:39 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:26:40.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:39 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:26:40.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:39 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:26:40.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:39 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:26:40.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:39 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:26:40.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:39 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:26:40.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:39 smithi132 ceph-mon[35941]: Adjusting osd_memory_target on smithi119 to 176.7M 2024-03-28T00:26:40.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:39 smithi132 ceph-mon[35941]: Unable to set osd_memory_target on smithi119 to 185372364: error parsing value: Value '185372364' is below minimum 939524096 2024-03-28T00:26:40.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:39 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:26:40.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:39 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:26:40.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:39 smithi132 ceph-mon[35941]: Updating smithi119:/etc/ceph/ceph.conf 2024-03-28T00:26:40.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:39 smithi132 ceph-mon[35941]: Updating smithi132:/etc/ceph/ceph.conf 2024-03-28T00:26:40.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:39 smithi132 ceph-mon[35941]: Updating smithi165:/etc/ceph/ceph.conf 2024-03-28T00:26:40.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:39 smithi132 ceph-mon[35941]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T00:26:41.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:41 smithi165 ceph-mon[36302]: Updating smithi165:/var/lib/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-28T00:26:41.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:41 smithi165 ceph-mon[36302]: Updating smithi132:/var/lib/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-28T00:26:41.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:41 smithi165 ceph-mon[36302]: Updating smithi132:/etc/ceph/ceph.client.admin.keyring 2024-03-28T00:26:41.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:41 smithi165 ceph-mon[36302]: Updating smithi165:/etc/ceph/ceph.client.admin.keyring 2024-03-28T00:26:41.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:41 smithi165 ceph-mon[36302]: Updating smithi119:/var/lib/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-28T00:26:41.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:41 smithi165 ceph-mon[36302]: Updating smithi132:/var/lib/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/config/ceph.client.admin.keyring 2024-03-28T00:26:41.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:41 smithi165 ceph-mon[36302]: Updating smithi165:/var/lib/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/config/ceph.client.admin.keyring 2024-03-28T00:26:41.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:41 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:26:41.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:41 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:26:41.384 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:41 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:26:41.384 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:41 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:26:41.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:41 smithi119 ceph-mon[31518]: Updating smithi165:/var/lib/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-28T00:26:41.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:41 smithi119 ceph-mon[31518]: Updating smithi132:/var/lib/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-28T00:26:41.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:41 smithi119 ceph-mon[31518]: Updating smithi132:/etc/ceph/ceph.client.admin.keyring 2024-03-28T00:26:41.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:41 smithi119 ceph-mon[31518]: Updating smithi165:/etc/ceph/ceph.client.admin.keyring 2024-03-28T00:26:41.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:41 smithi119 ceph-mon[31518]: Updating smithi119:/var/lib/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-28T00:26:41.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:41 smithi119 ceph-mon[31518]: Updating smithi132:/var/lib/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/config/ceph.client.admin.keyring 2024-03-28T00:26:41.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:41 smithi119 ceph-mon[31518]: Updating smithi165:/var/lib/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/config/ceph.client.admin.keyring 2024-03-28T00:26:41.407 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:41 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:26:41.407 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:41 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:26:41.407 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:41 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:26:41.407 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:41 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:26:41.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:41 smithi132 ceph-mon[35941]: Updating smithi165:/var/lib/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-28T00:26:41.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:41 smithi132 ceph-mon[35941]: Updating smithi132:/var/lib/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-28T00:26:41.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:41 smithi132 ceph-mon[35941]: Updating smithi132:/etc/ceph/ceph.client.admin.keyring 2024-03-28T00:26:41.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:41 smithi132 ceph-mon[35941]: Updating smithi165:/etc/ceph/ceph.client.admin.keyring 2024-03-28T00:26:41.454 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:41 smithi132 ceph-mon[35941]: Updating smithi119:/var/lib/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-28T00:26:41.454 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:41 smithi132 ceph-mon[35941]: Updating smithi132:/var/lib/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/config/ceph.client.admin.keyring 2024-03-28T00:26:41.454 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:41 smithi132 ceph-mon[35941]: Updating smithi165:/var/lib/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/config/ceph.client.admin.keyring 2024-03-28T00:26:41.454 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:41 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:26:41.454 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:41 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:26:41.454 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:41 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:26:41.454 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:41 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:26:42.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:42 smithi165 ceph-mon[36302]: Updating smithi119:/etc/ceph/ceph.client.admin.keyring 2024-03-28T00:26:42.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:42 smithi165 ceph-mon[36302]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 132 KiB/s wr, 80 op/s 2024-03-28T00:26:42.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:42 smithi165 ceph-mon[36302]: Updating smithi119:/var/lib/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/config/ceph.client.admin.keyring 2024-03-28T00:26:42.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:42 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:26:42.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:42 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:26:42.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:42 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:26:42.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:42 smithi119 ceph-mon[31518]: Updating smithi119:/etc/ceph/ceph.client.admin.keyring 2024-03-28T00:26:42.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:42 smithi119 ceph-mon[31518]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 132 KiB/s wr, 80 op/s 2024-03-28T00:26:42.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:42 smithi119 ceph-mon[31518]: Updating smithi119:/var/lib/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/config/ceph.client.admin.keyring 2024-03-28T00:26:42.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:42 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:26:42.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:42 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:26:42.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:42 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:26:42.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:42 smithi132 ceph-mon[35941]: Updating smithi119:/etc/ceph/ceph.client.admin.keyring 2024-03-28T00:26:42.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:42 smithi132 ceph-mon[35941]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 132 KiB/s wr, 80 op/s 2024-03-28T00:26:42.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:42 smithi132 ceph-mon[35941]: Updating smithi119:/var/lib/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/config/ceph.client.admin.keyring 2024-03-28T00:26:42.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:42 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:26:42.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:42 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:26:42.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:42 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:26:43.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:43 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:26:43.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:43 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:26:43.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:43 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:26:43.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:43 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:26:43.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:43 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:26:43.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:43 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:26:43.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:43 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:26:43.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:43 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:26:43.452 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:43 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:26:43.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:43 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:26:43.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:43 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:26:43.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:43 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:26:44.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:44 smithi165 ceph-mon[36302]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 103 KiB/s wr, 62 op/s 2024-03-28T00:26:44.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:44 smithi119 ceph-mon[31518]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 103 KiB/s wr, 62 op/s 2024-03-28T00:26:44.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:44 smithi132 ceph-mon[35941]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 103 KiB/s wr, 62 op/s 2024-03-28T00:26:46.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:46 smithi165 ceph-mon[36302]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 84 KiB/s wr, 55 op/s 2024-03-28T00:26:46.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:46 smithi119 ceph-mon[31518]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 84 KiB/s wr, 55 op/s 2024-03-28T00:26:46.452 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:46 smithi132 ceph-mon[35941]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 84 KiB/s wr, 55 op/s 2024-03-28T00:26:48.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:48 smithi119 ceph-mon[31518]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 77 KiB/s wr, 50 op/s 2024-03-28T00:26:48.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:48 smithi132 ceph-mon[35941]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 77 KiB/s wr, 50 op/s 2024-03-28T00:26:48.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:48 smithi165 ceph-mon[36302]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 77 KiB/s wr, 50 op/s 2024-03-28T00:26:50.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:50 smithi119 ceph-mon[31518]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 77 KiB/s wr, 50 op/s 2024-03-28T00:26:50.434 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:50 smithi165 ceph-mon[36302]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 77 KiB/s wr, 50 op/s 2024-03-28T00:26:50.452 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:50 smithi132 ceph-mon[35941]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 77 KiB/s wr, 50 op/s 2024-03-28T00:26:52.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:52 smithi119 ceph-mon[31518]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 77 KiB/s wr, 54 op/s 2024-03-28T00:26:52.452 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:52 smithi132 ceph-mon[35941]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 77 KiB/s wr, 54 op/s 2024-03-28T00:26:52.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:52 smithi165 ceph-mon[36302]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 77 KiB/s wr, 54 op/s 2024-03-28T00:26:54.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:54 smithi119 ceph-mon[31518]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T00:26:54.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:54 smithi132 ceph-mon[35941]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T00:26:54.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:54 smithi165 ceph-mon[36302]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T00:26:56.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:56 smithi119 ceph-mon[31518]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-28T00:26:56.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:56 smithi132 ceph-mon[35941]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-28T00:26:56.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:56 smithi165 ceph-mon[36302]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-28T00:26:58.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:26:58 smithi132 ceph-mon[35941]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T00:26:58.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:26:58 smithi165 ceph-mon[36302]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T00:26:58.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:26:58 smithi119 ceph-mon[31518]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T00:27:00.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:27:00 smithi132 ceph-mon[35941]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T00:27:00.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:27:00 smithi165 ceph-mon[36302]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T00:27:00.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:27:00 smithi119 ceph-mon[31518]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T00:27:01.156 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 00:27:00 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-x[31752]: 2024-03-28T00:27:00.681+0000 7fabca200700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-28T00:27:01.156 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 00:27:00 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-x[31752]: 2024-03-28T00:27:00.681+0000 7fabca200700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-28T00:27:01.156 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 00:27:00 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-x[31752]: 2024-03-28T00:27:00.681+0000 7fabca200700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-28T00:27:01.156 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 00:27:00 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-x[31752]: 2024-03-28T00:27:00.682+0000 7fabca200700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-28T00:27:01.156 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 00:27:00 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-x[31752]: 2024-03-28T00:27:00.682+0000 7fabca200700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-28T00:27:02.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:27:02 smithi132 ceph-mon[35941]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-28T00:27:02.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:27:02 smithi132 ceph-mon[35941]: mgrmap e26: x(active, since 28s), standbys: y, z 2024-03-28T00:27:02.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:27:02 smithi165 ceph-mon[36302]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-28T00:27:02.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:27:02 smithi165 ceph-mon[36302]: mgrmap e26: x(active, since 28s), standbys: y, z 2024-03-28T00:27:02.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:27:02 smithi119 ceph-mon[31518]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-28T00:27:02.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:27:02 smithi119 ceph-mon[31518]: mgrmap e26: x(active, since 28s), standbys: y, z 2024-03-28T00:27:04.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:27:04 smithi132 ceph-mon[35941]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T00:27:04.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:27:04 smithi165 ceph-mon[36302]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T00:27:04.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:27:04 smithi119 ceph-mon[31518]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T00:27:06.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:27:06 smithi132 ceph-mon[35941]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-03-28T00:27:06.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:27:06 smithi165 ceph-mon[36302]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-03-28T00:27:06.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:27:06 smithi119 ceph-mon[31518]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-03-28T00:27:08.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:27:08 smithi132 ceph-mon[35941]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-03-28T00:27:08.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:27:08 smithi165 ceph-mon[36302]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-03-28T00:27:08.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:27:08 smithi119 ceph-mon[31518]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-03-28T00:27:10.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:27:10 smithi132 ceph-mon[35941]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-03-28T00:27:10.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:27:10 smithi165 ceph-mon[36302]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-03-28T00:27:10.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:27:10 smithi119 ceph-mon[31518]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-03-28T00:27:12.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:27:12 smithi132 ceph-mon[35941]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 27 op/s 2024-03-28T00:27:12.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:27:12 smithi165 ceph-mon[36302]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 27 op/s 2024-03-28T00:27:12.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:27:12 smithi119 ceph-mon[31518]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 27 op/s 2024-03-28T00:27:14.458 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:27:14 smithi119 ceph-mon[31518]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-03-28T00:27:14.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:27:14 smithi165 ceph-mon[36302]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-03-28T00:27:14.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:27:14 smithi132 ceph-mon[35941]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-03-28T00:27:15.494 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:27:15 smithi119 ceph-mon[31518]: pgmap v24: 97 pgs: 97 active+clean; 1.4 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 24 KiB/s wr, 28 op/s 2024-03-28T00:27:15.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:27:15 smithi165 ceph-mon[36302]: pgmap v24: 97 pgs: 97 active+clean; 1.4 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 24 KiB/s wr, 28 op/s 2024-03-28T00:27:15.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:27:15 smithi132 ceph-mon[35941]: pgmap v24: 97 pgs: 97 active+clean; 1.4 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 24 KiB/s wr, 28 op/s 2024-03-28T00:27:18.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:27:18 smithi165 ceph-mon[36302]: pgmap v25: 97 pgs: 97 active+clean; 1.4 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.6 KiB/s wr, 8 op/s 2024-03-28T00:27:18.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:27:18 smithi119 ceph-mon[31518]: pgmap v25: 97 pgs: 97 active+clean; 1.4 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.6 KiB/s wr, 8 op/s 2024-03-28T00:27:18.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:27:18 smithi132 ceph-mon[35941]: pgmap v25: 97 pgs: 97 active+clean; 1.4 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.6 KiB/s wr, 8 op/s 2024-03-28T00:27:20.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:27:20 smithi165 ceph-mon[36302]: pgmap v26: 97 pgs: 97 active+clean; 1.4 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.6 KiB/s wr, 9 op/s 2024-03-28T00:27:20.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:27:20 smithi119 ceph-mon[31518]: pgmap v26: 97 pgs: 97 active+clean; 1.4 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.6 KiB/s wr, 9 op/s 2024-03-28T00:27:20.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:27:20 smithi132 ceph-mon[35941]: pgmap v26: 97 pgs: 97 active+clean; 1.4 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.6 KiB/s wr, 9 op/s 2024-03-28T00:27:22.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:27:22 smithi165 ceph-mon[36302]: pgmap v27: 97 pgs: 97 active+clean; 1.4 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 1.6 KiB/s wr, 12 op/s 2024-03-28T00:27:22.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:27:22 smithi119 ceph-mon[31518]: pgmap v27: 97 pgs: 97 active+clean; 1.4 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 1.6 KiB/s wr, 12 op/s 2024-03-28T00:27:22.452 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:27:22 smithi132 ceph-mon[35941]: pgmap v27: 97 pgs: 97 active+clean; 1.4 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 1.6 KiB/s wr, 12 op/s 2024-03-28T00:27:24.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:27:24 smithi165 ceph-mon[36302]: pgmap v28: 97 pgs: 97 active+clean; 1.4 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 938 B/s wr, 8 op/s 2024-03-28T00:27:24.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:27:24 smithi119 ceph-mon[31518]: pgmap v28: 97 pgs: 97 active+clean; 1.4 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 938 B/s wr, 8 op/s 2024-03-28T00:27:24.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:27:24 smithi132 ceph-mon[35941]: pgmap v28: 97 pgs: 97 active+clean; 1.4 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 938 B/s wr, 8 op/s 2024-03-28T00:27:26.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:27:26 smithi165 ceph-mon[36302]: pgmap v29: 97 pgs: 97 active+clean; 1.4 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 938 B/s wr, 12 op/s 2024-03-28T00:27:26.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:27:26 smithi119 ceph-mon[31518]: pgmap v29: 97 pgs: 97 active+clean; 1.4 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 938 B/s wr, 12 op/s 2024-03-28T00:27:26.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:27:26 smithi132 ceph-mon[35941]: pgmap v29: 97 pgs: 97 active+clean; 1.4 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 938 B/s wr, 12 op/s 2024-03-28T00:27:28.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:27:28 smithi165 ceph-mon[36302]: pgmap v30: 97 pgs: 97 active+clean; 1.4 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T00:27:28.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:27:28 smithi119 ceph-mon[31518]: pgmap v30: 97 pgs: 97 active+clean; 1.4 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T00:27:28.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:27:28 smithi132 ceph-mon[35941]: pgmap v30: 97 pgs: 97 active+clean; 1.4 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T00:27:30.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:27:30 smithi165 ceph-mon[36302]: pgmap v31: 97 pgs: 97 active+clean; 1.4 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-28T00:27:30.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:27:30 smithi119 ceph-mon[31518]: pgmap v31: 97 pgs: 97 active+clean; 1.4 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-28T00:27:30.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:27:30 smithi132 ceph-mon[35941]: pgmap v31: 97 pgs: 97 active+clean; 1.4 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-28T00:27:32.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:27:32 smithi165 ceph-mon[36302]: pgmap v32: 97 pgs: 97 active+clean; 1.4 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-28T00:27:32.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:27:32 smithi119 ceph-mon[31518]: pgmap v32: 97 pgs: 97 active+clean; 1.4 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-28T00:27:32.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:27:32 smithi132 ceph-mon[35941]: pgmap v32: 97 pgs: 97 active+clean; 1.4 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-28T00:27:34.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:27:34 smithi165 ceph-mon[36302]: pgmap v33: 97 pgs: 97 active+clean; 1.4 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T00:27:34.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:27:34 smithi119 ceph-mon[31518]: pgmap v33: 97 pgs: 97 active+clean; 1.4 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T00:27:34.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:27:34 smithi132 ceph-mon[35941]: pgmap v33: 97 pgs: 97 active+clean; 1.4 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T00:27:36.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:27:36 smithi165 ceph-mon[36302]: pgmap v34: 97 pgs: 97 active+clean; 1.4 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-28T00:27:36.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:27:36 smithi119 ceph-mon[31518]: pgmap v34: 97 pgs: 97 active+clean; 1.4 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-28T00:27:36.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:27:36 smithi132 ceph-mon[35941]: pgmap v34: 97 pgs: 97 active+clean; 1.4 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-28T00:27:38.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:27:38 smithi165 ceph-mon[36302]: pgmap v35: 97 pgs: 97 active+clean; 1.4 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T00:27:38.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:27:38 smithi119 ceph-mon[31518]: pgmap v35: 97 pgs: 97 active+clean; 1.4 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T00:27:38.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:27:38 smithi132 ceph-mon[35941]: pgmap v35: 97 pgs: 97 active+clean; 1.4 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T00:27:40.385 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:27:40 smithi165 ceph-mon[36302]: pgmap v36: 97 pgs: 97 active+clean; 1.4 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-28T00:27:40.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:27:40 smithi119 ceph-mon[31518]: pgmap v36: 97 pgs: 97 active+clean; 1.4 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-28T00:27:40.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:27:40 smithi132 ceph-mon[35941]: pgmap v36: 97 pgs: 97 active+clean; 1.4 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-28T00:27:42.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:27:42 smithi165 ceph-mon[36302]: pgmap v37: 97 pgs: 97 active+clean; 1.4 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-28T00:27:42.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:27:42 smithi119 ceph-mon[31518]: pgmap v37: 97 pgs: 97 active+clean; 1.4 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-28T00:27:42.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:27:42 smithi132 ceph-mon[35941]: pgmap v37: 97 pgs: 97 active+clean; 1.4 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-28T00:27:43.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:27:43 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:27:43.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:27:43 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:27:43.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:27:43 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:27:43.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:27:43 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:27:43.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:27:43 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:27:43.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:27:43 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:27:43.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:27:43 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:27:43.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:27:43 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:27:43.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:27:43 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:27:44.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:27:44 smithi165 ceph-mon[36302]: pgmap v38: 97 pgs: 97 active+clean; 1.4 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T00:27:44.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:27:44 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:27:44.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:27:44 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:27:44.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:27:44 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:27:44.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:27:44 smithi119 ceph-mon[31518]: pgmap v38: 97 pgs: 97 active+clean; 1.4 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T00:27:44.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:27:44 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:27:44.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:27:44 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:27:44.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:27:44 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:27:44.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:27:44 smithi132 ceph-mon[35941]: pgmap v38: 97 pgs: 97 active+clean; 1.4 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T00:27:44.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:27:44 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:27:44.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:27:44 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:27:44.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:27:44 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:27:46.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:27:46 smithi165 ceph-mon[36302]: pgmap v39: 97 pgs: 97 active+clean; 1.4 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-28T00:27:46.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:27:46 smithi119 ceph-mon[31518]: pgmap v39: 97 pgs: 97 active+clean; 1.4 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-28T00:27:46.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:27:46 smithi132 ceph-mon[35941]: pgmap v39: 97 pgs: 97 active+clean; 1.4 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-28T00:27:48.385 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:27:48 smithi165 ceph-mon[36302]: pgmap v40: 97 pgs: 97 active+clean; 1.4 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T00:27:48.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:27:48 smithi119 ceph-mon[31518]: pgmap v40: 97 pgs: 97 active+clean; 1.4 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T00:27:48.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:27:48 smithi132 ceph-mon[35941]: pgmap v40: 97 pgs: 97 active+clean; 1.4 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T00:27:50.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:27:50 smithi119 ceph-mon[31518]: pgmap v41: 97 pgs: 97 active+clean; 1.4 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-28T00:27:50.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:27:50 smithi132 ceph-mon[35941]: pgmap v41: 97 pgs: 97 active+clean; 1.4 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-28T00:27:50.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:27:50 smithi165 ceph-mon[36302]: pgmap v41: 97 pgs: 97 active+clean; 1.4 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-28T00:27:52.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:27:52 smithi119 ceph-mon[31518]: pgmap v42: 97 pgs: 97 active+clean; 1.4 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-28T00:27:52.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:27:52 smithi132 ceph-mon[35941]: pgmap v42: 97 pgs: 97 active+clean; 1.4 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-28T00:27:52.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:27:52 smithi165 ceph-mon[36302]: pgmap v42: 97 pgs: 97 active+clean; 1.4 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-28T00:27:54.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:27:54 smithi119 ceph-mon[31518]: pgmap v43: 97 pgs: 97 active+clean; 1.4 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T00:27:54.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:27:54 smithi132 ceph-mon[35941]: pgmap v43: 97 pgs: 97 active+clean; 1.4 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T00:27:54.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:27:54 smithi165 ceph-mon[36302]: pgmap v43: 97 pgs: 97 active+clean; 1.4 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T00:27:56.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:27:56 smithi119 ceph-mon[31518]: pgmap v44: 97 pgs: 97 active+clean; 1.4 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-28T00:27:56.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:27:56 smithi132 ceph-mon[35941]: pgmap v44: 97 pgs: 97 active+clean; 1.4 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-28T00:27:56.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:27:56 smithi165 ceph-mon[36302]: pgmap v44: 97 pgs: 97 active+clean; 1.4 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-28T00:27:58.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:27:58 smithi119 ceph-mon[31518]: pgmap v45: 97 pgs: 97 active+clean; 1.4 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T00:27:58.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:27:58 smithi132 ceph-mon[35941]: pgmap v45: 97 pgs: 97 active+clean; 1.4 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T00:27:58.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:27:58 smithi165 ceph-mon[36302]: pgmap v45: 97 pgs: 97 active+clean; 1.4 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T00:27:59.803 INFO:tasks.workunit.client.0.smithi119.stderr:Updating files: 62% (7720/12271) Updating files: 63% (7731/12271) Updating files: 64% (7854/12271) Updating files: 65% (7977/12271) Updating files: 66% (8099/12271) Updating files: 67% (8222/12271) Updating files: 68% (8345/12271) Updating files: 69% (8467/12271) Updating files: 70% (8590/12271) Updating files: 71% (8713/12271) Updating files: 72% (8836/12271) Updating files: 73% (8958/12271) Updating files: 74% (9081/12271) Updating files: 75% (9204/12271) Updating files: 76% (9326/12271) Updating files: 77% (9449/12271) Updating files: 78% (9572/12271) Updating files: 79% (9695/12271) Updating files: 80% (9817/12271) Updating files: 81% (9940/12271) Updating files: 82% (10063/12271) Updating files: 83% (10185/12271) Updating files: 84% (10308/12271) Updating files: 85% (10431/12271) Updating files: 86% (10554/12271) Updating files: 87% (10676/12271) Updating files: 88% (10799/12271) Updating files: 89% (10922/12271) Updating files: 90% (11044/12271) Updating files: 91% (11167/12271) Updating files: 92% (11290/12271) Updating files: 93% (11413/12271) Updating files: 94% (11535/12271) Updating files: 95% (11658/12271) Updating files: 96% (11781/12271) Updating files: 97% (11903/12271) Updating files: 98% (12026/12271) Updating files: 99% (12149/12271) Updating files: 100% (12271/12271) Updating files: 100% (12271/12271), done. 2024-03-28T00:28:00.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:28:00 smithi119 ceph-mon[31518]: pgmap v46: 97 pgs: 97 active+clean; 1.4 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-28T00:28:00.454 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:28:00 smithi132 ceph-mon[35941]: pgmap v46: 97 pgs: 97 active+clean; 1.4 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-28T00:28:00.544 INFO:tasks.workunit.client.0.smithi119.stderr:Note: switching to '3be9a5c9bc793e11e2800a8c0c696e8b46742033'. 2024-03-28T00:28:00.544 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-03-28T00:28:00.544 INFO:tasks.workunit.client.0.smithi119.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-03-28T00:28:00.544 INFO:tasks.workunit.client.0.smithi119.stderr:changes and commit them, and you can discard any commits you make in this 2024-03-28T00:28:00.544 INFO:tasks.workunit.client.0.smithi119.stderr:state without impacting any branches by switching back to a branch. 2024-03-28T00:28:00.544 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-03-28T00:28:00.544 INFO:tasks.workunit.client.0.smithi119.stderr:If you want to create a new branch to retain commits you create, you may 2024-03-28T00:28:00.544 INFO:tasks.workunit.client.0.smithi119.stderr:do so (now or later) by using -c with the switch command. Example: 2024-03-28T00:28:00.544 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-03-28T00:28:00.544 INFO:tasks.workunit.client.0.smithi119.stderr: git switch -c 2024-03-28T00:28:00.544 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-03-28T00:28:00.544 INFO:tasks.workunit.client.0.smithi119.stderr:Or undo this operation with: 2024-03-28T00:28:00.544 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-03-28T00:28:00.545 INFO:tasks.workunit.client.0.smithi119.stderr: git switch - 2024-03-28T00:28:00.545 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-03-28T00:28:00.545 INFO:tasks.workunit.client.0.smithi119.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-03-28T00:28:00.545 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-03-28T00:28:00.545 INFO:tasks.workunit.client.0.smithi119.stderr:HEAD is now at 3be9a5c9bc7 dummy commit #1 2024-03-28T00:28:00.553 DEBUG:teuthology.orchestra.run.smithi119:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-03-28T00:28:00.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:28:00 smithi165 ceph-mon[36302]: pgmap v46: 97 pgs: 97 active+clean; 1.4 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-28T00:28:00.672 INFO:tasks.workunit.client.0.smithi119.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-03-28T00:28:00.680 INFO:tasks.workunit.client.0.smithi119.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-28T00:28:00.680 INFO:tasks.workunit.client.0.smithi119.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-03-28T00:28:01.617 INFO:tasks.workunit.client.0.smithi119.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-03-28T00:28:01.749 INFO:tasks.workunit.client.0.smithi119.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-03-28T00:28:01.834 INFO:tasks.workunit.client.0.smithi119.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-28T00:28:01.839 INFO:tasks.workunit.client.0.smithi119.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-28T00:28:01.839 INFO:tasks.workunit.client.0.smithi119.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-03-28T00:28:01.918 INFO:tasks.workunit.client.0.smithi119.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-28T00:28:01.928 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-03-28T00:28:01.928 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-03-28T00:28:01.976 INFO:tasks.workunit:Running workunits matching fs/misc on client.0... 2024-03-28T00:28:01.979 INFO:tasks.workunit:Running workunit fs/misc/acl.sh... 2024-03-28T00:28:01.979 DEBUG:teuthology.orchestra.run.smithi119:workunit test fs/misc/acl.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=3be9a5c9bc793e11e2800a8c0c696e8b46742033 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/acl.sh 2024-03-28T00:28:02.094 INFO:tasks.workunit.client.0.smithi119.stderr:+ set -e 2024-03-28T00:28:02.094 INFO:tasks.workunit.client.0.smithi119.stderr:+ mkdir -p testdir 2024-03-28T00:28:02.099 INFO:tasks.workunit.client.0.smithi119.stderr:+ cd testdir 2024-03-28T00:28:02.099 INFO:tasks.workunit.client.0.smithi119.stderr:+ set +e 2024-03-28T00:28:02.100 INFO:tasks.workunit.client.0.smithi119.stderr:+ setfacl -d -m u:nobody:rw . 2024-03-28T00:28:02.103 INFO:tasks.workunit.client.0.smithi119.stderr:+ test 0 '!=' 0 2024-03-28T00:28:02.103 INFO:tasks.workunit.client.0.smithi119.stderr:+ set -e 2024-03-28T00:28:02.103 INFO:tasks.workunit.client.0.smithi119.stderr:+ c=0 2024-03-28T00:28:02.103 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 0 -lt 100 ']' 2024-03-28T00:28:02.104 INFO:tasks.workunit.client.0.smithi119.stderr:++ expr 0 + 1 2024-03-28T00:28:02.107 INFO:tasks.workunit.client.0.smithi119.stderr:+ c=1 2024-03-28T00:28:02.107 INFO:tasks.workunit.client.0.smithi119.stderr:+ mkdir d1 2024-03-28T00:28:02.115 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:28:02.115 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:28:02.119 INFO:tasks.workunit.client.0.smithi119.stderr:+ c1=2 2024-03-28T00:28:02.119 INFO:tasks.workunit.client.0.smithi119.stderr:+ echo 3 2024-03-28T00:28:02.120 INFO:tasks.workunit.client.0.smithi119.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-03-28T00:28:02.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:28:02 smithi119 ceph-mon[31518]: pgmap v47: 97 pgs: 97 active+clean; 1.4 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-28T00:28:02.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:28:02 smithi132 ceph-mon[35941]: pgmap v47: 97 pgs: 97 active+clean; 1.4 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-28T00:28:02.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:28:02 smithi165 ceph-mon[36302]: pgmap v47: 97 pgs: 97 active+clean; 1.4 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-28T00:28:04.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:28:04 smithi132 ceph-mon[35941]: pgmap v48: 97 pgs: 97 active+clean; 1.4 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T00:28:04.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:28:04 smithi165 ceph-mon[36302]: pgmap v48: 97 pgs: 97 active+clean; 1.4 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T00:28:04.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:28:04 smithi119 ceph-mon[31518]: pgmap v48: 97 pgs: 97 active+clean; 1.4 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T00:28:05.060 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:28:05.060 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:28:05.315 INFO:tasks.workunit.client.0.smithi119.stderr:+ c2=2 2024-03-28T00:28:05.315 INFO:tasks.workunit.client.0.smithi119.stderr:+ rmdir d1 2024-03-28T00:28:05.335 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:28:05.335 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:28:05.336 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 1 -lt 100 ']' 2024-03-28T00:28:05.337 INFO:tasks.workunit.client.0.smithi119.stderr:++ expr 1 + 1 2024-03-28T00:28:05.374 INFO:tasks.workunit.client.0.smithi119.stderr:+ c=2 2024-03-28T00:28:05.374 INFO:tasks.workunit.client.0.smithi119.stderr:+ mkdir d1 2024-03-28T00:28:05.411 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:28:05.412 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:28:05.416 INFO:tasks.workunit.client.0.smithi119.stderr:+ c1=2 2024-03-28T00:28:05.417 INFO:tasks.workunit.client.0.smithi119.stderr:+ echo 3 2024-03-28T00:28:05.418 INFO:tasks.workunit.client.0.smithi119.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-03-28T00:28:06.018 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:28:06.018 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:28:06.190 INFO:tasks.workunit.client.0.smithi119.stderr:+ c2=2 2024-03-28T00:28:06.190 INFO:tasks.workunit.client.0.smithi119.stderr:+ rmdir d1 2024-03-28T00:28:06.211 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:28:06.211 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:28:06.211 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -lt 100 ']' 2024-03-28T00:28:06.212 INFO:tasks.workunit.client.0.smithi119.stderr:++ expr 2 + 1 2024-03-28T00:28:06.258 INFO:tasks.workunit.client.0.smithi119.stderr:+ c=3 2024-03-28T00:28:06.258 INFO:tasks.workunit.client.0.smithi119.stderr:+ mkdir d1 2024-03-28T00:28:06.286 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:28:06.286 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:28:06.291 INFO:tasks.workunit.client.0.smithi119.stderr:+ c1=2 2024-03-28T00:28:06.292 INFO:tasks.workunit.client.0.smithi119.stderr:+ echo 3 2024-03-28T00:28:06.292 INFO:tasks.workunit.client.0.smithi119.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-03-28T00:28:06.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:28:06 smithi132 ceph-mon[35941]: pgmap v49: 97 pgs: 97 active+clean; 1.4 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 25 KiB/s wr, 29 op/s 2024-03-28T00:28:06.571 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:28:06 smithi119 ceph-mon[31518]: pgmap v49: 97 pgs: 97 active+clean; 1.4 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 25 KiB/s wr, 29 op/s 2024-03-28T00:28:06.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:28:06 smithi165 ceph-mon[36302]: pgmap v49: 97 pgs: 97 active+clean; 1.4 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 25 KiB/s wr, 29 op/s 2024-03-28T00:28:06.853 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:28:06.853 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:28:06.990 INFO:tasks.workunit.client.0.smithi119.stderr:+ c2=2 2024-03-28T00:28:06.991 INFO:tasks.workunit.client.0.smithi119.stderr:+ rmdir d1 2024-03-28T00:28:07.010 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:28:07.010 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:28:07.010 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 3 -lt 100 ']' 2024-03-28T00:28:07.011 INFO:tasks.workunit.client.0.smithi119.stderr:++ expr 3 + 1 2024-03-28T00:28:07.058 INFO:tasks.workunit.client.0.smithi119.stderr:+ c=4 2024-03-28T00:28:07.058 INFO:tasks.workunit.client.0.smithi119.stderr:+ mkdir d1 2024-03-28T00:28:07.086 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:28:07.086 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:28:07.091 INFO:tasks.workunit.client.0.smithi119.stderr:+ c1=2 2024-03-28T00:28:07.092 INFO:tasks.workunit.client.0.smithi119.stderr:+ echo 3 2024-03-28T00:28:07.093 INFO:tasks.workunit.client.0.smithi119.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-03-28T00:28:07.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:28:07 smithi132 ceph-mon[35941]: mgrmap e27: x(active, since 93s), standbys: y, z 2024-03-28T00:28:07.458 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:28:07 smithi119 ceph-mon[31518]: mgrmap e27: x(active, since 93s), standbys: y, z 2024-03-28T00:28:07.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:28:07 smithi165 ceph-mon[36302]: mgrmap e27: x(active, since 93s), standbys: y, z 2024-03-28T00:28:07.729 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:28:07.729 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:28:07.890 INFO:tasks.workunit.client.0.smithi119.stderr:+ c2=2 2024-03-28T00:28:07.890 INFO:tasks.workunit.client.0.smithi119.stderr:+ rmdir d1 2024-03-28T00:28:07.900 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:28:07.900 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:28:07.901 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 4 -lt 100 ']' 2024-03-28T00:28:07.902 INFO:tasks.workunit.client.0.smithi119.stderr:++ expr 4 + 1 2024-03-28T00:28:07.932 INFO:tasks.workunit.client.0.smithi119.stderr:+ c=5 2024-03-28T00:28:07.932 INFO:tasks.workunit.client.0.smithi119.stderr:+ mkdir d1 2024-03-28T00:28:07.960 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:28:07.960 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:28:07.965 INFO:tasks.workunit.client.0.smithi119.stderr:+ c1=2 2024-03-28T00:28:07.966 INFO:tasks.workunit.client.0.smithi119.stderr:+ echo 3 2024-03-28T00:28:07.967 INFO:tasks.workunit.client.0.smithi119.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-03-28T00:28:08.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:28:08 smithi132 ceph-mon[35941]: pgmap v50: 97 pgs: 97 active+clean; 1.4 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 25 KiB/s wr, 25 op/s 2024-03-28T00:28:08.493 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:28:08 smithi119 ceph-mon[31518]: pgmap v50: 97 pgs: 97 active+clean; 1.4 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 25 KiB/s wr, 25 op/s 2024-03-28T00:28:08.602 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:28:08.603 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:28:08.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:28:08 smithi165 ceph-mon[36302]: pgmap v50: 97 pgs: 97 active+clean; 1.4 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 25 KiB/s wr, 25 op/s 2024-03-28T00:28:08.757 INFO:tasks.workunit.client.0.smithi119.stderr:+ c2=2 2024-03-28T00:28:08.757 INFO:tasks.workunit.client.0.smithi119.stderr:+ rmdir d1 2024-03-28T00:28:08.777 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:28:08.777 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:28:08.777 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 5 -lt 100 ']' 2024-03-28T00:28:08.779 INFO:tasks.workunit.client.0.smithi119.stderr:++ expr 5 + 1 2024-03-28T00:28:08.832 INFO:tasks.workunit.client.0.smithi119.stderr:+ c=6 2024-03-28T00:28:08.832 INFO:tasks.workunit.client.0.smithi119.stderr:+ mkdir d1 2024-03-28T00:28:08.860 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:28:08.860 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:28:08.864 INFO:tasks.workunit.client.0.smithi119.stderr:+ c1=2 2024-03-28T00:28:08.865 INFO:tasks.workunit.client.0.smithi119.stderr:+ echo 3 2024-03-28T00:28:08.866 INFO:tasks.workunit.client.0.smithi119.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-03-28T00:28:09.395 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:28:09.395 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:28:09.548 INFO:tasks.workunit.client.0.smithi119.stderr:+ c2=2 2024-03-28T00:28:09.548 INFO:tasks.workunit.client.0.smithi119.stderr:+ rmdir d1 2024-03-28T00:28:09.559 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:28:09.560 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:28:09.560 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 6 -lt 100 ']' 2024-03-28T00:28:09.561 INFO:tasks.workunit.client.0.smithi119.stderr:++ expr 6 + 1 2024-03-28T00:28:09.590 INFO:tasks.workunit.client.0.smithi119.stderr:+ c=7 2024-03-28T00:28:09.591 INFO:tasks.workunit.client.0.smithi119.stderr:+ mkdir d1 2024-03-28T00:28:09.618 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:28:09.618 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:28:09.624 INFO:tasks.workunit.client.0.smithi119.stderr:+ c1=2 2024-03-28T00:28:09.625 INFO:tasks.workunit.client.0.smithi119.stderr:+ echo 3 2024-03-28T00:28:09.625 INFO:tasks.workunit.client.0.smithi119.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-03-28T00:28:10.135 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:28:10.136 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:28:10.306 INFO:tasks.workunit.client.0.smithi119.stderr:+ c2=2 2024-03-28T00:28:10.306 INFO:tasks.workunit.client.0.smithi119.stderr:+ rmdir d1 2024-03-28T00:28:10.315 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:28:10.315 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:28:10.316 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 7 -lt 100 ']' 2024-03-28T00:28:10.317 INFO:tasks.workunit.client.0.smithi119.stderr:++ expr 7 + 1 2024-03-28T00:28:10.348 INFO:tasks.workunit.client.0.smithi119.stderr:+ c=8 2024-03-28T00:28:10.348 INFO:tasks.workunit.client.0.smithi119.stderr:+ mkdir d1 2024-03-28T00:28:10.376 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:28:10.377 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:28:10.381 INFO:tasks.workunit.client.0.smithi119.stderr:+ c1=2 2024-03-28T00:28:10.382 INFO:tasks.workunit.client.0.smithi119.stderr:+ echo 3 2024-03-28T00:28:10.383 INFO:tasks.workunit.client.0.smithi119.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-03-28T00:28:10.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:28:10 smithi165 ceph-mon[36302]: pgmap v51: 97 pgs: 97 active+clean; 1.4 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 31 KiB/s wr, 30 op/s 2024-03-28T00:28:10.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:28:10 smithi119 ceph-mon[31518]: pgmap v51: 97 pgs: 97 active+clean; 1.4 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 31 KiB/s wr, 30 op/s 2024-03-28T00:28:10.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:28:10 smithi132 ceph-mon[35941]: pgmap v51: 97 pgs: 97 active+clean; 1.4 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 31 KiB/s wr, 30 op/s 2024-03-28T00:28:10.936 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:28:10.936 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:28:11.089 INFO:tasks.workunit.client.0.smithi119.stderr:+ c2=2 2024-03-28T00:28:11.089 INFO:tasks.workunit.client.0.smithi119.stderr:+ rmdir d1 2024-03-28T00:28:11.101 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:28:11.101 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:28:11.101 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 8 -lt 100 ']' 2024-03-28T00:28:11.102 INFO:tasks.workunit.client.0.smithi119.stderr:++ expr 8 + 1 2024-03-28T00:28:11.123 INFO:tasks.workunit.client.0.smithi119.stderr:+ c=9 2024-03-28T00:28:11.123 INFO:tasks.workunit.client.0.smithi119.stderr:+ mkdir d1 2024-03-28T00:28:11.151 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:28:11.152 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:28:11.156 INFO:tasks.workunit.client.0.smithi119.stderr:+ c1=2 2024-03-28T00:28:11.157 INFO:tasks.workunit.client.0.smithi119.stderr:+ echo 3 2024-03-28T00:28:11.158 INFO:tasks.workunit.client.0.smithi119.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-03-28T00:28:11.719 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:28:11.719 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:28:11.922 INFO:tasks.workunit.client.0.smithi119.stderr:+ c2=2 2024-03-28T00:28:11.922 INFO:tasks.workunit.client.0.smithi119.stderr:+ rmdir d1 2024-03-28T00:28:11.933 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:28:11.934 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:28:11.934 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 9 -lt 100 ']' 2024-03-28T00:28:11.935 INFO:tasks.workunit.client.0.smithi119.stderr:++ expr 9 + 1 2024-03-28T00:28:11.956 INFO:tasks.workunit.client.0.smithi119.stderr:+ c=10 2024-03-28T00:28:11.956 INFO:tasks.workunit.client.0.smithi119.stderr:+ mkdir d1 2024-03-28T00:28:11.984 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:28:11.986 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:28:11.990 INFO:tasks.workunit.client.0.smithi119.stderr:+ c1=2 2024-03-28T00:28:11.991 INFO:tasks.workunit.client.0.smithi119.stderr:+ echo 3 2024-03-28T00:28:11.991 INFO:tasks.workunit.client.0.smithi119.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-03-28T00:28:12.466 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:28:12 smithi119 ceph-mon[31518]: pgmap v52: 97 pgs: 97 active+clean; 1.4 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 31 KiB/s wr, 29 op/s 2024-03-28T00:28:12.502 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:28:12.503 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:28:12.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:28:12 smithi165 ceph-mon[36302]: pgmap v52: 97 pgs: 97 active+clean; 1.4 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 31 KiB/s wr, 29 op/s 2024-03-28T00:28:12.658 INFO:tasks.workunit.client.0.smithi119.stderr:+ c2=2 2024-03-28T00:28:12.659 INFO:tasks.workunit.client.0.smithi119.stderr:+ rmdir d1 2024-03-28T00:28:12.674 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:28:12.674 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:28:12.675 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 10 -lt 100 ']' 2024-03-28T00:28:12.676 INFO:tasks.workunit.client.0.smithi119.stderr:++ expr 10 + 1 2024-03-28T00:28:12.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:28:12 smithi132 ceph-mon[35941]: pgmap v52: 97 pgs: 97 active+clean; 1.4 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 31 KiB/s wr, 29 op/s 2024-03-28T00:28:12.706 INFO:tasks.workunit.client.0.smithi119.stderr:+ c=11 2024-03-28T00:28:12.706 INFO:tasks.workunit.client.0.smithi119.stderr:+ mkdir d1 2024-03-28T00:28:12.734 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:28:12.735 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:28:12.740 INFO:tasks.workunit.client.0.smithi119.stderr:+ c1=2 2024-03-28T00:28:12.741 INFO:tasks.workunit.client.0.smithi119.stderr:+ echo 3 2024-03-28T00:28:12.742 INFO:tasks.workunit.client.0.smithi119.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-03-28T00:28:13.268 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:28:13.269 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:28:13.498 INFO:tasks.workunit.client.0.smithi119.stderr:+ c2=2 2024-03-28T00:28:13.498 INFO:tasks.workunit.client.0.smithi119.stderr:+ rmdir d1 2024-03-28T00:28:13.510 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:28:13.510 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:28:13.510 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 11 -lt 100 ']' 2024-03-28T00:28:13.511 INFO:tasks.workunit.client.0.smithi119.stderr:++ expr 11 + 1 2024-03-28T00:28:13.608 INFO:tasks.workunit.client.0.smithi119.stderr:+ c=12 2024-03-28T00:28:13.608 INFO:tasks.workunit.client.0.smithi119.stderr:+ mkdir d1 2024-03-28T00:28:13.643 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:28:13.643 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:28:13.647 INFO:tasks.workunit.client.0.smithi119.stderr:+ c1=2 2024-03-28T00:28:13.648 INFO:tasks.workunit.client.0.smithi119.stderr:+ echo 3 2024-03-28T00:28:13.648 INFO:tasks.workunit.client.0.smithi119.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-03-28T00:28:14.260 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:28:14.260 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:28:14.422 INFO:tasks.workunit.client.0.smithi119.stderr:+ c2=2 2024-03-28T00:28:14.422 INFO:tasks.workunit.client.0.smithi119.stderr:+ rmdir d1 2024-03-28T00:28:14.442 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:28:14.442 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:28:14.442 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 12 -lt 100 ']' 2024-03-28T00:28:14.444 INFO:tasks.workunit.client.0.smithi119.stderr:++ expr 12 + 1 2024-03-28T00:28:14.494 INFO:tasks.workunit.client.0.smithi119.stderr:+ c=13 2024-03-28T00:28:14.494 INFO:tasks.workunit.client.0.smithi119.stderr:+ mkdir d1 2024-03-28T00:28:14.528 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:28:14.528 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:28:14.532 INFO:tasks.workunit.client.0.smithi119.stderr:+ c1=2 2024-03-28T00:28:14.533 INFO:tasks.workunit.client.0.smithi119.stderr:+ echo 3 2024-03-28T00:28:14.534 INFO:tasks.workunit.client.0.smithi119.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-03-28T00:28:14.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:28:14 smithi165 ceph-mon[36302]: pgmap v53: 97 pgs: 97 active+clean; 1.4 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 31 KiB/s wr, 27 op/s 2024-03-28T00:28:14.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:28:14 smithi119 ceph-mon[31518]: pgmap v53: 97 pgs: 97 active+clean; 1.4 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 31 KiB/s wr, 27 op/s 2024-03-28T00:28:14.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:28:14 smithi132 ceph-mon[35941]: pgmap v53: 97 pgs: 97 active+clean; 1.4 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 31 KiB/s wr, 27 op/s 2024-03-28T00:28:15.093 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:28:15.094 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:28:15.280 INFO:tasks.workunit.client.0.smithi119.stderr:+ c2=2 2024-03-28T00:28:15.280 INFO:tasks.workunit.client.0.smithi119.stderr:+ rmdir d1 2024-03-28T00:28:15.299 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:28:15.299 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:28:15.299 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 13 -lt 100 ']' 2024-03-28T00:28:15.300 INFO:tasks.workunit.client.0.smithi119.stderr:++ expr 13 + 1 2024-03-28T00:28:15.331 INFO:tasks.workunit.client.0.smithi119.stderr:+ c=14 2024-03-28T00:28:15.331 INFO:tasks.workunit.client.0.smithi119.stderr:+ mkdir d1 2024-03-28T00:28:15.358 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:28:15.359 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:28:15.363 INFO:tasks.workunit.client.0.smithi119.stderr:+ c1=2 2024-03-28T00:28:15.364 INFO:tasks.workunit.client.0.smithi119.stderr:+ echo 3 2024-03-28T00:28:15.365 INFO:tasks.workunit.client.0.smithi119.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-03-28T00:28:16.008 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:28:16.008 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:28:16.198 INFO:tasks.workunit.client.0.smithi119.stderr:+ c2=2 2024-03-28T00:28:16.198 INFO:tasks.workunit.client.0.smithi119.stderr:+ rmdir d1 2024-03-28T00:28:16.208 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:28:16.208 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:28:16.209 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 14 -lt 100 ']' 2024-03-28T00:28:16.210 INFO:tasks.workunit.client.0.smithi119.stderr:++ expr 14 + 1 2024-03-28T00:28:16.263 INFO:tasks.workunit.client.0.smithi119.stderr:+ c=15 2024-03-28T00:28:16.263 INFO:tasks.workunit.client.0.smithi119.stderr:+ mkdir d1 2024-03-28T00:28:16.293 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:28:16.293 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:28:16.299 INFO:tasks.workunit.client.0.smithi119.stderr:+ c1=2 2024-03-28T00:28:16.300 INFO:tasks.workunit.client.0.smithi119.stderr:+ echo 3 2024-03-28T00:28:16.300 INFO:tasks.workunit.client.0.smithi119.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-03-28T00:28:16.606 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:28:16 smithi119 ceph-mon[31518]: pgmap v54: 97 pgs: 97 active+clean; 1.5 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 39 KiB/s wr, 34 op/s 2024-03-28T00:28:16.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:28:16 smithi165 ceph-mon[36302]: pgmap v54: 97 pgs: 97 active+clean; 1.5 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 39 KiB/s wr, 34 op/s 2024-03-28T00:28:16.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:28:16 smithi132 ceph-mon[35941]: pgmap v54: 97 pgs: 97 active+clean; 1.5 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 39 KiB/s wr, 34 op/s 2024-03-28T00:28:16.851 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:28:16.852 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:28:17.007 INFO:tasks.workunit.client.0.smithi119.stderr:+ c2=2 2024-03-28T00:28:17.007 INFO:tasks.workunit.client.0.smithi119.stderr:+ rmdir d1 2024-03-28T00:28:17.024 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:28:17.024 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:28:17.024 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 15 -lt 100 ']' 2024-03-28T00:28:17.025 INFO:tasks.workunit.client.0.smithi119.stderr:++ expr 15 + 1 2024-03-28T00:28:17.073 INFO:tasks.workunit.client.0.smithi119.stderr:+ c=16 2024-03-28T00:28:17.073 INFO:tasks.workunit.client.0.smithi119.stderr:+ mkdir d1 2024-03-28T00:28:17.101 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:28:17.101 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:28:17.106 INFO:tasks.workunit.client.0.smithi119.stderr:+ c1=2 2024-03-28T00:28:17.107 INFO:tasks.workunit.client.0.smithi119.stderr:+ echo 3 2024-03-28T00:28:17.107 INFO:tasks.workunit.client.0.smithi119.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-03-28T00:28:17.618 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:28:17.618 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:28:17.891 INFO:tasks.workunit.client.0.smithi119.stderr:+ c2=2 2024-03-28T00:28:17.891 INFO:tasks.workunit.client.0.smithi119.stderr:+ rmdir d1 2024-03-28T00:28:17.909 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:28:17.909 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:28:17.909 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 16 -lt 100 ']' 2024-03-28T00:28:17.911 INFO:tasks.workunit.client.0.smithi119.stderr:++ expr 16 + 1 2024-03-28T00:28:17.939 INFO:tasks.workunit.client.0.smithi119.stderr:+ c=17 2024-03-28T00:28:17.939 INFO:tasks.workunit.client.0.smithi119.stderr:+ mkdir d1 2024-03-28T00:28:17.966 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:28:17.967 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:28:17.971 INFO:tasks.workunit.client.0.smithi119.stderr:+ c1=2 2024-03-28T00:28:17.972 INFO:tasks.workunit.client.0.smithi119.stderr:+ echo 3 2024-03-28T00:28:17.973 INFO:tasks.workunit.client.0.smithi119.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-03-28T00:28:18.534 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:28:18.534 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:28:18.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:28:18 smithi165 ceph-mon[36302]: pgmap v55: 97 pgs: 97 active+clean; 1.5 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-03-28T00:28:18.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:28:18 smithi119 ceph-mon[31518]: pgmap v55: 97 pgs: 97 active+clean; 1.5 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-03-28T00:28:18.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:28:18 smithi132 ceph-mon[35941]: pgmap v55: 97 pgs: 97 active+clean; 1.5 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-03-28T00:28:18.754 INFO:tasks.workunit.client.0.smithi119.stderr:+ c2=2 2024-03-28T00:28:18.754 INFO:tasks.workunit.client.0.smithi119.stderr:+ rmdir d1 2024-03-28T00:28:18.766 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:28:18.766 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:28:18.766 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 17 -lt 100 ']' 2024-03-28T00:28:18.768 INFO:tasks.workunit.client.0.smithi119.stderr:++ expr 17 + 1 2024-03-28T00:28:18.814 INFO:tasks.workunit.client.0.smithi119.stderr:+ c=18 2024-03-28T00:28:18.814 INFO:tasks.workunit.client.0.smithi119.stderr:+ mkdir d1 2024-03-28T00:28:18.849 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:28:18.850 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:28:18.854 INFO:tasks.workunit.client.0.smithi119.stderr:+ c1=2 2024-03-28T00:28:18.856 INFO:tasks.workunit.client.0.smithi119.stderr:+ echo 3 2024-03-28T00:28:18.856 INFO:tasks.workunit.client.0.smithi119.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-03-28T00:28:19.443 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:28:19.443 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:28:19.621 INFO:tasks.workunit.client.0.smithi119.stderr:+ c2=2 2024-03-28T00:28:19.621 INFO:tasks.workunit.client.0.smithi119.stderr:+ rmdir d1 2024-03-28T00:28:19.645 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:28:19.645 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:28:19.646 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 18 -lt 100 ']' 2024-03-28T00:28:19.647 INFO:tasks.workunit.client.0.smithi119.stderr:++ expr 18 + 1 2024-03-28T00:28:19.688 INFO:tasks.workunit.client.0.smithi119.stderr:+ c=19 2024-03-28T00:28:19.688 INFO:tasks.workunit.client.0.smithi119.stderr:+ mkdir d1 2024-03-28T00:28:19.725 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:28:19.725 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:28:19.730 INFO:tasks.workunit.client.0.smithi119.stderr:+ c1=2 2024-03-28T00:28:19.731 INFO:tasks.workunit.client.0.smithi119.stderr:+ echo 3 2024-03-28T00:28:19.732 INFO:tasks.workunit.client.0.smithi119.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-03-28T00:28:20.343 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:28:20.343 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:28:20.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:28:20 smithi119 ceph-mon[31518]: pgmap v56: 97 pgs: 97 active+clean; 1.6 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 23 KiB/s wr, 19 op/s 2024-03-28T00:28:20.496 INFO:tasks.workunit.client.0.smithi119.stderr:+ c2=2 2024-03-28T00:28:20.497 INFO:tasks.workunit.client.0.smithi119.stderr:+ rmdir d1 2024-03-28T00:28:20.514 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:28:20.515 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:28:20.515 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 19 -lt 100 ']' 2024-03-28T00:28:20.516 INFO:tasks.workunit.client.0.smithi119.stderr:++ expr 19 + 1 2024-03-28T00:28:20.547 INFO:tasks.workunit.client.0.smithi119.stderr:+ c=20 2024-03-28T00:28:20.547 INFO:tasks.workunit.client.0.smithi119.stderr:+ mkdir d1 2024-03-28T00:28:20.576 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:28:20.576 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:28:20.580 INFO:tasks.workunit.client.0.smithi119.stderr:+ c1=2 2024-03-28T00:28:20.581 INFO:tasks.workunit.client.0.smithi119.stderr:+ echo 3 2024-03-28T00:28:20.582 INFO:tasks.workunit.client.0.smithi119.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-03-28T00:28:20.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:28:20 smithi165 ceph-mon[36302]: pgmap v56: 97 pgs: 97 active+clean; 1.6 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 23 KiB/s wr, 19 op/s 2024-03-28T00:28:20.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:28:20 smithi132 ceph-mon[35941]: pgmap v56: 97 pgs: 97 active+clean; 1.6 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 23 KiB/s wr, 19 op/s 2024-03-28T00:28:21.209 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:28:21.210 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:28:21.365 INFO:tasks.workunit.client.0.smithi119.stderr:+ c2=2 2024-03-28T00:28:21.366 INFO:tasks.workunit.client.0.smithi119.stderr:+ rmdir d1 2024-03-28T00:28:21.384 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:28:21.384 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:28:21.384 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 20 -lt 100 ']' 2024-03-28T00:28:21.386 INFO:tasks.workunit.client.0.smithi119.stderr:++ expr 20 + 1 2024-03-28T00:28:21.422 INFO:tasks.workunit.client.0.smithi119.stderr:+ c=21 2024-03-28T00:28:21.422 INFO:tasks.workunit.client.0.smithi119.stderr:+ mkdir d1 2024-03-28T00:28:21.442 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:28:21.442 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:28:21.447 INFO:tasks.workunit.client.0.smithi119.stderr:+ c1=2 2024-03-28T00:28:21.448 INFO:tasks.workunit.client.0.smithi119.stderr:+ echo 3 2024-03-28T00:28:21.449 INFO:tasks.workunit.client.0.smithi119.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-03-28T00:28:22.053 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:28:22.053 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:28:22.229 INFO:tasks.workunit.client.0.smithi119.stderr:+ c2=2 2024-03-28T00:28:22.229 INFO:tasks.workunit.client.0.smithi119.stderr:+ rmdir d1 2024-03-28T00:28:22.239 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:28:22.239 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:28:22.239 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 21 -lt 100 ']' 2024-03-28T00:28:22.241 INFO:tasks.workunit.client.0.smithi119.stderr:++ expr 21 + 1 2024-03-28T00:28:22.280 INFO:tasks.workunit.client.0.smithi119.stderr:+ c=22 2024-03-28T00:28:22.280 INFO:tasks.workunit.client.0.smithi119.stderr:+ mkdir d1 2024-03-28T00:28:22.308 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:28:22.309 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:28:22.313 INFO:tasks.workunit.client.0.smithi119.stderr:+ c1=2 2024-03-28T00:28:22.313 INFO:tasks.workunit.client.0.smithi119.stderr:+ echo 3 2024-03-28T00:28:22.314 INFO:tasks.workunit.client.0.smithi119.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-03-28T00:28:22.622 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:28:22 smithi119 ceph-mon[31518]: pgmap v57: 97 pgs: 97 active+clean; 1.6 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 14 op/s 2024-03-28T00:28:22.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:28:22 smithi165 ceph-mon[36302]: pgmap v57: 97 pgs: 97 active+clean; 1.6 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 14 op/s 2024-03-28T00:28:22.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:28:22 smithi132 ceph-mon[35941]: pgmap v57: 97 pgs: 97 active+clean; 1.6 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 14 op/s 2024-03-28T00:28:22.875 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:28:22.876 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:28:23.037 INFO:tasks.workunit.client.0.smithi119.stderr:+ c2=2 2024-03-28T00:28:23.037 INFO:tasks.workunit.client.0.smithi119.stderr:+ rmdir d1 2024-03-28T00:28:23.058 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:28:23.058 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:28:23.058 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 22 -lt 100 ']' 2024-03-28T00:28:23.059 INFO:tasks.workunit.client.0.smithi119.stderr:++ expr 22 + 1 2024-03-28T00:28:23.102 INFO:tasks.workunit.client.0.smithi119.stderr:+ c=23 2024-03-28T00:28:23.102 INFO:tasks.workunit.client.0.smithi119.stderr:+ mkdir d1 2024-03-28T00:28:23.133 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:28:23.133 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:28:23.138 INFO:tasks.workunit.client.0.smithi119.stderr:+ c1=2 2024-03-28T00:28:23.139 INFO:tasks.workunit.client.0.smithi119.stderr:+ echo 3 2024-03-28T00:28:23.140 INFO:tasks.workunit.client.0.smithi119.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-03-28T00:28:23.742 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:28:23.743 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:28:23.973 INFO:tasks.workunit.client.0.smithi119.stderr:+ c2=2 2024-03-28T00:28:23.974 INFO:tasks.workunit.client.0.smithi119.stderr:+ rmdir d1 2024-03-28T00:28:23.990 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:28:23.990 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:28:23.990 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 23 -lt 100 ']' 2024-03-28T00:28:23.991 INFO:tasks.workunit.client.0.smithi119.stderr:++ expr 23 + 1 2024-03-28T00:28:24.029 INFO:tasks.workunit.client.0.smithi119.stderr:+ c=24 2024-03-28T00:28:24.029 INFO:tasks.workunit.client.0.smithi119.stderr:+ mkdir d1 2024-03-28T00:28:24.058 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:28:24.059 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:28:24.063 INFO:tasks.workunit.client.0.smithi119.stderr:+ c1=2 2024-03-28T00:28:24.064 INFO:tasks.workunit.client.0.smithi119.stderr:+ echo 3 2024-03-28T00:28:24.065 INFO:tasks.workunit.client.0.smithi119.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-03-28T00:28:24.580 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:28:24 smithi119 ceph-mon[31518]: pgmap v58: 97 pgs: 97 active+clean; 1.6 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 KiB/s wr, 14 op/s 2024-03-28T00:28:24.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:28:24 smithi165 ceph-mon[36302]: pgmap v58: 97 pgs: 97 active+clean; 1.6 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 KiB/s wr, 14 op/s 2024-03-28T00:28:24.683 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:28:24.683 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:28:24.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:28:24 smithi132 ceph-mon[35941]: pgmap v58: 97 pgs: 97 active+clean; 1.6 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 KiB/s wr, 14 op/s 2024-03-28T00:28:24.856 INFO:tasks.workunit.client.0.smithi119.stderr:+ c2=2 2024-03-28T00:28:24.857 INFO:tasks.workunit.client.0.smithi119.stderr:+ rmdir d1 2024-03-28T00:28:24.873 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:28:24.873 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:28:24.873 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 24 -lt 100 ']' 2024-03-28T00:28:24.875 INFO:tasks.workunit.client.0.smithi119.stderr:++ expr 24 + 1 2024-03-28T00:28:24.913 INFO:tasks.workunit.client.0.smithi119.stderr:+ c=25 2024-03-28T00:28:24.913 INFO:tasks.workunit.client.0.smithi119.stderr:+ mkdir d1 2024-03-28T00:28:24.941 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:28:24.942 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:28:24.946 INFO:tasks.workunit.client.0.smithi119.stderr:+ c1=2 2024-03-28T00:28:24.946 INFO:tasks.workunit.client.0.smithi119.stderr:+ echo 3 2024-03-28T00:28:24.947 INFO:tasks.workunit.client.0.smithi119.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-03-28T00:28:25.525 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:28:25.525 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:28:25.703 INFO:tasks.workunit.client.0.smithi119.stderr:+ c2=2 2024-03-28T00:28:25.703 INFO:tasks.workunit.client.0.smithi119.stderr:+ rmdir d1 2024-03-28T00:28:25.713 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:28:25.713 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:28:25.713 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 25 -lt 100 ']' 2024-03-28T00:28:25.714 INFO:tasks.workunit.client.0.smithi119.stderr:++ expr 25 + 1 2024-03-28T00:28:25.746 INFO:tasks.workunit.client.0.smithi119.stderr:+ c=26 2024-03-28T00:28:25.746 INFO:tasks.workunit.client.0.smithi119.stderr:+ mkdir d1 2024-03-28T00:28:25.774 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:28:25.774 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:28:25.778 INFO:tasks.workunit.client.0.smithi119.stderr:+ c1=2 2024-03-28T00:28:25.779 INFO:tasks.workunit.client.0.smithi119.stderr:+ echo 3 2024-03-28T00:28:25.780 INFO:tasks.workunit.client.0.smithi119.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-03-28T00:28:26.466 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:28:26.467 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:28:26.619 INFO:tasks.workunit.client.0.smithi119.stderr:+ c2=2 2024-03-28T00:28:26.620 INFO:tasks.workunit.client.0.smithi119.stderr:+ rmdir d1 2024-03-28T00:28:26.631 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:28:26.631 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:28:26.631 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 26 -lt 100 ']' 2024-03-28T00:28:26.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:28:26 smithi165 ceph-mon[36302]: pgmap v59: 97 pgs: 97 active+clean; 1.7 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 25 KiB/s wr, 21 op/s 2024-03-28T00:28:26.633 INFO:tasks.workunit.client.0.smithi119.stderr:++ expr 26 + 1 2024-03-28T00:28:26.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:28:26 smithi119 ceph-mon[31518]: pgmap v59: 97 pgs: 97 active+clean; 1.7 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 25 KiB/s wr, 21 op/s 2024-03-28T00:28:26.679 INFO:tasks.workunit.client.0.smithi119.stderr:+ c=27 2024-03-28T00:28:26.680 INFO:tasks.workunit.client.0.smithi119.stderr:+ mkdir d1 2024-03-28T00:28:26.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:28:26 smithi132 ceph-mon[35941]: pgmap v59: 97 pgs: 97 active+clean; 1.7 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 25 KiB/s wr, 21 op/s 2024-03-28T00:28:26.707 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:28:26.708 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:28:26.712 INFO:tasks.workunit.client.0.smithi119.stderr:+ c1=2 2024-03-28T00:28:26.714 INFO:tasks.workunit.client.0.smithi119.stderr:+ echo 3 2024-03-28T00:28:26.715 INFO:tasks.workunit.client.0.smithi119.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-03-28T00:28:27.268 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:28:27.268 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:28:27.470 INFO:tasks.workunit.client.0.smithi119.stderr:+ c2=2 2024-03-28T00:28:27.470 INFO:tasks.workunit.client.0.smithi119.stderr:+ rmdir d1 2024-03-28T00:28:27.481 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:28:27.481 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:28:27.481 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 27 -lt 100 ']' 2024-03-28T00:28:27.482 INFO:tasks.workunit.client.0.smithi119.stderr:++ expr 27 + 1 2024-03-28T00:28:27.512 INFO:tasks.workunit.client.0.smithi119.stderr:+ c=28 2024-03-28T00:28:27.513 INFO:tasks.workunit.client.0.smithi119.stderr:+ mkdir d1 2024-03-28T00:28:27.540 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:28:27.540 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:28:27.544 INFO:tasks.workunit.client.0.smithi119.stderr:+ c1=2 2024-03-28T00:28:27.545 INFO:tasks.workunit.client.0.smithi119.stderr:+ echo 3 2024-03-28T00:28:27.546 INFO:tasks.workunit.client.0.smithi119.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-03-28T00:28:28.466 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:28:28.466 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:28:28.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:28:28 smithi165 ceph-mon[36302]: pgmap v60: 97 pgs: 97 active+clean; 1.7 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 14 op/s 2024-03-28T00:28:28.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:28:28 smithi119 ceph-mon[31518]: pgmap v60: 97 pgs: 97 active+clean; 1.7 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 14 op/s 2024-03-28T00:28:28.700 INFO:tasks.workunit.client.0.smithi119.stderr:+ c2=2 2024-03-28T00:28:28.700 INFO:tasks.workunit.client.0.smithi119.stderr:+ rmdir d1 2024-03-28T00:28:28.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:28:28 smithi132 ceph-mon[35941]: pgmap v60: 97 pgs: 97 active+clean; 1.7 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 14 op/s 2024-03-28T00:28:28.724 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:28:28.724 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:28:28.724 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 28 -lt 100 ']' 2024-03-28T00:28:28.725 INFO:tasks.workunit.client.0.smithi119.stderr:++ expr 28 + 1 2024-03-28T00:28:28.754 INFO:tasks.workunit.client.0.smithi119.stderr:+ c=29 2024-03-28T00:28:28.754 INFO:tasks.workunit.client.0.smithi119.stderr:+ mkdir d1 2024-03-28T00:28:28.792 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:28:28.792 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:28:28.796 INFO:tasks.workunit.client.0.smithi119.stderr:+ c1=2 2024-03-28T00:28:28.797 INFO:tasks.workunit.client.0.smithi119.stderr:+ echo 3 2024-03-28T00:28:28.798 INFO:tasks.workunit.client.0.smithi119.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-03-28T00:28:29.623 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:28:29.623 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:28:29.779 INFO:tasks.workunit.client.0.smithi119.stderr:+ c2=2 2024-03-28T00:28:29.779 INFO:tasks.workunit.client.0.smithi119.stderr:+ rmdir d1 2024-03-28T00:28:29.799 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:28:29.799 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:28:29.799 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 29 -lt 100 ']' 2024-03-28T00:28:29.800 INFO:tasks.workunit.client.0.smithi119.stderr:++ expr 29 + 1 2024-03-28T00:28:29.846 INFO:tasks.workunit.client.0.smithi119.stderr:+ c=30 2024-03-28T00:28:29.846 INFO:tasks.workunit.client.0.smithi119.stderr:+ mkdir d1 2024-03-28T00:28:29.873 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:28:29.874 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:28:29.878 INFO:tasks.workunit.client.0.smithi119.stderr:+ c1=2 2024-03-28T00:28:29.879 INFO:tasks.workunit.client.0.smithi119.stderr:+ echo 3 2024-03-28T00:28:29.880 INFO:tasks.workunit.client.0.smithi119.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-03-28T00:28:30.514 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:28:30.515 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:28:30.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:28:30 smithi165 ceph-mon[36302]: pgmap v61: 97 pgs: 97 active+clean; 1.8 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 25 KiB/s wr, 20 op/s 2024-03-28T00:28:30.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:28:30 smithi119 ceph-mon[31518]: pgmap v61: 97 pgs: 97 active+clean; 1.8 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 25 KiB/s wr, 20 op/s 2024-03-28T00:28:30.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:28:30 smithi132 ceph-mon[35941]: pgmap v61: 97 pgs: 97 active+clean; 1.8 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 25 KiB/s wr, 20 op/s 2024-03-28T00:28:30.753 INFO:tasks.workunit.client.0.smithi119.stderr:+ c2=2 2024-03-28T00:28:30.754 INFO:tasks.workunit.client.0.smithi119.stderr:+ rmdir d1 2024-03-28T00:28:30.770 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:28:30.771 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:28:30.771 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 30 -lt 100 ']' 2024-03-28T00:28:30.772 INFO:tasks.workunit.client.0.smithi119.stderr:++ expr 30 + 1 2024-03-28T00:28:30.812 INFO:tasks.workunit.client.0.smithi119.stderr:+ c=31 2024-03-28T00:28:30.812 INFO:tasks.workunit.client.0.smithi119.stderr:+ mkdir d1 2024-03-28T00:28:30.841 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:28:30.841 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:28:30.846 INFO:tasks.workunit.client.0.smithi119.stderr:+ c1=2 2024-03-28T00:28:30.847 INFO:tasks.workunit.client.0.smithi119.stderr:+ echo 3 2024-03-28T00:28:30.847 INFO:tasks.workunit.client.0.smithi119.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-03-28T00:28:31.431 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:28:31.432 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:28:31.611 INFO:tasks.workunit.client.0.smithi119.stderr:+ c2=2 2024-03-28T00:28:31.611 INFO:tasks.workunit.client.0.smithi119.stderr:+ rmdir d1 2024-03-28T00:28:31.631 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:28:31.631 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:28:31.631 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 31 -lt 100 ']' 2024-03-28T00:28:31.632 INFO:tasks.workunit.client.0.smithi119.stderr:++ expr 31 + 1 2024-03-28T00:28:31.679 INFO:tasks.workunit.client.0.smithi119.stderr:+ c=32 2024-03-28T00:28:31.679 INFO:tasks.workunit.client.0.smithi119.stderr:+ mkdir d1 2024-03-28T00:28:31.707 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:28:31.707 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:28:31.711 INFO:tasks.workunit.client.0.smithi119.stderr:+ c1=2 2024-03-28T00:28:31.712 INFO:tasks.workunit.client.0.smithi119.stderr:+ echo 3 2024-03-28T00:28:31.713 INFO:tasks.workunit.client.0.smithi119.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-03-28T00:28:32.301 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:28:32.301 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:28:32.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:28:32 smithi119 ceph-mon[31518]: pgmap v62: 97 pgs: 97 active+clean; 1.8 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 14 op/s 2024-03-28T00:28:32.485 INFO:tasks.workunit.client.0.smithi119.stderr:+ c2=2 2024-03-28T00:28:32.485 INFO:tasks.workunit.client.0.smithi119.stderr:+ rmdir d1 2024-03-28T00:28:32.499 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:28:32.499 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:28:32.499 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 32 -lt 100 ']' 2024-03-28T00:28:32.500 INFO:tasks.workunit.client.0.smithi119.stderr:++ expr 32 + 1 2024-03-28T00:28:32.528 INFO:tasks.workunit.client.0.smithi119.stderr:+ c=33 2024-03-28T00:28:32.528 INFO:tasks.workunit.client.0.smithi119.stderr:+ mkdir d1 2024-03-28T00:28:32.556 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:28:32.556 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:28:32.560 INFO:tasks.workunit.client.0.smithi119.stderr:+ c1=2 2024-03-28T00:28:32.562 INFO:tasks.workunit.client.0.smithi119.stderr:+ echo 3 2024-03-28T00:28:32.563 INFO:tasks.workunit.client.0.smithi119.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-03-28T00:28:32.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:28:32 smithi165 ceph-mon[36302]: pgmap v62: 97 pgs: 97 active+clean; 1.8 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 14 op/s 2024-03-28T00:28:32.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:28:32 smithi132 ceph-mon[35941]: pgmap v62: 97 pgs: 97 active+clean; 1.8 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 14 op/s 2024-03-28T00:28:33.216 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:28:33.216 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:28:33.369 INFO:tasks.workunit.client.0.smithi119.stderr:+ c2=2 2024-03-28T00:28:33.370 INFO:tasks.workunit.client.0.smithi119.stderr:+ rmdir d1 2024-03-28T00:28:33.388 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:28:33.388 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:28:33.388 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 33 -lt 100 ']' 2024-03-28T00:28:33.389 INFO:tasks.workunit.client.0.smithi119.stderr:++ expr 33 + 1 2024-03-28T00:28:33.420 INFO:tasks.workunit.client.0.smithi119.stderr:+ c=34 2024-03-28T00:28:33.420 INFO:tasks.workunit.client.0.smithi119.stderr:+ mkdir d1 2024-03-28T00:28:33.456 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:28:33.457 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:28:33.461 INFO:tasks.workunit.client.0.smithi119.stderr:+ c1=2 2024-03-28T00:28:33.462 INFO:tasks.workunit.client.0.smithi119.stderr:+ echo 3 2024-03-28T00:28:33.463 INFO:tasks.workunit.client.0.smithi119.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-03-28T00:28:34.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:28:34 smithi165 ceph-mon[36302]: pgmap v63: 97 pgs: 97 active+clean; 1.8 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 14 op/s 2024-03-28T00:28:34.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:28:34 smithi119 ceph-mon[31518]: pgmap v63: 97 pgs: 97 active+clean; 1.8 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 14 op/s 2024-03-28T00:28:34.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:28:34 smithi132 ceph-mon[35941]: pgmap v63: 97 pgs: 97 active+clean; 1.8 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 14 op/s 2024-03-28T00:28:34.931 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:28:34.931 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:28:35.110 INFO:tasks.workunit.client.0.smithi119.stderr:+ c2=2 2024-03-28T00:28:35.110 INFO:tasks.workunit.client.0.smithi119.stderr:+ rmdir d1 2024-03-28T00:28:35.128 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:28:35.129 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:28:35.129 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 34 -lt 100 ']' 2024-03-28T00:28:35.129 INFO:tasks.workunit.client.0.smithi119.stderr:++ expr 34 + 1 2024-03-28T00:28:35.161 INFO:tasks.workunit.client.0.smithi119.stderr:+ c=35 2024-03-28T00:28:35.161 INFO:tasks.workunit.client.0.smithi119.stderr:+ mkdir d1 2024-03-28T00:28:35.197 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:28:35.197 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:28:35.201 INFO:tasks.workunit.client.0.smithi119.stderr:+ c1=2 2024-03-28T00:28:35.203 INFO:tasks.workunit.client.0.smithi119.stderr:+ echo 3 2024-03-28T00:28:35.204 INFO:tasks.workunit.client.0.smithi119.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-03-28T00:28:36.047 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:28:36.047 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:28:36.226 INFO:tasks.workunit.client.0.smithi119.stderr:+ c2=2 2024-03-28T00:28:36.227 INFO:tasks.workunit.client.0.smithi119.stderr:+ rmdir d1 2024-03-28T00:28:36.246 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:28:36.246 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:28:36.246 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 35 -lt 100 ']' 2024-03-28T00:28:36.248 INFO:tasks.workunit.client.0.smithi119.stderr:++ expr 35 + 1 2024-03-28T00:28:36.278 INFO:tasks.workunit.client.0.smithi119.stderr:+ c=36 2024-03-28T00:28:36.278 INFO:tasks.workunit.client.0.smithi119.stderr:+ mkdir d1 2024-03-28T00:28:36.314 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:28:36.314 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:28:36.319 INFO:tasks.workunit.client.0.smithi119.stderr:+ c1=2 2024-03-28T00:28:36.319 INFO:tasks.workunit.client.0.smithi119.stderr:+ echo 3 2024-03-28T00:28:36.320 INFO:tasks.workunit.client.0.smithi119.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-03-28T00:28:36.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:28:36 smithi165 ceph-mon[36302]: pgmap v64: 97 pgs: 97 active+clean; 1.8 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 24 KiB/s wr, 20 op/s 2024-03-28T00:28:36.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:28:36 smithi119 ceph-mon[31518]: pgmap v64: 97 pgs: 97 active+clean; 1.8 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 24 KiB/s wr, 20 op/s 2024-03-28T00:28:36.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:28:36 smithi132 ceph-mon[35941]: pgmap v64: 97 pgs: 97 active+clean; 1.8 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 24 KiB/s wr, 20 op/s 2024-03-28T00:28:37.047 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:28:37.047 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:28:37.257 INFO:tasks.workunit.client.0.smithi119.stderr:+ c2=2 2024-03-28T00:28:37.257 INFO:tasks.workunit.client.0.smithi119.stderr:+ rmdir d1 2024-03-28T00:28:37.280 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:28:37.280 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:28:37.280 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 36 -lt 100 ']' 2024-03-28T00:28:37.281 INFO:tasks.workunit.client.0.smithi119.stderr:++ expr 36 + 1 2024-03-28T00:28:37.311 INFO:tasks.workunit.client.0.smithi119.stderr:+ c=37 2024-03-28T00:28:37.311 INFO:tasks.workunit.client.0.smithi119.stderr:+ mkdir d1 2024-03-28T00:28:37.346 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:28:37.347 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:28:37.351 INFO:tasks.workunit.client.0.smithi119.stderr:+ c1=2 2024-03-28T00:28:37.352 INFO:tasks.workunit.client.0.smithi119.stderr:+ echo 3 2024-03-28T00:28:37.353 INFO:tasks.workunit.client.0.smithi119.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-03-28T00:28:38.421 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:28:38.421 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:28:38.585 INFO:tasks.workunit.client.0.smithi119.stderr:+ c2=2 2024-03-28T00:28:38.585 INFO:tasks.workunit.client.0.smithi119.stderr:+ rmdir d1 2024-03-28T00:28:38.604 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:28:38.605 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:28:38.605 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 37 -lt 100 ']' 2024-03-28T00:28:38.606 INFO:tasks.workunit.client.0.smithi119.stderr:++ expr 37 + 1 2024-03-28T00:28:38.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:28:38 smithi165 ceph-mon[36302]: pgmap v65: 97 pgs: 97 active+clean; 1.8 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 13 op/s 2024-03-28T00:28:38.636 INFO:tasks.workunit.client.0.smithi119.stderr:+ c=38 2024-03-28T00:28:38.636 INFO:tasks.workunit.client.0.smithi119.stderr:+ mkdir d1 2024-03-28T00:28:38.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:28:38 smithi119 ceph-mon[31518]: pgmap v65: 97 pgs: 97 active+clean; 1.8 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 13 op/s 2024-03-28T00:28:38.672 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:28:38.672 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:28:38.676 INFO:tasks.workunit.client.0.smithi119.stderr:+ c1=2 2024-03-28T00:28:38.678 INFO:tasks.workunit.client.0.smithi119.stderr:+ echo 3 2024-03-28T00:28:38.679 INFO:tasks.workunit.client.0.smithi119.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-03-28T00:28:38.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:28:38 smithi132 ceph-mon[35941]: pgmap v65: 97 pgs: 97 active+clean; 1.8 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 13 op/s 2024-03-28T00:28:39.141 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:28:39.141 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:28:39.326 INFO:tasks.workunit.client.0.smithi119.stderr:+ c2=2 2024-03-28T00:28:39.327 INFO:tasks.workunit.client.0.smithi119.stderr:+ rmdir d1 2024-03-28T00:28:39.338 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:28:39.338 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:28:39.338 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 38 -lt 100 ']' 2024-03-28T00:28:39.339 INFO:tasks.workunit.client.0.smithi119.stderr:++ expr 38 + 1 2024-03-28T00:28:39.394 INFO:tasks.workunit.client.0.smithi119.stderr:+ c=39 2024-03-28T00:28:39.394 INFO:tasks.workunit.client.0.smithi119.stderr:+ mkdir d1 2024-03-28T00:28:39.421 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:28:39.422 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:28:39.426 INFO:tasks.workunit.client.0.smithi119.stderr:+ c1=2 2024-03-28T00:28:39.427 INFO:tasks.workunit.client.0.smithi119.stderr:+ echo 3 2024-03-28T00:28:39.428 INFO:tasks.workunit.client.0.smithi119.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-03-28T00:28:40.006 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:28:40.006 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:28:40.155 INFO:tasks.workunit.client.0.smithi119.stderr:+ c2=2 2024-03-28T00:28:40.155 INFO:tasks.workunit.client.0.smithi119.stderr:+ rmdir d1 2024-03-28T00:28:40.169 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:28:40.169 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:28:40.169 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 39 -lt 100 ']' 2024-03-28T00:28:40.170 INFO:tasks.workunit.client.0.smithi119.stderr:++ expr 39 + 1 2024-03-28T00:28:40.211 INFO:tasks.workunit.client.0.smithi119.stderr:+ c=40 2024-03-28T00:28:40.211 INFO:tasks.workunit.client.0.smithi119.stderr:+ mkdir d1 2024-03-28T00:28:40.239 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:28:40.240 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:28:40.244 INFO:tasks.workunit.client.0.smithi119.stderr:+ c1=2 2024-03-28T00:28:40.245 INFO:tasks.workunit.client.0.smithi119.stderr:+ echo 3 2024-03-28T00:28:40.246 INFO:tasks.workunit.client.0.smithi119.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-03-28T00:28:40.561 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:28:40 smithi119 ceph-mon[31518]: pgmap v66: 97 pgs: 97 active+clean; 1.9 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 23 KiB/s wr, 19 op/s 2024-03-28T00:28:40.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:28:40 smithi165 ceph-mon[36302]: pgmap v66: 97 pgs: 97 active+clean; 1.9 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 23 KiB/s wr, 19 op/s 2024-03-28T00:28:40.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:28:40 smithi132 ceph-mon[35941]: pgmap v66: 97 pgs: 97 active+clean; 1.9 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 23 KiB/s wr, 19 op/s 2024-03-28T00:28:40.731 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:28:40.731 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:28:40.896 INFO:tasks.workunit.client.0.smithi119.stderr:+ c2=2 2024-03-28T00:28:40.896 INFO:tasks.workunit.client.0.smithi119.stderr:+ rmdir d1 2024-03-28T00:28:40.913 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:28:40.913 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:28:40.913 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 40 -lt 100 ']' 2024-03-28T00:28:40.914 INFO:tasks.workunit.client.0.smithi119.stderr:++ expr 40 + 1 2024-03-28T00:28:40.944 INFO:tasks.workunit.client.0.smithi119.stderr:+ c=41 2024-03-28T00:28:40.944 INFO:tasks.workunit.client.0.smithi119.stderr:+ mkdir d1 2024-03-28T00:28:40.972 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:28:40.972 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:28:40.976 INFO:tasks.workunit.client.0.smithi119.stderr:+ c1=2 2024-03-28T00:28:40.977 INFO:tasks.workunit.client.0.smithi119.stderr:+ echo 3 2024-03-28T00:28:40.977 INFO:tasks.workunit.client.0.smithi119.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-03-28T00:28:41.556 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:28:41.556 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:28:41.717 INFO:tasks.workunit.client.0.smithi119.stderr:+ c2=2 2024-03-28T00:28:41.717 INFO:tasks.workunit.client.0.smithi119.stderr:+ rmdir d1 2024-03-28T00:28:41.729 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:28:41.729 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:28:41.729 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 41 -lt 100 ']' 2024-03-28T00:28:41.730 INFO:tasks.workunit.client.0.smithi119.stderr:++ expr 41 + 1 2024-03-28T00:28:41.752 INFO:tasks.workunit.client.0.smithi119.stderr:+ c=42 2024-03-28T00:28:41.752 INFO:tasks.workunit.client.0.smithi119.stderr:+ mkdir d1 2024-03-28T00:28:41.789 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:28:41.790 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:28:41.794 INFO:tasks.workunit.client.0.smithi119.stderr:+ c1=2 2024-03-28T00:28:41.795 INFO:tasks.workunit.client.0.smithi119.stderr:+ echo 3 2024-03-28T00:28:41.796 INFO:tasks.workunit.client.0.smithi119.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-03-28T00:28:42.289 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:28:42.289 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:28:42.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:28:42 smithi119 ceph-mon[31518]: pgmap v67: 97 pgs: 97 active+clean; 1.9 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 15 KiB/s wr, 13 op/s 2024-03-28T00:28:42.442 INFO:tasks.workunit.client.0.smithi119.stderr:+ c2=2 2024-03-28T00:28:42.442 INFO:tasks.workunit.client.0.smithi119.stderr:+ rmdir d1 2024-03-28T00:28:42.453 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:28:42.453 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:28:42.453 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 42 -lt 100 ']' 2024-03-28T00:28:42.454 INFO:tasks.workunit.client.0.smithi119.stderr:++ expr 42 + 1 2024-03-28T00:28:42.477 INFO:tasks.workunit.client.0.smithi119.stderr:+ c=43 2024-03-28T00:28:42.477 INFO:tasks.workunit.client.0.smithi119.stderr:+ mkdir d1 2024-03-28T00:28:42.504 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:28:42.504 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:28:42.509 INFO:tasks.workunit.client.0.smithi119.stderr:+ c1=2 2024-03-28T00:28:42.510 INFO:tasks.workunit.client.0.smithi119.stderr:+ echo 3 2024-03-28T00:28:42.511 INFO:tasks.workunit.client.0.smithi119.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-03-28T00:28:42.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:28:42 smithi165 ceph-mon[36302]: pgmap v67: 97 pgs: 97 active+clean; 1.9 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 15 KiB/s wr, 13 op/s 2024-03-28T00:28:42.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:28:42 smithi132 ceph-mon[35941]: pgmap v67: 97 pgs: 97 active+clean; 1.9 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 15 KiB/s wr, 13 op/s 2024-03-28T00:28:42.988 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:28:42.989 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:28:43.150 INFO:tasks.workunit.client.0.smithi119.stderr:+ c2=2 2024-03-28T00:28:43.150 INFO:tasks.workunit.client.0.smithi119.stderr:+ rmdir d1 2024-03-28T00:28:43.162 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:28:43.162 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:28:43.162 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 43 -lt 100 ']' 2024-03-28T00:28:43.164 INFO:tasks.workunit.client.0.smithi119.stderr:++ expr 43 + 1 2024-03-28T00:28:43.293 INFO:tasks.workunit.client.0.smithi119.stderr:+ c=44 2024-03-28T00:28:43.293 INFO:tasks.workunit.client.0.smithi119.stderr:+ mkdir d1 2024-03-28T00:28:43.321 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:28:43.322 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:28:43.326 INFO:tasks.workunit.client.0.smithi119.stderr:+ c1=2 2024-03-28T00:28:43.327 INFO:tasks.workunit.client.0.smithi119.stderr:+ echo 3 2024-03-28T00:28:43.327 INFO:tasks.workunit.client.0.smithi119.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-03-28T00:28:43.888 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:28:43.889 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:28:44.023 INFO:tasks.workunit.client.0.smithi119.stderr:+ c2=2 2024-03-28T00:28:44.023 INFO:tasks.workunit.client.0.smithi119.stderr:+ rmdir d1 2024-03-28T00:28:44.045 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:28:44.045 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:28:44.045 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 44 -lt 100 ']' 2024-03-28T00:28:44.046 INFO:tasks.workunit.client.0.smithi119.stderr:++ expr 44 + 1 2024-03-28T00:28:44.076 INFO:tasks.workunit.client.0.smithi119.stderr:+ c=45 2024-03-28T00:28:44.076 INFO:tasks.workunit.client.0.smithi119.stderr:+ mkdir d1 2024-03-28T00:28:44.105 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:28:44.105 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:28:44.110 INFO:tasks.workunit.client.0.smithi119.stderr:+ c1=2 2024-03-28T00:28:44.111 INFO:tasks.workunit.client.0.smithi119.stderr:+ echo 3 2024-03-28T00:28:44.112 INFO:tasks.workunit.client.0.smithi119.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-03-28T00:28:44.568 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:28:44 smithi119 ceph-mon[31518]: pgmap v68: 97 pgs: 97 active+clean; 1.9 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 15 KiB/s wr, 12 op/s 2024-03-28T00:28:44.568 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:28:44 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:28:44.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:28:44 smithi165 ceph-mon[36302]: pgmap v68: 97 pgs: 97 active+clean; 1.9 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 15 KiB/s wr, 12 op/s 2024-03-28T00:28:44.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:28:44 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:28:44.698 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:28:44.699 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:28:44.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:28:44 smithi132 ceph-mon[35941]: pgmap v68: 97 pgs: 97 active+clean; 1.9 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 15 KiB/s wr, 12 op/s 2024-03-28T00:28:44.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:28:44 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:28:44.917 INFO:tasks.workunit.client.0.smithi119.stderr:+ c2=2 2024-03-28T00:28:44.917 INFO:tasks.workunit.client.0.smithi119.stderr:+ rmdir d1 2024-03-28T00:28:44.936 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:28:44.937 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:28:44.937 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 45 -lt 100 ']' 2024-03-28T00:28:44.938 INFO:tasks.workunit.client.0.smithi119.stderr:++ expr 45 + 1 2024-03-28T00:28:44.984 INFO:tasks.workunit.client.0.smithi119.stderr:+ c=46 2024-03-28T00:28:44.985 INFO:tasks.workunit.client.0.smithi119.stderr:+ mkdir d1 2024-03-28T00:28:45.030 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:28:45.031 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:28:45.035 INFO:tasks.workunit.client.0.smithi119.stderr:+ c1=2 2024-03-28T00:28:45.036 INFO:tasks.workunit.client.0.smithi119.stderr:+ echo 3 2024-03-28T00:28:45.037 INFO:tasks.workunit.client.0.smithi119.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-03-28T00:28:45.829 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:28:45.829 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:28:46.026 INFO:tasks.workunit.client.0.smithi119.stderr:+ c2=2 2024-03-28T00:28:46.027 INFO:tasks.workunit.client.0.smithi119.stderr:+ rmdir d1 2024-03-28T00:28:46.047 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:28:46.047 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:28:46.047 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 46 -lt 100 ']' 2024-03-28T00:28:46.048 INFO:tasks.workunit.client.0.smithi119.stderr:++ expr 46 + 1 2024-03-28T00:28:46.076 INFO:tasks.workunit.client.0.smithi119.stderr:+ c=47 2024-03-28T00:28:46.076 INFO:tasks.workunit.client.0.smithi119.stderr:+ mkdir d1 2024-03-28T00:28:46.122 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:28:46.123 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:28:46.127 INFO:tasks.workunit.client.0.smithi119.stderr:+ c1=2 2024-03-28T00:28:46.129 INFO:tasks.workunit.client.0.smithi119.stderr:+ echo 3 2024-03-28T00:28:46.130 INFO:tasks.workunit.client.0.smithi119.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-03-28T00:28:46.568 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:28:46 smithi119 ceph-mon[31518]: pgmap v69: 97 pgs: 97 active+clean; 2.0 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 24 KiB/s wr, 20 op/s 2024-03-28T00:28:46.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:28:46 smithi165 ceph-mon[36302]: pgmap v69: 97 pgs: 97 active+clean; 2.0 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 24 KiB/s wr, 20 op/s 2024-03-28T00:28:46.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:28:46 smithi132 ceph-mon[35941]: pgmap v69: 97 pgs: 97 active+clean; 2.0 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 24 KiB/s wr, 20 op/s 2024-03-28T00:28:46.778 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:28:46.779 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:28:46.962 INFO:tasks.workunit.client.0.smithi119.stderr:+ c2=2 2024-03-28T00:28:46.963 INFO:tasks.workunit.client.0.smithi119.stderr:+ rmdir d1 2024-03-28T00:28:46.977 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:28:46.977 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:28:46.977 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 47 -lt 100 ']' 2024-03-28T00:28:46.979 INFO:tasks.workunit.client.0.smithi119.stderr:++ expr 47 + 1 2024-03-28T00:28:47.035 INFO:tasks.workunit.client.0.smithi119.stderr:+ c=48 2024-03-28T00:28:47.035 INFO:tasks.workunit.client.0.smithi119.stderr:+ mkdir d1 2024-03-28T00:28:47.079 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:28:47.079 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:28:47.083 INFO:tasks.workunit.client.0.smithi119.stderr:+ c1=2 2024-03-28T00:28:47.085 INFO:tasks.workunit.client.0.smithi119.stderr:+ echo 3 2024-03-28T00:28:47.085 INFO:tasks.workunit.client.0.smithi119.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-03-28T00:28:47.920 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:28:47.920 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:28:48.289 INFO:tasks.workunit.client.0.smithi119.stderr:+ c2=2 2024-03-28T00:28:48.289 INFO:tasks.workunit.client.0.smithi119.stderr:+ rmdir d1 2024-03-28T00:28:48.320 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:28:48.320 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:28:48.321 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 48 -lt 100 ']' 2024-03-28T00:28:48.322 INFO:tasks.workunit.client.0.smithi119.stderr:++ expr 48 + 1 2024-03-28T00:28:48.377 INFO:tasks.workunit.client.0.smithi119.stderr:+ c=49 2024-03-28T00:28:48.377 INFO:tasks.workunit.client.0.smithi119.stderr:+ mkdir d1 2024-03-28T00:28:48.446 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:28:48.446 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:28:48.450 INFO:tasks.workunit.client.0.smithi119.stderr:+ c1=2 2024-03-28T00:28:48.451 INFO:tasks.workunit.client.0.smithi119.stderr:+ echo 3 2024-03-28T00:28:48.452 INFO:tasks.workunit.client.0.smithi119.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-03-28T00:28:48.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:28:48 smithi165 ceph-mon[36302]: pgmap v70: 97 pgs: 97 active+clean; 2.0 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 KiB/s wr, 13 op/s 2024-03-28T00:28:48.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:28:48 smithi119 ceph-mon[31518]: pgmap v70: 97 pgs: 97 active+clean; 2.0 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 KiB/s wr, 13 op/s 2024-03-28T00:28:48.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:28:48 smithi132 ceph-mon[35941]: pgmap v70: 97 pgs: 97 active+clean; 2.0 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 KiB/s wr, 13 op/s 2024-03-28T00:28:49.261 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:28:49.262 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:28:49.900 INFO:tasks.workunit.client.0.smithi119.stderr:+ c2=2 2024-03-28T00:28:49.900 INFO:tasks.workunit.client.0.smithi119.stderr:+ rmdir d1 2024-03-28T00:28:49.930 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:28:49.930 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:28:49.930 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 49 -lt 100 ']' 2024-03-28T00:28:49.932 INFO:tasks.workunit.client.0.smithi119.stderr:++ expr 49 + 1 2024-03-28T00:28:50.043 INFO:tasks.workunit.client.0.smithi119.stderr:+ c=50 2024-03-28T00:28:50.043 INFO:tasks.workunit.client.0.smithi119.stderr:+ mkdir d1 2024-03-28T00:28:50.093 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:28:50.093 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:28:50.098 INFO:tasks.workunit.client.0.smithi119.stderr:+ c1=2 2024-03-28T00:28:50.099 INFO:tasks.workunit.client.0.smithi119.stderr:+ echo 3 2024-03-28T00:28:50.099 INFO:tasks.workunit.client.0.smithi119.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-03-28T00:28:50.579 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:28:50 smithi119 ceph-mon[31518]: pgmap v71: 97 pgs: 97 active+clean; 2.1 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 25 KiB/s wr, 20 op/s 2024-03-28T00:28:50.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:28:50 smithi165 ceph-mon[36302]: pgmap v71: 97 pgs: 97 active+clean; 2.1 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 25 KiB/s wr, 20 op/s 2024-03-28T00:28:50.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:28:50 smithi132 ceph-mon[35941]: pgmap v71: 97 pgs: 97 active+clean; 2.1 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 25 KiB/s wr, 20 op/s 2024-03-28T00:28:50.886 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:28:50.886 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:28:50.957 INFO:tasks.workunit.client.0.smithi119.stderr:+ c2=2 2024-03-28T00:28:50.957 INFO:tasks.workunit.client.0.smithi119.stderr:+ rmdir d1 2024-03-28T00:28:50.968 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:28:50.968 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:28:50.969 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 50 -lt 100 ']' 2024-03-28T00:28:50.969 INFO:tasks.workunit.client.0.smithi119.stderr:++ expr 50 + 1 2024-03-28T00:28:50.993 INFO:tasks.workunit.client.0.smithi119.stderr:+ c=51 2024-03-28T00:28:50.993 INFO:tasks.workunit.client.0.smithi119.stderr:+ mkdir d1 2024-03-28T00:28:51.035 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:28:51.036 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:28:51.040 INFO:tasks.workunit.client.0.smithi119.stderr:+ c1=2 2024-03-28T00:28:51.041 INFO:tasks.workunit.client.0.smithi119.stderr:+ echo 3 2024-03-28T00:28:51.041 INFO:tasks.workunit.client.0.smithi119.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-03-28T00:28:51.544 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:28:51.544 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:28:51.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:28:51 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:28:51.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:28:51 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:28:51.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:28:51 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:28:51.656 INFO:tasks.workunit.client.0.smithi119.stderr:+ c2=2 2024-03-28T00:28:51.719 INFO:tasks.workunit.client.0.smithi119.stderr:+ rmdir d1 2024-03-28T00:28:51.720 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:28:51 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:28:51.720 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:28:51 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:28:51.720 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:28:51 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:28:51.720 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:28:51 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:28:51.720 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:28:51 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:28:51.721 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:28:51 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:28:51.721 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:28:51.721 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:28:51.721 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 51 -lt 100 ']' 2024-03-28T00:28:51.721 INFO:tasks.workunit.client.0.smithi119.stderr:++ expr 51 + 1 2024-03-28T00:28:51.721 INFO:tasks.workunit.client.0.smithi119.stderr:+ c=52 2024-03-28T00:28:51.722 INFO:tasks.workunit.client.0.smithi119.stderr:+ mkdir d1 2024-03-28T00:28:51.734 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:28:51.734 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:28:51.735 INFO:tasks.workunit.client.0.smithi119.stderr:+ c1=2 2024-03-28T00:28:51.736 INFO:tasks.workunit.client.0.smithi119.stderr:+ echo 3 2024-03-28T00:28:51.736 INFO:tasks.workunit.client.0.smithi119.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-03-28T00:28:52.304 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:28:52.304 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:28:52.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:28:52 smithi119 ceph-mon[31518]: pgmap v72: 97 pgs: 97 active+clean; 2.1 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-03-28T00:28:52.474 INFO:tasks.workunit.client.0.smithi119.stderr:+ c2=2 2024-03-28T00:28:52.474 INFO:tasks.workunit.client.0.smithi119.stderr:+ rmdir d1 2024-03-28T00:28:52.503 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:28:52.503 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:28:52.503 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 52 -lt 100 ']' 2024-03-28T00:28:52.504 INFO:tasks.workunit.client.0.smithi119.stderr:++ expr 52 + 1 2024-03-28T00:28:52.550 INFO:tasks.workunit.client.0.smithi119.stderr:+ c=53 2024-03-28T00:28:52.550 INFO:tasks.workunit.client.0.smithi119.stderr:+ mkdir d1 2024-03-28T00:28:52.578 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:28:52.578 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:28:52.582 INFO:tasks.workunit.client.0.smithi119.stderr:+ c1=2 2024-03-28T00:28:52.583 INFO:tasks.workunit.client.0.smithi119.stderr:+ echo 3 2024-03-28T00:28:52.584 INFO:tasks.workunit.client.0.smithi119.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-03-28T00:28:52.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:28:52 smithi165 ceph-mon[36302]: pgmap v72: 97 pgs: 97 active+clean; 2.1 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-03-28T00:28:52.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:28:52 smithi132 ceph-mon[35941]: pgmap v72: 97 pgs: 97 active+clean; 2.1 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-03-28T00:28:53.012 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:28:53.013 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:28:53.177 INFO:tasks.workunit.client.0.smithi119.stderr:+ c2=2 2024-03-28T00:28:53.177 INFO:tasks.workunit.client.0.smithi119.stderr:+ rmdir d1 2024-03-28T00:28:53.193 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:28:53.193 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:28:53.194 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 53 -lt 100 ']' 2024-03-28T00:28:53.195 INFO:tasks.workunit.client.0.smithi119.stderr:++ expr 53 + 1 2024-03-28T00:28:53.233 INFO:tasks.workunit.client.0.smithi119.stderr:+ c=54 2024-03-28T00:28:53.233 INFO:tasks.workunit.client.0.smithi119.stderr:+ mkdir d1 2024-03-28T00:28:53.252 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:28:53.252 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:28:53.257 INFO:tasks.workunit.client.0.smithi119.stderr:+ c1=2 2024-03-28T00:28:53.258 INFO:tasks.workunit.client.0.smithi119.stderr:+ echo 3 2024-03-28T00:28:53.259 INFO:tasks.workunit.client.0.smithi119.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-03-28T00:28:53.745 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:28:53.746 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:28:53.899 INFO:tasks.workunit.client.0.smithi119.stderr:+ c2=2 2024-03-28T00:28:53.899 INFO:tasks.workunit.client.0.smithi119.stderr:+ rmdir d1 2024-03-28T00:28:53.911 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:28:53.912 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:28:53.912 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 54 -lt 100 ']' 2024-03-28T00:28:53.913 INFO:tasks.workunit.client.0.smithi119.stderr:++ expr 54 + 1 2024-03-28T00:28:53.942 INFO:tasks.workunit.client.0.smithi119.stderr:+ c=55 2024-03-28T00:28:53.942 INFO:tasks.workunit.client.0.smithi119.stderr:+ mkdir d1 2024-03-28T00:28:53.960 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:28:53.961 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:28:53.966 INFO:tasks.workunit.client.0.smithi119.stderr:+ c1=2 2024-03-28T00:28:53.967 INFO:tasks.workunit.client.0.smithi119.stderr:+ echo 3 2024-03-28T00:28:53.968 INFO:tasks.workunit.client.0.smithi119.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-03-28T00:28:54.537 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:28:54.537 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:28:54.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:28:54 smithi165 ceph-mon[36302]: pgmap v73: 97 pgs: 97 active+clean; 2.1 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-03-28T00:28:54.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:28:54 smithi119 ceph-mon[31518]: pgmap v73: 97 pgs: 97 active+clean; 2.1 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-03-28T00:28:54.698 INFO:tasks.workunit.client.0.smithi119.stderr:+ c2=2 2024-03-28T00:28:54.699 INFO:tasks.workunit.client.0.smithi119.stderr:+ rmdir d1 2024-03-28T00:28:54.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:28:54 smithi132 ceph-mon[35941]: pgmap v73: 97 pgs: 97 active+clean; 2.1 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-03-28T00:28:54.710 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:28:54.710 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:28:54.710 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 55 -lt 100 ']' 2024-03-28T00:28:54.711 INFO:tasks.workunit.client.0.smithi119.stderr:++ expr 55 + 1 2024-03-28T00:28:54.741 INFO:tasks.workunit.client.0.smithi119.stderr:+ c=56 2024-03-28T00:28:54.741 INFO:tasks.workunit.client.0.smithi119.stderr:+ mkdir d1 2024-03-28T00:28:54.760 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:28:54.760 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:28:54.764 INFO:tasks.workunit.client.0.smithi119.stderr:+ c1=2 2024-03-28T00:28:54.766 INFO:tasks.workunit.client.0.smithi119.stderr:+ echo 3 2024-03-28T00:28:54.767 INFO:tasks.workunit.client.0.smithi119.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-03-28T00:28:55.254 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:28:55.254 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:28:55.507 INFO:tasks.workunit.client.0.smithi119.stderr:+ c2=2 2024-03-28T00:28:55.507 INFO:tasks.workunit.client.0.smithi119.stderr:+ rmdir d1 2024-03-28T00:28:55.518 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:28:55.518 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:28:55.518 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 56 -lt 100 ']' 2024-03-28T00:28:55.519 INFO:tasks.workunit.client.0.smithi119.stderr:++ expr 56 + 1 2024-03-28T00:28:55.549 INFO:tasks.workunit.client.0.smithi119.stderr:+ c=57 2024-03-28T00:28:55.549 INFO:tasks.workunit.client.0.smithi119.stderr:+ mkdir d1 2024-03-28T00:28:55.579 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:28:55.580 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:28:55.584 INFO:tasks.workunit.client.0.smithi119.stderr:+ c1=2 2024-03-28T00:28:55.585 INFO:tasks.workunit.client.0.smithi119.stderr:+ echo 3 2024-03-28T00:28:55.585 INFO:tasks.workunit.client.0.smithi119.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-03-28T00:28:56.145 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:28:56.145 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:28:56.385 INFO:tasks.workunit.client.0.smithi119.stderr:+ c2=2 2024-03-28T00:28:56.385 INFO:tasks.workunit.client.0.smithi119.stderr:+ rmdir d1 2024-03-28T00:28:56.401 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:28:56.401 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:28:56.401 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 57 -lt 100 ']' 2024-03-28T00:28:56.402 INFO:tasks.workunit.client.0.smithi119.stderr:++ expr 57 + 1 2024-03-28T00:28:56.441 INFO:tasks.workunit.client.0.smithi119.stderr:+ c=58 2024-03-28T00:28:56.441 INFO:tasks.workunit.client.0.smithi119.stderr:+ mkdir d1 2024-03-28T00:28:56.469 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:28:56.469 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:28:56.473 INFO:tasks.workunit.client.0.smithi119.stderr:+ c1=2 2024-03-28T00:28:56.474 INFO:tasks.workunit.client.0.smithi119.stderr:+ echo 3 2024-03-28T00:28:56.475 INFO:tasks.workunit.client.0.smithi119.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-03-28T00:28:56.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:28:56 smithi165 ceph-mon[36302]: pgmap v74: 97 pgs: 97 active+clean; 2.2 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 27 KiB/s wr, 21 op/s 2024-03-28T00:28:56.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:28:56 smithi119 ceph-mon[31518]: pgmap v74: 97 pgs: 97 active+clean; 2.2 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 27 KiB/s wr, 21 op/s 2024-03-28T00:28:56.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:28:56 smithi132 ceph-mon[35941]: pgmap v74: 97 pgs: 97 active+clean; 2.2 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 27 KiB/s wr, 21 op/s 2024-03-28T00:28:57.118 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:28:57.119 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:28:57.273 INFO:tasks.workunit.client.0.smithi119.stderr:+ c2=2 2024-03-28T00:28:57.273 INFO:tasks.workunit.client.0.smithi119.stderr:+ rmdir d1 2024-03-28T00:28:57.285 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:28:57.285 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:28:57.285 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 58 -lt 100 ']' 2024-03-28T00:28:57.286 INFO:tasks.workunit.client.0.smithi119.stderr:++ expr 58 + 1 2024-03-28T00:28:57.316 INFO:tasks.workunit.client.0.smithi119.stderr:+ c=59 2024-03-28T00:28:57.316 INFO:tasks.workunit.client.0.smithi119.stderr:+ mkdir d1 2024-03-28T00:28:57.343 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:28:57.344 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:28:57.349 INFO:tasks.workunit.client.0.smithi119.stderr:+ c1=2 2024-03-28T00:28:57.350 INFO:tasks.workunit.client.0.smithi119.stderr:+ echo 3 2024-03-28T00:28:57.351 INFO:tasks.workunit.client.0.smithi119.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-03-28T00:28:57.836 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:28:57.836 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:28:58.006 INFO:tasks.workunit.client.0.smithi119.stderr:+ c2=2 2024-03-28T00:28:58.007 INFO:tasks.workunit.client.0.smithi119.stderr:+ rmdir d1 2024-03-28T00:28:58.018 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:28:58.018 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:28:58.018 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 59 -lt 100 ']' 2024-03-28T00:28:58.020 INFO:tasks.workunit.client.0.smithi119.stderr:++ expr 59 + 1 2024-03-28T00:28:58.057 INFO:tasks.workunit.client.0.smithi119.stderr:+ c=60 2024-03-28T00:28:58.057 INFO:tasks.workunit.client.0.smithi119.stderr:+ mkdir d1 2024-03-28T00:28:58.077 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:28:58.077 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:28:58.081 INFO:tasks.workunit.client.0.smithi119.stderr:+ c1=2 2024-03-28T00:28:58.082 INFO:tasks.workunit.client.0.smithi119.stderr:+ echo 3 2024-03-28T00:28:58.083 INFO:tasks.workunit.client.0.smithi119.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-03-28T00:28:58.617 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:28:58 smithi119 ceph-mon[31518]: pgmap v75: 97 pgs: 97 active+clean; 2.2 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-03-28T00:28:58.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:28:58 smithi165 ceph-mon[36302]: pgmap v75: 97 pgs: 97 active+clean; 2.2 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-03-28T00:28:58.662 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:28:58.662 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:28:58.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:28:58 smithi132 ceph-mon[35941]: pgmap v75: 97 pgs: 97 active+clean; 2.2 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-03-28T00:28:58.831 INFO:tasks.workunit.client.0.smithi119.stderr:+ c2=2 2024-03-28T00:28:58.831 INFO:tasks.workunit.client.0.smithi119.stderr:+ rmdir d1 2024-03-28T00:28:58.844 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:28:58.844 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:28:58.844 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 60 -lt 100 ']' 2024-03-28T00:28:58.846 INFO:tasks.workunit.client.0.smithi119.stderr:++ expr 60 + 1 2024-03-28T00:28:58.907 INFO:tasks.workunit.client.0.smithi119.stderr:+ c=61 2024-03-28T00:28:58.907 INFO:tasks.workunit.client.0.smithi119.stderr:+ mkdir d1 2024-03-28T00:28:58.968 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:28:58.969 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:28:58.973 INFO:tasks.workunit.client.0.smithi119.stderr:+ c1=2 2024-03-28T00:28:58.975 INFO:tasks.workunit.client.0.smithi119.stderr:+ echo 3 2024-03-28T00:28:58.976 INFO:tasks.workunit.client.0.smithi119.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-03-28T00:28:59.511 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:28:59.511 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:28:59.689 INFO:tasks.workunit.client.0.smithi119.stderr:+ c2=2 2024-03-28T00:28:59.689 INFO:tasks.workunit.client.0.smithi119.stderr:+ rmdir d1 2024-03-28T00:28:59.701 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:28:59.701 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:28:59.701 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 61 -lt 100 ']' 2024-03-28T00:28:59.702 INFO:tasks.workunit.client.0.smithi119.stderr:++ expr 61 + 1 2024-03-28T00:28:59.750 INFO:tasks.workunit.client.0.smithi119.stderr:+ c=62 2024-03-28T00:28:59.750 INFO:tasks.workunit.client.0.smithi119.stderr:+ mkdir d1 2024-03-28T00:28:59.775 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:28:59.776 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:28:59.780 INFO:tasks.workunit.client.0.smithi119.stderr:+ c1=2 2024-03-28T00:28:59.781 INFO:tasks.workunit.client.0.smithi119.stderr:+ echo 3 2024-03-28T00:28:59.782 INFO:tasks.workunit.client.0.smithi119.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-03-28T00:29:00.261 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:29:00.261 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:29:00.422 INFO:tasks.workunit.client.0.smithi119.stderr:+ c2=2 2024-03-28T00:29:00.422 INFO:tasks.workunit.client.0.smithi119.stderr:+ rmdir d1 2024-03-28T00:29:00.445 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:29:00.445 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:29:00.445 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 62 -lt 100 ']' 2024-03-28T00:29:00.446 INFO:tasks.workunit.client.0.smithi119.stderr:++ expr 62 + 1 2024-03-28T00:29:00.542 INFO:tasks.workunit.client.0.smithi119.stderr:+ c=63 2024-03-28T00:29:00.542 INFO:tasks.workunit.client.0.smithi119.stderr:+ mkdir d1 2024-03-28T00:29:00.598 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:29:00.599 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:29:00.603 INFO:tasks.workunit.client.0.smithi119.stderr:+ c1=2 2024-03-28T00:29:00.604 INFO:tasks.workunit.client.0.smithi119.stderr:+ echo 3 2024-03-28T00:29:00.605 INFO:tasks.workunit.client.0.smithi119.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-03-28T00:29:00.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:29:00 smithi165 ceph-mon[36302]: pgmap v76: 97 pgs: 97 active+clean; 2.3 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 26 KiB/s wr, 20 op/s 2024-03-28T00:29:00.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:29:00 smithi119 ceph-mon[31518]: pgmap v76: 97 pgs: 97 active+clean; 2.3 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 26 KiB/s wr, 20 op/s 2024-03-28T00:29:00.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:29:00 smithi132 ceph-mon[35941]: pgmap v76: 97 pgs: 97 active+clean; 2.3 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 26 KiB/s wr, 20 op/s 2024-03-28T00:29:01.203 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:29:01.203 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:29:01.364 INFO:tasks.workunit.client.0.smithi119.stderr:+ c2=2 2024-03-28T00:29:01.364 INFO:tasks.workunit.client.0.smithi119.stderr:+ rmdir d1 2024-03-28T00:29:01.374 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:29:01.374 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:29:01.375 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 63 -lt 100 ']' 2024-03-28T00:29:01.376 INFO:tasks.workunit.client.0.smithi119.stderr:++ expr 63 + 1 2024-03-28T00:29:01.407 INFO:tasks.workunit.client.0.smithi119.stderr:+ c=64 2024-03-28T00:29:01.407 INFO:tasks.workunit.client.0.smithi119.stderr:+ mkdir d1 2024-03-28T00:29:01.434 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:29:01.434 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:29:01.438 INFO:tasks.workunit.client.0.smithi119.stderr:+ c1=2 2024-03-28T00:29:01.440 INFO:tasks.workunit.client.0.smithi119.stderr:+ echo 3 2024-03-28T00:29:01.441 INFO:tasks.workunit.client.0.smithi119.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-03-28T00:29:02.029 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:29:02.029 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:29:02.198 INFO:tasks.workunit.client.0.smithi119.stderr:+ c2=2 2024-03-28T00:29:02.198 INFO:tasks.workunit.client.0.smithi119.stderr:+ rmdir d1 2024-03-28T00:29:02.218 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:29:02.218 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:29:02.218 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 64 -lt 100 ']' 2024-03-28T00:29:02.220 INFO:tasks.workunit.client.0.smithi119.stderr:++ expr 64 + 1 2024-03-28T00:29:02.249 INFO:tasks.workunit.client.0.smithi119.stderr:+ c=65 2024-03-28T00:29:02.249 INFO:tasks.workunit.client.0.smithi119.stderr:+ mkdir d1 2024-03-28T00:29:02.277 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:29:02.278 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:29:02.282 INFO:tasks.workunit.client.0.smithi119.stderr:+ c1=2 2024-03-28T00:29:02.283 INFO:tasks.workunit.client.0.smithi119.stderr:+ echo 3 2024-03-28T00:29:02.283 INFO:tasks.workunit.client.0.smithi119.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-03-28T00:29:02.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:29:02 smithi165 ceph-mon[36302]: pgmap v77: 97 pgs: 97 active+clean; 2.3 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 14 op/s 2024-03-28T00:29:02.648 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:29:02 smithi119 ceph-mon[31518]: pgmap v77: 97 pgs: 97 active+clean; 2.3 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 14 op/s 2024-03-28T00:29:02.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:29:02 smithi132 ceph-mon[35941]: pgmap v77: 97 pgs: 97 active+clean; 2.3 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 14 op/s 2024-03-28T00:29:02.744 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:29:02.744 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:29:02.898 INFO:tasks.workunit.client.0.smithi119.stderr:+ c2=2 2024-03-28T00:29:02.898 INFO:tasks.workunit.client.0.smithi119.stderr:+ rmdir d1 2024-03-28T00:29:02.916 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:29:02.916 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:29:02.916 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 65 -lt 100 ']' 2024-03-28T00:29:02.918 INFO:tasks.workunit.client.0.smithi119.stderr:++ expr 65 + 1 2024-03-28T00:29:02.948 INFO:tasks.workunit.client.0.smithi119.stderr:+ c=66 2024-03-28T00:29:02.948 INFO:tasks.workunit.client.0.smithi119.stderr:+ mkdir d1 2024-03-28T00:29:02.976 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:29:02.976 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:29:02.980 INFO:tasks.workunit.client.0.smithi119.stderr:+ c1=2 2024-03-28T00:29:02.981 INFO:tasks.workunit.client.0.smithi119.stderr:+ echo 3 2024-03-28T00:29:02.981 INFO:tasks.workunit.client.0.smithi119.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-03-28T00:29:03.586 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:29:03.586 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:29:03.764 INFO:tasks.workunit.client.0.smithi119.stderr:+ c2=2 2024-03-28T00:29:03.764 INFO:tasks.workunit.client.0.smithi119.stderr:+ rmdir d1 2024-03-28T00:29:03.776 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:29:03.776 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:29:03.776 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 66 -lt 100 ']' 2024-03-28T00:29:03.777 INFO:tasks.workunit.client.0.smithi119.stderr:++ expr 66 + 1 2024-03-28T00:29:03.806 INFO:tasks.workunit.client.0.smithi119.stderr:+ c=67 2024-03-28T00:29:03.806 INFO:tasks.workunit.client.0.smithi119.stderr:+ mkdir d1 2024-03-28T00:29:03.836 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:29:03.836 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:29:03.841 INFO:tasks.workunit.client.0.smithi119.stderr:+ c1=2 2024-03-28T00:29:03.842 INFO:tasks.workunit.client.0.smithi119.stderr:+ echo 3 2024-03-28T00:29:03.843 INFO:tasks.workunit.client.0.smithi119.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-03-28T00:29:04.335 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:29:04.335 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:29:04.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:29:04 smithi119 ceph-mon[31518]: pgmap v78: 97 pgs: 97 active+clean; 2.3 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 14 op/s 2024-03-28T00:29:04.497 INFO:tasks.workunit.client.0.smithi119.stderr:+ c2=2 2024-03-28T00:29:04.497 INFO:tasks.workunit.client.0.smithi119.stderr:+ rmdir d1 2024-03-28T00:29:04.509 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:29:04.509 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:29:04.509 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 67 -lt 100 ']' 2024-03-28T00:29:04.510 INFO:tasks.workunit.client.0.smithi119.stderr:++ expr 67 + 1 2024-03-28T00:29:04.540 INFO:tasks.workunit.client.0.smithi119.stderr:+ c=68 2024-03-28T00:29:04.540 INFO:tasks.workunit.client.0.smithi119.stderr:+ mkdir d1 2024-03-28T00:29:04.560 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:29:04.560 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:29:04.565 INFO:tasks.workunit.client.0.smithi119.stderr:+ c1=2 2024-03-28T00:29:04.566 INFO:tasks.workunit.client.0.smithi119.stderr:+ echo 3 2024-03-28T00:29:04.566 INFO:tasks.workunit.client.0.smithi119.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-03-28T00:29:04.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:29:04 smithi165 ceph-mon[36302]: pgmap v78: 97 pgs: 97 active+clean; 2.3 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 14 op/s 2024-03-28T00:29:04.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:29:04 smithi132 ceph-mon[35941]: pgmap v78: 97 pgs: 97 active+clean; 2.3 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 14 op/s 2024-03-28T00:29:05.076 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:29:05.077 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:29:05.242 INFO:tasks.workunit.client.0.smithi119.stderr:+ c2=2 2024-03-28T00:29:05.242 INFO:tasks.workunit.client.0.smithi119.stderr:+ rmdir d1 2024-03-28T00:29:05.257 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:29:05.257 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:29:05.257 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 68 -lt 100 ']' 2024-03-28T00:29:05.258 INFO:tasks.workunit.client.0.smithi119.stderr:++ expr 68 + 1 2024-03-28T00:29:05.290 INFO:tasks.workunit.client.0.smithi119.stderr:+ c=69 2024-03-28T00:29:05.290 INFO:tasks.workunit.client.0.smithi119.stderr:+ mkdir d1 2024-03-28T00:29:05.321 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:29:05.322 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:29:05.323 INFO:tasks.workunit.client.0.smithi119.stderr:+ c1=2 2024-03-28T00:29:05.325 INFO:tasks.workunit.client.0.smithi119.stderr:+ echo 3 2024-03-28T00:29:05.325 INFO:tasks.workunit.client.0.smithi119.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-03-28T00:29:05.927 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:29:05.927 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:29:06.100 INFO:tasks.workunit.client.0.smithi119.stderr:+ c2=2 2024-03-28T00:29:06.100 INFO:tasks.workunit.client.0.smithi119.stderr:+ rmdir d1 2024-03-28T00:29:06.117 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:29:06.117 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:29:06.117 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 69 -lt 100 ']' 2024-03-28T00:29:06.118 INFO:tasks.workunit.client.0.smithi119.stderr:++ expr 69 + 1 2024-03-28T00:29:06.148 INFO:tasks.workunit.client.0.smithi119.stderr:+ c=70 2024-03-28T00:29:06.148 INFO:tasks.workunit.client.0.smithi119.stderr:+ mkdir d1 2024-03-28T00:29:06.184 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:29:06.185 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:29:06.190 INFO:tasks.workunit.client.0.smithi119.stderr:+ c1=2 2024-03-28T00:29:06.191 INFO:tasks.workunit.client.0.smithi119.stderr:+ echo 3 2024-03-28T00:29:06.192 INFO:tasks.workunit.client.0.smithi119.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-03-28T00:29:06.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:29:06 smithi165 ceph-mon[36302]: pgmap v79: 97 pgs: 97 active+clean; 2.4 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 50 KiB/s wr, 36 op/s 2024-03-28T00:29:06.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:29:06 smithi119 ceph-mon[31518]: pgmap v79: 97 pgs: 97 active+clean; 2.4 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 50 KiB/s wr, 36 op/s 2024-03-28T00:29:06.685 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:29:06.685 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:29:06.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:29:06 smithi132 ceph-mon[35941]: pgmap v79: 97 pgs: 97 active+clean; 2.4 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 50 KiB/s wr, 36 op/s 2024-03-28T00:29:06.888 INFO:tasks.workunit.client.0.smithi119.stderr:+ c2=2 2024-03-28T00:29:06.888 INFO:tasks.workunit.client.0.smithi119.stderr:+ rmdir d1 2024-03-28T00:29:06.902 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:29:06.902 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:29:06.902 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 70 -lt 100 ']' 2024-03-28T00:29:06.903 INFO:tasks.workunit.client.0.smithi119.stderr:++ expr 70 + 1 2024-03-28T00:29:06.948 INFO:tasks.workunit.client.0.smithi119.stderr:+ c=71 2024-03-28T00:29:06.948 INFO:tasks.workunit.client.0.smithi119.stderr:+ mkdir d1 2024-03-28T00:29:06.977 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:29:06.977 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:29:06.982 INFO:tasks.workunit.client.0.smithi119.stderr:+ c1=2 2024-03-28T00:29:06.983 INFO:tasks.workunit.client.0.smithi119.stderr:+ echo 3 2024-03-28T00:29:06.985 INFO:tasks.workunit.client.0.smithi119.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-03-28T00:29:07.526 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:29:07.527 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:29:07.707 INFO:tasks.workunit.client.0.smithi119.stderr:+ c2=2 2024-03-28T00:29:07.707 INFO:tasks.workunit.client.0.smithi119.stderr:+ rmdir d1 2024-03-28T00:29:07.725 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:29:07.725 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:29:07.725 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 71 -lt 100 ']' 2024-03-28T00:29:07.727 INFO:tasks.workunit.client.0.smithi119.stderr:++ expr 71 + 1 2024-03-28T00:29:07.764 INFO:tasks.workunit.client.0.smithi119.stderr:+ c=72 2024-03-28T00:29:07.764 INFO:tasks.workunit.client.0.smithi119.stderr:+ mkdir d1 2024-03-28T00:29:07.792 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:29:07.792 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:29:07.796 INFO:tasks.workunit.client.0.smithi119.stderr:+ c1=2 2024-03-28T00:29:07.797 INFO:tasks.workunit.client.0.smithi119.stderr:+ echo 3 2024-03-28T00:29:07.798 INFO:tasks.workunit.client.0.smithi119.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-03-28T00:29:08.276 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:29:08.277 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:29:08.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:29:08 smithi165 ceph-mon[36302]: pgmap v80: 97 pgs: 97 active+clean; 2.4 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 40 KiB/s wr, 28 op/s 2024-03-28T00:29:08.638 INFO:tasks.workunit.client.0.smithi119.stderr:+ c2=2 2024-03-28T00:29:08.638 INFO:tasks.workunit.client.0.smithi119.stderr:+ rmdir d1 2024-03-28T00:29:08.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:29:08 smithi119 ceph-mon[31518]: pgmap v80: 97 pgs: 97 active+clean; 2.4 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 40 KiB/s wr, 28 op/s 2024-03-28T00:29:08.658 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:29:08.658 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:29:08.658 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 72 -lt 100 ']' 2024-03-28T00:29:08.660 INFO:tasks.workunit.client.0.smithi119.stderr:++ expr 72 + 1 2024-03-28T00:29:08.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:29:08 smithi132 ceph-mon[35941]: pgmap v80: 97 pgs: 97 active+clean; 2.4 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 40 KiB/s wr, 28 op/s 2024-03-28T00:29:08.705 INFO:tasks.workunit.client.0.smithi119.stderr:+ c=73 2024-03-28T00:29:08.706 INFO:tasks.workunit.client.0.smithi119.stderr:+ mkdir d1 2024-03-28T00:29:08.734 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:29:08.734 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:29:08.739 INFO:tasks.workunit.client.0.smithi119.stderr:+ c1=2 2024-03-28T00:29:08.740 INFO:tasks.workunit.client.0.smithi119.stderr:+ echo 3 2024-03-28T00:29:08.741 INFO:tasks.workunit.client.0.smithi119.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-03-28T00:29:09.351 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:29:09.351 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:29:09.539 INFO:tasks.workunit.client.0.smithi119.stderr:+ c2=2 2024-03-28T00:29:09.539 INFO:tasks.workunit.client.0.smithi119.stderr:+ rmdir d1 2024-03-28T00:29:09.557 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:29:09.557 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:29:09.557 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 73 -lt 100 ']' 2024-03-28T00:29:09.558 INFO:tasks.workunit.client.0.smithi119.stderr:++ expr 73 + 1 2024-03-28T00:29:09.589 INFO:tasks.workunit.client.0.smithi119.stderr:+ c=74 2024-03-28T00:29:09.589 INFO:tasks.workunit.client.0.smithi119.stderr:+ mkdir d1 2024-03-28T00:29:09.617 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:29:09.618 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:29:09.623 INFO:tasks.workunit.client.0.smithi119.stderr:+ c1=2 2024-03-28T00:29:09.626 INFO:tasks.workunit.client.0.smithi119.stderr:+ echo 3 2024-03-28T00:29:09.626 INFO:tasks.workunit.client.0.smithi119.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-03-28T00:29:10.201 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:29:10.201 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:29:10.354 INFO:tasks.workunit.client.0.smithi119.stderr:+ c2=2 2024-03-28T00:29:10.354 INFO:tasks.workunit.client.0.smithi119.stderr:+ rmdir d1 2024-03-28T00:29:10.363 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:29:10.363 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:29:10.364 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 74 -lt 100 ']' 2024-03-28T00:29:10.366 INFO:tasks.workunit.client.0.smithi119.stderr:++ expr 74 + 1 2024-03-28T00:29:10.397 INFO:tasks.workunit.client.0.smithi119.stderr:+ c=75 2024-03-28T00:29:10.397 INFO:tasks.workunit.client.0.smithi119.stderr:+ mkdir d1 2024-03-28T00:29:10.425 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:29:10.426 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:29:10.430 INFO:tasks.workunit.client.0.smithi119.stderr:+ c1=2 2024-03-28T00:29:10.431 INFO:tasks.workunit.client.0.smithi119.stderr:+ echo 3 2024-03-28T00:29:10.432 INFO:tasks.workunit.client.0.smithi119.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-03-28T00:29:10.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:29:10 smithi165 ceph-mon[36302]: pgmap v81: 97 pgs: 97 active+clean; 2.5 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 50 KiB/s wr, 35 op/s 2024-03-28T00:29:10.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:29:10 smithi119 ceph-mon[31518]: pgmap v81: 97 pgs: 97 active+clean; 2.5 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 50 KiB/s wr, 35 op/s 2024-03-28T00:29:10.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:29:10 smithi132 ceph-mon[35941]: pgmap v81: 97 pgs: 97 active+clean; 2.5 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 50 KiB/s wr, 35 op/s 2024-03-28T00:29:11.001 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:29:11.001 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:29:11.229 INFO:tasks.workunit.client.0.smithi119.stderr:+ c2=2 2024-03-28T00:29:11.229 INFO:tasks.workunit.client.0.smithi119.stderr:+ rmdir d1 2024-03-28T00:29:11.240 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:29:11.241 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:29:11.241 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 75 -lt 100 ']' 2024-03-28T00:29:11.242 INFO:tasks.workunit.client.0.smithi119.stderr:++ expr 75 + 1 2024-03-28T00:29:11.272 INFO:tasks.workunit.client.0.smithi119.stderr:+ c=76 2024-03-28T00:29:11.272 INFO:tasks.workunit.client.0.smithi119.stderr:+ mkdir d1 2024-03-28T00:29:11.300 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:29:11.301 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:29:11.306 INFO:tasks.workunit.client.0.smithi119.stderr:+ c1=2 2024-03-28T00:29:11.307 INFO:tasks.workunit.client.0.smithi119.stderr:+ echo 3 2024-03-28T00:29:11.307 INFO:tasks.workunit.client.0.smithi119.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-03-28T00:29:11.792 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:29:11.792 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:29:11.962 INFO:tasks.workunit.client.0.smithi119.stderr:+ c2=2 2024-03-28T00:29:11.962 INFO:tasks.workunit.client.0.smithi119.stderr:+ rmdir d1 2024-03-28T00:29:11.975 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:29:11.976 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:29:11.976 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 76 -lt 100 ']' 2024-03-28T00:29:11.977 INFO:tasks.workunit.client.0.smithi119.stderr:++ expr 76 + 1 2024-03-28T00:29:12.022 INFO:tasks.workunit.client.0.smithi119.stderr:+ c=77 2024-03-28T00:29:12.023 INFO:tasks.workunit.client.0.smithi119.stderr:+ mkdir d1 2024-03-28T00:29:12.049 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:29:12.050 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:29:12.054 INFO:tasks.workunit.client.0.smithi119.stderr:+ c1=2 2024-03-28T00:29:12.056 INFO:tasks.workunit.client.0.smithi119.stderr:+ echo 3 2024-03-28T00:29:12.057 INFO:tasks.workunit.client.0.smithi119.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-03-28T00:29:12.559 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:29:12.559 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:29:12.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:29:12 smithi165 ceph-mon[36302]: pgmap v82: 97 pgs: 97 active+clean; 2.5 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 42 KiB/s wr, 28 op/s 2024-03-28T00:29:12.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:29:12 smithi119 ceph-mon[31518]: pgmap v82: 97 pgs: 97 active+clean; 2.5 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 42 KiB/s wr, 28 op/s 2024-03-28T00:29:12.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:29:12 smithi132 ceph-mon[35941]: pgmap v82: 97 pgs: 97 active+clean; 2.5 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 42 KiB/s wr, 28 op/s 2024-03-28T00:29:12.729 INFO:tasks.workunit.client.0.smithi119.stderr:+ c2=2 2024-03-28T00:29:12.729 INFO:tasks.workunit.client.0.smithi119.stderr:+ rmdir d1 2024-03-28T00:29:12.741 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:29:12.741 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:29:12.741 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 77 -lt 100 ']' 2024-03-28T00:29:12.742 INFO:tasks.workunit.client.0.smithi119.stderr:++ expr 77 + 1 2024-03-28T00:29:12.772 INFO:tasks.workunit.client.0.smithi119.stderr:+ c=78 2024-03-28T00:29:12.772 INFO:tasks.workunit.client.0.smithi119.stderr:+ mkdir d1 2024-03-28T00:29:12.791 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:29:12.791 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:29:12.796 INFO:tasks.workunit.client.0.smithi119.stderr:+ c1=2 2024-03-28T00:29:12.797 INFO:tasks.workunit.client.0.smithi119.stderr:+ echo 3 2024-03-28T00:29:12.797 INFO:tasks.workunit.client.0.smithi119.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-03-28T00:29:13.384 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:29:13.384 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:29:13.564 INFO:tasks.workunit.client.0.smithi119.stderr:+ c2=2 2024-03-28T00:29:13.564 INFO:tasks.workunit.client.0.smithi119.stderr:+ rmdir d1 2024-03-28T00:29:13.587 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:29:13.587 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:29:13.587 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 78 -lt 100 ']' 2024-03-28T00:29:13.588 INFO:tasks.workunit.client.0.smithi119.stderr:++ expr 78 + 1 2024-03-28T00:29:13.622 INFO:tasks.workunit.client.0.smithi119.stderr:+ c=79 2024-03-28T00:29:13.622 INFO:tasks.workunit.client.0.smithi119.stderr:+ mkdir d1 2024-03-28T00:29:13.650 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:29:13.650 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:29:13.654 INFO:tasks.workunit.client.0.smithi119.stderr:+ c1=2 2024-03-28T00:29:13.655 INFO:tasks.workunit.client.0.smithi119.stderr:+ echo 3 2024-03-28T00:29:13.656 INFO:tasks.workunit.client.0.smithi119.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-03-28T00:29:14.201 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:29:14.201 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:29:14.404 INFO:tasks.workunit.client.0.smithi119.stderr:+ c2=2 2024-03-28T00:29:14.404 INFO:tasks.workunit.client.0.smithi119.stderr:+ rmdir d1 2024-03-28T00:29:14.416 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:29:14.416 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:29:14.416 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 79 -lt 100 ']' 2024-03-28T00:29:14.417 INFO:tasks.workunit.client.0.smithi119.stderr:++ expr 79 + 1 2024-03-28T00:29:14.463 INFO:tasks.workunit.client.0.smithi119.stderr:+ c=80 2024-03-28T00:29:14.464 INFO:tasks.workunit.client.0.smithi119.stderr:+ mkdir d1 2024-03-28T00:29:14.491 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:29:14.492 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:29:14.496 INFO:tasks.workunit.client.0.smithi119.stderr:+ c1=2 2024-03-28T00:29:14.498 INFO:tasks.workunit.client.0.smithi119.stderr:+ echo 3 2024-03-28T00:29:14.498 INFO:tasks.workunit.client.0.smithi119.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-03-28T00:29:14.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:29:14 smithi119 ceph-mon[31518]: pgmap v83: 97 pgs: 97 active+clean; 2.5 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 42 KiB/s wr, 28 op/s 2024-03-28T00:29:14.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:29:14 smithi132 ceph-mon[35941]: pgmap v83: 97 pgs: 97 active+clean; 2.5 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 42 KiB/s wr, 28 op/s 2024-03-28T00:29:14.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:29:14 smithi165 ceph-mon[36302]: pgmap v83: 97 pgs: 97 active+clean; 2.5 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 42 KiB/s wr, 28 op/s 2024-03-28T00:29:15.042 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:29:15.043 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:29:15.312 INFO:tasks.workunit.client.0.smithi119.stderr:+ c2=2 2024-03-28T00:29:15.312 INFO:tasks.workunit.client.0.smithi119.stderr:+ rmdir d1 2024-03-28T00:29:15.341 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:29:15.341 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:29:15.341 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 80 -lt 100 ']' 2024-03-28T00:29:15.342 INFO:tasks.workunit.client.0.smithi119.stderr:++ expr 80 + 1 2024-03-28T00:29:15.371 INFO:tasks.workunit.client.0.smithi119.stderr:+ c=81 2024-03-28T00:29:15.371 INFO:tasks.workunit.client.0.smithi119.stderr:+ mkdir d1 2024-03-28T00:29:15.393 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:29:15.393 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:29:15.398 INFO:tasks.workunit.client.0.smithi119.stderr:+ c1=2 2024-03-28T00:29:15.399 INFO:tasks.workunit.client.0.smithi119.stderr:+ echo 3 2024-03-28T00:29:15.399 INFO:tasks.workunit.client.0.smithi119.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-03-28T00:29:15.950 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:29:15.951 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:29:16.145 INFO:tasks.workunit.client.0.smithi119.stderr:+ c2=2 2024-03-28T00:29:16.145 INFO:tasks.workunit.client.0.smithi119.stderr:+ rmdir d1 2024-03-28T00:29:16.157 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:29:16.157 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:29:16.157 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 81 -lt 100 ']' 2024-03-28T00:29:16.158 INFO:tasks.workunit.client.0.smithi119.stderr:++ expr 81 + 1 2024-03-28T00:29:16.188 INFO:tasks.workunit.client.0.smithi119.stderr:+ c=82 2024-03-28T00:29:16.188 INFO:tasks.workunit.client.0.smithi119.stderr:+ mkdir d1 2024-03-28T00:29:16.206 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:29:16.207 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:29:16.211 INFO:tasks.workunit.client.0.smithi119.stderr:+ c1=2 2024-03-28T00:29:16.213 INFO:tasks.workunit.client.0.smithi119.stderr:+ echo 3 2024-03-28T00:29:16.213 INFO:tasks.workunit.client.0.smithi119.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-03-28T00:29:16.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:29:16 smithi119 ceph-mon[31518]: pgmap v84: 97 pgs: 97 active+clean; 2.6 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 53 KiB/s wr, 36 op/s 2024-03-28T00:29:16.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:29:16 smithi132 ceph-mon[35941]: pgmap v84: 97 pgs: 97 active+clean; 2.6 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 53 KiB/s wr, 36 op/s 2024-03-28T00:29:16.724 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:29:16.725 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:29:16.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:29:16 smithi165 ceph-mon[36302]: pgmap v84: 97 pgs: 97 active+clean; 2.6 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 53 KiB/s wr, 36 op/s 2024-03-28T00:29:16.897 INFO:tasks.workunit.client.0.smithi119.stderr:+ c2=2 2024-03-28T00:29:16.897 INFO:tasks.workunit.client.0.smithi119.stderr:+ rmdir d1 2024-03-28T00:29:16.914 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:29:16.914 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:29:16.915 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 82 -lt 100 ']' 2024-03-28T00:29:16.916 INFO:tasks.workunit.client.0.smithi119.stderr:++ expr 82 + 1 2024-03-28T00:29:16.946 INFO:tasks.workunit.client.0.smithi119.stderr:+ c=83 2024-03-28T00:29:16.946 INFO:tasks.workunit.client.0.smithi119.stderr:+ mkdir d1 2024-03-28T00:29:16.974 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:29:16.975 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:29:16.979 INFO:tasks.workunit.client.0.smithi119.stderr:+ c1=2 2024-03-28T00:29:16.980 INFO:tasks.workunit.client.0.smithi119.stderr:+ echo 3 2024-03-28T00:29:16.981 INFO:tasks.workunit.client.0.smithi119.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-03-28T00:29:17.615 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:29:17.615 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:29:17.786 INFO:tasks.workunit.client.0.smithi119.stderr:+ c2=2 2024-03-28T00:29:17.786 INFO:tasks.workunit.client.0.smithi119.stderr:+ rmdir d1 2024-03-28T00:29:17.798 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:29:17.798 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:29:17.798 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 83 -lt 100 ']' 2024-03-28T00:29:17.799 INFO:tasks.workunit.client.0.smithi119.stderr:++ expr 83 + 1 2024-03-28T00:29:17.846 INFO:tasks.workunit.client.0.smithi119.stderr:+ c=84 2024-03-28T00:29:17.846 INFO:tasks.workunit.client.0.smithi119.stderr:+ mkdir d1 2024-03-28T00:29:17.873 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:29:17.874 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:29:17.879 INFO:tasks.workunit.client.0.smithi119.stderr:+ c1=2 2024-03-28T00:29:17.880 INFO:tasks.workunit.client.0.smithi119.stderr:+ echo 3 2024-03-28T00:29:17.881 INFO:tasks.workunit.client.0.smithi119.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-03-28T00:29:18.358 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:29:18.358 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:29:18.528 INFO:tasks.workunit.client.0.smithi119.stderr:+ c2=2 2024-03-28T00:29:18.528 INFO:tasks.workunit.client.0.smithi119.stderr:+ rmdir d1 2024-03-28T00:29:18.540 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:29:18.540 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:29:18.540 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 84 -lt 100 ']' 2024-03-28T00:29:18.542 INFO:tasks.workunit.client.0.smithi119.stderr:++ expr 84 + 1 2024-03-28T00:29:18.588 INFO:tasks.workunit.client.0.smithi119.stderr:+ c=85 2024-03-28T00:29:18.588 INFO:tasks.workunit.client.0.smithi119.stderr:+ mkdir d1 2024-03-28T00:29:18.617 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:29:18.617 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:29:18.621 INFO:tasks.workunit.client.0.smithi119.stderr:+ c1=2 2024-03-28T00:29:18.622 INFO:tasks.workunit.client.0.smithi119.stderr:+ echo 3 2024-03-28T00:29:18.626 INFO:tasks.workunit.client.0.smithi119.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-03-28T00:29:18.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:29:18 smithi119 ceph-mon[31518]: pgmap v85: 97 pgs: 97 active+clean; 2.6 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 21 KiB/s wr, 14 op/s 2024-03-28T00:29:18.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:29:18 smithi132 ceph-mon[35941]: pgmap v85: 97 pgs: 97 active+clean; 2.6 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 21 KiB/s wr, 14 op/s 2024-03-28T00:29:18.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:29:18 smithi165 ceph-mon[36302]: pgmap v85: 97 pgs: 97 active+clean; 2.6 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 21 KiB/s wr, 14 op/s 2024-03-28T00:29:19.182 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:29:19.182 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:29:19.336 INFO:tasks.workunit.client.0.smithi119.stderr:+ c2=2 2024-03-28T00:29:19.336 INFO:tasks.workunit.client.0.smithi119.stderr:+ rmdir d1 2024-03-28T00:29:19.348 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:29:19.348 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:29:19.348 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 85 -lt 100 ']' 2024-03-28T00:29:19.349 INFO:tasks.workunit.client.0.smithi119.stderr:++ expr 85 + 1 2024-03-28T00:29:19.379 INFO:tasks.workunit.client.0.smithi119.stderr:+ c=86 2024-03-28T00:29:19.379 INFO:tasks.workunit.client.0.smithi119.stderr:+ mkdir d1 2024-03-28T00:29:19.409 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:29:19.410 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:29:19.415 INFO:tasks.workunit.client.0.smithi119.stderr:+ c1=2 2024-03-28T00:29:19.416 INFO:tasks.workunit.client.0.smithi119.stderr:+ echo 3 2024-03-28T00:29:19.417 INFO:tasks.workunit.client.0.smithi119.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-03-28T00:29:20.048 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:29:20.048 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:29:20.219 INFO:tasks.workunit.client.0.smithi119.stderr:+ c2=2 2024-03-28T00:29:20.219 INFO:tasks.workunit.client.0.smithi119.stderr:+ rmdir d1 2024-03-28T00:29:20.238 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:29:20.238 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:29:20.239 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 86 -lt 100 ']' 2024-03-28T00:29:20.240 INFO:tasks.workunit.client.0.smithi119.stderr:++ expr 86 + 1 2024-03-28T00:29:20.270 INFO:tasks.workunit.client.0.smithi119.stderr:+ c=87 2024-03-28T00:29:20.270 INFO:tasks.workunit.client.0.smithi119.stderr:+ mkdir d1 2024-03-28T00:29:20.300 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:29:20.300 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:29:20.305 INFO:tasks.workunit.client.0.smithi119.stderr:+ c1=2 2024-03-28T00:29:20.306 INFO:tasks.workunit.client.0.smithi119.stderr:+ echo 3 2024-03-28T00:29:20.307 INFO:tasks.workunit.client.0.smithi119.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-03-28T00:29:20.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:29:20 smithi119 ceph-mon[31518]: pgmap v86: 97 pgs: 97 active+clean; 2.7 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 31 KiB/s wr, 21 op/s 2024-03-28T00:29:20.686 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:29:20 smithi165 ceph-mon[36302]: pgmap v86: 97 pgs: 97 active+clean; 2.7 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 31 KiB/s wr, 21 op/s 2024-03-28T00:29:20.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:29:20 smithi132 ceph-mon[35941]: pgmap v86: 97 pgs: 97 active+clean; 2.7 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 31 KiB/s wr, 21 op/s 2024-03-28T00:29:20.816 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:29:20.816 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:29:20.986 INFO:tasks.workunit.client.0.smithi119.stderr:+ c2=2 2024-03-28T00:29:20.986 INFO:tasks.workunit.client.0.smithi119.stderr:+ rmdir d1 2024-03-28T00:29:21.005 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:29:21.005 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:29:21.005 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 87 -lt 100 ']' 2024-03-28T00:29:21.007 INFO:tasks.workunit.client.0.smithi119.stderr:++ expr 87 + 1 2024-03-28T00:29:21.106 INFO:tasks.workunit.client.0.smithi119.stderr:+ c=88 2024-03-28T00:29:21.107 INFO:tasks.workunit.client.0.smithi119.stderr:+ mkdir d1 2024-03-28T00:29:21.148 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:29:21.149 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:29:21.154 INFO:tasks.workunit.client.0.smithi119.stderr:+ c1=2 2024-03-28T00:29:21.155 INFO:tasks.workunit.client.0.smithi119.stderr:+ echo 3 2024-03-28T00:29:21.155 INFO:tasks.workunit.client.0.smithi119.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-03-28T00:29:21.657 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:29:21.657 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:29:21.895 INFO:tasks.workunit.client.0.smithi119.stderr:+ c2=2 2024-03-28T00:29:21.895 INFO:tasks.workunit.client.0.smithi119.stderr:+ rmdir d1 2024-03-28T00:29:21.914 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:29:21.914 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:29:21.914 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 88 -lt 100 ']' 2024-03-28T00:29:21.916 INFO:tasks.workunit.client.0.smithi119.stderr:++ expr 88 + 1 2024-03-28T00:29:21.954 INFO:tasks.workunit.client.0.smithi119.stderr:+ c=89 2024-03-28T00:29:21.954 INFO:tasks.workunit.client.0.smithi119.stderr:+ mkdir d1 2024-03-28T00:29:21.983 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:29:21.983 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:29:21.987 INFO:tasks.workunit.client.0.smithi119.stderr:+ c1=2 2024-03-28T00:29:21.988 INFO:tasks.workunit.client.0.smithi119.stderr:+ echo 3 2024-03-28T00:29:21.989 INFO:tasks.workunit.client.0.smithi119.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-03-28T00:29:22.671 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:29:22 smithi119 ceph-mon[31518]: pgmap v87: 97 pgs: 97 active+clean; 2.7 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 21 KiB/s wr, 14 op/s 2024-03-28T00:29:22.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:29:22 smithi132 ceph-mon[35941]: pgmap v87: 97 pgs: 97 active+clean; 2.7 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 21 KiB/s wr, 14 op/s 2024-03-28T00:29:22.739 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:29:22.740 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:29:22.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:29:22 smithi165 ceph-mon[36302]: pgmap v87: 97 pgs: 97 active+clean; 2.7 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 21 KiB/s wr, 14 op/s 2024-03-28T00:29:22.902 INFO:tasks.workunit.client.0.smithi119.stderr:+ c2=2 2024-03-28T00:29:22.902 INFO:tasks.workunit.client.0.smithi119.stderr:+ rmdir d1 2024-03-28T00:29:22.914 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:29:22.914 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:29:22.914 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 89 -lt 100 ']' 2024-03-28T00:29:22.916 INFO:tasks.workunit.client.0.smithi119.stderr:++ expr 89 + 1 2024-03-28T00:29:22.945 INFO:tasks.workunit.client.0.smithi119.stderr:+ c=90 2024-03-28T00:29:22.945 INFO:tasks.workunit.client.0.smithi119.stderr:+ mkdir d1 2024-03-28T00:29:22.974 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:29:22.975 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:29:22.978 INFO:tasks.workunit.client.0.smithi119.stderr:+ c1=2 2024-03-28T00:29:22.980 INFO:tasks.workunit.client.0.smithi119.stderr:+ echo 3 2024-03-28T00:29:22.980 INFO:tasks.workunit.client.0.smithi119.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-03-28T00:29:23.467 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:29:23.468 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:29:23.635 INFO:tasks.workunit.client.0.smithi119.stderr:+ c2=2 2024-03-28T00:29:23.635 INFO:tasks.workunit.client.0.smithi119.stderr:+ rmdir d1 2024-03-28T00:29:23.647 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:29:23.647 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:29:23.647 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 90 -lt 100 ']' 2024-03-28T00:29:23.649 INFO:tasks.workunit.client.0.smithi119.stderr:++ expr 90 + 1 2024-03-28T00:29:23.695 INFO:tasks.workunit.client.0.smithi119.stderr:+ c=91 2024-03-28T00:29:23.695 INFO:tasks.workunit.client.0.smithi119.stderr:+ mkdir d1 2024-03-28T00:29:23.722 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:29:23.722 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:29:23.726 INFO:tasks.workunit.client.0.smithi119.stderr:+ c1=2 2024-03-28T00:29:23.727 INFO:tasks.workunit.client.0.smithi119.stderr:+ echo 3 2024-03-28T00:29:23.727 INFO:tasks.workunit.client.0.smithi119.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-03-28T00:29:24.298 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:29:24.299 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:29:24.489 INFO:tasks.workunit.client.0.smithi119.stderr:+ c2=2 2024-03-28T00:29:24.489 INFO:tasks.workunit.client.0.smithi119.stderr:+ rmdir d1 2024-03-28T00:29:24.506 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:29:24.506 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:29:24.507 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 91 -lt 100 ']' 2024-03-28T00:29:24.507 INFO:tasks.workunit.client.0.smithi119.stderr:++ expr 91 + 1 2024-03-28T00:29:24.537 INFO:tasks.workunit.client.0.smithi119.stderr:+ c=92 2024-03-28T00:29:24.537 INFO:tasks.workunit.client.0.smithi119.stderr:+ mkdir d1 2024-03-28T00:29:24.566 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:29:24.566 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:29:24.571 INFO:tasks.workunit.client.0.smithi119.stderr:+ c1=2 2024-03-28T00:29:24.571 INFO:tasks.workunit.client.0.smithi119.stderr:+ echo 3 2024-03-28T00:29:24.572 INFO:tasks.workunit.client.0.smithi119.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-03-28T00:29:24.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:29:24 smithi132 ceph-mon[35941]: pgmap v88: 97 pgs: 97 active+clean; 2.7 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 21 KiB/s wr, 14 op/s 2024-03-28T00:29:24.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:29:24 smithi165 ceph-mon[36302]: pgmap v88: 97 pgs: 97 active+clean; 2.7 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 21 KiB/s wr, 14 op/s 2024-03-28T00:29:24.883 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:29:24 smithi119 ceph-mon[31518]: pgmap v88: 97 pgs: 97 active+clean; 2.7 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 21 KiB/s wr, 14 op/s 2024-03-28T00:29:25.098 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:29:25.100 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:29:25.252 INFO:tasks.workunit.client.0.smithi119.stderr:+ c2=2 2024-03-28T00:29:25.252 INFO:tasks.workunit.client.0.smithi119.stderr:+ rmdir d1 2024-03-28T00:29:25.278 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:29:25.278 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:29:25.278 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 92 -lt 100 ']' 2024-03-28T00:29:25.279 INFO:tasks.workunit.client.0.smithi119.stderr:++ expr 92 + 1 2024-03-28T00:29:25.311 INFO:tasks.workunit.client.0.smithi119.stderr:+ c=93 2024-03-28T00:29:25.311 INFO:tasks.workunit.client.0.smithi119.stderr:+ mkdir d1 2024-03-28T00:29:25.347 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:29:25.348 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:29:25.352 INFO:tasks.workunit.client.0.smithi119.stderr:+ c1=2 2024-03-28T00:29:25.354 INFO:tasks.workunit.client.0.smithi119.stderr:+ echo 3 2024-03-28T00:29:25.354 INFO:tasks.workunit.client.0.smithi119.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-03-28T00:29:25.840 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:29:25.840 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:29:26.086 INFO:tasks.workunit.client.0.smithi119.stderr:+ c2=2 2024-03-28T00:29:26.086 INFO:tasks.workunit.client.0.smithi119.stderr:+ rmdir d1 2024-03-28T00:29:26.105 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:29:26.105 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:29:26.105 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 93 -lt 100 ']' 2024-03-28T00:29:26.107 INFO:tasks.workunit.client.0.smithi119.stderr:++ expr 93 + 1 2024-03-28T00:29:26.146 INFO:tasks.workunit.client.0.smithi119.stderr:+ c=94 2024-03-28T00:29:26.146 INFO:tasks.workunit.client.0.smithi119.stderr:+ mkdir d1 2024-03-28T00:29:26.180 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:29:26.181 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:29:26.185 INFO:tasks.workunit.client.0.smithi119.stderr:+ c1=2 2024-03-28T00:29:26.186 INFO:tasks.workunit.client.0.smithi119.stderr:+ echo 3 2024-03-28T00:29:26.186 INFO:tasks.workunit.client.0.smithi119.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-03-28T00:29:26.656 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:29:26.657 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:29:26.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:29:26 smithi132 ceph-mon[35941]: pgmap v89: 97 pgs: 97 active+clean; 2.8 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 31 KiB/s wr, 20 op/s 2024-03-28T00:29:26.827 INFO:tasks.workunit.client.0.smithi119.stderr:+ c2=2 2024-03-28T00:29:26.827 INFO:tasks.workunit.client.0.smithi119.stderr:+ rmdir d1 2024-03-28T00:29:26.837 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:29:26.838 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:29:26.838 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 94 -lt 100 ']' 2024-03-28T00:29:26.839 INFO:tasks.workunit.client.0.smithi119.stderr:++ expr 94 + 1 2024-03-28T00:29:26.869 INFO:tasks.workunit.client.0.smithi119.stderr:+ c=95 2024-03-28T00:29:26.869 INFO:tasks.workunit.client.0.smithi119.stderr:+ mkdir d1 2024-03-28T00:29:26.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:29:26 smithi165 ceph-mon[36302]: pgmap v89: 97 pgs: 97 active+clean; 2.8 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 31 KiB/s wr, 20 op/s 2024-03-28T00:29:26.896 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:29:26.897 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:29:26.902 INFO:tasks.workunit.client.0.smithi119.stderr:+ c1=2 2024-03-28T00:29:26.903 INFO:tasks.workunit.client.0.smithi119.stderr:+ echo 3 2024-03-28T00:29:26.904 INFO:tasks.workunit.client.0.smithi119.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-03-28T00:29:26.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:29:26 smithi119 ceph-mon[31518]: pgmap v89: 97 pgs: 97 active+clean; 2.8 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 31 KiB/s wr, 20 op/s 2024-03-28T00:29:27.406 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:29:27.407 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:29:27.602 INFO:tasks.workunit.client.0.smithi119.stderr:+ c2=2 2024-03-28T00:29:27.602 INFO:tasks.workunit.client.0.smithi119.stderr:+ rmdir d1 2024-03-28T00:29:27.630 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:29:27.630 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:29:27.630 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 95 -lt 100 ']' 2024-03-28T00:29:27.630 INFO:tasks.workunit.client.0.smithi119.stderr:++ expr 95 + 1 2024-03-28T00:29:27.661 INFO:tasks.workunit.client.0.smithi119.stderr:+ c=96 2024-03-28T00:29:27.661 INFO:tasks.workunit.client.0.smithi119.stderr:+ mkdir d1 2024-03-28T00:29:27.688 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:29:27.689 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:29:27.693 INFO:tasks.workunit.client.0.smithi119.stderr:+ c1=2 2024-03-28T00:29:27.695 INFO:tasks.workunit.client.0.smithi119.stderr:+ echo 3 2024-03-28T00:29:27.696 INFO:tasks.workunit.client.0.smithi119.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-03-28T00:29:28.215 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:29:28.216 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:29:28.388 INFO:tasks.workunit.client.0.smithi119.stderr:+ c2=2 2024-03-28T00:29:28.388 INFO:tasks.workunit.client.0.smithi119.stderr:+ rmdir d1 2024-03-28T00:29:28.404 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:29:28.404 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:29:28.404 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 96 -lt 100 ']' 2024-03-28T00:29:28.405 INFO:tasks.workunit.client.0.smithi119.stderr:++ expr 96 + 1 2024-03-28T00:29:28.453 INFO:tasks.workunit.client.0.smithi119.stderr:+ c=97 2024-03-28T00:29:28.453 INFO:tasks.workunit.client.0.smithi119.stderr:+ mkdir d1 2024-03-28T00:29:28.481 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:29:28.482 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:29:28.487 INFO:tasks.workunit.client.0.smithi119.stderr:+ c1=2 2024-03-28T00:29:28.488 INFO:tasks.workunit.client.0.smithi119.stderr:+ echo 3 2024-03-28T00:29:28.489 INFO:tasks.workunit.client.0.smithi119.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-03-28T00:29:28.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:29:28 smithi132 ceph-mon[35941]: pgmap v90: 97 pgs: 97 active+clean; 2.8 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 13 op/s 2024-03-28T00:29:28.818 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:29:28 smithi119 ceph-mon[31518]: pgmap v90: 97 pgs: 97 active+clean; 2.8 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 13 op/s 2024-03-28T00:29:28.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:29:28 smithi165 ceph-mon[36302]: pgmap v90: 97 pgs: 97 active+clean; 2.8 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 13 op/s 2024-03-28T00:29:28.989 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:29:28.989 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:29:29.251 INFO:tasks.workunit.client.0.smithi119.stderr:+ c2=2 2024-03-28T00:29:29.251 INFO:tasks.workunit.client.0.smithi119.stderr:+ rmdir d1 2024-03-28T00:29:29.273 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:29:29.273 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:29:29.273 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 97 -lt 100 ']' 2024-03-28T00:29:29.274 INFO:tasks.workunit.client.0.smithi119.stderr:++ expr 97 + 1 2024-03-28T00:29:29.298 INFO:tasks.workunit.client.0.smithi119.stderr:+ c=98 2024-03-28T00:29:29.298 INFO:tasks.workunit.client.0.smithi119.stderr:+ mkdir d1 2024-03-28T00:29:29.330 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:29:29.330 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:29:29.334 INFO:tasks.workunit.client.0.smithi119.stderr:+ c1=2 2024-03-28T00:29:29.335 INFO:tasks.workunit.client.0.smithi119.stderr:+ echo 3 2024-03-28T00:29:29.336 INFO:tasks.workunit.client.0.smithi119.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-03-28T00:29:29.839 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:29:29.839 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:29:30.027 INFO:tasks.workunit.client.0.smithi119.stderr:+ c2=2 2024-03-28T00:29:30.027 INFO:tasks.workunit.client.0.smithi119.stderr:+ rmdir d1 2024-03-28T00:29:30.044 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:29:30.045 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:29:30.045 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 98 -lt 100 ']' 2024-03-28T00:29:30.045 INFO:tasks.workunit.client.0.smithi119.stderr:++ expr 98 + 1 2024-03-28T00:29:30.086 INFO:tasks.workunit.client.0.smithi119.stderr:+ c=99 2024-03-28T00:29:30.086 INFO:tasks.workunit.client.0.smithi119.stderr:+ mkdir d1 2024-03-28T00:29:30.114 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:29:30.114 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:29:30.119 INFO:tasks.workunit.client.0.smithi119.stderr:+ c1=2 2024-03-28T00:29:30.119 INFO:tasks.workunit.client.0.smithi119.stderr:+ echo 3 2024-03-28T00:29:30.120 INFO:tasks.workunit.client.0.smithi119.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-03-28T00:29:30.696 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:29:30 smithi119 ceph-mon[31518]: pgmap v91: 97 pgs: 97 active+clean; 2.9 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 31 KiB/s wr, 20 op/s 2024-03-28T00:29:30.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:29:30 smithi132 ceph-mon[35941]: pgmap v91: 97 pgs: 97 active+clean; 2.9 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 31 KiB/s wr, 20 op/s 2024-03-28T00:29:30.771 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:29:30.771 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:29:30.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:29:30 smithi165 ceph-mon[36302]: pgmap v91: 97 pgs: 97 active+clean; 2.9 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 31 KiB/s wr, 20 op/s 2024-03-28T00:29:30.942 INFO:tasks.workunit.client.0.smithi119.stderr:+ c2=2 2024-03-28T00:29:30.943 INFO:tasks.workunit.client.0.smithi119.stderr:+ rmdir d1 2024-03-28T00:29:30.975 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:29:30.975 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:29:30.975 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 99 -lt 100 ']' 2024-03-28T00:29:30.976 INFO:tasks.workunit.client.0.smithi119.stderr:++ expr 99 + 1 2024-03-28T00:29:31.010 INFO:tasks.workunit.client.0.smithi119.stderr:+ c=100 2024-03-28T00:29:31.010 INFO:tasks.workunit.client.0.smithi119.stderr:+ mkdir d1 2024-03-28T00:29:31.038 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:29:31.038 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:29:31.042 INFO:tasks.workunit.client.0.smithi119.stderr:+ c1=2 2024-03-28T00:29:31.044 INFO:tasks.workunit.client.0.smithi119.stderr:+ echo 3 2024-03-28T00:29:31.044 INFO:tasks.workunit.client.0.smithi119.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-03-28T00:29:31.590 INFO:tasks.workunit.client.0.smithi119.stderr:++ getfacl d1 2024-03-28T00:29:31.590 INFO:tasks.workunit.client.0.smithi119.stderr:++ grep -c nobody:rw 2024-03-28T00:29:31.760 INFO:tasks.workunit.client.0.smithi119.stderr:+ c2=2 2024-03-28T00:29:31.760 INFO:tasks.workunit.client.0.smithi119.stderr:+ rmdir d1 2024-03-28T00:29:31.778 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:29:31.778 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 2 -ne 2 ']' 2024-03-28T00:29:31.779 INFO:tasks.workunit.client.0.smithi119.stderr:+ '[' 100 -lt 100 ']' 2024-03-28T00:29:31.783 INFO:tasks.workunit.client.0.smithi119.stderr:+ mkdir d1 2024-03-28T00:29:31.809 INFO:tasks.workunit.client.0.smithi119.stderr:+ setfattr -n system.posix_acl_access -v 0x02000000 d1 2024-03-28T00:29:31.821 INFO:tasks.workunit.client.0.smithi119.stderr:+ setfattr -n system.posix_acl_default -v 0x02000000 . 2024-03-28T00:29:31.826 INFO:tasks.workunit.client.0.smithi119.stderr:+ expect_failure getfattr -n system.posix_acl_access d1 2024-03-28T00:29:31.826 INFO:tasks.workunit.client.0.smithi119.stderr:+ getfattr -n system.posix_acl_access d1 2024-03-28T00:29:31.831 INFO:tasks.workunit.client.0.smithi119.stderr:d1: system.posix_acl_access: No such attribute 2024-03-28T00:29:31.831 INFO:tasks.workunit.client.0.smithi119.stderr:+ return 0 2024-03-28T00:29:31.831 INFO:tasks.workunit.client.0.smithi119.stderr:+ expect_failure getfattr -n system.posix_acl_default . 2024-03-28T00:29:31.832 INFO:tasks.workunit.client.0.smithi119.stderr:+ getfattr -n system.posix_acl_default . 2024-03-28T00:29:31.834 INFO:tasks.workunit.client.0.smithi119.stderr:.: system.posix_acl_default: No such attribute 2024-03-28T00:29:31.834 INFO:tasks.workunit.client.0.smithi119.stderr:+ return 0 2024-03-28T00:29:31.834 INFO:tasks.workunit.client.0.smithi119.stderr:+ rmdir d1 2024-03-28T00:29:31.844 INFO:tasks.workunit.client.0.smithi119.stderr:+ cd .. 2024-03-28T00:29:31.844 INFO:tasks.workunit.client.0.smithi119.stderr:+ rmdir testdir 2024-03-28T00:29:31.856 INFO:tasks.workunit.client.0.smithi119.stderr:+ echo OK 2024-03-28T00:29:31.856 INFO:tasks.workunit.client.0.smithi119.stdout:OK 2024-03-28T00:29:31.858 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-28T00:29:31.858 DEBUG:teuthology.orchestra.run.smithi119:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-28T00:29:32.520 INFO:tasks.workunit:Running workunit fs/misc/chmod.sh... 2024-03-28T00:29:32.521 DEBUG:teuthology.orchestra.run.smithi119:workunit test fs/misc/chmod.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=3be9a5c9bc793e11e2800a8c0c696e8b46742033 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/chmod.sh 2024-03-28T00:29:32.645 INFO:tasks.workunit.client.0.smithi119.stderr:+ set -e 2024-03-28T00:29:32.645 INFO:tasks.workunit.client.0.smithi119.stderr:+ file=test_chmod.84428 2024-03-28T00:29:32.645 INFO:tasks.workunit.client.0.smithi119.stderr:+ echo foo 2024-03-28T00:29:32.653 INFO:tasks.workunit.client.0.smithi119.stderr:+ test 0 '!=' 0 2024-03-28T00:29:32.653 INFO:tasks.workunit.client.0.smithi119.stderr:+ chmod 400 test_chmod.84428 2024-03-28T00:29:32.665 INFO:tasks.workunit.client.0.smithi119.stderr:+ test 0 '!=' 0 2024-03-28T00:29:32.665 INFO:tasks.workunit.client.0.smithi119.stderr:+ check_perms test_chmod.84428 -r-------- 2024-03-28T00:29:32.665 INFO:tasks.workunit.client.0.smithi119.stderr:+ file=test_chmod.84428 2024-03-28T00:29:32.665 INFO:tasks.workunit.client.0.smithi119.stderr:++ ls -la test_chmod.84428 2024-03-28T00:29:32.677 INFO:tasks.workunit.client.0.smithi119.stderr:+ r='-r--------. 1 ubuntu ubuntu 4 Mar 28 00:29 test_chmod.84428' 2024-03-28T00:29:32.678 INFO:tasks.workunit.client.0.smithi119.stderr:+ test 0 '!=' 0 2024-03-28T00:29:32.678 INFO:tasks.workunit.client.0.smithi119.stderr:+ perms=-r-------- 2024-03-28T00:29:32.680 INFO:tasks.workunit.client.0.smithi119.stderr:++ echo -r--------. 1 ubuntu ubuntu 4 Mar 28 00:29 test_chmod.84428 2024-03-28T00:29:32.680 INFO:tasks.workunit.client.0.smithi119.stderr:++ awk '{print $1}' 2024-03-28T00:29:32.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:29:32 smithi132 ceph-mon[35941]: pgmap v92: 97 pgs: 97 active+clean; 2.9 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 13 op/s 2024-03-28T00:29:32.741 INFO:tasks.workunit.client.0.smithi119.stderr:+ test -r-------- '!=' -r--------. 2024-03-28T00:29:32.743 INFO:tasks.workunit.client.0.smithi119.stderr:++ echo -r--------. 1 ubuntu ubuntu 4 Mar 28 00:29 test_chmod.84428 2024-03-28T00:29:32.743 INFO:tasks.workunit.client.0.smithi119.stderr:++ awk '{print $1}' 2024-03-28T00:29:32.749 INFO:tasks.workunit.client.0.smithi119.stderr:+ test -r--------. '!=' -r--------. 2024-03-28T00:29:32.749 INFO:tasks.workunit.client.0.smithi119.stderr:+ set +e 2024-03-28T00:29:32.750 INFO:tasks.workunit.client.0.smithi119.stderr:+ echo bar 2024-03-28T00:29:32.750 INFO:tasks.workunit.client.0.smithi119.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/chmod.sh: line 40: test_chmod.84428: Permission denied 2024-03-28T00:29:32.750 INFO:tasks.workunit.client.0.smithi119.stderr:+ test 1 = 0 2024-03-28T00:29:32.750 INFO:tasks.workunit.client.0.smithi119.stderr:+ set -e 2024-03-28T00:29:32.750 INFO:tasks.workunit.client.0.smithi119.stderr:+ chmod 600 test_chmod.84428 2024-03-28T00:29:32.753 INFO:tasks.workunit.client.0.smithi119.stderr:+ echo bar 2024-03-28T00:29:32.753 INFO:tasks.workunit.client.0.smithi119.stderr:+ test 0 '!=' 0 2024-03-28T00:29:32.753 INFO:tasks.workunit.client.0.smithi119.stderr:+ check_perms test_chmod.84428 -rw------- 2024-03-28T00:29:32.753 INFO:tasks.workunit.client.0.smithi119.stderr:+ file=test_chmod.84428 2024-03-28T00:29:32.754 INFO:tasks.workunit.client.0.smithi119.stderr:++ ls -la test_chmod.84428 2024-03-28T00:29:32.758 INFO:tasks.workunit.client.0.smithi119.stderr:+ r='-rw-------. 1 ubuntu ubuntu 8 Mar 28 00:29 test_chmod.84428' 2024-03-28T00:29:32.758 INFO:tasks.workunit.client.0.smithi119.stderr:+ test 0 '!=' 0 2024-03-28T00:29:32.759 INFO:tasks.workunit.client.0.smithi119.stderr:+ perms=-rw------- 2024-03-28T00:29:32.761 INFO:tasks.workunit.client.0.smithi119.stderr:++ echo -rw-------. 1 ubuntu ubuntu 8 Mar 28 00:29 test_chmod.84428 2024-03-28T00:29:32.761 INFO:tasks.workunit.client.0.smithi119.stderr:++ awk '{print $1}' 2024-03-28T00:29:32.767 INFO:tasks.workunit.client.0.smithi119.stderr:+ test -rw------- '!=' -rw-------. 2024-03-28T00:29:32.769 INFO:tasks.workunit.client.0.smithi119.stderr:++ echo -rw-------. 1 ubuntu ubuntu 8 Mar 28 00:29 test_chmod.84428 2024-03-28T00:29:32.770 INFO:tasks.workunit.client.0.smithi119.stderr:++ awk '{print $1}' 2024-03-28T00:29:32.777 INFO:tasks.workunit.client.0.smithi119.stderr:+ test -rw-------. '!=' -rw-------. 2024-03-28T00:29:32.777 INFO:tasks.workunit.client.0.smithi119.stderr:+ echo foo 2024-03-28T00:29:32.777 INFO:tasks.workunit.client.0.smithi119.stderr:+ test 0 '!=' 0 2024-03-28T00:29:32.778 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-28T00:29:32.778 DEBUG:teuthology.orchestra.run.smithi119:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-28T00:29:32.854 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:29:32 smithi119 ceph-mon[31518]: pgmap v92: 97 pgs: 97 active+clean; 2.9 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 13 op/s 2024-03-28T00:29:32.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:29:32 smithi165 ceph-mon[36302]: pgmap v92: 97 pgs: 97 active+clean; 2.9 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 13 op/s 2024-03-28T00:29:32.885 INFO:tasks.workunit:Running workunit fs/misc/dac_override.sh... 2024-03-28T00:29:32.885 DEBUG:teuthology.orchestra.run.smithi119:workunit test fs/misc/dac_override.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=3be9a5c9bc793e11e2800a8c0c696e8b46742033 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/dac_override.sh 2024-03-28T00:29:32.961 INFO:tasks.workunit.client.0.smithi119.stderr:+ set -e 2024-03-28T00:29:32.961 INFO:tasks.workunit.client.0.smithi119.stderr:+ mkdir -p testdir 2024-03-28T00:29:32.966 INFO:tasks.workunit.client.0.smithi119.stderr:+ file=test_chmod.84481 2024-03-28T00:29:32.966 INFO:tasks.workunit.client.0.smithi119.stderr:+ echo foo 2024-03-28T00:29:32.968 INFO:tasks.workunit.client.0.smithi119.stderr:+ sudo chmod 600 testdir 2024-03-28T00:29:33.013 INFO:tasks.workunit.client.0.smithi119.stderr:+ expect_failure cat testdir/test_chmod.84481 2024-03-28T00:29:33.013 INFO:tasks.workunit.client.0.smithi119.stderr:+ cat testdir/test_chmod.84481 2024-03-28T00:29:33.015 INFO:tasks.workunit.client.0.smithi119.stderr:cat: testdir/test_chmod.84481: Permission denied 2024-03-28T00:29:33.016 INFO:tasks.workunit.client.0.smithi119.stderr:+ return 0 2024-03-28T00:29:33.016 INFO:tasks.workunit.client.0.smithi119.stderr:+ sudo cat testdir/test_chmod.84481 2024-03-28T00:29:33.055 INFO:tasks.workunit.client.0.smithi119.stdout:foo 2024-03-28T00:29:33.060 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-28T00:29:33.060 DEBUG:teuthology.orchestra.run.smithi119:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-28T00:29:33.182 INFO:tasks.workunit:Running workunit fs/misc/direct_io.py... 2024-03-28T00:29:33.182 DEBUG:teuthology.orchestra.run.smithi119:workunit test fs/misc/direct_io.py> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=3be9a5c9bc793e11e2800a8c0c696e8b46742033 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/direct_io.py 2024-03-28T00:29:34.379 INFO:tasks.workunit.client.0.smithi119.stdout:ok 2024-03-28T00:29:34.386 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-28T00:29:34.386 DEBUG:teuthology.orchestra.run.smithi119:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-28T00:29:34.506 INFO:tasks.workunit:Running workunit fs/misc/dirfrag.sh... 2024-03-28T00:29:34.507 DEBUG:teuthology.orchestra.run.smithi119:workunit test fs/misc/dirfrag.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=3be9a5c9bc793e11e2800a8c0c696e8b46742033 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/dirfrag.sh 2024-03-28T00:29:34.617 INFO:tasks.workunit.client.0.smithi119.stdout:creating folder hierarchy 2024-03-28T00:29:34.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:29:34 smithi132 ceph-mon[35941]: pgmap v93: 97 pgs: 97 active+clean; 2.9 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 13 op/s 2024-03-28T00:29:34.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:29:34 smithi165 ceph-mon[36302]: pgmap v93: 97 pgs: 97 active+clean; 2.9 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 13 op/s 2024-03-28T00:29:34.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:29:34 smithi119 ceph-mon[31518]: pgmap v93: 97 pgs: 97 active+clean; 2.9 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 13 op/s 2024-03-28T00:29:36.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:29:36 smithi132 ceph-mon[35941]: pgmap v94: 97 pgs: 97 active+clean; 3.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 36 KiB/s wr, 21 op/s 2024-03-28T00:29:36.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:29:36 smithi165 ceph-mon[36302]: pgmap v94: 97 pgs: 97 active+clean; 3.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 36 KiB/s wr, 21 op/s 2024-03-28T00:29:36.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:29:36 smithi119 ceph-mon[31518]: pgmap v94: 97 pgs: 97 active+clean; 3.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 36 KiB/s wr, 21 op/s 2024-03-28T00:29:38.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:29:38 smithi132 ceph-mon[35941]: pgmap v95: 97 pgs: 97 active+clean; 3.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 26 KiB/s wr, 14 op/s 2024-03-28T00:29:38.884 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:29:38 smithi165 ceph-mon[36302]: pgmap v95: 97 pgs: 97 active+clean; 3.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 26 KiB/s wr, 14 op/s 2024-03-28T00:29:38.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:29:38 smithi119 ceph-mon[31518]: pgmap v95: 97 pgs: 97 active+clean; 3.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 26 KiB/s wr, 14 op/s 2024-03-28T00:29:40.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:29:40 smithi132 ceph-mon[35941]: pgmap v96: 97 pgs: 97 active+clean; 7.1 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 613 KiB/s wr, 191 op/s 2024-03-28T00:29:40.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:29:40 smithi165 ceph-mon[36302]: pgmap v96: 97 pgs: 97 active+clean; 7.1 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 613 KiB/s wr, 191 op/s 2024-03-28T00:29:40.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:29:40 smithi119 ceph-mon[31518]: pgmap v96: 97 pgs: 97 active+clean; 7.1 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 613 KiB/s wr, 191 op/s 2024-03-28T00:29:42.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:29:42 smithi132 ceph-mon[35941]: pgmap v97: 97 pgs: 97 active+clean; 7.1 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 603 KiB/s wr, 184 op/s 2024-03-28T00:29:42.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:29:42 smithi165 ceph-mon[36302]: pgmap v97: 97 pgs: 97 active+clean; 7.1 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 603 KiB/s wr, 184 op/s 2024-03-28T00:29:42.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:29:42 smithi119 ceph-mon[31518]: pgmap v97: 97 pgs: 97 active+clean; 7.1 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 603 KiB/s wr, 184 op/s 2024-03-28T00:29:44.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:29:44 smithi132 ceph-mon[35941]: pgmap v98: 97 pgs: 97 active+clean; 7.1 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 603 KiB/s wr, 184 op/s 2024-03-28T00:29:44.708 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:29:44 smithi119 ceph-mon[31518]: pgmap v98: 97 pgs: 97 active+clean; 7.1 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 603 KiB/s wr, 184 op/s 2024-03-28T00:29:44.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:29:44 smithi165 ceph-mon[36302]: pgmap v98: 97 pgs: 97 active+clean; 7.1 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 603 KiB/s wr, 184 op/s 2024-03-28T00:29:46.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:29:46 smithi165 ceph-mon[36302]: pgmap v99: 97 pgs: 97 active+clean; 15 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 988 KiB/s rd, 1.4 MiB/s wr, 333 op/s 2024-03-28T00:29:46.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:29:46 smithi165 ceph-mon[36302]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-28T00:29:46.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:29:46 smithi119 ceph-mon[31518]: pgmap v99: 97 pgs: 97 active+clean; 15 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 988 KiB/s rd, 1.4 MiB/s wr, 333 op/s 2024-03-28T00:29:46.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:29:46 smithi119 ceph-mon[31518]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-28T00:29:46.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:29:46 smithi132 ceph-mon[35941]: pgmap v99: 97 pgs: 97 active+clean; 15 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 988 KiB/s rd, 1.4 MiB/s wr, 333 op/s 2024-03-28T00:29:46.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:29:46 smithi132 ceph-mon[35941]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-28T00:29:48.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:29:48 smithi165 ceph-mon[36302]: pgmap v100: 97 pgs: 97 active+clean; 15 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 972 KiB/s rd, 1.4 MiB/s wr, 325 op/s 2024-03-28T00:29:48.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:29:48 smithi119 ceph-mon[31518]: pgmap v100: 97 pgs: 97 active+clean; 15 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 972 KiB/s rd, 1.4 MiB/s wr, 325 op/s 2024-03-28T00:29:48.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:29:48 smithi132 ceph-mon[35941]: pgmap v100: 97 pgs: 97 active+clean; 15 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 972 KiB/s rd, 1.4 MiB/s wr, 325 op/s 2024-03-28T00:29:50.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:29:50 smithi165 ceph-mon[36302]: pgmap v101: 97 pgs: 97 active+clean; 19 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.0 MiB/s wr, 532 op/s 2024-03-28T00:29:50.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:29:50 smithi119 ceph-mon[31518]: pgmap v101: 97 pgs: 97 active+clean; 19 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.0 MiB/s wr, 532 op/s 2024-03-28T00:29:50.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:29:50 smithi132 ceph-mon[35941]: pgmap v101: 97 pgs: 97 active+clean; 19 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.0 MiB/s wr, 532 op/s 2024-03-28T00:29:51.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:29:51 smithi165 ceph-mon[36302]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-28T00:29:51.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:29:51 smithi119 ceph-mon[31518]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-28T00:29:51.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:29:51 smithi132 ceph-mon[35941]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-28T00:29:52.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:29:52 smithi165 ceph-mon[36302]: pgmap v102: 97 pgs: 97 active+clean; 19 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 356 op/s 2024-03-28T00:29:52.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:29:52 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:29:52.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:29:52 smithi119 ceph-mon[31518]: pgmap v102: 97 pgs: 97 active+clean; 19 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 356 op/s 2024-03-28T00:29:52.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:29:52 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:29:52.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:29:52 smithi132 ceph-mon[35941]: pgmap v102: 97 pgs: 97 active+clean; 19 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 356 op/s 2024-03-28T00:29:52.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:29:52 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:29:54.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:29:54 smithi165 ceph-mon[36302]: pgmap v103: 97 pgs: 97 active+clean; 19 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 356 op/s 2024-03-28T00:29:54.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:29:54 smithi119 ceph-mon[31518]: pgmap v103: 97 pgs: 97 active+clean; 19 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 356 op/s 2024-03-28T00:29:54.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:29:54 smithi132 ceph-mon[35941]: pgmap v103: 97 pgs: 97 active+clean; 19 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 356 op/s 2024-03-28T00:29:56.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:29:56 smithi165 ceph-mon[36302]: pgmap v104: 97 pgs: 97 active+clean; 24 MiB data, 586 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.2 MiB/s wr, 573 op/s 2024-03-28T00:29:56.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:29:56 smithi165 ceph-mon[36302]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-28T00:29:56.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:29:56 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:29:56.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:29:56 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:29:56.884 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:29:56 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:29:56.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:29:56 smithi119 ceph-mon[31518]: pgmap v104: 97 pgs: 97 active+clean; 24 MiB data, 586 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.2 MiB/s wr, 573 op/s 2024-03-28T00:29:56.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:29:56 smithi119 ceph-mon[31518]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-28T00:29:56.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:29:56 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:29:56.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:29:56 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:29:56.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:29:56 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:29:56.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:29:56 smithi132 ceph-mon[35941]: pgmap v104: 97 pgs: 97 active+clean; 24 MiB data, 586 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.2 MiB/s wr, 573 op/s 2024-03-28T00:29:56.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:29:56 smithi132 ceph-mon[35941]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-28T00:29:56.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:29:56 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:29:56.954 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:29:56 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:29:56.954 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:29:56 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:29:58.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:29:58 smithi165 ceph-mon[36302]: pgmap v105: 97 pgs: 97 active+clean; 24 MiB data, 586 MiB used, 1.0 TiB / 1.0 TiB avail; 766 KiB/s rd, 1.4 MiB/s wr, 423 op/s 2024-03-28T00:29:58.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:29:58 smithi119 ceph-mon[31518]: pgmap v105: 97 pgs: 97 active+clean; 24 MiB data, 586 MiB used, 1.0 TiB / 1.0 TiB avail; 766 KiB/s rd, 1.4 MiB/s wr, 423 op/s 2024-03-28T00:29:58.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:29:58 smithi132 ceph-mon[35941]: pgmap v105: 97 pgs: 97 active+clean; 24 MiB data, 586 MiB used, 1.0 TiB / 1.0 TiB avail; 766 KiB/s rd, 1.4 MiB/s wr, 423 op/s 2024-03-28T00:30:00.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:30:00 smithi165 ceph-mon[36302]: pgmap v106: 97 pgs: 97 active+clean; 29 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 628 op/s 2024-03-28T00:30:00.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:30:00 smithi165 ceph-mon[36302]: overall HEALTH_OK 2024-03-28T00:30:00.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:30:00 smithi119 ceph-mon[31518]: pgmap v106: 97 pgs: 97 active+clean; 29 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 628 op/s 2024-03-28T00:30:00.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:30:00 smithi119 ceph-mon[31518]: overall HEALTH_OK 2024-03-28T00:30:00.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:30:00 smithi132 ceph-mon[35941]: pgmap v106: 97 pgs: 97 active+clean; 29 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 628 op/s 2024-03-28T00:30:00.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:30:00 smithi132 ceph-mon[35941]: overall HEALTH_OK 2024-03-28T00:30:02.755 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:30:02 smithi132 ceph-mon[35941]: pgmap v107: 97 pgs: 97 active+clean; 29 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 804 KiB/s rd, 1.4 MiB/s wr, 421 op/s 2024-03-28T00:30:02.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:30:02 smithi165 ceph-mon[36302]: pgmap v107: 97 pgs: 97 active+clean; 29 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 804 KiB/s rd, 1.4 MiB/s wr, 421 op/s 2024-03-28T00:30:02.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:30:02 smithi119 ceph-mon[31518]: pgmap v107: 97 pgs: 97 active+clean; 29 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 804 KiB/s rd, 1.4 MiB/s wr, 421 op/s 2024-03-28T00:30:04.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:30:04 smithi165 ceph-mon[36302]: pgmap v108: 97 pgs: 97 active+clean; 29 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 804 KiB/s rd, 1.4 MiB/s wr, 421 op/s 2024-03-28T00:30:04.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:30:04 smithi119 ceph-mon[31518]: pgmap v108: 97 pgs: 97 active+clean; 29 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 804 KiB/s rd, 1.4 MiB/s wr, 421 op/s 2024-03-28T00:30:04.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:30:04 smithi132 ceph-mon[35941]: pgmap v108: 97 pgs: 97 active+clean; 29 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 804 KiB/s rd, 1.4 MiB/s wr, 421 op/s 2024-03-28T00:30:06.884 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:30:06 smithi165 ceph-mon[36302]: pgmap v109: 97 pgs: 97 active+clean; 30 MiB data, 657 MiB used, 1.0 TiB / 1.0 TiB avail; 930 KiB/s rd, 1.5 MiB/s wr, 468 op/s 2024-03-28T00:30:06.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:30:06 smithi119 ceph-mon[31518]: pgmap v109: 97 pgs: 97 active+clean; 30 MiB data, 657 MiB used, 1.0 TiB / 1.0 TiB avail; 930 KiB/s rd, 1.5 MiB/s wr, 468 op/s 2024-03-28T00:30:06.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:30:06 smithi132 ceph-mon[35941]: pgmap v109: 97 pgs: 97 active+clean; 30 MiB data, 657 MiB used, 1.0 TiB / 1.0 TiB avail; 930 KiB/s rd, 1.5 MiB/s wr, 468 op/s 2024-03-28T00:30:08.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:30:08 smithi165 ceph-mon[36302]: pgmap v110: 97 pgs: 97 active+clean; 30 MiB data, 657 MiB used, 1.0 TiB / 1.0 TiB avail; 585 KiB/s rd, 756 KiB/s wr, 251 op/s 2024-03-28T00:30:08.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:30:08 smithi119 ceph-mon[31518]: pgmap v110: 97 pgs: 97 active+clean; 30 MiB data, 657 MiB used, 1.0 TiB / 1.0 TiB avail; 585 KiB/s rd, 756 KiB/s wr, 251 op/s 2024-03-28T00:30:08.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:30:08 smithi132 ceph-mon[35941]: pgmap v110: 97 pgs: 97 active+clean; 30 MiB data, 657 MiB used, 1.0 TiB / 1.0 TiB avail; 585 KiB/s rd, 756 KiB/s wr, 251 op/s 2024-03-28T00:30:10.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:30:10 smithi119 ceph-mon[31518]: pgmap v111: 97 pgs: 97 active+clean; 45 MiB data, 663 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 271 op/s 2024-03-28T00:30:10.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:30:10 smithi165 ceph-mon[36302]: pgmap v111: 97 pgs: 97 active+clean; 45 MiB data, 663 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 271 op/s 2024-03-28T00:30:10.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:30:10 smithi132 ceph-mon[35941]: pgmap v111: 97 pgs: 97 active+clean; 45 MiB data, 663 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 271 op/s 2024-03-28T00:30:12.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-mon[36302]: pgmap v112: 97 pgs: 97 active+clean; 45 MiB data, 663 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 66 op/s 2024-03-28T00:30:12.634 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7fe475334700 time 2024-03-28T00:30:12.317567+0000 2024-03-28T00:30:12.634 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T00:30:12.634 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:30:12.634 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fe47f95e4a1] 2024-03-28T00:30:12.635 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7fe47f95e667] 2024-03-28T00:30:12.635 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x555d87791223] 2024-03-28T00:30:12.635 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x555d8760b5d6] 2024-03-28T00:30:12.635 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x555d8760bda1] 2024-03-28T00:30:12.635 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: 6: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x555d8764009c] 2024-03-28T00:30:12.635 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x555d8765e233] 2024-03-28T00:30:12.635 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x555d87660197] 2024-03-28T00:30:12.636 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x555d8755f11b] 2024-03-28T00:30:12.636 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x555d8755f6bc] 2024-03-28T00:30:12.636 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x555d875491ff] 2024-03-28T00:30:12.636 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7fe47fbf0488] 2024-03-28T00:30:12.636 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: 13: (DispatchQueue::entry()+0x51f) [0x7fe47fbed60f] 2024-03-28T00:30:12.636 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fe47fcc0e81] 2024-03-28T00:30:12.636 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fe47e6db1ca] 2024-03-28T00:30:12.636 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: 16: clone() 2024-03-28T00:30:12.637 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: 2024-03-28T00:30:12.318+0000 7fe475334700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7fe475334700 time 2024-03-28T00:30:12.317567+0000 2024-03-28T00:30:12.638 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T00:30:12.638 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: 2024-03-28T00:30:12.638 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:30:12.638 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fe47f95e4a1] 2024-03-28T00:30:12.638 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7fe47f95e667] 2024-03-28T00:30:12.638 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x555d87791223] 2024-03-28T00:30:12.638 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x555d8760b5d6] 2024-03-28T00:30:12.638 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x555d8760bda1] 2024-03-28T00:30:12.638 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: 6: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x555d8764009c] 2024-03-28T00:30:12.638 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x555d8765e233] 2024-03-28T00:30:12.638 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x555d87660197] 2024-03-28T00:30:12.639 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x555d8755f11b] 2024-03-28T00:30:12.639 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x555d8755f6bc] 2024-03-28T00:30:12.639 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x555d875491ff] 2024-03-28T00:30:12.639 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7fe47fbf0488] 2024-03-28T00:30:12.639 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: 13: (DispatchQueue::entry()+0x51f) [0x7fe47fbed60f] 2024-03-28T00:30:12.639 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fe47fcc0e81] 2024-03-28T00:30:12.639 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fe47e6db1ca] 2024-03-28T00:30:12.639 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: 16: clone() 2024-03-28T00:30:12.639 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: 2024-03-28T00:30:12.639 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: *** Caught signal (Aborted) ** 2024-03-28T00:30:12.639 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: in thread 7fe475334700 thread_name:ms_dispatch 2024-03-28T00:30:12.640 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:30:12.640 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fe47e6e5d20] 2024-03-28T00:30:12.640 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: 2: gsignal() 2024-03-28T00:30:12.640 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: 3: abort() 2024-03-28T00:30:12.640 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fe47f95e4fb] 2024-03-28T00:30:12.640 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7fe47f95e667] 2024-03-28T00:30:12.640 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x555d87791223] 2024-03-28T00:30:12.640 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x555d8760b5d6] 2024-03-28T00:30:12.640 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x555d8760bda1] 2024-03-28T00:30:12.640 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: 9: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x555d8764009c] 2024-03-28T00:30:12.640 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x555d8765e233] 2024-03-28T00:30:12.640 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x555d87660197] 2024-03-28T00:30:12.640 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x555d8755f11b] 2024-03-28T00:30:12.641 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x555d8755f6bc] 2024-03-28T00:30:12.641 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x555d875491ff] 2024-03-28T00:30:12.641 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7fe47fbf0488] 2024-03-28T00:30:12.641 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: 16: (DispatchQueue::entry()+0x51f) [0x7fe47fbed60f] 2024-03-28T00:30:12.641 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fe47fcc0e81] 2024-03-28T00:30:12.641 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7fe47e6db1ca] 2024-03-28T00:30:12.641 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: 19: clone() 2024-03-28T00:30:12.641 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: 2024-03-28T00:30:12.320+0000 7fe475334700 -1 *** Caught signal (Aborted) ** 2024-03-28T00:30:12.641 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: in thread 7fe475334700 thread_name:ms_dispatch 2024-03-28T00:30:12.641 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: 2024-03-28T00:30:12.641 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:30:12.641 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fe47e6e5d20] 2024-03-28T00:30:12.641 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: 2: gsignal() 2024-03-28T00:30:12.641 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: 3: abort() 2024-03-28T00:30:12.642 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fe47f95e4fb] 2024-03-28T00:30:12.642 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7fe47f95e667] 2024-03-28T00:30:12.642 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x555d87791223] 2024-03-28T00:30:12.642 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x555d8760b5d6] 2024-03-28T00:30:12.642 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x555d8760bda1] 2024-03-28T00:30:12.642 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: 9: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x555d8764009c] 2024-03-28T00:30:12.642 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x555d8765e233] 2024-03-28T00:30:12.642 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x555d87660197] 2024-03-28T00:30:12.642 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x555d8755f11b] 2024-03-28T00:30:12.642 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x555d8755f6bc] 2024-03-28T00:30:12.642 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x555d875491ff] 2024-03-28T00:30:12.642 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7fe47fbf0488] 2024-03-28T00:30:12.643 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: 16: (DispatchQueue::entry()+0x51f) [0x7fe47fbed60f] 2024-03-28T00:30:12.643 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fe47fcc0e81] 2024-03-28T00:30:12.643 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7fe47e6db1ca] 2024-03-28T00:30:12.643 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: 19: clone() 2024-03-28T00:30:12.643 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T00:30:12.643 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: 2024-03-28T00:30:12.643 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: -1> 2024-03-28T00:30:12.318+0000 7fe475334700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7fe475334700 time 2024-03-28T00:30:12.317567+0000 2024-03-28T00:30:12.643 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T00:30:12.643 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: 2024-03-28T00:30:12.643 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:30:12.643 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fe47f95e4a1] 2024-03-28T00:30:12.643 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7fe47f95e667] 2024-03-28T00:30:12.643 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x555d87791223] 2024-03-28T00:30:12.644 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x555d8760b5d6] 2024-03-28T00:30:12.644 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x555d8760bda1] 2024-03-28T00:30:12.644 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: 6: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x555d8764009c] 2024-03-28T00:30:12.644 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x555d8765e233] 2024-03-28T00:30:12.644 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x555d87660197] 2024-03-28T00:30:12.644 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x555d8755f11b] 2024-03-28T00:30:12.644 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x555d8755f6bc] 2024-03-28T00:30:12.644 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x555d875491ff] 2024-03-28T00:30:12.644 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7fe47fbf0488] 2024-03-28T00:30:12.644 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: 13: (DispatchQueue::entry()+0x51f) [0x7fe47fbed60f] 2024-03-28T00:30:12.644 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fe47fcc0e81] 2024-03-28T00:30:12.644 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fe47e6db1ca] 2024-03-28T00:30:12.644 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: 16: clone() 2024-03-28T00:30:12.645 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: 2024-03-28T00:30:12.645 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: 0> 2024-03-28T00:30:12.320+0000 7fe475334700 -1 *** Caught signal (Aborted) ** 2024-03-28T00:30:12.645 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: in thread 7fe475334700 thread_name:ms_dispatch 2024-03-28T00:30:12.645 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: 2024-03-28T00:30:12.645 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:30:12.645 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fe47e6e5d20] 2024-03-28T00:30:12.645 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: 2: gsignal() 2024-03-28T00:30:12.645 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: 3: abort() 2024-03-28T00:30:12.645 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fe47f95e4fb] 2024-03-28T00:30:12.645 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7fe47f95e667] 2024-03-28T00:30:12.645 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x555d87791223] 2024-03-28T00:30:12.646 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x555d8760b5d6] 2024-03-28T00:30:12.646 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x555d8760bda1] 2024-03-28T00:30:12.646 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: 9: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x555d8764009c] 2024-03-28T00:30:12.646 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x555d8765e233] 2024-03-28T00:30:12.646 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x555d87660197] 2024-03-28T00:30:12.646 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x555d8755f11b] 2024-03-28T00:30:12.646 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x555d8755f6bc] 2024-03-28T00:30:12.646 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x555d875491ff] 2024-03-28T00:30:12.646 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7fe47fbf0488] 2024-03-28T00:30:12.646 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: 16: (DispatchQueue::entry()+0x51f) [0x7fe47fbed60f] 2024-03-28T00:30:12.646 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fe47fcc0e81] 2024-03-28T00:30:12.646 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7fe47e6db1ca] 2024-03-28T00:30:12.647 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: 19: clone() 2024-03-28T00:30:12.647 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T00:30:12.647 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: 2024-03-28T00:30:12.647 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: -1> 2024-03-28T00:30:12.318+0000 7fe475334700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7fe475334700 time 2024-03-28T00:30:12.317567+0000 2024-03-28T00:30:12.647 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T00:30:12.647 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: 2024-03-28T00:30:12.647 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:30:12.647 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fe47f95e4a1] 2024-03-28T00:30:12.647 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7fe47f95e667] 2024-03-28T00:30:12.647 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x555d87791223] 2024-03-28T00:30:12.647 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x555d8760b5d6] 2024-03-28T00:30:12.647 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x555d8760bda1] 2024-03-28T00:30:12.648 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: 6: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x555d8764009c] 2024-03-28T00:30:12.648 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x555d8765e233] 2024-03-28T00:30:12.648 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x555d87660197] 2024-03-28T00:30:12.648 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x555d8755f11b] 2024-03-28T00:30:12.648 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x555d8755f6bc] 2024-03-28T00:30:12.648 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x555d875491ff] 2024-03-28T00:30:12.648 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7fe47fbf0488] 2024-03-28T00:30:12.648 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: 13: (DispatchQueue::entry()+0x51f) [0x7fe47fbed60f] 2024-03-28T00:30:12.649 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fe47fcc0e81] 2024-03-28T00:30:12.649 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fe47e6db1ca] 2024-03-28T00:30:12.649 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: 16: clone() 2024-03-28T00:30:12.649 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: 2024-03-28T00:30:12.649 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: 0> 2024-03-28T00:30:12.320+0000 7fe475334700 -1 *** Caught signal (Aborted) ** 2024-03-28T00:30:12.649 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: in thread 7fe475334700 thread_name:ms_dispatch 2024-03-28T00:30:12.649 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: 2024-03-28T00:30:12.649 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:30:12.649 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fe47e6e5d20] 2024-03-28T00:30:12.649 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: 2: gsignal() 2024-03-28T00:30:12.649 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: 3: abort() 2024-03-28T00:30:12.649 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fe47f95e4fb] 2024-03-28T00:30:12.649 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7fe47f95e667] 2024-03-28T00:30:12.650 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x555d87791223] 2024-03-28T00:30:12.650 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x555d8760b5d6] 2024-03-28T00:30:12.650 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x555d8760bda1] 2024-03-28T00:30:12.650 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: 9: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x555d8764009c] 2024-03-28T00:30:12.650 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x555d8765e233] 2024-03-28T00:30:12.650 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x555d87660197] 2024-03-28T00:30:12.650 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x555d8755f11b] 2024-03-28T00:30:12.650 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x555d8755f6bc] 2024-03-28T00:30:12.650 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x555d875491ff] 2024-03-28T00:30:12.650 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7fe47fbf0488] 2024-03-28T00:30:12.650 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: 16: (DispatchQueue::entry()+0x51f) [0x7fe47fbed60f] 2024-03-28T00:30:12.650 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fe47fcc0e81] 2024-03-28T00:30:12.650 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7fe47e6db1ca] 2024-03-28T00:30:12.651 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: 19: clone() 2024-03-28T00:30:12.651 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T00:30:12.651 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[67764]: 2024-03-28T00:30:12.651 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:12 smithi165 podman[71443]: 2024-03-28 00:30:12.400373689 +0000 UTC m=+0.025117526 container died 1af8232810b86e2c9f926f5ce25fa81bd44baaeb9883d5ea6068a59d7732e4fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-03-28T00:30:12.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:30:12 smithi119 ceph-mon[31518]: pgmap v112: 97 pgs: 97 active+clean; 45 MiB data, 663 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 66 op/s 2024-03-28T00:30:12.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:30:12 smithi132 ceph-mon[35941]: pgmap v112: 97 pgs: 97 active+clean; 45 MiB data, 663 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 66 op/s 2024-03-28T00:30:13.455 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:13 smithi165 podman[71443]: 2024-03-28 00:30:13.183583444 +0000 UTC m=+0.808327268 container remove 1af8232810b86e2c9f926f5ce25fa81bd44baaeb9883d5ea6068a59d7732e4fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-28T00:30:13.456 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:13 smithi165 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.c.service: Main process exited, code=exited, status=134/n/a 2024-03-28T00:30:13.883 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:13 smithi165 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.c.service: Failed with result 'exit-code'. 2024-03-28T00:30:13.883 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:13 smithi165 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.c.service: Consumed 2min 358ms CPU time. 2024-03-28T00:30:14.878 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:30:14 smithi119 ceph-mon[31518]: pgmap v113: 97 pgs: 97 active+clean; 45 MiB data, 663 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 66 op/s 2024-03-28T00:30:14.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:30:14 smithi165 ceph-mon[36302]: pgmap v113: 97 pgs: 97 active+clean; 45 MiB data, 663 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 66 op/s 2024-03-28T00:30:14.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:30:14 smithi132 ceph-mon[35941]: pgmap v113: 97 pgs: 97 active+clean; 45 MiB data, 663 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 66 op/s 2024-03-28T00:30:16.862 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:30:16 smithi119 ceph-mon[31518]: pgmap v114: 97 pgs: 97 active+clean; 58 MiB data, 665 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.5 MiB/s wr, 113 op/s 2024-03-28T00:30:16.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:30:16 smithi165 ceph-mon[36302]: pgmap v114: 97 pgs: 97 active+clean; 58 MiB data, 665 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.5 MiB/s wr, 113 op/s 2024-03-28T00:30:16.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:30:16 smithi132 ceph-mon[35941]: pgmap v114: 97 pgs: 97 active+clean; 58 MiB data, 665 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.5 MiB/s wr, 113 op/s 2024-03-28T00:30:18.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:30:18 smithi165 ceph-mon[36302]: pgmap v115: 97 pgs: 97 active+clean; 58 MiB data, 665 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.4 MiB/s wr, 66 op/s 2024-03-28T00:30:18.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:30:18 smithi119 ceph-mon[31518]: pgmap v115: 97 pgs: 97 active+clean; 58 MiB data, 665 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.4 MiB/s wr, 66 op/s 2024-03-28T00:30:18.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:30:18 smithi132 ceph-mon[35941]: pgmap v115: 97 pgs: 97 active+clean; 58 MiB data, 665 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.4 MiB/s wr, 66 op/s 2024-03-28T00:30:20.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:30:20 smithi165 ceph-mon[36302]: pgmap v116: 97 pgs: 97 active+clean; 67 MiB data, 725 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.2 MiB/s wr, 231 op/s 2024-03-28T00:30:20.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:30:20 smithi119 ceph-mon[31518]: pgmap v116: 97 pgs: 97 active+clean; 67 MiB data, 725 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.2 MiB/s wr, 231 op/s 2024-03-28T00:30:20.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:30:20 smithi132 ceph-mon[35941]: pgmap v116: 97 pgs: 97 active+clean; 67 MiB data, 725 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.2 MiB/s wr, 231 op/s 2024-03-28T00:30:22.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:30:22 smithi165 ceph-mon[36302]: pgmap v117: 97 pgs: 97 active+clean; 67 MiB data, 725 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.9 MiB/s wr, 211 op/s 2024-03-28T00:30:22.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:30:22 smithi119 ceph-mon[31518]: pgmap v117: 97 pgs: 97 active+clean; 67 MiB data, 725 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.9 MiB/s wr, 211 op/s 2024-03-28T00:30:22.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:30:22 smithi132 ceph-mon[35941]: pgmap v117: 97 pgs: 97 active+clean; 67 MiB data, 725 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.9 MiB/s wr, 211 op/s 2024-03-28T00:30:23.898 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:23 smithi165 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.c.service: Scheduled restart job, restart counter is at 1. 2024-03-28T00:30:23.899 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:23 smithi165 systemd[1]: Stopped Ceph mds.c for 10f73b92-ec98-11ee-95d0-87774f69a715. 2024-03-28T00:30:23.899 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:23 smithi165 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.c.service: Consumed 2min 358ms CPU time. 2024-03-28T00:30:23.899 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:23 smithi165 systemd[1]: Starting Ceph mds.c for 10f73b92-ec98-11ee-95d0-87774f69a715... 2024-03-28T00:30:23.899 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:23 smithi165 podman[71536]: 2024-03-28 00:30:23.76032339 +0000 UTC m=+0.018634906 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-28T00:30:23.899 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:23 smithi165 podman[71536]: 2024-03-28 00:30:23.892882106 +0000 UTC m=+0.151193657 container create e1e5b5efd905cecd15d3ac02e60ca6ace279f424f76609b9350e629bd33c4be2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240319) 2024-03-28T00:30:24.210 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:24 smithi165 podman[71536]: 2024-03-28 00:30:24.103760816 +0000 UTC m=+0.362072354 container init e1e5b5efd905cecd15d3ac02e60ca6ace279f424f76609b9350e629bd33c4be2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-28T00:30:24.210 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:24 smithi165 podman[71536]: 2024-03-28 00:30:24.106782048 +0000 UTC m=+0.365093563 container start e1e5b5efd905cecd15d3ac02e60ca6ace279f424f76609b9350e629bd33c4be2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-03-28T00:30:24.210 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:24 smithi165 ceph-mds[71554]: -- 172.21.15.165:0/1127086818 <== mon.1 v2:172.21.15.165:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55d1fa2efa40 con 0x55d1fa2c3000 2024-03-28T00:30:24.210 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: starting mds.c at 2024-03-28T00:30:24.542 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:24 smithi165 bash[71536]: e1e5b5efd905cecd15d3ac02e60ca6ace279f424f76609b9350e629bd33c4be2 2024-03-28T00:30:24.542 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:24 smithi165 systemd[1]: Started Ceph mds.c for 10f73b92-ec98-11ee-95d0-87774f69a715. 2024-03-28T00:30:24.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:30:24 smithi165 ceph-mon[36302]: pgmap v118: 97 pgs: 97 active+clean; 67 MiB data, 725 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.9 MiB/s wr, 211 op/s 2024-03-28T00:30:24.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:30:24 smithi165 ceph-mon[36302]: daemon mds.c restarted 2024-03-28T00:30:24.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:30:24 smithi165 ceph-mon[36302]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-28T00:30:24.884 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:30:24 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-28T00:30:24.884 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:30:24 smithi165 ceph-mon[36302]: osdmap e89: 12 total, 12 up, 12 in 2024-03-28T00:30:24.884 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:30:24 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.165:6836/1456402510,v1:172.21.15.165:6837/1456402510] up:boot 2024-03-28T00:30:24.884 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:30:24 smithi165 ceph-mon[36302]: Standby daemon mds.l assigned to filesystem cephfs as rank 3 2024-03-28T00:30:24.884 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:30:24 smithi165 ceph-mon[36302]: fsmap cephfs:4/5 {0=b=up:active,1=i=up:active,2=h=up:active,4=f=up:active} 5 up:standby-replay 3 up:standby, 1 failed 2024-03-28T00:30:24.884 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:30:24 smithi165 ceph-mon[36302]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:replay 4 up:active 2024-03-28T00:30:24.907 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:30:24 smithi119 ceph-mon[31518]: pgmap v118: 97 pgs: 97 active+clean; 67 MiB data, 725 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.9 MiB/s wr, 211 op/s 2024-03-28T00:30:24.907 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:30:24 smithi119 ceph-mon[31518]: daemon mds.c restarted 2024-03-28T00:30:24.907 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:30:24 smithi119 ceph-mon[31518]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-28T00:30:24.907 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:30:24 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-28T00:30:24.907 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:30:24 smithi119 ceph-mon[31518]: osdmap e89: 12 total, 12 up, 12 in 2024-03-28T00:30:24.907 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:30:24 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.165:6836/1456402510,v1:172.21.15.165:6837/1456402510] up:boot 2024-03-28T00:30:24.907 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:30:24 smithi119 ceph-mon[31518]: Standby daemon mds.l assigned to filesystem cephfs as rank 3 2024-03-28T00:30:24.907 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:30:24 smithi119 ceph-mon[31518]: fsmap cephfs:4/5 {0=b=up:active,1=i=up:active,2=h=up:active,4=f=up:active} 5 up:standby-replay 3 up:standby, 1 failed 2024-03-28T00:30:24.907 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:30:24 smithi119 ceph-mon[31518]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:replay 4 up:active 2024-03-28T00:30:24.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:30:24 smithi132 ceph-mon[35941]: pgmap v118: 97 pgs: 97 active+clean; 67 MiB data, 725 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.9 MiB/s wr, 211 op/s 2024-03-28T00:30:24.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:30:24 smithi132 ceph-mon[35941]: daemon mds.c restarted 2024-03-28T00:30:24.954 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:30:24 smithi132 ceph-mon[35941]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-28T00:30:24.954 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:30:24 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-28T00:30:24.954 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:30:24 smithi132 ceph-mon[35941]: osdmap e89: 12 total, 12 up, 12 in 2024-03-28T00:30:24.954 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:30:24 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.165:6836/1456402510,v1:172.21.15.165:6837/1456402510] up:boot 2024-03-28T00:30:24.954 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:30:24 smithi132 ceph-mon[35941]: Standby daemon mds.l assigned to filesystem cephfs as rank 3 2024-03-28T00:30:24.954 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:30:24 smithi132 ceph-mon[35941]: fsmap cephfs:4/5 {0=b=up:active,1=i=up:active,2=h=up:active,4=f=up:active} 5 up:standby-replay 3 up:standby, 1 failed 2024-03-28T00:30:24.954 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:30:24 smithi132 ceph-mon[35941]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:replay 4 up:active 2024-03-28T00:30:26.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:30:26 smithi165 ceph-mon[36302]: pgmap v120: 97 pgs: 97 active+clean; 69 MiB data, 766 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 296 op/s 2024-03-28T00:30:26.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:30:26 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.165:6832/609963168,v1:172.21.15.165:6833/609963168] up:resolve 2024-03-28T00:30:26.884 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:30:26 smithi165 ceph-mon[36302]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:resolve 4 up:active 2024-03-28T00:30:26.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:30:26 smithi119 ceph-mon[31518]: pgmap v120: 97 pgs: 97 active+clean; 69 MiB data, 766 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 296 op/s 2024-03-28T00:30:26.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:30:26 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.165:6832/609963168,v1:172.21.15.165:6833/609963168] up:resolve 2024-03-28T00:30:26.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:30:26 smithi119 ceph-mon[31518]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:resolve 4 up:active 2024-03-28T00:30:26.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:30:26 smithi132 ceph-mon[35941]: pgmap v120: 97 pgs: 97 active+clean; 69 MiB data, 766 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 296 op/s 2024-03-28T00:30:26.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:30:26 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.165:6832/609963168,v1:172.21.15.165:6833/609963168] up:resolve 2024-03-28T00:30:26.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:30:26 smithi132 ceph-mon[35941]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:resolve 4 up:active 2024-03-28T00:30:27.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:30:27 smithi165 ceph-mon[36302]: reconnect by client.25355 172.21.15.119:0/3071152482 after 0.00199997 2024-03-28T00:30:27.884 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:30:27 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.165:6832/609963168,v1:172.21.15.165:6833/609963168] up:reconnect 2024-03-28T00:30:27.884 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:30:27 smithi165 ceph-mon[36302]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:reconnect 4 up:active 2024-03-28T00:30:27.884 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:30:27 smithi165 ceph-mon[36302]: reconnect by client.25264 v1:172.21.15.119:0/3863511511 after 0.0299996 2024-03-28T00:30:27.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:30:27 smithi119 ceph-mon[31518]: reconnect by client.25355 172.21.15.119:0/3071152482 after 0.00199997 2024-03-28T00:30:27.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:30:27 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.165:6832/609963168,v1:172.21.15.165:6833/609963168] up:reconnect 2024-03-28T00:30:27.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:30:27 smithi119 ceph-mon[31518]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:reconnect 4 up:active 2024-03-28T00:30:27.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:30:27 smithi119 ceph-mon[31518]: reconnect by client.25264 v1:172.21.15.119:0/3863511511 after 0.0299996 2024-03-28T00:30:27.954 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:30:27 smithi132 ceph-mon[35941]: reconnect by client.25355 172.21.15.119:0/3071152482 after 0.00199997 2024-03-28T00:30:27.954 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:30:27 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.165:6832/609963168,v1:172.21.15.165:6833/609963168] up:reconnect 2024-03-28T00:30:27.954 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:30:27 smithi132 ceph-mon[35941]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:reconnect 4 up:active 2024-03-28T00:30:27.954 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:30:27 smithi132 ceph-mon[35941]: reconnect by client.25264 v1:172.21.15.119:0/3863511511 after 0.0299996 2024-03-28T00:30:28.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:30:28 smithi165 ceph-mon[36302]: pgmap v121: 97 pgs: 97 active+clean; 69 MiB data, 766 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 296 op/s 2024-03-28T00:30:28.884 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:30:28 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.165:6832/609963168,v1:172.21.15.165:6833/609963168] up:rejoin 2024-03-28T00:30:28.884 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:30:28 smithi165 ceph-mon[36302]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:rejoin 4 up:active 2024-03-28T00:30:28.884 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:30:28 smithi165 ceph-mon[36302]: daemon mds.l is now active in filesystem cephfs as rank 3 2024-03-28T00:30:28.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:30:28 smithi119 ceph-mon[31518]: pgmap v121: 97 pgs: 97 active+clean; 69 MiB data, 766 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 296 op/s 2024-03-28T00:30:28.926 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:30:28 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.165:6832/609963168,v1:172.21.15.165:6833/609963168] up:rejoin 2024-03-28T00:30:28.927 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:30:28 smithi119 ceph-mon[31518]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:rejoin 4 up:active 2024-03-28T00:30:28.927 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:30:28 smithi119 ceph-mon[31518]: daemon mds.l is now active in filesystem cephfs as rank 3 2024-03-28T00:30:28.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:30:28 smithi132 ceph-mon[35941]: pgmap v121: 97 pgs: 97 active+clean; 69 MiB data, 766 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 296 op/s 2024-03-28T00:30:28.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:30:28 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.165:6832/609963168,v1:172.21.15.165:6833/609963168] up:rejoin 2024-03-28T00:30:28.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:30:28 smithi132 ceph-mon[35941]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:rejoin 4 up:active 2024-03-28T00:30:28.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:30:28 smithi132 ceph-mon[35941]: daemon mds.l is now active in filesystem cephfs as rank 3 2024-03-28T00:30:29.384 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:28 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7feb8936f700 time 2024-03-28T00:30:28.985737+0000 2024-03-28T00:30:29.384 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:28 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T00:30:29.384 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:28 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:30:29.384 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:28 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7feb939994a1] 2024-03-28T00:30:29.385 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:28 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7feb93999667] 2024-03-28T00:30:29.385 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:28 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55a2c189d223] 2024-03-28T00:30:29.385 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:28 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55a2c17175d6] 2024-03-28T00:30:29.385 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:28 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55a2c1717da1] 2024-03-28T00:30:29.385 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:28 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: 6: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x55a2c174c09c] 2024-03-28T00:30:29.385 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:28 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55a2c176a233] 2024-03-28T00:30:29.385 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:28 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55a2c176c197] 2024-03-28T00:30:29.385 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:28 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55a2c166b11b] 2024-03-28T00:30:29.385 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:28 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55a2c166b6bc] 2024-03-28T00:30:29.385 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:28 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55a2c16551ff] 2024-03-28T00:30:29.385 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:28 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7feb93c2b488] 2024-03-28T00:30:29.386 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:28 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: 13: (DispatchQueue::entry()+0x51f) [0x7feb93c2860f] 2024-03-28T00:30:29.386 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:28 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7feb93cfbe81] 2024-03-28T00:30:29.386 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:28 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7feb927161ca] 2024-03-28T00:30:29.386 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:28 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: 16: clone() 2024-03-28T00:30:29.386 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:28 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: *** Caught signal (Aborted) ** 2024-03-28T00:30:29.386 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:28 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: in thread 7feb8936f700 thread_name:ms_dispatch 2024-03-28T00:30:29.386 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:28 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: 2024-03-28T00:30:28.986+0000 7feb8936f700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7feb8936f700 time 2024-03-28T00:30:28.985737+0000 2024-03-28T00:30:29.386 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:28 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T00:30:29.386 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:28 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: 2024-03-28T00:30:29.386 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:28 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:30:29.386 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:28 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7feb939994a1] 2024-03-28T00:30:29.387 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:28 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7feb93999667] 2024-03-28T00:30:29.387 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:28 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55a2c189d223] 2024-03-28T00:30:29.387 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:28 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55a2c17175d6] 2024-03-28T00:30:29.387 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:28 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55a2c1717da1] 2024-03-28T00:30:29.387 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:28 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: 6: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x55a2c174c09c] 2024-03-28T00:30:29.387 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:28 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55a2c176a233] 2024-03-28T00:30:29.387 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:28 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55a2c176c197] 2024-03-28T00:30:29.387 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:28 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55a2c166b11b] 2024-03-28T00:30:29.387 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:28 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55a2c166b6bc] 2024-03-28T00:30:29.387 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:28 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55a2c16551ff] 2024-03-28T00:30:29.387 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:28 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7feb93c2b488] 2024-03-28T00:30:29.387 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:28 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: 13: (DispatchQueue::entry()+0x51f) [0x7feb93c2860f] 2024-03-28T00:30:29.387 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:28 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7feb93cfbe81] 2024-03-28T00:30:29.388 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:28 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7feb927161ca] 2024-03-28T00:30:29.388 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:28 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: 16: clone() 2024-03-28T00:30:29.388 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:28 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: 2024-03-28T00:30:29.388 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:28 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:30:29.388 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:28 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7feb92720d20] 2024-03-28T00:30:29.389 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:28 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: 2: gsignal() 2024-03-28T00:30:29.389 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:28 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: 3: abort() 2024-03-28T00:30:29.389 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:28 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7feb939994fb] 2024-03-28T00:30:29.389 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:28 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7feb93999667] 2024-03-28T00:30:29.390 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:28 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55a2c189d223] 2024-03-28T00:30:29.390 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:28 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55a2c17175d6] 2024-03-28T00:30:29.390 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:28 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55a2c1717da1] 2024-03-28T00:30:29.390 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:28 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: 9: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x55a2c174c09c] 2024-03-28T00:30:29.390 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:28 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55a2c176a233] 2024-03-28T00:30:29.390 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:28 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55a2c176c197] 2024-03-28T00:30:29.390 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:28 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55a2c166b11b] 2024-03-28T00:30:29.390 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:28 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55a2c166b6bc] 2024-03-28T00:30:29.390 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:28 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55a2c16551ff] 2024-03-28T00:30:29.390 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:28 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7feb93c2b488] 2024-03-28T00:30:29.390 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:28 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: 16: (DispatchQueue::entry()+0x51f) [0x7feb93c2860f] 2024-03-28T00:30:29.391 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:28 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7feb93cfbe81] 2024-03-28T00:30:29.391 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:28 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7feb927161ca] 2024-03-28T00:30:29.391 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:28 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: 19: clone() 2024-03-28T00:30:29.391 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:28 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: 2024-03-28T00:30:28.989+0000 7feb8936f700 -1 *** Caught signal (Aborted) ** 2024-03-28T00:30:29.391 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:28 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: in thread 7feb8936f700 thread_name:ms_dispatch 2024-03-28T00:30:29.391 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:28 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: 2024-03-28T00:30:29.391 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:28 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:30:29.391 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:28 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7feb92720d20] 2024-03-28T00:30:29.391 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:28 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: 2: gsignal() 2024-03-28T00:30:29.391 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:28 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: 3: abort() 2024-03-28T00:30:29.391 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:28 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7feb939994fb] 2024-03-28T00:30:29.391 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:28 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7feb93999667] 2024-03-28T00:30:29.391 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:28 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55a2c189d223] 2024-03-28T00:30:29.392 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:28 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55a2c17175d6] 2024-03-28T00:30:29.392 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:28 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55a2c1717da1] 2024-03-28T00:30:29.392 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:28 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: 9: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x55a2c174c09c] 2024-03-28T00:30:29.392 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:28 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55a2c176a233] 2024-03-28T00:30:29.392 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:28 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55a2c176c197] 2024-03-28T00:30:29.392 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:28 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55a2c166b11b] 2024-03-28T00:30:29.392 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:28 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55a2c166b6bc] 2024-03-28T00:30:29.392 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:28 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55a2c16551ff] 2024-03-28T00:30:29.392 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:28 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7feb93c2b488] 2024-03-28T00:30:29.392 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:28 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: 16: (DispatchQueue::entry()+0x51f) [0x7feb93c2860f] 2024-03-28T00:30:29.392 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:28 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7feb93cfbe81] 2024-03-28T00:30:29.392 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:28 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7feb927161ca] 2024-03-28T00:30:29.393 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:28 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: 19: clone() 2024-03-28T00:30:29.393 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:28 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T00:30:29.393 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:28 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: 2024-03-28T00:30:29.393 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:29 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: -2> 2024-03-28T00:30:28.986+0000 7feb8936f700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7feb8936f700 time 2024-03-28T00:30:28.985737+0000 2024-03-28T00:30:29.393 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:29 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T00:30:29.393 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:29 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: 2024-03-28T00:30:29.393 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:29 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:30:29.393 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:29 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7feb939994a1] 2024-03-28T00:30:29.393 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:29 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7feb93999667] 2024-03-28T00:30:29.393 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:29 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55a2c189d223] 2024-03-28T00:30:29.393 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:29 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55a2c17175d6] 2024-03-28T00:30:29.393 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:29 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55a2c1717da1] 2024-03-28T00:30:29.394 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:29 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: 6: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x55a2c174c09c] 2024-03-28T00:30:29.394 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:29 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55a2c176a233] 2024-03-28T00:30:29.394 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:29 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55a2c176c197] 2024-03-28T00:30:29.394 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:29 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55a2c166b11b] 2024-03-28T00:30:29.394 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:29 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55a2c166b6bc] 2024-03-28T00:30:29.394 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:29 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55a2c16551ff] 2024-03-28T00:30:29.394 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:29 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7feb93c2b488] 2024-03-28T00:30:29.394 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:29 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: 13: (DispatchQueue::entry()+0x51f) [0x7feb93c2860f] 2024-03-28T00:30:29.394 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:29 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7feb93cfbe81] 2024-03-28T00:30:29.394 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:29 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7feb927161ca] 2024-03-28T00:30:29.394 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:29 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: 16: clone() 2024-03-28T00:30:29.395 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:29 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: 2024-03-28T00:30:29.395 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:29 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: 0> 2024-03-28T00:30:28.989+0000 7feb8936f700 -1 *** Caught signal (Aborted) ** 2024-03-28T00:30:29.395 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:29 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: in thread 7feb8936f700 thread_name:ms_dispatch 2024-03-28T00:30:29.395 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:29 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: 2024-03-28T00:30:29.395 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:29 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:30:29.395 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:29 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7feb92720d20] 2024-03-28T00:30:29.395 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:29 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: 2: gsignal() 2024-03-28T00:30:29.395 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:29 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: 3: abort() 2024-03-28T00:30:29.395 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:29 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7feb939994fb] 2024-03-28T00:30:29.395 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:29 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7feb93999667] 2024-03-28T00:30:29.395 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:29 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55a2c189d223] 2024-03-28T00:30:29.395 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:29 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55a2c17175d6] 2024-03-28T00:30:29.396 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:29 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55a2c1717da1] 2024-03-28T00:30:29.396 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:29 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: 9: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x55a2c174c09c] 2024-03-28T00:30:29.396 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:29 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55a2c176a233] 2024-03-28T00:30:29.396 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:29 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55a2c176c197] 2024-03-28T00:30:29.396 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:29 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55a2c166b11b] 2024-03-28T00:30:29.396 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:29 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55a2c166b6bc] 2024-03-28T00:30:29.396 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:29 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55a2c16551ff] 2024-03-28T00:30:29.396 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:29 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7feb93c2b488] 2024-03-28T00:30:29.396 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:29 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: 16: (DispatchQueue::entry()+0x51f) [0x7feb93c2860f] 2024-03-28T00:30:29.396 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:29 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7feb93cfbe81] 2024-03-28T00:30:29.396 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:29 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7feb927161ca] 2024-03-28T00:30:29.396 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:29 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: 19: clone() 2024-03-28T00:30:29.396 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:29 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T00:30:29.397 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:29 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: 2024-03-28T00:30:29.397 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:29 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: -2> 2024-03-28T00:30:28.986+0000 7feb8936f700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7feb8936f700 time 2024-03-28T00:30:28.985737+0000 2024-03-28T00:30:29.397 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:29 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T00:30:29.397 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:29 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: 2024-03-28T00:30:29.397 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:29 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:30:29.397 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:29 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7feb939994a1] 2024-03-28T00:30:29.397 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:29 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7feb93999667] 2024-03-28T00:30:29.397 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:29 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55a2c189d223] 2024-03-28T00:30:29.397 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:29 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55a2c17175d6] 2024-03-28T00:30:29.397 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:29 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55a2c1717da1] 2024-03-28T00:30:29.397 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:29 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: 6: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x55a2c174c09c] 2024-03-28T00:30:29.397 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:29 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55a2c176a233] 2024-03-28T00:30:29.398 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:29 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55a2c176c197] 2024-03-28T00:30:29.398 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:29 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55a2c166b11b] 2024-03-28T00:30:29.398 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:29 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55a2c166b6bc] 2024-03-28T00:30:29.398 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:29 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55a2c16551ff] 2024-03-28T00:30:29.398 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:29 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7feb93c2b488] 2024-03-28T00:30:29.398 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:29 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: 13: (DispatchQueue::entry()+0x51f) [0x7feb93c2860f] 2024-03-28T00:30:29.398 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:29 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7feb93cfbe81] 2024-03-28T00:30:29.398 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:29 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7feb927161ca] 2024-03-28T00:30:29.398 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:29 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: 16: clone() 2024-03-28T00:30:29.398 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:29 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: 2024-03-28T00:30:29.398 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:29 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: 0> 2024-03-28T00:30:28.989+0000 7feb8936f700 -1 *** Caught signal (Aborted) ** 2024-03-28T00:30:29.399 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:29 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: in thread 7feb8936f700 thread_name:ms_dispatch 2024-03-28T00:30:29.399 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:29 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: 2024-03-28T00:30:29.399 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:29 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:30:29.399 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:29 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7feb92720d20] 2024-03-28T00:30:29.399 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:29 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: 2: gsignal() 2024-03-28T00:30:29.399 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:29 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: 3: abort() 2024-03-28T00:30:29.399 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:29 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7feb939994fb] 2024-03-28T00:30:29.399 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:29 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7feb93999667] 2024-03-28T00:30:29.399 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:29 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55a2c189d223] 2024-03-28T00:30:29.399 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:29 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55a2c17175d6] 2024-03-28T00:30:29.400 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:29 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55a2c1717da1] 2024-03-28T00:30:29.400 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:29 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: 9: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x55a2c174c09c] 2024-03-28T00:30:29.400 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:29 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55a2c176a233] 2024-03-28T00:30:29.400 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:29 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55a2c176c197] 2024-03-28T00:30:29.400 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:29 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55a2c166b11b] 2024-03-28T00:30:29.400 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:29 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55a2c166b6bc] 2024-03-28T00:30:29.400 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:29 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55a2c16551ff] 2024-03-28T00:30:29.400 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:29 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7feb93c2b488] 2024-03-28T00:30:29.400 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:29 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: 16: (DispatchQueue::entry()+0x51f) [0x7feb93c2860f] 2024-03-28T00:30:29.400 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:29 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7feb93cfbe81] 2024-03-28T00:30:29.400 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:29 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7feb927161ca] 2024-03-28T00:30:29.400 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:29 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: 19: clone() 2024-03-28T00:30:29.400 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:29 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T00:30:29.400 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:29 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[67144]: 2024-03-28T00:30:29.401 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:29 smithi165 podman[71588]: 2024-03-28 00:30:29.069338205 +0000 UTC m=+0.025227897 container died fa62f3785e42c16fb08c1a22e89922df56595b0015bf38d1fc63b909bde70e78 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-28T00:30:29.453 INFO:journalctl@ceph.mds.b.smithi132.stdout:Mar 28 00:30:29 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-b[66443]: 2024-03-28T00:30:29.035+0000 7f857cecc700 -1 mds.pinger is_rank_lagging: rank=3 was never sent ping request. 2024-03-28T00:30:29.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:30:29 smithi165 ceph-mon[36302]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-28T00:30:29.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:30:29 smithi165 ceph-mon[36302]: Cluster is now healthy 2024-03-28T00:30:29.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:30:29 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.165:6832/609963168,v1:172.21.15.165:6833/609963168] up:active 2024-03-28T00:30:29.884 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:30:29 smithi165 ceph-mon[36302]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-03-28T00:30:29.884 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:30:29 smithi165 ceph-mon[36302]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-28T00:30:29.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:30:29 smithi119 ceph-mon[31518]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-28T00:30:29.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:30:29 smithi119 ceph-mon[31518]: Cluster is now healthy 2024-03-28T00:30:29.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:30:29 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.165:6832/609963168,v1:172.21.15.165:6833/609963168] up:active 2024-03-28T00:30:29.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:30:29 smithi119 ceph-mon[31518]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-03-28T00:30:29.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:30:29 smithi119 ceph-mon[31518]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-28T00:30:29.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:30:29 smithi132 ceph-mon[35941]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-28T00:30:29.954 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:30:29 smithi132 ceph-mon[35941]: Cluster is now healthy 2024-03-28T00:30:29.954 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:30:29 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.165:6832/609963168,v1:172.21.15.165:6833/609963168] up:active 2024-03-28T00:30:29.954 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:30:29 smithi132 ceph-mon[35941]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-03-28T00:30:29.954 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:30:29 smithi132 ceph-mon[35941]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-28T00:30:30.259 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:29 smithi165 podman[71588]: 2024-03-28 00:30:29.967605772 +0000 UTC m=+0.923495453 container remove fa62f3785e42c16fb08c1a22e89922df56595b0015bf38d1fc63b909bde70e78 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319) 2024-03-28T00:30:30.259 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:29 smithi165 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.l.service: Main process exited, code=exited, status=134/n/a 2024-03-28T00:30:30.592 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:30 smithi165 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.l.service: Failed with result 'exit-code'. 2024-03-28T00:30:30.593 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:30 smithi165 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.l.service: Consumed 1.911s CPU time. 2024-03-28T00:30:30.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:30:30 smithi165 ceph-mon[36302]: pgmap v122: 97 pgs: 97 active+clean; 82 MiB data, 798 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 205 op/s 2024-03-28T00:30:30.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:30:30 smithi119 ceph-mon[31518]: pgmap v122: 97 pgs: 97 active+clean; 82 MiB data, 798 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 205 op/s 2024-03-28T00:30:30.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:30:30 smithi132 ceph-mon[35941]: pgmap v122: 97 pgs: 97 active+clean; 82 MiB data, 798 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 205 op/s 2024-03-28T00:30:32.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:30:32 smithi165 ceph-mon[36302]: pgmap v123: 97 pgs: 97 active+clean; 82 MiB data, 798 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 205 op/s 2024-03-28T00:30:32.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:30:32 smithi119 ceph-mon[31518]: pgmap v123: 97 pgs: 97 active+clean; 82 MiB data, 798 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 205 op/s 2024-03-28T00:30:32.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:30:32 smithi132 ceph-mon[35941]: pgmap v123: 97 pgs: 97 active+clean; 82 MiB data, 798 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 205 op/s 2024-03-28T00:30:34.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:30:34 smithi165 ceph-mon[36302]: pgmap v124: 97 pgs: 97 active+clean; 82 MiB data, 798 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 205 op/s 2024-03-28T00:30:34.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:30:34 smithi119 ceph-mon[31518]: pgmap v124: 97 pgs: 97 active+clean; 82 MiB data, 798 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 205 op/s 2024-03-28T00:30:34.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:30:34 smithi132 ceph-mon[35941]: pgmap v124: 97 pgs: 97 active+clean; 82 MiB data, 798 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 205 op/s 2024-03-28T00:30:36.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:30:36 smithi165 ceph-mon[36302]: pgmap v125: 97 pgs: 97 active+clean; 88 MiB data, 899 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 310 op/s 2024-03-28T00:30:36.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:30:36 smithi119 ceph-mon[31518]: pgmap v125: 97 pgs: 97 active+clean; 88 MiB data, 899 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 310 op/s 2024-03-28T00:30:36.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:30:36 smithi132 ceph-mon[35941]: pgmap v125: 97 pgs: 97 active+clean; 88 MiB data, 899 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 310 op/s 2024-03-28T00:30:38.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:30:38 smithi165 ceph-mon[36302]: pgmap v126: 97 pgs: 97 active+clean; 88 MiB data, 899 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 282 op/s 2024-03-28T00:30:38.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:30:38 smithi119 ceph-mon[31518]: pgmap v126: 97 pgs: 97 active+clean; 88 MiB data, 899 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 282 op/s 2024-03-28T00:30:38.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:30:38 smithi132 ceph-mon[35941]: pgmap v126: 97 pgs: 97 active+clean; 88 MiB data, 899 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 282 op/s 2024-03-28T00:30:40.726 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:30:40 smithi165 ceph-mon[36302]: pgmap v127: 97 pgs: 97 active+clean; 92 MiB data, 973 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 377 op/s 2024-03-28T00:30:40.726 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:40 smithi165 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.l.service: Scheduled restart job, restart counter is at 1. 2024-03-28T00:30:40.726 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:40 smithi165 systemd[1]: Stopped Ceph mds.l for 10f73b92-ec98-11ee-95d0-87774f69a715. 2024-03-28T00:30:40.726 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:40 smithi165 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.l.service: Consumed 1.911s CPU time. 2024-03-28T00:30:40.726 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:40 smithi165 systemd[1]: Starting Ceph mds.l for 10f73b92-ec98-11ee-95d0-87774f69a715... 2024-03-28T00:30:40.726 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:40 smithi165 podman[71679]: 2024-03-28 00:30:40.531195821 +0000 UTC m=+0.024569032 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-28T00:30:40.726 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:40 smithi165 podman[71679]: 2024-03-28 00:30:40.719983769 +0000 UTC m=+0.213356967 container create e64abea411d6f06622eeb4a41cd4517895cdab1ca3c8e014bc942203d8390807 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True) 2024-03-28T00:30:40.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:30:40 smithi119 ceph-mon[31518]: pgmap v127: 97 pgs: 97 active+clean; 92 MiB data, 973 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 377 op/s 2024-03-28T00:30:40.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:30:40 smithi132 ceph-mon[35941]: pgmap v127: 97 pgs: 97 active+clean; 92 MiB data, 973 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 377 op/s 2024-03-28T00:30:41.384 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:41 smithi165 podman[71679]: 2024-03-28 00:30:41.254039102 +0000 UTC m=+0.747412304 container init e64abea411d6f06622eeb4a41cd4517895cdab1ca3c8e014bc942203d8390807 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-03-28T00:30:41.384 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:41 smithi165 podman[71679]: 2024-03-28 00:30:41.257043611 +0000 UTC m=+0.750416802 container start e64abea411d6f06622eeb4a41cd4517895cdab1ca3c8e014bc942203d8390807 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-03-28T00:30:41.384 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:41 smithi165 ceph-mds[71697]: -- 172.21.15.165:0/153042977 <== mon.1 v2:172.21.15.165:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55e7a9c2fa40 con 0x55e7a9c03000 2024-03-28T00:30:41.384 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:41 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: starting mds.l at 2024-03-28T00:30:41.384 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:41 smithi165 bash[71679]: e64abea411d6f06622eeb4a41cd4517895cdab1ca3c8e014bc942203d8390807 2024-03-28T00:30:41.384 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:30:41 smithi165 systemd[1]: Started Ceph mds.l for 10f73b92-ec98-11ee-95d0-87774f69a715. 2024-03-28T00:30:41.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:30:41 smithi165 ceph-mon[36302]: daemon mds.l restarted 2024-03-28T00:30:41.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:30:41 smithi165 ceph-mon[36302]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-28T00:30:41.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:30:41 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-28T00:30:41.884 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:30:41 smithi165 ceph-mon[36302]: osdmap e90: 12 total, 12 up, 12 in 2024-03-28T00:30:41.884 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:30:41 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.165:6832/1389081045,v1:172.21.15.165:6833/1389081045] up:boot 2024-03-28T00:30:41.884 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:30:41 smithi165 ceph-mon[36302]: Standby daemon mds.c assigned to filesystem cephfs as rank 3 2024-03-28T00:30:41.884 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:30:41 smithi165 ceph-mon[36302]: fsmap cephfs:4/5 {0=b=up:active,1=i=up:active,2=h=up:active,4=f=up:active} 5 up:standby-replay 3 up:standby, 1 failed 2024-03-28T00:30:41.884 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:30:41 smithi165 ceph-mon[36302]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:replay 4 up:active 2024-03-28T00:30:41.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:30:41 smithi119 ceph-mon[31518]: daemon mds.l restarted 2024-03-28T00:30:41.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:30:41 smithi119 ceph-mon[31518]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-28T00:30:41.907 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:30:41 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-28T00:30:41.907 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:30:41 smithi119 ceph-mon[31518]: osdmap e90: 12 total, 12 up, 12 in 2024-03-28T00:30:41.907 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:30:41 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.165:6832/1389081045,v1:172.21.15.165:6833/1389081045] up:boot 2024-03-28T00:30:41.907 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:30:41 smithi119 ceph-mon[31518]: Standby daemon mds.c assigned to filesystem cephfs as rank 3 2024-03-28T00:30:41.907 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:30:41 smithi119 ceph-mon[31518]: fsmap cephfs:4/5 {0=b=up:active,1=i=up:active,2=h=up:active,4=f=up:active} 5 up:standby-replay 3 up:standby, 1 failed 2024-03-28T00:30:41.907 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:30:41 smithi119 ceph-mon[31518]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:replay 4 up:active 2024-03-28T00:30:41.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:30:41 smithi132 ceph-mon[35941]: daemon mds.l restarted 2024-03-28T00:30:41.954 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:30:41 smithi132 ceph-mon[35941]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-28T00:30:41.954 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:30:41 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-28T00:30:41.954 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:30:41 smithi132 ceph-mon[35941]: osdmap e90: 12 total, 12 up, 12 in 2024-03-28T00:30:41.954 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:30:41 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.165:6832/1389081045,v1:172.21.15.165:6833/1389081045] up:boot 2024-03-28T00:30:41.954 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:30:41 smithi132 ceph-mon[35941]: Standby daemon mds.c assigned to filesystem cephfs as rank 3 2024-03-28T00:30:41.954 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:30:41 smithi132 ceph-mon[35941]: fsmap cephfs:4/5 {0=b=up:active,1=i=up:active,2=h=up:active,4=f=up:active} 5 up:standby-replay 3 up:standby, 1 failed 2024-03-28T00:30:41.954 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:30:41 smithi132 ceph-mon[35941]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:replay 4 up:active 2024-03-28T00:30:42.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:30:42 smithi165 ceph-mon[36302]: pgmap v128: 97 pgs: 97 active+clean; 92 MiB data, 973 MiB used, 1.0 TiB / 1.0 TiB avail; 997 KiB/s rd, 1.0 MiB/s wr, 289 op/s 2024-03-28T00:30:42.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:30:42 smithi119 ceph-mon[31518]: pgmap v128: 97 pgs: 97 active+clean; 92 MiB data, 973 MiB used, 1.0 TiB / 1.0 TiB avail; 997 KiB/s rd, 1.0 MiB/s wr, 289 op/s 2024-03-28T00:30:42.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:30:42 smithi132 ceph-mon[35941]: pgmap v128: 97 pgs: 97 active+clean; 92 MiB data, 973 MiB used, 1.0 TiB / 1.0 TiB avail; 997 KiB/s rd, 1.0 MiB/s wr, 289 op/s 2024-03-28T00:30:43.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:30:43 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.165:6836/1456402510,v1:172.21.15.165:6837/1456402510] up:resolve 2024-03-28T00:30:43.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:30:43 smithi119 ceph-mon[31518]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:resolve 4 up:active 2024-03-28T00:30:43.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:30:43 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.165:6836/1456402510,v1:172.21.15.165:6837/1456402510] up:resolve 2024-03-28T00:30:43.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:30:43 smithi132 ceph-mon[35941]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:resolve 4 up:active 2024-03-28T00:30:44.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:30:43 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.165:6836/1456402510,v1:172.21.15.165:6837/1456402510] up:resolve 2024-03-28T00:30:44.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:30:43 smithi165 ceph-mon[36302]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:resolve 4 up:active 2024-03-28T00:30:44.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:30:44 smithi119 ceph-mon[31518]: pgmap v130: 97 pgs: 97 active+clean; 92 MiB data, 973 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 347 op/s 2024-03-28T00:30:44.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:30:44 smithi119 ceph-mon[31518]: reconnect by client.25355 172.21.15.119:0/3071152482 after 0.00199997 2024-03-28T00:30:44.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:30:44 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.165:6836/1456402510,v1:172.21.15.165:6837/1456402510] up:reconnect 2024-03-28T00:30:44.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:30:44 smithi119 ceph-mon[31518]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:reconnect 4 up:active 2024-03-28T00:30:44.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:30:44 smithi119 ceph-mon[31518]: reconnect by client.25264 v1:172.21.15.119:0/3863511511 after 0.0139998 2024-03-28T00:30:44.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:30:44 smithi132 ceph-mon[35941]: pgmap v130: 97 pgs: 97 active+clean; 92 MiB data, 973 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 347 op/s 2024-03-28T00:30:44.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:30:44 smithi132 ceph-mon[35941]: reconnect by client.25355 172.21.15.119:0/3071152482 after 0.00199997 2024-03-28T00:30:44.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:30:44 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.165:6836/1456402510,v1:172.21.15.165:6837/1456402510] up:reconnect 2024-03-28T00:30:44.954 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:30:44 smithi132 ceph-mon[35941]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:reconnect 4 up:active 2024-03-28T00:30:44.954 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:30:44 smithi132 ceph-mon[35941]: reconnect by client.25264 v1:172.21.15.119:0/3863511511 after 0.0139998 2024-03-28T00:30:45.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-mon[36302]: pgmap v130: 97 pgs: 97 active+clean; 92 MiB data, 973 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 347 op/s 2024-03-28T00:30:45.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-mon[36302]: reconnect by client.25355 172.21.15.119:0/3071152482 after 0.00199997 2024-03-28T00:30:45.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.165:6836/1456402510,v1:172.21.15.165:6837/1456402510] up:reconnect 2024-03-28T00:30:45.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-mon[36302]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:reconnect 4 up:active 2024-03-28T00:30:45.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-mon[36302]: reconnect by client.25264 v1:172.21.15.119:0/3863511511 after 0.0139998 2024-03-28T00:30:45.134 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7f74e410a700 time 2024-03-28T00:30:44.783239+0000 2024-03-28T00:30:45.134 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T00:30:45.134 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:30:45.134 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f74ee7344a1] 2024-03-28T00:30:45.135 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f74ee734667] 2024-03-28T00:30:45.135 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55d1f7e5f223] 2024-03-28T00:30:45.135 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55d1f7cd95d6] 2024-03-28T00:30:45.135 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55d1f7cd9da1] 2024-03-28T00:30:45.135 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: 6: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x55d1f7d0e09c] 2024-03-28T00:30:45.135 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55d1f7d2c233] 2024-03-28T00:30:45.135 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55d1f7d2e197] 2024-03-28T00:30:45.135 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55d1f7c2d11b] 2024-03-28T00:30:45.135 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55d1f7c2d6bc] 2024-03-28T00:30:45.135 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55d1f7c171ff] 2024-03-28T00:30:45.135 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f74ee9c6488] 2024-03-28T00:30:45.135 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: 13: (DispatchQueue::entry()+0x51f) [0x7f74ee9c360f] 2024-03-28T00:30:45.135 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f74eea96e81] 2024-03-28T00:30:45.135 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f74ed4b11ca] 2024-03-28T00:30:45.136 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: 16: clone() 2024-03-28T00:30:45.136 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: 2024-03-28T00:30:44.784+0000 7f74e410a700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7f74e410a700 time 2024-03-28T00:30:44.783239+0000 2024-03-28T00:30:45.136 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T00:30:45.136 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: 2024-03-28T00:30:45.136 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:30:45.136 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f74ee7344a1] 2024-03-28T00:30:45.136 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f74ee734667] 2024-03-28T00:30:45.136 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55d1f7e5f223] 2024-03-28T00:30:45.136 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55d1f7cd95d6] 2024-03-28T00:30:45.136 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55d1f7cd9da1] 2024-03-28T00:30:45.136 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: 6: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x55d1f7d0e09c] 2024-03-28T00:30:45.136 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55d1f7d2c233] 2024-03-28T00:30:45.136 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55d1f7d2e197] 2024-03-28T00:30:45.136 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55d1f7c2d11b] 2024-03-28T00:30:45.137 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55d1f7c2d6bc] 2024-03-28T00:30:45.137 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55d1f7c171ff] 2024-03-28T00:30:45.137 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f74ee9c6488] 2024-03-28T00:30:45.137 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: 13: (DispatchQueue::entry()+0x51f) [0x7f74ee9c360f] 2024-03-28T00:30:45.137 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f74eea96e81] 2024-03-28T00:30:45.137 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f74ed4b11ca] 2024-03-28T00:30:45.137 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: 16: clone() 2024-03-28T00:30:45.137 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: 2024-03-28T00:30:45.137 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: *** Caught signal (Aborted) ** 2024-03-28T00:30:45.137 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: in thread 7f74e410a700 thread_name:ms_dispatch 2024-03-28T00:30:45.137 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:30:45.137 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f74ed4bbd20] 2024-03-28T00:30:45.138 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: 2: gsignal() 2024-03-28T00:30:45.138 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: 3: abort() 2024-03-28T00:30:45.139 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f74ee7344fb] 2024-03-28T00:30:45.139 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f74ee734667] 2024-03-28T00:30:45.139 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55d1f7e5f223] 2024-03-28T00:30:45.139 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55d1f7cd95d6] 2024-03-28T00:30:45.139 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55d1f7cd9da1] 2024-03-28T00:30:45.139 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: 9: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x55d1f7d0e09c] 2024-03-28T00:30:45.139 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55d1f7d2c233] 2024-03-28T00:30:45.139 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55d1f7d2e197] 2024-03-28T00:30:45.139 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55d1f7c2d11b] 2024-03-28T00:30:45.139 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55d1f7c2d6bc] 2024-03-28T00:30:45.139 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55d1f7c171ff] 2024-03-28T00:30:45.139 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f74ee9c6488] 2024-03-28T00:30:45.139 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: 16: (DispatchQueue::entry()+0x51f) [0x7f74ee9c360f] 2024-03-28T00:30:45.139 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f74eea96e81] 2024-03-28T00:30:45.140 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f74ed4b11ca] 2024-03-28T00:30:45.140 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: 19: clone() 2024-03-28T00:30:45.140 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: 2024-03-28T00:30:44.786+0000 7f74e410a700 -1 *** Caught signal (Aborted) ** 2024-03-28T00:30:45.140 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: in thread 7f74e410a700 thread_name:ms_dispatch 2024-03-28T00:30:45.140 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: 2024-03-28T00:30:45.140 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:30:45.140 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f74ed4bbd20] 2024-03-28T00:30:45.140 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: 2: gsignal() 2024-03-28T00:30:45.140 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: 3: abort() 2024-03-28T00:30:45.140 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f74ee7344fb] 2024-03-28T00:30:45.140 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f74ee734667] 2024-03-28T00:30:45.140 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55d1f7e5f223] 2024-03-28T00:30:45.140 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55d1f7cd95d6] 2024-03-28T00:30:45.140 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55d1f7cd9da1] 2024-03-28T00:30:45.140 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: 9: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x55d1f7d0e09c] 2024-03-28T00:30:45.141 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55d1f7d2c233] 2024-03-28T00:30:45.141 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55d1f7d2e197] 2024-03-28T00:30:45.141 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55d1f7c2d11b] 2024-03-28T00:30:45.141 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55d1f7c2d6bc] 2024-03-28T00:30:45.141 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55d1f7c171ff] 2024-03-28T00:30:45.141 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f74ee9c6488] 2024-03-28T00:30:45.141 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: 16: (DispatchQueue::entry()+0x51f) [0x7f74ee9c360f] 2024-03-28T00:30:45.141 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f74eea96e81] 2024-03-28T00:30:45.141 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f74ed4b11ca] 2024-03-28T00:30:45.141 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: 19: clone() 2024-03-28T00:30:45.141 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T00:30:45.141 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: 2024-03-28T00:30:45.141 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: -1> 2024-03-28T00:30:44.784+0000 7f74e410a700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7f74e410a700 time 2024-03-28T00:30:44.783239+0000 2024-03-28T00:30:45.141 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T00:30:45.141 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: 2024-03-28T00:30:45.142 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:30:45.142 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f74ee7344a1] 2024-03-28T00:30:45.142 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f74ee734667] 2024-03-28T00:30:45.142 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55d1f7e5f223] 2024-03-28T00:30:45.142 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55d1f7cd95d6] 2024-03-28T00:30:45.142 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55d1f7cd9da1] 2024-03-28T00:30:45.142 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: 6: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x55d1f7d0e09c] 2024-03-28T00:30:45.142 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55d1f7d2c233] 2024-03-28T00:30:45.142 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55d1f7d2e197] 2024-03-28T00:30:45.142 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55d1f7c2d11b] 2024-03-28T00:30:45.142 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55d1f7c2d6bc] 2024-03-28T00:30:45.142 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55d1f7c171ff] 2024-03-28T00:30:45.142 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f74ee9c6488] 2024-03-28T00:30:45.142 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: 13: (DispatchQueue::entry()+0x51f) [0x7f74ee9c360f] 2024-03-28T00:30:45.143 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f74eea96e81] 2024-03-28T00:30:45.143 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f74ed4b11ca] 2024-03-28T00:30:45.143 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: 16: clone() 2024-03-28T00:30:45.143 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: 2024-03-28T00:30:45.143 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: 0> 2024-03-28T00:30:44.786+0000 7f74e410a700 -1 *** Caught signal (Aborted) ** 2024-03-28T00:30:45.143 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: in thread 7f74e410a700 thread_name:ms_dispatch 2024-03-28T00:30:45.143 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: 2024-03-28T00:30:45.143 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:30:45.143 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f74ed4bbd20] 2024-03-28T00:30:45.143 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: 2: gsignal() 2024-03-28T00:30:45.143 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: 3: abort() 2024-03-28T00:30:45.143 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f74ee7344fb] 2024-03-28T00:30:45.143 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f74ee734667] 2024-03-28T00:30:45.143 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55d1f7e5f223] 2024-03-28T00:30:45.143 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55d1f7cd95d6] 2024-03-28T00:30:45.144 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55d1f7cd9da1] 2024-03-28T00:30:45.144 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: 9: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x55d1f7d0e09c] 2024-03-28T00:30:45.144 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55d1f7d2c233] 2024-03-28T00:30:45.144 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55d1f7d2e197] 2024-03-28T00:30:45.144 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55d1f7c2d11b] 2024-03-28T00:30:45.144 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55d1f7c2d6bc] 2024-03-28T00:30:45.144 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55d1f7c171ff] 2024-03-28T00:30:45.144 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f74ee9c6488] 2024-03-28T00:30:45.144 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: 16: (DispatchQueue::entry()+0x51f) [0x7f74ee9c360f] 2024-03-28T00:30:45.144 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f74eea96e81] 2024-03-28T00:30:45.144 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f74ed4b11ca] 2024-03-28T00:30:45.144 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: 19: clone() 2024-03-28T00:30:45.144 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T00:30:45.144 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: 2024-03-28T00:30:45.144 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: -1> 2024-03-28T00:30:44.784+0000 7f74e410a700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7f74e410a700 time 2024-03-28T00:30:44.783239+0000 2024-03-28T00:30:45.145 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T00:30:45.145 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: 2024-03-28T00:30:45.145 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:30:45.145 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f74ee7344a1] 2024-03-28T00:30:45.145 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f74ee734667] 2024-03-28T00:30:45.145 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55d1f7e5f223] 2024-03-28T00:30:45.145 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55d1f7cd95d6] 2024-03-28T00:30:45.145 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55d1f7cd9da1] 2024-03-28T00:30:45.145 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: 6: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x55d1f7d0e09c] 2024-03-28T00:30:45.145 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55d1f7d2c233] 2024-03-28T00:30:45.145 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55d1f7d2e197] 2024-03-28T00:30:45.145 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55d1f7c2d11b] 2024-03-28T00:30:45.145 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55d1f7c2d6bc] 2024-03-28T00:30:45.145 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55d1f7c171ff] 2024-03-28T00:30:45.145 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f74ee9c6488] 2024-03-28T00:30:45.146 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: 13: (DispatchQueue::entry()+0x51f) [0x7f74ee9c360f] 2024-03-28T00:30:45.146 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f74eea96e81] 2024-03-28T00:30:45.146 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f74ed4b11ca] 2024-03-28T00:30:45.146 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: 16: clone() 2024-03-28T00:30:45.146 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: 2024-03-28T00:30:45.146 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: 0> 2024-03-28T00:30:44.786+0000 7f74e410a700 -1 *** Caught signal (Aborted) ** 2024-03-28T00:30:45.146 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: in thread 7f74e410a700 thread_name:ms_dispatch 2024-03-28T00:30:45.146 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: 2024-03-28T00:30:45.146 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:30:45.146 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f74ed4bbd20] 2024-03-28T00:30:45.146 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: 2: gsignal() 2024-03-28T00:30:45.146 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: 3: abort() 2024-03-28T00:30:45.146 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f74ee7344fb] 2024-03-28T00:30:45.146 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f74ee734667] 2024-03-28T00:30:45.146 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55d1f7e5f223] 2024-03-28T00:30:45.147 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55d1f7cd95d6] 2024-03-28T00:30:45.147 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55d1f7cd9da1] 2024-03-28T00:30:45.147 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: 9: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x55d1f7d0e09c] 2024-03-28T00:30:45.147 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55d1f7d2c233] 2024-03-28T00:30:45.147 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55d1f7d2e197] 2024-03-28T00:30:45.147 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55d1f7c2d11b] 2024-03-28T00:30:45.147 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55d1f7c2d6bc] 2024-03-28T00:30:45.147 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55d1f7c171ff] 2024-03-28T00:30:45.147 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f74ee9c6488] 2024-03-28T00:30:45.147 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: 16: (DispatchQueue::entry()+0x51f) [0x7f74ee9c360f] 2024-03-28T00:30:45.147 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f74eea96e81] 2024-03-28T00:30:45.147 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f74ed4b11ca] 2024-03-28T00:30:45.147 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: 19: clone() 2024-03-28T00:30:45.147 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T00:30:45.147 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71550]: 2024-03-28T00:30:45.148 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:44 smithi165 podman[71731]: 2024-03-28 00:30:44.867045718 +0000 UTC m=+0.028321610 container died e1e5b5efd905cecd15d3ac02e60ca6ace279f424f76609b9350e629bd33c4be2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319) 2024-03-28T00:30:45.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:30:45 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.165:6836/1456402510,v1:172.21.15.165:6837/1456402510] up:rejoin 2024-03-28T00:30:45.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:30:45 smithi165 ceph-mon[36302]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:rejoin 4 up:active 2024-03-28T00:30:45.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:30:45 smithi165 ceph-mon[36302]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-03-28T00:30:45.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:30:45 smithi165 ceph-mon[36302]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-28T00:30:45.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:30:45 smithi165 ceph-mon[36302]: Cluster is now healthy 2024-03-28T00:30:45.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:30:45 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.165:6836/1456402510,v1:172.21.15.165:6837/1456402510] up:active 2024-03-28T00:30:45.816 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:30:45 smithi165 ceph-mon[36302]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-28T00:30:45.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:30:45 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.165:6836/1456402510,v1:172.21.15.165:6837/1456402510] up:rejoin 2024-03-28T00:30:45.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:30:45 smithi119 ceph-mon[31518]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:rejoin 4 up:active 2024-03-28T00:30:45.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:30:45 smithi119 ceph-mon[31518]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-03-28T00:30:45.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:30:45 smithi119 ceph-mon[31518]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-28T00:30:45.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:30:45 smithi119 ceph-mon[31518]: Cluster is now healthy 2024-03-28T00:30:45.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:30:45 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.165:6836/1456402510,v1:172.21.15.165:6837/1456402510] up:active 2024-03-28T00:30:45.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:30:45 smithi119 ceph-mon[31518]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-28T00:30:45.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:30:45 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.165:6836/1456402510,v1:172.21.15.165:6837/1456402510] up:rejoin 2024-03-28T00:30:45.954 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:30:45 smithi132 ceph-mon[35941]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:rejoin 4 up:active 2024-03-28T00:30:45.954 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:30:45 smithi132 ceph-mon[35941]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-03-28T00:30:45.954 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:30:45 smithi132 ceph-mon[35941]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-28T00:30:45.954 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:30:45 smithi132 ceph-mon[35941]: Cluster is now healthy 2024-03-28T00:30:45.954 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:30:45 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.165:6836/1456402510,v1:172.21.15.165:6837/1456402510] up:active 2024-03-28T00:30:45.954 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:30:45 smithi132 ceph-mon[35941]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-28T00:30:46.088 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:45 smithi165 podman[71731]: 2024-03-28 00:30:45.809932063 +0000 UTC m=+0.971207954 container remove e1e5b5efd905cecd15d3ac02e60ca6ace279f424f76609b9350e629bd33c4be2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-03-28T00:30:46.088 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:45 smithi165 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.c.service: Main process exited, code=exited, status=134/n/a 2024-03-28T00:30:46.383 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:46 smithi165 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.c.service: Failed with result 'exit-code'. 2024-03-28T00:30:46.383 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:46 smithi165 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.c.service: Consumed 1.010s CPU time. 2024-03-28T00:30:46.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:30:46 smithi132 ceph-mon[35941]: pgmap v131: 97 pgs: 97 active+clean; 94 MiB data, 990 MiB used, 1.0 TiB / 1.0 TiB avail; 615 KiB/s rd, 785 KiB/s wr, 202 op/s 2024-03-28T00:30:47.093 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:30:46 smithi119 ceph-mon[31518]: pgmap v131: 97 pgs: 97 active+clean; 94 MiB data, 990 MiB used, 1.0 TiB / 1.0 TiB avail; 615 KiB/s rd, 785 KiB/s wr, 202 op/s 2024-03-28T00:30:47.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:30:46 smithi165 ceph-mon[36302]: pgmap v131: 97 pgs: 97 active+clean; 94 MiB data, 990 MiB used, 1.0 TiB / 1.0 TiB avail; 615 KiB/s rd, 785 KiB/s wr, 202 op/s 2024-03-28T00:30:48.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:30:48 smithi132 ceph-mon[35941]: pgmap v132: 97 pgs: 97 active+clean; 94 MiB data, 990 MiB used, 1.0 TiB / 1.0 TiB avail; 615 KiB/s rd, 785 KiB/s wr, 202 op/s 2024-03-28T00:30:49.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:30:48 smithi165 ceph-mon[36302]: pgmap v132: 97 pgs: 97 active+clean; 94 MiB data, 990 MiB used, 1.0 TiB / 1.0 TiB avail; 615 KiB/s rd, 785 KiB/s wr, 202 op/s 2024-03-28T00:30:49.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:30:48 smithi119 ceph-mon[31518]: pgmap v132: 97 pgs: 97 active+clean; 94 MiB data, 990 MiB used, 1.0 TiB / 1.0 TiB avail; 615 KiB/s rd, 785 KiB/s wr, 202 op/s 2024-03-28T00:30:50.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:30:50 smithi132 ceph-mon[35941]: pgmap v133: 97 pgs: 97 active+clean; 99 MiB data, 1011 MiB used, 1.0 TiB / 1.0 TiB avail; 839 KiB/s rd, 1.0 MiB/s wr, 246 op/s 2024-03-28T00:30:51.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:30:50 smithi165 ceph-mon[36302]: pgmap v133: 97 pgs: 97 active+clean; 99 MiB data, 1011 MiB used, 1.0 TiB / 1.0 TiB avail; 839 KiB/s rd, 1.0 MiB/s wr, 246 op/s 2024-03-28T00:30:51.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:30:50 smithi119 ceph-mon[31518]: pgmap v133: 97 pgs: 97 active+clean; 99 MiB data, 1011 MiB used, 1.0 TiB / 1.0 TiB avail; 839 KiB/s rd, 1.0 MiB/s wr, 246 op/s 2024-03-28T00:30:52.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:30:52 smithi132 ceph-mon[35941]: pgmap v134: 97 pgs: 97 active+clean; 99 MiB data, 1011 MiB used, 1.0 TiB / 1.0 TiB avail; 839 KiB/s rd, 1.0 MiB/s wr, 246 op/s 2024-03-28T00:30:53.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:30:52 smithi165 ceph-mon[36302]: pgmap v134: 97 pgs: 97 active+clean; 99 MiB data, 1011 MiB used, 1.0 TiB / 1.0 TiB avail; 839 KiB/s rd, 1.0 MiB/s wr, 246 op/s 2024-03-28T00:30:53.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:30:52 smithi119 ceph-mon[31518]: pgmap v134: 97 pgs: 97 active+clean; 99 MiB data, 1011 MiB used, 1.0 TiB / 1.0 TiB avail; 839 KiB/s rd, 1.0 MiB/s wr, 246 op/s 2024-03-28T00:30:55.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:30:54 smithi165 ceph-mon[36302]: pgmap v135: 97 pgs: 97 active+clean; 99 MiB data, 1011 MiB used, 1.0 TiB / 1.0 TiB avail; 712 KiB/s rd, 893 KiB/s wr, 208 op/s 2024-03-28T00:30:55.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:30:54 smithi119 ceph-mon[31518]: pgmap v135: 97 pgs: 97 active+clean; 99 MiB data, 1011 MiB used, 1.0 TiB / 1.0 TiB avail; 712 KiB/s rd, 893 KiB/s wr, 208 op/s 2024-03-28T00:30:55.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:30:54 smithi132 ceph-mon[35941]: pgmap v135: 97 pgs: 97 active+clean; 99 MiB data, 1011 MiB used, 1.0 TiB / 1.0 TiB avail; 712 KiB/s rd, 893 KiB/s wr, 208 op/s 2024-03-28T00:30:56.400 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:56 smithi165 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.c.service: Scheduled restart job, restart counter is at 2. 2024-03-28T00:30:56.400 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:56 smithi165 systemd[1]: Stopped Ceph mds.c for 10f73b92-ec98-11ee-95d0-87774f69a715. 2024-03-28T00:30:56.400 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:56 smithi165 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.c.service: Consumed 1.010s CPU time. 2024-03-28T00:30:56.400 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:56 smithi165 systemd[1]: Starting Ceph mds.c for 10f73b92-ec98-11ee-95d0-87774f69a715... 2024-03-28T00:30:56.400 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:56 smithi165 podman[71827]: 2024-03-28 00:30:56.265717055 +0000 UTC m=+0.019416495 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-28T00:30:56.400 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:56 smithi165 podman[71827]: 2024-03-28 00:30:56.394013881 +0000 UTC m=+0.147713308 container create 0d115520da96cf874a08bf0f58fcde1967850ff57c94e5d2e697ead2bffb8478 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-28T00:30:57.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:30:56 smithi165 ceph-mon[36302]: pgmap v136: 97 pgs: 97 active+clean; 104 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 271 op/s 2024-03-28T00:30:57.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:30:56 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:30:57.134 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:56 smithi165 podman[71827]: 2024-03-28 00:30:56.671768242 +0000 UTC m=+0.425467682 container init 0d115520da96cf874a08bf0f58fcde1967850ff57c94e5d2e697ead2bffb8478 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-28T00:30:57.134 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:56 smithi165 podman[71827]: 2024-03-28 00:30:56.675334597 +0000 UTC m=+0.429034027 container start 0d115520da96cf874a08bf0f58fcde1967850ff57c94e5d2e697ead2bffb8478 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c) 2024-03-28T00:30:57.134 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:56 smithi165 ceph-mds[71911]: -- 172.21.15.165:0/3783003737 <== mon.1 v2:172.21.15.165:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5557c4385a40 con 0x5557c4358800 2024-03-28T00:30:57.134 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:56 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: starting mds.c at 2024-03-28T00:30:57.134 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:56 smithi165 bash[71827]: 0d115520da96cf874a08bf0f58fcde1967850ff57c94e5d2e697ead2bffb8478 2024-03-28T00:30:57.134 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:30:56 smithi165 systemd[1]: Started Ceph mds.c for 10f73b92-ec98-11ee-95d0-87774f69a715. 2024-03-28T00:30:57.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:30:56 smithi119 ceph-mon[31518]: pgmap v136: 97 pgs: 97 active+clean; 104 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 271 op/s 2024-03-28T00:30:57.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:30:56 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:30:57.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:30:56 smithi132 ceph-mon[35941]: pgmap v136: 97 pgs: 97 active+clean; 104 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 271 op/s 2024-03-28T00:30:57.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:30:56 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:30:58.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:30:57 smithi165 ceph-mon[36302]: daemon mds.c restarted 2024-03-28T00:30:58.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:30:57 smithi165 ceph-mon[36302]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-28T00:30:58.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:30:57 smithi165 ceph-mon[36302]: osdmap e91: 12 total, 12 up, 12 in 2024-03-28T00:30:58.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:30:57 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.165:6836/1011925619,v1:172.21.15.165:6837/1011925619] up:boot 2024-03-28T00:30:58.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:30:57 smithi165 ceph-mon[36302]: Standby daemon mds.l assigned to filesystem cephfs as rank 3 2024-03-28T00:30:58.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:30:57 smithi165 ceph-mon[36302]: fsmap cephfs:4/5 {0=b=up:active,1=i=up:active,2=h=up:active,4=f=up:active} 5 up:standby-replay 3 up:standby, 1 failed 2024-03-28T00:30:58.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:30:57 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-28T00:30:58.135 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:30:57 smithi165 ceph-mon[36302]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:replay 4 up:active 2024-03-28T00:30:58.135 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:30:57 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:30:58.135 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:30:57 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:30:58.135 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:30:57 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:30:58.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:30:57 smithi119 ceph-mon[31518]: daemon mds.c restarted 2024-03-28T00:30:58.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:30:57 smithi119 ceph-mon[31518]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-28T00:30:58.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:30:57 smithi119 ceph-mon[31518]: osdmap e91: 12 total, 12 up, 12 in 2024-03-28T00:30:58.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:30:57 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.165:6836/1011925619,v1:172.21.15.165:6837/1011925619] up:boot 2024-03-28T00:30:58.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:30:57 smithi119 ceph-mon[31518]: Standby daemon mds.l assigned to filesystem cephfs as rank 3 2024-03-28T00:30:58.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:30:57 smithi119 ceph-mon[31518]: fsmap cephfs:4/5 {0=b=up:active,1=i=up:active,2=h=up:active,4=f=up:active} 5 up:standby-replay 3 up:standby, 1 failed 2024-03-28T00:30:58.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:30:57 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-28T00:30:58.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:30:57 smithi119 ceph-mon[31518]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:replay 4 up:active 2024-03-28T00:30:58.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:30:57 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:30:58.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:30:57 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:30:58.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:30:57 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:30:58.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:30:57 smithi132 ceph-mon[35941]: daemon mds.c restarted 2024-03-28T00:30:58.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:30:57 smithi132 ceph-mon[35941]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-28T00:30:58.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:30:57 smithi132 ceph-mon[35941]: osdmap e91: 12 total, 12 up, 12 in 2024-03-28T00:30:58.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:30:57 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.165:6836/1011925619,v1:172.21.15.165:6837/1011925619] up:boot 2024-03-28T00:30:58.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:30:57 smithi132 ceph-mon[35941]: Standby daemon mds.l assigned to filesystem cephfs as rank 3 2024-03-28T00:30:58.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:30:57 smithi132 ceph-mon[35941]: fsmap cephfs:4/5 {0=b=up:active,1=i=up:active,2=h=up:active,4=f=up:active} 5 up:standby-replay 3 up:standby, 1 failed 2024-03-28T00:30:58.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:30:57 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-28T00:30:58.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:30:57 smithi132 ceph-mon[35941]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:replay 4 up:active 2024-03-28T00:30:58.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:30:57 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:30:58.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:30:57 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:30:58.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:30:57 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:30:59.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:30:58 smithi165 ceph-mon[36302]: pgmap v138: 97 pgs: 97 active+clean; 104 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 237 op/s 2024-03-28T00:30:59.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:30:58 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.165:6832/1389081045,v1:172.21.15.165:6833/1389081045] up:resolve 2024-03-28T00:30:59.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:30:58 smithi165 ceph-mon[36302]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:resolve 4 up:active 2024-03-28T00:30:59.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:30:58 smithi119 ceph-mon[31518]: pgmap v138: 97 pgs: 97 active+clean; 104 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 237 op/s 2024-03-28T00:30:59.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:30:58 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.165:6832/1389081045,v1:172.21.15.165:6833/1389081045] up:resolve 2024-03-28T00:30:59.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:30:58 smithi119 ceph-mon[31518]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:resolve 4 up:active 2024-03-28T00:30:59.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:30:58 smithi132 ceph-mon[35941]: pgmap v138: 97 pgs: 97 active+clean; 104 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 237 op/s 2024-03-28T00:30:59.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:30:58 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.165:6832/1389081045,v1:172.21.15.165:6833/1389081045] up:resolve 2024-03-28T00:30:59.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:30:58 smithi132 ceph-mon[35941]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:resolve 4 up:active 2024-03-28T00:31:00.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:30:59 smithi165 ceph-mon[36302]: reconnect by client.25355 172.21.15.119:0/3071152482 after 0.00199997 2024-03-28T00:31:00.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:30:59 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.165:6832/1389081045,v1:172.21.15.165:6833/1389081045] up:reconnect 2024-03-28T00:31:00.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:30:59 smithi165 ceph-mon[36302]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:reconnect 4 up:active 2024-03-28T00:31:00.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:30:59 smithi165 ceph-mon[36302]: reconnect by client.25264 v1:172.21.15.119:0/3863511511 after 0.0129998 2024-03-28T00:31:00.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:30:59 smithi165 ceph-mon[36302]: pgmap v139: 97 pgs: 97 active+clean; 108 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 971 KiB/s rd, 1.1 MiB/s wr, 142 op/s 2024-03-28T00:31:00.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:30:59 smithi119 ceph-mon[31518]: reconnect by client.25355 172.21.15.119:0/3071152482 after 0.00199997 2024-03-28T00:31:00.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:30:59 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.165:6832/1389081045,v1:172.21.15.165:6833/1389081045] up:reconnect 2024-03-28T00:31:00.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:30:59 smithi119 ceph-mon[31518]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:reconnect 4 up:active 2024-03-28T00:31:00.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:30:59 smithi119 ceph-mon[31518]: reconnect by client.25264 v1:172.21.15.119:0/3863511511 after 0.0129998 2024-03-28T00:31:00.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:30:59 smithi119 ceph-mon[31518]: pgmap v139: 97 pgs: 97 active+clean; 108 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 971 KiB/s rd, 1.1 MiB/s wr, 142 op/s 2024-03-28T00:31:00.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:30:59 smithi132 ceph-mon[35941]: reconnect by client.25355 172.21.15.119:0/3071152482 after 0.00199997 2024-03-28T00:31:00.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:30:59 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.165:6832/1389081045,v1:172.21.15.165:6833/1389081045] up:reconnect 2024-03-28T00:31:00.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:30:59 smithi132 ceph-mon[35941]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:reconnect 4 up:active 2024-03-28T00:31:00.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:30:59 smithi132 ceph-mon[35941]: reconnect by client.25264 v1:172.21.15.119:0/3863511511 after 0.0129998 2024-03-28T00:31:00.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:30:59 smithi132 ceph-mon[35941]: pgmap v139: 97 pgs: 97 active+clean; 108 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 971 KiB/s rd, 1.1 MiB/s wr, 142 op/s 2024-03-28T00:31:01.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:31:00 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.165:6832/1389081045,v1:172.21.15.165:6833/1389081045] up:rejoin 2024-03-28T00:31:01.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:31:00 smithi165 ceph-mon[36302]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:rejoin 4 up:active 2024-03-28T00:31:01.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:31:00 smithi165 ceph-mon[36302]: daemon mds.l is now active in filesystem cephfs as rank 3 2024-03-28T00:31:01.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:31:00 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.165:6832/1389081045,v1:172.21.15.165:6833/1389081045] up:rejoin 2024-03-28T00:31:01.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:31:00 smithi119 ceph-mon[31518]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:rejoin 4 up:active 2024-03-28T00:31:01.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:31:00 smithi119 ceph-mon[31518]: daemon mds.l is now active in filesystem cephfs as rank 3 2024-03-28T00:31:01.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:31:00 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.165:6832/1389081045,v1:172.21.15.165:6833/1389081045] up:rejoin 2024-03-28T00:31:01.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:31:00 smithi132 ceph-mon[35941]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:rejoin 4 up:active 2024-03-28T00:31:01.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:31:00 smithi132 ceph-mon[35941]: daemon mds.l is now active in filesystem cephfs as rank 3 2024-03-28T00:31:02.105 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:31:01 smithi132 ceph-mon[35941]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-28T00:31:02.105 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:31:01 smithi132 ceph-mon[35941]: Cluster is now healthy 2024-03-28T00:31:02.105 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:31:01 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.165:6832/1389081045,v1:172.21.15.165:6833/1389081045] up:active 2024-03-28T00:31:02.105 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:31:01 smithi132 ceph-mon[35941]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-03-28T00:31:02.105 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:31:01 smithi132 ceph-mon[35941]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-28T00:31:02.105 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:31:01 smithi132 ceph-mon[35941]: pgmap v140: 97 pgs: 97 active+clean; 108 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 971 KiB/s rd, 1.1 MiB/s wr, 142 op/s 2024-03-28T00:31:02.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:31:01 smithi165 ceph-mon[36302]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-28T00:31:02.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:31:01 smithi165 ceph-mon[36302]: Cluster is now healthy 2024-03-28T00:31:02.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:31:01 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.165:6832/1389081045,v1:172.21.15.165:6833/1389081045] up:active 2024-03-28T00:31:02.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:31:01 smithi165 ceph-mon[36302]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-03-28T00:31:02.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:31:01 smithi165 ceph-mon[36302]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-28T00:31:02.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:31:01 smithi165 ceph-mon[36302]: pgmap v140: 97 pgs: 97 active+clean; 108 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 971 KiB/s rd, 1.1 MiB/s wr, 142 op/s 2024-03-28T00:31:02.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:31:01 smithi119 ceph-mon[31518]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-28T00:31:02.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:31:01 smithi119 ceph-mon[31518]: Cluster is now healthy 2024-03-28T00:31:02.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:31:01 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.165:6832/1389081045,v1:172.21.15.165:6833/1389081045] up:active 2024-03-28T00:31:02.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:31:01 smithi119 ceph-mon[31518]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-03-28T00:31:02.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:31:01 smithi119 ceph-mon[31518]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-28T00:31:02.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:31:01 smithi119 ceph-mon[31518]: pgmap v140: 97 pgs: 97 active+clean; 108 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 971 KiB/s rd, 1.1 MiB/s wr, 142 op/s 2024-03-28T00:31:04.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:31:04 smithi132 ceph-mon[35941]: pgmap v141: 97 pgs: 97 active+clean; 108 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 971 KiB/s rd, 1.1 MiB/s wr, 142 op/s 2024-03-28T00:31:04.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:31:04 smithi165 ceph-mon[36302]: pgmap v141: 97 pgs: 97 active+clean; 108 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 971 KiB/s rd, 1.1 MiB/s wr, 142 op/s 2024-03-28T00:31:04.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:31:04 smithi119 ceph-mon[31518]: pgmap v141: 97 pgs: 97 active+clean; 108 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 971 KiB/s rd, 1.1 MiB/s wr, 142 op/s 2024-03-28T00:31:06.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:31:06 smithi132 ceph-mon[35941]: pgmap v142: 97 pgs: 97 active+clean; 124 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.1 MiB/s wr, 139 op/s 2024-03-28T00:31:06.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:31:06 smithi165 ceph-mon[36302]: pgmap v142: 97 pgs: 97 active+clean; 124 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.1 MiB/s wr, 139 op/s 2024-03-28T00:31:06.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:31:06 smithi119 ceph-mon[31518]: pgmap v142: 97 pgs: 97 active+clean; 124 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.1 MiB/s wr, 139 op/s 2024-03-28T00:31:08.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:31:08 smithi132 ceph-mon[35941]: pgmap v143: 97 pgs: 97 active+clean; 124 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.0 MiB/s wr, 134 op/s 2024-03-28T00:31:08.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-mon[36302]: pgmap v143: 97 pgs: 97 active+clean; 124 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.0 MiB/s wr, 134 op/s 2024-03-28T00:31:08.634 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7f0ad911a700 time 2024-03-28T00:31:08.307944+0000 2024-03-28T00:31:08.634 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T00:31:08.634 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:31:08.634 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f0ae37444a1] 2024-03-28T00:31:08.634 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f0ae3744667] 2024-03-28T00:31:08.634 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55e7a8508223] 2024-03-28T00:31:08.634 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55e7a83825d6] 2024-03-28T00:31:08.635 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55e7a8382da1] 2024-03-28T00:31:08.635 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 6: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x55e7a83b709c] 2024-03-28T00:31:08.635 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 7: (MDSContext::complete(int)+0x5f) [0x55e7a865f68f] 2024-03-28T00:31:08.635 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 8: (void finish_contexts > >(ceph::common::CephContext*, std::vector >&, int)+0x8d) [0x55e7a82c74fd] 2024-03-28T00:31:08.635 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 9: (MDSCacheObject::finish_waiting(unsigned long, int)+0x5c) [0x55e7a86726ac] 2024-03-28T00:31:08.635 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 10: (Locker::dispatch(boost::intrusive_ptr const&)+0x174) [0x55e7a852e474] 2024-03-28T00:31:08.635 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 11: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55e7a82d611b] 2024-03-28T00:31:08.635 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 12: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55e7a82d66bc] 2024-03-28T00:31:08.635 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 13: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55e7a82c01ff] 2024-03-28T00:31:08.635 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 14: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f0ae39d6488] 2024-03-28T00:31:08.635 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 15: (DispatchQueue::entry()+0x51f) [0x7f0ae39d360f] 2024-03-28T00:31:08.635 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 16: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f0ae3aa6e81] 2024-03-28T00:31:08.635 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 17: /lib64/libpthread.so.0(+0x81ca) [0x7f0ae24c11ca] 2024-03-28T00:31:08.636 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 18: clone() 2024-03-28T00:31:08.637 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 2024-03-28T00:31:08.307+0000 7f0ad911a700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7f0ad911a700 time 2024-03-28T00:31:08.307944+0000 2024-03-28T00:31:08.637 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T00:31:08.637 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 2024-03-28T00:31:08.637 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:31:08.637 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f0ae37444a1] 2024-03-28T00:31:08.637 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f0ae3744667] 2024-03-28T00:31:08.637 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55e7a8508223] 2024-03-28T00:31:08.637 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55e7a83825d6] 2024-03-28T00:31:08.637 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55e7a8382da1] 2024-03-28T00:31:08.637 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 6: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x55e7a83b709c] 2024-03-28T00:31:08.638 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 7: (MDSContext::complete(int)+0x5f) [0x55e7a865f68f] 2024-03-28T00:31:08.638 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 8: (void finish_contexts > >(ceph::common::CephContext*, std::vector >&, int)+0x8d) [0x55e7a82c74fd] 2024-03-28T00:31:08.638 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 9: (MDSCacheObject::finish_waiting(unsigned long, int)+0x5c) [0x55e7a86726ac] 2024-03-28T00:31:08.638 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 10: (Locker::dispatch(boost::intrusive_ptr const&)+0x174) [0x55e7a852e474] 2024-03-28T00:31:08.638 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 11: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55e7a82d611b] 2024-03-28T00:31:08.638 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 12: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55e7a82d66bc] 2024-03-28T00:31:08.638 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 13: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55e7a82c01ff] 2024-03-28T00:31:08.638 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 14: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f0ae39d6488] 2024-03-28T00:31:08.638 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 15: (DispatchQueue::entry()+0x51f) [0x7f0ae39d360f] 2024-03-28T00:31:08.638 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 16: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f0ae3aa6e81] 2024-03-28T00:31:08.638 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 17: /lib64/libpthread.so.0(+0x81ca) [0x7f0ae24c11ca] 2024-03-28T00:31:08.639 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 18: clone() 2024-03-28T00:31:08.639 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 2024-03-28T00:31:08.639 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: *** Caught signal (Aborted) ** 2024-03-28T00:31:08.639 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: in thread 7f0ad911a700 thread_name:ms_dispatch 2024-03-28T00:31:08.639 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:31:08.639 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f0ae24cbd20] 2024-03-28T00:31:08.639 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 2: gsignal() 2024-03-28T00:31:08.639 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 3: abort() 2024-03-28T00:31:08.639 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f0ae37444fb] 2024-03-28T00:31:08.639 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f0ae3744667] 2024-03-28T00:31:08.639 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55e7a8508223] 2024-03-28T00:31:08.639 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55e7a83825d6] 2024-03-28T00:31:08.639 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55e7a8382da1] 2024-03-28T00:31:08.639 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 9: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x55e7a83b709c] 2024-03-28T00:31:08.640 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 10: (MDSContext::complete(int)+0x5f) [0x55e7a865f68f] 2024-03-28T00:31:08.640 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 11: (void finish_contexts > >(ceph::common::CephContext*, std::vector >&, int)+0x8d) [0x55e7a82c74fd] 2024-03-28T00:31:08.640 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 12: (MDSCacheObject::finish_waiting(unsigned long, int)+0x5c) [0x55e7a86726ac] 2024-03-28T00:31:08.640 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 13: (Locker::dispatch(boost::intrusive_ptr const&)+0x174) [0x55e7a852e474] 2024-03-28T00:31:08.640 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 14: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55e7a82d611b] 2024-03-28T00:31:08.640 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 15: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55e7a82d66bc] 2024-03-28T00:31:08.640 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 16: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55e7a82c01ff] 2024-03-28T00:31:08.640 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 17: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f0ae39d6488] 2024-03-28T00:31:08.640 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 18: (DispatchQueue::entry()+0x51f) [0x7f0ae39d360f] 2024-03-28T00:31:08.640 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 19: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f0ae3aa6e81] 2024-03-28T00:31:08.640 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 20: /lib64/libpthread.so.0(+0x81ca) [0x7f0ae24c11ca] 2024-03-28T00:31:08.640 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 21: clone() 2024-03-28T00:31:08.640 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 2024-03-28T00:31:08.309+0000 7f0ad911a700 -1 *** Caught signal (Aborted) ** 2024-03-28T00:31:08.641 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: in thread 7f0ad911a700 thread_name:ms_dispatch 2024-03-28T00:31:08.641 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 2024-03-28T00:31:08.641 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:31:08.641 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f0ae24cbd20] 2024-03-28T00:31:08.641 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 2: gsignal() 2024-03-28T00:31:08.641 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 3: abort() 2024-03-28T00:31:08.641 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f0ae37444fb] 2024-03-28T00:31:08.641 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f0ae3744667] 2024-03-28T00:31:08.641 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55e7a8508223] 2024-03-28T00:31:08.641 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55e7a83825d6] 2024-03-28T00:31:08.641 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55e7a8382da1] 2024-03-28T00:31:08.641 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 9: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x55e7a83b709c] 2024-03-28T00:31:08.641 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 10: (MDSContext::complete(int)+0x5f) [0x55e7a865f68f] 2024-03-28T00:31:08.642 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 11: (void finish_contexts > >(ceph::common::CephContext*, std::vector >&, int)+0x8d) [0x55e7a82c74fd] 2024-03-28T00:31:08.642 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 12: (MDSCacheObject::finish_waiting(unsigned long, int)+0x5c) [0x55e7a86726ac] 2024-03-28T00:31:08.642 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 13: (Locker::dispatch(boost::intrusive_ptr const&)+0x174) [0x55e7a852e474] 2024-03-28T00:31:08.642 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 14: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55e7a82d611b] 2024-03-28T00:31:08.642 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 15: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55e7a82d66bc] 2024-03-28T00:31:08.642 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 16: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55e7a82c01ff] 2024-03-28T00:31:08.642 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 17: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f0ae39d6488] 2024-03-28T00:31:08.642 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 18: (DispatchQueue::entry()+0x51f) [0x7f0ae39d360f] 2024-03-28T00:31:08.642 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 19: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f0ae3aa6e81] 2024-03-28T00:31:08.642 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 20: /lib64/libpthread.so.0(+0x81ca) [0x7f0ae24c11ca] 2024-03-28T00:31:08.642 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 21: clone() 2024-03-28T00:31:08.642 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T00:31:08.642 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 2024-03-28T00:31:08.643 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: -1> 2024-03-28T00:31:08.307+0000 7f0ad911a700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7f0ad911a700 time 2024-03-28T00:31:08.307944+0000 2024-03-28T00:31:08.643 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T00:31:08.643 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 2024-03-28T00:31:08.643 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:31:08.643 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f0ae37444a1] 2024-03-28T00:31:08.643 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f0ae3744667] 2024-03-28T00:31:08.643 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55e7a8508223] 2024-03-28T00:31:08.643 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55e7a83825d6] 2024-03-28T00:31:08.643 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55e7a8382da1] 2024-03-28T00:31:08.643 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 6: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x55e7a83b709c] 2024-03-28T00:31:08.643 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 7: (MDSContext::complete(int)+0x5f) [0x55e7a865f68f] 2024-03-28T00:31:08.643 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 8: (void finish_contexts > >(ceph::common::CephContext*, std::vector >&, int)+0x8d) [0x55e7a82c74fd] 2024-03-28T00:31:08.643 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 9: (MDSCacheObject::finish_waiting(unsigned long, int)+0x5c) [0x55e7a86726ac] 2024-03-28T00:31:08.644 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 10: (Locker::dispatch(boost::intrusive_ptr const&)+0x174) [0x55e7a852e474] 2024-03-28T00:31:08.644 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 11: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55e7a82d611b] 2024-03-28T00:31:08.644 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 12: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55e7a82d66bc] 2024-03-28T00:31:08.644 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 13: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55e7a82c01ff] 2024-03-28T00:31:08.644 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 14: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f0ae39d6488] 2024-03-28T00:31:08.644 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 15: (DispatchQueue::entry()+0x51f) [0x7f0ae39d360f] 2024-03-28T00:31:08.644 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 16: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f0ae3aa6e81] 2024-03-28T00:31:08.644 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 17: /lib64/libpthread.so.0(+0x81ca) [0x7f0ae24c11ca] 2024-03-28T00:31:08.644 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 18: clone() 2024-03-28T00:31:08.644 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 2024-03-28T00:31:08.644 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 0> 2024-03-28T00:31:08.309+0000 7f0ad911a700 -1 *** Caught signal (Aborted) ** 2024-03-28T00:31:08.644 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: in thread 7f0ad911a700 thread_name:ms_dispatch 2024-03-28T00:31:08.644 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 2024-03-28T00:31:08.645 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:31:08.645 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f0ae24cbd20] 2024-03-28T00:31:08.645 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 2: gsignal() 2024-03-28T00:31:08.645 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 3: abort() 2024-03-28T00:31:08.645 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f0ae37444fb] 2024-03-28T00:31:08.645 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f0ae3744667] 2024-03-28T00:31:08.645 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55e7a8508223] 2024-03-28T00:31:08.645 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55e7a83825d6] 2024-03-28T00:31:08.645 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55e7a8382da1] 2024-03-28T00:31:08.645 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 9: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x55e7a83b709c] 2024-03-28T00:31:08.645 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 10: (MDSContext::complete(int)+0x5f) [0x55e7a865f68f] 2024-03-28T00:31:08.645 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 11: (void finish_contexts > >(ceph::common::CephContext*, std::vector >&, int)+0x8d) [0x55e7a82c74fd] 2024-03-28T00:31:08.645 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 12: (MDSCacheObject::finish_waiting(unsigned long, int)+0x5c) [0x55e7a86726ac] 2024-03-28T00:31:08.645 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 13: (Locker::dispatch(boost::intrusive_ptr const&)+0x174) [0x55e7a852e474] 2024-03-28T00:31:08.645 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 14: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55e7a82d611b] 2024-03-28T00:31:08.646 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 15: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55e7a82d66bc] 2024-03-28T00:31:08.646 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 16: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55e7a82c01ff] 2024-03-28T00:31:08.646 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 17: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f0ae39d6488] 2024-03-28T00:31:08.646 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 18: (DispatchQueue::entry()+0x51f) [0x7f0ae39d360f] 2024-03-28T00:31:08.646 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 19: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f0ae3aa6e81] 2024-03-28T00:31:08.646 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 20: /lib64/libpthread.so.0(+0x81ca) [0x7f0ae24c11ca] 2024-03-28T00:31:08.647 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 21: clone() 2024-03-28T00:31:08.647 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T00:31:08.647 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 2024-03-28T00:31:08.647 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: -1> 2024-03-28T00:31:08.307+0000 7f0ad911a700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7f0ad911a700 time 2024-03-28T00:31:08.307944+0000 2024-03-28T00:31:08.647 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T00:31:08.647 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 2024-03-28T00:31:08.647 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:31:08.647 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f0ae37444a1] 2024-03-28T00:31:08.647 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f0ae3744667] 2024-03-28T00:31:08.647 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55e7a8508223] 2024-03-28T00:31:08.647 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55e7a83825d6] 2024-03-28T00:31:08.648 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55e7a8382da1] 2024-03-28T00:31:08.648 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 6: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x55e7a83b709c] 2024-03-28T00:31:08.648 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 7: (MDSContext::complete(int)+0x5f) [0x55e7a865f68f] 2024-03-28T00:31:08.648 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 8: (void finish_contexts > >(ceph::common::CephContext*, std::vector >&, int)+0x8d) [0x55e7a82c74fd] 2024-03-28T00:31:08.648 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 9: (MDSCacheObject::finish_waiting(unsigned long, int)+0x5c) [0x55e7a86726ac] 2024-03-28T00:31:08.648 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 10: (Locker::dispatch(boost::intrusive_ptr const&)+0x174) [0x55e7a852e474] 2024-03-28T00:31:08.648 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 11: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55e7a82d611b] 2024-03-28T00:31:08.648 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 12: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55e7a82d66bc] 2024-03-28T00:31:08.648 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 13: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55e7a82c01ff] 2024-03-28T00:31:08.648 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 14: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f0ae39d6488] 2024-03-28T00:31:08.648 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 15: (DispatchQueue::entry()+0x51f) [0x7f0ae39d360f] 2024-03-28T00:31:08.649 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 16: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f0ae3aa6e81] 2024-03-28T00:31:08.649 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 17: /lib64/libpthread.so.0(+0x81ca) [0x7f0ae24c11ca] 2024-03-28T00:31:08.649 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 18: clone() 2024-03-28T00:31:08.649 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 2024-03-28T00:31:08.649 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 0> 2024-03-28T00:31:08.309+0000 7f0ad911a700 -1 *** Caught signal (Aborted) ** 2024-03-28T00:31:08.649 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: in thread 7f0ad911a700 thread_name:ms_dispatch 2024-03-28T00:31:08.649 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 2024-03-28T00:31:08.649 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:31:08.649 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f0ae24cbd20] 2024-03-28T00:31:08.649 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 2: gsignal() 2024-03-28T00:31:08.649 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 3: abort() 2024-03-28T00:31:08.649 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f0ae37444fb] 2024-03-28T00:31:08.649 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f0ae3744667] 2024-03-28T00:31:08.649 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55e7a8508223] 2024-03-28T00:31:08.650 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55e7a83825d6] 2024-03-28T00:31:08.650 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55e7a8382da1] 2024-03-28T00:31:08.650 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 9: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x55e7a83b709c] 2024-03-28T00:31:08.650 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 10: (MDSContext::complete(int)+0x5f) [0x55e7a865f68f] 2024-03-28T00:31:08.650 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 11: (void finish_contexts > >(ceph::common::CephContext*, std::vector >&, int)+0x8d) [0x55e7a82c74fd] 2024-03-28T00:31:08.650 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 12: (MDSCacheObject::finish_waiting(unsigned long, int)+0x5c) [0x55e7a86726ac] 2024-03-28T00:31:08.650 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 13: (Locker::dispatch(boost::intrusive_ptr const&)+0x174) [0x55e7a852e474] 2024-03-28T00:31:08.650 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 14: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55e7a82d611b] 2024-03-28T00:31:08.650 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 15: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55e7a82d66bc] 2024-03-28T00:31:08.650 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 16: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55e7a82c01ff] 2024-03-28T00:31:08.650 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 17: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f0ae39d6488] 2024-03-28T00:31:08.650 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 18: (DispatchQueue::entry()+0x51f) [0x7f0ae39d360f] 2024-03-28T00:31:08.650 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 19: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f0ae3aa6e81] 2024-03-28T00:31:08.650 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 20: /lib64/libpthread.so.0(+0x81ca) [0x7f0ae24c11ca] 2024-03-28T00:31:08.651 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 21: clone() 2024-03-28T00:31:08.651 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T00:31:08.651 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[71693]: 2024-03-28T00:31:08.651 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:08 smithi165 podman[72009]: 2024-03-28 00:31:08.368652267 +0000 UTC m=+0.019240925 container died e64abea411d6f06622eeb4a41cd4517895cdab1ca3c8e014bc942203d8390807 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, GIT_CLEAN=True) 2024-03-28T00:31:08.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:31:08 smithi119 ceph-mon[31518]: pgmap v143: 97 pgs: 97 active+clean; 124 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.0 MiB/s wr, 134 op/s 2024-03-28T00:31:09.338 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:09 smithi165 podman[72009]: 2024-03-28 00:31:09.077079221 +0000 UTC m=+0.727667886 container remove e64abea411d6f06622eeb4a41cd4517895cdab1ca3c8e014bc942203d8390807 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l, GIT_CLEAN=True, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-03-28T00:31:09.338 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:09 smithi165 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.l.service: Main process exited, code=exited, status=134/n/a 2024-03-28T00:31:09.633 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:09 smithi165 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.l.service: Failed with result 'exit-code'. 2024-03-28T00:31:09.633 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:09 smithi165 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.l.service: Consumed 3.480s CPU time. 2024-03-28T00:31:10.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:31:10 smithi132 ceph-mon[35941]: pgmap v144: 97 pgs: 97 active+clean; 144 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.5 MiB/s wr, 156 op/s 2024-03-28T00:31:10.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:31:10 smithi165 ceph-mon[36302]: pgmap v144: 97 pgs: 97 active+clean; 144 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.5 MiB/s wr, 156 op/s 2024-03-28T00:31:10.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:31:10 smithi119 ceph-mon[31518]: pgmap v144: 97 pgs: 97 active+clean; 144 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.5 MiB/s wr, 156 op/s 2024-03-28T00:31:12.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:31:12 smithi132 ceph-mon[35941]: pgmap v145: 97 pgs: 97 active+clean; 144 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.1 MiB/s wr, 103 op/s 2024-03-28T00:31:12.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:31:12 smithi165 ceph-mon[36302]: pgmap v145: 97 pgs: 97 active+clean; 144 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.1 MiB/s wr, 103 op/s 2024-03-28T00:31:12.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:31:12 smithi119 ceph-mon[31518]: pgmap v145: 97 pgs: 97 active+clean; 144 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.1 MiB/s wr, 103 op/s 2024-03-28T00:31:14.239 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:31:14 smithi119 ceph-mon[31518]: pgmap v146: 97 pgs: 97 active+clean; 144 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.1 MiB/s wr, 103 op/s 2024-03-28T00:31:14.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:31:14 smithi132 ceph-mon[35941]: pgmap v146: 97 pgs: 97 active+clean; 144 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.1 MiB/s wr, 103 op/s 2024-03-28T00:31:14.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:31:14 smithi165 ceph-mon[36302]: pgmap v146: 97 pgs: 97 active+clean; 144 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.1 MiB/s wr, 103 op/s 2024-03-28T00:31:16.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:31:16 smithi132 ceph-mon[35941]: pgmap v147: 97 pgs: 97 active+clean; 161 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.6 MiB/s wr, 140 op/s 2024-03-28T00:31:16.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:31:16 smithi165 ceph-mon[36302]: pgmap v147: 97 pgs: 97 active+clean; 161 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.6 MiB/s wr, 140 op/s 2024-03-28T00:31:16.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:31:16 smithi119 ceph-mon[31518]: pgmap v147: 97 pgs: 97 active+clean; 161 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.6 MiB/s wr, 140 op/s 2024-03-28T00:31:18.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:31:18 smithi132 ceph-mon[35941]: pgmap v148: 97 pgs: 97 active+clean; 161 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.3 MiB/s wr, 76 op/s 2024-03-28T00:31:18.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:31:18 smithi165 ceph-mon[36302]: pgmap v148: 97 pgs: 97 active+clean; 161 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.3 MiB/s wr, 76 op/s 2024-03-28T00:31:18.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:31:18 smithi119 ceph-mon[31518]: pgmap v148: 97 pgs: 97 active+clean; 161 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.3 MiB/s wr, 76 op/s 2024-03-28T00:31:19.712 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:19 smithi165 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.l.service: Scheduled restart job, restart counter is at 2. 2024-03-28T00:31:19.712 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:19 smithi165 systemd[1]: Stopped Ceph mds.l for 10f73b92-ec98-11ee-95d0-87774f69a715. 2024-03-28T00:31:19.712 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:19 smithi165 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.l.service: Consumed 3.480s CPU time. 2024-03-28T00:31:19.712 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:19 smithi165 systemd[1]: Starting Ceph mds.l for 10f73b92-ec98-11ee-95d0-87774f69a715... 2024-03-28T00:31:19.712 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:19 smithi165 podman[72102]: 2024-03-28 00:31:19.510688149 +0000 UTC m=+0.025490770 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-28T00:31:20.027 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:19 smithi165 podman[72102]: 2024-03-28 00:31:19.705904272 +0000 UTC m=+0.220706871 container create 13b6de6cfef851c38b8dfc107dc9bb86ec1f898dfe1fabaaf0277a3207e74066 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-03-28T00:31:20.384 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-mon[36302]: pgmap v149: 97 pgs: 97 active+clean; 163 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.5 MiB/s wr, 88 op/s 2024-03-28T00:31:20.384 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-mon[36302]: daemon mds.l restarted 2024-03-28T00:31:20.384 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-mon[36302]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-28T00:31:20.384 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-28T00:31:20.384 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-mon[36302]: osdmap e92: 12 total, 12 up, 12 in 2024-03-28T00:31:20.384 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.165:6832/2013665520,v1:172.21.15.165:6833/2013665520] up:boot 2024-03-28T00:31:20.384 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-mon[36302]: Standby daemon mds.c assigned to filesystem cephfs as rank 3 2024-03-28T00:31:20.384 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-mon[36302]: fsmap cephfs:4/5 {0=b=up:active,1=i=up:active,2=h=up:active,4=f=up:active} 5 up:standby-replay 3 up:standby, 1 failed 2024-03-28T00:31:20.384 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-mon[36302]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:replay 4 up:active 2024-03-28T00:31:20.385 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:20 smithi165 podman[72102]: 2024-03-28 00:31:20.021791237 +0000 UTC m=+0.536593853 container init 13b6de6cfef851c38b8dfc107dc9bb86ec1f898dfe1fabaaf0277a3207e74066 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-03-28T00:31:20.385 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:20 smithi165 podman[72102]: 2024-03-28 00:31:20.024851294 +0000 UTC m=+0.539653891 container start 13b6de6cfef851c38b8dfc107dc9bb86ec1f898dfe1fabaaf0277a3207e74066 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-03-28T00:31:20.385 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-mds[72121]: -- 172.21.15.165:0/2603427729 <== mon.1 v2:172.21.15.165:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55b96e093a40 con 0x55b96e067000 2024-03-28T00:31:20.385 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: starting mds.l at 2024-03-28T00:31:20.385 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:20 smithi165 bash[72102]: 13b6de6cfef851c38b8dfc107dc9bb86ec1f898dfe1fabaaf0277a3207e74066 2024-03-28T00:31:20.385 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:20 smithi165 systemd[1]: Started Ceph mds.l for 10f73b92-ec98-11ee-95d0-87774f69a715. 2024-03-28T00:31:20.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:31:20 smithi132 ceph-mon[35941]: pgmap v149: 97 pgs: 97 active+clean; 163 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.5 MiB/s wr, 88 op/s 2024-03-28T00:31:20.454 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:31:20 smithi132 ceph-mon[35941]: daemon mds.l restarted 2024-03-28T00:31:20.454 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:31:20 smithi132 ceph-mon[35941]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-28T00:31:20.454 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:31:20 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-28T00:31:20.454 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:31:20 smithi132 ceph-mon[35941]: osdmap e92: 12 total, 12 up, 12 in 2024-03-28T00:31:20.454 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:31:20 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.165:6832/2013665520,v1:172.21.15.165:6833/2013665520] up:boot 2024-03-28T00:31:20.454 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:31:20 smithi132 ceph-mon[35941]: Standby daemon mds.c assigned to filesystem cephfs as rank 3 2024-03-28T00:31:20.454 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:31:20 smithi132 ceph-mon[35941]: fsmap cephfs:4/5 {0=b=up:active,1=i=up:active,2=h=up:active,4=f=up:active} 5 up:standby-replay 3 up:standby, 1 failed 2024-03-28T00:31:20.454 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:31:20 smithi132 ceph-mon[35941]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:replay 4 up:active 2024-03-28T00:31:20.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:31:20 smithi119 ceph-mon[31518]: pgmap v149: 97 pgs: 97 active+clean; 163 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.5 MiB/s wr, 88 op/s 2024-03-28T00:31:20.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:31:20 smithi119 ceph-mon[31518]: daemon mds.l restarted 2024-03-28T00:31:20.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:31:20 smithi119 ceph-mon[31518]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-28T00:31:20.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:31:20 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-28T00:31:20.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:31:20 smithi119 ceph-mon[31518]: osdmap e92: 12 total, 12 up, 12 in 2024-03-28T00:31:20.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:31:20 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.165:6832/2013665520,v1:172.21.15.165:6833/2013665520] up:boot 2024-03-28T00:31:20.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:31:20 smithi119 ceph-mon[31518]: Standby daemon mds.c assigned to filesystem cephfs as rank 3 2024-03-28T00:31:20.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:31:20 smithi119 ceph-mon[31518]: fsmap cephfs:4/5 {0=b=up:active,1=i=up:active,2=h=up:active,4=f=up:active} 5 up:standby-replay 3 up:standby, 1 failed 2024-03-28T00:31:20.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:31:20 smithi119 ceph-mon[31518]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:replay 4 up:active 2024-03-28T00:31:20.701 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7f2c436fa700 time 2024-03-28T00:31:20.666588+0000 2024-03-28T00:31:20.701 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T00:31:20.701 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:31:20.701 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f2c4dd244a1] 2024-03-28T00:31:20.701 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f2c4dd24667] 2024-03-28T00:31:20.701 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x561037b40223] 2024-03-28T00:31:20.702 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x5610379ba5d6] 2024-03-28T00:31:20.702 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x5610379bada1] 2024-03-28T00:31:20.702 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: 6: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x5610379ef09c] 2024-03-28T00:31:20.702 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x561037a0d233] 2024-03-28T00:31:20.702 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x561037a0f197] 2024-03-28T00:31:20.702 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x56103790e11b] 2024-03-28T00:31:20.702 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x56103790e6bc] 2024-03-28T00:31:20.702 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x5610378f81ff] 2024-03-28T00:31:20.702 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f2c4dfb6488] 2024-03-28T00:31:20.702 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: 13: (DispatchQueue::entry()+0x51f) [0x7f2c4dfb360f] 2024-03-28T00:31:20.702 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f2c4e086e81] 2024-03-28T00:31:20.702 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f2c4caa11ca] 2024-03-28T00:31:20.703 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: 16: clone() 2024-03-28T00:31:20.703 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: 2024-03-28T00:31:20.667+0000 7f2c436fa700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7f2c436fa700 time 2024-03-28T00:31:20.666588+0000 2024-03-28T00:31:20.703 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T00:31:20.703 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: 2024-03-28T00:31:20.703 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:31:20.703 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f2c4dd244a1] 2024-03-28T00:31:20.704 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f2c4dd24667] 2024-03-28T00:31:20.704 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x561037b40223] 2024-03-28T00:31:20.704 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x5610379ba5d6] 2024-03-28T00:31:20.704 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x5610379bada1] 2024-03-28T00:31:20.704 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: 6: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x5610379ef09c] 2024-03-28T00:31:20.704 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x561037a0d233] 2024-03-28T00:31:20.704 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x561037a0f197] 2024-03-28T00:31:20.704 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x56103790e11b] 2024-03-28T00:31:20.704 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x56103790e6bc] 2024-03-28T00:31:20.704 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x5610378f81ff] 2024-03-28T00:31:20.704 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f2c4dfb6488] 2024-03-28T00:31:20.704 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: 13: (DispatchQueue::entry()+0x51f) [0x7f2c4dfb360f] 2024-03-28T00:31:20.705 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f2c4e086e81] 2024-03-28T00:31:20.705 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f2c4caa11ca] 2024-03-28T00:31:20.705 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: 16: clone() 2024-03-28T00:31:20.705 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: 2024-03-28T00:31:20.705 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: *** Caught signal (Aborted) ** 2024-03-28T00:31:20.705 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: in thread 7f2c436fa700 thread_name:ms_dispatch 2024-03-28T00:31:20.705 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:31:20.705 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f2c4caabd20] 2024-03-28T00:31:20.705 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: 2: gsignal() 2024-03-28T00:31:20.705 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: 3: abort() 2024-03-28T00:31:20.705 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f2c4dd244fb] 2024-03-28T00:31:20.705 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f2c4dd24667] 2024-03-28T00:31:20.705 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x561037b40223] 2024-03-28T00:31:20.705 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x5610379ba5d6] 2024-03-28T00:31:20.706 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x5610379bada1] 2024-03-28T00:31:20.706 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: 9: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x5610379ef09c] 2024-03-28T00:31:20.706 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x561037a0d233] 2024-03-28T00:31:20.706 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x561037a0f197] 2024-03-28T00:31:20.706 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x56103790e11b] 2024-03-28T00:31:20.706 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x56103790e6bc] 2024-03-28T00:31:20.706 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x5610378f81ff] 2024-03-28T00:31:20.706 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f2c4dfb6488] 2024-03-28T00:31:20.706 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: 16: (DispatchQueue::entry()+0x51f) [0x7f2c4dfb360f] 2024-03-28T00:31:20.706 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f2c4e086e81] 2024-03-28T00:31:20.706 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f2c4caa11ca] 2024-03-28T00:31:20.706 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: 19: clone() 2024-03-28T00:31:20.706 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: 2024-03-28T00:31:20.670+0000 7f2c436fa700 -1 *** Caught signal (Aborted) ** 2024-03-28T00:31:20.707 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: in thread 7f2c436fa700 thread_name:ms_dispatch 2024-03-28T00:31:20.707 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: 2024-03-28T00:31:20.707 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:31:20.707 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f2c4caabd20] 2024-03-28T00:31:20.707 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: 2: gsignal() 2024-03-28T00:31:20.707 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: 3: abort() 2024-03-28T00:31:20.707 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f2c4dd244fb] 2024-03-28T00:31:20.707 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f2c4dd24667] 2024-03-28T00:31:20.708 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x561037b40223] 2024-03-28T00:31:20.708 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x5610379ba5d6] 2024-03-28T00:31:20.708 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x5610379bada1] 2024-03-28T00:31:20.708 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: 9: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x5610379ef09c] 2024-03-28T00:31:20.708 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x561037a0d233] 2024-03-28T00:31:20.708 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x561037a0f197] 2024-03-28T00:31:20.708 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x56103790e11b] 2024-03-28T00:31:20.708 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x56103790e6bc] 2024-03-28T00:31:20.708 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x5610378f81ff] 2024-03-28T00:31:20.708 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f2c4dfb6488] 2024-03-28T00:31:20.708 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: 16: (DispatchQueue::entry()+0x51f) [0x7f2c4dfb360f] 2024-03-28T00:31:20.709 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f2c4e086e81] 2024-03-28T00:31:20.709 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f2c4caa11ca] 2024-03-28T00:31:20.709 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: 19: clone() 2024-03-28T00:31:20.709 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T00:31:20.709 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: 2024-03-28T00:31:20.709 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: -1> 2024-03-28T00:31:20.667+0000 7f2c436fa700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7f2c436fa700 time 2024-03-28T00:31:20.666588+0000 2024-03-28T00:31:20.709 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T00:31:20.709 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: 2024-03-28T00:31:20.709 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:31:20.709 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f2c4dd244a1] 2024-03-28T00:31:20.709 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f2c4dd24667] 2024-03-28T00:31:20.709 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x561037b40223] 2024-03-28T00:31:20.709 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x5610379ba5d6] 2024-03-28T00:31:20.710 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x5610379bada1] 2024-03-28T00:31:20.710 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: 6: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x5610379ef09c] 2024-03-28T00:31:20.710 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x561037a0d233] 2024-03-28T00:31:20.710 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x561037a0f197] 2024-03-28T00:31:20.710 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x56103790e11b] 2024-03-28T00:31:20.710 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x56103790e6bc] 2024-03-28T00:31:20.710 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x5610378f81ff] 2024-03-28T00:31:20.710 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f2c4dfb6488] 2024-03-28T00:31:20.710 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: 13: (DispatchQueue::entry()+0x51f) [0x7f2c4dfb360f] 2024-03-28T00:31:20.711 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f2c4e086e81] 2024-03-28T00:31:20.711 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f2c4caa11ca] 2024-03-28T00:31:20.711 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: 16: clone() 2024-03-28T00:31:20.711 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: 2024-03-28T00:31:20.711 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: 0> 2024-03-28T00:31:20.670+0000 7f2c436fa700 -1 *** Caught signal (Aborted) ** 2024-03-28T00:31:20.711 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: in thread 7f2c436fa700 thread_name:ms_dispatch 2024-03-28T00:31:20.711 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: 2024-03-28T00:31:20.711 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:31:20.711 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f2c4caabd20] 2024-03-28T00:31:20.711 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: 2: gsignal() 2024-03-28T00:31:20.711 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: 3: abort() 2024-03-28T00:31:20.711 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f2c4dd244fb] 2024-03-28T00:31:20.711 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f2c4dd24667] 2024-03-28T00:31:20.712 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x561037b40223] 2024-03-28T00:31:20.712 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x5610379ba5d6] 2024-03-28T00:31:20.712 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x5610379bada1] 2024-03-28T00:31:20.712 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: 9: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x5610379ef09c] 2024-03-28T00:31:20.712 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x561037a0d233] 2024-03-28T00:31:20.712 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x561037a0f197] 2024-03-28T00:31:20.712 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x56103790e11b] 2024-03-28T00:31:20.712 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x56103790e6bc] 2024-03-28T00:31:20.712 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x5610378f81ff] 2024-03-28T00:31:20.712 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f2c4dfb6488] 2024-03-28T00:31:20.712 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: 16: (DispatchQueue::entry()+0x51f) [0x7f2c4dfb360f] 2024-03-28T00:31:20.712 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f2c4e086e81] 2024-03-28T00:31:20.712 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f2c4caa11ca] 2024-03-28T00:31:20.713 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: 19: clone() 2024-03-28T00:31:20.713 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T00:31:20.713 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: 2024-03-28T00:31:21.059 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: -1> 2024-03-28T00:31:20.667+0000 7f2c436fa700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7f2c436fa700 time 2024-03-28T00:31:20.666588+0000 2024-03-28T00:31:21.059 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T00:31:21.059 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: 2024-03-28T00:31:21.059 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:31:21.060 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f2c4dd244a1] 2024-03-28T00:31:21.060 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f2c4dd24667] 2024-03-28T00:31:21.060 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x561037b40223] 2024-03-28T00:31:21.060 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x5610379ba5d6] 2024-03-28T00:31:21.060 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x5610379bada1] 2024-03-28T00:31:21.060 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: 6: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x5610379ef09c] 2024-03-28T00:31:21.060 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x561037a0d233] 2024-03-28T00:31:21.060 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x561037a0f197] 2024-03-28T00:31:21.060 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x56103790e11b] 2024-03-28T00:31:21.060 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x56103790e6bc] 2024-03-28T00:31:21.060 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x5610378f81ff] 2024-03-28T00:31:21.060 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f2c4dfb6488] 2024-03-28T00:31:21.061 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: 13: (DispatchQueue::entry()+0x51f) [0x7f2c4dfb360f] 2024-03-28T00:31:21.061 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f2c4e086e81] 2024-03-28T00:31:21.061 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f2c4caa11ca] 2024-03-28T00:31:21.061 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: 16: clone() 2024-03-28T00:31:21.061 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: 2024-03-28T00:31:21.061 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: 0> 2024-03-28T00:31:20.670+0000 7f2c436fa700 -1 *** Caught signal (Aborted) ** 2024-03-28T00:31:21.061 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: in thread 7f2c436fa700 thread_name:ms_dispatch 2024-03-28T00:31:21.061 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: 2024-03-28T00:31:21.061 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:31:21.061 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f2c4caabd20] 2024-03-28T00:31:21.061 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: 2: gsignal() 2024-03-28T00:31:21.061 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: 3: abort() 2024-03-28T00:31:21.061 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f2c4dd244fb] 2024-03-28T00:31:21.061 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f2c4dd24667] 2024-03-28T00:31:21.061 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x561037b40223] 2024-03-28T00:31:21.062 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x5610379ba5d6] 2024-03-28T00:31:21.062 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x5610379bada1] 2024-03-28T00:31:21.062 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: 9: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x5610379ef09c] 2024-03-28T00:31:21.062 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x561037a0d233] 2024-03-28T00:31:21.062 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x561037a0f197] 2024-03-28T00:31:21.062 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x56103790e11b] 2024-03-28T00:31:21.062 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x56103790e6bc] 2024-03-28T00:31:21.062 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x5610378f81ff] 2024-03-28T00:31:21.062 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f2c4dfb6488] 2024-03-28T00:31:21.062 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: 16: (DispatchQueue::entry()+0x51f) [0x7f2c4dfb360f] 2024-03-28T00:31:21.062 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f2c4e086e81] 2024-03-28T00:31:21.062 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f2c4caa11ca] 2024-03-28T00:31:21.062 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: 19: clone() 2024-03-28T00:31:21.062 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T00:31:21.062 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[68074]: 2024-03-28T00:31:21.063 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:20 smithi165 podman[72150]: 2024-03-28 00:31:20.738143336 +0000 UTC m=+0.020273894 container died 19dc1822c951d8dfb1a7533592550913b638bcad6b225e1d337341dc7a36af1f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-03-28T00:31:21.633 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:21 smithi165 podman[72150]: 2024-03-28 00:31:21.360406406 +0000 UTC m=+0.642536950 container remove 19dc1822c951d8dfb1a7533592550913b638bcad6b225e1d337341dc7a36af1f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-03-28T00:31:21.633 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:21 smithi165 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.i.service: Main process exited, code=exited, status=134/n/a 2024-03-28T00:31:22.133 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:21 smithi165 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.i.service: Failed with result 'exit-code'. 2024-03-28T00:31:22.134 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:21 smithi165 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.i.service: Consumed 2min 23.327s CPU time. 2024-03-28T00:31:22.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:31:22 smithi165 ceph-mon[36302]: pgmap v151: 97 pgs: 97 active+clean; 163 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.0 MiB/s wr, 58 op/s 2024-03-28T00:31:22.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:31:22 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.165:6836/1011925619,v1:172.21.15.165:6837/1011925619] up:resolve 2024-03-28T00:31:22.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:31:22 smithi165 ceph-mon[36302]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:resolve 4 up:active 2024-03-28T00:31:22.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:31:22 smithi119 ceph-mon[31518]: pgmap v151: 97 pgs: 97 active+clean; 163 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.0 MiB/s wr, 58 op/s 2024-03-28T00:31:22.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:31:22 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.165:6836/1011925619,v1:172.21.15.165:6837/1011925619] up:resolve 2024-03-28T00:31:22.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:31:22 smithi119 ceph-mon[31518]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:resolve 4 up:active 2024-03-28T00:31:22.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:31:22 smithi132 ceph-mon[35941]: pgmap v151: 97 pgs: 97 active+clean; 163 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.0 MiB/s wr, 58 op/s 2024-03-28T00:31:22.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:31:22 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.165:6836/1011925619,v1:172.21.15.165:6837/1011925619] up:resolve 2024-03-28T00:31:22.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:31:22 smithi132 ceph-mon[35941]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:resolve 4 up:active 2024-03-28T00:31:24.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:31:24 smithi119 ceph-mon[31518]: pgmap v152: 97 pgs: 97 active+clean; 163 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.0 MiB/s wr, 58 op/s 2024-03-28T00:31:24.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:31:24 smithi165 ceph-mon[36302]: pgmap v152: 97 pgs: 97 active+clean; 163 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.0 MiB/s wr, 58 op/s 2024-03-28T00:31:24.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:31:24 smithi132 ceph-mon[35941]: pgmap v152: 97 pgs: 97 active+clean; 163 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.0 MiB/s wr, 58 op/s 2024-03-28T00:31:26.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:31:26 smithi165 ceph-mon[36302]: pgmap v153: 97 pgs: 97 active+clean; 166 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 606 KiB/s rd, 498 KiB/s wr, 27 op/s 2024-03-28T00:31:26.658 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:31:26 smithi119 ceph-mon[31518]: pgmap v153: 97 pgs: 97 active+clean; 166 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 606 KiB/s rd, 498 KiB/s wr, 27 op/s 2024-03-28T00:31:26.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:31:26 smithi132 ceph-mon[35941]: pgmap v153: 97 pgs: 97 active+clean; 166 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 606 KiB/s rd, 498 KiB/s wr, 27 op/s 2024-03-28T00:31:28.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:31:28 smithi165 ceph-mon[36302]: pgmap v154: 97 pgs: 97 active+clean; 166 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 606 KiB/s rd, 498 KiB/s wr, 27 op/s 2024-03-28T00:31:28.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:31:28 smithi119 ceph-mon[31518]: pgmap v154: 97 pgs: 97 active+clean; 166 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 606 KiB/s rd, 498 KiB/s wr, 27 op/s 2024-03-28T00:31:28.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:31:28 smithi132 ceph-mon[35941]: pgmap v154: 97 pgs: 97 active+clean; 166 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 606 KiB/s rd, 498 KiB/s wr, 27 op/s 2024-03-28T00:31:30.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:31:30 smithi165 ceph-mon[36302]: pgmap v155: 97 pgs: 97 active+clean; 168 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 542 KiB/s rd, 543 KiB/s wr, 18 op/s 2024-03-28T00:31:30.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:31:30 smithi119 ceph-mon[31518]: pgmap v155: 97 pgs: 97 active+clean; 168 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 542 KiB/s rd, 543 KiB/s wr, 18 op/s 2024-03-28T00:31:30.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:31:30 smithi132 ceph-mon[35941]: pgmap v155: 97 pgs: 97 active+clean; 168 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 542 KiB/s rd, 543 KiB/s wr, 18 op/s 2024-03-28T00:31:32.141 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:31 smithi165 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.i.service: Scheduled restart job, restart counter is at 1. 2024-03-28T00:31:32.142 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:31 smithi165 systemd[1]: Stopped Ceph mds.i for 10f73b92-ec98-11ee-95d0-87774f69a715. 2024-03-28T00:31:32.142 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:31 smithi165 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.i.service: Consumed 2min 23.327s CPU time. 2024-03-28T00:31:32.142 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:31 smithi165 systemd[1]: Starting Ceph mds.i for 10f73b92-ec98-11ee-95d0-87774f69a715... 2024-03-28T00:31:32.143 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:32 smithi165 podman[72250]: 2024-03-28 00:31:32.006046787 +0000 UTC m=+0.017839630 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-28T00:31:32.143 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:32 smithi165 podman[72250]: 2024-03-28 00:31:32.135620804 +0000 UTC m=+0.147413625 container create 90ac655df7c6427e5ffba59ca7bd034e0b657f8eaebc3c77624ba6e001074844 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-28T00:31:32.418 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:31:32 smithi165 ceph-mon[36302]: pgmap v156: 97 pgs: 97 active+clean; 168 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 492 KiB/s rd, 492 KiB/s wr, 17 op/s 2024-03-28T00:31:32.418 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:32 smithi165 podman[72250]: 2024-03-28 00:31:32.380150964 +0000 UTC m=+0.391943795 container init 90ac655df7c6427e5ffba59ca7bd034e0b657f8eaebc3c77624ba6e001074844 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c) 2024-03-28T00:31:32.418 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:32 smithi165 podman[72250]: 2024-03-28 00:31:32.382881667 +0000 UTC m=+0.394674505 container start 90ac655df7c6427e5ffba59ca7bd034e0b657f8eaebc3c77624ba6e001074844 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-03-28T00:31:32.505 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:31:32 smithi132 ceph-mon[35941]: pgmap v156: 97 pgs: 97 active+clean; 168 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 492 KiB/s rd, 492 KiB/s wr, 17 op/s 2024-03-28T00:31:32.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:31:32 smithi119 ceph-mon[31518]: pgmap v156: 97 pgs: 97 active+clean; 168 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 492 KiB/s rd, 492 KiB/s wr, 17 op/s 2024-03-28T00:31:32.883 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:32 smithi165 ceph-mds[72269]: -- 172.21.15.165:0/2507209270 <== mon.2 v2:172.21.15.132:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x556ff13d1a40 con 0x556ff13a4800 2024-03-28T00:31:32.884 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:32 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: starting mds.i at 2024-03-28T00:31:32.884 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:32 smithi165 bash[72250]: 90ac655df7c6427e5ffba59ca7bd034e0b657f8eaebc3c77624ba6e001074844 2024-03-28T00:31:32.884 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:32 smithi165 systemd[1]: Started Ceph mds.i for 10f73b92-ec98-11ee-95d0-87774f69a715. 2024-03-28T00:31:33.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:31:33 smithi165 ceph-mon[36302]: daemon mds.i restarted 2024-03-28T00:31:33.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:31:33 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-28T00:31:33.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:31:33 smithi165 ceph-mon[36302]: osdmap e93: 12 total, 12 up, 12 in 2024-03-28T00:31:33.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:31:33 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.165:6838/384483120,v1:172.21.15.165:6839/384483120] up:boot 2024-03-28T00:31:33.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:31:33 smithi165 ceph-mon[36302]: Standby daemon mds.j assigned to filesystem cephfs as rank 1 2024-03-28T00:31:33.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:31:33 smithi165 ceph-mon[36302]: fsmap cephfs:4/5 {0=b=up:active,2=h=up:active,3=c=up:resolve,4=f=up:active} 4 up:standby-replay 4 up:standby, 1 failed 2024-03-28T00:31:33.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:31:33 smithi165 ceph-mon[36302]: fsmap cephfs:5/5 3 up:standby-replay 4 up:standby 1 up:replay 1 up:resolve 3 up:active 2024-03-28T00:31:33.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:31:33 smithi119 ceph-mon[31518]: daemon mds.i restarted 2024-03-28T00:31:33.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:31:33 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-28T00:31:33.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:31:33 smithi119 ceph-mon[31518]: osdmap e93: 12 total, 12 up, 12 in 2024-03-28T00:31:33.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:31:33 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.165:6838/384483120,v1:172.21.15.165:6839/384483120] up:boot 2024-03-28T00:31:33.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:31:33 smithi119 ceph-mon[31518]: Standby daemon mds.j assigned to filesystem cephfs as rank 1 2024-03-28T00:31:33.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:31:33 smithi119 ceph-mon[31518]: fsmap cephfs:4/5 {0=b=up:active,2=h=up:active,3=c=up:resolve,4=f=up:active} 4 up:standby-replay 4 up:standby, 1 failed 2024-03-28T00:31:33.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:31:33 smithi119 ceph-mon[31518]: fsmap cephfs:5/5 3 up:standby-replay 4 up:standby 1 up:replay 1 up:resolve 3 up:active 2024-03-28T00:31:33.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:31:33 smithi132 ceph-mon[35941]: daemon mds.i restarted 2024-03-28T00:31:33.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:31:33 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-28T00:31:33.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:31:33 smithi132 ceph-mon[35941]: osdmap e93: 12 total, 12 up, 12 in 2024-03-28T00:31:33.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:31:33 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.165:6838/384483120,v1:172.21.15.165:6839/384483120] up:boot 2024-03-28T00:31:33.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:31:33 smithi132 ceph-mon[35941]: Standby daemon mds.j assigned to filesystem cephfs as rank 1 2024-03-28T00:31:33.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:31:33 smithi132 ceph-mon[35941]: fsmap cephfs:4/5 {0=b=up:active,2=h=up:active,3=c=up:resolve,4=f=up:active} 4 up:standby-replay 4 up:standby, 1 failed 2024-03-28T00:31:33.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:31:33 smithi132 ceph-mon[35941]: fsmap cephfs:5/5 3 up:standby-replay 4 up:standby 1 up:replay 1 up:resolve 3 up:active 2024-03-28T00:31:34.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:31:34 smithi165 ceph-mon[36302]: pgmap v158: 97 pgs: 97 active+clean; 168 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 542 KiB/s rd, 543 KiB/s wr, 18 op/s 2024-03-28T00:31:34.884 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:31:34 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.119:6834/744950487,v1:172.21.15.119:6835/744950487] up:resolve 2024-03-28T00:31:34.884 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:31:34 smithi165 ceph-mon[36302]: fsmap cephfs:5/5 3 up:standby-replay 4 up:standby 2 up:resolve 3 up:active 2024-03-28T00:31:34.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:31:34 smithi119 ceph-mon[31518]: pgmap v158: 97 pgs: 97 active+clean; 168 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 542 KiB/s rd, 543 KiB/s wr, 18 op/s 2024-03-28T00:31:34.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:31:34 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.119:6834/744950487,v1:172.21.15.119:6835/744950487] up:resolve 2024-03-28T00:31:34.907 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:31:34 smithi119 ceph-mon[31518]: fsmap cephfs:5/5 3 up:standby-replay 4 up:standby 2 up:resolve 3 up:active 2024-03-28T00:31:34.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:31:34 smithi132 ceph-mon[35941]: pgmap v158: 97 pgs: 97 active+clean; 168 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 542 KiB/s rd, 543 KiB/s wr, 18 op/s 2024-03-28T00:31:34.954 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:31:34 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.119:6834/744950487,v1:172.21.15.119:6835/744950487] up:resolve 2024-03-28T00:31:34.954 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:31:34 smithi132 ceph-mon[35941]: fsmap cephfs:5/5 3 up:standby-replay 4 up:standby 2 up:resolve 3 up:active 2024-03-28T00:31:36.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:31:36 smithi165 ceph-mon[36302]: pgmap v159: 97 pgs: 97 active+clean; 168 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 292 KiB/s rd, 288 KiB/s wr, 11 op/s 2024-03-28T00:31:36.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:31:36 smithi165 ceph-mon[36302]: reconnect by client.25355 172.21.15.119:0/3071152482 after 0.00299995 2024-03-28T00:31:36.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:31:36 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.165:6836/1011925619,v1:172.21.15.165:6837/1011925619] up:reconnect 2024-03-28T00:31:36.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:31:36 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.119:6834/744950487,v1:172.21.15.119:6835/744950487] up:reconnect 2024-03-28T00:31:36.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:31:36 smithi165 ceph-mon[36302]: fsmap cephfs:5/5 3 up:standby-replay 4 up:standby 2 up:reconnect 3 up:active 2024-03-28T00:31:36.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:31:36 smithi165 ceph-mon[36302]: reconnect by client.25264 v1:172.21.15.119:0/3863511511 after 0.0199997 2024-03-28T00:31:36.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:31:36 smithi165 ceph-mon[36302]: reconnect by client.25355 172.21.15.119:0/3071152482 after 0.0079998 2024-03-28T00:31:36.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:31:36 smithi165 ceph-mon[36302]: reconnect by client.25264 v1:172.21.15.119:0/3863511511 after 0.00899978 2024-03-28T00:31:36.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:31:36 smithi119 ceph-mon[31518]: pgmap v159: 97 pgs: 97 active+clean; 168 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 292 KiB/s rd, 288 KiB/s wr, 11 op/s 2024-03-28T00:31:36.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:31:36 smithi119 ceph-mon[31518]: reconnect by client.25355 172.21.15.119:0/3071152482 after 0.00299995 2024-03-28T00:31:36.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:31:36 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.165:6836/1011925619,v1:172.21.15.165:6837/1011925619] up:reconnect 2024-03-28T00:31:36.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:31:36 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.119:6834/744950487,v1:172.21.15.119:6835/744950487] up:reconnect 2024-03-28T00:31:36.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:31:36 smithi119 ceph-mon[31518]: fsmap cephfs:5/5 3 up:standby-replay 4 up:standby 2 up:reconnect 3 up:active 2024-03-28T00:31:36.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:31:36 smithi119 ceph-mon[31518]: reconnect by client.25264 v1:172.21.15.119:0/3863511511 after 0.0199997 2024-03-28T00:31:36.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:31:36 smithi119 ceph-mon[31518]: reconnect by client.25355 172.21.15.119:0/3071152482 after 0.0079998 2024-03-28T00:31:36.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:31:36 smithi119 ceph-mon[31518]: reconnect by client.25264 v1:172.21.15.119:0/3863511511 after 0.00899978 2024-03-28T00:31:36.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:31:36 smithi132 ceph-mon[35941]: pgmap v159: 97 pgs: 97 active+clean; 168 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 292 KiB/s rd, 288 KiB/s wr, 11 op/s 2024-03-28T00:31:36.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:31:36 smithi132 ceph-mon[35941]: reconnect by client.25355 172.21.15.119:0/3071152482 after 0.00299995 2024-03-28T00:31:36.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:31:36 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.165:6836/1011925619,v1:172.21.15.165:6837/1011925619] up:reconnect 2024-03-28T00:31:36.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:31:36 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.119:6834/744950487,v1:172.21.15.119:6835/744950487] up:reconnect 2024-03-28T00:31:36.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:31:36 smithi132 ceph-mon[35941]: fsmap cephfs:5/5 3 up:standby-replay 4 up:standby 2 up:reconnect 3 up:active 2024-03-28T00:31:36.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:31:36 smithi132 ceph-mon[35941]: reconnect by client.25264 v1:172.21.15.119:0/3863511511 after 0.0199997 2024-03-28T00:31:36.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:31:36 smithi132 ceph-mon[35941]: reconnect by client.25355 172.21.15.119:0/3071152482 after 0.0079998 2024-03-28T00:31:36.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:31:36 smithi132 ceph-mon[35941]: reconnect by client.25264 v1:172.21.15.119:0/3863511511 after 0.00899978 2024-03-28T00:31:37.588 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.165:6836/1011925619,v1:172.21.15.165:6837/1011925619] up:rejoin 2024-03-28T00:31:37.588 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.119:6834/744950487,v1:172.21.15.119:6835/744950487] up:rejoin 2024-03-28T00:31:37.588 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-mon[31518]: fsmap cephfs:5/5 3 up:standby-replay 4 up:standby 2 up:rejoin 3 up:active 2024-03-28T00:31:37.588 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-mon[31518]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-03-28T00:31:37.588 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-mon[31518]: daemon mds.j is now active in filesystem cephfs as rank 1 2024-03-28T00:31:37.589 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7f32b2a00700 time 2024-03-28T00:31:37.327754+0000 2024-03-28T00:31:37.589 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T00:31:37.589 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:31:37.589 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f32be30e4a1] 2024-03-28T00:31:37.589 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f32be30e667] 2024-03-28T00:31:37.589 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55f979598223] 2024-03-28T00:31:37.589 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55f9794125d6] 2024-03-28T00:31:37.590 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55f979412da1] 2024-03-28T00:31:37.590 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: 6: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x55f97944709c] 2024-03-28T00:31:37.590 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55f979465233] 2024-03-28T00:31:37.590 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55f979467197] 2024-03-28T00:31:37.590 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55f97936611b] 2024-03-28T00:31:37.590 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55f9793666bc] 2024-03-28T00:31:37.590 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55f9793501ff] 2024-03-28T00:31:37.590 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f32be5a0488] 2024-03-28T00:31:37.590 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: 13: (DispatchQueue::entry()+0x51f) [0x7f32be59d60f] 2024-03-28T00:31:37.590 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f32be670e81] 2024-03-28T00:31:37.590 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f32bd08b1ca] 2024-03-28T00:31:37.590 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: 16: clone() 2024-03-28T00:31:37.591 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: 2024-03-28T00:31:37.346+0000 7f32b2a00700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7f32b2a00700 time 2024-03-28T00:31:37.327754+0000 2024-03-28T00:31:37.591 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T00:31:37.591 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: 2024-03-28T00:31:37.591 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:31:37.591 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f32be30e4a1] 2024-03-28T00:31:37.592 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f32be30e667] 2024-03-28T00:31:37.592 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55f979598223] 2024-03-28T00:31:37.592 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55f9794125d6] 2024-03-28T00:31:37.592 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55f979412da1] 2024-03-28T00:31:37.592 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: 6: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x55f97944709c] 2024-03-28T00:31:37.592 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55f979465233] 2024-03-28T00:31:37.592 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55f979467197] 2024-03-28T00:31:37.592 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55f97936611b] 2024-03-28T00:31:37.592 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55f9793666bc] 2024-03-28T00:31:37.592 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55f9793501ff] 2024-03-28T00:31:37.592 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f32be5a0488] 2024-03-28T00:31:37.592 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: 13: (DispatchQueue::entry()+0x51f) [0x7f32be59d60f] 2024-03-28T00:31:37.593 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f32be670e81] 2024-03-28T00:31:37.593 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f32bd08b1ca] 2024-03-28T00:31:37.593 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: 16: clone() 2024-03-28T00:31:37.593 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: 2024-03-28T00:31:37.593 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: *** Caught signal (Aborted) ** 2024-03-28T00:31:37.593 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: in thread 7f32b2a00700 thread_name:ms_dispatch 2024-03-28T00:31:37.593 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:31:37.593 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f32bd095d20] 2024-03-28T00:31:37.593 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: 2: gsignal() 2024-03-28T00:31:37.593 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: 3: abort() 2024-03-28T00:31:37.593 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f32be30e4fb] 2024-03-28T00:31:37.593 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f32be30e667] 2024-03-28T00:31:37.594 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55f979598223] 2024-03-28T00:31:37.594 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55f9794125d6] 2024-03-28T00:31:37.594 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55f979412da1] 2024-03-28T00:31:37.594 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: 9: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x55f97944709c] 2024-03-28T00:31:37.594 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55f979465233] 2024-03-28T00:31:37.594 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55f979467197] 2024-03-28T00:31:37.594 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55f97936611b] 2024-03-28T00:31:37.594 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55f9793666bc] 2024-03-28T00:31:37.594 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55f9793501ff] 2024-03-28T00:31:37.594 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f32be5a0488] 2024-03-28T00:31:37.594 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: 16: (DispatchQueue::entry()+0x51f) [0x7f32be59d60f] 2024-03-28T00:31:37.594 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f32be670e81] 2024-03-28T00:31:37.594 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f32bd08b1ca] 2024-03-28T00:31:37.595 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: 19: clone() 2024-03-28T00:31:37.595 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: 2024-03-28T00:31:37.358+0000 7f32b2a00700 -1 *** Caught signal (Aborted) ** 2024-03-28T00:31:37.595 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: in thread 7f32b2a00700 thread_name:ms_dispatch 2024-03-28T00:31:37.595 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: 2024-03-28T00:31:37.595 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:31:37.595 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f32bd095d20] 2024-03-28T00:31:37.595 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: 2: gsignal() 2024-03-28T00:31:37.595 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: 3: abort() 2024-03-28T00:31:37.595 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f32be30e4fb] 2024-03-28T00:31:37.595 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f32be30e667] 2024-03-28T00:31:37.595 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55f979598223] 2024-03-28T00:31:37.595 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55f9794125d6] 2024-03-28T00:31:37.595 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55f979412da1] 2024-03-28T00:31:37.595 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: 9: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x55f97944709c] 2024-03-28T00:31:37.595 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55f979465233] 2024-03-28T00:31:37.596 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55f979467197] 2024-03-28T00:31:37.596 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55f97936611b] 2024-03-28T00:31:37.596 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55f9793666bc] 2024-03-28T00:31:37.596 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55f9793501ff] 2024-03-28T00:31:37.596 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f32be5a0488] 2024-03-28T00:31:37.596 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: 16: (DispatchQueue::entry()+0x51f) [0x7f32be59d60f] 2024-03-28T00:31:37.596 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f32be670e81] 2024-03-28T00:31:37.596 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f32bd08b1ca] 2024-03-28T00:31:37.596 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: 19: clone() 2024-03-28T00:31:37.596 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T00:31:37.596 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: 2024-03-28T00:31:37.596 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: -1> 2024-03-28T00:31:37.346+0000 7f32b2a00700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7f32b2a00700 time 2024-03-28T00:31:37.327754+0000 2024-03-28T00:31:37.596 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T00:31:37.596 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: 2024-03-28T00:31:37.597 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:31:37.597 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f32be30e4a1] 2024-03-28T00:31:37.597 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f32be30e667] 2024-03-28T00:31:37.597 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55f979598223] 2024-03-28T00:31:37.598 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55f9794125d6] 2024-03-28T00:31:37.598 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55f979412da1] 2024-03-28T00:31:37.598 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: 6: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x55f97944709c] 2024-03-28T00:31:37.598 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55f979465233] 2024-03-28T00:31:37.598 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55f979467197] 2024-03-28T00:31:37.598 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55f97936611b] 2024-03-28T00:31:37.598 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55f9793666bc] 2024-03-28T00:31:37.598 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55f9793501ff] 2024-03-28T00:31:37.598 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f32be5a0488] 2024-03-28T00:31:37.598 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: 13: (DispatchQueue::entry()+0x51f) [0x7f32be59d60f] 2024-03-28T00:31:37.598 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f32be670e81] 2024-03-28T00:31:37.598 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f32bd08b1ca] 2024-03-28T00:31:37.598 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: 16: clone() 2024-03-28T00:31:37.599 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: 2024-03-28T00:31:37.599 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: 0> 2024-03-28T00:31:37.358+0000 7f32b2a00700 -1 *** Caught signal (Aborted) ** 2024-03-28T00:31:37.599 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: in thread 7f32b2a00700 thread_name:ms_dispatch 2024-03-28T00:31:37.599 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: 2024-03-28T00:31:37.599 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:31:37.599 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f32bd095d20] 2024-03-28T00:31:37.599 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: 2: gsignal() 2024-03-28T00:31:37.599 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: 3: abort() 2024-03-28T00:31:37.599 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f32be30e4fb] 2024-03-28T00:31:37.599 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f32be30e667] 2024-03-28T00:31:37.599 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55f979598223] 2024-03-28T00:31:37.599 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55f9794125d6] 2024-03-28T00:31:37.599 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55f979412da1] 2024-03-28T00:31:37.600 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: 9: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x55f97944709c] 2024-03-28T00:31:37.600 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55f979465233] 2024-03-28T00:31:37.600 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55f979467197] 2024-03-28T00:31:37.600 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55f97936611b] 2024-03-28T00:31:37.600 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55f9793666bc] 2024-03-28T00:31:37.600 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55f9793501ff] 2024-03-28T00:31:37.600 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f32be5a0488] 2024-03-28T00:31:37.600 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: 16: (DispatchQueue::entry()+0x51f) [0x7f32be59d60f] 2024-03-28T00:31:37.600 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f32be670e81] 2024-03-28T00:31:37.600 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f32bd08b1ca] 2024-03-28T00:31:37.600 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: 19: clone() 2024-03-28T00:31:37.600 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T00:31:37.600 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: 2024-03-28T00:31:37.600 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: -5> 2024-03-28T00:31:37.346+0000 7f32b2a00700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7f32b2a00700 time 2024-03-28T00:31:37.327754+0000 2024-03-28T00:31:37.601 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T00:31:37.601 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: 2024-03-28T00:31:37.601 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:31:37.601 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f32be30e4a1] 2024-03-28T00:31:37.601 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f32be30e667] 2024-03-28T00:31:37.601 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55f979598223] 2024-03-28T00:31:37.601 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55f9794125d6] 2024-03-28T00:31:37.601 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55f979412da1] 2024-03-28T00:31:37.601 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: 6: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x55f97944709c] 2024-03-28T00:31:37.601 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55f979465233] 2024-03-28T00:31:37.601 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55f979467197] 2024-03-28T00:31:37.601 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55f97936611b] 2024-03-28T00:31:37.601 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55f9793666bc] 2024-03-28T00:31:37.601 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55f9793501ff] 2024-03-28T00:31:37.602 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f32be5a0488] 2024-03-28T00:31:37.602 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: 13: (DispatchQueue::entry()+0x51f) [0x7f32be59d60f] 2024-03-28T00:31:37.602 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f32be670e81] 2024-03-28T00:31:37.602 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f32bd08b1ca] 2024-03-28T00:31:37.602 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: 16: clone() 2024-03-28T00:31:37.602 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: 2024-03-28T00:31:37.602 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: -4> 2024-03-28T00:31:37.358+0000 7f32b2a00700 -1 *** Caught signal (Aborted) ** 2024-03-28T00:31:37.602 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: in thread 7f32b2a00700 thread_name:ms_dispatch 2024-03-28T00:31:37.602 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: 2024-03-28T00:31:37.602 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:31:37.602 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f32bd095d20] 2024-03-28T00:31:37.602 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: 2: gsignal() 2024-03-28T00:31:37.602 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: 3: abort() 2024-03-28T00:31:37.603 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f32be30e4fb] 2024-03-28T00:31:37.603 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f32be30e667] 2024-03-28T00:31:37.603 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55f979598223] 2024-03-28T00:31:37.603 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55f9794125d6] 2024-03-28T00:31:37.603 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55f979412da1] 2024-03-28T00:31:37.603 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: 9: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x55f97944709c] 2024-03-28T00:31:37.603 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55f979465233] 2024-03-28T00:31:37.603 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55f979467197] 2024-03-28T00:31:37.603 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55f97936611b] 2024-03-28T00:31:37.603 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55f9793666bc] 2024-03-28T00:31:37.603 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55f9793501ff] 2024-03-28T00:31:37.603 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f32be5a0488] 2024-03-28T00:31:37.603 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: 16: (DispatchQueue::entry()+0x51f) [0x7f32be59d60f] 2024-03-28T00:31:37.604 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f32be670e81] 2024-03-28T00:31:37.604 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f32bd08b1ca] 2024-03-28T00:31:37.604 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: 19: clone() 2024-03-28T00:31:37.604 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T00:31:37.604 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[68782]: 2024-03-28T00:31:37.604 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:37 smithi119 podman[106975]: 2024-03-28 00:31:37.574109703 +0000 UTC m=+0.126341116 container died 91c6d8f59d030cc4ae168abd47e8a10ab49251520429c439228e62e74fac9f42 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-03-28T00:31:37.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.165:6836/1011925619,v1:172.21.15.165:6837/1011925619] up:rejoin 2024-03-28T00:31:37.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.119:6834/744950487,v1:172.21.15.119:6835/744950487] up:rejoin 2024-03-28T00:31:37.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-mon[36302]: fsmap cephfs:5/5 3 up:standby-replay 4 up:standby 2 up:rejoin 3 up:active 2024-03-28T00:31:37.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-mon[36302]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-03-28T00:31:37.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-mon[36302]: daemon mds.j is now active in filesystem cephfs as rank 1 2024-03-28T00:31:37.634 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7f6b75e78700 time 2024-03-28T00:31:37.322408+0000 2024-03-28T00:31:37.634 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T00:31:37.634 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:31:37.634 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f6b804a24a1] 2024-03-28T00:31:37.634 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f6b804a2667] 2024-03-28T00:31:37.634 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x5557c2e3c223] 2024-03-28T00:31:37.634 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x5557c2cb65d6] 2024-03-28T00:31:37.634 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x5557c2cb6da1] 2024-03-28T00:31:37.634 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: 6: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x5557c2ceb09c] 2024-03-28T00:31:37.635 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x5557c2d09233] 2024-03-28T00:31:37.635 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5557c2d0b197] 2024-03-28T00:31:37.635 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x5557c2c0a11b] 2024-03-28T00:31:37.635 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5557c2c0a6bc] 2024-03-28T00:31:37.635 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x5557c2bf41ff] 2024-03-28T00:31:37.635 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f6b80734488] 2024-03-28T00:31:37.635 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: 13: (DispatchQueue::entry()+0x51f) [0x7f6b8073160f] 2024-03-28T00:31:37.635 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f6b80804e81] 2024-03-28T00:31:37.635 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f6b7f21f1ca] 2024-03-28T00:31:37.635 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: 16: clone() 2024-03-28T00:31:37.636 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: 2024-03-28T00:31:37.323+0000 7f6b75e78700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7f6b75e78700 time 2024-03-28T00:31:37.322408+0000 2024-03-28T00:31:37.636 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T00:31:37.636 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: 2024-03-28T00:31:37.636 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:31:37.636 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f6b804a24a1] 2024-03-28T00:31:37.636 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f6b804a2667] 2024-03-28T00:31:37.636 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x5557c2e3c223] 2024-03-28T00:31:37.636 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x5557c2cb65d6] 2024-03-28T00:31:37.636 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x5557c2cb6da1] 2024-03-28T00:31:37.636 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: 6: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x5557c2ceb09c] 2024-03-28T00:31:37.637 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x5557c2d09233] 2024-03-28T00:31:37.637 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5557c2d0b197] 2024-03-28T00:31:37.637 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x5557c2c0a11b] 2024-03-28T00:31:37.637 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5557c2c0a6bc] 2024-03-28T00:31:37.637 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x5557c2bf41ff] 2024-03-28T00:31:37.637 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f6b80734488] 2024-03-28T00:31:37.637 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: 13: (DispatchQueue::entry()+0x51f) [0x7f6b8073160f] 2024-03-28T00:31:37.637 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f6b80804e81] 2024-03-28T00:31:37.637 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f6b7f21f1ca] 2024-03-28T00:31:37.637 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: 16: clone() 2024-03-28T00:31:37.637 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: 2024-03-28T00:31:37.637 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: *** Caught signal (Aborted) ** 2024-03-28T00:31:37.637 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: in thread 7f6b75e78700 thread_name:ms_dispatch 2024-03-28T00:31:37.637 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:31:37.638 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f6b7f229d20] 2024-03-28T00:31:37.638 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: 2: gsignal() 2024-03-28T00:31:37.638 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: 3: abort() 2024-03-28T00:31:37.638 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f6b804a24fb] 2024-03-28T00:31:37.638 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f6b804a2667] 2024-03-28T00:31:37.638 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x5557c2e3c223] 2024-03-28T00:31:37.638 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x5557c2cb65d6] 2024-03-28T00:31:37.638 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x5557c2cb6da1] 2024-03-28T00:31:37.638 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: 9: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x5557c2ceb09c] 2024-03-28T00:31:37.638 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x5557c2d09233] 2024-03-28T00:31:37.638 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5557c2d0b197] 2024-03-28T00:31:37.638 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x5557c2c0a11b] 2024-03-28T00:31:37.638 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5557c2c0a6bc] 2024-03-28T00:31:37.638 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x5557c2bf41ff] 2024-03-28T00:31:37.639 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f6b80734488] 2024-03-28T00:31:37.639 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: 16: (DispatchQueue::entry()+0x51f) [0x7f6b8073160f] 2024-03-28T00:31:37.639 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f6b80804e81] 2024-03-28T00:31:37.639 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f6b7f21f1ca] 2024-03-28T00:31:37.639 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: 19: clone() 2024-03-28T00:31:37.639 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: 2024-03-28T00:31:37.325+0000 7f6b75e78700 -1 *** Caught signal (Aborted) ** 2024-03-28T00:31:37.639 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: in thread 7f6b75e78700 thread_name:ms_dispatch 2024-03-28T00:31:37.639 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: 2024-03-28T00:31:37.639 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:31:37.639 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f6b7f229d20] 2024-03-28T00:31:37.639 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: 2: gsignal() 2024-03-28T00:31:37.639 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: 3: abort() 2024-03-28T00:31:37.639 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f6b804a24fb] 2024-03-28T00:31:37.639 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f6b804a2667] 2024-03-28T00:31:37.639 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x5557c2e3c223] 2024-03-28T00:31:37.640 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x5557c2cb65d6] 2024-03-28T00:31:37.640 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x5557c2cb6da1] 2024-03-28T00:31:37.640 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: 9: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x5557c2ceb09c] 2024-03-28T00:31:37.640 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x5557c2d09233] 2024-03-28T00:31:37.640 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5557c2d0b197] 2024-03-28T00:31:37.640 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x5557c2c0a11b] 2024-03-28T00:31:37.640 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5557c2c0a6bc] 2024-03-28T00:31:37.640 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x5557c2bf41ff] 2024-03-28T00:31:37.640 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f6b80734488] 2024-03-28T00:31:37.640 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: 16: (DispatchQueue::entry()+0x51f) [0x7f6b8073160f] 2024-03-28T00:31:37.640 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f6b80804e81] 2024-03-28T00:31:37.640 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f6b7f21f1ca] 2024-03-28T00:31:37.640 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: 19: clone() 2024-03-28T00:31:37.640 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T00:31:37.641 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: 2024-03-28T00:31:37.641 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: -1> 2024-03-28T00:31:37.323+0000 7f6b75e78700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7f6b75e78700 time 2024-03-28T00:31:37.322408+0000 2024-03-28T00:31:37.641 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T00:31:37.641 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: 2024-03-28T00:31:37.641 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:31:37.641 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f6b804a24a1] 2024-03-28T00:31:37.641 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f6b804a2667] 2024-03-28T00:31:37.642 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x5557c2e3c223] 2024-03-28T00:31:37.642 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x5557c2cb65d6] 2024-03-28T00:31:37.642 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x5557c2cb6da1] 2024-03-28T00:31:37.642 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: 6: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x5557c2ceb09c] 2024-03-28T00:31:37.642 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x5557c2d09233] 2024-03-28T00:31:37.642 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5557c2d0b197] 2024-03-28T00:31:37.642 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x5557c2c0a11b] 2024-03-28T00:31:37.642 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5557c2c0a6bc] 2024-03-28T00:31:37.642 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x5557c2bf41ff] 2024-03-28T00:31:37.642 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f6b80734488] 2024-03-28T00:31:37.643 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: 13: (DispatchQueue::entry()+0x51f) [0x7f6b8073160f] 2024-03-28T00:31:37.643 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f6b80804e81] 2024-03-28T00:31:37.643 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f6b7f21f1ca] 2024-03-28T00:31:37.643 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: 16: clone() 2024-03-28T00:31:37.643 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: 2024-03-28T00:31:37.643 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: 0> 2024-03-28T00:31:37.325+0000 7f6b75e78700 -1 *** Caught signal (Aborted) ** 2024-03-28T00:31:37.643 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: in thread 7f6b75e78700 thread_name:ms_dispatch 2024-03-28T00:31:37.643 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: 2024-03-28T00:31:37.643 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:31:37.643 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f6b7f229d20] 2024-03-28T00:31:37.643 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: 2: gsignal() 2024-03-28T00:31:37.643 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: 3: abort() 2024-03-28T00:31:37.643 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f6b804a24fb] 2024-03-28T00:31:37.643 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f6b804a2667] 2024-03-28T00:31:37.644 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x5557c2e3c223] 2024-03-28T00:31:37.644 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x5557c2cb65d6] 2024-03-28T00:31:37.644 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x5557c2cb6da1] 2024-03-28T00:31:37.644 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: 9: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x5557c2ceb09c] 2024-03-28T00:31:37.644 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x5557c2d09233] 2024-03-28T00:31:37.644 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5557c2d0b197] 2024-03-28T00:31:37.644 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x5557c2c0a11b] 2024-03-28T00:31:37.644 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5557c2c0a6bc] 2024-03-28T00:31:37.644 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x5557c2bf41ff] 2024-03-28T00:31:37.644 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f6b80734488] 2024-03-28T00:31:37.644 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: 16: (DispatchQueue::entry()+0x51f) [0x7f6b8073160f] 2024-03-28T00:31:37.644 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f6b80804e81] 2024-03-28T00:31:37.644 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f6b7f21f1ca] 2024-03-28T00:31:37.644 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: 19: clone() 2024-03-28T00:31:37.645 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T00:31:37.645 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: 2024-03-28T00:31:37.645 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: -1> 2024-03-28T00:31:37.323+0000 7f6b75e78700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7f6b75e78700 time 2024-03-28T00:31:37.322408+0000 2024-03-28T00:31:37.645 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T00:31:37.645 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: 2024-03-28T00:31:37.645 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:31:37.645 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f6b804a24a1] 2024-03-28T00:31:37.645 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f6b804a2667] 2024-03-28T00:31:37.645 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x5557c2e3c223] 2024-03-28T00:31:37.645 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x5557c2cb65d6] 2024-03-28T00:31:37.645 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x5557c2cb6da1] 2024-03-28T00:31:37.645 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: 6: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x5557c2ceb09c] 2024-03-28T00:31:37.645 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x5557c2d09233] 2024-03-28T00:31:37.645 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5557c2d0b197] 2024-03-28T00:31:37.646 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x5557c2c0a11b] 2024-03-28T00:31:37.646 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5557c2c0a6bc] 2024-03-28T00:31:37.646 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x5557c2bf41ff] 2024-03-28T00:31:37.646 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f6b80734488] 2024-03-28T00:31:37.646 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: 13: (DispatchQueue::entry()+0x51f) [0x7f6b8073160f] 2024-03-28T00:31:37.646 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f6b80804e81] 2024-03-28T00:31:37.646 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f6b7f21f1ca] 2024-03-28T00:31:37.646 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: 16: clone() 2024-03-28T00:31:37.646 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: 2024-03-28T00:31:37.646 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: 0> 2024-03-28T00:31:37.325+0000 7f6b75e78700 -1 *** Caught signal (Aborted) ** 2024-03-28T00:31:37.646 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: in thread 7f6b75e78700 thread_name:ms_dispatch 2024-03-28T00:31:37.646 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: 2024-03-28T00:31:37.646 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:31:37.646 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f6b7f229d20] 2024-03-28T00:31:37.647 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: 2: gsignal() 2024-03-28T00:31:37.647 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: 3: abort() 2024-03-28T00:31:37.647 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f6b804a24fb] 2024-03-28T00:31:37.647 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f6b804a2667] 2024-03-28T00:31:37.647 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x5557c2e3c223] 2024-03-28T00:31:37.647 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x5557c2cb65d6] 2024-03-28T00:31:37.647 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x5557c2cb6da1] 2024-03-28T00:31:37.647 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: 9: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x5557c2ceb09c] 2024-03-28T00:31:37.647 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x5557c2d09233] 2024-03-28T00:31:37.647 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5557c2d0b197] 2024-03-28T00:31:37.647 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x5557c2c0a11b] 2024-03-28T00:31:37.647 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5557c2c0a6bc] 2024-03-28T00:31:37.647 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x5557c2bf41ff] 2024-03-28T00:31:37.647 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f6b80734488] 2024-03-28T00:31:37.648 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: 16: (DispatchQueue::entry()+0x51f) [0x7f6b8073160f] 2024-03-28T00:31:37.648 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f6b80804e81] 2024-03-28T00:31:37.648 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f6b7f21f1ca] 2024-03-28T00:31:37.648 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: 19: clone() 2024-03-28T00:31:37.648 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T00:31:37.648 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[71907]: 2024-03-28T00:31:37.648 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:37 smithi165 podman[72314]: 2024-03-28 00:31:37.399372455 +0000 UTC m=+0.023885764 container died 0d115520da96cf874a08bf0f58fcde1967850ff57c94e5d2e697ead2bffb8478 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c) 2024-03-28T00:31:37.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:31:37 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.165:6836/1011925619,v1:172.21.15.165:6837/1011925619] up:rejoin 2024-03-28T00:31:37.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:31:37 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.119:6834/744950487,v1:172.21.15.119:6835/744950487] up:rejoin 2024-03-28T00:31:37.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:31:37 smithi132 ceph-mon[35941]: fsmap cephfs:5/5 3 up:standby-replay 4 up:standby 2 up:rejoin 3 up:active 2024-03-28T00:31:37.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:31:37 smithi132 ceph-mon[35941]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-03-28T00:31:37.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:31:37 smithi132 ceph-mon[35941]: daemon mds.j is now active in filesystem cephfs as rank 1 2024-03-28T00:31:38.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:31:38 smithi119 ceph-mon[31518]: pgmap v160: 97 pgs: 97 active+clean; 168 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 292 KiB/s rd, 288 KiB/s wr, 11 op/s 2024-03-28T00:31:38.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:31:38 smithi119 ceph-mon[31518]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-28T00:31:38.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:31:38 smithi119 ceph-mon[31518]: Cluster is now healthy 2024-03-28T00:31:38.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:31:38 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.165:6836/1011925619,v1:172.21.15.165:6837/1011925619] up:active 2024-03-28T00:31:38.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:31:38 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.119:6834/744950487,v1:172.21.15.119:6835/744950487] up:active 2024-03-28T00:31:38.407 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:31:38 smithi119 ceph-mon[31518]: fsmap cephfs:5 3 up:standby-replay 4 up:standby 5 up:active 2024-03-28T00:31:38.407 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:31:38 smithi119 ceph-mon[31518]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-28T00:31:38.420 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:38 smithi165 podman[72314]: 2024-03-28 00:31:38.143678666 +0000 UTC m=+0.768191943 container remove 0d115520da96cf874a08bf0f58fcde1967850ff57c94e5d2e697ead2bffb8478 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2) 2024-03-28T00:31:38.420 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:38 smithi165 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.c.service: Main process exited, code=exited, status=134/n/a 2024-03-28T00:31:38.420 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:31:38 smithi165 ceph-mon[36302]: pgmap v160: 97 pgs: 97 active+clean; 168 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 292 KiB/s rd, 288 KiB/s wr, 11 op/s 2024-03-28T00:31:38.420 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:31:38 smithi165 ceph-mon[36302]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-28T00:31:38.421 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:31:38 smithi165 ceph-mon[36302]: Cluster is now healthy 2024-03-28T00:31:38.421 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:31:38 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.165:6836/1011925619,v1:172.21.15.165:6837/1011925619] up:active 2024-03-28T00:31:38.421 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:31:38 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.119:6834/744950487,v1:172.21.15.119:6835/744950487] up:active 2024-03-28T00:31:38.421 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:31:38 smithi165 ceph-mon[36302]: fsmap cephfs:5 3 up:standby-replay 4 up:standby 5 up:active 2024-03-28T00:31:38.421 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:31:38 smithi165 ceph-mon[36302]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-28T00:31:38.705 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:31:38 smithi132 ceph-mon[35941]: pgmap v160: 97 pgs: 97 active+clean; 168 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 292 KiB/s rd, 288 KiB/s wr, 11 op/s 2024-03-28T00:31:38.706 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:31:38 smithi132 ceph-mon[35941]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-28T00:31:38.706 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:31:38 smithi132 ceph-mon[35941]: Cluster is now healthy 2024-03-28T00:31:38.706 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:31:38 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.165:6836/1011925619,v1:172.21.15.165:6837/1011925619] up:active 2024-03-28T00:31:38.706 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:31:38 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.119:6834/744950487,v1:172.21.15.119:6835/744950487] up:active 2024-03-28T00:31:38.706 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:31:38 smithi132 ceph-mon[35941]: fsmap cephfs:5 3 up:standby-replay 4 up:standby 5 up:active 2024-03-28T00:31:38.706 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:31:38 smithi132 ceph-mon[35941]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-28T00:31:38.883 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:38 smithi165 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.c.service: Failed with result 'exit-code'. 2024-03-28T00:31:39.149 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:38 smithi119 podman[106975]: 2024-03-28 00:31:38.717588442 +0000 UTC m=+1.269819798 container remove 91c6d8f59d030cc4ae168abd47e8a10ab49251520429c439228e62e74fac9f42 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319) 2024-03-28T00:31:39.150 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:38 smithi119 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.j.service: Main process exited, code=exited, status=134/n/a 2024-03-28T00:31:39.656 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:39 smithi119 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.j.service: Failed with result 'exit-code'. 2024-03-28T00:31:39.656 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:39 smithi119 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.j.service: Consumed 3.073s CPU time. 2024-03-28T00:31:40.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:31:40 smithi165 ceph-mon[36302]: pgmap v161: 97 pgs: 97 active+clean; 177 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 936 KiB/s wr, 132 op/s 2024-03-28T00:31:40.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:31:40 smithi119 ceph-mon[31518]: pgmap v161: 97 pgs: 97 active+clean; 177 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 936 KiB/s wr, 132 op/s 2024-03-28T00:31:40.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:31:40 smithi132 ceph-mon[35941]: pgmap v161: 97 pgs: 97 active+clean; 177 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 936 KiB/s wr, 132 op/s 2024-03-28T00:31:42.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:31:42 smithi165 ceph-mon[36302]: pgmap v162: 97 pgs: 97 active+clean; 177 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 936 KiB/s wr, 132 op/s 2024-03-28T00:31:42.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:31:42 smithi119 ceph-mon[31518]: pgmap v162: 97 pgs: 97 active+clean; 177 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 936 KiB/s wr, 132 op/s 2024-03-28T00:31:42.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:31:42 smithi132 ceph-mon[35941]: pgmap v162: 97 pgs: 97 active+clean; 177 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 936 KiB/s wr, 132 op/s 2024-03-28T00:31:44.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:31:44 smithi165 ceph-mon[36302]: pgmap v163: 97 pgs: 97 active+clean; 177 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 969 KiB/s rd, 876 KiB/s wr, 124 op/s 2024-03-28T00:31:44.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:31:44 smithi119 ceph-mon[31518]: pgmap v163: 97 pgs: 97 active+clean; 177 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 969 KiB/s rd, 876 KiB/s wr, 124 op/s 2024-03-28T00:31:44.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:31:44 smithi132 ceph-mon[35941]: pgmap v163: 97 pgs: 97 active+clean; 177 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 969 KiB/s rd, 876 KiB/s wr, 124 op/s 2024-03-28T00:31:46.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:31:46 smithi165 ceph-mon[36302]: pgmap v164: 97 pgs: 97 active+clean; 179 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 174 op/s 2024-03-28T00:31:46.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:31:46 smithi119 ceph-mon[31518]: pgmap v164: 97 pgs: 97 active+clean; 179 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 174 op/s 2024-03-28T00:31:46.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:31:46 smithi132 ceph-mon[35941]: pgmap v164: 97 pgs: 97 active+clean; 179 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 174 op/s 2024-03-28T00:31:48.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:31:48 smithi165 ceph-mon[36302]: pgmap v165: 97 pgs: 97 active+clean; 179 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 170 op/s 2024-03-28T00:31:48.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:31:48 smithi119 ceph-mon[31518]: pgmap v165: 97 pgs: 97 active+clean; 179 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 170 op/s 2024-03-28T00:31:48.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:31:48 smithi132 ceph-mon[35941]: pgmap v165: 97 pgs: 97 active+clean; 179 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 170 op/s 2024-03-28T00:31:48.992 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:48 smithi165 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.c.service: Scheduled restart job, restart counter is at 3. 2024-03-28T00:31:48.992 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:48 smithi165 systemd[1]: Stopped Ceph mds.c for 10f73b92-ec98-11ee-95d0-87774f69a715. 2024-03-28T00:31:48.992 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:48 smithi165 systemd[1]: Starting Ceph mds.c for 10f73b92-ec98-11ee-95d0-87774f69a715... 2024-03-28T00:31:48.992 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:48 smithi165 podman[72407]: 2024-03-28 00:31:48.754506029 +0000 UTC m=+0.021974577 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-28T00:31:48.992 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:48 smithi165 podman[72407]: 2024-03-28 00:31:48.985931863 +0000 UTC m=+0.253400397 container create de384b7ccc8d47e7b1907bf6fdb4e88f7996ce19e634b4cea65ea1688af60cb6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True) 2024-03-28T00:31:49.260 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:49 smithi165 podman[72407]: 2024-03-28 00:31:49.229960703 +0000 UTC m=+0.497429254 container init de384b7ccc8d47e7b1907bf6fdb4e88f7996ce19e634b4cea65ea1688af60cb6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_CLEAN=True, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-03-28T00:31:49.260 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:49 smithi165 podman[72407]: 2024-03-28 00:31:49.232465124 +0000 UTC m=+0.499933673 container start de384b7ccc8d47e7b1907bf6fdb4e88f7996ce19e634b4cea65ea1688af60cb6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=) 2024-03-28T00:31:49.593 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:49 smithi119 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.j.service: Scheduled restart job, restart counter is at 1. 2024-03-28T00:31:49.593 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:49 smithi119 systemd[1]: Stopped Ceph mds.j for 10f73b92-ec98-11ee-95d0-87774f69a715. 2024-03-28T00:31:49.593 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:49 smithi119 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.j.service: Consumed 3.073s CPU time. 2024-03-28T00:31:49.593 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:49 smithi119 systemd[1]: Starting Ceph mds.j for 10f73b92-ec98-11ee-95d0-87774f69a715... 2024-03-28T00:31:49.593 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:31:49 smithi119 ceph-mon[31518]: daemon mds.c restarted 2024-03-28T00:31:49.593 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:31:49 smithi119 ceph-mon[31518]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-28T00:31:49.593 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:31:49 smithi119 ceph-mon[31518]: osdmap e94: 12 total, 12 up, 12 in 2024-03-28T00:31:49.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:31:49 smithi165 ceph-mon[36302]: daemon mds.c restarted 2024-03-28T00:31:49.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:31:49 smithi165 ceph-mon[36302]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-28T00:31:49.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:31:49 smithi165 ceph-mon[36302]: osdmap e94: 12 total, 12 up, 12 in 2024-03-28T00:31:49.634 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:49 smithi165 ceph-mds[72426]: -- 172.21.15.165:0/631362014 <== mon.1 v2:172.21.15.165:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x559296f81a40 con 0x559296f54c00 2024-03-28T00:31:49.634 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:49 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: starting mds.c at 2024-03-28T00:31:49.634 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:49 smithi165 bash[72407]: de384b7ccc8d47e7b1907bf6fdb4e88f7996ce19e634b4cea65ea1688af60cb6 2024-03-28T00:31:49.634 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:31:49 smithi165 systemd[1]: Started Ceph mds.c for 10f73b92-ec98-11ee-95d0-87774f69a715. 2024-03-28T00:31:49.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:31:49 smithi132 ceph-mon[35941]: daemon mds.c restarted 2024-03-28T00:31:49.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:31:49 smithi132 ceph-mon[35941]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-28T00:31:49.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:31:49 smithi132 ceph-mon[35941]: osdmap e94: 12 total, 12 up, 12 in 2024-03-28T00:31:49.906 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:49 smithi119 podman[108726]: 2024-03-28 00:31:49.725663884 +0000 UTC m=+0.067414310 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-28T00:31:50.299 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:49 smithi119 podman[108726]: 2024-03-28 00:31:49.969140556 +0000 UTC m=+0.310891011 container create 002ee9aafce575c635a79fb90cb329d0aa3ce302d8a921962d65e38bb75838d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True) 2024-03-28T00:31:50.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:31:50 smithi165 ceph-mon[36302]: pgmap v166: 97 pgs: 97 active+clean; 179 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 175 op/s 2024-03-28T00:31:50.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:31:50 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-28T00:31:50.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:31:50 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.165:6836/3170852275,v1:172.21.15.165:6837/3170852275] up:boot 2024-03-28T00:31:50.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:31:50 smithi165 ceph-mon[36302]: Standby daemon mds.l assigned to filesystem cephfs as rank 3 2024-03-28T00:31:50.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:31:50 smithi165 ceph-mon[36302]: fsmap cephfs:4/5 {0=b=up:active,1=j=up:active,2=h=up:active,4=f=up:active} 5 up:standby-replay 3 up:standby, 1 failed 2024-03-28T00:31:50.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:31:50 smithi165 ceph-mon[36302]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:replay 4 up:active 2024-03-28T00:31:50.637 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:50 smithi119 podman[108726]: 2024-03-28 00:31:50.454580744 +0000 UTC m=+0.796331259 container init 002ee9aafce575c635a79fb90cb329d0aa3ce302d8a921962d65e38bb75838d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD) 2024-03-28T00:31:50.637 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:50 smithi119 podman[108726]: 2024-03-28 00:31:50.464246777 +0000 UTC m=+0.805997255 container start 002ee9aafce575c635a79fb90cb329d0aa3ce302d8a921962d65e38bb75838d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-03-28T00:31:50.637 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:31:50 smithi119 ceph-mon[31518]: pgmap v166: 97 pgs: 97 active+clean; 179 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 175 op/s 2024-03-28T00:31:50.638 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:31:50 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-28T00:31:50.638 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:31:50 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.165:6836/3170852275,v1:172.21.15.165:6837/3170852275] up:boot 2024-03-28T00:31:50.638 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:31:50 smithi119 ceph-mon[31518]: Standby daemon mds.l assigned to filesystem cephfs as rank 3 2024-03-28T00:31:50.638 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:31:50 smithi119 ceph-mon[31518]: fsmap cephfs:4/5 {0=b=up:active,1=j=up:active,2=h=up:active,4=f=up:active} 5 up:standby-replay 3 up:standby, 1 failed 2024-03-28T00:31:50.638 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:31:50 smithi119 ceph-mon[31518]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:replay 4 up:active 2024-03-28T00:31:50.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:31:50 smithi132 ceph-mon[35941]: pgmap v166: 97 pgs: 97 active+clean; 179 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 175 op/s 2024-03-28T00:31:50.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:31:50 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-28T00:31:50.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:31:50 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.165:6836/3170852275,v1:172.21.15.165:6837/3170852275] up:boot 2024-03-28T00:31:50.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:31:50 smithi132 ceph-mon[35941]: Standby daemon mds.l assigned to filesystem cephfs as rank 3 2024-03-28T00:31:50.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:31:50 smithi132 ceph-mon[35941]: fsmap cephfs:4/5 {0=b=up:active,1=j=up:active,2=h=up:active,4=f=up:active} 5 up:standby-replay 3 up:standby, 1 failed 2024-03-28T00:31:50.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:31:50 smithi132 ceph-mon[35941]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:replay 4 up:active 2024-03-28T00:31:50.906 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:50 smithi119 bash[108726]: 002ee9aafce575c635a79fb90cb329d0aa3ce302d8a921962d65e38bb75838d6 2024-03-28T00:31:50.907 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:50 smithi119 systemd[1]: Started Ceph mds.j for 10f73b92-ec98-11ee-95d0-87774f69a715. 2024-03-28T00:31:51.406 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:51 smithi119 ceph-mds[108744]: -- 172.21.15.119:0/4140092031 <== mon.1 v2:172.21.15.165:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55cc6e82da40 con 0x55cc6e800c00 2024-03-28T00:31:51.407 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:31:51 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: starting mds.j at 2024-03-28T00:31:51.407 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:31:51 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.165:6832/2013665520,v1:172.21.15.165:6833/2013665520] up:resolve 2024-03-28T00:31:51.407 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:31:51 smithi119 ceph-mon[31518]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:resolve 4 up:active 2024-03-28T00:31:51.407 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:31:51 smithi119 ceph-mon[31518]: daemon mds.j restarted 2024-03-28T00:31:51.407 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:31:51 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-28T00:31:51.407 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:31:51 smithi119 ceph-mon[31518]: osdmap e95: 12 total, 12 up, 12 in 2024-03-28T00:31:51.408 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:31:51 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.119:6834/2101147835,v1:172.21.15.119:6835/2101147835] up:boot 2024-03-28T00:31:51.408 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:31:51 smithi119 ceph-mon[31518]: Standby daemon mds.i assigned to filesystem cephfs as rank 1 2024-03-28T00:31:51.408 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:31:51 smithi119 ceph-mon[31518]: fsmap cephfs:4/5 {0=b=up:active,2=h=up:active,3=l=up:resolve,4=f=up:active} 4 up:standby-replay 4 up:standby, 1 failed 2024-03-28T00:31:51.408 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:31:51 smithi119 ceph-mon[31518]: fsmap cephfs:5/5 3 up:standby-replay 4 up:standby 1 up:replay 1 up:resolve 3 up:active 2024-03-28T00:31:51.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:31:51 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.165:6832/2013665520,v1:172.21.15.165:6833/2013665520] up:resolve 2024-03-28T00:31:51.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:31:51 smithi165 ceph-mon[36302]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:resolve 4 up:active 2024-03-28T00:31:51.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:31:51 smithi165 ceph-mon[36302]: daemon mds.j restarted 2024-03-28T00:31:51.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:31:51 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-28T00:31:51.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:31:51 smithi165 ceph-mon[36302]: osdmap e95: 12 total, 12 up, 12 in 2024-03-28T00:31:51.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:31:51 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.119:6834/2101147835,v1:172.21.15.119:6835/2101147835] up:boot 2024-03-28T00:31:51.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:31:51 smithi165 ceph-mon[36302]: Standby daemon mds.i assigned to filesystem cephfs as rank 1 2024-03-28T00:31:51.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:31:51 smithi165 ceph-mon[36302]: fsmap cephfs:4/5 {0=b=up:active,2=h=up:active,3=l=up:resolve,4=f=up:active} 4 up:standby-replay 4 up:standby, 1 failed 2024-03-28T00:31:51.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:31:51 smithi165 ceph-mon[36302]: fsmap cephfs:5/5 3 up:standby-replay 4 up:standby 1 up:replay 1 up:resolve 3 up:active 2024-03-28T00:31:51.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:31:51 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.165:6832/2013665520,v1:172.21.15.165:6833/2013665520] up:resolve 2024-03-28T00:31:51.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:31:51 smithi132 ceph-mon[35941]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:resolve 4 up:active 2024-03-28T00:31:51.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:31:51 smithi132 ceph-mon[35941]: daemon mds.j restarted 2024-03-28T00:31:51.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:31:51 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-28T00:31:51.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:31:51 smithi132 ceph-mon[35941]: osdmap e95: 12 total, 12 up, 12 in 2024-03-28T00:31:51.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:31:51 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.119:6834/2101147835,v1:172.21.15.119:6835/2101147835] up:boot 2024-03-28T00:31:51.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:31:51 smithi132 ceph-mon[35941]: Standby daemon mds.i assigned to filesystem cephfs as rank 1 2024-03-28T00:31:51.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:31:51 smithi132 ceph-mon[35941]: fsmap cephfs:4/5 {0=b=up:active,2=h=up:active,3=l=up:resolve,4=f=up:active} 4 up:standby-replay 4 up:standby, 1 failed 2024-03-28T00:31:51.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:31:51 smithi132 ceph-mon[35941]: fsmap cephfs:5/5 3 up:standby-replay 4 up:standby 1 up:replay 1 up:resolve 3 up:active 2024-03-28T00:31:52.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:31:52 smithi165 ceph-mon[36302]: pgmap v169: 97 pgs: 97 active+clean; 179 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 391 KiB/s rd, 423 KiB/s wr, 104 op/s 2024-03-28T00:31:52.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:31:52 smithi119 ceph-mon[31518]: pgmap v169: 97 pgs: 97 active+clean; 179 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 391 KiB/s rd, 423 KiB/s wr, 104 op/s 2024-03-28T00:31:52.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:31:52 smithi132 ceph-mon[35941]: pgmap v169: 97 pgs: 97 active+clean; 179 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 391 KiB/s rd, 423 KiB/s wr, 104 op/s 2024-03-28T00:31:53.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:31:53 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.165:6838/384483120,v1:172.21.15.165:6839/384483120] up:resolve 2024-03-28T00:31:53.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:31:53 smithi165 ceph-mon[36302]: fsmap cephfs:5/5 3 up:standby-replay 4 up:standby 2 up:resolve 3 up:active 2024-03-28T00:31:53.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:31:53 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.165:6838/384483120,v1:172.21.15.165:6839/384483120] up:resolve 2024-03-28T00:31:53.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:31:53 smithi119 ceph-mon[31518]: fsmap cephfs:5/5 3 up:standby-replay 4 up:standby 2 up:resolve 3 up:active 2024-03-28T00:31:53.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:31:53 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.165:6838/384483120,v1:172.21.15.165:6839/384483120] up:resolve 2024-03-28T00:31:53.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:31:53 smithi132 ceph-mon[35941]: fsmap cephfs:5/5 3 up:standby-replay 4 up:standby 2 up:resolve 3 up:active 2024-03-28T00:31:54.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-mon[36302]: pgmap v170: 97 pgs: 97 active+clean; 179 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 5.4 KiB/s wr, 7 op/s 2024-03-28T00:31:54.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-mon[36302]: reconnect by client.25355 172.21.15.119:0/3071152482 after 0.00199997 2024-03-28T00:31:54.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-mon[36302]: reconnect by client.25355 172.21.15.119:0/3071152482 after 0.00299996 2024-03-28T00:31:54.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.165:6838/384483120,v1:172.21.15.165:6839/384483120] up:reconnect 2024-03-28T00:31:54.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.165:6832/2013665520,v1:172.21.15.165:6833/2013665520] up:reconnect 2024-03-28T00:31:54.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-mon[36302]: fsmap cephfs:5/5 3 up:standby-replay 4 up:standby 2 up:reconnect 3 up:active 2024-03-28T00:31:54.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-mon[36302]: reconnect by client.25264 v1:172.21.15.119:0/3863511511 after 0.0179997 2024-03-28T00:31:54.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-mon[36302]: reconnect by client.25264 v1:172.21.15.119:0/3863511511 after 0.0179997 2024-03-28T00:31:54.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:31:54 smithi119 ceph-mon[31518]: pgmap v170: 97 pgs: 97 active+clean; 179 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 5.4 KiB/s wr, 7 op/s 2024-03-28T00:31:54.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:31:54 smithi119 ceph-mon[31518]: reconnect by client.25355 172.21.15.119:0/3071152482 after 0.00199997 2024-03-28T00:31:54.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:31:54 smithi119 ceph-mon[31518]: reconnect by client.25355 172.21.15.119:0/3071152482 after 0.00299996 2024-03-28T00:31:54.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:31:54 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.165:6838/384483120,v1:172.21.15.165:6839/384483120] up:reconnect 2024-03-28T00:31:54.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:31:54 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.165:6832/2013665520,v1:172.21.15.165:6833/2013665520] up:reconnect 2024-03-28T00:31:54.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:31:54 smithi119 ceph-mon[31518]: fsmap cephfs:5/5 3 up:standby-replay 4 up:standby 2 up:reconnect 3 up:active 2024-03-28T00:31:54.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:31:54 smithi119 ceph-mon[31518]: reconnect by client.25264 v1:172.21.15.119:0/3863511511 after 0.0179997 2024-03-28T00:31:54.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:31:54 smithi119 ceph-mon[31518]: reconnect by client.25264 v1:172.21.15.119:0/3863511511 after 0.0179997 2024-03-28T00:31:54.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:31:54 smithi132 ceph-mon[35941]: pgmap v170: 97 pgs: 97 active+clean; 179 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 5.4 KiB/s wr, 7 op/s 2024-03-28T00:31:54.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:31:54 smithi132 ceph-mon[35941]: reconnect by client.25355 172.21.15.119:0/3071152482 after 0.00199997 2024-03-28T00:31:54.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:31:54 smithi132 ceph-mon[35941]: reconnect by client.25355 172.21.15.119:0/3071152482 after 0.00299996 2024-03-28T00:31:54.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:31:54 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.165:6838/384483120,v1:172.21.15.165:6839/384483120] up:reconnect 2024-03-28T00:31:54.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:31:54 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.165:6832/2013665520,v1:172.21.15.165:6833/2013665520] up:reconnect 2024-03-28T00:31:54.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:31:54 smithi132 ceph-mon[35941]: fsmap cephfs:5/5 3 up:standby-replay 4 up:standby 2 up:reconnect 3 up:active 2024-03-28T00:31:54.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:31:54 smithi132 ceph-mon[35941]: reconnect by client.25264 v1:172.21.15.119:0/3863511511 after 0.0179997 2024-03-28T00:31:54.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:31:54 smithi132 ceph-mon[35941]: reconnect by client.25264 v1:172.21.15.119:0/3863511511 after 0.0179997 2024-03-28T00:31:55.134 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7ff0f4d15700 time 2024-03-28T00:31:54.764425+0000 2024-03-28T00:31:55.134 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T00:31:55.134 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:31:55.134 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7ff0ff33f4a1] 2024-03-28T00:31:55.134 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7ff0ff33f667] 2024-03-28T00:31:55.135 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55b96bf85223] 2024-03-28T00:31:55.135 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55b96bdff5d6] 2024-03-28T00:31:55.135 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55b96bdffda1] 2024-03-28T00:31:55.135 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: 6: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x55b96be3409c] 2024-03-28T00:31:55.135 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55b96be52233] 2024-03-28T00:31:55.135 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55b96be54197] 2024-03-28T00:31:55.135 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55b96bd5311b] 2024-03-28T00:31:55.135 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55b96bd536bc] 2024-03-28T00:31:55.135 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55b96bd3d1ff] 2024-03-28T00:31:55.135 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7ff0ff5d1488] 2024-03-28T00:31:55.135 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: 13: (DispatchQueue::entry()+0x51f) [0x7ff0ff5ce60f] 2024-03-28T00:31:55.135 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7ff0ff6a1e81] 2024-03-28T00:31:55.135 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7ff0fe0bc1ca] 2024-03-28T00:31:55.135 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: 16: clone() 2024-03-28T00:31:55.136 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7fde612c1700 time 2024-03-28T00:31:54.775376+0000 2024-03-28T00:31:55.136 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T00:31:55.136 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:31:55.136 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fde6b8eb4a1] 2024-03-28T00:31:55.136 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7fde6b8eb667] 2024-03-28T00:31:55.136 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x556fef25c223] 2024-03-28T00:31:55.136 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x556fef0d65d6] 2024-03-28T00:31:55.136 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x556fef0d6da1] 2024-03-28T00:31:55.136 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: 6: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x556fef10b09c] 2024-03-28T00:31:55.136 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x556fef129233] 2024-03-28T00:31:55.137 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x556fef12b197] 2024-03-28T00:31:55.137 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x556fef02a11b] 2024-03-28T00:31:55.137 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x556fef02a6bc] 2024-03-28T00:31:55.137 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x556fef0141ff] 2024-03-28T00:31:55.137 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7fde6bb7d488] 2024-03-28T00:31:55.137 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: 13: (DispatchQueue::entry()+0x51f) [0x7fde6bb7a60f] 2024-03-28T00:31:55.137 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fde6bc4de81] 2024-03-28T00:31:55.137 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fde6a6681ca] 2024-03-28T00:31:55.137 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: 16: clone() 2024-03-28T00:31:55.137 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: *** Caught signal (Aborted) ** 2024-03-28T00:31:55.137 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: in thread 7fde612c1700 thread_name:ms_dispatch 2024-03-28T00:31:55.138 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: 2024-03-28T00:31:54.765+0000 7ff0f4d15700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7ff0f4d15700 time 2024-03-28T00:31:54.764425+0000 2024-03-28T00:31:55.138 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T00:31:55.138 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: 2024-03-28T00:31:55.138 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:31:55.138 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7ff0ff33f4a1] 2024-03-28T00:31:55.138 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7ff0ff33f667] 2024-03-28T00:31:55.138 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55b96bf85223] 2024-03-28T00:31:55.138 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55b96bdff5d6] 2024-03-28T00:31:55.138 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55b96bdffda1] 2024-03-28T00:31:55.139 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: 6: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x55b96be3409c] 2024-03-28T00:31:55.139 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55b96be52233] 2024-03-28T00:31:55.139 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55b96be54197] 2024-03-28T00:31:55.139 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55b96bd5311b] 2024-03-28T00:31:55.139 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55b96bd536bc] 2024-03-28T00:31:55.139 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55b96bd3d1ff] 2024-03-28T00:31:55.139 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7ff0ff5d1488] 2024-03-28T00:31:55.139 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: 13: (DispatchQueue::entry()+0x51f) [0x7ff0ff5ce60f] 2024-03-28T00:31:55.139 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7ff0ff6a1e81] 2024-03-28T00:31:55.139 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7ff0fe0bc1ca] 2024-03-28T00:31:55.139 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: 16: clone() 2024-03-28T00:31:55.139 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: 2024-03-28T00:31:55.139 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: *** Caught signal (Aborted) ** 2024-03-28T00:31:55.139 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: in thread 7ff0f4d15700 thread_name:ms_dispatch 2024-03-28T00:31:55.140 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:31:55.140 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7ff0fe0c6d20] 2024-03-28T00:31:55.140 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: 2024-03-28T00:31:54.776+0000 7fde612c1700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7fde612c1700 time 2024-03-28T00:31:54.775376+0000 2024-03-28T00:31:55.140 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T00:31:55.140 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: 2024-03-28T00:31:55.140 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:31:55.140 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fde6b8eb4a1] 2024-03-28T00:31:55.140 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7fde6b8eb667] 2024-03-28T00:31:55.140 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x556fef25c223] 2024-03-28T00:31:55.140 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x556fef0d65d6] 2024-03-28T00:31:55.141 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x556fef0d6da1] 2024-03-28T00:31:55.141 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: 6: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x556fef10b09c] 2024-03-28T00:31:55.141 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x556fef129233] 2024-03-28T00:31:55.141 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x556fef12b197] 2024-03-28T00:31:55.141 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x556fef02a11b] 2024-03-28T00:31:55.141 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x556fef02a6bc] 2024-03-28T00:31:55.141 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x556fef0141ff] 2024-03-28T00:31:55.141 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7fde6bb7d488] 2024-03-28T00:31:55.141 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: 13: (DispatchQueue::entry()+0x51f) [0x7fde6bb7a60f] 2024-03-28T00:31:55.141 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fde6bc4de81] 2024-03-28T00:31:55.141 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fde6a6681ca] 2024-03-28T00:31:55.141 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: 16: clone() 2024-03-28T00:31:55.141 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: 2024-03-28T00:31:55.142 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:31:55.142 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fde6a672d20] 2024-03-28T00:31:55.142 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: 2: gsignal() 2024-03-28T00:31:55.142 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: 3: abort() 2024-03-28T00:31:55.142 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7ff0ff33f4fb] 2024-03-28T00:31:55.142 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7ff0ff33f667] 2024-03-28T00:31:55.142 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55b96bf85223] 2024-03-28T00:31:55.142 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55b96bdff5d6] 2024-03-28T00:31:55.142 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55b96bdffda1] 2024-03-28T00:31:55.143 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: 9: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x55b96be3409c] 2024-03-28T00:31:55.143 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55b96be52233] 2024-03-28T00:31:55.143 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55b96be54197] 2024-03-28T00:31:55.143 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55b96bd5311b] 2024-03-28T00:31:55.143 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55b96bd536bc] 2024-03-28T00:31:55.143 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55b96bd3d1ff] 2024-03-28T00:31:55.143 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7ff0ff5d1488] 2024-03-28T00:31:55.143 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: 16: (DispatchQueue::entry()+0x51f) [0x7ff0ff5ce60f] 2024-03-28T00:31:55.143 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7ff0ff6a1e81] 2024-03-28T00:31:55.143 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7ff0fe0bc1ca] 2024-03-28T00:31:55.143 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: 19: clone() 2024-03-28T00:31:55.143 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: 2024-03-28T00:31:54.767+0000 7ff0f4d15700 -1 *** Caught signal (Aborted) ** 2024-03-28T00:31:55.143 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: in thread 7ff0f4d15700 thread_name:ms_dispatch 2024-03-28T00:31:55.143 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: 2024-03-28T00:31:55.144 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:31:55.144 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7ff0fe0c6d20] 2024-03-28T00:31:55.144 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: 2: gsignal() 2024-03-28T00:31:55.144 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: 3: abort() 2024-03-28T00:31:55.144 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7ff0ff33f4fb] 2024-03-28T00:31:55.144 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7ff0ff33f667] 2024-03-28T00:31:55.144 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: 2: gsignal() 2024-03-28T00:31:55.144 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: 3: abort() 2024-03-28T00:31:55.145 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fde6b8eb4fb] 2024-03-28T00:31:55.145 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7fde6b8eb667] 2024-03-28T00:31:55.145 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x556fef25c223] 2024-03-28T00:31:55.145 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x556fef0d65d6] 2024-03-28T00:31:55.145 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x556fef0d6da1] 2024-03-28T00:31:55.145 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: 9: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x556fef10b09c] 2024-03-28T00:31:55.145 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x556fef129233] 2024-03-28T00:31:55.145 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x556fef12b197] 2024-03-28T00:31:55.145 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x556fef02a11b] 2024-03-28T00:31:55.145 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x556fef02a6bc] 2024-03-28T00:31:55.145 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x556fef0141ff] 2024-03-28T00:31:55.145 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7fde6bb7d488] 2024-03-28T00:31:55.145 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: 16: (DispatchQueue::entry()+0x51f) [0x7fde6bb7a60f] 2024-03-28T00:31:55.146 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fde6bc4de81] 2024-03-28T00:31:55.146 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7fde6a6681ca] 2024-03-28T00:31:55.146 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: 19: clone() 2024-03-28T00:31:55.146 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: 2024-03-28T00:31:54.778+0000 7fde612c1700 -1 *** Caught signal (Aborted) ** 2024-03-28T00:31:55.146 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: in thread 7fde612c1700 thread_name:ms_dispatch 2024-03-28T00:31:55.146 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: 2024-03-28T00:31:55.146 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:31:55.146 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fde6a672d20] 2024-03-28T00:31:55.146 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: 2: gsignal() 2024-03-28T00:31:55.146 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: 3: abort() 2024-03-28T00:31:55.146 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fde6b8eb4fb] 2024-03-28T00:31:55.146 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7fde6b8eb667] 2024-03-28T00:31:55.147 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55b96bf85223] 2024-03-28T00:31:55.147 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55b96bdff5d6] 2024-03-28T00:31:55.147 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55b96bdffda1] 2024-03-28T00:31:55.147 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: 9: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x55b96be3409c] 2024-03-28T00:31:55.147 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55b96be52233] 2024-03-28T00:31:55.147 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55b96be54197] 2024-03-28T00:31:55.147 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55b96bd5311b] 2024-03-28T00:31:55.147 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55b96bd536bc] 2024-03-28T00:31:55.147 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55b96bd3d1ff] 2024-03-28T00:31:55.147 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7ff0ff5d1488] 2024-03-28T00:31:55.148 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: 16: (DispatchQueue::entry()+0x51f) [0x7ff0ff5ce60f] 2024-03-28T00:31:55.148 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7ff0ff6a1e81] 2024-03-28T00:31:55.148 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7ff0fe0bc1ca] 2024-03-28T00:31:55.148 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: 19: clone() 2024-03-28T00:31:55.148 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T00:31:55.148 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: 2024-03-28T00:31:55.148 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: -1> 2024-03-28T00:31:54.765+0000 7ff0f4d15700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7ff0f4d15700 time 2024-03-28T00:31:54.764425+0000 2024-03-28T00:31:55.148 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T00:31:55.148 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: 2024-03-28T00:31:55.148 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:31:55.148 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7ff0ff33f4a1] 2024-03-28T00:31:55.148 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7ff0ff33f667] 2024-03-28T00:31:55.149 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x556fef25c223] 2024-03-28T00:31:55.149 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x556fef0d65d6] 2024-03-28T00:31:55.149 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x556fef0d6da1] 2024-03-28T00:31:55.149 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: 9: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x556fef10b09c] 2024-03-28T00:31:55.149 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x556fef129233] 2024-03-28T00:31:55.149 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x556fef12b197] 2024-03-28T00:31:55.149 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x556fef02a11b] 2024-03-28T00:31:55.149 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x556fef02a6bc] 2024-03-28T00:31:55.149 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x556fef0141ff] 2024-03-28T00:31:55.150 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7fde6bb7d488] 2024-03-28T00:31:55.150 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: 16: (DispatchQueue::entry()+0x51f) [0x7fde6bb7a60f] 2024-03-28T00:31:55.150 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fde6bc4de81] 2024-03-28T00:31:55.150 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7fde6a6681ca] 2024-03-28T00:31:55.150 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: 19: clone() 2024-03-28T00:31:55.150 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T00:31:55.150 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: 2024-03-28T00:31:55.150 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: -1> 2024-03-28T00:31:54.776+0000 7fde612c1700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7fde612c1700 time 2024-03-28T00:31:54.775376+0000 2024-03-28T00:31:55.150 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T00:31:55.150 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: 2024-03-28T00:31:55.150 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:31:55.150 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fde6b8eb4a1] 2024-03-28T00:31:55.150 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7fde6b8eb667] 2024-03-28T00:31:55.151 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55b96bf85223] 2024-03-28T00:31:55.151 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55b96bdff5d6] 2024-03-28T00:31:55.151 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55b96bdffda1] 2024-03-28T00:31:55.151 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: 6: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x55b96be3409c] 2024-03-28T00:31:55.151 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55b96be52233] 2024-03-28T00:31:55.151 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55b96be54197] 2024-03-28T00:31:55.151 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55b96bd5311b] 2024-03-28T00:31:55.151 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55b96bd536bc] 2024-03-28T00:31:55.151 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55b96bd3d1ff] 2024-03-28T00:31:55.151 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7ff0ff5d1488] 2024-03-28T00:31:55.151 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: 13: (DispatchQueue::entry()+0x51f) [0x7ff0ff5ce60f] 2024-03-28T00:31:55.152 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7ff0ff6a1e81] 2024-03-28T00:31:55.152 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7ff0fe0bc1ca] 2024-03-28T00:31:55.152 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: 16: clone() 2024-03-28T00:31:55.152 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: 2024-03-28T00:31:55.152 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: 0> 2024-03-28T00:31:54.767+0000 7ff0f4d15700 -1 *** Caught signal (Aborted) ** 2024-03-28T00:31:55.152 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: in thread 7ff0f4d15700 thread_name:ms_dispatch 2024-03-28T00:31:55.152 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: 2024-03-28T00:31:55.152 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:31:55.152 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7ff0fe0c6d20] 2024-03-28T00:31:55.152 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: 2: gsignal() 2024-03-28T00:31:55.152 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: 3: abort() 2024-03-28T00:31:55.152 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7ff0ff33f4fb] 2024-03-28T00:31:55.153 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7ff0ff33f667] 2024-03-28T00:31:55.153 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55b96bf85223] 2024-03-28T00:31:55.153 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55b96bdff5d6] 2024-03-28T00:31:55.153 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x556fef25c223] 2024-03-28T00:31:55.153 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x556fef0d65d6] 2024-03-28T00:31:55.153 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x556fef0d6da1] 2024-03-28T00:31:55.153 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: 6: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x556fef10b09c] 2024-03-28T00:31:55.153 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x556fef129233] 2024-03-28T00:31:55.153 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x556fef12b197] 2024-03-28T00:31:55.153 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x556fef02a11b] 2024-03-28T00:31:55.154 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x556fef02a6bc] 2024-03-28T00:31:55.154 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x556fef0141ff] 2024-03-28T00:31:55.154 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7fde6bb7d488] 2024-03-28T00:31:55.154 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: 13: (DispatchQueue::entry()+0x51f) [0x7fde6bb7a60f] 2024-03-28T00:31:55.154 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fde6bc4de81] 2024-03-28T00:31:55.154 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fde6a6681ca] 2024-03-28T00:31:55.154 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: 16: clone() 2024-03-28T00:31:55.154 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: 2024-03-28T00:31:55.154 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: 0> 2024-03-28T00:31:54.778+0000 7fde612c1700 -1 *** Caught signal (Aborted) ** 2024-03-28T00:31:55.154 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: in thread 7fde612c1700 thread_name:ms_dispatch 2024-03-28T00:31:55.154 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: 2024-03-28T00:31:55.154 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:31:55.154 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fde6a672d20] 2024-03-28T00:31:55.155 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: 2: gsignal() 2024-03-28T00:31:55.155 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: 3: abort() 2024-03-28T00:31:55.155 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fde6b8eb4fb] 2024-03-28T00:31:55.155 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7fde6b8eb667] 2024-03-28T00:31:55.155 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x556fef25c223] 2024-03-28T00:31:55.155 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x556fef0d65d6] 2024-03-28T00:31:55.155 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55b96bdffda1] 2024-03-28T00:31:55.155 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: 9: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x55b96be3409c] 2024-03-28T00:31:55.156 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55b96be52233] 2024-03-28T00:31:55.156 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55b96be54197] 2024-03-28T00:31:55.156 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55b96bd5311b] 2024-03-28T00:31:55.156 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55b96bd536bc] 2024-03-28T00:31:55.156 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55b96bd3d1ff] 2024-03-28T00:31:55.156 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7ff0ff5d1488] 2024-03-28T00:31:55.156 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: 16: (DispatchQueue::entry()+0x51f) [0x7ff0ff5ce60f] 2024-03-28T00:31:55.156 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7ff0ff6a1e81] 2024-03-28T00:31:55.156 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7ff0fe0bc1ca] 2024-03-28T00:31:55.156 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: 19: clone() 2024-03-28T00:31:55.156 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T00:31:55.156 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: 2024-03-28T00:31:55.156 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: -1> 2024-03-28T00:31:54.765+0000 7ff0f4d15700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7ff0f4d15700 time 2024-03-28T00:31:54.764425+0000 2024-03-28T00:31:55.156 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T00:31:55.157 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: 2024-03-28T00:31:55.157 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:31:55.157 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7ff0ff33f4a1] 2024-03-28T00:31:55.157 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7ff0ff33f667] 2024-03-28T00:31:55.157 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55b96bf85223] 2024-03-28T00:31:55.157 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55b96bdff5d6] 2024-03-28T00:31:55.157 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55b96bdffda1] 2024-03-28T00:31:55.157 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x556fef0d6da1] 2024-03-28T00:31:55.157 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: 9: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x556fef10b09c] 2024-03-28T00:31:55.158 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x556fef129233] 2024-03-28T00:31:55.158 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x556fef12b197] 2024-03-28T00:31:55.158 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x556fef02a11b] 2024-03-28T00:31:55.158 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x556fef02a6bc] 2024-03-28T00:31:55.158 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x556fef0141ff] 2024-03-28T00:31:55.158 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7fde6bb7d488] 2024-03-28T00:31:55.158 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: 16: (DispatchQueue::entry()+0x51f) [0x7fde6bb7a60f] 2024-03-28T00:31:55.158 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fde6bc4de81] 2024-03-28T00:31:55.158 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7fde6a6681ca] 2024-03-28T00:31:55.158 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: 19: clone() 2024-03-28T00:31:55.158 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T00:31:55.158 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: 2024-03-28T00:31:55.158 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: -1> 2024-03-28T00:31:54.776+0000 7fde612c1700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7fde612c1700 time 2024-03-28T00:31:54.775376+0000 2024-03-28T00:31:55.159 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T00:31:55.159 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: 2024-03-28T00:31:55.159 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:31:55.159 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fde6b8eb4a1] 2024-03-28T00:31:55.159 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7fde6b8eb667] 2024-03-28T00:31:55.159 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x556fef25c223] 2024-03-28T00:31:55.159 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x556fef0d65d6] 2024-03-28T00:31:55.159 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x556fef0d6da1] 2024-03-28T00:31:55.159 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: 6: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x55b96be3409c] 2024-03-28T00:31:55.160 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55b96be52233] 2024-03-28T00:31:55.160 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55b96be54197] 2024-03-28T00:31:55.160 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55b96bd5311b] 2024-03-28T00:31:55.160 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55b96bd536bc] 2024-03-28T00:31:55.160 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55b96bd3d1ff] 2024-03-28T00:31:55.160 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7ff0ff5d1488] 2024-03-28T00:31:55.160 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: 13: (DispatchQueue::entry()+0x51f) [0x7ff0ff5ce60f] 2024-03-28T00:31:55.160 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7ff0ff6a1e81] 2024-03-28T00:31:55.160 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7ff0fe0bc1ca] 2024-03-28T00:31:55.160 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: 16: clone() 2024-03-28T00:31:55.160 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: 2024-03-28T00:31:55.160 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: 0> 2024-03-28T00:31:54.767+0000 7ff0f4d15700 -1 *** Caught signal (Aborted) ** 2024-03-28T00:31:55.160 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: in thread 7ff0f4d15700 thread_name:ms_dispatch 2024-03-28T00:31:55.161 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: 2024-03-28T00:31:55.161 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:31:55.161 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7ff0fe0c6d20] 2024-03-28T00:31:55.161 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: 2: gsignal() 2024-03-28T00:31:55.161 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: 3: abort() 2024-03-28T00:31:55.161 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7ff0ff33f4fb] 2024-03-28T00:31:55.161 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7ff0ff33f667] 2024-03-28T00:31:55.161 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55b96bf85223] 2024-03-28T00:31:55.161 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55b96bdff5d6] 2024-03-28T00:31:55.161 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55b96bdffda1] 2024-03-28T00:31:55.161 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: 9: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x55b96be3409c] 2024-03-28T00:31:55.161 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55b96be52233] 2024-03-28T00:31:55.162 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: 6: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x556fef10b09c] 2024-03-28T00:31:55.162 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x556fef129233] 2024-03-28T00:31:55.162 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x556fef12b197] 2024-03-28T00:31:55.162 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x556fef02a11b] 2024-03-28T00:31:55.162 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x556fef02a6bc] 2024-03-28T00:31:55.162 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x556fef0141ff] 2024-03-28T00:31:55.162 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7fde6bb7d488] 2024-03-28T00:31:55.162 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: 13: (DispatchQueue::entry()+0x51f) [0x7fde6bb7a60f] 2024-03-28T00:31:55.162 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fde6bc4de81] 2024-03-28T00:31:55.162 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fde6a6681ca] 2024-03-28T00:31:55.163 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: 16: clone() 2024-03-28T00:31:55.163 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: 2024-03-28T00:31:55.163 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: 0> 2024-03-28T00:31:54.778+0000 7fde612c1700 -1 *** Caught signal (Aborted) ** 2024-03-28T00:31:55.163 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: in thread 7fde612c1700 thread_name:ms_dispatch 2024-03-28T00:31:55.163 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: 2024-03-28T00:31:55.163 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:31:55.163 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fde6a672d20] 2024-03-28T00:31:55.163 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: 2: gsignal() 2024-03-28T00:31:55.163 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: 3: abort() 2024-03-28T00:31:55.163 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fde6b8eb4fb] 2024-03-28T00:31:55.163 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7fde6b8eb667] 2024-03-28T00:31:55.163 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x556fef25c223] 2024-03-28T00:31:55.163 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x556fef0d65d6] 2024-03-28T00:31:55.164 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x556fef0d6da1] 2024-03-28T00:31:55.164 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: 9: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x556fef10b09c] 2024-03-28T00:31:55.164 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x556fef129233] 2024-03-28T00:31:55.164 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55b96be54197] 2024-03-28T00:31:55.164 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55b96bd5311b] 2024-03-28T00:31:55.164 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55b96bd536bc] 2024-03-28T00:31:55.164 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55b96bd3d1ff] 2024-03-28T00:31:55.165 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7ff0ff5d1488] 2024-03-28T00:31:55.165 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: 16: (DispatchQueue::entry()+0x51f) [0x7ff0ff5ce60f] 2024-03-28T00:31:55.165 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7ff0ff6a1e81] 2024-03-28T00:31:55.165 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7ff0fe0bc1ca] 2024-03-28T00:31:55.165 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: 19: clone() 2024-03-28T00:31:55.165 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T00:31:55.165 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72117]: 2024-03-28T00:31:55.165 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:54 smithi165 podman[72463]: 2024-03-28 00:31:54.846790106 +0000 UTC m=+0.025734031 container died 13b6de6cfef851c38b8dfc107dc9bb86ec1f898dfe1fabaaf0277a3207e74066 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l, ceph=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-03-28T00:31:55.165 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x556fef12b197] 2024-03-28T00:31:55.165 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x556fef02a11b] 2024-03-28T00:31:55.165 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x556fef02a6bc] 2024-03-28T00:31:55.165 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x556fef0141ff] 2024-03-28T00:31:55.166 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7fde6bb7d488] 2024-03-28T00:31:55.166 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: 16: (DispatchQueue::entry()+0x51f) [0x7fde6bb7a60f] 2024-03-28T00:31:55.166 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fde6bc4de81] 2024-03-28T00:31:55.166 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7fde6a6681ca] 2024-03-28T00:31:55.166 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: 19: clone() 2024-03-28T00:31:55.166 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T00:31:55.166 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72265]: 2024-03-28T00:31:55.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:31:55 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.165:6832/2013665520,v1:172.21.15.165:6833/2013665520] up:rejoin 2024-03-28T00:31:55.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:31:55 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.165:6838/384483120,v1:172.21.15.165:6839/384483120] up:rejoin 2024-03-28T00:31:55.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:31:55 smithi165 ceph-mon[36302]: fsmap cephfs:5/5 3 up:standby-replay 4 up:standby 2 up:rejoin 3 up:active 2024-03-28T00:31:55.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:31:55 smithi165 ceph-mon[36302]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-03-28T00:31:55.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:31:55 smithi165 ceph-mon[36302]: daemon mds.l is now active in filesystem cephfs as rank 3 2024-03-28T00:31:55.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:31:55 smithi165 ceph-mon[36302]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-28T00:31:55.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:31:55 smithi165 ceph-mon[36302]: Cluster is now healthy 2024-03-28T00:31:55.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:31:55 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.165:6838/384483120,v1:172.21.15.165:6839/384483120] up:active 2024-03-28T00:31:55.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:31:55 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.165:6832/2013665520,v1:172.21.15.165:6833/2013665520] up:active 2024-03-28T00:31:55.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:31:55 smithi165 ceph-mon[36302]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-28T00:31:55.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:31:55 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.165:6832/2013665520,v1:172.21.15.165:6833/2013665520] up:rejoin 2024-03-28T00:31:55.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:31:55 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.165:6838/384483120,v1:172.21.15.165:6839/384483120] up:rejoin 2024-03-28T00:31:55.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:31:55 smithi119 ceph-mon[31518]: fsmap cephfs:5/5 3 up:standby-replay 4 up:standby 2 up:rejoin 3 up:active 2024-03-28T00:31:55.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:31:55 smithi119 ceph-mon[31518]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-03-28T00:31:55.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:31:55 smithi119 ceph-mon[31518]: daemon mds.l is now active in filesystem cephfs as rank 3 2024-03-28T00:31:55.658 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:31:55 smithi119 ceph-mon[31518]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-28T00:31:55.658 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:31:55 smithi119 ceph-mon[31518]: Cluster is now healthy 2024-03-28T00:31:55.658 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:31:55 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.165:6838/384483120,v1:172.21.15.165:6839/384483120] up:active 2024-03-28T00:31:55.658 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:31:55 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.165:6832/2013665520,v1:172.21.15.165:6833/2013665520] up:active 2024-03-28T00:31:55.658 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:31:55 smithi119 ceph-mon[31518]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-28T00:31:55.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:31:55 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.165:6832/2013665520,v1:172.21.15.165:6833/2013665520] up:rejoin 2024-03-28T00:31:55.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:31:55 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.165:6838/384483120,v1:172.21.15.165:6839/384483120] up:rejoin 2024-03-28T00:31:55.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:31:55 smithi132 ceph-mon[35941]: fsmap cephfs:5/5 3 up:standby-replay 4 up:standby 2 up:rejoin 3 up:active 2024-03-28T00:31:55.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:31:55 smithi132 ceph-mon[35941]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-03-28T00:31:55.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:31:55 smithi132 ceph-mon[35941]: daemon mds.l is now active in filesystem cephfs as rank 3 2024-03-28T00:31:55.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:31:55 smithi132 ceph-mon[35941]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-28T00:31:55.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:31:55 smithi132 ceph-mon[35941]: Cluster is now healthy 2024-03-28T00:31:55.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:31:55 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.165:6838/384483120,v1:172.21.15.165:6839/384483120] up:active 2024-03-28T00:31:55.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:31:55 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.165:6832/2013665520,v1:172.21.15.165:6833/2013665520] up:active 2024-03-28T00:31:55.705 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:31:55 smithi132 ceph-mon[35941]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-28T00:31:55.883 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:55 smithi165 podman[72469]: 2024-03-28 00:31:55.579313237 +0000 UTC m=+0.751473908 container died 90ac655df7c6427e5ffba59ca7bd034e0b657f8eaebc3c77624ba6e001074844 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-03-28T00:31:55.884 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:55 smithi165 podman[72463]: 2024-03-28 00:31:55.570077716 +0000 UTC m=+0.749021631 container remove 13b6de6cfef851c38b8dfc107dc9bb86ec1f898dfe1fabaaf0277a3207e74066 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-03-28T00:31:55.884 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:55 smithi165 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.l.service: Main process exited, code=exited, status=134/n/a 2024-03-28T00:31:56.472 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:56 smithi165 podman[72469]: 2024-03-28 00:31:56.216117937 +0000 UTC m=+1.388278581 container remove 90ac655df7c6427e5ffba59ca7bd034e0b657f8eaebc3c77624ba6e001074844 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i, io.buildah.version=1.33.5, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True) 2024-03-28T00:31:56.473 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:56 smithi165 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.i.service: Main process exited, code=exited, status=134/n/a 2024-03-28T00:31:56.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:31:56 smithi165 ceph-mon[36302]: pgmap v171: 97 pgs: 97 active+clean; 179 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.2 KiB/s wr, 15 op/s 2024-03-28T00:31:56.884 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:31:56 smithi165 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.i.service: Failed with result 'exit-code'. 2024-03-28T00:31:56.884 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:31:56 smithi165 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.l.service: Failed with result 'exit-code'. 2024-03-28T00:31:57.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:31:56 smithi119 ceph-mon[31518]: pgmap v171: 97 pgs: 97 active+clean; 179 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.2 KiB/s wr, 15 op/s 2024-03-28T00:31:57.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:31:56 smithi132 ceph-mon[35941]: pgmap v171: 97 pgs: 97 active+clean; 179 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.2 KiB/s wr, 15 op/s 2024-03-28T00:31:59.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:31:58 smithi165 ceph-mon[36302]: pgmap v172: 97 pgs: 97 active+clean; 179 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.9 KiB/s wr, 7 op/s 2024-03-28T00:31:59.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:31:58 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:31:59.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:31:58 smithi119 ceph-mon[31518]: pgmap v172: 97 pgs: 97 active+clean; 179 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.9 KiB/s wr, 7 op/s 2024-03-28T00:31:59.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:31:58 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:31:59.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:31:58 smithi132 ceph-mon[35941]: pgmap v172: 97 pgs: 97 active+clean; 179 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.9 KiB/s wr, 7 op/s 2024-03-28T00:31:59.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:31:58 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:32:00.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:31:59 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:32:00.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:31:59 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:32:00.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:31:59 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:32:00.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:31:59 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:32:00.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:31:59 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:32:00.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:31:59 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:32:00.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:31:59 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:32:00.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:31:59 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:32:00.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:31:59 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:32:01.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:32:00 smithi165 ceph-mon[36302]: pgmap v173: 97 pgs: 97 active+clean; 181 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 361 KiB/s rd, 145 KiB/s wr, 46 op/s 2024-03-28T00:32:01.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:32:00 smithi119 ceph-mon[31518]: pgmap v173: 97 pgs: 97 active+clean; 181 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 361 KiB/s rd, 145 KiB/s wr, 46 op/s 2024-03-28T00:32:01.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:32:00 smithi132 ceph-mon[35941]: pgmap v173: 97 pgs: 97 active+clean; 181 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 361 KiB/s rd, 145 KiB/s wr, 46 op/s 2024-03-28T00:32:02.105 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:32:01 smithi132 ceph-mon[35941]: pgmap v174: 97 pgs: 97 active+clean; 181 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 355 KiB/s rd, 143 KiB/s wr, 45 op/s 2024-03-28T00:32:02.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:32:01 smithi165 ceph-mon[36302]: pgmap v174: 97 pgs: 97 active+clean; 181 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 355 KiB/s rd, 143 KiB/s wr, 45 op/s 2024-03-28T00:32:02.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:32:01 smithi119 ceph-mon[31518]: pgmap v174: 97 pgs: 97 active+clean; 181 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 355 KiB/s rd, 143 KiB/s wr, 45 op/s 2024-03-28T00:32:04.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:32:04 smithi132 ceph-mon[35941]: pgmap v175: 97 pgs: 97 active+clean; 181 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 119 KiB/s wr, 38 op/s 2024-03-28T00:32:04.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:32:04 smithi165 ceph-mon[36302]: pgmap v175: 97 pgs: 97 active+clean; 181 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 119 KiB/s wr, 38 op/s 2024-03-28T00:32:04.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:32:04 smithi119 ceph-mon[31518]: pgmap v175: 97 pgs: 97 active+clean; 181 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 119 KiB/s wr, 38 op/s 2024-03-28T00:32:06.454 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:32:06 smithi132 ceph-mon[35941]: pgmap v176: 97 pgs: 97 active+clean; 181 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 123 KiB/s wr, 44 op/s 2024-03-28T00:32:06.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:32:06 smithi165 ceph-mon[36302]: pgmap v176: 97 pgs: 97 active+clean; 181 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 123 KiB/s wr, 44 op/s 2024-03-28T00:32:06.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:32:06 smithi119 ceph-mon[31518]: pgmap v176: 97 pgs: 97 active+clean; 181 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 123 KiB/s wr, 44 op/s 2024-03-28T00:32:06.889 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:06 smithi165 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.i.service: Scheduled restart job, restart counter is at 2. 2024-03-28T00:32:06.889 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:06 smithi165 systemd[1]: Stopped Ceph mds.i for 10f73b92-ec98-11ee-95d0-87774f69a715. 2024-03-28T00:32:06.889 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:06 smithi165 systemd[1]: Starting Ceph mds.i for 10f73b92-ec98-11ee-95d0-87774f69a715... 2024-03-28T00:32:06.889 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:06 smithi165 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.l.service: Scheduled restart job, restart counter is at 3. 2024-03-28T00:32:06.889 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:06 smithi165 systemd[1]: Stopped Ceph mds.l for 10f73b92-ec98-11ee-95d0-87774f69a715. 2024-03-28T00:32:06.889 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:06 smithi165 systemd[1]: Starting Ceph mds.l for 10f73b92-ec98-11ee-95d0-87774f69a715... 2024-03-28T00:32:07.226 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:06 smithi165 podman[72773]: 2024-03-28 00:32:06.78635252 +0000 UTC m=+0.023282902 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-28T00:32:07.226 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:06 smithi165 podman[72773]: 2024-03-28 00:32:06.908482582 +0000 UTC m=+0.145412937 container create 16e6cd19bdbe3f6021e0dd2438bdb676a60653b43e40b67bf379124de1547c3b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i, org.label-schema.build-date=20240319, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-28T00:32:07.227 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:06 smithi165 podman[72772]: 2024-03-28 00:32:06.787581197 +0000 UTC m=+0.024512002 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-28T00:32:07.227 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:07 smithi165 podman[72772]: 2024-03-28 00:32:07.127268427 +0000 UTC m=+0.364199207 container create 59e8ab3b38abc37a920297b6e1fd2dcb0aa3604b5277ecc5ba031cb07f51975e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2) 2024-03-28T00:32:07.480 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:07 smithi165 podman[72773]: 2024-03-28 00:32:07.220159297 +0000 UTC m=+0.457089673 container init 16e6cd19bdbe3f6021e0dd2438bdb676a60653b43e40b67bf379124de1547c3b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-03-28T00:32:07.480 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:07 smithi165 podman[72773]: 2024-03-28 00:32:07.223104633 +0000 UTC m=+0.460034989 container start 16e6cd19bdbe3f6021e0dd2438bdb676a60653b43e40b67bf379124de1547c3b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240319) 2024-03-28T00:32:07.480 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:07 smithi165 ceph-mds[72805]: -- 172.21.15.165:0/3342594070 <== mon.1 v2:172.21.15.165:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55eb31199a40 con 0x55eb3116cc00 2024-03-28T00:32:07.480 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: starting mds.i at 2024-03-28T00:32:07.480 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:07 smithi165 bash[72773]: 16e6cd19bdbe3f6021e0dd2438bdb676a60653b43e40b67bf379124de1547c3b 2024-03-28T00:32:07.480 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:07 smithi165 systemd[1]: Started Ceph mds.i for 10f73b92-ec98-11ee-95d0-87774f69a715. 2024-03-28T00:32:07.834 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:07 smithi165 podman[72772]: 2024-03-28 00:32:07.56497942 +0000 UTC m=+0.801910214 container init 59e8ab3b38abc37a920297b6e1fd2dcb0aa3604b5277ecc5ba031cb07f51975e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, ceph=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-03-28T00:32:07.834 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:07 smithi165 podman[72772]: 2024-03-28 00:32:07.567829211 +0000 UTC m=+0.804759983 container start 59e8ab3b38abc37a920297b6e1fd2dcb0aa3604b5277ecc5ba031cb07f51975e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_BRANCH=HEAD) 2024-03-28T00:32:07.834 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:07 smithi165 ceph-mds[72831]: -- 172.21.15.165:0/1124920632 <== mon.1 v2:172.21.15.165:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56285b2fda40 con 0x56285b2d1000 2024-03-28T00:32:07.834 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: starting mds.l at 2024-03-28T00:32:07.834 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:07 smithi165 bash[72772]: 59e8ab3b38abc37a920297b6e1fd2dcb0aa3604b5277ecc5ba031cb07f51975e 2024-03-28T00:32:07.834 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:07 smithi165 systemd[1]: Started Ceph mds.l for 10f73b92-ec98-11ee-95d0-87774f69a715. 2024-03-28T00:32:08.454 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:32:08 smithi132 ceph-mon[35941]: pgmap v177: 97 pgs: 97 active+clean; 181 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 122 KiB/s wr, 39 op/s 2024-03-28T00:32:08.454 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:32:08 smithi132 ceph-mon[35941]: daemon mds.i restarted 2024-03-28T00:32:08.454 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:32:08 smithi132 ceph-mon[35941]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-28T00:32:08.454 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:32:08 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-28T00:32:08.454 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:32:08 smithi132 ceph-mon[35941]: osdmap e96: 12 total, 12 up, 12 in 2024-03-28T00:32:08.454 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:32:08 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.165:6832/4070507195,v1:172.21.15.165:6833/4070507195] up:boot 2024-03-28T00:32:08.454 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:32:08 smithi132 ceph-mon[35941]: Standby daemon mds.j assigned to filesystem cephfs as rank 1 2024-03-28T00:32:08.454 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:32:08 smithi132 ceph-mon[35941]: fsmap cephfs:4/5 {0=b=up:active,2=h=up:active,3=l=up:active,4=f=up:active} 5 up:standby-replay 3 up:standby, 1 failed 2024-03-28T00:32:08.454 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:32:08 smithi132 ceph-mon[35941]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:replay 4 up:active 2024-03-28T00:32:08.454 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:32:08 smithi132 ceph-mon[35941]: daemon mds.l restarted 2024-03-28T00:32:08.455 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:32:08 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-28T00:32:08.455 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:32:08 smithi132 ceph-mon[35941]: osdmap e97: 12 total, 12 up, 12 in 2024-03-28T00:32:08.455 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:32:08 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.165:6838/1687390659,v1:172.21.15.165:6839/1687390659] up:boot 2024-03-28T00:32:08.455 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:32:08 smithi132 ceph-mon[35941]: Standby daemon mds.c assigned to filesystem cephfs as rank 3 2024-03-28T00:32:08.455 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:32:08 smithi132 ceph-mon[35941]: fsmap cephfs:4/5 {0=b=up:active,1=j=up:replay,2=h=up:active,4=f=up:active} 4 up:standby-replay 4 up:standby, 1 failed 2024-03-28T00:32:08.455 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:32:08 smithi132 ceph-mon[35941]: fsmap cephfs:5/5 3 up:standby-replay 4 up:standby 2 up:replay 3 up:active 2024-03-28T00:32:08.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:32:08 smithi165 ceph-mon[36302]: pgmap v177: 97 pgs: 97 active+clean; 181 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 122 KiB/s wr, 39 op/s 2024-03-28T00:32:08.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:32:08 smithi165 ceph-mon[36302]: daemon mds.i restarted 2024-03-28T00:32:08.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:32:08 smithi165 ceph-mon[36302]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-28T00:32:08.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:32:08 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-28T00:32:08.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:32:08 smithi165 ceph-mon[36302]: osdmap e96: 12 total, 12 up, 12 in 2024-03-28T00:32:08.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:32:08 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.165:6832/4070507195,v1:172.21.15.165:6833/4070507195] up:boot 2024-03-28T00:32:08.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:32:08 smithi165 ceph-mon[36302]: Standby daemon mds.j assigned to filesystem cephfs as rank 1 2024-03-28T00:32:08.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:32:08 smithi165 ceph-mon[36302]: fsmap cephfs:4/5 {0=b=up:active,2=h=up:active,3=l=up:active,4=f=up:active} 5 up:standby-replay 3 up:standby, 1 failed 2024-03-28T00:32:08.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:32:08 smithi165 ceph-mon[36302]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:replay 4 up:active 2024-03-28T00:32:08.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:32:08 smithi165 ceph-mon[36302]: daemon mds.l restarted 2024-03-28T00:32:08.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:32:08 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-28T00:32:08.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:32:08 smithi165 ceph-mon[36302]: osdmap e97: 12 total, 12 up, 12 in 2024-03-28T00:32:08.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:32:08 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.165:6838/1687390659,v1:172.21.15.165:6839/1687390659] up:boot 2024-03-28T00:32:08.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:32:08 smithi165 ceph-mon[36302]: Standby daemon mds.c assigned to filesystem cephfs as rank 3 2024-03-28T00:32:08.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:32:08 smithi165 ceph-mon[36302]: fsmap cephfs:4/5 {0=b=up:active,1=j=up:replay,2=h=up:active,4=f=up:active} 4 up:standby-replay 4 up:standby, 1 failed 2024-03-28T00:32:08.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:32:08 smithi165 ceph-mon[36302]: fsmap cephfs:5/5 3 up:standby-replay 4 up:standby 2 up:replay 3 up:active 2024-03-28T00:32:08.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:32:08 smithi119 ceph-mon[31518]: pgmap v177: 97 pgs: 97 active+clean; 181 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 122 KiB/s wr, 39 op/s 2024-03-28T00:32:08.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:32:08 smithi119 ceph-mon[31518]: daemon mds.i restarted 2024-03-28T00:32:08.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:32:08 smithi119 ceph-mon[31518]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-28T00:32:08.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:32:08 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-28T00:32:08.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:32:08 smithi119 ceph-mon[31518]: osdmap e96: 12 total, 12 up, 12 in 2024-03-28T00:32:08.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:32:08 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.165:6832/4070507195,v1:172.21.15.165:6833/4070507195] up:boot 2024-03-28T00:32:08.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:32:08 smithi119 ceph-mon[31518]: Standby daemon mds.j assigned to filesystem cephfs as rank 1 2024-03-28T00:32:08.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:32:08 smithi119 ceph-mon[31518]: fsmap cephfs:4/5 {0=b=up:active,2=h=up:active,3=l=up:active,4=f=up:active} 5 up:standby-replay 3 up:standby, 1 failed 2024-03-28T00:32:08.658 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:32:08 smithi119 ceph-mon[31518]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:replay 4 up:active 2024-03-28T00:32:08.658 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:32:08 smithi119 ceph-mon[31518]: daemon mds.l restarted 2024-03-28T00:32:08.658 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:32:08 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-28T00:32:08.658 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:32:08 smithi119 ceph-mon[31518]: osdmap e97: 12 total, 12 up, 12 in 2024-03-28T00:32:08.658 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:32:08 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.165:6838/1687390659,v1:172.21.15.165:6839/1687390659] up:boot 2024-03-28T00:32:08.658 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:32:08 smithi119 ceph-mon[31518]: Standby daemon mds.c assigned to filesystem cephfs as rank 3 2024-03-28T00:32:08.658 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:32:08 smithi119 ceph-mon[31518]: fsmap cephfs:4/5 {0=b=up:active,1=j=up:replay,2=h=up:active,4=f=up:active} 4 up:standby-replay 4 up:standby, 1 failed 2024-03-28T00:32:08.658 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:32:08 smithi119 ceph-mon[31518]: fsmap cephfs:5/5 3 up:standby-replay 4 up:standby 2 up:replay 3 up:active 2024-03-28T00:32:09.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:32:09 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.119:6834/2101147835,v1:172.21.15.119:6835/2101147835] up:resolve 2024-03-28T00:32:09.907 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:32:09 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.165:6836/3170852275,v1:172.21.15.165:6837/3170852275] up:resolve 2024-03-28T00:32:09.907 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:32:09 smithi119 ceph-mon[31518]: fsmap cephfs:5/5 3 up:standby-replay 4 up:standby 2 up:resolve 3 up:active 2024-03-28T00:32:09.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:32:09 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.119:6834/2101147835,v1:172.21.15.119:6835/2101147835] up:resolve 2024-03-28T00:32:09.954 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:32:09 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.165:6836/3170852275,v1:172.21.15.165:6837/3170852275] up:resolve 2024-03-28T00:32:09.954 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:32:09 smithi132 ceph-mon[35941]: fsmap cephfs:5/5 3 up:standby-replay 4 up:standby 2 up:resolve 3 up:active 2024-03-28T00:32:10.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:32:09 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.119:6834/2101147835,v1:172.21.15.119:6835/2101147835] up:resolve 2024-03-28T00:32:10.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:32:09 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.165:6836/3170852275,v1:172.21.15.165:6837/3170852275] up:resolve 2024-03-28T00:32:10.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:32:09 smithi165 ceph-mon[36302]: fsmap cephfs:5/5 3 up:standby-replay 4 up:standby 2 up:resolve 3 up:active 2024-03-28T00:32:10.954 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:32:10 smithi132 ceph-mon[35941]: pgmap v180: 97 pgs: 97 active+clean; 181 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.0 KiB/s wr, 16 op/s 2024-03-28T00:32:10.954 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:32:10 smithi132 ceph-mon[35941]: reconnect by client.25355 172.21.15.119:0/3071152482 after 0.00299995 2024-03-28T00:32:10.954 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:32:10 smithi132 ceph-mon[35941]: reconnect by client.25355 172.21.15.119:0/3071152482 after 0.00299993 2024-03-28T00:32:10.954 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:32:10 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.165:6836/3170852275,v1:172.21.15.165:6837/3170852275] up:reconnect 2024-03-28T00:32:10.954 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:32:10 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.119:6834/2101147835,v1:172.21.15.119:6835/2101147835] up:reconnect 2024-03-28T00:32:10.954 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:32:10 smithi132 ceph-mon[35941]: fsmap cephfs:5/5 3 up:standby-replay 4 up:standby 2 up:reconnect 3 up:active 2024-03-28T00:32:10.954 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:32:10 smithi132 ceph-mon[35941]: reconnect by client.25264 v1:172.21.15.119:0/3863511511 after 0.0179996 2024-03-28T00:32:10.954 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:32:10 smithi132 ceph-mon[35941]: reconnect by client.25264 v1:172.21.15.119:0/3863511511 after 0.0179997 2024-03-28T00:32:11.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:32:10 smithi165 ceph-mon[36302]: pgmap v180: 97 pgs: 97 active+clean; 181 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.0 KiB/s wr, 16 op/s 2024-03-28T00:32:11.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:32:10 smithi165 ceph-mon[36302]: reconnect by client.25355 172.21.15.119:0/3071152482 after 0.00299995 2024-03-28T00:32:11.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:32:10 smithi165 ceph-mon[36302]: reconnect by client.25355 172.21.15.119:0/3071152482 after 0.00299993 2024-03-28T00:32:11.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:32:10 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.165:6836/3170852275,v1:172.21.15.165:6837/3170852275] up:reconnect 2024-03-28T00:32:11.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:32:10 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.119:6834/2101147835,v1:172.21.15.119:6835/2101147835] up:reconnect 2024-03-28T00:32:11.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:32:10 smithi165 ceph-mon[36302]: fsmap cephfs:5/5 3 up:standby-replay 4 up:standby 2 up:reconnect 3 up:active 2024-03-28T00:32:11.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:32:10 smithi165 ceph-mon[36302]: reconnect by client.25264 v1:172.21.15.119:0/3863511511 after 0.0179996 2024-03-28T00:32:11.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:32:10 smithi165 ceph-mon[36302]: reconnect by client.25264 v1:172.21.15.119:0/3863511511 after 0.0179997 2024-03-28T00:32:11.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:32:10 smithi119 ceph-mon[31518]: pgmap v180: 97 pgs: 97 active+clean; 181 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.0 KiB/s wr, 16 op/s 2024-03-28T00:32:11.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:32:10 smithi119 ceph-mon[31518]: reconnect by client.25355 172.21.15.119:0/3071152482 after 0.00299995 2024-03-28T00:32:11.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:32:10 smithi119 ceph-mon[31518]: reconnect by client.25355 172.21.15.119:0/3071152482 after 0.00299993 2024-03-28T00:32:11.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:32:10 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.165:6836/3170852275,v1:172.21.15.165:6837/3170852275] up:reconnect 2024-03-28T00:32:11.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:32:10 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.119:6834/2101147835,v1:172.21.15.119:6835/2101147835] up:reconnect 2024-03-28T00:32:11.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:32:10 smithi119 ceph-mon[31518]: fsmap cephfs:5/5 3 up:standby-replay 4 up:standby 2 up:reconnect 3 up:active 2024-03-28T00:32:11.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:32:10 smithi119 ceph-mon[31518]: reconnect by client.25264 v1:172.21.15.119:0/3863511511 after 0.0179996 2024-03-28T00:32:11.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:32:10 smithi119 ceph-mon[31518]: reconnect by client.25264 v1:172.21.15.119:0/3863511511 after 0.0179997 2024-03-28T00:32:11.954 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:32:11 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.119:6834/2101147835,v1:172.21.15.119:6835/2101147835] up:rejoin 2024-03-28T00:32:11.954 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:32:11 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.165:6836/3170852275,v1:172.21.15.165:6837/3170852275] up:rejoin 2024-03-28T00:32:11.954 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:32:11 smithi132 ceph-mon[35941]: fsmap cephfs:5/5 3 up:standby-replay 4 up:standby 2 up:rejoin 3 up:active 2024-03-28T00:32:11.954 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:32:11 smithi132 ceph-mon[35941]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-03-28T00:32:11.954 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:32:11 smithi132 ceph-mon[35941]: daemon mds.j is now active in filesystem cephfs as rank 1 2024-03-28T00:32:12.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:32:11 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.119:6834/2101147835,v1:172.21.15.119:6835/2101147835] up:rejoin 2024-03-28T00:32:12.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:32:11 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.165:6836/3170852275,v1:172.21.15.165:6837/3170852275] up:rejoin 2024-03-28T00:32:12.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:32:11 smithi165 ceph-mon[36302]: fsmap cephfs:5/5 3 up:standby-replay 4 up:standby 2 up:rejoin 3 up:active 2024-03-28T00:32:12.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:32:11 smithi165 ceph-mon[36302]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-03-28T00:32:12.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:32:11 smithi165 ceph-mon[36302]: daemon mds.j is now active in filesystem cephfs as rank 1 2024-03-28T00:32:12.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:32:11 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.119:6834/2101147835,v1:172.21.15.119:6835/2101147835] up:rejoin 2024-03-28T00:32:12.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:32:11 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.165:6836/3170852275,v1:172.21.15.165:6837/3170852275] up:rejoin 2024-03-28T00:32:12.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:32:11 smithi119 ceph-mon[31518]: fsmap cephfs:5/5 3 up:standby-replay 4 up:standby 2 up:rejoin 3 up:active 2024-03-28T00:32:12.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:32:11 smithi119 ceph-mon[31518]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-03-28T00:32:12.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:32:11 smithi119 ceph-mon[31518]: daemon mds.j is now active in filesystem cephfs as rank 1 2024-03-28T00:32:12.954 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:32:12 smithi132 ceph-mon[35941]: pgmap v181: 97 pgs: 97 active+clean; 181 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.0 KiB/s wr, 16 op/s 2024-03-28T00:32:12.954 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:32:12 smithi132 ceph-mon[35941]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-28T00:32:12.954 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:32:12 smithi132 ceph-mon[35941]: Cluster is now healthy 2024-03-28T00:32:12.954 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:32:12 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.165:6836/3170852275,v1:172.21.15.165:6837/3170852275] up:active 2024-03-28T00:32:12.954 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:32:12 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.119:6834/2101147835,v1:172.21.15.119:6835/2101147835] up:active 2024-03-28T00:32:12.954 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:32:12 smithi132 ceph-mon[35941]: fsmap cephfs:5 3 up:standby-replay 4 up:standby 5 up:active 2024-03-28T00:32:12.954 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:32:12 smithi132 ceph-mon[35941]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-28T00:32:13.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:32:12 smithi165 ceph-mon[36302]: pgmap v181: 97 pgs: 97 active+clean; 181 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.0 KiB/s wr, 16 op/s 2024-03-28T00:32:13.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:32:12 smithi165 ceph-mon[36302]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-28T00:32:13.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:32:12 smithi165 ceph-mon[36302]: Cluster is now healthy 2024-03-28T00:32:13.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:32:12 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.165:6836/3170852275,v1:172.21.15.165:6837/3170852275] up:active 2024-03-28T00:32:13.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:32:12 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.119:6834/2101147835,v1:172.21.15.119:6835/2101147835] up:active 2024-03-28T00:32:13.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:32:12 smithi165 ceph-mon[36302]: fsmap cephfs:5 3 up:standby-replay 4 up:standby 5 up:active 2024-03-28T00:32:13.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:32:12 smithi165 ceph-mon[36302]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-28T00:32:13.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:32:12 smithi119 ceph-mon[31518]: pgmap v181: 97 pgs: 97 active+clean; 181 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.0 KiB/s wr, 16 op/s 2024-03-28T00:32:13.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:32:12 smithi119 ceph-mon[31518]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-28T00:32:13.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:32:12 smithi119 ceph-mon[31518]: Cluster is now healthy 2024-03-28T00:32:13.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:32:12 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.165:6836/3170852275,v1:172.21.15.165:6837/3170852275] up:active 2024-03-28T00:32:13.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:32:12 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.119:6834/2101147835,v1:172.21.15.119:6835/2101147835] up:active 2024-03-28T00:32:13.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:32:12 smithi119 ceph-mon[31518]: fsmap cephfs:5 3 up:standby-replay 4 up:standby 5 up:active 2024-03-28T00:32:13.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:32:12 smithi119 ceph-mon[31518]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-28T00:32:14.313 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7f08c337a700 time 2024-03-28T00:32:14.302663+0000 2024-03-28T00:32:14.313 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T00:32:14.313 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:32:14.313 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f08cd9a44a1] 2024-03-28T00:32:14.313 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f08cd9a4667] 2024-03-28T00:32:14.313 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x5592953b2223] 2024-03-28T00:32:14.313 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55929522c5d6] 2024-03-28T00:32:14.313 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55929522cda1] 2024-03-28T00:32:14.313 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: 6: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x55929526109c] 2024-03-28T00:32:14.314 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55929527f233] 2024-03-28T00:32:14.314 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x559295281197] 2024-03-28T00:32:14.314 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55929518011b] 2024-03-28T00:32:14.314 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5592951806bc] 2024-03-28T00:32:14.314 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55929516a1ff] 2024-03-28T00:32:14.314 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f08cdc36488] 2024-03-28T00:32:14.314 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: 13: (DispatchQueue::entry()+0x51f) [0x7f08cdc3360f] 2024-03-28T00:32:14.314 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f08cdd06e81] 2024-03-28T00:32:14.314 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f08cc7211ca] 2024-03-28T00:32:14.314 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: 16: clone() 2024-03-28T00:32:14.314 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: 2024-03-28T00:32:14.302+0000 7f08c337a700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7f08c337a700 time 2024-03-28T00:32:14.302663+0000 2024-03-28T00:32:14.314 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T00:32:14.314 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: 2024-03-28T00:32:14.315 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:32:14.634 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f08cd9a44a1] 2024-03-28T00:32:14.634 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f08cd9a4667] 2024-03-28T00:32:14.634 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x5592953b2223] 2024-03-28T00:32:14.635 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55929522c5d6] 2024-03-28T00:32:14.635 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55929522cda1] 2024-03-28T00:32:14.635 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: 6: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x55929526109c] 2024-03-28T00:32:14.635 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55929527f233] 2024-03-28T00:32:14.635 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x559295281197] 2024-03-28T00:32:14.635 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55929518011b] 2024-03-28T00:32:14.635 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5592951806bc] 2024-03-28T00:32:14.635 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55929516a1ff] 2024-03-28T00:32:14.635 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f08cdc36488] 2024-03-28T00:32:14.635 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: 13: (DispatchQueue::entry()+0x51f) [0x7f08cdc3360f] 2024-03-28T00:32:14.635 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f08cdd06e81] 2024-03-28T00:32:14.635 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f08cc7211ca] 2024-03-28T00:32:14.635 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: 16: clone() 2024-03-28T00:32:14.635 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: 2024-03-28T00:32:14.635 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: *** Caught signal (Aborted) ** 2024-03-28T00:32:14.636 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: in thread 7f08c337a700 thread_name:ms_dispatch 2024-03-28T00:32:14.636 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:32:14.636 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f08cc72bd20] 2024-03-28T00:32:14.636 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: 2: gsignal() 2024-03-28T00:32:14.636 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: 3: abort() 2024-03-28T00:32:14.636 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f08cd9a44fb] 2024-03-28T00:32:14.636 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f08cd9a4667] 2024-03-28T00:32:14.636 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x5592953b2223] 2024-03-28T00:32:14.637 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55929522c5d6] 2024-03-28T00:32:14.637 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55929522cda1] 2024-03-28T00:32:14.637 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: 9: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x55929526109c] 2024-03-28T00:32:14.637 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55929527f233] 2024-03-28T00:32:14.637 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x559295281197] 2024-03-28T00:32:14.638 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55929518011b] 2024-03-28T00:32:14.638 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5592951806bc] 2024-03-28T00:32:14.638 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55929516a1ff] 2024-03-28T00:32:14.638 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f08cdc36488] 2024-03-28T00:32:14.638 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: 16: (DispatchQueue::entry()+0x51f) [0x7f08cdc3360f] 2024-03-28T00:32:14.638 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f08cdd06e81] 2024-03-28T00:32:14.638 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f08cc7211ca] 2024-03-28T00:32:14.638 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: 19: clone() 2024-03-28T00:32:14.638 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: 2024-03-28T00:32:14.305+0000 7f08c337a700 -1 *** Caught signal (Aborted) ** 2024-03-28T00:32:14.638 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: in thread 7f08c337a700 thread_name:ms_dispatch 2024-03-28T00:32:14.638 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: 2024-03-28T00:32:14.638 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:32:14.638 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f08cc72bd20] 2024-03-28T00:32:14.639 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: 2: gsignal() 2024-03-28T00:32:14.639 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: 3: abort() 2024-03-28T00:32:14.639 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f08cd9a44fb] 2024-03-28T00:32:14.639 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f08cd9a4667] 2024-03-28T00:32:14.639 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x5592953b2223] 2024-03-28T00:32:14.639 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55929522c5d6] 2024-03-28T00:32:14.639 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55929522cda1] 2024-03-28T00:32:14.639 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: 9: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x55929526109c] 2024-03-28T00:32:14.639 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55929527f233] 2024-03-28T00:32:14.639 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x559295281197] 2024-03-28T00:32:14.639 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55929518011b] 2024-03-28T00:32:14.639 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5592951806bc] 2024-03-28T00:32:14.639 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55929516a1ff] 2024-03-28T00:32:14.639 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f08cdc36488] 2024-03-28T00:32:14.640 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: 16: (DispatchQueue::entry()+0x51f) [0x7f08cdc3360f] 2024-03-28T00:32:14.640 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f08cdd06e81] 2024-03-28T00:32:14.640 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f08cc7211ca] 2024-03-28T00:32:14.640 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: 19: clone() 2024-03-28T00:32:14.640 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T00:32:14.640 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: 2024-03-28T00:32:14.640 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: -1> 2024-03-28T00:32:14.302+0000 7f08c337a700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7f08c337a700 time 2024-03-28T00:32:14.302663+0000 2024-03-28T00:32:14.640 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T00:32:14.640 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: 2024-03-28T00:32:14.640 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:32:14.640 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f08cd9a44a1] 2024-03-28T00:32:14.640 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f08cd9a4667] 2024-03-28T00:32:14.640 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x5592953b2223] 2024-03-28T00:32:14.640 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55929522c5d6] 2024-03-28T00:32:14.640 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55929522cda1] 2024-03-28T00:32:14.641 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: 6: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x55929526109c] 2024-03-28T00:32:14.641 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55929527f233] 2024-03-28T00:32:14.641 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x559295281197] 2024-03-28T00:32:14.641 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55929518011b] 2024-03-28T00:32:14.641 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5592951806bc] 2024-03-28T00:32:14.641 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55929516a1ff] 2024-03-28T00:32:14.641 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f08cdc36488] 2024-03-28T00:32:14.641 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: 13: (DispatchQueue::entry()+0x51f) [0x7f08cdc3360f] 2024-03-28T00:32:14.641 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f08cdd06e81] 2024-03-28T00:32:14.641 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f08cc7211ca] 2024-03-28T00:32:14.641 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: 16: clone() 2024-03-28T00:32:14.641 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: 2024-03-28T00:32:14.641 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: 0> 2024-03-28T00:32:14.305+0000 7f08c337a700 -1 *** Caught signal (Aborted) ** 2024-03-28T00:32:14.641 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: in thread 7f08c337a700 thread_name:ms_dispatch 2024-03-28T00:32:14.641 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: 2024-03-28T00:32:14.642 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:32:14.642 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f08cc72bd20] 2024-03-28T00:32:14.642 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: 2: gsignal() 2024-03-28T00:32:14.642 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: 3: abort() 2024-03-28T00:32:14.642 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f08cd9a44fb] 2024-03-28T00:32:14.642 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f08cd9a4667] 2024-03-28T00:32:14.642 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x5592953b2223] 2024-03-28T00:32:14.642 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55929522c5d6] 2024-03-28T00:32:14.642 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55929522cda1] 2024-03-28T00:32:14.643 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: 9: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x55929526109c] 2024-03-28T00:32:14.643 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55929527f233] 2024-03-28T00:32:14.643 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x559295281197] 2024-03-28T00:32:14.643 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55929518011b] 2024-03-28T00:32:14.643 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5592951806bc] 2024-03-28T00:32:14.643 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55929516a1ff] 2024-03-28T00:32:14.643 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f08cdc36488] 2024-03-28T00:32:14.643 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: 16: (DispatchQueue::entry()+0x51f) [0x7f08cdc3360f] 2024-03-28T00:32:14.643 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f08cdd06e81] 2024-03-28T00:32:14.643 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f08cc7211ca] 2024-03-28T00:32:14.643 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: 19: clone() 2024-03-28T00:32:14.643 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T00:32:14.643 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: 2024-03-28T00:32:14.643 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: -1> 2024-03-28T00:32:14.302+0000 7f08c337a700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7f08c337a700 time 2024-03-28T00:32:14.302663+0000 2024-03-28T00:32:14.643 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T00:32:14.644 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: 2024-03-28T00:32:14.644 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:32:14.644 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f08cd9a44a1] 2024-03-28T00:32:14.644 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f08cd9a4667] 2024-03-28T00:32:14.644 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x5592953b2223] 2024-03-28T00:32:14.644 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55929522c5d6] 2024-03-28T00:32:14.644 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55929522cda1] 2024-03-28T00:32:14.644 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: 6: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x55929526109c] 2024-03-28T00:32:14.644 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55929527f233] 2024-03-28T00:32:14.644 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x559295281197] 2024-03-28T00:32:14.644 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55929518011b] 2024-03-28T00:32:14.644 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5592951806bc] 2024-03-28T00:32:14.644 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55929516a1ff] 2024-03-28T00:32:14.644 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f08cdc36488] 2024-03-28T00:32:14.645 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: 13: (DispatchQueue::entry()+0x51f) [0x7f08cdc3360f] 2024-03-28T00:32:14.645 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f08cdd06e81] 2024-03-28T00:32:14.645 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f08cc7211ca] 2024-03-28T00:32:14.645 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: 16: clone() 2024-03-28T00:32:14.645 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: 2024-03-28T00:32:14.645 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: 0> 2024-03-28T00:32:14.305+0000 7f08c337a700 -1 *** Caught signal (Aborted) ** 2024-03-28T00:32:14.645 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: in thread 7f08c337a700 thread_name:ms_dispatch 2024-03-28T00:32:14.645 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: 2024-03-28T00:32:14.645 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:32:14.645 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f08cc72bd20] 2024-03-28T00:32:14.645 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: 2: gsignal() 2024-03-28T00:32:14.645 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: 3: abort() 2024-03-28T00:32:14.645 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f08cd9a44fb] 2024-03-28T00:32:14.645 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f08cd9a4667] 2024-03-28T00:32:14.645 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x5592953b2223] 2024-03-28T00:32:14.646 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55929522c5d6] 2024-03-28T00:32:14.646 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55929522cda1] 2024-03-28T00:32:14.646 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: 9: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x55929526109c] 2024-03-28T00:32:14.646 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55929527f233] 2024-03-28T00:32:14.646 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x559295281197] 2024-03-28T00:32:14.646 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55929518011b] 2024-03-28T00:32:14.646 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5592951806bc] 2024-03-28T00:32:14.646 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55929516a1ff] 2024-03-28T00:32:14.646 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f08cdc36488] 2024-03-28T00:32:14.646 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: 16: (DispatchQueue::entry()+0x51f) [0x7f08cdc3360f] 2024-03-28T00:32:14.646 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f08cdd06e81] 2024-03-28T00:32:14.646 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f08cc7211ca] 2024-03-28T00:32:14.646 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: 19: clone() 2024-03-28T00:32:14.646 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T00:32:14.646 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72422]: 2024-03-28T00:32:14.647 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:14 smithi165 podman[72887]: 2024-03-28 00:32:14.374016139 +0000 UTC m=+0.020272043 container died de384b7ccc8d47e7b1907bf6fdb4e88f7996ce19e634b4cea65ea1688af60cb6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_CLEAN=True) 2024-03-28T00:32:14.954 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:32:14 smithi132 ceph-mon[35941]: pgmap v182: 97 pgs: 97 active+clean; 181 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 7 op/s 2024-03-28T00:32:15.009 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:32:14 smithi165 ceph-mon[36302]: pgmap v182: 97 pgs: 97 active+clean; 181 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 7 op/s 2024-03-28T00:32:15.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:32:14 smithi119 ceph-mon[31518]: pgmap v182: 97 pgs: 97 active+clean; 181 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 7 op/s 2024-03-28T00:32:15.384 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:15 smithi165 podman[72887]: 2024-03-28 00:32:15.002984529 +0000 UTC m=+0.649240426 container remove de384b7ccc8d47e7b1907bf6fdb4e88f7996ce19e634b4cea65ea1688af60cb6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c, ceph=True, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-03-28T00:32:15.384 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:15 smithi165 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.c.service: Main process exited, code=exited, status=134/n/a 2024-03-28T00:32:15.384 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:15 smithi165 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.c.service: Failed with result 'exit-code'. 2024-03-28T00:32:15.384 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:15 smithi165 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.c.service: Consumed 1.696s CPU time. 2024-03-28T00:32:16.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-mon[31518]: pgmap v183: 97 pgs: 97 active+clean; 181 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 335 KiB/s rd, 48 KiB/s wr, 33 op/s 2024-03-28T00:32:16.908 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7f7c28400700 time 2024-03-28T00:32:16.620108+0000 2024-03-28T00:32:16.908 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T00:32:16.908 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:32:16.908 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f7c33cbf4a1] 2024-03-28T00:32:16.908 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f7c33cbf667] 2024-03-28T00:32:16.908 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55cc6ca54223] 2024-03-28T00:32:16.908 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55cc6c8ce5d6] 2024-03-28T00:32:16.908 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55cc6c8ceda1] 2024-03-28T00:32:16.909 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: 6: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x55cc6c90309c] 2024-03-28T00:32:16.909 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55cc6c921233] 2024-03-28T00:32:16.909 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55cc6c923197] 2024-03-28T00:32:16.909 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55cc6c82211b] 2024-03-28T00:32:16.909 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55cc6c8226bc] 2024-03-28T00:32:16.909 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55cc6c80c1ff] 2024-03-28T00:32:16.909 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f7c33f51488] 2024-03-28T00:32:16.909 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: 13: (DispatchQueue::entry()+0x51f) [0x7f7c33f4e60f] 2024-03-28T00:32:16.909 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f7c34021e81] 2024-03-28T00:32:16.909 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f7c32a3c1ca] 2024-03-28T00:32:16.909 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: 16: clone() 2024-03-28T00:32:16.909 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: *** Caught signal (Aborted) ** 2024-03-28T00:32:16.910 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: in thread 7f7c28400700 thread_name:ms_dispatch 2024-03-28T00:32:16.910 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: 2024-03-28T00:32:16.623+0000 7f7c28400700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7f7c28400700 time 2024-03-28T00:32:16.620108+0000 2024-03-28T00:32:16.910 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T00:32:16.910 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: 2024-03-28T00:32:16.910 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:32:16.910 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f7c33cbf4a1] 2024-03-28T00:32:16.911 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f7c33cbf667] 2024-03-28T00:32:16.911 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55cc6ca54223] 2024-03-28T00:32:16.911 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55cc6c8ce5d6] 2024-03-28T00:32:16.911 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55cc6c8ceda1] 2024-03-28T00:32:16.911 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: 6: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x55cc6c90309c] 2024-03-28T00:32:16.911 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55cc6c921233] 2024-03-28T00:32:16.911 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55cc6c923197] 2024-03-28T00:32:16.911 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55cc6c82211b] 2024-03-28T00:32:16.911 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55cc6c8226bc] 2024-03-28T00:32:16.911 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55cc6c80c1ff] 2024-03-28T00:32:16.911 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f7c33f51488] 2024-03-28T00:32:16.911 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: 13: (DispatchQueue::entry()+0x51f) [0x7f7c33f4e60f] 2024-03-28T00:32:16.912 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f7c34021e81] 2024-03-28T00:32:16.912 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f7c32a3c1ca] 2024-03-28T00:32:16.912 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: 16: clone() 2024-03-28T00:32:16.912 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: 2024-03-28T00:32:16.912 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:32:16.912 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f7c32a46d20] 2024-03-28T00:32:16.912 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: 2: gsignal() 2024-03-28T00:32:16.912 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: 3: abort() 2024-03-28T00:32:16.913 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f7c33cbf4fb] 2024-03-28T00:32:16.913 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f7c33cbf667] 2024-03-28T00:32:16.913 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55cc6ca54223] 2024-03-28T00:32:16.913 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55cc6c8ce5d6] 2024-03-28T00:32:16.913 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55cc6c8ceda1] 2024-03-28T00:32:16.913 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: 9: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x55cc6c90309c] 2024-03-28T00:32:16.913 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55cc6c921233] 2024-03-28T00:32:16.913 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55cc6c923197] 2024-03-28T00:32:16.913 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55cc6c82211b] 2024-03-28T00:32:16.913 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55cc6c8226bc] 2024-03-28T00:32:16.913 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55cc6c80c1ff] 2024-03-28T00:32:16.913 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f7c33f51488] 2024-03-28T00:32:16.914 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: 16: (DispatchQueue::entry()+0x51f) [0x7f7c33f4e60f] 2024-03-28T00:32:16.914 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f7c34021e81] 2024-03-28T00:32:16.914 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f7c32a3c1ca] 2024-03-28T00:32:16.914 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: 19: clone() 2024-03-28T00:32:16.914 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: 2024-03-28T00:32:16.631+0000 7f7c28400700 -1 *** Caught signal (Aborted) ** 2024-03-28T00:32:16.914 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: in thread 7f7c28400700 thread_name:ms_dispatch 2024-03-28T00:32:16.914 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: 2024-03-28T00:32:16.914 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:32:16.914 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f7c32a46d20] 2024-03-28T00:32:16.914 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: 2: gsignal() 2024-03-28T00:32:16.914 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: 3: abort() 2024-03-28T00:32:16.914 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f7c33cbf4fb] 2024-03-28T00:32:16.915 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f7c33cbf667] 2024-03-28T00:32:16.915 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55cc6ca54223] 2024-03-28T00:32:16.915 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55cc6c8ce5d6] 2024-03-28T00:32:16.915 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55cc6c8ceda1] 2024-03-28T00:32:16.915 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: 9: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x55cc6c90309c] 2024-03-28T00:32:16.916 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55cc6c921233] 2024-03-28T00:32:16.916 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55cc6c923197] 2024-03-28T00:32:16.916 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55cc6c82211b] 2024-03-28T00:32:16.916 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55cc6c8226bc] 2024-03-28T00:32:16.916 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55cc6c80c1ff] 2024-03-28T00:32:16.916 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f7c33f51488] 2024-03-28T00:32:16.916 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: 16: (DispatchQueue::entry()+0x51f) [0x7f7c33f4e60f] 2024-03-28T00:32:16.916 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f7c34021e81] 2024-03-28T00:32:16.916 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f7c32a3c1ca] 2024-03-28T00:32:16.916 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: 19: clone() 2024-03-28T00:32:16.916 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T00:32:16.917 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: 2024-03-28T00:32:16.917 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: -1> 2024-03-28T00:32:16.623+0000 7f7c28400700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7f7c28400700 time 2024-03-28T00:32:16.620108+0000 2024-03-28T00:32:16.917 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T00:32:16.917 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: 2024-03-28T00:32:16.917 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:32:16.917 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f7c33cbf4a1] 2024-03-28T00:32:16.917 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f7c33cbf667] 2024-03-28T00:32:16.917 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55cc6ca54223] 2024-03-28T00:32:16.917 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55cc6c8ce5d6] 2024-03-28T00:32:16.917 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55cc6c8ceda1] 2024-03-28T00:32:16.917 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: 6: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x55cc6c90309c] 2024-03-28T00:32:16.917 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55cc6c921233] 2024-03-28T00:32:16.918 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55cc6c923197] 2024-03-28T00:32:16.918 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55cc6c82211b] 2024-03-28T00:32:16.918 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55cc6c8226bc] 2024-03-28T00:32:16.918 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55cc6c80c1ff] 2024-03-28T00:32:16.918 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f7c33f51488] 2024-03-28T00:32:16.918 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: 13: (DispatchQueue::entry()+0x51f) [0x7f7c33f4e60f] 2024-03-28T00:32:16.918 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f7c34021e81] 2024-03-28T00:32:16.918 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f7c32a3c1ca] 2024-03-28T00:32:16.918 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: 16: clone() 2024-03-28T00:32:16.918 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: 2024-03-28T00:32:16.918 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: 0> 2024-03-28T00:32:16.631+0000 7f7c28400700 -1 *** Caught signal (Aborted) ** 2024-03-28T00:32:16.918 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: in thread 7f7c28400700 thread_name:ms_dispatch 2024-03-28T00:32:16.919 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: 2024-03-28T00:32:16.919 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:32:16.919 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f7c32a46d20] 2024-03-28T00:32:16.919 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: 2: gsignal() 2024-03-28T00:32:16.919 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: 3: abort() 2024-03-28T00:32:16.919 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f7c33cbf4fb] 2024-03-28T00:32:16.919 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f7c33cbf667] 2024-03-28T00:32:16.919 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55cc6ca54223] 2024-03-28T00:32:16.919 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55cc6c8ce5d6] 2024-03-28T00:32:16.920 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55cc6c8ceda1] 2024-03-28T00:32:16.920 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: 9: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x55cc6c90309c] 2024-03-28T00:32:16.920 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55cc6c921233] 2024-03-28T00:32:16.920 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55cc6c923197] 2024-03-28T00:32:16.920 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55cc6c82211b] 2024-03-28T00:32:16.920 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55cc6c8226bc] 2024-03-28T00:32:16.920 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55cc6c80c1ff] 2024-03-28T00:32:16.920 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f7c33f51488] 2024-03-28T00:32:16.921 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: 16: (DispatchQueue::entry()+0x51f) [0x7f7c33f4e60f] 2024-03-28T00:32:16.921 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f7c34021e81] 2024-03-28T00:32:16.921 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f7c32a3c1ca] 2024-03-28T00:32:16.921 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: 19: clone() 2024-03-28T00:32:16.921 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T00:32:16.921 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: 2024-03-28T00:32:16.921 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: -1> 2024-03-28T00:32:16.623+0000 7f7c28400700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7f7c28400700 time 2024-03-28T00:32:16.620108+0000 2024-03-28T00:32:16.921 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T00:32:16.921 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: 2024-03-28T00:32:16.921 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:32:16.921 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f7c33cbf4a1] 2024-03-28T00:32:16.921 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f7c33cbf667] 2024-03-28T00:32:16.922 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55cc6ca54223] 2024-03-28T00:32:16.922 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55cc6c8ce5d6] 2024-03-28T00:32:16.922 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55cc6c8ceda1] 2024-03-28T00:32:16.922 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: 6: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x55cc6c90309c] 2024-03-28T00:32:16.922 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55cc6c921233] 2024-03-28T00:32:16.922 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55cc6c923197] 2024-03-28T00:32:16.922 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55cc6c82211b] 2024-03-28T00:32:16.922 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55cc6c8226bc] 2024-03-28T00:32:16.922 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55cc6c80c1ff] 2024-03-28T00:32:16.922 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f7c33f51488] 2024-03-28T00:32:16.923 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: 13: (DispatchQueue::entry()+0x51f) [0x7f7c33f4e60f] 2024-03-28T00:32:16.923 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f7c34021e81] 2024-03-28T00:32:16.923 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f7c32a3c1ca] 2024-03-28T00:32:16.923 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: 16: clone() 2024-03-28T00:32:16.923 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: 2024-03-28T00:32:16.923 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: 0> 2024-03-28T00:32:16.631+0000 7f7c28400700 -1 *** Caught signal (Aborted) ** 2024-03-28T00:32:16.923 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: in thread 7f7c28400700 thread_name:ms_dispatch 2024-03-28T00:32:16.923 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: 2024-03-28T00:32:16.923 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:32:16.923 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f7c32a46d20] 2024-03-28T00:32:16.923 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: 2: gsignal() 2024-03-28T00:32:16.923 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: 3: abort() 2024-03-28T00:32:16.923 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f7c33cbf4fb] 2024-03-28T00:32:16.924 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f7c33cbf667] 2024-03-28T00:32:16.924 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55cc6ca54223] 2024-03-28T00:32:16.924 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55cc6c8ce5d6] 2024-03-28T00:32:16.924 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55cc6c8ceda1] 2024-03-28T00:32:16.924 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: 9: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x55cc6c90309c] 2024-03-28T00:32:16.924 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55cc6c921233] 2024-03-28T00:32:16.924 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55cc6c923197] 2024-03-28T00:32:16.924 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55cc6c82211b] 2024-03-28T00:32:16.924 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55cc6c8226bc] 2024-03-28T00:32:16.924 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55cc6c80c1ff] 2024-03-28T00:32:16.924 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f7c33f51488] 2024-03-28T00:32:16.924 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: 16: (DispatchQueue::entry()+0x51f) [0x7f7c33f4e60f] 2024-03-28T00:32:16.924 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f7c34021e81] 2024-03-28T00:32:16.925 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f7c32a3c1ca] 2024-03-28T00:32:16.925 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: 19: clone() 2024-03-28T00:32:16.925 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T00:32:16.925 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[108740]: 2024-03-28T00:32:16.925 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:16 smithi119 podman[109129]: 2024-03-28 00:32:16.787084941 +0000 UTC m=+0.038507292 container died 002ee9aafce575c635a79fb90cb329d0aa3ce302d8a921962d65e38bb75838d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-28T00:32:16.954 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:32:16 smithi132 ceph-mon[35941]: pgmap v183: 97 pgs: 97 active+clean; 181 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 335 KiB/s rd, 48 KiB/s wr, 33 op/s 2024-03-28T00:32:17.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:32:16 smithi165 ceph-mon[36302]: pgmap v183: 97 pgs: 97 active+clean; 181 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 335 KiB/s rd, 48 KiB/s wr, 33 op/s 2024-03-28T00:32:17.906 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:17 smithi119 podman[109129]: 2024-03-28 00:32:17.569295809 +0000 UTC m=+0.820718200 container remove 002ee9aafce575c635a79fb90cb329d0aa3ce302d8a921962d65e38bb75838d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True) 2024-03-28T00:32:17.907 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:17 smithi119 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.j.service: Main process exited, code=exited, status=134/n/a 2024-03-28T00:32:18.406 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:18 smithi119 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.j.service: Failed with result 'exit-code'. 2024-03-28T00:32:18.407 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:18 smithi119 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.j.service: Consumed 5.232s CPU time. 2024-03-28T00:32:18.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:32:18 smithi132 ceph-mon[35941]: pgmap v184: 97 pgs: 97 active+clean; 181 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 39 KiB/s wr, 27 op/s 2024-03-28T00:32:19.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:32:18 smithi165 ceph-mon[36302]: pgmap v184: 97 pgs: 97 active+clean; 181 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 39 KiB/s wr, 27 op/s 2024-03-28T00:32:19.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:32:18 smithi119 ceph-mon[31518]: pgmap v184: 97 pgs: 97 active+clean; 181 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 39 KiB/s wr, 27 op/s 2024-03-28T00:32:20.954 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:32:20 smithi132 ceph-mon[35941]: pgmap v185: 97 pgs: 97 active+clean; 195 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.3 MiB/s wr, 45 op/s 2024-03-28T00:32:21.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:32:20 smithi165 ceph-mon[36302]: pgmap v185: 97 pgs: 97 active+clean; 195 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.3 MiB/s wr, 45 op/s 2024-03-28T00:32:21.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:32:20 smithi119 ceph-mon[31518]: pgmap v185: 97 pgs: 97 active+clean; 195 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.3 MiB/s wr, 45 op/s 2024-03-28T00:32:22.954 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:32:22 smithi132 ceph-mon[35941]: pgmap v186: 97 pgs: 97 active+clean; 195 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.2 MiB/s wr, 38 op/s 2024-03-28T00:32:23.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:32:22 smithi165 ceph-mon[36302]: pgmap v186: 97 pgs: 97 active+clean; 195 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.2 MiB/s wr, 38 op/s 2024-03-28T00:32:23.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:32:22 smithi119 ceph-mon[31518]: pgmap v186: 97 pgs: 97 active+clean; 195 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.2 MiB/s wr, 38 op/s 2024-03-28T00:32:24.954 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:32:24 smithi132 ceph-mon[35941]: pgmap v187: 97 pgs: 97 active+clean; 195 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.2 MiB/s wr, 38 op/s 2024-03-28T00:32:25.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:32:24 smithi165 ceph-mon[36302]: pgmap v187: 97 pgs: 97 active+clean; 195 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.2 MiB/s wr, 38 op/s 2024-03-28T00:32:25.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:32:24 smithi119 ceph-mon[31518]: pgmap v187: 97 pgs: 97 active+clean; 195 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.2 MiB/s wr, 38 op/s 2024-03-28T00:32:25.766 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:25 smithi165 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.c.service: Scheduled restart job, restart counter is at 4. 2024-03-28T00:32:25.766 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:25 smithi165 systemd[1]: Stopped Ceph mds.c for 10f73b92-ec98-11ee-95d0-87774f69a715. 2024-03-28T00:32:25.766 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:25 smithi165 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.c.service: Consumed 1.696s CPU time. 2024-03-28T00:32:25.766 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:25 smithi165 systemd[1]: Starting Ceph mds.c for 10f73b92-ec98-11ee-95d0-87774f69a715... 2024-03-28T00:32:25.766 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:25 smithi165 podman[72982]: 2024-03-28 00:32:25.506066382 +0000 UTC m=+0.017958579 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-28T00:32:25.766 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:25 smithi165 podman[72982]: 2024-03-28 00:32:25.62828214 +0000 UTC m=+0.140174336 container create 18a29d5287c93c76aeae1908b4d59391676b3c1aff8a6a79caae94d3fea0062b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=) 2024-03-28T00:32:26.384 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:25 smithi165 podman[72982]: 2024-03-28 00:32:25.897759952 +0000 UTC m=+0.409652156 container init 18a29d5287c93c76aeae1908b4d59391676b3c1aff8a6a79caae94d3fea0062b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True) 2024-03-28T00:32:26.384 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:25 smithi165 podman[72982]: 2024-03-28 00:32:25.900322479 +0000 UTC m=+0.412214664 container start 18a29d5287c93c76aeae1908b4d59391676b3c1aff8a6a79caae94d3fea0062b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-03-28T00:32:26.384 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:25 smithi165 ceph-mds[73001]: -- 172.21.15.165:0/1945040566 <== mon.1 v2:172.21.15.165:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55ad9ef97a40 con 0x55ad9ef6a800 2024-03-28T00:32:26.384 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:25 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: starting mds.c at 2024-03-28T00:32:26.384 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:25 smithi165 bash[72982]: 18a29d5287c93c76aeae1908b4d59391676b3c1aff8a6a79caae94d3fea0062b 2024-03-28T00:32:26.384 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:26 smithi165 systemd[1]: Started Ceph mds.c for 10f73b92-ec98-11ee-95d0-87774f69a715. 2024-03-28T00:32:26.954 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:32:26 smithi132 ceph-mon[35941]: pgmap v188: 97 pgs: 97 active+clean; 195 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 44 op/s 2024-03-28T00:32:26.954 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:32:26 smithi132 ceph-mon[35941]: daemon mds.c restarted 2024-03-28T00:32:26.954 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:32:26 smithi132 ceph-mon[35941]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-28T00:32:26.954 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:32:26 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-28T00:32:26.954 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:32:26 smithi132 ceph-mon[35941]: osdmap e98: 12 total, 12 up, 12 in 2024-03-28T00:32:26.954 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:32:26 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.165:6836/2717860763,v1:172.21.15.165:6837/2717860763] up:boot 2024-03-28T00:32:26.954 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:32:26 smithi132 ceph-mon[35941]: Standby daemon mds.i assigned to filesystem cephfs as rank 3 2024-03-28T00:32:26.954 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:32:26 smithi132 ceph-mon[35941]: fsmap cephfs:4/5 {0=b=up:active,1=j=up:active,2=h=up:active,4=f=up:active} 5 up:standby-replay 3 up:standby, 1 failed 2024-03-28T00:32:26.954 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:32:26 smithi132 ceph-mon[35941]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:replay 4 up:active 2024-03-28T00:32:27.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:32:26 smithi165 ceph-mon[36302]: pgmap v188: 97 pgs: 97 active+clean; 195 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 44 op/s 2024-03-28T00:32:27.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:32:26 smithi165 ceph-mon[36302]: daemon mds.c restarted 2024-03-28T00:32:27.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:32:26 smithi165 ceph-mon[36302]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-28T00:32:27.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:32:26 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-28T00:32:27.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:32:26 smithi165 ceph-mon[36302]: osdmap e98: 12 total, 12 up, 12 in 2024-03-28T00:32:27.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:32:26 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.165:6836/2717860763,v1:172.21.15.165:6837/2717860763] up:boot 2024-03-28T00:32:27.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:32:26 smithi165 ceph-mon[36302]: Standby daemon mds.i assigned to filesystem cephfs as rank 3 2024-03-28T00:32:27.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:32:26 smithi165 ceph-mon[36302]: fsmap cephfs:4/5 {0=b=up:active,1=j=up:active,2=h=up:active,4=f=up:active} 5 up:standby-replay 3 up:standby, 1 failed 2024-03-28T00:32:27.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:32:26 smithi165 ceph-mon[36302]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:replay 4 up:active 2024-03-28T00:32:27.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:32:26 smithi119 ceph-mon[31518]: pgmap v188: 97 pgs: 97 active+clean; 195 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 44 op/s 2024-03-28T00:32:27.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:32:26 smithi119 ceph-mon[31518]: daemon mds.c restarted 2024-03-28T00:32:27.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:32:26 smithi119 ceph-mon[31518]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-28T00:32:27.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:32:26 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-28T00:32:27.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:32:26 smithi119 ceph-mon[31518]: osdmap e98: 12 total, 12 up, 12 in 2024-03-28T00:32:27.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:32:26 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.165:6836/2717860763,v1:172.21.15.165:6837/2717860763] up:boot 2024-03-28T00:32:27.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:32:26 smithi119 ceph-mon[31518]: Standby daemon mds.i assigned to filesystem cephfs as rank 3 2024-03-28T00:32:27.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:32:26 smithi119 ceph-mon[31518]: fsmap cephfs:4/5 {0=b=up:active,1=j=up:active,2=h=up:active,4=f=up:active} 5 up:standby-replay 3 up:standby, 1 failed 2024-03-28T00:32:27.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:32:26 smithi119 ceph-mon[31518]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:replay 4 up:active 2024-03-28T00:32:28.538 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:28 smithi119 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.j.service: Scheduled restart job, restart counter is at 2. 2024-03-28T00:32:28.539 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:28 smithi119 systemd[1]: Stopped Ceph mds.j for 10f73b92-ec98-11ee-95d0-87774f69a715. 2024-03-28T00:32:28.539 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:28 smithi119 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.j.service: Consumed 5.232s CPU time. 2024-03-28T00:32:28.539 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:28 smithi119 systemd[1]: Starting Ceph mds.j for 10f73b92-ec98-11ee-95d0-87774f69a715... 2024-03-28T00:32:28.865 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:28 smithi119 podman[109558]: 2024-03-28 00:32:28.436998391 +0000 UTC m=+0.033620885 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-28T00:32:28.865 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:28 smithi119 podman[109558]: 2024-03-28 00:32:28.595099241 +0000 UTC m=+0.191721740 container create 6b6443e2c8ef702998870d58751aba8bc5f0174c06884c32bed3aba87f6a635e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD) 2024-03-28T00:32:28.865 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:32:28 smithi119 ceph-mon[31518]: pgmap v190: 97 pgs: 97 active+clean; 195 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 32 op/s 2024-03-28T00:32:28.865 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:32:28 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.165:6832/4070507195,v1:172.21.15.165:6833/4070507195] up:resolve 2024-03-28T00:32:28.865 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:32:28 smithi119 ceph-mon[31518]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:resolve 4 up:active 2024-03-28T00:32:29.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:32:28 smithi165 ceph-mon[36302]: pgmap v190: 97 pgs: 97 active+clean; 195 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 32 op/s 2024-03-28T00:32:29.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:32:28 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.165:6832/4070507195,v1:172.21.15.165:6833/4070507195] up:resolve 2024-03-28T00:32:29.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:32:28 smithi165 ceph-mon[36302]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:resolve 4 up:active 2024-03-28T00:32:29.156 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:28 smithi119 podman[109558]: 2024-03-28 00:32:28.858319587 +0000 UTC m=+0.454942124 container init 6b6443e2c8ef702998870d58751aba8bc5f0174c06884c32bed3aba87f6a635e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-03-28T00:32:29.156 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:28 smithi119 podman[109558]: 2024-03-28 00:32:28.867454307 +0000 UTC m=+0.464076806 container start 6b6443e2c8ef702998870d58751aba8bc5f0174c06884c32bed3aba87f6a635e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j, GIT_CLEAN=True, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c) 2024-03-28T00:32:29.156 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:28 smithi119 ceph-mds[109644]: -- 172.21.15.119:0/2149611052 <== mon.1 v2:172.21.15.165:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x558ea5b69a40 con 0x558ea5b3cc00 2024-03-28T00:32:29.156 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:28 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: starting mds.j at 2024-03-28T00:32:29.156 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:28 smithi119 bash[109558]: 6b6443e2c8ef702998870d58751aba8bc5f0174c06884c32bed3aba87f6a635e 2024-03-28T00:32:29.157 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:28 smithi119 systemd[1]: Started Ceph mds.j for 10f73b92-ec98-11ee-95d0-87774f69a715. 2024-03-28T00:32:29.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:32:28 smithi132 ceph-mon[35941]: pgmap v190: 97 pgs: 97 active+clean; 195 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 32 op/s 2024-03-28T00:32:29.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:32:28 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.165:6832/4070507195,v1:172.21.15.165:6833/4070507195] up:resolve 2024-03-28T00:32:29.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:32:28 smithi132 ceph-mon[35941]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:resolve 4 up:active 2024-03-28T00:32:30.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:32:29 smithi165 ceph-mon[36302]: daemon mds.j restarted 2024-03-28T00:32:30.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:32:29 smithi165 ceph-mon[36302]: osdmap e99: 12 total, 12 up, 12 in 2024-03-28T00:32:30.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:32:29 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.119:6834/2210305461,v1:172.21.15.119:6835/2210305461] up:boot 2024-03-28T00:32:30.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:32:29 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-28T00:32:30.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:32:29 smithi165 ceph-mon[36302]: Standby daemon mds.l assigned to filesystem cephfs as rank 1 2024-03-28T00:32:30.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:32:29 smithi165 ceph-mon[36302]: fsmap cephfs:4/5 {0=b=up:active,2=h=up:active,3=i=up:resolve,4=f=up:active} 4 up:standby-replay 4 up:standby, 1 failed 2024-03-28T00:32:30.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:32:29 smithi165 ceph-mon[36302]: fsmap cephfs:5/5 3 up:standby-replay 4 up:standby 1 up:replay 1 up:resolve 3 up:active 2024-03-28T00:32:30.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:32:29 smithi119 ceph-mon[31518]: daemon mds.j restarted 2024-03-28T00:32:30.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:32:29 smithi119 ceph-mon[31518]: osdmap e99: 12 total, 12 up, 12 in 2024-03-28T00:32:30.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:32:29 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.119:6834/2210305461,v1:172.21.15.119:6835/2210305461] up:boot 2024-03-28T00:32:30.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:32:29 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-28T00:32:30.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:32:29 smithi119 ceph-mon[31518]: Standby daemon mds.l assigned to filesystem cephfs as rank 1 2024-03-28T00:32:30.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:32:29 smithi119 ceph-mon[31518]: fsmap cephfs:4/5 {0=b=up:active,2=h=up:active,3=i=up:resolve,4=f=up:active} 4 up:standby-replay 4 up:standby, 1 failed 2024-03-28T00:32:30.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:32:29 smithi119 ceph-mon[31518]: fsmap cephfs:5/5 3 up:standby-replay 4 up:standby 1 up:replay 1 up:resolve 3 up:active 2024-03-28T00:32:30.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:32:29 smithi132 ceph-mon[35941]: daemon mds.j restarted 2024-03-28T00:32:30.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:32:29 smithi132 ceph-mon[35941]: osdmap e99: 12 total, 12 up, 12 in 2024-03-28T00:32:30.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:32:29 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.119:6834/2210305461,v1:172.21.15.119:6835/2210305461] up:boot 2024-03-28T00:32:30.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:32:29 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-28T00:32:30.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:32:29 smithi132 ceph-mon[35941]: Standby daemon mds.l assigned to filesystem cephfs as rank 1 2024-03-28T00:32:30.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:32:29 smithi132 ceph-mon[35941]: fsmap cephfs:4/5 {0=b=up:active,2=h=up:active,3=i=up:resolve,4=f=up:active} 4 up:standby-replay 4 up:standby, 1 failed 2024-03-28T00:32:30.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:32:29 smithi132 ceph-mon[35941]: fsmap cephfs:5/5 3 up:standby-replay 4 up:standby 1 up:replay 1 up:resolve 3 up:active 2024-03-28T00:32:31.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:32:30 smithi165 ceph-mon[36302]: pgmap v192: 97 pgs: 97 active+clean; 197 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 269 KiB/s wr, 72 op/s 2024-03-28T00:32:31.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:32:30 smithi119 ceph-mon[31518]: pgmap v192: 97 pgs: 97 active+clean; 197 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 269 KiB/s wr, 72 op/s 2024-03-28T00:32:31.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:32:30 smithi132 ceph-mon[35941]: pgmap v192: 97 pgs: 97 active+clean; 197 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 269 KiB/s wr, 72 op/s 2024-03-28T00:32:32.105 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:32:31 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.165:6838/1687390659,v1:172.21.15.165:6839/1687390659] up:resolve 2024-03-28T00:32:32.105 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:32:31 smithi132 ceph-mon[35941]: fsmap cephfs:5/5 3 up:standby-replay 4 up:standby 2 up:resolve 3 up:active 2024-03-28T00:32:32.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:32:31 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.165:6838/1687390659,v1:172.21.15.165:6839/1687390659] up:resolve 2024-03-28T00:32:32.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:32:31 smithi165 ceph-mon[36302]: fsmap cephfs:5/5 3 up:standby-replay 4 up:standby 2 up:resolve 3 up:active 2024-03-28T00:32:32.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:32:31 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.165:6838/1687390659,v1:172.21.15.165:6839/1687390659] up:resolve 2024-03-28T00:32:32.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:32:31 smithi119 ceph-mon[31518]: fsmap cephfs:5/5 3 up:standby-replay 4 up:standby 2 up:resolve 3 up:active 2024-03-28T00:32:33.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:32:32 smithi165 ceph-mon[36302]: pgmap v193: 97 pgs: 97 active+clean; 197 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 269 KiB/s wr, 72 op/s 2024-03-28T00:32:33.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:32:32 smithi165 ceph-mon[36302]: reconnect by client.25355 172.21.15.119:0/3071152482 after 0.00199997 2024-03-28T00:32:33.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:32:32 smithi165 ceph-mon[36302]: reconnect by client.25355 172.21.15.119:0/3071152482 after 0.00199997 2024-03-28T00:32:33.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:32:32 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.165:6838/1687390659,v1:172.21.15.165:6839/1687390659] up:reconnect 2024-03-28T00:32:33.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:32:32 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.165:6832/4070507195,v1:172.21.15.165:6833/4070507195] up:reconnect 2024-03-28T00:32:33.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:32:32 smithi165 ceph-mon[36302]: fsmap cephfs:5/5 3 up:standby-replay 4 up:standby 2 up:reconnect 3 up:active 2024-03-28T00:32:33.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:32:32 smithi165 ceph-mon[36302]: reconnect by client.25264 v1:172.21.15.119:0/3863511511 after 0.0179997 2024-03-28T00:32:33.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:32:32 smithi165 ceph-mon[36302]: reconnect by client.25264 v1:172.21.15.119:0/3863511511 after 0.0259996 2024-03-28T00:32:33.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:32:32 smithi119 ceph-mon[31518]: pgmap v193: 97 pgs: 97 active+clean; 197 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 269 KiB/s wr, 72 op/s 2024-03-28T00:32:33.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:32:32 smithi119 ceph-mon[31518]: reconnect by client.25355 172.21.15.119:0/3071152482 after 0.00199997 2024-03-28T00:32:33.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:32:32 smithi119 ceph-mon[31518]: reconnect by client.25355 172.21.15.119:0/3071152482 after 0.00199997 2024-03-28T00:32:33.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:32:32 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.165:6838/1687390659,v1:172.21.15.165:6839/1687390659] up:reconnect 2024-03-28T00:32:33.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:32:32 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.165:6832/4070507195,v1:172.21.15.165:6833/4070507195] up:reconnect 2024-03-28T00:32:33.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:32:32 smithi119 ceph-mon[31518]: fsmap cephfs:5/5 3 up:standby-replay 4 up:standby 2 up:reconnect 3 up:active 2024-03-28T00:32:33.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:32:32 smithi119 ceph-mon[31518]: reconnect by client.25264 v1:172.21.15.119:0/3863511511 after 0.0179997 2024-03-28T00:32:33.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:32:32 smithi119 ceph-mon[31518]: reconnect by client.25264 v1:172.21.15.119:0/3863511511 after 0.0259996 2024-03-28T00:32:33.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:32:32 smithi132 ceph-mon[35941]: pgmap v193: 97 pgs: 97 active+clean; 197 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 269 KiB/s wr, 72 op/s 2024-03-28T00:32:33.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:32:32 smithi132 ceph-mon[35941]: reconnect by client.25355 172.21.15.119:0/3071152482 after 0.00199997 2024-03-28T00:32:33.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:32:32 smithi132 ceph-mon[35941]: reconnect by client.25355 172.21.15.119:0/3071152482 after 0.00199997 2024-03-28T00:32:33.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:32:32 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.165:6838/1687390659,v1:172.21.15.165:6839/1687390659] up:reconnect 2024-03-28T00:32:33.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:32:32 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.165:6832/4070507195,v1:172.21.15.165:6833/4070507195] up:reconnect 2024-03-28T00:32:33.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:32:32 smithi132 ceph-mon[35941]: fsmap cephfs:5/5 3 up:standby-replay 4 up:standby 2 up:reconnect 3 up:active 2024-03-28T00:32:33.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:32:32 smithi132 ceph-mon[35941]: reconnect by client.25264 v1:172.21.15.119:0/3863511511 after 0.0179997 2024-03-28T00:32:33.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:32:32 smithi132 ceph-mon[35941]: reconnect by client.25264 v1:172.21.15.119:0/3863511511 after 0.0259996 2024-03-28T00:32:34.048 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:32:33 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.165:6838/1687390659,v1:172.21.15.165:6839/1687390659] up:rejoin 2024-03-28T00:32:34.049 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:32:33 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.165:6832/4070507195,v1:172.21.15.165:6833/4070507195] up:rejoin 2024-03-28T00:32:34.049 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:32:33 smithi132 ceph-mon[35941]: fsmap cephfs:5/5 3 up:standby-replay 4 up:standby 2 up:rejoin 3 up:active 2024-03-28T00:32:34.049 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:32:33 smithi132 ceph-mon[35941]: daemon mds.l is now active in filesystem cephfs as rank 1 2024-03-28T00:32:34.049 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:32:33 smithi132 ceph-mon[35941]: daemon mds.i is now active in filesystem cephfs as rank 3 2024-03-28T00:32:34.049 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:32:33 smithi132 ceph-mon[35941]: pgmap v194: 97 pgs: 97 active+clean; 197 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 244 KiB/s wr, 63 op/s 2024-03-28T00:32:34.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.165:6838/1687390659,v1:172.21.15.165:6839/1687390659] up:rejoin 2024-03-28T00:32:34.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.165:6832/4070507195,v1:172.21.15.165:6833/4070507195] up:rejoin 2024-03-28T00:32:34.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-mon[36302]: fsmap cephfs:5/5 3 up:standby-replay 4 up:standby 2 up:rejoin 3 up:active 2024-03-28T00:32:34.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-mon[36302]: daemon mds.l is now active in filesystem cephfs as rank 1 2024-03-28T00:32:34.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-mon[36302]: daemon mds.i is now active in filesystem cephfs as rank 3 2024-03-28T00:32:34.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-mon[36302]: pgmap v194: 97 pgs: 97 active+clean; 197 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 244 KiB/s wr, 63 op/s 2024-03-28T00:32:34.135 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7ff72fc24700 time 2024-03-28T00:32:33.818235+0000 2024-03-28T00:32:34.135 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T00:32:34.135 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:32:34.135 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7ff73a24e4a1] 2024-03-28T00:32:34.135 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7ff73a24e667] 2024-03-28T00:32:34.135 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x562859a23223] 2024-03-28T00:32:34.135 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x56285989d5d6] 2024-03-28T00:32:34.135 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x56285989dda1] 2024-03-28T00:32:34.135 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: 6: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x5628598d209c] 2024-03-28T00:32:34.135 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x5628598f0233] 2024-03-28T00:32:34.135 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5628598f2197] 2024-03-28T00:32:34.135 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x5628597f111b] 2024-03-28T00:32:34.136 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5628597f16bc] 2024-03-28T00:32:34.136 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x5628597db1ff] 2024-03-28T00:32:34.136 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7ff73a4e0488] 2024-03-28T00:32:34.136 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: 13: (DispatchQueue::entry()+0x51f) [0x7ff73a4dd60f] 2024-03-28T00:32:34.136 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7ff73a5b0e81] 2024-03-28T00:32:34.136 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7ff738fcb1ca] 2024-03-28T00:32:34.136 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: 16: clone() 2024-03-28T00:32:34.136 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7fbb8acd1700 time 2024-03-28T00:32:33.808481+0000 2024-03-28T00:32:34.136 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T00:32:34.136 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:32:34.137 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fbb952fb4a1] 2024-03-28T00:32:34.137 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7fbb952fb667] 2024-03-28T00:32:34.137 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55eb2ee89223] 2024-03-28T00:32:34.137 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55eb2ed035d6] 2024-03-28T00:32:34.137 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55eb2ed03da1] 2024-03-28T00:32:34.137 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: 6: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x55eb2ed3809c] 2024-03-28T00:32:34.137 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55eb2ed56233] 2024-03-28T00:32:34.137 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55eb2ed58197] 2024-03-28T00:32:34.137 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55eb2ec5711b] 2024-03-28T00:32:34.137 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55eb2ec576bc] 2024-03-28T00:32:34.137 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55eb2ec411ff] 2024-03-28T00:32:34.137 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7fbb9558d488] 2024-03-28T00:32:34.137 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: 13: (DispatchQueue::entry()+0x51f) [0x7fbb9558a60f] 2024-03-28T00:32:34.137 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fbb9565de81] 2024-03-28T00:32:34.138 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fbb940781ca] 2024-03-28T00:32:34.138 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: 16: clone() 2024-03-28T00:32:34.138 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: *** Caught signal (Aborted) ** 2024-03-28T00:32:34.138 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: in thread 7fbb8acd1700 thread_name:ms_dispatch 2024-03-28T00:32:34.138 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: 2024-03-28T00:32:33.819+0000 7ff72fc24700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7ff72fc24700 time 2024-03-28T00:32:33.818235+0000 2024-03-28T00:32:34.138 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T00:32:34.138 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: 2024-03-28T00:32:34.138 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:32:34.138 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7ff73a24e4a1] 2024-03-28T00:32:34.138 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7ff73a24e667] 2024-03-28T00:32:34.138 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x562859a23223] 2024-03-28T00:32:34.139 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x56285989d5d6] 2024-03-28T00:32:34.139 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x56285989dda1] 2024-03-28T00:32:34.139 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: 6: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x5628598d209c] 2024-03-28T00:32:34.139 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x5628598f0233] 2024-03-28T00:32:34.139 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5628598f2197] 2024-03-28T00:32:34.139 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x5628597f111b] 2024-03-28T00:32:34.139 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5628597f16bc] 2024-03-28T00:32:34.139 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x5628597db1ff] 2024-03-28T00:32:34.139 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7ff73a4e0488] 2024-03-28T00:32:34.139 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: 13: (DispatchQueue::entry()+0x51f) [0x7ff73a4dd60f] 2024-03-28T00:32:34.139 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7ff73a5b0e81] 2024-03-28T00:32:34.139 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7ff738fcb1ca] 2024-03-28T00:32:34.139 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: 16: clone() 2024-03-28T00:32:34.139 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: 2024-03-28T00:32:34.140 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: *** Caught signal (Aborted) ** 2024-03-28T00:32:34.140 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: in thread 7ff72fc24700 thread_name:ms_dispatch 2024-03-28T00:32:34.140 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:32:34.140 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7ff738fd5d20] 2024-03-28T00:32:34.140 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: 2024-03-28T00:32:33.809+0000 7fbb8acd1700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7fbb8acd1700 time 2024-03-28T00:32:33.808481+0000 2024-03-28T00:32:34.140 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T00:32:34.140 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: 2024-03-28T00:32:34.140 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:32:34.140 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fbb952fb4a1] 2024-03-28T00:32:34.140 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7fbb952fb667] 2024-03-28T00:32:34.141 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55eb2ee89223] 2024-03-28T00:32:34.141 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55eb2ed035d6] 2024-03-28T00:32:34.141 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55eb2ed03da1] 2024-03-28T00:32:34.141 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: 6: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x55eb2ed3809c] 2024-03-28T00:32:34.141 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55eb2ed56233] 2024-03-28T00:32:34.141 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55eb2ed58197] 2024-03-28T00:32:34.141 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55eb2ec5711b] 2024-03-28T00:32:34.141 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55eb2ec576bc] 2024-03-28T00:32:34.141 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55eb2ec411ff] 2024-03-28T00:32:34.141 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7fbb9558d488] 2024-03-28T00:32:34.141 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: 13: (DispatchQueue::entry()+0x51f) [0x7fbb9558a60f] 2024-03-28T00:32:34.141 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fbb9565de81] 2024-03-28T00:32:34.141 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fbb940781ca] 2024-03-28T00:32:34.141 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: 16: clone() 2024-03-28T00:32:34.142 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: 2024-03-28T00:32:34.142 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:32:34.142 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fbb94082d20] 2024-03-28T00:32:34.142 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: 2: gsignal() 2024-03-28T00:32:34.142 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: 3: abort() 2024-03-28T00:32:34.142 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7ff73a24e4fb] 2024-03-28T00:32:34.142 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7ff73a24e667] 2024-03-28T00:32:34.142 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x562859a23223] 2024-03-28T00:32:34.142 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x56285989d5d6] 2024-03-28T00:32:34.142 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x56285989dda1] 2024-03-28T00:32:34.143 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: 9: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x5628598d209c] 2024-03-28T00:32:34.143 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x5628598f0233] 2024-03-28T00:32:34.143 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5628598f2197] 2024-03-28T00:32:34.143 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x5628597f111b] 2024-03-28T00:32:34.143 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5628597f16bc] 2024-03-28T00:32:34.143 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x5628597db1ff] 2024-03-28T00:32:34.143 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7ff73a4e0488] 2024-03-28T00:32:34.143 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: 16: (DispatchQueue::entry()+0x51f) [0x7ff73a4dd60f] 2024-03-28T00:32:34.143 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7ff73a5b0e81] 2024-03-28T00:32:34.143 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7ff738fcb1ca] 2024-03-28T00:32:34.143 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: 19: clone() 2024-03-28T00:32:34.143 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: 2024-03-28T00:32:33.821+0000 7ff72fc24700 -1 *** Caught signal (Aborted) ** 2024-03-28T00:32:34.143 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: in thread 7ff72fc24700 thread_name:ms_dispatch 2024-03-28T00:32:34.143 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: 2024-03-28T00:32:34.143 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:32:34.144 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7ff738fd5d20] 2024-03-28T00:32:34.144 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: 2: gsignal() 2024-03-28T00:32:34.144 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: 3: abort() 2024-03-28T00:32:34.144 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7ff73a24e4fb] 2024-03-28T00:32:34.144 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7ff73a24e667] 2024-03-28T00:32:34.144 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: 2: gsignal() 2024-03-28T00:32:34.144 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: 3: abort() 2024-03-28T00:32:34.144 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fbb952fb4fb] 2024-03-28T00:32:34.144 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7fbb952fb667] 2024-03-28T00:32:34.144 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55eb2ee89223] 2024-03-28T00:32:34.145 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55eb2ed035d6] 2024-03-28T00:32:34.145 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55eb2ed03da1] 2024-03-28T00:32:34.145 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: 9: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x55eb2ed3809c] 2024-03-28T00:32:34.145 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55eb2ed56233] 2024-03-28T00:32:34.145 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55eb2ed58197] 2024-03-28T00:32:34.145 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55eb2ec5711b] 2024-03-28T00:32:34.145 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55eb2ec576bc] 2024-03-28T00:32:34.145 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55eb2ec411ff] 2024-03-28T00:32:34.145 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7fbb9558d488] 2024-03-28T00:32:34.145 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: 16: (DispatchQueue::entry()+0x51f) [0x7fbb9558a60f] 2024-03-28T00:32:34.145 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fbb9565de81] 2024-03-28T00:32:34.145 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7fbb940781ca] 2024-03-28T00:32:34.145 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: 19: clone() 2024-03-28T00:32:34.146 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: 2024-03-28T00:32:33.811+0000 7fbb8acd1700 -1 *** Caught signal (Aborted) ** 2024-03-28T00:32:34.146 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: in thread 7fbb8acd1700 thread_name:ms_dispatch 2024-03-28T00:32:34.146 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: 2024-03-28T00:32:34.146 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:32:34.146 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fbb94082d20] 2024-03-28T00:32:34.146 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: 2: gsignal() 2024-03-28T00:32:34.146 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: 3: abort() 2024-03-28T00:32:34.146 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fbb952fb4fb] 2024-03-28T00:32:34.146 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7fbb952fb667] 2024-03-28T00:32:34.146 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x562859a23223] 2024-03-28T00:32:34.147 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x56285989d5d6] 2024-03-28T00:32:34.147 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x56285989dda1] 2024-03-28T00:32:34.147 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: 9: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x5628598d209c] 2024-03-28T00:32:34.147 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x5628598f0233] 2024-03-28T00:32:34.147 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5628598f2197] 2024-03-28T00:32:34.147 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x5628597f111b] 2024-03-28T00:32:34.147 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5628597f16bc] 2024-03-28T00:32:34.147 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x5628597db1ff] 2024-03-28T00:32:34.147 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7ff73a4e0488] 2024-03-28T00:32:34.147 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: 16: (DispatchQueue::entry()+0x51f) [0x7ff73a4dd60f] 2024-03-28T00:32:34.147 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7ff73a5b0e81] 2024-03-28T00:32:34.147 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7ff738fcb1ca] 2024-03-28T00:32:34.147 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: 19: clone() 2024-03-28T00:32:34.147 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T00:32:34.148 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: 2024-03-28T00:32:34.148 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: -1> 2024-03-28T00:32:33.819+0000 7ff72fc24700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7ff72fc24700 time 2024-03-28T00:32:33.818235+0000 2024-03-28T00:32:34.148 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T00:32:34.148 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: 2024-03-28T00:32:34.148 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:32:34.148 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7ff73a24e4a1] 2024-03-28T00:32:34.148 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7ff73a24e667] 2024-03-28T00:32:34.148 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55eb2ee89223] 2024-03-28T00:32:34.149 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55eb2ed035d6] 2024-03-28T00:32:34.149 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55eb2ed03da1] 2024-03-28T00:32:34.149 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: 9: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x55eb2ed3809c] 2024-03-28T00:32:34.149 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55eb2ed56233] 2024-03-28T00:32:34.149 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55eb2ed58197] 2024-03-28T00:32:34.149 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55eb2ec5711b] 2024-03-28T00:32:34.149 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55eb2ec576bc] 2024-03-28T00:32:34.149 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55eb2ec411ff] 2024-03-28T00:32:34.149 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7fbb9558d488] 2024-03-28T00:32:34.149 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: 16: (DispatchQueue::entry()+0x51f) [0x7fbb9558a60f] 2024-03-28T00:32:34.149 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fbb9565de81] 2024-03-28T00:32:34.149 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7fbb940781ca] 2024-03-28T00:32:34.150 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: 19: clone() 2024-03-28T00:32:34.150 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T00:32:34.150 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: 2024-03-28T00:32:34.150 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: -1> 2024-03-28T00:32:33.809+0000 7fbb8acd1700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7fbb8acd1700 time 2024-03-28T00:32:33.808481+0000 2024-03-28T00:32:34.150 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T00:32:34.150 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: 2024-03-28T00:32:34.150 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:32:34.150 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fbb952fb4a1] 2024-03-28T00:32:34.150 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7fbb952fb667] 2024-03-28T00:32:34.150 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55eb2ee89223] 2024-03-28T00:32:34.150 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55eb2ed035d6] 2024-03-28T00:32:34.150 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55eb2ed03da1] 2024-03-28T00:32:34.151 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: 6: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x55eb2ed3809c] 2024-03-28T00:32:34.151 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55eb2ed56233] 2024-03-28T00:32:34.151 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55eb2ed58197] 2024-03-28T00:32:34.151 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55eb2ec5711b] 2024-03-28T00:32:34.151 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55eb2ec576bc] 2024-03-28T00:32:34.151 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55eb2ec411ff] 2024-03-28T00:32:34.151 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7fbb9558d488] 2024-03-28T00:32:34.151 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: 13: (DispatchQueue::entry()+0x51f) [0x7fbb9558a60f] 2024-03-28T00:32:34.151 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fbb9565de81] 2024-03-28T00:32:34.151 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fbb940781ca] 2024-03-28T00:32:34.151 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: 16: clone() 2024-03-28T00:32:34.151 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: 2024-03-28T00:32:34.151 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: 0> 2024-03-28T00:32:33.811+0000 7fbb8acd1700 -1 *** Caught signal (Aborted) ** 2024-03-28T00:32:34.151 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: in thread 7fbb8acd1700 thread_name:ms_dispatch 2024-03-28T00:32:34.152 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: 2024-03-28T00:32:34.152 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:32:34.152 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fbb94082d20] 2024-03-28T00:32:34.152 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: 2: gsignal() 2024-03-28T00:32:34.152 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: 3: abort() 2024-03-28T00:32:34.152 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fbb952fb4fb] 2024-03-28T00:32:34.152 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7fbb952fb667] 2024-03-28T00:32:34.152 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55eb2ee89223] 2024-03-28T00:32:34.152 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55eb2ed035d6] 2024-03-28T00:32:34.152 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x562859a23223] 2024-03-28T00:32:34.152 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x56285989d5d6] 2024-03-28T00:32:34.153 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x56285989dda1] 2024-03-28T00:32:34.153 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: 6: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x5628598d209c] 2024-03-28T00:32:34.153 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x5628598f0233] 2024-03-28T00:32:34.153 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5628598f2197] 2024-03-28T00:32:34.153 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x5628597f111b] 2024-03-28T00:32:34.153 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5628597f16bc] 2024-03-28T00:32:34.153 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x5628597db1ff] 2024-03-28T00:32:34.153 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7ff73a4e0488] 2024-03-28T00:32:34.153 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: 13: (DispatchQueue::entry()+0x51f) [0x7ff73a4dd60f] 2024-03-28T00:32:34.153 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7ff73a5b0e81] 2024-03-28T00:32:34.153 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7ff738fcb1ca] 2024-03-28T00:32:34.153 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: 16: clone() 2024-03-28T00:32:34.153 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: 2024-03-28T00:32:34.154 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: 0> 2024-03-28T00:32:33.821+0000 7ff72fc24700 -1 *** Caught signal (Aborted) ** 2024-03-28T00:32:34.154 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: in thread 7ff72fc24700 thread_name:ms_dispatch 2024-03-28T00:32:34.154 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: 2024-03-28T00:32:34.154 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:32:34.154 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7ff738fd5d20] 2024-03-28T00:32:34.154 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: 2: gsignal() 2024-03-28T00:32:34.154 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: 3: abort() 2024-03-28T00:32:34.154 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7ff73a24e4fb] 2024-03-28T00:32:34.154 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7ff73a24e667] 2024-03-28T00:32:34.154 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x562859a23223] 2024-03-28T00:32:34.154 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x56285989d5d6] 2024-03-28T00:32:34.155 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55eb2ed03da1] 2024-03-28T00:32:34.155 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: 9: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x55eb2ed3809c] 2024-03-28T00:32:34.155 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55eb2ed56233] 2024-03-28T00:32:34.155 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55eb2ed58197] 2024-03-28T00:32:34.156 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55eb2ec5711b] 2024-03-28T00:32:34.156 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55eb2ec576bc] 2024-03-28T00:32:34.156 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55eb2ec411ff] 2024-03-28T00:32:34.156 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7fbb9558d488] 2024-03-28T00:32:34.156 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: 16: (DispatchQueue::entry()+0x51f) [0x7fbb9558a60f] 2024-03-28T00:32:34.156 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fbb9565de81] 2024-03-28T00:32:34.156 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7fbb940781ca] 2024-03-28T00:32:34.156 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: 19: clone() 2024-03-28T00:32:34.156 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T00:32:34.156 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: 2024-03-28T00:32:34.156 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: -1> 2024-03-28T00:32:33.809+0000 7fbb8acd1700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7fbb8acd1700 time 2024-03-28T00:32:33.808481+0000 2024-03-28T00:32:34.156 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T00:32:34.157 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: 2024-03-28T00:32:34.157 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:32:34.157 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fbb952fb4a1] 2024-03-28T00:32:34.157 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7fbb952fb667] 2024-03-28T00:32:34.157 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55eb2ee89223] 2024-03-28T00:32:34.157 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55eb2ed035d6] 2024-03-28T00:32:34.157 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55eb2ed03da1] 2024-03-28T00:32:34.157 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: 6: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x55eb2ed3809c] 2024-03-28T00:32:34.157 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55eb2ed56233] 2024-03-28T00:32:34.157 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55eb2ed58197] 2024-03-28T00:32:34.157 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55eb2ec5711b] 2024-03-28T00:32:34.157 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55eb2ec576bc] 2024-03-28T00:32:34.157 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55eb2ec411ff] 2024-03-28T00:32:34.158 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7fbb9558d488] 2024-03-28T00:32:34.158 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: 13: (DispatchQueue::entry()+0x51f) [0x7fbb9558a60f] 2024-03-28T00:32:34.158 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fbb9565de81] 2024-03-28T00:32:34.158 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fbb940781ca] 2024-03-28T00:32:34.158 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: 16: clone() 2024-03-28T00:32:34.158 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: 2024-03-28T00:32:34.158 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: 0> 2024-03-28T00:32:33.811+0000 7fbb8acd1700 -1 *** Caught signal (Aborted) ** 2024-03-28T00:32:34.158 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: in thread 7fbb8acd1700 thread_name:ms_dispatch 2024-03-28T00:32:34.158 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: 2024-03-28T00:32:34.158 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:32:34.158 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fbb94082d20] 2024-03-28T00:32:34.158 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: 2: gsignal() 2024-03-28T00:32:34.158 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: 3: abort() 2024-03-28T00:32:34.158 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fbb952fb4fb] 2024-03-28T00:32:34.158 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7fbb952fb667] 2024-03-28T00:32:34.159 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55eb2ee89223] 2024-03-28T00:32:34.159 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55eb2ed035d6] 2024-03-28T00:32:34.159 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55eb2ed03da1] 2024-03-28T00:32:34.159 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: 9: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x55eb2ed3809c] 2024-03-28T00:32:34.159 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55eb2ed56233] 2024-03-28T00:32:34.159 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55eb2ed58197] 2024-03-28T00:32:34.159 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55eb2ec5711b] 2024-03-28T00:32:34.159 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55eb2ec576bc] 2024-03-28T00:32:34.159 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55eb2ec411ff] 2024-03-28T00:32:34.159 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7fbb9558d488] 2024-03-28T00:32:34.159 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: 16: (DispatchQueue::entry()+0x51f) [0x7fbb9558a60f] 2024-03-28T00:32:34.159 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fbb9565de81] 2024-03-28T00:32:34.159 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7fbb940781ca] 2024-03-28T00:32:34.160 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: 19: clone() 2024-03-28T00:32:34.160 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T00:32:34.160 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[72800]: 2024-03-28T00:32:34.160 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:33 smithi165 podman[73038]: 2024-03-28 00:32:33.890507624 +0000 UTC m=+0.025770616 container died 16e6cd19bdbe3f6021e0dd2438bdb676a60653b43e40b67bf379124de1547c3b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux ) 2024-03-28T00:32:34.160 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x56285989dda1] 2024-03-28T00:32:34.160 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: 9: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x5628598d209c] 2024-03-28T00:32:34.161 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x5628598f0233] 2024-03-28T00:32:34.161 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5628598f2197] 2024-03-28T00:32:34.161 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x5628597f111b] 2024-03-28T00:32:34.161 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5628597f16bc] 2024-03-28T00:32:34.161 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x5628597db1ff] 2024-03-28T00:32:34.161 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7ff73a4e0488] 2024-03-28T00:32:34.161 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: 16: (DispatchQueue::entry()+0x51f) [0x7ff73a4dd60f] 2024-03-28T00:32:34.161 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7ff73a5b0e81] 2024-03-28T00:32:34.161 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7ff738fcb1ca] 2024-03-28T00:32:34.161 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: 19: clone() 2024-03-28T00:32:34.161 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T00:32:34.161 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: 2024-03-28T00:32:34.161 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: -1> 2024-03-28T00:32:33.819+0000 7ff72fc24700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7ff72fc24700 time 2024-03-28T00:32:33.818235+0000 2024-03-28T00:32:34.162 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T00:32:34.162 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: 2024-03-28T00:32:34.162 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:32:34.162 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7ff73a24e4a1] 2024-03-28T00:32:34.162 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7ff73a24e667] 2024-03-28T00:32:34.162 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x562859a23223] 2024-03-28T00:32:34.162 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x56285989d5d6] 2024-03-28T00:32:34.162 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x56285989dda1] 2024-03-28T00:32:34.162 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: 6: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x5628598d209c] 2024-03-28T00:32:34.162 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x5628598f0233] 2024-03-28T00:32:34.162 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5628598f2197] 2024-03-28T00:32:34.162 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x5628597f111b] 2024-03-28T00:32:34.162 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5628597f16bc] 2024-03-28T00:32:34.163 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x5628597db1ff] 2024-03-28T00:32:34.163 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7ff73a4e0488] 2024-03-28T00:32:34.163 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: 13: (DispatchQueue::entry()+0x51f) [0x7ff73a4dd60f] 2024-03-28T00:32:34.163 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7ff73a5b0e81] 2024-03-28T00:32:34.163 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7ff738fcb1ca] 2024-03-28T00:32:34.163 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: 16: clone() 2024-03-28T00:32:34.163 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: 2024-03-28T00:32:34.163 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: 0> 2024-03-28T00:32:33.821+0000 7ff72fc24700 -1 *** Caught signal (Aborted) ** 2024-03-28T00:32:34.163 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: in thread 7ff72fc24700 thread_name:ms_dispatch 2024-03-28T00:32:34.163 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: 2024-03-28T00:32:34.163 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:32:34.163 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7ff738fd5d20] 2024-03-28T00:32:34.163 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: 2: gsignal() 2024-03-28T00:32:34.163 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: 3: abort() 2024-03-28T00:32:34.163 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7ff73a24e4fb] 2024-03-28T00:32:34.164 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7ff73a24e667] 2024-03-28T00:32:34.164 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x562859a23223] 2024-03-28T00:32:34.164 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x56285989d5d6] 2024-03-28T00:32:34.164 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x56285989dda1] 2024-03-28T00:32:34.164 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: 9: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x5628598d209c] 2024-03-28T00:32:34.164 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x5628598f0233] 2024-03-28T00:32:34.164 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5628598f2197] 2024-03-28T00:32:34.164 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x5628597f111b] 2024-03-28T00:32:34.164 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5628597f16bc] 2024-03-28T00:32:34.164 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x5628597db1ff] 2024-03-28T00:32:34.164 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7ff73a4e0488] 2024-03-28T00:32:34.164 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: 16: (DispatchQueue::entry()+0x51f) [0x7ff73a4dd60f] 2024-03-28T00:32:34.164 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7ff73a5b0e81] 2024-03-28T00:32:34.165 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7ff738fcb1ca] 2024-03-28T00:32:34.165 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: 19: clone() 2024-03-28T00:32:34.165 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T00:32:34.165 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[72827]: 2024-03-28T00:32:34.165 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:32:33 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.165:6838/1687390659,v1:172.21.15.165:6839/1687390659] up:rejoin 2024-03-28T00:32:34.165 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:32:33 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.165:6832/4070507195,v1:172.21.15.165:6833/4070507195] up:rejoin 2024-03-28T00:32:34.165 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:32:33 smithi119 ceph-mon[31518]: fsmap cephfs:5/5 3 up:standby-replay 4 up:standby 2 up:rejoin 3 up:active 2024-03-28T00:32:34.165 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:32:33 smithi119 ceph-mon[31518]: daemon mds.l is now active in filesystem cephfs as rank 1 2024-03-28T00:32:34.166 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:32:33 smithi119 ceph-mon[31518]: daemon mds.i is now active in filesystem cephfs as rank 3 2024-03-28T00:32:34.166 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:32:33 smithi119 ceph-mon[31518]: pgmap v194: 97 pgs: 97 active+clean; 197 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 244 KiB/s wr, 63 op/s 2024-03-28T00:32:34.454 INFO:journalctl@ceph.mds.b.smithi132.stdout:Mar 28 00:32:34 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-b[66443]: 2024-03-28T00:32:34.041+0000 7f857cecc700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-03-28T00:32:34.454 INFO:journalctl@ceph.mds.b.smithi132.stdout:Mar 28 00:32:34 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-b[66443]: 2024-03-28T00:32:34.041+0000 7f857cecc700 -1 mds.pinger is_rank_lagging: rank=3 was never sent ping request. 2024-03-28T00:32:35.030 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:34 smithi165 podman[73044]: 2024-03-28 00:32:34.730551293 +0000 UTC m=+0.855138379 container died 59e8ab3b38abc37a920297b6e1fd2dcb0aa3604b5277ecc5ba031cb07f51975e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-03-28T00:32:35.031 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:32:34 smithi165 ceph-mon[36302]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-28T00:32:35.031 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:32:34 smithi165 ceph-mon[36302]: Cluster is now healthy 2024-03-28T00:32:35.031 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:32:34 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.165:6838/1687390659,v1:172.21.15.165:6839/1687390659] up:active 2024-03-28T00:32:35.031 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:32:34 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.165:6832/4070507195,v1:172.21.15.165:6833/4070507195] up:active 2024-03-28T00:32:35.031 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:32:34 smithi165 ceph-mon[36302]: fsmap cephfs:5 3 up:standby-replay 4 up:standby 5 up:active 2024-03-28T00:32:35.031 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:32:34 smithi165 ceph-mon[36302]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-28T00:32:35.031 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:34 smithi165 podman[73038]: 2024-03-28 00:32:34.62897887 +0000 UTC m=+0.764241845 container remove 16e6cd19bdbe3f6021e0dd2438bdb676a60653b43e40b67bf379124de1547c3b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-03-28T00:32:35.031 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:34 smithi165 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.i.service: Main process exited, code=exited, status=134/n/a 2024-03-28T00:32:35.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:32:34 smithi119 ceph-mon[31518]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-28T00:32:35.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:32:34 smithi119 ceph-mon[31518]: Cluster is now healthy 2024-03-28T00:32:35.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:32:34 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.165:6838/1687390659,v1:172.21.15.165:6839/1687390659] up:active 2024-03-28T00:32:35.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:32:34 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.165:6832/4070507195,v1:172.21.15.165:6833/4070507195] up:active 2024-03-28T00:32:35.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:32:34 smithi119 ceph-mon[31518]: fsmap cephfs:5 3 up:standby-replay 4 up:standby 5 up:active 2024-03-28T00:32:35.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:32:34 smithi119 ceph-mon[31518]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-28T00:32:35.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:32:34 smithi132 ceph-mon[35941]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-28T00:32:35.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:32:34 smithi132 ceph-mon[35941]: Cluster is now healthy 2024-03-28T00:32:35.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:32:34 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.165:6838/1687390659,v1:172.21.15.165:6839/1687390659] up:active 2024-03-28T00:32:35.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:32:34 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.165:6832/4070507195,v1:172.21.15.165:6833/4070507195] up:active 2024-03-28T00:32:35.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:32:34 smithi132 ceph-mon[35941]: fsmap cephfs:5 3 up:standby-replay 4 up:standby 5 up:active 2024-03-28T00:32:35.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:32:34 smithi132 ceph-mon[35941]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-28T00:32:35.626 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:35 smithi165 podman[73044]: 2024-03-28 00:32:35.342276576 +0000 UTC m=+1.466863639 container remove 59e8ab3b38abc37a920297b6e1fd2dcb0aa3604b5277ecc5ba031cb07f51975e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=) 2024-03-28T00:32:35.626 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:35 smithi165 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.l.service: Main process exited, code=exited, status=134/n/a 2024-03-28T00:32:35.626 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:35 smithi165 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.i.service: Failed with result 'exit-code'. 2024-03-28T00:32:36.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:32:35 smithi165 ceph-mon[36302]: pgmap v195: 97 pgs: 97 active+clean; 200 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 714 KiB/s rd, 538 KiB/s wr, 112 op/s 2024-03-28T00:32:36.134 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:35 smithi165 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.l.service: Failed with result 'exit-code'. 2024-03-28T00:32:36.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:32:35 smithi119 ceph-mon[31518]: pgmap v195: 97 pgs: 97 active+clean; 200 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 714 KiB/s rd, 538 KiB/s wr, 112 op/s 2024-03-28T00:32:36.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:32:35 smithi132 ceph-mon[35941]: pgmap v195: 97 pgs: 97 active+clean; 200 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 714 KiB/s rd, 538 KiB/s wr, 112 op/s 2024-03-28T00:32:38.454 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:32:38 smithi132 ceph-mon[35941]: pgmap v196: 97 pgs: 97 active+clean; 200 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 656 KiB/s rd, 495 KiB/s wr, 102 op/s 2024-03-28T00:32:38.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:32:38 smithi165 ceph-mon[36302]: pgmap v196: 97 pgs: 97 active+clean; 200 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 656 KiB/s rd, 495 KiB/s wr, 102 op/s 2024-03-28T00:32:38.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:32:38 smithi119 ceph-mon[31518]: pgmap v196: 97 pgs: 97 active+clean; 200 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 656 KiB/s rd, 495 KiB/s wr, 102 op/s 2024-03-28T00:32:40.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:32:40 smithi165 ceph-mon[36302]: pgmap v197: 97 pgs: 97 active+clean; 205 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1018 KiB/s wr, 200 op/s 2024-03-28T00:32:40.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:32:40 smithi119 ceph-mon[31518]: pgmap v197: 97 pgs: 97 active+clean; 205 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1018 KiB/s wr, 200 op/s 2024-03-28T00:32:40.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:32:40 smithi132 ceph-mon[35941]: pgmap v197: 97 pgs: 97 active+clean; 205 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1018 KiB/s wr, 200 op/s 2024-03-28T00:32:42.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:32:42 smithi165 ceph-mon[36302]: pgmap v198: 97 pgs: 97 active+clean; 205 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 852 KiB/s rd, 704 KiB/s wr, 128 op/s 2024-03-28T00:32:42.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:32:42 smithi119 ceph-mon[31518]: pgmap v198: 97 pgs: 97 active+clean; 205 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 852 KiB/s rd, 704 KiB/s wr, 128 op/s 2024-03-28T00:32:42.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:32:42 smithi132 ceph-mon[35941]: pgmap v198: 97 pgs: 97 active+clean; 205 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 852 KiB/s rd, 704 KiB/s wr, 128 op/s 2024-03-28T00:32:44.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:32:44 smithi165 ceph-mon[36302]: pgmap v199: 97 pgs: 97 active+clean; 205 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 852 KiB/s rd, 704 KiB/s wr, 128 op/s 2024-03-28T00:32:44.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:32:44 smithi119 ceph-mon[31518]: pgmap v199: 97 pgs: 97 active+clean; 205 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 852 KiB/s rd, 704 KiB/s wr, 128 op/s 2024-03-28T00:32:44.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:32:44 smithi132 ceph-mon[35941]: pgmap v199: 97 pgs: 97 active+clean; 205 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 852 KiB/s rd, 704 KiB/s wr, 128 op/s 2024-03-28T00:32:46.004 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:45 smithi165 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.l.service: Scheduled restart job, restart counter is at 4. 2024-03-28T00:32:46.004 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:45 smithi165 systemd[1]: Stopped Ceph mds.l for 10f73b92-ec98-11ee-95d0-87774f69a715. 2024-03-28T00:32:46.004 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:45 smithi165 systemd[1]: Starting Ceph mds.l for 10f73b92-ec98-11ee-95d0-87774f69a715... 2024-03-28T00:32:46.004 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:45 smithi165 podman[73223]: 2024-03-28 00:32:45.776520856 +0000 UTC m=+0.021273257 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-28T00:32:46.004 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:45 smithi165 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.i.service: Scheduled restart job, restart counter is at 3. 2024-03-28T00:32:46.004 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:45 smithi165 systemd[1]: Stopped Ceph mds.i for 10f73b92-ec98-11ee-95d0-87774f69a715. 2024-03-28T00:32:46.005 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:45 smithi165 systemd[1]: Starting Ceph mds.i for 10f73b92-ec98-11ee-95d0-87774f69a715... 2024-03-28T00:32:46.005 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:45 smithi165 podman[73217]: 2024-03-28 00:32:45.765915208 +0000 UTC m=+0.018894424 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-28T00:32:46.005 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:45 smithi165 podman[73217]: 2024-03-28 00:32:45.876517521 +0000 UTC m=+0.129496731 container create 62cc90d67cd903aa97960596ef5c9dd218d14d3633b5177338f3c3e6ebe4f8e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS) 2024-03-28T00:32:46.370 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:32:46 smithi165 ceph-mon[36302]: pgmap v200: 97 pgs: 97 active+clean; 205 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 854 KiB/s rd, 704 KiB/s wr, 134 op/s 2024-03-28T00:32:46.370 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:46 smithi165 podman[73223]: 2024-03-28 00:32:46.137182526 +0000 UTC m=+0.381934929 container create f1ef1cba450f73c0cfc14ae61e24411d780bd29d29489f3d16f809dcee086e6c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-28T00:32:46.634 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:46 smithi165 podman[73217]: 2024-03-28 00:32:46.467272094 +0000 UTC m=+0.720251312 container init 62cc90d67cd903aa97960596ef5c9dd218d14d3633b5177338f3c3e6ebe4f8e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-03-28T00:32:46.634 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:46 smithi165 podman[73217]: 2024-03-28 00:32:46.469764732 +0000 UTC m=+0.722743940 container start 62cc90d67cd903aa97960596ef5c9dd218d14d3633b5177338f3c3e6ebe4f8e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-03-28T00:32:46.634 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:46 smithi165 ceph-mds[73256]: -- 172.21.15.165:0/4082434309 <== mon.1 v2:172.21.15.165:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56018d677a40 con 0x56018d64ac00 2024-03-28T00:32:46.634 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:46 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: starting mds.i at 2024-03-28T00:32:46.634 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:46 smithi165 bash[73217]: 62cc90d67cd903aa97960596ef5c9dd218d14d3633b5177338f3c3e6ebe4f8e9 2024-03-28T00:32:46.634 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:32:46 smithi165 systemd[1]: Started Ceph mds.i for 10f73b92-ec98-11ee-95d0-87774f69a715. 2024-03-28T00:32:46.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:32:46 smithi119 ceph-mon[31518]: pgmap v200: 97 pgs: 97 active+clean; 205 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 854 KiB/s rd, 704 KiB/s wr, 134 op/s 2024-03-28T00:32:46.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:32:46 smithi132 ceph-mon[35941]: pgmap v200: 97 pgs: 97 active+clean; 205 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 854 KiB/s rd, 704 KiB/s wr, 134 op/s 2024-03-28T00:32:47.134 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:46 smithi165 podman[73223]: 2024-03-28 00:32:46.736268858 +0000 UTC m=+0.981021254 container init f1ef1cba450f73c0cfc14ae61e24411d780bd29d29489f3d16f809dcee086e6c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-03-28T00:32:47.134 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:46 smithi165 podman[73223]: 2024-03-28 00:32:46.739561253 +0000 UTC m=+0.984313650 container start f1ef1cba450f73c0cfc14ae61e24411d780bd29d29489f3d16f809dcee086e6c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-28T00:32:47.134 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:46 smithi165 ceph-mds[73277]: -- 172.21.15.165:0/1831136401 <== mon.1 v2:172.21.15.165:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55e68c425a40 con 0x55e68c3f8800 2024-03-28T00:32:47.134 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:46 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: starting mds.l at 2024-03-28T00:32:47.134 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:46 smithi165 bash[73223]: f1ef1cba450f73c0cfc14ae61e24411d780bd29d29489f3d16f809dcee086e6c 2024-03-28T00:32:47.134 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:32:46 smithi165 systemd[1]: Started Ceph mds.l for 10f73b92-ec98-11ee-95d0-87774f69a715. 2024-03-28T00:32:47.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:32:47 smithi165 ceph-mon[36302]: daemon mds.i restarted 2024-03-28T00:32:47.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:32:47 smithi165 ceph-mon[36302]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-28T00:32:47.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:32:47 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-28T00:32:47.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:32:47 smithi165 ceph-mon[36302]: osdmap e100: 12 total, 12 up, 12 in 2024-03-28T00:32:47.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:32:47 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.165:6832/3417761142,v1:172.21.15.165:6833/3417761142] up:boot 2024-03-28T00:32:47.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:32:47 smithi165 ceph-mon[36302]: Standby daemon mds.c assigned to filesystem cephfs as rank 3 2024-03-28T00:32:47.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:32:47 smithi165 ceph-mon[36302]: fsmap cephfs:4/5 {0=b=up:active,1=l=up:active,2=h=up:active,4=f=up:active} 5 up:standby-replay 3 up:standby, 1 failed 2024-03-28T00:32:47.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:32:47 smithi165 ceph-mon[36302]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:replay 4 up:active 2024-03-28T00:32:47.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:32:47 smithi165 ceph-mon[36302]: daemon mds.l restarted 2024-03-28T00:32:47.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:32:47 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-28T00:32:47.635 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:32:47 smithi165 ceph-mon[36302]: osdmap e101: 12 total, 12 up, 12 in 2024-03-28T00:32:47.635 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:32:47 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.165:6838/768712672,v1:172.21.15.165:6839/768712672] up:boot 2024-03-28T00:32:47.635 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:32:47 smithi165 ceph-mon[36302]: Standby daemon mds.j assigned to filesystem cephfs as rank 1 2024-03-28T00:32:47.635 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:32:47 smithi165 ceph-mon[36302]: fsmap cephfs:4/5 {0=b=up:active,2=h=up:active,3=c=up:replay,4=f=up:active} 4 up:standby-replay 4 up:standby, 1 failed 2024-03-28T00:32:47.635 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:32:47 smithi165 ceph-mon[36302]: fsmap cephfs:5/5 3 up:standby-replay 4 up:standby 2 up:replay 3 up:active 2024-03-28T00:32:47.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:32:47 smithi119 ceph-mon[31518]: daemon mds.i restarted 2024-03-28T00:32:47.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:32:47 smithi119 ceph-mon[31518]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-28T00:32:47.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:32:47 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-28T00:32:47.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:32:47 smithi119 ceph-mon[31518]: osdmap e100: 12 total, 12 up, 12 in 2024-03-28T00:32:47.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:32:47 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.165:6832/3417761142,v1:172.21.15.165:6833/3417761142] up:boot 2024-03-28T00:32:47.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:32:47 smithi119 ceph-mon[31518]: Standby daemon mds.c assigned to filesystem cephfs as rank 3 2024-03-28T00:32:47.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:32:47 smithi119 ceph-mon[31518]: fsmap cephfs:4/5 {0=b=up:active,1=l=up:active,2=h=up:active,4=f=up:active} 5 up:standby-replay 3 up:standby, 1 failed 2024-03-28T00:32:47.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:32:47 smithi119 ceph-mon[31518]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:replay 4 up:active 2024-03-28T00:32:47.658 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:32:47 smithi119 ceph-mon[31518]: daemon mds.l restarted 2024-03-28T00:32:47.658 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:32:47 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-28T00:32:47.658 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:32:47 smithi119 ceph-mon[31518]: osdmap e101: 12 total, 12 up, 12 in 2024-03-28T00:32:47.658 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:32:47 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.165:6838/768712672,v1:172.21.15.165:6839/768712672] up:boot 2024-03-28T00:32:47.658 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:32:47 smithi119 ceph-mon[31518]: Standby daemon mds.j assigned to filesystem cephfs as rank 1 2024-03-28T00:32:47.658 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:32:47 smithi119 ceph-mon[31518]: fsmap cephfs:4/5 {0=b=up:active,2=h=up:active,3=c=up:replay,4=f=up:active} 4 up:standby-replay 4 up:standby, 1 failed 2024-03-28T00:32:47.658 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:32:47 smithi119 ceph-mon[31518]: fsmap cephfs:5/5 3 up:standby-replay 4 up:standby 2 up:replay 3 up:active 2024-03-28T00:32:47.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:32:47 smithi132 ceph-mon[35941]: daemon mds.i restarted 2024-03-28T00:32:47.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:32:47 smithi132 ceph-mon[35941]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-28T00:32:47.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:32:47 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-28T00:32:47.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:32:47 smithi132 ceph-mon[35941]: osdmap e100: 12 total, 12 up, 12 in 2024-03-28T00:32:47.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:32:47 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.165:6832/3417761142,v1:172.21.15.165:6833/3417761142] up:boot 2024-03-28T00:32:47.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:32:47 smithi132 ceph-mon[35941]: Standby daemon mds.c assigned to filesystem cephfs as rank 3 2024-03-28T00:32:47.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:32:47 smithi132 ceph-mon[35941]: fsmap cephfs:4/5 {0=b=up:active,1=l=up:active,2=h=up:active,4=f=up:active} 5 up:standby-replay 3 up:standby, 1 failed 2024-03-28T00:32:47.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:32:47 smithi132 ceph-mon[35941]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:replay 4 up:active 2024-03-28T00:32:47.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:32:47 smithi132 ceph-mon[35941]: daemon mds.l restarted 2024-03-28T00:32:47.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:32:47 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-28T00:32:47.705 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:32:47 smithi132 ceph-mon[35941]: osdmap e101: 12 total, 12 up, 12 in 2024-03-28T00:32:47.705 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:32:47 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.165:6838/768712672,v1:172.21.15.165:6839/768712672] up:boot 2024-03-28T00:32:47.705 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:32:47 smithi132 ceph-mon[35941]: Standby daemon mds.j assigned to filesystem cephfs as rank 1 2024-03-28T00:32:47.705 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:32:47 smithi132 ceph-mon[35941]: fsmap cephfs:4/5 {0=b=up:active,2=h=up:active,3=c=up:replay,4=f=up:active} 4 up:standby-replay 4 up:standby, 1 failed 2024-03-28T00:32:47.705 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:32:47 smithi132 ceph-mon[35941]: fsmap cephfs:5/5 3 up:standby-replay 4 up:standby 2 up:replay 3 up:active 2024-03-28T00:32:48.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:32:48 smithi165 ceph-mon[36302]: pgmap v203: 97 pgs: 97 active+clean; 205 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 8 op/s 2024-03-28T00:32:48.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:32:48 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.119:6834/2210305461,v1:172.21.15.119:6835/2210305461] up:resolve 2024-03-28T00:32:48.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:32:48 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.165:6836/2717860763,v1:172.21.15.165:6837/2717860763] up:resolve 2024-03-28T00:32:48.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:32:48 smithi165 ceph-mon[36302]: fsmap cephfs:5/5 3 up:standby-replay 4 up:standby 2 up:resolve 3 up:active 2024-03-28T00:32:48.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:32:48 smithi119 ceph-mon[31518]: pgmap v203: 97 pgs: 97 active+clean; 205 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 8 op/s 2024-03-28T00:32:48.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:32:48 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.119:6834/2210305461,v1:172.21.15.119:6835/2210305461] up:resolve 2024-03-28T00:32:48.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:32:48 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.165:6836/2717860763,v1:172.21.15.165:6837/2717860763] up:resolve 2024-03-28T00:32:48.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:32:48 smithi119 ceph-mon[31518]: fsmap cephfs:5/5 3 up:standby-replay 4 up:standby 2 up:resolve 3 up:active 2024-03-28T00:32:48.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:32:48 smithi132 ceph-mon[35941]: pgmap v203: 97 pgs: 97 active+clean; 205 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 8 op/s 2024-03-28T00:32:48.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:32:48 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.119:6834/2210305461,v1:172.21.15.119:6835/2210305461] up:resolve 2024-03-28T00:32:48.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:32:48 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.165:6836/2717860763,v1:172.21.15.165:6837/2717860763] up:resolve 2024-03-28T00:32:48.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:32:48 smithi132 ceph-mon[35941]: fsmap cephfs:5/5 3 up:standby-replay 4 up:standby 2 up:resolve 3 up:active 2024-03-28T00:32:50.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:32:49 smithi165 ceph-mon[36302]: reconnect by client.25355 172.21.15.119:0/3071152482 after 0.00199997 2024-03-28T00:32:50.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:32:49 smithi165 ceph-mon[36302]: reconnect by client.25355 172.21.15.119:0/3071152482 after 0.00299993 2024-03-28T00:32:50.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:32:49 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.165:6836/2717860763,v1:172.21.15.165:6837/2717860763] up:reconnect 2024-03-28T00:32:50.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:32:49 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.119:6834/2210305461,v1:172.21.15.119:6835/2210305461] up:reconnect 2024-03-28T00:32:50.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:32:49 smithi165 ceph-mon[36302]: fsmap cephfs:5/5 3 up:standby-replay 4 up:standby 2 up:reconnect 3 up:active 2024-03-28T00:32:50.135 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:32:49 smithi165 ceph-mon[36302]: reconnect by client.25264 v1:172.21.15.119:0/3863511511 after 0.0179997 2024-03-28T00:32:50.135 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:32:49 smithi165 ceph-mon[36302]: reconnect by client.25264 v1:172.21.15.119:0/3863511511 after 0.0179996 2024-03-28T00:32:50.135 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:32:49 smithi165 ceph-mon[36302]: pgmap v204: 97 pgs: 97 active+clean; 205 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 0 B/s wr, 16 op/s 2024-03-28T00:32:50.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:32:49 smithi119 ceph-mon[31518]: reconnect by client.25355 172.21.15.119:0/3071152482 after 0.00199997 2024-03-28T00:32:50.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:32:49 smithi119 ceph-mon[31518]: reconnect by client.25355 172.21.15.119:0/3071152482 after 0.00299993 2024-03-28T00:32:50.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:32:49 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.165:6836/2717860763,v1:172.21.15.165:6837/2717860763] up:reconnect 2024-03-28T00:32:50.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:32:49 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.119:6834/2210305461,v1:172.21.15.119:6835/2210305461] up:reconnect 2024-03-28T00:32:50.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:32:49 smithi119 ceph-mon[31518]: fsmap cephfs:5/5 3 up:standby-replay 4 up:standby 2 up:reconnect 3 up:active 2024-03-28T00:32:50.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:32:49 smithi119 ceph-mon[31518]: reconnect by client.25264 v1:172.21.15.119:0/3863511511 after 0.0179997 2024-03-28T00:32:50.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:32:49 smithi119 ceph-mon[31518]: reconnect by client.25264 v1:172.21.15.119:0/3863511511 after 0.0179996 2024-03-28T00:32:50.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:32:49 smithi119 ceph-mon[31518]: pgmap v204: 97 pgs: 97 active+clean; 205 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 0 B/s wr, 16 op/s 2024-03-28T00:32:50.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:32:49 smithi132 ceph-mon[35941]: reconnect by client.25355 172.21.15.119:0/3071152482 after 0.00199997 2024-03-28T00:32:50.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:32:49 smithi132 ceph-mon[35941]: reconnect by client.25355 172.21.15.119:0/3071152482 after 0.00299993 2024-03-28T00:32:50.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:32:49 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.165:6836/2717860763,v1:172.21.15.165:6837/2717860763] up:reconnect 2024-03-28T00:32:50.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:32:49 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.119:6834/2210305461,v1:172.21.15.119:6835/2210305461] up:reconnect 2024-03-28T00:32:50.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:32:49 smithi132 ceph-mon[35941]: fsmap cephfs:5/5 3 up:standby-replay 4 up:standby 2 up:reconnect 3 up:active 2024-03-28T00:32:50.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:32:49 smithi132 ceph-mon[35941]: reconnect by client.25264 v1:172.21.15.119:0/3863511511 after 0.0179997 2024-03-28T00:32:50.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:32:49 smithi132 ceph-mon[35941]: reconnect by client.25264 v1:172.21.15.119:0/3863511511 after 0.0179996 2024-03-28T00:32:50.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:32:49 smithi132 ceph-mon[35941]: pgmap v204: 97 pgs: 97 active+clean; 205 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 0 B/s wr, 16 op/s 2024-03-28T00:32:51.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.165:6836/2717860763,v1:172.21.15.165:6837/2717860763] up:rejoin 2024-03-28T00:32:51.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.119:6834/2210305461,v1:172.21.15.119:6835/2210305461] up:rejoin 2024-03-28T00:32:51.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-mon[36302]: fsmap cephfs:5/5 3 up:standby-replay 4 up:standby 2 up:rejoin 3 up:active 2024-03-28T00:32:51.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-mon[36302]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-03-28T00:32:51.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-mon[36302]: daemon mds.j is now active in filesystem cephfs as rank 1 2024-03-28T00:32:51.135 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7f93adc5a700 time 2024-03-28T00:32:50.871689+0000 2024-03-28T00:32:51.135 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T00:32:51.135 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:32:51.135 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f93b82844a1] 2024-03-28T00:32:51.135 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f93b8284667] 2024-03-28T00:32:51.135 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55ad9d475223] 2024-03-28T00:32:51.135 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55ad9d2ef5d6] 2024-03-28T00:32:51.135 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55ad9d2efda1] 2024-03-28T00:32:51.135 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: 6: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x55ad9d32409c] 2024-03-28T00:32:51.135 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55ad9d342233] 2024-03-28T00:32:51.136 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55ad9d344197] 2024-03-28T00:32:51.136 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55ad9d24311b] 2024-03-28T00:32:51.136 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55ad9d2436bc] 2024-03-28T00:32:51.136 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55ad9d22d1ff] 2024-03-28T00:32:51.136 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f93b8516488] 2024-03-28T00:32:51.136 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: 13: (DispatchQueue::entry()+0x51f) [0x7f93b851360f] 2024-03-28T00:32:51.136 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f93b85e6e81] 2024-03-28T00:32:51.136 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f93b70011ca] 2024-03-28T00:32:51.136 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: 16: clone() 2024-03-28T00:32:51.136 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: 2024-03-28T00:32:50.872+0000 7f93adc5a700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7f93adc5a700 time 2024-03-28T00:32:50.871689+0000 2024-03-28T00:32:51.136 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T00:32:51.136 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: 2024-03-28T00:32:51.136 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:32:51.137 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f93b82844a1] 2024-03-28T00:32:51.137 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f93b8284667] 2024-03-28T00:32:51.137 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55ad9d475223] 2024-03-28T00:32:51.137 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55ad9d2ef5d6] 2024-03-28T00:32:51.137 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55ad9d2efda1] 2024-03-28T00:32:51.137 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: 6: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x55ad9d32409c] 2024-03-28T00:32:51.137 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55ad9d342233] 2024-03-28T00:32:51.137 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55ad9d344197] 2024-03-28T00:32:51.137 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55ad9d24311b] 2024-03-28T00:32:51.137 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55ad9d2436bc] 2024-03-28T00:32:51.137 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55ad9d22d1ff] 2024-03-28T00:32:51.137 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f93b8516488] 2024-03-28T00:32:51.137 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: 13: (DispatchQueue::entry()+0x51f) [0x7f93b851360f] 2024-03-28T00:32:51.137 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f93b85e6e81] 2024-03-28T00:32:51.137 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f93b70011ca] 2024-03-28T00:32:51.137 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: 16: clone() 2024-03-28T00:32:51.138 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: 2024-03-28T00:32:51.138 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: *** Caught signal (Aborted) ** 2024-03-28T00:32:51.138 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: in thread 7f93adc5a700 thread_name:ms_dispatch 2024-03-28T00:32:51.138 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:32:51.138 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f93b700bd20] 2024-03-28T00:32:51.138 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: 2: gsignal() 2024-03-28T00:32:51.138 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: 3: abort() 2024-03-28T00:32:51.138 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f93b82844fb] 2024-03-28T00:32:51.138 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f93b8284667] 2024-03-28T00:32:51.138 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55ad9d475223] 2024-03-28T00:32:51.138 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55ad9d2ef5d6] 2024-03-28T00:32:51.138 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55ad9d2efda1] 2024-03-28T00:32:51.139 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: 9: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x55ad9d32409c] 2024-03-28T00:32:51.139 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55ad9d342233] 2024-03-28T00:32:51.139 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55ad9d344197] 2024-03-28T00:32:51.139 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55ad9d24311b] 2024-03-28T00:32:51.139 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55ad9d2436bc] 2024-03-28T00:32:51.139 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55ad9d22d1ff] 2024-03-28T00:32:51.139 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f93b8516488] 2024-03-28T00:32:51.139 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: 16: (DispatchQueue::entry()+0x51f) [0x7f93b851360f] 2024-03-28T00:32:51.139 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f93b85e6e81] 2024-03-28T00:32:51.139 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f93b70011ca] 2024-03-28T00:32:51.139 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: 19: clone() 2024-03-28T00:32:51.139 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: 2024-03-28T00:32:50.875+0000 7f93adc5a700 -1 *** Caught signal (Aborted) ** 2024-03-28T00:32:51.139 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: in thread 7f93adc5a700 thread_name:ms_dispatch 2024-03-28T00:32:51.139 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: 2024-03-28T00:32:51.139 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:32:51.140 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f93b700bd20] 2024-03-28T00:32:51.140 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: 2: gsignal() 2024-03-28T00:32:51.140 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: 3: abort() 2024-03-28T00:32:51.140 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f93b82844fb] 2024-03-28T00:32:51.140 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f93b8284667] 2024-03-28T00:32:51.141 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55ad9d475223] 2024-03-28T00:32:51.141 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55ad9d2ef5d6] 2024-03-28T00:32:51.141 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55ad9d2efda1] 2024-03-28T00:32:51.141 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: 9: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x55ad9d32409c] 2024-03-28T00:32:51.141 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55ad9d342233] 2024-03-28T00:32:51.141 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55ad9d344197] 2024-03-28T00:32:51.141 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55ad9d24311b] 2024-03-28T00:32:51.141 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55ad9d2436bc] 2024-03-28T00:32:51.141 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55ad9d22d1ff] 2024-03-28T00:32:51.141 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f93b8516488] 2024-03-28T00:32:51.142 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: 16: (DispatchQueue::entry()+0x51f) [0x7f93b851360f] 2024-03-28T00:32:51.142 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f93b85e6e81] 2024-03-28T00:32:51.142 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f93b70011ca] 2024-03-28T00:32:51.142 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: 19: clone() 2024-03-28T00:32:51.142 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T00:32:51.142 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: 2024-03-28T00:32:51.142 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: -1> 2024-03-28T00:32:50.872+0000 7f93adc5a700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7f93adc5a700 time 2024-03-28T00:32:50.871689+0000 2024-03-28T00:32:51.142 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T00:32:51.142 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: 2024-03-28T00:32:51.142 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:32:51.142 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f93b82844a1] 2024-03-28T00:32:51.142 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f93b8284667] 2024-03-28T00:32:51.142 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55ad9d475223] 2024-03-28T00:32:51.143 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55ad9d2ef5d6] 2024-03-28T00:32:51.143 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55ad9d2efda1] 2024-03-28T00:32:51.143 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: 6: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x55ad9d32409c] 2024-03-28T00:32:51.143 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55ad9d342233] 2024-03-28T00:32:51.143 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55ad9d344197] 2024-03-28T00:32:51.143 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55ad9d24311b] 2024-03-28T00:32:51.143 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55ad9d2436bc] 2024-03-28T00:32:51.143 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55ad9d22d1ff] 2024-03-28T00:32:51.143 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f93b8516488] 2024-03-28T00:32:51.143 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: 13: (DispatchQueue::entry()+0x51f) [0x7f93b851360f] 2024-03-28T00:32:51.143 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f93b85e6e81] 2024-03-28T00:32:51.143 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f93b70011ca] 2024-03-28T00:32:51.143 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: 16: clone() 2024-03-28T00:32:51.143 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: 2024-03-28T00:32:51.143 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: 0> 2024-03-28T00:32:50.875+0000 7f93adc5a700 -1 *** Caught signal (Aborted) ** 2024-03-28T00:32:51.144 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: in thread 7f93adc5a700 thread_name:ms_dispatch 2024-03-28T00:32:51.144 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: 2024-03-28T00:32:51.144 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:32:51.144 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f93b700bd20] 2024-03-28T00:32:51.144 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: 2: gsignal() 2024-03-28T00:32:51.144 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: 3: abort() 2024-03-28T00:32:51.144 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f93b82844fb] 2024-03-28T00:32:51.144 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f93b8284667] 2024-03-28T00:32:51.144 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55ad9d475223] 2024-03-28T00:32:51.144 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55ad9d2ef5d6] 2024-03-28T00:32:51.144 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55ad9d2efda1] 2024-03-28T00:32:51.144 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: 9: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x55ad9d32409c] 2024-03-28T00:32:51.144 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55ad9d342233] 2024-03-28T00:32:51.144 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55ad9d344197] 2024-03-28T00:32:51.145 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55ad9d24311b] 2024-03-28T00:32:51.145 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55ad9d2436bc] 2024-03-28T00:32:51.145 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55ad9d22d1ff] 2024-03-28T00:32:51.145 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f93b8516488] 2024-03-28T00:32:51.145 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: 16: (DispatchQueue::entry()+0x51f) [0x7f93b851360f] 2024-03-28T00:32:51.145 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f93b85e6e81] 2024-03-28T00:32:51.145 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f93b70011ca] 2024-03-28T00:32:51.145 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: 19: clone() 2024-03-28T00:32:51.145 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T00:32:51.145 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: 2024-03-28T00:32:51.145 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: -1> 2024-03-28T00:32:50.872+0000 7f93adc5a700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7f93adc5a700 time 2024-03-28T00:32:50.871689+0000 2024-03-28T00:32:51.145 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T00:32:51.145 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: 2024-03-28T00:32:51.145 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:32:51.146 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f93b82844a1] 2024-03-28T00:32:51.146 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f93b8284667] 2024-03-28T00:32:51.146 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55ad9d475223] 2024-03-28T00:32:51.146 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55ad9d2ef5d6] 2024-03-28T00:32:51.146 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55ad9d2efda1] 2024-03-28T00:32:51.146 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: 6: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x55ad9d32409c] 2024-03-28T00:32:51.146 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55ad9d342233] 2024-03-28T00:32:51.146 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55ad9d344197] 2024-03-28T00:32:51.146 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55ad9d24311b] 2024-03-28T00:32:51.146 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55ad9d2436bc] 2024-03-28T00:32:51.146 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55ad9d22d1ff] 2024-03-28T00:32:51.146 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f93b8516488] 2024-03-28T00:32:51.146 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: 13: (DispatchQueue::entry()+0x51f) [0x7f93b851360f] 2024-03-28T00:32:51.146 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f93b85e6e81] 2024-03-28T00:32:51.147 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f93b70011ca] 2024-03-28T00:32:51.147 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: 16: clone() 2024-03-28T00:32:51.147 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: 2024-03-28T00:32:51.147 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: 0> 2024-03-28T00:32:50.875+0000 7f93adc5a700 -1 *** Caught signal (Aborted) ** 2024-03-28T00:32:51.147 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: in thread 7f93adc5a700 thread_name:ms_dispatch 2024-03-28T00:32:51.147 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: 2024-03-28T00:32:51.147 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:32:51.147 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f93b700bd20] 2024-03-28T00:32:51.147 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: 2: gsignal() 2024-03-28T00:32:51.147 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: 3: abort() 2024-03-28T00:32:51.147 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f93b82844fb] 2024-03-28T00:32:51.147 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f93b8284667] 2024-03-28T00:32:51.147 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55ad9d475223] 2024-03-28T00:32:51.147 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55ad9d2ef5d6] 2024-03-28T00:32:51.148 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55ad9d2efda1] 2024-03-28T00:32:51.148 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: 9: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x55ad9d32409c] 2024-03-28T00:32:51.148 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55ad9d342233] 2024-03-28T00:32:51.148 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55ad9d344197] 2024-03-28T00:32:51.148 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55ad9d24311b] 2024-03-28T00:32:51.148 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55ad9d2436bc] 2024-03-28T00:32:51.148 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55ad9d22d1ff] 2024-03-28T00:32:51.148 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f93b8516488] 2024-03-28T00:32:51.148 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: 16: (DispatchQueue::entry()+0x51f) [0x7f93b851360f] 2024-03-28T00:32:51.148 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f93b85e6e81] 2024-03-28T00:32:51.148 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f93b70011ca] 2024-03-28T00:32:51.148 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: 19: clone() 2024-03-28T00:32:51.148 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T00:32:51.148 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[72997]: 2024-03-28T00:32:51.148 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:50 smithi165 podman[73322]: 2024-03-28 00:32:50.950811712 +0000 UTC m=+0.021232687 container died 18a29d5287c93c76aeae1908b4d59391676b3c1aff8a6a79caae94d3fea0062b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-03-28T00:32:51.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.165:6836/2717860763,v1:172.21.15.165:6837/2717860763] up:rejoin 2024-03-28T00:32:51.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.119:6834/2210305461,v1:172.21.15.119:6835/2210305461] up:rejoin 2024-03-28T00:32:51.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-mon[31518]: fsmap cephfs:5/5 3 up:standby-replay 4 up:standby 2 up:rejoin 3 up:active 2024-03-28T00:32:51.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-mon[31518]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-03-28T00:32:51.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-mon[31518]: daemon mds.j is now active in filesystem cephfs as rank 1 2024-03-28T00:32:51.157 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7fe40c200700 time 2024-03-28T00:32:50.884389+0000 2024-03-28T00:32:51.157 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T00:32:51.157 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:32:51.157 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fe417ab14a1] 2024-03-28T00:32:51.157 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7fe417ab1667] 2024-03-28T00:32:51.157 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x558ea446a223] 2024-03-28T00:32:51.158 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x558ea42e45d6] 2024-03-28T00:32:51.158 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x558ea42e4da1] 2024-03-28T00:32:51.158 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: 6: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x558ea431909c] 2024-03-28T00:32:51.158 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x558ea4337233] 2024-03-28T00:32:51.158 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x558ea4339197] 2024-03-28T00:32:51.158 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x558ea423811b] 2024-03-28T00:32:51.158 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x558ea42386bc] 2024-03-28T00:32:51.158 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x558ea42221ff] 2024-03-28T00:32:51.158 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7fe417d43488] 2024-03-28T00:32:51.158 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: 13: (DispatchQueue::entry()+0x51f) [0x7fe417d4060f] 2024-03-28T00:32:51.158 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fe417e13e81] 2024-03-28T00:32:51.158 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fe41682e1ca] 2024-03-28T00:32:51.159 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: 16: clone() 2024-03-28T00:32:51.159 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: 2024-03-28T00:32:50.886+0000 7fe40c200700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7fe40c200700 time 2024-03-28T00:32:50.884389+0000 2024-03-28T00:32:51.159 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T00:32:51.159 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: 2024-03-28T00:32:51.159 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:32:51.159 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fe417ab14a1] 2024-03-28T00:32:51.159 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7fe417ab1667] 2024-03-28T00:32:51.159 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x558ea446a223] 2024-03-28T00:32:51.159 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x558ea42e45d6] 2024-03-28T00:32:51.159 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x558ea42e4da1] 2024-03-28T00:32:51.159 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: 6: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x558ea431909c] 2024-03-28T00:32:51.159 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x558ea4337233] 2024-03-28T00:32:51.159 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x558ea4339197] 2024-03-28T00:32:51.160 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x558ea423811b] 2024-03-28T00:32:51.160 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x558ea42386bc] 2024-03-28T00:32:51.160 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x558ea42221ff] 2024-03-28T00:32:51.160 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7fe417d43488] 2024-03-28T00:32:51.160 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: 13: (DispatchQueue::entry()+0x51f) [0x7fe417d4060f] 2024-03-28T00:32:51.160 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fe417e13e81] 2024-03-28T00:32:51.160 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fe41682e1ca] 2024-03-28T00:32:51.160 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: 16: clone() 2024-03-28T00:32:51.160 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: 2024-03-28T00:32:51.160 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: *** Caught signal (Aborted) ** 2024-03-28T00:32:51.160 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: in thread 7fe40c200700 thread_name:ms_dispatch 2024-03-28T00:32:51.160 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:32:51.161 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fe416838d20] 2024-03-28T00:32:51.162 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: 2: gsignal() 2024-03-28T00:32:51.162 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: 3: abort() 2024-03-28T00:32:51.162 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fe417ab14fb] 2024-03-28T00:32:51.162 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7fe417ab1667] 2024-03-28T00:32:51.162 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x558ea446a223] 2024-03-28T00:32:51.162 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x558ea42e45d6] 2024-03-28T00:32:51.162 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x558ea42e4da1] 2024-03-28T00:32:51.162 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: 9: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x558ea431909c] 2024-03-28T00:32:51.162 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x558ea4337233] 2024-03-28T00:32:51.162 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x558ea4339197] 2024-03-28T00:32:51.163 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x558ea423811b] 2024-03-28T00:32:51.163 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x558ea42386bc] 2024-03-28T00:32:51.163 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x558ea42221ff] 2024-03-28T00:32:51.163 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7fe417d43488] 2024-03-28T00:32:51.163 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: 16: (DispatchQueue::entry()+0x51f) [0x7fe417d4060f] 2024-03-28T00:32:51.163 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fe417e13e81] 2024-03-28T00:32:51.163 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7fe41682e1ca] 2024-03-28T00:32:51.163 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: 19: clone() 2024-03-28T00:32:51.163 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: 2024-03-28T00:32:50.891+0000 7fe40c200700 -1 *** Caught signal (Aborted) ** 2024-03-28T00:32:51.163 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: in thread 7fe40c200700 thread_name:ms_dispatch 2024-03-28T00:32:51.163 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: 2024-03-28T00:32:51.163 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:32:51.163 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fe416838d20] 2024-03-28T00:32:51.163 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: 2: gsignal() 2024-03-28T00:32:51.164 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: 3: abort() 2024-03-28T00:32:51.164 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fe417ab14fb] 2024-03-28T00:32:51.164 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7fe417ab1667] 2024-03-28T00:32:51.164 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x558ea446a223] 2024-03-28T00:32:51.164 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x558ea42e45d6] 2024-03-28T00:32:51.164 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x558ea42e4da1] 2024-03-28T00:32:51.164 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: 9: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x558ea431909c] 2024-03-28T00:32:51.164 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x558ea4337233] 2024-03-28T00:32:51.164 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x558ea4339197] 2024-03-28T00:32:51.164 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x558ea423811b] 2024-03-28T00:32:51.164 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x558ea42386bc] 2024-03-28T00:32:51.164 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x558ea42221ff] 2024-03-28T00:32:51.164 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7fe417d43488] 2024-03-28T00:32:51.164 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: 16: (DispatchQueue::entry()+0x51f) [0x7fe417d4060f] 2024-03-28T00:32:51.165 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fe417e13e81] 2024-03-28T00:32:51.165 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7fe41682e1ca] 2024-03-28T00:32:51.165 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: 19: clone() 2024-03-28T00:32:51.165 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T00:32:51.165 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: 2024-03-28T00:32:51.165 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: -1> 2024-03-28T00:32:50.886+0000 7fe40c200700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7fe40c200700 time 2024-03-28T00:32:50.884389+0000 2024-03-28T00:32:51.165 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T00:32:51.165 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: 2024-03-28T00:32:51.165 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:32:51.165 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fe417ab14a1] 2024-03-28T00:32:51.166 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7fe417ab1667] 2024-03-28T00:32:51.166 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x558ea446a223] 2024-03-28T00:32:51.166 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x558ea42e45d6] 2024-03-28T00:32:51.166 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x558ea42e4da1] 2024-03-28T00:32:51.166 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: 6: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x558ea431909c] 2024-03-28T00:32:51.166 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x558ea4337233] 2024-03-28T00:32:51.166 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x558ea4339197] 2024-03-28T00:32:51.166 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x558ea423811b] 2024-03-28T00:32:51.166 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x558ea42386bc] 2024-03-28T00:32:51.166 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x558ea42221ff] 2024-03-28T00:32:51.166 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7fe417d43488] 2024-03-28T00:32:51.166 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: 13: (DispatchQueue::entry()+0x51f) [0x7fe417d4060f] 2024-03-28T00:32:51.166 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fe417e13e81] 2024-03-28T00:32:51.167 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fe41682e1ca] 2024-03-28T00:32:51.167 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: 16: clone() 2024-03-28T00:32:51.167 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: 2024-03-28T00:32:51.167 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: 0> 2024-03-28T00:32:50.891+0000 7fe40c200700 -1 *** Caught signal (Aborted) ** 2024-03-28T00:32:51.167 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: in thread 7fe40c200700 thread_name:ms_dispatch 2024-03-28T00:32:51.167 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: 2024-03-28T00:32:51.167 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:32:51.167 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fe416838d20] 2024-03-28T00:32:51.167 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: 2: gsignal() 2024-03-28T00:32:51.167 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: 3: abort() 2024-03-28T00:32:51.167 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fe417ab14fb] 2024-03-28T00:32:51.167 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7fe417ab1667] 2024-03-28T00:32:51.167 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x558ea446a223] 2024-03-28T00:32:51.167 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x558ea42e45d6] 2024-03-28T00:32:51.168 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x558ea42e4da1] 2024-03-28T00:32:51.168 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: 9: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x558ea431909c] 2024-03-28T00:32:51.168 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x558ea4337233] 2024-03-28T00:32:51.168 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x558ea4339197] 2024-03-28T00:32:51.168 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x558ea423811b] 2024-03-28T00:32:51.168 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x558ea42386bc] 2024-03-28T00:32:51.168 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x558ea42221ff] 2024-03-28T00:32:51.168 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7fe417d43488] 2024-03-28T00:32:51.168 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: 16: (DispatchQueue::entry()+0x51f) [0x7fe417d4060f] 2024-03-28T00:32:51.168 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fe417e13e81] 2024-03-28T00:32:51.168 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7fe41682e1ca] 2024-03-28T00:32:51.168 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: 19: clone() 2024-03-28T00:32:51.168 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T00:32:51.168 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: 2024-03-28T00:32:51.168 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: -8> 2024-03-28T00:32:50.886+0000 7fe40c200700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7fe40c200700 time 2024-03-28T00:32:50.884389+0000 2024-03-28T00:32:51.169 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T00:32:51.169 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: 2024-03-28T00:32:51.169 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:32:51.169 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fe417ab14a1] 2024-03-28T00:32:51.169 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7fe417ab1667] 2024-03-28T00:32:51.169 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x558ea446a223] 2024-03-28T00:32:51.169 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x558ea42e45d6] 2024-03-28T00:32:51.169 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x558ea42e4da1] 2024-03-28T00:32:51.169 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: 6: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x558ea431909c] 2024-03-28T00:32:51.169 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x558ea4337233] 2024-03-28T00:32:51.169 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x558ea4339197] 2024-03-28T00:32:51.169 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x558ea423811b] 2024-03-28T00:32:51.169 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x558ea42386bc] 2024-03-28T00:32:51.169 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x558ea42221ff] 2024-03-28T00:32:51.170 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7fe417d43488] 2024-03-28T00:32:51.170 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: 13: (DispatchQueue::entry()+0x51f) [0x7fe417d4060f] 2024-03-28T00:32:51.170 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fe417e13e81] 2024-03-28T00:32:51.170 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fe41682e1ca] 2024-03-28T00:32:51.170 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: 16: clone() 2024-03-28T00:32:51.170 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: 2024-03-28T00:32:51.170 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: -7> 2024-03-28T00:32:50.891+0000 7fe40c200700 -1 *** Caught signal (Aborted) ** 2024-03-28T00:32:51.170 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: in thread 7fe40c200700 thread_name:ms_dispatch 2024-03-28T00:32:51.170 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: 2024-03-28T00:32:51.170 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:32:51.170 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fe416838d20] 2024-03-28T00:32:51.170 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: 2: gsignal() 2024-03-28T00:32:51.170 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: 3: abort() 2024-03-28T00:32:51.170 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fe417ab14fb] 2024-03-28T00:32:51.170 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7fe417ab1667] 2024-03-28T00:32:51.171 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x558ea446a223] 2024-03-28T00:32:51.171 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x558ea42e45d6] 2024-03-28T00:32:51.171 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x558ea42e4da1] 2024-03-28T00:32:51.171 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: 9: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x558ea431909c] 2024-03-28T00:32:51.171 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x558ea4337233] 2024-03-28T00:32:51.171 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x558ea4339197] 2024-03-28T00:32:51.171 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x558ea423811b] 2024-03-28T00:32:51.171 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x558ea42386bc] 2024-03-28T00:32:51.171 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x558ea42221ff] 2024-03-28T00:32:51.171 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7fe417d43488] 2024-03-28T00:32:51.171 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: 16: (DispatchQueue::entry()+0x51f) [0x7fe417d4060f] 2024-03-28T00:32:51.171 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fe417e13e81] 2024-03-28T00:32:51.171 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7fe41682e1ca] 2024-03-28T00:32:51.171 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: 19: clone() 2024-03-28T00:32:51.172 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T00:32:51.172 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[109623]: 2024-03-28T00:32:51.172 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:51 smithi119 podman[111945]: 2024-03-28 00:32:51.032340178 +0000 UTC m=+0.033660069 container died 6b6443e2c8ef702998870d58751aba8bc5f0174c06884c32bed3aba87f6a635e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-28T00:32:51.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:32:50 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.165:6836/2717860763,v1:172.21.15.165:6837/2717860763] up:rejoin 2024-03-28T00:32:51.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:32:50 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.119:6834/2210305461,v1:172.21.15.119:6835/2210305461] up:rejoin 2024-03-28T00:32:51.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:32:50 smithi132 ceph-mon[35941]: fsmap cephfs:5/5 3 up:standby-replay 4 up:standby 2 up:rejoin 3 up:active 2024-03-28T00:32:51.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:32:50 smithi132 ceph-mon[35941]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-03-28T00:32:51.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:32:50 smithi132 ceph-mon[35941]: daemon mds.j is now active in filesystem cephfs as rank 1 2024-03-28T00:32:51.980 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:51 smithi165 podman[73322]: 2024-03-28 00:32:51.716298239 +0000 UTC m=+0.786719196 container remove 18a29d5287c93c76aeae1908b4d59391676b3c1aff8a6a79caae94d3fea0062b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-03-28T00:32:51.980 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:51 smithi165 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.c.service: Main process exited, code=exited, status=134/n/a 2024-03-28T00:32:51.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:32:51 smithi165 ceph-mon[36302]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-28T00:32:51.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:32:51 smithi165 ceph-mon[36302]: Cluster is now healthy 2024-03-28T00:32:51.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:32:51 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.165:6836/2717860763,v1:172.21.15.165:6837/2717860763] up:active 2024-03-28T00:32:51.981 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:32:51 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.119:6834/2210305461,v1:172.21.15.119:6835/2210305461] up:active 2024-03-28T00:32:51.981 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:32:51 smithi165 ceph-mon[36302]: fsmap cephfs:5 3 up:standby-replay 4 up:standby 5 up:active 2024-03-28T00:32:51.981 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:32:51 smithi165 ceph-mon[36302]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-28T00:32:51.981 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:32:51 smithi165 ceph-mon[36302]: pgmap v205: 97 pgs: 97 active+clean; 205 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 0 B/s wr, 16 op/s 2024-03-28T00:32:52.081 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:51 smithi119 podman[111945]: 2024-03-28 00:32:51.753659572 +0000 UTC m=+0.754979455 container remove 6b6443e2c8ef702998870d58751aba8bc5f0174c06884c32bed3aba87f6a635e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319) 2024-03-28T00:32:52.081 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:51 smithi119 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.j.service: Main process exited, code=exited, status=134/n/a 2024-03-28T00:32:52.081 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:32:51 smithi119 ceph-mon[31518]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-28T00:32:52.081 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:32:51 smithi119 ceph-mon[31518]: Cluster is now healthy 2024-03-28T00:32:52.081 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:32:51 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.165:6836/2717860763,v1:172.21.15.165:6837/2717860763] up:active 2024-03-28T00:32:52.081 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:32:51 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.119:6834/2210305461,v1:172.21.15.119:6835/2210305461] up:active 2024-03-28T00:32:52.081 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:32:51 smithi119 ceph-mon[31518]: fsmap cephfs:5 3 up:standby-replay 4 up:standby 5 up:active 2024-03-28T00:32:52.082 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:32:51 smithi119 ceph-mon[31518]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-28T00:32:52.082 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:32:51 smithi119 ceph-mon[31518]: pgmap v205: 97 pgs: 97 active+clean; 205 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 0 B/s wr, 16 op/s 2024-03-28T00:32:52.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:32:51 smithi132 ceph-mon[35941]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-28T00:32:52.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:32:51 smithi132 ceph-mon[35941]: Cluster is now healthy 2024-03-28T00:32:52.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:32:51 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.165:6836/2717860763,v1:172.21.15.165:6837/2717860763] up:active 2024-03-28T00:32:52.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:32:51 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.119:6834/2210305461,v1:172.21.15.119:6835/2210305461] up:active 2024-03-28T00:32:52.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:32:51 smithi132 ceph-mon[35941]: fsmap cephfs:5 3 up:standby-replay 4 up:standby 5 up:active 2024-03-28T00:32:52.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:32:51 smithi132 ceph-mon[35941]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-28T00:32:52.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:32:51 smithi132 ceph-mon[35941]: pgmap v205: 97 pgs: 97 active+clean; 205 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 0 B/s wr, 16 op/s 2024-03-28T00:32:52.384 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:32:51 smithi165 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.c.service: Failed with result 'exit-code'. 2024-03-28T00:32:52.406 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:52 smithi119 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.j.service: Failed with result 'exit-code'. 2024-03-28T00:32:52.406 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:32:52 smithi119 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.j.service: Consumed 1.395s CPU time. 2024-03-28T00:32:54.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:32:54 smithi119 ceph-mon[31518]: pgmap v206: 97 pgs: 97 active+clean; 205 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 0 B/s wr, 8 op/s 2024-03-28T00:32:54.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:32:54 smithi165 ceph-mon[36302]: pgmap v206: 97 pgs: 97 active+clean; 205 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 0 B/s wr, 8 op/s 2024-03-28T00:32:54.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:32:54 smithi132 ceph-mon[35941]: pgmap v206: 97 pgs: 97 active+clean; 205 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 0 B/s wr, 8 op/s 2024-03-28T00:32:56.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:32:56 smithi165 ceph-mon[36302]: pgmap v207: 97 pgs: 97 active+clean; 211 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 931 KiB/s rd, 859 KiB/s wr, 222 op/s 2024-03-28T00:32:56.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:32:56 smithi119 ceph-mon[31518]: pgmap v207: 97 pgs: 97 active+clean; 211 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 931 KiB/s rd, 859 KiB/s wr, 222 op/s 2024-03-28T00:32:56.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:32:56 smithi132 ceph-mon[35941]: pgmap v207: 97 pgs: 97 active+clean; 211 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 931 KiB/s rd, 859 KiB/s wr, 222 op/s 2024-03-28T00:32:58.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:32:58 smithi165 ceph-mon[36302]: pgmap v208: 97 pgs: 97 active+clean; 211 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 775 KiB/s rd, 715 KiB/s wr, 185 op/s 2024-03-28T00:32:58.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:32:58 smithi119 ceph-mon[31518]: pgmap v208: 97 pgs: 97 active+clean; 211 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 775 KiB/s rd, 715 KiB/s wr, 185 op/s 2024-03-28T00:32:58.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:32:58 smithi132 ceph-mon[35941]: pgmap v208: 97 pgs: 97 active+clean; 211 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 775 KiB/s rd, 715 KiB/s wr, 185 op/s 2024-03-28T00:33:00.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:33:00 smithi165 ceph-mon[36302]: pgmap v209: 97 pgs: 97 active+clean; 212 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 863 KiB/s rd, 732 KiB/s wr, 190 op/s 2024-03-28T00:33:00.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:33:00 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:33:00.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:33:00 smithi119 ceph-mon[31518]: pgmap v209: 97 pgs: 97 active+clean; 212 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 863 KiB/s rd, 732 KiB/s wr, 190 op/s 2024-03-28T00:33:00.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:33:00 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:33:00.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:33:00 smithi132 ceph-mon[35941]: pgmap v209: 97 pgs: 97 active+clean; 212 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 863 KiB/s rd, 732 KiB/s wr, 190 op/s 2024-03-28T00:33:00.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:33:00 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:33:01.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:33:01 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:33:01.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:33:01 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:33:01.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:33:01 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:33:01.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:33:01 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:33:01.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:33:01 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:33:01.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:33:01 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:33:01.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:33:01 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:33:01.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:33:01 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:33:01.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:33:01 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:33:02.389 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:33:02 smithi165 ceph-mon[36302]: pgmap v210: 97 pgs: 97 active+clean; 212 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 860 KiB/s rd, 732 KiB/s wr, 184 op/s 2024-03-28T00:33:02.389 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:02 smithi165 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.c.service: Scheduled restart job, restart counter is at 5. 2024-03-28T00:33:02.390 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:02 smithi165 systemd[1]: Stopped Ceph mds.c for 10f73b92-ec98-11ee-95d0-87774f69a715. 2024-03-28T00:33:02.390 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:02 smithi165 systemd[1]: Starting Ceph mds.c for 10f73b92-ec98-11ee-95d0-87774f69a715... 2024-03-28T00:33:02.390 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:02 smithi165 podman[73549]: 2024-03-28 00:33:02.264545886 +0000 UTC m=+0.019767944 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-28T00:33:02.550 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:33:02 smithi119 ceph-mon[31518]: pgmap v210: 97 pgs: 97 active+clean; 212 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 860 KiB/s rd, 732 KiB/s wr, 184 op/s 2024-03-28T00:33:02.550 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:02 smithi119 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.j.service: Scheduled restart job, restart counter is at 3. 2024-03-28T00:33:02.551 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:02 smithi119 systemd[1]: Stopped Ceph mds.j for 10f73b92-ec98-11ee-95d0-87774f69a715. 2024-03-28T00:33:02.551 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:02 smithi119 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.j.service: Consumed 1.395s CPU time. 2024-03-28T00:33:02.551 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:02 smithi119 systemd[1]: Starting Ceph mds.j for 10f73b92-ec98-11ee-95d0-87774f69a715... 2024-03-28T00:33:02.605 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:33:02 smithi132 ceph-mon[35941]: pgmap v210: 97 pgs: 97 active+clean; 212 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 860 KiB/s rd, 732 KiB/s wr, 184 op/s 2024-03-28T00:33:02.667 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:02 smithi165 podman[73549]: 2024-03-28 00:33:02.383272663 +0000 UTC m=+0.138494713 container create e50f2f72ad4921599196dc06f38bf8e6166fd6c2714c6986f461bc803fd924a9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-03-28T00:33:02.882 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:02 smithi119 podman[113746]: 2024-03-28 00:33:02.450149537 +0000 UTC m=+0.037478967 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-28T00:33:02.882 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:02 smithi119 podman[113746]: 2024-03-28 00:33:02.59677254 +0000 UTC m=+0.184102011 container create f519af739077e8d2209d74929537c0431b390e73dc83b35975867db4429386fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-03-28T00:33:03.134 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:02 smithi165 podman[73549]: 2024-03-28 00:33:02.660853251 +0000 UTC m=+0.416075302 container init e50f2f72ad4921599196dc06f38bf8e6166fd6c2714c6986f461bc803fd924a9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c) 2024-03-28T00:33:03.134 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:02 smithi165 podman[73549]: 2024-03-28 00:33:02.663429442 +0000 UTC m=+0.418651493 container start e50f2f72ad4921599196dc06f38bf8e6166fd6c2714c6986f461bc803fd924a9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-03-28T00:33:03.134 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:02 smithi165 ceph-mds[73568]: -- 172.21.15.165:0/1032545890 <== mon.1 v2:172.21.15.165:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5595b8ce7a40 con 0x5595b8cbac00 2024-03-28T00:33:03.134 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:02 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: starting mds.c at 2024-03-28T00:33:03.134 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:02 smithi165 bash[73549]: e50f2f72ad4921599196dc06f38bf8e6166fd6c2714c6986f461bc803fd924a9 2024-03-28T00:33:03.134 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:02 smithi165 systemd[1]: Started Ceph mds.c for 10f73b92-ec98-11ee-95d0-87774f69a715. 2024-03-28T00:33:03.157 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:02 smithi119 podman[113746]: 2024-03-28 00:33:02.875629918 +0000 UTC m=+0.462959373 container init f519af739077e8d2209d74929537c0431b390e73dc83b35975867db4429386fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-03-28T00:33:03.157 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:02 smithi119 podman[113746]: 2024-03-28 00:33:02.886337251 +0000 UTC m=+0.473666681 container start f519af739077e8d2209d74929537c0431b390e73dc83b35975867db4429386fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-03-28T00:33:03.157 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:02 smithi119 ceph-mds[113766]: -- 172.21.15.119:0/1094834301 <== mon.1 v2:172.21.15.165:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x556495c5da40 con 0x556495c30800 2024-03-28T00:33:03.157 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:02 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: starting mds.j at 2024-03-28T00:33:03.158 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:02 smithi119 bash[113746]: f519af739077e8d2209d74929537c0431b390e73dc83b35975867db4429386fd 2024-03-28T00:33:03.158 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:02 smithi119 systemd[1]: Started Ceph mds.j for 10f73b92-ec98-11ee-95d0-87774f69a715. 2024-03-28T00:33:03.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:33:03 smithi165 ceph-mon[36302]: daemon mds.c restarted 2024-03-28T00:33:03.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:33:03 smithi165 ceph-mon[36302]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-28T00:33:03.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:33:03 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-28T00:33:03.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:33:03 smithi165 ceph-mon[36302]: osdmap e102: 12 total, 12 up, 12 in 2024-03-28T00:33:03.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:33:03 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.165:6836/823338130,v1:172.21.15.165:6837/823338130] up:boot 2024-03-28T00:33:03.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:33:03 smithi165 ceph-mon[36302]: Standby daemon mds.i assigned to filesystem cephfs as rank 3 2024-03-28T00:33:03.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:33:03 smithi165 ceph-mon[36302]: fsmap cephfs:4/5 {0=b=up:active,1=j=up:active,2=h=up:active,4=f=up:active} 5 up:standby-replay 3 up:standby, 1 failed 2024-03-28T00:33:03.635 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:33:03 smithi165 ceph-mon[36302]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:replay 4 up:active 2024-03-28T00:33:03.635 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:33:03 smithi165 ceph-mon[36302]: daemon mds.j restarted 2024-03-28T00:33:03.635 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:33:03 smithi165 ceph-mon[36302]: osdmap e103: 12 total, 12 up, 12 in 2024-03-28T00:33:03.635 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:33:03 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.119:6834/2941089726,v1:172.21.15.119:6835/2941089726] up:boot 2024-03-28T00:33:03.635 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:33:03 smithi165 ceph-mon[36302]: Standby daemon mds.l assigned to filesystem cephfs as rank 1 2024-03-28T00:33:03.635 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:33:03 smithi165 ceph-mon[36302]: fsmap cephfs:4/5 {0=b=up:active,2=h=up:active,3=i=up:replay,4=f=up:active} 4 up:standby-replay 4 up:standby, 1 failed 2024-03-28T00:33:03.635 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:33:03 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-28T00:33:03.635 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:33:03 smithi165 ceph-mon[36302]: fsmap cephfs:5/5 3 up:standby-replay 4 up:standby 2 up:replay 3 up:active 2024-03-28T00:33:03.658 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:33:03 smithi119 ceph-mon[31518]: daemon mds.c restarted 2024-03-28T00:33:03.658 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:33:03 smithi119 ceph-mon[31518]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-28T00:33:03.658 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:33:03 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-28T00:33:03.658 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:33:03 smithi119 ceph-mon[31518]: osdmap e102: 12 total, 12 up, 12 in 2024-03-28T00:33:03.658 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:33:03 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.165:6836/823338130,v1:172.21.15.165:6837/823338130] up:boot 2024-03-28T00:33:03.658 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:33:03 smithi119 ceph-mon[31518]: Standby daemon mds.i assigned to filesystem cephfs as rank 3 2024-03-28T00:33:03.658 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:33:03 smithi119 ceph-mon[31518]: fsmap cephfs:4/5 {0=b=up:active,1=j=up:active,2=h=up:active,4=f=up:active} 5 up:standby-replay 3 up:standby, 1 failed 2024-03-28T00:33:03.658 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:33:03 smithi119 ceph-mon[31518]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:replay 4 up:active 2024-03-28T00:33:03.658 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:33:03 smithi119 ceph-mon[31518]: daemon mds.j restarted 2024-03-28T00:33:03.659 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:33:03 smithi119 ceph-mon[31518]: osdmap e103: 12 total, 12 up, 12 in 2024-03-28T00:33:03.659 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:33:03 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.119:6834/2941089726,v1:172.21.15.119:6835/2941089726] up:boot 2024-03-28T00:33:03.659 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:33:03 smithi119 ceph-mon[31518]: Standby daemon mds.l assigned to filesystem cephfs as rank 1 2024-03-28T00:33:03.659 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:33:03 smithi119 ceph-mon[31518]: fsmap cephfs:4/5 {0=b=up:active,2=h=up:active,3=i=up:replay,4=f=up:active} 4 up:standby-replay 4 up:standby, 1 failed 2024-03-28T00:33:03.659 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:33:03 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-28T00:33:03.659 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:33:03 smithi119 ceph-mon[31518]: fsmap cephfs:5/5 3 up:standby-replay 4 up:standby 2 up:replay 3 up:active 2024-03-28T00:33:03.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:33:03 smithi132 ceph-mon[35941]: daemon mds.c restarted 2024-03-28T00:33:03.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:33:03 smithi132 ceph-mon[35941]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-28T00:33:03.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:33:03 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-28T00:33:03.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:33:03 smithi132 ceph-mon[35941]: osdmap e102: 12 total, 12 up, 12 in 2024-03-28T00:33:03.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:33:03 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.165:6836/823338130,v1:172.21.15.165:6837/823338130] up:boot 2024-03-28T00:33:03.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:33:03 smithi132 ceph-mon[35941]: Standby daemon mds.i assigned to filesystem cephfs as rank 3 2024-03-28T00:33:03.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:33:03 smithi132 ceph-mon[35941]: fsmap cephfs:4/5 {0=b=up:active,1=j=up:active,2=h=up:active,4=f=up:active} 5 up:standby-replay 3 up:standby, 1 failed 2024-03-28T00:33:03.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:33:03 smithi132 ceph-mon[35941]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:replay 4 up:active 2024-03-28T00:33:03.705 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:33:03 smithi132 ceph-mon[35941]: daemon mds.j restarted 2024-03-28T00:33:03.705 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:33:03 smithi132 ceph-mon[35941]: osdmap e103: 12 total, 12 up, 12 in 2024-03-28T00:33:03.705 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:33:03 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.119:6834/2941089726,v1:172.21.15.119:6835/2941089726] up:boot 2024-03-28T00:33:03.705 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:33:03 smithi132 ceph-mon[35941]: Standby daemon mds.l assigned to filesystem cephfs as rank 1 2024-03-28T00:33:03.705 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:33:03 smithi132 ceph-mon[35941]: fsmap cephfs:4/5 {0=b=up:active,2=h=up:active,3=i=up:replay,4=f=up:active} 4 up:standby-replay 4 up:standby, 1 failed 2024-03-28T00:33:03.705 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:33:03 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-28T00:33:03.705 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:33:03 smithi132 ceph-mon[35941]: fsmap cephfs:5/5 3 up:standby-replay 4 up:standby 2 up:replay 3 up:active 2024-03-28T00:33:04.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:33:04 smithi165 ceph-mon[36302]: pgmap v213: 97 pgs: 97 active+clean; 212 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 171 KiB/s wr, 44 op/s 2024-03-28T00:33:04.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:33:04 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.165:6832/3417761142,v1:172.21.15.165:6833/3417761142] up:resolve 2024-03-28T00:33:04.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:33:04 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.165:6838/768712672,v1:172.21.15.165:6839/768712672] up:resolve 2024-03-28T00:33:04.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:33:04 smithi165 ceph-mon[36302]: fsmap cephfs:5/5 3 up:standby-replay 4 up:standby 2 up:resolve 3 up:active 2024-03-28T00:33:04.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:33:04 smithi119 ceph-mon[31518]: pgmap v213: 97 pgs: 97 active+clean; 212 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 171 KiB/s wr, 44 op/s 2024-03-28T00:33:04.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:33:04 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.165:6832/3417761142,v1:172.21.15.165:6833/3417761142] up:resolve 2024-03-28T00:33:04.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:33:04 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.165:6838/768712672,v1:172.21.15.165:6839/768712672] up:resolve 2024-03-28T00:33:04.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:33:04 smithi119 ceph-mon[31518]: fsmap cephfs:5/5 3 up:standby-replay 4 up:standby 2 up:resolve 3 up:active 2024-03-28T00:33:04.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:33:04 smithi132 ceph-mon[35941]: pgmap v213: 97 pgs: 97 active+clean; 212 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 171 KiB/s wr, 44 op/s 2024-03-28T00:33:04.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:33:04 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.165:6832/3417761142,v1:172.21.15.165:6833/3417761142] up:resolve 2024-03-28T00:33:04.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:33:04 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.165:6838/768712672,v1:172.21.15.165:6839/768712672] up:resolve 2024-03-28T00:33:04.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:33:04 smithi132 ceph-mon[35941]: fsmap cephfs:5/5 3 up:standby-replay 4 up:standby 2 up:resolve 3 up:active 2024-03-28T00:33:06.384 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:33:05 smithi165 ceph-mon[36302]: reconnect by client.25355 172.21.15.119:0/3071152482 after 0.00299995 2024-03-28T00:33:06.384 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:33:05 smithi165 ceph-mon[36302]: reconnect by client.25355 172.21.15.119:0/3071152482 after 0.00299995 2024-03-28T00:33:06.384 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:33:05 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.165:6838/768712672,v1:172.21.15.165:6839/768712672] up:reconnect 2024-03-28T00:33:06.384 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:33:05 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.165:6832/3417761142,v1:172.21.15.165:6833/3417761142] up:reconnect 2024-03-28T00:33:06.384 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:33:05 smithi165 ceph-mon[36302]: fsmap cephfs:5/5 3 up:standby-replay 4 up:standby 2 up:reconnect 3 up:active 2024-03-28T00:33:06.384 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:33:05 smithi165 ceph-mon[36302]: reconnect by client.25264 v1:172.21.15.119:0/3863511511 after 0.0169997 2024-03-28T00:33:06.384 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:33:05 smithi165 ceph-mon[36302]: reconnect by client.25264 v1:172.21.15.119:0/3863511511 after 0.0269996 2024-03-28T00:33:06.384 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:33:05 smithi165 ceph-mon[36302]: pgmap v214: 97 pgs: 97 active+clean; 213 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 312 KiB/s rd, 188 KiB/s wr, 55 op/s 2024-03-28T00:33:06.407 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:33:05 smithi119 ceph-mon[31518]: reconnect by client.25355 172.21.15.119:0/3071152482 after 0.00299995 2024-03-28T00:33:06.407 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:33:05 smithi119 ceph-mon[31518]: reconnect by client.25355 172.21.15.119:0/3071152482 after 0.00299995 2024-03-28T00:33:06.407 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:33:05 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.165:6838/768712672,v1:172.21.15.165:6839/768712672] up:reconnect 2024-03-28T00:33:06.407 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:33:05 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.165:6832/3417761142,v1:172.21.15.165:6833/3417761142] up:reconnect 2024-03-28T00:33:06.407 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:33:05 smithi119 ceph-mon[31518]: fsmap cephfs:5/5 3 up:standby-replay 4 up:standby 2 up:reconnect 3 up:active 2024-03-28T00:33:06.407 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:33:05 smithi119 ceph-mon[31518]: reconnect by client.25264 v1:172.21.15.119:0/3863511511 after 0.0169997 2024-03-28T00:33:06.407 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:33:05 smithi119 ceph-mon[31518]: reconnect by client.25264 v1:172.21.15.119:0/3863511511 after 0.0269996 2024-03-28T00:33:06.407 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:33:05 smithi119 ceph-mon[31518]: pgmap v214: 97 pgs: 97 active+clean; 213 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 312 KiB/s rd, 188 KiB/s wr, 55 op/s 2024-03-28T00:33:06.454 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:33:05 smithi132 ceph-mon[35941]: reconnect by client.25355 172.21.15.119:0/3071152482 after 0.00299995 2024-03-28T00:33:06.454 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:33:05 smithi132 ceph-mon[35941]: reconnect by client.25355 172.21.15.119:0/3071152482 after 0.00299995 2024-03-28T00:33:06.454 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:33:05 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.165:6838/768712672,v1:172.21.15.165:6839/768712672] up:reconnect 2024-03-28T00:33:06.454 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:33:05 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.165:6832/3417761142,v1:172.21.15.165:6833/3417761142] up:reconnect 2024-03-28T00:33:06.454 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:33:05 smithi132 ceph-mon[35941]: fsmap cephfs:5/5 3 up:standby-replay 4 up:standby 2 up:reconnect 3 up:active 2024-03-28T00:33:06.454 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:33:05 smithi132 ceph-mon[35941]: reconnect by client.25264 v1:172.21.15.119:0/3863511511 after 0.0169997 2024-03-28T00:33:06.454 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:33:05 smithi132 ceph-mon[35941]: reconnect by client.25264 v1:172.21.15.119:0/3863511511 after 0.0269996 2024-03-28T00:33:06.454 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:33:05 smithi132 ceph-mon[35941]: pgmap v214: 97 pgs: 97 active+clean; 213 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 312 KiB/s rd, 188 KiB/s wr, 55 op/s 2024-03-28T00:33:07.384 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:33:06 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.165:6838/768712672,v1:172.21.15.165:6839/768712672] up:rejoin 2024-03-28T00:33:07.384 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:33:06 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.165:6832/3417761142,v1:172.21.15.165:6833/3417761142] up:rejoin 2024-03-28T00:33:07.384 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:33:06 smithi165 ceph-mon[36302]: fsmap cephfs:5/5 3 up:standby-replay 4 up:standby 2 up:rejoin 3 up:active 2024-03-28T00:33:07.384 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:33:06 smithi165 ceph-mon[36302]: daemon mds.l is now active in filesystem cephfs as rank 1 2024-03-28T00:33:07.384 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:33:06 smithi165 ceph-mon[36302]: daemon mds.i is now active in filesystem cephfs as rank 3 2024-03-28T00:33:07.407 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:33:06 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.165:6838/768712672,v1:172.21.15.165:6839/768712672] up:rejoin 2024-03-28T00:33:07.407 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:33:06 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.165:6832/3417761142,v1:172.21.15.165:6833/3417761142] up:rejoin 2024-03-28T00:33:07.407 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:33:06 smithi119 ceph-mon[31518]: fsmap cephfs:5/5 3 up:standby-replay 4 up:standby 2 up:rejoin 3 up:active 2024-03-28T00:33:07.407 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:33:06 smithi119 ceph-mon[31518]: daemon mds.l is now active in filesystem cephfs as rank 1 2024-03-28T00:33:07.407 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:33:06 smithi119 ceph-mon[31518]: daemon mds.i is now active in filesystem cephfs as rank 3 2024-03-28T00:33:07.454 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:33:06 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.165:6838/768712672,v1:172.21.15.165:6839/768712672] up:rejoin 2024-03-28T00:33:07.454 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:33:06 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.165:6832/3417761142,v1:172.21.15.165:6833/3417761142] up:rejoin 2024-03-28T00:33:07.454 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:33:06 smithi132 ceph-mon[35941]: fsmap cephfs:5/5 3 up:standby-replay 4 up:standby 2 up:rejoin 3 up:active 2024-03-28T00:33:07.454 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:33:06 smithi132 ceph-mon[35941]: daemon mds.l is now active in filesystem cephfs as rank 1 2024-03-28T00:33:07.454 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:33:06 smithi132 ceph-mon[35941]: daemon mds.i is now active in filesystem cephfs as rank 3 2024-03-28T00:33:08.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-mon[36302]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-28T00:33:08.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-mon[36302]: Cluster is now healthy 2024-03-28T00:33:08.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.165:6838/768712672,v1:172.21.15.165:6839/768712672] up:active 2024-03-28T00:33:08.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.165:6832/3417761142,v1:172.21.15.165:6833/3417761142] up:active 2024-03-28T00:33:08.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-mon[36302]: fsmap cephfs:5 3 up:standby-replay 4 up:standby 5 up:active 2024-03-28T00:33:08.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-mon[36302]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-28T00:33:08.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-mon[36302]: pgmap v215: 97 pgs: 97 active+clean; 213 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 17 KiB/s wr, 11 op/s 2024-03-28T00:33:08.135 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7f709da10700 time 2024-03-28T00:33:07.748565+0000 2024-03-28T00:33:08.135 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T00:33:08.135 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:33:08.135 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f70a803a4a1] 2024-03-28T00:33:08.135 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f70a803a667] 2024-03-28T00:33:08.135 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55e689c0c223] 2024-03-28T00:33:08.135 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55e689a865d6] 2024-03-28T00:33:08.135 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55e689a86da1] 2024-03-28T00:33:08.135 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: 6: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x55e689abb09c] 2024-03-28T00:33:08.135 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55e689ad9233] 2024-03-28T00:33:08.135 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55e689adb197] 2024-03-28T00:33:08.136 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55e6899da11b] 2024-03-28T00:33:08.136 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55e6899da6bc] 2024-03-28T00:33:08.136 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55e6899c41ff] 2024-03-28T00:33:08.136 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f70a82cc488] 2024-03-28T00:33:08.136 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: 13: (DispatchQueue::entry()+0x51f) [0x7f70a82c960f] 2024-03-28T00:33:08.136 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f70a839ce81] 2024-03-28T00:33:08.136 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f70a6db71ca] 2024-03-28T00:33:08.136 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: 16: clone() 2024-03-28T00:33:08.136 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7f96b5f4e700 time 2024-03-28T00:33:07.737781+0000 2024-03-28T00:33:08.136 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T00:33:08.137 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:33:08.137 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f96c05784a1] 2024-03-28T00:33:08.137 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f96c0578667] 2024-03-28T00:33:08.137 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x56018afd7223] 2024-03-28T00:33:08.137 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x56018ae515d6] 2024-03-28T00:33:08.137 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x56018ae51da1] 2024-03-28T00:33:08.137 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: 6: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x56018ae8609c] 2024-03-28T00:33:08.137 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x56018aea4233] 2024-03-28T00:33:08.137 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x56018aea6197] 2024-03-28T00:33:08.137 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x56018ada511b] 2024-03-28T00:33:08.137 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x56018ada56bc] 2024-03-28T00:33:08.137 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x56018ad8f1ff] 2024-03-28T00:33:08.137 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f96c080a488] 2024-03-28T00:33:08.137 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: 13: (DispatchQueue::entry()+0x51f) [0x7f96c080760f] 2024-03-28T00:33:08.138 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f96c08dae81] 2024-03-28T00:33:08.138 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f96bf2f51ca] 2024-03-28T00:33:08.138 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: 16: clone() 2024-03-28T00:33:08.138 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: 2024-03-28T00:33:07.749+0000 7f709da10700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7f709da10700 time 2024-03-28T00:33:07.748565+0000 2024-03-28T00:33:08.138 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T00:33:08.138 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: 2024-03-28T00:33:08.138 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:33:08.138 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f70a803a4a1] 2024-03-28T00:33:08.138 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f70a803a667] 2024-03-28T00:33:08.139 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55e689c0c223] 2024-03-28T00:33:08.139 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55e689a865d6] 2024-03-28T00:33:08.139 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55e689a86da1] 2024-03-28T00:33:08.139 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: 6: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x55e689abb09c] 2024-03-28T00:33:08.139 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55e689ad9233] 2024-03-28T00:33:08.139 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55e689adb197] 2024-03-28T00:33:08.139 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55e6899da11b] 2024-03-28T00:33:08.139 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55e6899da6bc] 2024-03-28T00:33:08.139 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55e6899c41ff] 2024-03-28T00:33:08.139 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f70a82cc488] 2024-03-28T00:33:08.139 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: 13: (DispatchQueue::entry()+0x51f) [0x7f70a82c960f] 2024-03-28T00:33:08.139 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f70a839ce81] 2024-03-28T00:33:08.139 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f70a6db71ca] 2024-03-28T00:33:08.139 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: 16: clone() 2024-03-28T00:33:08.139 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: 2024-03-28T00:33:08.140 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: *** Caught signal (Aborted) ** 2024-03-28T00:33:08.140 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: in thread 7f709da10700 thread_name:ms_dispatch 2024-03-28T00:33:08.140 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:33:08.140 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f70a6dc1d20] 2024-03-28T00:33:08.140 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: 2024-03-28T00:33:07.738+0000 7f96b5f4e700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7f96b5f4e700 time 2024-03-28T00:33:07.737781+0000 2024-03-28T00:33:08.140 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T00:33:08.140 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: 2024-03-28T00:33:08.140 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:33:08.140 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f96c05784a1] 2024-03-28T00:33:08.140 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f96c0578667] 2024-03-28T00:33:08.140 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x56018afd7223] 2024-03-28T00:33:08.141 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x56018ae515d6] 2024-03-28T00:33:08.141 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x56018ae51da1] 2024-03-28T00:33:08.141 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: 6: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x56018ae8609c] 2024-03-28T00:33:08.141 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x56018aea4233] 2024-03-28T00:33:08.141 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x56018aea6197] 2024-03-28T00:33:08.141 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x56018ada511b] 2024-03-28T00:33:08.141 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x56018ada56bc] 2024-03-28T00:33:08.141 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x56018ad8f1ff] 2024-03-28T00:33:08.141 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f96c080a488] 2024-03-28T00:33:08.141 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: 13: (DispatchQueue::entry()+0x51f) [0x7f96c080760f] 2024-03-28T00:33:08.141 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f96c08dae81] 2024-03-28T00:33:08.141 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f96bf2f51ca] 2024-03-28T00:33:08.141 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: 16: clone() 2024-03-28T00:33:08.141 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: 2024-03-28T00:33:08.141 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: *** Caught signal (Aborted) ** 2024-03-28T00:33:08.142 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: in thread 7f96b5f4e700 thread_name:ms_dispatch 2024-03-28T00:33:08.142 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:33:08.142 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f96bf2ffd20] 2024-03-28T00:33:08.142 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: 2: gsignal() 2024-03-28T00:33:08.142 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: 3: abort() 2024-03-28T00:33:08.142 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f70a803a4fb] 2024-03-28T00:33:08.142 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f70a803a667] 2024-03-28T00:33:08.142 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55e689c0c223] 2024-03-28T00:33:08.142 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55e689a865d6] 2024-03-28T00:33:08.142 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55e689a86da1] 2024-03-28T00:33:08.143 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: 9: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x55e689abb09c] 2024-03-28T00:33:08.143 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55e689ad9233] 2024-03-28T00:33:08.143 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55e689adb197] 2024-03-28T00:33:08.143 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55e6899da11b] 2024-03-28T00:33:08.143 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55e6899da6bc] 2024-03-28T00:33:08.143 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55e6899c41ff] 2024-03-28T00:33:08.143 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f70a82cc488] 2024-03-28T00:33:08.143 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: 16: (DispatchQueue::entry()+0x51f) [0x7f70a82c960f] 2024-03-28T00:33:08.143 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f70a839ce81] 2024-03-28T00:33:08.143 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f70a6db71ca] 2024-03-28T00:33:08.143 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: 19: clone() 2024-03-28T00:33:08.143 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: 2024-03-28T00:33:07.752+0000 7f709da10700 -1 *** Caught signal (Aborted) ** 2024-03-28T00:33:08.143 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: in thread 7f709da10700 thread_name:ms_dispatch 2024-03-28T00:33:08.143 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: 2024-03-28T00:33:08.143 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:33:08.143 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f70a6dc1d20] 2024-03-28T00:33:08.144 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: 2: gsignal() 2024-03-28T00:33:08.144 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: 3: abort() 2024-03-28T00:33:08.144 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f70a803a4fb] 2024-03-28T00:33:08.144 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f70a803a667] 2024-03-28T00:33:08.144 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: 2: gsignal() 2024-03-28T00:33:08.144 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: 3: abort() 2024-03-28T00:33:08.144 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f96c05784fb] 2024-03-28T00:33:08.144 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f96c0578667] 2024-03-28T00:33:08.144 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x56018afd7223] 2024-03-28T00:33:08.144 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x56018ae515d6] 2024-03-28T00:33:08.144 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x56018ae51da1] 2024-03-28T00:33:08.144 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: 9: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x56018ae8609c] 2024-03-28T00:33:08.145 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x56018aea4233] 2024-03-28T00:33:08.145 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x56018aea6197] 2024-03-28T00:33:08.145 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x56018ada511b] 2024-03-28T00:33:08.145 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x56018ada56bc] 2024-03-28T00:33:08.145 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x56018ad8f1ff] 2024-03-28T00:33:08.145 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f96c080a488] 2024-03-28T00:33:08.145 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: 16: (DispatchQueue::entry()+0x51f) [0x7f96c080760f] 2024-03-28T00:33:08.145 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f96c08dae81] 2024-03-28T00:33:08.145 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f96bf2f51ca] 2024-03-28T00:33:08.145 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: 19: clone() 2024-03-28T00:33:08.145 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: 2024-03-28T00:33:07.740+0000 7f96b5f4e700 -1 *** Caught signal (Aborted) ** 2024-03-28T00:33:08.145 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: in thread 7f96b5f4e700 thread_name:ms_dispatch 2024-03-28T00:33:08.145 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: 2024-03-28T00:33:08.145 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:33:08.145 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f96bf2ffd20] 2024-03-28T00:33:08.145 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: 2: gsignal() 2024-03-28T00:33:08.146 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: 3: abort() 2024-03-28T00:33:08.146 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f96c05784fb] 2024-03-28T00:33:08.146 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f96c0578667] 2024-03-28T00:33:08.146 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55e689c0c223] 2024-03-28T00:33:08.146 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55e689a865d6] 2024-03-28T00:33:08.146 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55e689a86da1] 2024-03-28T00:33:08.146 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: 9: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x55e689abb09c] 2024-03-28T00:33:08.146 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55e689ad9233] 2024-03-28T00:33:08.146 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55e689adb197] 2024-03-28T00:33:08.146 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55e6899da11b] 2024-03-28T00:33:08.146 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55e6899da6bc] 2024-03-28T00:33:08.147 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55e6899c41ff] 2024-03-28T00:33:08.147 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f70a82cc488] 2024-03-28T00:33:08.147 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: 16: (DispatchQueue::entry()+0x51f) [0x7f70a82c960f] 2024-03-28T00:33:08.147 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f70a839ce81] 2024-03-28T00:33:08.147 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f70a6db71ca] 2024-03-28T00:33:08.147 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: 19: clone() 2024-03-28T00:33:08.147 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T00:33:08.147 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: 2024-03-28T00:33:08.147 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: -1> 2024-03-28T00:33:07.749+0000 7f709da10700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7f709da10700 time 2024-03-28T00:33:07.748565+0000 2024-03-28T00:33:08.147 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T00:33:08.147 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: 2024-03-28T00:33:08.147 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:33:08.147 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f70a803a4a1] 2024-03-28T00:33:08.147 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f70a803a667] 2024-03-28T00:33:08.148 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x56018afd7223] 2024-03-28T00:33:08.148 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x56018ae515d6] 2024-03-28T00:33:08.148 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x56018ae51da1] 2024-03-28T00:33:08.148 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: 9: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x56018ae8609c] 2024-03-28T00:33:08.148 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x56018aea4233] 2024-03-28T00:33:08.148 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x56018aea6197] 2024-03-28T00:33:08.148 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x56018ada511b] 2024-03-28T00:33:08.148 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x56018ada56bc] 2024-03-28T00:33:08.148 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x56018ad8f1ff] 2024-03-28T00:33:08.148 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f96c080a488] 2024-03-28T00:33:08.148 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: 16: (DispatchQueue::entry()+0x51f) [0x7f96c080760f] 2024-03-28T00:33:08.148 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f96c08dae81] 2024-03-28T00:33:08.148 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f96bf2f51ca] 2024-03-28T00:33:08.148 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: 19: clone() 2024-03-28T00:33:08.149 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T00:33:08.149 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: 2024-03-28T00:33:08.149 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: -1> 2024-03-28T00:33:07.738+0000 7f96b5f4e700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7f96b5f4e700 time 2024-03-28T00:33:07.737781+0000 2024-03-28T00:33:08.149 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T00:33:08.149 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: 2024-03-28T00:33:08.149 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:33:08.149 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f96c05784a1] 2024-03-28T00:33:08.149 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f96c0578667] 2024-03-28T00:33:08.149 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55e689c0c223] 2024-03-28T00:33:08.149 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55e689a865d6] 2024-03-28T00:33:08.149 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55e689a86da1] 2024-03-28T00:33:08.149 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: 6: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x55e689abb09c] 2024-03-28T00:33:08.150 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55e689ad9233] 2024-03-28T00:33:08.150 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55e689adb197] 2024-03-28T00:33:08.150 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55e6899da11b] 2024-03-28T00:33:08.150 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55e6899da6bc] 2024-03-28T00:33:08.150 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55e6899c41ff] 2024-03-28T00:33:08.150 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f70a82cc488] 2024-03-28T00:33:08.150 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: 13: (DispatchQueue::entry()+0x51f) [0x7f70a82c960f] 2024-03-28T00:33:08.150 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f70a839ce81] 2024-03-28T00:33:08.150 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f70a6db71ca] 2024-03-28T00:33:08.150 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: 16: clone() 2024-03-28T00:33:08.150 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: 2024-03-28T00:33:08.150 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: 0> 2024-03-28T00:33:07.752+0000 7f709da10700 -1 *** Caught signal (Aborted) ** 2024-03-28T00:33:08.150 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: in thread 7f709da10700 thread_name:ms_dispatch 2024-03-28T00:33:08.150 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: 2024-03-28T00:33:08.150 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:33:08.151 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f70a6dc1d20] 2024-03-28T00:33:08.151 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: 2: gsignal() 2024-03-28T00:33:08.151 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: 3: abort() 2024-03-28T00:33:08.151 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f70a803a4fb] 2024-03-28T00:33:08.151 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f70a803a667] 2024-03-28T00:33:08.151 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55e689c0c223] 2024-03-28T00:33:08.151 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55e689a865d6] 2024-03-28T00:33:08.151 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x56018afd7223] 2024-03-28T00:33:08.151 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x56018ae515d6] 2024-03-28T00:33:08.151 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x56018ae51da1] 2024-03-28T00:33:08.151 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: 6: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x56018ae8609c] 2024-03-28T00:33:08.151 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x56018aea4233] 2024-03-28T00:33:08.152 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x56018aea6197] 2024-03-28T00:33:08.152 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x56018ada511b] 2024-03-28T00:33:08.152 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x56018ada56bc] 2024-03-28T00:33:08.152 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x56018ad8f1ff] 2024-03-28T00:33:08.152 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f96c080a488] 2024-03-28T00:33:08.152 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: 13: (DispatchQueue::entry()+0x51f) [0x7f96c080760f] 2024-03-28T00:33:08.152 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f96c08dae81] 2024-03-28T00:33:08.152 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f96bf2f51ca] 2024-03-28T00:33:08.152 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: 16: clone() 2024-03-28T00:33:08.152 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: 2024-03-28T00:33:08.152 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: 0> 2024-03-28T00:33:07.740+0000 7f96b5f4e700 -1 *** Caught signal (Aborted) ** 2024-03-28T00:33:08.152 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: in thread 7f96b5f4e700 thread_name:ms_dispatch 2024-03-28T00:33:08.152 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: 2024-03-28T00:33:08.152 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:33:08.152 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f96bf2ffd20] 2024-03-28T00:33:08.152 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: 2: gsignal() 2024-03-28T00:33:08.153 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: 3: abort() 2024-03-28T00:33:08.153 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f96c05784fb] 2024-03-28T00:33:08.153 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f96c0578667] 2024-03-28T00:33:08.153 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x56018afd7223] 2024-03-28T00:33:08.153 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x56018ae515d6] 2024-03-28T00:33:08.153 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55e689a86da1] 2024-03-28T00:33:08.153 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: 9: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x55e689abb09c] 2024-03-28T00:33:08.153 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55e689ad9233] 2024-03-28T00:33:08.153 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55e689adb197] 2024-03-28T00:33:08.153 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55e6899da11b] 2024-03-28T00:33:08.153 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55e6899da6bc] 2024-03-28T00:33:08.154 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55e6899c41ff] 2024-03-28T00:33:08.154 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f70a82cc488] 2024-03-28T00:33:08.154 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: 16: (DispatchQueue::entry()+0x51f) [0x7f70a82c960f] 2024-03-28T00:33:08.154 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f70a839ce81] 2024-03-28T00:33:08.154 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f70a6db71ca] 2024-03-28T00:33:08.154 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: 19: clone() 2024-03-28T00:33:08.154 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T00:33:08.154 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: 2024-03-28T00:33:08.154 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: -4> 2024-03-28T00:33:07.749+0000 7f709da10700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7f709da10700 time 2024-03-28T00:33:07.748565+0000 2024-03-28T00:33:08.154 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T00:33:08.154 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: 2024-03-28T00:33:08.154 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:33:08.154 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f70a803a4a1] 2024-03-28T00:33:08.154 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f70a803a667] 2024-03-28T00:33:08.154 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55e689c0c223] 2024-03-28T00:33:08.155 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55e689a865d6] 2024-03-28T00:33:08.155 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55e689a86da1] 2024-03-28T00:33:08.155 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x56018ae51da1] 2024-03-28T00:33:08.155 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: 9: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x56018ae8609c] 2024-03-28T00:33:08.155 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x56018aea4233] 2024-03-28T00:33:08.155 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x56018aea6197] 2024-03-28T00:33:08.155 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x56018ada511b] 2024-03-28T00:33:08.155 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x56018ada56bc] 2024-03-28T00:33:08.155 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x56018ad8f1ff] 2024-03-28T00:33:08.155 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f96c080a488] 2024-03-28T00:33:08.155 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: 16: (DispatchQueue::entry()+0x51f) [0x7f96c080760f] 2024-03-28T00:33:08.155 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f96c08dae81] 2024-03-28T00:33:08.156 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f96bf2f51ca] 2024-03-28T00:33:08.156 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: 19: clone() 2024-03-28T00:33:08.156 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T00:33:08.156 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: 2024-03-28T00:33:08.156 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: -1> 2024-03-28T00:33:07.738+0000 7f96b5f4e700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7f96b5f4e700 time 2024-03-28T00:33:07.737781+0000 2024-03-28T00:33:08.156 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T00:33:08.156 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: 2024-03-28T00:33:08.156 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:33:08.156 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f96c05784a1] 2024-03-28T00:33:08.156 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f96c0578667] 2024-03-28T00:33:08.156 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x56018afd7223] 2024-03-28T00:33:08.156 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x56018ae515d6] 2024-03-28T00:33:08.156 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x56018ae51da1] 2024-03-28T00:33:08.157 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: 6: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x55e689abb09c] 2024-03-28T00:33:08.157 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55e689ad9233] 2024-03-28T00:33:08.157 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55e689adb197] 2024-03-28T00:33:08.157 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55e6899da11b] 2024-03-28T00:33:08.157 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55e6899da6bc] 2024-03-28T00:33:08.157 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55e6899c41ff] 2024-03-28T00:33:08.157 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f70a82cc488] 2024-03-28T00:33:08.157 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: 13: (DispatchQueue::entry()+0x51f) [0x7f70a82c960f] 2024-03-28T00:33:08.157 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f70a839ce81] 2024-03-28T00:33:08.157 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f70a6db71ca] 2024-03-28T00:33:08.157 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: 16: clone() 2024-03-28T00:33:08.157 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: 2024-03-28T00:33:08.157 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: -3> 2024-03-28T00:33:07.752+0000 7f709da10700 -1 *** Caught signal (Aborted) ** 2024-03-28T00:33:08.158 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: in thread 7f709da10700 thread_name:ms_dispatch 2024-03-28T00:33:08.158 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: 2024-03-28T00:33:08.158 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:33:08.158 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f70a6dc1d20] 2024-03-28T00:33:08.158 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: 2: gsignal() 2024-03-28T00:33:08.158 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: 3: abort() 2024-03-28T00:33:08.158 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f70a803a4fb] 2024-03-28T00:33:08.158 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f70a803a667] 2024-03-28T00:33:08.158 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55e689c0c223] 2024-03-28T00:33:08.158 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55e689a865d6] 2024-03-28T00:33:08.158 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55e689a86da1] 2024-03-28T00:33:08.158 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: 9: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x55e689abb09c] 2024-03-28T00:33:08.158 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55e689ad9233] 2024-03-28T00:33:08.158 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55e689adb197] 2024-03-28T00:33:08.159 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55e6899da11b] 2024-03-28T00:33:08.159 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55e6899da6bc] 2024-03-28T00:33:08.159 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55e6899c41ff] 2024-03-28T00:33:08.159 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f70a82cc488] 2024-03-28T00:33:08.159 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: 16: (DispatchQueue::entry()+0x51f) [0x7f70a82c960f] 2024-03-28T00:33:08.159 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f70a839ce81] 2024-03-28T00:33:08.159 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f70a6db71ca] 2024-03-28T00:33:08.159 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: 19: clone() 2024-03-28T00:33:08.159 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T00:33:08.159 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73252]: 2024-03-28T00:33:08.160 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: 6: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x56018ae8609c] 2024-03-28T00:33:08.160 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x56018aea4233] 2024-03-28T00:33:08.160 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x56018aea6197] 2024-03-28T00:33:08.160 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x56018ada511b] 2024-03-28T00:33:08.160 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x56018ada56bc] 2024-03-28T00:33:08.160 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x56018ad8f1ff] 2024-03-28T00:33:08.160 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f96c080a488] 2024-03-28T00:33:08.160 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: 13: (DispatchQueue::entry()+0x51f) [0x7f96c080760f] 2024-03-28T00:33:08.160 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f96c08dae81] 2024-03-28T00:33:08.160 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f96bf2f51ca] 2024-03-28T00:33:08.160 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: 16: clone() 2024-03-28T00:33:08.160 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: 2024-03-28T00:33:08.160 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: 0> 2024-03-28T00:33:07.740+0000 7f96b5f4e700 -1 *** Caught signal (Aborted) ** 2024-03-28T00:33:08.161 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: in thread 7f96b5f4e700 thread_name:ms_dispatch 2024-03-28T00:33:08.161 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: 2024-03-28T00:33:08.161 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:33:08.161 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f96bf2ffd20] 2024-03-28T00:33:08.161 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: 2: gsignal() 2024-03-28T00:33:08.161 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: 3: abort() 2024-03-28T00:33:08.161 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f96c05784fb] 2024-03-28T00:33:08.161 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f96c0578667] 2024-03-28T00:33:08.161 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x56018afd7223] 2024-03-28T00:33:08.161 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x56018ae515d6] 2024-03-28T00:33:08.161 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x56018ae51da1] 2024-03-28T00:33:08.161 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: 9: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x56018ae8609c] 2024-03-28T00:33:08.161 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x56018aea4233] 2024-03-28T00:33:08.161 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x56018aea6197] 2024-03-28T00:33:08.161 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x56018ada511b] 2024-03-28T00:33:08.161 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x56018ada56bc] 2024-03-28T00:33:08.162 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x56018ad8f1ff] 2024-03-28T00:33:08.162 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f96c080a488] 2024-03-28T00:33:08.162 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: 16: (DispatchQueue::entry()+0x51f) [0x7f96c080760f] 2024-03-28T00:33:08.162 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f96c08dae81] 2024-03-28T00:33:08.162 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f96bf2f51ca] 2024-03-28T00:33:08.162 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: 19: clone() 2024-03-28T00:33:08.162 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T00:33:08.162 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73247]: 2024-03-28T00:33:08.162 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:07 smithi165 podman[73605]: 2024-03-28 00:33:07.816544763 +0000 UTC m=+0.022168711 container died 62cc90d67cd903aa97960596ef5c9dd218d14d3633b5177338f3c3e6ebe4f8e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319) 2024-03-28T00:33:08.162 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:33:07 smithi119 ceph-mon[31518]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-28T00:33:08.162 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:33:07 smithi119 ceph-mon[31518]: Cluster is now healthy 2024-03-28T00:33:08.162 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:33:07 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.165:6838/768712672,v1:172.21.15.165:6839/768712672] up:active 2024-03-28T00:33:08.162 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:33:07 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.165:6832/3417761142,v1:172.21.15.165:6833/3417761142] up:active 2024-03-28T00:33:08.162 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:33:07 smithi119 ceph-mon[31518]: fsmap cephfs:5 3 up:standby-replay 4 up:standby 5 up:active 2024-03-28T00:33:08.162 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:33:07 smithi119 ceph-mon[31518]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-28T00:33:08.163 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:33:07 smithi119 ceph-mon[31518]: pgmap v215: 97 pgs: 97 active+clean; 213 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 17 KiB/s wr, 11 op/s 2024-03-28T00:33:08.454 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:33:07 smithi132 ceph-mon[35941]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-28T00:33:08.454 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:33:07 smithi132 ceph-mon[35941]: Cluster is now healthy 2024-03-28T00:33:08.454 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:33:07 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.165:6838/768712672,v1:172.21.15.165:6839/768712672] up:active 2024-03-28T00:33:08.454 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:33:07 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.165:6832/3417761142,v1:172.21.15.165:6833/3417761142] up:active 2024-03-28T00:33:08.454 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:33:07 smithi132 ceph-mon[35941]: fsmap cephfs:5 3 up:standby-replay 4 up:standby 5 up:active 2024-03-28T00:33:08.455 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:33:07 smithi132 ceph-mon[35941]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-28T00:33:08.455 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:33:07 smithi132 ceph-mon[35941]: pgmap v215: 97 pgs: 97 active+clean; 213 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 17 KiB/s wr, 11 op/s 2024-03-28T00:33:08.884 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:08 smithi165 podman[73611]: 2024-03-28 00:33:08.652147419 +0000 UTC m=+0.851186351 container died f1ef1cba450f73c0cfc14ae61e24411d780bd29d29489f3d16f809dcee086e6c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l, org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-03-28T00:33:08.884 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:08 smithi165 podman[73605]: 2024-03-28 00:33:08.610798364 +0000 UTC m=+0.816422300 container remove 62cc90d67cd903aa97960596ef5c9dd218d14d3633b5177338f3c3e6ebe4f8e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-03-28T00:33:08.884 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:08 smithi165 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.i.service: Main process exited, code=exited, status=134/n/a 2024-03-28T00:33:09.664 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:09 smithi165 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.i.service: Failed with result 'exit-code'. 2024-03-28T00:33:09.665 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:09 smithi165 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.i.service: Consumed 1.163s CPU time. 2024-03-28T00:33:09.665 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:09 smithi165 podman[73611]: 2024-03-28 00:33:09.40798651 +0000 UTC m=+1.607025422 container remove f1ef1cba450f73c0cfc14ae61e24411d780bd29d29489f3d16f809dcee086e6c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-03-28T00:33:09.665 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:09 smithi165 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.l.service: Main process exited, code=exited, status=134/n/a 2024-03-28T00:33:10.134 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:09 smithi165 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.l.service: Failed with result 'exit-code'. 2024-03-28T00:33:10.407 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:33:10 smithi119 ceph-mon[31518]: pgmap v216: 97 pgs: 97 active+clean; 220 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.0 MiB/s wr, 60 op/s 2024-03-28T00:33:10.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:33:10 smithi165 ceph-mon[36302]: pgmap v216: 97 pgs: 97 active+clean; 220 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.0 MiB/s wr, 60 op/s 2024-03-28T00:33:10.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:33:10 smithi132 ceph-mon[35941]: pgmap v216: 97 pgs: 97 active+clean; 220 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.0 MiB/s wr, 60 op/s 2024-03-28T00:33:12.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:33:12 smithi165 ceph-mon[36302]: pgmap v217: 97 pgs: 97 active+clean; 220 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 985 KiB/s wr, 57 op/s 2024-03-28T00:33:12.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:33:12 smithi119 ceph-mon[31518]: pgmap v217: 97 pgs: 97 active+clean; 220 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 985 KiB/s wr, 57 op/s 2024-03-28T00:33:12.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:33:12 smithi132 ceph-mon[35941]: pgmap v217: 97 pgs: 97 active+clean; 220 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 985 KiB/s wr, 57 op/s 2024-03-28T00:33:14.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:33:14 smithi165 ceph-mon[36302]: pgmap v218: 97 pgs: 97 active+clean; 220 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1014 KiB/s rd, 815 KiB/s wr, 47 op/s 2024-03-28T00:33:14.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:33:14 smithi119 ceph-mon[31518]: pgmap v218: 97 pgs: 97 active+clean; 220 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1014 KiB/s rd, 815 KiB/s wr, 47 op/s 2024-03-28T00:33:14.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:33:14 smithi132 ceph-mon[35941]: pgmap v218: 97 pgs: 97 active+clean; 220 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1014 KiB/s rd, 815 KiB/s wr, 47 op/s 2024-03-28T00:33:16.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:33:16 smithi165 ceph-mon[36302]: pgmap v219: 97 pgs: 97 active+clean; 236 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.0 MiB/s wr, 52 op/s 2024-03-28T00:33:16.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:33:16 smithi119 ceph-mon[31518]: pgmap v219: 97 pgs: 97 active+clean; 236 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.0 MiB/s wr, 52 op/s 2024-03-28T00:33:16.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:33:16 smithi132 ceph-mon[35941]: pgmap v219: 97 pgs: 97 active+clean; 236 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.0 MiB/s wr, 52 op/s 2024-03-28T00:33:18.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:33:18 smithi165 ceph-mon[36302]: pgmap v220: 97 pgs: 97 active+clean; 236 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.0 MiB/s wr, 44 op/s 2024-03-28T00:33:18.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:33:18 smithi119 ceph-mon[31518]: pgmap v220: 97 pgs: 97 active+clean; 236 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.0 MiB/s wr, 44 op/s 2024-03-28T00:33:18.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:33:18 smithi132 ceph-mon[35941]: pgmap v220: 97 pgs: 97 active+clean; 236 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.0 MiB/s wr, 44 op/s 2024-03-28T00:33:19.885 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:19 smithi165 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.l.service: Scheduled restart job, restart counter is at 5. 2024-03-28T00:33:19.885 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:19 smithi165 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.i.service: Scheduled restart job, restart counter is at 4. 2024-03-28T00:33:19.885 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:19 smithi165 systemd[1]: Stopped Ceph mds.i for 10f73b92-ec98-11ee-95d0-87774f69a715. 2024-03-28T00:33:19.885 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:19 smithi165 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.i.service: Consumed 1.163s CPU time. 2024-03-28T00:33:19.885 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:19 smithi165 systemd[1]: Starting Ceph mds.i for 10f73b92-ec98-11ee-95d0-87774f69a715... 2024-03-28T00:33:19.885 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:19 smithi165 podman[73744]: 2024-03-28 00:33:19.763587342 +0000 UTC m=+0.017995672 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-28T00:33:20.217 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:19 smithi165 systemd[1]: Stopped Ceph mds.l for 10f73b92-ec98-11ee-95d0-87774f69a715. 2024-03-28T00:33:20.217 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:19 smithi165 systemd[1]: Starting Ceph mds.l for 10f73b92-ec98-11ee-95d0-87774f69a715... 2024-03-28T00:33:20.217 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:19 smithi165 podman[73744]: 2024-03-28 00:33:19.894311603 +0000 UTC m=+0.148719920 container create e88bd49e10a33560a689618bb1a43ffed1447887f37b532a1980168d19824d06 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS) 2024-03-28T00:33:20.518 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:33:20 smithi165 ceph-mon[36302]: pgmap v221: 97 pgs: 97 active+clean; 236 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.0 MiB/s wr, 51 op/s 2024-03-28T00:33:20.519 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:20 smithi165 podman[73744]: 2024-03-28 00:33:20.210887823 +0000 UTC m=+0.465296155 container init e88bd49e10a33560a689618bb1a43ffed1447887f37b532a1980168d19824d06 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-28T00:33:20.519 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:20 smithi165 podman[73744]: 2024-03-28 00:33:20.214534911 +0000 UTC m=+0.468943244 container start e88bd49e10a33560a689618bb1a43ffed1447887f37b532a1980168d19824d06 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c) 2024-03-28T00:33:20.519 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: starting mds.i at 2024-03-28T00:33:20.519 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:20 smithi165 bash[73744]: e88bd49e10a33560a689618bb1a43ffed1447887f37b532a1980168d19824d06 2024-03-28T00:33:20.519 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:20 smithi165 systemd[1]: Started Ceph mds.i for 10f73b92-ec98-11ee-95d0-87774f69a715. 2024-03-28T00:33:20.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:33:20 smithi119 ceph-mon[31518]: pgmap v221: 97 pgs: 97 active+clean; 236 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.0 MiB/s wr, 51 op/s 2024-03-28T00:33:20.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:33:20 smithi132 ceph-mon[35941]: pgmap v221: 97 pgs: 97 active+clean; 236 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.0 MiB/s wr, 51 op/s 2024-03-28T00:33:20.805 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:20 smithi165 podman[73832]: 2024-03-28 00:33:20.424000854 +0000 UTC m=+0.018276444 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-28T00:33:20.805 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:20 smithi165 podman[73832]: 2024-03-28 00:33:20.57105347 +0000 UTC m=+0.165329020 container create 9fef9895027f3ffd477a2a49316400b684e21d9f00da5f58ae88e73497aef267 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-28T00:33:21.134 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:20 smithi165 podman[73832]: 2024-03-28 00:33:20.798461057 +0000 UTC m=+0.392736613 container init 9fef9895027f3ffd477a2a49316400b684e21d9f00da5f58ae88e73497aef267 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD) 2024-03-28T00:33:21.134 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:20 smithi165 podman[73832]: 2024-03-28 00:33:20.801370773 +0000 UTC m=+0.395646333 container start 9fef9895027f3ffd477a2a49316400b684e21d9f00da5f58ae88e73497aef267 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-28T00:33:21.134 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:20 smithi165 ceph-mds[73859]: -- 172.21.15.165:0/3757010716 <== mon.1 v2:172.21.15.165:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5629c661da40 con 0x5629c65f1000 2024-03-28T00:33:21.134 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: starting mds.l at 2024-03-28T00:33:21.134 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:20 smithi165 bash[73832]: 9fef9895027f3ffd477a2a49316400b684e21d9f00da5f58ae88e73497aef267 2024-03-28T00:33:21.134 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:20 smithi165 systemd[1]: Started Ceph mds.l for 10f73b92-ec98-11ee-95d0-87774f69a715. 2024-03-28T00:33:21.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:33:21 smithi165 ceph-mon[36302]: daemon mds.i restarted 2024-03-28T00:33:21.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:33:21 smithi165 ceph-mon[36302]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-28T00:33:21.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:33:21 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-28T00:33:21.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:33:21 smithi165 ceph-mon[36302]: osdmap e104: 12 total, 12 up, 12 in 2024-03-28T00:33:21.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:33:21 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.165:6832/1344198130,v1:172.21.15.165:6833/1344198130] up:boot 2024-03-28T00:33:21.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:33:21 smithi165 ceph-mon[36302]: Standby daemon mds.j assigned to filesystem cephfs as rank 3 2024-03-28T00:33:21.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:33:21 smithi165 ceph-mon[36302]: fsmap cephfs:4/5 {0=b=up:active,1=l=up:active,2=h=up:active,4=f=up:active} 5 up:standby-replay 3 up:standby, 1 failed 2024-03-28T00:33:21.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:33:21 smithi165 ceph-mon[36302]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:replay 4 up:active 2024-03-28T00:33:21.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:33:21 smithi165 ceph-mon[36302]: daemon mds.l restarted 2024-03-28T00:33:21.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:33:21 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-28T00:33:21.635 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:33:21 smithi165 ceph-mon[36302]: osdmap e105: 12 total, 12 up, 12 in 2024-03-28T00:33:21.635 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:33:21 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.165:6838/2762859081,v1:172.21.15.165:6839/2762859081] up:boot 2024-03-28T00:33:21.635 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:33:21 smithi165 ceph-mon[36302]: Standby daemon mds.c assigned to filesystem cephfs as rank 1 2024-03-28T00:33:21.635 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:33:21 smithi165 ceph-mon[36302]: fsmap cephfs:4/5 {0=b=up:active,2=h=up:active,3=j=up:replay,4=f=up:active} 4 up:standby-replay 4 up:standby, 1 failed 2024-03-28T00:33:21.635 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:33:21 smithi165 ceph-mon[36302]: fsmap cephfs:5/5 3 up:standby-replay 4 up:standby 2 up:replay 3 up:active 2024-03-28T00:33:21.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:33:21 smithi119 ceph-mon[31518]: daemon mds.i restarted 2024-03-28T00:33:21.658 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:33:21 smithi119 ceph-mon[31518]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-28T00:33:21.658 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:33:21 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-28T00:33:21.658 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:33:21 smithi119 ceph-mon[31518]: osdmap e104: 12 total, 12 up, 12 in 2024-03-28T00:33:21.658 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:33:21 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.165:6832/1344198130,v1:172.21.15.165:6833/1344198130] up:boot 2024-03-28T00:33:21.659 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:33:21 smithi119 ceph-mon[31518]: Standby daemon mds.j assigned to filesystem cephfs as rank 3 2024-03-28T00:33:21.659 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:33:21 smithi119 ceph-mon[31518]: fsmap cephfs:4/5 {0=b=up:active,1=l=up:active,2=h=up:active,4=f=up:active} 5 up:standby-replay 3 up:standby, 1 failed 2024-03-28T00:33:21.659 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:33:21 smithi119 ceph-mon[31518]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:replay 4 up:active 2024-03-28T00:33:21.659 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:33:21 smithi119 ceph-mon[31518]: daemon mds.l restarted 2024-03-28T00:33:21.659 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:33:21 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-28T00:33:21.659 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:33:21 smithi119 ceph-mon[31518]: osdmap e105: 12 total, 12 up, 12 in 2024-03-28T00:33:21.659 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:33:21 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.165:6838/2762859081,v1:172.21.15.165:6839/2762859081] up:boot 2024-03-28T00:33:21.659 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:33:21 smithi119 ceph-mon[31518]: Standby daemon mds.c assigned to filesystem cephfs as rank 1 2024-03-28T00:33:21.659 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:33:21 smithi119 ceph-mon[31518]: fsmap cephfs:4/5 {0=b=up:active,2=h=up:active,3=j=up:replay,4=f=up:active} 4 up:standby-replay 4 up:standby, 1 failed 2024-03-28T00:33:21.660 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:33:21 smithi119 ceph-mon[31518]: fsmap cephfs:5/5 3 up:standby-replay 4 up:standby 2 up:replay 3 up:active 2024-03-28T00:33:21.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:33:21 smithi132 ceph-mon[35941]: daemon mds.i restarted 2024-03-28T00:33:21.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:33:21 smithi132 ceph-mon[35941]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-28T00:33:21.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:33:21 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-28T00:33:21.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:33:21 smithi132 ceph-mon[35941]: osdmap e104: 12 total, 12 up, 12 in 2024-03-28T00:33:21.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:33:21 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.165:6832/1344198130,v1:172.21.15.165:6833/1344198130] up:boot 2024-03-28T00:33:21.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:33:21 smithi132 ceph-mon[35941]: Standby daemon mds.j assigned to filesystem cephfs as rank 3 2024-03-28T00:33:21.705 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:33:21 smithi132 ceph-mon[35941]: fsmap cephfs:4/5 {0=b=up:active,1=l=up:active,2=h=up:active,4=f=up:active} 5 up:standby-replay 3 up:standby, 1 failed 2024-03-28T00:33:21.705 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:33:21 smithi132 ceph-mon[35941]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:replay 4 up:active 2024-03-28T00:33:21.705 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:33:21 smithi132 ceph-mon[35941]: daemon mds.l restarted 2024-03-28T00:33:21.705 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:33:21 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-28T00:33:21.705 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:33:21 smithi132 ceph-mon[35941]: osdmap e105: 12 total, 12 up, 12 in 2024-03-28T00:33:21.705 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:33:21 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.165:6838/2762859081,v1:172.21.15.165:6839/2762859081] up:boot 2024-03-28T00:33:21.705 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:33:21 smithi132 ceph-mon[35941]: Standby daemon mds.c assigned to filesystem cephfs as rank 1 2024-03-28T00:33:21.705 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:33:21 smithi132 ceph-mon[35941]: fsmap cephfs:4/5 {0=b=up:active,2=h=up:active,3=j=up:replay,4=f=up:active} 4 up:standby-replay 4 up:standby, 1 failed 2024-03-28T00:33:21.705 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:33:21 smithi132 ceph-mon[35941]: fsmap cephfs:5/5 3 up:standby-replay 4 up:standby 2 up:replay 3 up:active 2024-03-28T00:33:22.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:33:22 smithi165 ceph-mon[36302]: pgmap v224: 97 pgs: 97 active+clean; 236 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 28 op/s 2024-03-28T00:33:22.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:33:22 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.165:6836/823338130,v1:172.21.15.165:6837/823338130] up:resolve 2024-03-28T00:33:22.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:33:22 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.119:6834/2941089726,v1:172.21.15.119:6835/2941089726] up:resolve 2024-03-28T00:33:22.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:33:22 smithi165 ceph-mon[36302]: fsmap cephfs:5/5 3 up:standby-replay 4 up:standby 2 up:resolve 3 up:active 2024-03-28T00:33:22.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:33:22 smithi119 ceph-mon[31518]: pgmap v224: 97 pgs: 97 active+clean; 236 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 28 op/s 2024-03-28T00:33:22.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:33:22 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.165:6836/823338130,v1:172.21.15.165:6837/823338130] up:resolve 2024-03-28T00:33:22.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:33:22 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.119:6834/2941089726,v1:172.21.15.119:6835/2941089726] up:resolve 2024-03-28T00:33:22.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:33:22 smithi119 ceph-mon[31518]: fsmap cephfs:5/5 3 up:standby-replay 4 up:standby 2 up:resolve 3 up:active 2024-03-28T00:33:22.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:33:22 smithi132 ceph-mon[35941]: pgmap v224: 97 pgs: 97 active+clean; 236 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 28 op/s 2024-03-28T00:33:22.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:33:22 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.165:6836/823338130,v1:172.21.15.165:6837/823338130] up:resolve 2024-03-28T00:33:22.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:33:22 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.119:6834/2941089726,v1:172.21.15.119:6835/2941089726] up:resolve 2024-03-28T00:33:22.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:33:22 smithi132 ceph-mon[35941]: fsmap cephfs:5/5 3 up:standby-replay 4 up:standby 2 up:resolve 3 up:active 2024-03-28T00:33:24.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:33:23 smithi119 ceph-mon[31518]: reconnect by client.25355 172.21.15.119:0/3071152482 after 0.00299996 2024-03-28T00:33:24.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:33:23 smithi119 ceph-mon[31518]: reconnect by client.25355 172.21.15.119:0/3071152482 after 0.00299993 2024-03-28T00:33:24.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:33:23 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.165:6836/823338130,v1:172.21.15.165:6837/823338130] up:reconnect 2024-03-28T00:33:24.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:33:23 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.119:6834/2941089726,v1:172.21.15.119:6835/2941089726] up:reconnect 2024-03-28T00:33:24.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:33:23 smithi119 ceph-mon[31518]: fsmap cephfs:5/5 3 up:standby-replay 4 up:standby 2 up:reconnect 3 up:active 2024-03-28T00:33:24.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:33:23 smithi119 ceph-mon[31518]: reconnect by client.25264 v1:172.21.15.119:0/3863511511 after 0.0169997 2024-03-28T00:33:24.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:33:23 smithi119 ceph-mon[31518]: reconnect by client.25264 v1:172.21.15.119:0/3863511511 after 0.0179996 2024-03-28T00:33:24.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:33:23 smithi119 ceph-mon[31518]: pgmap v225: 97 pgs: 97 active+clean; 236 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 21 KiB/s wr, 10 op/s 2024-03-28T00:33:24.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:33:23 smithi132 ceph-mon[35941]: reconnect by client.25355 172.21.15.119:0/3071152482 after 0.00299996 2024-03-28T00:33:24.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:33:23 smithi132 ceph-mon[35941]: reconnect by client.25355 172.21.15.119:0/3071152482 after 0.00299993 2024-03-28T00:33:24.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:33:23 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.165:6836/823338130,v1:172.21.15.165:6837/823338130] up:reconnect 2024-03-28T00:33:24.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:33:23 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.119:6834/2941089726,v1:172.21.15.119:6835/2941089726] up:reconnect 2024-03-28T00:33:24.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:33:23 smithi132 ceph-mon[35941]: fsmap cephfs:5/5 3 up:standby-replay 4 up:standby 2 up:reconnect 3 up:active 2024-03-28T00:33:24.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:33:23 smithi132 ceph-mon[35941]: reconnect by client.25264 v1:172.21.15.119:0/3863511511 after 0.0169997 2024-03-28T00:33:24.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:33:23 smithi132 ceph-mon[35941]: reconnect by client.25264 v1:172.21.15.119:0/3863511511 after 0.0179996 2024-03-28T00:33:24.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:33:23 smithi132 ceph-mon[35941]: pgmap v225: 97 pgs: 97 active+clean; 236 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 21 KiB/s wr, 10 op/s 2024-03-28T00:33:24.384 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:33:23 smithi165 ceph-mon[36302]: reconnect by client.25355 172.21.15.119:0/3071152482 after 0.00299996 2024-03-28T00:33:24.384 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:33:23 smithi165 ceph-mon[36302]: reconnect by client.25355 172.21.15.119:0/3071152482 after 0.00299993 2024-03-28T00:33:24.384 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:33:23 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.165:6836/823338130,v1:172.21.15.165:6837/823338130] up:reconnect 2024-03-28T00:33:24.384 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:33:23 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.119:6834/2941089726,v1:172.21.15.119:6835/2941089726] up:reconnect 2024-03-28T00:33:24.384 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:33:23 smithi165 ceph-mon[36302]: fsmap cephfs:5/5 3 up:standby-replay 4 up:standby 2 up:reconnect 3 up:active 2024-03-28T00:33:24.384 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:33:23 smithi165 ceph-mon[36302]: reconnect by client.25264 v1:172.21.15.119:0/3863511511 after 0.0169997 2024-03-28T00:33:24.384 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:33:23 smithi165 ceph-mon[36302]: reconnect by client.25264 v1:172.21.15.119:0/3863511511 after 0.0179996 2024-03-28T00:33:24.384 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:33:23 smithi165 ceph-mon[36302]: pgmap v225: 97 pgs: 97 active+clean; 236 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 21 KiB/s wr, 10 op/s 2024-03-28T00:33:25.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.165:6836/823338130,v1:172.21.15.165:6837/823338130] up:rejoin 2024-03-28T00:33:25.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.119:6834/2941089726,v1:172.21.15.119:6835/2941089726] up:rejoin 2024-03-28T00:33:25.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-mon[36302]: fsmap cephfs:5/5 3 up:standby-replay 4 up:standby 2 up:rejoin 3 up:active 2024-03-28T00:33:25.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-mon[36302]: daemon mds.c is now active in filesystem cephfs as rank 1 2024-03-28T00:33:25.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-mon[36302]: daemon mds.j is now active in filesystem cephfs as rank 3 2024-03-28T00:33:25.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-mon[36302]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-28T00:33:25.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-mon[36302]: Cluster is now healthy 2024-03-28T00:33:25.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.165:6836/823338130,v1:172.21.15.165:6837/823338130] up:active 2024-03-28T00:33:25.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.119:6834/2941089726,v1:172.21.15.119:6835/2941089726] up:active 2024-03-28T00:33:25.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-mon[36302]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-28T00:33:25.135 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7f5dc9950700 time 2024-03-28T00:33:24.837343+0000 2024-03-28T00:33:25.135 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T00:33:25.135 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:33:25.135 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f5dd3f7a4a1] 2024-03-28T00:33:25.135 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f5dd3f7a667] 2024-03-28T00:33:25.135 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x5595b71fe223] 2024-03-28T00:33:25.135 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x5595b70785d6] 2024-03-28T00:33:25.135 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x5595b7078da1] 2024-03-28T00:33:25.135 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: 6: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x5595b70ad09c] 2024-03-28T00:33:25.136 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x5595b70cb233] 2024-03-28T00:33:25.136 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5595b70cd197] 2024-03-28T00:33:25.136 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x5595b6fcc11b] 2024-03-28T00:33:25.136 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5595b6fcc6bc] 2024-03-28T00:33:25.136 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x5595b6fb61ff] 2024-03-28T00:33:25.136 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f5dd420c488] 2024-03-28T00:33:25.136 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: 13: (DispatchQueue::entry()+0x51f) [0x7f5dd420960f] 2024-03-28T00:33:25.136 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f5dd42dce81] 2024-03-28T00:33:25.136 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f5dd2cf71ca] 2024-03-28T00:33:25.136 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: 16: clone() 2024-03-28T00:33:25.137 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: 2024-03-28T00:33:24.837+0000 7f5dc9950700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7f5dc9950700 time 2024-03-28T00:33:24.837343+0000 2024-03-28T00:33:25.137 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T00:33:25.137 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: 2024-03-28T00:33:25.137 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:33:25.137 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f5dd3f7a4a1] 2024-03-28T00:33:25.137 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f5dd3f7a667] 2024-03-28T00:33:25.137 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x5595b71fe223] 2024-03-28T00:33:25.137 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x5595b70785d6] 2024-03-28T00:33:25.137 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x5595b7078da1] 2024-03-28T00:33:25.137 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: 6: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x5595b70ad09c] 2024-03-28T00:33:25.137 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x5595b70cb233] 2024-03-28T00:33:25.137 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5595b70cd197] 2024-03-28T00:33:25.137 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x5595b6fcc11b] 2024-03-28T00:33:25.138 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5595b6fcc6bc] 2024-03-28T00:33:25.138 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x5595b6fb61ff] 2024-03-28T00:33:25.138 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f5dd420c488] 2024-03-28T00:33:25.138 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: 13: (DispatchQueue::entry()+0x51f) [0x7f5dd420960f] 2024-03-28T00:33:25.138 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f5dd42dce81] 2024-03-28T00:33:25.138 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f5dd2cf71ca] 2024-03-28T00:33:25.138 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: 16: clone() 2024-03-28T00:33:25.138 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: 2024-03-28T00:33:25.138 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: *** Caught signal (Aborted) ** 2024-03-28T00:33:25.138 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: in thread 7f5dc9950700 thread_name:ms_dispatch 2024-03-28T00:33:25.138 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:33:25.138 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f5dd2d01d20] 2024-03-28T00:33:25.138 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: 2: gsignal() 2024-03-28T00:33:25.139 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: 3: abort() 2024-03-28T00:33:25.139 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f5dd3f7a4fb] 2024-03-28T00:33:25.139 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f5dd3f7a667] 2024-03-28T00:33:25.139 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x5595b71fe223] 2024-03-28T00:33:25.139 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x5595b70785d6] 2024-03-28T00:33:25.139 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x5595b7078da1] 2024-03-28T00:33:25.139 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: 9: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x5595b70ad09c] 2024-03-28T00:33:25.139 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x5595b70cb233] 2024-03-28T00:33:25.139 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5595b70cd197] 2024-03-28T00:33:25.139 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x5595b6fcc11b] 2024-03-28T00:33:25.139 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5595b6fcc6bc] 2024-03-28T00:33:25.139 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x5595b6fb61ff] 2024-03-28T00:33:25.139 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f5dd420c488] 2024-03-28T00:33:25.139 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: 16: (DispatchQueue::entry()+0x51f) [0x7f5dd420960f] 2024-03-28T00:33:25.139 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f5dd42dce81] 2024-03-28T00:33:25.140 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f5dd2cf71ca] 2024-03-28T00:33:25.140 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: 19: clone() 2024-03-28T00:33:25.140 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: 2024-03-28T00:33:24.839+0000 7f5dc9950700 -1 *** Caught signal (Aborted) ** 2024-03-28T00:33:25.140 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: in thread 7f5dc9950700 thread_name:ms_dispatch 2024-03-28T00:33:25.140 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: 2024-03-28T00:33:25.140 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:33:25.140 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f5dd2d01d20] 2024-03-28T00:33:25.140 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: 2: gsignal() 2024-03-28T00:33:25.140 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: 3: abort() 2024-03-28T00:33:25.140 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f5dd3f7a4fb] 2024-03-28T00:33:25.140 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f5dd3f7a667] 2024-03-28T00:33:25.141 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x5595b71fe223] 2024-03-28T00:33:25.141 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x5595b70785d6] 2024-03-28T00:33:25.141 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x5595b7078da1] 2024-03-28T00:33:25.141 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: 9: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x5595b70ad09c] 2024-03-28T00:33:25.141 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x5595b70cb233] 2024-03-28T00:33:25.141 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5595b70cd197] 2024-03-28T00:33:25.142 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x5595b6fcc11b] 2024-03-28T00:33:25.142 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5595b6fcc6bc] 2024-03-28T00:33:25.142 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x5595b6fb61ff] 2024-03-28T00:33:25.142 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f5dd420c488] 2024-03-28T00:33:25.142 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: 16: (DispatchQueue::entry()+0x51f) [0x7f5dd420960f] 2024-03-28T00:33:25.142 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f5dd42dce81] 2024-03-28T00:33:25.142 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f5dd2cf71ca] 2024-03-28T00:33:25.142 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: 19: clone() 2024-03-28T00:33:25.142 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T00:33:25.142 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: 2024-03-28T00:33:25.142 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: -1> 2024-03-28T00:33:24.837+0000 7f5dc9950700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7f5dc9950700 time 2024-03-28T00:33:24.837343+0000 2024-03-28T00:33:25.142 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T00:33:25.142 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: 2024-03-28T00:33:25.143 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:33:25.143 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f5dd3f7a4a1] 2024-03-28T00:33:25.143 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f5dd3f7a667] 2024-03-28T00:33:25.143 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x5595b71fe223] 2024-03-28T00:33:25.143 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x5595b70785d6] 2024-03-28T00:33:25.143 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x5595b7078da1] 2024-03-28T00:33:25.143 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: 6: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x5595b70ad09c] 2024-03-28T00:33:25.143 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x5595b70cb233] 2024-03-28T00:33:25.143 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5595b70cd197] 2024-03-28T00:33:25.143 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x5595b6fcc11b] 2024-03-28T00:33:25.143 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5595b6fcc6bc] 2024-03-28T00:33:25.143 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x5595b6fb61ff] 2024-03-28T00:33:25.143 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f5dd420c488] 2024-03-28T00:33:25.143 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: 13: (DispatchQueue::entry()+0x51f) [0x7f5dd420960f] 2024-03-28T00:33:25.144 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f5dd42dce81] 2024-03-28T00:33:25.144 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f5dd2cf71ca] 2024-03-28T00:33:25.144 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: 16: clone() 2024-03-28T00:33:25.144 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: 2024-03-28T00:33:25.144 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: 0> 2024-03-28T00:33:24.839+0000 7f5dc9950700 -1 *** Caught signal (Aborted) ** 2024-03-28T00:33:25.144 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: in thread 7f5dc9950700 thread_name:ms_dispatch 2024-03-28T00:33:25.144 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: 2024-03-28T00:33:25.144 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:33:25.144 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f5dd2d01d20] 2024-03-28T00:33:25.144 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: 2: gsignal() 2024-03-28T00:33:25.144 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: 3: abort() 2024-03-28T00:33:25.144 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f5dd3f7a4fb] 2024-03-28T00:33:25.144 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f5dd3f7a667] 2024-03-28T00:33:25.144 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x5595b71fe223] 2024-03-28T00:33:25.144 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x5595b70785d6] 2024-03-28T00:33:25.145 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x5595b7078da1] 2024-03-28T00:33:25.145 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: 9: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x5595b70ad09c] 2024-03-28T00:33:25.145 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x5595b70cb233] 2024-03-28T00:33:25.145 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5595b70cd197] 2024-03-28T00:33:25.145 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x5595b6fcc11b] 2024-03-28T00:33:25.145 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5595b6fcc6bc] 2024-03-28T00:33:25.145 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x5595b6fb61ff] 2024-03-28T00:33:25.145 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f5dd420c488] 2024-03-28T00:33:25.145 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: 16: (DispatchQueue::entry()+0x51f) [0x7f5dd420960f] 2024-03-28T00:33:25.145 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f5dd42dce81] 2024-03-28T00:33:25.145 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f5dd2cf71ca] 2024-03-28T00:33:25.146 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: 19: clone() 2024-03-28T00:33:25.146 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T00:33:25.146 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: 2024-03-28T00:33:25.146 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: -1> 2024-03-28T00:33:24.837+0000 7f5dc9950700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7f5dc9950700 time 2024-03-28T00:33:24.837343+0000 2024-03-28T00:33:25.146 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T00:33:25.146 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: 2024-03-28T00:33:25.146 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:33:25.146 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f5dd3f7a4a1] 2024-03-28T00:33:25.146 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f5dd3f7a667] 2024-03-28T00:33:25.146 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x5595b71fe223] 2024-03-28T00:33:25.146 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x5595b70785d6] 2024-03-28T00:33:25.146 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x5595b7078da1] 2024-03-28T00:33:25.146 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: 6: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x5595b70ad09c] 2024-03-28T00:33:25.146 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x5595b70cb233] 2024-03-28T00:33:25.147 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5595b70cd197] 2024-03-28T00:33:25.147 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x5595b6fcc11b] 2024-03-28T00:33:25.147 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5595b6fcc6bc] 2024-03-28T00:33:25.147 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x5595b6fb61ff] 2024-03-28T00:33:25.147 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f5dd420c488] 2024-03-28T00:33:25.147 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: 13: (DispatchQueue::entry()+0x51f) [0x7f5dd420960f] 2024-03-28T00:33:25.147 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f5dd42dce81] 2024-03-28T00:33:25.147 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f5dd2cf71ca] 2024-03-28T00:33:25.147 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: 16: clone() 2024-03-28T00:33:25.147 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: 2024-03-28T00:33:25.147 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: 0> 2024-03-28T00:33:24.839+0000 7f5dc9950700 -1 *** Caught signal (Aborted) ** 2024-03-28T00:33:25.147 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: in thread 7f5dc9950700 thread_name:ms_dispatch 2024-03-28T00:33:25.147 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: 2024-03-28T00:33:25.147 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:33:25.148 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f5dd2d01d20] 2024-03-28T00:33:25.148 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: 2: gsignal() 2024-03-28T00:33:25.148 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: 3: abort() 2024-03-28T00:33:25.148 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f5dd3f7a4fb] 2024-03-28T00:33:25.148 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f5dd3f7a667] 2024-03-28T00:33:25.148 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x5595b71fe223] 2024-03-28T00:33:25.148 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x5595b70785d6] 2024-03-28T00:33:25.148 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x5595b7078da1] 2024-03-28T00:33:25.148 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: 9: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x5595b70ad09c] 2024-03-28T00:33:25.148 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x5595b70cb233] 2024-03-28T00:33:25.148 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5595b70cd197] 2024-03-28T00:33:25.148 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x5595b6fcc11b] 2024-03-28T00:33:25.148 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5595b6fcc6bc] 2024-03-28T00:33:25.148 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x5595b6fb61ff] 2024-03-28T00:33:25.149 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f5dd420c488] 2024-03-28T00:33:25.149 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: 16: (DispatchQueue::entry()+0x51f) [0x7f5dd420960f] 2024-03-28T00:33:25.149 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f5dd42dce81] 2024-03-28T00:33:25.149 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f5dd2cf71ca] 2024-03-28T00:33:25.149 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: 19: clone() 2024-03-28T00:33:25.149 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T00:33:25.149 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c[73564]: 2024-03-28T00:33:25.149 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:24 smithi165 podman[73907]: 2024-03-28 00:33:24.896894212 +0000 UTC m=+0.021481981 container died e50f2f72ad4921599196dc06f38bf8e6166fd6c2714c6986f461bc803fd924a9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319) 2024-03-28T00:33:25.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.165:6836/823338130,v1:172.21.15.165:6837/823338130] up:rejoin 2024-03-28T00:33:25.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.119:6834/2941089726,v1:172.21.15.119:6835/2941089726] up:rejoin 2024-03-28T00:33:25.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-mon[31518]: fsmap cephfs:5/5 3 up:standby-replay 4 up:standby 2 up:rejoin 3 up:active 2024-03-28T00:33:25.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-mon[31518]: daemon mds.c is now active in filesystem cephfs as rank 1 2024-03-28T00:33:25.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-mon[31518]: daemon mds.j is now active in filesystem cephfs as rank 3 2024-03-28T00:33:25.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-mon[31518]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-28T00:33:25.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-mon[31518]: Cluster is now healthy 2024-03-28T00:33:25.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.165:6836/823338130,v1:172.21.15.165:6837/823338130] up:active 2024-03-28T00:33:25.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.119:6834/2941089726,v1:172.21.15.119:6835/2941089726] up:active 2024-03-28T00:33:25.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-mon[31518]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-28T00:33:25.157 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7f50e4c00700 time 2024-03-28T00:33:24.844450+0000 2024-03-28T00:33:25.157 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T00:33:25.157 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:33:25.157 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f50f04db4a1] 2024-03-28T00:33:25.158 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f50f04db667] 2024-03-28T00:33:25.158 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x5564946ec223] 2024-03-28T00:33:25.158 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x5564945665d6] 2024-03-28T00:33:25.158 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x556494566da1] 2024-03-28T00:33:25.158 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: 6: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x55649459b09c] 2024-03-28T00:33:25.158 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x5564945b9233] 2024-03-28T00:33:25.158 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5564945bb197] 2024-03-28T00:33:25.158 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x5564944ba11b] 2024-03-28T00:33:25.158 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5564944ba6bc] 2024-03-28T00:33:25.158 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x5564944a41ff] 2024-03-28T00:33:25.158 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f50f076d488] 2024-03-28T00:33:25.158 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: 13: (DispatchQueue::entry()+0x51f) [0x7f50f076a60f] 2024-03-28T00:33:25.158 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f50f083de81] 2024-03-28T00:33:25.159 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f50ef2581ca] 2024-03-28T00:33:25.159 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: 16: clone() 2024-03-28T00:33:25.159 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: *** Caught signal (Aborted) ** 2024-03-28T00:33:25.159 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: in thread 7f50e4c00700 thread_name:ms_dispatch 2024-03-28T00:33:25.159 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: 2024-03-28T00:33:24.845+0000 7f50e4c00700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7f50e4c00700 time 2024-03-28T00:33:24.844450+0000 2024-03-28T00:33:25.159 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T00:33:25.159 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: 2024-03-28T00:33:25.159 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:33:25.159 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f50f04db4a1] 2024-03-28T00:33:25.159 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f50f04db667] 2024-03-28T00:33:25.159 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x5564946ec223] 2024-03-28T00:33:25.159 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x5564945665d6] 2024-03-28T00:33:25.159 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x556494566da1] 2024-03-28T00:33:25.159 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: 6: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x55649459b09c] 2024-03-28T00:33:25.159 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x5564945b9233] 2024-03-28T00:33:25.160 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5564945bb197] 2024-03-28T00:33:25.160 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x5564944ba11b] 2024-03-28T00:33:25.160 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5564944ba6bc] 2024-03-28T00:33:25.160 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x5564944a41ff] 2024-03-28T00:33:25.160 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f50f076d488] 2024-03-28T00:33:25.160 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: 13: (DispatchQueue::entry()+0x51f) [0x7f50f076a60f] 2024-03-28T00:33:25.160 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f50f083de81] 2024-03-28T00:33:25.160 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f50ef2581ca] 2024-03-28T00:33:25.160 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: 16: clone() 2024-03-28T00:33:25.160 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: 2024-03-28T00:33:25.160 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:33:25.160 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f50ef262d20] 2024-03-28T00:33:25.161 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: 2: gsignal() 2024-03-28T00:33:25.161 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: 3: abort() 2024-03-28T00:33:25.161 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f50f04db4fb] 2024-03-28T00:33:25.162 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f50f04db667] 2024-03-28T00:33:25.162 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x5564946ec223] 2024-03-28T00:33:25.162 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x5564945665d6] 2024-03-28T00:33:25.162 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x556494566da1] 2024-03-28T00:33:25.162 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: 9: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x55649459b09c] 2024-03-28T00:33:25.162 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x5564945b9233] 2024-03-28T00:33:25.162 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5564945bb197] 2024-03-28T00:33:25.162 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x5564944ba11b] 2024-03-28T00:33:25.162 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5564944ba6bc] 2024-03-28T00:33:25.162 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x5564944a41ff] 2024-03-28T00:33:25.162 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f50f076d488] 2024-03-28T00:33:25.162 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: 16: (DispatchQueue::entry()+0x51f) [0x7f50f076a60f] 2024-03-28T00:33:25.162 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f50f083de81] 2024-03-28T00:33:25.162 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f50ef2581ca] 2024-03-28T00:33:25.163 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: 19: clone() 2024-03-28T00:33:25.163 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: 2024-03-28T00:33:24.847+0000 7f50e4c00700 -1 *** Caught signal (Aborted) ** 2024-03-28T00:33:25.163 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: in thread 7f50e4c00700 thread_name:ms_dispatch 2024-03-28T00:33:25.163 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: 2024-03-28T00:33:25.163 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:33:25.163 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f50ef262d20] 2024-03-28T00:33:25.163 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: 2: gsignal() 2024-03-28T00:33:25.163 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: 3: abort() 2024-03-28T00:33:25.163 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f50f04db4fb] 2024-03-28T00:33:25.163 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f50f04db667] 2024-03-28T00:33:25.163 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x5564946ec223] 2024-03-28T00:33:25.163 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x5564945665d6] 2024-03-28T00:33:25.163 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x556494566da1] 2024-03-28T00:33:25.163 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: 9: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x55649459b09c] 2024-03-28T00:33:25.163 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x5564945b9233] 2024-03-28T00:33:25.164 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5564945bb197] 2024-03-28T00:33:25.164 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x5564944ba11b] 2024-03-28T00:33:25.164 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5564944ba6bc] 2024-03-28T00:33:25.164 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x5564944a41ff] 2024-03-28T00:33:25.164 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f50f076d488] 2024-03-28T00:33:25.164 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: 16: (DispatchQueue::entry()+0x51f) [0x7f50f076a60f] 2024-03-28T00:33:25.164 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f50f083de81] 2024-03-28T00:33:25.164 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f50ef2581ca] 2024-03-28T00:33:25.164 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: 19: clone() 2024-03-28T00:33:25.164 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T00:33:25.164 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: 2024-03-28T00:33:25.164 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: -1> 2024-03-28T00:33:24.845+0000 7f50e4c00700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7f50e4c00700 time 2024-03-28T00:33:24.844450+0000 2024-03-28T00:33:25.164 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T00:33:25.164 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: 2024-03-28T00:33:25.164 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:33:25.165 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f50f04db4a1] 2024-03-28T00:33:25.165 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f50f04db667] 2024-03-28T00:33:25.165 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x5564946ec223] 2024-03-28T00:33:25.165 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x5564945665d6] 2024-03-28T00:33:25.165 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x556494566da1] 2024-03-28T00:33:25.165 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: 6: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x55649459b09c] 2024-03-28T00:33:25.165 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x5564945b9233] 2024-03-28T00:33:25.165 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5564945bb197] 2024-03-28T00:33:25.165 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x5564944ba11b] 2024-03-28T00:33:25.165 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5564944ba6bc] 2024-03-28T00:33:25.165 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x5564944a41ff] 2024-03-28T00:33:25.165 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f50f076d488] 2024-03-28T00:33:25.165 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: 13: (DispatchQueue::entry()+0x51f) [0x7f50f076a60f] 2024-03-28T00:33:25.165 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f50f083de81] 2024-03-28T00:33:25.166 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f50ef2581ca] 2024-03-28T00:33:25.166 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: 16: clone() 2024-03-28T00:33:25.166 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: 2024-03-28T00:33:25.166 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: 0> 2024-03-28T00:33:24.847+0000 7f50e4c00700 -1 *** Caught signal (Aborted) ** 2024-03-28T00:33:25.166 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: in thread 7f50e4c00700 thread_name:ms_dispatch 2024-03-28T00:33:25.166 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: 2024-03-28T00:33:25.166 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:33:25.166 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f50ef262d20] 2024-03-28T00:33:25.166 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: 2: gsignal() 2024-03-28T00:33:25.166 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: 3: abort() 2024-03-28T00:33:25.166 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f50f04db4fb] 2024-03-28T00:33:25.166 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f50f04db667] 2024-03-28T00:33:25.166 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x5564946ec223] 2024-03-28T00:33:25.166 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x5564945665d6] 2024-03-28T00:33:25.166 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x556494566da1] 2024-03-28T00:33:25.167 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: 9: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x55649459b09c] 2024-03-28T00:33:25.167 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x5564945b9233] 2024-03-28T00:33:25.167 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5564945bb197] 2024-03-28T00:33:25.167 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x5564944ba11b] 2024-03-28T00:33:25.167 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5564944ba6bc] 2024-03-28T00:33:25.167 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x5564944a41ff] 2024-03-28T00:33:25.167 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f50f076d488] 2024-03-28T00:33:25.167 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: 16: (DispatchQueue::entry()+0x51f) [0x7f50f076a60f] 2024-03-28T00:33:25.167 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f50f083de81] 2024-03-28T00:33:25.167 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f50ef2581ca] 2024-03-28T00:33:25.167 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: 19: clone() 2024-03-28T00:33:25.167 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T00:33:25.167 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: 2024-03-28T00:33:25.167 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: -5> 2024-03-28T00:33:24.845+0000 7f50e4c00700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7f50e4c00700 time 2024-03-28T00:33:24.844450+0000 2024-03-28T00:33:25.167 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T00:33:25.168 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: 2024-03-28T00:33:25.168 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:33:25.168 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f50f04db4a1] 2024-03-28T00:33:25.168 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f50f04db667] 2024-03-28T00:33:25.168 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x5564946ec223] 2024-03-28T00:33:25.168 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x5564945665d6] 2024-03-28T00:33:25.168 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x556494566da1] 2024-03-28T00:33:25.168 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: 6: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x55649459b09c] 2024-03-28T00:33:25.168 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x5564945b9233] 2024-03-28T00:33:25.168 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5564945bb197] 2024-03-28T00:33:25.168 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x5564944ba11b] 2024-03-28T00:33:25.168 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5564944ba6bc] 2024-03-28T00:33:25.168 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x5564944a41ff] 2024-03-28T00:33:25.168 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f50f076d488] 2024-03-28T00:33:25.168 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: 13: (DispatchQueue::entry()+0x51f) [0x7f50f076a60f] 2024-03-28T00:33:25.169 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f50f083de81] 2024-03-28T00:33:25.169 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f50ef2581ca] 2024-03-28T00:33:25.169 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: 16: clone() 2024-03-28T00:33:25.169 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: 2024-03-28T00:33:25.169 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: -4> 2024-03-28T00:33:24.847+0000 7f50e4c00700 -1 *** Caught signal (Aborted) ** 2024-03-28T00:33:25.169 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: in thread 7f50e4c00700 thread_name:ms_dispatch 2024-03-28T00:33:25.169 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: 2024-03-28T00:33:25.169 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:33:25.169 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f50ef262d20] 2024-03-28T00:33:25.169 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: 2: gsignal() 2024-03-28T00:33:25.169 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: 3: abort() 2024-03-28T00:33:25.169 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f50f04db4fb] 2024-03-28T00:33:25.169 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f50f04db667] 2024-03-28T00:33:25.169 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x5564946ec223] 2024-03-28T00:33:25.169 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x5564945665d6] 2024-03-28T00:33:25.170 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x556494566da1] 2024-03-28T00:33:25.170 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: 9: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x55649459b09c] 2024-03-28T00:33:25.170 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x5564945b9233] 2024-03-28T00:33:25.170 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5564945bb197] 2024-03-28T00:33:25.170 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x5564944ba11b] 2024-03-28T00:33:25.170 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5564944ba6bc] 2024-03-28T00:33:25.170 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x5564944a41ff] 2024-03-28T00:33:25.170 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f50f076d488] 2024-03-28T00:33:25.170 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: 16: (DispatchQueue::entry()+0x51f) [0x7f50f076a60f] 2024-03-28T00:33:25.170 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f50f083de81] 2024-03-28T00:33:25.170 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f50ef2581ca] 2024-03-28T00:33:25.170 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: 19: clone() 2024-03-28T00:33:25.170 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T00:33:25.173 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[113762]: 2024-03-28T00:33:25.173 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:24 smithi119 podman[114176]: 2024-03-28 00:33:24.980463602 +0000 UTC m=+0.038751550 container died f519af739077e8d2209d74929537c0431b390e73dc83b35975867db4429386fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-28T00:33:25.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:33:24 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.165:6836/823338130,v1:172.21.15.165:6837/823338130] up:rejoin 2024-03-28T00:33:25.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:33:24 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.119:6834/2941089726,v1:172.21.15.119:6835/2941089726] up:rejoin 2024-03-28T00:33:25.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:33:24 smithi132 ceph-mon[35941]: fsmap cephfs:5/5 3 up:standby-replay 4 up:standby 2 up:rejoin 3 up:active 2024-03-28T00:33:25.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:33:24 smithi132 ceph-mon[35941]: daemon mds.c is now active in filesystem cephfs as rank 1 2024-03-28T00:33:25.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:33:24 smithi132 ceph-mon[35941]: daemon mds.j is now active in filesystem cephfs as rank 3 2024-03-28T00:33:25.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:33:24 smithi132 ceph-mon[35941]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-28T00:33:25.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:33:24 smithi132 ceph-mon[35941]: Cluster is now healthy 2024-03-28T00:33:25.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:33:24 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.165:6836/823338130,v1:172.21.15.165:6837/823338130] up:active 2024-03-28T00:33:25.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:33:24 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.119:6834/2941089726,v1:172.21.15.119:6835/2941089726] up:active 2024-03-28T00:33:25.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:33:24 smithi132 ceph-mon[35941]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-28T00:33:25.850 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:25 smithi165 podman[73907]: 2024-03-28 00:33:25.578825266 +0000 UTC m=+0.703413039 container remove e50f2f72ad4921599196dc06f38bf8e6166fd6c2714c6986f461bc803fd924a9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-28T00:33:25.850 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:25 smithi165 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.c.service: Main process exited, code=exited, status=134/n/a 2024-03-28T00:33:26.075 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:25 smithi119 podman[114176]: 2024-03-28 00:33:25.814034259 +0000 UTC m=+0.872322199 container remove f519af739077e8d2209d74929537c0431b390e73dc83b35975867db4429386fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j, GIT_CLEAN=True, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-28T00:33:26.075 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:25 smithi119 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.j.service: Main process exited, code=exited, status=134/n/a 2024-03-28T00:33:26.076 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:33:25 smithi119 ceph-mon[31518]: pgmap v226: 97 pgs: 97 active+clean; 238 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 189 KiB/s wr, 24 op/s 2024-03-28T00:33:26.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:33:25 smithi165 ceph-mon[36302]: pgmap v226: 97 pgs: 97 active+clean; 238 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 189 KiB/s wr, 24 op/s 2024-03-28T00:33:26.134 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:25 smithi165 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.c.service: Failed with result 'exit-code'. 2024-03-28T00:33:26.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:33:25 smithi132 ceph-mon[35941]: pgmap v226: 97 pgs: 97 active+clean; 238 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 189 KiB/s wr, 24 op/s 2024-03-28T00:33:26.656 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:26 smithi119 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.j.service: Failed with result 'exit-code'. 2024-03-28T00:33:26.657 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:26 smithi119 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.j.service: Consumed 1.640s CPU time. 2024-03-28T00:33:27.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:33:26 smithi119 ceph-mon[31518]: osdmap e106: 12 total, 12 up, 12 in 2024-03-28T00:33:27.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:33:26 smithi132 ceph-mon[35941]: osdmap e106: 12 total, 12 up, 12 in 2024-03-28T00:33:27.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:33:26 smithi165 ceph-mon[36302]: osdmap e106: 12 total, 12 up, 12 in 2024-03-28T00:33:28.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:33:27 smithi132 ceph-mon[35941]: pgmap v228: 97 pgs: 97 active+clean; 238 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 194 KiB/s wr, 15 op/s 2024-03-28T00:33:28.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:33:27 smithi165 ceph-mon[36302]: pgmap v228: 97 pgs: 97 active+clean; 238 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 194 KiB/s wr, 15 op/s 2024-03-28T00:33:28.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:33:27 smithi119 ceph-mon[31518]: pgmap v228: 97 pgs: 97 active+clean; 238 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 194 KiB/s wr, 15 op/s 2024-03-28T00:33:30.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:33:30 smithi165 ceph-mon[36302]: pgmap v229: 97 pgs: 97 active+clean; 244 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 133 op/s 2024-03-28T00:33:30.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:33:30 smithi165 ceph-mon[36302]: osdmap e107: 12 total, 12 up, 12 in 2024-03-28T00:33:30.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:33:30 smithi119 ceph-mon[31518]: pgmap v229: 97 pgs: 97 active+clean; 244 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 133 op/s 2024-03-28T00:33:30.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:33:30 smithi119 ceph-mon[31518]: osdmap e107: 12 total, 12 up, 12 in 2024-03-28T00:33:30.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:33:30 smithi132 ceph-mon[35941]: pgmap v229: 97 pgs: 97 active+clean; 244 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 133 op/s 2024-03-28T00:33:30.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:33:30 smithi132 ceph-mon[35941]: osdmap e107: 12 total, 12 up, 12 in 2024-03-28T00:33:32.506 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:33:32 smithi132 ceph-mon[35941]: pgmap v231: 97 pgs: 97 active+clean; 244 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.1 MiB/s wr, 138 op/s 2024-03-28T00:33:32.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:33:32 smithi165 ceph-mon[36302]: pgmap v231: 97 pgs: 97 active+clean; 244 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.1 MiB/s wr, 138 op/s 2024-03-28T00:33:32.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:33:32 smithi119 ceph-mon[31518]: pgmap v231: 97 pgs: 97 active+clean; 244 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.1 MiB/s wr, 138 op/s 2024-03-28T00:33:34.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:33:34 smithi165 ceph-mon[36302]: pgmap v232: 97 pgs: 97 active+clean; 244 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 963 KiB/s wr, 125 op/s 2024-03-28T00:33:34.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:33:34 smithi119 ceph-mon[31518]: pgmap v232: 97 pgs: 97 active+clean; 244 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 963 KiB/s wr, 125 op/s 2024-03-28T00:33:34.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:33:34 smithi132 ceph-mon[35941]: pgmap v232: 97 pgs: 97 active+clean; 244 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 963 KiB/s wr, 125 op/s 2024-03-28T00:33:36.238 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:35 smithi165 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.c.service: Scheduled restart job, restart counter is at 6. 2024-03-28T00:33:36.239 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:35 smithi165 systemd[1]: Stopped Ceph mds.c for 10f73b92-ec98-11ee-95d0-87774f69a715. 2024-03-28T00:33:36.239 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:35 smithi165 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.c.service: Start request repeated too quickly. 2024-03-28T00:33:36.239 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:35 smithi165 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.c.service: Failed with result 'exit-code'. 2024-03-28T00:33:36.239 INFO:journalctl@ceph.mds.c.smithi165.stdout:Mar 28 00:33:35 smithi165 systemd[1]: Failed to start Ceph mds.c for 10f73b92-ec98-11ee-95d0-87774f69a715. 2024-03-28T00:33:36.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:33:36 smithi165 ceph-mon[36302]: pgmap v233: 97 pgs: 97 active+clean; 246 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 964 KiB/s wr, 115 op/s 2024-03-28T00:33:36.656 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:36 smithi119 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.j.service: Scheduled restart job, restart counter is at 4. 2024-03-28T00:33:36.656 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:36 smithi119 systemd[1]: Stopped Ceph mds.j for 10f73b92-ec98-11ee-95d0-87774f69a715. 2024-03-28T00:33:36.656 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:36 smithi119 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.j.service: Consumed 1.640s CPU time. 2024-03-28T00:33:36.657 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:36 smithi119 systemd[1]: Starting Ceph mds.j for 10f73b92-ec98-11ee-95d0-87774f69a715... 2024-03-28T00:33:36.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:33:36 smithi119 ceph-mon[31518]: pgmap v233: 97 pgs: 97 active+clean; 246 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 964 KiB/s wr, 115 op/s 2024-03-28T00:33:36.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:33:36 smithi132 ceph-mon[35941]: pgmap v233: 97 pgs: 97 active+clean; 246 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 964 KiB/s wr, 115 op/s 2024-03-28T00:33:37.080 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:36 smithi119 podman[114737]: 2024-03-28 00:33:36.71870265 +0000 UTC m=+0.033236383 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-28T00:33:37.080 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:36 smithi119 podman[114737]: 2024-03-28 00:33:36.840302507 +0000 UTC m=+0.154836263 container create e87ff1b71b8829ec0814a259d6bd7ef381f0bddd2e660745ae00aa8d5a12ec27 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319) 2024-03-28T00:33:37.657 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:37 smithi119 podman[114737]: 2024-03-28 00:33:37.169521202 +0000 UTC m=+0.484055032 container init e87ff1b71b8829ec0814a259d6bd7ef381f0bddd2e660745ae00aa8d5a12ec27 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j, org.label-schema.build-date=20240319, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-03-28T00:33:37.657 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:37 smithi119 podman[114737]: 2024-03-28 00:33:37.179105327 +0000 UTC m=+0.493639086 container start e87ff1b71b8829ec0814a259d6bd7ef381f0bddd2e660745ae00aa8d5a12ec27 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j, ceph=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-28T00:33:37.657 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:37 smithi119 ceph-mds[114755]: -- 172.21.15.119:0/755227831 <== mon.1 v2:172.21.15.165:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55cf45ef1a40 con 0x55cf45ec4800 2024-03-28T00:33:37.657 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: starting mds.j at 2024-03-28T00:33:37.658 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:37 smithi119 bash[114737]: e87ff1b71b8829ec0814a259d6bd7ef381f0bddd2e660745ae00aa8d5a12ec27 2024-03-28T00:33:37.658 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:33:37 smithi119 systemd[1]: Started Ceph mds.j for 10f73b92-ec98-11ee-95d0-87774f69a715. 2024-03-28T00:33:38.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:33:38 smithi165 ceph-mon[36302]: pgmap v234: 97 pgs: 97 active+clean; 246 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 893 KiB/s wr, 107 op/s 2024-03-28T00:33:38.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:33:38 smithi165 ceph-mon[36302]: daemon mds.j restarted 2024-03-28T00:33:38.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:33:38 smithi165 ceph-mon[36302]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-28T00:33:38.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:33:38 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-28T00:33:38.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:33:38 smithi165 ceph-mon[36302]: osdmap e108: 12 total, 12 up, 12 in 2024-03-28T00:33:38.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:33:38 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.119:6834/1659925849,v1:172.21.15.119:6835/1659925849] up:boot 2024-03-28T00:33:38.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:33:38 smithi165 ceph-mon[36302]: Standby daemon mds.i assigned to filesystem cephfs as rank 3 2024-03-28T00:33:38.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:33:38 smithi165 ceph-mon[36302]: fsmap cephfs:4/5 {0=b=up:active,1=c=up:active,2=h=up:active,4=f=up:active} 5 up:standby-replay 3 up:standby, 1 failed 2024-03-28T00:33:38.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:33:38 smithi165 ceph-mon[36302]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:replay 4 up:active 2024-03-28T00:33:38.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:33:38 smithi119 ceph-mon[31518]: pgmap v234: 97 pgs: 97 active+clean; 246 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 893 KiB/s wr, 107 op/s 2024-03-28T00:33:38.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:33:38 smithi119 ceph-mon[31518]: daemon mds.j restarted 2024-03-28T00:33:38.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:33:38 smithi119 ceph-mon[31518]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-28T00:33:38.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:33:38 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-28T00:33:38.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:33:38 smithi119 ceph-mon[31518]: osdmap e108: 12 total, 12 up, 12 in 2024-03-28T00:33:38.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:33:38 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.119:6834/1659925849,v1:172.21.15.119:6835/1659925849] up:boot 2024-03-28T00:33:38.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:33:38 smithi119 ceph-mon[31518]: Standby daemon mds.i assigned to filesystem cephfs as rank 3 2024-03-28T00:33:38.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:33:38 smithi119 ceph-mon[31518]: fsmap cephfs:4/5 {0=b=up:active,1=c=up:active,2=h=up:active,4=f=up:active} 5 up:standby-replay 3 up:standby, 1 failed 2024-03-28T00:33:38.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:33:38 smithi119 ceph-mon[31518]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:replay 4 up:active 2024-03-28T00:33:38.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:33:38 smithi132 ceph-mon[35941]: pgmap v234: 97 pgs: 97 active+clean; 246 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 893 KiB/s wr, 107 op/s 2024-03-28T00:33:38.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:33:38 smithi132 ceph-mon[35941]: daemon mds.j restarted 2024-03-28T00:33:38.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:33:38 smithi132 ceph-mon[35941]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-28T00:33:38.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:33:38 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-28T00:33:38.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:33:38 smithi132 ceph-mon[35941]: osdmap e108: 12 total, 12 up, 12 in 2024-03-28T00:33:38.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:33:38 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.119:6834/1659925849,v1:172.21.15.119:6835/1659925849] up:boot 2024-03-28T00:33:38.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:33:38 smithi132 ceph-mon[35941]: Standby daemon mds.i assigned to filesystem cephfs as rank 3 2024-03-28T00:33:38.705 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:33:38 smithi132 ceph-mon[35941]: fsmap cephfs:4/5 {0=b=up:active,1=c=up:active,2=h=up:active,4=f=up:active} 5 up:standby-replay 3 up:standby, 1 failed 2024-03-28T00:33:38.705 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:33:38 smithi132 ceph-mon[35941]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:replay 4 up:active 2024-03-28T00:33:39.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:33:39 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.165:6832/1344198130,v1:172.21.15.165:6833/1344198130] up:resolve 2024-03-28T00:33:39.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:33:39 smithi165 ceph-mon[36302]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:resolve 4 up:active 2024-03-28T00:33:39.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:33:39 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.165:6832/1344198130,v1:172.21.15.165:6833/1344198130] up:resolve 2024-03-28T00:33:39.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:33:39 smithi119 ceph-mon[31518]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:resolve 4 up:active 2024-03-28T00:33:39.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:33:39 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.165:6832/1344198130,v1:172.21.15.165:6833/1344198130] up:resolve 2024-03-28T00:33:39.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:33:39 smithi132 ceph-mon[35941]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:resolve 4 up:active 2024-03-28T00:33:40.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:33:40 smithi165 ceph-mon[36302]: pgmap v236: 97 pgs: 97 active+clean; 246 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 131 KiB/s wr, 14 op/s 2024-03-28T00:33:40.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:33:40 smithi165 ceph-mon[36302]: Replacing daemon mds.c as rank 1 with standby daemon mds.l 2024-03-28T00:33:40.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:33:40 smithi165 ceph-mon[36302]: MDS daemon mds.c is removed because it is dead or otherwise unavailable. 2024-03-28T00:33:40.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:33:40 smithi165 ceph-mon[36302]: osdmap e109: 12 total, 12 up, 12 in 2024-03-28T00:33:40.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:33:40 smithi165 ceph-mon[36302]: fsmap cephfs:5/5 3 up:standby-replay 3 up:standby 1 up:replay 1 up:resolve 3 up:active 2024-03-28T00:33:40.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:33:40 smithi119 ceph-mon[31518]: pgmap v236: 97 pgs: 97 active+clean; 246 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 131 KiB/s wr, 14 op/s 2024-03-28T00:33:40.658 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:33:40 smithi119 ceph-mon[31518]: Replacing daemon mds.c as rank 1 with standby daemon mds.l 2024-03-28T00:33:40.658 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:33:40 smithi119 ceph-mon[31518]: MDS daemon mds.c is removed because it is dead or otherwise unavailable. 2024-03-28T00:33:40.658 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:33:40 smithi119 ceph-mon[31518]: osdmap e109: 12 total, 12 up, 12 in 2024-03-28T00:33:40.658 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:33:40 smithi119 ceph-mon[31518]: fsmap cephfs:5/5 3 up:standby-replay 3 up:standby 1 up:replay 1 up:resolve 3 up:active 2024-03-28T00:33:40.705 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:33:40 smithi132 ceph-mon[35941]: pgmap v236: 97 pgs: 97 active+clean; 246 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 131 KiB/s wr, 14 op/s 2024-03-28T00:33:40.706 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:33:40 smithi132 ceph-mon[35941]: Replacing daemon mds.c as rank 1 with standby daemon mds.l 2024-03-28T00:33:40.706 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:33:40 smithi132 ceph-mon[35941]: MDS daemon mds.c is removed because it is dead or otherwise unavailable. 2024-03-28T00:33:40.706 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:33:40 smithi132 ceph-mon[35941]: osdmap e109: 12 total, 12 up, 12 in 2024-03-28T00:33:40.706 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:33:40 smithi132 ceph-mon[35941]: fsmap cephfs:5/5 3 up:standby-replay 3 up:standby 1 up:replay 1 up:resolve 3 up:active 2024-03-28T00:33:42.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:33:41 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.165:6838/2762859081,v1:172.21.15.165:6839/2762859081] up:resolve 2024-03-28T00:33:42.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:33:41 smithi165 ceph-mon[36302]: fsmap cephfs:5/5 3 up:standby-replay 3 up:standby 2 up:resolve 3 up:active 2024-03-28T00:33:42.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:33:41 smithi165 ceph-mon[36302]: pgmap v238: 97 pgs: 97 active+clean; 246 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 154 KiB/s wr, 16 op/s 2024-03-28T00:33:42.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:33:41 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.165:6838/2762859081,v1:172.21.15.165:6839/2762859081] up:resolve 2024-03-28T00:33:42.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:33:41 smithi119 ceph-mon[31518]: fsmap cephfs:5/5 3 up:standby-replay 3 up:standby 2 up:resolve 3 up:active 2024-03-28T00:33:42.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:33:41 smithi119 ceph-mon[31518]: pgmap v238: 97 pgs: 97 active+clean; 246 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 154 KiB/s wr, 16 op/s 2024-03-28T00:33:42.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:33:41 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.165:6838/2762859081,v1:172.21.15.165:6839/2762859081] up:resolve 2024-03-28T00:33:42.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:33:41 smithi132 ceph-mon[35941]: fsmap cephfs:5/5 3 up:standby-replay 3 up:standby 2 up:resolve 3 up:active 2024-03-28T00:33:42.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:33:41 smithi132 ceph-mon[35941]: pgmap v238: 97 pgs: 97 active+clean; 246 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 154 KiB/s wr, 16 op/s 2024-03-28T00:33:43.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:33:42 smithi165 ceph-mon[36302]: reconnect by client.25355 172.21.15.119:0/3071152482 after 0.00299996 2024-03-28T00:33:43.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:33:42 smithi165 ceph-mon[36302]: reconnect by client.25355 172.21.15.119:0/3071152482 after 0.00199997 2024-03-28T00:33:43.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:33:42 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.165:6838/2762859081,v1:172.21.15.165:6839/2762859081] up:reconnect 2024-03-28T00:33:43.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:33:42 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.165:6832/1344198130,v1:172.21.15.165:6833/1344198130] up:reconnect 2024-03-28T00:33:43.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:33:42 smithi165 ceph-mon[36302]: fsmap cephfs:5/5 3 up:standby-replay 3 up:standby 2 up:reconnect 3 up:active 2024-03-28T00:33:43.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:33:42 smithi165 ceph-mon[36302]: reconnect by client.25264 v1:172.21.15.119:0/3863511511 after 0.0179997 2024-03-28T00:33:43.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:33:42 smithi165 ceph-mon[36302]: reconnect by client.25264 v1:172.21.15.119:0/3863511511 after 0.0279996 2024-03-28T00:33:43.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:33:42 smithi119 ceph-mon[31518]: reconnect by client.25355 172.21.15.119:0/3071152482 after 0.00299996 2024-03-28T00:33:43.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:33:42 smithi119 ceph-mon[31518]: reconnect by client.25355 172.21.15.119:0/3071152482 after 0.00199997 2024-03-28T00:33:43.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:33:42 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.165:6838/2762859081,v1:172.21.15.165:6839/2762859081] up:reconnect 2024-03-28T00:33:43.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:33:42 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.165:6832/1344198130,v1:172.21.15.165:6833/1344198130] up:reconnect 2024-03-28T00:33:43.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:33:42 smithi119 ceph-mon[31518]: fsmap cephfs:5/5 3 up:standby-replay 3 up:standby 2 up:reconnect 3 up:active 2024-03-28T00:33:43.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:33:42 smithi119 ceph-mon[31518]: reconnect by client.25264 v1:172.21.15.119:0/3863511511 after 0.0179997 2024-03-28T00:33:43.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:33:42 smithi119 ceph-mon[31518]: reconnect by client.25264 v1:172.21.15.119:0/3863511511 after 0.0279996 2024-03-28T00:33:43.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:33:42 smithi132 ceph-mon[35941]: reconnect by client.25355 172.21.15.119:0/3071152482 after 0.00299996 2024-03-28T00:33:43.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:33:42 smithi132 ceph-mon[35941]: reconnect by client.25355 172.21.15.119:0/3071152482 after 0.00199997 2024-03-28T00:33:43.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:33:42 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.165:6838/2762859081,v1:172.21.15.165:6839/2762859081] up:reconnect 2024-03-28T00:33:43.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:33:42 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.165:6832/1344198130,v1:172.21.15.165:6833/1344198130] up:reconnect 2024-03-28T00:33:43.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:33:42 smithi132 ceph-mon[35941]: fsmap cephfs:5/5 3 up:standby-replay 3 up:standby 2 up:reconnect 3 up:active 2024-03-28T00:33:43.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:33:42 smithi132 ceph-mon[35941]: reconnect by client.25264 v1:172.21.15.119:0/3863511511 after 0.0179997 2024-03-28T00:33:43.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:33:42 smithi132 ceph-mon[35941]: reconnect by client.25264 v1:172.21.15.119:0/3863511511 after 0.0279996 2024-03-28T00:33:44.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.165:6838/2762859081,v1:172.21.15.165:6839/2762859081] up:rejoin 2024-03-28T00:33:44.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.165:6832/1344198130,v1:172.21.15.165:6833/1344198130] up:rejoin 2024-03-28T00:33:44.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-mon[36302]: fsmap cephfs:5/5 3 up:standby-replay 3 up:standby 2 up:rejoin 3 up:active 2024-03-28T00:33:44.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-mon[36302]: daemon mds.l is now active in filesystem cephfs as rank 1 2024-03-28T00:33:44.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-mon[36302]: daemon mds.i is now active in filesystem cephfs as rank 3 2024-03-28T00:33:44.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-mon[36302]: pgmap v239: 97 pgs: 97 active+clean; 246 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-28T00:33:44.135 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7fe715328700 time 2024-03-28T00:33:43.884920+0000 2024-03-28T00:33:44.135 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T00:33:44.135 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:33:44.135 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fe71f9524a1] 2024-03-28T00:33:44.135 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7fe71f952667] 2024-03-28T00:33:44.135 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x5638ad634223] 2024-03-28T00:33:44.135 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x5638ad4ae5d6] 2024-03-28T00:33:44.135 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x5638ad4aeda1] 2024-03-28T00:33:44.135 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: 6: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x5638ad4e309c] 2024-03-28T00:33:44.135 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x5638ad501233] 2024-03-28T00:33:44.136 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5638ad503197] 2024-03-28T00:33:44.136 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x5638ad40211b] 2024-03-28T00:33:44.136 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5638ad4026bc] 2024-03-28T00:33:44.136 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x5638ad3ec1ff] 2024-03-28T00:33:44.136 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7fe71fbe4488] 2024-03-28T00:33:44.136 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: 13: (DispatchQueue::entry()+0x51f) [0x7fe71fbe160f] 2024-03-28T00:33:44.136 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fe71fcb4e81] 2024-03-28T00:33:44.136 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fe71e6cf1ca] 2024-03-28T00:33:44.136 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: 16: clone() 2024-03-28T00:33:44.136 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7f05a4298700 time 2024-03-28T00:33:43.874022+0000 2024-03-28T00:33:44.137 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T00:33:44.137 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:33:44.137 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f05ae8c24a1] 2024-03-28T00:33:44.137 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f05ae8c2667] 2024-03-28T00:33:44.137 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x5629c4b6c223] 2024-03-28T00:33:44.137 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x5629c49e65d6] 2024-03-28T00:33:44.137 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x5629c49e6da1] 2024-03-28T00:33:44.137 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: 6: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x5629c4a1b09c] 2024-03-28T00:33:44.137 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x5629c4a39233] 2024-03-28T00:33:44.137 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5629c4a3b197] 2024-03-28T00:33:44.137 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x5629c493a11b] 2024-03-28T00:33:44.137 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5629c493a6bc] 2024-03-28T00:33:44.138 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x5629c49241ff] 2024-03-28T00:33:44.138 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f05aeb54488] 2024-03-28T00:33:44.138 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: 13: (DispatchQueue::entry()+0x51f) [0x7f05aeb5160f] 2024-03-28T00:33:44.138 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f05aec24e81] 2024-03-28T00:33:44.138 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f05ad63f1ca] 2024-03-28T00:33:44.138 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: 16: clone() 2024-03-28T00:33:44.138 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: 2024-03-28T00:33:43.886+0000 7fe715328700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7fe715328700 time 2024-03-28T00:33:43.884920+0000 2024-03-28T00:33:44.138 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T00:33:44.138 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: 2024-03-28T00:33:44.139 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:33:44.139 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fe71f9524a1] 2024-03-28T00:33:44.139 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7fe71f952667] 2024-03-28T00:33:44.139 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x5638ad634223] 2024-03-28T00:33:44.139 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x5638ad4ae5d6] 2024-03-28T00:33:44.139 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x5638ad4aeda1] 2024-03-28T00:33:44.139 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: 6: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x5638ad4e309c] 2024-03-28T00:33:44.139 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x5638ad501233] 2024-03-28T00:33:44.139 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5638ad503197] 2024-03-28T00:33:44.139 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x5638ad40211b] 2024-03-28T00:33:44.139 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5638ad4026bc] 2024-03-28T00:33:44.139 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x5638ad3ec1ff] 2024-03-28T00:33:44.139 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7fe71fbe4488] 2024-03-28T00:33:44.139 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: 13: (DispatchQueue::entry()+0x51f) [0x7fe71fbe160f] 2024-03-28T00:33:44.140 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fe71fcb4e81] 2024-03-28T00:33:44.140 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fe71e6cf1ca] 2024-03-28T00:33:44.140 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: 16: clone() 2024-03-28T00:33:44.140 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: 2024-03-28T00:33:44.140 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: *** Caught signal (Aborted) ** 2024-03-28T00:33:44.140 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: in thread 7fe715328700 thread_name:ms_dispatch 2024-03-28T00:33:44.140 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:33:44.140 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fe71e6d9d20] 2024-03-28T00:33:44.140 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: 2024-03-28T00:33:43.875+0000 7f05a4298700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7f05a4298700 time 2024-03-28T00:33:43.874022+0000 2024-03-28T00:33:44.140 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T00:33:44.141 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: 2024-03-28T00:33:44.141 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:33:44.141 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f05ae8c24a1] 2024-03-28T00:33:44.141 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f05ae8c2667] 2024-03-28T00:33:44.141 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x5629c4b6c223] 2024-03-28T00:33:44.141 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x5629c49e65d6] 2024-03-28T00:33:44.141 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x5629c49e6da1] 2024-03-28T00:33:44.141 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: 6: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x5629c4a1b09c] 2024-03-28T00:33:44.141 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x5629c4a39233] 2024-03-28T00:33:44.141 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5629c4a3b197] 2024-03-28T00:33:44.141 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x5629c493a11b] 2024-03-28T00:33:44.141 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5629c493a6bc] 2024-03-28T00:33:44.141 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x5629c49241ff] 2024-03-28T00:33:44.141 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f05aeb54488] 2024-03-28T00:33:44.142 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: 13: (DispatchQueue::entry()+0x51f) [0x7f05aeb5160f] 2024-03-28T00:33:44.142 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f05aec24e81] 2024-03-28T00:33:44.142 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f05ad63f1ca] 2024-03-28T00:33:44.142 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: 16: clone() 2024-03-28T00:33:44.142 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: 2024-03-28T00:33:44.142 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: *** Caught signal (Aborted) ** 2024-03-28T00:33:44.142 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: in thread 7f05a4298700 thread_name:ms_dispatch 2024-03-28T00:33:44.142 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:33:44.142 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f05ad649d20] 2024-03-28T00:33:44.143 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: 2: gsignal() 2024-03-28T00:33:44.143 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: 3: abort() 2024-03-28T00:33:44.143 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fe71f9524fb] 2024-03-28T00:33:44.143 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7fe71f952667] 2024-03-28T00:33:44.143 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x5638ad634223] 2024-03-28T00:33:44.143 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x5638ad4ae5d6] 2024-03-28T00:33:44.143 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x5638ad4aeda1] 2024-03-28T00:33:44.143 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: 9: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x5638ad4e309c] 2024-03-28T00:33:44.143 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x5638ad501233] 2024-03-28T00:33:44.143 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5638ad503197] 2024-03-28T00:33:44.143 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x5638ad40211b] 2024-03-28T00:33:44.144 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5638ad4026bc] 2024-03-28T00:33:44.144 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x5638ad3ec1ff] 2024-03-28T00:33:44.144 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7fe71fbe4488] 2024-03-28T00:33:44.144 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: 16: (DispatchQueue::entry()+0x51f) [0x7fe71fbe160f] 2024-03-28T00:33:44.144 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fe71fcb4e81] 2024-03-28T00:33:44.144 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7fe71e6cf1ca] 2024-03-28T00:33:44.144 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: 19: clone() 2024-03-28T00:33:44.144 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: 2024-03-28T00:33:43.888+0000 7fe715328700 -1 *** Caught signal (Aborted) ** 2024-03-28T00:33:44.144 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: in thread 7fe715328700 thread_name:ms_dispatch 2024-03-28T00:33:44.144 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: 2024-03-28T00:33:44.144 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:33:44.144 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fe71e6d9d20] 2024-03-28T00:33:44.144 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: 2: gsignal() 2024-03-28T00:33:44.145 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: 3: abort() 2024-03-28T00:33:44.145 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fe71f9524fb] 2024-03-28T00:33:44.145 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7fe71f952667] 2024-03-28T00:33:44.145 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x5638ad634223] 2024-03-28T00:33:44.145 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x5638ad4ae5d6] 2024-03-28T00:33:44.145 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x5638ad4aeda1] 2024-03-28T00:33:44.145 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: 9: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x5638ad4e309c] 2024-03-28T00:33:44.145 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x5638ad501233] 2024-03-28T00:33:44.145 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5638ad503197] 2024-03-28T00:33:44.145 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x5638ad40211b] 2024-03-28T00:33:44.145 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5638ad4026bc] 2024-03-28T00:33:44.145 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x5638ad3ec1ff] 2024-03-28T00:33:44.145 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7fe71fbe4488] 2024-03-28T00:33:44.145 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: 16: (DispatchQueue::entry()+0x51f) [0x7fe71fbe160f] 2024-03-28T00:33:44.145 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fe71fcb4e81] 2024-03-28T00:33:44.146 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7fe71e6cf1ca] 2024-03-28T00:33:44.146 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: 19: clone() 2024-03-28T00:33:44.146 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T00:33:44.146 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: 2024-03-28T00:33:44.146 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: -1> 2024-03-28T00:33:43.886+0000 7fe715328700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7fe715328700 time 2024-03-28T00:33:43.884920+0000 2024-03-28T00:33:44.146 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T00:33:44.146 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: 2024-03-28T00:33:44.146 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:33:44.146 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fe71f9524a1] 2024-03-28T00:33:44.146 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7fe71f952667] 2024-03-28T00:33:44.147 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: 2: gsignal() 2024-03-28T00:33:44.147 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: 3: abort() 2024-03-28T00:33:44.147 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f05ae8c24fb] 2024-03-28T00:33:44.147 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f05ae8c2667] 2024-03-28T00:33:44.147 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x5629c4b6c223] 2024-03-28T00:33:44.147 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x5629c49e65d6] 2024-03-28T00:33:44.147 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x5629c49e6da1] 2024-03-28T00:33:44.147 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: 9: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x5629c4a1b09c] 2024-03-28T00:33:44.147 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x5629c4a39233] 2024-03-28T00:33:44.147 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5629c4a3b197] 2024-03-28T00:33:44.147 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x5629c493a11b] 2024-03-28T00:33:44.147 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5629c493a6bc] 2024-03-28T00:33:44.147 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x5629c49241ff] 2024-03-28T00:33:44.148 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f05aeb54488] 2024-03-28T00:33:44.148 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: 16: (DispatchQueue::entry()+0x51f) [0x7f05aeb5160f] 2024-03-28T00:33:44.148 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f05aec24e81] 2024-03-28T00:33:44.148 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f05ad63f1ca] 2024-03-28T00:33:44.148 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: 19: clone() 2024-03-28T00:33:44.148 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: 2024-03-28T00:33:43.877+0000 7f05a4298700 -1 *** Caught signal (Aborted) ** 2024-03-28T00:33:44.148 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: in thread 7f05a4298700 thread_name:ms_dispatch 2024-03-28T00:33:44.148 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: 2024-03-28T00:33:44.148 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:33:44.148 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f05ad649d20] 2024-03-28T00:33:44.148 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: 2: gsignal() 2024-03-28T00:33:44.148 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: 3: abort() 2024-03-28T00:33:44.148 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f05ae8c24fb] 2024-03-28T00:33:44.148 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f05ae8c2667] 2024-03-28T00:33:44.149 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x5638ad634223] 2024-03-28T00:33:44.149 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x5638ad4ae5d6] 2024-03-28T00:33:44.149 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x5638ad4aeda1] 2024-03-28T00:33:44.149 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: 6: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x5638ad4e309c] 2024-03-28T00:33:44.149 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x5638ad501233] 2024-03-28T00:33:44.149 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5638ad503197] 2024-03-28T00:33:44.149 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x5638ad40211b] 2024-03-28T00:33:44.149 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5638ad4026bc] 2024-03-28T00:33:44.149 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x5638ad3ec1ff] 2024-03-28T00:33:44.149 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7fe71fbe4488] 2024-03-28T00:33:44.150 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: 13: (DispatchQueue::entry()+0x51f) [0x7fe71fbe160f] 2024-03-28T00:33:44.150 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fe71fcb4e81] 2024-03-28T00:33:44.150 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fe71e6cf1ca] 2024-03-28T00:33:44.150 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: 16: clone() 2024-03-28T00:33:44.150 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: 2024-03-28T00:33:44.150 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: 0> 2024-03-28T00:33:43.888+0000 7fe715328700 -1 *** Caught signal (Aborted) ** 2024-03-28T00:33:44.150 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: in thread 7fe715328700 thread_name:ms_dispatch 2024-03-28T00:33:44.150 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: 2024-03-28T00:33:44.150 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:33:44.150 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fe71e6d9d20] 2024-03-28T00:33:44.150 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: 2: gsignal() 2024-03-28T00:33:44.150 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: 3: abort() 2024-03-28T00:33:44.150 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fe71f9524fb] 2024-03-28T00:33:44.150 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7fe71f952667] 2024-03-28T00:33:44.151 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x5638ad634223] 2024-03-28T00:33:44.151 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x5638ad4ae5d6] 2024-03-28T00:33:44.151 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x5629c4b6c223] 2024-03-28T00:33:44.151 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x5629c49e65d6] 2024-03-28T00:33:44.151 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x5629c49e6da1] 2024-03-28T00:33:44.151 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: 9: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x5629c4a1b09c] 2024-03-28T00:33:44.151 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x5629c4a39233] 2024-03-28T00:33:44.151 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5629c4a3b197] 2024-03-28T00:33:44.151 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x5629c493a11b] 2024-03-28T00:33:44.151 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5629c493a6bc] 2024-03-28T00:33:44.152 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x5629c49241ff] 2024-03-28T00:33:44.152 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f05aeb54488] 2024-03-28T00:33:44.152 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: 16: (DispatchQueue::entry()+0x51f) [0x7f05aeb5160f] 2024-03-28T00:33:44.152 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f05aec24e81] 2024-03-28T00:33:44.152 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f05ad63f1ca] 2024-03-28T00:33:44.152 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: 19: clone() 2024-03-28T00:33:44.152 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T00:33:44.152 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: 2024-03-28T00:33:44.152 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: -1> 2024-03-28T00:33:43.875+0000 7f05a4298700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7f05a4298700 time 2024-03-28T00:33:43.874022+0000 2024-03-28T00:33:44.152 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T00:33:44.152 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: 2024-03-28T00:33:44.152 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:33:44.152 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f05ae8c24a1] 2024-03-28T00:33:44.152 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f05ae8c2667] 2024-03-28T00:33:44.153 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x5638ad4aeda1] 2024-03-28T00:33:44.153 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: 9: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x5638ad4e309c] 2024-03-28T00:33:44.153 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x5638ad501233] 2024-03-28T00:33:44.153 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5638ad503197] 2024-03-28T00:33:44.153 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x5638ad40211b] 2024-03-28T00:33:44.153 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5638ad4026bc] 2024-03-28T00:33:44.153 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x5638ad3ec1ff] 2024-03-28T00:33:44.153 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7fe71fbe4488] 2024-03-28T00:33:44.153 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: 16: (DispatchQueue::entry()+0x51f) [0x7fe71fbe160f] 2024-03-28T00:33:44.154 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fe71fcb4e81] 2024-03-28T00:33:44.154 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7fe71e6cf1ca] 2024-03-28T00:33:44.154 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: 19: clone() 2024-03-28T00:33:44.154 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T00:33:44.154 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: 2024-03-28T00:33:44.154 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: -1> 2024-03-28T00:33:43.886+0000 7fe715328700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7fe715328700 time 2024-03-28T00:33:43.884920+0000 2024-03-28T00:33:44.154 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T00:33:44.154 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: 2024-03-28T00:33:44.154 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:33:44.154 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fe71f9524a1] 2024-03-28T00:33:44.154 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7fe71f952667] 2024-03-28T00:33:44.154 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x5638ad634223] 2024-03-28T00:33:44.154 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x5638ad4ae5d6] 2024-03-28T00:33:44.154 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x5638ad4aeda1] 2024-03-28T00:33:44.155 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x5629c4b6c223] 2024-03-28T00:33:44.155 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x5629c49e65d6] 2024-03-28T00:33:44.155 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x5629c49e6da1] 2024-03-28T00:33:44.155 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: 6: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x5629c4a1b09c] 2024-03-28T00:33:44.155 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x5629c4a39233] 2024-03-28T00:33:44.155 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5629c4a3b197] 2024-03-28T00:33:44.155 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x5629c493a11b] 2024-03-28T00:33:44.155 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5629c493a6bc] 2024-03-28T00:33:44.155 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x5629c49241ff] 2024-03-28T00:33:44.155 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f05aeb54488] 2024-03-28T00:33:44.155 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: 13: (DispatchQueue::entry()+0x51f) [0x7f05aeb5160f] 2024-03-28T00:33:44.155 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f05aec24e81] 2024-03-28T00:33:44.156 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f05ad63f1ca] 2024-03-28T00:33:44.156 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: 16: clone() 2024-03-28T00:33:44.156 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: 2024-03-28T00:33:44.156 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: 0> 2024-03-28T00:33:43.877+0000 7f05a4298700 -1 *** Caught signal (Aborted) ** 2024-03-28T00:33:44.156 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: in thread 7f05a4298700 thread_name:ms_dispatch 2024-03-28T00:33:44.156 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: 2024-03-28T00:33:44.156 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:33:44.156 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f05ad649d20] 2024-03-28T00:33:44.156 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: 2: gsignal() 2024-03-28T00:33:44.156 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: 3: abort() 2024-03-28T00:33:44.156 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f05ae8c24fb] 2024-03-28T00:33:44.156 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f05ae8c2667] 2024-03-28T00:33:44.156 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x5629c4b6c223] 2024-03-28T00:33:44.156 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x5629c49e65d6] 2024-03-28T00:33:44.157 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: 6: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x5638ad4e309c] 2024-03-28T00:33:44.157 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x5638ad501233] 2024-03-28T00:33:44.157 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5638ad503197] 2024-03-28T00:33:44.157 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x5638ad40211b] 2024-03-28T00:33:44.157 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5638ad4026bc] 2024-03-28T00:33:44.157 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x5638ad3ec1ff] 2024-03-28T00:33:44.157 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7fe71fbe4488] 2024-03-28T00:33:44.157 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: 13: (DispatchQueue::entry()+0x51f) [0x7fe71fbe160f] 2024-03-28T00:33:44.158 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fe71fcb4e81] 2024-03-28T00:33:44.158 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fe71e6cf1ca] 2024-03-28T00:33:44.158 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: 16: clone() 2024-03-28T00:33:44.158 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: 2024-03-28T00:33:44.158 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: 0> 2024-03-28T00:33:43.888+0000 7fe715328700 -1 *** Caught signal (Aborted) ** 2024-03-28T00:33:44.158 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: in thread 7fe715328700 thread_name:ms_dispatch 2024-03-28T00:33:44.158 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: 2024-03-28T00:33:44.158 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:33:44.158 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fe71e6d9d20] 2024-03-28T00:33:44.158 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: 2: gsignal() 2024-03-28T00:33:44.158 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: 3: abort() 2024-03-28T00:33:44.158 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fe71f9524fb] 2024-03-28T00:33:44.158 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7fe71f952667] 2024-03-28T00:33:44.158 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x5638ad634223] 2024-03-28T00:33:44.158 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x5638ad4ae5d6] 2024-03-28T00:33:44.159 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x5638ad4aeda1] 2024-03-28T00:33:44.159 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: 9: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x5638ad4e309c] 2024-03-28T00:33:44.159 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x5638ad501233] 2024-03-28T00:33:44.159 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5638ad503197] 2024-03-28T00:33:44.159 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x5638ad40211b] 2024-03-28T00:33:44.159 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5638ad4026bc] 2024-03-28T00:33:44.159 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x5638ad3ec1ff] 2024-03-28T00:33:44.159 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7fe71fbe4488] 2024-03-28T00:33:44.159 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: 16: (DispatchQueue::entry()+0x51f) [0x7fe71fbe160f] 2024-03-28T00:33:44.159 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fe71fcb4e81] 2024-03-28T00:33:44.159 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7fe71e6cf1ca] 2024-03-28T00:33:44.159 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: 19: clone() 2024-03-28T00:33:44.159 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T00:33:44.159 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i[73774]: 2024-03-28T00:33:44.160 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x5629c49e6da1] 2024-03-28T00:33:44.160 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: 9: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x5629c4a1b09c] 2024-03-28T00:33:44.160 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x5629c4a39233] 2024-03-28T00:33:44.160 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5629c4a3b197] 2024-03-28T00:33:44.161 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x5629c493a11b] 2024-03-28T00:33:44.161 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5629c493a6bc] 2024-03-28T00:33:44.161 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x5629c49241ff] 2024-03-28T00:33:44.161 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f05aeb54488] 2024-03-28T00:33:44.161 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: 16: (DispatchQueue::entry()+0x51f) [0x7f05aeb5160f] 2024-03-28T00:33:44.161 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f05aec24e81] 2024-03-28T00:33:44.161 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f05ad63f1ca] 2024-03-28T00:33:44.161 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: 19: clone() 2024-03-28T00:33:44.161 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T00:33:44.161 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: 2024-03-28T00:33:44.161 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: -1> 2024-03-28T00:33:43.875+0000 7f05a4298700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7f05a4298700 time 2024-03-28T00:33:43.874022+0000 2024-03-28T00:33:44.161 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T00:33:44.162 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: 2024-03-28T00:33:44.162 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:33:44.162 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f05ae8c24a1] 2024-03-28T00:33:44.162 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f05ae8c2667] 2024-03-28T00:33:44.162 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x5629c4b6c223] 2024-03-28T00:33:44.162 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x5629c49e65d6] 2024-03-28T00:33:44.162 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x5629c49e6da1] 2024-03-28T00:33:44.162 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: 6: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x5629c4a1b09c] 2024-03-28T00:33:44.162 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x5629c4a39233] 2024-03-28T00:33:44.162 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5629c4a3b197] 2024-03-28T00:33:44.162 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x5629c493a11b] 2024-03-28T00:33:44.163 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5629c493a6bc] 2024-03-28T00:33:44.163 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x5629c49241ff] 2024-03-28T00:33:44.163 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f05aeb54488] 2024-03-28T00:33:44.163 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: 13: (DispatchQueue::entry()+0x51f) [0x7f05aeb5160f] 2024-03-28T00:33:44.163 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f05aec24e81] 2024-03-28T00:33:44.163 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f05ad63f1ca] 2024-03-28T00:33:44.163 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: 16: clone() 2024-03-28T00:33:44.163 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: 2024-03-28T00:33:44.163 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: 0> 2024-03-28T00:33:43.877+0000 7f05a4298700 -1 *** Caught signal (Aborted) ** 2024-03-28T00:33:44.163 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: in thread 7f05a4298700 thread_name:ms_dispatch 2024-03-28T00:33:44.163 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: 2024-03-28T00:33:44.163 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:33:44.163 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f05ad649d20] 2024-03-28T00:33:44.163 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: 2: gsignal() 2024-03-28T00:33:44.163 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: 3: abort() 2024-03-28T00:33:44.164 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f05ae8c24fb] 2024-03-28T00:33:44.164 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f05ae8c2667] 2024-03-28T00:33:44.164 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x5629c4b6c223] 2024-03-28T00:33:44.164 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x5629c49e65d6] 2024-03-28T00:33:44.164 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x5629c49e6da1] 2024-03-28T00:33:44.164 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: 9: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x5629c4a1b09c] 2024-03-28T00:33:44.164 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x5629c4a39233] 2024-03-28T00:33:44.164 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5629c4a3b197] 2024-03-28T00:33:44.164 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x5629c493a11b] 2024-03-28T00:33:44.164 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5629c493a6bc] 2024-03-28T00:33:44.164 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x5629c49241ff] 2024-03-28T00:33:44.164 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f05aeb54488] 2024-03-28T00:33:44.164 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: 16: (DispatchQueue::entry()+0x51f) [0x7f05aeb5160f] 2024-03-28T00:33:44.164 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f05aec24e81] 2024-03-28T00:33:44.165 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f05ad63f1ca] 2024-03-28T00:33:44.165 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: 19: clone() 2024-03-28T00:33:44.165 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T00:33:44.165 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l[73854]: 2024-03-28T00:33:44.165 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:43 smithi165 podman[73956]: 2024-03-28 00:33:43.95763266 +0000 UTC m=+0.022511935 container died 9fef9895027f3ffd477a2a49316400b684e21d9f00da5f58ae88e73497aef267 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-03-28T00:33:44.165 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:33:43 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.165:6838/2762859081,v1:172.21.15.165:6839/2762859081] up:rejoin 2024-03-28T00:33:44.165 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:33:43 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.165:6832/1344198130,v1:172.21.15.165:6833/1344198130] up:rejoin 2024-03-28T00:33:44.165 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:33:43 smithi119 ceph-mon[31518]: fsmap cephfs:5/5 3 up:standby-replay 3 up:standby 2 up:rejoin 3 up:active 2024-03-28T00:33:44.165 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:33:43 smithi119 ceph-mon[31518]: daemon mds.l is now active in filesystem cephfs as rank 1 2024-03-28T00:33:44.166 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:33:43 smithi119 ceph-mon[31518]: daemon mds.i is now active in filesystem cephfs as rank 3 2024-03-28T00:33:44.166 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:33:43 smithi119 ceph-mon[31518]: pgmap v239: 97 pgs: 97 active+clean; 246 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-28T00:33:44.204 INFO:journalctl@ceph.mds.b.smithi132.stdout:Mar 28 00:33:44 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-b[66443]: 2024-03-28T00:33:44.045+0000 7f857cecc700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-03-28T00:33:44.204 INFO:journalctl@ceph.mds.b.smithi132.stdout:Mar 28 00:33:44 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-b[66443]: 2024-03-28T00:33:44.045+0000 7f857cecc700 -1 mds.pinger is_rank_lagging: rank=3 was never sent ping request. 2024-03-28T00:33:44.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:33:43 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.165:6838/2762859081,v1:172.21.15.165:6839/2762859081] up:rejoin 2024-03-28T00:33:44.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:33:43 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.165:6832/1344198130,v1:172.21.15.165:6833/1344198130] up:rejoin 2024-03-28T00:33:44.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:33:43 smithi132 ceph-mon[35941]: fsmap cephfs:5/5 3 up:standby-replay 3 up:standby 2 up:rejoin 3 up:active 2024-03-28T00:33:44.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:33:43 smithi132 ceph-mon[35941]: daemon mds.l is now active in filesystem cephfs as rank 1 2024-03-28T00:33:44.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:33:43 smithi132 ceph-mon[35941]: daemon mds.i is now active in filesystem cephfs as rank 3 2024-03-28T00:33:44.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:33:43 smithi132 ceph-mon[35941]: pgmap v239: 97 pgs: 97 active+clean; 246 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-28T00:33:44.995 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:44 smithi165 podman[73963]: 2024-03-28 00:33:44.698047066 +0000 UTC m=+0.750568587 container died e88bd49e10a33560a689618bb1a43ffed1447887f37b532a1980168d19824d06 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-28T00:33:44.995 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:33:44 smithi165 ceph-mon[36302]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-28T00:33:44.996 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:33:44 smithi165 ceph-mon[36302]: Cluster is now healthy 2024-03-28T00:33:44.996 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:33:44 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.165:6838/2762859081,v1:172.21.15.165:6839/2762859081] up:active 2024-03-28T00:33:44.996 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:33:44 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.165:6832/1344198130,v1:172.21.15.165:6833/1344198130] up:active 2024-03-28T00:33:44.996 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:33:44 smithi165 ceph-mon[36302]: fsmap cephfs:5 3 up:standby-replay 3 up:standby 5 up:active 2024-03-28T00:33:44.996 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:33:44 smithi165 ceph-mon[36302]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2024-03-28T00:33:44.996 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:44 smithi165 podman[73956]: 2024-03-28 00:33:44.666036415 +0000 UTC m=+0.730915682 container remove 9fef9895027f3ffd477a2a49316400b684e21d9f00da5f58ae88e73497aef267 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-l, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319) 2024-03-28T00:33:44.996 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:44 smithi165 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.l.service: Main process exited, code=exited, status=134/n/a 2024-03-28T00:33:45.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:33:44 smithi119 ceph-mon[31518]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-28T00:33:45.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:33:44 smithi119 ceph-mon[31518]: Cluster is now healthy 2024-03-28T00:33:45.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:33:44 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.165:6838/2762859081,v1:172.21.15.165:6839/2762859081] up:active 2024-03-28T00:33:45.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:33:44 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.165:6832/1344198130,v1:172.21.15.165:6833/1344198130] up:active 2024-03-28T00:33:45.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:33:44 smithi119 ceph-mon[31518]: fsmap cephfs:5 3 up:standby-replay 3 up:standby 5 up:active 2024-03-28T00:33:45.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:33:44 smithi119 ceph-mon[31518]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2024-03-28T00:33:45.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:33:44 smithi132 ceph-mon[35941]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-28T00:33:45.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:33:44 smithi132 ceph-mon[35941]: Cluster is now healthy 2024-03-28T00:33:45.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:33:44 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.165:6838/2762859081,v1:172.21.15.165:6839/2762859081] up:active 2024-03-28T00:33:45.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:33:44 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.165:6832/1344198130,v1:172.21.15.165:6833/1344198130] up:active 2024-03-28T00:33:45.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:33:44 smithi132 ceph-mon[35941]: fsmap cephfs:5 3 up:standby-replay 3 up:standby 5 up:active 2024-03-28T00:33:45.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:33:44 smithi132 ceph-mon[35941]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2024-03-28T00:33:45.566 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:45 smithi165 podman[73963]: 2024-03-28 00:33:45.306757047 +0000 UTC m=+1.359278570 container remove e88bd49e10a33560a689618bb1a43ffed1447887f37b532a1980168d19824d06 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-i, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c) 2024-03-28T00:33:45.566 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:45 smithi165 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.i.service: Main process exited, code=exited, status=134/n/a 2024-03-28T00:33:45.834 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:45 smithi165 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.l.service: Failed with result 'exit-code'. 2024-03-28T00:33:45.834 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:45 smithi165 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.i.service: Failed with result 'exit-code'. 2024-03-28T00:33:46.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:33:45 smithi165 ceph-mon[36302]: pgmap v240: 97 pgs: 97 active+clean; 249 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 660 KiB/s rd, 485 KiB/s wr, 80 op/s 2024-03-28T00:33:46.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:33:45 smithi119 ceph-mon[31518]: pgmap v240: 97 pgs: 97 active+clean; 249 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 660 KiB/s rd, 485 KiB/s wr, 80 op/s 2024-03-28T00:33:46.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:33:45 smithi132 ceph-mon[35941]: pgmap v240: 97 pgs: 97 active+clean; 249 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 660 KiB/s rd, 485 KiB/s wr, 80 op/s 2024-03-28T00:33:48.342 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:33:48 smithi119 ceph-mon[31518]: pgmap v241: 97 pgs: 97 active+clean; 249 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 532 KiB/s rd, 391 KiB/s wr, 64 op/s 2024-03-28T00:33:48.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:33:48 smithi165 ceph-mon[36302]: pgmap v241: 97 pgs: 97 active+clean; 249 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 532 KiB/s rd, 391 KiB/s wr, 64 op/s 2024-03-28T00:33:48.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:33:48 smithi132 ceph-mon[35941]: pgmap v241: 97 pgs: 97 active+clean; 249 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 532 KiB/s rd, 391 KiB/s wr, 64 op/s 2024-03-28T00:33:50.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:33:50 smithi165 ceph-mon[36302]: pgmap v242: 97 pgs: 97 active+clean; 251 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 801 KiB/s rd, 599 KiB/s wr, 104 op/s 2024-03-28T00:33:50.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:33:50 smithi119 ceph-mon[31518]: pgmap v242: 97 pgs: 97 active+clean; 251 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 801 KiB/s rd, 599 KiB/s wr, 104 op/s 2024-03-28T00:33:50.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:33:50 smithi132 ceph-mon[35941]: pgmap v242: 97 pgs: 97 active+clean; 251 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 801 KiB/s rd, 599 KiB/s wr, 104 op/s 2024-03-28T00:33:52.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:33:52 smithi165 ceph-mon[36302]: pgmap v243: 97 pgs: 97 active+clean; 251 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 704 KiB/s rd, 526 KiB/s wr, 91 op/s 2024-03-28T00:33:52.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:33:52 smithi119 ceph-mon[31518]: pgmap v243: 97 pgs: 97 active+clean; 251 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 704 KiB/s rd, 526 KiB/s wr, 91 op/s 2024-03-28T00:33:52.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:33:52 smithi132 ceph-mon[35941]: pgmap v243: 97 pgs: 97 active+clean; 251 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 704 KiB/s rd, 526 KiB/s wr, 91 op/s 2024-03-28T00:33:54.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:33:54 smithi165 ceph-mon[36302]: pgmap v244: 97 pgs: 97 active+clean; 251 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 667 KiB/s rd, 499 KiB/s wr, 86 op/s 2024-03-28T00:33:54.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:33:54 smithi119 ceph-mon[31518]: pgmap v244: 97 pgs: 97 active+clean; 251 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 667 KiB/s rd, 499 KiB/s wr, 86 op/s 2024-03-28T00:33:54.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:33:54 smithi132 ceph-mon[35941]: pgmap v244: 97 pgs: 97 active+clean; 251 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 667 KiB/s rd, 499 KiB/s wr, 86 op/s 2024-03-28T00:33:56.134 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:55 smithi165 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.i.service: Scheduled restart job, restart counter is at 5. 2024-03-28T00:33:56.134 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:55 smithi165 systemd[1]: Stopped Ceph mds.i for 10f73b92-ec98-11ee-95d0-87774f69a715. 2024-03-28T00:33:56.134 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:55 smithi165 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.i.service: Start request repeated too quickly. 2024-03-28T00:33:56.134 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:55 smithi165 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.i.service: Failed with result 'exit-code'. 2024-03-28T00:33:56.134 INFO:journalctl@ceph.mds.i.smithi165.stdout:Mar 28 00:33:55 smithi165 systemd[1]: Failed to start Ceph mds.i for 10f73b92-ec98-11ee-95d0-87774f69a715. 2024-03-28T00:33:56.134 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:55 smithi165 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.l.service: Scheduled restart job, restart counter is at 6. 2024-03-28T00:33:56.135 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:55 smithi165 systemd[1]: Stopped Ceph mds.l for 10f73b92-ec98-11ee-95d0-87774f69a715. 2024-03-28T00:33:56.135 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:55 smithi165 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.l.service: Start request repeated too quickly. 2024-03-28T00:33:56.135 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:55 smithi165 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.l.service: Failed with result 'exit-code'. 2024-03-28T00:33:56.135 INFO:journalctl@ceph.mds.l.smithi165.stdout:Mar 28 00:33:55 smithi165 systemd[1]: Failed to start Ceph mds.l for 10f73b92-ec98-11ee-95d0-87774f69a715. 2024-03-28T00:33:56.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:33:56 smithi165 ceph-mon[36302]: pgmap v245: 97 pgs: 97 active+clean; 251 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 670 KiB/s rd, 499 KiB/s wr, 91 op/s 2024-03-28T00:33:56.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:33:56 smithi119 ceph-mon[31518]: pgmap v245: 97 pgs: 97 active+clean; 251 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 670 KiB/s rd, 499 KiB/s wr, 91 op/s 2024-03-28T00:33:56.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:33:56 smithi132 ceph-mon[35941]: pgmap v245: 97 pgs: 97 active+clean; 251 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 670 KiB/s rd, 499 KiB/s wr, 91 op/s 2024-03-28T00:33:58.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:33:58 smithi165 ceph-mon[36302]: pgmap v246: 97 pgs: 97 active+clean; 251 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 176 KiB/s wr, 43 op/s 2024-03-28T00:33:58.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:33:58 smithi119 ceph-mon[31518]: pgmap v246: 97 pgs: 97 active+clean; 251 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 176 KiB/s wr, 43 op/s 2024-03-28T00:33:58.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:33:58 smithi132 ceph-mon[35941]: pgmap v246: 97 pgs: 97 active+clean; 251 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 176 KiB/s wr, 43 op/s 2024-03-28T00:34:00.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:34:00 smithi165 ceph-mon[36302]: pgmap v247: 97 pgs: 97 active+clean; 251 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 176 KiB/s wr, 47 op/s 2024-03-28T00:34:00.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:34:00 smithi165 ceph-mon[36302]: Replacing daemon mds.i as rank 3 with standby daemon mds.g 2024-03-28T00:34:00.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:34:00 smithi165 ceph-mon[36302]: MDS daemon mds.i is removed because it is dead or otherwise unavailable. 2024-03-28T00:34:00.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:34:00 smithi165 ceph-mon[36302]: Replacing daemon mds.l as rank 1 with standby daemon mds.j 2024-03-28T00:34:00.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:34:00 smithi165 ceph-mon[36302]: MDS daemon mds.l is removed because it is dead or otherwise unavailable. 2024-03-28T00:34:00.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:34:00 smithi165 ceph-mon[36302]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-28T00:34:00.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:34:00 smithi165 ceph-mon[36302]: osdmap e110: 12 total, 12 up, 12 in 2024-03-28T00:34:00.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:34:00 smithi165 ceph-mon[36302]: fsmap cephfs:5/5 3 up:standby-replay 1 up:standby 2 up:replay 3 up:active 2024-03-28T00:34:00.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:34:00 smithi119 ceph-mon[31518]: pgmap v247: 97 pgs: 97 active+clean; 251 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 176 KiB/s wr, 47 op/s 2024-03-28T00:34:00.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:34:00 smithi119 ceph-mon[31518]: Replacing daemon mds.i as rank 3 with standby daemon mds.g 2024-03-28T00:34:00.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:34:00 smithi119 ceph-mon[31518]: MDS daemon mds.i is removed because it is dead or otherwise unavailable. 2024-03-28T00:34:00.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:34:00 smithi119 ceph-mon[31518]: Replacing daemon mds.l as rank 1 with standby daemon mds.j 2024-03-28T00:34:00.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:34:00 smithi119 ceph-mon[31518]: MDS daemon mds.l is removed because it is dead or otherwise unavailable. 2024-03-28T00:34:00.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:34:00 smithi119 ceph-mon[31518]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-28T00:34:00.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:34:00 smithi119 ceph-mon[31518]: osdmap e110: 12 total, 12 up, 12 in 2024-03-28T00:34:00.658 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:34:00 smithi119 ceph-mon[31518]: fsmap cephfs:5/5 3 up:standby-replay 1 up:standby 2 up:replay 3 up:active 2024-03-28T00:34:00.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:34:00 smithi132 ceph-mon[35941]: pgmap v247: 97 pgs: 97 active+clean; 251 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 176 KiB/s wr, 47 op/s 2024-03-28T00:34:00.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:34:00 smithi132 ceph-mon[35941]: Replacing daemon mds.i as rank 3 with standby daemon mds.g 2024-03-28T00:34:00.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:34:00 smithi132 ceph-mon[35941]: MDS daemon mds.i is removed because it is dead or otherwise unavailable. 2024-03-28T00:34:00.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:34:00 smithi132 ceph-mon[35941]: Replacing daemon mds.l as rank 1 with standby daemon mds.j 2024-03-28T00:34:00.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:34:00 smithi132 ceph-mon[35941]: MDS daemon mds.l is removed because it is dead or otherwise unavailable. 2024-03-28T00:34:00.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:34:00 smithi132 ceph-mon[35941]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-28T00:34:00.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:34:00 smithi132 ceph-mon[35941]: osdmap e110: 12 total, 12 up, 12 in 2024-03-28T00:34:00.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:34:00 smithi132 ceph-mon[35941]: fsmap cephfs:5/5 3 up:standby-replay 1 up:standby 2 up:replay 3 up:active 2024-03-28T00:34:02.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:34:01 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.119:6834/1659925849,v1:172.21.15.119:6835/1659925849] up:resolve 2024-03-28T00:34:02.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:34:01 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.119:6838/1595343240,v1:172.21.15.119:6839/1595343240] up:resolve 2024-03-28T00:34:02.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:34:01 smithi165 ceph-mon[36302]: fsmap cephfs:5/5 3 up:standby-replay 1 up:standby 2 up:resolve 3 up:active 2024-03-28T00:34:02.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:34:01 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:34:02.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:34:01 smithi165 ceph-mon[36302]: pgmap v249: 97 pgs: 97 active+clean; 251 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-03-28T00:34:02.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:34:01 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.119:6834/1659925849,v1:172.21.15.119:6835/1659925849] up:resolve 2024-03-28T00:34:02.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:34:01 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.119:6838/1595343240,v1:172.21.15.119:6839/1595343240] up:resolve 2024-03-28T00:34:02.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:34:01 smithi119 ceph-mon[31518]: fsmap cephfs:5/5 3 up:standby-replay 1 up:standby 2 up:resolve 3 up:active 2024-03-28T00:34:02.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:34:01 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:34:02.158 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:34:01 smithi119 ceph-mon[31518]: pgmap v249: 97 pgs: 97 active+clean; 251 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-03-28T00:34:02.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:34:01 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.119:6834/1659925849,v1:172.21.15.119:6835/1659925849] up:resolve 2024-03-28T00:34:02.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:34:01 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.119:6838/1595343240,v1:172.21.15.119:6839/1595343240] up:resolve 2024-03-28T00:34:02.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:34:01 smithi132 ceph-mon[35941]: fsmap cephfs:5/5 3 up:standby-replay 1 up:standby 2 up:resolve 3 up:active 2024-03-28T00:34:02.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:34:01 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:34:02.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:34:01 smithi132 ceph-mon[35941]: pgmap v249: 97 pgs: 97 active+clean; 251 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-03-28T00:34:03.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:34:02 smithi165 ceph-mon[36302]: reconnect by client.25355 172.21.15.119:0/3071152482 after 0.00199995 2024-03-28T00:34:03.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:34:02 smithi165 ceph-mon[36302]: reconnect by client.25355 172.21.15.119:0/3071152482 after 0.00199995 2024-03-28T00:34:03.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:34:02 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.119:6834/1659925849,v1:172.21.15.119:6835/1659925849] up:reconnect 2024-03-28T00:34:03.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:34:02 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.119:6838/1595343240,v1:172.21.15.119:6839/1595343240] up:reconnect 2024-03-28T00:34:03.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:34:02 smithi165 ceph-mon[36302]: fsmap cephfs:5/5 3 up:standby-replay 1 up:standby 2 up:reconnect 3 up:active 2024-03-28T00:34:03.135 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:34:02 smithi165 ceph-mon[36302]: reconnect by client.25264 v1:172.21.15.119:0/3863511511 after 0.0169996 2024-03-28T00:34:03.135 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:34:02 smithi165 ceph-mon[36302]: reconnect by client.25264 v1:172.21.15.119:0/3863511511 after 0.0179996 2024-03-28T00:34:03.135 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:34:02 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:34:03.135 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:34:02 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:34:03.135 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:34:02 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:34:03.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:34:02 smithi119 ceph-mon[31518]: reconnect by client.25355 172.21.15.119:0/3071152482 after 0.00199995 2024-03-28T00:34:03.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:34:02 smithi119 ceph-mon[31518]: reconnect by client.25355 172.21.15.119:0/3071152482 after 0.00199995 2024-03-28T00:34:03.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:34:02 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.119:6834/1659925849,v1:172.21.15.119:6835/1659925849] up:reconnect 2024-03-28T00:34:03.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:34:02 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.119:6838/1595343240,v1:172.21.15.119:6839/1595343240] up:reconnect 2024-03-28T00:34:03.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:34:02 smithi119 ceph-mon[31518]: fsmap cephfs:5/5 3 up:standby-replay 1 up:standby 2 up:reconnect 3 up:active 2024-03-28T00:34:03.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:34:02 smithi119 ceph-mon[31518]: reconnect by client.25264 v1:172.21.15.119:0/3863511511 after 0.0169996 2024-03-28T00:34:03.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:34:02 smithi119 ceph-mon[31518]: reconnect by client.25264 v1:172.21.15.119:0/3863511511 after 0.0179996 2024-03-28T00:34:03.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:34:02 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:34:03.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:34:02 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:34:03.158 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:34:02 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:34:03.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:34:02 smithi132 ceph-mon[35941]: reconnect by client.25355 172.21.15.119:0/3071152482 after 0.00199995 2024-03-28T00:34:03.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:34:02 smithi132 ceph-mon[35941]: reconnect by client.25355 172.21.15.119:0/3071152482 after 0.00199995 2024-03-28T00:34:03.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:34:02 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.119:6834/1659925849,v1:172.21.15.119:6835/1659925849] up:reconnect 2024-03-28T00:34:03.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:34:02 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.119:6838/1595343240,v1:172.21.15.119:6839/1595343240] up:reconnect 2024-03-28T00:34:03.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:34:02 smithi132 ceph-mon[35941]: fsmap cephfs:5/5 3 up:standby-replay 1 up:standby 2 up:reconnect 3 up:active 2024-03-28T00:34:03.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:34:02 smithi132 ceph-mon[35941]: reconnect by client.25264 v1:172.21.15.119:0/3863511511 after 0.0169996 2024-03-28T00:34:03.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:34:02 smithi132 ceph-mon[35941]: reconnect by client.25264 v1:172.21.15.119:0/3863511511 after 0.0179996 2024-03-28T00:34:03.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:34:02 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:34:03.205 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:34:02 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:34:03.205 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:34:02 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:34:04.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:34:03 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.119:6834/1659925849,v1:172.21.15.119:6835/1659925849] up:rejoin 2024-03-28T00:34:04.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:34:03 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.119:6838/1595343240,v1:172.21.15.119:6839/1595343240] up:rejoin 2024-03-28T00:34:04.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:34:03 smithi165 ceph-mon[36302]: fsmap cephfs:5/5 3 up:standby-replay 1 up:standby 2 up:rejoin 3 up:active 2024-03-28T00:34:04.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:34:03 smithi165 ceph-mon[36302]: daemon mds.j is now active in filesystem cephfs as rank 1 2024-03-28T00:34:04.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:34:03 smithi165 ceph-mon[36302]: daemon mds.g is now active in filesystem cephfs as rank 3 2024-03-28T00:34:04.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:34:03 smithi165 ceph-mon[36302]: pgmap v250: 97 pgs: 97 active+clean; 251 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-03-28T00:34:04.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.119:6834/1659925849,v1:172.21.15.119:6835/1659925849] up:rejoin 2024-03-28T00:34:04.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.119:6838/1595343240,v1:172.21.15.119:6839/1595343240] up:rejoin 2024-03-28T00:34:04.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-mon[31518]: fsmap cephfs:5/5 3 up:standby-replay 1 up:standby 2 up:rejoin 3 up:active 2024-03-28T00:34:04.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-mon[31518]: daemon mds.j is now active in filesystem cephfs as rank 1 2024-03-28T00:34:04.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-mon[31518]: daemon mds.g is now active in filesystem cephfs as rank 3 2024-03-28T00:34:04.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-mon[31518]: pgmap v250: 97 pgs: 97 active+clean; 251 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-03-28T00:34:04.158 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7f9535000700 time 2024-03-28T00:34:03.935013+0000 2024-03-28T00:34:04.158 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T00:34:04.158 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:34:04.158 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f954096c4a1] 2024-03-28T00:34:04.158 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f954096c667] 2024-03-28T00:34:04.158 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55cf44f9a223] 2024-03-28T00:34:04.158 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55cf44e145d6] 2024-03-28T00:34:04.158 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55cf44e14da1] 2024-03-28T00:34:04.158 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: 6: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x55cf44e4909c] 2024-03-28T00:34:04.158 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55cf44e67233] 2024-03-28T00:34:04.158 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55cf44e69197] 2024-03-28T00:34:04.158 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55cf44d6811b] 2024-03-28T00:34:04.158 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55cf44d686bc] 2024-03-28T00:34:04.159 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55cf44d521ff] 2024-03-28T00:34:04.159 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f9540bfe488] 2024-03-28T00:34:04.159 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: 13: (DispatchQueue::entry()+0x51f) [0x7f9540bfb60f] 2024-03-28T00:34:04.159 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f9540ccee81] 2024-03-28T00:34:04.159 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f953f6e91ca] 2024-03-28T00:34:04.159 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: 16: clone() 2024-03-28T00:34:04.159 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7ff75e600700 time 2024-03-28T00:34:03.941108+0000 2024-03-28T00:34:04.160 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T00:34:04.160 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:34:04.160 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7ff769e324a1] 2024-03-28T00:34:04.160 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7ff769e32667] 2024-03-28T00:34:04.160 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55cfc78b8223] 2024-03-28T00:34:04.160 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55cfc77325d6] 2024-03-28T00:34:04.160 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55cfc7732da1] 2024-03-28T00:34:04.160 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: 6: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x55cfc776709c] 2024-03-28T00:34:04.160 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55cfc7785233] 2024-03-28T00:34:04.160 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55cfc7787197] 2024-03-28T00:34:04.160 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55cfc768611b] 2024-03-28T00:34:04.160 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55cfc76866bc] 2024-03-28T00:34:04.161 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55cfc76701ff] 2024-03-28T00:34:04.161 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7ff76a0c4488] 2024-03-28T00:34:04.161 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: 13: (DispatchQueue::entry()+0x51f) [0x7ff76a0c160f] 2024-03-28T00:34:04.161 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7ff76a194e81] 2024-03-28T00:34:04.161 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7ff768baf1ca] 2024-03-28T00:34:04.161 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: 16: clone() 2024-03-28T00:34:04.161 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: *** Caught signal (Aborted) ** 2024-03-28T00:34:04.161 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: in thread 7ff75e600700 thread_name:ms_dispatch 2024-03-28T00:34:04.161 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: 2024-03-28T00:34:03.935+0000 7f9535000700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7f9535000700 time 2024-03-28T00:34:03.935013+0000 2024-03-28T00:34:04.161 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T00:34:04.162 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: 2024-03-28T00:34:04.162 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:34:04.162 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f954096c4a1] 2024-03-28T00:34:04.162 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f954096c667] 2024-03-28T00:34:04.162 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55cf44f9a223] 2024-03-28T00:34:04.162 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55cf44e145d6] 2024-03-28T00:34:04.162 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55cf44e14da1] 2024-03-28T00:34:04.162 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: 6: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x55cf44e4909c] 2024-03-28T00:34:04.162 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55cf44e67233] 2024-03-28T00:34:04.162 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55cf44e69197] 2024-03-28T00:34:04.162 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55cf44d6811b] 2024-03-28T00:34:04.162 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55cf44d686bc] 2024-03-28T00:34:04.162 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55cf44d521ff] 2024-03-28T00:34:04.162 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f9540bfe488] 2024-03-28T00:34:04.163 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: 13: (DispatchQueue::entry()+0x51f) [0x7f9540bfb60f] 2024-03-28T00:34:04.163 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f9540ccee81] 2024-03-28T00:34:04.163 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f953f6e91ca] 2024-03-28T00:34:04.163 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: 16: clone() 2024-03-28T00:34:04.163 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: 2024-03-28T00:34:04.163 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: *** Caught signal (Aborted) ** 2024-03-28T00:34:04.163 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: in thread 7f9535000700 thread_name:ms_dispatch 2024-03-28T00:34:04.163 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:34:04.163 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f953f6f3d20] 2024-03-28T00:34:04.163 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: 2024-03-28T00:34:03.941+0000 7ff75e600700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7ff75e600700 time 2024-03-28T00:34:03.941108+0000 2024-03-28T00:34:04.163 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T00:34:04.164 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: 2024-03-28T00:34:04.164 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:34:04.164 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7ff769e324a1] 2024-03-28T00:34:04.164 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7ff769e32667] 2024-03-28T00:34:04.164 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55cfc78b8223] 2024-03-28T00:34:04.164 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55cfc77325d6] 2024-03-28T00:34:04.164 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55cfc7732da1] 2024-03-28T00:34:04.164 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: 6: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x55cfc776709c] 2024-03-28T00:34:04.164 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55cfc7785233] 2024-03-28T00:34:04.164 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55cfc7787197] 2024-03-28T00:34:04.164 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55cfc768611b] 2024-03-28T00:34:04.164 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55cfc76866bc] 2024-03-28T00:34:04.164 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55cfc76701ff] 2024-03-28T00:34:04.164 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7ff76a0c4488] 2024-03-28T00:34:04.164 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: 13: (DispatchQueue::entry()+0x51f) [0x7ff76a0c160f] 2024-03-28T00:34:04.165 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7ff76a194e81] 2024-03-28T00:34:04.165 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7ff768baf1ca] 2024-03-28T00:34:04.165 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: 16: clone() 2024-03-28T00:34:04.165 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: 2024-03-28T00:34:04.165 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:34:04.165 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7ff768bb9d20] 2024-03-28T00:34:04.165 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: 2: gsignal() 2024-03-28T00:34:04.165 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: 3: abort() 2024-03-28T00:34:04.165 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f954096c4fb] 2024-03-28T00:34:04.165 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f954096c667] 2024-03-28T00:34:04.165 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55cf44f9a223] 2024-03-28T00:34:04.165 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55cf44e145d6] 2024-03-28T00:34:04.166 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55cf44e14da1] 2024-03-28T00:34:04.166 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: 9: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x55cf44e4909c] 2024-03-28T00:34:04.166 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55cf44e67233] 2024-03-28T00:34:04.166 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55cf44e69197] 2024-03-28T00:34:04.166 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55cf44d6811b] 2024-03-28T00:34:04.166 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55cf44d686bc] 2024-03-28T00:34:04.166 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55cf44d521ff] 2024-03-28T00:34:04.166 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f9540bfe488] 2024-03-28T00:34:04.166 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: 16: (DispatchQueue::entry()+0x51f) [0x7f9540bfb60f] 2024-03-28T00:34:04.166 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f9540ccee81] 2024-03-28T00:34:04.166 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f953f6e91ca] 2024-03-28T00:34:04.166 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: 19: clone() 2024-03-28T00:34:04.166 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: 2024-03-28T00:34:03.937+0000 7f9535000700 -1 *** Caught signal (Aborted) ** 2024-03-28T00:34:04.166 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: in thread 7f9535000700 thread_name:ms_dispatch 2024-03-28T00:34:04.166 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: 2024-03-28T00:34:04.167 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:34:04.167 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f953f6f3d20] 2024-03-28T00:34:04.167 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: 2: gsignal() 2024-03-28T00:34:04.167 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: 3: abort() 2024-03-28T00:34:04.167 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f954096c4fb] 2024-03-28T00:34:04.167 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f954096c667] 2024-03-28T00:34:04.167 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: 2: gsignal() 2024-03-28T00:34:04.167 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: 3: abort() 2024-03-28T00:34:04.167 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7ff769e324fb] 2024-03-28T00:34:04.167 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7ff769e32667] 2024-03-28T00:34:04.167 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55cfc78b8223] 2024-03-28T00:34:04.168 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55cfc77325d6] 2024-03-28T00:34:04.168 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55cfc7732da1] 2024-03-28T00:34:04.168 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: 9: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x55cfc776709c] 2024-03-28T00:34:04.168 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55cfc7785233] 2024-03-28T00:34:04.168 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55cfc7787197] 2024-03-28T00:34:04.168 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55cfc768611b] 2024-03-28T00:34:04.168 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55cfc76866bc] 2024-03-28T00:34:04.168 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55cfc76701ff] 2024-03-28T00:34:04.168 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7ff76a0c4488] 2024-03-28T00:34:04.168 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: 16: (DispatchQueue::entry()+0x51f) [0x7ff76a0c160f] 2024-03-28T00:34:04.168 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7ff76a194e81] 2024-03-28T00:34:04.168 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7ff768baf1ca] 2024-03-28T00:34:04.168 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: 19: clone() 2024-03-28T00:34:04.168 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: 2024-03-28T00:34:03.955+0000 7ff75e600700 -1 *** Caught signal (Aborted) ** 2024-03-28T00:34:04.169 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: in thread 7ff75e600700 thread_name:ms_dispatch 2024-03-28T00:34:04.169 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: 2024-03-28T00:34:04.169 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:34:04.169 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7ff768bb9d20] 2024-03-28T00:34:04.169 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: 2: gsignal() 2024-03-28T00:34:04.169 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: 3: abort() 2024-03-28T00:34:04.169 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7ff769e324fb] 2024-03-28T00:34:04.169 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7ff769e32667] 2024-03-28T00:34:04.169 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55cf44f9a223] 2024-03-28T00:34:04.169 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55cf44e145d6] 2024-03-28T00:34:04.169 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55cf44e14da1] 2024-03-28T00:34:04.170 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: 9: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x55cf44e4909c] 2024-03-28T00:34:04.170 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55cf44e67233] 2024-03-28T00:34:04.170 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55cf44e69197] 2024-03-28T00:34:04.170 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55cf44d6811b] 2024-03-28T00:34:04.170 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55cf44d686bc] 2024-03-28T00:34:04.170 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55cf44d521ff] 2024-03-28T00:34:04.170 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f9540bfe488] 2024-03-28T00:34:04.170 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: 16: (DispatchQueue::entry()+0x51f) [0x7f9540bfb60f] 2024-03-28T00:34:04.170 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f9540ccee81] 2024-03-28T00:34:04.170 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f953f6e91ca] 2024-03-28T00:34:04.170 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: 19: clone() 2024-03-28T00:34:04.170 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T00:34:04.170 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: 2024-03-28T00:34:04.170 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: -1> 2024-03-28T00:34:03.935+0000 7f9535000700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7f9535000700 time 2024-03-28T00:34:03.935013+0000 2024-03-28T00:34:04.171 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T00:34:04.171 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: 2024-03-28T00:34:04.171 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:34:04.171 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f954096c4a1] 2024-03-28T00:34:04.171 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f954096c667] 2024-03-28T00:34:04.171 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55cfc78b8223] 2024-03-28T00:34:04.171 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55cfc77325d6] 2024-03-28T00:34:04.171 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55cfc7732da1] 2024-03-28T00:34:04.171 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: 9: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x55cfc776709c] 2024-03-28T00:34:04.171 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55cfc7785233] 2024-03-28T00:34:04.172 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55cfc7787197] 2024-03-28T00:34:04.172 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55cfc768611b] 2024-03-28T00:34:04.172 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55cfc76866bc] 2024-03-28T00:34:04.172 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55cfc76701ff] 2024-03-28T00:34:04.172 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7ff76a0c4488] 2024-03-28T00:34:04.172 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: 16: (DispatchQueue::entry()+0x51f) [0x7ff76a0c160f] 2024-03-28T00:34:04.172 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7ff76a194e81] 2024-03-28T00:34:04.172 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7ff768baf1ca] 2024-03-28T00:34:04.172 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: 19: clone() 2024-03-28T00:34:04.172 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T00:34:04.172 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: 2024-03-28T00:34:04.172 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: -1> 2024-03-28T00:34:03.941+0000 7ff75e600700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7ff75e600700 time 2024-03-28T00:34:03.941108+0000 2024-03-28T00:34:04.172 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T00:34:04.172 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: 2024-03-28T00:34:04.172 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:34:04.173 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7ff769e324a1] 2024-03-28T00:34:04.173 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7ff769e32667] 2024-03-28T00:34:04.173 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55cf44f9a223] 2024-03-28T00:34:04.173 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55cf44e145d6] 2024-03-28T00:34:04.173 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55cf44e14da1] 2024-03-28T00:34:04.173 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: 6: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x55cf44e4909c] 2024-03-28T00:34:04.173 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55cf44e67233] 2024-03-28T00:34:04.174 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55cf44e69197] 2024-03-28T00:34:04.174 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55cf44d6811b] 2024-03-28T00:34:04.174 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55cf44d686bc] 2024-03-28T00:34:04.174 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55cf44d521ff] 2024-03-28T00:34:04.174 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f9540bfe488] 2024-03-28T00:34:04.174 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: 13: (DispatchQueue::entry()+0x51f) [0x7f9540bfb60f] 2024-03-28T00:34:04.174 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f9540ccee81] 2024-03-28T00:34:04.174 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f953f6e91ca] 2024-03-28T00:34:04.174 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: 16: clone() 2024-03-28T00:34:04.174 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: 2024-03-28T00:34:04.174 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: 0> 2024-03-28T00:34:03.937+0000 7f9535000700 -1 *** Caught signal (Aborted) ** 2024-03-28T00:34:04.174 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: in thread 7f9535000700 thread_name:ms_dispatch 2024-03-28T00:34:04.174 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: 2024-03-28T00:34:04.175 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:34:04.175 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f953f6f3d20] 2024-03-28T00:34:04.175 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: 2: gsignal() 2024-03-28T00:34:04.175 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: 3: abort() 2024-03-28T00:34:04.175 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f954096c4fb] 2024-03-28T00:34:04.175 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f954096c667] 2024-03-28T00:34:04.175 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55cf44f9a223] 2024-03-28T00:34:04.175 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55cf44e145d6] 2024-03-28T00:34:04.175 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55cf44e14da1] 2024-03-28T00:34:04.175 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: 9: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x55cf44e4909c] 2024-03-28T00:34:04.175 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55cf44e67233] 2024-03-28T00:34:04.175 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55cf44e69197] 2024-03-28T00:34:04.175 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55cf44d6811b] 2024-03-28T00:34:04.175 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55cf44d686bc] 2024-03-28T00:34:04.175 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55cf44d521ff] 2024-03-28T00:34:04.176 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f9540bfe488] 2024-03-28T00:34:04.176 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: 16: (DispatchQueue::entry()+0x51f) [0x7f9540bfb60f] 2024-03-28T00:34:04.176 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f9540ccee81] 2024-03-28T00:34:04.176 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f953f6e91ca] 2024-03-28T00:34:04.176 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: 19: clone() 2024-03-28T00:34:04.176 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T00:34:04.176 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: 2024-03-28T00:34:04.176 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: -1> 2024-03-28T00:34:03.935+0000 7f9535000700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7f9535000700 time 2024-03-28T00:34:03.935013+0000 2024-03-28T00:34:04.176 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T00:34:04.176 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: 2024-03-28T00:34:04.176 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:34:04.176 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f954096c4a1] 2024-03-28T00:34:04.176 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f954096c667] 2024-03-28T00:34:04.176 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55cf44f9a223] 2024-03-28T00:34:04.176 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55cf44e145d6] 2024-03-28T00:34:04.177 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55cfc78b8223] 2024-03-28T00:34:04.177 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55cfc77325d6] 2024-03-28T00:34:04.177 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55cfc7732da1] 2024-03-28T00:34:04.177 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: 6: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x55cfc776709c] 2024-03-28T00:34:04.177 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55cfc7785233] 2024-03-28T00:34:04.177 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55cfc7787197] 2024-03-28T00:34:04.177 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55cfc768611b] 2024-03-28T00:34:04.177 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55cfc76866bc] 2024-03-28T00:34:04.177 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55cfc76701ff] 2024-03-28T00:34:04.177 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7ff76a0c4488] 2024-03-28T00:34:04.177 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: 13: (DispatchQueue::entry()+0x51f) [0x7ff76a0c160f] 2024-03-28T00:34:04.177 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7ff76a194e81] 2024-03-28T00:34:04.178 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7ff768baf1ca] 2024-03-28T00:34:04.178 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: 16: clone() 2024-03-28T00:34:04.178 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: 2024-03-28T00:34:04.178 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: 0> 2024-03-28T00:34:03.955+0000 7ff75e600700 -1 *** Caught signal (Aborted) ** 2024-03-28T00:34:04.178 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: in thread 7ff75e600700 thread_name:ms_dispatch 2024-03-28T00:34:04.178 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: 2024-03-28T00:34:04.178 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:34:04.178 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7ff768bb9d20] 2024-03-28T00:34:04.178 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: 2: gsignal() 2024-03-28T00:34:04.178 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: 3: abort() 2024-03-28T00:34:04.178 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7ff769e324fb] 2024-03-28T00:34:04.178 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7ff769e32667] 2024-03-28T00:34:04.178 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55cfc78b8223] 2024-03-28T00:34:04.178 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55cfc77325d6] 2024-03-28T00:34:04.179 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55cf44e14da1] 2024-03-28T00:34:04.179 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: 6: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x55cf44e4909c] 2024-03-28T00:34:04.179 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55cf44e67233] 2024-03-28T00:34:04.179 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55cf44e69197] 2024-03-28T00:34:04.179 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55cf44d6811b] 2024-03-28T00:34:04.179 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55cf44d686bc] 2024-03-28T00:34:04.179 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55cf44d521ff] 2024-03-28T00:34:04.179 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f9540bfe488] 2024-03-28T00:34:04.179 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: 13: (DispatchQueue::entry()+0x51f) [0x7f9540bfb60f] 2024-03-28T00:34:04.179 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f9540ccee81] 2024-03-28T00:34:04.179 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f953f6e91ca] 2024-03-28T00:34:04.180 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: 16: clone() 2024-03-28T00:34:04.180 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: 2024-03-28T00:34:04.180 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: 0> 2024-03-28T00:34:03.937+0000 7f9535000700 -1 *** Caught signal (Aborted) ** 2024-03-28T00:34:04.180 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: in thread 7f9535000700 thread_name:ms_dispatch 2024-03-28T00:34:04.180 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: 2024-03-28T00:34:04.180 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:34:04.180 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f953f6f3d20] 2024-03-28T00:34:04.180 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: 2: gsignal() 2024-03-28T00:34:04.180 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: 3: abort() 2024-03-28T00:34:04.180 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f954096c4fb] 2024-03-28T00:34:04.180 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f954096c667] 2024-03-28T00:34:04.180 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55cf44f9a223] 2024-03-28T00:34:04.180 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55cf44e145d6] 2024-03-28T00:34:04.180 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55cf44e14da1] 2024-03-28T00:34:04.180 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: 9: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x55cf44e4909c] 2024-03-28T00:34:04.181 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55cfc7732da1] 2024-03-28T00:34:04.181 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: 9: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x55cfc776709c] 2024-03-28T00:34:04.181 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55cfc7785233] 2024-03-28T00:34:04.181 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55cfc7787197] 2024-03-28T00:34:04.181 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55cfc768611b] 2024-03-28T00:34:04.181 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55cfc76866bc] 2024-03-28T00:34:04.181 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55cfc76701ff] 2024-03-28T00:34:04.181 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7ff76a0c4488] 2024-03-28T00:34:04.181 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: 16: (DispatchQueue::entry()+0x51f) [0x7ff76a0c160f] 2024-03-28T00:34:04.181 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7ff76a194e81] 2024-03-28T00:34:04.181 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7ff768baf1ca] 2024-03-28T00:34:04.182 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: 19: clone() 2024-03-28T00:34:04.182 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T00:34:04.182 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: 2024-03-28T00:34:04.182 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: -1> 2024-03-28T00:34:03.941+0000 7ff75e600700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7ff75e600700 time 2024-03-28T00:34:03.941108+0000 2024-03-28T00:34:04.182 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T00:34:04.182 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: 2024-03-28T00:34:04.182 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:34:04.182 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7ff769e324a1] 2024-03-28T00:34:04.182 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7ff769e32667] 2024-03-28T00:34:04.182 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55cfc78b8223] 2024-03-28T00:34:04.182 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55cfc77325d6] 2024-03-28T00:34:04.182 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55cfc7732da1] 2024-03-28T00:34:04.183 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55cf44e67233] 2024-03-28T00:34:04.183 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55cf44e69197] 2024-03-28T00:34:04.183 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55cf44d6811b] 2024-03-28T00:34:04.183 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55cf44d686bc] 2024-03-28T00:34:04.183 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55cf44d521ff] 2024-03-28T00:34:04.183 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f9540bfe488] 2024-03-28T00:34:04.183 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: 16: (DispatchQueue::entry()+0x51f) [0x7f9540bfb60f] 2024-03-28T00:34:04.183 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f9540ccee81] 2024-03-28T00:34:04.183 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f953f6e91ca] 2024-03-28T00:34:04.183 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: 19: clone() 2024-03-28T00:34:04.183 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T00:34:04.183 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[114751]: 2024-03-28T00:34:04.184 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: 6: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x55cfc776709c] 2024-03-28T00:34:04.184 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55cfc7785233] 2024-03-28T00:34:04.184 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55cfc7787197] 2024-03-28T00:34:04.184 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55cfc768611b] 2024-03-28T00:34:04.184 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55cfc76866bc] 2024-03-28T00:34:04.184 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55cfc76701ff] 2024-03-28T00:34:04.184 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7ff76a0c4488] 2024-03-28T00:34:04.184 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: 13: (DispatchQueue::entry()+0x51f) [0x7ff76a0c160f] 2024-03-28T00:34:04.184 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7ff76a194e81] 2024-03-28T00:34:04.184 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7ff768baf1ca] 2024-03-28T00:34:04.184 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: 16: clone() 2024-03-28T00:34:04.184 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: 2024-03-28T00:34:04.184 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: 0> 2024-03-28T00:34:03.955+0000 7ff75e600700 -1 *** Caught signal (Aborted) ** 2024-03-28T00:34:04.185 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: in thread 7ff75e600700 thread_name:ms_dispatch 2024-03-28T00:34:04.185 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: 2024-03-28T00:34:04.185 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:34:04.185 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7ff768bb9d20] 2024-03-28T00:34:04.185 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: 2: gsignal() 2024-03-28T00:34:04.185 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: 3: abort() 2024-03-28T00:34:04.185 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7ff769e324fb] 2024-03-28T00:34:04.185 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7ff769e32667] 2024-03-28T00:34:04.185 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55cfc78b8223] 2024-03-28T00:34:04.185 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55cfc77325d6] 2024-03-28T00:34:04.185 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55cfc7732da1] 2024-03-28T00:34:04.185 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: 9: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x55cfc776709c] 2024-03-28T00:34:04.185 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55cfc7785233] 2024-03-28T00:34:04.185 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55cfc7787197] 2024-03-28T00:34:04.185 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55cfc768611b] 2024-03-28T00:34:04.186 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55cfc76866bc] 2024-03-28T00:34:04.186 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55cfc76701ff] 2024-03-28T00:34:04.186 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7ff76a0c4488] 2024-03-28T00:34:04.186 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: 16: (DispatchQueue::entry()+0x51f) [0x7ff76a0c160f] 2024-03-28T00:34:04.186 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7ff76a194e81] 2024-03-28T00:34:04.186 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7ff768baf1ca] 2024-03-28T00:34:04.186 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: 19: clone() 2024-03-28T00:34:04.186 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T00:34:04.186 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[69757]: 2024-03-28T00:34:04.186 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:04 smithi119 podman[116136]: 2024-03-28 00:34:04.068622648 +0000 UTC m=+0.035867249 container died 68ec75c44d59c37dc7bf95509cef9c053c1fd7a68de316d11a697648ed87d0a8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-28T00:34:04.205 INFO:journalctl@ceph.mds.b.smithi132.stdout:Mar 28 00:34:04 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-b[66443]: 2024-03-28T00:34:04.046+0000 7f857cecc700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-03-28T00:34:04.205 INFO:journalctl@ceph.mds.b.smithi132.stdout:Mar 28 00:34:04 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-b[66443]: 2024-03-28T00:34:04.046+0000 7f857cecc700 -1 mds.pinger is_rank_lagging: rank=3 was never sent ping request. 2024-03-28T00:34:04.205 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:34:03 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.119:6834/1659925849,v1:172.21.15.119:6835/1659925849] up:rejoin 2024-03-28T00:34:04.205 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:34:03 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.119:6838/1595343240,v1:172.21.15.119:6839/1595343240] up:rejoin 2024-03-28T00:34:04.205 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:34:03 smithi132 ceph-mon[35941]: fsmap cephfs:5/5 3 up:standby-replay 1 up:standby 2 up:rejoin 3 up:active 2024-03-28T00:34:04.205 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:34:03 smithi132 ceph-mon[35941]: daemon mds.j is now active in filesystem cephfs as rank 1 2024-03-28T00:34:04.205 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:34:03 smithi132 ceph-mon[35941]: daemon mds.g is now active in filesystem cephfs as rank 3 2024-03-28T00:34:04.206 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:34:03 smithi132 ceph-mon[35941]: pgmap v250: 97 pgs: 97 active+clean; 251 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-03-28T00:34:04.907 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:34:04 smithi119 ceph-mon[31518]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-28T00:34:04.907 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:34:04 smithi119 ceph-mon[31518]: Cluster is now healthy 2024-03-28T00:34:04.907 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:34:04 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.119:6834/1659925849,v1:172.21.15.119:6835/1659925849] up:active 2024-03-28T00:34:04.907 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:34:04 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.119:6838/1595343240,v1:172.21.15.119:6839/1595343240] up:active 2024-03-28T00:34:04.908 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:34:04 smithi119 ceph-mon[31518]: fsmap cephfs:5 3 up:standby-replay 1 up:standby 5 up:active 2024-03-28T00:34:04.908 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:34:04 smithi119 ceph-mon[31518]: fsmap cephfs:5 4 up:standby-replay 5 up:active 2024-03-28T00:34:05.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:34:04 smithi165 ceph-mon[36302]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-28T00:34:05.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:34:04 smithi165 ceph-mon[36302]: Cluster is now healthy 2024-03-28T00:34:05.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:34:04 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.119:6834/1659925849,v1:172.21.15.119:6835/1659925849] up:active 2024-03-28T00:34:05.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:34:04 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.119:6838/1595343240,v1:172.21.15.119:6839/1595343240] up:active 2024-03-28T00:34:05.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:34:04 smithi165 ceph-mon[36302]: fsmap cephfs:5 3 up:standby-replay 1 up:standby 5 up:active 2024-03-28T00:34:05.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:34:04 smithi165 ceph-mon[36302]: fsmap cephfs:5 4 up:standby-replay 5 up:active 2024-03-28T00:34:05.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:34:04 smithi132 ceph-mon[35941]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-28T00:34:05.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:34:04 smithi132 ceph-mon[35941]: Cluster is now healthy 2024-03-28T00:34:05.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:34:04 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.119:6834/1659925849,v1:172.21.15.119:6835/1659925849] up:active 2024-03-28T00:34:05.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:34:04 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.119:6838/1595343240,v1:172.21.15.119:6839/1595343240] up:active 2024-03-28T00:34:05.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:34:04 smithi132 ceph-mon[35941]: fsmap cephfs:5 3 up:standby-replay 1 up:standby 5 up:active 2024-03-28T00:34:05.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:34:04 smithi132 ceph-mon[35941]: fsmap cephfs:5 4 up:standby-replay 5 up:active 2024-03-28T00:34:05.235 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:05 smithi119 podman[116136]: 2024-03-28 00:34:05.116257899 +0000 UTC m=+1.083502531 container remove 68ec75c44d59c37dc7bf95509cef9c053c1fd7a68de316d11a697648ed87d0a8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-03-28T00:34:05.235 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:05 smithi119 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.g.service: Main process exited, code=exited, status=134/n/a 2024-03-28T00:34:05.235 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:05 smithi119 podman[116130]: 2024-03-28 00:34:05.209903076 +0000 UTC m=+1.168512143 container died e87ff1b71b8829ec0814a259d6bd7ef381f0bddd2e660745ae00aa8d5a12ec27 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j, ceph=True, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-28T00:34:06.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:34:05 smithi165 ceph-mon[36302]: pgmap v251: 97 pgs: 97 active+clean; 255 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 447 KiB/s rd, 455 KiB/s wr, 78 op/s 2024-03-28T00:34:06.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:34:05 smithi119 ceph-mon[31518]: pgmap v251: 97 pgs: 97 active+clean; 255 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 447 KiB/s rd, 455 KiB/s wr, 78 op/s 2024-03-28T00:34:06.157 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:05 smithi119 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.g.service: Failed with result 'exit-code'. 2024-03-28T00:34:06.157 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:05 smithi119 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.g.service: Consumed 2.237s CPU time. 2024-03-28T00:34:06.157 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:05 smithi119 podman[116130]: 2024-03-28 00:34:05.946545943 +0000 UTC m=+1.905155040 container remove e87ff1b71b8829ec0814a259d6bd7ef381f0bddd2e660745ae00aa8d5a12ec27 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2) 2024-03-28T00:34:06.157 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:05 smithi119 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.j.service: Main process exited, code=exited, status=134/n/a 2024-03-28T00:34:06.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:34:05 smithi132 ceph-mon[35941]: pgmap v251: 97 pgs: 97 active+clean; 255 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 447 KiB/s rd, 455 KiB/s wr, 78 op/s 2024-03-28T00:34:06.885 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:06 smithi119 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.j.service: Failed with result 'exit-code'. 2024-03-28T00:34:06.885 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:06 smithi119 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.j.service: Consumed 1.721s CPU time. 2024-03-28T00:34:08.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:34:08 smithi165 ceph-mon[36302]: pgmap v252: 97 pgs: 97 active+clean; 255 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 447 KiB/s rd, 455 KiB/s wr, 78 op/s 2024-03-28T00:34:08.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:34:08 smithi119 ceph-mon[31518]: pgmap v252: 97 pgs: 97 active+clean; 255 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 447 KiB/s rd, 455 KiB/s wr, 78 op/s 2024-03-28T00:34:08.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:34:08 smithi132 ceph-mon[35941]: pgmap v252: 97 pgs: 97 active+clean; 255 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 447 KiB/s rd, 455 KiB/s wr, 78 op/s 2024-03-28T00:34:10.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:34:10 smithi165 ceph-mon[36302]: pgmap v253: 97 pgs: 97 active+clean; 262 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 79 op/s 2024-03-28T00:34:10.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:34:10 smithi119 ceph-mon[31518]: pgmap v253: 97 pgs: 97 active+clean; 262 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 79 op/s 2024-03-28T00:34:10.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:34:10 smithi132 ceph-mon[35941]: pgmap v253: 97 pgs: 97 active+clean; 262 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 79 op/s 2024-03-28T00:34:11.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:34:11 smithi165 ceph-mon[36302]: fsmap cephfs:5 4 up:standby-replay 5 up:active 2024-03-28T00:34:11.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:34:11 smithi119 ceph-mon[31518]: fsmap cephfs:5 4 up:standby-replay 5 up:active 2024-03-28T00:34:11.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:34:11 smithi132 ceph-mon[35941]: fsmap cephfs:5 4 up:standby-replay 5 up:active 2024-03-28T00:34:12.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:34:12 smithi165 ceph-mon[36302]: pgmap v254: 97 pgs: 97 active+clean; 262 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 999 KiB/s rd, 1008 KiB/s wr, 69 op/s 2024-03-28T00:34:12.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:34:12 smithi119 ceph-mon[31518]: pgmap v254: 97 pgs: 97 active+clean; 262 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 999 KiB/s rd, 1008 KiB/s wr, 69 op/s 2024-03-28T00:34:12.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:34:12 smithi132 ceph-mon[35941]: pgmap v254: 97 pgs: 97 active+clean; 262 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 999 KiB/s rd, 1008 KiB/s wr, 69 op/s 2024-03-28T00:34:14.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:34:14 smithi165 ceph-mon[36302]: pgmap v255: 97 pgs: 97 active+clean; 262 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 948 KiB/s rd, 956 KiB/s wr, 66 op/s 2024-03-28T00:34:14.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:34:14 smithi119 ceph-mon[31518]: pgmap v255: 97 pgs: 97 active+clean; 262 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 948 KiB/s rd, 956 KiB/s wr, 66 op/s 2024-03-28T00:34:14.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:34:14 smithi132 ceph-mon[35941]: pgmap v255: 97 pgs: 97 active+clean; 262 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 948 KiB/s rd, 956 KiB/s wr, 66 op/s 2024-03-28T00:34:16.283 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:15 smithi119 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.g.service: Scheduled restart job, restart counter is at 1. 2024-03-28T00:34:16.284 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:15 smithi119 systemd[1]: Stopped Ceph mds.g for 10f73b92-ec98-11ee-95d0-87774f69a715. 2024-03-28T00:34:16.284 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:15 smithi119 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.g.service: Consumed 2.237s CPU time. 2024-03-28T00:34:16.284 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:15 smithi119 systemd[1]: Starting Ceph mds.g for 10f73b92-ec98-11ee-95d0-87774f69a715... 2024-03-28T00:34:16.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:34:16 smithi165 ceph-mon[36302]: pgmap v256: 97 pgs: 97 active+clean; 262 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 949 KiB/s rd, 956 KiB/s wr, 70 op/s 2024-03-28T00:34:16.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:34:16 smithi165 ceph-mon[36302]: fsmap cephfs:5 4 up:standby-replay 5 up:active 2024-03-28T00:34:16.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:34:16 smithi119 ceph-mon[31518]: pgmap v256: 97 pgs: 97 active+clean; 262 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 949 KiB/s rd, 956 KiB/s wr, 70 op/s 2024-03-28T00:34:16.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:34:16 smithi119 ceph-mon[31518]: fsmap cephfs:5 4 up:standby-replay 5 up:active 2024-03-28T00:34:16.651 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:16 smithi119 podman[116266]: 2024-03-28 00:34:16.215990237 +0000 UTC m=+0.035817441 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-28T00:34:16.651 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:16 smithi119 podman[116266]: 2024-03-28 00:34:16.331004509 +0000 UTC m=+0.150831747 container create 2d8b060c1281c86f4eb7f5e59ad8b16387bfb81f9ec23dccd907c0e6a2620b73 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-28T00:34:16.651 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:16 smithi119 podman[116266]: 2024-03-28 00:34:16.643432251 +0000 UTC m=+0.463259496 container init 2d8b060c1281c86f4eb7f5e59ad8b16387bfb81f9ec23dccd907c0e6a2620b73 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-03-28T00:34:16.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:34:16 smithi132 ceph-mon[35941]: pgmap v256: 97 pgs: 97 active+clean; 262 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 949 KiB/s rd, 956 KiB/s wr, 70 op/s 2024-03-28T00:34:16.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:34:16 smithi132 ceph-mon[35941]: fsmap cephfs:5 4 up:standby-replay 5 up:active 2024-03-28T00:34:16.906 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:16 smithi119 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.j.service: Scheduled restart job, restart counter is at 5. 2024-03-28T00:34:16.907 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:16 smithi119 systemd[1]: Stopped Ceph mds.j for 10f73b92-ec98-11ee-95d0-87774f69a715. 2024-03-28T00:34:16.907 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:16 smithi119 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.j.service: Consumed 1.721s CPU time. 2024-03-28T00:34:16.907 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:16 smithi119 systemd[1]: Starting Ceph mds.j for 10f73b92-ec98-11ee-95d0-87774f69a715... 2024-03-28T00:34:16.907 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:16 smithi119 podman[116266]: 2024-03-28 00:34:16.655128051 +0000 UTC m=+0.474955257 container start 2d8b060c1281c86f4eb7f5e59ad8b16387bfb81f9ec23dccd907c0e6a2620b73 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-03-28T00:34:16.907 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:16 smithi119 ceph-mds[116284]: -- 172.21.15.119:0/3469191442 <== mon.2 v2:172.21.15.132:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x556649abba40 con 0x556649a8ec00 2024-03-28T00:34:16.907 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: starting mds.g at 2024-03-28T00:34:16.907 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:16 smithi119 bash[116266]: 2d8b060c1281c86f4eb7f5e59ad8b16387bfb81f9ec23dccd907c0e6a2620b73 2024-03-28T00:34:16.907 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:16 smithi119 systemd[1]: Started Ceph mds.g for 10f73b92-ec98-11ee-95d0-87774f69a715. 2024-03-28T00:34:17.405 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:17 smithi119 podman[116362]: 2024-03-28 00:34:16.967719737 +0000 UTC m=+0.033354796 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-28T00:34:17.405 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:17 smithi119 podman[116362]: 2024-03-28 00:34:17.135675109 +0000 UTC m=+0.201310194 container create 6d699ce8e535466d3bfa776c07efad31cc4e4741c4452684447f1edb995cb013 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-28T00:34:17.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:34:17 smithi119 ceph-mon[31518]: daemon mds.g restarted 2024-03-28T00:34:17.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:34:17 smithi119 ceph-mon[31518]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-28T00:34:17.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:34:17 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-28T00:34:17.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:34:17 smithi119 ceph-mon[31518]: osdmap e111: 12 total, 12 up, 12 in 2024-03-28T00:34:17.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:34:17 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.119:6834/4197433095,v1:172.21.15.119:6835/4197433095] up:boot 2024-03-28T00:34:17.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:34:17 smithi119 ceph-mon[31518]: Standby daemon mds.g assigned to filesystem cephfs as rank 3 2024-03-28T00:34:17.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:34:17 smithi119 ceph-mon[31518]: fsmap cephfs:4/5 {0=b=up:active,1=j=up:active,2=h=up:active,4=f=up:active} 4 up:standby-replay 1 up:standby, 1 failed 2024-03-28T00:34:17.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:34:17 smithi119 ceph-mon[31518]: fsmap cephfs:5/5 4 up:standby-replay 1 up:replay 4 up:active 2024-03-28T00:34:17.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:34:17 smithi165 ceph-mon[36302]: daemon mds.g restarted 2024-03-28T00:34:17.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:34:17 smithi165 ceph-mon[36302]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-28T00:34:17.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:34:17 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-28T00:34:17.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:34:17 smithi165 ceph-mon[36302]: osdmap e111: 12 total, 12 up, 12 in 2024-03-28T00:34:17.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:34:17 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.119:6834/4197433095,v1:172.21.15.119:6835/4197433095] up:boot 2024-03-28T00:34:17.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:34:17 smithi165 ceph-mon[36302]: Standby daemon mds.g assigned to filesystem cephfs as rank 3 2024-03-28T00:34:17.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:34:17 smithi165 ceph-mon[36302]: fsmap cephfs:4/5 {0=b=up:active,1=j=up:active,2=h=up:active,4=f=up:active} 4 up:standby-replay 1 up:standby, 1 failed 2024-03-28T00:34:17.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:34:17 smithi165 ceph-mon[36302]: fsmap cephfs:5/5 4 up:standby-replay 1 up:replay 4 up:active 2024-03-28T00:34:17.657 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:17 smithi119 podman[116362]: 2024-03-28 00:34:17.398153012 +0000 UTC m=+0.463788104 container init 6d699ce8e535466d3bfa776c07efad31cc4e4741c4452684447f1edb995cb013 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-28T00:34:17.657 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:17 smithi119 podman[116362]: 2024-03-28 00:34:17.408952662 +0000 UTC m=+0.474587716 container start 6d699ce8e535466d3bfa776c07efad31cc4e4741c4452684447f1edb995cb013 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-03-28T00:34:17.657 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:17 smithi119 ceph-mds[116381]: -- 172.21.15.119:0/3557436527 <== mon.1 v2:172.21.15.165:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55b8bb24ba40 con 0x55b8bb21e800 2024-03-28T00:34:17.658 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:17 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: starting mds.j at 2024-03-28T00:34:17.658 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:17 smithi119 bash[116362]: 6d699ce8e535466d3bfa776c07efad31cc4e4741c4452684447f1edb995cb013 2024-03-28T00:34:17.658 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:17 smithi119 systemd[1]: Started Ceph mds.j for 10f73b92-ec98-11ee-95d0-87774f69a715. 2024-03-28T00:34:17.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:34:17 smithi132 ceph-mon[35941]: daemon mds.g restarted 2024-03-28T00:34:17.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:34:17 smithi132 ceph-mon[35941]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-28T00:34:17.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:34:17 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-28T00:34:17.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:34:17 smithi132 ceph-mon[35941]: osdmap e111: 12 total, 12 up, 12 in 2024-03-28T00:34:17.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:34:17 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.119:6834/4197433095,v1:172.21.15.119:6835/4197433095] up:boot 2024-03-28T00:34:17.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:34:17 smithi132 ceph-mon[35941]: Standby daemon mds.g assigned to filesystem cephfs as rank 3 2024-03-28T00:34:17.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:34:17 smithi132 ceph-mon[35941]: fsmap cephfs:4/5 {0=b=up:active,1=j=up:active,2=h=up:active,4=f=up:active} 4 up:standby-replay 1 up:standby, 1 failed 2024-03-28T00:34:17.705 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:34:17 smithi132 ceph-mon[35941]: fsmap cephfs:5/5 4 up:standby-replay 1 up:replay 4 up:active 2024-03-28T00:34:18.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:34:18 smithi165 ceph-mon[36302]: pgmap v258: 97 pgs: 97 active+clean; 262 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 694 KiB/s rd, 692 KiB/s wr, 12 op/s 2024-03-28T00:34:18.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:34:18 smithi165 ceph-mon[36302]: daemon mds.j restarted 2024-03-28T00:34:18.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:34:18 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-28T00:34:18.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:34:18 smithi165 ceph-mon[36302]: osdmap e112: 12 total, 12 up, 12 in 2024-03-28T00:34:18.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:34:18 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.119:6834/4197433095,v1:172.21.15.119:6835/4197433095] up:resolve 2024-03-28T00:34:18.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:34:18 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.119:6838/3947365264,v1:172.21.15.119:6839/3947365264] up:boot 2024-03-28T00:34:18.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:34:18 smithi165 ceph-mon[36302]: Standby daemon mds.d assigned to filesystem cephfs as rank 1 2024-03-28T00:34:18.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:34:18 smithi165 ceph-mon[36302]: fsmap cephfs:4/5 {0=b=up:active,2=h=up:active,3=g=up:resolve,4=f=up:active} 4 up:standby-replay 1 up:standby, 1 failed 2024-03-28T00:34:18.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:34:18 smithi165 ceph-mon[36302]: fsmap cephfs:5/5 3 up:standby-replay 1 up:standby 1 up:replay 1 up:resolve 3 up:active 2024-03-28T00:34:18.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:34:18 smithi119 ceph-mon[31518]: pgmap v258: 97 pgs: 97 active+clean; 262 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 694 KiB/s rd, 692 KiB/s wr, 12 op/s 2024-03-28T00:34:18.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:34:18 smithi119 ceph-mon[31518]: daemon mds.j restarted 2024-03-28T00:34:18.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:34:18 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-28T00:34:18.658 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:34:18 smithi119 ceph-mon[31518]: osdmap e112: 12 total, 12 up, 12 in 2024-03-28T00:34:18.658 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:34:18 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.119:6834/4197433095,v1:172.21.15.119:6835/4197433095] up:resolve 2024-03-28T00:34:18.658 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:34:18 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.119:6838/3947365264,v1:172.21.15.119:6839/3947365264] up:boot 2024-03-28T00:34:18.658 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:34:18 smithi119 ceph-mon[31518]: Standby daemon mds.d assigned to filesystem cephfs as rank 1 2024-03-28T00:34:18.658 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:34:18 smithi119 ceph-mon[31518]: fsmap cephfs:4/5 {0=b=up:active,2=h=up:active,3=g=up:resolve,4=f=up:active} 4 up:standby-replay 1 up:standby, 1 failed 2024-03-28T00:34:18.658 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:34:18 smithi119 ceph-mon[31518]: fsmap cephfs:5/5 3 up:standby-replay 1 up:standby 1 up:replay 1 up:resolve 3 up:active 2024-03-28T00:34:18.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:34:18 smithi132 ceph-mon[35941]: pgmap v258: 97 pgs: 97 active+clean; 262 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 694 KiB/s rd, 692 KiB/s wr, 12 op/s 2024-03-28T00:34:18.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:34:18 smithi132 ceph-mon[35941]: daemon mds.j restarted 2024-03-28T00:34:18.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:34:18 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-28T00:34:18.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:34:18 smithi132 ceph-mon[35941]: osdmap e112: 12 total, 12 up, 12 in 2024-03-28T00:34:18.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:34:18 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.119:6834/4197433095,v1:172.21.15.119:6835/4197433095] up:resolve 2024-03-28T00:34:18.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:34:18 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.119:6838/3947365264,v1:172.21.15.119:6839/3947365264] up:boot 2024-03-28T00:34:18.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:34:18 smithi132 ceph-mon[35941]: Standby daemon mds.d assigned to filesystem cephfs as rank 1 2024-03-28T00:34:18.705 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:34:18 smithi132 ceph-mon[35941]: fsmap cephfs:4/5 {0=b=up:active,2=h=up:active,3=g=up:resolve,4=f=up:active} 4 up:standby-replay 1 up:standby, 1 failed 2024-03-28T00:34:18.705 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:34:18 smithi132 ceph-mon[35941]: fsmap cephfs:5/5 3 up:standby-replay 1 up:standby 1 up:replay 1 up:resolve 3 up:active 2024-03-28T00:34:20.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:34:19 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.119:6836/3988635716,v1:172.21.15.119:6837/3988635716] up:resolve 2024-03-28T00:34:20.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:34:19 smithi165 ceph-mon[36302]: fsmap cephfs:5/5 3 up:standby-replay 1 up:standby 2 up:resolve 3 up:active 2024-03-28T00:34:20.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:34:19 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.119:6836/3988635716,v1:172.21.15.119:6837/3988635716] up:resolve 2024-03-28T00:34:20.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:34:19 smithi119 ceph-mon[31518]: fsmap cephfs:5/5 3 up:standby-replay 1 up:standby 2 up:resolve 3 up:active 2024-03-28T00:34:20.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:34:19 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.119:6836/3988635716,v1:172.21.15.119:6837/3988635716] up:resolve 2024-03-28T00:34:20.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:34:19 smithi132 ceph-mon[35941]: fsmap cephfs:5/5 3 up:standby-replay 1 up:standby 2 up:resolve 3 up:active 2024-03-28T00:34:21.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:34:20 smithi165 ceph-mon[36302]: pgmap v260: 97 pgs: 97 active+clean; 262 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 127 B/s wr, 16 op/s 2024-03-28T00:34:21.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:34:20 smithi165 ceph-mon[36302]: reconnect by client.25355 172.21.15.119:0/3071152482 after 0.0039999 2024-03-28T00:34:21.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:34:20 smithi165 ceph-mon[36302]: reconnect by client.25355 172.21.15.119:0/3071152482 after 0.0039999 2024-03-28T00:34:21.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:34:20 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.119:6836/3988635716,v1:172.21.15.119:6837/3988635716] up:reconnect 2024-03-28T00:34:21.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:34:20 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.119:6834/4197433095,v1:172.21.15.119:6835/4197433095] up:reconnect 2024-03-28T00:34:21.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:34:20 smithi165 ceph-mon[36302]: fsmap cephfs:5/5 3 up:standby-replay 1 up:standby 2 up:reconnect 3 up:active 2024-03-28T00:34:21.135 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:34:20 smithi165 ceph-mon[36302]: reconnect by client.25264 v1:172.21.15.119:0/3863511511 after 0.0179996 2024-03-28T00:34:21.135 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:34:20 smithi165 ceph-mon[36302]: reconnect by client.25264 v1:172.21.15.119:0/3863511511 after 0.0179996 2024-03-28T00:34:21.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:34:20 smithi119 ceph-mon[31518]: pgmap v260: 97 pgs: 97 active+clean; 262 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 127 B/s wr, 16 op/s 2024-03-28T00:34:21.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:34:20 smithi119 ceph-mon[31518]: reconnect by client.25355 172.21.15.119:0/3071152482 after 0.0039999 2024-03-28T00:34:21.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:34:20 smithi119 ceph-mon[31518]: reconnect by client.25355 172.21.15.119:0/3071152482 after 0.0039999 2024-03-28T00:34:21.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:34:20 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.119:6836/3988635716,v1:172.21.15.119:6837/3988635716] up:reconnect 2024-03-28T00:34:21.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:34:20 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.119:6834/4197433095,v1:172.21.15.119:6835/4197433095] up:reconnect 2024-03-28T00:34:21.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:34:20 smithi119 ceph-mon[31518]: fsmap cephfs:5/5 3 up:standby-replay 1 up:standby 2 up:reconnect 3 up:active 2024-03-28T00:34:21.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:34:20 smithi119 ceph-mon[31518]: reconnect by client.25264 v1:172.21.15.119:0/3863511511 after 0.0179996 2024-03-28T00:34:21.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:34:20 smithi119 ceph-mon[31518]: reconnect by client.25264 v1:172.21.15.119:0/3863511511 after 0.0179996 2024-03-28T00:34:21.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:34:20 smithi132 ceph-mon[35941]: pgmap v260: 97 pgs: 97 active+clean; 262 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 127 B/s wr, 16 op/s 2024-03-28T00:34:21.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:34:20 smithi132 ceph-mon[35941]: reconnect by client.25355 172.21.15.119:0/3071152482 after 0.0039999 2024-03-28T00:34:21.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:34:20 smithi132 ceph-mon[35941]: reconnect by client.25355 172.21.15.119:0/3071152482 after 0.0039999 2024-03-28T00:34:21.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:34:20 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.119:6836/3988635716,v1:172.21.15.119:6837/3988635716] up:reconnect 2024-03-28T00:34:21.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:34:20 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.119:6834/4197433095,v1:172.21.15.119:6835/4197433095] up:reconnect 2024-03-28T00:34:21.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:34:20 smithi132 ceph-mon[35941]: fsmap cephfs:5/5 3 up:standby-replay 1 up:standby 2 up:reconnect 3 up:active 2024-03-28T00:34:21.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:34:20 smithi132 ceph-mon[35941]: reconnect by client.25264 v1:172.21.15.119:0/3863511511 after 0.0179996 2024-03-28T00:34:21.205 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:34:20 smithi132 ceph-mon[35941]: reconnect by client.25264 v1:172.21.15.119:0/3863511511 after 0.0179996 2024-03-28T00:34:22.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:34:21 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.119:6836/3988635716,v1:172.21.15.119:6837/3988635716] up:rejoin 2024-03-28T00:34:22.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:34:21 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.119:6834/4197433095,v1:172.21.15.119:6835/4197433095] up:rejoin 2024-03-28T00:34:22.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:34:21 smithi165 ceph-mon[36302]: fsmap cephfs:5/5 3 up:standby-replay 1 up:standby 2 up:rejoin 3 up:active 2024-03-28T00:34:22.135 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:34:21 smithi165 ceph-mon[36302]: daemon mds.d is now active in filesystem cephfs as rank 1 2024-03-28T00:34:22.135 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:34:21 smithi165 ceph-mon[36302]: daemon mds.g is now active in filesystem cephfs as rank 3 2024-03-28T00:34:22.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:34:21 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.119:6836/3988635716,v1:172.21.15.119:6837/3988635716] up:rejoin 2024-03-28T00:34:22.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:34:21 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.119:6834/4197433095,v1:172.21.15.119:6835/4197433095] up:rejoin 2024-03-28T00:34:22.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:34:21 smithi119 ceph-mon[31518]: fsmap cephfs:5/5 3 up:standby-replay 1 up:standby 2 up:rejoin 3 up:active 2024-03-28T00:34:22.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:34:21 smithi119 ceph-mon[31518]: daemon mds.d is now active in filesystem cephfs as rank 1 2024-03-28T00:34:22.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:34:21 smithi119 ceph-mon[31518]: daemon mds.g is now active in filesystem cephfs as rank 3 2024-03-28T00:34:22.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:34:21 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.119:6836/3988635716,v1:172.21.15.119:6837/3988635716] up:rejoin 2024-03-28T00:34:22.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:34:21 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.119:6834/4197433095,v1:172.21.15.119:6835/4197433095] up:rejoin 2024-03-28T00:34:22.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:34:21 smithi132 ceph-mon[35941]: fsmap cephfs:5/5 3 up:standby-replay 1 up:standby 2 up:rejoin 3 up:active 2024-03-28T00:34:22.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:34:21 smithi132 ceph-mon[35941]: daemon mds.d is now active in filesystem cephfs as rank 1 2024-03-28T00:34:22.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:34:21 smithi132 ceph-mon[35941]: daemon mds.g is now active in filesystem cephfs as rank 3 2024-03-28T00:34:23.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:34:22 smithi165 ceph-mon[36302]: pgmap v261: 97 pgs: 97 active+clean; 262 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 127 B/s wr, 16 op/s 2024-03-28T00:34:23.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:34:22 smithi165 ceph-mon[36302]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-28T00:34:23.135 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:34:22 smithi165 ceph-mon[36302]: Cluster is now healthy 2024-03-28T00:34:23.135 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:34:22 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.119:6836/3988635716,v1:172.21.15.119:6837/3988635716] up:active 2024-03-28T00:34:23.135 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:34:22 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.119:6834/4197433095,v1:172.21.15.119:6835/4197433095] up:active 2024-03-28T00:34:23.135 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:34:22 smithi165 ceph-mon[36302]: fsmap cephfs:5 3 up:standby-replay 1 up:standby 5 up:active 2024-03-28T00:34:23.135 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:34:22 smithi165 ceph-mon[36302]: fsmap cephfs:5 4 up:standby-replay 5 up:active 2024-03-28T00:34:23.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:34:22 smithi119 ceph-mon[31518]: pgmap v261: 97 pgs: 97 active+clean; 262 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 127 B/s wr, 16 op/s 2024-03-28T00:34:23.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:34:22 smithi119 ceph-mon[31518]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-28T00:34:23.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:34:22 smithi119 ceph-mon[31518]: Cluster is now healthy 2024-03-28T00:34:23.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:34:22 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.119:6836/3988635716,v1:172.21.15.119:6837/3988635716] up:active 2024-03-28T00:34:23.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:34:22 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.119:6834/4197433095,v1:172.21.15.119:6835/4197433095] up:active 2024-03-28T00:34:23.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:34:22 smithi119 ceph-mon[31518]: fsmap cephfs:5 3 up:standby-replay 1 up:standby 5 up:active 2024-03-28T00:34:23.158 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:34:22 smithi119 ceph-mon[31518]: fsmap cephfs:5 4 up:standby-replay 5 up:active 2024-03-28T00:34:23.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:34:22 smithi132 ceph-mon[35941]: pgmap v261: 97 pgs: 97 active+clean; 262 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 127 B/s wr, 16 op/s 2024-03-28T00:34:23.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:34:22 smithi132 ceph-mon[35941]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-28T00:34:23.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:34:22 smithi132 ceph-mon[35941]: Cluster is now healthy 2024-03-28T00:34:23.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:34:22 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.119:6836/3988635716,v1:172.21.15.119:6837/3988635716] up:active 2024-03-28T00:34:23.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:34:22 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.119:6834/4197433095,v1:172.21.15.119:6835/4197433095] up:active 2024-03-28T00:34:23.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:34:22 smithi132 ceph-mon[35941]: fsmap cephfs:5 3 up:standby-replay 1 up:standby 5 up:active 2024-03-28T00:34:23.205 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:34:22 smithi132 ceph-mon[35941]: fsmap cephfs:5 4 up:standby-replay 5 up:active 2024-03-28T00:34:24.776 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-mon[31518]: pgmap v262: 97 pgs: 97 active+clean; 262 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 0 B/s wr, 9 op/s 2024-03-28T00:34:24.778 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7f08db400700 time 2024-03-28T00:34:24.438113+0000 2024-03-28T00:34:24.779 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T00:34:24.779 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:34:24.779 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f08e6c684a1] 2024-03-28T00:34:24.779 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f08e6c68667] 2024-03-28T00:34:24.779 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x556a4585a223] 2024-03-28T00:34:24.779 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x556a456d45d6] 2024-03-28T00:34:24.779 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x556a456d4da1] 2024-03-28T00:34:24.779 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: 6: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x556a4570909c] 2024-03-28T00:34:24.779 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x556a45727233] 2024-03-28T00:34:24.779 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x556a45729197] 2024-03-28T00:34:24.779 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x556a4562811b] 2024-03-28T00:34:24.780 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x556a456286bc] 2024-03-28T00:34:24.780 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x556a456121ff] 2024-03-28T00:34:24.780 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f08e6efa488] 2024-03-28T00:34:24.780 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: 13: (DispatchQueue::entry()+0x51f) [0x7f08e6ef760f] 2024-03-28T00:34:24.780 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f08e6fcae81] 2024-03-28T00:34:24.780 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f08e59e51ca] 2024-03-28T00:34:24.780 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: 16: clone() 2024-03-28T00:34:24.780 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: *** Caught signal (Aborted) ** 2024-03-28T00:34:24.780 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: in thread 7f08db400700 thread_name:ms_dispatch 2024-03-28T00:34:24.780 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: 2024-03-28T00:34:24.438+0000 7f08db400700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7f08db400700 time 2024-03-28T00:34:24.438113+0000 2024-03-28T00:34:24.780 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T00:34:24.781 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: 2024-03-28T00:34:24.781 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:34:24.781 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f08e6c684a1] 2024-03-28T00:34:24.781 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f08e6c68667] 2024-03-28T00:34:24.781 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x556a4585a223] 2024-03-28T00:34:24.781 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x556a456d45d6] 2024-03-28T00:34:24.781 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x556a456d4da1] 2024-03-28T00:34:24.781 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: 6: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x556a4570909c] 2024-03-28T00:34:24.781 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x556a45727233] 2024-03-28T00:34:24.781 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x556a45729197] 2024-03-28T00:34:24.781 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x556a4562811b] 2024-03-28T00:34:24.781 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x556a456286bc] 2024-03-28T00:34:24.782 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x556a456121ff] 2024-03-28T00:34:24.782 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f08e6efa488] 2024-03-28T00:34:24.782 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: 13: (DispatchQueue::entry()+0x51f) [0x7f08e6ef760f] 2024-03-28T00:34:24.782 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f08e6fcae81] 2024-03-28T00:34:24.782 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f08e59e51ca] 2024-03-28T00:34:24.782 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: 16: clone() 2024-03-28T00:34:24.782 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: 2024-03-28T00:34:24.782 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:34:24.782 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f08e59efd20] 2024-03-28T00:34:24.782 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: 2: gsignal() 2024-03-28T00:34:24.782 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: 3: abort() 2024-03-28T00:34:24.783 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f08e6c684fb] 2024-03-28T00:34:24.783 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f08e6c68667] 2024-03-28T00:34:24.783 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x556a4585a223] 2024-03-28T00:34:24.783 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x556a456d45d6] 2024-03-28T00:34:24.783 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x556a456d4da1] 2024-03-28T00:34:24.783 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: 9: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x556a4570909c] 2024-03-28T00:34:24.783 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x556a45727233] 2024-03-28T00:34:24.783 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x556a45729197] 2024-03-28T00:34:24.783 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x556a4562811b] 2024-03-28T00:34:24.783 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x556a456286bc] 2024-03-28T00:34:24.783 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x556a456121ff] 2024-03-28T00:34:24.783 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f08e6efa488] 2024-03-28T00:34:24.783 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: 16: (DispatchQueue::entry()+0x51f) [0x7f08e6ef760f] 2024-03-28T00:34:24.783 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f08e6fcae81] 2024-03-28T00:34:24.784 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f08e59e51ca] 2024-03-28T00:34:24.784 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: 19: clone() 2024-03-28T00:34:24.784 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: 2024-03-28T00:34:24.451+0000 7f08db400700 -1 *** Caught signal (Aborted) ** 2024-03-28T00:34:24.784 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: in thread 7f08db400700 thread_name:ms_dispatch 2024-03-28T00:34:24.784 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: 2024-03-28T00:34:24.784 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:34:24.784 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f08e59efd20] 2024-03-28T00:34:24.784 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: 2: gsignal() 2024-03-28T00:34:24.784 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: 3: abort() 2024-03-28T00:34:24.784 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f08e6c684fb] 2024-03-28T00:34:24.784 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f08e6c68667] 2024-03-28T00:34:24.785 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7fee65e00700 time 2024-03-28T00:34:24.426888+0000 2024-03-28T00:34:24.785 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T00:34:24.785 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:34:24.785 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fee7177f4a1] 2024-03-28T00:34:24.785 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7fee7177f667] 2024-03-28T00:34:24.785 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x556648cac223] 2024-03-28T00:34:24.785 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x556648b265d6] 2024-03-28T00:34:24.785 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x556648b26da1] 2024-03-28T00:34:24.785 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: 6: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x556648b5b09c] 2024-03-28T00:34:24.785 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x556648b79233] 2024-03-28T00:34:24.786 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x556648b7b197] 2024-03-28T00:34:24.786 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x556648a7a11b] 2024-03-28T00:34:24.786 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x556648a7a6bc] 2024-03-28T00:34:24.786 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x556648a641ff] 2024-03-28T00:34:24.786 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7fee71a11488] 2024-03-28T00:34:24.786 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: 13: (DispatchQueue::entry()+0x51f) [0x7fee71a0e60f] 2024-03-28T00:34:24.786 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fee71ae1e81] 2024-03-28T00:34:24.786 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fee704fc1ca] 2024-03-28T00:34:24.786 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: 16: clone() 2024-03-28T00:34:24.786 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x556a4585a223] 2024-03-28T00:34:24.787 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x556a456d45d6] 2024-03-28T00:34:24.787 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x556a456d4da1] 2024-03-28T00:34:24.787 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: 9: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x556a4570909c] 2024-03-28T00:34:24.787 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x556a45727233] 2024-03-28T00:34:24.787 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x556a45729197] 2024-03-28T00:34:24.787 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x556a4562811b] 2024-03-28T00:34:24.787 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x556a456286bc] 2024-03-28T00:34:24.787 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x556a456121ff] 2024-03-28T00:34:24.787 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f08e6efa488] 2024-03-28T00:34:24.787 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: 16: (DispatchQueue::entry()+0x51f) [0x7f08e6ef760f] 2024-03-28T00:34:24.787 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f08e6fcae81] 2024-03-28T00:34:24.787 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f08e59e51ca] 2024-03-28T00:34:24.787 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: 19: clone() 2024-03-28T00:34:24.787 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T00:34:24.788 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: 2024-03-28T00:34:24.788 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: -1> 2024-03-28T00:34:24.438+0000 7f08db400700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7f08db400700 time 2024-03-28T00:34:24.438113+0000 2024-03-28T00:34:24.788 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T00:34:24.788 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: 2024-03-28T00:34:24.788 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:34:24.788 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f08e6c684a1] 2024-03-28T00:34:24.788 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f08e6c68667] 2024-03-28T00:34:24.788 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: 2024-03-28T00:34:24.428+0000 7fee65e00700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7fee65e00700 time 2024-03-28T00:34:24.426888+0000 2024-03-28T00:34:24.788 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T00:34:24.788 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: 2024-03-28T00:34:24.788 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:34:24.789 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fee7177f4a1] 2024-03-28T00:34:24.789 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7fee7177f667] 2024-03-28T00:34:24.789 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x556648cac223] 2024-03-28T00:34:24.789 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x556648b265d6] 2024-03-28T00:34:24.789 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x556648b26da1] 2024-03-28T00:34:24.789 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: 6: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x556648b5b09c] 2024-03-28T00:34:24.789 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x556648b79233] 2024-03-28T00:34:24.789 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x556648b7b197] 2024-03-28T00:34:24.789 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x556648a7a11b] 2024-03-28T00:34:24.789 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x556648a7a6bc] 2024-03-28T00:34:24.789 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x556648a641ff] 2024-03-28T00:34:24.789 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7fee71a11488] 2024-03-28T00:34:24.789 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: 13: (DispatchQueue::entry()+0x51f) [0x7fee71a0e60f] 2024-03-28T00:34:24.789 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fee71ae1e81] 2024-03-28T00:34:24.790 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fee704fc1ca] 2024-03-28T00:34:24.790 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: 16: clone() 2024-03-28T00:34:24.790 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: 2024-03-28T00:34:24.790 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: *** Caught signal (Aborted) ** 2024-03-28T00:34:24.790 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: in thread 7fee65e00700 thread_name:ms_dispatch 2024-03-28T00:34:24.790 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:34:24.790 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fee70506d20] 2024-03-28T00:34:24.790 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x556a4585a223] 2024-03-28T00:34:24.790 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x556a456d45d6] 2024-03-28T00:34:24.790 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x556a456d4da1] 2024-03-28T00:34:24.791 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: 6: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x556a4570909c] 2024-03-28T00:34:24.791 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x556a45727233] 2024-03-28T00:34:24.791 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x556a45729197] 2024-03-28T00:34:24.791 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x556a4562811b] 2024-03-28T00:34:24.791 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x556a456286bc] 2024-03-28T00:34:24.791 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x556a456121ff] 2024-03-28T00:34:24.791 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f08e6efa488] 2024-03-28T00:34:24.791 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: 13: (DispatchQueue::entry()+0x51f) [0x7f08e6ef760f] 2024-03-28T00:34:24.791 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f08e6fcae81] 2024-03-28T00:34:24.791 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f08e59e51ca] 2024-03-28T00:34:24.791 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: 16: clone() 2024-03-28T00:34:24.791 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: 2024-03-28T00:34:24.792 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: 0> 2024-03-28T00:34:24.451+0000 7f08db400700 -1 *** Caught signal (Aborted) ** 2024-03-28T00:34:24.792 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: in thread 7f08db400700 thread_name:ms_dispatch 2024-03-28T00:34:24.792 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: 2024-03-28T00:34:24.792 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:34:24.792 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f08e59efd20] 2024-03-28T00:34:24.792 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: 2: gsignal() 2024-03-28T00:34:24.792 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: 3: abort() 2024-03-28T00:34:24.792 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f08e6c684fb] 2024-03-28T00:34:24.792 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f08e6c68667] 2024-03-28T00:34:24.792 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x556a4585a223] 2024-03-28T00:34:24.792 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x556a456d45d6] 2024-03-28T00:34:24.792 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: 2: gsignal() 2024-03-28T00:34:24.793 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: 3: abort() 2024-03-28T00:34:24.793 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fee7177f4fb] 2024-03-28T00:34:24.793 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7fee7177f667] 2024-03-28T00:34:24.793 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x556648cac223] 2024-03-28T00:34:24.793 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x556648b265d6] 2024-03-28T00:34:24.793 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x556648b26da1] 2024-03-28T00:34:24.793 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: 9: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x556648b5b09c] 2024-03-28T00:34:24.793 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x556648b79233] 2024-03-28T00:34:24.793 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x556648b7b197] 2024-03-28T00:34:24.793 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x556648a7a11b] 2024-03-28T00:34:24.793 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x556648a7a6bc] 2024-03-28T00:34:24.793 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x556648a641ff] 2024-03-28T00:34:24.793 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7fee71a11488] 2024-03-28T00:34:24.794 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: 16: (DispatchQueue::entry()+0x51f) [0x7fee71a0e60f] 2024-03-28T00:34:24.794 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fee71ae1e81] 2024-03-28T00:34:24.794 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7fee704fc1ca] 2024-03-28T00:34:24.794 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: 19: clone() 2024-03-28T00:34:24.794 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: 2024-03-28T00:34:24.430+0000 7fee65e00700 -1 *** Caught signal (Aborted) ** 2024-03-28T00:34:24.794 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: in thread 7fee65e00700 thread_name:ms_dispatch 2024-03-28T00:34:24.794 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: 2024-03-28T00:34:24.794 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:34:24.794 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fee70506d20] 2024-03-28T00:34:24.794 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: 2: gsignal() 2024-03-28T00:34:24.794 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: 3: abort() 2024-03-28T00:34:24.794 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fee7177f4fb] 2024-03-28T00:34:24.794 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7fee7177f667] 2024-03-28T00:34:24.795 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x556a456d4da1] 2024-03-28T00:34:24.795 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: 9: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x556a4570909c] 2024-03-28T00:34:24.795 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x556a45727233] 2024-03-28T00:34:24.795 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x556a45729197] 2024-03-28T00:34:24.795 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x556a4562811b] 2024-03-28T00:34:24.795 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x556a456286bc] 2024-03-28T00:34:24.795 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x556a456121ff] 2024-03-28T00:34:24.795 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f08e6efa488] 2024-03-28T00:34:24.795 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: 16: (DispatchQueue::entry()+0x51f) [0x7f08e6ef760f] 2024-03-28T00:34:24.795 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f08e6fcae81] 2024-03-28T00:34:24.796 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f08e59e51ca] 2024-03-28T00:34:24.796 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: 19: clone() 2024-03-28T00:34:24.796 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T00:34:24.796 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: 2024-03-28T00:34:24.796 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: -1> 2024-03-28T00:34:24.438+0000 7f08db400700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7f08db400700 time 2024-03-28T00:34:24.438113+0000 2024-03-28T00:34:24.796 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T00:34:24.796 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: 2024-03-28T00:34:24.796 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:34:24.796 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f08e6c684a1] 2024-03-28T00:34:24.796 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f08e6c68667] 2024-03-28T00:34:24.796 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x556a4585a223] 2024-03-28T00:34:24.796 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x556a456d45d6] 2024-03-28T00:34:24.796 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x556a456d4da1] 2024-03-28T00:34:24.797 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x556648cac223] 2024-03-28T00:34:24.797 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x556648b265d6] 2024-03-28T00:34:24.797 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x556648b26da1] 2024-03-28T00:34:24.797 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: 9: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x556648b5b09c] 2024-03-28T00:34:24.797 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x556648b79233] 2024-03-28T00:34:24.797 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x556648b7b197] 2024-03-28T00:34:24.797 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x556648a7a11b] 2024-03-28T00:34:24.797 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x556648a7a6bc] 2024-03-28T00:34:24.797 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x556648a641ff] 2024-03-28T00:34:24.797 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7fee71a11488] 2024-03-28T00:34:24.797 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: 16: (DispatchQueue::entry()+0x51f) [0x7fee71a0e60f] 2024-03-28T00:34:24.797 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fee71ae1e81] 2024-03-28T00:34:24.798 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7fee704fc1ca] 2024-03-28T00:34:24.798 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: 19: clone() 2024-03-28T00:34:24.798 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T00:34:24.798 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: 2024-03-28T00:34:24.798 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: -1> 2024-03-28T00:34:24.428+0000 7fee65e00700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7fee65e00700 time 2024-03-28T00:34:24.426888+0000 2024-03-28T00:34:24.798 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T00:34:24.798 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: 2024-03-28T00:34:24.798 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:34:24.798 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fee7177f4a1] 2024-03-28T00:34:24.798 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7fee7177f667] 2024-03-28T00:34:24.799 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: 6: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x556a4570909c] 2024-03-28T00:34:24.799 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x556a45727233] 2024-03-28T00:34:24.799 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x556a45729197] 2024-03-28T00:34:24.799 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x556a4562811b] 2024-03-28T00:34:24.799 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x556a456286bc] 2024-03-28T00:34:24.799 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x556a456121ff] 2024-03-28T00:34:24.799 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f08e6efa488] 2024-03-28T00:34:24.799 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: 13: (DispatchQueue::entry()+0x51f) [0x7f08e6ef760f] 2024-03-28T00:34:24.799 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f08e6fcae81] 2024-03-28T00:34:24.799 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f08e59e51ca] 2024-03-28T00:34:24.799 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: 16: clone() 2024-03-28T00:34:24.799 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: 2024-03-28T00:34:24.800 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: 0> 2024-03-28T00:34:24.451+0000 7f08db400700 -1 *** Caught signal (Aborted) ** 2024-03-28T00:34:24.800 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: in thread 7f08db400700 thread_name:ms_dispatch 2024-03-28T00:34:24.800 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: 2024-03-28T00:34:24.800 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:34:24.800 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f08e59efd20] 2024-03-28T00:34:24.800 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: 2: gsignal() 2024-03-28T00:34:24.800 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: 3: abort() 2024-03-28T00:34:24.800 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f08e6c684fb] 2024-03-28T00:34:24.800 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f08e6c68667] 2024-03-28T00:34:24.800 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x556a4585a223] 2024-03-28T00:34:24.800 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x556a456d45d6] 2024-03-28T00:34:24.800 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x556a456d4da1] 2024-03-28T00:34:24.800 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: 9: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x556a4570909c] 2024-03-28T00:34:24.800 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x556a45727233] 2024-03-28T00:34:24.801 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x556a45729197] 2024-03-28T00:34:24.801 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x556a4562811b] 2024-03-28T00:34:24.801 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x556a456286bc] 2024-03-28T00:34:24.801 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x556a456121ff] 2024-03-28T00:34:24.801 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f08e6efa488] 2024-03-28T00:34:24.801 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: 16: (DispatchQueue::entry()+0x51f) [0x7f08e6ef760f] 2024-03-28T00:34:24.801 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f08e6fcae81] 2024-03-28T00:34:24.801 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f08e59e51ca] 2024-03-28T00:34:24.801 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: 19: clone() 2024-03-28T00:34:24.801 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T00:34:24.801 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[69120]: 2024-03-28T00:34:24.802 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x556648cac223] 2024-03-28T00:34:24.802 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x556648b265d6] 2024-03-28T00:34:24.802 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x556648b26da1] 2024-03-28T00:34:24.802 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: 6: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x556648b5b09c] 2024-03-28T00:34:24.802 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x556648b79233] 2024-03-28T00:34:24.802 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x556648b7b197] 2024-03-28T00:34:24.802 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x556648a7a11b] 2024-03-28T00:34:24.803 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x556648a7a6bc] 2024-03-28T00:34:24.803 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x556648a641ff] 2024-03-28T00:34:24.803 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7fee71a11488] 2024-03-28T00:34:24.803 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: 13: (DispatchQueue::entry()+0x51f) [0x7fee71a0e60f] 2024-03-28T00:34:24.803 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fee71ae1e81] 2024-03-28T00:34:24.803 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fee704fc1ca] 2024-03-28T00:34:24.803 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: 16: clone() 2024-03-28T00:34:24.803 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: 2024-03-28T00:34:24.803 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: 0> 2024-03-28T00:34:24.430+0000 7fee65e00700 -1 *** Caught signal (Aborted) ** 2024-03-28T00:34:24.803 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: in thread 7fee65e00700 thread_name:ms_dispatch 2024-03-28T00:34:24.803 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: 2024-03-28T00:34:24.803 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:34:24.803 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fee70506d20] 2024-03-28T00:34:24.803 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: 2: gsignal() 2024-03-28T00:34:24.804 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: 3: abort() 2024-03-28T00:34:24.804 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fee7177f4fb] 2024-03-28T00:34:24.804 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7fee7177f667] 2024-03-28T00:34:24.804 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x556648cac223] 2024-03-28T00:34:24.804 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x556648b265d6] 2024-03-28T00:34:24.804 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x556648b26da1] 2024-03-28T00:34:24.804 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: 9: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x556648b5b09c] 2024-03-28T00:34:24.804 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x556648b79233] 2024-03-28T00:34:24.804 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x556648b7b197] 2024-03-28T00:34:24.804 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x556648a7a11b] 2024-03-28T00:34:24.804 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x556648a7a6bc] 2024-03-28T00:34:24.804 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x556648a641ff] 2024-03-28T00:34:24.804 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7fee71a11488] 2024-03-28T00:34:24.804 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: 16: (DispatchQueue::entry()+0x51f) [0x7fee71a0e60f] 2024-03-28T00:34:24.805 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fee71ae1e81] 2024-03-28T00:34:24.805 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7fee704fc1ca] 2024-03-28T00:34:24.805 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: 19: clone() 2024-03-28T00:34:24.805 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T00:34:24.805 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: 2024-03-28T00:34:24.805 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: -1> 2024-03-28T00:34:24.428+0000 7fee65e00700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7fee65e00700 time 2024-03-28T00:34:24.426888+0000 2024-03-28T00:34:24.805 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T00:34:24.805 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: 2024-03-28T00:34:24.805 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:34:24.805 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fee7177f4a1] 2024-03-28T00:34:24.805 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7fee7177f667] 2024-03-28T00:34:24.805 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x556648cac223] 2024-03-28T00:34:24.805 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x556648b265d6] 2024-03-28T00:34:24.806 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x556648b26da1] 2024-03-28T00:34:24.806 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: 6: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x556648b5b09c] 2024-03-28T00:34:24.806 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x556648b79233] 2024-03-28T00:34:24.806 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x556648b7b197] 2024-03-28T00:34:24.806 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x556648a7a11b] 2024-03-28T00:34:24.806 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x556648a7a6bc] 2024-03-28T00:34:24.806 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x556648a641ff] 2024-03-28T00:34:24.806 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7fee71a11488] 2024-03-28T00:34:24.806 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: 13: (DispatchQueue::entry()+0x51f) [0x7fee71a0e60f] 2024-03-28T00:34:24.806 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fee71ae1e81] 2024-03-28T00:34:24.806 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fee704fc1ca] 2024-03-28T00:34:24.806 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: 16: clone() 2024-03-28T00:34:24.806 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: 2024-03-28T00:34:24.807 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: 0> 2024-03-28T00:34:24.430+0000 7fee65e00700 -1 *** Caught signal (Aborted) ** 2024-03-28T00:34:24.807 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: in thread 7fee65e00700 thread_name:ms_dispatch 2024-03-28T00:34:24.807 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: 2024-03-28T00:34:24.807 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:34:24.807 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fee70506d20] 2024-03-28T00:34:24.807 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: 2: gsignal() 2024-03-28T00:34:24.807 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: 3: abort() 2024-03-28T00:34:24.807 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fee7177f4fb] 2024-03-28T00:34:24.807 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7fee7177f667] 2024-03-28T00:34:24.807 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x556648cac223] 2024-03-28T00:34:24.807 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x556648b265d6] 2024-03-28T00:34:24.807 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x556648b26da1] 2024-03-28T00:34:24.807 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: 9: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x556648b5b09c] 2024-03-28T00:34:24.807 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x556648b79233] 2024-03-28T00:34:24.808 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x556648b7b197] 2024-03-28T00:34:24.808 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x556648a7a11b] 2024-03-28T00:34:24.808 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x556648a7a6bc] 2024-03-28T00:34:24.808 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x556648a641ff] 2024-03-28T00:34:24.808 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7fee71a11488] 2024-03-28T00:34:24.808 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: 16: (DispatchQueue::entry()+0x51f) [0x7fee71a0e60f] 2024-03-28T00:34:24.808 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fee71ae1e81] 2024-03-28T00:34:24.808 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7fee704fc1ca] 2024-03-28T00:34:24.808 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: 19: clone() 2024-03-28T00:34:24.808 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T00:34:24.808 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[116280]: 2024-03-28T00:34:24.808 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:24 smithi119 podman[116676]: 2024-03-28 00:34:24.567726697 +0000 UTC m=+0.036825321 container died 2d8b060c1281c86f4eb7f5e59ad8b16387bfb81f9ec23dccd907c0e6a2620b73 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, ceph=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-03-28T00:34:25.029 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:25 smithi119 podman[116679]: 2024-03-28 00:34:25.002959871 +0000 UTC m=+0.466469301 container died b6a9b8c07b0db4f6ddf2798322d3605dad93cedc394640dbb4bc3b63a167cec7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-03-28T00:34:25.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:34:24 smithi165 ceph-mon[36302]: pgmap v262: 97 pgs: 97 active+clean; 262 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 0 B/s wr, 9 op/s 2024-03-28T00:34:25.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:34:24 smithi132 ceph-mon[35941]: pgmap v262: 97 pgs: 97 active+clean; 262 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 0 B/s wr, 9 op/s 2024-03-28T00:34:26.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:34:25 smithi165 ceph-mon[36302]: pgmap v263: 97 pgs: 97 active+clean; 267 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 920 KiB/s rd, 669 KiB/s wr, 56 op/s 2024-03-28T00:34:26.156 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:25 smithi119 podman[116676]: 2024-03-28 00:34:25.739402526 +0000 UTC m=+1.208501207 container remove 2d8b060c1281c86f4eb7f5e59ad8b16387bfb81f9ec23dccd907c0e6a2620b73 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-03-28T00:34:26.157 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:25 smithi119 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.g.service: Main process exited, code=exited, status=134/n/a 2024-03-28T00:34:26.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:34:25 smithi119 ceph-mon[31518]: pgmap v263: 97 pgs: 97 active+clean; 267 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 920 KiB/s rd, 669 KiB/s wr, 56 op/s 2024-03-28T00:34:26.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:34:25 smithi132 ceph-mon[35941]: pgmap v263: 97 pgs: 97 active+clean; 267 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 920 KiB/s rd, 669 KiB/s wr, 56 op/s 2024-03-28T00:34:26.626 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:26 smithi119 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.g.service: Failed with result 'exit-code'. 2024-03-28T00:34:26.626 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:26 smithi119 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.g.service: Consumed 3.348s CPU time. 2024-03-28T00:34:26.626 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:26 smithi119 podman[116679]: 2024-03-28 00:34:26.192153888 +0000 UTC m=+1.655663355 container remove b6a9b8c07b0db4f6ddf2798322d3605dad93cedc394640dbb4bc3b63a167cec7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-28T00:34:26.626 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:26 smithi119 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.d.service: Main process exited, code=exited, status=134/n/a 2024-03-28T00:34:26.907 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:34:26 smithi119 ceph-mon[31518]: osdmap e113: 12 total, 12 up, 12 in 2024-03-28T00:34:26.907 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:26 smithi119 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.d.service: Failed with result 'exit-code'. 2024-03-28T00:34:26.907 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:26 smithi119 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.d.service: Consumed 3.714s CPU time. 2024-03-28T00:34:27.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:34:26 smithi165 ceph-mon[36302]: osdmap e113: 12 total, 12 up, 12 in 2024-03-28T00:34:27.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:34:26 smithi132 ceph-mon[35941]: osdmap e113: 12 total, 12 up, 12 in 2024-03-28T00:34:28.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:34:27 smithi165 ceph-mon[36302]: pgmap v265: 97 pgs: 97 active+clean; 267 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 802 KiB/s rd, 583 KiB/s wr, 48 op/s 2024-03-28T00:34:28.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:34:27 smithi119 ceph-mon[31518]: pgmap v265: 97 pgs: 97 active+clean; 267 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 802 KiB/s rd, 583 KiB/s wr, 48 op/s 2024-03-28T00:34:28.206 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:34:27 smithi132 ceph-mon[35941]: pgmap v265: 97 pgs: 97 active+clean; 267 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 802 KiB/s rd, 583 KiB/s wr, 48 op/s 2024-03-28T00:34:30.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:34:30 smithi165 ceph-mon[36302]: pgmap v266: 97 pgs: 97 active+clean; 269 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 801 KiB/s rd, 809 KiB/s wr, 70 op/s 2024-03-28T00:34:30.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:34:30 smithi165 ceph-mon[36302]: osdmap e114: 12 total, 12 up, 12 in 2024-03-28T00:34:30.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:34:30 smithi119 ceph-mon[31518]: pgmap v266: 97 pgs: 97 active+clean; 269 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 801 KiB/s rd, 809 KiB/s wr, 70 op/s 2024-03-28T00:34:30.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:34:30 smithi119 ceph-mon[31518]: osdmap e114: 12 total, 12 up, 12 in 2024-03-28T00:34:30.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:34:30 smithi132 ceph-mon[35941]: pgmap v266: 97 pgs: 97 active+clean; 269 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 801 KiB/s rd, 809 KiB/s wr, 70 op/s 2024-03-28T00:34:30.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:34:30 smithi132 ceph-mon[35941]: osdmap e114: 12 total, 12 up, 12 in 2024-03-28T00:34:32.506 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:34:32 smithi132 ceph-mon[35941]: pgmap v268: 97 pgs: 97 active+clean; 269 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1002 KiB/s rd, 1012 KiB/s wr, 88 op/s 2024-03-28T00:34:32.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:34:32 smithi165 ceph-mon[36302]: pgmap v268: 97 pgs: 97 active+clean; 269 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1002 KiB/s rd, 1012 KiB/s wr, 88 op/s 2024-03-28T00:34:32.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:34:32 smithi119 ceph-mon[31518]: pgmap v268: 97 pgs: 97 active+clean; 269 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1002 KiB/s rd, 1012 KiB/s wr, 88 op/s 2024-03-28T00:34:34.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:34:34 smithi165 ceph-mon[36302]: pgmap v269: 97 pgs: 97 active+clean; 269 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 304 KiB/s wr, 38 op/s 2024-03-28T00:34:34.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:34:34 smithi119 ceph-mon[31518]: pgmap v269: 97 pgs: 97 active+clean; 269 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 304 KiB/s wr, 38 op/s 2024-03-28T00:34:34.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:34:34 smithi132 ceph-mon[35941]: pgmap v269: 97 pgs: 97 active+clean; 269 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 304 KiB/s wr, 38 op/s 2024-03-28T00:34:36.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:34:36 smithi165 ceph-mon[36302]: pgmap v270: 97 pgs: 97 active+clean; 269 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 258 KiB/s wr, 37 op/s 2024-03-28T00:34:36.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:34:36 smithi119 ceph-mon[31518]: pgmap v270: 97 pgs: 97 active+clean; 269 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 258 KiB/s wr, 37 op/s 2024-03-28T00:34:36.657 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:36 smithi119 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.g.service: Scheduled restart job, restart counter is at 2. 2024-03-28T00:34:36.657 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:36 smithi119 systemd[1]: Stopped Ceph mds.g for 10f73b92-ec98-11ee-95d0-87774f69a715. 2024-03-28T00:34:36.657 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:36 smithi119 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.g.service: Consumed 3.348s CPU time. 2024-03-28T00:34:36.657 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:36 smithi119 systemd[1]: Starting Ceph mds.g for 10f73b92-ec98-11ee-95d0-87774f69a715... 2024-03-28T00:34:36.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:34:36 smithi132 ceph-mon[35941]: pgmap v270: 97 pgs: 97 active+clean; 269 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 258 KiB/s wr, 37 op/s 2024-03-28T00:34:37.012 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:36 smithi119 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.d.service: Scheduled restart job, restart counter is at 1. 2024-03-28T00:34:37.012 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:36 smithi119 systemd[1]: Stopped Ceph mds.d for 10f73b92-ec98-11ee-95d0-87774f69a715. 2024-03-28T00:34:37.012 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:36 smithi119 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.d.service: Consumed 3.714s CPU time. 2024-03-28T00:34:37.012 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:36 smithi119 systemd[1]: Starting Ceph mds.d for 10f73b92-ec98-11ee-95d0-87774f69a715... 2024-03-28T00:34:37.013 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:36 smithi119 podman[117038]: 2024-03-28 00:34:36.710340627 +0000 UTC m=+0.034350469 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-28T00:34:37.013 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:36 smithi119 podman[117038]: 2024-03-28 00:34:36.851817051 +0000 UTC m=+0.175826927 container create 4cbf4f0e0a822f19d23e503d4730b5011323549bd088e94c58392ee23d7f4b3b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-03-28T00:34:37.407 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:37 smithi119 podman[117038]: 2024-03-28 00:34:37.122430088 +0000 UTC m=+0.446440014 container init 4cbf4f0e0a822f19d23e503d4730b5011323549bd088e94c58392ee23d7f4b3b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-03-28T00:34:37.407 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:37 smithi119 podman[117038]: 2024-03-28 00:34:37.132960295 +0000 UTC m=+0.456970144 container start 4cbf4f0e0a822f19d23e503d4730b5011323549bd088e94c58392ee23d7f4b3b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True) 2024-03-28T00:34:37.407 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:37 smithi119 ceph-mds[117094]: -- 172.21.15.119:0/1871080725 <== mon.1 v2:172.21.15.165:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55899bfeba40 con 0x55899bfbe800 2024-03-28T00:34:37.407 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: starting mds.g at 2024-03-28T00:34:37.407 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:37 smithi119 bash[117038]: 4cbf4f0e0a822f19d23e503d4730b5011323549bd088e94c58392ee23d7f4b3b 2024-03-28T00:34:37.407 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:37 smithi119 systemd[1]: Started Ceph mds.g for 10f73b92-ec98-11ee-95d0-87774f69a715. 2024-03-28T00:34:37.407 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:34:37 smithi119 ceph-mon[31518]: daemon mds.g restarted 2024-03-28T00:34:37.407 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:34:37 smithi119 ceph-mon[31518]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-28T00:34:37.408 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:34:37 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-28T00:34:37.408 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:34:37 smithi119 ceph-mon[31518]: osdmap e115: 12 total, 12 up, 12 in 2024-03-28T00:34:37.408 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:34:37 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.119:6834/3816731508,v1:172.21.15.119:6835/3816731508] up:boot 2024-03-28T00:34:37.408 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:34:37 smithi119 ceph-mon[31518]: Replacing daemon mds.d as rank 1 with standby daemon mds.j 2024-03-28T00:34:37.408 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:34:37 smithi119 ceph-mon[31518]: MDS daemon mds.d is removed because it is dead or otherwise unavailable. 2024-03-28T00:34:37.408 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:34:37 smithi119 ceph-mon[31518]: Standby daemon mds.g assigned to filesystem cephfs as rank 3 2024-03-28T00:34:37.408 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:34:37 smithi119 ceph-mon[31518]: fsmap cephfs:4/5 {0=b=up:active,1=d=up:active,2=h=up:active,4=f=up:active} 4 up:standby-replay 1 up:standby, 1 failed 2024-03-28T00:34:37.408 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:34:37 smithi119 ceph-mon[31518]: osdmap e116: 12 total, 12 up, 12 in 2024-03-28T00:34:37.408 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:34:37 smithi119 ceph-mon[31518]: fsmap cephfs:5/5 3 up:standby-replay 2 up:replay 3 up:active 2024-03-28T00:34:37.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:34:37 smithi165 ceph-mon[36302]: daemon mds.g restarted 2024-03-28T00:34:37.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:34:37 smithi165 ceph-mon[36302]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-28T00:34:37.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:34:37 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-28T00:34:37.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:34:37 smithi165 ceph-mon[36302]: osdmap e115: 12 total, 12 up, 12 in 2024-03-28T00:34:37.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:34:37 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.119:6834/3816731508,v1:172.21.15.119:6835/3816731508] up:boot 2024-03-28T00:34:37.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:34:37 smithi165 ceph-mon[36302]: Replacing daemon mds.d as rank 1 with standby daemon mds.j 2024-03-28T00:34:37.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:34:37 smithi165 ceph-mon[36302]: MDS daemon mds.d is removed because it is dead or otherwise unavailable. 2024-03-28T00:34:37.635 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:34:37 smithi165 ceph-mon[36302]: Standby daemon mds.g assigned to filesystem cephfs as rank 3 2024-03-28T00:34:37.635 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:34:37 smithi165 ceph-mon[36302]: fsmap cephfs:4/5 {0=b=up:active,1=d=up:active,2=h=up:active,4=f=up:active} 4 up:standby-replay 1 up:standby, 1 failed 2024-03-28T00:34:37.635 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:34:37 smithi165 ceph-mon[36302]: osdmap e116: 12 total, 12 up, 12 in 2024-03-28T00:34:37.635 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:34:37 smithi165 ceph-mon[36302]: fsmap cephfs:5/5 3 up:standby-replay 2 up:replay 3 up:active 2024-03-28T00:34:37.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:34:37 smithi132 ceph-mon[35941]: daemon mds.g restarted 2024-03-28T00:34:37.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:34:37 smithi132 ceph-mon[35941]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-28T00:34:37.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:34:37 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-28T00:34:37.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:34:37 smithi132 ceph-mon[35941]: osdmap e115: 12 total, 12 up, 12 in 2024-03-28T00:34:37.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:34:37 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.119:6834/3816731508,v1:172.21.15.119:6835/3816731508] up:boot 2024-03-28T00:34:37.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:34:37 smithi132 ceph-mon[35941]: Replacing daemon mds.d as rank 1 with standby daemon mds.j 2024-03-28T00:34:37.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:34:37 smithi132 ceph-mon[35941]: MDS daemon mds.d is removed because it is dead or otherwise unavailable. 2024-03-28T00:34:37.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:34:37 smithi132 ceph-mon[35941]: Standby daemon mds.g assigned to filesystem cephfs as rank 3 2024-03-28T00:34:37.705 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:34:37 smithi132 ceph-mon[35941]: fsmap cephfs:4/5 {0=b=up:active,1=d=up:active,2=h=up:active,4=f=up:active} 4 up:standby-replay 1 up:standby, 1 failed 2024-03-28T00:34:37.705 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:34:37 smithi132 ceph-mon[35941]: osdmap e116: 12 total, 12 up, 12 in 2024-03-28T00:34:37.705 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:34:37 smithi132 ceph-mon[35941]: fsmap cephfs:5/5 3 up:standby-replay 2 up:replay 3 up:active 2024-03-28T00:34:37.893 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:37 smithi119 podman[117138]: 2024-03-28 00:34:37.473705296 +0000 UTC m=+0.033742731 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-28T00:34:37.893 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:37 smithi119 podman[117138]: 2024-03-28 00:34:37.589778068 +0000 UTC m=+0.149815526 container create 50102c2aedd4271caac8481669f65eb8864480e98cc491d92f71fce33bdef1f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-03-28T00:34:38.157 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:37 smithi119 podman[117138]: 2024-03-28 00:34:37.88576662 +0000 UTC m=+0.445804124 container init 50102c2aedd4271caac8481669f65eb8864480e98cc491d92f71fce33bdef1f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0) 2024-03-28T00:34:38.158 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:37 smithi119 podman[117138]: 2024-03-28 00:34:37.895082986 +0000 UTC m=+0.455120478 container start 50102c2aedd4271caac8481669f65eb8864480e98cc491d92f71fce33bdef1f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-03-28T00:34:38.158 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:37 smithi119 ceph-mds[117156]: -- 172.21.15.119:0/4015771469 <== mon.1 v2:172.21.15.165:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55f86aa0da40 con 0x55f86a9e0800 2024-03-28T00:34:38.158 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:37 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: starting mds.d at 2024-03-28T00:34:38.158 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:37 smithi119 bash[117138]: 50102c2aedd4271caac8481669f65eb8864480e98cc491d92f71fce33bdef1f9 2024-03-28T00:34:38.158 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:34:37 smithi119 systemd[1]: Started Ceph mds.d for 10f73b92-ec98-11ee-95d0-87774f69a715. 2024-03-28T00:34:38.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:34:38 smithi165 ceph-mon[36302]: pgmap v272: 97 pgs: 97 active+clean; 269 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 6 op/s 2024-03-28T00:34:38.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:34:38 smithi119 ceph-mon[31518]: pgmap v272: 97 pgs: 97 active+clean; 269 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 6 op/s 2024-03-28T00:34:38.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:34:38 smithi132 ceph-mon[35941]: pgmap v272: 97 pgs: 97 active+clean; 269 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 6 op/s 2024-03-28T00:34:39.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:34:39 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-28T00:34:39.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:34:39 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.119:6834/3816731508,v1:172.21.15.119:6835/3816731508] up:resolve 2024-03-28T00:34:39.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:34:39 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.119:6838/3947365264,v1:172.21.15.119:6839/3947365264] up:resolve 2024-03-28T00:34:39.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:34:39 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.119:6836/482178170,v1:172.21.15.119:6837/482178170] up:boot 2024-03-28T00:34:39.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:34:39 smithi165 ceph-mon[36302]: fsmap cephfs:5/5 3 up:standby-replay 1 up:standby 2 up:resolve 3 up:active 2024-03-28T00:34:39.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:34:39 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.119:6838/3947365264,v1:172.21.15.119:6839/3947365264] up:reconnect 2024-03-28T00:34:39.635 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:34:39 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.119:6834/3816731508,v1:172.21.15.119:6835/3816731508] up:reconnect 2024-03-28T00:34:39.635 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:34:39 smithi165 ceph-mon[36302]: fsmap cephfs:5/5 3 up:standby-replay 1 up:standby 2 up:reconnect 3 up:active 2024-03-28T00:34:39.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:34:39 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-28T00:34:39.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:34:39 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.119:6834/3816731508,v1:172.21.15.119:6835/3816731508] up:resolve 2024-03-28T00:34:39.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:34:39 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.119:6838/3947365264,v1:172.21.15.119:6839/3947365264] up:resolve 2024-03-28T00:34:39.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:34:39 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.119:6836/482178170,v1:172.21.15.119:6837/482178170] up:boot 2024-03-28T00:34:39.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:34:39 smithi119 ceph-mon[31518]: fsmap cephfs:5/5 3 up:standby-replay 1 up:standby 2 up:resolve 3 up:active 2024-03-28T00:34:39.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:34:39 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.119:6838/3947365264,v1:172.21.15.119:6839/3947365264] up:reconnect 2024-03-28T00:34:39.658 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:34:39 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.119:6834/3816731508,v1:172.21.15.119:6835/3816731508] up:reconnect 2024-03-28T00:34:39.658 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:34:39 smithi119 ceph-mon[31518]: fsmap cephfs:5/5 3 up:standby-replay 1 up:standby 2 up:reconnect 3 up:active 2024-03-28T00:34:39.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:34:39 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-28T00:34:39.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:34:39 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.119:6834/3816731508,v1:172.21.15.119:6835/3816731508] up:resolve 2024-03-28T00:34:39.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:34:39 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.119:6838/3947365264,v1:172.21.15.119:6839/3947365264] up:resolve 2024-03-28T00:34:39.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:34:39 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.119:6836/482178170,v1:172.21.15.119:6837/482178170] up:boot 2024-03-28T00:34:39.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:34:39 smithi132 ceph-mon[35941]: fsmap cephfs:5/5 3 up:standby-replay 1 up:standby 2 up:resolve 3 up:active 2024-03-28T00:34:39.705 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:34:39 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.119:6838/3947365264,v1:172.21.15.119:6839/3947365264] up:reconnect 2024-03-28T00:34:39.705 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:34:39 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.119:6834/3816731508,v1:172.21.15.119:6835/3816731508] up:reconnect 2024-03-28T00:34:39.705 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:34:39 smithi132 ceph-mon[35941]: fsmap cephfs:5/5 3 up:standby-replay 1 up:standby 2 up:reconnect 3 up:active 2024-03-28T00:34:40.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:34:40 smithi165 ceph-mon[36302]: pgmap v274: 97 pgs: 97 active+clean; 270 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 152 KiB/s wr, 14 op/s 2024-03-28T00:34:40.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:34:40 smithi165 ceph-mon[36302]: reconnect by client.25355 172.21.15.119:0/3071152482 after 0.00199995 2024-03-28T00:34:40.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:34:40 smithi165 ceph-mon[36302]: reconnect by client.25355 172.21.15.119:0/3071152482 after 0.00299993 2024-03-28T00:34:40.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:34:40 smithi165 ceph-mon[36302]: reconnect by client.25264 v1:172.21.15.119:0/3863511511 after 0.0179996 2024-03-28T00:34:40.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:34:40 smithi165 ceph-mon[36302]: reconnect by client.25264 v1:172.21.15.119:0/3863511511 after 0.0179996 2024-03-28T00:34:40.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:34:40 smithi119 ceph-mon[31518]: pgmap v274: 97 pgs: 97 active+clean; 270 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 152 KiB/s wr, 14 op/s 2024-03-28T00:34:40.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:34:40 smithi119 ceph-mon[31518]: reconnect by client.25355 172.21.15.119:0/3071152482 after 0.00199995 2024-03-28T00:34:40.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:34:40 smithi119 ceph-mon[31518]: reconnect by client.25355 172.21.15.119:0/3071152482 after 0.00299993 2024-03-28T00:34:40.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:34:40 smithi119 ceph-mon[31518]: reconnect by client.25264 v1:172.21.15.119:0/3863511511 after 0.0179996 2024-03-28T00:34:40.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:34:40 smithi119 ceph-mon[31518]: reconnect by client.25264 v1:172.21.15.119:0/3863511511 after 0.0179996 2024-03-28T00:34:40.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:34:40 smithi132 ceph-mon[35941]: pgmap v274: 97 pgs: 97 active+clean; 270 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 152 KiB/s wr, 14 op/s 2024-03-28T00:34:40.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:34:40 smithi132 ceph-mon[35941]: reconnect by client.25355 172.21.15.119:0/3071152482 after 0.00199995 2024-03-28T00:34:40.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:34:40 smithi132 ceph-mon[35941]: reconnect by client.25355 172.21.15.119:0/3071152482 after 0.00299993 2024-03-28T00:34:40.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:34:40 smithi132 ceph-mon[35941]: reconnect by client.25264 v1:172.21.15.119:0/3863511511 after 0.0179996 2024-03-28T00:34:40.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:34:40 smithi132 ceph-mon[35941]: reconnect by client.25264 v1:172.21.15.119:0/3863511511 after 0.0179996 2024-03-28T00:34:41.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:34:41 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.119:6834/3816731508,v1:172.21.15.119:6835/3816731508] up:rejoin 2024-03-28T00:34:41.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:34:41 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.119:6838/3947365264,v1:172.21.15.119:6839/3947365264] up:rejoin 2024-03-28T00:34:41.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:34:41 smithi165 ceph-mon[36302]: fsmap cephfs:5/5 3 up:standby-replay 1 up:standby 2 up:rejoin 3 up:active 2024-03-28T00:34:41.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:34:41 smithi165 ceph-mon[36302]: daemon mds.g is now active in filesystem cephfs as rank 3 2024-03-28T00:34:41.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:34:41 smithi165 ceph-mon[36302]: daemon mds.j is now active in filesystem cephfs as rank 1 2024-03-28T00:34:41.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.119:6834/3816731508,v1:172.21.15.119:6835/3816731508] up:rejoin 2024-03-28T00:34:41.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.119:6838/3947365264,v1:172.21.15.119:6839/3947365264] up:rejoin 2024-03-28T00:34:41.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-mon[31518]: fsmap cephfs:5/5 3 up:standby-replay 1 up:standby 2 up:rejoin 3 up:active 2024-03-28T00:34:41.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-mon[31518]: daemon mds.g is now active in filesystem cephfs as rank 3 2024-03-28T00:34:41.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-mon[31518]: daemon mds.j is now active in filesystem cephfs as rank 1 2024-03-28T00:34:41.657 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7f655c000700 time 2024-03-28T00:34:41.337299+0000 2024-03-28T00:34:41.657 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T00:34:41.657 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:34:41.657 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f65679964a1] 2024-03-28T00:34:41.657 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f6567996667] 2024-03-28T00:34:41.657 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x5589995bb223] 2024-03-28T00:34:41.658 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x5589994355d6] 2024-03-28T00:34:41.658 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x558999435da1] 2024-03-28T00:34:41.658 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: 6: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x55899946a09c] 2024-03-28T00:34:41.658 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x558999488233] 2024-03-28T00:34:41.658 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55899948a197] 2024-03-28T00:34:41.658 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55899938911b] 2024-03-28T00:34:41.658 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5589993896bc] 2024-03-28T00:34:41.658 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x5589993731ff] 2024-03-28T00:34:41.658 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f6567c28488] 2024-03-28T00:34:41.658 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: 13: (DispatchQueue::entry()+0x51f) [0x7f6567c2560f] 2024-03-28T00:34:41.658 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f6567cf8e81] 2024-03-28T00:34:41.658 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f65667131ca] 2024-03-28T00:34:41.658 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: 16: clone() 2024-03-28T00:34:41.658 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: *** Caught signal (Aborted) ** 2024-03-28T00:34:41.659 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: in thread 7f655c000700 thread_name:ms_dispatch 2024-03-28T00:34:41.659 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7f9adae00700 time 2024-03-28T00:34:41.386154+0000 2024-03-28T00:34:41.659 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T00:34:41.659 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:34:41.659 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f9ae66654a1] 2024-03-28T00:34:41.659 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f9ae6665667] 2024-03-28T00:34:41.659 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55b8b87af223] 2024-03-28T00:34:41.659 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55b8b86295d6] 2024-03-28T00:34:41.659 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55b8b8629da1] 2024-03-28T00:34:41.659 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: 6: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x55b8b865e09c] 2024-03-28T00:34:41.660 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55b8b867c233] 2024-03-28T00:34:41.660 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55b8b867e197] 2024-03-28T00:34:41.660 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55b8b857d11b] 2024-03-28T00:34:41.660 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55b8b857d6bc] 2024-03-28T00:34:41.660 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55b8b85671ff] 2024-03-28T00:34:41.660 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f9ae68f7488] 2024-03-28T00:34:41.660 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: 13: (DispatchQueue::entry()+0x51f) [0x7f9ae68f460f] 2024-03-28T00:34:41.660 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f9ae69c7e81] 2024-03-28T00:34:41.660 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f9ae53e21ca] 2024-03-28T00:34:41.660 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: 16: clone() 2024-03-28T00:34:41.660 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: *** Caught signal (Aborted) ** 2024-03-28T00:34:41.660 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: in thread 7f9adae00700 thread_name:ms_dispatch 2024-03-28T00:34:41.661 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: 2024-03-28T00:34:41.337+0000 7f655c000700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7f655c000700 time 2024-03-28T00:34:41.337299+0000 2024-03-28T00:34:41.661 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T00:34:41.661 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: 2024-03-28T00:34:41.661 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:34:41.661 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f65679964a1] 2024-03-28T00:34:41.661 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f6567996667] 2024-03-28T00:34:41.661 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x5589995bb223] 2024-03-28T00:34:41.661 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x5589994355d6] 2024-03-28T00:34:41.661 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x558999435da1] 2024-03-28T00:34:41.661 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: 6: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x55899946a09c] 2024-03-28T00:34:41.661 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x558999488233] 2024-03-28T00:34:41.661 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55899948a197] 2024-03-28T00:34:41.661 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55899938911b] 2024-03-28T00:34:41.661 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5589993896bc] 2024-03-28T00:34:41.662 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x5589993731ff] 2024-03-28T00:34:41.662 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f6567c28488] 2024-03-28T00:34:41.662 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: 13: (DispatchQueue::entry()+0x51f) [0x7f6567c2560f] 2024-03-28T00:34:41.662 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f6567cf8e81] 2024-03-28T00:34:41.662 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f65667131ca] 2024-03-28T00:34:41.662 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: 16: clone() 2024-03-28T00:34:41.662 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: 2024-03-28T00:34:41.662 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:34:41.662 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f656671dd20] 2024-03-28T00:34:41.662 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: 2024-03-28T00:34:41.386+0000 7f9adae00700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7f9adae00700 time 2024-03-28T00:34:41.386154+0000 2024-03-28T00:34:41.662 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T00:34:41.663 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: 2024-03-28T00:34:41.663 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:34:41.663 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f9ae66654a1] 2024-03-28T00:34:41.663 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f9ae6665667] 2024-03-28T00:34:41.663 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55b8b87af223] 2024-03-28T00:34:41.663 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55b8b86295d6] 2024-03-28T00:34:41.663 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55b8b8629da1] 2024-03-28T00:34:41.663 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: 6: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x55b8b865e09c] 2024-03-28T00:34:41.663 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55b8b867c233] 2024-03-28T00:34:41.663 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55b8b867e197] 2024-03-28T00:34:41.663 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55b8b857d11b] 2024-03-28T00:34:41.663 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55b8b857d6bc] 2024-03-28T00:34:41.663 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55b8b85671ff] 2024-03-28T00:34:41.663 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f9ae68f7488] 2024-03-28T00:34:41.664 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: 13: (DispatchQueue::entry()+0x51f) [0x7f9ae68f460f] 2024-03-28T00:34:41.664 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f9ae69c7e81] 2024-03-28T00:34:41.664 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f9ae53e21ca] 2024-03-28T00:34:41.664 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: 16: clone() 2024-03-28T00:34:41.664 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: 2024-03-28T00:34:41.664 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:34:41.664 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f9ae53ecd20] 2024-03-28T00:34:41.664 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: 2: gsignal() 2024-03-28T00:34:41.664 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: 3: abort() 2024-03-28T00:34:41.664 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f65679964fb] 2024-03-28T00:34:41.664 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f6567996667] 2024-03-28T00:34:41.664 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x5589995bb223] 2024-03-28T00:34:41.664 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x5589994355d6] 2024-03-28T00:34:41.665 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x558999435da1] 2024-03-28T00:34:41.665 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: 9: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x55899946a09c] 2024-03-28T00:34:41.665 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x558999488233] 2024-03-28T00:34:41.665 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55899948a197] 2024-03-28T00:34:41.665 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55899938911b] 2024-03-28T00:34:41.665 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5589993896bc] 2024-03-28T00:34:41.665 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x5589993731ff] 2024-03-28T00:34:41.665 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f6567c28488] 2024-03-28T00:34:41.665 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: 16: (DispatchQueue::entry()+0x51f) [0x7f6567c2560f] 2024-03-28T00:34:41.665 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f6567cf8e81] 2024-03-28T00:34:41.665 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f65667131ca] 2024-03-28T00:34:41.665 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: 19: clone() 2024-03-28T00:34:41.665 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: 2024-03-28T00:34:41.339+0000 7f655c000700 -1 *** Caught signal (Aborted) ** 2024-03-28T00:34:41.665 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: in thread 7f655c000700 thread_name:ms_dispatch 2024-03-28T00:34:41.665 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: 2024-03-28T00:34:41.666 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:34:41.666 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f656671dd20] 2024-03-28T00:34:41.666 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: 2: gsignal() 2024-03-28T00:34:41.666 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: 3: abort() 2024-03-28T00:34:41.666 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f65679964fb] 2024-03-28T00:34:41.666 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f6567996667] 2024-03-28T00:34:41.666 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: 2: gsignal() 2024-03-28T00:34:41.666 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: 3: abort() 2024-03-28T00:34:41.667 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f9ae66654fb] 2024-03-28T00:34:41.667 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f9ae6665667] 2024-03-28T00:34:41.667 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55b8b87af223] 2024-03-28T00:34:41.667 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55b8b86295d6] 2024-03-28T00:34:41.667 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55b8b8629da1] 2024-03-28T00:34:41.667 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: 9: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x55b8b865e09c] 2024-03-28T00:34:41.667 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55b8b867c233] 2024-03-28T00:34:41.667 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55b8b867e197] 2024-03-28T00:34:41.667 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55b8b857d11b] 2024-03-28T00:34:41.667 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55b8b857d6bc] 2024-03-28T00:34:41.667 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55b8b85671ff] 2024-03-28T00:34:41.667 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f9ae68f7488] 2024-03-28T00:34:41.667 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: 16: (DispatchQueue::entry()+0x51f) [0x7f9ae68f460f] 2024-03-28T00:34:41.668 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f9ae69c7e81] 2024-03-28T00:34:41.668 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f9ae53e21ca] 2024-03-28T00:34:41.668 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: 19: clone() 2024-03-28T00:34:41.668 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: 2024-03-28T00:34:41.388+0000 7f9adae00700 -1 *** Caught signal (Aborted) ** 2024-03-28T00:34:41.668 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: in thread 7f9adae00700 thread_name:ms_dispatch 2024-03-28T00:34:41.668 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: 2024-03-28T00:34:41.668 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:34:41.668 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f9ae53ecd20] 2024-03-28T00:34:41.668 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: 2: gsignal() 2024-03-28T00:34:41.668 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: 3: abort() 2024-03-28T00:34:41.668 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f9ae66654fb] 2024-03-28T00:34:41.668 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f9ae6665667] 2024-03-28T00:34:41.668 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55b8b87af223] 2024-03-28T00:34:41.668 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55b8b86295d6] 2024-03-28T00:34:41.669 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55b8b8629da1] 2024-03-28T00:34:41.669 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: 9: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x55b8b865e09c] 2024-03-28T00:34:41.669 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55b8b867c233] 2024-03-28T00:34:41.669 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55b8b867e197] 2024-03-28T00:34:41.669 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55b8b857d11b] 2024-03-28T00:34:41.669 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55b8b857d6bc] 2024-03-28T00:34:41.669 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55b8b85671ff] 2024-03-28T00:34:41.669 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f9ae68f7488] 2024-03-28T00:34:41.669 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: 16: (DispatchQueue::entry()+0x51f) [0x7f9ae68f460f] 2024-03-28T00:34:41.669 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f9ae69c7e81] 2024-03-28T00:34:41.669 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f9ae53e21ca] 2024-03-28T00:34:41.669 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: 19: clone() 2024-03-28T00:34:41.669 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T00:34:41.669 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: 2024-03-28T00:34:41.669 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: -1> 2024-03-28T00:34:41.386+0000 7f9adae00700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7f9adae00700 time 2024-03-28T00:34:41.386154+0000 2024-03-28T00:34:41.669 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T00:34:41.670 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: 2024-03-28T00:34:41.670 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:34:41.670 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f9ae66654a1] 2024-03-28T00:34:41.670 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f9ae6665667] 2024-03-28T00:34:41.670 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x5589995bb223] 2024-03-28T00:34:41.670 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x5589994355d6] 2024-03-28T00:34:41.670 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x558999435da1] 2024-03-28T00:34:41.670 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: 9: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x55899946a09c] 2024-03-28T00:34:41.670 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x558999488233] 2024-03-28T00:34:41.670 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55899948a197] 2024-03-28T00:34:41.670 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55899938911b] 2024-03-28T00:34:41.671 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5589993896bc] 2024-03-28T00:34:41.671 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x5589993731ff] 2024-03-28T00:34:41.671 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f6567c28488] 2024-03-28T00:34:41.671 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: 16: (DispatchQueue::entry()+0x51f) [0x7f6567c2560f] 2024-03-28T00:34:41.671 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f6567cf8e81] 2024-03-28T00:34:41.671 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f65667131ca] 2024-03-28T00:34:41.671 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: 19: clone() 2024-03-28T00:34:41.671 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T00:34:41.671 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: 2024-03-28T00:34:41.671 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: -1> 2024-03-28T00:34:41.337+0000 7f655c000700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7f655c000700 time 2024-03-28T00:34:41.337299+0000 2024-03-28T00:34:41.671 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T00:34:41.671 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: 2024-03-28T00:34:41.671 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:34:41.671 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f65679964a1] 2024-03-28T00:34:41.672 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f6567996667] 2024-03-28T00:34:41.672 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55b8b87af223] 2024-03-28T00:34:41.672 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55b8b86295d6] 2024-03-28T00:34:41.672 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55b8b8629da1] 2024-03-28T00:34:41.672 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: 6: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x55b8b865e09c] 2024-03-28T00:34:41.672 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55b8b867c233] 2024-03-28T00:34:41.672 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55b8b867e197] 2024-03-28T00:34:41.672 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55b8b857d11b] 2024-03-28T00:34:41.672 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55b8b857d6bc] 2024-03-28T00:34:41.672 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55b8b85671ff] 2024-03-28T00:34:41.672 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f9ae68f7488] 2024-03-28T00:34:41.673 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: 13: (DispatchQueue::entry()+0x51f) [0x7f9ae68f460f] 2024-03-28T00:34:41.673 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f9ae69c7e81] 2024-03-28T00:34:41.673 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f9ae53e21ca] 2024-03-28T00:34:41.673 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: 16: clone() 2024-03-28T00:34:41.673 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: 2024-03-28T00:34:41.673 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: 0> 2024-03-28T00:34:41.388+0000 7f9adae00700 -1 *** Caught signal (Aborted) ** 2024-03-28T00:34:41.673 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: in thread 7f9adae00700 thread_name:ms_dispatch 2024-03-28T00:34:41.673 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: 2024-03-28T00:34:41.673 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:34:41.673 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f9ae53ecd20] 2024-03-28T00:34:41.673 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: 2: gsignal() 2024-03-28T00:34:41.673 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: 3: abort() 2024-03-28T00:34:41.673 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f9ae66654fb] 2024-03-28T00:34:41.673 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f9ae6665667] 2024-03-28T00:34:41.673 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55b8b87af223] 2024-03-28T00:34:41.673 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55b8b86295d6] 2024-03-28T00:34:41.674 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x5589995bb223] 2024-03-28T00:34:41.674 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x5589994355d6] 2024-03-28T00:34:41.674 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x558999435da1] 2024-03-28T00:34:41.674 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: 6: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x55899946a09c] 2024-03-28T00:34:41.674 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x558999488233] 2024-03-28T00:34:41.674 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55899948a197] 2024-03-28T00:34:41.674 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55899938911b] 2024-03-28T00:34:41.674 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5589993896bc] 2024-03-28T00:34:41.674 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x5589993731ff] 2024-03-28T00:34:41.674 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f6567c28488] 2024-03-28T00:34:41.674 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: 13: (DispatchQueue::entry()+0x51f) [0x7f6567c2560f] 2024-03-28T00:34:41.675 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f6567cf8e81] 2024-03-28T00:34:41.675 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f65667131ca] 2024-03-28T00:34:41.675 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: 16: clone() 2024-03-28T00:34:41.675 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: 2024-03-28T00:34:41.675 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: 0> 2024-03-28T00:34:41.339+0000 7f655c000700 -1 *** Caught signal (Aborted) ** 2024-03-28T00:34:41.675 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: in thread 7f655c000700 thread_name:ms_dispatch 2024-03-28T00:34:41.675 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: 2024-03-28T00:34:41.675 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:34:41.675 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f656671dd20] 2024-03-28T00:34:41.675 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: 2: gsignal() 2024-03-28T00:34:41.675 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: 3: abort() 2024-03-28T00:34:41.675 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f65679964fb] 2024-03-28T00:34:41.675 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f6567996667] 2024-03-28T00:34:41.675 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x5589995bb223] 2024-03-28T00:34:41.675 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x5589994355d6] 2024-03-28T00:34:41.676 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55b8b8629da1] 2024-03-28T00:34:41.676 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: 9: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x55b8b865e09c] 2024-03-28T00:34:41.676 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55b8b867c233] 2024-03-28T00:34:41.676 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55b8b867e197] 2024-03-28T00:34:41.676 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55b8b857d11b] 2024-03-28T00:34:41.676 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55b8b857d6bc] 2024-03-28T00:34:41.676 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55b8b85671ff] 2024-03-28T00:34:41.676 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f9ae68f7488] 2024-03-28T00:34:41.676 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: 16: (DispatchQueue::entry()+0x51f) [0x7f9ae68f460f] 2024-03-28T00:34:41.676 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f9ae69c7e81] 2024-03-28T00:34:41.676 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f9ae53e21ca] 2024-03-28T00:34:41.676 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: 19: clone() 2024-03-28T00:34:41.677 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T00:34:41.677 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: 2024-03-28T00:34:41.677 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: -5> 2024-03-28T00:34:41.386+0000 7f9adae00700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7f9adae00700 time 2024-03-28T00:34:41.386154+0000 2024-03-28T00:34:41.677 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T00:34:41.677 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: 2024-03-28T00:34:41.677 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:34:41.677 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f9ae66654a1] 2024-03-28T00:34:41.677 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f9ae6665667] 2024-03-28T00:34:41.677 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55b8b87af223] 2024-03-28T00:34:41.677 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55b8b86295d6] 2024-03-28T00:34:41.677 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x558999435da1] 2024-03-28T00:34:41.677 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: 9: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x55899946a09c] 2024-03-28T00:34:41.678 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x558999488233] 2024-03-28T00:34:41.678 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55899948a197] 2024-03-28T00:34:41.678 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55899938911b] 2024-03-28T00:34:41.678 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5589993896bc] 2024-03-28T00:34:41.678 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x5589993731ff] 2024-03-28T00:34:41.678 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f6567c28488] 2024-03-28T00:34:41.678 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: 16: (DispatchQueue::entry()+0x51f) [0x7f6567c2560f] 2024-03-28T00:34:41.678 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f6567cf8e81] 2024-03-28T00:34:41.678 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f65667131ca] 2024-03-28T00:34:41.678 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: 19: clone() 2024-03-28T00:34:41.678 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T00:34:41.678 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: 2024-03-28T00:34:41.678 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: -1> 2024-03-28T00:34:41.337+0000 7f655c000700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7f655c000700 time 2024-03-28T00:34:41.337299+0000 2024-03-28T00:34:41.678 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T00:34:41.679 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: 2024-03-28T00:34:41.679 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:34:41.679 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f65679964a1] 2024-03-28T00:34:41.679 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f6567996667] 2024-03-28T00:34:41.679 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x5589995bb223] 2024-03-28T00:34:41.679 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x5589994355d6] 2024-03-28T00:34:41.679 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55b8b8629da1] 2024-03-28T00:34:41.680 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: 6: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x55b8b865e09c] 2024-03-28T00:34:41.680 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55b8b867c233] 2024-03-28T00:34:41.680 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55b8b867e197] 2024-03-28T00:34:41.680 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55b8b857d11b] 2024-03-28T00:34:41.680 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55b8b857d6bc] 2024-03-28T00:34:41.680 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55b8b85671ff] 2024-03-28T00:34:41.680 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f9ae68f7488] 2024-03-28T00:34:41.680 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: 13: (DispatchQueue::entry()+0x51f) [0x7f9ae68f460f] 2024-03-28T00:34:41.680 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f9ae69c7e81] 2024-03-28T00:34:41.680 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f9ae53e21ca] 2024-03-28T00:34:41.680 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: 16: clone() 2024-03-28T00:34:41.680 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: 2024-03-28T00:34:41.680 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: -4> 2024-03-28T00:34:41.388+0000 7f9adae00700 -1 *** Caught signal (Aborted) ** 2024-03-28T00:34:41.680 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: in thread 7f9adae00700 thread_name:ms_dispatch 2024-03-28T00:34:41.681 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: 2024-03-28T00:34:41.681 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:34:41.681 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f9ae53ecd20] 2024-03-28T00:34:41.681 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: 2: gsignal() 2024-03-28T00:34:41.681 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: 3: abort() 2024-03-28T00:34:41.681 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f9ae66654fb] 2024-03-28T00:34:41.681 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f9ae6665667] 2024-03-28T00:34:41.681 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55b8b87af223] 2024-03-28T00:34:41.681 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55b8b86295d6] 2024-03-28T00:34:41.681 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55b8b8629da1] 2024-03-28T00:34:41.681 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: 9: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x55b8b865e09c] 2024-03-28T00:34:41.681 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55b8b867c233] 2024-03-28T00:34:41.681 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55b8b867e197] 2024-03-28T00:34:41.681 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55b8b857d11b] 2024-03-28T00:34:41.681 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55b8b857d6bc] 2024-03-28T00:34:41.681 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55b8b85671ff] 2024-03-28T00:34:41.682 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f9ae68f7488] 2024-03-28T00:34:41.682 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: 16: (DispatchQueue::entry()+0x51f) [0x7f9ae68f460f] 2024-03-28T00:34:41.682 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f9ae69c7e81] 2024-03-28T00:34:41.682 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f9ae53e21ca] 2024-03-28T00:34:41.682 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: 19: clone() 2024-03-28T00:34:41.682 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T00:34:41.682 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j[116377]: 2024-03-28T00:34:41.682 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x558999435da1] 2024-03-28T00:34:41.682 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: 6: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x55899946a09c] 2024-03-28T00:34:41.683 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x558999488233] 2024-03-28T00:34:41.683 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55899948a197] 2024-03-28T00:34:41.683 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55899938911b] 2024-03-28T00:34:41.683 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5589993896bc] 2024-03-28T00:34:41.683 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x5589993731ff] 2024-03-28T00:34:41.683 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f6567c28488] 2024-03-28T00:34:41.683 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: 13: (DispatchQueue::entry()+0x51f) [0x7f6567c2560f] 2024-03-28T00:34:41.683 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f6567cf8e81] 2024-03-28T00:34:41.683 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f65667131ca] 2024-03-28T00:34:41.683 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: 16: clone() 2024-03-28T00:34:41.683 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: 2024-03-28T00:34:41.683 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: 0> 2024-03-28T00:34:41.339+0000 7f655c000700 -1 *** Caught signal (Aborted) ** 2024-03-28T00:34:41.683 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: in thread 7f655c000700 thread_name:ms_dispatch 2024-03-28T00:34:41.683 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: 2024-03-28T00:34:41.683 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:34:41.684 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f656671dd20] 2024-03-28T00:34:41.684 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: 2: gsignal() 2024-03-28T00:34:41.684 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: 3: abort() 2024-03-28T00:34:41.684 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f65679964fb] 2024-03-28T00:34:41.684 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f6567996667] 2024-03-28T00:34:41.684 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x5589995bb223] 2024-03-28T00:34:41.684 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x5589994355d6] 2024-03-28T00:34:41.684 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x558999435da1] 2024-03-28T00:34:41.684 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: 9: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x55899946a09c] 2024-03-28T00:34:41.684 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x558999488233] 2024-03-28T00:34:41.684 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55899948a197] 2024-03-28T00:34:41.684 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55899938911b] 2024-03-28T00:34:41.684 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5589993896bc] 2024-03-28T00:34:41.684 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x5589993731ff] 2024-03-28T00:34:41.684 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f6567c28488] 2024-03-28T00:34:41.684 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: 16: (DispatchQueue::entry()+0x51f) [0x7f6567c2560f] 2024-03-28T00:34:41.685 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f6567cf8e81] 2024-03-28T00:34:41.685 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f65667131ca] 2024-03-28T00:34:41.685 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: 19: clone() 2024-03-28T00:34:41.685 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T00:34:41.685 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[117079]: 2024-03-28T00:34:41.685 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:41 smithi119 podman[117612]: 2024-03-28 00:34:41.478630023 +0000 UTC m=+0.038014827 container died 4cbf4f0e0a822f19d23e503d4730b5011323549bd088e94c58392ee23d7f4b3b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0) 2024-03-28T00:34:41.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:34:41 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.119:6834/3816731508,v1:172.21.15.119:6835/3816731508] up:rejoin 2024-03-28T00:34:41.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:34:41 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.119:6838/3947365264,v1:172.21.15.119:6839/3947365264] up:rejoin 2024-03-28T00:34:41.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:34:41 smithi132 ceph-mon[35941]: fsmap cephfs:5/5 3 up:standby-replay 1 up:standby 2 up:rejoin 3 up:active 2024-03-28T00:34:41.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:34:41 smithi132 ceph-mon[35941]: daemon mds.g is now active in filesystem cephfs as rank 3 2024-03-28T00:34:41.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:34:41 smithi132 ceph-mon[35941]: daemon mds.j is now active in filesystem cephfs as rank 1 2024-03-28T00:34:42.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:34:42 smithi165 ceph-mon[36302]: pgmap v275: 97 pgs: 97 active+clean; 270 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 152 KiB/s wr, 14 op/s 2024-03-28T00:34:42.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:34:42 smithi165 ceph-mon[36302]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-28T00:34:42.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:34:42 smithi165 ceph-mon[36302]: Cluster is now healthy 2024-03-28T00:34:42.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:34:42 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.119:6834/3816731508,v1:172.21.15.119:6835/3816731508] up:active 2024-03-28T00:34:42.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:34:42 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.119:6838/3947365264,v1:172.21.15.119:6839/3947365264] up:active 2024-03-28T00:34:42.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:34:42 smithi165 ceph-mon[36302]: fsmap cephfs:5 3 up:standby-replay 1 up:standby 5 up:active 2024-03-28T00:34:42.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:34:42 smithi165 ceph-mon[36302]: fsmap cephfs:5 4 up:standby-replay 5 up:active 2024-03-28T00:34:42.642 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:42 smithi119 podman[117612]: 2024-03-28 00:34:42.290725156 +0000 UTC m=+0.850109957 container remove 4cbf4f0e0a822f19d23e503d4730b5011323549bd088e94c58392ee23d7f4b3b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-03-28T00:34:42.642 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:42 smithi119 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.g.service: Main process exited, code=exited, status=134/n/a 2024-03-28T00:34:42.643 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:42 smithi119 podman[117632]: 2024-03-28 00:34:42.333744521 +0000 UTC m=+0.861989269 container died 6d699ce8e535466d3bfa776c07efad31cc4e4741c4452684447f1edb995cb013 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, ceph=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-28T00:34:42.643 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:34:42 smithi119 ceph-mon[31518]: pgmap v275: 97 pgs: 97 active+clean; 270 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 152 KiB/s wr, 14 op/s 2024-03-28T00:34:42.643 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:34:42 smithi119 ceph-mon[31518]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-28T00:34:42.643 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:34:42 smithi119 ceph-mon[31518]: Cluster is now healthy 2024-03-28T00:34:42.643 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:34:42 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.119:6834/3816731508,v1:172.21.15.119:6835/3816731508] up:active 2024-03-28T00:34:42.643 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:34:42 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.119:6838/3947365264,v1:172.21.15.119:6839/3947365264] up:active 2024-03-28T00:34:42.643 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:34:42 smithi119 ceph-mon[31518]: fsmap cephfs:5 3 up:standby-replay 1 up:standby 5 up:active 2024-03-28T00:34:42.643 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:34:42 smithi119 ceph-mon[31518]: fsmap cephfs:5 4 up:standby-replay 5 up:active 2024-03-28T00:34:42.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:34:42 smithi132 ceph-mon[35941]: pgmap v275: 97 pgs: 97 active+clean; 270 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 152 KiB/s wr, 14 op/s 2024-03-28T00:34:42.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:34:42 smithi132 ceph-mon[35941]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-28T00:34:42.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:34:42 smithi132 ceph-mon[35941]: Cluster is now healthy 2024-03-28T00:34:42.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:34:42 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.119:6834/3816731508,v1:172.21.15.119:6835/3816731508] up:active 2024-03-28T00:34:42.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:34:42 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.119:6838/3947365264,v1:172.21.15.119:6839/3947365264] up:active 2024-03-28T00:34:42.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:34:42 smithi132 ceph-mon[35941]: fsmap cephfs:5 3 up:standby-replay 1 up:standby 5 up:active 2024-03-28T00:34:42.705 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:34:42 smithi132 ceph-mon[35941]: fsmap cephfs:5 4 up:standby-replay 5 up:active 2024-03-28T00:34:43.319 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:42 smithi119 podman[117632]: 2024-03-28 00:34:42.978784506 +0000 UTC m=+1.507029282 container remove 6d699ce8e535466d3bfa776c07efad31cc4e4741c4452684447f1edb995cb013 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-j, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-03-28T00:34:43.319 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:42 smithi119 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.j.service: Main process exited, code=exited, status=134/n/a 2024-03-28T00:34:43.627 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:43 smithi119 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.g.service: Failed with result 'exit-code'. 2024-03-28T00:34:43.627 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:43 smithi119 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.g.service: Consumed 1.567s CPU time. 2024-03-28T00:34:43.628 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:43 smithi119 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.j.service: Failed with result 'exit-code'. 2024-03-28T00:34:43.628 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:43 smithi119 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.j.service: Consumed 1.515s CPU time. 2024-03-28T00:34:44.407 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:34:44 smithi119 ceph-mon[31518]: pgmap v276: 97 pgs: 97 active+clean; 270 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 390 KiB/s rd, 152 KiB/s wr, 8 op/s 2024-03-28T00:34:44.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:34:44 smithi165 ceph-mon[36302]: pgmap v276: 97 pgs: 97 active+clean; 270 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 390 KiB/s rd, 152 KiB/s wr, 8 op/s 2024-03-28T00:34:44.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:34:44 smithi132 ceph-mon[35941]: pgmap v276: 97 pgs: 97 active+clean; 270 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 390 KiB/s rd, 152 KiB/s wr, 8 op/s 2024-03-28T00:34:46.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:34:46 smithi165 ceph-mon[36302]: pgmap v277: 97 pgs: 97 active+clean; 275 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 781 KiB/s wr, 103 op/s 2024-03-28T00:34:46.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:34:46 smithi119 ceph-mon[31518]: pgmap v277: 97 pgs: 97 active+clean; 275 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 781 KiB/s wr, 103 op/s 2024-03-28T00:34:46.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:34:46 smithi132 ceph-mon[35941]: pgmap v277: 97 pgs: 97 active+clean; 275 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 781 KiB/s wr, 103 op/s 2024-03-28T00:34:48.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:34:48 smithi165 ceph-mon[36302]: pgmap v278: 97 pgs: 97 active+clean; 275 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 834 KiB/s rd, 625 KiB/s wr, 82 op/s 2024-03-28T00:34:48.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:34:48 smithi119 ceph-mon[31518]: pgmap v278: 97 pgs: 97 active+clean; 275 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 834 KiB/s rd, 625 KiB/s wr, 82 op/s 2024-03-28T00:34:48.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:34:48 smithi132 ceph-mon[35941]: pgmap v278: 97 pgs: 97 active+clean; 275 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 834 KiB/s rd, 625 KiB/s wr, 82 op/s 2024-03-28T00:34:50.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:34:50 smithi165 ceph-mon[36302]: pgmap v279: 97 pgs: 97 active+clean; 275 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 701 KiB/s rd, 525 KiB/s wr, 72 op/s 2024-03-28T00:34:50.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:34:50 smithi119 ceph-mon[31518]: pgmap v279: 97 pgs: 97 active+clean; 275 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 701 KiB/s rd, 525 KiB/s wr, 72 op/s 2024-03-28T00:34:50.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:34:50 smithi132 ceph-mon[35941]: pgmap v279: 97 pgs: 97 active+clean; 275 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 701 KiB/s rd, 525 KiB/s wr, 72 op/s 2024-03-28T00:34:52.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:34:52 smithi165 ceph-mon[36302]: pgmap v280: 97 pgs: 97 active+clean; 275 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 440 KiB/s rd, 423 KiB/s wr, 66 op/s 2024-03-28T00:34:52.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:34:52 smithi119 ceph-mon[31518]: pgmap v280: 97 pgs: 97 active+clean; 275 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 440 KiB/s rd, 423 KiB/s wr, 66 op/s 2024-03-28T00:34:52.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:34:52 smithi132 ceph-mon[35941]: pgmap v280: 97 pgs: 97 active+clean; 275 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 440 KiB/s rd, 423 KiB/s wr, 66 op/s 2024-03-28T00:34:53.793 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:53 smithi119 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.g.service: Scheduled restart job, restart counter is at 3. 2024-03-28T00:34:53.793 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:53 smithi119 systemd[1]: Stopped Ceph mds.g for 10f73b92-ec98-11ee-95d0-87774f69a715. 2024-03-28T00:34:53.794 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:53 smithi119 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.g.service: Consumed 1.567s CPU time. 2024-03-28T00:34:53.794 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:53 smithi119 systemd[1]: Starting Ceph mds.g for 10f73b92-ec98-11ee-95d0-87774f69a715... 2024-03-28T00:34:53.794 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:53 smithi119 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.j.service: Scheduled restart job, restart counter is at 6. 2024-03-28T00:34:53.794 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:53 smithi119 systemd[1]: Stopped Ceph mds.j for 10f73b92-ec98-11ee-95d0-87774f69a715. 2024-03-28T00:34:53.794 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:53 smithi119 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.j.service: Consumed 1.515s CPU time. 2024-03-28T00:34:53.794 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:53 smithi119 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.j.service: Start request repeated too quickly. 2024-03-28T00:34:53.794 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:53 smithi119 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.j.service: Failed with result 'exit-code'. 2024-03-28T00:34:53.794 INFO:journalctl@ceph.mds.j.smithi119.stdout:Mar 28 00:34:53 smithi119 systemd[1]: Failed to start Ceph mds.j for 10f73b92-ec98-11ee-95d0-87774f69a715. 2024-03-28T00:34:54.135 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:53 smithi119 podman[118056]: 2024-03-28 00:34:53.692530513 +0000 UTC m=+0.034656698 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-28T00:34:54.135 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:53 smithi119 podman[118056]: 2024-03-28 00:34:53.855379078 +0000 UTC m=+0.197505301 container create 59f4cd1c1e8712186b7cca291250551393e02cfdab5e334ab7139be2c8cb8ae9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-03-28T00:34:54.407 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:54 smithi119 podman[118056]: 2024-03-28 00:34:54.128280392 +0000 UTC m=+0.470406619 container init 59f4cd1c1e8712186b7cca291250551393e02cfdab5e334ab7139be2c8cb8ae9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-03-28T00:34:54.407 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:54 smithi119 podman[118056]: 2024-03-28 00:34:54.138093115 +0000 UTC m=+0.480219339 container start 59f4cd1c1e8712186b7cca291250551393e02cfdab5e334ab7139be2c8cb8ae9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-28T00:34:54.408 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:54 smithi119 ceph-mds[118074]: -- 172.21.15.119:0/3348059760 <== mon.1 v2:172.21.15.165:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5587d12dda40 con 0x5587d12b0c00 2024-03-28T00:34:54.408 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:54 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: starting mds.g at 2024-03-28T00:34:54.408 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:54 smithi119 bash[118056]: 59f4cd1c1e8712186b7cca291250551393e02cfdab5e334ab7139be2c8cb8ae9 2024-03-28T00:34:54.408 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:34:54 smithi119 systemd[1]: Started Ceph mds.g for 10f73b92-ec98-11ee-95d0-87774f69a715. 2024-03-28T00:34:54.408 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:34:54 smithi119 ceph-mon[31518]: pgmap v281: 97 pgs: 97 active+clean; 275 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 441 KiB/s rd, 423 KiB/s wr, 66 op/s 2024-03-28T00:34:54.408 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:34:54 smithi119 ceph-mon[31518]: daemon mds.g restarted 2024-03-28T00:34:54.408 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:34:54 smithi119 ceph-mon[31518]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-28T00:34:54.408 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:34:54 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-28T00:34:54.408 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:34:54 smithi119 ceph-mon[31518]: osdmap e117: 12 total, 12 up, 12 in 2024-03-28T00:34:54.408 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:34:54 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.119:6834/3517971302,v1:172.21.15.119:6835/3517971302] up:boot 2024-03-28T00:34:54.408 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:34:54 smithi119 ceph-mon[31518]: Standby daemon mds.g assigned to filesystem cephfs as rank 3 2024-03-28T00:34:54.408 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:34:54 smithi119 ceph-mon[31518]: fsmap cephfs:4/5 {0=b=up:active,1=j=up:active,2=h=up:active,4=f=up:active} 4 up:standby-replay 1 up:standby, 1 failed 2024-03-28T00:34:54.409 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:34:54 smithi119 ceph-mon[31518]: fsmap cephfs:5/5 4 up:standby-replay 1 up:replay 4 up:active 2024-03-28T00:34:54.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:34:54 smithi165 ceph-mon[36302]: pgmap v281: 97 pgs: 97 active+clean; 275 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 441 KiB/s rd, 423 KiB/s wr, 66 op/s 2024-03-28T00:34:54.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:34:54 smithi165 ceph-mon[36302]: daemon mds.g restarted 2024-03-28T00:34:54.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:34:54 smithi165 ceph-mon[36302]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-28T00:34:54.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:34:54 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-28T00:34:54.635 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:34:54 smithi165 ceph-mon[36302]: osdmap e117: 12 total, 12 up, 12 in 2024-03-28T00:34:54.635 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:34:54 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.119:6834/3517971302,v1:172.21.15.119:6835/3517971302] up:boot 2024-03-28T00:34:54.635 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:34:54 smithi165 ceph-mon[36302]: Standby daemon mds.g assigned to filesystem cephfs as rank 3 2024-03-28T00:34:54.635 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:34:54 smithi165 ceph-mon[36302]: fsmap cephfs:4/5 {0=b=up:active,1=j=up:active,2=h=up:active,4=f=up:active} 4 up:standby-replay 1 up:standby, 1 failed 2024-03-28T00:34:54.635 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:34:54 smithi165 ceph-mon[36302]: fsmap cephfs:5/5 4 up:standby-replay 1 up:replay 4 up:active 2024-03-28T00:34:54.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:34:54 smithi132 ceph-mon[35941]: pgmap v281: 97 pgs: 97 active+clean; 275 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 441 KiB/s rd, 423 KiB/s wr, 66 op/s 2024-03-28T00:34:54.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:34:54 smithi132 ceph-mon[35941]: daemon mds.g restarted 2024-03-28T00:34:54.705 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:34:54 smithi132 ceph-mon[35941]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-28T00:34:54.705 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:34:54 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-28T00:34:54.705 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:34:54 smithi132 ceph-mon[35941]: osdmap e117: 12 total, 12 up, 12 in 2024-03-28T00:34:54.705 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:34:54 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.119:6834/3517971302,v1:172.21.15.119:6835/3517971302] up:boot 2024-03-28T00:34:54.705 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:34:54 smithi132 ceph-mon[35941]: Standby daemon mds.g assigned to filesystem cephfs as rank 3 2024-03-28T00:34:54.705 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:34:54 smithi132 ceph-mon[35941]: fsmap cephfs:4/5 {0=b=up:active,1=j=up:active,2=h=up:active,4=f=up:active} 4 up:standby-replay 1 up:standby, 1 failed 2024-03-28T00:34:54.705 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:34:54 smithi132 ceph-mon[35941]: fsmap cephfs:5/5 4 up:standby-replay 1 up:replay 4 up:active 2024-03-28T00:34:56.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:34:56 smithi165 ceph-mon[36302]: pgmap v283: 97 pgs: 97 active+clean; 275 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 4.8 KiB/s wr, 8 op/s 2024-03-28T00:34:56.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:34:56 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.119:6834/3517971302,v1:172.21.15.119:6835/3517971302] up:resolve 2024-03-28T00:34:56.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:34:56 smithi165 ceph-mon[36302]: fsmap cephfs:5/5 4 up:standby-replay 1 up:resolve 4 up:active 2024-03-28T00:34:56.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:34:56 smithi119 ceph-mon[31518]: pgmap v283: 97 pgs: 97 active+clean; 275 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 4.8 KiB/s wr, 8 op/s 2024-03-28T00:34:56.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:34:56 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.119:6834/3517971302,v1:172.21.15.119:6835/3517971302] up:resolve 2024-03-28T00:34:56.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:34:56 smithi119 ceph-mon[31518]: fsmap cephfs:5/5 4 up:standby-replay 1 up:resolve 4 up:active 2024-03-28T00:34:56.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:34:56 smithi132 ceph-mon[35941]: pgmap v283: 97 pgs: 97 active+clean; 275 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 4.8 KiB/s wr, 8 op/s 2024-03-28T00:34:56.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:34:56 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.119:6834/3517971302,v1:172.21.15.119:6835/3517971302] up:resolve 2024-03-28T00:34:56.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:34:56 smithi132 ceph-mon[35941]: fsmap cephfs:5/5 4 up:standby-replay 1 up:resolve 4 up:active 2024-03-28T00:34:58.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:34:58 smithi165 ceph-mon[36302]: pgmap v284: 97 pgs: 97 active+clean; 275 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 4.8 KiB/s wr, 8 op/s 2024-03-28T00:34:58.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:34:58 smithi119 ceph-mon[31518]: pgmap v284: 97 pgs: 97 active+clean; 275 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 4.8 KiB/s wr, 8 op/s 2024-03-28T00:34:58.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:34:58 smithi132 ceph-mon[35941]: pgmap v284: 97 pgs: 97 active+clean; 275 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 4.8 KiB/s wr, 8 op/s 2024-03-28T00:35:00.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:35:00 smithi165 ceph-mon[36302]: pgmap v285: 97 pgs: 97 active+clean; 275 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-03-28T00:35:00.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:35:00 smithi165 ceph-mon[36302]: Replacing daemon mds.j as rank 1 with standby daemon mds.d 2024-03-28T00:35:00.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:35:00 smithi165 ceph-mon[36302]: MDS daemon mds.j is removed because it is dead or otherwise unavailable. 2024-03-28T00:35:00.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:35:00 smithi165 ceph-mon[36302]: osdmap e118: 12 total, 12 up, 12 in 2024-03-28T00:35:00.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:35:00 smithi165 ceph-mon[36302]: fsmap cephfs:5/5 3 up:standby-replay 1 up:replay 1 up:resolve 3 up:active 2024-03-28T00:35:00.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:35:00 smithi119 ceph-mon[31518]: pgmap v285: 97 pgs: 97 active+clean; 275 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-03-28T00:35:00.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:35:00 smithi119 ceph-mon[31518]: Replacing daemon mds.j as rank 1 with standby daemon mds.d 2024-03-28T00:35:00.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:35:00 smithi119 ceph-mon[31518]: MDS daemon mds.j is removed because it is dead or otherwise unavailable. 2024-03-28T00:35:00.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:35:00 smithi119 ceph-mon[31518]: osdmap e118: 12 total, 12 up, 12 in 2024-03-28T00:35:00.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:35:00 smithi119 ceph-mon[31518]: fsmap cephfs:5/5 3 up:standby-replay 1 up:replay 1 up:resolve 3 up:active 2024-03-28T00:35:00.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:35:00 smithi132 ceph-mon[35941]: pgmap v285: 97 pgs: 97 active+clean; 275 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-03-28T00:35:00.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:35:00 smithi132 ceph-mon[35941]: Replacing daemon mds.j as rank 1 with standby daemon mds.d 2024-03-28T00:35:00.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:35:00 smithi132 ceph-mon[35941]: MDS daemon mds.j is removed because it is dead or otherwise unavailable. 2024-03-28T00:35:00.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:35:00 smithi132 ceph-mon[35941]: osdmap e118: 12 total, 12 up, 12 in 2024-03-28T00:35:00.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:35:00 smithi132 ceph-mon[35941]: fsmap cephfs:5/5 3 up:standby-replay 1 up:replay 1 up:resolve 3 up:active 2024-03-28T00:35:02.605 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:35:02 smithi132 ceph-mon[35941]: pgmap v287: 97 pgs: 97 active+clean; 275 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 2.1 KiB/s wr, 12 op/s 2024-03-28T00:35:02.606 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:35:02 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.119:6836/482178170,v1:172.21.15.119:6837/482178170] up:resolve 2024-03-28T00:35:02.606 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:35:02 smithi132 ceph-mon[35941]: fsmap cephfs:5/5 3 up:standby-replay 2 up:resolve 3 up:active 2024-03-28T00:35:02.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:35:02 smithi165 ceph-mon[36302]: pgmap v287: 97 pgs: 97 active+clean; 275 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 2.1 KiB/s wr, 12 op/s 2024-03-28T00:35:02.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:35:02 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.119:6836/482178170,v1:172.21.15.119:6837/482178170] up:resolve 2024-03-28T00:35:02.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:35:02 smithi165 ceph-mon[36302]: fsmap cephfs:5/5 3 up:standby-replay 2 up:resolve 3 up:active 2024-03-28T00:35:02.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:35:02 smithi119 ceph-mon[31518]: pgmap v287: 97 pgs: 97 active+clean; 275 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 2.1 KiB/s wr, 12 op/s 2024-03-28T00:35:02.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:35:02 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.119:6836/482178170,v1:172.21.15.119:6837/482178170] up:resolve 2024-03-28T00:35:02.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:35:02 smithi119 ceph-mon[31518]: fsmap cephfs:5/5 3 up:standby-replay 2 up:resolve 3 up:active 2024-03-28T00:35:03.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:35:03 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.119:6836/482178170,v1:172.21.15.119:6837/482178170] up:reconnect 2024-03-28T00:35:03.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:35:03 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.119:6834/3517971302,v1:172.21.15.119:6835/3517971302] up:reconnect 2024-03-28T00:35:03.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:35:03 smithi165 ceph-mon[36302]: fsmap cephfs:5/5 3 up:standby-replay 2 up:reconnect 3 up:active 2024-03-28T00:35:03.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:35:03 smithi165 ceph-mon[36302]: reconnect by client.25355 172.21.15.119:0/3071152482 after 0.0109997 2024-03-28T00:35:03.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:35:03 smithi165 ceph-mon[36302]: reconnect by client.25355 172.21.15.119:0/3071152482 after 0.0109997 2024-03-28T00:35:03.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:35:03 smithi165 ceph-mon[36302]: reconnect by client.25264 v1:172.21.15.119:0/3863511511 after 0.0179996 2024-03-28T00:35:03.635 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:35:03 smithi165 ceph-mon[36302]: reconnect by client.25264 v1:172.21.15.119:0/3863511511 after 0.0179996 2024-03-28T00:35:03.635 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:35:03 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:35:03.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:35:03 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.119:6836/482178170,v1:172.21.15.119:6837/482178170] up:reconnect 2024-03-28T00:35:03.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:35:03 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.119:6834/3517971302,v1:172.21.15.119:6835/3517971302] up:reconnect 2024-03-28T00:35:03.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:35:03 smithi119 ceph-mon[31518]: fsmap cephfs:5/5 3 up:standby-replay 2 up:reconnect 3 up:active 2024-03-28T00:35:03.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:35:03 smithi119 ceph-mon[31518]: reconnect by client.25355 172.21.15.119:0/3071152482 after 0.0109997 2024-03-28T00:35:03.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:35:03 smithi119 ceph-mon[31518]: reconnect by client.25355 172.21.15.119:0/3071152482 after 0.0109997 2024-03-28T00:35:03.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:35:03 smithi119 ceph-mon[31518]: reconnect by client.25264 v1:172.21.15.119:0/3863511511 after 0.0179996 2024-03-28T00:35:03.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:35:03 smithi119 ceph-mon[31518]: reconnect by client.25264 v1:172.21.15.119:0/3863511511 after 0.0179996 2024-03-28T00:35:03.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:35:03 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:35:03.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:35:03 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.119:6836/482178170,v1:172.21.15.119:6837/482178170] up:reconnect 2024-03-28T00:35:03.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:35:03 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.119:6834/3517971302,v1:172.21.15.119:6835/3517971302] up:reconnect 2024-03-28T00:35:03.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:35:03 smithi132 ceph-mon[35941]: fsmap cephfs:5/5 3 up:standby-replay 2 up:reconnect 3 up:active 2024-03-28T00:35:03.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:35:03 smithi132 ceph-mon[35941]: reconnect by client.25355 172.21.15.119:0/3071152482 after 0.0109997 2024-03-28T00:35:03.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:35:03 smithi132 ceph-mon[35941]: reconnect by client.25355 172.21.15.119:0/3071152482 after 0.0109997 2024-03-28T00:35:03.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:35:03 smithi132 ceph-mon[35941]: reconnect by client.25264 v1:172.21.15.119:0/3863511511 after 0.0179996 2024-03-28T00:35:03.705 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:35:03 smithi132 ceph-mon[35941]: reconnect by client.25264 v1:172.21.15.119:0/3863511511 after 0.0179996 2024-03-28T00:35:03.705 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:35:03 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:35:04.507 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-mon[31518]: pgmap v288: 97 pgs: 97 active+clean; 275 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 1.9 KiB/s wr, 6 op/s 2024-03-28T00:35:04.508 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.119:6836/482178170,v1:172.21.15.119:6837/482178170] up:rejoin 2024-03-28T00:35:04.508 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.119:6834/3517971302,v1:172.21.15.119:6835/3517971302] up:rejoin 2024-03-28T00:35:04.508 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-mon[31518]: fsmap cephfs:5/5 3 up:standby-replay 2 up:rejoin 3 up:active 2024-03-28T00:35:04.508 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-mon[31518]: daemon mds.g is now active in filesystem cephfs as rank 3 2024-03-28T00:35:04.508 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-mon[31518]: daemon mds.d is now active in filesystem cephfs as rank 1 2024-03-28T00:35:04.508 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:35:04.508 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:35:04.508 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:35:04.509 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7f219aa00700 time 2024-03-28T00:35:04.336361+0000 2024-03-28T00:35:04.509 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T00:35:04.509 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:35:04.509 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f21a62e34a1] 2024-03-28T00:35:04.509 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f21a62e3667] 2024-03-28T00:35:04.509 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55f8690cd223] 2024-03-28T00:35:04.509 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55f868f475d6] 2024-03-28T00:35:04.509 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55f868f47da1] 2024-03-28T00:35:04.509 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: 6: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x55f868f7c09c] 2024-03-28T00:35:04.510 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55f868f9a233] 2024-03-28T00:35:04.510 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55f868f9c197] 2024-03-28T00:35:04.510 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55f868e9b11b] 2024-03-28T00:35:04.510 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55f868e9b6bc] 2024-03-28T00:35:04.510 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55f868e851ff] 2024-03-28T00:35:04.510 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f21a6575488] 2024-03-28T00:35:04.510 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: 13: (DispatchQueue::entry()+0x51f) [0x7f21a657260f] 2024-03-28T00:35:04.510 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f21a6645e81] 2024-03-28T00:35:04.510 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f21a50601ca] 2024-03-28T00:35:04.510 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: 16: clone() 2024-03-28T00:35:04.511 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7f0563000700 time 2024-03-28T00:35:04.341366+0000 2024-03-28T00:35:04.511 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T00:35:04.511 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:35:04.511 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f056e94a4a1] 2024-03-28T00:35:04.511 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f056e94a667] 2024-03-28T00:35:04.511 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x5587cf042223] 2024-03-28T00:35:04.511 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x5587ceebc5d6] 2024-03-28T00:35:04.511 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x5587ceebcda1] 2024-03-28T00:35:04.511 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: 6: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x5587ceef109c] 2024-03-28T00:35:04.511 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x5587cef0f233] 2024-03-28T00:35:04.512 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5587cef11197] 2024-03-28T00:35:04.512 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x5587cee1011b] 2024-03-28T00:35:04.512 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5587cee106bc] 2024-03-28T00:35:04.512 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x5587cedfa1ff] 2024-03-28T00:35:04.512 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f056ebdc488] 2024-03-28T00:35:04.512 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: 13: (DispatchQueue::entry()+0x51f) [0x7f056ebd960f] 2024-03-28T00:35:04.512 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f056ecace81] 2024-03-28T00:35:04.512 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f056d6c71ca] 2024-03-28T00:35:04.512 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: 16: clone() 2024-03-28T00:35:04.513 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: 2024-03-28T00:35:04.337+0000 7f219aa00700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7f219aa00700 time 2024-03-28T00:35:04.336361+0000 2024-03-28T00:35:04.513 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T00:35:04.513 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: 2024-03-28T00:35:04.513 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:35:04.513 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f21a62e34a1] 2024-03-28T00:35:04.513 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f21a62e3667] 2024-03-28T00:35:04.513 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55f8690cd223] 2024-03-28T00:35:04.513 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55f868f475d6] 2024-03-28T00:35:04.513 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55f868f47da1] 2024-03-28T00:35:04.514 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: 6: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x55f868f7c09c] 2024-03-28T00:35:04.514 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55f868f9a233] 2024-03-28T00:35:04.514 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55f868f9c197] 2024-03-28T00:35:04.514 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55f868e9b11b] 2024-03-28T00:35:04.514 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55f868e9b6bc] 2024-03-28T00:35:04.514 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55f868e851ff] 2024-03-28T00:35:04.514 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f21a6575488] 2024-03-28T00:35:04.514 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: 13: (DispatchQueue::entry()+0x51f) [0x7f21a657260f] 2024-03-28T00:35:04.514 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f21a6645e81] 2024-03-28T00:35:04.514 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f21a50601ca] 2024-03-28T00:35:04.514 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: 16: clone() 2024-03-28T00:35:04.514 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: 2024-03-28T00:35:04.515 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: *** Caught signal (Aborted) ** 2024-03-28T00:35:04.515 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: in thread 7f219aa00700 thread_name:ms_dispatch 2024-03-28T00:35:04.515 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:35:04.515 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f21a506ad20] 2024-03-28T00:35:04.515 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: 2024-03-28T00:35:04.342+0000 7f0563000700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7f0563000700 time 2024-03-28T00:35:04.341366+0000 2024-03-28T00:35:04.515 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T00:35:04.515 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: 2024-03-28T00:35:04.515 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:35:04.515 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f056e94a4a1] 2024-03-28T00:35:04.515 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f056e94a667] 2024-03-28T00:35:04.516 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x5587cf042223] 2024-03-28T00:35:04.516 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x5587ceebc5d6] 2024-03-28T00:35:04.516 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x5587ceebcda1] 2024-03-28T00:35:04.516 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: 6: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x5587ceef109c] 2024-03-28T00:35:04.516 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x5587cef0f233] 2024-03-28T00:35:04.516 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5587cef11197] 2024-03-28T00:35:04.516 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x5587cee1011b] 2024-03-28T00:35:04.516 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5587cee106bc] 2024-03-28T00:35:04.516 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x5587cedfa1ff] 2024-03-28T00:35:04.516 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f056ebdc488] 2024-03-28T00:35:04.516 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: 13: (DispatchQueue::entry()+0x51f) [0x7f056ebd960f] 2024-03-28T00:35:04.516 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f056ecace81] 2024-03-28T00:35:04.516 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f056d6c71ca] 2024-03-28T00:35:04.516 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: 16: clone() 2024-03-28T00:35:04.516 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: 2024-03-28T00:35:04.517 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: *** Caught signal (Aborted) ** 2024-03-28T00:35:04.517 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: in thread 7f0563000700 thread_name:ms_dispatch 2024-03-28T00:35:04.517 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:35:04.517 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f056d6d1d20] 2024-03-28T00:35:04.517 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: 2: gsignal() 2024-03-28T00:35:04.517 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: 3: abort() 2024-03-28T00:35:04.517 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f21a62e34fb] 2024-03-28T00:35:04.517 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f21a62e3667] 2024-03-28T00:35:04.518 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55f8690cd223] 2024-03-28T00:35:04.518 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55f868f475d6] 2024-03-28T00:35:04.518 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55f868f47da1] 2024-03-28T00:35:04.518 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: 9: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x55f868f7c09c] 2024-03-28T00:35:04.518 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55f868f9a233] 2024-03-28T00:35:04.518 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55f868f9c197] 2024-03-28T00:35:04.518 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55f868e9b11b] 2024-03-28T00:35:04.518 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55f868e9b6bc] 2024-03-28T00:35:04.518 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55f868e851ff] 2024-03-28T00:35:04.518 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f21a6575488] 2024-03-28T00:35:04.518 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: 16: (DispatchQueue::entry()+0x51f) [0x7f21a657260f] 2024-03-28T00:35:04.518 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f21a6645e81] 2024-03-28T00:35:04.518 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f21a50601ca] 2024-03-28T00:35:04.519 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: 19: clone() 2024-03-28T00:35:04.519 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: 2024-03-28T00:35:04.340+0000 7f219aa00700 -1 *** Caught signal (Aborted) ** 2024-03-28T00:35:04.519 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: in thread 7f219aa00700 thread_name:ms_dispatch 2024-03-28T00:35:04.519 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: 2024-03-28T00:35:04.519 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:35:04.519 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f21a506ad20] 2024-03-28T00:35:04.519 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: 2: gsignal() 2024-03-28T00:35:04.519 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: 3: abort() 2024-03-28T00:35:04.519 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f21a62e34fb] 2024-03-28T00:35:04.519 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f21a62e3667] 2024-03-28T00:35:04.519 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55f8690cd223] 2024-03-28T00:35:04.519 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55f868f475d6] 2024-03-28T00:35:04.519 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55f868f47da1] 2024-03-28T00:35:04.519 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: 9: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x55f868f7c09c] 2024-03-28T00:35:04.520 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55f868f9a233] 2024-03-28T00:35:04.520 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55f868f9c197] 2024-03-28T00:35:04.520 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55f868e9b11b] 2024-03-28T00:35:04.520 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55f868e9b6bc] 2024-03-28T00:35:04.520 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55f868e851ff] 2024-03-28T00:35:04.520 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f21a6575488] 2024-03-28T00:35:04.520 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: 16: (DispatchQueue::entry()+0x51f) [0x7f21a657260f] 2024-03-28T00:35:04.520 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f21a6645e81] 2024-03-28T00:35:04.520 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f21a50601ca] 2024-03-28T00:35:04.520 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: 19: clone() 2024-03-28T00:35:04.520 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T00:35:04.520 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: 2024-03-28T00:35:04.520 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: -1> 2024-03-28T00:35:04.337+0000 7f219aa00700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7f219aa00700 time 2024-03-28T00:35:04.336361+0000 2024-03-28T00:35:04.520 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T00:35:04.521 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: 2024-03-28T00:35:04.521 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:35:04.521 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f21a62e34a1] 2024-03-28T00:35:04.521 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f21a62e3667] 2024-03-28T00:35:04.521 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: 2: gsignal() 2024-03-28T00:35:04.521 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: 3: abort() 2024-03-28T00:35:04.521 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f056e94a4fb] 2024-03-28T00:35:04.521 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f056e94a667] 2024-03-28T00:35:04.521 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x5587cf042223] 2024-03-28T00:35:04.521 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x5587ceebc5d6] 2024-03-28T00:35:04.521 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x5587ceebcda1] 2024-03-28T00:35:04.522 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: 9: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x5587ceef109c] 2024-03-28T00:35:04.522 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x5587cef0f233] 2024-03-28T00:35:04.522 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5587cef11197] 2024-03-28T00:35:04.522 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x5587cee1011b] 2024-03-28T00:35:04.522 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5587cee106bc] 2024-03-28T00:35:04.522 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x5587cedfa1ff] 2024-03-28T00:35:04.522 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f056ebdc488] 2024-03-28T00:35:04.522 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: 16: (DispatchQueue::entry()+0x51f) [0x7f056ebd960f] 2024-03-28T00:35:04.522 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f056ecace81] 2024-03-28T00:35:04.522 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f056d6c71ca] 2024-03-28T00:35:04.522 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: 19: clone() 2024-03-28T00:35:04.522 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: 2024-03-28T00:35:04.345+0000 7f0563000700 -1 *** Caught signal (Aborted) ** 2024-03-28T00:35:04.522 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: in thread 7f0563000700 thread_name:ms_dispatch 2024-03-28T00:35:04.522 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: 2024-03-28T00:35:04.523 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:35:04.523 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f056d6d1d20] 2024-03-28T00:35:04.523 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: 2: gsignal() 2024-03-28T00:35:04.523 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: 3: abort() 2024-03-28T00:35:04.523 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f056e94a4fb] 2024-03-28T00:35:04.523 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f056e94a667] 2024-03-28T00:35:04.523 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55f8690cd223] 2024-03-28T00:35:04.523 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55f868f475d6] 2024-03-28T00:35:04.523 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55f868f47da1] 2024-03-28T00:35:04.524 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: 6: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x55f868f7c09c] 2024-03-28T00:35:04.524 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55f868f9a233] 2024-03-28T00:35:04.524 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55f868f9c197] 2024-03-28T00:35:04.524 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55f868e9b11b] 2024-03-28T00:35:04.524 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55f868e9b6bc] 2024-03-28T00:35:04.524 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55f868e851ff] 2024-03-28T00:35:04.524 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f21a6575488] 2024-03-28T00:35:04.524 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: 13: (DispatchQueue::entry()+0x51f) [0x7f21a657260f] 2024-03-28T00:35:04.524 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f21a6645e81] 2024-03-28T00:35:04.524 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f21a50601ca] 2024-03-28T00:35:04.524 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: 16: clone() 2024-03-28T00:35:04.524 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: 2024-03-28T00:35:04.524 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: 0> 2024-03-28T00:35:04.340+0000 7f219aa00700 -1 *** Caught signal (Aborted) ** 2024-03-28T00:35:04.524 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: in thread 7f219aa00700 thread_name:ms_dispatch 2024-03-28T00:35:04.524 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: 2024-03-28T00:35:04.525 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:35:04.525 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f21a506ad20] 2024-03-28T00:35:04.525 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: 2: gsignal() 2024-03-28T00:35:04.525 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: 3: abort() 2024-03-28T00:35:04.525 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f21a62e34fb] 2024-03-28T00:35:04.525 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f21a62e3667] 2024-03-28T00:35:04.525 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55f8690cd223] 2024-03-28T00:35:04.525 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55f868f475d6] 2024-03-28T00:35:04.525 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55f868f47da1] 2024-03-28T00:35:04.525 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: 9: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x55f868f7c09c] 2024-03-28T00:35:04.525 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55f868f9a233] 2024-03-28T00:35:04.525 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55f868f9c197] 2024-03-28T00:35:04.525 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55f868e9b11b] 2024-03-28T00:35:04.525 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55f868e9b6bc] 2024-03-28T00:35:04.526 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55f868e851ff] 2024-03-28T00:35:04.526 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f21a6575488] 2024-03-28T00:35:04.526 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: 16: (DispatchQueue::entry()+0x51f) [0x7f21a657260f] 2024-03-28T00:35:04.526 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f21a6645e81] 2024-03-28T00:35:04.526 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f21a50601ca] 2024-03-28T00:35:04.526 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: 19: clone() 2024-03-28T00:35:04.526 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T00:35:04.526 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: 2024-03-28T00:35:04.526 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: -1> 2024-03-28T00:35:04.337+0000 7f219aa00700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7f219aa00700 time 2024-03-28T00:35:04.336361+0000 2024-03-28T00:35:04.526 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T00:35:04.526 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: 2024-03-28T00:35:04.526 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:35:04.526 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f21a62e34a1] 2024-03-28T00:35:04.526 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f21a62e3667] 2024-03-28T00:35:04.527 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55f8690cd223] 2024-03-28T00:35:04.527 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55f868f475d6] 2024-03-28T00:35:04.527 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x5587cf042223] 2024-03-28T00:35:04.527 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x5587ceebc5d6] 2024-03-28T00:35:04.527 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x5587ceebcda1] 2024-03-28T00:35:04.527 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: 9: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x5587ceef109c] 2024-03-28T00:35:04.527 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x5587cef0f233] 2024-03-28T00:35:04.527 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5587cef11197] 2024-03-28T00:35:04.527 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x5587cee1011b] 2024-03-28T00:35:04.527 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5587cee106bc] 2024-03-28T00:35:04.527 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x5587cedfa1ff] 2024-03-28T00:35:04.527 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f056ebdc488] 2024-03-28T00:35:04.528 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: 16: (DispatchQueue::entry()+0x51f) [0x7f056ebd960f] 2024-03-28T00:35:04.528 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f056ecace81] 2024-03-28T00:35:04.528 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f056d6c71ca] 2024-03-28T00:35:04.528 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: 19: clone() 2024-03-28T00:35:04.528 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T00:35:04.528 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: 2024-03-28T00:35:04.528 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: -1> 2024-03-28T00:35:04.342+0000 7f0563000700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7f0563000700 time 2024-03-28T00:35:04.341366+0000 2024-03-28T00:35:04.528 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T00:35:04.528 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: 2024-03-28T00:35:04.528 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:35:04.528 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f056e94a4a1] 2024-03-28T00:35:04.528 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f056e94a667] 2024-03-28T00:35:04.529 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55f868f47da1] 2024-03-28T00:35:04.529 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: 6: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x55f868f7c09c] 2024-03-28T00:35:04.529 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55f868f9a233] 2024-03-28T00:35:04.529 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55f868f9c197] 2024-03-28T00:35:04.529 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55f868e9b11b] 2024-03-28T00:35:04.529 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55f868e9b6bc] 2024-03-28T00:35:04.529 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55f868e851ff] 2024-03-28T00:35:04.529 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f21a6575488] 2024-03-28T00:35:04.529 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: 13: (DispatchQueue::entry()+0x51f) [0x7f21a657260f] 2024-03-28T00:35:04.529 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f21a6645e81] 2024-03-28T00:35:04.530 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f21a50601ca] 2024-03-28T00:35:04.530 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: 16: clone() 2024-03-28T00:35:04.530 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: 2024-03-28T00:35:04.530 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: 0> 2024-03-28T00:35:04.340+0000 7f219aa00700 -1 *** Caught signal (Aborted) ** 2024-03-28T00:35:04.530 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: in thread 7f219aa00700 thread_name:ms_dispatch 2024-03-28T00:35:04.530 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: 2024-03-28T00:35:04.530 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:35:04.530 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f21a506ad20] 2024-03-28T00:35:04.530 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: 2: gsignal() 2024-03-28T00:35:04.530 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: 3: abort() 2024-03-28T00:35:04.530 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f21a62e34fb] 2024-03-28T00:35:04.530 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f21a62e3667] 2024-03-28T00:35:04.530 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55f8690cd223] 2024-03-28T00:35:04.530 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55f868f475d6] 2024-03-28T00:35:04.531 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55f868f47da1] 2024-03-28T00:35:04.531 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: 9: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x55f868f7c09c] 2024-03-28T00:35:04.531 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55f868f9a233] 2024-03-28T00:35:04.531 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55f868f9c197] 2024-03-28T00:35:04.531 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55f868e9b11b] 2024-03-28T00:35:04.531 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55f868e9b6bc] 2024-03-28T00:35:04.531 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55f868e851ff] 2024-03-28T00:35:04.531 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f21a6575488] 2024-03-28T00:35:04.531 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: 16: (DispatchQueue::entry()+0x51f) [0x7f21a657260f] 2024-03-28T00:35:04.531 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f21a6645e81] 2024-03-28T00:35:04.531 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f21a50601ca] 2024-03-28T00:35:04.531 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: 19: clone() 2024-03-28T00:35:04.531 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T00:35:04.531 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[117152]: 2024-03-28T00:35:04.531 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:04 smithi119 podman[118655]: 2024-03-28 00:35:04.501232034 +0000 UTC m=+0.042134733 container died 50102c2aedd4271caac8481669f65eb8864480e98cc491d92f71fce33bdef1f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.schema-version=1.0) 2024-03-28T00:35:04.532 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x5587cf042223] 2024-03-28T00:35:04.532 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x5587ceebc5d6] 2024-03-28T00:35:04.532 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x5587ceebcda1] 2024-03-28T00:35:04.532 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: 6: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x5587ceef109c] 2024-03-28T00:35:04.532 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x5587cef0f233] 2024-03-28T00:35:04.532 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5587cef11197] 2024-03-28T00:35:04.533 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x5587cee1011b] 2024-03-28T00:35:04.533 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5587cee106bc] 2024-03-28T00:35:04.533 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x5587cedfa1ff] 2024-03-28T00:35:04.533 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f056ebdc488] 2024-03-28T00:35:04.533 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: 13: (DispatchQueue::entry()+0x51f) [0x7f056ebd960f] 2024-03-28T00:35:04.533 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f056ecace81] 2024-03-28T00:35:04.533 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f056d6c71ca] 2024-03-28T00:35:04.533 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: 16: clone() 2024-03-28T00:35:04.533 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: 2024-03-28T00:35:04.533 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: 0> 2024-03-28T00:35:04.345+0000 7f0563000700 -1 *** Caught signal (Aborted) ** 2024-03-28T00:35:04.533 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: in thread 7f0563000700 thread_name:ms_dispatch 2024-03-28T00:35:04.533 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: 2024-03-28T00:35:04.533 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:35:04.533 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f056d6d1d20] 2024-03-28T00:35:04.533 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: 2: gsignal() 2024-03-28T00:35:04.534 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: 3: abort() 2024-03-28T00:35:04.534 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f056e94a4fb] 2024-03-28T00:35:04.534 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f056e94a667] 2024-03-28T00:35:04.534 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x5587cf042223] 2024-03-28T00:35:04.534 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x5587ceebc5d6] 2024-03-28T00:35:04.534 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x5587ceebcda1] 2024-03-28T00:35:04.534 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: 9: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x5587ceef109c] 2024-03-28T00:35:04.534 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x5587cef0f233] 2024-03-28T00:35:04.534 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5587cef11197] 2024-03-28T00:35:04.534 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x5587cee1011b] 2024-03-28T00:35:04.534 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5587cee106bc] 2024-03-28T00:35:04.534 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x5587cedfa1ff] 2024-03-28T00:35:04.534 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f056ebdc488] 2024-03-28T00:35:04.534 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: 16: (DispatchQueue::entry()+0x51f) [0x7f056ebd960f] 2024-03-28T00:35:04.534 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f056ecace81] 2024-03-28T00:35:04.535 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f056d6c71ca] 2024-03-28T00:35:04.535 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: 19: clone() 2024-03-28T00:35:04.535 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T00:35:04.535 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: 2024-03-28T00:35:04.535 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: -1> 2024-03-28T00:35:04.342+0000 7f0563000700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7f0563000700 time 2024-03-28T00:35:04.341366+0000 2024-03-28T00:35:04.535 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T00:35:04.535 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: 2024-03-28T00:35:04.535 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:35:04.535 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f056e94a4a1] 2024-03-28T00:35:04.535 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f056e94a667] 2024-03-28T00:35:04.535 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x5587cf042223] 2024-03-28T00:35:04.535 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x5587ceebc5d6] 2024-03-28T00:35:04.535 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x5587ceebcda1] 2024-03-28T00:35:04.535 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: 6: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x5587ceef109c] 2024-03-28T00:35:04.535 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x5587cef0f233] 2024-03-28T00:35:04.535 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5587cef11197] 2024-03-28T00:35:04.536 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x5587cee1011b] 2024-03-28T00:35:04.536 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5587cee106bc] 2024-03-28T00:35:04.536 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x5587cedfa1ff] 2024-03-28T00:35:04.536 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f056ebdc488] 2024-03-28T00:35:04.536 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: 13: (DispatchQueue::entry()+0x51f) [0x7f056ebd960f] 2024-03-28T00:35:04.536 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f056ecace81] 2024-03-28T00:35:04.536 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f056d6c71ca] 2024-03-28T00:35:04.536 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: 16: clone() 2024-03-28T00:35:04.536 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: 2024-03-28T00:35:04.536 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: 0> 2024-03-28T00:35:04.345+0000 7f0563000700 -1 *** Caught signal (Aborted) ** 2024-03-28T00:35:04.536 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: in thread 7f0563000700 thread_name:ms_dispatch 2024-03-28T00:35:04.536 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: 2024-03-28T00:35:04.536 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:35:04.536 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f056d6d1d20] 2024-03-28T00:35:04.536 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: 2: gsignal() 2024-03-28T00:35:04.537 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: 3: abort() 2024-03-28T00:35:04.537 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f056e94a4fb] 2024-03-28T00:35:04.537 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f056e94a667] 2024-03-28T00:35:04.537 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x5587cf042223] 2024-03-28T00:35:04.537 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x5587ceebc5d6] 2024-03-28T00:35:04.537 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x5587ceebcda1] 2024-03-28T00:35:04.537 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: 9: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x5587ceef109c] 2024-03-28T00:35:04.537 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x5587cef0f233] 2024-03-28T00:35:04.537 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5587cef11197] 2024-03-28T00:35:04.537 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x5587cee1011b] 2024-03-28T00:35:04.537 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5587cee106bc] 2024-03-28T00:35:04.537 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x5587cedfa1ff] 2024-03-28T00:35:04.538 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f056ebdc488] 2024-03-28T00:35:04.538 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: 16: (DispatchQueue::entry()+0x51f) [0x7f056ebd960f] 2024-03-28T00:35:04.538 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f056ecace81] 2024-03-28T00:35:04.538 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f056d6c71ca] 2024-03-28T00:35:04.538 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: 19: clone() 2024-03-28T00:35:04.538 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T00:35:04.538 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118070]: 2024-03-28T00:35:04.538 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:04 smithi119 podman[118654]: 2024-03-28 00:35:04.501019497 +0000 UTC m=+0.045002771 container died 59f4cd1c1e8712186b7cca291250551393e02cfdab5e334ab7139be2c8cb8ae9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-03-28T00:35:04.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:35:04 smithi165 ceph-mon[36302]: pgmap v288: 97 pgs: 97 active+clean; 275 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 1.9 KiB/s wr, 6 op/s 2024-03-28T00:35:04.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:35:04 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.119:6836/482178170,v1:172.21.15.119:6837/482178170] up:rejoin 2024-03-28T00:35:04.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:35:04 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.119:6834/3517971302,v1:172.21.15.119:6835/3517971302] up:rejoin 2024-03-28T00:35:04.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:35:04 smithi165 ceph-mon[36302]: fsmap cephfs:5/5 3 up:standby-replay 2 up:rejoin 3 up:active 2024-03-28T00:35:04.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:35:04 smithi165 ceph-mon[36302]: daemon mds.g is now active in filesystem cephfs as rank 3 2024-03-28T00:35:04.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:35:04 smithi165 ceph-mon[36302]: daemon mds.d is now active in filesystem cephfs as rank 1 2024-03-28T00:35:04.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:35:04 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:35:04.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:35:04 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:35:04.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:35:04 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:35:04.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:35:04 smithi132 ceph-mon[35941]: pgmap v288: 97 pgs: 97 active+clean; 275 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 1.9 KiB/s wr, 6 op/s 2024-03-28T00:35:04.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:35:04 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.119:6836/482178170,v1:172.21.15.119:6837/482178170] up:rejoin 2024-03-28T00:35:04.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:35:04 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.119:6834/3517971302,v1:172.21.15.119:6835/3517971302] up:rejoin 2024-03-28T00:35:04.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:35:04 smithi132 ceph-mon[35941]: fsmap cephfs:5/5 3 up:standby-replay 2 up:rejoin 3 up:active 2024-03-28T00:35:04.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:35:04 smithi132 ceph-mon[35941]: daemon mds.g is now active in filesystem cephfs as rank 3 2024-03-28T00:35:04.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:35:04 smithi132 ceph-mon[35941]: daemon mds.d is now active in filesystem cephfs as rank 1 2024-03-28T00:35:04.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:35:04 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:35:04.705 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:35:04 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:35:04.705 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:35:04 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:35:05.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:35:05 smithi165 ceph-mon[36302]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-28T00:35:05.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:35:05 smithi165 ceph-mon[36302]: Cluster is now healthy 2024-03-28T00:35:05.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:35:05 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.119:6834/3517971302,v1:172.21.15.119:6835/3517971302] up:active 2024-03-28T00:35:05.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:35:05 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.119:6836/482178170,v1:172.21.15.119:6837/482178170] up:active 2024-03-28T00:35:05.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:35:05 smithi165 ceph-mon[36302]: fsmap cephfs:5 3 up:standby-replay 5 up:active 2024-03-28T00:35:05.656 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:05 smithi119 podman[118654]: 2024-03-28 00:35:05.323705668 +0000 UTC m=+0.867688986 container remove 59f4cd1c1e8712186b7cca291250551393e02cfdab5e334ab7139be2c8cb8ae9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-28T00:35:05.657 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:05 smithi119 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.g.service: Main process exited, code=exited, status=134/n/a 2024-03-28T00:35:05.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:35:05 smithi119 ceph-mon[31518]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-28T00:35:05.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:35:05 smithi119 ceph-mon[31518]: Cluster is now healthy 2024-03-28T00:35:05.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:35:05 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.119:6834/3517971302,v1:172.21.15.119:6835/3517971302] up:active 2024-03-28T00:35:05.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:35:05 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.119:6836/482178170,v1:172.21.15.119:6837/482178170] up:active 2024-03-28T00:35:05.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:35:05 smithi119 ceph-mon[31518]: fsmap cephfs:5 3 up:standby-replay 5 up:active 2024-03-28T00:35:05.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:35:05 smithi132 ceph-mon[35941]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-28T00:35:05.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:35:05 smithi132 ceph-mon[35941]: Cluster is now healthy 2024-03-28T00:35:05.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:35:05 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.119:6834/3517971302,v1:172.21.15.119:6835/3517971302] up:active 2024-03-28T00:35:05.705 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:35:05 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.119:6836/482178170,v1:172.21.15.119:6837/482178170] up:active 2024-03-28T00:35:05.705 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:35:05 smithi132 ceph-mon[35941]: fsmap cephfs:5 3 up:standby-replay 5 up:active 2024-03-28T00:35:06.628 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:35:06 smithi119 ceph-mon[31518]: pgmap v289: 97 pgs: 97 active+clean; 279 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 468 KiB/s rd, 385 KiB/s wr, 158 op/s 2024-03-28T00:35:06.628 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:06 smithi119 podman[118655]: 2024-03-28 00:35:06.326164174 +0000 UTC m=+1.867066913 container remove 50102c2aedd4271caac8481669f65eb8864480e98cc491d92f71fce33bdef1f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-28T00:35:06.629 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:06 smithi119 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.d.service: Main process exited, code=exited, status=134/n/a 2024-03-28T00:35:06.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:35:06 smithi165 ceph-mon[36302]: pgmap v289: 97 pgs: 97 active+clean; 279 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 468 KiB/s rd, 385 KiB/s wr, 158 op/s 2024-03-28T00:35:06.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:35:06 smithi132 ceph-mon[35941]: pgmap v289: 97 pgs: 97 active+clean; 279 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 468 KiB/s rd, 385 KiB/s wr, 158 op/s 2024-03-28T00:35:06.907 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:06 smithi119 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.d.service: Failed with result 'exit-code'. 2024-03-28T00:35:06.907 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:06 smithi119 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.d.service: Consumed 1.533s CPU time. 2024-03-28T00:35:06.907 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:06 smithi119 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.g.service: Failed with result 'exit-code'. 2024-03-28T00:35:06.907 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:06 smithi119 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.g.service: Consumed 1.600s CPU time. 2024-03-28T00:35:08.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:35:08 smithi165 ceph-mon[36302]: pgmap v290: 97 pgs: 97 active+clean; 279 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 468 KiB/s rd, 385 KiB/s wr, 158 op/s 2024-03-28T00:35:08.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:35:08 smithi119 ceph-mon[31518]: pgmap v290: 97 pgs: 97 active+clean; 279 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 468 KiB/s rd, 385 KiB/s wr, 158 op/s 2024-03-28T00:35:08.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:35:08 smithi132 ceph-mon[35941]: pgmap v290: 97 pgs: 97 active+clean; 279 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 468 KiB/s rd, 385 KiB/s wr, 158 op/s 2024-03-28T00:35:10.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:35:10 smithi165 ceph-mon[36302]: pgmap v291: 97 pgs: 97 active+clean; 286 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 452 op/s 2024-03-28T00:35:10.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:35:10 smithi119 ceph-mon[31518]: pgmap v291: 97 pgs: 97 active+clean; 286 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 452 op/s 2024-03-28T00:35:10.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:35:10 smithi132 ceph-mon[35941]: pgmap v291: 97 pgs: 97 active+clean; 286 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 452 op/s 2024-03-28T00:35:12.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:35:12 smithi165 ceph-mon[36302]: pgmap v292: 97 pgs: 97 active+clean; 286 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 987 KiB/s rd, 1.0 MiB/s wr, 397 op/s 2024-03-28T00:35:12.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:35:12 smithi119 ceph-mon[31518]: pgmap v292: 97 pgs: 97 active+clean; 286 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 987 KiB/s rd, 1.0 MiB/s wr, 397 op/s 2024-03-28T00:35:12.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:35:12 smithi132 ceph-mon[35941]: pgmap v292: 97 pgs: 97 active+clean; 286 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 987 KiB/s rd, 1.0 MiB/s wr, 397 op/s 2024-03-28T00:35:14.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:35:14 smithi165 ceph-mon[36302]: pgmap v293: 97 pgs: 97 active+clean; 286 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 935 KiB/s rd, 975 KiB/s wr, 376 op/s 2024-03-28T00:35:14.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:35:14 smithi119 ceph-mon[31518]: pgmap v293: 97 pgs: 97 active+clean; 286 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 935 KiB/s rd, 975 KiB/s wr, 376 op/s 2024-03-28T00:35:14.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:35:14 smithi132 ceph-mon[35941]: pgmap v293: 97 pgs: 97 active+clean; 286 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 935 KiB/s rd, 975 KiB/s wr, 376 op/s 2024-03-28T00:35:16.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:35:16 smithi165 ceph-mon[36302]: pgmap v294: 97 pgs: 97 active+clean; 286 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 936 KiB/s rd, 975 KiB/s wr, 380 op/s 2024-03-28T00:35:16.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:35:16 smithi119 ceph-mon[31518]: pgmap v294: 97 pgs: 97 active+clean; 286 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 936 KiB/s rd, 975 KiB/s wr, 380 op/s 2024-03-28T00:35:16.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:35:16 smithi132 ceph-mon[35941]: pgmap v294: 97 pgs: 97 active+clean; 286 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 936 KiB/s rd, 975 KiB/s wr, 380 op/s 2024-03-28T00:35:17.034 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:16 smithi119 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.g.service: Scheduled restart job, restart counter is at 4. 2024-03-28T00:35:17.034 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:16 smithi119 systemd[1]: Stopped Ceph mds.g for 10f73b92-ec98-11ee-95d0-87774f69a715. 2024-03-28T00:35:17.034 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:16 smithi119 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.g.service: Consumed 1.600s CPU time. 2024-03-28T00:35:17.034 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:16 smithi119 systemd[1]: Starting Ceph mds.g for 10f73b92-ec98-11ee-95d0-87774f69a715... 2024-03-28T00:35:17.035 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:16 smithi119 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.d.service: Scheduled restart job, restart counter is at 2. 2024-03-28T00:35:17.035 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:16 smithi119 systemd[1]: Stopped Ceph mds.d for 10f73b92-ec98-11ee-95d0-87774f69a715. 2024-03-28T00:35:17.035 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:16 smithi119 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.d.service: Consumed 1.533s CPU time. 2024-03-28T00:35:17.035 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:16 smithi119 systemd[1]: Starting Ceph mds.d for 10f73b92-ec98-11ee-95d0-87774f69a715... 2024-03-28T00:35:17.338 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:17 smithi119 podman[118794]: 2024-03-28 00:35:16.931420838 +0000 UTC m=+0.034951538 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-28T00:35:17.338 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:17 smithi119 podman[118794]: 2024-03-28 00:35:17.052051175 +0000 UTC m=+0.155581912 container create 2c561737e27ab09d4055ddd4c480209d3f3614a130b67c197792ea12e375fccf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True) 2024-03-28T00:35:17.654 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:17 smithi119 podman[118794]: 2024-03-28 00:35:17.331072273 +0000 UTC m=+0.434602989 container init 2c561737e27ab09d4055ddd4c480209d3f3614a130b67c197792ea12e375fccf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux ) 2024-03-28T00:35:17.655 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:17 smithi119 podman[118794]: 2024-03-28 00:35:17.343453917 +0000 UTC m=+0.446984677 container start 2c561737e27ab09d4055ddd4c480209d3f3614a130b67c197792ea12e375fccf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-28T00:35:17.655 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:17 smithi119 ceph-mds[118848]: -- 172.21.15.119:0/599627728 <== mon.2 v2:172.21.15.132:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55fe1c7b1a40 con 0x55fe1c784c00 2024-03-28T00:35:17.655 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:17 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: starting mds.g at 2024-03-28T00:35:17.655 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:17 smithi119 bash[118794]: 2c561737e27ab09d4055ddd4c480209d3f3614a130b67c197792ea12e375fccf 2024-03-28T00:35:17.655 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:17 smithi119 systemd[1]: Started Ceph mds.g for 10f73b92-ec98-11ee-95d0-87774f69a715. 2024-03-28T00:35:17.907 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:17 smithi119 podman[118891]: 2024-03-28 00:35:17.551966356 +0000 UTC m=+0.031554082 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-28T00:35:17.907 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:17 smithi119 podman[118891]: 2024-03-28 00:35:17.656488473 +0000 UTC m=+0.136076236 container create 532386f0ed8b2791629a5c4e2649430e291d7d7eddca11be935c3ffbbc22f643 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d, org.label-schema.build-date=20240319, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-03-28T00:35:18.296 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:17 smithi119 podman[118891]: 2024-03-28 00:35:17.918902937 +0000 UTC m=+0.398490700 container init 532386f0ed8b2791629a5c4e2649430e291d7d7eddca11be935c3ffbbc22f643 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d, io.buildah.version=1.33.5, ceph=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-03-28T00:35:18.296 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:17 smithi119 podman[118891]: 2024-03-28 00:35:17.92653726 +0000 UTC m=+0.406125020 container start 532386f0ed8b2791629a5c4e2649430e291d7d7eddca11be935c3ffbbc22f643 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-03-28T00:35:18.296 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:17 smithi119 ceph-mds[118909]: -- 172.21.15.119:0/1450487578 <== mon.2 v2:172.21.15.132:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x557afe7c5a40 con 0x557afe799000 2024-03-28T00:35:18.296 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:17 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: starting mds.d at 2024-03-28T00:35:18.296 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:18 smithi119 bash[118891]: 532386f0ed8b2791629a5c4e2649430e291d7d7eddca11be935c3ffbbc22f643 2024-03-28T00:35:18.296 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:18 smithi119 systemd[1]: Started Ceph mds.d for 10f73b92-ec98-11ee-95d0-87774f69a715. 2024-03-28T00:35:18.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:35:18 smithi165 ceph-mon[36302]: pgmap v295: 97 pgs: 97 active+clean; 286 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 706 KiB/s rd, 655 KiB/s wr, 253 op/s 2024-03-28T00:35:18.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:35:18 smithi165 ceph-mon[36302]: daemon mds.g restarted 2024-03-28T00:35:18.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:35:18 smithi165 ceph-mon[36302]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-28T00:35:18.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:35:18 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-28T00:35:18.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:35:18 smithi165 ceph-mon[36302]: osdmap e119: 12 total, 12 up, 12 in 2024-03-28T00:35:18.635 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:35:18 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.119:6834/2793569574,v1:172.21.15.119:6835/2793569574] up:boot 2024-03-28T00:35:18.635 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:35:18 smithi165 ceph-mon[36302]: Standby daemon mds.g assigned to filesystem cephfs as rank 3 2024-03-28T00:35:18.635 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:35:18 smithi165 ceph-mon[36302]: fsmap cephfs:4/5 {0=b=up:active,1=d=up:active,2=h=up:active,4=f=up:active} 3 up:standby-replay 1 up:standby, 1 failed 2024-03-28T00:35:18.635 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:35:18 smithi165 ceph-mon[36302]: fsmap cephfs:5/5 3 up:standby-replay 1 up:replay 4 up:active 2024-03-28T00:35:18.635 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:35:18 smithi165 ceph-mon[36302]: daemon mds.d restarted 2024-03-28T00:35:18.635 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:35:18 smithi165 ceph-mon[36302]: osdmap e120: 12 total, 12 up, 12 in 2024-03-28T00:35:18.635 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:35:18 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.119:6834/2793569574,v1:172.21.15.119:6835/2793569574] up:resolve 2024-03-28T00:35:18.635 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:35:18 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.119:6836/3511403986,v1:172.21.15.119:6837/3511403986] up:boot 2024-03-28T00:35:18.635 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:35:18 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-28T00:35:18.635 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:35:18 smithi165 ceph-mon[36302]: Standby daemon mds.d assigned to filesystem cephfs as rank 1 2024-03-28T00:35:18.635 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:35:18 smithi165 ceph-mon[36302]: fsmap cephfs:4/5 {0=b=up:active,2=h=up:active,3=g=up:resolve,4=f=up:active} 3 up:standby-replay 1 up:standby, 1 failed 2024-03-28T00:35:18.635 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:35:18 smithi165 ceph-mon[36302]: fsmap cephfs:5/5 3 up:standby-replay 1 up:replay 1 up:resolve 3 up:active 2024-03-28T00:35:18.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:35:18 smithi119 ceph-mon[31518]: pgmap v295: 97 pgs: 97 active+clean; 286 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 706 KiB/s rd, 655 KiB/s wr, 253 op/s 2024-03-28T00:35:18.658 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:35:18 smithi119 ceph-mon[31518]: daemon mds.g restarted 2024-03-28T00:35:18.658 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:35:18 smithi119 ceph-mon[31518]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-28T00:35:18.658 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:35:18 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-28T00:35:18.658 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:35:18 smithi119 ceph-mon[31518]: osdmap e119: 12 total, 12 up, 12 in 2024-03-28T00:35:18.658 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:35:18 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.119:6834/2793569574,v1:172.21.15.119:6835/2793569574] up:boot 2024-03-28T00:35:18.658 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:35:18 smithi119 ceph-mon[31518]: Standby daemon mds.g assigned to filesystem cephfs as rank 3 2024-03-28T00:35:18.658 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:35:18 smithi119 ceph-mon[31518]: fsmap cephfs:4/5 {0=b=up:active,1=d=up:active,2=h=up:active,4=f=up:active} 3 up:standby-replay 1 up:standby, 1 failed 2024-03-28T00:35:18.658 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:35:18 smithi119 ceph-mon[31518]: fsmap cephfs:5/5 3 up:standby-replay 1 up:replay 4 up:active 2024-03-28T00:35:18.658 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:35:18 smithi119 ceph-mon[31518]: daemon mds.d restarted 2024-03-28T00:35:18.658 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:35:18 smithi119 ceph-mon[31518]: osdmap e120: 12 total, 12 up, 12 in 2024-03-28T00:35:18.658 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:35:18 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.119:6834/2793569574,v1:172.21.15.119:6835/2793569574] up:resolve 2024-03-28T00:35:18.658 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:35:18 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.119:6836/3511403986,v1:172.21.15.119:6837/3511403986] up:boot 2024-03-28T00:35:18.658 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:35:18 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-28T00:35:18.659 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:35:18 smithi119 ceph-mon[31518]: Standby daemon mds.d assigned to filesystem cephfs as rank 1 2024-03-28T00:35:18.659 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:35:18 smithi119 ceph-mon[31518]: fsmap cephfs:4/5 {0=b=up:active,2=h=up:active,3=g=up:resolve,4=f=up:active} 3 up:standby-replay 1 up:standby, 1 failed 2024-03-28T00:35:18.659 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:35:18 smithi119 ceph-mon[31518]: fsmap cephfs:5/5 3 up:standby-replay 1 up:replay 1 up:resolve 3 up:active 2024-03-28T00:35:18.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:35:18 smithi132 ceph-mon[35941]: pgmap v295: 97 pgs: 97 active+clean; 286 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 706 KiB/s rd, 655 KiB/s wr, 253 op/s 2024-03-28T00:35:18.705 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:35:18 smithi132 ceph-mon[35941]: daemon mds.g restarted 2024-03-28T00:35:18.705 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:35:18 smithi132 ceph-mon[35941]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-28T00:35:18.705 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:35:18 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-28T00:35:18.705 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:35:18 smithi132 ceph-mon[35941]: osdmap e119: 12 total, 12 up, 12 in 2024-03-28T00:35:18.705 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:35:18 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.119:6834/2793569574,v1:172.21.15.119:6835/2793569574] up:boot 2024-03-28T00:35:18.705 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:35:18 smithi132 ceph-mon[35941]: Standby daemon mds.g assigned to filesystem cephfs as rank 3 2024-03-28T00:35:18.705 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:35:18 smithi132 ceph-mon[35941]: fsmap cephfs:4/5 {0=b=up:active,1=d=up:active,2=h=up:active,4=f=up:active} 3 up:standby-replay 1 up:standby, 1 failed 2024-03-28T00:35:18.705 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:35:18 smithi132 ceph-mon[35941]: fsmap cephfs:5/5 3 up:standby-replay 1 up:replay 4 up:active 2024-03-28T00:35:18.705 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:35:18 smithi132 ceph-mon[35941]: daemon mds.d restarted 2024-03-28T00:35:18.705 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:35:18 smithi132 ceph-mon[35941]: osdmap e120: 12 total, 12 up, 12 in 2024-03-28T00:35:18.705 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:35:18 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.119:6834/2793569574,v1:172.21.15.119:6835/2793569574] up:resolve 2024-03-28T00:35:18.705 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:35:18 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.119:6836/3511403986,v1:172.21.15.119:6837/3511403986] up:boot 2024-03-28T00:35:18.706 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:35:18 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-28T00:35:18.706 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:35:18 smithi132 ceph-mon[35941]: Standby daemon mds.d assigned to filesystem cephfs as rank 1 2024-03-28T00:35:18.706 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:35:18 smithi132 ceph-mon[35941]: fsmap cephfs:4/5 {0=b=up:active,2=h=up:active,3=g=up:resolve,4=f=up:active} 3 up:standby-replay 1 up:standby, 1 failed 2024-03-28T00:35:18.706 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:35:18 smithi132 ceph-mon[35941]: fsmap cephfs:5/5 3 up:standby-replay 1 up:replay 1 up:resolve 3 up:active 2024-03-28T00:35:20.384 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:35:19 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.119:6836/3511403986,v1:172.21.15.119:6837/3511403986] up:resolve 2024-03-28T00:35:20.384 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:35:19 smithi165 ceph-mon[36302]: fsmap cephfs:5/5 3 up:standby-replay 2 up:resolve 3 up:active 2024-03-28T00:35:20.384 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:35:19 smithi165 ceph-mon[36302]: pgmap v298: 97 pgs: 97 active+clean; 288 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 594 KiB/s rd, 324 KiB/s wr, 152 op/s 2024-03-28T00:35:20.407 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:35:19 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.119:6836/3511403986,v1:172.21.15.119:6837/3511403986] up:resolve 2024-03-28T00:35:20.407 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:35:19 smithi119 ceph-mon[31518]: fsmap cephfs:5/5 3 up:standby-replay 2 up:resolve 3 up:active 2024-03-28T00:35:20.407 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:35:19 smithi119 ceph-mon[31518]: pgmap v298: 97 pgs: 97 active+clean; 288 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 594 KiB/s rd, 324 KiB/s wr, 152 op/s 2024-03-28T00:35:20.454 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:35:19 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.119:6836/3511403986,v1:172.21.15.119:6837/3511403986] up:resolve 2024-03-28T00:35:20.455 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:35:19 smithi132 ceph-mon[35941]: fsmap cephfs:5/5 3 up:standby-replay 2 up:resolve 3 up:active 2024-03-28T00:35:20.455 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:35:19 smithi132 ceph-mon[35941]: pgmap v298: 97 pgs: 97 active+clean; 288 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 594 KiB/s rd, 324 KiB/s wr, 152 op/s 2024-03-28T00:35:21.384 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:35:20 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.119:6836/3511403986,v1:172.21.15.119:6837/3511403986] up:reconnect 2024-03-28T00:35:21.384 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:35:20 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.119:6834/2793569574,v1:172.21.15.119:6835/2793569574] up:reconnect 2024-03-28T00:35:21.384 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:35:20 smithi165 ceph-mon[36302]: fsmap cephfs:5/5 3 up:standby-replay 2 up:reconnect 3 up:active 2024-03-28T00:35:21.384 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:35:20 smithi165 ceph-mon[36302]: reconnect by client.25355 172.21.15.119:0/3071152482 after 0.00799981 2024-03-28T00:35:21.385 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:35:20 smithi165 ceph-mon[36302]: reconnect by client.25355 172.21.15.119:0/3071152482 after 0.00799981 2024-03-28T00:35:21.385 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:35:20 smithi165 ceph-mon[36302]: reconnect by client.25264 v1:172.21.15.119:0/3863511511 after 0.0169996 2024-03-28T00:35:21.385 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:35:20 smithi165 ceph-mon[36302]: reconnect by client.25264 v1:172.21.15.119:0/3863511511 after 0.0169996 2024-03-28T00:35:21.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:35:20 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.119:6836/3511403986,v1:172.21.15.119:6837/3511403986] up:reconnect 2024-03-28T00:35:21.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:35:20 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.119:6834/2793569574,v1:172.21.15.119:6835/2793569574] up:reconnect 2024-03-28T00:35:21.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:35:20 smithi119 ceph-mon[31518]: fsmap cephfs:5/5 3 up:standby-replay 2 up:reconnect 3 up:active 2024-03-28T00:35:21.407 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:35:20 smithi119 ceph-mon[31518]: reconnect by client.25355 172.21.15.119:0/3071152482 after 0.00799981 2024-03-28T00:35:21.407 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:35:20 smithi119 ceph-mon[31518]: reconnect by client.25355 172.21.15.119:0/3071152482 after 0.00799981 2024-03-28T00:35:21.407 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:35:20 smithi119 ceph-mon[31518]: reconnect by client.25264 v1:172.21.15.119:0/3863511511 after 0.0169996 2024-03-28T00:35:21.407 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:35:20 smithi119 ceph-mon[31518]: reconnect by client.25264 v1:172.21.15.119:0/3863511511 after 0.0169996 2024-03-28T00:35:21.454 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:35:20 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.119:6836/3511403986,v1:172.21.15.119:6837/3511403986] up:reconnect 2024-03-28T00:35:21.455 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:35:20 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.119:6834/2793569574,v1:172.21.15.119:6835/2793569574] up:reconnect 2024-03-28T00:35:21.455 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:35:20 smithi132 ceph-mon[35941]: fsmap cephfs:5/5 3 up:standby-replay 2 up:reconnect 3 up:active 2024-03-28T00:35:21.455 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:35:20 smithi132 ceph-mon[35941]: reconnect by client.25355 172.21.15.119:0/3071152482 after 0.00799981 2024-03-28T00:35:21.455 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:35:20 smithi132 ceph-mon[35941]: reconnect by client.25355 172.21.15.119:0/3071152482 after 0.00799981 2024-03-28T00:35:21.455 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:35:20 smithi132 ceph-mon[35941]: reconnect by client.25264 v1:172.21.15.119:0/3863511511 after 0.0169996 2024-03-28T00:35:21.455 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:35:20 smithi132 ceph-mon[35941]: reconnect by client.25264 v1:172.21.15.119:0/3863511511 after 0.0169996 2024-03-28T00:35:22.308 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.119:6836/3511403986,v1:172.21.15.119:6837/3511403986] up:rejoin 2024-03-28T00:35:22.309 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.119:6834/2793569574,v1:172.21.15.119:6835/2793569574] up:rejoin 2024-03-28T00:35:22.309 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-mon[31518]: fsmap cephfs:5/5 3 up:standby-replay 2 up:rejoin 3 up:active 2024-03-28T00:35:22.309 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-mon[31518]: daemon mds.d is now active in filesystem cephfs as rank 1 2024-03-28T00:35:22.309 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-mon[31518]: daemon mds.g is now active in filesystem cephfs as rank 3 2024-03-28T00:35:22.309 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-mon[31518]: pgmap v299: 97 pgs: 97 active+clean; 288 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 594 KiB/s rd, 324 KiB/s wr, 152 op/s 2024-03-28T00:35:22.309 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7fc6d3600700 time 2024-03-28T00:35:22.088988+0000 2024-03-28T00:35:22.309 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T00:35:22.310 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:35:22.310 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fc6deedb4a1] 2024-03-28T00:35:22.310 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7fc6deedb667] 2024-03-28T00:35:22.310 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x557afbe63223] 2024-03-28T00:35:22.310 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x557afbcdd5d6] 2024-03-28T00:35:22.310 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x557afbcddda1] 2024-03-28T00:35:22.310 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: 6: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x557afbd1209c] 2024-03-28T00:35:22.310 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x557afbd30233] 2024-03-28T00:35:22.310 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x557afbd32197] 2024-03-28T00:35:22.310 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x557afbc3111b] 2024-03-28T00:35:22.310 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x557afbc316bc] 2024-03-28T00:35:22.310 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x557afbc1b1ff] 2024-03-28T00:35:22.310 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7fc6df16d488] 2024-03-28T00:35:22.310 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: 13: (DispatchQueue::entry()+0x51f) [0x7fc6df16a60f] 2024-03-28T00:35:22.311 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fc6df23de81] 2024-03-28T00:35:22.311 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fc6ddc581ca] 2024-03-28T00:35:22.311 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: 16: clone() 2024-03-28T00:35:22.311 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: *** Caught signal (Aborted) ** 2024-03-28T00:35:22.311 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: in thread 7fc6d3600700 thread_name:ms_dispatch 2024-03-28T00:35:22.311 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7ff6c3200700 time 2024-03-28T00:35:22.080348+0000 2024-03-28T00:35:22.311 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T00:35:22.311 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:35:22.311 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7ff6cea3a4a1] 2024-03-28T00:35:22.311 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7ff6cea3a667] 2024-03-28T00:35:22.311 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55fe1a638223] 2024-03-28T00:35:22.312 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55fe1a4b25d6] 2024-03-28T00:35:22.312 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55fe1a4b2da1] 2024-03-28T00:35:22.312 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: 6: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x55fe1a4e709c] 2024-03-28T00:35:22.312 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55fe1a505233] 2024-03-28T00:35:22.312 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55fe1a507197] 2024-03-28T00:35:22.312 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55fe1a40611b] 2024-03-28T00:35:22.312 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55fe1a4066bc] 2024-03-28T00:35:22.312 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55fe1a3f01ff] 2024-03-28T00:35:22.312 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7ff6ceccc488] 2024-03-28T00:35:22.312 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: 13: (DispatchQueue::entry()+0x51f) [0x7ff6cecc960f] 2024-03-28T00:35:22.312 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7ff6ced9ce81] 2024-03-28T00:35:22.312 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7ff6cd7b71ca] 2024-03-28T00:35:22.312 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: 16: clone() 2024-03-28T00:35:22.313 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: 2024-03-28T00:35:22.089+0000 7fc6d3600700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7fc6d3600700 time 2024-03-28T00:35:22.088988+0000 2024-03-28T00:35:22.313 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T00:35:22.313 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: 2024-03-28T00:35:22.313 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:35:22.313 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fc6deedb4a1] 2024-03-28T00:35:22.313 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7fc6deedb667] 2024-03-28T00:35:22.313 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x557afbe63223] 2024-03-28T00:35:22.313 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x557afbcdd5d6] 2024-03-28T00:35:22.313 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x557afbcddda1] 2024-03-28T00:35:22.313 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: 6: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x557afbd1209c] 2024-03-28T00:35:22.313 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x557afbd30233] 2024-03-28T00:35:22.313 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x557afbd32197] 2024-03-28T00:35:22.314 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x557afbc3111b] 2024-03-28T00:35:22.314 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x557afbc316bc] 2024-03-28T00:35:22.314 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x557afbc1b1ff] 2024-03-28T00:35:22.314 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7fc6df16d488] 2024-03-28T00:35:22.314 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: 13: (DispatchQueue::entry()+0x51f) [0x7fc6df16a60f] 2024-03-28T00:35:22.314 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fc6df23de81] 2024-03-28T00:35:22.314 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fc6ddc581ca] 2024-03-28T00:35:22.314 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: 16: clone() 2024-03-28T00:35:22.314 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: 2024-03-28T00:35:22.314 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:35:22.314 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fc6ddc62d20] 2024-03-28T00:35:22.315 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: 2024-03-28T00:35:22.081+0000 7ff6c3200700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7ff6c3200700 time 2024-03-28T00:35:22.080348+0000 2024-03-28T00:35:22.315 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T00:35:22.315 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: 2024-03-28T00:35:22.315 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:35:22.315 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7ff6cea3a4a1] 2024-03-28T00:35:22.315 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7ff6cea3a667] 2024-03-28T00:35:22.315 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55fe1a638223] 2024-03-28T00:35:22.315 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55fe1a4b25d6] 2024-03-28T00:35:22.315 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55fe1a4b2da1] 2024-03-28T00:35:22.315 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: 6: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x55fe1a4e709c] 2024-03-28T00:35:22.315 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55fe1a505233] 2024-03-28T00:35:22.315 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55fe1a507197] 2024-03-28T00:35:22.315 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55fe1a40611b] 2024-03-28T00:35:22.315 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55fe1a4066bc] 2024-03-28T00:35:22.316 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55fe1a3f01ff] 2024-03-28T00:35:22.316 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7ff6ceccc488] 2024-03-28T00:35:22.316 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: 13: (DispatchQueue::entry()+0x51f) [0x7ff6cecc960f] 2024-03-28T00:35:22.316 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7ff6ced9ce81] 2024-03-28T00:35:22.316 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7ff6cd7b71ca] 2024-03-28T00:35:22.316 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: 16: clone() 2024-03-28T00:35:22.316 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: 2024-03-28T00:35:22.316 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: *** Caught signal (Aborted) ** 2024-03-28T00:35:22.316 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: in thread 7ff6c3200700 thread_name:ms_dispatch 2024-03-28T00:35:22.316 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:35:22.316 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7ff6cd7c1d20] 2024-03-28T00:35:22.316 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: 2: gsignal() 2024-03-28T00:35:22.317 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: 3: abort() 2024-03-28T00:35:22.317 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fc6deedb4fb] 2024-03-28T00:35:22.317 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7fc6deedb667] 2024-03-28T00:35:22.317 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x557afbe63223] 2024-03-28T00:35:22.317 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x557afbcdd5d6] 2024-03-28T00:35:22.317 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x557afbcddda1] 2024-03-28T00:35:22.317 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: 9: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x557afbd1209c] 2024-03-28T00:35:22.317 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x557afbd30233] 2024-03-28T00:35:22.317 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x557afbd32197] 2024-03-28T00:35:22.317 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x557afbc3111b] 2024-03-28T00:35:22.317 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x557afbc316bc] 2024-03-28T00:35:22.317 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x557afbc1b1ff] 2024-03-28T00:35:22.317 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7fc6df16d488] 2024-03-28T00:35:22.317 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: 16: (DispatchQueue::entry()+0x51f) [0x7fc6df16a60f] 2024-03-28T00:35:22.317 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fc6df23de81] 2024-03-28T00:35:22.318 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7fc6ddc581ca] 2024-03-28T00:35:22.318 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: 19: clone() 2024-03-28T00:35:22.318 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: 2024-03-28T00:35:22.092+0000 7fc6d3600700 -1 *** Caught signal (Aborted) ** 2024-03-28T00:35:22.318 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: in thread 7fc6d3600700 thread_name:ms_dispatch 2024-03-28T00:35:22.318 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: 2024-03-28T00:35:22.318 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:35:22.318 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fc6ddc62d20] 2024-03-28T00:35:22.318 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: 2: gsignal() 2024-03-28T00:35:22.318 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: 3: abort() 2024-03-28T00:35:22.318 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fc6deedb4fb] 2024-03-28T00:35:22.318 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7fc6deedb667] 2024-03-28T00:35:22.318 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: 2: gsignal() 2024-03-28T00:35:22.319 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: 3: abort() 2024-03-28T00:35:22.319 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7ff6cea3a4fb] 2024-03-28T00:35:22.319 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7ff6cea3a667] 2024-03-28T00:35:22.319 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55fe1a638223] 2024-03-28T00:35:22.319 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55fe1a4b25d6] 2024-03-28T00:35:22.319 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55fe1a4b2da1] 2024-03-28T00:35:22.319 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: 9: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x55fe1a4e709c] 2024-03-28T00:35:22.319 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55fe1a505233] 2024-03-28T00:35:22.319 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55fe1a507197] 2024-03-28T00:35:22.319 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55fe1a40611b] 2024-03-28T00:35:22.319 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55fe1a4066bc] 2024-03-28T00:35:22.319 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55fe1a3f01ff] 2024-03-28T00:35:22.319 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7ff6ceccc488] 2024-03-28T00:35:22.320 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: 16: (DispatchQueue::entry()+0x51f) [0x7ff6cecc960f] 2024-03-28T00:35:22.320 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7ff6ced9ce81] 2024-03-28T00:35:22.320 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7ff6cd7b71ca] 2024-03-28T00:35:22.320 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: 19: clone() 2024-03-28T00:35:22.320 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: 2024-03-28T00:35:22.083+0000 7ff6c3200700 -1 *** Caught signal (Aborted) ** 2024-03-28T00:35:22.320 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: in thread 7ff6c3200700 thread_name:ms_dispatch 2024-03-28T00:35:22.320 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: 2024-03-28T00:35:22.320 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:35:22.320 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7ff6cd7c1d20] 2024-03-28T00:35:22.320 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: 2: gsignal() 2024-03-28T00:35:22.320 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: 3: abort() 2024-03-28T00:35:22.320 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7ff6cea3a4fb] 2024-03-28T00:35:22.320 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7ff6cea3a667] 2024-03-28T00:35:22.321 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x557afbe63223] 2024-03-28T00:35:22.321 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x557afbcdd5d6] 2024-03-28T00:35:22.321 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x557afbcddda1] 2024-03-28T00:35:22.321 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: 9: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x557afbd1209c] 2024-03-28T00:35:22.321 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x557afbd30233] 2024-03-28T00:35:22.321 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x557afbd32197] 2024-03-28T00:35:22.321 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x557afbc3111b] 2024-03-28T00:35:22.321 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x557afbc316bc] 2024-03-28T00:35:22.321 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x557afbc1b1ff] 2024-03-28T00:35:22.321 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7fc6df16d488] 2024-03-28T00:35:22.321 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: 16: (DispatchQueue::entry()+0x51f) [0x7fc6df16a60f] 2024-03-28T00:35:22.321 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fc6df23de81] 2024-03-28T00:35:22.321 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7fc6ddc581ca] 2024-03-28T00:35:22.321 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: 19: clone() 2024-03-28T00:35:22.322 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T00:35:22.322 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: 2024-03-28T00:35:22.322 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: -1> 2024-03-28T00:35:22.089+0000 7fc6d3600700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7fc6d3600700 time 2024-03-28T00:35:22.088988+0000 2024-03-28T00:35:22.322 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T00:35:22.322 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: 2024-03-28T00:35:22.322 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:35:22.322 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fc6deedb4a1] 2024-03-28T00:35:22.322 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7fc6deedb667] 2024-03-28T00:35:22.322 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55fe1a638223] 2024-03-28T00:35:22.322 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55fe1a4b25d6] 2024-03-28T00:35:22.322 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55fe1a4b2da1] 2024-03-28T00:35:22.323 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: 9: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x55fe1a4e709c] 2024-03-28T00:35:22.323 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55fe1a505233] 2024-03-28T00:35:22.323 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55fe1a507197] 2024-03-28T00:35:22.323 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55fe1a40611b] 2024-03-28T00:35:22.323 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55fe1a4066bc] 2024-03-28T00:35:22.323 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55fe1a3f01ff] 2024-03-28T00:35:22.323 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7ff6ceccc488] 2024-03-28T00:35:22.323 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: 16: (DispatchQueue::entry()+0x51f) [0x7ff6cecc960f] 2024-03-28T00:35:22.323 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7ff6ced9ce81] 2024-03-28T00:35:22.323 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7ff6cd7b71ca] 2024-03-28T00:35:22.323 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: 19: clone() 2024-03-28T00:35:22.323 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T00:35:22.323 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: 2024-03-28T00:35:22.323 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: -1> 2024-03-28T00:35:22.081+0000 7ff6c3200700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7ff6c3200700 time 2024-03-28T00:35:22.080348+0000 2024-03-28T00:35:22.323 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T00:35:22.324 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: 2024-03-28T00:35:22.324 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:35:22.324 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7ff6cea3a4a1] 2024-03-28T00:35:22.324 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7ff6cea3a667] 2024-03-28T00:35:22.324 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x557afbe63223] 2024-03-28T00:35:22.324 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x557afbcdd5d6] 2024-03-28T00:35:22.324 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x557afbcddda1] 2024-03-28T00:35:22.324 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: 6: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x557afbd1209c] 2024-03-28T00:35:22.324 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x557afbd30233] 2024-03-28T00:35:22.324 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x557afbd32197] 2024-03-28T00:35:22.324 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x557afbc3111b] 2024-03-28T00:35:22.324 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x557afbc316bc] 2024-03-28T00:35:22.325 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x557afbc1b1ff] 2024-03-28T00:35:22.325 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7fc6df16d488] 2024-03-28T00:35:22.325 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: 13: (DispatchQueue::entry()+0x51f) [0x7fc6df16a60f] 2024-03-28T00:35:22.325 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fc6df23de81] 2024-03-28T00:35:22.325 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fc6ddc581ca] 2024-03-28T00:35:22.325 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: 16: clone() 2024-03-28T00:35:22.325 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: 2024-03-28T00:35:22.325 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: 0> 2024-03-28T00:35:22.092+0000 7fc6d3600700 -1 *** Caught signal (Aborted) ** 2024-03-28T00:35:22.325 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: in thread 7fc6d3600700 thread_name:ms_dispatch 2024-03-28T00:35:22.325 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: 2024-03-28T00:35:22.325 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:35:22.325 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fc6ddc62d20] 2024-03-28T00:35:22.325 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: 2: gsignal() 2024-03-28T00:35:22.325 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: 3: abort() 2024-03-28T00:35:22.325 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fc6deedb4fb] 2024-03-28T00:35:22.325 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7fc6deedb667] 2024-03-28T00:35:22.326 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x557afbe63223] 2024-03-28T00:35:22.326 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x557afbcdd5d6] 2024-03-28T00:35:22.326 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55fe1a638223] 2024-03-28T00:35:22.326 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55fe1a4b25d6] 2024-03-28T00:35:22.326 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55fe1a4b2da1] 2024-03-28T00:35:22.326 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: 6: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x55fe1a4e709c] 2024-03-28T00:35:22.326 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55fe1a505233] 2024-03-28T00:35:22.326 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55fe1a507197] 2024-03-28T00:35:22.326 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55fe1a40611b] 2024-03-28T00:35:22.326 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55fe1a4066bc] 2024-03-28T00:35:22.326 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55fe1a3f01ff] 2024-03-28T00:35:22.327 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7ff6ceccc488] 2024-03-28T00:35:22.327 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: 13: (DispatchQueue::entry()+0x51f) [0x7ff6cecc960f] 2024-03-28T00:35:22.327 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7ff6ced9ce81] 2024-03-28T00:35:22.327 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7ff6cd7b71ca] 2024-03-28T00:35:22.327 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: 16: clone() 2024-03-28T00:35:22.327 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: 2024-03-28T00:35:22.327 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: 0> 2024-03-28T00:35:22.083+0000 7ff6c3200700 -1 *** Caught signal (Aborted) ** 2024-03-28T00:35:22.327 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: in thread 7ff6c3200700 thread_name:ms_dispatch 2024-03-28T00:35:22.327 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: 2024-03-28T00:35:22.327 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:35:22.327 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7ff6cd7c1d20] 2024-03-28T00:35:22.327 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: 2: gsignal() 2024-03-28T00:35:22.327 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: 3: abort() 2024-03-28T00:35:22.327 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7ff6cea3a4fb] 2024-03-28T00:35:22.327 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7ff6cea3a667] 2024-03-28T00:35:22.328 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55fe1a638223] 2024-03-28T00:35:22.328 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55fe1a4b25d6] 2024-03-28T00:35:22.328 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x557afbcddda1] 2024-03-28T00:35:22.328 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: 9: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x557afbd1209c] 2024-03-28T00:35:22.328 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x557afbd30233] 2024-03-28T00:35:22.328 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x557afbd32197] 2024-03-28T00:35:22.328 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x557afbc3111b] 2024-03-28T00:35:22.328 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x557afbc316bc] 2024-03-28T00:35:22.328 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x557afbc1b1ff] 2024-03-28T00:35:22.328 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7fc6df16d488] 2024-03-28T00:35:22.328 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: 16: (DispatchQueue::entry()+0x51f) [0x7fc6df16a60f] 2024-03-28T00:35:22.328 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fc6df23de81] 2024-03-28T00:35:22.329 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7fc6ddc581ca] 2024-03-28T00:35:22.329 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: 19: clone() 2024-03-28T00:35:22.329 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T00:35:22.329 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: 2024-03-28T00:35:22.329 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: -1> 2024-03-28T00:35:22.089+0000 7fc6d3600700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7fc6d3600700 time 2024-03-28T00:35:22.088988+0000 2024-03-28T00:35:22.329 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T00:35:22.329 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: 2024-03-28T00:35:22.329 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:35:22.329 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fc6deedb4a1] 2024-03-28T00:35:22.329 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7fc6deedb667] 2024-03-28T00:35:22.329 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x557afbe63223] 2024-03-28T00:35:22.329 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x557afbcdd5d6] 2024-03-28T00:35:22.330 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55fe1a4b2da1] 2024-03-28T00:35:22.330 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: 9: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x55fe1a4e709c] 2024-03-28T00:35:22.330 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55fe1a505233] 2024-03-28T00:35:22.330 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55fe1a507197] 2024-03-28T00:35:22.330 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55fe1a40611b] 2024-03-28T00:35:22.330 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55fe1a4066bc] 2024-03-28T00:35:22.330 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55fe1a3f01ff] 2024-03-28T00:35:22.330 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7ff6ceccc488] 2024-03-28T00:35:22.330 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: 16: (DispatchQueue::entry()+0x51f) [0x7ff6cecc960f] 2024-03-28T00:35:22.330 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7ff6ced9ce81] 2024-03-28T00:35:22.330 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7ff6cd7b71ca] 2024-03-28T00:35:22.330 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: 19: clone() 2024-03-28T00:35:22.330 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T00:35:22.330 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: 2024-03-28T00:35:22.330 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: -1> 2024-03-28T00:35:22.081+0000 7ff6c3200700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7ff6c3200700 time 2024-03-28T00:35:22.080348+0000 2024-03-28T00:35:22.331 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T00:35:22.331 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: 2024-03-28T00:35:22.331 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:35:22.331 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7ff6cea3a4a1] 2024-03-28T00:35:22.331 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7ff6cea3a667] 2024-03-28T00:35:22.331 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55fe1a638223] 2024-03-28T00:35:22.331 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55fe1a4b25d6] 2024-03-28T00:35:22.331 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x557afbcddda1] 2024-03-28T00:35:22.331 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: 6: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x557afbd1209c] 2024-03-28T00:35:22.331 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x557afbd30233] 2024-03-28T00:35:22.331 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x557afbd32197] 2024-03-28T00:35:22.332 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x557afbc3111b] 2024-03-28T00:35:22.332 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x557afbc316bc] 2024-03-28T00:35:22.332 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x557afbc1b1ff] 2024-03-28T00:35:22.332 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7fc6df16d488] 2024-03-28T00:35:22.332 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: 13: (DispatchQueue::entry()+0x51f) [0x7fc6df16a60f] 2024-03-28T00:35:22.332 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fc6df23de81] 2024-03-28T00:35:22.332 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fc6ddc581ca] 2024-03-28T00:35:22.332 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: 16: clone() 2024-03-28T00:35:22.332 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: 2024-03-28T00:35:22.332 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: 0> 2024-03-28T00:35:22.092+0000 7fc6d3600700 -1 *** Caught signal (Aborted) ** 2024-03-28T00:35:22.332 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: in thread 7fc6d3600700 thread_name:ms_dispatch 2024-03-28T00:35:22.332 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: 2024-03-28T00:35:22.332 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:35:22.332 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fc6ddc62d20] 2024-03-28T00:35:22.332 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: 2: gsignal() 2024-03-28T00:35:22.332 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: 3: abort() 2024-03-28T00:35:22.333 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fc6deedb4fb] 2024-03-28T00:35:22.333 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7fc6deedb667] 2024-03-28T00:35:22.333 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x557afbe63223] 2024-03-28T00:35:22.333 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x557afbcdd5d6] 2024-03-28T00:35:22.333 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x557afbcddda1] 2024-03-28T00:35:22.333 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: 9: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x557afbd1209c] 2024-03-28T00:35:22.333 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55fe1a4b2da1] 2024-03-28T00:35:22.333 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: 6: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x55fe1a4e709c] 2024-03-28T00:35:22.333 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55fe1a505233] 2024-03-28T00:35:22.333 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55fe1a507197] 2024-03-28T00:35:22.333 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55fe1a40611b] 2024-03-28T00:35:22.333 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55fe1a4066bc] 2024-03-28T00:35:22.334 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55fe1a3f01ff] 2024-03-28T00:35:22.334 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7ff6ceccc488] 2024-03-28T00:35:22.334 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: 13: (DispatchQueue::entry()+0x51f) [0x7ff6cecc960f] 2024-03-28T00:35:22.334 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7ff6ced9ce81] 2024-03-28T00:35:22.334 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7ff6cd7b71ca] 2024-03-28T00:35:22.334 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: 16: clone() 2024-03-28T00:35:22.334 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: 2024-03-28T00:35:22.334 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: 0> 2024-03-28T00:35:22.083+0000 7ff6c3200700 -1 *** Caught signal (Aborted) ** 2024-03-28T00:35:22.334 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: in thread 7ff6c3200700 thread_name:ms_dispatch 2024-03-28T00:35:22.334 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: 2024-03-28T00:35:22.334 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:35:22.334 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7ff6cd7c1d20] 2024-03-28T00:35:22.335 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: 2: gsignal() 2024-03-28T00:35:22.335 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: 3: abort() 2024-03-28T00:35:22.335 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7ff6cea3a4fb] 2024-03-28T00:35:22.335 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7ff6cea3a667] 2024-03-28T00:35:22.335 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55fe1a638223] 2024-03-28T00:35:22.335 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55fe1a4b25d6] 2024-03-28T00:35:22.335 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55fe1a4b2da1] 2024-03-28T00:35:22.335 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: 9: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x55fe1a4e709c] 2024-03-28T00:35:22.335 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x557afbd30233] 2024-03-28T00:35:22.335 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x557afbd32197] 2024-03-28T00:35:22.335 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x557afbc3111b] 2024-03-28T00:35:22.335 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x557afbc316bc] 2024-03-28T00:35:22.336 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x557afbc1b1ff] 2024-03-28T00:35:22.336 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7fc6df16d488] 2024-03-28T00:35:22.336 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: 16: (DispatchQueue::entry()+0x51f) [0x7fc6df16a60f] 2024-03-28T00:35:22.336 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fc6df23de81] 2024-03-28T00:35:22.336 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7fc6ddc581ca] 2024-03-28T00:35:22.336 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: 19: clone() 2024-03-28T00:35:22.336 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T00:35:22.336 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[118905]: 2024-03-28T00:35:22.336 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55fe1a505233] 2024-03-28T00:35:22.336 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55fe1a507197] 2024-03-28T00:35:22.336 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55fe1a40611b] 2024-03-28T00:35:22.336 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55fe1a4066bc] 2024-03-28T00:35:22.336 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55fe1a3f01ff] 2024-03-28T00:35:22.337 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7ff6ceccc488] 2024-03-28T00:35:22.337 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: 16: (DispatchQueue::entry()+0x51f) [0x7ff6cecc960f] 2024-03-28T00:35:22.337 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7ff6ced9ce81] 2024-03-28T00:35:22.337 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7ff6cd7b71ca] 2024-03-28T00:35:22.337 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: 19: clone() 2024-03-28T00:35:22.337 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T00:35:22.337 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[118840]: 2024-03-28T00:35:22.337 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 podman[119340]: 2024-03-28 00:35:22.220106027 +0000 UTC m=+0.038595285 container died 2c561737e27ab09d4055ddd4c480209d3f3614a130b67c197792ea12e375fccf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-28T00:35:22.384 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:35:22 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.119:6836/3511403986,v1:172.21.15.119:6837/3511403986] up:rejoin 2024-03-28T00:35:22.384 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:35:22 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.119:6834/2793569574,v1:172.21.15.119:6835/2793569574] up:rejoin 2024-03-28T00:35:22.384 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:35:22 smithi165 ceph-mon[36302]: fsmap cephfs:5/5 3 up:standby-replay 2 up:rejoin 3 up:active 2024-03-28T00:35:22.384 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:35:22 smithi165 ceph-mon[36302]: daemon mds.d is now active in filesystem cephfs as rank 1 2024-03-28T00:35:22.384 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:35:22 smithi165 ceph-mon[36302]: daemon mds.g is now active in filesystem cephfs as rank 3 2024-03-28T00:35:22.384 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:35:22 smithi165 ceph-mon[36302]: pgmap v299: 97 pgs: 97 active+clean; 288 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 594 KiB/s rd, 324 KiB/s wr, 152 op/s 2024-03-28T00:35:22.454 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:35:22 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.119:6836/3511403986,v1:172.21.15.119:6837/3511403986] up:rejoin 2024-03-28T00:35:22.454 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:35:22 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.119:6834/2793569574,v1:172.21.15.119:6835/2793569574] up:rejoin 2024-03-28T00:35:22.454 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:35:22 smithi132 ceph-mon[35941]: fsmap cephfs:5/5 3 up:standby-replay 2 up:rejoin 3 up:active 2024-03-28T00:35:22.454 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:35:22 smithi132 ceph-mon[35941]: daemon mds.d is now active in filesystem cephfs as rank 1 2024-03-28T00:35:22.454 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:35:22 smithi132 ceph-mon[35941]: daemon mds.g is now active in filesystem cephfs as rank 3 2024-03-28T00:35:22.455 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:35:22 smithi132 ceph-mon[35941]: pgmap v299: 97 pgs: 97 active+clean; 288 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 594 KiB/s rd, 324 KiB/s wr, 152 op/s 2024-03-28T00:35:23.384 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:35:23 smithi165 ceph-mon[36302]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-28T00:35:23.384 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:35:23 smithi165 ceph-mon[36302]: Cluster is now healthy 2024-03-28T00:35:23.384 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:35:23 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.119:6836/3511403986,v1:172.21.15.119:6837/3511403986] up:active 2024-03-28T00:35:23.384 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:35:23 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.119:6834/2793569574,v1:172.21.15.119:6835/2793569574] up:active 2024-03-28T00:35:23.384 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:35:23 smithi165 ceph-mon[36302]: fsmap cephfs:5 3 up:standby-replay 5 up:active 2024-03-28T00:35:23.406 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:23 smithi119 podman[119345]: 2024-03-28 00:35:23.091984353 +0000 UTC m=+0.898296809 container died 532386f0ed8b2791629a5c4e2649430e291d7d7eddca11be935c3ffbbc22f643 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-03-28T00:35:23.407 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:22 smithi119 podman[119340]: 2024-03-28 00:35:22.999564023 +0000 UTC m=+0.818053342 container remove 2c561737e27ab09d4055ddd4c480209d3f3614a130b67c197792ea12e375fccf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-03-28T00:35:23.407 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:23 smithi119 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.g.service: Main process exited, code=exited, status=134/n/a 2024-03-28T00:35:23.407 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:35:23 smithi119 ceph-mon[31518]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-28T00:35:23.407 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:35:23 smithi119 ceph-mon[31518]: Cluster is now healthy 2024-03-28T00:35:23.407 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:35:23 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.119:6836/3511403986,v1:172.21.15.119:6837/3511403986] up:active 2024-03-28T00:35:23.407 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:35:23 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.119:6834/2793569574,v1:172.21.15.119:6835/2793569574] up:active 2024-03-28T00:35:23.407 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:35:23 smithi119 ceph-mon[31518]: fsmap cephfs:5 3 up:standby-replay 5 up:active 2024-03-28T00:35:23.454 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:35:23 smithi132 ceph-mon[35941]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-28T00:35:23.454 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:35:23 smithi132 ceph-mon[35941]: Cluster is now healthy 2024-03-28T00:35:23.454 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:35:23 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.119:6836/3511403986,v1:172.21.15.119:6837/3511403986] up:active 2024-03-28T00:35:23.454 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:35:23 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.119:6834/2793569574,v1:172.21.15.119:6835/2793569574] up:active 2024-03-28T00:35:23.454 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:35:23 smithi132 ceph-mon[35941]: fsmap cephfs:5 3 up:standby-replay 5 up:active 2024-03-28T00:35:23.869 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:23 smithi119 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.g.service: Failed with result 'exit-code'. 2024-03-28T00:35:23.869 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:23 smithi119 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.g.service: Consumed 1.677s CPU time. 2024-03-28T00:35:24.157 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:23 smithi119 podman[119345]: 2024-03-28 00:35:23.885599712 +0000 UTC m=+1.691912199 container remove 532386f0ed8b2791629a5c4e2649430e291d7d7eddca11be935c3ffbbc22f643 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-28T00:35:24.157 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:23 smithi119 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.d.service: Main process exited, code=exited, status=134/n/a 2024-03-28T00:35:24.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:35:24 smithi119 ceph-mon[31518]: pgmap v300: 97 pgs: 97 active+clean; 288 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 592 KiB/s rd, 324 KiB/s wr, 147 op/s 2024-03-28T00:35:24.384 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:35:24 smithi165 ceph-mon[36302]: pgmap v300: 97 pgs: 97 active+clean; 288 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 592 KiB/s rd, 324 KiB/s wr, 147 op/s 2024-03-28T00:35:24.454 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:35:24 smithi132 ceph-mon[35941]: pgmap v300: 97 pgs: 97 active+clean; 288 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 592 KiB/s rd, 324 KiB/s wr, 147 op/s 2024-03-28T00:35:24.907 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:24 smithi119 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.d.service: Failed with result 'exit-code'. 2024-03-28T00:35:24.907 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:24 smithi119 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.d.service: Consumed 1.564s CPU time. 2024-03-28T00:35:26.384 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:35:26 smithi165 ceph-mon[36302]: osdmap e121: 12 total, 12 up, 12 in 2024-03-28T00:35:26.384 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:35:26 smithi165 ceph-mon[36302]: pgmap v302: 97 pgs: 97 active+clean; 293 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 998 KiB/s wr, 241 op/s 2024-03-28T00:35:26.407 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:35:26 smithi119 ceph-mon[31518]: osdmap e121: 12 total, 12 up, 12 in 2024-03-28T00:35:26.407 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:35:26 smithi119 ceph-mon[31518]: pgmap v302: 97 pgs: 97 active+clean; 293 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 998 KiB/s wr, 241 op/s 2024-03-28T00:35:26.454 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:35:26 smithi132 ceph-mon[35941]: osdmap e121: 12 total, 12 up, 12 in 2024-03-28T00:35:26.454 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:35:26 smithi132 ceph-mon[35941]: pgmap v302: 97 pgs: 97 active+clean; 293 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 998 KiB/s wr, 241 op/s 2024-03-28T00:35:28.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:35:28 smithi165 ceph-mon[36302]: pgmap v303: 97 pgs: 97 active+clean; 293 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 845 KiB/s wr, 204 op/s 2024-03-28T00:35:28.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:35:28 smithi119 ceph-mon[31518]: pgmap v303: 97 pgs: 97 active+clean; 293 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 845 KiB/s wr, 204 op/s 2024-03-28T00:35:28.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:35:28 smithi132 ceph-mon[35941]: pgmap v303: 97 pgs: 97 active+clean; 293 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 845 KiB/s wr, 204 op/s 2024-03-28T00:35:30.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:35:30 smithi165 ceph-mon[36302]: pgmap v304: 97 pgs: 97 active+clean; 293 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 484 KiB/s rd, 521 KiB/s wr, 74 op/s 2024-03-28T00:35:30.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:35:30 smithi165 ceph-mon[36302]: osdmap e122: 12 total, 12 up, 12 in 2024-03-28T00:35:30.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:35:30 smithi119 ceph-mon[31518]: pgmap v304: 97 pgs: 97 active+clean; 293 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 484 KiB/s rd, 521 KiB/s wr, 74 op/s 2024-03-28T00:35:30.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:35:30 smithi119 ceph-mon[31518]: osdmap e122: 12 total, 12 up, 12 in 2024-03-28T00:35:30.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:35:30 smithi132 ceph-mon[35941]: pgmap v304: 97 pgs: 97 active+clean; 293 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 484 KiB/s rd, 521 KiB/s wr, 74 op/s 2024-03-28T00:35:30.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:35:30 smithi132 ceph-mon[35941]: osdmap e122: 12 total, 12 up, 12 in 2024-03-28T00:35:32.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:35:32 smithi165 ceph-mon[36302]: pgmap v306: 97 pgs: 97 active+clean; 293 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 605 KiB/s rd, 651 KiB/s wr, 93 op/s 2024-03-28T00:35:32.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:35:32 smithi119 ceph-mon[31518]: pgmap v306: 97 pgs: 97 active+clean; 293 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 605 KiB/s rd, 651 KiB/s wr, 93 op/s 2024-03-28T00:35:32.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:35:32 smithi132 ceph-mon[35941]: pgmap v306: 97 pgs: 97 active+clean; 293 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 605 KiB/s rd, 651 KiB/s wr, 93 op/s 2024-03-28T00:35:34.288 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:33 smithi119 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.g.service: Scheduled restart job, restart counter is at 5. 2024-03-28T00:35:34.289 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:33 smithi119 systemd[1]: Stopped Ceph mds.g for 10f73b92-ec98-11ee-95d0-87774f69a715. 2024-03-28T00:35:34.289 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:33 smithi119 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.g.service: Consumed 1.677s CPU time. 2024-03-28T00:35:34.289 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:33 smithi119 systemd[1]: Starting Ceph mds.g for 10f73b92-ec98-11ee-95d0-87774f69a715... 2024-03-28T00:35:34.544 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:34 smithi119 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.d.service: Scheduled restart job, restart counter is at 3. 2024-03-28T00:35:34.544 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:34 smithi119 systemd[1]: Stopped Ceph mds.d for 10f73b92-ec98-11ee-95d0-87774f69a715. 2024-03-28T00:35:34.544 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:34 smithi119 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.d.service: Consumed 1.564s CPU time. 2024-03-28T00:35:34.544 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:34 smithi119 systemd[1]: Starting Ceph mds.d for 10f73b92-ec98-11ee-95d0-87774f69a715... 2024-03-28T00:35:34.544 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:35:34 smithi119 ceph-mon[31518]: pgmap v307: 97 pgs: 97 active+clean; 293 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 125 B/s wr, 5 op/s 2024-03-28T00:35:34.544 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:34 smithi119 podman[119731]: 2024-03-28 00:35:34.190182462 +0000 UTC m=+0.033055203 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-28T00:35:34.545 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:34 smithi119 podman[119731]: 2024-03-28 00:35:34.323608049 +0000 UTC m=+0.166480827 container create 8619117ec89b37e78705aea709041738e1dd460a60582688981732aef41f8f91 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.build-date=20240319, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-03-28T00:35:34.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:35:34 smithi165 ceph-mon[36302]: pgmap v307: 97 pgs: 97 active+clean; 293 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 125 B/s wr, 5 op/s 2024-03-28T00:35:34.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:35:34 smithi132 ceph-mon[35941]: pgmap v307: 97 pgs: 97 active+clean; 293 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 125 B/s wr, 5 op/s 2024-03-28T00:35:34.907 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:34 smithi119 podman[119731]: 2024-03-28 00:35:34.617847873 +0000 UTC m=+0.460720627 container init 8619117ec89b37e78705aea709041738e1dd460a60582688981732aef41f8f91 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-28T00:35:34.907 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:34 smithi119 podman[119731]: 2024-03-28 00:35:34.627757235 +0000 UTC m=+0.470630004 container start 8619117ec89b37e78705aea709041738e1dd460a60582688981732aef41f8f91 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g, io.buildah.version=1.33.5, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-03-28T00:35:34.907 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:34 smithi119 ceph-mds[119763]: -- 172.21.15.119:0/958098665 <== mon.1 v2:172.21.15.165:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x559090b13a40 con 0x559090ae6800 2024-03-28T00:35:34.907 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:34 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: starting mds.g at 2024-03-28T00:35:34.908 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:34 smithi119 bash[119731]: 8619117ec89b37e78705aea709041738e1dd460a60582688981732aef41f8f91 2024-03-28T00:35:34.908 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:34 smithi119 systemd[1]: Started Ceph mds.g for 10f73b92-ec98-11ee-95d0-87774f69a715. 2024-03-28T00:35:35.256 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:34 smithi119 podman[119849]: 2024-03-28 00:35:34.89747358 +0000 UTC m=+0.032337171 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-28T00:35:35.256 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:35 smithi119 podman[119849]: 2024-03-28 00:35:35.010420109 +0000 UTC m=+0.145283758 container create 450ac508eef1f0f0c4b03570e6e529d736fa2801a2d5ec065a5d3dcd6ef2cfc8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-03-28T00:35:35.256 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:35 smithi119 podman[119849]: 2024-03-28 00:35:35.238904889 +0000 UTC m=+0.373768512 container init 450ac508eef1f0f0c4b03570e6e529d736fa2801a2d5ec065a5d3dcd6ef2cfc8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-03-28T00:35:35.257 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:35 smithi119 podman[119849]: 2024-03-28 00:35:35.248677802 +0000 UTC m=+0.383541448 container start 450ac508eef1f0f0c4b03570e6e529d736fa2801a2d5ec065a5d3dcd6ef2cfc8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d, io.buildah.version=1.33.5, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319) 2024-03-28T00:35:35.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:35:35 smithi165 ceph-mon[36302]: daemon mds.g restarted 2024-03-28T00:35:35.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:35:35 smithi165 ceph-mon[36302]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-28T00:35:35.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:35:35 smithi165 ceph-mon[36302]: osdmap e123: 12 total, 12 up, 12 in 2024-03-28T00:35:35.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:35:35 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.119:6834/1945411848,v1:172.21.15.119:6835/1945411848] up:boot 2024-03-28T00:35:35.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:35:35 smithi165 ceph-mon[36302]: Standby daemon mds.g assigned to filesystem cephfs as rank 3 2024-03-28T00:35:35.635 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:35:35 smithi165 ceph-mon[36302]: fsmap cephfs:4/5 {0=b=up:active,1=d=up:active,2=h=up:active,4=f=up:active} 3 up:standby-replay 1 up:standby, 1 failed 2024-03-28T00:35:35.635 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:35:35 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-28T00:35:35.635 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:35:35 smithi165 ceph-mon[36302]: fsmap cephfs:5/5 3 up:standby-replay 1 up:replay 4 up:active 2024-03-28T00:35:35.657 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:35 smithi119 ceph-mds[119894]: -- 172.21.15.119:0/3912536266 <== mon.1 v2:172.21.15.165:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55bec4771a40 con 0x55bec4744c00 2024-03-28T00:35:35.657 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:35 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: starting mds.d at 2024-03-28T00:35:35.657 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:35 smithi119 bash[119849]: 450ac508eef1f0f0c4b03570e6e529d736fa2801a2d5ec065a5d3dcd6ef2cfc8 2024-03-28T00:35:35.658 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:35 smithi119 systemd[1]: Started Ceph mds.d for 10f73b92-ec98-11ee-95d0-87774f69a715. 2024-03-28T00:35:35.658 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:35:35 smithi119 ceph-mon[31518]: daemon mds.g restarted 2024-03-28T00:35:35.658 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:35:35 smithi119 ceph-mon[31518]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-28T00:35:35.658 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:35:35 smithi119 ceph-mon[31518]: osdmap e123: 12 total, 12 up, 12 in 2024-03-28T00:35:35.658 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:35:35 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.119:6834/1945411848,v1:172.21.15.119:6835/1945411848] up:boot 2024-03-28T00:35:35.658 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:35:35 smithi119 ceph-mon[31518]: Standby daemon mds.g assigned to filesystem cephfs as rank 3 2024-03-28T00:35:35.658 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:35:35 smithi119 ceph-mon[31518]: fsmap cephfs:4/5 {0=b=up:active,1=d=up:active,2=h=up:active,4=f=up:active} 3 up:standby-replay 1 up:standby, 1 failed 2024-03-28T00:35:35.658 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:35:35 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-28T00:35:35.658 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:35:35 smithi119 ceph-mon[31518]: fsmap cephfs:5/5 3 up:standby-replay 1 up:replay 4 up:active 2024-03-28T00:35:35.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:35:35 smithi132 ceph-mon[35941]: daemon mds.g restarted 2024-03-28T00:35:35.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:35:35 smithi132 ceph-mon[35941]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-28T00:35:35.705 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:35:35 smithi132 ceph-mon[35941]: osdmap e123: 12 total, 12 up, 12 in 2024-03-28T00:35:35.705 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:35:35 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.119:6834/1945411848,v1:172.21.15.119:6835/1945411848] up:boot 2024-03-28T00:35:35.705 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:35:35 smithi132 ceph-mon[35941]: Standby daemon mds.g assigned to filesystem cephfs as rank 3 2024-03-28T00:35:35.705 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:35:35 smithi132 ceph-mon[35941]: fsmap cephfs:4/5 {0=b=up:active,1=d=up:active,2=h=up:active,4=f=up:active} 3 up:standby-replay 1 up:standby, 1 failed 2024-03-28T00:35:35.705 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:35:35 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-28T00:35:35.705 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:35:35 smithi132 ceph-mon[35941]: fsmap cephfs:5/5 3 up:standby-replay 1 up:replay 4 up:active 2024-03-28T00:35:36.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:35:36 smithi165 ceph-mon[36302]: pgmap v309: 97 pgs: 97 active+clean; 293 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 127 B/s wr, 10 op/s 2024-03-28T00:35:36.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:35:36 smithi165 ceph-mon[36302]: daemon mds.d restarted 2024-03-28T00:35:36.635 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:35:36 smithi165 ceph-mon[36302]: osdmap e124: 12 total, 12 up, 12 in 2024-03-28T00:35:36.635 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:35:36 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.119:6834/1945411848,v1:172.21.15.119:6835/1945411848] up:resolve 2024-03-28T00:35:36.635 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:35:36 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.119:6836/2109630829,v1:172.21.15.119:6837/2109630829] up:boot 2024-03-28T00:35:36.635 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:35:36 smithi165 ceph-mon[36302]: Standby daemon mds.d assigned to filesystem cephfs as rank 1 2024-03-28T00:35:36.635 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:35:36 smithi165 ceph-mon[36302]: fsmap cephfs:4/5 {0=b=up:active,2=h=up:active,3=g=up:resolve,4=f=up:active} 3 up:standby-replay 1 up:standby, 1 failed 2024-03-28T00:35:36.635 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:35:36 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-28T00:35:36.635 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:35:36 smithi165 ceph-mon[36302]: fsmap cephfs:5/5 3 up:standby-replay 1 up:replay 1 up:resolve 3 up:active 2024-03-28T00:35:36.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:35:36 smithi119 ceph-mon[31518]: pgmap v309: 97 pgs: 97 active+clean; 293 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 127 B/s wr, 10 op/s 2024-03-28T00:35:36.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:35:36 smithi119 ceph-mon[31518]: daemon mds.d restarted 2024-03-28T00:35:36.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:35:36 smithi119 ceph-mon[31518]: osdmap e124: 12 total, 12 up, 12 in 2024-03-28T00:35:36.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:35:36 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.119:6834/1945411848,v1:172.21.15.119:6835/1945411848] up:resolve 2024-03-28T00:35:36.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:35:36 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.119:6836/2109630829,v1:172.21.15.119:6837/2109630829] up:boot 2024-03-28T00:35:36.658 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:35:36 smithi119 ceph-mon[31518]: Standby daemon mds.d assigned to filesystem cephfs as rank 1 2024-03-28T00:35:36.658 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:35:36 smithi119 ceph-mon[31518]: fsmap cephfs:4/5 {0=b=up:active,2=h=up:active,3=g=up:resolve,4=f=up:active} 3 up:standby-replay 1 up:standby, 1 failed 2024-03-28T00:35:36.658 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:35:36 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-28T00:35:36.658 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:35:36 smithi119 ceph-mon[31518]: fsmap cephfs:5/5 3 up:standby-replay 1 up:replay 1 up:resolve 3 up:active 2024-03-28T00:35:36.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:35:36 smithi132 ceph-mon[35941]: pgmap v309: 97 pgs: 97 active+clean; 293 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 127 B/s wr, 10 op/s 2024-03-28T00:35:36.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:35:36 smithi132 ceph-mon[35941]: daemon mds.d restarted 2024-03-28T00:35:36.705 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:35:36 smithi132 ceph-mon[35941]: osdmap e124: 12 total, 12 up, 12 in 2024-03-28T00:35:36.705 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:35:36 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.119:6834/1945411848,v1:172.21.15.119:6835/1945411848] up:resolve 2024-03-28T00:35:36.705 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:35:36 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.119:6836/2109630829,v1:172.21.15.119:6837/2109630829] up:boot 2024-03-28T00:35:36.705 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:35:36 smithi132 ceph-mon[35941]: Standby daemon mds.d assigned to filesystem cephfs as rank 1 2024-03-28T00:35:36.705 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:35:36 smithi132 ceph-mon[35941]: fsmap cephfs:4/5 {0=b=up:active,2=h=up:active,3=g=up:resolve,4=f=up:active} 3 up:standby-replay 1 up:standby, 1 failed 2024-03-28T00:35:36.705 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:35:36 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-28T00:35:36.705 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:35:36 smithi132 ceph-mon[35941]: fsmap cephfs:5/5 3 up:standby-replay 1 up:replay 1 up:resolve 3 up:active 2024-03-28T00:35:37.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:35:37 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.119:6836/2109630829,v1:172.21.15.119:6837/2109630829] up:resolve 2024-03-28T00:35:37.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:35:37 smithi165 ceph-mon[36302]: fsmap cephfs:5/5 3 up:standby-replay 2 up:resolve 3 up:active 2024-03-28T00:35:37.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:35:37 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.119:6836/2109630829,v1:172.21.15.119:6837/2109630829] up:resolve 2024-03-28T00:35:37.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:35:37 smithi119 ceph-mon[31518]: fsmap cephfs:5/5 3 up:standby-replay 2 up:resolve 3 up:active 2024-03-28T00:35:37.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:35:37 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.119:6836/2109630829,v1:172.21.15.119:6837/2109630829] up:resolve 2024-03-28T00:35:37.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:35:37 smithi132 ceph-mon[35941]: fsmap cephfs:5/5 3 up:standby-replay 2 up:resolve 3 up:active 2024-03-28T00:35:38.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:35:38 smithi165 ceph-mon[36302]: pgmap v311: 97 pgs: 97 active+clean; 293 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 5 op/s 2024-03-28T00:35:38.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:35:38 smithi165 ceph-mon[36302]: reconnect by client.25355 172.21.15.119:0/3071152482 after 0.00299993 2024-03-28T00:35:38.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:35:38 smithi165 ceph-mon[36302]: reconnect by client.25355 172.21.15.119:0/3071152482 after 0.00199995 2024-03-28T00:35:38.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:35:38 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.119:6836/2109630829,v1:172.21.15.119:6837/2109630829] up:reconnect 2024-03-28T00:35:38.635 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:35:38 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.119:6834/1945411848,v1:172.21.15.119:6835/1945411848] up:reconnect 2024-03-28T00:35:38.635 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:35:38 smithi165 ceph-mon[36302]: fsmap cephfs:5/5 3 up:standby-replay 2 up:reconnect 3 up:active 2024-03-28T00:35:38.635 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:35:38 smithi165 ceph-mon[36302]: reconnect by client.25264 v1:172.21.15.119:0/3863511511 after 0.0179996 2024-03-28T00:35:38.635 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:35:38 smithi165 ceph-mon[36302]: reconnect by client.25264 v1:172.21.15.119:0/3863511511 after 0.0169996 2024-03-28T00:35:38.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:35:38 smithi119 ceph-mon[31518]: pgmap v311: 97 pgs: 97 active+clean; 293 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 5 op/s 2024-03-28T00:35:38.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:35:38 smithi119 ceph-mon[31518]: reconnect by client.25355 172.21.15.119:0/3071152482 after 0.00299993 2024-03-28T00:35:38.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:35:38 smithi119 ceph-mon[31518]: reconnect by client.25355 172.21.15.119:0/3071152482 after 0.00199995 2024-03-28T00:35:38.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:35:38 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.119:6836/2109630829,v1:172.21.15.119:6837/2109630829] up:reconnect 2024-03-28T00:35:38.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:35:38 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.119:6834/1945411848,v1:172.21.15.119:6835/1945411848] up:reconnect 2024-03-28T00:35:38.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:35:38 smithi119 ceph-mon[31518]: fsmap cephfs:5/5 3 up:standby-replay 2 up:reconnect 3 up:active 2024-03-28T00:35:38.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:35:38 smithi119 ceph-mon[31518]: reconnect by client.25264 v1:172.21.15.119:0/3863511511 after 0.0179996 2024-03-28T00:35:38.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:35:38 smithi119 ceph-mon[31518]: reconnect by client.25264 v1:172.21.15.119:0/3863511511 after 0.0169996 2024-03-28T00:35:38.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:35:38 smithi132 ceph-mon[35941]: pgmap v311: 97 pgs: 97 active+clean; 293 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 5 op/s 2024-03-28T00:35:38.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:35:38 smithi132 ceph-mon[35941]: reconnect by client.25355 172.21.15.119:0/3071152482 after 0.00299993 2024-03-28T00:35:38.705 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:35:38 smithi132 ceph-mon[35941]: reconnect by client.25355 172.21.15.119:0/3071152482 after 0.00199995 2024-03-28T00:35:38.705 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:35:38 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.119:6836/2109630829,v1:172.21.15.119:6837/2109630829] up:reconnect 2024-03-28T00:35:38.705 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:35:38 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.119:6834/1945411848,v1:172.21.15.119:6835/1945411848] up:reconnect 2024-03-28T00:35:38.705 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:35:38 smithi132 ceph-mon[35941]: fsmap cephfs:5/5 3 up:standby-replay 2 up:reconnect 3 up:active 2024-03-28T00:35:38.705 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:35:38 smithi132 ceph-mon[35941]: reconnect by client.25264 v1:172.21.15.119:0/3863511511 after 0.0179996 2024-03-28T00:35:38.705 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:35:38 smithi132 ceph-mon[35941]: reconnect by client.25264 v1:172.21.15.119:0/3863511511 after 0.0169996 2024-03-28T00:35:39.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:35:39 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.119:6836/2109630829,v1:172.21.15.119:6837/2109630829] up:rejoin 2024-03-28T00:35:39.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:35:39 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.119:6834/1945411848,v1:172.21.15.119:6835/1945411848] up:rejoin 2024-03-28T00:35:39.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:35:39 smithi165 ceph-mon[36302]: fsmap cephfs:5/5 3 up:standby-replay 2 up:rejoin 3 up:active 2024-03-28T00:35:39.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:35:39 smithi165 ceph-mon[36302]: daemon mds.d is now active in filesystem cephfs as rank 1 2024-03-28T00:35:39.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:35:39 smithi165 ceph-mon[36302]: daemon mds.g is now active in filesystem cephfs as rank 3 2024-03-28T00:35:39.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.119:6836/2109630829,v1:172.21.15.119:6837/2109630829] up:rejoin 2024-03-28T00:35:39.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.119:6834/1945411848,v1:172.21.15.119:6835/1945411848] up:rejoin 2024-03-28T00:35:39.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-mon[31518]: fsmap cephfs:5/5 3 up:standby-replay 2 up:rejoin 3 up:active 2024-03-28T00:35:39.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-mon[31518]: daemon mds.d is now active in filesystem cephfs as rank 1 2024-03-28T00:35:39.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-mon[31518]: daemon mds.g is now active in filesystem cephfs as rank 3 2024-03-28T00:35:39.657 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7fca2b200700 time 2024-03-28T00:35:39.370090+0000 2024-03-28T00:35:39.658 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T00:35:39.658 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:35:39.658 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fca36b884a1] 2024-03-28T00:35:39.658 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7fca36b88667] 2024-03-28T00:35:39.658 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55908f89a223] 2024-03-28T00:35:39.658 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55908f7145d6] 2024-03-28T00:35:39.658 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55908f714da1] 2024-03-28T00:35:39.658 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: 6: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x55908f74909c] 2024-03-28T00:35:39.658 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55908f767233] 2024-03-28T00:35:39.658 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55908f769197] 2024-03-28T00:35:39.659 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55908f66811b] 2024-03-28T00:35:39.659 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55908f6686bc] 2024-03-28T00:35:39.659 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55908f6521ff] 2024-03-28T00:35:39.659 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7fca36e1a488] 2024-03-28T00:35:39.659 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: 13: (DispatchQueue::entry()+0x51f) [0x7fca36e1760f] 2024-03-28T00:35:39.659 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fca36eeae81] 2024-03-28T00:35:39.659 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fca359051ca] 2024-03-28T00:35:39.659 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: 16: clone() 2024-03-28T00:35:39.659 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7f2450a00700 time 2024-03-28T00:35:39.425056+0000 2024-03-28T00:35:39.659 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T00:35:39.660 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:35:39.660 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f245c2084a1] 2024-03-28T00:35:39.660 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f245c208667] 2024-03-28T00:35:39.660 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55bec1f51223] 2024-03-28T00:35:39.660 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55bec1dcb5d6] 2024-03-28T00:35:39.660 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55bec1dcbda1] 2024-03-28T00:35:39.660 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: 6: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x55bec1e0009c] 2024-03-28T00:35:39.660 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55bec1e1e233] 2024-03-28T00:35:39.660 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55bec1e20197] 2024-03-28T00:35:39.660 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55bec1d1f11b] 2024-03-28T00:35:39.660 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55bec1d1f6bc] 2024-03-28T00:35:39.660 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55bec1d091ff] 2024-03-28T00:35:39.661 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f245c49a488] 2024-03-28T00:35:39.661 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: 13: (DispatchQueue::entry()+0x51f) [0x7f245c49760f] 2024-03-28T00:35:39.661 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f245c56ae81] 2024-03-28T00:35:39.661 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f245af851ca] 2024-03-28T00:35:39.661 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: 16: clone() 2024-03-28T00:35:39.661 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: 2024-03-28T00:35:39.371+0000 7fca2b200700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7fca2b200700 time 2024-03-28T00:35:39.370090+0000 2024-03-28T00:35:39.661 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T00:35:39.661 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: 2024-03-28T00:35:39.662 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:35:39.662 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fca36b884a1] 2024-03-28T00:35:39.662 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7fca36b88667] 2024-03-28T00:35:39.662 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55908f89a223] 2024-03-28T00:35:39.662 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55908f7145d6] 2024-03-28T00:35:39.662 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55908f714da1] 2024-03-28T00:35:39.662 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: 6: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x55908f74909c] 2024-03-28T00:35:39.662 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55908f767233] 2024-03-28T00:35:39.662 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55908f769197] 2024-03-28T00:35:39.662 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55908f66811b] 2024-03-28T00:35:39.662 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55908f6686bc] 2024-03-28T00:35:39.662 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55908f6521ff] 2024-03-28T00:35:39.663 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7fca36e1a488] 2024-03-28T00:35:39.663 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: 13: (DispatchQueue::entry()+0x51f) [0x7fca36e1760f] 2024-03-28T00:35:39.663 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fca36eeae81] 2024-03-28T00:35:39.663 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fca359051ca] 2024-03-28T00:35:39.663 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: 16: clone() 2024-03-28T00:35:39.663 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: 2024-03-28T00:35:39.663 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: *** Caught signal (Aborted) ** 2024-03-28T00:35:39.663 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: in thread 7fca2b200700 thread_name:ms_dispatch 2024-03-28T00:35:39.663 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:35:39.663 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fca3590fd20] 2024-03-28T00:35:39.663 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: 2024-03-28T00:35:39.425+0000 7f2450a00700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7f2450a00700 time 2024-03-28T00:35:39.425056+0000 2024-03-28T00:35:39.664 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T00:35:39.664 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: 2024-03-28T00:35:39.664 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:35:39.664 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f245c2084a1] 2024-03-28T00:35:39.664 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f245c208667] 2024-03-28T00:35:39.664 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55bec1f51223] 2024-03-28T00:35:39.664 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55bec1dcb5d6] 2024-03-28T00:35:39.664 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55bec1dcbda1] 2024-03-28T00:35:39.664 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: 6: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x55bec1e0009c] 2024-03-28T00:35:39.664 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55bec1e1e233] 2024-03-28T00:35:39.664 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55bec1e20197] 2024-03-28T00:35:39.664 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55bec1d1f11b] 2024-03-28T00:35:39.664 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55bec1d1f6bc] 2024-03-28T00:35:39.665 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55bec1d091ff] 2024-03-28T00:35:39.665 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f245c49a488] 2024-03-28T00:35:39.665 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: 13: (DispatchQueue::entry()+0x51f) [0x7f245c49760f] 2024-03-28T00:35:39.665 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f245c56ae81] 2024-03-28T00:35:39.665 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f245af851ca] 2024-03-28T00:35:39.665 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: 16: clone() 2024-03-28T00:35:39.665 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: 2024-03-28T00:35:39.665 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: *** Caught signal (Aborted) ** 2024-03-28T00:35:39.665 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: in thread 7f2450a00700 thread_name:ms_dispatch 2024-03-28T00:35:39.665 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:35:39.665 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f245af8fd20] 2024-03-28T00:35:39.666 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: 2: gsignal() 2024-03-28T00:35:39.666 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: 3: abort() 2024-03-28T00:35:39.666 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fca36b884fb] 2024-03-28T00:35:39.666 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7fca36b88667] 2024-03-28T00:35:39.666 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55908f89a223] 2024-03-28T00:35:39.666 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55908f7145d6] 2024-03-28T00:35:39.666 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55908f714da1] 2024-03-28T00:35:39.666 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: 9: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x55908f74909c] 2024-03-28T00:35:39.666 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55908f767233] 2024-03-28T00:35:39.666 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55908f769197] 2024-03-28T00:35:39.666 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55908f66811b] 2024-03-28T00:35:39.667 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55908f6686bc] 2024-03-28T00:35:39.667 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55908f6521ff] 2024-03-28T00:35:39.667 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7fca36e1a488] 2024-03-28T00:35:39.667 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: 16: (DispatchQueue::entry()+0x51f) [0x7fca36e1760f] 2024-03-28T00:35:39.667 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fca36eeae81] 2024-03-28T00:35:39.667 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7fca359051ca] 2024-03-28T00:35:39.667 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: 19: clone() 2024-03-28T00:35:39.667 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: 2024-03-28T00:35:39.373+0000 7fca2b200700 -1 *** Caught signal (Aborted) ** 2024-03-28T00:35:39.667 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: in thread 7fca2b200700 thread_name:ms_dispatch 2024-03-28T00:35:39.667 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: 2024-03-28T00:35:39.667 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:35:39.667 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fca3590fd20] 2024-03-28T00:35:39.668 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: 2: gsignal() 2024-03-28T00:35:39.668 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: 3: abort() 2024-03-28T00:35:39.668 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fca36b884fb] 2024-03-28T00:35:39.668 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7fca36b88667] 2024-03-28T00:35:39.668 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: 2: gsignal() 2024-03-28T00:35:39.668 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: 3: abort() 2024-03-28T00:35:39.668 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f245c2084fb] 2024-03-28T00:35:39.669 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f245c208667] 2024-03-28T00:35:39.669 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55bec1f51223] 2024-03-28T00:35:39.669 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55bec1dcb5d6] 2024-03-28T00:35:39.669 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55bec1dcbda1] 2024-03-28T00:35:39.669 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: 9: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x55bec1e0009c] 2024-03-28T00:35:39.669 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55bec1e1e233] 2024-03-28T00:35:39.669 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55bec1e20197] 2024-03-28T00:35:39.669 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55bec1d1f11b] 2024-03-28T00:35:39.669 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55bec1d1f6bc] 2024-03-28T00:35:39.669 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55bec1d091ff] 2024-03-28T00:35:39.669 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f245c49a488] 2024-03-28T00:35:39.669 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: 16: (DispatchQueue::entry()+0x51f) [0x7f245c49760f] 2024-03-28T00:35:39.669 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f245c56ae81] 2024-03-28T00:35:39.670 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f245af851ca] 2024-03-28T00:35:39.670 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: 19: clone() 2024-03-28T00:35:39.670 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: 2024-03-28T00:35:39.427+0000 7f2450a00700 -1 *** Caught signal (Aborted) ** 2024-03-28T00:35:39.670 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: in thread 7f2450a00700 thread_name:ms_dispatch 2024-03-28T00:35:39.670 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: 2024-03-28T00:35:39.670 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:35:39.670 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f245af8fd20] 2024-03-28T00:35:39.670 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: 2: gsignal() 2024-03-28T00:35:39.670 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: 3: abort() 2024-03-28T00:35:39.670 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f245c2084fb] 2024-03-28T00:35:39.670 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f245c208667] 2024-03-28T00:35:39.671 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55908f89a223] 2024-03-28T00:35:39.671 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55908f7145d6] 2024-03-28T00:35:39.671 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55908f714da1] 2024-03-28T00:35:39.671 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: 9: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x55908f74909c] 2024-03-28T00:35:39.671 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55908f767233] 2024-03-28T00:35:39.671 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55908f769197] 2024-03-28T00:35:39.671 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55908f66811b] 2024-03-28T00:35:39.671 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55908f6686bc] 2024-03-28T00:35:39.671 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55908f6521ff] 2024-03-28T00:35:39.672 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7fca36e1a488] 2024-03-28T00:35:39.672 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: 16: (DispatchQueue::entry()+0x51f) [0x7fca36e1760f] 2024-03-28T00:35:39.672 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fca36eeae81] 2024-03-28T00:35:39.672 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7fca359051ca] 2024-03-28T00:35:39.672 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: 19: clone() 2024-03-28T00:35:39.672 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T00:35:39.672 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: 2024-03-28T00:35:39.672 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: -1> 2024-03-28T00:35:39.371+0000 7fca2b200700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7fca2b200700 time 2024-03-28T00:35:39.370090+0000 2024-03-28T00:35:39.672 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T00:35:39.672 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: 2024-03-28T00:35:39.672 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:35:39.672 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fca36b884a1] 2024-03-28T00:35:39.672 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7fca36b88667] 2024-03-28T00:35:39.673 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55bec1f51223] 2024-03-28T00:35:39.673 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55bec1dcb5d6] 2024-03-28T00:35:39.673 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55bec1dcbda1] 2024-03-28T00:35:39.673 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: 9: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x55bec1e0009c] 2024-03-28T00:35:39.673 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55bec1e1e233] 2024-03-28T00:35:39.673 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55bec1e20197] 2024-03-28T00:35:39.673 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55bec1d1f11b] 2024-03-28T00:35:39.673 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55bec1d1f6bc] 2024-03-28T00:35:39.673 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55bec1d091ff] 2024-03-28T00:35:39.673 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f245c49a488] 2024-03-28T00:35:39.673 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: 16: (DispatchQueue::entry()+0x51f) [0x7f245c49760f] 2024-03-28T00:35:39.673 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f245c56ae81] 2024-03-28T00:35:39.674 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f245af851ca] 2024-03-28T00:35:39.674 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: 19: clone() 2024-03-28T00:35:39.674 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T00:35:39.674 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: 2024-03-28T00:35:39.674 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: -1> 2024-03-28T00:35:39.425+0000 7f2450a00700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7f2450a00700 time 2024-03-28T00:35:39.425056+0000 2024-03-28T00:35:39.674 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T00:35:39.674 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: 2024-03-28T00:35:39.674 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:35:39.674 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f245c2084a1] 2024-03-28T00:35:39.674 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f245c208667] 2024-03-28T00:35:39.675 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55908f89a223] 2024-03-28T00:35:39.675 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55908f7145d6] 2024-03-28T00:35:39.675 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55908f714da1] 2024-03-28T00:35:39.675 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: 6: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x55908f74909c] 2024-03-28T00:35:39.675 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55908f767233] 2024-03-28T00:35:39.675 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55908f769197] 2024-03-28T00:35:39.675 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55908f66811b] 2024-03-28T00:35:39.675 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55908f6686bc] 2024-03-28T00:35:39.675 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55908f6521ff] 2024-03-28T00:35:39.675 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7fca36e1a488] 2024-03-28T00:35:39.675 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: 13: (DispatchQueue::entry()+0x51f) [0x7fca36e1760f] 2024-03-28T00:35:39.676 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fca36eeae81] 2024-03-28T00:35:39.676 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fca359051ca] 2024-03-28T00:35:39.676 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: 16: clone() 2024-03-28T00:35:39.676 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: 2024-03-28T00:35:39.676 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: 0> 2024-03-28T00:35:39.373+0000 7fca2b200700 -1 *** Caught signal (Aborted) ** 2024-03-28T00:35:39.676 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: in thread 7fca2b200700 thread_name:ms_dispatch 2024-03-28T00:35:39.676 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: 2024-03-28T00:35:39.676 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:35:39.676 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fca3590fd20] 2024-03-28T00:35:39.676 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: 2: gsignal() 2024-03-28T00:35:39.676 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: 3: abort() 2024-03-28T00:35:39.676 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fca36b884fb] 2024-03-28T00:35:39.676 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7fca36b88667] 2024-03-28T00:35:39.676 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55908f89a223] 2024-03-28T00:35:39.677 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55908f7145d6] 2024-03-28T00:35:39.677 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55bec1f51223] 2024-03-28T00:35:39.677 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55bec1dcb5d6] 2024-03-28T00:35:39.677 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55bec1dcbda1] 2024-03-28T00:35:39.677 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: 6: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x55bec1e0009c] 2024-03-28T00:35:39.677 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55bec1e1e233] 2024-03-28T00:35:39.677 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55bec1e20197] 2024-03-28T00:35:39.677 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55bec1d1f11b] 2024-03-28T00:35:39.677 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55bec1d1f6bc] 2024-03-28T00:35:39.677 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55bec1d091ff] 2024-03-28T00:35:39.678 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f245c49a488] 2024-03-28T00:35:39.678 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: 13: (DispatchQueue::entry()+0x51f) [0x7f245c49760f] 2024-03-28T00:35:39.678 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f245c56ae81] 2024-03-28T00:35:39.678 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f245af851ca] 2024-03-28T00:35:39.678 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: 16: clone() 2024-03-28T00:35:39.678 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: 2024-03-28T00:35:39.678 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: 0> 2024-03-28T00:35:39.427+0000 7f2450a00700 -1 *** Caught signal (Aborted) ** 2024-03-28T00:35:39.678 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: in thread 7f2450a00700 thread_name:ms_dispatch 2024-03-28T00:35:39.678 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: 2024-03-28T00:35:39.678 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:35:39.678 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f245af8fd20] 2024-03-28T00:35:39.678 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: 2: gsignal() 2024-03-28T00:35:39.679 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: 3: abort() 2024-03-28T00:35:39.679 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f245c2084fb] 2024-03-28T00:35:39.679 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f245c208667] 2024-03-28T00:35:39.679 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55bec1f51223] 2024-03-28T00:35:39.679 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55bec1dcb5d6] 2024-03-28T00:35:39.679 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55908f714da1] 2024-03-28T00:35:39.679 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: 9: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x55908f74909c] 2024-03-28T00:35:39.679 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55908f767233] 2024-03-28T00:35:39.680 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55908f769197] 2024-03-28T00:35:39.680 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55908f66811b] 2024-03-28T00:35:39.680 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55908f6686bc] 2024-03-28T00:35:39.680 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55908f6521ff] 2024-03-28T00:35:39.680 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7fca36e1a488] 2024-03-28T00:35:39.680 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: 16: (DispatchQueue::entry()+0x51f) [0x7fca36e1760f] 2024-03-28T00:35:39.680 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fca36eeae81] 2024-03-28T00:35:39.680 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7fca359051ca] 2024-03-28T00:35:39.680 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: 19: clone() 2024-03-28T00:35:39.680 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T00:35:39.680 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: 2024-03-28T00:35:39.680 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: -1> 2024-03-28T00:35:39.371+0000 7fca2b200700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7fca2b200700 time 2024-03-28T00:35:39.370090+0000 2024-03-28T00:35:39.680 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T00:35:39.680 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: 2024-03-28T00:35:39.680 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:35:39.681 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fca36b884a1] 2024-03-28T00:35:39.681 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7fca36b88667] 2024-03-28T00:35:39.681 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55908f89a223] 2024-03-28T00:35:39.681 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55908f7145d6] 2024-03-28T00:35:39.681 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55bec1dcbda1] 2024-03-28T00:35:39.681 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: 9: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x55bec1e0009c] 2024-03-28T00:35:39.681 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55bec1e1e233] 2024-03-28T00:35:39.681 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55bec1e20197] 2024-03-28T00:35:39.682 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55bec1d1f11b] 2024-03-28T00:35:39.682 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55bec1d1f6bc] 2024-03-28T00:35:39.682 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55bec1d091ff] 2024-03-28T00:35:39.682 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f245c49a488] 2024-03-28T00:35:39.682 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: 16: (DispatchQueue::entry()+0x51f) [0x7f245c49760f] 2024-03-28T00:35:39.682 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f245c56ae81] 2024-03-28T00:35:39.682 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f245af851ca] 2024-03-28T00:35:39.682 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: 19: clone() 2024-03-28T00:35:39.682 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T00:35:39.682 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: 2024-03-28T00:35:39.682 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: -1> 2024-03-28T00:35:39.425+0000 7f2450a00700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7f2450a00700 time 2024-03-28T00:35:39.425056+0000 2024-03-28T00:35:39.682 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T00:35:39.683 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: 2024-03-28T00:35:39.683 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:35:39.683 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f245c2084a1] 2024-03-28T00:35:39.683 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f245c208667] 2024-03-28T00:35:39.683 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55bec1f51223] 2024-03-28T00:35:39.683 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55bec1dcb5d6] 2024-03-28T00:35:39.683 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55908f714da1] 2024-03-28T00:35:39.683 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: 6: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x55908f74909c] 2024-03-28T00:35:39.684 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55908f767233] 2024-03-28T00:35:39.684 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55908f769197] 2024-03-28T00:35:39.684 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55908f66811b] 2024-03-28T00:35:39.684 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55908f6686bc] 2024-03-28T00:35:39.684 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55908f6521ff] 2024-03-28T00:35:39.684 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7fca36e1a488] 2024-03-28T00:35:39.684 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: 13: (DispatchQueue::entry()+0x51f) [0x7fca36e1760f] 2024-03-28T00:35:39.684 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fca36eeae81] 2024-03-28T00:35:39.684 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fca359051ca] 2024-03-28T00:35:39.684 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: 16: clone() 2024-03-28T00:35:39.684 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: 2024-03-28T00:35:39.684 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: 0> 2024-03-28T00:35:39.373+0000 7fca2b200700 -1 *** Caught signal (Aborted) ** 2024-03-28T00:35:39.685 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: in thread 7fca2b200700 thread_name:ms_dispatch 2024-03-28T00:35:39.685 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: 2024-03-28T00:35:39.685 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:35:39.685 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fca3590fd20] 2024-03-28T00:35:39.685 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: 2: gsignal() 2024-03-28T00:35:39.685 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: 3: abort() 2024-03-28T00:35:39.685 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fca36b884fb] 2024-03-28T00:35:39.685 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7fca36b88667] 2024-03-28T00:35:39.685 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55908f89a223] 2024-03-28T00:35:39.685 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55908f7145d6] 2024-03-28T00:35:39.685 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55908f714da1] 2024-03-28T00:35:39.685 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: 9: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x55908f74909c] 2024-03-28T00:35:39.686 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55bec1dcbda1] 2024-03-28T00:35:39.686 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: 6: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x55bec1e0009c] 2024-03-28T00:35:39.686 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55bec1e1e233] 2024-03-28T00:35:39.686 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55bec1e20197] 2024-03-28T00:35:39.686 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55bec1d1f11b] 2024-03-28T00:35:39.686 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55bec1d1f6bc] 2024-03-28T00:35:39.686 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55bec1d091ff] 2024-03-28T00:35:39.686 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f245c49a488] 2024-03-28T00:35:39.686 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: 13: (DispatchQueue::entry()+0x51f) [0x7f245c49760f] 2024-03-28T00:35:39.686 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f245c56ae81] 2024-03-28T00:35:39.686 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f245af851ca] 2024-03-28T00:35:39.687 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: 16: clone() 2024-03-28T00:35:39.687 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: 2024-03-28T00:35:39.687 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: 0> 2024-03-28T00:35:39.427+0000 7f2450a00700 -1 *** Caught signal (Aborted) ** 2024-03-28T00:35:39.687 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: in thread 7f2450a00700 thread_name:ms_dispatch 2024-03-28T00:35:39.687 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: 2024-03-28T00:35:39.687 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T00:35:39.687 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f245af8fd20] 2024-03-28T00:35:39.687 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: 2: gsignal() 2024-03-28T00:35:39.687 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: 3: abort() 2024-03-28T00:35:39.687 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f245c2084fb] 2024-03-28T00:35:39.687 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f245c208667] 2024-03-28T00:35:39.687 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55bec1f51223] 2024-03-28T00:35:39.687 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55bec1dcb5d6] 2024-03-28T00:35:39.687 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55bec1dcbda1] 2024-03-28T00:35:39.688 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: 9: (Server::handle_client_openc(boost::intrusive_ptr const&)+0xa9c) [0x55bec1e0009c] 2024-03-28T00:35:39.688 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55908f767233] 2024-03-28T00:35:39.688 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55908f769197] 2024-03-28T00:35:39.688 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55908f66811b] 2024-03-28T00:35:39.688 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55908f6686bc] 2024-03-28T00:35:39.688 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55908f6521ff] 2024-03-28T00:35:39.688 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7fca36e1a488] 2024-03-28T00:35:39.688 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: 16: (DispatchQueue::entry()+0x51f) [0x7fca36e1760f] 2024-03-28T00:35:39.688 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fca36eeae81] 2024-03-28T00:35:39.689 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7fca359051ca] 2024-03-28T00:35:39.689 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: 19: clone() 2024-03-28T00:35:39.689 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T00:35:39.689 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g[119745]: 2024-03-28T00:35:39.689 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:39 smithi119 podman[120259]: 2024-03-28 00:35:39.487845065 +0000 UTC m=+0.038177202 container died 8619117ec89b37e78705aea709041738e1dd460a60582688981732aef41f8f91 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-03-28T00:35:39.689 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55bec1e1e233] 2024-03-28T00:35:39.689 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55bec1e20197] 2024-03-28T00:35:39.689 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55bec1d1f11b] 2024-03-28T00:35:39.689 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55bec1d1f6bc] 2024-03-28T00:35:39.689 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55bec1d091ff] 2024-03-28T00:35:39.690 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f245c49a488] 2024-03-28T00:35:39.690 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: 16: (DispatchQueue::entry()+0x51f) [0x7f245c49760f] 2024-03-28T00:35:39.690 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f245c56ae81] 2024-03-28T00:35:39.690 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f245af851ca] 2024-03-28T00:35:39.690 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: 19: clone() 2024-03-28T00:35:39.690 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T00:35:39.690 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:39 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[119890]: 2024-03-28T00:35:39.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:35:39 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.119:6836/2109630829,v1:172.21.15.119:6837/2109630829] up:rejoin 2024-03-28T00:35:39.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:35:39 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.119:6834/1945411848,v1:172.21.15.119:6835/1945411848] up:rejoin 2024-03-28T00:35:39.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:35:39 smithi132 ceph-mon[35941]: fsmap cephfs:5/5 3 up:standby-replay 2 up:rejoin 3 up:active 2024-03-28T00:35:39.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:35:39 smithi132 ceph-mon[35941]: daemon mds.d is now active in filesystem cephfs as rank 1 2024-03-28T00:35:39.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:35:39 smithi132 ceph-mon[35941]: daemon mds.g is now active in filesystem cephfs as rank 3 2024-03-28T00:35:40.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:35:40 smithi165 ceph-mon[36302]: pgmap v312: 97 pgs: 97 active+clean; 296 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 312 KiB/s rd, 358 KiB/s wr, 44 op/s 2024-03-28T00:35:40.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:35:40 smithi165 ceph-mon[36302]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-28T00:35:40.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:35:40 smithi165 ceph-mon[36302]: Cluster is now healthy 2024-03-28T00:35:40.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:35:40 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.119:6836/2109630829,v1:172.21.15.119:6837/2109630829] up:active 2024-03-28T00:35:40.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:35:40 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.119:6834/1945411848,v1:172.21.15.119:6835/1945411848] up:active 2024-03-28T00:35:40.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:35:40 smithi165 ceph-mon[36302]: fsmap cephfs:5 3 up:standby-replay 5 up:active 2024-03-28T00:35:40.656 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:40 smithi119 podman[120294]: 2024-03-28 00:35:40.376647339 +0000 UTC m=+0.861699270 container died 450ac508eef1f0f0c4b03570e6e529d736fa2801a2d5ec065a5d3dcd6ef2cfc8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c) 2024-03-28T00:35:40.657 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:40 smithi119 podman[120259]: 2024-03-28 00:35:40.277484447 +0000 UTC m=+0.827816583 container remove 8619117ec89b37e78705aea709041738e1dd460a60582688981732aef41f8f91 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-g, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-03-28T00:35:40.657 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:40 smithi119 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.g.service: Main process exited, code=exited, status=134/n/a 2024-03-28T00:35:40.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:35:40 smithi119 ceph-mon[31518]: pgmap v312: 97 pgs: 97 active+clean; 296 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 312 KiB/s rd, 358 KiB/s wr, 44 op/s 2024-03-28T00:35:40.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:35:40 smithi119 ceph-mon[31518]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-28T00:35:40.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:35:40 smithi119 ceph-mon[31518]: Cluster is now healthy 2024-03-28T00:35:40.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:35:40 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.119:6836/2109630829,v1:172.21.15.119:6837/2109630829] up:active 2024-03-28T00:35:40.658 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:35:40 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.119:6834/1945411848,v1:172.21.15.119:6835/1945411848] up:active 2024-03-28T00:35:40.658 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:35:40 smithi119 ceph-mon[31518]: fsmap cephfs:5 3 up:standby-replay 5 up:active 2024-03-28T00:35:40.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:35:40 smithi132 ceph-mon[35941]: pgmap v312: 97 pgs: 97 active+clean; 296 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 312 KiB/s rd, 358 KiB/s wr, 44 op/s 2024-03-28T00:35:40.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:35:40 smithi132 ceph-mon[35941]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-28T00:35:40.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:35:40 smithi132 ceph-mon[35941]: Cluster is now healthy 2024-03-28T00:35:40.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:35:40 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.119:6836/2109630829,v1:172.21.15.119:6837/2109630829] up:active 2024-03-28T00:35:40.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:35:40 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.119:6834/1945411848,v1:172.21.15.119:6835/1945411848] up:active 2024-03-28T00:35:40.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:35:40 smithi132 ceph-mon[35941]: fsmap cephfs:5 3 up:standby-replay 5 up:active 2024-03-28T00:35:41.372 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:41 smithi119 podman[120294]: 2024-03-28 00:35:41.099543887 +0000 UTC m=+1.584595809 container remove 450ac508eef1f0f0c4b03570e6e529d736fa2801a2d5ec065a5d3dcd6ef2cfc8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c) 2024-03-28T00:35:41.372 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:41 smithi119 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.d.service: Main process exited, code=exited, status=134/n/a 2024-03-28T00:35:41.657 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:41 smithi119 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.d.service: Failed with result 'exit-code'. 2024-03-28T00:35:41.657 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:41 smithi119 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.d.service: Consumed 1.387s CPU time. 2024-03-28T00:35:41.657 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:41 smithi119 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.g.service: Failed with result 'exit-code'. 2024-03-28T00:35:41.658 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:41 smithi119 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.g.service: Consumed 1.496s CPU time. 2024-03-28T00:35:42.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:35:42 smithi165 ceph-mon[36302]: pgmap v313: 97 pgs: 97 active+clean; 296 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 312 KiB/s rd, 358 KiB/s wr, 44 op/s 2024-03-28T00:35:42.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:35:42 smithi119 ceph-mon[31518]: pgmap v313: 97 pgs: 97 active+clean; 296 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 312 KiB/s rd, 358 KiB/s wr, 44 op/s 2024-03-28T00:35:42.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:35:42 smithi132 ceph-mon[35941]: pgmap v313: 97 pgs: 97 active+clean; 296 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 312 KiB/s rd, 358 KiB/s wr, 44 op/s 2024-03-28T00:35:44.555 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:35:44 smithi119 ceph-mon[31518]: pgmap v314: 97 pgs: 97 active+clean; 296 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 336 KiB/s wr, 37 op/s 2024-03-28T00:35:44.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:35:44 smithi165 ceph-mon[36302]: pgmap v314: 97 pgs: 97 active+clean; 296 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 336 KiB/s wr, 37 op/s 2024-03-28T00:35:44.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:35:44 smithi132 ceph-mon[35941]: pgmap v314: 97 pgs: 97 active+clean; 296 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 336 KiB/s wr, 37 op/s 2024-03-28T00:35:46.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:35:46 smithi165 ceph-mon[36302]: pgmap v315: 97 pgs: 97 active+clean; 303 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.1 MiB/s wr, 394 op/s 2024-03-28T00:35:46.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:35:46 smithi119 ceph-mon[31518]: pgmap v315: 97 pgs: 97 active+clean; 303 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.1 MiB/s wr, 394 op/s 2024-03-28T00:35:46.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:35:46 smithi132 ceph-mon[35941]: pgmap v315: 97 pgs: 97 active+clean; 303 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.1 MiB/s wr, 394 op/s 2024-03-28T00:35:48.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:35:48 smithi165 ceph-mon[36302]: pgmap v316: 97 pgs: 97 active+clean; 303 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 920 KiB/s wr, 331 op/s 2024-03-28T00:35:48.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:35:48 smithi119 ceph-mon[31518]: pgmap v316: 97 pgs: 97 active+clean; 303 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 920 KiB/s wr, 331 op/s 2024-03-28T00:35:48.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:35:48 smithi132 ceph-mon[35941]: pgmap v316: 97 pgs: 97 active+clean; 303 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 920 KiB/s wr, 331 op/s 2024-03-28T00:35:50.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:35:50 smithi165 ceph-mon[36302]: pgmap v317: 97 pgs: 97 active+clean; 303 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 912 KiB/s wr, 332 op/s 2024-03-28T00:35:50.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:35:50 smithi119 ceph-mon[31518]: pgmap v317: 97 pgs: 97 active+clean; 303 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 912 KiB/s wr, 332 op/s 2024-03-28T00:35:50.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:35:50 smithi132 ceph-mon[35941]: pgmap v317: 97 pgs: 97 active+clean; 303 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 912 KiB/s wr, 332 op/s 2024-03-28T00:35:51.681 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:51 smithi119 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.d.service: Scheduled restart job, restart counter is at 4. 2024-03-28T00:35:51.681 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:51 smithi119 systemd[1]: Stopped Ceph mds.d for 10f73b92-ec98-11ee-95d0-87774f69a715. 2024-03-28T00:35:51.681 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:51 smithi119 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.d.service: Consumed 1.387s CPU time. 2024-03-28T00:35:51.682 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:51 smithi119 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.g.service: Scheduled restart job, restart counter is at 6. 2024-03-28T00:35:51.682 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:51 smithi119 systemd[1]: Stopped Ceph mds.g for 10f73b92-ec98-11ee-95d0-87774f69a715. 2024-03-28T00:35:51.682 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:51 smithi119 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.g.service: Consumed 1.496s CPU time. 2024-03-28T00:35:51.682 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:51 smithi119 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.g.service: Start request repeated too quickly. 2024-03-28T00:35:51.682 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:51 smithi119 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.g.service: Failed with result 'exit-code'. 2024-03-28T00:35:51.682 INFO:journalctl@ceph.mds.g.smithi119.stdout:Mar 28 00:35:51 smithi119 systemd[1]: Failed to start Ceph mds.g for 10f73b92-ec98-11ee-95d0-87774f69a715. 2024-03-28T00:35:52.048 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:51 smithi119 systemd[1]: Starting Ceph mds.d for 10f73b92-ec98-11ee-95d0-87774f69a715... 2024-03-28T00:35:52.353 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:52 smithi119 podman[121370]: 2024-03-28 00:35:51.945721136 +0000 UTC m=+0.031676024 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-28T00:35:52.353 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:52 smithi119 podman[121370]: 2024-03-28 00:35:52.108773257 +0000 UTC m=+0.194728182 container create 49458320a2c67c738c4eb9535b3ccb34df887bf7469714ed67f6aa6e8aef6ff8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240319) 2024-03-28T00:35:52.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:35:52 smithi165 ceph-mon[36302]: pgmap v318: 97 pgs: 97 active+clean; 303 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 869 KiB/s rd, 674 KiB/s wr, 306 op/s 2024-03-28T00:35:52.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:35:52 smithi119 ceph-mon[31518]: pgmap v318: 97 pgs: 97 active+clean; 303 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 869 KiB/s rd, 674 KiB/s wr, 306 op/s 2024-03-28T00:35:52.657 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:52 smithi119 podman[121370]: 2024-03-28 00:35:52.345922946 +0000 UTC m=+0.431877878 container init 49458320a2c67c738c4eb9535b3ccb34df887bf7469714ed67f6aa6e8aef6ff8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d, org.label-schema.build-date=20240319, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-03-28T00:35:52.657 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:52 smithi119 podman[121370]: 2024-03-28 00:35:52.360299213 +0000 UTC m=+0.446254108 container start 49458320a2c67c738c4eb9535b3ccb34df887bf7469714ed67f6aa6e8aef6ff8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-03-28T00:35:52.658 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:52 smithi119 ceph-mds[121389]: -- 172.21.15.119:0/3820118132 <== mon.2 v2:172.21.15.132:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x555d19ac5a40 con 0x555d19a98800 2024-03-28T00:35:52.658 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:52 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[121385]: starting mds.d at 2024-03-28T00:35:52.658 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:52 smithi119 bash[121370]: 49458320a2c67c738c4eb9535b3ccb34df887bf7469714ed67f6aa6e8aef6ff8 2024-03-28T00:35:52.658 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 00:35:52 smithi119 systemd[1]: Started Ceph mds.d for 10f73b92-ec98-11ee-95d0-87774f69a715. 2024-03-28T00:35:52.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:35:52 smithi132 ceph-mon[35941]: pgmap v318: 97 pgs: 97 active+clean; 303 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 869 KiB/s rd, 674 KiB/s wr, 306 op/s 2024-03-28T00:35:53.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:35:53 smithi165 ceph-mon[36302]: daemon mds.d restarted 2024-03-28T00:35:53.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:35:53 smithi165 ceph-mon[36302]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-28T00:35:53.635 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:35:53 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-28T00:35:53.635 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:35:53 smithi165 ceph-mon[36302]: osdmap e125: 12 total, 12 up, 12 in 2024-03-28T00:35:53.635 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:35:53 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.119:6834/1473458343,v1:172.21.15.119:6835/1473458343] up:boot 2024-03-28T00:35:53.635 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:35:53 smithi165 ceph-mon[36302]: Standby daemon mds.d assigned to filesystem cephfs as rank 1 2024-03-28T00:35:53.635 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:35:53 smithi165 ceph-mon[36302]: fsmap cephfs:4/5 {0=b=up:active,2=h=up:active,3=g=up:active,4=f=up:active} 3 up:standby-replay 1 up:standby, 1 failed 2024-03-28T00:35:53.635 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:35:53 smithi165 ceph-mon[36302]: fsmap cephfs:5/5 3 up:standby-replay 1 up:replay 4 up:active 2024-03-28T00:35:53.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:35:53 smithi119 ceph-mon[31518]: daemon mds.d restarted 2024-03-28T00:35:53.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:35:53 smithi119 ceph-mon[31518]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-28T00:35:53.658 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:35:53 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-28T00:35:53.658 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:35:53 smithi119 ceph-mon[31518]: osdmap e125: 12 total, 12 up, 12 in 2024-03-28T00:35:53.658 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:35:53 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.119:6834/1473458343,v1:172.21.15.119:6835/1473458343] up:boot 2024-03-28T00:35:53.658 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:35:53 smithi119 ceph-mon[31518]: Standby daemon mds.d assigned to filesystem cephfs as rank 1 2024-03-28T00:35:53.658 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:35:53 smithi119 ceph-mon[31518]: fsmap cephfs:4/5 {0=b=up:active,2=h=up:active,3=g=up:active,4=f=up:active} 3 up:standby-replay 1 up:standby, 1 failed 2024-03-28T00:35:53.659 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:35:53 smithi119 ceph-mon[31518]: fsmap cephfs:5/5 3 up:standby-replay 1 up:replay 4 up:active 2024-03-28T00:35:53.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:35:53 smithi132 ceph-mon[35941]: daemon mds.d restarted 2024-03-28T00:35:53.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:35:53 smithi132 ceph-mon[35941]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-28T00:35:53.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:35:53 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-28T00:35:53.705 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:35:53 smithi132 ceph-mon[35941]: osdmap e125: 12 total, 12 up, 12 in 2024-03-28T00:35:53.705 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:35:53 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.119:6834/1473458343,v1:172.21.15.119:6835/1473458343] up:boot 2024-03-28T00:35:53.705 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:35:53 smithi132 ceph-mon[35941]: Standby daemon mds.d assigned to filesystem cephfs as rank 1 2024-03-28T00:35:53.705 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:35:53 smithi132 ceph-mon[35941]: fsmap cephfs:4/5 {0=b=up:active,2=h=up:active,3=g=up:active,4=f=up:active} 3 up:standby-replay 1 up:standby, 1 failed 2024-03-28T00:35:53.705 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:35:53 smithi132 ceph-mon[35941]: fsmap cephfs:5/5 3 up:standby-replay 1 up:replay 4 up:active 2024-03-28T00:35:54.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:35:54 smithi132 ceph-mon[35941]: pgmap v320: 97 pgs: 97 active+clean; 303 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 808 KiB/s wr, 367 op/s 2024-03-28T00:35:54.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:35:54 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.119:6834/1473458343,v1:172.21.15.119:6835/1473458343] up:resolve 2024-03-28T00:35:54.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:35:54 smithi132 ceph-mon[35941]: fsmap cephfs:5/5 3 up:standby-replay 1 up:resolve 4 up:active 2024-03-28T00:35:54.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:35:54 smithi132 ceph-mon[35941]: fsmap cephfs:5/5 3 up:standby-replay 1 up:resolve 4 up:active 2024-03-28T00:35:54.884 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:35:54 smithi165 ceph-mon[36302]: pgmap v320: 97 pgs: 97 active+clean; 303 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 808 KiB/s wr, 367 op/s 2024-03-28T00:35:54.884 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:35:54 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.119:6834/1473458343,v1:172.21.15.119:6835/1473458343] up:resolve 2024-03-28T00:35:54.884 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:35:54 smithi165 ceph-mon[36302]: fsmap cephfs:5/5 3 up:standby-replay 1 up:resolve 4 up:active 2024-03-28T00:35:54.884 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:35:54 smithi165 ceph-mon[36302]: fsmap cephfs:5/5 3 up:standby-replay 1 up:resolve 4 up:active 2024-03-28T00:35:54.907 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:35:54 smithi119 ceph-mon[31518]: pgmap v320: 97 pgs: 97 active+clean; 303 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 808 KiB/s wr, 367 op/s 2024-03-28T00:35:54.907 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:35:54 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.119:6834/1473458343,v1:172.21.15.119:6835/1473458343] up:resolve 2024-03-28T00:35:54.907 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:35:54 smithi119 ceph-mon[31518]: fsmap cephfs:5/5 3 up:standby-replay 1 up:resolve 4 up:active 2024-03-28T00:35:54.907 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:35:54 smithi119 ceph-mon[31518]: fsmap cephfs:5/5 3 up:standby-replay 1 up:resolve 4 up:active 2024-03-28T00:35:56.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:35:56 smithi132 ceph-mon[35941]: pgmap v321: 97 pgs: 97 active+clean; 303 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 204 B/s wr, 9 op/s 2024-03-28T00:35:56.884 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:35:56 smithi165 ceph-mon[36302]: pgmap v321: 97 pgs: 97 active+clean; 303 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 204 B/s wr, 9 op/s 2024-03-28T00:35:56.907 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:35:56 smithi119 ceph-mon[31518]: pgmap v321: 97 pgs: 97 active+clean; 303 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 204 B/s wr, 9 op/s 2024-03-28T00:35:58.884 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:35:58 smithi165 ceph-mon[36302]: pgmap v322: 97 pgs: 97 active+clean; 303 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 204 B/s wr, 9 op/s 2024-03-28T00:35:58.907 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:35:58 smithi119 ceph-mon[31518]: pgmap v322: 97 pgs: 97 active+clean; 303 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 204 B/s wr, 9 op/s 2024-03-28T00:35:58.954 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:35:58 smithi132 ceph-mon[35941]: pgmap v322: 97 pgs: 97 active+clean; 303 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 204 B/s wr, 9 op/s 2024-03-28T00:36:00.884 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:36:00 smithi165 ceph-mon[36302]: pgmap v323: 97 pgs: 97 active+clean; 303 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 0 B/s wr, 9 op/s 2024-03-28T00:36:00.907 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:36:00 smithi119 ceph-mon[31518]: pgmap v323: 97 pgs: 97 active+clean; 303 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 0 B/s wr, 9 op/s 2024-03-28T00:36:00.954 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:36:00 smithi132 ceph-mon[35941]: pgmap v323: 97 pgs: 97 active+clean; 303 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 0 B/s wr, 9 op/s 2024-03-28T00:36:02.755 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:36:02 smithi132 ceph-mon[35941]: pgmap v324: 97 pgs: 97 active+clean; 303 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 0 B/s wr, 9 op/s 2024-03-28T00:36:02.884 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:36:02 smithi165 ceph-mon[36302]: pgmap v324: 97 pgs: 97 active+clean; 303 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 0 B/s wr, 9 op/s 2024-03-28T00:36:02.907 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:36:02 smithi119 ceph-mon[31518]: pgmap v324: 97 pgs: 97 active+clean; 303 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 0 B/s wr, 9 op/s 2024-03-28T00:36:04.884 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:36:04 smithi165 ceph-mon[36302]: pgmap v325: 97 pgs: 97 active+clean; 303 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 0 B/s wr, 8 op/s 2024-03-28T00:36:04.884 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:36:04 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:36:04.907 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:36:04 smithi119 ceph-mon[31518]: pgmap v325: 97 pgs: 97 active+clean; 303 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 0 B/s wr, 8 op/s 2024-03-28T00:36:04.907 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:36:04 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:36:04.954 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:36:04 smithi132 ceph-mon[35941]: pgmap v325: 97 pgs: 97 active+clean; 303 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 0 B/s wr, 8 op/s 2024-03-28T00:36:04.954 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:36:04 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:36:05.884 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:36:05 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:36:05.884 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:36:05 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:36:05.884 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:36:05 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:36:05.907 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:36:05 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:36:05.907 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:36:05 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:36:05.907 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:36:05 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:36:05.954 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:36:05 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:36:05.954 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:36:05 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:36:05.954 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:36:05 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:36:06.884 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:36:06 smithi165 ceph-mon[36302]: pgmap v326: 97 pgs: 97 active+clean; 305 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 166 KiB/s wr, 159 op/s 2024-03-28T00:36:06.907 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:36:06 smithi119 ceph-mon[31518]: pgmap v326: 97 pgs: 97 active+clean; 305 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 166 KiB/s wr, 159 op/s 2024-03-28T00:36:06.954 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:36:06 smithi132 ceph-mon[35941]: pgmap v326: 97 pgs: 97 active+clean; 305 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 166 KiB/s wr, 159 op/s 2024-03-28T00:36:08.884 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:36:08 smithi165 ceph-mon[36302]: pgmap v327: 97 pgs: 97 active+clean; 305 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 166 KiB/s wr, 154 op/s 2024-03-28T00:36:08.907 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:36:08 smithi119 ceph-mon[31518]: pgmap v327: 97 pgs: 97 active+clean; 305 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 166 KiB/s wr, 154 op/s 2024-03-28T00:36:08.954 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:36:08 smithi132 ceph-mon[35941]: pgmap v327: 97 pgs: 97 active+clean; 305 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 166 KiB/s wr, 154 op/s 2024-03-28T00:36:10.884 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:36:10 smithi165 ceph-mon[36302]: pgmap v328: 97 pgs: 97 active+clean; 305 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 166 KiB/s wr, 158 op/s 2024-03-28T00:36:10.907 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:36:10 smithi119 ceph-mon[31518]: pgmap v328: 97 pgs: 97 active+clean; 305 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 166 KiB/s wr, 158 op/s 2024-03-28T00:36:10.954 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:36:10 smithi132 ceph-mon[35941]: pgmap v328: 97 pgs: 97 active+clean; 305 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 166 KiB/s wr, 158 op/s 2024-03-28T00:36:12.884 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:36:12 smithi165 ceph-mon[36302]: pgmap v329: 97 pgs: 97 active+clean; 305 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 166 KiB/s wr, 154 op/s 2024-03-28T00:36:12.907 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:36:12 smithi119 ceph-mon[31518]: pgmap v329: 97 pgs: 97 active+clean; 305 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 166 KiB/s wr, 154 op/s 2024-03-28T00:36:12.954 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:36:12 smithi132 ceph-mon[35941]: pgmap v329: 97 pgs: 97 active+clean; 305 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 166 KiB/s wr, 154 op/s 2024-03-28T00:36:14.884 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:36:14 smithi165 ceph-mon[36302]: pgmap v330: 97 pgs: 97 active+clean; 305 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 166 KiB/s wr, 154 op/s 2024-03-28T00:36:14.907 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:36:14 smithi119 ceph-mon[31518]: pgmap v330: 97 pgs: 97 active+clean; 305 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 166 KiB/s wr, 154 op/s 2024-03-28T00:36:14.954 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:36:14 smithi132 ceph-mon[35941]: pgmap v330: 97 pgs: 97 active+clean; 305 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 166 KiB/s wr, 154 op/s 2024-03-28T00:36:16.884 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:36:16 smithi165 ceph-mon[36302]: pgmap v331: 97 pgs: 97 active+clean; 309 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 459 KiB/s rd, 454 KiB/s wr, 447 op/s 2024-03-28T00:36:16.907 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:36:16 smithi119 ceph-mon[31518]: pgmap v331: 97 pgs: 97 active+clean; 309 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 459 KiB/s rd, 454 KiB/s wr, 447 op/s 2024-03-28T00:36:16.954 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:36:16 smithi132 ceph-mon[35941]: pgmap v331: 97 pgs: 97 active+clean; 309 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 459 KiB/s rd, 454 KiB/s wr, 447 op/s 2024-03-28T00:36:18.582 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:36:18 smithi119 ceph-mon[31518]: pgmap v332: 97 pgs: 97 active+clean; 309 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 288 KiB/s wr, 295 op/s 2024-03-28T00:36:18.884 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:36:18 smithi165 ceph-mon[36302]: pgmap v332: 97 pgs: 97 active+clean; 309 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 288 KiB/s wr, 295 op/s 2024-03-28T00:36:18.954 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:36:18 smithi132 ceph-mon[35941]: pgmap v332: 97 pgs: 97 active+clean; 309 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 288 KiB/s wr, 295 op/s 2024-03-28T00:36:20.884 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:36:20 smithi165 ceph-mon[36302]: pgmap v333: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 441 KiB/s rd, 437 KiB/s wr, 299 op/s 2024-03-28T00:36:20.907 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:36:20 smithi119 ceph-mon[31518]: pgmap v333: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 441 KiB/s rd, 437 KiB/s wr, 299 op/s 2024-03-28T00:36:20.954 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:36:20 smithi132 ceph-mon[35941]: pgmap v333: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 441 KiB/s rd, 437 KiB/s wr, 299 op/s 2024-03-28T00:36:22.884 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:36:22 smithi165 ceph-mon[36302]: pgmap v334: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 439 KiB/s rd, 437 KiB/s wr, 295 op/s 2024-03-28T00:36:22.907 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:36:22 smithi119 ceph-mon[31518]: pgmap v334: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 439 KiB/s rd, 437 KiB/s wr, 295 op/s 2024-03-28T00:36:22.954 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:36:22 smithi132 ceph-mon[35941]: pgmap v334: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 439 KiB/s rd, 437 KiB/s wr, 295 op/s 2024-03-28T00:36:24.884 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:36:24 smithi165 ceph-mon[36302]: pgmap v335: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 439 KiB/s rd, 437 KiB/s wr, 295 op/s 2024-03-28T00:36:24.907 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:36:24 smithi119 ceph-mon[31518]: pgmap v335: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 439 KiB/s rd, 437 KiB/s wr, 295 op/s 2024-03-28T00:36:24.954 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:36:24 smithi132 ceph-mon[35941]: pgmap v335: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 439 KiB/s rd, 437 KiB/s wr, 295 op/s 2024-03-28T00:36:26.884 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:36:26 smithi165 ceph-mon[36302]: pgmap v336: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 441 KiB/s rd, 437 KiB/s wr, 298 op/s 2024-03-28T00:36:26.907 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:36:26 smithi119 ceph-mon[31518]: pgmap v336: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 441 KiB/s rd, 437 KiB/s wr, 298 op/s 2024-03-28T00:36:26.954 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:36:26 smithi132 ceph-mon[35941]: pgmap v336: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 441 KiB/s rd, 437 KiB/s wr, 298 op/s 2024-03-28T00:36:28.884 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:36:28 smithi165 ceph-mon[36302]: pgmap v337: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 149 KiB/s wr, 6 op/s 2024-03-28T00:36:28.907 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:36:28 smithi119 ceph-mon[31518]: pgmap v337: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 149 KiB/s wr, 6 op/s 2024-03-28T00:36:28.954 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:36:28 smithi132 ceph-mon[35941]: pgmap v337: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 149 KiB/s wr, 6 op/s 2024-03-28T00:36:30.884 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:36:30 smithi165 ceph-mon[36302]: pgmap v338: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 149 KiB/s wr, 8 op/s 2024-03-28T00:36:30.887 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:36:30 smithi119 ceph-mon[31518]: pgmap v338: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 149 KiB/s wr, 8 op/s 2024-03-28T00:36:30.954 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:36:30 smithi132 ceph-mon[35941]: pgmap v338: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 149 KiB/s wr, 8 op/s 2024-03-28T00:36:32.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:36:32 smithi165 ceph-mon[36302]: pgmap v339: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-03-28T00:36:32.907 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:36:32 smithi119 ceph-mon[31518]: pgmap v339: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-03-28T00:36:32.954 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:36:32 smithi132 ceph-mon[35941]: pgmap v339: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-03-28T00:36:34.884 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:36:34 smithi165 ceph-mon[36302]: pgmap v340: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-03-28T00:36:34.907 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:36:34 smithi119 ceph-mon[31518]: pgmap v340: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-03-28T00:36:34.954 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:36:34 smithi132 ceph-mon[35941]: pgmap v340: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-03-28T00:36:36.884 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:36:36 smithi165 ceph-mon[36302]: pgmap v341: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 85 B/s wr, 7 op/s 2024-03-28T00:36:36.907 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:36:36 smithi119 ceph-mon[31518]: pgmap v341: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 85 B/s wr, 7 op/s 2024-03-28T00:36:36.954 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:36:36 smithi132 ceph-mon[35941]: pgmap v341: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 85 B/s wr, 7 op/s 2024-03-28T00:36:38.884 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:36:38 smithi165 ceph-mon[36302]: pgmap v342: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:36:38.907 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:36:38 smithi119 ceph-mon[31518]: pgmap v342: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:36:38.954 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:36:38 smithi132 ceph-mon[35941]: pgmap v342: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:36:40.884 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:36:40 smithi165 ceph-mon[36302]: pgmap v343: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:36:40.907 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:36:40 smithi119 ceph-mon[31518]: pgmap v343: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:36:40.954 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:36:40 smithi132 ceph-mon[35941]: pgmap v343: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:36:42.884 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:36:42 smithi165 ceph-mon[36302]: pgmap v344: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T00:36:42.907 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:36:42 smithi119 ceph-mon[31518]: pgmap v344: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T00:36:42.954 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:36:42 smithi132 ceph-mon[35941]: pgmap v344: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T00:36:44.884 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:36:44 smithi165 ceph-mon[36302]: pgmap v345: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T00:36:44.907 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:36:44 smithi119 ceph-mon[31518]: pgmap v345: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T00:36:44.954 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:36:44 smithi132 ceph-mon[35941]: pgmap v345: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T00:36:46.884 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:36:46 smithi165 ceph-mon[36302]: pgmap v346: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:36:46.907 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:36:46 smithi119 ceph-mon[31518]: pgmap v346: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:36:46.954 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:36:46 smithi132 ceph-mon[35941]: pgmap v346: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:36:48.884 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:36:48 smithi165 ceph-mon[36302]: pgmap v347: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T00:36:48.907 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:36:48 smithi119 ceph-mon[31518]: pgmap v347: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T00:36:48.954 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:36:48 smithi132 ceph-mon[35941]: pgmap v347: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T00:36:50.884 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:36:50 smithi165 ceph-mon[36302]: pgmap v348: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:36:50.907 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:36:50 smithi119 ceph-mon[31518]: pgmap v348: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:36:50.954 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:36:50 smithi132 ceph-mon[35941]: pgmap v348: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:36:52.884 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:36:52 smithi165 ceph-mon[36302]: pgmap v349: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:36:52.907 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:36:52 smithi119 ceph-mon[31518]: pgmap v349: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:36:52.954 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:36:52 smithi132 ceph-mon[35941]: pgmap v349: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:36:54.884 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:36:54 smithi165 ceph-mon[36302]: pgmap v350: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:36:54.907 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:36:54 smithi119 ceph-mon[31518]: pgmap v350: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:36:54.954 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:36:54 smithi132 ceph-mon[35941]: pgmap v350: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:36:56.884 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:36:56 smithi165 ceph-mon[36302]: pgmap v351: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:36:56.907 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:36:56 smithi119 ceph-mon[31518]: pgmap v351: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:36:56.954 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:36:56 smithi132 ceph-mon[35941]: pgmap v351: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:36:58.884 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:36:58 smithi165 ceph-mon[36302]: pgmap v352: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:36:58.907 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:36:58 smithi119 ceph-mon[31518]: pgmap v352: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:36:58.954 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:36:58 smithi132 ceph-mon[35941]: pgmap v352: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:37:00.884 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:37:00 smithi165 ceph-mon[36302]: pgmap v353: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:37:00.909 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:37:00 smithi119 ceph-mon[31518]: pgmap v353: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:37:00.954 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:37:00 smithi132 ceph-mon[35941]: pgmap v353: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:37:02.884 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:37:02 smithi165 ceph-mon[36302]: pgmap v354: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:37:02.907 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:37:02 smithi119 ceph-mon[31518]: pgmap v354: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:37:02.954 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:37:02 smithi132 ceph-mon[35941]: pgmap v354: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:37:04.884 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:37:04 smithi165 ceph-mon[36302]: pgmap v355: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:37:04.907 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:37:04 smithi119 ceph-mon[31518]: pgmap v355: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:37:04.954 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:37:04 smithi132 ceph-mon[35941]: pgmap v355: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:37:06.884 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:37:06 smithi165 ceph-mon[36302]: pgmap v356: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:37:06.885 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:37:06 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:37:06.885 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:37:06 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:37:06.885 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:37:06 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:37:06.885 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:37:06 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:37:06.907 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:37:06 smithi119 ceph-mon[31518]: pgmap v356: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:37:06.907 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:37:06 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:37:06.907 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:37:06 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:37:06.907 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:37:06 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:37:06.907 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:37:06 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:37:06.954 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:37:06 smithi132 ceph-mon[35941]: pgmap v356: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:37:06.954 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:37:06 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:37:06.954 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:37:06 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:37:06.955 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:37:06 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:37:06.955 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:37:06 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:37:07.884 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:37:07 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:37:07.907 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:37:07 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:37:07.954 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:37:07 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:37:08.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:37:08 smithi119 ceph-mon[31518]: pgmap v357: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:37:08.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:37:08 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:37:08.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:37:08 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:37:08.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:37:08 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:37:08.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:37:08 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:37:08.885 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:37:08 smithi165 ceph-mon[36302]: pgmap v357: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:37:08.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:37:08 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:37:08.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:37:08 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:37:08.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:37:08 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:37:08.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:37:08 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:37:08.954 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:37:08 smithi132 ceph-mon[35941]: pgmap v357: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:37:08.954 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:37:08 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:37:08.954 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:37:08 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:37:08.954 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:37:08 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:37:08.954 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:37:08 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:37:09.907 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:37:09 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:37:09.907 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:37:09 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:37:09.907 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:37:09 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:37:09.908 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:37:09 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:37:09.908 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:37:09 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:37:09.908 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:37:09 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:37:09.908 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:37:09 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:37:09.908 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:37:09 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:37:09.908 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:37:09 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:37:09.908 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:37:09 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:37:09.908 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:37:09 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:37:09.908 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:37:09 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:37:09.908 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:37:09 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:37:09.908 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:37:09 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:37:09.908 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:37:09 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:37:09.908 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:37:09 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:37:09.954 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:37:09 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:37:09.955 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:37:09 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:37:09.955 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:37:09 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:37:09.955 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:37:09 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:37:09.955 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:37:09 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:37:09.955 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:37:09 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:37:09.955 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:37:09 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:37:09.955 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:37:09 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:37:09.955 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:37:09 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:37:09.955 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:37:09 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:37:09.955 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:37:09 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:37:09.955 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:37:09 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:37:09.955 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:37:09 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:37:09.955 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:37:09 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:37:09.955 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:37:09 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:37:09.956 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:37:09 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:37:10.135 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:37:09 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:37:10.135 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:37:09 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:37:10.135 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:37:09 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:37:10.135 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:37:09 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:37:10.135 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:37:09 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:37:10.135 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:37:09 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:37:10.135 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:37:09 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:37:10.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:37:09 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:37:10.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:37:09 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:37:10.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:37:09 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:37:10.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:37:09 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:37:10.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:37:09 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:37:10.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:37:09 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:37:10.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:37:09 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:37:10.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:37:09 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:37:10.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:37:09 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:37:10.908 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:37:10 smithi119 ceph-mon[31518]: Adjusting osd_memory_target on smithi132 to 174.1M 2024-03-28T00:37:10.908 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:37:10 smithi119 ceph-mon[31518]: Unable to set osd_memory_target on smithi132 to 182640640: error parsing value: Value '182640640' is below minimum 939524096 2024-03-28T00:37:10.908 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:37:10 smithi119 ceph-mon[31518]: Adjusting osd_memory_target on smithi165 to 174.1M 2024-03-28T00:37:10.908 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:37:10 smithi119 ceph-mon[31518]: Unable to set osd_memory_target on smithi165 to 182635622: error parsing value: Value '182635622' is below minimum 939524096 2024-03-28T00:37:10.908 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:37:10 smithi119 ceph-mon[31518]: pgmap v358: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:37:10.954 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:37:10 smithi132 ceph-mon[35941]: Adjusting osd_memory_target on smithi132 to 174.1M 2024-03-28T00:37:10.955 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:37:10 smithi132 ceph-mon[35941]: Unable to set osd_memory_target on smithi132 to 182640640: error parsing value: Value '182640640' is below minimum 939524096 2024-03-28T00:37:10.955 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:37:10 smithi132 ceph-mon[35941]: Adjusting osd_memory_target on smithi165 to 174.1M 2024-03-28T00:37:10.955 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:37:10 smithi132 ceph-mon[35941]: Unable to set osd_memory_target on smithi165 to 182635622: error parsing value: Value '182635622' is below minimum 939524096 2024-03-28T00:37:10.955 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:37:10 smithi132 ceph-mon[35941]: pgmap v358: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:37:11.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:37:10 smithi165 ceph-mon[36302]: Adjusting osd_memory_target on smithi132 to 174.1M 2024-03-28T00:37:11.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:37:10 smithi165 ceph-mon[36302]: Unable to set osd_memory_target on smithi132 to 182640640: error parsing value: Value '182640640' is below minimum 939524096 2024-03-28T00:37:11.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:37:10 smithi165 ceph-mon[36302]: Adjusting osd_memory_target on smithi165 to 174.1M 2024-03-28T00:37:11.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:37:10 smithi165 ceph-mon[36302]: Unable to set osd_memory_target on smithi165 to 182635622: error parsing value: Value '182635622' is below minimum 939524096 2024-03-28T00:37:11.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:37:10 smithi165 ceph-mon[36302]: pgmap v358: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:37:12.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:37:12 smithi165 ceph-mon[36302]: pgmap v359: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:37:12.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:37:12 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:37:12.635 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:37:12 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:37:12.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:37:12 smithi119 ceph-mon[31518]: pgmap v359: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:37:12.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:37:12 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:37:12.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:37:12 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:37:12.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:37:12 smithi132 ceph-mon[35941]: pgmap v359: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:37:12.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:37:12 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:37:12.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:37:12 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:37:13.658 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:37:13 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:37:13.658 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:37:13 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:37:13.658 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:37:13 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:37:13.658 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:37:13 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:37:13.658 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:37:13 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:37:13.658 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:37:13 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:37:13.658 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:37:13 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:37:13.658 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:37:13 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:37:13.658 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:37:13 smithi119 ceph-mon[31518]: Adjusting osd_memory_target on smithi119 to 176.7M 2024-03-28T00:37:13.658 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:37:13 smithi119 ceph-mon[31518]: Unable to set osd_memory_target on smithi119 to 185372364: error parsing value: Value '185372364' is below minimum 939524096 2024-03-28T00:37:13.658 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:37:13 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:37:13.659 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:37:13 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:37:13.659 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:37:13 smithi119 ceph-mon[31518]: pgmap v360: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T00:37:13.659 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:37:13 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:37:13.705 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:37:13 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:37:13.705 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:37:13 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:37:13.705 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:37:13 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:37:13.705 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:37:13 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:37:13.705 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:37:13 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:37:13.705 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:37:13 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:37:13.705 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:37:13 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:37:13.705 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:37:13 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:37:13.705 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:37:13 smithi132 ceph-mon[35941]: Adjusting osd_memory_target on smithi119 to 176.7M 2024-03-28T00:37:13.705 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:37:13 smithi132 ceph-mon[35941]: Unable to set osd_memory_target on smithi119 to 185372364: error parsing value: Value '185372364' is below minimum 939524096 2024-03-28T00:37:13.705 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:37:13 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:37:13.705 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:37:13 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:37:13.705 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:37:13 smithi132 ceph-mon[35941]: pgmap v360: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T00:37:13.706 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:37:13 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:37:13.884 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:37:13 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:37:13.885 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:37:13 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:37:13.885 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:37:13 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:37:13.885 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:37:13 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:37:13.885 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:37:13 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:37:13.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:37:13 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:37:13.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:37:13 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:37:13.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:37:13 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:37:13.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:37:13 smithi165 ceph-mon[36302]: Adjusting osd_memory_target on smithi119 to 176.7M 2024-03-28T00:37:13.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:37:13 smithi165 ceph-mon[36302]: Unable to set osd_memory_target on smithi119 to 185372364: error parsing value: Value '185372364' is below minimum 939524096 2024-03-28T00:37:13.887 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:37:13 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:37:13.887 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:37:13 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:37:13.887 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:37:13 smithi165 ceph-mon[36302]: pgmap v360: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T00:37:13.887 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:37:13 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:37:14.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:37:14 smithi119 ceph-mon[31518]: Health check failed: 5 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-03-28T00:37:14.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:37:14 smithi132 ceph-mon[35941]: Health check failed: 5 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-03-28T00:37:14.884 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:37:14 smithi165 ceph-mon[36302]: Health check failed: 5 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-03-28T00:37:15.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:37:15 smithi119 ceph-mon[31518]: pgmap v361: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T00:37:15.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:37:15 smithi132 ceph-mon[35941]: pgmap v361: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T00:37:15.884 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:37:15 smithi165 ceph-mon[36302]: pgmap v361: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T00:37:17.954 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:37:17 smithi132 ceph-mon[35941]: pgmap v362: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T00:37:17.991 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:37:17 smithi119 ceph-mon[31518]: pgmap v362: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T00:37:18.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:37:17 smithi165 ceph-mon[36302]: pgmap v362: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T00:37:20.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:37:19 smithi165 ceph-mon[36302]: pgmap v363: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T00:37:20.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:37:19 smithi119 ceph-mon[31518]: pgmap v363: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T00:37:20.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:37:19 smithi132 ceph-mon[35941]: pgmap v363: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T00:37:22.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:37:21 smithi165 ceph-mon[36302]: pgmap v364: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T00:37:22.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:37:21 smithi119 ceph-mon[31518]: pgmap v364: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T00:37:22.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:37:21 smithi132 ceph-mon[35941]: pgmap v364: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T00:37:24.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:37:23 smithi165 ceph-mon[36302]: pgmap v365: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T00:37:24.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:37:23 smithi119 ceph-mon[31518]: pgmap v365: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T00:37:24.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:37:23 smithi132 ceph-mon[35941]: pgmap v365: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T00:37:26.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:37:25 smithi165 ceph-mon[36302]: pgmap v366: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:37:26.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:37:25 smithi119 ceph-mon[31518]: pgmap v366: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:37:26.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:37:25 smithi132 ceph-mon[35941]: pgmap v366: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:37:28.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:37:27 smithi165 ceph-mon[36302]: pgmap v367: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:37:28.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:37:27 smithi119 ceph-mon[31518]: pgmap v367: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:37:28.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:37:27 smithi132 ceph-mon[35941]: pgmap v367: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:37:30.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:37:29 smithi165 ceph-mon[36302]: pgmap v368: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:37:30.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:37:29 smithi119 ceph-mon[31518]: pgmap v368: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:37:30.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:37:29 smithi132 ceph-mon[35941]: pgmap v368: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:37:32.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:37:31 smithi165 ceph-mon[36302]: pgmap v369: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:37:32.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:37:31 smithi119 ceph-mon[31518]: pgmap v369: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:37:32.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:37:31 smithi132 ceph-mon[35941]: pgmap v369: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:37:34.005 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:37:33 smithi132 ceph-mon[35941]: pgmap v370: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:37:34.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:37:33 smithi165 ceph-mon[36302]: pgmap v370: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:37:34.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:37:33 smithi119 ceph-mon[31518]: pgmap v370: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:37:36.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:37:35 smithi165 ceph-mon[36302]: pgmap v371: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:37:36.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:37:35 smithi119 ceph-mon[31518]: pgmap v371: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:37:36.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:37:35 smithi132 ceph-mon[35941]: pgmap v371: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:37:38.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:37:37 smithi165 ceph-mon[36302]: pgmap v372: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:37:38.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:37:37 smithi119 ceph-mon[31518]: pgmap v372: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:37:38.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:37:37 smithi132 ceph-mon[35941]: pgmap v372: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:37:40.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:37:39 smithi165 ceph-mon[36302]: pgmap v373: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:37:40.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:37:39 smithi119 ceph-mon[31518]: pgmap v373: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:37:40.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:37:39 smithi132 ceph-mon[35941]: pgmap v373: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:37:42.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:37:41 smithi165 ceph-mon[36302]: pgmap v374: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:37:42.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:37:41 smithi119 ceph-mon[31518]: pgmap v374: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:37:42.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:37:41 smithi132 ceph-mon[35941]: pgmap v374: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:37:44.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:37:43 smithi165 ceph-mon[36302]: pgmap v375: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:37:44.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:37:43 smithi119 ceph-mon[31518]: pgmap v375: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:37:44.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:37:43 smithi132 ceph-mon[35941]: pgmap v375: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:37:46.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:37:45 smithi165 ceph-mon[36302]: pgmap v376: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:37:46.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:37:45 smithi119 ceph-mon[31518]: pgmap v376: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:37:46.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:37:45 smithi132 ceph-mon[35941]: pgmap v376: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:37:48.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:37:47 smithi165 ceph-mon[36302]: pgmap v377: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:37:48.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:37:47 smithi119 ceph-mon[31518]: pgmap v377: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:37:48.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:37:47 smithi132 ceph-mon[35941]: pgmap v377: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:37:50.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:37:49 smithi165 ceph-mon[36302]: pgmap v378: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:37:50.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:37:49 smithi119 ceph-mon[31518]: pgmap v378: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:37:50.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:37:49 smithi132 ceph-mon[35941]: pgmap v378: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:37:52.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:37:51 smithi165 ceph-mon[36302]: pgmap v379: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:37:52.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:37:51 smithi119 ceph-mon[31518]: pgmap v379: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:37:52.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:37:51 smithi132 ceph-mon[35941]: pgmap v379: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:37:54.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:37:53 smithi165 ceph-mon[36302]: pgmap v380: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:37:54.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:37:53 smithi119 ceph-mon[31518]: pgmap v380: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:37:54.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:37:53 smithi132 ceph-mon[35941]: pgmap v380: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:37:56.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:37:55 smithi165 ceph-mon[36302]: pgmap v381: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:37:56.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:37:55 smithi119 ceph-mon[31518]: pgmap v381: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:37:56.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:37:55 smithi132 ceph-mon[35941]: pgmap v381: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:37:58.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:37:57 smithi165 ceph-mon[36302]: pgmap v382: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T00:37:58.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:37:57 smithi119 ceph-mon[31518]: pgmap v382: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T00:37:58.205 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:37:57 smithi132 ceph-mon[35941]: pgmap v382: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T00:38:00.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:37:59 smithi165 ceph-mon[36302]: pgmap v383: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T00:38:00.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:37:59 smithi119 ceph-mon[31518]: pgmap v383: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T00:38:00.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:37:59 smithi132 ceph-mon[35941]: pgmap v383: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T00:38:02.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:38:01 smithi165 ceph-mon[36302]: pgmap v384: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:38:02.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:38:01 smithi119 ceph-mon[31518]: pgmap v384: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:38:02.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:38:01 smithi132 ceph-mon[35941]: pgmap v384: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:38:04.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:38:03 smithi165 ceph-mon[36302]: pgmap v385: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T00:38:04.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:38:03 smithi119 ceph-mon[31518]: pgmap v385: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T00:38:04.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:38:03 smithi132 ceph-mon[35941]: pgmap v385: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T00:38:06.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:38:05 smithi165 ceph-mon[36302]: pgmap v386: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-03-28T00:38:06.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:38:05 smithi119 ceph-mon[31518]: pgmap v386: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-03-28T00:38:06.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:38:05 smithi132 ceph-mon[35941]: pgmap v386: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-03-28T00:38:08.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:38:07 smithi165 ceph-mon[36302]: pgmap v387: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T00:38:08.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:38:07 smithi119 ceph-mon[31518]: pgmap v387: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T00:38:08.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:38:07 smithi132 ceph-mon[35941]: pgmap v387: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T00:38:10.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:38:09 smithi165 ceph-mon[36302]: pgmap v388: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T00:38:10.158 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:38:09 smithi119 ceph-mon[31518]: pgmap v388: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T00:38:10.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:38:09 smithi132 ceph-mon[35941]: pgmap v388: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T00:38:12.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:38:11 smithi165 ceph-mon[36302]: pgmap v389: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:38:12.158 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:38:11 smithi119 ceph-mon[31518]: pgmap v389: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:38:12.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:38:11 smithi132 ceph-mon[35941]: pgmap v389: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:38:13.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:38:12 smithi165 ceph-mon[36302]: pgmap v390: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T00:38:13.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:38:12 smithi119 ceph-mon[31518]: pgmap v390: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T00:38:13.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:38:12 smithi132 ceph-mon[35941]: pgmap v390: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T00:38:14.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:38:13 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:38:14.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:38:13 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:38:14.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:38:13 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:38:14.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:38:13 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:38:14.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:38:13 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:38:14.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:38:13 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:38:14.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:38:13 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:38:14.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:38:13 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:38:14.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:38:13 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:38:15.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:38:14 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:38:15.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:38:14 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:38:15.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:38:14 smithi165 ceph-mon[36302]: pgmap v391: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 7 op/s 2024-03-28T00:38:15.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:38:14 smithi165 ceph-mon[36302]: pgmap v392: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T00:38:15.135 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:38:14 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:38:15.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:38:14 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:38:15.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:38:14 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:38:15.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:38:14 smithi119 ceph-mon[31518]: pgmap v391: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 7 op/s 2024-03-28T00:38:15.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:38:14 smithi119 ceph-mon[31518]: pgmap v392: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T00:38:15.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:38:14 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:38:15.206 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:38:14 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:38:15.206 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:38:14 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:38:15.206 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:38:14 smithi132 ceph-mon[35941]: pgmap v391: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 7 op/s 2024-03-28T00:38:15.207 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:38:14 smithi132 ceph-mon[35941]: pgmap v392: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T00:38:15.207 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:38:14 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:38:17.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:38:17 smithi165 ceph-mon[36302]: pgmap v393: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T00:38:17.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:38:17 smithi119 ceph-mon[31518]: pgmap v393: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T00:38:17.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:38:17 smithi132 ceph-mon[35941]: pgmap v393: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T00:38:19.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:38:19 smithi165 ceph-mon[36302]: pgmap v394: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T00:38:19.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:38:19 smithi119 ceph-mon[31518]: pgmap v394: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T00:38:19.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:38:19 smithi132 ceph-mon[35941]: pgmap v394: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T00:38:21.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:38:21 smithi165 ceph-mon[36302]: pgmap v395: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T00:38:21.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:38:21 smithi119 ceph-mon[31518]: pgmap v395: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T00:38:21.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:38:21 smithi132 ceph-mon[35941]: pgmap v395: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T00:38:23.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:38:23 smithi165 ceph-mon[36302]: pgmap v396: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T00:38:23.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:38:23 smithi119 ceph-mon[31518]: pgmap v396: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T00:38:23.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:38:23 smithi132 ceph-mon[35941]: pgmap v396: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T00:38:25.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:38:25 smithi165 ceph-mon[36302]: pgmap v397: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T00:38:25.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:38:25 smithi119 ceph-mon[31518]: pgmap v397: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T00:38:25.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:38:25 smithi132 ceph-mon[35941]: pgmap v397: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T00:38:27.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:38:27 smithi165 ceph-mon[36302]: pgmap v398: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:38:27.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:38:27 smithi119 ceph-mon[31518]: pgmap v398: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:38:27.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:38:27 smithi132 ceph-mon[35941]: pgmap v398: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:38:29.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:38:29 smithi165 ceph-mon[36302]: pgmap v399: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:38:29.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:38:29 smithi119 ceph-mon[31518]: pgmap v399: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:38:29.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:38:29 smithi132 ceph-mon[35941]: pgmap v399: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:38:31.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:38:31 smithi165 ceph-mon[36302]: pgmap v400: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:38:31.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:38:31 smithi119 ceph-mon[31518]: pgmap v400: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:38:31.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:38:31 smithi132 ceph-mon[35941]: pgmap v400: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:38:33.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:38:33 smithi165 ceph-mon[36302]: pgmap v401: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:38:33.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:38:33 smithi119 ceph-mon[31518]: pgmap v401: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:38:33.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:38:33 smithi132 ceph-mon[35941]: pgmap v401: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:38:35.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:38:35 smithi165 ceph-mon[36302]: pgmap v402: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:38:35.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:38:35 smithi119 ceph-mon[31518]: pgmap v402: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:38:35.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:38:35 smithi132 ceph-mon[35941]: pgmap v402: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:38:37.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:38:37 smithi165 ceph-mon[36302]: pgmap v403: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:38:37.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:38:37 smithi119 ceph-mon[31518]: pgmap v403: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:38:37.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:38:37 smithi132 ceph-mon[35941]: pgmap v403: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:38:39.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:38:39 smithi165 ceph-mon[36302]: pgmap v404: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:38:39.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:38:39 smithi119 ceph-mon[31518]: pgmap v404: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:38:39.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:38:39 smithi132 ceph-mon[35941]: pgmap v404: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:38:41.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:38:41 smithi165 ceph-mon[36302]: pgmap v405: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:38:41.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:38:41 smithi119 ceph-mon[31518]: pgmap v405: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:38:41.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:38:41 smithi132 ceph-mon[35941]: pgmap v405: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:38:43.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:38:43 smithi165 ceph-mon[36302]: pgmap v406: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:38:43.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:38:43 smithi119 ceph-mon[31518]: pgmap v406: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:38:43.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:38:43 smithi132 ceph-mon[35941]: pgmap v406: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:38:45.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:38:45 smithi165 ceph-mon[36302]: pgmap v407: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:38:45.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:38:45 smithi165 ceph-mon[36302]: 47 slow requests, 5 included below; oldest blocked for > 185.664018 secs 2024-03-28T00:38:45.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:38:45 smithi165 ceph-mon[36302]: slow request 181.571355 seconds old, received at 2024-03-28T00:35:43.451608+0000: internal op fragmentdir:mds.0:3794 currently failed to wrlock, waiting 2024-03-28T00:38:45.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:38:45 smithi165 ceph-mon[36302]: slow request 181.553687 seconds old, received at 2024-03-28T00:35:43.469275+0000: internal op fragmentdir:mds.0:3825 currently failed to wrlock, waiting 2024-03-28T00:38:45.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:38:45 smithi165 ceph-mon[36302]: slow request 181.539595 seconds old, received at 2024-03-28T00:35:43.483367+0000: internal op fragmentdir:mds.0:3857 currently failed to wrlock, waiting 2024-03-28T00:38:45.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:38:45 smithi165 ceph-mon[36302]: slow request 181.526534 seconds old, received at 2024-03-28T00:35:43.496429+0000: internal op fragmentdir:mds.0:3889 currently failed to wrlock, waiting 2024-03-28T00:38:45.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:38:45 smithi165 ceph-mon[36302]: slow request 181.569379 seconds old, received at 2024-03-28T00:35:43.453584+0000: internal op fragmentdir:mds.0:3795 currently failed to wrlock, waiting 2024-03-28T00:38:45.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:38:45 smithi119 ceph-mon[31518]: pgmap v407: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:38:45.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:38:45 smithi119 ceph-mon[31518]: 47 slow requests, 5 included below; oldest blocked for > 185.664018 secs 2024-03-28T00:38:45.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:38:45 smithi119 ceph-mon[31518]: slow request 181.571355 seconds old, received at 2024-03-28T00:35:43.451608+0000: internal op fragmentdir:mds.0:3794 currently failed to wrlock, waiting 2024-03-28T00:38:45.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:38:45 smithi119 ceph-mon[31518]: slow request 181.553687 seconds old, received at 2024-03-28T00:35:43.469275+0000: internal op fragmentdir:mds.0:3825 currently failed to wrlock, waiting 2024-03-28T00:38:45.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:38:45 smithi119 ceph-mon[31518]: slow request 181.539595 seconds old, received at 2024-03-28T00:35:43.483367+0000: internal op fragmentdir:mds.0:3857 currently failed to wrlock, waiting 2024-03-28T00:38:45.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:38:45 smithi119 ceph-mon[31518]: slow request 181.526534 seconds old, received at 2024-03-28T00:35:43.496429+0000: internal op fragmentdir:mds.0:3889 currently failed to wrlock, waiting 2024-03-28T00:38:45.658 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:38:45 smithi119 ceph-mon[31518]: slow request 181.569379 seconds old, received at 2024-03-28T00:35:43.453584+0000: internal op fragmentdir:mds.0:3795 currently failed to wrlock, waiting 2024-03-28T00:38:45.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:38:45 smithi132 ceph-mon[35941]: pgmap v407: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:38:45.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:38:45 smithi132 ceph-mon[35941]: 47 slow requests, 5 included below; oldest blocked for > 185.664018 secs 2024-03-28T00:38:45.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:38:45 smithi132 ceph-mon[35941]: slow request 181.571355 seconds old, received at 2024-03-28T00:35:43.451608+0000: internal op fragmentdir:mds.0:3794 currently failed to wrlock, waiting 2024-03-28T00:38:45.705 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:38:45 smithi132 ceph-mon[35941]: slow request 181.553687 seconds old, received at 2024-03-28T00:35:43.469275+0000: internal op fragmentdir:mds.0:3825 currently failed to wrlock, waiting 2024-03-28T00:38:45.705 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:38:45 smithi132 ceph-mon[35941]: slow request 181.539595 seconds old, received at 2024-03-28T00:35:43.483367+0000: internal op fragmentdir:mds.0:3857 currently failed to wrlock, waiting 2024-03-28T00:38:45.705 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:38:45 smithi132 ceph-mon[35941]: slow request 181.526534 seconds old, received at 2024-03-28T00:35:43.496429+0000: internal op fragmentdir:mds.0:3889 currently failed to wrlock, waiting 2024-03-28T00:38:45.705 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:38:45 smithi132 ceph-mon[35941]: slow request 181.569379 seconds old, received at 2024-03-28T00:35:43.453584+0000: internal op fragmentdir:mds.0:3795 currently failed to wrlock, waiting 2024-03-28T00:38:46.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:38:46 smithi165 ceph-mon[36302]: Health check failed: 1 MDSs report slow requests (MDS_SLOW_REQUEST) 2024-03-28T00:38:46.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:38:46 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.132:6838/3305822525,v1:172.21.15.132:6839/3305822525] up:active 2024-03-28T00:38:46.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:38:46 smithi165 ceph-mon[36302]: fsmap cephfs:5/5 3 up:standby-replay 1 up:resolve 4 up:active 2024-03-28T00:38:46.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:38:46 smithi119 ceph-mon[31518]: Health check failed: 1 MDSs report slow requests (MDS_SLOW_REQUEST) 2024-03-28T00:38:46.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:38:46 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.132:6838/3305822525,v1:172.21.15.132:6839/3305822525] up:active 2024-03-28T00:38:46.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:38:46 smithi119 ceph-mon[31518]: fsmap cephfs:5/5 3 up:standby-replay 1 up:resolve 4 up:active 2024-03-28T00:38:46.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:38:46 smithi132 ceph-mon[35941]: Health check failed: 1 MDSs report slow requests (MDS_SLOW_REQUEST) 2024-03-28T00:38:46.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:38:46 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.132:6838/3305822525,v1:172.21.15.132:6839/3305822525] up:active 2024-03-28T00:38:46.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:38:46 smithi132 ceph-mon[35941]: fsmap cephfs:5/5 3 up:standby-replay 1 up:resolve 4 up:active 2024-03-28T00:38:47.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:38:47 smithi165 ceph-mon[36302]: pgmap v408: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:38:47.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:38:47 smithi119 ceph-mon[31518]: pgmap v408: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:38:47.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:38:47 smithi132 ceph-mon[35941]: pgmap v408: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:38:49.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:38:49 smithi165 ceph-mon[36302]: pgmap v409: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:38:49.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:38:49 smithi119 ceph-mon[31518]: pgmap v409: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:38:49.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:38:49 smithi132 ceph-mon[35941]: pgmap v409: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:38:50.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:38:50 smithi165 ceph-mon[36302]: 47 slow requests, 5 included below; oldest blocked for > 190.664106 secs 2024-03-28T00:38:50.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:38:50 smithi165 ceph-mon[36302]: slow request 186.488204 seconds old, received at 2024-03-28T00:35:43.534847+0000: internal op fragmentdir:mds.0:3922 currently failed to wrlock, waiting 2024-03-28T00:38:50.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:38:50 smithi165 ceph-mon[36302]: slow request 190.444131 seconds old, received at 2024-03-28T00:35:39.578920+0000: internal op fragmentdir:mds.0:3783 currently failed to wrlock, waiting 2024-03-28T00:38:50.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:38:50 smithi165 ceph-mon[36302]: slow request 186.568946 seconds old, received at 2024-03-28T00:35:43.454104+0000: internal op fragmentdir:mds.0:3796 currently failed to wrlock, waiting 2024-03-28T00:38:50.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:38:50 smithi165 ceph-mon[36302]: slow request 186.552887 seconds old, received at 2024-03-28T00:35:43.470163+0000: internal op fragmentdir:mds.0:3827 currently failed to wrlock, waiting 2024-03-28T00:38:50.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:38:50 smithi165 ceph-mon[36302]: slow request 186.525724 seconds old, received at 2024-03-28T00:35:43.497326+0000: internal op fragmentdir:mds.0:3891 currently failed to wrlock, waiting 2024-03-28T00:38:50.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:38:50 smithi119 ceph-mon[31518]: 47 slow requests, 5 included below; oldest blocked for > 190.664106 secs 2024-03-28T00:38:50.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:38:50 smithi119 ceph-mon[31518]: slow request 186.488204 seconds old, received at 2024-03-28T00:35:43.534847+0000: internal op fragmentdir:mds.0:3922 currently failed to wrlock, waiting 2024-03-28T00:38:50.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:38:50 smithi119 ceph-mon[31518]: slow request 190.444131 seconds old, received at 2024-03-28T00:35:39.578920+0000: internal op fragmentdir:mds.0:3783 currently failed to wrlock, waiting 2024-03-28T00:38:50.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:38:50 smithi119 ceph-mon[31518]: slow request 186.568946 seconds old, received at 2024-03-28T00:35:43.454104+0000: internal op fragmentdir:mds.0:3796 currently failed to wrlock, waiting 2024-03-28T00:38:50.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:38:50 smithi119 ceph-mon[31518]: slow request 186.552887 seconds old, received at 2024-03-28T00:35:43.470163+0000: internal op fragmentdir:mds.0:3827 currently failed to wrlock, waiting 2024-03-28T00:38:50.658 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:38:50 smithi119 ceph-mon[31518]: slow request 186.525724 seconds old, received at 2024-03-28T00:35:43.497326+0000: internal op fragmentdir:mds.0:3891 currently failed to wrlock, waiting 2024-03-28T00:38:50.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:38:50 smithi132 ceph-mon[35941]: 47 slow requests, 5 included below; oldest blocked for > 190.664106 secs 2024-03-28T00:38:50.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:38:50 smithi132 ceph-mon[35941]: slow request 186.488204 seconds old, received at 2024-03-28T00:35:43.534847+0000: internal op fragmentdir:mds.0:3922 currently failed to wrlock, waiting 2024-03-28T00:38:50.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:38:50 smithi132 ceph-mon[35941]: slow request 190.444131 seconds old, received at 2024-03-28T00:35:39.578920+0000: internal op fragmentdir:mds.0:3783 currently failed to wrlock, waiting 2024-03-28T00:38:50.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:38:50 smithi132 ceph-mon[35941]: slow request 186.568946 seconds old, received at 2024-03-28T00:35:43.454104+0000: internal op fragmentdir:mds.0:3796 currently failed to wrlock, waiting 2024-03-28T00:38:50.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:38:50 smithi132 ceph-mon[35941]: slow request 186.552887 seconds old, received at 2024-03-28T00:35:43.470163+0000: internal op fragmentdir:mds.0:3827 currently failed to wrlock, waiting 2024-03-28T00:38:50.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:38:50 smithi132 ceph-mon[35941]: slow request 186.525724 seconds old, received at 2024-03-28T00:35:43.497326+0000: internal op fragmentdir:mds.0:3891 currently failed to wrlock, waiting 2024-03-28T00:38:51.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:38:51 smithi165 ceph-mon[36302]: pgmap v410: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:38:51.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:38:51 smithi119 ceph-mon[31518]: pgmap v410: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:38:51.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:38:51 smithi132 ceph-mon[35941]: pgmap v410: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:38:53.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:38:53 smithi165 ceph-mon[36302]: pgmap v411: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:38:53.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:38:53 smithi119 ceph-mon[31518]: pgmap v411: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:38:53.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:38:53 smithi132 ceph-mon[35941]: pgmap v411: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:38:55.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:38:55 smithi165 ceph-mon[36302]: pgmap v412: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:38:55.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:38:55 smithi165 ceph-mon[36302]: 47 slow requests, 5 included below; oldest blocked for > 195.664196 secs 2024-03-28T00:38:55.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:38:55 smithi165 ceph-mon[36302]: slow request 191.461564 seconds old, received at 2024-03-28T00:35:43.561577+0000: internal op fragmentdir:mds.0:3923 currently failed to wrlock, waiting 2024-03-28T00:38:55.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:38:55 smithi165 ceph-mon[36302]: slow request 191.568563 seconds old, received at 2024-03-28T00:35:43.454578+0000: internal op fragmentdir:mds.0:3797 currently failed to wrlock, waiting 2024-03-28T00:38:55.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:38:55 smithi165 ceph-mon[36302]: slow request 191.552536 seconds old, received at 2024-03-28T00:35:43.470605+0000: internal op fragmentdir:mds.0:3828 currently failed to wrlock, waiting 2024-03-28T00:38:55.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:38:55 smithi165 ceph-mon[36302]: slow request 191.538571 seconds old, received at 2024-03-28T00:35:43.484570+0000: internal op fragmentdir:mds.0:3860 currently failed to wrlock, waiting 2024-03-28T00:38:55.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:38:55 smithi165 ceph-mon[36302]: slow request 191.525365 seconds old, received at 2024-03-28T00:35:43.497776+0000: internal op fragmentdir:mds.0:3892 currently failed to wrlock, waiting 2024-03-28T00:38:55.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:38:55 smithi119 ceph-mon[31518]: pgmap v412: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:38:55.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:38:55 smithi119 ceph-mon[31518]: 47 slow requests, 5 included below; oldest blocked for > 195.664196 secs 2024-03-28T00:38:55.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:38:55 smithi119 ceph-mon[31518]: slow request 191.461564 seconds old, received at 2024-03-28T00:35:43.561577+0000: internal op fragmentdir:mds.0:3923 currently failed to wrlock, waiting 2024-03-28T00:38:55.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:38:55 smithi119 ceph-mon[31518]: slow request 191.568563 seconds old, received at 2024-03-28T00:35:43.454578+0000: internal op fragmentdir:mds.0:3797 currently failed to wrlock, waiting 2024-03-28T00:38:55.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:38:55 smithi119 ceph-mon[31518]: slow request 191.552536 seconds old, received at 2024-03-28T00:35:43.470605+0000: internal op fragmentdir:mds.0:3828 currently failed to wrlock, waiting 2024-03-28T00:38:55.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:38:55 smithi119 ceph-mon[31518]: slow request 191.538571 seconds old, received at 2024-03-28T00:35:43.484570+0000: internal op fragmentdir:mds.0:3860 currently failed to wrlock, waiting 2024-03-28T00:38:55.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:38:55 smithi119 ceph-mon[31518]: slow request 191.525365 seconds old, received at 2024-03-28T00:35:43.497776+0000: internal op fragmentdir:mds.0:3892 currently failed to wrlock, waiting 2024-03-28T00:38:55.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:38:55 smithi132 ceph-mon[35941]: pgmap v412: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:38:55.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:38:55 smithi132 ceph-mon[35941]: 47 slow requests, 5 included below; oldest blocked for > 195.664196 secs 2024-03-28T00:38:55.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:38:55 smithi132 ceph-mon[35941]: slow request 191.461564 seconds old, received at 2024-03-28T00:35:43.561577+0000: internal op fragmentdir:mds.0:3923 currently failed to wrlock, waiting 2024-03-28T00:38:55.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:38:55 smithi132 ceph-mon[35941]: slow request 191.568563 seconds old, received at 2024-03-28T00:35:43.454578+0000: internal op fragmentdir:mds.0:3797 currently failed to wrlock, waiting 2024-03-28T00:38:55.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:38:55 smithi132 ceph-mon[35941]: slow request 191.552536 seconds old, received at 2024-03-28T00:35:43.470605+0000: internal op fragmentdir:mds.0:3828 currently failed to wrlock, waiting 2024-03-28T00:38:55.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:38:55 smithi132 ceph-mon[35941]: slow request 191.538571 seconds old, received at 2024-03-28T00:35:43.484570+0000: internal op fragmentdir:mds.0:3860 currently failed to wrlock, waiting 2024-03-28T00:38:55.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:38:55 smithi132 ceph-mon[35941]: slow request 191.525365 seconds old, received at 2024-03-28T00:35:43.497776+0000: internal op fragmentdir:mds.0:3892 currently failed to wrlock, waiting 2024-03-28T00:38:57.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:38:57 smithi165 ceph-mon[36302]: pgmap v413: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:38:57.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:38:57 smithi119 ceph-mon[31518]: pgmap v413: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:38:57.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:38:57 smithi132 ceph-mon[35941]: pgmap v413: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:38:59.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:38:59 smithi165 ceph-mon[36302]: pgmap v414: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:38:59.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:38:59 smithi119 ceph-mon[31518]: pgmap v414: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:38:59.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:38:59 smithi132 ceph-mon[35941]: pgmap v414: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:39:01.384 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:39:00 smithi165 ceph-mon[36302]: 47 slow requests, 5 included below; oldest blocked for > 200.664284 secs 2024-03-28T00:39:01.384 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:39:00 smithi165 ceph-mon[36302]: slow request 196.568174 seconds old, received at 2024-03-28T00:35:43.455055+0000: internal op fragmentdir:mds.0:3798 currently failed to wrlock, waiting 2024-03-28T00:39:01.384 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:39:00 smithi165 ceph-mon[36302]: slow request 196.524111 seconds old, received at 2024-03-28T00:35:43.499118+0000: internal op fragmentdir:mds.0:3893 currently failed to wrlock, waiting 2024-03-28T00:39:01.384 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:39:00 smithi165 ceph-mon[36302]: slow request 196.567700 seconds old, received at 2024-03-28T00:35:43.455529+0000: internal op fragmentdir:mds.0:3799 currently failed to wrlock, waiting 2024-03-28T00:39:01.384 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:39:00 smithi165 ceph-mon[36302]: slow request 196.537831 seconds old, received at 2024-03-28T00:35:43.485397+0000: internal op fragmentdir:mds.0:3862 currently failed to wrlock, waiting 2024-03-28T00:39:01.384 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:39:00 smithi165 ceph-mon[36302]: slow request 196.523661 seconds old, received at 2024-03-28T00:35:43.499568+0000: internal op fragmentdir:mds.0:3894 currently failed to wrlock, waiting 2024-03-28T00:39:01.384 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:39:00 smithi165 ceph-mon[36302]: pgmap v415: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:39:01.407 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:39:00 smithi119 ceph-mon[31518]: 47 slow requests, 5 included below; oldest blocked for > 200.664284 secs 2024-03-28T00:39:01.407 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:39:00 smithi119 ceph-mon[31518]: slow request 196.568174 seconds old, received at 2024-03-28T00:35:43.455055+0000: internal op fragmentdir:mds.0:3798 currently failed to wrlock, waiting 2024-03-28T00:39:01.407 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:39:00 smithi119 ceph-mon[31518]: slow request 196.524111 seconds old, received at 2024-03-28T00:35:43.499118+0000: internal op fragmentdir:mds.0:3893 currently failed to wrlock, waiting 2024-03-28T00:39:01.407 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:39:00 smithi119 ceph-mon[31518]: slow request 196.567700 seconds old, received at 2024-03-28T00:35:43.455529+0000: internal op fragmentdir:mds.0:3799 currently failed to wrlock, waiting 2024-03-28T00:39:01.407 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:39:00 smithi119 ceph-mon[31518]: slow request 196.537831 seconds old, received at 2024-03-28T00:35:43.485397+0000: internal op fragmentdir:mds.0:3862 currently failed to wrlock, waiting 2024-03-28T00:39:01.407 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:39:00 smithi119 ceph-mon[31518]: slow request 196.523661 seconds old, received at 2024-03-28T00:35:43.499568+0000: internal op fragmentdir:mds.0:3894 currently failed to wrlock, waiting 2024-03-28T00:39:01.407 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:39:00 smithi119 ceph-mon[31518]: pgmap v415: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:39:01.454 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:39:00 smithi132 ceph-mon[35941]: 47 slow requests, 5 included below; oldest blocked for > 200.664284 secs 2024-03-28T00:39:01.454 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:39:00 smithi132 ceph-mon[35941]: slow request 196.568174 seconds old, received at 2024-03-28T00:35:43.455055+0000: internal op fragmentdir:mds.0:3798 currently failed to wrlock, waiting 2024-03-28T00:39:01.454 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:39:00 smithi132 ceph-mon[35941]: slow request 196.524111 seconds old, received at 2024-03-28T00:35:43.499118+0000: internal op fragmentdir:mds.0:3893 currently failed to wrlock, waiting 2024-03-28T00:39:01.454 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:39:00 smithi132 ceph-mon[35941]: slow request 196.567700 seconds old, received at 2024-03-28T00:35:43.455529+0000: internal op fragmentdir:mds.0:3799 currently failed to wrlock, waiting 2024-03-28T00:39:01.454 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:39:00 smithi132 ceph-mon[35941]: slow request 196.537831 seconds old, received at 2024-03-28T00:35:43.485397+0000: internal op fragmentdir:mds.0:3862 currently failed to wrlock, waiting 2024-03-28T00:39:01.454 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:39:00 smithi132 ceph-mon[35941]: slow request 196.523661 seconds old, received at 2024-03-28T00:35:43.499568+0000: internal op fragmentdir:mds.0:3894 currently failed to wrlock, waiting 2024-03-28T00:39:01.454 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:39:00 smithi132 ceph-mon[35941]: pgmap v415: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:39:03.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:39:03 smithi165 ceph-mon[36302]: pgmap v416: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:39:03.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:39:03 smithi119 ceph-mon[31518]: pgmap v416: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:39:03.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:39:03 smithi132 ceph-mon[35941]: pgmap v416: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:39:05.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:39:05 smithi165 ceph-mon[36302]: pgmap v417: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:39:05.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:39:05 smithi165 ceph-mon[36302]: 98 slow requests, 5 included below; oldest blocked for > 205.664344 secs 2024-03-28T00:39:05.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:39:05 smithi165 ceph-mon[36302]: slow request 201.567281 seconds old, received at 2024-03-28T00:35:43.456007+0000: internal op fragmentdir:mds.0:3800 currently failed to wrlock, waiting 2024-03-28T00:39:05.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:39:05 smithi165 ceph-mon[36302]: slow request 201.551616 seconds old, received at 2024-03-28T00:35:43.471672+0000: internal op fragmentdir:mds.0:3831 currently failed to wrlock, waiting 2024-03-28T00:39:05.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:39:05 smithi165 ceph-mon[36302]: slow request 201.537500 seconds old, received at 2024-03-28T00:35:43.485788+0000: internal op fragmentdir:mds.0:3863 currently failed to wrlock, waiting 2024-03-28T00:39:05.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:39:05 smithi165 ceph-mon[36302]: slow request 201.523257 seconds old, received at 2024-03-28T00:35:43.500032+0000: internal op fragmentdir:mds.0:3895 currently failed to wrlock, waiting 2024-03-28T00:39:05.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:39:05 smithi165 ceph-mon[36302]: slow request 205.435421 seconds old, received at 2024-03-28T00:35:39.587868+0000: client_request(client.25264:33202 create owner_uid=1000, owner_gid=1301 #0x10000000270/file8309 2024-03-28T00:35:39.586399+0000 caller_uid=1000, caller_gid=1301{6,36,1000,1301,}) currently failed to xlock, waiting 2024-03-28T00:39:05.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:39:05 smithi119 ceph-mon[31518]: pgmap v417: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:39:05.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:39:05 smithi119 ceph-mon[31518]: 98 slow requests, 5 included below; oldest blocked for > 205.664344 secs 2024-03-28T00:39:05.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:39:05 smithi119 ceph-mon[31518]: slow request 201.567281 seconds old, received at 2024-03-28T00:35:43.456007+0000: internal op fragmentdir:mds.0:3800 currently failed to wrlock, waiting 2024-03-28T00:39:05.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:39:05 smithi119 ceph-mon[31518]: slow request 201.551616 seconds old, received at 2024-03-28T00:35:43.471672+0000: internal op fragmentdir:mds.0:3831 currently failed to wrlock, waiting 2024-03-28T00:39:05.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:39:05 smithi119 ceph-mon[31518]: slow request 201.537500 seconds old, received at 2024-03-28T00:35:43.485788+0000: internal op fragmentdir:mds.0:3863 currently failed to wrlock, waiting 2024-03-28T00:39:05.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:39:05 smithi119 ceph-mon[31518]: slow request 201.523257 seconds old, received at 2024-03-28T00:35:43.500032+0000: internal op fragmentdir:mds.0:3895 currently failed to wrlock, waiting 2024-03-28T00:39:05.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:39:05 smithi119 ceph-mon[31518]: slow request 205.435421 seconds old, received at 2024-03-28T00:35:39.587868+0000: client_request(client.25264:33202 create owner_uid=1000, owner_gid=1301 #0x10000000270/file8309 2024-03-28T00:35:39.586399+0000 caller_uid=1000, caller_gid=1301{6,36,1000,1301,}) currently failed to xlock, waiting 2024-03-28T00:39:05.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:39:05 smithi132 ceph-mon[35941]: pgmap v417: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:39:05.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:39:05 smithi132 ceph-mon[35941]: 98 slow requests, 5 included below; oldest blocked for > 205.664344 secs 2024-03-28T00:39:05.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:39:05 smithi132 ceph-mon[35941]: slow request 201.567281 seconds old, received at 2024-03-28T00:35:43.456007+0000: internal op fragmentdir:mds.0:3800 currently failed to wrlock, waiting 2024-03-28T00:39:05.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:39:05 smithi132 ceph-mon[35941]: slow request 201.551616 seconds old, received at 2024-03-28T00:35:43.471672+0000: internal op fragmentdir:mds.0:3831 currently failed to wrlock, waiting 2024-03-28T00:39:05.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:39:05 smithi132 ceph-mon[35941]: slow request 201.537500 seconds old, received at 2024-03-28T00:35:43.485788+0000: internal op fragmentdir:mds.0:3863 currently failed to wrlock, waiting 2024-03-28T00:39:05.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:39:05 smithi132 ceph-mon[35941]: slow request 201.523257 seconds old, received at 2024-03-28T00:35:43.500032+0000: internal op fragmentdir:mds.0:3895 currently failed to wrlock, waiting 2024-03-28T00:39:05.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:39:05 smithi132 ceph-mon[35941]: slow request 205.435421 seconds old, received at 2024-03-28T00:35:39.587868+0000: client_request(client.25264:33202 create owner_uid=1000, owner_gid=1301 #0x10000000270/file8309 2024-03-28T00:35:39.586399+0000 caller_uid=1000, caller_gid=1301{6,36,1000,1301,}) currently failed to xlock, waiting 2024-03-28T00:39:06.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:39:06 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.132:6838/3305822525,v1:172.21.15.132:6839/3305822525] up:active 2024-03-28T00:39:06.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:39:06 smithi165 ceph-mon[36302]: fsmap cephfs:5/5 3 up:standby-replay 1 up:resolve 4 up:active 2024-03-28T00:39:06.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:39:06 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.132:6838/3305822525,v1:172.21.15.132:6839/3305822525] up:active 2024-03-28T00:39:06.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:39:06 smithi119 ceph-mon[31518]: fsmap cephfs:5/5 3 up:standby-replay 1 up:resolve 4 up:active 2024-03-28T00:39:06.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:39:06 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.132:6838/3305822525,v1:172.21.15.132:6839/3305822525] up:active 2024-03-28T00:39:06.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:39:06 smithi132 ceph-mon[35941]: fsmap cephfs:5/5 3 up:standby-replay 1 up:resolve 4 up:active 2024-03-28T00:39:07.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:39:07 smithi165 ceph-mon[36302]: pgmap v418: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:39:07.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:39:07 smithi119 ceph-mon[31518]: pgmap v418: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:39:07.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:39:07 smithi132 ceph-mon[35941]: pgmap v418: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:39:09.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:39:09 smithi165 ceph-mon[36302]: pgmap v419: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:39:09.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:39:09 smithi119 ceph-mon[31518]: pgmap v419: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:39:09.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:39:09 smithi132 ceph-mon[35941]: pgmap v419: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:39:10.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:39:10 smithi165 ceph-mon[36302]: 98 slow requests, 5 included below; oldest blocked for > 210.664434 secs 2024-03-28T00:39:10.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:39:10 smithi165 ceph-mon[36302]: slow request 206.566915 seconds old, received at 2024-03-28T00:35:43.456464+0000: internal op fragmentdir:mds.0:3801 currently failed to wrlock, waiting 2024-03-28T00:39:10.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:39:10 smithi165 ceph-mon[36302]: slow request 206.551248 seconds old, received at 2024-03-28T00:35:43.472131+0000: internal op fragmentdir:mds.0:3832 currently failed to wrlock, waiting 2024-03-28T00:39:10.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:39:10 smithi165 ceph-mon[36302]: slow request 206.566398 seconds old, received at 2024-03-28T00:35:43.456981+0000: internal op fragmentdir:mds.0:3802 currently failed to wrlock, waiting 2024-03-28T00:39:10.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:39:10 smithi165 ceph-mon[36302]: slow request 206.536789 seconds old, received at 2024-03-28T00:35:43.486590+0000: internal op fragmentdir:mds.0:3865 currently failed to wrlock, waiting 2024-03-28T00:39:10.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:39:10 smithi165 ceph-mon[36302]: slow request 206.522475 seconds old, received at 2024-03-28T00:35:43.500904+0000: internal op fragmentdir:mds.0:3897 currently failed to wrlock, waiting 2024-03-28T00:39:10.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:39:10 smithi119 ceph-mon[31518]: 98 slow requests, 5 included below; oldest blocked for > 210.664434 secs 2024-03-28T00:39:10.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:39:10 smithi119 ceph-mon[31518]: slow request 206.566915 seconds old, received at 2024-03-28T00:35:43.456464+0000: internal op fragmentdir:mds.0:3801 currently failed to wrlock, waiting 2024-03-28T00:39:10.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:39:10 smithi119 ceph-mon[31518]: slow request 206.551248 seconds old, received at 2024-03-28T00:35:43.472131+0000: internal op fragmentdir:mds.0:3832 currently failed to wrlock, waiting 2024-03-28T00:39:10.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:39:10 smithi119 ceph-mon[31518]: slow request 206.566398 seconds old, received at 2024-03-28T00:35:43.456981+0000: internal op fragmentdir:mds.0:3802 currently failed to wrlock, waiting 2024-03-28T00:39:10.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:39:10 smithi119 ceph-mon[31518]: slow request 206.536789 seconds old, received at 2024-03-28T00:35:43.486590+0000: internal op fragmentdir:mds.0:3865 currently failed to wrlock, waiting 2024-03-28T00:39:10.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:39:10 smithi119 ceph-mon[31518]: slow request 206.522475 seconds old, received at 2024-03-28T00:35:43.500904+0000: internal op fragmentdir:mds.0:3897 currently failed to wrlock, waiting 2024-03-28T00:39:10.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:39:10 smithi132 ceph-mon[35941]: 98 slow requests, 5 included below; oldest blocked for > 210.664434 secs 2024-03-28T00:39:10.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:39:10 smithi132 ceph-mon[35941]: slow request 206.566915 seconds old, received at 2024-03-28T00:35:43.456464+0000: internal op fragmentdir:mds.0:3801 currently failed to wrlock, waiting 2024-03-28T00:39:10.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:39:10 smithi132 ceph-mon[35941]: slow request 206.551248 seconds old, received at 2024-03-28T00:35:43.472131+0000: internal op fragmentdir:mds.0:3832 currently failed to wrlock, waiting 2024-03-28T00:39:10.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:39:10 smithi132 ceph-mon[35941]: slow request 206.566398 seconds old, received at 2024-03-28T00:35:43.456981+0000: internal op fragmentdir:mds.0:3802 currently failed to wrlock, waiting 2024-03-28T00:39:10.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:39:10 smithi132 ceph-mon[35941]: slow request 206.536789 seconds old, received at 2024-03-28T00:35:43.486590+0000: internal op fragmentdir:mds.0:3865 currently failed to wrlock, waiting 2024-03-28T00:39:10.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:39:10 smithi132 ceph-mon[35941]: slow request 206.522475 seconds old, received at 2024-03-28T00:35:43.500904+0000: internal op fragmentdir:mds.0:3897 currently failed to wrlock, waiting 2024-03-28T00:39:11.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:39:11 smithi165 ceph-mon[36302]: pgmap v420: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:39:11.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:39:11 smithi119 ceph-mon[31518]: pgmap v420: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:39:11.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:39:11 smithi132 ceph-mon[35941]: pgmap v420: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:39:13.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:39:13 smithi165 ceph-mon[36302]: pgmap v421: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:39:13.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:39:13 smithi119 ceph-mon[31518]: pgmap v421: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:39:13.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:39:13 smithi132 ceph-mon[35941]: pgmap v421: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:39:15.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:39:15 smithi165 ceph-mon[36302]: pgmap v422: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:39:15.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:39:15 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:39:15.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:39:15 smithi165 ceph-mon[36302]: 98 slow requests, 5 included below; oldest blocked for > 215.664524 secs 2024-03-28T00:39:15.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:39:15 smithi165 ceph-mon[36302]: slow request 211.566039 seconds old, received at 2024-03-28T00:35:43.457430+0000: internal op fragmentdir:mds.0:3803 currently failed to wrlock, waiting 2024-03-28T00:39:15.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:39:15 smithi165 ceph-mon[36302]: slow request 211.550453 seconds old, received at 2024-03-28T00:35:43.473016+0000: internal op fragmentdir:mds.0:3834 currently failed to wrlock, waiting 2024-03-28T00:39:15.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:39:15 smithi165 ceph-mon[36302]: slow request 211.565533 seconds old, received at 2024-03-28T00:35:43.457937+0000: internal op fragmentdir:mds.0:3804 currently failed to wrlock, waiting 2024-03-28T00:39:15.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:39:15 smithi165 ceph-mon[36302]: slow request 211.536084 seconds old, received at 2024-03-28T00:35:43.487385+0000: internal op fragmentdir:mds.0:3867 currently failed to wrlock, waiting 2024-03-28T00:39:15.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:39:15 smithi165 ceph-mon[36302]: slow request 211.521698 seconds old, received at 2024-03-28T00:35:43.501771+0000: internal op fragmentdir:mds.0:3899 currently failed to wrlock, waiting 2024-03-28T00:39:15.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:39:15 smithi119 ceph-mon[31518]: pgmap v422: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:39:15.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:39:15 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:39:15.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:39:15 smithi119 ceph-mon[31518]: 98 slow requests, 5 included below; oldest blocked for > 215.664524 secs 2024-03-28T00:39:15.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:39:15 smithi119 ceph-mon[31518]: slow request 211.566039 seconds old, received at 2024-03-28T00:35:43.457430+0000: internal op fragmentdir:mds.0:3803 currently failed to wrlock, waiting 2024-03-28T00:39:15.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:39:15 smithi119 ceph-mon[31518]: slow request 211.550453 seconds old, received at 2024-03-28T00:35:43.473016+0000: internal op fragmentdir:mds.0:3834 currently failed to wrlock, waiting 2024-03-28T00:39:15.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:39:15 smithi119 ceph-mon[31518]: slow request 211.565533 seconds old, received at 2024-03-28T00:35:43.457937+0000: internal op fragmentdir:mds.0:3804 currently failed to wrlock, waiting 2024-03-28T00:39:15.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:39:15 smithi119 ceph-mon[31518]: slow request 211.536084 seconds old, received at 2024-03-28T00:35:43.487385+0000: internal op fragmentdir:mds.0:3867 currently failed to wrlock, waiting 2024-03-28T00:39:15.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:39:15 smithi119 ceph-mon[31518]: slow request 211.521698 seconds old, received at 2024-03-28T00:35:43.501771+0000: internal op fragmentdir:mds.0:3899 currently failed to wrlock, waiting 2024-03-28T00:39:15.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:39:15 smithi132 ceph-mon[35941]: pgmap v422: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:39:15.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:39:15 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:39:15.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:39:15 smithi132 ceph-mon[35941]: 98 slow requests, 5 included below; oldest blocked for > 215.664524 secs 2024-03-28T00:39:15.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:39:15 smithi132 ceph-mon[35941]: slow request 211.566039 seconds old, received at 2024-03-28T00:35:43.457430+0000: internal op fragmentdir:mds.0:3803 currently failed to wrlock, waiting 2024-03-28T00:39:15.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:39:15 smithi132 ceph-mon[35941]: slow request 211.550453 seconds old, received at 2024-03-28T00:35:43.473016+0000: internal op fragmentdir:mds.0:3834 currently failed to wrlock, waiting 2024-03-28T00:39:15.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:39:15 smithi132 ceph-mon[35941]: slow request 211.565533 seconds old, received at 2024-03-28T00:35:43.457937+0000: internal op fragmentdir:mds.0:3804 currently failed to wrlock, waiting 2024-03-28T00:39:15.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:39:15 smithi132 ceph-mon[35941]: slow request 211.536084 seconds old, received at 2024-03-28T00:35:43.487385+0000: internal op fragmentdir:mds.0:3867 currently failed to wrlock, waiting 2024-03-28T00:39:15.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:39:15 smithi132 ceph-mon[35941]: slow request 211.521698 seconds old, received at 2024-03-28T00:35:43.501771+0000: internal op fragmentdir:mds.0:3899 currently failed to wrlock, waiting 2024-03-28T00:39:16.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:39:16 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:39:16.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:39:16 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:39:16.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:39:16 smithi165 ceph-mon[36302]: pgmap v423: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T00:39:16.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:39:16 smithi165 ceph-mon[36302]: pgmap v424: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T00:39:16.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:39:16 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:39:16.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:39:16 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:39:16.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:39:16 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:39:16.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:39:16 smithi119 ceph-mon[31518]: pgmap v423: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T00:39:16.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:39:16 smithi119 ceph-mon[31518]: pgmap v424: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T00:39:16.657 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:39:16 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:39:16.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:39:16 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:39:16.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:39:16 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:39:16.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:39:16 smithi132 ceph-mon[35941]: pgmap v423: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T00:39:16.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:39:16 smithi132 ceph-mon[35941]: pgmap v424: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T00:39:16.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:39:16 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:39:19.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:39:18 smithi165 ceph-mon[36302]: pgmap v425: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T00:39:19.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:39:18 smithi119 ceph-mon[31518]: pgmap v425: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T00:39:19.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:39:18 smithi132 ceph-mon[35941]: pgmap v425: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T00:39:21.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:39:20 smithi165 ceph-mon[36302]: pgmap v426: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T00:39:21.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:39:20 smithi165 ceph-mon[36302]: 98 slow requests, 5 included below; oldest blocked for > 220.664613 secs 2024-03-28T00:39:21.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:39:20 smithi165 ceph-mon[36302]: slow request 216.565132 seconds old, received at 2024-03-28T00:35:43.458426+0000: internal op fragmentdir:mds.0:3805 currently failed to wrlock, waiting 2024-03-28T00:39:21.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:39:20 smithi165 ceph-mon[36302]: slow request 216.549655 seconds old, received at 2024-03-28T00:35:43.473902+0000: internal op fragmentdir:mds.0:3836 currently failed to wrlock, waiting 2024-03-28T00:39:21.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:39:20 smithi165 ceph-mon[36302]: slow request 216.535776 seconds old, received at 2024-03-28T00:35:43.487781+0000: internal op fragmentdir:mds.0:3868 currently failed to wrlock, waiting 2024-03-28T00:39:21.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:39:20 smithi165 ceph-mon[36302]: slow request 216.564645 seconds old, received at 2024-03-28T00:35:43.458913+0000: internal op fragmentdir:mds.0:3806 currently failed to wrlock, waiting 2024-03-28T00:39:21.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:39:20 smithi165 ceph-mon[36302]: slow request 216.549220 seconds old, received at 2024-03-28T00:35:43.474338+0000: internal op fragmentdir:mds.0:3837 currently failed to wrlock, waiting 2024-03-28T00:39:21.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:39:20 smithi119 ceph-mon[31518]: pgmap v426: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T00:39:21.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:39:20 smithi119 ceph-mon[31518]: 98 slow requests, 5 included below; oldest blocked for > 220.664613 secs 2024-03-28T00:39:21.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:39:20 smithi119 ceph-mon[31518]: slow request 216.565132 seconds old, received at 2024-03-28T00:35:43.458426+0000: internal op fragmentdir:mds.0:3805 currently failed to wrlock, waiting 2024-03-28T00:39:21.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:39:20 smithi119 ceph-mon[31518]: slow request 216.549655 seconds old, received at 2024-03-28T00:35:43.473902+0000: internal op fragmentdir:mds.0:3836 currently failed to wrlock, waiting 2024-03-28T00:39:21.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:39:20 smithi119 ceph-mon[31518]: slow request 216.535776 seconds old, received at 2024-03-28T00:35:43.487781+0000: internal op fragmentdir:mds.0:3868 currently failed to wrlock, waiting 2024-03-28T00:39:21.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:39:20 smithi119 ceph-mon[31518]: slow request 216.564645 seconds old, received at 2024-03-28T00:35:43.458913+0000: internal op fragmentdir:mds.0:3806 currently failed to wrlock, waiting 2024-03-28T00:39:21.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:39:20 smithi119 ceph-mon[31518]: slow request 216.549220 seconds old, received at 2024-03-28T00:35:43.474338+0000: internal op fragmentdir:mds.0:3837 currently failed to wrlock, waiting 2024-03-28T00:39:21.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:39:20 smithi132 ceph-mon[35941]: pgmap v426: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T00:39:21.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:39:20 smithi132 ceph-mon[35941]: 98 slow requests, 5 included below; oldest blocked for > 220.664613 secs 2024-03-28T00:39:21.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:39:20 smithi132 ceph-mon[35941]: slow request 216.565132 seconds old, received at 2024-03-28T00:35:43.458426+0000: internal op fragmentdir:mds.0:3805 currently failed to wrlock, waiting 2024-03-28T00:39:21.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:39:20 smithi132 ceph-mon[35941]: slow request 216.549655 seconds old, received at 2024-03-28T00:35:43.473902+0000: internal op fragmentdir:mds.0:3836 currently failed to wrlock, waiting 2024-03-28T00:39:21.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:39:20 smithi132 ceph-mon[35941]: slow request 216.535776 seconds old, received at 2024-03-28T00:35:43.487781+0000: internal op fragmentdir:mds.0:3868 currently failed to wrlock, waiting 2024-03-28T00:39:21.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:39:20 smithi132 ceph-mon[35941]: slow request 216.564645 seconds old, received at 2024-03-28T00:35:43.458913+0000: internal op fragmentdir:mds.0:3806 currently failed to wrlock, waiting 2024-03-28T00:39:21.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:39:20 smithi132 ceph-mon[35941]: slow request 216.549220 seconds old, received at 2024-03-28T00:35:43.474338+0000: internal op fragmentdir:mds.0:3837 currently failed to wrlock, waiting 2024-03-28T00:39:23.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:39:22 smithi165 ceph-mon[36302]: pgmap v427: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T00:39:23.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:39:22 smithi119 ceph-mon[31518]: pgmap v427: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T00:39:23.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:39:22 smithi132 ceph-mon[35941]: pgmap v427: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T00:39:25.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:39:24 smithi165 ceph-mon[36302]: pgmap v428: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-28T00:39:25.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:39:24 smithi119 ceph-mon[31518]: pgmap v428: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-28T00:39:25.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:39:24 smithi132 ceph-mon[35941]: pgmap v428: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-28T00:39:26.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:39:25 smithi165 ceph-mon[36302]: 98 slow requests, 5 included below; oldest blocked for > 225.664702 secs 2024-03-28T00:39:26.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:39:25 smithi165 ceph-mon[36302]: slow request 221.520992 seconds old, received at 2024-03-28T00:35:43.502655+0000: internal op fragmentdir:mds.0:3901 currently failed to wrlock, waiting 2024-03-28T00:39:26.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:39:25 smithi165 ceph-mon[36302]: slow request 221.565380 seconds old, received at 2024-03-28T00:35:43.458267+0000: client_request(client.25264:34113 create owner_uid=1000, owner_gid=1301 #0x10000000272/file7594 2024-03-28T00:35:43.457305+0000 caller_uid=1000, caller_gid=1301{6,36,1000,1301,}) currently failed to xlock, waiting 2024-03-28T00:39:26.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:39:25 smithi165 ceph-mon[36302]: slow request 221.535342 seconds old, received at 2024-03-28T00:35:43.488305+0000: internal op fragmentdir:mds.0:3870 currently failed to wrlock, waiting 2024-03-28T00:39:26.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:39:25 smithi165 ceph-mon[36302]: slow request 221.561799 seconds old, received at 2024-03-28T00:35:43.461848+0000: internal op fragmentdir:mds.0:3807 currently failed to wrlock, waiting 2024-03-28T00:39:26.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:39:25 smithi165 ceph-mon[36302]: slow request 221.548452 seconds old, received at 2024-03-28T00:35:43.475195+0000: internal op fragmentdir:mds.0:3839 currently failed to wrlock, waiting 2024-03-28T00:39:26.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:39:25 smithi119 ceph-mon[31518]: 98 slow requests, 5 included below; oldest blocked for > 225.664702 secs 2024-03-28T00:39:26.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:39:25 smithi119 ceph-mon[31518]: slow request 221.520992 seconds old, received at 2024-03-28T00:35:43.502655+0000: internal op fragmentdir:mds.0:3901 currently failed to wrlock, waiting 2024-03-28T00:39:26.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:39:25 smithi119 ceph-mon[31518]: slow request 221.565380 seconds old, received at 2024-03-28T00:35:43.458267+0000: client_request(client.25264:34113 create owner_uid=1000, owner_gid=1301 #0x10000000272/file7594 2024-03-28T00:35:43.457305+0000 caller_uid=1000, caller_gid=1301{6,36,1000,1301,}) currently failed to xlock, waiting 2024-03-28T00:39:26.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:39:25 smithi119 ceph-mon[31518]: slow request 221.535342 seconds old, received at 2024-03-28T00:35:43.488305+0000: internal op fragmentdir:mds.0:3870 currently failed to wrlock, waiting 2024-03-28T00:39:26.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:39:25 smithi119 ceph-mon[31518]: slow request 221.561799 seconds old, received at 2024-03-28T00:35:43.461848+0000: internal op fragmentdir:mds.0:3807 currently failed to wrlock, waiting 2024-03-28T00:39:26.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:39:25 smithi119 ceph-mon[31518]: slow request 221.548452 seconds old, received at 2024-03-28T00:35:43.475195+0000: internal op fragmentdir:mds.0:3839 currently failed to wrlock, waiting 2024-03-28T00:39:26.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:39:25 smithi132 ceph-mon[35941]: 98 slow requests, 5 included below; oldest blocked for > 225.664702 secs 2024-03-28T00:39:26.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:39:25 smithi132 ceph-mon[35941]: slow request 221.520992 seconds old, received at 2024-03-28T00:35:43.502655+0000: internal op fragmentdir:mds.0:3901 currently failed to wrlock, waiting 2024-03-28T00:39:26.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:39:25 smithi132 ceph-mon[35941]: slow request 221.565380 seconds old, received at 2024-03-28T00:35:43.458267+0000: client_request(client.25264:34113 create owner_uid=1000, owner_gid=1301 #0x10000000272/file7594 2024-03-28T00:35:43.457305+0000 caller_uid=1000, caller_gid=1301{6,36,1000,1301,}) currently failed to xlock, waiting 2024-03-28T00:39:26.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:39:25 smithi132 ceph-mon[35941]: slow request 221.535342 seconds old, received at 2024-03-28T00:35:43.488305+0000: internal op fragmentdir:mds.0:3870 currently failed to wrlock, waiting 2024-03-28T00:39:26.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:39:25 smithi132 ceph-mon[35941]: slow request 221.561799 seconds old, received at 2024-03-28T00:35:43.461848+0000: internal op fragmentdir:mds.0:3807 currently failed to wrlock, waiting 2024-03-28T00:39:26.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:39:25 smithi132 ceph-mon[35941]: slow request 221.548452 seconds old, received at 2024-03-28T00:35:43.475195+0000: internal op fragmentdir:mds.0:3839 currently failed to wrlock, waiting 2024-03-28T00:39:27.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:39:26 smithi165 ceph-mon[36302]: pgmap v429: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T00:39:27.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:39:26 smithi119 ceph-mon[31518]: pgmap v429: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T00:39:27.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:39:26 smithi132 ceph-mon[35941]: pgmap v429: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T00:39:28.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:39:27 smithi165 ceph-mon[36302]: pgmap v430: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:39:28.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:39:27 smithi119 ceph-mon[31518]: pgmap v430: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:39:28.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:39:27 smithi132 ceph-mon[35941]: pgmap v430: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:39:31.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:39:30 smithi165 ceph-mon[36302]: pgmap v431: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:39:31.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:39:30 smithi165 ceph-mon[36302]: 98 slow requests, 5 included below; oldest blocked for > 230.664792 secs 2024-03-28T00:39:31.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:39:30 smithi165 ceph-mon[36302]: slow request 226.535012 seconds old, received at 2024-03-28T00:35:43.488725+0000: internal op fragmentdir:mds.0:3871 currently failed to wrlock, waiting 2024-03-28T00:39:31.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:39:30 smithi165 ceph-mon[36302]: slow request 226.520429 seconds old, received at 2024-03-28T00:35:43.503308+0000: internal op fragmentdir:mds.0:3903 currently failed to wrlock, waiting 2024-03-28T00:39:31.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:39:30 smithi165 ceph-mon[36302]: slow request 209.646905 seconds old, received at 2024-03-28T00:36:00.376832+0000: client_request(client.25355:96 mksnap #0x1//scheduled-2024-03-28-00_36_00_UTC 2024-03-28T00:36:00.376777+0000 caller_uid=0, caller_gid=0{}) currently failed to acquire_locks 2024-03-28T00:39:31.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:39:30 smithi165 ceph-mon[36302]: slow request 226.561435 seconds old, received at 2024-03-28T00:35:43.462302+0000: internal op fragmentdir:mds.0:3808 currently failed to wrlock, waiting 2024-03-28T00:39:31.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:39:30 smithi165 ceph-mon[36302]: slow request 226.625041 seconds old, received at 2024-03-28T00:35:43.398696+0000: internal op fragmentdir:mds.0:3785 currently failed to wrlock, waiting 2024-03-28T00:39:31.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:39:30 smithi119 ceph-mon[31518]: pgmap v431: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:39:31.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:39:30 smithi119 ceph-mon[31518]: 98 slow requests, 5 included below; oldest blocked for > 230.664792 secs 2024-03-28T00:39:31.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:39:30 smithi119 ceph-mon[31518]: slow request 226.535012 seconds old, received at 2024-03-28T00:35:43.488725+0000: internal op fragmentdir:mds.0:3871 currently failed to wrlock, waiting 2024-03-28T00:39:31.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:39:30 smithi119 ceph-mon[31518]: slow request 226.520429 seconds old, received at 2024-03-28T00:35:43.503308+0000: internal op fragmentdir:mds.0:3903 currently failed to wrlock, waiting 2024-03-28T00:39:31.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:39:30 smithi119 ceph-mon[31518]: slow request 209.646905 seconds old, received at 2024-03-28T00:36:00.376832+0000: client_request(client.25355:96 mksnap #0x1//scheduled-2024-03-28-00_36_00_UTC 2024-03-28T00:36:00.376777+0000 caller_uid=0, caller_gid=0{}) currently failed to acquire_locks 2024-03-28T00:39:31.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:39:30 smithi119 ceph-mon[31518]: slow request 226.561435 seconds old, received at 2024-03-28T00:35:43.462302+0000: internal op fragmentdir:mds.0:3808 currently failed to wrlock, waiting 2024-03-28T00:39:31.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:39:30 smithi119 ceph-mon[31518]: slow request 226.625041 seconds old, received at 2024-03-28T00:35:43.398696+0000: internal op fragmentdir:mds.0:3785 currently failed to wrlock, waiting 2024-03-28T00:39:31.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:39:30 smithi132 ceph-mon[35941]: pgmap v431: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:39:31.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:39:30 smithi132 ceph-mon[35941]: 98 slow requests, 5 included below; oldest blocked for > 230.664792 secs 2024-03-28T00:39:31.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:39:30 smithi132 ceph-mon[35941]: slow request 226.535012 seconds old, received at 2024-03-28T00:35:43.488725+0000: internal op fragmentdir:mds.0:3871 currently failed to wrlock, waiting 2024-03-28T00:39:31.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:39:30 smithi132 ceph-mon[35941]: slow request 226.520429 seconds old, received at 2024-03-28T00:35:43.503308+0000: internal op fragmentdir:mds.0:3903 currently failed to wrlock, waiting 2024-03-28T00:39:31.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:39:30 smithi132 ceph-mon[35941]: slow request 209.646905 seconds old, received at 2024-03-28T00:36:00.376832+0000: client_request(client.25355:96 mksnap #0x1//scheduled-2024-03-28-00_36_00_UTC 2024-03-28T00:36:00.376777+0000 caller_uid=0, caller_gid=0{}) currently failed to acquire_locks 2024-03-28T00:39:31.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:39:30 smithi132 ceph-mon[35941]: slow request 226.561435 seconds old, received at 2024-03-28T00:35:43.462302+0000: internal op fragmentdir:mds.0:3808 currently failed to wrlock, waiting 2024-03-28T00:39:31.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:39:30 smithi132 ceph-mon[35941]: slow request 226.625041 seconds old, received at 2024-03-28T00:35:43.398696+0000: internal op fragmentdir:mds.0:3785 currently failed to wrlock, waiting 2024-03-28T00:39:33.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:39:32 smithi165 ceph-mon[36302]: pgmap v432: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:39:33.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:39:32 smithi119 ceph-mon[31518]: pgmap v432: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:39:33.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:39:32 smithi132 ceph-mon[35941]: pgmap v432: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:39:35.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:39:34 smithi165 ceph-mon[36302]: pgmap v433: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:39:35.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:39:34 smithi119 ceph-mon[31518]: pgmap v433: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:39:35.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:39:34 smithi132 ceph-mon[35941]: pgmap v433: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:39:36.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:39:35 smithi165 ceph-mon[36302]: 98 slow requests, 5 included below; oldest blocked for > 235.664832 secs 2024-03-28T00:39:36.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:39:35 smithi165 ceph-mon[36302]: slow request 231.561013 seconds old, received at 2024-03-28T00:35:43.462764+0000: internal op fragmentdir:mds.0:3809 currently failed to wrlock, waiting 2024-03-28T00:39:36.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:39:35 smithi165 ceph-mon[36302]: slow request 231.547683 seconds old, received at 2024-03-28T00:35:43.476094+0000: internal op fragmentdir:mds.0:3841 currently failed to wrlock, waiting 2024-03-28T00:39:36.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:39:35 smithi165 ceph-mon[36302]: slow request 231.534126 seconds old, received at 2024-03-28T00:35:43.489651+0000: internal op fragmentdir:mds.0:3873 currently failed to wrlock, waiting 2024-03-28T00:39:36.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:39:35 smithi165 ceph-mon[36302]: slow request 230.696156 seconds old, received at 2024-03-28T00:35:44.327621+0000: internal op fragmentdir:mds.0:3937 currently failed to wrlock, waiting 2024-03-28T00:39:36.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:39:35 smithi165 ceph-mon[36302]: slow request 231.560577 seconds old, received at 2024-03-28T00:35:43.463200+0000: internal op fragmentdir:mds.0:3810 currently failed to wrlock, waiting 2024-03-28T00:39:36.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:39:35 smithi165 ceph-mon[36302]: pgmap v434: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:39:36.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:39:35 smithi119 ceph-mon[31518]: 98 slow requests, 5 included below; oldest blocked for > 235.664832 secs 2024-03-28T00:39:36.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:39:35 smithi119 ceph-mon[31518]: slow request 231.561013 seconds old, received at 2024-03-28T00:35:43.462764+0000: internal op fragmentdir:mds.0:3809 currently failed to wrlock, waiting 2024-03-28T00:39:36.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:39:35 smithi119 ceph-mon[31518]: slow request 231.547683 seconds old, received at 2024-03-28T00:35:43.476094+0000: internal op fragmentdir:mds.0:3841 currently failed to wrlock, waiting 2024-03-28T00:39:36.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:39:35 smithi119 ceph-mon[31518]: slow request 231.534126 seconds old, received at 2024-03-28T00:35:43.489651+0000: internal op fragmentdir:mds.0:3873 currently failed to wrlock, waiting 2024-03-28T00:39:36.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:39:35 smithi119 ceph-mon[31518]: slow request 230.696156 seconds old, received at 2024-03-28T00:35:44.327621+0000: internal op fragmentdir:mds.0:3937 currently failed to wrlock, waiting 2024-03-28T00:39:36.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:39:35 smithi119 ceph-mon[31518]: slow request 231.560577 seconds old, received at 2024-03-28T00:35:43.463200+0000: internal op fragmentdir:mds.0:3810 currently failed to wrlock, waiting 2024-03-28T00:39:36.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:39:35 smithi119 ceph-mon[31518]: pgmap v434: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:39:36.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:39:35 smithi132 ceph-mon[35941]: 98 slow requests, 5 included below; oldest blocked for > 235.664832 secs 2024-03-28T00:39:36.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:39:35 smithi132 ceph-mon[35941]: slow request 231.561013 seconds old, received at 2024-03-28T00:35:43.462764+0000: internal op fragmentdir:mds.0:3809 currently failed to wrlock, waiting 2024-03-28T00:39:36.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:39:35 smithi132 ceph-mon[35941]: slow request 231.547683 seconds old, received at 2024-03-28T00:35:43.476094+0000: internal op fragmentdir:mds.0:3841 currently failed to wrlock, waiting 2024-03-28T00:39:36.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:39:35 smithi132 ceph-mon[35941]: slow request 231.534126 seconds old, received at 2024-03-28T00:35:43.489651+0000: internal op fragmentdir:mds.0:3873 currently failed to wrlock, waiting 2024-03-28T00:39:36.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:39:35 smithi132 ceph-mon[35941]: slow request 230.696156 seconds old, received at 2024-03-28T00:35:44.327621+0000: internal op fragmentdir:mds.0:3937 currently failed to wrlock, waiting 2024-03-28T00:39:36.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:39:35 smithi132 ceph-mon[35941]: slow request 231.560577 seconds old, received at 2024-03-28T00:35:43.463200+0000: internal op fragmentdir:mds.0:3810 currently failed to wrlock, waiting 2024-03-28T00:39:36.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:39:35 smithi132 ceph-mon[35941]: pgmap v434: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:39:39.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:39:38 smithi165 ceph-mon[36302]: pgmap v435: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:39:39.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:39:38 smithi119 ceph-mon[31518]: pgmap v435: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:39:39.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:39:38 smithi132 ceph-mon[35941]: pgmap v435: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:39:41.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:39:40 smithi165 ceph-mon[36302]: pgmap v436: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:39:41.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:39:40 smithi165 ceph-mon[36302]: 98 slow requests, 5 included below; oldest blocked for > 240.664873 secs 2024-03-28T00:39:41.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:39:40 smithi165 ceph-mon[36302]: slow request 236.547287 seconds old, received at 2024-03-28T00:35:43.476531+0000: internal op fragmentdir:mds.0:3842 currently failed to wrlock, waiting 2024-03-28T00:39:41.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:39:40 smithi165 ceph-mon[36302]: slow request 236.560217 seconds old, received at 2024-03-28T00:35:43.463601+0000: internal op fragmentdir:mds.0:3811 currently failed to wrlock, waiting 2024-03-28T00:39:41.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:39:40 smithi165 ceph-mon[36302]: slow request 236.517780 seconds old, received at 2024-03-28T00:35:43.506038+0000: internal op fragmentdir:mds.0:3907 currently failed to wrlock, waiting 2024-03-28T00:39:41.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:39:40 smithi165 ceph-mon[36302]: slow request 236.559782 seconds old, received at 2024-03-28T00:35:43.464036+0000: internal op fragmentdir:mds.0:3812 currently failed to wrlock, waiting 2024-03-28T00:39:41.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:39:40 smithi165 ceph-mon[36302]: slow request 236.546404 seconds old, received at 2024-03-28T00:35:43.477414+0000: internal op fragmentdir:mds.0:3844 currently failed to wrlock, waiting 2024-03-28T00:39:41.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:39:40 smithi119 ceph-mon[31518]: pgmap v436: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:39:41.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:39:40 smithi119 ceph-mon[31518]: 98 slow requests, 5 included below; oldest blocked for > 240.664873 secs 2024-03-28T00:39:41.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:39:40 smithi119 ceph-mon[31518]: slow request 236.547287 seconds old, received at 2024-03-28T00:35:43.476531+0000: internal op fragmentdir:mds.0:3842 currently failed to wrlock, waiting 2024-03-28T00:39:41.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:39:40 smithi119 ceph-mon[31518]: slow request 236.560217 seconds old, received at 2024-03-28T00:35:43.463601+0000: internal op fragmentdir:mds.0:3811 currently failed to wrlock, waiting 2024-03-28T00:39:41.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:39:40 smithi119 ceph-mon[31518]: slow request 236.517780 seconds old, received at 2024-03-28T00:35:43.506038+0000: internal op fragmentdir:mds.0:3907 currently failed to wrlock, waiting 2024-03-28T00:39:41.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:39:40 smithi119 ceph-mon[31518]: slow request 236.559782 seconds old, received at 2024-03-28T00:35:43.464036+0000: internal op fragmentdir:mds.0:3812 currently failed to wrlock, waiting 2024-03-28T00:39:41.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:39:40 smithi119 ceph-mon[31518]: slow request 236.546404 seconds old, received at 2024-03-28T00:35:43.477414+0000: internal op fragmentdir:mds.0:3844 currently failed to wrlock, waiting 2024-03-28T00:39:41.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:39:40 smithi132 ceph-mon[35941]: pgmap v436: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:39:41.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:39:40 smithi132 ceph-mon[35941]: 98 slow requests, 5 included below; oldest blocked for > 240.664873 secs 2024-03-28T00:39:41.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:39:40 smithi132 ceph-mon[35941]: slow request 236.547287 seconds old, received at 2024-03-28T00:35:43.476531+0000: internal op fragmentdir:mds.0:3842 currently failed to wrlock, waiting 2024-03-28T00:39:41.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:39:40 smithi132 ceph-mon[35941]: slow request 236.560217 seconds old, received at 2024-03-28T00:35:43.463601+0000: internal op fragmentdir:mds.0:3811 currently failed to wrlock, waiting 2024-03-28T00:39:41.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:39:40 smithi132 ceph-mon[35941]: slow request 236.517780 seconds old, received at 2024-03-28T00:35:43.506038+0000: internal op fragmentdir:mds.0:3907 currently failed to wrlock, waiting 2024-03-28T00:39:41.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:39:40 smithi132 ceph-mon[35941]: slow request 236.559782 seconds old, received at 2024-03-28T00:35:43.464036+0000: internal op fragmentdir:mds.0:3812 currently failed to wrlock, waiting 2024-03-28T00:39:41.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:39:40 smithi132 ceph-mon[35941]: slow request 236.546404 seconds old, received at 2024-03-28T00:35:43.477414+0000: internal op fragmentdir:mds.0:3844 currently failed to wrlock, waiting 2024-03-28T00:39:43.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:39:42 smithi165 ceph-mon[36302]: pgmap v437: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:39:43.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:39:42 smithi119 ceph-mon[31518]: pgmap v437: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:39:43.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:39:42 smithi132 ceph-mon[35941]: pgmap v437: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:39:44.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:39:43 smithi119 ceph-mon[31518]: pgmap v438: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:39:44.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:39:43 smithi132 ceph-mon[35941]: pgmap v438: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:39:44.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:39:43 smithi165 ceph-mon[36302]: pgmap v438: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:39:45.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:39:45 smithi165 ceph-mon[36302]: 98 slow requests, 5 included below; oldest blocked for > 245.664936 secs 2024-03-28T00:39:45.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:39:45 smithi165 ceph-mon[36302]: slow request 241.516115 seconds old, received at 2024-03-28T00:35:43.507766+0000: internal op fragmentdir:mds.0:3908 currently failed to wrlock, waiting 2024-03-28T00:39:45.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:39:45 smithi165 ceph-mon[36302]: slow request 242.884516 seconds old, received at 2024-03-28T00:35:42.139365+0000: internal op fragmentdir:mds.0:3784 currently failed to wrlock, waiting 2024-03-28T00:39:45.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:39:45 smithi165 ceph-mon[36302]: slow request 241.559448 seconds old, received at 2024-03-28T00:35:43.464432+0000: internal op fragmentdir:mds.0:3813 currently failed to wrlock, waiting 2024-03-28T00:39:45.384 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:39:45 smithi165 ceph-mon[36302]: slow request 241.546023 seconds old, received at 2024-03-28T00:35:43.477858+0000: internal op fragmentdir:mds.0:3845 currently failed to wrlock, waiting 2024-03-28T00:39:45.384 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:39:45 smithi165 ceph-mon[36302]: slow request 241.532333 seconds old, received at 2024-03-28T00:35:43.491547+0000: internal op fragmentdir:mds.0:3877 currently failed to wrlock, waiting 2024-03-28T00:39:45.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:39:45 smithi119 ceph-mon[31518]: 98 slow requests, 5 included below; oldest blocked for > 245.664936 secs 2024-03-28T00:39:45.407 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:39:45 smithi119 ceph-mon[31518]: slow request 241.516115 seconds old, received at 2024-03-28T00:35:43.507766+0000: internal op fragmentdir:mds.0:3908 currently failed to wrlock, waiting 2024-03-28T00:39:45.407 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:39:45 smithi119 ceph-mon[31518]: slow request 242.884516 seconds old, received at 2024-03-28T00:35:42.139365+0000: internal op fragmentdir:mds.0:3784 currently failed to wrlock, waiting 2024-03-28T00:39:45.407 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:39:45 smithi119 ceph-mon[31518]: slow request 241.559448 seconds old, received at 2024-03-28T00:35:43.464432+0000: internal op fragmentdir:mds.0:3813 currently failed to wrlock, waiting 2024-03-28T00:39:45.407 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:39:45 smithi119 ceph-mon[31518]: slow request 241.546023 seconds old, received at 2024-03-28T00:35:43.477858+0000: internal op fragmentdir:mds.0:3845 currently failed to wrlock, waiting 2024-03-28T00:39:45.407 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:39:45 smithi119 ceph-mon[31518]: slow request 241.532333 seconds old, received at 2024-03-28T00:35:43.491547+0000: internal op fragmentdir:mds.0:3877 currently failed to wrlock, waiting 2024-03-28T00:39:45.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:39:45 smithi132 ceph-mon[35941]: 98 slow requests, 5 included below; oldest blocked for > 245.664936 secs 2024-03-28T00:39:45.454 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:39:45 smithi132 ceph-mon[35941]: slow request 241.516115 seconds old, received at 2024-03-28T00:35:43.507766+0000: internal op fragmentdir:mds.0:3908 currently failed to wrlock, waiting 2024-03-28T00:39:45.454 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:39:45 smithi132 ceph-mon[35941]: slow request 242.884516 seconds old, received at 2024-03-28T00:35:42.139365+0000: internal op fragmentdir:mds.0:3784 currently failed to wrlock, waiting 2024-03-28T00:39:45.454 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:39:45 smithi132 ceph-mon[35941]: slow request 241.559448 seconds old, received at 2024-03-28T00:35:43.464432+0000: internal op fragmentdir:mds.0:3813 currently failed to wrlock, waiting 2024-03-28T00:39:45.454 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:39:45 smithi132 ceph-mon[35941]: slow request 241.546023 seconds old, received at 2024-03-28T00:35:43.477858+0000: internal op fragmentdir:mds.0:3845 currently failed to wrlock, waiting 2024-03-28T00:39:45.454 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:39:45 smithi132 ceph-mon[35941]: slow request 241.532333 seconds old, received at 2024-03-28T00:35:43.491547+0000: internal op fragmentdir:mds.0:3877 currently failed to wrlock, waiting 2024-03-28T00:39:46.384 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:39:46 smithi165 ceph-mon[36302]: pgmap v439: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:39:46.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:39:46 smithi119 ceph-mon[31518]: pgmap v439: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:39:46.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:39:46 smithi132 ceph-mon[35941]: pgmap v439: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:39:49.110 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:39:48 smithi119 ceph-mon[31518]: pgmap v440: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:39:49.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:39:48 smithi165 ceph-mon[36302]: pgmap v440: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:39:49.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:39:48 smithi132 ceph-mon[35941]: pgmap v440: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:39:51.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:39:50 smithi165 ceph-mon[36302]: pgmap v441: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:39:51.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:39:50 smithi165 ceph-mon[36302]: 98 slow requests, 5 included below; oldest blocked for > 250.664977 secs 2024-03-28T00:39:51.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:39:50 smithi165 ceph-mon[36302]: slow request 246.559051 seconds old, received at 2024-03-28T00:35:43.464871+0000: internal op fragmentdir:mds.0:3814 currently failed to wrlock, waiting 2024-03-28T00:39:51.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:39:50 smithi165 ceph-mon[36302]: slow request 246.514098 seconds old, received at 2024-03-28T00:35:43.509824+0000: internal op fragmentdir:mds.0:3910 currently failed to wrlock, waiting 2024-03-28T00:39:51.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:39:50 smithi165 ceph-mon[36302]: slow request 246.598995 seconds old, received at 2024-03-28T00:35:43.424926+0000: internal op fragmentdir:mds.0:3787 currently failed to wrlock, waiting 2024-03-28T00:39:51.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:39:50 smithi165 ceph-mon[36302]: slow request 246.558611 seconds old, received at 2024-03-28T00:35:43.465310+0000: internal op fragmentdir:mds.0:3815 currently failed to wrlock, waiting 2024-03-28T00:39:51.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:39:50 smithi165 ceph-mon[36302]: slow request 246.531446 seconds old, received at 2024-03-28T00:35:43.492476+0000: internal op fragmentdir:mds.0:3879 currently failed to wrlock, waiting 2024-03-28T00:39:51.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:39:50 smithi119 ceph-mon[31518]: pgmap v441: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:39:51.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:39:50 smithi119 ceph-mon[31518]: 98 slow requests, 5 included below; oldest blocked for > 250.664977 secs 2024-03-28T00:39:51.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:39:50 smithi119 ceph-mon[31518]: slow request 246.559051 seconds old, received at 2024-03-28T00:35:43.464871+0000: internal op fragmentdir:mds.0:3814 currently failed to wrlock, waiting 2024-03-28T00:39:51.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:39:50 smithi119 ceph-mon[31518]: slow request 246.514098 seconds old, received at 2024-03-28T00:35:43.509824+0000: internal op fragmentdir:mds.0:3910 currently failed to wrlock, waiting 2024-03-28T00:39:51.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:39:50 smithi119 ceph-mon[31518]: slow request 246.598995 seconds old, received at 2024-03-28T00:35:43.424926+0000: internal op fragmentdir:mds.0:3787 currently failed to wrlock, waiting 2024-03-28T00:39:51.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:39:50 smithi119 ceph-mon[31518]: slow request 246.558611 seconds old, received at 2024-03-28T00:35:43.465310+0000: internal op fragmentdir:mds.0:3815 currently failed to wrlock, waiting 2024-03-28T00:39:51.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:39:50 smithi119 ceph-mon[31518]: slow request 246.531446 seconds old, received at 2024-03-28T00:35:43.492476+0000: internal op fragmentdir:mds.0:3879 currently failed to wrlock, waiting 2024-03-28T00:39:51.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:39:50 smithi132 ceph-mon[35941]: pgmap v441: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:39:51.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:39:50 smithi132 ceph-mon[35941]: 98 slow requests, 5 included below; oldest blocked for > 250.664977 secs 2024-03-28T00:39:51.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:39:50 smithi132 ceph-mon[35941]: slow request 246.559051 seconds old, received at 2024-03-28T00:35:43.464871+0000: internal op fragmentdir:mds.0:3814 currently failed to wrlock, waiting 2024-03-28T00:39:51.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:39:50 smithi132 ceph-mon[35941]: slow request 246.514098 seconds old, received at 2024-03-28T00:35:43.509824+0000: internal op fragmentdir:mds.0:3910 currently failed to wrlock, waiting 2024-03-28T00:39:51.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:39:50 smithi132 ceph-mon[35941]: slow request 246.598995 seconds old, received at 2024-03-28T00:35:43.424926+0000: internal op fragmentdir:mds.0:3787 currently failed to wrlock, waiting 2024-03-28T00:39:51.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:39:50 smithi132 ceph-mon[35941]: slow request 246.558611 seconds old, received at 2024-03-28T00:35:43.465310+0000: internal op fragmentdir:mds.0:3815 currently failed to wrlock, waiting 2024-03-28T00:39:51.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:39:50 smithi132 ceph-mon[35941]: slow request 246.531446 seconds old, received at 2024-03-28T00:35:43.492476+0000: internal op fragmentdir:mds.0:3879 currently failed to wrlock, waiting 2024-03-28T00:39:52.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:39:51 smithi119 ceph-mon[31518]: pgmap v442: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:39:52.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:39:51 smithi132 ceph-mon[35941]: pgmap v442: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:39:52.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:39:51 smithi165 ceph-mon[36302]: pgmap v442: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:39:55.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:39:54 smithi165 ceph-mon[36302]: pgmap v443: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:39:55.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:39:54 smithi119 ceph-mon[31518]: pgmap v443: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:39:55.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:39:54 smithi132 ceph-mon[35941]: pgmap v443: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:39:56.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:39:55 smithi165 ceph-mon[36302]: 98 slow requests, 5 included below; oldest blocked for > 255.665066 secs 2024-03-28T00:39:56.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:39:55 smithi165 ceph-mon[36302]: slow request 251.558307 seconds old, received at 2024-03-28T00:35:43.465705+0000: internal op fragmentdir:mds.0:3816 currently failed to wrlock, waiting 2024-03-28T00:39:56.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:39:55 smithi165 ceph-mon[36302]: slow request 251.544745 seconds old, received at 2024-03-28T00:35:43.479266+0000: internal op fragmentdir:mds.0:3848 currently failed to wrlock, waiting 2024-03-28T00:39:56.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:39:55 smithi165 ceph-mon[36302]: slow request 251.531049 seconds old, received at 2024-03-28T00:35:43.492962+0000: internal op fragmentdir:mds.0:3880 currently failed to wrlock, waiting 2024-03-28T00:39:56.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:39:55 smithi165 ceph-mon[36302]: slow request 252.875940 seconds old, received at 2024-03-28T00:35:42.148071+0000: client_request(client.25264:33954 create owner_uid=1000, owner_gid=1301 #0x10000000271/file8309 2024-03-28T00:35:42.146337+0000 caller_uid=1000, caller_gid=1301{6,36,1000,1301,}) currently failed to xlock, waiting 2024-03-28T00:39:56.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:39:55 smithi165 ceph-mon[36302]: slow request 251.557911 seconds old, received at 2024-03-28T00:35:43.466100+0000: internal op fragmentdir:mds.0:3817 currently failed to wrlock, waiting 2024-03-28T00:39:56.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:39:55 smithi119 ceph-mon[31518]: 98 slow requests, 5 included below; oldest blocked for > 255.665066 secs 2024-03-28T00:39:56.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:39:55 smithi119 ceph-mon[31518]: slow request 251.558307 seconds old, received at 2024-03-28T00:35:43.465705+0000: internal op fragmentdir:mds.0:3816 currently failed to wrlock, waiting 2024-03-28T00:39:56.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:39:55 smithi119 ceph-mon[31518]: slow request 251.544745 seconds old, received at 2024-03-28T00:35:43.479266+0000: internal op fragmentdir:mds.0:3848 currently failed to wrlock, waiting 2024-03-28T00:39:56.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:39:55 smithi119 ceph-mon[31518]: slow request 251.531049 seconds old, received at 2024-03-28T00:35:43.492962+0000: internal op fragmentdir:mds.0:3880 currently failed to wrlock, waiting 2024-03-28T00:39:56.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:39:55 smithi119 ceph-mon[31518]: slow request 252.875940 seconds old, received at 2024-03-28T00:35:42.148071+0000: client_request(client.25264:33954 create owner_uid=1000, owner_gid=1301 #0x10000000271/file8309 2024-03-28T00:35:42.146337+0000 caller_uid=1000, caller_gid=1301{6,36,1000,1301,}) currently failed to xlock, waiting 2024-03-28T00:39:56.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:39:55 smithi119 ceph-mon[31518]: slow request 251.557911 seconds old, received at 2024-03-28T00:35:43.466100+0000: internal op fragmentdir:mds.0:3817 currently failed to wrlock, waiting 2024-03-28T00:39:56.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:39:55 smithi132 ceph-mon[35941]: 98 slow requests, 5 included below; oldest blocked for > 255.665066 secs 2024-03-28T00:39:56.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:39:55 smithi132 ceph-mon[35941]: slow request 251.558307 seconds old, received at 2024-03-28T00:35:43.465705+0000: internal op fragmentdir:mds.0:3816 currently failed to wrlock, waiting 2024-03-28T00:39:56.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:39:55 smithi132 ceph-mon[35941]: slow request 251.544745 seconds old, received at 2024-03-28T00:35:43.479266+0000: internal op fragmentdir:mds.0:3848 currently failed to wrlock, waiting 2024-03-28T00:39:56.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:39:55 smithi132 ceph-mon[35941]: slow request 251.531049 seconds old, received at 2024-03-28T00:35:43.492962+0000: internal op fragmentdir:mds.0:3880 currently failed to wrlock, waiting 2024-03-28T00:39:56.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:39:55 smithi132 ceph-mon[35941]: slow request 252.875940 seconds old, received at 2024-03-28T00:35:42.148071+0000: client_request(client.25264:33954 create owner_uid=1000, owner_gid=1301 #0x10000000271/file8309 2024-03-28T00:35:42.146337+0000 caller_uid=1000, caller_gid=1301{6,36,1000,1301,}) currently failed to xlock, waiting 2024-03-28T00:39:56.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:39:55 smithi132 ceph-mon[35941]: slow request 251.557911 seconds old, received at 2024-03-28T00:35:43.466100+0000: internal op fragmentdir:mds.0:3817 currently failed to wrlock, waiting 2024-03-28T00:39:57.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:39:56 smithi165 ceph-mon[36302]: pgmap v444: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:39:57.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:39:56 smithi119 ceph-mon[31518]: pgmap v444: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:39:57.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:39:56 smithi132 ceph-mon[35941]: pgmap v444: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:39:58.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:39:57 smithi165 ceph-mon[36302]: pgmap v445: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:39:58.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:39:57 smithi119 ceph-mon[31518]: pgmap v445: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:39:58.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:39:57 smithi132 ceph-mon[35941]: pgmap v445: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:40:01.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:40:00 smithi165 ceph-mon[36302]: pgmap v446: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:40:01.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:40:00 smithi165 ceph-mon[36302]: Health detail: HEALTH_WARN 5 failed cephadm daemon(s); 1 filesystem is degraded; 1 MDSs report slow requests 2024-03-28T00:40:01.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:40:00 smithi165 ceph-mon[36302]: [WRN] CEPHADM_FAILED_DAEMON: 5 failed cephadm daemon(s) 2024-03-28T00:40:01.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:40:00 smithi165 ceph-mon[36302]: daemon mds.j on smithi119 is in error state 2024-03-28T00:40:01.135 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:40:00 smithi165 ceph-mon[36302]: daemon mds.g on smithi119 is in error state 2024-03-28T00:40:01.135 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:40:00 smithi165 ceph-mon[36302]: daemon mds.l on smithi165 is in error state 2024-03-28T00:40:01.135 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:40:00 smithi165 ceph-mon[36302]: daemon mds.c on smithi165 is in error state 2024-03-28T00:40:01.135 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:40:00 smithi165 ceph-mon[36302]: daemon mds.i on smithi165 is in error state 2024-03-28T00:40:01.135 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:40:00 smithi165 ceph-mon[36302]: [WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-28T00:40:01.135 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:40:00 smithi165 ceph-mon[36302]: fs cephfs is degraded 2024-03-28T00:40:01.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:40:00 smithi165 ceph-mon[36302]: [WRN] MDS_SLOW_REQUEST: 1 MDSs report slow requests 2024-03-28T00:40:01.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:40:00 smithi165 ceph-mon[36302]: mds.b(mds.0): 98 slow requests are blocked > 180 secs 2024-03-28T00:40:01.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:40:00 smithi165 ceph-mon[36302]: 98 slow requests, 5 included below; oldest blocked for > 260.665151 secs 2024-03-28T00:40:01.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:40:00 smithi165 ceph-mon[36302]: slow request 256.530806 seconds old, received at 2024-03-28T00:35:43.493290+0000: internal op fragmentdir:mds.0:3881 currently failed to wrlock, waiting 2024-03-28T00:40:01.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:40:00 smithi165 ceph-mon[36302]: slow request 256.557549 seconds old, received at 2024-03-28T00:35:43.466548+0000: internal op fragmentdir:mds.0:3818 currently failed to wrlock, waiting 2024-03-28T00:40:01.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:40:00 smithi165 ceph-mon[36302]: slow request 256.543904 seconds old, received at 2024-03-28T00:35:43.480192+0000: internal op fragmentdir:mds.0:3850 currently failed to wrlock, waiting 2024-03-28T00:40:01.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:40:00 smithi165 ceph-mon[36302]: slow request 256.575352 seconds old, received at 2024-03-28T00:35:43.448744+0000: internal op fragmentdir:mds.0:3788 currently failed to wrlock, waiting 2024-03-28T00:40:01.137 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:40:00 smithi165 ceph-mon[36302]: slow request 256.557141 seconds old, received at 2024-03-28T00:35:43.466955+0000: internal op fragmentdir:mds.0:3819 currently failed to wrlock, waiting 2024-03-28T00:40:01.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:40:00 smithi119 ceph-mon[31518]: pgmap v446: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:40:01.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:40:00 smithi119 ceph-mon[31518]: Health detail: HEALTH_WARN 5 failed cephadm daemon(s); 1 filesystem is degraded; 1 MDSs report slow requests 2024-03-28T00:40:01.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:40:00 smithi119 ceph-mon[31518]: [WRN] CEPHADM_FAILED_DAEMON: 5 failed cephadm daemon(s) 2024-03-28T00:40:01.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:40:00 smithi119 ceph-mon[31518]: daemon mds.j on smithi119 is in error state 2024-03-28T00:40:01.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:40:00 smithi119 ceph-mon[31518]: daemon mds.g on smithi119 is in error state 2024-03-28T00:40:01.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:40:00 smithi119 ceph-mon[31518]: daemon mds.l on smithi165 is in error state 2024-03-28T00:40:01.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:40:00 smithi119 ceph-mon[31518]: daemon mds.c on smithi165 is in error state 2024-03-28T00:40:01.158 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:40:00 smithi119 ceph-mon[31518]: daemon mds.i on smithi165 is in error state 2024-03-28T00:40:01.158 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:40:00 smithi119 ceph-mon[31518]: [WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-28T00:40:01.158 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:40:00 smithi119 ceph-mon[31518]: fs cephfs is degraded 2024-03-28T00:40:01.158 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:40:00 smithi119 ceph-mon[31518]: [WRN] MDS_SLOW_REQUEST: 1 MDSs report slow requests 2024-03-28T00:40:01.158 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:40:00 smithi119 ceph-mon[31518]: mds.b(mds.0): 98 slow requests are blocked > 180 secs 2024-03-28T00:40:01.158 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:40:00 smithi119 ceph-mon[31518]: 98 slow requests, 5 included below; oldest blocked for > 260.665151 secs 2024-03-28T00:40:01.158 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:40:00 smithi119 ceph-mon[31518]: slow request 256.530806 seconds old, received at 2024-03-28T00:35:43.493290+0000: internal op fragmentdir:mds.0:3881 currently failed to wrlock, waiting 2024-03-28T00:40:01.158 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:40:00 smithi119 ceph-mon[31518]: slow request 256.557549 seconds old, received at 2024-03-28T00:35:43.466548+0000: internal op fragmentdir:mds.0:3818 currently failed to wrlock, waiting 2024-03-28T00:40:01.158 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:40:00 smithi119 ceph-mon[31518]: slow request 256.543904 seconds old, received at 2024-03-28T00:35:43.480192+0000: internal op fragmentdir:mds.0:3850 currently failed to wrlock, waiting 2024-03-28T00:40:01.158 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:40:00 smithi119 ceph-mon[31518]: slow request 256.575352 seconds old, received at 2024-03-28T00:35:43.448744+0000: internal op fragmentdir:mds.0:3788 currently failed to wrlock, waiting 2024-03-28T00:40:01.158 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:40:00 smithi119 ceph-mon[31518]: slow request 256.557141 seconds old, received at 2024-03-28T00:35:43.466955+0000: internal op fragmentdir:mds.0:3819 currently failed to wrlock, waiting 2024-03-28T00:40:01.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:40:00 smithi132 ceph-mon[35941]: pgmap v446: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:40:01.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:40:00 smithi132 ceph-mon[35941]: Health detail: HEALTH_WARN 5 failed cephadm daemon(s); 1 filesystem is degraded; 1 MDSs report slow requests 2024-03-28T00:40:01.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:40:00 smithi132 ceph-mon[35941]: [WRN] CEPHADM_FAILED_DAEMON: 5 failed cephadm daemon(s) 2024-03-28T00:40:01.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:40:00 smithi132 ceph-mon[35941]: daemon mds.j on smithi119 is in error state 2024-03-28T00:40:01.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:40:00 smithi132 ceph-mon[35941]: daemon mds.g on smithi119 is in error state 2024-03-28T00:40:01.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:40:00 smithi132 ceph-mon[35941]: daemon mds.l on smithi165 is in error state 2024-03-28T00:40:01.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:40:00 smithi132 ceph-mon[35941]: daemon mds.c on smithi165 is in error state 2024-03-28T00:40:01.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:40:00 smithi132 ceph-mon[35941]: daemon mds.i on smithi165 is in error state 2024-03-28T00:40:01.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:40:00 smithi132 ceph-mon[35941]: [WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-28T00:40:01.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:40:00 smithi132 ceph-mon[35941]: fs cephfs is degraded 2024-03-28T00:40:01.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:40:00 smithi132 ceph-mon[35941]: [WRN] MDS_SLOW_REQUEST: 1 MDSs report slow requests 2024-03-28T00:40:01.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:40:00 smithi132 ceph-mon[35941]: mds.b(mds.0): 98 slow requests are blocked > 180 secs 2024-03-28T00:40:01.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:40:00 smithi132 ceph-mon[35941]: 98 slow requests, 5 included below; oldest blocked for > 260.665151 secs 2024-03-28T00:40:01.205 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:40:00 smithi132 ceph-mon[35941]: slow request 256.530806 seconds old, received at 2024-03-28T00:35:43.493290+0000: internal op fragmentdir:mds.0:3881 currently failed to wrlock, waiting 2024-03-28T00:40:01.205 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:40:00 smithi132 ceph-mon[35941]: slow request 256.557549 seconds old, received at 2024-03-28T00:35:43.466548+0000: internal op fragmentdir:mds.0:3818 currently failed to wrlock, waiting 2024-03-28T00:40:01.205 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:40:00 smithi132 ceph-mon[35941]: slow request 256.543904 seconds old, received at 2024-03-28T00:35:43.480192+0000: internal op fragmentdir:mds.0:3850 currently failed to wrlock, waiting 2024-03-28T00:40:01.205 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:40:00 smithi132 ceph-mon[35941]: slow request 256.575352 seconds old, received at 2024-03-28T00:35:43.448744+0000: internal op fragmentdir:mds.0:3788 currently failed to wrlock, waiting 2024-03-28T00:40:01.205 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:40:00 smithi132 ceph-mon[35941]: slow request 256.557141 seconds old, received at 2024-03-28T00:35:43.466955+0000: internal op fragmentdir:mds.0:3819 currently failed to wrlock, waiting 2024-03-28T00:40:02.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:40:01 smithi119 ceph-mon[31518]: pgmap v447: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:40:02.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:40:01 smithi132 ceph-mon[35941]: pgmap v447: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:40:02.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:40:01 smithi165 ceph-mon[36302]: pgmap v447: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:40:05.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:40:04 smithi165 ceph-mon[36302]: pgmap v448: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:40:05.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:40:04 smithi119 ceph-mon[31518]: pgmap v448: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:40:05.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:40:04 smithi132 ceph-mon[35941]: pgmap v448: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:40:06.104 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:40:05 smithi132 ceph-mon[35941]: 98 slow requests, 5 included below; oldest blocked for > 265.665212 secs 2024-03-28T00:40:06.105 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:40:05 smithi132 ceph-mon[35941]: slow request 261.543490 seconds old, received at 2024-03-28T00:35:43.480667+0000: internal op fragmentdir:mds.0:3851 currently failed to wrlock, waiting 2024-03-28T00:40:06.105 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:40:05 smithi132 ceph-mon[35941]: slow request 261.530318 seconds old, received at 2024-03-28T00:35:43.493839+0000: internal op fragmentdir:mds.0:3883 currently failed to wrlock, waiting 2024-03-28T00:40:06.105 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:40:05 smithi132 ceph-mon[35941]: slow request 261.574996 seconds old, received at 2024-03-28T00:35:43.449161+0000: internal op fragmentdir:mds.0:3789 currently failed to wrlock, waiting 2024-03-28T00:40:06.105 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:40:05 smithi132 ceph-mon[35941]: slow request 261.556766 seconds old, received at 2024-03-28T00:35:43.467391+0000: internal op fragmentdir:mds.0:3820 currently failed to wrlock, waiting 2024-03-28T00:40:06.105 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:40:05 smithi132 ceph-mon[35941]: slow request 261.543016 seconds old, received at 2024-03-28T00:35:43.481141+0000: internal op fragmentdir:mds.0:3852 currently failed to wrlock, waiting 2024-03-28T00:40:06.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:40:05 smithi165 ceph-mon[36302]: 98 slow requests, 5 included below; oldest blocked for > 265.665212 secs 2024-03-28T00:40:06.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:40:05 smithi165 ceph-mon[36302]: slow request 261.543490 seconds old, received at 2024-03-28T00:35:43.480667+0000: internal op fragmentdir:mds.0:3851 currently failed to wrlock, waiting 2024-03-28T00:40:06.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:40:05 smithi165 ceph-mon[36302]: slow request 261.530318 seconds old, received at 2024-03-28T00:35:43.493839+0000: internal op fragmentdir:mds.0:3883 currently failed to wrlock, waiting 2024-03-28T00:40:06.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:40:05 smithi165 ceph-mon[36302]: slow request 261.574996 seconds old, received at 2024-03-28T00:35:43.449161+0000: internal op fragmentdir:mds.0:3789 currently failed to wrlock, waiting 2024-03-28T00:40:06.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:40:05 smithi165 ceph-mon[36302]: slow request 261.556766 seconds old, received at 2024-03-28T00:35:43.467391+0000: internal op fragmentdir:mds.0:3820 currently failed to wrlock, waiting 2024-03-28T00:40:06.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:40:05 smithi165 ceph-mon[36302]: slow request 261.543016 seconds old, received at 2024-03-28T00:35:43.481141+0000: internal op fragmentdir:mds.0:3852 currently failed to wrlock, waiting 2024-03-28T00:40:06.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:40:05 smithi119 ceph-mon[31518]: 98 slow requests, 5 included below; oldest blocked for > 265.665212 secs 2024-03-28T00:40:06.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:40:05 smithi119 ceph-mon[31518]: slow request 261.543490 seconds old, received at 2024-03-28T00:35:43.480667+0000: internal op fragmentdir:mds.0:3851 currently failed to wrlock, waiting 2024-03-28T00:40:06.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:40:05 smithi119 ceph-mon[31518]: slow request 261.530318 seconds old, received at 2024-03-28T00:35:43.493839+0000: internal op fragmentdir:mds.0:3883 currently failed to wrlock, waiting 2024-03-28T00:40:06.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:40:05 smithi119 ceph-mon[31518]: slow request 261.574996 seconds old, received at 2024-03-28T00:35:43.449161+0000: internal op fragmentdir:mds.0:3789 currently failed to wrlock, waiting 2024-03-28T00:40:06.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:40:05 smithi119 ceph-mon[31518]: slow request 261.556766 seconds old, received at 2024-03-28T00:35:43.467391+0000: internal op fragmentdir:mds.0:3820 currently failed to wrlock, waiting 2024-03-28T00:40:06.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:40:05 smithi119 ceph-mon[31518]: slow request 261.543016 seconds old, received at 2024-03-28T00:35:43.481141+0000: internal op fragmentdir:mds.0:3852 currently failed to wrlock, waiting 2024-03-28T00:40:07.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:40:06 smithi165 ceph-mon[36302]: pgmap v449: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:40:07.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:40:06 smithi119 ceph-mon[31518]: pgmap v449: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:40:07.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:40:06 smithi132 ceph-mon[35941]: pgmap v449: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:40:08.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:40:07 smithi165 ceph-mon[36302]: pgmap v450: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:40:08.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:40:07 smithi119 ceph-mon[31518]: pgmap v450: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:40:08.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:40:07 smithi132 ceph-mon[35941]: pgmap v450: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:40:11.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:40:10 smithi165 ceph-mon[36302]: pgmap v451: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:40:11.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:40:10 smithi165 ceph-mon[36302]: 98 slow requests, 5 included below; oldest blocked for > 270.665299 secs 2024-03-28T00:40:11.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:40:10 smithi165 ceph-mon[36302]: slow request 266.501311 seconds old, received at 2024-03-28T00:35:43.522934+0000: internal op fragmentdir:mds.0:3916 currently failed to wrlock, waiting 2024-03-28T00:40:11.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:40:10 smithi165 ceph-mon[36302]: slow request 270.665299 seconds old, received at 2024-03-28T00:35:39.358945+0000: peer_request:client.25264:11489 currently acquired locks 2024-03-28T00:40:11.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:40:10 smithi165 ceph-mon[36302]: slow request 266.574624 seconds old, received at 2024-03-28T00:35:43.449621+0000: internal op fragmentdir:mds.0:3790 currently failed to wrlock, waiting 2024-03-28T00:40:11.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:40:10 smithi165 ceph-mon[36302]: slow request 266.556450 seconds old, received at 2024-03-28T00:35:43.467794+0000: internal op fragmentdir:mds.0:3821 currently failed to wrlock, waiting 2024-03-28T00:40:11.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:40:10 smithi165 ceph-mon[36302]: slow request 266.529622 seconds old, received at 2024-03-28T00:35:43.494622+0000: internal op fragmentdir:mds.0:3885 currently failed to wrlock, waiting 2024-03-28T00:40:11.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:40:10 smithi119 ceph-mon[31518]: pgmap v451: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:40:11.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:40:10 smithi119 ceph-mon[31518]: 98 slow requests, 5 included below; oldest blocked for > 270.665299 secs 2024-03-28T00:40:11.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:40:10 smithi119 ceph-mon[31518]: slow request 266.501311 seconds old, received at 2024-03-28T00:35:43.522934+0000: internal op fragmentdir:mds.0:3916 currently failed to wrlock, waiting 2024-03-28T00:40:11.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:40:10 smithi119 ceph-mon[31518]: slow request 270.665299 seconds old, received at 2024-03-28T00:35:39.358945+0000: peer_request:client.25264:11489 currently acquired locks 2024-03-28T00:40:11.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:40:10 smithi119 ceph-mon[31518]: slow request 266.574624 seconds old, received at 2024-03-28T00:35:43.449621+0000: internal op fragmentdir:mds.0:3790 currently failed to wrlock, waiting 2024-03-28T00:40:11.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:40:10 smithi119 ceph-mon[31518]: slow request 266.556450 seconds old, received at 2024-03-28T00:35:43.467794+0000: internal op fragmentdir:mds.0:3821 currently failed to wrlock, waiting 2024-03-28T00:40:11.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:40:10 smithi119 ceph-mon[31518]: slow request 266.529622 seconds old, received at 2024-03-28T00:35:43.494622+0000: internal op fragmentdir:mds.0:3885 currently failed to wrlock, waiting 2024-03-28T00:40:11.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:40:10 smithi132 ceph-mon[35941]: pgmap v451: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:40:11.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:40:10 smithi132 ceph-mon[35941]: 98 slow requests, 5 included below; oldest blocked for > 270.665299 secs 2024-03-28T00:40:11.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:40:10 smithi132 ceph-mon[35941]: slow request 266.501311 seconds old, received at 2024-03-28T00:35:43.522934+0000: internal op fragmentdir:mds.0:3916 currently failed to wrlock, waiting 2024-03-28T00:40:11.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:40:10 smithi132 ceph-mon[35941]: slow request 270.665299 seconds old, received at 2024-03-28T00:35:39.358945+0000: peer_request:client.25264:11489 currently acquired locks 2024-03-28T00:40:11.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:40:10 smithi132 ceph-mon[35941]: slow request 266.574624 seconds old, received at 2024-03-28T00:35:43.449621+0000: internal op fragmentdir:mds.0:3790 currently failed to wrlock, waiting 2024-03-28T00:40:11.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:40:10 smithi132 ceph-mon[35941]: slow request 266.556450 seconds old, received at 2024-03-28T00:35:43.467794+0000: internal op fragmentdir:mds.0:3821 currently failed to wrlock, waiting 2024-03-28T00:40:11.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:40:10 smithi132 ceph-mon[35941]: slow request 266.529622 seconds old, received at 2024-03-28T00:35:43.494622+0000: internal op fragmentdir:mds.0:3885 currently failed to wrlock, waiting 2024-03-28T00:40:12.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:40:11 smithi165 ceph-mon[36302]: pgmap v452: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:40:12.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:40:11 smithi119 ceph-mon[31518]: pgmap v452: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:40:12.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:40:11 smithi132 ceph-mon[35941]: pgmap v452: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:40:15.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:40:14 smithi165 ceph-mon[36302]: pgmap v453: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:40:15.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:40:14 smithi119 ceph-mon[31518]: pgmap v453: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:40:15.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:40:14 smithi132 ceph-mon[35941]: pgmap v453: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:40:16.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:40:15 smithi165 ceph-mon[36302]: 98 slow requests, 5 included below; oldest blocked for > 275.665361 secs 2024-03-28T00:40:16.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:40:15 smithi165 ceph-mon[36302]: slow request 271.574143 seconds old, received at 2024-03-28T00:35:43.450163+0000: internal op fragmentdir:mds.0:3791 currently failed to wrlock, waiting 2024-03-28T00:40:16.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:40:15 smithi165 ceph-mon[36302]: slow request 271.542247 seconds old, received at 2024-03-28T00:35:43.482059+0000: internal op fragmentdir:mds.0:3854 currently failed to wrlock, waiting 2024-03-28T00:40:16.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:40:15 smithi165 ceph-mon[36302]: slow request 271.529232 seconds old, received at 2024-03-28T00:35:43.495074+0000: internal op fragmentdir:mds.0:3886 currently failed to wrlock, waiting 2024-03-28T00:40:16.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:40:15 smithi165 ceph-mon[36302]: slow request 271.500252 seconds old, received at 2024-03-28T00:35:43.524054+0000: internal op fragmentdir:mds.0:3918 currently failed to wrlock, waiting 2024-03-28T00:40:16.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:40:15 smithi165 ceph-mon[36302]: slow request 271.573693 seconds old, received at 2024-03-28T00:35:43.450613+0000: internal op fragmentdir:mds.0:3792 currently failed to wrlock, waiting 2024-03-28T00:40:16.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:40:15 smithi165 ceph-mon[36302]: pgmap v454: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:40:16.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:40:15 smithi119 ceph-mon[31518]: 98 slow requests, 5 included below; oldest blocked for > 275.665361 secs 2024-03-28T00:40:16.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:40:15 smithi119 ceph-mon[31518]: slow request 271.574143 seconds old, received at 2024-03-28T00:35:43.450163+0000: internal op fragmentdir:mds.0:3791 currently failed to wrlock, waiting 2024-03-28T00:40:16.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:40:15 smithi119 ceph-mon[31518]: slow request 271.542247 seconds old, received at 2024-03-28T00:35:43.482059+0000: internal op fragmentdir:mds.0:3854 currently failed to wrlock, waiting 2024-03-28T00:40:16.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:40:15 smithi119 ceph-mon[31518]: slow request 271.529232 seconds old, received at 2024-03-28T00:35:43.495074+0000: internal op fragmentdir:mds.0:3886 currently failed to wrlock, waiting 2024-03-28T00:40:16.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:40:15 smithi119 ceph-mon[31518]: slow request 271.500252 seconds old, received at 2024-03-28T00:35:43.524054+0000: internal op fragmentdir:mds.0:3918 currently failed to wrlock, waiting 2024-03-28T00:40:16.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:40:15 smithi119 ceph-mon[31518]: slow request 271.573693 seconds old, received at 2024-03-28T00:35:43.450613+0000: internal op fragmentdir:mds.0:3792 currently failed to wrlock, waiting 2024-03-28T00:40:16.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:40:15 smithi119 ceph-mon[31518]: pgmap v454: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:40:16.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:40:15 smithi132 ceph-mon[35941]: 98 slow requests, 5 included below; oldest blocked for > 275.665361 secs 2024-03-28T00:40:16.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:40:15 smithi132 ceph-mon[35941]: slow request 271.574143 seconds old, received at 2024-03-28T00:35:43.450163+0000: internal op fragmentdir:mds.0:3791 currently failed to wrlock, waiting 2024-03-28T00:40:16.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:40:15 smithi132 ceph-mon[35941]: slow request 271.542247 seconds old, received at 2024-03-28T00:35:43.482059+0000: internal op fragmentdir:mds.0:3854 currently failed to wrlock, waiting 2024-03-28T00:40:16.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:40:15 smithi132 ceph-mon[35941]: slow request 271.529232 seconds old, received at 2024-03-28T00:35:43.495074+0000: internal op fragmentdir:mds.0:3886 currently failed to wrlock, waiting 2024-03-28T00:40:16.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:40:15 smithi132 ceph-mon[35941]: slow request 271.500252 seconds old, received at 2024-03-28T00:35:43.524054+0000: internal op fragmentdir:mds.0:3918 currently failed to wrlock, waiting 2024-03-28T00:40:16.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:40:15 smithi132 ceph-mon[35941]: slow request 271.573693 seconds old, received at 2024-03-28T00:35:43.450613+0000: internal op fragmentdir:mds.0:3792 currently failed to wrlock, waiting 2024-03-28T00:40:16.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:40:15 smithi132 ceph-mon[35941]: pgmap v454: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:40:17.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:40:16 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:40:17.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:40:16 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:40:17.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:40:16 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:40:18.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:40:17 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:40:18.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:40:17 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:40:18.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:40:17 smithi165 ceph-mon[36302]: pgmap v455: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T00:40:18.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:40:17 smithi165 ceph-mon[36302]: pgmap v456: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T00:40:18.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:40:17 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:40:18.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:40:17 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:40:18.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:40:17 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:40:18.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:40:17 smithi119 ceph-mon[31518]: pgmap v455: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T00:40:18.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:40:17 smithi119 ceph-mon[31518]: pgmap v456: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T00:40:18.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:40:17 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:40:18.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:40:17 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:40:18.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:40:17 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:40:18.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:40:17 smithi132 ceph-mon[35941]: pgmap v455: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T00:40:18.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:40:17 smithi132 ceph-mon[35941]: pgmap v456: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T00:40:18.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:40:17 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:40:20.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:40:20 smithi119 ceph-mon[31518]: pgmap v457: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T00:40:20.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:40:20 smithi119 ceph-mon[31518]: 98 slow requests, 3 included below; oldest blocked for > 280.665380 secs 2024-03-28T00:40:20.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:40:20 smithi119 ceph-mon[31518]: slow request 276.555936 seconds old, received at 2024-03-28T00:35:43.468389+0000: internal op fragmentdir:mds.0:3823 currently failed to wrlock, waiting 2024-03-28T00:40:20.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:40:20 smithi119 ceph-mon[31518]: slow request 276.573219 seconds old, received at 2024-03-28T00:35:43.451105+0000: internal op fragmentdir:mds.0:3793 currently failed to wrlock, waiting 2024-03-28T00:40:20.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:40:20 smithi119 ceph-mon[31518]: slow request 276.528377 seconds old, received at 2024-03-28T00:35:43.495948+0000: internal op fragmentdir:mds.0:3888 currently failed to wrlock, waiting 2024-03-28T00:40:20.705 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:40:20 smithi132 ceph-mon[35941]: pgmap v457: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T00:40:20.706 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:40:20 smithi132 ceph-mon[35941]: 98 slow requests, 3 included below; oldest blocked for > 280.665380 secs 2024-03-28T00:40:20.706 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:40:20 smithi132 ceph-mon[35941]: slow request 276.555936 seconds old, received at 2024-03-28T00:35:43.468389+0000: internal op fragmentdir:mds.0:3823 currently failed to wrlock, waiting 2024-03-28T00:40:20.706 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:40:20 smithi132 ceph-mon[35941]: slow request 276.573219 seconds old, received at 2024-03-28T00:35:43.451105+0000: internal op fragmentdir:mds.0:3793 currently failed to wrlock, waiting 2024-03-28T00:40:20.706 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:40:20 smithi132 ceph-mon[35941]: slow request 276.528377 seconds old, received at 2024-03-28T00:35:43.495948+0000: internal op fragmentdir:mds.0:3888 currently failed to wrlock, waiting 2024-03-28T00:40:20.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:40:20 smithi165 ceph-mon[36302]: pgmap v457: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T00:40:20.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:40:20 smithi165 ceph-mon[36302]: 98 slow requests, 3 included below; oldest blocked for > 280.665380 secs 2024-03-28T00:40:20.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:40:20 smithi165 ceph-mon[36302]: slow request 276.555936 seconds old, received at 2024-03-28T00:35:43.468389+0000: internal op fragmentdir:mds.0:3823 currently failed to wrlock, waiting 2024-03-28T00:40:20.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:40:20 smithi165 ceph-mon[36302]: slow request 276.573219 seconds old, received at 2024-03-28T00:35:43.451105+0000: internal op fragmentdir:mds.0:3793 currently failed to wrlock, waiting 2024-03-28T00:40:20.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:40:20 smithi165 ceph-mon[36302]: slow request 276.528377 seconds old, received at 2024-03-28T00:35:43.495948+0000: internal op fragmentdir:mds.0:3888 currently failed to wrlock, waiting 2024-03-28T00:40:22.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:40:22 smithi119 ceph-mon[31518]: pgmap v458: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T00:40:22.685 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:40:22 smithi165 ceph-mon[36302]: pgmap v458: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T00:40:22.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:40:22 smithi132 ceph-mon[35941]: pgmap v458: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T00:40:24.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:40:24 smithi119 ceph-mon[31518]: pgmap v459: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T00:40:24.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:40:24 smithi132 ceph-mon[35941]: pgmap v459: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T00:40:24.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:40:24 smithi165 ceph-mon[36302]: pgmap v459: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T00:40:25.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:40:25 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 285.665422 secs 2024-03-28T00:40:25.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:40:25 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 285.665422 secs 2024-03-28T00:40:25.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:40:25 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 285.665422 secs 2024-03-28T00:40:26.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:40:26 smithi132 ceph-mon[35941]: pgmap v460: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T00:40:26.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:40:26 smithi165 ceph-mon[36302]: pgmap v460: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T00:40:26.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:40:26 smithi119 ceph-mon[31518]: pgmap v460: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T00:40:28.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:40:28 smithi132 ceph-mon[35941]: pgmap v461: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T00:40:28.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:40:28 smithi165 ceph-mon[36302]: pgmap v461: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T00:40:28.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:40:28 smithi119 ceph-mon[31518]: pgmap v461: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T00:40:30.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:40:30 smithi132 ceph-mon[35941]: pgmap v462: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:40:30.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:40:30 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 290.665510 secs 2024-03-28T00:40:30.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:40:30 smithi165 ceph-mon[36302]: pgmap v462: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:40:30.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:40:30 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 290.665510 secs 2024-03-28T00:40:30.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:40:30 smithi119 ceph-mon[31518]: pgmap v462: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:40:30.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:40:30 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 290.665510 secs 2024-03-28T00:40:32.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:40:32 smithi132 ceph-mon[35941]: pgmap v463: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:40:32.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:40:32 smithi165 ceph-mon[36302]: pgmap v463: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:40:32.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:40:32 smithi119 ceph-mon[31518]: pgmap v463: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:40:34.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:40:34 smithi132 ceph-mon[35941]: pgmap v464: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:40:34.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:40:34 smithi165 ceph-mon[36302]: pgmap v464: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:40:34.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:40:34 smithi119 ceph-mon[31518]: pgmap v464: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:40:35.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:40:35 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 295.665572 secs 2024-03-28T00:40:35.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:40:35 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 295.665572 secs 2024-03-28T00:40:35.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:40:35 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 295.665572 secs 2024-03-28T00:40:36.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:40:36 smithi132 ceph-mon[35941]: pgmap v465: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:40:36.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:40:36 smithi165 ceph-mon[36302]: pgmap v465: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:40:36.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:40:36 smithi119 ceph-mon[31518]: pgmap v465: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:40:38.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:40:38 smithi132 ceph-mon[35941]: pgmap v466: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:40:38.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:40:38 smithi165 ceph-mon[36302]: pgmap v466: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:40:38.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:40:38 smithi119 ceph-mon[31518]: pgmap v466: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:40:40.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:40:40 smithi119 ceph-mon[31518]: pgmap v467: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:40:40.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:40:40 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 300.665610 secs 2024-03-28T00:40:40.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:40:40 smithi132 ceph-mon[35941]: pgmap v467: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:40:40.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:40:40 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 300.665610 secs 2024-03-28T00:40:40.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:40:40 smithi165 ceph-mon[36302]: pgmap v467: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:40:40.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:40:40 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 300.665610 secs 2024-03-28T00:40:42.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:40:42 smithi119 ceph-mon[31518]: pgmap v468: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:40:42.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:40:42 smithi165 ceph-mon[36302]: pgmap v468: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:40:42.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:40:42 smithi132 ceph-mon[35941]: pgmap v468: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:40:44.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:40:44 smithi165 ceph-mon[36302]: pgmap v469: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:40:44.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:40:44 smithi119 ceph-mon[31518]: pgmap v469: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:40:44.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:40:44 smithi132 ceph-mon[35941]: pgmap v469: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:40:45.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:40:45 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 305.665699 secs 2024-03-28T00:40:45.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:40:45 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 305.665699 secs 2024-03-28T00:40:45.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:40:45 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 305.665699 secs 2024-03-28T00:40:46.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:40:46 smithi165 ceph-mon[36302]: pgmap v470: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:40:46.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:40:46 smithi119 ceph-mon[31518]: pgmap v470: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:40:46.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:40:46 smithi132 ceph-mon[35941]: pgmap v470: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:40:48.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:40:48 smithi165 ceph-mon[36302]: pgmap v471: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:40:48.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:40:48 smithi119 ceph-mon[31518]: pgmap v471: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:40:48.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:40:48 smithi132 ceph-mon[35941]: pgmap v471: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:40:50.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:40:50 smithi165 ceph-mon[36302]: pgmap v472: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:40:50.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:40:50 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 310.665788 secs 2024-03-28T00:40:50.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:40:50 smithi119 ceph-mon[31518]: pgmap v472: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:40:50.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:40:50 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 310.665788 secs 2024-03-28T00:40:50.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:40:50 smithi132 ceph-mon[35941]: pgmap v472: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:40:50.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:40:50 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 310.665788 secs 2024-03-28T00:40:52.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:40:52 smithi165 ceph-mon[36302]: pgmap v473: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:40:52.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:40:52 smithi119 ceph-mon[31518]: pgmap v473: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:40:52.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:40:52 smithi132 ceph-mon[35941]: pgmap v473: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:40:54.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:40:54 smithi165 ceph-mon[36302]: pgmap v474: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:40:54.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:40:54 smithi119 ceph-mon[31518]: pgmap v474: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:40:54.954 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:40:54 smithi132 ceph-mon[35941]: pgmap v474: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:40:55.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:40:55 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 315.665889 secs 2024-03-28T00:40:55.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:40:55 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 315.665889 secs 2024-03-28T00:40:55.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:40:55 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 315.665889 secs 2024-03-28T00:40:56.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:40:56 smithi165 ceph-mon[36302]: pgmap v475: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:40:56.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:40:56 smithi119 ceph-mon[31518]: pgmap v475: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:40:56.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:40:56 smithi132 ceph-mon[35941]: pgmap v475: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:40:58.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:40:58 smithi165 ceph-mon[36302]: pgmap v476: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:40:58.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:40:58 smithi119 ceph-mon[31518]: pgmap v476: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:40:58.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:40:58 smithi132 ceph-mon[35941]: pgmap v476: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:41:00.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:41:00 smithi165 ceph-mon[36302]: pgmap v477: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:41:00.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:41:00 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 320.665975 secs 2024-03-28T00:41:00.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:41:00 smithi119 ceph-mon[31518]: pgmap v477: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:41:00.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:41:00 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 320.665975 secs 2024-03-28T00:41:00.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:41:00 smithi132 ceph-mon[35941]: pgmap v477: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:41:00.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:41:00 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 320.665975 secs 2024-03-28T00:41:02.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:41:02 smithi165 ceph-mon[36302]: pgmap v478: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:41:02.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:41:02 smithi119 ceph-mon[31518]: pgmap v478: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:41:02.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:41:02 smithi132 ceph-mon[35941]: pgmap v478: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:41:04.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:41:04 smithi165 ceph-mon[36302]: pgmap v479: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:41:04.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:41:04 smithi119 ceph-mon[31518]: pgmap v479: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:41:04.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:41:04 smithi132 ceph-mon[35941]: pgmap v479: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:41:05.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:41:05 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 325.666039 secs 2024-03-28T00:41:05.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:41:05 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 325.666039 secs 2024-03-28T00:41:05.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:41:05 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 325.666039 secs 2024-03-28T00:41:06.854 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:41:06 smithi132 ceph-mon[35941]: pgmap v480: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:41:06.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:41:06 smithi165 ceph-mon[36302]: pgmap v480: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:41:06.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:41:06 smithi119 ceph-mon[31518]: pgmap v480: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:41:08.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:41:08 smithi165 ceph-mon[36302]: pgmap v481: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:41:08.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:41:08 smithi119 ceph-mon[31518]: pgmap v481: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:41:08.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:41:08 smithi132 ceph-mon[35941]: pgmap v481: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:41:10.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:41:10 smithi165 ceph-mon[36302]: pgmap v482: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:41:10.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:41:10 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 330.666099 secs 2024-03-28T00:41:10.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:41:10 smithi119 ceph-mon[31518]: pgmap v482: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:41:10.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:41:10 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 330.666099 secs 2024-03-28T00:41:10.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:41:10 smithi132 ceph-mon[35941]: pgmap v482: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:41:10.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:41:10 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 330.666099 secs 2024-03-28T00:41:12.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:41:12 smithi165 ceph-mon[36302]: pgmap v483: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:41:12.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:41:12 smithi119 ceph-mon[31518]: pgmap v483: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:41:12.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:41:12 smithi132 ceph-mon[35941]: pgmap v483: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:41:14.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:41:14 smithi165 ceph-mon[36302]: pgmap v484: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:41:14.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:41:14 smithi119 ceph-mon[31518]: pgmap v484: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:41:14.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:41:14 smithi132 ceph-mon[35941]: pgmap v484: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:41:15.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:41:15 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 335.666185 secs 2024-03-28T00:41:15.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:41:15 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 335.666185 secs 2024-03-28T00:41:15.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:41:15 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 335.666185 secs 2024-03-28T00:41:16.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:41:16 smithi165 ceph-mon[36302]: pgmap v485: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:41:16.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:41:16 smithi119 ceph-mon[31518]: pgmap v485: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:41:16.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:41:16 smithi132 ceph-mon[35941]: pgmap v485: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:41:18.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:41:18 smithi165 ceph-mon[36302]: pgmap v486: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:41:18.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:41:18 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:41:18.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:41:18 smithi119 ceph-mon[31518]: pgmap v486: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:41:18.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:41:18 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:41:18.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:41:18 smithi132 ceph-mon[35941]: pgmap v486: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:41:18.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:41:18 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:41:19.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:41:19 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:41:19.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:41:19 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:41:19.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:41:19 smithi165 ceph-mon[36302]: pgmap v487: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T00:41:19.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:41:19 smithi165 ceph-mon[36302]: pgmap v488: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-28T00:41:19.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:41:19 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:41:19.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:41:19 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:41:19.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:41:19 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:41:19.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:41:19 smithi119 ceph-mon[31518]: pgmap v487: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T00:41:19.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:41:19 smithi119 ceph-mon[31518]: pgmap v488: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-28T00:41:19.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:41:19 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:41:19.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:41:19 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:41:19.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:41:19 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:41:19.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:41:19 smithi132 ceph-mon[35941]: pgmap v487: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T00:41:19.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:41:19 smithi132 ceph-mon[35941]: pgmap v488: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-28T00:41:19.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:41:19 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:41:20.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:41:20 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 340.666276 secs 2024-03-28T00:41:20.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:41:20 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 340.666276 secs 2024-03-28T00:41:20.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:41:20 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 340.666276 secs 2024-03-28T00:41:21.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:41:21 smithi165 ceph-mon[36302]: pgmap v489: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T00:41:21.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:41:21 smithi119 ceph-mon[31518]: pgmap v489: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T00:41:21.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:41:21 smithi132 ceph-mon[35941]: pgmap v489: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T00:41:23.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:41:22 smithi132 ceph-mon[35941]: pgmap v490: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T00:41:23.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:41:22 smithi165 ceph-mon[36302]: pgmap v490: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T00:41:23.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:41:22 smithi119 ceph-mon[31518]: pgmap v490: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T00:41:25.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:41:24 smithi132 ceph-mon[35941]: pgmap v491: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T00:41:25.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:41:24 smithi165 ceph-mon[36302]: pgmap v491: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T00:41:25.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:41:24 smithi119 ceph-mon[31518]: pgmap v491: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T00:41:26.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:41:25 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 345.666366 secs 2024-03-28T00:41:26.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:41:25 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 345.666366 secs 2024-03-28T00:41:26.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:41:25 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 345.666366 secs 2024-03-28T00:41:27.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:41:26 smithi132 ceph-mon[35941]: pgmap v492: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T00:41:27.333 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:41:26 smithi119 ceph-mon[31518]: pgmap v492: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T00:41:27.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:41:26 smithi165 ceph-mon[36302]: pgmap v492: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T00:41:29.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:41:28 smithi132 ceph-mon[35941]: pgmap v493: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T00:41:29.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:41:28 smithi165 ceph-mon[36302]: pgmap v493: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T00:41:29.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:41:28 smithi119 ceph-mon[31518]: pgmap v493: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T00:41:30.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:41:30 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 350.666428 secs 2024-03-28T00:41:30.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:41:30 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 350.666428 secs 2024-03-28T00:41:30.452 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:41:30 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 350.666428 secs 2024-03-28T00:41:31.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:41:31 smithi165 ceph-mon[36302]: pgmap v494: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:41:31.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:41:31 smithi119 ceph-mon[31518]: pgmap v494: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:41:31.452 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:41:31 smithi132 ceph-mon[35941]: pgmap v494: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:41:33.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:41:32 smithi132 ceph-mon[35941]: pgmap v495: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:41:33.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:41:32 smithi165 ceph-mon[36302]: pgmap v495: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:41:33.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:41:32 smithi119 ceph-mon[31518]: pgmap v495: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:41:35.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:41:34 smithi132 ceph-mon[35941]: pgmap v496: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:41:35.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:41:34 smithi165 ceph-mon[36302]: pgmap v496: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:41:35.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:41:34 smithi119 ceph-mon[31518]: pgmap v496: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:41:36.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:41:35 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 355.666490 secs 2024-03-28T00:41:36.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:41:35 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 355.666490 secs 2024-03-28T00:41:36.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:41:35 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 355.666490 secs 2024-03-28T00:41:37.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:41:36 smithi132 ceph-mon[35941]: pgmap v497: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:41:37.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:41:36 smithi165 ceph-mon[36302]: pgmap v497: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:41:37.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:41:36 smithi119 ceph-mon[31518]: pgmap v497: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:41:39.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:41:38 smithi132 ceph-mon[35941]: pgmap v498: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:41:39.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:41:38 smithi165 ceph-mon[36302]: pgmap v498: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:41:39.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:41:38 smithi119 ceph-mon[31518]: pgmap v498: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:41:40.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:41:40 smithi165 ceph-mon[36302]: 98 slow requests, 3 included below; oldest blocked for > 360.666577 secs 2024-03-28T00:41:40.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:41:40 smithi165 ceph-mon[36302]: slow request 360.446602 seconds old, received at 2024-03-28T00:35:39.578920+0000: internal op fragmentdir:mds.0:3783 currently failed to wrlock, waiting 2024-03-28T00:41:40.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:41:40 smithi165 ceph-mon[36302]: slow request 360.437654 seconds old, received at 2024-03-28T00:35:39.587868+0000: client_request(client.25264:33202 create owner_uid=1000, owner_gid=1301 #0x10000000270/file8309 2024-03-28T00:35:39.586399+0000 caller_uid=1000, caller_gid=1301{6,36,1000,1301,}) currently failed to xlock, waiting 2024-03-28T00:41:40.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:41:40 smithi165 ceph-mon[36302]: slow request 360.666577 seconds old, received at 2024-03-28T00:35:39.358945+0000: peer_request:client.25264:11489 currently acquired locks 2024-03-28T00:41:40.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:41:40 smithi119 ceph-mon[31518]: 98 slow requests, 3 included below; oldest blocked for > 360.666577 secs 2024-03-28T00:41:40.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:41:40 smithi119 ceph-mon[31518]: slow request 360.446602 seconds old, received at 2024-03-28T00:35:39.578920+0000: internal op fragmentdir:mds.0:3783 currently failed to wrlock, waiting 2024-03-28T00:41:40.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:41:40 smithi119 ceph-mon[31518]: slow request 360.437654 seconds old, received at 2024-03-28T00:35:39.587868+0000: client_request(client.25264:33202 create owner_uid=1000, owner_gid=1301 #0x10000000270/file8309 2024-03-28T00:35:39.586399+0000 caller_uid=1000, caller_gid=1301{6,36,1000,1301,}) currently failed to xlock, waiting 2024-03-28T00:41:40.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:41:40 smithi119 ceph-mon[31518]: slow request 360.666577 seconds old, received at 2024-03-28T00:35:39.358945+0000: peer_request:client.25264:11489 currently acquired locks 2024-03-28T00:41:40.452 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:41:40 smithi132 ceph-mon[35941]: 98 slow requests, 3 included below; oldest blocked for > 360.666577 secs 2024-03-28T00:41:40.452 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:41:40 smithi132 ceph-mon[35941]: slow request 360.446602 seconds old, received at 2024-03-28T00:35:39.578920+0000: internal op fragmentdir:mds.0:3783 currently failed to wrlock, waiting 2024-03-28T00:41:40.452 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:41:40 smithi132 ceph-mon[35941]: slow request 360.437654 seconds old, received at 2024-03-28T00:35:39.587868+0000: client_request(client.25264:33202 create owner_uid=1000, owner_gid=1301 #0x10000000270/file8309 2024-03-28T00:35:39.586399+0000 caller_uid=1000, caller_gid=1301{6,36,1000,1301,}) currently failed to xlock, waiting 2024-03-28T00:41:40.452 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:41:40 smithi132 ceph-mon[35941]: slow request 360.666577 seconds old, received at 2024-03-28T00:35:39.358945+0000: peer_request:client.25264:11489 currently acquired locks 2024-03-28T00:41:41.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:41:41 smithi165 ceph-mon[36302]: pgmap v499: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:41:41.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:41:41 smithi119 ceph-mon[31518]: pgmap v499: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:41:41.452 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:41:41 smithi132 ceph-mon[35941]: pgmap v499: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:41:43.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:41:42 smithi132 ceph-mon[35941]: pgmap v500: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:41:43.206 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:41:42 smithi119 ceph-mon[31518]: pgmap v500: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:41:43.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:41:42 smithi165 ceph-mon[36302]: pgmap v500: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:41:45.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:41:44 smithi132 ceph-mon[35941]: pgmap v501: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:41:45.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:41:44 smithi165 ceph-mon[36302]: pgmap v501: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:41:45.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:41:44 smithi119 ceph-mon[31518]: pgmap v501: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:41:46.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:41:46 smithi165 ceph-mon[36302]: 98 slow requests, 5 included below; oldest blocked for > 365.666667 secs 2024-03-28T00:41:46.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:41:46 smithi165 ceph-mon[36302]: slow request 361.574004 seconds old, received at 2024-03-28T00:35:43.451608+0000: internal op fragmentdir:mds.0:3794 currently failed to wrlock, waiting 2024-03-28T00:41:46.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:41:46 smithi165 ceph-mon[36302]: slow request 361.556336 seconds old, received at 2024-03-28T00:35:43.469275+0000: internal op fragmentdir:mds.0:3825 currently failed to wrlock, waiting 2024-03-28T00:41:46.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:41:46 smithi165 ceph-mon[36302]: slow request 361.542244 seconds old, received at 2024-03-28T00:35:43.483367+0000: internal op fragmentdir:mds.0:3857 currently failed to wrlock, waiting 2024-03-28T00:41:46.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:41:46 smithi165 ceph-mon[36302]: slow request 361.529183 seconds old, received at 2024-03-28T00:35:43.496429+0000: internal op fragmentdir:mds.0:3889 currently failed to wrlock, waiting 2024-03-28T00:41:46.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:41:46 smithi165 ceph-mon[36302]: slow request 361.572028 seconds old, received at 2024-03-28T00:35:43.453584+0000: internal op fragmentdir:mds.0:3795 currently failed to wrlock, waiting 2024-03-28T00:41:46.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:41:46 smithi119 ceph-mon[31518]: 98 slow requests, 5 included below; oldest blocked for > 365.666667 secs 2024-03-28T00:41:46.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:41:46 smithi119 ceph-mon[31518]: slow request 361.574004 seconds old, received at 2024-03-28T00:35:43.451608+0000: internal op fragmentdir:mds.0:3794 currently failed to wrlock, waiting 2024-03-28T00:41:46.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:41:46 smithi119 ceph-mon[31518]: slow request 361.556336 seconds old, received at 2024-03-28T00:35:43.469275+0000: internal op fragmentdir:mds.0:3825 currently failed to wrlock, waiting 2024-03-28T00:41:46.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:41:46 smithi119 ceph-mon[31518]: slow request 361.542244 seconds old, received at 2024-03-28T00:35:43.483367+0000: internal op fragmentdir:mds.0:3857 currently failed to wrlock, waiting 2024-03-28T00:41:46.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:41:46 smithi119 ceph-mon[31518]: slow request 361.529183 seconds old, received at 2024-03-28T00:35:43.496429+0000: internal op fragmentdir:mds.0:3889 currently failed to wrlock, waiting 2024-03-28T00:41:46.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:41:46 smithi119 ceph-mon[31518]: slow request 361.572028 seconds old, received at 2024-03-28T00:35:43.453584+0000: internal op fragmentdir:mds.0:3795 currently failed to wrlock, waiting 2024-03-28T00:41:46.452 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:41:46 smithi132 ceph-mon[35941]: 98 slow requests, 5 included below; oldest blocked for > 365.666667 secs 2024-03-28T00:41:46.452 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:41:46 smithi132 ceph-mon[35941]: slow request 361.574004 seconds old, received at 2024-03-28T00:35:43.451608+0000: internal op fragmentdir:mds.0:3794 currently failed to wrlock, waiting 2024-03-28T00:41:46.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:41:46 smithi132 ceph-mon[35941]: slow request 361.556336 seconds old, received at 2024-03-28T00:35:43.469275+0000: internal op fragmentdir:mds.0:3825 currently failed to wrlock, waiting 2024-03-28T00:41:46.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:41:46 smithi132 ceph-mon[35941]: slow request 361.542244 seconds old, received at 2024-03-28T00:35:43.483367+0000: internal op fragmentdir:mds.0:3857 currently failed to wrlock, waiting 2024-03-28T00:41:46.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:41:46 smithi132 ceph-mon[35941]: slow request 361.529183 seconds old, received at 2024-03-28T00:35:43.496429+0000: internal op fragmentdir:mds.0:3889 currently failed to wrlock, waiting 2024-03-28T00:41:46.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:41:46 smithi132 ceph-mon[35941]: slow request 361.572028 seconds old, received at 2024-03-28T00:35:43.453584+0000: internal op fragmentdir:mds.0:3795 currently failed to wrlock, waiting 2024-03-28T00:41:47.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:41:47 smithi165 ceph-mon[36302]: pgmap v502: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:41:47.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:41:47 smithi119 ceph-mon[31518]: pgmap v502: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:41:47.452 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:41:47 smithi132 ceph-mon[35941]: pgmap v502: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:41:49.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:41:48 smithi132 ceph-mon[35941]: pgmap v503: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:41:49.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:41:48 smithi165 ceph-mon[36302]: pgmap v503: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:41:49.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:41:48 smithi119 ceph-mon[31518]: pgmap v503: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:41:50.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:41:50 smithi165 ceph-mon[36302]: 98 slow requests, 5 included below; oldest blocked for > 370.666727 secs 2024-03-28T00:41:50.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:41:50 smithi165 ceph-mon[36302]: slow request 366.490825 seconds old, received at 2024-03-28T00:35:43.534847+0000: internal op fragmentdir:mds.0:3922 currently failed to wrlock, waiting 2024-03-28T00:41:50.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:41:50 smithi165 ceph-mon[36302]: slow request 366.571568 seconds old, received at 2024-03-28T00:35:43.454104+0000: internal op fragmentdir:mds.0:3796 currently failed to wrlock, waiting 2024-03-28T00:41:50.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:41:50 smithi165 ceph-mon[36302]: slow request 366.555508 seconds old, received at 2024-03-28T00:35:43.470163+0000: internal op fragmentdir:mds.0:3827 currently failed to wrlock, waiting 2024-03-28T00:41:50.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:41:50 smithi165 ceph-mon[36302]: slow request 366.528346 seconds old, received at 2024-03-28T00:35:43.497326+0000: internal op fragmentdir:mds.0:3891 currently failed to wrlock, waiting 2024-03-28T00:41:50.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:41:50 smithi165 ceph-mon[36302]: slow request 366.464095 seconds old, received at 2024-03-28T00:35:43.561577+0000: internal op fragmentdir:mds.0:3923 currently failed to wrlock, waiting 2024-03-28T00:41:50.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:41:50 smithi119 ceph-mon[31518]: 98 slow requests, 5 included below; oldest blocked for > 370.666727 secs 2024-03-28T00:41:50.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:41:50 smithi119 ceph-mon[31518]: slow request 366.490825 seconds old, received at 2024-03-28T00:35:43.534847+0000: internal op fragmentdir:mds.0:3922 currently failed to wrlock, waiting 2024-03-28T00:41:50.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:41:50 smithi119 ceph-mon[31518]: slow request 366.571568 seconds old, received at 2024-03-28T00:35:43.454104+0000: internal op fragmentdir:mds.0:3796 currently failed to wrlock, waiting 2024-03-28T00:41:50.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:41:50 smithi119 ceph-mon[31518]: slow request 366.555508 seconds old, received at 2024-03-28T00:35:43.470163+0000: internal op fragmentdir:mds.0:3827 currently failed to wrlock, waiting 2024-03-28T00:41:50.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:41:50 smithi119 ceph-mon[31518]: slow request 366.528346 seconds old, received at 2024-03-28T00:35:43.497326+0000: internal op fragmentdir:mds.0:3891 currently failed to wrlock, waiting 2024-03-28T00:41:50.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:41:50 smithi119 ceph-mon[31518]: slow request 366.464095 seconds old, received at 2024-03-28T00:35:43.561577+0000: internal op fragmentdir:mds.0:3923 currently failed to wrlock, waiting 2024-03-28T00:41:50.452 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:41:50 smithi132 ceph-mon[35941]: 98 slow requests, 5 included below; oldest blocked for > 370.666727 secs 2024-03-28T00:41:50.452 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:41:50 smithi132 ceph-mon[35941]: slow request 366.490825 seconds old, received at 2024-03-28T00:35:43.534847+0000: internal op fragmentdir:mds.0:3922 currently failed to wrlock, waiting 2024-03-28T00:41:50.452 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:41:50 smithi132 ceph-mon[35941]: slow request 366.571568 seconds old, received at 2024-03-28T00:35:43.454104+0000: internal op fragmentdir:mds.0:3796 currently failed to wrlock, waiting 2024-03-28T00:41:50.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:41:50 smithi132 ceph-mon[35941]: slow request 366.555508 seconds old, received at 2024-03-28T00:35:43.470163+0000: internal op fragmentdir:mds.0:3827 currently failed to wrlock, waiting 2024-03-28T00:41:50.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:41:50 smithi132 ceph-mon[35941]: slow request 366.528346 seconds old, received at 2024-03-28T00:35:43.497326+0000: internal op fragmentdir:mds.0:3891 currently failed to wrlock, waiting 2024-03-28T00:41:50.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:41:50 smithi132 ceph-mon[35941]: slow request 366.464095 seconds old, received at 2024-03-28T00:35:43.561577+0000: internal op fragmentdir:mds.0:3923 currently failed to wrlock, waiting 2024-03-28T00:41:51.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:41:51 smithi119 ceph-mon[31518]: pgmap v504: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:41:51.452 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:41:51 smithi132 ceph-mon[35941]: pgmap v504: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:41:51.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:41:51 smithi165 ceph-mon[36302]: pgmap v504: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:41:53.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:41:52 smithi132 ceph-mon[35941]: pgmap v505: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:41:53.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:41:52 smithi165 ceph-mon[36302]: pgmap v505: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:41:53.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:41:52 smithi119 ceph-mon[31518]: pgmap v505: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:41:55.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:41:54 smithi132 ceph-mon[35941]: pgmap v506: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:41:55.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:41:54 smithi165 ceph-mon[36302]: pgmap v506: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:41:55.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:41:54 smithi119 ceph-mon[31518]: pgmap v506: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:41:56.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:41:55 smithi132 ceph-mon[35941]: 98 slow requests, 5 included below; oldest blocked for > 375.666823 secs 2024-03-28T00:41:56.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:41:55 smithi132 ceph-mon[35941]: slow request 371.571190 seconds old, received at 2024-03-28T00:35:43.454578+0000: internal op fragmentdir:mds.0:3797 currently failed to wrlock, waiting 2024-03-28T00:41:56.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:41:55 smithi132 ceph-mon[35941]: slow request 371.555163 seconds old, received at 2024-03-28T00:35:43.470605+0000: internal op fragmentdir:mds.0:3828 currently failed to wrlock, waiting 2024-03-28T00:41:56.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:41:55 smithi132 ceph-mon[35941]: slow request 371.541198 seconds old, received at 2024-03-28T00:35:43.484570+0000: internal op fragmentdir:mds.0:3860 currently failed to wrlock, waiting 2024-03-28T00:41:56.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:41:55 smithi132 ceph-mon[35941]: slow request 371.527992 seconds old, received at 2024-03-28T00:35:43.497776+0000: internal op fragmentdir:mds.0:3892 currently failed to wrlock, waiting 2024-03-28T00:41:56.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:41:55 smithi132 ceph-mon[35941]: slow request 371.570713 seconds old, received at 2024-03-28T00:35:43.455055+0000: internal op fragmentdir:mds.0:3798 currently failed to wrlock, waiting 2024-03-28T00:41:56.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:41:55 smithi165 ceph-mon[36302]: 98 slow requests, 5 included below; oldest blocked for > 375.666823 secs 2024-03-28T00:41:56.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:41:55 smithi165 ceph-mon[36302]: slow request 371.571190 seconds old, received at 2024-03-28T00:35:43.454578+0000: internal op fragmentdir:mds.0:3797 currently failed to wrlock, waiting 2024-03-28T00:41:56.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:41:55 smithi165 ceph-mon[36302]: slow request 371.555163 seconds old, received at 2024-03-28T00:35:43.470605+0000: internal op fragmentdir:mds.0:3828 currently failed to wrlock, waiting 2024-03-28T00:41:56.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:41:55 smithi165 ceph-mon[36302]: slow request 371.541198 seconds old, received at 2024-03-28T00:35:43.484570+0000: internal op fragmentdir:mds.0:3860 currently failed to wrlock, waiting 2024-03-28T00:41:56.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:41:55 smithi165 ceph-mon[36302]: slow request 371.527992 seconds old, received at 2024-03-28T00:35:43.497776+0000: internal op fragmentdir:mds.0:3892 currently failed to wrlock, waiting 2024-03-28T00:41:56.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:41:55 smithi165 ceph-mon[36302]: slow request 371.570713 seconds old, received at 2024-03-28T00:35:43.455055+0000: internal op fragmentdir:mds.0:3798 currently failed to wrlock, waiting 2024-03-28T00:41:56.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:41:55 smithi119 ceph-mon[31518]: 98 slow requests, 5 included below; oldest blocked for > 375.666823 secs 2024-03-28T00:41:56.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:41:55 smithi119 ceph-mon[31518]: slow request 371.571190 seconds old, received at 2024-03-28T00:35:43.454578+0000: internal op fragmentdir:mds.0:3797 currently failed to wrlock, waiting 2024-03-28T00:41:56.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:41:55 smithi119 ceph-mon[31518]: slow request 371.555163 seconds old, received at 2024-03-28T00:35:43.470605+0000: internal op fragmentdir:mds.0:3828 currently failed to wrlock, waiting 2024-03-28T00:41:56.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:41:55 smithi119 ceph-mon[31518]: slow request 371.541198 seconds old, received at 2024-03-28T00:35:43.484570+0000: internal op fragmentdir:mds.0:3860 currently failed to wrlock, waiting 2024-03-28T00:41:56.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:41:55 smithi119 ceph-mon[31518]: slow request 371.527992 seconds old, received at 2024-03-28T00:35:43.497776+0000: internal op fragmentdir:mds.0:3892 currently failed to wrlock, waiting 2024-03-28T00:41:56.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:41:55 smithi119 ceph-mon[31518]: slow request 371.570713 seconds old, received at 2024-03-28T00:35:43.455055+0000: internal op fragmentdir:mds.0:3798 currently failed to wrlock, waiting 2024-03-28T00:41:57.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:41:56 smithi132 ceph-mon[35941]: pgmap v507: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:41:57.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:41:56 smithi165 ceph-mon[36302]: pgmap v507: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:41:57.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:41:56 smithi119 ceph-mon[31518]: pgmap v507: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:41:59.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:41:58 smithi132 ceph-mon[35941]: pgmap v508: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:41:59.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:41:58 smithi165 ceph-mon[36302]: pgmap v508: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:41:59.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:41:58 smithi119 ceph-mon[31518]: pgmap v508: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:42:00.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:42:00 smithi165 ceph-mon[36302]: 98 slow requests, 5 included below; oldest blocked for > 380.666915 secs 2024-03-28T00:42:00.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:42:00 smithi165 ceph-mon[36302]: slow request 376.526743 seconds old, received at 2024-03-28T00:35:43.499118+0000: internal op fragmentdir:mds.0:3893 currently failed to wrlock, waiting 2024-03-28T00:42:00.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:42:00 smithi165 ceph-mon[36302]: slow request 376.570332 seconds old, received at 2024-03-28T00:35:43.455529+0000: internal op fragmentdir:mds.0:3799 currently failed to wrlock, waiting 2024-03-28T00:42:00.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:42:00 smithi165 ceph-mon[36302]: slow request 376.540463 seconds old, received at 2024-03-28T00:35:43.485397+0000: internal op fragmentdir:mds.0:3862 currently failed to wrlock, waiting 2024-03-28T00:42:00.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:42:00 smithi165 ceph-mon[36302]: slow request 376.526292 seconds old, received at 2024-03-28T00:35:43.499568+0000: internal op fragmentdir:mds.0:3894 currently failed to wrlock, waiting 2024-03-28T00:42:00.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:42:00 smithi165 ceph-mon[36302]: slow request 376.569853 seconds old, received at 2024-03-28T00:35:43.456007+0000: internal op fragmentdir:mds.0:3800 currently failed to wrlock, waiting 2024-03-28T00:42:00.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:42:00 smithi119 ceph-mon[31518]: 98 slow requests, 5 included below; oldest blocked for > 380.666915 secs 2024-03-28T00:42:00.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:42:00 smithi119 ceph-mon[31518]: slow request 376.526743 seconds old, received at 2024-03-28T00:35:43.499118+0000: internal op fragmentdir:mds.0:3893 currently failed to wrlock, waiting 2024-03-28T00:42:00.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:42:00 smithi119 ceph-mon[31518]: slow request 376.570332 seconds old, received at 2024-03-28T00:35:43.455529+0000: internal op fragmentdir:mds.0:3799 currently failed to wrlock, waiting 2024-03-28T00:42:00.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:42:00 smithi119 ceph-mon[31518]: slow request 376.540463 seconds old, received at 2024-03-28T00:35:43.485397+0000: internal op fragmentdir:mds.0:3862 currently failed to wrlock, waiting 2024-03-28T00:42:00.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:42:00 smithi119 ceph-mon[31518]: slow request 376.526292 seconds old, received at 2024-03-28T00:35:43.499568+0000: internal op fragmentdir:mds.0:3894 currently failed to wrlock, waiting 2024-03-28T00:42:00.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:42:00 smithi119 ceph-mon[31518]: slow request 376.569853 seconds old, received at 2024-03-28T00:35:43.456007+0000: internal op fragmentdir:mds.0:3800 currently failed to wrlock, waiting 2024-03-28T00:42:00.452 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:42:00 smithi132 ceph-mon[35941]: 98 slow requests, 5 included below; oldest blocked for > 380.666915 secs 2024-03-28T00:42:00.452 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:42:00 smithi132 ceph-mon[35941]: slow request 376.526743 seconds old, received at 2024-03-28T00:35:43.499118+0000: internal op fragmentdir:mds.0:3893 currently failed to wrlock, waiting 2024-03-28T00:42:00.452 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:42:00 smithi132 ceph-mon[35941]: slow request 376.570332 seconds old, received at 2024-03-28T00:35:43.455529+0000: internal op fragmentdir:mds.0:3799 currently failed to wrlock, waiting 2024-03-28T00:42:00.452 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:42:00 smithi132 ceph-mon[35941]: slow request 376.540463 seconds old, received at 2024-03-28T00:35:43.485397+0000: internal op fragmentdir:mds.0:3862 currently failed to wrlock, waiting 2024-03-28T00:42:00.452 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:42:00 smithi132 ceph-mon[35941]: slow request 376.526292 seconds old, received at 2024-03-28T00:35:43.499568+0000: internal op fragmentdir:mds.0:3894 currently failed to wrlock, waiting 2024-03-28T00:42:00.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:42:00 smithi132 ceph-mon[35941]: slow request 376.569853 seconds old, received at 2024-03-28T00:35:43.456007+0000: internal op fragmentdir:mds.0:3800 currently failed to wrlock, waiting 2024-03-28T00:42:01.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:42:01 smithi165 ceph-mon[36302]: pgmap v509: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:42:01.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:42:01 smithi119 ceph-mon[31518]: pgmap v509: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:42:01.452 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:42:01 smithi132 ceph-mon[35941]: pgmap v509: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:42:03.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:42:02 smithi132 ceph-mon[35941]: pgmap v510: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:42:03.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:42:02 smithi165 ceph-mon[36302]: pgmap v510: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:42:03.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:42:02 smithi119 ceph-mon[31518]: pgmap v510: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:42:05.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:42:04 smithi132 ceph-mon[35941]: pgmap v511: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:42:05.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:42:04 smithi165 ceph-mon[36302]: pgmap v511: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:42:05.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:42:04 smithi119 ceph-mon[31518]: pgmap v511: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:42:06.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:42:05 smithi132 ceph-mon[35941]: 98 slow requests, 5 included below; oldest blocked for > 385.666952 secs 2024-03-28T00:42:06.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:42:05 smithi132 ceph-mon[35941]: slow request 381.554224 seconds old, received at 2024-03-28T00:35:43.471672+0000: internal op fragmentdir:mds.0:3831 currently failed to wrlock, waiting 2024-03-28T00:42:06.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:42:05 smithi132 ceph-mon[35941]: slow request 381.540108 seconds old, received at 2024-03-28T00:35:43.485788+0000: internal op fragmentdir:mds.0:3863 currently failed to wrlock, waiting 2024-03-28T00:42:06.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:42:05 smithi132 ceph-mon[35941]: slow request 381.525865 seconds old, received at 2024-03-28T00:35:43.500032+0000: internal op fragmentdir:mds.0:3895 currently failed to wrlock, waiting 2024-03-28T00:42:06.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:42:05 smithi132 ceph-mon[35941]: slow request 381.569433 seconds old, received at 2024-03-28T00:35:43.456464+0000: internal op fragmentdir:mds.0:3801 currently failed to wrlock, waiting 2024-03-28T00:42:06.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:42:05 smithi132 ceph-mon[35941]: slow request 381.553765 seconds old, received at 2024-03-28T00:35:43.472131+0000: internal op fragmentdir:mds.0:3832 currently failed to wrlock, waiting 2024-03-28T00:42:06.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:42:05 smithi165 ceph-mon[36302]: 98 slow requests, 5 included below; oldest blocked for > 385.666952 secs 2024-03-28T00:42:06.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:42:05 smithi165 ceph-mon[36302]: slow request 381.554224 seconds old, received at 2024-03-28T00:35:43.471672+0000: internal op fragmentdir:mds.0:3831 currently failed to wrlock, waiting 2024-03-28T00:42:06.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:42:05 smithi165 ceph-mon[36302]: slow request 381.540108 seconds old, received at 2024-03-28T00:35:43.485788+0000: internal op fragmentdir:mds.0:3863 currently failed to wrlock, waiting 2024-03-28T00:42:06.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:42:05 smithi165 ceph-mon[36302]: slow request 381.525865 seconds old, received at 2024-03-28T00:35:43.500032+0000: internal op fragmentdir:mds.0:3895 currently failed to wrlock, waiting 2024-03-28T00:42:06.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:42:05 smithi165 ceph-mon[36302]: slow request 381.569433 seconds old, received at 2024-03-28T00:35:43.456464+0000: internal op fragmentdir:mds.0:3801 currently failed to wrlock, waiting 2024-03-28T00:42:06.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:42:05 smithi165 ceph-mon[36302]: slow request 381.553765 seconds old, received at 2024-03-28T00:35:43.472131+0000: internal op fragmentdir:mds.0:3832 currently failed to wrlock, waiting 2024-03-28T00:42:06.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:42:05 smithi119 ceph-mon[31518]: 98 slow requests, 5 included below; oldest blocked for > 385.666952 secs 2024-03-28T00:42:06.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:42:05 smithi119 ceph-mon[31518]: slow request 381.554224 seconds old, received at 2024-03-28T00:35:43.471672+0000: internal op fragmentdir:mds.0:3831 currently failed to wrlock, waiting 2024-03-28T00:42:06.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:42:05 smithi119 ceph-mon[31518]: slow request 381.540108 seconds old, received at 2024-03-28T00:35:43.485788+0000: internal op fragmentdir:mds.0:3863 currently failed to wrlock, waiting 2024-03-28T00:42:06.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:42:05 smithi119 ceph-mon[31518]: slow request 381.525865 seconds old, received at 2024-03-28T00:35:43.500032+0000: internal op fragmentdir:mds.0:3895 currently failed to wrlock, waiting 2024-03-28T00:42:06.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:42:05 smithi119 ceph-mon[31518]: slow request 381.569433 seconds old, received at 2024-03-28T00:35:43.456464+0000: internal op fragmentdir:mds.0:3801 currently failed to wrlock, waiting 2024-03-28T00:42:06.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:42:05 smithi119 ceph-mon[31518]: slow request 381.553765 seconds old, received at 2024-03-28T00:35:43.472131+0000: internal op fragmentdir:mds.0:3832 currently failed to wrlock, waiting 2024-03-28T00:42:07.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:42:06 smithi132 ceph-mon[35941]: pgmap v512: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:42:07.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:42:06 smithi165 ceph-mon[36302]: pgmap v512: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:42:07.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:42:06 smithi119 ceph-mon[31518]: pgmap v512: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:42:09.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:42:08 smithi132 ceph-mon[35941]: pgmap v513: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:42:09.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:42:08 smithi165 ceph-mon[36302]: pgmap v513: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:42:09.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:42:08 smithi119 ceph-mon[31518]: pgmap v513: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:42:10.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:42:10 smithi165 ceph-mon[36302]: 98 slow requests, 5 included below; oldest blocked for > 390.667014 secs 2024-03-28T00:42:10.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:42:10 smithi165 ceph-mon[36302]: slow request 386.568978 seconds old, received at 2024-03-28T00:35:43.456981+0000: internal op fragmentdir:mds.0:3802 currently failed to wrlock, waiting 2024-03-28T00:42:10.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:42:10 smithi165 ceph-mon[36302]: slow request 386.539369 seconds old, received at 2024-03-28T00:35:43.486590+0000: internal op fragmentdir:mds.0:3865 currently failed to wrlock, waiting 2024-03-28T00:42:10.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:42:10 smithi165 ceph-mon[36302]: slow request 386.525054 seconds old, received at 2024-03-28T00:35:43.500904+0000: internal op fragmentdir:mds.0:3897 currently failed to wrlock, waiting 2024-03-28T00:42:10.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:42:10 smithi165 ceph-mon[36302]: slow request 386.568529 seconds old, received at 2024-03-28T00:35:43.457430+0000: internal op fragmentdir:mds.0:3803 currently failed to wrlock, waiting 2024-03-28T00:42:10.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:42:10 smithi165 ceph-mon[36302]: slow request 386.552943 seconds old, received at 2024-03-28T00:35:43.473016+0000: internal op fragmentdir:mds.0:3834 currently failed to wrlock, waiting 2024-03-28T00:42:10.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:42:10 smithi119 ceph-mon[31518]: 98 slow requests, 5 included below; oldest blocked for > 390.667014 secs 2024-03-28T00:42:10.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:42:10 smithi119 ceph-mon[31518]: slow request 386.568978 seconds old, received at 2024-03-28T00:35:43.456981+0000: internal op fragmentdir:mds.0:3802 currently failed to wrlock, waiting 2024-03-28T00:42:10.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:42:10 smithi119 ceph-mon[31518]: slow request 386.539369 seconds old, received at 2024-03-28T00:35:43.486590+0000: internal op fragmentdir:mds.0:3865 currently failed to wrlock, waiting 2024-03-28T00:42:10.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:42:10 smithi119 ceph-mon[31518]: slow request 386.525054 seconds old, received at 2024-03-28T00:35:43.500904+0000: internal op fragmentdir:mds.0:3897 currently failed to wrlock, waiting 2024-03-28T00:42:10.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:42:10 smithi119 ceph-mon[31518]: slow request 386.568529 seconds old, received at 2024-03-28T00:35:43.457430+0000: internal op fragmentdir:mds.0:3803 currently failed to wrlock, waiting 2024-03-28T00:42:10.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:42:10 smithi119 ceph-mon[31518]: slow request 386.552943 seconds old, received at 2024-03-28T00:35:43.473016+0000: internal op fragmentdir:mds.0:3834 currently failed to wrlock, waiting 2024-03-28T00:42:10.452 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:42:10 smithi132 ceph-mon[35941]: 98 slow requests, 5 included below; oldest blocked for > 390.667014 secs 2024-03-28T00:42:10.452 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:42:10 smithi132 ceph-mon[35941]: slow request 386.568978 seconds old, received at 2024-03-28T00:35:43.456981+0000: internal op fragmentdir:mds.0:3802 currently failed to wrlock, waiting 2024-03-28T00:42:10.452 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:42:10 smithi132 ceph-mon[35941]: slow request 386.539369 seconds old, received at 2024-03-28T00:35:43.486590+0000: internal op fragmentdir:mds.0:3865 currently failed to wrlock, waiting 2024-03-28T00:42:10.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:42:10 smithi132 ceph-mon[35941]: slow request 386.525054 seconds old, received at 2024-03-28T00:35:43.500904+0000: internal op fragmentdir:mds.0:3897 currently failed to wrlock, waiting 2024-03-28T00:42:10.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:42:10 smithi132 ceph-mon[35941]: slow request 386.568529 seconds old, received at 2024-03-28T00:35:43.457430+0000: internal op fragmentdir:mds.0:3803 currently failed to wrlock, waiting 2024-03-28T00:42:10.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:42:10 smithi132 ceph-mon[35941]: slow request 386.552943 seconds old, received at 2024-03-28T00:35:43.473016+0000: internal op fragmentdir:mds.0:3834 currently failed to wrlock, waiting 2024-03-28T00:42:11.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:42:11 smithi165 ceph-mon[36302]: pgmap v514: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:42:11.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:42:11 smithi119 ceph-mon[31518]: pgmap v514: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:42:11.452 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:42:11 smithi132 ceph-mon[35941]: pgmap v514: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:42:13.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:42:12 smithi132 ceph-mon[35941]: pgmap v515: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:42:13.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:42:12 smithi165 ceph-mon[36302]: pgmap v515: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:42:13.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:42:12 smithi119 ceph-mon[31518]: pgmap v515: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:42:15.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:42:14 smithi132 ceph-mon[35941]: pgmap v516: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:42:15.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:42:14 smithi165 ceph-mon[36302]: pgmap v516: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:42:15.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:42:14 smithi119 ceph-mon[31518]: pgmap v516: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:42:16.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:42:15 smithi132 ceph-mon[35941]: 98 slow requests, 5 included below; oldest blocked for > 395.667100 secs 2024-03-28T00:42:16.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:42:15 smithi132 ceph-mon[35941]: slow request 391.568108 seconds old, received at 2024-03-28T00:35:43.457937+0000: internal op fragmentdir:mds.0:3804 currently failed to wrlock, waiting 2024-03-28T00:42:16.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:42:15 smithi132 ceph-mon[35941]: slow request 391.538659 seconds old, received at 2024-03-28T00:35:43.487385+0000: internal op fragmentdir:mds.0:3867 currently failed to wrlock, waiting 2024-03-28T00:42:16.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:42:15 smithi132 ceph-mon[35941]: slow request 391.524274 seconds old, received at 2024-03-28T00:35:43.501771+0000: internal op fragmentdir:mds.0:3899 currently failed to wrlock, waiting 2024-03-28T00:42:16.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:42:15 smithi132 ceph-mon[35941]: slow request 391.567619 seconds old, received at 2024-03-28T00:35:43.458426+0000: internal op fragmentdir:mds.0:3805 currently failed to wrlock, waiting 2024-03-28T00:42:16.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:42:15 smithi132 ceph-mon[35941]: slow request 391.552142 seconds old, received at 2024-03-28T00:35:43.473902+0000: internal op fragmentdir:mds.0:3836 currently failed to wrlock, waiting 2024-03-28T00:42:16.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:42:15 smithi165 ceph-mon[36302]: 98 slow requests, 5 included below; oldest blocked for > 395.667100 secs 2024-03-28T00:42:16.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:42:15 smithi165 ceph-mon[36302]: slow request 391.568108 seconds old, received at 2024-03-28T00:35:43.457937+0000: internal op fragmentdir:mds.0:3804 currently failed to wrlock, waiting 2024-03-28T00:42:16.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:42:15 smithi165 ceph-mon[36302]: slow request 391.538659 seconds old, received at 2024-03-28T00:35:43.487385+0000: internal op fragmentdir:mds.0:3867 currently failed to wrlock, waiting 2024-03-28T00:42:16.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:42:15 smithi165 ceph-mon[36302]: slow request 391.524274 seconds old, received at 2024-03-28T00:35:43.501771+0000: internal op fragmentdir:mds.0:3899 currently failed to wrlock, waiting 2024-03-28T00:42:16.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:42:15 smithi165 ceph-mon[36302]: slow request 391.567619 seconds old, received at 2024-03-28T00:35:43.458426+0000: internal op fragmentdir:mds.0:3805 currently failed to wrlock, waiting 2024-03-28T00:42:16.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:42:15 smithi165 ceph-mon[36302]: slow request 391.552142 seconds old, received at 2024-03-28T00:35:43.473902+0000: internal op fragmentdir:mds.0:3836 currently failed to wrlock, waiting 2024-03-28T00:42:16.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:42:15 smithi119 ceph-mon[31518]: 98 slow requests, 5 included below; oldest blocked for > 395.667100 secs 2024-03-28T00:42:16.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:42:15 smithi119 ceph-mon[31518]: slow request 391.568108 seconds old, received at 2024-03-28T00:35:43.457937+0000: internal op fragmentdir:mds.0:3804 currently failed to wrlock, waiting 2024-03-28T00:42:16.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:42:15 smithi119 ceph-mon[31518]: slow request 391.538659 seconds old, received at 2024-03-28T00:35:43.487385+0000: internal op fragmentdir:mds.0:3867 currently failed to wrlock, waiting 2024-03-28T00:42:16.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:42:15 smithi119 ceph-mon[31518]: slow request 391.524274 seconds old, received at 2024-03-28T00:35:43.501771+0000: internal op fragmentdir:mds.0:3899 currently failed to wrlock, waiting 2024-03-28T00:42:16.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:42:15 smithi119 ceph-mon[31518]: slow request 391.567619 seconds old, received at 2024-03-28T00:35:43.458426+0000: internal op fragmentdir:mds.0:3805 currently failed to wrlock, waiting 2024-03-28T00:42:16.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:42:15 smithi119 ceph-mon[31518]: slow request 391.552142 seconds old, received at 2024-03-28T00:35:43.473902+0000: internal op fragmentdir:mds.0:3836 currently failed to wrlock, waiting 2024-03-28T00:42:17.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:42:16 smithi132 ceph-mon[35941]: pgmap v517: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:42:17.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:42:16 smithi165 ceph-mon[36302]: pgmap v517: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:42:17.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:42:16 smithi119 ceph-mon[31518]: pgmap v517: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:42:19.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:42:18 smithi132 ceph-mon[35941]: pgmap v518: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:42:19.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:42:18 smithi165 ceph-mon[36302]: pgmap v518: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:42:19.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:42:18 smithi119 ceph-mon[31518]: pgmap v518: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:42:20.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:42:19 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:42:20.306 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:42:19 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:42:20.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:42:19 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:42:21.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:42:20 smithi132 ceph-mon[35941]: 98 slow requests, 5 included below; oldest blocked for > 400.667187 secs 2024-03-28T00:42:21.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:42:20 smithi132 ceph-mon[35941]: slow request 396.538350 seconds old, received at 2024-03-28T00:35:43.487781+0000: internal op fragmentdir:mds.0:3868 currently failed to wrlock, waiting 2024-03-28T00:42:21.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:42:20 smithi132 ceph-mon[35941]: slow request 396.567219 seconds old, received at 2024-03-28T00:35:43.458913+0000: internal op fragmentdir:mds.0:3806 currently failed to wrlock, waiting 2024-03-28T00:42:21.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:42:20 smithi132 ceph-mon[35941]: slow request 396.551793 seconds old, received at 2024-03-28T00:35:43.474338+0000: internal op fragmentdir:mds.0:3837 currently failed to wrlock, waiting 2024-03-28T00:42:21.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:42:20 smithi132 ceph-mon[35941]: slow request 396.523477 seconds old, received at 2024-03-28T00:35:43.502655+0000: internal op fragmentdir:mds.0:3901 currently failed to wrlock, waiting 2024-03-28T00:42:21.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:42:20 smithi132 ceph-mon[35941]: slow request 396.567864 seconds old, received at 2024-03-28T00:35:43.458267+0000: client_request(client.25264:34113 create owner_uid=1000, owner_gid=1301 #0x10000000272/file7594 2024-03-28T00:35:43.457305+0000 caller_uid=1000, caller_gid=1301{6,36,1000,1301,}) currently failed to xlock, waiting 2024-03-28T00:42:21.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:42:20 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:42:21.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:42:20 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:42:21.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:42:20 smithi132 ceph-mon[35941]: pgmap v519: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T00:42:21.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:42:20 smithi132 ceph-mon[35941]: pgmap v520: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T00:42:21.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:42:20 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:42:21.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:42:20 smithi165 ceph-mon[36302]: 98 slow requests, 5 included below; oldest blocked for > 400.667187 secs 2024-03-28T00:42:21.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:42:20 smithi165 ceph-mon[36302]: slow request 396.538350 seconds old, received at 2024-03-28T00:35:43.487781+0000: internal op fragmentdir:mds.0:3868 currently failed to wrlock, waiting 2024-03-28T00:42:21.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:42:20 smithi165 ceph-mon[36302]: slow request 396.567219 seconds old, received at 2024-03-28T00:35:43.458913+0000: internal op fragmentdir:mds.0:3806 currently failed to wrlock, waiting 2024-03-28T00:42:21.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:42:20 smithi165 ceph-mon[36302]: slow request 396.551793 seconds old, received at 2024-03-28T00:35:43.474338+0000: internal op fragmentdir:mds.0:3837 currently failed to wrlock, waiting 2024-03-28T00:42:21.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:42:20 smithi165 ceph-mon[36302]: slow request 396.523477 seconds old, received at 2024-03-28T00:35:43.502655+0000: internal op fragmentdir:mds.0:3901 currently failed to wrlock, waiting 2024-03-28T00:42:21.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:42:20 smithi165 ceph-mon[36302]: slow request 396.567864 seconds old, received at 2024-03-28T00:35:43.458267+0000: client_request(client.25264:34113 create owner_uid=1000, owner_gid=1301 #0x10000000272/file7594 2024-03-28T00:35:43.457305+0000 caller_uid=1000, caller_gid=1301{6,36,1000,1301,}) currently failed to xlock, waiting 2024-03-28T00:42:21.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:42:20 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:42:21.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:42:20 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:42:21.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:42:20 smithi165 ceph-mon[36302]: pgmap v519: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T00:42:21.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:42:20 smithi165 ceph-mon[36302]: pgmap v520: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T00:42:21.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:42:20 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:42:21.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:42:20 smithi119 ceph-mon[31518]: 98 slow requests, 5 included below; oldest blocked for > 400.667187 secs 2024-03-28T00:42:21.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:42:20 smithi119 ceph-mon[31518]: slow request 396.538350 seconds old, received at 2024-03-28T00:35:43.487781+0000: internal op fragmentdir:mds.0:3868 currently failed to wrlock, waiting 2024-03-28T00:42:21.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:42:20 smithi119 ceph-mon[31518]: slow request 396.567219 seconds old, received at 2024-03-28T00:35:43.458913+0000: internal op fragmentdir:mds.0:3806 currently failed to wrlock, waiting 2024-03-28T00:42:21.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:42:20 smithi119 ceph-mon[31518]: slow request 396.551793 seconds old, received at 2024-03-28T00:35:43.474338+0000: internal op fragmentdir:mds.0:3837 currently failed to wrlock, waiting 2024-03-28T00:42:21.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:42:20 smithi119 ceph-mon[31518]: slow request 396.523477 seconds old, received at 2024-03-28T00:35:43.502655+0000: internal op fragmentdir:mds.0:3901 currently failed to wrlock, waiting 2024-03-28T00:42:21.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:42:20 smithi119 ceph-mon[31518]: slow request 396.567864 seconds old, received at 2024-03-28T00:35:43.458267+0000: client_request(client.25264:34113 create owner_uid=1000, owner_gid=1301 #0x10000000272/file7594 2024-03-28T00:35:43.457305+0000 caller_uid=1000, caller_gid=1301{6,36,1000,1301,}) currently failed to xlock, waiting 2024-03-28T00:42:21.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:42:20 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:42:21.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:42:20 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:42:21.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:42:20 smithi119 ceph-mon[31518]: pgmap v519: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T00:42:21.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:42:20 smithi119 ceph-mon[31518]: pgmap v520: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T00:42:21.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:42:20 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:42:23.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:42:23 smithi132 ceph-mon[35941]: pgmap v521: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T00:42:23.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:42:23 smithi165 ceph-mon[36302]: pgmap v521: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T00:42:23.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:42:23 smithi119 ceph-mon[31518]: pgmap v521: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T00:42:25.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:42:25 smithi132 ceph-mon[35941]: pgmap v522: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T00:42:25.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:42:25 smithi132 ceph-mon[35941]: 98 slow requests, 5 included below; oldest blocked for > 405.667228 secs 2024-03-28T00:42:25.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:42:25 smithi132 ceph-mon[35941]: slow request 401.537867 seconds old, received at 2024-03-28T00:35:43.488305+0000: internal op fragmentdir:mds.0:3870 currently failed to wrlock, waiting 2024-03-28T00:42:25.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:42:25 smithi132 ceph-mon[35941]: slow request 401.564325 seconds old, received at 2024-03-28T00:35:43.461848+0000: internal op fragmentdir:mds.0:3807 currently failed to wrlock, waiting 2024-03-28T00:42:25.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:42:25 smithi132 ceph-mon[35941]: slow request 401.550977 seconds old, received at 2024-03-28T00:35:43.475195+0000: internal op fragmentdir:mds.0:3839 currently failed to wrlock, waiting 2024-03-28T00:42:25.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:42:25 smithi132 ceph-mon[35941]: slow request 401.537448 seconds old, received at 2024-03-28T00:35:43.488725+0000: internal op fragmentdir:mds.0:3871 currently failed to wrlock, waiting 2024-03-28T00:42:25.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:42:25 smithi132 ceph-mon[35941]: slow request 401.522865 seconds old, received at 2024-03-28T00:35:43.503308+0000: internal op fragmentdir:mds.0:3903 currently failed to wrlock, waiting 2024-03-28T00:42:25.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:42:25 smithi165 ceph-mon[36302]: pgmap v522: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T00:42:25.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:42:25 smithi165 ceph-mon[36302]: 98 slow requests, 5 included below; oldest blocked for > 405.667228 secs 2024-03-28T00:42:25.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:42:25 smithi165 ceph-mon[36302]: slow request 401.537867 seconds old, received at 2024-03-28T00:35:43.488305+0000: internal op fragmentdir:mds.0:3870 currently failed to wrlock, waiting 2024-03-28T00:42:25.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:42:25 smithi165 ceph-mon[36302]: slow request 401.564325 seconds old, received at 2024-03-28T00:35:43.461848+0000: internal op fragmentdir:mds.0:3807 currently failed to wrlock, waiting 2024-03-28T00:42:25.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:42:25 smithi165 ceph-mon[36302]: slow request 401.550977 seconds old, received at 2024-03-28T00:35:43.475195+0000: internal op fragmentdir:mds.0:3839 currently failed to wrlock, waiting 2024-03-28T00:42:25.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:42:25 smithi165 ceph-mon[36302]: slow request 401.537448 seconds old, received at 2024-03-28T00:35:43.488725+0000: internal op fragmentdir:mds.0:3871 currently failed to wrlock, waiting 2024-03-28T00:42:25.883 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:42:25 smithi165 ceph-mon[36302]: slow request 401.522865 seconds old, received at 2024-03-28T00:35:43.503308+0000: internal op fragmentdir:mds.0:3903 currently failed to wrlock, waiting 2024-03-28T00:42:25.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:42:25 smithi119 ceph-mon[31518]: pgmap v522: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T00:42:25.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:42:25 smithi119 ceph-mon[31518]: 98 slow requests, 5 included below; oldest blocked for > 405.667228 secs 2024-03-28T00:42:25.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:42:25 smithi119 ceph-mon[31518]: slow request 401.537867 seconds old, received at 2024-03-28T00:35:43.488305+0000: internal op fragmentdir:mds.0:3870 currently failed to wrlock, waiting 2024-03-28T00:42:25.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:42:25 smithi119 ceph-mon[31518]: slow request 401.564325 seconds old, received at 2024-03-28T00:35:43.461848+0000: internal op fragmentdir:mds.0:3807 currently failed to wrlock, waiting 2024-03-28T00:42:25.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:42:25 smithi119 ceph-mon[31518]: slow request 401.550977 seconds old, received at 2024-03-28T00:35:43.475195+0000: internal op fragmentdir:mds.0:3839 currently failed to wrlock, waiting 2024-03-28T00:42:25.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:42:25 smithi119 ceph-mon[31518]: slow request 401.537448 seconds old, received at 2024-03-28T00:35:43.488725+0000: internal op fragmentdir:mds.0:3871 currently failed to wrlock, waiting 2024-03-28T00:42:25.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:42:25 smithi119 ceph-mon[31518]: slow request 401.522865 seconds old, received at 2024-03-28T00:35:43.503308+0000: internal op fragmentdir:mds.0:3903 currently failed to wrlock, waiting 2024-03-28T00:42:27.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:42:27 smithi132 ceph-mon[35941]: pgmap v523: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T00:42:27.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:42:27 smithi165 ceph-mon[36302]: pgmap v523: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T00:42:27.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:42:27 smithi119 ceph-mon[31518]: pgmap v523: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T00:42:29.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:42:29 smithi119 ceph-mon[31518]: pgmap v524: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T00:42:29.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:42:29 smithi132 ceph-mon[35941]: pgmap v524: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T00:42:29.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:42:29 smithi165 ceph-mon[36302]: pgmap v524: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T00:42:30.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:42:30 smithi132 ceph-mon[35941]: 98 slow requests, 5 included below; oldest blocked for > 410.667316 secs 2024-03-28T00:42:30.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:42:30 smithi132 ceph-mon[35941]: slow request 389.649429 seconds old, received at 2024-03-28T00:36:00.376832+0000: client_request(client.25355:96 mksnap #0x1//scheduled-2024-03-28-00_36_00_UTC 2024-03-28T00:36:00.376777+0000 caller_uid=0, caller_gid=0{}) currently failed to acquire_locks 2024-03-28T00:42:30.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:42:30 smithi132 ceph-mon[35941]: slow request 406.563959 seconds old, received at 2024-03-28T00:35:43.462302+0000: internal op fragmentdir:mds.0:3808 currently failed to wrlock, waiting 2024-03-28T00:42:30.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:42:30 smithi132 ceph-mon[35941]: slow request 406.627566 seconds old, received at 2024-03-28T00:35:43.398696+0000: internal op fragmentdir:mds.0:3785 currently failed to wrlock, waiting 2024-03-28T00:42:30.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:42:30 smithi132 ceph-mon[35941]: slow request 406.563497 seconds old, received at 2024-03-28T00:35:43.462764+0000: internal op fragmentdir:mds.0:3809 currently failed to wrlock, waiting 2024-03-28T00:42:30.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:42:30 smithi132 ceph-mon[35941]: slow request 406.550167 seconds old, received at 2024-03-28T00:35:43.476094+0000: internal op fragmentdir:mds.0:3841 currently failed to wrlock, waiting 2024-03-28T00:42:30.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:42:30 smithi165 ceph-mon[36302]: 98 slow requests, 5 included below; oldest blocked for > 410.667316 secs 2024-03-28T00:42:30.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:42:30 smithi165 ceph-mon[36302]: slow request 389.649429 seconds old, received at 2024-03-28T00:36:00.376832+0000: client_request(client.25355:96 mksnap #0x1//scheduled-2024-03-28-00_36_00_UTC 2024-03-28T00:36:00.376777+0000 caller_uid=0, caller_gid=0{}) currently failed to acquire_locks 2024-03-28T00:42:30.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:42:30 smithi165 ceph-mon[36302]: slow request 406.563959 seconds old, received at 2024-03-28T00:35:43.462302+0000: internal op fragmentdir:mds.0:3808 currently failed to wrlock, waiting 2024-03-28T00:42:30.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:42:30 smithi165 ceph-mon[36302]: slow request 406.627566 seconds old, received at 2024-03-28T00:35:43.398696+0000: internal op fragmentdir:mds.0:3785 currently failed to wrlock, waiting 2024-03-28T00:42:30.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:42:30 smithi165 ceph-mon[36302]: slow request 406.563497 seconds old, received at 2024-03-28T00:35:43.462764+0000: internal op fragmentdir:mds.0:3809 currently failed to wrlock, waiting 2024-03-28T00:42:30.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:42:30 smithi165 ceph-mon[36302]: slow request 406.550167 seconds old, received at 2024-03-28T00:35:43.476094+0000: internal op fragmentdir:mds.0:3841 currently failed to wrlock, waiting 2024-03-28T00:42:30.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:42:30 smithi119 ceph-mon[31518]: 98 slow requests, 5 included below; oldest blocked for > 410.667316 secs 2024-03-28T00:42:30.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:42:30 smithi119 ceph-mon[31518]: slow request 389.649429 seconds old, received at 2024-03-28T00:36:00.376832+0000: client_request(client.25355:96 mksnap #0x1//scheduled-2024-03-28-00_36_00_UTC 2024-03-28T00:36:00.376777+0000 caller_uid=0, caller_gid=0{}) currently failed to acquire_locks 2024-03-28T00:42:30.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:42:30 smithi119 ceph-mon[31518]: slow request 406.563959 seconds old, received at 2024-03-28T00:35:43.462302+0000: internal op fragmentdir:mds.0:3808 currently failed to wrlock, waiting 2024-03-28T00:42:30.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:42:30 smithi119 ceph-mon[31518]: slow request 406.627566 seconds old, received at 2024-03-28T00:35:43.398696+0000: internal op fragmentdir:mds.0:3785 currently failed to wrlock, waiting 2024-03-28T00:42:30.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:42:30 smithi119 ceph-mon[31518]: slow request 406.563497 seconds old, received at 2024-03-28T00:35:43.462764+0000: internal op fragmentdir:mds.0:3809 currently failed to wrlock, waiting 2024-03-28T00:42:30.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:42:30 smithi119 ceph-mon[31518]: slow request 406.550167 seconds old, received at 2024-03-28T00:35:43.476094+0000: internal op fragmentdir:mds.0:3841 currently failed to wrlock, waiting 2024-03-28T00:42:31.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:42:31 smithi132 ceph-mon[35941]: pgmap v525: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T00:42:31.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:42:31 smithi165 ceph-mon[36302]: pgmap v525: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T00:42:31.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:42:31 smithi119 ceph-mon[31518]: pgmap v525: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T00:42:33.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:42:33 smithi132 ceph-mon[35941]: pgmap v526: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:42:33.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:42:33 smithi165 ceph-mon[36302]: pgmap v526: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:42:33.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:42:33 smithi119 ceph-mon[31518]: pgmap v526: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:42:35.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:42:35 smithi132 ceph-mon[35941]: pgmap v527: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:42:35.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:42:35 smithi132 ceph-mon[35941]: 98 slow requests, 5 included below; oldest blocked for > 415.667376 secs 2024-03-28T00:42:35.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:42:35 smithi132 ceph-mon[35941]: slow request 411.536670 seconds old, received at 2024-03-28T00:35:43.489651+0000: internal op fragmentdir:mds.0:3873 currently failed to wrlock, waiting 2024-03-28T00:42:35.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:42:35 smithi132 ceph-mon[35941]: slow request 410.698700 seconds old, received at 2024-03-28T00:35:44.327621+0000: internal op fragmentdir:mds.0:3937 currently failed to wrlock, waiting 2024-03-28T00:42:35.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:42:35 smithi132 ceph-mon[35941]: slow request 411.563121 seconds old, received at 2024-03-28T00:35:43.463200+0000: internal op fragmentdir:mds.0:3810 currently failed to wrlock, waiting 2024-03-28T00:42:35.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:42:35 smithi132 ceph-mon[35941]: slow request 411.549790 seconds old, received at 2024-03-28T00:35:43.476531+0000: internal op fragmentdir:mds.0:3842 currently failed to wrlock, waiting 2024-03-28T00:42:35.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:42:35 smithi132 ceph-mon[35941]: slow request 411.562720 seconds old, received at 2024-03-28T00:35:43.463601+0000: internal op fragmentdir:mds.0:3811 currently failed to wrlock, waiting 2024-03-28T00:42:35.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:42:35 smithi165 ceph-mon[36302]: pgmap v527: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:42:35.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:42:35 smithi165 ceph-mon[36302]: 98 slow requests, 5 included below; oldest blocked for > 415.667376 secs 2024-03-28T00:42:35.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:42:35 smithi165 ceph-mon[36302]: slow request 411.536670 seconds old, received at 2024-03-28T00:35:43.489651+0000: internal op fragmentdir:mds.0:3873 currently failed to wrlock, waiting 2024-03-28T00:42:35.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:42:35 smithi165 ceph-mon[36302]: slow request 410.698700 seconds old, received at 2024-03-28T00:35:44.327621+0000: internal op fragmentdir:mds.0:3937 currently failed to wrlock, waiting 2024-03-28T00:42:35.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:42:35 smithi165 ceph-mon[36302]: slow request 411.563121 seconds old, received at 2024-03-28T00:35:43.463200+0000: internal op fragmentdir:mds.0:3810 currently failed to wrlock, waiting 2024-03-28T00:42:35.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:42:35 smithi165 ceph-mon[36302]: slow request 411.549790 seconds old, received at 2024-03-28T00:35:43.476531+0000: internal op fragmentdir:mds.0:3842 currently failed to wrlock, waiting 2024-03-28T00:42:35.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:42:35 smithi165 ceph-mon[36302]: slow request 411.562720 seconds old, received at 2024-03-28T00:35:43.463601+0000: internal op fragmentdir:mds.0:3811 currently failed to wrlock, waiting 2024-03-28T00:42:35.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:42:35 smithi119 ceph-mon[31518]: pgmap v527: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:42:35.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:42:35 smithi119 ceph-mon[31518]: 98 slow requests, 5 included below; oldest blocked for > 415.667376 secs 2024-03-28T00:42:35.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:42:35 smithi119 ceph-mon[31518]: slow request 411.536670 seconds old, received at 2024-03-28T00:35:43.489651+0000: internal op fragmentdir:mds.0:3873 currently failed to wrlock, waiting 2024-03-28T00:42:35.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:42:35 smithi119 ceph-mon[31518]: slow request 410.698700 seconds old, received at 2024-03-28T00:35:44.327621+0000: internal op fragmentdir:mds.0:3937 currently failed to wrlock, waiting 2024-03-28T00:42:35.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:42:35 smithi119 ceph-mon[31518]: slow request 411.563121 seconds old, received at 2024-03-28T00:35:43.463200+0000: internal op fragmentdir:mds.0:3810 currently failed to wrlock, waiting 2024-03-28T00:42:35.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:42:35 smithi119 ceph-mon[31518]: slow request 411.549790 seconds old, received at 2024-03-28T00:35:43.476531+0000: internal op fragmentdir:mds.0:3842 currently failed to wrlock, waiting 2024-03-28T00:42:35.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:42:35 smithi119 ceph-mon[31518]: slow request 411.562720 seconds old, received at 2024-03-28T00:35:43.463601+0000: internal op fragmentdir:mds.0:3811 currently failed to wrlock, waiting 2024-03-28T00:42:37.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:42:37 smithi132 ceph-mon[35941]: pgmap v528: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:42:37.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:42:37 smithi165 ceph-mon[36302]: pgmap v528: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:42:37.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:42:37 smithi119 ceph-mon[31518]: pgmap v528: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:42:39.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:42:39 smithi165 ceph-mon[36302]: pgmap v529: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:42:39.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:42:39 smithi119 ceph-mon[31518]: pgmap v529: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:42:39.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:42:39 smithi132 ceph-mon[35941]: pgmap v529: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:42:40.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:42:40 smithi165 ceph-mon[36302]: 98 slow requests, 5 included below; oldest blocked for > 420.667435 secs 2024-03-28T00:42:40.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:42:40 smithi165 ceph-mon[36302]: slow request 416.520342 seconds old, received at 2024-03-28T00:35:43.506038+0000: internal op fragmentdir:mds.0:3907 currently failed to wrlock, waiting 2024-03-28T00:42:40.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:42:40 smithi165 ceph-mon[36302]: slow request 416.562344 seconds old, received at 2024-03-28T00:35:43.464036+0000: internal op fragmentdir:mds.0:3812 currently failed to wrlock, waiting 2024-03-28T00:42:40.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:42:40 smithi165 ceph-mon[36302]: slow request 416.548965 seconds old, received at 2024-03-28T00:35:43.477414+0000: internal op fragmentdir:mds.0:3844 currently failed to wrlock, waiting 2024-03-28T00:42:40.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:42:40 smithi165 ceph-mon[36302]: slow request 416.518614 seconds old, received at 2024-03-28T00:35:43.507766+0000: internal op fragmentdir:mds.0:3908 currently failed to wrlock, waiting 2024-03-28T00:42:40.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:42:40 smithi165 ceph-mon[36302]: slow request 417.887014 seconds old, received at 2024-03-28T00:35:42.139365+0000: internal op fragmentdir:mds.0:3784 currently failed to wrlock, waiting 2024-03-28T00:42:40.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:42:40 smithi119 ceph-mon[31518]: 98 slow requests, 5 included below; oldest blocked for > 420.667435 secs 2024-03-28T00:42:40.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:42:40 smithi119 ceph-mon[31518]: slow request 416.520342 seconds old, received at 2024-03-28T00:35:43.506038+0000: internal op fragmentdir:mds.0:3907 currently failed to wrlock, waiting 2024-03-28T00:42:40.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:42:40 smithi119 ceph-mon[31518]: slow request 416.562344 seconds old, received at 2024-03-28T00:35:43.464036+0000: internal op fragmentdir:mds.0:3812 currently failed to wrlock, waiting 2024-03-28T00:42:40.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:42:40 smithi119 ceph-mon[31518]: slow request 416.548965 seconds old, received at 2024-03-28T00:35:43.477414+0000: internal op fragmentdir:mds.0:3844 currently failed to wrlock, waiting 2024-03-28T00:42:40.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:42:40 smithi119 ceph-mon[31518]: slow request 416.518614 seconds old, received at 2024-03-28T00:35:43.507766+0000: internal op fragmentdir:mds.0:3908 currently failed to wrlock, waiting 2024-03-28T00:42:40.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:42:40 smithi119 ceph-mon[31518]: slow request 417.887014 seconds old, received at 2024-03-28T00:35:42.139365+0000: internal op fragmentdir:mds.0:3784 currently failed to wrlock, waiting 2024-03-28T00:42:40.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:42:40 smithi132 ceph-mon[35941]: 98 slow requests, 5 included below; oldest blocked for > 420.667435 secs 2024-03-28T00:42:40.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:42:40 smithi132 ceph-mon[35941]: slow request 416.520342 seconds old, received at 2024-03-28T00:35:43.506038+0000: internal op fragmentdir:mds.0:3907 currently failed to wrlock, waiting 2024-03-28T00:42:40.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:42:40 smithi132 ceph-mon[35941]: slow request 416.562344 seconds old, received at 2024-03-28T00:35:43.464036+0000: internal op fragmentdir:mds.0:3812 currently failed to wrlock, waiting 2024-03-28T00:42:40.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:42:40 smithi132 ceph-mon[35941]: slow request 416.548965 seconds old, received at 2024-03-28T00:35:43.477414+0000: internal op fragmentdir:mds.0:3844 currently failed to wrlock, waiting 2024-03-28T00:42:40.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:42:40 smithi132 ceph-mon[35941]: slow request 416.518614 seconds old, received at 2024-03-28T00:35:43.507766+0000: internal op fragmentdir:mds.0:3908 currently failed to wrlock, waiting 2024-03-28T00:42:40.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:42:40 smithi132 ceph-mon[35941]: slow request 417.887014 seconds old, received at 2024-03-28T00:35:42.139365+0000: internal op fragmentdir:mds.0:3784 currently failed to wrlock, waiting 2024-03-28T00:42:41.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:42:41 smithi165 ceph-mon[36302]: pgmap v530: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:42:41.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:42:41 smithi119 ceph-mon[31518]: pgmap v530: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:42:41.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:42:41 smithi132 ceph-mon[35941]: pgmap v530: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:42:43.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:42:43 smithi165 ceph-mon[36302]: pgmap v531: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:42:43.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:42:43 smithi119 ceph-mon[31518]: pgmap v531: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:42:43.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:42:43 smithi132 ceph-mon[35941]: pgmap v531: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:42:45.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:42:45 smithi165 ceph-mon[36302]: pgmap v532: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:42:45.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:42:45 smithi165 ceph-mon[36302]: 98 slow requests, 5 included below; oldest blocked for > 425.667495 secs 2024-03-28T00:42:45.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:42:45 smithi165 ceph-mon[36302]: slow request 421.562008 seconds old, received at 2024-03-28T00:35:43.464432+0000: internal op fragmentdir:mds.0:3813 currently failed to wrlock, waiting 2024-03-28T00:42:45.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:42:45 smithi165 ceph-mon[36302]: slow request 421.548582 seconds old, received at 2024-03-28T00:35:43.477858+0000: internal op fragmentdir:mds.0:3845 currently failed to wrlock, waiting 2024-03-28T00:42:45.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:42:45 smithi165 ceph-mon[36302]: slow request 421.534893 seconds old, received at 2024-03-28T00:35:43.491547+0000: internal op fragmentdir:mds.0:3877 currently failed to wrlock, waiting 2024-03-28T00:42:45.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:42:45 smithi165 ceph-mon[36302]: slow request 421.561570 seconds old, received at 2024-03-28T00:35:43.464871+0000: internal op fragmentdir:mds.0:3814 currently failed to wrlock, waiting 2024-03-28T00:42:45.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:42:45 smithi165 ceph-mon[36302]: slow request 421.516616 seconds old, received at 2024-03-28T00:35:43.509824+0000: internal op fragmentdir:mds.0:3910 currently failed to wrlock, waiting 2024-03-28T00:42:45.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:42:45 smithi119 ceph-mon[31518]: pgmap v532: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:42:45.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:42:45 smithi119 ceph-mon[31518]: 98 slow requests, 5 included below; oldest blocked for > 425.667495 secs 2024-03-28T00:42:45.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:42:45 smithi119 ceph-mon[31518]: slow request 421.562008 seconds old, received at 2024-03-28T00:35:43.464432+0000: internal op fragmentdir:mds.0:3813 currently failed to wrlock, waiting 2024-03-28T00:42:45.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:42:45 smithi119 ceph-mon[31518]: slow request 421.548582 seconds old, received at 2024-03-28T00:35:43.477858+0000: internal op fragmentdir:mds.0:3845 currently failed to wrlock, waiting 2024-03-28T00:42:45.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:42:45 smithi119 ceph-mon[31518]: slow request 421.534893 seconds old, received at 2024-03-28T00:35:43.491547+0000: internal op fragmentdir:mds.0:3877 currently failed to wrlock, waiting 2024-03-28T00:42:45.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:42:45 smithi119 ceph-mon[31518]: slow request 421.561570 seconds old, received at 2024-03-28T00:35:43.464871+0000: internal op fragmentdir:mds.0:3814 currently failed to wrlock, waiting 2024-03-28T00:42:45.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:42:45 smithi119 ceph-mon[31518]: slow request 421.516616 seconds old, received at 2024-03-28T00:35:43.509824+0000: internal op fragmentdir:mds.0:3910 currently failed to wrlock, waiting 2024-03-28T00:42:45.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:42:45 smithi132 ceph-mon[35941]: pgmap v532: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:42:45.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:42:45 smithi132 ceph-mon[35941]: 98 slow requests, 5 included below; oldest blocked for > 425.667495 secs 2024-03-28T00:42:45.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:42:45 smithi132 ceph-mon[35941]: slow request 421.562008 seconds old, received at 2024-03-28T00:35:43.464432+0000: internal op fragmentdir:mds.0:3813 currently failed to wrlock, waiting 2024-03-28T00:42:45.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:42:45 smithi132 ceph-mon[35941]: slow request 421.548582 seconds old, received at 2024-03-28T00:35:43.477858+0000: internal op fragmentdir:mds.0:3845 currently failed to wrlock, waiting 2024-03-28T00:42:45.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:42:45 smithi132 ceph-mon[35941]: slow request 421.534893 seconds old, received at 2024-03-28T00:35:43.491547+0000: internal op fragmentdir:mds.0:3877 currently failed to wrlock, waiting 2024-03-28T00:42:45.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:42:45 smithi132 ceph-mon[35941]: slow request 421.561570 seconds old, received at 2024-03-28T00:35:43.464871+0000: internal op fragmentdir:mds.0:3814 currently failed to wrlock, waiting 2024-03-28T00:42:45.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:42:45 smithi132 ceph-mon[35941]: slow request 421.516616 seconds old, received at 2024-03-28T00:35:43.509824+0000: internal op fragmentdir:mds.0:3910 currently failed to wrlock, waiting 2024-03-28T00:42:47.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:42:47 smithi165 ceph-mon[36302]: pgmap v533: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:42:47.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:42:47 smithi119 ceph-mon[31518]: pgmap v533: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:42:47.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:42:47 smithi132 ceph-mon[35941]: pgmap v533: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:42:49.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:42:49 smithi165 ceph-mon[36302]: pgmap v534: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:42:49.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:42:49 smithi119 ceph-mon[31518]: pgmap v534: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:42:49.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:42:49 smithi132 ceph-mon[35941]: pgmap v534: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:42:50.806 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:42:50 smithi119 ceph-mon[31518]: 98 slow requests, 5 included below; oldest blocked for > 430.667556 secs 2024-03-28T00:42:50.806 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:42:50 smithi119 ceph-mon[31518]: slow request 426.601575 seconds old, received at 2024-03-28T00:35:43.424926+0000: internal op fragmentdir:mds.0:3787 currently failed to wrlock, waiting 2024-03-28T00:42:50.807 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:42:50 smithi119 ceph-mon[31518]: slow request 426.561191 seconds old, received at 2024-03-28T00:35:43.465310+0000: internal op fragmentdir:mds.0:3815 currently failed to wrlock, waiting 2024-03-28T00:42:50.807 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:42:50 smithi119 ceph-mon[31518]: slow request 426.534025 seconds old, received at 2024-03-28T00:35:43.492476+0000: internal op fragmentdir:mds.0:3879 currently failed to wrlock, waiting 2024-03-28T00:42:50.807 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:42:50 smithi119 ceph-mon[31518]: slow request 426.560797 seconds old, received at 2024-03-28T00:35:43.465705+0000: internal op fragmentdir:mds.0:3816 currently failed to wrlock, waiting 2024-03-28T00:42:50.807 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:42:50 smithi119 ceph-mon[31518]: slow request 426.547235 seconds old, received at 2024-03-28T00:35:43.479266+0000: internal op fragmentdir:mds.0:3848 currently failed to wrlock, waiting 2024-03-28T00:42:50.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:42:50 smithi165 ceph-mon[36302]: 98 slow requests, 5 included below; oldest blocked for > 430.667556 secs 2024-03-28T00:42:50.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:42:50 smithi165 ceph-mon[36302]: slow request 426.601575 seconds old, received at 2024-03-28T00:35:43.424926+0000: internal op fragmentdir:mds.0:3787 currently failed to wrlock, waiting 2024-03-28T00:42:50.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:42:50 smithi165 ceph-mon[36302]: slow request 426.561191 seconds old, received at 2024-03-28T00:35:43.465310+0000: internal op fragmentdir:mds.0:3815 currently failed to wrlock, waiting 2024-03-28T00:42:50.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:42:50 smithi165 ceph-mon[36302]: slow request 426.534025 seconds old, received at 2024-03-28T00:35:43.492476+0000: internal op fragmentdir:mds.0:3879 currently failed to wrlock, waiting 2024-03-28T00:42:50.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:42:50 smithi165 ceph-mon[36302]: slow request 426.560797 seconds old, received at 2024-03-28T00:35:43.465705+0000: internal op fragmentdir:mds.0:3816 currently failed to wrlock, waiting 2024-03-28T00:42:50.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:42:50 smithi165 ceph-mon[36302]: slow request 426.547235 seconds old, received at 2024-03-28T00:35:43.479266+0000: internal op fragmentdir:mds.0:3848 currently failed to wrlock, waiting 2024-03-28T00:42:50.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:42:50 smithi132 ceph-mon[35941]: 98 slow requests, 5 included below; oldest blocked for > 430.667556 secs 2024-03-28T00:42:50.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:42:50 smithi132 ceph-mon[35941]: slow request 426.601575 seconds old, received at 2024-03-28T00:35:43.424926+0000: internal op fragmentdir:mds.0:3787 currently failed to wrlock, waiting 2024-03-28T00:42:50.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:42:50 smithi132 ceph-mon[35941]: slow request 426.561191 seconds old, received at 2024-03-28T00:35:43.465310+0000: internal op fragmentdir:mds.0:3815 currently failed to wrlock, waiting 2024-03-28T00:42:50.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:42:50 smithi132 ceph-mon[35941]: slow request 426.534025 seconds old, received at 2024-03-28T00:35:43.492476+0000: internal op fragmentdir:mds.0:3879 currently failed to wrlock, waiting 2024-03-28T00:42:50.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:42:50 smithi132 ceph-mon[35941]: slow request 426.560797 seconds old, received at 2024-03-28T00:35:43.465705+0000: internal op fragmentdir:mds.0:3816 currently failed to wrlock, waiting 2024-03-28T00:42:50.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:42:50 smithi132 ceph-mon[35941]: slow request 426.547235 seconds old, received at 2024-03-28T00:35:43.479266+0000: internal op fragmentdir:mds.0:3848 currently failed to wrlock, waiting 2024-03-28T00:42:51.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:42:51 smithi165 ceph-mon[36302]: pgmap v535: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:42:51.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:42:51 smithi119 ceph-mon[31518]: pgmap v535: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:42:51.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:42:51 smithi132 ceph-mon[35941]: pgmap v535: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:42:53.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:42:53 smithi165 ceph-mon[36302]: pgmap v536: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:42:53.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:42:53 smithi119 ceph-mon[31518]: pgmap v536: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:42:53.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:42:53 smithi132 ceph-mon[35941]: pgmap v536: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:42:55.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:42:55 smithi165 ceph-mon[36302]: pgmap v537: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:42:55.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:42:55 smithi165 ceph-mon[36302]: 98 slow requests, 5 included below; oldest blocked for > 435.667616 secs 2024-03-28T00:42:55.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:42:55 smithi165 ceph-mon[36302]: slow request 431.533599 seconds old, received at 2024-03-28T00:35:43.492962+0000: internal op fragmentdir:mds.0:3880 currently failed to wrlock, waiting 2024-03-28T00:42:55.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:42:55 smithi165 ceph-mon[36302]: slow request 432.878489 seconds old, received at 2024-03-28T00:35:42.148071+0000: client_request(client.25264:33954 create owner_uid=1000, owner_gid=1301 #0x10000000271/file8309 2024-03-28T00:35:42.146337+0000 caller_uid=1000, caller_gid=1301{6,36,1000,1301,}) currently failed to xlock, waiting 2024-03-28T00:42:55.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:42:55 smithi165 ceph-mon[36302]: slow request 431.560461 seconds old, received at 2024-03-28T00:35:43.466100+0000: internal op fragmentdir:mds.0:3817 currently failed to wrlock, waiting 2024-03-28T00:42:55.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:42:55 smithi165 ceph-mon[36302]: slow request 431.533271 seconds old, received at 2024-03-28T00:35:43.493290+0000: internal op fragmentdir:mds.0:3881 currently failed to wrlock, waiting 2024-03-28T00:42:55.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:42:55 smithi165 ceph-mon[36302]: slow request 431.560013 seconds old, received at 2024-03-28T00:35:43.466548+0000: internal op fragmentdir:mds.0:3818 currently failed to wrlock, waiting 2024-03-28T00:42:55.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:42:55 smithi119 ceph-mon[31518]: pgmap v537: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:42:55.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:42:55 smithi119 ceph-mon[31518]: 98 slow requests, 5 included below; oldest blocked for > 435.667616 secs 2024-03-28T00:42:55.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:42:55 smithi119 ceph-mon[31518]: slow request 431.533599 seconds old, received at 2024-03-28T00:35:43.492962+0000: internal op fragmentdir:mds.0:3880 currently failed to wrlock, waiting 2024-03-28T00:42:55.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:42:55 smithi119 ceph-mon[31518]: slow request 432.878489 seconds old, received at 2024-03-28T00:35:42.148071+0000: client_request(client.25264:33954 create owner_uid=1000, owner_gid=1301 #0x10000000271/file8309 2024-03-28T00:35:42.146337+0000 caller_uid=1000, caller_gid=1301{6,36,1000,1301,}) currently failed to xlock, waiting 2024-03-28T00:42:55.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:42:55 smithi119 ceph-mon[31518]: slow request 431.560461 seconds old, received at 2024-03-28T00:35:43.466100+0000: internal op fragmentdir:mds.0:3817 currently failed to wrlock, waiting 2024-03-28T00:42:55.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:42:55 smithi119 ceph-mon[31518]: slow request 431.533271 seconds old, received at 2024-03-28T00:35:43.493290+0000: internal op fragmentdir:mds.0:3881 currently failed to wrlock, waiting 2024-03-28T00:42:55.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:42:55 smithi119 ceph-mon[31518]: slow request 431.560013 seconds old, received at 2024-03-28T00:35:43.466548+0000: internal op fragmentdir:mds.0:3818 currently failed to wrlock, waiting 2024-03-28T00:42:55.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:42:55 smithi132 ceph-mon[35941]: pgmap v537: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:42:55.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:42:55 smithi132 ceph-mon[35941]: 98 slow requests, 5 included below; oldest blocked for > 435.667616 secs 2024-03-28T00:42:55.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:42:55 smithi132 ceph-mon[35941]: slow request 431.533599 seconds old, received at 2024-03-28T00:35:43.492962+0000: internal op fragmentdir:mds.0:3880 currently failed to wrlock, waiting 2024-03-28T00:42:55.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:42:55 smithi132 ceph-mon[35941]: slow request 432.878489 seconds old, received at 2024-03-28T00:35:42.148071+0000: client_request(client.25264:33954 create owner_uid=1000, owner_gid=1301 #0x10000000271/file8309 2024-03-28T00:35:42.146337+0000 caller_uid=1000, caller_gid=1301{6,36,1000,1301,}) currently failed to xlock, waiting 2024-03-28T00:42:55.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:42:55 smithi132 ceph-mon[35941]: slow request 431.560461 seconds old, received at 2024-03-28T00:35:43.466100+0000: internal op fragmentdir:mds.0:3817 currently failed to wrlock, waiting 2024-03-28T00:42:55.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:42:55 smithi132 ceph-mon[35941]: slow request 431.533271 seconds old, received at 2024-03-28T00:35:43.493290+0000: internal op fragmentdir:mds.0:3881 currently failed to wrlock, waiting 2024-03-28T00:42:55.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:42:55 smithi132 ceph-mon[35941]: slow request 431.560013 seconds old, received at 2024-03-28T00:35:43.466548+0000: internal op fragmentdir:mds.0:3818 currently failed to wrlock, waiting 2024-03-28T00:42:57.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:42:57 smithi165 ceph-mon[36302]: pgmap v538: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:42:57.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:42:57 smithi119 ceph-mon[31518]: pgmap v538: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:42:57.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:42:57 smithi132 ceph-mon[35941]: pgmap v538: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:42:59.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:42:59 smithi165 ceph-mon[36302]: pgmap v539: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:42:59.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:42:59 smithi119 ceph-mon[31518]: pgmap v539: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:42:59.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:42:59 smithi132 ceph-mon[35941]: pgmap v539: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:43:00.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:43:00 smithi165 ceph-mon[36302]: 98 slow requests, 5 included below; oldest blocked for > 440.667655 secs 2024-03-28T00:43:00.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:43:00 smithi165 ceph-mon[36302]: slow request 436.546407 seconds old, received at 2024-03-28T00:35:43.480192+0000: internal op fragmentdir:mds.0:3850 currently failed to wrlock, waiting 2024-03-28T00:43:00.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:43:00 smithi165 ceph-mon[36302]: slow request 436.577856 seconds old, received at 2024-03-28T00:35:43.448744+0000: internal op fragmentdir:mds.0:3788 currently failed to wrlock, waiting 2024-03-28T00:43:00.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:43:00 smithi165 ceph-mon[36302]: slow request 436.559644 seconds old, received at 2024-03-28T00:35:43.466955+0000: internal op fragmentdir:mds.0:3819 currently failed to wrlock, waiting 2024-03-28T00:43:00.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:43:00 smithi165 ceph-mon[36302]: slow request 436.545933 seconds old, received at 2024-03-28T00:35:43.480667+0000: internal op fragmentdir:mds.0:3851 currently failed to wrlock, waiting 2024-03-28T00:43:00.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:43:00 smithi165 ceph-mon[36302]: slow request 436.532761 seconds old, received at 2024-03-28T00:35:43.493839+0000: internal op fragmentdir:mds.0:3883 currently failed to wrlock, waiting 2024-03-28T00:43:00.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:43:00 smithi119 ceph-mon[31518]: 98 slow requests, 5 included below; oldest blocked for > 440.667655 secs 2024-03-28T00:43:00.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:43:00 smithi119 ceph-mon[31518]: slow request 436.546407 seconds old, received at 2024-03-28T00:35:43.480192+0000: internal op fragmentdir:mds.0:3850 currently failed to wrlock, waiting 2024-03-28T00:43:00.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:43:00 smithi119 ceph-mon[31518]: slow request 436.577856 seconds old, received at 2024-03-28T00:35:43.448744+0000: internal op fragmentdir:mds.0:3788 currently failed to wrlock, waiting 2024-03-28T00:43:00.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:43:00 smithi119 ceph-mon[31518]: slow request 436.559644 seconds old, received at 2024-03-28T00:35:43.466955+0000: internal op fragmentdir:mds.0:3819 currently failed to wrlock, waiting 2024-03-28T00:43:00.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:43:00 smithi119 ceph-mon[31518]: slow request 436.545933 seconds old, received at 2024-03-28T00:35:43.480667+0000: internal op fragmentdir:mds.0:3851 currently failed to wrlock, waiting 2024-03-28T00:43:00.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:43:00 smithi119 ceph-mon[31518]: slow request 436.532761 seconds old, received at 2024-03-28T00:35:43.493839+0000: internal op fragmentdir:mds.0:3883 currently failed to wrlock, waiting 2024-03-28T00:43:00.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:43:00 smithi132 ceph-mon[35941]: 98 slow requests, 5 included below; oldest blocked for > 440.667655 secs 2024-03-28T00:43:00.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:43:00 smithi132 ceph-mon[35941]: slow request 436.546407 seconds old, received at 2024-03-28T00:35:43.480192+0000: internal op fragmentdir:mds.0:3850 currently failed to wrlock, waiting 2024-03-28T00:43:00.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:43:00 smithi132 ceph-mon[35941]: slow request 436.577856 seconds old, received at 2024-03-28T00:35:43.448744+0000: internal op fragmentdir:mds.0:3788 currently failed to wrlock, waiting 2024-03-28T00:43:00.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:43:00 smithi132 ceph-mon[35941]: slow request 436.559644 seconds old, received at 2024-03-28T00:35:43.466955+0000: internal op fragmentdir:mds.0:3819 currently failed to wrlock, waiting 2024-03-28T00:43:00.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:43:00 smithi132 ceph-mon[35941]: slow request 436.545933 seconds old, received at 2024-03-28T00:35:43.480667+0000: internal op fragmentdir:mds.0:3851 currently failed to wrlock, waiting 2024-03-28T00:43:00.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:43:00 smithi132 ceph-mon[35941]: slow request 436.532761 seconds old, received at 2024-03-28T00:35:43.493839+0000: internal op fragmentdir:mds.0:3883 currently failed to wrlock, waiting 2024-03-28T00:43:01.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:43:01 smithi165 ceph-mon[36302]: pgmap v540: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:43:01.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:43:01 smithi119 ceph-mon[31518]: pgmap v540: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:43:01.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:43:01 smithi132 ceph-mon[35941]: pgmap v540: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:43:03.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:43:03 smithi165 ceph-mon[36302]: pgmap v541: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:43:03.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:43:03 smithi119 ceph-mon[31518]: pgmap v541: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:43:03.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:43:03 smithi132 ceph-mon[35941]: pgmap v541: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:43:05.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:43:05 smithi165 ceph-mon[36302]: pgmap v542: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:43:05.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:43:05 smithi165 ceph-mon[36302]: 98 slow requests, 5 included below; oldest blocked for > 445.667743 secs 2024-03-28T00:43:05.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:43:05 smithi165 ceph-mon[36302]: slow request 441.577527 seconds old, received at 2024-03-28T00:35:43.449161+0000: internal op fragmentdir:mds.0:3789 currently failed to wrlock, waiting 2024-03-28T00:43:05.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:43:05 smithi165 ceph-mon[36302]: slow request 441.559296 seconds old, received at 2024-03-28T00:35:43.467391+0000: internal op fragmentdir:mds.0:3820 currently failed to wrlock, waiting 2024-03-28T00:43:05.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:43:05 smithi165 ceph-mon[36302]: slow request 441.545546 seconds old, received at 2024-03-28T00:35:43.481141+0000: internal op fragmentdir:mds.0:3852 currently failed to wrlock, waiting 2024-03-28T00:43:05.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:43:05 smithi165 ceph-mon[36302]: slow request 441.503754 seconds old, received at 2024-03-28T00:35:43.522934+0000: internal op fragmentdir:mds.0:3916 currently failed to wrlock, waiting 2024-03-28T00:43:05.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:43:05 smithi165 ceph-mon[36302]: slow request 441.577067 seconds old, received at 2024-03-28T00:35:43.449621+0000: internal op fragmentdir:mds.0:3790 currently failed to wrlock, waiting 2024-03-28T00:43:05.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:43:05 smithi119 ceph-mon[31518]: pgmap v542: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:43:05.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:43:05 smithi119 ceph-mon[31518]: 98 slow requests, 5 included below; oldest blocked for > 445.667743 secs 2024-03-28T00:43:05.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:43:05 smithi119 ceph-mon[31518]: slow request 441.577527 seconds old, received at 2024-03-28T00:35:43.449161+0000: internal op fragmentdir:mds.0:3789 currently failed to wrlock, waiting 2024-03-28T00:43:05.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:43:05 smithi119 ceph-mon[31518]: slow request 441.559296 seconds old, received at 2024-03-28T00:35:43.467391+0000: internal op fragmentdir:mds.0:3820 currently failed to wrlock, waiting 2024-03-28T00:43:05.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:43:05 smithi119 ceph-mon[31518]: slow request 441.545546 seconds old, received at 2024-03-28T00:35:43.481141+0000: internal op fragmentdir:mds.0:3852 currently failed to wrlock, waiting 2024-03-28T00:43:05.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:43:05 smithi119 ceph-mon[31518]: slow request 441.503754 seconds old, received at 2024-03-28T00:35:43.522934+0000: internal op fragmentdir:mds.0:3916 currently failed to wrlock, waiting 2024-03-28T00:43:05.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:43:05 smithi119 ceph-mon[31518]: slow request 441.577067 seconds old, received at 2024-03-28T00:35:43.449621+0000: internal op fragmentdir:mds.0:3790 currently failed to wrlock, waiting 2024-03-28T00:43:05.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:43:05 smithi132 ceph-mon[35941]: pgmap v542: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:43:05.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:43:05 smithi132 ceph-mon[35941]: 98 slow requests, 5 included below; oldest blocked for > 445.667743 secs 2024-03-28T00:43:05.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:43:05 smithi132 ceph-mon[35941]: slow request 441.577527 seconds old, received at 2024-03-28T00:35:43.449161+0000: internal op fragmentdir:mds.0:3789 currently failed to wrlock, waiting 2024-03-28T00:43:05.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:43:05 smithi132 ceph-mon[35941]: slow request 441.559296 seconds old, received at 2024-03-28T00:35:43.467391+0000: internal op fragmentdir:mds.0:3820 currently failed to wrlock, waiting 2024-03-28T00:43:05.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:43:05 smithi132 ceph-mon[35941]: slow request 441.545546 seconds old, received at 2024-03-28T00:35:43.481141+0000: internal op fragmentdir:mds.0:3852 currently failed to wrlock, waiting 2024-03-28T00:43:05.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:43:05 smithi132 ceph-mon[35941]: slow request 441.503754 seconds old, received at 2024-03-28T00:35:43.522934+0000: internal op fragmentdir:mds.0:3916 currently failed to wrlock, waiting 2024-03-28T00:43:05.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:43:05 smithi132 ceph-mon[35941]: slow request 441.577067 seconds old, received at 2024-03-28T00:35:43.449621+0000: internal op fragmentdir:mds.0:3790 currently failed to wrlock, waiting 2024-03-28T00:43:07.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:43:07 smithi165 ceph-mon[36302]: pgmap v543: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:43:07.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:43:07 smithi119 ceph-mon[31518]: pgmap v543: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:43:07.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:43:07 smithi132 ceph-mon[35941]: pgmap v543: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:43:09.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:43:09 smithi165 ceph-mon[36302]: pgmap v544: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:43:09.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:43:09 smithi119 ceph-mon[31518]: pgmap v544: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:43:09.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:43:09 smithi132 ceph-mon[35941]: pgmap v544: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:43:10.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:43:10 smithi165 ceph-mon[36302]: 98 slow requests, 5 included below; oldest blocked for > 450.667827 secs 2024-03-28T00:43:10.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:43:10 smithi165 ceph-mon[36302]: slow request 446.558977 seconds old, received at 2024-03-28T00:35:43.467794+0000: internal op fragmentdir:mds.0:3821 currently failed to wrlock, waiting 2024-03-28T00:43:10.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:43:10 smithi165 ceph-mon[36302]: slow request 446.532149 seconds old, received at 2024-03-28T00:35:43.494622+0000: internal op fragmentdir:mds.0:3885 currently failed to wrlock, waiting 2024-03-28T00:43:10.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:43:10 smithi165 ceph-mon[36302]: slow request 446.576608 seconds old, received at 2024-03-28T00:35:43.450163+0000: internal op fragmentdir:mds.0:3791 currently failed to wrlock, waiting 2024-03-28T00:43:10.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:43:10 smithi165 ceph-mon[36302]: slow request 446.544712 seconds old, received at 2024-03-28T00:35:43.482059+0000: internal op fragmentdir:mds.0:3854 currently failed to wrlock, waiting 2024-03-28T00:43:10.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:43:10 smithi165 ceph-mon[36302]: slow request 446.531697 seconds old, received at 2024-03-28T00:35:43.495074+0000: internal op fragmentdir:mds.0:3886 currently failed to wrlock, waiting 2024-03-28T00:43:10.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:43:10 smithi119 ceph-mon[31518]: 98 slow requests, 5 included below; oldest blocked for > 450.667827 secs 2024-03-28T00:43:10.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:43:10 smithi119 ceph-mon[31518]: slow request 446.558977 seconds old, received at 2024-03-28T00:35:43.467794+0000: internal op fragmentdir:mds.0:3821 currently failed to wrlock, waiting 2024-03-28T00:43:10.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:43:10 smithi119 ceph-mon[31518]: slow request 446.532149 seconds old, received at 2024-03-28T00:35:43.494622+0000: internal op fragmentdir:mds.0:3885 currently failed to wrlock, waiting 2024-03-28T00:43:10.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:43:10 smithi119 ceph-mon[31518]: slow request 446.576608 seconds old, received at 2024-03-28T00:35:43.450163+0000: internal op fragmentdir:mds.0:3791 currently failed to wrlock, waiting 2024-03-28T00:43:10.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:43:10 smithi119 ceph-mon[31518]: slow request 446.544712 seconds old, received at 2024-03-28T00:35:43.482059+0000: internal op fragmentdir:mds.0:3854 currently failed to wrlock, waiting 2024-03-28T00:43:10.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:43:10 smithi119 ceph-mon[31518]: slow request 446.531697 seconds old, received at 2024-03-28T00:35:43.495074+0000: internal op fragmentdir:mds.0:3886 currently failed to wrlock, waiting 2024-03-28T00:43:10.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:43:10 smithi132 ceph-mon[35941]: 98 slow requests, 5 included below; oldest blocked for > 450.667827 secs 2024-03-28T00:43:10.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:43:10 smithi132 ceph-mon[35941]: slow request 446.558977 seconds old, received at 2024-03-28T00:35:43.467794+0000: internal op fragmentdir:mds.0:3821 currently failed to wrlock, waiting 2024-03-28T00:43:10.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:43:10 smithi132 ceph-mon[35941]: slow request 446.532149 seconds old, received at 2024-03-28T00:35:43.494622+0000: internal op fragmentdir:mds.0:3885 currently failed to wrlock, waiting 2024-03-28T00:43:10.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:43:10 smithi132 ceph-mon[35941]: slow request 446.576608 seconds old, received at 2024-03-28T00:35:43.450163+0000: internal op fragmentdir:mds.0:3791 currently failed to wrlock, waiting 2024-03-28T00:43:10.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:43:10 smithi132 ceph-mon[35941]: slow request 446.544712 seconds old, received at 2024-03-28T00:35:43.482059+0000: internal op fragmentdir:mds.0:3854 currently failed to wrlock, waiting 2024-03-28T00:43:10.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:43:10 smithi132 ceph-mon[35941]: slow request 446.531697 seconds old, received at 2024-03-28T00:35:43.495074+0000: internal op fragmentdir:mds.0:3886 currently failed to wrlock, waiting 2024-03-28T00:43:11.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:43:11 smithi165 ceph-mon[36302]: pgmap v545: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:43:11.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:43:11 smithi119 ceph-mon[31518]: pgmap v545: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:43:11.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:43:11 smithi132 ceph-mon[35941]: pgmap v545: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:43:13.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:43:13 smithi165 ceph-mon[36302]: pgmap v546: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:43:13.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:43:13 smithi119 ceph-mon[31518]: pgmap v546: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:43:13.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:43:13 smithi132 ceph-mon[35941]: pgmap v546: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:43:15.877 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:43:15 smithi119 ceph-mon[31518]: pgmap v547: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:43:15.877 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:43:15 smithi119 ceph-mon[31518]: 98 slow requests, 5 included below; oldest blocked for > 455.667915 secs 2024-03-28T00:43:15.877 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:43:15 smithi119 ceph-mon[31518]: slow request 451.502805 seconds old, received at 2024-03-28T00:35:43.524054+0000: internal op fragmentdir:mds.0:3918 currently failed to wrlock, waiting 2024-03-28T00:43:15.877 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:43:15 smithi119 ceph-mon[31518]: slow request 451.576247 seconds old, received at 2024-03-28T00:35:43.450613+0000: internal op fragmentdir:mds.0:3792 currently failed to wrlock, waiting 2024-03-28T00:43:15.878 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:43:15 smithi119 ceph-mon[31518]: slow request 451.558471 seconds old, received at 2024-03-28T00:35:43.468389+0000: internal op fragmentdir:mds.0:3823 currently failed to wrlock, waiting 2024-03-28T00:43:15.878 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:43:15 smithi119 ceph-mon[31518]: slow request 451.575754 seconds old, received at 2024-03-28T00:35:43.451105+0000: internal op fragmentdir:mds.0:3793 currently failed to wrlock, waiting 2024-03-28T00:43:15.878 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:43:15 smithi119 ceph-mon[31518]: slow request 451.530912 seconds old, received at 2024-03-28T00:35:43.495948+0000: internal op fragmentdir:mds.0:3888 currently failed to wrlock, waiting 2024-03-28T00:43:15.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:43:15 smithi165 ceph-mon[36302]: pgmap v547: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:43:15.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:43:15 smithi165 ceph-mon[36302]: 98 slow requests, 5 included below; oldest blocked for > 455.667915 secs 2024-03-28T00:43:15.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:43:15 smithi165 ceph-mon[36302]: slow request 451.502805 seconds old, received at 2024-03-28T00:35:43.524054+0000: internal op fragmentdir:mds.0:3918 currently failed to wrlock, waiting 2024-03-28T00:43:15.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:43:15 smithi165 ceph-mon[36302]: slow request 451.576247 seconds old, received at 2024-03-28T00:35:43.450613+0000: internal op fragmentdir:mds.0:3792 currently failed to wrlock, waiting 2024-03-28T00:43:15.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:43:15 smithi165 ceph-mon[36302]: slow request 451.558471 seconds old, received at 2024-03-28T00:35:43.468389+0000: internal op fragmentdir:mds.0:3823 currently failed to wrlock, waiting 2024-03-28T00:43:15.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:43:15 smithi165 ceph-mon[36302]: slow request 451.575754 seconds old, received at 2024-03-28T00:35:43.451105+0000: internal op fragmentdir:mds.0:3793 currently failed to wrlock, waiting 2024-03-28T00:43:15.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:43:15 smithi165 ceph-mon[36302]: slow request 451.530912 seconds old, received at 2024-03-28T00:35:43.495948+0000: internal op fragmentdir:mds.0:3888 currently failed to wrlock, waiting 2024-03-28T00:43:15.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:43:15 smithi132 ceph-mon[35941]: pgmap v547: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:43:15.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:43:15 smithi132 ceph-mon[35941]: 98 slow requests, 5 included below; oldest blocked for > 455.667915 secs 2024-03-28T00:43:15.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:43:15 smithi132 ceph-mon[35941]: slow request 451.502805 seconds old, received at 2024-03-28T00:35:43.524054+0000: internal op fragmentdir:mds.0:3918 currently failed to wrlock, waiting 2024-03-28T00:43:15.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:43:15 smithi132 ceph-mon[35941]: slow request 451.576247 seconds old, received at 2024-03-28T00:35:43.450613+0000: internal op fragmentdir:mds.0:3792 currently failed to wrlock, waiting 2024-03-28T00:43:15.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:43:15 smithi132 ceph-mon[35941]: slow request 451.558471 seconds old, received at 2024-03-28T00:35:43.468389+0000: internal op fragmentdir:mds.0:3823 currently failed to wrlock, waiting 2024-03-28T00:43:15.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:43:15 smithi132 ceph-mon[35941]: slow request 451.575754 seconds old, received at 2024-03-28T00:35:43.451105+0000: internal op fragmentdir:mds.0:3793 currently failed to wrlock, waiting 2024-03-28T00:43:15.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:43:15 smithi132 ceph-mon[35941]: slow request 451.530912 seconds old, received at 2024-03-28T00:35:43.495948+0000: internal op fragmentdir:mds.0:3888 currently failed to wrlock, waiting 2024-03-28T00:43:17.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:43:17 smithi165 ceph-mon[36302]: pgmap v548: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:43:17.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:43:17 smithi119 ceph-mon[31518]: pgmap v548: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:43:17.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:43:17 smithi132 ceph-mon[35941]: pgmap v548: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:43:19.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:43:19 smithi165 ceph-mon[36302]: pgmap v549: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:43:19.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:43:19 smithi119 ceph-mon[31518]: pgmap v549: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:43:19.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:43:19 smithi132 ceph-mon[35941]: pgmap v549: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:43:20.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:43:20 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 460.667999 secs 2024-03-28T00:43:20.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:43:20 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 460.667999 secs 2024-03-28T00:43:20.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:43:20 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 460.667999 secs 2024-03-28T00:43:21.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:43:21 smithi165 ceph-mon[36302]: pgmap v550: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:43:21.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:43:21 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:43:21.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:43:21 smithi119 ceph-mon[31518]: pgmap v550: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:43:21.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:43:21 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:43:21.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:43:21 smithi132 ceph-mon[35941]: pgmap v550: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:43:21.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:43:21 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:43:22.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:43:22 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:43:22.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:43:22 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:43:22.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:43:22 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:43:22.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:43:22 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:43:22.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:43:22 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:43:22.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:43:22 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:43:22.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:43:22 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:43:22.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:43:22 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:43:22.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:43:22 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:43:23.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:43:23 smithi165 ceph-mon[36302]: pgmap v551: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T00:43:23.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:43:23 smithi165 ceph-mon[36302]: pgmap v552: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T00:43:23.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:43:23 smithi119 ceph-mon[31518]: pgmap v551: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T00:43:23.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:43:23 smithi119 ceph-mon[31518]: pgmap v552: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T00:43:23.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:43:23 smithi132 ceph-mon[35941]: pgmap v551: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T00:43:23.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:43:23 smithi132 ceph-mon[35941]: pgmap v552: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T00:43:25.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:43:25 smithi165 ceph-mon[36302]: pgmap v553: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-28T00:43:25.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:43:25 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 465.668060 secs 2024-03-28T00:43:25.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:43:25 smithi119 ceph-mon[31518]: pgmap v553: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-28T00:43:25.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:43:25 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 465.668060 secs 2024-03-28T00:43:25.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:43:25 smithi132 ceph-mon[35941]: pgmap v553: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-28T00:43:25.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:43:25 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 465.668060 secs 2024-03-28T00:43:27.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:43:27 smithi165 ceph-mon[36302]: pgmap v554: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T00:43:27.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:43:27 smithi119 ceph-mon[31518]: pgmap v554: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T00:43:27.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:43:27 smithi132 ceph-mon[35941]: pgmap v554: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T00:43:29.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:43:29 smithi165 ceph-mon[36302]: pgmap v555: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T00:43:29.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:43:29 smithi119 ceph-mon[31518]: pgmap v555: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T00:43:29.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:43:29 smithi132 ceph-mon[35941]: pgmap v555: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T00:43:30.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:43:30 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 470.668147 secs 2024-03-28T00:43:30.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:43:30 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 470.668147 secs 2024-03-28T00:43:30.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:43:30 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 470.668147 secs 2024-03-28T00:43:31.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:43:31 smithi165 ceph-mon[36302]: pgmap v556: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T00:43:31.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:43:31 smithi119 ceph-mon[31518]: pgmap v556: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T00:43:31.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:43:31 smithi132 ceph-mon[35941]: pgmap v556: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T00:43:33.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:43:33 smithi165 ceph-mon[36302]: pgmap v557: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T00:43:33.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:43:33 smithi119 ceph-mon[31518]: pgmap v557: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T00:43:33.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:43:33 smithi132 ceph-mon[35941]: pgmap v557: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T00:43:35.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:43:35 smithi119 ceph-mon[31518]: pgmap v558: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:43:35.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:43:35 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 475.668237 secs 2024-03-28T00:43:35.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:43:35 smithi132 ceph-mon[35941]: pgmap v558: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:43:35.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:43:35 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 475.668237 secs 2024-03-28T00:43:36.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:43:35 smithi165 ceph-mon[36302]: pgmap v558: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:43:36.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:43:35 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 475.668237 secs 2024-03-28T00:43:37.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:43:37 smithi119 ceph-mon[31518]: pgmap v559: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:43:37.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:43:37 smithi132 ceph-mon[35941]: pgmap v559: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:43:38.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:43:37 smithi165 ceph-mon[36302]: pgmap v559: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:43:39.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:43:39 smithi119 ceph-mon[31518]: pgmap v560: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:43:39.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:43:39 smithi132 ceph-mon[35941]: pgmap v560: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:43:40.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:43:39 smithi165 ceph-mon[36302]: pgmap v560: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:43:40.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:43:40 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 480.668274 secs 2024-03-28T00:43:41.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:43:40 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 480.668274 secs 2024-03-28T00:43:41.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:43:40 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 480.668274 secs 2024-03-28T00:43:41.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:43:41 smithi132 ceph-mon[35941]: pgmap v561: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:43:42.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:43:41 smithi165 ceph-mon[36302]: pgmap v561: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:43:42.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:43:41 smithi119 ceph-mon[31518]: pgmap v561: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:43:43.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:43:43 smithi132 ceph-mon[35941]: pgmap v562: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:43:44.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:43:43 smithi165 ceph-mon[36302]: pgmap v562: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:43:44.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:43:43 smithi119 ceph-mon[31518]: pgmap v562: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:43:45.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:43:45 smithi132 ceph-mon[35941]: pgmap v563: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:43:45.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:43:45 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 485.668362 secs 2024-03-28T00:43:46.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:43:45 smithi165 ceph-mon[36302]: pgmap v563: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:43:46.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:43:45 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 485.668362 secs 2024-03-28T00:43:46.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:43:45 smithi119 ceph-mon[31518]: pgmap v563: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:43:46.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:43:45 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 485.668362 secs 2024-03-28T00:43:47.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:43:47 smithi132 ceph-mon[35941]: pgmap v564: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:43:48.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:43:47 smithi165 ceph-mon[36302]: pgmap v564: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:43:48.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:43:47 smithi119 ceph-mon[31518]: pgmap v564: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:43:49.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:43:49 smithi132 ceph-mon[35941]: pgmap v565: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:43:50.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:43:49 smithi165 ceph-mon[36302]: pgmap v565: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:43:50.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:43:49 smithi119 ceph-mon[31518]: pgmap v565: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:43:50.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:43:50 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 490.668451 secs 2024-03-28T00:43:51.056 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:43:50 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 490.668451 secs 2024-03-28T00:43:51.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:43:50 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 490.668451 secs 2024-03-28T00:43:51.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:43:51 smithi132 ceph-mon[35941]: pgmap v566: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:43:52.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:43:51 smithi165 ceph-mon[36302]: pgmap v566: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:43:52.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:43:51 smithi119 ceph-mon[31518]: pgmap v566: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:43:53.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:43:53 smithi132 ceph-mon[35941]: pgmap v567: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:43:54.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:43:53 smithi165 ceph-mon[36302]: pgmap v567: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:43:54.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:43:53 smithi119 ceph-mon[31518]: pgmap v567: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:43:55.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:43:55 smithi132 ceph-mon[35941]: pgmap v568: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:43:55.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:43:55 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 495.668539 secs 2024-03-28T00:43:56.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:43:55 smithi165 ceph-mon[36302]: pgmap v568: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:43:56.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:43:55 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 495.668539 secs 2024-03-28T00:43:56.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:43:55 smithi119 ceph-mon[31518]: pgmap v568: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:43:56.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:43:55 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 495.668539 secs 2024-03-28T00:43:57.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:43:57 smithi132 ceph-mon[35941]: pgmap v569: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:43:58.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:43:57 smithi165 ceph-mon[36302]: pgmap v569: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:43:58.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:43:57 smithi119 ceph-mon[31518]: pgmap v569: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:44:00.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:43:59 smithi165 ceph-mon[36302]: pgmap v570: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:44:00.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:43:59 smithi119 ceph-mon[31518]: pgmap v570: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:44:00.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:43:59 smithi132 ceph-mon[35941]: pgmap v570: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:44:01.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:44:00 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 500.668601 secs 2024-03-28T00:44:01.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:44:00 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 500.668601 secs 2024-03-28T00:44:01.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:44:00 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 500.668601 secs 2024-03-28T00:44:02.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:44:01 smithi165 ceph-mon[36302]: pgmap v571: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:44:02.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:44:01 smithi119 ceph-mon[31518]: pgmap v571: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:44:02.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:44:01 smithi132 ceph-mon[35941]: pgmap v571: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:44:04.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:44:03 smithi165 ceph-mon[36302]: pgmap v572: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:44:04.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:44:03 smithi119 ceph-mon[31518]: pgmap v572: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:44:04.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:44:03 smithi132 ceph-mon[35941]: pgmap v572: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:44:06.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:44:05 smithi165 ceph-mon[36302]: pgmap v573: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:44:06.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:44:05 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 505.668689 secs 2024-03-28T00:44:06.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:44:05 smithi119 ceph-mon[31518]: pgmap v573: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:44:06.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:44:05 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 505.668689 secs 2024-03-28T00:44:06.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:44:05 smithi132 ceph-mon[35941]: pgmap v573: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:44:06.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:44:05 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 505.668689 secs 2024-03-28T00:44:08.002 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:44:07 smithi132 ceph-mon[35941]: pgmap v574: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:44:08.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:44:07 smithi165 ceph-mon[36302]: pgmap v574: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:44:08.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:44:07 smithi119 ceph-mon[31518]: pgmap v574: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:44:10.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:44:09 smithi165 ceph-mon[36302]: pgmap v575: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:44:10.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:44:09 smithi119 ceph-mon[31518]: pgmap v575: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:44:10.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:44:09 smithi132 ceph-mon[35941]: pgmap v575: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:44:11.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:44:10 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 510.668779 secs 2024-03-28T00:44:11.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:44:10 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 510.668779 secs 2024-03-28T00:44:11.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:44:10 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 510.668779 secs 2024-03-28T00:44:12.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:44:11 smithi165 ceph-mon[36302]: pgmap v576: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:44:12.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:44:11 smithi119 ceph-mon[31518]: pgmap v576: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:44:12.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:44:11 smithi132 ceph-mon[35941]: pgmap v576: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:44:14.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:44:13 smithi165 ceph-mon[36302]: pgmap v577: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:44:14.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:44:13 smithi119 ceph-mon[31518]: pgmap v577: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:44:14.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:44:13 smithi132 ceph-mon[35941]: pgmap v577: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:44:16.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:44:15 smithi165 ceph-mon[36302]: pgmap v578: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:44:16.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:44:15 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 515.668870 secs 2024-03-28T00:44:16.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:44:15 smithi119 ceph-mon[31518]: pgmap v578: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:44:16.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:44:15 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 515.668870 secs 2024-03-28T00:44:16.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:44:15 smithi132 ceph-mon[35941]: pgmap v578: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:44:16.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:44:15 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 515.668870 secs 2024-03-28T00:44:18.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:44:17 smithi165 ceph-mon[36302]: pgmap v579: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:44:18.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:44:17 smithi119 ceph-mon[31518]: pgmap v579: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:44:18.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:44:17 smithi132 ceph-mon[35941]: pgmap v579: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:44:20.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:44:19 smithi165 ceph-mon[36302]: pgmap v580: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T00:44:20.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:44:19 smithi119 ceph-mon[31518]: pgmap v580: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T00:44:20.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:44:19 smithi132 ceph-mon[35941]: pgmap v580: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T00:44:21.056 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:44:20 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 520.668955 secs 2024-03-28T00:44:21.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:44:20 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 520.668955 secs 2024-03-28T00:44:21.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:44:20 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 520.668955 secs 2024-03-28T00:44:22.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:44:21 smithi165 ceph-mon[36302]: pgmap v581: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:44:22.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:44:21 smithi119 ceph-mon[31518]: pgmap v581: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:44:22.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:44:21 smithi132 ceph-mon[35941]: pgmap v581: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:44:23.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:44:22 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:44:23.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:44:22 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:44:23.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:44:22 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:44:24.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:44:23 smithi165 ceph-mon[36302]: pgmap v582: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T00:44:24.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:44:23 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:44:24.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:44:23 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:44:24.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:44:23 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:44:24.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:44:23 smithi119 ceph-mon[31518]: pgmap v582: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T00:44:24.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:44:23 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:44:24.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:44:23 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:44:24.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:44:23 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:44:24.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:44:23 smithi132 ceph-mon[35941]: pgmap v582: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T00:44:24.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:44:23 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:44:24.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:44:23 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:44:24.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:44:23 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:44:25.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:44:24 smithi165 ceph-mon[36302]: pgmap v583: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T00:44:25.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:44:24 smithi165 ceph-mon[36302]: pgmap v584: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T00:44:25.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:44:24 smithi119 ceph-mon[31518]: pgmap v583: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T00:44:25.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:44:24 smithi119 ceph-mon[31518]: pgmap v584: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T00:44:25.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:44:24 smithi132 ceph-mon[35941]: pgmap v583: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T00:44:25.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:44:24 smithi132 ceph-mon[35941]: pgmap v584: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T00:44:26.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:44:25 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 525.669038 secs 2024-03-28T00:44:26.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:44:25 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 525.669038 secs 2024-03-28T00:44:26.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:44:25 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 525.669038 secs 2024-03-28T00:44:27.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:44:26 smithi165 ceph-mon[36302]: pgmap v585: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T00:44:27.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:44:26 smithi119 ceph-mon[31518]: pgmap v585: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T00:44:27.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:44:26 smithi132 ceph-mon[35941]: pgmap v585: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T00:44:29.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:44:28 smithi165 ceph-mon[36302]: pgmap v586: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T00:44:29.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:44:28 smithi119 ceph-mon[31518]: pgmap v586: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T00:44:29.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:44:28 smithi132 ceph-mon[35941]: pgmap v586: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T00:44:31.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:44:30 smithi165 ceph-mon[36302]: pgmap v587: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T00:44:31.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:44:30 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 530.669100 secs 2024-03-28T00:44:31.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:44:30 smithi119 ceph-mon[31518]: pgmap v587: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T00:44:31.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:44:30 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 530.669100 secs 2024-03-28T00:44:31.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:44:30 smithi132 ceph-mon[35941]: pgmap v587: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T00:44:31.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:44:30 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 530.669100 secs 2024-03-28T00:44:33.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:44:32 smithi165 ceph-mon[36302]: pgmap v588: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T00:44:33.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:44:32 smithi119 ceph-mon[31518]: pgmap v588: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T00:44:33.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:44:32 smithi132 ceph-mon[35941]: pgmap v588: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T00:44:35.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:44:34 smithi165 ceph-mon[36302]: pgmap v589: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T00:44:35.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:44:34 smithi119 ceph-mon[31518]: pgmap v589: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T00:44:35.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:44:34 smithi132 ceph-mon[35941]: pgmap v589: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T00:44:36.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:44:35 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 535.669170 secs 2024-03-28T00:44:36.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:44:35 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 535.669170 secs 2024-03-28T00:44:36.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:44:35 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 535.669170 secs 2024-03-28T00:44:37.103 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:44:36 smithi132 ceph-mon[35941]: pgmap v590: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:44:37.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:44:36 smithi165 ceph-mon[36302]: pgmap v590: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:44:37.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:44:36 smithi119 ceph-mon[31518]: pgmap v590: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:44:39.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:44:38 smithi165 ceph-mon[36302]: pgmap v591: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:44:39.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:44:38 smithi119 ceph-mon[31518]: pgmap v591: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:44:39.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:44:38 smithi132 ceph-mon[35941]: pgmap v591: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:44:41.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:44:40 smithi165 ceph-mon[36302]: pgmap v592: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:44:41.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:44:40 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 540.669259 secs 2024-03-28T00:44:41.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:44:40 smithi119 ceph-mon[31518]: pgmap v592: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:44:41.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:44:40 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 540.669259 secs 2024-03-28T00:44:41.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:44:40 smithi132 ceph-mon[35941]: pgmap v592: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:44:41.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:44:40 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 540.669259 secs 2024-03-28T00:44:43.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:44:42 smithi165 ceph-mon[36302]: pgmap v593: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:44:43.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:44:42 smithi119 ceph-mon[31518]: pgmap v593: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:44:43.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:44:42 smithi132 ceph-mon[35941]: pgmap v593: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:44:45.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:44:44 smithi165 ceph-mon[36302]: pgmap v594: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:44:45.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:44:44 smithi119 ceph-mon[31518]: pgmap v594: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:44:45.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:44:44 smithi132 ceph-mon[35941]: pgmap v594: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:44:46.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:44:45 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 545.669342 secs 2024-03-28T00:44:46.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:44:45 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 545.669342 secs 2024-03-28T00:44:46.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:44:45 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 545.669342 secs 2024-03-28T00:44:47.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:44:46 smithi165 ceph-mon[36302]: pgmap v595: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:44:47.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:44:46 smithi119 ceph-mon[31518]: pgmap v595: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:44:47.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:44:46 smithi132 ceph-mon[35941]: pgmap v595: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:44:49.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:44:48 smithi165 ceph-mon[36302]: pgmap v596: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:44:49.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:44:48 smithi119 ceph-mon[31518]: pgmap v596: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:44:49.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:44:48 smithi132 ceph-mon[35941]: pgmap v596: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:44:51.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:44:50 smithi119 ceph-mon[31518]: pgmap v597: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:44:51.248 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:44:50 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 550.669401 secs 2024-03-28T00:44:51.249 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:44:50 smithi132 ceph-mon[35941]: pgmap v597: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:44:51.249 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:44:50 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 550.669401 secs 2024-03-28T00:44:51.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:44:50 smithi165 ceph-mon[36302]: pgmap v597: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:44:51.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:44:50 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 550.669401 secs 2024-03-28T00:44:53.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:44:52 smithi119 ceph-mon[31518]: pgmap v598: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:44:53.183 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:44:52 smithi165 ceph-mon[36302]: pgmap v598: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:44:53.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:44:52 smithi132 ceph-mon[35941]: pgmap v598: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:44:55.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:44:54 smithi119 ceph-mon[31518]: pgmap v599: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:44:55.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:44:54 smithi132 ceph-mon[35941]: pgmap v599: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:44:55.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:44:54 smithi165 ceph-mon[36302]: pgmap v599: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:44:56.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:44:55 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 555.669460 secs 2024-03-28T00:44:56.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:44:55 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 555.669460 secs 2024-03-28T00:44:56.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:44:55 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 555.669460 secs 2024-03-28T00:44:57.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:44:56 smithi132 ceph-mon[35941]: pgmap v600: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:44:57.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:44:56 smithi165 ceph-mon[36302]: pgmap v600: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:44:57.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:44:56 smithi119 ceph-mon[31518]: pgmap v600: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:44:58.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:44:57 smithi132 ceph-mon[35941]: pgmap v601: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:44:58.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:44:57 smithi165 ceph-mon[36302]: pgmap v601: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:44:58.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:44:57 smithi119 ceph-mon[31518]: pgmap v601: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:45:00.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:45:00 smithi165 ceph-mon[36302]: pgmap v602: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:45:00.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:45:00 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 560.669502 secs 2024-03-28T00:45:00.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:45:00 smithi119 ceph-mon[31518]: pgmap v602: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:45:00.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:45:00 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 560.669502 secs 2024-03-28T00:45:00.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:45:00 smithi132 ceph-mon[35941]: pgmap v602: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:45:00.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:45:00 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 560.669502 secs 2024-03-28T00:45:02.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:45:02 smithi165 ceph-mon[36302]: pgmap v603: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:45:02.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:45:02 smithi119 ceph-mon[31518]: pgmap v603: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:45:02.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:45:02 smithi132 ceph-mon[35941]: pgmap v603: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:45:04.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:45:04 smithi165 ceph-mon[36302]: pgmap v604: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:45:04.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:45:04 smithi119 ceph-mon[31518]: pgmap v604: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:45:04.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:45:04 smithi132 ceph-mon[35941]: pgmap v604: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:45:05.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:45:05 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 565.669563 secs 2024-03-28T00:45:05.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:45:05 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 565.669563 secs 2024-03-28T00:45:05.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:45:05 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 565.669563 secs 2024-03-28T00:45:06.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:45:06 smithi165 ceph-mon[36302]: pgmap v605: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:45:06.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:45:06 smithi119 ceph-mon[31518]: pgmap v605: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:45:06.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:45:06 smithi132 ceph-mon[35941]: pgmap v605: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:45:08.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:45:08 smithi165 ceph-mon[36302]: pgmap v606: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:45:08.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:45:08 smithi119 ceph-mon[31518]: pgmap v606: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:45:08.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:45:08 smithi132 ceph-mon[35941]: pgmap v606: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:45:10.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:45:10 smithi165 ceph-mon[36302]: pgmap v607: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:45:10.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:45:10 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 570.669622 secs 2024-03-28T00:45:10.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:45:10 smithi119 ceph-mon[31518]: pgmap v607: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:45:10.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:45:10 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 570.669622 secs 2024-03-28T00:45:10.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:45:10 smithi132 ceph-mon[35941]: pgmap v607: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:45:10.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:45:10 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 570.669622 secs 2024-03-28T00:45:12.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:45:12 smithi165 ceph-mon[36302]: pgmap v608: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:45:12.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:45:12 smithi119 ceph-mon[31518]: pgmap v608: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:45:12.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:45:12 smithi132 ceph-mon[35941]: pgmap v608: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:45:14.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:45:14 smithi165 ceph-mon[36302]: pgmap v609: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:45:14.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:45:14 smithi119 ceph-mon[31518]: pgmap v609: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:45:14.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:45:14 smithi132 ceph-mon[35941]: pgmap v609: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:45:15.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:45:15 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 575.669682 secs 2024-03-28T00:45:15.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:45:15 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 575.669682 secs 2024-03-28T00:45:15.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:45:15 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 575.669682 secs 2024-03-28T00:45:16.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:45:16 smithi165 ceph-mon[36302]: pgmap v610: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:45:16.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:45:16 smithi119 ceph-mon[31518]: pgmap v610: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:45:16.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:45:16 smithi132 ceph-mon[35941]: pgmap v610: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:45:18.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:45:18 smithi165 ceph-mon[36302]: pgmap v611: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:45:18.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:45:18 smithi119 ceph-mon[31518]: pgmap v611: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:45:18.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:45:18 smithi132 ceph-mon[35941]: pgmap v611: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:45:20.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:45:20 smithi165 ceph-mon[36302]: pgmap v612: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:45:20.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:45:20 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 580.669745 secs 2024-03-28T00:45:20.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:45:20 smithi119 ceph-mon[31518]: pgmap v612: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:45:20.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:45:20 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 580.669745 secs 2024-03-28T00:45:20.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:45:20 smithi132 ceph-mon[35941]: pgmap v612: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:45:20.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:45:20 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 580.669745 secs 2024-03-28T00:45:22.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:45:22 smithi165 ceph-mon[36302]: pgmap v613: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:45:22.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:45:22 smithi119 ceph-mon[31518]: pgmap v613: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:45:22.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:45:22 smithi132 ceph-mon[35941]: pgmap v613: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:45:24.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:45:24 smithi165 ceph-mon[36302]: pgmap v614: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:45:24.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:45:24 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:45:24.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:45:24 smithi119 ceph-mon[31518]: pgmap v614: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:45:24.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:45:24 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:45:24.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:45:24 smithi132 ceph-mon[35941]: pgmap v614: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:45:24.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:45:24 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:45:25.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:45:25 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:45:25.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:45:25 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:45:25.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:45:25 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:45:25.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:45:25 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 585.669834 secs 2024-03-28T00:45:25.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:45:25 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:45:25.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:45:25 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:45:25.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:45:25 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:45:25.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:45:25 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 585.669834 secs 2024-03-28T00:45:25.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:45:25 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:45:25.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:45:25 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:45:25.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:45:25 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:45:25.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:45:25 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 585.669834 secs 2024-03-28T00:45:26.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:45:26 smithi165 ceph-mon[36302]: pgmap v615: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T00:45:26.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:45:26 smithi165 ceph-mon[36302]: pgmap v616: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T00:45:26.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:45:26 smithi119 ceph-mon[31518]: pgmap v615: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T00:45:26.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:45:26 smithi119 ceph-mon[31518]: pgmap v616: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T00:45:26.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:45:26 smithi132 ceph-mon[35941]: pgmap v615: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T00:45:26.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:45:26 smithi132 ceph-mon[35941]: pgmap v616: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T00:45:28.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:45:28 smithi165 ceph-mon[36302]: pgmap v617: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T00:45:28.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:45:28 smithi119 ceph-mon[31518]: pgmap v617: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T00:45:28.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:45:28 smithi132 ceph-mon[35941]: pgmap v617: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T00:45:30.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:45:30 smithi165 ceph-mon[36302]: pgmap v618: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-28T00:45:30.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:45:30 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 590.669897 secs 2024-03-28T00:45:30.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:45:30 smithi119 ceph-mon[31518]: pgmap v618: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-28T00:45:30.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:45:30 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 590.669897 secs 2024-03-28T00:45:30.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:45:30 smithi132 ceph-mon[35941]: pgmap v618: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-28T00:45:30.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:45:30 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 590.669897 secs 2024-03-28T00:45:32.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:45:32 smithi165 ceph-mon[36302]: pgmap v619: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T00:45:32.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:45:32 smithi119 ceph-mon[31518]: pgmap v619: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T00:45:32.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:45:32 smithi132 ceph-mon[35941]: pgmap v619: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T00:45:34.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:45:34 smithi165 ceph-mon[36302]: pgmap v620: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T00:45:34.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:45:34 smithi119 ceph-mon[31518]: pgmap v620: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T00:45:34.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:45:34 smithi132 ceph-mon[35941]: pgmap v620: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T00:45:35.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:45:35 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 595.669939 secs 2024-03-28T00:45:35.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:45:35 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 595.669939 secs 2024-03-28T00:45:35.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:45:35 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 595.669939 secs 2024-03-28T00:45:36.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:45:36 smithi165 ceph-mon[36302]: pgmap v621: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T00:45:36.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:45:36 smithi119 ceph-mon[31518]: pgmap v621: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T00:45:36.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:45:36 smithi132 ceph-mon[35941]: pgmap v621: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T00:45:38.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:45:38 smithi165 ceph-mon[36302]: pgmap v622: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:45:38.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:45:38 smithi119 ceph-mon[31518]: pgmap v622: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:45:38.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:45:38 smithi132 ceph-mon[35941]: pgmap v622: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:45:40.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:45:40 smithi165 ceph-mon[36302]: pgmap v623: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:45:40.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:45:40 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 600.670023 secs 2024-03-28T00:45:40.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:45:40 smithi119 ceph-mon[31518]: pgmap v623: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:45:40.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:45:40 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 600.670023 secs 2024-03-28T00:45:40.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:45:40 smithi132 ceph-mon[35941]: pgmap v623: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:45:40.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:45:40 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 600.670023 secs 2024-03-28T00:45:42.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:45:42 smithi119 ceph-mon[31518]: pgmap v624: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:45:42.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:45:42 smithi132 ceph-mon[35941]: pgmap v624: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:45:43.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:45:42 smithi165 ceph-mon[36302]: pgmap v624: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:45:44.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:45:44 smithi119 ceph-mon[31518]: pgmap v625: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:45:44.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:45:44 smithi132 ceph-mon[35941]: pgmap v625: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:45:45.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:45:44 smithi165 ceph-mon[36302]: pgmap v625: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:45:45.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:45:45 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 605.670084 secs 2024-03-28T00:45:45.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:45:45 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 605.670084 secs 2024-03-28T00:45:46.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:45:45 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 605.670084 secs 2024-03-28T00:45:46.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:45:46 smithi119 ceph-mon[31518]: pgmap v626: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:45:46.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:45:46 smithi132 ceph-mon[35941]: pgmap v626: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:45:47.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:45:46 smithi165 ceph-mon[36302]: pgmap v626: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:45:48.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:45:48 smithi132 ceph-mon[35941]: pgmap v627: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:45:49.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:45:48 smithi165 ceph-mon[36302]: pgmap v627: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:45:49.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:45:48 smithi119 ceph-mon[31518]: pgmap v627: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:45:50.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:45:50 smithi132 ceph-mon[35941]: pgmap v628: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:45:50.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:45:50 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 610.670159 secs 2024-03-28T00:45:51.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:45:50 smithi165 ceph-mon[36302]: pgmap v628: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:45:51.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:45:50 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 610.670159 secs 2024-03-28T00:45:51.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:45:50 smithi119 ceph-mon[31518]: pgmap v628: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:45:51.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:45:50 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 610.670159 secs 2024-03-28T00:45:52.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:45:52 smithi132 ceph-mon[35941]: pgmap v629: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:45:52.956 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:45:52 smithi119 ceph-mon[31518]: pgmap v629: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:45:53.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:45:52 smithi165 ceph-mon[36302]: pgmap v629: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:45:54.933 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:45:54 smithi165 ceph-mon[36302]: pgmap v630: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:45:54.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:45:54 smithi132 ceph-mon[35941]: pgmap v630: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:45:55.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:45:54 smithi119 ceph-mon[31518]: pgmap v630: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:45:55.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:45:55 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 615.670227 secs 2024-03-28T00:45:56.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:45:55 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 615.670227 secs 2024-03-28T00:45:56.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:45:55 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 615.670227 secs 2024-03-28T00:45:56.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:45:56 smithi132 ceph-mon[35941]: pgmap v631: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:45:57.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:45:56 smithi165 ceph-mon[36302]: pgmap v631: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:45:57.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:45:56 smithi119 ceph-mon[31518]: pgmap v631: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:45:58.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:45:58 smithi132 ceph-mon[35941]: pgmap v632: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:45:59.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:45:58 smithi165 ceph-mon[36302]: pgmap v632: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:45:59.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:45:58 smithi119 ceph-mon[31518]: pgmap v632: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:46:00.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:46:00 smithi132 ceph-mon[35941]: pgmap v633: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:46:00.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:46:00 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 620.670317 secs 2024-03-28T00:46:01.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:46:00 smithi165 ceph-mon[36302]: pgmap v633: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:46:01.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:46:00 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 620.670317 secs 2024-03-28T00:46:01.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:46:00 smithi119 ceph-mon[31518]: pgmap v633: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:46:01.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:46:00 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 620.670317 secs 2024-03-28T00:46:02.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:46:02 smithi132 ceph-mon[35941]: pgmap v634: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:46:03.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:46:02 smithi165 ceph-mon[36302]: pgmap v634: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:46:03.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:46:02 smithi119 ceph-mon[31518]: pgmap v634: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:46:05.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:46:04 smithi165 ceph-mon[36302]: pgmap v635: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:46:05.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:46:04 smithi119 ceph-mon[31518]: pgmap v635: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:46:05.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:46:04 smithi132 ceph-mon[35941]: pgmap v635: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:46:06.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:46:05 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 625.670404 secs 2024-03-28T00:46:06.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:46:05 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 625.670404 secs 2024-03-28T00:46:06.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:46:05 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 625.670404 secs 2024-03-28T00:46:07.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:46:06 smithi165 ceph-mon[36302]: pgmap v636: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:46:07.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:46:06 smithi119 ceph-mon[31518]: pgmap v636: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:46:07.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:46:06 smithi132 ceph-mon[35941]: pgmap v636: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:46:09.002 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:46:08 smithi132 ceph-mon[35941]: pgmap v637: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:46:09.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:46:08 smithi165 ceph-mon[36302]: pgmap v637: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:46:09.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:46:08 smithi119 ceph-mon[31518]: pgmap v637: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:46:11.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:46:10 smithi165 ceph-mon[36302]: pgmap v638: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:46:11.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:46:10 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 630.670492 secs 2024-03-28T00:46:11.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:46:10 smithi119 ceph-mon[31518]: pgmap v638: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:46:11.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:46:10 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 630.670492 secs 2024-03-28T00:46:11.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:46:10 smithi132 ceph-mon[35941]: pgmap v638: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:46:11.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:46:10 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 630.670492 secs 2024-03-28T00:46:13.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:46:12 smithi165 ceph-mon[36302]: pgmap v639: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:46:13.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:46:12 smithi119 ceph-mon[31518]: pgmap v639: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:46:13.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:46:12 smithi132 ceph-mon[35941]: pgmap v639: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:46:15.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:46:14 smithi165 ceph-mon[36302]: pgmap v640: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:46:15.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:46:14 smithi119 ceph-mon[31518]: pgmap v640: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:46:15.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:46:14 smithi132 ceph-mon[35941]: pgmap v640: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:46:16.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:46:15 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 635.670581 secs 2024-03-28T00:46:16.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:46:15 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 635.670581 secs 2024-03-28T00:46:16.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:46:15 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 635.670581 secs 2024-03-28T00:46:17.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:46:16 smithi165 ceph-mon[36302]: pgmap v641: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:46:17.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:46:16 smithi119 ceph-mon[31518]: pgmap v641: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:46:17.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:46:16 smithi132 ceph-mon[35941]: pgmap v641: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:46:19.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:46:18 smithi165 ceph-mon[36302]: pgmap v642: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:46:19.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:46:18 smithi119 ceph-mon[31518]: pgmap v642: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:46:19.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:46:18 smithi132 ceph-mon[35941]: pgmap v642: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:46:21.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:46:20 smithi165 ceph-mon[36302]: pgmap v643: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:46:21.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:46:20 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 640.670623 secs 2024-03-28T00:46:21.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:46:20 smithi119 ceph-mon[31518]: pgmap v643: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:46:21.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:46:20 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 640.670623 secs 2024-03-28T00:46:21.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:46:20 smithi132 ceph-mon[35941]: pgmap v643: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:46:21.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:46:20 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 640.670623 secs 2024-03-28T00:46:23.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:46:22 smithi165 ceph-mon[36302]: pgmap v644: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:46:23.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:46:22 smithi119 ceph-mon[31518]: pgmap v644: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:46:23.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:46:22 smithi132 ceph-mon[35941]: pgmap v644: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:46:25.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:46:24 smithi165 ceph-mon[36302]: pgmap v645: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:46:25.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:46:24 smithi119 ceph-mon[31518]: pgmap v645: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:46:25.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:46:24 smithi132 ceph-mon[35941]: pgmap v645: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:46:26.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:46:25 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 645.670710 secs 2024-03-28T00:46:26.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:46:25 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:46:26.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:46:25 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 645.670710 secs 2024-03-28T00:46:26.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:46:25 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:46:26.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:46:25 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 645.670710 secs 2024-03-28T00:46:26.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:46:25 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:46:27.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:46:26 smithi165 ceph-mon[36302]: pgmap v646: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:46:27.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:46:26 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:46:27.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:46:26 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:46:27.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:46:26 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:46:27.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:46:26 smithi119 ceph-mon[31518]: pgmap v646: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:46:27.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:46:26 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:46:27.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:46:26 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:46:27.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:46:26 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:46:27.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:46:26 smithi132 ceph-mon[35941]: pgmap v646: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:46:27.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:46:26 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:46:27.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:46:26 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:46:27.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:46:26 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:46:28.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:46:27 smithi165 ceph-mon[36302]: pgmap v647: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T00:46:28.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:46:27 smithi165 ceph-mon[36302]: pgmap v648: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T00:46:28.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:46:27 smithi119 ceph-mon[31518]: pgmap v647: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T00:46:28.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:46:27 smithi119 ceph-mon[31518]: pgmap v648: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T00:46:28.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:46:27 smithi132 ceph-mon[35941]: pgmap v647: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T00:46:28.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:46:27 smithi132 ceph-mon[35941]: pgmap v648: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T00:46:30.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:46:29 smithi165 ceph-mon[36302]: pgmap v649: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T00:46:30.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:46:29 smithi119 ceph-mon[31518]: pgmap v649: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T00:46:30.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:46:29 smithi132 ceph-mon[35941]: pgmap v649: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T00:46:31.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:46:30 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 650.670799 secs 2024-03-28T00:46:31.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:46:30 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 650.670799 secs 2024-03-28T00:46:31.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:46:30 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 650.670799 secs 2024-03-28T00:46:32.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:46:31 smithi165 ceph-mon[36302]: pgmap v650: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T00:46:32.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:46:31 smithi119 ceph-mon[31518]: pgmap v650: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T00:46:32.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:46:31 smithi132 ceph-mon[35941]: pgmap v650: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T00:46:34.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:46:33 smithi165 ceph-mon[36302]: pgmap v651: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T00:46:34.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:46:33 smithi119 ceph-mon[31518]: pgmap v651: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T00:46:34.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:46:33 smithi132 ceph-mon[35941]: pgmap v651: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T00:46:36.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:46:35 smithi165 ceph-mon[36302]: pgmap v652: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T00:46:36.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:46:35 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 655.670881 secs 2024-03-28T00:46:36.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:46:35 smithi119 ceph-mon[31518]: pgmap v652: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T00:46:36.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:46:35 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 655.670881 secs 2024-03-28T00:46:36.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:46:35 smithi132 ceph-mon[35941]: pgmap v652: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T00:46:36.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:46:35 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 655.670881 secs 2024-03-28T00:46:38.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:46:37 smithi165 ceph-mon[36302]: pgmap v653: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-28T00:46:38.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:46:37 smithi119 ceph-mon[31518]: pgmap v653: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-28T00:46:38.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:46:37 smithi132 ceph-mon[35941]: pgmap v653: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-28T00:46:40.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:46:39 smithi165 ceph-mon[36302]: pgmap v654: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T00:46:40.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:46:39 smithi119 ceph-mon[31518]: pgmap v654: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T00:46:40.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:46:39 smithi132 ceph-mon[35941]: pgmap v654: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T00:46:41.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:46:40 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 660.670952 secs 2024-03-28T00:46:41.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:46:40 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 660.670952 secs 2024-03-28T00:46:41.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:46:40 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 660.670952 secs 2024-03-28T00:46:42.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:46:41 smithi165 ceph-mon[36302]: pgmap v655: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:46:42.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:46:41 smithi119 ceph-mon[31518]: pgmap v655: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:46:42.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:46:41 smithi132 ceph-mon[35941]: pgmap v655: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:46:44.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:46:43 smithi165 ceph-mon[36302]: pgmap v656: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T00:46:44.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:46:43 smithi119 ceph-mon[31518]: pgmap v656: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T00:46:44.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:46:43 smithi132 ceph-mon[35941]: pgmap v656: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T00:46:46.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:46:45 smithi165 ceph-mon[36302]: pgmap v657: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:46:46.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:46:45 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 665.671037 secs 2024-03-28T00:46:46.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:46:45 smithi119 ceph-mon[31518]: pgmap v657: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:46:46.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:46:45 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 665.671037 secs 2024-03-28T00:46:46.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:46:45 smithi132 ceph-mon[35941]: pgmap v657: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:46:46.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:46:45 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 665.671037 secs 2024-03-28T00:46:48.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:46:47 smithi165 ceph-mon[36302]: pgmap v658: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:46:48.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:46:47 smithi119 ceph-mon[31518]: pgmap v658: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:46:48.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:46:47 smithi132 ceph-mon[35941]: pgmap v658: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:46:50.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:46:49 smithi165 ceph-mon[36302]: pgmap v659: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:46:50.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:46:49 smithi119 ceph-mon[31518]: pgmap v659: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:46:50.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:46:49 smithi132 ceph-mon[35941]: pgmap v659: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:46:51.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:46:50 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 670.671123 secs 2024-03-28T00:46:51.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:46:50 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 670.671123 secs 2024-03-28T00:46:51.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:46:50 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 670.671123 secs 2024-03-28T00:46:52.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:46:51 smithi165 ceph-mon[36302]: pgmap v660: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:46:52.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:46:51 smithi119 ceph-mon[31518]: pgmap v660: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:46:52.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:46:51 smithi132 ceph-mon[35941]: pgmap v660: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:46:54.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:46:53 smithi165 ceph-mon[36302]: pgmap v661: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:46:54.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:46:53 smithi119 ceph-mon[31518]: pgmap v661: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:46:54.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:46:53 smithi132 ceph-mon[35941]: pgmap v661: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:46:56.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:46:55 smithi165 ceph-mon[36302]: pgmap v662: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:46:56.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:46:55 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 675.671134 secs 2024-03-28T00:46:56.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:46:55 smithi119 ceph-mon[31518]: pgmap v662: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:46:56.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:46:55 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 675.671134 secs 2024-03-28T00:46:56.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:46:55 smithi132 ceph-mon[35941]: pgmap v662: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:46:56.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:46:55 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 675.671134 secs 2024-03-28T00:46:58.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:46:57 smithi165 ceph-mon[36302]: pgmap v663: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:46:58.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:46:57 smithi119 ceph-mon[31518]: pgmap v663: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:46:58.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:46:57 smithi132 ceph-mon[35941]: pgmap v663: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:47:00.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:46:59 smithi165 ceph-mon[36302]: pgmap v664: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:47:00.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:46:59 smithi119 ceph-mon[31518]: pgmap v664: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:47:00.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:46:59 smithi132 ceph-mon[35941]: pgmap v664: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:47:01.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:47:00 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 680.671168 secs 2024-03-28T00:47:01.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:47:00 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 680.671168 secs 2024-03-28T00:47:01.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:47:00 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 680.671168 secs 2024-03-28T00:47:02.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:47:01 smithi119 ceph-mon[31518]: pgmap v665: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:47:02.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:47:01 smithi132 ceph-mon[35941]: pgmap v665: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:47:02.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:47:01 smithi165 ceph-mon[36302]: pgmap v665: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:47:04.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:47:03 smithi119 ceph-mon[31518]: pgmap v666: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:47:04.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:47:03 smithi132 ceph-mon[35941]: pgmap v666: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:47:04.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:47:03 smithi165 ceph-mon[36302]: pgmap v666: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:47:06.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:47:05 smithi119 ceph-mon[31518]: pgmap v667: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:47:06.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:47:05 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 685.671230 secs 2024-03-28T00:47:06.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:47:05 smithi132 ceph-mon[35941]: pgmap v667: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:47:06.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:47:05 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 685.671230 secs 2024-03-28T00:47:06.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:47:05 smithi165 ceph-mon[36302]: pgmap v667: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:47:06.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:47:05 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 685.671230 secs 2024-03-28T00:47:08.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:47:07 smithi119 ceph-mon[31518]: pgmap v668: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:47:08.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:47:07 smithi132 ceph-mon[35941]: pgmap v668: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:47:08.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:47:07 smithi165 ceph-mon[36302]: pgmap v668: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:47:10.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:47:09 smithi119 ceph-mon[31518]: pgmap v669: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:47:10.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:47:09 smithi132 ceph-mon[35941]: pgmap v669: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:47:10.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:47:09 smithi165 ceph-mon[36302]: pgmap v669: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:47:11.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:47:10 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 690.671291 secs 2024-03-28T00:47:11.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:47:10 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 690.671291 secs 2024-03-28T00:47:11.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:47:10 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 690.671291 secs 2024-03-28T00:47:12.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:47:11 smithi132 ceph-mon[35941]: pgmap v670: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:47:12.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:47:11 smithi165 ceph-mon[36302]: pgmap v670: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:47:12.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:47:11 smithi119 ceph-mon[31518]: pgmap v670: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:47:14.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:47:13 smithi132 ceph-mon[35941]: pgmap v671: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:47:14.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:47:13 smithi165 ceph-mon[36302]: pgmap v671: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:47:14.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:47:13 smithi119 ceph-mon[31518]: pgmap v671: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:47:16.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:47:15 smithi132 ceph-mon[35941]: pgmap v672: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:47:16.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:47:15 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 695.671351 secs 2024-03-28T00:47:16.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:47:15 smithi165 ceph-mon[36302]: pgmap v672: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:47:16.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:47:15 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 695.671351 secs 2024-03-28T00:47:16.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:47:15 smithi119 ceph-mon[31518]: pgmap v672: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:47:16.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:47:15 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 695.671351 secs 2024-03-28T00:47:18.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:47:17 smithi132 ceph-mon[35941]: pgmap v673: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:47:18.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:47:17 smithi165 ceph-mon[36302]: pgmap v673: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:47:18.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:47:17 smithi119 ceph-mon[31518]: pgmap v673: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:47:19.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:47:18 smithi132 ceph-mon[35941]: pgmap v674: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:47:19.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:47:18 smithi165 ceph-mon[36302]: pgmap v674: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:47:19.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:47:18 smithi119 ceph-mon[31518]: pgmap v674: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:47:20.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:47:20 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 700.671439 secs 2024-03-28T00:47:20.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:47:20 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 700.671439 secs 2024-03-28T00:47:20.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:47:20 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 700.671439 secs 2024-03-28T00:47:21.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:47:21 smithi132 ceph-mon[35941]: pgmap v675: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:47:21.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:47:21 smithi165 ceph-mon[36302]: pgmap v675: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:47:21.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:47:21 smithi119 ceph-mon[31518]: pgmap v675: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:47:23.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:47:23 smithi165 ceph-mon[36302]: pgmap v676: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:47:23.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:47:23 smithi119 ceph-mon[31518]: pgmap v676: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:47:23.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:47:23 smithi132 ceph-mon[35941]: pgmap v676: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:47:25.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:47:25 smithi165 ceph-mon[36302]: pgmap v677: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:47:25.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:47:25 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 705.671482 secs 2024-03-28T00:47:25.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:47:25 smithi119 ceph-mon[31518]: pgmap v677: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:47:25.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:47:25 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 705.671482 secs 2024-03-28T00:47:25.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:47:25 smithi132 ceph-mon[35941]: pgmap v677: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:47:25.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:47:25 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 705.671482 secs 2024-03-28T00:47:27.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:47:27 smithi165 ceph-mon[36302]: pgmap v678: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:47:27.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:47:27 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:47:27.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:47:27 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:47:27.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:47:27 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:47:27.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:47:27 smithi119 ceph-mon[31518]: pgmap v678: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:47:27.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:47:27 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:47:27.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:47:27 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:47:27.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:47:27 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:47:27.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:47:27 smithi132 ceph-mon[35941]: pgmap v678: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:47:27.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:47:27 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:47:27.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:47:27 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:47:27.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:47:27 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:47:28.850 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:47:28 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:47:28.851 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:47:28 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:47:28.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:47:28 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:47:28.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:47:28 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:47:28.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:47:28 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:47:28.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:47:28 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:47:29.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:47:29 smithi165 ceph-mon[36302]: pgmap v679: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:47:29.920 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:47:29 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:47:29.920 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:47:29 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:47:29.920 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:47:29 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:47:29.920 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:47:29 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:47:29.920 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:47:29 smithi119 ceph-mon[31518]: pgmap v679: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:47:29.920 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:47:29 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:47:29.921 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:47:29 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:47:29.921 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:47:29 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:47:29.921 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:47:29 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:47:29.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:47:29 smithi132 ceph-mon[35941]: pgmap v679: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:47:29.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:47:29 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:47:29.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:47:29 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:47:29.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:47:29 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:47:29.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:47:29 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:47:30.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:47:30 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:47:30.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:47:30 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:47:30.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:47:30 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:47:30.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:47:30 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:47:30.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:47:30 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:47:30.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:47:30 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:47:30.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:47:30 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:47:30.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:47:30 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:47:30.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:47:30 smithi165 ceph-mon[36302]: Adjusting osd_memory_target on smithi165 to 174.1M 2024-03-28T00:47:30.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:47:30 smithi165 ceph-mon[36302]: Unable to set osd_memory_target on smithi165 to 182635622: error parsing value: Value '182635622' is below minimum 939524096 2024-03-28T00:47:30.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:47:30 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:47:30.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:47:30 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:47:30.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:47:30 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:47:30.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:47:30 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:47:30.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:47:30 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:47:30.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:47:30 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:47:30.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:47:30 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:47:30.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:47:30 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:47:30.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:47:30 smithi165 ceph-mon[36302]: Adjusting osd_memory_target on smithi132 to 174.1M 2024-03-28T00:47:30.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:47:30 smithi165 ceph-mon[36302]: Unable to set osd_memory_target on smithi132 to 182640640: error parsing value: Value '182640640' is below minimum 939524096 2024-03-28T00:47:30.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:47:30 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 710.671568 secs 2024-03-28T00:47:30.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:47:30 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:47:30.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:47:30 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:47:30.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:47:30 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:47:30.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:47:30 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:47:30.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:47:30 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:47:30.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:47:30 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:47:30.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:47:30 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:47:30.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:47:30 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:47:30.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:47:30 smithi119 ceph-mon[31518]: Adjusting osd_memory_target on smithi165 to 174.1M 2024-03-28T00:47:30.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:47:30 smithi119 ceph-mon[31518]: Unable to set osd_memory_target on smithi165 to 182635622: error parsing value: Value '182635622' is below minimum 939524096 2024-03-28T00:47:30.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:47:30 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:47:30.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:47:30 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:47:30.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:47:30 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:47:30.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:47:30 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:47:30.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:47:30 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:47:30.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:47:30 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:47:30.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:47:30 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:47:30.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:47:30 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:47:30.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:47:30 smithi119 ceph-mon[31518]: Adjusting osd_memory_target on smithi132 to 174.1M 2024-03-28T00:47:30.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:47:30 smithi119 ceph-mon[31518]: Unable to set osd_memory_target on smithi132 to 182640640: error parsing value: Value '182640640' is below minimum 939524096 2024-03-28T00:47:30.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:47:30 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 710.671568 secs 2024-03-28T00:47:30.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:47:30 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:47:30.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:47:30 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:47:30.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:47:30 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:47:30.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:47:30 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:47:30.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:47:30 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:47:30.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:47:30 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:47:30.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:47:30 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:47:30.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:47:30 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:47:30.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:47:30 smithi132 ceph-mon[35941]: Adjusting osd_memory_target on smithi165 to 174.1M 2024-03-28T00:47:30.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:47:30 smithi132 ceph-mon[35941]: Unable to set osd_memory_target on smithi165 to 182635622: error parsing value: Value '182635622' is below minimum 939524096 2024-03-28T00:47:30.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:47:30 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:47:30.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:47:30 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:47:30.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:47:30 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:47:30.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:47:30 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:47:30.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:47:30 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:47:30.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:47:30 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:47:30.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:47:30 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:47:30.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:47:30 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:47:30.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:47:30 smithi132 ceph-mon[35941]: Adjusting osd_memory_target on smithi132 to 174.1M 2024-03-28T00:47:30.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:47:30 smithi132 ceph-mon[35941]: Unable to set osd_memory_target on smithi132 to 182640640: error parsing value: Value '182640640' is below minimum 939524096 2024-03-28T00:47:30.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:47:30 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 710.671568 secs 2024-03-28T00:47:31.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:47:31 smithi165 ceph-mon[36302]: pgmap v680: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:47:31.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:47:31 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:47:31.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:47:31 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:47:31.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:47:31 smithi119 ceph-mon[31518]: pgmap v680: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:47:31.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:47:31 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:47:31.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:47:31 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:47:31.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:47:31 smithi132 ceph-mon[35941]: pgmap v680: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:47:31.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:47:31 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:47:31.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:47:31 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:47:32.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:47:32 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:47:32.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:47:32 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:47:32.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:47:32 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:47:32.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:47:32 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:47:32.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:47:32 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:47:32.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:47:32 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:47:32.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:47:32 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:47:32.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:47:32 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:47:32.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:47:32 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:47:32.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:47:32 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:47:32.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:47:32 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:47:32.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:47:32 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:47:32.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:47:32 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:47:32.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:47:32 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:47:32.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:47:32 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:47:32.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:47:32 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:47:32.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:47:32 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:47:32.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:47:32 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:47:32.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:47:32 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:47:32.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:47:32 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:47:32.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:47:32 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:47:32.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:47:32 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:47:32.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:47:32 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:47:32.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:47:32 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:47:32.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:47:32 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:47:32.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:47:32 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:47:32.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:47:32 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:47:32.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:47:32 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:47:32.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:47:32 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:47:32.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:47:32 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:47:32.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:47:32 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:47:32.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:47:32 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:47:32.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:47:32 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:47:33.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:47:33 smithi165 ceph-mon[36302]: Adjusting osd_memory_target on smithi119 to 176.7M 2024-03-28T00:47:33.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:47:33 smithi165 ceph-mon[36302]: Unable to set osd_memory_target on smithi119 to 185372364: error parsing value: Value '185372364' is below minimum 939524096 2024-03-28T00:47:33.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:47:33 smithi165 ceph-mon[36302]: pgmap v681: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:47:33.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:47:33 smithi165 ceph-mon[36302]: pgmap v682: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T00:47:33.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:47:33 smithi119 ceph-mon[31518]: Adjusting osd_memory_target on smithi119 to 176.7M 2024-03-28T00:47:33.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:47:33 smithi119 ceph-mon[31518]: Unable to set osd_memory_target on smithi119 to 185372364: error parsing value: Value '185372364' is below minimum 939524096 2024-03-28T00:47:33.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:47:33 smithi119 ceph-mon[31518]: pgmap v681: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:47:33.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:47:33 smithi119 ceph-mon[31518]: pgmap v682: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T00:47:33.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:47:33 smithi132 ceph-mon[35941]: Adjusting osd_memory_target on smithi119 to 176.7M 2024-03-28T00:47:33.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:47:33 smithi132 ceph-mon[35941]: Unable to set osd_memory_target on smithi119 to 185372364: error parsing value: Value '185372364' is below minimum 939524096 2024-03-28T00:47:33.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:47:33 smithi132 ceph-mon[35941]: pgmap v681: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:47:33.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:47:33 smithi132 ceph-mon[35941]: pgmap v682: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T00:47:35.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:47:35 smithi165 ceph-mon[36302]: pgmap v683: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T00:47:35.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:47:35 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 715.671657 secs 2024-03-28T00:47:35.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:47:35 smithi119 ceph-mon[31518]: pgmap v683: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T00:47:35.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:47:35 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 715.671657 secs 2024-03-28T00:47:35.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:47:35 smithi132 ceph-mon[35941]: pgmap v683: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T00:47:35.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:47:35 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 715.671657 secs 2024-03-28T00:47:37.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:47:37 smithi165 ceph-mon[36302]: pgmap v684: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T00:47:37.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:47:37 smithi119 ceph-mon[31518]: pgmap v684: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T00:47:37.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:47:37 smithi132 ceph-mon[35941]: pgmap v684: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T00:47:39.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:47:39 smithi119 ceph-mon[31518]: pgmap v685: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T00:47:39.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:47:39 smithi132 ceph-mon[35941]: pgmap v685: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T00:47:40.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:47:39 smithi165 ceph-mon[36302]: pgmap v685: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T00:47:40.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:47:40 smithi119 ceph-mon[31518]: 98 slow requests, 3 included below; oldest blocked for > 720.671745 secs 2024-03-28T00:47:40.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:47:40 smithi119 ceph-mon[31518]: slow request 720.451770 seconds old, received at 2024-03-28T00:35:39.578920+0000: internal op fragmentdir:mds.0:3783 currently failed to wrlock, waiting 2024-03-28T00:47:40.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:47:40 smithi119 ceph-mon[31518]: slow request 720.442822 seconds old, received at 2024-03-28T00:35:39.587868+0000: client_request(client.25264:33202 create owner_uid=1000, owner_gid=1301 #0x10000000270/file8309 2024-03-28T00:35:39.586399+0000 caller_uid=1000, caller_gid=1301{6,36,1000,1301,}) currently failed to xlock, waiting 2024-03-28T00:47:40.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:47:40 smithi119 ceph-mon[31518]: slow request 720.671745 seconds old, received at 2024-03-28T00:35:39.358945+0000: peer_request:client.25264:11489 currently acquired locks 2024-03-28T00:47:40.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:47:40 smithi132 ceph-mon[35941]: 98 slow requests, 3 included below; oldest blocked for > 720.671745 secs 2024-03-28T00:47:40.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:47:40 smithi132 ceph-mon[35941]: slow request 720.451770 seconds old, received at 2024-03-28T00:35:39.578920+0000: internal op fragmentdir:mds.0:3783 currently failed to wrlock, waiting 2024-03-28T00:47:40.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:47:40 smithi132 ceph-mon[35941]: slow request 720.442822 seconds old, received at 2024-03-28T00:35:39.587868+0000: client_request(client.25264:33202 create owner_uid=1000, owner_gid=1301 #0x10000000270/file8309 2024-03-28T00:35:39.586399+0000 caller_uid=1000, caller_gid=1301{6,36,1000,1301,}) currently failed to xlock, waiting 2024-03-28T00:47:40.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:47:40 smithi132 ceph-mon[35941]: slow request 720.671745 seconds old, received at 2024-03-28T00:35:39.358945+0000: peer_request:client.25264:11489 currently acquired locks 2024-03-28T00:47:41.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:47:40 smithi165 ceph-mon[36302]: 98 slow requests, 3 included below; oldest blocked for > 720.671745 secs 2024-03-28T00:47:41.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:47:40 smithi165 ceph-mon[36302]: slow request 720.451770 seconds old, received at 2024-03-28T00:35:39.578920+0000: internal op fragmentdir:mds.0:3783 currently failed to wrlock, waiting 2024-03-28T00:47:41.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:47:40 smithi165 ceph-mon[36302]: slow request 720.442822 seconds old, received at 2024-03-28T00:35:39.587868+0000: client_request(client.25264:33202 create owner_uid=1000, owner_gid=1301 #0x10000000270/file8309 2024-03-28T00:35:39.586399+0000 caller_uid=1000, caller_gid=1301{6,36,1000,1301,}) currently failed to xlock, waiting 2024-03-28T00:47:41.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:47:40 smithi165 ceph-mon[36302]: slow request 720.671745 seconds old, received at 2024-03-28T00:35:39.358945+0000: peer_request:client.25264:11489 currently acquired locks 2024-03-28T00:47:41.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:47:41 smithi119 ceph-mon[31518]: pgmap v686: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T00:47:41.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:47:41 smithi132 ceph-mon[35941]: pgmap v686: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T00:47:42.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:47:41 smithi165 ceph-mon[36302]: pgmap v686: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T00:47:43.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:47:43 smithi132 ceph-mon[35941]: pgmap v687: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T00:47:44.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:47:43 smithi165 ceph-mon[36302]: pgmap v687: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T00:47:44.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:47:43 smithi119 ceph-mon[31518]: pgmap v687: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T00:47:45.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:47:45 smithi132 ceph-mon[35941]: pgmap v688: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:47:45.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:47:45 smithi132 ceph-mon[35941]: 98 slow requests, 5 included below; oldest blocked for > 725.671833 secs 2024-03-28T00:47:45.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:47:45 smithi132 ceph-mon[35941]: slow request 721.579171 seconds old, received at 2024-03-28T00:35:43.451608+0000: internal op fragmentdir:mds.0:3794 currently failed to wrlock, waiting 2024-03-28T00:47:45.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:47:45 smithi132 ceph-mon[35941]: slow request 721.561503 seconds old, received at 2024-03-28T00:35:43.469275+0000: internal op fragmentdir:mds.0:3825 currently failed to wrlock, waiting 2024-03-28T00:47:45.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:47:45 smithi132 ceph-mon[35941]: slow request 721.547411 seconds old, received at 2024-03-28T00:35:43.483367+0000: internal op fragmentdir:mds.0:3857 currently failed to wrlock, waiting 2024-03-28T00:47:45.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:47:45 smithi132 ceph-mon[35941]: slow request 721.534350 seconds old, received at 2024-03-28T00:35:43.496429+0000: internal op fragmentdir:mds.0:3889 currently failed to wrlock, waiting 2024-03-28T00:47:45.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:47:45 smithi132 ceph-mon[35941]: slow request 721.577195 seconds old, received at 2024-03-28T00:35:43.453584+0000: internal op fragmentdir:mds.0:3795 currently failed to wrlock, waiting 2024-03-28T00:47:46.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:47:45 smithi165 ceph-mon[36302]: pgmap v688: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:47:46.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:47:45 smithi165 ceph-mon[36302]: 98 slow requests, 5 included below; oldest blocked for > 725.671833 secs 2024-03-28T00:47:46.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:47:45 smithi165 ceph-mon[36302]: slow request 721.579171 seconds old, received at 2024-03-28T00:35:43.451608+0000: internal op fragmentdir:mds.0:3794 currently failed to wrlock, waiting 2024-03-28T00:47:46.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:47:45 smithi165 ceph-mon[36302]: slow request 721.561503 seconds old, received at 2024-03-28T00:35:43.469275+0000: internal op fragmentdir:mds.0:3825 currently failed to wrlock, waiting 2024-03-28T00:47:46.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:47:45 smithi165 ceph-mon[36302]: slow request 721.547411 seconds old, received at 2024-03-28T00:35:43.483367+0000: internal op fragmentdir:mds.0:3857 currently failed to wrlock, waiting 2024-03-28T00:47:46.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:47:45 smithi165 ceph-mon[36302]: slow request 721.534350 seconds old, received at 2024-03-28T00:35:43.496429+0000: internal op fragmentdir:mds.0:3889 currently failed to wrlock, waiting 2024-03-28T00:47:46.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:47:45 smithi165 ceph-mon[36302]: slow request 721.577195 seconds old, received at 2024-03-28T00:35:43.453584+0000: internal op fragmentdir:mds.0:3795 currently failed to wrlock, waiting 2024-03-28T00:47:46.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:47:45 smithi119 ceph-mon[31518]: pgmap v688: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:47:46.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:47:45 smithi119 ceph-mon[31518]: 98 slow requests, 5 included below; oldest blocked for > 725.671833 secs 2024-03-28T00:47:46.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:47:45 smithi119 ceph-mon[31518]: slow request 721.579171 seconds old, received at 2024-03-28T00:35:43.451608+0000: internal op fragmentdir:mds.0:3794 currently failed to wrlock, waiting 2024-03-28T00:47:46.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:47:45 smithi119 ceph-mon[31518]: slow request 721.561503 seconds old, received at 2024-03-28T00:35:43.469275+0000: internal op fragmentdir:mds.0:3825 currently failed to wrlock, waiting 2024-03-28T00:47:46.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:47:45 smithi119 ceph-mon[31518]: slow request 721.547411 seconds old, received at 2024-03-28T00:35:43.483367+0000: internal op fragmentdir:mds.0:3857 currently failed to wrlock, waiting 2024-03-28T00:47:46.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:47:45 smithi119 ceph-mon[31518]: slow request 721.534350 seconds old, received at 2024-03-28T00:35:43.496429+0000: internal op fragmentdir:mds.0:3889 currently failed to wrlock, waiting 2024-03-28T00:47:46.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:47:45 smithi119 ceph-mon[31518]: slow request 721.577195 seconds old, received at 2024-03-28T00:35:43.453584+0000: internal op fragmentdir:mds.0:3795 currently failed to wrlock, waiting 2024-03-28T00:47:47.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:47:47 smithi132 ceph-mon[35941]: pgmap v689: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:47:48.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:47:47 smithi165 ceph-mon[36302]: pgmap v689: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:47:48.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:47:47 smithi119 ceph-mon[31518]: pgmap v689: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:47:49.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:47:49 smithi132 ceph-mon[35941]: pgmap v690: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:47:50.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:47:49 smithi165 ceph-mon[36302]: pgmap v690: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:47:50.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:47:49 smithi119 ceph-mon[31518]: pgmap v690: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:47:50.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:47:50 smithi132 ceph-mon[35941]: 98 slow requests, 5 included below; oldest blocked for > 730.671902 secs 2024-03-28T00:47:50.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:47:50 smithi132 ceph-mon[35941]: slow request 726.496000 seconds old, received at 2024-03-28T00:35:43.534847+0000: internal op fragmentdir:mds.0:3922 currently failed to wrlock, waiting 2024-03-28T00:47:50.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:47:50 smithi132 ceph-mon[35941]: slow request 726.576742 seconds old, received at 2024-03-28T00:35:43.454104+0000: internal op fragmentdir:mds.0:3796 currently failed to wrlock, waiting 2024-03-28T00:47:50.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:47:50 smithi132 ceph-mon[35941]: slow request 726.560683 seconds old, received at 2024-03-28T00:35:43.470163+0000: internal op fragmentdir:mds.0:3827 currently failed to wrlock, waiting 2024-03-28T00:47:50.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:47:50 smithi132 ceph-mon[35941]: slow request 726.533521 seconds old, received at 2024-03-28T00:35:43.497326+0000: internal op fragmentdir:mds.0:3891 currently failed to wrlock, waiting 2024-03-28T00:47:50.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:47:50 smithi132 ceph-mon[35941]: slow request 726.469270 seconds old, received at 2024-03-28T00:35:43.561577+0000: internal op fragmentdir:mds.0:3923 currently failed to wrlock, waiting 2024-03-28T00:47:51.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:47:50 smithi165 ceph-mon[36302]: 98 slow requests, 5 included below; oldest blocked for > 730.671902 secs 2024-03-28T00:47:51.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:47:50 smithi165 ceph-mon[36302]: slow request 726.496000 seconds old, received at 2024-03-28T00:35:43.534847+0000: internal op fragmentdir:mds.0:3922 currently failed to wrlock, waiting 2024-03-28T00:47:51.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:47:50 smithi165 ceph-mon[36302]: slow request 726.576742 seconds old, received at 2024-03-28T00:35:43.454104+0000: internal op fragmentdir:mds.0:3796 currently failed to wrlock, waiting 2024-03-28T00:47:51.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:47:50 smithi165 ceph-mon[36302]: slow request 726.560683 seconds old, received at 2024-03-28T00:35:43.470163+0000: internal op fragmentdir:mds.0:3827 currently failed to wrlock, waiting 2024-03-28T00:47:51.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:47:50 smithi165 ceph-mon[36302]: slow request 726.533521 seconds old, received at 2024-03-28T00:35:43.497326+0000: internal op fragmentdir:mds.0:3891 currently failed to wrlock, waiting 2024-03-28T00:47:51.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:47:50 smithi165 ceph-mon[36302]: slow request 726.469270 seconds old, received at 2024-03-28T00:35:43.561577+0000: internal op fragmentdir:mds.0:3923 currently failed to wrlock, waiting 2024-03-28T00:47:51.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:47:50 smithi119 ceph-mon[31518]: 98 slow requests, 5 included below; oldest blocked for > 730.671902 secs 2024-03-28T00:47:51.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:47:50 smithi119 ceph-mon[31518]: slow request 726.496000 seconds old, received at 2024-03-28T00:35:43.534847+0000: internal op fragmentdir:mds.0:3922 currently failed to wrlock, waiting 2024-03-28T00:47:51.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:47:50 smithi119 ceph-mon[31518]: slow request 726.576742 seconds old, received at 2024-03-28T00:35:43.454104+0000: internal op fragmentdir:mds.0:3796 currently failed to wrlock, waiting 2024-03-28T00:47:51.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:47:50 smithi119 ceph-mon[31518]: slow request 726.560683 seconds old, received at 2024-03-28T00:35:43.470163+0000: internal op fragmentdir:mds.0:3827 currently failed to wrlock, waiting 2024-03-28T00:47:51.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:47:50 smithi119 ceph-mon[31518]: slow request 726.533521 seconds old, received at 2024-03-28T00:35:43.497326+0000: internal op fragmentdir:mds.0:3891 currently failed to wrlock, waiting 2024-03-28T00:47:51.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:47:50 smithi119 ceph-mon[31518]: slow request 726.469270 seconds old, received at 2024-03-28T00:35:43.561577+0000: internal op fragmentdir:mds.0:3923 currently failed to wrlock, waiting 2024-03-28T00:47:51.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:47:51 smithi132 ceph-mon[35941]: pgmap v691: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:47:52.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:47:51 smithi165 ceph-mon[36302]: pgmap v691: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:47:52.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:47:51 smithi119 ceph-mon[31518]: pgmap v691: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:47:53.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:47:53 smithi132 ceph-mon[35941]: pgmap v692: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:47:53.956 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:47:53 smithi119 ceph-mon[31518]: pgmap v692: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:47:54.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:47:53 smithi165 ceph-mon[36302]: pgmap v692: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:47:55.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:47:55 smithi132 ceph-mon[35941]: pgmap v693: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:47:55.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:47:55 smithi132 ceph-mon[35941]: 98 slow requests, 5 included below; oldest blocked for > 735.671988 secs 2024-03-28T00:47:55.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:47:55 smithi132 ceph-mon[35941]: slow request 731.576355 seconds old, received at 2024-03-28T00:35:43.454578+0000: internal op fragmentdir:mds.0:3797 currently failed to wrlock, waiting 2024-03-28T00:47:55.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:47:55 smithi132 ceph-mon[35941]: slow request 731.560327 seconds old, received at 2024-03-28T00:35:43.470605+0000: internal op fragmentdir:mds.0:3828 currently failed to wrlock, waiting 2024-03-28T00:47:55.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:47:55 smithi132 ceph-mon[35941]: slow request 731.546362 seconds old, received at 2024-03-28T00:35:43.484570+0000: internal op fragmentdir:mds.0:3860 currently failed to wrlock, waiting 2024-03-28T00:47:55.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:47:55 smithi132 ceph-mon[35941]: slow request 731.533156 seconds old, received at 2024-03-28T00:35:43.497776+0000: internal op fragmentdir:mds.0:3892 currently failed to wrlock, waiting 2024-03-28T00:47:55.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:47:55 smithi132 ceph-mon[35941]: slow request 731.575878 seconds old, received at 2024-03-28T00:35:43.455055+0000: internal op fragmentdir:mds.0:3798 currently failed to wrlock, waiting 2024-03-28T00:47:56.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:47:55 smithi165 ceph-mon[36302]: pgmap v693: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:47:56.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:47:55 smithi165 ceph-mon[36302]: 98 slow requests, 5 included below; oldest blocked for > 735.671988 secs 2024-03-28T00:47:56.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:47:55 smithi165 ceph-mon[36302]: slow request 731.576355 seconds old, received at 2024-03-28T00:35:43.454578+0000: internal op fragmentdir:mds.0:3797 currently failed to wrlock, waiting 2024-03-28T00:47:56.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:47:55 smithi165 ceph-mon[36302]: slow request 731.560327 seconds old, received at 2024-03-28T00:35:43.470605+0000: internal op fragmentdir:mds.0:3828 currently failed to wrlock, waiting 2024-03-28T00:47:56.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:47:55 smithi165 ceph-mon[36302]: slow request 731.546362 seconds old, received at 2024-03-28T00:35:43.484570+0000: internal op fragmentdir:mds.0:3860 currently failed to wrlock, waiting 2024-03-28T00:47:56.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:47:55 smithi165 ceph-mon[36302]: slow request 731.533156 seconds old, received at 2024-03-28T00:35:43.497776+0000: internal op fragmentdir:mds.0:3892 currently failed to wrlock, waiting 2024-03-28T00:47:56.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:47:55 smithi165 ceph-mon[36302]: slow request 731.575878 seconds old, received at 2024-03-28T00:35:43.455055+0000: internal op fragmentdir:mds.0:3798 currently failed to wrlock, waiting 2024-03-28T00:47:56.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:47:55 smithi119 ceph-mon[31518]: pgmap v693: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:47:56.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:47:55 smithi119 ceph-mon[31518]: 98 slow requests, 5 included below; oldest blocked for > 735.671988 secs 2024-03-28T00:47:56.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:47:55 smithi119 ceph-mon[31518]: slow request 731.576355 seconds old, received at 2024-03-28T00:35:43.454578+0000: internal op fragmentdir:mds.0:3797 currently failed to wrlock, waiting 2024-03-28T00:47:56.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:47:55 smithi119 ceph-mon[31518]: slow request 731.560327 seconds old, received at 2024-03-28T00:35:43.470605+0000: internal op fragmentdir:mds.0:3828 currently failed to wrlock, waiting 2024-03-28T00:47:56.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:47:55 smithi119 ceph-mon[31518]: slow request 731.546362 seconds old, received at 2024-03-28T00:35:43.484570+0000: internal op fragmentdir:mds.0:3860 currently failed to wrlock, waiting 2024-03-28T00:47:56.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:47:55 smithi119 ceph-mon[31518]: slow request 731.533156 seconds old, received at 2024-03-28T00:35:43.497776+0000: internal op fragmentdir:mds.0:3892 currently failed to wrlock, waiting 2024-03-28T00:47:56.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:47:55 smithi119 ceph-mon[31518]: slow request 731.575878 seconds old, received at 2024-03-28T00:35:43.455055+0000: internal op fragmentdir:mds.0:3798 currently failed to wrlock, waiting 2024-03-28T00:47:57.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:47:57 smithi132 ceph-mon[35941]: pgmap v694: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:47:58.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:47:57 smithi165 ceph-mon[36302]: pgmap v694: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:47:58.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:47:57 smithi119 ceph-mon[31518]: pgmap v694: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:47:59.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:47:59 smithi132 ceph-mon[35941]: pgmap v695: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:48:00.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:47:59 smithi165 ceph-mon[36302]: pgmap v695: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:48:00.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:47:59 smithi119 ceph-mon[31518]: pgmap v695: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:48:00.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:48:00 smithi132 ceph-mon[35941]: 98 slow requests, 5 included below; oldest blocked for > 740.672077 secs 2024-03-28T00:48:00.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:48:00 smithi132 ceph-mon[35941]: slow request 736.531904 seconds old, received at 2024-03-28T00:35:43.499118+0000: internal op fragmentdir:mds.0:3893 currently failed to wrlock, waiting 2024-03-28T00:48:00.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:48:00 smithi132 ceph-mon[35941]: slow request 736.575493 seconds old, received at 2024-03-28T00:35:43.455529+0000: internal op fragmentdir:mds.0:3799 currently failed to wrlock, waiting 2024-03-28T00:48:00.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:48:00 smithi132 ceph-mon[35941]: slow request 736.545624 seconds old, received at 2024-03-28T00:35:43.485397+0000: internal op fragmentdir:mds.0:3862 currently failed to wrlock, waiting 2024-03-28T00:48:00.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:48:00 smithi132 ceph-mon[35941]: slow request 736.531454 seconds old, received at 2024-03-28T00:35:43.499568+0000: internal op fragmentdir:mds.0:3894 currently failed to wrlock, waiting 2024-03-28T00:48:00.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:48:00 smithi132 ceph-mon[35941]: slow request 736.575014 seconds old, received at 2024-03-28T00:35:43.456007+0000: internal op fragmentdir:mds.0:3800 currently failed to wrlock, waiting 2024-03-28T00:48:01.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:48:00 smithi165 ceph-mon[36302]: 98 slow requests, 5 included below; oldest blocked for > 740.672077 secs 2024-03-28T00:48:01.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:48:00 smithi165 ceph-mon[36302]: slow request 736.531904 seconds old, received at 2024-03-28T00:35:43.499118+0000: internal op fragmentdir:mds.0:3893 currently failed to wrlock, waiting 2024-03-28T00:48:01.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:48:00 smithi165 ceph-mon[36302]: slow request 736.575493 seconds old, received at 2024-03-28T00:35:43.455529+0000: internal op fragmentdir:mds.0:3799 currently failed to wrlock, waiting 2024-03-28T00:48:01.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:48:00 smithi165 ceph-mon[36302]: slow request 736.545624 seconds old, received at 2024-03-28T00:35:43.485397+0000: internal op fragmentdir:mds.0:3862 currently failed to wrlock, waiting 2024-03-28T00:48:01.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:48:00 smithi165 ceph-mon[36302]: slow request 736.531454 seconds old, received at 2024-03-28T00:35:43.499568+0000: internal op fragmentdir:mds.0:3894 currently failed to wrlock, waiting 2024-03-28T00:48:01.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:48:00 smithi165 ceph-mon[36302]: slow request 736.575014 seconds old, received at 2024-03-28T00:35:43.456007+0000: internal op fragmentdir:mds.0:3800 currently failed to wrlock, waiting 2024-03-28T00:48:01.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:48:00 smithi119 ceph-mon[31518]: 98 slow requests, 5 included below; oldest blocked for > 740.672077 secs 2024-03-28T00:48:01.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:48:00 smithi119 ceph-mon[31518]: slow request 736.531904 seconds old, received at 2024-03-28T00:35:43.499118+0000: internal op fragmentdir:mds.0:3893 currently failed to wrlock, waiting 2024-03-28T00:48:01.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:48:00 smithi119 ceph-mon[31518]: slow request 736.575493 seconds old, received at 2024-03-28T00:35:43.455529+0000: internal op fragmentdir:mds.0:3799 currently failed to wrlock, waiting 2024-03-28T00:48:01.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:48:00 smithi119 ceph-mon[31518]: slow request 736.545624 seconds old, received at 2024-03-28T00:35:43.485397+0000: internal op fragmentdir:mds.0:3862 currently failed to wrlock, waiting 2024-03-28T00:48:01.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:48:00 smithi119 ceph-mon[31518]: slow request 736.531454 seconds old, received at 2024-03-28T00:35:43.499568+0000: internal op fragmentdir:mds.0:3894 currently failed to wrlock, waiting 2024-03-28T00:48:01.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:48:00 smithi119 ceph-mon[31518]: slow request 736.575014 seconds old, received at 2024-03-28T00:35:43.456007+0000: internal op fragmentdir:mds.0:3800 currently failed to wrlock, waiting 2024-03-28T00:48:02.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:48:01 smithi165 ceph-mon[36302]: pgmap v696: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:48:02.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:48:01 smithi119 ceph-mon[31518]: pgmap v696: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:48:02.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:48:01 smithi132 ceph-mon[35941]: pgmap v696: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:48:04.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:48:03 smithi165 ceph-mon[36302]: pgmap v697: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:48:04.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:48:03 smithi119 ceph-mon[31518]: pgmap v697: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:48:04.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:48:03 smithi132 ceph-mon[35941]: pgmap v697: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:48:06.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:48:05 smithi165 ceph-mon[36302]: pgmap v698: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:48:06.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:48:05 smithi165 ceph-mon[36302]: 98 slow requests, 5 included below; oldest blocked for > 745.672112 secs 2024-03-28T00:48:06.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:48:05 smithi165 ceph-mon[36302]: slow request 741.559385 seconds old, received at 2024-03-28T00:35:43.471672+0000: internal op fragmentdir:mds.0:3831 currently failed to wrlock, waiting 2024-03-28T00:48:06.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:48:05 smithi165 ceph-mon[36302]: slow request 741.545268 seconds old, received at 2024-03-28T00:35:43.485788+0000: internal op fragmentdir:mds.0:3863 currently failed to wrlock, waiting 2024-03-28T00:48:06.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:48:05 smithi165 ceph-mon[36302]: slow request 741.531025 seconds old, received at 2024-03-28T00:35:43.500032+0000: internal op fragmentdir:mds.0:3895 currently failed to wrlock, waiting 2024-03-28T00:48:06.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:48:05 smithi165 ceph-mon[36302]: slow request 741.574593 seconds old, received at 2024-03-28T00:35:43.456464+0000: internal op fragmentdir:mds.0:3801 currently failed to wrlock, waiting 2024-03-28T00:48:06.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:48:05 smithi165 ceph-mon[36302]: slow request 741.558926 seconds old, received at 2024-03-28T00:35:43.472131+0000: internal op fragmentdir:mds.0:3832 currently failed to wrlock, waiting 2024-03-28T00:48:06.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:48:05 smithi119 ceph-mon[31518]: pgmap v698: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:48:06.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:48:05 smithi119 ceph-mon[31518]: 98 slow requests, 5 included below; oldest blocked for > 745.672112 secs 2024-03-28T00:48:06.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:48:05 smithi119 ceph-mon[31518]: slow request 741.559385 seconds old, received at 2024-03-28T00:35:43.471672+0000: internal op fragmentdir:mds.0:3831 currently failed to wrlock, waiting 2024-03-28T00:48:06.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:48:05 smithi119 ceph-mon[31518]: slow request 741.545268 seconds old, received at 2024-03-28T00:35:43.485788+0000: internal op fragmentdir:mds.0:3863 currently failed to wrlock, waiting 2024-03-28T00:48:06.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:48:05 smithi119 ceph-mon[31518]: slow request 741.531025 seconds old, received at 2024-03-28T00:35:43.500032+0000: internal op fragmentdir:mds.0:3895 currently failed to wrlock, waiting 2024-03-28T00:48:06.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:48:05 smithi119 ceph-mon[31518]: slow request 741.574593 seconds old, received at 2024-03-28T00:35:43.456464+0000: internal op fragmentdir:mds.0:3801 currently failed to wrlock, waiting 2024-03-28T00:48:06.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:48:05 smithi119 ceph-mon[31518]: slow request 741.558926 seconds old, received at 2024-03-28T00:35:43.472131+0000: internal op fragmentdir:mds.0:3832 currently failed to wrlock, waiting 2024-03-28T00:48:06.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:48:05 smithi132 ceph-mon[35941]: pgmap v698: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:48:06.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:48:05 smithi132 ceph-mon[35941]: 98 slow requests, 5 included below; oldest blocked for > 745.672112 secs 2024-03-28T00:48:06.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:48:05 smithi132 ceph-mon[35941]: slow request 741.559385 seconds old, received at 2024-03-28T00:35:43.471672+0000: internal op fragmentdir:mds.0:3831 currently failed to wrlock, waiting 2024-03-28T00:48:06.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:48:05 smithi132 ceph-mon[35941]: slow request 741.545268 seconds old, received at 2024-03-28T00:35:43.485788+0000: internal op fragmentdir:mds.0:3863 currently failed to wrlock, waiting 2024-03-28T00:48:06.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:48:05 smithi132 ceph-mon[35941]: slow request 741.531025 seconds old, received at 2024-03-28T00:35:43.500032+0000: internal op fragmentdir:mds.0:3895 currently failed to wrlock, waiting 2024-03-28T00:48:06.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:48:05 smithi132 ceph-mon[35941]: slow request 741.574593 seconds old, received at 2024-03-28T00:35:43.456464+0000: internal op fragmentdir:mds.0:3801 currently failed to wrlock, waiting 2024-03-28T00:48:06.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:48:05 smithi132 ceph-mon[35941]: slow request 741.558926 seconds old, received at 2024-03-28T00:35:43.472131+0000: internal op fragmentdir:mds.0:3832 currently failed to wrlock, waiting 2024-03-28T00:48:08.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:48:07 smithi165 ceph-mon[36302]: pgmap v699: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:48:08.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:48:07 smithi119 ceph-mon[31518]: pgmap v699: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:48:08.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:48:07 smithi132 ceph-mon[35941]: pgmap v699: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:48:10.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:48:09 smithi165 ceph-mon[36302]: pgmap v700: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:48:10.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:48:09 smithi119 ceph-mon[31518]: pgmap v700: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:48:10.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:48:09 smithi132 ceph-mon[35941]: pgmap v700: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:48:11.102 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:48:10 smithi132 ceph-mon[35941]: 98 slow requests, 5 included below; oldest blocked for > 750.672175 secs 2024-03-28T00:48:11.102 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:48:10 smithi132 ceph-mon[35941]: slow request 746.574139 seconds old, received at 2024-03-28T00:35:43.456981+0000: internal op fragmentdir:mds.0:3802 currently failed to wrlock, waiting 2024-03-28T00:48:11.102 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:48:10 smithi132 ceph-mon[35941]: slow request 746.544530 seconds old, received at 2024-03-28T00:35:43.486590+0000: internal op fragmentdir:mds.0:3865 currently failed to wrlock, waiting 2024-03-28T00:48:11.102 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:48:10 smithi132 ceph-mon[35941]: slow request 746.530215 seconds old, received at 2024-03-28T00:35:43.500904+0000: internal op fragmentdir:mds.0:3897 currently failed to wrlock, waiting 2024-03-28T00:48:11.102 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:48:10 smithi132 ceph-mon[35941]: slow request 746.573690 seconds old, received at 2024-03-28T00:35:43.457430+0000: internal op fragmentdir:mds.0:3803 currently failed to wrlock, waiting 2024-03-28T00:48:11.102 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:48:10 smithi132 ceph-mon[35941]: slow request 746.558104 seconds old, received at 2024-03-28T00:35:43.473016+0000: internal op fragmentdir:mds.0:3834 currently failed to wrlock, waiting 2024-03-28T00:48:11.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:48:10 smithi165 ceph-mon[36302]: 98 slow requests, 5 included below; oldest blocked for > 750.672175 secs 2024-03-28T00:48:11.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:48:10 smithi165 ceph-mon[36302]: slow request 746.574139 seconds old, received at 2024-03-28T00:35:43.456981+0000: internal op fragmentdir:mds.0:3802 currently failed to wrlock, waiting 2024-03-28T00:48:11.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:48:10 smithi165 ceph-mon[36302]: slow request 746.544530 seconds old, received at 2024-03-28T00:35:43.486590+0000: internal op fragmentdir:mds.0:3865 currently failed to wrlock, waiting 2024-03-28T00:48:11.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:48:10 smithi165 ceph-mon[36302]: slow request 746.530215 seconds old, received at 2024-03-28T00:35:43.500904+0000: internal op fragmentdir:mds.0:3897 currently failed to wrlock, waiting 2024-03-28T00:48:11.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:48:10 smithi165 ceph-mon[36302]: slow request 746.573690 seconds old, received at 2024-03-28T00:35:43.457430+0000: internal op fragmentdir:mds.0:3803 currently failed to wrlock, waiting 2024-03-28T00:48:11.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:48:10 smithi165 ceph-mon[36302]: slow request 746.558104 seconds old, received at 2024-03-28T00:35:43.473016+0000: internal op fragmentdir:mds.0:3834 currently failed to wrlock, waiting 2024-03-28T00:48:11.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:48:10 smithi119 ceph-mon[31518]: 98 slow requests, 5 included below; oldest blocked for > 750.672175 secs 2024-03-28T00:48:11.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:48:10 smithi119 ceph-mon[31518]: slow request 746.574139 seconds old, received at 2024-03-28T00:35:43.456981+0000: internal op fragmentdir:mds.0:3802 currently failed to wrlock, waiting 2024-03-28T00:48:11.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:48:10 smithi119 ceph-mon[31518]: slow request 746.544530 seconds old, received at 2024-03-28T00:35:43.486590+0000: internal op fragmentdir:mds.0:3865 currently failed to wrlock, waiting 2024-03-28T00:48:11.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:48:10 smithi119 ceph-mon[31518]: slow request 746.530215 seconds old, received at 2024-03-28T00:35:43.500904+0000: internal op fragmentdir:mds.0:3897 currently failed to wrlock, waiting 2024-03-28T00:48:11.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:48:10 smithi119 ceph-mon[31518]: slow request 746.573690 seconds old, received at 2024-03-28T00:35:43.457430+0000: internal op fragmentdir:mds.0:3803 currently failed to wrlock, waiting 2024-03-28T00:48:11.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:48:10 smithi119 ceph-mon[31518]: slow request 746.558104 seconds old, received at 2024-03-28T00:35:43.473016+0000: internal op fragmentdir:mds.0:3834 currently failed to wrlock, waiting 2024-03-28T00:48:12.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:48:11 smithi165 ceph-mon[36302]: pgmap v701: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:48:12.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:48:11 smithi119 ceph-mon[31518]: pgmap v701: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:48:12.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:48:11 smithi132 ceph-mon[35941]: pgmap v701: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:48:14.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:48:13 smithi165 ceph-mon[36302]: pgmap v702: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:48:14.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:48:13 smithi119 ceph-mon[31518]: pgmap v702: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:48:14.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:48:13 smithi132 ceph-mon[35941]: pgmap v702: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:48:16.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:48:15 smithi165 ceph-mon[36302]: pgmap v703: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:48:16.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:48:15 smithi165 ceph-mon[36302]: 98 slow requests, 5 included below; oldest blocked for > 755.672262 secs 2024-03-28T00:48:16.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:48:15 smithi165 ceph-mon[36302]: slow request 751.573271 seconds old, received at 2024-03-28T00:35:43.457937+0000: internal op fragmentdir:mds.0:3804 currently failed to wrlock, waiting 2024-03-28T00:48:16.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:48:15 smithi165 ceph-mon[36302]: slow request 751.543822 seconds old, received at 2024-03-28T00:35:43.487385+0000: internal op fragmentdir:mds.0:3867 currently failed to wrlock, waiting 2024-03-28T00:48:16.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:48:15 smithi165 ceph-mon[36302]: slow request 751.529436 seconds old, received at 2024-03-28T00:35:43.501771+0000: internal op fragmentdir:mds.0:3899 currently failed to wrlock, waiting 2024-03-28T00:48:16.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:48:15 smithi165 ceph-mon[36302]: slow request 751.572781 seconds old, received at 2024-03-28T00:35:43.458426+0000: internal op fragmentdir:mds.0:3805 currently failed to wrlock, waiting 2024-03-28T00:48:16.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:48:15 smithi165 ceph-mon[36302]: slow request 751.557305 seconds old, received at 2024-03-28T00:35:43.473902+0000: internal op fragmentdir:mds.0:3836 currently failed to wrlock, waiting 2024-03-28T00:48:16.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:48:15 smithi119 ceph-mon[31518]: pgmap v703: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:48:16.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:48:15 smithi119 ceph-mon[31518]: 98 slow requests, 5 included below; oldest blocked for > 755.672262 secs 2024-03-28T00:48:16.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:48:15 smithi119 ceph-mon[31518]: slow request 751.573271 seconds old, received at 2024-03-28T00:35:43.457937+0000: internal op fragmentdir:mds.0:3804 currently failed to wrlock, waiting 2024-03-28T00:48:16.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:48:15 smithi119 ceph-mon[31518]: slow request 751.543822 seconds old, received at 2024-03-28T00:35:43.487385+0000: internal op fragmentdir:mds.0:3867 currently failed to wrlock, waiting 2024-03-28T00:48:16.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:48:15 smithi119 ceph-mon[31518]: slow request 751.529436 seconds old, received at 2024-03-28T00:35:43.501771+0000: internal op fragmentdir:mds.0:3899 currently failed to wrlock, waiting 2024-03-28T00:48:16.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:48:15 smithi119 ceph-mon[31518]: slow request 751.572781 seconds old, received at 2024-03-28T00:35:43.458426+0000: internal op fragmentdir:mds.0:3805 currently failed to wrlock, waiting 2024-03-28T00:48:16.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:48:15 smithi119 ceph-mon[31518]: slow request 751.557305 seconds old, received at 2024-03-28T00:35:43.473902+0000: internal op fragmentdir:mds.0:3836 currently failed to wrlock, waiting 2024-03-28T00:48:16.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:48:15 smithi132 ceph-mon[35941]: pgmap v703: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:48:16.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:48:15 smithi132 ceph-mon[35941]: 98 slow requests, 5 included below; oldest blocked for > 755.672262 secs 2024-03-28T00:48:16.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:48:15 smithi132 ceph-mon[35941]: slow request 751.573271 seconds old, received at 2024-03-28T00:35:43.457937+0000: internal op fragmentdir:mds.0:3804 currently failed to wrlock, waiting 2024-03-28T00:48:16.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:48:15 smithi132 ceph-mon[35941]: slow request 751.543822 seconds old, received at 2024-03-28T00:35:43.487385+0000: internal op fragmentdir:mds.0:3867 currently failed to wrlock, waiting 2024-03-28T00:48:16.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:48:15 smithi132 ceph-mon[35941]: slow request 751.529436 seconds old, received at 2024-03-28T00:35:43.501771+0000: internal op fragmentdir:mds.0:3899 currently failed to wrlock, waiting 2024-03-28T00:48:16.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:48:15 smithi132 ceph-mon[35941]: slow request 751.572781 seconds old, received at 2024-03-28T00:35:43.458426+0000: internal op fragmentdir:mds.0:3805 currently failed to wrlock, waiting 2024-03-28T00:48:16.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:48:15 smithi132 ceph-mon[35941]: slow request 751.557305 seconds old, received at 2024-03-28T00:35:43.473902+0000: internal op fragmentdir:mds.0:3836 currently failed to wrlock, waiting 2024-03-28T00:48:18.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:48:17 smithi165 ceph-mon[36302]: pgmap v704: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:48:18.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:48:17 smithi119 ceph-mon[31518]: pgmap v704: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:48:18.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:48:17 smithi132 ceph-mon[35941]: pgmap v704: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:48:20.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:48:19 smithi165 ceph-mon[36302]: pgmap v705: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:48:20.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:48:19 smithi119 ceph-mon[31518]: pgmap v705: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:48:20.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:48:19 smithi132 ceph-mon[35941]: pgmap v705: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:48:21.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:48:20 smithi165 ceph-mon[36302]: 98 slow requests, 5 included below; oldest blocked for > 760.672353 secs 2024-03-28T00:48:21.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:48:20 smithi165 ceph-mon[36302]: slow request 756.543516 seconds old, received at 2024-03-28T00:35:43.487781+0000: internal op fragmentdir:mds.0:3868 currently failed to wrlock, waiting 2024-03-28T00:48:21.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:48:20 smithi165 ceph-mon[36302]: slow request 756.572385 seconds old, received at 2024-03-28T00:35:43.458913+0000: internal op fragmentdir:mds.0:3806 currently failed to wrlock, waiting 2024-03-28T00:48:21.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:48:20 smithi165 ceph-mon[36302]: slow request 756.556959 seconds old, received at 2024-03-28T00:35:43.474338+0000: internal op fragmentdir:mds.0:3837 currently failed to wrlock, waiting 2024-03-28T00:48:21.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:48:20 smithi165 ceph-mon[36302]: slow request 756.528643 seconds old, received at 2024-03-28T00:35:43.502655+0000: internal op fragmentdir:mds.0:3901 currently failed to wrlock, waiting 2024-03-28T00:48:21.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:48:20 smithi165 ceph-mon[36302]: slow request 756.573031 seconds old, received at 2024-03-28T00:35:43.458267+0000: client_request(client.25264:34113 create owner_uid=1000, owner_gid=1301 #0x10000000272/file7594 2024-03-28T00:35:43.457305+0000 caller_uid=1000, caller_gid=1301{6,36,1000,1301,}) currently failed to xlock, waiting 2024-03-28T00:48:21.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:48:20 smithi119 ceph-mon[31518]: 98 slow requests, 5 included below; oldest blocked for > 760.672353 secs 2024-03-28T00:48:21.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:48:20 smithi119 ceph-mon[31518]: slow request 756.543516 seconds old, received at 2024-03-28T00:35:43.487781+0000: internal op fragmentdir:mds.0:3868 currently failed to wrlock, waiting 2024-03-28T00:48:21.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:48:20 smithi119 ceph-mon[31518]: slow request 756.572385 seconds old, received at 2024-03-28T00:35:43.458913+0000: internal op fragmentdir:mds.0:3806 currently failed to wrlock, waiting 2024-03-28T00:48:21.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:48:20 smithi119 ceph-mon[31518]: slow request 756.556959 seconds old, received at 2024-03-28T00:35:43.474338+0000: internal op fragmentdir:mds.0:3837 currently failed to wrlock, waiting 2024-03-28T00:48:21.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:48:20 smithi119 ceph-mon[31518]: slow request 756.528643 seconds old, received at 2024-03-28T00:35:43.502655+0000: internal op fragmentdir:mds.0:3901 currently failed to wrlock, waiting 2024-03-28T00:48:21.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:48:20 smithi119 ceph-mon[31518]: slow request 756.573031 seconds old, received at 2024-03-28T00:35:43.458267+0000: client_request(client.25264:34113 create owner_uid=1000, owner_gid=1301 #0x10000000272/file7594 2024-03-28T00:35:43.457305+0000 caller_uid=1000, caller_gid=1301{6,36,1000,1301,}) currently failed to xlock, waiting 2024-03-28T00:48:21.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:48:20 smithi132 ceph-mon[35941]: 98 slow requests, 5 included below; oldest blocked for > 760.672353 secs 2024-03-28T00:48:21.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:48:20 smithi132 ceph-mon[35941]: slow request 756.543516 seconds old, received at 2024-03-28T00:35:43.487781+0000: internal op fragmentdir:mds.0:3868 currently failed to wrlock, waiting 2024-03-28T00:48:21.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:48:20 smithi132 ceph-mon[35941]: slow request 756.572385 seconds old, received at 2024-03-28T00:35:43.458913+0000: internal op fragmentdir:mds.0:3806 currently failed to wrlock, waiting 2024-03-28T00:48:21.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:48:20 smithi132 ceph-mon[35941]: slow request 756.556959 seconds old, received at 2024-03-28T00:35:43.474338+0000: internal op fragmentdir:mds.0:3837 currently failed to wrlock, waiting 2024-03-28T00:48:21.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:48:20 smithi132 ceph-mon[35941]: slow request 756.528643 seconds old, received at 2024-03-28T00:35:43.502655+0000: internal op fragmentdir:mds.0:3901 currently failed to wrlock, waiting 2024-03-28T00:48:21.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:48:20 smithi132 ceph-mon[35941]: slow request 756.573031 seconds old, received at 2024-03-28T00:35:43.458267+0000: client_request(client.25264:34113 create owner_uid=1000, owner_gid=1301 #0x10000000272/file7594 2024-03-28T00:35:43.457305+0000 caller_uid=1000, caller_gid=1301{6,36,1000,1301,}) currently failed to xlock, waiting 2024-03-28T00:48:22.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:48:21 smithi165 ceph-mon[36302]: pgmap v706: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:48:22.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:48:21 smithi119 ceph-mon[31518]: pgmap v706: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:48:22.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:48:21 smithi132 ceph-mon[35941]: pgmap v706: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:48:24.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:48:23 smithi165 ceph-mon[36302]: pgmap v707: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:48:24.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:48:23 smithi119 ceph-mon[31518]: pgmap v707: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:48:24.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:48:23 smithi132 ceph-mon[35941]: pgmap v707: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:48:26.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:48:25 smithi165 ceph-mon[36302]: pgmap v708: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:48:26.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:48:25 smithi165 ceph-mon[36302]: 98 slow requests, 5 included below; oldest blocked for > 765.672442 secs 2024-03-28T00:48:26.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:48:25 smithi165 ceph-mon[36302]: slow request 761.543081 seconds old, received at 2024-03-28T00:35:43.488305+0000: internal op fragmentdir:mds.0:3870 currently failed to wrlock, waiting 2024-03-28T00:48:26.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:48:25 smithi165 ceph-mon[36302]: slow request 761.569539 seconds old, received at 2024-03-28T00:35:43.461848+0000: internal op fragmentdir:mds.0:3807 currently failed to wrlock, waiting 2024-03-28T00:48:26.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:48:25 smithi165 ceph-mon[36302]: slow request 761.556191 seconds old, received at 2024-03-28T00:35:43.475195+0000: internal op fragmentdir:mds.0:3839 currently failed to wrlock, waiting 2024-03-28T00:48:26.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:48:25 smithi165 ceph-mon[36302]: slow request 761.542661 seconds old, received at 2024-03-28T00:35:43.488725+0000: internal op fragmentdir:mds.0:3871 currently failed to wrlock, waiting 2024-03-28T00:48:26.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:48:25 smithi165 ceph-mon[36302]: slow request 761.528079 seconds old, received at 2024-03-28T00:35:43.503308+0000: internal op fragmentdir:mds.0:3903 currently failed to wrlock, waiting 2024-03-28T00:48:26.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:48:25 smithi119 ceph-mon[31518]: pgmap v708: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:48:26.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:48:25 smithi119 ceph-mon[31518]: 98 slow requests, 5 included below; oldest blocked for > 765.672442 secs 2024-03-28T00:48:26.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:48:25 smithi119 ceph-mon[31518]: slow request 761.543081 seconds old, received at 2024-03-28T00:35:43.488305+0000: internal op fragmentdir:mds.0:3870 currently failed to wrlock, waiting 2024-03-28T00:48:26.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:48:25 smithi119 ceph-mon[31518]: slow request 761.569539 seconds old, received at 2024-03-28T00:35:43.461848+0000: internal op fragmentdir:mds.0:3807 currently failed to wrlock, waiting 2024-03-28T00:48:26.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:48:25 smithi119 ceph-mon[31518]: slow request 761.556191 seconds old, received at 2024-03-28T00:35:43.475195+0000: internal op fragmentdir:mds.0:3839 currently failed to wrlock, waiting 2024-03-28T00:48:26.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:48:25 smithi119 ceph-mon[31518]: slow request 761.542661 seconds old, received at 2024-03-28T00:35:43.488725+0000: internal op fragmentdir:mds.0:3871 currently failed to wrlock, waiting 2024-03-28T00:48:26.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:48:25 smithi119 ceph-mon[31518]: slow request 761.528079 seconds old, received at 2024-03-28T00:35:43.503308+0000: internal op fragmentdir:mds.0:3903 currently failed to wrlock, waiting 2024-03-28T00:48:26.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:48:25 smithi132 ceph-mon[35941]: pgmap v708: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:48:26.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:48:25 smithi132 ceph-mon[35941]: 98 slow requests, 5 included below; oldest blocked for > 765.672442 secs 2024-03-28T00:48:26.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:48:25 smithi132 ceph-mon[35941]: slow request 761.543081 seconds old, received at 2024-03-28T00:35:43.488305+0000: internal op fragmentdir:mds.0:3870 currently failed to wrlock, waiting 2024-03-28T00:48:26.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:48:25 smithi132 ceph-mon[35941]: slow request 761.569539 seconds old, received at 2024-03-28T00:35:43.461848+0000: internal op fragmentdir:mds.0:3807 currently failed to wrlock, waiting 2024-03-28T00:48:26.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:48:25 smithi132 ceph-mon[35941]: slow request 761.556191 seconds old, received at 2024-03-28T00:35:43.475195+0000: internal op fragmentdir:mds.0:3839 currently failed to wrlock, waiting 2024-03-28T00:48:26.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:48:25 smithi132 ceph-mon[35941]: slow request 761.542661 seconds old, received at 2024-03-28T00:35:43.488725+0000: internal op fragmentdir:mds.0:3871 currently failed to wrlock, waiting 2024-03-28T00:48:26.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:48:25 smithi132 ceph-mon[35941]: slow request 761.528079 seconds old, received at 2024-03-28T00:35:43.503308+0000: internal op fragmentdir:mds.0:3903 currently failed to wrlock, waiting 2024-03-28T00:48:28.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:48:27 smithi165 ceph-mon[36302]: pgmap v709: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:48:28.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:48:27 smithi119 ceph-mon[31518]: pgmap v709: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:48:28.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:48:27 smithi132 ceph-mon[35941]: pgmap v709: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:48:30.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:48:29 smithi165 ceph-mon[36302]: pgmap v710: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:48:30.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:48:29 smithi119 ceph-mon[31518]: pgmap v710: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:48:30.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:48:29 smithi132 ceph-mon[35941]: pgmap v710: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:48:31.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:48:30 smithi165 ceph-mon[36302]: 98 slow requests, 5 included below; oldest blocked for > 770.672504 secs 2024-03-28T00:48:31.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:48:30 smithi165 ceph-mon[36302]: slow request 749.654616 seconds old, received at 2024-03-28T00:36:00.376832+0000: client_request(client.25355:96 mksnap #0x1//scheduled-2024-03-28-00_36_00_UTC 2024-03-28T00:36:00.376777+0000 caller_uid=0, caller_gid=0{}) currently failed to acquire_locks 2024-03-28T00:48:31.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:48:30 smithi165 ceph-mon[36302]: slow request 766.569146 seconds old, received at 2024-03-28T00:35:43.462302+0000: internal op fragmentdir:mds.0:3808 currently failed to wrlock, waiting 2024-03-28T00:48:31.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:48:30 smithi165 ceph-mon[36302]: slow request 766.632753 seconds old, received at 2024-03-28T00:35:43.398696+0000: internal op fragmentdir:mds.0:3785 currently failed to wrlock, waiting 2024-03-28T00:48:31.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:48:30 smithi165 ceph-mon[36302]: slow request 766.568684 seconds old, received at 2024-03-28T00:35:43.462764+0000: internal op fragmentdir:mds.0:3809 currently failed to wrlock, waiting 2024-03-28T00:48:31.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:48:30 smithi165 ceph-mon[36302]: slow request 766.555354 seconds old, received at 2024-03-28T00:35:43.476094+0000: internal op fragmentdir:mds.0:3841 currently failed to wrlock, waiting 2024-03-28T00:48:31.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:48:30 smithi119 ceph-mon[31518]: 98 slow requests, 5 included below; oldest blocked for > 770.672504 secs 2024-03-28T00:48:31.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:48:30 smithi119 ceph-mon[31518]: slow request 749.654616 seconds old, received at 2024-03-28T00:36:00.376832+0000: client_request(client.25355:96 mksnap #0x1//scheduled-2024-03-28-00_36_00_UTC 2024-03-28T00:36:00.376777+0000 caller_uid=0, caller_gid=0{}) currently failed to acquire_locks 2024-03-28T00:48:31.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:48:30 smithi119 ceph-mon[31518]: slow request 766.569146 seconds old, received at 2024-03-28T00:35:43.462302+0000: internal op fragmentdir:mds.0:3808 currently failed to wrlock, waiting 2024-03-28T00:48:31.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:48:30 smithi119 ceph-mon[31518]: slow request 766.632753 seconds old, received at 2024-03-28T00:35:43.398696+0000: internal op fragmentdir:mds.0:3785 currently failed to wrlock, waiting 2024-03-28T00:48:31.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:48:30 smithi119 ceph-mon[31518]: slow request 766.568684 seconds old, received at 2024-03-28T00:35:43.462764+0000: internal op fragmentdir:mds.0:3809 currently failed to wrlock, waiting 2024-03-28T00:48:31.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:48:30 smithi119 ceph-mon[31518]: slow request 766.555354 seconds old, received at 2024-03-28T00:35:43.476094+0000: internal op fragmentdir:mds.0:3841 currently failed to wrlock, waiting 2024-03-28T00:48:31.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:48:30 smithi132 ceph-mon[35941]: 98 slow requests, 5 included below; oldest blocked for > 770.672504 secs 2024-03-28T00:48:31.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:48:30 smithi132 ceph-mon[35941]: slow request 749.654616 seconds old, received at 2024-03-28T00:36:00.376832+0000: client_request(client.25355:96 mksnap #0x1//scheduled-2024-03-28-00_36_00_UTC 2024-03-28T00:36:00.376777+0000 caller_uid=0, caller_gid=0{}) currently failed to acquire_locks 2024-03-28T00:48:31.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:48:30 smithi132 ceph-mon[35941]: slow request 766.569146 seconds old, received at 2024-03-28T00:35:43.462302+0000: internal op fragmentdir:mds.0:3808 currently failed to wrlock, waiting 2024-03-28T00:48:31.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:48:30 smithi132 ceph-mon[35941]: slow request 766.632753 seconds old, received at 2024-03-28T00:35:43.398696+0000: internal op fragmentdir:mds.0:3785 currently failed to wrlock, waiting 2024-03-28T00:48:31.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:48:30 smithi132 ceph-mon[35941]: slow request 766.568684 seconds old, received at 2024-03-28T00:35:43.462764+0000: internal op fragmentdir:mds.0:3809 currently failed to wrlock, waiting 2024-03-28T00:48:31.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:48:30 smithi132 ceph-mon[35941]: slow request 766.555354 seconds old, received at 2024-03-28T00:35:43.476094+0000: internal op fragmentdir:mds.0:3841 currently failed to wrlock, waiting 2024-03-28T00:48:32.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:48:31 smithi165 ceph-mon[36302]: pgmap v711: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:48:32.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:48:31 smithi119 ceph-mon[31518]: pgmap v711: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:48:32.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:48:31 smithi132 ceph-mon[35941]: pgmap v711: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:48:34.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:48:33 smithi165 ceph-mon[36302]: pgmap v712: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:48:34.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:48:33 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:48:34.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:48:33 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:48:34.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:48:33 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:48:34.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:48:33 smithi119 ceph-mon[31518]: pgmap v712: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:48:34.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:48:33 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:48:34.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:48:33 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:48:34.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:48:33 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:48:34.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:48:33 smithi132 ceph-mon[35941]: pgmap v712: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:48:34.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:48:33 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:48:34.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:48:33 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:48:34.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:48:33 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:48:35.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:48:34 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:48:35.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:48:34 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:48:35.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:48:34 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:48:35.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:48:34 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:48:35.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:48:34 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:48:35.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:48:34 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:48:35.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:48:34 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:48:35.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:48:34 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:48:35.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:48:34 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:48:36.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:48:35 smithi165 ceph-mon[36302]: pgmap v713: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T00:48:36.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:48:35 smithi165 ceph-mon[36302]: pgmap v714: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 5 op/s 2024-03-28T00:48:36.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:48:35 smithi165 ceph-mon[36302]: 98 slow requests, 5 included below; oldest blocked for > 775.672590 secs 2024-03-28T00:48:36.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:48:35 smithi165 ceph-mon[36302]: slow request 771.541884 seconds old, received at 2024-03-28T00:35:43.489651+0000: internal op fragmentdir:mds.0:3873 currently failed to wrlock, waiting 2024-03-28T00:48:36.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:48:35 smithi165 ceph-mon[36302]: slow request 770.703914 seconds old, received at 2024-03-28T00:35:44.327621+0000: internal op fragmentdir:mds.0:3937 currently failed to wrlock, waiting 2024-03-28T00:48:36.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:48:35 smithi165 ceph-mon[36302]: slow request 771.568335 seconds old, received at 2024-03-28T00:35:43.463200+0000: internal op fragmentdir:mds.0:3810 currently failed to wrlock, waiting 2024-03-28T00:48:36.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:48:35 smithi165 ceph-mon[36302]: slow request 771.555004 seconds old, received at 2024-03-28T00:35:43.476531+0000: internal op fragmentdir:mds.0:3842 currently failed to wrlock, waiting 2024-03-28T00:48:36.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:48:35 smithi165 ceph-mon[36302]: slow request 771.567934 seconds old, received at 2024-03-28T00:35:43.463601+0000: internal op fragmentdir:mds.0:3811 currently failed to wrlock, waiting 2024-03-28T00:48:36.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:48:35 smithi119 ceph-mon[31518]: pgmap v713: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T00:48:36.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:48:35 smithi119 ceph-mon[31518]: pgmap v714: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 5 op/s 2024-03-28T00:48:36.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:48:35 smithi119 ceph-mon[31518]: 98 slow requests, 5 included below; oldest blocked for > 775.672590 secs 2024-03-28T00:48:36.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:48:35 smithi119 ceph-mon[31518]: slow request 771.541884 seconds old, received at 2024-03-28T00:35:43.489651+0000: internal op fragmentdir:mds.0:3873 currently failed to wrlock, waiting 2024-03-28T00:48:36.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:48:35 smithi119 ceph-mon[31518]: slow request 770.703914 seconds old, received at 2024-03-28T00:35:44.327621+0000: internal op fragmentdir:mds.0:3937 currently failed to wrlock, waiting 2024-03-28T00:48:36.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:48:35 smithi119 ceph-mon[31518]: slow request 771.568335 seconds old, received at 2024-03-28T00:35:43.463200+0000: internal op fragmentdir:mds.0:3810 currently failed to wrlock, waiting 2024-03-28T00:48:36.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:48:35 smithi119 ceph-mon[31518]: slow request 771.555004 seconds old, received at 2024-03-28T00:35:43.476531+0000: internal op fragmentdir:mds.0:3842 currently failed to wrlock, waiting 2024-03-28T00:48:36.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:48:35 smithi119 ceph-mon[31518]: slow request 771.567934 seconds old, received at 2024-03-28T00:35:43.463601+0000: internal op fragmentdir:mds.0:3811 currently failed to wrlock, waiting 2024-03-28T00:48:36.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:48:35 smithi132 ceph-mon[35941]: pgmap v713: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T00:48:36.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:48:35 smithi132 ceph-mon[35941]: pgmap v714: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 5 op/s 2024-03-28T00:48:36.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:48:35 smithi132 ceph-mon[35941]: 98 slow requests, 5 included below; oldest blocked for > 775.672590 secs 2024-03-28T00:48:36.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:48:35 smithi132 ceph-mon[35941]: slow request 771.541884 seconds old, received at 2024-03-28T00:35:43.489651+0000: internal op fragmentdir:mds.0:3873 currently failed to wrlock, waiting 2024-03-28T00:48:36.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:48:35 smithi132 ceph-mon[35941]: slow request 770.703914 seconds old, received at 2024-03-28T00:35:44.327621+0000: internal op fragmentdir:mds.0:3937 currently failed to wrlock, waiting 2024-03-28T00:48:36.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:48:35 smithi132 ceph-mon[35941]: slow request 771.568335 seconds old, received at 2024-03-28T00:35:43.463200+0000: internal op fragmentdir:mds.0:3810 currently failed to wrlock, waiting 2024-03-28T00:48:36.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:48:35 smithi132 ceph-mon[35941]: slow request 771.555004 seconds old, received at 2024-03-28T00:35:43.476531+0000: internal op fragmentdir:mds.0:3842 currently failed to wrlock, waiting 2024-03-28T00:48:36.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:48:35 smithi132 ceph-mon[35941]: slow request 771.567934 seconds old, received at 2024-03-28T00:35:43.463601+0000: internal op fragmentdir:mds.0:3811 currently failed to wrlock, waiting 2024-03-28T00:48:38.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:48:37 smithi165 ceph-mon[36302]: pgmap v715: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T00:48:38.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:48:37 smithi119 ceph-mon[31518]: pgmap v715: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T00:48:38.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:48:37 smithi132 ceph-mon[35941]: pgmap v715: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T00:48:40.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:48:39 smithi165 ceph-mon[36302]: pgmap v716: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T00:48:40.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:48:39 smithi119 ceph-mon[31518]: pgmap v716: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T00:48:40.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:48:39 smithi132 ceph-mon[35941]: pgmap v716: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T00:48:41.101 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:48:40 smithi132 ceph-mon[35941]: 98 slow requests, 5 included below; oldest blocked for > 780.672651 secs 2024-03-28T00:48:41.101 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:48:40 smithi132 ceph-mon[35941]: slow request 776.525559 seconds old, received at 2024-03-28T00:35:43.506038+0000: internal op fragmentdir:mds.0:3907 currently failed to wrlock, waiting 2024-03-28T00:48:41.101 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:48:40 smithi132 ceph-mon[35941]: slow request 776.567560 seconds old, received at 2024-03-28T00:35:43.464036+0000: internal op fragmentdir:mds.0:3812 currently failed to wrlock, waiting 2024-03-28T00:48:41.101 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:48:40 smithi132 ceph-mon[35941]: slow request 776.554182 seconds old, received at 2024-03-28T00:35:43.477414+0000: internal op fragmentdir:mds.0:3844 currently failed to wrlock, waiting 2024-03-28T00:48:41.101 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:48:40 smithi132 ceph-mon[35941]: slow request 776.523831 seconds old, received at 2024-03-28T00:35:43.507766+0000: internal op fragmentdir:mds.0:3908 currently failed to wrlock, waiting 2024-03-28T00:48:41.101 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:48:40 smithi132 ceph-mon[35941]: slow request 777.892231 seconds old, received at 2024-03-28T00:35:42.139365+0000: internal op fragmentdir:mds.0:3784 currently failed to wrlock, waiting 2024-03-28T00:48:41.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:48:40 smithi165 ceph-mon[36302]: 98 slow requests, 5 included below; oldest blocked for > 780.672651 secs 2024-03-28T00:48:41.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:48:40 smithi165 ceph-mon[36302]: slow request 776.525559 seconds old, received at 2024-03-28T00:35:43.506038+0000: internal op fragmentdir:mds.0:3907 currently failed to wrlock, waiting 2024-03-28T00:48:41.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:48:40 smithi165 ceph-mon[36302]: slow request 776.567560 seconds old, received at 2024-03-28T00:35:43.464036+0000: internal op fragmentdir:mds.0:3812 currently failed to wrlock, waiting 2024-03-28T00:48:41.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:48:40 smithi165 ceph-mon[36302]: slow request 776.554182 seconds old, received at 2024-03-28T00:35:43.477414+0000: internal op fragmentdir:mds.0:3844 currently failed to wrlock, waiting 2024-03-28T00:48:41.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:48:40 smithi165 ceph-mon[36302]: slow request 776.523831 seconds old, received at 2024-03-28T00:35:43.507766+0000: internal op fragmentdir:mds.0:3908 currently failed to wrlock, waiting 2024-03-28T00:48:41.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:48:40 smithi165 ceph-mon[36302]: slow request 777.892231 seconds old, received at 2024-03-28T00:35:42.139365+0000: internal op fragmentdir:mds.0:3784 currently failed to wrlock, waiting 2024-03-28T00:48:41.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:48:40 smithi119 ceph-mon[31518]: 98 slow requests, 5 included below; oldest blocked for > 780.672651 secs 2024-03-28T00:48:41.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:48:40 smithi119 ceph-mon[31518]: slow request 776.525559 seconds old, received at 2024-03-28T00:35:43.506038+0000: internal op fragmentdir:mds.0:3907 currently failed to wrlock, waiting 2024-03-28T00:48:41.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:48:40 smithi119 ceph-mon[31518]: slow request 776.567560 seconds old, received at 2024-03-28T00:35:43.464036+0000: internal op fragmentdir:mds.0:3812 currently failed to wrlock, waiting 2024-03-28T00:48:41.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:48:40 smithi119 ceph-mon[31518]: slow request 776.554182 seconds old, received at 2024-03-28T00:35:43.477414+0000: internal op fragmentdir:mds.0:3844 currently failed to wrlock, waiting 2024-03-28T00:48:41.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:48:40 smithi119 ceph-mon[31518]: slow request 776.523831 seconds old, received at 2024-03-28T00:35:43.507766+0000: internal op fragmentdir:mds.0:3908 currently failed to wrlock, waiting 2024-03-28T00:48:41.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:48:40 smithi119 ceph-mon[31518]: slow request 777.892231 seconds old, received at 2024-03-28T00:35:42.139365+0000: internal op fragmentdir:mds.0:3784 currently failed to wrlock, waiting 2024-03-28T00:48:42.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:48:41 smithi165 ceph-mon[36302]: pgmap v717: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T00:48:42.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:48:41 smithi119 ceph-mon[31518]: pgmap v717: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T00:48:42.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:48:41 smithi132 ceph-mon[35941]: pgmap v717: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T00:48:44.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:48:43 smithi165 ceph-mon[36302]: pgmap v718: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T00:48:44.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:48:43 smithi119 ceph-mon[31518]: pgmap v718: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T00:48:44.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:48:43 smithi132 ceph-mon[35941]: pgmap v718: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T00:48:46.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:48:45 smithi165 ceph-mon[36302]: pgmap v719: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T00:48:46.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:48:45 smithi165 ceph-mon[36302]: 98 slow requests, 5 included below; oldest blocked for > 785.672713 secs 2024-03-28T00:48:46.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:48:45 smithi165 ceph-mon[36302]: slow request 781.567226 seconds old, received at 2024-03-28T00:35:43.464432+0000: internal op fragmentdir:mds.0:3813 currently failed to wrlock, waiting 2024-03-28T00:48:46.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:48:45 smithi165 ceph-mon[36302]: slow request 781.553800 seconds old, received at 2024-03-28T00:35:43.477858+0000: internal op fragmentdir:mds.0:3845 currently failed to wrlock, waiting 2024-03-28T00:48:46.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:48:45 smithi165 ceph-mon[36302]: slow request 781.540110 seconds old, received at 2024-03-28T00:35:43.491547+0000: internal op fragmentdir:mds.0:3877 currently failed to wrlock, waiting 2024-03-28T00:48:46.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:48:45 smithi165 ceph-mon[36302]: slow request 781.566787 seconds old, received at 2024-03-28T00:35:43.464871+0000: internal op fragmentdir:mds.0:3814 currently failed to wrlock, waiting 2024-03-28T00:48:46.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:48:45 smithi165 ceph-mon[36302]: slow request 781.521834 seconds old, received at 2024-03-28T00:35:43.509824+0000: internal op fragmentdir:mds.0:3910 currently failed to wrlock, waiting 2024-03-28T00:48:46.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:48:45 smithi119 ceph-mon[31518]: pgmap v719: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T00:48:46.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:48:45 smithi119 ceph-mon[31518]: 98 slow requests, 5 included below; oldest blocked for > 785.672713 secs 2024-03-28T00:48:46.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:48:45 smithi119 ceph-mon[31518]: slow request 781.567226 seconds old, received at 2024-03-28T00:35:43.464432+0000: internal op fragmentdir:mds.0:3813 currently failed to wrlock, waiting 2024-03-28T00:48:46.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:48:45 smithi119 ceph-mon[31518]: slow request 781.553800 seconds old, received at 2024-03-28T00:35:43.477858+0000: internal op fragmentdir:mds.0:3845 currently failed to wrlock, waiting 2024-03-28T00:48:46.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:48:45 smithi119 ceph-mon[31518]: slow request 781.540110 seconds old, received at 2024-03-28T00:35:43.491547+0000: internal op fragmentdir:mds.0:3877 currently failed to wrlock, waiting 2024-03-28T00:48:46.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:48:45 smithi119 ceph-mon[31518]: slow request 781.566787 seconds old, received at 2024-03-28T00:35:43.464871+0000: internal op fragmentdir:mds.0:3814 currently failed to wrlock, waiting 2024-03-28T00:48:46.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:48:45 smithi119 ceph-mon[31518]: slow request 781.521834 seconds old, received at 2024-03-28T00:35:43.509824+0000: internal op fragmentdir:mds.0:3910 currently failed to wrlock, waiting 2024-03-28T00:48:46.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:48:45 smithi132 ceph-mon[35941]: pgmap v719: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T00:48:46.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:48:45 smithi132 ceph-mon[35941]: 98 slow requests, 5 included below; oldest blocked for > 785.672713 secs 2024-03-28T00:48:46.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:48:45 smithi132 ceph-mon[35941]: slow request 781.567226 seconds old, received at 2024-03-28T00:35:43.464432+0000: internal op fragmentdir:mds.0:3813 currently failed to wrlock, waiting 2024-03-28T00:48:46.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:48:45 smithi132 ceph-mon[35941]: slow request 781.553800 seconds old, received at 2024-03-28T00:35:43.477858+0000: internal op fragmentdir:mds.0:3845 currently failed to wrlock, waiting 2024-03-28T00:48:46.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:48:45 smithi132 ceph-mon[35941]: slow request 781.540110 seconds old, received at 2024-03-28T00:35:43.491547+0000: internal op fragmentdir:mds.0:3877 currently failed to wrlock, waiting 2024-03-28T00:48:46.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:48:45 smithi132 ceph-mon[35941]: slow request 781.566787 seconds old, received at 2024-03-28T00:35:43.464871+0000: internal op fragmentdir:mds.0:3814 currently failed to wrlock, waiting 2024-03-28T00:48:46.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:48:45 smithi132 ceph-mon[35941]: slow request 781.521834 seconds old, received at 2024-03-28T00:35:43.509824+0000: internal op fragmentdir:mds.0:3910 currently failed to wrlock, waiting 2024-03-28T00:48:48.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:48:47 smithi165 ceph-mon[36302]: pgmap v720: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-03-28T00:48:48.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:48:47 smithi119 ceph-mon[31518]: pgmap v720: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-03-28T00:48:48.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:48:47 smithi132 ceph-mon[35941]: pgmap v720: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-03-28T00:48:50.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:48:49 smithi165 ceph-mon[36302]: pgmap v721: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:48:50.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:48:49 smithi119 ceph-mon[31518]: pgmap v721: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:48:50.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:48:49 smithi132 ceph-mon[35941]: pgmap v721: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:48:51.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:48:50 smithi165 ceph-mon[36302]: 98 slow requests, 5 included below; oldest blocked for > 790.672775 secs 2024-03-28T00:48:51.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:48:50 smithi165 ceph-mon[36302]: slow request 786.606793 seconds old, received at 2024-03-28T00:35:43.424926+0000: internal op fragmentdir:mds.0:3787 currently failed to wrlock, waiting 2024-03-28T00:48:51.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:48:50 smithi165 ceph-mon[36302]: slow request 786.566409 seconds old, received at 2024-03-28T00:35:43.465310+0000: internal op fragmentdir:mds.0:3815 currently failed to wrlock, waiting 2024-03-28T00:48:51.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:48:50 smithi165 ceph-mon[36302]: slow request 786.539244 seconds old, received at 2024-03-28T00:35:43.492476+0000: internal op fragmentdir:mds.0:3879 currently failed to wrlock, waiting 2024-03-28T00:48:51.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:48:50 smithi165 ceph-mon[36302]: slow request 786.566015 seconds old, received at 2024-03-28T00:35:43.465705+0000: internal op fragmentdir:mds.0:3816 currently failed to wrlock, waiting 2024-03-28T00:48:51.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:48:50 smithi165 ceph-mon[36302]: slow request 786.552454 seconds old, received at 2024-03-28T00:35:43.479266+0000: internal op fragmentdir:mds.0:3848 currently failed to wrlock, waiting 2024-03-28T00:48:51.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:48:50 smithi119 ceph-mon[31518]: 98 slow requests, 5 included below; oldest blocked for > 790.672775 secs 2024-03-28T00:48:51.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:48:50 smithi119 ceph-mon[31518]: slow request 786.606793 seconds old, received at 2024-03-28T00:35:43.424926+0000: internal op fragmentdir:mds.0:3787 currently failed to wrlock, waiting 2024-03-28T00:48:51.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:48:50 smithi119 ceph-mon[31518]: slow request 786.566409 seconds old, received at 2024-03-28T00:35:43.465310+0000: internal op fragmentdir:mds.0:3815 currently failed to wrlock, waiting 2024-03-28T00:48:51.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:48:50 smithi119 ceph-mon[31518]: slow request 786.539244 seconds old, received at 2024-03-28T00:35:43.492476+0000: internal op fragmentdir:mds.0:3879 currently failed to wrlock, waiting 2024-03-28T00:48:51.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:48:50 smithi119 ceph-mon[31518]: slow request 786.566015 seconds old, received at 2024-03-28T00:35:43.465705+0000: internal op fragmentdir:mds.0:3816 currently failed to wrlock, waiting 2024-03-28T00:48:51.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:48:50 smithi119 ceph-mon[31518]: slow request 786.552454 seconds old, received at 2024-03-28T00:35:43.479266+0000: internal op fragmentdir:mds.0:3848 currently failed to wrlock, waiting 2024-03-28T00:48:51.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:48:50 smithi132 ceph-mon[35941]: 98 slow requests, 5 included below; oldest blocked for > 790.672775 secs 2024-03-28T00:48:51.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:48:50 smithi132 ceph-mon[35941]: slow request 786.606793 seconds old, received at 2024-03-28T00:35:43.424926+0000: internal op fragmentdir:mds.0:3787 currently failed to wrlock, waiting 2024-03-28T00:48:51.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:48:50 smithi132 ceph-mon[35941]: slow request 786.566409 seconds old, received at 2024-03-28T00:35:43.465310+0000: internal op fragmentdir:mds.0:3815 currently failed to wrlock, waiting 2024-03-28T00:48:51.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:48:50 smithi132 ceph-mon[35941]: slow request 786.539244 seconds old, received at 2024-03-28T00:35:43.492476+0000: internal op fragmentdir:mds.0:3879 currently failed to wrlock, waiting 2024-03-28T00:48:51.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:48:50 smithi132 ceph-mon[35941]: slow request 786.566015 seconds old, received at 2024-03-28T00:35:43.465705+0000: internal op fragmentdir:mds.0:3816 currently failed to wrlock, waiting 2024-03-28T00:48:51.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:48:50 smithi132 ceph-mon[35941]: slow request 786.552454 seconds old, received at 2024-03-28T00:35:43.479266+0000: internal op fragmentdir:mds.0:3848 currently failed to wrlock, waiting 2024-03-28T00:48:52.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:48:51 smithi165 ceph-mon[36302]: pgmap v722: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:48:52.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:48:51 smithi119 ceph-mon[31518]: pgmap v722: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:48:52.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:48:51 smithi132 ceph-mon[35941]: pgmap v722: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:48:54.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:48:53 smithi165 ceph-mon[36302]: pgmap v723: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:48:54.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:48:53 smithi119 ceph-mon[31518]: pgmap v723: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:48:54.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:48:53 smithi132 ceph-mon[35941]: pgmap v723: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:48:56.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:48:55 smithi165 ceph-mon[36302]: pgmap v724: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-28T00:48:56.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:48:55 smithi165 ceph-mon[36302]: 98 slow requests, 5 included below; oldest blocked for > 795.672880 secs 2024-03-28T00:48:56.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:48:55 smithi165 ceph-mon[36302]: slow request 791.538863 seconds old, received at 2024-03-28T00:35:43.492962+0000: internal op fragmentdir:mds.0:3880 currently failed to wrlock, waiting 2024-03-28T00:48:56.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:48:55 smithi165 ceph-mon[36302]: slow request 792.883754 seconds old, received at 2024-03-28T00:35:42.148071+0000: client_request(client.25264:33954 create owner_uid=1000, owner_gid=1301 #0x10000000271/file8309 2024-03-28T00:35:42.146337+0000 caller_uid=1000, caller_gid=1301{6,36,1000,1301,}) currently failed to xlock, waiting 2024-03-28T00:48:56.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:48:55 smithi165 ceph-mon[36302]: slow request 791.565725 seconds old, received at 2024-03-28T00:35:43.466100+0000: internal op fragmentdir:mds.0:3817 currently failed to wrlock, waiting 2024-03-28T00:48:56.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:48:55 smithi165 ceph-mon[36302]: slow request 791.538535 seconds old, received at 2024-03-28T00:35:43.493290+0000: internal op fragmentdir:mds.0:3881 currently failed to wrlock, waiting 2024-03-28T00:48:56.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:48:55 smithi165 ceph-mon[36302]: slow request 791.565277 seconds old, received at 2024-03-28T00:35:43.466548+0000: internal op fragmentdir:mds.0:3818 currently failed to wrlock, waiting 2024-03-28T00:48:56.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:48:55 smithi119 ceph-mon[31518]: pgmap v724: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-28T00:48:56.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:48:55 smithi119 ceph-mon[31518]: 98 slow requests, 5 included below; oldest blocked for > 795.672880 secs 2024-03-28T00:48:56.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:48:55 smithi119 ceph-mon[31518]: slow request 791.538863 seconds old, received at 2024-03-28T00:35:43.492962+0000: internal op fragmentdir:mds.0:3880 currently failed to wrlock, waiting 2024-03-28T00:48:56.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:48:55 smithi119 ceph-mon[31518]: slow request 792.883754 seconds old, received at 2024-03-28T00:35:42.148071+0000: client_request(client.25264:33954 create owner_uid=1000, owner_gid=1301 #0x10000000271/file8309 2024-03-28T00:35:42.146337+0000 caller_uid=1000, caller_gid=1301{6,36,1000,1301,}) currently failed to xlock, waiting 2024-03-28T00:48:56.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:48:55 smithi119 ceph-mon[31518]: slow request 791.565725 seconds old, received at 2024-03-28T00:35:43.466100+0000: internal op fragmentdir:mds.0:3817 currently failed to wrlock, waiting 2024-03-28T00:48:56.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:48:55 smithi119 ceph-mon[31518]: slow request 791.538535 seconds old, received at 2024-03-28T00:35:43.493290+0000: internal op fragmentdir:mds.0:3881 currently failed to wrlock, waiting 2024-03-28T00:48:56.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:48:55 smithi119 ceph-mon[31518]: slow request 791.565277 seconds old, received at 2024-03-28T00:35:43.466548+0000: internal op fragmentdir:mds.0:3818 currently failed to wrlock, waiting 2024-03-28T00:48:56.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:48:55 smithi132 ceph-mon[35941]: pgmap v724: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-28T00:48:56.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:48:55 smithi132 ceph-mon[35941]: 98 slow requests, 5 included below; oldest blocked for > 795.672880 secs 2024-03-28T00:48:56.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:48:55 smithi132 ceph-mon[35941]: slow request 791.538863 seconds old, received at 2024-03-28T00:35:43.492962+0000: internal op fragmentdir:mds.0:3880 currently failed to wrlock, waiting 2024-03-28T00:48:56.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:48:55 smithi132 ceph-mon[35941]: slow request 792.883754 seconds old, received at 2024-03-28T00:35:42.148071+0000: client_request(client.25264:33954 create owner_uid=1000, owner_gid=1301 #0x10000000271/file8309 2024-03-28T00:35:42.146337+0000 caller_uid=1000, caller_gid=1301{6,36,1000,1301,}) currently failed to xlock, waiting 2024-03-28T00:48:56.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:48:55 smithi132 ceph-mon[35941]: slow request 791.565725 seconds old, received at 2024-03-28T00:35:43.466100+0000: internal op fragmentdir:mds.0:3817 currently failed to wrlock, waiting 2024-03-28T00:48:56.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:48:55 smithi132 ceph-mon[35941]: slow request 791.538535 seconds old, received at 2024-03-28T00:35:43.493290+0000: internal op fragmentdir:mds.0:3881 currently failed to wrlock, waiting 2024-03-28T00:48:56.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:48:55 smithi132 ceph-mon[35941]: slow request 791.565277 seconds old, received at 2024-03-28T00:35:43.466548+0000: internal op fragmentdir:mds.0:3818 currently failed to wrlock, waiting 2024-03-28T00:48:58.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:48:57 smithi165 ceph-mon[36302]: pgmap v725: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-03-28T00:48:58.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:48:57 smithi119 ceph-mon[31518]: pgmap v725: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-03-28T00:48:58.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:48:57 smithi132 ceph-mon[35941]: pgmap v725: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-03-28T00:49:00.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:48:59 smithi119 ceph-mon[31518]: pgmap v726: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:49:00.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:48:59 smithi132 ceph-mon[35941]: pgmap v726: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:49:00.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:48:59 smithi165 ceph-mon[36302]: pgmap v726: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:49:01.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:49:00 smithi119 ceph-mon[31518]: 98 slow requests, 5 included below; oldest blocked for > 800.672902 secs 2024-03-28T00:49:01.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:49:00 smithi119 ceph-mon[31518]: slow request 796.551655 seconds old, received at 2024-03-28T00:35:43.480192+0000: internal op fragmentdir:mds.0:3850 currently failed to wrlock, waiting 2024-03-28T00:49:01.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:49:00 smithi119 ceph-mon[31518]: slow request 796.583103 seconds old, received at 2024-03-28T00:35:43.448744+0000: internal op fragmentdir:mds.0:3788 currently failed to wrlock, waiting 2024-03-28T00:49:01.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:49:00 smithi119 ceph-mon[31518]: slow request 796.564891 seconds old, received at 2024-03-28T00:35:43.466955+0000: internal op fragmentdir:mds.0:3819 currently failed to wrlock, waiting 2024-03-28T00:49:01.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:49:00 smithi119 ceph-mon[31518]: slow request 796.551180 seconds old, received at 2024-03-28T00:35:43.480667+0000: internal op fragmentdir:mds.0:3851 currently failed to wrlock, waiting 2024-03-28T00:49:01.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:49:00 smithi119 ceph-mon[31518]: slow request 796.538008 seconds old, received at 2024-03-28T00:35:43.493839+0000: internal op fragmentdir:mds.0:3883 currently failed to wrlock, waiting 2024-03-28T00:49:01.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:49:00 smithi132 ceph-mon[35941]: 98 slow requests, 5 included below; oldest blocked for > 800.672902 secs 2024-03-28T00:49:01.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:49:00 smithi132 ceph-mon[35941]: slow request 796.551655 seconds old, received at 2024-03-28T00:35:43.480192+0000: internal op fragmentdir:mds.0:3850 currently failed to wrlock, waiting 2024-03-28T00:49:01.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:49:00 smithi132 ceph-mon[35941]: slow request 796.583103 seconds old, received at 2024-03-28T00:35:43.448744+0000: internal op fragmentdir:mds.0:3788 currently failed to wrlock, waiting 2024-03-28T00:49:01.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:49:00 smithi132 ceph-mon[35941]: slow request 796.564891 seconds old, received at 2024-03-28T00:35:43.466955+0000: internal op fragmentdir:mds.0:3819 currently failed to wrlock, waiting 2024-03-28T00:49:01.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:49:00 smithi132 ceph-mon[35941]: slow request 796.551180 seconds old, received at 2024-03-28T00:35:43.480667+0000: internal op fragmentdir:mds.0:3851 currently failed to wrlock, waiting 2024-03-28T00:49:01.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:49:00 smithi132 ceph-mon[35941]: slow request 796.538008 seconds old, received at 2024-03-28T00:35:43.493839+0000: internal op fragmentdir:mds.0:3883 currently failed to wrlock, waiting 2024-03-28T00:49:01.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:49:00 smithi165 ceph-mon[36302]: 98 slow requests, 5 included below; oldest blocked for > 800.672902 secs 2024-03-28T00:49:01.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:49:00 smithi165 ceph-mon[36302]: slow request 796.551655 seconds old, received at 2024-03-28T00:35:43.480192+0000: internal op fragmentdir:mds.0:3850 currently failed to wrlock, waiting 2024-03-28T00:49:01.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:49:00 smithi165 ceph-mon[36302]: slow request 796.583103 seconds old, received at 2024-03-28T00:35:43.448744+0000: internal op fragmentdir:mds.0:3788 currently failed to wrlock, waiting 2024-03-28T00:49:01.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:49:00 smithi165 ceph-mon[36302]: slow request 796.564891 seconds old, received at 2024-03-28T00:35:43.466955+0000: internal op fragmentdir:mds.0:3819 currently failed to wrlock, waiting 2024-03-28T00:49:01.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:49:00 smithi165 ceph-mon[36302]: slow request 796.551180 seconds old, received at 2024-03-28T00:35:43.480667+0000: internal op fragmentdir:mds.0:3851 currently failed to wrlock, waiting 2024-03-28T00:49:01.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:49:00 smithi165 ceph-mon[36302]: slow request 796.538008 seconds old, received at 2024-03-28T00:35:43.493839+0000: internal op fragmentdir:mds.0:3883 currently failed to wrlock, waiting 2024-03-28T00:49:02.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:49:01 smithi119 ceph-mon[31518]: pgmap v727: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:49:02.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:49:01 smithi132 ceph-mon[35941]: pgmap v727: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:49:02.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:49:01 smithi165 ceph-mon[36302]: pgmap v727: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:49:04.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:49:03 smithi119 ceph-mon[31518]: pgmap v728: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:49:04.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:49:03 smithi132 ceph-mon[35941]: pgmap v728: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:49:04.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:49:03 smithi165 ceph-mon[36302]: pgmap v728: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:49:06.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:49:05 smithi132 ceph-mon[35941]: pgmap v729: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-28T00:49:06.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:49:05 smithi132 ceph-mon[35941]: 98 slow requests, 5 included below; oldest blocked for > 805.672964 secs 2024-03-28T00:49:06.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:49:05 smithi132 ceph-mon[35941]: slow request 801.582748 seconds old, received at 2024-03-28T00:35:43.449161+0000: internal op fragmentdir:mds.0:3789 currently failed to wrlock, waiting 2024-03-28T00:49:06.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:49:05 smithi132 ceph-mon[35941]: slow request 801.564518 seconds old, received at 2024-03-28T00:35:43.467391+0000: internal op fragmentdir:mds.0:3820 currently failed to wrlock, waiting 2024-03-28T00:49:06.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:49:05 smithi132 ceph-mon[35941]: slow request 801.550768 seconds old, received at 2024-03-28T00:35:43.481141+0000: internal op fragmentdir:mds.0:3852 currently failed to wrlock, waiting 2024-03-28T00:49:06.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:49:05 smithi132 ceph-mon[35941]: slow request 801.508975 seconds old, received at 2024-03-28T00:35:43.522934+0000: internal op fragmentdir:mds.0:3916 currently failed to wrlock, waiting 2024-03-28T00:49:06.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:49:05 smithi132 ceph-mon[35941]: slow request 801.582288 seconds old, received at 2024-03-28T00:35:43.449621+0000: internal op fragmentdir:mds.0:3790 currently failed to wrlock, waiting 2024-03-28T00:49:06.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:49:05 smithi165 ceph-mon[36302]: pgmap v729: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-28T00:49:06.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:49:05 smithi165 ceph-mon[36302]: 98 slow requests, 5 included below; oldest blocked for > 805.672964 secs 2024-03-28T00:49:06.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:49:05 smithi165 ceph-mon[36302]: slow request 801.582748 seconds old, received at 2024-03-28T00:35:43.449161+0000: internal op fragmentdir:mds.0:3789 currently failed to wrlock, waiting 2024-03-28T00:49:06.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:49:05 smithi165 ceph-mon[36302]: slow request 801.564518 seconds old, received at 2024-03-28T00:35:43.467391+0000: internal op fragmentdir:mds.0:3820 currently failed to wrlock, waiting 2024-03-28T00:49:06.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:49:05 smithi165 ceph-mon[36302]: slow request 801.550768 seconds old, received at 2024-03-28T00:35:43.481141+0000: internal op fragmentdir:mds.0:3852 currently failed to wrlock, waiting 2024-03-28T00:49:06.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:49:05 smithi165 ceph-mon[36302]: slow request 801.508975 seconds old, received at 2024-03-28T00:35:43.522934+0000: internal op fragmentdir:mds.0:3916 currently failed to wrlock, waiting 2024-03-28T00:49:06.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:49:05 smithi165 ceph-mon[36302]: slow request 801.582288 seconds old, received at 2024-03-28T00:35:43.449621+0000: internal op fragmentdir:mds.0:3790 currently failed to wrlock, waiting 2024-03-28T00:49:06.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:49:05 smithi119 ceph-mon[31518]: pgmap v729: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-28T00:49:06.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:49:05 smithi119 ceph-mon[31518]: 98 slow requests, 5 included below; oldest blocked for > 805.672964 secs 2024-03-28T00:49:06.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:49:05 smithi119 ceph-mon[31518]: slow request 801.582748 seconds old, received at 2024-03-28T00:35:43.449161+0000: internal op fragmentdir:mds.0:3789 currently failed to wrlock, waiting 2024-03-28T00:49:06.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:49:05 smithi119 ceph-mon[31518]: slow request 801.564518 seconds old, received at 2024-03-28T00:35:43.467391+0000: internal op fragmentdir:mds.0:3820 currently failed to wrlock, waiting 2024-03-28T00:49:06.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:49:05 smithi119 ceph-mon[31518]: slow request 801.550768 seconds old, received at 2024-03-28T00:35:43.481141+0000: internal op fragmentdir:mds.0:3852 currently failed to wrlock, waiting 2024-03-28T00:49:06.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:49:05 smithi119 ceph-mon[31518]: slow request 801.508975 seconds old, received at 2024-03-28T00:35:43.522934+0000: internal op fragmentdir:mds.0:3916 currently failed to wrlock, waiting 2024-03-28T00:49:06.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:49:05 smithi119 ceph-mon[31518]: slow request 801.582288 seconds old, received at 2024-03-28T00:35:43.449621+0000: internal op fragmentdir:mds.0:3790 currently failed to wrlock, waiting 2024-03-28T00:49:08.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:49:07 smithi132 ceph-mon[35941]: pgmap v730: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-03-28T00:49:08.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:49:07 smithi165 ceph-mon[36302]: pgmap v730: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-03-28T00:49:08.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:49:07 smithi119 ceph-mon[31518]: pgmap v730: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-03-28T00:49:10.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:49:09 smithi132 ceph-mon[35941]: pgmap v731: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:49:10.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:49:09 smithi165 ceph-mon[36302]: pgmap v731: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:49:10.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:49:09 smithi119 ceph-mon[31518]: pgmap v731: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:49:11.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:49:10 smithi132 ceph-mon[35941]: 98 slow requests, 5 included below; oldest blocked for > 810.673026 secs 2024-03-28T00:49:11.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:49:10 smithi132 ceph-mon[35941]: slow request 806.564176 seconds old, received at 2024-03-28T00:35:43.467794+0000: internal op fragmentdir:mds.0:3821 currently failed to wrlock, waiting 2024-03-28T00:49:11.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:49:10 smithi132 ceph-mon[35941]: slow request 806.537348 seconds old, received at 2024-03-28T00:35:43.494622+0000: internal op fragmentdir:mds.0:3885 currently failed to wrlock, waiting 2024-03-28T00:49:11.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:49:10 smithi132 ceph-mon[35941]: slow request 806.581807 seconds old, received at 2024-03-28T00:35:43.450163+0000: internal op fragmentdir:mds.0:3791 currently failed to wrlock, waiting 2024-03-28T00:49:11.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:49:10 smithi132 ceph-mon[35941]: slow request 806.549911 seconds old, received at 2024-03-28T00:35:43.482059+0000: internal op fragmentdir:mds.0:3854 currently failed to wrlock, waiting 2024-03-28T00:49:11.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:49:10 smithi132 ceph-mon[35941]: slow request 806.536896 seconds old, received at 2024-03-28T00:35:43.495074+0000: internal op fragmentdir:mds.0:3886 currently failed to wrlock, waiting 2024-03-28T00:49:11.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:49:10 smithi165 ceph-mon[36302]: 98 slow requests, 5 included below; oldest blocked for > 810.673026 secs 2024-03-28T00:49:11.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:49:10 smithi165 ceph-mon[36302]: slow request 806.564176 seconds old, received at 2024-03-28T00:35:43.467794+0000: internal op fragmentdir:mds.0:3821 currently failed to wrlock, waiting 2024-03-28T00:49:11.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:49:10 smithi165 ceph-mon[36302]: slow request 806.537348 seconds old, received at 2024-03-28T00:35:43.494622+0000: internal op fragmentdir:mds.0:3885 currently failed to wrlock, waiting 2024-03-28T00:49:11.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:49:10 smithi165 ceph-mon[36302]: slow request 806.581807 seconds old, received at 2024-03-28T00:35:43.450163+0000: internal op fragmentdir:mds.0:3791 currently failed to wrlock, waiting 2024-03-28T00:49:11.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:49:10 smithi165 ceph-mon[36302]: slow request 806.549911 seconds old, received at 2024-03-28T00:35:43.482059+0000: internal op fragmentdir:mds.0:3854 currently failed to wrlock, waiting 2024-03-28T00:49:11.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:49:10 smithi165 ceph-mon[36302]: slow request 806.536896 seconds old, received at 2024-03-28T00:35:43.495074+0000: internal op fragmentdir:mds.0:3886 currently failed to wrlock, waiting 2024-03-28T00:49:11.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:49:10 smithi119 ceph-mon[31518]: 98 slow requests, 5 included below; oldest blocked for > 810.673026 secs 2024-03-28T00:49:11.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:49:10 smithi119 ceph-mon[31518]: slow request 806.564176 seconds old, received at 2024-03-28T00:35:43.467794+0000: internal op fragmentdir:mds.0:3821 currently failed to wrlock, waiting 2024-03-28T00:49:11.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:49:10 smithi119 ceph-mon[31518]: slow request 806.537348 seconds old, received at 2024-03-28T00:35:43.494622+0000: internal op fragmentdir:mds.0:3885 currently failed to wrlock, waiting 2024-03-28T00:49:11.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:49:10 smithi119 ceph-mon[31518]: slow request 806.581807 seconds old, received at 2024-03-28T00:35:43.450163+0000: internal op fragmentdir:mds.0:3791 currently failed to wrlock, waiting 2024-03-28T00:49:11.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:49:10 smithi119 ceph-mon[31518]: slow request 806.549911 seconds old, received at 2024-03-28T00:35:43.482059+0000: internal op fragmentdir:mds.0:3854 currently failed to wrlock, waiting 2024-03-28T00:49:11.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:49:10 smithi119 ceph-mon[31518]: slow request 806.536896 seconds old, received at 2024-03-28T00:35:43.495074+0000: internal op fragmentdir:mds.0:3886 currently failed to wrlock, waiting 2024-03-28T00:49:12.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:49:11 smithi132 ceph-mon[35941]: pgmap v732: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:49:12.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:49:11 smithi165 ceph-mon[36302]: pgmap v732: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:49:12.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:49:11 smithi119 ceph-mon[31518]: pgmap v732: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:49:14.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:49:13 smithi132 ceph-mon[35941]: pgmap v733: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:49:14.275 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:49:13 smithi119 ceph-mon[31518]: pgmap v733: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:49:14.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:49:13 smithi165 ceph-mon[36302]: pgmap v733: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:49:16.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:49:15 smithi132 ceph-mon[35941]: pgmap v734: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-28T00:49:16.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:49:15 smithi132 ceph-mon[35941]: 98 slow requests, 5 included below; oldest blocked for > 815.673087 secs 2024-03-28T00:49:16.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:49:15 smithi132 ceph-mon[35941]: slow request 811.507978 seconds old, received at 2024-03-28T00:35:43.524054+0000: internal op fragmentdir:mds.0:3918 currently failed to wrlock, waiting 2024-03-28T00:49:16.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:49:15 smithi132 ceph-mon[35941]: slow request 811.581419 seconds old, received at 2024-03-28T00:35:43.450613+0000: internal op fragmentdir:mds.0:3792 currently failed to wrlock, waiting 2024-03-28T00:49:16.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:49:15 smithi132 ceph-mon[35941]: slow request 811.563643 seconds old, received at 2024-03-28T00:35:43.468389+0000: internal op fragmentdir:mds.0:3823 currently failed to wrlock, waiting 2024-03-28T00:49:16.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:49:15 smithi132 ceph-mon[35941]: slow request 811.580927 seconds old, received at 2024-03-28T00:35:43.451105+0000: internal op fragmentdir:mds.0:3793 currently failed to wrlock, waiting 2024-03-28T00:49:16.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:49:15 smithi132 ceph-mon[35941]: slow request 811.536084 seconds old, received at 2024-03-28T00:35:43.495948+0000: internal op fragmentdir:mds.0:3888 currently failed to wrlock, waiting 2024-03-28T00:49:16.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:49:15 smithi165 ceph-mon[36302]: pgmap v734: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-28T00:49:16.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:49:15 smithi165 ceph-mon[36302]: 98 slow requests, 5 included below; oldest blocked for > 815.673087 secs 2024-03-28T00:49:16.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:49:15 smithi165 ceph-mon[36302]: slow request 811.507978 seconds old, received at 2024-03-28T00:35:43.524054+0000: internal op fragmentdir:mds.0:3918 currently failed to wrlock, waiting 2024-03-28T00:49:16.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:49:15 smithi165 ceph-mon[36302]: slow request 811.581419 seconds old, received at 2024-03-28T00:35:43.450613+0000: internal op fragmentdir:mds.0:3792 currently failed to wrlock, waiting 2024-03-28T00:49:16.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:49:15 smithi165 ceph-mon[36302]: slow request 811.563643 seconds old, received at 2024-03-28T00:35:43.468389+0000: internal op fragmentdir:mds.0:3823 currently failed to wrlock, waiting 2024-03-28T00:49:16.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:49:15 smithi165 ceph-mon[36302]: slow request 811.580927 seconds old, received at 2024-03-28T00:35:43.451105+0000: internal op fragmentdir:mds.0:3793 currently failed to wrlock, waiting 2024-03-28T00:49:16.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:49:15 smithi165 ceph-mon[36302]: slow request 811.536084 seconds old, received at 2024-03-28T00:35:43.495948+0000: internal op fragmentdir:mds.0:3888 currently failed to wrlock, waiting 2024-03-28T00:49:16.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:49:15 smithi119 ceph-mon[31518]: pgmap v734: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-28T00:49:16.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:49:15 smithi119 ceph-mon[31518]: 98 slow requests, 5 included below; oldest blocked for > 815.673087 secs 2024-03-28T00:49:16.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:49:15 smithi119 ceph-mon[31518]: slow request 811.507978 seconds old, received at 2024-03-28T00:35:43.524054+0000: internal op fragmentdir:mds.0:3918 currently failed to wrlock, waiting 2024-03-28T00:49:16.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:49:15 smithi119 ceph-mon[31518]: slow request 811.581419 seconds old, received at 2024-03-28T00:35:43.450613+0000: internal op fragmentdir:mds.0:3792 currently failed to wrlock, waiting 2024-03-28T00:49:16.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:49:15 smithi119 ceph-mon[31518]: slow request 811.563643 seconds old, received at 2024-03-28T00:35:43.468389+0000: internal op fragmentdir:mds.0:3823 currently failed to wrlock, waiting 2024-03-28T00:49:16.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:49:15 smithi119 ceph-mon[31518]: slow request 811.580927 seconds old, received at 2024-03-28T00:35:43.451105+0000: internal op fragmentdir:mds.0:3793 currently failed to wrlock, waiting 2024-03-28T00:49:16.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:49:15 smithi119 ceph-mon[31518]: slow request 811.536084 seconds old, received at 2024-03-28T00:35:43.495948+0000: internal op fragmentdir:mds.0:3888 currently failed to wrlock, waiting 2024-03-28T00:49:18.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:49:17 smithi132 ceph-mon[35941]: pgmap v735: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-03-28T00:49:18.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:49:17 smithi165 ceph-mon[36302]: pgmap v735: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-03-28T00:49:18.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:49:17 smithi119 ceph-mon[31518]: pgmap v735: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-03-28T00:49:20.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:49:19 smithi132 ceph-mon[35941]: pgmap v736: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:49:20.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:49:19 smithi165 ceph-mon[36302]: pgmap v736: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:49:20.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:49:19 smithi119 ceph-mon[31518]: pgmap v736: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:49:21.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:49:20 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 820.673167 secs 2024-03-28T00:49:21.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:49:20 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 820.673167 secs 2024-03-28T00:49:21.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:49:20 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 820.673167 secs 2024-03-28T00:49:22.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:49:21 smithi165 ceph-mon[36302]: pgmap v737: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:49:22.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:49:21 smithi119 ceph-mon[31518]: pgmap v737: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:49:22.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:49:21 smithi132 ceph-mon[35941]: pgmap v737: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:49:23.305 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:49:22 smithi119 ceph-mon[31518]: pgmap v738: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:49:23.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:49:22 smithi165 ceph-mon[36302]: pgmap v738: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:49:23.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:49:22 smithi132 ceph-mon[35941]: pgmap v738: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:49:25.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:49:25 smithi165 ceph-mon[36302]: pgmap v739: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-28T00:49:25.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:49:25 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 825.673227 secs 2024-03-28T00:49:25.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:49:25 smithi119 ceph-mon[31518]: pgmap v739: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-28T00:49:25.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:49:25 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 825.673227 secs 2024-03-28T00:49:25.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:49:25 smithi132 ceph-mon[35941]: pgmap v739: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-28T00:49:25.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:49:25 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 825.673227 secs 2024-03-28T00:49:27.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:49:27 smithi165 ceph-mon[36302]: pgmap v740: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-03-28T00:49:27.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:49:27 smithi119 ceph-mon[31518]: pgmap v740: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-03-28T00:49:27.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:49:27 smithi132 ceph-mon[35941]: pgmap v740: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-03-28T00:49:29.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:49:29 smithi165 ceph-mon[36302]: pgmap v741: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:49:29.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:49:29 smithi119 ceph-mon[31518]: pgmap v741: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:49:29.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:49:29 smithi132 ceph-mon[35941]: pgmap v741: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:49:30.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:49:30 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 830.673289 secs 2024-03-28T00:49:30.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:49:30 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 830.673289 secs 2024-03-28T00:49:30.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:49:30 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 830.673289 secs 2024-03-28T00:49:31.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:49:31 smithi165 ceph-mon[36302]: pgmap v742: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:49:31.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:49:31 smithi119 ceph-mon[31518]: pgmap v742: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:49:31.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:49:31 smithi132 ceph-mon[35941]: pgmap v742: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:49:33.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:49:33 smithi165 ceph-mon[36302]: pgmap v743: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:49:33.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:49:33 smithi119 ceph-mon[31518]: pgmap v743: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:49:33.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:49:33 smithi132 ceph-mon[35941]: pgmap v743: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:49:35.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:49:35 smithi165 ceph-mon[36302]: pgmap v744: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-28T00:49:35.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:49:35 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:49:35.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:49:35 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 835.673381 secs 2024-03-28T00:49:35.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:49:35 smithi119 ceph-mon[31518]: pgmap v744: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-28T00:49:35.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:49:35 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:49:35.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:49:35 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 835.673381 secs 2024-03-28T00:49:35.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:49:35 smithi132 ceph-mon[35941]: pgmap v744: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-28T00:49:35.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:49:35 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:49:35.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:49:35 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 835.673381 secs 2024-03-28T00:49:36.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:49:36 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:49:36.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:49:36 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:49:36.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:49:36 smithi165 ceph-mon[36302]: pgmap v745: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-03-28T00:49:36.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:49:36 smithi165 ceph-mon[36302]: pgmap v746: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T00:49:36.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:49:36 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:49:36.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:49:36 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:49:36.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:49:36 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:49:36.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:49:36 smithi119 ceph-mon[31518]: pgmap v745: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-03-28T00:49:36.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:49:36 smithi119 ceph-mon[31518]: pgmap v746: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T00:49:36.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:49:36 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:49:36.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:49:36 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:49:36.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:49:36 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:49:36.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:49:36 smithi132 ceph-mon[35941]: pgmap v745: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-03-28T00:49:36.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:49:36 smithi132 ceph-mon[35941]: pgmap v746: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T00:49:36.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:49:36 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:49:39.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:49:38 smithi165 ceph-mon[36302]: pgmap v747: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T00:49:39.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:49:38 smithi119 ceph-mon[31518]: pgmap v747: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T00:49:39.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:49:38 smithi132 ceph-mon[35941]: pgmap v747: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T00:49:41.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:49:40 smithi165 ceph-mon[36302]: pgmap v748: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T00:49:41.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:49:40 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 840.673470 secs 2024-03-28T00:49:41.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:49:40 smithi119 ceph-mon[31518]: pgmap v748: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T00:49:41.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:49:40 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 840.673470 secs 2024-03-28T00:49:41.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:49:40 smithi132 ceph-mon[35941]: pgmap v748: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T00:49:41.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:49:40 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 840.673470 secs 2024-03-28T00:49:43.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:49:42 smithi165 ceph-mon[36302]: pgmap v749: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T00:49:43.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:49:42 smithi119 ceph-mon[31518]: pgmap v749: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T00:49:43.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:49:42 smithi132 ceph-mon[35941]: pgmap v749: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T00:49:45.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:49:44 smithi165 ceph-mon[36302]: pgmap v750: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:49:45.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:49:44 smithi119 ceph-mon[31518]: pgmap v750: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:49:45.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:49:44 smithi132 ceph-mon[35941]: pgmap v750: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:49:46.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:49:45 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 845.673560 secs 2024-03-28T00:49:46.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:49:45 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 845.673560 secs 2024-03-28T00:49:46.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:49:45 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 845.673560 secs 2024-03-28T00:49:47.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:49:46 smithi165 ceph-mon[36302]: pgmap v751: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-28T00:49:47.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:49:46 smithi119 ceph-mon[31518]: pgmap v751: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-28T00:49:47.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:49:46 smithi132 ceph-mon[35941]: pgmap v751: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-28T00:49:49.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:49:48 smithi165 ceph-mon[36302]: pgmap v752: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T00:49:49.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:49:48 smithi119 ceph-mon[31518]: pgmap v752: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T00:49:49.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:49:48 smithi132 ceph-mon[35941]: pgmap v752: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T00:49:51.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:49:50 smithi165 ceph-mon[36302]: pgmap v753: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:49:51.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:49:50 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 850.673649 secs 2024-03-28T00:49:51.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:49:50 smithi119 ceph-mon[31518]: pgmap v753: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:49:51.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:49:50 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 850.673649 secs 2024-03-28T00:49:51.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:49:50 smithi132 ceph-mon[35941]: pgmap v753: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:49:51.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:49:50 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 850.673649 secs 2024-03-28T00:49:53.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:49:52 smithi165 ceph-mon[36302]: pgmap v754: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:49:53.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:49:52 smithi119 ceph-mon[31518]: pgmap v754: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:49:53.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:49:52 smithi132 ceph-mon[35941]: pgmap v754: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:49:55.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:49:54 smithi165 ceph-mon[36302]: pgmap v755: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:49:55.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:49:54 smithi119 ceph-mon[31518]: pgmap v755: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:49:55.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:49:54 smithi132 ceph-mon[35941]: pgmap v755: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:49:56.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:49:55 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 855.673738 secs 2024-03-28T00:49:56.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:49:55 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 855.673738 secs 2024-03-28T00:49:56.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:49:55 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 855.673738 secs 2024-03-28T00:49:57.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:49:56 smithi165 ceph-mon[36302]: pgmap v756: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:49:57.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:49:56 smithi119 ceph-mon[31518]: pgmap v756: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:49:57.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:49:56 smithi132 ceph-mon[35941]: pgmap v756: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:49:59.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:49:58 smithi165 ceph-mon[36302]: pgmap v757: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:49:59.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:49:58 smithi119 ceph-mon[31518]: pgmap v757: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:49:59.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:49:58 smithi132 ceph-mon[35941]: pgmap v757: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:50:01.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:50:00 smithi165 ceph-mon[36302]: pgmap v758: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:50:01.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:50:00 smithi165 ceph-mon[36302]: overall HEALTH_WARN 5 failed cephadm daemon(s); 1 filesystem is degraded; 1 MDSs report slow requests 2024-03-28T00:50:01.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:50:00 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 860.673799 secs 2024-03-28T00:50:01.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:50:00 smithi119 ceph-mon[31518]: pgmap v758: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:50:01.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:50:00 smithi119 ceph-mon[31518]: overall HEALTH_WARN 5 failed cephadm daemon(s); 1 filesystem is degraded; 1 MDSs report slow requests 2024-03-28T00:50:01.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:50:00 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 860.673799 secs 2024-03-28T00:50:01.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:50:00 smithi132 ceph-mon[35941]: pgmap v758: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:50:01.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:50:00 smithi132 ceph-mon[35941]: overall HEALTH_WARN 5 failed cephadm daemon(s); 1 filesystem is degraded; 1 MDSs report slow requests 2024-03-28T00:50:01.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:50:00 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 860.673799 secs 2024-03-28T00:50:03.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:50:02 smithi165 ceph-mon[36302]: pgmap v759: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:50:03.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:50:02 smithi119 ceph-mon[31518]: pgmap v759: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:50:03.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:50:02 smithi132 ceph-mon[35941]: pgmap v759: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:50:05.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:50:04 smithi165 ceph-mon[36302]: pgmap v760: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:50:05.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:50:04 smithi119 ceph-mon[31518]: pgmap v760: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:50:05.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:50:04 smithi132 ceph-mon[35941]: pgmap v760: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:50:06.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:50:05 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 865.673888 secs 2024-03-28T00:50:06.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:50:05 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 865.673888 secs 2024-03-28T00:50:06.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:50:05 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 865.673888 secs 2024-03-28T00:50:07.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:50:06 smithi165 ceph-mon[36302]: pgmap v761: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:50:07.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:50:06 smithi119 ceph-mon[31518]: pgmap v761: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:50:07.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:50:06 smithi132 ceph-mon[35941]: pgmap v761: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:50:09.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:50:08 smithi165 ceph-mon[36302]: pgmap v762: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:50:09.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:50:08 smithi119 ceph-mon[31518]: pgmap v762: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:50:09.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:50:08 smithi132 ceph-mon[35941]: pgmap v762: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:50:11.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:50:10 smithi119 ceph-mon[31518]: pgmap v763: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:50:11.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:50:10 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 870.673952 secs 2024-03-28T00:50:11.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:50:10 smithi132 ceph-mon[35941]: pgmap v763: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:50:11.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:50:10 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 870.673952 secs 2024-03-28T00:50:11.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:50:10 smithi165 ceph-mon[36302]: pgmap v763: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:50:11.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:50:10 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 870.673952 secs 2024-03-28T00:50:13.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:50:12 smithi119 ceph-mon[31518]: pgmap v764: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:50:13.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:50:12 smithi132 ceph-mon[35941]: pgmap v764: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:50:13.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:50:12 smithi165 ceph-mon[36302]: pgmap v764: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:50:15.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:50:14 smithi119 ceph-mon[31518]: pgmap v765: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:50:15.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:50:14 smithi132 ceph-mon[35941]: pgmap v765: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:50:15.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:50:14 smithi165 ceph-mon[36302]: pgmap v765: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:50:16.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:50:15 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 875.673965 secs 2024-03-28T00:50:16.227 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:50:15 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 875.673965 secs 2024-03-28T00:50:16.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:50:15 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 875.673965 secs 2024-03-28T00:50:17.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:50:16 smithi132 ceph-mon[35941]: pgmap v766: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:50:17.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:50:16 smithi165 ceph-mon[36302]: pgmap v766: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:50:17.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:50:16 smithi119 ceph-mon[31518]: pgmap v766: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:50:19.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:50:18 smithi132 ceph-mon[35941]: pgmap v767: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:50:19.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:50:18 smithi165 ceph-mon[36302]: pgmap v767: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:50:19.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:50:18 smithi119 ceph-mon[31518]: pgmap v767: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:50:21.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:50:20 smithi132 ceph-mon[35941]: pgmap v768: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:50:21.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:50:20 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 880.674029 secs 2024-03-28T00:50:21.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:50:20 smithi165 ceph-mon[36302]: pgmap v768: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:50:21.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:50:20 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 880.674029 secs 2024-03-28T00:50:21.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:50:20 smithi119 ceph-mon[31518]: pgmap v768: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:50:21.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:50:20 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 880.674029 secs 2024-03-28T00:50:23.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:50:22 smithi132 ceph-mon[35941]: pgmap v769: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:50:23.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:50:22 smithi165 ceph-mon[36302]: pgmap v769: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:50:23.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:50:22 smithi119 ceph-mon[31518]: pgmap v769: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:50:25.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:50:24 smithi132 ceph-mon[35941]: pgmap v770: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:50:25.204 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:50:24 smithi119 ceph-mon[31518]: pgmap v770: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:50:25.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:50:24 smithi165 ceph-mon[36302]: pgmap v770: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:50:26.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:50:25 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 885.674090 secs 2024-03-28T00:50:26.281 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:50:25 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 885.674090 secs 2024-03-28T00:50:26.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:50:25 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 885.674090 secs 2024-03-28T00:50:27.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:50:26 smithi132 ceph-mon[35941]: pgmap v771: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:50:27.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:50:26 smithi165 ceph-mon[36302]: pgmap v771: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:50:27.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:50:26 smithi119 ceph-mon[31518]: pgmap v771: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:50:29.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:50:28 smithi165 ceph-mon[36302]: pgmap v772: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:50:29.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:50:28 smithi119 ceph-mon[31518]: pgmap v772: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:50:29.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:50:28 smithi132 ceph-mon[35941]: pgmap v772: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:50:31.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:50:30 smithi165 ceph-mon[36302]: pgmap v773: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:50:31.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:50:30 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 890.674153 secs 2024-03-28T00:50:31.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:50:30 smithi119 ceph-mon[31518]: pgmap v773: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:50:31.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:50:30 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 890.674153 secs 2024-03-28T00:50:31.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:50:30 smithi132 ceph-mon[35941]: pgmap v773: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:50:31.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:50:30 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 890.674153 secs 2024-03-28T00:50:33.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:50:32 smithi165 ceph-mon[36302]: pgmap v774: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:50:33.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:50:32 smithi119 ceph-mon[31518]: pgmap v774: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:50:33.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:50:32 smithi132 ceph-mon[35941]: pgmap v774: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:50:35.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:50:34 smithi165 ceph-mon[36302]: pgmap v775: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:50:35.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:50:34 smithi119 ceph-mon[31518]: pgmap v775: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:50:35.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:50:34 smithi132 ceph-mon[35941]: pgmap v775: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:50:36.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:50:35 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 895.674243 secs 2024-03-28T00:50:36.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:50:35 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 895.674243 secs 2024-03-28T00:50:36.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:50:35 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 895.674243 secs 2024-03-28T00:50:37.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:50:36 smithi165 ceph-mon[36302]: pgmap v776: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:50:37.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:50:36 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:50:37.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:50:36 smithi119 ceph-mon[31518]: pgmap v776: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:50:37.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:50:36 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:50:37.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:50:36 smithi132 ceph-mon[35941]: pgmap v776: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:50:37.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:50:36 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:50:38.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:50:37 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:50:38.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:50:37 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:50:38.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:50:37 smithi165 ceph-mon[36302]: pgmap v777: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T00:50:38.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:50:37 smithi165 ceph-mon[36302]: pgmap v778: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T00:50:38.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:50:37 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:50:38.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:50:37 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:50:38.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:50:37 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:50:38.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:50:37 smithi119 ceph-mon[31518]: pgmap v777: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T00:50:38.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:50:37 smithi119 ceph-mon[31518]: pgmap v778: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T00:50:38.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:50:37 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:50:38.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:50:37 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:50:38.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:50:37 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:50:38.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:50:37 smithi132 ceph-mon[35941]: pgmap v777: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T00:50:38.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:50:37 smithi132 ceph-mon[35941]: pgmap v778: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T00:50:38.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:50:37 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:50:40.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:50:40 smithi165 ceph-mon[36302]: pgmap v779: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T00:50:40.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:50:40 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 900.674333 secs 2024-03-28T00:50:40.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:50:40 smithi119 ceph-mon[31518]: pgmap v779: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T00:50:40.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:50:40 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 900.674333 secs 2024-03-28T00:50:40.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:50:40 smithi132 ceph-mon[35941]: pgmap v779: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T00:50:40.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:50:40 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 900.674333 secs 2024-03-28T00:50:42.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:50:42 smithi165 ceph-mon[36302]: pgmap v780: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T00:50:42.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:50:42 smithi119 ceph-mon[31518]: pgmap v780: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T00:50:42.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:50:42 smithi132 ceph-mon[35941]: pgmap v780: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T00:50:44.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:50:44 smithi165 ceph-mon[36302]: pgmap v781: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T00:50:44.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:50:44 smithi119 ceph-mon[31518]: pgmap v781: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T00:50:44.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:50:44 smithi132 ceph-mon[35941]: pgmap v781: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T00:50:45.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:50:45 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 905.674396 secs 2024-03-28T00:50:45.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:50:45 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 905.674396 secs 2024-03-28T00:50:45.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:50:45 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 905.674396 secs 2024-03-28T00:50:46.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:50:46 smithi119 ceph-mon[31518]: pgmap v782: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T00:50:46.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:50:46 smithi132 ceph-mon[35941]: pgmap v782: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T00:50:46.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:50:46 smithi165 ceph-mon[36302]: pgmap v782: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T00:50:48.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:50:48 smithi119 ceph-mon[31518]: pgmap v783: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T00:50:48.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:50:48 smithi132 ceph-mon[35941]: pgmap v783: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T00:50:48.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:50:48 smithi165 ceph-mon[36302]: pgmap v783: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T00:50:50.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:50:50 smithi119 ceph-mon[31518]: pgmap v784: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:50:50.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:50:50 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 910.674484 secs 2024-03-28T00:50:50.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:50:50 smithi132 ceph-mon[35941]: pgmap v784: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:50:50.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:50:50 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 910.674484 secs 2024-03-28T00:50:50.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:50:50 smithi165 ceph-mon[36302]: pgmap v784: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:50:50.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:50:50 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 910.674484 secs 2024-03-28T00:50:52.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:50:52 smithi132 ceph-mon[35941]: pgmap v785: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:50:52.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:50:52 smithi165 ceph-mon[36302]: pgmap v785: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:50:52.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:50:52 smithi119 ceph-mon[31518]: pgmap v785: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:50:54.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:50:54 smithi132 ceph-mon[35941]: pgmap v786: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:50:54.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:50:54 smithi165 ceph-mon[36302]: pgmap v786: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:50:54.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:50:54 smithi119 ceph-mon[31518]: pgmap v786: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:50:55.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:50:55 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 915.674574 secs 2024-03-28T00:50:55.804 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:50:55 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 915.674574 secs 2024-03-28T00:50:55.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:50:55 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 915.674574 secs 2024-03-28T00:50:56.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:50:56 smithi132 ceph-mon[35941]: pgmap v787: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:50:56.781 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:50:56 smithi165 ceph-mon[36302]: pgmap v787: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:50:56.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:50:56 smithi119 ceph-mon[31518]: pgmap v787: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:50:58.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:50:58 smithi132 ceph-mon[35941]: pgmap v788: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:50:58.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:50:58 smithi165 ceph-mon[36302]: pgmap v788: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:50:58.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:50:58 smithi119 ceph-mon[31518]: pgmap v788: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:51:00.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:51:00 smithi132 ceph-mon[35941]: pgmap v789: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:51:00.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:51:00 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 920.674636 secs 2024-03-28T00:51:00.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:51:00 smithi165 ceph-mon[36302]: pgmap v789: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:51:00.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:51:00 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 920.674636 secs 2024-03-28T00:51:00.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:51:00 smithi119 ceph-mon[31518]: pgmap v789: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:51:00.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:51:00 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 920.674636 secs 2024-03-28T00:51:02.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:51:02 smithi132 ceph-mon[35941]: pgmap v790: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:51:02.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:51:02 smithi165 ceph-mon[36302]: pgmap v790: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:51:02.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:51:02 smithi119 ceph-mon[31518]: pgmap v790: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:51:04.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:51:04 smithi132 ceph-mon[35941]: pgmap v791: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:51:04.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:51:04 smithi165 ceph-mon[36302]: pgmap v791: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:51:04.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:51:04 smithi119 ceph-mon[31518]: pgmap v791: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:51:05.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:51:05 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 925.674727 secs 2024-03-28T00:51:05.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:51:05 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 925.674727 secs 2024-03-28T00:51:05.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:51:05 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 925.674727 secs 2024-03-28T00:51:06.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:51:06 smithi165 ceph-mon[36302]: pgmap v792: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:51:06.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:51:06 smithi119 ceph-mon[31518]: pgmap v792: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:51:06.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:51:06 smithi132 ceph-mon[35941]: pgmap v792: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:51:08.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:51:08 smithi165 ceph-mon[36302]: pgmap v793: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:51:08.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:51:08 smithi119 ceph-mon[31518]: pgmap v793: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:51:08.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:51:08 smithi132 ceph-mon[35941]: pgmap v793: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:51:10.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:51:10 smithi165 ceph-mon[36302]: pgmap v794: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:51:10.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:51:10 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 930.674812 secs 2024-03-28T00:51:10.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:51:10 smithi119 ceph-mon[31518]: pgmap v794: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:51:10.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:51:10 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 930.674812 secs 2024-03-28T00:51:10.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:51:10 smithi132 ceph-mon[35941]: pgmap v794: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:51:10.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:51:10 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 930.674812 secs 2024-03-28T00:51:12.753 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:51:12 smithi132 ceph-mon[35941]: pgmap v795: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:51:12.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:51:12 smithi165 ceph-mon[36302]: pgmap v795: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:51:12.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:51:12 smithi119 ceph-mon[31518]: pgmap v795: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:51:14.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:51:14 smithi165 ceph-mon[36302]: pgmap v796: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:51:14.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:51:14 smithi119 ceph-mon[31518]: pgmap v796: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:51:14.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:51:14 smithi132 ceph-mon[35941]: pgmap v796: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:51:15.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:51:15 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 935.674850 secs 2024-03-28T00:51:15.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:51:15 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 935.674850 secs 2024-03-28T00:51:15.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:51:15 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 935.674850 secs 2024-03-28T00:51:16.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:51:16 smithi165 ceph-mon[36302]: pgmap v797: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:51:16.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:51:16 smithi119 ceph-mon[31518]: pgmap v797: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:51:16.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:51:16 smithi132 ceph-mon[35941]: pgmap v797: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:51:18.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:51:18 smithi165 ceph-mon[36302]: pgmap v798: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:51:18.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:51:18 smithi119 ceph-mon[31518]: pgmap v798: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:51:18.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:51:18 smithi132 ceph-mon[35941]: pgmap v798: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:51:20.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:51:20 smithi165 ceph-mon[36302]: pgmap v799: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:51:20.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:51:20 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 940.674912 secs 2024-03-28T00:51:20.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:51:20 smithi119 ceph-mon[31518]: pgmap v799: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:51:20.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:51:20 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 940.674912 secs 2024-03-28T00:51:20.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:51:20 smithi132 ceph-mon[35941]: pgmap v799: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:51:20.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:51:20 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 940.674912 secs 2024-03-28T00:51:22.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:51:22 smithi165 ceph-mon[36302]: pgmap v800: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:51:22.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:51:22 smithi119 ceph-mon[31518]: pgmap v800: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:51:22.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:51:22 smithi132 ceph-mon[35941]: pgmap v800: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:51:24.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:51:24 smithi165 ceph-mon[36302]: pgmap v801: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:51:24.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:51:24 smithi119 ceph-mon[31518]: pgmap v801: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:51:24.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:51:24 smithi132 ceph-mon[35941]: pgmap v801: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:51:25.804 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:51:25 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 945.674973 secs 2024-03-28T00:51:25.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:51:25 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 945.674973 secs 2024-03-28T00:51:25.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:51:25 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 945.674973 secs 2024-03-28T00:51:26.781 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:51:26 smithi165 ceph-mon[36302]: pgmap v802: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:51:26.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:51:26 smithi119 ceph-mon[31518]: pgmap v802: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:51:26.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:51:26 smithi132 ceph-mon[35941]: pgmap v802: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:51:28.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:51:28 smithi165 ceph-mon[36302]: pgmap v803: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:51:28.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:51:28 smithi119 ceph-mon[31518]: pgmap v803: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:51:28.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:51:28 smithi132 ceph-mon[35941]: pgmap v803: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:51:30.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:51:30 smithi165 ceph-mon[36302]: pgmap v804: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:51:30.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:51:30 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 950.675032 secs 2024-03-28T00:51:30.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:51:30 smithi119 ceph-mon[31518]: pgmap v804: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:51:30.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:51:30 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 950.675032 secs 2024-03-28T00:51:30.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:51:30 smithi132 ceph-mon[35941]: pgmap v804: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:51:30.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:51:30 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 950.675032 secs 2024-03-28T00:51:32.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:51:32 smithi165 ceph-mon[36302]: pgmap v805: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:51:32.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:51:32 smithi119 ceph-mon[31518]: pgmap v805: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:51:32.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:51:32 smithi132 ceph-mon[35941]: pgmap v805: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:51:34.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:51:34 smithi165 ceph-mon[36302]: pgmap v806: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:51:34.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:51:34 smithi119 ceph-mon[31518]: pgmap v806: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:51:34.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:51:34 smithi132 ceph-mon[35941]: pgmap v806: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:51:35.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:51:35 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 955.675092 secs 2024-03-28T00:51:35.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:51:35 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 955.675092 secs 2024-03-28T00:51:35.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:51:35 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 955.675092 secs 2024-03-28T00:51:36.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:51:36 smithi165 ceph-mon[36302]: pgmap v807: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:51:36.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:51:36 smithi119 ceph-mon[31518]: pgmap v807: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:51:36.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:51:36 smithi132 ceph-mon[35941]: pgmap v807: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:51:38.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:51:38 smithi165 ceph-mon[36302]: pgmap v808: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:51:38.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:51:38 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:51:38.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:51:38 smithi119 ceph-mon[31518]: pgmap v808: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:51:38.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:51:38 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:51:38.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:51:38 smithi132 ceph-mon[35941]: pgmap v808: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:51:38.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:51:38 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:51:39.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:51:39 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:51:39.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:51:39 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:51:39.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:51:39 smithi165 ceph-mon[36302]: pgmap v809: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T00:51:39.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:51:39 smithi165 ceph-mon[36302]: pgmap v810: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-28T00:51:39.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:51:39 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:51:39.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:51:39 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:51:39.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:51:39 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:51:39.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:51:39 smithi119 ceph-mon[31518]: pgmap v809: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T00:51:39.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:51:39 smithi119 ceph-mon[31518]: pgmap v810: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-28T00:51:39.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:51:39 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:51:39.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:51:39 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:51:39.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:51:39 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:51:39.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:51:39 smithi132 ceph-mon[35941]: pgmap v809: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T00:51:39.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:51:39 smithi132 ceph-mon[35941]: pgmap v810: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-28T00:51:39.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:51:39 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:51:40.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:51:40 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 960.675169 secs 2024-03-28T00:51:40.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:51:40 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 960.675169 secs 2024-03-28T00:51:40.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:51:40 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 960.675169 secs 2024-03-28T00:51:41.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:51:41 smithi165 ceph-mon[36302]: pgmap v811: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T00:51:41.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:51:41 smithi119 ceph-mon[31518]: pgmap v811: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T00:51:41.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:51:41 smithi132 ceph-mon[35941]: pgmap v811: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T00:51:44.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:51:43 smithi165 ceph-mon[36302]: pgmap v812: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T00:51:44.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:51:43 smithi119 ceph-mon[31518]: pgmap v812: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T00:51:44.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:51:43 smithi132 ceph-mon[35941]: pgmap v812: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T00:51:46.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:51:45 smithi119 ceph-mon[31518]: pgmap v813: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T00:51:46.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:51:45 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 965.675229 secs 2024-03-28T00:51:46.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:51:45 smithi132 ceph-mon[35941]: pgmap v813: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T00:51:46.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:51:45 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 965.675229 secs 2024-03-28T00:51:46.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:51:45 smithi165 ceph-mon[36302]: pgmap v813: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T00:51:46.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:51:45 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 965.675229 secs 2024-03-28T00:51:48.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:51:47 smithi119 ceph-mon[31518]: pgmap v814: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T00:51:48.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:51:47 smithi132 ceph-mon[35941]: pgmap v814: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T00:51:48.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:51:47 smithi165 ceph-mon[36302]: pgmap v814: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T00:51:50.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:51:49 smithi119 ceph-mon[31518]: pgmap v815: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-28T00:51:50.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:51:49 smithi132 ceph-mon[35941]: pgmap v815: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-28T00:51:50.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:51:49 smithi165 ceph-mon[36302]: pgmap v815: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-28T00:51:51.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:51:50 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 970.675316 secs 2024-03-28T00:51:51.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:51:50 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 970.675316 secs 2024-03-28T00:51:51.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:51:50 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 970.675316 secs 2024-03-28T00:51:52.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:51:51 smithi132 ceph-mon[35941]: pgmap v816: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:51:52.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:51:51 smithi165 ceph-mon[36302]: pgmap v816: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:51:52.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:51:51 smithi119 ceph-mon[31518]: pgmap v816: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:51:54.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:51:53 smithi132 ceph-mon[35941]: pgmap v817: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T00:51:54.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:51:53 smithi165 ceph-mon[36302]: pgmap v817: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T00:51:54.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:51:53 smithi119 ceph-mon[31518]: pgmap v817: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T00:51:56.182 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:51:55 smithi165 ceph-mon[36302]: pgmap v818: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:51:56.182 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:51:55 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 975.675377 secs 2024-03-28T00:51:56.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:51:55 smithi132 ceph-mon[35941]: pgmap v818: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:51:56.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:51:55 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 975.675377 secs 2024-03-28T00:51:56.205 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:51:55 smithi119 ceph-mon[31518]: pgmap v818: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:51:56.205 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:51:55 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 975.675377 secs 2024-03-28T00:51:58.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:51:57 smithi132 ceph-mon[35941]: pgmap v819: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:51:58.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:51:57 smithi165 ceph-mon[36302]: pgmap v819: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:51:58.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:51:57 smithi119 ceph-mon[31518]: pgmap v819: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:52:00.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:51:59 smithi132 ceph-mon[35941]: pgmap v820: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:52:00.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:51:59 smithi165 ceph-mon[36302]: pgmap v820: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:52:00.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:51:59 smithi119 ceph-mon[31518]: pgmap v820: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:52:01.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:52:00 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 980.675463 secs 2024-03-28T00:52:01.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:52:00 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 980.675463 secs 2024-03-28T00:52:01.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:52:00 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 980.675463 secs 2024-03-28T00:52:02.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:52:01 smithi132 ceph-mon[35941]: pgmap v821: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:52:02.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:52:01 smithi165 ceph-mon[36302]: pgmap v821: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:52:02.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:52:01 smithi119 ceph-mon[31518]: pgmap v821: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:52:04.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:52:03 smithi132 ceph-mon[35941]: pgmap v822: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:52:04.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:52:03 smithi165 ceph-mon[36302]: pgmap v822: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:52:04.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:52:03 smithi119 ceph-mon[31518]: pgmap v822: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:52:06.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:52:05 smithi165 ceph-mon[36302]: pgmap v823: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:52:06.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:52:05 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 985.675521 secs 2024-03-28T00:52:06.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:52:05 smithi119 ceph-mon[31518]: pgmap v823: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:52:06.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:52:05 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 985.675521 secs 2024-03-28T00:52:06.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:52:05 smithi132 ceph-mon[35941]: pgmap v823: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:52:06.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:52:05 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 985.675521 secs 2024-03-28T00:52:08.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:52:07 smithi165 ceph-mon[36302]: pgmap v824: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:52:08.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:52:07 smithi119 ceph-mon[31518]: pgmap v824: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:52:08.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:52:07 smithi132 ceph-mon[35941]: pgmap v824: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:52:10.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:52:09 smithi165 ceph-mon[36302]: pgmap v825: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:52:10.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:52:09 smithi119 ceph-mon[31518]: pgmap v825: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:52:10.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:52:09 smithi132 ceph-mon[35941]: pgmap v825: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:52:11.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:52:10 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 990.675582 secs 2024-03-28T00:52:11.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:52:10 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 990.675582 secs 2024-03-28T00:52:11.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:52:10 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 990.675582 secs 2024-03-28T00:52:12.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:52:11 smithi165 ceph-mon[36302]: pgmap v826: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:52:12.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:52:11 smithi119 ceph-mon[31518]: pgmap v826: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:52:12.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:52:11 smithi132 ceph-mon[35941]: pgmap v826: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:52:14.252 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:52:13 smithi132 ceph-mon[35941]: pgmap v827: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:52:14.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:52:13 smithi165 ceph-mon[36302]: pgmap v827: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:52:14.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:52:13 smithi119 ceph-mon[31518]: pgmap v827: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:52:16.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:52:15 smithi165 ceph-mon[36302]: pgmap v828: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:52:16.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:52:15 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 995.675671 secs 2024-03-28T00:52:16.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:52:15 smithi119 ceph-mon[31518]: pgmap v828: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:52:16.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:52:15 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 995.675671 secs 2024-03-28T00:52:16.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:52:15 smithi132 ceph-mon[35941]: pgmap v828: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:52:16.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:52:15 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 995.675671 secs 2024-03-28T00:52:18.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:52:17 smithi165 ceph-mon[36302]: pgmap v829: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:52:18.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:52:17 smithi119 ceph-mon[31518]: pgmap v829: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:52:18.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:52:17 smithi132 ceph-mon[35941]: pgmap v829: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:52:20.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:52:19 smithi165 ceph-mon[36302]: pgmap v830: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:52:20.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:52:19 smithi119 ceph-mon[31518]: pgmap v830: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:52:20.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:52:19 smithi132 ceph-mon[35941]: pgmap v830: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:52:21.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:52:20 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1000.675711 secs 2024-03-28T00:52:21.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:52:20 smithi165 ceph-mon[36302]: pgmap v831: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:52:21.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:52:20 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1000.675711 secs 2024-03-28T00:52:21.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:52:20 smithi119 ceph-mon[31518]: pgmap v831: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:52:21.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:52:20 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1000.675711 secs 2024-03-28T00:52:21.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:52:20 smithi132 ceph-mon[35941]: pgmap v831: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:52:24.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:52:23 smithi119 ceph-mon[31518]: pgmap v832: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:52:24.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:52:23 smithi132 ceph-mon[35941]: pgmap v832: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:52:24.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:52:23 smithi165 ceph-mon[36302]: pgmap v832: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:52:26.182 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:52:25 smithi165 ceph-mon[36302]: pgmap v833: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:52:26.182 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:52:25 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1005.675801 secs 2024-03-28T00:52:26.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:52:25 smithi132 ceph-mon[35941]: pgmap v833: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:52:26.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:52:25 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1005.675801 secs 2024-03-28T00:52:26.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:52:25 smithi119 ceph-mon[31518]: pgmap v833: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:52:26.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:52:25 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1005.675801 secs 2024-03-28T00:52:28.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:52:27 smithi132 ceph-mon[35941]: pgmap v834: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:52:28.205 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:52:27 smithi119 ceph-mon[31518]: pgmap v834: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:52:28.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:52:27 smithi165 ceph-mon[36302]: pgmap v834: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:52:30.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:52:29 smithi132 ceph-mon[35941]: pgmap v835: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:52:30.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:52:29 smithi165 ceph-mon[36302]: pgmap v835: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:52:30.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:52:29 smithi119 ceph-mon[31518]: pgmap v835: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:52:31.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:52:30 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1010.675887 secs 2024-03-28T00:52:31.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:52:30 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1010.675887 secs 2024-03-28T00:52:31.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:52:30 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1010.675887 secs 2024-03-28T00:52:32.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:52:31 smithi132 ceph-mon[35941]: pgmap v836: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:52:32.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:52:31 smithi165 ceph-mon[36302]: pgmap v836: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:52:32.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:52:31 smithi119 ceph-mon[31518]: pgmap v836: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:52:34.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:52:33 smithi132 ceph-mon[35941]: pgmap v837: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:52:34.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:52:33 smithi165 ceph-mon[36302]: pgmap v837: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:52:34.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:52:33 smithi119 ceph-mon[31518]: pgmap v837: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:52:36.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:52:35 smithi132 ceph-mon[35941]: pgmap v838: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:52:36.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:52:35 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1015.675970 secs 2024-03-28T00:52:36.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:52:35 smithi165 ceph-mon[36302]: pgmap v838: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:52:36.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:52:35 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1015.675970 secs 2024-03-28T00:52:36.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:52:35 smithi119 ceph-mon[31518]: pgmap v838: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:52:36.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:52:35 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1015.675970 secs 2024-03-28T00:52:38.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:52:37 smithi132 ceph-mon[35941]: pgmap v839: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:52:38.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:52:37 smithi165 ceph-mon[36302]: pgmap v839: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:52:38.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:52:37 smithi119 ceph-mon[31518]: pgmap v839: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:52:40.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:52:39 smithi132 ceph-mon[35941]: pgmap v840: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:52:40.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:52:39 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:52:40.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:52:39 smithi165 ceph-mon[36302]: pgmap v840: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:52:40.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:52:39 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:52:40.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:52:39 smithi119 ceph-mon[31518]: pgmap v840: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:52:40.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:52:39 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:52:41.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:52:40 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1020.676036 secs 2024-03-28T00:52:41.327 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:52:40 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1020.676036 secs 2024-03-28T00:52:41.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:52:40 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1020.676036 secs 2024-03-28T00:52:42.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:52:41 smithi132 ceph-mon[35941]: pgmap v841: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:52:42.213 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:52:41 smithi119 ceph-mon[31518]: pgmap v841: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:52:42.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:52:41 smithi165 ceph-mon[36302]: pgmap v841: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:52:44.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:52:43 smithi165 ceph-mon[36302]: pgmap v842: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:52:44.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:52:43 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:52:44.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:52:43 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:52:44.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:52:43 smithi119 ceph-mon[31518]: pgmap v842: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:52:44.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:52:43 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:52:44.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:52:43 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:52:44.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:52:43 smithi132 ceph-mon[35941]: pgmap v842: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:52:44.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:52:43 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:52:44.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:52:43 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:52:46.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:52:45 smithi165 ceph-mon[36302]: pgmap v843: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:52:46.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:52:45 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1025.676120 secs 2024-03-28T00:52:46.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:52:45 smithi119 ceph-mon[31518]: pgmap v843: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:52:46.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:52:45 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1025.676120 secs 2024-03-28T00:52:46.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:52:45 smithi132 ceph-mon[35941]: pgmap v843: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:52:46.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:52:45 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1025.676120 secs 2024-03-28T00:52:48.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:52:47 smithi165 ceph-mon[36302]: pgmap v844: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:52:48.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:52:47 smithi119 ceph-mon[31518]: pgmap v844: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:52:48.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:52:47 smithi132 ceph-mon[35941]: pgmap v844: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:52:49.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:52:49 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:52:49.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:52:49 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:52:49.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:52:49 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:52:49.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:52:49 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:52:49.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:52:49 smithi165 ceph-mon[36302]: pgmap v845: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T00:52:49.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:52:49 smithi165 ceph-mon[36302]: pgmap v846: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T00:52:49.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:52:49 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:52:49.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:52:49 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:52:49.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:52:49 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:52:49.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:52:49 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:52:49.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:52:49 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:52:49.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:52:49 smithi119 ceph-mon[31518]: pgmap v845: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T00:52:49.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:52:49 smithi119 ceph-mon[31518]: pgmap v846: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T00:52:49.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:52:49 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:52:49.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:52:49 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:52:49.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:52:49 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:52:49.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:52:49 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:52:49.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:52:49 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:52:49.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:52:49 smithi132 ceph-mon[35941]: pgmap v845: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T00:52:49.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:52:49 smithi132 ceph-mon[35941]: pgmap v846: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T00:52:49.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:52:49 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:52:50.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:52:50 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1030.676209 secs 2024-03-28T00:52:50.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:52:50 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1030.676209 secs 2024-03-28T00:52:50.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:52:50 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1030.676209 secs 2024-03-28T00:52:51.875 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:52:51 smithi119 ceph-mon[31518]: pgmap v847: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T00:52:51.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:52:51 smithi165 ceph-mon[36302]: pgmap v847: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T00:52:51.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:52:51 smithi132 ceph-mon[35941]: pgmap v847: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T00:52:53.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:52:53 smithi165 ceph-mon[36302]: pgmap v848: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T00:52:53.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:52:53 smithi119 ceph-mon[31518]: pgmap v848: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T00:52:53.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:52:53 smithi132 ceph-mon[35941]: pgmap v848: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T00:52:55.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:52:55 smithi165 ceph-mon[36302]: pgmap v849: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T00:52:55.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:52:55 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1035.676270 secs 2024-03-28T00:52:55.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:52:55 smithi119 ceph-mon[31518]: pgmap v849: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T00:52:55.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:52:55 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1035.676270 secs 2024-03-28T00:52:55.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:52:55 smithi132 ceph-mon[35941]: pgmap v849: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T00:52:55.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:52:55 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1035.676270 secs 2024-03-28T00:52:57.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:52:57 smithi165 ceph-mon[36302]: pgmap v850: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T00:52:57.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:52:57 smithi119 ceph-mon[31518]: pgmap v850: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T00:52:57.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:52:57 smithi132 ceph-mon[35941]: pgmap v850: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T00:52:59.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:52:59 smithi165 ceph-mon[36302]: pgmap v851: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T00:52:59.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:52:59 smithi119 ceph-mon[31518]: pgmap v851: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T00:52:59.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:52:59 smithi132 ceph-mon[35941]: pgmap v851: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T00:53:00.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:53:00 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1040.676360 secs 2024-03-28T00:53:00.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:53:00 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1040.676360 secs 2024-03-28T00:53:00.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:53:00 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1040.676360 secs 2024-03-28T00:53:01.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:53:01 smithi119 ceph-mon[31518]: pgmap v852: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:53:01.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:53:01 smithi165 ceph-mon[36302]: pgmap v852: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:53:01.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:53:01 smithi132 ceph-mon[35941]: pgmap v852: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:53:03.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:53:03 smithi165 ceph-mon[36302]: pgmap v853: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:53:03.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:53:03 smithi119 ceph-mon[31518]: pgmap v853: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:53:03.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:53:03 smithi132 ceph-mon[35941]: pgmap v853: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:53:05.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:53:05 smithi165 ceph-mon[36302]: pgmap v854: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:53:05.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:53:05 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1045.676447 secs 2024-03-28T00:53:05.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:53:05 smithi119 ceph-mon[31518]: pgmap v854: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:53:05.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:53:05 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1045.676447 secs 2024-03-28T00:53:05.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:53:05 smithi132 ceph-mon[35941]: pgmap v854: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:53:05.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:53:05 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1045.676447 secs 2024-03-28T00:53:07.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:53:07 smithi165 ceph-mon[36302]: pgmap v855: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:53:07.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:53:07 smithi119 ceph-mon[31518]: pgmap v855: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:53:07.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:53:07 smithi132 ceph-mon[35941]: pgmap v855: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:53:09.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:53:09 smithi165 ceph-mon[36302]: pgmap v856: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:53:09.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:53:09 smithi119 ceph-mon[31518]: pgmap v856: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:53:09.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:53:09 smithi132 ceph-mon[35941]: pgmap v856: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:53:10.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:53:10 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1050.676538 secs 2024-03-28T00:53:10.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:53:10 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1050.676538 secs 2024-03-28T00:53:10.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:53:10 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1050.676538 secs 2024-03-28T00:53:11.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:53:11 smithi165 ceph-mon[36302]: pgmap v857: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:53:11.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:53:11 smithi119 ceph-mon[31518]: pgmap v857: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:53:11.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:53:11 smithi132 ceph-mon[35941]: pgmap v857: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:53:13.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:53:13 smithi165 ceph-mon[36302]: pgmap v858: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:53:13.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:53:13 smithi119 ceph-mon[31518]: pgmap v858: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:53:13.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:53:13 smithi132 ceph-mon[35941]: pgmap v858: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:53:15.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:53:15 smithi165 ceph-mon[36302]: pgmap v859: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:53:15.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:53:15 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1055.676626 secs 2024-03-28T00:53:15.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:53:15 smithi119 ceph-mon[31518]: pgmap v859: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:53:15.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:53:15 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1055.676626 secs 2024-03-28T00:53:15.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:53:15 smithi132 ceph-mon[35941]: pgmap v859: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:53:15.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:53:15 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1055.676626 secs 2024-03-28T00:53:17.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:53:17 smithi165 ceph-mon[36302]: pgmap v860: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:53:17.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:53:17 smithi119 ceph-mon[31518]: pgmap v860: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:53:17.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:53:17 smithi132 ceph-mon[35941]: pgmap v860: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:53:19.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:53:19 smithi165 ceph-mon[36302]: pgmap v861: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:53:19.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:53:19 smithi119 ceph-mon[31518]: pgmap v861: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:53:19.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:53:19 smithi132 ceph-mon[35941]: pgmap v861: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:53:20.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:53:20 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1060.676686 secs 2024-03-28T00:53:20.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:53:20 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1060.676686 secs 2024-03-28T00:53:20.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:53:20 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1060.676686 secs 2024-03-28T00:53:21.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:53:21 smithi119 ceph-mon[31518]: pgmap v862: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:53:21.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:53:21 smithi132 ceph-mon[35941]: pgmap v862: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:53:22.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:53:21 smithi165 ceph-mon[36302]: pgmap v862: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:53:23.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:53:23 smithi119 ceph-mon[31518]: pgmap v863: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:53:23.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:53:23 smithi132 ceph-mon[35941]: pgmap v863: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:53:24.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:53:23 smithi165 ceph-mon[36302]: pgmap v863: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:53:25.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:53:25 smithi119 ceph-mon[31518]: pgmap v864: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:53:25.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:53:25 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1065.676776 secs 2024-03-28T00:53:25.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:53:25 smithi132 ceph-mon[35941]: pgmap v864: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:53:25.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:53:25 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1065.676776 secs 2024-03-28T00:53:26.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:53:25 smithi165 ceph-mon[36302]: pgmap v864: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:53:26.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:53:25 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1065.676776 secs 2024-03-28T00:53:27.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:53:27 smithi119 ceph-mon[31518]: pgmap v865: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:53:27.931 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:53:27 smithi165 ceph-mon[36302]: pgmap v865: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:53:27.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:53:27 smithi132 ceph-mon[35941]: pgmap v865: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:53:29.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:53:29 smithi119 ceph-mon[31518]: pgmap v866: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:53:29.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:53:29 smithi132 ceph-mon[35941]: pgmap v866: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:53:30.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:53:29 smithi165 ceph-mon[36302]: pgmap v866: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:53:30.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:53:30 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1070.676863 secs 2024-03-28T00:53:31.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:53:30 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1070.676863 secs 2024-03-28T00:53:31.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:53:30 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1070.676863 secs 2024-03-28T00:53:31.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:53:31 smithi132 ceph-mon[35941]: pgmap v867: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:53:32.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:53:31 smithi165 ceph-mon[36302]: pgmap v867: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:53:32.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:53:31 smithi119 ceph-mon[31518]: pgmap v867: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:53:33.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:53:33 smithi132 ceph-mon[35941]: pgmap v868: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:53:34.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:53:33 smithi165 ceph-mon[36302]: pgmap v868: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:53:34.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:53:33 smithi119 ceph-mon[31518]: pgmap v868: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:53:35.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:53:35 smithi132 ceph-mon[35941]: pgmap v869: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:53:35.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:53:35 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1075.676924 secs 2024-03-28T00:53:36.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:53:35 smithi165 ceph-mon[36302]: pgmap v869: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:53:36.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:53:35 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1075.676924 secs 2024-03-28T00:53:36.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:53:35 smithi119 ceph-mon[31518]: pgmap v869: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:53:36.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:53:35 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1075.676924 secs 2024-03-28T00:53:37.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:53:37 smithi132 ceph-mon[35941]: pgmap v870: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:53:38.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:53:37 smithi165 ceph-mon[36302]: pgmap v870: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:53:38.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:53:37 smithi119 ceph-mon[31518]: pgmap v870: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:53:39.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:53:39 smithi132 ceph-mon[35941]: pgmap v871: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:53:40.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:53:39 smithi165 ceph-mon[36302]: pgmap v871: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:53:40.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:53:39 smithi119 ceph-mon[31518]: pgmap v871: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:53:40.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:53:40 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1080.677012 secs 2024-03-28T00:53:41.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:53:40 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1080.677012 secs 2024-03-28T00:53:41.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:53:40 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1080.677012 secs 2024-03-28T00:53:41.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:53:41 smithi132 ceph-mon[35941]: pgmap v872: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:53:42.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:53:41 smithi165 ceph-mon[36302]: pgmap v872: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:53:42.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:53:41 smithi119 ceph-mon[31518]: pgmap v872: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:53:43.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:53:43 smithi132 ceph-mon[35941]: pgmap v873: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:53:44.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:53:43 smithi165 ceph-mon[36302]: pgmap v873: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:53:44.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:53:43 smithi119 ceph-mon[31518]: pgmap v873: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:53:45.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:53:45 smithi132 ceph-mon[35941]: pgmap v874: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:53:45.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:53:45 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1085.677074 secs 2024-03-28T00:53:46.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:53:45 smithi165 ceph-mon[36302]: pgmap v874: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:53:46.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:53:45 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1085.677074 secs 2024-03-28T00:53:46.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:53:45 smithi119 ceph-mon[31518]: pgmap v874: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:53:46.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:53:45 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1085.677074 secs 2024-03-28T00:53:48.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:53:47 smithi165 ceph-mon[36302]: pgmap v875: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:53:48.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:53:47 smithi119 ceph-mon[31518]: pgmap v875: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:53:48.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:53:47 smithi132 ceph-mon[35941]: pgmap v875: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:53:50.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:53:49 smithi165 ceph-mon[36302]: pgmap v876: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:53:50.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:53:49 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:53:50.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:53:49 smithi119 ceph-mon[31518]: pgmap v876: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:53:50.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:53:49 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:53:50.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:53:49 smithi132 ceph-mon[35941]: pgmap v876: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:53:50.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:53:49 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:53:50.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:53:50 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1090.677135 secs 2024-03-28T00:53:51.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:53:50 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1090.677135 secs 2024-03-28T00:53:51.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:53:50 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1090.677135 secs 2024-03-28T00:53:51.773 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:53:51 smithi165 ceph-mon[36302]: pgmap v877: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:53:52.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:53:51 smithi119 ceph-mon[31518]: pgmap v877: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:53:52.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:53:51 smithi132 ceph-mon[35941]: pgmap v877: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:53:54.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:53:53 smithi165 ceph-mon[36302]: pgmap v878: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:53:54.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:53:53 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:53:54.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:53:53 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:53:54.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:53:53 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:53:54.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:53:53 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:53:54.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:53:53 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:53:54.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:53:53 smithi119 ceph-mon[31518]: pgmap v878: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:53:54.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:53:53 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:53:54.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:53:53 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:53:54.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:53:53 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:53:54.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:53:53 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:53:54.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:53:53 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:53:54.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:53:53 smithi132 ceph-mon[35941]: pgmap v878: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:53:54.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:53:53 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:53:54.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:53:53 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:53:54.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:53:53 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:53:54.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:53:53 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:53:54.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:53:53 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:53:55.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:53:54 smithi165 ceph-mon[36302]: pgmap v879: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-03-28T00:53:55.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:53:54 smithi165 ceph-mon[36302]: pgmap v880: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-28T00:53:55.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:53:54 smithi119 ceph-mon[31518]: pgmap v879: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-03-28T00:53:55.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:53:54 smithi119 ceph-mon[31518]: pgmap v880: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-28T00:53:55.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:53:54 smithi132 ceph-mon[35941]: pgmap v879: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-03-28T00:53:55.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:53:54 smithi132 ceph-mon[35941]: pgmap v880: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-28T00:53:56.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:53:55 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1095.677194 secs 2024-03-28T00:53:56.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:53:55 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1095.677194 secs 2024-03-28T00:53:56.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:53:55 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1095.677194 secs 2024-03-28T00:53:57.030 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:53:56 smithi165 ceph-mon[36302]: pgmap v881: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T00:53:57.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:53:56 smithi119 ceph-mon[31518]: pgmap v881: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T00:53:57.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:53:56 smithi132 ceph-mon[35941]: pgmap v881: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T00:53:59.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:53:58 smithi165 ceph-mon[36302]: pgmap v882: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T00:53:59.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:53:58 smithi119 ceph-mon[31518]: pgmap v882: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T00:53:59.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:53:58 smithi132 ceph-mon[35941]: pgmap v882: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T00:54:01.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:54:00 smithi165 ceph-mon[36302]: pgmap v883: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T00:54:01.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:54:00 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1100.677280 secs 2024-03-28T00:54:01.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:54:00 smithi119 ceph-mon[31518]: pgmap v883: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T00:54:01.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:54:00 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1100.677280 secs 2024-03-28T00:54:01.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:54:00 smithi132 ceph-mon[35941]: pgmap v883: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T00:54:01.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:54:00 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1100.677280 secs 2024-03-28T00:54:03.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:54:02 smithi165 ceph-mon[36302]: pgmap v884: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T00:54:03.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:54:02 smithi119 ceph-mon[31518]: pgmap v884: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T00:54:03.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:54:02 smithi132 ceph-mon[35941]: pgmap v884: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T00:54:05.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:54:04 smithi165 ceph-mon[36302]: pgmap v885: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T00:54:05.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:54:04 smithi119 ceph-mon[31518]: pgmap v885: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T00:54:05.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:54:04 smithi132 ceph-mon[35941]: pgmap v885: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T00:54:06.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:54:05 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1105.677341 secs 2024-03-28T00:54:06.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:54:05 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1105.677341 secs 2024-03-28T00:54:06.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:54:05 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1105.677341 secs 2024-03-28T00:54:07.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:54:06 smithi165 ceph-mon[36302]: pgmap v886: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:54:07.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:54:06 smithi119 ceph-mon[31518]: pgmap v886: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:54:07.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:54:06 smithi132 ceph-mon[35941]: pgmap v886: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:54:09.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:54:08 smithi165 ceph-mon[36302]: pgmap v887: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:54:09.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:54:08 smithi119 ceph-mon[31518]: pgmap v887: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:54:09.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:54:08 smithi132 ceph-mon[35941]: pgmap v887: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:54:11.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:54:10 smithi165 ceph-mon[36302]: pgmap v888: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:54:11.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:54:10 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1110.677428 secs 2024-03-28T00:54:11.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:54:10 smithi119 ceph-mon[31518]: pgmap v888: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:54:11.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:54:10 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1110.677428 secs 2024-03-28T00:54:11.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:54:10 smithi132 ceph-mon[35941]: pgmap v888: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:54:11.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:54:10 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1110.677428 secs 2024-03-28T00:54:13.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:54:12 smithi165 ceph-mon[36302]: pgmap v889: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:54:13.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:54:12 smithi119 ceph-mon[31518]: pgmap v889: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:54:13.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:54:12 smithi132 ceph-mon[35941]: pgmap v889: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:54:15.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:54:14 smithi165 ceph-mon[36302]: pgmap v890: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:54:15.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:54:14 smithi119 ceph-mon[31518]: pgmap v890: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:54:15.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:54:14 smithi132 ceph-mon[35941]: pgmap v890: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:54:16.101 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:54:15 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1115.677515 secs 2024-03-28T00:54:16.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:54:15 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1115.677515 secs 2024-03-28T00:54:16.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:54:15 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1115.677515 secs 2024-03-28T00:54:17.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:54:16 smithi165 ceph-mon[36302]: pgmap v891: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:54:17.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:54:16 smithi119 ceph-mon[31518]: pgmap v891: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:54:17.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:54:16 smithi132 ceph-mon[35941]: pgmap v891: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:54:19.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:54:18 smithi165 ceph-mon[36302]: pgmap v892: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:54:19.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:54:18 smithi119 ceph-mon[31518]: pgmap v892: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:54:19.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:54:18 smithi132 ceph-mon[35941]: pgmap v892: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:54:21.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:54:20 smithi165 ceph-mon[36302]: pgmap v893: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:54:21.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:54:20 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1120.677602 secs 2024-03-28T00:54:21.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:54:20 smithi119 ceph-mon[31518]: pgmap v893: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:54:21.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:54:20 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1120.677602 secs 2024-03-28T00:54:21.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:54:20 smithi132 ceph-mon[35941]: pgmap v893: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:54:21.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:54:20 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1120.677602 secs 2024-03-28T00:54:23.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:54:22 smithi165 ceph-mon[36302]: pgmap v894: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:54:23.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:54:22 smithi119 ceph-mon[31518]: pgmap v894: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:54:23.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:54:22 smithi132 ceph-mon[35941]: pgmap v894: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:54:25.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:54:24 smithi165 ceph-mon[36302]: pgmap v895: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:54:25.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:54:24 smithi119 ceph-mon[31518]: pgmap v895: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:54:25.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:54:24 smithi132 ceph-mon[35941]: pgmap v895: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:54:26.083 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:54:25 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1125.677643 secs 2024-03-28T00:54:26.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:54:25 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1125.677643 secs 2024-03-28T00:54:26.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:54:25 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1125.677643 secs 2024-03-28T00:54:27.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:54:26 smithi165 ceph-mon[36302]: pgmap v896: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:54:27.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:54:26 smithi119 ceph-mon[31518]: pgmap v896: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:54:27.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:54:26 smithi132 ceph-mon[35941]: pgmap v896: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:54:29.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:54:28 smithi165 ceph-mon[36302]: pgmap v897: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:54:29.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:54:28 smithi119 ceph-mon[31518]: pgmap v897: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:54:29.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:54:28 smithi132 ceph-mon[35941]: pgmap v897: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:54:31.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:54:30 smithi165 ceph-mon[36302]: pgmap v898: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:54:31.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:54:30 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1130.677732 secs 2024-03-28T00:54:31.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:54:30 smithi119 ceph-mon[31518]: pgmap v898: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:54:31.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:54:30 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1130.677732 secs 2024-03-28T00:54:31.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:54:30 smithi132 ceph-mon[35941]: pgmap v898: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:54:31.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:54:30 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1130.677732 secs 2024-03-28T00:54:33.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:54:32 smithi165 ceph-mon[36302]: pgmap v899: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:54:33.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:54:32 smithi119 ceph-mon[31518]: pgmap v899: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:54:33.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:54:32 smithi132 ceph-mon[35941]: pgmap v899: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:54:35.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:54:34 smithi165 ceph-mon[36302]: pgmap v900: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:54:35.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:54:34 smithi119 ceph-mon[31518]: pgmap v900: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:54:35.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:54:34 smithi132 ceph-mon[35941]: pgmap v900: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:54:36.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:54:35 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1135.677822 secs 2024-03-28T00:54:36.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:54:35 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1135.677822 secs 2024-03-28T00:54:36.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:54:35 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1135.677822 secs 2024-03-28T00:54:37.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:54:36 smithi165 ceph-mon[36302]: pgmap v901: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:54:37.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:54:36 smithi119 ceph-mon[31518]: pgmap v901: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:54:37.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:54:36 smithi132 ceph-mon[35941]: pgmap v901: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:54:39.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:54:38 smithi165 ceph-mon[36302]: pgmap v902: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:54:39.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:54:38 smithi119 ceph-mon[31518]: pgmap v902: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:54:39.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:54:38 smithi132 ceph-mon[35941]: pgmap v902: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:54:41.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:54:40 smithi165 ceph-mon[36302]: pgmap v903: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:54:41.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:54:40 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1140.677917 secs 2024-03-28T00:54:41.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:54:40 smithi119 ceph-mon[31518]: pgmap v903: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:54:41.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:54:40 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1140.677917 secs 2024-03-28T00:54:41.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:54:40 smithi132 ceph-mon[35941]: pgmap v903: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:54:41.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:54:40 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1140.677917 secs 2024-03-28T00:54:43.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:54:42 smithi165 ceph-mon[36302]: pgmap v904: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:54:43.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:54:42 smithi119 ceph-mon[31518]: pgmap v904: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:54:43.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:54:42 smithi132 ceph-mon[35941]: pgmap v904: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:54:45.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:54:44 smithi165 ceph-mon[36302]: pgmap v905: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:54:45.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:54:44 smithi119 ceph-mon[31518]: pgmap v905: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:54:45.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:54:44 smithi132 ceph-mon[35941]: pgmap v905: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:54:46.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:54:45 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1145.677953 secs 2024-03-28T00:54:46.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:54:45 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1145.677953 secs 2024-03-28T00:54:46.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:54:45 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1145.677953 secs 2024-03-28T00:54:47.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:54:46 smithi119 ceph-mon[31518]: pgmap v906: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:54:47.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:54:46 smithi132 ceph-mon[35941]: pgmap v906: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:54:47.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:54:46 smithi165 ceph-mon[36302]: pgmap v906: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:54:49.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:54:48 smithi119 ceph-mon[31518]: pgmap v907: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:54:49.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:54:48 smithi132 ceph-mon[35941]: pgmap v907: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:54:49.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:54:48 smithi165 ceph-mon[36302]: pgmap v907: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:54:51.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:54:50 smithi119 ceph-mon[31518]: pgmap v908: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:54:51.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:54:50 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1150.678013 secs 2024-03-28T00:54:51.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:54:50 smithi132 ceph-mon[35941]: pgmap v908: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:54:51.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:54:50 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1150.678013 secs 2024-03-28T00:54:51.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:54:50 smithi165 ceph-mon[36302]: pgmap v908: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:54:51.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:54:50 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1150.678013 secs 2024-03-28T00:54:53.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:54:52 smithi132 ceph-mon[35941]: pgmap v909: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:54:53.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:54:52 smithi165 ceph-mon[36302]: pgmap v909: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:54:53.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:54:52 smithi119 ceph-mon[31518]: pgmap v909: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:54:54.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:54:53 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:54:54.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:54:53 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:54:54.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:54:53 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:54:55.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:54:54 smithi132 ceph-mon[35941]: pgmap v910: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:54:55.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:54:54 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:54:55.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:54:54 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:54:55.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:54:54 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:54:55.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:54:54 smithi165 ceph-mon[36302]: pgmap v910: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:54:55.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:54:54 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:54:55.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:54:54 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:54:55.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:54:54 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:54:55.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:54:54 smithi119 ceph-mon[31518]: pgmap v910: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:54:55.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:54:54 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:54:55.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:54:54 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:54:55.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:54:54 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:54:56.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:54:55 smithi132 ceph-mon[35941]: pgmap v911: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T00:54:56.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:54:55 smithi132 ceph-mon[35941]: pgmap v912: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T00:54:56.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:54:55 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1155.678033 secs 2024-03-28T00:54:56.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:54:55 smithi165 ceph-mon[36302]: pgmap v911: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T00:54:56.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:54:55 smithi165 ceph-mon[36302]: pgmap v912: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T00:54:56.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:54:55 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1155.678033 secs 2024-03-28T00:54:56.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:54:55 smithi119 ceph-mon[31518]: pgmap v911: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T00:54:56.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:54:55 smithi119 ceph-mon[31518]: pgmap v912: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T00:54:56.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:54:55 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1155.678033 secs 2024-03-28T00:54:58.181 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:54:57 smithi165 ceph-mon[36302]: pgmap v913: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T00:54:58.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:54:57 smithi132 ceph-mon[35941]: pgmap v913: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T00:54:58.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:54:57 smithi119 ceph-mon[31518]: pgmap v913: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T00:55:00.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:54:59 smithi132 ceph-mon[35941]: pgmap v914: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-28T00:55:00.206 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:54:59 smithi119 ceph-mon[31518]: pgmap v914: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-28T00:55:00.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:54:59 smithi165 ceph-mon[36302]: pgmap v914: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-28T00:55:01.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:55:00 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1160.678097 secs 2024-03-28T00:55:01.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:55:00 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1160.678097 secs 2024-03-28T00:55:01.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:55:00 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1160.678097 secs 2024-03-28T00:55:02.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:55:01 smithi132 ceph-mon[35941]: pgmap v915: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T00:55:02.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:55:01 smithi165 ceph-mon[36302]: pgmap v915: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T00:55:02.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:55:01 smithi119 ceph-mon[31518]: pgmap v915: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T00:55:04.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:55:03 smithi132 ceph-mon[35941]: pgmap v916: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T00:55:04.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:55:03 smithi165 ceph-mon[36302]: pgmap v916: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T00:55:04.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:55:03 smithi119 ceph-mon[31518]: pgmap v916: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T00:55:06.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:55:05 smithi165 ceph-mon[36302]: pgmap v917: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-28T00:55:06.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:55:05 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1165.678111 secs 2024-03-28T00:55:06.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:55:05 smithi119 ceph-mon[31518]: pgmap v917: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-28T00:55:06.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:55:05 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1165.678111 secs 2024-03-28T00:55:06.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:55:05 smithi132 ceph-mon[35941]: pgmap v917: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-28T00:55:06.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:55:05 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1165.678111 secs 2024-03-28T00:55:08.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:55:07 smithi165 ceph-mon[36302]: pgmap v918: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T00:55:08.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:55:07 smithi119 ceph-mon[31518]: pgmap v918: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T00:55:08.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:55:07 smithi132 ceph-mon[35941]: pgmap v918: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T00:55:10.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:55:09 smithi165 ceph-mon[36302]: pgmap v919: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T00:55:10.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:55:09 smithi119 ceph-mon[31518]: pgmap v919: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T00:55:10.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:55:09 smithi132 ceph-mon[35941]: pgmap v919: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T00:55:11.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:55:10 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1170.678176 secs 2024-03-28T00:55:11.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:55:10 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1170.678176 secs 2024-03-28T00:55:11.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:55:10 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1170.678176 secs 2024-03-28T00:55:12.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:55:11 smithi165 ceph-mon[36302]: pgmap v920: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:55:12.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:55:11 smithi119 ceph-mon[31518]: pgmap v920: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:55:12.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:55:11 smithi132 ceph-mon[35941]: pgmap v920: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:55:14.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:55:13 smithi165 ceph-mon[36302]: pgmap v921: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T00:55:14.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:55:13 smithi119 ceph-mon[31518]: pgmap v921: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T00:55:14.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:55:13 smithi132 ceph-mon[35941]: pgmap v921: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T00:55:16.252 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:55:15 smithi132 ceph-mon[35941]: pgmap v922: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:55:16.252 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:55:15 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1175.678237 secs 2024-03-28T00:55:16.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:55:15 smithi165 ceph-mon[36302]: pgmap v922: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:55:16.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:55:15 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1175.678237 secs 2024-03-28T00:55:16.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:55:15 smithi119 ceph-mon[31518]: pgmap v922: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:55:16.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:55:15 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1175.678237 secs 2024-03-28T00:55:18.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:55:17 smithi165 ceph-mon[36302]: pgmap v923: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:55:18.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:55:17 smithi119 ceph-mon[31518]: pgmap v923: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:55:18.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:55:17 smithi132 ceph-mon[35941]: pgmap v923: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:55:20.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:55:19 smithi165 ceph-mon[36302]: pgmap v924: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:55:20.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:55:19 smithi119 ceph-mon[31518]: pgmap v924: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:55:20.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:55:19 smithi132 ceph-mon[35941]: pgmap v924: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:55:21.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:55:20 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1180.678276 secs 2024-03-28T00:55:21.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:55:20 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1180.678276 secs 2024-03-28T00:55:21.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:55:20 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1180.678276 secs 2024-03-28T00:55:22.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:55:22 smithi165 ceph-mon[36302]: pgmap v925: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:55:22.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:55:22 smithi119 ceph-mon[31518]: pgmap v925: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:55:22.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:55:22 smithi132 ceph-mon[35941]: pgmap v925: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:55:24.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:55:24 smithi165 ceph-mon[36302]: pgmap v926: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:55:24.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:55:24 smithi119 ceph-mon[31518]: pgmap v926: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:55:24.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:55:24 smithi132 ceph-mon[35941]: pgmap v926: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:55:26.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:55:26 smithi165 ceph-mon[36302]: pgmap v927: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:55:26.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:55:26 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1185.678366 secs 2024-03-28T00:55:26.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:55:26 smithi119 ceph-mon[31518]: pgmap v927: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:55:26.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:55:26 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1185.678366 secs 2024-03-28T00:55:26.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:55:26 smithi132 ceph-mon[35941]: pgmap v927: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:55:26.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:55:26 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1185.678366 secs 2024-03-28T00:55:28.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:55:28 smithi165 ceph-mon[36302]: pgmap v928: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:55:28.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:55:28 smithi119 ceph-mon[31518]: pgmap v928: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:55:28.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:55:28 smithi132 ceph-mon[35941]: pgmap v928: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:55:30.304 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:55:30 smithi119 ceph-mon[31518]: pgmap v929: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:55:30.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:55:30 smithi165 ceph-mon[36302]: pgmap v929: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:55:30.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:55:30 smithi132 ceph-mon[35941]: pgmap v929: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:55:31.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:55:31 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1190.678456 secs 2024-03-28T00:55:31.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:55:31 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1190.678456 secs 2024-03-28T00:55:31.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:55:31 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1190.678456 secs 2024-03-28T00:55:32.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:55:32 smithi165 ceph-mon[36302]: pgmap v930: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:55:32.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:55:32 smithi119 ceph-mon[31518]: pgmap v930: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:55:32.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:55:32 smithi132 ceph-mon[35941]: pgmap v930: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:55:33.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:55:33 smithi165 ceph-mon[36302]: pgmap v931: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:55:33.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:55:33 smithi119 ceph-mon[31518]: pgmap v931: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:55:33.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:55:33 smithi132 ceph-mon[35941]: pgmap v931: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:55:36.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:55:35 smithi132 ceph-mon[35941]: pgmap v932: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:55:36.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:55:35 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1195.678517 secs 2024-03-28T00:55:36.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:55:35 smithi165 ceph-mon[36302]: pgmap v932: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:55:36.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:55:35 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1195.678517 secs 2024-03-28T00:55:36.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:55:35 smithi119 ceph-mon[31518]: pgmap v932: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:55:36.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:55:35 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1195.678517 secs 2024-03-28T00:55:38.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:55:37 smithi132 ceph-mon[35941]: pgmap v933: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:55:38.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:55:37 smithi165 ceph-mon[36302]: pgmap v933: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:55:38.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:55:37 smithi119 ceph-mon[31518]: pgmap v933: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:55:40.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:55:39 smithi132 ceph-mon[35941]: pgmap v934: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:55:40.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:55:39 smithi165 ceph-mon[36302]: pgmap v934: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:55:40.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:55:39 smithi119 ceph-mon[31518]: pgmap v934: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:55:41.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:55:40 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1200.678559 secs 2024-03-28T00:55:41.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:55:40 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1200.678559 secs 2024-03-28T00:55:41.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:55:40 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1200.678559 secs 2024-03-28T00:55:42.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:55:41 smithi165 ceph-mon[36302]: pgmap v935: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:55:42.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:55:41 smithi119 ceph-mon[31518]: pgmap v935: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:55:42.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:55:41 smithi132 ceph-mon[35941]: pgmap v935: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:55:44.205 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:55:43 smithi119 ceph-mon[31518]: pgmap v936: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:55:44.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:55:43 smithi165 ceph-mon[36302]: pgmap v936: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:55:44.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:55:43 smithi132 ceph-mon[35941]: pgmap v936: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:55:46.252 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:55:45 smithi132 ceph-mon[35941]: pgmap v937: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:55:46.252 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:55:45 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1205.678646 secs 2024-03-28T00:55:46.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:55:45 smithi165 ceph-mon[36302]: pgmap v937: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:55:46.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:55:45 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1205.678646 secs 2024-03-28T00:55:46.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:55:45 smithi119 ceph-mon[31518]: pgmap v937: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:55:46.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:55:45 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1205.678646 secs 2024-03-28T00:55:48.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:55:47 smithi165 ceph-mon[36302]: pgmap v938: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:55:48.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:55:47 smithi119 ceph-mon[31518]: pgmap v938: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:55:48.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:55:47 smithi132 ceph-mon[35941]: pgmap v938: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:55:50.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:55:49 smithi165 ceph-mon[36302]: pgmap v939: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:55:50.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:55:49 smithi119 ceph-mon[31518]: pgmap v939: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:55:50.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:55:49 smithi132 ceph-mon[35941]: pgmap v939: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:55:51.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:55:50 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1210.678737 secs 2024-03-28T00:55:51.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:55:50 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1210.678737 secs 2024-03-28T00:55:51.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:55:50 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1210.678737 secs 2024-03-28T00:55:52.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:55:51 smithi165 ceph-mon[36302]: pgmap v940: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:55:52.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:55:51 smithi119 ceph-mon[31518]: pgmap v940: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:55:52.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:55:51 smithi132 ceph-mon[35941]: pgmap v940: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:55:54.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:55:53 smithi165 ceph-mon[36302]: pgmap v941: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:55:54.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:55:53 smithi119 ceph-mon[31518]: pgmap v941: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:55:54.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:55:53 smithi132 ceph-mon[35941]: pgmap v941: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:55:56.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:55:55 smithi165 ceph-mon[36302]: pgmap v942: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:55:56.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:55:55 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1215.678825 secs 2024-03-28T00:55:56.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:55:55 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:55:56.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:55:55 smithi119 ceph-mon[31518]: pgmap v942: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:55:56.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:55:55 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1215.678825 secs 2024-03-28T00:55:56.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:55:55 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:55:56.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:55:55 smithi132 ceph-mon[35941]: pgmap v942: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:55:56.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:55:55 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1215.678825 secs 2024-03-28T00:55:56.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:55:55 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:55:57.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:55:57 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:55:57.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:55:57 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:55:57.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:55:57 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:55:57.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:55:57 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:55:57.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:55:57 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:55:57.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:55:57 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:55:57.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:55:57 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:55:57.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:55:57 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:55:57.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:55:57 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:55:58.280 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:55:58 smithi165 ceph-mon[36302]: pgmap v943: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T00:55:58.281 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:55:58 smithi165 ceph-mon[36302]: pgmap v944: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T00:55:58.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:55:58 smithi119 ceph-mon[31518]: pgmap v943: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T00:55:58.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:55:58 smithi119 ceph-mon[31518]: pgmap v944: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T00:55:58.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:55:58 smithi132 ceph-mon[35941]: pgmap v943: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T00:55:58.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:55:58 smithi132 ceph-mon[35941]: pgmap v944: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T00:56:00.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:56:00 smithi165 ceph-mon[36302]: pgmap v945: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T00:56:00.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:56:00 smithi119 ceph-mon[31518]: pgmap v945: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T00:56:00.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:56:00 smithi132 ceph-mon[35941]: pgmap v945: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T00:56:01.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:56:01 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1220.678915 secs 2024-03-28T00:56:01.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:56:01 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1220.678915 secs 2024-03-28T00:56:01.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:56:01 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1220.678915 secs 2024-03-28T00:56:02.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:56:02 smithi165 ceph-mon[36302]: pgmap v946: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T00:56:02.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:56:02 smithi119 ceph-mon[31518]: pgmap v946: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T00:56:02.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:56:02 smithi132 ceph-mon[35941]: pgmap v946: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T00:56:04.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:56:04 smithi165 ceph-mon[36302]: pgmap v947: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T00:56:04.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:56:04 smithi119 ceph-mon[31518]: pgmap v947: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T00:56:04.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:56:04 smithi132 ceph-mon[35941]: pgmap v947: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T00:56:06.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:56:06 smithi165 ceph-mon[36302]: pgmap v948: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T00:56:06.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:56:06 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1225.678980 secs 2024-03-28T00:56:06.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:56:06 smithi119 ceph-mon[31518]: pgmap v948: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T00:56:06.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:56:06 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1225.678980 secs 2024-03-28T00:56:06.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:56:06 smithi132 ceph-mon[35941]: pgmap v948: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T00:56:06.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:56:06 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1225.678980 secs 2024-03-28T00:56:07.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:56:07 smithi165 ceph-mon[36302]: pgmap v949: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T00:56:07.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:56:07 smithi119 ceph-mon[31518]: pgmap v949: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T00:56:07.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:56:07 smithi132 ceph-mon[35941]: pgmap v949: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T00:56:09.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:56:09 smithi165 ceph-mon[36302]: pgmap v950: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:56:09.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:56:09 smithi119 ceph-mon[31518]: pgmap v950: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:56:09.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:56:09 smithi132 ceph-mon[35941]: pgmap v950: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:56:10.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:56:10 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1230.679015 secs 2024-03-28T00:56:10.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:56:10 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1230.679015 secs 2024-03-28T00:56:10.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:56:10 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1230.679015 secs 2024-03-28T00:56:11.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:56:11 smithi165 ceph-mon[36302]: pgmap v951: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:56:11.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:56:11 smithi119 ceph-mon[31518]: pgmap v951: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:56:11.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:56:11 smithi132 ceph-mon[35941]: pgmap v951: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:56:13.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:56:13 smithi165 ceph-mon[36302]: pgmap v952: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:56:13.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:56:13 smithi119 ceph-mon[31518]: pgmap v952: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:56:13.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:56:13 smithi132 ceph-mon[35941]: pgmap v952: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:56:15.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:56:15 smithi165 ceph-mon[36302]: pgmap v953: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:56:15.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:56:15 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1235.679077 secs 2024-03-28T00:56:15.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:56:15 smithi119 ceph-mon[31518]: pgmap v953: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:56:15.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:56:15 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1235.679077 secs 2024-03-28T00:56:15.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:56:15 smithi132 ceph-mon[35941]: pgmap v953: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:56:15.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:56:15 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1235.679077 secs 2024-03-28T00:56:17.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:56:17 smithi165 ceph-mon[36302]: pgmap v954: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:56:17.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:56:17 smithi119 ceph-mon[31518]: pgmap v954: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:56:17.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:56:17 smithi132 ceph-mon[35941]: pgmap v954: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:56:19.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:56:19 smithi165 ceph-mon[36302]: pgmap v955: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:56:19.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:56:19 smithi119 ceph-mon[31518]: pgmap v955: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:56:19.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:56:19 smithi132 ceph-mon[35941]: pgmap v955: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:56:20.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:56:20 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1240.679139 secs 2024-03-28T00:56:20.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:56:20 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1240.679139 secs 2024-03-28T00:56:20.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:56:20 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1240.679139 secs 2024-03-28T00:56:21.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:56:21 smithi165 ceph-mon[36302]: pgmap v956: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:56:21.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:56:21 smithi119 ceph-mon[31518]: pgmap v956: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:56:21.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:56:21 smithi132 ceph-mon[35941]: pgmap v956: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:56:23.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:56:23 smithi165 ceph-mon[36302]: pgmap v957: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:56:23.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:56:23 smithi119 ceph-mon[31518]: pgmap v957: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:56:23.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:56:23 smithi132 ceph-mon[35941]: pgmap v957: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:56:25.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:56:25 smithi165 ceph-mon[36302]: pgmap v958: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:56:25.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:56:25 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1245.679228 secs 2024-03-28T00:56:25.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:56:25 smithi119 ceph-mon[31518]: pgmap v958: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:56:25.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:56:25 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1245.679228 secs 2024-03-28T00:56:25.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:56:25 smithi132 ceph-mon[35941]: pgmap v958: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:56:25.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:56:25 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1245.679228 secs 2024-03-28T00:56:27.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:56:27 smithi165 ceph-mon[36302]: pgmap v959: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:56:27.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:56:27 smithi119 ceph-mon[31518]: pgmap v959: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:56:27.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:56:27 smithi132 ceph-mon[35941]: pgmap v959: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:56:29.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:56:29 smithi165 ceph-mon[36302]: pgmap v960: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:56:29.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:56:29 smithi119 ceph-mon[31518]: pgmap v960: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:56:29.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:56:29 smithi132 ceph-mon[35941]: pgmap v960: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:56:30.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:56:30 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1250.679317 secs 2024-03-28T00:56:30.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:56:30 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1250.679317 secs 2024-03-28T00:56:30.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:56:30 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1250.679317 secs 2024-03-28T00:56:31.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:56:31 smithi165 ceph-mon[36302]: pgmap v961: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:56:31.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:56:31 smithi119 ceph-mon[31518]: pgmap v961: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:56:31.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:56:31 smithi132 ceph-mon[35941]: pgmap v961: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:56:33.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:56:33 smithi165 ceph-mon[36302]: pgmap v962: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:56:33.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:56:33 smithi119 ceph-mon[31518]: pgmap v962: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:56:33.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:56:33 smithi132 ceph-mon[35941]: pgmap v962: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:56:35.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:56:35 smithi165 ceph-mon[36302]: pgmap v963: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:56:35.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:56:35 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1255.679378 secs 2024-03-28T00:56:35.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:56:35 smithi119 ceph-mon[31518]: pgmap v963: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:56:35.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:56:35 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1255.679378 secs 2024-03-28T00:56:35.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:56:35 smithi132 ceph-mon[35941]: pgmap v963: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:56:35.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:56:35 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1255.679378 secs 2024-03-28T00:56:37.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:56:37 smithi165 ceph-mon[36302]: pgmap v964: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:56:37.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:56:37 smithi119 ceph-mon[31518]: pgmap v964: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:56:37.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:56:37 smithi132 ceph-mon[35941]: pgmap v964: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:56:39.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:56:39 smithi165 ceph-mon[36302]: pgmap v965: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:56:39.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:56:39 smithi119 ceph-mon[31518]: pgmap v965: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:56:39.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:56:39 smithi132 ceph-mon[35941]: pgmap v965: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:56:40.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:56:40 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1260.679418 secs 2024-03-28T00:56:40.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:56:40 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1260.679418 secs 2024-03-28T00:56:40.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:56:40 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1260.679418 secs 2024-03-28T00:56:41.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:56:41 smithi165 ceph-mon[36302]: pgmap v966: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:56:41.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:56:41 smithi119 ceph-mon[31518]: pgmap v966: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:56:41.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:56:41 smithi132 ceph-mon[35941]: pgmap v966: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:56:43.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:56:43 smithi165 ceph-mon[36302]: pgmap v967: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:56:43.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:56:43 smithi119 ceph-mon[31518]: pgmap v967: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:56:43.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:56:43 smithi132 ceph-mon[35941]: pgmap v967: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:56:45.859 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:56:45 smithi119 ceph-mon[31518]: pgmap v968: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:56:45.859 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:56:45 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1265.679505 secs 2024-03-28T00:56:45.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:56:45 smithi165 ceph-mon[36302]: pgmap v968: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:56:45.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:56:45 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1265.679505 secs 2024-03-28T00:56:45.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:56:45 smithi132 ceph-mon[35941]: pgmap v968: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:56:45.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:56:45 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1265.679505 secs 2024-03-28T00:56:47.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:56:47 smithi165 ceph-mon[36302]: pgmap v969: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:56:47.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:56:47 smithi119 ceph-mon[31518]: pgmap v969: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:56:47.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:56:47 smithi132 ceph-mon[35941]: pgmap v969: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:56:49.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:56:49 smithi165 ceph-mon[36302]: pgmap v970: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:56:49.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:56:49 smithi119 ceph-mon[31518]: pgmap v970: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:56:49.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:56:49 smithi132 ceph-mon[35941]: pgmap v970: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:56:50.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:56:50 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1270.679548 secs 2024-03-28T00:56:50.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:56:50 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1270.679548 secs 2024-03-28T00:56:50.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:56:50 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1270.679548 secs 2024-03-28T00:56:51.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:56:51 smithi165 ceph-mon[36302]: pgmap v971: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:56:51.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:56:51 smithi119 ceph-mon[31518]: pgmap v971: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:56:51.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:56:51 smithi132 ceph-mon[35941]: pgmap v971: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:56:53.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:56:53 smithi119 ceph-mon[31518]: pgmap v972: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:56:53.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:56:53 smithi132 ceph-mon[35941]: pgmap v972: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:56:54.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:56:53 smithi165 ceph-mon[36302]: pgmap v972: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:56:55.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:56:55 smithi119 ceph-mon[31518]: pgmap v973: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:56:55.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:56:55 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1275.679637 secs 2024-03-28T00:56:55.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:56:55 smithi132 ceph-mon[35941]: pgmap v973: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:56:55.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:56:55 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1275.679637 secs 2024-03-28T00:56:56.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:56:55 smithi165 ceph-mon[36302]: pgmap v973: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:56:56.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:56:55 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1275.679637 secs 2024-03-28T00:56:57.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:56:57 smithi119 ceph-mon[31518]: pgmap v974: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T00:56:57.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:56:57 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:56:57.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:56:57 smithi132 ceph-mon[35941]: pgmap v974: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T00:56:57.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:56:57 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:56:58.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:56:57 smithi165 ceph-mon[36302]: pgmap v974: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T00:56:58.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:56:57 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:56:59.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:56:58 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:56:59.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:56:58 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:56:59.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:56:58 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:56:59.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:56:58 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:56:59.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:56:58 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:56:59.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:56:58 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:56:59.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:56:58 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:56:59.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:56:58 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:56:59.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:56:58 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:56:59.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:56:58 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:56:59.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:56:58 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:56:59.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:56:58 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:56:59.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:56:58 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:56:59.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:56:58 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:56:59.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:56:58 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:56:59.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:56:58 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:56:59.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:56:58 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:56:59.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:56:58 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:56:59.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:56:58 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:56:59.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:56:58 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:56:59.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:56:58 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:56:59.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:56:58 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:56:59.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:56:58 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:56:59.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:56:58 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:56:59.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:56:58 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:56:59.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:56:58 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:56:59.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:56:58 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:57:00.306 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:57:00 smithi119 ceph-mon[31518]: pgmap v975: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T00:57:00.306 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:57:00 smithi119 ceph-mon[31518]: pgmap v976: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-03-28T00:57:00.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:57:00 smithi165 ceph-mon[36302]: pgmap v975: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T00:57:00.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:57:00 smithi165 ceph-mon[36302]: pgmap v976: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-03-28T00:57:00.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:57:00 smithi132 ceph-mon[35941]: pgmap v975: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T00:57:00.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:57:00 smithi132 ceph-mon[35941]: pgmap v976: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-03-28T00:57:01.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:57:01 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1280.679727 secs 2024-03-28T00:57:01.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:57:01 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1280.679727 secs 2024-03-28T00:57:01.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:57:01 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1280.679727 secs 2024-03-28T00:57:02.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:57:02 smithi165 ceph-mon[36302]: pgmap v977: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-03-28T00:57:02.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:57:02 smithi119 ceph-mon[31518]: pgmap v977: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-03-28T00:57:02.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:57:02 smithi132 ceph-mon[35941]: pgmap v977: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-03-28T00:57:04.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:57:04 smithi165 ceph-mon[36302]: pgmap v978: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-03-28T00:57:04.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:57:04 smithi119 ceph-mon[31518]: pgmap v978: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-03-28T00:57:04.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:57:04 smithi132 ceph-mon[35941]: pgmap v978: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-03-28T00:57:06.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:57:06 smithi165 ceph-mon[36302]: pgmap v979: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-28T00:57:06.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:57:06 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1285.679744 secs 2024-03-28T00:57:06.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:57:06 smithi119 ceph-mon[31518]: pgmap v979: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-28T00:57:06.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:57:06 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1285.679744 secs 2024-03-28T00:57:06.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:57:06 smithi132 ceph-mon[35941]: pgmap v979: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-28T00:57:06.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:57:06 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1285.679744 secs 2024-03-28T00:57:08.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:57:08 smithi165 ceph-mon[36302]: pgmap v980: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-28T00:57:08.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:57:08 smithi119 ceph-mon[31518]: pgmap v980: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-28T00:57:08.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:57:08 smithi132 ceph-mon[35941]: pgmap v980: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-28T00:57:09.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:57:09 smithi165 ceph-mon[36302]: pgmap v981: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-28T00:57:09.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:57:09 smithi119 ceph-mon[31518]: pgmap v981: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-28T00:57:09.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:57:09 smithi132 ceph-mon[35941]: pgmap v981: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-28T00:57:10.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:57:10 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1290.679832 secs 2024-03-28T00:57:10.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:57:10 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1290.679832 secs 2024-03-28T00:57:10.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:57:10 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1290.679832 secs 2024-03-28T00:57:11.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:57:11 smithi165 ceph-mon[36302]: pgmap v982: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:57:11.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:57:11 smithi119 ceph-mon[31518]: pgmap v982: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:57:11.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:57:11 smithi132 ceph-mon[35941]: pgmap v982: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:57:14.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:57:13 smithi132 ceph-mon[35941]: pgmap v983: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:57:14.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:57:13 smithi165 ceph-mon[36302]: pgmap v983: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:57:14.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:57:13 smithi119 ceph-mon[31518]: pgmap v983: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:57:16.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:57:15 smithi132 ceph-mon[35941]: pgmap v984: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:57:16.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:57:15 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1295.679900 secs 2024-03-28T00:57:16.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:57:15 smithi165 ceph-mon[36302]: pgmap v984: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:57:16.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:57:15 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1295.679900 secs 2024-03-28T00:57:16.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:57:15 smithi119 ceph-mon[31518]: pgmap v984: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:57:16.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:57:15 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1295.679900 secs 2024-03-28T00:57:18.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:57:17 smithi132 ceph-mon[35941]: pgmap v985: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:57:18.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:57:17 smithi165 ceph-mon[36302]: pgmap v985: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:57:18.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:57:17 smithi119 ceph-mon[31518]: pgmap v985: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:57:20.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:57:19 smithi132 ceph-mon[35941]: pgmap v986: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:57:20.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:57:19 smithi165 ceph-mon[36302]: pgmap v986: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:57:20.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:57:19 smithi119 ceph-mon[31518]: pgmap v986: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:57:21.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:57:20 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1300.679962 secs 2024-03-28T00:57:21.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:57:20 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1300.679962 secs 2024-03-28T00:57:21.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:57:20 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1300.679962 secs 2024-03-28T00:57:22.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:57:21 smithi132 ceph-mon[35941]: pgmap v987: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:57:22.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:57:21 smithi165 ceph-mon[36302]: pgmap v987: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:57:22.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:57:21 smithi119 ceph-mon[31518]: pgmap v987: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:57:24.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:57:23 smithi132 ceph-mon[35941]: pgmap v988: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:57:24.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:57:23 smithi165 ceph-mon[36302]: pgmap v988: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:57:24.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:57:23 smithi119 ceph-mon[31518]: pgmap v988: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:57:26.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:57:25 smithi132 ceph-mon[35941]: pgmap v989: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:57:26.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:57:25 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1305.680023 secs 2024-03-28T00:57:26.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:57:25 smithi165 ceph-mon[36302]: pgmap v989: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:57:26.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:57:25 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1305.680023 secs 2024-03-28T00:57:26.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:57:25 smithi119 ceph-mon[31518]: pgmap v989: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:57:26.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:57:25 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1305.680023 secs 2024-03-28T00:57:28.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:57:27 smithi132 ceph-mon[35941]: pgmap v990: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:57:28.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:57:27 smithi165 ceph-mon[36302]: pgmap v990: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:57:28.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:57:27 smithi119 ceph-mon[31518]: pgmap v990: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:57:30.207 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:57:29 smithi119 ceph-mon[31518]: pgmap v991: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:57:30.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:57:29 smithi165 ceph-mon[36302]: pgmap v991: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:57:30.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:57:29 smithi132 ceph-mon[35941]: pgmap v991: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:57:31.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:57:30 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1310.680084 secs 2024-03-28T00:57:31.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:57:30 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1310.680084 secs 2024-03-28T00:57:31.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:57:30 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1310.680084 secs 2024-03-28T00:57:32.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:57:31 smithi165 ceph-mon[36302]: pgmap v992: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:57:32.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:57:31 smithi119 ceph-mon[31518]: pgmap v992: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:57:32.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:57:31 smithi132 ceph-mon[35941]: pgmap v992: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:57:34.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:57:33 smithi165 ceph-mon[36302]: pgmap v993: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:57:34.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:57:33 smithi119 ceph-mon[31518]: pgmap v993: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:57:34.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:57:33 smithi132 ceph-mon[35941]: pgmap v993: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:57:36.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:57:35 smithi165 ceph-mon[36302]: pgmap v994: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:57:36.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:57:35 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1315.680146 secs 2024-03-28T00:57:36.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:57:35 smithi119 ceph-mon[31518]: pgmap v994: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:57:36.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:57:35 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1315.680146 secs 2024-03-28T00:57:36.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:57:35 smithi132 ceph-mon[35941]: pgmap v994: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:57:36.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:57:35 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1315.680146 secs 2024-03-28T00:57:38.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:57:37 smithi165 ceph-mon[36302]: pgmap v995: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:57:38.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:57:37 smithi119 ceph-mon[31518]: pgmap v995: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:57:38.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:57:37 smithi132 ceph-mon[35941]: pgmap v995: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:57:40.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:57:39 smithi165 ceph-mon[36302]: pgmap v996: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:57:40.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:57:39 smithi119 ceph-mon[31518]: pgmap v996: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:57:40.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:57:39 smithi132 ceph-mon[35941]: pgmap v996: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:57:41.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:57:40 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1320.680206 secs 2024-03-28T00:57:41.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:57:40 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1320.680206 secs 2024-03-28T00:57:41.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:57:40 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1320.680206 secs 2024-03-28T00:57:42.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:57:41 smithi165 ceph-mon[36302]: pgmap v997: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:57:42.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:57:41 smithi119 ceph-mon[31518]: pgmap v997: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:57:42.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:57:41 smithi132 ceph-mon[35941]: pgmap v997: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:57:44.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:57:44 smithi165 ceph-mon[36302]: pgmap v998: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:57:44.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:57:44 smithi119 ceph-mon[31518]: pgmap v998: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:57:44.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:57:44 smithi132 ceph-mon[35941]: pgmap v998: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:57:46.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:57:46 smithi165 ceph-mon[36302]: pgmap v999: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:57:46.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:57:46 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1325.680265 secs 2024-03-28T00:57:46.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:57:46 smithi119 ceph-mon[31518]: pgmap v999: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:57:46.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:57:46 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1325.680265 secs 2024-03-28T00:57:46.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:57:46 smithi132 ceph-mon[35941]: pgmap v999: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:57:46.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:57:46 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1325.680265 secs 2024-03-28T00:57:48.323 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:57:48 smithi119 ceph-mon[31518]: pgmap v1000: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:57:48.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:57:48 smithi165 ceph-mon[36302]: pgmap v1000: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:57:48.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:57:48 smithi132 ceph-mon[35941]: pgmap v1000: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:57:50.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:57:50 smithi165 ceph-mon[36302]: pgmap v1001: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:57:50.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:57:50 smithi119 ceph-mon[31518]: pgmap v1001: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:57:50.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:57:50 smithi132 ceph-mon[35941]: pgmap v1001: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:57:51.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:57:51 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1330.680329 secs 2024-03-28T00:57:51.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:57:51 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1330.680329 secs 2024-03-28T00:57:51.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:57:51 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1330.680329 secs 2024-03-28T00:57:52.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:57:52 smithi165 ceph-mon[36302]: pgmap v1002: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:57:52.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:57:52 smithi119 ceph-mon[31518]: pgmap v1002: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:57:52.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:57:52 smithi132 ceph-mon[35941]: pgmap v1002: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:57:54.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:57:54 smithi165 ceph-mon[36302]: pgmap v1003: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:57:54.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:57:54 smithi119 ceph-mon[31518]: pgmap v1003: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:57:54.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:57:54 smithi132 ceph-mon[35941]: pgmap v1003: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:57:56.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:57:56 smithi165 ceph-mon[36302]: pgmap v1004: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:57:56.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:57:56 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1335.680390 secs 2024-03-28T00:57:56.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:57:56 smithi119 ceph-mon[31518]: pgmap v1004: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:57:56.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:57:56 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1335.680390 secs 2024-03-28T00:57:56.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:57:56 smithi132 ceph-mon[35941]: pgmap v1004: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:57:56.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:57:56 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1335.680390 secs 2024-03-28T00:57:58.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:57:58 smithi165 ceph-mon[36302]: pgmap v1005: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:57:58.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:57:58 smithi119 ceph-mon[31518]: pgmap v1005: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:57:58.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:57:58 smithi132 ceph-mon[35941]: pgmap v1005: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:58:00.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:58:00 smithi165 ceph-mon[36302]: pgmap v1006: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:58:00.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:58:00 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:58:00.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:58:00 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:58:00.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:58:00 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:58:00.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:58:00 smithi119 ceph-mon[31518]: pgmap v1006: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:58:00.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:58:00 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:58:00.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:58:00 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:58:00.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:58:00 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:58:00.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:58:00 smithi132 ceph-mon[35941]: pgmap v1006: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:58:00.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:58:00 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:58:00.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:58:00 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:58:00.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:58:00 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:58:01.367 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:58:01 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1340.680460 secs 2024-03-28T00:58:01.367 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:58:01 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:58:01.367 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:58:01 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:58:01.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:58:01 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1340.680460 secs 2024-03-28T00:58:01.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:58:01 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:58:01.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:58:01 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:58:01.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:58:01 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1340.680460 secs 2024-03-28T00:58:01.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:58:01 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:58:01.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:58:01 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:58:02.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:58:02 smithi165 ceph-mon[36302]: pgmap v1007: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:58:02.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:58:02 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:58:02.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:58:02 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:58:02.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:58:02 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:58:02.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:58:02 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:58:02.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:58:02 smithi119 ceph-mon[31518]: pgmap v1007: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:58:02.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:58:02 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:58:02.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:58:02 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:58:02.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:58:02 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:58:02.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:58:02 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:58:02.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:58:02 smithi132 ceph-mon[35941]: pgmap v1007: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:58:02.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:58:02 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:58:02.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:58:02 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:58:02.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:58:02 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:58:02.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:58:02 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:58:03.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:58:03 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:58:03.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:58:03 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:58:03.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:58:03 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:58:03.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:58:03 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:58:03.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:58:03 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:58:03.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:58:03 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:58:03.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:58:03 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:58:03.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:58:03 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:58:03.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:58:03 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:58:03.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:58:03 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:58:03.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:58:03 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:58:03.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:58:03 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:58:03.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:58:03 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:58:03.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:58:03 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:58:03.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:58:03 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:58:03.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:58:03 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:58:03.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:58:03 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:58:03.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:58:03 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:58:03.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:58:03 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:58:03.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:58:03 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:58:03.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:58:03 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:58:03.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:58:03 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:58:03.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:58:03 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:58:03.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:58:03 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:58:03.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:58:03 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:58:03.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:58:03 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:58:03.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:58:03 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:58:03.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:58:03 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:58:03.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:58:03 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:58:03.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:58:03 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:58:03.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:58:03 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:58:03.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:58:03 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:58:03.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:58:03 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:58:03.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:58:03 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:58:03.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:58:03 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:58:03.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:58:03 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:58:03.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:58:03 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:58:03.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:58:03 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:58:03.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:58:03 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:58:03.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:58:03 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:58:03.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:58:03 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:58:03.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:58:03 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:58:03.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:58:03 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:58:03.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:58:03 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:58:03.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:58:03 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:58:03.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:58:03 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:58:03.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:58:03 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:58:03.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:58:03 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:58:04.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:58:04 smithi165 ceph-mon[36302]: Adjusting osd_memory_target on smithi132 to 174.1M 2024-03-28T00:58:04.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:58:04 smithi165 ceph-mon[36302]: Unable to set osd_memory_target on smithi132 to 182640640: error parsing value: Value '182640640' is below minimum 939524096 2024-03-28T00:58:04.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:58:04 smithi165 ceph-mon[36302]: Adjusting osd_memory_target on smithi165 to 174.1M 2024-03-28T00:58:04.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:58:04 smithi165 ceph-mon[36302]: Unable to set osd_memory_target on smithi165 to 182635622: error parsing value: Value '182635622' is below minimum 939524096 2024-03-28T00:58:04.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:58:04 smithi165 ceph-mon[36302]: pgmap v1008: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:58:04.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:58:04 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:58:04.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:58:04 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:58:04.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:58:04 smithi119 ceph-mon[31518]: Adjusting osd_memory_target on smithi132 to 174.1M 2024-03-28T00:58:04.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:58:04 smithi119 ceph-mon[31518]: Unable to set osd_memory_target on smithi132 to 182640640: error parsing value: Value '182640640' is below minimum 939524096 2024-03-28T00:58:04.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:58:04 smithi119 ceph-mon[31518]: Adjusting osd_memory_target on smithi165 to 174.1M 2024-03-28T00:58:04.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:58:04 smithi119 ceph-mon[31518]: Unable to set osd_memory_target on smithi165 to 182635622: error parsing value: Value '182635622' is below minimum 939524096 2024-03-28T00:58:04.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:58:04 smithi119 ceph-mon[31518]: pgmap v1008: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:58:04.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:58:04 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:58:04.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:58:04 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:58:04.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:58:04 smithi132 ceph-mon[35941]: Adjusting osd_memory_target on smithi132 to 174.1M 2024-03-28T00:58:04.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:58:04 smithi132 ceph-mon[35941]: Unable to set osd_memory_target on smithi132 to 182640640: error parsing value: Value '182640640' is below minimum 939524096 2024-03-28T00:58:04.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:58:04 smithi132 ceph-mon[35941]: Adjusting osd_memory_target on smithi165 to 174.1M 2024-03-28T00:58:04.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:58:04 smithi132 ceph-mon[35941]: Unable to set osd_memory_target on smithi165 to 182635622: error parsing value: Value '182635622' is below minimum 939524096 2024-03-28T00:58:04.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:58:04 smithi132 ceph-mon[35941]: pgmap v1008: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:58:04.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:58:04 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:58:04.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:58:04 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:58:05.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:58:05 smithi165 ceph-mon[36302]: pgmap v1009: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:58:05.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:58:05 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:58:05.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:58:05 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:58:05.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:58:05 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:58:05.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:58:05 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:58:05.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:58:05 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:58:05.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:58:05 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:58:05.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:58:05 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:58:05.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:58:05 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:58:05.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:58:05 smithi165 ceph-mon[36302]: Adjusting osd_memory_target on smithi119 to 176.7M 2024-03-28T00:58:05.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:58:05 smithi165 ceph-mon[36302]: Unable to set osd_memory_target on smithi119 to 185372364: error parsing value: Value '185372364' is below minimum 939524096 2024-03-28T00:58:05.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:58:05 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:58:05.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:58:05 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:58:05.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:58:05 smithi165 ceph-mon[36302]: pgmap v1010: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T00:58:05.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:58:05 smithi165 ceph-mon[36302]: pgmap v1011: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:58:05.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:58:05 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:58:05.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:58:05 smithi119 ceph-mon[31518]: pgmap v1009: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:58:05.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:58:05 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:58:05.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:58:05 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:58:05.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:58:05 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:58:05.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:58:05 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:58:05.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:58:05 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:58:05.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:58:05 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:58:05.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:58:05 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:58:05.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:58:05 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:58:05.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:58:05 smithi119 ceph-mon[31518]: Adjusting osd_memory_target on smithi119 to 176.7M 2024-03-28T00:58:05.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:58:05 smithi119 ceph-mon[31518]: Unable to set osd_memory_target on smithi119 to 185372364: error parsing value: Value '185372364' is below minimum 939524096 2024-03-28T00:58:05.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:58:05 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:58:05.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:58:05 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:58:05.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:58:05 smithi119 ceph-mon[31518]: pgmap v1010: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T00:58:05.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:58:05 smithi119 ceph-mon[31518]: pgmap v1011: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:58:05.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:58:05 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:58:05.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:58:05 smithi132 ceph-mon[35941]: pgmap v1009: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:58:05.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:58:05 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:58:05.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:58:05 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:58:05.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:58:05 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:58:05.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:58:05 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:58:05.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:58:05 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:58:05.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:58:05 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:58:05.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:58:05 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:58:05.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:58:05 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T00:58:05.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:58:05 smithi132 ceph-mon[35941]: Adjusting osd_memory_target on smithi119 to 176.7M 2024-03-28T00:58:05.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:58:05 smithi132 ceph-mon[35941]: Unable to set osd_memory_target on smithi119 to 185372364: error parsing value: Value '185372364' is below minimum 939524096 2024-03-28T00:58:05.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:58:05 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:58:05.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:58:05 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:58:05.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:58:05 smithi132 ceph-mon[35941]: pgmap v1010: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T00:58:05.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:58:05 smithi132 ceph-mon[35941]: pgmap v1011: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:58:05.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:58:05 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:58:06.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:58:06 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1345.680549 secs 2024-03-28T00:58:06.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:58:06 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1345.680549 secs 2024-03-28T00:58:06.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:58:06 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1345.680549 secs 2024-03-28T00:58:07.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:58:07 smithi165 ceph-mon[36302]: pgmap v1012: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:58:07.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:58:07 smithi119 ceph-mon[31518]: pgmap v1012: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:58:07.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:58:07 smithi132 ceph-mon[35941]: pgmap v1012: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:58:10.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:58:09 smithi165 ceph-mon[36302]: pgmap v1013: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 3 op/s 2024-03-28T00:58:10.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:58:09 smithi119 ceph-mon[31518]: pgmap v1013: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 3 op/s 2024-03-28T00:58:10.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:58:09 smithi132 ceph-mon[35941]: pgmap v1013: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 3 op/s 2024-03-28T00:58:11.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:58:10 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1350.680636 secs 2024-03-28T00:58:11.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:58:10 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1350.680636 secs 2024-03-28T00:58:11.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:58:10 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1350.680636 secs 2024-03-28T00:58:12.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:58:11 smithi165 ceph-mon[36302]: pgmap v1014: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:58:12.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:58:11 smithi119 ceph-mon[31518]: pgmap v1014: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:58:12.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:58:11 smithi132 ceph-mon[35941]: pgmap v1014: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:58:14.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:58:14 smithi165 ceph-mon[36302]: pgmap v1015: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 3 op/s 2024-03-28T00:58:14.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:58:13 smithi119 ceph-mon[31518]: pgmap v1015: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 3 op/s 2024-03-28T00:58:14.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:58:13 smithi132 ceph-mon[35941]: pgmap v1015: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 3 op/s 2024-03-28T00:58:16.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:58:16 smithi165 ceph-mon[36302]: pgmap v1016: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T00:58:16.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:58:16 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1355.680724 secs 2024-03-28T00:58:16.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:58:16 smithi119 ceph-mon[31518]: pgmap v1016: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T00:58:16.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:58:16 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1355.680724 secs 2024-03-28T00:58:16.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:58:16 smithi132 ceph-mon[35941]: pgmap v1016: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T00:58:16.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:58:16 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1355.680724 secs 2024-03-28T00:58:18.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:58:18 smithi165 ceph-mon[36302]: pgmap v1017: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:58:18.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:58:18 smithi119 ceph-mon[31518]: pgmap v1017: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:58:18.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:58:18 smithi132 ceph-mon[35941]: pgmap v1017: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:58:20.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:58:20 smithi165 ceph-mon[36302]: pgmap v1018: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:58:20.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:58:20 smithi119 ceph-mon[31518]: pgmap v1018: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:58:20.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:58:20 smithi132 ceph-mon[35941]: pgmap v1018: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:58:21.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:58:21 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1360.680786 secs 2024-03-28T00:58:21.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:58:21 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1360.680786 secs 2024-03-28T00:58:21.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:58:21 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1360.680786 secs 2024-03-28T00:58:22.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:58:22 smithi165 ceph-mon[36302]: pgmap v1019: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:58:22.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:58:22 smithi119 ceph-mon[31518]: pgmap v1019: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:58:22.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:58:22 smithi132 ceph-mon[35941]: pgmap v1019: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:58:24.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:58:24 smithi165 ceph-mon[36302]: pgmap v1020: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:58:24.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:58:24 smithi119 ceph-mon[31518]: pgmap v1020: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:58:24.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:58:24 smithi132 ceph-mon[35941]: pgmap v1020: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:58:26.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:58:26 smithi165 ceph-mon[36302]: pgmap v1021: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:58:26.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:58:26 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1365.680847 secs 2024-03-28T00:58:26.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:58:26 smithi119 ceph-mon[31518]: pgmap v1021: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:58:26.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:58:26 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1365.680847 secs 2024-03-28T00:58:26.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:58:26 smithi132 ceph-mon[35941]: pgmap v1021: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:58:26.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:58:26 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1365.680847 secs 2024-03-28T00:58:28.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:58:28 smithi165 ceph-mon[36302]: pgmap v1022: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:58:28.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:58:28 smithi119 ceph-mon[31518]: pgmap v1022: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:58:28.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:58:28 smithi132 ceph-mon[35941]: pgmap v1022: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:58:30.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:58:30 smithi165 ceph-mon[36302]: pgmap v1023: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:58:30.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:58:30 smithi119 ceph-mon[31518]: pgmap v1023: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:58:30.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:58:30 smithi132 ceph-mon[35941]: pgmap v1023: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:58:31.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:58:31 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1370.680909 secs 2024-03-28T00:58:31.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:58:31 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1370.680909 secs 2024-03-28T00:58:31.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:58:31 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1370.680909 secs 2024-03-28T00:58:32.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:58:32 smithi165 ceph-mon[36302]: pgmap v1024: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:58:32.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:58:32 smithi119 ceph-mon[31518]: pgmap v1024: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:58:32.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:58:32 smithi132 ceph-mon[35941]: pgmap v1024: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:58:34.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:58:34 smithi165 ceph-mon[36302]: pgmap v1025: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:58:34.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:58:34 smithi119 ceph-mon[31518]: pgmap v1025: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:58:34.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:58:34 smithi132 ceph-mon[35941]: pgmap v1025: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:58:35.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:58:35 smithi165 ceph-mon[36302]: pgmap v1026: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:58:35.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:58:35 smithi119 ceph-mon[31518]: pgmap v1026: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:58:35.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:58:35 smithi132 ceph-mon[35941]: pgmap v1026: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:58:36.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:58:36 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1375.680970 secs 2024-03-28T00:58:36.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:58:36 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1375.680970 secs 2024-03-28T00:58:36.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:58:36 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1375.680970 secs 2024-03-28T00:58:37.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:58:37 smithi165 ceph-mon[36302]: pgmap v1027: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:58:37.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:58:37 smithi119 ceph-mon[31518]: pgmap v1027: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:58:37.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:58:37 smithi132 ceph-mon[35941]: pgmap v1027: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:58:40.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:58:39 smithi165 ceph-mon[36302]: pgmap v1028: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:58:40.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:58:39 smithi119 ceph-mon[31518]: pgmap v1028: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:58:40.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:58:39 smithi132 ceph-mon[35941]: pgmap v1028: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:58:41.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:58:41 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1380.681030 secs 2024-03-28T00:58:41.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:58:41 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1380.681030 secs 2024-03-28T00:58:41.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:58:41 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1380.681030 secs 2024-03-28T00:58:42.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:58:42 smithi165 ceph-mon[36302]: pgmap v1029: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:58:42.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:58:42 smithi119 ceph-mon[31518]: pgmap v1029: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:58:42.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:58:42 smithi132 ceph-mon[35941]: pgmap v1029: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:58:44.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:58:44 smithi165 ceph-mon[36302]: pgmap v1030: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:58:44.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:58:44 smithi119 ceph-mon[31518]: pgmap v1030: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:58:44.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:58:44 smithi132 ceph-mon[35941]: pgmap v1030: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:58:46.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:58:46 smithi165 ceph-mon[36302]: pgmap v1031: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:58:46.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:58:46 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1385.681122 secs 2024-03-28T00:58:46.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:58:46 smithi119 ceph-mon[31518]: pgmap v1031: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:58:46.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:58:46 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1385.681122 secs 2024-03-28T00:58:46.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:58:46 smithi132 ceph-mon[35941]: pgmap v1031: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:58:46.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:58:46 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1385.681122 secs 2024-03-28T00:58:48.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:58:48 smithi165 ceph-mon[36302]: pgmap v1032: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:58:48.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:58:48 smithi119 ceph-mon[31518]: pgmap v1032: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:58:48.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:58:48 smithi132 ceph-mon[35941]: pgmap v1032: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:58:50.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:58:50 smithi165 ceph-mon[36302]: pgmap v1033: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:58:50.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:58:50 smithi119 ceph-mon[31518]: pgmap v1033: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:58:50.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:58:50 smithi132 ceph-mon[35941]: pgmap v1033: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:58:51.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:58:51 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1390.681212 secs 2024-03-28T00:58:51.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:58:51 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1390.681212 secs 2024-03-28T00:58:51.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:58:51 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1390.681212 secs 2024-03-28T00:58:52.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:58:52 smithi165 ceph-mon[36302]: pgmap v1034: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:58:52.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:58:52 smithi119 ceph-mon[31518]: pgmap v1034: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:58:52.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:58:52 smithi132 ceph-mon[35941]: pgmap v1034: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:58:54.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:58:54 smithi165 ceph-mon[36302]: pgmap v1035: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:58:54.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:58:54 smithi119 ceph-mon[31518]: pgmap v1035: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:58:54.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:58:54 smithi132 ceph-mon[35941]: pgmap v1035: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:58:56.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:58:56 smithi165 ceph-mon[36302]: pgmap v1036: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:58:56.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:58:56 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1395.681253 secs 2024-03-28T00:58:56.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:58:56 smithi119 ceph-mon[31518]: pgmap v1036: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:58:56.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:58:56 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1395.681253 secs 2024-03-28T00:58:56.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:58:56 smithi132 ceph-mon[35941]: pgmap v1036: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:58:56.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:58:56 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1395.681253 secs 2024-03-28T00:58:58.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:58:58 smithi165 ceph-mon[36302]: pgmap v1037: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:58:58.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:58:58 smithi119 ceph-mon[31518]: pgmap v1037: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:58:58.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:58:58 smithi132 ceph-mon[35941]: pgmap v1037: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:59:00.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:59:00 smithi165 ceph-mon[36302]: pgmap v1038: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:59:00.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:59:00 smithi119 ceph-mon[31518]: pgmap v1038: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:59:00.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:59:00 smithi132 ceph-mon[35941]: pgmap v1038: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:59:01.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:59:01 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1400.681343 secs 2024-03-28T00:59:01.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:59:01 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1400.681343 secs 2024-03-28T00:59:01.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:59:01 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1400.681343 secs 2024-03-28T00:59:02.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:59:02 smithi165 ceph-mon[36302]: pgmap v1039: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:59:02.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:59:02 smithi119 ceph-mon[31518]: pgmap v1039: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:59:02.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:59:02 smithi132 ceph-mon[35941]: pgmap v1039: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:59:04.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:59:04 smithi165 ceph-mon[36302]: pgmap v1040: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:59:04.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:59:04 smithi119 ceph-mon[31518]: pgmap v1040: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:59:04.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:59:04 smithi132 ceph-mon[35941]: pgmap v1040: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:59:05.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:59:05 smithi165 ceph-mon[36302]: pgmap v1041: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:59:05.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:59:05 smithi119 ceph-mon[31518]: pgmap v1041: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:59:05.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:59:05 smithi132 ceph-mon[35941]: pgmap v1041: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:59:06.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:59:06 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1405.681431 secs 2024-03-28T00:59:06.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:59:06 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:59:06.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:59:06 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:59:06.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:59:06 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:59:06.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:59:06 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1405.681431 secs 2024-03-28T00:59:06.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:59:06 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:59:06.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:59:06 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:59:06.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:59:06 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:59:06.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:59:06 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1405.681431 secs 2024-03-28T00:59:06.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:59:06 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T00:59:06.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:59:06 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:59:06.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:59:06 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:59:07.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:59:07 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:59:07.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:59:07 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:59:07.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:59:07 smithi119 ceph-mon[31518]: pgmap v1042: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T00:59:07.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:59:07 smithi119 ceph-mon[31518]: pgmap v1043: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T00:59:07.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:59:07 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:59:07.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:59:07 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:59:07.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:59:07 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:59:07.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:59:07 smithi132 ceph-mon[35941]: pgmap v1042: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T00:59:07.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:59:07 smithi132 ceph-mon[35941]: pgmap v1043: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T00:59:07.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:59:07 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:59:07.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:59:07 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T00:59:07.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:59:07 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T00:59:07.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:59:07 smithi165 ceph-mon[36302]: pgmap v1042: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T00:59:07.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:59:07 smithi165 ceph-mon[36302]: pgmap v1043: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T00:59:07.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:59:07 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T00:59:09.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:59:09 smithi165 ceph-mon[36302]: pgmap v1044: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T00:59:09.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:59:09 smithi119 ceph-mon[31518]: pgmap v1044: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T00:59:09.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:59:09 smithi132 ceph-mon[35941]: pgmap v1044: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T00:59:10.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:59:10 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1410.681490 secs 2024-03-28T00:59:10.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:59:10 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1410.681490 secs 2024-03-28T00:59:10.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:59:10 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1410.681490 secs 2024-03-28T00:59:11.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:59:11 smithi165 ceph-mon[36302]: pgmap v1045: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T00:59:11.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:59:11 smithi119 ceph-mon[31518]: pgmap v1045: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T00:59:11.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:59:11 smithi132 ceph-mon[35941]: pgmap v1045: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T00:59:13.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:59:13 smithi165 ceph-mon[36302]: pgmap v1046: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T00:59:13.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:59:13 smithi119 ceph-mon[31518]: pgmap v1046: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T00:59:13.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:59:13 smithi132 ceph-mon[35941]: pgmap v1046: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T00:59:15.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:59:15 smithi119 ceph-mon[31518]: pgmap v1047: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T00:59:15.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:59:15 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1415.681580 secs 2024-03-28T00:59:15.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:59:15 smithi132 ceph-mon[35941]: pgmap v1047: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T00:59:15.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:59:15 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1415.681580 secs 2024-03-28T00:59:16.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:59:15 smithi165 ceph-mon[36302]: pgmap v1047: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T00:59:16.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:59:15 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1415.681580 secs 2024-03-28T00:59:17.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:59:17 smithi119 ceph-mon[31518]: pgmap v1048: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T00:59:17.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:59:17 smithi132 ceph-mon[35941]: pgmap v1048: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T00:59:18.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:59:17 smithi165 ceph-mon[36302]: pgmap v1048: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T00:59:19.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:59:19 smithi119 ceph-mon[31518]: pgmap v1049: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:59:19.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:59:19 smithi132 ceph-mon[35941]: pgmap v1049: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:59:20.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:59:19 smithi165 ceph-mon[36302]: pgmap v1049: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:59:20.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:59:20 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1420.681641 secs 2024-03-28T00:59:20.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:59:20 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1420.681641 secs 2024-03-28T00:59:21.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:59:20 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1420.681641 secs 2024-03-28T00:59:21.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:59:21 smithi132 ceph-mon[35941]: pgmap v1050: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:59:22.018 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:59:21 smithi119 ceph-mon[31518]: pgmap v1050: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:59:22.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:59:21 smithi165 ceph-mon[36302]: pgmap v1050: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:59:23.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:59:23 smithi132 ceph-mon[35941]: pgmap v1051: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:59:24.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:59:23 smithi165 ceph-mon[36302]: pgmap v1051: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:59:24.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:59:23 smithi119 ceph-mon[31518]: pgmap v1051: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:59:25.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:59:25 smithi132 ceph-mon[35941]: pgmap v1052: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:59:25.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:59:25 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1425.681701 secs 2024-03-28T00:59:26.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:59:25 smithi165 ceph-mon[36302]: pgmap v1052: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:59:26.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:59:25 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1425.681701 secs 2024-03-28T00:59:26.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:59:25 smithi119 ceph-mon[31518]: pgmap v1052: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:59:26.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:59:25 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1425.681701 secs 2024-03-28T00:59:27.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:59:27 smithi132 ceph-mon[35941]: pgmap v1053: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:59:28.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:59:27 smithi165 ceph-mon[36302]: pgmap v1053: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:59:28.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:59:27 smithi119 ceph-mon[31518]: pgmap v1053: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:59:29.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:59:29 smithi132 ceph-mon[35941]: pgmap v1054: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:59:30.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:59:29 smithi165 ceph-mon[36302]: pgmap v1054: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:59:30.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:59:29 smithi119 ceph-mon[31518]: pgmap v1054: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:59:30.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:59:30 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1430.681761 secs 2024-03-28T00:59:31.030 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:59:30 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1430.681761 secs 2024-03-28T00:59:31.054 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:59:30 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1430.681761 secs 2024-03-28T00:59:31.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:59:31 smithi132 ceph-mon[35941]: pgmap v1055: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:59:32.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:59:31 smithi165 ceph-mon[36302]: pgmap v1055: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:59:32.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:59:31 smithi119 ceph-mon[31518]: pgmap v1055: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:59:33.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:59:33 smithi132 ceph-mon[35941]: pgmap v1056: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:59:34.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:59:33 smithi165 ceph-mon[36302]: pgmap v1056: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:59:34.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:59:33 smithi119 ceph-mon[31518]: pgmap v1056: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:59:35.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:59:35 smithi132 ceph-mon[35941]: pgmap v1057: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:59:35.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:59:35 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1435.681848 secs 2024-03-28T00:59:36.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:59:35 smithi165 ceph-mon[36302]: pgmap v1057: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:59:36.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:59:35 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1435.681848 secs 2024-03-28T00:59:36.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:59:35 smithi119 ceph-mon[31518]: pgmap v1057: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:59:36.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:59:35 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1435.681848 secs 2024-03-28T00:59:37.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:59:37 smithi132 ceph-mon[35941]: pgmap v1058: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:59:38.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:59:37 smithi165 ceph-mon[36302]: pgmap v1058: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:59:38.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:59:37 smithi119 ceph-mon[31518]: pgmap v1058: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:59:40.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:59:39 smithi165 ceph-mon[36302]: pgmap v1059: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:59:40.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:59:39 smithi119 ceph-mon[31518]: pgmap v1059: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:59:40.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:59:39 smithi132 ceph-mon[35941]: pgmap v1059: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:59:41.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:59:40 smithi165 ceph-mon[36302]: 98 slow requests, 3 included below; oldest blocked for > 1440.681912 secs 2024-03-28T00:59:41.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:59:40 smithi165 ceph-mon[36302]: slow request 1440.461937 seconds old, received at 2024-03-28T00:35:39.578920+0000: internal op fragmentdir:mds.0:3783 currently failed to wrlock, waiting 2024-03-28T00:59:41.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:59:40 smithi165 ceph-mon[36302]: slow request 1440.452989 seconds old, received at 2024-03-28T00:35:39.587868+0000: client_request(client.25264:33202 create owner_uid=1000, owner_gid=1301 #0x10000000270/file8309 2024-03-28T00:35:39.586399+0000 caller_uid=1000, caller_gid=1301{6,36,1000,1301,}) currently failed to xlock, waiting 2024-03-28T00:59:41.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:59:40 smithi165 ceph-mon[36302]: slow request 1440.681911 seconds old, received at 2024-03-28T00:35:39.358945+0000: peer_request:client.25264:11489 currently acquired locks 2024-03-28T00:59:41.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:59:40 smithi119 ceph-mon[31518]: 98 slow requests, 3 included below; oldest blocked for > 1440.681912 secs 2024-03-28T00:59:41.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:59:40 smithi119 ceph-mon[31518]: slow request 1440.461937 seconds old, received at 2024-03-28T00:35:39.578920+0000: internal op fragmentdir:mds.0:3783 currently failed to wrlock, waiting 2024-03-28T00:59:41.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:59:40 smithi119 ceph-mon[31518]: slow request 1440.452989 seconds old, received at 2024-03-28T00:35:39.587868+0000: client_request(client.25264:33202 create owner_uid=1000, owner_gid=1301 #0x10000000270/file8309 2024-03-28T00:35:39.586399+0000 caller_uid=1000, caller_gid=1301{6,36,1000,1301,}) currently failed to xlock, waiting 2024-03-28T00:59:41.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:59:40 smithi119 ceph-mon[31518]: slow request 1440.681911 seconds old, received at 2024-03-28T00:35:39.358945+0000: peer_request:client.25264:11489 currently acquired locks 2024-03-28T00:59:41.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:59:40 smithi132 ceph-mon[35941]: 98 slow requests, 3 included below; oldest blocked for > 1440.681912 secs 2024-03-28T00:59:41.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:59:40 smithi132 ceph-mon[35941]: slow request 1440.461937 seconds old, received at 2024-03-28T00:35:39.578920+0000: internal op fragmentdir:mds.0:3783 currently failed to wrlock, waiting 2024-03-28T00:59:41.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:59:40 smithi132 ceph-mon[35941]: slow request 1440.452989 seconds old, received at 2024-03-28T00:35:39.587868+0000: client_request(client.25264:33202 create owner_uid=1000, owner_gid=1301 #0x10000000270/file8309 2024-03-28T00:35:39.586399+0000 caller_uid=1000, caller_gid=1301{6,36,1000,1301,}) currently failed to xlock, waiting 2024-03-28T00:59:41.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:59:40 smithi132 ceph-mon[35941]: slow request 1440.681911 seconds old, received at 2024-03-28T00:35:39.358945+0000: peer_request:client.25264:11489 currently acquired locks 2024-03-28T00:59:42.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:59:41 smithi165 ceph-mon[36302]: pgmap v1060: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:59:42.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:59:41 smithi119 ceph-mon[31518]: pgmap v1060: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:59:42.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:59:41 smithi132 ceph-mon[35941]: pgmap v1060: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:59:44.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:59:43 smithi165 ceph-mon[36302]: pgmap v1061: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:59:44.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:59:43 smithi119 ceph-mon[31518]: pgmap v1061: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:59:44.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:59:43 smithi132 ceph-mon[35941]: pgmap v1061: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:59:46.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:59:45 smithi165 ceph-mon[36302]: pgmap v1062: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:59:46.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:59:45 smithi165 ceph-mon[36302]: 98 slow requests, 5 included below; oldest blocked for > 1445.681975 secs 2024-03-28T00:59:46.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:59:45 smithi165 ceph-mon[36302]: slow request 1441.589312 seconds old, received at 2024-03-28T00:35:43.451608+0000: internal op fragmentdir:mds.0:3794 currently failed to wrlock, waiting 2024-03-28T00:59:46.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:59:45 smithi165 ceph-mon[36302]: slow request 1441.571645 seconds old, received at 2024-03-28T00:35:43.469275+0000: internal op fragmentdir:mds.0:3825 currently failed to wrlock, waiting 2024-03-28T00:59:46.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:59:45 smithi165 ceph-mon[36302]: slow request 1441.557553 seconds old, received at 2024-03-28T00:35:43.483367+0000: internal op fragmentdir:mds.0:3857 currently failed to wrlock, waiting 2024-03-28T00:59:46.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:59:45 smithi165 ceph-mon[36302]: slow request 1441.544491 seconds old, received at 2024-03-28T00:35:43.496429+0000: internal op fragmentdir:mds.0:3889 currently failed to wrlock, waiting 2024-03-28T00:59:46.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:59:45 smithi165 ceph-mon[36302]: slow request 1441.587336 seconds old, received at 2024-03-28T00:35:43.453584+0000: internal op fragmentdir:mds.0:3795 currently failed to wrlock, waiting 2024-03-28T00:59:46.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:59:45 smithi119 ceph-mon[31518]: pgmap v1062: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:59:46.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:59:45 smithi119 ceph-mon[31518]: 98 slow requests, 5 included below; oldest blocked for > 1445.681975 secs 2024-03-28T00:59:46.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:59:45 smithi119 ceph-mon[31518]: slow request 1441.589312 seconds old, received at 2024-03-28T00:35:43.451608+0000: internal op fragmentdir:mds.0:3794 currently failed to wrlock, waiting 2024-03-28T00:59:46.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:59:45 smithi119 ceph-mon[31518]: slow request 1441.571645 seconds old, received at 2024-03-28T00:35:43.469275+0000: internal op fragmentdir:mds.0:3825 currently failed to wrlock, waiting 2024-03-28T00:59:46.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:59:45 smithi119 ceph-mon[31518]: slow request 1441.557553 seconds old, received at 2024-03-28T00:35:43.483367+0000: internal op fragmentdir:mds.0:3857 currently failed to wrlock, waiting 2024-03-28T00:59:46.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:59:45 smithi119 ceph-mon[31518]: slow request 1441.544491 seconds old, received at 2024-03-28T00:35:43.496429+0000: internal op fragmentdir:mds.0:3889 currently failed to wrlock, waiting 2024-03-28T00:59:46.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:59:45 smithi119 ceph-mon[31518]: slow request 1441.587336 seconds old, received at 2024-03-28T00:35:43.453584+0000: internal op fragmentdir:mds.0:3795 currently failed to wrlock, waiting 2024-03-28T00:59:46.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:59:45 smithi132 ceph-mon[35941]: pgmap v1062: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:59:46.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:59:45 smithi132 ceph-mon[35941]: 98 slow requests, 5 included below; oldest blocked for > 1445.681975 secs 2024-03-28T00:59:46.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:59:45 smithi132 ceph-mon[35941]: slow request 1441.589312 seconds old, received at 2024-03-28T00:35:43.451608+0000: internal op fragmentdir:mds.0:3794 currently failed to wrlock, waiting 2024-03-28T00:59:46.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:59:45 smithi132 ceph-mon[35941]: slow request 1441.571645 seconds old, received at 2024-03-28T00:35:43.469275+0000: internal op fragmentdir:mds.0:3825 currently failed to wrlock, waiting 2024-03-28T00:59:46.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:59:45 smithi132 ceph-mon[35941]: slow request 1441.557553 seconds old, received at 2024-03-28T00:35:43.483367+0000: internal op fragmentdir:mds.0:3857 currently failed to wrlock, waiting 2024-03-28T00:59:46.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:59:45 smithi132 ceph-mon[35941]: slow request 1441.544491 seconds old, received at 2024-03-28T00:35:43.496429+0000: internal op fragmentdir:mds.0:3889 currently failed to wrlock, waiting 2024-03-28T00:59:46.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:59:45 smithi132 ceph-mon[35941]: slow request 1441.587336 seconds old, received at 2024-03-28T00:35:43.453584+0000: internal op fragmentdir:mds.0:3795 currently failed to wrlock, waiting 2024-03-28T00:59:48.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:59:47 smithi165 ceph-mon[36302]: pgmap v1063: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:59:48.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:59:47 smithi119 ceph-mon[31518]: pgmap v1063: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:59:48.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:59:47 smithi132 ceph-mon[35941]: pgmap v1063: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:59:50.001 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:59:49 smithi132 ceph-mon[35941]: pgmap v1064: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:59:50.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:59:49 smithi165 ceph-mon[36302]: pgmap v1064: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:59:50.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:59:49 smithi119 ceph-mon[31518]: pgmap v1064: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T00:59:51.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:59:50 smithi165 ceph-mon[36302]: 98 slow requests, 5 included below; oldest blocked for > 1450.682064 secs 2024-03-28T00:59:51.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:59:50 smithi165 ceph-mon[36302]: slow request 1446.506162 seconds old, received at 2024-03-28T00:35:43.534847+0000: internal op fragmentdir:mds.0:3922 currently failed to wrlock, waiting 2024-03-28T00:59:51.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:59:50 smithi165 ceph-mon[36302]: slow request 1446.586905 seconds old, received at 2024-03-28T00:35:43.454104+0000: internal op fragmentdir:mds.0:3796 currently failed to wrlock, waiting 2024-03-28T00:59:51.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:59:50 smithi165 ceph-mon[36302]: slow request 1446.570845 seconds old, received at 2024-03-28T00:35:43.470163+0000: internal op fragmentdir:mds.0:3827 currently failed to wrlock, waiting 2024-03-28T00:59:51.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:59:50 smithi165 ceph-mon[36302]: slow request 1446.543683 seconds old, received at 2024-03-28T00:35:43.497326+0000: internal op fragmentdir:mds.0:3891 currently failed to wrlock, waiting 2024-03-28T00:59:51.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:59:50 smithi165 ceph-mon[36302]: slow request 1446.479432 seconds old, received at 2024-03-28T00:35:43.561577+0000: internal op fragmentdir:mds.0:3923 currently failed to wrlock, waiting 2024-03-28T00:59:51.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:59:50 smithi119 ceph-mon[31518]: 98 slow requests, 5 included below; oldest blocked for > 1450.682064 secs 2024-03-28T00:59:51.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:59:50 smithi119 ceph-mon[31518]: slow request 1446.506162 seconds old, received at 2024-03-28T00:35:43.534847+0000: internal op fragmentdir:mds.0:3922 currently failed to wrlock, waiting 2024-03-28T00:59:51.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:59:50 smithi119 ceph-mon[31518]: slow request 1446.586905 seconds old, received at 2024-03-28T00:35:43.454104+0000: internal op fragmentdir:mds.0:3796 currently failed to wrlock, waiting 2024-03-28T00:59:51.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:59:50 smithi119 ceph-mon[31518]: slow request 1446.570845 seconds old, received at 2024-03-28T00:35:43.470163+0000: internal op fragmentdir:mds.0:3827 currently failed to wrlock, waiting 2024-03-28T00:59:51.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:59:50 smithi119 ceph-mon[31518]: slow request 1446.543683 seconds old, received at 2024-03-28T00:35:43.497326+0000: internal op fragmentdir:mds.0:3891 currently failed to wrlock, waiting 2024-03-28T00:59:51.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:59:50 smithi119 ceph-mon[31518]: slow request 1446.479432 seconds old, received at 2024-03-28T00:35:43.561577+0000: internal op fragmentdir:mds.0:3923 currently failed to wrlock, waiting 2024-03-28T00:59:51.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:59:50 smithi132 ceph-mon[35941]: 98 slow requests, 5 included below; oldest blocked for > 1450.682064 secs 2024-03-28T00:59:51.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:59:50 smithi132 ceph-mon[35941]: slow request 1446.506162 seconds old, received at 2024-03-28T00:35:43.534847+0000: internal op fragmentdir:mds.0:3922 currently failed to wrlock, waiting 2024-03-28T00:59:51.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:59:50 smithi132 ceph-mon[35941]: slow request 1446.586905 seconds old, received at 2024-03-28T00:35:43.454104+0000: internal op fragmentdir:mds.0:3796 currently failed to wrlock, waiting 2024-03-28T00:59:51.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:59:50 smithi132 ceph-mon[35941]: slow request 1446.570845 seconds old, received at 2024-03-28T00:35:43.470163+0000: internal op fragmentdir:mds.0:3827 currently failed to wrlock, waiting 2024-03-28T00:59:51.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:59:50 smithi132 ceph-mon[35941]: slow request 1446.543683 seconds old, received at 2024-03-28T00:35:43.497326+0000: internal op fragmentdir:mds.0:3891 currently failed to wrlock, waiting 2024-03-28T00:59:51.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:59:50 smithi132 ceph-mon[35941]: slow request 1446.479432 seconds old, received at 2024-03-28T00:35:43.561577+0000: internal op fragmentdir:mds.0:3923 currently failed to wrlock, waiting 2024-03-28T00:59:52.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:59:51 smithi165 ceph-mon[36302]: pgmap v1065: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:59:52.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:59:51 smithi119 ceph-mon[31518]: pgmap v1065: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:59:52.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:59:51 smithi132 ceph-mon[35941]: pgmap v1065: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:59:54.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:59:53 smithi165 ceph-mon[36302]: pgmap v1066: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T00:59:54.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:59:53 smithi119 ceph-mon[31518]: pgmap v1066: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T00:59:54.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:59:53 smithi132 ceph-mon[35941]: pgmap v1066: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T00:59:56.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:59:55 smithi165 ceph-mon[36302]: pgmap v1067: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:59:56.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:59:55 smithi165 ceph-mon[36302]: 98 slow requests, 5 included below; oldest blocked for > 1455.682099 secs 2024-03-28T00:59:56.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:59:55 smithi165 ceph-mon[36302]: slow request 1451.586466 seconds old, received at 2024-03-28T00:35:43.454578+0000: internal op fragmentdir:mds.0:3797 currently failed to wrlock, waiting 2024-03-28T00:59:56.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:59:55 smithi165 ceph-mon[36302]: slow request 1451.570439 seconds old, received at 2024-03-28T00:35:43.470605+0000: internal op fragmentdir:mds.0:3828 currently failed to wrlock, waiting 2024-03-28T00:59:56.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:59:55 smithi165 ceph-mon[36302]: slow request 1451.556474 seconds old, received at 2024-03-28T00:35:43.484570+0000: internal op fragmentdir:mds.0:3860 currently failed to wrlock, waiting 2024-03-28T00:59:56.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:59:55 smithi165 ceph-mon[36302]: slow request 1451.543268 seconds old, received at 2024-03-28T00:35:43.497776+0000: internal op fragmentdir:mds.0:3892 currently failed to wrlock, waiting 2024-03-28T00:59:56.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:59:55 smithi165 ceph-mon[36302]: slow request 1451.585989 seconds old, received at 2024-03-28T00:35:43.455055+0000: internal op fragmentdir:mds.0:3798 currently failed to wrlock, waiting 2024-03-28T00:59:56.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:59:55 smithi119 ceph-mon[31518]: pgmap v1067: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:59:56.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:59:55 smithi119 ceph-mon[31518]: 98 slow requests, 5 included below; oldest blocked for > 1455.682099 secs 2024-03-28T00:59:56.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:59:55 smithi119 ceph-mon[31518]: slow request 1451.586466 seconds old, received at 2024-03-28T00:35:43.454578+0000: internal op fragmentdir:mds.0:3797 currently failed to wrlock, waiting 2024-03-28T00:59:56.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:59:55 smithi119 ceph-mon[31518]: slow request 1451.570439 seconds old, received at 2024-03-28T00:35:43.470605+0000: internal op fragmentdir:mds.0:3828 currently failed to wrlock, waiting 2024-03-28T00:59:56.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:59:55 smithi119 ceph-mon[31518]: slow request 1451.556474 seconds old, received at 2024-03-28T00:35:43.484570+0000: internal op fragmentdir:mds.0:3860 currently failed to wrlock, waiting 2024-03-28T00:59:56.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:59:55 smithi119 ceph-mon[31518]: slow request 1451.543268 seconds old, received at 2024-03-28T00:35:43.497776+0000: internal op fragmentdir:mds.0:3892 currently failed to wrlock, waiting 2024-03-28T00:59:56.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:59:55 smithi119 ceph-mon[31518]: slow request 1451.585989 seconds old, received at 2024-03-28T00:35:43.455055+0000: internal op fragmentdir:mds.0:3798 currently failed to wrlock, waiting 2024-03-28T00:59:56.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:59:55 smithi132 ceph-mon[35941]: pgmap v1067: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T00:59:56.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:59:55 smithi132 ceph-mon[35941]: 98 slow requests, 5 included below; oldest blocked for > 1455.682099 secs 2024-03-28T00:59:56.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:59:55 smithi132 ceph-mon[35941]: slow request 1451.586466 seconds old, received at 2024-03-28T00:35:43.454578+0000: internal op fragmentdir:mds.0:3797 currently failed to wrlock, waiting 2024-03-28T00:59:56.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:59:55 smithi132 ceph-mon[35941]: slow request 1451.570439 seconds old, received at 2024-03-28T00:35:43.470605+0000: internal op fragmentdir:mds.0:3828 currently failed to wrlock, waiting 2024-03-28T00:59:56.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:59:55 smithi132 ceph-mon[35941]: slow request 1451.556474 seconds old, received at 2024-03-28T00:35:43.484570+0000: internal op fragmentdir:mds.0:3860 currently failed to wrlock, waiting 2024-03-28T00:59:56.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:59:55 smithi132 ceph-mon[35941]: slow request 1451.543268 seconds old, received at 2024-03-28T00:35:43.497776+0000: internal op fragmentdir:mds.0:3892 currently failed to wrlock, waiting 2024-03-28T00:59:56.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:59:55 smithi132 ceph-mon[35941]: slow request 1451.585989 seconds old, received at 2024-03-28T00:35:43.455055+0000: internal op fragmentdir:mds.0:3798 currently failed to wrlock, waiting 2024-03-28T00:59:58.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:59:57 smithi165 ceph-mon[36302]: pgmap v1068: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T00:59:58.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:59:57 smithi119 ceph-mon[31518]: pgmap v1068: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T00:59:58.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:59:57 smithi132 ceph-mon[35941]: pgmap v1068: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T01:00:00.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 00:59:59 smithi165 ceph-mon[36302]: pgmap v1069: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T01:00:00.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 00:59:59 smithi119 ceph-mon[31518]: pgmap v1069: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T01:00:00.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 00:59:59 smithi132 ceph-mon[35941]: pgmap v1069: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T01:00:01.030 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:00:00 smithi165 ceph-mon[36302]: overall HEALTH_WARN 5 failed cephadm daemon(s); 1 filesystem is degraded; 1 MDSs report slow requests 2024-03-28T01:00:01.031 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:00:00 smithi165 ceph-mon[36302]: 98 slow requests, 5 included below; oldest blocked for > 1460.682198 secs 2024-03-28T01:00:01.031 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:00:00 smithi165 ceph-mon[36302]: slow request 1456.542025 seconds old, received at 2024-03-28T00:35:43.499118+0000: internal op fragmentdir:mds.0:3893 currently failed to wrlock, waiting 2024-03-28T01:00:01.031 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:00:00 smithi165 ceph-mon[36302]: slow request 1456.585614 seconds old, received at 2024-03-28T00:35:43.455529+0000: internal op fragmentdir:mds.0:3799 currently failed to wrlock, waiting 2024-03-28T01:00:01.031 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:00:00 smithi165 ceph-mon[36302]: slow request 1456.555745 seconds old, received at 2024-03-28T00:35:43.485397+0000: internal op fragmentdir:mds.0:3862 currently failed to wrlock, waiting 2024-03-28T01:00:01.031 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:00:00 smithi165 ceph-mon[36302]: slow request 1456.541575 seconds old, received at 2024-03-28T00:35:43.499568+0000: internal op fragmentdir:mds.0:3894 currently failed to wrlock, waiting 2024-03-28T01:00:01.031 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:00:00 smithi165 ceph-mon[36302]: slow request 1456.585136 seconds old, received at 2024-03-28T00:35:43.456007+0000: internal op fragmentdir:mds.0:3800 currently failed to wrlock, waiting 2024-03-28T01:00:01.054 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:00:00 smithi119 ceph-mon[31518]: overall HEALTH_WARN 5 failed cephadm daemon(s); 1 filesystem is degraded; 1 MDSs report slow requests 2024-03-28T01:00:01.054 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:00:00 smithi119 ceph-mon[31518]: 98 slow requests, 5 included below; oldest blocked for > 1460.682198 secs 2024-03-28T01:00:01.055 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:00:00 smithi119 ceph-mon[31518]: slow request 1456.542025 seconds old, received at 2024-03-28T00:35:43.499118+0000: internal op fragmentdir:mds.0:3893 currently failed to wrlock, waiting 2024-03-28T01:00:01.055 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:00:00 smithi119 ceph-mon[31518]: slow request 1456.585614 seconds old, received at 2024-03-28T00:35:43.455529+0000: internal op fragmentdir:mds.0:3799 currently failed to wrlock, waiting 2024-03-28T01:00:01.055 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:00:00 smithi119 ceph-mon[31518]: slow request 1456.555745 seconds old, received at 2024-03-28T00:35:43.485397+0000: internal op fragmentdir:mds.0:3862 currently failed to wrlock, waiting 2024-03-28T01:00:01.055 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:00:00 smithi119 ceph-mon[31518]: slow request 1456.541575 seconds old, received at 2024-03-28T00:35:43.499568+0000: internal op fragmentdir:mds.0:3894 currently failed to wrlock, waiting 2024-03-28T01:00:01.055 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:00:00 smithi119 ceph-mon[31518]: slow request 1456.585136 seconds old, received at 2024-03-28T00:35:43.456007+0000: internal op fragmentdir:mds.0:3800 currently failed to wrlock, waiting 2024-03-28T01:00:01.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:00:00 smithi132 ceph-mon[35941]: overall HEALTH_WARN 5 failed cephadm daemon(s); 1 filesystem is degraded; 1 MDSs report slow requests 2024-03-28T01:00:01.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:00:00 smithi132 ceph-mon[35941]: 98 slow requests, 5 included below; oldest blocked for > 1460.682198 secs 2024-03-28T01:00:01.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:00:00 smithi132 ceph-mon[35941]: slow request 1456.542025 seconds old, received at 2024-03-28T00:35:43.499118+0000: internal op fragmentdir:mds.0:3893 currently failed to wrlock, waiting 2024-03-28T01:00:01.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:00:00 smithi132 ceph-mon[35941]: slow request 1456.585614 seconds old, received at 2024-03-28T00:35:43.455529+0000: internal op fragmentdir:mds.0:3799 currently failed to wrlock, waiting 2024-03-28T01:00:01.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:00:00 smithi132 ceph-mon[35941]: slow request 1456.555745 seconds old, received at 2024-03-28T00:35:43.485397+0000: internal op fragmentdir:mds.0:3862 currently failed to wrlock, waiting 2024-03-28T01:00:01.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:00:00 smithi132 ceph-mon[35941]: slow request 1456.541575 seconds old, received at 2024-03-28T00:35:43.499568+0000: internal op fragmentdir:mds.0:3894 currently failed to wrlock, waiting 2024-03-28T01:00:01.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:00:00 smithi132 ceph-mon[35941]: slow request 1456.585136 seconds old, received at 2024-03-28T00:35:43.456007+0000: internal op fragmentdir:mds.0:3800 currently failed to wrlock, waiting 2024-03-28T01:00:02.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:00:01 smithi165 ceph-mon[36302]: pgmap v1070: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:00:02.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:00:01 smithi119 ceph-mon[31518]: pgmap v1070: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:00:02.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:00:01 smithi132 ceph-mon[35941]: pgmap v1070: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:00:04.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:00:03 smithi165 ceph-mon[36302]: pgmap v1071: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:00:04.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:00:03 smithi119 ceph-mon[31518]: pgmap v1071: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:00:04.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:00:03 smithi132 ceph-mon[35941]: pgmap v1071: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:00:06.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:00:05 smithi165 ceph-mon[36302]: pgmap v1072: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:00:06.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:00:05 smithi165 ceph-mon[36302]: 98 slow requests, 5 included below; oldest blocked for > 1465.682288 secs 2024-03-28T01:00:06.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:00:05 smithi165 ceph-mon[36302]: slow request 1461.569561 seconds old, received at 2024-03-28T00:35:43.471672+0000: internal op fragmentdir:mds.0:3831 currently failed to wrlock, waiting 2024-03-28T01:00:06.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:00:05 smithi165 ceph-mon[36302]: slow request 1461.555444 seconds old, received at 2024-03-28T00:35:43.485788+0000: internal op fragmentdir:mds.0:3863 currently failed to wrlock, waiting 2024-03-28T01:00:06.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:00:05 smithi165 ceph-mon[36302]: slow request 1461.541201 seconds old, received at 2024-03-28T00:35:43.500032+0000: internal op fragmentdir:mds.0:3895 currently failed to wrlock, waiting 2024-03-28T01:00:06.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:00:05 smithi165 ceph-mon[36302]: slow request 1461.584769 seconds old, received at 2024-03-28T00:35:43.456464+0000: internal op fragmentdir:mds.0:3801 currently failed to wrlock, waiting 2024-03-28T01:00:06.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:00:05 smithi165 ceph-mon[36302]: slow request 1461.569102 seconds old, received at 2024-03-28T00:35:43.472131+0000: internal op fragmentdir:mds.0:3832 currently failed to wrlock, waiting 2024-03-28T01:00:06.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:00:05 smithi119 ceph-mon[31518]: pgmap v1072: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:00:06.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:00:05 smithi119 ceph-mon[31518]: 98 slow requests, 5 included below; oldest blocked for > 1465.682288 secs 2024-03-28T01:00:06.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:00:05 smithi119 ceph-mon[31518]: slow request 1461.569561 seconds old, received at 2024-03-28T00:35:43.471672+0000: internal op fragmentdir:mds.0:3831 currently failed to wrlock, waiting 2024-03-28T01:00:06.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:00:05 smithi119 ceph-mon[31518]: slow request 1461.555444 seconds old, received at 2024-03-28T00:35:43.485788+0000: internal op fragmentdir:mds.0:3863 currently failed to wrlock, waiting 2024-03-28T01:00:06.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:00:05 smithi119 ceph-mon[31518]: slow request 1461.541201 seconds old, received at 2024-03-28T00:35:43.500032+0000: internal op fragmentdir:mds.0:3895 currently failed to wrlock, waiting 2024-03-28T01:00:06.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:00:05 smithi119 ceph-mon[31518]: slow request 1461.584769 seconds old, received at 2024-03-28T00:35:43.456464+0000: internal op fragmentdir:mds.0:3801 currently failed to wrlock, waiting 2024-03-28T01:00:06.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:00:05 smithi119 ceph-mon[31518]: slow request 1461.569102 seconds old, received at 2024-03-28T00:35:43.472131+0000: internal op fragmentdir:mds.0:3832 currently failed to wrlock, waiting 2024-03-28T01:00:06.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:00:05 smithi132 ceph-mon[35941]: pgmap v1072: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:00:06.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:00:05 smithi132 ceph-mon[35941]: 98 slow requests, 5 included below; oldest blocked for > 1465.682288 secs 2024-03-28T01:00:06.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:00:05 smithi132 ceph-mon[35941]: slow request 1461.569561 seconds old, received at 2024-03-28T00:35:43.471672+0000: internal op fragmentdir:mds.0:3831 currently failed to wrlock, waiting 2024-03-28T01:00:06.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:00:05 smithi132 ceph-mon[35941]: slow request 1461.555444 seconds old, received at 2024-03-28T00:35:43.485788+0000: internal op fragmentdir:mds.0:3863 currently failed to wrlock, waiting 2024-03-28T01:00:06.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:00:05 smithi132 ceph-mon[35941]: slow request 1461.541201 seconds old, received at 2024-03-28T00:35:43.500032+0000: internal op fragmentdir:mds.0:3895 currently failed to wrlock, waiting 2024-03-28T01:00:06.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:00:05 smithi132 ceph-mon[35941]: slow request 1461.584769 seconds old, received at 2024-03-28T00:35:43.456464+0000: internal op fragmentdir:mds.0:3801 currently failed to wrlock, waiting 2024-03-28T01:00:06.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:00:05 smithi132 ceph-mon[35941]: slow request 1461.569102 seconds old, received at 2024-03-28T00:35:43.472131+0000: internal op fragmentdir:mds.0:3832 currently failed to wrlock, waiting 2024-03-28T01:00:08.098 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:00:07 smithi119 ceph-mon[31518]: pgmap v1073: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:00:08.099 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:00:07 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:00:08.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:00:07 smithi165 ceph-mon[36302]: pgmap v1073: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:00:08.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:00:07 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:00:08.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:00:07 smithi132 ceph-mon[35941]: pgmap v1073: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:00:08.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:00:07 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:00:09.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:00:08 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:00:09.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:00:08 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:00:09.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:00:08 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:00:09.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:00:08 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:00:09.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:00:08 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:00:09.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:00:08 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:00:09.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:00:08 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:00:09.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:00:08 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:00:09.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:00:08 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:00:10.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:00:09 smithi165 ceph-mon[36302]: pgmap v1074: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T01:00:10.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:00:09 smithi165 ceph-mon[36302]: pgmap v1075: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:00:10.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:00:09 smithi119 ceph-mon[31518]: pgmap v1074: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T01:00:10.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:00:09 smithi119 ceph-mon[31518]: pgmap v1075: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:00:10.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:00:09 smithi132 ceph-mon[35941]: pgmap v1074: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T01:00:10.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:00:09 smithi132 ceph-mon[35941]: pgmap v1075: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:00:11.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:00:10 smithi165 ceph-mon[36302]: 98 slow requests, 5 included below; oldest blocked for > 1470.682349 secs 2024-03-28T01:00:11.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:00:10 smithi165 ceph-mon[36302]: slow request 1466.584313 seconds old, received at 2024-03-28T00:35:43.456981+0000: internal op fragmentdir:mds.0:3802 currently failed to wrlock, waiting 2024-03-28T01:00:11.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:00:10 smithi165 ceph-mon[36302]: slow request 1466.554704 seconds old, received at 2024-03-28T00:35:43.486590+0000: internal op fragmentdir:mds.0:3865 currently failed to wrlock, waiting 2024-03-28T01:00:11.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:00:10 smithi165 ceph-mon[36302]: slow request 1466.540389 seconds old, received at 2024-03-28T00:35:43.500904+0000: internal op fragmentdir:mds.0:3897 currently failed to wrlock, waiting 2024-03-28T01:00:11.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:00:10 smithi165 ceph-mon[36302]: slow request 1466.583864 seconds old, received at 2024-03-28T00:35:43.457430+0000: internal op fragmentdir:mds.0:3803 currently failed to wrlock, waiting 2024-03-28T01:00:11.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:00:10 smithi165 ceph-mon[36302]: slow request 1466.568278 seconds old, received at 2024-03-28T00:35:43.473016+0000: internal op fragmentdir:mds.0:3834 currently failed to wrlock, waiting 2024-03-28T01:00:11.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:00:10 smithi119 ceph-mon[31518]: 98 slow requests, 5 included below; oldest blocked for > 1470.682349 secs 2024-03-28T01:00:11.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:00:10 smithi119 ceph-mon[31518]: slow request 1466.584313 seconds old, received at 2024-03-28T00:35:43.456981+0000: internal op fragmentdir:mds.0:3802 currently failed to wrlock, waiting 2024-03-28T01:00:11.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:00:10 smithi119 ceph-mon[31518]: slow request 1466.554704 seconds old, received at 2024-03-28T00:35:43.486590+0000: internal op fragmentdir:mds.0:3865 currently failed to wrlock, waiting 2024-03-28T01:00:11.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:00:10 smithi119 ceph-mon[31518]: slow request 1466.540389 seconds old, received at 2024-03-28T00:35:43.500904+0000: internal op fragmentdir:mds.0:3897 currently failed to wrlock, waiting 2024-03-28T01:00:11.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:00:10 smithi119 ceph-mon[31518]: slow request 1466.583864 seconds old, received at 2024-03-28T00:35:43.457430+0000: internal op fragmentdir:mds.0:3803 currently failed to wrlock, waiting 2024-03-28T01:00:11.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:00:10 smithi119 ceph-mon[31518]: slow request 1466.568278 seconds old, received at 2024-03-28T00:35:43.473016+0000: internal op fragmentdir:mds.0:3834 currently failed to wrlock, waiting 2024-03-28T01:00:11.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:00:10 smithi132 ceph-mon[35941]: 98 slow requests, 5 included below; oldest blocked for > 1470.682349 secs 2024-03-28T01:00:11.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:00:10 smithi132 ceph-mon[35941]: slow request 1466.584313 seconds old, received at 2024-03-28T00:35:43.456981+0000: internal op fragmentdir:mds.0:3802 currently failed to wrlock, waiting 2024-03-28T01:00:11.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:00:10 smithi132 ceph-mon[35941]: slow request 1466.554704 seconds old, received at 2024-03-28T00:35:43.486590+0000: internal op fragmentdir:mds.0:3865 currently failed to wrlock, waiting 2024-03-28T01:00:11.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:00:10 smithi132 ceph-mon[35941]: slow request 1466.540389 seconds old, received at 2024-03-28T00:35:43.500904+0000: internal op fragmentdir:mds.0:3897 currently failed to wrlock, waiting 2024-03-28T01:00:11.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:00:10 smithi132 ceph-mon[35941]: slow request 1466.583864 seconds old, received at 2024-03-28T00:35:43.457430+0000: internal op fragmentdir:mds.0:3803 currently failed to wrlock, waiting 2024-03-28T01:00:11.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:00:10 smithi132 ceph-mon[35941]: slow request 1466.568278 seconds old, received at 2024-03-28T00:35:43.473016+0000: internal op fragmentdir:mds.0:3834 currently failed to wrlock, waiting 2024-03-28T01:00:12.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:00:11 smithi165 ceph-mon[36302]: pgmap v1076: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:00:12.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:00:11 smithi119 ceph-mon[31518]: pgmap v1076: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:00:12.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:00:11 smithi132 ceph-mon[35941]: pgmap v1076: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:00:14.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:00:13 smithi165 ceph-mon[36302]: pgmap v1077: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:00:14.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:00:13 smithi119 ceph-mon[31518]: pgmap v1077: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:00:14.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:00:13 smithi132 ceph-mon[35941]: pgmap v1077: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:00:16.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:00:15 smithi165 ceph-mon[36302]: pgmap v1078: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:00:16.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:00:15 smithi165 ceph-mon[36302]: 98 slow requests, 5 included below; oldest blocked for > 1475.682409 secs 2024-03-28T01:00:16.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:00:15 smithi165 ceph-mon[36302]: slow request 1471.583417 seconds old, received at 2024-03-28T00:35:43.457937+0000: internal op fragmentdir:mds.0:3804 currently failed to wrlock, waiting 2024-03-28T01:00:16.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:00:15 smithi165 ceph-mon[36302]: slow request 1471.553968 seconds old, received at 2024-03-28T00:35:43.487385+0000: internal op fragmentdir:mds.0:3867 currently failed to wrlock, waiting 2024-03-28T01:00:16.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:00:15 smithi165 ceph-mon[36302]: slow request 1471.539582 seconds old, received at 2024-03-28T00:35:43.501771+0000: internal op fragmentdir:mds.0:3899 currently failed to wrlock, waiting 2024-03-28T01:00:16.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:00:15 smithi165 ceph-mon[36302]: slow request 1471.582927 seconds old, received at 2024-03-28T00:35:43.458426+0000: internal op fragmentdir:mds.0:3805 currently failed to wrlock, waiting 2024-03-28T01:00:16.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:00:15 smithi165 ceph-mon[36302]: slow request 1471.567451 seconds old, received at 2024-03-28T00:35:43.473902+0000: internal op fragmentdir:mds.0:3836 currently failed to wrlock, waiting 2024-03-28T01:00:16.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:00:15 smithi119 ceph-mon[31518]: pgmap v1078: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:00:16.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:00:15 smithi119 ceph-mon[31518]: 98 slow requests, 5 included below; oldest blocked for > 1475.682409 secs 2024-03-28T01:00:16.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:00:15 smithi119 ceph-mon[31518]: slow request 1471.583417 seconds old, received at 2024-03-28T00:35:43.457937+0000: internal op fragmentdir:mds.0:3804 currently failed to wrlock, waiting 2024-03-28T01:00:16.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:00:15 smithi119 ceph-mon[31518]: slow request 1471.553968 seconds old, received at 2024-03-28T00:35:43.487385+0000: internal op fragmentdir:mds.0:3867 currently failed to wrlock, waiting 2024-03-28T01:00:16.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:00:15 smithi119 ceph-mon[31518]: slow request 1471.539582 seconds old, received at 2024-03-28T00:35:43.501771+0000: internal op fragmentdir:mds.0:3899 currently failed to wrlock, waiting 2024-03-28T01:00:16.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:00:15 smithi119 ceph-mon[31518]: slow request 1471.582927 seconds old, received at 2024-03-28T00:35:43.458426+0000: internal op fragmentdir:mds.0:3805 currently failed to wrlock, waiting 2024-03-28T01:00:16.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:00:15 smithi119 ceph-mon[31518]: slow request 1471.567451 seconds old, received at 2024-03-28T00:35:43.473902+0000: internal op fragmentdir:mds.0:3836 currently failed to wrlock, waiting 2024-03-28T01:00:16.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:00:15 smithi132 ceph-mon[35941]: pgmap v1078: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:00:16.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:00:15 smithi132 ceph-mon[35941]: 98 slow requests, 5 included below; oldest blocked for > 1475.682409 secs 2024-03-28T01:00:16.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:00:15 smithi132 ceph-mon[35941]: slow request 1471.583417 seconds old, received at 2024-03-28T00:35:43.457937+0000: internal op fragmentdir:mds.0:3804 currently failed to wrlock, waiting 2024-03-28T01:00:16.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:00:15 smithi132 ceph-mon[35941]: slow request 1471.553968 seconds old, received at 2024-03-28T00:35:43.487385+0000: internal op fragmentdir:mds.0:3867 currently failed to wrlock, waiting 2024-03-28T01:00:16.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:00:15 smithi132 ceph-mon[35941]: slow request 1471.539582 seconds old, received at 2024-03-28T00:35:43.501771+0000: internal op fragmentdir:mds.0:3899 currently failed to wrlock, waiting 2024-03-28T01:00:16.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:00:15 smithi132 ceph-mon[35941]: slow request 1471.582927 seconds old, received at 2024-03-28T00:35:43.458426+0000: internal op fragmentdir:mds.0:3805 currently failed to wrlock, waiting 2024-03-28T01:00:16.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:00:15 smithi132 ceph-mon[35941]: slow request 1471.567451 seconds old, received at 2024-03-28T00:35:43.473902+0000: internal op fragmentdir:mds.0:3836 currently failed to wrlock, waiting 2024-03-28T01:00:18.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:00:17 smithi165 ceph-mon[36302]: pgmap v1079: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:00:18.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:00:17 smithi119 ceph-mon[31518]: pgmap v1079: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:00:18.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:00:17 smithi132 ceph-mon[35941]: pgmap v1079: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:00:20.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:00:19 smithi165 ceph-mon[36302]: pgmap v1080: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T01:00:20.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:00:19 smithi119 ceph-mon[31518]: pgmap v1080: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T01:00:20.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:00:19 smithi132 ceph-mon[35941]: pgmap v1080: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T01:00:21.100 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:00:20 smithi132 ceph-mon[35941]: 98 slow requests, 5 included below; oldest blocked for > 1480.682471 secs 2024-03-28T01:00:21.101 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:00:20 smithi132 ceph-mon[35941]: slow request 1476.553635 seconds old, received at 2024-03-28T00:35:43.487781+0000: internal op fragmentdir:mds.0:3868 currently failed to wrlock, waiting 2024-03-28T01:00:21.101 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:00:20 smithi132 ceph-mon[35941]: slow request 1476.582504 seconds old, received at 2024-03-28T00:35:43.458913+0000: internal op fragmentdir:mds.0:3806 currently failed to wrlock, waiting 2024-03-28T01:00:21.101 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:00:20 smithi132 ceph-mon[35941]: slow request 1476.567078 seconds old, received at 2024-03-28T00:35:43.474338+0000: internal op fragmentdir:mds.0:3837 currently failed to wrlock, waiting 2024-03-28T01:00:21.101 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:00:20 smithi132 ceph-mon[35941]: slow request 1476.538761 seconds old, received at 2024-03-28T00:35:43.502655+0000: internal op fragmentdir:mds.0:3901 currently failed to wrlock, waiting 2024-03-28T01:00:21.101 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:00:20 smithi132 ceph-mon[35941]: slow request 1476.583149 seconds old, received at 2024-03-28T00:35:43.458267+0000: client_request(client.25264:34113 create owner_uid=1000, owner_gid=1301 #0x10000000272/file7594 2024-03-28T00:35:43.457305+0000 caller_uid=1000, caller_gid=1301{6,36,1000,1301,}) currently failed to xlock, waiting 2024-03-28T01:00:21.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:00:20 smithi165 ceph-mon[36302]: 98 slow requests, 5 included below; oldest blocked for > 1480.682471 secs 2024-03-28T01:00:21.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:00:20 smithi165 ceph-mon[36302]: slow request 1476.553635 seconds old, received at 2024-03-28T00:35:43.487781+0000: internal op fragmentdir:mds.0:3868 currently failed to wrlock, waiting 2024-03-28T01:00:21.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:00:20 smithi165 ceph-mon[36302]: slow request 1476.582504 seconds old, received at 2024-03-28T00:35:43.458913+0000: internal op fragmentdir:mds.0:3806 currently failed to wrlock, waiting 2024-03-28T01:00:21.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:00:20 smithi165 ceph-mon[36302]: slow request 1476.567078 seconds old, received at 2024-03-28T00:35:43.474338+0000: internal op fragmentdir:mds.0:3837 currently failed to wrlock, waiting 2024-03-28T01:00:21.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:00:20 smithi165 ceph-mon[36302]: slow request 1476.538761 seconds old, received at 2024-03-28T00:35:43.502655+0000: internal op fragmentdir:mds.0:3901 currently failed to wrlock, waiting 2024-03-28T01:00:21.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:00:20 smithi165 ceph-mon[36302]: slow request 1476.583149 seconds old, received at 2024-03-28T00:35:43.458267+0000: client_request(client.25264:34113 create owner_uid=1000, owner_gid=1301 #0x10000000272/file7594 2024-03-28T00:35:43.457305+0000 caller_uid=1000, caller_gid=1301{6,36,1000,1301,}) currently failed to xlock, waiting 2024-03-28T01:00:21.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:00:20 smithi119 ceph-mon[31518]: 98 slow requests, 5 included below; oldest blocked for > 1480.682471 secs 2024-03-28T01:00:21.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:00:20 smithi119 ceph-mon[31518]: slow request 1476.553635 seconds old, received at 2024-03-28T00:35:43.487781+0000: internal op fragmentdir:mds.0:3868 currently failed to wrlock, waiting 2024-03-28T01:00:21.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:00:20 smithi119 ceph-mon[31518]: slow request 1476.582504 seconds old, received at 2024-03-28T00:35:43.458913+0000: internal op fragmentdir:mds.0:3806 currently failed to wrlock, waiting 2024-03-28T01:00:21.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:00:20 smithi119 ceph-mon[31518]: slow request 1476.567078 seconds old, received at 2024-03-28T00:35:43.474338+0000: internal op fragmentdir:mds.0:3837 currently failed to wrlock, waiting 2024-03-28T01:00:21.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:00:20 smithi119 ceph-mon[31518]: slow request 1476.538761 seconds old, received at 2024-03-28T00:35:43.502655+0000: internal op fragmentdir:mds.0:3901 currently failed to wrlock, waiting 2024-03-28T01:00:21.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:00:20 smithi119 ceph-mon[31518]: slow request 1476.583149 seconds old, received at 2024-03-28T00:35:43.458267+0000: client_request(client.25264:34113 create owner_uid=1000, owner_gid=1301 #0x10000000272/file7594 2024-03-28T00:35:43.457305+0000 caller_uid=1000, caller_gid=1301{6,36,1000,1301,}) currently failed to xlock, waiting 2024-03-28T01:00:22.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:00:21 smithi165 ceph-mon[36302]: pgmap v1081: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:00:22.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:00:21 smithi119 ceph-mon[31518]: pgmap v1081: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:00:22.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:00:21 smithi132 ceph-mon[35941]: pgmap v1081: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:00:24.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:00:23 smithi165 ceph-mon[36302]: pgmap v1082: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:00:24.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:00:23 smithi119 ceph-mon[31518]: pgmap v1082: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:00:24.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:00:23 smithi132 ceph-mon[35941]: pgmap v1082: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:00:26.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:00:25 smithi165 ceph-mon[36302]: pgmap v1083: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:00:26.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:00:25 smithi165 ceph-mon[36302]: 98 slow requests, 5 included below; oldest blocked for > 1485.682560 secs 2024-03-28T01:00:26.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:00:25 smithi165 ceph-mon[36302]: slow request 1481.553199 seconds old, received at 2024-03-28T00:35:43.488305+0000: internal op fragmentdir:mds.0:3870 currently failed to wrlock, waiting 2024-03-28T01:00:26.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:00:25 smithi165 ceph-mon[36302]: slow request 1481.579657 seconds old, received at 2024-03-28T00:35:43.461848+0000: internal op fragmentdir:mds.0:3807 currently failed to wrlock, waiting 2024-03-28T01:00:26.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:00:25 smithi165 ceph-mon[36302]: slow request 1481.566309 seconds old, received at 2024-03-28T00:35:43.475195+0000: internal op fragmentdir:mds.0:3839 currently failed to wrlock, waiting 2024-03-28T01:00:26.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:00:25 smithi165 ceph-mon[36302]: slow request 1481.552779 seconds old, received at 2024-03-28T00:35:43.488725+0000: internal op fragmentdir:mds.0:3871 currently failed to wrlock, waiting 2024-03-28T01:00:26.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:00:25 smithi165 ceph-mon[36302]: slow request 1481.538197 seconds old, received at 2024-03-28T00:35:43.503308+0000: internal op fragmentdir:mds.0:3903 currently failed to wrlock, waiting 2024-03-28T01:00:26.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:00:25 smithi119 ceph-mon[31518]: pgmap v1083: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:00:26.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:00:25 smithi119 ceph-mon[31518]: 98 slow requests, 5 included below; oldest blocked for > 1485.682560 secs 2024-03-28T01:00:26.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:00:25 smithi119 ceph-mon[31518]: slow request 1481.553199 seconds old, received at 2024-03-28T00:35:43.488305+0000: internal op fragmentdir:mds.0:3870 currently failed to wrlock, waiting 2024-03-28T01:00:26.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:00:25 smithi119 ceph-mon[31518]: slow request 1481.579657 seconds old, received at 2024-03-28T00:35:43.461848+0000: internal op fragmentdir:mds.0:3807 currently failed to wrlock, waiting 2024-03-28T01:00:26.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:00:25 smithi119 ceph-mon[31518]: slow request 1481.566309 seconds old, received at 2024-03-28T00:35:43.475195+0000: internal op fragmentdir:mds.0:3839 currently failed to wrlock, waiting 2024-03-28T01:00:26.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:00:25 smithi119 ceph-mon[31518]: slow request 1481.552779 seconds old, received at 2024-03-28T00:35:43.488725+0000: internal op fragmentdir:mds.0:3871 currently failed to wrlock, waiting 2024-03-28T01:00:26.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:00:25 smithi119 ceph-mon[31518]: slow request 1481.538197 seconds old, received at 2024-03-28T00:35:43.503308+0000: internal op fragmentdir:mds.0:3903 currently failed to wrlock, waiting 2024-03-28T01:00:26.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:00:25 smithi132 ceph-mon[35941]: pgmap v1083: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:00:26.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:00:25 smithi132 ceph-mon[35941]: 98 slow requests, 5 included below; oldest blocked for > 1485.682560 secs 2024-03-28T01:00:26.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:00:25 smithi132 ceph-mon[35941]: slow request 1481.553199 seconds old, received at 2024-03-28T00:35:43.488305+0000: internal op fragmentdir:mds.0:3870 currently failed to wrlock, waiting 2024-03-28T01:00:26.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:00:25 smithi132 ceph-mon[35941]: slow request 1481.579657 seconds old, received at 2024-03-28T00:35:43.461848+0000: internal op fragmentdir:mds.0:3807 currently failed to wrlock, waiting 2024-03-28T01:00:26.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:00:25 smithi132 ceph-mon[35941]: slow request 1481.566309 seconds old, received at 2024-03-28T00:35:43.475195+0000: internal op fragmentdir:mds.0:3839 currently failed to wrlock, waiting 2024-03-28T01:00:26.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:00:25 smithi132 ceph-mon[35941]: slow request 1481.552779 seconds old, received at 2024-03-28T00:35:43.488725+0000: internal op fragmentdir:mds.0:3871 currently failed to wrlock, waiting 2024-03-28T01:00:26.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:00:25 smithi132 ceph-mon[35941]: slow request 1481.538197 seconds old, received at 2024-03-28T00:35:43.503308+0000: internal op fragmentdir:mds.0:3903 currently failed to wrlock, waiting 2024-03-28T01:00:28.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:00:27 smithi165 ceph-mon[36302]: pgmap v1084: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:00:28.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:00:27 smithi119 ceph-mon[31518]: pgmap v1084: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:00:28.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:00:27 smithi132 ceph-mon[35941]: pgmap v1084: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:00:30.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:00:29 smithi165 ceph-mon[36302]: pgmap v1085: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:00:30.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:00:29 smithi119 ceph-mon[31518]: pgmap v1085: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:00:30.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:00:29 smithi132 ceph-mon[35941]: pgmap v1085: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:00:31.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:00:30 smithi119 ceph-mon[31518]: 98 slow requests, 5 included below; oldest blocked for > 1490.682648 secs 2024-03-28T01:00:31.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:00:30 smithi119 ceph-mon[31518]: slow request 1469.664761 seconds old, received at 2024-03-28T00:36:00.376832+0000: client_request(client.25355:96 mksnap #0x1//scheduled-2024-03-28-00_36_00_UTC 2024-03-28T00:36:00.376777+0000 caller_uid=0, caller_gid=0{}) currently failed to acquire_locks 2024-03-28T01:00:31.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:00:30 smithi119 ceph-mon[31518]: slow request 1486.579291 seconds old, received at 2024-03-28T00:35:43.462302+0000: internal op fragmentdir:mds.0:3808 currently failed to wrlock, waiting 2024-03-28T01:00:31.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:00:30 smithi119 ceph-mon[31518]: slow request 1486.642898 seconds old, received at 2024-03-28T00:35:43.398696+0000: internal op fragmentdir:mds.0:3785 currently failed to wrlock, waiting 2024-03-28T01:00:31.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:00:30 smithi119 ceph-mon[31518]: slow request 1486.578829 seconds old, received at 2024-03-28T00:35:43.462764+0000: internal op fragmentdir:mds.0:3809 currently failed to wrlock, waiting 2024-03-28T01:00:31.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:00:30 smithi119 ceph-mon[31518]: slow request 1486.565499 seconds old, received at 2024-03-28T00:35:43.476094+0000: internal op fragmentdir:mds.0:3841 currently failed to wrlock, waiting 2024-03-28T01:00:31.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:00:30 smithi132 ceph-mon[35941]: 98 slow requests, 5 included below; oldest blocked for > 1490.682648 secs 2024-03-28T01:00:31.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:00:30 smithi132 ceph-mon[35941]: slow request 1469.664761 seconds old, received at 2024-03-28T00:36:00.376832+0000: client_request(client.25355:96 mksnap #0x1//scheduled-2024-03-28-00_36_00_UTC 2024-03-28T00:36:00.376777+0000 caller_uid=0, caller_gid=0{}) currently failed to acquire_locks 2024-03-28T01:00:31.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:00:30 smithi132 ceph-mon[35941]: slow request 1486.579291 seconds old, received at 2024-03-28T00:35:43.462302+0000: internal op fragmentdir:mds.0:3808 currently failed to wrlock, waiting 2024-03-28T01:00:31.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:00:30 smithi132 ceph-mon[35941]: slow request 1486.642898 seconds old, received at 2024-03-28T00:35:43.398696+0000: internal op fragmentdir:mds.0:3785 currently failed to wrlock, waiting 2024-03-28T01:00:31.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:00:30 smithi132 ceph-mon[35941]: slow request 1486.578829 seconds old, received at 2024-03-28T00:35:43.462764+0000: internal op fragmentdir:mds.0:3809 currently failed to wrlock, waiting 2024-03-28T01:00:31.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:00:30 smithi132 ceph-mon[35941]: slow request 1486.565499 seconds old, received at 2024-03-28T00:35:43.476094+0000: internal op fragmentdir:mds.0:3841 currently failed to wrlock, waiting 2024-03-28T01:00:31.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:00:30 smithi165 ceph-mon[36302]: 98 slow requests, 5 included below; oldest blocked for > 1490.682648 secs 2024-03-28T01:00:31.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:00:30 smithi165 ceph-mon[36302]: slow request 1469.664761 seconds old, received at 2024-03-28T00:36:00.376832+0000: client_request(client.25355:96 mksnap #0x1//scheduled-2024-03-28-00_36_00_UTC 2024-03-28T00:36:00.376777+0000 caller_uid=0, caller_gid=0{}) currently failed to acquire_locks 2024-03-28T01:00:31.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:00:30 smithi165 ceph-mon[36302]: slow request 1486.579291 seconds old, received at 2024-03-28T00:35:43.462302+0000: internal op fragmentdir:mds.0:3808 currently failed to wrlock, waiting 2024-03-28T01:00:31.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:00:30 smithi165 ceph-mon[36302]: slow request 1486.642898 seconds old, received at 2024-03-28T00:35:43.398696+0000: internal op fragmentdir:mds.0:3785 currently failed to wrlock, waiting 2024-03-28T01:00:31.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:00:30 smithi165 ceph-mon[36302]: slow request 1486.578829 seconds old, received at 2024-03-28T00:35:43.462764+0000: internal op fragmentdir:mds.0:3809 currently failed to wrlock, waiting 2024-03-28T01:00:31.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:00:30 smithi165 ceph-mon[36302]: slow request 1486.565499 seconds old, received at 2024-03-28T00:35:43.476094+0000: internal op fragmentdir:mds.0:3841 currently failed to wrlock, waiting 2024-03-28T01:00:32.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:00:31 smithi119 ceph-mon[31518]: pgmap v1086: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:00:32.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:00:31 smithi132 ceph-mon[35941]: pgmap v1086: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:00:32.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:00:31 smithi165 ceph-mon[36302]: pgmap v1086: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:00:34.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:00:33 smithi119 ceph-mon[31518]: pgmap v1087: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:00:34.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:00:33 smithi132 ceph-mon[35941]: pgmap v1087: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:00:34.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:00:33 smithi165 ceph-mon[36302]: pgmap v1087: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:00:36.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:00:35 smithi119 ceph-mon[31518]: pgmap v1088: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:00:36.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:00:35 smithi119 ceph-mon[31518]: 98 slow requests, 5 included below; oldest blocked for > 1495.682737 secs 2024-03-28T01:00:36.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:00:35 smithi119 ceph-mon[31518]: slow request 1491.552031 seconds old, received at 2024-03-28T00:35:43.489651+0000: internal op fragmentdir:mds.0:3873 currently failed to wrlock, waiting 2024-03-28T01:00:36.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:00:35 smithi119 ceph-mon[31518]: slow request 1490.714061 seconds old, received at 2024-03-28T00:35:44.327621+0000: internal op fragmentdir:mds.0:3937 currently failed to wrlock, waiting 2024-03-28T01:00:36.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:00:35 smithi119 ceph-mon[31518]: slow request 1491.578482 seconds old, received at 2024-03-28T00:35:43.463200+0000: internal op fragmentdir:mds.0:3810 currently failed to wrlock, waiting 2024-03-28T01:00:36.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:00:35 smithi119 ceph-mon[31518]: slow request 1491.565151 seconds old, received at 2024-03-28T00:35:43.476531+0000: internal op fragmentdir:mds.0:3842 currently failed to wrlock, waiting 2024-03-28T01:00:36.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:00:35 smithi119 ceph-mon[31518]: slow request 1491.578081 seconds old, received at 2024-03-28T00:35:43.463601+0000: internal op fragmentdir:mds.0:3811 currently failed to wrlock, waiting 2024-03-28T01:00:36.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:00:35 smithi132 ceph-mon[35941]: pgmap v1088: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:00:36.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:00:35 smithi132 ceph-mon[35941]: 98 slow requests, 5 included below; oldest blocked for > 1495.682737 secs 2024-03-28T01:00:36.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:00:35 smithi132 ceph-mon[35941]: slow request 1491.552031 seconds old, received at 2024-03-28T00:35:43.489651+0000: internal op fragmentdir:mds.0:3873 currently failed to wrlock, waiting 2024-03-28T01:00:36.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:00:35 smithi132 ceph-mon[35941]: slow request 1490.714061 seconds old, received at 2024-03-28T00:35:44.327621+0000: internal op fragmentdir:mds.0:3937 currently failed to wrlock, waiting 2024-03-28T01:00:36.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:00:35 smithi132 ceph-mon[35941]: slow request 1491.578482 seconds old, received at 2024-03-28T00:35:43.463200+0000: internal op fragmentdir:mds.0:3810 currently failed to wrlock, waiting 2024-03-28T01:00:36.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:00:35 smithi132 ceph-mon[35941]: slow request 1491.565151 seconds old, received at 2024-03-28T00:35:43.476531+0000: internal op fragmentdir:mds.0:3842 currently failed to wrlock, waiting 2024-03-28T01:00:36.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:00:35 smithi132 ceph-mon[35941]: slow request 1491.578081 seconds old, received at 2024-03-28T00:35:43.463601+0000: internal op fragmentdir:mds.0:3811 currently failed to wrlock, waiting 2024-03-28T01:00:36.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:00:35 smithi165 ceph-mon[36302]: pgmap v1088: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:00:36.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:00:35 smithi165 ceph-mon[36302]: 98 slow requests, 5 included below; oldest blocked for > 1495.682737 secs 2024-03-28T01:00:36.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:00:35 smithi165 ceph-mon[36302]: slow request 1491.552031 seconds old, received at 2024-03-28T00:35:43.489651+0000: internal op fragmentdir:mds.0:3873 currently failed to wrlock, waiting 2024-03-28T01:00:36.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:00:35 smithi165 ceph-mon[36302]: slow request 1490.714061 seconds old, received at 2024-03-28T00:35:44.327621+0000: internal op fragmentdir:mds.0:3937 currently failed to wrlock, waiting 2024-03-28T01:00:36.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:00:35 smithi165 ceph-mon[36302]: slow request 1491.578482 seconds old, received at 2024-03-28T00:35:43.463200+0000: internal op fragmentdir:mds.0:3810 currently failed to wrlock, waiting 2024-03-28T01:00:36.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:00:35 smithi165 ceph-mon[36302]: slow request 1491.565151 seconds old, received at 2024-03-28T00:35:43.476531+0000: internal op fragmentdir:mds.0:3842 currently failed to wrlock, waiting 2024-03-28T01:00:36.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:00:35 smithi165 ceph-mon[36302]: slow request 1491.578081 seconds old, received at 2024-03-28T00:35:43.463601+0000: internal op fragmentdir:mds.0:3811 currently failed to wrlock, waiting 2024-03-28T01:00:38.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:00:37 smithi119 ceph-mon[31518]: pgmap v1089: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:00:38.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:00:37 smithi132 ceph-mon[35941]: pgmap v1089: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:00:38.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:00:37 smithi165 ceph-mon[36302]: pgmap v1089: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:00:40.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:00:39 smithi132 ceph-mon[35941]: pgmap v1090: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:00:40.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:00:39 smithi165 ceph-mon[36302]: pgmap v1090: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:00:40.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:00:39 smithi119 ceph-mon[31518]: pgmap v1090: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:00:41.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:00:40 smithi132 ceph-mon[35941]: 98 slow requests, 5 included below; oldest blocked for > 1500.682777 secs 2024-03-28T01:00:41.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:00:40 smithi132 ceph-mon[35941]: slow request 1496.535684 seconds old, received at 2024-03-28T00:35:43.506038+0000: internal op fragmentdir:mds.0:3907 currently failed to wrlock, waiting 2024-03-28T01:00:41.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:00:40 smithi132 ceph-mon[35941]: slow request 1496.577686 seconds old, received at 2024-03-28T00:35:43.464036+0000: internal op fragmentdir:mds.0:3812 currently failed to wrlock, waiting 2024-03-28T01:00:41.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:00:40 smithi132 ceph-mon[35941]: slow request 1496.564308 seconds old, received at 2024-03-28T00:35:43.477414+0000: internal op fragmentdir:mds.0:3844 currently failed to wrlock, waiting 2024-03-28T01:00:41.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:00:40 smithi132 ceph-mon[35941]: slow request 1496.533956 seconds old, received at 2024-03-28T00:35:43.507766+0000: internal op fragmentdir:mds.0:3908 currently failed to wrlock, waiting 2024-03-28T01:00:41.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:00:40 smithi132 ceph-mon[35941]: slow request 1497.902357 seconds old, received at 2024-03-28T00:35:42.139365+0000: internal op fragmentdir:mds.0:3784 currently failed to wrlock, waiting 2024-03-28T01:00:41.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:00:40 smithi165 ceph-mon[36302]: 98 slow requests, 5 included below; oldest blocked for > 1500.682777 secs 2024-03-28T01:00:41.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:00:40 smithi165 ceph-mon[36302]: slow request 1496.535684 seconds old, received at 2024-03-28T00:35:43.506038+0000: internal op fragmentdir:mds.0:3907 currently failed to wrlock, waiting 2024-03-28T01:00:41.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:00:40 smithi165 ceph-mon[36302]: slow request 1496.577686 seconds old, received at 2024-03-28T00:35:43.464036+0000: internal op fragmentdir:mds.0:3812 currently failed to wrlock, waiting 2024-03-28T01:00:41.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:00:40 smithi165 ceph-mon[36302]: slow request 1496.564308 seconds old, received at 2024-03-28T00:35:43.477414+0000: internal op fragmentdir:mds.0:3844 currently failed to wrlock, waiting 2024-03-28T01:00:41.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:00:40 smithi165 ceph-mon[36302]: slow request 1496.533956 seconds old, received at 2024-03-28T00:35:43.507766+0000: internal op fragmentdir:mds.0:3908 currently failed to wrlock, waiting 2024-03-28T01:00:41.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:00:40 smithi165 ceph-mon[36302]: slow request 1497.902357 seconds old, received at 2024-03-28T00:35:42.139365+0000: internal op fragmentdir:mds.0:3784 currently failed to wrlock, waiting 2024-03-28T01:00:41.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:00:40 smithi119 ceph-mon[31518]: 98 slow requests, 5 included below; oldest blocked for > 1500.682777 secs 2024-03-28T01:00:41.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:00:40 smithi119 ceph-mon[31518]: slow request 1496.535684 seconds old, received at 2024-03-28T00:35:43.506038+0000: internal op fragmentdir:mds.0:3907 currently failed to wrlock, waiting 2024-03-28T01:00:41.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:00:40 smithi119 ceph-mon[31518]: slow request 1496.577686 seconds old, received at 2024-03-28T00:35:43.464036+0000: internal op fragmentdir:mds.0:3812 currently failed to wrlock, waiting 2024-03-28T01:00:41.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:00:40 smithi119 ceph-mon[31518]: slow request 1496.564308 seconds old, received at 2024-03-28T00:35:43.477414+0000: internal op fragmentdir:mds.0:3844 currently failed to wrlock, waiting 2024-03-28T01:00:41.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:00:40 smithi119 ceph-mon[31518]: slow request 1496.533956 seconds old, received at 2024-03-28T00:35:43.507766+0000: internal op fragmentdir:mds.0:3908 currently failed to wrlock, waiting 2024-03-28T01:00:41.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:00:40 smithi119 ceph-mon[31518]: slow request 1497.902357 seconds old, received at 2024-03-28T00:35:42.139365+0000: internal op fragmentdir:mds.0:3784 currently failed to wrlock, waiting 2024-03-28T01:00:42.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:00:41 smithi132 ceph-mon[35941]: pgmap v1091: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:00:42.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:00:41 smithi165 ceph-mon[36302]: pgmap v1091: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:00:42.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:00:41 smithi119 ceph-mon[31518]: pgmap v1091: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:00:44.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:00:43 smithi132 ceph-mon[35941]: pgmap v1092: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:00:44.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:00:43 smithi165 ceph-mon[36302]: pgmap v1092: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:00:44.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:00:43 smithi119 ceph-mon[31518]: pgmap v1092: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:00:46.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:00:45 smithi132 ceph-mon[35941]: pgmap v1093: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:00:46.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:00:45 smithi132 ceph-mon[35941]: 98 slow requests, 5 included below; oldest blocked for > 1505.682838 secs 2024-03-28T01:00:46.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:00:45 smithi132 ceph-mon[35941]: slow request 1501.577350 seconds old, received at 2024-03-28T00:35:43.464432+0000: internal op fragmentdir:mds.0:3813 currently failed to wrlock, waiting 2024-03-28T01:00:46.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:00:45 smithi132 ceph-mon[35941]: slow request 1501.563925 seconds old, received at 2024-03-28T00:35:43.477858+0000: internal op fragmentdir:mds.0:3845 currently failed to wrlock, waiting 2024-03-28T01:00:46.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:00:45 smithi132 ceph-mon[35941]: slow request 1501.550235 seconds old, received at 2024-03-28T00:35:43.491547+0000: internal op fragmentdir:mds.0:3877 currently failed to wrlock, waiting 2024-03-28T01:00:46.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:00:45 smithi132 ceph-mon[35941]: slow request 1501.576912 seconds old, received at 2024-03-28T00:35:43.464871+0000: internal op fragmentdir:mds.0:3814 currently failed to wrlock, waiting 2024-03-28T01:00:46.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:00:45 smithi132 ceph-mon[35941]: slow request 1501.531959 seconds old, received at 2024-03-28T00:35:43.509824+0000: internal op fragmentdir:mds.0:3910 currently failed to wrlock, waiting 2024-03-28T01:00:46.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:00:45 smithi165 ceph-mon[36302]: pgmap v1093: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:00:46.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:00:45 smithi165 ceph-mon[36302]: 98 slow requests, 5 included below; oldest blocked for > 1505.682838 secs 2024-03-28T01:00:46.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:00:45 smithi165 ceph-mon[36302]: slow request 1501.577350 seconds old, received at 2024-03-28T00:35:43.464432+0000: internal op fragmentdir:mds.0:3813 currently failed to wrlock, waiting 2024-03-28T01:00:46.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:00:45 smithi165 ceph-mon[36302]: slow request 1501.563925 seconds old, received at 2024-03-28T00:35:43.477858+0000: internal op fragmentdir:mds.0:3845 currently failed to wrlock, waiting 2024-03-28T01:00:46.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:00:45 smithi165 ceph-mon[36302]: slow request 1501.550235 seconds old, received at 2024-03-28T00:35:43.491547+0000: internal op fragmentdir:mds.0:3877 currently failed to wrlock, waiting 2024-03-28T01:00:46.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:00:45 smithi165 ceph-mon[36302]: slow request 1501.576912 seconds old, received at 2024-03-28T00:35:43.464871+0000: internal op fragmentdir:mds.0:3814 currently failed to wrlock, waiting 2024-03-28T01:00:46.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:00:45 smithi165 ceph-mon[36302]: slow request 1501.531959 seconds old, received at 2024-03-28T00:35:43.509824+0000: internal op fragmentdir:mds.0:3910 currently failed to wrlock, waiting 2024-03-28T01:00:46.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:00:45 smithi119 ceph-mon[31518]: pgmap v1093: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:00:46.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:00:45 smithi119 ceph-mon[31518]: 98 slow requests, 5 included below; oldest blocked for > 1505.682838 secs 2024-03-28T01:00:46.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:00:45 smithi119 ceph-mon[31518]: slow request 1501.577350 seconds old, received at 2024-03-28T00:35:43.464432+0000: internal op fragmentdir:mds.0:3813 currently failed to wrlock, waiting 2024-03-28T01:00:46.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:00:45 smithi119 ceph-mon[31518]: slow request 1501.563925 seconds old, received at 2024-03-28T00:35:43.477858+0000: internal op fragmentdir:mds.0:3845 currently failed to wrlock, waiting 2024-03-28T01:00:46.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:00:45 smithi119 ceph-mon[31518]: slow request 1501.550235 seconds old, received at 2024-03-28T00:35:43.491547+0000: internal op fragmentdir:mds.0:3877 currently failed to wrlock, waiting 2024-03-28T01:00:46.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:00:45 smithi119 ceph-mon[31518]: slow request 1501.576912 seconds old, received at 2024-03-28T00:35:43.464871+0000: internal op fragmentdir:mds.0:3814 currently failed to wrlock, waiting 2024-03-28T01:00:46.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:00:45 smithi119 ceph-mon[31518]: slow request 1501.531959 seconds old, received at 2024-03-28T00:35:43.509824+0000: internal op fragmentdir:mds.0:3910 currently failed to wrlock, waiting 2024-03-28T01:00:48.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:00:47 smithi132 ceph-mon[35941]: pgmap v1094: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:00:48.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:00:47 smithi165 ceph-mon[36302]: pgmap v1094: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:00:48.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:00:47 smithi119 ceph-mon[31518]: pgmap v1094: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:00:50.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:00:49 smithi132 ceph-mon[35941]: pgmap v1095: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:00:50.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:00:49 smithi165 ceph-mon[36302]: pgmap v1095: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:00:50.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:00:49 smithi119 ceph-mon[31518]: pgmap v1095: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:00:51.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:00:50 smithi132 ceph-mon[35941]: 98 slow requests, 5 included below; oldest blocked for > 1510.682899 secs 2024-03-28T01:00:51.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:00:50 smithi132 ceph-mon[35941]: slow request 1506.616918 seconds old, received at 2024-03-28T00:35:43.424926+0000: internal op fragmentdir:mds.0:3787 currently failed to wrlock, waiting 2024-03-28T01:00:51.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:00:50 smithi132 ceph-mon[35941]: slow request 1506.576533 seconds old, received at 2024-03-28T00:35:43.465310+0000: internal op fragmentdir:mds.0:3815 currently failed to wrlock, waiting 2024-03-28T01:00:51.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:00:50 smithi132 ceph-mon[35941]: slow request 1506.549368 seconds old, received at 2024-03-28T00:35:43.492476+0000: internal op fragmentdir:mds.0:3879 currently failed to wrlock, waiting 2024-03-28T01:00:51.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:00:50 smithi132 ceph-mon[35941]: slow request 1506.576139 seconds old, received at 2024-03-28T00:35:43.465705+0000: internal op fragmentdir:mds.0:3816 currently failed to wrlock, waiting 2024-03-28T01:00:51.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:00:50 smithi132 ceph-mon[35941]: slow request 1506.562578 seconds old, received at 2024-03-28T00:35:43.479266+0000: internal op fragmentdir:mds.0:3848 currently failed to wrlock, waiting 2024-03-28T01:00:51.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:00:50 smithi165 ceph-mon[36302]: 98 slow requests, 5 included below; oldest blocked for > 1510.682899 secs 2024-03-28T01:00:51.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:00:50 smithi165 ceph-mon[36302]: slow request 1506.616918 seconds old, received at 2024-03-28T00:35:43.424926+0000: internal op fragmentdir:mds.0:3787 currently failed to wrlock, waiting 2024-03-28T01:00:51.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:00:50 smithi165 ceph-mon[36302]: slow request 1506.576533 seconds old, received at 2024-03-28T00:35:43.465310+0000: internal op fragmentdir:mds.0:3815 currently failed to wrlock, waiting 2024-03-28T01:00:51.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:00:50 smithi165 ceph-mon[36302]: slow request 1506.549368 seconds old, received at 2024-03-28T00:35:43.492476+0000: internal op fragmentdir:mds.0:3879 currently failed to wrlock, waiting 2024-03-28T01:00:51.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:00:50 smithi165 ceph-mon[36302]: slow request 1506.576139 seconds old, received at 2024-03-28T00:35:43.465705+0000: internal op fragmentdir:mds.0:3816 currently failed to wrlock, waiting 2024-03-28T01:00:51.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:00:50 smithi165 ceph-mon[36302]: slow request 1506.562578 seconds old, received at 2024-03-28T00:35:43.479266+0000: internal op fragmentdir:mds.0:3848 currently failed to wrlock, waiting 2024-03-28T01:00:51.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:00:50 smithi119 ceph-mon[31518]: 98 slow requests, 5 included below; oldest blocked for > 1510.682899 secs 2024-03-28T01:00:51.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:00:50 smithi119 ceph-mon[31518]: slow request 1506.616918 seconds old, received at 2024-03-28T00:35:43.424926+0000: internal op fragmentdir:mds.0:3787 currently failed to wrlock, waiting 2024-03-28T01:00:51.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:00:50 smithi119 ceph-mon[31518]: slow request 1506.576533 seconds old, received at 2024-03-28T00:35:43.465310+0000: internal op fragmentdir:mds.0:3815 currently failed to wrlock, waiting 2024-03-28T01:00:51.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:00:50 smithi119 ceph-mon[31518]: slow request 1506.549368 seconds old, received at 2024-03-28T00:35:43.492476+0000: internal op fragmentdir:mds.0:3879 currently failed to wrlock, waiting 2024-03-28T01:00:51.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:00:50 smithi119 ceph-mon[31518]: slow request 1506.576139 seconds old, received at 2024-03-28T00:35:43.465705+0000: internal op fragmentdir:mds.0:3816 currently failed to wrlock, waiting 2024-03-28T01:00:51.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:00:50 smithi119 ceph-mon[31518]: slow request 1506.562578 seconds old, received at 2024-03-28T00:35:43.479266+0000: internal op fragmentdir:mds.0:3848 currently failed to wrlock, waiting 2024-03-28T01:00:52.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:00:51 smithi132 ceph-mon[35941]: pgmap v1096: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:00:52.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:00:51 smithi165 ceph-mon[36302]: pgmap v1096: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:00:52.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:00:51 smithi119 ceph-mon[31518]: pgmap v1096: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:00:54.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:00:53 smithi165 ceph-mon[36302]: pgmap v1097: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:00:54.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:00:53 smithi119 ceph-mon[31518]: pgmap v1097: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:00:54.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:00:53 smithi132 ceph-mon[35941]: pgmap v1097: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:00:56.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:00:55 smithi165 ceph-mon[36302]: pgmap v1098: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:00:56.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:00:55 smithi165 ceph-mon[36302]: 98 slow requests, 5 included below; oldest blocked for > 1515.682935 secs 2024-03-28T01:00:56.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:00:55 smithi165 ceph-mon[36302]: slow request 1511.548918 seconds old, received at 2024-03-28T00:35:43.492962+0000: internal op fragmentdir:mds.0:3880 currently failed to wrlock, waiting 2024-03-28T01:00:56.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:00:55 smithi165 ceph-mon[36302]: slow request 1512.893808 seconds old, received at 2024-03-28T00:35:42.148071+0000: client_request(client.25264:33954 create owner_uid=1000, owner_gid=1301 #0x10000000271/file8309 2024-03-28T00:35:42.146337+0000 caller_uid=1000, caller_gid=1301{6,36,1000,1301,}) currently failed to xlock, waiting 2024-03-28T01:00:56.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:00:55 smithi165 ceph-mon[36302]: slow request 1511.575779 seconds old, received at 2024-03-28T00:35:43.466100+0000: internal op fragmentdir:mds.0:3817 currently failed to wrlock, waiting 2024-03-28T01:00:56.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:00:55 smithi165 ceph-mon[36302]: slow request 1511.548589 seconds old, received at 2024-03-28T00:35:43.493290+0000: internal op fragmentdir:mds.0:3881 currently failed to wrlock, waiting 2024-03-28T01:00:56.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:00:55 smithi165 ceph-mon[36302]: slow request 1511.575332 seconds old, received at 2024-03-28T00:35:43.466548+0000: internal op fragmentdir:mds.0:3818 currently failed to wrlock, waiting 2024-03-28T01:00:56.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:00:55 smithi119 ceph-mon[31518]: pgmap v1098: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:00:56.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:00:55 smithi119 ceph-mon[31518]: 98 slow requests, 5 included below; oldest blocked for > 1515.682935 secs 2024-03-28T01:00:56.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:00:55 smithi119 ceph-mon[31518]: slow request 1511.548918 seconds old, received at 2024-03-28T00:35:43.492962+0000: internal op fragmentdir:mds.0:3880 currently failed to wrlock, waiting 2024-03-28T01:00:56.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:00:55 smithi119 ceph-mon[31518]: slow request 1512.893808 seconds old, received at 2024-03-28T00:35:42.148071+0000: client_request(client.25264:33954 create owner_uid=1000, owner_gid=1301 #0x10000000271/file8309 2024-03-28T00:35:42.146337+0000 caller_uid=1000, caller_gid=1301{6,36,1000,1301,}) currently failed to xlock, waiting 2024-03-28T01:00:56.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:00:55 smithi119 ceph-mon[31518]: slow request 1511.575779 seconds old, received at 2024-03-28T00:35:43.466100+0000: internal op fragmentdir:mds.0:3817 currently failed to wrlock, waiting 2024-03-28T01:00:56.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:00:55 smithi119 ceph-mon[31518]: slow request 1511.548589 seconds old, received at 2024-03-28T00:35:43.493290+0000: internal op fragmentdir:mds.0:3881 currently failed to wrlock, waiting 2024-03-28T01:00:56.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:00:55 smithi119 ceph-mon[31518]: slow request 1511.575332 seconds old, received at 2024-03-28T00:35:43.466548+0000: internal op fragmentdir:mds.0:3818 currently failed to wrlock, waiting 2024-03-28T01:00:56.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:00:55 smithi132 ceph-mon[35941]: pgmap v1098: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:00:56.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:00:55 smithi132 ceph-mon[35941]: 98 slow requests, 5 included below; oldest blocked for > 1515.682935 secs 2024-03-28T01:00:56.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:00:55 smithi132 ceph-mon[35941]: slow request 1511.548918 seconds old, received at 2024-03-28T00:35:43.492962+0000: internal op fragmentdir:mds.0:3880 currently failed to wrlock, waiting 2024-03-28T01:00:56.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:00:55 smithi132 ceph-mon[35941]: slow request 1512.893808 seconds old, received at 2024-03-28T00:35:42.148071+0000: client_request(client.25264:33954 create owner_uid=1000, owner_gid=1301 #0x10000000271/file8309 2024-03-28T00:35:42.146337+0000 caller_uid=1000, caller_gid=1301{6,36,1000,1301,}) currently failed to xlock, waiting 2024-03-28T01:00:56.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:00:55 smithi132 ceph-mon[35941]: slow request 1511.575779 seconds old, received at 2024-03-28T00:35:43.466100+0000: internal op fragmentdir:mds.0:3817 currently failed to wrlock, waiting 2024-03-28T01:00:56.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:00:55 smithi132 ceph-mon[35941]: slow request 1511.548589 seconds old, received at 2024-03-28T00:35:43.493290+0000: internal op fragmentdir:mds.0:3881 currently failed to wrlock, waiting 2024-03-28T01:00:56.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:00:55 smithi132 ceph-mon[35941]: slow request 1511.575332 seconds old, received at 2024-03-28T00:35:43.466548+0000: internal op fragmentdir:mds.0:3818 currently failed to wrlock, waiting 2024-03-28T01:00:58.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:00:57 smithi165 ceph-mon[36302]: pgmap v1099: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:00:58.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:00:57 smithi119 ceph-mon[31518]: pgmap v1099: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:00:58.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:00:57 smithi132 ceph-mon[35941]: pgmap v1099: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:01:00.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:00:59 smithi165 ceph-mon[36302]: pgmap v1100: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:01:00.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:00:59 smithi119 ceph-mon[31518]: pgmap v1100: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:01:00.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:00:59 smithi132 ceph-mon[35941]: pgmap v1100: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:01:01.280 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:01:00 smithi165 ceph-mon[36302]: 98 slow requests, 5 included below; oldest blocked for > 1520.682998 secs 2024-03-28T01:01:01.280 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:01:00 smithi165 ceph-mon[36302]: slow request 1516.561751 seconds old, received at 2024-03-28T00:35:43.480192+0000: internal op fragmentdir:mds.0:3850 currently failed to wrlock, waiting 2024-03-28T01:01:01.281 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:01:00 smithi165 ceph-mon[36302]: slow request 1516.593199 seconds old, received at 2024-03-28T00:35:43.448744+0000: internal op fragmentdir:mds.0:3788 currently failed to wrlock, waiting 2024-03-28T01:01:01.281 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:01:00 smithi165 ceph-mon[36302]: slow request 1516.574988 seconds old, received at 2024-03-28T00:35:43.466955+0000: internal op fragmentdir:mds.0:3819 currently failed to wrlock, waiting 2024-03-28T01:01:01.281 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:01:00 smithi165 ceph-mon[36302]: slow request 1516.561277 seconds old, received at 2024-03-28T00:35:43.480667+0000: internal op fragmentdir:mds.0:3851 currently failed to wrlock, waiting 2024-03-28T01:01:01.281 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:01:00 smithi165 ceph-mon[36302]: slow request 1516.548104 seconds old, received at 2024-03-28T00:35:43.493839+0000: internal op fragmentdir:mds.0:3883 currently failed to wrlock, waiting 2024-03-28T01:01:01.304 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:01:00 smithi119 ceph-mon[31518]: 98 slow requests, 5 included below; oldest blocked for > 1520.682998 secs 2024-03-28T01:01:01.304 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:01:00 smithi119 ceph-mon[31518]: slow request 1516.561751 seconds old, received at 2024-03-28T00:35:43.480192+0000: internal op fragmentdir:mds.0:3850 currently failed to wrlock, waiting 2024-03-28T01:01:01.304 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:01:00 smithi119 ceph-mon[31518]: slow request 1516.593199 seconds old, received at 2024-03-28T00:35:43.448744+0000: internal op fragmentdir:mds.0:3788 currently failed to wrlock, waiting 2024-03-28T01:01:01.304 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:01:00 smithi119 ceph-mon[31518]: slow request 1516.574988 seconds old, received at 2024-03-28T00:35:43.466955+0000: internal op fragmentdir:mds.0:3819 currently failed to wrlock, waiting 2024-03-28T01:01:01.304 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:01:00 smithi119 ceph-mon[31518]: slow request 1516.561277 seconds old, received at 2024-03-28T00:35:43.480667+0000: internal op fragmentdir:mds.0:3851 currently failed to wrlock, waiting 2024-03-28T01:01:01.304 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:01:00 smithi119 ceph-mon[31518]: slow request 1516.548104 seconds old, received at 2024-03-28T00:35:43.493839+0000: internal op fragmentdir:mds.0:3883 currently failed to wrlock, waiting 2024-03-28T01:01:01.308 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:01:00 smithi132 ceph-mon[35941]: 98 slow requests, 5 included below; oldest blocked for > 1520.682998 secs 2024-03-28T01:01:01.308 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:01:00 smithi132 ceph-mon[35941]: slow request 1516.561751 seconds old, received at 2024-03-28T00:35:43.480192+0000: internal op fragmentdir:mds.0:3850 currently failed to wrlock, waiting 2024-03-28T01:01:01.308 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:01:00 smithi132 ceph-mon[35941]: slow request 1516.593199 seconds old, received at 2024-03-28T00:35:43.448744+0000: internal op fragmentdir:mds.0:3788 currently failed to wrlock, waiting 2024-03-28T01:01:01.308 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:01:00 smithi132 ceph-mon[35941]: slow request 1516.574988 seconds old, received at 2024-03-28T00:35:43.466955+0000: internal op fragmentdir:mds.0:3819 currently failed to wrlock, waiting 2024-03-28T01:01:01.308 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:01:00 smithi132 ceph-mon[35941]: slow request 1516.561277 seconds old, received at 2024-03-28T00:35:43.480667+0000: internal op fragmentdir:mds.0:3851 currently failed to wrlock, waiting 2024-03-28T01:01:01.308 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:01:00 smithi132 ceph-mon[35941]: slow request 1516.548104 seconds old, received at 2024-03-28T00:35:43.493839+0000: internal op fragmentdir:mds.0:3883 currently failed to wrlock, waiting 2024-03-28T01:01:02.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:01:01 smithi119 ceph-mon[31518]: pgmap v1101: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:01:02.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:01:01 smithi165 ceph-mon[36302]: pgmap v1101: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:01:02.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:01:01 smithi132 ceph-mon[35941]: pgmap v1101: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:01:04.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:01:03 smithi165 ceph-mon[36302]: pgmap v1102: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:01:04.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:01:03 smithi119 ceph-mon[31518]: pgmap v1102: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:01:04.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:01:03 smithi132 ceph-mon[35941]: pgmap v1102: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:01:06.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:01:05 smithi165 ceph-mon[36302]: pgmap v1103: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:01:06.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:01:05 smithi165 ceph-mon[36302]: 98 slow requests, 5 included below; oldest blocked for > 1525.683013 secs 2024-03-28T01:01:06.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:01:05 smithi165 ceph-mon[36302]: slow request 1521.592797 seconds old, received at 2024-03-28T00:35:43.449161+0000: internal op fragmentdir:mds.0:3789 currently failed to wrlock, waiting 2024-03-28T01:01:06.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:01:05 smithi165 ceph-mon[36302]: slow request 1521.574567 seconds old, received at 2024-03-28T00:35:43.467391+0000: internal op fragmentdir:mds.0:3820 currently failed to wrlock, waiting 2024-03-28T01:01:06.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:01:05 smithi165 ceph-mon[36302]: slow request 1521.560817 seconds old, received at 2024-03-28T00:35:43.481141+0000: internal op fragmentdir:mds.0:3852 currently failed to wrlock, waiting 2024-03-28T01:01:06.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:01:05 smithi165 ceph-mon[36302]: slow request 1521.519024 seconds old, received at 2024-03-28T00:35:43.522934+0000: internal op fragmentdir:mds.0:3916 currently failed to wrlock, waiting 2024-03-28T01:01:06.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:01:05 smithi165 ceph-mon[36302]: slow request 1521.592337 seconds old, received at 2024-03-28T00:35:43.449621+0000: internal op fragmentdir:mds.0:3790 currently failed to wrlock, waiting 2024-03-28T01:01:06.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:01:05 smithi119 ceph-mon[31518]: pgmap v1103: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:01:06.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:01:05 smithi119 ceph-mon[31518]: 98 slow requests, 5 included below; oldest blocked for > 1525.683013 secs 2024-03-28T01:01:06.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:01:05 smithi119 ceph-mon[31518]: slow request 1521.592797 seconds old, received at 2024-03-28T00:35:43.449161+0000: internal op fragmentdir:mds.0:3789 currently failed to wrlock, waiting 2024-03-28T01:01:06.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:01:05 smithi119 ceph-mon[31518]: slow request 1521.574567 seconds old, received at 2024-03-28T00:35:43.467391+0000: internal op fragmentdir:mds.0:3820 currently failed to wrlock, waiting 2024-03-28T01:01:06.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:01:05 smithi119 ceph-mon[31518]: slow request 1521.560817 seconds old, received at 2024-03-28T00:35:43.481141+0000: internal op fragmentdir:mds.0:3852 currently failed to wrlock, waiting 2024-03-28T01:01:06.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:01:05 smithi119 ceph-mon[31518]: slow request 1521.519024 seconds old, received at 2024-03-28T00:35:43.522934+0000: internal op fragmentdir:mds.0:3916 currently failed to wrlock, waiting 2024-03-28T01:01:06.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:01:05 smithi119 ceph-mon[31518]: slow request 1521.592337 seconds old, received at 2024-03-28T00:35:43.449621+0000: internal op fragmentdir:mds.0:3790 currently failed to wrlock, waiting 2024-03-28T01:01:06.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:01:05 smithi132 ceph-mon[35941]: pgmap v1103: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:01:06.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:01:05 smithi132 ceph-mon[35941]: 98 slow requests, 5 included below; oldest blocked for > 1525.683013 secs 2024-03-28T01:01:06.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:01:05 smithi132 ceph-mon[35941]: slow request 1521.592797 seconds old, received at 2024-03-28T00:35:43.449161+0000: internal op fragmentdir:mds.0:3789 currently failed to wrlock, waiting 2024-03-28T01:01:06.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:01:05 smithi132 ceph-mon[35941]: slow request 1521.574567 seconds old, received at 2024-03-28T00:35:43.467391+0000: internal op fragmentdir:mds.0:3820 currently failed to wrlock, waiting 2024-03-28T01:01:06.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:01:05 smithi132 ceph-mon[35941]: slow request 1521.560817 seconds old, received at 2024-03-28T00:35:43.481141+0000: internal op fragmentdir:mds.0:3852 currently failed to wrlock, waiting 2024-03-28T01:01:06.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:01:05 smithi132 ceph-mon[35941]: slow request 1521.519024 seconds old, received at 2024-03-28T00:35:43.522934+0000: internal op fragmentdir:mds.0:3916 currently failed to wrlock, waiting 2024-03-28T01:01:06.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:01:05 smithi132 ceph-mon[35941]: slow request 1521.592337 seconds old, received at 2024-03-28T00:35:43.449621+0000: internal op fragmentdir:mds.0:3790 currently failed to wrlock, waiting 2024-03-28T01:01:08.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:01:07 smithi165 ceph-mon[36302]: pgmap v1104: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:01:08.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:01:07 smithi119 ceph-mon[31518]: pgmap v1104: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:01:08.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:01:07 smithi132 ceph-mon[35941]: pgmap v1104: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:01:09.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:01:09 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:01:09.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:01:09 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:01:09.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:01:09 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:01:10.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:01:10 smithi165 ceph-mon[36302]: pgmap v1105: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:01:10.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:01:10 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:01:10.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:01:10 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:01:10.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:01:10 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:01:10.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:01:10 smithi119 ceph-mon[31518]: pgmap v1105: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:01:10.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:01:10 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:01:10.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:01:10 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:01:10.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:01:10 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:01:10.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:01:10 smithi132 ceph-mon[35941]: pgmap v1105: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:01:10.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:01:10 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:01:10.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:01:10 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:01:10.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:01:10 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:01:11.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:01:11 smithi165 ceph-mon[36302]: pgmap v1106: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 7 op/s 2024-03-28T01:01:11.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:01:11 smithi165 ceph-mon[36302]: 98 slow requests, 5 included below; oldest blocked for > 1530.683077 secs 2024-03-28T01:01:11.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:01:11 smithi165 ceph-mon[36302]: slow request 1526.574228 seconds old, received at 2024-03-28T00:35:43.467794+0000: internal op fragmentdir:mds.0:3821 currently failed to wrlock, waiting 2024-03-28T01:01:11.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:01:11 smithi165 ceph-mon[36302]: slow request 1526.547400 seconds old, received at 2024-03-28T00:35:43.494622+0000: internal op fragmentdir:mds.0:3885 currently failed to wrlock, waiting 2024-03-28T01:01:11.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:01:11 smithi165 ceph-mon[36302]: slow request 1526.591859 seconds old, received at 2024-03-28T00:35:43.450163+0000: internal op fragmentdir:mds.0:3791 currently failed to wrlock, waiting 2024-03-28T01:01:11.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:01:11 smithi165 ceph-mon[36302]: slow request 1526.559963 seconds old, received at 2024-03-28T00:35:43.482059+0000: internal op fragmentdir:mds.0:3854 currently failed to wrlock, waiting 2024-03-28T01:01:11.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:01:11 smithi165 ceph-mon[36302]: slow request 1526.546948 seconds old, received at 2024-03-28T00:35:43.495074+0000: internal op fragmentdir:mds.0:3886 currently failed to wrlock, waiting 2024-03-28T01:01:11.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:01:11 smithi119 ceph-mon[31518]: pgmap v1106: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 7 op/s 2024-03-28T01:01:11.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:01:11 smithi119 ceph-mon[31518]: 98 slow requests, 5 included below; oldest blocked for > 1530.683077 secs 2024-03-28T01:01:11.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:01:11 smithi119 ceph-mon[31518]: slow request 1526.574228 seconds old, received at 2024-03-28T00:35:43.467794+0000: internal op fragmentdir:mds.0:3821 currently failed to wrlock, waiting 2024-03-28T01:01:11.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:01:11 smithi119 ceph-mon[31518]: slow request 1526.547400 seconds old, received at 2024-03-28T00:35:43.494622+0000: internal op fragmentdir:mds.0:3885 currently failed to wrlock, waiting 2024-03-28T01:01:11.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:01:11 smithi119 ceph-mon[31518]: slow request 1526.591859 seconds old, received at 2024-03-28T00:35:43.450163+0000: internal op fragmentdir:mds.0:3791 currently failed to wrlock, waiting 2024-03-28T01:01:11.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:01:11 smithi119 ceph-mon[31518]: slow request 1526.559963 seconds old, received at 2024-03-28T00:35:43.482059+0000: internal op fragmentdir:mds.0:3854 currently failed to wrlock, waiting 2024-03-28T01:01:11.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:01:11 smithi119 ceph-mon[31518]: slow request 1526.546948 seconds old, received at 2024-03-28T00:35:43.495074+0000: internal op fragmentdir:mds.0:3886 currently failed to wrlock, waiting 2024-03-28T01:01:11.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:01:11 smithi132 ceph-mon[35941]: pgmap v1106: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 7 op/s 2024-03-28T01:01:11.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:01:11 smithi132 ceph-mon[35941]: 98 slow requests, 5 included below; oldest blocked for > 1530.683077 secs 2024-03-28T01:01:11.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:01:11 smithi132 ceph-mon[35941]: slow request 1526.574228 seconds old, received at 2024-03-28T00:35:43.467794+0000: internal op fragmentdir:mds.0:3821 currently failed to wrlock, waiting 2024-03-28T01:01:11.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:01:11 smithi132 ceph-mon[35941]: slow request 1526.547400 seconds old, received at 2024-03-28T00:35:43.494622+0000: internal op fragmentdir:mds.0:3885 currently failed to wrlock, waiting 2024-03-28T01:01:11.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:01:11 smithi132 ceph-mon[35941]: slow request 1526.591859 seconds old, received at 2024-03-28T00:35:43.450163+0000: internal op fragmentdir:mds.0:3791 currently failed to wrlock, waiting 2024-03-28T01:01:11.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:01:11 smithi132 ceph-mon[35941]: slow request 1526.559963 seconds old, received at 2024-03-28T00:35:43.482059+0000: internal op fragmentdir:mds.0:3854 currently failed to wrlock, waiting 2024-03-28T01:01:11.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:01:11 smithi132 ceph-mon[35941]: slow request 1526.546948 seconds old, received at 2024-03-28T00:35:43.495074+0000: internal op fragmentdir:mds.0:3886 currently failed to wrlock, waiting 2024-03-28T01:01:13.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:01:13 smithi165 ceph-mon[36302]: pgmap v1107: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T01:01:13.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:01:13 smithi119 ceph-mon[31518]: pgmap v1107: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T01:01:13.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:01:13 smithi132 ceph-mon[35941]: pgmap v1107: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T01:01:15.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:01:15 smithi165 ceph-mon[36302]: pgmap v1108: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T01:01:15.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:01:15 smithi119 ceph-mon[31518]: pgmap v1108: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T01:01:15.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:01:15 smithi132 ceph-mon[35941]: pgmap v1108: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T01:01:16.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:01:16 smithi165 ceph-mon[36302]: 98 slow requests, 5 included below; oldest blocked for > 1535.683161 secs 2024-03-28T01:01:16.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:01:16 smithi165 ceph-mon[36302]: slow request 1531.518051 seconds old, received at 2024-03-28T00:35:43.524054+0000: internal op fragmentdir:mds.0:3918 currently failed to wrlock, waiting 2024-03-28T01:01:16.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:01:16 smithi165 ceph-mon[36302]: slow request 1531.591492 seconds old, received at 2024-03-28T00:35:43.450613+0000: internal op fragmentdir:mds.0:3792 currently failed to wrlock, waiting 2024-03-28T01:01:16.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:01:16 smithi165 ceph-mon[36302]: slow request 1531.573717 seconds old, received at 2024-03-28T00:35:43.468389+0000: internal op fragmentdir:mds.0:3823 currently failed to wrlock, waiting 2024-03-28T01:01:16.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:01:16 smithi165 ceph-mon[36302]: slow request 1531.591000 seconds old, received at 2024-03-28T00:35:43.451105+0000: internal op fragmentdir:mds.0:3793 currently failed to wrlock, waiting 2024-03-28T01:01:16.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:01:16 smithi165 ceph-mon[36302]: slow request 1531.546158 seconds old, received at 2024-03-28T00:35:43.495948+0000: internal op fragmentdir:mds.0:3888 currently failed to wrlock, waiting 2024-03-28T01:01:16.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:01:16 smithi119 ceph-mon[31518]: 98 slow requests, 5 included below; oldest blocked for > 1535.683161 secs 2024-03-28T01:01:16.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:01:16 smithi119 ceph-mon[31518]: slow request 1531.518051 seconds old, received at 2024-03-28T00:35:43.524054+0000: internal op fragmentdir:mds.0:3918 currently failed to wrlock, waiting 2024-03-28T01:01:16.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:01:16 smithi119 ceph-mon[31518]: slow request 1531.591492 seconds old, received at 2024-03-28T00:35:43.450613+0000: internal op fragmentdir:mds.0:3792 currently failed to wrlock, waiting 2024-03-28T01:01:16.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:01:16 smithi119 ceph-mon[31518]: slow request 1531.573717 seconds old, received at 2024-03-28T00:35:43.468389+0000: internal op fragmentdir:mds.0:3823 currently failed to wrlock, waiting 2024-03-28T01:01:16.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:01:16 smithi119 ceph-mon[31518]: slow request 1531.591000 seconds old, received at 2024-03-28T00:35:43.451105+0000: internal op fragmentdir:mds.0:3793 currently failed to wrlock, waiting 2024-03-28T01:01:16.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:01:16 smithi119 ceph-mon[31518]: slow request 1531.546158 seconds old, received at 2024-03-28T00:35:43.495948+0000: internal op fragmentdir:mds.0:3888 currently failed to wrlock, waiting 2024-03-28T01:01:16.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:01:16 smithi132 ceph-mon[35941]: 98 slow requests, 5 included below; oldest blocked for > 1535.683161 secs 2024-03-28T01:01:16.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:01:16 smithi132 ceph-mon[35941]: slow request 1531.518051 seconds old, received at 2024-03-28T00:35:43.524054+0000: internal op fragmentdir:mds.0:3918 currently failed to wrlock, waiting 2024-03-28T01:01:16.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:01:16 smithi132 ceph-mon[35941]: slow request 1531.591492 seconds old, received at 2024-03-28T00:35:43.450613+0000: internal op fragmentdir:mds.0:3792 currently failed to wrlock, waiting 2024-03-28T01:01:16.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:01:16 smithi132 ceph-mon[35941]: slow request 1531.573717 seconds old, received at 2024-03-28T00:35:43.468389+0000: internal op fragmentdir:mds.0:3823 currently failed to wrlock, waiting 2024-03-28T01:01:16.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:01:16 smithi132 ceph-mon[35941]: slow request 1531.591000 seconds old, received at 2024-03-28T00:35:43.451105+0000: internal op fragmentdir:mds.0:3793 currently failed to wrlock, waiting 2024-03-28T01:01:16.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:01:16 smithi132 ceph-mon[35941]: slow request 1531.546158 seconds old, received at 2024-03-28T00:35:43.495948+0000: internal op fragmentdir:mds.0:3888 currently failed to wrlock, waiting 2024-03-28T01:01:17.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:01:17 smithi165 ceph-mon[36302]: pgmap v1109: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T01:01:17.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:01:17 smithi119 ceph-mon[31518]: pgmap v1109: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T01:01:17.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:01:17 smithi132 ceph-mon[35941]: pgmap v1109: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T01:01:19.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:01:19 smithi165 ceph-mon[36302]: pgmap v1110: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T01:01:19.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:01:19 smithi119 ceph-mon[31518]: pgmap v1110: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T01:01:19.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:01:19 smithi132 ceph-mon[35941]: pgmap v1110: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T01:01:21.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:01:21 smithi165 ceph-mon[36302]: pgmap v1111: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 7 op/s 2024-03-28T01:01:21.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:01:21 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1540.683244 secs 2024-03-28T01:01:21.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:01:21 smithi119 ceph-mon[31518]: pgmap v1111: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 7 op/s 2024-03-28T01:01:21.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:01:21 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1540.683244 secs 2024-03-28T01:01:21.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:01:21 smithi132 ceph-mon[35941]: pgmap v1111: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 7 op/s 2024-03-28T01:01:21.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:01:21 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1540.683244 secs 2024-03-28T01:01:23.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:01:23 smithi165 ceph-mon[36302]: pgmap v1112: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:01:23.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:01:23 smithi119 ceph-mon[31518]: pgmap v1112: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:01:23.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:01:23 smithi132 ceph-mon[35941]: pgmap v1112: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:01:25.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:01:25 smithi165 ceph-mon[36302]: pgmap v1113: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:01:25.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:01:25 smithi119 ceph-mon[31518]: pgmap v1113: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:01:25.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:01:25 smithi132 ceph-mon[35941]: pgmap v1113: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:01:26.304 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:01:26 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1545.683306 secs 2024-03-28T01:01:26.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:01:26 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1545.683306 secs 2024-03-28T01:01:26.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:01:26 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1545.683306 secs 2024-03-28T01:01:27.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:01:27 smithi165 ceph-mon[36302]: pgmap v1114: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:01:27.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:01:27 smithi119 ceph-mon[31518]: pgmap v1114: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:01:27.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:01:27 smithi132 ceph-mon[35941]: pgmap v1114: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:01:29.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:01:29 smithi165 ceph-mon[36302]: pgmap v1115: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:01:29.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:01:29 smithi119 ceph-mon[31518]: pgmap v1115: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:01:29.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:01:29 smithi132 ceph-mon[35941]: pgmap v1115: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:01:31.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:01:31 smithi165 ceph-mon[36302]: pgmap v1116: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:01:31.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:01:31 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1550.683368 secs 2024-03-28T01:01:31.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:01:31 smithi119 ceph-mon[31518]: pgmap v1116: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:01:31.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:01:31 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1550.683368 secs 2024-03-28T01:01:31.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:01:31 smithi132 ceph-mon[35941]: pgmap v1116: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:01:31.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:01:31 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1550.683368 secs 2024-03-28T01:01:33.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:01:33 smithi165 ceph-mon[36302]: pgmap v1117: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:01:33.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:01:33 smithi119 ceph-mon[31518]: pgmap v1117: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:01:33.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:01:33 smithi132 ceph-mon[35941]: pgmap v1117: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:01:35.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:01:35 smithi165 ceph-mon[36302]: pgmap v1118: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:01:35.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:01:35 smithi119 ceph-mon[31518]: pgmap v1118: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:01:35.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:01:35 smithi132 ceph-mon[35941]: pgmap v1118: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:01:36.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:01:36 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1555.683457 secs 2024-03-28T01:01:36.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:01:36 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1555.683457 secs 2024-03-28T01:01:36.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:01:36 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1555.683457 secs 2024-03-28T01:01:37.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:01:37 smithi165 ceph-mon[36302]: pgmap v1119: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:01:37.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:01:37 smithi119 ceph-mon[31518]: pgmap v1119: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:01:37.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:01:37 smithi132 ceph-mon[35941]: pgmap v1119: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:01:38.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:01:38 smithi165 ceph-mon[36302]: pgmap v1120: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:01:38.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:01:38 smithi119 ceph-mon[31518]: pgmap v1120: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:01:38.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:01:38 smithi132 ceph-mon[35941]: pgmap v1120: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:01:41.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:01:40 smithi165 ceph-mon[36302]: pgmap v1121: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:01:41.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:01:40 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1560.683547 secs 2024-03-28T01:01:41.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:01:40 smithi119 ceph-mon[31518]: pgmap v1121: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:01:41.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:01:40 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1560.683547 secs 2024-03-28T01:01:41.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:01:40 smithi132 ceph-mon[35941]: pgmap v1121: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:01:41.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:01:40 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1560.683547 secs 2024-03-28T01:01:43.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:01:42 smithi165 ceph-mon[36302]: pgmap v1122: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:01:43.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:01:42 smithi119 ceph-mon[31518]: pgmap v1122: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:01:43.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:01:42 smithi132 ceph-mon[35941]: pgmap v1122: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:01:45.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:01:44 smithi165 ceph-mon[36302]: pgmap v1123: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:01:45.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:01:44 smithi119 ceph-mon[31518]: pgmap v1123: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:01:45.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:01:44 smithi132 ceph-mon[35941]: pgmap v1123: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:01:46.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:01:45 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1565.683637 secs 2024-03-28T01:01:46.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:01:45 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1565.683637 secs 2024-03-28T01:01:46.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:01:45 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1565.683637 secs 2024-03-28T01:01:47.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:01:46 smithi165 ceph-mon[36302]: pgmap v1124: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:01:47.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:01:46 smithi119 ceph-mon[31518]: pgmap v1124: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:01:47.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:01:46 smithi132 ceph-mon[35941]: pgmap v1124: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:01:49.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:01:48 smithi165 ceph-mon[36302]: pgmap v1125: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:01:49.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:01:48 smithi119 ceph-mon[31518]: pgmap v1125: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:01:49.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:01:48 smithi132 ceph-mon[35941]: pgmap v1125: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:01:51.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:01:50 smithi165 ceph-mon[36302]: pgmap v1126: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:01:51.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:01:50 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1570.683726 secs 2024-03-28T01:01:51.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:01:50 smithi119 ceph-mon[31518]: pgmap v1126: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:01:51.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:01:50 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1570.683726 secs 2024-03-28T01:01:51.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:01:50 smithi132 ceph-mon[35941]: pgmap v1126: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:01:51.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:01:50 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1570.683726 secs 2024-03-28T01:01:53.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:01:52 smithi165 ceph-mon[36302]: pgmap v1127: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:01:53.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:01:52 smithi119 ceph-mon[31518]: pgmap v1127: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:01:53.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:01:52 smithi132 ceph-mon[35941]: pgmap v1127: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:01:55.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:01:54 smithi165 ceph-mon[36302]: pgmap v1128: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:01:55.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:01:54 smithi119 ceph-mon[31518]: pgmap v1128: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:01:55.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:01:54 smithi132 ceph-mon[35941]: pgmap v1128: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:01:56.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:01:55 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1575.683815 secs 2024-03-28T01:01:56.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:01:55 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1575.683815 secs 2024-03-28T01:01:56.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:01:55 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1575.683815 secs 2024-03-28T01:01:57.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:01:56 smithi165 ceph-mon[36302]: pgmap v1129: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:01:57.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:01:56 smithi119 ceph-mon[31518]: pgmap v1129: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:01:57.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:01:56 smithi132 ceph-mon[35941]: pgmap v1129: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:01:59.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:01:58 smithi165 ceph-mon[36302]: pgmap v1130: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:01:59.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:01:58 smithi119 ceph-mon[31518]: pgmap v1130: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:01:59.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:01:58 smithi132 ceph-mon[35941]: pgmap v1130: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:02:01.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:02:00 smithi165 ceph-mon[36302]: pgmap v1131: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:02:01.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:02:00 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1580.683902 secs 2024-03-28T01:02:01.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:02:00 smithi119 ceph-mon[31518]: pgmap v1131: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:02:01.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:02:00 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1580.683902 secs 2024-03-28T01:02:01.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:02:00 smithi132 ceph-mon[35941]: pgmap v1131: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:02:01.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:02:00 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1580.683902 secs 2024-03-28T01:02:03.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:02:02 smithi165 ceph-mon[36302]: pgmap v1132: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:02:03.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:02:02 smithi119 ceph-mon[31518]: pgmap v1132: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:02:03.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:02:02 smithi132 ceph-mon[35941]: pgmap v1132: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:02:05.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:02:04 smithi165 ceph-mon[36302]: pgmap v1133: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:02:05.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:02:04 smithi119 ceph-mon[31518]: pgmap v1133: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:02:05.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:02:04 smithi132 ceph-mon[35941]: pgmap v1133: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:02:06.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:02:05 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1585.683994 secs 2024-03-28T01:02:06.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:02:05 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1585.683994 secs 2024-03-28T01:02:06.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:02:05 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1585.683994 secs 2024-03-28T01:02:07.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:02:06 smithi165 ceph-mon[36302]: pgmap v1134: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:02:07.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:02:06 smithi119 ceph-mon[31518]: pgmap v1134: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:02:07.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:02:06 smithi132 ceph-mon[35941]: pgmap v1134: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:02:09.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:02:08 smithi165 ceph-mon[36302]: pgmap v1135: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:02:09.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:02:08 smithi119 ceph-mon[31518]: pgmap v1135: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:02:09.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:02:08 smithi132 ceph-mon[35941]: pgmap v1135: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:02:11.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:02:10 smithi165 ceph-mon[36302]: pgmap v1136: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:02:11.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:02:10 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:02:11.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:02:10 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1590.684061 secs 2024-03-28T01:02:11.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:02:10 smithi119 ceph-mon[31518]: pgmap v1136: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:02:11.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:02:10 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:02:11.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:02:10 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1590.684061 secs 2024-03-28T01:02:11.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:02:10 smithi132 ceph-mon[35941]: pgmap v1136: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:02:11.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:02:10 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:02:11.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:02:10 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1590.684061 secs 2024-03-28T01:02:12.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:02:11 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:02:12.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:02:11 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:02:12.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:02:11 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:02:12.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:02:11 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:02:12.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:02:11 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:02:12.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:02:11 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:02:12.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:02:11 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:02:12.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:02:11 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:02:12.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:02:11 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:02:13.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:02:12 smithi165 ceph-mon[36302]: pgmap v1137: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T01:02:13.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:02:12 smithi165 ceph-mon[36302]: pgmap v1138: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:02:13.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:02:12 smithi119 ceph-mon[31518]: pgmap v1137: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T01:02:13.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:02:12 smithi119 ceph-mon[31518]: pgmap v1138: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:02:13.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:02:12 smithi132 ceph-mon[35941]: pgmap v1137: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T01:02:13.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:02:12 smithi132 ceph-mon[35941]: pgmap v1138: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:02:15.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:02:14 smithi165 ceph-mon[36302]: pgmap v1139: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:02:15.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:02:14 smithi119 ceph-mon[31518]: pgmap v1139: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:02:15.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:02:14 smithi132 ceph-mon[35941]: pgmap v1139: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:02:16.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:02:15 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1595.684099 secs 2024-03-28T01:02:16.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:02:15 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1595.684099 secs 2024-03-28T01:02:16.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:02:15 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1595.684099 secs 2024-03-28T01:02:17.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:02:16 smithi119 ceph-mon[31518]: pgmap v1140: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:02:17.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:02:16 smithi132 ceph-mon[35941]: pgmap v1140: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:02:17.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:02:16 smithi165 ceph-mon[36302]: pgmap v1140: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:02:19.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:02:18 smithi119 ceph-mon[31518]: pgmap v1141: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:02:19.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:02:18 smithi132 ceph-mon[35941]: pgmap v1141: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:02:19.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:02:18 smithi165 ceph-mon[36302]: pgmap v1141: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:02:21.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:02:20 smithi119 ceph-mon[31518]: pgmap v1142: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:02:21.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:02:20 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1600.684163 secs 2024-03-28T01:02:21.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:02:20 smithi132 ceph-mon[35941]: pgmap v1142: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:02:21.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:02:20 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1600.684163 secs 2024-03-28T01:02:21.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:02:20 smithi165 ceph-mon[36302]: pgmap v1142: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:02:21.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:02:20 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1600.684163 secs 2024-03-28T01:02:23.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:02:22 smithi132 ceph-mon[35941]: pgmap v1143: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T01:02:23.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:02:22 smithi165 ceph-mon[36302]: pgmap v1143: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T01:02:23.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:02:22 smithi119 ceph-mon[31518]: pgmap v1143: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T01:02:25.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:02:24 smithi132 ceph-mon[35941]: pgmap v1144: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:02:25.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:02:24 smithi165 ceph-mon[36302]: pgmap v1144: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:02:25.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:02:24 smithi119 ceph-mon[31518]: pgmap v1144: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:02:26.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:02:25 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1605.684224 secs 2024-03-28T01:02:26.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:02:25 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1605.684224 secs 2024-03-28T01:02:26.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:02:25 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1605.684224 secs 2024-03-28T01:02:27.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:02:26 smithi132 ceph-mon[35941]: pgmap v1145: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:02:27.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:02:26 smithi165 ceph-mon[36302]: pgmap v1145: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:02:27.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:02:26 smithi119 ceph-mon[31518]: pgmap v1145: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:02:29.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:02:28 smithi132 ceph-mon[35941]: pgmap v1146: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:02:29.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:02:28 smithi165 ceph-mon[36302]: pgmap v1146: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:02:29.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:02:28 smithi119 ceph-mon[31518]: pgmap v1146: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:02:31.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:02:30 smithi132 ceph-mon[35941]: pgmap v1147: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:02:31.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:02:30 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1610.684312 secs 2024-03-28T01:02:31.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:02:30 smithi165 ceph-mon[36302]: pgmap v1147: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:02:31.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:02:30 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1610.684312 secs 2024-03-28T01:02:31.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:02:30 smithi119 ceph-mon[31518]: pgmap v1147: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:02:31.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:02:30 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1610.684312 secs 2024-03-28T01:02:33.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:02:32 smithi132 ceph-mon[35941]: pgmap v1148: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:02:33.204 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:02:32 smithi119 ceph-mon[31518]: pgmap v1148: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:02:33.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:02:32 smithi165 ceph-mon[36302]: pgmap v1148: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:02:35.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:02:34 smithi132 ceph-mon[35941]: pgmap v1149: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:02:35.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:02:34 smithi165 ceph-mon[36302]: pgmap v1149: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:02:35.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:02:34 smithi119 ceph-mon[31518]: pgmap v1149: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:02:36.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:02:35 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1615.684376 secs 2024-03-28T01:02:36.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:02:35 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1615.684376 secs 2024-03-28T01:02:36.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:02:35 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1615.684376 secs 2024-03-28T01:02:37.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:02:36 smithi165 ceph-mon[36302]: pgmap v1150: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:02:37.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:02:36 smithi119 ceph-mon[31518]: pgmap v1150: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:02:37.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:02:36 smithi132 ceph-mon[35941]: pgmap v1150: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:02:39.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:02:38 smithi165 ceph-mon[36302]: pgmap v1151: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:02:39.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:02:38 smithi119 ceph-mon[31518]: pgmap v1151: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:02:39.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:02:38 smithi132 ceph-mon[35941]: pgmap v1151: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:02:41.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:02:40 smithi165 ceph-mon[36302]: pgmap v1152: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:02:41.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:02:40 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1620.684465 secs 2024-03-28T01:02:41.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:02:40 smithi119 ceph-mon[31518]: pgmap v1152: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:02:41.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:02:40 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1620.684465 secs 2024-03-28T01:02:41.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:02:40 smithi132 ceph-mon[35941]: pgmap v1152: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:02:41.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:02:40 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1620.684465 secs 2024-03-28T01:02:43.234 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:02:42 smithi119 ceph-mon[31518]: pgmap v1153: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:02:43.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:02:42 smithi165 ceph-mon[36302]: pgmap v1153: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:02:43.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:02:42 smithi132 ceph-mon[35941]: pgmap v1153: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:02:45.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:02:44 smithi165 ceph-mon[36302]: pgmap v1154: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:02:45.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:02:44 smithi119 ceph-mon[31518]: pgmap v1154: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:02:45.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:02:44 smithi132 ceph-mon[35941]: pgmap v1154: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:02:46.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:02:45 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1625.684526 secs 2024-03-28T01:02:46.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:02:45 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1625.684526 secs 2024-03-28T01:02:46.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:02:45 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1625.684526 secs 2024-03-28T01:02:47.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:02:46 smithi165 ceph-mon[36302]: pgmap v1155: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:02:47.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:02:46 smithi119 ceph-mon[31518]: pgmap v1155: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:02:47.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:02:46 smithi132 ceph-mon[35941]: pgmap v1155: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:02:49.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:02:48 smithi165 ceph-mon[36302]: pgmap v1156: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:02:49.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:02:48 smithi119 ceph-mon[31518]: pgmap v1156: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:02:49.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:02:48 smithi132 ceph-mon[35941]: pgmap v1156: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:02:51.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:02:50 smithi165 ceph-mon[36302]: pgmap v1157: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:02:51.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:02:50 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1630.684587 secs 2024-03-28T01:02:51.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:02:50 smithi119 ceph-mon[31518]: pgmap v1157: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:02:51.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:02:50 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1630.684587 secs 2024-03-28T01:02:51.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:02:50 smithi132 ceph-mon[35941]: pgmap v1157: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:02:51.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:02:50 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1630.684587 secs 2024-03-28T01:02:53.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:02:52 smithi165 ceph-mon[36302]: pgmap v1158: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:02:53.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:02:52 smithi119 ceph-mon[31518]: pgmap v1158: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:02:53.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:02:52 smithi132 ceph-mon[35941]: pgmap v1158: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:02:55.251 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:02:54 smithi132 ceph-mon[35941]: pgmap v1159: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:02:55.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:02:54 smithi165 ceph-mon[36302]: pgmap v1159: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:02:55.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:02:54 smithi119 ceph-mon[31518]: pgmap v1159: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:02:56.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:02:56 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1635.684675 secs 2024-03-28T01:02:56.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:02:56 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1635.684675 secs 2024-03-28T01:02:56.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:02:56 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1635.684675 secs 2024-03-28T01:02:57.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:02:57 smithi165 ceph-mon[36302]: pgmap v1160: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:02:57.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:02:57 smithi119 ceph-mon[31518]: pgmap v1160: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:02:57.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:02:57 smithi132 ceph-mon[35941]: pgmap v1160: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:02:59.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:02:59 smithi165 ceph-mon[36302]: pgmap v1161: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:02:59.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:02:59 smithi119 ceph-mon[31518]: pgmap v1161: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:02:59.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:02:59 smithi132 ceph-mon[35941]: pgmap v1161: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:03:01.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:03:01 smithi165 ceph-mon[36302]: pgmap v1162: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:03:01.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:03:01 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1640.684764 secs 2024-03-28T01:03:01.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:03:01 smithi119 ceph-mon[31518]: pgmap v1162: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:03:01.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:03:01 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1640.684764 secs 2024-03-28T01:03:01.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:03:01 smithi132 ceph-mon[35941]: pgmap v1162: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:03:01.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:03:01 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1640.684764 secs 2024-03-28T01:03:03.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:03:03 smithi165 ceph-mon[36302]: pgmap v1163: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:03:03.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:03:03 smithi119 ceph-mon[31518]: pgmap v1163: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:03:03.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:03:03 smithi132 ceph-mon[35941]: pgmap v1163: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:03:05.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:03:05 smithi165 ceph-mon[36302]: pgmap v1164: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:03:05.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:03:05 smithi119 ceph-mon[31518]: pgmap v1164: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:03:05.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:03:05 smithi132 ceph-mon[35941]: pgmap v1164: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:03:06.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:03:06 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1645.684853 secs 2024-03-28T01:03:06.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:03:06 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1645.684853 secs 2024-03-28T01:03:06.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:03:06 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1645.684853 secs 2024-03-28T01:03:07.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:03:07 smithi165 ceph-mon[36302]: pgmap v1165: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:03:07.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:03:07 smithi119 ceph-mon[31518]: pgmap v1165: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:03:07.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:03:07 smithi132 ceph-mon[35941]: pgmap v1165: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:03:09.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:03:09 smithi165 ceph-mon[36302]: pgmap v1166: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:03:09.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:03:09 smithi119 ceph-mon[31518]: pgmap v1166: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:03:09.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:03:09 smithi132 ceph-mon[35941]: pgmap v1166: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:03:11.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:03:11 smithi165 ceph-mon[36302]: pgmap v1167: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:03:11.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:03:11 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1650.684913 secs 2024-03-28T01:03:11.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:03:11 smithi119 ceph-mon[31518]: pgmap v1167: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:03:11.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:03:11 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1650.684913 secs 2024-03-28T01:03:11.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:03:11 smithi132 ceph-mon[35941]: pgmap v1167: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:03:11.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:03:11 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1650.684913 secs 2024-03-28T01:03:12.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:03:12 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:03:12.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:03:12 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:03:12.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:03:12 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:03:13.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:03:13 smithi165 ceph-mon[36302]: pgmap v1168: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:03:13.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:03:13 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:03:13.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:03:13 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:03:13.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:03:13 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:03:13.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:03:13 smithi119 ceph-mon[31518]: pgmap v1168: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:03:13.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:03:13 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:03:13.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:03:13 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:03:13.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:03:13 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:03:13.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:03:13 smithi132 ceph-mon[35941]: pgmap v1168: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:03:13.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:03:13 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:03:13.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:03:13 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:03:13.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:03:13 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:03:14.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:03:14 smithi165 ceph-mon[36302]: pgmap v1169: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T01:03:14.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:03:14 smithi165 ceph-mon[36302]: pgmap v1170: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:03:14.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:03:14 smithi119 ceph-mon[31518]: pgmap v1169: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T01:03:14.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:03:14 smithi119 ceph-mon[31518]: pgmap v1170: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:03:14.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:03:14 smithi132 ceph-mon[35941]: pgmap v1169: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T01:03:14.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:03:14 smithi132 ceph-mon[35941]: pgmap v1170: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:03:16.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:03:16 smithi165 ceph-mon[36302]: pgmap v1171: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:03:16.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:03:16 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1655.685001 secs 2024-03-28T01:03:16.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:03:16 smithi119 ceph-mon[31518]: pgmap v1171: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:03:16.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:03:16 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1655.685001 secs 2024-03-28T01:03:16.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:03:16 smithi132 ceph-mon[35941]: pgmap v1171: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:03:16.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:03:16 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1655.685001 secs 2024-03-28T01:03:18.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:03:18 smithi165 ceph-mon[36302]: pgmap v1172: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:03:18.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:03:18 smithi119 ceph-mon[31518]: pgmap v1172: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:03:18.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:03:18 smithi132 ceph-mon[35941]: pgmap v1172: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:03:20.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:03:20 smithi165 ceph-mon[36302]: pgmap v1173: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-28T01:03:20.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:03:20 smithi119 ceph-mon[31518]: pgmap v1173: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-28T01:03:20.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:03:20 smithi132 ceph-mon[35941]: pgmap v1173: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-28T01:03:21.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:03:21 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1660.685090 secs 2024-03-28T01:03:21.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:03:21 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1660.685090 secs 2024-03-28T01:03:21.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:03:21 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1660.685090 secs 2024-03-28T01:03:22.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:03:22 smithi165 ceph-mon[36302]: pgmap v1174: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:03:22.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:03:22 smithi119 ceph-mon[31518]: pgmap v1174: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:03:22.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:03:22 smithi132 ceph-mon[35941]: pgmap v1174: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:03:24.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:03:24 smithi165 ceph-mon[36302]: pgmap v1175: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T01:03:24.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:03:24 smithi119 ceph-mon[31518]: pgmap v1175: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T01:03:24.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:03:24 smithi132 ceph-mon[35941]: pgmap v1175: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T01:03:26.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:03:26 smithi165 ceph-mon[36302]: pgmap v1176: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:03:26.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:03:26 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1665.685200 secs 2024-03-28T01:03:26.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:03:26 smithi119 ceph-mon[31518]: pgmap v1176: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:03:26.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:03:26 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1665.685200 secs 2024-03-28T01:03:26.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:03:26 smithi132 ceph-mon[35941]: pgmap v1176: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:03:26.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:03:26 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1665.685200 secs 2024-03-28T01:03:28.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:03:28 smithi165 ceph-mon[36302]: pgmap v1177: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:03:28.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:03:28 smithi119 ceph-mon[31518]: pgmap v1177: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:03:28.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:03:28 smithi132 ceph-mon[35941]: pgmap v1177: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:03:30.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:03:30 smithi165 ceph-mon[36302]: pgmap v1178: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:03:30.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:03:30 smithi119 ceph-mon[31518]: pgmap v1178: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:03:30.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:03:30 smithi132 ceph-mon[35941]: pgmap v1178: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:03:31.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:03:31 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1670.685260 secs 2024-03-28T01:03:31.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:03:31 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1670.685260 secs 2024-03-28T01:03:31.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:03:31 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1670.685260 secs 2024-03-28T01:03:32.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:03:32 smithi165 ceph-mon[36302]: pgmap v1179: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:03:32.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:03:32 smithi119 ceph-mon[31518]: pgmap v1179: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:03:32.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:03:32 smithi132 ceph-mon[35941]: pgmap v1179: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:03:33.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:03:33 smithi132 ceph-mon[35941]: pgmap v1180: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:03:33.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:03:33 smithi165 ceph-mon[36302]: pgmap v1180: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:03:33.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:03:33 smithi119 ceph-mon[31518]: pgmap v1180: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:03:36.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:03:35 smithi165 ceph-mon[36302]: pgmap v1181: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:03:36.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:03:35 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1675.685320 secs 2024-03-28T01:03:36.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:03:35 smithi119 ceph-mon[31518]: pgmap v1181: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:03:36.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:03:35 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1675.685320 secs 2024-03-28T01:03:36.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:03:35 smithi132 ceph-mon[35941]: pgmap v1181: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:03:36.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:03:35 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1675.685320 secs 2024-03-28T01:03:38.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:03:37 smithi165 ceph-mon[36302]: pgmap v1182: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:03:38.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:03:37 smithi119 ceph-mon[31518]: pgmap v1182: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:03:38.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:03:37 smithi132 ceph-mon[35941]: pgmap v1182: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:03:40.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:03:39 smithi165 ceph-mon[36302]: pgmap v1183: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:03:40.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:03:39 smithi119 ceph-mon[31518]: pgmap v1183: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:03:40.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:03:39 smithi132 ceph-mon[35941]: pgmap v1183: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:03:41.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:03:40 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1680.685405 secs 2024-03-28T01:03:41.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:03:40 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1680.685405 secs 2024-03-28T01:03:41.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:03:40 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1680.685405 secs 2024-03-28T01:03:42.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:03:41 smithi165 ceph-mon[36302]: pgmap v1184: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:03:42.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:03:41 smithi119 ceph-mon[31518]: pgmap v1184: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:03:42.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:03:41 smithi132 ceph-mon[35941]: pgmap v1184: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:03:44.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:03:43 smithi165 ceph-mon[36302]: pgmap v1185: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:03:44.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:03:43 smithi119 ceph-mon[31518]: pgmap v1185: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:03:44.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:03:43 smithi132 ceph-mon[35941]: pgmap v1185: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:03:46.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:03:45 smithi165 ceph-mon[36302]: pgmap v1186: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:03:46.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:03:45 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1685.685494 secs 2024-03-28T01:03:46.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:03:45 smithi119 ceph-mon[31518]: pgmap v1186: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:03:46.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:03:45 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1685.685494 secs 2024-03-28T01:03:46.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:03:45 smithi132 ceph-mon[35941]: pgmap v1186: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:03:46.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:03:45 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1685.685494 secs 2024-03-28T01:03:48.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:03:47 smithi165 ceph-mon[36302]: pgmap v1187: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:03:48.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:03:47 smithi119 ceph-mon[31518]: pgmap v1187: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:03:48.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:03:47 smithi132 ceph-mon[35941]: pgmap v1187: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:03:50.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:03:49 smithi165 ceph-mon[36302]: pgmap v1188: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:03:50.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:03:49 smithi119 ceph-mon[31518]: pgmap v1188: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:03:50.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:03:49 smithi132 ceph-mon[35941]: pgmap v1188: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:03:51.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:03:50 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1690.685553 secs 2024-03-28T01:03:51.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:03:50 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1690.685553 secs 2024-03-28T01:03:51.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:03:50 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1690.685553 secs 2024-03-28T01:03:52.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:03:51 smithi119 ceph-mon[31518]: pgmap v1189: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:03:52.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:03:51 smithi132 ceph-mon[35941]: pgmap v1189: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:03:52.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:03:51 smithi165 ceph-mon[36302]: pgmap v1189: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:03:54.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:03:53 smithi119 ceph-mon[31518]: pgmap v1190: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:03:54.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:03:53 smithi132 ceph-mon[35941]: pgmap v1190: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:03:54.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:03:53 smithi165 ceph-mon[36302]: pgmap v1190: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:03:56.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:03:55 smithi119 ceph-mon[31518]: pgmap v1191: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:03:56.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:03:55 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1695.685641 secs 2024-03-28T01:03:56.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:03:55 smithi132 ceph-mon[35941]: pgmap v1191: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:03:56.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:03:55 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1695.685641 secs 2024-03-28T01:03:56.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:03:55 smithi165 ceph-mon[36302]: pgmap v1191: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:03:56.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:03:55 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1695.685641 secs 2024-03-28T01:03:58.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:03:57 smithi119 ceph-mon[31518]: pgmap v1192: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:03:58.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:03:57 smithi132 ceph-mon[35941]: pgmap v1192: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:03:58.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:03:57 smithi165 ceph-mon[36302]: pgmap v1192: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:04:00.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:03:59 smithi132 ceph-mon[35941]: pgmap v1193: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:04:00.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:03:59 smithi165 ceph-mon[36302]: pgmap v1193: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:04:00.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:03:59 smithi119 ceph-mon[31518]: pgmap v1193: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:04:01.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:04:00 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1700.685703 secs 2024-03-28T01:04:01.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:04:00 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1700.685703 secs 2024-03-28T01:04:01.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:04:00 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1700.685703 secs 2024-03-28T01:04:02.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:04:01 smithi132 ceph-mon[35941]: pgmap v1194: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:04:02.304 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:04:01 smithi119 ceph-mon[31518]: pgmap v1194: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:04:02.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:04:01 smithi165 ceph-mon[36302]: pgmap v1194: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:04:04.180 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:04:03 smithi165 ceph-mon[36302]: pgmap v1195: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:04:04.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:04:03 smithi132 ceph-mon[35941]: pgmap v1195: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:04:04.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:04:03 smithi119 ceph-mon[31518]: pgmap v1195: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:04:06.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:04:05 smithi132 ceph-mon[35941]: pgmap v1196: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:04:06.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:04:05 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1705.685740 secs 2024-03-28T01:04:06.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:04:05 smithi165 ceph-mon[36302]: pgmap v1196: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:04:06.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:04:05 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1705.685740 secs 2024-03-28T01:04:06.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:04:05 smithi119 ceph-mon[31518]: pgmap v1196: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:04:06.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:04:05 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1705.685740 secs 2024-03-28T01:04:08.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:04:07 smithi132 ceph-mon[35941]: pgmap v1197: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:04:08.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:04:07 smithi165 ceph-mon[36302]: pgmap v1197: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:04:08.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:04:07 smithi119 ceph-mon[31518]: pgmap v1197: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:04:10.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:04:09 smithi132 ceph-mon[35941]: pgmap v1198: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:04:10.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:04:09 smithi165 ceph-mon[36302]: pgmap v1198: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:04:10.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:04:09 smithi119 ceph-mon[31518]: pgmap v1198: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:04:11.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:04:10 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1710.685830 secs 2024-03-28T01:04:11.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:04:10 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1710.685830 secs 2024-03-28T01:04:11.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:04:10 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1710.685830 secs 2024-03-28T01:04:12.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:04:11 smithi132 ceph-mon[35941]: pgmap v1199: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:04:12.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:04:11 smithi165 ceph-mon[36302]: pgmap v1199: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:04:12.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:04:11 smithi119 ceph-mon[31518]: pgmap v1199: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:04:14.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:04:13 smithi132 ceph-mon[35941]: pgmap v1200: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:04:14.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:04:13 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:04:14.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:04:13 smithi165 ceph-mon[36302]: pgmap v1200: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:04:14.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:04:13 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:04:14.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:04:13 smithi119 ceph-mon[31518]: pgmap v1200: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:04:14.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:04:13 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:04:15.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:04:14 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:04:15.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:04:14 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:04:15.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:04:14 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:04:15.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:04:14 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:04:15.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:04:14 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:04:15.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:04:14 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:04:15.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:04:14 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:04:15.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:04:14 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:04:15.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:04:14 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:04:16.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:04:15 smithi165 ceph-mon[36302]: pgmap v1201: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T01:04:16.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:04:15 smithi165 ceph-mon[36302]: pgmap v1202: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:04:16.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:04:15 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1715.685921 secs 2024-03-28T01:04:16.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:04:15 smithi119 ceph-mon[31518]: pgmap v1201: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T01:04:16.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:04:15 smithi119 ceph-mon[31518]: pgmap v1202: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:04:16.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:04:15 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1715.685921 secs 2024-03-28T01:04:16.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:04:15 smithi132 ceph-mon[35941]: pgmap v1201: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T01:04:16.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:04:15 smithi132 ceph-mon[35941]: pgmap v1202: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:04:16.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:04:15 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1715.685921 secs 2024-03-28T01:04:18.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:04:17 smithi165 ceph-mon[36302]: pgmap v1203: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:04:18.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:04:17 smithi119 ceph-mon[31518]: pgmap v1203: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:04:18.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:04:17 smithi132 ceph-mon[35941]: pgmap v1203: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:04:20.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:04:19 smithi165 ceph-mon[36302]: pgmap v1204: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:04:20.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:04:19 smithi119 ceph-mon[31518]: pgmap v1204: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:04:20.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:04:19 smithi132 ceph-mon[35941]: pgmap v1204: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:04:21.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:04:20 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1720.685983 secs 2024-03-28T01:04:21.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:04:20 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1720.685983 secs 2024-03-28T01:04:21.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:04:20 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1720.685983 secs 2024-03-28T01:04:22.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:04:21 smithi165 ceph-mon[36302]: pgmap v1205: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:04:22.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:04:21 smithi119 ceph-mon[31518]: pgmap v1205: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:04:22.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:04:21 smithi132 ceph-mon[35941]: pgmap v1205: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:04:24.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:04:23 smithi165 ceph-mon[36302]: pgmap v1206: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:04:24.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:04:23 smithi119 ceph-mon[31518]: pgmap v1206: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:04:24.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:04:23 smithi132 ceph-mon[35941]: pgmap v1206: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:04:26.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:04:25 smithi165 ceph-mon[36302]: pgmap v1207: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-28T01:04:26.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:04:25 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1725.686068 secs 2024-03-28T01:04:26.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:04:25 smithi119 ceph-mon[31518]: pgmap v1207: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-28T01:04:26.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:04:25 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1725.686068 secs 2024-03-28T01:04:26.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:04:25 smithi132 ceph-mon[35941]: pgmap v1207: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-28T01:04:26.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:04:25 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1725.686068 secs 2024-03-28T01:04:28.251 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:04:27 smithi132 ceph-mon[35941]: pgmap v1208: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T01:04:28.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:04:27 smithi165 ceph-mon[36302]: pgmap v1208: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T01:04:28.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:04:27 smithi119 ceph-mon[31518]: pgmap v1208: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T01:04:30.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:04:29 smithi165 ceph-mon[36302]: pgmap v1209: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T01:04:30.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:04:29 smithi119 ceph-mon[31518]: pgmap v1209: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T01:04:30.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:04:29 smithi132 ceph-mon[35941]: pgmap v1209: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T01:04:31.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:04:31 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1730.686131 secs 2024-03-28T01:04:31.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:04:31 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1730.686131 secs 2024-03-28T01:04:31.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:04:31 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1730.686131 secs 2024-03-28T01:04:32.290 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:04:32 smithi119 ceph-mon[31518]: pgmap v1210: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:04:32.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:04:32 smithi165 ceph-mon[36302]: pgmap v1210: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:04:32.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:04:32 smithi132 ceph-mon[35941]: pgmap v1210: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:04:34.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:04:34 smithi165 ceph-mon[36302]: pgmap v1211: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:04:34.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:04:34 smithi119 ceph-mon[31518]: pgmap v1211: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:04:34.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:04:34 smithi132 ceph-mon[35941]: pgmap v1211: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:04:36.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:04:36 smithi165 ceph-mon[36302]: pgmap v1212: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:04:36.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:04:36 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1735.686223 secs 2024-03-28T01:04:36.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:04:36 smithi119 ceph-mon[31518]: pgmap v1212: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:04:36.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:04:36 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1735.686223 secs 2024-03-28T01:04:36.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:04:36 smithi132 ceph-mon[35941]: pgmap v1212: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:04:36.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:04:36 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1735.686223 secs 2024-03-28T01:04:38.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:04:38 smithi165 ceph-mon[36302]: pgmap v1213: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:04:38.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:04:38 smithi119 ceph-mon[31518]: pgmap v1213: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:04:38.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:04:38 smithi132 ceph-mon[35941]: pgmap v1213: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:04:40.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:04:40 smithi165 ceph-mon[36302]: pgmap v1214: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:04:40.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:04:40 smithi119 ceph-mon[31518]: pgmap v1214: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:04:40.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:04:40 smithi132 ceph-mon[35941]: pgmap v1214: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:04:41.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:04:41 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1740.686285 secs 2024-03-28T01:04:41.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:04:41 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1740.686285 secs 2024-03-28T01:04:41.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:04:41 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1740.686285 secs 2024-03-28T01:04:42.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:04:42 smithi165 ceph-mon[36302]: pgmap v1215: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:04:42.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:04:42 smithi119 ceph-mon[31518]: pgmap v1215: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:04:42.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:04:42 smithi132 ceph-mon[35941]: pgmap v1215: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:04:44.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:04:44 smithi165 ceph-mon[36302]: pgmap v1216: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:04:44.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:04:44 smithi119 ceph-mon[31518]: pgmap v1216: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:04:44.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:04:44 smithi132 ceph-mon[35941]: pgmap v1216: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:04:46.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:04:46 smithi165 ceph-mon[36302]: pgmap v1217: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:04:46.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:04:46 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1745.686347 secs 2024-03-28T01:04:46.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:04:46 smithi119 ceph-mon[31518]: pgmap v1217: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:04:46.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:04:46 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1745.686347 secs 2024-03-28T01:04:46.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:04:46 smithi132 ceph-mon[35941]: pgmap v1217: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:04:46.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:04:46 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1745.686347 secs 2024-03-28T01:04:48.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:04:48 smithi165 ceph-mon[36302]: pgmap v1218: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:04:48.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:04:48 smithi119 ceph-mon[31518]: pgmap v1218: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:04:48.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:04:48 smithi132 ceph-mon[35941]: pgmap v1218: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:04:50.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:04:50 smithi165 ceph-mon[36302]: pgmap v1219: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:04:50.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:04:50 smithi119 ceph-mon[31518]: pgmap v1219: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:04:50.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:04:50 smithi132 ceph-mon[35941]: pgmap v1219: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:04:51.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:04:51 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1750.686358 secs 2024-03-28T01:04:51.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:04:51 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1750.686358 secs 2024-03-28T01:04:51.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:04:51 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1750.686358 secs 2024-03-28T01:04:52.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:04:52 smithi165 ceph-mon[36302]: pgmap v1220: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:04:52.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:04:52 smithi119 ceph-mon[31518]: pgmap v1220: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:04:52.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:04:52 smithi132 ceph-mon[35941]: pgmap v1220: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:04:54.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:04:54 smithi165 ceph-mon[36302]: pgmap v1221: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:04:54.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:04:54 smithi119 ceph-mon[31518]: pgmap v1221: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:04:54.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:04:54 smithi132 ceph-mon[35941]: pgmap v1221: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:04:56.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:04:56 smithi165 ceph-mon[36302]: pgmap v1222: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:04:56.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:04:56 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1755.686442 secs 2024-03-28T01:04:56.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:04:56 smithi119 ceph-mon[31518]: pgmap v1222: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:04:56.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:04:56 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1755.686442 secs 2024-03-28T01:04:56.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:04:56 smithi132 ceph-mon[35941]: pgmap v1222: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:04:56.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:04:56 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1755.686442 secs 2024-03-28T01:04:58.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:04:58 smithi165 ceph-mon[36302]: pgmap v1223: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:04:58.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:04:58 smithi119 ceph-mon[31518]: pgmap v1223: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:04:58.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:04:58 smithi132 ceph-mon[35941]: pgmap v1223: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:05:00.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:05:00 smithi165 ceph-mon[36302]: pgmap v1224: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:05:00.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:05:00 smithi119 ceph-mon[31518]: pgmap v1224: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:05:00.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:05:00 smithi132 ceph-mon[35941]: pgmap v1224: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:05:01.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:05:01 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1760.686532 secs 2024-03-28T01:05:01.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:05:01 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1760.686532 secs 2024-03-28T01:05:01.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:05:01 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1760.686532 secs 2024-03-28T01:05:02.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:05:02 smithi165 ceph-mon[36302]: pgmap v1225: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:05:02.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:05:02 smithi119 ceph-mon[31518]: pgmap v1225: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:05:02.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:05:02 smithi132 ceph-mon[35941]: pgmap v1225: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:05:04.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:05:04 smithi165 ceph-mon[36302]: pgmap v1226: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:05:04.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:05:04 smithi119 ceph-mon[31518]: pgmap v1226: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:05:04.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:05:04 smithi132 ceph-mon[35941]: pgmap v1226: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:05:06.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:05:06 smithi165 ceph-mon[36302]: pgmap v1227: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:05:06.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:05:06 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1765.686574 secs 2024-03-28T01:05:06.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:05:06 smithi119 ceph-mon[31518]: pgmap v1227: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:05:06.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:05:06 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1765.686574 secs 2024-03-28T01:05:06.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:05:06 smithi132 ceph-mon[35941]: pgmap v1227: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:05:06.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:05:06 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1765.686574 secs 2024-03-28T01:05:08.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:05:08 smithi119 ceph-mon[31518]: pgmap v1228: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:05:08.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:05:08 smithi132 ceph-mon[35941]: pgmap v1228: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:05:08.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:05:08 smithi165 ceph-mon[36302]: pgmap v1228: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:05:10.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:05:10 smithi119 ceph-mon[31518]: pgmap v1229: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:05:10.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:05:10 smithi132 ceph-mon[35941]: pgmap v1229: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:05:10.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:05:10 smithi165 ceph-mon[36302]: pgmap v1229: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:05:11.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:05:11 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1770.686615 secs 2024-03-28T01:05:11.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:05:11 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1770.686615 secs 2024-03-28T01:05:11.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:05:11 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1770.686615 secs 2024-03-28T01:05:12.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:05:12 smithi119 ceph-mon[31518]: pgmap v1230: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:05:12.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:05:12 smithi132 ceph-mon[35941]: pgmap v1230: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:05:12.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:05:12 smithi165 ceph-mon[36302]: pgmap v1230: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:05:13.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:05:13 smithi132 ceph-mon[35941]: pgmap v1231: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:05:13.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:05:13 smithi165 ceph-mon[36302]: pgmap v1231: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:05:13.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:05:13 smithi119 ceph-mon[31518]: pgmap v1231: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:05:15.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:05:15 smithi165 ceph-mon[36302]: pgmap v1232: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:05:15.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:05:15 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:05:15.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:05:15 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1775.686678 secs 2024-03-28T01:05:15.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:05:15 smithi119 ceph-mon[31518]: pgmap v1232: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:05:15.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:05:15 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:05:15.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:05:15 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1775.686678 secs 2024-03-28T01:05:15.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:05:15 smithi132 ceph-mon[35941]: pgmap v1232: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:05:15.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:05:15 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:05:15.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:05:15 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1775.686678 secs 2024-03-28T01:05:16.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:05:16 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:05:16.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:05:16 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:05:16.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:05:16 smithi119 ceph-mon[31518]: pgmap v1233: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T01:05:16.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:05:16 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:05:16.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:05:16 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:05:16.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:05:16 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:05:16.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:05:16 smithi132 ceph-mon[35941]: pgmap v1233: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T01:05:16.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:05:16 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:05:16.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:05:16 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:05:16.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:05:16 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:05:16.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:05:16 smithi165 ceph-mon[36302]: pgmap v1233: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T01:05:16.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:05:16 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:05:19.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:05:18 smithi165 ceph-mon[36302]: pgmap v1234: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T01:05:19.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:05:18 smithi119 ceph-mon[31518]: pgmap v1234: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T01:05:19.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:05:18 smithi132 ceph-mon[35941]: pgmap v1234: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T01:05:21.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:05:20 smithi165 ceph-mon[36302]: pgmap v1235: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T01:05:21.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:05:20 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1780.686767 secs 2024-03-28T01:05:21.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:05:20 smithi119 ceph-mon[31518]: pgmap v1235: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T01:05:21.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:05:20 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1780.686767 secs 2024-03-28T01:05:21.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:05:20 smithi132 ceph-mon[35941]: pgmap v1235: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T01:05:21.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:05:20 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1780.686767 secs 2024-03-28T01:05:23.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:05:22 smithi165 ceph-mon[36302]: pgmap v1236: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T01:05:23.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:05:22 smithi119 ceph-mon[31518]: pgmap v1236: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T01:05:23.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:05:22 smithi132 ceph-mon[35941]: pgmap v1236: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T01:05:25.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:05:24 smithi165 ceph-mon[36302]: pgmap v1237: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T01:05:25.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:05:24 smithi119 ceph-mon[31518]: pgmap v1237: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T01:05:25.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:05:24 smithi132 ceph-mon[35941]: pgmap v1237: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T01:05:26.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:05:25 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1785.686853 secs 2024-03-28T01:05:26.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:05:25 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1785.686853 secs 2024-03-28T01:05:26.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:05:25 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1785.686853 secs 2024-03-28T01:05:27.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:05:26 smithi119 ceph-mon[31518]: pgmap v1238: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T01:05:27.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:05:26 smithi132 ceph-mon[35941]: pgmap v1238: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T01:05:27.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:05:26 smithi165 ceph-mon[36302]: pgmap v1238: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T01:05:29.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:05:28 smithi119 ceph-mon[31518]: pgmap v1239: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:05:29.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:05:28 smithi132 ceph-mon[35941]: pgmap v1239: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:05:29.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:05:28 smithi165 ceph-mon[36302]: pgmap v1239: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:05:31.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:05:30 smithi119 ceph-mon[31518]: pgmap v1240: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:05:31.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:05:30 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1790.686901 secs 2024-03-28T01:05:31.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:05:30 smithi132 ceph-mon[35941]: pgmap v1240: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:05:31.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:05:30 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1790.686901 secs 2024-03-28T01:05:31.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:05:30 smithi165 ceph-mon[36302]: pgmap v1240: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:05:31.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:05:30 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1790.686901 secs 2024-03-28T01:05:33.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:05:32 smithi119 ceph-mon[31518]: pgmap v1241: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:05:33.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:05:32 smithi132 ceph-mon[35941]: pgmap v1241: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:05:33.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:05:32 smithi165 ceph-mon[36302]: pgmap v1241: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:05:35.181 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:05:34 smithi165 ceph-mon[36302]: pgmap v1242: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:05:35.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:05:34 smithi132 ceph-mon[35941]: pgmap v1242: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:05:35.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:05:34 smithi119 ceph-mon[31518]: pgmap v1242: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:05:36.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:05:35 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1795.686965 secs 2024-03-28T01:05:36.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:05:35 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1795.686965 secs 2024-03-28T01:05:36.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:05:35 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1795.686965 secs 2024-03-28T01:05:37.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:05:36 smithi132 ceph-mon[35941]: pgmap v1243: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:05:37.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:05:36 smithi165 ceph-mon[36302]: pgmap v1243: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:05:37.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:05:36 smithi119 ceph-mon[31518]: pgmap v1243: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:05:39.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:05:38 smithi132 ceph-mon[35941]: pgmap v1244: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:05:39.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:05:38 smithi165 ceph-mon[36302]: pgmap v1244: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:05:39.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:05:38 smithi119 ceph-mon[31518]: pgmap v1244: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:05:41.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:05:40 smithi132 ceph-mon[35941]: pgmap v1245: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:05:41.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:05:40 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1800.687029 secs 2024-03-28T01:05:41.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:05:40 smithi165 ceph-mon[36302]: pgmap v1245: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:05:41.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:05:40 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1800.687029 secs 2024-03-28T01:05:41.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:05:40 smithi119 ceph-mon[31518]: pgmap v1245: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:05:41.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:05:40 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1800.687029 secs 2024-03-28T01:05:43.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:05:42 smithi132 ceph-mon[35941]: pgmap v1246: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:05:43.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:05:42 smithi165 ceph-mon[36302]: pgmap v1246: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:05:43.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:05:42 smithi119 ceph-mon[31518]: pgmap v1246: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:05:45.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:05:44 smithi132 ceph-mon[35941]: pgmap v1247: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:05:45.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:05:44 smithi165 ceph-mon[36302]: pgmap v1247: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:05:45.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:05:44 smithi119 ceph-mon[31518]: pgmap v1247: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:05:46.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:05:45 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1805.687066 secs 2024-03-28T01:05:46.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:05:45 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1805.687066 secs 2024-03-28T01:05:46.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:05:45 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1805.687066 secs 2024-03-28T01:05:47.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:05:46 smithi165 ceph-mon[36302]: pgmap v1248: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:05:47.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:05:46 smithi119 ceph-mon[31518]: pgmap v1248: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:05:47.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:05:46 smithi132 ceph-mon[35941]: pgmap v1248: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:05:49.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:05:48 smithi165 ceph-mon[36302]: pgmap v1249: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:05:49.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:05:48 smithi119 ceph-mon[31518]: pgmap v1249: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:05:49.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:05:48 smithi132 ceph-mon[35941]: pgmap v1249: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:05:51.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:05:50 smithi165 ceph-mon[36302]: pgmap v1250: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:05:51.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:05:50 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1810.687139 secs 2024-03-28T01:05:51.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:05:50 smithi119 ceph-mon[31518]: pgmap v1250: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:05:51.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:05:50 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1810.687139 secs 2024-03-28T01:05:51.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:05:50 smithi132 ceph-mon[35941]: pgmap v1250: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:05:51.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:05:50 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1810.687139 secs 2024-03-28T01:05:53.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:05:52 smithi165 ceph-mon[36302]: pgmap v1251: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:05:53.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:05:52 smithi119 ceph-mon[31518]: pgmap v1251: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:05:53.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:05:52 smithi132 ceph-mon[35941]: pgmap v1251: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:05:55.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:05:54 smithi165 ceph-mon[36302]: pgmap v1252: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:05:55.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:05:54 smithi119 ceph-mon[31518]: pgmap v1252: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:05:55.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:05:54 smithi132 ceph-mon[35941]: pgmap v1252: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:05:56.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:05:55 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1815.687228 secs 2024-03-28T01:05:56.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:05:55 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1815.687228 secs 2024-03-28T01:05:56.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:05:55 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1815.687228 secs 2024-03-28T01:05:57.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:05:56 smithi165 ceph-mon[36302]: pgmap v1253: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:05:57.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:05:56 smithi119 ceph-mon[31518]: pgmap v1253: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:05:57.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:05:56 smithi132 ceph-mon[35941]: pgmap v1253: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:05:59.251 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:05:58 smithi132 ceph-mon[35941]: pgmap v1254: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:05:59.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:05:58 smithi165 ceph-mon[36302]: pgmap v1254: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:05:59.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:05:58 smithi119 ceph-mon[31518]: pgmap v1254: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:06:01.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:06:00 smithi165 ceph-mon[36302]: pgmap v1255: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:06:01.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:06:00 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1820.687320 secs 2024-03-28T01:06:01.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:06:00 smithi119 ceph-mon[31518]: pgmap v1255: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:06:01.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:06:00 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1820.687320 secs 2024-03-28T01:06:01.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:06:00 smithi132 ceph-mon[35941]: pgmap v1255: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:06:01.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:06:00 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1820.687320 secs 2024-03-28T01:06:03.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:06:02 smithi165 ceph-mon[36302]: pgmap v1256: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:06:03.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:06:02 smithi119 ceph-mon[31518]: pgmap v1256: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:06:03.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:06:02 smithi132 ceph-mon[35941]: pgmap v1256: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:06:05.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:06:04 smithi165 ceph-mon[36302]: pgmap v1257: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:06:05.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:06:04 smithi119 ceph-mon[31518]: pgmap v1257: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:06:05.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:06:04 smithi132 ceph-mon[35941]: pgmap v1257: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:06:06.308 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:06:06 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1825.687411 secs 2024-03-28T01:06:06.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:06:06 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1825.687411 secs 2024-03-28T01:06:06.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:06:06 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1825.687411 secs 2024-03-28T01:06:07.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:06:07 smithi165 ceph-mon[36302]: pgmap v1258: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:06:07.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:06:07 smithi119 ceph-mon[31518]: pgmap v1258: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:06:07.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:06:07 smithi132 ceph-mon[35941]: pgmap v1258: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:06:09.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:06:09 smithi165 ceph-mon[36302]: pgmap v1259: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:06:09.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:06:09 smithi119 ceph-mon[31518]: pgmap v1259: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:06:09.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:06:09 smithi132 ceph-mon[35941]: pgmap v1259: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:06:11.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:06:11 smithi165 ceph-mon[36302]: pgmap v1260: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:06:11.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:06:11 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1830.687502 secs 2024-03-28T01:06:11.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:06:11 smithi119 ceph-mon[31518]: pgmap v1260: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:06:11.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:06:11 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1830.687502 secs 2024-03-28T01:06:11.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:06:11 smithi132 ceph-mon[35941]: pgmap v1260: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:06:11.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:06:11 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1830.687502 secs 2024-03-28T01:06:13.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:06:13 smithi165 ceph-mon[36302]: pgmap v1261: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:06:13.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:06:13 smithi119 ceph-mon[31518]: pgmap v1261: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:06:13.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:06:13 smithi132 ceph-mon[35941]: pgmap v1261: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:06:15.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:06:15 smithi165 ceph-mon[36302]: pgmap v1262: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:06:15.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:06:15 smithi119 ceph-mon[31518]: pgmap v1262: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:06:15.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:06:15 smithi132 ceph-mon[35941]: pgmap v1262: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:06:16.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:06:16 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1835.687516 secs 2024-03-28T01:06:16.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:06:16 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1835.687516 secs 2024-03-28T01:06:16.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:06:16 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1835.687516 secs 2024-03-28T01:06:17.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:06:17 smithi165 ceph-mon[36302]: pgmap v1263: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:06:17.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:06:17 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:06:17.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:06:17 smithi119 ceph-mon[31518]: pgmap v1263: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:06:17.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:06:17 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:06:17.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:06:17 smithi132 ceph-mon[35941]: pgmap v1263: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:06:17.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:06:17 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:06:18.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:06:18 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:06:18.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:06:18 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:06:18.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:06:18 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:06:18.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:06:18 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:06:18.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:06:18 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:06:18.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:06:18 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:06:18.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:06:18 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:06:18.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:06:18 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:06:18.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:06:18 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:06:19.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:06:19 smithi165 ceph-mon[36302]: pgmap v1264: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T01:06:19.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:06:19 smithi165 ceph-mon[36302]: pgmap v1265: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:06:19.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:06:19 smithi119 ceph-mon[31518]: pgmap v1264: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T01:06:19.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:06:19 smithi119 ceph-mon[31518]: pgmap v1265: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:06:19.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:06:19 smithi132 ceph-mon[35941]: pgmap v1264: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T01:06:19.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:06:19 smithi132 ceph-mon[35941]: pgmap v1265: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:06:21.347 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:06:21 smithi119 ceph-mon[31518]: pgmap v1266: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:06:21.347 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:06:21 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1840.687605 secs 2024-03-28T01:06:21.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:06:21 smithi165 ceph-mon[36302]: pgmap v1266: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:06:21.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:06:21 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1840.687605 secs 2024-03-28T01:06:21.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:06:21 smithi132 ceph-mon[35941]: pgmap v1266: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:06:21.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:06:21 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1840.687605 secs 2024-03-28T01:06:23.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:06:23 smithi165 ceph-mon[36302]: pgmap v1267: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:06:23.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:06:23 smithi119 ceph-mon[31518]: pgmap v1267: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:06:23.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:06:23 smithi132 ceph-mon[35941]: pgmap v1267: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:06:25.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:06:25 smithi165 ceph-mon[36302]: pgmap v1268: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:06:25.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:06:25 smithi119 ceph-mon[31518]: pgmap v1268: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:06:25.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:06:25 smithi132 ceph-mon[35941]: pgmap v1268: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:06:26.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:06:26 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1845.687691 secs 2024-03-28T01:06:26.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:06:26 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1845.687691 secs 2024-03-28T01:06:26.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:06:26 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1845.687691 secs 2024-03-28T01:06:27.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:06:27 smithi165 ceph-mon[36302]: pgmap v1269: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:06:27.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:06:27 smithi119 ceph-mon[31518]: pgmap v1269: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:06:27.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:06:27 smithi132 ceph-mon[35941]: pgmap v1269: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:06:29.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:06:29 smithi165 ceph-mon[36302]: pgmap v1270: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T01:06:29.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:06:29 smithi119 ceph-mon[31518]: pgmap v1270: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T01:06:29.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:06:29 smithi132 ceph-mon[35941]: pgmap v1270: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T01:06:31.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:06:31 smithi165 ceph-mon[36302]: pgmap v1271: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:06:31.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:06:31 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1850.687780 secs 2024-03-28T01:06:31.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:06:31 smithi119 ceph-mon[31518]: pgmap v1271: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:06:31.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:06:31 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1850.687780 secs 2024-03-28T01:06:31.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:06:31 smithi132 ceph-mon[35941]: pgmap v1271: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:06:31.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:06:31 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1850.687780 secs 2024-03-28T01:06:33.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:06:33 smithi165 ceph-mon[36302]: pgmap v1272: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:06:33.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:06:33 smithi119 ceph-mon[31518]: pgmap v1272: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:06:33.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:06:33 smithi132 ceph-mon[35941]: pgmap v1272: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:06:35.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:06:35 smithi165 ceph-mon[36302]: pgmap v1273: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:06:35.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:06:35 smithi119 ceph-mon[31518]: pgmap v1273: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:06:35.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:06:35 smithi132 ceph-mon[35941]: pgmap v1273: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:06:36.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:06:36 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1855.687842 secs 2024-03-28T01:06:36.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:06:36 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1855.687842 secs 2024-03-28T01:06:36.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:06:36 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1855.687842 secs 2024-03-28T01:06:37.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:06:37 smithi165 ceph-mon[36302]: pgmap v1274: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:06:37.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:06:37 smithi119 ceph-mon[31518]: pgmap v1274: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:06:37.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:06:37 smithi132 ceph-mon[35941]: pgmap v1274: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:06:39.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:06:39 smithi165 ceph-mon[36302]: pgmap v1275: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:06:39.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:06:39 smithi119 ceph-mon[31518]: pgmap v1275: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:06:39.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:06:39 smithi132 ceph-mon[35941]: pgmap v1275: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:06:41.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:06:41 smithi165 ceph-mon[36302]: pgmap v1276: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:06:41.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:06:41 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1860.687880 secs 2024-03-28T01:06:41.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:06:41 smithi119 ceph-mon[31518]: pgmap v1276: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:06:41.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:06:41 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1860.687880 secs 2024-03-28T01:06:41.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:06:41 smithi132 ceph-mon[35941]: pgmap v1276: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:06:41.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:06:41 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1860.687880 secs 2024-03-28T01:06:43.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:06:43 smithi165 ceph-mon[36302]: pgmap v1277: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:06:43.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:06:43 smithi119 ceph-mon[31518]: pgmap v1277: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:06:43.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:06:43 smithi132 ceph-mon[35941]: pgmap v1277: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:06:45.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:06:45 smithi165 ceph-mon[36302]: pgmap v1278: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:06:45.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:06:45 smithi119 ceph-mon[31518]: pgmap v1278: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:06:45.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:06:45 smithi132 ceph-mon[35941]: pgmap v1278: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:06:46.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:06:46 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1865.687944 secs 2024-03-28T01:06:46.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:06:46 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1865.687944 secs 2024-03-28T01:06:46.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:06:46 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1865.687944 secs 2024-03-28T01:06:47.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:06:47 smithi119 ceph-mon[31518]: pgmap v1279: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:06:47.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:06:47 smithi132 ceph-mon[35941]: pgmap v1279: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:06:47.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:06:47 smithi165 ceph-mon[36302]: pgmap v1279: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:06:49.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:06:49 smithi119 ceph-mon[31518]: pgmap v1280: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:06:49.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:06:49 smithi132 ceph-mon[35941]: pgmap v1280: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:06:49.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:06:49 smithi165 ceph-mon[36302]: pgmap v1280: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:06:51.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:06:51 smithi119 ceph-mon[31518]: pgmap v1281: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:06:51.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:06:51 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1870.687980 secs 2024-03-28T01:06:51.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:06:51 smithi132 ceph-mon[35941]: pgmap v1281: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:06:51.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:06:51 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1870.687980 secs 2024-03-28T01:06:51.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:06:51 smithi165 ceph-mon[36302]: pgmap v1281: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:06:51.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:06:51 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1870.687980 secs 2024-03-28T01:06:53.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:06:53 smithi119 ceph-mon[31518]: pgmap v1282: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:06:53.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:06:53 smithi132 ceph-mon[35941]: pgmap v1282: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:06:53.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:06:53 smithi165 ceph-mon[36302]: pgmap v1282: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:06:55.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:06:55 smithi132 ceph-mon[35941]: pgmap v1283: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:06:55.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:06:55 smithi165 ceph-mon[36302]: pgmap v1283: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:06:55.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:06:55 smithi119 ceph-mon[31518]: pgmap v1283: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:06:56.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:06:56 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1875.688069 secs 2024-03-28T01:06:56.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:06:56 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1875.688069 secs 2024-03-28T01:06:56.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:06:56 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1875.688069 secs 2024-03-28T01:06:57.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:06:57 smithi132 ceph-mon[35941]: pgmap v1284: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:06:57.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:06:57 smithi165 ceph-mon[36302]: pgmap v1284: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:06:57.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:06:57 smithi119 ceph-mon[31518]: pgmap v1284: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:06:58.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:06:58 smithi132 ceph-mon[35941]: pgmap v1285: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:06:58.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:06:58 smithi165 ceph-mon[36302]: pgmap v1285: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:06:58.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:06:58 smithi119 ceph-mon[31518]: pgmap v1285: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:07:00.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:07:00 smithi165 ceph-mon[36302]: pgmap v1286: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:07:00.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:07:00 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1880.688177 secs 2024-03-28T01:07:00.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:07:00 smithi119 ceph-mon[31518]: pgmap v1286: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:07:00.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:07:00 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1880.688177 secs 2024-03-28T01:07:00.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:07:00 smithi132 ceph-mon[35941]: pgmap v1286: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:07:00.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:07:00 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1880.688177 secs 2024-03-28T01:07:03.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:07:02 smithi165 ceph-mon[36302]: pgmap v1287: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:07:03.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:07:02 smithi119 ceph-mon[31518]: pgmap v1287: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:07:03.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:07:02 smithi132 ceph-mon[35941]: pgmap v1287: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:07:05.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:07:04 smithi165 ceph-mon[36302]: pgmap v1288: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:07:05.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:07:04 smithi119 ceph-mon[31518]: pgmap v1288: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:07:05.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:07:04 smithi132 ceph-mon[35941]: pgmap v1288: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:07:06.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:07:05 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1885.688240 secs 2024-03-28T01:07:06.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:07:05 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1885.688240 secs 2024-03-28T01:07:06.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:07:05 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1885.688240 secs 2024-03-28T01:07:07.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:07:06 smithi165 ceph-mon[36302]: pgmap v1289: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:07:07.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:07:06 smithi119 ceph-mon[31518]: pgmap v1289: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:07:07.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:07:06 smithi132 ceph-mon[35941]: pgmap v1289: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:07:09.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:07:09 smithi165 ceph-mon[36302]: pgmap v1290: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:07:09.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:07:09 smithi119 ceph-mon[31518]: pgmap v1290: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:07:09.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:07:09 smithi132 ceph-mon[35941]: pgmap v1290: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:07:11.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:07:11 smithi165 ceph-mon[36302]: pgmap v1291: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:07:11.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:07:11 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1890.688303 secs 2024-03-28T01:07:11.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:07:11 smithi119 ceph-mon[31518]: pgmap v1291: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:07:11.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:07:11 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1890.688303 secs 2024-03-28T01:07:11.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:07:11 smithi132 ceph-mon[35941]: pgmap v1291: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:07:11.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:07:11 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1890.688303 secs 2024-03-28T01:07:13.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:07:13 smithi165 ceph-mon[36302]: pgmap v1292: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:07:13.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:07:13 smithi119 ceph-mon[31518]: pgmap v1292: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:07:13.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:07:13 smithi132 ceph-mon[35941]: pgmap v1292: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:07:15.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:07:15 smithi165 ceph-mon[36302]: pgmap v1293: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:07:15.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:07:15 smithi119 ceph-mon[31518]: pgmap v1293: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:07:15.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:07:15 smithi132 ceph-mon[35941]: pgmap v1293: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:07:16.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:07:16 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1895.688364 secs 2024-03-28T01:07:16.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:07:16 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1895.688364 secs 2024-03-28T01:07:16.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:07:16 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1895.688364 secs 2024-03-28T01:07:17.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:07:17 smithi165 ceph-mon[36302]: pgmap v1294: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:07:17.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:07:17 smithi119 ceph-mon[31518]: pgmap v1294: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:07:17.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:07:17 smithi132 ceph-mon[35941]: pgmap v1294: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:07:18.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:07:18 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:07:18.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:07:18 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:07:18.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:07:18 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:07:19.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:07:19 smithi165 ceph-mon[36302]: pgmap v1295: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:07:19.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:07:19 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:07:19.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:07:19 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:07:19.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:07:19 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:07:19.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:07:19 smithi119 ceph-mon[31518]: pgmap v1295: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:07:19.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:07:19 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:07:19.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:07:19 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:07:19.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:07:19 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:07:19.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:07:19 smithi132 ceph-mon[35941]: pgmap v1295: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:07:19.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:07:19 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:07:19.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:07:19 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:07:19.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:07:19 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:07:20.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:07:20 smithi165 ceph-mon[36302]: pgmap v1296: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T01:07:20.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:07:20 smithi165 ceph-mon[36302]: pgmap v1297: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-28T01:07:20.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:07:20 smithi119 ceph-mon[31518]: pgmap v1296: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T01:07:20.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:07:20 smithi119 ceph-mon[31518]: pgmap v1297: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-28T01:07:20.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:07:20 smithi132 ceph-mon[35941]: pgmap v1296: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T01:07:20.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:07:20 smithi132 ceph-mon[35941]: pgmap v1297: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-28T01:07:21.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:07:21 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1900.688379 secs 2024-03-28T01:07:21.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:07:21 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1900.688379 secs 2024-03-28T01:07:21.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:07:21 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1900.688379 secs 2024-03-28T01:07:22.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:07:22 smithi165 ceph-mon[36302]: pgmap v1298: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:07:22.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:07:22 smithi119 ceph-mon[31518]: pgmap v1298: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:07:22.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:07:22 smithi132 ceph-mon[35941]: pgmap v1298: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:07:24.322 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:07:24 smithi119 ceph-mon[31518]: pgmap v1299: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:07:24.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:07:24 smithi165 ceph-mon[36302]: pgmap v1299: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:07:24.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:07:24 smithi132 ceph-mon[35941]: pgmap v1299: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:07:26.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:07:26 smithi165 ceph-mon[36302]: pgmap v1300: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:07:26.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:07:26 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1905.688464 secs 2024-03-28T01:07:26.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:07:26 smithi119 ceph-mon[31518]: pgmap v1300: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:07:26.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:07:26 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1905.688464 secs 2024-03-28T01:07:26.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:07:26 smithi132 ceph-mon[35941]: pgmap v1300: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:07:26.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:07:26 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1905.688464 secs 2024-03-28T01:07:28.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:07:28 smithi165 ceph-mon[36302]: pgmap v1301: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:07:28.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:07:28 smithi119 ceph-mon[31518]: pgmap v1301: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:07:28.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:07:28 smithi132 ceph-mon[35941]: pgmap v1301: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:07:30.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:07:30 smithi165 ceph-mon[36302]: pgmap v1302: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T01:07:30.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:07:30 smithi119 ceph-mon[31518]: pgmap v1302: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T01:07:30.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:07:30 smithi132 ceph-mon[35941]: pgmap v1302: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T01:07:31.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:07:31 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1910.688554 secs 2024-03-28T01:07:31.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:07:31 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1910.688554 secs 2024-03-28T01:07:31.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:07:31 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1910.688554 secs 2024-03-28T01:07:32.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:07:32 smithi165 ceph-mon[36302]: pgmap v1303: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:07:32.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:07:32 smithi119 ceph-mon[31518]: pgmap v1303: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:07:32.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:07:32 smithi132 ceph-mon[35941]: pgmap v1303: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:07:34.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:07:34 smithi165 ceph-mon[36302]: pgmap v1304: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:07:34.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:07:34 smithi119 ceph-mon[31518]: pgmap v1304: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:07:34.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:07:34 smithi132 ceph-mon[35941]: pgmap v1304: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:07:36.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:07:36 smithi165 ceph-mon[36302]: pgmap v1305: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:07:36.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:07:36 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1915.688616 secs 2024-03-28T01:07:36.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:07:36 smithi119 ceph-mon[31518]: pgmap v1305: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:07:36.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:07:36 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1915.688616 secs 2024-03-28T01:07:36.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:07:36 smithi132 ceph-mon[35941]: pgmap v1305: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:07:36.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:07:36 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1915.688616 secs 2024-03-28T01:07:38.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:07:38 smithi165 ceph-mon[36302]: pgmap v1306: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:07:38.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:07:38 smithi119 ceph-mon[31518]: pgmap v1306: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:07:38.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:07:38 smithi132 ceph-mon[35941]: pgmap v1306: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:07:40.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:07:40 smithi165 ceph-mon[36302]: pgmap v1307: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:07:40.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:07:40 smithi119 ceph-mon[31518]: pgmap v1307: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:07:40.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:07:40 smithi132 ceph-mon[35941]: pgmap v1307: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:07:41.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:07:41 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1920.688700 secs 2024-03-28T01:07:41.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:07:41 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1920.688700 secs 2024-03-28T01:07:41.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:07:41 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1920.688700 secs 2024-03-28T01:07:42.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:07:42 smithi165 ceph-mon[36302]: pgmap v1308: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:07:42.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:07:42 smithi119 ceph-mon[31518]: pgmap v1308: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:07:42.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:07:42 smithi132 ceph-mon[35941]: pgmap v1308: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:07:44.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:07:44 smithi165 ceph-mon[36302]: pgmap v1309: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:07:44.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:07:44 smithi119 ceph-mon[31518]: pgmap v1309: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:07:44.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:07:44 smithi132 ceph-mon[35941]: pgmap v1309: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:07:46.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:07:46 smithi165 ceph-mon[36302]: pgmap v1310: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:07:46.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:07:46 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1925.688790 secs 2024-03-28T01:07:46.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:07:46 smithi119 ceph-mon[31518]: pgmap v1310: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:07:46.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:07:46 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1925.688790 secs 2024-03-28T01:07:46.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:07:46 smithi132 ceph-mon[35941]: pgmap v1310: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:07:46.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:07:46 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1925.688790 secs 2024-03-28T01:07:48.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:07:48 smithi119 ceph-mon[31518]: pgmap v1311: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:07:48.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:07:48 smithi132 ceph-mon[35941]: pgmap v1311: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:07:48.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:07:48 smithi165 ceph-mon[36302]: pgmap v1311: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:07:50.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:07:50 smithi119 ceph-mon[31518]: pgmap v1312: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:07:50.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:07:50 smithi132 ceph-mon[35941]: pgmap v1312: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:07:50.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:07:50 smithi165 ceph-mon[36302]: pgmap v1312: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:07:51.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:07:51 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1930.688880 secs 2024-03-28T01:07:51.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:07:51 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1930.688880 secs 2024-03-28T01:07:51.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:07:51 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1930.688880 secs 2024-03-28T01:07:52.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:07:52 smithi132 ceph-mon[35941]: pgmap v1313: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:07:52.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:07:52 smithi165 ceph-mon[36302]: pgmap v1313: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:07:52.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:07:52 smithi119 ceph-mon[31518]: pgmap v1313: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:07:54.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:07:54 smithi132 ceph-mon[35941]: pgmap v1314: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:07:54.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:07:54 smithi165 ceph-mon[36302]: pgmap v1314: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:07:54.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:07:54 smithi119 ceph-mon[31518]: pgmap v1314: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:07:56.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:07:56 smithi119 ceph-mon[31518]: pgmap v1315: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:07:56.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:07:56 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1935.688902 secs 2024-03-28T01:07:56.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:07:56 smithi132 ceph-mon[35941]: pgmap v1315: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:07:56.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:07:56 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1935.688902 secs 2024-03-28T01:07:56.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:07:56 smithi165 ceph-mon[36302]: pgmap v1315: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:07:56.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:07:56 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1935.688902 secs 2024-03-28T01:07:58.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:07:58 smithi132 ceph-mon[35941]: pgmap v1316: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:07:58.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:07:58 smithi165 ceph-mon[36302]: pgmap v1316: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:07:58.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:07:58 smithi119 ceph-mon[31518]: pgmap v1316: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:08:00.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:08:00 smithi132 ceph-mon[35941]: pgmap v1317: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:08:00.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:08:00 smithi165 ceph-mon[36302]: pgmap v1317: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:08:00.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:08:00 smithi119 ceph-mon[31518]: pgmap v1317: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:08:01.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:08:01 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1940.688965 secs 2024-03-28T01:08:01.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:08:01 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1940.688965 secs 2024-03-28T01:08:01.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:08:01 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1940.688965 secs 2024-03-28T01:08:02.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:08:02 smithi132 ceph-mon[35941]: pgmap v1318: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:08:02.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:08:02 smithi165 ceph-mon[36302]: pgmap v1318: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:08:02.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:08:02 smithi119 ceph-mon[31518]: pgmap v1318: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:08:03.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:08:03 smithi132 ceph-mon[35941]: pgmap v1319: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:08:03.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:08:03 smithi165 ceph-mon[36302]: pgmap v1319: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:08:03.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:08:03 smithi119 ceph-mon[31518]: pgmap v1319: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:08:06.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:08:05 smithi165 ceph-mon[36302]: pgmap v1320: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:08:06.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:08:05 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1945.689051 secs 2024-03-28T01:08:06.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:08:05 smithi119 ceph-mon[31518]: pgmap v1320: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:08:06.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:08:05 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1945.689051 secs 2024-03-28T01:08:06.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:08:05 smithi132 ceph-mon[35941]: pgmap v1320: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:08:06.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:08:05 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1945.689051 secs 2024-03-28T01:08:08.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:08:07 smithi165 ceph-mon[36302]: pgmap v1321: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:08:08.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:08:07 smithi119 ceph-mon[31518]: pgmap v1321: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:08:08.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:08:07 smithi132 ceph-mon[35941]: pgmap v1321: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:08:10.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:08:09 smithi165 ceph-mon[36302]: pgmap v1322: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:08:10.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:08:09 smithi119 ceph-mon[31518]: pgmap v1322: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:08:10.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:08:09 smithi132 ceph-mon[35941]: pgmap v1322: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:08:11.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:08:10 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1950.689120 secs 2024-03-28T01:08:11.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:08:10 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1950.689120 secs 2024-03-28T01:08:11.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:08:10 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1950.689120 secs 2024-03-28T01:08:12.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:08:11 smithi165 ceph-mon[36302]: pgmap v1323: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:08:12.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:08:11 smithi119 ceph-mon[31518]: pgmap v1323: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:08:12.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:08:11 smithi132 ceph-mon[35941]: pgmap v1323: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:08:14.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:08:13 smithi119 ceph-mon[31518]: pgmap v1324: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:08:14.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:08:13 smithi132 ceph-mon[35941]: pgmap v1324: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:08:14.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:08:13 smithi165 ceph-mon[36302]: pgmap v1324: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:08:16.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:08:15 smithi119 ceph-mon[31518]: pgmap v1325: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:08:16.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:08:15 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1955.689182 secs 2024-03-28T01:08:16.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:08:15 smithi132 ceph-mon[35941]: pgmap v1325: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:08:16.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:08:15 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1955.689182 secs 2024-03-28T01:08:16.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:08:15 smithi165 ceph-mon[36302]: pgmap v1325: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:08:16.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:08:15 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1955.689182 secs 2024-03-28T01:08:18.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:08:17 smithi119 ceph-mon[31518]: pgmap v1326: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:08:18.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:08:17 smithi132 ceph-mon[35941]: pgmap v1326: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:08:18.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:08:17 smithi165 ceph-mon[36302]: pgmap v1326: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:08:20.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:08:19 smithi119 ceph-mon[31518]: pgmap v1327: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:08:20.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:08:19 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:08:20.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:08:19 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:08:20.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:08:19 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:08:20.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:08:19 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:08:20.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:08:19 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:08:20.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:08:19 smithi132 ceph-mon[35941]: pgmap v1327: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:08:20.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:08:19 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:08:20.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:08:19 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:08:20.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:08:19 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:08:20.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:08:19 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:08:20.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:08:19 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:08:20.248 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:08:19 smithi165 ceph-mon[36302]: pgmap v1327: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:08:20.248 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:08:19 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:08:20.248 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:08:19 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:08:20.249 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:08:19 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:08:20.249 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:08:19 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:08:20.249 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:08:19 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:08:21.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:08:20 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1960.689270 secs 2024-03-28T01:08:21.251 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:08:20 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1960.689270 secs 2024-03-28T01:08:21.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:08:20 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1960.689270 secs 2024-03-28T01:08:22.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:08:21 smithi132 ceph-mon[35941]: pgmap v1328: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:08:22.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:08:21 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:08:22.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:08:21 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:08:22.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:08:21 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:08:22.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:08:21 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:08:22.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:08:21 smithi165 ceph-mon[36302]: pgmap v1328: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:08:22.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:08:21 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:08:22.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:08:21 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:08:22.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:08:21 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:08:22.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:08:21 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:08:22.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:08:21 smithi119 ceph-mon[31518]: pgmap v1328: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:08:22.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:08:21 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:08:22.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:08:21 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:08:22.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:08:21 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:08:22.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:08:21 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:08:23.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:08:22 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:08:23.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:08:22 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:08:23.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:08:22 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:08:23.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:08:22 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:08:23.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:08:22 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:08:23.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:08:22 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:08:23.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:08:22 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:08:23.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:08:22 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:08:23.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:08:22 smithi132 ceph-mon[35941]: Adjusting osd_memory_target on smithi132 to 174.1M 2024-03-28T01:08:23.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:08:22 smithi132 ceph-mon[35941]: Unable to set osd_memory_target on smithi132 to 182640640: error parsing value: Value '182640640' is below minimum 939524096 2024-03-28T01:08:23.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:08:22 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:08:23.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:08:22 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:08:23.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:08:22 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:08:23.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:08:22 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:08:23.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:08:22 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:08:23.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:08:22 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:08:23.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:08:22 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:08:23.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:08:22 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:08:23.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:08:22 smithi132 ceph-mon[35941]: Adjusting osd_memory_target on smithi165 to 174.1M 2024-03-28T01:08:23.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:08:22 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:08:23.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:08:22 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:08:23.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:08:22 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:08:23.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:08:22 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:08:23.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:08:22 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:08:23.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:08:22 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:08:23.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:08:22 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:08:23.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:08:22 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:08:23.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:08:22 smithi165 ceph-mon[36302]: Adjusting osd_memory_target on smithi132 to 174.1M 2024-03-28T01:08:23.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:08:22 smithi165 ceph-mon[36302]: Unable to set osd_memory_target on smithi132 to 182640640: error parsing value: Value '182640640' is below minimum 939524096 2024-03-28T01:08:23.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:08:22 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:08:23.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:08:22 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:08:23.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:08:22 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:08:23.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:08:22 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:08:23.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:08:22 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:08:23.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:08:22 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:08:23.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:08:22 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:08:23.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:08:22 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:08:23.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:08:22 smithi165 ceph-mon[36302]: Adjusting osd_memory_target on smithi165 to 174.1M 2024-03-28T01:08:23.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:08:22 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:08:23.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:08:22 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:08:23.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:08:22 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:08:23.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:08:22 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:08:23.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:08:22 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:08:23.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:08:22 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:08:23.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:08:22 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:08:23.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:08:22 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:08:23.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:08:22 smithi119 ceph-mon[31518]: Adjusting osd_memory_target on smithi132 to 174.1M 2024-03-28T01:08:23.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:08:22 smithi119 ceph-mon[31518]: Unable to set osd_memory_target on smithi132 to 182640640: error parsing value: Value '182640640' is below minimum 939524096 2024-03-28T01:08:23.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:08:22 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:08:23.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:08:22 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:08:23.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:08:22 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:08:23.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:08:22 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:08:23.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:08:22 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:08:23.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:08:22 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:08:23.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:08:22 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:08:23.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:08:22 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:08:23.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:08:22 smithi119 ceph-mon[31518]: Adjusting osd_memory_target on smithi165 to 174.1M 2024-03-28T01:08:24.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:08:23 smithi132 ceph-mon[35941]: Unable to set osd_memory_target on smithi165 to 182635622: error parsing value: Value '182635622' is below minimum 939524096 2024-03-28T01:08:24.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:08:23 smithi132 ceph-mon[35941]: pgmap v1329: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:08:24.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:08:23 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:08:24.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:08:23 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:08:24.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:08:23 smithi165 ceph-mon[36302]: Unable to set osd_memory_target on smithi165 to 182635622: error parsing value: Value '182635622' is below minimum 939524096 2024-03-28T01:08:24.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:08:23 smithi165 ceph-mon[36302]: pgmap v1329: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:08:24.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:08:23 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:08:24.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:08:23 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:08:24.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:08:23 smithi119 ceph-mon[31518]: Unable to set osd_memory_target on smithi165 to 182635622: error parsing value: Value '182635622' is below minimum 939524096 2024-03-28T01:08:24.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:08:23 smithi119 ceph-mon[31518]: pgmap v1329: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:08:24.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:08:23 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:08:24.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:08:23 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:08:25.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:08:24 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:08:25.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:08:24 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:08:25.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:08:24 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:08:25.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:08:24 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:08:25.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:08:24 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:08:25.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:08:24 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:08:25.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:08:24 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:08:25.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:08:24 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:08:25.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:08:24 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:08:25.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:08:24 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:08:25.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:08:24 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:08:25.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:08:24 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:08:25.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:08:24 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:08:25.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:08:24 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:08:25.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:08:24 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:08:25.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:08:24 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:08:25.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:08:24 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:08:25.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:08:24 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:08:25.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:08:24 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:08:25.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:08:24 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:08:25.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:08:24 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:08:25.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:08:24 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:08:25.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:08:24 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:08:25.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:08:24 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:08:25.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:08:24 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:08:25.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:08:24 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:08:25.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:08:24 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:08:25.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:08:24 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:08:25.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:08:24 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:08:25.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:08:24 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:08:25.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:08:24 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:08:25.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:08:24 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:08:25.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:08:24 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:08:26.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:08:25 smithi132 ceph-mon[35941]: Adjusting osd_memory_target on smithi119 to 176.7M 2024-03-28T01:08:26.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:08:25 smithi132 ceph-mon[35941]: Unable to set osd_memory_target on smithi119 to 185372364: error parsing value: Value '185372364' is below minimum 939524096 2024-03-28T01:08:26.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:08:25 smithi132 ceph-mon[35941]: pgmap v1330: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 7 op/s 2024-03-28T01:08:26.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:08:25 smithi132 ceph-mon[35941]: pgmap v1331: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T01:08:26.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:08:25 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1965.689361 secs 2024-03-28T01:08:26.275 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:08:25 smithi119 ceph-mon[31518]: Adjusting osd_memory_target on smithi119 to 176.7M 2024-03-28T01:08:26.275 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:08:25 smithi119 ceph-mon[31518]: Unable to set osd_memory_target on smithi119 to 185372364: error parsing value: Value '185372364' is below minimum 939524096 2024-03-28T01:08:26.275 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:08:25 smithi119 ceph-mon[31518]: pgmap v1330: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 7 op/s 2024-03-28T01:08:26.275 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:08:25 smithi119 ceph-mon[31518]: pgmap v1331: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T01:08:26.275 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:08:25 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1965.689361 secs 2024-03-28T01:08:26.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:08:25 smithi165 ceph-mon[36302]: Adjusting osd_memory_target on smithi119 to 176.7M 2024-03-28T01:08:26.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:08:25 smithi165 ceph-mon[36302]: Unable to set osd_memory_target on smithi119 to 185372364: error parsing value: Value '185372364' is below minimum 939524096 2024-03-28T01:08:26.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:08:25 smithi165 ceph-mon[36302]: pgmap v1330: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 7 op/s 2024-03-28T01:08:26.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:08:25 smithi165 ceph-mon[36302]: pgmap v1331: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T01:08:26.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:08:25 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1965.689361 secs 2024-03-28T01:08:28.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:08:27 smithi132 ceph-mon[35941]: pgmap v1332: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T01:08:28.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:08:27 smithi165 ceph-mon[36302]: pgmap v1332: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T01:08:28.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:08:27 smithi119 ceph-mon[31518]: pgmap v1332: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T01:08:30.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:08:29 smithi132 ceph-mon[35941]: pgmap v1333: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T01:08:30.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:08:29 smithi165 ceph-mon[36302]: pgmap v1333: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T01:08:30.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:08:29 smithi119 ceph-mon[31518]: pgmap v1333: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T01:08:31.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:08:30 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1970.689454 secs 2024-03-28T01:08:31.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:08:30 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1970.689454 secs 2024-03-28T01:08:31.404 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:08:30 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1970.689454 secs 2024-03-28T01:08:32.355 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:08:31 smithi132 ceph-mon[35941]: pgmap v1334: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T01:08:32.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:08:31 smithi165 ceph-mon[36302]: pgmap v1334: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T01:08:32.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:08:31 smithi119 ceph-mon[31518]: pgmap v1334: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T01:08:34.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:08:33 smithi165 ceph-mon[36302]: pgmap v1335: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T01:08:34.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:08:33 smithi119 ceph-mon[31518]: pgmap v1335: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T01:08:34.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:08:33 smithi132 ceph-mon[35941]: pgmap v1335: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T01:08:36.235 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:08:35 smithi119 ceph-mon[31518]: pgmap v1336: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T01:08:36.235 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:08:35 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1975.689495 secs 2024-03-28T01:08:36.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:08:35 smithi165 ceph-mon[36302]: pgmap v1336: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T01:08:36.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:08:35 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1975.689495 secs 2024-03-28T01:08:36.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:08:35 smithi132 ceph-mon[35941]: pgmap v1336: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T01:08:36.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:08:35 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1975.689495 secs 2024-03-28T01:08:38.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:08:37 smithi119 ceph-mon[31518]: pgmap v1337: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:08:38.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:08:37 smithi165 ceph-mon[36302]: pgmap v1337: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:08:38.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:08:37 smithi132 ceph-mon[35941]: pgmap v1337: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:08:40.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:08:39 smithi165 ceph-mon[36302]: pgmap v1338: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:08:40.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:08:39 smithi119 ceph-mon[31518]: pgmap v1338: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:08:40.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:08:39 smithi132 ceph-mon[35941]: pgmap v1338: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:08:41.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:08:40 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1980.689580 secs 2024-03-28T01:08:41.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:08:40 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1980.689580 secs 2024-03-28T01:08:41.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:08:40 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1980.689580 secs 2024-03-28T01:08:42.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:08:41 smithi132 ceph-mon[35941]: pgmap v1339: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:08:42.304 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:08:41 smithi119 ceph-mon[31518]: pgmap v1339: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:08:42.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:08:41 smithi165 ceph-mon[36302]: pgmap v1339: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:08:44.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:08:43 smithi165 ceph-mon[36302]: pgmap v1340: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:08:44.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:08:43 smithi119 ceph-mon[31518]: pgmap v1340: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:08:44.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:08:43 smithi132 ceph-mon[35941]: pgmap v1340: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:08:46.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:08:45 smithi165 ceph-mon[36302]: pgmap v1341: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:08:46.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:08:45 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1985.689672 secs 2024-03-28T01:08:46.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:08:45 smithi119 ceph-mon[31518]: pgmap v1341: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:08:46.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:08:45 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1985.689672 secs 2024-03-28T01:08:46.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:08:45 smithi132 ceph-mon[35941]: pgmap v1341: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:08:46.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:08:45 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1985.689672 secs 2024-03-28T01:08:48.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:08:47 smithi119 ceph-mon[31518]: pgmap v1342: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:08:48.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:08:47 smithi165 ceph-mon[36302]: pgmap v1342: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:08:48.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:08:47 smithi132 ceph-mon[35941]: pgmap v1342: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:08:50.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:08:50 smithi165 ceph-mon[36302]: pgmap v1343: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:08:50.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:08:50 smithi119 ceph-mon[31518]: pgmap v1343: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:08:50.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:08:50 smithi132 ceph-mon[35941]: pgmap v1343: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:08:51.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:08:51 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1990.689758 secs 2024-03-28T01:08:51.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:08:51 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1990.689758 secs 2024-03-28T01:08:51.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:08:51 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1990.689758 secs 2024-03-28T01:08:52.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:08:52 smithi165 ceph-mon[36302]: pgmap v1344: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:08:52.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:08:52 smithi119 ceph-mon[31518]: pgmap v1344: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:08:52.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:08:52 smithi132 ceph-mon[35941]: pgmap v1344: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:08:54.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:08:54 smithi165 ceph-mon[36302]: pgmap v1345: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:08:54.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:08:54 smithi119 ceph-mon[31518]: pgmap v1345: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:08:54.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:08:54 smithi132 ceph-mon[35941]: pgmap v1345: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:08:56.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:08:56 smithi165 ceph-mon[36302]: pgmap v1346: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:08:56.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:08:56 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 1995.689851 secs 2024-03-28T01:08:56.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:08:56 smithi119 ceph-mon[31518]: pgmap v1346: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:08:56.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:08:56 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 1995.689851 secs 2024-03-28T01:08:56.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:08:56 smithi132 ceph-mon[35941]: pgmap v1346: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:08:56.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:08:56 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 1995.689851 secs 2024-03-28T01:08:58.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:08:58 smithi165 ceph-mon[36302]: pgmap v1347: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:08:58.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:08:58 smithi119 ceph-mon[31518]: pgmap v1347: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:08:58.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:08:58 smithi132 ceph-mon[35941]: pgmap v1347: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:09:00.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:09:00 smithi165 ceph-mon[36302]: pgmap v1348: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:09:00.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:09:00 smithi119 ceph-mon[31518]: pgmap v1348: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:09:00.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:09:00 smithi132 ceph-mon[35941]: pgmap v1348: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:09:01.350 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:09:01 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2000.689892 secs 2024-03-28T01:09:01.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:09:01 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2000.689892 secs 2024-03-28T01:09:01.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:09:01 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2000.689892 secs 2024-03-28T01:09:02.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:09:02 smithi165 ceph-mon[36302]: pgmap v1349: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:09:02.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:09:02 smithi119 ceph-mon[31518]: pgmap v1349: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:09:02.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:09:02 smithi132 ceph-mon[35941]: pgmap v1349: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:09:04.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:09:04 smithi165 ceph-mon[36302]: pgmap v1350: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:09:04.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:09:04 smithi119 ceph-mon[31518]: pgmap v1350: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:09:04.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:09:04 smithi132 ceph-mon[35941]: pgmap v1350: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:09:06.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:09:06 smithi165 ceph-mon[36302]: pgmap v1351: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:09:06.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:09:06 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2005.689980 secs 2024-03-28T01:09:06.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:09:06 smithi119 ceph-mon[31518]: pgmap v1351: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:09:06.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:09:06 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2005.689980 secs 2024-03-28T01:09:06.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:09:06 smithi132 ceph-mon[35941]: pgmap v1351: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:09:06.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:09:06 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2005.689980 secs 2024-03-28T01:09:08.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:09:08 smithi165 ceph-mon[36302]: pgmap v1352: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:09:08.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:09:08 smithi119 ceph-mon[31518]: pgmap v1352: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:09:08.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:09:08 smithi132 ceph-mon[35941]: pgmap v1352: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:09:10.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:09:10 smithi165 ceph-mon[36302]: pgmap v1353: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:09:10.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:09:10 smithi119 ceph-mon[31518]: pgmap v1353: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:09:10.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:09:10 smithi132 ceph-mon[35941]: pgmap v1353: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:09:11.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:09:11 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2010.690045 secs 2024-03-28T01:09:11.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:09:11 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2010.690045 secs 2024-03-28T01:09:11.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:09:11 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2010.690045 secs 2024-03-28T01:09:12.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:09:12 smithi165 ceph-mon[36302]: pgmap v1354: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:09:12.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:09:12 smithi119 ceph-mon[31518]: pgmap v1354: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:09:12.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:09:12 smithi132 ceph-mon[35941]: pgmap v1354: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:09:14.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:09:14 smithi165 ceph-mon[36302]: pgmap v1355: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:09:14.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:09:14 smithi119 ceph-mon[31518]: pgmap v1355: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:09:14.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:09:14 smithi132 ceph-mon[35941]: pgmap v1355: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:09:16.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:09:16 smithi165 ceph-mon[36302]: pgmap v1356: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:09:16.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:09:16 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2015.690108 secs 2024-03-28T01:09:16.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:09:16 smithi119 ceph-mon[31518]: pgmap v1356: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:09:16.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:09:16 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2015.690108 secs 2024-03-28T01:09:16.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:09:16 smithi132 ceph-mon[35941]: pgmap v1356: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:09:16.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:09:16 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2015.690108 secs 2024-03-28T01:09:18.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:09:18 smithi165 ceph-mon[36302]: pgmap v1357: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:09:18.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:09:18 smithi119 ceph-mon[31518]: pgmap v1357: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:09:18.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:09:18 smithi132 ceph-mon[35941]: pgmap v1357: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:09:20.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:09:20 smithi165 ceph-mon[36302]: pgmap v1358: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:09:20.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:09:20 smithi119 ceph-mon[31518]: pgmap v1358: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:09:20.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:09:20 smithi132 ceph-mon[35941]: pgmap v1358: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:09:21.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:09:21 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2020.690169 secs 2024-03-28T01:09:21.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:09:21 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2020.690169 secs 2024-03-28T01:09:21.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:09:21 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2020.690169 secs 2024-03-28T01:09:22.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:09:22 smithi165 ceph-mon[36302]: pgmap v1359: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:09:22.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:09:22 smithi119 ceph-mon[31518]: pgmap v1359: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:09:22.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:09:22 smithi132 ceph-mon[35941]: pgmap v1359: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:09:24.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:09:24 smithi165 ceph-mon[36302]: pgmap v1360: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:09:24.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:09:24 smithi119 ceph-mon[31518]: pgmap v1360: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:09:24.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:09:24 smithi132 ceph-mon[35941]: pgmap v1360: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:09:25.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:09:25 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:09:25.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:09:25 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:09:25.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:09:25 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:09:26.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:09:26 smithi119 ceph-mon[31518]: pgmap v1361: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:09:26.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:09:26 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2025.690258 secs 2024-03-28T01:09:26.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:09:26 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:09:26.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:09:26 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:09:26.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:09:26 smithi132 ceph-mon[35941]: pgmap v1361: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:09:26.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:09:26 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2025.690258 secs 2024-03-28T01:09:26.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:09:26 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:09:26.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:09:26 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:09:26.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:09:26 smithi165 ceph-mon[36302]: pgmap v1361: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:09:26.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:09:26 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2025.690258 secs 2024-03-28T01:09:26.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:09:26 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:09:26.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:09:26 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:09:27.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:09:27 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:09:27.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:09:27 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:09:27.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:09:27 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:09:27.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:09:27 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:09:27.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:09:27 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:09:27.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:09:27 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:09:27.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:09:27 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:09:27.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:09:27 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:09:27.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:09:27 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:09:28.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:09:28 smithi119 ceph-mon[31518]: pgmap v1362: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T01:09:28.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:09:28 smithi119 ceph-mon[31518]: pgmap v1363: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:09:28.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:09:28 smithi132 ceph-mon[35941]: pgmap v1362: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T01:09:28.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:09:28 smithi132 ceph-mon[35941]: pgmap v1363: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:09:28.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:09:28 smithi165 ceph-mon[36302]: pgmap v1362: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T01:09:28.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:09:28 smithi165 ceph-mon[36302]: pgmap v1363: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:09:30.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:09:30 smithi119 ceph-mon[31518]: pgmap v1364: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:09:30.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:09:30 smithi132 ceph-mon[35941]: pgmap v1364: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:09:30.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:09:30 smithi165 ceph-mon[36302]: pgmap v1364: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:09:31.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:09:31 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2030.690348 secs 2024-03-28T01:09:31.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:09:31 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2030.690348 secs 2024-03-28T01:09:31.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:09:31 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2030.690348 secs 2024-03-28T01:09:32.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:09:32 smithi132 ceph-mon[35941]: pgmap v1365: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:09:32.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:09:32 smithi165 ceph-mon[36302]: pgmap v1365: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:09:32.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:09:32 smithi119 ceph-mon[31518]: pgmap v1365: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:09:34.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:09:34 smithi132 ceph-mon[35941]: pgmap v1366: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:09:34.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:09:34 smithi165 ceph-mon[36302]: pgmap v1366: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:09:34.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:09:34 smithi119 ceph-mon[31518]: pgmap v1366: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:09:36.431 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:09:36 smithi165 ceph-mon[36302]: pgmap v1367: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:09:36.431 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:09:36 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2035.690390 secs 2024-03-28T01:09:36.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:09:36 smithi132 ceph-mon[35941]: pgmap v1367: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:09:36.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:09:36 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2035.690390 secs 2024-03-28T01:09:36.455 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:09:36 smithi119 ceph-mon[31518]: pgmap v1367: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:09:36.455 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:09:36 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2035.690390 secs 2024-03-28T01:09:38.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:09:38 smithi132 ceph-mon[35941]: pgmap v1368: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T01:09:38.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:09:38 smithi165 ceph-mon[36302]: pgmap v1368: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T01:09:38.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:09:38 smithi119 ceph-mon[31518]: pgmap v1368: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T01:09:40.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:09:40 smithi132 ceph-mon[35941]: pgmap v1369: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:09:40.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:09:40 smithi165 ceph-mon[36302]: pgmap v1369: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:09:40.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:09:40 smithi119 ceph-mon[31518]: pgmap v1369: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:09:41.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:09:41 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2040.690450 secs 2024-03-28T01:09:41.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:09:41 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2040.690450 secs 2024-03-28T01:09:41.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:09:41 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2040.690450 secs 2024-03-28T01:09:42.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:09:42 smithi132 ceph-mon[35941]: pgmap v1370: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:09:42.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:09:42 smithi165 ceph-mon[36302]: pgmap v1370: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:09:42.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:09:42 smithi119 ceph-mon[31518]: pgmap v1370: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:09:44.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:09:44 smithi119 ceph-mon[31518]: pgmap v1371: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:09:44.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:09:44 smithi132 ceph-mon[35941]: pgmap v1371: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:09:44.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:09:44 smithi165 ceph-mon[36302]: pgmap v1371: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:09:46.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:09:46 smithi165 ceph-mon[36302]: pgmap v1372: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:09:46.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:09:46 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2045.690542 secs 2024-03-28T01:09:46.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:09:46 smithi119 ceph-mon[31518]: pgmap v1372: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:09:46.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:09:46 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2045.690542 secs 2024-03-28T01:09:46.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:09:46 smithi132 ceph-mon[35941]: pgmap v1372: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:09:46.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:09:46 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2045.690542 secs 2024-03-28T01:09:47.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:09:47 smithi165 ceph-mon[36302]: pgmap v1373: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:09:47.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:09:47 smithi119 ceph-mon[31518]: pgmap v1373: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:09:47.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:09:47 smithi132 ceph-mon[35941]: pgmap v1373: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:09:49.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:09:49 smithi165 ceph-mon[36302]: pgmap v1374: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:09:49.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:09:49 smithi119 ceph-mon[31518]: pgmap v1374: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:09:49.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:09:49 smithi132 ceph-mon[35941]: pgmap v1374: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:09:50.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:09:50 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2050.690635 secs 2024-03-28T01:09:50.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:09:50 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2050.690635 secs 2024-03-28T01:09:50.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:09:50 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2050.690635 secs 2024-03-28T01:09:51.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:09:51 smithi165 ceph-mon[36302]: pgmap v1375: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:09:51.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:09:51 smithi119 ceph-mon[31518]: pgmap v1375: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:09:51.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:09:51 smithi132 ceph-mon[35941]: pgmap v1375: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:09:53.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:09:53 smithi165 ceph-mon[36302]: pgmap v1376: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T01:09:53.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:09:53 smithi119 ceph-mon[31518]: pgmap v1376: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T01:09:53.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:09:53 smithi132 ceph-mon[35941]: pgmap v1376: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T01:09:55.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:09:55 smithi165 ceph-mon[36302]: pgmap v1377: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:09:55.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:09:55 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2055.690724 secs 2024-03-28T01:09:55.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:09:55 smithi119 ceph-mon[31518]: pgmap v1377: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:09:55.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:09:55 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2055.690724 secs 2024-03-28T01:09:55.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:09:55 smithi132 ceph-mon[35941]: pgmap v1377: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:09:55.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:09:55 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2055.690724 secs 2024-03-28T01:09:57.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:09:57 smithi119 ceph-mon[31518]: pgmap v1378: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T01:09:57.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:09:57 smithi132 ceph-mon[35941]: pgmap v1378: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T01:09:58.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:09:57 smithi165 ceph-mon[36302]: pgmap v1378: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T01:09:59.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:09:59 smithi119 ceph-mon[31518]: pgmap v1379: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T01:09:59.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:09:59 smithi132 ceph-mon[35941]: pgmap v1379: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T01:10:00.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:09:59 smithi165 ceph-mon[36302]: pgmap v1379: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T01:10:00.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:10:00 smithi119 ceph-mon[31518]: overall HEALTH_WARN 5 failed cephadm daemon(s); 1 filesystem is degraded; 1 MDSs report slow requests 2024-03-28T01:10:00.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:10:00 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2060.690784 secs 2024-03-28T01:10:00.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:10:00 smithi132 ceph-mon[35941]: overall HEALTH_WARN 5 failed cephadm daemon(s); 1 filesystem is degraded; 1 MDSs report slow requests 2024-03-28T01:10:00.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:10:00 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2060.690784 secs 2024-03-28T01:10:01.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:10:00 smithi165 ceph-mon[36302]: overall HEALTH_WARN 5 failed cephadm daemon(s); 1 filesystem is degraded; 1 MDSs report slow requests 2024-03-28T01:10:01.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:10:00 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2060.690784 secs 2024-03-28T01:10:02.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:10:02 smithi165 ceph-mon[36302]: pgmap v1380: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:10:02.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:10:02 smithi119 ceph-mon[31518]: pgmap v1380: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:10:02.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:10:02 smithi132 ceph-mon[35941]: pgmap v1380: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:10:04.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:10:04 smithi165 ceph-mon[36302]: pgmap v1381: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:10:04.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:10:04 smithi119 ceph-mon[31518]: pgmap v1381: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:10:04.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:10:04 smithi132 ceph-mon[35941]: pgmap v1381: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:10:06.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:10:06 smithi165 ceph-mon[36302]: pgmap v1382: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:10:06.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:10:06 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2065.690877 secs 2024-03-28T01:10:06.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:10:06 smithi119 ceph-mon[31518]: pgmap v1382: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:10:06.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:10:06 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2065.690877 secs 2024-03-28T01:10:06.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:10:06 smithi132 ceph-mon[35941]: pgmap v1382: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:10:06.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:10:06 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2065.690877 secs 2024-03-28T01:10:08.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:10:08 smithi165 ceph-mon[36302]: pgmap v1383: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:10:08.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:10:08 smithi119 ceph-mon[31518]: pgmap v1383: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:10:08.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:10:08 smithi132 ceph-mon[35941]: pgmap v1383: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:10:10.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:10:10 smithi165 ceph-mon[36302]: pgmap v1384: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:10:10.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:10:10 smithi119 ceph-mon[31518]: pgmap v1384: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:10:10.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:10:10 smithi132 ceph-mon[35941]: pgmap v1384: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:10:11.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:10:11 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2070.690964 secs 2024-03-28T01:10:11.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:10:11 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2070.690964 secs 2024-03-28T01:10:11.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:10:11 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2070.690964 secs 2024-03-28T01:10:12.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:10:12 smithi165 ceph-mon[36302]: pgmap v1385: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:10:12.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:10:12 smithi119 ceph-mon[31518]: pgmap v1385: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:10:12.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:10:12 smithi132 ceph-mon[35941]: pgmap v1385: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:10:14.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:10:14 smithi165 ceph-mon[36302]: pgmap v1386: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:10:14.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:10:14 smithi119 ceph-mon[31518]: pgmap v1386: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:10:14.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:10:14 smithi132 ceph-mon[35941]: pgmap v1386: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:10:16.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:10:16 smithi165 ceph-mon[36302]: pgmap v1387: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:10:16.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:10:16 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2075.691055 secs 2024-03-28T01:10:16.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:10:16 smithi119 ceph-mon[31518]: pgmap v1387: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:10:16.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:10:16 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2075.691055 secs 2024-03-28T01:10:16.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:10:16 smithi132 ceph-mon[35941]: pgmap v1387: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:10:16.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:10:16 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2075.691055 secs 2024-03-28T01:10:18.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:10:18 smithi119 ceph-mon[31518]: pgmap v1388: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:10:18.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:10:18 smithi132 ceph-mon[35941]: pgmap v1388: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:10:18.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:10:18 smithi165 ceph-mon[36302]: pgmap v1388: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:10:20.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:10:20 smithi119 ceph-mon[31518]: pgmap v1389: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:10:20.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:10:20 smithi132 ceph-mon[35941]: pgmap v1389: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:10:20.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:10:20 smithi165 ceph-mon[36302]: pgmap v1389: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:10:21.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:10:21 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2080.691092 secs 2024-03-28T01:10:21.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:10:21 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2080.691092 secs 2024-03-28T01:10:21.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:10:21 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2080.691092 secs 2024-03-28T01:10:22.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:10:22 smithi132 ceph-mon[35941]: pgmap v1390: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:10:22.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:10:22 smithi165 ceph-mon[36302]: pgmap v1390: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:10:22.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:10:22 smithi119 ceph-mon[31518]: pgmap v1390: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:10:24.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:10:24 smithi132 ceph-mon[35941]: pgmap v1391: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:10:24.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:10:24 smithi165 ceph-mon[36302]: pgmap v1391: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:10:24.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:10:24 smithi119 ceph-mon[31518]: pgmap v1391: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:10:26.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:10:26 smithi132 ceph-mon[35941]: pgmap v1392: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:10:26.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:10:26 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2085.691198 secs 2024-03-28T01:10:26.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:10:26 smithi165 ceph-mon[36302]: pgmap v1392: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:10:26.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:10:26 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2085.691198 secs 2024-03-28T01:10:26.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:10:26 smithi119 ceph-mon[31518]: pgmap v1392: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:10:26.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:10:26 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2085.691198 secs 2024-03-28T01:10:27.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:10:27 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:10:27.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:10:27 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:10:27.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:10:27 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:10:28.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:10:28 smithi132 ceph-mon[35941]: pgmap v1393: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:10:28.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:10:28 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:10:28.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:10:28 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:10:28.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:10:28 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:10:28.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:10:28 smithi165 ceph-mon[36302]: pgmap v1393: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:10:28.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:10:28 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:10:28.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:10:28 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:10:28.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:10:28 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:10:28.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:10:28 smithi119 ceph-mon[31518]: pgmap v1393: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:10:28.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:10:28 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:10:28.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:10:28 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:10:28.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:10:28 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:10:29.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:10:29 smithi132 ceph-mon[35941]: pgmap v1394: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T01:10:29.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:10:29 smithi132 ceph-mon[35941]: pgmap v1395: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:10:29.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:10:29 smithi165 ceph-mon[36302]: pgmap v1394: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T01:10:29.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:10:29 smithi165 ceph-mon[36302]: pgmap v1395: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:10:29.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:10:29 smithi119 ceph-mon[31518]: pgmap v1394: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T01:10:29.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:10:29 smithi119 ceph-mon[31518]: pgmap v1395: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:10:31.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:10:31 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2090.691287 secs 2024-03-28T01:10:31.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:10:31 smithi132 ceph-mon[35941]: pgmap v1396: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:10:31.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:10:31 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2090.691287 secs 2024-03-28T01:10:31.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:10:31 smithi165 ceph-mon[36302]: pgmap v1396: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:10:31.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:10:31 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2090.691287 secs 2024-03-28T01:10:31.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:10:31 smithi119 ceph-mon[31518]: pgmap v1396: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:10:33.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:10:33 smithi132 ceph-mon[35941]: pgmap v1397: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:10:33.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:10:33 smithi165 ceph-mon[36302]: pgmap v1397: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:10:33.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:10:33 smithi119 ceph-mon[31518]: pgmap v1397: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:10:35.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:10:35 smithi132 ceph-mon[35941]: pgmap v1398: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 5 op/s 2024-03-28T01:10:35.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:10:35 smithi165 ceph-mon[36302]: pgmap v1398: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 5 op/s 2024-03-28T01:10:35.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:10:35 smithi119 ceph-mon[31518]: pgmap v1398: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 5 op/s 2024-03-28T01:10:36.530 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:10:36 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2095.691352 secs 2024-03-28T01:10:36.554 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:10:36 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2095.691352 secs 2024-03-28T01:10:36.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:10:36 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2095.691352 secs 2024-03-28T01:10:37.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:10:37 smithi165 ceph-mon[36302]: pgmap v1399: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:10:37.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:10:37 smithi119 ceph-mon[31518]: pgmap v1399: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:10:37.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:10:37 smithi132 ceph-mon[35941]: pgmap v1399: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:10:38.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:10:38 smithi165 ceph-mon[36302]: pgmap v1400: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T01:10:38.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:10:38 smithi119 ceph-mon[31518]: pgmap v1400: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T01:10:38.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:10:38 smithi132 ceph-mon[35941]: pgmap v1400: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T01:10:41.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:10:41 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2100.691441 secs 2024-03-28T01:10:41.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:10:41 smithi132 ceph-mon[35941]: pgmap v1401: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:10:41.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:10:41 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2100.691441 secs 2024-03-28T01:10:41.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:10:41 smithi165 ceph-mon[36302]: pgmap v1401: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:10:41.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:10:41 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2100.691441 secs 2024-03-28T01:10:41.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:10:41 smithi119 ceph-mon[31518]: pgmap v1401: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:10:43.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:10:43 smithi132 ceph-mon[35941]: pgmap v1402: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:10:43.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:10:43 smithi165 ceph-mon[36302]: pgmap v1402: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:10:43.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:10:43 smithi119 ceph-mon[31518]: pgmap v1402: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:10:45.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:10:45 smithi132 ceph-mon[35941]: pgmap v1403: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-28T01:10:45.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:10:45 smithi165 ceph-mon[36302]: pgmap v1403: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-28T01:10:45.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:10:45 smithi119 ceph-mon[31518]: pgmap v1403: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-28T01:10:46.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:10:46 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2105.691530 secs 2024-03-28T01:10:46.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:10:46 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2105.691530 secs 2024-03-28T01:10:46.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:10:46 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2105.691530 secs 2024-03-28T01:10:47.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:10:47 smithi119 ceph-mon[31518]: pgmap v1404: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-03-28T01:10:47.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:10:47 smithi132 ceph-mon[35941]: pgmap v1404: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-03-28T01:10:47.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:10:47 smithi165 ceph-mon[36302]: pgmap v1404: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-03-28T01:10:49.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:10:49 smithi132 ceph-mon[35941]: pgmap v1405: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:10:49.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:10:49 smithi165 ceph-mon[36302]: pgmap v1405: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:10:49.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:10:49 smithi119 ceph-mon[31518]: pgmap v1405: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:10:51.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:10:51 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2110.691620 secs 2024-03-28T01:10:51.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:10:51 smithi132 ceph-mon[35941]: pgmap v1406: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:10:51.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:10:51 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2110.691620 secs 2024-03-28T01:10:51.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:10:51 smithi165 ceph-mon[36302]: pgmap v1406: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:10:51.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:10:51 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2110.691620 secs 2024-03-28T01:10:51.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:10:51 smithi119 ceph-mon[31518]: pgmap v1406: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:10:53.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:10:53 smithi165 ceph-mon[36302]: pgmap v1407: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:10:53.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:10:53 smithi119 ceph-mon[31518]: pgmap v1407: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:10:53.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:10:53 smithi132 ceph-mon[35941]: pgmap v1407: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:10:55.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:10:55 smithi165 ceph-mon[36302]: pgmap v1408: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-28T01:10:55.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:10:55 smithi119 ceph-mon[31518]: pgmap v1408: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-28T01:10:55.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:10:55 smithi132 ceph-mon[35941]: pgmap v1408: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-28T01:10:56.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:10:56 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2115.691667 secs 2024-03-28T01:10:56.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:10:56 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2115.691667 secs 2024-03-28T01:10:56.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:10:56 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2115.691667 secs 2024-03-28T01:10:57.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:10:57 smithi165 ceph-mon[36302]: pgmap v1409: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-03-28T01:10:57.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:10:57 smithi119 ceph-mon[31518]: pgmap v1409: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-03-28T01:10:57.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:10:57 smithi132 ceph-mon[35941]: pgmap v1409: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-03-28T01:10:58.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:10:58 smithi165 ceph-mon[36302]: pgmap v1410: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:10:58.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:10:58 smithi119 ceph-mon[31518]: pgmap v1410: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:10:58.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:10:58 smithi132 ceph-mon[35941]: pgmap v1410: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:11:01.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:11:01 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2120.691730 secs 2024-03-28T01:11:01.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:11:01 smithi132 ceph-mon[35941]: pgmap v1411: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:11:01.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:11:01 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2120.691730 secs 2024-03-28T01:11:01.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:11:01 smithi165 ceph-mon[36302]: pgmap v1411: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:11:01.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:11:01 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2120.691730 secs 2024-03-28T01:11:01.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:11:01 smithi119 ceph-mon[31518]: pgmap v1411: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:11:03.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:11:03 smithi119 ceph-mon[31518]: pgmap v1412: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:11:03.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:11:03 smithi132 ceph-mon[35941]: pgmap v1412: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:11:03.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:11:03 smithi165 ceph-mon[36302]: pgmap v1412: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:11:05.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:11:05 smithi132 ceph-mon[35941]: pgmap v1413: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-28T01:11:05.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:11:05 smithi165 ceph-mon[36302]: pgmap v1413: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-28T01:11:05.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:11:05 smithi119 ceph-mon[31518]: pgmap v1413: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-28T01:11:06.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:11:06 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2125.691791 secs 2024-03-28T01:11:06.530 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:11:06 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2125.691791 secs 2024-03-28T01:11:06.578 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:11:06 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2125.691791 secs 2024-03-28T01:11:07.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:11:07 smithi132 ceph-mon[35941]: pgmap v1414: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-03-28T01:11:07.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:11:07 smithi165 ceph-mon[36302]: pgmap v1414: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-03-28T01:11:07.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:11:07 smithi119 ceph-mon[31518]: pgmap v1414: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-03-28T01:11:09.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:11:09 smithi165 ceph-mon[36302]: pgmap v1415: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:11:09.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:11:09 smithi119 ceph-mon[31518]: pgmap v1415: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:11:09.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:11:09 smithi132 ceph-mon[35941]: pgmap v1415: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:11:11.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:11:11 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2130.691834 secs 2024-03-28T01:11:11.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:11:11 smithi165 ceph-mon[36302]: pgmap v1416: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:11:11.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:11:11 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2130.691834 secs 2024-03-28T01:11:11.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:11:11 smithi119 ceph-mon[31518]: pgmap v1416: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:11:11.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:11:11 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2130.691834 secs 2024-03-28T01:11:11.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:11:11 smithi132 ceph-mon[35941]: pgmap v1416: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:11:13.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:11:13 smithi165 ceph-mon[36302]: pgmap v1417: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:11:13.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:11:13 smithi119 ceph-mon[31518]: pgmap v1417: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:11:13.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:11:13 smithi132 ceph-mon[35941]: pgmap v1417: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:11:15.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:11:15 smithi165 ceph-mon[36302]: pgmap v1418: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-28T01:11:15.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:11:15 smithi119 ceph-mon[31518]: pgmap v1418: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-28T01:11:15.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:11:15 smithi132 ceph-mon[35941]: pgmap v1418: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-28T01:11:16.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:11:16 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2135.691923 secs 2024-03-28T01:11:16.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:11:16 smithi165 ceph-mon[36302]: pgmap v1419: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-03-28T01:11:16.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:11:16 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2135.691923 secs 2024-03-28T01:11:16.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:11:16 smithi119 ceph-mon[31518]: pgmap v1419: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-03-28T01:11:16.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:11:16 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2135.691923 secs 2024-03-28T01:11:16.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:11:16 smithi132 ceph-mon[35941]: pgmap v1419: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-03-28T01:11:19.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:11:19 smithi132 ceph-mon[35941]: pgmap v1420: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:11:19.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:11:19 smithi165 ceph-mon[36302]: pgmap v1420: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:11:19.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:11:19 smithi119 ceph-mon[31518]: pgmap v1420: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:11:21.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:11:21 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2140.691985 secs 2024-03-28T01:11:21.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:11:21 smithi132 ceph-mon[35941]: pgmap v1421: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:11:21.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:11:21 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2140.691985 secs 2024-03-28T01:11:21.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:11:21 smithi165 ceph-mon[36302]: pgmap v1421: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:11:21.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:11:21 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2140.691985 secs 2024-03-28T01:11:21.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:11:21 smithi119 ceph-mon[31518]: pgmap v1421: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:11:23.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:11:23 smithi132 ceph-mon[35941]: pgmap v1422: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:11:23.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:11:23 smithi165 ceph-mon[36302]: pgmap v1422: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:11:23.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:11:23 smithi119 ceph-mon[31518]: pgmap v1422: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:11:25.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:11:25 smithi132 ceph-mon[35941]: pgmap v1423: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-28T01:11:25.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:11:25 smithi165 ceph-mon[36302]: pgmap v1423: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-28T01:11:25.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:11:25 smithi119 ceph-mon[31518]: pgmap v1423: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-28T01:11:26.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:11:26 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2145.692047 secs 2024-03-28T01:11:26.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:11:26 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2145.692047 secs 2024-03-28T01:11:26.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:11:26 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2145.692047 secs 2024-03-28T01:11:27.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:11:27 smithi165 ceph-mon[36302]: pgmap v1424: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-03-28T01:11:27.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:11:27 smithi119 ceph-mon[31518]: pgmap v1424: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-03-28T01:11:27.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:11:27 smithi132 ceph-mon[35941]: pgmap v1424: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-03-28T01:11:29.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:11:29 smithi165 ceph-mon[36302]: pgmap v1425: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:11:29.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:11:29 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:11:29.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:11:29 smithi119 ceph-mon[31518]: pgmap v1425: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:11:29.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:11:29 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:11:29.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:11:29 smithi132 ceph-mon[35941]: pgmap v1425: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:11:29.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:11:29 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:11:30.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:11:30 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:11:30.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:11:30 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:11:30.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:11:30 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:11:30.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:11:30 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:11:30.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:11:30 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:11:30.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:11:30 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:11:30.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:11:30 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:11:30.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:11:30 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:11:30.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:11:30 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:11:31.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:11:31 smithi165 ceph-mon[36302]: pgmap v1426: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T01:11:31.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:11:31 smithi165 ceph-mon[36302]: pgmap v1427: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:11:31.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:11:31 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2150.692110 secs 2024-03-28T01:11:31.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:11:31 smithi119 ceph-mon[31518]: pgmap v1426: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T01:11:31.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:11:31 smithi119 ceph-mon[31518]: pgmap v1427: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:11:31.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:11:31 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2150.692110 secs 2024-03-28T01:11:31.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:11:31 smithi132 ceph-mon[35941]: pgmap v1426: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T01:11:31.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:11:31 smithi132 ceph-mon[35941]: pgmap v1427: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:11:31.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:11:31 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2150.692110 secs 2024-03-28T01:11:32.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:11:32 smithi165 ceph-mon[36302]: pgmap v1428: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:11:32.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:11:32 smithi119 ceph-mon[31518]: pgmap v1428: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:11:32.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:11:32 smithi132 ceph-mon[35941]: pgmap v1428: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:11:34.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:11:34 smithi132 ceph-mon[35941]: pgmap v1429: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 4 op/s 2024-03-28T01:11:35.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:11:34 smithi165 ceph-mon[36302]: pgmap v1429: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 4 op/s 2024-03-28T01:11:35.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:11:34 smithi119 ceph-mon[31518]: pgmap v1429: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 4 op/s 2024-03-28T01:11:35.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:11:35 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2155.692170 secs 2024-03-28T01:11:36.030 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:11:35 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2155.692170 secs 2024-03-28T01:11:36.054 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:11:35 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2155.692170 secs 2024-03-28T01:11:37.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:11:36 smithi165 ceph-mon[36302]: pgmap v1430: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:11:37.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:11:36 smithi119 ceph-mon[31518]: pgmap v1430: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:11:37.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:11:36 smithi132 ceph-mon[35941]: pgmap v1430: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:11:39.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:11:38 smithi165 ceph-mon[36302]: pgmap v1431: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:11:39.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:11:38 smithi119 ceph-mon[31518]: pgmap v1431: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:11:39.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:11:38 smithi132 ceph-mon[35941]: pgmap v1431: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:11:41.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:11:40 smithi165 ceph-mon[36302]: pgmap v1432: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T01:11:41.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:11:40 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2160.692253 secs 2024-03-28T01:11:41.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:11:40 smithi119 ceph-mon[31518]: pgmap v1432: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T01:11:41.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:11:40 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2160.692253 secs 2024-03-28T01:11:41.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:11:40 smithi132 ceph-mon[35941]: pgmap v1432: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T01:11:41.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:11:40 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2160.692253 secs 2024-03-28T01:11:43.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:11:42 smithi165 ceph-mon[36302]: pgmap v1433: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:11:43.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:11:42 smithi119 ceph-mon[31518]: pgmap v1433: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:11:43.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:11:42 smithi132 ceph-mon[35941]: pgmap v1433: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:11:45.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:11:44 smithi165 ceph-mon[36302]: pgmap v1434: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:11:45.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:11:44 smithi119 ceph-mon[31518]: pgmap v1434: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:11:45.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:11:44 smithi132 ceph-mon[35941]: pgmap v1434: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:11:46.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:11:45 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2165.692342 secs 2024-03-28T01:11:46.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:11:45 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2165.692342 secs 2024-03-28T01:11:46.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:11:45 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2165.692342 secs 2024-03-28T01:11:47.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:11:46 smithi165 ceph-mon[36302]: pgmap v1435: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:11:47.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:11:46 smithi119 ceph-mon[31518]: pgmap v1435: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:11:47.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:11:46 smithi132 ceph-mon[35941]: pgmap v1435: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:11:49.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:11:48 smithi165 ceph-mon[36302]: pgmap v1436: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T01:11:49.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:11:48 smithi119 ceph-mon[31518]: pgmap v1436: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T01:11:49.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:11:48 smithi132 ceph-mon[35941]: pgmap v1436: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T01:11:51.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:11:50 smithi165 ceph-mon[36302]: pgmap v1437: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-03-28T01:11:51.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:11:50 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2170.692430 secs 2024-03-28T01:11:51.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:11:50 smithi119 ceph-mon[31518]: pgmap v1437: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-03-28T01:11:51.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:11:50 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2170.692430 secs 2024-03-28T01:11:51.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:11:50 smithi132 ceph-mon[35941]: pgmap v1437: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-03-28T01:11:51.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:11:50 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2170.692430 secs 2024-03-28T01:11:53.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:11:52 smithi165 ceph-mon[36302]: pgmap v1438: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-03-28T01:11:53.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:11:52 smithi119 ceph-mon[31518]: pgmap v1438: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-03-28T01:11:53.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:11:52 smithi132 ceph-mon[35941]: pgmap v1438: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-03-28T01:11:55.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:11:54 smithi165 ceph-mon[36302]: pgmap v1439: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-03-28T01:11:55.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:11:54 smithi119 ceph-mon[31518]: pgmap v1439: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-03-28T01:11:55.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:11:54 smithi132 ceph-mon[35941]: pgmap v1439: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-03-28T01:11:56.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:11:55 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2175.692520 secs 2024-03-28T01:11:56.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:11:55 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2175.692520 secs 2024-03-28T01:11:56.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:11:55 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2175.692520 secs 2024-03-28T01:11:57.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:11:56 smithi165 ceph-mon[36302]: pgmap v1440: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-03-28T01:11:57.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:11:56 smithi119 ceph-mon[31518]: pgmap v1440: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-03-28T01:11:57.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:11:56 smithi132 ceph-mon[35941]: pgmap v1440: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-03-28T01:11:59.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:11:58 smithi165 ceph-mon[36302]: pgmap v1441: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T01:11:59.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:11:58 smithi119 ceph-mon[31518]: pgmap v1441: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T01:11:59.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:11:58 smithi132 ceph-mon[35941]: pgmap v1441: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T01:12:01.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:12:00 smithi165 ceph-mon[36302]: pgmap v1442: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:12:01.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:12:00 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2180.692628 secs 2024-03-28T01:12:01.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:12:00 smithi119 ceph-mon[31518]: pgmap v1442: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:12:01.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:12:00 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2180.692628 secs 2024-03-28T01:12:01.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:12:00 smithi132 ceph-mon[35941]: pgmap v1442: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:12:01.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:12:00 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2180.692628 secs 2024-03-28T01:12:03.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:12:02 smithi165 ceph-mon[36302]: pgmap v1443: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:12:03.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:12:02 smithi119 ceph-mon[31518]: pgmap v1443: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:12:03.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:12:02 smithi132 ceph-mon[35941]: pgmap v1443: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:12:05.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:12:04 smithi165 ceph-mon[36302]: pgmap v1444: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:12:05.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:12:04 smithi119 ceph-mon[31518]: pgmap v1444: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:12:05.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:12:04 smithi132 ceph-mon[35941]: pgmap v1444: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:12:06.054 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:12:05 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2185.692641 secs 2024-03-28T01:12:06.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:12:05 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2185.692641 secs 2024-03-28T01:12:06.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:12:05 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2185.692641 secs 2024-03-28T01:12:07.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:12:06 smithi165 ceph-mon[36302]: pgmap v1445: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:12:07.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:12:06 smithi119 ceph-mon[31518]: pgmap v1445: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:12:07.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:12:06 smithi132 ceph-mon[35941]: pgmap v1445: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:12:09.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:12:08 smithi165 ceph-mon[36302]: pgmap v1446: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:12:09.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:12:08 smithi119 ceph-mon[31518]: pgmap v1446: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:12:09.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:12:08 smithi132 ceph-mon[35941]: pgmap v1446: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:12:11.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:12:10 smithi165 ceph-mon[36302]: pgmap v1447: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:12:11.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:12:10 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2190.692730 secs 2024-03-28T01:12:11.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:12:10 smithi119 ceph-mon[31518]: pgmap v1447: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:12:11.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:12:10 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2190.692730 secs 2024-03-28T01:12:11.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:12:10 smithi132 ceph-mon[35941]: pgmap v1447: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:12:11.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:12:10 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2190.692730 secs 2024-03-28T01:12:13.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:12:12 smithi165 ceph-mon[36302]: pgmap v1448: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:12:13.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:12:12 smithi119 ceph-mon[31518]: pgmap v1448: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:12:13.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:12:12 smithi132 ceph-mon[35941]: pgmap v1448: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:12:15.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:12:14 smithi165 ceph-mon[36302]: pgmap v1449: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:12:15.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:12:14 smithi119 ceph-mon[31518]: pgmap v1449: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:12:15.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:12:14 smithi132 ceph-mon[35941]: pgmap v1449: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:12:16.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:12:15 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2195.692819 secs 2024-03-28T01:12:16.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:12:15 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2195.692819 secs 2024-03-28T01:12:16.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:12:15 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2195.692819 secs 2024-03-28T01:12:17.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:12:16 smithi165 ceph-mon[36302]: pgmap v1450: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:12:17.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:12:16 smithi119 ceph-mon[31518]: pgmap v1450: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:12:17.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:12:16 smithi132 ceph-mon[35941]: pgmap v1450: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:12:19.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:12:18 smithi165 ceph-mon[36302]: pgmap v1451: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:12:19.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:12:18 smithi119 ceph-mon[31518]: pgmap v1451: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:12:19.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:12:18 smithi132 ceph-mon[35941]: pgmap v1451: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:12:21.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:12:20 smithi165 ceph-mon[36302]: pgmap v1452: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:12:21.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:12:20 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2200.692881 secs 2024-03-28T01:12:21.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:12:20 smithi119 ceph-mon[31518]: pgmap v1452: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:12:21.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:12:20 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2200.692881 secs 2024-03-28T01:12:21.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:12:20 smithi132 ceph-mon[35941]: pgmap v1452: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:12:21.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:12:20 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2200.692881 secs 2024-03-28T01:12:23.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:12:22 smithi165 ceph-mon[36302]: pgmap v1453: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:12:23.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:12:22 smithi119 ceph-mon[31518]: pgmap v1453: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:12:23.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:12:22 smithi132 ceph-mon[35941]: pgmap v1453: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:12:25.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:12:24 smithi165 ceph-mon[36302]: pgmap v1454: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:12:25.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:12:24 smithi119 ceph-mon[31518]: pgmap v1454: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:12:25.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:12:24 smithi132 ceph-mon[35941]: pgmap v1454: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:12:26.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:12:25 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2205.692897 secs 2024-03-28T01:12:26.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:12:25 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2205.692897 secs 2024-03-28T01:12:26.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:12:25 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2205.692897 secs 2024-03-28T01:12:27.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:12:26 smithi165 ceph-mon[36302]: pgmap v1455: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:12:27.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:12:26 smithi119 ceph-mon[31518]: pgmap v1455: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:12:27.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:12:26 smithi132 ceph-mon[35941]: pgmap v1455: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:12:29.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:12:28 smithi165 ceph-mon[36302]: pgmap v1456: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:12:29.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:12:28 smithi119 ceph-mon[31518]: pgmap v1456: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:12:29.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:12:28 smithi132 ceph-mon[35941]: pgmap v1456: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:12:31.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:12:30 smithi165 ceph-mon[36302]: pgmap v1457: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:12:31.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:12:30 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:12:31.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:12:30 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2210.692967 secs 2024-03-28T01:12:31.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:12:30 smithi119 ceph-mon[31518]: pgmap v1457: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:12:31.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:12:30 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:12:31.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:12:30 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2210.692967 secs 2024-03-28T01:12:31.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:12:30 smithi132 ceph-mon[35941]: pgmap v1457: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:12:31.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:12:30 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:12:31.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:12:30 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2210.692967 secs 2024-03-28T01:12:32.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:12:31 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:12:32.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:12:31 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:12:32.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:12:31 smithi165 ceph-mon[36302]: pgmap v1458: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T01:12:32.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:12:31 smithi165 ceph-mon[36302]: pgmap v1459: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:12:32.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:12:31 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:12:32.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:12:31 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:12:32.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:12:31 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:12:32.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:12:31 smithi119 ceph-mon[31518]: pgmap v1458: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T01:12:32.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:12:31 smithi119 ceph-mon[31518]: pgmap v1459: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:12:32.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:12:31 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:12:32.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:12:31 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:12:32.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:12:31 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:12:32.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:12:31 smithi132 ceph-mon[35941]: pgmap v1458: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T01:12:32.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:12:31 smithi132 ceph-mon[35941]: pgmap v1459: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:12:32.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:12:31 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:12:34.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:12:34 smithi165 ceph-mon[36302]: pgmap v1460: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:12:34.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:12:34 smithi119 ceph-mon[31518]: pgmap v1460: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:12:34.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:12:34 smithi132 ceph-mon[35941]: pgmap v1460: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:12:36.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:12:36 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2215.693031 secs 2024-03-28T01:12:36.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:12:36 smithi119 ceph-mon[31518]: pgmap v1461: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:12:36.501 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:12:36 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2215.693031 secs 2024-03-28T01:12:36.502 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:12:36 smithi132 ceph-mon[35941]: pgmap v1461: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:12:36.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:12:36 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2215.693031 secs 2024-03-28T01:12:36.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:12:36 smithi165 ceph-mon[36302]: pgmap v1461: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:12:38.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:12:38 smithi165 ceph-mon[36302]: pgmap v1462: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:12:38.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:12:38 smithi119 ceph-mon[31518]: pgmap v1462: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:12:38.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:12:38 smithi132 ceph-mon[35941]: pgmap v1462: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:12:40.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:12:40 smithi165 ceph-mon[36302]: pgmap v1463: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 5 op/s 2024-03-28T01:12:40.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:12:40 smithi119 ceph-mon[31518]: pgmap v1463: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 5 op/s 2024-03-28T01:12:40.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:12:40 smithi132 ceph-mon[35941]: pgmap v1463: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 5 op/s 2024-03-28T01:12:41.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:12:41 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2220.693091 secs 2024-03-28T01:12:41.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:12:41 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2220.693091 secs 2024-03-28T01:12:41.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:12:41 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2220.693091 secs 2024-03-28T01:12:42.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:12:42 smithi165 ceph-mon[36302]: pgmap v1464: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T01:12:42.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:12:42 smithi119 ceph-mon[31518]: pgmap v1464: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T01:12:42.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:12:42 smithi132 ceph-mon[35941]: pgmap v1464: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T01:12:44.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:12:44 smithi165 ceph-mon[36302]: pgmap v1465: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:12:44.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:12:44 smithi119 ceph-mon[31518]: pgmap v1465: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:12:44.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:12:44 smithi132 ceph-mon[35941]: pgmap v1465: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:12:45.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:12:45 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2225.693168 secs 2024-03-28T01:12:45.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:12:45 smithi165 ceph-mon[36302]: pgmap v1466: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:12:45.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:12:45 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2225.693168 secs 2024-03-28T01:12:45.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:12:45 smithi119 ceph-mon[31518]: pgmap v1466: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:12:45.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:12:45 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2225.693168 secs 2024-03-28T01:12:45.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:12:45 smithi132 ceph-mon[35941]: pgmap v1466: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:12:48.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:12:48 smithi165 ceph-mon[36302]: pgmap v1467: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:12:48.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:12:48 smithi119 ceph-mon[31518]: pgmap v1467: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:12:48.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:12:48 smithi132 ceph-mon[35941]: pgmap v1467: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:12:50.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:12:50 smithi165 ceph-mon[36302]: pgmap v1468: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-28T01:12:50.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:12:50 smithi119 ceph-mon[31518]: pgmap v1468: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-28T01:12:50.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:12:50 smithi132 ceph-mon[35941]: pgmap v1468: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-28T01:12:51.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:12:51 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2230.693252 secs 2024-03-28T01:12:51.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:12:51 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2230.693252 secs 2024-03-28T01:12:51.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:12:51 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2230.693252 secs 2024-03-28T01:12:52.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:12:52 smithi119 ceph-mon[31518]: pgmap v1469: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-03-28T01:12:52.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:12:52 smithi165 ceph-mon[36302]: pgmap v1469: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-03-28T01:12:52.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:12:52 smithi132 ceph-mon[35941]: pgmap v1469: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-03-28T01:12:54.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:12:54 smithi165 ceph-mon[36302]: pgmap v1470: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:12:54.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:12:54 smithi119 ceph-mon[31518]: pgmap v1470: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:12:54.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:12:54 smithi132 ceph-mon[35941]: pgmap v1470: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:12:55.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:12:55 smithi165 ceph-mon[36302]: pgmap v1471: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:12:55.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:12:55 smithi119 ceph-mon[31518]: pgmap v1471: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:12:55.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:12:55 smithi132 ceph-mon[35941]: pgmap v1471: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:12:56.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:12:56 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2235.693338 secs 2024-03-28T01:12:56.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:12:56 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2235.693338 secs 2024-03-28T01:12:56.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:12:56 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2235.693338 secs 2024-03-28T01:12:57.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:12:57 smithi165 ceph-mon[36302]: pgmap v1472: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:12:57.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:12:57 smithi119 ceph-mon[31518]: pgmap v1472: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:12:57.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:12:57 smithi132 ceph-mon[35941]: pgmap v1472: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:13:00.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:13:00 smithi165 ceph-mon[36302]: pgmap v1473: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-28T01:13:00.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:13:00 smithi119 ceph-mon[31518]: pgmap v1473: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-28T01:13:00.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:13:00 smithi132 ceph-mon[35941]: pgmap v1473: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-28T01:13:01.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:13:01 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2240.693428 secs 2024-03-28T01:13:01.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:13:01 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2240.693428 secs 2024-03-28T01:13:01.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:13:01 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2240.693428 secs 2024-03-28T01:13:02.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:13:02 smithi165 ceph-mon[36302]: pgmap v1474: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-03-28T01:13:02.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:13:02 smithi119 ceph-mon[31518]: pgmap v1474: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-03-28T01:13:02.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:13:02 smithi132 ceph-mon[35941]: pgmap v1474: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-03-28T01:13:03.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:13:03 smithi165 ceph-mon[36302]: pgmap v1475: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:13:03.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:13:03 smithi119 ceph-mon[31518]: pgmap v1475: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:13:03.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:13:03 smithi132 ceph-mon[35941]: pgmap v1475: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:13:06.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:13:06 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2245.693517 secs 2024-03-28T01:13:06.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:13:06 smithi165 ceph-mon[36302]: pgmap v1476: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:13:06.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:13:06 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2245.693517 secs 2024-03-28T01:13:06.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:13:06 smithi119 ceph-mon[31518]: pgmap v1476: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:13:06.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:13:06 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2245.693517 secs 2024-03-28T01:13:06.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:13:06 smithi132 ceph-mon[35941]: pgmap v1476: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:13:08.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:13:08 smithi165 ceph-mon[36302]: pgmap v1477: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:13:08.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:13:08 smithi119 ceph-mon[31518]: pgmap v1477: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:13:08.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:13:08 smithi132 ceph-mon[35941]: pgmap v1477: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:13:10.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:13:10 smithi165 ceph-mon[36302]: pgmap v1478: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-28T01:13:10.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:13:10 smithi119 ceph-mon[31518]: pgmap v1478: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-28T01:13:10.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:13:10 smithi132 ceph-mon[35941]: pgmap v1478: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-28T01:13:11.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:13:11 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2250.693560 secs 2024-03-28T01:13:11.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:13:11 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2250.693560 secs 2024-03-28T01:13:11.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:13:11 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2250.693560 secs 2024-03-28T01:13:12.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:13:12 smithi165 ceph-mon[36302]: pgmap v1479: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-03-28T01:13:12.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:13:12 smithi119 ceph-mon[31518]: pgmap v1479: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-03-28T01:13:12.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:13:12 smithi132 ceph-mon[35941]: pgmap v1479: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-03-28T01:13:13.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:13:13 smithi165 ceph-mon[36302]: pgmap v1480: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:13:13.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:13:13 smithi119 ceph-mon[31518]: pgmap v1480: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:13:13.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:13:13 smithi132 ceph-mon[35941]: pgmap v1480: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:13:16.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:13:16 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2255.693650 secs 2024-03-28T01:13:16.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:13:16 smithi165 ceph-mon[36302]: pgmap v1481: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:13:16.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:13:16 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2255.693650 secs 2024-03-28T01:13:16.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:13:16 smithi119 ceph-mon[31518]: pgmap v1481: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:13:16.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:13:16 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2255.693650 secs 2024-03-28T01:13:16.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:13:16 smithi132 ceph-mon[35941]: pgmap v1481: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:13:18.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:13:18 smithi165 ceph-mon[36302]: pgmap v1482: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:13:18.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:13:18 smithi119 ceph-mon[31518]: pgmap v1482: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:13:18.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:13:18 smithi132 ceph-mon[35941]: pgmap v1482: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:13:20.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:13:20 smithi165 ceph-mon[36302]: pgmap v1483: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-28T01:13:20.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:13:20 smithi119 ceph-mon[31518]: pgmap v1483: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-28T01:13:20.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:13:20 smithi132 ceph-mon[35941]: pgmap v1483: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-28T01:13:21.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:13:21 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2260.693691 secs 2024-03-28T01:13:21.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:13:21 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2260.693691 secs 2024-03-28T01:13:21.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:13:21 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2260.693691 secs 2024-03-28T01:13:22.554 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:13:22 smithi119 ceph-mon[31518]: pgmap v1484: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-03-28T01:13:22.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:13:22 smithi165 ceph-mon[36302]: pgmap v1484: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-03-28T01:13:22.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:13:22 smithi132 ceph-mon[35941]: pgmap v1484: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-03-28T01:13:23.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:13:23 smithi165 ceph-mon[36302]: pgmap v1485: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:13:23.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:13:23 smithi119 ceph-mon[31518]: pgmap v1485: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:13:23.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:13:23 smithi132 ceph-mon[35941]: pgmap v1485: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:13:26.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:13:26 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2265.693710 secs 2024-03-28T01:13:26.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:13:26 smithi165 ceph-mon[36302]: pgmap v1486: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:13:26.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:13:26 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2265.693710 secs 2024-03-28T01:13:26.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:13:26 smithi119 ceph-mon[31518]: pgmap v1486: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:13:26.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:13:26 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2265.693710 secs 2024-03-28T01:13:26.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:13:26 smithi132 ceph-mon[35941]: pgmap v1486: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:13:28.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:13:28 smithi165 ceph-mon[36302]: pgmap v1487: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:13:28.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:13:28 smithi119 ceph-mon[31518]: pgmap v1487: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:13:28.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:13:28 smithi132 ceph-mon[35941]: pgmap v1487: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:13:30.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:13:30 smithi165 ceph-mon[36302]: pgmap v1488: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-28T01:13:30.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:13:30 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2270.693797 secs 2024-03-28T01:13:30.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:13:30 smithi119 ceph-mon[31518]: pgmap v1488: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-28T01:13:30.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:13:30 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2270.693797 secs 2024-03-28T01:13:30.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:13:30 smithi132 ceph-mon[35941]: pgmap v1488: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-28T01:13:30.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:13:30 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2270.693797 secs 2024-03-28T01:13:31.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:13:31 smithi165 ceph-mon[36302]: pgmap v1489: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-03-28T01:13:31.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:13:31 smithi119 ceph-mon[31518]: pgmap v1489: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-03-28T01:13:31.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:13:31 smithi132 ceph-mon[35941]: pgmap v1489: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-03-28T01:13:32.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:13:32 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:13:32.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:13:32 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:13:32.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:13:32 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:13:33.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:13:33 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:13:33.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:13:33 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:13:33.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:13:33 smithi165 ceph-mon[36302]: pgmap v1490: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T01:13:33.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:13:33 smithi165 ceph-mon[36302]: pgmap v1491: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:13:33.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:13:33 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:13:33.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:13:33 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:13:33.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:13:33 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:13:33.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:13:33 smithi119 ceph-mon[31518]: pgmap v1490: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T01:13:33.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:13:33 smithi119 ceph-mon[31518]: pgmap v1491: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:13:33.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:13:33 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:13:33.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:13:33 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:13:33.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:13:33 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:13:33.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:13:33 smithi132 ceph-mon[35941]: pgmap v1490: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T01:13:33.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:13:33 smithi132 ceph-mon[35941]: pgmap v1491: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:13:33.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:13:33 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:13:36.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:13:35 smithi165 ceph-mon[36302]: pgmap v1492: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:13:36.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:13:35 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2275.693887 secs 2024-03-28T01:13:36.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:13:35 smithi119 ceph-mon[31518]: pgmap v1492: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:13:36.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:13:35 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2275.693887 secs 2024-03-28T01:13:36.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:13:35 smithi132 ceph-mon[35941]: pgmap v1492: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:13:36.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:13:35 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2275.693887 secs 2024-03-28T01:13:38.082 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:13:37 smithi119 ceph-mon[31518]: pgmap v1493: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:13:38.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:13:37 smithi165 ceph-mon[36302]: pgmap v1493: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:13:38.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:13:37 smithi132 ceph-mon[35941]: pgmap v1493: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:13:40.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:13:39 smithi165 ceph-mon[36302]: pgmap v1494: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-28T01:13:40.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:13:39 smithi119 ceph-mon[31518]: pgmap v1494: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-28T01:13:40.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:13:39 smithi132 ceph-mon[35941]: pgmap v1494: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-28T01:13:41.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:13:40 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2280.693951 secs 2024-03-28T01:13:41.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:13:40 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2280.693951 secs 2024-03-28T01:13:41.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:13:40 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2280.693951 secs 2024-03-28T01:13:42.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:13:41 smithi165 ceph-mon[36302]: pgmap v1495: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:13:42.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:13:41 smithi119 ceph-mon[31518]: pgmap v1495: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:13:42.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:13:41 smithi132 ceph-mon[35941]: pgmap v1495: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:13:44.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:13:43 smithi165 ceph-mon[36302]: pgmap v1496: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T01:13:44.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:13:43 smithi119 ceph-mon[31518]: pgmap v1496: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T01:13:44.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:13:43 smithi132 ceph-mon[35941]: pgmap v1496: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T01:13:46.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:13:45 smithi165 ceph-mon[36302]: pgmap v1497: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:13:46.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:13:45 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2285.694015 secs 2024-03-28T01:13:46.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:13:45 smithi119 ceph-mon[31518]: pgmap v1497: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:13:46.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:13:45 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2285.694015 secs 2024-03-28T01:13:46.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:13:45 smithi132 ceph-mon[35941]: pgmap v1497: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:13:46.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:13:45 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2285.694015 secs 2024-03-28T01:13:48.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:13:47 smithi165 ceph-mon[36302]: pgmap v1498: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:13:48.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:13:47 smithi119 ceph-mon[31518]: pgmap v1498: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:13:48.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:13:47 smithi132 ceph-mon[35941]: pgmap v1498: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:13:50.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:13:49 smithi165 ceph-mon[36302]: pgmap v1499: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:13:50.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:13:49 smithi119 ceph-mon[31518]: pgmap v1499: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:13:50.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:13:49 smithi132 ceph-mon[35941]: pgmap v1499: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:13:51.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:13:50 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2290.694076 secs 2024-03-28T01:13:51.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:13:50 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2290.694076 secs 2024-03-28T01:13:51.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:13:50 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2290.694076 secs 2024-03-28T01:13:52.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:13:51 smithi165 ceph-mon[36302]: pgmap v1500: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:13:52.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:13:51 smithi119 ceph-mon[31518]: pgmap v1500: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:13:52.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:13:51 smithi132 ceph-mon[35941]: pgmap v1500: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:13:54.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:13:53 smithi165 ceph-mon[36302]: pgmap v1501: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:13:54.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:13:53 smithi119 ceph-mon[31518]: pgmap v1501: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:13:54.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:13:53 smithi132 ceph-mon[35941]: pgmap v1501: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:13:56.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:13:55 smithi165 ceph-mon[36302]: pgmap v1502: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:13:56.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:13:55 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2295.694113 secs 2024-03-28T01:13:56.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:13:55 smithi119 ceph-mon[31518]: pgmap v1502: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:13:56.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:13:55 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2295.694113 secs 2024-03-28T01:13:56.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:13:55 smithi132 ceph-mon[35941]: pgmap v1502: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:13:56.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:13:55 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2295.694113 secs 2024-03-28T01:13:58.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:13:57 smithi165 ceph-mon[36302]: pgmap v1503: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:13:58.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:13:57 smithi119 ceph-mon[31518]: pgmap v1503: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:13:58.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:13:57 smithi132 ceph-mon[35941]: pgmap v1503: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:14:00.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:13:59 smithi165 ceph-mon[36302]: pgmap v1504: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:14:00.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:13:59 smithi119 ceph-mon[31518]: pgmap v1504: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:14:00.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:13:59 smithi132 ceph-mon[35941]: pgmap v1504: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:14:01.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:14:00 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2300.694174 secs 2024-03-28T01:14:01.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:14:00 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2300.694174 secs 2024-03-28T01:14:01.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:14:00 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2300.694174 secs 2024-03-28T01:14:02.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:14:01 smithi165 ceph-mon[36302]: pgmap v1505: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:14:02.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:14:01 smithi119 ceph-mon[31518]: pgmap v1505: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:14:02.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:14:01 smithi132 ceph-mon[35941]: pgmap v1505: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:14:04.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:14:03 smithi165 ceph-mon[36302]: pgmap v1506: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:14:04.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:14:03 smithi119 ceph-mon[31518]: pgmap v1506: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:14:04.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:14:03 smithi132 ceph-mon[35941]: pgmap v1506: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:14:06.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:14:05 smithi165 ceph-mon[36302]: pgmap v1507: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:14:06.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:14:05 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2305.694265 secs 2024-03-28T01:14:06.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:14:05 smithi119 ceph-mon[31518]: pgmap v1507: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:14:06.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:14:05 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2305.694265 secs 2024-03-28T01:14:06.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:14:05 smithi132 ceph-mon[35941]: pgmap v1507: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:14:06.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:14:05 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2305.694265 secs 2024-03-28T01:14:08.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:14:07 smithi165 ceph-mon[36302]: pgmap v1508: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:14:08.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:14:07 smithi119 ceph-mon[31518]: pgmap v1508: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:14:08.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:14:07 smithi132 ceph-mon[35941]: pgmap v1508: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:14:10.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:14:09 smithi165 ceph-mon[36302]: pgmap v1509: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:14:10.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:14:09 smithi119 ceph-mon[31518]: pgmap v1509: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:14:10.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:14:09 smithi132 ceph-mon[35941]: pgmap v1509: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:14:11.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:14:10 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2310.694356 secs 2024-03-28T01:14:11.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:14:10 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2310.694356 secs 2024-03-28T01:14:11.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:14:10 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2310.694356 secs 2024-03-28T01:14:12.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:14:11 smithi165 ceph-mon[36302]: pgmap v1510: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:14:12.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:14:11 smithi119 ceph-mon[31518]: pgmap v1510: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:14:12.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:14:11 smithi132 ceph-mon[35941]: pgmap v1510: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:14:14.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:14:13 smithi165 ceph-mon[36302]: pgmap v1511: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:14:14.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:14:13 smithi119 ceph-mon[31518]: pgmap v1511: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:14:14.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:14:13 smithi132 ceph-mon[35941]: pgmap v1511: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:14:16.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:14:15 smithi165 ceph-mon[36302]: pgmap v1512: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:14:16.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:14:15 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2315.694441 secs 2024-03-28T01:14:16.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:14:15 smithi119 ceph-mon[31518]: pgmap v1512: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:14:16.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:14:15 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2315.694441 secs 2024-03-28T01:14:16.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:14:15 smithi132 ceph-mon[35941]: pgmap v1512: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:14:16.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:14:15 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2315.694441 secs 2024-03-28T01:14:18.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:14:17 smithi165 ceph-mon[36302]: pgmap v1513: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:14:18.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:14:17 smithi119 ceph-mon[31518]: pgmap v1513: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:14:18.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:14:17 smithi132 ceph-mon[35941]: pgmap v1513: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:14:20.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:14:19 smithi165 ceph-mon[36302]: pgmap v1514: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:14:20.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:14:19 smithi119 ceph-mon[31518]: pgmap v1514: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:14:20.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:14:19 smithi132 ceph-mon[35941]: pgmap v1514: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:14:21.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:14:20 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2320.694530 secs 2024-03-28T01:14:21.186 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:14:20 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2320.694530 secs 2024-03-28T01:14:21.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:14:20 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2320.694530 secs 2024-03-28T01:14:22.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:14:21 smithi165 ceph-mon[36302]: pgmap v1515: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:14:22.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:14:21 smithi119 ceph-mon[31518]: pgmap v1515: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:14:22.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:14:21 smithi132 ceph-mon[35941]: pgmap v1515: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:14:24.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:14:23 smithi165 ceph-mon[36302]: pgmap v1516: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:14:24.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:14:23 smithi119 ceph-mon[31518]: pgmap v1516: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:14:24.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:14:23 smithi132 ceph-mon[35941]: pgmap v1516: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:14:26.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:14:25 smithi165 ceph-mon[36302]: pgmap v1517: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:14:26.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:14:25 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2325.694621 secs 2024-03-28T01:14:26.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:14:25 smithi119 ceph-mon[31518]: pgmap v1517: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:14:26.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:14:25 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2325.694621 secs 2024-03-28T01:14:26.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:14:25 smithi132 ceph-mon[35941]: pgmap v1517: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:14:26.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:14:25 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2325.694621 secs 2024-03-28T01:14:28.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:14:27 smithi119 ceph-mon[31518]: pgmap v1518: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:14:28.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:14:27 smithi132 ceph-mon[35941]: pgmap v1518: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:14:28.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:14:27 smithi165 ceph-mon[36302]: pgmap v1518: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:14:30.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:14:29 smithi119 ceph-mon[31518]: pgmap v1519: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:14:30.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:14:29 smithi132 ceph-mon[35941]: pgmap v1519: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:14:30.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:14:29 smithi165 ceph-mon[36302]: pgmap v1519: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:14:31.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:14:30 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2330.694711 secs 2024-03-28T01:14:31.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:14:30 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2330.694711 secs 2024-03-28T01:14:31.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:14:30 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2330.694711 secs 2024-03-28T01:14:32.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:14:31 smithi119 ceph-mon[31518]: pgmap v1520: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:14:32.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:14:31 smithi132 ceph-mon[35941]: pgmap v1520: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:14:32.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:14:31 smithi165 ceph-mon[36302]: pgmap v1520: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:14:34.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:14:33 smithi132 ceph-mon[35941]: pgmap v1521: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:14:34.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:14:33 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:14:34.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:14:33 smithi165 ceph-mon[36302]: pgmap v1521: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:14:34.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:14:33 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:14:34.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:14:33 smithi119 ceph-mon[31518]: pgmap v1521: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:14:34.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:14:33 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:14:35.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:14:34 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:14:35.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:14:34 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:14:35.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:14:34 smithi132 ceph-mon[35941]: pgmap v1522: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T01:14:35.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:14:34 smithi132 ceph-mon[35941]: pgmap v1523: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 5 op/s 2024-03-28T01:14:35.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:14:34 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:14:35.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:14:34 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:14:35.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:14:34 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:14:35.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:14:34 smithi165 ceph-mon[36302]: pgmap v1522: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T01:14:35.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:14:34 smithi165 ceph-mon[36302]: pgmap v1523: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 5 op/s 2024-03-28T01:14:35.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:14:34 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:14:35.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:14:34 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:14:35.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:14:34 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:14:35.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:14:34 smithi119 ceph-mon[31518]: pgmap v1522: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T01:14:35.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:14:34 smithi119 ceph-mon[31518]: pgmap v1523: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 5 op/s 2024-03-28T01:14:35.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:14:34 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:14:36.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:14:35 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2335.694753 secs 2024-03-28T01:14:36.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:14:35 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2335.694753 secs 2024-03-28T01:14:36.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:14:35 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2335.694753 secs 2024-03-28T01:14:37.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:14:36 smithi132 ceph-mon[35941]: pgmap v1524: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:14:37.280 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:14:36 smithi165 ceph-mon[36302]: pgmap v1524: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:14:37.303 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:14:36 smithi119 ceph-mon[31518]: pgmap v1524: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:14:39.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:14:39 smithi165 ceph-mon[36302]: pgmap v1525: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:14:39.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:14:39 smithi119 ceph-mon[31518]: pgmap v1525: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:14:39.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:14:39 smithi132 ceph-mon[35941]: pgmap v1525: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:14:41.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:14:41 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2340.694843 secs 2024-03-28T01:14:41.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:14:41 smithi119 ceph-mon[31518]: pgmap v1526: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:14:41.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:14:41 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2340.694843 secs 2024-03-28T01:14:41.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:14:41 smithi165 ceph-mon[36302]: pgmap v1526: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:14:41.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:14:41 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2340.694843 secs 2024-03-28T01:14:41.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:14:41 smithi132 ceph-mon[35941]: pgmap v1526: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:14:43.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:14:43 smithi165 ceph-mon[36302]: pgmap v1527: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:14:43.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:14:43 smithi119 ceph-mon[31518]: pgmap v1527: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:14:43.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:14:43 smithi132 ceph-mon[35941]: pgmap v1527: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:14:45.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:14:45 smithi165 ceph-mon[36302]: pgmap v1528: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T01:14:45.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:14:45 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2345.694930 secs 2024-03-28T01:14:45.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:14:45 smithi119 ceph-mon[31518]: pgmap v1528: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T01:14:45.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:14:45 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2345.694930 secs 2024-03-28T01:14:45.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:14:45 smithi132 ceph-mon[35941]: pgmap v1528: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T01:14:45.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:14:45 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2345.694930 secs 2024-03-28T01:14:47.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:14:47 smithi165 ceph-mon[36302]: pgmap v1529: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-03-28T01:14:47.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:14:47 smithi119 ceph-mon[31518]: pgmap v1529: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-03-28T01:14:47.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:14:47 smithi132 ceph-mon[35941]: pgmap v1529: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-03-28T01:14:48.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:14:48 smithi165 ceph-mon[36302]: pgmap v1530: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:14:48.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:14:48 smithi119 ceph-mon[31518]: pgmap v1530: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:14:48.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:14:48 smithi132 ceph-mon[35941]: pgmap v1530: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:14:51.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:14:51 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2350.694991 secs 2024-03-28T01:14:51.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:14:51 smithi165 ceph-mon[36302]: pgmap v1531: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:14:51.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:14:51 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2350.694991 secs 2024-03-28T01:14:51.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:14:51 smithi119 ceph-mon[31518]: pgmap v1531: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:14:51.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:14:51 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2350.694991 secs 2024-03-28T01:14:51.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:14:51 smithi132 ceph-mon[35941]: pgmap v1531: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:14:53.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:14:53 smithi165 ceph-mon[36302]: pgmap v1532: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:14:53.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:14:53 smithi119 ceph-mon[31518]: pgmap v1532: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:14:53.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:14:53 smithi132 ceph-mon[35941]: pgmap v1532: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:14:55.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:14:55 smithi165 ceph-mon[36302]: pgmap v1533: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-28T01:14:55.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:14:55 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2355.695050 secs 2024-03-28T01:14:55.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:14:55 smithi119 ceph-mon[31518]: pgmap v1533: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-28T01:14:55.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:14:55 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2355.695050 secs 2024-03-28T01:14:55.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:14:55 smithi132 ceph-mon[35941]: pgmap v1533: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-28T01:14:55.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:14:55 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2355.695050 secs 2024-03-28T01:14:57.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:14:57 smithi165 ceph-mon[36302]: pgmap v1534: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-03-28T01:14:57.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:14:57 smithi119 ceph-mon[31518]: pgmap v1534: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-03-28T01:14:57.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:14:57 smithi132 ceph-mon[35941]: pgmap v1534: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-03-28T01:14:58.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:14:58 smithi165 ceph-mon[36302]: pgmap v1535: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:14:58.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:14:58 smithi119 ceph-mon[31518]: pgmap v1535: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:14:58.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:14:58 smithi132 ceph-mon[35941]: pgmap v1535: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:15:01.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:15:01 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2360.695113 secs 2024-03-28T01:15:01.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:15:01 smithi165 ceph-mon[36302]: pgmap v1536: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:15:01.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:15:01 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2360.695113 secs 2024-03-28T01:15:01.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:15:01 smithi119 ceph-mon[31518]: pgmap v1536: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:15:01.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:15:01 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2360.695113 secs 2024-03-28T01:15:01.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:15:01 smithi132 ceph-mon[35941]: pgmap v1536: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:15:02.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:15:02 smithi165 ceph-mon[36302]: pgmap v1537: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:15:02.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:15:02 smithi119 ceph-mon[31518]: pgmap v1537: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:15:02.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:15:02 smithi132 ceph-mon[35941]: pgmap v1537: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:15:05.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:15:05 smithi165 ceph-mon[36302]: pgmap v1538: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-28T01:15:05.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:15:05 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2365.695177 secs 2024-03-28T01:15:05.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:15:05 smithi119 ceph-mon[31518]: pgmap v1538: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-28T01:15:05.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:15:05 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2365.695177 secs 2024-03-28T01:15:05.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:15:05 smithi132 ceph-mon[35941]: pgmap v1538: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-28T01:15:05.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:15:05 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2365.695177 secs 2024-03-28T01:15:07.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:15:07 smithi165 ceph-mon[36302]: pgmap v1539: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-28T01:15:07.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:15:07 smithi119 ceph-mon[31518]: pgmap v1539: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-28T01:15:07.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:15:07 smithi132 ceph-mon[35941]: pgmap v1539: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-28T01:15:09.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:15:09 smithi165 ceph-mon[36302]: pgmap v1540: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:15:09.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:15:09 smithi119 ceph-mon[31518]: pgmap v1540: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:15:09.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:15:09 smithi132 ceph-mon[35941]: pgmap v1540: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:15:10.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:15:10 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2370.695240 secs 2024-03-28T01:15:10.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:15:10 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2370.695240 secs 2024-03-28T01:15:10.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:15:10 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2370.695240 secs 2024-03-28T01:15:11.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:15:11 smithi165 ceph-mon[36302]: pgmap v1541: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:15:11.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:15:11 smithi119 ceph-mon[31518]: pgmap v1541: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:15:11.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:15:11 smithi132 ceph-mon[35941]: pgmap v1541: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:15:12.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:15:12 smithi165 ceph-mon[36302]: pgmap v1542: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:15:12.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:15:12 smithi119 ceph-mon[31518]: pgmap v1542: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:15:12.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:15:12 smithi132 ceph-mon[35941]: pgmap v1542: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:15:15.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:15:15 smithi165 ceph-mon[36302]: pgmap v1543: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-28T01:15:15.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:15:15 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2375.695329 secs 2024-03-28T01:15:15.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:15:15 smithi119 ceph-mon[31518]: pgmap v1543: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-28T01:15:15.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:15:15 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2375.695329 secs 2024-03-28T01:15:15.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:15:15 smithi132 ceph-mon[35941]: pgmap v1543: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-28T01:15:15.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:15:15 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2375.695329 secs 2024-03-28T01:15:17.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:15:17 smithi165 ceph-mon[36302]: pgmap v1544: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-28T01:15:17.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:15:17 smithi119 ceph-mon[31518]: pgmap v1544: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-28T01:15:17.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:15:17 smithi132 ceph-mon[35941]: pgmap v1544: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-28T01:15:19.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:15:19 smithi165 ceph-mon[36302]: pgmap v1545: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:15:19.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:15:19 smithi119 ceph-mon[31518]: pgmap v1545: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:15:19.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:15:19 smithi132 ceph-mon[35941]: pgmap v1545: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:15:20.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:15:20 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2380.695420 secs 2024-03-28T01:15:20.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:15:20 smithi165 ceph-mon[36302]: pgmap v1546: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:15:20.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:15:20 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2380.695420 secs 2024-03-28T01:15:20.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:15:20 smithi119 ceph-mon[31518]: pgmap v1546: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:15:20.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:15:20 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2380.695420 secs 2024-03-28T01:15:20.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:15:20 smithi132 ceph-mon[35941]: pgmap v1546: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:15:23.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:15:23 smithi165 ceph-mon[36302]: pgmap v1547: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:15:23.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:15:23 smithi119 ceph-mon[31518]: pgmap v1547: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:15:23.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:15:23 smithi132 ceph-mon[35941]: pgmap v1547: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:15:25.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:15:25 smithi165 ceph-mon[36302]: pgmap v1548: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-28T01:15:25.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:15:25 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2385.695509 secs 2024-03-28T01:15:25.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:15:25 smithi119 ceph-mon[31518]: pgmap v1548: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-28T01:15:25.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:15:25 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2385.695509 secs 2024-03-28T01:15:25.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:15:25 smithi132 ceph-mon[35941]: pgmap v1548: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-28T01:15:25.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:15:25 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2385.695509 secs 2024-03-28T01:15:26.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:15:26 smithi165 ceph-mon[36302]: pgmap v1549: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-28T01:15:26.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:15:26 smithi119 ceph-mon[31518]: pgmap v1549: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-28T01:15:26.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:15:26 smithi132 ceph-mon[35941]: pgmap v1549: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-28T01:15:29.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:15:29 smithi165 ceph-mon[36302]: pgmap v1550: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:15:29.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:15:29 smithi119 ceph-mon[31518]: pgmap v1550: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:15:29.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:15:29 smithi132 ceph-mon[35941]: pgmap v1550: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:15:30.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:15:30 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2390.695596 secs 2024-03-28T01:15:30.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:15:30 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2390.695596 secs 2024-03-28T01:15:30.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:15:30 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2390.695596 secs 2024-03-28T01:15:31.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:15:31 smithi165 ceph-mon[36302]: pgmap v1551: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:15:31.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:15:31 smithi119 ceph-mon[31518]: pgmap v1551: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:15:31.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:15:31 smithi132 ceph-mon[35941]: pgmap v1551: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:15:32.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:15:32 smithi165 ceph-mon[36302]: pgmap v1552: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:15:32.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:15:32 smithi119 ceph-mon[31518]: pgmap v1552: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:15:32.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:15:32 smithi132 ceph-mon[35941]: pgmap v1552: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:15:35.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:15:35 smithi165 ceph-mon[36302]: pgmap v1553: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-28T01:15:35.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:15:35 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:15:35.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:15:35 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2395.695668 secs 2024-03-28T01:15:35.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:15:35 smithi119 ceph-mon[31518]: pgmap v1553: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-28T01:15:35.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:15:35 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:15:35.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:15:35 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2395.695668 secs 2024-03-28T01:15:35.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:15:35 smithi132 ceph-mon[35941]: pgmap v1553: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-28T01:15:35.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:15:35 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:15:35.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:15:35 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2395.695668 secs 2024-03-28T01:15:36.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:15:36 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:15:36.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:15:36 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:15:36.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:15:36 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:15:36.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:15:36 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:15:36.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:15:36 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:15:36.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:15:36 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:15:36.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:15:36 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:15:36.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:15:36 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:15:36.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:15:36 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:15:37.530 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:15:37 smithi165 ceph-mon[36302]: pgmap v1554: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T01:15:37.530 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:15:37 smithi165 ceph-mon[36302]: pgmap v1555: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:15:37.553 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:15:37 smithi119 ceph-mon[31518]: pgmap v1554: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T01:15:37.553 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:15:37 smithi119 ceph-mon[31518]: pgmap v1555: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:15:37.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:15:37 smithi132 ceph-mon[35941]: pgmap v1554: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T01:15:37.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:15:37 smithi132 ceph-mon[35941]: pgmap v1555: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:15:38.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:15:38 smithi165 ceph-mon[36302]: pgmap v1556: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:15:38.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:15:38 smithi119 ceph-mon[31518]: pgmap v1556: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:15:38.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:15:38 smithi132 ceph-mon[35941]: pgmap v1556: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:15:41.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:15:40 smithi165 ceph-mon[36302]: pgmap v1557: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:15:41.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:15:40 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2400.695730 secs 2024-03-28T01:15:41.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:15:40 smithi119 ceph-mon[31518]: pgmap v1557: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:15:41.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:15:40 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2400.695730 secs 2024-03-28T01:15:41.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:15:40 smithi132 ceph-mon[35941]: pgmap v1557: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:15:41.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:15:40 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2400.695730 secs 2024-03-28T01:15:43.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:15:42 smithi165 ceph-mon[36302]: pgmap v1558: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:15:43.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:15:42 smithi119 ceph-mon[31518]: pgmap v1558: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:15:43.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:15:42 smithi132 ceph-mon[35941]: pgmap v1558: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:15:45.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:15:44 smithi165 ceph-mon[36302]: pgmap v1559: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-28T01:15:45.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:15:44 smithi119 ceph-mon[31518]: pgmap v1559: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-28T01:15:45.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:15:44 smithi132 ceph-mon[35941]: pgmap v1559: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-28T01:15:46.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:15:45 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2405.695790 secs 2024-03-28T01:15:46.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:15:45 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2405.695790 secs 2024-03-28T01:15:46.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:15:45 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2405.695790 secs 2024-03-28T01:15:47.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:15:46 smithi165 ceph-mon[36302]: pgmap v1560: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T01:15:47.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:15:46 smithi119 ceph-mon[31518]: pgmap v1560: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T01:15:47.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:15:46 smithi132 ceph-mon[35941]: pgmap v1560: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T01:15:49.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:15:48 smithi165 ceph-mon[36302]: pgmap v1561: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:15:49.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:15:48 smithi119 ceph-mon[31518]: pgmap v1561: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:15:49.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:15:48 smithi132 ceph-mon[35941]: pgmap v1561: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:15:51.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:15:50 smithi165 ceph-mon[36302]: pgmap v1562: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:15:51.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:15:50 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2410.695881 secs 2024-03-28T01:15:51.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:15:50 smithi119 ceph-mon[31518]: pgmap v1562: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:15:51.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:15:50 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2410.695881 secs 2024-03-28T01:15:51.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:15:50 smithi132 ceph-mon[35941]: pgmap v1562: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:15:51.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:15:50 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2410.695881 secs 2024-03-28T01:15:53.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:15:52 smithi165 ceph-mon[36302]: pgmap v1563: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:15:53.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:15:52 smithi119 ceph-mon[31518]: pgmap v1563: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:15:53.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:15:52 smithi132 ceph-mon[35941]: pgmap v1563: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:15:55.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:15:54 smithi165 ceph-mon[36302]: pgmap v1564: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:15:55.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:15:54 smithi119 ceph-mon[31518]: pgmap v1564: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:15:55.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:15:54 smithi132 ceph-mon[35941]: pgmap v1564: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:15:56.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:15:55 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2415.695898 secs 2024-03-28T01:15:56.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:15:55 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2415.695898 secs 2024-03-28T01:15:56.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:15:55 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2415.695898 secs 2024-03-28T01:15:57.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:15:56 smithi165 ceph-mon[36302]: pgmap v1565: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:15:57.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:15:56 smithi119 ceph-mon[31518]: pgmap v1565: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:15:57.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:15:56 smithi132 ceph-mon[35941]: pgmap v1565: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:15:59.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:15:58 smithi165 ceph-mon[36302]: pgmap v1566: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:15:59.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:15:58 smithi119 ceph-mon[31518]: pgmap v1566: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:15:59.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:15:58 smithi132 ceph-mon[35941]: pgmap v1566: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:16:01.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:16:00 smithi165 ceph-mon[36302]: pgmap v1567: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:16:01.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:16:00 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2420.695957 secs 2024-03-28T01:16:01.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:16:00 smithi119 ceph-mon[31518]: pgmap v1567: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:16:01.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:16:00 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2420.695957 secs 2024-03-28T01:16:01.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:16:00 smithi132 ceph-mon[35941]: pgmap v1567: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:16:01.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:16:00 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2420.695957 secs 2024-03-28T01:16:02.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:16:02 smithi165 ceph-mon[36302]: pgmap v1568: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:16:02.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:16:02 smithi119 ceph-mon[31518]: pgmap v1568: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:16:02.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:16:02 smithi132 ceph-mon[35941]: pgmap v1568: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:16:05.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:16:04 smithi165 ceph-mon[36302]: pgmap v1569: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:16:05.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:16:04 smithi119 ceph-mon[31518]: pgmap v1569: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:16:05.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:16:04 smithi132 ceph-mon[35941]: pgmap v1569: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:16:06.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:16:05 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2425.696021 secs 2024-03-28T01:16:06.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:16:05 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2425.696021 secs 2024-03-28T01:16:06.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:16:05 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2425.696021 secs 2024-03-28T01:16:07.053 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:16:06 smithi119 ceph-mon[31518]: pgmap v1570: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:16:07.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:16:06 smithi165 ceph-mon[36302]: pgmap v1570: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:16:07.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:16:06 smithi132 ceph-mon[35941]: pgmap v1570: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:16:09.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:16:08 smithi165 ceph-mon[36302]: pgmap v1571: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:16:09.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:16:08 smithi119 ceph-mon[31518]: pgmap v1571: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:16:09.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:16:08 smithi132 ceph-mon[35941]: pgmap v1571: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:16:11.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:16:10 smithi165 ceph-mon[36302]: pgmap v1572: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:16:11.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:16:10 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2430.696036 secs 2024-03-28T01:16:11.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:16:10 smithi119 ceph-mon[31518]: pgmap v1572: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:16:11.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:16:10 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2430.696036 secs 2024-03-28T01:16:11.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:16:10 smithi132 ceph-mon[35941]: pgmap v1572: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:16:11.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:16:10 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2430.696036 secs 2024-03-28T01:16:13.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:16:12 smithi165 ceph-mon[36302]: pgmap v1573: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:16:13.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:16:12 smithi119 ceph-mon[31518]: pgmap v1573: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:16:13.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:16:12 smithi132 ceph-mon[35941]: pgmap v1573: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:16:15.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:16:14 smithi165 ceph-mon[36302]: pgmap v1574: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:16:15.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:16:14 smithi119 ceph-mon[31518]: pgmap v1574: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:16:15.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:16:14 smithi132 ceph-mon[35941]: pgmap v1574: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:16:16.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:16:15 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2435.696099 secs 2024-03-28T01:16:16.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:16:15 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2435.696099 secs 2024-03-28T01:16:16.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:16:15 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2435.696099 secs 2024-03-28T01:16:17.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:16:16 smithi165 ceph-mon[36302]: pgmap v1575: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:16:17.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:16:16 smithi119 ceph-mon[31518]: pgmap v1575: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:16:17.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:16:16 smithi132 ceph-mon[35941]: pgmap v1575: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:16:19.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:16:18 smithi165 ceph-mon[36302]: pgmap v1576: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:16:19.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:16:18 smithi119 ceph-mon[31518]: pgmap v1576: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:16:19.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:16:18 smithi132 ceph-mon[35941]: pgmap v1576: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:16:21.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:16:20 smithi165 ceph-mon[36302]: pgmap v1577: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:16:21.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:16:20 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2440.696169 secs 2024-03-28T01:16:21.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:16:20 smithi119 ceph-mon[31518]: pgmap v1577: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:16:21.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:16:20 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2440.696169 secs 2024-03-28T01:16:21.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:16:20 smithi132 ceph-mon[35941]: pgmap v1577: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:16:21.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:16:20 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2440.696169 secs 2024-03-28T01:16:23.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:16:22 smithi165 ceph-mon[36302]: pgmap v1578: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:16:23.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:16:22 smithi119 ceph-mon[31518]: pgmap v1578: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:16:23.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:16:22 smithi132 ceph-mon[35941]: pgmap v1578: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:16:25.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:16:24 smithi165 ceph-mon[36302]: pgmap v1579: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:16:25.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:16:24 smithi119 ceph-mon[31518]: pgmap v1579: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:16:25.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:16:24 smithi132 ceph-mon[35941]: pgmap v1579: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:16:26.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:16:25 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2445.696231 secs 2024-03-28T01:16:26.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:16:25 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2445.696231 secs 2024-03-28T01:16:26.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:16:25 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2445.696231 secs 2024-03-28T01:16:27.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:16:26 smithi165 ceph-mon[36302]: pgmap v1580: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:16:27.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:16:26 smithi119 ceph-mon[31518]: pgmap v1580: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:16:27.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:16:26 smithi132 ceph-mon[35941]: pgmap v1580: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:16:29.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:16:28 smithi165 ceph-mon[36302]: pgmap v1581: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:16:29.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:16:28 smithi119 ceph-mon[31518]: pgmap v1581: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:16:29.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:16:28 smithi132 ceph-mon[35941]: pgmap v1581: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:16:31.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:16:30 smithi165 ceph-mon[36302]: pgmap v1582: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:16:31.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:16:30 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2450.696322 secs 2024-03-28T01:16:31.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:16:30 smithi119 ceph-mon[31518]: pgmap v1582: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:16:31.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:16:30 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2450.696322 secs 2024-03-28T01:16:31.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:16:30 smithi132 ceph-mon[35941]: pgmap v1582: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:16:31.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:16:30 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2450.696322 secs 2024-03-28T01:16:33.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:16:32 smithi165 ceph-mon[36302]: pgmap v1583: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:16:33.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:16:32 smithi119 ceph-mon[31518]: pgmap v1583: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:16:33.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:16:32 smithi132 ceph-mon[35941]: pgmap v1583: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:16:35.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:16:34 smithi165 ceph-mon[36302]: pgmap v1584: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:16:35.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:16:34 smithi119 ceph-mon[31518]: pgmap v1584: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:16:35.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:16:34 smithi132 ceph-mon[35941]: pgmap v1584: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:16:36.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:16:35 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2455.696335 secs 2024-03-28T01:16:36.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:16:35 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2455.696335 secs 2024-03-28T01:16:36.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:16:35 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2455.696335 secs 2024-03-28T01:16:37.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:16:36 smithi119 ceph-mon[31518]: pgmap v1585: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:16:37.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:16:36 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:16:37.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:16:36 smithi132 ceph-mon[35941]: pgmap v1585: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:16:37.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:16:36 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:16:37.279 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:16:36 smithi165 ceph-mon[36302]: pgmap v1585: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:16:37.279 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:16:36 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:16:38.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:16:37 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:16:38.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:16:37 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:16:38.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:16:37 smithi119 ceph-mon[31518]: pgmap v1586: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T01:16:38.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:16:37 smithi119 ceph-mon[31518]: pgmap v1587: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:16:38.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:16:37 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:16:38.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:16:37 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:16:38.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:16:37 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:16:38.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:16:37 smithi132 ceph-mon[35941]: pgmap v1586: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T01:16:38.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:16:37 smithi132 ceph-mon[35941]: pgmap v1587: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:16:38.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:16:37 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:16:38.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:16:37 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:16:38.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:16:37 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:16:38.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:16:37 smithi165 ceph-mon[36302]: pgmap v1586: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T01:16:38.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:16:37 smithi165 ceph-mon[36302]: pgmap v1587: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:16:38.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:16:37 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:16:39.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:16:39 smithi165 ceph-mon[36302]: pgmap v1588: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:16:39.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:16:39 smithi119 ceph-mon[31518]: pgmap v1588: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:16:39.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:16:39 smithi132 ceph-mon[35941]: pgmap v1588: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:16:40.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:16:40 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2460.696396 secs 2024-03-28T01:16:40.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:16:40 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2460.696396 secs 2024-03-28T01:16:40.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:16:40 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2460.696396 secs 2024-03-28T01:16:41.599 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:16:41 smithi132 ceph-mon[35941]: pgmap v1589: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:16:41.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:16:41 smithi165 ceph-mon[36302]: pgmap v1589: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:16:41.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:16:41 smithi119 ceph-mon[31518]: pgmap v1589: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:16:43.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:16:43 smithi165 ceph-mon[36302]: pgmap v1590: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:16:43.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:16:43 smithi119 ceph-mon[31518]: pgmap v1590: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:16:43.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:16:43 smithi132 ceph-mon[35941]: pgmap v1590: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:16:45.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:16:45 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2465.696487 secs 2024-03-28T01:16:45.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:16:45 smithi165 ceph-mon[36302]: pgmap v1591: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:16:45.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:16:45 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2465.696487 secs 2024-03-28T01:16:45.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:16:45 smithi119 ceph-mon[31518]: pgmap v1591: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:16:45.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:16:45 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2465.696487 secs 2024-03-28T01:16:45.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:16:45 smithi132 ceph-mon[35941]: pgmap v1591: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:16:47.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:16:47 smithi165 ceph-mon[36302]: pgmap v1592: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T01:16:47.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:16:47 smithi119 ceph-mon[31518]: pgmap v1592: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T01:16:47.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:16:47 smithi132 ceph-mon[35941]: pgmap v1592: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T01:16:49.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:16:49 smithi165 ceph-mon[36302]: pgmap v1593: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:16:49.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:16:49 smithi119 ceph-mon[31518]: pgmap v1593: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:16:49.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:16:49 smithi132 ceph-mon[35941]: pgmap v1593: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:16:50.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:16:50 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2470.696549 secs 2024-03-28T01:16:50.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:16:50 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2470.696549 secs 2024-03-28T01:16:50.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:16:50 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2470.696549 secs 2024-03-28T01:16:51.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:16:51 smithi165 ceph-mon[36302]: pgmap v1594: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:16:51.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:16:51 smithi119 ceph-mon[31518]: pgmap v1594: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:16:51.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:16:51 smithi132 ceph-mon[35941]: pgmap v1594: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:16:53.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:16:53 smithi165 ceph-mon[36302]: pgmap v1595: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:16:53.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:16:53 smithi119 ceph-mon[31518]: pgmap v1595: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:16:53.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:16:53 smithi132 ceph-mon[35941]: pgmap v1595: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:16:55.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:16:55 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2475.696613 secs 2024-03-28T01:16:55.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:16:55 smithi165 ceph-mon[36302]: pgmap v1596: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:16:55.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:16:55 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2475.696613 secs 2024-03-28T01:16:55.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:16:55 smithi119 ceph-mon[31518]: pgmap v1596: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:16:55.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:16:55 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2475.696613 secs 2024-03-28T01:16:55.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:16:55 smithi132 ceph-mon[35941]: pgmap v1596: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:16:57.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:16:57 smithi165 ceph-mon[36302]: pgmap v1597: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:16:57.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:16:57 smithi119 ceph-mon[31518]: pgmap v1597: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:16:57.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:16:57 smithi132 ceph-mon[35941]: pgmap v1597: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:16:59.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:16:59 smithi165 ceph-mon[36302]: pgmap v1598: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:16:59.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:16:59 smithi119 ceph-mon[31518]: pgmap v1598: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:16:59.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:16:59 smithi132 ceph-mon[35941]: pgmap v1598: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:17:00.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:17:00 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2480.696676 secs 2024-03-28T01:17:00.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:17:00 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2480.696676 secs 2024-03-28T01:17:00.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:17:00 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2480.696676 secs 2024-03-28T01:17:01.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:17:01 smithi165 ceph-mon[36302]: pgmap v1599: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:17:01.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:17:01 smithi119 ceph-mon[31518]: pgmap v1599: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:17:01.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:17:01 smithi132 ceph-mon[35941]: pgmap v1599: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:17:03.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:17:03 smithi165 ceph-mon[36302]: pgmap v1600: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:17:03.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:17:03 smithi119 ceph-mon[31518]: pgmap v1600: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:17:03.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:17:03 smithi132 ceph-mon[35941]: pgmap v1600: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:17:05.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:17:05 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2485.696735 secs 2024-03-28T01:17:05.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:17:05 smithi165 ceph-mon[36302]: pgmap v1601: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:17:05.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:17:05 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2485.696735 secs 2024-03-28T01:17:05.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:17:05 smithi119 ceph-mon[31518]: pgmap v1601: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:17:05.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:17:05 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2485.696735 secs 2024-03-28T01:17:05.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:17:05 smithi132 ceph-mon[35941]: pgmap v1601: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:17:07.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:17:07 smithi165 ceph-mon[36302]: pgmap v1602: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:17:07.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:17:07 smithi119 ceph-mon[31518]: pgmap v1602: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:17:07.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:17:07 smithi132 ceph-mon[35941]: pgmap v1602: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:17:09.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:17:09 smithi165 ceph-mon[36302]: pgmap v1603: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:17:09.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:17:09 smithi119 ceph-mon[31518]: pgmap v1603: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:17:09.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:17:09 smithi132 ceph-mon[35941]: pgmap v1603: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:17:10.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:17:10 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2490.696750 secs 2024-03-28T01:17:10.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:17:10 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2490.696750 secs 2024-03-28T01:17:10.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:17:10 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2490.696750 secs 2024-03-28T01:17:11.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:17:11 smithi165 ceph-mon[36302]: pgmap v1604: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:17:11.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:17:11 smithi119 ceph-mon[31518]: pgmap v1604: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:17:11.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:17:11 smithi132 ceph-mon[35941]: pgmap v1604: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:17:13.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:17:13 smithi165 ceph-mon[36302]: pgmap v1605: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:17:13.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:17:13 smithi119 ceph-mon[31518]: pgmap v1605: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:17:13.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:17:13 smithi132 ceph-mon[35941]: pgmap v1605: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:17:15.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:17:15 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2495.696810 secs 2024-03-28T01:17:15.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:17:15 smithi165 ceph-mon[36302]: pgmap v1606: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:17:15.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:17:15 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2495.696810 secs 2024-03-28T01:17:15.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:17:15 smithi119 ceph-mon[31518]: pgmap v1606: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:17:15.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:17:15 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2495.696810 secs 2024-03-28T01:17:15.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:17:15 smithi132 ceph-mon[35941]: pgmap v1606: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:17:17.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:17:17 smithi165 ceph-mon[36302]: pgmap v1607: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:17:17.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:17:17 smithi119 ceph-mon[31518]: pgmap v1607: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:17:17.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:17:17 smithi132 ceph-mon[35941]: pgmap v1607: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:17:19.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:17:19 smithi165 ceph-mon[36302]: pgmap v1608: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:17:19.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:17:19 smithi119 ceph-mon[31518]: pgmap v1608: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:17:19.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:17:19 smithi132 ceph-mon[35941]: pgmap v1608: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:17:20.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:17:20 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2500.696898 secs 2024-03-28T01:17:20.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:17:20 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2500.696898 secs 2024-03-28T01:17:20.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:17:20 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2500.696898 secs 2024-03-28T01:17:21.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:17:21 smithi165 ceph-mon[36302]: pgmap v1609: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:17:21.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:17:21 smithi119 ceph-mon[31518]: pgmap v1609: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:17:21.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:17:21 smithi132 ceph-mon[35941]: pgmap v1609: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:17:23.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:17:23 smithi165 ceph-mon[36302]: pgmap v1610: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:17:23.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:17:23 smithi119 ceph-mon[31518]: pgmap v1610: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:17:23.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:17:23 smithi132 ceph-mon[35941]: pgmap v1610: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:17:25.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:17:25 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2505.696960 secs 2024-03-28T01:17:25.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:17:25 smithi165 ceph-mon[36302]: pgmap v1611: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:17:25.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:17:25 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2505.696960 secs 2024-03-28T01:17:25.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:17:25 smithi119 ceph-mon[31518]: pgmap v1611: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:17:25.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:17:25 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2505.696960 secs 2024-03-28T01:17:25.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:17:25 smithi132 ceph-mon[35941]: pgmap v1611: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:17:27.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:17:27 smithi165 ceph-mon[36302]: pgmap v1612: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:17:27.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:17:27 smithi119 ceph-mon[31518]: pgmap v1612: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:17:27.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:17:27 smithi132 ceph-mon[35941]: pgmap v1612: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:17:29.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:17:29 smithi165 ceph-mon[36302]: pgmap v1613: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:17:29.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:17:29 smithi119 ceph-mon[31518]: pgmap v1613: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:17:29.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:17:29 smithi132 ceph-mon[35941]: pgmap v1613: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:17:30.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:17:30 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2510.697022 secs 2024-03-28T01:17:30.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:17:30 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2510.697022 secs 2024-03-28T01:17:30.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:17:30 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2510.697022 secs 2024-03-28T01:17:31.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:17:31 smithi165 ceph-mon[36302]: pgmap v1614: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:17:31.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:17:31 smithi119 ceph-mon[31518]: pgmap v1614: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:17:31.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:17:31 smithi132 ceph-mon[35941]: pgmap v1614: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:17:33.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:17:33 smithi165 ceph-mon[36302]: pgmap v1615: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:17:33.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:17:33 smithi119 ceph-mon[31518]: pgmap v1615: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:17:33.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:17:33 smithi132 ceph-mon[35941]: pgmap v1615: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:17:35.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:17:35 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2515.697110 secs 2024-03-28T01:17:35.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:17:35 smithi165 ceph-mon[36302]: pgmap v1616: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:17:35.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:17:35 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2515.697110 secs 2024-03-28T01:17:35.693 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:17:35 smithi119 ceph-mon[31518]: pgmap v1616: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:17:35.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:17:35 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2515.697110 secs 2024-03-28T01:17:35.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:17:35 smithi132 ceph-mon[35941]: pgmap v1616: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:17:37.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:17:37 smithi165 ceph-mon[36302]: pgmap v1617: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:17:37.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:17:37 smithi119 ceph-mon[31518]: pgmap v1617: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:17:37.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:17:37 smithi132 ceph-mon[35941]: pgmap v1617: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:17:38.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:17:38 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:17:38.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:17:38 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:17:38.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:17:38 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:17:39.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:17:39 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:17:39.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:17:39 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:17:39.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:17:39 smithi165 ceph-mon[36302]: pgmap v1618: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T01:17:39.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:17:39 smithi165 ceph-mon[36302]: pgmap v1619: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-28T01:17:39.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:17:39 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:17:39.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:17:39 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:17:39.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:17:39 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:17:39.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:17:39 smithi119 ceph-mon[31518]: pgmap v1618: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T01:17:39.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:17:39 smithi119 ceph-mon[31518]: pgmap v1619: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-28T01:17:39.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:17:39 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:17:39.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:17:39 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:17:39.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:17:39 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:17:39.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:17:39 smithi132 ceph-mon[35941]: pgmap v1618: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T01:17:39.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:17:39 smithi132 ceph-mon[35941]: pgmap v1619: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-28T01:17:39.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:17:39 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:17:40.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:17:40 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2520.697172 secs 2024-03-28T01:17:40.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:17:40 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2520.697172 secs 2024-03-28T01:17:40.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:17:40 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2520.697172 secs 2024-03-28T01:17:41.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:17:41 smithi165 ceph-mon[36302]: pgmap v1620: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:17:41.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:17:41 smithi119 ceph-mon[31518]: pgmap v1620: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:17:41.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:17:41 smithi132 ceph-mon[35941]: pgmap v1620: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:17:44.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:17:43 smithi165 ceph-mon[36302]: pgmap v1621: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:17:44.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:17:43 smithi119 ceph-mon[31518]: pgmap v1621: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:17:44.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:17:43 smithi132 ceph-mon[35941]: pgmap v1621: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:17:46.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:17:45 smithi165 ceph-mon[36302]: pgmap v1622: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:17:46.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:17:45 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2525.697234 secs 2024-03-28T01:17:46.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:17:45 smithi119 ceph-mon[31518]: pgmap v1622: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:17:46.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:17:45 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2525.697234 secs 2024-03-28T01:17:46.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:17:45 smithi132 ceph-mon[35941]: pgmap v1622: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:17:46.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:17:45 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2525.697234 secs 2024-03-28T01:17:48.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:17:47 smithi165 ceph-mon[36302]: pgmap v1623: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:17:48.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:17:47 smithi119 ceph-mon[31518]: pgmap v1623: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:17:48.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:17:47 smithi132 ceph-mon[35941]: pgmap v1623: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:17:50.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:17:49 smithi165 ceph-mon[36302]: pgmap v1624: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T01:17:50.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:17:49 smithi119 ceph-mon[31518]: pgmap v1624: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T01:17:50.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:17:49 smithi132 ceph-mon[35941]: pgmap v1624: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T01:17:51.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:17:50 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2530.697274 secs 2024-03-28T01:17:51.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:17:50 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2530.697274 secs 2024-03-28T01:17:51.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:17:50 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2530.697274 secs 2024-03-28T01:17:52.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:17:51 smithi165 ceph-mon[36302]: pgmap v1625: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:17:52.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:17:51 smithi119 ceph-mon[31518]: pgmap v1625: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:17:52.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:17:51 smithi132 ceph-mon[35941]: pgmap v1625: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:17:54.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:17:53 smithi165 ceph-mon[36302]: pgmap v1626: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:17:54.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:17:53 smithi119 ceph-mon[31518]: pgmap v1626: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:17:54.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:17:53 smithi132 ceph-mon[35941]: pgmap v1626: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:17:56.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:17:55 smithi165 ceph-mon[36302]: pgmap v1627: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:17:56.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:17:55 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2535.697313 secs 2024-03-28T01:17:56.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:17:55 smithi119 ceph-mon[31518]: pgmap v1627: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:17:56.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:17:55 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2535.697313 secs 2024-03-28T01:17:56.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:17:55 smithi132 ceph-mon[35941]: pgmap v1627: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:17:56.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:17:55 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2535.697313 secs 2024-03-28T01:17:58.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:17:57 smithi165 ceph-mon[36302]: pgmap v1628: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:17:58.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:17:57 smithi119 ceph-mon[31518]: pgmap v1628: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:17:58.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:17:57 smithi132 ceph-mon[35941]: pgmap v1628: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:18:00.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:17:59 smithi165 ceph-mon[36302]: pgmap v1629: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:18:00.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:17:59 smithi119 ceph-mon[31518]: pgmap v1629: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:18:00.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:17:59 smithi132 ceph-mon[35941]: pgmap v1629: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:18:01.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:18:00 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2540.697403 secs 2024-03-28T01:18:01.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:18:00 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2540.697403 secs 2024-03-28T01:18:01.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:18:00 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2540.697403 secs 2024-03-28T01:18:02.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:18:01 smithi165 ceph-mon[36302]: pgmap v1630: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:18:02.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:18:01 smithi119 ceph-mon[31518]: pgmap v1630: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:18:02.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:18:01 smithi132 ceph-mon[35941]: pgmap v1630: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:18:04.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:18:03 smithi165 ceph-mon[36302]: pgmap v1631: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:18:04.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:18:03 smithi119 ceph-mon[31518]: pgmap v1631: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:18:04.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:18:03 smithi132 ceph-mon[35941]: pgmap v1631: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:18:06.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:18:05 smithi165 ceph-mon[36302]: pgmap v1632: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:18:06.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:18:05 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2545.697489 secs 2024-03-28T01:18:06.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:18:05 smithi119 ceph-mon[31518]: pgmap v1632: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:18:06.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:18:05 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2545.697489 secs 2024-03-28T01:18:06.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:18:05 smithi132 ceph-mon[35941]: pgmap v1632: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:18:06.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:18:05 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2545.697489 secs 2024-03-28T01:18:08.127 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:18:07 smithi165 ceph-mon[36302]: pgmap v1633: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:18:08.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:18:07 smithi119 ceph-mon[31518]: pgmap v1633: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:18:08.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:18:07 smithi132 ceph-mon[35941]: pgmap v1633: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:18:10.127 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:18:09 smithi165 ceph-mon[36302]: pgmap v1634: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:18:10.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:18:09 smithi119 ceph-mon[31518]: pgmap v1634: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:18:10.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:18:09 smithi132 ceph-mon[35941]: pgmap v1634: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:18:11.127 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:18:10 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2550.697577 secs 2024-03-28T01:18:11.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:18:10 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2550.697577 secs 2024-03-28T01:18:11.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:18:10 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2550.697577 secs 2024-03-28T01:18:12.127 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:18:11 smithi165 ceph-mon[36302]: pgmap v1635: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:18:12.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:18:11 smithi119 ceph-mon[31518]: pgmap v1635: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:18:12.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:18:11 smithi132 ceph-mon[35941]: pgmap v1635: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:18:14.127 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:18:13 smithi165 ceph-mon[36302]: pgmap v1636: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:18:14.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:18:13 smithi119 ceph-mon[31518]: pgmap v1636: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:18:14.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:18:13 smithi132 ceph-mon[35941]: pgmap v1636: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:18:16.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:18:15 smithi165 ceph-mon[36302]: pgmap v1637: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:18:16.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:18:15 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2555.697666 secs 2024-03-28T01:18:16.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:18:15 smithi119 ceph-mon[31518]: pgmap v1637: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:18:16.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:18:15 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2555.697666 secs 2024-03-28T01:18:16.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:18:15 smithi132 ceph-mon[35941]: pgmap v1637: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:18:16.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:18:15 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2555.697666 secs 2024-03-28T01:18:18.127 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:18:17 smithi165 ceph-mon[36302]: pgmap v1638: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:18:18.145 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:18:17 smithi119 ceph-mon[31518]: pgmap v1638: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:18:18.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:18:17 smithi132 ceph-mon[35941]: pgmap v1638: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:18:20.127 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:18:19 smithi165 ceph-mon[36302]: pgmap v1639: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:18:20.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:18:19 smithi119 ceph-mon[31518]: pgmap v1639: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:18:20.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:18:19 smithi132 ceph-mon[35941]: pgmap v1639: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:18:21.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:18:20 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2560.697757 secs 2024-03-28T01:18:21.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:18:20 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2560.697757 secs 2024-03-28T01:18:21.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:18:20 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2560.697757 secs 2024-03-28T01:18:22.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:18:21 smithi119 ceph-mon[31518]: pgmap v1640: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:18:22.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:18:21 smithi132 ceph-mon[35941]: pgmap v1640: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:18:22.377 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:18:21 smithi165 ceph-mon[36302]: pgmap v1640: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:18:24.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:18:23 smithi119 ceph-mon[31518]: pgmap v1641: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:18:24.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:18:23 smithi132 ceph-mon[35941]: pgmap v1641: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:18:24.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:18:23 smithi165 ceph-mon[36302]: pgmap v1641: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:18:26.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:18:25 smithi119 ceph-mon[31518]: pgmap v1642: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:18:26.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:18:25 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2565.697847 secs 2024-03-28T01:18:26.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:18:25 smithi132 ceph-mon[35941]: pgmap v1642: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:18:26.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:18:25 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2565.697847 secs 2024-03-28T01:18:26.377 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:18:25 smithi165 ceph-mon[36302]: pgmap v1642: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:18:26.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:18:25 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2565.697847 secs 2024-03-28T01:18:28.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:18:27 smithi119 ceph-mon[31518]: pgmap v1643: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T01:18:28.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:18:27 smithi132 ceph-mon[35941]: pgmap v1643: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T01:18:28.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:18:27 smithi165 ceph-mon[36302]: pgmap v1643: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T01:18:30.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:18:29 smithi132 ceph-mon[35941]: pgmap v1644: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T01:18:30.377 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:18:29 smithi165 ceph-mon[36302]: pgmap v1644: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T01:18:30.400 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:18:29 smithi119 ceph-mon[31518]: pgmap v1644: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T01:18:31.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:18:30 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2570.697913 secs 2024-03-28T01:18:31.377 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:18:30 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2570.697913 secs 2024-03-28T01:18:31.400 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:18:30 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2570.697913 secs 2024-03-28T01:18:32.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:18:31 smithi132 ceph-mon[35941]: pgmap v1645: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:18:32.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:18:31 smithi165 ceph-mon[36302]: pgmap v1645: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:18:32.400 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:18:31 smithi119 ceph-mon[31518]: pgmap v1645: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:18:34.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:18:33 smithi132 ceph-mon[35941]: pgmap v1646: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T01:18:34.203 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:18:33 smithi119 ceph-mon[31518]: pgmap v1646: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T01:18:34.377 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:18:33 smithi165 ceph-mon[36302]: pgmap v1646: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T01:18:36.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:18:35 smithi132 ceph-mon[35941]: pgmap v1647: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:18:36.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:18:35 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2575.698004 secs 2024-03-28T01:18:36.377 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:18:35 smithi165 ceph-mon[36302]: pgmap v1647: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:18:36.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:18:35 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2575.698004 secs 2024-03-28T01:18:36.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:18:35 smithi119 ceph-mon[31518]: pgmap v1647: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:18:36.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:18:35 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2575.698004 secs 2024-03-28T01:18:38.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:18:37 smithi132 ceph-mon[35941]: pgmap v1648: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:18:38.204 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:18:37 smithi119 ceph-mon[31518]: pgmap v1648: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:18:38.377 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:18:37 smithi165 ceph-mon[36302]: pgmap v1648: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:18:40.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:18:39 smithi165 ceph-mon[36302]: pgmap v1649: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:18:40.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:18:39 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:18:40.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:18:39 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:18:40.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:18:39 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:18:40.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:18:39 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:18:40.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:18:39 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:18:40.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:18:39 smithi132 ceph-mon[35941]: pgmap v1649: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:18:40.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:18:39 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:18:40.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:18:39 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:18:40.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:18:39 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:18:40.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:18:39 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:18:40.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:18:39 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:18:40.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:18:39 smithi119 ceph-mon[31518]: pgmap v1649: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:18:40.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:18:39 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:18:40.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:18:39 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:18:40.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:18:39 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:18:40.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:18:39 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:18:40.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:18:39 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:18:41.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:18:40 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2580.698071 secs 2024-03-28T01:18:41.205 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:18:40 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2580.698071 secs 2024-03-28T01:18:41.377 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:18:40 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2580.698071 secs 2024-03-28T01:18:42.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:18:42 smithi165 ceph-mon[36302]: pgmap v1650: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:18:42.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:18:42 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:18:42.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:18:42 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:18:42.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:18:42 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:18:42.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:18:42 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:18:42.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:18:42 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:18:42.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:18:42 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:18:42.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:18:42 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:18:42.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:18:42 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:18:42.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:18:42 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:18:42.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:18:42 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:18:42.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:18:42 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:18:42.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:18:42 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:18:42.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:18:42 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:18:42.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:18:42 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:18:42.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:18:42 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:18:42.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:18:42 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:18:42.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:18:42 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:18:42.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:18:42 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:18:42.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:18:42 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:18:42.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:18:42 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:18:42.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:18:42 smithi119 ceph-mon[31518]: pgmap v1650: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:18:42.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:18:42 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:18:42.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:18:42 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:18:42.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:18:42 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:18:42.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:18:42 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:18:42.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:18:42 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:18:42.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:18:42 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:18:42.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:18:42 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:18:42.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:18:42 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:18:42.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:18:42 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:18:42.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:18:42 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:18:42.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:18:42 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:18:42.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:18:42 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:18:42.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:18:42 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:18:42.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:18:42 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:18:42.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:18:42 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:18:42.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:18:42 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:18:42.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:18:42 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:18:42.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:18:42 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:18:42.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:18:42 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:18:42.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:18:42 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:18:42.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:18:42 smithi132 ceph-mon[35941]: pgmap v1650: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:18:42.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:18:42 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:18:42.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:18:42 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:18:42.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:18:42 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:18:42.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:18:42 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:18:42.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:18:42 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:18:42.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:18:42 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:18:42.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:18:42 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:18:42.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:18:42 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:18:42.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:18:42 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:18:42.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:18:42 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:18:42.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:18:42 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:18:42.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:18:42 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:18:42.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:18:42 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:18:42.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:18:42 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:18:42.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:18:42 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:18:42.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:18:42 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:18:42.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:18:42 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:18:42.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:18:42 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:18:42.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:18:42 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:18:42.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:18:42 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:18:43.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:18:43 smithi165 ceph-mon[36302]: Adjusting osd_memory_target on smithi165 to 174.1M 2024-03-28T01:18:43.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:18:43 smithi165 ceph-mon[36302]: Unable to set osd_memory_target on smithi165 to 182635622: error parsing value: Value '182635622' is below minimum 939524096 2024-03-28T01:18:43.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:18:43 smithi165 ceph-mon[36302]: Adjusting osd_memory_target on smithi132 to 174.1M 2024-03-28T01:18:43.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:18:43 smithi165 ceph-mon[36302]: Unable to set osd_memory_target on smithi132 to 182640640: error parsing value: Value '182640640' is below minimum 939524096 2024-03-28T01:18:43.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:18:43 smithi119 ceph-mon[31518]: Adjusting osd_memory_target on smithi165 to 174.1M 2024-03-28T01:18:43.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:18:43 smithi119 ceph-mon[31518]: Unable to set osd_memory_target on smithi165 to 182635622: error parsing value: Value '182635622' is below minimum 939524096 2024-03-28T01:18:43.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:18:43 smithi119 ceph-mon[31518]: Adjusting osd_memory_target on smithi132 to 174.1M 2024-03-28T01:18:43.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:18:43 smithi119 ceph-mon[31518]: Unable to set osd_memory_target on smithi132 to 182640640: error parsing value: Value '182640640' is below minimum 939524096 2024-03-28T01:18:43.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:18:43 smithi132 ceph-mon[35941]: Adjusting osd_memory_target on smithi165 to 174.1M 2024-03-28T01:18:43.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:18:43 smithi132 ceph-mon[35941]: Unable to set osd_memory_target on smithi165 to 182635622: error parsing value: Value '182635622' is below minimum 939524096 2024-03-28T01:18:43.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:18:43 smithi132 ceph-mon[35941]: Adjusting osd_memory_target on smithi132 to 174.1M 2024-03-28T01:18:43.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:18:43 smithi132 ceph-mon[35941]: Unable to set osd_memory_target on smithi132 to 182640640: error parsing value: Value '182640640' is below minimum 939524096 2024-03-28T01:18:44.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:18:44 smithi165 ceph-mon[36302]: pgmap v1651: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:18:44.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:18:44 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:18:44.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:18:44 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:18:44.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:18:44 smithi119 ceph-mon[31518]: pgmap v1651: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:18:44.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:18:44 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:18:44.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:18:44 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:18:44.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:18:44 smithi132 ceph-mon[35941]: pgmap v1651: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:18:44.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:18:44 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:18:44.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:18:44 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:18:45.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:18:45 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:18:45.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:18:45 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:18:45.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:18:45 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:18:45.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:18:45 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:18:45.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:18:45 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:18:45.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:18:45 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:18:45.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:18:45 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:18:45.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:18:45 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:18:45.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:18:45 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:18:45.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:18:45 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:18:45.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:18:45 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:18:45.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:18:45 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:18:45.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:18:45 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:18:45.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:18:45 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:18:45.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:18:45 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:18:45.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:18:45 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:18:45.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:18:45 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:18:45.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:18:45 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:18:45.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:18:45 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:18:45.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:18:45 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:18:45.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:18:45 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:18:45.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:18:45 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:18:45.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:18:45 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:18:45.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:18:45 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:18:45.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:18:45 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:18:45.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:18:45 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:18:45.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:18:45 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:18:45.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:18:45 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:18:45.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:18:45 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:18:45.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:18:45 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:18:45.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:18:45 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:18:45.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:18:45 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:18:45.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:18:45 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:18:46.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:18:46 smithi165 ceph-mon[36302]: Adjusting osd_memory_target on smithi119 to 176.7M 2024-03-28T01:18:46.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:18:46 smithi165 ceph-mon[36302]: Unable to set osd_memory_target on smithi119 to 185372364: error parsing value: Value '185372364' is below minimum 939524096 2024-03-28T01:18:46.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:18:46 smithi165 ceph-mon[36302]: pgmap v1652: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 7 op/s 2024-03-28T01:18:46.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:18:46 smithi165 ceph-mon[36302]: pgmap v1653: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T01:18:46.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:18:46 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2585.698132 secs 2024-03-28T01:18:46.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:18:46 smithi119 ceph-mon[31518]: Adjusting osd_memory_target on smithi119 to 176.7M 2024-03-28T01:18:46.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:18:46 smithi119 ceph-mon[31518]: Unable to set osd_memory_target on smithi119 to 185372364: error parsing value: Value '185372364' is below minimum 939524096 2024-03-28T01:18:46.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:18:46 smithi119 ceph-mon[31518]: pgmap v1652: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 7 op/s 2024-03-28T01:18:46.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:18:46 smithi119 ceph-mon[31518]: pgmap v1653: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T01:18:46.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:18:46 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2585.698132 secs 2024-03-28T01:18:46.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:18:46 smithi132 ceph-mon[35941]: Adjusting osd_memory_target on smithi119 to 176.7M 2024-03-28T01:18:46.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:18:46 smithi132 ceph-mon[35941]: Unable to set osd_memory_target on smithi119 to 185372364: error parsing value: Value '185372364' is below minimum 939524096 2024-03-28T01:18:46.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:18:46 smithi132 ceph-mon[35941]: pgmap v1652: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 7 op/s 2024-03-28T01:18:46.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:18:46 smithi132 ceph-mon[35941]: pgmap v1653: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T01:18:46.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:18:46 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2585.698132 secs 2024-03-28T01:18:48.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:18:48 smithi165 ceph-mon[36302]: pgmap v1654: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T01:18:48.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:18:48 smithi119 ceph-mon[31518]: pgmap v1654: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T01:18:48.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:18:48 smithi132 ceph-mon[35941]: pgmap v1654: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T01:18:50.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:18:50 smithi165 ceph-mon[36302]: pgmap v1655: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T01:18:50.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:18:50 smithi119 ceph-mon[31518]: pgmap v1655: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T01:18:50.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:18:50 smithi132 ceph-mon[35941]: pgmap v1655: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T01:18:51.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:18:51 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2590.698194 secs 2024-03-28T01:18:51.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:18:51 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2590.698194 secs 2024-03-28T01:18:51.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:18:51 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2590.698194 secs 2024-03-28T01:18:52.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:18:52 smithi165 ceph-mon[36302]: pgmap v1656: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T01:18:52.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:18:52 smithi119 ceph-mon[31518]: pgmap v1656: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T01:18:52.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:18:52 smithi132 ceph-mon[35941]: pgmap v1656: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T01:18:54.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:18:54 smithi165 ceph-mon[36302]: pgmap v1657: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T01:18:54.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:18:54 smithi119 ceph-mon[31518]: pgmap v1657: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T01:18:54.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:18:54 smithi132 ceph-mon[35941]: pgmap v1657: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T01:18:55.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:18:55 smithi165 ceph-mon[36302]: pgmap v1658: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T01:18:55.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:18:55 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2595.698253 secs 2024-03-28T01:18:55.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:18:55 smithi119 ceph-mon[31518]: pgmap v1658: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T01:18:55.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:18:55 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2595.698253 secs 2024-03-28T01:18:55.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:18:55 smithi132 ceph-mon[35941]: pgmap v1658: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T01:18:55.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:18:55 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2595.698253 secs 2024-03-28T01:18:58.127 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:18:57 smithi165 ceph-mon[36302]: pgmap v1659: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:18:58.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:18:57 smithi119 ceph-mon[31518]: pgmap v1659: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:18:58.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:18:57 smithi132 ceph-mon[35941]: pgmap v1659: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:19:00.127 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:18:59 smithi165 ceph-mon[36302]: pgmap v1660: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:19:00.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:18:59 smithi119 ceph-mon[31518]: pgmap v1660: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:19:00.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:18:59 smithi132 ceph-mon[35941]: pgmap v1660: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:19:01.127 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:19:00 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2600.698341 secs 2024-03-28T01:19:01.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:19:00 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2600.698341 secs 2024-03-28T01:19:01.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:19:00 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2600.698341 secs 2024-03-28T01:19:02.127 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:19:01 smithi165 ceph-mon[36302]: pgmap v1661: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:19:02.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:19:01 smithi119 ceph-mon[31518]: pgmap v1661: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:19:02.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:19:01 smithi132 ceph-mon[35941]: pgmap v1661: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:19:04.127 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:19:03 smithi165 ceph-mon[36302]: pgmap v1662: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:19:04.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:19:03 smithi119 ceph-mon[31518]: pgmap v1662: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:19:04.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:19:03 smithi132 ceph-mon[35941]: pgmap v1662: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:19:06.025 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:19:05 smithi119 ceph-mon[31518]: pgmap v1663: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:19:06.026 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:19:05 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2605.698431 secs 2024-03-28T01:19:06.127 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:19:05 smithi165 ceph-mon[36302]: pgmap v1663: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:19:06.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:19:05 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2605.698431 secs 2024-03-28T01:19:06.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:19:05 smithi132 ceph-mon[35941]: pgmap v1663: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:19:06.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:19:05 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2605.698431 secs 2024-03-28T01:19:08.127 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:19:07 smithi165 ceph-mon[36302]: pgmap v1664: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:19:08.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:19:07 smithi119 ceph-mon[31518]: pgmap v1664: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:19:08.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:19:07 smithi132 ceph-mon[35941]: pgmap v1664: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:19:10.127 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:19:09 smithi165 ceph-mon[36302]: pgmap v1665: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:19:10.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:19:09 smithi119 ceph-mon[31518]: pgmap v1665: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:19:10.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:19:09 smithi132 ceph-mon[35941]: pgmap v1665: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:19:11.099 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:19:10 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2610.698490 secs 2024-03-28T01:19:11.127 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:19:10 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2610.698490 secs 2024-03-28T01:19:11.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:19:10 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2610.698490 secs 2024-03-28T01:19:12.127 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:19:11 smithi165 ceph-mon[36302]: pgmap v1666: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:19:12.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:19:11 smithi119 ceph-mon[31518]: pgmap v1666: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:19:12.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:19:11 smithi132 ceph-mon[35941]: pgmap v1666: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:19:14.127 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:19:13 smithi165 ceph-mon[36302]: pgmap v1667: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:19:14.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:19:13 smithi119 ceph-mon[31518]: pgmap v1667: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:19:14.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:19:13 smithi132 ceph-mon[35941]: pgmap v1667: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:19:16.127 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:19:15 smithi165 ceph-mon[36302]: pgmap v1668: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:19:16.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:19:15 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2615.698579 secs 2024-03-28T01:19:16.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:19:15 smithi119 ceph-mon[31518]: pgmap v1668: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:19:16.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:19:15 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2615.698579 secs 2024-03-28T01:19:16.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:19:15 smithi132 ceph-mon[35941]: pgmap v1668: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:19:16.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:19:15 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2615.698579 secs 2024-03-28T01:19:18.127 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:19:17 smithi165 ceph-mon[36302]: pgmap v1669: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:19:18.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:19:17 smithi119 ceph-mon[31518]: pgmap v1669: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:19:18.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:19:17 smithi132 ceph-mon[35941]: pgmap v1669: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:19:20.097 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:19:19 smithi119 ceph-mon[31518]: pgmap v1670: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:19:20.127 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:19:19 smithi165 ceph-mon[36302]: pgmap v1670: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:19:20.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:19:19 smithi132 ceph-mon[35941]: pgmap v1670: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:19:21.127 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:19:20 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2620.698669 secs 2024-03-28T01:19:21.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:19:20 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2620.698669 secs 2024-03-28T01:19:21.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:19:20 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2620.698669 secs 2024-03-28T01:19:22.127 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:19:21 smithi165 ceph-mon[36302]: pgmap v1671: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:19:22.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:19:21 smithi119 ceph-mon[31518]: pgmap v1671: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:19:22.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:19:21 smithi132 ceph-mon[35941]: pgmap v1671: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:19:24.127 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:19:23 smithi165 ceph-mon[36302]: pgmap v1672: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T01:19:24.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:19:23 smithi119 ceph-mon[31518]: pgmap v1672: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T01:19:24.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:19:23 smithi132 ceph-mon[35941]: pgmap v1672: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T01:19:26.127 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:19:25 smithi165 ceph-mon[36302]: pgmap v1673: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:19:26.127 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:19:25 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2625.698759 secs 2024-03-28T01:19:26.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:19:25 smithi119 ceph-mon[31518]: pgmap v1673: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:19:26.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:19:25 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2625.698759 secs 2024-03-28T01:19:26.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:19:25 smithi132 ceph-mon[35941]: pgmap v1673: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:19:26.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:19:25 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2625.698759 secs 2024-03-28T01:19:28.127 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:19:27 smithi165 ceph-mon[36302]: pgmap v1674: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T01:19:28.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:19:27 smithi119 ceph-mon[31518]: pgmap v1674: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T01:19:28.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:19:27 smithi132 ceph-mon[35941]: pgmap v1674: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T01:19:30.127 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:19:29 smithi165 ceph-mon[36302]: pgmap v1675: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T01:19:30.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:19:29 smithi119 ceph-mon[31518]: pgmap v1675: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T01:19:30.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:19:29 smithi132 ceph-mon[35941]: pgmap v1675: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T01:19:31.127 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:19:30 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2630.698845 secs 2024-03-28T01:19:31.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:19:30 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2630.698845 secs 2024-03-28T01:19:31.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:19:30 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2630.698845 secs 2024-03-28T01:19:32.127 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:19:31 smithi165 ceph-mon[36302]: pgmap v1676: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:19:32.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:19:31 smithi119 ceph-mon[31518]: pgmap v1676: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:19:32.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:19:31 smithi132 ceph-mon[35941]: pgmap v1676: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:19:34.127 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:19:33 smithi165 ceph-mon[36302]: pgmap v1677: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:19:34.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:19:33 smithi119 ceph-mon[31518]: pgmap v1677: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:19:34.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:19:33 smithi132 ceph-mon[35941]: pgmap v1677: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:19:36.127 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:19:35 smithi165 ceph-mon[36302]: pgmap v1678: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:19:36.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:19:35 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2635.698930 secs 2024-03-28T01:19:36.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:19:35 smithi119 ceph-mon[31518]: pgmap v1678: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:19:36.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:19:35 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2635.698930 secs 2024-03-28T01:19:36.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:19:35 smithi132 ceph-mon[35941]: pgmap v1678: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:19:36.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:19:35 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2635.698930 secs 2024-03-28T01:19:38.127 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:19:37 smithi165 ceph-mon[36302]: pgmap v1679: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:19:38.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:19:37 smithi119 ceph-mon[31518]: pgmap v1679: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:19:38.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:19:37 smithi132 ceph-mon[35941]: pgmap v1679: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:19:40.127 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:19:39 smithi165 ceph-mon[36302]: pgmap v1680: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:19:40.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:19:39 smithi119 ceph-mon[31518]: pgmap v1680: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:19:40.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:19:39 smithi132 ceph-mon[35941]: pgmap v1680: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:19:41.127 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:19:40 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2640.699020 secs 2024-03-28T01:19:41.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:19:40 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2640.699020 secs 2024-03-28T01:19:41.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:19:40 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2640.699020 secs 2024-03-28T01:19:42.127 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:19:41 smithi165 ceph-mon[36302]: pgmap v1681: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:19:42.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:19:41 smithi119 ceph-mon[31518]: pgmap v1681: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:19:42.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:19:41 smithi132 ceph-mon[35941]: pgmap v1681: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:19:44.127 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:19:43 smithi165 ceph-mon[36302]: pgmap v1682: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:19:44.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:19:43 smithi119 ceph-mon[31518]: pgmap v1682: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:19:44.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:19:43 smithi132 ceph-mon[35941]: pgmap v1682: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:19:46.127 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:19:45 smithi165 ceph-mon[36302]: pgmap v1683: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:19:46.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:19:45 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:19:46.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:19:45 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2645.699092 secs 2024-03-28T01:19:46.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:19:45 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:19:46.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:19:45 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:19:46.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:19:45 smithi119 ceph-mon[31518]: pgmap v1683: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:19:46.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:19:45 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:19:46.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:19:45 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2645.699092 secs 2024-03-28T01:19:46.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:19:45 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:19:46.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:19:45 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:19:46.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:19:45 smithi132 ceph-mon[35941]: pgmap v1683: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:19:46.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:19:45 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:19:46.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:19:45 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2645.699092 secs 2024-03-28T01:19:46.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:19:45 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:19:46.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:19:45 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:19:47.127 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:19:46 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:19:47.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:19:46 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:19:47.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:19:46 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:19:47.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:19:46 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:19:47.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:19:46 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:19:47.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:19:46 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:19:47.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:19:46 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:19:47.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:19:46 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:19:47.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:19:46 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:19:48.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:19:47 smithi165 ceph-mon[36302]: pgmap v1684: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T01:19:48.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:19:47 smithi165 ceph-mon[36302]: pgmap v1685: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:19:48.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:19:47 smithi119 ceph-mon[31518]: pgmap v1684: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T01:19:48.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:19:47 smithi119 ceph-mon[31518]: pgmap v1685: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:19:48.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:19:47 smithi132 ceph-mon[35941]: pgmap v1684: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T01:19:48.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:19:47 smithi132 ceph-mon[35941]: pgmap v1685: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:19:50.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:19:49 smithi119 ceph-mon[31518]: pgmap v1686: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:19:50.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:19:49 smithi132 ceph-mon[35941]: pgmap v1686: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:19:50.377 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:19:49 smithi165 ceph-mon[36302]: pgmap v1686: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:19:51.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:19:50 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2650.699167 secs 2024-03-28T01:19:51.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:19:50 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2650.699167 secs 2024-03-28T01:19:51.377 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:19:50 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2650.699167 secs 2024-03-28T01:19:52.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:19:51 smithi119 ceph-mon[31518]: pgmap v1687: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:19:52.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:19:51 smithi132 ceph-mon[35941]: pgmap v1687: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:19:52.377 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:19:51 smithi165 ceph-mon[36302]: pgmap v1687: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:19:54.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:19:53 smithi119 ceph-mon[31518]: pgmap v1688: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:19:54.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:19:53 smithi132 ceph-mon[35941]: pgmap v1688: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:19:54.377 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:19:53 smithi165 ceph-mon[36302]: pgmap v1688: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:19:56.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:19:55 smithi132 ceph-mon[35941]: pgmap v1689: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:19:56.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:19:55 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2655.699258 secs 2024-03-28T01:19:56.377 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:19:55 smithi165 ceph-mon[36302]: pgmap v1689: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:19:56.377 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:19:55 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2655.699258 secs 2024-03-28T01:19:56.400 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:19:55 smithi119 ceph-mon[31518]: pgmap v1689: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:19:56.400 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:19:55 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2655.699258 secs 2024-03-28T01:19:58.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:19:57 smithi132 ceph-mon[35941]: pgmap v1690: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T01:19:58.377 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:19:57 smithi165 ceph-mon[36302]: pgmap v1690: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T01:19:58.400 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:19:57 smithi119 ceph-mon[31518]: pgmap v1690: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T01:20:00.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:19:59 smithi132 ceph-mon[35941]: pgmap v1691: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:20:00.377 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:19:59 smithi165 ceph-mon[36302]: pgmap v1691: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:20:00.400 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:19:59 smithi119 ceph-mon[31518]: pgmap v1691: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:20:01.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:20:00 smithi132 ceph-mon[35941]: overall HEALTH_WARN 5 failed cephadm daemon(s); 1 filesystem is degraded; 1 MDSs report slow requests 2024-03-28T01:20:01.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:20:00 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2660.699347 secs 2024-03-28T01:20:01.377 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:20:00 smithi165 ceph-mon[36302]: overall HEALTH_WARN 5 failed cephadm daemon(s); 1 filesystem is degraded; 1 MDSs report slow requests 2024-03-28T01:20:01.377 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:20:00 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2660.699347 secs 2024-03-28T01:20:01.400 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:20:00 smithi119 ceph-mon[31518]: overall HEALTH_WARN 5 failed cephadm daemon(s); 1 filesystem is degraded; 1 MDSs report slow requests 2024-03-28T01:20:01.400 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:20:00 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2660.699347 secs 2024-03-28T01:20:02.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:20:01 smithi132 ceph-mon[35941]: pgmap v1692: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:20:02.377 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:20:01 smithi165 ceph-mon[36302]: pgmap v1692: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:20:02.400 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:20:01 smithi119 ceph-mon[31518]: pgmap v1692: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:20:04.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:20:03 smithi132 ceph-mon[35941]: pgmap v1693: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:20:04.377 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:20:03 smithi165 ceph-mon[36302]: pgmap v1693: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:20:04.400 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:20:03 smithi119 ceph-mon[31518]: pgmap v1693: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:20:06.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:20:05 smithi132 ceph-mon[35941]: pgmap v1694: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:20:06.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:20:05 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2665.699410 secs 2024-03-28T01:20:06.377 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:20:05 smithi165 ceph-mon[36302]: pgmap v1694: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:20:06.377 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:20:05 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2665.699410 secs 2024-03-28T01:20:06.400 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:20:05 smithi119 ceph-mon[31518]: pgmap v1694: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:20:06.400 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:20:05 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2665.699410 secs 2024-03-28T01:20:08.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:20:07 smithi132 ceph-mon[35941]: pgmap v1695: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:20:08.377 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:20:07 smithi165 ceph-mon[36302]: pgmap v1695: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:20:08.400 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:20:07 smithi119 ceph-mon[31518]: pgmap v1695: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:20:10.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:20:09 smithi132 ceph-mon[35941]: pgmap v1696: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:20:10.198 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:20:09 smithi165 ceph-mon[36302]: pgmap v1696: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:20:10.272 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:20:09 smithi119 ceph-mon[31518]: pgmap v1696: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:20:11.377 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:20:10 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2670.699470 secs 2024-03-28T01:20:11.400 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:20:10 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2670.699470 secs 2024-03-28T01:20:11.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:20:10 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2670.699470 secs 2024-03-28T01:20:12.377 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:20:11 smithi165 ceph-mon[36302]: pgmap v1697: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:20:12.400 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:20:11 smithi119 ceph-mon[31518]: pgmap v1697: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:20:12.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:20:11 smithi132 ceph-mon[35941]: pgmap v1697: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:20:14.377 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:20:13 smithi165 ceph-mon[36302]: pgmap v1698: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:20:14.400 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:20:13 smithi119 ceph-mon[31518]: pgmap v1698: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:20:14.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:20:13 smithi132 ceph-mon[35941]: pgmap v1698: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:20:16.377 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:20:15 smithi165 ceph-mon[36302]: pgmap v1699: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:20:16.377 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:20:15 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2675.699483 secs 2024-03-28T01:20:16.400 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:20:15 smithi119 ceph-mon[31518]: pgmap v1699: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:20:16.400 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:20:15 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2675.699483 secs 2024-03-28T01:20:16.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:20:15 smithi132 ceph-mon[35941]: pgmap v1699: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:20:16.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:20:15 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2675.699483 secs 2024-03-28T01:20:18.377 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:20:17 smithi165 ceph-mon[36302]: pgmap v1700: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:20:18.400 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:20:17 smithi119 ceph-mon[31518]: pgmap v1700: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:20:18.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:20:17 smithi132 ceph-mon[35941]: pgmap v1700: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:20:20.377 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:20:19 smithi165 ceph-mon[36302]: pgmap v1701: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:20:20.400 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:20:19 smithi119 ceph-mon[31518]: pgmap v1701: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:20:20.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:20:19 smithi132 ceph-mon[35941]: pgmap v1701: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:20:21.377 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:20:20 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2680.699571 secs 2024-03-28T01:20:21.400 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:20:20 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2680.699571 secs 2024-03-28T01:20:21.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:20:20 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2680.699571 secs 2024-03-28T01:20:22.377 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:20:21 smithi165 ceph-mon[36302]: pgmap v1702: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:20:22.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:20:21 smithi119 ceph-mon[31518]: pgmap v1702: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:20:22.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:20:21 smithi132 ceph-mon[35941]: pgmap v1702: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:20:24.377 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:20:23 smithi165 ceph-mon[36302]: pgmap v1703: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:20:24.400 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:20:23 smithi119 ceph-mon[31518]: pgmap v1703: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:20:24.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:20:23 smithi132 ceph-mon[35941]: pgmap v1703: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:20:26.377 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:20:25 smithi165 ceph-mon[36302]: pgmap v1704: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:20:26.377 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:20:25 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2685.699662 secs 2024-03-28T01:20:26.400 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:20:25 smithi119 ceph-mon[31518]: pgmap v1704: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:20:26.400 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:20:25 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2685.699662 secs 2024-03-28T01:20:26.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:20:25 smithi132 ceph-mon[35941]: pgmap v1704: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:20:26.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:20:25 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2685.699662 secs 2024-03-28T01:20:28.377 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:20:27 smithi165 ceph-mon[36302]: pgmap v1705: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:20:28.400 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:20:27 smithi119 ceph-mon[31518]: pgmap v1705: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:20:28.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:20:27 smithi132 ceph-mon[35941]: pgmap v1705: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:20:30.377 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:20:30 smithi165 ceph-mon[36302]: pgmap v1706: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:20:30.400 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:20:30 smithi119 ceph-mon[31518]: pgmap v1706: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:20:30.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:20:30 smithi132 ceph-mon[35941]: pgmap v1706: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:20:31.377 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:20:31 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2690.699680 secs 2024-03-28T01:20:31.400 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:20:31 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2690.699680 secs 2024-03-28T01:20:31.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:20:31 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2690.699680 secs 2024-03-28T01:20:32.377 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:20:32 smithi165 ceph-mon[36302]: pgmap v1707: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:20:32.400 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:20:32 smithi119 ceph-mon[31518]: pgmap v1707: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:20:32.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:20:32 smithi132 ceph-mon[35941]: pgmap v1707: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:20:34.377 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:20:34 smithi165 ceph-mon[36302]: pgmap v1708: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:20:34.400 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:20:34 smithi119 ceph-mon[31518]: pgmap v1708: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:20:34.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:20:34 smithi132 ceph-mon[35941]: pgmap v1708: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:20:36.377 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:20:36 smithi165 ceph-mon[36302]: pgmap v1709: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:20:36.377 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:20:36 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2695.699750 secs 2024-03-28T01:20:36.400 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:20:36 smithi119 ceph-mon[31518]: pgmap v1709: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:20:36.400 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:20:36 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2695.699750 secs 2024-03-28T01:20:36.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:20:36 smithi132 ceph-mon[35941]: pgmap v1709: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:20:36.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:20:36 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2695.699750 secs 2024-03-28T01:20:38.377 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:20:38 smithi165 ceph-mon[36302]: pgmap v1710: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:20:38.400 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:20:38 smithi119 ceph-mon[31518]: pgmap v1710: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:20:38.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:20:38 smithi132 ceph-mon[35941]: pgmap v1710: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:20:40.377 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:20:40 smithi165 ceph-mon[36302]: pgmap v1711: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:20:40.400 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:20:40 smithi119 ceph-mon[31518]: pgmap v1711: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:20:40.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:20:40 smithi132 ceph-mon[35941]: pgmap v1711: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:20:41.349 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:20:41 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2700.699839 secs 2024-03-28T01:20:41.377 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:20:41 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2700.699839 secs 2024-03-28T01:20:41.400 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:20:41 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2700.699839 secs 2024-03-28T01:20:42.377 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:20:42 smithi165 ceph-mon[36302]: pgmap v1712: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:20:42.400 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:20:42 smithi119 ceph-mon[31518]: pgmap v1712: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:20:42.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:20:42 smithi132 ceph-mon[35941]: pgmap v1712: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:20:44.377 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:20:44 smithi165 ceph-mon[36302]: pgmap v1713: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:20:44.400 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:20:44 smithi119 ceph-mon[31518]: pgmap v1713: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:20:44.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:20:44 smithi132 ceph-mon[35941]: pgmap v1713: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:20:46.377 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:20:46 smithi165 ceph-mon[36302]: pgmap v1714: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:20:46.377 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:20:46 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2705.699912 secs 2024-03-28T01:20:46.400 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:20:46 smithi119 ceph-mon[31518]: pgmap v1714: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:20:46.400 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:20:46 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2705.699912 secs 2024-03-28T01:20:46.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:20:46 smithi132 ceph-mon[35941]: pgmap v1714: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:20:46.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:20:46 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2705.699912 secs 2024-03-28T01:20:47.377 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:20:47 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:20:47.400 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:20:47 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:20:47.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:20:47 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:20:48.377 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:20:48 smithi165 ceph-mon[36302]: pgmap v1715: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:20:48.377 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:20:48 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:20:48.377 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:20:48 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:20:48.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:20:48 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:20:48.400 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:20:48 smithi119 ceph-mon[31518]: pgmap v1715: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:20:48.400 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:20:48 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:20:48.400 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:20:48 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:20:48.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:20:48 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:20:48.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:20:48 smithi132 ceph-mon[35941]: pgmap v1715: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:20:48.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:20:48 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:20:48.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:20:48 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:20:48.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:20:48 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:20:49.377 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:20:49 smithi165 ceph-mon[36302]: pgmap v1716: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T01:20:49.377 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:20:49 smithi165 ceph-mon[36302]: pgmap v1717: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:20:49.400 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:20:49 smithi119 ceph-mon[31518]: pgmap v1716: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T01:20:49.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:20:49 smithi119 ceph-mon[31518]: pgmap v1717: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:20:49.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:20:49 smithi132 ceph-mon[35941]: pgmap v1716: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T01:20:49.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:20:49 smithi132 ceph-mon[35941]: pgmap v1717: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:20:51.377 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:20:51 smithi165 ceph-mon[36302]: pgmap v1718: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:20:51.377 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:20:51 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2710.699926 secs 2024-03-28T01:20:51.400 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:20:51 smithi119 ceph-mon[31518]: pgmap v1718: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:20:51.400 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:20:51 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2710.699926 secs 2024-03-28T01:20:51.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:20:51 smithi132 ceph-mon[35941]: pgmap v1718: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:20:51.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:20:51 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2710.699926 secs 2024-03-28T01:20:53.377 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:20:53 smithi165 ceph-mon[36302]: pgmap v1719: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:20:53.400 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:20:53 smithi119 ceph-mon[31518]: pgmap v1719: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:20:53.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:20:53 smithi132 ceph-mon[35941]: pgmap v1719: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:20:55.377 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:20:55 smithi165 ceph-mon[36302]: pgmap v1720: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-28T01:20:55.400 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:20:55 smithi119 ceph-mon[31518]: pgmap v1720: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-28T01:20:55.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:20:55 smithi132 ceph-mon[35941]: pgmap v1720: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-28T01:20:56.377 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:20:56 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2715.699989 secs 2024-03-28T01:20:56.400 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:20:56 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2715.699989 secs 2024-03-28T01:20:56.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:20:56 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2715.699989 secs 2024-03-28T01:20:57.377 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:20:57 smithi165 ceph-mon[36302]: pgmap v1721: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:20:57.400 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:20:57 smithi119 ceph-mon[31518]: pgmap v1721: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:20:57.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:20:57 smithi132 ceph-mon[35941]: pgmap v1721: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:20:59.377 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:20:59 smithi165 ceph-mon[36302]: pgmap v1722: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T01:20:59.400 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:20:59 smithi119 ceph-mon[31518]: pgmap v1722: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T01:20:59.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:20:59 smithi132 ceph-mon[35941]: pgmap v1722: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T01:21:01.377 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:21:01 smithi165 ceph-mon[36302]: pgmap v1723: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:21:01.377 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:21:01 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2720.700052 secs 2024-03-28T01:21:01.400 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:21:01 smithi119 ceph-mon[31518]: pgmap v1723: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:21:01.400 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:21:01 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2720.700052 secs 2024-03-28T01:21:01.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:21:01 smithi132 ceph-mon[35941]: pgmap v1723: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:21:01.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:21:01 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2720.700052 secs 2024-03-28T01:21:02.877 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:21:02 smithi165 ceph-mon[36302]: pgmap v1724: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:21:02.900 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:21:02 smithi119 ceph-mon[31518]: pgmap v1724: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:21:02.947 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:21:02 smithi132 ceph-mon[35941]: pgmap v1724: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:21:05.377 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:21:05 smithi165 ceph-mon[36302]: pgmap v1725: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:21:05.400 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:21:05 smithi119 ceph-mon[31518]: pgmap v1725: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:21:05.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:21:05 smithi132 ceph-mon[35941]: pgmap v1725: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:21:06.377 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:21:06 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2725.700248 secs 2024-03-28T01:21:06.400 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:21:06 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2725.700248 secs 2024-03-28T01:21:06.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:21:06 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2725.700248 secs 2024-03-28T01:21:07.377 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:21:07 smithi165 ceph-mon[36302]: pgmap v1726: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:21:07.400 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:21:07 smithi119 ceph-mon[31518]: pgmap v1726: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:21:07.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:21:07 smithi132 ceph-mon[35941]: pgmap v1726: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:21:09.377 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:21:09 smithi165 ceph-mon[36302]: pgmap v1727: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:21:09.400 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:21:09 smithi119 ceph-mon[31518]: pgmap v1727: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:21:09.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:21:09 smithi132 ceph-mon[35941]: pgmap v1727: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:21:11.377 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:21:11 smithi165 ceph-mon[36302]: pgmap v1728: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:21:11.377 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:21:11 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2730.700286 secs 2024-03-28T01:21:11.400 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:21:11 smithi119 ceph-mon[31518]: pgmap v1728: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:21:11.400 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:21:11 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2730.700286 secs 2024-03-28T01:21:11.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:21:11 smithi132 ceph-mon[35941]: pgmap v1728: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:21:11.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:21:11 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2730.700286 secs 2024-03-28T01:21:13.377 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:21:13 smithi165 ceph-mon[36302]: pgmap v1729: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:21:13.400 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:21:13 smithi119 ceph-mon[31518]: pgmap v1729: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:21:13.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:21:13 smithi132 ceph-mon[35941]: pgmap v1729: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:21:15.377 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:21:15 smithi165 ceph-mon[36302]: pgmap v1730: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:21:15.400 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:21:15 smithi119 ceph-mon[31518]: pgmap v1730: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:21:15.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:21:15 smithi132 ceph-mon[35941]: pgmap v1730: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:21:16.377 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:21:16 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2735.700375 secs 2024-03-28T01:21:16.400 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:21:16 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2735.700375 secs 2024-03-28T01:21:16.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:21:16 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2735.700375 secs 2024-03-28T01:21:17.377 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:21:17 smithi165 ceph-mon[36302]: pgmap v1731: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:21:17.400 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:21:17 smithi119 ceph-mon[31518]: pgmap v1731: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:21:17.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:21:17 smithi132 ceph-mon[35941]: pgmap v1731: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:21:19.377 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:21:19 smithi165 ceph-mon[36302]: pgmap v1732: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:21:19.400 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:21:19 smithi119 ceph-mon[31518]: pgmap v1732: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:21:19.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:21:19 smithi132 ceph-mon[35941]: pgmap v1732: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:21:21.377 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:21:21 smithi165 ceph-mon[36302]: pgmap v1733: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:21:21.377 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:21:21 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2740.700465 secs 2024-03-28T01:21:21.400 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:21:21 smithi119 ceph-mon[31518]: pgmap v1733: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:21:21.400 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:21:21 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2740.700465 secs 2024-03-28T01:21:21.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:21:21 smithi132 ceph-mon[35941]: pgmap v1733: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:21:21.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:21:21 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2740.700465 secs 2024-03-28T01:21:23.377 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:21:23 smithi165 ceph-mon[36302]: pgmap v1734: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:21:23.400 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:21:23 smithi119 ceph-mon[31518]: pgmap v1734: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:21:23.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:21:23 smithi132 ceph-mon[35941]: pgmap v1734: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:21:25.377 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:21:25 smithi165 ceph-mon[36302]: pgmap v1735: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:21:25.400 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:21:25 smithi119 ceph-mon[31518]: pgmap v1735: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:21:25.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:21:25 smithi132 ceph-mon[35941]: pgmap v1735: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:21:26.377 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:21:26 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2745.700503 secs 2024-03-28T01:21:26.400 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:21:26 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2745.700503 secs 2024-03-28T01:21:26.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:21:26 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2745.700503 secs 2024-03-28T01:21:27.377 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:21:27 smithi165 ceph-mon[36302]: pgmap v1736: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:21:27.400 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:21:27 smithi119 ceph-mon[31518]: pgmap v1736: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:21:27.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:21:27 smithi132 ceph-mon[35941]: pgmap v1736: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:21:29.377 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:21:29 smithi165 ceph-mon[36302]: pgmap v1737: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:21:29.400 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:21:29 smithi119 ceph-mon[31518]: pgmap v1737: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:21:29.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:21:29 smithi132 ceph-mon[35941]: pgmap v1737: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:21:31.377 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:21:31 smithi165 ceph-mon[36302]: pgmap v1738: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:21:31.377 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:21:31 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2750.700545 secs 2024-03-28T01:21:31.400 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:21:31 smithi119 ceph-mon[31518]: pgmap v1738: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:21:31.400 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:21:31 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2750.700545 secs 2024-03-28T01:21:31.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:21:31 smithi132 ceph-mon[35941]: pgmap v1738: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:21:31.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:21:31 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2750.700545 secs 2024-03-28T01:21:33.377 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:21:33 smithi165 ceph-mon[36302]: pgmap v1739: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:21:33.400 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:21:33 smithi119 ceph-mon[31518]: pgmap v1739: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:21:33.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:21:33 smithi132 ceph-mon[35941]: pgmap v1739: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:21:35.377 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:21:35 smithi165 ceph-mon[36302]: pgmap v1740: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:21:35.400 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:21:35 smithi119 ceph-mon[31518]: pgmap v1740: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:21:35.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:21:35 smithi132 ceph-mon[35941]: pgmap v1740: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:21:36.400 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:21:36 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2755.700634 secs 2024-03-28T01:21:36.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:21:36 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2755.700634 secs 2024-03-28T01:21:36.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:21:36 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2755.700634 secs 2024-03-28T01:21:37.400 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:21:37 smithi119 ceph-mon[31518]: pgmap v1741: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:21:37.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:21:37 smithi132 ceph-mon[35941]: pgmap v1741: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:21:37.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:21:37 smithi165 ceph-mon[36302]: pgmap v1741: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:21:39.400 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:21:39 smithi119 ceph-mon[31518]: pgmap v1742: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:21:39.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:21:39 smithi132 ceph-mon[35941]: pgmap v1742: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:21:39.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:21:39 smithi165 ceph-mon[36302]: pgmap v1742: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:21:41.400 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:21:41 smithi119 ceph-mon[31518]: pgmap v1743: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:21:41.400 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:21:41 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2760.700723 secs 2024-03-28T01:21:41.429 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:21:41 smithi165 ceph-mon[36302]: pgmap v1743: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:21:41.469 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:21:41 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2760.700723 secs 2024-03-28T01:21:41.470 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:21:41 smithi132 ceph-mon[35941]: pgmap v1743: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:21:41.470 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:21:41 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2760.700723 secs 2024-03-28T01:21:43.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:21:43 smithi132 ceph-mon[35941]: pgmap v1744: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:21:43.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:21:43 smithi165 ceph-mon[36302]: pgmap v1744: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:21:43.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:21:43 smithi119 ceph-mon[31518]: pgmap v1744: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:21:45.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:21:45 smithi132 ceph-mon[35941]: pgmap v1745: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:21:45.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:21:45 smithi165 ceph-mon[36302]: pgmap v1745: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:21:45.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:21:45 smithi119 ceph-mon[31518]: pgmap v1745: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:21:46.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:21:46 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2765.700811 secs 2024-03-28T01:21:46.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:21:46 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2765.700811 secs 2024-03-28T01:21:46.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:21:46 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2765.700811 secs 2024-03-28T01:21:47.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:21:47 smithi132 ceph-mon[35941]: pgmap v1746: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:21:47.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:21:47 smithi165 ceph-mon[36302]: pgmap v1746: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:21:47.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:21:47 smithi119 ceph-mon[31518]: pgmap v1746: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:21:49.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:21:49 smithi132 ceph-mon[35941]: pgmap v1747: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:21:49.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:21:49 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:21:49.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:21:49 smithi165 ceph-mon[36302]: pgmap v1747: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:21:49.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:21:49 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:21:49.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:21:49 smithi119 ceph-mon[31518]: pgmap v1747: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:21:49.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:21:49 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:21:50.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:21:50 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:21:50.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:21:50 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:21:50.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:21:50 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:21:50.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:21:50 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:21:50.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:21:50 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:21:50.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:21:50 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:21:50.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:21:50 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:21:50.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:21:50 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:21:50.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:21:50 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:21:51.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:21:51 smithi132 ceph-mon[35941]: pgmap v1748: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T01:21:51.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:21:51 smithi132 ceph-mon[35941]: pgmap v1749: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:21:51.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:21:51 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2770.700921 secs 2024-03-28T01:21:51.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:21:51 smithi165 ceph-mon[36302]: pgmap v1748: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T01:21:51.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:21:51 smithi165 ceph-mon[36302]: pgmap v1749: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:21:51.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:21:51 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2770.700921 secs 2024-03-28T01:21:51.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:21:51 smithi119 ceph-mon[31518]: pgmap v1748: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T01:21:51.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:21:51 smithi119 ceph-mon[31518]: pgmap v1749: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:21:51.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:21:51 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2770.700921 secs 2024-03-28T01:21:53.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:21:53 smithi132 ceph-mon[35941]: pgmap v1750: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:21:53.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:21:53 smithi165 ceph-mon[36302]: pgmap v1750: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:21:53.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:21:53 smithi119 ceph-mon[31518]: pgmap v1750: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:21:55.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:21:55 smithi132 ceph-mon[35941]: pgmap v1751: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:21:55.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:21:55 smithi165 ceph-mon[36302]: pgmap v1751: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:21:55.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:21:55 smithi119 ceph-mon[31518]: pgmap v1751: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:21:56.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:21:56 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2775.700983 secs 2024-03-28T01:21:56.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:21:56 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2775.700983 secs 2024-03-28T01:21:56.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:21:56 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2775.700983 secs 2024-03-28T01:21:57.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:21:57 smithi165 ceph-mon[36302]: pgmap v1752: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:21:57.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:21:57 smithi119 ceph-mon[31518]: pgmap v1752: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:21:57.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:21:57 smithi132 ceph-mon[35941]: pgmap v1752: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:21:59.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:21:59 smithi165 ceph-mon[36302]: pgmap v1753: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:21:59.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:21:59 smithi119 ceph-mon[31518]: pgmap v1753: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:21:59.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:21:59 smithi132 ceph-mon[35941]: pgmap v1753: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:22:01.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:22:01 smithi165 ceph-mon[36302]: pgmap v1754: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T01:22:01.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:22:01 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2780.701045 secs 2024-03-28T01:22:01.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:22:01 smithi119 ceph-mon[31518]: pgmap v1754: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T01:22:01.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:22:01 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2780.701045 secs 2024-03-28T01:22:01.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:22:01 smithi132 ceph-mon[35941]: pgmap v1754: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T01:22:01.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:22:01 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2780.701045 secs 2024-03-28T01:22:03.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:22:03 smithi165 ceph-mon[36302]: pgmap v1755: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:22:03.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:22:03 smithi119 ceph-mon[31518]: pgmap v1755: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:22:03.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:22:03 smithi132 ceph-mon[35941]: pgmap v1755: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:22:05.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:22:05 smithi165 ceph-mon[36302]: pgmap v1756: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:22:05.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:22:05 smithi119 ceph-mon[31518]: pgmap v1756: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:22:05.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:22:05 smithi132 ceph-mon[35941]: pgmap v1756: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:22:06.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:22:06 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2785.701133 secs 2024-03-28T01:22:06.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:22:06 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2785.701133 secs 2024-03-28T01:22:06.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:22:06 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2785.701133 secs 2024-03-28T01:22:07.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:22:07 smithi165 ceph-mon[36302]: pgmap v1757: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:22:07.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:22:07 smithi119 ceph-mon[31518]: pgmap v1757: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:22:07.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:22:07 smithi132 ceph-mon[35941]: pgmap v1757: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:22:09.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:22:09 smithi165 ceph-mon[36302]: pgmap v1758: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:22:09.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:22:09 smithi119 ceph-mon[31518]: pgmap v1758: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:22:09.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:22:09 smithi132 ceph-mon[35941]: pgmap v1758: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:22:11.499 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:22:11 smithi132 ceph-mon[35941]: pgmap v1759: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:22:11.499 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:22:11 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2790.701194 secs 2024-03-28T01:22:11.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:22:11 smithi165 ceph-mon[36302]: pgmap v1759: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:22:11.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:22:11 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2790.701194 secs 2024-03-28T01:22:11.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:22:11 smithi119 ceph-mon[31518]: pgmap v1759: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:22:11.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:22:11 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2790.701194 secs 2024-03-28T01:22:13.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:22:13 smithi165 ceph-mon[36302]: pgmap v1760: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:22:13.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:22:13 smithi119 ceph-mon[31518]: pgmap v1760: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:22:13.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:22:13 smithi132 ceph-mon[35941]: pgmap v1760: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:22:15.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:22:15 smithi165 ceph-mon[36302]: pgmap v1761: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:22:15.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:22:15 smithi119 ceph-mon[31518]: pgmap v1761: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:22:15.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:22:15 smithi132 ceph-mon[35941]: pgmap v1761: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:22:16.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:22:16 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2795.701283 secs 2024-03-28T01:22:16.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:22:16 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2795.701283 secs 2024-03-28T01:22:16.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:22:16 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2795.701283 secs 2024-03-28T01:22:17.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:22:17 smithi165 ceph-mon[36302]: pgmap v1762: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:22:17.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:22:17 smithi119 ceph-mon[31518]: pgmap v1762: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:22:17.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:22:17 smithi132 ceph-mon[35941]: pgmap v1762: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:22:19.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:22:19 smithi165 ceph-mon[36302]: pgmap v1763: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:22:19.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:22:19 smithi119 ceph-mon[31518]: pgmap v1763: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:22:19.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:22:19 smithi132 ceph-mon[35941]: pgmap v1763: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:22:21.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:22:21 smithi165 ceph-mon[36302]: pgmap v1764: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:22:21.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:22:21 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2800.701343 secs 2024-03-28T01:22:21.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:22:21 smithi119 ceph-mon[31518]: pgmap v1764: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:22:21.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:22:21 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2800.701343 secs 2024-03-28T01:22:21.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:22:21 smithi132 ceph-mon[35941]: pgmap v1764: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:22:21.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:22:21 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2800.701343 secs 2024-03-28T01:22:23.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:22:23 smithi165 ceph-mon[36302]: pgmap v1765: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:22:23.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:22:23 smithi119 ceph-mon[31518]: pgmap v1765: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:22:23.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:22:23 smithi132 ceph-mon[35941]: pgmap v1765: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:22:25.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:22:25 smithi165 ceph-mon[36302]: pgmap v1766: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:22:25.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:22:25 smithi119 ceph-mon[31518]: pgmap v1766: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:22:25.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:22:25 smithi132 ceph-mon[35941]: pgmap v1766: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:22:26.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:22:26 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2805.701431 secs 2024-03-28T01:22:26.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:22:26 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2805.701431 secs 2024-03-28T01:22:26.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:22:26 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2805.701431 secs 2024-03-28T01:22:27.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:22:27 smithi165 ceph-mon[36302]: pgmap v1767: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:22:27.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:22:27 smithi119 ceph-mon[31518]: pgmap v1767: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:22:27.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:22:27 smithi132 ceph-mon[35941]: pgmap v1767: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:22:29.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:22:29 smithi165 ceph-mon[36302]: pgmap v1768: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:22:29.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:22:29 smithi119 ceph-mon[31518]: pgmap v1768: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:22:29.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:22:29 smithi132 ceph-mon[35941]: pgmap v1768: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:22:30.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:22:30 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2810.701519 secs 2024-03-28T01:22:30.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:22:30 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2810.701519 secs 2024-03-28T01:22:30.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:22:30 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2810.701519 secs 2024-03-28T01:22:31.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:22:31 smithi165 ceph-mon[36302]: pgmap v1769: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:22:31.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:22:31 smithi119 ceph-mon[31518]: pgmap v1769: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:22:31.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:22:31 smithi132 ceph-mon[35941]: pgmap v1769: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:22:33.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:22:33 smithi165 ceph-mon[36302]: pgmap v1770: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:22:33.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:22:33 smithi119 ceph-mon[31518]: pgmap v1770: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:22:33.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:22:33 smithi132 ceph-mon[35941]: pgmap v1770: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:22:35.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:22:35 smithi165 ceph-mon[36302]: pgmap v1771: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:22:35.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:22:35 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2815.701608 secs 2024-03-28T01:22:35.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:22:35 smithi119 ceph-mon[31518]: pgmap v1771: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:22:35.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:22:35 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2815.701608 secs 2024-03-28T01:22:35.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:22:35 smithi132 ceph-mon[35941]: pgmap v1771: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:22:35.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:22:35 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2815.701608 secs 2024-03-28T01:22:37.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:22:37 smithi165 ceph-mon[36302]: pgmap v1772: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:22:37.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:22:37 smithi119 ceph-mon[31518]: pgmap v1772: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:22:37.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:22:37 smithi132 ceph-mon[35941]: pgmap v1772: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:22:39.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:22:39 smithi165 ceph-mon[36302]: pgmap v1773: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:22:39.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:22:39 smithi119 ceph-mon[31518]: pgmap v1773: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:22:39.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:22:39 smithi132 ceph-mon[35941]: pgmap v1773: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:22:40.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:22:40 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2820.701698 secs 2024-03-28T01:22:40.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:22:40 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2820.701698 secs 2024-03-28T01:22:40.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:22:40 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2820.701698 secs 2024-03-28T01:22:41.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:22:41 smithi165 ceph-mon[36302]: pgmap v1774: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:22:41.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:22:41 smithi119 ceph-mon[31518]: pgmap v1774: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:22:41.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:22:41 smithi132 ceph-mon[35941]: pgmap v1774: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:22:42.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:22:42 smithi132 ceph-mon[35941]: pgmap v1775: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:22:42.848 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:22:42 smithi119 ceph-mon[31518]: pgmap v1775: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:22:42.877 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:22:42 smithi165 ceph-mon[36302]: pgmap v1775: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:22:44.877 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:22:44 smithi165 ceph-mon[36302]: pgmap v1776: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:22:44.900 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:22:44 smithi119 ceph-mon[31518]: pgmap v1776: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:22:44.947 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:22:44 smithi132 ceph-mon[35941]: pgmap v1776: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:22:45.900 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:22:45 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2825.701787 secs 2024-03-28T01:22:45.947 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:22:45 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2825.701787 secs 2024-03-28T01:22:46.127 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:22:45 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2825.701787 secs 2024-03-28T01:22:46.900 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:22:46 smithi119 ceph-mon[31518]: pgmap v1777: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:22:46.947 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:22:46 smithi132 ceph-mon[35941]: pgmap v1777: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:22:47.127 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:22:46 smithi165 ceph-mon[36302]: pgmap v1777: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:22:48.900 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:22:48 smithi119 ceph-mon[31518]: pgmap v1778: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:22:48.947 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:22:48 smithi132 ceph-mon[35941]: pgmap v1778: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:22:49.127 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:22:48 smithi165 ceph-mon[36302]: pgmap v1778: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:22:50.900 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:22:50 smithi119 ceph-mon[31518]: pgmap v1779: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:22:50.900 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:22:50 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:22:50.900 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:22:50 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2830.701881 secs 2024-03-28T01:22:50.947 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:22:50 smithi132 ceph-mon[35941]: pgmap v1779: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:22:50.947 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:22:50 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:22:50.947 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:22:50 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2830.701881 secs 2024-03-28T01:22:51.127 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:22:50 smithi165 ceph-mon[36302]: pgmap v1779: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:22:51.127 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:22:50 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:22:51.127 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:22:50 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2830.701881 secs 2024-03-28T01:22:52.781 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:22:52 smithi132 ceph-mon[35941]: pgmap v1780: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:22:53.057 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:22:52 smithi119 ceph-mon[31518]: pgmap v1780: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:22:53.127 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:22:52 smithi165 ceph-mon[36302]: pgmap v1780: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:22:55.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:22:55 smithi165 ceph-mon[36302]: pgmap v1781: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:22:55.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:22:55 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:22:55.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:22:55 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:22:55.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:22:55 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2835.701973 secs 2024-03-28T01:22:55.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:22:55 smithi119 ceph-mon[31518]: pgmap v1781: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:22:55.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:22:55 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:22:55.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:22:55 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:22:55.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:22:55 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2835.701973 secs 2024-03-28T01:22:55.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:22:55 smithi132 ceph-mon[35941]: pgmap v1781: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:22:55.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:22:55 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:22:55.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:22:55 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:22:55.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:22:55 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2835.701973 secs 2024-03-28T01:22:57.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:22:57 smithi165 ceph-mon[36302]: pgmap v1782: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:22:57.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:22:57 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:22:57.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:22:57 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:22:57.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:22:57 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:22:57.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:22:57 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:22:57.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:22:57 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:22:57.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:22:57 smithi119 ceph-mon[31518]: pgmap v1782: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:22:57.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:22:57 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:22:57.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:22:57 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:22:57.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:22:57 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:22:57.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:22:57 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:22:57.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:22:57 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:22:57.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:22:57 smithi132 ceph-mon[35941]: pgmap v1782: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:22:57.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:22:57 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:22:57.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:22:57 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:22:57.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:22:57 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:22:57.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:22:57 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:22:57.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:22:57 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:22:58.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:22:58 smithi165 ceph-mon[36302]: pgmap v1783: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-28T01:22:58.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:22:58 smithi119 ceph-mon[31518]: pgmap v1783: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-28T01:22:58.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:22:58 smithi132 ceph-mon[35941]: pgmap v1783: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-28T01:23:00.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:23:00 smithi165 ceph-mon[36302]: pgmap v1784: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-28T01:23:00.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:23:00 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2840.701990 secs 2024-03-28T01:23:00.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:23:00 smithi119 ceph-mon[31518]: pgmap v1784: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-28T01:23:00.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:23:00 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2840.701990 secs 2024-03-28T01:23:00.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:23:00 smithi132 ceph-mon[35941]: pgmap v1784: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-28T01:23:00.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:23:00 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2840.701990 secs 2024-03-28T01:23:02.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:23:02 smithi165 ceph-mon[36302]: pgmap v1785: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-28T01:23:02.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:23:02 smithi119 ceph-mon[31518]: pgmap v1785: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-28T01:23:02.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:23:02 smithi132 ceph-mon[35941]: pgmap v1785: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-28T01:23:04.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:23:04 smithi165 ceph-mon[36302]: pgmap v1786: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-28T01:23:04.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:23:04 smithi119 ceph-mon[31518]: pgmap v1786: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-28T01:23:04.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:23:04 smithi132 ceph-mon[35941]: pgmap v1786: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-28T01:23:05.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:23:05 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2845.702060 secs 2024-03-28T01:23:05.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:23:05 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2845.702060 secs 2024-03-28T01:23:05.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:23:05 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2845.702060 secs 2024-03-28T01:23:06.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:23:06 smithi165 ceph-mon[36302]: pgmap v1787: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T01:23:06.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:23:06 smithi119 ceph-mon[31518]: pgmap v1787: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T01:23:06.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:23:06 smithi132 ceph-mon[35941]: pgmap v1787: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T01:23:07.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:23:07 smithi119 ceph-mon[31518]: pgmap v1788: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-28T01:23:07.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:23:07 smithi132 ceph-mon[35941]: pgmap v1788: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-28T01:23:07.877 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:23:07 smithi165 ceph-mon[36302]: pgmap v1788: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-28T01:23:09.900 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:23:09 smithi119 ceph-mon[31518]: pgmap v1789: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:23:09.947 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:23:09 smithi132 ceph-mon[35941]: pgmap v1789: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:23:10.127 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:23:09 smithi165 ceph-mon[36302]: pgmap v1789: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:23:10.900 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:23:10 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2850.702122 secs 2024-03-28T01:23:10.947 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:23:10 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2850.702122 secs 2024-03-28T01:23:11.028 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:23:10 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2850.702122 secs 2024-03-28T01:23:11.900 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:23:11 smithi119 ceph-mon[31518]: pgmap v1790: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:23:11.947 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:23:11 smithi132 ceph-mon[35941]: pgmap v1790: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:23:12.127 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:23:11 smithi165 ceph-mon[36302]: pgmap v1790: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:23:13.947 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:23:13 smithi132 ceph-mon[35941]: pgmap v1791: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:23:14.080 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:23:13 smithi119 ceph-mon[31518]: pgmap v1791: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:23:14.127 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:23:13 smithi165 ceph-mon[36302]: pgmap v1791: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:23:15.947 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:23:15 smithi132 ceph-mon[35941]: pgmap v1792: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:23:15.947 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:23:15 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2855.702184 secs 2024-03-28T01:23:16.127 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:23:15 smithi165 ceph-mon[36302]: pgmap v1792: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:23:16.127 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:23:15 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2855.702184 secs 2024-03-28T01:23:16.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:23:15 smithi119 ceph-mon[31518]: pgmap v1792: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:23:16.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:23:15 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2855.702184 secs 2024-03-28T01:23:17.947 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:23:17 smithi132 ceph-mon[35941]: pgmap v1793: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:23:18.127 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:23:17 smithi165 ceph-mon[36302]: pgmap v1793: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:23:18.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:23:17 smithi119 ceph-mon[31518]: pgmap v1793: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:23:19.947 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:23:19 smithi132 ceph-mon[35941]: pgmap v1794: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:23:20.127 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:23:19 smithi165 ceph-mon[36302]: pgmap v1794: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:23:20.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:23:19 smithi119 ceph-mon[31518]: pgmap v1794: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:23:20.947 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:23:20 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2860.702273 secs 2024-03-28T01:23:21.127 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:23:20 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2860.702273 secs 2024-03-28T01:23:21.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:23:20 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2860.702273 secs 2024-03-28T01:23:21.947 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:23:21 smithi132 ceph-mon[35941]: pgmap v1795: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:23:22.127 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:23:21 smithi165 ceph-mon[36302]: pgmap v1795: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:23:22.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:23:21 smithi119 ceph-mon[31518]: pgmap v1795: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:23:23.947 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:23:23 smithi132 ceph-mon[35941]: pgmap v1796: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:23:24.127 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:23:23 smithi165 ceph-mon[36302]: pgmap v1796: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:23:24.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:23:23 smithi119 ceph-mon[31518]: pgmap v1796: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:23:25.947 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:23:25 smithi132 ceph-mon[35941]: pgmap v1797: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:23:25.947 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:23:25 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2865.702336 secs 2024-03-28T01:23:26.127 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:23:25 smithi165 ceph-mon[36302]: pgmap v1797: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:23:26.127 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:23:25 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2865.702336 secs 2024-03-28T01:23:26.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:23:25 smithi119 ceph-mon[31518]: pgmap v1797: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:23:26.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:23:25 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2865.702336 secs 2024-03-28T01:23:27.947 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:23:27 smithi132 ceph-mon[35941]: pgmap v1798: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:23:28.127 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:23:27 smithi165 ceph-mon[36302]: pgmap v1798: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:23:28.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:23:27 smithi119 ceph-mon[31518]: pgmap v1798: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:23:29.947 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:23:29 smithi132 ceph-mon[35941]: pgmap v1799: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:23:29.952 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:23:29 smithi119 ceph-mon[31518]: pgmap v1799: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:23:30.127 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:23:29 smithi165 ceph-mon[36302]: pgmap v1799: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:23:31.127 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:23:30 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2870.702397 secs 2024-03-28T01:23:31.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:23:30 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2870.702397 secs 2024-03-28T01:23:31.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:23:30 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2870.702397 secs 2024-03-28T01:23:32.127 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:23:31 smithi165 ceph-mon[36302]: pgmap v1800: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:23:32.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:23:31 smithi119 ceph-mon[31518]: pgmap v1800: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:23:32.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:23:31 smithi132 ceph-mon[35941]: pgmap v1800: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:23:34.127 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:23:33 smithi165 ceph-mon[36302]: pgmap v1801: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:23:34.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:23:33 smithi119 ceph-mon[31518]: pgmap v1801: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:23:34.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:23:33 smithi132 ceph-mon[35941]: pgmap v1801: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:23:36.127 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:23:35 smithi165 ceph-mon[36302]: pgmap v1802: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:23:36.127 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:23:35 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2875.702484 secs 2024-03-28T01:23:36.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:23:35 smithi119 ceph-mon[31518]: pgmap v1802: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:23:36.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:23:35 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2875.702484 secs 2024-03-28T01:23:36.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:23:35 smithi132 ceph-mon[35941]: pgmap v1802: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:23:36.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:23:35 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2875.702484 secs 2024-03-28T01:23:38.127 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:23:37 smithi165 ceph-mon[36302]: pgmap v1803: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:23:38.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:23:37 smithi119 ceph-mon[31518]: pgmap v1803: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:23:38.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:23:37 smithi132 ceph-mon[35941]: pgmap v1803: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:23:40.127 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:23:39 smithi165 ceph-mon[36302]: pgmap v1804: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:23:40.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:23:39 smithi119 ceph-mon[31518]: pgmap v1804: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:23:40.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:23:39 smithi132 ceph-mon[35941]: pgmap v1804: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:23:41.028 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:23:40 smithi165 ceph-mon[36302]: 98 slow requests, 3 included below; oldest blocked for > 2880.702573 secs 2024-03-28T01:23:41.028 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:23:40 smithi165 ceph-mon[36302]: slow request 2880.482598 seconds old, received at 2024-03-28T00:35:39.578920+0000: internal op fragmentdir:mds.0:3783 currently failed to wrlock, waiting 2024-03-28T01:23:41.028 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:23:40 smithi165 ceph-mon[36302]: slow request 2880.473650 seconds old, received at 2024-03-28T00:35:39.587868+0000: client_request(client.25264:33202 create owner_uid=1000, owner_gid=1301 #0x10000000270/file8309 2024-03-28T00:35:39.586399+0000 caller_uid=1000, caller_gid=1301{6,36,1000,1301,}) currently failed to xlock, waiting 2024-03-28T01:23:41.028 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:23:40 smithi165 ceph-mon[36302]: slow request 2880.702573 seconds old, received at 2024-03-28T00:35:39.358945+0000: peer_request:client.25264:11489 currently acquired locks 2024-03-28T01:23:41.098 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:23:40 smithi132 ceph-mon[35941]: 98 slow requests, 3 included below; oldest blocked for > 2880.702573 secs 2024-03-28T01:23:41.098 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:23:40 smithi132 ceph-mon[35941]: slow request 2880.482598 seconds old, received at 2024-03-28T00:35:39.578920+0000: internal op fragmentdir:mds.0:3783 currently failed to wrlock, waiting 2024-03-28T01:23:41.098 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:23:40 smithi132 ceph-mon[35941]: slow request 2880.473650 seconds old, received at 2024-03-28T00:35:39.587868+0000: client_request(client.25264:33202 create owner_uid=1000, owner_gid=1301 #0x10000000270/file8309 2024-03-28T00:35:39.586399+0000 caller_uid=1000, caller_gid=1301{6,36,1000,1301,}) currently failed to xlock, waiting 2024-03-28T01:23:41.098 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:23:40 smithi132 ceph-mon[35941]: slow request 2880.702573 seconds old, received at 2024-03-28T00:35:39.358945+0000: peer_request:client.25264:11489 currently acquired locks 2024-03-28T01:23:41.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:23:40 smithi119 ceph-mon[31518]: 98 slow requests, 3 included below; oldest blocked for > 2880.702573 secs 2024-03-28T01:23:41.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:23:40 smithi119 ceph-mon[31518]: slow request 2880.482598 seconds old, received at 2024-03-28T00:35:39.578920+0000: internal op fragmentdir:mds.0:3783 currently failed to wrlock, waiting 2024-03-28T01:23:41.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:23:40 smithi119 ceph-mon[31518]: slow request 2880.473650 seconds old, received at 2024-03-28T00:35:39.587868+0000: client_request(client.25264:33202 create owner_uid=1000, owner_gid=1301 #0x10000000270/file8309 2024-03-28T00:35:39.586399+0000 caller_uid=1000, caller_gid=1301{6,36,1000,1301,}) currently failed to xlock, waiting 2024-03-28T01:23:41.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:23:40 smithi119 ceph-mon[31518]: slow request 2880.702573 seconds old, received at 2024-03-28T00:35:39.358945+0000: peer_request:client.25264:11489 currently acquired locks 2024-03-28T01:23:42.127 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:23:41 smithi165 ceph-mon[36302]: pgmap v1805: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:23:42.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:23:41 smithi119 ceph-mon[31518]: pgmap v1805: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:23:42.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:23:41 smithi132 ceph-mon[35941]: pgmap v1805: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:23:44.127 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:23:43 smithi165 ceph-mon[36302]: pgmap v1806: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T01:23:44.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:23:43 smithi119 ceph-mon[31518]: pgmap v1806: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T01:23:44.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:23:43 smithi132 ceph-mon[35941]: pgmap v1806: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T01:23:46.127 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:23:45 smithi165 ceph-mon[36302]: pgmap v1807: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:23:46.127 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:23:45 smithi165 ceph-mon[36302]: 98 slow requests, 5 included below; oldest blocked for > 2885.702662 secs 2024-03-28T01:23:46.127 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:23:45 smithi165 ceph-mon[36302]: slow request 2881.609999 seconds old, received at 2024-03-28T00:35:43.451608+0000: internal op fragmentdir:mds.0:3794 currently failed to wrlock, waiting 2024-03-28T01:23:46.127 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:23:45 smithi165 ceph-mon[36302]: slow request 2881.592332 seconds old, received at 2024-03-28T00:35:43.469275+0000: internal op fragmentdir:mds.0:3825 currently failed to wrlock, waiting 2024-03-28T01:23:46.127 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:23:45 smithi165 ceph-mon[36302]: slow request 2881.578240 seconds old, received at 2024-03-28T00:35:43.483367+0000: internal op fragmentdir:mds.0:3857 currently failed to wrlock, waiting 2024-03-28T01:23:46.127 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:23:45 smithi165 ceph-mon[36302]: slow request 2881.565178 seconds old, received at 2024-03-28T00:35:43.496429+0000: internal op fragmentdir:mds.0:3889 currently failed to wrlock, waiting 2024-03-28T01:23:46.127 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:23:45 smithi165 ceph-mon[36302]: slow request 2881.608023 seconds old, received at 2024-03-28T00:35:43.453584+0000: internal op fragmentdir:mds.0:3795 currently failed to wrlock, waiting 2024-03-28T01:23:46.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:23:45 smithi119 ceph-mon[31518]: pgmap v1807: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:23:46.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:23:45 smithi119 ceph-mon[31518]: 98 slow requests, 5 included below; oldest blocked for > 2885.702662 secs 2024-03-28T01:23:46.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:23:45 smithi119 ceph-mon[31518]: slow request 2881.609999 seconds old, received at 2024-03-28T00:35:43.451608+0000: internal op fragmentdir:mds.0:3794 currently failed to wrlock, waiting 2024-03-28T01:23:46.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:23:45 smithi119 ceph-mon[31518]: slow request 2881.592332 seconds old, received at 2024-03-28T00:35:43.469275+0000: internal op fragmentdir:mds.0:3825 currently failed to wrlock, waiting 2024-03-28T01:23:46.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:23:45 smithi119 ceph-mon[31518]: slow request 2881.578240 seconds old, received at 2024-03-28T00:35:43.483367+0000: internal op fragmentdir:mds.0:3857 currently failed to wrlock, waiting 2024-03-28T01:23:46.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:23:45 smithi119 ceph-mon[31518]: slow request 2881.565178 seconds old, received at 2024-03-28T00:35:43.496429+0000: internal op fragmentdir:mds.0:3889 currently failed to wrlock, waiting 2024-03-28T01:23:46.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:23:45 smithi119 ceph-mon[31518]: slow request 2881.608023 seconds old, received at 2024-03-28T00:35:43.453584+0000: internal op fragmentdir:mds.0:3795 currently failed to wrlock, waiting 2024-03-28T01:23:46.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:23:45 smithi132 ceph-mon[35941]: pgmap v1807: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:23:46.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:23:45 smithi132 ceph-mon[35941]: 98 slow requests, 5 included below; oldest blocked for > 2885.702662 secs 2024-03-28T01:23:46.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:23:45 smithi132 ceph-mon[35941]: slow request 2881.609999 seconds old, received at 2024-03-28T00:35:43.451608+0000: internal op fragmentdir:mds.0:3794 currently failed to wrlock, waiting 2024-03-28T01:23:46.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:23:45 smithi132 ceph-mon[35941]: slow request 2881.592332 seconds old, received at 2024-03-28T00:35:43.469275+0000: internal op fragmentdir:mds.0:3825 currently failed to wrlock, waiting 2024-03-28T01:23:46.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:23:45 smithi132 ceph-mon[35941]: slow request 2881.578240 seconds old, received at 2024-03-28T00:35:43.483367+0000: internal op fragmentdir:mds.0:3857 currently failed to wrlock, waiting 2024-03-28T01:23:46.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:23:45 smithi132 ceph-mon[35941]: slow request 2881.565178 seconds old, received at 2024-03-28T00:35:43.496429+0000: internal op fragmentdir:mds.0:3889 currently failed to wrlock, waiting 2024-03-28T01:23:46.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:23:45 smithi132 ceph-mon[35941]: slow request 2881.608023 seconds old, received at 2024-03-28T00:35:43.453584+0000: internal op fragmentdir:mds.0:3795 currently failed to wrlock, waiting 2024-03-28T01:23:48.127 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:23:47 smithi165 ceph-mon[36302]: pgmap v1808: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T01:23:48.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:23:47 smithi119 ceph-mon[31518]: pgmap v1808: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T01:23:48.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:23:47 smithi132 ceph-mon[35941]: pgmap v1808: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T01:23:50.127 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:23:49 smithi165 ceph-mon[36302]: pgmap v1809: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T01:23:50.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:23:49 smithi119 ceph-mon[31518]: pgmap v1809: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T01:23:50.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:23:49 smithi132 ceph-mon[35941]: pgmap v1809: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T01:23:51.127 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:23:50 smithi165 ceph-mon[36302]: 98 slow requests, 5 included below; oldest blocked for > 2890.702724 secs 2024-03-28T01:23:51.127 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:23:50 smithi165 ceph-mon[36302]: slow request 2886.526822 seconds old, received at 2024-03-28T00:35:43.534847+0000: internal op fragmentdir:mds.0:3922 currently failed to wrlock, waiting 2024-03-28T01:23:51.127 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:23:50 smithi165 ceph-mon[36302]: slow request 2886.607564 seconds old, received at 2024-03-28T00:35:43.454104+0000: internal op fragmentdir:mds.0:3796 currently failed to wrlock, waiting 2024-03-28T01:23:51.127 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:23:50 smithi165 ceph-mon[36302]: slow request 2886.591505 seconds old, received at 2024-03-28T00:35:43.470163+0000: internal op fragmentdir:mds.0:3827 currently failed to wrlock, waiting 2024-03-28T01:23:51.127 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:23:50 smithi165 ceph-mon[36302]: slow request 2886.564343 seconds old, received at 2024-03-28T00:35:43.497326+0000: internal op fragmentdir:mds.0:3891 currently failed to wrlock, waiting 2024-03-28T01:23:51.127 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:23:50 smithi165 ceph-mon[36302]: slow request 2886.500092 seconds old, received at 2024-03-28T00:35:43.561577+0000: internal op fragmentdir:mds.0:3923 currently failed to wrlock, waiting 2024-03-28T01:23:51.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:23:50 smithi119 ceph-mon[31518]: 98 slow requests, 5 included below; oldest blocked for > 2890.702724 secs 2024-03-28T01:23:51.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:23:50 smithi119 ceph-mon[31518]: slow request 2886.526822 seconds old, received at 2024-03-28T00:35:43.534847+0000: internal op fragmentdir:mds.0:3922 currently failed to wrlock, waiting 2024-03-28T01:23:51.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:23:50 smithi119 ceph-mon[31518]: slow request 2886.607564 seconds old, received at 2024-03-28T00:35:43.454104+0000: internal op fragmentdir:mds.0:3796 currently failed to wrlock, waiting 2024-03-28T01:23:51.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:23:50 smithi119 ceph-mon[31518]: slow request 2886.591505 seconds old, received at 2024-03-28T00:35:43.470163+0000: internal op fragmentdir:mds.0:3827 currently failed to wrlock, waiting 2024-03-28T01:23:51.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:23:50 smithi119 ceph-mon[31518]: slow request 2886.564343 seconds old, received at 2024-03-28T00:35:43.497326+0000: internal op fragmentdir:mds.0:3891 currently failed to wrlock, waiting 2024-03-28T01:23:51.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:23:50 smithi119 ceph-mon[31518]: slow request 2886.500092 seconds old, received at 2024-03-28T00:35:43.561577+0000: internal op fragmentdir:mds.0:3923 currently failed to wrlock, waiting 2024-03-28T01:23:51.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:23:50 smithi132 ceph-mon[35941]: 98 slow requests, 5 included below; oldest blocked for > 2890.702724 secs 2024-03-28T01:23:51.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:23:50 smithi132 ceph-mon[35941]: slow request 2886.526822 seconds old, received at 2024-03-28T00:35:43.534847+0000: internal op fragmentdir:mds.0:3922 currently failed to wrlock, waiting 2024-03-28T01:23:51.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:23:50 smithi132 ceph-mon[35941]: slow request 2886.607564 seconds old, received at 2024-03-28T00:35:43.454104+0000: internal op fragmentdir:mds.0:3796 currently failed to wrlock, waiting 2024-03-28T01:23:51.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:23:50 smithi132 ceph-mon[35941]: slow request 2886.591505 seconds old, received at 2024-03-28T00:35:43.470163+0000: internal op fragmentdir:mds.0:3827 currently failed to wrlock, waiting 2024-03-28T01:23:51.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:23:50 smithi132 ceph-mon[35941]: slow request 2886.564343 seconds old, received at 2024-03-28T00:35:43.497326+0000: internal op fragmentdir:mds.0:3891 currently failed to wrlock, waiting 2024-03-28T01:23:51.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:23:50 smithi132 ceph-mon[35941]: slow request 2886.500092 seconds old, received at 2024-03-28T00:35:43.561577+0000: internal op fragmentdir:mds.0:3923 currently failed to wrlock, waiting 2024-03-28T01:23:52.126 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:23:51 smithi165 ceph-mon[36302]: pgmap v1810: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:23:52.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:23:51 smithi119 ceph-mon[31518]: pgmap v1810: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:23:52.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:23:51 smithi132 ceph-mon[35941]: pgmap v1810: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:23:54.126 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:23:53 smithi165 ceph-mon[36302]: pgmap v1811: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:23:54.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:23:53 smithi119 ceph-mon[31518]: pgmap v1811: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:23:54.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:23:53 smithi132 ceph-mon[35941]: pgmap v1811: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:23:56.127 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:23:55 smithi165 ceph-mon[36302]: pgmap v1812: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:23:56.127 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:23:55 smithi165 ceph-mon[36302]: 98 slow requests, 5 included below; oldest blocked for > 2895.702786 secs 2024-03-28T01:23:56.127 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:23:55 smithi165 ceph-mon[36302]: slow request 2891.607153 seconds old, received at 2024-03-28T00:35:43.454578+0000: internal op fragmentdir:mds.0:3797 currently failed to wrlock, waiting 2024-03-28T01:23:56.127 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:23:55 smithi165 ceph-mon[36302]: slow request 2891.591125 seconds old, received at 2024-03-28T00:35:43.470605+0000: internal op fragmentdir:mds.0:3828 currently failed to wrlock, waiting 2024-03-28T01:23:56.127 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:23:55 smithi165 ceph-mon[36302]: slow request 2891.577161 seconds old, received at 2024-03-28T00:35:43.484570+0000: internal op fragmentdir:mds.0:3860 currently failed to wrlock, waiting 2024-03-28T01:23:56.127 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:23:55 smithi165 ceph-mon[36302]: slow request 2891.563954 seconds old, received at 2024-03-28T00:35:43.497776+0000: internal op fragmentdir:mds.0:3892 currently failed to wrlock, waiting 2024-03-28T01:23:56.127 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:23:55 smithi165 ceph-mon[36302]: slow request 2891.606676 seconds old, received at 2024-03-28T00:35:43.455055+0000: internal op fragmentdir:mds.0:3798 currently failed to wrlock, waiting 2024-03-28T01:23:56.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:23:55 smithi119 ceph-mon[31518]: pgmap v1812: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:23:56.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:23:55 smithi119 ceph-mon[31518]: 98 slow requests, 5 included below; oldest blocked for > 2895.702786 secs 2024-03-28T01:23:56.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:23:55 smithi119 ceph-mon[31518]: slow request 2891.607153 seconds old, received at 2024-03-28T00:35:43.454578+0000: internal op fragmentdir:mds.0:3797 currently failed to wrlock, waiting 2024-03-28T01:23:56.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:23:55 smithi119 ceph-mon[31518]: slow request 2891.591125 seconds old, received at 2024-03-28T00:35:43.470605+0000: internal op fragmentdir:mds.0:3828 currently failed to wrlock, waiting 2024-03-28T01:23:56.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:23:55 smithi119 ceph-mon[31518]: slow request 2891.577161 seconds old, received at 2024-03-28T00:35:43.484570+0000: internal op fragmentdir:mds.0:3860 currently failed to wrlock, waiting 2024-03-28T01:23:56.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:23:55 smithi119 ceph-mon[31518]: slow request 2891.563954 seconds old, received at 2024-03-28T00:35:43.497776+0000: internal op fragmentdir:mds.0:3892 currently failed to wrlock, waiting 2024-03-28T01:23:56.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:23:55 smithi119 ceph-mon[31518]: slow request 2891.606676 seconds old, received at 2024-03-28T00:35:43.455055+0000: internal op fragmentdir:mds.0:3798 currently failed to wrlock, waiting 2024-03-28T01:23:56.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:23:55 smithi132 ceph-mon[35941]: pgmap v1812: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:23:56.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:23:55 smithi132 ceph-mon[35941]: 98 slow requests, 5 included below; oldest blocked for > 2895.702786 secs 2024-03-28T01:23:56.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:23:55 smithi132 ceph-mon[35941]: slow request 2891.607153 seconds old, received at 2024-03-28T00:35:43.454578+0000: internal op fragmentdir:mds.0:3797 currently failed to wrlock, waiting 2024-03-28T01:23:56.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:23:55 smithi132 ceph-mon[35941]: slow request 2891.591125 seconds old, received at 2024-03-28T00:35:43.470605+0000: internal op fragmentdir:mds.0:3828 currently failed to wrlock, waiting 2024-03-28T01:23:56.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:23:55 smithi132 ceph-mon[35941]: slow request 2891.577161 seconds old, received at 2024-03-28T00:35:43.484570+0000: internal op fragmentdir:mds.0:3860 currently failed to wrlock, waiting 2024-03-28T01:23:56.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:23:55 smithi132 ceph-mon[35941]: slow request 2891.563954 seconds old, received at 2024-03-28T00:35:43.497776+0000: internal op fragmentdir:mds.0:3892 currently failed to wrlock, waiting 2024-03-28T01:23:56.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:23:55 smithi132 ceph-mon[35941]: slow request 2891.606676 seconds old, received at 2024-03-28T00:35:43.455055+0000: internal op fragmentdir:mds.0:3798 currently failed to wrlock, waiting 2024-03-28T01:23:58.127 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:23:57 smithi165 ceph-mon[36302]: pgmap v1813: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:23:58.127 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:23:57 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:23:58.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:23:57 smithi119 ceph-mon[31518]: pgmap v1813: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:23:58.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:23:57 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:23:58.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:23:57 smithi132 ceph-mon[35941]: pgmap v1813: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:23:58.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:23:57 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:23:59.877 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:23:59 smithi165 ceph-mon[36302]: pgmap v1814: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:24:00.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:23:59 smithi119 ceph-mon[31518]: pgmap v1814: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:24:00.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:23:59 smithi132 ceph-mon[35941]: pgmap v1814: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:24:01.114 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:24:00 smithi165 ceph-mon[36302]: 98 slow requests, 5 included below; oldest blocked for > 2900.702876 secs 2024-03-28T01:24:01.115 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:24:00 smithi165 ceph-mon[36302]: slow request 2896.562703 seconds old, received at 2024-03-28T00:35:43.499118+0000: internal op fragmentdir:mds.0:3893 currently failed to wrlock, waiting 2024-03-28T01:24:01.115 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:24:00 smithi165 ceph-mon[36302]: slow request 2896.606292 seconds old, received at 2024-03-28T00:35:43.455529+0000: internal op fragmentdir:mds.0:3799 currently failed to wrlock, waiting 2024-03-28T01:24:01.115 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:24:00 smithi165 ceph-mon[36302]: slow request 2896.576424 seconds old, received at 2024-03-28T00:35:43.485397+0000: internal op fragmentdir:mds.0:3862 currently failed to wrlock, waiting 2024-03-28T01:24:01.115 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:24:00 smithi165 ceph-mon[36302]: slow request 2896.562253 seconds old, received at 2024-03-28T00:35:43.499568+0000: internal op fragmentdir:mds.0:3894 currently failed to wrlock, waiting 2024-03-28T01:24:01.115 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:24:00 smithi165 ceph-mon[36302]: slow request 2896.605814 seconds old, received at 2024-03-28T00:35:43.456007+0000: internal op fragmentdir:mds.0:3800 currently failed to wrlock, waiting 2024-03-28T01:24:01.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:24:00 smithi119 ceph-mon[31518]: 98 slow requests, 5 included below; oldest blocked for > 2900.702876 secs 2024-03-28T01:24:01.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:24:00 smithi119 ceph-mon[31518]: slow request 2896.562703 seconds old, received at 2024-03-28T00:35:43.499118+0000: internal op fragmentdir:mds.0:3893 currently failed to wrlock, waiting 2024-03-28T01:24:01.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:24:00 smithi119 ceph-mon[31518]: slow request 2896.606292 seconds old, received at 2024-03-28T00:35:43.455529+0000: internal op fragmentdir:mds.0:3799 currently failed to wrlock, waiting 2024-03-28T01:24:01.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:24:00 smithi119 ceph-mon[31518]: slow request 2896.576424 seconds old, received at 2024-03-28T00:35:43.485397+0000: internal op fragmentdir:mds.0:3862 currently failed to wrlock, waiting 2024-03-28T01:24:01.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:24:00 smithi119 ceph-mon[31518]: slow request 2896.562253 seconds old, received at 2024-03-28T00:35:43.499568+0000: internal op fragmentdir:mds.0:3894 currently failed to wrlock, waiting 2024-03-28T01:24:01.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:24:00 smithi119 ceph-mon[31518]: slow request 2896.605814 seconds old, received at 2024-03-28T00:35:43.456007+0000: internal op fragmentdir:mds.0:3800 currently failed to wrlock, waiting 2024-03-28T01:24:01.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:24:00 smithi132 ceph-mon[35941]: 98 slow requests, 5 included below; oldest blocked for > 2900.702876 secs 2024-03-28T01:24:01.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:24:00 smithi132 ceph-mon[35941]: slow request 2896.562703 seconds old, received at 2024-03-28T00:35:43.499118+0000: internal op fragmentdir:mds.0:3893 currently failed to wrlock, waiting 2024-03-28T01:24:01.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:24:00 smithi132 ceph-mon[35941]: slow request 2896.606292 seconds old, received at 2024-03-28T00:35:43.455529+0000: internal op fragmentdir:mds.0:3799 currently failed to wrlock, waiting 2024-03-28T01:24:01.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:24:00 smithi132 ceph-mon[35941]: slow request 2896.576424 seconds old, received at 2024-03-28T00:35:43.485397+0000: internal op fragmentdir:mds.0:3862 currently failed to wrlock, waiting 2024-03-28T01:24:01.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:24:00 smithi132 ceph-mon[35941]: slow request 2896.562253 seconds old, received at 2024-03-28T00:35:43.499568+0000: internal op fragmentdir:mds.0:3894 currently failed to wrlock, waiting 2024-03-28T01:24:01.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:24:00 smithi132 ceph-mon[35941]: slow request 2896.605814 seconds old, received at 2024-03-28T00:35:43.456007+0000: internal op fragmentdir:mds.0:3800 currently failed to wrlock, waiting 2024-03-28T01:24:02.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:24:02 smithi132 ceph-mon[35941]: pgmap v1815: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:24:02.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:24:02 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:24:02.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:24:02 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:24:02.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:24:02 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:24:02.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:24:02 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:24:02.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:24:02 smithi132 ceph-mon[35941]: pgmap v1816: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-03-28T01:24:02.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:24:02 smithi132 ceph-mon[35941]: pgmap v1817: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-03-28T01:24:02.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:24:02 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:24:02.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:24:02 smithi165 ceph-mon[36302]: pgmap v1815: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:24:02.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:24:02 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:24:02.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:24:02 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:24:02.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:24:02 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:24:02.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:24:02 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:24:02.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:24:02 smithi165 ceph-mon[36302]: pgmap v1816: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-03-28T01:24:02.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:24:02 smithi165 ceph-mon[36302]: pgmap v1817: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-03-28T01:24:02.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:24:02 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:24:02.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:24:02 smithi119 ceph-mon[31518]: pgmap v1815: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:24:02.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:24:02 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:24:02.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:24:02 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:24:02.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:24:02 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:24:02.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:24:02 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:24:02.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:24:02 smithi119 ceph-mon[31518]: pgmap v1816: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-03-28T01:24:02.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:24:02 smithi119 ceph-mon[31518]: pgmap v1817: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-03-28T01:24:02.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:24:02 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:24:04.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:24:04 smithi165 ceph-mon[36302]: pgmap v1818: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-28T01:24:04.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:24:04 smithi119 ceph-mon[31518]: pgmap v1818: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-28T01:24:04.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:24:04 smithi132 ceph-mon[35941]: pgmap v1818: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-28T01:24:06.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:24:06 smithi165 ceph-mon[36302]: 98 slow requests, 5 included below; oldest blocked for > 2905.702940 secs 2024-03-28T01:24:06.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:24:06 smithi165 ceph-mon[36302]: slow request 2901.590212 seconds old, received at 2024-03-28T00:35:43.471672+0000: internal op fragmentdir:mds.0:3831 currently failed to wrlock, waiting 2024-03-28T01:24:06.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:24:06 smithi165 ceph-mon[36302]: slow request 2901.576096 seconds old, received at 2024-03-28T00:35:43.485788+0000: internal op fragmentdir:mds.0:3863 currently failed to wrlock, waiting 2024-03-28T01:24:06.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:24:06 smithi165 ceph-mon[36302]: slow request 2901.561853 seconds old, received at 2024-03-28T00:35:43.500032+0000: internal op fragmentdir:mds.0:3895 currently failed to wrlock, waiting 2024-03-28T01:24:06.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:24:06 smithi165 ceph-mon[36302]: slow request 2901.605421 seconds old, received at 2024-03-28T00:35:43.456464+0000: internal op fragmentdir:mds.0:3801 currently failed to wrlock, waiting 2024-03-28T01:24:06.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:24:06 smithi165 ceph-mon[36302]: slow request 2901.589754 seconds old, received at 2024-03-28T00:35:43.472131+0000: internal op fragmentdir:mds.0:3832 currently failed to wrlock, waiting 2024-03-28T01:24:06.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:24:06 smithi165 ceph-mon[36302]: pgmap v1819: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-03-28T01:24:06.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:24:06 smithi119 ceph-mon[31518]: 98 slow requests, 5 included below; oldest blocked for > 2905.702940 secs 2024-03-28T01:24:06.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:24:06 smithi119 ceph-mon[31518]: slow request 2901.590212 seconds old, received at 2024-03-28T00:35:43.471672+0000: internal op fragmentdir:mds.0:3831 currently failed to wrlock, waiting 2024-03-28T01:24:06.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:24:06 smithi119 ceph-mon[31518]: slow request 2901.576096 seconds old, received at 2024-03-28T00:35:43.485788+0000: internal op fragmentdir:mds.0:3863 currently failed to wrlock, waiting 2024-03-28T01:24:06.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:24:06 smithi119 ceph-mon[31518]: slow request 2901.561853 seconds old, received at 2024-03-28T00:35:43.500032+0000: internal op fragmentdir:mds.0:3895 currently failed to wrlock, waiting 2024-03-28T01:24:06.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:24:06 smithi119 ceph-mon[31518]: slow request 2901.605421 seconds old, received at 2024-03-28T00:35:43.456464+0000: internal op fragmentdir:mds.0:3801 currently failed to wrlock, waiting 2024-03-28T01:24:06.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:24:06 smithi119 ceph-mon[31518]: slow request 2901.589754 seconds old, received at 2024-03-28T00:35:43.472131+0000: internal op fragmentdir:mds.0:3832 currently failed to wrlock, waiting 2024-03-28T01:24:06.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:24:06 smithi119 ceph-mon[31518]: pgmap v1819: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-03-28T01:24:06.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:24:06 smithi132 ceph-mon[35941]: 98 slow requests, 5 included below; oldest blocked for > 2905.702940 secs 2024-03-28T01:24:06.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:24:06 smithi132 ceph-mon[35941]: slow request 2901.590212 seconds old, received at 2024-03-28T00:35:43.471672+0000: internal op fragmentdir:mds.0:3831 currently failed to wrlock, waiting 2024-03-28T01:24:06.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:24:06 smithi132 ceph-mon[35941]: slow request 2901.576096 seconds old, received at 2024-03-28T00:35:43.485788+0000: internal op fragmentdir:mds.0:3863 currently failed to wrlock, waiting 2024-03-28T01:24:06.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:24:06 smithi132 ceph-mon[35941]: slow request 2901.561853 seconds old, received at 2024-03-28T00:35:43.500032+0000: internal op fragmentdir:mds.0:3895 currently failed to wrlock, waiting 2024-03-28T01:24:06.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:24:06 smithi132 ceph-mon[35941]: slow request 2901.605421 seconds old, received at 2024-03-28T00:35:43.456464+0000: internal op fragmentdir:mds.0:3801 currently failed to wrlock, waiting 2024-03-28T01:24:06.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:24:06 smithi132 ceph-mon[35941]: slow request 2901.589754 seconds old, received at 2024-03-28T00:35:43.472131+0000: internal op fragmentdir:mds.0:3832 currently failed to wrlock, waiting 2024-03-28T01:24:06.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:24:06 smithi132 ceph-mon[35941]: pgmap v1819: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-03-28T01:24:08.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:24:08 smithi165 ceph-mon[36302]: pgmap v1820: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-03-28T01:24:08.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:24:08 smithi119 ceph-mon[31518]: pgmap v1820: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-03-28T01:24:08.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:24:08 smithi132 ceph-mon[35941]: pgmap v1820: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-03-28T01:24:10.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:24:10 smithi165 ceph-mon[36302]: pgmap v1821: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 5 op/s 2024-03-28T01:24:10.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:24:10 smithi119 ceph-mon[31518]: pgmap v1821: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 5 op/s 2024-03-28T01:24:10.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:24:10 smithi132 ceph-mon[35941]: pgmap v1821: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 5 op/s 2024-03-28T01:24:11.528 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:24:11 smithi165 ceph-mon[36302]: 98 slow requests, 5 included below; oldest blocked for > 2910.703025 secs 2024-03-28T01:24:11.528 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:24:11 smithi165 ceph-mon[36302]: slow request 2906.604989 seconds old, received at 2024-03-28T00:35:43.456981+0000: internal op fragmentdir:mds.0:3802 currently failed to wrlock, waiting 2024-03-28T01:24:11.528 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:24:11 smithi165 ceph-mon[36302]: slow request 2906.575380 seconds old, received at 2024-03-28T00:35:43.486590+0000: internal op fragmentdir:mds.0:3865 currently failed to wrlock, waiting 2024-03-28T01:24:11.528 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:24:11 smithi165 ceph-mon[36302]: slow request 2906.561065 seconds old, received at 2024-03-28T00:35:43.500904+0000: internal op fragmentdir:mds.0:3897 currently failed to wrlock, waiting 2024-03-28T01:24:11.528 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:24:11 smithi165 ceph-mon[36302]: slow request 2906.604539 seconds old, received at 2024-03-28T00:35:43.457430+0000: internal op fragmentdir:mds.0:3803 currently failed to wrlock, waiting 2024-03-28T01:24:11.528 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:24:11 smithi165 ceph-mon[36302]: slow request 2906.588954 seconds old, received at 2024-03-28T00:35:43.473016+0000: internal op fragmentdir:mds.0:3834 currently failed to wrlock, waiting 2024-03-28T01:24:11.598 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:24:11 smithi132 ceph-mon[35941]: 98 slow requests, 5 included below; oldest blocked for > 2910.703025 secs 2024-03-28T01:24:11.599 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:24:11 smithi132 ceph-mon[35941]: slow request 2906.604989 seconds old, received at 2024-03-28T00:35:43.456981+0000: internal op fragmentdir:mds.0:3802 currently failed to wrlock, waiting 2024-03-28T01:24:11.599 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:24:11 smithi132 ceph-mon[35941]: slow request 2906.575380 seconds old, received at 2024-03-28T00:35:43.486590+0000: internal op fragmentdir:mds.0:3865 currently failed to wrlock, waiting 2024-03-28T01:24:11.599 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:24:11 smithi132 ceph-mon[35941]: slow request 2906.561065 seconds old, received at 2024-03-28T00:35:43.500904+0000: internal op fragmentdir:mds.0:3897 currently failed to wrlock, waiting 2024-03-28T01:24:11.599 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:24:11 smithi132 ceph-mon[35941]: slow request 2906.604539 seconds old, received at 2024-03-28T00:35:43.457430+0000: internal op fragmentdir:mds.0:3803 currently failed to wrlock, waiting 2024-03-28T01:24:11.599 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:24:11 smithi132 ceph-mon[35941]: slow request 2906.588954 seconds old, received at 2024-03-28T00:35:43.473016+0000: internal op fragmentdir:mds.0:3834 currently failed to wrlock, waiting 2024-03-28T01:24:11.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:24:11 smithi119 ceph-mon[31518]: 98 slow requests, 5 included below; oldest blocked for > 2910.703025 secs 2024-03-28T01:24:11.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:24:11 smithi119 ceph-mon[31518]: slow request 2906.604989 seconds old, received at 2024-03-28T00:35:43.456981+0000: internal op fragmentdir:mds.0:3802 currently failed to wrlock, waiting 2024-03-28T01:24:11.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:24:11 smithi119 ceph-mon[31518]: slow request 2906.575380 seconds old, received at 2024-03-28T00:35:43.486590+0000: internal op fragmentdir:mds.0:3865 currently failed to wrlock, waiting 2024-03-28T01:24:11.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:24:11 smithi119 ceph-mon[31518]: slow request 2906.561065 seconds old, received at 2024-03-28T00:35:43.500904+0000: internal op fragmentdir:mds.0:3897 currently failed to wrlock, waiting 2024-03-28T01:24:11.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:24:11 smithi119 ceph-mon[31518]: slow request 2906.604539 seconds old, received at 2024-03-28T00:35:43.457430+0000: internal op fragmentdir:mds.0:3803 currently failed to wrlock, waiting 2024-03-28T01:24:11.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:24:11 smithi119 ceph-mon[31518]: slow request 2906.588954 seconds old, received at 2024-03-28T00:35:43.473016+0000: internal op fragmentdir:mds.0:3834 currently failed to wrlock, waiting 2024-03-28T01:24:12.610 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:24:12 smithi132 ceph-mon[35941]: pgmap v1822: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T01:24:12.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:24:12 smithi165 ceph-mon[36302]: pgmap v1822: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T01:24:12.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:24:12 smithi119 ceph-mon[31518]: pgmap v1822: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T01:24:14.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:24:14 smithi165 ceph-mon[36302]: pgmap v1823: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:24:14.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:24:14 smithi119 ceph-mon[31518]: pgmap v1823: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:24:14.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:24:14 smithi132 ceph-mon[35941]: pgmap v1823: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:24:16.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:24:16 smithi165 ceph-mon[36302]: 98 slow requests, 5 included below; oldest blocked for > 2915.703063 secs 2024-03-28T01:24:16.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:24:16 smithi165 ceph-mon[36302]: slow request 2911.604071 seconds old, received at 2024-03-28T00:35:43.457937+0000: internal op fragmentdir:mds.0:3804 currently failed to wrlock, waiting 2024-03-28T01:24:16.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:24:16 smithi165 ceph-mon[36302]: slow request 2911.574622 seconds old, received at 2024-03-28T00:35:43.487385+0000: internal op fragmentdir:mds.0:3867 currently failed to wrlock, waiting 2024-03-28T01:24:16.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:24:16 smithi165 ceph-mon[36302]: slow request 2911.560236 seconds old, received at 2024-03-28T00:35:43.501771+0000: internal op fragmentdir:mds.0:3899 currently failed to wrlock, waiting 2024-03-28T01:24:16.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:24:16 smithi165 ceph-mon[36302]: slow request 2911.603582 seconds old, received at 2024-03-28T00:35:43.458426+0000: internal op fragmentdir:mds.0:3805 currently failed to wrlock, waiting 2024-03-28T01:24:16.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:24:16 smithi165 ceph-mon[36302]: slow request 2911.588105 seconds old, received at 2024-03-28T00:35:43.473902+0000: internal op fragmentdir:mds.0:3836 currently failed to wrlock, waiting 2024-03-28T01:24:16.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:24:16 smithi165 ceph-mon[36302]: pgmap v1824: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:24:16.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:24:16 smithi119 ceph-mon[31518]: 98 slow requests, 5 included below; oldest blocked for > 2915.703063 secs 2024-03-28T01:24:16.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:24:16 smithi119 ceph-mon[31518]: slow request 2911.604071 seconds old, received at 2024-03-28T00:35:43.457937+0000: internal op fragmentdir:mds.0:3804 currently failed to wrlock, waiting 2024-03-28T01:24:16.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:24:16 smithi119 ceph-mon[31518]: slow request 2911.574622 seconds old, received at 2024-03-28T00:35:43.487385+0000: internal op fragmentdir:mds.0:3867 currently failed to wrlock, waiting 2024-03-28T01:24:16.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:24:16 smithi119 ceph-mon[31518]: slow request 2911.560236 seconds old, received at 2024-03-28T00:35:43.501771+0000: internal op fragmentdir:mds.0:3899 currently failed to wrlock, waiting 2024-03-28T01:24:16.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:24:16 smithi119 ceph-mon[31518]: slow request 2911.603582 seconds old, received at 2024-03-28T00:35:43.458426+0000: internal op fragmentdir:mds.0:3805 currently failed to wrlock, waiting 2024-03-28T01:24:16.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:24:16 smithi119 ceph-mon[31518]: slow request 2911.588105 seconds old, received at 2024-03-28T00:35:43.473902+0000: internal op fragmentdir:mds.0:3836 currently failed to wrlock, waiting 2024-03-28T01:24:16.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:24:16 smithi119 ceph-mon[31518]: pgmap v1824: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:24:16.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:24:16 smithi132 ceph-mon[35941]: 98 slow requests, 5 included below; oldest blocked for > 2915.703063 secs 2024-03-28T01:24:16.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:24:16 smithi132 ceph-mon[35941]: slow request 2911.604071 seconds old, received at 2024-03-28T00:35:43.457937+0000: internal op fragmentdir:mds.0:3804 currently failed to wrlock, waiting 2024-03-28T01:24:16.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:24:16 smithi132 ceph-mon[35941]: slow request 2911.574622 seconds old, received at 2024-03-28T00:35:43.487385+0000: internal op fragmentdir:mds.0:3867 currently failed to wrlock, waiting 2024-03-28T01:24:16.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:24:16 smithi132 ceph-mon[35941]: slow request 2911.560236 seconds old, received at 2024-03-28T00:35:43.501771+0000: internal op fragmentdir:mds.0:3899 currently failed to wrlock, waiting 2024-03-28T01:24:16.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:24:16 smithi132 ceph-mon[35941]: slow request 2911.603582 seconds old, received at 2024-03-28T00:35:43.458426+0000: internal op fragmentdir:mds.0:3805 currently failed to wrlock, waiting 2024-03-28T01:24:16.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:24:16 smithi132 ceph-mon[35941]: slow request 2911.588105 seconds old, received at 2024-03-28T00:35:43.473902+0000: internal op fragmentdir:mds.0:3836 currently failed to wrlock, waiting 2024-03-28T01:24:16.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:24:16 smithi132 ceph-mon[35941]: pgmap v1824: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:24:18.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:24:18 smithi165 ceph-mon[36302]: pgmap v1825: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:24:18.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:24:18 smithi119 ceph-mon[31518]: pgmap v1825: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:24:18.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:24:18 smithi132 ceph-mon[35941]: pgmap v1825: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:24:20.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:24:20 smithi165 ceph-mon[36302]: pgmap v1826: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-28T01:24:20.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:24:20 smithi119 ceph-mon[31518]: pgmap v1826: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-28T01:24:20.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:24:20 smithi132 ceph-mon[35941]: pgmap v1826: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-28T01:24:21.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:24:21 smithi165 ceph-mon[36302]: 98 slow requests, 5 included below; oldest blocked for > 2920.703125 secs 2024-03-28T01:24:21.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:24:21 smithi165 ceph-mon[36302]: slow request 2916.574288 seconds old, received at 2024-03-28T00:35:43.487781+0000: internal op fragmentdir:mds.0:3868 currently failed to wrlock, waiting 2024-03-28T01:24:21.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:24:21 smithi165 ceph-mon[36302]: slow request 2916.603156 seconds old, received at 2024-03-28T00:35:43.458913+0000: internal op fragmentdir:mds.0:3806 currently failed to wrlock, waiting 2024-03-28T01:24:21.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:24:21 smithi165 ceph-mon[36302]: slow request 2916.587731 seconds old, received at 2024-03-28T00:35:43.474338+0000: internal op fragmentdir:mds.0:3837 currently failed to wrlock, waiting 2024-03-28T01:24:21.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:24:21 smithi165 ceph-mon[36302]: slow request 2916.559414 seconds old, received at 2024-03-28T00:35:43.502655+0000: internal op fragmentdir:mds.0:3901 currently failed to wrlock, waiting 2024-03-28T01:24:21.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:24:21 smithi165 ceph-mon[36302]: slow request 2916.603802 seconds old, received at 2024-03-28T00:35:43.458267+0000: client_request(client.25264:34113 create owner_uid=1000, owner_gid=1301 #0x10000000272/file7594 2024-03-28T00:35:43.457305+0000 caller_uid=1000, caller_gid=1301{6,36,1000,1301,}) currently failed to xlock, waiting 2024-03-28T01:24:21.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:24:21 smithi119 ceph-mon[31518]: 98 slow requests, 5 included below; oldest blocked for > 2920.703125 secs 2024-03-28T01:24:21.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:24:21 smithi119 ceph-mon[31518]: slow request 2916.574288 seconds old, received at 2024-03-28T00:35:43.487781+0000: internal op fragmentdir:mds.0:3868 currently failed to wrlock, waiting 2024-03-28T01:24:21.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:24:21 smithi119 ceph-mon[31518]: slow request 2916.603156 seconds old, received at 2024-03-28T00:35:43.458913+0000: internal op fragmentdir:mds.0:3806 currently failed to wrlock, waiting 2024-03-28T01:24:21.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:24:21 smithi119 ceph-mon[31518]: slow request 2916.587731 seconds old, received at 2024-03-28T00:35:43.474338+0000: internal op fragmentdir:mds.0:3837 currently failed to wrlock, waiting 2024-03-28T01:24:21.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:24:21 smithi119 ceph-mon[31518]: slow request 2916.559414 seconds old, received at 2024-03-28T00:35:43.502655+0000: internal op fragmentdir:mds.0:3901 currently failed to wrlock, waiting 2024-03-28T01:24:21.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:24:21 smithi119 ceph-mon[31518]: slow request 2916.603802 seconds old, received at 2024-03-28T00:35:43.458267+0000: client_request(client.25264:34113 create owner_uid=1000, owner_gid=1301 #0x10000000272/file7594 2024-03-28T00:35:43.457305+0000 caller_uid=1000, caller_gid=1301{6,36,1000,1301,}) currently failed to xlock, waiting 2024-03-28T01:24:21.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:24:21 smithi132 ceph-mon[35941]: 98 slow requests, 5 included below; oldest blocked for > 2920.703125 secs 2024-03-28T01:24:21.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:24:21 smithi132 ceph-mon[35941]: slow request 2916.574288 seconds old, received at 2024-03-28T00:35:43.487781+0000: internal op fragmentdir:mds.0:3868 currently failed to wrlock, waiting 2024-03-28T01:24:21.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:24:21 smithi132 ceph-mon[35941]: slow request 2916.603156 seconds old, received at 2024-03-28T00:35:43.458913+0000: internal op fragmentdir:mds.0:3806 currently failed to wrlock, waiting 2024-03-28T01:24:21.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:24:21 smithi132 ceph-mon[35941]: slow request 2916.587731 seconds old, received at 2024-03-28T00:35:43.474338+0000: internal op fragmentdir:mds.0:3837 currently failed to wrlock, waiting 2024-03-28T01:24:21.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:24:21 smithi132 ceph-mon[35941]: slow request 2916.559414 seconds old, received at 2024-03-28T00:35:43.502655+0000: internal op fragmentdir:mds.0:3901 currently failed to wrlock, waiting 2024-03-28T01:24:21.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:24:21 smithi132 ceph-mon[35941]: slow request 2916.603802 seconds old, received at 2024-03-28T00:35:43.458267+0000: client_request(client.25264:34113 create owner_uid=1000, owner_gid=1301 #0x10000000272/file7594 2024-03-28T00:35:43.457305+0000 caller_uid=1000, caller_gid=1301{6,36,1000,1301,}) currently failed to xlock, waiting 2024-03-28T01:24:22.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:24:22 smithi165 ceph-mon[36302]: pgmap v1827: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-03-28T01:24:22.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:24:22 smithi119 ceph-mon[31518]: pgmap v1827: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-03-28T01:24:22.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:24:22 smithi132 ceph-mon[35941]: pgmap v1827: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-03-28T01:24:24.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:24:24 smithi165 ceph-mon[36302]: pgmap v1828: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:24:24.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:24:24 smithi119 ceph-mon[31518]: pgmap v1828: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:24:24.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:24:24 smithi132 ceph-mon[35941]: pgmap v1828: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:24:25.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:24:25 smithi165 ceph-mon[36302]: 98 slow requests, 5 included below; oldest blocked for > 2925.703186 secs 2024-03-28T01:24:25.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:24:25 smithi165 ceph-mon[36302]: slow request 2921.573825 seconds old, received at 2024-03-28T00:35:43.488305+0000: internal op fragmentdir:mds.0:3870 currently failed to wrlock, waiting 2024-03-28T01:24:25.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:24:25 smithi165 ceph-mon[36302]: slow request 2921.600282 seconds old, received at 2024-03-28T00:35:43.461848+0000: internal op fragmentdir:mds.0:3807 currently failed to wrlock, waiting 2024-03-28T01:24:25.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:24:25 smithi165 ceph-mon[36302]: slow request 2921.586935 seconds old, received at 2024-03-28T00:35:43.475195+0000: internal op fragmentdir:mds.0:3839 currently failed to wrlock, waiting 2024-03-28T01:24:25.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:24:25 smithi165 ceph-mon[36302]: slow request 2921.573405 seconds old, received at 2024-03-28T00:35:43.488725+0000: internal op fragmentdir:mds.0:3871 currently failed to wrlock, waiting 2024-03-28T01:24:25.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:24:25 smithi165 ceph-mon[36302]: slow request 2921.558822 seconds old, received at 2024-03-28T00:35:43.503308+0000: internal op fragmentdir:mds.0:3903 currently failed to wrlock, waiting 2024-03-28T01:24:25.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:24:25 smithi119 ceph-mon[31518]: 98 slow requests, 5 included below; oldest blocked for > 2925.703186 secs 2024-03-28T01:24:25.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:24:25 smithi119 ceph-mon[31518]: slow request 2921.573825 seconds old, received at 2024-03-28T00:35:43.488305+0000: internal op fragmentdir:mds.0:3870 currently failed to wrlock, waiting 2024-03-28T01:24:25.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:24:25 smithi119 ceph-mon[31518]: slow request 2921.600282 seconds old, received at 2024-03-28T00:35:43.461848+0000: internal op fragmentdir:mds.0:3807 currently failed to wrlock, waiting 2024-03-28T01:24:25.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:24:25 smithi119 ceph-mon[31518]: slow request 2921.586935 seconds old, received at 2024-03-28T00:35:43.475195+0000: internal op fragmentdir:mds.0:3839 currently failed to wrlock, waiting 2024-03-28T01:24:25.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:24:25 smithi119 ceph-mon[31518]: slow request 2921.573405 seconds old, received at 2024-03-28T00:35:43.488725+0000: internal op fragmentdir:mds.0:3871 currently failed to wrlock, waiting 2024-03-28T01:24:25.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:24:25 smithi119 ceph-mon[31518]: slow request 2921.558822 seconds old, received at 2024-03-28T00:35:43.503308+0000: internal op fragmentdir:mds.0:3903 currently failed to wrlock, waiting 2024-03-28T01:24:25.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:24:25 smithi132 ceph-mon[35941]: 98 slow requests, 5 included below; oldest blocked for > 2925.703186 secs 2024-03-28T01:24:25.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:24:25 smithi132 ceph-mon[35941]: slow request 2921.573825 seconds old, received at 2024-03-28T00:35:43.488305+0000: internal op fragmentdir:mds.0:3870 currently failed to wrlock, waiting 2024-03-28T01:24:25.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:24:25 smithi132 ceph-mon[35941]: slow request 2921.600282 seconds old, received at 2024-03-28T00:35:43.461848+0000: internal op fragmentdir:mds.0:3807 currently failed to wrlock, waiting 2024-03-28T01:24:25.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:24:25 smithi132 ceph-mon[35941]: slow request 2921.586935 seconds old, received at 2024-03-28T00:35:43.475195+0000: internal op fragmentdir:mds.0:3839 currently failed to wrlock, waiting 2024-03-28T01:24:25.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:24:25 smithi132 ceph-mon[35941]: slow request 2921.573405 seconds old, received at 2024-03-28T00:35:43.488725+0000: internal op fragmentdir:mds.0:3871 currently failed to wrlock, waiting 2024-03-28T01:24:25.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:24:25 smithi132 ceph-mon[35941]: slow request 2921.558822 seconds old, received at 2024-03-28T00:35:43.503308+0000: internal op fragmentdir:mds.0:3903 currently failed to wrlock, waiting 2024-03-28T01:24:26.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:24:26 smithi165 ceph-mon[36302]: pgmap v1829: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:24:26.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:24:26 smithi119 ceph-mon[31518]: pgmap v1829: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:24:26.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:24:26 smithi132 ceph-mon[35941]: pgmap v1829: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:24:28.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:24:28 smithi165 ceph-mon[36302]: pgmap v1830: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:24:28.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:24:28 smithi119 ceph-mon[31518]: pgmap v1830: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:24:28.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:24:28 smithi132 ceph-mon[35941]: pgmap v1830: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:24:30.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:24:30 smithi165 ceph-mon[36302]: pgmap v1831: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-28T01:24:30.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:24:30 smithi165 ceph-mon[36302]: 98 slow requests, 5 included below; oldest blocked for > 2930.703275 secs 2024-03-28T01:24:30.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:24:30 smithi165 ceph-mon[36302]: slow request 2909.685387 seconds old, received at 2024-03-28T00:36:00.376832+0000: client_request(client.25355:96 mksnap #0x1//scheduled-2024-03-28-00_36_00_UTC 2024-03-28T00:36:00.376777+0000 caller_uid=0, caller_gid=0{}) currently failed to acquire_locks 2024-03-28T01:24:30.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:24:30 smithi165 ceph-mon[36302]: slow request 2926.599917 seconds old, received at 2024-03-28T00:35:43.462302+0000: internal op fragmentdir:mds.0:3808 currently failed to wrlock, waiting 2024-03-28T01:24:30.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:24:30 smithi165 ceph-mon[36302]: slow request 2926.663524 seconds old, received at 2024-03-28T00:35:43.398696+0000: internal op fragmentdir:mds.0:3785 currently failed to wrlock, waiting 2024-03-28T01:24:30.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:24:30 smithi165 ceph-mon[36302]: slow request 2926.599455 seconds old, received at 2024-03-28T00:35:43.462764+0000: internal op fragmentdir:mds.0:3809 currently failed to wrlock, waiting 2024-03-28T01:24:30.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:24:30 smithi165 ceph-mon[36302]: slow request 2926.586125 seconds old, received at 2024-03-28T00:35:43.476094+0000: internal op fragmentdir:mds.0:3841 currently failed to wrlock, waiting 2024-03-28T01:24:30.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:24:30 smithi119 ceph-mon[31518]: pgmap v1831: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-28T01:24:30.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:24:30 smithi119 ceph-mon[31518]: 98 slow requests, 5 included below; oldest blocked for > 2930.703275 secs 2024-03-28T01:24:30.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:24:30 smithi119 ceph-mon[31518]: slow request 2909.685387 seconds old, received at 2024-03-28T00:36:00.376832+0000: client_request(client.25355:96 mksnap #0x1//scheduled-2024-03-28-00_36_00_UTC 2024-03-28T00:36:00.376777+0000 caller_uid=0, caller_gid=0{}) currently failed to acquire_locks 2024-03-28T01:24:30.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:24:30 smithi119 ceph-mon[31518]: slow request 2926.599917 seconds old, received at 2024-03-28T00:35:43.462302+0000: internal op fragmentdir:mds.0:3808 currently failed to wrlock, waiting 2024-03-28T01:24:30.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:24:30 smithi119 ceph-mon[31518]: slow request 2926.663524 seconds old, received at 2024-03-28T00:35:43.398696+0000: internal op fragmentdir:mds.0:3785 currently failed to wrlock, waiting 2024-03-28T01:24:30.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:24:30 smithi119 ceph-mon[31518]: slow request 2926.599455 seconds old, received at 2024-03-28T00:35:43.462764+0000: internal op fragmentdir:mds.0:3809 currently failed to wrlock, waiting 2024-03-28T01:24:30.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:24:30 smithi119 ceph-mon[31518]: slow request 2926.586125 seconds old, received at 2024-03-28T00:35:43.476094+0000: internal op fragmentdir:mds.0:3841 currently failed to wrlock, waiting 2024-03-28T01:24:30.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:24:30 smithi132 ceph-mon[35941]: pgmap v1831: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-28T01:24:30.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:24:30 smithi132 ceph-mon[35941]: 98 slow requests, 5 included below; oldest blocked for > 2930.703275 secs 2024-03-28T01:24:30.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:24:30 smithi132 ceph-mon[35941]: slow request 2909.685387 seconds old, received at 2024-03-28T00:36:00.376832+0000: client_request(client.25355:96 mksnap #0x1//scheduled-2024-03-28-00_36_00_UTC 2024-03-28T00:36:00.376777+0000 caller_uid=0, caller_gid=0{}) currently failed to acquire_locks 2024-03-28T01:24:30.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:24:30 smithi132 ceph-mon[35941]: slow request 2926.599917 seconds old, received at 2024-03-28T00:35:43.462302+0000: internal op fragmentdir:mds.0:3808 currently failed to wrlock, waiting 2024-03-28T01:24:30.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:24:30 smithi132 ceph-mon[35941]: slow request 2926.663524 seconds old, received at 2024-03-28T00:35:43.398696+0000: internal op fragmentdir:mds.0:3785 currently failed to wrlock, waiting 2024-03-28T01:24:30.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:24:30 smithi132 ceph-mon[35941]: slow request 2926.599455 seconds old, received at 2024-03-28T00:35:43.462764+0000: internal op fragmentdir:mds.0:3809 currently failed to wrlock, waiting 2024-03-28T01:24:30.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:24:30 smithi132 ceph-mon[35941]: slow request 2926.586125 seconds old, received at 2024-03-28T00:35:43.476094+0000: internal op fragmentdir:mds.0:3841 currently failed to wrlock, waiting 2024-03-28T01:24:32.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:24:32 smithi165 ceph-mon[36302]: pgmap v1832: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-03-28T01:24:32.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:24:32 smithi119 ceph-mon[31518]: pgmap v1832: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-03-28T01:24:32.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:24:32 smithi132 ceph-mon[35941]: pgmap v1832: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-03-28T01:24:34.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:24:34 smithi165 ceph-mon[36302]: pgmap v1833: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:24:34.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:24:34 smithi119 ceph-mon[31518]: pgmap v1833: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:24:34.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:24:34 smithi132 ceph-mon[35941]: pgmap v1833: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:24:35.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:24:35 smithi165 ceph-mon[36302]: 98 slow requests, 5 included below; oldest blocked for > 2935.703360 secs 2024-03-28T01:24:35.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:24:35 smithi165 ceph-mon[36302]: slow request 2931.572654 seconds old, received at 2024-03-28T00:35:43.489651+0000: internal op fragmentdir:mds.0:3873 currently failed to wrlock, waiting 2024-03-28T01:24:35.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:24:35 smithi165 ceph-mon[36302]: slow request 2930.734684 seconds old, received at 2024-03-28T00:35:44.327621+0000: internal op fragmentdir:mds.0:3937 currently failed to wrlock, waiting 2024-03-28T01:24:35.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:24:35 smithi165 ceph-mon[36302]: slow request 2931.599104 seconds old, received at 2024-03-28T00:35:43.463200+0000: internal op fragmentdir:mds.0:3810 currently failed to wrlock, waiting 2024-03-28T01:24:35.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:24:35 smithi165 ceph-mon[36302]: slow request 2931.585774 seconds old, received at 2024-03-28T00:35:43.476531+0000: internal op fragmentdir:mds.0:3842 currently failed to wrlock, waiting 2024-03-28T01:24:35.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:24:35 smithi165 ceph-mon[36302]: slow request 2931.598703 seconds old, received at 2024-03-28T00:35:43.463601+0000: internal op fragmentdir:mds.0:3811 currently failed to wrlock, waiting 2024-03-28T01:24:35.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:24:35 smithi119 ceph-mon[31518]: 98 slow requests, 5 included below; oldest blocked for > 2935.703360 secs 2024-03-28T01:24:35.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:24:35 smithi119 ceph-mon[31518]: slow request 2931.572654 seconds old, received at 2024-03-28T00:35:43.489651+0000: internal op fragmentdir:mds.0:3873 currently failed to wrlock, waiting 2024-03-28T01:24:35.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:24:35 smithi119 ceph-mon[31518]: slow request 2930.734684 seconds old, received at 2024-03-28T00:35:44.327621+0000: internal op fragmentdir:mds.0:3937 currently failed to wrlock, waiting 2024-03-28T01:24:35.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:24:35 smithi119 ceph-mon[31518]: slow request 2931.599104 seconds old, received at 2024-03-28T00:35:43.463200+0000: internal op fragmentdir:mds.0:3810 currently failed to wrlock, waiting 2024-03-28T01:24:35.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:24:35 smithi119 ceph-mon[31518]: slow request 2931.585774 seconds old, received at 2024-03-28T00:35:43.476531+0000: internal op fragmentdir:mds.0:3842 currently failed to wrlock, waiting 2024-03-28T01:24:35.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:24:35 smithi119 ceph-mon[31518]: slow request 2931.598703 seconds old, received at 2024-03-28T00:35:43.463601+0000: internal op fragmentdir:mds.0:3811 currently failed to wrlock, waiting 2024-03-28T01:24:35.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:24:35 smithi132 ceph-mon[35941]: 98 slow requests, 5 included below; oldest blocked for > 2935.703360 secs 2024-03-28T01:24:35.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:24:35 smithi132 ceph-mon[35941]: slow request 2931.572654 seconds old, received at 2024-03-28T00:35:43.489651+0000: internal op fragmentdir:mds.0:3873 currently failed to wrlock, waiting 2024-03-28T01:24:35.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:24:35 smithi132 ceph-mon[35941]: slow request 2930.734684 seconds old, received at 2024-03-28T00:35:44.327621+0000: internal op fragmentdir:mds.0:3937 currently failed to wrlock, waiting 2024-03-28T01:24:35.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:24:35 smithi132 ceph-mon[35941]: slow request 2931.599104 seconds old, received at 2024-03-28T00:35:43.463200+0000: internal op fragmentdir:mds.0:3810 currently failed to wrlock, waiting 2024-03-28T01:24:35.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:24:35 smithi132 ceph-mon[35941]: slow request 2931.585774 seconds old, received at 2024-03-28T00:35:43.476531+0000: internal op fragmentdir:mds.0:3842 currently failed to wrlock, waiting 2024-03-28T01:24:35.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:24:35 smithi132 ceph-mon[35941]: slow request 2931.598703 seconds old, received at 2024-03-28T00:35:43.463601+0000: internal op fragmentdir:mds.0:3811 currently failed to wrlock, waiting 2024-03-28T01:24:36.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:24:36 smithi165 ceph-mon[36302]: pgmap v1834: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:24:36.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:24:36 smithi119 ceph-mon[31518]: pgmap v1834: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:24:36.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:24:36 smithi132 ceph-mon[35941]: pgmap v1834: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:24:38.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:24:38 smithi165 ceph-mon[36302]: pgmap v1835: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:24:38.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:24:38 smithi119 ceph-mon[31518]: pgmap v1835: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:24:38.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:24:38 smithi132 ceph-mon[35941]: pgmap v1835: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:24:40.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:24:40 smithi165 ceph-mon[36302]: pgmap v1836: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-28T01:24:40.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:24:40 smithi165 ceph-mon[36302]: 98 slow requests, 5 included below; oldest blocked for > 2940.703420 secs 2024-03-28T01:24:40.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:24:40 smithi165 ceph-mon[36302]: slow request 2936.556327 seconds old, received at 2024-03-28T00:35:43.506038+0000: internal op fragmentdir:mds.0:3907 currently failed to wrlock, waiting 2024-03-28T01:24:40.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:24:40 smithi165 ceph-mon[36302]: slow request 2936.598329 seconds old, received at 2024-03-28T00:35:43.464036+0000: internal op fragmentdir:mds.0:3812 currently failed to wrlock, waiting 2024-03-28T01:24:40.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:24:40 smithi165 ceph-mon[36302]: slow request 2936.584951 seconds old, received at 2024-03-28T00:35:43.477414+0000: internal op fragmentdir:mds.0:3844 currently failed to wrlock, waiting 2024-03-28T01:24:40.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:24:40 smithi165 ceph-mon[36302]: slow request 2936.554600 seconds old, received at 2024-03-28T00:35:43.507766+0000: internal op fragmentdir:mds.0:3908 currently failed to wrlock, waiting 2024-03-28T01:24:40.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:24:40 smithi165 ceph-mon[36302]: slow request 2937.923000 seconds old, received at 2024-03-28T00:35:42.139365+0000: internal op fragmentdir:mds.0:3784 currently failed to wrlock, waiting 2024-03-28T01:24:40.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:24:40 smithi119 ceph-mon[31518]: pgmap v1836: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-28T01:24:40.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:24:40 smithi119 ceph-mon[31518]: 98 slow requests, 5 included below; oldest blocked for > 2940.703420 secs 2024-03-28T01:24:40.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:24:40 smithi119 ceph-mon[31518]: slow request 2936.556327 seconds old, received at 2024-03-28T00:35:43.506038+0000: internal op fragmentdir:mds.0:3907 currently failed to wrlock, waiting 2024-03-28T01:24:40.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:24:40 smithi119 ceph-mon[31518]: slow request 2936.598329 seconds old, received at 2024-03-28T00:35:43.464036+0000: internal op fragmentdir:mds.0:3812 currently failed to wrlock, waiting 2024-03-28T01:24:40.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:24:40 smithi119 ceph-mon[31518]: slow request 2936.584951 seconds old, received at 2024-03-28T00:35:43.477414+0000: internal op fragmentdir:mds.0:3844 currently failed to wrlock, waiting 2024-03-28T01:24:40.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:24:40 smithi119 ceph-mon[31518]: slow request 2936.554600 seconds old, received at 2024-03-28T00:35:43.507766+0000: internal op fragmentdir:mds.0:3908 currently failed to wrlock, waiting 2024-03-28T01:24:40.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:24:40 smithi119 ceph-mon[31518]: slow request 2937.923000 seconds old, received at 2024-03-28T00:35:42.139365+0000: internal op fragmentdir:mds.0:3784 currently failed to wrlock, waiting 2024-03-28T01:24:40.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:24:40 smithi132 ceph-mon[35941]: pgmap v1836: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-28T01:24:40.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:24:40 smithi132 ceph-mon[35941]: 98 slow requests, 5 included below; oldest blocked for > 2940.703420 secs 2024-03-28T01:24:40.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:24:40 smithi132 ceph-mon[35941]: slow request 2936.556327 seconds old, received at 2024-03-28T00:35:43.506038+0000: internal op fragmentdir:mds.0:3907 currently failed to wrlock, waiting 2024-03-28T01:24:40.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:24:40 smithi132 ceph-mon[35941]: slow request 2936.598329 seconds old, received at 2024-03-28T00:35:43.464036+0000: internal op fragmentdir:mds.0:3812 currently failed to wrlock, waiting 2024-03-28T01:24:40.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:24:40 smithi132 ceph-mon[35941]: slow request 2936.584951 seconds old, received at 2024-03-28T00:35:43.477414+0000: internal op fragmentdir:mds.0:3844 currently failed to wrlock, waiting 2024-03-28T01:24:40.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:24:40 smithi132 ceph-mon[35941]: slow request 2936.554600 seconds old, received at 2024-03-28T00:35:43.507766+0000: internal op fragmentdir:mds.0:3908 currently failed to wrlock, waiting 2024-03-28T01:24:40.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:24:40 smithi132 ceph-mon[35941]: slow request 2937.923000 seconds old, received at 2024-03-28T00:35:42.139365+0000: internal op fragmentdir:mds.0:3784 currently failed to wrlock, waiting 2024-03-28T01:24:42.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:24:42 smithi165 ceph-mon[36302]: pgmap v1837: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-03-28T01:24:42.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:24:42 smithi119 ceph-mon[31518]: pgmap v1837: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-03-28T01:24:42.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:24:42 smithi132 ceph-mon[35941]: pgmap v1837: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-03-28T01:24:44.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:24:44 smithi165 ceph-mon[36302]: pgmap v1838: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:24:44.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:24:44 smithi119 ceph-mon[31518]: pgmap v1838: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:24:44.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:24:44 smithi132 ceph-mon[35941]: pgmap v1838: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:24:45.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:24:45 smithi119 ceph-mon[31518]: 98 slow requests, 5 included below; oldest blocked for > 2945.703509 secs 2024-03-28T01:24:45.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:24:45 smithi119 ceph-mon[31518]: slow request 2941.598021 seconds old, received at 2024-03-28T00:35:43.464432+0000: internal op fragmentdir:mds.0:3813 currently failed to wrlock, waiting 2024-03-28T01:24:45.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:24:45 smithi119 ceph-mon[31518]: slow request 2941.584596 seconds old, received at 2024-03-28T00:35:43.477858+0000: internal op fragmentdir:mds.0:3845 currently failed to wrlock, waiting 2024-03-28T01:24:45.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:24:45 smithi119 ceph-mon[31518]: slow request 2941.570906 seconds old, received at 2024-03-28T00:35:43.491547+0000: internal op fragmentdir:mds.0:3877 currently failed to wrlock, waiting 2024-03-28T01:24:45.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:24:45 smithi119 ceph-mon[31518]: slow request 2941.597583 seconds old, received at 2024-03-28T00:35:43.464871+0000: internal op fragmentdir:mds.0:3814 currently failed to wrlock, waiting 2024-03-28T01:24:45.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:24:45 smithi119 ceph-mon[31518]: slow request 2941.552630 seconds old, received at 2024-03-28T00:35:43.509824+0000: internal op fragmentdir:mds.0:3910 currently failed to wrlock, waiting 2024-03-28T01:24:45.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:24:45 smithi132 ceph-mon[35941]: 98 slow requests, 5 included below; oldest blocked for > 2945.703509 secs 2024-03-28T01:24:45.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:24:45 smithi132 ceph-mon[35941]: slow request 2941.598021 seconds old, received at 2024-03-28T00:35:43.464432+0000: internal op fragmentdir:mds.0:3813 currently failed to wrlock, waiting 2024-03-28T01:24:45.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:24:45 smithi132 ceph-mon[35941]: slow request 2941.584596 seconds old, received at 2024-03-28T00:35:43.477858+0000: internal op fragmentdir:mds.0:3845 currently failed to wrlock, waiting 2024-03-28T01:24:45.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:24:45 smithi132 ceph-mon[35941]: slow request 2941.570906 seconds old, received at 2024-03-28T00:35:43.491547+0000: internal op fragmentdir:mds.0:3877 currently failed to wrlock, waiting 2024-03-28T01:24:45.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:24:45 smithi132 ceph-mon[35941]: slow request 2941.597583 seconds old, received at 2024-03-28T00:35:43.464871+0000: internal op fragmentdir:mds.0:3814 currently failed to wrlock, waiting 2024-03-28T01:24:45.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:24:45 smithi132 ceph-mon[35941]: slow request 2941.552630 seconds old, received at 2024-03-28T00:35:43.509824+0000: internal op fragmentdir:mds.0:3910 currently failed to wrlock, waiting 2024-03-28T01:24:45.877 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:24:45 smithi165 ceph-mon[36302]: 98 slow requests, 5 included below; oldest blocked for > 2945.703509 secs 2024-03-28T01:24:45.877 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:24:45 smithi165 ceph-mon[36302]: slow request 2941.598021 seconds old, received at 2024-03-28T00:35:43.464432+0000: internal op fragmentdir:mds.0:3813 currently failed to wrlock, waiting 2024-03-28T01:24:45.877 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:24:45 smithi165 ceph-mon[36302]: slow request 2941.584596 seconds old, received at 2024-03-28T00:35:43.477858+0000: internal op fragmentdir:mds.0:3845 currently failed to wrlock, waiting 2024-03-28T01:24:45.877 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:24:45 smithi165 ceph-mon[36302]: slow request 2941.570906 seconds old, received at 2024-03-28T00:35:43.491547+0000: internal op fragmentdir:mds.0:3877 currently failed to wrlock, waiting 2024-03-28T01:24:45.877 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:24:45 smithi165 ceph-mon[36302]: slow request 2941.597583 seconds old, received at 2024-03-28T00:35:43.464871+0000: internal op fragmentdir:mds.0:3814 currently failed to wrlock, waiting 2024-03-28T01:24:45.877 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:24:45 smithi165 ceph-mon[36302]: slow request 2941.552630 seconds old, received at 2024-03-28T00:35:43.509824+0000: internal op fragmentdir:mds.0:3910 currently failed to wrlock, waiting 2024-03-28T01:24:46.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:24:46 smithi119 ceph-mon[31518]: pgmap v1839: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:24:46.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:24:46 smithi132 ceph-mon[35941]: pgmap v1839: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:24:46.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:24:46 smithi165 ceph-mon[36302]: pgmap v1839: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:24:47.552 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:24:47 smithi119 ceph-mon[31518]: pgmap v1840: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:24:47.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:24:47 smithi132 ceph-mon[35941]: pgmap v1840: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:24:47.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:24:47 smithi165 ceph-mon[36302]: pgmap v1840: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:24:50.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:24:50 smithi165 ceph-mon[36302]: pgmap v1841: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-28T01:24:50.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:24:50 smithi119 ceph-mon[31518]: pgmap v1841: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-28T01:24:50.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:24:50 smithi132 ceph-mon[35941]: pgmap v1841: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-28T01:24:51.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:24:51 smithi165 ceph-mon[36302]: 98 slow requests, 5 included below; oldest blocked for > 2950.703593 secs 2024-03-28T01:24:51.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:24:51 smithi165 ceph-mon[36302]: slow request 2946.637612 seconds old, received at 2024-03-28T00:35:43.424926+0000: internal op fragmentdir:mds.0:3787 currently failed to wrlock, waiting 2024-03-28T01:24:51.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:24:51 smithi165 ceph-mon[36302]: slow request 2946.597227 seconds old, received at 2024-03-28T00:35:43.465310+0000: internal op fragmentdir:mds.0:3815 currently failed to wrlock, waiting 2024-03-28T01:24:51.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:24:51 smithi165 ceph-mon[36302]: slow request 2946.570062 seconds old, received at 2024-03-28T00:35:43.492476+0000: internal op fragmentdir:mds.0:3879 currently failed to wrlock, waiting 2024-03-28T01:24:51.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:24:51 smithi165 ceph-mon[36302]: slow request 2946.596833 seconds old, received at 2024-03-28T00:35:43.465705+0000: internal op fragmentdir:mds.0:3816 currently failed to wrlock, waiting 2024-03-28T01:24:51.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:24:51 smithi165 ceph-mon[36302]: slow request 2946.583272 seconds old, received at 2024-03-28T00:35:43.479266+0000: internal op fragmentdir:mds.0:3848 currently failed to wrlock, waiting 2024-03-28T01:24:51.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:24:51 smithi119 ceph-mon[31518]: 98 slow requests, 5 included below; oldest blocked for > 2950.703593 secs 2024-03-28T01:24:51.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:24:51 smithi119 ceph-mon[31518]: slow request 2946.637612 seconds old, received at 2024-03-28T00:35:43.424926+0000: internal op fragmentdir:mds.0:3787 currently failed to wrlock, waiting 2024-03-28T01:24:51.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:24:51 smithi119 ceph-mon[31518]: slow request 2946.597227 seconds old, received at 2024-03-28T00:35:43.465310+0000: internal op fragmentdir:mds.0:3815 currently failed to wrlock, waiting 2024-03-28T01:24:51.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:24:51 smithi119 ceph-mon[31518]: slow request 2946.570062 seconds old, received at 2024-03-28T00:35:43.492476+0000: internal op fragmentdir:mds.0:3879 currently failed to wrlock, waiting 2024-03-28T01:24:51.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:24:51 smithi119 ceph-mon[31518]: slow request 2946.596833 seconds old, received at 2024-03-28T00:35:43.465705+0000: internal op fragmentdir:mds.0:3816 currently failed to wrlock, waiting 2024-03-28T01:24:51.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:24:51 smithi119 ceph-mon[31518]: slow request 2946.583272 seconds old, received at 2024-03-28T00:35:43.479266+0000: internal op fragmentdir:mds.0:3848 currently failed to wrlock, waiting 2024-03-28T01:24:51.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:24:51 smithi132 ceph-mon[35941]: 98 slow requests, 5 included below; oldest blocked for > 2950.703593 secs 2024-03-28T01:24:51.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:24:51 smithi132 ceph-mon[35941]: slow request 2946.637612 seconds old, received at 2024-03-28T00:35:43.424926+0000: internal op fragmentdir:mds.0:3787 currently failed to wrlock, waiting 2024-03-28T01:24:51.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:24:51 smithi132 ceph-mon[35941]: slow request 2946.597227 seconds old, received at 2024-03-28T00:35:43.465310+0000: internal op fragmentdir:mds.0:3815 currently failed to wrlock, waiting 2024-03-28T01:24:51.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:24:51 smithi132 ceph-mon[35941]: slow request 2946.570062 seconds old, received at 2024-03-28T00:35:43.492476+0000: internal op fragmentdir:mds.0:3879 currently failed to wrlock, waiting 2024-03-28T01:24:51.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:24:51 smithi132 ceph-mon[35941]: slow request 2946.596833 seconds old, received at 2024-03-28T00:35:43.465705+0000: internal op fragmentdir:mds.0:3816 currently failed to wrlock, waiting 2024-03-28T01:24:51.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:24:51 smithi132 ceph-mon[35941]: slow request 2946.583272 seconds old, received at 2024-03-28T00:35:43.479266+0000: internal op fragmentdir:mds.0:3848 currently failed to wrlock, waiting 2024-03-28T01:24:52.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:24:52 smithi165 ceph-mon[36302]: pgmap v1842: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-03-28T01:24:52.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:24:52 smithi119 ceph-mon[31518]: pgmap v1842: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-03-28T01:24:52.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:24:52 smithi132 ceph-mon[35941]: pgmap v1842: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-03-28T01:24:54.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:24:54 smithi165 ceph-mon[36302]: pgmap v1843: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:24:54.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:24:54 smithi119 ceph-mon[31518]: pgmap v1843: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:24:54.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:24:54 smithi132 ceph-mon[35941]: pgmap v1843: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:24:56.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:24:56 smithi165 ceph-mon[36302]: 98 slow requests, 5 included below; oldest blocked for > 2955.703681 secs 2024-03-28T01:24:56.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:24:56 smithi165 ceph-mon[36302]: slow request 2951.569665 seconds old, received at 2024-03-28T00:35:43.492962+0000: internal op fragmentdir:mds.0:3880 currently failed to wrlock, waiting 2024-03-28T01:24:56.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:24:56 smithi165 ceph-mon[36302]: slow request 2952.914555 seconds old, received at 2024-03-28T00:35:42.148071+0000: client_request(client.25264:33954 create owner_uid=1000, owner_gid=1301 #0x10000000271/file8309 2024-03-28T00:35:42.146337+0000 caller_uid=1000, caller_gid=1301{6,36,1000,1301,}) currently failed to xlock, waiting 2024-03-28T01:24:56.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:24:56 smithi165 ceph-mon[36302]: slow request 2951.596526 seconds old, received at 2024-03-28T00:35:43.466100+0000: internal op fragmentdir:mds.0:3817 currently failed to wrlock, waiting 2024-03-28T01:24:56.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:24:56 smithi165 ceph-mon[36302]: slow request 2951.569336 seconds old, received at 2024-03-28T00:35:43.493290+0000: internal op fragmentdir:mds.0:3881 currently failed to wrlock, waiting 2024-03-28T01:24:56.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:24:56 smithi165 ceph-mon[36302]: slow request 2951.596079 seconds old, received at 2024-03-28T00:35:43.466548+0000: internal op fragmentdir:mds.0:3818 currently failed to wrlock, waiting 2024-03-28T01:24:56.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:24:56 smithi165 ceph-mon[36302]: pgmap v1844: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:24:56.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:24:56 smithi119 ceph-mon[31518]: 98 slow requests, 5 included below; oldest blocked for > 2955.703681 secs 2024-03-28T01:24:56.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:24:56 smithi119 ceph-mon[31518]: slow request 2951.569665 seconds old, received at 2024-03-28T00:35:43.492962+0000: internal op fragmentdir:mds.0:3880 currently failed to wrlock, waiting 2024-03-28T01:24:56.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:24:56 smithi119 ceph-mon[31518]: slow request 2952.914555 seconds old, received at 2024-03-28T00:35:42.148071+0000: client_request(client.25264:33954 create owner_uid=1000, owner_gid=1301 #0x10000000271/file8309 2024-03-28T00:35:42.146337+0000 caller_uid=1000, caller_gid=1301{6,36,1000,1301,}) currently failed to xlock, waiting 2024-03-28T01:24:56.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:24:56 smithi119 ceph-mon[31518]: slow request 2951.596526 seconds old, received at 2024-03-28T00:35:43.466100+0000: internal op fragmentdir:mds.0:3817 currently failed to wrlock, waiting 2024-03-28T01:24:56.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:24:56 smithi119 ceph-mon[31518]: slow request 2951.569336 seconds old, received at 2024-03-28T00:35:43.493290+0000: internal op fragmentdir:mds.0:3881 currently failed to wrlock, waiting 2024-03-28T01:24:56.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:24:56 smithi119 ceph-mon[31518]: slow request 2951.596079 seconds old, received at 2024-03-28T00:35:43.466548+0000: internal op fragmentdir:mds.0:3818 currently failed to wrlock, waiting 2024-03-28T01:24:56.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:24:56 smithi119 ceph-mon[31518]: pgmap v1844: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:24:56.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:24:56 smithi132 ceph-mon[35941]: 98 slow requests, 5 included below; oldest blocked for > 2955.703681 secs 2024-03-28T01:24:56.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:24:56 smithi132 ceph-mon[35941]: slow request 2951.569665 seconds old, received at 2024-03-28T00:35:43.492962+0000: internal op fragmentdir:mds.0:3880 currently failed to wrlock, waiting 2024-03-28T01:24:56.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:24:56 smithi132 ceph-mon[35941]: slow request 2952.914555 seconds old, received at 2024-03-28T00:35:42.148071+0000: client_request(client.25264:33954 create owner_uid=1000, owner_gid=1301 #0x10000000271/file8309 2024-03-28T00:35:42.146337+0000 caller_uid=1000, caller_gid=1301{6,36,1000,1301,}) currently failed to xlock, waiting 2024-03-28T01:24:56.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:24:56 smithi132 ceph-mon[35941]: slow request 2951.596526 seconds old, received at 2024-03-28T00:35:43.466100+0000: internal op fragmentdir:mds.0:3817 currently failed to wrlock, waiting 2024-03-28T01:24:56.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:24:56 smithi132 ceph-mon[35941]: slow request 2951.569336 seconds old, received at 2024-03-28T00:35:43.493290+0000: internal op fragmentdir:mds.0:3881 currently failed to wrlock, waiting 2024-03-28T01:24:56.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:24:56 smithi132 ceph-mon[35941]: slow request 2951.596079 seconds old, received at 2024-03-28T00:35:43.466548+0000: internal op fragmentdir:mds.0:3818 currently failed to wrlock, waiting 2024-03-28T01:24:56.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:24:56 smithi132 ceph-mon[35941]: pgmap v1844: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:24:58.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:24:58 smithi165 ceph-mon[36302]: pgmap v1845: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:24:58.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:24:58 smithi119 ceph-mon[31518]: pgmap v1845: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:24:58.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:24:58 smithi132 ceph-mon[35941]: pgmap v1845: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:25:00.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:25:00 smithi165 ceph-mon[36302]: pgmap v1846: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-28T01:25:00.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:25:00 smithi119 ceph-mon[31518]: pgmap v1846: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-28T01:25:00.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:25:00 smithi132 ceph-mon[35941]: pgmap v1846: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-28T01:25:01.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:25:01 smithi165 ceph-mon[36302]: 98 slow requests, 5 included below; oldest blocked for > 2960.703697 secs 2024-03-28T01:25:01.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:25:01 smithi165 ceph-mon[36302]: slow request 2956.582449 seconds old, received at 2024-03-28T00:35:43.480192+0000: internal op fragmentdir:mds.0:3850 currently failed to wrlock, waiting 2024-03-28T01:25:01.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:25:01 smithi165 ceph-mon[36302]: slow request 2956.613898 seconds old, received at 2024-03-28T00:35:43.448744+0000: internal op fragmentdir:mds.0:3788 currently failed to wrlock, waiting 2024-03-28T01:25:01.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:25:01 smithi165 ceph-mon[36302]: slow request 2956.595686 seconds old, received at 2024-03-28T00:35:43.466955+0000: internal op fragmentdir:mds.0:3819 currently failed to wrlock, waiting 2024-03-28T01:25:01.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:25:01 smithi165 ceph-mon[36302]: slow request 2956.581975 seconds old, received at 2024-03-28T00:35:43.480667+0000: internal op fragmentdir:mds.0:3851 currently failed to wrlock, waiting 2024-03-28T01:25:01.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:25:01 smithi165 ceph-mon[36302]: slow request 2956.568803 seconds old, received at 2024-03-28T00:35:43.493839+0000: internal op fragmentdir:mds.0:3883 currently failed to wrlock, waiting 2024-03-28T01:25:01.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:25:01 smithi119 ceph-mon[31518]: 98 slow requests, 5 included below; oldest blocked for > 2960.703697 secs 2024-03-28T01:25:01.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:25:01 smithi119 ceph-mon[31518]: slow request 2956.582449 seconds old, received at 2024-03-28T00:35:43.480192+0000: internal op fragmentdir:mds.0:3850 currently failed to wrlock, waiting 2024-03-28T01:25:01.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:25:01 smithi119 ceph-mon[31518]: slow request 2956.613898 seconds old, received at 2024-03-28T00:35:43.448744+0000: internal op fragmentdir:mds.0:3788 currently failed to wrlock, waiting 2024-03-28T01:25:01.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:25:01 smithi119 ceph-mon[31518]: slow request 2956.595686 seconds old, received at 2024-03-28T00:35:43.466955+0000: internal op fragmentdir:mds.0:3819 currently failed to wrlock, waiting 2024-03-28T01:25:01.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:25:01 smithi119 ceph-mon[31518]: slow request 2956.581975 seconds old, received at 2024-03-28T00:35:43.480667+0000: internal op fragmentdir:mds.0:3851 currently failed to wrlock, waiting 2024-03-28T01:25:01.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:25:01 smithi119 ceph-mon[31518]: slow request 2956.568803 seconds old, received at 2024-03-28T00:35:43.493839+0000: internal op fragmentdir:mds.0:3883 currently failed to wrlock, waiting 2024-03-28T01:25:01.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:25:01 smithi132 ceph-mon[35941]: 98 slow requests, 5 included below; oldest blocked for > 2960.703697 secs 2024-03-28T01:25:01.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:25:01 smithi132 ceph-mon[35941]: slow request 2956.582449 seconds old, received at 2024-03-28T00:35:43.480192+0000: internal op fragmentdir:mds.0:3850 currently failed to wrlock, waiting 2024-03-28T01:25:01.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:25:01 smithi132 ceph-mon[35941]: slow request 2956.613898 seconds old, received at 2024-03-28T00:35:43.448744+0000: internal op fragmentdir:mds.0:3788 currently failed to wrlock, waiting 2024-03-28T01:25:01.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:25:01 smithi132 ceph-mon[35941]: slow request 2956.595686 seconds old, received at 2024-03-28T00:35:43.466955+0000: internal op fragmentdir:mds.0:3819 currently failed to wrlock, waiting 2024-03-28T01:25:01.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:25:01 smithi132 ceph-mon[35941]: slow request 2956.581975 seconds old, received at 2024-03-28T00:35:43.480667+0000: internal op fragmentdir:mds.0:3851 currently failed to wrlock, waiting 2024-03-28T01:25:01.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:25:01 smithi132 ceph-mon[35941]: slow request 2956.568803 seconds old, received at 2024-03-28T00:35:43.493839+0000: internal op fragmentdir:mds.0:3883 currently failed to wrlock, waiting 2024-03-28T01:25:02.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:25:02 smithi165 ceph-mon[36302]: pgmap v1847: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-03-28T01:25:02.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:25:02 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:25:02.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:25:02 smithi119 ceph-mon[31518]: pgmap v1847: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-03-28T01:25:02.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:25:02 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:25:02.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:25:02 smithi132 ceph-mon[35941]: pgmap v1847: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-03-28T01:25:02.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:25:02 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:25:03.400 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:25:03 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:25:03.400 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:25:03 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:25:03.400 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:25:03 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:25:03.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:25:03 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:25:03.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:25:03 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:25:03.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:25:03 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:25:03.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:25:03 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:25:03.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:25:03 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:25:03.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:25:03 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:25:04.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:25:04 smithi165 ceph-mon[36302]: pgmap v1848: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T01:25:04.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:25:04 smithi165 ceph-mon[36302]: pgmap v1849: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:25:04.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:25:04 smithi119 ceph-mon[31518]: pgmap v1848: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T01:25:04.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:25:04 smithi119 ceph-mon[31518]: pgmap v1849: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:25:04.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:25:04 smithi132 ceph-mon[35941]: pgmap v1848: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T01:25:04.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:25:04 smithi132 ceph-mon[35941]: pgmap v1849: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:25:05.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:25:05 smithi165 ceph-mon[36302]: 98 slow requests, 5 included below; oldest blocked for > 2965.703762 secs 2024-03-28T01:25:05.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:25:05 smithi165 ceph-mon[36302]: slow request 2961.613545 seconds old, received at 2024-03-28T00:35:43.449161+0000: internal op fragmentdir:mds.0:3789 currently failed to wrlock, waiting 2024-03-28T01:25:05.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:25:05 smithi165 ceph-mon[36302]: slow request 2961.595315 seconds old, received at 2024-03-28T00:35:43.467391+0000: internal op fragmentdir:mds.0:3820 currently failed to wrlock, waiting 2024-03-28T01:25:05.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:25:05 smithi165 ceph-mon[36302]: slow request 2961.581565 seconds old, received at 2024-03-28T00:35:43.481141+0000: internal op fragmentdir:mds.0:3852 currently failed to wrlock, waiting 2024-03-28T01:25:05.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:25:05 smithi165 ceph-mon[36302]: slow request 2961.539773 seconds old, received at 2024-03-28T00:35:43.522934+0000: internal op fragmentdir:mds.0:3916 currently failed to wrlock, waiting 2024-03-28T01:25:05.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:25:05 smithi165 ceph-mon[36302]: slow request 2961.613085 seconds old, received at 2024-03-28T00:35:43.449621+0000: internal op fragmentdir:mds.0:3790 currently failed to wrlock, waiting 2024-03-28T01:25:05.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:25:05 smithi119 ceph-mon[31518]: 98 slow requests, 5 included below; oldest blocked for > 2965.703762 secs 2024-03-28T01:25:05.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:25:05 smithi119 ceph-mon[31518]: slow request 2961.613545 seconds old, received at 2024-03-28T00:35:43.449161+0000: internal op fragmentdir:mds.0:3789 currently failed to wrlock, waiting 2024-03-28T01:25:05.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:25:05 smithi119 ceph-mon[31518]: slow request 2961.595315 seconds old, received at 2024-03-28T00:35:43.467391+0000: internal op fragmentdir:mds.0:3820 currently failed to wrlock, waiting 2024-03-28T01:25:05.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:25:05 smithi119 ceph-mon[31518]: slow request 2961.581565 seconds old, received at 2024-03-28T00:35:43.481141+0000: internal op fragmentdir:mds.0:3852 currently failed to wrlock, waiting 2024-03-28T01:25:05.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:25:05 smithi119 ceph-mon[31518]: slow request 2961.539773 seconds old, received at 2024-03-28T00:35:43.522934+0000: internal op fragmentdir:mds.0:3916 currently failed to wrlock, waiting 2024-03-28T01:25:05.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:25:05 smithi119 ceph-mon[31518]: slow request 2961.613085 seconds old, received at 2024-03-28T00:35:43.449621+0000: internal op fragmentdir:mds.0:3790 currently failed to wrlock, waiting 2024-03-28T01:25:05.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:25:05 smithi132 ceph-mon[35941]: 98 slow requests, 5 included below; oldest blocked for > 2965.703762 secs 2024-03-28T01:25:05.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:25:05 smithi132 ceph-mon[35941]: slow request 2961.613545 seconds old, received at 2024-03-28T00:35:43.449161+0000: internal op fragmentdir:mds.0:3789 currently failed to wrlock, waiting 2024-03-28T01:25:05.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:25:05 smithi132 ceph-mon[35941]: slow request 2961.595315 seconds old, received at 2024-03-28T00:35:43.467391+0000: internal op fragmentdir:mds.0:3820 currently failed to wrlock, waiting 2024-03-28T01:25:05.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:25:05 smithi132 ceph-mon[35941]: slow request 2961.581565 seconds old, received at 2024-03-28T00:35:43.481141+0000: internal op fragmentdir:mds.0:3852 currently failed to wrlock, waiting 2024-03-28T01:25:05.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:25:05 smithi132 ceph-mon[35941]: slow request 2961.539773 seconds old, received at 2024-03-28T00:35:43.522934+0000: internal op fragmentdir:mds.0:3916 currently failed to wrlock, waiting 2024-03-28T01:25:05.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:25:05 smithi132 ceph-mon[35941]: slow request 2961.613085 seconds old, received at 2024-03-28T00:35:43.449621+0000: internal op fragmentdir:mds.0:3790 currently failed to wrlock, waiting 2024-03-28T01:25:06.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:25:06 smithi165 ceph-mon[36302]: pgmap v1850: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:25:06.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:25:06 smithi119 ceph-mon[31518]: pgmap v1850: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:25:06.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:25:06 smithi132 ceph-mon[35941]: pgmap v1850: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:25:08.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:25:08 smithi165 ceph-mon[36302]: pgmap v1851: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:25:08.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:25:08 smithi119 ceph-mon[31518]: pgmap v1851: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:25:08.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:25:08 smithi132 ceph-mon[35941]: pgmap v1851: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:25:10.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:25:10 smithi165 ceph-mon[36302]: pgmap v1852: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 4 op/s 2024-03-28T01:25:10.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:25:10 smithi165 ceph-mon[36302]: 98 slow requests, 5 included below; oldest blocked for > 2970.703822 secs 2024-03-28T01:25:10.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:25:10 smithi165 ceph-mon[36302]: slow request 2966.594973 seconds old, received at 2024-03-28T00:35:43.467794+0000: internal op fragmentdir:mds.0:3821 currently failed to wrlock, waiting 2024-03-28T01:25:10.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:25:10 smithi165 ceph-mon[36302]: slow request 2966.568145 seconds old, received at 2024-03-28T00:35:43.494622+0000: internal op fragmentdir:mds.0:3885 currently failed to wrlock, waiting 2024-03-28T01:25:10.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:25:10 smithi165 ceph-mon[36302]: slow request 2966.612604 seconds old, received at 2024-03-28T00:35:43.450163+0000: internal op fragmentdir:mds.0:3791 currently failed to wrlock, waiting 2024-03-28T01:25:10.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:25:10 smithi165 ceph-mon[36302]: slow request 2966.580708 seconds old, received at 2024-03-28T00:35:43.482059+0000: internal op fragmentdir:mds.0:3854 currently failed to wrlock, waiting 2024-03-28T01:25:10.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:25:10 smithi165 ceph-mon[36302]: slow request 2966.567693 seconds old, received at 2024-03-28T00:35:43.495074+0000: internal op fragmentdir:mds.0:3886 currently failed to wrlock, waiting 2024-03-28T01:25:10.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:25:10 smithi119 ceph-mon[31518]: pgmap v1852: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 4 op/s 2024-03-28T01:25:10.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:25:10 smithi119 ceph-mon[31518]: 98 slow requests, 5 included below; oldest blocked for > 2970.703822 secs 2024-03-28T01:25:10.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:25:10 smithi119 ceph-mon[31518]: slow request 2966.594973 seconds old, received at 2024-03-28T00:35:43.467794+0000: internal op fragmentdir:mds.0:3821 currently failed to wrlock, waiting 2024-03-28T01:25:10.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:25:10 smithi119 ceph-mon[31518]: slow request 2966.568145 seconds old, received at 2024-03-28T00:35:43.494622+0000: internal op fragmentdir:mds.0:3885 currently failed to wrlock, waiting 2024-03-28T01:25:10.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:25:10 smithi119 ceph-mon[31518]: slow request 2966.612604 seconds old, received at 2024-03-28T00:35:43.450163+0000: internal op fragmentdir:mds.0:3791 currently failed to wrlock, waiting 2024-03-28T01:25:10.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:25:10 smithi119 ceph-mon[31518]: slow request 2966.580708 seconds old, received at 2024-03-28T00:35:43.482059+0000: internal op fragmentdir:mds.0:3854 currently failed to wrlock, waiting 2024-03-28T01:25:10.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:25:10 smithi119 ceph-mon[31518]: slow request 2966.567693 seconds old, received at 2024-03-28T00:35:43.495074+0000: internal op fragmentdir:mds.0:3886 currently failed to wrlock, waiting 2024-03-28T01:25:10.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:25:10 smithi132 ceph-mon[35941]: pgmap v1852: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 4 op/s 2024-03-28T01:25:10.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:25:10 smithi132 ceph-mon[35941]: 98 slow requests, 5 included below; oldest blocked for > 2970.703822 secs 2024-03-28T01:25:10.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:25:10 smithi132 ceph-mon[35941]: slow request 2966.594973 seconds old, received at 2024-03-28T00:35:43.467794+0000: internal op fragmentdir:mds.0:3821 currently failed to wrlock, waiting 2024-03-28T01:25:10.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:25:10 smithi132 ceph-mon[35941]: slow request 2966.568145 seconds old, received at 2024-03-28T00:35:43.494622+0000: internal op fragmentdir:mds.0:3885 currently failed to wrlock, waiting 2024-03-28T01:25:10.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:25:10 smithi132 ceph-mon[35941]: slow request 2966.612604 seconds old, received at 2024-03-28T00:35:43.450163+0000: internal op fragmentdir:mds.0:3791 currently failed to wrlock, waiting 2024-03-28T01:25:10.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:25:10 smithi132 ceph-mon[35941]: slow request 2966.580708 seconds old, received at 2024-03-28T00:35:43.482059+0000: internal op fragmentdir:mds.0:3854 currently failed to wrlock, waiting 2024-03-28T01:25:10.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:25:10 smithi132 ceph-mon[35941]: slow request 2966.567693 seconds old, received at 2024-03-28T00:35:43.495074+0000: internal op fragmentdir:mds.0:3886 currently failed to wrlock, waiting 2024-03-28T01:25:12.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:25:12 smithi165 ceph-mon[36302]: pgmap v1853: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:25:12.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:25:12 smithi119 ceph-mon[31518]: pgmap v1853: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:25:12.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:25:12 smithi132 ceph-mon[35941]: pgmap v1853: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:25:14.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:25:14 smithi165 ceph-mon[36302]: pgmap v1854: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T01:25:14.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:25:14 smithi119 ceph-mon[31518]: pgmap v1854: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T01:25:14.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:25:14 smithi132 ceph-mon[35941]: pgmap v1854: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T01:25:15.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:25:15 smithi165 ceph-mon[36302]: 98 slow requests, 5 included below; oldest blocked for > 2975.703894 secs 2024-03-28T01:25:15.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:25:15 smithi165 ceph-mon[36302]: slow request 2971.538784 seconds old, received at 2024-03-28T00:35:43.524054+0000: internal op fragmentdir:mds.0:3918 currently failed to wrlock, waiting 2024-03-28T01:25:15.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:25:15 smithi165 ceph-mon[36302]: slow request 2971.612225 seconds old, received at 2024-03-28T00:35:43.450613+0000: internal op fragmentdir:mds.0:3792 currently failed to wrlock, waiting 2024-03-28T01:25:15.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:25:15 smithi165 ceph-mon[36302]: slow request 2971.594449 seconds old, received at 2024-03-28T00:35:43.468389+0000: internal op fragmentdir:mds.0:3823 currently failed to wrlock, waiting 2024-03-28T01:25:15.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:25:15 smithi165 ceph-mon[36302]: slow request 2971.611733 seconds old, received at 2024-03-28T00:35:43.451105+0000: internal op fragmentdir:mds.0:3793 currently failed to wrlock, waiting 2024-03-28T01:25:15.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:25:15 smithi165 ceph-mon[36302]: slow request 2971.566891 seconds old, received at 2024-03-28T00:35:43.495948+0000: internal op fragmentdir:mds.0:3888 currently failed to wrlock, waiting 2024-03-28T01:25:15.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:25:15 smithi119 ceph-mon[31518]: 98 slow requests, 5 included below; oldest blocked for > 2975.703894 secs 2024-03-28T01:25:15.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:25:15 smithi119 ceph-mon[31518]: slow request 2971.538784 seconds old, received at 2024-03-28T00:35:43.524054+0000: internal op fragmentdir:mds.0:3918 currently failed to wrlock, waiting 2024-03-28T01:25:15.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:25:15 smithi119 ceph-mon[31518]: slow request 2971.612225 seconds old, received at 2024-03-28T00:35:43.450613+0000: internal op fragmentdir:mds.0:3792 currently failed to wrlock, waiting 2024-03-28T01:25:15.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:25:15 smithi119 ceph-mon[31518]: slow request 2971.594449 seconds old, received at 2024-03-28T00:35:43.468389+0000: internal op fragmentdir:mds.0:3823 currently failed to wrlock, waiting 2024-03-28T01:25:15.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:25:15 smithi119 ceph-mon[31518]: slow request 2971.611733 seconds old, received at 2024-03-28T00:35:43.451105+0000: internal op fragmentdir:mds.0:3793 currently failed to wrlock, waiting 2024-03-28T01:25:15.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:25:15 smithi119 ceph-mon[31518]: slow request 2971.566891 seconds old, received at 2024-03-28T00:35:43.495948+0000: internal op fragmentdir:mds.0:3888 currently failed to wrlock, waiting 2024-03-28T01:25:15.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:25:15 smithi132 ceph-mon[35941]: 98 slow requests, 5 included below; oldest blocked for > 2975.703894 secs 2024-03-28T01:25:15.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:25:15 smithi132 ceph-mon[35941]: slow request 2971.538784 seconds old, received at 2024-03-28T00:35:43.524054+0000: internal op fragmentdir:mds.0:3918 currently failed to wrlock, waiting 2024-03-28T01:25:15.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:25:15 smithi132 ceph-mon[35941]: slow request 2971.612225 seconds old, received at 2024-03-28T00:35:43.450613+0000: internal op fragmentdir:mds.0:3792 currently failed to wrlock, waiting 2024-03-28T01:25:15.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:25:15 smithi132 ceph-mon[35941]: slow request 2971.594449 seconds old, received at 2024-03-28T00:35:43.468389+0000: internal op fragmentdir:mds.0:3823 currently failed to wrlock, waiting 2024-03-28T01:25:15.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:25:15 smithi132 ceph-mon[35941]: slow request 2971.611733 seconds old, received at 2024-03-28T00:35:43.451105+0000: internal op fragmentdir:mds.0:3793 currently failed to wrlock, waiting 2024-03-28T01:25:15.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:25:15 smithi132 ceph-mon[35941]: slow request 2971.566891 seconds old, received at 2024-03-28T00:35:43.495948+0000: internal op fragmentdir:mds.0:3888 currently failed to wrlock, waiting 2024-03-28T01:25:16.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:25:16 smithi165 ceph-mon[36302]: pgmap v1855: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:25:16.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:25:16 smithi119 ceph-mon[31518]: pgmap v1855: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:25:16.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:25:16 smithi132 ceph-mon[35941]: pgmap v1855: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:25:18.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:25:18 smithi119 ceph-mon[31518]: pgmap v1856: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:25:18.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:25:18 smithi132 ceph-mon[35941]: pgmap v1856: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:25:18.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:25:18 smithi165 ceph-mon[36302]: pgmap v1856: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:25:20.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:25:20 smithi119 ceph-mon[31518]: pgmap v1857: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:25:20.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:25:20 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2980.703956 secs 2024-03-28T01:25:20.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:25:20 smithi132 ceph-mon[35941]: pgmap v1857: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:25:20.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:25:20 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2980.703956 secs 2024-03-28T01:25:20.877 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:25:20 smithi165 ceph-mon[36302]: pgmap v1857: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:25:20.877 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:25:20 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2980.703956 secs 2024-03-28T01:25:22.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:25:22 smithi119 ceph-mon[31518]: pgmap v1858: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:25:22.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:25:22 smithi132 ceph-mon[35941]: pgmap v1858: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:25:22.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:25:22 smithi165 ceph-mon[36302]: pgmap v1858: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:25:23.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:25:23 smithi165 ceph-mon[36302]: pgmap v1859: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:25:23.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:25:23 smithi119 ceph-mon[31518]: pgmap v1859: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:25:23.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:25:23 smithi132 ceph-mon[35941]: pgmap v1859: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:25:26.126 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:25:25 smithi165 ceph-mon[36302]: pgmap v1860: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:25:26.127 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:25:25 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2985.704021 secs 2024-03-28T01:25:26.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:25:25 smithi119 ceph-mon[31518]: pgmap v1860: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:25:26.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:25:25 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2985.704021 secs 2024-03-28T01:25:26.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:25:25 smithi132 ceph-mon[35941]: pgmap v1860: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:25:26.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:25:25 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2985.704021 secs 2024-03-28T01:25:28.126 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:25:27 smithi165 ceph-mon[36302]: pgmap v1861: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:25:28.149 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:25:27 smithi119 ceph-mon[31518]: pgmap v1861: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:25:28.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:25:27 smithi132 ceph-mon[35941]: pgmap v1861: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:25:30.126 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:25:29 smithi165 ceph-mon[36302]: pgmap v1862: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:25:30.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:25:29 smithi119 ceph-mon[31518]: pgmap v1862: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:25:30.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:25:29 smithi132 ceph-mon[35941]: pgmap v1862: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:25:31.126 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:25:30 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2990.704079 secs 2024-03-28T01:25:31.149 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:25:30 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2990.704079 secs 2024-03-28T01:25:31.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:25:30 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2990.704079 secs 2024-03-28T01:25:32.126 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:25:31 smithi165 ceph-mon[36302]: pgmap v1863: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:25:32.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:25:31 smithi119 ceph-mon[31518]: pgmap v1863: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:25:32.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:25:31 smithi132 ceph-mon[35941]: pgmap v1863: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:25:34.127 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:25:33 smithi165 ceph-mon[36302]: pgmap v1864: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:25:34.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:25:33 smithi119 ceph-mon[31518]: pgmap v1864: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:25:34.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:25:33 smithi132 ceph-mon[35941]: pgmap v1864: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:25:36.127 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:25:35 smithi165 ceph-mon[36302]: pgmap v1865: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:25:36.127 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:25:35 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 2995.704141 secs 2024-03-28T01:25:36.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:25:35 smithi119 ceph-mon[31518]: pgmap v1865: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:25:36.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:25:35 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 2995.704141 secs 2024-03-28T01:25:36.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:25:35 smithi132 ceph-mon[35941]: pgmap v1865: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:25:36.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:25:35 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 2995.704141 secs 2024-03-28T01:25:38.126 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:25:37 smithi165 ceph-mon[36302]: pgmap v1866: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:25:38.149 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:25:37 smithi119 ceph-mon[31518]: pgmap v1866: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:25:38.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:25:37 smithi132 ceph-mon[35941]: pgmap v1866: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:25:40.126 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:25:39 smithi165 ceph-mon[36302]: pgmap v1867: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:25:40.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:25:39 smithi119 ceph-mon[31518]: pgmap v1867: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:25:40.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:25:39 smithi132 ceph-mon[35941]: pgmap v1867: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:25:41.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:25:40 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3000.704204 secs 2024-03-28T01:25:41.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:25:40 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3000.704204 secs 2024-03-28T01:25:41.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:25:40 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3000.704204 secs 2024-03-28T01:25:42.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:25:41 smithi119 ceph-mon[31518]: pgmap v1868: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:25:42.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:25:41 smithi132 ceph-mon[35941]: pgmap v1868: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:25:42.278 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:25:41 smithi165 ceph-mon[36302]: pgmap v1868: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:25:44.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:25:43 smithi119 ceph-mon[31518]: pgmap v1869: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:25:44.196 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:25:43 smithi132 ceph-mon[35941]: pgmap v1869: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:25:44.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:25:43 smithi165 ceph-mon[36302]: pgmap v1869: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:25:46.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:25:45 smithi119 ceph-mon[31518]: pgmap v1870: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:25:46.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:25:45 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3005.704267 secs 2024-03-28T01:25:46.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:25:45 smithi132 ceph-mon[35941]: pgmap v1870: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:25:46.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:25:45 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3005.704267 secs 2024-03-28T01:25:46.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:25:45 smithi165 ceph-mon[36302]: pgmap v1870: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:25:46.377 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:25:45 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3005.704267 secs 2024-03-28T01:25:48.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:25:47 smithi119 ceph-mon[31518]: pgmap v1871: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:25:48.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:25:47 smithi132 ceph-mon[35941]: pgmap v1871: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:25:48.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:25:47 smithi165 ceph-mon[36302]: pgmap v1871: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:25:50.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:25:49 smithi132 ceph-mon[35941]: pgmap v1872: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:25:50.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:25:49 smithi165 ceph-mon[36302]: pgmap v1872: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:25:50.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:25:49 smithi119 ceph-mon[31518]: pgmap v1872: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:25:51.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:25:50 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3010.704327 secs 2024-03-28T01:25:51.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:25:50 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3010.704327 secs 2024-03-28T01:25:51.400 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:25:50 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3010.704327 secs 2024-03-28T01:25:52.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:25:51 smithi132 ceph-mon[35941]: pgmap v1873: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:25:52.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:25:51 smithi165 ceph-mon[36302]: pgmap v1873: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:25:52.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:25:51 smithi119 ceph-mon[31518]: pgmap v1873: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:25:54.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:25:53 smithi132 ceph-mon[35941]: pgmap v1874: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:25:54.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:25:53 smithi165 ceph-mon[36302]: pgmap v1874: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:25:54.400 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:25:53 smithi119 ceph-mon[31518]: pgmap v1874: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:25:56.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:25:55 smithi132 ceph-mon[35941]: pgmap v1875: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:25:56.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:25:55 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3015.704370 secs 2024-03-28T01:25:56.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:25:55 smithi165 ceph-mon[36302]: pgmap v1875: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:25:56.377 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:25:55 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3015.704370 secs 2024-03-28T01:25:56.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:25:55 smithi119 ceph-mon[31518]: pgmap v1875: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:25:56.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:25:55 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3015.704370 secs 2024-03-28T01:25:58.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:25:57 smithi132 ceph-mon[35941]: pgmap v1876: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:25:58.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:25:57 smithi165 ceph-mon[36302]: pgmap v1876: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:25:58.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:25:57 smithi119 ceph-mon[31518]: pgmap v1876: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:26:00.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:25:59 smithi132 ceph-mon[35941]: pgmap v1877: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:26:00.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:25:59 smithi165 ceph-mon[36302]: pgmap v1877: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:26:00.400 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:25:59 smithi119 ceph-mon[31518]: pgmap v1877: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:26:01.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:26:00 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3020.704459 secs 2024-03-28T01:26:01.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:26:00 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3020.704459 secs 2024-03-28T01:26:01.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:26:00 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3020.704459 secs 2024-03-28T01:26:02.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:26:01 smithi165 ceph-mon[36302]: pgmap v1878: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:26:02.400 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:26:01 smithi119 ceph-mon[31518]: pgmap v1878: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:26:02.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:26:01 smithi132 ceph-mon[35941]: pgmap v1878: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:26:04.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:26:03 smithi165 ceph-mon[36302]: pgmap v1879: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:26:04.377 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:26:03 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:26:04.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:26:03 smithi119 ceph-mon[31518]: pgmap v1879: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:26:04.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:26:03 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:26:04.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:26:03 smithi132 ceph-mon[35941]: pgmap v1879: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:26:04.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:26:03 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:26:05.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:26:04 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:26:05.377 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:26:04 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:26:05.377 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:26:04 smithi165 ceph-mon[36302]: pgmap v1880: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T01:26:05.377 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:26:04 smithi165 ceph-mon[36302]: pgmap v1881: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 5 op/s 2024-03-28T01:26:05.377 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:26:04 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:26:05.400 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:26:04 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:26:05.400 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:26:04 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:26:05.400 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:26:04 smithi119 ceph-mon[31518]: pgmap v1880: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T01:26:05.400 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:26:04 smithi119 ceph-mon[31518]: pgmap v1881: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 5 op/s 2024-03-28T01:26:05.400 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:26:04 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:26:05.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:26:04 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:26:05.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:26:04 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:26:05.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:26:04 smithi132 ceph-mon[35941]: pgmap v1880: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T01:26:05.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:26:04 smithi132 ceph-mon[35941]: pgmap v1881: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 5 op/s 2024-03-28T01:26:05.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:26:04 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:26:06.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:26:05 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3025.704548 secs 2024-03-28T01:26:06.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:26:05 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3025.704548 secs 2024-03-28T01:26:06.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:26:05 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3025.704548 secs 2024-03-28T01:26:07.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:26:06 smithi165 ceph-mon[36302]: pgmap v1882: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:26:07.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:26:06 smithi119 ceph-mon[31518]: pgmap v1882: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:26:07.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:26:06 smithi132 ceph-mon[35941]: pgmap v1882: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:26:09.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:26:09 smithi165 ceph-mon[36302]: pgmap v1883: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:26:09.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:26:09 smithi119 ceph-mon[31518]: pgmap v1883: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:26:09.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:26:09 smithi132 ceph-mon[35941]: pgmap v1883: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:26:10.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:26:10 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3030.704637 secs 2024-03-28T01:26:10.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:26:10 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3030.704637 secs 2024-03-28T01:26:10.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:26:10 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3030.704637 secs 2024-03-28T01:26:11.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:26:11 smithi165 ceph-mon[36302]: pgmap v1884: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:26:11.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:26:11 smithi119 ceph-mon[31518]: pgmap v1884: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:26:11.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:26:11 smithi132 ceph-mon[35941]: pgmap v1884: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:26:13.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:26:13 smithi165 ceph-mon[36302]: pgmap v1885: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:26:13.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:26:13 smithi119 ceph-mon[31518]: pgmap v1885: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:26:13.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:26:13 smithi132 ceph-mon[35941]: pgmap v1885: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:26:15.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:26:15 smithi165 ceph-mon[36302]: pgmap v1886: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T01:26:15.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:26:15 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3035.704724 secs 2024-03-28T01:26:15.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:26:15 smithi119 ceph-mon[31518]: pgmap v1886: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T01:26:15.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:26:15 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3035.704724 secs 2024-03-28T01:26:15.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:26:15 smithi132 ceph-mon[35941]: pgmap v1886: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T01:26:15.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:26:15 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3035.704724 secs 2024-03-28T01:26:17.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:26:17 smithi165 ceph-mon[36302]: pgmap v1887: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-03-28T01:26:17.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:26:17 smithi119 ceph-mon[31518]: pgmap v1887: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-03-28T01:26:17.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:26:17 smithi132 ceph-mon[35941]: pgmap v1887: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-03-28T01:26:19.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:26:19 smithi165 ceph-mon[36302]: pgmap v1888: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:26:19.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:26:19 smithi119 ceph-mon[31518]: pgmap v1888: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:26:19.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:26:19 smithi132 ceph-mon[35941]: pgmap v1888: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:26:20.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:26:20 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3040.704814 secs 2024-03-28T01:26:20.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:26:20 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3040.704814 secs 2024-03-28T01:26:20.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:26:20 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3040.704814 secs 2024-03-28T01:26:21.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:26:21 smithi165 ceph-mon[36302]: pgmap v1889: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:26:21.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:26:21 smithi119 ceph-mon[31518]: pgmap v1889: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:26:21.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:26:21 smithi132 ceph-mon[35941]: pgmap v1889: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:26:23.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:26:23 smithi165 ceph-mon[36302]: pgmap v1890: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:26:23.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:26:23 smithi119 ceph-mon[31518]: pgmap v1890: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:26:23.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:26:23 smithi132 ceph-mon[35941]: pgmap v1890: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:26:25.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:26:25 smithi119 ceph-mon[31518]: pgmap v1891: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-28T01:26:25.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:26:25 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3045.704899 secs 2024-03-28T01:26:25.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:26:25 smithi132 ceph-mon[35941]: pgmap v1891: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-28T01:26:25.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:26:25 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3045.704899 secs 2024-03-28T01:26:25.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:26:25 smithi165 ceph-mon[36302]: pgmap v1891: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-28T01:26:25.877 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:26:25 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3045.704899 secs 2024-03-28T01:26:27.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:26:27 smithi119 ceph-mon[31518]: pgmap v1892: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-03-28T01:26:27.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:26:27 smithi132 ceph-mon[35941]: pgmap v1892: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-03-28T01:26:27.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:26:27 smithi165 ceph-mon[36302]: pgmap v1892: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-03-28T01:26:29.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:26:29 smithi119 ceph-mon[31518]: pgmap v1893: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:26:29.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:26:29 smithi132 ceph-mon[35941]: pgmap v1893: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:26:29.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:26:29 smithi165 ceph-mon[36302]: pgmap v1893: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:26:30.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:26:30 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3050.704960 secs 2024-03-28T01:26:30.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:26:30 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3050.704960 secs 2024-03-28T01:26:30.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:26:30 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3050.704960 secs 2024-03-28T01:26:31.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:26:31 smithi132 ceph-mon[35941]: pgmap v1894: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:26:31.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:26:31 smithi165 ceph-mon[36302]: pgmap v1894: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:26:31.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:26:31 smithi119 ceph-mon[31518]: pgmap v1894: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:26:32.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:26:32 smithi132 ceph-mon[35941]: pgmap v1895: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:26:32.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:26:32 smithi165 ceph-mon[36302]: pgmap v1895: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:26:32.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:26:32 smithi119 ceph-mon[31518]: pgmap v1895: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:26:35.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:26:35 smithi165 ceph-mon[36302]: pgmap v1896: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-03-28T01:26:35.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:26:35 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3055.705051 secs 2024-03-28T01:26:35.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:26:35 smithi119 ceph-mon[31518]: pgmap v1896: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-03-28T01:26:35.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:26:35 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3055.705051 secs 2024-03-28T01:26:35.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:26:35 smithi132 ceph-mon[35941]: pgmap v1896: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-03-28T01:26:35.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:26:35 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3055.705051 secs 2024-03-28T01:26:37.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:26:37 smithi165 ceph-mon[36302]: pgmap v1897: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 5 op/s 2024-03-28T01:26:37.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:26:37 smithi119 ceph-mon[31518]: pgmap v1897: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 5 op/s 2024-03-28T01:26:37.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:26:37 smithi132 ceph-mon[35941]: pgmap v1897: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 5 op/s 2024-03-28T01:26:39.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:26:39 smithi165 ceph-mon[36302]: pgmap v1898: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-03-28T01:26:39.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:26:39 smithi119 ceph-mon[31518]: pgmap v1898: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-03-28T01:26:39.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:26:39 smithi132 ceph-mon[35941]: pgmap v1898: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-03-28T01:26:40.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:26:40 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3060.705135 secs 2024-03-28T01:26:40.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:26:40 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3060.705135 secs 2024-03-28T01:26:40.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:26:40 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3060.705135 secs 2024-03-28T01:26:41.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:26:41 smithi165 ceph-mon[36302]: pgmap v1899: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-03-28T01:26:41.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:26:41 smithi119 ceph-mon[31518]: pgmap v1899: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-03-28T01:26:41.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:26:41 smithi132 ceph-mon[35941]: pgmap v1899: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-03-28T01:26:43.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:26:43 smithi165 ceph-mon[36302]: pgmap v1900: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-03-28T01:26:43.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:26:43 smithi119 ceph-mon[31518]: pgmap v1900: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-03-28T01:26:43.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:26:43 smithi132 ceph-mon[35941]: pgmap v1900: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-03-28T01:26:45.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:26:45 smithi165 ceph-mon[36302]: pgmap v1901: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-03-28T01:26:45.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:26:45 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3065.705198 secs 2024-03-28T01:26:45.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:26:45 smithi119 ceph-mon[31518]: pgmap v1901: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-03-28T01:26:45.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:26:45 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3065.705198 secs 2024-03-28T01:26:45.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:26:45 smithi132 ceph-mon[35941]: pgmap v1901: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-03-28T01:26:45.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:26:45 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3065.705198 secs 2024-03-28T01:26:47.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:26:47 smithi165 ceph-mon[36302]: pgmap v1902: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:26:47.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:26:47 smithi119 ceph-mon[31518]: pgmap v1902: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:26:47.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:26:47 smithi132 ceph-mon[35941]: pgmap v1902: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:26:49.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:26:49 smithi119 ceph-mon[31518]: pgmap v1903: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:26:49.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:26:49 smithi132 ceph-mon[35941]: pgmap v1903: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:26:49.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:26:49 smithi165 ceph-mon[36302]: pgmap v1903: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:26:50.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:26:50 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3070.705288 secs 2024-03-28T01:26:50.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:26:50 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3070.705288 secs 2024-03-28T01:26:50.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:26:50 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3070.705288 secs 2024-03-28T01:26:51.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:26:51 smithi119 ceph-mon[31518]: pgmap v1904: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:26:51.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:26:51 smithi132 ceph-mon[35941]: pgmap v1904: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:26:51.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:26:51 smithi165 ceph-mon[36302]: pgmap v1904: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:26:53.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:26:53 smithi119 ceph-mon[31518]: pgmap v1905: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:26:53.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:26:53 smithi132 ceph-mon[35941]: pgmap v1905: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:26:53.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:26:53 smithi165 ceph-mon[36302]: pgmap v1905: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:26:55.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:26:55 smithi132 ceph-mon[35941]: pgmap v1906: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:26:55.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:26:55 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3075.705355 secs 2024-03-28T01:26:55.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:26:55 smithi165 ceph-mon[36302]: pgmap v1906: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:26:55.877 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:26:55 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3075.705355 secs 2024-03-28T01:26:55.900 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:26:55 smithi119 ceph-mon[31518]: pgmap v1906: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:26:55.900 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:26:55 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3075.705355 secs 2024-03-28T01:26:57.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:26:57 smithi132 ceph-mon[35941]: pgmap v1907: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:26:57.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:26:57 smithi165 ceph-mon[36302]: pgmap v1907: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:26:57.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:26:57 smithi119 ceph-mon[31518]: pgmap v1907: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:26:58.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:26:58 smithi165 ceph-mon[36302]: pgmap v1908: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:26:58.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:26:58 smithi119 ceph-mon[31518]: pgmap v1908: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:26:58.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:26:58 smithi132 ceph-mon[35941]: pgmap v1908: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:27:00.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:27:00 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3080.705414 secs 2024-03-28T01:27:00.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:27:00 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3080.705414 secs 2024-03-28T01:27:00.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:27:00 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3080.705414 secs 2024-03-28T01:27:01.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:27:01 smithi165 ceph-mon[36302]: pgmap v1909: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:27:01.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:27:01 smithi119 ceph-mon[31518]: pgmap v1909: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:27:01.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:27:01 smithi132 ceph-mon[35941]: pgmap v1909: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:27:03.126 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:27:02 smithi165 ceph-mon[36302]: pgmap v1910: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:27:03.149 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:27:02 smithi119 ceph-mon[31518]: pgmap v1910: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:27:03.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:27:02 smithi132 ceph-mon[35941]: pgmap v1910: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:27:05.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:27:05 smithi165 ceph-mon[36302]: pgmap v1911: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:27:05.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:27:05 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:27:05.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:27:05 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3085.705480 secs 2024-03-28T01:27:05.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:27:05 smithi119 ceph-mon[31518]: pgmap v1911: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:27:05.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:27:05 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:27:05.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:27:05 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3085.705480 secs 2024-03-28T01:27:05.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:27:05 smithi132 ceph-mon[35941]: pgmap v1911: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:27:05.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:27:05 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:27:05.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:27:05 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3085.705480 secs 2024-03-28T01:27:07.127 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:27:06 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:27:07.127 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:27:06 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:27:07.127 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:27:06 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:27:07.127 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:27:06 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:27:07.127 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:27:06 smithi165 ceph-mon[36302]: pgmap v1912: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:27:07.127 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:27:06 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:27:07.127 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:27:06 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:27:07.127 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:27:06 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:27:07.127 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:27:06 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:27:07.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:27:06 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:27:07.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:27:06 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:27:07.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:27:06 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:27:07.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:27:06 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:27:07.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:27:06 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:27:07.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:27:06 smithi119 ceph-mon[31518]: pgmap v1912: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:27:07.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:27:06 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:27:07.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:27:06 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:27:07.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:27:06 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:27:07.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:27:06 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:27:07.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:27:06 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:27:07.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:27:06 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:27:07.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:27:06 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:27:07.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:27:06 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:27:07.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:27:06 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:27:07.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:27:06 smithi132 ceph-mon[35941]: pgmap v1912: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:27:07.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:27:06 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:27:07.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:27:06 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:27:07.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:27:06 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:27:07.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:27:06 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:27:07.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:27:06 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:27:08.126 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:27:07 smithi165 ceph-mon[36302]: pgmap v1913: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-28T01:27:08.127 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:27:07 smithi165 ceph-mon[36302]: pgmap v1914: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-03-28T01:27:08.149 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:27:07 smithi119 ceph-mon[31518]: pgmap v1913: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-28T01:27:08.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:27:07 smithi119 ceph-mon[31518]: pgmap v1914: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-03-28T01:27:08.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:27:07 smithi132 ceph-mon[35941]: pgmap v1913: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-28T01:27:08.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:27:07 smithi132 ceph-mon[35941]: pgmap v1914: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-03-28T01:27:10.126 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:27:09 smithi165 ceph-mon[36302]: pgmap v1915: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 3 op/s 2024-03-28T01:27:10.149 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:27:09 smithi119 ceph-mon[31518]: pgmap v1915: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 3 op/s 2024-03-28T01:27:10.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:27:09 smithi132 ceph-mon[35941]: pgmap v1915: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 3 op/s 2024-03-28T01:27:11.126 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:27:10 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3090.705567 secs 2024-03-28T01:27:11.149 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:27:10 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3090.705567 secs 2024-03-28T01:27:11.196 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:27:10 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3090.705567 secs 2024-03-28T01:27:12.126 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:27:11 smithi165 ceph-mon[36302]: pgmap v1916: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-03-28T01:27:12.149 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:27:11 smithi119 ceph-mon[31518]: pgmap v1916: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-03-28T01:27:12.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:27:11 smithi132 ceph-mon[35941]: pgmap v1916: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-03-28T01:27:14.126 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:27:13 smithi165 ceph-mon[36302]: pgmap v1917: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 3 op/s 2024-03-28T01:27:14.149 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:27:13 smithi119 ceph-mon[31518]: pgmap v1917: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 3 op/s 2024-03-28T01:27:14.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:27:13 smithi132 ceph-mon[35941]: pgmap v1917: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 3 op/s 2024-03-28T01:27:16.126 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:27:15 smithi165 ceph-mon[36302]: pgmap v1918: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-03-28T01:27:16.126 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:27:15 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3095.705646 secs 2024-03-28T01:27:16.149 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:27:15 smithi119 ceph-mon[31518]: pgmap v1918: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-03-28T01:27:16.149 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:27:15 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3095.705646 secs 2024-03-28T01:27:16.196 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:27:15 smithi132 ceph-mon[35941]: pgmap v1918: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-03-28T01:27:16.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:27:15 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3095.705646 secs 2024-03-28T01:27:18.126 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:27:17 smithi165 ceph-mon[36302]: pgmap v1919: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T01:27:18.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:27:17 smithi119 ceph-mon[31518]: pgmap v1919: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T01:27:18.196 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:27:17 smithi132 ceph-mon[35941]: pgmap v1919: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T01:27:20.126 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:27:19 smithi165 ceph-mon[36302]: pgmap v1920: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:27:20.149 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:27:19 smithi119 ceph-mon[31518]: pgmap v1920: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:27:20.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:27:19 smithi132 ceph-mon[35941]: pgmap v1920: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:27:21.126 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:27:20 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3100.705707 secs 2024-03-28T01:27:21.149 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:27:20 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3100.705707 secs 2024-03-28T01:27:21.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:27:20 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3100.705707 secs 2024-03-28T01:27:22.149 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:27:21 smithi119 ceph-mon[31518]: pgmap v1921: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:27:22.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:27:21 smithi132 ceph-mon[35941]: pgmap v1921: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:27:22.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:27:21 smithi165 ceph-mon[36302]: pgmap v1921: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:27:24.149 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:27:23 smithi119 ceph-mon[31518]: pgmap v1922: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:27:24.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:27:23 smithi132 ceph-mon[35941]: pgmap v1922: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:27:24.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:27:23 smithi165 ceph-mon[36302]: pgmap v1922: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:27:26.149 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:27:25 smithi119 ceph-mon[31518]: pgmap v1923: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:27:26.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:27:25 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3105.705768 secs 2024-03-28T01:27:26.196 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:27:25 smithi132 ceph-mon[35941]: pgmap v1923: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:27:26.196 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:27:25 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3105.705768 secs 2024-03-28T01:27:26.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:27:25 smithi165 ceph-mon[36302]: pgmap v1923: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:27:26.377 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:27:25 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3105.705768 secs 2024-03-28T01:27:28.149 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:27:27 smithi119 ceph-mon[31518]: pgmap v1924: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:27:28.196 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:27:27 smithi132 ceph-mon[35941]: pgmap v1924: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:27:28.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:27:27 smithi165 ceph-mon[36302]: pgmap v1924: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:27:30.149 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:27:29 smithi119 ceph-mon[31518]: pgmap v1925: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:27:30.196 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:27:29 smithi132 ceph-mon[35941]: pgmap v1925: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:27:30.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:27:29 smithi165 ceph-mon[36302]: pgmap v1925: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:27:31.196 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:27:30 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3110.705830 secs 2024-03-28T01:27:31.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:27:30 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3110.705830 secs 2024-03-28T01:27:31.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:27:30 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3110.705830 secs 2024-03-28T01:27:32.196 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:27:31 smithi132 ceph-mon[35941]: pgmap v1926: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:27:32.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:27:31 smithi165 ceph-mon[36302]: pgmap v1926: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:27:32.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:27:31 smithi119 ceph-mon[31518]: pgmap v1926: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:27:34.196 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:27:33 smithi132 ceph-mon[35941]: pgmap v1927: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:27:34.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:27:33 smithi165 ceph-mon[36302]: pgmap v1927: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:27:34.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:27:33 smithi119 ceph-mon[31518]: pgmap v1927: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:27:36.196 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:27:35 smithi132 ceph-mon[35941]: pgmap v1928: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:27:36.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:27:35 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3115.705892 secs 2024-03-28T01:27:36.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:27:35 smithi165 ceph-mon[36302]: pgmap v1928: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:27:36.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:27:35 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3115.705892 secs 2024-03-28T01:27:36.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:27:35 smithi119 ceph-mon[31518]: pgmap v1928: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:27:36.400 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:27:35 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3115.705892 secs 2024-03-28T01:27:38.196 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:27:37 smithi132 ceph-mon[35941]: pgmap v1929: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:27:38.272 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:27:37 smithi119 ceph-mon[31518]: pgmap v1929: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:27:38.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:27:37 smithi165 ceph-mon[36302]: pgmap v1929: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:27:40.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:27:39 smithi132 ceph-mon[35941]: pgmap v1930: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:27:40.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:27:39 smithi165 ceph-mon[36302]: pgmap v1930: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:27:40.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:27:39 smithi119 ceph-mon[31518]: pgmap v1930: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:27:41.196 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:27:40 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3120.705955 secs 2024-03-28T01:27:41.301 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:27:40 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3120.705955 secs 2024-03-28T01:27:41.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:27:40 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3120.705955 secs 2024-03-28T01:27:42.196 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:27:41 smithi132 ceph-mon[35941]: pgmap v1931: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:27:42.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:27:41 smithi165 ceph-mon[36302]: pgmap v1931: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:27:42.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:27:41 smithi119 ceph-mon[31518]: pgmap v1931: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:27:44.196 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:27:43 smithi132 ceph-mon[35941]: pgmap v1932: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:27:44.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:27:43 smithi165 ceph-mon[36302]: pgmap v1932: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:27:44.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:27:43 smithi119 ceph-mon[31518]: pgmap v1932: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:27:46.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:27:45 smithi165 ceph-mon[36302]: pgmap v1933: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:27:46.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:27:45 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3125.706017 secs 2024-03-28T01:27:46.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:27:45 smithi119 ceph-mon[31518]: pgmap v1933: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:27:46.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:27:45 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3125.706017 secs 2024-03-28T01:27:46.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:27:45 smithi132 ceph-mon[35941]: pgmap v1933: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:27:46.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:27:45 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3125.706017 secs 2024-03-28T01:27:48.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:27:47 smithi165 ceph-mon[36302]: pgmap v1934: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:27:48.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:27:47 smithi119 ceph-mon[31518]: pgmap v1934: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:27:48.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:27:47 smithi132 ceph-mon[35941]: pgmap v1934: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:27:50.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:27:49 smithi165 ceph-mon[36302]: pgmap v1935: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:27:50.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:27:49 smithi119 ceph-mon[31518]: pgmap v1935: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:27:50.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:27:49 smithi132 ceph-mon[35941]: pgmap v1935: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:27:51.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:27:50 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3130.706082 secs 2024-03-28T01:27:51.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:27:50 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3130.706082 secs 2024-03-28T01:27:51.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:27:50 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3130.706082 secs 2024-03-28T01:27:52.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:27:51 smithi165 ceph-mon[36302]: pgmap v1936: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:27:52.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:27:51 smithi119 ceph-mon[31518]: pgmap v1936: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:27:52.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:27:51 smithi132 ceph-mon[35941]: pgmap v1936: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:27:54.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:27:53 smithi165 ceph-mon[36302]: pgmap v1937: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:27:54.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:27:53 smithi119 ceph-mon[31518]: pgmap v1937: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:27:54.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:27:53 smithi132 ceph-mon[35941]: pgmap v1937: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:27:56.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:27:55 smithi165 ceph-mon[36302]: pgmap v1938: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:27:56.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:27:55 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3135.706170 secs 2024-03-28T01:27:56.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:27:55 smithi119 ceph-mon[31518]: pgmap v1938: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:27:56.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:27:55 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3135.706170 secs 2024-03-28T01:27:56.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:27:55 smithi132 ceph-mon[35941]: pgmap v1938: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:27:56.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:27:55 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3135.706170 secs 2024-03-28T01:27:58.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:27:57 smithi165 ceph-mon[36302]: pgmap v1939: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:27:58.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:27:57 smithi119 ceph-mon[31518]: pgmap v1939: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:27:58.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:27:57 smithi132 ceph-mon[35941]: pgmap v1939: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:28:00.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:27:59 smithi165 ceph-mon[36302]: pgmap v1940: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:28:00.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:27:59 smithi119 ceph-mon[31518]: pgmap v1940: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:28:00.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:27:59 smithi132 ceph-mon[35941]: pgmap v1940: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:28:01.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:28:01 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3140.706259 secs 2024-03-28T01:28:01.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:28:00 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3140.706259 secs 2024-03-28T01:28:01.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:28:00 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3140.706259 secs 2024-03-28T01:28:02.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:28:02 smithi165 ceph-mon[36302]: pgmap v1941: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:28:02.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:28:02 smithi119 ceph-mon[31518]: pgmap v1941: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:28:02.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:28:02 smithi132 ceph-mon[35941]: pgmap v1941: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:28:04.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:28:04 smithi165 ceph-mon[36302]: pgmap v1942: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:28:04.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:28:04 smithi119 ceph-mon[31518]: pgmap v1942: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:28:04.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:28:04 smithi132 ceph-mon[35941]: pgmap v1942: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:28:06.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:28:06 smithi165 ceph-mon[36302]: pgmap v1943: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:28:06.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:28:06 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3145.706349 secs 2024-03-28T01:28:06.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:28:06 smithi119 ceph-mon[31518]: pgmap v1943: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:28:06.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:28:06 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3145.706349 secs 2024-03-28T01:28:06.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:28:06 smithi132 ceph-mon[35941]: pgmap v1943: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:28:06.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:28:06 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3145.706349 secs 2024-03-28T01:28:07.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:28:07 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:28:07.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:28:07 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:28:07.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:28:07 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:28:08.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:28:08 smithi165 ceph-mon[36302]: pgmap v1944: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:28:08.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:28:08 smithi119 ceph-mon[31518]: pgmap v1944: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:28:08.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:28:08 smithi132 ceph-mon[35941]: pgmap v1944: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:28:09.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:28:09 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:28:09.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:28:09 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:28:09.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:28:09 smithi165 ceph-mon[36302]: pgmap v1945: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T01:28:09.377 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:28:09 smithi165 ceph-mon[36302]: pgmap v1946: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:28:09.377 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:28:09 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:28:09.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:28:09 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:28:09.400 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:28:09 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:28:09.400 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:28:09 smithi119 ceph-mon[31518]: pgmap v1945: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T01:28:09.400 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:28:09 smithi119 ceph-mon[31518]: pgmap v1946: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:28:09.400 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:28:09 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:28:09.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:28:09 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:28:09.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:28:09 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:28:09.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:28:09 smithi132 ceph-mon[35941]: pgmap v1945: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T01:28:09.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:28:09 smithi132 ceph-mon[35941]: pgmap v1946: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:28:09.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:28:09 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:28:11.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:28:11 smithi165 ceph-mon[36302]: pgmap v1947: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:28:11.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:28:11 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3150.706435 secs 2024-03-28T01:28:11.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:28:11 smithi119 ceph-mon[31518]: pgmap v1947: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:28:11.400 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:28:11 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3150.706435 secs 2024-03-28T01:28:11.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:28:11 smithi132 ceph-mon[35941]: pgmap v1947: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:28:11.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:28:11 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3150.706435 secs 2024-03-28T01:28:13.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:28:13 smithi165 ceph-mon[36302]: pgmap v1948: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:28:13.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:28:13 smithi119 ceph-mon[31518]: pgmap v1948: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:28:13.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:28:13 smithi132 ceph-mon[35941]: pgmap v1948: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:28:15.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:28:15 smithi165 ceph-mon[36302]: pgmap v1949: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T01:28:15.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:28:15 smithi119 ceph-mon[31518]: pgmap v1949: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T01:28:15.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:28:15 smithi132 ceph-mon[35941]: pgmap v1949: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T01:28:16.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:28:16 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3155.706469 secs 2024-03-28T01:28:16.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:28:16 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3155.706469 secs 2024-03-28T01:28:16.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:28:16 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3155.706469 secs 2024-03-28T01:28:17.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:28:17 smithi165 ceph-mon[36302]: pgmap v1950: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:28:17.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:28:17 smithi119 ceph-mon[31518]: pgmap v1950: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:28:17.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:28:17 smithi132 ceph-mon[35941]: pgmap v1950: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:28:19.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:28:19 smithi165 ceph-mon[36302]: pgmap v1951: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T01:28:19.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:28:19 smithi119 ceph-mon[31518]: pgmap v1951: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T01:28:19.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:28:19 smithi132 ceph-mon[35941]: pgmap v1951: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T01:28:21.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:28:21 smithi119 ceph-mon[31518]: pgmap v1952: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:28:21.400 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:28:21 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3160.706530 secs 2024-03-28T01:28:21.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:28:21 smithi132 ceph-mon[35941]: pgmap v1952: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:28:21.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:28:21 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3160.706530 secs 2024-03-28T01:28:21.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:28:21 smithi165 ceph-mon[36302]: pgmap v1952: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:28:21.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:28:21 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3160.706530 secs 2024-03-28T01:28:23.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:28:23 smithi119 ceph-mon[31518]: pgmap v1953: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:28:23.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:28:23 smithi132 ceph-mon[35941]: pgmap v1953: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:28:23.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:28:23 smithi165 ceph-mon[36302]: pgmap v1953: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:28:25.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:28:25 smithi119 ceph-mon[31518]: pgmap v1954: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-28T01:28:25.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:28:25 smithi132 ceph-mon[35941]: pgmap v1954: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-28T01:28:25.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:28:25 smithi165 ceph-mon[36302]: pgmap v1954: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-28T01:28:26.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:28:26 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3165.706616 secs 2024-03-28T01:28:26.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:28:26 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3165.706616 secs 2024-03-28T01:28:26.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:28:26 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3165.706616 secs 2024-03-28T01:28:27.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:28:27 smithi132 ceph-mon[35941]: pgmap v1955: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-28T01:28:27.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:28:27 smithi165 ceph-mon[36302]: pgmap v1955: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-28T01:28:27.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:28:27 smithi119 ceph-mon[31518]: pgmap v1955: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-28T01:28:29.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:28:29 smithi132 ceph-mon[35941]: pgmap v1956: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:28:29.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:28:29 smithi165 ceph-mon[36302]: pgmap v1956: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:28:29.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:28:29 smithi119 ceph-mon[31518]: pgmap v1956: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:28:31.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:28:31 smithi132 ceph-mon[35941]: pgmap v1957: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:28:31.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:28:31 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3170.706702 secs 2024-03-28T01:28:31.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:28:31 smithi165 ceph-mon[36302]: pgmap v1957: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:28:31.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:28:31 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3170.706702 secs 2024-03-28T01:28:31.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:28:31 smithi119 ceph-mon[31518]: pgmap v1957: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:28:31.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:28:31 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3170.706702 secs 2024-03-28T01:28:33.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:28:33 smithi132 ceph-mon[35941]: pgmap v1958: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:28:33.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:28:33 smithi165 ceph-mon[36302]: pgmap v1958: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:28:33.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:28:33 smithi119 ceph-mon[31518]: pgmap v1958: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:28:35.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:28:35 smithi132 ceph-mon[35941]: pgmap v1959: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-28T01:28:35.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:28:35 smithi165 ceph-mon[36302]: pgmap v1959: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-28T01:28:35.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:28:35 smithi119 ceph-mon[31518]: pgmap v1959: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-28T01:28:36.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:28:36 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3175.706791 secs 2024-03-28T01:28:36.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:28:36 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3175.706791 secs 2024-03-28T01:28:36.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:28:36 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3175.706791 secs 2024-03-28T01:28:37.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:28:37 smithi132 ceph-mon[35941]: pgmap v1960: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-28T01:28:37.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:28:37 smithi165 ceph-mon[36302]: pgmap v1960: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-28T01:28:37.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:28:37 smithi119 ceph-mon[31518]: pgmap v1960: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-28T01:28:39.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:28:39 smithi132 ceph-mon[35941]: pgmap v1961: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:28:39.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:28:39 smithi165 ceph-mon[36302]: pgmap v1961: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:28:39.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:28:39 smithi119 ceph-mon[31518]: pgmap v1961: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:28:41.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:28:41 smithi132 ceph-mon[35941]: pgmap v1962: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:28:41.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:28:41 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3180.706886 secs 2024-03-28T01:28:41.551 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:28:41 smithi119 ceph-mon[31518]: pgmap v1962: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:28:41.551 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:28:41 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3180.706886 secs 2024-03-28T01:28:41.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:28:41 smithi165 ceph-mon[36302]: pgmap v1962: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:28:41.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:28:41 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3180.706886 secs 2024-03-28T01:28:43.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:28:43 smithi165 ceph-mon[36302]: pgmap v1963: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:28:43.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:28:43 smithi119 ceph-mon[31518]: pgmap v1963: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:28:43.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:28:43 smithi132 ceph-mon[35941]: pgmap v1963: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:28:45.498 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:28:45 smithi132 ceph-mon[35941]: pgmap v1964: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-28T01:28:45.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:28:45 smithi165 ceph-mon[36302]: pgmap v1964: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-28T01:28:45.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:28:45 smithi119 ceph-mon[31518]: pgmap v1964: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-28T01:28:46.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:28:46 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3185.706949 secs 2024-03-28T01:28:46.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:28:46 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3185.706949 secs 2024-03-28T01:28:46.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:28:46 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3185.706949 secs 2024-03-28T01:28:47.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:28:47 smithi165 ceph-mon[36302]: pgmap v1965: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-28T01:28:47.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:28:47 smithi119 ceph-mon[31518]: pgmap v1965: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-28T01:28:47.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:28:47 smithi132 ceph-mon[35941]: pgmap v1965: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-28T01:28:49.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:28:49 smithi165 ceph-mon[36302]: pgmap v1966: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:28:49.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:28:49 smithi119 ceph-mon[31518]: pgmap v1966: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:28:49.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:28:49 smithi132 ceph-mon[35941]: pgmap v1966: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:28:51.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:28:51 smithi165 ceph-mon[36302]: pgmap v1967: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:28:51.650 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:28:51 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3190.707009 secs 2024-03-28T01:28:51.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:28:51 smithi119 ceph-mon[31518]: pgmap v1967: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:28:51.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:28:51 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3190.707009 secs 2024-03-28T01:28:51.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:28:51 smithi132 ceph-mon[35941]: pgmap v1967: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:28:51.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:28:51 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3190.707009 secs 2024-03-28T01:28:53.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:28:53 smithi165 ceph-mon[36302]: pgmap v1968: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:28:53.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:28:53 smithi119 ceph-mon[31518]: pgmap v1968: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:28:53.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:28:53 smithi132 ceph-mon[35941]: pgmap v1968: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:28:55.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:28:55 smithi165 ceph-mon[36302]: pgmap v1969: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:28:55.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:28:55 smithi119 ceph-mon[31518]: pgmap v1969: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:28:55.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:28:55 smithi132 ceph-mon[35941]: pgmap v1969: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:28:56.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:28:56 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3195.707068 secs 2024-03-28T01:28:56.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:28:56 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3195.707068 secs 2024-03-28T01:28:56.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:28:56 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3195.707068 secs 2024-03-28T01:28:57.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:28:57 smithi165 ceph-mon[36302]: pgmap v1970: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-28T01:28:57.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:28:57 smithi119 ceph-mon[31518]: pgmap v1970: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-28T01:28:57.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:28:57 smithi132 ceph-mon[35941]: pgmap v1970: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-28T01:28:59.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:28:59 smithi165 ceph-mon[36302]: pgmap v1971: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:28:59.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:28:59 smithi119 ceph-mon[31518]: pgmap v1971: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:28:59.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:28:59 smithi132 ceph-mon[35941]: pgmap v1971: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:29:01.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:29:01 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3200.707158 secs 2024-03-28T01:29:01.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:29:01 smithi165 ceph-mon[36302]: pgmap v1972: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:29:01.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:29:01 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3200.707158 secs 2024-03-28T01:29:01.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:29:01 smithi119 ceph-mon[31518]: pgmap v1972: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:29:01.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:29:01 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3200.707158 secs 2024-03-28T01:29:01.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:29:01 smithi132 ceph-mon[35941]: pgmap v1972: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:29:03.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:29:03 smithi165 ceph-mon[36302]: pgmap v1973: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:29:03.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:29:03 smithi119 ceph-mon[31518]: pgmap v1973: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:29:03.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:29:03 smithi132 ceph-mon[35941]: pgmap v1973: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:29:05.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:29:05 smithi165 ceph-mon[36302]: pgmap v1974: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:29:05.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:29:05 smithi119 ceph-mon[31518]: pgmap v1974: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:29:05.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:29:05 smithi132 ceph-mon[35941]: pgmap v1974: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:29:06.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:29:06 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3205.707219 secs 2024-03-28T01:29:06.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:29:06 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3205.707219 secs 2024-03-28T01:29:06.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:29:06 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3205.707219 secs 2024-03-28T01:29:07.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:29:07 smithi165 ceph-mon[36302]: pgmap v1975: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:29:07.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:29:07 smithi119 ceph-mon[31518]: pgmap v1975: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:29:07.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:29:07 smithi132 ceph-mon[35941]: pgmap v1975: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:29:09.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:29:09 smithi165 ceph-mon[36302]: pgmap v1976: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:29:09.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:29:09 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:29:09.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:29:09 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:29:09.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:29:09 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:29:09.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:29:09 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:29:09.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:29:09 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:29:09.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:29:09 smithi119 ceph-mon[31518]: pgmap v1976: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:29:09.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:29:09 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:29:09.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:29:09 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:29:09.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:29:09 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:29:09.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:29:09 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:29:09.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:29:09 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:29:09.688 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:29:09 smithi132 ceph-mon[35941]: pgmap v1976: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:29:09.688 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:29:09 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:29:09.688 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:29:09 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:29:09.689 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:29:09 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:29:09.689 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:29:09 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:29:09.689 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:29:09 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:29:11.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:29:11 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3210.707282 secs 2024-03-28T01:29:11.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:29:11 smithi165 ceph-mon[36302]: pgmap v1977: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:29:11.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:29:11 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:29:11.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:29:11 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:29:11.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:29:11 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:29:11.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:29:11 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:29:11.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:29:11 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:29:11.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:29:11 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:29:11.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:29:11 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:29:11.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:29:11 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:29:11.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:29:11 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:29:11.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:29:11 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:29:11.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:29:11 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:29:11.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:29:11 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:29:11.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:29:11 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:29:11.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:29:11 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:29:11.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:29:11 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:29:11.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:29:11 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:29:11.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:29:11 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:29:11.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:29:11 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:29:11.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:29:11 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:29:11.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:29:11 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:29:11.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:29:11 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3210.707282 secs 2024-03-28T01:29:11.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:29:11 smithi119 ceph-mon[31518]: pgmap v1977: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:29:11.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:29:11 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:29:11.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:29:11 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:29:11.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:29:11 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:29:11.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:29:11 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:29:11.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:29:11 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:29:11.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:29:11 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:29:11.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:29:11 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:29:11.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:29:11 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:29:11.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:29:11 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:29:11.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:29:11 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:29:11.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:29:11 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:29:11.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:29:11 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:29:11.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:29:11 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:29:11.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:29:11 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:29:11.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:29:11 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:29:11.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:29:11 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:29:11.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:29:11 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:29:11.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:29:11 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:29:11.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:29:11 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:29:11.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:29:11 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:29:11.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:29:11 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3210.707282 secs 2024-03-28T01:29:11.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:29:11 smithi132 ceph-mon[35941]: pgmap v1977: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:29:11.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:29:11 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:29:11.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:29:11 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:29:11.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:29:11 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:29:11.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:29:11 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:29:11.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:29:11 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:29:11.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:29:11 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:29:11.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:29:11 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:29:11.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:29:11 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:29:11.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:29:11 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:29:11.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:29:11 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:29:11.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:29:11 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:29:11.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:29:11 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:29:11.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:29:11 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:29:11.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:29:11 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:29:11.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:29:11 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:29:11.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:29:11 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:29:11.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:29:11 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:29:11.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:29:11 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:29:11.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:29:11 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:29:11.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:29:11 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:29:12.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:29:12 smithi165 ceph-mon[36302]: Adjusting osd_memory_target on smithi165 to 174.1M 2024-03-28T01:29:12.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:29:12 smithi165 ceph-mon[36302]: Unable to set osd_memory_target on smithi165 to 182635622: error parsing value: Value '182635622' is below minimum 939524096 2024-03-28T01:29:12.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:29:12 smithi165 ceph-mon[36302]: Adjusting osd_memory_target on smithi132 to 174.1M 2024-03-28T01:29:12.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:29:12 smithi165 ceph-mon[36302]: Unable to set osd_memory_target on smithi132 to 182640640: error parsing value: Value '182640640' is below minimum 939524096 2024-03-28T01:29:12.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:29:12 smithi119 ceph-mon[31518]: Adjusting osd_memory_target on smithi165 to 174.1M 2024-03-28T01:29:12.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:29:12 smithi119 ceph-mon[31518]: Unable to set osd_memory_target on smithi165 to 182635622: error parsing value: Value '182635622' is below minimum 939524096 2024-03-28T01:29:12.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:29:12 smithi119 ceph-mon[31518]: Adjusting osd_memory_target on smithi132 to 174.1M 2024-03-28T01:29:12.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:29:12 smithi119 ceph-mon[31518]: Unable to set osd_memory_target on smithi132 to 182640640: error parsing value: Value '182640640' is below minimum 939524096 2024-03-28T01:29:12.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:29:12 smithi132 ceph-mon[35941]: Adjusting osd_memory_target on smithi165 to 174.1M 2024-03-28T01:29:12.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:29:12 smithi132 ceph-mon[35941]: Unable to set osd_memory_target on smithi165 to 182635622: error parsing value: Value '182635622' is below minimum 939524096 2024-03-28T01:29:12.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:29:12 smithi132 ceph-mon[35941]: Adjusting osd_memory_target on smithi132 to 174.1M 2024-03-28T01:29:12.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:29:12 smithi132 ceph-mon[35941]: Unable to set osd_memory_target on smithi132 to 182640640: error parsing value: Value '182640640' is below minimum 939524096 2024-03-28T01:29:13.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:29:13 smithi165 ceph-mon[36302]: pgmap v1978: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:29:13.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:29:13 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:29:13.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:29:13 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:29:13.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:29:13 smithi119 ceph-mon[31518]: pgmap v1978: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:29:13.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:29:13 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:29:13.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:29:13 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:29:13.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:29:13 smithi132 ceph-mon[35941]: pgmap v1978: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:29:13.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:29:13 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:29:13.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:29:13 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:29:14.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:29:14 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:29:14.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:29:14 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:29:14.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:29:14 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:29:14.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:29:14 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:29:14.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:29:14 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:29:14.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:29:14 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:29:14.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:29:14 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:29:14.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:29:14 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:29:14.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:29:14 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:29:14.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:29:14 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:29:14.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:29:14 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:29:14.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:29:14 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:29:14.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:29:14 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:29:14.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:29:14 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:29:14.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:29:14 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:29:14.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:29:14 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:29:14.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:29:14 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:29:14.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:29:14 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:29:14.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:29:14 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:29:14.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:29:14 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:29:14.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:29:14 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:29:14.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:29:14 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:29:14.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:29:14 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:29:14.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:29:14 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:29:14.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:29:14 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:29:14.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:29:14 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:29:14.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:29:14 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:29:14.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:29:14 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:29:14.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:29:14 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:29:14.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:29:14 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:29:14.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:29:14 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:29:14.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:29:14 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:29:14.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:29:14 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:29:15.598 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:29:15 smithi132 ceph-mon[35941]: Adjusting osd_memory_target on smithi119 to 176.7M 2024-03-28T01:29:15.598 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:29:15 smithi132 ceph-mon[35941]: Unable to set osd_memory_target on smithi119 to 185372364: error parsing value: Value '185372364' is below minimum 939524096 2024-03-28T01:29:15.598 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:29:15 smithi132 ceph-mon[35941]: pgmap v1979: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T01:29:15.598 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:29:15 smithi132 ceph-mon[35941]: pgmap v1980: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 3 op/s 2024-03-28T01:29:15.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:29:15 smithi165 ceph-mon[36302]: Adjusting osd_memory_target on smithi119 to 176.7M 2024-03-28T01:29:15.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:29:15 smithi165 ceph-mon[36302]: Unable to set osd_memory_target on smithi119 to 185372364: error parsing value: Value '185372364' is below minimum 939524096 2024-03-28T01:29:15.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:29:15 smithi165 ceph-mon[36302]: pgmap v1979: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T01:29:15.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:29:15 smithi165 ceph-mon[36302]: pgmap v1980: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 3 op/s 2024-03-28T01:29:15.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:29:15 smithi119 ceph-mon[31518]: Adjusting osd_memory_target on smithi119 to 176.7M 2024-03-28T01:29:15.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:29:15 smithi119 ceph-mon[31518]: Unable to set osd_memory_target on smithi119 to 185372364: error parsing value: Value '185372364' is below minimum 939524096 2024-03-28T01:29:15.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:29:15 smithi119 ceph-mon[31518]: pgmap v1979: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T01:29:15.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:29:15 smithi119 ceph-mon[31518]: pgmap v1980: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 3 op/s 2024-03-28T01:29:16.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:29:16 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3215.707342 secs 2024-03-28T01:29:16.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:29:16 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3215.707342 secs 2024-03-28T01:29:16.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:29:16 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3215.707342 secs 2024-03-28T01:29:17.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:29:17 smithi165 ceph-mon[36302]: pgmap v1981: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T01:29:17.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:29:17 smithi119 ceph-mon[31518]: pgmap v1981: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T01:29:17.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:29:17 smithi132 ceph-mon[35941]: pgmap v1981: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T01:29:19.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:29:19 smithi165 ceph-mon[36302]: pgmap v1982: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T01:29:19.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:29:19 smithi119 ceph-mon[31518]: pgmap v1982: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T01:29:19.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:29:19 smithi132 ceph-mon[35941]: pgmap v1982: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T01:29:21.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:29:21 smithi165 ceph-mon[36302]: pgmap v1983: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T01:29:21.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:29:21 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3220.707401 secs 2024-03-28T01:29:21.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:29:21 smithi119 ceph-mon[31518]: pgmap v1983: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T01:29:21.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:29:21 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3220.707401 secs 2024-03-28T01:29:21.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:29:21 smithi132 ceph-mon[35941]: pgmap v1983: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T01:29:21.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:29:21 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3220.707401 secs 2024-03-28T01:29:23.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:29:23 smithi165 ceph-mon[36302]: pgmap v1984: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T01:29:23.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:29:23 smithi119 ceph-mon[31518]: pgmap v1984: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T01:29:23.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:29:23 smithi132 ceph-mon[35941]: pgmap v1984: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T01:29:25.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:29:25 smithi165 ceph-mon[36302]: pgmap v1985: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T01:29:25.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:29:25 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3225.707464 secs 2024-03-28T01:29:25.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:29:25 smithi119 ceph-mon[31518]: pgmap v1985: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T01:29:25.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:29:25 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3225.707464 secs 2024-03-28T01:29:25.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:29:25 smithi132 ceph-mon[35941]: pgmap v1985: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T01:29:25.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:29:25 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3225.707464 secs 2024-03-28T01:29:27.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:29:27 smithi165 ceph-mon[36302]: pgmap v1986: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:29:27.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:29:27 smithi119 ceph-mon[31518]: pgmap v1986: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:29:27.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:29:27 smithi132 ceph-mon[35941]: pgmap v1986: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:29:29.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:29:29 smithi165 ceph-mon[36302]: pgmap v1987: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:29:29.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:29:29 smithi119 ceph-mon[31518]: pgmap v1987: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:29:29.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:29:29 smithi132 ceph-mon[35941]: pgmap v1987: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:29:30.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:29:30 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3230.707525 secs 2024-03-28T01:29:30.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:29:30 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3230.707525 secs 2024-03-28T01:29:30.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:29:30 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3230.707525 secs 2024-03-28T01:29:31.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:29:31 smithi165 ceph-mon[36302]: pgmap v1988: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:29:31.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:29:31 smithi119 ceph-mon[31518]: pgmap v1988: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:29:31.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:29:31 smithi132 ceph-mon[35941]: pgmap v1988: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:29:33.126 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:29:32 smithi165 ceph-mon[36302]: pgmap v1989: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:29:33.149 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:29:32 smithi119 ceph-mon[31518]: pgmap v1989: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:29:33.196 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:29:32 smithi132 ceph-mon[35941]: pgmap v1989: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:29:35.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:29:35 smithi165 ceph-mon[36302]: pgmap v1990: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T01:29:35.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:29:35 smithi119 ceph-mon[31518]: pgmap v1990: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T01:29:35.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:29:35 smithi132 ceph-mon[35941]: pgmap v1990: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T01:29:36.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:29:36 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3235.707612 secs 2024-03-28T01:29:36.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:29:36 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3235.707612 secs 2024-03-28T01:29:36.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:29:36 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3235.707612 secs 2024-03-28T01:29:37.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:29:37 smithi165 ceph-mon[36302]: pgmap v1991: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:29:37.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:29:37 smithi119 ceph-mon[31518]: pgmap v1991: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:29:37.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:29:37 smithi132 ceph-mon[35941]: pgmap v1991: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:29:39.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:29:39 smithi165 ceph-mon[36302]: pgmap v1992: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:29:39.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:29:39 smithi119 ceph-mon[31518]: pgmap v1992: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:29:39.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:29:39 smithi132 ceph-mon[35941]: pgmap v1992: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:29:41.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:29:41 smithi165 ceph-mon[36302]: pgmap v1993: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:29:41.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:29:41 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3240.707676 secs 2024-03-28T01:29:41.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:29:41 smithi119 ceph-mon[31518]: pgmap v1993: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:29:41.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:29:41 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3240.707676 secs 2024-03-28T01:29:41.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:29:41 smithi132 ceph-mon[35941]: pgmap v1993: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:29:41.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:29:41 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3240.707676 secs 2024-03-28T01:29:43.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:29:43 smithi165 ceph-mon[36302]: pgmap v1994: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:29:43.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:29:43 smithi119 ceph-mon[31518]: pgmap v1994: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:29:43.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:29:43 smithi132 ceph-mon[35941]: pgmap v1994: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:29:45.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:29:45 smithi165 ceph-mon[36302]: pgmap v1995: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T01:29:45.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:29:45 smithi119 ceph-mon[31518]: pgmap v1995: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T01:29:45.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:29:45 smithi132 ceph-mon[35941]: pgmap v1995: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T01:29:46.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:29:46 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3245.707737 secs 2024-03-28T01:29:46.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:29:46 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3245.707737 secs 2024-03-28T01:29:46.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:29:46 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3245.707737 secs 2024-03-28T01:29:47.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:29:47 smithi165 ceph-mon[36302]: pgmap v1996: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:29:47.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:29:47 smithi119 ceph-mon[31518]: pgmap v1996: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:29:47.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:29:47 smithi132 ceph-mon[35941]: pgmap v1996: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:29:49.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:29:49 smithi119 ceph-mon[31518]: pgmap v1997: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:29:49.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:29:49 smithi132 ceph-mon[35941]: pgmap v1997: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:29:49.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:29:49 smithi165 ceph-mon[36302]: pgmap v1997: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:29:51.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:29:51 smithi119 ceph-mon[31518]: pgmap v1998: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:29:51.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:29:51 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3250.707797 secs 2024-03-28T01:29:51.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:29:51 smithi132 ceph-mon[35941]: pgmap v1998: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:29:51.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:29:51 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3250.707797 secs 2024-03-28T01:29:51.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:29:51 smithi165 ceph-mon[36302]: pgmap v1998: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:29:51.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:29:51 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3250.707797 secs 2024-03-28T01:29:53.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:29:53 smithi119 ceph-mon[31518]: pgmap v1999: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:29:53.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:29:53 smithi132 ceph-mon[35941]: pgmap v1999: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:29:53.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:29:53 smithi165 ceph-mon[36302]: pgmap v1999: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:29:55.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:29:55 smithi119 ceph-mon[31518]: pgmap v2000: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T01:29:55.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:29:55 smithi132 ceph-mon[35941]: pgmap v2000: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T01:29:55.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:29:55 smithi165 ceph-mon[36302]: pgmap v2000: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T01:29:56.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:29:56 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3255.707863 secs 2024-03-28T01:29:56.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:29:56 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3255.707863 secs 2024-03-28T01:29:56.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:29:56 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3255.707863 secs 2024-03-28T01:29:57.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:29:57 smithi132 ceph-mon[35941]: pgmap v2001: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:29:57.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:29:57 smithi165 ceph-mon[36302]: pgmap v2001: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:29:57.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:29:57 smithi119 ceph-mon[31518]: pgmap v2001: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:29:59.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:29:59 smithi119 ceph-mon[31518]: pgmap v2002: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:29:59.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:29:59 smithi132 ceph-mon[35941]: pgmap v2002: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:29:59.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:29:59 smithi165 ceph-mon[36302]: pgmap v2002: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:30:00.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:30:00 smithi132 ceph-mon[35941]: overall HEALTH_WARN 5 failed cephadm daemon(s); 1 filesystem is degraded; 1 MDSs report slow requests 2024-03-28T01:30:00.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:30:00 smithi165 ceph-mon[36302]: overall HEALTH_WARN 5 failed cephadm daemon(s); 1 filesystem is degraded; 1 MDSs report slow requests 2024-03-28T01:30:00.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:30:00 smithi119 ceph-mon[31518]: overall HEALTH_WARN 5 failed cephadm daemon(s); 1 filesystem is degraded; 1 MDSs report slow requests 2024-03-28T01:30:01.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:30:01 smithi132 ceph-mon[35941]: pgmap v2003: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:30:01.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:30:01 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3260.707920 secs 2024-03-28T01:30:01.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:30:01 smithi165 ceph-mon[36302]: pgmap v2003: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:30:01.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:30:01 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3260.707920 secs 2024-03-28T01:30:01.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:30:01 smithi119 ceph-mon[31518]: pgmap v2003: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:30:01.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:30:01 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3260.707920 secs 2024-03-28T01:30:03.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:30:03 smithi132 ceph-mon[35941]: pgmap v2004: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:30:03.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:30:03 smithi165 ceph-mon[36302]: pgmap v2004: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:30:03.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:30:03 smithi119 ceph-mon[31518]: pgmap v2004: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:30:05.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:30:05 smithi132 ceph-mon[35941]: pgmap v2005: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T01:30:05.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:30:05 smithi165 ceph-mon[36302]: pgmap v2005: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T01:30:05.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:30:05 smithi119 ceph-mon[31518]: pgmap v2005: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T01:30:06.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:30:06 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3265.707981 secs 2024-03-28T01:30:06.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:30:06 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3265.707981 secs 2024-03-28T01:30:06.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:30:06 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3265.707981 secs 2024-03-28T01:30:07.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:30:07 smithi132 ceph-mon[35941]: pgmap v2006: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-28T01:30:07.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:30:07 smithi165 ceph-mon[36302]: pgmap v2006: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-28T01:30:07.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:30:07 smithi119 ceph-mon[31518]: pgmap v2006: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-28T01:30:09.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:30:09 smithi132 ceph-mon[35941]: pgmap v2007: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:30:09.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:30:09 smithi165 ceph-mon[36302]: pgmap v2007: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:30:09.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:30:09 smithi119 ceph-mon[31518]: pgmap v2007: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:30:11.552 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:30:11 smithi119 ceph-mon[31518]: pgmap v2008: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:30:11.552 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:30:11 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3270.708067 secs 2024-03-28T01:30:11.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:30:11 smithi165 ceph-mon[36302]: pgmap v2008: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:30:11.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:30:11 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3270.708067 secs 2024-03-28T01:30:11.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:30:11 smithi132 ceph-mon[35941]: pgmap v2008: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:30:11.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:30:11 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3270.708067 secs 2024-03-28T01:30:13.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:30:13 smithi165 ceph-mon[36302]: pgmap v2009: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:30:13.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:30:13 smithi119 ceph-mon[31518]: pgmap v2009: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:30:13.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:30:13 smithi132 ceph-mon[35941]: pgmap v2009: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:30:15.498 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:30:15 smithi132 ceph-mon[35941]: pgmap v2010: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 6 op/s 2024-03-28T01:30:15.499 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:30:15 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:30:15.499 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:30:15 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:30:15.499 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:30:15 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:30:15.597 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:30:15 smithi165 ceph-mon[36302]: pgmap v2010: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 6 op/s 2024-03-28T01:30:15.597 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:30:15 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:30:15.597 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:30:15 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:30:15.598 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:30:15 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:30:15.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:30:15 smithi119 ceph-mon[31518]: pgmap v2010: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 6 op/s 2024-03-28T01:30:15.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:30:15 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:30:15.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:30:15 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:30:15.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:30:15 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:30:16.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:30:16 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3275.708132 secs 2024-03-28T01:30:16.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:30:16 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:30:16.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:30:16 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:30:16.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:30:16 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:30:16.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:30:16 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3275.708132 secs 2024-03-28T01:30:16.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:30:16 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:30:16.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:30:16 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:30:16.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:30:16 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:30:16.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:30:16 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3275.708132 secs 2024-03-28T01:30:16.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:30:16 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:30:16.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:30:16 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:30:16.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:30:16 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:30:17.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:30:17 smithi165 ceph-mon[36302]: pgmap v2011: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 6 op/s 2024-03-28T01:30:17.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:30:17 smithi119 ceph-mon[31518]: pgmap v2011: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 6 op/s 2024-03-28T01:30:17.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:30:17 smithi132 ceph-mon[35941]: pgmap v2011: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 6 op/s 2024-03-28T01:30:19.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:30:19 smithi165 ceph-mon[36302]: pgmap v2012: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T01:30:19.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:30:19 smithi119 ceph-mon[31518]: pgmap v2012: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T01:30:19.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:30:19 smithi132 ceph-mon[35941]: pgmap v2012: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T01:30:21.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:30:21 smithi165 ceph-mon[36302]: pgmap v2013: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T01:30:21.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:30:21 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3280.708143 secs 2024-03-28T01:30:21.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:30:21 smithi119 ceph-mon[31518]: pgmap v2013: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T01:30:21.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:30:21 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3280.708143 secs 2024-03-28T01:30:21.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:30:21 smithi132 ceph-mon[35941]: pgmap v2013: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T01:30:21.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:30:21 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3280.708143 secs 2024-03-28T01:30:23.574 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:30:23 smithi119 ceph-mon[31518]: pgmap v2014: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T01:30:23.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:30:23 smithi165 ceph-mon[36302]: pgmap v2014: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T01:30:23.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:30:23 smithi132 ceph-mon[35941]: pgmap v2014: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T01:30:25.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:30:25 smithi165 ceph-mon[36302]: pgmap v2015: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T01:30:25.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:30:25 smithi119 ceph-mon[31518]: pgmap v2015: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T01:30:25.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:30:25 smithi132 ceph-mon[35941]: pgmap v2015: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T01:30:26.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:30:26 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3285.708231 secs 2024-03-28T01:30:26.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:30:26 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3285.708231 secs 2024-03-28T01:30:26.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:30:26 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3285.708231 secs 2024-03-28T01:30:27.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:30:27 smithi165 ceph-mon[36302]: pgmap v2016: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 6 op/s 2024-03-28T01:30:27.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:30:27 smithi119 ceph-mon[31518]: pgmap v2016: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 6 op/s 2024-03-28T01:30:27.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:30:27 smithi132 ceph-mon[35941]: pgmap v2016: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 6 op/s 2024-03-28T01:30:29.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:30:29 smithi165 ceph-mon[36302]: pgmap v2017: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:30:29.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:30:29 smithi119 ceph-mon[31518]: pgmap v2017: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:30:29.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:30:29 smithi132 ceph-mon[35941]: pgmap v2017: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:30:31.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:30:31 smithi165 ceph-mon[36302]: pgmap v2018: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:30:31.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:30:31 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3290.708274 secs 2024-03-28T01:30:31.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:30:31 smithi119 ceph-mon[31518]: pgmap v2018: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:30:31.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:30:31 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3290.708274 secs 2024-03-28T01:30:31.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:30:31 smithi132 ceph-mon[35941]: pgmap v2018: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:30:31.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:30:31 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3290.708274 secs 2024-03-28T01:30:33.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:30:33 smithi165 ceph-mon[36302]: pgmap v2019: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:30:33.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:30:33 smithi119 ceph-mon[31518]: pgmap v2019: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:30:33.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:30:33 smithi132 ceph-mon[35941]: pgmap v2019: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:30:35.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:30:35 smithi165 ceph-mon[36302]: pgmap v2020: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:30:35.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:30:35 smithi119 ceph-mon[31518]: pgmap v2020: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:30:35.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:30:35 smithi132 ceph-mon[35941]: pgmap v2020: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:30:36.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:30:36 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3295.708314 secs 2024-03-28T01:30:36.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:30:36 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3295.708314 secs 2024-03-28T01:30:36.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:30:36 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3295.708314 secs 2024-03-28T01:30:37.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:30:37 smithi165 ceph-mon[36302]: pgmap v2021: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:30:37.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:30:37 smithi119 ceph-mon[31518]: pgmap v2021: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:30:37.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:30:37 smithi132 ceph-mon[35941]: pgmap v2021: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:30:39.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:30:39 smithi165 ceph-mon[36302]: pgmap v2022: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:30:39.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:30:39 smithi119 ceph-mon[31518]: pgmap v2022: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:30:39.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:30:39 smithi132 ceph-mon[35941]: pgmap v2022: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:30:41.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:30:41 smithi165 ceph-mon[36302]: pgmap v2023: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:30:41.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:30:41 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3300.708353 secs 2024-03-28T01:30:41.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:30:41 smithi119 ceph-mon[31518]: pgmap v2023: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:30:41.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:30:41 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3300.708353 secs 2024-03-28T01:30:41.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:30:41 smithi132 ceph-mon[35941]: pgmap v2023: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:30:41.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:30:41 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3300.708353 secs 2024-03-28T01:30:43.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:30:43 smithi165 ceph-mon[36302]: pgmap v2024: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:30:43.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:30:43 smithi119 ceph-mon[31518]: pgmap v2024: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:30:43.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:30:43 smithi132 ceph-mon[35941]: pgmap v2024: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:30:45.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:30:45 smithi165 ceph-mon[36302]: pgmap v2025: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:30:45.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:30:45 smithi119 ceph-mon[31518]: pgmap v2025: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:30:45.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:30:45 smithi132 ceph-mon[35941]: pgmap v2025: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:30:46.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:30:46 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3305.708439 secs 2024-03-28T01:30:46.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:30:46 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3305.708439 secs 2024-03-28T01:30:46.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:30:46 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3305.708439 secs 2024-03-28T01:30:47.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:30:47 smithi165 ceph-mon[36302]: pgmap v2026: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:30:47.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:30:47 smithi119 ceph-mon[31518]: pgmap v2026: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:30:47.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:30:47 smithi132 ceph-mon[35941]: pgmap v2026: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:30:49.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:30:49 smithi165 ceph-mon[36302]: pgmap v2027: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:30:49.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:30:49 smithi119 ceph-mon[31518]: pgmap v2027: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:30:49.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:30:49 smithi132 ceph-mon[35941]: pgmap v2027: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:30:51.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:30:51 smithi165 ceph-mon[36302]: pgmap v2028: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:30:51.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:30:51 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3310.708527 secs 2024-03-28T01:30:51.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:30:51 smithi119 ceph-mon[31518]: pgmap v2028: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:30:51.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:30:51 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3310.708527 secs 2024-03-28T01:30:51.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:30:51 smithi132 ceph-mon[35941]: pgmap v2028: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:30:51.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:30:51 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3310.708527 secs 2024-03-28T01:30:53.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:30:53 smithi165 ceph-mon[36302]: pgmap v2029: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:30:53.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:30:53 smithi119 ceph-mon[31518]: pgmap v2029: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:30:53.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:30:53 smithi132 ceph-mon[35941]: pgmap v2029: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:30:55.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:30:55 smithi165 ceph-mon[36302]: pgmap v2030: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:30:55.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:30:55 smithi119 ceph-mon[31518]: pgmap v2030: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:30:55.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:30:55 smithi132 ceph-mon[35941]: pgmap v2030: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:30:56.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:30:56 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3315.708589 secs 2024-03-28T01:30:56.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:30:56 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3315.708589 secs 2024-03-28T01:30:56.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:30:56 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3315.708589 secs 2024-03-28T01:30:57.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:30:57 smithi165 ceph-mon[36302]: pgmap v2031: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:30:57.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:30:57 smithi119 ceph-mon[31518]: pgmap v2031: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:30:57.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:30:57 smithi132 ceph-mon[35941]: pgmap v2031: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:30:59.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:30:59 smithi165 ceph-mon[36302]: pgmap v2032: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:30:59.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:30:59 smithi119 ceph-mon[31518]: pgmap v2032: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:30:59.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:30:59 smithi132 ceph-mon[35941]: pgmap v2032: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:31:00.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:31:00 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3320.708651 secs 2024-03-28T01:31:00.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:31:00 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3320.708651 secs 2024-03-28T01:31:00.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:31:00 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3320.708651 secs 2024-03-28T01:31:01.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:31:01 smithi165 ceph-mon[36302]: pgmap v2033: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:31:01.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:31:01 smithi119 ceph-mon[31518]: pgmap v2033: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:31:01.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:31:01 smithi132 ceph-mon[35941]: pgmap v2033: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:31:03.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:31:03 smithi165 ceph-mon[36302]: pgmap v2034: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:31:03.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:31:03 smithi119 ceph-mon[31518]: pgmap v2034: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:31:03.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:31:03 smithi132 ceph-mon[35941]: pgmap v2034: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:31:05.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:31:05 smithi119 ceph-mon[31518]: pgmap v2035: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:31:05.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:31:05 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3325.708738 secs 2024-03-28T01:31:05.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:31:05 smithi132 ceph-mon[35941]: pgmap v2035: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:31:05.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:31:05 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3325.708738 secs 2024-03-28T01:31:05.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:31:05 smithi165 ceph-mon[36302]: pgmap v2035: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:31:05.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:31:05 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3325.708738 secs 2024-03-28T01:31:07.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:31:07 smithi119 ceph-mon[31518]: pgmap v2036: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:31:07.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:31:07 smithi132 ceph-mon[35941]: pgmap v2036: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:31:07.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:31:07 smithi165 ceph-mon[36302]: pgmap v2036: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:31:09.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:31:09 smithi119 ceph-mon[31518]: pgmap v2037: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:31:09.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:31:09 smithi132 ceph-mon[35941]: pgmap v2037: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:31:09.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:31:09 smithi165 ceph-mon[36302]: pgmap v2037: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:31:10.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:31:10 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3330.708827 secs 2024-03-28T01:31:10.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:31:10 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3330.708827 secs 2024-03-28T01:31:10.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:31:10 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3330.708827 secs 2024-03-28T01:31:11.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:31:11 smithi119 ceph-mon[31518]: pgmap v2038: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:31:11.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:31:11 smithi132 ceph-mon[35941]: pgmap v2038: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:31:11.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:31:11 smithi165 ceph-mon[36302]: pgmap v2038: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:31:13.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:31:13 smithi132 ceph-mon[35941]: pgmap v2039: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:31:13.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:31:13 smithi165 ceph-mon[36302]: pgmap v2039: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:31:13.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:31:13 smithi119 ceph-mon[31518]: pgmap v2039: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:31:15.678 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:31:15 smithi165 ceph-mon[36302]: pgmap v2040: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:31:15.678 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:31:15 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3335.708892 secs 2024-03-28T01:31:15.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:31:15 smithi132 ceph-mon[35941]: pgmap v2040: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:31:15.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:31:15 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3335.708892 secs 2024-03-28T01:31:15.872 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:31:15 smithi119 ceph-mon[31518]: pgmap v2040: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:31:15.872 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:31:15 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3335.708892 secs 2024-03-28T01:31:16.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:31:16 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:31:16.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:31:16 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:31:16.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:31:16 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:31:17.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:31:17 smithi132 ceph-mon[35941]: pgmap v2041: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:31:17.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:31:17 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:31:17.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:31:17 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:31:17.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:31:17 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:31:17.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:31:17 smithi165 ceph-mon[36302]: pgmap v2041: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:31:17.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:31:17 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:31:17.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:31:17 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:31:17.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:31:17 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:31:17.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:31:17 smithi119 ceph-mon[31518]: pgmap v2041: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:31:17.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:31:17 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:31:17.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:31:17 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:31:17.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:31:17 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:31:18.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:31:18 smithi132 ceph-mon[35941]: pgmap v2042: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T01:31:18.877 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:31:18 smithi165 ceph-mon[36302]: pgmap v2042: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T01:31:18.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:31:18 smithi119 ceph-mon[31518]: pgmap v2042: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T01:31:20.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:31:20 smithi132 ceph-mon[35941]: pgmap v2043: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T01:31:20.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:31:20 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3340.708956 secs 2024-03-28T01:31:20.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:31:20 smithi165 ceph-mon[36302]: pgmap v2043: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T01:31:20.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:31:20 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3340.708956 secs 2024-03-28T01:31:20.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:31:20 smithi119 ceph-mon[31518]: pgmap v2043: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T01:31:20.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:31:20 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3340.708956 secs 2024-03-28T01:31:22.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:31:22 smithi132 ceph-mon[35941]: pgmap v2044: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T01:31:22.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:31:22 smithi165 ceph-mon[36302]: pgmap v2044: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T01:31:22.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:31:22 smithi119 ceph-mon[31518]: pgmap v2044: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T01:31:24.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:31:24 smithi132 ceph-mon[35941]: pgmap v2045: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T01:31:24.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:31:24 smithi165 ceph-mon[36302]: pgmap v2045: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T01:31:24.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:31:24 smithi119 ceph-mon[31518]: pgmap v2045: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T01:31:25.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:31:25 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3345.709043 secs 2024-03-28T01:31:25.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:31:25 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3345.709043 secs 2024-03-28T01:31:25.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:31:25 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3345.709043 secs 2024-03-28T01:31:26.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:31:26 smithi165 ceph-mon[36302]: pgmap v2046: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T01:31:26.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:31:26 smithi119 ceph-mon[31518]: pgmap v2046: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T01:31:26.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:31:26 smithi132 ceph-mon[35941]: pgmap v2046: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T01:31:28.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:31:28 smithi165 ceph-mon[36302]: pgmap v2047: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T01:31:28.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:31:28 smithi119 ceph-mon[31518]: pgmap v2047: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T01:31:28.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:31:28 smithi132 ceph-mon[35941]: pgmap v2047: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T01:31:29.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:31:29 smithi165 ceph-mon[36302]: pgmap v2048: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:31:29.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:31:29 smithi119 ceph-mon[31518]: pgmap v2048: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:31:29.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:31:29 smithi132 ceph-mon[35941]: pgmap v2048: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:31:30.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:31:30 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3350.709105 secs 2024-03-28T01:31:30.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:31:30 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3350.709105 secs 2024-03-28T01:31:30.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:31:30 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3350.709105 secs 2024-03-28T01:31:31.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:31:31 smithi165 ceph-mon[36302]: pgmap v2049: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:31:31.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:31:31 smithi119 ceph-mon[31518]: pgmap v2049: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:31:31.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:31:31 smithi132 ceph-mon[35941]: pgmap v2049: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:31:34.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:31:34 smithi165 ceph-mon[36302]: pgmap v2050: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:31:34.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:31:34 smithi119 ceph-mon[31518]: pgmap v2050: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:31:34.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:31:34 smithi132 ceph-mon[35941]: pgmap v2050: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:31:36.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:31:36 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3355.709167 secs 2024-03-28T01:31:36.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:31:36 smithi165 ceph-mon[36302]: pgmap v2051: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:31:36.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:31:36 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3355.709167 secs 2024-03-28T01:31:36.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:31:36 smithi119 ceph-mon[31518]: pgmap v2051: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:31:36.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:31:36 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3355.709167 secs 2024-03-28T01:31:36.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:31:36 smithi132 ceph-mon[35941]: pgmap v2051: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:31:38.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:31:38 smithi165 ceph-mon[36302]: pgmap v2052: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:31:38.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:31:38 smithi119 ceph-mon[31518]: pgmap v2052: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:31:38.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:31:38 smithi132 ceph-mon[35941]: pgmap v2052: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:31:40.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:31:40 smithi165 ceph-mon[36302]: pgmap v2053: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:31:40.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:31:40 smithi119 ceph-mon[31518]: pgmap v2053: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:31:40.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:31:40 smithi132 ceph-mon[35941]: pgmap v2053: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:31:41.550 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:31:41 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3360.709226 secs 2024-03-28T01:31:41.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:31:41 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3360.709226 secs 2024-03-28T01:31:41.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:31:41 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3360.709226 secs 2024-03-28T01:31:42.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:31:42 smithi165 ceph-mon[36302]: pgmap v2054: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:31:42.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:31:42 smithi119 ceph-mon[31518]: pgmap v2054: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:31:42.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:31:42 smithi132 ceph-mon[35941]: pgmap v2054: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:31:44.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:31:44 smithi165 ceph-mon[36302]: pgmap v2055: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:31:44.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:31:44 smithi119 ceph-mon[31518]: pgmap v2055: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:31:44.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:31:44 smithi132 ceph-mon[35941]: pgmap v2055: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:31:46.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:31:46 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3365.709313 secs 2024-03-28T01:31:46.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:31:46 smithi165 ceph-mon[36302]: pgmap v2056: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:31:46.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:31:46 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3365.709313 secs 2024-03-28T01:31:46.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:31:46 smithi119 ceph-mon[31518]: pgmap v2056: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:31:46.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:31:46 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3365.709313 secs 2024-03-28T01:31:46.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:31:46 smithi132 ceph-mon[35941]: pgmap v2056: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:31:48.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:31:48 smithi165 ceph-mon[36302]: pgmap v2057: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:31:48.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:31:48 smithi119 ceph-mon[31518]: pgmap v2057: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:31:48.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:31:48 smithi132 ceph-mon[35941]: pgmap v2057: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:31:50.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:31:50 smithi165 ceph-mon[36302]: pgmap v2058: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:31:50.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:31:50 smithi119 ceph-mon[31518]: pgmap v2058: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:31:50.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:31:50 smithi132 ceph-mon[35941]: pgmap v2058: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:31:51.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:31:51 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3370.709375 secs 2024-03-28T01:31:51.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:31:51 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3370.709375 secs 2024-03-28T01:31:51.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:31:51 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3370.709375 secs 2024-03-28T01:31:52.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:31:52 smithi165 ceph-mon[36302]: pgmap v2059: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:31:52.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:31:52 smithi119 ceph-mon[31518]: pgmap v2059: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:31:52.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:31:52 smithi132 ceph-mon[35941]: pgmap v2059: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:31:54.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:31:54 smithi165 ceph-mon[36302]: pgmap v2060: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:31:54.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:31:54 smithi119 ceph-mon[31518]: pgmap v2060: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:31:54.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:31:54 smithi132 ceph-mon[35941]: pgmap v2060: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:31:56.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:31:56 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3375.709435 secs 2024-03-28T01:31:56.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:31:56 smithi165 ceph-mon[36302]: pgmap v2061: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:31:56.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:31:56 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3375.709435 secs 2024-03-28T01:31:56.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:31:56 smithi119 ceph-mon[31518]: pgmap v2061: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:31:56.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:31:56 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3375.709435 secs 2024-03-28T01:31:56.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:31:56 smithi132 ceph-mon[35941]: pgmap v2061: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:31:58.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:31:58 smithi165 ceph-mon[36302]: pgmap v2062: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:31:58.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:31:58 smithi119 ceph-mon[31518]: pgmap v2062: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:31:58.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:31:58 smithi132 ceph-mon[35941]: pgmap v2062: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:32:00.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:32:00 smithi165 ceph-mon[36302]: pgmap v2063: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:32:00.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:32:00 smithi119 ceph-mon[31518]: pgmap v2063: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:32:00.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:32:00 smithi132 ceph-mon[35941]: pgmap v2063: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:32:01.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:32:01 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3380.709496 secs 2024-03-28T01:32:01.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:32:01 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3380.709496 secs 2024-03-28T01:32:01.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:32:01 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3380.709496 secs 2024-03-28T01:32:02.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:32:02 smithi165 ceph-mon[36302]: pgmap v2064: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:32:02.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:32:02 smithi119 ceph-mon[31518]: pgmap v2064: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:32:02.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:32:02 smithi132 ceph-mon[35941]: pgmap v2064: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:32:04.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:32:04 smithi165 ceph-mon[36302]: pgmap v2065: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:32:04.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:32:04 smithi119 ceph-mon[31518]: pgmap v2065: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:32:04.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:32:04 smithi132 ceph-mon[35941]: pgmap v2065: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:32:06.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:32:06 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3385.709558 secs 2024-03-28T01:32:06.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:32:06 smithi165 ceph-mon[36302]: pgmap v2066: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:32:06.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:32:06 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3385.709558 secs 2024-03-28T01:32:06.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:32:06 smithi119 ceph-mon[31518]: pgmap v2066: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:32:06.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:32:06 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3385.709558 secs 2024-03-28T01:32:06.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:32:06 smithi132 ceph-mon[35941]: pgmap v2066: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:32:08.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:32:08 smithi165 ceph-mon[36302]: pgmap v2067: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:32:08.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:32:08 smithi119 ceph-mon[31518]: pgmap v2067: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:32:08.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:32:08 smithi132 ceph-mon[35941]: pgmap v2067: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:32:10.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:32:10 smithi165 ceph-mon[36302]: pgmap v2068: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:32:10.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:32:10 smithi119 ceph-mon[31518]: pgmap v2068: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:32:10.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:32:10 smithi132 ceph-mon[35941]: pgmap v2068: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:32:11.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:32:11 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3390.709647 secs 2024-03-28T01:32:11.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:32:11 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3390.709647 secs 2024-03-28T01:32:11.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:32:11 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3390.709647 secs 2024-03-28T01:32:12.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:32:12 smithi165 ceph-mon[36302]: pgmap v2069: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T01:32:12.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:32:12 smithi119 ceph-mon[31518]: pgmap v2069: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T01:32:12.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:32:12 smithi132 ceph-mon[35941]: pgmap v2069: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T01:32:14.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:32:14 smithi165 ceph-mon[36302]: pgmap v2070: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T01:32:14.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:32:14 smithi119 ceph-mon[31518]: pgmap v2070: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T01:32:14.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:32:14 smithi132 ceph-mon[35941]: pgmap v2070: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T01:32:15.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:32:15 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3395.709736 secs 2024-03-28T01:32:15.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:32:15 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3395.709736 secs 2024-03-28T01:32:15.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:32:15 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3395.709736 secs 2024-03-28T01:32:16.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:32:16 smithi165 ceph-mon[36302]: pgmap v2071: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:32:16.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:32:16 smithi119 ceph-mon[31518]: pgmap v2071: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:32:16.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:32:16 smithi132 ceph-mon[35941]: pgmap v2071: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:32:18.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:32:18 smithi165 ceph-mon[36302]: pgmap v2072: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T01:32:18.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:32:18 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:32:18.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:32:18 smithi119 ceph-mon[31518]: pgmap v2072: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T01:32:18.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:32:18 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:32:18.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:32:18 smithi132 ceph-mon[35941]: pgmap v2072: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T01:32:18.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:32:18 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:32:19.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:32:19 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:32:19.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:32:19 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:32:19.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:32:19 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:32:19.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:32:19 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:32:19.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:32:19 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:32:19.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:32:19 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:32:19.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:32:19 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:32:19.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:32:19 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:32:19.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:32:19 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:32:20.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:32:20 smithi119 ceph-mon[31518]: pgmap v2073: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T01:32:20.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:32:20 smithi119 ceph-mon[31518]: pgmap v2074: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-28T01:32:20.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:32:20 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3400.709798 secs 2024-03-28T01:32:20.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:32:20 smithi132 ceph-mon[35941]: pgmap v2073: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T01:32:20.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:32:20 smithi132 ceph-mon[35941]: pgmap v2074: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-28T01:32:20.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:32:20 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3400.709798 secs 2024-03-28T01:32:20.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:32:20 smithi165 ceph-mon[36302]: pgmap v2073: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T01:32:20.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:32:20 smithi165 ceph-mon[36302]: pgmap v2074: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-28T01:32:20.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:32:20 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3400.709798 secs 2024-03-28T01:32:22.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:32:22 smithi119 ceph-mon[31518]: pgmap v2075: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:32:22.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:32:22 smithi132 ceph-mon[35941]: pgmap v2075: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:32:22.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:32:22 smithi165 ceph-mon[36302]: pgmap v2075: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:32:24.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:32:24 smithi119 ceph-mon[31518]: pgmap v2076: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:32:24.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:32:24 smithi132 ceph-mon[35941]: pgmap v2076: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:32:24.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:32:24 smithi165 ceph-mon[36302]: pgmap v2076: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:32:25.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:32:25 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3405.709889 secs 2024-03-28T01:32:25.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:32:25 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3405.709889 secs 2024-03-28T01:32:25.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:32:25 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3405.709889 secs 2024-03-28T01:32:26.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:32:26 smithi132 ceph-mon[35941]: pgmap v2077: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:32:26.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:32:26 smithi165 ceph-mon[36302]: pgmap v2077: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:32:26.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:32:26 smithi119 ceph-mon[31518]: pgmap v2077: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:32:28.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:32:28 smithi132 ceph-mon[35941]: pgmap v2078: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:32:28.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:32:28 smithi165 ceph-mon[36302]: pgmap v2078: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:32:28.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:32:28 smithi119 ceph-mon[31518]: pgmap v2078: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:32:30.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:32:30 smithi132 ceph-mon[35941]: pgmap v2079: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T01:32:30.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:32:30 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3410.709977 secs 2024-03-28T01:32:30.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:32:30 smithi165 ceph-mon[36302]: pgmap v2079: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T01:32:30.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:32:30 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3410.709977 secs 2024-03-28T01:32:30.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:32:30 smithi119 ceph-mon[31518]: pgmap v2079: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T01:32:30.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:32:30 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3410.709977 secs 2024-03-28T01:32:32.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:32:32 smithi132 ceph-mon[35941]: pgmap v2080: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:32:32.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:32:32 smithi165 ceph-mon[36302]: pgmap v2080: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:32:32.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:32:32 smithi119 ceph-mon[31518]: pgmap v2080: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:32:34.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:32:34 smithi119 ceph-mon[31518]: pgmap v2081: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:32:34.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:32:34 smithi132 ceph-mon[35941]: pgmap v2081: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:32:34.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:32:34 smithi165 ceph-mon[36302]: pgmap v2081: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:32:35.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:32:35 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3415.710036 secs 2024-03-28T01:32:35.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:32:35 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3415.710036 secs 2024-03-28T01:32:35.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:32:35 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3415.710036 secs 2024-03-28T01:32:36.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:32:36 smithi132 ceph-mon[35941]: pgmap v2082: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:32:36.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:32:36 smithi165 ceph-mon[36302]: pgmap v2082: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:32:36.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:32:36 smithi119 ceph-mon[31518]: pgmap v2082: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:32:38.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:32:38 smithi132 ceph-mon[35941]: pgmap v2083: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:32:38.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:32:38 smithi165 ceph-mon[36302]: pgmap v2083: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:32:38.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:32:38 smithi119 ceph-mon[31518]: pgmap v2083: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:32:40.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:32:40 smithi132 ceph-mon[35941]: pgmap v2084: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:32:40.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:32:40 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3420.710096 secs 2024-03-28T01:32:40.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:32:40 smithi165 ceph-mon[36302]: pgmap v2084: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:32:40.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:32:40 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3420.710096 secs 2024-03-28T01:32:40.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:32:40 smithi119 ceph-mon[31518]: pgmap v2084: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:32:40.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:32:40 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3420.710096 secs 2024-03-28T01:32:42.702 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:32:42 smithi119 ceph-mon[31518]: pgmap v2085: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:32:42.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:32:42 smithi165 ceph-mon[36302]: pgmap v2085: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:32:42.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:32:42 smithi132 ceph-mon[35941]: pgmap v2085: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:32:44.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:32:44 smithi165 ceph-mon[36302]: pgmap v2086: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:32:44.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:32:44 smithi119 ceph-mon[31518]: pgmap v2086: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:32:44.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:32:44 smithi132 ceph-mon[35941]: pgmap v2086: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:32:45.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:32:45 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3425.710170 secs 2024-03-28T01:32:45.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:32:45 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3425.710170 secs 2024-03-28T01:32:45.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:32:45 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3425.710170 secs 2024-03-28T01:32:46.777 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:32:46 smithi165 ceph-mon[36302]: pgmap v2087: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:32:46.847 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:32:46 smithi132 ceph-mon[35941]: pgmap v2087: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:32:46.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:32:46 smithi119 ceph-mon[31518]: pgmap v2087: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:32:48.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:32:48 smithi165 ceph-mon[36302]: pgmap v2088: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:32:48.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:32:48 smithi119 ceph-mon[31518]: pgmap v2088: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:32:48.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:32:48 smithi132 ceph-mon[35941]: pgmap v2088: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:32:50.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:32:50 smithi165 ceph-mon[36302]: pgmap v2089: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:32:50.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:32:50 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3430.710259 secs 2024-03-28T01:32:50.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:32:50 smithi119 ceph-mon[31518]: pgmap v2089: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:32:50.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:32:50 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3430.710259 secs 2024-03-28T01:32:50.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:32:50 smithi132 ceph-mon[35941]: pgmap v2089: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:32:50.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:32:50 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3430.710259 secs 2024-03-28T01:32:52.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:32:52 smithi165 ceph-mon[36302]: pgmap v2090: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:32:52.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:32:52 smithi119 ceph-mon[31518]: pgmap v2090: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:32:52.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:32:52 smithi132 ceph-mon[35941]: pgmap v2090: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:32:53.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:32:53 smithi165 ceph-mon[36302]: pgmap v2091: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:32:53.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:32:53 smithi119 ceph-mon[31518]: pgmap v2091: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:32:53.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:32:53 smithi132 ceph-mon[35941]: pgmap v2091: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:32:56.126 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:32:55 smithi165 ceph-mon[36302]: pgmap v2092: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:32:56.126 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:32:55 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3435.710347 secs 2024-03-28T01:32:56.149 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:32:55 smithi119 ceph-mon[31518]: pgmap v2092: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:32:56.149 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:32:55 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3435.710347 secs 2024-03-28T01:32:56.196 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:32:55 smithi132 ceph-mon[35941]: pgmap v2092: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:32:56.196 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:32:55 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3435.710347 secs 2024-03-28T01:32:58.126 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:32:57 smithi165 ceph-mon[36302]: pgmap v2093: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:32:58.149 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:32:57 smithi119 ceph-mon[31518]: pgmap v2093: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:32:58.196 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:32:57 smithi132 ceph-mon[35941]: pgmap v2093: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:33:00.126 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:32:59 smithi165 ceph-mon[36302]: pgmap v2094: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:33:00.149 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:32:59 smithi119 ceph-mon[31518]: pgmap v2094: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:33:00.196 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:32:59 smithi132 ceph-mon[35941]: pgmap v2094: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:33:01.126 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:33:00 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3440.710435 secs 2024-03-28T01:33:01.149 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:33:00 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3440.710435 secs 2024-03-28T01:33:01.196 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:33:00 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3440.710435 secs 2024-03-28T01:33:02.126 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:33:01 smithi165 ceph-mon[36302]: pgmap v2095: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:33:02.149 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:33:01 smithi119 ceph-mon[31518]: pgmap v2095: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:33:02.196 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:33:01 smithi132 ceph-mon[35941]: pgmap v2095: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:33:04.126 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:33:03 smithi165 ceph-mon[36302]: pgmap v2096: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:33:04.149 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:33:03 smithi119 ceph-mon[31518]: pgmap v2096: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:33:04.196 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:33:03 smithi132 ceph-mon[35941]: pgmap v2096: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:33:06.126 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:33:05 smithi165 ceph-mon[36302]: pgmap v2097: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:33:06.126 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:33:05 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3445.710523 secs 2024-03-28T01:33:06.149 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:33:05 smithi119 ceph-mon[31518]: pgmap v2097: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:33:06.149 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:33:05 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3445.710523 secs 2024-03-28T01:33:06.196 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:33:05 smithi132 ceph-mon[35941]: pgmap v2097: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:33:06.196 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:33:05 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3445.710523 secs 2024-03-28T01:33:08.126 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:33:07 smithi165 ceph-mon[36302]: pgmap v2098: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:33:08.149 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:33:07 smithi119 ceph-mon[31518]: pgmap v2098: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:33:08.196 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:33:07 smithi132 ceph-mon[35941]: pgmap v2098: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:33:10.126 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:33:09 smithi165 ceph-mon[36302]: pgmap v2099: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:33:10.149 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:33:09 smithi119 ceph-mon[31518]: pgmap v2099: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:33:10.196 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:33:09 smithi132 ceph-mon[35941]: pgmap v2099: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:33:11.126 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:33:10 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3450.710614 secs 2024-03-28T01:33:11.149 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:33:10 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3450.710614 secs 2024-03-28T01:33:11.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:33:10 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3450.710614 secs 2024-03-28T01:33:12.126 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:33:11 smithi165 ceph-mon[36302]: pgmap v2100: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:33:12.149 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:33:11 smithi119 ceph-mon[31518]: pgmap v2100: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:33:12.196 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:33:11 smithi132 ceph-mon[35941]: pgmap v2100: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:33:14.126 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:33:13 smithi165 ceph-mon[36302]: pgmap v2101: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:33:14.149 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:33:13 smithi119 ceph-mon[31518]: pgmap v2101: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:33:14.196 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:33:13 smithi132 ceph-mon[35941]: pgmap v2101: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:33:16.126 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:33:15 smithi165 ceph-mon[36302]: pgmap v2102: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:33:16.126 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:33:15 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3455.710698 secs 2024-03-28T01:33:16.149 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:33:15 smithi119 ceph-mon[31518]: pgmap v2102: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:33:16.149 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:33:15 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3455.710698 secs 2024-03-28T01:33:16.196 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:33:15 smithi132 ceph-mon[35941]: pgmap v2102: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:33:16.196 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:33:15 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3455.710698 secs 2024-03-28T01:33:18.126 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:33:17 smithi165 ceph-mon[36302]: pgmap v2103: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T01:33:18.149 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:33:17 smithi119 ceph-mon[31518]: pgmap v2103: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T01:33:18.196 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:33:17 smithi132 ceph-mon[35941]: pgmap v2103: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T01:33:20.149 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:33:19 smithi119 ceph-mon[31518]: pgmap v2104: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T01:33:20.149 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:33:19 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:33:20.196 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:33:19 smithi132 ceph-mon[35941]: pgmap v2104: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T01:33:20.196 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:33:19 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:33:20.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:33:19 smithi165 ceph-mon[36302]: pgmap v2104: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T01:33:20.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:33:19 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:33:21.149 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:33:20 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3460.710757 secs 2024-03-28T01:33:21.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:33:20 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:33:21.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:33:20 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:33:21.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:33:20 smithi119 ceph-mon[31518]: pgmap v2105: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 7 op/s 2024-03-28T01:33:21.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:33:20 smithi119 ceph-mon[31518]: pgmap v2106: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T01:33:21.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:33:20 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:33:21.196 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:33:20 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3460.710757 secs 2024-03-28T01:33:21.196 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:33:20 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:33:21.196 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:33:20 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:33:21.196 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:33:20 smithi132 ceph-mon[35941]: pgmap v2105: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 7 op/s 2024-03-28T01:33:21.196 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:33:20 smithi132 ceph-mon[35941]: pgmap v2106: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T01:33:21.196 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:33:20 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:33:21.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:33:20 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3460.710757 secs 2024-03-28T01:33:21.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:33:20 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:33:21.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:33:20 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:33:21.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:33:20 smithi165 ceph-mon[36302]: pgmap v2105: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 7 op/s 2024-03-28T01:33:21.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:33:20 smithi165 ceph-mon[36302]: pgmap v2106: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T01:33:21.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:33:20 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:33:23.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:33:23 smithi165 ceph-mon[36302]: pgmap v2107: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T01:33:23.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:33:23 smithi119 ceph-mon[31518]: pgmap v2107: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T01:33:23.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:33:23 smithi132 ceph-mon[35941]: pgmap v2107: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T01:33:25.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:33:25 smithi165 ceph-mon[36302]: pgmap v2108: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:33:25.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:33:25 smithi119 ceph-mon[31518]: pgmap v2108: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:33:25.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:33:25 smithi132 ceph-mon[35941]: pgmap v2108: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:33:26.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:33:26 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3465.710818 secs 2024-03-28T01:33:26.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:33:26 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3465.710818 secs 2024-03-28T01:33:26.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:33:26 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3465.710818 secs 2024-03-28T01:33:27.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:33:27 smithi165 ceph-mon[36302]: pgmap v2109: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:33:27.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:33:27 smithi119 ceph-mon[31518]: pgmap v2109: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:33:27.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:33:27 smithi132 ceph-mon[35941]: pgmap v2109: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:33:29.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:33:29 smithi165 ceph-mon[36302]: pgmap v2110: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:33:29.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:33:29 smithi119 ceph-mon[31518]: pgmap v2110: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:33:29.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:33:29 smithi132 ceph-mon[35941]: pgmap v2110: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:33:30.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:33:30 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3470.710915 secs 2024-03-28T01:33:30.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:33:30 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3470.710915 secs 2024-03-28T01:33:30.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:33:30 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3470.710915 secs 2024-03-28T01:33:31.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:33:31 smithi119 ceph-mon[31518]: pgmap v2111: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T01:33:31.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:33:31 smithi132 ceph-mon[35941]: pgmap v2111: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T01:33:31.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:33:31 smithi165 ceph-mon[36302]: pgmap v2111: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T01:33:33.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:33:33 smithi119 ceph-mon[31518]: pgmap v2112: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:33:33.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:33:33 smithi132 ceph-mon[35941]: pgmap v2112: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:33:33.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:33:33 smithi165 ceph-mon[36302]: pgmap v2112: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:33:35.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:33:35 smithi119 ceph-mon[31518]: pgmap v2113: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:33:35.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:33:35 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3475.710976 secs 2024-03-28T01:33:35.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:33:35 smithi132 ceph-mon[35941]: pgmap v2113: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:33:35.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:33:35 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3475.710976 secs 2024-03-28T01:33:35.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:33:35 smithi165 ceph-mon[36302]: pgmap v2113: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:33:35.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:33:35 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3475.710976 secs 2024-03-28T01:33:37.452 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:33:37 smithi119 ceph-mon[31518]: pgmap v2114: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:33:37.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:33:37 smithi132 ceph-mon[35941]: pgmap v2114: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:33:37.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:33:37 smithi165 ceph-mon[36302]: pgmap v2114: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:33:39.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:33:39 smithi119 ceph-mon[31518]: pgmap v2115: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:33:39.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:33:39 smithi132 ceph-mon[35941]: pgmap v2115: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:33:39.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:33:39 smithi165 ceph-mon[36302]: pgmap v2115: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:33:40.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:33:40 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3480.710987 secs 2024-03-28T01:33:40.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:33:40 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3480.710987 secs 2024-03-28T01:33:40.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:33:40 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3480.710987 secs 2024-03-28T01:33:41.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:33:41 smithi132 ceph-mon[35941]: pgmap v2116: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:33:41.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:33:41 smithi165 ceph-mon[36302]: pgmap v2116: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:33:41.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:33:41 smithi119 ceph-mon[31518]: pgmap v2116: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:33:43.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:33:43 smithi132 ceph-mon[35941]: pgmap v2117: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:33:43.701 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:33:43 smithi119 ceph-mon[31518]: pgmap v2117: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:33:43.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:33:43 smithi165 ceph-mon[36302]: pgmap v2117: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:33:45.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:33:45 smithi132 ceph-mon[35941]: pgmap v2118: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:33:45.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:33:45 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3485.711048 secs 2024-03-28T01:33:45.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:33:45 smithi165 ceph-mon[36302]: pgmap v2118: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:33:45.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:33:45 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3485.711048 secs 2024-03-28T01:33:45.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:33:45 smithi119 ceph-mon[31518]: pgmap v2118: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:33:45.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:33:45 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3485.711048 secs 2024-03-28T01:33:47.678 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:33:47 smithi165 ceph-mon[36302]: pgmap v2119: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:33:47.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:33:47 smithi132 ceph-mon[35941]: pgmap v2119: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:33:47.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:33:47 smithi119 ceph-mon[31518]: pgmap v2119: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:33:49.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:33:49 smithi132 ceph-mon[35941]: pgmap v2120: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:33:49.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:33:49 smithi165 ceph-mon[36302]: pgmap v2120: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:33:49.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:33:49 smithi119 ceph-mon[31518]: pgmap v2120: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:33:50.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:33:50 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3490.711111 secs 2024-03-28T01:33:50.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:33:50 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3490.711111 secs 2024-03-28T01:33:50.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:33:50 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3490.711111 secs 2024-03-28T01:33:51.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:33:51 smithi132 ceph-mon[35941]: pgmap v2121: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:33:51.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:33:51 smithi165 ceph-mon[36302]: pgmap v2121: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:33:51.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:33:51 smithi119 ceph-mon[31518]: pgmap v2121: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:33:53.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:33:53 smithi132 ceph-mon[35941]: pgmap v2122: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:33:53.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:33:53 smithi165 ceph-mon[36302]: pgmap v2122: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:33:53.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:33:53 smithi119 ceph-mon[31518]: pgmap v2122: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:33:55.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:33:55 smithi132 ceph-mon[35941]: pgmap v2123: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:33:55.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:33:55 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3495.711199 secs 2024-03-28T01:33:55.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:33:55 smithi165 ceph-mon[36302]: pgmap v2123: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:33:55.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:33:55 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3495.711199 secs 2024-03-28T01:33:55.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:33:55 smithi119 ceph-mon[31518]: pgmap v2123: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:33:55.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:33:55 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3495.711199 secs 2024-03-28T01:33:57.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:33:57 smithi165 ceph-mon[36302]: pgmap v2124: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:33:57.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:33:57 smithi119 ceph-mon[31518]: pgmap v2124: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:33:57.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:33:57 smithi132 ceph-mon[35941]: pgmap v2124: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:33:59.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:33:59 smithi165 ceph-mon[36302]: pgmap v2125: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:33:59.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:33:59 smithi119 ceph-mon[31518]: pgmap v2125: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:33:59.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:33:59 smithi132 ceph-mon[35941]: pgmap v2125: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:34:00.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:34:00 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3500.711238 secs 2024-03-28T01:34:00.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:34:00 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3500.711238 secs 2024-03-28T01:34:00.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:34:00 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3500.711238 secs 2024-03-28T01:34:01.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:34:01 smithi165 ceph-mon[36302]: pgmap v2126: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:34:01.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:34:01 smithi119 ceph-mon[31518]: pgmap v2126: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:34:01.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:34:01 smithi132 ceph-mon[35941]: pgmap v2126: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:34:03.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:34:03 smithi165 ceph-mon[36302]: pgmap v2127: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T01:34:03.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:34:03 smithi119 ceph-mon[31518]: pgmap v2127: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T01:34:03.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:34:03 smithi132 ceph-mon[35941]: pgmap v2127: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T01:34:05.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:34:05 smithi165 ceph-mon[36302]: pgmap v2128: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:34:05.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:34:05 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3505.711326 secs 2024-03-28T01:34:05.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:34:05 smithi119 ceph-mon[31518]: pgmap v2128: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:34:05.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:34:05 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3505.711326 secs 2024-03-28T01:34:05.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:34:05 smithi132 ceph-mon[35941]: pgmap v2128: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:34:05.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:34:05 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3505.711326 secs 2024-03-28T01:34:07.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:34:07 smithi165 ceph-mon[36302]: pgmap v2129: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T01:34:07.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:34:07 smithi119 ceph-mon[31518]: pgmap v2129: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T01:34:07.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:34:07 smithi132 ceph-mon[35941]: pgmap v2129: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T01:34:09.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:34:09 smithi165 ceph-mon[36302]: pgmap v2130: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T01:34:09.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:34:09 smithi119 ceph-mon[31518]: pgmap v2130: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T01:34:09.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:34:09 smithi132 ceph-mon[35941]: pgmap v2130: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T01:34:10.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:34:10 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3510.711412 secs 2024-03-28T01:34:10.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:34:10 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3510.711412 secs 2024-03-28T01:34:10.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:34:10 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3510.711412 secs 2024-03-28T01:34:11.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:34:11 smithi165 ceph-mon[36302]: pgmap v2131: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:34:11.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:34:11 smithi119 ceph-mon[31518]: pgmap v2131: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:34:11.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:34:11 smithi132 ceph-mon[35941]: pgmap v2131: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:34:12.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:34:12 smithi165 ceph-mon[36302]: pgmap v2132: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:34:12.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:34:12 smithi119 ceph-mon[31518]: pgmap v2132: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:34:12.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:34:12 smithi132 ceph-mon[35941]: pgmap v2132: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:34:15.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:34:15 smithi165 ceph-mon[36302]: pgmap v2133: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:34:15.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:34:15 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3515.711501 secs 2024-03-28T01:34:15.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:34:15 smithi119 ceph-mon[31518]: pgmap v2133: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:34:15.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:34:15 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3515.711501 secs 2024-03-28T01:34:15.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:34:15 smithi132 ceph-mon[35941]: pgmap v2133: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:34:15.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:34:15 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3515.711501 secs 2024-03-28T01:34:17.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:34:17 smithi165 ceph-mon[36302]: pgmap v2134: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:34:17.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:34:17 smithi119 ceph-mon[31518]: pgmap v2134: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:34:17.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:34:17 smithi132 ceph-mon[35941]: pgmap v2134: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:34:19.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:34:19 smithi119 ceph-mon[31518]: pgmap v2135: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:34:19.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:34:19 smithi132 ceph-mon[35941]: pgmap v2135: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:34:19.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:34:19 smithi165 ceph-mon[36302]: pgmap v2135: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:34:20.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:34:20 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3520.711588 secs 2024-03-28T01:34:20.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:34:20 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3520.711588 secs 2024-03-28T01:34:20.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:34:20 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3520.711588 secs 2024-03-28T01:34:21.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:34:21 smithi119 ceph-mon[31518]: pgmap v2136: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:34:21.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:34:21 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:34:21.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:34:21 smithi132 ceph-mon[35941]: pgmap v2136: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:34:21.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:34:21 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:34:21.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:34:21 smithi165 ceph-mon[36302]: pgmap v2136: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:34:21.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:34:21 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:34:22.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:34:22 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:34:22.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:34:22 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:34:22.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:34:22 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:34:22.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:34:22 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:34:22.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:34:22 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:34:22.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:34:22 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:34:22.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:34:22 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:34:22.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:34:22 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:34:22.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:34:22 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:34:23.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:34:23 smithi119 ceph-mon[31518]: pgmap v2137: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T01:34:23.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:34:23 smithi119 ceph-mon[31518]: pgmap v2138: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:34:23.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:34:23 smithi132 ceph-mon[35941]: pgmap v2137: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T01:34:23.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:34:23 smithi132 ceph-mon[35941]: pgmap v2138: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:34:23.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:34:23 smithi165 ceph-mon[36302]: pgmap v2137: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T01:34:23.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:34:23 smithi165 ceph-mon[36302]: pgmap v2138: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:34:25.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:34:25 smithi132 ceph-mon[35941]: pgmap v2139: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-28T01:34:25.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:34:25 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3525.711649 secs 2024-03-28T01:34:25.877 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:34:25 smithi165 ceph-mon[36302]: pgmap v2139: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-28T01:34:25.877 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:34:25 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3525.711649 secs 2024-03-28T01:34:25.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:34:25 smithi119 ceph-mon[31518]: pgmap v2139: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-28T01:34:25.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:34:25 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3525.711649 secs 2024-03-28T01:34:27.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:34:27 smithi132 ceph-mon[35941]: pgmap v2140: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:34:27.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:34:27 smithi165 ceph-mon[36302]: pgmap v2140: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:34:27.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:34:27 smithi119 ceph-mon[31518]: pgmap v2140: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:34:29.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:34:29 smithi132 ceph-mon[35941]: pgmap v2141: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:34:29.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:34:29 smithi165 ceph-mon[36302]: pgmap v2141: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:34:29.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:34:29 smithi119 ceph-mon[31518]: pgmap v2141: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:34:30.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:34:30 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3530.711660 secs 2024-03-28T01:34:30.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:34:30 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3530.711660 secs 2024-03-28T01:34:30.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:34:30 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3530.711660 secs 2024-03-28T01:34:31.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:34:31 smithi132 ceph-mon[35941]: pgmap v2142: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:34:31.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:34:31 smithi165 ceph-mon[36302]: pgmap v2142: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:34:31.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:34:31 smithi119 ceph-mon[31518]: pgmap v2142: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:34:33.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:34:33 smithi132 ceph-mon[35941]: pgmap v2143: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T01:34:33.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:34:33 smithi165 ceph-mon[36302]: pgmap v2143: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T01:34:33.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:34:33 smithi119 ceph-mon[31518]: pgmap v2143: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T01:34:35.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:34:35 smithi132 ceph-mon[35941]: pgmap v2144: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:34:35.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:34:35 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3535.711748 secs 2024-03-28T01:34:35.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:34:35 smithi165 ceph-mon[36302]: pgmap v2144: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:34:35.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:34:35 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3535.711748 secs 2024-03-28T01:34:35.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:34:35 smithi119 ceph-mon[31518]: pgmap v2144: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:34:35.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:34:35 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3535.711748 secs 2024-03-28T01:34:37.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:34:37 smithi165 ceph-mon[36302]: pgmap v2145: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:34:37.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:34:37 smithi119 ceph-mon[31518]: pgmap v2145: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:34:37.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:34:37 smithi132 ceph-mon[35941]: pgmap v2145: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:34:39.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:34:39 smithi165 ceph-mon[36302]: pgmap v2146: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:34:39.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:34:39 smithi119 ceph-mon[31518]: pgmap v2146: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:34:39.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:34:39 smithi132 ceph-mon[35941]: pgmap v2146: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:34:40.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:34:40 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3540.711812 secs 2024-03-28T01:34:40.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:34:40 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3540.711812 secs 2024-03-28T01:34:40.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:34:40 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3540.711812 secs 2024-03-28T01:34:41.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:34:41 smithi165 ceph-mon[36302]: pgmap v2147: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:34:41.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:34:41 smithi119 ceph-mon[31518]: pgmap v2147: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:34:41.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:34:41 smithi132 ceph-mon[35941]: pgmap v2147: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:34:43.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:34:43 smithi165 ceph-mon[36302]: pgmap v2148: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:34:43.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:34:43 smithi119 ceph-mon[31518]: pgmap v2148: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:34:43.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:34:43 smithi132 ceph-mon[35941]: pgmap v2148: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:34:45.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:34:45 smithi165 ceph-mon[36302]: pgmap v2149: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:34:45.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:34:45 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3545.711902 secs 2024-03-28T01:34:45.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:34:45 smithi119 ceph-mon[31518]: pgmap v2149: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:34:45.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:34:45 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3545.711902 secs 2024-03-28T01:34:45.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:34:45 smithi132 ceph-mon[35941]: pgmap v2149: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:34:45.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:34:45 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3545.711902 secs 2024-03-28T01:34:47.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:34:47 smithi165 ceph-mon[36302]: pgmap v2150: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:34:47.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:34:47 smithi119 ceph-mon[31518]: pgmap v2150: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:34:47.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:34:47 smithi132 ceph-mon[35941]: pgmap v2150: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:34:49.748 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:34:49 smithi132 ceph-mon[35941]: pgmap v2151: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:34:49.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:34:49 smithi165 ceph-mon[36302]: pgmap v2151: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:34:49.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:34:49 smithi119 ceph-mon[31518]: pgmap v2151: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:34:50.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:34:50 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3550.711990 secs 2024-03-28T01:34:50.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:34:50 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3550.711990 secs 2024-03-28T01:34:50.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:34:50 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3550.711990 secs 2024-03-28T01:34:51.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:34:51 smithi165 ceph-mon[36302]: pgmap v2152: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:34:51.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:34:51 smithi119 ceph-mon[31518]: pgmap v2152: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:34:51.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:34:51 smithi132 ceph-mon[35941]: pgmap v2152: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:34:53.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:34:53 smithi165 ceph-mon[36302]: pgmap v2153: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:34:53.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:34:53 smithi119 ceph-mon[31518]: pgmap v2153: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:34:53.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:34:53 smithi132 ceph-mon[35941]: pgmap v2153: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:34:55.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:34:55 smithi165 ceph-mon[36302]: pgmap v2154: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:34:55.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:34:55 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3555.712076 secs 2024-03-28T01:34:55.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:34:55 smithi119 ceph-mon[31518]: pgmap v2154: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:34:55.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:34:55 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3555.712076 secs 2024-03-28T01:34:55.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:34:55 smithi132 ceph-mon[35941]: pgmap v2154: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:34:55.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:34:55 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3555.712076 secs 2024-03-28T01:34:56.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:34:56 smithi165 ceph-mon[36302]: pgmap v2155: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:34:56.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:34:56 smithi119 ceph-mon[31518]: pgmap v2155: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:34:56.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:34:56 smithi132 ceph-mon[35941]: pgmap v2155: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:34:59.149 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:34:58 smithi119 ceph-mon[31518]: pgmap v2156: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:34:59.196 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:34:58 smithi132 ceph-mon[35941]: pgmap v2156: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:34:59.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:34:58 smithi165 ceph-mon[36302]: pgmap v2156: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:35:01.149 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:35:00 smithi119 ceph-mon[31518]: pgmap v2157: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:35:01.149 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:35:00 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3560.712163 secs 2024-03-28T01:35:01.196 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:35:00 smithi132 ceph-mon[35941]: pgmap v2157: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:35:01.196 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:35:00 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3560.712163 secs 2024-03-28T01:35:01.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:35:00 smithi165 ceph-mon[36302]: pgmap v2157: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:35:01.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:35:00 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3560.712163 secs 2024-03-28T01:35:03.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:35:02 smithi165 ceph-mon[36302]: pgmap v2158: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:35:03.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:35:02 smithi119 ceph-mon[31518]: pgmap v2158: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:35:03.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:35:02 smithi132 ceph-mon[35941]: pgmap v2158: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:35:05.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:35:04 smithi165 ceph-mon[36302]: pgmap v2159: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:35:05.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:35:04 smithi119 ceph-mon[31518]: pgmap v2159: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:35:05.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:35:04 smithi132 ceph-mon[35941]: pgmap v2159: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:35:06.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:35:05 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3565.712224 secs 2024-03-28T01:35:06.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:35:05 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3565.712224 secs 2024-03-28T01:35:06.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:35:05 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3565.712224 secs 2024-03-28T01:35:07.375 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:35:06 smithi165 ceph-mon[36302]: pgmap v2160: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:35:07.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:35:06 smithi119 ceph-mon[31518]: pgmap v2160: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:35:07.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:35:06 smithi132 ceph-mon[35941]: pgmap v2160: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:35:09.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:35:08 smithi165 ceph-mon[36302]: pgmap v2161: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:35:09.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:35:08 smithi119 ceph-mon[31518]: pgmap v2161: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:35:09.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:35:09 smithi132 ceph-mon[35941]: pgmap v2161: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:35:11.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:35:11 smithi165 ceph-mon[36302]: pgmap v2162: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:35:11.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:35:11 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3570.712312 secs 2024-03-28T01:35:11.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:35:11 smithi119 ceph-mon[31518]: pgmap v2162: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:35:11.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:35:11 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3570.712312 secs 2024-03-28T01:35:11.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:35:11 smithi132 ceph-mon[35941]: pgmap v2162: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:35:11.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:35:11 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3570.712312 secs 2024-03-28T01:35:13.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:35:13 smithi165 ceph-mon[36302]: pgmap v2163: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:35:13.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:35:13 smithi119 ceph-mon[31518]: pgmap v2163: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:35:13.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:35:13 smithi132 ceph-mon[35941]: pgmap v2163: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:35:15.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:35:15 smithi165 ceph-mon[36302]: pgmap v2164: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:35:15.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:35:15 smithi119 ceph-mon[31518]: pgmap v2164: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:35:15.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:35:15 smithi132 ceph-mon[35941]: pgmap v2164: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:35:16.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:35:16 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3575.712402 secs 2024-03-28T01:35:16.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:35:16 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3575.712402 secs 2024-03-28T01:35:16.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:35:16 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3575.712402 secs 2024-03-28T01:35:17.277 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:35:17 smithi165 ceph-mon[36302]: pgmap v2165: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:35:17.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:35:17 smithi119 ceph-mon[31518]: pgmap v2165: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:35:17.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:35:17 smithi132 ceph-mon[35941]: pgmap v2165: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:35:19.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:35:19 smithi165 ceph-mon[36302]: pgmap v2166: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:35:19.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:35:19 smithi119 ceph-mon[31518]: pgmap v2166: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:35:19.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:35:19 smithi132 ceph-mon[35941]: pgmap v2166: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:35:21.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:35:21 smithi165 ceph-mon[36302]: pgmap v2167: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:35:21.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:35:21 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3580.712491 secs 2024-03-28T01:35:21.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:35:21 smithi119 ceph-mon[31518]: pgmap v2167: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:35:21.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:35:21 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3580.712491 secs 2024-03-28T01:35:21.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:35:21 smithi132 ceph-mon[35941]: pgmap v2167: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:35:21.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:35:21 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3580.712491 secs 2024-03-28T01:35:23.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:35:23 smithi165 ceph-mon[36302]: pgmap v2168: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:35:23.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:35:23 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:35:23.398 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:35:23 smithi119 ceph-mon[31518]: pgmap v2168: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:35:23.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:35:23 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:35:23.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:35:23 smithi132 ceph-mon[35941]: pgmap v2168: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:35:23.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:35:23 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:35:24.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:35:24 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:35:24.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:35:24 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:35:24.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:35:24 smithi165 ceph-mon[36302]: pgmap v2169: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T01:35:24.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:35:24 smithi165 ceph-mon[36302]: pgmap v2170: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:35:24.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:35:24 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:35:24.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:35:24 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:35:24.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:35:24 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:35:24.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:35:24 smithi119 ceph-mon[31518]: pgmap v2169: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T01:35:24.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:35:24 smithi119 ceph-mon[31518]: pgmap v2170: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:35:24.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:35:24 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:35:24.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:35:24 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:35:24.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:35:24 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:35:24.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:35:24 smithi132 ceph-mon[35941]: pgmap v2169: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T01:35:24.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:35:24 smithi132 ceph-mon[35941]: pgmap v2170: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:35:24.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:35:24 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:35:25.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:35:25 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3585.712553 secs 2024-03-28T01:35:25.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:35:25 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3585.712553 secs 2024-03-28T01:35:25.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:35:25 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3585.712553 secs 2024-03-28T01:35:26.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:35:26 smithi132 ceph-mon[35941]: pgmap v2171: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:35:26.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:35:26 smithi165 ceph-mon[36302]: pgmap v2171: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:35:26.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:35:26 smithi119 ceph-mon[31518]: pgmap v2171: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:35:28.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:35:28 smithi132 ceph-mon[35941]: pgmap v2172: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:35:28.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:35:28 smithi165 ceph-mon[36302]: pgmap v2172: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:35:28.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:35:28 smithi119 ceph-mon[31518]: pgmap v2172: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:35:30.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:35:30 smithi132 ceph-mon[35941]: pgmap v2173: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:35:30.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:35:30 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3590.712614 secs 2024-03-28T01:35:30.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:35:30 smithi165 ceph-mon[36302]: pgmap v2173: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:35:30.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:35:30 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3590.712614 secs 2024-03-28T01:35:30.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:35:30 smithi119 ceph-mon[31518]: pgmap v2173: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:35:30.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:35:30 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3590.712614 secs 2024-03-28T01:35:32.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:35:32 smithi132 ceph-mon[35941]: pgmap v2174: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:35:32.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:35:32 smithi165 ceph-mon[36302]: pgmap v2174: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:35:32.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:35:32 smithi119 ceph-mon[31518]: pgmap v2174: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:35:34.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:35:34 smithi132 ceph-mon[35941]: pgmap v2175: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T01:35:34.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:35:34 smithi165 ceph-mon[36302]: pgmap v2175: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T01:35:34.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:35:34 smithi119 ceph-mon[31518]: pgmap v2175: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T01:35:35.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:35:35 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3595.712673 secs 2024-03-28T01:35:35.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:35:35 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3595.712673 secs 2024-03-28T01:35:35.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:35:35 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3595.712673 secs 2024-03-28T01:35:36.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:35:36 smithi165 ceph-mon[36302]: pgmap v2176: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:35:36.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:35:36 smithi119 ceph-mon[31518]: pgmap v2176: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:35:36.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:35:36 smithi132 ceph-mon[35941]: pgmap v2176: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:35:38.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:35:38 smithi165 ceph-mon[36302]: pgmap v2177: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:35:38.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:35:38 smithi119 ceph-mon[31518]: pgmap v2177: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:35:38.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:35:38 smithi132 ceph-mon[35941]: pgmap v2177: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:35:40.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:35:40 smithi165 ceph-mon[36302]: pgmap v2178: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:35:40.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:35:40 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3600.712761 secs 2024-03-28T01:35:40.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:35:40 smithi119 ceph-mon[31518]: pgmap v2178: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:35:40.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:35:40 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3600.712761 secs 2024-03-28T01:35:40.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:35:40 smithi132 ceph-mon[35941]: pgmap v2178: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:35:40.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:35:40 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3600.712761 secs 2024-03-28T01:35:42.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:35:42 smithi165 ceph-mon[36302]: pgmap v2179: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:35:42.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:35:42 smithi119 ceph-mon[31518]: pgmap v2179: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:35:42.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:35:42 smithi132 ceph-mon[35941]: pgmap v2179: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:35:44.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:35:44 smithi165 ceph-mon[36302]: pgmap v2180: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:35:44.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:35:44 smithi119 ceph-mon[31518]: pgmap v2180: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:35:44.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:35:44 smithi132 ceph-mon[35941]: pgmap v2180: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:35:45.801 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:35:45 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3605.712851 secs 2024-03-28T01:35:45.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:35:45 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3605.712851 secs 2024-03-28T01:35:45.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:35:45 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3605.712851 secs 2024-03-28T01:35:46.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:35:46 smithi165 ceph-mon[36302]: pgmap v2181: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:35:46.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:35:46 smithi119 ceph-mon[31518]: pgmap v2181: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:35:46.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:35:46 smithi132 ceph-mon[35941]: pgmap v2181: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:35:48.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:35:48 smithi165 ceph-mon[36302]: pgmap v2182: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:35:48.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:35:48 smithi119 ceph-mon[31518]: pgmap v2182: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:35:48.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:35:48 smithi132 ceph-mon[35941]: pgmap v2182: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:35:50.748 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:35:50 smithi132 ceph-mon[35941]: pgmap v2183: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:35:50.748 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:35:50 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3610.712941 secs 2024-03-28T01:35:50.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:35:50 smithi165 ceph-mon[36302]: pgmap v2183: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:35:50.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:35:50 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3610.712941 secs 2024-03-28T01:35:50.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:35:50 smithi119 ceph-mon[31518]: pgmap v2183: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:35:50.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:35:50 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3610.712941 secs 2024-03-28T01:35:52.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:35:52 smithi165 ceph-mon[36302]: pgmap v2184: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:35:52.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:35:52 smithi119 ceph-mon[31518]: pgmap v2184: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:35:52.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:35:52 smithi132 ceph-mon[35941]: pgmap v2184: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:35:54.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:35:54 smithi165 ceph-mon[36302]: pgmap v2185: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:35:54.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:35:54 smithi119 ceph-mon[31518]: pgmap v2185: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:35:54.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:35:54 smithi132 ceph-mon[35941]: pgmap v2185: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:35:55.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:35:55 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3615.713027 secs 2024-03-28T01:35:55.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:35:55 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3615.713027 secs 2024-03-28T01:35:55.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:35:55 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3615.713027 secs 2024-03-28T01:35:56.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:35:56 smithi165 ceph-mon[36302]: pgmap v2186: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:35:56.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:35:56 smithi119 ceph-mon[31518]: pgmap v2186: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:35:56.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:35:56 smithi132 ceph-mon[35941]: pgmap v2186: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:35:58.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:35:58 smithi165 ceph-mon[36302]: pgmap v2187: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:35:58.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:35:58 smithi119 ceph-mon[31518]: pgmap v2187: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:35:58.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:35:58 smithi132 ceph-mon[35941]: pgmap v2187: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:36:00.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:36:00 smithi165 ceph-mon[36302]: pgmap v2188: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:36:00.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:36:00 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3620.713089 secs 2024-03-28T01:36:00.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:36:00 smithi119 ceph-mon[31518]: pgmap v2188: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:36:00.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:36:00 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3620.713089 secs 2024-03-28T01:36:00.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:36:00 smithi132 ceph-mon[35941]: pgmap v2188: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:36:00.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:36:00 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3620.713089 secs 2024-03-28T01:36:01.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:36:01 smithi165 ceph-mon[36302]: pgmap v2189: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:36:01.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:36:01 smithi119 ceph-mon[31518]: pgmap v2189: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:36:01.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:36:01 smithi132 ceph-mon[35941]: pgmap v2189: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:36:04.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:36:04 smithi132 ceph-mon[35941]: pgmap v2190: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:36:04.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:36:04 smithi165 ceph-mon[36302]: pgmap v2190: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:36:04.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:36:04 smithi119 ceph-mon[31518]: pgmap v2190: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:36:05.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:36:05 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3625.713165 secs 2024-03-28T01:36:05.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:36:05 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3625.713165 secs 2024-03-28T01:36:05.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:36:05 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3625.713165 secs 2024-03-28T01:36:06.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:36:06 smithi132 ceph-mon[35941]: pgmap v2191: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:36:06.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:36:06 smithi165 ceph-mon[36302]: pgmap v2191: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:36:06.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:36:06 smithi119 ceph-mon[31518]: pgmap v2191: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:36:08.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:36:08 smithi132 ceph-mon[35941]: pgmap v2192: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:36:08.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:36:08 smithi165 ceph-mon[36302]: pgmap v2192: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:36:08.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:36:08 smithi119 ceph-mon[31518]: pgmap v2192: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:36:10.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:36:10 smithi132 ceph-mon[35941]: pgmap v2193: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:36:10.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:36:10 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3630.713205 secs 2024-03-28T01:36:10.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:36:10 smithi165 ceph-mon[36302]: pgmap v2193: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:36:10.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:36:10 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3630.713205 secs 2024-03-28T01:36:10.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:36:10 smithi119 ceph-mon[31518]: pgmap v2193: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:36:10.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:36:10 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3630.713205 secs 2024-03-28T01:36:12.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:36:12 smithi119 ceph-mon[31518]: pgmap v2194: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:36:12.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:36:12 smithi132 ceph-mon[35941]: pgmap v2194: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:36:12.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:36:12 smithi165 ceph-mon[36302]: pgmap v2194: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:36:14.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:36:14 smithi132 ceph-mon[35941]: pgmap v2195: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:36:14.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:36:14 smithi165 ceph-mon[36302]: pgmap v2195: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:36:14.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:36:14 smithi119 ceph-mon[31518]: pgmap v2195: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:36:15.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:36:15 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3635.713293 secs 2024-03-28T01:36:15.801 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:36:15 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3635.713293 secs 2024-03-28T01:36:15.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:36:15 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3635.713293 secs 2024-03-28T01:36:16.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:36:16 smithi165 ceph-mon[36302]: pgmap v2196: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:36:16.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:36:16 smithi119 ceph-mon[31518]: pgmap v2196: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:36:16.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:36:16 smithi132 ceph-mon[35941]: pgmap v2196: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:36:18.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:36:18 smithi165 ceph-mon[36302]: pgmap v2197: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:36:18.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:36:18 smithi119 ceph-mon[31518]: pgmap v2197: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:36:18.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:36:18 smithi132 ceph-mon[35941]: pgmap v2197: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:36:20.748 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:36:20 smithi132 ceph-mon[35941]: pgmap v2198: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:36:20.748 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:36:20 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3640.713355 secs 2024-03-28T01:36:20.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:36:20 smithi165 ceph-mon[36302]: pgmap v2198: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:36:20.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:36:20 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3640.713355 secs 2024-03-28T01:36:20.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:36:20 smithi119 ceph-mon[31518]: pgmap v2198: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:36:20.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:36:20 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3640.713355 secs 2024-03-28T01:36:22.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:36:22 smithi165 ceph-mon[36302]: pgmap v2199: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:36:22.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:36:22 smithi119 ceph-mon[31518]: pgmap v2199: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:36:22.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:36:22 smithi132 ceph-mon[35941]: pgmap v2199: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:36:24.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:36:24 smithi165 ceph-mon[36302]: pgmap v2200: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:36:24.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:36:24 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:36:24.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:36:24 smithi119 ceph-mon[31518]: pgmap v2200: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:36:24.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:36:24 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:36:24.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:36:24 smithi132 ceph-mon[35941]: pgmap v2200: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:36:24.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:36:24 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:36:25.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:36:25 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:36:25.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:36:25 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:36:25.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:36:25 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:36:25.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:36:25 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3645.713422 secs 2024-03-28T01:36:25.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:36:25 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:36:25.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:36:25 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:36:25.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:36:25 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:36:25.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:36:25 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3645.713422 secs 2024-03-28T01:36:25.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:36:25 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:36:25.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:36:25 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:36:25.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:36:25 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:36:25.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:36:25 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3645.713422 secs 2024-03-28T01:36:26.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:36:26 smithi165 ceph-mon[36302]: pgmap v2201: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T01:36:26.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:36:26 smithi165 ceph-mon[36302]: pgmap v2202: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:36:26.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:36:26 smithi119 ceph-mon[31518]: pgmap v2201: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T01:36:26.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:36:26 smithi119 ceph-mon[31518]: pgmap v2202: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:36:26.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:36:26 smithi132 ceph-mon[35941]: pgmap v2201: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T01:36:26.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:36:26 smithi132 ceph-mon[35941]: pgmap v2202: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:36:28.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:36:28 smithi165 ceph-mon[36302]: pgmap v2203: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:36:28.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:36:28 smithi119 ceph-mon[31518]: pgmap v2203: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:36:28.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:36:28 smithi132 ceph-mon[35941]: pgmap v2203: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:36:30.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:36:30 smithi165 ceph-mon[36302]: pgmap v2204: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-28T01:36:30.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:36:30 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3650.713512 secs 2024-03-28T01:36:30.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:36:30 smithi119 ceph-mon[31518]: pgmap v2204: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-28T01:36:30.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:36:30 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3650.713512 secs 2024-03-28T01:36:30.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:36:30 smithi132 ceph-mon[35941]: pgmap v2204: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-28T01:36:30.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:36:30 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3650.713512 secs 2024-03-28T01:36:32.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:36:32 smithi165 ceph-mon[36302]: pgmap v2205: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:36:32.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:36:32 smithi119 ceph-mon[31518]: pgmap v2205: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:36:32.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:36:32 smithi132 ceph-mon[35941]: pgmap v2205: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:36:34.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:36:34 smithi165 ceph-mon[36302]: pgmap v2206: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:36:34.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:36:34 smithi119 ceph-mon[31518]: pgmap v2206: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:36:34.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:36:34 smithi132 ceph-mon[35941]: pgmap v2206: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:36:35.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:36:35 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3655.713601 secs 2024-03-28T01:36:35.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:36:35 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3655.713601 secs 2024-03-28T01:36:35.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:36:35 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3655.713601 secs 2024-03-28T01:36:36.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:36:36 smithi165 ceph-mon[36302]: pgmap v2207: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T01:36:36.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:36:36 smithi119 ceph-mon[31518]: pgmap v2207: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T01:36:36.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:36:36 smithi132 ceph-mon[35941]: pgmap v2207: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T01:36:38.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:36:38 smithi165 ceph-mon[36302]: pgmap v2208: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:36:38.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:36:38 smithi119 ceph-mon[31518]: pgmap v2208: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:36:38.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:36:38 smithi132 ceph-mon[35941]: pgmap v2208: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:36:40.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:36:40 smithi165 ceph-mon[36302]: pgmap v2209: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:36:40.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:36:40 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3660.713643 secs 2024-03-28T01:36:40.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:36:40 smithi119 ceph-mon[31518]: pgmap v2209: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:36:40.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:36:40 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3660.713643 secs 2024-03-28T01:36:40.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:36:40 smithi132 ceph-mon[35941]: pgmap v2209: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:36:40.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:36:40 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3660.713643 secs 2024-03-28T01:36:41.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:36:41 smithi165 ceph-mon[36302]: pgmap v2210: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:36:41.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:36:41 smithi119 ceph-mon[31518]: pgmap v2210: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:36:41.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:36:41 smithi132 ceph-mon[35941]: pgmap v2210: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:36:44.196 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:36:43 smithi132 ceph-mon[35941]: pgmap v2211: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:36:44.375 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:36:43 smithi165 ceph-mon[36302]: pgmap v2211: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:36:44.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:36:43 smithi119 ceph-mon[31518]: pgmap v2211: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:36:46.196 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:36:45 smithi132 ceph-mon[35941]: pgmap v2212: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:36:46.196 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:36:45 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3665.713732 secs 2024-03-28T01:36:46.375 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:36:45 smithi165 ceph-mon[36302]: pgmap v2212: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:36:46.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:36:45 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3665.713732 secs 2024-03-28T01:36:46.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:36:45 smithi119 ceph-mon[31518]: pgmap v2212: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:36:46.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:36:45 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3665.713732 secs 2024-03-28T01:36:48.178 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:36:47 smithi165 ceph-mon[36302]: pgmap v2213: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:36:48.196 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:36:47 smithi132 ceph-mon[35941]: pgmap v2213: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:36:48.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:36:47 smithi119 ceph-mon[31518]: pgmap v2213: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:36:50.196 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:36:49 smithi132 ceph-mon[35941]: pgmap v2214: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:36:50.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:36:49 smithi165 ceph-mon[36302]: pgmap v2214: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:36:50.398 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:36:49 smithi119 ceph-mon[31518]: pgmap v2214: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:36:51.196 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:36:50 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3670.713794 secs 2024-03-28T01:36:51.375 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:36:50 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3670.713794 secs 2024-03-28T01:36:51.398 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:36:50 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3670.713794 secs 2024-03-28T01:36:52.196 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:36:51 smithi132 ceph-mon[35941]: pgmap v2215: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:36:52.375 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:36:51 smithi165 ceph-mon[36302]: pgmap v2215: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:36:52.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:36:51 smithi119 ceph-mon[31518]: pgmap v2215: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:36:54.196 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:36:53 smithi132 ceph-mon[35941]: pgmap v2216: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:36:54.375 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:36:53 smithi165 ceph-mon[36302]: pgmap v2216: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:36:54.398 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:36:53 smithi119 ceph-mon[31518]: pgmap v2216: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:36:56.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:36:55 smithi165 ceph-mon[36302]: pgmap v2217: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:36:56.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:36:55 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3675.713857 secs 2024-03-28T01:36:56.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:36:55 smithi119 ceph-mon[31518]: pgmap v2217: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:36:56.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:36:55 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3675.713857 secs 2024-03-28T01:36:56.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:36:55 smithi132 ceph-mon[35941]: pgmap v2217: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:36:56.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:36:55 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3675.713857 secs 2024-03-28T01:36:58.375 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:36:57 smithi165 ceph-mon[36302]: pgmap v2218: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:36:58.398 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:36:57 smithi119 ceph-mon[31518]: pgmap v2218: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:36:58.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:36:57 smithi132 ceph-mon[35941]: pgmap v2218: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:37:00.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:36:59 smithi165 ceph-mon[36302]: pgmap v2219: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:37:00.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:36:59 smithi119 ceph-mon[31518]: pgmap v2219: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:37:00.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:36:59 smithi132 ceph-mon[35941]: pgmap v2219: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:37:01.375 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:37:00 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3680.713921 secs 2024-03-28T01:37:01.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:37:00 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3680.713921 secs 2024-03-28T01:37:01.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:37:00 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3680.713921 secs 2024-03-28T01:37:02.375 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:37:01 smithi165 ceph-mon[36302]: pgmap v2220: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:37:02.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:37:01 smithi119 ceph-mon[31518]: pgmap v2220: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:37:02.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:37:01 smithi132 ceph-mon[35941]: pgmap v2220: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:37:04.375 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:37:03 smithi165 ceph-mon[36302]: pgmap v2221: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:37:04.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:37:03 smithi119 ceph-mon[31518]: pgmap v2221: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:37:04.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:37:03 smithi132 ceph-mon[35941]: pgmap v2221: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:37:06.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:37:05 smithi165 ceph-mon[36302]: pgmap v2222: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:37:06.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:37:05 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3685.713984 secs 2024-03-28T01:37:06.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:37:05 smithi119 ceph-mon[31518]: pgmap v2222: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:37:06.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:37:05 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3685.713984 secs 2024-03-28T01:37:06.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:37:05 smithi132 ceph-mon[35941]: pgmap v2222: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:37:06.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:37:05 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3685.713984 secs 2024-03-28T01:37:08.375 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:37:07 smithi165 ceph-mon[36302]: pgmap v2223: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:37:08.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:37:07 smithi119 ceph-mon[31518]: pgmap v2223: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:37:08.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:37:07 smithi132 ceph-mon[35941]: pgmap v2223: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:37:10.375 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:37:09 smithi165 ceph-mon[36302]: pgmap v2224: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:37:10.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:37:09 smithi119 ceph-mon[31518]: pgmap v2224: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:37:10.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:37:09 smithi132 ceph-mon[35941]: pgmap v2224: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:37:11.375 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:37:10 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3690.714046 secs 2024-03-28T01:37:11.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:37:10 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3690.714046 secs 2024-03-28T01:37:11.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:37:10 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3690.714046 secs 2024-03-28T01:37:12.375 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:37:12 smithi165 ceph-mon[36302]: pgmap v2225: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:37:12.398 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:37:12 smithi119 ceph-mon[31518]: pgmap v2225: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:37:12.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:37:12 smithi132 ceph-mon[35941]: pgmap v2225: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:37:14.271 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:37:14 smithi119 ceph-mon[31518]: pgmap v2226: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:37:14.375 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:37:14 smithi165 ceph-mon[36302]: pgmap v2226: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:37:14.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:37:14 smithi132 ceph-mon[35941]: pgmap v2226: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:37:16.375 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:37:16 smithi165 ceph-mon[36302]: pgmap v2227: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:37:16.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:37:16 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3695.714109 secs 2024-03-28T01:37:16.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:37:16 smithi119 ceph-mon[31518]: pgmap v2227: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:37:16.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:37:16 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3695.714109 secs 2024-03-28T01:37:16.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:37:16 smithi132 ceph-mon[35941]: pgmap v2227: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:37:16.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:37:16 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3695.714109 secs 2024-03-28T01:37:18.375 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:37:18 smithi165 ceph-mon[36302]: pgmap v2228: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:37:18.421 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:37:18 smithi119 ceph-mon[31518]: pgmap v2228: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:37:18.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:37:18 smithi132 ceph-mon[35941]: pgmap v2228: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:37:20.375 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:37:20 smithi165 ceph-mon[36302]: pgmap v2229: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:37:20.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:37:20 smithi119 ceph-mon[31518]: pgmap v2229: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:37:20.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:37:20 smithi132 ceph-mon[35941]: pgmap v2229: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:37:21.347 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:37:21 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3700.714198 secs 2024-03-28T01:37:21.375 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:37:21 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3700.714198 secs 2024-03-28T01:37:21.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:37:21 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3700.714198 secs 2024-03-28T01:37:22.375 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:37:22 smithi165 ceph-mon[36302]: pgmap v2230: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:37:22.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:37:22 smithi119 ceph-mon[31518]: pgmap v2230: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:37:22.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:37:22 smithi132 ceph-mon[35941]: pgmap v2230: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:37:24.375 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:37:24 smithi165 ceph-mon[36302]: pgmap v2231: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:37:24.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:37:24 smithi119 ceph-mon[31518]: pgmap v2231: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:37:24.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:37:24 smithi132 ceph-mon[35941]: pgmap v2231: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:37:26.375 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:37:26 smithi165 ceph-mon[36302]: pgmap v2232: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:37:26.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:37:26 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3705.714260 secs 2024-03-28T01:37:26.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:37:26 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:37:26.398 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:37:26 smithi119 ceph-mon[31518]: pgmap v2232: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:37:26.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:37:26 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3705.714260 secs 2024-03-28T01:37:26.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:37:26 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:37:26.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:37:26 smithi132 ceph-mon[35941]: pgmap v2232: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:37:26.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:37:26 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3705.714260 secs 2024-03-28T01:37:26.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:37:26 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:37:27.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:37:27 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:37:27.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:37:27 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:37:27.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:37:27 smithi165 ceph-mon[36302]: pgmap v2233: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T01:37:27.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:37:27 smithi165 ceph-mon[36302]: pgmap v2234: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:37:27.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:37:27 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:37:27.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:37:27 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:37:27.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:37:27 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:37:27.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:37:27 smithi119 ceph-mon[31518]: pgmap v2233: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T01:37:27.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:37:27 smithi119 ceph-mon[31518]: pgmap v2234: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:37:27.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:37:27 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:37:27.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:37:27 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:37:27.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:37:27 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:37:27.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:37:27 smithi132 ceph-mon[35941]: pgmap v2233: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T01:37:27.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:37:27 smithi132 ceph-mon[35941]: pgmap v2234: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:37:27.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:37:27 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:37:29.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:37:29 smithi165 ceph-mon[36302]: pgmap v2235: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:37:29.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:37:29 smithi119 ceph-mon[31518]: pgmap v2235: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:37:29.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:37:29 smithi132 ceph-mon[35941]: pgmap v2235: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:37:30.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:37:30 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3710.714323 secs 2024-03-28T01:37:30.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:37:30 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3710.714323 secs 2024-03-28T01:37:30.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:37:30 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3710.714323 secs 2024-03-28T01:37:31.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:37:31 smithi165 ceph-mon[36302]: pgmap v2236: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:37:31.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:37:31 smithi119 ceph-mon[31518]: pgmap v2236: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:37:31.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:37:31 smithi132 ceph-mon[35941]: pgmap v2236: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:37:32.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:37:32 smithi165 ceph-mon[36302]: pgmap v2237: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:37:32.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:37:32 smithi119 ceph-mon[31518]: pgmap v2237: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:37:32.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:37:32 smithi132 ceph-mon[35941]: pgmap v2237: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:37:35.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:37:35 smithi165 ceph-mon[36302]: pgmap v2238: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:37:35.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:37:35 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3715.714412 secs 2024-03-28T01:37:35.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:37:35 smithi119 ceph-mon[31518]: pgmap v2238: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:37:35.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:37:35 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3715.714412 secs 2024-03-28T01:37:35.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:37:35 smithi132 ceph-mon[35941]: pgmap v2238: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:37:35.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:37:35 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3715.714412 secs 2024-03-28T01:37:36.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:37:36 smithi165 ceph-mon[36302]: pgmap v2239: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T01:37:36.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:37:36 smithi119 ceph-mon[31518]: pgmap v2239: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T01:37:36.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:37:36 smithi132 ceph-mon[35941]: pgmap v2239: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T01:37:39.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:37:39 smithi165 ceph-mon[36302]: pgmap v2240: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:37:39.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:37:39 smithi119 ceph-mon[31518]: pgmap v2240: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:37:39.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:37:39 smithi132 ceph-mon[35941]: pgmap v2240: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:37:40.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:37:40 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3720.714501 secs 2024-03-28T01:37:40.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:37:40 smithi165 ceph-mon[36302]: pgmap v2241: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:37:40.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:37:40 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3720.714501 secs 2024-03-28T01:37:40.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:37:40 smithi119 ceph-mon[31518]: pgmap v2241: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:37:40.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:37:40 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3720.714501 secs 2024-03-28T01:37:40.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:37:40 smithi132 ceph-mon[35941]: pgmap v2241: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:37:43.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:37:43 smithi165 ceph-mon[36302]: pgmap v2242: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:37:43.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:37:43 smithi119 ceph-mon[31518]: pgmap v2242: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:37:43.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:37:43 smithi132 ceph-mon[35941]: pgmap v2242: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:37:44.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:37:44 smithi165 ceph-mon[36302]: pgmap v2243: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:37:44.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:37:44 smithi119 ceph-mon[31518]: pgmap v2243: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:37:44.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:37:44 smithi132 ceph-mon[35941]: pgmap v2243: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:37:45.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:37:45 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3725.714592 secs 2024-03-28T01:37:45.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:37:45 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3725.714592 secs 2024-03-28T01:37:45.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:37:45 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3725.714592 secs 2024-03-28T01:37:46.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:37:46 smithi165 ceph-mon[36302]: pgmap v2244: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:37:46.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:37:46 smithi119 ceph-mon[31518]: pgmap v2244: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:37:46.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:37:46 smithi132 ceph-mon[35941]: pgmap v2244: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:37:48.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:37:48 smithi165 ceph-mon[36302]: pgmap v2245: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:37:48.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:37:48 smithi119 ceph-mon[31518]: pgmap v2245: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:37:48.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:37:48 smithi132 ceph-mon[35941]: pgmap v2245: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:37:50.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:37:50 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3730.714679 secs 2024-03-28T01:37:50.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:37:50 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3730.714679 secs 2024-03-28T01:37:50.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:37:50 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3730.714679 secs 2024-03-28T01:37:51.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:37:51 smithi132 ceph-mon[35941]: pgmap v2246: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:37:51.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:37:51 smithi165 ceph-mon[36302]: pgmap v2246: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:37:51.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:37:51 smithi119 ceph-mon[31518]: pgmap v2246: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:37:52.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:37:52 smithi165 ceph-mon[36302]: pgmap v2247: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:37:52.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:37:52 smithi119 ceph-mon[31518]: pgmap v2247: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:37:52.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:37:52 smithi132 ceph-mon[35941]: pgmap v2247: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:37:54.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:37:54 smithi165 ceph-mon[36302]: pgmap v2248: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:37:54.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:37:54 smithi119 ceph-mon[31518]: pgmap v2248: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:37:54.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:37:54 smithi132 ceph-mon[35941]: pgmap v2248: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:37:55.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:37:55 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3735.714766 secs 2024-03-28T01:37:55.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:37:55 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3735.714766 secs 2024-03-28T01:37:55.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:37:55 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3735.714766 secs 2024-03-28T01:37:56.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:37:56 smithi165 ceph-mon[36302]: pgmap v2249: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:37:56.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:37:56 smithi119 ceph-mon[31518]: pgmap v2249: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:37:56.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:37:56 smithi132 ceph-mon[35941]: pgmap v2249: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:37:58.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:37:58 smithi165 ceph-mon[36302]: pgmap v2250: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:37:58.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:37:58 smithi119 ceph-mon[31518]: pgmap v2250: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:37:58.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:37:58 smithi132 ceph-mon[35941]: pgmap v2250: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:38:00.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:38:00 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3740.714828 secs 2024-03-28T01:38:00.863 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:38:00 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3740.714828 secs 2024-03-28T01:38:00.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:38:00 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3740.714828 secs 2024-03-28T01:38:01.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:38:01 smithi132 ceph-mon[35941]: pgmap v2251: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:38:01.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:38:01 smithi165 ceph-mon[36302]: pgmap v2251: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:38:01.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:38:01 smithi119 ceph-mon[31518]: pgmap v2251: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:38:02.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:38:02 smithi165 ceph-mon[36302]: pgmap v2252: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:38:02.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:38:02 smithi119 ceph-mon[31518]: pgmap v2252: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:38:02.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:38:02 smithi132 ceph-mon[35941]: pgmap v2252: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:38:04.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:38:04 smithi165 ceph-mon[36302]: pgmap v2253: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:38:04.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:38:04 smithi119 ceph-mon[31518]: pgmap v2253: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:38:04.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:38:04 smithi132 ceph-mon[35941]: pgmap v2253: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:38:05.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:38:05 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3745.714903 secs 2024-03-28T01:38:05.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:38:05 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3745.714903 secs 2024-03-28T01:38:05.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:38:05 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3745.714903 secs 2024-03-28T01:38:06.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:38:06 smithi165 ceph-mon[36302]: pgmap v2254: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:38:06.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:38:06 smithi119 ceph-mon[31518]: pgmap v2254: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:38:06.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:38:06 smithi132 ceph-mon[35941]: pgmap v2254: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:38:08.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:38:08 smithi165 ceph-mon[36302]: pgmap v2255: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:38:08.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:38:08 smithi119 ceph-mon[31518]: pgmap v2255: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:38:08.945 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:38:08 smithi132 ceph-mon[35941]: pgmap v2255: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:38:10.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:38:10 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3750.714962 secs 2024-03-28T01:38:10.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:38:10 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3750.714962 secs 2024-03-28T01:38:10.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:38:10 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3750.714962 secs 2024-03-28T01:38:11.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:38:11 smithi132 ceph-mon[35941]: pgmap v2256: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:38:11.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:38:11 smithi165 ceph-mon[36302]: pgmap v2256: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:38:11.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:38:11 smithi119 ceph-mon[31518]: pgmap v2256: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:38:12.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:38:12 smithi165 ceph-mon[36302]: pgmap v2257: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:38:12.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:38:12 smithi119 ceph-mon[31518]: pgmap v2257: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:38:12.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:38:12 smithi132 ceph-mon[35941]: pgmap v2257: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:38:14.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:38:14 smithi165 ceph-mon[36302]: pgmap v2258: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:38:14.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:38:14 smithi119 ceph-mon[31518]: pgmap v2258: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:38:14.947 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:38:14 smithi132 ceph-mon[35941]: pgmap v2258: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:38:15.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:38:15 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3755.715023 secs 2024-03-28T01:38:15.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:38:15 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3755.715023 secs 2024-03-28T01:38:15.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:38:15 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3755.715023 secs 2024-03-28T01:38:16.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:38:16 smithi165 ceph-mon[36302]: pgmap v2259: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:38:16.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:38:16 smithi119 ceph-mon[31518]: pgmap v2259: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:38:16.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:38:16 smithi132 ceph-mon[35941]: pgmap v2259: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:38:18.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:38:18 smithi165 ceph-mon[36302]: pgmap v2260: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:38:18.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:38:18 smithi119 ceph-mon[31518]: pgmap v2260: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:38:18.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:38:18 smithi132 ceph-mon[35941]: pgmap v2260: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:38:20.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:38:20 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3760.715086 secs 2024-03-28T01:38:20.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:38:20 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3760.715086 secs 2024-03-28T01:38:20.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:38:20 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3760.715086 secs 2024-03-28T01:38:21.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:38:21 smithi132 ceph-mon[35941]: pgmap v2261: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:38:21.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:38:21 smithi165 ceph-mon[36302]: pgmap v2261: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:38:21.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:38:21 smithi119 ceph-mon[31518]: pgmap v2261: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:38:22.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:38:22 smithi165 ceph-mon[36302]: pgmap v2262: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:38:22.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:38:22 smithi119 ceph-mon[31518]: pgmap v2262: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:38:22.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:38:22 smithi132 ceph-mon[35941]: pgmap v2262: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:38:24.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:38:24 smithi165 ceph-mon[36302]: pgmap v2263: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:38:24.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:38:24 smithi119 ceph-mon[31518]: pgmap v2263: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:38:24.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:38:24 smithi132 ceph-mon[35941]: pgmap v2263: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:38:25.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:38:25 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3765.715148 secs 2024-03-28T01:38:25.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:38:25 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3765.715148 secs 2024-03-28T01:38:25.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:38:25 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3765.715148 secs 2024-03-28T01:38:26.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:38:26 smithi165 ceph-mon[36302]: pgmap v2264: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:38:26.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:38:26 smithi119 ceph-mon[31518]: pgmap v2264: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:38:26.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:38:26 smithi132 ceph-mon[35941]: pgmap v2264: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:38:27.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:38:27 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:38:27.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:38:27 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:38:27.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:38:27 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:38:28.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:38:28 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:38:28.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:38:28 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:38:28.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:38:28 smithi165 ceph-mon[36302]: pgmap v2265: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T01:38:28.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:38:28 smithi165 ceph-mon[36302]: pgmap v2266: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:38:28.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:38:28 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:38:28.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:38:28 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:38:28.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:38:28 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:38:28.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:38:28 smithi119 ceph-mon[31518]: pgmap v2265: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T01:38:28.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:38:28 smithi119 ceph-mon[31518]: pgmap v2266: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:38:28.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:38:28 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:38:28.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:38:28 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:38:28.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:38:28 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:38:28.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:38:28 smithi132 ceph-mon[35941]: pgmap v2265: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T01:38:28.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:38:28 smithi132 ceph-mon[35941]: pgmap v2266: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:38:28.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:38:28 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:38:31.375 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:38:31 smithi165 ceph-mon[36302]: pgmap v2267: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:38:31.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:38:31 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3770.715209 secs 2024-03-28T01:38:31.398 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:38:31 smithi119 ceph-mon[31518]: pgmap v2267: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:38:31.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:38:31 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3770.715209 secs 2024-03-28T01:38:31.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:38:31 smithi132 ceph-mon[35941]: pgmap v2267: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:38:31.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:38:31 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3770.715209 secs 2024-03-28T01:38:33.377 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:38:33 smithi165 ceph-mon[36302]: pgmap v2268: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:38:33.398 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:38:33 smithi119 ceph-mon[31518]: pgmap v2268: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:38:33.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:38:33 smithi132 ceph-mon[35941]: pgmap v2268: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:38:35.375 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:38:35 smithi165 ceph-mon[36302]: pgmap v2269: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-28T01:38:35.398 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:38:35 smithi119 ceph-mon[31518]: pgmap v2269: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-28T01:38:35.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:38:35 smithi132 ceph-mon[35941]: pgmap v2269: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-28T01:38:36.375 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:38:36 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3775.715268 secs 2024-03-28T01:38:36.398 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:38:36 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3775.715268 secs 2024-03-28T01:38:36.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:38:36 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3775.715268 secs 2024-03-28T01:38:37.375 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:38:37 smithi165 ceph-mon[36302]: pgmap v2270: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:38:37.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:38:37 smithi119 ceph-mon[31518]: pgmap v2270: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:38:37.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:38:37 smithi132 ceph-mon[35941]: pgmap v2270: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:38:39.375 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:38:39 smithi165 ceph-mon[36302]: pgmap v2271: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T01:38:39.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:38:39 smithi119 ceph-mon[31518]: pgmap v2271: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T01:38:39.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:38:39 smithi132 ceph-mon[35941]: pgmap v2271: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T01:38:41.375 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:38:41 smithi165 ceph-mon[36302]: pgmap v2272: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:38:41.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:38:41 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3780.715308 secs 2024-03-28T01:38:41.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:38:41 smithi119 ceph-mon[31518]: pgmap v2272: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:38:41.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:38:41 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3780.715308 secs 2024-03-28T01:38:41.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:38:41 smithi132 ceph-mon[35941]: pgmap v2272: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:38:41.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:38:41 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3780.715308 secs 2024-03-28T01:38:43.375 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:38:43 smithi165 ceph-mon[36302]: pgmap v2273: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:38:43.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:38:43 smithi119 ceph-mon[31518]: pgmap v2273: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:38:43.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:38:43 smithi132 ceph-mon[35941]: pgmap v2273: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:38:45.375 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:38:45 smithi165 ceph-mon[36302]: pgmap v2274: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:38:45.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:38:45 smithi119 ceph-mon[31518]: pgmap v2274: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:38:45.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:38:45 smithi132 ceph-mon[35941]: pgmap v2274: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:38:46.375 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:38:46 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3785.715368 secs 2024-03-28T01:38:46.398 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:38:46 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3785.715368 secs 2024-03-28T01:38:46.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:38:46 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3785.715368 secs 2024-03-28T01:38:47.375 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:38:47 smithi165 ceph-mon[36302]: pgmap v2275: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:38:47.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:38:47 smithi119 ceph-mon[31518]: pgmap v2275: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:38:47.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:38:47 smithi132 ceph-mon[35941]: pgmap v2275: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:38:49.375 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:38:49 smithi165 ceph-mon[36302]: pgmap v2276: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:38:49.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:38:49 smithi119 ceph-mon[31518]: pgmap v2276: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:38:49.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:38:49 smithi132 ceph-mon[35941]: pgmap v2276: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:38:51.375 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:38:51 smithi165 ceph-mon[36302]: pgmap v2277: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:38:51.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:38:51 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3790.715451 secs 2024-03-28T01:38:51.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:38:51 smithi119 ceph-mon[31518]: pgmap v2277: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:38:51.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:38:51 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3790.715451 secs 2024-03-28T01:38:51.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:38:51 smithi132 ceph-mon[35941]: pgmap v2277: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:38:51.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:38:51 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3790.715451 secs 2024-03-28T01:38:53.375 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:38:53 smithi165 ceph-mon[36302]: pgmap v2278: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:38:53.398 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:38:53 smithi119 ceph-mon[31518]: pgmap v2278: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:38:53.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:38:53 smithi132 ceph-mon[35941]: pgmap v2278: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:38:55.375 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:38:55 smithi165 ceph-mon[36302]: pgmap v2279: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:38:55.398 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:38:55 smithi119 ceph-mon[31518]: pgmap v2279: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:38:55.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:38:55 smithi132 ceph-mon[35941]: pgmap v2279: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:38:56.375 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:38:56 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3795.715541 secs 2024-03-28T01:38:56.398 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:38:56 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3795.715541 secs 2024-03-28T01:38:56.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:38:56 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3795.715541 secs 2024-03-28T01:38:57.375 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:38:57 smithi165 ceph-mon[36302]: pgmap v2280: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:38:57.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:38:57 smithi119 ceph-mon[31518]: pgmap v2280: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:38:57.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:38:57 smithi132 ceph-mon[35941]: pgmap v2280: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:38:59.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:38:59 smithi119 ceph-mon[31518]: pgmap v2281: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:38:59.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:38:59 smithi132 ceph-mon[35941]: pgmap v2281: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:38:59.625 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:38:59 smithi165 ceph-mon[36302]: pgmap v2281: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:39:01.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:39:01 smithi119 ceph-mon[31518]: pgmap v2282: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:39:01.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:39:01 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3800.715634 secs 2024-03-28T01:39:01.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:39:01 smithi132 ceph-mon[35941]: pgmap v2282: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:39:01.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:39:01 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3800.715634 secs 2024-03-28T01:39:01.625 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:39:01 smithi165 ceph-mon[36302]: pgmap v2282: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:39:01.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:39:01 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3800.715634 secs 2024-03-28T01:39:03.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:39:03 smithi119 ceph-mon[31518]: pgmap v2283: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:39:03.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:39:03 smithi132 ceph-mon[35941]: pgmap v2283: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:39:03.625 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:39:03 smithi165 ceph-mon[36302]: pgmap v2283: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:39:05.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:39:05 smithi119 ceph-mon[31518]: pgmap v2284: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:39:05.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:39:05 smithi132 ceph-mon[35941]: pgmap v2284: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:39:05.625 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:39:05 smithi165 ceph-mon[36302]: pgmap v2284: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:39:06.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:39:06 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3805.715723 secs 2024-03-28T01:39:06.625 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:39:06 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3805.715723 secs 2024-03-28T01:39:06.648 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:39:06 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3805.715723 secs 2024-03-28T01:39:07.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:39:07 smithi132 ceph-mon[35941]: pgmap v2285: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:39:07.625 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:39:07 smithi165 ceph-mon[36302]: pgmap v2285: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:39:07.648 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:39:07 smithi119 ceph-mon[31518]: pgmap v2285: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:39:09.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:39:09 smithi132 ceph-mon[35941]: pgmap v2286: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:39:09.625 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:39:09 smithi165 ceph-mon[36302]: pgmap v2286: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:39:09.648 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:39:09 smithi119 ceph-mon[31518]: pgmap v2286: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:39:11.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:39:11 smithi132 ceph-mon[35941]: pgmap v2287: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:39:11.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:39:11 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3810.715784 secs 2024-03-28T01:39:11.625 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:39:11 smithi165 ceph-mon[36302]: pgmap v2287: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:39:11.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:39:11 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3810.715784 secs 2024-03-28T01:39:11.648 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:39:11 smithi119 ceph-mon[31518]: pgmap v2287: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:39:11.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:39:11 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3810.715784 secs 2024-03-28T01:39:13.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:39:13 smithi132 ceph-mon[35941]: pgmap v2288: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:39:13.625 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:39:13 smithi165 ceph-mon[36302]: pgmap v2288: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:39:13.648 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:39:13 smithi119 ceph-mon[31518]: pgmap v2288: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:39:15.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:39:15 smithi132 ceph-mon[35941]: pgmap v2289: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:39:15.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:39:15 smithi165 ceph-mon[36302]: pgmap v2289: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:39:15.648 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:39:15 smithi119 ceph-mon[31518]: pgmap v2289: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:39:16.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:39:16 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3815.715870 secs 2024-03-28T01:39:16.550 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:39:16 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3815.715870 secs 2024-03-28T01:39:16.625 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:39:16 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3815.715870 secs 2024-03-28T01:39:17.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:39:17 smithi132 ceph-mon[35941]: pgmap v2290: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:39:17.625 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:39:17 smithi165 ceph-mon[36302]: pgmap v2290: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:39:17.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:39:17 smithi119 ceph-mon[31518]: pgmap v2290: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:39:19.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:39:19 smithi132 ceph-mon[35941]: pgmap v2291: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:39:19.452 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:39:19 smithi119 ceph-mon[31518]: pgmap v2291: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:39:19.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:39:19 smithi165 ceph-mon[36302]: pgmap v2291: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:39:21.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:39:21 smithi165 ceph-mon[36302]: pgmap v2292: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:39:21.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:39:21 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3820.715934 secs 2024-03-28T01:39:21.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:39:21 smithi119 ceph-mon[31518]: pgmap v2292: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:39:21.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:39:21 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3820.715934 secs 2024-03-28T01:39:21.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:39:21 smithi132 ceph-mon[35941]: pgmap v2292: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:39:21.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:39:21 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3820.715934 secs 2024-03-28T01:39:23.625 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:39:23 smithi165 ceph-mon[36302]: pgmap v2293: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:39:23.648 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:39:23 smithi119 ceph-mon[31518]: pgmap v2293: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:39:23.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:39:23 smithi132 ceph-mon[35941]: pgmap v2293: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:39:25.625 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:39:25 smithi165 ceph-mon[36302]: pgmap v2294: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:39:25.648 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:39:25 smithi119 ceph-mon[31518]: pgmap v2294: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:39:25.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:39:25 smithi132 ceph-mon[35941]: pgmap v2294: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:39:26.625 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:39:26 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3825.716020 secs 2024-03-28T01:39:26.648 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:39:26 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3825.716020 secs 2024-03-28T01:39:26.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:39:26 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3825.716020 secs 2024-03-28T01:39:27.625 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:39:27 smithi165 ceph-mon[36302]: pgmap v2295: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:39:27.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:39:27 smithi119 ceph-mon[31518]: pgmap v2295: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:39:27.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:39:27 smithi132 ceph-mon[35941]: pgmap v2295: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:39:29.625 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:39:29 smithi165 ceph-mon[36302]: pgmap v2296: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:39:29.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:39:29 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:39:29.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:39:29 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:39:29.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:39:29 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:39:29.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:39:29 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:39:29.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:39:29 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:39:29.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:39:29 smithi119 ceph-mon[31518]: pgmap v2296: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:39:29.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:39:29 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:39:29.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:39:29 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:39:29.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:39:29 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:39:29.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:39:29 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:39:29.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:39:29 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:39:29.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:39:29 smithi132 ceph-mon[35941]: pgmap v2296: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:39:29.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:39:29 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:39:29.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:39:29 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:39:29.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:39:29 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:39:29.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:39:29 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:39:29.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:39:29 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:39:31.625 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:39:31 smithi165 ceph-mon[36302]: pgmap v2297: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:39:31.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:39:31 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3830.716111 secs 2024-03-28T01:39:31.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:39:31 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:39:31.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:39:31 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:39:31.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:39:31 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:39:31.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:39:31 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:39:31.648 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:39:31 smithi119 ceph-mon[31518]: pgmap v2297: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:39:31.648 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:39:31 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3830.716111 secs 2024-03-28T01:39:31.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:39:31 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:39:31.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:39:31 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:39:31.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:39:31 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:39:31.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:39:31 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:39:31.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:39:31 smithi132 ceph-mon[35941]: pgmap v2297: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:39:31.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:39:31 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3830.716111 secs 2024-03-28T01:39:31.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:39:31 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:39:31.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:39:31 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:39:31.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:39:31 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:39:31.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:39:31 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:39:32.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:39:32 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:39:32.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:39:32 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:39:32.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:39:32 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:39:32.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:39:32 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:39:32.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:39:32 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:39:32.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:39:32 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:39:32.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:39:32 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:39:32.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:39:32 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:39:32.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:39:32 smithi165 ceph-mon[36302]: Adjusting osd_memory_target on smithi132 to 174.1M 2024-03-28T01:39:32.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:39:32 smithi165 ceph-mon[36302]: Unable to set osd_memory_target on smithi132 to 182640640: error parsing value: Value '182640640' is below minimum 939524096 2024-03-28T01:39:32.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:39:32 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:39:32.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:39:32 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:39:32.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:39:32 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:39:32.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:39:32 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:39:32.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:39:32 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:39:32.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:39:32 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:39:32.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:39:32 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:39:32.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:39:32 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:39:32.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:39:32 smithi165 ceph-mon[36302]: Adjusting osd_memory_target on smithi165 to 174.1M 2024-03-28T01:39:32.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:39:32 smithi165 ceph-mon[36302]: Unable to set osd_memory_target on smithi165 to 182635622: error parsing value: Value '182635622' is below minimum 939524096 2024-03-28T01:39:32.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:39:32 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:39:32.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:39:32 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:39:32.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:39:32 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:39:32.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:39:32 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:39:32.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:39:32 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:39:32.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:39:32 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:39:32.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:39:32 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:39:32.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:39:32 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:39:32.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:39:32 smithi119 ceph-mon[31518]: Adjusting osd_memory_target on smithi132 to 174.1M 2024-03-28T01:39:32.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:39:32 smithi119 ceph-mon[31518]: Unable to set osd_memory_target on smithi132 to 182640640: error parsing value: Value '182640640' is below minimum 939524096 2024-03-28T01:39:32.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:39:32 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:39:32.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:39:32 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:39:32.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:39:32 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:39:32.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:39:32 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:39:32.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:39:32 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:39:32.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:39:32 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:39:32.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:39:32 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:39:32.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:39:32 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:39:32.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:39:32 smithi119 ceph-mon[31518]: Adjusting osd_memory_target on smithi165 to 174.1M 2024-03-28T01:39:32.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:39:32 smithi119 ceph-mon[31518]: Unable to set osd_memory_target on smithi165 to 182635622: error parsing value: Value '182635622' is below minimum 939524096 2024-03-28T01:39:32.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:39:32 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:39:32.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:39:32 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:39:32.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:39:32 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:39:32.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:39:32 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:39:32.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:39:32 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:39:32.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:39:32 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:39:32.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:39:32 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:39:32.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:39:32 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:39:32.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:39:32 smithi132 ceph-mon[35941]: Adjusting osd_memory_target on smithi132 to 174.1M 2024-03-28T01:39:32.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:39:32 smithi132 ceph-mon[35941]: Unable to set osd_memory_target on smithi132 to 182640640: error parsing value: Value '182640640' is below minimum 939524096 2024-03-28T01:39:32.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:39:32 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:39:32.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:39:32 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:39:32.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:39:32 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:39:32.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:39:32 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:39:32.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:39:32 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:39:32.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:39:32 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:39:32.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:39:32 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:39:32.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:39:32 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:39:32.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:39:32 smithi132 ceph-mon[35941]: Adjusting osd_memory_target on smithi165 to 174.1M 2024-03-28T01:39:32.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:39:32 smithi132 ceph-mon[35941]: Unable to set osd_memory_target on smithi165 to 182635622: error parsing value: Value '182635622' is below minimum 939524096 2024-03-28T01:39:33.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:39:33 smithi165 ceph-mon[36302]: pgmap v2298: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:39:33.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:39:33 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:39:33.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:39:33 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:39:33.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:39:33 smithi119 ceph-mon[31518]: pgmap v2298: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:39:33.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:39:33 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:39:33.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:39:33 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:39:33.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:39:33 smithi132 ceph-mon[35941]: pgmap v2298: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:39:33.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:39:33 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:39:33.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:39:33 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:39:34.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:39:34 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:39:34.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:39:34 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:39:34.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:39:34 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:39:34.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:39:34 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:39:34.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:39:34 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:39:34.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:39:34 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:39:34.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:39:34 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:39:34.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:39:34 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:39:34.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:39:34 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:39:34.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:39:34 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:39:34.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:39:34 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:39:34.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:39:34 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:39:34.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:39:34 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:39:34.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:39:34 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:39:34.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:39:34 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:39:34.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:39:34 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:39:34.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:39:34 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:39:34.900 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:39:34 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:39:34.900 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:39:34 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:39:34.900 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:39:34 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:39:34.900 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:39:34 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:39:34.900 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:39:34 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:39:34.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:39:34 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:39:34.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:39:34 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:39:34.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:39:34 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:39:34.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:39:34 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:39:34.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:39:34 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:39:34.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:39:34 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:39:34.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:39:34 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:39:34.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:39:34 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:39:34.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:39:34 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:39:34.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:39:34 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:39:34.947 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:39:34 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:39:35.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:39:35 smithi165 ceph-mon[36302]: Adjusting osd_memory_target on smithi119 to 176.7M 2024-03-28T01:39:35.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:39:35 smithi165 ceph-mon[36302]: Unable to set osd_memory_target on smithi119 to 185372364: error parsing value: Value '185372364' is below minimum 939524096 2024-03-28T01:39:35.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:39:35 smithi165 ceph-mon[36302]: pgmap v2299: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T01:39:35.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:39:35 smithi165 ceph-mon[36302]: pgmap v2300: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:39:35.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:39:35 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3835.716202 secs 2024-03-28T01:39:35.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:39:35 smithi119 ceph-mon[31518]: Adjusting osd_memory_target on smithi119 to 176.7M 2024-03-28T01:39:35.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:39:35 smithi119 ceph-mon[31518]: Unable to set osd_memory_target on smithi119 to 185372364: error parsing value: Value '185372364' is below minimum 939524096 2024-03-28T01:39:35.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:39:35 smithi119 ceph-mon[31518]: pgmap v2299: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T01:39:35.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:39:35 smithi119 ceph-mon[31518]: pgmap v2300: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:39:35.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:39:35 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3835.716202 secs 2024-03-28T01:39:35.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:39:35 smithi132 ceph-mon[35941]: Adjusting osd_memory_target on smithi119 to 176.7M 2024-03-28T01:39:35.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:39:35 smithi132 ceph-mon[35941]: Unable to set osd_memory_target on smithi119 to 185372364: error parsing value: Value '185372364' is below minimum 939524096 2024-03-28T01:39:35.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:39:35 smithi132 ceph-mon[35941]: pgmap v2299: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T01:39:35.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:39:35 smithi132 ceph-mon[35941]: pgmap v2300: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:39:35.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:39:35 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3835.716202 secs 2024-03-28T01:39:36.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:39:36 smithi165 ceph-mon[36302]: pgmap v2301: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:39:36.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:39:36 smithi119 ceph-mon[31518]: pgmap v2301: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:39:36.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:39:36 smithi132 ceph-mon[35941]: pgmap v2301: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:39:39.149 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:39:38 smithi119 ceph-mon[31518]: pgmap v2302: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:39:39.196 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:39:38 smithi132 ceph-mon[35941]: pgmap v2302: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:39:39.375 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:39:38 smithi165 ceph-mon[36302]: pgmap v2302: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:39:41.149 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:39:40 smithi119 ceph-mon[31518]: pgmap v2303: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:39:41.149 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:39:40 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3840.716290 secs 2024-03-28T01:39:41.196 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:39:40 smithi132 ceph-mon[35941]: pgmap v2303: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:39:41.196 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:39:40 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3840.716290 secs 2024-03-28T01:39:41.375 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:39:40 smithi165 ceph-mon[36302]: pgmap v2303: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:39:41.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:39:40 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3840.716290 secs 2024-03-28T01:39:43.149 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:39:42 smithi119 ceph-mon[31518]: pgmap v2304: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:39:43.196 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:39:42 smithi132 ceph-mon[35941]: pgmap v2304: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:39:43.375 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:39:42 smithi165 ceph-mon[36302]: pgmap v2304: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:39:45.149 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:39:44 smithi119 ceph-mon[31518]: pgmap v2305: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T01:39:45.195 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:39:44 smithi132 ceph-mon[35941]: pgmap v2305: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T01:39:45.375 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:39:44 smithi165 ceph-mon[36302]: pgmap v2305: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T01:39:46.196 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:39:45 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3845.716378 secs 2024-03-28T01:39:46.375 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:39:45 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3845.716378 secs 2024-03-28T01:39:46.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:39:45 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3845.716378 secs 2024-03-28T01:39:47.196 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:39:46 smithi132 ceph-mon[35941]: pgmap v2306: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:39:47.375 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:39:46 smithi165 ceph-mon[36302]: pgmap v2306: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:39:47.398 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:39:46 smithi119 ceph-mon[31518]: pgmap v2306: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:39:49.177 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:39:48 smithi165 ceph-mon[36302]: pgmap v2307: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T01:39:49.196 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:39:48 smithi132 ceph-mon[35941]: pgmap v2307: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T01:39:49.398 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:39:48 smithi119 ceph-mon[31518]: pgmap v2307: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T01:39:51.196 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:39:50 smithi132 ceph-mon[35941]: pgmap v2308: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:39:51.196 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:39:50 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3850.716465 secs 2024-03-28T01:39:51.375 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:39:50 smithi165 ceph-mon[36302]: pgmap v2308: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:39:51.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:39:50 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3850.716465 secs 2024-03-28T01:39:51.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:39:50 smithi119 ceph-mon[31518]: pgmap v2308: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:39:51.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:39:50 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3850.716465 secs 2024-03-28T01:39:53.196 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:39:52 smithi132 ceph-mon[35941]: pgmap v2309: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T01:39:53.375 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:39:52 smithi165 ceph-mon[36302]: pgmap v2309: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T01:39:53.398 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:39:52 smithi119 ceph-mon[31518]: pgmap v2309: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T01:39:55.196 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:39:54 smithi132 ceph-mon[35941]: pgmap v2310: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T01:39:55.375 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:39:54 smithi165 ceph-mon[36302]: pgmap v2310: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T01:39:55.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:39:54 smithi119 ceph-mon[31518]: pgmap v2310: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T01:39:56.196 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:39:55 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3855.716504 secs 2024-03-28T01:39:56.375 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:39:55 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3855.716504 secs 2024-03-28T01:39:56.398 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:39:55 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3855.716504 secs 2024-03-28T01:39:57.196 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:39:56 smithi132 ceph-mon[35941]: pgmap v2311: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:39:57.375 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:39:56 smithi165 ceph-mon[36302]: pgmap v2311: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:39:57.398 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:39:56 smithi119 ceph-mon[31518]: pgmap v2311: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:39:59.196 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:39:58 smithi132 ceph-mon[35941]: pgmap v2312: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:39:59.375 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:39:58 smithi165 ceph-mon[36302]: pgmap v2312: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:39:59.398 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:39:58 smithi119 ceph-mon[31518]: pgmap v2312: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:40:01.375 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:40:00 smithi165 ceph-mon[36302]: pgmap v2313: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:40:01.375 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:40:00 smithi165 ceph-mon[36302]: overall HEALTH_WARN 5 failed cephadm daemon(s); 1 filesystem is degraded; 1 MDSs report slow requests 2024-03-28T01:40:01.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:40:00 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3860.716592 secs 2024-03-28T01:40:01.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:40:00 smithi119 ceph-mon[31518]: pgmap v2313: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:40:01.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:40:00 smithi119 ceph-mon[31518]: overall HEALTH_WARN 5 failed cephadm daemon(s); 1 filesystem is degraded; 1 MDSs report slow requests 2024-03-28T01:40:01.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:40:00 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3860.716592 secs 2024-03-28T01:40:01.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:40:00 smithi132 ceph-mon[35941]: pgmap v2313: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:40:01.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:40:00 smithi132 ceph-mon[35941]: overall HEALTH_WARN 5 failed cephadm daemon(s); 1 filesystem is degraded; 1 MDSs report slow requests 2024-03-28T01:40:01.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:40:00 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3860.716592 secs 2024-03-28T01:40:03.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:40:03 smithi132 ceph-mon[35941]: pgmap v2314: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:40:03.625 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:40:03 smithi165 ceph-mon[36302]: pgmap v2314: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:40:03.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:40:03 smithi119 ceph-mon[31518]: pgmap v2314: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:40:05.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:40:05 smithi132 ceph-mon[35941]: pgmap v2315: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:40:05.625 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:40:05 smithi165 ceph-mon[36302]: pgmap v2315: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:40:05.648 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:40:05 smithi119 ceph-mon[31518]: pgmap v2315: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:40:06.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:40:06 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3865.716679 secs 2024-03-28T01:40:06.625 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:40:06 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3865.716679 secs 2024-03-28T01:40:06.648 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:40:06 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3865.716679 secs 2024-03-28T01:40:07.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:40:07 smithi132 ceph-mon[35941]: pgmap v2316: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:40:07.625 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:40:07 smithi165 ceph-mon[36302]: pgmap v2316: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:40:07.648 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:40:07 smithi119 ceph-mon[31518]: pgmap v2316: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:40:09.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:40:09 smithi132 ceph-mon[35941]: pgmap v2317: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:40:09.625 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:40:09 smithi165 ceph-mon[36302]: pgmap v2317: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:40:09.648 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:40:09 smithi119 ceph-mon[31518]: pgmap v2317: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:40:11.625 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:40:11 smithi165 ceph-mon[36302]: pgmap v2318: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:40:11.625 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:40:11 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3870.716719 secs 2024-03-28T01:40:11.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:40:11 smithi119 ceph-mon[31518]: pgmap v2318: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:40:11.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:40:11 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3870.716719 secs 2024-03-28T01:40:11.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:40:11 smithi132 ceph-mon[35941]: pgmap v2318: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:40:11.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:40:11 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3870.716719 secs 2024-03-28T01:40:13.625 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:40:13 smithi165 ceph-mon[36302]: pgmap v2319: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:40:13.648 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:40:13 smithi119 ceph-mon[31518]: pgmap v2319: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:40:13.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:40:13 smithi132 ceph-mon[35941]: pgmap v2319: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:40:15.625 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:40:15 smithi165 ceph-mon[36302]: pgmap v2320: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:40:15.648 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:40:15 smithi119 ceph-mon[31518]: pgmap v2320: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:40:15.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:40:15 smithi132 ceph-mon[35941]: pgmap v2320: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:40:16.551 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:40:16 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3875.716782 secs 2024-03-28T01:40:16.625 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:40:16 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3875.716782 secs 2024-03-28T01:40:16.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:40:16 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3875.716782 secs 2024-03-28T01:40:17.587 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:40:17 smithi119 ceph-mon[31518]: pgmap v2321: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:40:17.625 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:40:17 smithi165 ceph-mon[36302]: pgmap v2321: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:40:17.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:40:17 smithi132 ceph-mon[35941]: pgmap v2321: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:40:19.625 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:40:19 smithi165 ceph-mon[36302]: pgmap v2322: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:40:19.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:40:19 smithi119 ceph-mon[31518]: pgmap v2322: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:40:19.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:40:19 smithi132 ceph-mon[35941]: pgmap v2322: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:40:21.451 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:40:21 smithi119 ceph-mon[31518]: pgmap v2323: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:40:21.452 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:40:21 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3880.716843 secs 2024-03-28T01:40:21.498 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:40:21 smithi132 ceph-mon[35941]: pgmap v2323: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:40:21.498 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:40:21 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3880.716843 secs 2024-03-28T01:40:21.625 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:40:21 smithi165 ceph-mon[36302]: pgmap v2323: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:40:21.625 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:40:21 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3880.716843 secs 2024-03-28T01:40:23.625 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:40:23 smithi165 ceph-mon[36302]: pgmap v2324: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:40:23.648 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:40:23 smithi119 ceph-mon[31518]: pgmap v2324: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:40:23.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:40:23 smithi132 ceph-mon[35941]: pgmap v2324: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:40:25.625 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:40:25 smithi165 ceph-mon[36302]: pgmap v2325: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:40:25.648 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:40:25 smithi119 ceph-mon[31518]: pgmap v2325: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:40:25.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:40:25 smithi132 ceph-mon[35941]: pgmap v2325: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:40:26.625 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:40:26 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3885.716905 secs 2024-03-28T01:40:26.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:40:26 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3885.716905 secs 2024-03-28T01:40:26.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:40:26 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3885.716905 secs 2024-03-28T01:40:27.625 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:40:27 smithi165 ceph-mon[36302]: pgmap v2326: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:40:27.648 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:40:27 smithi119 ceph-mon[31518]: pgmap v2326: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:40:27.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:40:27 smithi132 ceph-mon[35941]: pgmap v2326: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:40:29.625 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:40:29 smithi165 ceph-mon[36302]: pgmap v2327: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:40:29.648 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:40:29 smithi119 ceph-mon[31518]: pgmap v2327: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:40:29.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:40:29 smithi132 ceph-mon[35941]: pgmap v2327: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:40:31.625 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:40:31 smithi165 ceph-mon[36302]: pgmap v2328: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:40:31.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:40:31 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3890.716992 secs 2024-03-28T01:40:31.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:40:31 smithi119 ceph-mon[31518]: pgmap v2328: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:40:31.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:40:31 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3890.716992 secs 2024-03-28T01:40:31.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:40:31 smithi132 ceph-mon[35941]: pgmap v2328: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:40:31.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:40:31 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3890.716992 secs 2024-03-28T01:40:33.625 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:40:33 smithi165 ceph-mon[36302]: pgmap v2329: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:40:33.648 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:40:33 smithi119 ceph-mon[31518]: pgmap v2329: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:40:33.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:40:33 smithi132 ceph-mon[35941]: pgmap v2329: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:40:34.625 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:40:34 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:40:34.648 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:40:34 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:40:34.695 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:40:34 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:40:35.625 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:40:35 smithi165 ceph-mon[36302]: pgmap v2330: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:40:35.625 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:40:35 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:40:35.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:40:35 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:40:35.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:40:35 smithi119 ceph-mon[31518]: pgmap v2330: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:40:35.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:40:35 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:40:35.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:40:35 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:40:35.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:40:35 smithi132 ceph-mon[35941]: pgmap v2330: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:40:35.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:40:35 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:40:35.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:40:35 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:40:36.625 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:40:36 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3895.717031 secs 2024-03-28T01:40:36.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:40:36 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:40:36.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:40:36 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:40:36.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:40:36 smithi165 ceph-mon[36302]: pgmap v2331: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T01:40:36.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:40:36 smithi165 ceph-mon[36302]: pgmap v2332: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:40:36.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:40:36 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:40:36.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:40:36 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3895.717031 secs 2024-03-28T01:40:36.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:40:36 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:40:36.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:40:36 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:40:36.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:40:36 smithi119 ceph-mon[31518]: pgmap v2331: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T01:40:36.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:40:36 smithi119 ceph-mon[31518]: pgmap v2332: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:40:36.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:40:36 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:40:36.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:40:36 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3895.717031 secs 2024-03-28T01:40:36.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:40:36 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:40:36.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:40:36 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:40:36.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:40:36 smithi132 ceph-mon[35941]: pgmap v2331: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T01:40:36.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:40:36 smithi132 ceph-mon[35941]: pgmap v2332: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:40:36.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:40:36 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:40:38.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:40:38 smithi165 ceph-mon[36302]: pgmap v2333: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:40:38.907 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:40:38 smithi119 ceph-mon[31518]: pgmap v2333: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:40:38.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:40:38 smithi132 ceph-mon[35941]: pgmap v2333: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:40:40.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:40:40 smithi165 ceph-mon[36302]: pgmap v2334: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:40:40.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:40:40 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3900.717121 secs 2024-03-28T01:40:40.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:40:40 smithi119 ceph-mon[31518]: pgmap v2334: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:40:40.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:40:40 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3900.717121 secs 2024-03-28T01:40:40.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:40:40 smithi132 ceph-mon[35941]: pgmap v2334: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:40:40.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:40:40 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3900.717121 secs 2024-03-28T01:40:42.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:40:42 smithi165 ceph-mon[36302]: pgmap v2335: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:40:42.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:40:42 smithi119 ceph-mon[31518]: pgmap v2335: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:40:42.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:40:42 smithi132 ceph-mon[35941]: pgmap v2335: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:40:44.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:40:44 smithi165 ceph-mon[36302]: pgmap v2336: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:40:44.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:40:44 smithi119 ceph-mon[31518]: pgmap v2336: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:40:44.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:40:44 smithi132 ceph-mon[35941]: pgmap v2336: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:40:45.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:40:45 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3905.717210 secs 2024-03-28T01:40:45.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:40:45 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3905.717210 secs 2024-03-28T01:40:45.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:40:45 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3905.717210 secs 2024-03-28T01:40:46.801 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:40:46 smithi119 ceph-mon[31518]: pgmap v2337: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T01:40:46.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:40:46 smithi165 ceph-mon[36302]: pgmap v2337: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T01:40:46.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:40:46 smithi132 ceph-mon[35941]: pgmap v2337: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T01:40:48.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:40:48 smithi165 ceph-mon[36302]: pgmap v2338: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:40:48.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:40:48 smithi119 ceph-mon[31518]: pgmap v2338: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:40:48.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:40:48 smithi132 ceph-mon[35941]: pgmap v2338: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:40:50.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:40:50 smithi165 ceph-mon[36302]: pgmap v2339: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:40:50.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:40:50 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3910.717299 secs 2024-03-28T01:40:50.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:40:50 smithi119 ceph-mon[31518]: pgmap v2339: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:40:50.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:40:50 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3910.717299 secs 2024-03-28T01:40:50.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:40:50 smithi132 ceph-mon[35941]: pgmap v2339: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:40:50.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:40:50 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3910.717299 secs 2024-03-28T01:40:52.747 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:40:52 smithi132 ceph-mon[35941]: pgmap v2340: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:40:52.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:40:52 smithi165 ceph-mon[36302]: pgmap v2340: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:40:52.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:40:52 smithi119 ceph-mon[31518]: pgmap v2340: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:40:54.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:40:54 smithi165 ceph-mon[36302]: pgmap v2341: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:40:54.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:40:54 smithi119 ceph-mon[31518]: pgmap v2341: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:40:54.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:40:54 smithi132 ceph-mon[35941]: pgmap v2341: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:40:55.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:40:55 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3915.717388 secs 2024-03-28T01:40:55.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:40:55 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3915.717388 secs 2024-03-28T01:40:55.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:40:55 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3915.717388 secs 2024-03-28T01:40:56.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:40:56 smithi165 ceph-mon[36302]: pgmap v2342: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:40:56.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:40:56 smithi119 ceph-mon[31518]: pgmap v2342: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:40:56.945 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:40:56 smithi132 ceph-mon[35941]: pgmap v2342: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:40:58.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:40:58 smithi165 ceph-mon[36302]: pgmap v2343: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:40:58.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:40:58 smithi119 ceph-mon[31518]: pgmap v2343: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:40:58.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:40:58 smithi132 ceph-mon[35941]: pgmap v2343: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:41:00.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:41:00 smithi165 ceph-mon[36302]: pgmap v2344: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:41:00.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:41:00 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3920.717449 secs 2024-03-28T01:41:00.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:41:00 smithi119 ceph-mon[31518]: pgmap v2344: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:41:00.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:41:00 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3920.717449 secs 2024-03-28T01:41:00.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:41:00 smithi132 ceph-mon[35941]: pgmap v2344: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:41:00.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:41:00 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3920.717449 secs 2024-03-28T01:41:02.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:41:02 smithi165 ceph-mon[36302]: pgmap v2345: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:41:02.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:41:02 smithi119 ceph-mon[31518]: pgmap v2345: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:41:02.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:41:02 smithi132 ceph-mon[35941]: pgmap v2345: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:41:04.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:41:04 smithi165 ceph-mon[36302]: pgmap v2346: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:41:04.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:41:04 smithi119 ceph-mon[31518]: pgmap v2346: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:41:04.945 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:41:04 smithi132 ceph-mon[35941]: pgmap v2346: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:41:05.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:41:05 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3925.717536 secs 2024-03-28T01:41:05.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:41:05 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3925.717536 secs 2024-03-28T01:41:05.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:41:05 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3925.717536 secs 2024-03-28T01:41:06.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:41:06 smithi165 ceph-mon[36302]: pgmap v2347: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:41:06.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:41:06 smithi119 ceph-mon[31518]: pgmap v2347: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:41:06.945 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:41:06 smithi132 ceph-mon[35941]: pgmap v2347: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:41:08.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:41:08 smithi165 ceph-mon[36302]: pgmap v2348: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:41:08.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:41:08 smithi119 ceph-mon[31518]: pgmap v2348: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:41:08.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:41:08 smithi132 ceph-mon[35941]: pgmap v2348: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:41:10.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:41:10 smithi165 ceph-mon[36302]: pgmap v2349: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:41:10.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:41:10 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3930.717622 secs 2024-03-28T01:41:10.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:41:10 smithi119 ceph-mon[31518]: pgmap v2349: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:41:10.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:41:10 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3930.717622 secs 2024-03-28T01:41:10.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:41:10 smithi132 ceph-mon[35941]: pgmap v2349: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:41:10.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:41:10 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3930.717622 secs 2024-03-28T01:41:12.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:41:12 smithi165 ceph-mon[36302]: pgmap v2350: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:41:12.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:41:12 smithi119 ceph-mon[31518]: pgmap v2350: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:41:12.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:41:12 smithi132 ceph-mon[35941]: pgmap v2350: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:41:14.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:41:14 smithi165 ceph-mon[36302]: pgmap v2351: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:41:14.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:41:14 smithi119 ceph-mon[31518]: pgmap v2351: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:41:14.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:41:14 smithi132 ceph-mon[35941]: pgmap v2351: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:41:15.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:41:15 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3935.717712 secs 2024-03-28T01:41:15.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:41:15 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3935.717712 secs 2024-03-28T01:41:15.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:41:15 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3935.717712 secs 2024-03-28T01:41:16.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:41:16 smithi165 ceph-mon[36302]: pgmap v2352: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:41:16.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:41:16 smithi119 ceph-mon[31518]: pgmap v2352: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:41:16.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:41:16 smithi132 ceph-mon[35941]: pgmap v2352: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:41:18.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:41:18 smithi165 ceph-mon[36302]: pgmap v2353: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:41:18.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:41:18 smithi119 ceph-mon[31518]: pgmap v2353: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:41:18.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:41:18 smithi132 ceph-mon[35941]: pgmap v2353: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:41:20.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:41:20 smithi165 ceph-mon[36302]: pgmap v2354: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:41:20.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:41:20 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3940.717802 secs 2024-03-28T01:41:20.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:41:20 smithi119 ceph-mon[31518]: pgmap v2354: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:41:20.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:41:20 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3940.717802 secs 2024-03-28T01:41:20.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:41:20 smithi132 ceph-mon[35941]: pgmap v2354: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:41:20.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:41:20 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3940.717802 secs 2024-03-28T01:41:21.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:41:21 smithi119 ceph-mon[31518]: pgmap v2355: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:41:21.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:41:21 smithi132 ceph-mon[35941]: pgmap v2355: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:41:22.125 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:41:21 smithi165 ceph-mon[36302]: pgmap v2355: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:41:24.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:41:24 smithi165 ceph-mon[36302]: pgmap v2356: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:41:24.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:41:24 smithi119 ceph-mon[31518]: pgmap v2356: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:41:24.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:41:24 smithi132 ceph-mon[35941]: pgmap v2356: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:41:25.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:41:25 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3945.717893 secs 2024-03-28T01:41:25.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:41:25 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3945.717893 secs 2024-03-28T01:41:25.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:41:25 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3945.717893 secs 2024-03-28T01:41:26.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:41:26 smithi165 ceph-mon[36302]: pgmap v2357: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:41:26.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:41:26 smithi119 ceph-mon[31518]: pgmap v2357: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:41:26.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:41:26 smithi132 ceph-mon[35941]: pgmap v2357: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:41:28.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:41:28 smithi165 ceph-mon[36302]: pgmap v2358: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:41:28.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:41:28 smithi119 ceph-mon[31518]: pgmap v2358: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:41:28.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:41:28 smithi132 ceph-mon[35941]: pgmap v2358: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:41:30.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:41:30 smithi165 ceph-mon[36302]: pgmap v2359: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:41:30.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:41:30 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3950.717956 secs 2024-03-28T01:41:30.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:41:30 smithi119 ceph-mon[31518]: pgmap v2359: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:41:30.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:41:30 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3950.717956 secs 2024-03-28T01:41:30.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:41:30 smithi132 ceph-mon[35941]: pgmap v2359: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:41:30.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:41:30 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3950.717956 secs 2024-03-28T01:41:32.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:41:32 smithi165 ceph-mon[36302]: pgmap v2360: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:41:32.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:41:32 smithi119 ceph-mon[31518]: pgmap v2360: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:41:32.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:41:32 smithi132 ceph-mon[35941]: pgmap v2360: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:41:34.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:41:34 smithi165 ceph-mon[36302]: pgmap v2361: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:41:34.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:41:34 smithi119 ceph-mon[31518]: pgmap v2361: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:41:34.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:41:34 smithi132 ceph-mon[35941]: pgmap v2361: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:41:35.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:41:35 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3955.718043 secs 2024-03-28T01:41:35.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:41:35 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3955.718043 secs 2024-03-28T01:41:35.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:41:35 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3955.718043 secs 2024-03-28T01:41:36.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:41:36 smithi165 ceph-mon[36302]: pgmap v2362: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:41:36.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:41:36 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:41:36.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:41:36 smithi119 ceph-mon[31518]: pgmap v2362: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:41:36.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:41:36 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:41:36.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:41:36 smithi132 ceph-mon[35941]: pgmap v2362: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:41:36.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:41:36 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:41:37.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:41:37 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:41:37.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:41:37 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:41:37.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:41:37 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:41:37.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:41:37 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:41:37.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:41:37 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:41:37.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:41:37 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:41:37.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:41:37 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:41:37.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:41:37 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:41:37.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:41:37 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:41:38.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:41:38 smithi165 ceph-mon[36302]: pgmap v2363: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T01:41:38.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:41:38 smithi165 ceph-mon[36302]: pgmap v2364: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:41:38.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:41:38 smithi119 ceph-mon[31518]: pgmap v2363: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T01:41:38.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:41:38 smithi119 ceph-mon[31518]: pgmap v2364: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:41:38.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:41:38 smithi132 ceph-mon[35941]: pgmap v2363: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T01:41:38.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:41:38 smithi132 ceph-mon[35941]: pgmap v2364: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:41:40.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:41:40 smithi165 ceph-mon[36302]: pgmap v2365: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-28T01:41:40.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:41:40 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3960.718082 secs 2024-03-28T01:41:40.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:41:40 smithi119 ceph-mon[31518]: pgmap v2365: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-28T01:41:40.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:41:40 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3960.718082 secs 2024-03-28T01:41:40.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:41:40 smithi132 ceph-mon[35941]: pgmap v2365: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-28T01:41:40.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:41:40 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3960.718082 secs 2024-03-28T01:41:42.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:41:42 smithi165 ceph-mon[36302]: pgmap v2366: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:41:42.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:41:42 smithi119 ceph-mon[31518]: pgmap v2366: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:41:42.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:41:42 smithi132 ceph-mon[35941]: pgmap v2366: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:41:44.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:41:44 smithi165 ceph-mon[36302]: pgmap v2367: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:41:44.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:41:44 smithi119 ceph-mon[31518]: pgmap v2367: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:41:44.945 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:41:44 smithi132 ceph-mon[35941]: pgmap v2367: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:41:45.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:41:45 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3965.718144 secs 2024-03-28T01:41:45.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:41:45 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3965.718144 secs 2024-03-28T01:41:45.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:41:45 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3965.718144 secs 2024-03-28T01:41:46.803 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:41:46 smithi119 ceph-mon[31518]: pgmap v2368: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:41:46.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:41:46 smithi165 ceph-mon[36302]: pgmap v2368: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:41:46.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:41:46 smithi132 ceph-mon[35941]: pgmap v2368: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:41:48.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:41:48 smithi165 ceph-mon[36302]: pgmap v2369: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T01:41:48.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:41:48 smithi119 ceph-mon[31518]: pgmap v2369: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T01:41:48.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:41:48 smithi132 ceph-mon[35941]: pgmap v2369: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T01:41:50.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:41:50 smithi165 ceph-mon[36302]: pgmap v2370: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T01:41:50.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:41:50 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3970.718234 secs 2024-03-28T01:41:50.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:41:50 smithi119 ceph-mon[31518]: pgmap v2370: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T01:41:50.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:41:50 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3970.718234 secs 2024-03-28T01:41:50.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:41:50 smithi132 ceph-mon[35941]: pgmap v2370: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T01:41:50.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:41:50 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3970.718234 secs 2024-03-28T01:41:52.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:41:52 smithi165 ceph-mon[36302]: pgmap v2371: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:41:52.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:41:52 smithi119 ceph-mon[31518]: pgmap v2371: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:41:52.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:41:52 smithi132 ceph-mon[35941]: pgmap v2371: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:41:54.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:41:54 smithi119 ceph-mon[31518]: pgmap v2372: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:41:54.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:41:54 smithi165 ceph-mon[36302]: pgmap v2372: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:41:54.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:41:54 smithi132 ceph-mon[35941]: pgmap v2372: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:41:55.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:41:55 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3975.718292 secs 2024-03-28T01:41:55.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:41:55 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3975.718292 secs 2024-03-28T01:41:55.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:41:55 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3975.718292 secs 2024-03-28T01:41:56.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:41:56 smithi165 ceph-mon[36302]: pgmap v2373: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:41:56.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:41:56 smithi119 ceph-mon[31518]: pgmap v2373: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:41:56.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:41:56 smithi132 ceph-mon[35941]: pgmap v2373: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:41:58.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:41:58 smithi165 ceph-mon[36302]: pgmap v2374: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:41:58.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:41:58 smithi119 ceph-mon[31518]: pgmap v2374: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:41:58.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:41:58 smithi132 ceph-mon[35941]: pgmap v2374: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:41:59.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:41:59 smithi119 ceph-mon[31518]: pgmap v2375: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T01:41:59.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:41:59 smithi132 ceph-mon[35941]: pgmap v2375: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T01:42:00.125 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:41:59 smithi165 ceph-mon[36302]: pgmap v2375: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T01:42:00.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:42:00 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3980.718354 secs 2024-03-28T01:42:00.945 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:42:00 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3980.718354 secs 2024-03-28T01:42:01.125 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:42:00 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3980.718354 secs 2024-03-28T01:42:01.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:42:01 smithi132 ceph-mon[35941]: pgmap v2376: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-28T01:42:02.125 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:42:01 smithi165 ceph-mon[36302]: pgmap v2376: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-28T01:42:02.148 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:42:01 smithi119 ceph-mon[31518]: pgmap v2376: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-28T01:42:04.375 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:42:04 smithi165 ceph-mon[36302]: pgmap v2377: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T01:42:04.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:42:04 smithi119 ceph-mon[31518]: pgmap v2377: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T01:42:04.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:42:04 smithi132 ceph-mon[35941]: pgmap v2377: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T01:42:06.375 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:42:06 smithi165 ceph-mon[36302]: pgmap v2378: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:42:06.375 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:42:06 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3985.718440 secs 2024-03-28T01:42:06.398 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:42:06 smithi119 ceph-mon[31518]: pgmap v2378: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:42:06.398 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:42:06 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3985.718440 secs 2024-03-28T01:42:06.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:42:06 smithi132 ceph-mon[35941]: pgmap v2378: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:42:06.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:42:06 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3985.718440 secs 2024-03-28T01:42:08.375 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:42:08 smithi165 ceph-mon[36302]: pgmap v2379: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T01:42:08.398 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:42:08 smithi119 ceph-mon[31518]: pgmap v2379: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T01:42:08.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:42:08 smithi132 ceph-mon[35941]: pgmap v2379: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T01:42:10.375 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:42:10 smithi165 ceph-mon[36302]: pgmap v2380: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-03-28T01:42:10.398 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:42:10 smithi119 ceph-mon[31518]: pgmap v2380: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-03-28T01:42:10.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:42:10 smithi132 ceph-mon[35941]: pgmap v2380: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-03-28T01:42:11.375 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:42:11 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3990.718500 secs 2024-03-28T01:42:11.398 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:42:11 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3990.718500 secs 2024-03-28T01:42:11.445 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:42:11 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3990.718500 secs 2024-03-28T01:42:12.375 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:42:12 smithi165 ceph-mon[36302]: pgmap v2381: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-28T01:42:12.398 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:42:12 smithi119 ceph-mon[31518]: pgmap v2381: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-28T01:42:12.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:42:12 smithi132 ceph-mon[35941]: pgmap v2381: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-28T01:42:14.375 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:42:14 smithi165 ceph-mon[36302]: pgmap v2382: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:42:14.398 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:42:14 smithi119 ceph-mon[31518]: pgmap v2382: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:42:14.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:42:14 smithi132 ceph-mon[35941]: pgmap v2382: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:42:16.375 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:42:16 smithi165 ceph-mon[36302]: pgmap v2383: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:42:16.375 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:42:16 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 3995.718540 secs 2024-03-28T01:42:16.398 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:42:16 smithi119 ceph-mon[31518]: pgmap v2383: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:42:16.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:42:16 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 3995.718540 secs 2024-03-28T01:42:16.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:42:16 smithi132 ceph-mon[35941]: pgmap v2383: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:42:16.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:42:16 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 3995.718540 secs 2024-03-28T01:42:18.375 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:42:18 smithi165 ceph-mon[36302]: pgmap v2384: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:42:18.398 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:42:18 smithi119 ceph-mon[31518]: pgmap v2384: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:42:18.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:42:18 smithi132 ceph-mon[35941]: pgmap v2384: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:42:20.375 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:42:20 smithi165 ceph-mon[36302]: pgmap v2385: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T01:42:20.398 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:42:20 smithi119 ceph-mon[31518]: pgmap v2385: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T01:42:20.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:42:20 smithi132 ceph-mon[35941]: pgmap v2385: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T01:42:21.398 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:42:21 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4000.718630 secs 2024-03-28T01:42:21.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:42:21 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4000.718630 secs 2024-03-28T01:42:21.625 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:42:21 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4000.718630 secs 2024-03-28T01:42:22.398 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:42:22 smithi119 ceph-mon[31518]: pgmap v2386: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-28T01:42:22.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:42:22 smithi132 ceph-mon[35941]: pgmap v2386: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-28T01:42:22.625 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:42:22 smithi165 ceph-mon[36302]: pgmap v2386: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-28T01:42:24.398 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:42:24 smithi119 ceph-mon[31518]: pgmap v2387: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:42:24.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:42:24 smithi132 ceph-mon[35941]: pgmap v2387: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:42:24.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:42:24 smithi165 ceph-mon[36302]: pgmap v2387: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:42:26.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:42:26 smithi119 ceph-mon[31518]: pgmap v2388: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:42:26.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:42:26 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4005.718735 secs 2024-03-28T01:42:26.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:42:26 smithi132 ceph-mon[35941]: pgmap v2388: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:42:26.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:42:26 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4005.718735 secs 2024-03-28T01:42:26.625 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:42:26 smithi165 ceph-mon[36302]: pgmap v2388: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:42:26.625 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:42:26 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4005.718735 secs 2024-03-28T01:42:28.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:42:28 smithi132 ceph-mon[35941]: pgmap v2389: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:42:28.625 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:42:28 smithi165 ceph-mon[36302]: pgmap v2389: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:42:28.648 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:42:28 smithi119 ceph-mon[31518]: pgmap v2389: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:42:30.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:42:30 smithi132 ceph-mon[35941]: pgmap v2390: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T01:42:30.625 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:42:30 smithi165 ceph-mon[36302]: pgmap v2390: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T01:42:30.648 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:42:30 smithi119 ceph-mon[31518]: pgmap v2390: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T01:42:31.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:42:31 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4010.718825 secs 2024-03-28T01:42:31.625 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:42:31 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4010.718825 secs 2024-03-28T01:42:31.648 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:42:31 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4010.718825 secs 2024-03-28T01:42:32.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:42:32 smithi132 ceph-mon[35941]: pgmap v2391: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-28T01:42:32.625 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:42:32 smithi165 ceph-mon[36302]: pgmap v2391: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-28T01:42:32.648 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:42:32 smithi119 ceph-mon[31518]: pgmap v2391: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-28T01:42:34.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:42:34 smithi132 ceph-mon[35941]: pgmap v2392: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:42:34.625 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:42:34 smithi165 ceph-mon[36302]: pgmap v2392: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:42:34.648 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:42:34 smithi119 ceph-mon[31518]: pgmap v2392: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:42:36.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:42:36 smithi132 ceph-mon[35941]: pgmap v2393: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:42:36.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:42:36 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4015.718895 secs 2024-03-28T01:42:36.625 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:42:36 smithi165 ceph-mon[36302]: pgmap v2393: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:42:36.625 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:42:36 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4015.718895 secs 2024-03-28T01:42:36.648 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:42:36 smithi119 ceph-mon[31518]: pgmap v2393: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:42:36.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:42:36 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4015.718895 secs 2024-03-28T01:42:38.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:42:38 smithi132 ceph-mon[35941]: pgmap v2394: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:42:38.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:42:38 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:42:38.625 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:42:38 smithi165 ceph-mon[36302]: pgmap v2394: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:42:38.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:42:38 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:42:38.648 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:42:38 smithi119 ceph-mon[31518]: pgmap v2394: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:42:38.648 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:42:38 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:42:39.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:42:39 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:42:39.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:42:39 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:42:39.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:42:39 smithi132 ceph-mon[35941]: pgmap v2395: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T01:42:39.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:42:39 smithi132 ceph-mon[35941]: pgmap v2396: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-03-28T01:42:39.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:42:39 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:42:39.625 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:42:39 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:42:39.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:42:39 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:42:39.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:42:39 smithi165 ceph-mon[36302]: pgmap v2395: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T01:42:39.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:42:39 smithi165 ceph-mon[36302]: pgmap v2396: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-03-28T01:42:39.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:42:39 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:42:39.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:42:39 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:42:39.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:42:39 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:42:39.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:42:39 smithi119 ceph-mon[31518]: pgmap v2395: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T01:42:39.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:42:39 smithi119 ceph-mon[31518]: pgmap v2396: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-03-28T01:42:39.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:42:39 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:42:40.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:42:40 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4020.718957 secs 2024-03-28T01:42:40.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:42:40 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4020.718957 secs 2024-03-28T01:42:40.945 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:42:40 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4020.718957 secs 2024-03-28T01:42:41.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:42:41 smithi165 ceph-mon[36302]: pgmap v2397: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:42:41.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:42:41 smithi119 ceph-mon[31518]: pgmap v2397: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:42:41.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:42:41 smithi132 ceph-mon[35941]: pgmap v2397: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:42:42.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:42:42 smithi119 ceph-mon[31518]: pgmap v2398: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:42:42.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:42:42 smithi132 ceph-mon[35941]: pgmap v2398: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:42:43.125 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:42:42 smithi165 ceph-mon[36302]: pgmap v2398: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:42:44.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:42:44 smithi119 ceph-mon[31518]: pgmap v2399: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:42:44.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:42:44 smithi132 ceph-mon[35941]: pgmap v2399: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:42:45.125 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:42:44 smithi165 ceph-mon[36302]: pgmap v2399: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:42:45.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:42:45 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4025.719021 secs 2024-03-28T01:42:46.125 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:42:45 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4025.719021 secs 2024-03-28T01:42:46.148 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:42:45 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4025.719021 secs 2024-03-28T01:42:46.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:42:46 smithi132 ceph-mon[35941]: pgmap v2400: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:42:47.050 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:42:46 smithi119 ceph-mon[31518]: pgmap v2400: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:42:47.125 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:42:46 smithi165 ceph-mon[36302]: pgmap v2400: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:42:48.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:42:48 smithi119 ceph-mon[31518]: pgmap v2401: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T01:42:48.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:42:48 smithi132 ceph-mon[35941]: pgmap v2401: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T01:42:49.125 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:42:48 smithi165 ceph-mon[36302]: pgmap v2401: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T01:42:50.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:42:50 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4030.719037 secs 2024-03-28T01:42:50.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:42:50 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4030.719037 secs 2024-03-28T01:42:50.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:42:50 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4030.719037 secs 2024-03-28T01:42:51.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:42:51 smithi165 ceph-mon[36302]: pgmap v2402: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:42:51.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:42:51 smithi119 ceph-mon[31518]: pgmap v2402: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:42:51.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:42:51 smithi132 ceph-mon[35941]: pgmap v2402: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:42:52.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:42:52 smithi132 ceph-mon[35941]: pgmap v2403: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:42:53.125 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:42:52 smithi165 ceph-mon[36302]: pgmap v2403: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:42:53.148 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:42:52 smithi119 ceph-mon[31518]: pgmap v2403: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:42:54.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:42:54 smithi132 ceph-mon[35941]: pgmap v2404: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:42:55.125 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:42:54 smithi165 ceph-mon[36302]: pgmap v2404: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:42:55.148 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:42:54 smithi119 ceph-mon[31518]: pgmap v2404: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:42:55.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:42:55 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4035.719127 secs 2024-03-28T01:42:56.125 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:42:55 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4035.719127 secs 2024-03-28T01:42:56.148 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:42:55 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4035.719127 secs 2024-03-28T01:42:56.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:42:56 smithi132 ceph-mon[35941]: pgmap v2405: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:42:57.125 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:42:56 smithi165 ceph-mon[36302]: pgmap v2405: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:42:57.148 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:42:56 smithi119 ceph-mon[31518]: pgmap v2405: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:42:58.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:42:58 smithi132 ceph-mon[35941]: pgmap v2406: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:42:59.125 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:42:58 smithi165 ceph-mon[36302]: pgmap v2406: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:42:59.148 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:42:58 smithi119 ceph-mon[31518]: pgmap v2406: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:43:00.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:43:00 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4040.719215 secs 2024-03-28T01:43:00.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:43:00 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4040.719215 secs 2024-03-28T01:43:00.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:43:00 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4040.719215 secs 2024-03-28T01:43:01.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:43:01 smithi165 ceph-mon[36302]: pgmap v2407: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:43:01.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:43:01 smithi119 ceph-mon[31518]: pgmap v2407: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:43:01.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:43:01 smithi132 ceph-mon[35941]: pgmap v2407: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:43:03.625 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:43:03 smithi165 ceph-mon[36302]: pgmap v2408: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:43:03.648 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:43:03 smithi119 ceph-mon[31518]: pgmap v2408: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:43:03.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:43:03 smithi132 ceph-mon[35941]: pgmap v2408: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:43:04.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:43:04 smithi132 ceph-mon[35941]: pgmap v2409: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:43:05.125 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:43:04 smithi165 ceph-mon[36302]: pgmap v2409: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:43:05.148 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:43:04 smithi119 ceph-mon[31518]: pgmap v2409: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:43:05.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:43:05 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4045.719279 secs 2024-03-28T01:43:06.125 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:43:05 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4045.719279 secs 2024-03-28T01:43:06.148 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:43:05 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4045.719279 secs 2024-03-28T01:43:06.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:43:06 smithi132 ceph-mon[35941]: pgmap v2410: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:43:07.125 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:43:06 smithi165 ceph-mon[36302]: pgmap v2410: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:43:07.148 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:43:06 smithi119 ceph-mon[31518]: pgmap v2410: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:43:08.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:43:08 smithi132 ceph-mon[35941]: pgmap v2411: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:43:09.125 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:43:08 smithi165 ceph-mon[36302]: pgmap v2411: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:43:09.148 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:43:08 smithi119 ceph-mon[31518]: pgmap v2411: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:43:10.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:43:10 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4050.719341 secs 2024-03-28T01:43:10.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:43:10 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4050.719341 secs 2024-03-28T01:43:10.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:43:10 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4050.719341 secs 2024-03-28T01:43:11.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:43:11 smithi165 ceph-mon[36302]: pgmap v2412: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:43:11.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:43:11 smithi119 ceph-mon[31518]: pgmap v2412: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:43:11.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:43:11 smithi132 ceph-mon[35941]: pgmap v2412: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:43:12.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:43:12 smithi132 ceph-mon[35941]: pgmap v2413: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:43:13.125 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:43:12 smithi165 ceph-mon[36302]: pgmap v2413: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:43:13.148 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:43:12 smithi119 ceph-mon[31518]: pgmap v2413: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:43:14.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:43:14 smithi132 ceph-mon[35941]: pgmap v2414: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:43:15.125 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:43:14 smithi165 ceph-mon[36302]: pgmap v2414: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:43:15.148 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:43:14 smithi119 ceph-mon[31518]: pgmap v2414: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:43:15.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:43:15 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4055.719404 secs 2024-03-28T01:43:16.125 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:43:15 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4055.719404 secs 2024-03-28T01:43:16.148 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:43:15 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4055.719404 secs 2024-03-28T01:43:16.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:43:16 smithi132 ceph-mon[35941]: pgmap v2415: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:43:17.050 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:43:16 smithi119 ceph-mon[31518]: pgmap v2415: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:43:17.125 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:43:16 smithi165 ceph-mon[36302]: pgmap v2415: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:43:18.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:43:18 smithi132 ceph-mon[35941]: pgmap v2416: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:43:19.125 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:43:18 smithi165 ceph-mon[36302]: pgmap v2416: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:43:19.148 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:43:18 smithi119 ceph-mon[31518]: pgmap v2416: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:43:20.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:43:20 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4060.719465 secs 2024-03-28T01:43:20.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:43:20 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4060.719465 secs 2024-03-28T01:43:20.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:43:20 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4060.719465 secs 2024-03-28T01:43:21.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:43:21 smithi165 ceph-mon[36302]: pgmap v2417: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:43:21.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:43:21 smithi119 ceph-mon[31518]: pgmap v2417: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:43:21.945 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:43:21 smithi132 ceph-mon[35941]: pgmap v2417: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:43:22.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:43:22 smithi132 ceph-mon[35941]: pgmap v2418: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:43:23.125 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:43:22 smithi165 ceph-mon[36302]: pgmap v2418: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:43:23.148 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:43:22 smithi119 ceph-mon[31518]: pgmap v2418: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:43:24.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:43:24 smithi132 ceph-mon[35941]: pgmap v2419: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:43:25.125 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:43:24 smithi165 ceph-mon[36302]: pgmap v2419: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:43:25.148 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:43:24 smithi119 ceph-mon[31518]: pgmap v2419: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:43:25.945 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:43:25 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4065.719556 secs 2024-03-28T01:43:26.125 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:43:25 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4065.719556 secs 2024-03-28T01:43:26.148 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:43:25 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4065.719556 secs 2024-03-28T01:43:26.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:43:26 smithi132 ceph-mon[35941]: pgmap v2420: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:43:27.125 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:43:26 smithi165 ceph-mon[36302]: pgmap v2420: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:43:27.148 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:43:26 smithi119 ceph-mon[31518]: pgmap v2420: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:43:28.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:43:28 smithi132 ceph-mon[35941]: pgmap v2421: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:43:29.055 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:43:28 smithi119 ceph-mon[31518]: pgmap v2421: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:43:29.125 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:43:28 smithi165 ceph-mon[36302]: pgmap v2421: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:43:30.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:43:30 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4070.719645 secs 2024-03-28T01:43:30.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:43:30 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4070.719645 secs 2024-03-28T01:43:30.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:43:30 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4070.719645 secs 2024-03-28T01:43:31.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:43:31 smithi165 ceph-mon[36302]: pgmap v2422: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:43:31.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:43:31 smithi119 ceph-mon[31518]: pgmap v2422: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:43:31.945 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:43:31 smithi132 ceph-mon[35941]: pgmap v2422: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:43:32.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:43:32 smithi132 ceph-mon[35941]: pgmap v2423: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:43:33.125 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:43:32 smithi165 ceph-mon[36302]: pgmap v2423: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:43:33.148 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:43:32 smithi119 ceph-mon[31518]: pgmap v2423: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:43:35.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:43:35 smithi132 ceph-mon[35941]: pgmap v2424: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:43:35.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:43:35 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4075.719706 secs 2024-03-28T01:43:36.125 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:43:35 smithi165 ceph-mon[36302]: pgmap v2424: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:43:36.125 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:43:35 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4075.719706 secs 2024-03-28T01:43:36.148 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:43:35 smithi119 ceph-mon[31518]: pgmap v2424: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:43:36.149 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:43:35 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4075.719706 secs 2024-03-28T01:43:37.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:43:37 smithi132 ceph-mon[35941]: pgmap v2425: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:43:38.125 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:43:37 smithi165 ceph-mon[36302]: pgmap v2425: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:43:38.148 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:43:37 smithi119 ceph-mon[31518]: pgmap v2425: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:43:39.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:43:39 smithi132 ceph-mon[35941]: pgmap v2426: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:43:39.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:43:39 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:43:40.125 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:43:39 smithi165 ceph-mon[36302]: pgmap v2426: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:43:40.125 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:43:39 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:43:40.148 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:43:39 smithi119 ceph-mon[31518]: pgmap v2426: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:43:40.148 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:43:39 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:43:40.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:43:40 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:43:40.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:43:40 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:43:40.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:43:40 smithi132 ceph-mon[35941]: pgmap v2427: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T01:43:40.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:43:40 smithi132 ceph-mon[35941]: pgmap v2428: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:43:40.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:43:40 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:43:40.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:43:40 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4080.719791 secs 2024-03-28T01:43:41.125 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:43:40 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:43:41.125 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:43:40 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:43:41.125 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:43:40 smithi165 ceph-mon[36302]: pgmap v2427: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T01:43:41.126 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:43:40 smithi165 ceph-mon[36302]: pgmap v2428: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:43:41.126 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:43:40 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:43:41.126 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:43:40 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4080.719791 secs 2024-03-28T01:43:41.149 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:43:40 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:43:41.149 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:43:40 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:43:41.149 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:43:40 smithi119 ceph-mon[31518]: pgmap v2427: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T01:43:41.149 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:43:40 smithi119 ceph-mon[31518]: pgmap v2428: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:43:41.149 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:43:40 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:43:41.149 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:43:40 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4080.719791 secs 2024-03-28T01:43:43.375 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:43:43 smithi165 ceph-mon[36302]: pgmap v2429: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:43:43.398 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:43:43 smithi119 ceph-mon[31518]: pgmap v2429: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:43:43.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:43:43 smithi132 ceph-mon[35941]: pgmap v2429: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:43:45.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:43:45 smithi165 ceph-mon[36302]: pgmap v2430: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-28T01:43:45.398 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:43:45 smithi119 ceph-mon[31518]: pgmap v2430: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-28T01:43:45.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:43:45 smithi132 ceph-mon[35941]: pgmap v2430: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-28T01:43:46.375 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:43:46 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4085.719895 secs 2024-03-28T01:43:46.398 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:43:46 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4085.719895 secs 2024-03-28T01:43:46.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:43:46 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4085.719895 secs 2024-03-28T01:43:47.375 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:43:47 smithi165 ceph-mon[36302]: pgmap v2431: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:43:47.398 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:43:47 smithi119 ceph-mon[31518]: pgmap v2431: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:43:47.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:43:47 smithi132 ceph-mon[35941]: pgmap v2431: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:43:49.375 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:43:49 smithi165 ceph-mon[36302]: pgmap v2432: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:43:49.398 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:43:49 smithi119 ceph-mon[31518]: pgmap v2432: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:43:49.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:43:49 smithi132 ceph-mon[35941]: pgmap v2432: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:43:51.375 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:43:51 smithi165 ceph-mon[36302]: pgmap v2433: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T01:43:51.375 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:43:51 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4090.719986 secs 2024-03-28T01:43:51.398 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:43:51 smithi119 ceph-mon[31518]: pgmap v2433: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T01:43:51.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:43:51 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4090.719986 secs 2024-03-28T01:43:51.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:43:51 smithi132 ceph-mon[35941]: pgmap v2433: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T01:43:51.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:43:51 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4090.719986 secs 2024-03-28T01:43:53.375 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:43:53 smithi165 ceph-mon[36302]: pgmap v2434: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:43:53.398 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:43:53 smithi119 ceph-mon[31518]: pgmap v2434: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:43:53.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:43:53 smithi132 ceph-mon[35941]: pgmap v2434: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:43:55.375 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:43:55 smithi165 ceph-mon[36302]: pgmap v2435: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:43:55.398 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:43:55 smithi119 ceph-mon[31518]: pgmap v2435: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:43:55.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:43:55 smithi132 ceph-mon[35941]: pgmap v2435: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:43:56.375 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:43:56 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4095.720050 secs 2024-03-28T01:43:56.398 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:43:56 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4095.720050 secs 2024-03-28T01:43:56.445 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:43:56 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4095.720050 secs 2024-03-28T01:43:57.375 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:43:57 smithi165 ceph-mon[36302]: pgmap v2436: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:43:57.398 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:43:57 smithi119 ceph-mon[31518]: pgmap v2436: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:43:57.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:43:57 smithi132 ceph-mon[35941]: pgmap v2436: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:43:59.375 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:43:59 smithi165 ceph-mon[36302]: pgmap v2437: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:43:59.398 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:43:59 smithi119 ceph-mon[31518]: pgmap v2437: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:43:59.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:43:59 smithi132 ceph-mon[35941]: pgmap v2437: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:44:01.375 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:44:01 smithi165 ceph-mon[36302]: pgmap v2438: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:44:01.375 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:44:01 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4100.720154 secs 2024-03-28T01:44:01.398 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:44:01 smithi119 ceph-mon[31518]: pgmap v2438: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:44:01.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:44:01 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4100.720154 secs 2024-03-28T01:44:01.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:44:01 smithi132 ceph-mon[35941]: pgmap v2438: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:44:01.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:44:01 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4100.720154 secs 2024-03-28T01:44:03.375 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:44:03 smithi165 ceph-mon[36302]: pgmap v2439: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:44:03.398 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:44:03 smithi119 ceph-mon[31518]: pgmap v2439: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:44:03.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:44:03 smithi132 ceph-mon[35941]: pgmap v2439: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:44:05.398 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:44:05 smithi119 ceph-mon[31518]: pgmap v2440: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:44:05.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:44:05 smithi132 ceph-mon[35941]: pgmap v2440: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:44:05.625 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:44:05 smithi165 ceph-mon[36302]: pgmap v2440: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:44:06.398 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:44:06 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4105.720218 secs 2024-03-28T01:44:06.445 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:44:06 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4105.720218 secs 2024-03-28T01:44:06.625 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:44:06 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4105.720218 secs 2024-03-28T01:44:07.398 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:44:07 smithi119 ceph-mon[31518]: pgmap v2441: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:44:07.445 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:44:07 smithi132 ceph-mon[35941]: pgmap v2441: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:44:07.625 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:44:07 smithi165 ceph-mon[36302]: pgmap v2441: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:44:09.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:44:09 smithi119 ceph-mon[31518]: pgmap v2442: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:44:09.445 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:44:09 smithi132 ceph-mon[35941]: pgmap v2442: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:44:09.625 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:44:09 smithi165 ceph-mon[36302]: pgmap v2442: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:44:11.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:44:11 smithi132 ceph-mon[35941]: pgmap v2443: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:44:11.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:44:11 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4110.720305 secs 2024-03-28T01:44:11.625 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:44:11 smithi165 ceph-mon[36302]: pgmap v2443: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:44:11.625 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:44:11 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4110.720305 secs 2024-03-28T01:44:11.648 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:44:11 smithi119 ceph-mon[31518]: pgmap v2443: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:44:11.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:44:11 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4110.720305 secs 2024-03-28T01:44:13.445 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:44:13 smithi132 ceph-mon[35941]: pgmap v2444: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:44:13.625 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:44:13 smithi165 ceph-mon[36302]: pgmap v2444: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:44:13.648 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:44:13 smithi119 ceph-mon[31518]: pgmap v2444: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:44:15.398 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:44:15 smithi119 ceph-mon[31518]: pgmap v2445: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:44:15.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:44:15 smithi132 ceph-mon[35941]: pgmap v2445: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:44:15.625 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:44:15 smithi165 ceph-mon[36302]: pgmap v2445: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:44:16.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:44:16 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4115.720367 secs 2024-03-28T01:44:16.625 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:44:16 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4115.720367 secs 2024-03-28T01:44:16.648 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:44:16 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4115.720367 secs 2024-03-28T01:44:17.445 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:44:17 smithi132 ceph-mon[35941]: pgmap v2446: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:44:17.550 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:44:17 smithi119 ceph-mon[31518]: pgmap v2446: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:44:17.625 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:44:17 smithi165 ceph-mon[36302]: pgmap v2446: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:44:19.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:44:19 smithi132 ceph-mon[35941]: pgmap v2447: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:44:19.625 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:44:19 smithi165 ceph-mon[36302]: pgmap v2447: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:44:19.648 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:44:19 smithi119 ceph-mon[31518]: pgmap v2447: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:44:21.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:44:21 smithi132 ceph-mon[35941]: pgmap v2448: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:44:21.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:44:21 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4120.720455 secs 2024-03-28T01:44:21.625 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:44:21 smithi165 ceph-mon[36302]: pgmap v2448: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:44:21.625 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:44:21 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4120.720455 secs 2024-03-28T01:44:21.648 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:44:21 smithi119 ceph-mon[31518]: pgmap v2448: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:44:21.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:44:21 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4120.720455 secs 2024-03-28T01:44:23.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:44:23 smithi132 ceph-mon[35941]: pgmap v2449: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:44:23.625 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:44:23 smithi165 ceph-mon[36302]: pgmap v2449: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:44:23.648 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:44:23 smithi119 ceph-mon[31518]: pgmap v2449: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:44:25.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:44:25 smithi132 ceph-mon[35941]: pgmap v2450: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:44:25.625 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:44:25 smithi165 ceph-mon[36302]: pgmap v2450: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:44:25.648 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:44:25 smithi119 ceph-mon[31518]: pgmap v2450: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:44:26.625 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:44:26 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4125.720515 secs 2024-03-28T01:44:26.648 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:44:26 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4125.720515 secs 2024-03-28T01:44:26.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:44:26 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4125.720515 secs 2024-03-28T01:44:27.625 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:44:27 smithi165 ceph-mon[36302]: pgmap v2451: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:44:27.648 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:44:27 smithi119 ceph-mon[31518]: pgmap v2451: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:44:27.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:44:27 smithi132 ceph-mon[35941]: pgmap v2451: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:44:29.625 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:44:29 smithi165 ceph-mon[36302]: pgmap v2452: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:44:29.648 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:44:29 smithi119 ceph-mon[31518]: pgmap v2452: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:44:29.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:44:29 smithi132 ceph-mon[35941]: pgmap v2452: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:44:31.398 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:44:31 smithi119 ceph-mon[31518]: pgmap v2453: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:44:31.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:44:31 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4130.720605 secs 2024-03-28T01:44:31.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:44:31 smithi165 ceph-mon[36302]: pgmap v2453: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:44:31.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:44:31 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4130.720605 secs 2024-03-28T01:44:31.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:44:31 smithi132 ceph-mon[35941]: pgmap v2453: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:44:31.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:44:31 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4130.720605 secs 2024-03-28T01:44:33.625 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:44:33 smithi165 ceph-mon[36302]: pgmap v2454: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:44:33.648 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:44:33 smithi119 ceph-mon[31518]: pgmap v2454: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:44:33.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:44:33 smithi132 ceph-mon[35941]: pgmap v2454: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:44:35.625 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:44:35 smithi165 ceph-mon[36302]: pgmap v2455: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T01:44:35.648 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:44:35 smithi119 ceph-mon[31518]: pgmap v2455: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T01:44:35.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:44:35 smithi132 ceph-mon[35941]: pgmap v2455: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T01:44:36.625 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:44:36 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4135.720693 secs 2024-03-28T01:44:36.648 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:44:36 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4135.720693 secs 2024-03-28T01:44:36.695 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:44:36 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4135.720693 secs 2024-03-28T01:44:37.625 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:44:37 smithi165 ceph-mon[36302]: pgmap v2456: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:44:37.648 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:44:37 smithi119 ceph-mon[31518]: pgmap v2456: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:44:37.695 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:44:37 smithi132 ceph-mon[35941]: pgmap v2456: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:44:39.625 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:44:39 smithi165 ceph-mon[36302]: pgmap v2457: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:44:39.648 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:44:39 smithi119 ceph-mon[31518]: pgmap v2457: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:44:39.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:44:39 smithi132 ceph-mon[35941]: pgmap v2457: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:44:41.625 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:44:41 smithi165 ceph-mon[36302]: pgmap v2458: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:44:41.625 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:44:41 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4140.720782 secs 2024-03-28T01:44:41.625 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:44:41 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:44:41.648 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:44:41 smithi119 ceph-mon[31518]: pgmap v2458: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:44:41.648 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:44:41 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4140.720782 secs 2024-03-28T01:44:41.648 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:44:41 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:44:41.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:44:41 smithi132 ceph-mon[35941]: pgmap v2458: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:44:41.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:44:41 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4140.720782 secs 2024-03-28T01:44:41.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:44:41 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:44:42.625 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:44:42 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:44:42.625 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:44:42 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:44:42.625 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:44:42 smithi165 ceph-mon[36302]: pgmap v2459: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T01:44:42.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:44:42 smithi165 ceph-mon[36302]: pgmap v2460: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:44:42.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:44:42 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:44:42.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:44:42 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:44:42.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:44:42 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:44:42.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:44:42 smithi119 ceph-mon[31518]: pgmap v2459: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T01:44:42.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:44:42 smithi119 ceph-mon[31518]: pgmap v2460: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:44:42.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:44:42 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:44:42.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:44:42 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:44:42.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:44:42 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:44:42.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:44:42 smithi132 ceph-mon[35941]: pgmap v2459: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T01:44:42.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:44:42 smithi132 ceph-mon[35941]: pgmap v2460: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:44:42.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:44:42 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:44:44.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:44:44 smithi165 ceph-mon[36302]: pgmap v2461: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 5 op/s 2024-03-28T01:44:44.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:44:44 smithi119 ceph-mon[31518]: pgmap v2461: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 5 op/s 2024-03-28T01:44:44.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:44:44 smithi132 ceph-mon[35941]: pgmap v2461: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 5 op/s 2024-03-28T01:44:45.855 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:44:45 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4145.720871 secs 2024-03-28T01:44:45.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:44:45 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4145.720871 secs 2024-03-28T01:44:45.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:44:45 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4145.720871 secs 2024-03-28T01:44:46.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:44:46 smithi165 ceph-mon[36302]: pgmap v2462: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:44:46.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:44:46 smithi119 ceph-mon[31518]: pgmap v2462: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:44:46.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:44:46 smithi132 ceph-mon[35941]: pgmap v2462: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:44:48.877 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:44:48 smithi165 ceph-mon[36302]: pgmap v2463: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:44:48.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:44:48 smithi119 ceph-mon[31518]: pgmap v2463: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:44:48.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:44:48 smithi132 ceph-mon[35941]: pgmap v2463: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:44:50.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:44:50 smithi165 ceph-mon[36302]: pgmap v2464: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:44:50.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:44:50 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4150.720960 secs 2024-03-28T01:44:50.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:44:50 smithi119 ceph-mon[31518]: pgmap v2464: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:44:50.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:44:50 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4150.720960 secs 2024-03-28T01:44:50.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:44:50 smithi132 ceph-mon[35941]: pgmap v2464: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:44:50.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:44:50 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4150.720960 secs 2024-03-28T01:44:52.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:44:52 smithi165 ceph-mon[36302]: pgmap v2465: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T01:44:52.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:44:52 smithi119 ceph-mon[31518]: pgmap v2465: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T01:44:52.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:44:52 smithi132 ceph-mon[35941]: pgmap v2465: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T01:44:54.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:44:54 smithi165 ceph-mon[36302]: pgmap v2466: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:44:54.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:44:54 smithi119 ceph-mon[31518]: pgmap v2466: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:44:54.945 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:44:54 smithi132 ceph-mon[35941]: pgmap v2466: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:44:55.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:44:55 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4155.720975 secs 2024-03-28T01:44:55.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:44:55 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4155.720975 secs 2024-03-28T01:44:55.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:44:55 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4155.720975 secs 2024-03-28T01:44:56.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:44:56 smithi165 ceph-mon[36302]: pgmap v2467: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:44:56.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:44:56 smithi119 ceph-mon[31518]: pgmap v2467: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:44:56.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:44:56 smithi132 ceph-mon[35941]: pgmap v2467: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:44:58.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:44:58 smithi165 ceph-mon[36302]: pgmap v2468: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:44:58.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:44:58 smithi119 ceph-mon[31518]: pgmap v2468: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:44:58.945 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:44:58 smithi132 ceph-mon[35941]: pgmap v2468: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:44:59.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:44:59 smithi119 ceph-mon[31518]: pgmap v2469: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:44:59.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:44:59 smithi132 ceph-mon[35941]: pgmap v2469: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:45:00.125 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:44:59 smithi165 ceph-mon[36302]: pgmap v2469: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:45:00.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:45:00 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4160.721037 secs 2024-03-28T01:45:00.945 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:45:00 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4160.721037 secs 2024-03-28T01:45:01.125 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:45:00 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4160.721037 secs 2024-03-28T01:45:01.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:45:01 smithi119 ceph-mon[31518]: pgmap v2470: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:45:01.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:45:01 smithi132 ceph-mon[35941]: pgmap v2470: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:45:02.125 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:45:01 smithi165 ceph-mon[36302]: pgmap v2470: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:45:04.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:45:04 smithi165 ceph-mon[36302]: pgmap v2471: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:45:04.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:45:04 smithi119 ceph-mon[31518]: pgmap v2471: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:45:04.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:45:04 smithi132 ceph-mon[35941]: pgmap v2471: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:45:05.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:45:05 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4165.721097 secs 2024-03-28T01:45:05.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:45:05 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4165.721097 secs 2024-03-28T01:45:05.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:45:05 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4165.721097 secs 2024-03-28T01:45:06.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:45:06 smithi165 ceph-mon[36302]: pgmap v2472: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:45:06.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:45:06 smithi119 ceph-mon[31518]: pgmap v2472: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:45:06.945 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:45:06 smithi132 ceph-mon[35941]: pgmap v2472: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:45:08.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:45:08 smithi165 ceph-mon[36302]: pgmap v2473: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:45:08.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:45:08 smithi119 ceph-mon[31518]: pgmap v2473: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:45:08.945 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:45:08 smithi132 ceph-mon[35941]: pgmap v2473: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:45:10.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:45:10 smithi165 ceph-mon[36302]: pgmap v2474: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:45:10.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:45:10 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4170.721157 secs 2024-03-28T01:45:10.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:45:10 smithi119 ceph-mon[31518]: pgmap v2474: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:45:10.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:45:10 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4170.721157 secs 2024-03-28T01:45:10.945 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:45:10 smithi132 ceph-mon[35941]: pgmap v2474: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:45:10.945 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:45:10 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4170.721157 secs 2024-03-28T01:45:12.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:45:12 smithi165 ceph-mon[36302]: pgmap v2475: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:45:12.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:45:12 smithi119 ceph-mon[31518]: pgmap v2475: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:45:12.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:45:12 smithi132 ceph-mon[35941]: pgmap v2475: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:45:14.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:45:14 smithi165 ceph-mon[36302]: pgmap v2476: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:45:14.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:45:14 smithi119 ceph-mon[31518]: pgmap v2476: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:45:14.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:45:14 smithi132 ceph-mon[35941]: pgmap v2476: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:45:15.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:45:15 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4175.721246 secs 2024-03-28T01:45:15.945 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:45:15 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4175.721246 secs 2024-03-28T01:45:16.125 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:45:15 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4175.721246 secs 2024-03-28T01:45:16.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:45:16 smithi119 ceph-mon[31518]: pgmap v2477: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:45:16.945 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:45:16 smithi132 ceph-mon[35941]: pgmap v2477: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:45:17.125 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:45:16 smithi165 ceph-mon[36302]: pgmap v2477: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:45:17.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:45:17 smithi119 ceph-mon[31518]: pgmap v2478: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:45:17.945 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:45:17 smithi132 ceph-mon[35941]: pgmap v2478: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:45:18.125 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:45:17 smithi165 ceph-mon[36302]: pgmap v2478: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:45:20.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:45:20 smithi165 ceph-mon[36302]: pgmap v2479: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:45:20.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:45:20 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4180.721336 secs 2024-03-28T01:45:20.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:45:20 smithi119 ceph-mon[31518]: pgmap v2479: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:45:20.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:45:20 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4180.721336 secs 2024-03-28T01:45:20.945 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:45:20 smithi132 ceph-mon[35941]: pgmap v2479: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:45:20.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:45:20 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4180.721336 secs 2024-03-28T01:45:22.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:45:22 smithi165 ceph-mon[36302]: pgmap v2480: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:45:22.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:45:22 smithi119 ceph-mon[31518]: pgmap v2480: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:45:22.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:45:22 smithi132 ceph-mon[35941]: pgmap v2480: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:45:24.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:45:24 smithi165 ceph-mon[36302]: pgmap v2481: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:45:24.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:45:24 smithi119 ceph-mon[31518]: pgmap v2481: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:45:24.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:45:24 smithi132 ceph-mon[35941]: pgmap v2481: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:45:25.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:45:25 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4185.721427 secs 2024-03-28T01:45:25.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:45:25 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4185.721427 secs 2024-03-28T01:45:25.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:45:25 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4185.721427 secs 2024-03-28T01:45:26.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:45:26 smithi165 ceph-mon[36302]: pgmap v2482: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:45:26.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:45:26 smithi119 ceph-mon[31518]: pgmap v2482: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:45:26.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:45:26 smithi132 ceph-mon[35941]: pgmap v2482: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:45:28.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:45:28 smithi165 ceph-mon[36302]: pgmap v2483: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:45:28.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:45:28 smithi119 ceph-mon[31518]: pgmap v2483: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:45:28.945 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:45:28 smithi132 ceph-mon[35941]: pgmap v2483: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:45:30.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:45:30 smithi119 ceph-mon[31518]: pgmap v2484: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:45:30.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:45:30 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4190.721515 secs 2024-03-28T01:45:30.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:45:30 smithi132 ceph-mon[35941]: pgmap v2484: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:45:30.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:45:30 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4190.721515 secs 2024-03-28T01:45:31.125 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:45:30 smithi165 ceph-mon[36302]: pgmap v2484: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:45:31.125 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:45:30 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4190.721515 secs 2024-03-28T01:45:31.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:45:31 smithi132 ceph-mon[35941]: pgmap v2485: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:45:32.125 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:45:31 smithi165 ceph-mon[36302]: pgmap v2485: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:45:32.148 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:45:31 smithi119 ceph-mon[31518]: pgmap v2485: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:45:34.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:45:34 smithi165 ceph-mon[36302]: pgmap v2486: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:45:34.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:45:34 smithi119 ceph-mon[31518]: pgmap v2486: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:45:34.945 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:45:34 smithi132 ceph-mon[35941]: pgmap v2486: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:45:35.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:45:35 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4195.721604 secs 2024-03-28T01:45:35.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:45:35 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4195.721604 secs 2024-03-28T01:45:35.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:45:35 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4195.721604 secs 2024-03-28T01:45:36.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:45:36 smithi165 ceph-mon[36302]: pgmap v2487: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:45:36.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:45:36 smithi119 ceph-mon[31518]: pgmap v2487: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:45:36.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:45:36 smithi132 ceph-mon[35941]: pgmap v2487: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:45:38.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:45:38 smithi165 ceph-mon[36302]: pgmap v2488: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T01:45:38.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:45:38 smithi119 ceph-mon[31518]: pgmap v2488: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T01:45:38.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:45:38 smithi132 ceph-mon[35941]: pgmap v2488: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T01:45:40.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:45:40 smithi165 ceph-mon[36302]: pgmap v2489: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:45:40.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:45:40 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4200.721648 secs 2024-03-28T01:45:40.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:45:40 smithi119 ceph-mon[31518]: pgmap v2489: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:45:40.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:45:40 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4200.721648 secs 2024-03-28T01:45:40.945 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:45:40 smithi132 ceph-mon[35941]: pgmap v2489: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:45:40.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:45:40 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4200.721648 secs 2024-03-28T01:45:42.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:45:42 smithi119 ceph-mon[31518]: pgmap v2490: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T01:45:42.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:45:42 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:45:42.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:45:42 smithi132 ceph-mon[35941]: pgmap v2490: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T01:45:42.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:45:42 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:45:43.125 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:45:42 smithi165 ceph-mon[36302]: pgmap v2490: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T01:45:43.125 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:45:42 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:45:43.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:45:43 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:45:43.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:45:43 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:45:43.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:45:43 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:45:43.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:45:43 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:45:43.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:45:43 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:45:43.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:45:43 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:45:44.125 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:45:43 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:45:44.125 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:45:43 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:45:44.125 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:45:43 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:45:44.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:45:44 smithi119 ceph-mon[31518]: pgmap v2491: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T01:45:44.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:45:44 smithi119 ceph-mon[31518]: pgmap v2492: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T01:45:44.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:45:44 smithi132 ceph-mon[35941]: pgmap v2491: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T01:45:44.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:45:44 smithi132 ceph-mon[35941]: pgmap v2492: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T01:45:45.125 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:45:44 smithi165 ceph-mon[36302]: pgmap v2491: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T01:45:45.125 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:45:44 smithi165 ceph-mon[36302]: pgmap v2492: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T01:45:45.945 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:45:45 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4205.721711 secs 2024-03-28T01:45:45.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:45:45 smithi132 ceph-mon[35941]: pgmap v2493: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:45:46.125 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:45:45 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4205.721711 secs 2024-03-28T01:45:46.125 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:45:45 smithi165 ceph-mon[36302]: pgmap v2493: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:45:46.148 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:45:45 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4205.721711 secs 2024-03-28T01:45:46.148 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:45:45 smithi119 ceph-mon[31518]: pgmap v2493: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:45:48.398 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:45:48 smithi119 ceph-mon[31518]: pgmap v2494: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:45:48.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:45:48 smithi132 ceph-mon[35941]: pgmap v2494: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:45:48.625 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:45:48 smithi165 ceph-mon[36302]: pgmap v2494: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:45:50.398 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:45:50 smithi119 ceph-mon[31518]: pgmap v2495: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 4 op/s 2024-03-28T01:45:50.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:45:50 smithi132 ceph-mon[35941]: pgmap v2495: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 4 op/s 2024-03-28T01:45:50.625 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:45:50 smithi165 ceph-mon[36302]: pgmap v2495: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 4 op/s 2024-03-28T01:45:51.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:45:51 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4210.721771 secs 2024-03-28T01:45:51.625 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:45:51 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4210.721771 secs 2024-03-28T01:45:51.648 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:45:51 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4210.721771 secs 2024-03-28T01:45:52.445 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:45:52 smithi132 ceph-mon[35941]: pgmap v2496: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:45:52.526 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:45:52 smithi165 ceph-mon[36302]: pgmap v2496: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:45:52.648 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:45:52 smithi119 ceph-mon[31518]: pgmap v2496: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:45:54.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:45:54 smithi132 ceph-mon[35941]: pgmap v2497: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T01:45:54.625 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:45:54 smithi165 ceph-mon[36302]: pgmap v2497: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T01:45:54.648 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:45:54 smithi119 ceph-mon[31518]: pgmap v2497: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T01:45:56.445 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:45:56 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4215.721863 secs 2024-03-28T01:45:56.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:45:56 smithi132 ceph-mon[35941]: pgmap v2498: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:45:56.625 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:45:56 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4215.721863 secs 2024-03-28T01:45:56.625 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:45:56 smithi165 ceph-mon[36302]: pgmap v2498: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:45:56.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:45:56 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4215.721863 secs 2024-03-28T01:45:56.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:45:56 smithi119 ceph-mon[31518]: pgmap v2498: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:45:58.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:45:58 smithi132 ceph-mon[35941]: pgmap v2499: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:45:58.625 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:45:58 smithi165 ceph-mon[36302]: pgmap v2499: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:45:58.648 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:45:58 smithi119 ceph-mon[31518]: pgmap v2499: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:46:00.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:46:00 smithi132 ceph-mon[35941]: pgmap v2500: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T01:46:00.625 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:46:00 smithi165 ceph-mon[36302]: pgmap v2500: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T01:46:00.648 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:46:00 smithi119 ceph-mon[31518]: pgmap v2500: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T01:46:01.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:46:01 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4220.721904 secs 2024-03-28T01:46:01.625 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:46:01 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4220.721904 secs 2024-03-28T01:46:01.648 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:46:01 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4220.721904 secs 2024-03-28T01:46:02.445 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:46:02 smithi132 ceph-mon[35941]: pgmap v2501: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-28T01:46:02.625 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:46:02 smithi165 ceph-mon[36302]: pgmap v2501: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-28T01:46:02.648 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:46:02 smithi119 ceph-mon[31518]: pgmap v2501: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-28T01:46:04.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:46:04 smithi132 ceph-mon[35941]: pgmap v2502: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:46:04.451 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:46:04 smithi119 ceph-mon[31518]: pgmap v2502: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:46:04.625 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:46:04 smithi165 ceph-mon[36302]: pgmap v2502: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:46:06.625 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:46:06 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4225.721993 secs 2024-03-28T01:46:06.625 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:46:06 smithi165 ceph-mon[36302]: pgmap v2503: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:46:06.648 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:46:06 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4225.721993 secs 2024-03-28T01:46:06.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:46:06 smithi119 ceph-mon[31518]: pgmap v2503: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:46:06.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:46:06 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4225.721993 secs 2024-03-28T01:46:06.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:46:06 smithi132 ceph-mon[35941]: pgmap v2503: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:46:08.625 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:46:08 smithi165 ceph-mon[36302]: pgmap v2504: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:46:08.648 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:46:08 smithi119 ceph-mon[31518]: pgmap v2504: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:46:08.695 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:46:08 smithi132 ceph-mon[35941]: pgmap v2504: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:46:10.625 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:46:10 smithi165 ceph-mon[36302]: pgmap v2505: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T01:46:10.648 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:46:10 smithi119 ceph-mon[31518]: pgmap v2505: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T01:46:10.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:46:10 smithi132 ceph-mon[35941]: pgmap v2505: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T01:46:11.625 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:46:11 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4230.722077 secs 2024-03-28T01:46:11.648 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:46:11 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4230.722077 secs 2024-03-28T01:46:11.695 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:46:11 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4230.722077 secs 2024-03-28T01:46:12.625 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:46:12 smithi165 ceph-mon[36302]: pgmap v2506: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-28T01:46:12.648 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:46:12 smithi119 ceph-mon[31518]: pgmap v2506: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-28T01:46:12.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:46:12 smithi132 ceph-mon[35941]: pgmap v2506: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-28T01:46:14.625 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:46:14 smithi165 ceph-mon[36302]: pgmap v2507: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:46:14.648 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:46:14 smithi119 ceph-mon[31518]: pgmap v2507: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:46:14.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:46:14 smithi132 ceph-mon[35941]: pgmap v2507: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:46:16.625 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:46:16 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4235.722163 secs 2024-03-28T01:46:16.625 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:46:16 smithi165 ceph-mon[36302]: pgmap v2508: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:46:16.648 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:46:16 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4235.722163 secs 2024-03-28T01:46:16.648 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:46:16 smithi119 ceph-mon[31518]: pgmap v2508: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:46:16.695 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:46:16 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4235.722163 secs 2024-03-28T01:46:16.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:46:16 smithi132 ceph-mon[35941]: pgmap v2508: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:46:18.625 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:46:18 smithi165 ceph-mon[36302]: pgmap v2509: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:46:18.648 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:46:18 smithi119 ceph-mon[31518]: pgmap v2509: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:46:18.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:46:18 smithi132 ceph-mon[35941]: pgmap v2509: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:46:20.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:46:20 smithi119 ceph-mon[31518]: pgmap v2510: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T01:46:20.625 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:46:20 smithi165 ceph-mon[36302]: pgmap v2510: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T01:46:20.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:46:20 smithi132 ceph-mon[35941]: pgmap v2510: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T01:46:21.625 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:46:21 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4240.722252 secs 2024-03-28T01:46:21.648 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:46:21 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4240.722252 secs 2024-03-28T01:46:21.695 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:46:21 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4240.722252 secs 2024-03-28T01:46:22.526 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:46:22 smithi165 ceph-mon[36302]: pgmap v2511: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-28T01:46:22.648 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:46:22 smithi119 ceph-mon[31518]: pgmap v2511: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-28T01:46:22.695 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:46:22 smithi132 ceph-mon[35941]: pgmap v2511: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-28T01:46:24.625 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:46:24 smithi165 ceph-mon[36302]: pgmap v2512: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:46:24.648 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:46:24 smithi119 ceph-mon[31518]: pgmap v2512: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:46:24.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:46:24 smithi132 ceph-mon[35941]: pgmap v2512: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:46:26.625 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:46:26 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4245.722335 secs 2024-03-28T01:46:26.625 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:46:26 smithi165 ceph-mon[36302]: pgmap v2513: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:46:26.648 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:46:26 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4245.722335 secs 2024-03-28T01:46:26.648 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:46:26 smithi119 ceph-mon[31518]: pgmap v2513: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:46:26.695 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:46:26 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4245.722335 secs 2024-03-28T01:46:26.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:46:26 smithi132 ceph-mon[35941]: pgmap v2513: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:46:28.625 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:46:28 smithi165 ceph-mon[36302]: pgmap v2514: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:46:28.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:46:28 smithi119 ceph-mon[31518]: pgmap v2514: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:46:28.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:46:28 smithi132 ceph-mon[35941]: pgmap v2514: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:46:30.625 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:46:30 smithi165 ceph-mon[36302]: pgmap v2515: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T01:46:30.648 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:46:30 smithi119 ceph-mon[31518]: pgmap v2515: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T01:46:30.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:46:30 smithi132 ceph-mon[35941]: pgmap v2515: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T01:46:31.625 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:46:31 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4250.722397 secs 2024-03-28T01:46:31.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:46:31 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4250.722397 secs 2024-03-28T01:46:31.695 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:46:31 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4250.722397 secs 2024-03-28T01:46:32.625 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:46:32 smithi165 ceph-mon[36302]: pgmap v2516: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-28T01:46:32.648 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:46:32 smithi119 ceph-mon[31518]: pgmap v2516: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-28T01:46:32.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:46:32 smithi132 ceph-mon[35941]: pgmap v2516: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-28T01:46:34.625 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:46:34 smithi165 ceph-mon[36302]: pgmap v2517: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:46:34.648 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:46:34 smithi119 ceph-mon[31518]: pgmap v2517: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:46:34.695 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:46:34 smithi132 ceph-mon[35941]: pgmap v2517: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:46:36.625 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:46:36 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4255.722458 secs 2024-03-28T01:46:36.625 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:46:36 smithi165 ceph-mon[36302]: pgmap v2518: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:46:36.648 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:46:36 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4255.722458 secs 2024-03-28T01:46:36.648 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:46:36 smithi119 ceph-mon[31518]: pgmap v2518: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:46:36.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:46:36 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4255.722458 secs 2024-03-28T01:46:36.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:46:36 smithi132 ceph-mon[35941]: pgmap v2518: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:46:38.625 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:46:38 smithi165 ceph-mon[36302]: pgmap v2519: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:46:38.648 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:46:38 smithi119 ceph-mon[31518]: pgmap v2519: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:46:38.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:46:38 smithi132 ceph-mon[35941]: pgmap v2519: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:46:40.625 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:46:40 smithi165 ceph-mon[36302]: pgmap v2520: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T01:46:40.648 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:46:40 smithi119 ceph-mon[31518]: pgmap v2520: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T01:46:40.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:46:40 smithi132 ceph-mon[35941]: pgmap v2520: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T01:46:41.625 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:46:41 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4260.722516 secs 2024-03-28T01:46:41.648 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:46:41 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4260.722516 secs 2024-03-28T01:46:41.695 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:46:41 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4260.722516 secs 2024-03-28T01:46:42.625 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:46:42 smithi165 ceph-mon[36302]: pgmap v2521: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-28T01:46:42.648 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:46:42 smithi119 ceph-mon[31518]: pgmap v2521: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-28T01:46:42.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:46:42 smithi132 ceph-mon[35941]: pgmap v2521: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-28T01:46:44.625 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:46:44 smithi165 ceph-mon[36302]: pgmap v2522: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:46:44.625 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:46:44 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:46:44.648 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:46:44 smithi119 ceph-mon[31518]: pgmap v2522: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:46:44.648 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:46:44 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:46:44.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:46:44 smithi132 ceph-mon[35941]: pgmap v2522: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:46:44.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:46:44 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:46:45.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:46:45 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:46:45.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:46:45 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:46:45.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:46:45 smithi165 ceph-mon[36302]: pgmap v2523: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T01:46:45.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:46:45 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:46:45.648 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:46:45 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:46:45.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:46:45 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:46:45.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:46:45 smithi119 ceph-mon[31518]: pgmap v2523: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T01:46:45.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:46:45 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:46:45.695 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:46:45 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:46:45.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:46:45 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:46:45.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:46:45 smithi132 ceph-mon[35941]: pgmap v2523: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T01:46:45.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:46:45 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:46:46.625 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:46:46 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4265.722578 secs 2024-03-28T01:46:46.648 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:46:46 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4265.722578 secs 2024-03-28T01:46:46.695 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:46:46 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4265.722578 secs 2024-03-28T01:46:47.625 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:46:47 smithi165 ceph-mon[36302]: pgmap v2524: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T01:46:47.648 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:46:47 smithi119 ceph-mon[31518]: pgmap v2524: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T01:46:47.695 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:46:47 smithi132 ceph-mon[35941]: pgmap v2524: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T01:46:49.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:46:49 smithi165 ceph-mon[36302]: pgmap v2525: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T01:46:49.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:46:49 smithi119 ceph-mon[31518]: pgmap v2525: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T01:46:49.945 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:46:49 smithi132 ceph-mon[35941]: pgmap v2525: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T01:46:50.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:46:50 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4270.722668 secs 2024-03-28T01:46:50.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:46:50 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4270.722668 secs 2024-03-28T01:46:50.945 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:46:50 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4270.722668 secs 2024-03-28T01:46:51.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:46:51 smithi165 ceph-mon[36302]: pgmap v2526: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 6 op/s 2024-03-28T01:46:51.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:46:51 smithi119 ceph-mon[31518]: pgmap v2526: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 6 op/s 2024-03-28T01:46:51.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:46:51 smithi132 ceph-mon[35941]: pgmap v2526: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 6 op/s 2024-03-28T01:46:53.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:46:53 smithi119 ceph-mon[31518]: pgmap v2527: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T01:46:53.945 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:46:53 smithi132 ceph-mon[35941]: pgmap v2527: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T01:46:54.125 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:46:53 smithi165 ceph-mon[36302]: pgmap v2527: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T01:46:55.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:46:55 smithi119 ceph-mon[31518]: pgmap v2528: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T01:46:55.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:46:55 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4275.722730 secs 2024-03-28T01:46:55.945 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:46:55 smithi132 ceph-mon[35941]: pgmap v2528: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T01:46:55.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:46:55 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4275.722730 secs 2024-03-28T01:46:56.125 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:46:55 smithi165 ceph-mon[36302]: pgmap v2528: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T01:46:56.125 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:46:55 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4275.722730 secs 2024-03-28T01:46:57.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:46:57 smithi119 ceph-mon[31518]: pgmap v2529: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:46:57.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:46:57 smithi132 ceph-mon[35941]: pgmap v2529: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:46:58.125 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:46:57 smithi165 ceph-mon[36302]: pgmap v2529: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:46:59.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:46:59 smithi119 ceph-mon[31518]: pgmap v2530: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:46:59.945 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:46:59 smithi132 ceph-mon[35941]: pgmap v2530: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:47:00.125 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:46:59 smithi165 ceph-mon[36302]: pgmap v2530: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:47:00.945 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:47:00 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4280.722744 secs 2024-03-28T01:47:01.125 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:47:00 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4280.722744 secs 2024-03-28T01:47:01.148 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:47:00 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4280.722744 secs 2024-03-28T01:47:01.945 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:47:01 smithi132 ceph-mon[35941]: pgmap v2531: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:47:02.125 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:47:01 smithi165 ceph-mon[36302]: pgmap v2531: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:47:02.148 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:47:01 smithi119 ceph-mon[31518]: pgmap v2531: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:47:03.695 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:47:03 smithi132 ceph-mon[35941]: pgmap v2532: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:47:03.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:47:03 smithi165 ceph-mon[36302]: pgmap v2532: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:47:03.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:47:03 smithi119 ceph-mon[31518]: pgmap v2532: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:47:05.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:47:05 smithi165 ceph-mon[36302]: pgmap v2533: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:47:05.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:47:05 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4285.722802 secs 2024-03-28T01:47:05.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:47:05 smithi119 ceph-mon[31518]: pgmap v2533: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:47:05.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:47:05 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4285.722802 secs 2024-03-28T01:47:05.945 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:47:05 smithi132 ceph-mon[35941]: pgmap v2533: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:47:05.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:47:05 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4285.722802 secs 2024-03-28T01:47:07.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:47:07 smithi165 ceph-mon[36302]: pgmap v2534: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:47:07.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:47:07 smithi119 ceph-mon[31518]: pgmap v2534: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:47:07.945 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:47:07 smithi132 ceph-mon[35941]: pgmap v2534: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:47:09.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:47:09 smithi119 ceph-mon[31518]: pgmap v2535: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:47:09.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:47:09 smithi132 ceph-mon[35941]: pgmap v2535: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:47:10.125 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:47:09 smithi165 ceph-mon[36302]: pgmap v2535: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:47:10.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:47:10 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4290.722884 secs 2024-03-28T01:47:10.945 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:47:10 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4290.722884 secs 2024-03-28T01:47:11.125 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:47:10 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4290.722884 secs 2024-03-28T01:47:11.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:47:11 smithi119 ceph-mon[31518]: pgmap v2536: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:47:11.945 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:47:11 smithi132 ceph-mon[35941]: pgmap v2536: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:47:12.125 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:47:11 smithi165 ceph-mon[36302]: pgmap v2536: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:47:13.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:47:13 smithi119 ceph-mon[31518]: pgmap v2537: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:47:13.945 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:47:13 smithi132 ceph-mon[35941]: pgmap v2537: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:47:14.125 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:47:13 smithi165 ceph-mon[36302]: pgmap v2537: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:47:15.945 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:47:15 smithi132 ceph-mon[35941]: pgmap v2538: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:47:15.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:47:15 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4295.722944 secs 2024-03-28T01:47:16.125 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:47:15 smithi165 ceph-mon[36302]: pgmap v2538: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:47:16.125 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:47:15 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4295.722944 secs 2024-03-28T01:47:16.149 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:47:15 smithi119 ceph-mon[31518]: pgmap v2538: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:47:16.149 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:47:15 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4295.722944 secs 2024-03-28T01:47:17.125 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:47:16 smithi165 ceph-mon[36302]: pgmap v2539: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T01:47:17.148 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:47:16 smithi119 ceph-mon[31518]: pgmap v2539: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T01:47:17.195 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:47:16 smithi132 ceph-mon[35941]: pgmap v2539: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T01:47:19.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:47:19 smithi119 ceph-mon[31518]: pgmap v2540: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T01:47:19.945 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:47:19 smithi132 ceph-mon[35941]: pgmap v2540: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T01:47:20.125 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:47:19 smithi165 ceph-mon[36302]: pgmap v2540: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T01:47:20.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:47:20 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4300.722956 secs 2024-03-28T01:47:20.945 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:47:20 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4300.722956 secs 2024-03-28T01:47:21.125 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:47:20 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4300.722956 secs 2024-03-28T01:47:21.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:47:21 smithi119 ceph-mon[31518]: pgmap v2541: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:47:21.945 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:47:21 smithi132 ceph-mon[35941]: pgmap v2541: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:47:22.125 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:47:21 smithi165 ceph-mon[36302]: pgmap v2541: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:47:23.927 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:47:23 smithi165 ceph-mon[36302]: pgmap v2542: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T01:47:23.945 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:47:23 smithi132 ceph-mon[35941]: pgmap v2542: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T01:47:24.148 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:47:23 smithi119 ceph-mon[31518]: pgmap v2542: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T01:47:24.945 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:47:24 smithi132 ceph-mon[35941]: pgmap v2543: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:47:25.125 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:47:24 smithi165 ceph-mon[36302]: pgmap v2543: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:47:25.148 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:47:24 smithi119 ceph-mon[31518]: pgmap v2543: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:47:25.945 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:47:25 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4305.723019 secs 2024-03-28T01:47:26.125 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:47:25 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4305.723019 secs 2024-03-28T01:47:26.148 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:47:25 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4305.723019 secs 2024-03-28T01:47:26.945 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:47:26 smithi132 ceph-mon[35941]: pgmap v2544: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:47:27.125 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:47:26 smithi165 ceph-mon[36302]: pgmap v2544: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:47:27.148 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:47:26 smithi119 ceph-mon[31518]: pgmap v2544: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:47:29.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:47:29 smithi119 ceph-mon[31518]: pgmap v2545: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:47:29.945 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:47:29 smithi132 ceph-mon[35941]: pgmap v2545: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:47:30.125 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:47:29 smithi165 ceph-mon[36302]: pgmap v2545: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:47:30.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:47:30 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4310.723079 secs 2024-03-28T01:47:30.945 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:47:30 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4310.723079 secs 2024-03-28T01:47:31.125 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:47:30 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4310.723079 secs 2024-03-28T01:47:31.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:47:31 smithi119 ceph-mon[31518]: pgmap v2546: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:47:31.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:47:31 smithi132 ceph-mon[35941]: pgmap v2546: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:47:32.125 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:47:31 smithi165 ceph-mon[36302]: pgmap v2546: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:47:33.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:47:33 smithi119 ceph-mon[31518]: pgmap v2547: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:47:33.945 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:47:33 smithi132 ceph-mon[35941]: pgmap v2547: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:47:34.125 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:47:33 smithi165 ceph-mon[36302]: pgmap v2547: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:47:35.945 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:47:35 smithi132 ceph-mon[35941]: pgmap v2548: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:47:35.945 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:47:35 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4315.723139 secs 2024-03-28T01:47:36.125 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:47:35 smithi165 ceph-mon[36302]: pgmap v2548: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:47:36.125 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:47:35 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4315.723139 secs 2024-03-28T01:47:36.148 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:47:35 smithi119 ceph-mon[31518]: pgmap v2548: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:47:36.148 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:47:35 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4315.723139 secs 2024-03-28T01:47:37.050 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:47:36 smithi119 ceph-mon[31518]: pgmap v2549: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:47:37.125 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:47:36 smithi165 ceph-mon[36302]: pgmap v2549: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:47:37.195 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:47:36 smithi132 ceph-mon[35941]: pgmap v2549: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:47:39.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:47:39 smithi119 ceph-mon[31518]: pgmap v2550: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:47:39.945 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:47:39 smithi132 ceph-mon[35941]: pgmap v2550: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:47:40.125 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:47:39 smithi165 ceph-mon[36302]: pgmap v2550: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:47:40.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:47:40 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4320.723199 secs 2024-03-28T01:47:40.945 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:47:40 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4320.723199 secs 2024-03-28T01:47:41.125 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:47:40 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4320.723199 secs 2024-03-28T01:47:41.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:47:41 smithi119 ceph-mon[31518]: pgmap v2551: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:47:41.945 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:47:41 smithi132 ceph-mon[35941]: pgmap v2551: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:47:42.124 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:47:41 smithi165 ceph-mon[36302]: pgmap v2551: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:47:43.945 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:47:43 smithi132 ceph-mon[35941]: pgmap v2552: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:47:44.124 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:47:43 smithi165 ceph-mon[36302]: pgmap v2552: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:47:44.147 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:47:43 smithi119 ceph-mon[31518]: pgmap v2552: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:47:45.945 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:47:45 smithi132 ceph-mon[35941]: pgmap v2553: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:47:45.945 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:47:45 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:47:45.945 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:47:45 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4325.723264 secs 2024-03-28T01:47:46.124 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:47:45 smithi165 ceph-mon[36302]: pgmap v2553: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:47:46.125 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:47:45 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:47:46.125 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:47:45 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4325.723264 secs 2024-03-28T01:47:46.147 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:47:45 smithi119 ceph-mon[31518]: pgmap v2553: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:47:46.147 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:47:45 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:47:46.148 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:47:45 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4325.723264 secs 2024-03-28T01:47:46.945 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:47:46 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:47:46.945 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:47:46 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:47:46.945 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:47:46 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:47:47.124 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:47:46 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:47:47.125 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:47:46 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:47:47.125 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:47:46 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:47:47.148 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:47:46 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:47:47.148 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:47:46 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:47:47.148 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:47:46 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:47:47.945 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:47:47 smithi132 ceph-mon[35941]: pgmap v2554: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T01:47:47.945 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:47:47 smithi132 ceph-mon[35941]: pgmap v2555: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:47:48.125 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:47:47 smithi165 ceph-mon[36302]: pgmap v2554: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T01:47:48.125 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:47:47 smithi165 ceph-mon[36302]: pgmap v2555: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:47:48.148 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:47:47 smithi119 ceph-mon[31518]: pgmap v2554: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T01:47:48.148 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:47:47 smithi119 ceph-mon[31518]: pgmap v2555: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:47:48.945 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:47:48 smithi132 ceph-mon[35941]: pgmap v2556: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:47:49.124 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:47:48 smithi165 ceph-mon[36302]: pgmap v2556: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:47:49.148 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:47:48 smithi119 ceph-mon[31518]: pgmap v2556: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:47:51.398 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:47:51 smithi119 ceph-mon[31518]: pgmap v2557: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:47:51.398 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:47:51 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4330.723353 secs 2024-03-28T01:47:51.445 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:47:51 smithi132 ceph-mon[35941]: pgmap v2557: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:47:51.445 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:47:51 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4330.723353 secs 2024-03-28T01:47:51.625 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:47:51 smithi165 ceph-mon[36302]: pgmap v2557: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:47:51.625 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:47:51 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4330.723353 secs 2024-03-28T01:47:53.398 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:47:53 smithi119 ceph-mon[31518]: pgmap v2558: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:47:53.426 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:47:53 smithi165 ceph-mon[36302]: pgmap v2558: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:47:53.445 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:47:53 smithi132 ceph-mon[35941]: pgmap v2558: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:47:55.398 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:47:55 smithi119 ceph-mon[31518]: pgmap v2559: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 4 op/s 2024-03-28T01:47:55.445 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:47:55 smithi132 ceph-mon[35941]: pgmap v2559: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 4 op/s 2024-03-28T01:47:55.624 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:47:55 smithi165 ceph-mon[36302]: pgmap v2559: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 4 op/s 2024-03-28T01:47:56.398 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:47:56 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4335.723415 secs 2024-03-28T01:47:56.445 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:47:56 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4335.723415 secs 2024-03-28T01:47:56.624 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:47:56 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4335.723415 secs 2024-03-28T01:47:57.398 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:47:57 smithi119 ceph-mon[31518]: pgmap v2560: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T01:47:57.445 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:47:57 smithi132 ceph-mon[35941]: pgmap v2560: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T01:47:57.624 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:47:57 smithi165 ceph-mon[36302]: pgmap v2560: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T01:47:59.445 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:47:59 smithi132 ceph-mon[35941]: pgmap v2561: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:47:59.625 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:47:59 smithi165 ceph-mon[36302]: pgmap v2561: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:47:59.648 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:47:59 smithi119 ceph-mon[31518]: pgmap v2561: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:48:01.445 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:48:01 smithi132 ceph-mon[35941]: pgmap v2562: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:48:01.445 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:48:01 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4340.723503 secs 2024-03-28T01:48:01.624 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:48:01 smithi165 ceph-mon[36302]: pgmap v2562: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:48:01.625 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:48:01 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4340.723503 secs 2024-03-28T01:48:01.647 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:48:01 smithi119 ceph-mon[31518]: pgmap v2562: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:48:01.648 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:48:01 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4340.723503 secs 2024-03-28T01:48:03.445 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:48:03 smithi132 ceph-mon[35941]: pgmap v2563: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:48:03.624 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:48:03 smithi165 ceph-mon[36302]: pgmap v2563: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:48:03.647 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:48:03 smithi119 ceph-mon[31518]: pgmap v2563: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:48:05.445 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:48:05 smithi132 ceph-mon[35941]: pgmap v2564: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T01:48:05.624 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:48:05 smithi165 ceph-mon[36302]: pgmap v2564: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T01:48:05.647 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:48:05 smithi119 ceph-mon[31518]: pgmap v2564: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T01:48:06.445 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:48:06 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4345.723593 secs 2024-03-28T01:48:06.624 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:48:06 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4345.723593 secs 2024-03-28T01:48:06.647 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:48:06 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4345.723593 secs 2024-03-28T01:48:07.445 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:48:07 smithi132 ceph-mon[35941]: pgmap v2565: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-28T01:48:07.624 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:48:07 smithi165 ceph-mon[36302]: pgmap v2565: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-28T01:48:07.647 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:48:07 smithi119 ceph-mon[31518]: pgmap v2565: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-28T01:48:09.445 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:48:09 smithi132 ceph-mon[35941]: pgmap v2566: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:48:09.624 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:48:09 smithi165 ceph-mon[36302]: pgmap v2566: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:48:09.647 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:48:09 smithi119 ceph-mon[31518]: pgmap v2566: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:48:11.445 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:48:11 smithi132 ceph-mon[35941]: pgmap v2567: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:48:11.445 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:48:11 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4350.723631 secs 2024-03-28T01:48:11.624 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:48:11 smithi165 ceph-mon[36302]: pgmap v2567: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:48:11.625 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:48:11 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4350.723631 secs 2024-03-28T01:48:11.647 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:48:11 smithi119 ceph-mon[31518]: pgmap v2567: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:48:11.648 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:48:11 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4350.723631 secs 2024-03-28T01:48:13.445 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:48:13 smithi132 ceph-mon[35941]: pgmap v2568: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:48:13.624 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:48:13 smithi165 ceph-mon[36302]: pgmap v2568: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:48:13.647 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:48:13 smithi119 ceph-mon[31518]: pgmap v2568: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:48:15.445 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:48:15 smithi132 ceph-mon[35941]: pgmap v2569: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T01:48:15.624 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:48:15 smithi165 ceph-mon[36302]: pgmap v2569: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T01:48:15.647 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:48:15 smithi119 ceph-mon[31518]: pgmap v2569: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T01:48:16.624 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:48:16 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4355.723673 secs 2024-03-28T01:48:16.647 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:48:16 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4355.723673 secs 2024-03-28T01:48:16.694 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:48:16 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4355.723673 secs 2024-03-28T01:48:17.624 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:48:17 smithi165 ceph-mon[36302]: pgmap v2570: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-28T01:48:17.647 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:48:17 smithi119 ceph-mon[31518]: pgmap v2570: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-28T01:48:17.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:48:17 smithi132 ceph-mon[35941]: pgmap v2570: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-28T01:48:19.624 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:48:19 smithi165 ceph-mon[36302]: pgmap v2571: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:48:19.647 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:48:19 smithi119 ceph-mon[31518]: pgmap v2571: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:48:19.694 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:48:19 smithi132 ceph-mon[35941]: pgmap v2571: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:48:21.624 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:48:21 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4360.723761 secs 2024-03-28T01:48:21.624 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:48:21 smithi165 ceph-mon[36302]: pgmap v2572: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:48:21.647 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:48:21 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4360.723761 secs 2024-03-28T01:48:21.648 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:48:21 smithi119 ceph-mon[31518]: pgmap v2572: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:48:21.695 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:48:21 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4360.723761 secs 2024-03-28T01:48:21.695 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:48:21 smithi132 ceph-mon[35941]: pgmap v2572: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:48:23.874 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:48:23 smithi165 ceph-mon[36302]: pgmap v2573: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:48:23.897 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:48:23 smithi119 ceph-mon[31518]: pgmap v2573: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:48:23.944 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:48:23 smithi132 ceph-mon[35941]: pgmap v2573: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:48:25.874 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:48:25 smithi165 ceph-mon[36302]: pgmap v2574: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T01:48:25.874 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:48:25 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4365.723851 secs 2024-03-28T01:48:25.897 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:48:25 smithi119 ceph-mon[31518]: pgmap v2574: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T01:48:25.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:48:25 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4365.723851 secs 2024-03-28T01:48:25.945 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:48:25 smithi132 ceph-mon[35941]: pgmap v2574: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T01:48:25.945 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:48:25 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4365.723851 secs 2024-03-28T01:48:27.874 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:48:27 smithi165 ceph-mon[36302]: pgmap v2575: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-28T01:48:27.897 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:48:27 smithi119 ceph-mon[31518]: pgmap v2575: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-28T01:48:27.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:48:27 smithi132 ceph-mon[35941]: pgmap v2575: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-28T01:48:29.747 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:48:29 smithi132 ceph-mon[35941]: pgmap v2576: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:48:29.874 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:48:29 smithi165 ceph-mon[36302]: pgmap v2576: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:48:29.897 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:48:29 smithi119 ceph-mon[31518]: pgmap v2576: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:48:30.874 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:48:30 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4370.723892 secs 2024-03-28T01:48:30.897 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:48:30 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4370.723892 secs 2024-03-28T01:48:30.944 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:48:30 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4370.723892 secs 2024-03-28T01:48:31.874 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:48:31 smithi165 ceph-mon[36302]: pgmap v2577: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:48:31.897 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:48:31 smithi119 ceph-mon[31518]: pgmap v2577: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:48:31.944 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:48:31 smithi132 ceph-mon[35941]: pgmap v2577: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:48:33.874 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:48:33 smithi165 ceph-mon[36302]: pgmap v2578: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:48:33.897 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:48:33 smithi119 ceph-mon[31518]: pgmap v2578: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:48:33.944 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:48:33 smithi132 ceph-mon[35941]: pgmap v2578: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:48:35.874 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:48:35 smithi165 ceph-mon[36302]: pgmap v2579: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T01:48:35.874 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:48:35 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4375.723976 secs 2024-03-28T01:48:35.897 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:48:35 smithi119 ceph-mon[31518]: pgmap v2579: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T01:48:35.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:48:35 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4375.723976 secs 2024-03-28T01:48:35.945 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:48:35 smithi132 ceph-mon[35941]: pgmap v2579: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T01:48:35.945 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:48:35 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4375.723976 secs 2024-03-28T01:48:37.874 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:48:37 smithi165 ceph-mon[36302]: pgmap v2580: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-28T01:48:37.897 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:48:37 smithi119 ceph-mon[31518]: pgmap v2580: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-28T01:48:37.944 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:48:37 smithi132 ceph-mon[35941]: pgmap v2580: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-28T01:48:39.647 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:48:39 smithi119 ceph-mon[31518]: pgmap v2581: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:48:39.874 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:48:39 smithi165 ceph-mon[36302]: pgmap v2581: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:48:39.945 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:48:39 smithi132 ceph-mon[35941]: pgmap v2581: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:48:40.874 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:48:40 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4380.724065 secs 2024-03-28T01:48:40.897 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:48:40 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4380.724065 secs 2024-03-28T01:48:40.944 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:48:40 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4380.724065 secs 2024-03-28T01:48:41.874 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:48:41 smithi165 ceph-mon[36302]: pgmap v2582: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:48:41.897 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:48:41 smithi119 ceph-mon[31518]: pgmap v2582: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:48:41.944 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:48:41 smithi132 ceph-mon[35941]: pgmap v2582: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:48:43.874 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:48:43 smithi165 ceph-mon[36302]: pgmap v2583: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:48:43.897 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:48:43 smithi119 ceph-mon[31518]: pgmap v2583: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:48:43.944 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:48:43 smithi132 ceph-mon[35941]: pgmap v2583: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:48:45.874 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:48:45 smithi165 ceph-mon[36302]: pgmap v2584: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T01:48:45.874 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:48:45 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4385.724129 secs 2024-03-28T01:48:45.897 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:48:45 smithi119 ceph-mon[31518]: pgmap v2584: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T01:48:45.897 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:48:45 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4385.724129 secs 2024-03-28T01:48:45.944 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:48:45 smithi132 ceph-mon[35941]: pgmap v2584: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T01:48:45.944 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:48:45 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4385.724129 secs 2024-03-28T01:48:46.874 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:48:46 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:48:46.897 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:48:46 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:48:46.944 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:48:46 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:48:47.874 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:48:47 smithi165 ceph-mon[36302]: pgmap v2585: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-28T01:48:47.897 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:48:47 smithi119 ceph-mon[31518]: pgmap v2585: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-28T01:48:47.944 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:48:47 smithi132 ceph-mon[35941]: pgmap v2585: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-28T01:48:48.874 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:48:48 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:48:48.874 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:48:48 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:48:48.874 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:48:48 smithi165 ceph-mon[36302]: pgmap v2586: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T01:48:48.874 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:48:48 smithi165 ceph-mon[36302]: pgmap v2587: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:48:48.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:48:48 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:48:48.897 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:48:48 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:48:48.897 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:48:48 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:48:48.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:48:48 smithi119 ceph-mon[31518]: pgmap v2586: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T01:48:48.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:48:48 smithi119 ceph-mon[31518]: pgmap v2587: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:48:48.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:48:48 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:48:48.944 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:48:48 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:48:48.945 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:48:48 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:48:48.945 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:48:48 smithi132 ceph-mon[35941]: pgmap v2586: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T01:48:48.945 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:48:48 smithi132 ceph-mon[35941]: pgmap v2587: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:48:48.945 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:48:48 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:48:50.897 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:48:50 smithi119 ceph-mon[31518]: pgmap v2588: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:48:50.897 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:48:50 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4390.724165 secs 2024-03-28T01:48:50.944 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:48:50 smithi132 ceph-mon[35941]: pgmap v2588: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:48:50.945 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:48:50 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4390.724165 secs 2024-03-28T01:48:51.124 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:48:50 smithi165 ceph-mon[36302]: pgmap v2588: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:48:51.124 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:48:50 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4390.724165 secs 2024-03-28T01:48:52.944 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:48:52 smithi132 ceph-mon[35941]: pgmap v2589: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:48:53.126 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:48:52 smithi165 ceph-mon[36302]: pgmap v2589: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:48:53.147 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:48:52 smithi119 ceph-mon[31518]: pgmap v2589: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:48:54.926 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:48:54 smithi165 ceph-mon[36302]: pgmap v2590: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 5 op/s 2024-03-28T01:48:54.944 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:48:54 smithi132 ceph-mon[35941]: pgmap v2590: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 5 op/s 2024-03-28T01:48:55.147 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:48:54 smithi119 ceph-mon[31518]: pgmap v2590: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 5 op/s 2024-03-28T01:48:55.944 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:48:55 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4395.724248 secs 2024-03-28T01:48:56.124 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:48:55 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4395.724248 secs 2024-03-28T01:48:56.147 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:48:55 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4395.724248 secs 2024-03-28T01:48:56.944 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:48:56 smithi132 ceph-mon[35941]: pgmap v2591: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:48:57.124 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:48:56 smithi165 ceph-mon[36302]: pgmap v2591: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:48:57.147 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:48:56 smithi119 ceph-mon[31518]: pgmap v2591: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:48:58.944 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:48:58 smithi132 ceph-mon[35941]: pgmap v2592: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T01:48:59.124 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:48:58 smithi165 ceph-mon[36302]: pgmap v2592: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T01:48:59.147 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:48:58 smithi119 ceph-mon[31518]: pgmap v2592: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T01:48:59.945 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:48:59 smithi132 ceph-mon[35941]: pgmap v2593: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:49:00.124 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:48:59 smithi165 ceph-mon[36302]: pgmap v2593: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:49:00.147 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:48:59 smithi119 ceph-mon[31518]: pgmap v2593: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:49:00.944 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:49:00 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4400.724264 secs 2024-03-28T01:49:01.124 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:49:00 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4400.724264 secs 2024-03-28T01:49:01.147 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:49:00 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4400.724264 secs 2024-03-28T01:49:01.944 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:49:01 smithi132 ceph-mon[35941]: pgmap v2594: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:49:02.124 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:49:01 smithi165 ceph-mon[36302]: pgmap v2594: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:49:02.147 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:49:01 smithi119 ceph-mon[31518]: pgmap v2594: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:49:04.944 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:49:04 smithi132 ceph-mon[35941]: pgmap v2595: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:49:05.124 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:49:04 smithi165 ceph-mon[36302]: pgmap v2595: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:49:05.147 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:49:04 smithi119 ceph-mon[31518]: pgmap v2595: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:49:05.944 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:49:05 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4405.724327 secs 2024-03-28T01:49:06.124 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:49:05 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4405.724327 secs 2024-03-28T01:49:06.147 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:49:05 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4405.724327 secs 2024-03-28T01:49:06.944 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:49:06 smithi132 ceph-mon[35941]: pgmap v2596: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:49:07.124 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:49:06 smithi165 ceph-mon[36302]: pgmap v2596: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:49:07.147 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:49:06 smithi119 ceph-mon[31518]: pgmap v2596: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:49:08.944 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:49:08 smithi132 ceph-mon[35941]: pgmap v2597: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:49:09.124 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:49:08 smithi165 ceph-mon[36302]: pgmap v2597: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:49:09.147 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:49:08 smithi119 ceph-mon[31518]: pgmap v2597: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:49:10.944 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:49:10 smithi132 ceph-mon[35941]: pgmap v2598: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:49:10.944 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:49:10 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4410.724410 secs 2024-03-28T01:49:11.070 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:49:10 smithi119 ceph-mon[31518]: pgmap v2598: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:49:11.070 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:49:10 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4410.724410 secs 2024-03-28T01:49:11.124 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:49:10 smithi165 ceph-mon[36302]: pgmap v2598: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:49:11.124 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:49:10 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4410.724410 secs 2024-03-28T01:49:11.944 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:49:11 smithi132 ceph-mon[35941]: pgmap v2599: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:49:12.124 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:49:11 smithi165 ceph-mon[36302]: pgmap v2599: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:49:12.147 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:49:11 smithi119 ceph-mon[31518]: pgmap v2599: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:49:14.944 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:49:14 smithi132 ceph-mon[35941]: pgmap v2600: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:49:15.124 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:49:14 smithi165 ceph-mon[36302]: pgmap v2600: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:49:15.147 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:49:14 smithi119 ceph-mon[31518]: pgmap v2600: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:49:15.944 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:49:15 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4415.724500 secs 2024-03-28T01:49:16.124 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:49:15 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4415.724500 secs 2024-03-28T01:49:16.147 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:49:15 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4415.724500 secs 2024-03-28T01:49:16.944 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:49:16 smithi132 ceph-mon[35941]: pgmap v2601: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:49:17.124 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:49:16 smithi165 ceph-mon[36302]: pgmap v2601: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:49:17.147 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:49:16 smithi119 ceph-mon[31518]: pgmap v2601: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:49:18.944 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:49:18 smithi132 ceph-mon[35941]: pgmap v2602: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:49:19.124 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:49:18 smithi165 ceph-mon[36302]: pgmap v2602: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:49:19.147 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:49:18 smithi119 ceph-mon[31518]: pgmap v2602: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:49:20.944 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:49:20 smithi132 ceph-mon[35941]: pgmap v2603: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:49:20.944 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:49:20 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4420.724590 secs 2024-03-28T01:49:21.124 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:49:20 smithi165 ceph-mon[36302]: pgmap v2603: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:49:21.124 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:49:20 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4420.724590 secs 2024-03-28T01:49:21.147 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:49:20 smithi119 ceph-mon[31518]: pgmap v2603: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:49:21.147 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:49:20 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4420.724590 secs 2024-03-28T01:49:21.944 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:49:21 smithi132 ceph-mon[35941]: pgmap v2604: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:49:22.049 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:49:21 smithi119 ceph-mon[31518]: pgmap v2604: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:49:22.124 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:49:21 smithi165 ceph-mon[36302]: pgmap v2604: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:49:24.944 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:49:24 smithi132 ceph-mon[35941]: pgmap v2605: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:49:25.124 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:49:24 smithi165 ceph-mon[36302]: pgmap v2605: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:49:25.147 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:49:24 smithi119 ceph-mon[31518]: pgmap v2605: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:49:25.944 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:49:25 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4425.724653 secs 2024-03-28T01:49:26.124 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:49:25 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4425.724653 secs 2024-03-28T01:49:26.147 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:49:25 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4425.724653 secs 2024-03-28T01:49:26.944 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:49:26 smithi132 ceph-mon[35941]: pgmap v2606: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:49:27.124 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:49:26 smithi165 ceph-mon[36302]: pgmap v2606: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:49:27.147 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:49:26 smithi119 ceph-mon[31518]: pgmap v2606: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:49:28.944 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:49:28 smithi132 ceph-mon[35941]: pgmap v2607: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:49:29.124 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:49:28 smithi165 ceph-mon[36302]: pgmap v2607: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:49:29.147 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:49:28 smithi119 ceph-mon[31518]: pgmap v2607: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:49:29.944 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:49:29 smithi132 ceph-mon[35941]: pgmap v2608: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:49:30.124 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:49:29 smithi165 ceph-mon[36302]: pgmap v2608: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:49:30.147 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:49:29 smithi119 ceph-mon[31518]: pgmap v2608: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:49:30.944 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:49:30 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4430.724715 secs 2024-03-28T01:49:31.124 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:49:30 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4430.724715 secs 2024-03-28T01:49:31.147 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:49:30 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4430.724715 secs 2024-03-28T01:49:32.124 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:49:31 smithi165 ceph-mon[36302]: pgmap v2609: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:49:32.147 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:49:31 smithi119 ceph-mon[31518]: pgmap v2609: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:49:32.194 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:49:31 smithi132 ceph-mon[35941]: pgmap v2609: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:49:34.944 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:49:34 smithi132 ceph-mon[35941]: pgmap v2610: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:49:35.124 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:49:34 smithi165 ceph-mon[36302]: pgmap v2610: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:49:35.147 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:49:34 smithi119 ceph-mon[31518]: pgmap v2610: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:49:35.944 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:49:35 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4435.724804 secs 2024-03-28T01:49:36.124 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:49:35 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4435.724804 secs 2024-03-28T01:49:36.147 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:49:35 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4435.724804 secs 2024-03-28T01:49:36.944 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:49:36 smithi132 ceph-mon[35941]: pgmap v2611: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:49:37.124 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:49:36 smithi165 ceph-mon[36302]: pgmap v2611: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:49:37.147 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:49:36 smithi119 ceph-mon[31518]: pgmap v2611: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:49:37.944 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:49:37 smithi132 ceph-mon[35941]: pgmap v2612: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:49:38.124 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:49:37 smithi165 ceph-mon[36302]: pgmap v2612: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:49:38.146 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:49:37 smithi119 ceph-mon[31518]: pgmap v2612: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:49:40.944 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:49:40 smithi132 ceph-mon[35941]: pgmap v2613: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:49:40.944 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:49:40 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4440.724890 secs 2024-03-28T01:49:41.124 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:49:40 smithi165 ceph-mon[36302]: pgmap v2613: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:49:41.124 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:49:40 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4440.724890 secs 2024-03-28T01:49:41.147 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:49:40 smithi119 ceph-mon[31518]: pgmap v2613: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:49:41.147 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:49:40 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4440.724890 secs 2024-03-28T01:49:42.944 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:49:42 smithi132 ceph-mon[35941]: pgmap v2614: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:49:43.123 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:49:42 smithi165 ceph-mon[36302]: pgmap v2614: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:49:43.147 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:49:42 smithi119 ceph-mon[31518]: pgmap v2614: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:49:44.944 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:49:44 smithi132 ceph-mon[35941]: pgmap v2615: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:49:45.123 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:49:44 smithi165 ceph-mon[36302]: pgmap v2615: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:49:45.147 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:49:44 smithi119 ceph-mon[31518]: pgmap v2615: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:49:45.944 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:49:45 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4445.724975 secs 2024-03-28T01:49:46.124 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:49:45 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4445.724975 secs 2024-03-28T01:49:46.146 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:49:45 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4445.724975 secs 2024-03-28T01:49:46.944 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:49:46 smithi132 ceph-mon[35941]: pgmap v2616: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:49:47.124 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:49:46 smithi165 ceph-mon[36302]: pgmap v2616: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:49:47.146 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:49:46 smithi119 ceph-mon[31518]: pgmap v2616: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:49:48.874 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:49:48 smithi165 ceph-mon[36302]: pgmap v2617: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:49:48.874 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:49:48 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:49:48.874 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:49:48 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:49:48.874 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:49:48 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:49:48.897 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:49:48 smithi119 ceph-mon[31518]: pgmap v2617: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:49:48.897 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:49:48 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:49:48.897 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:49:48 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:49:48.897 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:49:48 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:49:48.944 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:49:48 smithi132 ceph-mon[35941]: pgmap v2617: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:49:48.944 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:49:48 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:49:48.944 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:49:48 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:49:48.944 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:49:48 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:49:49.944 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:49:49 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:49:49.944 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:49:49 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:49:49.944 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:49:49 smithi132 ceph-mon[35941]: pgmap v2618: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:49:50.021 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:49:49 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:49:50.021 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:49:49 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:49:50.021 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:49:49 smithi119 ceph-mon[31518]: pgmap v2618: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:49:50.123 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:49:49 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:49:50.124 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:49:49 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:49:50.124 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:49:49 smithi165 ceph-mon[36302]: pgmap v2618: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:49:51.374 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:49:51 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4450.725043 secs 2024-03-28T01:49:51.374 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:49:51 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:49:51.374 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:49:51 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:49:51.374 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:49:51 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:49:51.374 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:49:51 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:49:51.374 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:49:51 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:49:51.375 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:49:51 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:49:51.375 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:49:51 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:49:51.375 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:49:51 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:49:51.375 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:49:51 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:49:51.375 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:49:51 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:49:51.375 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:49:51 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:49:51.375 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:49:51 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:49:51.375 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:49:51 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:49:51.375 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:49:51 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:49:51.375 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:49:51 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:49:51.375 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:49:51 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:49:51.375 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:49:51 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:49:51.375 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:49:51 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:49:51.375 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:49:51 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:49:51.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:49:51 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:49:51.398 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:49:51 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4450.725043 secs 2024-03-28T01:49:51.398 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:49:51 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:49:51.398 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:49:51 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:49:51.398 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:49:51 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:49:51.398 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:49:51 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:49:51.398 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:49:51 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:49:51.398 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:49:51 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:49:51.398 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:49:51 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:49:51.398 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:49:51 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:49:51.398 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:49:51 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:49:51.398 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:49:51 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:49:51.398 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:49:51 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:49:51.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:49:51 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:49:51.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:49:51 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:49:51.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:49:51 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:49:51.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:49:51 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:49:51.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:49:51 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:49:51.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:49:51 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:49:51.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:49:51 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:49:51.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:49:51 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:49:51.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:49:51 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:49:51.444 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:49:51 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4450.725043 secs 2024-03-28T01:49:51.444 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:49:51 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:49:51.444 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:49:51 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:49:51.444 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:49:51 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:49:51.445 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:49:51 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:49:51.445 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:49:51 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:49:51.445 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:49:51 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:49:51.445 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:49:51 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:49:51.445 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:49:51 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:49:51.445 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:49:51 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:49:51.445 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:49:51 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:49:51.445 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:49:51 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:49:51.445 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:49:51 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:49:51.445 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:49:51 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:49:51.445 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:49:51 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:49:51.445 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:49:51 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:49:51.445 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:49:51 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:49:51.445 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:49:51 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:49:51.445 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:49:51 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:49:51.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:49:51 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:49:51.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:49:51 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:49:52.374 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:49:52 smithi165 ceph-mon[36302]: Adjusting osd_memory_target on smithi165 to 174.1M 2024-03-28T01:49:52.374 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:49:52 smithi165 ceph-mon[36302]: Unable to set osd_memory_target on smithi165 to 182635622: error parsing value: Value '182635622' is below minimum 939524096 2024-03-28T01:49:52.374 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:49:52 smithi165 ceph-mon[36302]: Adjusting osd_memory_target on smithi132 to 174.1M 2024-03-28T01:49:52.374 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:49:52 smithi165 ceph-mon[36302]: Unable to set osd_memory_target on smithi132 to 182640640: error parsing value: Value '182640640' is below minimum 939524096 2024-03-28T01:49:52.374 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:49:52 smithi165 ceph-mon[36302]: pgmap v2619: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:49:52.396 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:49:52 smithi119 ceph-mon[31518]: Adjusting osd_memory_target on smithi165 to 174.1M 2024-03-28T01:49:52.397 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:49:52 smithi119 ceph-mon[31518]: Unable to set osd_memory_target on smithi165 to 182635622: error parsing value: Value '182635622' is below minimum 939524096 2024-03-28T01:49:52.397 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:49:52 smithi119 ceph-mon[31518]: Adjusting osd_memory_target on smithi132 to 174.1M 2024-03-28T01:49:52.397 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:49:52 smithi119 ceph-mon[31518]: Unable to set osd_memory_target on smithi132 to 182640640: error parsing value: Value '182640640' is below minimum 939524096 2024-03-28T01:49:52.397 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:49:52 smithi119 ceph-mon[31518]: pgmap v2619: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:49:52.444 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:49:52 smithi132 ceph-mon[35941]: Adjusting osd_memory_target on smithi165 to 174.1M 2024-03-28T01:49:52.444 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:49:52 smithi132 ceph-mon[35941]: Unable to set osd_memory_target on smithi165 to 182635622: error parsing value: Value '182635622' is below minimum 939524096 2024-03-28T01:49:52.444 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:49:52 smithi132 ceph-mon[35941]: Adjusting osd_memory_target on smithi132 to 174.1M 2024-03-28T01:49:52.444 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:49:52 smithi132 ceph-mon[35941]: Unable to set osd_memory_target on smithi132 to 182640640: error parsing value: Value '182640640' is below minimum 939524096 2024-03-28T01:49:52.444 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:49:52 smithi132 ceph-mon[35941]: pgmap v2619: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:49:53.623 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:49:53 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:49:53.624 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:49:53 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:49:53.646 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:49:53 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:49:53.646 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:49:53 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:49:53.694 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:49:53 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:49:53.694 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:49:53 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:49:54.525 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:49:54 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:49:54.525 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:49:54 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:49:54.525 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:49:54 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:49:54.525 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:49:54 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:49:54.525 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:49:54 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:49:54.525 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:49:54 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:49:54.525 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:49:54 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:49:54.525 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:49:54 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:49:54.525 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:49:54 smithi165 ceph-mon[36302]: Adjusting osd_memory_target on smithi119 to 176.7M 2024-03-28T01:49:54.526 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:49:54 smithi165 ceph-mon[36302]: Unable to set osd_memory_target on smithi119 to 185372364: error parsing value: Value '185372364' is below minimum 939524096 2024-03-28T01:49:54.526 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:49:54 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:49:54.526 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:49:54 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:49:54.526 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:49:54 smithi165 ceph-mon[36302]: pgmap v2620: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T01:49:54.526 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:49:54 smithi165 ceph-mon[36302]: pgmap v2621: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T01:49:54.526 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:49:54 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:49:54.647 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:49:54 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:49:54.647 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:49:54 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:49:54.647 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:49:54 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:49:54.648 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:49:54 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:49:54.648 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:49:54 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:49:54.648 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:49:54 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:49:54.648 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:49:54 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:49:54.648 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:49:54 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:49:54.648 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:49:54 smithi119 ceph-mon[31518]: Adjusting osd_memory_target on smithi119 to 176.7M 2024-03-28T01:49:54.648 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:49:54 smithi119 ceph-mon[31518]: Unable to set osd_memory_target on smithi119 to 185372364: error parsing value: Value '185372364' is below minimum 939524096 2024-03-28T01:49:54.648 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:49:54 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:49:54.648 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:49:54 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:49:54.648 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:49:54 smithi119 ceph-mon[31518]: pgmap v2620: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T01:49:54.648 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:49:54 smithi119 ceph-mon[31518]: pgmap v2621: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T01:49:54.648 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:49:54 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:49:54.694 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:49:54 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:49:54.694 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:49:54 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:49:54.694 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:49:54 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:49:54.695 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:49:54 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:49:54.695 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:49:54 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:49:54.695 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:49:54 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:49:54.695 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:49:54 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:49:54.695 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:49:54 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T01:49:54.695 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:49:54 smithi132 ceph-mon[35941]: Adjusting osd_memory_target on smithi119 to 176.7M 2024-03-28T01:49:54.695 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:49:54 smithi132 ceph-mon[35941]: Unable to set osd_memory_target on smithi119 to 185372364: error parsing value: Value '185372364' is below minimum 939524096 2024-03-28T01:49:54.695 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:49:54 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:49:54.695 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:49:54 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:49:54.695 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:49:54 smithi132 ceph-mon[35941]: pgmap v2620: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T01:49:54.695 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:49:54 smithi132 ceph-mon[35941]: pgmap v2621: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T01:49:54.695 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:49:54 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:49:55.873 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:49:55 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4455.725103 secs 2024-03-28T01:49:55.897 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:49:55 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4455.725103 secs 2024-03-28T01:49:55.944 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:49:55 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4455.725103 secs 2024-03-28T01:49:56.873 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:49:56 smithi165 ceph-mon[36302]: pgmap v2622: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T01:49:56.897 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:49:56 smithi119 ceph-mon[31518]: pgmap v2622: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T01:49:56.944 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:49:56 smithi132 ceph-mon[35941]: pgmap v2622: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T01:49:58.873 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:49:58 smithi165 ceph-mon[36302]: pgmap v2623: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T01:49:58.896 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:49:58 smithi119 ceph-mon[31518]: pgmap v2623: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T01:49:58.944 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:49:58 smithi132 ceph-mon[35941]: pgmap v2623: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T01:50:00.873 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:50:00 smithi165 ceph-mon[36302]: pgmap v2624: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T01:50:00.874 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:50:00 smithi165 ceph-mon[36302]: overall HEALTH_WARN 5 failed cephadm daemon(s); 1 filesystem is degraded; 1 MDSs report slow requests 2024-03-28T01:50:00.874 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:50:00 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4460.725163 secs 2024-03-28T01:50:00.897 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:50:00 smithi119 ceph-mon[31518]: pgmap v2624: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T01:50:00.897 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:50:00 smithi119 ceph-mon[31518]: overall HEALTH_WARN 5 failed cephadm daemon(s); 1 filesystem is degraded; 1 MDSs report slow requests 2024-03-28T01:50:00.897 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:50:00 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4460.725163 secs 2024-03-28T01:50:00.944 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:50:00 smithi132 ceph-mon[35941]: pgmap v2624: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T01:50:00.944 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:50:00 smithi132 ceph-mon[35941]: overall HEALTH_WARN 5 failed cephadm daemon(s); 1 filesystem is degraded; 1 MDSs report slow requests 2024-03-28T01:50:00.944 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:50:00 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4460.725163 secs 2024-03-28T01:50:02.897 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:50:02 smithi119 ceph-mon[31518]: pgmap v2625: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T01:50:02.944 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:50:02 smithi132 ceph-mon[35941]: pgmap v2625: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T01:50:03.123 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:50:02 smithi165 ceph-mon[36302]: pgmap v2625: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T01:50:04.897 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:50:04 smithi119 ceph-mon[31518]: pgmap v2626: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T01:50:04.944 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:50:04 smithi132 ceph-mon[35941]: pgmap v2626: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T01:50:05.123 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:50:04 smithi165 ceph-mon[36302]: pgmap v2626: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T01:50:05.897 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:50:05 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4465.725225 secs 2024-03-28T01:50:05.944 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:50:05 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4465.725225 secs 2024-03-28T01:50:06.123 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:50:05 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4465.725225 secs 2024-03-28T01:50:06.896 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:50:06 smithi119 ceph-mon[31518]: pgmap v2627: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:50:06.944 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:50:06 smithi132 ceph-mon[35941]: pgmap v2627: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:50:07.123 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:50:06 smithi165 ceph-mon[36302]: pgmap v2627: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:50:08.944 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:50:08 smithi132 ceph-mon[35941]: pgmap v2628: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:50:09.123 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:50:08 smithi165 ceph-mon[36302]: pgmap v2628: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:50:09.146 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:50:08 smithi119 ceph-mon[31518]: pgmap v2628: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:50:10.944 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:50:10 smithi132 ceph-mon[35941]: pgmap v2629: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:50:10.944 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:50:10 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4470.725285 secs 2024-03-28T01:50:11.123 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:50:10 smithi165 ceph-mon[36302]: pgmap v2629: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:50:11.123 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:50:10 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4470.725285 secs 2024-03-28T01:50:11.147 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:50:10 smithi119 ceph-mon[31518]: pgmap v2629: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:50:11.147 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:50:10 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4470.725285 secs 2024-03-28T01:50:12.944 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:50:12 smithi132 ceph-mon[35941]: pgmap v2630: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:50:13.021 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:50:12 smithi119 ceph-mon[31518]: pgmap v2630: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:50:13.123 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:50:12 smithi165 ceph-mon[36302]: pgmap v2630: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:50:14.944 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:50:14 smithi132 ceph-mon[35941]: pgmap v2631: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:50:15.123 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:50:14 smithi165 ceph-mon[36302]: pgmap v2631: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:50:15.146 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:50:14 smithi119 ceph-mon[31518]: pgmap v2631: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:50:15.944 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:50:15 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4475.725345 secs 2024-03-28T01:50:16.123 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:50:15 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4475.725345 secs 2024-03-28T01:50:16.146 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:50:15 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4475.725345 secs 2024-03-28T01:50:16.944 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:50:16 smithi132 ceph-mon[35941]: pgmap v2632: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:50:17.123 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:50:16 smithi165 ceph-mon[36302]: pgmap v2632: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:50:17.146 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:50:16 smithi119 ceph-mon[31518]: pgmap v2632: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:50:18.944 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:50:18 smithi132 ceph-mon[35941]: pgmap v2633: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:50:19.123 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:50:18 smithi165 ceph-mon[36302]: pgmap v2633: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:50:19.146 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:50:18 smithi119 ceph-mon[31518]: pgmap v2633: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:50:20.944 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:50:20 smithi132 ceph-mon[35941]: pgmap v2634: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:50:20.944 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:50:20 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4480.725433 secs 2024-03-28T01:50:21.123 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:50:20 smithi165 ceph-mon[36302]: pgmap v2634: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:50:21.123 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:50:20 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4480.725433 secs 2024-03-28T01:50:21.146 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:50:20 smithi119 ceph-mon[31518]: pgmap v2634: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:50:21.147 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:50:20 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4480.725433 secs 2024-03-28T01:50:22.048 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:50:21 smithi119 ceph-mon[31518]: pgmap v2635: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:50:22.123 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:50:21 smithi165 ceph-mon[36302]: pgmap v2635: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:50:22.193 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:50:21 smithi132 ceph-mon[35941]: pgmap v2635: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:50:24.896 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:50:24 smithi119 ceph-mon[31518]: pgmap v2636: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:50:24.943 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:50:24 smithi132 ceph-mon[35941]: pgmap v2636: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:50:25.123 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:50:24 smithi165 ceph-mon[36302]: pgmap v2636: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:50:25.896 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:50:25 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4485.725522 secs 2024-03-28T01:50:25.944 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:50:25 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4485.725522 secs 2024-03-28T01:50:26.123 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:50:25 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4485.725522 secs 2024-03-28T01:50:26.896 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:50:26 smithi119 ceph-mon[31518]: pgmap v2637: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:50:26.944 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:50:26 smithi132 ceph-mon[35941]: pgmap v2637: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:50:27.123 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:50:26 smithi165 ceph-mon[36302]: pgmap v2637: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:50:28.896 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:50:28 smithi119 ceph-mon[31518]: pgmap v2638: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:50:28.943 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:50:28 smithi132 ceph-mon[35941]: pgmap v2638: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:50:29.123 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:50:28 smithi165 ceph-mon[36302]: pgmap v2638: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:50:30.944 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:50:30 smithi132 ceph-mon[35941]: pgmap v2639: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:50:30.944 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:50:30 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4490.725612 secs 2024-03-28T01:50:31.123 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:50:30 smithi165 ceph-mon[36302]: pgmap v2639: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:50:31.123 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:50:30 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4490.725612 secs 2024-03-28T01:50:31.146 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:50:30 smithi119 ceph-mon[31518]: pgmap v2639: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:50:31.146 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:50:30 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4490.725612 secs 2024-03-28T01:50:32.943 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:50:32 smithi132 ceph-mon[35941]: pgmap v2640: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T01:50:33.123 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:50:32 smithi165 ceph-mon[36302]: pgmap v2640: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T01:50:33.146 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:50:32 smithi119 ceph-mon[31518]: pgmap v2640: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T01:50:34.943 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:50:34 smithi132 ceph-mon[35941]: pgmap v2641: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T01:50:35.123 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:50:34 smithi165 ceph-mon[36302]: pgmap v2641: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T01:50:35.146 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:50:34 smithi119 ceph-mon[31518]: pgmap v2641: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T01:50:35.943 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:50:35 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4495.725655 secs 2024-03-28T01:50:36.123 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:50:35 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4495.725655 secs 2024-03-28T01:50:36.146 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:50:35 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4495.725655 secs 2024-03-28T01:50:36.943 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:50:36 smithi132 ceph-mon[35941]: pgmap v2642: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:50:37.123 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:50:36 smithi165 ceph-mon[36302]: pgmap v2642: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:50:37.146 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:50:36 smithi119 ceph-mon[31518]: pgmap v2642: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:50:38.943 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:50:38 smithi132 ceph-mon[35941]: pgmap v2643: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T01:50:39.123 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:50:38 smithi165 ceph-mon[36302]: pgmap v2643: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T01:50:39.146 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:50:38 smithi119 ceph-mon[31518]: pgmap v2643: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T01:50:40.943 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:50:40 smithi132 ceph-mon[35941]: pgmap v2644: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:50:40.944 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:50:40 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4500.725746 secs 2024-03-28T01:50:41.123 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:50:40 smithi165 ceph-mon[36302]: pgmap v2644: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:50:41.123 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:50:40 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4500.725746 secs 2024-03-28T01:50:41.146 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:50:40 smithi119 ceph-mon[31518]: pgmap v2644: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:50:41.146 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:50:40 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4500.725746 secs 2024-03-28T01:50:42.943 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:50:42 smithi132 ceph-mon[35941]: pgmap v2645: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:50:43.123 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:50:42 smithi165 ceph-mon[36302]: pgmap v2645: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:50:43.146 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:50:42 smithi119 ceph-mon[31518]: pgmap v2645: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:50:44.943 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:50:44 smithi132 ceph-mon[35941]: pgmap v2646: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:50:45.123 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:50:44 smithi165 ceph-mon[36302]: pgmap v2646: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:50:45.146 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:50:44 smithi119 ceph-mon[31518]: pgmap v2646: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:50:46.123 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:50:45 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4505.725830 secs 2024-03-28T01:50:46.125 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:50:45 smithi165 ceph-mon[36302]: pgmap v2647: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:50:46.146 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:50:45 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4505.725830 secs 2024-03-28T01:50:46.146 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:50:45 smithi119 ceph-mon[31518]: pgmap v2647: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:50:46.193 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:50:45 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4505.725830 secs 2024-03-28T01:50:46.193 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:50:45 smithi132 ceph-mon[35941]: pgmap v2647: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:50:48.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:50:48 smithi119 ceph-mon[31518]: pgmap v2648: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:50:48.943 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:50:48 smithi132 ceph-mon[35941]: pgmap v2648: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:50:49.123 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:50:48 smithi165 ceph-mon[36302]: pgmap v2648: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:50:50.943 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:50:50 smithi132 ceph-mon[35941]: pgmap v2649: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:50:50.944 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:50:50 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4510.725897 secs 2024-03-28T01:50:51.123 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:50:50 smithi165 ceph-mon[36302]: pgmap v2649: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:50:51.123 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:50:50 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4510.725897 secs 2024-03-28T01:50:51.146 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:50:50 smithi119 ceph-mon[31518]: pgmap v2649: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:50:51.146 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:50:50 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4510.725897 secs 2024-03-28T01:50:52.943 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:50:52 smithi132 ceph-mon[35941]: pgmap v2650: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:50:52.949 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:50:52 smithi119 ceph-mon[31518]: pgmap v2650: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:50:53.123 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:50:52 smithi165 ceph-mon[36302]: pgmap v2650: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:50:54.924 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:50:54 smithi165 ceph-mon[36302]: pgmap v2651: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:50:54.924 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:50:54 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:50:54.924 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:50:54 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:50:54.925 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:50:54 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:50:54.945 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:50:54 smithi132 ceph-mon[35941]: pgmap v2651: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:50:54.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:50:54 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:50:54.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:50:54 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:50:54.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:50:54 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:50:55.146 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:50:54 smithi119 ceph-mon[31518]: pgmap v2651: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:50:55.147 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:50:54 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:50:55.147 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:50:54 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:50:55.147 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:50:54 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:50:55.943 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:50:55 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4515.725958 secs 2024-03-28T01:50:55.944 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:50:55 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:50:55.944 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:50:55 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:50:55.944 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:50:55 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:50:56.123 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:50:55 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4515.725958 secs 2024-03-28T01:50:56.123 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:50:55 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:50:56.123 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:50:55 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:50:56.123 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:50:55 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:50:56.146 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:50:55 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4515.725958 secs 2024-03-28T01:50:56.146 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:50:55 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:50:56.147 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:50:55 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:50:56.147 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:50:55 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:50:56.943 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:50:56 smithi132 ceph-mon[35941]: pgmap v2652: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 7 op/s 2024-03-28T01:50:56.943 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:50:56 smithi132 ceph-mon[35941]: pgmap v2653: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:50:57.123 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:50:56 smithi165 ceph-mon[36302]: pgmap v2652: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 7 op/s 2024-03-28T01:50:57.123 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:50:56 smithi165 ceph-mon[36302]: pgmap v2653: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:50:57.146 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:50:56 smithi119 ceph-mon[31518]: pgmap v2652: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 7 op/s 2024-03-28T01:50:57.146 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:50:56 smithi119 ceph-mon[31518]: pgmap v2653: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:50:58.943 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:50:58 smithi132 ceph-mon[35941]: pgmap v2654: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:50:59.101 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:50:58 smithi119 ceph-mon[31518]: pgmap v2654: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:50:59.123 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:50:58 smithi165 ceph-mon[36302]: pgmap v2654: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:51:00.943 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:51:00 smithi132 ceph-mon[35941]: pgmap v2655: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:51:00.944 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:51:00 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4520.726049 secs 2024-03-28T01:51:01.123 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:51:00 smithi165 ceph-mon[36302]: pgmap v2655: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:51:01.123 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:51:00 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4520.726049 secs 2024-03-28T01:51:01.146 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:51:00 smithi119 ceph-mon[31518]: pgmap v2655: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:51:01.146 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:51:00 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4520.726049 secs 2024-03-28T01:51:02.943 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:51:02 smithi132 ceph-mon[35941]: pgmap v2656: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:51:03.123 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:51:02 smithi165 ceph-mon[36302]: pgmap v2656: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:51:03.146 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:51:02 smithi119 ceph-mon[31518]: pgmap v2656: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:51:04.943 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:51:04 smithi132 ceph-mon[35941]: pgmap v2657: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:51:05.123 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:51:04 smithi165 ceph-mon[36302]: pgmap v2657: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:51:05.146 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:51:04 smithi119 ceph-mon[31518]: pgmap v2657: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:51:06.123 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:51:05 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4525.726135 secs 2024-03-28T01:51:06.123 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:51:05 smithi165 ceph-mon[36302]: pgmap v2658: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T01:51:06.146 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:51:05 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4525.726135 secs 2024-03-28T01:51:06.146 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:51:05 smithi119 ceph-mon[31518]: pgmap v2658: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T01:51:06.193 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:51:05 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4525.726135 secs 2024-03-28T01:51:06.194 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:51:05 smithi132 ceph-mon[35941]: pgmap v2658: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T01:51:08.625 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:51:08 smithi165 ceph-mon[36302]: pgmap v2659: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:51:08.646 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:51:08 smithi119 ceph-mon[31518]: pgmap v2659: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:51:08.693 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:51:08 smithi132 ceph-mon[35941]: pgmap v2659: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:51:10.623 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:51:10 smithi165 ceph-mon[36302]: pgmap v2660: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:51:10.646 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:51:10 smithi119 ceph-mon[31518]: pgmap v2660: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:51:10.693 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:51:10 smithi132 ceph-mon[35941]: pgmap v2660: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:51:11.623 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:51:11 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4530.726223 secs 2024-03-28T01:51:11.646 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:51:11 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4530.726223 secs 2024-03-28T01:51:11.693 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:51:11 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4530.726223 secs 2024-03-28T01:51:12.623 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:51:12 smithi165 ceph-mon[36302]: pgmap v2661: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:51:12.646 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:51:12 smithi119 ceph-mon[31518]: pgmap v2661: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:51:12.693 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:51:12 smithi132 ceph-mon[35941]: pgmap v2661: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:51:14.623 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:51:14 smithi165 ceph-mon[36302]: pgmap v2662: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:51:14.646 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:51:14 smithi119 ceph-mon[31518]: pgmap v2662: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:51:14.693 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:51:14 smithi132 ceph-mon[35941]: pgmap v2662: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:51:16.623 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:51:16 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4535.726284 secs 2024-03-28T01:51:16.624 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:51:16 smithi165 ceph-mon[36302]: pgmap v2663: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:51:16.646 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:51:16 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4535.726284 secs 2024-03-28T01:51:16.646 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:51:16 smithi119 ceph-mon[31518]: pgmap v2663: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:51:16.693 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:51:16 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4535.726284 secs 2024-03-28T01:51:16.693 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:51:16 smithi132 ceph-mon[35941]: pgmap v2663: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:51:18.646 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:51:18 smithi119 ceph-mon[31518]: pgmap v2664: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:51:18.693 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:51:18 smithi132 ceph-mon[35941]: pgmap v2664: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:51:18.873 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:51:18 smithi165 ceph-mon[36302]: pgmap v2664: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:51:20.646 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:51:20 smithi119 ceph-mon[31518]: pgmap v2665: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:51:20.693 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:51:20 smithi132 ceph-mon[35941]: pgmap v2665: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:51:20.873 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:51:20 smithi165 ceph-mon[36302]: pgmap v2665: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:51:21.646 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:51:21 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4540.726346 secs 2024-03-28T01:51:21.693 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:51:21 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4540.726346 secs 2024-03-28T01:51:21.873 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:51:21 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4540.726346 secs 2024-03-28T01:51:22.646 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:51:22 smithi119 ceph-mon[31518]: pgmap v2666: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:51:22.693 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:51:22 smithi132 ceph-mon[35941]: pgmap v2666: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:51:22.873 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:51:22 smithi165 ceph-mon[36302]: pgmap v2666: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:51:24.646 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:51:24 smithi119 ceph-mon[31518]: pgmap v2667: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:51:24.674 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:51:24 smithi165 ceph-mon[36302]: pgmap v2667: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:51:24.693 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:51:24 smithi132 ceph-mon[35941]: pgmap v2667: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:51:26.693 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:51:26 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4545.726452 secs 2024-03-28T01:51:26.693 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:51:26 smithi132 ceph-mon[35941]: pgmap v2668: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:51:26.873 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:51:26 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4545.726452 secs 2024-03-28T01:51:26.873 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:51:26 smithi165 ceph-mon[36302]: pgmap v2668: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:51:26.896 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:51:26 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4545.726452 secs 2024-03-28T01:51:26.896 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:51:26 smithi119 ceph-mon[31518]: pgmap v2668: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:51:28.693 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:51:28 smithi132 ceph-mon[35941]: pgmap v2669: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:51:28.873 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:51:28 smithi165 ceph-mon[36302]: pgmap v2669: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:51:28.896 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:51:28 smithi119 ceph-mon[31518]: pgmap v2669: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:51:30.646 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:51:30 smithi119 ceph-mon[31518]: pgmap v2670: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:51:30.693 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:51:30 smithi132 ceph-mon[35941]: pgmap v2670: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:51:30.873 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:51:30 smithi165 ceph-mon[36302]: pgmap v2670: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:51:31.693 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:51:31 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4550.726515 secs 2024-03-28T01:51:31.873 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:51:31 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4550.726515 secs 2024-03-28T01:51:31.896 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:51:31 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4550.726515 secs 2024-03-28T01:51:32.693 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:51:32 smithi132 ceph-mon[35941]: pgmap v2671: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:51:32.873 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:51:32 smithi165 ceph-mon[36302]: pgmap v2671: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:51:32.896 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:51:32 smithi119 ceph-mon[31518]: pgmap v2671: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:51:34.693 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:51:34 smithi132 ceph-mon[35941]: pgmap v2672: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:51:34.873 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:51:34 smithi165 ceph-mon[36302]: pgmap v2672: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:51:34.896 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:51:34 smithi119 ceph-mon[31518]: pgmap v2672: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:51:36.693 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:51:36 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4555.726602 secs 2024-03-28T01:51:36.693 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:51:36 smithi132 ceph-mon[35941]: pgmap v2673: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:51:36.873 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:51:36 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4555.726602 secs 2024-03-28T01:51:36.873 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:51:36 smithi165 ceph-mon[36302]: pgmap v2673: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:51:36.896 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:51:36 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4555.726602 secs 2024-03-28T01:51:36.896 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:51:36 smithi119 ceph-mon[31518]: pgmap v2673: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:51:38.693 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:51:38 smithi132 ceph-mon[35941]: pgmap v2674: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:51:38.873 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:51:38 smithi165 ceph-mon[36302]: pgmap v2674: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:51:38.896 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:51:38 smithi119 ceph-mon[31518]: pgmap v2674: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:51:40.693 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:51:40 smithi132 ceph-mon[35941]: pgmap v2675: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:51:40.873 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:51:40 smithi165 ceph-mon[36302]: pgmap v2675: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:51:40.896 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:51:40 smithi119 ceph-mon[31518]: pgmap v2675: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:51:41.873 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:51:41 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4560.726665 secs 2024-03-28T01:51:41.896 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:51:41 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4560.726665 secs 2024-03-28T01:51:41.943 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:51:41 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4560.726665 secs 2024-03-28T01:51:42.873 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:51:42 smithi165 ceph-mon[36302]: pgmap v2676: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:51:42.896 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:51:42 smithi119 ceph-mon[31518]: pgmap v2676: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:51:42.943 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:51:42 smithi132 ceph-mon[35941]: pgmap v2676: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:51:44.873 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:51:44 smithi165 ceph-mon[36302]: pgmap v2677: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:51:44.896 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:51:44 smithi119 ceph-mon[31518]: pgmap v2677: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:51:44.943 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:51:44 smithi132 ceph-mon[35941]: pgmap v2677: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:51:45.874 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:51:45 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4565.726728 secs 2024-03-28T01:51:45.896 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:51:45 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4565.726728 secs 2024-03-28T01:51:45.943 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:51:45 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4565.726728 secs 2024-03-28T01:51:46.873 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:51:46 smithi165 ceph-mon[36302]: pgmap v2678: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:51:46.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:51:46 smithi119 ceph-mon[31518]: pgmap v2678: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:51:46.943 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:51:46 smithi132 ceph-mon[35941]: pgmap v2678: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:51:48.873 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:51:48 smithi165 ceph-mon[36302]: pgmap v2679: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:51:48.896 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:51:48 smithi119 ceph-mon[31518]: pgmap v2679: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:51:48.945 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:51:48 smithi132 ceph-mon[35941]: pgmap v2679: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:51:50.873 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:51:50 smithi165 ceph-mon[36302]: pgmap v2680: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:51:50.873 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:51:50 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4570.726743 secs 2024-03-28T01:51:50.896 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:51:50 smithi119 ceph-mon[31518]: pgmap v2680: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:51:50.896 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:51:50 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4570.726743 secs 2024-03-28T01:51:50.943 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:51:50 smithi132 ceph-mon[35941]: pgmap v2680: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:51:50.943 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:51:50 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4570.726743 secs 2024-03-28T01:51:52.873 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:51:52 smithi165 ceph-mon[36302]: pgmap v2681: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:51:52.896 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:51:52 smithi119 ceph-mon[31518]: pgmap v2681: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:51:52.943 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:51:52 smithi132 ceph-mon[35941]: pgmap v2681: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:51:54.873 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:51:54 smithi165 ceph-mon[36302]: pgmap v2682: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:51:54.896 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:51:54 smithi119 ceph-mon[31518]: pgmap v2682: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:51:54.943 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:51:54 smithi132 ceph-mon[35941]: pgmap v2682: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:51:55.873 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:51:55 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4575.726803 secs 2024-03-28T01:51:55.896 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:51:55 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4575.726803 secs 2024-03-28T01:51:55.943 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:51:55 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4575.726803 secs 2024-03-28T01:51:56.873 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:51:56 smithi165 ceph-mon[36302]: pgmap v2683: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:51:56.873 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:51:56 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:51:56.895 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:51:56 smithi119 ceph-mon[31518]: pgmap v2683: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:51:56.896 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:51:56 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:51:56.943 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:51:56 smithi132 ceph-mon[35941]: pgmap v2683: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:51:56.943 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:51:56 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:51:57.873 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:51:57 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:51:57.873 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:51:57 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:51:57.873 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:51:57 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:51:57.896 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:51:57 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:51:57.896 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:51:57 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:51:57.896 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:51:57 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:51:57.943 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:51:57 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:51:57.943 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:51:57 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:51:57.943 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:51:57 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:51:58.873 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:51:58 smithi165 ceph-mon[36302]: pgmap v2684: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T01:51:58.873 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:51:58 smithi165 ceph-mon[36302]: pgmap v2685: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:51:58.896 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:51:58 smithi119 ceph-mon[31518]: pgmap v2684: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T01:51:58.896 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:51:58 smithi119 ceph-mon[31518]: pgmap v2685: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:51:58.943 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:51:58 smithi132 ceph-mon[35941]: pgmap v2684: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T01:51:58.943 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:51:58 smithi132 ceph-mon[35941]: pgmap v2685: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:52:00.873 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:52:00 smithi165 ceph-mon[36302]: pgmap v2686: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-28T01:52:00.873 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:52:00 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4580.726893 secs 2024-03-28T01:52:00.896 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:52:00 smithi119 ceph-mon[31518]: pgmap v2686: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-28T01:52:00.896 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:52:00 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4580.726893 secs 2024-03-28T01:52:00.943 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:52:00 smithi132 ceph-mon[35941]: pgmap v2686: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-28T01:52:00.943 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:52:00 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4580.726893 secs 2024-03-28T01:52:02.873 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:52:02 smithi165 ceph-mon[36302]: pgmap v2687: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:52:02.896 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:52:02 smithi119 ceph-mon[31518]: pgmap v2687: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:52:02.943 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:52:02 smithi132 ceph-mon[35941]: pgmap v2687: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:52:04.872 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:52:04 smithi165 ceph-mon[36302]: pgmap v2688: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:52:04.896 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:52:04 smithi119 ceph-mon[31518]: pgmap v2688: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:52:04.943 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:52:04 smithi132 ceph-mon[35941]: pgmap v2688: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:52:05.872 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:52:05 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4585.726955 secs 2024-03-28T01:52:05.896 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:52:05 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4585.726955 secs 2024-03-28T01:52:05.943 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:52:05 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4585.726955 secs 2024-03-28T01:52:06.872 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:52:06 smithi165 ceph-mon[36302]: pgmap v2689: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:52:06.896 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:52:06 smithi119 ceph-mon[31518]: pgmap v2689: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:52:06.943 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:52:06 smithi132 ceph-mon[35941]: pgmap v2689: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:52:08.872 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:52:08 smithi165 ceph-mon[36302]: pgmap v2690: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T01:52:08.896 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:52:08 smithi119 ceph-mon[31518]: pgmap v2690: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T01:52:08.943 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:52:08 smithi132 ceph-mon[35941]: pgmap v2690: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T01:52:10.873 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:52:10 smithi165 ceph-mon[36302]: pgmap v2691: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:52:10.873 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:52:10 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4590.727039 secs 2024-03-28T01:52:10.896 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:52:10 smithi119 ceph-mon[31518]: pgmap v2691: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:52:10.896 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:52:10 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4590.727039 secs 2024-03-28T01:52:10.943 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:52:10 smithi132 ceph-mon[35941]: pgmap v2691: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:52:10.943 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:52:10 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4590.727039 secs 2024-03-28T01:52:12.872 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:52:12 smithi165 ceph-mon[36302]: pgmap v2692: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:52:12.896 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:52:12 smithi119 ceph-mon[31518]: pgmap v2692: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:52:12.943 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:52:12 smithi132 ceph-mon[35941]: pgmap v2692: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:52:14.873 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:52:14 smithi165 ceph-mon[36302]: pgmap v2693: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:52:14.896 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:52:14 smithi119 ceph-mon[31518]: pgmap v2693: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:52:14.943 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:52:14 smithi132 ceph-mon[35941]: pgmap v2693: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:52:15.872 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:52:15 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4595.727129 secs 2024-03-28T01:52:15.895 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:52:15 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4595.727129 secs 2024-03-28T01:52:15.943 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:52:15 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4595.727129 secs 2024-03-28T01:52:16.872 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:52:16 smithi165 ceph-mon[36302]: pgmap v2694: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:52:16.896 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:52:16 smithi119 ceph-mon[31518]: pgmap v2694: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:52:16.943 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:52:16 smithi132 ceph-mon[35941]: pgmap v2694: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:52:18.872 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:52:18 smithi165 ceph-mon[36302]: pgmap v2695: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:52:18.896 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:52:18 smithi119 ceph-mon[31518]: pgmap v2695: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:52:18.943 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:52:18 smithi132 ceph-mon[35941]: pgmap v2695: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:52:20.873 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:52:20 smithi165 ceph-mon[36302]: pgmap v2696: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:52:20.873 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:52:20 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4600.727218 secs 2024-03-28T01:52:20.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:52:20 smithi119 ceph-mon[31518]: pgmap v2696: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:52:20.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:52:20 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4600.727218 secs 2024-03-28T01:52:20.943 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:52:20 smithi132 ceph-mon[35941]: pgmap v2696: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:52:20.943 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:52:20 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4600.727218 secs 2024-03-28T01:52:22.872 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:52:22 smithi165 ceph-mon[36302]: pgmap v2697: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:52:22.896 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:52:22 smithi119 ceph-mon[31518]: pgmap v2697: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:52:22.943 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:52:22 smithi132 ceph-mon[35941]: pgmap v2697: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:52:24.872 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:52:24 smithi165 ceph-mon[36302]: pgmap v2698: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:52:24.896 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:52:24 smithi119 ceph-mon[31518]: pgmap v2698: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:52:24.943 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:52:24 smithi132 ceph-mon[35941]: pgmap v2698: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:52:25.872 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:52:25 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4605.727305 secs 2024-03-28T01:52:25.896 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:52:25 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4605.727305 secs 2024-03-28T01:52:25.943 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:52:25 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4605.727305 secs 2024-03-28T01:52:26.872 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:52:26 smithi165 ceph-mon[36302]: pgmap v2699: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:52:26.896 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:52:26 smithi119 ceph-mon[31518]: pgmap v2699: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:52:26.943 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:52:26 smithi132 ceph-mon[35941]: pgmap v2699: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:52:28.872 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:52:28 smithi165 ceph-mon[36302]: pgmap v2700: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:52:28.895 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:52:28 smithi119 ceph-mon[31518]: pgmap v2700: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:52:28.943 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:52:28 smithi132 ceph-mon[35941]: pgmap v2700: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:52:30.873 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:52:30 smithi165 ceph-mon[36302]: pgmap v2701: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:52:30.873 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:52:30 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4610.727394 secs 2024-03-28T01:52:30.896 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:52:30 smithi119 ceph-mon[31518]: pgmap v2701: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:52:30.896 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:52:30 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4610.727394 secs 2024-03-28T01:52:30.943 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:52:30 smithi132 ceph-mon[35941]: pgmap v2701: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:52:30.943 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:52:30 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4610.727394 secs 2024-03-28T01:52:32.873 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:52:32 smithi165 ceph-mon[36302]: pgmap v2702: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:52:32.895 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:52:32 smithi119 ceph-mon[31518]: pgmap v2702: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:52:32.943 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:52:32 smithi132 ceph-mon[35941]: pgmap v2702: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:52:34.872 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:52:34 smithi165 ceph-mon[36302]: pgmap v2703: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:52:34.896 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:52:34 smithi119 ceph-mon[31518]: pgmap v2703: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:52:34.943 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:52:34 smithi132 ceph-mon[35941]: pgmap v2703: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:52:35.872 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:52:35 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4615.727482 secs 2024-03-28T01:52:35.895 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:52:35 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4615.727482 secs 2024-03-28T01:52:35.943 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:52:35 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4615.727482 secs 2024-03-28T01:52:36.872 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:52:36 smithi165 ceph-mon[36302]: pgmap v2704: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:52:36.895 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:52:36 smithi119 ceph-mon[31518]: pgmap v2704: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:52:36.943 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:52:36 smithi132 ceph-mon[35941]: pgmap v2704: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:52:38.896 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:52:38 smithi119 ceph-mon[31518]: pgmap v2705: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:52:38.943 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:52:38 smithi132 ceph-mon[35941]: pgmap v2705: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:52:39.122 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:52:38 smithi165 ceph-mon[36302]: pgmap v2705: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:52:40.896 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:52:40 smithi119 ceph-mon[31518]: pgmap v2706: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:52:40.896 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:52:40 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4620.727525 secs 2024-03-28T01:52:40.943 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:52:40 smithi132 ceph-mon[35941]: pgmap v2706: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:52:40.943 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:52:40 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4620.727525 secs 2024-03-28T01:52:41.122 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:52:40 smithi165 ceph-mon[36302]: pgmap v2706: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:52:41.123 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:52:40 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4620.727525 secs 2024-03-28T01:52:42.895 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:52:42 smithi119 ceph-mon[31518]: pgmap v2707: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:52:42.943 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:52:42 smithi132 ceph-mon[35941]: pgmap v2707: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:52:43.122 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:52:42 smithi165 ceph-mon[36302]: pgmap v2707: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:52:44.895 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:52:44 smithi119 ceph-mon[31518]: pgmap v2708: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:52:44.942 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:52:44 smithi132 ceph-mon[35941]: pgmap v2708: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:52:45.122 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:52:44 smithi165 ceph-mon[36302]: pgmap v2708: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:52:45.943 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:52:45 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4625.727612 secs 2024-03-28T01:52:46.122 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:52:45 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4625.727612 secs 2024-03-28T01:52:46.145 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:52:45 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4625.727612 secs 2024-03-28T01:52:46.943 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:52:46 smithi132 ceph-mon[35941]: pgmap v2709: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:52:47.122 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:52:46 smithi165 ceph-mon[36302]: pgmap v2709: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:52:47.145 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:52:46 smithi119 ceph-mon[31518]: pgmap v2709: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:52:48.943 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:52:48 smithi132 ceph-mon[35941]: pgmap v2710: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:52:49.122 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:52:48 smithi165 ceph-mon[36302]: pgmap v2710: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:52:49.145 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:52:48 smithi119 ceph-mon[31518]: pgmap v2710: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:52:50.943 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:52:50 smithi132 ceph-mon[35941]: pgmap v2711: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:52:50.943 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:52:50 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4630.727703 secs 2024-03-28T01:52:51.123 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:52:50 smithi165 ceph-mon[36302]: pgmap v2711: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:52:51.123 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:52:50 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4630.727703 secs 2024-03-28T01:52:51.145 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:52:50 smithi119 ceph-mon[31518]: pgmap v2711: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:52:51.146 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:52:50 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4630.727703 secs 2024-03-28T01:52:52.943 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:52:52 smithi132 ceph-mon[35941]: pgmap v2712: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:52:52.948 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:52:52 smithi119 ceph-mon[31518]: pgmap v2712: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:52:53.122 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:52:52 smithi165 ceph-mon[36302]: pgmap v2712: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:52:54.924 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:52:54 smithi165 ceph-mon[36302]: pgmap v2713: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:52:54.942 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:52:54 smithi132 ceph-mon[35941]: pgmap v2713: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:52:55.145 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:52:54 smithi119 ceph-mon[31518]: pgmap v2713: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:52:55.942 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:52:55 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4635.727744 secs 2024-03-28T01:52:56.122 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:52:55 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4635.727744 secs 2024-03-28T01:52:56.145 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:52:55 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4635.727744 secs 2024-03-28T01:52:56.942 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:52:56 smithi132 ceph-mon[35941]: pgmap v2714: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:52:57.122 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:52:56 smithi165 ceph-mon[36302]: pgmap v2714: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:52:57.145 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:52:56 smithi119 ceph-mon[31518]: pgmap v2714: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:52:57.943 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:52:57 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:52:58.122 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:52:57 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:52:58.145 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:52:57 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:52:58.942 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:52:58 smithi132 ceph-mon[35941]: pgmap v2715: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:52:59.123 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:52:58 smithi165 ceph-mon[36302]: pgmap v2715: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:52:59.145 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:52:58 smithi119 ceph-mon[31518]: pgmap v2715: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:53:01.001 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:53:00 smithi132 ceph-mon[35941]: pgmap v2716: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:53:01.001 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:53:00 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4640.727789 secs 2024-03-28T01:53:01.057 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:53:00 smithi119 ceph-mon[31518]: pgmap v2716: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:53:01.057 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:53:00 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4640.727789 secs 2024-03-28T01:53:01.122 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:53:00 smithi165 ceph-mon[36302]: pgmap v2716: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:53:01.122 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:53:00 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4640.727789 secs 2024-03-28T01:53:02.693 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:53:02 smithi132 ceph-mon[35941]: pgmap v2717: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:53:02.693 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:53:02 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:53:02.693 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:53:02 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:53:02.872 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:53:02 smithi165 ceph-mon[36302]: pgmap v2717: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:53:02.872 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:53:02 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:53:02.872 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:53:02 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:53:02.896 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:53:02 smithi119 ceph-mon[31518]: pgmap v2717: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:53:02.896 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:53:02 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:53:02.896 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:53:02 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:53:04.693 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:53:04 smithi132 ceph-mon[35941]: pgmap v2718: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:53:04.693 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:53:04 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:53:04.693 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:53:04 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:53:04.693 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:53:04 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:53:04.693 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:53:04 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:53:04.693 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:53:04 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:53:04.872 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:53:04 smithi165 ceph-mon[36302]: pgmap v2718: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:53:04.873 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:53:04 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:53:04.873 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:53:04 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:53:04.873 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:53:04 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:53:04.873 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:53:04 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:53:04.873 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:53:04 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:53:04.896 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:53:04 smithi119 ceph-mon[31518]: pgmap v2718: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:53:04.896 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:53:04 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:53:04.896 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:53:04 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:53:04.896 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:53:04 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:53:04.896 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:53:04 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:53:04.896 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:53:04 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:53:05.692 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:53:05 smithi132 ceph-mon[35941]: pgmap v2719: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-28T01:53:05.693 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:53:05 smithi132 ceph-mon[35941]: pgmap v2720: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-28T01:53:05.872 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:53:05 smithi165 ceph-mon[36302]: pgmap v2719: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-28T01:53:05.872 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:53:05 smithi165 ceph-mon[36302]: pgmap v2720: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-28T01:53:05.895 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:53:05 smithi119 ceph-mon[31518]: pgmap v2719: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-28T01:53:05.895 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:53:05 smithi119 ceph-mon[31518]: pgmap v2720: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-28T01:53:06.872 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:53:06 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4645.727880 secs 2024-03-28T01:53:06.895 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:53:06 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4645.727880 secs 2024-03-28T01:53:06.942 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:53:06 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4645.727880 secs 2024-03-28T01:53:07.872 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:53:07 smithi165 ceph-mon[36302]: pgmap v2721: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T01:53:07.895 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:53:07 smithi119 ceph-mon[31518]: pgmap v2721: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T01:53:07.942 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:53:07 smithi132 ceph-mon[35941]: pgmap v2721: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T01:53:09.872 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:53:09 smithi165 ceph-mon[36302]: pgmap v2722: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T01:53:09.895 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:53:09 smithi119 ceph-mon[31518]: pgmap v2722: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T01:53:09.942 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:53:09 smithi132 ceph-mon[35941]: pgmap v2722: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T01:53:11.872 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:53:11 smithi165 ceph-mon[36302]: pgmap v2723: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T01:53:11.872 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:53:11 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4650.727904 secs 2024-03-28T01:53:11.895 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:53:11 smithi119 ceph-mon[31518]: pgmap v2723: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T01:53:11.895 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:53:11 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4650.727904 secs 2024-03-28T01:53:11.942 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:53:11 smithi132 ceph-mon[35941]: pgmap v2723: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T01:53:11.942 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:53:11 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4650.727904 secs 2024-03-28T01:53:13.872 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:53:13 smithi165 ceph-mon[36302]: pgmap v2724: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T01:53:13.895 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:53:13 smithi119 ceph-mon[31518]: pgmap v2724: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T01:53:13.943 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:53:13 smithi132 ceph-mon[35941]: pgmap v2724: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T01:53:15.872 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:53:15 smithi165 ceph-mon[36302]: pgmap v2725: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T01:53:15.872 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:53:15 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4655.727965 secs 2024-03-28T01:53:15.895 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:53:15 smithi119 ceph-mon[31518]: pgmap v2725: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T01:53:15.895 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:53:15 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4655.727965 secs 2024-03-28T01:53:15.942 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:53:15 smithi132 ceph-mon[35941]: pgmap v2725: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T01:53:15.942 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:53:15 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4655.727965 secs 2024-03-28T01:53:17.872 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:53:17 smithi165 ceph-mon[36302]: pgmap v2726: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:53:17.895 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:53:17 smithi119 ceph-mon[31518]: pgmap v2726: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:53:17.942 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:53:17 smithi132 ceph-mon[35941]: pgmap v2726: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:53:19.872 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:53:19 smithi165 ceph-mon[36302]: pgmap v2727: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:53:19.895 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:53:19 smithi119 ceph-mon[31518]: pgmap v2727: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:53:19.942 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:53:19 smithi132 ceph-mon[35941]: pgmap v2727: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:53:20.872 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:53:20 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4660.728025 secs 2024-03-28T01:53:20.895 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:53:20 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4660.728025 secs 2024-03-28T01:53:20.942 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:53:20 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4660.728025 secs 2024-03-28T01:53:21.771 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:53:21 smithi119 ceph-mon[31518]: pgmap v2728: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:53:21.872 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:53:21 smithi165 ceph-mon[36302]: pgmap v2728: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:53:21.942 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:53:21 smithi132 ceph-mon[35941]: pgmap v2728: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:53:23.872 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:53:23 smithi165 ceph-mon[36302]: pgmap v2729: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:53:23.895 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:53:23 smithi119 ceph-mon[31518]: pgmap v2729: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:53:23.942 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:53:23 smithi132 ceph-mon[35941]: pgmap v2729: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:53:25.872 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:53:25 smithi165 ceph-mon[36302]: pgmap v2730: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:53:25.872 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:53:25 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4665.728110 secs 2024-03-28T01:53:25.895 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:53:25 smithi119 ceph-mon[31518]: pgmap v2730: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:53:25.895 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:53:25 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4665.728110 secs 2024-03-28T01:53:25.942 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:53:25 smithi132 ceph-mon[35941]: pgmap v2730: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:53:25.943 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:53:25 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4665.728110 secs 2024-03-28T01:53:27.872 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:53:27 smithi165 ceph-mon[36302]: pgmap v2731: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:53:27.895 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:53:27 smithi119 ceph-mon[31518]: pgmap v2731: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:53:27.942 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:53:27 smithi132 ceph-mon[35941]: pgmap v2731: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:53:29.872 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:53:29 smithi165 ceph-mon[36302]: pgmap v2732: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:53:29.895 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:53:29 smithi119 ceph-mon[31518]: pgmap v2732: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:53:29.942 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:53:29 smithi132 ceph-mon[35941]: pgmap v2732: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:53:30.844 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:53:30 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4670.728170 secs 2024-03-28T01:53:30.872 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:53:30 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4670.728170 secs 2024-03-28T01:53:30.895 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:53:30 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4670.728170 secs 2024-03-28T01:53:31.872 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:53:31 smithi165 ceph-mon[36302]: pgmap v2733: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:53:31.895 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:53:31 smithi119 ceph-mon[31518]: pgmap v2733: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:53:31.942 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:53:31 smithi132 ceph-mon[35941]: pgmap v2733: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:53:33.872 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:53:33 smithi165 ceph-mon[36302]: pgmap v2734: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:53:33.895 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:53:33 smithi119 ceph-mon[31518]: pgmap v2734: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:53:33.942 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:53:33 smithi132 ceph-mon[35941]: pgmap v2734: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:53:35.872 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:53:35 smithi165 ceph-mon[36302]: pgmap v2735: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:53:35.872 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:53:35 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4675.728229 secs 2024-03-28T01:53:35.895 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:53:35 smithi119 ceph-mon[31518]: pgmap v2735: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:53:35.895 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:53:35 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4675.728229 secs 2024-03-28T01:53:35.942 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:53:35 smithi132 ceph-mon[35941]: pgmap v2735: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:53:35.942 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:53:35 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4675.728229 secs 2024-03-28T01:53:37.872 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:53:37 smithi165 ceph-mon[36302]: pgmap v2736: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:53:37.895 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:53:37 smithi119 ceph-mon[31518]: pgmap v2736: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:53:37.943 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:53:37 smithi132 ceph-mon[35941]: pgmap v2736: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:53:39.872 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:53:39 smithi165 ceph-mon[36302]: pgmap v2737: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:53:39.895 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:53:39 smithi119 ceph-mon[31518]: pgmap v2737: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:53:39.942 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:53:39 smithi132 ceph-mon[35941]: pgmap v2737: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:53:40.872 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:53:40 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4680.728290 secs 2024-03-28T01:53:40.895 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:53:40 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4680.728290 secs 2024-03-28T01:53:40.942 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:53:40 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4680.728290 secs 2024-03-28T01:53:41.872 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:53:41 smithi165 ceph-mon[36302]: pgmap v2738: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:53:41.895 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:53:41 smithi119 ceph-mon[31518]: pgmap v2738: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:53:41.942 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:53:41 smithi132 ceph-mon[35941]: pgmap v2738: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:53:43.872 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:53:43 smithi165 ceph-mon[36302]: pgmap v2739: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:53:43.895 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:53:43 smithi119 ceph-mon[31518]: pgmap v2739: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:53:43.942 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:53:43 smithi132 ceph-mon[35941]: pgmap v2739: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:53:45.872 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:53:45 smithi165 ceph-mon[36302]: pgmap v2740: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:53:45.872 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:53:45 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4685.728378 secs 2024-03-28T01:53:45.895 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:53:45 smithi119 ceph-mon[31518]: pgmap v2740: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:53:45.895 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:53:45 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4685.728378 secs 2024-03-28T01:53:45.942 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:53:45 smithi132 ceph-mon[35941]: pgmap v2740: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:53:45.942 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:53:45 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4685.728378 secs 2024-03-28T01:53:47.872 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:53:47 smithi165 ceph-mon[36302]: pgmap v2741: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:53:47.895 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:53:47 smithi119 ceph-mon[31518]: pgmap v2741: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:53:47.942 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:53:47 smithi132 ceph-mon[35941]: pgmap v2741: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:53:49.872 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:53:49 smithi165 ceph-mon[36302]: pgmap v2742: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:53:49.895 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:53:49 smithi119 ceph-mon[31518]: pgmap v2742: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:53:49.942 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:53:49 smithi132 ceph-mon[35941]: pgmap v2742: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:53:50.872 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:53:50 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4690.728441 secs 2024-03-28T01:53:50.895 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:53:50 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4690.728441 secs 2024-03-28T01:53:50.942 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:53:50 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4690.728441 secs 2024-03-28T01:53:51.872 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:53:51 smithi165 ceph-mon[36302]: pgmap v2743: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:53:51.895 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:53:51 smithi119 ceph-mon[31518]: pgmap v2743: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:53:51.942 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:53:51 smithi132 ceph-mon[35941]: pgmap v2743: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:53:53.872 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:53:53 smithi165 ceph-mon[36302]: pgmap v2744: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:53:53.895 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:53:53 smithi119 ceph-mon[31518]: pgmap v2744: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:53:53.942 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:53:53 smithi132 ceph-mon[35941]: pgmap v2744: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:53:55.872 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:53:55 smithi165 ceph-mon[36302]: pgmap v2745: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:53:55.872 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:53:55 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4695.728502 secs 2024-03-28T01:53:55.895 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:53:55 smithi119 ceph-mon[31518]: pgmap v2745: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:53:55.895 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:53:55 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4695.728502 secs 2024-03-28T01:53:55.942 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:53:55 smithi132 ceph-mon[35941]: pgmap v2745: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:53:55.942 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:53:55 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4695.728502 secs 2024-03-28T01:53:57.872 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:53:57 smithi165 ceph-mon[36302]: pgmap v2746: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:53:57.895 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:53:57 smithi119 ceph-mon[31518]: pgmap v2746: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:53:57.942 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:53:57 smithi132 ceph-mon[35941]: pgmap v2746: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:53:59.872 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:53:59 smithi165 ceph-mon[36302]: pgmap v2747: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:53:59.895 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:53:59 smithi119 ceph-mon[31518]: pgmap v2747: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:53:59.942 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:53:59 smithi132 ceph-mon[35941]: pgmap v2747: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:54:00.872 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:54:00 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4700.728585 secs 2024-03-28T01:54:00.895 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:54:00 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4700.728585 secs 2024-03-28T01:54:00.942 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:54:00 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4700.728585 secs 2024-03-28T01:54:01.872 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:54:01 smithi165 ceph-mon[36302]: pgmap v2748: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:54:01.895 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:54:01 smithi119 ceph-mon[31518]: pgmap v2748: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:54:01.942 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:54:01 smithi132 ceph-mon[35941]: pgmap v2748: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:54:03.942 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:54:03 smithi132 ceph-mon[35941]: pgmap v2749: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:54:04.122 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:54:03 smithi165 ceph-mon[36302]: pgmap v2749: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:54:04.145 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:54:03 smithi119 ceph-mon[31518]: pgmap v2749: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:54:04.942 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:54:04 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:54:05.122 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:54:04 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:54:05.145 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:54:04 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:54:05.942 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:54:05 smithi132 ceph-mon[35941]: pgmap v2750: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:54:05.942 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:54:05 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4705.728672 secs 2024-03-28T01:54:05.973 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:54:05 smithi165 ceph-mon[36302]: pgmap v2750: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:54:05.974 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:54:05 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4705.728672 secs 2024-03-28T01:54:06.146 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:54:05 smithi119 ceph-mon[31518]: pgmap v2750: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:54:06.146 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:54:05 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4705.728672 secs 2024-03-28T01:54:07.942 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:54:07 smithi132 ceph-mon[35941]: pgmap v2751: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:54:08.122 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:54:07 smithi165 ceph-mon[36302]: pgmap v2751: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:54:08.145 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:54:07 smithi119 ceph-mon[31518]: pgmap v2751: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:54:09.872 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:54:09 smithi165 ceph-mon[36302]: pgmap v2752: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:54:09.872 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:54:09 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:54:09.872 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:54:09 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:54:09.872 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:54:09 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:54:09.872 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:54:09 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:54:09.872 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:54:09 smithi165 ceph-mon[36302]: pgmap v2753: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-03-28T01:54:09.872 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:54:09 smithi165 ceph-mon[36302]: pgmap v2754: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-28T01:54:09.872 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:54:09 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:54:09.895 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:54:09 smithi119 ceph-mon[31518]: pgmap v2752: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:54:09.895 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:54:09 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:54:09.895 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:54:09 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:54:09.896 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:54:09 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:54:09.896 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:54:09 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:54:09.896 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:54:09 smithi119 ceph-mon[31518]: pgmap v2753: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-03-28T01:54:09.896 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:54:09 smithi119 ceph-mon[31518]: pgmap v2754: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-28T01:54:09.896 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:54:09 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:54:09.942 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:54:09 smithi132 ceph-mon[35941]: pgmap v2752: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:54:09.942 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:54:09 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:54:09.942 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:54:09 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:54:09.942 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:54:09 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:54:09.943 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:54:09 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:54:09.943 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:54:09 smithi132 ceph-mon[35941]: pgmap v2753: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-03-28T01:54:09.943 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:54:09 smithi132 ceph-mon[35941]: pgmap v2754: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-28T01:54:09.943 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:54:09 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:54:10.872 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:54:10 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4710.728763 secs 2024-03-28T01:54:10.895 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:54:10 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4710.728763 secs 2024-03-28T01:54:10.942 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:54:10 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4710.728763 secs 2024-03-28T01:54:11.872 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:54:11 smithi165 ceph-mon[36302]: pgmap v2755: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-03-28T01:54:11.895 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:54:11 smithi119 ceph-mon[31518]: pgmap v2755: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-03-28T01:54:11.942 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:54:11 smithi132 ceph-mon[35941]: pgmap v2755: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-03-28T01:54:13.942 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:54:13 smithi132 ceph-mon[35941]: pgmap v2756: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-03-28T01:54:14.122 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:54:13 smithi165 ceph-mon[36302]: pgmap v2756: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-03-28T01:54:14.145 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:54:13 smithi119 ceph-mon[31518]: pgmap v2756: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-03-28T01:54:15.942 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:54:15 smithi132 ceph-mon[35941]: pgmap v2757: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-03-28T01:54:15.942 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:54:15 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4715.728801 secs 2024-03-28T01:54:16.122 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:54:15 smithi165 ceph-mon[36302]: pgmap v2757: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-03-28T01:54:16.122 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:54:15 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4715.728801 secs 2024-03-28T01:54:16.146 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:54:15 smithi119 ceph-mon[31518]: pgmap v2757: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-03-28T01:54:16.146 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:54:15 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4715.728801 secs 2024-03-28T01:54:18.122 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:54:17 smithi165 ceph-mon[36302]: pgmap v2758: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-03-28T01:54:18.145 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:54:17 smithi119 ceph-mon[31518]: pgmap v2758: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-03-28T01:54:18.192 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:54:17 smithi132 ceph-mon[35941]: pgmap v2758: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-03-28T01:54:20.122 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:54:19 smithi165 ceph-mon[36302]: pgmap v2759: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T01:54:20.145 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:54:19 smithi119 ceph-mon[31518]: pgmap v2759: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T01:54:20.192 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:54:19 smithi132 ceph-mon[35941]: pgmap v2759: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T01:54:21.122 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:54:20 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4720.728886 secs 2024-03-28T01:54:21.547 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:54:20 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4720.728886 secs 2024-03-28T01:54:21.547 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:54:20 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4720.728886 secs 2024-03-28T01:54:22.047 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:54:21 smithi119 ceph-mon[31518]: pgmap v2760: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:54:22.122 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:54:21 smithi165 ceph-mon[36302]: pgmap v2760: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:54:22.192 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:54:21 smithi132 ceph-mon[35941]: pgmap v2760: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:54:24.122 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:54:23 smithi165 ceph-mon[36302]: pgmap v2761: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:54:24.145 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:54:23 smithi119 ceph-mon[31518]: pgmap v2761: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:54:24.192 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:54:23 smithi132 ceph-mon[35941]: pgmap v2761: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:54:26.122 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:54:25 smithi165 ceph-mon[36302]: pgmap v2762: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:54:26.122 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:54:25 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4725.728971 secs 2024-03-28T01:54:26.145 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:54:25 smithi119 ceph-mon[31518]: pgmap v2762: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:54:26.145 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:54:25 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4725.728971 secs 2024-03-28T01:54:26.192 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:54:25 smithi132 ceph-mon[35941]: pgmap v2762: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:54:26.192 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:54:25 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4725.728971 secs 2024-03-28T01:54:27.122 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:54:26 smithi165 ceph-mon[36302]: pgmap v2763: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:54:27.145 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:54:26 smithi119 ceph-mon[31518]: pgmap v2763: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:54:27.192 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:54:26 smithi132 ceph-mon[35941]: pgmap v2763: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:54:29.942 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:54:29 smithi132 ceph-mon[35941]: pgmap v2764: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:54:30.122 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:54:29 smithi165 ceph-mon[36302]: pgmap v2764: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:54:30.145 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:54:29 smithi119 ceph-mon[31518]: pgmap v2764: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:54:31.122 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:54:30 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4730.729034 secs 2024-03-28T01:54:31.145 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:54:30 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4730.729034 secs 2024-03-28T01:54:31.192 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:54:30 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4730.729034 secs 2024-03-28T01:54:32.122 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:54:31 smithi165 ceph-mon[36302]: pgmap v2765: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:54:32.145 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:54:31 smithi119 ceph-mon[31518]: pgmap v2765: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:54:32.192 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:54:31 smithi132 ceph-mon[35941]: pgmap v2765: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:54:34.122 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:54:33 smithi165 ceph-mon[36302]: pgmap v2766: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:54:34.145 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:54:33 smithi119 ceph-mon[31518]: pgmap v2766: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:54:34.192 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:54:33 smithi132 ceph-mon[35941]: pgmap v2766: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:54:36.122 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:54:35 smithi165 ceph-mon[36302]: pgmap v2767: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:54:36.122 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:54:35 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4735.729119 secs 2024-03-28T01:54:36.145 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:54:35 smithi119 ceph-mon[31518]: pgmap v2767: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:54:36.145 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:54:35 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4735.729119 secs 2024-03-28T01:54:36.192 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:54:35 smithi132 ceph-mon[35941]: pgmap v2767: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:54:36.192 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:54:35 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4735.729119 secs 2024-03-28T01:54:38.122 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:54:37 smithi165 ceph-mon[36302]: pgmap v2768: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:54:38.145 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:54:37 smithi119 ceph-mon[31518]: pgmap v2768: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:54:38.192 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:54:37 smithi132 ceph-mon[35941]: pgmap v2768: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:54:40.122 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:54:39 smithi165 ceph-mon[36302]: pgmap v2769: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:54:40.145 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:54:39 smithi119 ceph-mon[31518]: pgmap v2769: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:54:40.192 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:54:39 smithi132 ceph-mon[35941]: pgmap v2769: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:54:41.122 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:54:40 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4740.729211 secs 2024-03-28T01:54:41.145 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:54:40 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4740.729211 secs 2024-03-28T01:54:41.192 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:54:40 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4740.729211 secs 2024-03-28T01:54:42.122 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:54:41 smithi165 ceph-mon[36302]: pgmap v2770: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:54:42.145 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:54:41 smithi119 ceph-mon[31518]: pgmap v2770: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:54:42.192 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:54:41 smithi132 ceph-mon[35941]: pgmap v2770: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:54:43.122 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:54:42 smithi165 ceph-mon[36302]: pgmap v2771: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:54:43.145 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:54:42 smithi119 ceph-mon[31518]: pgmap v2771: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:54:43.192 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:54:42 smithi132 ceph-mon[35941]: pgmap v2771: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:54:46.122 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:54:45 smithi165 ceph-mon[36302]: pgmap v2772: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:54:46.122 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:54:45 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4745.729301 secs 2024-03-28T01:54:46.145 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:54:45 smithi119 ceph-mon[31518]: pgmap v2772: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:54:46.145 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:54:45 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4745.729301 secs 2024-03-28T01:54:46.192 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:54:45 smithi132 ceph-mon[35941]: pgmap v2772: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:54:46.192 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:54:45 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4745.729301 secs 2024-03-28T01:54:48.122 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:54:47 smithi165 ceph-mon[36302]: pgmap v2773: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T01:54:48.145 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:54:47 smithi119 ceph-mon[31518]: pgmap v2773: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T01:54:48.192 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:54:47 smithi132 ceph-mon[35941]: pgmap v2773: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T01:54:50.122 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:54:49 smithi165 ceph-mon[36302]: pgmap v2774: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T01:54:50.145 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:54:49 smithi119 ceph-mon[31518]: pgmap v2774: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T01:54:50.192 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:54:49 smithi132 ceph-mon[35941]: pgmap v2774: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T01:54:51.122 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:54:50 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4750.729390 secs 2024-03-28T01:54:51.145 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:54:50 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4750.729390 secs 2024-03-28T01:54:51.192 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:54:50 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4750.729390 secs 2024-03-28T01:54:52.122 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:54:51 smithi165 ceph-mon[36302]: pgmap v2775: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:54:52.145 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:54:51 smithi119 ceph-mon[31518]: pgmap v2775: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:54:52.192 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:54:51 smithi132 ceph-mon[35941]: pgmap v2775: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:54:54.122 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:54:53 smithi165 ceph-mon[36302]: pgmap v2776: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T01:54:54.145 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:54:53 smithi119 ceph-mon[31518]: pgmap v2776: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T01:54:54.192 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:54:53 smithi132 ceph-mon[35941]: pgmap v2776: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T01:54:56.122 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:54:55 smithi165 ceph-mon[36302]: pgmap v2777: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:54:56.122 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:54:55 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4755.729478 secs 2024-03-28T01:54:56.145 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:54:55 smithi119 ceph-mon[31518]: pgmap v2777: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:54:56.145 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:54:55 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4755.729478 secs 2024-03-28T01:54:56.192 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:54:55 smithi132 ceph-mon[35941]: pgmap v2777: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:54:56.192 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:54:55 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4755.729478 secs 2024-03-28T01:54:58.122 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:54:57 smithi165 ceph-mon[36302]: pgmap v2778: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:54:58.145 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:54:57 smithi119 ceph-mon[31518]: pgmap v2778: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:54:58.192 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:54:57 smithi132 ceph-mon[35941]: pgmap v2778: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:55:00.122 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:54:59 smithi165 ceph-mon[36302]: pgmap v2779: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:55:00.145 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:54:59 smithi119 ceph-mon[31518]: pgmap v2779: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:55:00.192 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:54:59 smithi132 ceph-mon[35941]: pgmap v2779: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:55:01.093 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:55:00 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4760.729567 secs 2024-03-28T01:55:01.093 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:55:00 smithi132 ceph-mon[35941]: pgmap v2780: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:55:01.122 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:55:00 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4760.729567 secs 2024-03-28T01:55:01.122 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:55:00 smithi165 ceph-mon[36302]: pgmap v2780: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:55:01.145 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:55:00 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4760.729567 secs 2024-03-28T01:55:01.145 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:55:00 smithi119 ceph-mon[31518]: pgmap v2780: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:55:04.122 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:55:03 smithi165 ceph-mon[36302]: pgmap v2781: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:55:04.145 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:55:03 smithi119 ceph-mon[31518]: pgmap v2781: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:55:04.192 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:55:03 smithi132 ceph-mon[35941]: pgmap v2781: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:55:06.122 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:55:05 smithi165 ceph-mon[36302]: pgmap v2782: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:55:06.122 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:55:05 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4765.729657 secs 2024-03-28T01:55:06.145 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:55:05 smithi119 ceph-mon[31518]: pgmap v2782: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:55:06.145 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:55:05 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4765.729657 secs 2024-03-28T01:55:06.192 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:55:05 smithi132 ceph-mon[35941]: pgmap v2782: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:55:06.192 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:55:05 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4765.729657 secs 2024-03-28T01:55:08.122 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:55:07 smithi165 ceph-mon[36302]: pgmap v2783: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:55:08.145 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:55:07 smithi119 ceph-mon[31518]: pgmap v2783: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:55:08.192 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:55:07 smithi132 ceph-mon[35941]: pgmap v2783: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:55:10.122 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:55:09 smithi165 ceph-mon[36302]: pgmap v2784: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:55:10.122 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:55:09 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:55:10.145 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:55:09 smithi119 ceph-mon[31518]: pgmap v2784: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:55:10.145 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:55:09 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:55:10.192 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:55:09 smithi132 ceph-mon[35941]: pgmap v2784: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:55:10.192 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:55:09 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:55:11.122 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:55:10 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4770.729748 secs 2024-03-28T01:55:11.122 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:55:10 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:55:11.122 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:55:10 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:55:11.122 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:55:10 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:55:11.145 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:55:10 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4770.729748 secs 2024-03-28T01:55:11.145 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:55:10 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:55:11.145 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:55:10 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:55:11.145 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:55:10 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:55:11.192 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:55:10 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4770.729748 secs 2024-03-28T01:55:11.192 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:55:10 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:55:11.192 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:55:10 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:55:11.192 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:55:10 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:55:12.121 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:55:11 smithi165 ceph-mon[36302]: pgmap v2785: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T01:55:12.144 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:55:11 smithi119 ceph-mon[31518]: pgmap v2785: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T01:55:12.192 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:55:11 smithi132 ceph-mon[35941]: pgmap v2785: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T01:55:14.121 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:55:13 smithi165 ceph-mon[36302]: pgmap v2786: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T01:55:14.145 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:55:13 smithi119 ceph-mon[31518]: pgmap v2786: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T01:55:14.192 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:55:13 smithi132 ceph-mon[35941]: pgmap v2786: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T01:55:16.122 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:55:15 smithi165 ceph-mon[36302]: pgmap v2787: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 6 op/s 2024-03-28T01:55:16.122 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:55:15 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4775.729837 secs 2024-03-28T01:55:16.145 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:55:15 smithi119 ceph-mon[31518]: pgmap v2787: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 6 op/s 2024-03-28T01:55:16.145 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:55:15 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4775.729837 secs 2024-03-28T01:55:16.192 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:55:15 smithi132 ceph-mon[35941]: pgmap v2787: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 6 op/s 2024-03-28T01:55:16.192 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:55:15 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4775.729837 secs 2024-03-28T01:55:17.121 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:55:16 smithi165 ceph-mon[36302]: pgmap v2788: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T01:55:17.145 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:55:16 smithi119 ceph-mon[31518]: pgmap v2788: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T01:55:17.192 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:55:16 smithi132 ceph-mon[35941]: pgmap v2788: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T01:55:19.621 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:55:19 smithi165 ceph-mon[36302]: pgmap v2789: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T01:55:19.645 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:55:19 smithi119 ceph-mon[31518]: pgmap v2789: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T01:55:19.692 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:55:19 smithi132 ceph-mon[35941]: pgmap v2789: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T01:55:21.621 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:55:21 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4780.729925 secs 2024-03-28T01:55:21.622 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:55:21 smithi165 ceph-mon[36302]: pgmap v2790: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T01:55:21.645 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:55:21 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4780.729925 secs 2024-03-28T01:55:21.645 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:55:21 smithi119 ceph-mon[31518]: pgmap v2790: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T01:55:21.692 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:55:21 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4780.729925 secs 2024-03-28T01:55:21.692 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:55:21 smithi132 ceph-mon[35941]: pgmap v2790: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T01:55:23.621 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:55:23 smithi165 ceph-mon[36302]: pgmap v2791: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:55:23.645 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:55:23 smithi119 ceph-mon[31518]: pgmap v2791: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:55:23.692 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:55:23 smithi132 ceph-mon[35941]: pgmap v2791: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:55:25.551 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:55:25 smithi119 ceph-mon[31518]: pgmap v2792: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T01:55:25.621 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:55:25 smithi165 ceph-mon[36302]: pgmap v2792: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T01:55:25.692 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:55:25 smithi132 ceph-mon[35941]: pgmap v2792: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T01:55:26.621 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:55:26 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4785.729987 secs 2024-03-28T01:55:26.645 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:55:26 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4785.729987 secs 2024-03-28T01:55:26.692 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:55:26 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4785.729987 secs 2024-03-28T01:55:27.621 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:55:27 smithi165 ceph-mon[36302]: pgmap v2793: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-28T01:55:27.645 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:55:27 smithi119 ceph-mon[31518]: pgmap v2793: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-28T01:55:27.694 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:55:27 smithi132 ceph-mon[35941]: pgmap v2793: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-28T01:55:29.622 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:55:29 smithi165 ceph-mon[36302]: pgmap v2794: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:55:29.644 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:55:29 smithi119 ceph-mon[31518]: pgmap v2794: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:55:29.692 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:55:29 smithi132 ceph-mon[35941]: pgmap v2794: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:55:31.493 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:55:31 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4790.730074 secs 2024-03-28T01:55:31.493 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:55:31 smithi132 ceph-mon[35941]: pgmap v2795: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:55:31.621 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:55:31 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4790.730074 secs 2024-03-28T01:55:31.622 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:55:31 smithi165 ceph-mon[36302]: pgmap v2795: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:55:31.644 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:55:31 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4790.730074 secs 2024-03-28T01:55:31.645 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:55:31 smithi119 ceph-mon[31518]: pgmap v2795: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:55:33.621 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:55:33 smithi165 ceph-mon[36302]: pgmap v2796: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:55:33.645 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:55:33 smithi119 ceph-mon[31518]: pgmap v2796: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:55:33.692 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:55:33 smithi132 ceph-mon[35941]: pgmap v2796: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:55:35.621 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:55:35 smithi165 ceph-mon[36302]: pgmap v2797: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T01:55:35.645 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:55:35 smithi119 ceph-mon[31518]: pgmap v2797: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T01:55:35.692 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:55:35 smithi132 ceph-mon[35941]: pgmap v2797: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T01:55:36.621 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:55:36 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4795.730162 secs 2024-03-28T01:55:36.645 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:55:36 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4795.730162 secs 2024-03-28T01:55:36.692 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:55:36 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4795.730162 secs 2024-03-28T01:55:37.621 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:55:37 smithi165 ceph-mon[36302]: pgmap v2798: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-28T01:55:37.645 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:55:37 smithi119 ceph-mon[31518]: pgmap v2798: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-28T01:55:37.692 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:55:37 smithi132 ceph-mon[35941]: pgmap v2798: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-28T01:55:39.621 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:55:39 smithi165 ceph-mon[36302]: pgmap v2799: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:55:39.644 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:55:39 smithi119 ceph-mon[31518]: pgmap v2799: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:55:39.692 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:55:39 smithi132 ceph-mon[35941]: pgmap v2799: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:55:41.621 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:55:41 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4800.730225 secs 2024-03-28T01:55:41.622 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:55:41 smithi165 ceph-mon[36302]: pgmap v2800: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:55:41.645 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:55:41 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4800.730225 secs 2024-03-28T01:55:41.645 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:55:41 smithi119 ceph-mon[31518]: pgmap v2800: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:55:41.692 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:55:41 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4800.730225 secs 2024-03-28T01:55:41.692 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:55:41 smithi132 ceph-mon[35941]: pgmap v2800: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:55:43.622 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:55:43 smithi165 ceph-mon[36302]: pgmap v2801: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:55:43.644 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:55:43 smithi119 ceph-mon[31518]: pgmap v2801: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:55:43.692 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:55:43 smithi132 ceph-mon[35941]: pgmap v2801: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:55:45.621 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:55:45 smithi165 ceph-mon[36302]: pgmap v2802: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T01:55:45.645 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:55:45 smithi119 ceph-mon[31518]: pgmap v2802: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T01:55:45.692 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:55:45 smithi132 ceph-mon[35941]: pgmap v2802: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T01:55:46.621 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:55:46 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4805.730287 secs 2024-03-28T01:55:46.645 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:55:46 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4805.730287 secs 2024-03-28T01:55:46.692 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:55:46 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4805.730287 secs 2024-03-28T01:55:47.621 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:55:47 smithi165 ceph-mon[36302]: pgmap v2803: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-28T01:55:47.644 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:55:47 smithi119 ceph-mon[31518]: pgmap v2803: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-28T01:55:47.692 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:55:47 smithi132 ceph-mon[35941]: pgmap v2803: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-28T01:55:49.621 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:55:49 smithi165 ceph-mon[36302]: pgmap v2804: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:55:49.644 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:55:49 smithi119 ceph-mon[31518]: pgmap v2804: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:55:49.692 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:55:49 smithi132 ceph-mon[35941]: pgmap v2804: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:55:51.621 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:55:51 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4810.730379 secs 2024-03-28T01:55:51.622 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:55:51 smithi165 ceph-mon[36302]: pgmap v2805: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:55:51.645 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:55:51 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4810.730379 secs 2024-03-28T01:55:51.645 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:55:51 smithi119 ceph-mon[31518]: pgmap v2805: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:55:51.692 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:55:51 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4810.730379 secs 2024-03-28T01:55:51.692 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:55:51 smithi132 ceph-mon[35941]: pgmap v2805: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:55:53.621 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:55:53 smithi165 ceph-mon[36302]: pgmap v2806: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:55:53.644 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:55:53 smithi119 ceph-mon[31518]: pgmap v2806: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:55:53.692 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:55:53 smithi132 ceph-mon[35941]: pgmap v2806: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:55:55.621 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:55:55 smithi165 ceph-mon[36302]: pgmap v2807: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T01:55:55.644 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:55:55 smithi119 ceph-mon[31518]: pgmap v2807: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T01:55:55.691 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:55:55 smithi132 ceph-mon[35941]: pgmap v2807: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T01:55:56.621 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:55:56 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4815.730462 secs 2024-03-28T01:55:56.645 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:55:56 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4815.730462 secs 2024-03-28T01:55:56.692 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:55:56 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4815.730462 secs 2024-03-28T01:55:57.621 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:55:57 smithi165 ceph-mon[36302]: pgmap v2808: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-28T01:55:57.645 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:55:57 smithi119 ceph-mon[31518]: pgmap v2808: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-28T01:55:57.692 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:55:57 smithi132 ceph-mon[35941]: pgmap v2808: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-28T01:55:59.621 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:55:59 smithi165 ceph-mon[36302]: pgmap v2809: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:55:59.644 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:55:59 smithi119 ceph-mon[31518]: pgmap v2809: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:55:59.692 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:55:59 smithi132 ceph-mon[35941]: pgmap v2809: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:56:01.621 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:56:01 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4820.730478 secs 2024-03-28T01:56:01.622 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:56:01 smithi165 ceph-mon[36302]: pgmap v2810: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:56:01.645 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:56:01 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4820.730478 secs 2024-03-28T01:56:01.645 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:56:01 smithi119 ceph-mon[31518]: pgmap v2810: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:56:01.692 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:56:01 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4820.730478 secs 2024-03-28T01:56:01.693 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:56:01 smithi132 ceph-mon[35941]: pgmap v2810: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:56:03.621 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:56:03 smithi165 ceph-mon[36302]: pgmap v2811: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:56:03.644 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:56:03 smithi119 ceph-mon[31518]: pgmap v2811: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:56:03.691 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:56:03 smithi132 ceph-mon[35941]: pgmap v2811: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:56:05.621 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:56:05 smithi165 ceph-mon[36302]: pgmap v2812: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-28T01:56:05.644 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:56:05 smithi119 ceph-mon[31518]: pgmap v2812: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-28T01:56:05.692 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:56:05 smithi132 ceph-mon[35941]: pgmap v2812: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-28T01:56:06.621 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:56:06 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4825.730569 secs 2024-03-28T01:56:06.644 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:56:06 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4825.730569 secs 2024-03-28T01:56:06.691 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:56:06 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4825.730569 secs 2024-03-28T01:56:07.621 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:56:07 smithi165 ceph-mon[36302]: pgmap v2813: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-28T01:56:07.645 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:56:07 smithi119 ceph-mon[31518]: pgmap v2813: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-28T01:56:07.692 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:56:07 smithi132 ceph-mon[35941]: pgmap v2813: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-28T01:56:09.621 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:56:09 smithi165 ceph-mon[36302]: pgmap v2814: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:56:09.645 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:56:09 smithi119 ceph-mon[31518]: pgmap v2814: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:56:09.692 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:56:09 smithi132 ceph-mon[35941]: pgmap v2814: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:56:11.621 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:56:11 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4830.730658 secs 2024-03-28T01:56:11.622 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:56:11 smithi165 ceph-mon[36302]: pgmap v2815: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:56:11.622 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:56:11 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:56:11.644 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:56:11 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4830.730658 secs 2024-03-28T01:56:11.645 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:56:11 smithi119 ceph-mon[31518]: pgmap v2815: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:56:11.645 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:56:11 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:56:11.692 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:56:11 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4830.730658 secs 2024-03-28T01:56:11.692 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:56:11 smithi132 ceph-mon[35941]: pgmap v2815: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:56:11.692 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:56:11 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:56:12.621 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:56:12 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:56:12.622 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:56:12 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:56:12.622 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:56:12 smithi165 ceph-mon[36302]: pgmap v2816: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T01:56:12.622 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:56:12 smithi165 ceph-mon[36302]: pgmap v2817: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:56:12.622 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:56:12 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:56:12.644 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:56:12 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:56:12.645 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:56:12 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:56:12.645 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:56:12 smithi119 ceph-mon[31518]: pgmap v2816: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T01:56:12.645 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:56:12 smithi119 ceph-mon[31518]: pgmap v2817: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:56:12.645 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:56:12 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:56:12.692 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:56:12 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:56:12.692 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:56:12 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:56:12.692 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:56:12 smithi132 ceph-mon[35941]: pgmap v2816: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T01:56:12.692 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:56:12 smithi132 ceph-mon[35941]: pgmap v2817: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:56:12.692 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:56:12 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:56:14.121 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:56:13 smithi165 ceph-mon[36302]: pgmap v2818: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 4 op/s 2024-03-28T01:56:14.144 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:56:13 smithi119 ceph-mon[31518]: pgmap v2818: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 4 op/s 2024-03-28T01:56:14.192 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:56:13 smithi132 ceph-mon[35941]: pgmap v2818: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 4 op/s 2024-03-28T01:56:15.871 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:56:15 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4835.730747 secs 2024-03-28T01:56:15.894 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:56:15 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4835.730747 secs 2024-03-28T01:56:15.941 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:56:15 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4835.730747 secs 2024-03-28T01:56:16.871 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:56:16 smithi165 ceph-mon[36302]: pgmap v2819: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:56:16.894 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:56:16 smithi119 ceph-mon[31518]: pgmap v2819: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:56:16.942 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:56:16 smithi132 ceph-mon[35941]: pgmap v2819: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:56:18.121 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:56:17 smithi165 ceph-mon[36302]: pgmap v2820: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:56:18.144 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:56:17 smithi119 ceph-mon[31518]: pgmap v2820: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:56:18.191 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:56:17 smithi132 ceph-mon[35941]: pgmap v2820: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:56:20.121 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:56:19 smithi165 ceph-mon[36302]: pgmap v2821: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:56:20.144 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:56:19 smithi119 ceph-mon[31518]: pgmap v2821: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:56:20.192 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:56:19 smithi132 ceph-mon[35941]: pgmap v2821: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T01:56:21.121 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:56:20 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4840.730840 secs 2024-03-28T01:56:21.144 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:56:20 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4840.730840 secs 2024-03-28T01:56:21.191 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:56:20 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4840.730840 secs 2024-03-28T01:56:22.121 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:56:21 smithi165 ceph-mon[36302]: pgmap v2822: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T01:56:22.144 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:56:21 smithi119 ceph-mon[31518]: pgmap v2822: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T01:56:22.192 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:56:21 smithi132 ceph-mon[35941]: pgmap v2822: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T01:56:24.121 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:56:23 smithi165 ceph-mon[36302]: pgmap v2823: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:56:24.144 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:56:23 smithi119 ceph-mon[31518]: pgmap v2823: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:56:24.191 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:56:23 smithi132 ceph-mon[35941]: pgmap v2823: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:56:25.873 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:56:25 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4845.730925 secs 2024-03-28T01:56:25.894 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:56:25 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4845.730925 secs 2024-03-28T01:56:25.941 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:56:25 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4845.730925 secs 2024-03-28T01:56:26.871 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:56:26 smithi165 ceph-mon[36302]: pgmap v2824: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:56:26.894 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:56:26 smithi119 ceph-mon[31518]: pgmap v2824: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:56:26.942 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:56:26 smithi132 ceph-mon[35941]: pgmap v2824: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:56:28.121 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:56:27 smithi165 ceph-mon[36302]: pgmap v2825: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:56:28.145 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:56:27 smithi119 ceph-mon[31518]: pgmap v2825: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:56:28.192 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:56:27 smithi132 ceph-mon[35941]: pgmap v2825: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:56:30.121 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:56:29 smithi165 ceph-mon[36302]: pgmap v2826: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:56:30.144 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:56:29 smithi119 ceph-mon[31518]: pgmap v2826: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:56:30.191 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:56:29 smithi132 ceph-mon[35941]: pgmap v2826: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:56:31.093 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:56:30 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4850.730985 secs 2024-03-28T01:56:31.121 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:56:30 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4850.730985 secs 2024-03-28T01:56:31.144 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:56:30 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4850.730985 secs 2024-03-28T01:56:32.121 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:56:31 smithi165 ceph-mon[36302]: pgmap v2827: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:56:32.144 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:56:31 smithi119 ceph-mon[31518]: pgmap v2827: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:56:32.192 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:56:31 smithi132 ceph-mon[35941]: pgmap v2827: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:56:34.121 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:56:33 smithi165 ceph-mon[36302]: pgmap v2828: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:56:34.144 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:56:33 smithi119 ceph-mon[31518]: pgmap v2828: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:56:34.191 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:56:33 smithi132 ceph-mon[35941]: pgmap v2828: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:56:35.871 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:56:35 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4855.731049 secs 2024-03-28T01:56:35.894 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:56:35 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4855.731049 secs 2024-03-28T01:56:35.941 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:56:35 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4855.731049 secs 2024-03-28T01:56:36.871 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:56:36 smithi165 ceph-mon[36302]: pgmap v2829: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-03-28T01:56:36.894 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:56:36 smithi119 ceph-mon[31518]: pgmap v2829: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-03-28T01:56:36.941 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:56:36 smithi132 ceph-mon[35941]: pgmap v2829: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-03-28T01:56:39.121 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:56:38 smithi165 ceph-mon[36302]: pgmap v2830: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-03-28T01:56:39.144 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:56:38 smithi119 ceph-mon[31518]: pgmap v2830: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-03-28T01:56:39.191 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:56:38 smithi132 ceph-mon[35941]: pgmap v2830: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-03-28T01:56:41.121 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:56:40 smithi165 ceph-mon[36302]: pgmap v2831: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-03-28T01:56:41.121 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:56:40 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4860.731133 secs 2024-03-28T01:56:41.144 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:56:40 smithi119 ceph-mon[31518]: pgmap v2831: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-03-28T01:56:41.144 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:56:40 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4860.731133 secs 2024-03-28T01:56:41.191 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:56:40 smithi132 ceph-mon[35941]: pgmap v2831: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-03-28T01:56:41.191 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:56:40 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4860.731133 secs 2024-03-28T01:56:43.121 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:56:42 smithi165 ceph-mon[36302]: pgmap v2832: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-03-28T01:56:43.144 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:56:42 smithi119 ceph-mon[31518]: pgmap v2832: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-03-28T01:56:43.191 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:56:42 smithi132 ceph-mon[35941]: pgmap v2832: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-03-28T01:56:45.121 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:56:44 smithi165 ceph-mon[36302]: pgmap v2833: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-03-28T01:56:45.144 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:56:44 smithi119 ceph-mon[31518]: pgmap v2833: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-03-28T01:56:45.192 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:56:44 smithi132 ceph-mon[35941]: pgmap v2833: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-03-28T01:56:46.121 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:56:45 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4865.731225 secs 2024-03-28T01:56:46.121 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:56:45 smithi165 ceph-mon[36302]: pgmap v2834: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-03-28T01:56:46.144 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:56:45 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4865.731225 secs 2024-03-28T01:56:46.144 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:56:45 smithi119 ceph-mon[31518]: pgmap v2834: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-03-28T01:56:46.191 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:56:45 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4865.731225 secs 2024-03-28T01:56:46.192 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:56:45 smithi132 ceph-mon[35941]: pgmap v2834: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-03-28T01:56:48.121 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:56:47 smithi165 ceph-mon[36302]: pgmap v2835: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:56:48.144 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:56:47 smithi119 ceph-mon[31518]: pgmap v2835: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:56:48.191 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:56:47 smithi132 ceph-mon[35941]: pgmap v2835: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:56:50.121 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:56:49 smithi165 ceph-mon[36302]: pgmap v2836: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:56:50.144 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:56:49 smithi119 ceph-mon[31518]: pgmap v2836: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:56:50.191 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:56:49 smithi132 ceph-mon[35941]: pgmap v2836: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:56:51.121 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:56:50 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4870.731241 secs 2024-03-28T01:56:51.144 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:56:50 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4870.731241 secs 2024-03-28T01:56:51.191 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:56:50 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4870.731241 secs 2024-03-28T01:56:52.121 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:56:51 smithi165 ceph-mon[36302]: pgmap v2837: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:56:52.144 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:56:51 smithi119 ceph-mon[31518]: pgmap v2837: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:56:52.191 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:56:51 smithi132 ceph-mon[35941]: pgmap v2837: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:56:54.121 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:56:53 smithi165 ceph-mon[36302]: pgmap v2838: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:56:54.144 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:56:53 smithi119 ceph-mon[31518]: pgmap v2838: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:56:54.192 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:56:53 smithi132 ceph-mon[35941]: pgmap v2838: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:56:55.871 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:56:55 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4875.731257 secs 2024-03-28T01:56:55.894 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:56:55 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4875.731257 secs 2024-03-28T01:56:55.941 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:56:55 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4875.731257 secs 2024-03-28T01:56:56.871 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:56:56 smithi165 ceph-mon[36302]: pgmap v2839: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:56:56.894 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:56:56 smithi119 ceph-mon[31518]: pgmap v2839: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:56:56.941 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:56:56 smithi132 ceph-mon[35941]: pgmap v2839: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:56:58.121 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:56:57 smithi165 ceph-mon[36302]: pgmap v2840: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:56:58.144 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:56:57 smithi119 ceph-mon[31518]: pgmap v2840: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:56:58.191 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:56:57 smithi132 ceph-mon[35941]: pgmap v2840: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:57:00.121 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:56:59 smithi165 ceph-mon[36302]: pgmap v2841: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:57:00.144 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:56:59 smithi119 ceph-mon[31518]: pgmap v2841: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:57:00.191 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:56:59 smithi132 ceph-mon[35941]: pgmap v2841: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:57:01.093 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:57:00 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4880.731320 secs 2024-03-28T01:57:01.121 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:57:00 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4880.731320 secs 2024-03-28T01:57:01.144 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:57:00 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4880.731320 secs 2024-03-28T01:57:02.121 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:57:01 smithi165 ceph-mon[36302]: pgmap v2842: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:57:02.144 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:57:01 smithi119 ceph-mon[31518]: pgmap v2842: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:57:02.191 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:57:01 smithi132 ceph-mon[35941]: pgmap v2842: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:57:04.121 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:57:03 smithi165 ceph-mon[36302]: pgmap v2843: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:57:04.144 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:57:03 smithi119 ceph-mon[31518]: pgmap v2843: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:57:04.191 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:57:03 smithi132 ceph-mon[35941]: pgmap v2843: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:57:05.871 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:57:05 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4885.731380 secs 2024-03-28T01:57:05.894 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:57:05 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4885.731380 secs 2024-03-28T01:57:05.941 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:57:05 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4885.731380 secs 2024-03-28T01:57:06.871 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:57:06 smithi165 ceph-mon[36302]: pgmap v2844: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:57:06.894 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:57:06 smithi119 ceph-mon[31518]: pgmap v2844: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:57:06.942 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:57:06 smithi132 ceph-mon[35941]: pgmap v2844: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:57:08.121 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:57:07 smithi165 ceph-mon[36302]: pgmap v2845: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:57:08.144 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:57:07 smithi119 ceph-mon[31518]: pgmap v2845: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:57:08.191 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:57:07 smithi132 ceph-mon[35941]: pgmap v2845: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:57:10.121 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:57:09 smithi165 ceph-mon[36302]: pgmap v2846: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:57:10.144 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:57:09 smithi119 ceph-mon[31518]: pgmap v2846: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:57:10.191 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:57:09 smithi132 ceph-mon[35941]: pgmap v2846: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:57:11.121 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:57:10 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4890.731469 secs 2024-03-28T01:57:11.144 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:57:10 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4890.731469 secs 2024-03-28T01:57:11.191 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:57:10 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4890.731469 secs 2024-03-28T01:57:12.121 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:57:11 smithi165 ceph-mon[36302]: pgmap v2847: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:57:12.144 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:57:11 smithi119 ceph-mon[31518]: pgmap v2847: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:57:12.191 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:57:11 smithi132 ceph-mon[35941]: pgmap v2847: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:57:13.121 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:57:12 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:57:13.144 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:57:12 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:57:13.191 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:57:12 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:57:14.621 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:57:14 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:57:14.622 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:57:14 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:57:14.622 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:57:14 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:57:14.622 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:57:14 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:57:14.622 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:57:14 smithi165 ceph-mon[36302]: pgmap v2848: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:57:14.622 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:57:14 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:57:14.622 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:57:14 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:57:14.622 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:57:14 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:57:14.622 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:57:14 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:57:14.622 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:57:14 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:57:14.645 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:57:14 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:57:14.645 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:57:14 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:57:14.645 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:57:14 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:57:14.645 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:57:14 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:57:14.645 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:57:14 smithi119 ceph-mon[31518]: pgmap v2848: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:57:14.645 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:57:14 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:57:14.645 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:57:14 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:57:14.646 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:57:14 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:57:14.646 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:57:14 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:57:14.646 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:57:14 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:57:14.691 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:57:14 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:57:14.692 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:57:14 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:57:14.692 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:57:14 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:57:14.692 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:57:14 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:57:14.692 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:57:14 smithi132 ceph-mon[35941]: pgmap v2848: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:57:14.692 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:57:14 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:57:14.692 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:57:14 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:57:14.692 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:57:14 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:57:14.692 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:57:14 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:57:14.692 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:57:14 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:57:15.691 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:57:15 smithi132 ceph-mon[35941]: pgmap v2849: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-28T01:57:15.692 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:57:15 smithi132 ceph-mon[35941]: pgmap v2850: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-03-28T01:57:15.871 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:57:15 smithi165 ceph-mon[36302]: pgmap v2849: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-28T01:57:15.871 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:57:15 smithi165 ceph-mon[36302]: pgmap v2850: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-03-28T01:57:15.894 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:57:15 smithi119 ceph-mon[31518]: pgmap v2849: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-28T01:57:15.894 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:57:15 smithi119 ceph-mon[31518]: pgmap v2850: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-03-28T01:57:16.871 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:57:16 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4895.731559 secs 2024-03-28T01:57:16.894 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:57:16 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4895.731559 secs 2024-03-28T01:57:16.941 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:57:16 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4895.731559 secs 2024-03-28T01:57:17.871 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:57:17 smithi165 ceph-mon[36302]: pgmap v2851: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-03-28T01:57:17.894 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:57:17 smithi119 ceph-mon[31518]: pgmap v2851: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-03-28T01:57:17.941 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:57:17 smithi132 ceph-mon[35941]: pgmap v2851: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-03-28T01:57:19.121 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:57:18 smithi165 ceph-mon[36302]: pgmap v2852: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 3 op/s 2024-03-28T01:57:19.144 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:57:18 smithi119 ceph-mon[31518]: pgmap v2852: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 3 op/s 2024-03-28T01:57:19.191 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:57:18 smithi132 ceph-mon[35941]: pgmap v2852: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 3 op/s 2024-03-28T01:57:21.621 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:57:21 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4900.731621 secs 2024-03-28T01:57:21.621 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:57:21 smithi165 ceph-mon[36302]: pgmap v2853: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-03-28T01:57:21.644 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:57:21 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4900.731621 secs 2024-03-28T01:57:21.644 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:57:21 smithi119 ceph-mon[31518]: pgmap v2853: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-03-28T01:57:21.691 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:57:21 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4900.731621 secs 2024-03-28T01:57:21.691 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:57:21 smithi132 ceph-mon[35941]: pgmap v2853: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-03-28T01:57:23.621 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:57:23 smithi165 ceph-mon[36302]: pgmap v2854: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-03-28T01:57:23.644 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:57:23 smithi119 ceph-mon[31518]: pgmap v2854: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-03-28T01:57:23.691 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:57:23 smithi132 ceph-mon[35941]: pgmap v2854: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-03-28T01:57:25.621 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:57:25 smithi165 ceph-mon[36302]: pgmap v2855: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T01:57:25.644 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:57:25 smithi119 ceph-mon[31518]: pgmap v2855: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T01:57:25.691 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:57:25 smithi132 ceph-mon[35941]: pgmap v2855: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T01:57:26.621 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:57:26 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4905.731681 secs 2024-03-28T01:57:26.644 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:57:26 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4905.731681 secs 2024-03-28T01:57:26.691 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:57:26 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4905.731681 secs 2024-03-28T01:57:27.621 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:57:27 smithi165 ceph-mon[36302]: pgmap v2856: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:57:27.644 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:57:27 smithi119 ceph-mon[31518]: pgmap v2856: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:57:27.691 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:57:27 smithi132 ceph-mon[35941]: pgmap v2856: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:57:29.621 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:57:29 smithi165 ceph-mon[36302]: pgmap v2857: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:57:29.644 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:57:29 smithi119 ceph-mon[31518]: pgmap v2857: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:57:29.691 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:57:29 smithi132 ceph-mon[35941]: pgmap v2857: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:57:31.621 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:57:31 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4910.731777 secs 2024-03-28T01:57:31.621 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:57:31 smithi165 ceph-mon[36302]: pgmap v2858: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:57:31.644 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:57:31 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4910.731777 secs 2024-03-28T01:57:31.644 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:57:31 smithi119 ceph-mon[31518]: pgmap v2858: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:57:31.691 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:57:31 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4910.731777 secs 2024-03-28T01:57:31.691 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:57:31 smithi132 ceph-mon[35941]: pgmap v2858: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:57:33.621 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:57:33 smithi165 ceph-mon[36302]: pgmap v2859: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:57:33.644 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:57:33 smithi119 ceph-mon[31518]: pgmap v2859: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:57:33.691 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:57:33 smithi132 ceph-mon[35941]: pgmap v2859: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:57:35.621 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:57:35 smithi165 ceph-mon[36302]: pgmap v2860: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:57:35.644 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:57:35 smithi119 ceph-mon[31518]: pgmap v2860: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:57:35.691 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:57:35 smithi132 ceph-mon[35941]: pgmap v2860: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:57:36.621 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:57:36 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4915.731872 secs 2024-03-28T01:57:36.644 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:57:36 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4915.731872 secs 2024-03-28T01:57:36.691 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:57:36 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4915.731872 secs 2024-03-28T01:57:37.621 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:57:37 smithi165 ceph-mon[36302]: pgmap v2861: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:57:37.644 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:57:37 smithi119 ceph-mon[31518]: pgmap v2861: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:57:37.691 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:57:37 smithi132 ceph-mon[35941]: pgmap v2861: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:57:39.621 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:57:39 smithi165 ceph-mon[36302]: pgmap v2862: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:57:39.644 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:57:39 smithi119 ceph-mon[31518]: pgmap v2862: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:57:39.691 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:57:39 smithi132 ceph-mon[35941]: pgmap v2862: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:57:41.621 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:57:41 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4920.731901 secs 2024-03-28T01:57:41.621 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:57:41 smithi165 ceph-mon[36302]: pgmap v2863: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:57:41.644 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:57:41 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4920.731901 secs 2024-03-28T01:57:41.644 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:57:41 smithi119 ceph-mon[31518]: pgmap v2863: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:57:41.691 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:57:41 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4920.731901 secs 2024-03-28T01:57:41.691 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:57:41 smithi132 ceph-mon[35941]: pgmap v2863: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:57:43.621 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:57:43 smithi165 ceph-mon[36302]: pgmap v2864: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:57:43.644 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:57:43 smithi119 ceph-mon[31518]: pgmap v2864: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:57:43.691 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:57:43 smithi132 ceph-mon[35941]: pgmap v2864: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:57:45.621 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:57:45 smithi165 ceph-mon[36302]: pgmap v2865: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:57:45.644 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:57:45 smithi119 ceph-mon[31518]: pgmap v2865: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:57:45.691 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:57:45 smithi132 ceph-mon[35941]: pgmap v2865: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:57:46.621 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:57:46 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4925.731986 secs 2024-03-28T01:57:46.644 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:57:46 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4925.731986 secs 2024-03-28T01:57:46.691 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:57:46 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4925.731986 secs 2024-03-28T01:57:47.621 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:57:47 smithi165 ceph-mon[36302]: pgmap v2866: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:57:47.644 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:57:47 smithi119 ceph-mon[31518]: pgmap v2866: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:57:47.691 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:57:47 smithi132 ceph-mon[35941]: pgmap v2866: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:57:49.621 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:57:49 smithi165 ceph-mon[36302]: pgmap v2867: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:57:49.644 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:57:49 smithi119 ceph-mon[31518]: pgmap v2867: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:57:49.691 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:57:49 smithi132 ceph-mon[35941]: pgmap v2867: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:57:51.621 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:57:51 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4930.732074 secs 2024-03-28T01:57:51.621 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:57:51 smithi165 ceph-mon[36302]: pgmap v2868: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:57:51.644 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:57:51 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4930.732074 secs 2024-03-28T01:57:51.644 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:57:51 smithi119 ceph-mon[31518]: pgmap v2868: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:57:51.691 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:57:51 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4930.732074 secs 2024-03-28T01:57:51.691 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:57:51 smithi132 ceph-mon[35941]: pgmap v2868: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:57:53.621 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:57:53 smithi165 ceph-mon[36302]: pgmap v2869: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:57:53.644 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:57:53 smithi119 ceph-mon[31518]: pgmap v2869: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:57:53.691 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:57:53 smithi132 ceph-mon[35941]: pgmap v2869: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:57:55.621 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:57:55 smithi165 ceph-mon[36302]: pgmap v2870: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:57:55.644 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:57:55 smithi119 ceph-mon[31518]: pgmap v2870: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:57:55.691 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:57:55 smithi132 ceph-mon[35941]: pgmap v2870: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:57:56.621 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:57:56 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4935.732136 secs 2024-03-28T01:57:56.644 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:57:56 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4935.732136 secs 2024-03-28T01:57:56.691 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:57:56 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4935.732136 secs 2024-03-28T01:57:57.621 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:57:57 smithi165 ceph-mon[36302]: pgmap v2871: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:57:57.644 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:57:57 smithi119 ceph-mon[31518]: pgmap v2871: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:57:57.691 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:57:57 smithi132 ceph-mon[35941]: pgmap v2871: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:57:59.621 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:57:59 smithi165 ceph-mon[36302]: pgmap v2872: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:57:59.644 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:57:59 smithi119 ceph-mon[31518]: pgmap v2872: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:57:59.691 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:57:59 smithi132 ceph-mon[35941]: pgmap v2872: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:58:01.621 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:58:01 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4940.732247 secs 2024-03-28T01:58:01.621 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:58:01 smithi165 ceph-mon[36302]: pgmap v2873: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:58:01.644 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:58:01 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4940.732247 secs 2024-03-28T01:58:01.644 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:58:01 smithi119 ceph-mon[31518]: pgmap v2873: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:58:01.691 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:58:01 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4940.732247 secs 2024-03-28T01:58:01.691 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:58:01 smithi132 ceph-mon[35941]: pgmap v2873: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:58:03.621 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:58:03 smithi165 ceph-mon[36302]: pgmap v2874: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:58:03.644 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:58:03 smithi119 ceph-mon[31518]: pgmap v2874: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:58:03.691 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:58:03 smithi132 ceph-mon[35941]: pgmap v2874: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:58:05.621 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:58:05 smithi165 ceph-mon[36302]: pgmap v2875: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:58:05.644 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:58:05 smithi119 ceph-mon[31518]: pgmap v2875: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:58:05.691 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:58:05 smithi132 ceph-mon[35941]: pgmap v2875: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:58:06.644 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:58:06 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4945.732333 secs 2024-03-28T01:58:06.691 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:58:06 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4945.732333 secs 2024-03-28T01:58:06.871 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:58:06 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4945.732333 secs 2024-03-28T01:58:07.644 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:58:07 smithi119 ceph-mon[31518]: pgmap v2876: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:58:07.691 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:58:07 smithi132 ceph-mon[35941]: pgmap v2876: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:58:07.871 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:58:07 smithi165 ceph-mon[36302]: pgmap v2876: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:58:09.644 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:58:09 smithi119 ceph-mon[31518]: pgmap v2877: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:58:09.691 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:58:09 smithi132 ceph-mon[35941]: pgmap v2877: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:58:09.871 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:58:09 smithi165 ceph-mon[36302]: pgmap v2877: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:58:11.644 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:58:11 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4950.732393 secs 2024-03-28T01:58:11.644 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:58:11 smithi119 ceph-mon[31518]: pgmap v2878: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:58:11.691 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:58:11 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4950.732393 secs 2024-03-28T01:58:11.691 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:58:11 smithi132 ceph-mon[35941]: pgmap v2878: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:58:11.871 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:58:11 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4950.732393 secs 2024-03-28T01:58:11.871 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:58:11 smithi165 ceph-mon[36302]: pgmap v2878: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:58:13.644 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:58:13 smithi119 ceph-mon[31518]: pgmap v2879: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:58:13.691 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:58:13 smithi132 ceph-mon[35941]: pgmap v2879: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:58:13.871 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:58:13 smithi165 ceph-mon[36302]: pgmap v2879: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:58:15.644 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:58:15 smithi119 ceph-mon[31518]: pgmap v2880: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:58:15.644 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:58:15 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:58:15.691 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:58:15 smithi132 ceph-mon[35941]: pgmap v2880: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:58:15.691 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:58:15 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:58:15.871 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:58:15 smithi165 ceph-mon[36302]: pgmap v2880: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:58:15.871 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:58:15 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:58:16.691 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:58:16 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4955.732463 secs 2024-03-28T01:58:16.691 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:58:16 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:58:16.692 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:58:16 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:58:16.692 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:58:16 smithi132 ceph-mon[35941]: pgmap v2881: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T01:58:16.692 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:58:16 smithi132 ceph-mon[35941]: pgmap v2882: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:58:16.692 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:58:16 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:58:16.871 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:58:16 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4955.732463 secs 2024-03-28T01:58:16.871 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:58:16 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:58:16.871 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:58:16 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:58:16.871 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:58:16 smithi165 ceph-mon[36302]: pgmap v2881: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T01:58:16.871 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:58:16 smithi165 ceph-mon[36302]: pgmap v2882: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:58:16.871 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:58:16 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:58:16.894 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:58:16 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4955.732463 secs 2024-03-28T01:58:16.894 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:58:16 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:58:16.894 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:58:16 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:58:16.894 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:58:16 smithi119 ceph-mon[31518]: pgmap v2881: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T01:58:16.894 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:58:16 smithi119 ceph-mon[31518]: pgmap v2882: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:58:16.895 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:58:16 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:58:19.121 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:58:18 smithi165 ceph-mon[36302]: pgmap v2883: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:58:19.144 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:58:18 smithi119 ceph-mon[31518]: pgmap v2883: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:58:19.191 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:58:18 smithi132 ceph-mon[35941]: pgmap v2883: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:58:21.121 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:58:20 smithi165 ceph-mon[36302]: pgmap v2884: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:58:21.121 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:58:20 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4960.732477 secs 2024-03-28T01:58:21.144 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:58:20 smithi119 ceph-mon[31518]: pgmap v2884: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:58:21.144 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:58:20 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4960.732477 secs 2024-03-28T01:58:21.191 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:58:20 smithi132 ceph-mon[35941]: pgmap v2884: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:58:21.191 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:58:20 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4960.732477 secs 2024-03-28T01:58:23.121 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:58:22 smithi165 ceph-mon[36302]: pgmap v2885: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:58:23.144 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:58:22 smithi119 ceph-mon[31518]: pgmap v2885: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:58:23.191 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:58:22 smithi132 ceph-mon[35941]: pgmap v2885: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:58:25.121 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:58:24 smithi165 ceph-mon[36302]: pgmap v2886: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-28T01:58:25.144 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:58:24 smithi119 ceph-mon[31518]: pgmap v2886: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-28T01:58:25.191 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:58:24 smithi132 ceph-mon[35941]: pgmap v2886: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-28T01:58:26.022 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:58:25 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4965.732565 secs 2024-03-28T01:58:26.046 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:58:25 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4965.732565 secs 2024-03-28T01:58:26.191 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:58:25 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4965.732565 secs 2024-03-28T01:58:27.121 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:58:26 smithi165 ceph-mon[36302]: pgmap v2887: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T01:58:27.144 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:58:26 smithi119 ceph-mon[31518]: pgmap v2887: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T01:58:27.191 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:58:26 smithi132 ceph-mon[35941]: pgmap v2887: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T01:58:29.121 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:58:28 smithi165 ceph-mon[36302]: pgmap v2888: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:58:29.144 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:58:28 smithi119 ceph-mon[31518]: pgmap v2888: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:58:29.191 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:58:28 smithi132 ceph-mon[35941]: pgmap v2888: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:58:31.121 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:58:30 smithi165 ceph-mon[36302]: pgmap v2889: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:58:31.121 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:58:30 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4970.732655 secs 2024-03-28T01:58:31.144 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:58:30 smithi119 ceph-mon[31518]: pgmap v2889: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:58:31.144 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:58:30 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4970.732655 secs 2024-03-28T01:58:31.191 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:58:30 smithi132 ceph-mon[35941]: pgmap v2889: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:58:31.191 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:58:30 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4970.732655 secs 2024-03-28T01:58:33.121 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:58:32 smithi165 ceph-mon[36302]: pgmap v2890: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:58:33.144 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:58:32 smithi119 ceph-mon[31518]: pgmap v2890: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:58:33.191 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:58:32 smithi132 ceph-mon[35941]: pgmap v2890: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:58:35.121 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:58:34 smithi165 ceph-mon[36302]: pgmap v2891: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:58:35.144 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:58:34 smithi119 ceph-mon[31518]: pgmap v2891: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:58:35.191 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:58:34 smithi132 ceph-mon[35941]: pgmap v2891: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:58:36.121 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:58:35 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4975.732745 secs 2024-03-28T01:58:36.144 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:58:35 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4975.732745 secs 2024-03-28T01:58:36.191 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:58:35 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4975.732745 secs 2024-03-28T01:58:37.121 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:58:36 smithi165 ceph-mon[36302]: pgmap v2892: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:58:37.144 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:58:36 smithi119 ceph-mon[31518]: pgmap v2892: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:58:37.191 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:58:36 smithi132 ceph-mon[35941]: pgmap v2892: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:58:38.121 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:58:37 smithi165 ceph-mon[36302]: pgmap v2893: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:58:38.144 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:58:37 smithi119 ceph-mon[31518]: pgmap v2893: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:58:38.191 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:58:37 smithi132 ceph-mon[35941]: pgmap v2893: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:58:41.121 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:58:40 smithi165 ceph-mon[36302]: pgmap v2894: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:58:41.121 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:58:40 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4980.732807 secs 2024-03-28T01:58:41.144 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:58:40 smithi119 ceph-mon[31518]: pgmap v2894: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:58:41.144 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:58:40 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4980.732807 secs 2024-03-28T01:58:41.191 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:58:40 smithi132 ceph-mon[35941]: pgmap v2894: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:58:41.191 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:58:40 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4980.732807 secs 2024-03-28T01:58:43.121 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:58:42 smithi165 ceph-mon[36302]: pgmap v2895: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:58:43.144 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:58:42 smithi119 ceph-mon[31518]: pgmap v2895: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:58:43.191 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:58:42 smithi132 ceph-mon[35941]: pgmap v2895: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:58:45.121 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:58:44 smithi165 ceph-mon[36302]: pgmap v2896: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:58:45.144 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:58:44 smithi119 ceph-mon[31518]: pgmap v2896: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:58:45.191 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:58:44 smithi132 ceph-mon[35941]: pgmap v2896: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:58:46.121 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:58:45 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4985.732890 secs 2024-03-28T01:58:46.144 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:58:45 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4985.732890 secs 2024-03-28T01:58:46.191 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:58:45 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4985.732890 secs 2024-03-28T01:58:47.066 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:58:46 smithi119 ceph-mon[31518]: pgmap v2897: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:58:47.121 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:58:46 smithi165 ceph-mon[36302]: pgmap v2897: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:58:47.191 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:58:46 smithi132 ceph-mon[35941]: pgmap v2897: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:58:49.121 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:58:48 smithi165 ceph-mon[36302]: pgmap v2898: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:58:49.144 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:58:48 smithi119 ceph-mon[31518]: pgmap v2898: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:58:49.191 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:58:48 smithi132 ceph-mon[35941]: pgmap v2898: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:58:51.121 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:58:50 smithi165 ceph-mon[36302]: pgmap v2899: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:58:51.121 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:58:50 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4990.732978 secs 2024-03-28T01:58:51.144 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:58:50 smithi119 ceph-mon[31518]: pgmap v2899: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:58:51.144 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:58:50 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4990.732978 secs 2024-03-28T01:58:51.191 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:58:50 smithi132 ceph-mon[35941]: pgmap v2899: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:58:51.191 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:58:50 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4990.732978 secs 2024-03-28T01:58:53.121 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:58:52 smithi165 ceph-mon[36302]: pgmap v2900: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:58:53.144 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:58:52 smithi119 ceph-mon[31518]: pgmap v2900: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:58:53.191 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:58:52 smithi132 ceph-mon[35941]: pgmap v2900: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:58:55.121 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:58:54 smithi165 ceph-mon[36302]: pgmap v2901: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:58:55.144 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:58:54 smithi119 ceph-mon[31518]: pgmap v2901: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:58:55.191 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:58:54 smithi132 ceph-mon[35941]: pgmap v2901: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:58:56.046 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:58:55 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 4995.732990 secs 2024-03-28T01:58:56.046 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:58:55 smithi119 ceph-mon[31518]: pgmap v2902: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:58:56.121 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:58:55 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 4995.732990 secs 2024-03-28T01:58:56.121 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:58:55 smithi165 ceph-mon[36302]: pgmap v2902: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:58:56.191 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:58:55 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 4995.732990 secs 2024-03-28T01:58:56.191 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:58:55 smithi132 ceph-mon[35941]: pgmap v2902: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:58:59.121 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:58:58 smithi165 ceph-mon[36302]: pgmap v2903: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:58:59.144 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:58:58 smithi119 ceph-mon[31518]: pgmap v2903: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:58:59.191 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:58:58 smithi132 ceph-mon[35941]: pgmap v2903: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:59:01.121 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:59:00 smithi165 ceph-mon[36302]: pgmap v2904: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:59:01.121 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:59:00 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5000.733051 secs 2024-03-28T01:59:01.144 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:59:00 smithi119 ceph-mon[31518]: pgmap v2904: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:59:01.144 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:59:00 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5000.733051 secs 2024-03-28T01:59:01.191 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:59:00 smithi132 ceph-mon[35941]: pgmap v2904: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:59:01.191 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:59:00 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5000.733051 secs 2024-03-28T01:59:03.121 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:59:02 smithi165 ceph-mon[36302]: pgmap v2905: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:59:03.144 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:59:02 smithi119 ceph-mon[31518]: pgmap v2905: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:59:03.191 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:59:02 smithi132 ceph-mon[35941]: pgmap v2905: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:59:04.092 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:59:03 smithi132 ceph-mon[35941]: pgmap v2906: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:59:04.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:59:03 smithi165 ceph-mon[36302]: pgmap v2906: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:59:04.144 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:59:03 smithi119 ceph-mon[31518]: pgmap v2906: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:59:05.871 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:59:05 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5005.733135 secs 2024-03-28T01:59:05.894 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:59:05 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5005.733135 secs 2024-03-28T01:59:05.941 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:59:05 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5005.733135 secs 2024-03-28T01:59:06.871 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:59:06 smithi165 ceph-mon[36302]: pgmap v2907: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:59:06.894 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:59:06 smithi119 ceph-mon[31518]: pgmap v2907: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:59:06.941 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:59:06 smithi132 ceph-mon[35941]: pgmap v2907: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:59:09.121 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:59:08 smithi165 ceph-mon[36302]: pgmap v2908: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:59:09.144 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:59:08 smithi119 ceph-mon[31518]: pgmap v2908: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:59:09.191 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:59:08 smithi132 ceph-mon[35941]: pgmap v2908: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:59:11.121 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:59:10 smithi165 ceph-mon[36302]: pgmap v2909: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:59:11.121 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:59:10 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5010.733197 secs 2024-03-28T01:59:11.144 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:59:10 smithi119 ceph-mon[31518]: pgmap v2909: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:59:11.144 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:59:10 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5010.733197 secs 2024-03-28T01:59:11.191 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:59:10 smithi132 ceph-mon[35941]: pgmap v2909: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:59:11.191 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:59:10 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5010.733197 secs 2024-03-28T01:59:13.121 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:59:12 smithi165 ceph-mon[36302]: pgmap v2910: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:59:13.144 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:59:12 smithi119 ceph-mon[31518]: pgmap v2910: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:59:13.191 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:59:12 smithi132 ceph-mon[35941]: pgmap v2910: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:59:15.121 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:59:14 smithi165 ceph-mon[36302]: pgmap v2911: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:59:15.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:59:14 smithi119 ceph-mon[31518]: pgmap v2911: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:59:15.191 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:59:14 smithi132 ceph-mon[35941]: pgmap v2911: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:59:16.121 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:59:15 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5015.733257 secs 2024-03-28T01:59:16.144 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:59:15 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5015.733257 secs 2024-03-28T01:59:16.191 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:59:15 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5015.733257 secs 2024-03-28T01:59:17.121 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:59:16 smithi165 ceph-mon[36302]: pgmap v2912: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:59:17.121 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:59:16 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:59:17.144 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:59:16 smithi119 ceph-mon[31518]: pgmap v2912: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:59:17.144 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:59:16 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:59:17.191 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:59:16 smithi132 ceph-mon[35941]: pgmap v2912: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:59:17.191 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:59:16 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T01:59:18.121 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:59:17 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:59:18.121 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:59:17 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:59:18.121 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:59:17 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:59:18.144 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:59:17 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:59:18.144 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:59:17 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:59:18.144 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:59:17 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:59:18.191 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:59:17 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T01:59:18.191 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:59:17 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T01:59:18.191 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:59:17 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T01:59:19.121 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:59:18 smithi165 ceph-mon[36302]: pgmap v2913: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T01:59:19.121 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:59:18 smithi165 ceph-mon[36302]: pgmap v2914: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:59:19.144 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:59:18 smithi119 ceph-mon[31518]: pgmap v2913: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T01:59:19.144 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:59:18 smithi119 ceph-mon[31518]: pgmap v2914: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:59:19.191 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:59:18 smithi132 ceph-mon[35941]: pgmap v2913: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T01:59:19.191 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:59:18 smithi132 ceph-mon[35941]: pgmap v2914: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:59:20.121 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:59:19 smithi165 ceph-mon[36302]: pgmap v2915: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:59:20.144 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:59:19 smithi119 ceph-mon[31518]: pgmap v2915: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:59:20.191 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:59:19 smithi132 ceph-mon[35941]: pgmap v2915: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:59:21.121 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:59:20 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5020.733344 secs 2024-03-28T01:59:21.144 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:59:20 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5020.733344 secs 2024-03-28T01:59:21.191 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:59:20 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5020.733344 secs 2024-03-28T01:59:22.121 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:59:21 smithi165 ceph-mon[36302]: pgmap v2916: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:59:22.144 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:59:21 smithi119 ceph-mon[31518]: pgmap v2916: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:59:22.191 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:59:21 smithi132 ceph-mon[35941]: pgmap v2916: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:59:24.621 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:59:24 smithi165 ceph-mon[36302]: pgmap v2917: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:59:24.643 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:59:24 smithi119 ceph-mon[31518]: pgmap v2917: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:59:24.691 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:59:24 smithi132 ceph-mon[35941]: pgmap v2917: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:59:26.621 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:59:26 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5025.733384 secs 2024-03-28T01:59:26.621 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:59:26 smithi165 ceph-mon[36302]: pgmap v2918: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:59:26.644 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:59:26 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5025.733384 secs 2024-03-28T01:59:26.644 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:59:26 smithi119 ceph-mon[31518]: pgmap v2918: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:59:26.691 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:59:26 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5025.733384 secs 2024-03-28T01:59:26.691 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:59:26 smithi132 ceph-mon[35941]: pgmap v2918: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T01:59:28.621 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:59:28 smithi165 ceph-mon[36302]: pgmap v2919: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T01:59:28.644 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:59:28 smithi119 ceph-mon[31518]: pgmap v2919: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T01:59:28.691 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:59:28 smithi132 ceph-mon[35941]: pgmap v2919: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T01:59:30.621 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:59:30 smithi165 ceph-mon[36302]: pgmap v2920: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:59:30.644 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:59:30 smithi119 ceph-mon[31518]: pgmap v2920: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:59:30.691 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:59:30 smithi132 ceph-mon[35941]: pgmap v2920: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:59:31.620 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:59:31 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5030.733400 secs 2024-03-28T01:59:31.644 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:59:31 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5030.733400 secs 2024-03-28T01:59:31.691 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:59:31 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5030.733400 secs 2024-03-28T01:59:32.621 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:59:32 smithi165 ceph-mon[36302]: pgmap v2921: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:59:32.644 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:59:32 smithi119 ceph-mon[31518]: pgmap v2921: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:59:32.691 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:59:32 smithi132 ceph-mon[35941]: pgmap v2921: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:59:34.620 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:59:34 smithi165 ceph-mon[36302]: pgmap v2922: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:59:34.644 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:59:34 smithi119 ceph-mon[31518]: pgmap v2922: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:59:34.691 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:59:34 smithi132 ceph-mon[35941]: pgmap v2922: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:59:36.621 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:59:36 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5035.733487 secs 2024-03-28T01:59:36.621 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:59:36 smithi165 ceph-mon[36302]: pgmap v2923: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:59:36.644 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:59:36 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5035.733487 secs 2024-03-28T01:59:36.644 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:59:36 smithi119 ceph-mon[31518]: pgmap v2923: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:59:36.691 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:59:36 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5035.733487 secs 2024-03-28T01:59:36.691 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:59:36 smithi132 ceph-mon[35941]: pgmap v2923: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:59:38.620 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:59:38 smithi165 ceph-mon[36302]: pgmap v2924: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:59:38.643 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:59:38 smithi119 ceph-mon[31518]: pgmap v2924: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:59:38.691 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:59:38 smithi132 ceph-mon[35941]: pgmap v2924: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:59:40.621 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:59:40 smithi165 ceph-mon[36302]: pgmap v2925: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:59:40.643 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:59:40 smithi119 ceph-mon[31518]: pgmap v2925: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:59:40.691 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:59:40 smithi132 ceph-mon[35941]: pgmap v2925: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:59:41.620 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:59:41 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5040.733548 secs 2024-03-28T01:59:41.643 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:59:41 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5040.733548 secs 2024-03-28T01:59:41.691 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:59:41 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5040.733548 secs 2024-03-28T01:59:42.621 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:59:42 smithi165 ceph-mon[36302]: pgmap v2926: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:59:42.644 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:59:42 smithi119 ceph-mon[31518]: pgmap v2926: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:59:42.691 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:59:42 smithi132 ceph-mon[35941]: pgmap v2926: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:59:44.620 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:59:44 smithi165 ceph-mon[36302]: pgmap v2927: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:59:44.644 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:59:44 smithi119 ceph-mon[31518]: pgmap v2927: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:59:44.691 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:59:44 smithi132 ceph-mon[35941]: pgmap v2927: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:59:46.621 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:59:46 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5045.733637 secs 2024-03-28T01:59:46.621 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:59:46 smithi165 ceph-mon[36302]: pgmap v2928: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:59:46.644 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:59:46 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5045.733637 secs 2024-03-28T01:59:46.644 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:59:46 smithi119 ceph-mon[31518]: pgmap v2928: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:59:46.691 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:59:46 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5045.733637 secs 2024-03-28T01:59:46.691 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:59:46 smithi132 ceph-mon[35941]: pgmap v2928: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:59:48.620 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:59:48 smithi165 ceph-mon[36302]: pgmap v2929: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:59:48.644 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:59:48 smithi119 ceph-mon[31518]: pgmap v2929: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:59:48.691 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:59:48 smithi132 ceph-mon[35941]: pgmap v2929: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:59:50.620 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:59:50 smithi165 ceph-mon[36302]: pgmap v2930: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:59:50.644 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:59:50 smithi119 ceph-mon[31518]: pgmap v2930: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:59:50.691 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:59:50 smithi132 ceph-mon[35941]: pgmap v2930: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:59:51.620 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:59:51 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5050.733725 secs 2024-03-28T01:59:51.643 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:59:51 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5050.733725 secs 2024-03-28T01:59:51.691 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:59:51 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5050.733725 secs 2024-03-28T01:59:52.621 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:59:52 smithi165 ceph-mon[36302]: pgmap v2931: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:59:52.644 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:59:52 smithi119 ceph-mon[31518]: pgmap v2931: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:59:52.691 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:59:52 smithi132 ceph-mon[35941]: pgmap v2931: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:59:54.620 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:59:54 smithi165 ceph-mon[36302]: pgmap v2932: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:59:54.644 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:59:54 smithi119 ceph-mon[31518]: pgmap v2932: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:59:54.691 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:59:54 smithi132 ceph-mon[35941]: pgmap v2932: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:59:56.620 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:59:56 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5055.733814 secs 2024-03-28T01:59:56.621 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:59:56 smithi165 ceph-mon[36302]: pgmap v2933: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:59:56.644 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:59:56 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5055.733814 secs 2024-03-28T01:59:56.644 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:59:56 smithi119 ceph-mon[31518]: pgmap v2933: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:59:56.691 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:59:56 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5055.733814 secs 2024-03-28T01:59:56.691 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:59:56 smithi132 ceph-mon[35941]: pgmap v2933: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T01:59:58.620 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 01:59:58 smithi165 ceph-mon[36302]: pgmap v2934: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:59:58.644 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 01:59:58 smithi119 ceph-mon[31518]: pgmap v2934: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T01:59:58.691 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 01:59:58 smithi132 ceph-mon[35941]: pgmap v2934: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:00:00.620 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:00:00 smithi165 ceph-mon[36302]: pgmap v2935: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:00:00.621 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:00:00 smithi165 ceph-mon[36302]: overall HEALTH_WARN 5 failed cephadm daemon(s); 1 filesystem is degraded; 1 MDSs report slow requests 2024-03-28T02:00:00.644 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:00:00 smithi119 ceph-mon[31518]: pgmap v2935: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:00:00.644 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:00:00 smithi119 ceph-mon[31518]: overall HEALTH_WARN 5 failed cephadm daemon(s); 1 filesystem is degraded; 1 MDSs report slow requests 2024-03-28T02:00:00.691 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:00:00 smithi132 ceph-mon[35941]: pgmap v2935: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:00:00.691 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:00:00 smithi132 ceph-mon[35941]: overall HEALTH_WARN 5 failed cephadm daemon(s); 1 filesystem is degraded; 1 MDSs report slow requests 2024-03-28T02:00:01.644 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:00:01 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5060.733919 secs 2024-03-28T02:00:01.691 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:00:01 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5060.733919 secs 2024-03-28T02:00:01.870 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:00:01 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5060.733919 secs 2024-03-28T02:00:02.644 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:00:02 smithi119 ceph-mon[31518]: pgmap v2936: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:00:02.691 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:00:02 smithi132 ceph-mon[35941]: pgmap v2936: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:00:02.870 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:00:02 smithi165 ceph-mon[36302]: pgmap v2936: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:00:04.644 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:00:04 smithi119 ceph-mon[31518]: pgmap v2937: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:00:04.691 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:00:04 smithi132 ceph-mon[35941]: pgmap v2937: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:00:04.870 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:00:04 smithi165 ceph-mon[36302]: pgmap v2937: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:00:06.644 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:00:06 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5065.734007 secs 2024-03-28T02:00:06.644 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:00:06 smithi119 ceph-mon[31518]: pgmap v2938: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:00:06.691 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:00:06 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5065.734007 secs 2024-03-28T02:00:06.691 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:00:06 smithi132 ceph-mon[35941]: pgmap v2938: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:00:06.871 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:00:06 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5065.734007 secs 2024-03-28T02:00:06.871 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:00:06 smithi165 ceph-mon[36302]: pgmap v2938: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:00:08.691 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:00:08 smithi132 ceph-mon[35941]: pgmap v2939: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T02:00:08.870 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:00:08 smithi165 ceph-mon[36302]: pgmap v2939: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T02:00:08.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:00:08 smithi119 ceph-mon[31518]: pgmap v2939: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T02:00:10.644 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:00:10 smithi119 ceph-mon[31518]: pgmap v2940: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:00:10.691 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:00:10 smithi132 ceph-mon[35941]: pgmap v2940: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:00:10.870 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:00:10 smithi165 ceph-mon[36302]: pgmap v2940: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:00:11.691 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:00:11 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5070.734023 secs 2024-03-28T02:00:11.870 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:00:11 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5070.734023 secs 2024-03-28T02:00:11.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:00:11 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5070.734023 secs 2024-03-28T02:00:12.691 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:00:12 smithi132 ceph-mon[35941]: pgmap v2941: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T02:00:12.870 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:00:12 smithi165 ceph-mon[36302]: pgmap v2941: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T02:00:12.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:00:12 smithi119 ceph-mon[31518]: pgmap v2941: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T02:00:14.691 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:00:14 smithi132 ceph-mon[35941]: pgmap v2942: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T02:00:14.870 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:00:14 smithi165 ceph-mon[36302]: pgmap v2942: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T02:00:14.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:00:14 smithi119 ceph-mon[31518]: pgmap v2942: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T02:00:16.691 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:00:16 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5075.734060 secs 2024-03-28T02:00:16.691 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:00:16 smithi132 ceph-mon[35941]: pgmap v2943: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:00:16.870 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:00:16 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5075.734060 secs 2024-03-28T02:00:16.871 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:00:16 smithi165 ceph-mon[36302]: pgmap v2943: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:00:16.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:00:16 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5075.734060 secs 2024-03-28T02:00:16.894 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:00:16 smithi119 ceph-mon[31518]: pgmap v2943: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:00:18.620 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:00:18 smithi165 ceph-mon[36302]: pgmap v2944: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:00:18.621 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:00:18 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:00:18.621 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:00:18 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:00:18.621 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:00:18 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:00:18.621 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:00:18 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:00:18.621 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:00:18 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:00:18.691 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:00:18 smithi132 ceph-mon[35941]: pgmap v2944: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:00:18.691 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:00:18 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:00:18.691 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:00:18 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:00:18.691 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:00:18 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:00:18.691 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:00:18 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:00:18.691 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:00:18 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:00:18.894 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:00:18 smithi119 ceph-mon[31518]: pgmap v2944: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:00:18.894 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:00:18 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:00:18.894 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:00:18 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:00:18.894 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:00:18 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:00:18.894 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:00:18 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:00:18.894 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:00:18 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:00:20.691 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:00:20 smithi132 ceph-mon[35941]: pgmap v2945: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:00:20.691 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:00:20 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:00:20.691 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:00:20 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:00:20.691 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:00:20 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:00:20.691 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:00:20 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:00:20.870 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:00:20 smithi165 ceph-mon[36302]: pgmap v2945: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:00:20.870 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:00:20 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:00:20.871 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:00:20 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:00:20.871 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:00:20 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:00:20.871 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:00:20 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:00:20.894 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:00:20 smithi119 ceph-mon[31518]: pgmap v2945: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:00:20.894 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:00:20 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:00:20.894 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:00:20 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:00:20.894 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:00:20 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:00:20.894 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:00:20 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:00:21.691 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:00:21 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5080.734081 secs 2024-03-28T02:00:21.691 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:00:21 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:00:21.692 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:00:21 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:00:21.692 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:00:21 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:00:21.692 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:00:21 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:00:21.692 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:00:21 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:00:21.692 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:00:21 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:00:21.692 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:00:21 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:00:21.692 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:00:21 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:00:21.692 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:00:21 smithi132 ceph-mon[35941]: Adjusting osd_memory_target on smithi132 to 174.1M 2024-03-28T02:00:21.692 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:00:21 smithi132 ceph-mon[35941]: Unable to set osd_memory_target on smithi132 to 182640640: error parsing value: Value '182640640' is below minimum 939524096 2024-03-28T02:00:21.692 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:00:21 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:00:21.692 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:00:21 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:00:21.692 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:00:21 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:00:21.692 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:00:21 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:00:21.692 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:00:21 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:00:21.692 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:00:21 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:00:21.692 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:00:21 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:00:21.693 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:00:21 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:00:21.693 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:00:21 smithi132 ceph-mon[35941]: Adjusting osd_memory_target on smithi165 to 174.1M 2024-03-28T02:00:21.693 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:00:21 smithi132 ceph-mon[35941]: Unable to set osd_memory_target on smithi165 to 182635622: error parsing value: Value '182635622' is below minimum 939524096 2024-03-28T02:00:21.871 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:00:21 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5080.734081 secs 2024-03-28T02:00:21.871 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:00:21 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:00:21.871 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:00:21 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:00:21.871 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:00:21 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:00:21.871 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:00:21 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:00:21.871 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:00:21 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:00:21.871 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:00:21 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:00:21.871 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:00:21 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:00:21.871 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:00:21 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:00:21.871 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:00:21 smithi165 ceph-mon[36302]: Adjusting osd_memory_target on smithi132 to 174.1M 2024-03-28T02:00:21.872 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:00:21 smithi165 ceph-mon[36302]: Unable to set osd_memory_target on smithi132 to 182640640: error parsing value: Value '182640640' is below minimum 939524096 2024-03-28T02:00:21.872 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:00:21 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:00:21.872 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:00:21 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:00:21.872 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:00:21 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:00:21.872 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:00:21 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:00:21.872 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:00:21 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:00:21.872 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:00:21 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:00:21.872 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:00:21 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:00:21.872 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:00:21 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:00:21.872 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:00:21 smithi165 ceph-mon[36302]: Adjusting osd_memory_target on smithi165 to 174.1M 2024-03-28T02:00:21.872 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:00:21 smithi165 ceph-mon[36302]: Unable to set osd_memory_target on smithi165 to 182635622: error parsing value: Value '182635622' is below minimum 939524096 2024-03-28T02:00:21.894 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:00:21 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5080.734081 secs 2024-03-28T02:00:21.894 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:00:21 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:00:21.894 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:00:21 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:00:21.894 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:00:21 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:00:21.894 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:00:21 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:00:21.895 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:00:21 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:00:21.895 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:00:21 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:00:21.895 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:00:21 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:00:21.895 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:00:21 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:00:21.895 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:00:21 smithi119 ceph-mon[31518]: Adjusting osd_memory_target on smithi132 to 174.1M 2024-03-28T02:00:21.895 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:00:21 smithi119 ceph-mon[31518]: Unable to set osd_memory_target on smithi132 to 182640640: error parsing value: Value '182640640' is below minimum 939524096 2024-03-28T02:00:21.895 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:00:21 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:00:21.895 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:00:21 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:00:21.895 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:00:21 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:00:21.895 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:00:21 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:00:21.895 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:00:21 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:00:21.895 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:00:21 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:00:21.895 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:00:21 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:00:21.895 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:00:21 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:00:21.895 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:00:21 smithi119 ceph-mon[31518]: Adjusting osd_memory_target on smithi165 to 174.1M 2024-03-28T02:00:21.895 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:00:21 smithi119 ceph-mon[31518]: Unable to set osd_memory_target on smithi165 to 182635622: error parsing value: Value '182635622' is below minimum 939524096 2024-03-28T02:00:22.691 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:00:22 smithi132 ceph-mon[35941]: pgmap v2946: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:00:22.691 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:00:22 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:00:22.691 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:00:22 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:00:22.870 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:00:22 smithi165 ceph-mon[36302]: pgmap v2946: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:00:22.870 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:00:22 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:00:22.871 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:00:22 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:00:22.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:00:22 smithi119 ceph-mon[31518]: pgmap v2946: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:00:22.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:00:22 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:00:22.894 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:00:22 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:00:23.871 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:00:23 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:00:23.871 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:00:23 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:00:23.871 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:00:23 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:00:23.871 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:00:23 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:00:23.871 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:00:23 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:00:23.871 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:00:23 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:00:23.871 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:00:23 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:00:23.871 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:00:23 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:00:23.871 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:00:23 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:00:23.871 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:00:23 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:00:23.871 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:00:23 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:00:23.894 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:00:23 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:00:23.894 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:00:23 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:00:23.894 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:00:23 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:00:23.894 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:00:23 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:00:23.894 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:00:23 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:00:23.894 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:00:23 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:00:23.894 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:00:23 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:00:23.894 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:00:23 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:00:23.895 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:00:23 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:00:23.895 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:00:23 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:00:23.895 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:00:23 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:00:23.941 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:00:23 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:00:23.941 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:00:23 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:00:23.941 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:00:23 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:00:23.941 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:00:23 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:00:23.941 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:00:23 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:00:23.941 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:00:23 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:00:23.941 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:00:23 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:00:23.941 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:00:23 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:00:23.941 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:00:23 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:00:23.941 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:00:23 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:00:23.942 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:00:23 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:00:24.870 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:00:24 smithi165 ceph-mon[36302]: Adjusting osd_memory_target on smithi119 to 176.7M 2024-03-28T02:00:24.871 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:00:24 smithi165 ceph-mon[36302]: Unable to set osd_memory_target on smithi119 to 185372364: error parsing value: Value '185372364' is below minimum 939524096 2024-03-28T02:00:24.871 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:00:24 smithi165 ceph-mon[36302]: pgmap v2947: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T02:00:24.871 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:00:24 smithi165 ceph-mon[36302]: pgmap v2948: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T02:00:24.894 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:00:24 smithi119 ceph-mon[31518]: Adjusting osd_memory_target on smithi119 to 176.7M 2024-03-28T02:00:24.894 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:00:24 smithi119 ceph-mon[31518]: Unable to set osd_memory_target on smithi119 to 185372364: error parsing value: Value '185372364' is below minimum 939524096 2024-03-28T02:00:24.894 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:00:24 smithi119 ceph-mon[31518]: pgmap v2947: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T02:00:24.894 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:00:24 smithi119 ceph-mon[31518]: pgmap v2948: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T02:00:24.941 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:00:24 smithi132 ceph-mon[35941]: Adjusting osd_memory_target on smithi119 to 176.7M 2024-03-28T02:00:24.941 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:00:24 smithi132 ceph-mon[35941]: Unable to set osd_memory_target on smithi119 to 185372364: error parsing value: Value '185372364' is below minimum 939524096 2024-03-28T02:00:24.941 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:00:24 smithi132 ceph-mon[35941]: pgmap v2947: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T02:00:24.941 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:00:24 smithi132 ceph-mon[35941]: pgmap v2948: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T02:00:26.871 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:00:26 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5085.734142 secs 2024-03-28T02:00:26.871 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:00:26 smithi165 ceph-mon[36302]: pgmap v2949: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T02:00:26.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:00:26 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5085.734142 secs 2024-03-28T02:00:26.894 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:00:26 smithi119 ceph-mon[31518]: pgmap v2949: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T02:00:26.941 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:00:26 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5085.734142 secs 2024-03-28T02:00:26.941 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:00:26 smithi132 ceph-mon[35941]: pgmap v2949: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T02:00:28.643 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:00:28 smithi119 ceph-mon[31518]: pgmap v2950: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T02:00:28.870 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:00:28 smithi165 ceph-mon[36302]: pgmap v2950: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T02:00:28.941 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:00:28 smithi132 ceph-mon[35941]: pgmap v2950: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T02:00:30.870 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:00:30 smithi165 ceph-mon[36302]: pgmap v2951: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T02:00:30.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:00:30 smithi119 ceph-mon[31518]: pgmap v2951: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T02:00:30.941 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:00:30 smithi132 ceph-mon[35941]: pgmap v2951: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T02:00:31.870 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:00:31 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5090.734203 secs 2024-03-28T02:00:31.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:00:31 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5090.734203 secs 2024-03-28T02:00:31.941 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:00:31 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5090.734203 secs 2024-03-28T02:00:32.870 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:00:32 smithi165 ceph-mon[36302]: pgmap v2952: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T02:00:32.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:00:32 smithi119 ceph-mon[31518]: pgmap v2952: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T02:00:32.941 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:00:32 smithi132 ceph-mon[35941]: pgmap v2952: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T02:00:34.743 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:00:34 smithi132 ceph-mon[35941]: pgmap v2953: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T02:00:34.870 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:00:34 smithi165 ceph-mon[36302]: pgmap v2953: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T02:00:34.894 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:00:34 smithi119 ceph-mon[31518]: pgmap v2953: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T02:00:36.870 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:00:36 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5095.734274 secs 2024-03-28T02:00:36.871 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:00:36 smithi165 ceph-mon[36302]: pgmap v2954: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:00:36.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:00:36 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5095.734274 secs 2024-03-28T02:00:36.894 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:00:36 smithi119 ceph-mon[31518]: pgmap v2954: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:00:36.941 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:00:36 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5095.734274 secs 2024-03-28T02:00:36.941 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:00:36 smithi132 ceph-mon[35941]: pgmap v2954: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:00:38.870 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:00:38 smithi165 ceph-mon[36302]: pgmap v2955: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:00:38.894 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:00:38 smithi119 ceph-mon[31518]: pgmap v2955: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:00:38.941 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:00:38 smithi132 ceph-mon[35941]: pgmap v2955: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:00:40.870 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:00:40 smithi165 ceph-mon[36302]: pgmap v2956: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:00:40.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:00:40 smithi119 ceph-mon[31518]: pgmap v2956: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:00:40.941 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:00:40 smithi132 ceph-mon[35941]: pgmap v2956: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:00:41.870 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:00:41 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5100.734363 secs 2024-03-28T02:00:41.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:00:41 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5100.734363 secs 2024-03-28T02:00:41.941 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:00:41 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5100.734363 secs 2024-03-28T02:00:42.870 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:00:42 smithi165 ceph-mon[36302]: pgmap v2957: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:00:42.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:00:42 smithi119 ceph-mon[31518]: pgmap v2957: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:00:42.940 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:00:42 smithi132 ceph-mon[35941]: pgmap v2957: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:00:44.870 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:00:44 smithi165 ceph-mon[36302]: pgmap v2958: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:00:44.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:00:44 smithi119 ceph-mon[31518]: pgmap v2958: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:00:44.941 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:00:44 smithi132 ceph-mon[35941]: pgmap v2958: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:00:45.870 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:00:45 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5105.734400 secs 2024-03-28T02:00:45.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:00:45 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5105.734400 secs 2024-03-28T02:00:45.941 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:00:45 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5105.734400 secs 2024-03-28T02:00:46.870 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:00:46 smithi165 ceph-mon[36302]: pgmap v2959: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:00:46.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:00:46 smithi119 ceph-mon[31518]: pgmap v2959: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:00:46.940 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:00:46 smithi132 ceph-mon[35941]: pgmap v2959: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:00:48.870 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:00:48 smithi165 ceph-mon[36302]: pgmap v2960: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:00:48.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:00:48 smithi119 ceph-mon[31518]: pgmap v2960: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:00:48.941 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:00:48 smithi132 ceph-mon[35941]: pgmap v2960: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:00:50.871 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:00:50 smithi165 ceph-mon[36302]: pgmap v2961: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:00:50.871 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:00:50 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5110.734463 secs 2024-03-28T02:00:50.894 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:00:50 smithi119 ceph-mon[31518]: pgmap v2961: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:00:50.894 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:00:50 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5110.734463 secs 2024-03-28T02:00:50.940 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:00:50 smithi132 ceph-mon[35941]: pgmap v2961: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:00:50.941 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:00:50 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5110.734463 secs 2024-03-28T02:00:52.870 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:00:52 smithi165 ceph-mon[36302]: pgmap v2962: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:00:52.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:00:52 smithi119 ceph-mon[31518]: pgmap v2962: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:00:52.940 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:00:52 smithi132 ceph-mon[35941]: pgmap v2962: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:00:54.870 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:00:54 smithi165 ceph-mon[36302]: pgmap v2963: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:00:54.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:00:54 smithi119 ceph-mon[31518]: pgmap v2963: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:00:54.941 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:00:54 smithi132 ceph-mon[35941]: pgmap v2963: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:00:55.870 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:00:55 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5115.734549 secs 2024-03-28T02:00:55.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:00:55 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5115.734549 secs 2024-03-28T02:00:55.941 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:00:55 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5115.734549 secs 2024-03-28T02:00:56.870 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:00:56 smithi165 ceph-mon[36302]: pgmap v2964: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:00:56.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:00:56 smithi119 ceph-mon[31518]: pgmap v2964: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:00:56.940 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:00:56 smithi132 ceph-mon[35941]: pgmap v2964: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:00:58.870 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:00:58 smithi165 ceph-mon[36302]: pgmap v2965: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:00:58.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:00:58 smithi119 ceph-mon[31518]: pgmap v2965: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:00:58.940 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:00:58 smithi132 ceph-mon[35941]: pgmap v2965: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:01:00.870 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:01:00 smithi165 ceph-mon[36302]: pgmap v2966: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:01:00.870 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:01:00 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5120.734639 secs 2024-03-28T02:01:00.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:01:00 smithi119 ceph-mon[31518]: pgmap v2966: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:01:00.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:01:00 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5120.734639 secs 2024-03-28T02:01:00.941 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:01:00 smithi132 ceph-mon[35941]: pgmap v2966: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:01:00.941 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:01:00 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5120.734639 secs 2024-03-28T02:01:02.874 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:01:02 smithi165 ceph-mon[36302]: pgmap v2967: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:01:02.894 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:01:02 smithi119 ceph-mon[31518]: pgmap v2967: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:01:02.941 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:01:02 smithi132 ceph-mon[35941]: pgmap v2967: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:01:04.870 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:01:04 smithi165 ceph-mon[36302]: pgmap v2968: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:01:04.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:01:04 smithi119 ceph-mon[31518]: pgmap v2968: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:01:04.941 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:01:04 smithi132 ceph-mon[35941]: pgmap v2968: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:01:05.870 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:01:05 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5125.734728 secs 2024-03-28T02:01:05.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:01:05 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5125.734728 secs 2024-03-28T02:01:05.941 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:01:05 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5125.734728 secs 2024-03-28T02:01:06.870 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:01:06 smithi165 ceph-mon[36302]: pgmap v2969: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:01:06.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:01:06 smithi119 ceph-mon[31518]: pgmap v2969: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:01:06.941 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:01:06 smithi132 ceph-mon[35941]: pgmap v2969: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:01:08.870 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:01:08 smithi165 ceph-mon[36302]: pgmap v2970: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:01:08.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:01:08 smithi119 ceph-mon[31518]: pgmap v2970: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:01:08.941 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:01:08 smithi132 ceph-mon[35941]: pgmap v2970: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:01:10.870 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:01:10 smithi165 ceph-mon[36302]: pgmap v2971: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:01:10.870 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:01:10 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5130.734817 secs 2024-03-28T02:01:10.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:01:10 smithi119 ceph-mon[31518]: pgmap v2971: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:01:10.894 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:01:10 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5130.734817 secs 2024-03-28T02:01:10.940 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:01:10 smithi132 ceph-mon[35941]: pgmap v2971: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:01:10.941 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:01:10 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5130.734817 secs 2024-03-28T02:01:12.870 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:01:12 smithi165 ceph-mon[36302]: pgmap v2972: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:01:12.894 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:01:12 smithi119 ceph-mon[31518]: pgmap v2972: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:01:12.940 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:01:12 smithi132 ceph-mon[35941]: pgmap v2972: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:01:14.870 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:01:14 smithi165 ceph-mon[36302]: pgmap v2973: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:01:14.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:01:14 smithi119 ceph-mon[31518]: pgmap v2973: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:01:14.940 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:01:14 smithi132 ceph-mon[35941]: pgmap v2973: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:01:15.870 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:01:15 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5135.734868 secs 2024-03-28T02:01:15.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:01:15 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5135.734868 secs 2024-03-28T02:01:15.940 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:01:15 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5135.734868 secs 2024-03-28T02:01:16.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:01:16 smithi119 ceph-mon[31518]: pgmap v2974: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:01:16.940 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:01:16 smithi132 ceph-mon[35941]: pgmap v2974: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:01:17.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:01:16 smithi165 ceph-mon[36302]: pgmap v2974: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:01:18.894 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:01:18 smithi119 ceph-mon[31518]: pgmap v2975: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:01:18.940 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:01:18 smithi132 ceph-mon[35941]: pgmap v2975: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:01:19.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:01:18 smithi165 ceph-mon[36302]: pgmap v2975: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:01:20.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:01:20 smithi119 ceph-mon[31518]: pgmap v2976: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:01:20.894 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:01:20 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5140.734929 secs 2024-03-28T02:01:20.940 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:01:20 smithi132 ceph-mon[35941]: pgmap v2976: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:01:20.941 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:01:20 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5140.734929 secs 2024-03-28T02:01:21.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:01:20 smithi165 ceph-mon[36302]: pgmap v2976: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:01:21.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:01:20 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5140.734929 secs 2024-03-28T02:01:22.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:01:22 smithi119 ceph-mon[31518]: pgmap v2977: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:01:22.940 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:01:22 smithi132 ceph-mon[35941]: pgmap v2977: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:01:23.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:01:22 smithi165 ceph-mon[36302]: pgmap v2977: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:01:23.894 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:01:23 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:01:23.941 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:01:23 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:01:24.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:01:23 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:01:24.941 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:01:24 smithi132 ceph-mon[35941]: pgmap v2978: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:01:24.941 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:01:24 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:01:24.941 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:01:24 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:01:25.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:01:24 smithi165 ceph-mon[36302]: pgmap v2978: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:01:25.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:01:24 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:01:25.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:01:24 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:01:25.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:01:24 smithi119 ceph-mon[31518]: pgmap v2978: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:01:25.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:01:24 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:01:25.144 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:01:24 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:01:25.941 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:01:25 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:01:25.941 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:01:25 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:01:25.941 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:01:25 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:01:25.941 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:01:25 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5145.734992 secs 2024-03-28T02:01:26.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:01:25 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:01:26.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:01:25 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:01:26.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:01:25 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:01:26.022 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:01:25 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5145.734992 secs 2024-03-28T02:01:26.045 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:01:25 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:01:26.045 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:01:25 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:01:26.046 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:01:25 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:01:26.046 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:01:25 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5145.734992 secs 2024-03-28T02:01:26.941 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:01:26 smithi132 ceph-mon[35941]: pgmap v2979: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T02:01:26.941 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:01:26 smithi132 ceph-mon[35941]: pgmap v2980: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:01:27.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:01:26 smithi165 ceph-mon[36302]: pgmap v2979: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T02:01:27.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:01:26 smithi165 ceph-mon[36302]: pgmap v2980: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:01:27.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:01:26 smithi119 ceph-mon[31518]: pgmap v2979: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T02:01:27.144 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:01:26 smithi119 ceph-mon[31518]: pgmap v2980: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:01:28.941 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:01:28 smithi132 ceph-mon[35941]: pgmap v2981: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:01:29.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:01:28 smithi165 ceph-mon[36302]: pgmap v2981: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:01:29.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:01:28 smithi119 ceph-mon[31518]: pgmap v2981: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:01:30.941 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:01:30 smithi132 ceph-mon[35941]: pgmap v2982: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-28T02:01:30.941 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:01:30 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5150.735081 secs 2024-03-28T02:01:31.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:01:30 smithi165 ceph-mon[36302]: pgmap v2982: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-28T02:01:31.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:01:30 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5150.735081 secs 2024-03-28T02:01:31.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:01:30 smithi119 ceph-mon[31518]: pgmap v2982: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-28T02:01:31.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:01:30 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5150.735081 secs 2024-03-28T02:01:32.940 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:01:32 smithi132 ceph-mon[35941]: pgmap v2983: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:01:33.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:01:32 smithi165 ceph-mon[36302]: pgmap v2983: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:01:33.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:01:32 smithi119 ceph-mon[31518]: pgmap v2983: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:01:34.940 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:01:34 smithi132 ceph-mon[35941]: pgmap v2984: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:01:35.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:01:34 smithi165 ceph-mon[36302]: pgmap v2984: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:01:35.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:01:34 smithi119 ceph-mon[31518]: pgmap v2984: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:01:35.940 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:01:35 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5155.735170 secs 2024-03-28T02:01:36.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:01:35 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5155.735170 secs 2024-03-28T02:01:36.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:01:35 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5155.735170 secs 2024-03-28T02:01:36.941 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:01:36 smithi132 ceph-mon[35941]: pgmap v2985: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T02:01:37.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:01:36 smithi165 ceph-mon[36302]: pgmap v2985: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T02:01:37.144 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:01:36 smithi119 ceph-mon[31518]: pgmap v2985: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T02:01:39.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:01:38 smithi165 ceph-mon[36302]: pgmap v2986: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:01:39.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:01:38 smithi119 ceph-mon[31518]: pgmap v2986: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:01:39.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:01:38 smithi132 ceph-mon[35941]: pgmap v2986: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:01:41.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:01:40 smithi165 ceph-mon[36302]: pgmap v2987: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:01:41.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:01:40 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5160.735259 secs 2024-03-28T02:01:41.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:01:40 smithi119 ceph-mon[31518]: pgmap v2987: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:01:41.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:01:40 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5160.735259 secs 2024-03-28T02:01:41.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:01:40 smithi132 ceph-mon[35941]: pgmap v2987: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:01:41.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:01:40 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5160.735259 secs 2024-03-28T02:01:43.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:01:42 smithi165 ceph-mon[36302]: pgmap v2988: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:01:43.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:01:42 smithi119 ceph-mon[31518]: pgmap v2988: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:01:43.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:01:42 smithi132 ceph-mon[35941]: pgmap v2988: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:01:45.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:01:44 smithi165 ceph-mon[36302]: pgmap v2989: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:01:45.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:01:44 smithi119 ceph-mon[31518]: pgmap v2989: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:01:45.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:01:44 smithi132 ceph-mon[35941]: pgmap v2989: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:01:46.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:01:45 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5165.735348 secs 2024-03-28T02:01:46.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:01:45 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5165.735348 secs 2024-03-28T02:01:46.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:01:45 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5165.735348 secs 2024-03-28T02:01:47.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:01:46 smithi165 ceph-mon[36302]: pgmap v2990: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:01:47.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:01:46 smithi119 ceph-mon[31518]: pgmap v2990: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:01:47.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:01:46 smithi132 ceph-mon[35941]: pgmap v2990: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:01:49.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:01:48 smithi165 ceph-mon[36302]: pgmap v2991: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:01:49.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:01:48 smithi119 ceph-mon[31518]: pgmap v2991: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:01:49.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:01:48 smithi132 ceph-mon[35941]: pgmap v2991: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:01:51.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:01:50 smithi165 ceph-mon[36302]: pgmap v2992: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:01:51.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:01:50 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5170.735438 secs 2024-03-28T02:01:51.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:01:50 smithi119 ceph-mon[31518]: pgmap v2992: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:01:51.144 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:01:50 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5170.735438 secs 2024-03-28T02:01:51.191 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:01:50 smithi132 ceph-mon[35941]: pgmap v2992: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:01:51.191 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:01:50 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5170.735438 secs 2024-03-28T02:01:53.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:01:52 smithi165 ceph-mon[36302]: pgmap v2993: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:01:53.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:01:52 smithi119 ceph-mon[31518]: pgmap v2993: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:01:53.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:01:52 smithi132 ceph-mon[35941]: pgmap v2993: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:01:55.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:01:54 smithi165 ceph-mon[36302]: pgmap v2994: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:01:55.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:01:54 smithi119 ceph-mon[31518]: pgmap v2994: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:01:55.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:01:54 smithi132 ceph-mon[35941]: pgmap v2994: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:01:56.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:01:55 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5175.735526 secs 2024-03-28T02:01:56.045 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:01:55 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5175.735526 secs 2024-03-28T02:01:56.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:01:55 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5175.735526 secs 2024-03-28T02:01:57.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:01:56 smithi165 ceph-mon[36302]: pgmap v2995: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:01:57.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:01:56 smithi119 ceph-mon[31518]: pgmap v2995: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:01:57.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:01:56 smithi132 ceph-mon[35941]: pgmap v2995: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:01:59.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:01:58 smithi165 ceph-mon[36302]: pgmap v2996: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:01:59.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:01:58 smithi119 ceph-mon[31518]: pgmap v2996: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:01:59.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:01:58 smithi132 ceph-mon[35941]: pgmap v2996: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:02:01.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:02:00 smithi165 ceph-mon[36302]: pgmap v2997: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:02:01.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:02:00 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5180.735616 secs 2024-03-28T02:02:01.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:02:00 smithi119 ceph-mon[31518]: pgmap v2997: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:02:01.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:02:00 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5180.735616 secs 2024-03-28T02:02:01.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:02:00 smithi132 ceph-mon[35941]: pgmap v2997: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:02:01.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:02:00 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5180.735616 secs 2024-03-28T02:02:03.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:02:02 smithi165 ceph-mon[36302]: pgmap v2998: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:02:03.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:02:02 smithi119 ceph-mon[31518]: pgmap v2998: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:02:03.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:02:02 smithi132 ceph-mon[35941]: pgmap v2998: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:02:05.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:02:04 smithi165 ceph-mon[36302]: pgmap v2999: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:02:05.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:02:04 smithi119 ceph-mon[31518]: pgmap v2999: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:02:05.191 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:02:04 smithi132 ceph-mon[35941]: pgmap v2999: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:02:06.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:02:05 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5185.735707 secs 2024-03-28T02:02:06.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:02:05 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5185.735707 secs 2024-03-28T02:02:06.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:02:05 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5185.735707 secs 2024-03-28T02:02:07.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:02:06 smithi165 ceph-mon[36302]: pgmap v3000: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:02:07.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:02:06 smithi119 ceph-mon[31518]: pgmap v3000: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:02:07.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:02:06 smithi132 ceph-mon[35941]: pgmap v3000: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:02:09.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:02:08 smithi165 ceph-mon[36302]: pgmap v3001: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:02:09.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:02:08 smithi119 ceph-mon[31518]: pgmap v3001: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:02:09.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:02:08 smithi132 ceph-mon[35941]: pgmap v3001: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:02:11.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:02:10 smithi165 ceph-mon[36302]: pgmap v3002: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:02:11.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:02:10 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5190.735768 secs 2024-03-28T02:02:11.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:02:10 smithi119 ceph-mon[31518]: pgmap v3002: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:02:11.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:02:10 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5190.735768 secs 2024-03-28T02:02:11.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:02:10 smithi132 ceph-mon[35941]: pgmap v3002: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:02:11.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:02:10 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5190.735768 secs 2024-03-28T02:02:13.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:02:12 smithi165 ceph-mon[36302]: pgmap v3003: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:02:13.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:02:12 smithi119 ceph-mon[31518]: pgmap v3003: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:02:13.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:02:12 smithi132 ceph-mon[35941]: pgmap v3003: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:02:15.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:02:14 smithi165 ceph-mon[36302]: pgmap v3004: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:02:15.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:02:14 smithi119 ceph-mon[31518]: pgmap v3004: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:02:15.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:02:14 smithi132 ceph-mon[35941]: pgmap v3004: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:02:16.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:02:15 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5195.735865 secs 2024-03-28T02:02:16.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:02:15 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5195.735865 secs 2024-03-28T02:02:16.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:02:15 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5195.735865 secs 2024-03-28T02:02:17.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:02:16 smithi165 ceph-mon[36302]: pgmap v3005: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:02:17.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:02:16 smithi119 ceph-mon[31518]: pgmap v3005: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:02:17.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:02:16 smithi132 ceph-mon[35941]: pgmap v3005: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:02:19.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:02:18 smithi165 ceph-mon[36302]: pgmap v3006: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:02:19.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:02:18 smithi119 ceph-mon[31518]: pgmap v3006: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:02:19.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:02:18 smithi132 ceph-mon[35941]: pgmap v3006: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:02:21.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:02:20 smithi165 ceph-mon[36302]: pgmap v3007: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:02:21.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:02:20 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5200.735901 secs 2024-03-28T02:02:21.144 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:02:20 smithi119 ceph-mon[31518]: pgmap v3007: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:02:21.144 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:02:20 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5200.735901 secs 2024-03-28T02:02:21.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:02:20 smithi132 ceph-mon[35941]: pgmap v3007: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:02:21.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:02:20 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5200.735901 secs 2024-03-28T02:02:23.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:02:22 smithi165 ceph-mon[36302]: pgmap v3008: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:02:23.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:02:22 smithi119 ceph-mon[31518]: pgmap v3008: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:02:23.193 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:02:22 smithi132 ceph-mon[35941]: pgmap v3008: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:02:25.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:02:24 smithi165 ceph-mon[36302]: pgmap v3009: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:02:25.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:02:24 smithi119 ceph-mon[31518]: pgmap v3009: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:02:25.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:02:24 smithi132 ceph-mon[35941]: pgmap v3009: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:02:26.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:02:25 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5205.735993 secs 2024-03-28T02:02:26.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:02:25 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:02:26.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:02:25 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5205.735993 secs 2024-03-28T02:02:26.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:02:25 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:02:26.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:02:25 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5205.735993 secs 2024-03-28T02:02:26.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:02:25 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:02:27.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:02:26 smithi165 ceph-mon[36302]: pgmap v3010: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:02:27.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:02:26 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:02:27.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:02:26 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:02:27.121 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:02:26 smithi165 ceph-mon[36302]: pgmap v3011: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T02:02:27.121 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:02:26 smithi165 ceph-mon[36302]: pgmap v3012: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:02:27.121 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:02:26 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:02:27.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:02:26 smithi119 ceph-mon[31518]: pgmap v3010: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:02:27.144 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:02:26 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:02:27.144 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:02:26 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:02:27.144 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:02:26 smithi119 ceph-mon[31518]: pgmap v3011: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T02:02:27.144 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:02:26 smithi119 ceph-mon[31518]: pgmap v3012: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:02:27.144 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:02:26 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:02:27.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:02:26 smithi132 ceph-mon[35941]: pgmap v3010: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:02:27.191 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:02:26 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:02:27.191 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:02:26 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:02:27.191 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:02:26 smithi132 ceph-mon[35941]: pgmap v3011: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T02:02:27.191 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:02:26 smithi132 ceph-mon[35941]: pgmap v3012: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:02:27.191 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:02:26 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:02:29.690 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:02:29 smithi132 ceph-mon[35941]: pgmap v3013: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:02:29.870 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:02:29 smithi165 ceph-mon[36302]: pgmap v3013: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:02:29.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:02:29 smithi119 ceph-mon[31518]: pgmap v3013: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:02:31.690 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:02:31 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5210.736029 secs 2024-03-28T02:02:31.691 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:02:31 smithi132 ceph-mon[35941]: pgmap v3014: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:02:31.870 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:02:31 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5210.736029 secs 2024-03-28T02:02:31.870 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:02:31 smithi165 ceph-mon[36302]: pgmap v3014: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:02:31.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:02:31 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5210.736029 secs 2024-03-28T02:02:31.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:02:31 smithi119 ceph-mon[31518]: pgmap v3014: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:02:33.691 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:02:33 smithi132 ceph-mon[35941]: pgmap v3015: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:02:33.870 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:02:33 smithi165 ceph-mon[36302]: pgmap v3015: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:02:33.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:02:33 smithi119 ceph-mon[31518]: pgmap v3015: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:02:35.690 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:02:35 smithi132 ceph-mon[35941]: pgmap v3016: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:02:35.870 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:02:35 smithi165 ceph-mon[36302]: pgmap v3016: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:02:35.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:02:35 smithi119 ceph-mon[31518]: pgmap v3016: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:02:36.870 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:02:36 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5215.736091 secs 2024-03-28T02:02:36.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:02:36 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5215.736091 secs 2024-03-28T02:02:36.940 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:02:36 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5215.736091 secs 2024-03-28T02:02:37.870 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:02:37 smithi165 ceph-mon[36302]: pgmap v3017: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T02:02:37.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:02:37 smithi119 ceph-mon[31518]: pgmap v3017: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T02:02:37.940 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:02:37 smithi132 ceph-mon[35941]: pgmap v3017: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T02:02:39.870 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:02:39 smithi165 ceph-mon[36302]: pgmap v3018: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:02:39.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:02:39 smithi119 ceph-mon[31518]: pgmap v3018: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:02:39.940 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:02:39 smithi132 ceph-mon[35941]: pgmap v3018: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:02:41.870 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:02:41 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5220.736150 secs 2024-03-28T02:02:41.870 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:02:41 smithi165 ceph-mon[36302]: pgmap v3019: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:02:41.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:02:41 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5220.736150 secs 2024-03-28T02:02:41.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:02:41 smithi119 ceph-mon[31518]: pgmap v3019: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:02:41.940 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:02:41 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5220.736150 secs 2024-03-28T02:02:41.940 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:02:41 smithi132 ceph-mon[35941]: pgmap v3019: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:02:43.870 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:02:43 smithi165 ceph-mon[36302]: pgmap v3020: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:02:43.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:02:43 smithi119 ceph-mon[31518]: pgmap v3020: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:02:43.940 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:02:43 smithi132 ceph-mon[35941]: pgmap v3020: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:02:45.870 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:02:45 smithi165 ceph-mon[36302]: pgmap v3021: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:02:45.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:02:45 smithi119 ceph-mon[31518]: pgmap v3021: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:02:45.940 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:02:45 smithi132 ceph-mon[35941]: pgmap v3021: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:02:46.870 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:02:46 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5225.736213 secs 2024-03-28T02:02:46.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:02:46 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5225.736213 secs 2024-03-28T02:02:46.940 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:02:46 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5225.736213 secs 2024-03-28T02:02:47.870 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:02:47 smithi165 ceph-mon[36302]: pgmap v3022: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:02:47.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:02:47 smithi119 ceph-mon[31518]: pgmap v3022: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:02:47.940 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:02:47 smithi132 ceph-mon[35941]: pgmap v3022: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:02:49.870 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:02:49 smithi165 ceph-mon[36302]: pgmap v3023: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:02:49.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:02:49 smithi119 ceph-mon[31518]: pgmap v3023: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:02:49.940 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:02:49 smithi132 ceph-mon[35941]: pgmap v3023: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:02:51.870 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:02:51 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5230.736274 secs 2024-03-28T02:02:51.870 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:02:51 smithi165 ceph-mon[36302]: pgmap v3024: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:02:51.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:02:51 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5230.736274 secs 2024-03-28T02:02:51.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:02:51 smithi119 ceph-mon[31518]: pgmap v3024: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:02:51.940 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:02:51 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5230.736274 secs 2024-03-28T02:02:51.940 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:02:51 smithi132 ceph-mon[35941]: pgmap v3024: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:02:53.870 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:02:53 smithi165 ceph-mon[36302]: pgmap v3025: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:02:53.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:02:53 smithi119 ceph-mon[31518]: pgmap v3025: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:02:53.940 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:02:53 smithi132 ceph-mon[35941]: pgmap v3025: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:02:55.870 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:02:55 smithi165 ceph-mon[36302]: pgmap v3026: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:02:55.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:02:55 smithi119 ceph-mon[31518]: pgmap v3026: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:02:55.940 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:02:55 smithi132 ceph-mon[35941]: pgmap v3026: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:02:56.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:02:56 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5235.736335 secs 2024-03-28T02:02:56.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:02:56 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5235.736335 secs 2024-03-28T02:02:56.940 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:02:56 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5235.736335 secs 2024-03-28T02:02:57.870 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:02:57 smithi165 ceph-mon[36302]: pgmap v3027: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:02:57.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:02:57 smithi119 ceph-mon[31518]: pgmap v3027: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:02:57.940 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:02:57 smithi132 ceph-mon[35941]: pgmap v3027: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:02:59.870 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:02:59 smithi165 ceph-mon[36302]: pgmap v3028: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:02:59.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:02:59 smithi119 ceph-mon[31518]: pgmap v3028: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:02:59.940 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:02:59 smithi132 ceph-mon[35941]: pgmap v3028: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:03:00.870 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:03:00 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5240.736397 secs 2024-03-28T02:03:00.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:03:00 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5240.736397 secs 2024-03-28T02:03:00.940 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:03:00 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5240.736397 secs 2024-03-28T02:03:01.870 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:03:01 smithi165 ceph-mon[36302]: pgmap v3029: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:03:01.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:03:01 smithi119 ceph-mon[31518]: pgmap v3029: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:03:01.940 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:03:01 smithi132 ceph-mon[35941]: pgmap v3029: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:03:03.870 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:03:03 smithi165 ceph-mon[36302]: pgmap v3030: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:03:03.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:03:03 smithi119 ceph-mon[31518]: pgmap v3030: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:03:03.940 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:03:03 smithi132 ceph-mon[35941]: pgmap v3030: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:03:05.870 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:03:05 smithi165 ceph-mon[36302]: pgmap v3031: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:03:05.870 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:03:05 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5245.736480 secs 2024-03-28T02:03:05.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:03:05 smithi119 ceph-mon[31518]: pgmap v3031: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:03:05.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:03:05 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5245.736480 secs 2024-03-28T02:03:05.940 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:03:05 smithi132 ceph-mon[35941]: pgmap v3031: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:03:05.940 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:03:05 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5245.736480 secs 2024-03-28T02:03:07.870 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:03:07 smithi165 ceph-mon[36302]: pgmap v3032: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:03:07.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:03:07 smithi119 ceph-mon[31518]: pgmap v3032: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:03:07.940 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:03:07 smithi132 ceph-mon[35941]: pgmap v3032: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:03:09.870 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:03:09 smithi165 ceph-mon[36302]: pgmap v3033: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:03:09.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:03:09 smithi119 ceph-mon[31518]: pgmap v3033: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:03:09.940 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:03:09 smithi132 ceph-mon[35941]: pgmap v3033: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:03:10.870 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:03:10 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5250.736571 secs 2024-03-28T02:03:10.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:03:10 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5250.736571 secs 2024-03-28T02:03:10.940 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:03:10 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5250.736571 secs 2024-03-28T02:03:11.870 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:03:11 smithi165 ceph-mon[36302]: pgmap v3034: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:03:11.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:03:11 smithi119 ceph-mon[31518]: pgmap v3034: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:03:11.940 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:03:11 smithi132 ceph-mon[35941]: pgmap v3034: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:03:13.870 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:03:13 smithi165 ceph-mon[36302]: pgmap v3035: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:03:13.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:03:13 smithi119 ceph-mon[31518]: pgmap v3035: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:03:13.940 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:03:13 smithi132 ceph-mon[35941]: pgmap v3035: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:03:15.870 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:03:15 smithi165 ceph-mon[36302]: pgmap v3036: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:03:15.870 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:03:15 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5255.736662 secs 2024-03-28T02:03:15.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:03:15 smithi119 ceph-mon[31518]: pgmap v3036: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:03:15.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:03:15 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5255.736662 secs 2024-03-28T02:03:15.940 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:03:15 smithi132 ceph-mon[35941]: pgmap v3036: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:03:15.940 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:03:15 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5255.736662 secs 2024-03-28T02:03:17.870 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:03:17 smithi165 ceph-mon[36302]: pgmap v3037: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:03:17.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:03:17 smithi119 ceph-mon[31518]: pgmap v3037: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:03:17.940 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:03:17 smithi132 ceph-mon[35941]: pgmap v3037: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:03:19.870 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:03:19 smithi165 ceph-mon[36302]: pgmap v3038: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:03:19.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:03:19 smithi119 ceph-mon[31518]: pgmap v3038: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:03:19.940 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:03:19 smithi132 ceph-mon[35941]: pgmap v3038: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:03:20.870 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:03:20 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5260.736722 secs 2024-03-28T02:03:20.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:03:20 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5260.736722 secs 2024-03-28T02:03:20.940 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:03:20 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5260.736722 secs 2024-03-28T02:03:21.870 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:03:21 smithi165 ceph-mon[36302]: pgmap v3039: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:03:21.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:03:21 smithi119 ceph-mon[31518]: pgmap v3039: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:03:21.940 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:03:21 smithi132 ceph-mon[35941]: pgmap v3039: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:03:23.870 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:03:23 smithi165 ceph-mon[36302]: pgmap v3040: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:03:23.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:03:23 smithi119 ceph-mon[31518]: pgmap v3040: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:03:23.940 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:03:23 smithi132 ceph-mon[35941]: pgmap v3040: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:03:25.870 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:03:25 smithi165 ceph-mon[36302]: pgmap v3041: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:03:25.870 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:03:25 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5265.736762 secs 2024-03-28T02:03:25.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:03:25 smithi119 ceph-mon[31518]: pgmap v3041: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:03:25.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:03:25 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5265.736762 secs 2024-03-28T02:03:25.940 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:03:25 smithi132 ceph-mon[35941]: pgmap v3041: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:03:25.940 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:03:25 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5265.736762 secs 2024-03-28T02:03:27.870 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:03:27 smithi165 ceph-mon[36302]: pgmap v3042: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:03:27.870 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:03:27 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:03:27.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:03:27 smithi119 ceph-mon[31518]: pgmap v3042: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:03:27.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:03:27 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:03:27.940 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:03:27 smithi132 ceph-mon[35941]: pgmap v3042: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:03:27.940 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:03:27 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:03:28.870 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:03:28 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:03:28.870 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:03:28 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:03:28.870 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:03:28 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:03:28.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:03:28 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:03:28.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:03:28 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:03:28.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:03:28 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:03:28.940 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:03:28 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:03:28.940 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:03:28 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:03:28.940 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:03:28 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:03:29.870 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:03:29 smithi165 ceph-mon[36302]: pgmap v3043: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T02:03:29.870 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:03:29 smithi165 ceph-mon[36302]: pgmap v3044: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:03:29.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:03:29 smithi119 ceph-mon[31518]: pgmap v3043: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T02:03:29.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:03:29 smithi119 ceph-mon[31518]: pgmap v3044: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:03:29.940 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:03:29 smithi132 ceph-mon[35941]: pgmap v3043: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T02:03:29.940 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:03:29 smithi132 ceph-mon[35941]: pgmap v3044: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:03:30.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:03:30 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5270.736825 secs 2024-03-28T02:03:30.940 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:03:30 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5270.736825 secs 2024-03-28T02:03:31.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:03:30 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5270.736825 secs 2024-03-28T02:03:31.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:03:31 smithi119 ceph-mon[31518]: pgmap v3045: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:03:31.940 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:03:31 smithi132 ceph-mon[35941]: pgmap v3045: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:03:32.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:03:31 smithi165 ceph-mon[36302]: pgmap v3045: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:03:33.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:03:33 smithi119 ceph-mon[31518]: pgmap v3046: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:03:33.940 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:03:33 smithi132 ceph-mon[35941]: pgmap v3046: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:03:34.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:03:33 smithi165 ceph-mon[36302]: pgmap v3046: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:03:35.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:03:35 smithi119 ceph-mon[31518]: pgmap v3047: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-28T02:03:35.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:03:35 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5275.736887 secs 2024-03-28T02:03:35.940 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:03:35 smithi132 ceph-mon[35941]: pgmap v3047: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-28T02:03:35.940 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:03:35 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5275.736887 secs 2024-03-28T02:03:36.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:03:35 smithi165 ceph-mon[36302]: pgmap v3047: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-28T02:03:36.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:03:35 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5275.736887 secs 2024-03-28T02:03:37.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:03:37 smithi119 ceph-mon[31518]: pgmap v3048: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:03:37.940 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:03:37 smithi132 ceph-mon[35941]: pgmap v3048: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:03:38.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:03:37 smithi165 ceph-mon[36302]: pgmap v3048: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:03:39.940 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:03:39 smithi132 ceph-mon[35941]: pgmap v3049: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T02:03:40.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:03:39 smithi165 ceph-mon[36302]: pgmap v3049: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T02:03:40.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:03:39 smithi119 ceph-mon[31518]: pgmap v3049: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T02:03:40.940 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:03:40 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5280.736950 secs 2024-03-28T02:03:41.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:03:40 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5280.736950 secs 2024-03-28T02:03:41.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:03:40 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5280.736950 secs 2024-03-28T02:03:41.940 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:03:41 smithi132 ceph-mon[35941]: pgmap v3050: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:03:42.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:03:41 smithi165 ceph-mon[36302]: pgmap v3050: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:03:42.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:03:41 smithi119 ceph-mon[31518]: pgmap v3050: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:03:43.940 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:03:43 smithi132 ceph-mon[35941]: pgmap v3051: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:03:44.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:03:43 smithi165 ceph-mon[36302]: pgmap v3051: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:03:44.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:03:43 smithi119 ceph-mon[31518]: pgmap v3051: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:03:45.940 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:03:45 smithi132 ceph-mon[35941]: pgmap v3052: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:03:45.940 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:03:45 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5285.737010 secs 2024-03-28T02:03:46.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:03:45 smithi165 ceph-mon[36302]: pgmap v3052: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:03:46.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:03:45 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5285.737010 secs 2024-03-28T02:03:46.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:03:45 smithi119 ceph-mon[31518]: pgmap v3052: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:03:46.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:03:45 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5285.737010 secs 2024-03-28T02:03:47.940 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:03:47 smithi132 ceph-mon[35941]: pgmap v3053: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:03:48.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:03:47 smithi165 ceph-mon[36302]: pgmap v3053: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:03:48.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:03:47 smithi119 ceph-mon[31518]: pgmap v3053: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:03:49.940 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:03:49 smithi132 ceph-mon[35941]: pgmap v3054: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:03:50.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:03:49 smithi165 ceph-mon[36302]: pgmap v3054: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:03:50.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:03:49 smithi119 ceph-mon[31518]: pgmap v3054: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:03:50.940 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:03:50 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5290.737069 secs 2024-03-28T02:03:51.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:03:50 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5290.737069 secs 2024-03-28T02:03:51.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:03:50 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5290.737069 secs 2024-03-28T02:03:51.940 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:03:51 smithi132 ceph-mon[35941]: pgmap v3055: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:03:52.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:03:51 smithi165 ceph-mon[36302]: pgmap v3055: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:03:52.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:03:51 smithi119 ceph-mon[31518]: pgmap v3055: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:03:54.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:03:53 smithi165 ceph-mon[36302]: pgmap v3056: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:03:54.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:03:53 smithi119 ceph-mon[31518]: pgmap v3056: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:03:54.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:03:53 smithi132 ceph-mon[35941]: pgmap v3056: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:03:56.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:03:55 smithi165 ceph-mon[36302]: pgmap v3057: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:03:56.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:03:55 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5295.737134 secs 2024-03-28T02:03:56.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:03:55 smithi119 ceph-mon[31518]: pgmap v3057: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:03:56.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:03:55 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5295.737134 secs 2024-03-28T02:03:56.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:03:55 smithi132 ceph-mon[35941]: pgmap v3057: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:03:56.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:03:55 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5295.737134 secs 2024-03-28T02:03:58.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:03:57 smithi165 ceph-mon[36302]: pgmap v3058: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:03:58.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:03:57 smithi119 ceph-mon[31518]: pgmap v3058: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:03:58.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:03:57 smithi132 ceph-mon[35941]: pgmap v3058: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:04:00.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:03:59 smithi165 ceph-mon[36302]: pgmap v3059: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:04:00.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:03:59 smithi119 ceph-mon[31518]: pgmap v3059: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:04:00.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:03:59 smithi132 ceph-mon[35941]: pgmap v3059: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:04:01.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:04:00 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5300.737195 secs 2024-03-28T02:04:01.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:04:00 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5300.737195 secs 2024-03-28T02:04:01.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:04:00 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5300.737195 secs 2024-03-28T02:04:02.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:04:01 smithi165 ceph-mon[36302]: pgmap v3060: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:04:02.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:04:01 smithi119 ceph-mon[31518]: pgmap v3060: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:04:02.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:04:01 smithi132 ceph-mon[35941]: pgmap v3060: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:04:04.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:04:03 smithi165 ceph-mon[36302]: pgmap v3061: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:04:04.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:04:03 smithi119 ceph-mon[31518]: pgmap v3061: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:04:04.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:04:03 smithi132 ceph-mon[35941]: pgmap v3061: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:04:05.992 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:04:05 smithi132 ceph-mon[35941]: pgmap v3062: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:04:05.992 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:04:05 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5305.737256 secs 2024-03-28T02:04:06.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:04:05 smithi165 ceph-mon[36302]: pgmap v3062: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:04:06.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:04:05 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5305.737256 secs 2024-03-28T02:04:06.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:04:05 smithi119 ceph-mon[31518]: pgmap v3062: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:04:06.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:04:05 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5305.737256 secs 2024-03-28T02:04:08.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:04:07 smithi165 ceph-mon[36302]: pgmap v3063: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:04:08.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:04:07 smithi119 ceph-mon[31518]: pgmap v3063: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:04:08.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:04:07 smithi132 ceph-mon[35941]: pgmap v3063: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:04:10.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:04:09 smithi165 ceph-mon[36302]: pgmap v3064: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:04:10.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:04:09 smithi119 ceph-mon[31518]: pgmap v3064: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:04:10.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:04:09 smithi132 ceph-mon[35941]: pgmap v3064: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:04:11.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:04:10 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5310.737318 secs 2024-03-28T02:04:11.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:04:10 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5310.737318 secs 2024-03-28T02:04:11.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:04:10 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5310.737318 secs 2024-03-28T02:04:12.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:04:11 smithi165 ceph-mon[36302]: pgmap v3065: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:04:12.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:04:11 smithi119 ceph-mon[31518]: pgmap v3065: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:04:12.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:04:11 smithi132 ceph-mon[35941]: pgmap v3065: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:04:14.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:04:13 smithi165 ceph-mon[36302]: pgmap v3066: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:04:14.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:04:13 smithi119 ceph-mon[31518]: pgmap v3066: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:04:14.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:04:13 smithi132 ceph-mon[35941]: pgmap v3066: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:04:16.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:04:15 smithi165 ceph-mon[36302]: pgmap v3067: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:04:16.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:04:15 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5315.737378 secs 2024-03-28T02:04:16.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:04:15 smithi119 ceph-mon[31518]: pgmap v3067: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:04:16.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:04:15 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5315.737378 secs 2024-03-28T02:04:16.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:04:15 smithi132 ceph-mon[35941]: pgmap v3067: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:04:16.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:04:15 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5315.737378 secs 2024-03-28T02:04:18.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:04:17 smithi165 ceph-mon[36302]: pgmap v3068: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:04:18.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:04:17 smithi119 ceph-mon[31518]: pgmap v3068: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:04:18.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:04:17 smithi132 ceph-mon[35941]: pgmap v3068: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:04:20.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:04:19 smithi165 ceph-mon[36302]: pgmap v3069: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:04:20.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:04:19 smithi119 ceph-mon[31518]: pgmap v3069: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:04:20.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:04:19 smithi132 ceph-mon[35941]: pgmap v3069: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:04:21.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:04:20 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5320.737438 secs 2024-03-28T02:04:21.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:04:20 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5320.737438 secs 2024-03-28T02:04:21.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:04:20 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5320.737438 secs 2024-03-28T02:04:22.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:04:21 smithi165 ceph-mon[36302]: pgmap v3070: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:04:22.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:04:21 smithi119 ceph-mon[31518]: pgmap v3070: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:04:22.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:04:21 smithi132 ceph-mon[35941]: pgmap v3070: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:04:24.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:04:23 smithi165 ceph-mon[36302]: pgmap v3071: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:04:24.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:04:23 smithi119 ceph-mon[31518]: pgmap v3071: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:04:24.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:04:23 smithi132 ceph-mon[35941]: pgmap v3071: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:04:26.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:04:25 smithi165 ceph-mon[36302]: pgmap v3072: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:04:26.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:04:25 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5325.737500 secs 2024-03-28T02:04:26.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:04:25 smithi119 ceph-mon[31518]: pgmap v3072: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:04:26.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:04:25 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5325.737500 secs 2024-03-28T02:04:26.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:04:25 smithi132 ceph-mon[35941]: pgmap v3072: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:04:26.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:04:25 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5325.737500 secs 2024-03-28T02:04:28.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:04:27 smithi165 ceph-mon[36302]: pgmap v3073: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:04:28.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:04:27 smithi119 ceph-mon[31518]: pgmap v3073: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:04:28.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:04:27 smithi132 ceph-mon[35941]: pgmap v3073: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:04:29.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:04:28 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:04:29.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:04:28 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:04:29.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:04:28 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:04:30.105 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:04:29 smithi119 ceph-mon[31518]: pgmap v3074: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:04:30.105 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:04:29 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:04:30.106 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:04:29 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:04:30.106 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:04:29 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:04:30.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:04:29 smithi165 ceph-mon[36302]: pgmap v3074: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:04:30.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:04:29 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:04:30.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:04:29 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:04:30.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:04:29 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:04:30.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:04:29 smithi132 ceph-mon[35941]: pgmap v3074: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:04:30.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:04:29 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:04:30.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:04:29 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:04:30.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:04:29 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:04:31.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:04:30 smithi165 ceph-mon[36302]: pgmap v3075: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T02:04:31.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:04:30 smithi165 ceph-mon[36302]: pgmap v3076: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:04:31.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:04:30 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5330.737562 secs 2024-03-28T02:04:31.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:04:30 smithi119 ceph-mon[31518]: pgmap v3075: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T02:04:31.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:04:30 smithi119 ceph-mon[31518]: pgmap v3076: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:04:31.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:04:30 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5330.737562 secs 2024-03-28T02:04:31.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:04:30 smithi132 ceph-mon[35941]: pgmap v3075: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T02:04:31.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:04:30 smithi132 ceph-mon[35941]: pgmap v3076: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:04:31.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:04:30 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5330.737562 secs 2024-03-28T02:04:33.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:04:32 smithi165 ceph-mon[36302]: pgmap v3077: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:04:33.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:04:32 smithi119 ceph-mon[31518]: pgmap v3077: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:04:33.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:04:32 smithi132 ceph-mon[35941]: pgmap v3077: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:04:35.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:04:34 smithi165 ceph-mon[36302]: pgmap v3078: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:04:35.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:04:34 smithi119 ceph-mon[31518]: pgmap v3078: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:04:35.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:04:34 smithi132 ceph-mon[35941]: pgmap v3078: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:04:36.091 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:04:35 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5335.737623 secs 2024-03-28T02:04:36.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:04:35 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5335.737623 secs 2024-03-28T02:04:36.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:04:35 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5335.737623 secs 2024-03-28T02:04:37.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:04:36 smithi165 ceph-mon[36302]: pgmap v3079: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:04:37.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:04:36 smithi119 ceph-mon[31518]: pgmap v3079: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:04:37.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:04:36 smithi132 ceph-mon[35941]: pgmap v3079: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:04:39.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:04:38 smithi165 ceph-mon[36302]: pgmap v3080: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:04:39.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:04:38 smithi119 ceph-mon[31518]: pgmap v3080: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:04:39.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:04:38 smithi132 ceph-mon[35941]: pgmap v3080: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:04:41.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:04:40 smithi165 ceph-mon[36302]: pgmap v3081: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T02:04:41.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:04:40 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5340.737684 secs 2024-03-28T02:04:41.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:04:40 smithi119 ceph-mon[31518]: pgmap v3081: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T02:04:41.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:04:40 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5340.737684 secs 2024-03-28T02:04:41.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:04:40 smithi132 ceph-mon[35941]: pgmap v3081: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T02:04:41.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:04:40 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5340.737684 secs 2024-03-28T02:04:43.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:04:42 smithi165 ceph-mon[36302]: pgmap v3082: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:04:43.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:04:42 smithi119 ceph-mon[31518]: pgmap v3082: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:04:43.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:04:42 smithi132 ceph-mon[35941]: pgmap v3082: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:04:44.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:04:43 smithi132 ceph-mon[35941]: pgmap v3083: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:04:44.370 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:04:43 smithi165 ceph-mon[36302]: pgmap v3083: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:04:44.393 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:04:43 smithi119 ceph-mon[31518]: pgmap v3083: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:04:46.870 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:04:46 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5345.737746 secs 2024-03-28T02:04:46.870 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:04:46 smithi165 ceph-mon[36302]: pgmap v3084: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:04:46.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:04:46 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5345.737746 secs 2024-03-28T02:04:46.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:04:46 smithi119 ceph-mon[31518]: pgmap v3084: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:04:46.940 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:04:46 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5345.737746 secs 2024-03-28T02:04:46.940 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:04:46 smithi132 ceph-mon[35941]: pgmap v3084: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:04:48.870 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:04:48 smithi165 ceph-mon[36302]: pgmap v3085: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:04:48.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:04:48 smithi119 ceph-mon[31518]: pgmap v3085: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:04:48.940 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:04:48 smithi132 ceph-mon[35941]: pgmap v3085: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:04:50.870 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:04:50 smithi165 ceph-mon[36302]: pgmap v3086: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:04:50.870 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:04:50 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5350.737833 secs 2024-03-28T02:04:50.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:04:50 smithi119 ceph-mon[31518]: pgmap v3086: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:04:50.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:04:50 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5350.737833 secs 2024-03-28T02:04:50.940 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:04:50 smithi132 ceph-mon[35941]: pgmap v3086: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:04:50.940 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:04:50 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5350.737833 secs 2024-03-28T02:04:52.870 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:04:52 smithi165 ceph-mon[36302]: pgmap v3087: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:04:52.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:04:52 smithi119 ceph-mon[31518]: pgmap v3087: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:04:52.940 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:04:52 smithi132 ceph-mon[35941]: pgmap v3087: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:04:54.870 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:04:54 smithi165 ceph-mon[36302]: pgmap v3088: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:04:54.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:04:54 smithi119 ceph-mon[31518]: pgmap v3088: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:04:54.940 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:04:54 smithi132 ceph-mon[35941]: pgmap v3088: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:04:55.870 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:04:55 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5355.737895 secs 2024-03-28T02:04:55.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:04:55 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5355.737895 secs 2024-03-28T02:04:55.940 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:04:55 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5355.737895 secs 2024-03-28T02:04:56.870 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:04:56 smithi165 ceph-mon[36302]: pgmap v3089: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:04:56.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:04:56 smithi119 ceph-mon[31518]: pgmap v3089: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:04:56.940 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:04:56 smithi132 ceph-mon[35941]: pgmap v3089: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:04:58.870 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:04:58 smithi165 ceph-mon[36302]: pgmap v3090: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:04:58.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:04:58 smithi119 ceph-mon[31518]: pgmap v3090: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:04:58.940 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:04:58 smithi132 ceph-mon[35941]: pgmap v3090: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:05:00.870 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:05:00 smithi165 ceph-mon[36302]: pgmap v3091: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:05:00.870 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:05:00 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5360.737957 secs 2024-03-28T02:05:00.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:05:00 smithi119 ceph-mon[31518]: pgmap v3091: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:05:00.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:05:00 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5360.737957 secs 2024-03-28T02:05:00.940 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:05:00 smithi132 ceph-mon[35941]: pgmap v3091: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:05:00.940 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:05:00 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5360.737957 secs 2024-03-28T02:05:02.870 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:05:02 smithi165 ceph-mon[36302]: pgmap v3092: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:05:02.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:05:02 smithi119 ceph-mon[31518]: pgmap v3092: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:05:02.940 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:05:02 smithi132 ceph-mon[35941]: pgmap v3092: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:05:04.370 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:05:04 smithi165 ceph-mon[36302]: pgmap v3093: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:05:04.393 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:05:04 smithi119 ceph-mon[31518]: pgmap v3093: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:05:04.440 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:05:04 smithi132 ceph-mon[35941]: pgmap v3093: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:05:05.870 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:05:05 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5365.738017 secs 2024-03-28T02:05:05.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:05:05 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5365.738017 secs 2024-03-28T02:05:05.940 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:05:05 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5365.738017 secs 2024-03-28T02:05:06.841 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:05:06 smithi132 ceph-mon[35941]: pgmap v3094: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:05:06.870 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:05:06 smithi165 ceph-mon[36302]: pgmap v3094: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:05:06.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:05:06 smithi119 ceph-mon[31518]: pgmap v3094: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:05:08.870 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:05:08 smithi165 ceph-mon[36302]: pgmap v3095: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:05:08.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:05:08 smithi119 ceph-mon[31518]: pgmap v3095: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:05:08.940 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:05:08 smithi132 ceph-mon[35941]: pgmap v3095: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:05:10.870 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:05:10 smithi165 ceph-mon[36302]: pgmap v3096: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:05:10.870 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:05:10 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5370.738077 secs 2024-03-28T02:05:10.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:05:10 smithi119 ceph-mon[31518]: pgmap v3096: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:05:10.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:05:10 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5370.738077 secs 2024-03-28T02:05:10.940 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:05:10 smithi132 ceph-mon[35941]: pgmap v3096: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:05:10.940 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:05:10 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5370.738077 secs 2024-03-28T02:05:12.870 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:05:12 smithi165 ceph-mon[36302]: pgmap v3097: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:05:12.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:05:12 smithi119 ceph-mon[31518]: pgmap v3097: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:05:12.940 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:05:12 smithi132 ceph-mon[35941]: pgmap v3097: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:05:14.870 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:05:14 smithi165 ceph-mon[36302]: pgmap v3098: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:05:14.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:05:14 smithi119 ceph-mon[31518]: pgmap v3098: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:05:14.940 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:05:14 smithi132 ceph-mon[35941]: pgmap v3098: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:05:15.870 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:05:15 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5375.738113 secs 2024-03-28T02:05:15.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:05:15 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5375.738113 secs 2024-03-28T02:05:15.940 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:05:15 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5375.738113 secs 2024-03-28T02:05:16.870 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:05:16 smithi165 ceph-mon[36302]: pgmap v3099: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:05:16.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:05:16 smithi119 ceph-mon[31518]: pgmap v3099: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:05:16.940 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:05:16 smithi132 ceph-mon[35941]: pgmap v3099: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:05:18.870 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:05:18 smithi165 ceph-mon[36302]: pgmap v3100: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T02:05:18.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:05:18 smithi119 ceph-mon[31518]: pgmap v3100: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T02:05:18.940 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:05:18 smithi132 ceph-mon[35941]: pgmap v3100: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T02:05:20.870 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:05:20 smithi165 ceph-mon[36302]: pgmap v3101: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:05:20.870 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:05:20 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5380.738174 secs 2024-03-28T02:05:20.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:05:20 smithi119 ceph-mon[31518]: pgmap v3101: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:05:20.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:05:20 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5380.738174 secs 2024-03-28T02:05:20.940 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:05:20 smithi132 ceph-mon[35941]: pgmap v3101: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:05:20.940 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:05:20 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5380.738174 secs 2024-03-28T02:05:22.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:05:22 smithi119 ceph-mon[31518]: pgmap v3102: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T02:05:22.940 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:05:22 smithi132 ceph-mon[35941]: pgmap v3102: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T02:05:23.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:05:22 smithi165 ceph-mon[36302]: pgmap v3102: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T02:05:24.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:05:24 smithi119 ceph-mon[31518]: pgmap v3103: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T02:05:24.940 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:05:24 smithi132 ceph-mon[35941]: pgmap v3103: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T02:05:25.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:05:24 smithi165 ceph-mon[36302]: pgmap v3103: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T02:05:25.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:05:25 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5385.738235 secs 2024-03-28T02:05:25.940 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:05:25 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5385.738235 secs 2024-03-28T02:05:26.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:05:25 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5385.738235 secs 2024-03-28T02:05:26.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:05:26 smithi119 ceph-mon[31518]: pgmap v3104: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:05:26.940 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:05:26 smithi132 ceph-mon[35941]: pgmap v3104: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:05:27.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:05:26 smithi165 ceph-mon[36302]: pgmap v3104: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:05:28.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:05:28 smithi119 ceph-mon[31518]: pgmap v3105: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:05:28.922 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:05:28 smithi165 ceph-mon[36302]: pgmap v3105: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:05:28.940 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:05:28 smithi132 ceph-mon[35941]: pgmap v3105: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:05:30.940 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:05:30 smithi132 ceph-mon[35941]: pgmap v3106: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:05:30.940 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:05:30 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:05:30.940 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:05:30 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5390.738299 secs 2024-03-28T02:05:31.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:05:30 smithi165 ceph-mon[36302]: pgmap v3106: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:05:31.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:05:30 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:05:31.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:05:30 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5390.738299 secs 2024-03-28T02:05:31.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:05:30 smithi119 ceph-mon[31518]: pgmap v3106: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:05:31.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:05:30 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:05:31.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:05:30 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5390.738299 secs 2024-03-28T02:05:31.940 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:05:31 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:05:31.940 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:05:31 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:05:31.940 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:05:31 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:05:32.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:05:31 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:05:32.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:05:31 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:05:32.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:05:31 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:05:32.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:05:31 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:05:32.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:05:31 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:05:32.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:05:31 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:05:32.940 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:05:32 smithi132 ceph-mon[35941]: pgmap v3107: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T02:05:32.940 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:05:32 smithi132 ceph-mon[35941]: pgmap v3108: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:05:33.081 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:05:32 smithi119 ceph-mon[31518]: pgmap v3107: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T02:05:33.082 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:05:32 smithi119 ceph-mon[31518]: pgmap v3108: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:05:33.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:05:32 smithi165 ceph-mon[36302]: pgmap v3107: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T02:05:33.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:05:32 smithi165 ceph-mon[36302]: pgmap v3108: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:05:34.940 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:05:34 smithi132 ceph-mon[35941]: pgmap v3109: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:05:35.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:05:34 smithi165 ceph-mon[36302]: pgmap v3109: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:05:35.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:05:34 smithi119 ceph-mon[31518]: pgmap v3109: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:05:35.940 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:05:35 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5395.738358 secs 2024-03-28T02:05:36.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:05:35 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5395.738358 secs 2024-03-28T02:05:36.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:05:35 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5395.738358 secs 2024-03-28T02:05:36.940 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:05:36 smithi132 ceph-mon[35941]: pgmap v3110: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:05:37.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:05:36 smithi165 ceph-mon[36302]: pgmap v3110: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:05:37.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:05:36 smithi119 ceph-mon[31518]: pgmap v3110: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:05:38.940 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:05:38 smithi132 ceph-mon[35941]: pgmap v3111: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:05:39.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:05:38 smithi165 ceph-mon[36302]: pgmap v3111: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:05:39.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:05:38 smithi119 ceph-mon[31518]: pgmap v3111: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:05:40.940 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:05:40 smithi132 ceph-mon[35941]: pgmap v3112: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-28T02:05:40.940 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:05:40 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5400.738447 secs 2024-03-28T02:05:41.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:05:40 smithi165 ceph-mon[36302]: pgmap v3112: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-28T02:05:41.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:05:40 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5400.738447 secs 2024-03-28T02:05:41.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:05:40 smithi119 ceph-mon[31518]: pgmap v3112: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-28T02:05:41.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:05:40 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5400.738447 secs 2024-03-28T02:05:42.940 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:05:42 smithi132 ceph-mon[35941]: pgmap v3113: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T02:05:43.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:05:42 smithi165 ceph-mon[36302]: pgmap v3113: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T02:05:43.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:05:42 smithi119 ceph-mon[31518]: pgmap v3113: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T02:05:45.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:05:44 smithi165 ceph-mon[36302]: pgmap v3114: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:05:45.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:05:44 smithi119 ceph-mon[31518]: pgmap v3114: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:05:45.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:05:44 smithi132 ceph-mon[35941]: pgmap v3114: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:05:46.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:05:45 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5405.738537 secs 2024-03-28T02:05:46.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:05:45 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5405.738537 secs 2024-03-28T02:05:46.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:05:45 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5405.738537 secs 2024-03-28T02:05:47.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:05:46 smithi165 ceph-mon[36302]: pgmap v3115: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:05:47.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:05:46 smithi119 ceph-mon[31518]: pgmap v3115: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:05:47.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:05:46 smithi132 ceph-mon[35941]: pgmap v3115: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:05:49.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:05:48 smithi165 ceph-mon[36302]: pgmap v3116: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:05:49.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:05:48 smithi119 ceph-mon[31518]: pgmap v3116: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:05:49.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:05:48 smithi132 ceph-mon[35941]: pgmap v3116: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:05:51.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:05:50 smithi165 ceph-mon[36302]: pgmap v3117: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:05:51.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:05:50 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5410.738597 secs 2024-03-28T02:05:51.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:05:50 smithi119 ceph-mon[31518]: pgmap v3117: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:05:51.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:05:50 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5410.738597 secs 2024-03-28T02:05:51.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:05:50 smithi132 ceph-mon[35941]: pgmap v3117: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:05:51.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:05:50 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5410.738597 secs 2024-03-28T02:05:53.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:05:52 smithi165 ceph-mon[36302]: pgmap v3118: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:05:53.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:05:52 smithi119 ceph-mon[31518]: pgmap v3118: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:05:53.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:05:52 smithi132 ceph-mon[35941]: pgmap v3118: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:05:55.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:05:54 smithi165 ceph-mon[36302]: pgmap v3119: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:05:55.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:05:54 smithi119 ceph-mon[31518]: pgmap v3119: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:05:55.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:05:54 smithi132 ceph-mon[35941]: pgmap v3119: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:05:56.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:05:55 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5415.738656 secs 2024-03-28T02:05:56.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:05:55 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5415.738656 secs 2024-03-28T02:05:56.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:05:55 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5415.738656 secs 2024-03-28T02:05:57.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:05:56 smithi165 ceph-mon[36302]: pgmap v3120: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:05:57.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:05:56 smithi119 ceph-mon[31518]: pgmap v3120: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:05:57.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:05:56 smithi132 ceph-mon[35941]: pgmap v3120: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:05:58.967 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:05:58 smithi119 ceph-mon[31518]: pgmap v3121: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:05:59.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:05:58 smithi165 ceph-mon[36302]: pgmap v3121: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:05:59.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:05:58 smithi132 ceph-mon[35941]: pgmap v3121: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:06:01.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:06:00 smithi165 ceph-mon[36302]: pgmap v3122: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:06:01.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:06:00 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5420.738742 secs 2024-03-28T02:06:01.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:06:00 smithi119 ceph-mon[31518]: pgmap v3122: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:06:01.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:06:00 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5420.738742 secs 2024-03-28T02:06:01.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:06:00 smithi132 ceph-mon[35941]: pgmap v3122: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:06:01.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:06:00 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5420.738742 secs 2024-03-28T02:06:03.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:06:02 smithi165 ceph-mon[36302]: pgmap v3123: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:06:03.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:06:02 smithi119 ceph-mon[31518]: pgmap v3123: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:06:03.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:06:02 smithi132 ceph-mon[35941]: pgmap v3123: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:06:05.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:06:04 smithi165 ceph-mon[36302]: pgmap v3124: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:06:05.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:06:04 smithi119 ceph-mon[31518]: pgmap v3124: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:06:05.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:06:04 smithi132 ceph-mon[35941]: pgmap v3124: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:06:06.091 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:06:05 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5425.738802 secs 2024-03-28T02:06:06.121 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:06:05 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5425.738802 secs 2024-03-28T02:06:06.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:06:05 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5425.738802 secs 2024-03-28T02:06:07.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:06:06 smithi165 ceph-mon[36302]: pgmap v3125: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:06:07.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:06:06 smithi119 ceph-mon[31518]: pgmap v3125: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:06:07.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:06:06 smithi132 ceph-mon[35941]: pgmap v3125: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:06:09.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:06:08 smithi165 ceph-mon[36302]: pgmap v3126: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:06:09.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:06:08 smithi119 ceph-mon[31518]: pgmap v3126: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:06:09.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:06:08 smithi132 ceph-mon[35941]: pgmap v3126: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:06:11.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:06:10 smithi165 ceph-mon[36302]: pgmap v3127: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:06:11.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:06:10 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5430.738870 secs 2024-03-28T02:06:11.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:06:10 smithi119 ceph-mon[31518]: pgmap v3127: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:06:11.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:06:10 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5430.738870 secs 2024-03-28T02:06:11.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:06:10 smithi132 ceph-mon[35941]: pgmap v3127: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:06:11.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:06:10 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5430.738870 secs 2024-03-28T02:06:13.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:06:12 smithi165 ceph-mon[36302]: pgmap v3128: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:06:13.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:06:12 smithi119 ceph-mon[31518]: pgmap v3128: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:06:13.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:06:12 smithi132 ceph-mon[35941]: pgmap v3128: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:06:15.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:06:14 smithi165 ceph-mon[36302]: pgmap v3129: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:06:15.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:06:14 smithi119 ceph-mon[31518]: pgmap v3129: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:06:15.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:06:14 smithi132 ceph-mon[35941]: pgmap v3129: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:06:16.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:06:15 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5435.738959 secs 2024-03-28T02:06:16.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:06:15 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5435.738959 secs 2024-03-28T02:06:16.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:06:15 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5435.738959 secs 2024-03-28T02:06:17.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:06:16 smithi165 ceph-mon[36302]: pgmap v3130: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:06:17.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:06:16 smithi119 ceph-mon[31518]: pgmap v3130: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:06:17.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:06:16 smithi132 ceph-mon[35941]: pgmap v3130: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:06:19.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:06:18 smithi165 ceph-mon[36302]: pgmap v3131: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:06:19.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:06:18 smithi119 ceph-mon[31518]: pgmap v3131: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:06:19.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:06:18 smithi132 ceph-mon[35941]: pgmap v3131: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:06:21.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:06:20 smithi165 ceph-mon[36302]: pgmap v3132: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:06:21.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:06:20 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5440.738976 secs 2024-03-28T02:06:21.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:06:20 smithi119 ceph-mon[31518]: pgmap v3132: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:06:21.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:06:20 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5440.738976 secs 2024-03-28T02:06:21.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:06:20 smithi132 ceph-mon[35941]: pgmap v3132: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:06:21.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:06:20 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5440.738976 secs 2024-03-28T02:06:23.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:06:22 smithi165 ceph-mon[36302]: pgmap v3133: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:06:23.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:06:22 smithi119 ceph-mon[31518]: pgmap v3133: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:06:23.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:06:22 smithi132 ceph-mon[35941]: pgmap v3133: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:06:25.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:06:24 smithi165 ceph-mon[36302]: pgmap v3134: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:06:25.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:06:24 smithi119 ceph-mon[31518]: pgmap v3134: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:06:25.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:06:24 smithi132 ceph-mon[35941]: pgmap v3134: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:06:26.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:06:25 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5445.739037 secs 2024-03-28T02:06:26.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:06:25 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5445.739037 secs 2024-03-28T02:06:26.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:06:25 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5445.739037 secs 2024-03-28T02:06:27.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:06:26 smithi165 ceph-mon[36302]: pgmap v3135: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:06:27.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:06:26 smithi119 ceph-mon[31518]: pgmap v3135: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:06:27.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:06:26 smithi132 ceph-mon[35941]: pgmap v3135: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:06:29.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:06:28 smithi165 ceph-mon[36302]: pgmap v3136: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:06:29.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:06:28 smithi119 ceph-mon[31518]: pgmap v3136: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:06:29.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:06:28 smithi132 ceph-mon[35941]: pgmap v3136: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:06:31.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:06:30 smithi165 ceph-mon[36302]: pgmap v3137: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:06:31.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:06:30 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5450.739122 secs 2024-03-28T02:06:31.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:06:30 smithi119 ceph-mon[31518]: pgmap v3137: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:06:31.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:06:30 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5450.739122 secs 2024-03-28T02:06:31.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:06:30 smithi132 ceph-mon[35941]: pgmap v3137: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:06:31.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:06:30 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5450.739122 secs 2024-03-28T02:06:32.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:06:31 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:06:32.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:06:31 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:06:32.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:06:31 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:06:33.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:06:32 smithi165 ceph-mon[36302]: pgmap v3138: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:06:33.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:06:32 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:06:33.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:06:32 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:06:33.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:06:32 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:06:33.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:06:32 smithi119 ceph-mon[31518]: pgmap v3138: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:06:33.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:06:32 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:06:33.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:06:32 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:06:33.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:06:32 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:06:33.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:06:32 smithi132 ceph-mon[35941]: pgmap v3138: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:06:33.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:06:32 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:06:33.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:06:32 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:06:33.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:06:32 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:06:34.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:06:33 smithi165 ceph-mon[36302]: pgmap v3139: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T02:06:34.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:06:33 smithi165 ceph-mon[36302]: pgmap v3140: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:06:34.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:06:33 smithi119 ceph-mon[31518]: pgmap v3139: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T02:06:34.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:06:33 smithi119 ceph-mon[31518]: pgmap v3140: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:06:34.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:06:33 smithi132 ceph-mon[35941]: pgmap v3139: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T02:06:34.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:06:33 smithi132 ceph-mon[35941]: pgmap v3140: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:06:36.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:06:35 smithi165 ceph-mon[36302]: pgmap v3141: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:06:36.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:06:35 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5455.739236 secs 2024-03-28T02:06:36.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:06:35 smithi119 ceph-mon[31518]: pgmap v3141: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:06:36.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:06:35 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5455.739236 secs 2024-03-28T02:06:36.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:06:35 smithi132 ceph-mon[35941]: pgmap v3141: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:06:36.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:06:35 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5455.739236 secs 2024-03-28T02:06:38.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:06:37 smithi165 ceph-mon[36302]: pgmap v3142: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:06:38.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:06:37 smithi119 ceph-mon[31518]: pgmap v3142: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:06:38.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:06:37 smithi132 ceph-mon[35941]: pgmap v3142: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:06:40.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:06:39 smithi119 ceph-mon[31518]: pgmap v3143: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:06:40.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:06:39 smithi132 ceph-mon[35941]: pgmap v3143: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:06:40.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:06:39 smithi165 ceph-mon[36302]: pgmap v3143: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:06:41.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:06:40 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5460.739324 secs 2024-03-28T02:06:41.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:06:40 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5460.739324 secs 2024-03-28T02:06:41.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:06:40 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5460.739324 secs 2024-03-28T02:06:42.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:06:41 smithi119 ceph-mon[31518]: pgmap v3144: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:06:42.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:06:41 smithi132 ceph-mon[35941]: pgmap v3144: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:06:42.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:06:41 smithi165 ceph-mon[36302]: pgmap v3144: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:06:43.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:06:42 smithi119 ceph-mon[31518]: pgmap v3145: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T02:06:43.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:06:42 smithi132 ceph-mon[35941]: pgmap v3145: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T02:06:43.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:06:42 smithi165 ceph-mon[36302]: pgmap v3145: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T02:06:45.870 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:06:45 smithi165 ceph-mon[36302]: pgmap v3146: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:06:45.870 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:06:45 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5465.739384 secs 2024-03-28T02:06:45.892 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:06:45 smithi119 ceph-mon[31518]: pgmap v3146: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:06:45.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:06:45 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5465.739384 secs 2024-03-28T02:06:45.940 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:06:45 smithi132 ceph-mon[35941]: pgmap v3146: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:06:45.940 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:06:45 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5465.739384 secs 2024-03-28T02:06:47.870 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:06:47 smithi165 ceph-mon[36302]: pgmap v3147: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:06:47.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:06:47 smithi119 ceph-mon[31518]: pgmap v3147: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:06:47.940 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:06:47 smithi132 ceph-mon[35941]: pgmap v3147: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:06:49.870 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:06:49 smithi165 ceph-mon[36302]: pgmap v3148: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:06:49.892 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:06:49 smithi119 ceph-mon[31518]: pgmap v3148: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:06:49.940 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:06:49 smithi132 ceph-mon[35941]: pgmap v3148: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:06:50.869 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:06:50 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5470.739472 secs 2024-03-28T02:06:50.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:06:50 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5470.739472 secs 2024-03-28T02:06:50.940 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:06:50 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5470.739472 secs 2024-03-28T02:06:51.869 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:06:51 smithi165 ceph-mon[36302]: pgmap v3149: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:06:51.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:06:51 smithi119 ceph-mon[31518]: pgmap v3149: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:06:51.940 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:06:51 smithi132 ceph-mon[35941]: pgmap v3149: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:06:53.869 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:06:53 smithi165 ceph-mon[36302]: pgmap v3150: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:06:53.892 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:06:53 smithi119 ceph-mon[31518]: pgmap v3150: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:06:53.940 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:06:53 smithi132 ceph-mon[35941]: pgmap v3150: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:06:55.869 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:06:55 smithi165 ceph-mon[36302]: pgmap v3151: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:06:55.870 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:06:55 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5475.739512 secs 2024-03-28T02:06:55.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:06:55 smithi119 ceph-mon[31518]: pgmap v3151: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:06:55.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:06:55 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5475.739512 secs 2024-03-28T02:06:55.940 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:06:55 smithi132 ceph-mon[35941]: pgmap v3151: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:06:55.940 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:06:55 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5475.739512 secs 2024-03-28T02:06:57.869 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:06:57 smithi165 ceph-mon[36302]: pgmap v3152: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:06:57.892 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:06:57 smithi119 ceph-mon[31518]: pgmap v3152: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:06:57.940 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:06:57 smithi132 ceph-mon[35941]: pgmap v3152: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:06:59.869 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:06:59 smithi165 ceph-mon[36302]: pgmap v3153: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:06:59.892 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:06:59 smithi119 ceph-mon[31518]: pgmap v3153: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:06:59.940 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:06:59 smithi132 ceph-mon[35941]: pgmap v3153: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:07:00.869 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:07:00 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5480.739602 secs 2024-03-28T02:07:00.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:07:00 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5480.739602 secs 2024-03-28T02:07:00.939 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:07:00 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5480.739602 secs 2024-03-28T02:07:01.869 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:07:01 smithi165 ceph-mon[36302]: pgmap v3154: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:07:01.892 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:07:01 smithi119 ceph-mon[31518]: pgmap v3154: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:07:01.939 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:07:01 smithi132 ceph-mon[35941]: pgmap v3154: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:07:03.869 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:07:03 smithi165 ceph-mon[36302]: pgmap v3155: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:07:03.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:07:03 smithi119 ceph-mon[31518]: pgmap v3155: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:07:03.939 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:07:03 smithi132 ceph-mon[35941]: pgmap v3155: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:07:05.869 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:07:05 smithi165 ceph-mon[36302]: pgmap v3156: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:07:05.870 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:07:05 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5485.739664 secs 2024-03-28T02:07:05.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:07:05 smithi119 ceph-mon[31518]: pgmap v3156: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:07:05.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:07:05 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5485.739664 secs 2024-03-28T02:07:05.940 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:07:05 smithi132 ceph-mon[35941]: pgmap v3156: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:07:05.940 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:07:05 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5485.739664 secs 2024-03-28T02:07:07.869 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:07:07 smithi165 ceph-mon[36302]: pgmap v3157: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:07:07.892 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:07:07 smithi119 ceph-mon[31518]: pgmap v3157: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:07:07.940 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:07:07 smithi132 ceph-mon[35941]: pgmap v3157: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:07:09.870 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:07:09 smithi165 ceph-mon[36302]: pgmap v3158: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:07:09.892 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:07:09 smithi119 ceph-mon[31518]: pgmap v3158: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:07:09.940 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:07:09 smithi132 ceph-mon[35941]: pgmap v3158: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:07:10.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:07:10 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5490.739703 secs 2024-03-28T02:07:10.939 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:07:10 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5490.739703 secs 2024-03-28T02:07:11.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:07:10 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5490.739703 secs 2024-03-28T02:07:11.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:07:11 smithi119 ceph-mon[31518]: pgmap v3159: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:07:11.940 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:07:11 smithi132 ceph-mon[35941]: pgmap v3159: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:07:12.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:07:11 smithi165 ceph-mon[36302]: pgmap v3159: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:07:13.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:07:13 smithi119 ceph-mon[31518]: pgmap v3160: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:07:13.939 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:07:13 smithi132 ceph-mon[35941]: pgmap v3160: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:07:14.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:07:13 smithi165 ceph-mon[36302]: pgmap v3160: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:07:15.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:07:15 smithi119 ceph-mon[31518]: pgmap v3161: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:07:15.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:07:15 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5495.739765 secs 2024-03-28T02:07:15.940 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:07:15 smithi132 ceph-mon[35941]: pgmap v3161: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:07:15.940 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:07:15 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5495.739765 secs 2024-03-28T02:07:16.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:07:15 smithi165 ceph-mon[36302]: pgmap v3161: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:07:16.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:07:15 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5495.739765 secs 2024-03-28T02:07:17.892 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:07:17 smithi119 ceph-mon[31518]: pgmap v3162: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:07:17.940 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:07:17 smithi132 ceph-mon[35941]: pgmap v3162: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:07:18.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:07:17 smithi165 ceph-mon[36302]: pgmap v3162: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:07:19.940 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:07:19 smithi132 ceph-mon[35941]: pgmap v3163: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:07:20.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:07:19 smithi165 ceph-mon[36302]: pgmap v3163: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:07:20.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:07:19 smithi119 ceph-mon[31518]: pgmap v3163: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:07:20.940 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:07:20 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5500.739827 secs 2024-03-28T02:07:21.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:07:20 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5500.739827 secs 2024-03-28T02:07:21.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:07:20 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5500.739827 secs 2024-03-28T02:07:21.940 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:07:21 smithi132 ceph-mon[35941]: pgmap v3164: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:07:22.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:07:21 smithi165 ceph-mon[36302]: pgmap v3164: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:07:22.137 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:07:21 smithi119 ceph-mon[31518]: pgmap v3164: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:07:23.940 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:07:23 smithi132 ceph-mon[35941]: pgmap v3165: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:07:24.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:07:23 smithi165 ceph-mon[36302]: pgmap v3165: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:07:24.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:07:23 smithi119 ceph-mon[31518]: pgmap v3165: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:07:25.940 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:07:25 smithi132 ceph-mon[35941]: pgmap v3166: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:07:25.940 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:07:25 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5505.739899 secs 2024-03-28T02:07:26.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:07:25 smithi165 ceph-mon[36302]: pgmap v3166: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:07:26.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:07:25 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5505.739899 secs 2024-03-28T02:07:26.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:07:25 smithi119 ceph-mon[31518]: pgmap v3166: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:07:26.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:07:25 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5505.739899 secs 2024-03-28T02:07:27.940 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:07:27 smithi132 ceph-mon[35941]: pgmap v3167: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:07:28.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:07:27 smithi165 ceph-mon[36302]: pgmap v3167: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:07:28.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:07:27 smithi119 ceph-mon[31518]: pgmap v3167: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:07:29.940 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:07:29 smithi132 ceph-mon[35941]: pgmap v3168: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:07:29.945 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:07:29 smithi119 ceph-mon[31518]: pgmap v3168: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:07:30.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:07:29 smithi165 ceph-mon[36302]: pgmap v3168: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:07:30.940 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:07:30 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5510.739916 secs 2024-03-28T02:07:31.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:07:30 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5510.739916 secs 2024-03-28T02:07:31.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:07:30 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5510.739916 secs 2024-03-28T02:07:31.940 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:07:31 smithi132 ceph-mon[35941]: pgmap v3169: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:07:32.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:07:31 smithi165 ceph-mon[36302]: pgmap v3169: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:07:32.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:07:31 smithi119 ceph-mon[31518]: pgmap v3169: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:07:34.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:07:33 smithi165 ceph-mon[36302]: pgmap v3170: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T02:07:34.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:07:33 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:07:34.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:07:33 smithi119 ceph-mon[31518]: pgmap v3170: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T02:07:34.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:07:33 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:07:34.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:07:33 smithi132 ceph-mon[35941]: pgmap v3170: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T02:07:34.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:07:33 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:07:35.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:07:34 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:07:35.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:07:34 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:07:35.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:07:34 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:07:35.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:07:34 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:07:35.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:07:34 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:07:35.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:07:34 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:07:35.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:07:34 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:07:35.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:07:34 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:07:35.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:07:34 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:07:36.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:07:35 smithi165 ceph-mon[36302]: pgmap v3171: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-28T02:07:36.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:07:35 smithi165 ceph-mon[36302]: pgmap v3172: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 4 op/s 2024-03-28T02:07:36.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:07:35 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5515.739979 secs 2024-03-28T02:07:36.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:07:35 smithi119 ceph-mon[31518]: pgmap v3171: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-28T02:07:36.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:07:35 smithi119 ceph-mon[31518]: pgmap v3172: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 4 op/s 2024-03-28T02:07:36.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:07:35 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5515.739979 secs 2024-03-28T02:07:36.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:07:35 smithi132 ceph-mon[35941]: pgmap v3171: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-28T02:07:36.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:07:35 smithi132 ceph-mon[35941]: pgmap v3172: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 4 op/s 2024-03-28T02:07:36.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:07:35 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5515.739979 secs 2024-03-28T02:07:38.009 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:07:37 smithi119 ceph-mon[31518]: pgmap v3173: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T02:07:38.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:07:37 smithi165 ceph-mon[36302]: pgmap v3173: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T02:07:38.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:07:37 smithi132 ceph-mon[35941]: pgmap v3173: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T02:07:39.992 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:07:39 smithi132 ceph-mon[35941]: pgmap v3174: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T02:07:40.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:07:39 smithi165 ceph-mon[36302]: pgmap v3174: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T02:07:40.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:07:39 smithi119 ceph-mon[31518]: pgmap v3174: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T02:07:41.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:07:40 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5520.740041 secs 2024-03-28T02:07:41.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:07:40 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5520.740041 secs 2024-03-28T02:07:41.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:07:40 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5520.740041 secs 2024-03-28T02:07:42.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:07:41 smithi165 ceph-mon[36302]: pgmap v3175: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:07:42.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:07:41 smithi119 ceph-mon[31518]: pgmap v3175: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:07:42.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:07:41 smithi132 ceph-mon[35941]: pgmap v3175: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:07:44.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:07:43 smithi165 ceph-mon[36302]: pgmap v3176: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:07:44.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:07:43 smithi119 ceph-mon[31518]: pgmap v3176: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:07:44.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:07:43 smithi132 ceph-mon[35941]: pgmap v3176: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:07:46.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:07:45 smithi165 ceph-mon[36302]: pgmap v3177: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-28T02:07:46.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:07:45 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5525.740103 secs 2024-03-28T02:07:46.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:07:45 smithi119 ceph-mon[31518]: pgmap v3177: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-28T02:07:46.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:07:45 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5525.740103 secs 2024-03-28T02:07:46.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:07:45 smithi132 ceph-mon[35941]: pgmap v3177: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-28T02:07:46.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:07:45 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5525.740103 secs 2024-03-28T02:07:48.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:07:47 smithi165 ceph-mon[36302]: pgmap v3178: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:07:48.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:07:47 smithi119 ceph-mon[31518]: pgmap v3178: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:07:48.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:07:47 smithi132 ceph-mon[35941]: pgmap v3178: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:07:50.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:07:49 smithi165 ceph-mon[36302]: pgmap v3179: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T02:07:50.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:07:49 smithi119 ceph-mon[31518]: pgmap v3179: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T02:07:50.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:07:49 smithi132 ceph-mon[35941]: pgmap v3179: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T02:07:51.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:07:50 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5530.740162 secs 2024-03-28T02:07:51.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:07:50 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5530.740162 secs 2024-03-28T02:07:51.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:07:50 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5530.740162 secs 2024-03-28T02:07:52.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:07:51 smithi165 ceph-mon[36302]: pgmap v3180: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:07:52.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:07:51 smithi119 ceph-mon[31518]: pgmap v3180: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:07:52.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:07:51 smithi132 ceph-mon[35941]: pgmap v3180: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:07:54.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:07:53 smithi165 ceph-mon[36302]: pgmap v3181: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T02:07:54.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:07:53 smithi119 ceph-mon[31518]: pgmap v3181: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T02:07:54.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:07:53 smithi132 ceph-mon[35941]: pgmap v3181: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T02:07:56.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:07:55 smithi165 ceph-mon[36302]: pgmap v3182: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-03-28T02:07:56.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:07:55 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5535.740177 secs 2024-03-28T02:07:56.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:07:55 smithi119 ceph-mon[31518]: pgmap v3182: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-03-28T02:07:56.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:07:55 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5535.740177 secs 2024-03-28T02:07:56.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:07:55 smithi132 ceph-mon[35941]: pgmap v3182: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-03-28T02:07:56.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:07:55 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5535.740177 secs 2024-03-28T02:07:58.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:07:57 smithi165 ceph-mon[36302]: pgmap v3183: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T02:07:58.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:07:57 smithi119 ceph-mon[31518]: pgmap v3183: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T02:07:58.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:07:57 smithi132 ceph-mon[35941]: pgmap v3183: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T02:08:00.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:07:59 smithi165 ceph-mon[36302]: pgmap v3184: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:08:00.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:07:59 smithi119 ceph-mon[31518]: pgmap v3184: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:08:00.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:07:59 smithi132 ceph-mon[35941]: pgmap v3184: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:08:01.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:08:00 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5540.740266 secs 2024-03-28T02:08:01.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:08:00 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5540.740266 secs 2024-03-28T02:08:01.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:08:00 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5540.740266 secs 2024-03-28T02:08:02.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:08:01 smithi165 ceph-mon[36302]: pgmap v3185: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:08:02.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:08:01 smithi119 ceph-mon[31518]: pgmap v3185: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:08:02.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:08:01 smithi132 ceph-mon[35941]: pgmap v3185: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:08:04.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:08:03 smithi165 ceph-mon[36302]: pgmap v3186: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:08:04.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:08:03 smithi119 ceph-mon[31518]: pgmap v3186: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:08:04.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:08:03 smithi132 ceph-mon[35941]: pgmap v3186: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:08:06.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:08:05 smithi165 ceph-mon[36302]: pgmap v3187: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-28T02:08:06.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:08:05 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5545.740329 secs 2024-03-28T02:08:06.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:08:05 smithi119 ceph-mon[31518]: pgmap v3187: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-28T02:08:06.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:08:05 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5545.740329 secs 2024-03-28T02:08:06.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:08:05 smithi132 ceph-mon[35941]: pgmap v3187: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-28T02:08:06.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:08:05 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5545.740329 secs 2024-03-28T02:08:08.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:08:07 smithi165 ceph-mon[36302]: pgmap v3188: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T02:08:08.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:08:07 smithi119 ceph-mon[31518]: pgmap v3188: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T02:08:08.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:08:07 smithi132 ceph-mon[35941]: pgmap v3188: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T02:08:10.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:08:09 smithi165 ceph-mon[36302]: pgmap v3189: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:08:10.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:08:09 smithi119 ceph-mon[31518]: pgmap v3189: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:08:10.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:08:09 smithi132 ceph-mon[35941]: pgmap v3189: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:08:11.091 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:08:10 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5550.740417 secs 2024-03-28T02:08:11.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:08:10 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5550.740417 secs 2024-03-28T02:08:11.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:08:10 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5550.740417 secs 2024-03-28T02:08:12.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:08:11 smithi165 ceph-mon[36302]: pgmap v3190: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:08:12.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:08:11 smithi119 ceph-mon[31518]: pgmap v3190: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:08:12.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:08:11 smithi132 ceph-mon[35941]: pgmap v3190: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:08:14.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:08:13 smithi165 ceph-mon[36302]: pgmap v3191: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:08:14.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:08:13 smithi119 ceph-mon[31518]: pgmap v3191: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:08:14.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:08:13 smithi132 ceph-mon[35941]: pgmap v3191: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:08:16.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:08:15 smithi165 ceph-mon[36302]: pgmap v3192: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-28T02:08:16.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:08:15 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5555.740506 secs 2024-03-28T02:08:16.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:08:15 smithi119 ceph-mon[31518]: pgmap v3192: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-28T02:08:16.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:08:15 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5555.740506 secs 2024-03-28T02:08:16.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:08:15 smithi132 ceph-mon[35941]: pgmap v3192: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-28T02:08:16.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:08:15 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5555.740506 secs 2024-03-28T02:08:18.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:08:17 smithi165 ceph-mon[36302]: pgmap v3193: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T02:08:18.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:08:17 smithi119 ceph-mon[31518]: pgmap v3193: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T02:08:18.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:08:17 smithi132 ceph-mon[35941]: pgmap v3193: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T02:08:20.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:08:19 smithi165 ceph-mon[36302]: pgmap v3194: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:08:20.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:08:19 smithi119 ceph-mon[31518]: pgmap v3194: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:08:20.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:08:19 smithi132 ceph-mon[35941]: pgmap v3194: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:08:21.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:08:20 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5560.740568 secs 2024-03-28T02:08:21.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:08:20 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5560.740568 secs 2024-03-28T02:08:21.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:08:20 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5560.740568 secs 2024-03-28T02:08:22.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:08:21 smithi165 ceph-mon[36302]: pgmap v3195: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:08:22.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:08:21 smithi119 ceph-mon[31518]: pgmap v3195: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:08:22.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:08:21 smithi132 ceph-mon[35941]: pgmap v3195: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:08:24.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:08:23 smithi165 ceph-mon[36302]: pgmap v3196: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:08:24.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:08:23 smithi119 ceph-mon[31518]: pgmap v3196: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:08:24.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:08:23 smithi132 ceph-mon[35941]: pgmap v3196: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:08:26.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:08:25 smithi165 ceph-mon[36302]: pgmap v3197: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-28T02:08:26.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:08:25 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5565.740605 secs 2024-03-28T02:08:26.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:08:25 smithi119 ceph-mon[31518]: pgmap v3197: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-28T02:08:26.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:08:25 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5565.740605 secs 2024-03-28T02:08:26.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:08:25 smithi132 ceph-mon[35941]: pgmap v3197: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-28T02:08:26.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:08:25 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5565.740605 secs 2024-03-28T02:08:28.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:08:27 smithi165 ceph-mon[36302]: pgmap v3198: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T02:08:28.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:08:27 smithi119 ceph-mon[31518]: pgmap v3198: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T02:08:28.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:08:27 smithi132 ceph-mon[35941]: pgmap v3198: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T02:08:30.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:08:29 smithi165 ceph-mon[36302]: pgmap v3199: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:08:30.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:08:29 smithi119 ceph-mon[31518]: pgmap v3199: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:08:30.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:08:29 smithi132 ceph-mon[35941]: pgmap v3199: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:08:31.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:08:30 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5570.740667 secs 2024-03-28T02:08:31.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:08:30 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5570.740667 secs 2024-03-28T02:08:31.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:08:30 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5570.740667 secs 2024-03-28T02:08:32.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:08:31 smithi119 ceph-mon[31518]: pgmap v3200: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:08:32.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:08:31 smithi132 ceph-mon[35941]: pgmap v3200: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:08:32.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:08:31 smithi165 ceph-mon[36302]: pgmap v3200: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:08:34.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:08:33 smithi119 ceph-mon[31518]: pgmap v3201: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:08:34.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:08:33 smithi132 ceph-mon[35941]: pgmap v3201: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:08:34.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:08:33 smithi165 ceph-mon[36302]: pgmap v3201: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:08:35.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:08:34 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:08:35.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:08:34 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:08:35.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:08:34 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:08:36.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:08:35 smithi132 ceph-mon[35941]: pgmap v3202: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-28T02:08:36.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:08:35 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5575.740756 secs 2024-03-28T02:08:36.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:08:35 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:08:36.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:08:35 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:08:36.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:08:35 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:08:36.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:08:35 smithi165 ceph-mon[36302]: pgmap v3202: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-28T02:08:36.370 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:08:35 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5575.740756 secs 2024-03-28T02:08:36.370 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:08:35 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:08:36.370 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:08:35 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:08:36.370 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:08:35 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:08:36.393 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:08:35 smithi119 ceph-mon[31518]: pgmap v3202: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-28T02:08:36.393 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:08:35 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5575.740756 secs 2024-03-28T02:08:36.393 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:08:35 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:08:36.393 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:08:35 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:08:36.393 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:08:35 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:08:37.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:08:36 smithi132 ceph-mon[35941]: pgmap v3203: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-03-28T02:08:37.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:08:36 smithi132 ceph-mon[35941]: pgmap v3204: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:08:37.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:08:36 smithi165 ceph-mon[36302]: pgmap v3203: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-03-28T02:08:37.370 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:08:36 smithi165 ceph-mon[36302]: pgmap v3204: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:08:37.392 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:08:36 smithi119 ceph-mon[31518]: pgmap v3203: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-03-28T02:08:37.392 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:08:36 smithi119 ceph-mon[31518]: pgmap v3204: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:08:38.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:08:37 smithi132 ceph-mon[35941]: pgmap v3205: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:08:38.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:08:37 smithi165 ceph-mon[36302]: pgmap v3205: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:08:38.392 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:08:37 smithi119 ceph-mon[31518]: pgmap v3205: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:08:41.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:08:40 smithi165 ceph-mon[36302]: pgmap v3206: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:08:41.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:08:40 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5580.740797 secs 2024-03-28T02:08:41.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:08:40 smithi119 ceph-mon[31518]: pgmap v3206: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:08:41.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:08:40 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5580.740797 secs 2024-03-28T02:08:41.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:08:40 smithi132 ceph-mon[35941]: pgmap v3206: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:08:41.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:08:40 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5580.740797 secs 2024-03-28T02:08:42.992 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:08:42 smithi132 ceph-mon[35941]: pgmap v3207: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:08:43.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:08:42 smithi165 ceph-mon[36302]: pgmap v3207: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:08:43.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:08:42 smithi119 ceph-mon[31518]: pgmap v3207: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:08:45.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:08:44 smithi165 ceph-mon[36302]: pgmap v3208: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T02:08:45.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:08:44 smithi119 ceph-mon[31518]: pgmap v3208: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T02:08:45.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:08:44 smithi132 ceph-mon[35941]: pgmap v3208: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T02:08:46.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:08:45 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5585.740897 secs 2024-03-28T02:08:46.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:08:45 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5585.740897 secs 2024-03-28T02:08:46.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:08:45 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5585.740897 secs 2024-03-28T02:08:47.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:08:46 smithi165 ceph-mon[36302]: pgmap v3209: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T02:08:47.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:08:46 smithi119 ceph-mon[31518]: pgmap v3209: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T02:08:47.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:08:46 smithi132 ceph-mon[35941]: pgmap v3209: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T02:08:49.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:08:48 smithi165 ceph-mon[36302]: pgmap v3210: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:08:49.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:08:48 smithi119 ceph-mon[31518]: pgmap v3210: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:08:49.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:08:48 smithi132 ceph-mon[35941]: pgmap v3210: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:08:51.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:08:50 smithi165 ceph-mon[36302]: pgmap v3211: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:08:51.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:08:50 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5590.740982 secs 2024-03-28T02:08:51.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:08:50 smithi119 ceph-mon[31518]: pgmap v3211: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:08:51.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:08:50 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5590.740982 secs 2024-03-28T02:08:51.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:08:50 smithi132 ceph-mon[35941]: pgmap v3211: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:08:51.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:08:50 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5590.740982 secs 2024-03-28T02:08:53.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:08:52 smithi165 ceph-mon[36302]: pgmap v3212: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:08:53.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:08:52 smithi119 ceph-mon[31518]: pgmap v3212: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:08:53.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:08:52 smithi132 ceph-mon[35941]: pgmap v3212: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:08:55.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:08:54 smithi165 ceph-mon[36302]: pgmap v3213: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:08:55.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:08:54 smithi119 ceph-mon[31518]: pgmap v3213: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:08:55.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:08:54 smithi132 ceph-mon[35941]: pgmap v3213: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:08:56.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:08:55 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5595.741042 secs 2024-03-28T02:08:56.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:08:55 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5595.741042 secs 2024-03-28T02:08:56.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:08:55 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5595.741042 secs 2024-03-28T02:08:57.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:08:56 smithi165 ceph-mon[36302]: pgmap v3214: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:08:57.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:08:56 smithi119 ceph-mon[31518]: pgmap v3214: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:08:57.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:08:56 smithi132 ceph-mon[35941]: pgmap v3214: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:08:59.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:08:58 smithi165 ceph-mon[36302]: pgmap v3215: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:08:59.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:08:58 smithi119 ceph-mon[31518]: pgmap v3215: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:08:59.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:08:58 smithi132 ceph-mon[35941]: pgmap v3215: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:09:01.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:09:00 smithi165 ceph-mon[36302]: pgmap v3216: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:09:01.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:09:00 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5600.741058 secs 2024-03-28T02:09:01.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:09:00 smithi119 ceph-mon[31518]: pgmap v3216: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:09:01.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:09:00 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5600.741058 secs 2024-03-28T02:09:01.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:09:00 smithi132 ceph-mon[35941]: pgmap v3216: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:09:01.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:09:00 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5600.741058 secs 2024-03-28T02:09:03.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:09:02 smithi165 ceph-mon[36302]: pgmap v3217: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:09:03.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:09:02 smithi119 ceph-mon[31518]: pgmap v3217: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:09:03.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:09:02 smithi132 ceph-mon[35941]: pgmap v3217: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:09:05.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:09:04 smithi165 ceph-mon[36302]: pgmap v3218: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:09:05.146 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:09:04 smithi119 ceph-mon[31518]: pgmap v3218: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:09:05.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:09:04 smithi132 ceph-mon[35941]: pgmap v3218: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:09:06.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:09:05 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5605.741121 secs 2024-03-28T02:09:06.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:09:05 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5605.741121 secs 2024-03-28T02:09:06.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:09:05 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5605.741121 secs 2024-03-28T02:09:07.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:09:06 smithi165 ceph-mon[36302]: pgmap v3219: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:09:07.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:09:06 smithi119 ceph-mon[31518]: pgmap v3219: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:09:07.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:09:06 smithi132 ceph-mon[35941]: pgmap v3219: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:09:09.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:09:08 smithi165 ceph-mon[36302]: pgmap v3220: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:09:09.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:09:08 smithi119 ceph-mon[31518]: pgmap v3220: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:09:09.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:09:08 smithi132 ceph-mon[35941]: pgmap v3220: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:09:11.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:09:10 smithi165 ceph-mon[36302]: pgmap v3221: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:09:11.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:09:10 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5610.741184 secs 2024-03-28T02:09:11.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:09:10 smithi119 ceph-mon[31518]: pgmap v3221: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:09:11.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:09:10 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5610.741184 secs 2024-03-28T02:09:11.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:09:10 smithi132 ceph-mon[35941]: pgmap v3221: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:09:11.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:09:10 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5610.741184 secs 2024-03-28T02:09:13.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:09:12 smithi165 ceph-mon[36302]: pgmap v3222: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:09:13.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:09:12 smithi119 ceph-mon[31518]: pgmap v3222: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:09:13.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:09:12 smithi132 ceph-mon[35941]: pgmap v3222: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:09:15.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:09:14 smithi165 ceph-mon[36302]: pgmap v3223: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:09:15.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:09:14 smithi119 ceph-mon[31518]: pgmap v3223: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:09:15.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:09:14 smithi132 ceph-mon[35941]: pgmap v3223: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:09:16.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:09:15 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5615.741272 secs 2024-03-28T02:09:16.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:09:15 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5615.741272 secs 2024-03-28T02:09:16.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:09:15 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5615.741272 secs 2024-03-28T02:09:17.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:09:16 smithi165 ceph-mon[36302]: pgmap v3224: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:09:17.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:09:16 smithi119 ceph-mon[31518]: pgmap v3224: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:09:17.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:09:16 smithi132 ceph-mon[35941]: pgmap v3224: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:09:19.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:09:18 smithi165 ceph-mon[36302]: pgmap v3225: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:09:19.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:09:18 smithi119 ceph-mon[31518]: pgmap v3225: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:09:19.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:09:18 smithi132 ceph-mon[35941]: pgmap v3225: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:09:21.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:09:20 smithi165 ceph-mon[36302]: pgmap v3226: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:09:21.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:09:20 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5620.741361 secs 2024-03-28T02:09:21.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:09:20 smithi119 ceph-mon[31518]: pgmap v3226: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:09:21.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:09:20 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5620.741361 secs 2024-03-28T02:09:21.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:09:20 smithi132 ceph-mon[35941]: pgmap v3226: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:09:21.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:09:20 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5620.741361 secs 2024-03-28T02:09:23.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:09:22 smithi165 ceph-mon[36302]: pgmap v3227: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:09:23.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:09:22 smithi119 ceph-mon[31518]: pgmap v3227: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:09:23.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:09:22 smithi132 ceph-mon[35941]: pgmap v3227: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:09:25.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:09:24 smithi165 ceph-mon[36302]: pgmap v3228: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:09:25.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:09:24 smithi119 ceph-mon[31518]: pgmap v3228: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:09:25.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:09:24 smithi132 ceph-mon[35941]: pgmap v3228: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:09:26.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:09:25 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5625.741452 secs 2024-03-28T02:09:26.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:09:25 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5625.741452 secs 2024-03-28T02:09:26.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:09:25 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5625.741452 secs 2024-03-28T02:09:27.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:09:26 smithi165 ceph-mon[36302]: pgmap v3229: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:09:27.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:09:26 smithi119 ceph-mon[31518]: pgmap v3229: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:09:27.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:09:26 smithi132 ceph-mon[35941]: pgmap v3229: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:09:29.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:09:28 smithi165 ceph-mon[36302]: pgmap v3230: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:09:29.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:09:28 smithi119 ceph-mon[31518]: pgmap v3230: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:09:29.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:09:28 smithi132 ceph-mon[35941]: pgmap v3230: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:09:31.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:09:30 smithi165 ceph-mon[36302]: pgmap v3231: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:09:31.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:09:30 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5630.741538 secs 2024-03-28T02:09:31.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:09:30 smithi119 ceph-mon[31518]: pgmap v3231: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:09:31.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:09:30 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5630.741538 secs 2024-03-28T02:09:31.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:09:30 smithi132 ceph-mon[35941]: pgmap v3231: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:09:31.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:09:30 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5630.741538 secs 2024-03-28T02:09:33.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:09:32 smithi119 ceph-mon[31518]: pgmap v3232: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:09:33.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:09:32 smithi132 ceph-mon[35941]: pgmap v3232: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:09:33.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:09:32 smithi165 ceph-mon[36302]: pgmap v3232: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:09:35.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:09:34 smithi119 ceph-mon[31518]: pgmap v3233: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:09:35.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:09:34 smithi132 ceph-mon[35941]: pgmap v3233: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:09:35.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:09:34 smithi165 ceph-mon[36302]: pgmap v3233: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:09:36.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:09:35 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5635.741627 secs 2024-03-28T02:09:36.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:09:35 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5635.741627 secs 2024-03-28T02:09:36.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:09:35 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5635.741627 secs 2024-03-28T02:09:37.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:09:36 smithi119 ceph-mon[31518]: pgmap v3234: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:09:37.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:09:36 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:09:37.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:09:36 smithi132 ceph-mon[35941]: pgmap v3234: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:09:37.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:09:36 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:09:37.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:09:36 smithi165 ceph-mon[36302]: pgmap v3234: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:09:37.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:09:36 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:09:38.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:09:37 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:09:38.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:09:37 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:09:38.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:09:37 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:09:38.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:09:37 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:09:38.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:09:37 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:09:38.370 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:09:37 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:09:38.392 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:09:37 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:09:38.393 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:09:37 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:09:38.393 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:09:37 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:09:39.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:09:38 smithi132 ceph-mon[35941]: pgmap v3235: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T02:09:39.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:09:38 smithi132 ceph-mon[35941]: pgmap v3236: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:09:39.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:09:38 smithi165 ceph-mon[36302]: pgmap v3235: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T02:09:39.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:09:38 smithi165 ceph-mon[36302]: pgmap v3236: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:09:39.392 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:09:38 smithi119 ceph-mon[31518]: pgmap v3235: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T02:09:39.392 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:09:38 smithi119 ceph-mon[31518]: pgmap v3236: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:09:41.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:09:40 smithi132 ceph-mon[35941]: pgmap v3237: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:09:41.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:09:40 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5640.741717 secs 2024-03-28T02:09:41.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:09:40 smithi165 ceph-mon[36302]: pgmap v3237: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:09:41.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:09:40 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5640.741717 secs 2024-03-28T02:09:41.392 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:09:40 smithi119 ceph-mon[31518]: pgmap v3237: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:09:41.392 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:09:40 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5640.741717 secs 2024-03-28T02:09:43.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:09:42 smithi132 ceph-mon[35941]: pgmap v3238: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:09:43.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:09:42 smithi165 ceph-mon[36302]: pgmap v3238: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:09:43.392 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:09:42 smithi119 ceph-mon[31518]: pgmap v3238: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:09:45.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:09:44 smithi132 ceph-mon[35941]: pgmap v3239: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:09:45.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:09:44 smithi165 ceph-mon[36302]: pgmap v3239: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:09:45.392 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:09:44 smithi119 ceph-mon[31518]: pgmap v3239: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:09:46.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:09:45 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5645.741807 secs 2024-03-28T02:09:46.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:09:45 smithi132 ceph-mon[35941]: pgmap v3240: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:09:46.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:09:45 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5645.741807 secs 2024-03-28T02:09:46.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:09:45 smithi165 ceph-mon[36302]: pgmap v3240: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:09:46.392 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:09:45 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5645.741807 secs 2024-03-28T02:09:46.392 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:09:45 smithi119 ceph-mon[31518]: pgmap v3240: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:09:48.619 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:09:48 smithi165 ceph-mon[36302]: pgmap v3241: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T02:09:48.642 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:09:48 smithi119 ceph-mon[31518]: pgmap v3241: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T02:09:48.689 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:09:48 smithi132 ceph-mon[35941]: pgmap v3241: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T02:09:50.619 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:09:50 smithi165 ceph-mon[36302]: pgmap v3242: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:09:50.642 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:09:50 smithi119 ceph-mon[31518]: pgmap v3242: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:09:50.689 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:09:50 smithi132 ceph-mon[35941]: pgmap v3242: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:09:51.619 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:09:51 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5650.741898 secs 2024-03-28T02:09:51.642 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:09:51 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5650.741898 secs 2024-03-28T02:09:51.689 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:09:51 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5650.741898 secs 2024-03-28T02:09:52.619 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:09:52 smithi165 ceph-mon[36302]: pgmap v3243: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:09:52.642 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:09:52 smithi119 ceph-mon[31518]: pgmap v3243: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:09:52.689 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:09:52 smithi132 ceph-mon[35941]: pgmap v3243: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:09:54.619 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:09:54 smithi165 ceph-mon[36302]: pgmap v3244: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:09:54.642 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:09:54 smithi119 ceph-mon[31518]: pgmap v3244: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:09:54.689 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:09:54 smithi132 ceph-mon[35941]: pgmap v3244: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:09:56.619 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:09:56 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5655.741914 secs 2024-03-28T02:09:56.619 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:09:56 smithi165 ceph-mon[36302]: pgmap v3245: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:09:56.642 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:09:56 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5655.741914 secs 2024-03-28T02:09:56.642 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:09:56 smithi119 ceph-mon[31518]: pgmap v3245: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:09:56.689 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:09:56 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5655.741914 secs 2024-03-28T02:09:56.689 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:09:56 smithi132 ceph-mon[35941]: pgmap v3245: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:09:58.619 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:09:58 smithi165 ceph-mon[36302]: pgmap v3246: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:09:58.642 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:09:58 smithi119 ceph-mon[31518]: pgmap v3246: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:09:58.689 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:09:58 smithi132 ceph-mon[35941]: pgmap v3246: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:10:00.619 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:10:00 smithi165 ceph-mon[36302]: pgmap v3247: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:10:00.619 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:10:00 smithi165 ceph-mon[36302]: overall HEALTH_WARN 5 failed cephadm daemon(s); 1 filesystem is degraded; 1 MDSs report slow requests 2024-03-28T02:10:00.642 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:10:00 smithi119 ceph-mon[31518]: pgmap v3247: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:10:00.642 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:10:00 smithi119 ceph-mon[31518]: overall HEALTH_WARN 5 failed cephadm daemon(s); 1 filesystem is degraded; 1 MDSs report slow requests 2024-03-28T02:10:00.689 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:10:00 smithi132 ceph-mon[35941]: pgmap v3247: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:10:00.689 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:10:00 smithi132 ceph-mon[35941]: overall HEALTH_WARN 5 failed cephadm daemon(s); 1 filesystem is degraded; 1 MDSs report slow requests 2024-03-28T02:10:01.619 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:10:01 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5660.741999 secs 2024-03-28T02:10:01.642 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:10:01 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5660.741999 secs 2024-03-28T02:10:01.689 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:10:01 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5660.741999 secs 2024-03-28T02:10:02.619 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:10:02 smithi165 ceph-mon[36302]: pgmap v3248: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:10:02.642 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:10:02 smithi119 ceph-mon[31518]: pgmap v3248: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:10:02.689 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:10:02 smithi132 ceph-mon[35941]: pgmap v3248: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:10:04.619 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:10:04 smithi165 ceph-mon[36302]: pgmap v3249: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:10:04.642 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:10:04 smithi119 ceph-mon[31518]: pgmap v3249: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:10:04.689 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:10:04 smithi132 ceph-mon[35941]: pgmap v3249: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:10:06.619 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:10:06 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5665.742059 secs 2024-03-28T02:10:06.619 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:10:06 smithi165 ceph-mon[36302]: pgmap v3250: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:10:06.642 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:10:06 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5665.742059 secs 2024-03-28T02:10:06.642 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:10:06 smithi119 ceph-mon[31518]: pgmap v3250: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:10:06.689 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:10:06 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5665.742059 secs 2024-03-28T02:10:06.689 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:10:06 smithi132 ceph-mon[35941]: pgmap v3250: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:10:08.619 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:10:08 smithi165 ceph-mon[36302]: pgmap v3251: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:10:08.642 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:10:08 smithi119 ceph-mon[31518]: pgmap v3251: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:10:08.689 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:10:08 smithi132 ceph-mon[35941]: pgmap v3251: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:10:10.619 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:10:10 smithi165 ceph-mon[36302]: pgmap v3252: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:10:10.642 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:10:10 smithi119 ceph-mon[31518]: pgmap v3252: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:10:10.689 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:10:10 smithi132 ceph-mon[35941]: pgmap v3252: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:10:11.619 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:10:11 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5670.742146 secs 2024-03-28T02:10:11.642 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:10:11 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5670.742146 secs 2024-03-28T02:10:11.689 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:10:11 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5670.742146 secs 2024-03-28T02:10:12.619 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:10:12 smithi165 ceph-mon[36302]: pgmap v3253: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:10:12.642 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:10:12 smithi119 ceph-mon[31518]: pgmap v3253: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:10:12.689 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:10:12 smithi132 ceph-mon[35941]: pgmap v3253: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:10:14.619 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:10:14 smithi165 ceph-mon[36302]: pgmap v3254: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:10:14.642 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:10:14 smithi119 ceph-mon[31518]: pgmap v3254: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:10:14.689 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:10:14 smithi132 ceph-mon[35941]: pgmap v3254: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:10:16.619 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:10:16 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5675.742206 secs 2024-03-28T02:10:16.619 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:10:16 smithi165 ceph-mon[36302]: pgmap v3255: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:10:16.642 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:10:16 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5675.742206 secs 2024-03-28T02:10:16.642 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:10:16 smithi119 ceph-mon[31518]: pgmap v3255: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:10:16.689 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:10:16 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5675.742206 secs 2024-03-28T02:10:16.689 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:10:16 smithi132 ceph-mon[35941]: pgmap v3255: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:10:18.619 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:10:18 smithi165 ceph-mon[36302]: pgmap v3256: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:10:18.642 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:10:18 smithi119 ceph-mon[31518]: pgmap v3256: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:10:18.689 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:10:18 smithi132 ceph-mon[35941]: pgmap v3256: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:10:20.619 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:10:20 smithi165 ceph-mon[36302]: pgmap v3257: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:10:20.642 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:10:20 smithi119 ceph-mon[31518]: pgmap v3257: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:10:20.689 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:10:20 smithi132 ceph-mon[35941]: pgmap v3257: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:10:21.619 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:10:21 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5680.742300 secs 2024-03-28T02:10:21.642 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:10:21 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5680.742300 secs 2024-03-28T02:10:21.689 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:10:21 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5680.742300 secs 2024-03-28T02:10:22.619 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:10:22 smithi165 ceph-mon[36302]: pgmap v3258: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:10:22.642 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:10:22 smithi119 ceph-mon[31518]: pgmap v3258: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:10:22.689 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:10:22 smithi132 ceph-mon[35941]: pgmap v3258: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:10:24.619 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:10:24 smithi165 ceph-mon[36302]: pgmap v3259: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:10:24.642 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:10:24 smithi119 ceph-mon[31518]: pgmap v3259: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:10:24.689 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:10:24 smithi132 ceph-mon[35941]: pgmap v3259: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:10:26.619 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:10:26 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5685.742386 secs 2024-03-28T02:10:26.619 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:10:26 smithi165 ceph-mon[36302]: pgmap v3260: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:10:26.642 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:10:26 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5685.742386 secs 2024-03-28T02:10:26.642 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:10:26 smithi119 ceph-mon[31518]: pgmap v3260: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:10:26.689 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:10:26 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5685.742386 secs 2024-03-28T02:10:26.689 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:10:26 smithi132 ceph-mon[35941]: pgmap v3260: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:10:28.619 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:10:28 smithi165 ceph-mon[36302]: pgmap v3261: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:10:28.642 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:10:28 smithi119 ceph-mon[31518]: pgmap v3261: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:10:28.689 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:10:28 smithi132 ceph-mon[35941]: pgmap v3261: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:10:30.619 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:10:30 smithi165 ceph-mon[36302]: pgmap v3262: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:10:30.642 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:10:30 smithi119 ceph-mon[31518]: pgmap v3262: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:10:30.689 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:10:30 smithi132 ceph-mon[35941]: pgmap v3262: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:10:31.619 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:10:31 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5690.742468 secs 2024-03-28T02:10:31.642 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:10:31 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5690.742468 secs 2024-03-28T02:10:31.689 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:10:31 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5690.742468 secs 2024-03-28T02:10:32.619 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:10:32 smithi165 ceph-mon[36302]: pgmap v3263: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:10:32.642 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:10:32 smithi119 ceph-mon[31518]: pgmap v3263: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:10:32.689 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:10:32 smithi132 ceph-mon[35941]: pgmap v3263: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:10:34.642 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:10:34 smithi119 ceph-mon[31518]: pgmap v3264: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:10:34.689 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:10:34 smithi132 ceph-mon[35941]: pgmap v3264: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:10:34.869 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:10:34 smithi165 ceph-mon[36302]: pgmap v3264: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:10:36.642 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:10:36 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5695.742529 secs 2024-03-28T02:10:36.642 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:10:36 smithi119 ceph-mon[31518]: pgmap v3265: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:10:36.689 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:10:36 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5695.742529 secs 2024-03-28T02:10:36.689 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:10:36 smithi132 ceph-mon[35941]: pgmap v3265: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:10:36.869 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:10:36 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5695.742529 secs 2024-03-28T02:10:36.869 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:10:36 smithi165 ceph-mon[36302]: pgmap v3265: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:10:38.619 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:10:38 smithi165 ceph-mon[36302]: pgmap v3266: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:10:38.619 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:10:38 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:10:38.619 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:10:38 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:10:38.619 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:10:38 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:10:38.619 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:10:38 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:10:38.620 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:10:38 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:10:38.642 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:10:38 smithi119 ceph-mon[31518]: pgmap v3266: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:10:38.642 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:10:38 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:10:38.642 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:10:38 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:10:38.642 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:10:38 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:10:38.642 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:10:38 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:10:38.642 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:10:38 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:10:38.689 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:10:38 smithi132 ceph-mon[35941]: pgmap v3266: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:10:38.689 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:10:38 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:10:38.689 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:10:38 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:10:38.689 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:10:38 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:10:38.690 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:10:38 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:10:38.690 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:10:38 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:10:41.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:10:40 smithi165 ceph-mon[36302]: pgmap v3267: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:10:41.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:10:40 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:10:41.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:10:40 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:10:41.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:10:40 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:10:41.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:10:40 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:10:41.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:10:40 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5700.742623 secs 2024-03-28T02:10:41.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:10:40 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:10:41.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:10:40 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:10:41.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:10:40 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:10:41.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:10:40 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:10:41.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:10:40 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:10:41.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:10:40 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:10:41.121 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:10:40 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:10:41.121 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:10:40 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:10:41.121 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:10:40 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:10:41.121 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:10:40 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:10:41.121 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:10:40 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:10:41.121 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:10:40 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:10:41.121 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:10:40 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:10:41.121 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:10:40 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:10:41.121 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:10:40 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:10:41.121 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:10:40 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:10:41.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:10:40 smithi119 ceph-mon[31518]: pgmap v3267: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:10:41.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:10:40 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:10:41.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:10:40 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:10:41.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:10:40 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:10:41.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:10:40 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:10:41.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:10:40 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5700.742623 secs 2024-03-28T02:10:41.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:10:40 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:10:41.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:10:40 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:10:41.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:10:40 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:10:41.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:10:40 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:10:41.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:10:40 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:10:41.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:10:40 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:10:41.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:10:40 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:10:41.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:10:40 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:10:41.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:10:40 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:10:41.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:10:40 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:10:41.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:10:40 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:10:41.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:10:40 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:10:41.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:10:40 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:10:41.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:10:40 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:10:41.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:10:40 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:10:41.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:10:40 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:10:41.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:10:40 smithi132 ceph-mon[35941]: pgmap v3267: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:10:41.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:10:40 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:10:41.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:10:40 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:10:41.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:10:40 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:10:41.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:10:40 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:10:41.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:10:40 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5700.742623 secs 2024-03-28T02:10:41.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:10:40 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:10:41.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:10:40 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:10:41.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:10:40 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:10:41.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:10:40 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:10:41.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:10:40 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:10:41.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:10:40 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:10:41.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:10:40 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:10:41.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:10:40 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:10:41.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:10:40 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:10:41.191 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:10:40 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:10:41.191 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:10:40 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:10:41.191 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:10:40 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:10:41.191 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:10:40 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:10:41.191 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:10:40 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:10:41.191 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:10:40 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:10:41.191 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:10:40 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:10:42.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:10:41 smithi165 ceph-mon[36302]: Adjusting osd_memory_target on smithi132 to 174.1M 2024-03-28T02:10:42.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:10:41 smithi165 ceph-mon[36302]: Unable to set osd_memory_target on smithi132 to 182640640: error parsing value: Value '182640640' is below minimum 939524096 2024-03-28T02:10:42.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:10:41 smithi165 ceph-mon[36302]: Adjusting osd_memory_target on smithi165 to 174.1M 2024-03-28T02:10:42.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:10:41 smithi165 ceph-mon[36302]: Unable to set osd_memory_target on smithi165 to 182635622: error parsing value: Value '182635622' is below minimum 939524096 2024-03-28T02:10:42.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:10:41 smithi119 ceph-mon[31518]: Adjusting osd_memory_target on smithi132 to 174.1M 2024-03-28T02:10:42.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:10:41 smithi119 ceph-mon[31518]: Unable to set osd_memory_target on smithi132 to 182640640: error parsing value: Value '182640640' is below minimum 939524096 2024-03-28T02:10:42.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:10:41 smithi119 ceph-mon[31518]: Adjusting osd_memory_target on smithi165 to 174.1M 2024-03-28T02:10:42.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:10:41 smithi119 ceph-mon[31518]: Unable to set osd_memory_target on smithi165 to 182635622: error parsing value: Value '182635622' is below minimum 939524096 2024-03-28T02:10:42.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:10:41 smithi132 ceph-mon[35941]: Adjusting osd_memory_target on smithi132 to 174.1M 2024-03-28T02:10:42.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:10:41 smithi132 ceph-mon[35941]: Unable to set osd_memory_target on smithi132 to 182640640: error parsing value: Value '182640640' is below minimum 939524096 2024-03-28T02:10:42.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:10:41 smithi132 ceph-mon[35941]: Adjusting osd_memory_target on smithi165 to 174.1M 2024-03-28T02:10:42.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:10:41 smithi132 ceph-mon[35941]: Unable to set osd_memory_target on smithi165 to 182635622: error parsing value: Value '182635622' is below minimum 939524096 2024-03-28T02:10:43.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:10:42 smithi165 ceph-mon[36302]: pgmap v3268: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:10:43.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:10:42 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:10:43.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:10:42 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:10:43.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:10:42 smithi119 ceph-mon[31518]: pgmap v3268: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:10:43.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:10:42 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:10:43.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:10:42 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:10:43.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:10:42 smithi132 ceph-mon[35941]: pgmap v3268: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:10:43.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:10:42 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:10:43.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:10:42 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:10:44.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:10:43 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:10:44.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:10:43 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:10:44.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:10:43 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:10:44.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:10:43 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:10:44.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:10:43 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:10:44.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:10:43 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:10:44.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:10:43 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:10:44.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:10:43 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:10:44.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:10:43 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:10:44.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:10:43 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:10:44.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:10:43 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:10:44.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:10:43 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:10:44.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:10:43 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:10:44.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:10:43 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:10:44.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:10:43 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:10:44.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:10:43 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:10:44.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:10:43 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:10:44.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:10:43 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:10:44.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:10:43 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:10:44.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:10:43 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:10:44.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:10:43 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:10:44.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:10:43 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:10:44.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:10:43 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:10:44.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:10:43 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:10:44.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:10:43 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:10:44.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:10:43 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:10:44.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:10:43 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:10:44.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:10:43 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:10:44.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:10:43 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:10:44.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:10:43 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:10:44.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:10:43 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:10:44.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:10:43 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:10:44.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:10:43 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:10:45.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:10:44 smithi165 ceph-mon[36302]: Adjusting osd_memory_target on smithi119 to 176.7M 2024-03-28T02:10:45.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:10:44 smithi165 ceph-mon[36302]: Unable to set osd_memory_target on smithi119 to 185372364: error parsing value: Value '185372364' is below minimum 939524096 2024-03-28T02:10:45.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:10:44 smithi165 ceph-mon[36302]: pgmap v3269: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T02:10:45.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:10:44 smithi119 ceph-mon[31518]: Adjusting osd_memory_target on smithi119 to 176.7M 2024-03-28T02:10:45.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:10:44 smithi119 ceph-mon[31518]: Unable to set osd_memory_target on smithi119 to 185372364: error parsing value: Value '185372364' is below minimum 939524096 2024-03-28T02:10:45.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:10:44 smithi119 ceph-mon[31518]: pgmap v3269: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T02:10:45.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:10:44 smithi132 ceph-mon[35941]: Adjusting osd_memory_target on smithi119 to 176.7M 2024-03-28T02:10:45.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:10:44 smithi132 ceph-mon[35941]: Unable to set osd_memory_target on smithi119 to 185372364: error parsing value: Value '185372364' is below minimum 939524096 2024-03-28T02:10:45.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:10:44 smithi132 ceph-mon[35941]: pgmap v3269: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T02:10:46.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:10:45 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5705.742712 secs 2024-03-28T02:10:46.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:10:45 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5705.742712 secs 2024-03-28T02:10:46.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:10:45 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5705.742712 secs 2024-03-28T02:10:47.103 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:10:46 smithi132 ceph-mon[35941]: pgmap v3270: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 7 op/s 2024-03-28T02:10:47.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:10:46 smithi165 ceph-mon[36302]: pgmap v3270: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 7 op/s 2024-03-28T02:10:47.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:10:46 smithi119 ceph-mon[31518]: pgmap v3270: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 7 op/s 2024-03-28T02:10:49.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:10:48 smithi165 ceph-mon[36302]: pgmap v3271: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T02:10:49.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:10:48 smithi119 ceph-mon[31518]: pgmap v3271: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T02:10:49.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:10:48 smithi132 ceph-mon[35941]: pgmap v3271: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T02:10:51.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:10:50 smithi165 ceph-mon[36302]: pgmap v3272: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-28T02:10:51.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:10:50 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5710.742725 secs 2024-03-28T02:10:51.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:10:50 smithi119 ceph-mon[31518]: pgmap v3272: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-28T02:10:51.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:10:50 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5710.742725 secs 2024-03-28T02:10:51.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:10:50 smithi132 ceph-mon[35941]: pgmap v3272: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-28T02:10:51.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:10:50 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5710.742725 secs 2024-03-28T02:10:53.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:10:52 smithi165 ceph-mon[36302]: pgmap v3273: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T02:10:53.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:10:52 smithi119 ceph-mon[31518]: pgmap v3273: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T02:10:53.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:10:52 smithi132 ceph-mon[35941]: pgmap v3273: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T02:10:55.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:10:54 smithi165 ceph-mon[36302]: pgmap v3274: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T02:10:55.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:10:54 smithi119 ceph-mon[31518]: pgmap v3274: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T02:10:55.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:10:54 smithi132 ceph-mon[35941]: pgmap v3274: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T02:10:56.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:10:55 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5715.742814 secs 2024-03-28T02:10:56.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:10:55 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5715.742814 secs 2024-03-28T02:10:56.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:10:55 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5715.742814 secs 2024-03-28T02:10:57.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:10:56 smithi119 ceph-mon[31518]: pgmap v3275: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:10:57.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:10:56 smithi132 ceph-mon[35941]: pgmap v3275: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:10:57.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:10:56 smithi165 ceph-mon[36302]: pgmap v3275: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:10:59.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:10:58 smithi119 ceph-mon[31518]: pgmap v3276: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:10:59.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:10:58 smithi132 ceph-mon[35941]: pgmap v3276: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:10:59.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:10:58 smithi165 ceph-mon[36302]: pgmap v3276: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:11:01.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:11:00 smithi119 ceph-mon[31518]: pgmap v3277: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-28T02:11:01.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:11:00 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5720.742908 secs 2024-03-28T02:11:01.171 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:11:00 smithi165 ceph-mon[36302]: pgmap v3277: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-28T02:11:01.171 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:11:00 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5720.742908 secs 2024-03-28T02:11:01.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:11:00 smithi132 ceph-mon[35941]: pgmap v3277: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-28T02:11:01.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:11:00 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5720.742908 secs 2024-03-28T02:11:03.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:11:02 smithi132 ceph-mon[35941]: pgmap v3278: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-03-28T02:11:03.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:11:02 smithi165 ceph-mon[36302]: pgmap v3278: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-03-28T02:11:03.392 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:11:02 smithi119 ceph-mon[31518]: pgmap v3278: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-03-28T02:11:05.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:11:04 smithi132 ceph-mon[35941]: pgmap v3279: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:11:05.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:11:04 smithi165 ceph-mon[36302]: pgmap v3279: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:11:05.392 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:11:04 smithi119 ceph-mon[31518]: pgmap v3279: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:11:06.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:11:05 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5725.742969 secs 2024-03-28T02:11:06.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:11:05 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5725.742969 secs 2024-03-28T02:11:06.392 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:11:05 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5725.742969 secs 2024-03-28T02:11:07.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:11:06 smithi132 ceph-mon[35941]: pgmap v3280: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:11:07.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:11:06 smithi165 ceph-mon[36302]: pgmap v3280: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:11:07.392 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:11:06 smithi119 ceph-mon[31518]: pgmap v3280: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:11:09.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:11:08 smithi132 ceph-mon[35941]: pgmap v3281: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:11:09.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:11:08 smithi165 ceph-mon[36302]: pgmap v3281: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:11:09.392 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:11:08 smithi119 ceph-mon[31518]: pgmap v3281: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:11:11.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:11:10 smithi119 ceph-mon[31518]: pgmap v3282: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-28T02:11:11.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:11:10 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5730.743028 secs 2024-03-28T02:11:11.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:11:10 smithi132 ceph-mon[35941]: pgmap v3282: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-28T02:11:11.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:11:10 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5730.743028 secs 2024-03-28T02:11:11.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:11:10 smithi165 ceph-mon[36302]: pgmap v3282: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-28T02:11:11.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:11:10 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5730.743028 secs 2024-03-28T02:11:13.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:11:12 smithi132 ceph-mon[35941]: pgmap v3283: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-03-28T02:11:13.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:11:12 smithi165 ceph-mon[36302]: pgmap v3283: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-03-28T02:11:13.392 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:11:12 smithi119 ceph-mon[31518]: pgmap v3283: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-03-28T02:11:15.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:11:14 smithi132 ceph-mon[35941]: pgmap v3284: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:11:15.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:11:14 smithi165 ceph-mon[36302]: pgmap v3284: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:11:15.392 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:11:14 smithi119 ceph-mon[31518]: pgmap v3284: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:11:16.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:11:15 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5735.743089 secs 2024-03-28T02:11:16.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:11:15 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5735.743089 secs 2024-03-28T02:11:16.392 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:11:15 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5735.743089 secs 2024-03-28T02:11:17.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:11:16 smithi165 ceph-mon[36302]: pgmap v3285: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:11:17.392 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:11:16 smithi119 ceph-mon[31518]: pgmap v3285: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:11:17.439 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:11:16 smithi132 ceph-mon[35941]: pgmap v3285: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:11:18.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:11:17 smithi165 ceph-mon[36302]: pgmap v3286: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:11:18.392 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:11:17 smithi119 ceph-mon[31518]: pgmap v3286: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:11:18.439 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:11:17 smithi132 ceph-mon[35941]: pgmap v3286: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:11:20.619 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:11:20 smithi165 ceph-mon[36302]: pgmap v3287: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-28T02:11:20.642 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:11:20 smithi119 ceph-mon[31518]: pgmap v3287: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-28T02:11:20.689 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:11:20 smithi132 ceph-mon[35941]: pgmap v3287: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-28T02:11:21.619 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:11:21 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5740.743149 secs 2024-03-28T02:11:21.642 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:11:21 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5740.743149 secs 2024-03-28T02:11:21.689 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:11:21 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5740.743149 secs 2024-03-28T02:11:22.619 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:11:22 smithi165 ceph-mon[36302]: pgmap v3288: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-28T02:11:22.642 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:11:22 smithi119 ceph-mon[31518]: pgmap v3288: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-28T02:11:22.689 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:11:22 smithi132 ceph-mon[35941]: pgmap v3288: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-28T02:11:24.619 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:11:24 smithi165 ceph-mon[36302]: pgmap v3289: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:11:24.642 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:11:24 smithi119 ceph-mon[31518]: pgmap v3289: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:11:24.689 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:11:24 smithi132 ceph-mon[35941]: pgmap v3289: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:11:26.619 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:11:26 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5745.743235 secs 2024-03-28T02:11:26.619 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:11:26 smithi165 ceph-mon[36302]: pgmap v3290: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:11:26.642 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:11:26 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5745.743235 secs 2024-03-28T02:11:26.642 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:11:26 smithi119 ceph-mon[31518]: pgmap v3290: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:11:26.689 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:11:26 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5745.743235 secs 2024-03-28T02:11:26.689 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:11:26 smithi132 ceph-mon[35941]: pgmap v3290: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:11:28.619 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:11:28 smithi165 ceph-mon[36302]: pgmap v3291: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:11:28.642 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:11:28 smithi119 ceph-mon[31518]: pgmap v3291: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:11:28.689 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:11:28 smithi132 ceph-mon[35941]: pgmap v3291: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:11:30.619 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:11:30 smithi165 ceph-mon[36302]: pgmap v3292: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-28T02:11:30.642 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:11:30 smithi119 ceph-mon[31518]: pgmap v3292: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-28T02:11:30.689 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:11:30 smithi132 ceph-mon[35941]: pgmap v3292: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-28T02:11:31.520 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:11:31 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5750.743326 secs 2024-03-28T02:11:31.642 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:11:31 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5750.743326 secs 2024-03-28T02:11:31.689 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:11:31 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5750.743326 secs 2024-03-28T02:11:32.619 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:11:32 smithi165 ceph-mon[36302]: pgmap v3293: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-28T02:11:32.642 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:11:32 smithi119 ceph-mon[31518]: pgmap v3293: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-28T02:11:32.689 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:11:32 smithi132 ceph-mon[35941]: pgmap v3293: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-28T02:11:34.619 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:11:34 smithi165 ceph-mon[36302]: pgmap v3294: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:11:34.642 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:11:34 smithi119 ceph-mon[31518]: pgmap v3294: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:11:34.689 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:11:34 smithi132 ceph-mon[35941]: pgmap v3294: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:11:36.619 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:11:36 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5755.743416 secs 2024-03-28T02:11:36.619 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:11:36 smithi165 ceph-mon[36302]: pgmap v3295: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:11:36.642 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:11:36 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5755.743416 secs 2024-03-28T02:11:36.642 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:11:36 smithi119 ceph-mon[31518]: pgmap v3295: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:11:36.689 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:11:36 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5755.743416 secs 2024-03-28T02:11:36.689 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:11:36 smithi132 ceph-mon[35941]: pgmap v3295: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:11:38.619 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:11:38 smithi165 ceph-mon[36302]: pgmap v3296: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:11:38.642 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:11:38 smithi119 ceph-mon[31518]: pgmap v3296: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:11:38.689 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:11:38 smithi132 ceph-mon[35941]: pgmap v3296: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:11:40.619 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:11:40 smithi165 ceph-mon[36302]: pgmap v3297: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-28T02:11:40.642 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:11:40 smithi119 ceph-mon[31518]: pgmap v3297: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-28T02:11:40.689 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:11:40 smithi132 ceph-mon[35941]: pgmap v3297: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-28T02:11:41.619 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:11:41 smithi165 ceph-mon[36302]: 98 slow requests, 3 included below; oldest blocked for > 5760.743476 secs 2024-03-28T02:11:41.619 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:11:41 smithi165 ceph-mon[36302]: slow request 5760.523501 seconds old, received at 2024-03-28T00:35:39.578920+0000: internal op fragmentdir:mds.0:3783 currently failed to wrlock, waiting 2024-03-28T02:11:41.619 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:11:41 smithi165 ceph-mon[36302]: slow request 5760.514553 seconds old, received at 2024-03-28T00:35:39.587868+0000: client_request(client.25264:33202 create owner_uid=1000, owner_gid=1301 #0x10000000270/file8309 2024-03-28T00:35:39.586399+0000 caller_uid=1000, caller_gid=1301{6,36,1000,1301,}) currently failed to xlock, waiting 2024-03-28T02:11:41.619 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:11:41 smithi165 ceph-mon[36302]: slow request 5760.743476 seconds old, received at 2024-03-28T00:35:39.358945+0000: peer_request:client.25264:11489 currently acquired locks 2024-03-28T02:11:41.642 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:11:41 smithi119 ceph-mon[31518]: 98 slow requests, 3 included below; oldest blocked for > 5760.743476 secs 2024-03-28T02:11:41.642 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:11:41 smithi119 ceph-mon[31518]: slow request 5760.523501 seconds old, received at 2024-03-28T00:35:39.578920+0000: internal op fragmentdir:mds.0:3783 currently failed to wrlock, waiting 2024-03-28T02:11:41.643 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:11:41 smithi119 ceph-mon[31518]: slow request 5760.514553 seconds old, received at 2024-03-28T00:35:39.587868+0000: client_request(client.25264:33202 create owner_uid=1000, owner_gid=1301 #0x10000000270/file8309 2024-03-28T00:35:39.586399+0000 caller_uid=1000, caller_gid=1301{6,36,1000,1301,}) currently failed to xlock, waiting 2024-03-28T02:11:41.643 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:11:41 smithi119 ceph-mon[31518]: slow request 5760.743476 seconds old, received at 2024-03-28T00:35:39.358945+0000: peer_request:client.25264:11489 currently acquired locks 2024-03-28T02:11:41.689 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:11:41 smithi132 ceph-mon[35941]: 98 slow requests, 3 included below; oldest blocked for > 5760.743476 secs 2024-03-28T02:11:41.689 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:11:41 smithi132 ceph-mon[35941]: slow request 5760.523501 seconds old, received at 2024-03-28T00:35:39.578920+0000: internal op fragmentdir:mds.0:3783 currently failed to wrlock, waiting 2024-03-28T02:11:41.689 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:11:41 smithi132 ceph-mon[35941]: slow request 5760.514553 seconds old, received at 2024-03-28T00:35:39.587868+0000: client_request(client.25264:33202 create owner_uid=1000, owner_gid=1301 #0x10000000270/file8309 2024-03-28T00:35:39.586399+0000 caller_uid=1000, caller_gid=1301{6,36,1000,1301,}) currently failed to xlock, waiting 2024-03-28T02:11:41.689 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:11:41 smithi132 ceph-mon[35941]: slow request 5760.743476 seconds old, received at 2024-03-28T00:35:39.358945+0000: peer_request:client.25264:11489 currently acquired locks 2024-03-28T02:11:42.619 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:11:42 smithi165 ceph-mon[36302]: pgmap v3298: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-28T02:11:42.642 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:11:42 smithi119 ceph-mon[31518]: pgmap v3298: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-28T02:11:42.689 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:11:42 smithi132 ceph-mon[35941]: pgmap v3298: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-28T02:11:44.619 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:11:44 smithi165 ceph-mon[36302]: pgmap v3299: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:11:44.619 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:11:44 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:11:44.619 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:11:44 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:11:44.619 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:11:44 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:11:44.642 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:11:44 smithi119 ceph-mon[31518]: pgmap v3299: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:11:44.642 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:11:44 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:11:44.642 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:11:44 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:11:44.642 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:11:44 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:11:44.689 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:11:44 smithi132 ceph-mon[35941]: pgmap v3299: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:11:44.689 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:11:44 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:11:44.689 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:11:44 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:11:44.689 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:11:44 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:11:45.619 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:11:45 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:11:45.619 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:11:45 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:11:45.619 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:11:45 smithi165 ceph-mon[36302]: pgmap v3300: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T02:11:45.619 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:11:45 smithi165 ceph-mon[36302]: pgmap v3301: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:11:45.619 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:11:45 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:11:45.642 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:11:45 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:11:45.642 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:11:45 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:11:45.642 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:11:45 smithi119 ceph-mon[31518]: pgmap v3300: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T02:11:45.642 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:11:45 smithi119 ceph-mon[31518]: pgmap v3301: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:11:45.642 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:11:45 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:11:45.689 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:11:45 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:11:45.689 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:11:45 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:11:45.689 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:11:45 smithi132 ceph-mon[35941]: pgmap v3300: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T02:11:45.689 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:11:45 smithi132 ceph-mon[35941]: pgmap v3301: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:11:45.689 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:11:45 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:11:46.591 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:11:46 smithi132 ceph-mon[35941]: 98 slow requests, 5 included below; oldest blocked for > 5765.743566 secs 2024-03-28T02:11:46.591 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:11:46 smithi132 ceph-mon[35941]: slow request 5761.650903 seconds old, received at 2024-03-28T00:35:43.451608+0000: internal op fragmentdir:mds.0:3794 currently failed to wrlock, waiting 2024-03-28T02:11:46.591 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:11:46 smithi132 ceph-mon[35941]: slow request 5761.633235 seconds old, received at 2024-03-28T00:35:43.469275+0000: internal op fragmentdir:mds.0:3825 currently failed to wrlock, waiting 2024-03-28T02:11:46.591 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:11:46 smithi132 ceph-mon[35941]: slow request 5761.619143 seconds old, received at 2024-03-28T00:35:43.483367+0000: internal op fragmentdir:mds.0:3857 currently failed to wrlock, waiting 2024-03-28T02:11:46.591 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:11:46 smithi132 ceph-mon[35941]: slow request 5761.606082 seconds old, received at 2024-03-28T00:35:43.496429+0000: internal op fragmentdir:mds.0:3889 currently failed to wrlock, waiting 2024-03-28T02:11:46.591 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:11:46 smithi132 ceph-mon[35941]: slow request 5761.648927 seconds old, received at 2024-03-28T00:35:43.453584+0000: internal op fragmentdir:mds.0:3795 currently failed to wrlock, waiting 2024-03-28T02:11:46.619 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:11:46 smithi165 ceph-mon[36302]: 98 slow requests, 5 included below; oldest blocked for > 5765.743566 secs 2024-03-28T02:11:46.619 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:11:46 smithi165 ceph-mon[36302]: slow request 5761.650903 seconds old, received at 2024-03-28T00:35:43.451608+0000: internal op fragmentdir:mds.0:3794 currently failed to wrlock, waiting 2024-03-28T02:11:46.619 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:11:46 smithi165 ceph-mon[36302]: slow request 5761.633235 seconds old, received at 2024-03-28T00:35:43.469275+0000: internal op fragmentdir:mds.0:3825 currently failed to wrlock, waiting 2024-03-28T02:11:46.619 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:11:46 smithi165 ceph-mon[36302]: slow request 5761.619143 seconds old, received at 2024-03-28T00:35:43.483367+0000: internal op fragmentdir:mds.0:3857 currently failed to wrlock, waiting 2024-03-28T02:11:46.619 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:11:46 smithi165 ceph-mon[36302]: slow request 5761.606082 seconds old, received at 2024-03-28T00:35:43.496429+0000: internal op fragmentdir:mds.0:3889 currently failed to wrlock, waiting 2024-03-28T02:11:46.619 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:11:46 smithi165 ceph-mon[36302]: slow request 5761.648927 seconds old, received at 2024-03-28T00:35:43.453584+0000: internal op fragmentdir:mds.0:3795 currently failed to wrlock, waiting 2024-03-28T02:11:46.642 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:11:46 smithi119 ceph-mon[31518]: 98 slow requests, 5 included below; oldest blocked for > 5765.743566 secs 2024-03-28T02:11:46.642 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:11:46 smithi119 ceph-mon[31518]: slow request 5761.650903 seconds old, received at 2024-03-28T00:35:43.451608+0000: internal op fragmentdir:mds.0:3794 currently failed to wrlock, waiting 2024-03-28T02:11:46.642 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:11:46 smithi119 ceph-mon[31518]: slow request 5761.633235 seconds old, received at 2024-03-28T00:35:43.469275+0000: internal op fragmentdir:mds.0:3825 currently failed to wrlock, waiting 2024-03-28T02:11:46.642 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:11:46 smithi119 ceph-mon[31518]: slow request 5761.619143 seconds old, received at 2024-03-28T00:35:43.483367+0000: internal op fragmentdir:mds.0:3857 currently failed to wrlock, waiting 2024-03-28T02:11:46.642 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:11:46 smithi119 ceph-mon[31518]: slow request 5761.606082 seconds old, received at 2024-03-28T00:35:43.496429+0000: internal op fragmentdir:mds.0:3889 currently failed to wrlock, waiting 2024-03-28T02:11:46.643 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:11:46 smithi119 ceph-mon[31518]: slow request 5761.648927 seconds old, received at 2024-03-28T00:35:43.453584+0000: internal op fragmentdir:mds.0:3795 currently failed to wrlock, waiting 2024-03-28T02:11:47.619 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:11:47 smithi165 ceph-mon[36302]: pgmap v3302: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:11:47.642 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:11:47 smithi119 ceph-mon[31518]: pgmap v3302: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:11:47.689 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:11:47 smithi132 ceph-mon[35941]: pgmap v3302: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:11:50.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:11:49 smithi165 ceph-mon[36302]: pgmap v3303: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-28T02:11:50.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:11:49 smithi119 ceph-mon[31518]: pgmap v3303: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-28T02:11:50.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:11:49 smithi132 ceph-mon[35941]: pgmap v3303: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-28T02:11:51.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:11:50 smithi165 ceph-mon[36302]: 98 slow requests, 5 included below; oldest blocked for > 5770.743626 secs 2024-03-28T02:11:51.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:11:50 smithi165 ceph-mon[36302]: slow request 5766.567724 seconds old, received at 2024-03-28T00:35:43.534847+0000: internal op fragmentdir:mds.0:3922 currently failed to wrlock, waiting 2024-03-28T02:11:51.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:11:50 smithi165 ceph-mon[36302]: slow request 5766.648466 seconds old, received at 2024-03-28T00:35:43.454104+0000: internal op fragmentdir:mds.0:3796 currently failed to wrlock, waiting 2024-03-28T02:11:51.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:11:50 smithi165 ceph-mon[36302]: slow request 5766.632407 seconds old, received at 2024-03-28T00:35:43.470163+0000: internal op fragmentdir:mds.0:3827 currently failed to wrlock, waiting 2024-03-28T02:11:51.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:11:50 smithi165 ceph-mon[36302]: slow request 5766.605245 seconds old, received at 2024-03-28T00:35:43.497326+0000: internal op fragmentdir:mds.0:3891 currently failed to wrlock, waiting 2024-03-28T02:11:51.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:11:50 smithi165 ceph-mon[36302]: slow request 5766.540994 seconds old, received at 2024-03-28T00:35:43.561577+0000: internal op fragmentdir:mds.0:3923 currently failed to wrlock, waiting 2024-03-28T02:11:51.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:11:50 smithi119 ceph-mon[31518]: 98 slow requests, 5 included below; oldest blocked for > 5770.743626 secs 2024-03-28T02:11:51.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:11:50 smithi119 ceph-mon[31518]: slow request 5766.567724 seconds old, received at 2024-03-28T00:35:43.534847+0000: internal op fragmentdir:mds.0:3922 currently failed to wrlock, waiting 2024-03-28T02:11:51.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:11:50 smithi119 ceph-mon[31518]: slow request 5766.648466 seconds old, received at 2024-03-28T00:35:43.454104+0000: internal op fragmentdir:mds.0:3796 currently failed to wrlock, waiting 2024-03-28T02:11:51.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:11:50 smithi119 ceph-mon[31518]: slow request 5766.632407 seconds old, received at 2024-03-28T00:35:43.470163+0000: internal op fragmentdir:mds.0:3827 currently failed to wrlock, waiting 2024-03-28T02:11:51.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:11:50 smithi119 ceph-mon[31518]: slow request 5766.605245 seconds old, received at 2024-03-28T00:35:43.497326+0000: internal op fragmentdir:mds.0:3891 currently failed to wrlock, waiting 2024-03-28T02:11:51.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:11:50 smithi119 ceph-mon[31518]: slow request 5766.540994 seconds old, received at 2024-03-28T00:35:43.561577+0000: internal op fragmentdir:mds.0:3923 currently failed to wrlock, waiting 2024-03-28T02:11:51.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:11:50 smithi132 ceph-mon[35941]: 98 slow requests, 5 included below; oldest blocked for > 5770.743626 secs 2024-03-28T02:11:51.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:11:50 smithi132 ceph-mon[35941]: slow request 5766.567724 seconds old, received at 2024-03-28T00:35:43.534847+0000: internal op fragmentdir:mds.0:3922 currently failed to wrlock, waiting 2024-03-28T02:11:51.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:11:50 smithi132 ceph-mon[35941]: slow request 5766.648466 seconds old, received at 2024-03-28T00:35:43.454104+0000: internal op fragmentdir:mds.0:3796 currently failed to wrlock, waiting 2024-03-28T02:11:51.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:11:50 smithi132 ceph-mon[35941]: slow request 5766.632407 seconds old, received at 2024-03-28T00:35:43.470163+0000: internal op fragmentdir:mds.0:3827 currently failed to wrlock, waiting 2024-03-28T02:11:51.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:11:50 smithi132 ceph-mon[35941]: slow request 5766.605245 seconds old, received at 2024-03-28T00:35:43.497326+0000: internal op fragmentdir:mds.0:3891 currently failed to wrlock, waiting 2024-03-28T02:11:51.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:11:50 smithi132 ceph-mon[35941]: slow request 5766.540994 seconds old, received at 2024-03-28T00:35:43.561577+0000: internal op fragmentdir:mds.0:3923 currently failed to wrlock, waiting 2024-03-28T02:11:52.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:11:51 smithi165 ceph-mon[36302]: pgmap v3304: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:11:52.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:11:51 smithi119 ceph-mon[31518]: pgmap v3304: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:11:52.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:11:51 smithi132 ceph-mon[35941]: pgmap v3304: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:11:54.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:11:53 smithi165 ceph-mon[36302]: pgmap v3305: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:11:54.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:11:53 smithi119 ceph-mon[31518]: pgmap v3305: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:11:54.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:11:53 smithi132 ceph-mon[35941]: pgmap v3305: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:11:56.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:11:55 smithi165 ceph-mon[36302]: pgmap v3306: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T02:11:56.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:11:55 smithi165 ceph-mon[36302]: 98 slow requests, 5 included below; oldest blocked for > 5775.743660 secs 2024-03-28T02:11:56.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:11:55 smithi165 ceph-mon[36302]: slow request 5771.648027 seconds old, received at 2024-03-28T00:35:43.454578+0000: internal op fragmentdir:mds.0:3797 currently failed to wrlock, waiting 2024-03-28T02:11:56.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:11:55 smithi165 ceph-mon[36302]: slow request 5771.632000 seconds old, received at 2024-03-28T00:35:43.470605+0000: internal op fragmentdir:mds.0:3828 currently failed to wrlock, waiting 2024-03-28T02:11:56.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:11:55 smithi165 ceph-mon[36302]: slow request 5771.618035 seconds old, received at 2024-03-28T00:35:43.484570+0000: internal op fragmentdir:mds.0:3860 currently failed to wrlock, waiting 2024-03-28T02:11:56.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:11:55 smithi165 ceph-mon[36302]: slow request 5771.604829 seconds old, received at 2024-03-28T00:35:43.497776+0000: internal op fragmentdir:mds.0:3892 currently failed to wrlock, waiting 2024-03-28T02:11:56.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:11:55 smithi165 ceph-mon[36302]: slow request 5771.647550 seconds old, received at 2024-03-28T00:35:43.455055+0000: internal op fragmentdir:mds.0:3798 currently failed to wrlock, waiting 2024-03-28T02:11:56.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:11:55 smithi119 ceph-mon[31518]: pgmap v3306: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T02:11:56.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:11:55 smithi119 ceph-mon[31518]: 98 slow requests, 5 included below; oldest blocked for > 5775.743660 secs 2024-03-28T02:11:56.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:11:55 smithi119 ceph-mon[31518]: slow request 5771.648027 seconds old, received at 2024-03-28T00:35:43.454578+0000: internal op fragmentdir:mds.0:3797 currently failed to wrlock, waiting 2024-03-28T02:11:56.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:11:55 smithi119 ceph-mon[31518]: slow request 5771.632000 seconds old, received at 2024-03-28T00:35:43.470605+0000: internal op fragmentdir:mds.0:3828 currently failed to wrlock, waiting 2024-03-28T02:11:56.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:11:55 smithi119 ceph-mon[31518]: slow request 5771.618035 seconds old, received at 2024-03-28T00:35:43.484570+0000: internal op fragmentdir:mds.0:3860 currently failed to wrlock, waiting 2024-03-28T02:11:56.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:11:55 smithi119 ceph-mon[31518]: slow request 5771.604829 seconds old, received at 2024-03-28T00:35:43.497776+0000: internal op fragmentdir:mds.0:3892 currently failed to wrlock, waiting 2024-03-28T02:11:56.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:11:55 smithi119 ceph-mon[31518]: slow request 5771.647550 seconds old, received at 2024-03-28T00:35:43.455055+0000: internal op fragmentdir:mds.0:3798 currently failed to wrlock, waiting 2024-03-28T02:11:56.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:11:55 smithi132 ceph-mon[35941]: pgmap v3306: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T02:11:56.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:11:55 smithi132 ceph-mon[35941]: 98 slow requests, 5 included below; oldest blocked for > 5775.743660 secs 2024-03-28T02:11:56.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:11:55 smithi132 ceph-mon[35941]: slow request 5771.648027 seconds old, received at 2024-03-28T00:35:43.454578+0000: internal op fragmentdir:mds.0:3797 currently failed to wrlock, waiting 2024-03-28T02:11:56.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:11:55 smithi132 ceph-mon[35941]: slow request 5771.632000 seconds old, received at 2024-03-28T00:35:43.470605+0000: internal op fragmentdir:mds.0:3828 currently failed to wrlock, waiting 2024-03-28T02:11:56.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:11:55 smithi132 ceph-mon[35941]: slow request 5771.618035 seconds old, received at 2024-03-28T00:35:43.484570+0000: internal op fragmentdir:mds.0:3860 currently failed to wrlock, waiting 2024-03-28T02:11:56.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:11:55 smithi132 ceph-mon[35941]: slow request 5771.604829 seconds old, received at 2024-03-28T00:35:43.497776+0000: internal op fragmentdir:mds.0:3892 currently failed to wrlock, waiting 2024-03-28T02:11:56.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:11:55 smithi132 ceph-mon[35941]: slow request 5771.647550 seconds old, received at 2024-03-28T00:35:43.455055+0000: internal op fragmentdir:mds.0:3798 currently failed to wrlock, waiting 2024-03-28T02:11:58.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:11:57 smithi119 ceph-mon[31518]: pgmap v3307: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:11:58.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:11:57 smithi132 ceph-mon[35941]: pgmap v3307: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:11:58.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:11:57 smithi165 ceph-mon[36302]: pgmap v3307: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:12:00.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:11:59 smithi119 ceph-mon[31518]: pgmap v3308: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:12:00.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:11:59 smithi132 ceph-mon[35941]: pgmap v3308: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:12:00.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:11:59 smithi165 ceph-mon[36302]: pgmap v3308: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:12:01.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:12:00 smithi119 ceph-mon[31518]: 98 slow requests, 5 included below; oldest blocked for > 5780.743749 secs 2024-03-28T02:12:01.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:12:00 smithi119 ceph-mon[31518]: slow request 5776.603576 seconds old, received at 2024-03-28T00:35:43.499118+0000: internal op fragmentdir:mds.0:3893 currently failed to wrlock, waiting 2024-03-28T02:12:01.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:12:00 smithi119 ceph-mon[31518]: slow request 5776.647165 seconds old, received at 2024-03-28T00:35:43.455529+0000: internal op fragmentdir:mds.0:3799 currently failed to wrlock, waiting 2024-03-28T02:12:01.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:12:00 smithi119 ceph-mon[31518]: slow request 5776.617297 seconds old, received at 2024-03-28T00:35:43.485397+0000: internal op fragmentdir:mds.0:3862 currently failed to wrlock, waiting 2024-03-28T02:12:01.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:12:00 smithi119 ceph-mon[31518]: slow request 5776.603126 seconds old, received at 2024-03-28T00:35:43.499568+0000: internal op fragmentdir:mds.0:3894 currently failed to wrlock, waiting 2024-03-28T02:12:01.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:12:00 smithi119 ceph-mon[31518]: slow request 5776.646687 seconds old, received at 2024-03-28T00:35:43.456007+0000: internal op fragmentdir:mds.0:3800 currently failed to wrlock, waiting 2024-03-28T02:12:01.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:12:00 smithi132 ceph-mon[35941]: 98 slow requests, 5 included below; oldest blocked for > 5780.743749 secs 2024-03-28T02:12:01.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:12:00 smithi132 ceph-mon[35941]: slow request 5776.603576 seconds old, received at 2024-03-28T00:35:43.499118+0000: internal op fragmentdir:mds.0:3893 currently failed to wrlock, waiting 2024-03-28T02:12:01.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:12:00 smithi132 ceph-mon[35941]: slow request 5776.647165 seconds old, received at 2024-03-28T00:35:43.455529+0000: internal op fragmentdir:mds.0:3799 currently failed to wrlock, waiting 2024-03-28T02:12:01.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:12:00 smithi132 ceph-mon[35941]: slow request 5776.617297 seconds old, received at 2024-03-28T00:35:43.485397+0000: internal op fragmentdir:mds.0:3862 currently failed to wrlock, waiting 2024-03-28T02:12:01.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:12:00 smithi132 ceph-mon[35941]: slow request 5776.603126 seconds old, received at 2024-03-28T00:35:43.499568+0000: internal op fragmentdir:mds.0:3894 currently failed to wrlock, waiting 2024-03-28T02:12:01.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:12:00 smithi132 ceph-mon[35941]: slow request 5776.646687 seconds old, received at 2024-03-28T00:35:43.456007+0000: internal op fragmentdir:mds.0:3800 currently failed to wrlock, waiting 2024-03-28T02:12:01.270 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:12:00 smithi165 ceph-mon[36302]: 98 slow requests, 5 included below; oldest blocked for > 5780.743749 secs 2024-03-28T02:12:01.270 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:12:00 smithi165 ceph-mon[36302]: slow request 5776.603576 seconds old, received at 2024-03-28T00:35:43.499118+0000: internal op fragmentdir:mds.0:3893 currently failed to wrlock, waiting 2024-03-28T02:12:01.270 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:12:00 smithi165 ceph-mon[36302]: slow request 5776.647165 seconds old, received at 2024-03-28T00:35:43.455529+0000: internal op fragmentdir:mds.0:3799 currently failed to wrlock, waiting 2024-03-28T02:12:01.270 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:12:00 smithi165 ceph-mon[36302]: slow request 5776.617297 seconds old, received at 2024-03-28T00:35:43.485397+0000: internal op fragmentdir:mds.0:3862 currently failed to wrlock, waiting 2024-03-28T02:12:01.271 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:12:00 smithi165 ceph-mon[36302]: slow request 5776.603126 seconds old, received at 2024-03-28T00:35:43.499568+0000: internal op fragmentdir:mds.0:3894 currently failed to wrlock, waiting 2024-03-28T02:12:01.271 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:12:00 smithi165 ceph-mon[36302]: slow request 5776.646687 seconds old, received at 2024-03-28T00:35:43.456007+0000: internal op fragmentdir:mds.0:3800 currently failed to wrlock, waiting 2024-03-28T02:12:02.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:12:01 smithi119 ceph-mon[31518]: pgmap v3309: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:12:02.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:12:01 smithi132 ceph-mon[35941]: pgmap v3309: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:12:02.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:12:01 smithi165 ceph-mon[36302]: pgmap v3309: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:12:04.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:12:03 smithi132 ceph-mon[35941]: pgmap v3310: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:12:04.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:12:03 smithi165 ceph-mon[36302]: pgmap v3310: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:12:04.392 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:12:03 smithi119 ceph-mon[31518]: pgmap v3310: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:12:06.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:12:05 smithi132 ceph-mon[35941]: pgmap v3311: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:12:06.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:12:05 smithi132 ceph-mon[35941]: 98 slow requests, 5 included below; oldest blocked for > 5785.743792 secs 2024-03-28T02:12:06.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:12:05 smithi132 ceph-mon[35941]: slow request 5781.631065 seconds old, received at 2024-03-28T00:35:43.471672+0000: internal op fragmentdir:mds.0:3831 currently failed to wrlock, waiting 2024-03-28T02:12:06.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:12:05 smithi132 ceph-mon[35941]: slow request 5781.616949 seconds old, received at 2024-03-28T00:35:43.485788+0000: internal op fragmentdir:mds.0:3863 currently failed to wrlock, waiting 2024-03-28T02:12:06.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:12:05 smithi132 ceph-mon[35941]: slow request 5781.602706 seconds old, received at 2024-03-28T00:35:43.500032+0000: internal op fragmentdir:mds.0:3895 currently failed to wrlock, waiting 2024-03-28T02:12:06.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:12:05 smithi132 ceph-mon[35941]: slow request 5781.646274 seconds old, received at 2024-03-28T00:35:43.456464+0000: internal op fragmentdir:mds.0:3801 currently failed to wrlock, waiting 2024-03-28T02:12:06.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:12:05 smithi132 ceph-mon[35941]: slow request 5781.630606 seconds old, received at 2024-03-28T00:35:43.472131+0000: internal op fragmentdir:mds.0:3832 currently failed to wrlock, waiting 2024-03-28T02:12:06.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:12:05 smithi165 ceph-mon[36302]: pgmap v3311: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:12:06.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:12:05 smithi165 ceph-mon[36302]: 98 slow requests, 5 included below; oldest blocked for > 5785.743792 secs 2024-03-28T02:12:06.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:12:05 smithi165 ceph-mon[36302]: slow request 5781.631065 seconds old, received at 2024-03-28T00:35:43.471672+0000: internal op fragmentdir:mds.0:3831 currently failed to wrlock, waiting 2024-03-28T02:12:06.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:12:05 smithi165 ceph-mon[36302]: slow request 5781.616949 seconds old, received at 2024-03-28T00:35:43.485788+0000: internal op fragmentdir:mds.0:3863 currently failed to wrlock, waiting 2024-03-28T02:12:06.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:12:05 smithi165 ceph-mon[36302]: slow request 5781.602706 seconds old, received at 2024-03-28T00:35:43.500032+0000: internal op fragmentdir:mds.0:3895 currently failed to wrlock, waiting 2024-03-28T02:12:06.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:12:05 smithi165 ceph-mon[36302]: slow request 5781.646274 seconds old, received at 2024-03-28T00:35:43.456464+0000: internal op fragmentdir:mds.0:3801 currently failed to wrlock, waiting 2024-03-28T02:12:06.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:12:05 smithi165 ceph-mon[36302]: slow request 5781.630606 seconds old, received at 2024-03-28T00:35:43.472131+0000: internal op fragmentdir:mds.0:3832 currently failed to wrlock, waiting 2024-03-28T02:12:06.392 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:12:05 smithi119 ceph-mon[31518]: pgmap v3311: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:12:06.392 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:12:05 smithi119 ceph-mon[31518]: 98 slow requests, 5 included below; oldest blocked for > 5785.743792 secs 2024-03-28T02:12:06.392 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:12:05 smithi119 ceph-mon[31518]: slow request 5781.631065 seconds old, received at 2024-03-28T00:35:43.471672+0000: internal op fragmentdir:mds.0:3831 currently failed to wrlock, waiting 2024-03-28T02:12:06.392 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:12:05 smithi119 ceph-mon[31518]: slow request 5781.616949 seconds old, received at 2024-03-28T00:35:43.485788+0000: internal op fragmentdir:mds.0:3863 currently failed to wrlock, waiting 2024-03-28T02:12:06.392 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:12:05 smithi119 ceph-mon[31518]: slow request 5781.602706 seconds old, received at 2024-03-28T00:35:43.500032+0000: internal op fragmentdir:mds.0:3895 currently failed to wrlock, waiting 2024-03-28T02:12:06.393 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:12:05 smithi119 ceph-mon[31518]: slow request 5781.646274 seconds old, received at 2024-03-28T00:35:43.456464+0000: internal op fragmentdir:mds.0:3801 currently failed to wrlock, waiting 2024-03-28T02:12:06.393 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:12:05 smithi119 ceph-mon[31518]: slow request 5781.630606 seconds old, received at 2024-03-28T00:35:43.472131+0000: internal op fragmentdir:mds.0:3832 currently failed to wrlock, waiting 2024-03-28T02:12:08.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:12:07 smithi132 ceph-mon[35941]: pgmap v3312: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:12:08.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:12:07 smithi165 ceph-mon[36302]: pgmap v3312: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:12:08.392 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:12:07 smithi119 ceph-mon[31518]: pgmap v3312: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:12:10.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:12:09 smithi132 ceph-mon[35941]: pgmap v3313: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:12:10.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:12:09 smithi165 ceph-mon[36302]: pgmap v3313: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:12:10.392 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:12:09 smithi119 ceph-mon[31518]: pgmap v3313: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:12:11.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:12:10 smithi132 ceph-mon[35941]: 98 slow requests, 5 included below; oldest blocked for > 5790.743883 secs 2024-03-28T02:12:11.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:12:10 smithi132 ceph-mon[35941]: slow request 5786.645847 seconds old, received at 2024-03-28T00:35:43.456981+0000: internal op fragmentdir:mds.0:3802 currently failed to wrlock, waiting 2024-03-28T02:12:11.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:12:10 smithi132 ceph-mon[35941]: slow request 5786.616238 seconds old, received at 2024-03-28T00:35:43.486590+0000: internal op fragmentdir:mds.0:3865 currently failed to wrlock, waiting 2024-03-28T02:12:11.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:12:10 smithi132 ceph-mon[35941]: slow request 5786.601924 seconds old, received at 2024-03-28T00:35:43.500904+0000: internal op fragmentdir:mds.0:3897 currently failed to wrlock, waiting 2024-03-28T02:12:11.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:12:10 smithi132 ceph-mon[35941]: slow request 5786.645398 seconds old, received at 2024-03-28T00:35:43.457430+0000: internal op fragmentdir:mds.0:3803 currently failed to wrlock, waiting 2024-03-28T02:12:11.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:12:10 smithi132 ceph-mon[35941]: slow request 5786.629812 seconds old, received at 2024-03-28T00:35:43.473016+0000: internal op fragmentdir:mds.0:3834 currently failed to wrlock, waiting 2024-03-28T02:12:11.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:12:10 smithi165 ceph-mon[36302]: 98 slow requests, 5 included below; oldest blocked for > 5790.743883 secs 2024-03-28T02:12:11.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:12:10 smithi165 ceph-mon[36302]: slow request 5786.645847 seconds old, received at 2024-03-28T00:35:43.456981+0000: internal op fragmentdir:mds.0:3802 currently failed to wrlock, waiting 2024-03-28T02:12:11.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:12:10 smithi165 ceph-mon[36302]: slow request 5786.616238 seconds old, received at 2024-03-28T00:35:43.486590+0000: internal op fragmentdir:mds.0:3865 currently failed to wrlock, waiting 2024-03-28T02:12:11.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:12:10 smithi165 ceph-mon[36302]: slow request 5786.601924 seconds old, received at 2024-03-28T00:35:43.500904+0000: internal op fragmentdir:mds.0:3897 currently failed to wrlock, waiting 2024-03-28T02:12:11.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:12:10 smithi165 ceph-mon[36302]: slow request 5786.645398 seconds old, received at 2024-03-28T00:35:43.457430+0000: internal op fragmentdir:mds.0:3803 currently failed to wrlock, waiting 2024-03-28T02:12:11.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:12:10 smithi165 ceph-mon[36302]: slow request 5786.629812 seconds old, received at 2024-03-28T00:35:43.473016+0000: internal op fragmentdir:mds.0:3834 currently failed to wrlock, waiting 2024-03-28T02:12:11.392 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:12:10 smithi119 ceph-mon[31518]: 98 slow requests, 5 included below; oldest blocked for > 5790.743883 secs 2024-03-28T02:12:11.392 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:12:10 smithi119 ceph-mon[31518]: slow request 5786.645847 seconds old, received at 2024-03-28T00:35:43.456981+0000: internal op fragmentdir:mds.0:3802 currently failed to wrlock, waiting 2024-03-28T02:12:11.392 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:12:10 smithi119 ceph-mon[31518]: slow request 5786.616238 seconds old, received at 2024-03-28T00:35:43.486590+0000: internal op fragmentdir:mds.0:3865 currently failed to wrlock, waiting 2024-03-28T02:12:11.392 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:12:10 smithi119 ceph-mon[31518]: slow request 5786.601924 seconds old, received at 2024-03-28T00:35:43.500904+0000: internal op fragmentdir:mds.0:3897 currently failed to wrlock, waiting 2024-03-28T02:12:11.392 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:12:10 smithi119 ceph-mon[31518]: slow request 5786.645398 seconds old, received at 2024-03-28T00:35:43.457430+0000: internal op fragmentdir:mds.0:3803 currently failed to wrlock, waiting 2024-03-28T02:12:11.392 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:12:10 smithi119 ceph-mon[31518]: slow request 5786.629812 seconds old, received at 2024-03-28T00:35:43.473016+0000: internal op fragmentdir:mds.0:3834 currently failed to wrlock, waiting 2024-03-28T02:12:12.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:12:11 smithi132 ceph-mon[35941]: pgmap v3314: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:12:12.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:12:11 smithi165 ceph-mon[36302]: pgmap v3314: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:12:12.392 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:12:11 smithi119 ceph-mon[31518]: pgmap v3314: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:12:14.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:12:13 smithi132 ceph-mon[35941]: pgmap v3315: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:12:14.370 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:12:13 smithi165 ceph-mon[36302]: pgmap v3315: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:12:14.392 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:12:13 smithi119 ceph-mon[31518]: pgmap v3315: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:12:16.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:12:15 smithi132 ceph-mon[35941]: pgmap v3316: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:12:16.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:12:15 smithi132 ceph-mon[35941]: 98 slow requests, 5 included below; oldest blocked for > 5795.743904 secs 2024-03-28T02:12:16.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:12:15 smithi132 ceph-mon[35941]: slow request 5791.644912 seconds old, received at 2024-03-28T00:35:43.457937+0000: internal op fragmentdir:mds.0:3804 currently failed to wrlock, waiting 2024-03-28T02:12:16.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:12:15 smithi132 ceph-mon[35941]: slow request 5791.615464 seconds old, received at 2024-03-28T00:35:43.487385+0000: internal op fragmentdir:mds.0:3867 currently failed to wrlock, waiting 2024-03-28T02:12:16.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:12:15 smithi132 ceph-mon[35941]: slow request 5791.601078 seconds old, received at 2024-03-28T00:35:43.501771+0000: internal op fragmentdir:mds.0:3899 currently failed to wrlock, waiting 2024-03-28T02:12:16.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:12:15 smithi132 ceph-mon[35941]: slow request 5791.644423 seconds old, received at 2024-03-28T00:35:43.458426+0000: internal op fragmentdir:mds.0:3805 currently failed to wrlock, waiting 2024-03-28T02:12:16.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:12:15 smithi132 ceph-mon[35941]: slow request 5791.628946 seconds old, received at 2024-03-28T00:35:43.473902+0000: internal op fragmentdir:mds.0:3836 currently failed to wrlock, waiting 2024-03-28T02:12:16.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:12:15 smithi165 ceph-mon[36302]: pgmap v3316: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:12:16.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:12:15 smithi165 ceph-mon[36302]: 98 slow requests, 5 included below; oldest blocked for > 5795.743904 secs 2024-03-28T02:12:16.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:12:15 smithi165 ceph-mon[36302]: slow request 5791.644912 seconds old, received at 2024-03-28T00:35:43.457937+0000: internal op fragmentdir:mds.0:3804 currently failed to wrlock, waiting 2024-03-28T02:12:16.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:12:15 smithi165 ceph-mon[36302]: slow request 5791.615464 seconds old, received at 2024-03-28T00:35:43.487385+0000: internal op fragmentdir:mds.0:3867 currently failed to wrlock, waiting 2024-03-28T02:12:16.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:12:15 smithi165 ceph-mon[36302]: slow request 5791.601078 seconds old, received at 2024-03-28T00:35:43.501771+0000: internal op fragmentdir:mds.0:3899 currently failed to wrlock, waiting 2024-03-28T02:12:16.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:12:15 smithi165 ceph-mon[36302]: slow request 5791.644423 seconds old, received at 2024-03-28T00:35:43.458426+0000: internal op fragmentdir:mds.0:3805 currently failed to wrlock, waiting 2024-03-28T02:12:16.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:12:15 smithi165 ceph-mon[36302]: slow request 5791.628946 seconds old, received at 2024-03-28T00:35:43.473902+0000: internal op fragmentdir:mds.0:3836 currently failed to wrlock, waiting 2024-03-28T02:12:16.392 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:12:15 smithi119 ceph-mon[31518]: pgmap v3316: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:12:16.392 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:12:15 smithi119 ceph-mon[31518]: 98 slow requests, 5 included below; oldest blocked for > 5795.743904 secs 2024-03-28T02:12:16.392 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:12:15 smithi119 ceph-mon[31518]: slow request 5791.644912 seconds old, received at 2024-03-28T00:35:43.457937+0000: internal op fragmentdir:mds.0:3804 currently failed to wrlock, waiting 2024-03-28T02:12:16.392 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:12:15 smithi119 ceph-mon[31518]: slow request 5791.615464 seconds old, received at 2024-03-28T00:35:43.487385+0000: internal op fragmentdir:mds.0:3867 currently failed to wrlock, waiting 2024-03-28T02:12:16.392 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:12:15 smithi119 ceph-mon[31518]: slow request 5791.601078 seconds old, received at 2024-03-28T00:35:43.501771+0000: internal op fragmentdir:mds.0:3899 currently failed to wrlock, waiting 2024-03-28T02:12:16.392 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:12:15 smithi119 ceph-mon[31518]: slow request 5791.644423 seconds old, received at 2024-03-28T00:35:43.458426+0000: internal op fragmentdir:mds.0:3805 currently failed to wrlock, waiting 2024-03-28T02:12:16.393 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:12:15 smithi119 ceph-mon[31518]: slow request 5791.628946 seconds old, received at 2024-03-28T00:35:43.473902+0000: internal op fragmentdir:mds.0:3836 currently failed to wrlock, waiting 2024-03-28T02:12:18.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:12:17 smithi165 ceph-mon[36302]: pgmap v3317: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:12:18.392 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:12:17 smithi119 ceph-mon[31518]: pgmap v3317: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:12:18.439 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:12:17 smithi132 ceph-mon[35941]: pgmap v3317: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:12:19.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:12:18 smithi165 ceph-mon[36302]: pgmap v3318: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:12:19.392 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:12:18 smithi119 ceph-mon[31518]: pgmap v3318: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:12:19.439 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:12:18 smithi132 ceph-mon[35941]: pgmap v3318: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:12:20.869 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:12:20 smithi165 ceph-mon[36302]: 98 slow requests, 5 included below; oldest blocked for > 5800.743965 secs 2024-03-28T02:12:20.869 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:12:20 smithi165 ceph-mon[36302]: slow request 5796.615128 seconds old, received at 2024-03-28T00:35:43.487781+0000: internal op fragmentdir:mds.0:3868 currently failed to wrlock, waiting 2024-03-28T02:12:20.869 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:12:20 smithi165 ceph-mon[36302]: slow request 5796.643997 seconds old, received at 2024-03-28T00:35:43.458913+0000: internal op fragmentdir:mds.0:3806 currently failed to wrlock, waiting 2024-03-28T02:12:20.869 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:12:20 smithi165 ceph-mon[36302]: slow request 5796.628571 seconds old, received at 2024-03-28T00:35:43.474338+0000: internal op fragmentdir:mds.0:3837 currently failed to wrlock, waiting 2024-03-28T02:12:20.870 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:12:20 smithi165 ceph-mon[36302]: slow request 5796.600254 seconds old, received at 2024-03-28T00:35:43.502655+0000: internal op fragmentdir:mds.0:3901 currently failed to wrlock, waiting 2024-03-28T02:12:20.870 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:12:20 smithi165 ceph-mon[36302]: slow request 5796.644642 seconds old, received at 2024-03-28T00:35:43.458267+0000: client_request(client.25264:34113 create owner_uid=1000, owner_gid=1301 #0x10000000272/file7594 2024-03-28T00:35:43.457305+0000 caller_uid=1000, caller_gid=1301{6,36,1000,1301,}) currently failed to xlock, waiting 2024-03-28T02:12:20.892 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:12:20 smithi119 ceph-mon[31518]: 98 slow requests, 5 included below; oldest blocked for > 5800.743965 secs 2024-03-28T02:12:20.892 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:12:20 smithi119 ceph-mon[31518]: slow request 5796.615128 seconds old, received at 2024-03-28T00:35:43.487781+0000: internal op fragmentdir:mds.0:3868 currently failed to wrlock, waiting 2024-03-28T02:12:20.892 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:12:20 smithi119 ceph-mon[31518]: slow request 5796.643997 seconds old, received at 2024-03-28T00:35:43.458913+0000: internal op fragmentdir:mds.0:3806 currently failed to wrlock, waiting 2024-03-28T02:12:20.892 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:12:20 smithi119 ceph-mon[31518]: slow request 5796.628571 seconds old, received at 2024-03-28T00:35:43.474338+0000: internal op fragmentdir:mds.0:3837 currently failed to wrlock, waiting 2024-03-28T02:12:20.892 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:12:20 smithi119 ceph-mon[31518]: slow request 5796.600254 seconds old, received at 2024-03-28T00:35:43.502655+0000: internal op fragmentdir:mds.0:3901 currently failed to wrlock, waiting 2024-03-28T02:12:20.892 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:12:20 smithi119 ceph-mon[31518]: slow request 5796.644642 seconds old, received at 2024-03-28T00:35:43.458267+0000: client_request(client.25264:34113 create owner_uid=1000, owner_gid=1301 #0x10000000272/file7594 2024-03-28T00:35:43.457305+0000 caller_uid=1000, caller_gid=1301{6,36,1000,1301,}) currently failed to xlock, waiting 2024-03-28T02:12:20.939 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:12:20 smithi132 ceph-mon[35941]: 98 slow requests, 5 included below; oldest blocked for > 5800.743965 secs 2024-03-28T02:12:20.939 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:12:20 smithi132 ceph-mon[35941]: slow request 5796.615128 seconds old, received at 2024-03-28T00:35:43.487781+0000: internal op fragmentdir:mds.0:3868 currently failed to wrlock, waiting 2024-03-28T02:12:20.939 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:12:20 smithi132 ceph-mon[35941]: slow request 5796.643997 seconds old, received at 2024-03-28T00:35:43.458913+0000: internal op fragmentdir:mds.0:3806 currently failed to wrlock, waiting 2024-03-28T02:12:20.939 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:12:20 smithi132 ceph-mon[35941]: slow request 5796.628571 seconds old, received at 2024-03-28T00:35:43.474338+0000: internal op fragmentdir:mds.0:3837 currently failed to wrlock, waiting 2024-03-28T02:12:20.939 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:12:20 smithi132 ceph-mon[35941]: slow request 5796.600254 seconds old, received at 2024-03-28T00:35:43.502655+0000: internal op fragmentdir:mds.0:3901 currently failed to wrlock, waiting 2024-03-28T02:12:20.939 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:12:20 smithi132 ceph-mon[35941]: slow request 5796.644642 seconds old, received at 2024-03-28T00:35:43.458267+0000: client_request(client.25264:34113 create owner_uid=1000, owner_gid=1301 #0x10000000272/file7594 2024-03-28T00:35:43.457305+0000 caller_uid=1000, caller_gid=1301{6,36,1000,1301,}) currently failed to xlock, waiting 2024-03-28T02:12:21.892 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:12:21 smithi119 ceph-mon[31518]: pgmap v3319: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:12:21.939 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:12:21 smithi132 ceph-mon[35941]: pgmap v3319: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:12:22.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:12:21 smithi165 ceph-mon[36302]: pgmap v3319: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:12:24.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:12:23 smithi165 ceph-mon[36302]: pgmap v3320: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:12:24.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:12:23 smithi119 ceph-mon[31518]: pgmap v3320: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:12:24.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:12:23 smithi132 ceph-mon[35941]: pgmap v3320: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:12:26.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:12:25 smithi165 ceph-mon[36302]: pgmap v3321: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:12:26.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:12:25 smithi165 ceph-mon[36302]: 98 slow requests, 5 included below; oldest blocked for > 5805.744048 secs 2024-03-28T02:12:26.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:12:25 smithi165 ceph-mon[36302]: slow request 5801.614687 seconds old, received at 2024-03-28T00:35:43.488305+0000: internal op fragmentdir:mds.0:3870 currently failed to wrlock, waiting 2024-03-28T02:12:26.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:12:25 smithi165 ceph-mon[36302]: slow request 5801.641145 seconds old, received at 2024-03-28T00:35:43.461848+0000: internal op fragmentdir:mds.0:3807 currently failed to wrlock, waiting 2024-03-28T02:12:26.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:12:25 smithi165 ceph-mon[36302]: slow request 5801.627797 seconds old, received at 2024-03-28T00:35:43.475195+0000: internal op fragmentdir:mds.0:3839 currently failed to wrlock, waiting 2024-03-28T02:12:26.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:12:25 smithi165 ceph-mon[36302]: slow request 5801.614268 seconds old, received at 2024-03-28T00:35:43.488725+0000: internal op fragmentdir:mds.0:3871 currently failed to wrlock, waiting 2024-03-28T02:12:26.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:12:25 smithi165 ceph-mon[36302]: slow request 5801.599685 seconds old, received at 2024-03-28T00:35:43.503308+0000: internal op fragmentdir:mds.0:3903 currently failed to wrlock, waiting 2024-03-28T02:12:26.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:12:25 smithi119 ceph-mon[31518]: pgmap v3321: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:12:26.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:12:25 smithi119 ceph-mon[31518]: 98 slow requests, 5 included below; oldest blocked for > 5805.744048 secs 2024-03-28T02:12:26.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:12:25 smithi119 ceph-mon[31518]: slow request 5801.614687 seconds old, received at 2024-03-28T00:35:43.488305+0000: internal op fragmentdir:mds.0:3870 currently failed to wrlock, waiting 2024-03-28T02:12:26.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:12:25 smithi119 ceph-mon[31518]: slow request 5801.641145 seconds old, received at 2024-03-28T00:35:43.461848+0000: internal op fragmentdir:mds.0:3807 currently failed to wrlock, waiting 2024-03-28T02:12:26.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:12:25 smithi119 ceph-mon[31518]: slow request 5801.627797 seconds old, received at 2024-03-28T00:35:43.475195+0000: internal op fragmentdir:mds.0:3839 currently failed to wrlock, waiting 2024-03-28T02:12:26.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:12:25 smithi119 ceph-mon[31518]: slow request 5801.614268 seconds old, received at 2024-03-28T00:35:43.488725+0000: internal op fragmentdir:mds.0:3871 currently failed to wrlock, waiting 2024-03-28T02:12:26.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:12:25 smithi119 ceph-mon[31518]: slow request 5801.599685 seconds old, received at 2024-03-28T00:35:43.503308+0000: internal op fragmentdir:mds.0:3903 currently failed to wrlock, waiting 2024-03-28T02:12:26.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:12:25 smithi132 ceph-mon[35941]: pgmap v3321: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:12:26.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:12:25 smithi132 ceph-mon[35941]: 98 slow requests, 5 included below; oldest blocked for > 5805.744048 secs 2024-03-28T02:12:26.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:12:25 smithi132 ceph-mon[35941]: slow request 5801.614687 seconds old, received at 2024-03-28T00:35:43.488305+0000: internal op fragmentdir:mds.0:3870 currently failed to wrlock, waiting 2024-03-28T02:12:26.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:12:25 smithi132 ceph-mon[35941]: slow request 5801.641145 seconds old, received at 2024-03-28T00:35:43.461848+0000: internal op fragmentdir:mds.0:3807 currently failed to wrlock, waiting 2024-03-28T02:12:26.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:12:25 smithi132 ceph-mon[35941]: slow request 5801.627797 seconds old, received at 2024-03-28T00:35:43.475195+0000: internal op fragmentdir:mds.0:3839 currently failed to wrlock, waiting 2024-03-28T02:12:26.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:12:25 smithi132 ceph-mon[35941]: slow request 5801.614268 seconds old, received at 2024-03-28T00:35:43.488725+0000: internal op fragmentdir:mds.0:3871 currently failed to wrlock, waiting 2024-03-28T02:12:26.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:12:25 smithi132 ceph-mon[35941]: slow request 5801.599685 seconds old, received at 2024-03-28T00:35:43.503308+0000: internal op fragmentdir:mds.0:3903 currently failed to wrlock, waiting 2024-03-28T02:12:28.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:12:27 smithi119 ceph-mon[31518]: pgmap v3322: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:12:28.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:12:27 smithi132 ceph-mon[35941]: pgmap v3322: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:12:28.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:12:27 smithi165 ceph-mon[36302]: pgmap v3322: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:12:30.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:12:29 smithi119 ceph-mon[31518]: pgmap v3323: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:12:30.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:12:29 smithi132 ceph-mon[35941]: pgmap v3323: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:12:30.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:12:29 smithi165 ceph-mon[36302]: pgmap v3323: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:12:31.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:12:30 smithi119 ceph-mon[31518]: 98 slow requests, 5 included below; oldest blocked for > 5810.744113 secs 2024-03-28T02:12:31.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:12:30 smithi119 ceph-mon[31518]: slow request 5789.726225 seconds old, received at 2024-03-28T00:36:00.376832+0000: client_request(client.25355:96 mksnap #0x1//scheduled-2024-03-28-00_36_00_UTC 2024-03-28T00:36:00.376777+0000 caller_uid=0, caller_gid=0{}) currently failed to acquire_locks 2024-03-28T02:12:31.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:12:30 smithi119 ceph-mon[31518]: slow request 5806.640756 seconds old, received at 2024-03-28T00:35:43.462302+0000: internal op fragmentdir:mds.0:3808 currently failed to wrlock, waiting 2024-03-28T02:12:31.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:12:30 smithi119 ceph-mon[31518]: slow request 5806.704362 seconds old, received at 2024-03-28T00:35:43.398696+0000: internal op fragmentdir:mds.0:3785 currently failed to wrlock, waiting 2024-03-28T02:12:31.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:12:30 smithi119 ceph-mon[31518]: slow request 5806.640294 seconds old, received at 2024-03-28T00:35:43.462764+0000: internal op fragmentdir:mds.0:3809 currently failed to wrlock, waiting 2024-03-28T02:12:31.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:12:30 smithi119 ceph-mon[31518]: slow request 5806.626964 seconds old, received at 2024-03-28T00:35:43.476094+0000: internal op fragmentdir:mds.0:3841 currently failed to wrlock, waiting 2024-03-28T02:12:31.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:12:30 smithi132 ceph-mon[35941]: 98 slow requests, 5 included below; oldest blocked for > 5810.744113 secs 2024-03-28T02:12:31.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:12:30 smithi132 ceph-mon[35941]: slow request 5789.726225 seconds old, received at 2024-03-28T00:36:00.376832+0000: client_request(client.25355:96 mksnap #0x1//scheduled-2024-03-28-00_36_00_UTC 2024-03-28T00:36:00.376777+0000 caller_uid=0, caller_gid=0{}) currently failed to acquire_locks 2024-03-28T02:12:31.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:12:30 smithi132 ceph-mon[35941]: slow request 5806.640756 seconds old, received at 2024-03-28T00:35:43.462302+0000: internal op fragmentdir:mds.0:3808 currently failed to wrlock, waiting 2024-03-28T02:12:31.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:12:30 smithi132 ceph-mon[35941]: slow request 5806.704362 seconds old, received at 2024-03-28T00:35:43.398696+0000: internal op fragmentdir:mds.0:3785 currently failed to wrlock, waiting 2024-03-28T02:12:31.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:12:30 smithi132 ceph-mon[35941]: slow request 5806.640294 seconds old, received at 2024-03-28T00:35:43.462764+0000: internal op fragmentdir:mds.0:3809 currently failed to wrlock, waiting 2024-03-28T02:12:31.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:12:30 smithi132 ceph-mon[35941]: slow request 5806.626964 seconds old, received at 2024-03-28T00:35:43.476094+0000: internal op fragmentdir:mds.0:3841 currently failed to wrlock, waiting 2024-03-28T02:12:31.270 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:12:30 smithi165 ceph-mon[36302]: 98 slow requests, 5 included below; oldest blocked for > 5810.744113 secs 2024-03-28T02:12:31.270 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:12:30 smithi165 ceph-mon[36302]: slow request 5789.726225 seconds old, received at 2024-03-28T00:36:00.376832+0000: client_request(client.25355:96 mksnap #0x1//scheduled-2024-03-28-00_36_00_UTC 2024-03-28T00:36:00.376777+0000 caller_uid=0, caller_gid=0{}) currently failed to acquire_locks 2024-03-28T02:12:31.270 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:12:30 smithi165 ceph-mon[36302]: slow request 5806.640756 seconds old, received at 2024-03-28T00:35:43.462302+0000: internal op fragmentdir:mds.0:3808 currently failed to wrlock, waiting 2024-03-28T02:12:31.270 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:12:30 smithi165 ceph-mon[36302]: slow request 5806.704362 seconds old, received at 2024-03-28T00:35:43.398696+0000: internal op fragmentdir:mds.0:3785 currently failed to wrlock, waiting 2024-03-28T02:12:31.271 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:12:30 smithi165 ceph-mon[36302]: slow request 5806.640294 seconds old, received at 2024-03-28T00:35:43.462764+0000: internal op fragmentdir:mds.0:3809 currently failed to wrlock, waiting 2024-03-28T02:12:31.271 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:12:30 smithi165 ceph-mon[36302]: slow request 5806.626964 seconds old, received at 2024-03-28T00:35:43.476094+0000: internal op fragmentdir:mds.0:3841 currently failed to wrlock, waiting 2024-03-28T02:12:32.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:12:31 smithi119 ceph-mon[31518]: pgmap v3324: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:12:32.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:12:31 smithi132 ceph-mon[35941]: pgmap v3324: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:12:32.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:12:31 smithi165 ceph-mon[36302]: pgmap v3324: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:12:34.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:12:33 smithi132 ceph-mon[35941]: pgmap v3325: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:12:34.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:12:33 smithi165 ceph-mon[36302]: pgmap v3325: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:12:34.392 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:12:33 smithi119 ceph-mon[31518]: pgmap v3325: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:12:36.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:12:35 smithi132 ceph-mon[35941]: pgmap v3326: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:12:36.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:12:35 smithi132 ceph-mon[35941]: 98 slow requests, 5 included below; oldest blocked for > 5815.744126 secs 2024-03-28T02:12:36.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:12:35 smithi132 ceph-mon[35941]: slow request 5811.613420 seconds old, received at 2024-03-28T00:35:43.489651+0000: internal op fragmentdir:mds.0:3873 currently failed to wrlock, waiting 2024-03-28T02:12:36.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:12:35 smithi132 ceph-mon[35941]: slow request 5810.775449 seconds old, received at 2024-03-28T00:35:44.327621+0000: internal op fragmentdir:mds.0:3937 currently failed to wrlock, waiting 2024-03-28T02:12:36.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:12:35 smithi132 ceph-mon[35941]: slow request 5811.639870 seconds old, received at 2024-03-28T00:35:43.463200+0000: internal op fragmentdir:mds.0:3810 currently failed to wrlock, waiting 2024-03-28T02:12:36.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:12:35 smithi132 ceph-mon[35941]: slow request 5811.626540 seconds old, received at 2024-03-28T00:35:43.476531+0000: internal op fragmentdir:mds.0:3842 currently failed to wrlock, waiting 2024-03-28T02:12:36.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:12:35 smithi132 ceph-mon[35941]: slow request 5811.639469 seconds old, received at 2024-03-28T00:35:43.463601+0000: internal op fragmentdir:mds.0:3811 currently failed to wrlock, waiting 2024-03-28T02:12:36.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:12:35 smithi165 ceph-mon[36302]: pgmap v3326: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:12:36.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:12:35 smithi165 ceph-mon[36302]: 98 slow requests, 5 included below; oldest blocked for > 5815.744126 secs 2024-03-28T02:12:36.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:12:35 smithi165 ceph-mon[36302]: slow request 5811.613420 seconds old, received at 2024-03-28T00:35:43.489651+0000: internal op fragmentdir:mds.0:3873 currently failed to wrlock, waiting 2024-03-28T02:12:36.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:12:35 smithi165 ceph-mon[36302]: slow request 5810.775449 seconds old, received at 2024-03-28T00:35:44.327621+0000: internal op fragmentdir:mds.0:3937 currently failed to wrlock, waiting 2024-03-28T02:12:36.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:12:35 smithi165 ceph-mon[36302]: slow request 5811.639870 seconds old, received at 2024-03-28T00:35:43.463200+0000: internal op fragmentdir:mds.0:3810 currently failed to wrlock, waiting 2024-03-28T02:12:36.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:12:35 smithi165 ceph-mon[36302]: slow request 5811.626540 seconds old, received at 2024-03-28T00:35:43.476531+0000: internal op fragmentdir:mds.0:3842 currently failed to wrlock, waiting 2024-03-28T02:12:36.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:12:35 smithi165 ceph-mon[36302]: slow request 5811.639469 seconds old, received at 2024-03-28T00:35:43.463601+0000: internal op fragmentdir:mds.0:3811 currently failed to wrlock, waiting 2024-03-28T02:12:36.392 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:12:35 smithi119 ceph-mon[31518]: pgmap v3326: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:12:36.392 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:12:35 smithi119 ceph-mon[31518]: 98 slow requests, 5 included below; oldest blocked for > 5815.744126 secs 2024-03-28T02:12:36.392 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:12:35 smithi119 ceph-mon[31518]: slow request 5811.613420 seconds old, received at 2024-03-28T00:35:43.489651+0000: internal op fragmentdir:mds.0:3873 currently failed to wrlock, waiting 2024-03-28T02:12:36.392 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:12:35 smithi119 ceph-mon[31518]: slow request 5810.775449 seconds old, received at 2024-03-28T00:35:44.327621+0000: internal op fragmentdir:mds.0:3937 currently failed to wrlock, waiting 2024-03-28T02:12:36.392 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:12:35 smithi119 ceph-mon[31518]: slow request 5811.639870 seconds old, received at 2024-03-28T00:35:43.463200+0000: internal op fragmentdir:mds.0:3810 currently failed to wrlock, waiting 2024-03-28T02:12:36.393 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:12:35 smithi119 ceph-mon[31518]: slow request 5811.626540 seconds old, received at 2024-03-28T00:35:43.476531+0000: internal op fragmentdir:mds.0:3842 currently failed to wrlock, waiting 2024-03-28T02:12:36.393 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:12:35 smithi119 ceph-mon[31518]: slow request 5811.639469 seconds old, received at 2024-03-28T00:35:43.463601+0000: internal op fragmentdir:mds.0:3811 currently failed to wrlock, waiting 2024-03-28T02:12:38.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:12:37 smithi132 ceph-mon[35941]: pgmap v3327: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:12:38.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:12:37 smithi165 ceph-mon[36302]: pgmap v3327: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:12:38.392 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:12:37 smithi119 ceph-mon[31518]: pgmap v3327: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:12:40.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:12:39 smithi132 ceph-mon[35941]: pgmap v3328: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:12:40.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:12:39 smithi165 ceph-mon[36302]: pgmap v3328: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:12:40.392 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:12:39 smithi119 ceph-mon[31518]: pgmap v3328: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:12:41.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:12:40 smithi132 ceph-mon[35941]: 98 slow requests, 5 included below; oldest blocked for > 5820.744187 secs 2024-03-28T02:12:41.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:12:40 smithi132 ceph-mon[35941]: slow request 5816.597094 seconds old, received at 2024-03-28T00:35:43.506038+0000: internal op fragmentdir:mds.0:3907 currently failed to wrlock, waiting 2024-03-28T02:12:41.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:12:40 smithi132 ceph-mon[35941]: slow request 5816.639096 seconds old, received at 2024-03-28T00:35:43.464036+0000: internal op fragmentdir:mds.0:3812 currently failed to wrlock, waiting 2024-03-28T02:12:41.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:12:40 smithi132 ceph-mon[35941]: slow request 5816.625718 seconds old, received at 2024-03-28T00:35:43.477414+0000: internal op fragmentdir:mds.0:3844 currently failed to wrlock, waiting 2024-03-28T02:12:41.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:12:40 smithi132 ceph-mon[35941]: slow request 5816.595367 seconds old, received at 2024-03-28T00:35:43.507766+0000: internal op fragmentdir:mds.0:3908 currently failed to wrlock, waiting 2024-03-28T02:12:41.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:12:40 smithi132 ceph-mon[35941]: slow request 5817.963767 seconds old, received at 2024-03-28T00:35:42.139365+0000: internal op fragmentdir:mds.0:3784 currently failed to wrlock, waiting 2024-03-28T02:12:41.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:12:40 smithi165 ceph-mon[36302]: 98 slow requests, 5 included below; oldest blocked for > 5820.744187 secs 2024-03-28T02:12:41.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:12:40 smithi165 ceph-mon[36302]: slow request 5816.597094 seconds old, received at 2024-03-28T00:35:43.506038+0000: internal op fragmentdir:mds.0:3907 currently failed to wrlock, waiting 2024-03-28T02:12:41.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:12:40 smithi165 ceph-mon[36302]: slow request 5816.639096 seconds old, received at 2024-03-28T00:35:43.464036+0000: internal op fragmentdir:mds.0:3812 currently failed to wrlock, waiting 2024-03-28T02:12:41.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:12:40 smithi165 ceph-mon[36302]: slow request 5816.625718 seconds old, received at 2024-03-28T00:35:43.477414+0000: internal op fragmentdir:mds.0:3844 currently failed to wrlock, waiting 2024-03-28T02:12:41.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:12:40 smithi165 ceph-mon[36302]: slow request 5816.595367 seconds old, received at 2024-03-28T00:35:43.507766+0000: internal op fragmentdir:mds.0:3908 currently failed to wrlock, waiting 2024-03-28T02:12:41.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:12:40 smithi165 ceph-mon[36302]: slow request 5817.963767 seconds old, received at 2024-03-28T00:35:42.139365+0000: internal op fragmentdir:mds.0:3784 currently failed to wrlock, waiting 2024-03-28T02:12:41.392 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:12:40 smithi119 ceph-mon[31518]: 98 slow requests, 5 included below; oldest blocked for > 5820.744187 secs 2024-03-28T02:12:41.392 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:12:40 smithi119 ceph-mon[31518]: slow request 5816.597094 seconds old, received at 2024-03-28T00:35:43.506038+0000: internal op fragmentdir:mds.0:3907 currently failed to wrlock, waiting 2024-03-28T02:12:41.392 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:12:40 smithi119 ceph-mon[31518]: slow request 5816.639096 seconds old, received at 2024-03-28T00:35:43.464036+0000: internal op fragmentdir:mds.0:3812 currently failed to wrlock, waiting 2024-03-28T02:12:41.392 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:12:40 smithi119 ceph-mon[31518]: slow request 5816.625718 seconds old, received at 2024-03-28T00:35:43.477414+0000: internal op fragmentdir:mds.0:3844 currently failed to wrlock, waiting 2024-03-28T02:12:41.392 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:12:40 smithi119 ceph-mon[31518]: slow request 5816.595367 seconds old, received at 2024-03-28T00:35:43.507766+0000: internal op fragmentdir:mds.0:3908 currently failed to wrlock, waiting 2024-03-28T02:12:41.393 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:12:40 smithi119 ceph-mon[31518]: slow request 5817.963767 seconds old, received at 2024-03-28T00:35:42.139365+0000: internal op fragmentdir:mds.0:3784 currently failed to wrlock, waiting 2024-03-28T02:12:42.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:12:41 smithi132 ceph-mon[35941]: pgmap v3329: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:12:42.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:12:41 smithi165 ceph-mon[36302]: pgmap v3329: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:12:42.392 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:12:41 smithi119 ceph-mon[31518]: pgmap v3329: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:12:44.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:12:43 smithi132 ceph-mon[35941]: pgmap v3330: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:12:44.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:12:43 smithi165 ceph-mon[36302]: pgmap v3330: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:12:44.392 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:12:43 smithi119 ceph-mon[31518]: pgmap v3330: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:12:46.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:12:45 smithi132 ceph-mon[35941]: pgmap v3331: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:12:46.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:12:45 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:12:46.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:12:45 smithi132 ceph-mon[35941]: 98 slow requests, 5 included below; oldest blocked for > 5825.744250 secs 2024-03-28T02:12:46.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:12:45 smithi132 ceph-mon[35941]: slow request 5821.638763 seconds old, received at 2024-03-28T00:35:43.464432+0000: internal op fragmentdir:mds.0:3813 currently failed to wrlock, waiting 2024-03-28T02:12:46.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:12:45 smithi132 ceph-mon[35941]: slow request 5821.625337 seconds old, received at 2024-03-28T00:35:43.477858+0000: internal op fragmentdir:mds.0:3845 currently failed to wrlock, waiting 2024-03-28T02:12:46.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:12:45 smithi132 ceph-mon[35941]: slow request 5821.611648 seconds old, received at 2024-03-28T00:35:43.491547+0000: internal op fragmentdir:mds.0:3877 currently failed to wrlock, waiting 2024-03-28T02:12:46.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:12:45 smithi132 ceph-mon[35941]: slow request 5821.638324 seconds old, received at 2024-03-28T00:35:43.464871+0000: internal op fragmentdir:mds.0:3814 currently failed to wrlock, waiting 2024-03-28T02:12:46.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:12:45 smithi132 ceph-mon[35941]: slow request 5821.593371 seconds old, received at 2024-03-28T00:35:43.509824+0000: internal op fragmentdir:mds.0:3910 currently failed to wrlock, waiting 2024-03-28T02:12:46.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:12:45 smithi165 ceph-mon[36302]: pgmap v3331: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:12:46.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:12:45 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:12:46.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:12:45 smithi165 ceph-mon[36302]: 98 slow requests, 5 included below; oldest blocked for > 5825.744250 secs 2024-03-28T02:12:46.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:12:45 smithi165 ceph-mon[36302]: slow request 5821.638763 seconds old, received at 2024-03-28T00:35:43.464432+0000: internal op fragmentdir:mds.0:3813 currently failed to wrlock, waiting 2024-03-28T02:12:46.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:12:45 smithi165 ceph-mon[36302]: slow request 5821.625337 seconds old, received at 2024-03-28T00:35:43.477858+0000: internal op fragmentdir:mds.0:3845 currently failed to wrlock, waiting 2024-03-28T02:12:46.370 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:12:45 smithi165 ceph-mon[36302]: slow request 5821.611648 seconds old, received at 2024-03-28T00:35:43.491547+0000: internal op fragmentdir:mds.0:3877 currently failed to wrlock, waiting 2024-03-28T02:12:46.370 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:12:45 smithi165 ceph-mon[36302]: slow request 5821.638324 seconds old, received at 2024-03-28T00:35:43.464871+0000: internal op fragmentdir:mds.0:3814 currently failed to wrlock, waiting 2024-03-28T02:12:46.370 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:12:45 smithi165 ceph-mon[36302]: slow request 5821.593371 seconds old, received at 2024-03-28T00:35:43.509824+0000: internal op fragmentdir:mds.0:3910 currently failed to wrlock, waiting 2024-03-28T02:12:46.392 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:12:45 smithi119 ceph-mon[31518]: pgmap v3331: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:12:46.392 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:12:45 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:12:46.392 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:12:45 smithi119 ceph-mon[31518]: 98 slow requests, 5 included below; oldest blocked for > 5825.744250 secs 2024-03-28T02:12:46.392 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:12:45 smithi119 ceph-mon[31518]: slow request 5821.638763 seconds old, received at 2024-03-28T00:35:43.464432+0000: internal op fragmentdir:mds.0:3813 currently failed to wrlock, waiting 2024-03-28T02:12:46.392 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:12:45 smithi119 ceph-mon[31518]: slow request 5821.625337 seconds old, received at 2024-03-28T00:35:43.477858+0000: internal op fragmentdir:mds.0:3845 currently failed to wrlock, waiting 2024-03-28T02:12:46.392 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:12:45 smithi119 ceph-mon[31518]: slow request 5821.611648 seconds old, received at 2024-03-28T00:35:43.491547+0000: internal op fragmentdir:mds.0:3877 currently failed to wrlock, waiting 2024-03-28T02:12:46.392 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:12:45 smithi119 ceph-mon[31518]: slow request 5821.638324 seconds old, received at 2024-03-28T00:35:43.464871+0000: internal op fragmentdir:mds.0:3814 currently failed to wrlock, waiting 2024-03-28T02:12:46.392 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:12:45 smithi119 ceph-mon[31518]: slow request 5821.593371 seconds old, received at 2024-03-28T00:35:43.509824+0000: internal op fragmentdir:mds.0:3910 currently failed to wrlock, waiting 2024-03-28T02:12:47.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:12:46 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:12:47.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:12:46 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:12:47.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:12:46 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:12:47.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:12:46 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:12:47.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:12:46 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:12:47.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:12:46 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:12:47.392 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:12:46 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:12:47.392 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:12:46 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:12:47.392 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:12:46 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:12:48.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:12:47 smithi165 ceph-mon[36302]: pgmap v3332: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T02:12:48.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:12:47 smithi165 ceph-mon[36302]: pgmap v3333: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:12:48.392 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:12:47 smithi119 ceph-mon[31518]: pgmap v3332: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T02:12:48.392 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:12:47 smithi119 ceph-mon[31518]: pgmap v3333: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:12:48.439 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:12:47 smithi132 ceph-mon[35941]: pgmap v3332: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T02:12:48.439 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:12:47 smithi132 ceph-mon[35941]: pgmap v3333: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:12:50.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:12:49 smithi165 ceph-mon[36302]: pgmap v3334: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:12:50.391 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:12:49 smithi119 ceph-mon[31518]: pgmap v3334: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:12:50.439 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:12:49 smithi132 ceph-mon[35941]: pgmap v3334: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:12:51.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:12:50 smithi165 ceph-mon[36302]: 98 slow requests, 5 included below; oldest blocked for > 5830.744340 secs 2024-03-28T02:12:51.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:12:50 smithi165 ceph-mon[36302]: slow request 5826.678359 seconds old, received at 2024-03-28T00:35:43.424926+0000: internal op fragmentdir:mds.0:3787 currently failed to wrlock, waiting 2024-03-28T02:12:51.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:12:50 smithi165 ceph-mon[36302]: slow request 5826.637974 seconds old, received at 2024-03-28T00:35:43.465310+0000: internal op fragmentdir:mds.0:3815 currently failed to wrlock, waiting 2024-03-28T02:12:51.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:12:50 smithi165 ceph-mon[36302]: slow request 5826.610809 seconds old, received at 2024-03-28T00:35:43.492476+0000: internal op fragmentdir:mds.0:3879 currently failed to wrlock, waiting 2024-03-28T02:12:51.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:12:50 smithi165 ceph-mon[36302]: slow request 5826.637580 seconds old, received at 2024-03-28T00:35:43.465705+0000: internal op fragmentdir:mds.0:3816 currently failed to wrlock, waiting 2024-03-28T02:12:51.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:12:50 smithi165 ceph-mon[36302]: slow request 5826.624019 seconds old, received at 2024-03-28T00:35:43.479266+0000: internal op fragmentdir:mds.0:3848 currently failed to wrlock, waiting 2024-03-28T02:12:51.393 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:12:50 smithi119 ceph-mon[31518]: 98 slow requests, 5 included below; oldest blocked for > 5830.744340 secs 2024-03-28T02:12:51.393 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:12:50 smithi119 ceph-mon[31518]: slow request 5826.678359 seconds old, received at 2024-03-28T00:35:43.424926+0000: internal op fragmentdir:mds.0:3787 currently failed to wrlock, waiting 2024-03-28T02:12:51.393 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:12:50 smithi119 ceph-mon[31518]: slow request 5826.637974 seconds old, received at 2024-03-28T00:35:43.465310+0000: internal op fragmentdir:mds.0:3815 currently failed to wrlock, waiting 2024-03-28T02:12:51.394 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:12:50 smithi119 ceph-mon[31518]: slow request 5826.610809 seconds old, received at 2024-03-28T00:35:43.492476+0000: internal op fragmentdir:mds.0:3879 currently failed to wrlock, waiting 2024-03-28T02:12:51.394 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:12:50 smithi119 ceph-mon[31518]: slow request 5826.637580 seconds old, received at 2024-03-28T00:35:43.465705+0000: internal op fragmentdir:mds.0:3816 currently failed to wrlock, waiting 2024-03-28T02:12:51.394 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:12:50 smithi119 ceph-mon[31518]: slow request 5826.624019 seconds old, received at 2024-03-28T00:35:43.479266+0000: internal op fragmentdir:mds.0:3848 currently failed to wrlock, waiting 2024-03-28T02:12:51.439 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:12:50 smithi132 ceph-mon[35941]: 98 slow requests, 5 included below; oldest blocked for > 5830.744340 secs 2024-03-28T02:12:51.439 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:12:50 smithi132 ceph-mon[35941]: slow request 5826.678359 seconds old, received at 2024-03-28T00:35:43.424926+0000: internal op fragmentdir:mds.0:3787 currently failed to wrlock, waiting 2024-03-28T02:12:51.439 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:12:50 smithi132 ceph-mon[35941]: slow request 5826.637974 seconds old, received at 2024-03-28T00:35:43.465310+0000: internal op fragmentdir:mds.0:3815 currently failed to wrlock, waiting 2024-03-28T02:12:51.439 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:12:50 smithi132 ceph-mon[35941]: slow request 5826.610809 seconds old, received at 2024-03-28T00:35:43.492476+0000: internal op fragmentdir:mds.0:3879 currently failed to wrlock, waiting 2024-03-28T02:12:51.439 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:12:50 smithi132 ceph-mon[35941]: slow request 5826.637580 seconds old, received at 2024-03-28T00:35:43.465705+0000: internal op fragmentdir:mds.0:3816 currently failed to wrlock, waiting 2024-03-28T02:12:51.439 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:12:50 smithi132 ceph-mon[35941]: slow request 5826.624019 seconds old, received at 2024-03-28T00:35:43.479266+0000: internal op fragmentdir:mds.0:3848 currently failed to wrlock, waiting 2024-03-28T02:12:52.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:12:51 smithi165 ceph-mon[36302]: pgmap v3335: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:12:52.392 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:12:51 smithi119 ceph-mon[31518]: pgmap v3335: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:12:52.439 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:12:51 smithi132 ceph-mon[35941]: pgmap v3335: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:12:53.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:12:52 smithi165 ceph-mon[36302]: pgmap v3336: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:12:53.392 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:12:52 smithi119 ceph-mon[31518]: pgmap v3336: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:12:53.439 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:12:52 smithi132 ceph-mon[35941]: pgmap v3336: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:12:55.642 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:12:55 smithi119 ceph-mon[31518]: pgmap v3337: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:12:55.689 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:12:55 smithi132 ceph-mon[35941]: pgmap v3337: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:12:55.869 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:12:55 smithi165 ceph-mon[36302]: pgmap v3337: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:12:56.642 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:12:56 smithi119 ceph-mon[31518]: 98 slow requests, 5 included below; oldest blocked for > 5835.744402 secs 2024-03-28T02:12:56.642 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:12:56 smithi119 ceph-mon[31518]: slow request 5831.610385 seconds old, received at 2024-03-28T00:35:43.492962+0000: internal op fragmentdir:mds.0:3880 currently failed to wrlock, waiting 2024-03-28T02:12:56.642 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:12:56 smithi119 ceph-mon[31518]: slow request 5832.955275 seconds old, received at 2024-03-28T00:35:42.148071+0000: client_request(client.25264:33954 create owner_uid=1000, owner_gid=1301 #0x10000000271/file8309 2024-03-28T00:35:42.146337+0000 caller_uid=1000, caller_gid=1301{6,36,1000,1301,}) currently failed to xlock, waiting 2024-03-28T02:12:56.643 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:12:56 smithi119 ceph-mon[31518]: slow request 5831.637246 seconds old, received at 2024-03-28T00:35:43.466100+0000: internal op fragmentdir:mds.0:3817 currently failed to wrlock, waiting 2024-03-28T02:12:56.643 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:12:56 smithi119 ceph-mon[31518]: slow request 5831.610056 seconds old, received at 2024-03-28T00:35:43.493290+0000: internal op fragmentdir:mds.0:3881 currently failed to wrlock, waiting 2024-03-28T02:12:56.643 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:12:56 smithi119 ceph-mon[31518]: slow request 5831.636799 seconds old, received at 2024-03-28T00:35:43.466548+0000: internal op fragmentdir:mds.0:3818 currently failed to wrlock, waiting 2024-03-28T02:12:56.689 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:12:56 smithi132 ceph-mon[35941]: 98 slow requests, 5 included below; oldest blocked for > 5835.744402 secs 2024-03-28T02:12:56.689 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:12:56 smithi132 ceph-mon[35941]: slow request 5831.610385 seconds old, received at 2024-03-28T00:35:43.492962+0000: internal op fragmentdir:mds.0:3880 currently failed to wrlock, waiting 2024-03-28T02:12:56.689 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:12:56 smithi132 ceph-mon[35941]: slow request 5832.955275 seconds old, received at 2024-03-28T00:35:42.148071+0000: client_request(client.25264:33954 create owner_uid=1000, owner_gid=1301 #0x10000000271/file8309 2024-03-28T00:35:42.146337+0000 caller_uid=1000, caller_gid=1301{6,36,1000,1301,}) currently failed to xlock, waiting 2024-03-28T02:12:56.689 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:12:56 smithi132 ceph-mon[35941]: slow request 5831.637246 seconds old, received at 2024-03-28T00:35:43.466100+0000: internal op fragmentdir:mds.0:3817 currently failed to wrlock, waiting 2024-03-28T02:12:56.689 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:12:56 smithi132 ceph-mon[35941]: slow request 5831.610056 seconds old, received at 2024-03-28T00:35:43.493290+0000: internal op fragmentdir:mds.0:3881 currently failed to wrlock, waiting 2024-03-28T02:12:56.689 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:12:56 smithi132 ceph-mon[35941]: slow request 5831.636799 seconds old, received at 2024-03-28T00:35:43.466548+0000: internal op fragmentdir:mds.0:3818 currently failed to wrlock, waiting 2024-03-28T02:12:56.869 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:12:56 smithi165 ceph-mon[36302]: 98 slow requests, 5 included below; oldest blocked for > 5835.744402 secs 2024-03-28T02:12:56.869 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:12:56 smithi165 ceph-mon[36302]: slow request 5831.610385 seconds old, received at 2024-03-28T00:35:43.492962+0000: internal op fragmentdir:mds.0:3880 currently failed to wrlock, waiting 2024-03-28T02:12:56.869 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:12:56 smithi165 ceph-mon[36302]: slow request 5832.955275 seconds old, received at 2024-03-28T00:35:42.148071+0000: client_request(client.25264:33954 create owner_uid=1000, owner_gid=1301 #0x10000000271/file8309 2024-03-28T00:35:42.146337+0000 caller_uid=1000, caller_gid=1301{6,36,1000,1301,}) currently failed to xlock, waiting 2024-03-28T02:12:56.869 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:12:56 smithi165 ceph-mon[36302]: slow request 5831.637246 seconds old, received at 2024-03-28T00:35:43.466100+0000: internal op fragmentdir:mds.0:3817 currently failed to wrlock, waiting 2024-03-28T02:12:56.869 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:12:56 smithi165 ceph-mon[36302]: slow request 5831.610056 seconds old, received at 2024-03-28T00:35:43.493290+0000: internal op fragmentdir:mds.0:3881 currently failed to wrlock, waiting 2024-03-28T02:12:56.869 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:12:56 smithi165 ceph-mon[36302]: slow request 5831.636799 seconds old, received at 2024-03-28T00:35:43.466548+0000: internal op fragmentdir:mds.0:3818 currently failed to wrlock, waiting 2024-03-28T02:12:57.642 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:12:57 smithi119 ceph-mon[31518]: pgmap v3338: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T02:12:57.689 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:12:57 smithi132 ceph-mon[35941]: pgmap v3338: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T02:12:57.869 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:12:57 smithi165 ceph-mon[36302]: pgmap v3338: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T02:12:59.642 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:12:59 smithi119 ceph-mon[31518]: pgmap v3339: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:12:59.689 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:12:59 smithi132 ceph-mon[35941]: pgmap v3339: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:12:59.869 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:12:59 smithi165 ceph-mon[36302]: pgmap v3339: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:13:01.671 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:13:01 smithi165 ceph-mon[36302]: 98 slow requests, 5 included below; oldest blocked for > 5840.744463 secs 2024-03-28T02:13:01.671 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:13:01 smithi165 ceph-mon[36302]: slow request 5836.623216 seconds old, received at 2024-03-28T00:35:43.480192+0000: internal op fragmentdir:mds.0:3850 currently failed to wrlock, waiting 2024-03-28T02:13:01.671 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:13:01 smithi165 ceph-mon[36302]: slow request 5836.654664 seconds old, received at 2024-03-28T00:35:43.448744+0000: internal op fragmentdir:mds.0:3788 currently failed to wrlock, waiting 2024-03-28T02:13:01.671 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:13:01 smithi165 ceph-mon[36302]: slow request 5836.636452 seconds old, received at 2024-03-28T00:35:43.466955+0000: internal op fragmentdir:mds.0:3819 currently failed to wrlock, waiting 2024-03-28T02:13:01.672 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:13:01 smithi165 ceph-mon[36302]: slow request 5836.622741 seconds old, received at 2024-03-28T00:35:43.480667+0000: internal op fragmentdir:mds.0:3851 currently failed to wrlock, waiting 2024-03-28T02:13:01.672 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:13:01 smithi165 ceph-mon[36302]: slow request 5836.609569 seconds old, received at 2024-03-28T00:35:43.493839+0000: internal op fragmentdir:mds.0:3883 currently failed to wrlock, waiting 2024-03-28T02:13:01.672 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:13:01 smithi165 ceph-mon[36302]: pgmap v3340: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:13:01.689 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:13:01 smithi132 ceph-mon[35941]: 98 slow requests, 5 included below; oldest blocked for > 5840.744463 secs 2024-03-28T02:13:01.689 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:13:01 smithi132 ceph-mon[35941]: slow request 5836.623216 seconds old, received at 2024-03-28T00:35:43.480192+0000: internal op fragmentdir:mds.0:3850 currently failed to wrlock, waiting 2024-03-28T02:13:01.689 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:13:01 smithi132 ceph-mon[35941]: slow request 5836.654664 seconds old, received at 2024-03-28T00:35:43.448744+0000: internal op fragmentdir:mds.0:3788 currently failed to wrlock, waiting 2024-03-28T02:13:01.689 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:13:01 smithi132 ceph-mon[35941]: slow request 5836.636452 seconds old, received at 2024-03-28T00:35:43.466955+0000: internal op fragmentdir:mds.0:3819 currently failed to wrlock, waiting 2024-03-28T02:13:01.689 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:13:01 smithi132 ceph-mon[35941]: slow request 5836.622741 seconds old, received at 2024-03-28T00:35:43.480667+0000: internal op fragmentdir:mds.0:3851 currently failed to wrlock, waiting 2024-03-28T02:13:01.689 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:13:01 smithi132 ceph-mon[35941]: slow request 5836.609569 seconds old, received at 2024-03-28T00:35:43.493839+0000: internal op fragmentdir:mds.0:3883 currently failed to wrlock, waiting 2024-03-28T02:13:01.689 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:13:01 smithi132 ceph-mon[35941]: pgmap v3340: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:13:01.892 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:13:01 smithi119 ceph-mon[31518]: 98 slow requests, 5 included below; oldest blocked for > 5840.744463 secs 2024-03-28T02:13:01.892 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:13:01 smithi119 ceph-mon[31518]: slow request 5836.623216 seconds old, received at 2024-03-28T00:35:43.480192+0000: internal op fragmentdir:mds.0:3850 currently failed to wrlock, waiting 2024-03-28T02:13:01.892 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:13:01 smithi119 ceph-mon[31518]: slow request 5836.654664 seconds old, received at 2024-03-28T00:35:43.448744+0000: internal op fragmentdir:mds.0:3788 currently failed to wrlock, waiting 2024-03-28T02:13:01.892 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:13:01 smithi119 ceph-mon[31518]: slow request 5836.636452 seconds old, received at 2024-03-28T00:35:43.466955+0000: internal op fragmentdir:mds.0:3819 currently failed to wrlock, waiting 2024-03-28T02:13:01.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:13:01 smithi119 ceph-mon[31518]: slow request 5836.622741 seconds old, received at 2024-03-28T00:35:43.480667+0000: internal op fragmentdir:mds.0:3851 currently failed to wrlock, waiting 2024-03-28T02:13:01.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:13:01 smithi119 ceph-mon[31518]: slow request 5836.609569 seconds old, received at 2024-03-28T00:35:43.493839+0000: internal op fragmentdir:mds.0:3883 currently failed to wrlock, waiting 2024-03-28T02:13:01.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:13:01 smithi119 ceph-mon[31518]: pgmap v3340: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:13:03.689 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:13:03 smithi132 ceph-mon[35941]: pgmap v3341: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:13:03.869 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:13:03 smithi165 ceph-mon[36302]: pgmap v3341: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:13:03.892 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:13:03 smithi119 ceph-mon[31518]: pgmap v3341: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:13:05.642 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:13:05 smithi119 ceph-mon[31518]: pgmap v3342: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:13:05.689 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:13:05 smithi132 ceph-mon[35941]: pgmap v3342: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:13:05.869 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:13:05 smithi165 ceph-mon[36302]: pgmap v3342: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:13:06.689 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:13:06 smithi132 ceph-mon[35941]: 98 slow requests, 5 included below; oldest blocked for > 5845.744551 secs 2024-03-28T02:13:06.689 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:13:06 smithi132 ceph-mon[35941]: slow request 5841.654335 seconds old, received at 2024-03-28T00:35:43.449161+0000: internal op fragmentdir:mds.0:3789 currently failed to wrlock, waiting 2024-03-28T02:13:06.689 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:13:06 smithi132 ceph-mon[35941]: slow request 5841.636104 seconds old, received at 2024-03-28T00:35:43.467391+0000: internal op fragmentdir:mds.0:3820 currently failed to wrlock, waiting 2024-03-28T02:13:06.689 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:13:06 smithi132 ceph-mon[35941]: slow request 5841.622354 seconds old, received at 2024-03-28T00:35:43.481141+0000: internal op fragmentdir:mds.0:3852 currently failed to wrlock, waiting 2024-03-28T02:13:06.689 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:13:06 smithi132 ceph-mon[35941]: slow request 5841.580562 seconds old, received at 2024-03-28T00:35:43.522934+0000: internal op fragmentdir:mds.0:3916 currently failed to wrlock, waiting 2024-03-28T02:13:06.689 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:13:06 smithi132 ceph-mon[35941]: slow request 5841.653875 seconds old, received at 2024-03-28T00:35:43.449621+0000: internal op fragmentdir:mds.0:3790 currently failed to wrlock, waiting 2024-03-28T02:13:06.869 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:13:06 smithi165 ceph-mon[36302]: 98 slow requests, 5 included below; oldest blocked for > 5845.744551 secs 2024-03-28T02:13:06.869 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:13:06 smithi165 ceph-mon[36302]: slow request 5841.654335 seconds old, received at 2024-03-28T00:35:43.449161+0000: internal op fragmentdir:mds.0:3789 currently failed to wrlock, waiting 2024-03-28T02:13:06.869 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:13:06 smithi165 ceph-mon[36302]: slow request 5841.636104 seconds old, received at 2024-03-28T00:35:43.467391+0000: internal op fragmentdir:mds.0:3820 currently failed to wrlock, waiting 2024-03-28T02:13:06.869 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:13:06 smithi165 ceph-mon[36302]: slow request 5841.622354 seconds old, received at 2024-03-28T00:35:43.481141+0000: internal op fragmentdir:mds.0:3852 currently failed to wrlock, waiting 2024-03-28T02:13:06.869 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:13:06 smithi165 ceph-mon[36302]: slow request 5841.580562 seconds old, received at 2024-03-28T00:35:43.522934+0000: internal op fragmentdir:mds.0:3916 currently failed to wrlock, waiting 2024-03-28T02:13:06.869 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:13:06 smithi165 ceph-mon[36302]: slow request 5841.653875 seconds old, received at 2024-03-28T00:35:43.449621+0000: internal op fragmentdir:mds.0:3790 currently failed to wrlock, waiting 2024-03-28T02:13:06.892 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:13:06 smithi119 ceph-mon[31518]: 98 slow requests, 5 included below; oldest blocked for > 5845.744551 secs 2024-03-28T02:13:06.892 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:13:06 smithi119 ceph-mon[31518]: slow request 5841.654335 seconds old, received at 2024-03-28T00:35:43.449161+0000: internal op fragmentdir:mds.0:3789 currently failed to wrlock, waiting 2024-03-28T02:13:06.892 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:13:06 smithi119 ceph-mon[31518]: slow request 5841.636104 seconds old, received at 2024-03-28T00:35:43.467391+0000: internal op fragmentdir:mds.0:3820 currently failed to wrlock, waiting 2024-03-28T02:13:06.892 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:13:06 smithi119 ceph-mon[31518]: slow request 5841.622354 seconds old, received at 2024-03-28T00:35:43.481141+0000: internal op fragmentdir:mds.0:3852 currently failed to wrlock, waiting 2024-03-28T02:13:06.892 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:13:06 smithi119 ceph-mon[31518]: slow request 5841.580562 seconds old, received at 2024-03-28T00:35:43.522934+0000: internal op fragmentdir:mds.0:3916 currently failed to wrlock, waiting 2024-03-28T02:13:06.892 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:13:06 smithi119 ceph-mon[31518]: slow request 5841.653875 seconds old, received at 2024-03-28T00:35:43.449621+0000: internal op fragmentdir:mds.0:3790 currently failed to wrlock, waiting 2024-03-28T02:13:07.689 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:13:07 smithi132 ceph-mon[35941]: pgmap v3343: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:13:07.869 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:13:07 smithi165 ceph-mon[36302]: pgmap v3343: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:13:07.892 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:13:07 smithi119 ceph-mon[31518]: pgmap v3343: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:13:09.689 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:13:09 smithi132 ceph-mon[35941]: pgmap v3344: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:13:09.869 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:13:09 smithi165 ceph-mon[36302]: pgmap v3344: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:13:09.892 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:13:09 smithi119 ceph-mon[31518]: pgmap v3344: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:13:11.689 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:13:11 smithi132 ceph-mon[35941]: 98 slow requests, 5 included below; oldest blocked for > 5850.744639 secs 2024-03-28T02:13:11.689 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:13:11 smithi132 ceph-mon[35941]: slow request 5846.635789 seconds old, received at 2024-03-28T00:35:43.467794+0000: internal op fragmentdir:mds.0:3821 currently failed to wrlock, waiting 2024-03-28T02:13:11.689 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:13:11 smithi132 ceph-mon[35941]: slow request 5846.608961 seconds old, received at 2024-03-28T00:35:43.494622+0000: internal op fragmentdir:mds.0:3885 currently failed to wrlock, waiting 2024-03-28T02:13:11.689 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:13:11 smithi132 ceph-mon[35941]: slow request 5846.653420 seconds old, received at 2024-03-28T00:35:43.450163+0000: internal op fragmentdir:mds.0:3791 currently failed to wrlock, waiting 2024-03-28T02:13:11.689 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:13:11 smithi132 ceph-mon[35941]: slow request 5846.621524 seconds old, received at 2024-03-28T00:35:43.482059+0000: internal op fragmentdir:mds.0:3854 currently failed to wrlock, waiting 2024-03-28T02:13:11.689 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:13:11 smithi132 ceph-mon[35941]: slow request 5846.608509 seconds old, received at 2024-03-28T00:35:43.495074+0000: internal op fragmentdir:mds.0:3886 currently failed to wrlock, waiting 2024-03-28T02:13:11.690 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:13:11 smithi132 ceph-mon[35941]: pgmap v3345: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:13:11.869 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:13:11 smithi165 ceph-mon[36302]: 98 slow requests, 5 included below; oldest blocked for > 5850.744639 secs 2024-03-28T02:13:11.869 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:13:11 smithi165 ceph-mon[36302]: slow request 5846.635789 seconds old, received at 2024-03-28T00:35:43.467794+0000: internal op fragmentdir:mds.0:3821 currently failed to wrlock, waiting 2024-03-28T02:13:11.869 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:13:11 smithi165 ceph-mon[36302]: slow request 5846.608961 seconds old, received at 2024-03-28T00:35:43.494622+0000: internal op fragmentdir:mds.0:3885 currently failed to wrlock, waiting 2024-03-28T02:13:11.869 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:13:11 smithi165 ceph-mon[36302]: slow request 5846.653420 seconds old, received at 2024-03-28T00:35:43.450163+0000: internal op fragmentdir:mds.0:3791 currently failed to wrlock, waiting 2024-03-28T02:13:11.869 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:13:11 smithi165 ceph-mon[36302]: slow request 5846.621524 seconds old, received at 2024-03-28T00:35:43.482059+0000: internal op fragmentdir:mds.0:3854 currently failed to wrlock, waiting 2024-03-28T02:13:11.869 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:13:11 smithi165 ceph-mon[36302]: slow request 5846.608509 seconds old, received at 2024-03-28T00:35:43.495074+0000: internal op fragmentdir:mds.0:3886 currently failed to wrlock, waiting 2024-03-28T02:13:11.869 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:13:11 smithi165 ceph-mon[36302]: pgmap v3345: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:13:11.892 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:13:11 smithi119 ceph-mon[31518]: 98 slow requests, 5 included below; oldest blocked for > 5850.744639 secs 2024-03-28T02:13:11.892 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:13:11 smithi119 ceph-mon[31518]: slow request 5846.635789 seconds old, received at 2024-03-28T00:35:43.467794+0000: internal op fragmentdir:mds.0:3821 currently failed to wrlock, waiting 2024-03-28T02:13:11.892 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:13:11 smithi119 ceph-mon[31518]: slow request 5846.608961 seconds old, received at 2024-03-28T00:35:43.494622+0000: internal op fragmentdir:mds.0:3885 currently failed to wrlock, waiting 2024-03-28T02:13:11.892 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:13:11 smithi119 ceph-mon[31518]: slow request 5846.653420 seconds old, received at 2024-03-28T00:35:43.450163+0000: internal op fragmentdir:mds.0:3791 currently failed to wrlock, waiting 2024-03-28T02:13:11.892 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:13:11 smithi119 ceph-mon[31518]: slow request 5846.621524 seconds old, received at 2024-03-28T00:35:43.482059+0000: internal op fragmentdir:mds.0:3854 currently failed to wrlock, waiting 2024-03-28T02:13:11.892 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:13:11 smithi119 ceph-mon[31518]: slow request 5846.608509 seconds old, received at 2024-03-28T00:35:43.495074+0000: internal op fragmentdir:mds.0:3886 currently failed to wrlock, waiting 2024-03-28T02:13:11.892 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:13:11 smithi119 ceph-mon[31518]: pgmap v3345: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:13:13.689 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:13:13 smithi132 ceph-mon[35941]: pgmap v3346: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:13:13.869 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:13:13 smithi165 ceph-mon[36302]: pgmap v3346: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:13:13.892 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:13:13 smithi119 ceph-mon[31518]: pgmap v3346: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:13:15.689 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:13:15 smithi132 ceph-mon[35941]: pgmap v3347: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:13:15.869 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:13:15 smithi165 ceph-mon[36302]: pgmap v3347: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:13:15.892 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:13:15 smithi119 ceph-mon[31518]: pgmap v3347: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:13:16.869 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:13:16 smithi165 ceph-mon[36302]: 98 slow requests, 5 included below; oldest blocked for > 5855.744727 secs 2024-03-28T02:13:16.869 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:13:16 smithi165 ceph-mon[36302]: slow request 5851.579617 seconds old, received at 2024-03-28T00:35:43.524054+0000: internal op fragmentdir:mds.0:3918 currently failed to wrlock, waiting 2024-03-28T02:13:16.869 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:13:16 smithi165 ceph-mon[36302]: slow request 5851.653059 seconds old, received at 2024-03-28T00:35:43.450613+0000: internal op fragmentdir:mds.0:3792 currently failed to wrlock, waiting 2024-03-28T02:13:16.869 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:13:16 smithi165 ceph-mon[36302]: slow request 5851.635283 seconds old, received at 2024-03-28T00:35:43.468389+0000: internal op fragmentdir:mds.0:3823 currently failed to wrlock, waiting 2024-03-28T02:13:16.869 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:13:16 smithi165 ceph-mon[36302]: slow request 5851.652566 seconds old, received at 2024-03-28T00:35:43.451105+0000: internal op fragmentdir:mds.0:3793 currently failed to wrlock, waiting 2024-03-28T02:13:16.869 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:13:16 smithi165 ceph-mon[36302]: slow request 5851.607724 seconds old, received at 2024-03-28T00:35:43.495948+0000: internal op fragmentdir:mds.0:3888 currently failed to wrlock, waiting 2024-03-28T02:13:16.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:13:16 smithi119 ceph-mon[31518]: 98 slow requests, 5 included below; oldest blocked for > 5855.744727 secs 2024-03-28T02:13:16.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:13:16 smithi119 ceph-mon[31518]: slow request 5851.579617 seconds old, received at 2024-03-28T00:35:43.524054+0000: internal op fragmentdir:mds.0:3918 currently failed to wrlock, waiting 2024-03-28T02:13:16.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:13:16 smithi119 ceph-mon[31518]: slow request 5851.653059 seconds old, received at 2024-03-28T00:35:43.450613+0000: internal op fragmentdir:mds.0:3792 currently failed to wrlock, waiting 2024-03-28T02:13:16.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:13:16 smithi119 ceph-mon[31518]: slow request 5851.635283 seconds old, received at 2024-03-28T00:35:43.468389+0000: internal op fragmentdir:mds.0:3823 currently failed to wrlock, waiting 2024-03-28T02:13:16.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:13:16 smithi119 ceph-mon[31518]: slow request 5851.652566 seconds old, received at 2024-03-28T00:35:43.451105+0000: internal op fragmentdir:mds.0:3793 currently failed to wrlock, waiting 2024-03-28T02:13:16.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:13:16 smithi119 ceph-mon[31518]: slow request 5851.607724 seconds old, received at 2024-03-28T00:35:43.495948+0000: internal op fragmentdir:mds.0:3888 currently failed to wrlock, waiting 2024-03-28T02:13:16.939 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:13:16 smithi132 ceph-mon[35941]: 98 slow requests, 5 included below; oldest blocked for > 5855.744727 secs 2024-03-28T02:13:16.939 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:13:16 smithi132 ceph-mon[35941]: slow request 5851.579617 seconds old, received at 2024-03-28T00:35:43.524054+0000: internal op fragmentdir:mds.0:3918 currently failed to wrlock, waiting 2024-03-28T02:13:16.939 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:13:16 smithi132 ceph-mon[35941]: slow request 5851.653059 seconds old, received at 2024-03-28T00:35:43.450613+0000: internal op fragmentdir:mds.0:3792 currently failed to wrlock, waiting 2024-03-28T02:13:16.939 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:13:16 smithi132 ceph-mon[35941]: slow request 5851.635283 seconds old, received at 2024-03-28T00:35:43.468389+0000: internal op fragmentdir:mds.0:3823 currently failed to wrlock, waiting 2024-03-28T02:13:16.939 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:13:16 smithi132 ceph-mon[35941]: slow request 5851.652566 seconds old, received at 2024-03-28T00:35:43.451105+0000: internal op fragmentdir:mds.0:3793 currently failed to wrlock, waiting 2024-03-28T02:13:16.939 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:13:16 smithi132 ceph-mon[35941]: slow request 5851.607724 seconds old, received at 2024-03-28T00:35:43.495948+0000: internal op fragmentdir:mds.0:3888 currently failed to wrlock, waiting 2024-03-28T02:13:17.869 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:13:17 smithi165 ceph-mon[36302]: pgmap v3348: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:13:17.892 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:13:17 smithi119 ceph-mon[31518]: pgmap v3348: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:13:17.939 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:13:17 smithi132 ceph-mon[35941]: pgmap v3348: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:13:19.869 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:13:19 smithi165 ceph-mon[36302]: pgmap v3349: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:13:19.892 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:13:19 smithi119 ceph-mon[31518]: pgmap v3349: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:13:19.939 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:13:19 smithi132 ceph-mon[35941]: pgmap v3349: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:13:21.869 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:13:21 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5860.744817 secs 2024-03-28T02:13:21.869 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:13:21 smithi165 ceph-mon[36302]: pgmap v3350: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:13:21.892 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:13:21 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5860.744817 secs 2024-03-28T02:13:21.892 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:13:21 smithi119 ceph-mon[31518]: pgmap v3350: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:13:21.939 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:13:21 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5860.744817 secs 2024-03-28T02:13:21.939 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:13:21 smithi132 ceph-mon[35941]: pgmap v3350: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:13:23.869 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:13:23 smithi165 ceph-mon[36302]: pgmap v3351: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:13:23.892 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:13:23 smithi119 ceph-mon[31518]: pgmap v3351: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:13:23.939 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:13:23 smithi132 ceph-mon[35941]: pgmap v3351: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:13:25.869 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:13:25 smithi165 ceph-mon[36302]: pgmap v3352: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:13:25.892 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:13:25 smithi119 ceph-mon[31518]: pgmap v3352: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:13:25.939 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:13:25 smithi132 ceph-mon[35941]: pgmap v3352: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:13:26.869 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:13:26 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5865.744880 secs 2024-03-28T02:13:26.892 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:13:26 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5865.744880 secs 2024-03-28T02:13:26.939 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:13:26 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5865.744880 secs 2024-03-28T02:13:27.869 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:13:27 smithi165 ceph-mon[36302]: pgmap v3353: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:13:27.892 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:13:27 smithi119 ceph-mon[31518]: pgmap v3353: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:13:27.939 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:13:27 smithi132 ceph-mon[35941]: pgmap v3353: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:13:29.869 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:13:29 smithi165 ceph-mon[36302]: pgmap v3354: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:13:29.892 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:13:29 smithi119 ceph-mon[31518]: pgmap v3354: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:13:29.939 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:13:29 smithi132 ceph-mon[35941]: pgmap v3354: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:13:31.869 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:13:31 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5870.744942 secs 2024-03-28T02:13:31.869 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:13:31 smithi165 ceph-mon[36302]: pgmap v3355: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:13:31.892 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:13:31 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5870.744942 secs 2024-03-28T02:13:31.892 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:13:31 smithi119 ceph-mon[31518]: pgmap v3355: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:13:31.939 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:13:31 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5870.744942 secs 2024-03-28T02:13:31.939 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:13:31 smithi132 ceph-mon[35941]: pgmap v3355: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:13:33.869 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:13:33 smithi165 ceph-mon[36302]: pgmap v3356: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:13:33.892 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:13:33 smithi119 ceph-mon[31518]: pgmap v3356: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:13:33.939 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:13:33 smithi132 ceph-mon[35941]: pgmap v3356: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:13:35.869 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:13:35 smithi165 ceph-mon[36302]: pgmap v3357: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:13:35.892 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:13:35 smithi119 ceph-mon[31518]: pgmap v3357: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:13:35.939 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:13:35 smithi132 ceph-mon[35941]: pgmap v3357: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:13:36.869 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:13:36 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5875.745005 secs 2024-03-28T02:13:36.892 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:13:36 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5875.745005 secs 2024-03-28T02:13:36.939 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:13:36 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5875.745005 secs 2024-03-28T02:13:37.869 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:13:37 smithi165 ceph-mon[36302]: pgmap v3358: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:13:37.892 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:13:37 smithi119 ceph-mon[31518]: pgmap v3358: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:13:37.939 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:13:37 smithi132 ceph-mon[35941]: pgmap v3358: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:13:39.869 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:13:39 smithi165 ceph-mon[36302]: pgmap v3359: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:13:39.892 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:13:39 smithi119 ceph-mon[31518]: pgmap v3359: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:13:39.939 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:13:39 smithi132 ceph-mon[35941]: pgmap v3359: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:13:41.869 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:13:41 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5880.745066 secs 2024-03-28T02:13:41.869 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:13:41 smithi165 ceph-mon[36302]: pgmap v3360: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:13:41.892 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:13:41 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5880.745066 secs 2024-03-28T02:13:41.892 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:13:41 smithi119 ceph-mon[31518]: pgmap v3360: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:13:41.939 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:13:41 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5880.745066 secs 2024-03-28T02:13:41.939 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:13:41 smithi132 ceph-mon[35941]: pgmap v3360: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:13:43.869 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:13:43 smithi165 ceph-mon[36302]: pgmap v3361: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:13:43.892 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:13:43 smithi119 ceph-mon[31518]: pgmap v3361: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:13:43.939 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:13:43 smithi132 ceph-mon[35941]: pgmap v3361: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:13:45.869 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:13:45 smithi165 ceph-mon[36302]: pgmap v3362: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:13:45.892 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:13:45 smithi119 ceph-mon[31518]: pgmap v3362: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:13:45.939 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:13:45 smithi132 ceph-mon[35941]: pgmap v3362: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:13:46.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:13:46 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5885.745153 secs 2024-03-28T02:13:46.869 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:13:46 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5885.745153 secs 2024-03-28T02:13:46.891 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:13:46 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5885.745153 secs 2024-03-28T02:13:47.869 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:13:47 smithi165 ceph-mon[36302]: pgmap v3363: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:13:47.870 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:13:47 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:13:47.892 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:13:47 smithi119 ceph-mon[31518]: pgmap v3363: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:13:47.892 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:13:47 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:13:47.939 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:13:47 smithi132 ceph-mon[35941]: pgmap v3363: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:13:47.939 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:13:47 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:13:48.869 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:13:48 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:13:48.869 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:13:48 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:13:48.869 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:13:48 smithi165 ceph-mon[36302]: pgmap v3364: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T02:13:48.869 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:13:48 smithi165 ceph-mon[36302]: pgmap v3365: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:13:48.869 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:13:48 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:13:48.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:13:48 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:13:48.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:13:48 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:13:48.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:13:48 smithi119 ceph-mon[31518]: pgmap v3364: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T02:13:48.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:13:48 smithi119 ceph-mon[31518]: pgmap v3365: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:13:48.894 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:13:48 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:13:48.939 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:13:48 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:13:48.939 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:13:48 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:13:48.939 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:13:48 smithi132 ceph-mon[35941]: pgmap v3364: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T02:13:48.939 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:13:48 smithi132 ceph-mon[35941]: pgmap v3365: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:13:48.939 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:13:48 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:13:50.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:13:50 smithi165 ceph-mon[36302]: pgmap v3366: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:13:50.392 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:13:50 smithi119 ceph-mon[31518]: pgmap v3366: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:13:50.439 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:13:50 smithi132 ceph-mon[35941]: pgmap v3366: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:13:51.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:13:51 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5890.745197 secs 2024-03-28T02:13:51.392 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:13:51 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5890.745197 secs 2024-03-28T02:13:51.439 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:13:51 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5890.745197 secs 2024-03-28T02:13:52.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:13:52 smithi165 ceph-mon[36302]: pgmap v3367: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:13:52.392 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:13:52 smithi119 ceph-mon[31518]: pgmap v3367: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:13:52.439 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:13:52 smithi132 ceph-mon[35941]: pgmap v3367: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:13:54.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:13:54 smithi165 ceph-mon[36302]: pgmap v3368: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-28T02:13:54.392 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:13:54 smithi119 ceph-mon[31518]: pgmap v3368: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-28T02:13:54.439 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:13:54 smithi132 ceph-mon[35941]: pgmap v3368: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-28T02:13:55.892 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:13:55 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5895.745259 secs 2024-03-28T02:13:55.939 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:13:55 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5895.745259 secs 2024-03-28T02:13:56.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:13:55 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5895.745259 secs 2024-03-28T02:13:56.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:13:56 smithi119 ceph-mon[31518]: pgmap v3369: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T02:13:56.939 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:13:56 smithi132 ceph-mon[35941]: pgmap v3369: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T02:13:57.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:13:56 smithi165 ceph-mon[36302]: pgmap v3369: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T02:13:58.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:13:58 smithi165 ceph-mon[36302]: pgmap v3370: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-28T02:13:58.392 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:13:58 smithi119 ceph-mon[31518]: pgmap v3370: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-28T02:13:58.439 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:13:58 smithi132 ceph-mon[35941]: pgmap v3370: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-28T02:14:00.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:14:00 smithi165 ceph-mon[36302]: pgmap v3371: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:14:00.392 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:14:00 smithi119 ceph-mon[31518]: pgmap v3371: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:14:00.439 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:14:00 smithi132 ceph-mon[35941]: pgmap v3371: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:14:01.270 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:14:01 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5900.745320 secs 2024-03-28T02:14:01.294 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:14:01 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5900.745320 secs 2024-03-28T02:14:01.439 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:14:01 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5900.745320 secs 2024-03-28T02:14:02.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:14:02 smithi165 ceph-mon[36302]: pgmap v3372: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T02:14:02.392 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:14:02 smithi119 ceph-mon[31518]: pgmap v3372: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T02:14:02.439 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:14:02 smithi132 ceph-mon[35941]: pgmap v3372: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T02:14:04.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:14:04 smithi165 ceph-mon[36302]: pgmap v3373: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T02:14:04.392 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:14:04 smithi119 ceph-mon[31518]: pgmap v3373: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T02:14:04.439 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:14:04 smithi132 ceph-mon[35941]: pgmap v3373: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T02:14:05.892 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:14:05 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5905.745404 secs 2024-03-28T02:14:05.939 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:14:05 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5905.745404 secs 2024-03-28T02:14:06.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:14:05 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5905.745404 secs 2024-03-28T02:14:06.892 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:14:06 smithi119 ceph-mon[31518]: pgmap v3374: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:14:06.939 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:14:06 smithi132 ceph-mon[35941]: pgmap v3374: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:14:07.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:14:06 smithi165 ceph-mon[36302]: pgmap v3374: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:14:08.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:14:08 smithi165 ceph-mon[36302]: pgmap v3375: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:14:08.392 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:14:08 smithi119 ceph-mon[31518]: pgmap v3375: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:14:08.439 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:14:08 smithi132 ceph-mon[35941]: pgmap v3375: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:14:10.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:14:10 smithi165 ceph-mon[36302]: pgmap v3376: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:14:10.392 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:14:10 smithi119 ceph-mon[31518]: pgmap v3376: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:14:10.439 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:14:10 smithi132 ceph-mon[35941]: pgmap v3376: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:14:11.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:14:11 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5910.745490 secs 2024-03-28T02:14:11.392 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:14:11 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5910.745490 secs 2024-03-28T02:14:11.439 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:14:11 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5910.745490 secs 2024-03-28T02:14:12.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:14:12 smithi165 ceph-mon[36302]: pgmap v3377: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:14:12.392 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:14:12 smithi119 ceph-mon[31518]: pgmap v3377: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:14:12.439 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:14:12 smithi132 ceph-mon[35941]: pgmap v3377: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:14:14.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:14:14 smithi165 ceph-mon[36302]: pgmap v3378: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:14:14.392 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:14:14 smithi119 ceph-mon[31518]: pgmap v3378: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:14:14.439 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:14:14 smithi132 ceph-mon[35941]: pgmap v3378: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:14:15.892 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:14:15 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5915.745549 secs 2024-03-28T02:14:15.939 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:14:15 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5915.745549 secs 2024-03-28T02:14:16.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:14:15 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5915.745549 secs 2024-03-28T02:14:16.892 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:14:16 smithi119 ceph-mon[31518]: pgmap v3379: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:14:16.939 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:14:16 smithi132 ceph-mon[35941]: pgmap v3379: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:14:17.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:14:16 smithi165 ceph-mon[36302]: pgmap v3379: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:14:18.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:14:18 smithi165 ceph-mon[36302]: pgmap v3380: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:14:18.392 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:14:18 smithi119 ceph-mon[31518]: pgmap v3380: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:14:18.439 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:14:18 smithi132 ceph-mon[35941]: pgmap v3380: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:14:20.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:14:20 smithi165 ceph-mon[36302]: pgmap v3381: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:14:20.392 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:14:20 smithi119 ceph-mon[31518]: pgmap v3381: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:14:20.439 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:14:20 smithi132 ceph-mon[35941]: pgmap v3381: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:14:21.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:14:21 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5920.745609 secs 2024-03-28T02:14:21.391 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:14:21 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5920.745609 secs 2024-03-28T02:14:21.439 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:14:21 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5920.745609 secs 2024-03-28T02:14:22.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:14:22 smithi165 ceph-mon[36302]: pgmap v3382: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:14:22.392 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:14:22 smithi119 ceph-mon[31518]: pgmap v3382: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:14:22.439 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:14:22 smithi132 ceph-mon[35941]: pgmap v3382: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:14:24.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:14:24 smithi165 ceph-mon[36302]: pgmap v3383: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:14:24.392 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:14:24 smithi119 ceph-mon[31518]: pgmap v3383: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:14:24.439 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:14:24 smithi132 ceph-mon[35941]: pgmap v3383: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:14:25.892 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:14:25 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5925.745647 secs 2024-03-28T02:14:25.939 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:14:25 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5925.745647 secs 2024-03-28T02:14:26.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:14:25 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5925.745647 secs 2024-03-28T02:14:26.892 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:14:26 smithi119 ceph-mon[31518]: pgmap v3384: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:14:26.939 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:14:26 smithi132 ceph-mon[35941]: pgmap v3384: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:14:27.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:14:26 smithi165 ceph-mon[36302]: pgmap v3384: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:14:28.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:14:28 smithi165 ceph-mon[36302]: pgmap v3385: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:14:28.391 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:14:28 smithi119 ceph-mon[31518]: pgmap v3385: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:14:28.439 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:14:28 smithi132 ceph-mon[35941]: pgmap v3385: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:14:30.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:14:30 smithi165 ceph-mon[36302]: pgmap v3386: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:14:30.392 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:14:30 smithi119 ceph-mon[31518]: pgmap v3386: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:14:30.439 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:14:30 smithi132 ceph-mon[35941]: pgmap v3386: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:14:31.293 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:14:31 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5930.745709 secs 2024-03-28T02:14:31.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:14:31 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5930.745709 secs 2024-03-28T02:14:31.439 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:14:31 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5930.745709 secs 2024-03-28T02:14:32.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:14:32 smithi165 ceph-mon[36302]: pgmap v3387: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:14:32.392 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:14:32 smithi119 ceph-mon[31518]: pgmap v3387: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:14:32.439 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:14:32 smithi132 ceph-mon[35941]: pgmap v3387: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:14:34.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:14:34 smithi165 ceph-mon[36302]: pgmap v3388: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:14:34.391 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:14:34 smithi119 ceph-mon[31518]: pgmap v3388: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:14:34.439 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:14:34 smithi132 ceph-mon[35941]: pgmap v3388: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:14:35.892 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:14:35 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5935.745752 secs 2024-03-28T02:14:35.939 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:14:35 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5935.745752 secs 2024-03-28T02:14:36.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:14:35 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5935.745752 secs 2024-03-28T02:14:36.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:14:36 smithi119 ceph-mon[31518]: pgmap v3389: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:14:36.939 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:14:36 smithi132 ceph-mon[35941]: pgmap v3389: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:14:37.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:14:36 smithi165 ceph-mon[36302]: pgmap v3389: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:14:38.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:14:38 smithi119 ceph-mon[31518]: pgmap v3390: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:14:38.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:14:38 smithi165 ceph-mon[36302]: pgmap v3390: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:14:38.439 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:14:38 smithi132 ceph-mon[35941]: pgmap v3390: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:14:40.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:14:40 smithi165 ceph-mon[36302]: pgmap v3391: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:14:40.392 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:14:40 smithi119 ceph-mon[31518]: pgmap v3391: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:14:40.439 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:14:40 smithi132 ceph-mon[35941]: pgmap v3391: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:14:41.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:14:41 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5940.745811 secs 2024-03-28T02:14:41.392 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:14:41 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5940.745811 secs 2024-03-28T02:14:41.439 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:14:41 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5940.745811 secs 2024-03-28T02:14:42.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:14:42 smithi165 ceph-mon[36302]: pgmap v3392: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:14:42.392 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:14:42 smithi119 ceph-mon[31518]: pgmap v3392: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:14:42.439 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:14:42 smithi132 ceph-mon[35941]: pgmap v3392: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:14:44.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:14:44 smithi165 ceph-mon[36302]: pgmap v3393: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:14:44.391 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:14:44 smithi119 ceph-mon[31518]: pgmap v3393: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:14:44.439 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:14:44 smithi132 ceph-mon[35941]: pgmap v3393: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:14:45.892 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:14:45 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5945.745824 secs 2024-03-28T02:14:45.939 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:14:45 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5945.745824 secs 2024-03-28T02:14:46.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:14:45 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5945.745824 secs 2024-03-28T02:14:46.892 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:14:46 smithi119 ceph-mon[31518]: pgmap v3394: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:14:46.939 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:14:46 smithi132 ceph-mon[35941]: pgmap v3394: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:14:47.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:14:46 smithi165 ceph-mon[36302]: pgmap v3394: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:14:48.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:14:48 smithi165 ceph-mon[36302]: pgmap v3395: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:14:48.391 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:14:48 smithi119 ceph-mon[31518]: pgmap v3395: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:14:48.439 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:14:48 smithi132 ceph-mon[35941]: pgmap v3395: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:14:49.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:14:49 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:14:49.392 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:14:49 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:14:49.439 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:14:49 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:14:50.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:14:50 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:14:50.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:14:50 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:14:50.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:14:50 smithi165 ceph-mon[36302]: pgmap v3396: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T02:14:50.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:14:50 smithi165 ceph-mon[36302]: pgmap v3397: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:14:50.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:14:50 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:14:50.392 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:14:50 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:14:50.392 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:14:50 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:14:50.392 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:14:50 smithi119 ceph-mon[31518]: pgmap v3396: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T02:14:50.392 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:14:50 smithi119 ceph-mon[31518]: pgmap v3397: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:14:50.392 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:14:50 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:14:50.439 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:14:50 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:14:50.439 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:14:50 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:14:50.439 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:14:50 smithi132 ceph-mon[35941]: pgmap v3396: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T02:14:50.439 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:14:50 smithi132 ceph-mon[35941]: pgmap v3397: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:14:50.439 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:14:50 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:14:51.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:14:51 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5950.745894 secs 2024-03-28T02:14:51.391 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:14:51 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5950.745894 secs 2024-03-28T02:14:51.439 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:14:51 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5950.745894 secs 2024-03-28T02:14:52.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:14:52 smithi165 ceph-mon[36302]: pgmap v3398: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:14:52.391 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:14:52 smithi119 ceph-mon[31518]: pgmap v3398: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:14:52.439 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:14:52 smithi132 ceph-mon[35941]: pgmap v3398: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:14:54.869 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:14:54 smithi165 ceph-mon[36302]: pgmap v3399: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:14:54.892 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:14:54 smithi119 ceph-mon[31518]: pgmap v3399: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:14:54.939 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:14:54 smithi132 ceph-mon[35941]: pgmap v3399: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:14:55.869 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:14:55 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5955.745938 secs 2024-03-28T02:14:55.891 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:14:55 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5955.745938 secs 2024-03-28T02:14:55.939 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:14:55 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5955.745938 secs 2024-03-28T02:14:56.869 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:14:56 smithi165 ceph-mon[36302]: pgmap v3400: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:14:56.892 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:14:56 smithi119 ceph-mon[31518]: pgmap v3400: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:14:56.939 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:14:56 smithi132 ceph-mon[35941]: pgmap v3400: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:14:58.892 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:14:58 smithi119 ceph-mon[31518]: pgmap v3401: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:14:58.942 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:14:58 smithi132 ceph-mon[35941]: pgmap v3401: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:14:59.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:14:58 smithi165 ceph-mon[36302]: pgmap v3401: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:15:00.892 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:15:00 smithi119 ceph-mon[31518]: pgmap v3402: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-28T02:15:00.892 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:15:00 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5960.746002 secs 2024-03-28T02:15:00.939 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:15:00 smithi132 ceph-mon[35941]: pgmap v3402: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-28T02:15:00.939 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:15:00 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5960.746002 secs 2024-03-28T02:15:01.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:15:00 smithi165 ceph-mon[36302]: pgmap v3402: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-28T02:15:01.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:15:00 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5960.746002 secs 2024-03-28T02:15:02.892 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:15:02 smithi119 ceph-mon[31518]: pgmap v3403: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T02:15:02.921 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:15:02 smithi165 ceph-mon[36302]: pgmap v3403: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T02:15:02.939 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:15:02 smithi132 ceph-mon[35941]: pgmap v3403: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T02:15:04.642 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:15:04 smithi119 ceph-mon[31518]: pgmap v3404: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T02:15:04.689 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:15:04 smithi132 ceph-mon[35941]: pgmap v3404: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T02:15:04.870 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:15:04 smithi165 ceph-mon[36302]: pgmap v3404: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T02:15:05.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:15:05 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5965.746062 secs 2024-03-28T02:15:05.939 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:15:05 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5965.746062 secs 2024-03-28T02:15:06.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:15:05 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5965.746062 secs 2024-03-28T02:15:06.892 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:15:06 smithi119 ceph-mon[31518]: pgmap v3405: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:15:06.939 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:15:06 smithi132 ceph-mon[35941]: pgmap v3405: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:15:07.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:15:06 smithi165 ceph-mon[36302]: pgmap v3405: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:15:08.892 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:15:08 smithi119 ceph-mon[31518]: pgmap v3406: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T02:15:08.939 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:15:08 smithi132 ceph-mon[35941]: pgmap v3406: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T02:15:09.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:15:08 smithi165 ceph-mon[36302]: pgmap v3406: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T02:15:10.939 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:15:10 smithi132 ceph-mon[35941]: pgmap v3407: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:15:10.939 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:15:10 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5970.746125 secs 2024-03-28T02:15:11.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:15:10 smithi165 ceph-mon[36302]: pgmap v3407: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:15:11.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:15:10 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5970.746125 secs 2024-03-28T02:15:11.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:15:10 smithi119 ceph-mon[31518]: pgmap v3407: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:15:11.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:15:10 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5970.746125 secs 2024-03-28T02:15:12.939 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:15:12 smithi132 ceph-mon[35941]: pgmap v3408: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:15:13.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:15:12 smithi165 ceph-mon[36302]: pgmap v3408: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:15:13.141 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:15:12 smithi119 ceph-mon[31518]: pgmap v3408: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:15:14.939 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:15:14 smithi132 ceph-mon[35941]: pgmap v3409: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:15:15.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:15:14 smithi165 ceph-mon[36302]: pgmap v3409: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:15:15.141 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:15:14 smithi119 ceph-mon[31518]: pgmap v3409: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:15:15.939 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:15:15 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5975.746185 secs 2024-03-28T02:15:16.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:15:15 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5975.746185 secs 2024-03-28T02:15:16.141 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:15:15 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5975.746185 secs 2024-03-28T02:15:16.939 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:15:16 smithi132 ceph-mon[35941]: pgmap v3410: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:15:17.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:15:16 smithi165 ceph-mon[36302]: pgmap v3410: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:15:17.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:15:16 smithi119 ceph-mon[31518]: pgmap v3410: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:15:18.939 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:15:18 smithi132 ceph-mon[35941]: pgmap v3411: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:15:19.118 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:15:18 smithi165 ceph-mon[36302]: pgmap v3411: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:15:19.141 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:15:18 smithi119 ceph-mon[31518]: pgmap v3411: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:15:20.939 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:15:20 smithi132 ceph-mon[35941]: pgmap v3412: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:15:20.939 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:15:20 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5980.746272 secs 2024-03-28T02:15:21.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:15:20 smithi165 ceph-mon[36302]: pgmap v3412: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:15:21.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:15:20 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5980.746272 secs 2024-03-28T02:15:21.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:15:20 smithi119 ceph-mon[31518]: pgmap v3412: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:15:21.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:15:20 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5980.746272 secs 2024-03-28T02:15:22.939 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:15:22 smithi132 ceph-mon[35941]: pgmap v3413: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:15:23.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:15:22 smithi165 ceph-mon[36302]: pgmap v3413: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:15:23.141 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:15:22 smithi119 ceph-mon[31518]: pgmap v3413: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:15:24.939 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:15:24 smithi132 ceph-mon[35941]: pgmap v3414: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:15:24.952 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:15:24 smithi119 ceph-mon[31518]: pgmap v3414: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:15:25.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:15:24 smithi165 ceph-mon[36302]: pgmap v3414: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:15:26.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:15:25 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5985.746362 secs 2024-03-28T02:15:26.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:15:25 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5985.746362 secs 2024-03-28T02:15:26.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:15:25 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5985.746362 secs 2024-03-28T02:15:27.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:15:26 smithi165 ceph-mon[36302]: pgmap v3415: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:15:27.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:15:26 smithi119 ceph-mon[31518]: pgmap v3415: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:15:27.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:15:26 smithi132 ceph-mon[35941]: pgmap v3415: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:15:29.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:15:28 smithi165 ceph-mon[36302]: pgmap v3416: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:15:29.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:15:28 smithi119 ceph-mon[31518]: pgmap v3416: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:15:29.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:15:28 smithi132 ceph-mon[35941]: pgmap v3416: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:15:31.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:15:30 smithi165 ceph-mon[36302]: pgmap v3417: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:15:31.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:15:30 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5990.746450 secs 2024-03-28T02:15:31.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:15:30 smithi119 ceph-mon[31518]: pgmap v3417: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:15:31.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:15:30 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5990.746450 secs 2024-03-28T02:15:31.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:15:30 smithi132 ceph-mon[35941]: pgmap v3417: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:15:31.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:15:30 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5990.746450 secs 2024-03-28T02:15:33.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:15:32 smithi165 ceph-mon[36302]: pgmap v3418: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:15:33.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:15:32 smithi119 ceph-mon[31518]: pgmap v3418: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:15:33.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:15:32 smithi132 ceph-mon[35941]: pgmap v3418: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:15:35.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:15:34 smithi165 ceph-mon[36302]: pgmap v3419: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:15:35.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:15:34 smithi119 ceph-mon[31518]: pgmap v3419: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:15:35.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:15:34 smithi132 ceph-mon[35941]: pgmap v3419: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:15:36.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:15:35 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 5995.746511 secs 2024-03-28T02:15:36.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:15:35 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 5995.746511 secs 2024-03-28T02:15:36.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:15:35 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 5995.746511 secs 2024-03-28T02:15:37.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:15:36 smithi165 ceph-mon[36302]: pgmap v3420: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:15:37.141 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:15:36 smithi119 ceph-mon[31518]: pgmap v3420: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:15:37.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:15:36 smithi132 ceph-mon[35941]: pgmap v3420: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:15:39.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:15:38 smithi165 ceph-mon[36302]: pgmap v3421: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:15:39.141 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:15:38 smithi119 ceph-mon[31518]: pgmap v3421: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:15:39.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:15:38 smithi132 ceph-mon[35941]: pgmap v3421: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:15:41.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:15:40 smithi165 ceph-mon[36302]: pgmap v3422: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:15:41.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:15:40 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6000.746600 secs 2024-03-28T02:15:41.141 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:15:40 smithi119 ceph-mon[31518]: pgmap v3422: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:15:41.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:15:40 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6000.746600 secs 2024-03-28T02:15:41.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:15:40 smithi132 ceph-mon[35941]: pgmap v3422: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:15:41.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:15:40 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6000.746600 secs 2024-03-28T02:15:43.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:15:42 smithi165 ceph-mon[36302]: pgmap v3423: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:15:43.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:15:42 smithi119 ceph-mon[31518]: pgmap v3423: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:15:43.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:15:42 smithi132 ceph-mon[35941]: pgmap v3423: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:15:45.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:15:44 smithi165 ceph-mon[36302]: pgmap v3424: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:15:45.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:15:44 smithi119 ceph-mon[31518]: pgmap v3424: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:15:45.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:15:44 smithi132 ceph-mon[35941]: pgmap v3424: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:15:46.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:15:45 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6005.746678 secs 2024-03-28T02:15:46.118 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:15:45 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6005.746678 secs 2024-03-28T02:15:46.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:15:45 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6005.746678 secs 2024-03-28T02:15:47.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:15:46 smithi165 ceph-mon[36302]: pgmap v3425: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:15:47.141 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:15:46 smithi119 ceph-mon[31518]: pgmap v3425: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:15:47.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:15:46 smithi132 ceph-mon[35941]: pgmap v3425: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:15:49.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:15:48 smithi165 ceph-mon[36302]: pgmap v3426: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:15:49.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:15:48 smithi119 ceph-mon[31518]: pgmap v3426: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:15:49.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:15:48 smithi132 ceph-mon[35941]: pgmap v3426: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:15:51.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:15:50 smithi165 ceph-mon[36302]: pgmap v3427: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:15:51.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:15:50 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:15:51.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:15:50 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6010.746697 secs 2024-03-28T02:15:51.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:15:50 smithi119 ceph-mon[31518]: pgmap v3427: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:15:51.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:15:50 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:15:51.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:15:50 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6010.746697 secs 2024-03-28T02:15:51.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:15:50 smithi132 ceph-mon[35941]: pgmap v3427: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:15:51.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:15:50 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:15:51.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:15:50 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6010.746697 secs 2024-03-28T02:15:52.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:15:51 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:15:52.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:15:51 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:15:52.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:15:51 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:15:52.141 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:15:51 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:15:52.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:15:51 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:15:52.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:15:51 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:15:52.188 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:15:51 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:15:52.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:15:51 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:15:52.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:15:51 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:15:53.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:15:52 smithi165 ceph-mon[36302]: pgmap v3428: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T02:15:53.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:15:52 smithi165 ceph-mon[36302]: pgmap v3429: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:15:53.141 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:15:52 smithi119 ceph-mon[31518]: pgmap v3428: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T02:15:53.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:15:52 smithi119 ceph-mon[31518]: pgmap v3429: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:15:53.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:15:52 smithi132 ceph-mon[35941]: pgmap v3428: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T02:15:53.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:15:52 smithi132 ceph-mon[35941]: pgmap v3429: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:15:55.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:15:54 smithi165 ceph-mon[36302]: pgmap v3430: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:15:55.141 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:15:54 smithi119 ceph-mon[31518]: pgmap v3430: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:15:55.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:15:54 smithi132 ceph-mon[35941]: pgmap v3430: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:15:56.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:15:55 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6015.746758 secs 2024-03-28T02:15:56.141 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:15:55 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6015.746758 secs 2024-03-28T02:15:56.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:15:55 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6015.746758 secs 2024-03-28T02:15:57.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:15:56 smithi165 ceph-mon[36302]: pgmap v3431: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:15:57.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:15:56 smithi119 ceph-mon[31518]: pgmap v3431: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:15:57.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:15:56 smithi132 ceph-mon[35941]: pgmap v3431: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:15:59.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:15:58 smithi165 ceph-mon[36302]: pgmap v3432: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:15:59.141 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:15:58 smithi119 ceph-mon[31518]: pgmap v3432: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:15:59.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:15:58 smithi132 ceph-mon[35941]: pgmap v3432: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:16:01.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:16:00 smithi165 ceph-mon[36302]: pgmap v3433: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 5 op/s 2024-03-28T02:16:01.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:16:00 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6020.746846 secs 2024-03-28T02:16:01.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:16:00 smithi119 ceph-mon[31518]: pgmap v3433: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 5 op/s 2024-03-28T02:16:01.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:16:00 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6020.746846 secs 2024-03-28T02:16:01.188 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:16:00 smithi132 ceph-mon[35941]: pgmap v3433: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 5 op/s 2024-03-28T02:16:01.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:16:00 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6020.746846 secs 2024-03-28T02:16:03.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:16:02 smithi165 ceph-mon[36302]: pgmap v3434: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T02:16:03.141 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:16:02 smithi119 ceph-mon[31518]: pgmap v3434: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T02:16:03.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:16:02 smithi132 ceph-mon[35941]: pgmap v3434: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T02:16:05.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:16:04 smithi165 ceph-mon[36302]: pgmap v3435: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:16:05.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:16:04 smithi119 ceph-mon[31518]: pgmap v3435: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:16:05.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:16:04 smithi132 ceph-mon[35941]: pgmap v3435: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:16:06.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:16:05 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6025.746934 secs 2024-03-28T02:16:06.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:16:05 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6025.746934 secs 2024-03-28T02:16:06.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:16:05 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6025.746934 secs 2024-03-28T02:16:07.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:16:06 smithi165 ceph-mon[36302]: pgmap v3436: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:16:07.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:16:06 smithi119 ceph-mon[31518]: pgmap v3436: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:16:07.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:16:06 smithi132 ceph-mon[35941]: pgmap v3436: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:16:09.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:16:08 smithi165 ceph-mon[36302]: pgmap v3437: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:16:09.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:16:08 smithi119 ceph-mon[31518]: pgmap v3437: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:16:09.188 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:16:08 smithi132 ceph-mon[35941]: pgmap v3437: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:16:11.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:16:10 smithi165 ceph-mon[36302]: pgmap v3438: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-28T02:16:11.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:16:10 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6030.746973 secs 2024-03-28T02:16:11.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:16:10 smithi119 ceph-mon[31518]: pgmap v3438: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-28T02:16:11.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:16:10 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6030.746973 secs 2024-03-28T02:16:11.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:16:10 smithi132 ceph-mon[35941]: pgmap v3438: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-28T02:16:11.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:16:10 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6030.746973 secs 2024-03-28T02:16:13.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:16:12 smithi165 ceph-mon[36302]: pgmap v3439: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-03-28T02:16:13.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:16:12 smithi119 ceph-mon[31518]: pgmap v3439: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-03-28T02:16:13.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:16:12 smithi132 ceph-mon[35941]: pgmap v3439: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-03-28T02:16:15.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:16:14 smithi165 ceph-mon[36302]: pgmap v3440: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:16:15.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:16:14 smithi119 ceph-mon[31518]: pgmap v3440: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:16:15.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:16:14 smithi132 ceph-mon[35941]: pgmap v3440: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:16:16.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:16:15 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6035.747011 secs 2024-03-28T02:16:16.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:16:15 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6035.747011 secs 2024-03-28T02:16:16.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:16:15 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6035.747011 secs 2024-03-28T02:16:17.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:16:16 smithi119 ceph-mon[31518]: pgmap v3441: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:16:17.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:16:16 smithi132 ceph-mon[35941]: pgmap v3441: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:16:17.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:16:16 smithi165 ceph-mon[36302]: pgmap v3441: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:16:19.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:16:18 smithi119 ceph-mon[31518]: pgmap v3442: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:16:19.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:16:18 smithi132 ceph-mon[35941]: pgmap v3442: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:16:19.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:16:18 smithi165 ceph-mon[36302]: pgmap v3442: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:16:21.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:16:20 smithi119 ceph-mon[31518]: pgmap v3443: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-28T02:16:21.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:16:20 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6040.747074 secs 2024-03-28T02:16:21.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:16:20 smithi132 ceph-mon[35941]: pgmap v3443: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-28T02:16:21.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:16:20 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6040.747074 secs 2024-03-28T02:16:21.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:16:20 smithi165 ceph-mon[36302]: pgmap v3443: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-28T02:16:21.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:16:20 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6040.747074 secs 2024-03-28T02:16:23.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:16:22 smithi119 ceph-mon[31518]: pgmap v3444: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-03-28T02:16:23.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:16:22 smithi132 ceph-mon[35941]: pgmap v3444: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-03-28T02:16:23.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:16:22 smithi165 ceph-mon[36302]: pgmap v3444: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-03-28T02:16:25.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:16:24 smithi119 ceph-mon[31518]: pgmap v3445: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:16:25.188 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:16:24 smithi132 ceph-mon[35941]: pgmap v3445: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:16:25.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:16:24 smithi165 ceph-mon[36302]: pgmap v3445: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:16:26.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:16:25 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6045.747150 secs 2024-03-28T02:16:26.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:16:25 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6045.747150 secs 2024-03-28T02:16:26.391 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:16:25 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6045.747150 secs 2024-03-28T02:16:27.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:16:26 smithi132 ceph-mon[35941]: pgmap v3446: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:16:27.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:16:26 smithi165 ceph-mon[36302]: pgmap v3446: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:16:27.392 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:16:26 smithi119 ceph-mon[31518]: pgmap v3446: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:16:29.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:16:28 smithi132 ceph-mon[35941]: pgmap v3447: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:16:29.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:16:28 smithi165 ceph-mon[36302]: pgmap v3447: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:16:29.391 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:16:28 smithi119 ceph-mon[31518]: pgmap v3447: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:16:31.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:16:30 smithi132 ceph-mon[35941]: pgmap v3448: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-28T02:16:31.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:16:30 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6050.747239 secs 2024-03-28T02:16:31.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:16:30 smithi165 ceph-mon[36302]: pgmap v3448: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-28T02:16:31.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:16:30 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6050.747239 secs 2024-03-28T02:16:31.392 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:16:30 smithi119 ceph-mon[31518]: pgmap v3448: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-28T02:16:31.392 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:16:30 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6050.747239 secs 2024-03-28T02:16:33.171 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:16:32 smithi165 ceph-mon[36302]: pgmap v3449: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-03-28T02:16:33.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:16:32 smithi132 ceph-mon[35941]: pgmap v3449: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-03-28T02:16:33.194 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:16:32 smithi119 ceph-mon[31518]: pgmap v3449: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-03-28T02:16:35.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:16:34 smithi132 ceph-mon[35941]: pgmap v3450: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:16:35.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:16:34 smithi165 ceph-mon[36302]: pgmap v3450: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:16:35.391 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:16:34 smithi119 ceph-mon[31518]: pgmap v3450: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:16:36.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:16:35 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6055.747330 secs 2024-03-28T02:16:36.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:16:35 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6055.747330 secs 2024-03-28T02:16:36.392 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:16:35 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6055.747330 secs 2024-03-28T02:16:37.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:16:36 smithi132 ceph-mon[35941]: pgmap v3451: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:16:37.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:16:36 smithi165 ceph-mon[36302]: pgmap v3451: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:16:37.392 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:16:36 smithi119 ceph-mon[31518]: pgmap v3451: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:16:39.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:16:38 smithi165 ceph-mon[36302]: pgmap v3452: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:16:39.392 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:16:38 smithi119 ceph-mon[31518]: pgmap v3452: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:16:39.439 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:16:38 smithi132 ceph-mon[35941]: pgmap v3452: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:16:41.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:16:40 smithi165 ceph-mon[36302]: pgmap v3453: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-28T02:16:41.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:16:40 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6060.747414 secs 2024-03-28T02:16:41.392 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:16:40 smithi119 ceph-mon[31518]: pgmap v3453: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-28T02:16:41.392 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:16:40 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6060.747414 secs 2024-03-28T02:16:41.438 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:16:40 smithi132 ceph-mon[35941]: pgmap v3453: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-28T02:16:41.438 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:16:40 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6060.747414 secs 2024-03-28T02:16:43.368 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:16:42 smithi165 ceph-mon[36302]: pgmap v3454: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-03-28T02:16:43.391 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:16:42 smithi119 ceph-mon[31518]: pgmap v3454: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-03-28T02:16:43.438 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:16:42 smithi132 ceph-mon[35941]: pgmap v3454: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-03-28T02:16:45.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:16:44 smithi165 ceph-mon[36302]: pgmap v3455: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:16:45.391 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:16:44 smithi119 ceph-mon[31518]: pgmap v3455: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:16:45.438 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:16:44 smithi132 ceph-mon[35941]: pgmap v3455: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:16:46.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:16:45 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6065.747459 secs 2024-03-28T02:16:46.368 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:16:45 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6065.747459 secs 2024-03-28T02:16:46.392 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:16:45 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6065.747459 secs 2024-03-28T02:16:47.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:16:46 smithi165 ceph-mon[36302]: pgmap v3456: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:16:47.391 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:16:46 smithi119 ceph-mon[31518]: pgmap v3456: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:16:47.439 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:16:46 smithi132 ceph-mon[35941]: pgmap v3456: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:16:49.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:16:48 smithi165 ceph-mon[36302]: pgmap v3457: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:16:49.391 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:16:48 smithi119 ceph-mon[31518]: pgmap v3457: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:16:49.439 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:16:48 smithi132 ceph-mon[35941]: pgmap v3457: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:16:51.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:16:50 smithi165 ceph-mon[36302]: pgmap v3458: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-28T02:16:51.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:16:50 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6070.747548 secs 2024-03-28T02:16:51.391 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:16:50 smithi119 ceph-mon[31518]: pgmap v3458: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-28T02:16:51.392 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:16:50 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6070.747548 secs 2024-03-28T02:16:51.439 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:16:50 smithi132 ceph-mon[35941]: pgmap v3458: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-28T02:16:51.439 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:16:50 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6070.747548 secs 2024-03-28T02:16:52.368 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:16:51 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:16:52.391 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:16:51 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:16:52.441 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:16:51 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:16:53.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:16:53 smithi165 ceph-mon[36302]: pgmap v3459: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-03-28T02:16:53.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:16:53 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:16:53.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:16:53 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:16:53.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:16:53 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:16:53.392 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:16:53 smithi119 ceph-mon[31518]: pgmap v3459: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-03-28T02:16:53.392 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:16:53 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:16:53.392 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:16:53 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:16:53.392 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:16:53 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:16:53.438 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:16:53 smithi132 ceph-mon[35941]: pgmap v3459: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-03-28T02:16:53.439 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:16:53 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:16:53.439 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:16:53 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:16:53.439 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:16:53 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:16:54.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:16:54 smithi165 ceph-mon[36302]: pgmap v3460: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T02:16:54.391 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:16:54 smithi119 ceph-mon[31518]: pgmap v3460: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T02:16:54.438 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:16:54 smithi132 ceph-mon[35941]: pgmap v3460: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T02:16:56.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:16:56 smithi165 ceph-mon[36302]: pgmap v3461: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T02:16:56.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:16:56 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6075.747639 secs 2024-03-28T02:16:56.391 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:16:56 smithi119 ceph-mon[31518]: pgmap v3461: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T02:16:56.391 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:16:56 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6075.747639 secs 2024-03-28T02:16:56.439 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:16:56 smithi132 ceph-mon[35941]: pgmap v3461: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T02:16:56.439 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:16:56 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6075.747639 secs 2024-03-28T02:16:57.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:16:57 smithi165 ceph-mon[36302]: pgmap v3462: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T02:16:57.391 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:16:57 smithi119 ceph-mon[31518]: pgmap v3462: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T02:16:57.438 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:16:57 smithi132 ceph-mon[35941]: pgmap v3462: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T02:17:00.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:16:59 smithi165 ceph-mon[36302]: pgmap v3463: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T02:17:00.141 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:16:59 smithi119 ceph-mon[31518]: pgmap v3463: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T02:17:00.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:16:59 smithi132 ceph-mon[35941]: pgmap v3463: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T02:17:01.118 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:17:00 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6080.747702 secs 2024-03-28T02:17:01.141 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:17:00 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6080.747702 secs 2024-03-28T02:17:01.188 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:17:00 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6080.747702 secs 2024-03-28T02:17:02.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:17:01 smithi165 ceph-mon[36302]: pgmap v3464: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-03-28T02:17:02.141 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:17:01 smithi119 ceph-mon[31518]: pgmap v3464: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-03-28T02:17:02.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:17:01 smithi132 ceph-mon[35941]: pgmap v3464: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-03-28T02:17:04.118 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:17:03 smithi165 ceph-mon[36302]: pgmap v3465: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T02:17:04.141 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:17:03 smithi119 ceph-mon[31518]: pgmap v3465: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T02:17:04.188 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:17:03 smithi132 ceph-mon[35941]: pgmap v3465: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T02:17:06.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:17:05 smithi165 ceph-mon[36302]: pgmap v3466: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:17:06.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:17:05 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6085.747791 secs 2024-03-28T02:17:06.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:17:05 smithi119 ceph-mon[31518]: pgmap v3466: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:17:06.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:17:05 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6085.747791 secs 2024-03-28T02:17:06.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:17:05 smithi132 ceph-mon[35941]: pgmap v3466: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:17:06.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:17:05 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6085.747791 secs 2024-03-28T02:17:08.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:17:07 smithi165 ceph-mon[36302]: pgmap v3467: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:17:08.141 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:17:07 smithi119 ceph-mon[31518]: pgmap v3467: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:17:08.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:17:07 smithi132 ceph-mon[35941]: pgmap v3467: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:17:10.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:17:09 smithi165 ceph-mon[36302]: pgmap v3468: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:17:10.141 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:17:09 smithi119 ceph-mon[31518]: pgmap v3468: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:17:10.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:17:09 smithi132 ceph-mon[35941]: pgmap v3468: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:17:11.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:17:10 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6090.747834 secs 2024-03-28T02:17:11.141 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:17:10 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6090.747834 secs 2024-03-28T02:17:11.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:17:10 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6090.747834 secs 2024-03-28T02:17:12.118 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:17:11 smithi165 ceph-mon[36302]: pgmap v3469: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:17:12.141 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:17:11 smithi119 ceph-mon[31518]: pgmap v3469: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:17:12.188 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:17:11 smithi132 ceph-mon[35941]: pgmap v3469: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:17:14.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:17:13 smithi165 ceph-mon[36302]: pgmap v3470: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:17:14.141 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:17:13 smithi119 ceph-mon[31518]: pgmap v3470: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:17:14.188 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:17:13 smithi132 ceph-mon[35941]: pgmap v3470: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:17:16.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:17:15 smithi165 ceph-mon[36302]: pgmap v3471: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:17:16.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:17:15 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6095.747924 secs 2024-03-28T02:17:16.141 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:17:15 smithi119 ceph-mon[31518]: pgmap v3471: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:17:16.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:17:15 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6095.747924 secs 2024-03-28T02:17:16.188 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:17:15 smithi132 ceph-mon[35941]: pgmap v3471: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:17:16.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:17:15 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6095.747924 secs 2024-03-28T02:17:18.118 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:17:17 smithi165 ceph-mon[36302]: pgmap v3472: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:17:18.141 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:17:17 smithi119 ceph-mon[31518]: pgmap v3472: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:17:18.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:17:17 smithi132 ceph-mon[35941]: pgmap v3472: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:17:20.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:17:19 smithi165 ceph-mon[36302]: pgmap v3473: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:17:20.141 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:17:19 smithi119 ceph-mon[31518]: pgmap v3473: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:17:20.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:17:19 smithi132 ceph-mon[35941]: pgmap v3473: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:17:21.118 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:17:20 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6100.747988 secs 2024-03-28T02:17:21.141 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:17:20 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6100.747988 secs 2024-03-28T02:17:21.188 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:17:20 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6100.747988 secs 2024-03-28T02:17:22.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:17:21 smithi165 ceph-mon[36302]: pgmap v3474: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:17:22.141 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:17:21 smithi119 ceph-mon[31518]: pgmap v3474: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:17:22.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:17:21 smithi132 ceph-mon[35941]: pgmap v3474: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:17:24.118 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:17:23 smithi165 ceph-mon[36302]: pgmap v3475: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:17:24.141 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:17:23 smithi119 ceph-mon[31518]: pgmap v3475: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:17:24.188 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:17:23 smithi132 ceph-mon[35941]: pgmap v3475: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:17:26.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:17:25 smithi165 ceph-mon[36302]: pgmap v3476: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:17:26.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:17:25 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6105.748073 secs 2024-03-28T02:17:26.141 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:17:25 smithi119 ceph-mon[31518]: pgmap v3476: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:17:26.141 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:17:25 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6105.748073 secs 2024-03-28T02:17:26.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:17:25 smithi132 ceph-mon[35941]: pgmap v3476: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:17:26.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:17:25 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6105.748073 secs 2024-03-28T02:17:28.118 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:17:27 smithi165 ceph-mon[36302]: pgmap v3477: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:17:28.141 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:17:27 smithi119 ceph-mon[31518]: pgmap v3477: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:17:28.188 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:17:27 smithi132 ceph-mon[35941]: pgmap v3477: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:17:30.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:17:29 smithi165 ceph-mon[36302]: pgmap v3478: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:17:30.141 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:17:29 smithi119 ceph-mon[31518]: pgmap v3478: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:17:30.188 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:17:29 smithi132 ceph-mon[35941]: pgmap v3478: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:17:31.141 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:17:30 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6110.748134 secs 2024-03-28T02:17:31.188 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:17:30 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6110.748134 secs 2024-03-28T02:17:31.368 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:17:30 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6110.748134 secs 2024-03-28T02:17:32.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:17:31 smithi119 ceph-mon[31518]: pgmap v3479: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:17:32.188 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:17:31 smithi132 ceph-mon[35941]: pgmap v3479: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:17:32.368 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:17:31 smithi165 ceph-mon[36302]: pgmap v3479: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:17:34.141 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:17:33 smithi119 ceph-mon[31518]: pgmap v3480: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:17:34.171 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:17:33 smithi165 ceph-mon[36302]: pgmap v3480: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:17:34.188 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:17:33 smithi132 ceph-mon[35941]: pgmap v3480: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:17:36.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:17:35 smithi119 ceph-mon[31518]: pgmap v3481: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:17:36.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:17:35 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6115.748224 secs 2024-03-28T02:17:36.188 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:17:35 smithi132 ceph-mon[35941]: pgmap v3481: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:17:36.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:17:35 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6115.748224 secs 2024-03-28T02:17:36.368 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:17:35 smithi165 ceph-mon[36302]: pgmap v3481: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:17:36.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:17:35 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6115.748224 secs 2024-03-28T02:17:38.141 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:17:37 smithi119 ceph-mon[31518]: pgmap v3482: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:17:38.188 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:17:37 smithi132 ceph-mon[35941]: pgmap v3482: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:17:38.368 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:17:37 smithi165 ceph-mon[36302]: pgmap v3482: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:17:40.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:17:39 smithi119 ceph-mon[31518]: pgmap v3483: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:17:40.188 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:17:39 smithi132 ceph-mon[35941]: pgmap v3483: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:17:40.368 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:17:39 smithi165 ceph-mon[36302]: pgmap v3483: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:17:41.188 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:17:40 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6120.748315 secs 2024-03-28T02:17:41.368 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:17:40 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6120.748315 secs 2024-03-28T02:17:41.391 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:17:40 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6120.748315 secs 2024-03-28T02:17:42.188 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:17:41 smithi132 ceph-mon[35941]: pgmap v3484: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:17:42.368 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:17:41 smithi165 ceph-mon[36302]: pgmap v3484: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:17:42.391 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:17:41 smithi119 ceph-mon[31518]: pgmap v3484: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:17:44.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:17:43 smithi132 ceph-mon[35941]: pgmap v3485: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:17:44.216 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:17:43 smithi119 ceph-mon[31518]: pgmap v3485: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:17:44.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:17:43 smithi165 ceph-mon[36302]: pgmap v3485: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:17:46.188 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:17:45 smithi132 ceph-mon[35941]: pgmap v3486: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:17:46.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:17:45 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6125.748357 secs 2024-03-28T02:17:46.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:17:45 smithi165 ceph-mon[36302]: pgmap v3486: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:17:46.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:17:45 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6125.748357 secs 2024-03-28T02:17:46.391 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:17:45 smithi119 ceph-mon[31518]: pgmap v3486: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:17:46.392 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:17:45 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6125.748357 secs 2024-03-28T02:17:48.188 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:17:47 smithi132 ceph-mon[35941]: pgmap v3487: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:17:48.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:17:47 smithi165 ceph-mon[36302]: pgmap v3487: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:17:48.391 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:17:47 smithi119 ceph-mon[31518]: pgmap v3487: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:17:50.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:17:49 smithi132 ceph-mon[35941]: pgmap v3488: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:17:50.368 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:17:49 smithi165 ceph-mon[36302]: pgmap v3488: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:17:50.391 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:17:49 smithi119 ceph-mon[31518]: pgmap v3488: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:17:51.188 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:17:50 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6130.748398 secs 2024-03-28T02:17:51.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:17:50 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6130.748398 secs 2024-03-28T02:17:51.391 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:17:50 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6130.748398 secs 2024-03-28T02:17:52.188 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:17:51 smithi132 ceph-mon[35941]: pgmap v3489: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:17:52.368 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:17:51 smithi165 ceph-mon[36302]: pgmap v3489: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:17:52.391 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:17:51 smithi119 ceph-mon[31518]: pgmap v3489: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:17:54.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:17:53 smithi132 ceph-mon[35941]: pgmap v3490: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:17:54.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:17:53 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:17:54.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:17:53 smithi165 ceph-mon[36302]: pgmap v3490: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:17:54.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:17:53 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:17:54.391 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:17:53 smithi119 ceph-mon[31518]: pgmap v3490: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:17:54.391 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:17:53 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:17:55.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:17:54 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:17:55.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:17:54 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:17:55.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:17:54 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:17:55.391 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:17:54 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:17:55.392 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:17:54 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:17:55.392 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:17:54 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:17:55.438 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:17:54 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:17:55.438 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:17:54 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:17:55.439 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:17:54 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:17:56.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:17:55 smithi165 ceph-mon[36302]: pgmap v3491: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T02:17:56.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:17:55 smithi165 ceph-mon[36302]: pgmap v3492: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:17:56.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:17:55 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6135.748486 secs 2024-03-28T02:17:56.391 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:17:55 smithi119 ceph-mon[31518]: pgmap v3491: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T02:17:56.391 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:17:55 smithi119 ceph-mon[31518]: pgmap v3492: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:17:56.391 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:17:55 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6135.748486 secs 2024-03-28T02:17:56.439 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:17:55 smithi132 ceph-mon[35941]: pgmap v3491: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T02:17:56.439 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:17:55 smithi132 ceph-mon[35941]: pgmap v3492: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:17:56.439 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:17:55 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6135.748486 secs 2024-03-28T02:17:58.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:17:57 smithi165 ceph-mon[36302]: pgmap v3493: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:17:58.391 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:17:57 smithi119 ceph-mon[31518]: pgmap v3493: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:17:58.438 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:17:57 smithi132 ceph-mon[35941]: pgmap v3493: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:18:00.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:17:59 smithi165 ceph-mon[36302]: pgmap v3494: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:18:00.391 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:17:59 smithi119 ceph-mon[31518]: pgmap v3494: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:18:00.438 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:17:59 smithi132 ceph-mon[35941]: pgmap v3494: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:18:01.368 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:18:00 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6140.748549 secs 2024-03-28T02:18:01.391 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:18:00 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6140.748549 secs 2024-03-28T02:18:01.438 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:18:00 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6140.748549 secs 2024-03-28T02:18:02.368 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:18:01 smithi165 ceph-mon[36302]: pgmap v3495: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:18:02.391 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:18:01 smithi119 ceph-mon[31518]: pgmap v3495: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:18:02.438 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:18:01 smithi132 ceph-mon[35941]: pgmap v3495: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:18:04.368 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:18:03 smithi165 ceph-mon[36302]: pgmap v3496: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:18:04.391 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:18:03 smithi119 ceph-mon[31518]: pgmap v3496: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:18:04.438 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:18:03 smithi132 ceph-mon[35941]: pgmap v3496: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:18:06.368 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:18:05 smithi165 ceph-mon[36302]: pgmap v3497: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T02:18:06.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:18:05 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6145.748610 secs 2024-03-28T02:18:06.391 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:18:05 smithi119 ceph-mon[31518]: pgmap v3497: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T02:18:06.391 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:18:05 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6145.748610 secs 2024-03-28T02:18:06.438 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:18:05 smithi132 ceph-mon[35941]: pgmap v3497: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T02:18:06.438 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:18:05 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6145.748610 secs 2024-03-28T02:18:08.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:18:07 smithi165 ceph-mon[36302]: pgmap v3498: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:18:08.391 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:18:07 smithi119 ceph-mon[31518]: pgmap v3498: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:18:08.439 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:18:07 smithi132 ceph-mon[35941]: pgmap v3498: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:18:10.368 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:18:09 smithi165 ceph-mon[36302]: pgmap v3499: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:18:10.392 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:18:09 smithi119 ceph-mon[31518]: pgmap v3499: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:18:10.438 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:18:09 smithi132 ceph-mon[35941]: pgmap v3499: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:18:11.368 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:18:11 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6150.748670 secs 2024-03-28T02:18:11.391 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:18:11 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6150.748670 secs 2024-03-28T02:18:11.438 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:18:11 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6150.748670 secs 2024-03-28T02:18:12.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:18:12 smithi165 ceph-mon[36302]: pgmap v3500: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:18:12.391 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:18:12 smithi119 ceph-mon[31518]: pgmap v3500: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:18:12.438 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:18:12 smithi132 ceph-mon[35941]: pgmap v3500: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:18:14.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:18:14 smithi165 ceph-mon[36302]: pgmap v3501: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:18:14.391 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:18:14 smithi119 ceph-mon[31518]: pgmap v3501: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:18:14.438 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:18:14 smithi132 ceph-mon[35941]: pgmap v3501: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:18:16.368 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:18:16 smithi165 ceph-mon[36302]: pgmap v3502: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:18:16.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:18:16 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6155.748710 secs 2024-03-28T02:18:16.392 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:18:16 smithi119 ceph-mon[31518]: pgmap v3502: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:18:16.392 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:18:16 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6155.748710 secs 2024-03-28T02:18:16.438 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:18:16 smithi132 ceph-mon[35941]: pgmap v3502: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:18:16.439 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:18:16 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6155.748710 secs 2024-03-28T02:18:18.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:18:18 smithi165 ceph-mon[36302]: pgmap v3503: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:18:18.391 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:18:18 smithi119 ceph-mon[31518]: pgmap v3503: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:18:18.438 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:18:18 smithi132 ceph-mon[35941]: pgmap v3503: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:18:20.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:18:20 smithi165 ceph-mon[36302]: pgmap v3504: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:18:20.391 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:18:20 smithi119 ceph-mon[31518]: pgmap v3504: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:18:20.439 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:18:20 smithi132 ceph-mon[35941]: pgmap v3504: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:18:21.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:18:21 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6160.748801 secs 2024-03-28T02:18:21.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:18:21 smithi165 ceph-mon[36302]: pgmap v3505: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:18:21.391 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:18:21 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6160.748801 secs 2024-03-28T02:18:21.391 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:18:21 smithi119 ceph-mon[31518]: pgmap v3505: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:18:21.438 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:18:21 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6160.748801 secs 2024-03-28T02:18:21.438 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:18:21 smithi132 ceph-mon[35941]: pgmap v3505: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:18:23.618 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:18:23 smithi165 ceph-mon[36302]: pgmap v3506: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:18:23.641 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:18:23 smithi119 ceph-mon[31518]: pgmap v3506: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:18:23.688 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:18:23 smithi132 ceph-mon[35941]: pgmap v3506: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:18:25.619 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:18:25 smithi165 ceph-mon[36302]: pgmap v3507: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:18:25.641 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:18:25 smithi119 ceph-mon[31518]: pgmap v3507: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:18:25.688 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:18:25 smithi132 ceph-mon[35941]: pgmap v3507: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:18:26.618 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:18:26 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6165.748889 secs 2024-03-28T02:18:26.641 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:18:26 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6165.748889 secs 2024-03-28T02:18:26.688 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:18:26 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6165.748889 secs 2024-03-28T02:18:27.618 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:18:27 smithi165 ceph-mon[36302]: pgmap v3508: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:18:27.641 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:18:27 smithi119 ceph-mon[31518]: pgmap v3508: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:18:27.688 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:18:27 smithi132 ceph-mon[35941]: pgmap v3508: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:18:29.619 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:18:29 smithi165 ceph-mon[36302]: pgmap v3509: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:18:29.641 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:18:29 smithi119 ceph-mon[31518]: pgmap v3509: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:18:29.688 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:18:29 smithi132 ceph-mon[35941]: pgmap v3509: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:18:31.619 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:18:31 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6170.748951 secs 2024-03-28T02:18:31.619 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:18:31 smithi165 ceph-mon[36302]: pgmap v3510: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:18:31.641 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:18:31 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6170.748951 secs 2024-03-28T02:18:31.642 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:18:31 smithi119 ceph-mon[31518]: pgmap v3510: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:18:31.688 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:18:31 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6170.748951 secs 2024-03-28T02:18:31.688 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:18:31 smithi132 ceph-mon[35941]: pgmap v3510: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:18:33.619 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:18:33 smithi165 ceph-mon[36302]: pgmap v3511: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:18:33.641 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:18:33 smithi119 ceph-mon[31518]: pgmap v3511: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:18:33.688 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:18:33 smithi132 ceph-mon[35941]: pgmap v3511: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:18:35.641 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:18:35 smithi119 ceph-mon[31518]: pgmap v3512: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:18:35.688 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:18:35 smithi132 ceph-mon[35941]: pgmap v3512: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:18:35.869 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:18:35 smithi165 ceph-mon[36302]: pgmap v3512: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:18:36.641 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:18:36 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6175.749036 secs 2024-03-28T02:18:36.688 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:18:36 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6175.749036 secs 2024-03-28T02:18:36.868 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:18:36 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6175.749036 secs 2024-03-28T02:18:37.642 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:18:37 smithi119 ceph-mon[31518]: pgmap v3513: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:18:37.688 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:18:37 smithi132 ceph-mon[35941]: pgmap v3513: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:18:37.868 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:18:37 smithi165 ceph-mon[36302]: pgmap v3513: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:18:39.641 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:18:39 smithi119 ceph-mon[31518]: pgmap v3514: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:18:39.688 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:18:39 smithi132 ceph-mon[35941]: pgmap v3514: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:18:39.868 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:18:39 smithi165 ceph-mon[36302]: pgmap v3514: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:18:41.641 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:18:41 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6180.749122 secs 2024-03-28T02:18:41.642 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:18:41 smithi119 ceph-mon[31518]: pgmap v3515: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:18:41.688 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:18:41 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6180.749122 secs 2024-03-28T02:18:41.689 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:18:41 smithi132 ceph-mon[35941]: pgmap v3515: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:18:41.868 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:18:41 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6180.749122 secs 2024-03-28T02:18:41.869 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:18:41 smithi165 ceph-mon[36302]: pgmap v3515: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:18:43.688 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:18:43 smithi132 ceph-mon[35941]: pgmap v3516: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:18:43.868 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:18:43 smithi165 ceph-mon[36302]: pgmap v3516: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:18:43.891 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:18:43 smithi119 ceph-mon[31518]: pgmap v3516: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:18:45.688 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:18:45 smithi132 ceph-mon[35941]: pgmap v3517: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:18:45.868 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:18:45 smithi165 ceph-mon[36302]: pgmap v3517: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:18:45.892 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:18:45 smithi119 ceph-mon[31518]: pgmap v3517: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:18:46.688 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:18:46 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6185.749211 secs 2024-03-28T02:18:46.868 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:18:46 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6185.749211 secs 2024-03-28T02:18:46.891 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:18:46 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6185.749211 secs 2024-03-28T02:18:47.641 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:18:47 smithi119 ceph-mon[31518]: pgmap v3518: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:18:47.688 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:18:47 smithi132 ceph-mon[35941]: pgmap v3518: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:18:47.868 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:18:47 smithi165 ceph-mon[36302]: pgmap v3518: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:18:49.688 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:18:49 smithi132 ceph-mon[35941]: pgmap v3519: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:18:49.868 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:18:49 smithi165 ceph-mon[36302]: pgmap v3519: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:18:49.891 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:18:49 smithi119 ceph-mon[31518]: pgmap v3519: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:18:51.688 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:18:51 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6190.749300 secs 2024-03-28T02:18:51.689 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:18:51 smithi132 ceph-mon[35941]: pgmap v3520: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:18:51.868 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:18:51 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6190.749300 secs 2024-03-28T02:18:51.869 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:18:51 smithi165 ceph-mon[36302]: pgmap v3520: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:18:51.891 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:18:51 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6190.749300 secs 2024-03-28T02:18:51.891 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:18:51 smithi119 ceph-mon[31518]: pgmap v3520: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:18:53.688 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:18:53 smithi132 ceph-mon[35941]: pgmap v3521: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:18:53.868 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:18:53 smithi165 ceph-mon[36302]: pgmap v3521: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:18:53.891 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:18:53 smithi119 ceph-mon[31518]: pgmap v3521: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:18:55.688 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:18:55 smithi132 ceph-mon[35941]: pgmap v3522: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:18:55.689 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:18:55 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:18:55.868 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:18:55 smithi165 ceph-mon[36302]: pgmap v3522: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:18:55.869 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:18:55 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:18:55.891 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:18:55 smithi119 ceph-mon[31518]: pgmap v3522: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:18:55.891 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:18:55 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:18:56.688 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:18:56 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6195.749340 secs 2024-03-28T02:18:56.689 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:18:56 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:18:56.689 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:18:56 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:18:56.689 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:18:56 smithi132 ceph-mon[35941]: pgmap v3523: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T02:18:56.689 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:18:56 smithi132 ceph-mon[35941]: pgmap v3524: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:18:56.689 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:18:56 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:18:56.868 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:18:56 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6195.749340 secs 2024-03-28T02:18:56.869 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:18:56 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:18:56.869 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:18:56 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:18:56.869 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:18:56 smithi165 ceph-mon[36302]: pgmap v3523: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T02:18:56.869 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:18:56 smithi165 ceph-mon[36302]: pgmap v3524: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:18:56.869 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:18:56 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:18:56.891 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:18:56 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6195.749340 secs 2024-03-28T02:18:56.892 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:18:56 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:18:56.892 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:18:56 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:18:56.892 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:18:56 smithi119 ceph-mon[31518]: pgmap v3523: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T02:18:56.892 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:18:56 smithi119 ceph-mon[31518]: pgmap v3524: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:18:56.892 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:18:56 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:18:59.141 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:18:58 smithi119 ceph-mon[31518]: pgmap v3525: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:18:59.188 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:18:58 smithi132 ceph-mon[35941]: pgmap v3525: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:18:59.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:18:58 smithi165 ceph-mon[36302]: pgmap v3525: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:19:01.141 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:19:00 smithi119 ceph-mon[31518]: pgmap v3526: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:19:01.141 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:19:00 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6200.749400 secs 2024-03-28T02:19:01.188 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:19:00 smithi132 ceph-mon[35941]: pgmap v3526: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:19:01.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:19:00 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6200.749400 secs 2024-03-28T02:19:01.368 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:19:00 smithi165 ceph-mon[36302]: pgmap v3526: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:19:01.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:19:00 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6200.749400 secs 2024-03-28T02:19:03.188 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:19:02 smithi132 ceph-mon[35941]: pgmap v3527: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:19:03.368 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:19:02 smithi165 ceph-mon[36302]: pgmap v3527: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:19:03.391 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:19:02 smithi119 ceph-mon[31518]: pgmap v3527: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:19:04.868 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:19:04 smithi165 ceph-mon[36302]: pgmap v3528: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-28T02:19:04.891 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:19:04 smithi119 ceph-mon[31518]: pgmap v3528: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-28T02:19:04.938 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:19:04 smithi132 ceph-mon[35941]: pgmap v3528: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-28T02:19:05.938 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:19:05 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6205.749435 secs 2024-03-28T02:19:06.118 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:19:05 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6205.749435 secs 2024-03-28T02:19:06.141 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:19:05 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6205.749435 secs 2024-03-28T02:19:06.938 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:19:06 smithi132 ceph-mon[35941]: pgmap v3529: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T02:19:07.118 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:19:06 smithi165 ceph-mon[36302]: pgmap v3529: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T02:19:07.141 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:19:06 smithi119 ceph-mon[31518]: pgmap v3529: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T02:19:09.141 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:19:08 smithi119 ceph-mon[31518]: pgmap v3530: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:19:09.188 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:19:08 smithi132 ceph-mon[35941]: pgmap v3530: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:19:09.368 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:19:08 smithi165 ceph-mon[36302]: pgmap v3530: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:19:11.188 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:19:10 smithi132 ceph-mon[35941]: pgmap v3531: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:19:11.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:19:10 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6210.749526 secs 2024-03-28T02:19:11.368 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:19:10 smithi165 ceph-mon[36302]: pgmap v3531: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:19:11.368 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:19:10 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6210.749526 secs 2024-03-28T02:19:11.391 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:19:10 smithi119 ceph-mon[31518]: pgmap v3531: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:19:11.391 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:19:10 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6210.749526 secs 2024-03-28T02:19:13.188 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:19:12 smithi132 ceph-mon[35941]: pgmap v3532: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:19:13.368 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:19:12 smithi165 ceph-mon[36302]: pgmap v3532: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:19:13.391 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:19:12 smithi119 ceph-mon[31518]: pgmap v3532: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:19:15.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:19:14 smithi132 ceph-mon[35941]: pgmap v3533: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:19:15.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:19:14 smithi165 ceph-mon[36302]: pgmap v3533: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:19:15.391 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:19:14 smithi119 ceph-mon[31518]: pgmap v3533: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:19:16.188 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:19:15 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6215.749616 secs 2024-03-28T02:19:16.368 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:19:15 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6215.749616 secs 2024-03-28T02:19:16.391 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:19:15 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6215.749616 secs 2024-03-28T02:19:17.188 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:19:16 smithi132 ceph-mon[35941]: pgmap v3534: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:19:17.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:19:16 smithi165 ceph-mon[36302]: pgmap v3534: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:19:17.391 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:19:16 smithi119 ceph-mon[31518]: pgmap v3534: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:19:19.188 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:19:18 smithi132 ceph-mon[35941]: pgmap v3535: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:19:19.368 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:19:18 smithi165 ceph-mon[36302]: pgmap v3535: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:19:19.391 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:19:18 smithi119 ceph-mon[31518]: pgmap v3535: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:19:21.188 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:19:20 smithi132 ceph-mon[35941]: pgmap v3536: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:19:21.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:19:20 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6220.749706 secs 2024-03-28T02:19:21.368 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:19:20 smithi165 ceph-mon[36302]: pgmap v3536: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:19:21.368 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:19:20 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6220.749706 secs 2024-03-28T02:19:21.391 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:19:20 smithi119 ceph-mon[31518]: pgmap v3536: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:19:21.391 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:19:20 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6220.749706 secs 2024-03-28T02:19:23.188 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:19:22 smithi132 ceph-mon[35941]: pgmap v3537: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:19:23.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:19:22 smithi165 ceph-mon[36302]: pgmap v3537: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:19:23.391 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:19:22 smithi119 ceph-mon[31518]: pgmap v3537: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:19:25.368 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:19:24 smithi165 ceph-mon[36302]: pgmap v3538: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:19:25.391 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:19:24 smithi119 ceph-mon[31518]: pgmap v3538: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:19:25.438 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:19:24 smithi132 ceph-mon[35941]: pgmap v3538: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:19:26.368 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:19:25 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6225.749744 secs 2024-03-28T02:19:26.391 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:19:25 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6225.749744 secs 2024-03-28T02:19:26.438 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:19:25 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6225.749744 secs 2024-03-28T02:19:27.368 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:19:26 smithi165 ceph-mon[36302]: pgmap v3539: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:19:27.391 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:19:26 smithi119 ceph-mon[31518]: pgmap v3539: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:19:27.438 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:19:26 smithi132 ceph-mon[35941]: pgmap v3539: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:19:29.368 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:19:28 smithi165 ceph-mon[36302]: pgmap v3540: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T02:19:29.391 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:19:28 smithi119 ceph-mon[31518]: pgmap v3540: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T02:19:29.438 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:19:28 smithi132 ceph-mon[35941]: pgmap v3540: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T02:19:31.368 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:19:30 smithi165 ceph-mon[36302]: pgmap v3541: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:19:31.368 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:19:30 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6230.749833 secs 2024-03-28T02:19:31.391 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:19:30 smithi119 ceph-mon[31518]: pgmap v3541: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:19:31.391 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:19:30 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6230.749833 secs 2024-03-28T02:19:31.438 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:19:30 smithi132 ceph-mon[35941]: pgmap v3541: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:19:31.438 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:19:30 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6230.749833 secs 2024-03-28T02:19:33.272 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:19:32 smithi119 ceph-mon[31518]: pgmap v3542: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T02:19:33.368 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:19:32 smithi165 ceph-mon[36302]: pgmap v3542: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T02:19:33.438 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:19:32 smithi132 ceph-mon[35941]: pgmap v3542: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T02:19:35.368 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:19:34 smithi165 ceph-mon[36302]: pgmap v3543: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T02:19:35.391 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:19:34 smithi119 ceph-mon[31518]: pgmap v3543: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T02:19:35.438 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:19:34 smithi132 ceph-mon[35941]: pgmap v3543: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T02:19:36.368 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:19:35 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6235.749898 secs 2024-03-28T02:19:36.391 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:19:35 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6235.749898 secs 2024-03-28T02:19:36.438 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:19:35 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6235.749898 secs 2024-03-28T02:19:37.368 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:19:36 smithi165 ceph-mon[36302]: pgmap v3544: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:19:37.391 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:19:36 smithi119 ceph-mon[31518]: pgmap v3544: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:19:37.438 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:19:36 smithi132 ceph-mon[35941]: pgmap v3544: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:19:39.368 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:19:38 smithi165 ceph-mon[36302]: pgmap v3545: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:19:39.391 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:19:38 smithi119 ceph-mon[31518]: pgmap v3545: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:19:39.438 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:19:38 smithi132 ceph-mon[35941]: pgmap v3545: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:19:41.368 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:19:41 smithi165 ceph-mon[36302]: pgmap v3546: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:19:41.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:19:41 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6240.749917 secs 2024-03-28T02:19:41.391 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:19:41 smithi119 ceph-mon[31518]: pgmap v3546: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:19:41.391 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:19:41 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6240.749917 secs 2024-03-28T02:19:41.438 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:19:41 smithi132 ceph-mon[35941]: pgmap v3546: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:19:41.438 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:19:41 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6240.749917 secs 2024-03-28T02:19:43.368 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:19:43 smithi165 ceph-mon[36302]: pgmap v3547: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:19:43.391 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:19:43 smithi119 ceph-mon[31518]: pgmap v3547: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:19:43.438 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:19:43 smithi132 ceph-mon[35941]: pgmap v3547: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:19:45.368 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:19:45 smithi165 ceph-mon[36302]: pgmap v3548: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:19:45.391 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:19:45 smithi119 ceph-mon[31518]: pgmap v3548: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:19:45.438 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:19:45 smithi132 ceph-mon[35941]: pgmap v3548: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:19:46.368 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:19:46 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6245.750004 secs 2024-03-28T02:19:46.391 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:19:46 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6245.750004 secs 2024-03-28T02:19:46.438 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:19:46 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6245.750004 secs 2024-03-28T02:19:47.368 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:19:47 smithi165 ceph-mon[36302]: pgmap v3549: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:19:47.423 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:19:47 smithi119 ceph-mon[31518]: pgmap v3549: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:19:47.438 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:19:47 smithi132 ceph-mon[35941]: pgmap v3549: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:19:49.368 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:19:49 smithi165 ceph-mon[36302]: pgmap v3550: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:19:49.391 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:19:49 smithi119 ceph-mon[31518]: pgmap v3550: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:19:49.438 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:19:49 smithi132 ceph-mon[35941]: pgmap v3550: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:19:51.368 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:19:51 smithi165 ceph-mon[36302]: pgmap v3551: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:19:51.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:19:51 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6250.750090 secs 2024-03-28T02:19:51.391 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:19:51 smithi119 ceph-mon[31518]: pgmap v3551: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:19:51.391 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:19:51 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6250.750090 secs 2024-03-28T02:19:51.441 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:19:51 smithi132 ceph-mon[35941]: pgmap v3551: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:19:51.441 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:19:51 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6250.750090 secs 2024-03-28T02:19:53.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:19:53 smithi165 ceph-mon[36302]: pgmap v3552: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:19:53.391 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:19:53 smithi119 ceph-mon[31518]: pgmap v3552: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:19:53.438 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:19:53 smithi132 ceph-mon[35941]: pgmap v3552: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:19:55.368 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:19:55 smithi165 ceph-mon[36302]: pgmap v3553: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:19:55.391 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:19:55 smithi119 ceph-mon[31518]: pgmap v3553: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:19:55.438 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:19:55 smithi132 ceph-mon[35941]: pgmap v3553: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:19:56.368 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:19:56 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6255.750128 secs 2024-03-28T02:19:56.391 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:19:56 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6255.750128 secs 2024-03-28T02:19:56.438 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:19:56 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6255.750128 secs 2024-03-28T02:19:57.368 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:19:57 smithi165 ceph-mon[36302]: pgmap v3554: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:19:57.368 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:19:57 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:19:57.391 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:19:57 smithi119 ceph-mon[31518]: pgmap v3554: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:19:57.391 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:19:57 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:19:57.438 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:19:57 smithi132 ceph-mon[35941]: pgmap v3554: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:19:57.438 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:19:57 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:19:58.368 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:19:58 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:19:58.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:19:58 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:19:58.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:19:58 smithi165 ceph-mon[36302]: pgmap v3555: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T02:19:58.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:19:58 smithi165 ceph-mon[36302]: pgmap v3556: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:19:58.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:19:58 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:19:58.391 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:19:58 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:19:58.391 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:19:58 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:19:58.392 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:19:58 smithi119 ceph-mon[31518]: pgmap v3555: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T02:19:58.392 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:19:58 smithi119 ceph-mon[31518]: pgmap v3556: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:19:58.392 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:19:58 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:19:58.438 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:19:58 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:19:58.438 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:19:58 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:19:58.438 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:19:58 smithi132 ceph-mon[35941]: pgmap v3555: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T02:19:58.438 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:19:58 smithi132 ceph-mon[35941]: pgmap v3556: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:19:58.439 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:19:58 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:20:00.868 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:20:00 smithi165 ceph-mon[36302]: pgmap v3557: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:20:00.869 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:20:00 smithi165 ceph-mon[36302]: overall HEALTH_WARN 5 failed cephadm daemon(s); 1 filesystem is degraded; 1 MDSs report slow requests 2024-03-28T02:20:00.891 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:20:00 smithi119 ceph-mon[31518]: pgmap v3557: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:20:00.892 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:20:00 smithi119 ceph-mon[31518]: overall HEALTH_WARN 5 failed cephadm daemon(s); 1 filesystem is degraded; 1 MDSs report slow requests 2024-03-28T02:20:00.938 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:20:00 smithi132 ceph-mon[35941]: pgmap v3557: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:20:00.938 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:20:00 smithi132 ceph-mon[35941]: overall HEALTH_WARN 5 failed cephadm daemon(s); 1 filesystem is degraded; 1 MDSs report slow requests 2024-03-28T02:20:01.868 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:20:01 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6260.750218 secs 2024-03-28T02:20:01.891 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:20:01 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6260.750218 secs 2024-03-28T02:20:01.938 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:20:01 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6260.750218 secs 2024-03-28T02:20:02.868 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:20:02 smithi165 ceph-mon[36302]: pgmap v3558: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:20:02.891 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:20:02 smithi119 ceph-mon[31518]: pgmap v3558: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:20:02.938 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:20:02 smithi132 ceph-mon[35941]: pgmap v3558: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:20:04.868 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:20:04 smithi165 ceph-mon[36302]: pgmap v3559: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:20:04.891 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:20:04 smithi119 ceph-mon[31518]: pgmap v3559: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:20:04.938 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:20:04 smithi132 ceph-mon[35941]: pgmap v3559: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:20:06.868 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:20:06 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6265.750306 secs 2024-03-28T02:20:06.869 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:20:06 smithi165 ceph-mon[36302]: pgmap v3560: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:20:06.891 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:20:06 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6265.750306 secs 2024-03-28T02:20:06.891 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:20:06 smithi119 ceph-mon[31518]: pgmap v3560: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:20:06.938 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:20:06 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6265.750306 secs 2024-03-28T02:20:06.938 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:20:06 smithi132 ceph-mon[35941]: pgmap v3560: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:20:08.868 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:20:08 smithi165 ceph-mon[36302]: pgmap v3561: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T02:20:08.891 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:20:08 smithi119 ceph-mon[31518]: pgmap v3561: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T02:20:08.938 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:20:08 smithi132 ceph-mon[35941]: pgmap v3561: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T02:20:10.641 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:20:10 smithi119 ceph-mon[31518]: pgmap v3562: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:20:10.868 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:20:10 smithi165 ceph-mon[36302]: pgmap v3562: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:20:10.938 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:20:10 smithi132 ceph-mon[35941]: pgmap v3562: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:20:11.868 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:20:11 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6270.750395 secs 2024-03-28T02:20:11.891 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:20:11 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6270.750395 secs 2024-03-28T02:20:11.938 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:20:11 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6270.750395 secs 2024-03-28T02:20:12.868 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:20:12 smithi165 ceph-mon[36302]: pgmap v3563: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:20:12.891 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:20:12 smithi119 ceph-mon[31518]: pgmap v3563: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:20:12.938 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:20:12 smithi132 ceph-mon[35941]: pgmap v3563: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:20:14.868 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:20:14 smithi165 ceph-mon[36302]: pgmap v3564: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:20:14.891 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:20:14 smithi119 ceph-mon[31518]: pgmap v3564: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:20:14.938 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:20:14 smithi132 ceph-mon[35941]: pgmap v3564: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:20:16.868 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:20:16 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6275.750485 secs 2024-03-28T02:20:16.869 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:20:16 smithi165 ceph-mon[36302]: pgmap v3565: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:20:16.891 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:20:16 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6275.750485 secs 2024-03-28T02:20:16.891 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:20:16 smithi119 ceph-mon[31518]: pgmap v3565: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:20:16.938 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:20:16 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6275.750485 secs 2024-03-28T02:20:16.938 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:20:16 smithi132 ceph-mon[35941]: pgmap v3565: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:20:18.868 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:20:18 smithi165 ceph-mon[36302]: pgmap v3566: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:20:18.891 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:20:18 smithi119 ceph-mon[31518]: pgmap v3566: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:20:18.938 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:20:18 smithi132 ceph-mon[35941]: pgmap v3566: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:20:20.868 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:20:20 smithi165 ceph-mon[36302]: pgmap v3567: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:20:20.891 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:20:20 smithi119 ceph-mon[31518]: pgmap v3567: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:20:20.938 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:20:20 smithi132 ceph-mon[35941]: pgmap v3567: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:20:21.868 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:20:21 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6280.750574 secs 2024-03-28T02:20:21.891 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:20:21 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6280.750574 secs 2024-03-28T02:20:21.938 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:20:21 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6280.750574 secs 2024-03-28T02:20:22.868 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:20:22 smithi165 ceph-mon[36302]: pgmap v3568: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:20:22.891 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:20:22 smithi119 ceph-mon[31518]: pgmap v3568: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:20:22.938 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:20:22 smithi132 ceph-mon[35941]: pgmap v3568: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:20:24.868 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:20:24 smithi165 ceph-mon[36302]: pgmap v3569: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:20:24.891 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:20:24 smithi119 ceph-mon[31518]: pgmap v3569: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:20:24.938 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:20:24 smithi132 ceph-mon[35941]: pgmap v3569: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:20:26.870 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:20:26 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6285.750662 secs 2024-03-28T02:20:26.870 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:20:26 smithi165 ceph-mon[36302]: pgmap v3570: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:20:26.891 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:20:26 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6285.750662 secs 2024-03-28T02:20:26.891 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:20:26 smithi119 ceph-mon[31518]: pgmap v3570: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:20:26.938 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:20:26 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6285.750662 secs 2024-03-28T02:20:26.939 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:20:26 smithi132 ceph-mon[35941]: pgmap v3570: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:20:28.868 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:20:28 smithi165 ceph-mon[36302]: pgmap v3571: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:20:28.891 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:20:28 smithi119 ceph-mon[31518]: pgmap v3571: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:20:28.938 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:20:28 smithi132 ceph-mon[35941]: pgmap v3571: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:20:30.868 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:20:30 smithi165 ceph-mon[36302]: pgmap v3572: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:20:30.891 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:20:30 smithi119 ceph-mon[31518]: pgmap v3572: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:20:30.938 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:20:30 smithi132 ceph-mon[35941]: pgmap v3572: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:20:31.868 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:20:31 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6290.750702 secs 2024-03-28T02:20:31.891 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:20:31 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6290.750702 secs 2024-03-28T02:20:31.938 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:20:31 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6290.750702 secs 2024-03-28T02:20:32.868 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:20:32 smithi165 ceph-mon[36302]: pgmap v3573: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:20:32.891 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:20:32 smithi119 ceph-mon[31518]: pgmap v3573: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:20:32.938 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:20:32 smithi132 ceph-mon[35941]: pgmap v3573: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:20:34.868 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:20:34 smithi165 ceph-mon[36302]: pgmap v3574: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:20:34.891 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:20:34 smithi119 ceph-mon[31518]: pgmap v3574: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:20:34.938 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:20:34 smithi132 ceph-mon[35941]: pgmap v3574: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:20:36.868 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:20:36 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6295.750762 secs 2024-03-28T02:20:36.868 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:20:36 smithi165 ceph-mon[36302]: pgmap v3575: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:20:36.891 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:20:36 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6295.750762 secs 2024-03-28T02:20:36.891 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:20:36 smithi119 ceph-mon[31518]: pgmap v3575: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:20:36.938 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:20:36 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6295.750762 secs 2024-03-28T02:20:36.938 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:20:36 smithi132 ceph-mon[35941]: pgmap v3575: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:20:38.868 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:20:38 smithi165 ceph-mon[36302]: pgmap v3576: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:20:38.891 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:20:38 smithi119 ceph-mon[31518]: pgmap v3576: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:20:38.938 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:20:38 smithi132 ceph-mon[35941]: pgmap v3576: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:20:40.868 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:20:40 smithi165 ceph-mon[36302]: pgmap v3577: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:20:40.891 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:20:40 smithi119 ceph-mon[31518]: pgmap v3577: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:20:40.938 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:20:40 smithi132 ceph-mon[35941]: pgmap v3577: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:20:41.868 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:20:41 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6300.750849 secs 2024-03-28T02:20:41.891 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:20:41 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6300.750849 secs 2024-03-28T02:20:41.938 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:20:41 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6300.750849 secs 2024-03-28T02:20:42.868 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:20:42 smithi165 ceph-mon[36302]: pgmap v3578: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:20:42.891 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:20:42 smithi119 ceph-mon[31518]: pgmap v3578: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:20:42.938 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:20:42 smithi132 ceph-mon[35941]: pgmap v3578: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:20:44.868 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:20:44 smithi165 ceph-mon[36302]: pgmap v3579: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:20:44.891 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:20:44 smithi119 ceph-mon[31518]: pgmap v3579: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:20:44.938 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:20:44 smithi132 ceph-mon[35941]: pgmap v3579: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:20:46.868 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:20:46 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6305.750900 secs 2024-03-28T02:20:46.869 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:20:46 smithi165 ceph-mon[36302]: pgmap v3580: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:20:46.891 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:20:46 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6305.750900 secs 2024-03-28T02:20:46.891 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:20:46 smithi119 ceph-mon[31518]: pgmap v3580: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:20:46.938 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:20:46 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6305.750900 secs 2024-03-28T02:20:46.938 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:20:46 smithi132 ceph-mon[35941]: pgmap v3580: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:20:48.868 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:20:48 smithi165 ceph-mon[36302]: pgmap v3581: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:20:48.891 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:20:48 smithi119 ceph-mon[31518]: pgmap v3581: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:20:48.938 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:20:48 smithi132 ceph-mon[35941]: pgmap v3581: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:20:50.868 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:20:50 smithi165 ceph-mon[36302]: pgmap v3582: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:20:50.891 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:20:50 smithi119 ceph-mon[31518]: pgmap v3582: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:20:50.938 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:20:50 smithi132 ceph-mon[35941]: pgmap v3582: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:20:51.868 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:20:51 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6310.750986 secs 2024-03-28T02:20:51.891 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:20:51 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6310.750986 secs 2024-03-28T02:20:51.938 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:20:51 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6310.750986 secs 2024-03-28T02:20:52.868 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:20:52 smithi165 ceph-mon[36302]: pgmap v3583: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:20:52.891 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:20:52 smithi119 ceph-mon[31518]: pgmap v3583: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:20:52.938 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:20:52 smithi132 ceph-mon[35941]: pgmap v3583: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:20:54.868 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:20:54 smithi165 ceph-mon[36302]: pgmap v3584: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:20:54.891 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:20:54 smithi119 ceph-mon[31518]: pgmap v3584: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:20:54.938 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:20:54 smithi132 ceph-mon[35941]: pgmap v3584: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:20:55.938 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:20:55 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6315.751047 secs 2024-03-28T02:20:56.118 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:20:55 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6315.751047 secs 2024-03-28T02:20:56.141 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:20:55 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6315.751047 secs 2024-03-28T02:20:56.938 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:20:56 smithi132 ceph-mon[35941]: pgmap v3585: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:20:57.118 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:20:56 smithi165 ceph-mon[36302]: pgmap v3585: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:20:57.141 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:20:56 smithi119 ceph-mon[31518]: pgmap v3585: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:20:57.938 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:20:57 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:20:58.118 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:20:57 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:20:58.141 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:20:57 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:20:58.938 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:20:58 smithi132 ceph-mon[35941]: pgmap v3586: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:20:58.938 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:20:58 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:20:58.939 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:20:58 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:20:58.939 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:20:58 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:20:58.939 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:20:58 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:20:59.118 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:20:58 smithi165 ceph-mon[36302]: pgmap v3586: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:20:59.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:20:58 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:20:59.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:20:58 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:20:59.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:20:58 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:20:59.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:20:58 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:20:59.141 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:20:58 smithi119 ceph-mon[31518]: pgmap v3586: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:20:59.141 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:20:58 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:20:59.141 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:20:58 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:20:59.141 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:20:58 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:20:59.141 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:20:58 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:21:00.939 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:21:00 smithi132 ceph-mon[35941]: pgmap v3587: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:21:00.939 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:21:00 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6320.751111 secs 2024-03-28T02:21:00.939 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:21:00 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:21:00.939 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:21:00 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:21:00.939 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:21:00 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:21:00.939 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:21:00 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:21:01.118 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:21:00 smithi165 ceph-mon[36302]: pgmap v3587: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:21:01.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:21:00 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6320.751111 secs 2024-03-28T02:21:01.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:21:00 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:21:01.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:21:00 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:21:01.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:21:00 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:21:01.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:21:00 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:21:01.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:21:00 smithi119 ceph-mon[31518]: pgmap v3587: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:21:01.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:21:00 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6320.751111 secs 2024-03-28T02:21:01.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:21:00 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:21:01.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:21:00 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:21:01.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:21:00 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:21:01.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:21:00 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:21:02.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:21:01 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:21:02.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:21:01 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:21:02.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:21:01 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:21:02.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:21:01 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:21:02.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:21:01 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:21:02.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:21:01 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:21:02.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:21:01 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:21:02.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:21:01 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:21:02.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:21:01 smithi165 ceph-mon[36302]: Adjusting osd_memory_target on smithi165 to 174.1M 2024-03-28T02:21:02.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:21:01 smithi165 ceph-mon[36302]: Unable to set osd_memory_target on smithi165 to 182635622: error parsing value: Value '182635622' is below minimum 939524096 2024-03-28T02:21:02.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:21:01 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:21:02.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:21:01 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:21:02.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:21:01 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:21:02.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:21:01 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:21:02.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:21:01 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:21:02.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:21:01 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:21:02.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:21:01 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:21:02.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:21:01 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:21:02.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:21:01 smithi165 ceph-mon[36302]: Adjusting osd_memory_target on smithi132 to 174.1M 2024-03-28T02:21:02.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:21:01 smithi165 ceph-mon[36302]: Unable to set osd_memory_target on smithi132 to 182640640: error parsing value: Value '182640640' is below minimum 939524096 2024-03-28T02:21:02.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:21:01 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:21:02.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:21:01 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:21:02.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:21:01 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:21:02.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:21:01 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:21:02.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:21:01 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:21:02.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:21:01 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:21:02.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:21:01 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:21:02.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:21:01 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:21:02.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:21:01 smithi119 ceph-mon[31518]: Adjusting osd_memory_target on smithi165 to 174.1M 2024-03-28T02:21:02.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:21:01 smithi119 ceph-mon[31518]: Unable to set osd_memory_target on smithi165 to 182635622: error parsing value: Value '182635622' is below minimum 939524096 2024-03-28T02:21:02.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:21:01 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:21:02.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:21:01 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:21:02.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:21:01 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:21:02.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:21:01 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:21:02.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:21:01 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:21:02.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:21:01 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:21:02.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:21:01 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:21:02.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:21:01 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:21:02.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:21:01 smithi119 ceph-mon[31518]: Adjusting osd_memory_target on smithi132 to 174.1M 2024-03-28T02:21:02.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:21:01 smithi119 ceph-mon[31518]: Unable to set osd_memory_target on smithi132 to 182640640: error parsing value: Value '182640640' is below minimum 939524096 2024-03-28T02:21:02.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:21:01 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:21:02.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:21:01 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:21:02.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:21:01 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:21:02.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:21:01 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:21:02.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:21:01 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:21:02.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:21:01 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:21:02.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:21:01 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:21:02.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:21:01 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:21:02.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:21:01 smithi132 ceph-mon[35941]: Adjusting osd_memory_target on smithi165 to 174.1M 2024-03-28T02:21:02.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:21:01 smithi132 ceph-mon[35941]: Unable to set osd_memory_target on smithi165 to 182635622: error parsing value: Value '182635622' is below minimum 939524096 2024-03-28T02:21:02.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:21:01 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:21:02.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:21:01 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:21:02.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:21:01 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:21:02.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:21:01 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:21:02.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:21:01 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:21:02.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:21:01 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:21:02.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:21:01 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:21:02.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:21:01 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:21:02.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:21:01 smithi132 ceph-mon[35941]: Adjusting osd_memory_target on smithi132 to 174.1M 2024-03-28T02:21:02.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:21:01 smithi132 ceph-mon[35941]: Unable to set osd_memory_target on smithi132 to 182640640: error parsing value: Value '182640640' is below minimum 939524096 2024-03-28T02:21:03.368 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:21:03 smithi165 ceph-mon[36302]: pgmap v3588: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:21:03.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:21:03 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:21:03.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:21:03 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:21:03.391 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:21:03 smithi119 ceph-mon[31518]: pgmap v3588: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:21:03.391 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:21:03 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:21:03.391 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:21:03 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:21:03.438 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:21:03 smithi132 ceph-mon[35941]: pgmap v3588: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:21:03.439 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:21:03 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:21:03.439 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:21:03 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:21:04.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:21:04 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:21:04.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:21:04 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:21:04.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:21:04 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:21:04.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:21:04 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:21:04.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:21:04 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:21:04.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:21:04 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:21:04.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:21:04 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:21:04.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:21:04 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:21:04.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:21:04 smithi165 ceph-mon[36302]: Adjusting osd_memory_target on smithi119 to 176.7M 2024-03-28T02:21:04.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:21:04 smithi165 ceph-mon[36302]: Unable to set osd_memory_target on smithi119 to 185372364: error parsing value: Value '185372364' is below minimum 939524096 2024-03-28T02:21:04.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:21:04 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:21:04.370 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:21:04 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:21:04.370 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:21:04 smithi165 ceph-mon[36302]: pgmap v3589: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T02:21:04.370 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:21:04 smithi165 ceph-mon[36302]: pgmap v3590: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T02:21:04.370 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:21:04 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:21:04.392 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:21:04 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:21:04.392 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:21:04 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:21:04.392 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:21:04 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:21:04.392 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:21:04 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:21:04.392 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:21:04 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:21:04.392 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:21:04 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:21:04.392 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:21:04 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:21:04.392 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:21:04 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:21:04.392 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:21:04 smithi119 ceph-mon[31518]: Adjusting osd_memory_target on smithi119 to 176.7M 2024-03-28T02:21:04.392 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:21:04 smithi119 ceph-mon[31518]: Unable to set osd_memory_target on smithi119 to 185372364: error parsing value: Value '185372364' is below minimum 939524096 2024-03-28T02:21:04.392 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:21:04 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:21:04.392 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:21:04 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:21:04.393 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:21:04 smithi119 ceph-mon[31518]: pgmap v3589: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T02:21:04.393 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:21:04 smithi119 ceph-mon[31518]: pgmap v3590: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T02:21:04.393 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:21:04 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:21:04.439 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:21:04 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:21:04.439 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:21:04 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:21:04.439 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:21:04 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:21:04.439 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:21:04 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:21:04.439 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:21:04 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:21:04.439 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:21:04 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:21:04.439 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:21:04 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:21:04.439 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:21:04 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:21:04.439 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:21:04 smithi132 ceph-mon[35941]: Adjusting osd_memory_target on smithi119 to 176.7M 2024-03-28T02:21:04.439 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:21:04 smithi132 ceph-mon[35941]: Unable to set osd_memory_target on smithi119 to 185372364: error parsing value: Value '185372364' is below minimum 939524096 2024-03-28T02:21:04.439 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:21:04 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:21:04.439 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:21:04 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:21:04.440 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:21:04 smithi132 ceph-mon[35941]: pgmap v3589: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T02:21:04.440 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:21:04 smithi132 ceph-mon[35941]: pgmap v3590: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T02:21:04.440 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:21:04 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:21:06.689 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:21:06 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6325.751200 secs 2024-03-28T02:21:06.689 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:21:06 smithi132 ceph-mon[35941]: pgmap v3591: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T02:21:06.868 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:21:06 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6325.751200 secs 2024-03-28T02:21:06.868 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:21:06 smithi165 ceph-mon[36302]: pgmap v3591: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T02:21:06.891 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:21:06 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6325.751200 secs 2024-03-28T02:21:06.891 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:21:06 smithi119 ceph-mon[31518]: pgmap v3591: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T02:21:08.573 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:21:08 smithi119 ceph-mon[31518]: pgmap v3592: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T02:21:08.619 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:21:08 smithi165 ceph-mon[36302]: pgmap v3592: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T02:21:08.688 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:21:08 smithi132 ceph-mon[35941]: pgmap v3592: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T02:21:10.688 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:21:10 smithi132 ceph-mon[35941]: pgmap v3593: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T02:21:10.869 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:21:10 smithi165 ceph-mon[36302]: pgmap v3593: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T02:21:10.891 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:21:10 smithi119 ceph-mon[31518]: pgmap v3593: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T02:21:11.868 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:21:11 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6330.751238 secs 2024-03-28T02:21:11.891 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:21:11 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6330.751238 secs 2024-03-28T02:21:11.938 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:21:11 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6330.751238 secs 2024-03-28T02:21:12.868 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:21:12 smithi165 ceph-mon[36302]: pgmap v3594: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T02:21:12.891 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:21:12 smithi119 ceph-mon[31518]: pgmap v3594: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T02:21:12.939 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:21:12 smithi132 ceph-mon[35941]: pgmap v3594: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T02:21:14.869 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:21:14 smithi165 ceph-mon[36302]: pgmap v3595: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T02:21:14.891 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:21:14 smithi119 ceph-mon[31518]: pgmap v3595: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T02:21:14.938 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:21:14 smithi132 ceph-mon[35941]: pgmap v3595: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T02:21:16.869 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:21:16 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6335.751297 secs 2024-03-28T02:21:16.869 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:21:16 smithi165 ceph-mon[36302]: pgmap v3596: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:21:16.892 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:21:16 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6335.751297 secs 2024-03-28T02:21:16.892 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:21:16 smithi119 ceph-mon[31518]: pgmap v3596: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:21:16.939 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:21:16 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6335.751297 secs 2024-03-28T02:21:16.939 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:21:16 smithi132 ceph-mon[35941]: pgmap v3596: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:21:18.619 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:21:18 smithi165 ceph-mon[36302]: pgmap v3597: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:21:18.892 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:21:18 smithi119 ceph-mon[31518]: pgmap v3597: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:21:18.938 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:21:18 smithi132 ceph-mon[35941]: pgmap v3597: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:21:20.869 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:21:20 smithi165 ceph-mon[36302]: pgmap v3598: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:21:20.891 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:21:20 smithi119 ceph-mon[31518]: pgmap v3598: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:21:20.939 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:21:20 smithi132 ceph-mon[35941]: pgmap v3598: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:21:21.869 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:21:21 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6340.751381 secs 2024-03-28T02:21:21.892 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:21:21 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6340.751381 secs 2024-03-28T02:21:21.939 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:21:21 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6340.751381 secs 2024-03-28T02:21:22.642 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:21:22 smithi119 ceph-mon[31518]: pgmap v3599: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:21:22.869 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:21:22 smithi165 ceph-mon[36302]: pgmap v3599: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:21:22.939 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:21:22 smithi132 ceph-mon[35941]: pgmap v3599: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:21:24.741 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:21:24 smithi132 ceph-mon[35941]: pgmap v3600: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:21:24.869 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:21:24 smithi165 ceph-mon[36302]: pgmap v3600: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:21:24.892 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:21:24 smithi119 ceph-mon[31518]: pgmap v3600: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:21:26.869 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:21:26 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6345.751470 secs 2024-03-28T02:21:26.869 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:21:26 smithi165 ceph-mon[36302]: pgmap v3601: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:21:26.892 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:21:26 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6345.751470 secs 2024-03-28T02:21:26.892 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:21:26 smithi119 ceph-mon[31518]: pgmap v3601: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:21:26.939 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:21:26 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6345.751470 secs 2024-03-28T02:21:26.939 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:21:26 smithi132 ceph-mon[35941]: pgmap v3601: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:21:28.869 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:21:28 smithi165 ceph-mon[36302]: pgmap v3602: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:21:28.892 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:21:28 smithi119 ceph-mon[31518]: pgmap v3602: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:21:28.939 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:21:28 smithi132 ceph-mon[35941]: pgmap v3602: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:21:30.869 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:21:30 smithi165 ceph-mon[36302]: pgmap v3603: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:21:30.892 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:21:30 smithi119 ceph-mon[31518]: pgmap v3603: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:21:30.939 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:21:30 smithi132 ceph-mon[35941]: pgmap v3603: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:21:31.869 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:21:31 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6350.751560 secs 2024-03-28T02:21:31.892 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:21:31 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6350.751560 secs 2024-03-28T02:21:31.939 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:21:31 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6350.751560 secs 2024-03-28T02:21:32.869 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:21:32 smithi165 ceph-mon[36302]: pgmap v3604: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:21:32.892 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:21:32 smithi119 ceph-mon[31518]: pgmap v3604: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:21:32.939 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:21:32 smithi132 ceph-mon[35941]: pgmap v3604: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:21:34.869 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:21:34 smithi165 ceph-mon[36302]: pgmap v3605: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:21:34.892 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:21:34 smithi119 ceph-mon[31518]: pgmap v3605: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:21:34.939 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:21:34 smithi132 ceph-mon[35941]: pgmap v3605: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:21:36.869 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:21:36 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6355.751648 secs 2024-03-28T02:21:36.869 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:21:36 smithi165 ceph-mon[36302]: pgmap v3606: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:21:36.892 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:21:36 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6355.751648 secs 2024-03-28T02:21:36.892 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:21:36 smithi119 ceph-mon[31518]: pgmap v3606: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:21:36.939 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:21:36 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6355.751648 secs 2024-03-28T02:21:36.939 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:21:36 smithi132 ceph-mon[35941]: pgmap v3606: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:21:38.869 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:21:38 smithi165 ceph-mon[36302]: pgmap v3607: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:21:38.892 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:21:38 smithi119 ceph-mon[31518]: pgmap v3607: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:21:38.939 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:21:38 smithi132 ceph-mon[35941]: pgmap v3607: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:21:40.869 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:21:40 smithi165 ceph-mon[36302]: pgmap v3608: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:21:40.892 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:21:40 smithi119 ceph-mon[31518]: pgmap v3608: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:21:40.940 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:21:40 smithi132 ceph-mon[35941]: pgmap v3608: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:21:41.869 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:21:41 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6360.751707 secs 2024-03-28T02:21:41.892 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:21:41 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6360.751707 secs 2024-03-28T02:21:41.939 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:21:41 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6360.751707 secs 2024-03-28T02:21:42.869 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:21:42 smithi165 ceph-mon[36302]: pgmap v3609: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:21:42.892 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:21:42 smithi119 ceph-mon[31518]: pgmap v3609: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:21:42.939 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:21:42 smithi132 ceph-mon[35941]: pgmap v3609: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:21:44.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:21:44 smithi119 ceph-mon[31518]: pgmap v3610: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:21:44.939 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:21:44 smithi132 ceph-mon[35941]: pgmap v3610: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:21:45.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:21:44 smithi165 ceph-mon[36302]: pgmap v3610: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:21:45.892 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:21:45 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6365.751768 secs 2024-03-28T02:21:45.939 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:21:45 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6365.751768 secs 2024-03-28T02:21:46.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:21:45 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6365.751768 secs 2024-03-28T02:21:46.892 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:21:46 smithi119 ceph-mon[31518]: pgmap v3611: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:21:46.939 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:21:46 smithi132 ceph-mon[35941]: pgmap v3611: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:21:47.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:21:46 smithi165 ceph-mon[36302]: pgmap v3611: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:21:48.892 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:21:48 smithi119 ceph-mon[31518]: pgmap v3612: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:21:48.939 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:21:48 smithi132 ceph-mon[35941]: pgmap v3612: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:21:49.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:21:48 smithi165 ceph-mon[36302]: pgmap v3612: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:21:50.939 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:21:50 smithi132 ceph-mon[35941]: pgmap v3613: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:21:50.940 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:21:50 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6370.751828 secs 2024-03-28T02:21:51.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:21:50 smithi165 ceph-mon[36302]: pgmap v3613: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:21:51.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:21:50 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6370.751828 secs 2024-03-28T02:21:51.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:21:50 smithi119 ceph-mon[31518]: pgmap v3613: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:21:51.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:21:50 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6370.751828 secs 2024-03-28T02:21:52.939 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:21:52 smithi132 ceph-mon[35941]: pgmap v3614: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:21:53.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:21:52 smithi165 ceph-mon[36302]: pgmap v3614: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:21:53.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:21:52 smithi119 ceph-mon[31518]: pgmap v3614: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:21:54.939 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:21:54 smithi132 ceph-mon[35941]: pgmap v3615: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:21:55.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:21:54 smithi165 ceph-mon[36302]: pgmap v3615: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:21:55.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:21:54 smithi119 ceph-mon[31518]: pgmap v3615: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:21:55.939 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:21:55 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6375.751901 secs 2024-03-28T02:21:56.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:21:55 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6375.751901 secs 2024-03-28T02:21:56.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:21:55 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6375.751901 secs 2024-03-28T02:21:56.939 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:21:56 smithi132 ceph-mon[35941]: pgmap v3616: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:21:57.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:21:56 smithi165 ceph-mon[36302]: pgmap v3616: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:21:57.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:21:56 smithi119 ceph-mon[31518]: pgmap v3616: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:21:58.940 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:21:58 smithi132 ceph-mon[35941]: pgmap v3617: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:21:59.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:21:58 smithi165 ceph-mon[36302]: pgmap v3617: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:21:59.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:21:58 smithi119 ceph-mon[31518]: pgmap v3617: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:22:00.940 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:22:00 smithi132 ceph-mon[35941]: pgmap v3618: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:22:00.940 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:22:00 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6380.751963 secs 2024-03-28T02:22:01.119 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:22:00 smithi165 ceph-mon[36302]: pgmap v3618: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:22:01.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:22:00 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6380.751963 secs 2024-03-28T02:22:01.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:22:00 smithi119 ceph-mon[31518]: pgmap v3618: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:22:01.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:22:00 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6380.751963 secs 2024-03-28T02:22:02.939 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:22:02 smithi132 ceph-mon[35941]: pgmap v3619: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:22:03.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:22:02 smithi165 ceph-mon[36302]: pgmap v3619: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:22:03.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:22:02 smithi119 ceph-mon[31518]: pgmap v3619: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:22:03.940 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:22:03 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:22:04.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:22:03 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:22:04.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:22:03 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:22:05.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:22:04 smithi165 ceph-mon[36302]: pgmap v3620: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:22:05.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:22:04 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:22:05.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:22:04 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:22:05.144 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:22:04 smithi119 ceph-mon[31518]: pgmap v3620: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:22:05.144 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:22:04 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:22:05.144 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:22:04 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:22:05.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:22:04 smithi132 ceph-mon[35941]: pgmap v3620: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:22:05.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:22:04 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:22:05.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:22:04 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:22:06.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:22:05 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:22:06.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:22:05 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:22:06.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:22:05 smithi165 ceph-mon[36302]: pgmap v3621: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T02:22:06.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:22:05 smithi165 ceph-mon[36302]: pgmap v3622: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:22:06.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:22:05 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:22:06.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:22:05 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6385.752028 secs 2024-03-28T02:22:06.045 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:22:05 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:22:06.045 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:22:05 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:22:06.046 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:22:05 smithi119 ceph-mon[31518]: pgmap v3621: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T02:22:06.046 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:22:05 smithi119 ceph-mon[31518]: pgmap v3622: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:22:06.046 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:22:05 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:22:06.046 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:22:05 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6385.752028 secs 2024-03-28T02:22:06.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:22:05 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:22:06.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:22:05 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:22:06.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:22:05 smithi132 ceph-mon[35941]: pgmap v3621: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T02:22:06.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:22:05 smithi132 ceph-mon[35941]: pgmap v3622: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:22:06.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:22:05 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:22:06.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:22:05 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6385.752028 secs 2024-03-28T02:22:08.370 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:22:08 smithi165 ceph-mon[36302]: pgmap v3623: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:22:08.393 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:22:08 smithi119 ceph-mon[31518]: pgmap v3623: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:22:08.440 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:22:08 smithi132 ceph-mon[35941]: pgmap v3623: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:22:10.370 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:22:10 smithi165 ceph-mon[36302]: pgmap v3624: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-28T02:22:10.393 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:22:10 smithi119 ceph-mon[31518]: pgmap v3624: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-28T02:22:10.440 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:22:10 smithi132 ceph-mon[35941]: pgmap v3624: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-28T02:22:11.370 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:22:11 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6390.752114 secs 2024-03-28T02:22:11.392 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:22:11 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6390.752114 secs 2024-03-28T02:22:11.439 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:22:11 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6390.752114 secs 2024-03-28T02:22:12.370 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:22:12 smithi165 ceph-mon[36302]: pgmap v3625: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:22:12.392 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:22:12 smithi119 ceph-mon[31518]: pgmap v3625: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:22:12.440 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:22:12 smithi132 ceph-mon[35941]: pgmap v3625: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:22:14.370 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:22:14 smithi165 ceph-mon[36302]: pgmap v3626: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:22:14.393 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:22:14 smithi119 ceph-mon[31518]: pgmap v3626: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:22:14.441 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:22:14 smithi132 ceph-mon[35941]: pgmap v3626: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:22:16.370 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:22:16 smithi165 ceph-mon[36302]: pgmap v3627: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T02:22:16.370 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:22:16 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6395.752176 secs 2024-03-28T02:22:16.393 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:22:16 smithi119 ceph-mon[31518]: pgmap v3627: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T02:22:16.393 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:22:16 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6395.752176 secs 2024-03-28T02:22:16.440 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:22:16 smithi132 ceph-mon[35941]: pgmap v3627: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T02:22:16.440 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:22:16 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6395.752176 secs 2024-03-28T02:22:18.370 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:22:18 smithi165 ceph-mon[36302]: pgmap v3628: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:22:18.393 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:22:18 smithi119 ceph-mon[31518]: pgmap v3628: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:22:18.440 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:22:18 smithi132 ceph-mon[35941]: pgmap v3628: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:22:20.370 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:22:20 smithi165 ceph-mon[36302]: pgmap v3629: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:22:20.393 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:22:20 smithi119 ceph-mon[31518]: pgmap v3629: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:22:20.440 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:22:20 smithi132 ceph-mon[35941]: pgmap v3629: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:22:21.370 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:22:21 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6400.752267 secs 2024-03-28T02:22:21.370 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:22:21 smithi165 ceph-mon[36302]: pgmap v3630: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:22:21.393 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:22:21 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6400.752267 secs 2024-03-28T02:22:21.393 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:22:21 smithi119 ceph-mon[31518]: pgmap v3630: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:22:21.440 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:22:21 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6400.752267 secs 2024-03-28T02:22:21.440 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:22:21 smithi132 ceph-mon[35941]: pgmap v3630: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:22:24.370 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:22:24 smithi165 ceph-mon[36302]: pgmap v3631: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T02:22:24.393 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:22:24 smithi119 ceph-mon[31518]: pgmap v3631: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T02:22:24.440 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:22:24 smithi132 ceph-mon[35941]: pgmap v3631: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T02:22:26.370 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:22:26 smithi165 ceph-mon[36302]: pgmap v3632: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:22:26.370 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:22:26 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6405.752330 secs 2024-03-28T02:22:26.393 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:22:26 smithi119 ceph-mon[31518]: pgmap v3632: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:22:26.393 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:22:26 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6405.752330 secs 2024-03-28T02:22:26.440 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:22:26 smithi132 ceph-mon[35941]: pgmap v3632: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:22:26.440 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:22:26 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6405.752330 secs 2024-03-28T02:22:28.370 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:22:28 smithi165 ceph-mon[36302]: pgmap v3633: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T02:22:28.393 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:22:28 smithi119 ceph-mon[31518]: pgmap v3633: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T02:22:28.440 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:22:28 smithi132 ceph-mon[35941]: pgmap v3633: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T02:22:30.370 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:22:30 smithi165 ceph-mon[36302]: pgmap v3634: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T02:22:30.393 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:22:30 smithi119 ceph-mon[31518]: pgmap v3634: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T02:22:30.440 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:22:30 smithi132 ceph-mon[35941]: pgmap v3634: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T02:22:31.370 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:22:31 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6410.752422 secs 2024-03-28T02:22:31.393 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:22:31 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6410.752422 secs 2024-03-28T02:22:31.440 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:22:31 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6410.752422 secs 2024-03-28T02:22:32.370 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:22:32 smithi165 ceph-mon[36302]: pgmap v3635: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:22:32.393 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:22:32 smithi119 ceph-mon[31518]: pgmap v3635: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:22:32.440 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:22:32 smithi132 ceph-mon[35941]: pgmap v3635: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:22:33.370 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:22:33 smithi165 ceph-mon[36302]: pgmap v3636: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:22:33.393 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:22:33 smithi119 ceph-mon[31518]: pgmap v3636: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:22:33.440 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:22:33 smithi132 ceph-mon[35941]: pgmap v3636: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:22:36.370 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:22:36 smithi165 ceph-mon[36302]: pgmap v3637: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:22:36.370 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:22:36 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6415.752510 secs 2024-03-28T02:22:36.393 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:22:36 smithi119 ceph-mon[31518]: pgmap v3637: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:22:36.393 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:22:36 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6415.752510 secs 2024-03-28T02:22:36.440 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:22:36 smithi132 ceph-mon[35941]: pgmap v3637: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:22:36.440 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:22:36 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6415.752510 secs 2024-03-28T02:22:38.370 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:22:38 smithi165 ceph-mon[36302]: pgmap v3638: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T02:22:38.393 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:22:38 smithi119 ceph-mon[31518]: pgmap v3638: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T02:22:38.440 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:22:38 smithi132 ceph-mon[35941]: pgmap v3638: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T02:22:40.373 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:22:40 smithi165 ceph-mon[36302]: pgmap v3639: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T02:22:40.393 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:22:40 smithi119 ceph-mon[31518]: pgmap v3639: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T02:22:40.440 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:22:40 smithi132 ceph-mon[35941]: pgmap v3639: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T02:22:41.370 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:22:41 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6420.752573 secs 2024-03-28T02:22:41.393 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:22:41 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6420.752573 secs 2024-03-28T02:22:41.440 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:22:41 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6420.752573 secs 2024-03-28T02:22:42.370 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:22:42 smithi165 ceph-mon[36302]: pgmap v3640: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:22:42.393 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:22:42 smithi119 ceph-mon[31518]: pgmap v3640: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:22:42.440 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:22:42 smithi132 ceph-mon[35941]: pgmap v3640: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:22:43.370 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:22:43 smithi165 ceph-mon[36302]: pgmap v3641: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T02:22:43.393 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:22:43 smithi119 ceph-mon[31518]: pgmap v3641: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T02:22:43.440 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:22:43 smithi132 ceph-mon[35941]: pgmap v3641: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T02:22:46.370 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:22:46 smithi165 ceph-mon[36302]: pgmap v3642: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:22:46.371 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:22:46 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6425.752614 secs 2024-03-28T02:22:46.393 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:22:46 smithi119 ceph-mon[31518]: pgmap v3642: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:22:46.393 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:22:46 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6425.752614 secs 2024-03-28T02:22:46.440 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:22:46 smithi132 ceph-mon[35941]: pgmap v3642: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:22:46.440 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:22:46 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6425.752614 secs 2024-03-28T02:22:48.370 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:22:48 smithi165 ceph-mon[36302]: pgmap v3643: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:22:48.393 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:22:48 smithi119 ceph-mon[31518]: pgmap v3643: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:22:48.440 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:22:48 smithi132 ceph-mon[35941]: pgmap v3643: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:22:50.371 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:22:50 smithi165 ceph-mon[36302]: pgmap v3644: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:22:50.393 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:22:50 smithi119 ceph-mon[31518]: pgmap v3644: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:22:50.440 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:22:50 smithi132 ceph-mon[35941]: pgmap v3644: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:22:51.370 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:22:51 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6430.752704 secs 2024-03-28T02:22:51.393 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:22:51 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6430.752704 secs 2024-03-28T02:22:51.440 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:22:51 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6430.752704 secs 2024-03-28T02:22:52.370 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:22:52 smithi165 ceph-mon[36302]: pgmap v3645: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:22:52.393 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:22:52 smithi119 ceph-mon[31518]: pgmap v3645: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:22:52.440 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:22:52 smithi132 ceph-mon[35941]: pgmap v3645: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:22:53.370 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:22:53 smithi165 ceph-mon[36302]: pgmap v3646: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:22:53.393 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:22:53 smithi119 ceph-mon[31518]: pgmap v3646: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:22:53.440 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:22:53 smithi132 ceph-mon[35941]: pgmap v3646: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:22:56.371 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:22:56 smithi165 ceph-mon[36302]: pgmap v3647: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:22:56.371 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:22:56 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6435.752744 secs 2024-03-28T02:22:56.394 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:22:56 smithi119 ceph-mon[31518]: pgmap v3647: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:22:56.394 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:22:56 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6435.752744 secs 2024-03-28T02:22:56.440 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:22:56 smithi132 ceph-mon[35941]: pgmap v3647: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:22:56.441 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:22:56 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6435.752744 secs 2024-03-28T02:22:58.371 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:22:58 smithi165 ceph-mon[36302]: pgmap v3648: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:22:58.393 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:22:58 smithi119 ceph-mon[31518]: pgmap v3648: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:22:58.440 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:22:58 smithi132 ceph-mon[35941]: pgmap v3648: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:23:00.371 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:23:00 smithi165 ceph-mon[36302]: pgmap v3649: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:23:00.394 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:23:00 smithi119 ceph-mon[31518]: pgmap v3649: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:23:00.440 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:23:00 smithi132 ceph-mon[35941]: pgmap v3649: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:23:01.371 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:23:01 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6440.752808 secs 2024-03-28T02:23:01.393 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:23:01 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6440.752808 secs 2024-03-28T02:23:01.441 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:23:01 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6440.752808 secs 2024-03-28T02:23:02.371 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:23:02 smithi165 ceph-mon[36302]: pgmap v3650: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:23:02.393 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:23:02 smithi119 ceph-mon[31518]: pgmap v3650: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:23:02.441 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:23:02 smithi132 ceph-mon[35941]: pgmap v3650: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:23:04.371 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:23:04 smithi165 ceph-mon[36302]: pgmap v3651: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:23:04.393 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:23:04 smithi119 ceph-mon[31518]: pgmap v3651: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:23:04.441 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:23:04 smithi132 ceph-mon[35941]: pgmap v3651: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:23:05.371 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:23:05 smithi165 ceph-mon[36302]: pgmap v3652: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:23:05.393 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:23:05 smithi119 ceph-mon[31518]: pgmap v3652: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:23:05.441 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:23:05 smithi132 ceph-mon[35941]: pgmap v3652: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:23:06.371 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:23:06 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6445.752823 secs 2024-03-28T02:23:06.371 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:23:06 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:23:06.394 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:23:06 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6445.752823 secs 2024-03-28T02:23:06.394 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:23:06 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:23:06.441 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:23:06 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6445.752823 secs 2024-03-28T02:23:06.441 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:23:06 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:23:07.191 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:23:07 smithi132 ceph-mon[35941]: pgmap v3653: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:23:07.371 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:23:07 smithi165 ceph-mon[36302]: pgmap v3653: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:23:07.393 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:23:07 smithi119 ceph-mon[31518]: pgmap v3653: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:23:10.371 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:23:10 smithi165 ceph-mon[36302]: pgmap v3654: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:23:10.371 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:23:10 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:23:10.371 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:23:10 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:23:10.394 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:23:10 smithi119 ceph-mon[31518]: pgmap v3654: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:23:10.394 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:23:10 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:23:10.394 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:23:10 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:23:10.441 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:23:10 smithi132 ceph-mon[35941]: pgmap v3654: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:23:10.441 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:23:10 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:23:10.441 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:23:10 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:23:11.144 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:23:11 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6450.752862 secs 2024-03-28T02:23:11.371 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:23:11 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6450.752862 secs 2024-03-28T02:23:11.441 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:23:11 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6450.752862 secs 2024-03-28T02:23:12.371 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:23:12 smithi165 ceph-mon[36302]: pgmap v3655: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:23:12.394 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:23:12 smithi119 ceph-mon[31518]: pgmap v3655: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:23:12.441 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:23:12 smithi132 ceph-mon[35941]: pgmap v3655: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:23:13.621 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:23:13 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:23:13.621 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:23:13 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:23:13.622 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:23:13 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:23:13.622 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:23:13 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:23:13.622 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:23:13 smithi165 ceph-mon[36302]: pgmap v3656: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-28T02:23:13.622 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:23:13 smithi165 ceph-mon[36302]: pgmap v3657: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:23:13.622 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:23:13 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:23:13.644 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:23:13 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:23:13.644 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:23:13 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:23:13.644 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:23:13 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:23:13.645 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:23:13 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:23:13.645 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:23:13 smithi119 ceph-mon[31518]: pgmap v3656: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-28T02:23:13.645 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:23:13 smithi119 ceph-mon[31518]: pgmap v3657: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:23:13.645 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:23:13 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:23:13.691 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:23:13 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:23:13.691 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:23:13 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:23:13.691 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:23:13 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:23:13.691 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:23:13 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:23:13.692 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:23:13 smithi132 ceph-mon[35941]: pgmap v3656: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-28T02:23:13.692 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:23:13 smithi132 ceph-mon[35941]: pgmap v3657: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:23:13.692 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:23:13 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:23:15.621 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:23:15 smithi165 ceph-mon[36302]: pgmap v3658: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 5 op/s 2024-03-28T02:23:15.644 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:23:15 smithi119 ceph-mon[31518]: pgmap v3658: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 5 op/s 2024-03-28T02:23:15.691 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:23:15 smithi132 ceph-mon[35941]: pgmap v3658: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 5 op/s 2024-03-28T02:23:16.621 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:23:16 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6455.752945 secs 2024-03-28T02:23:16.644 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:23:16 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6455.752945 secs 2024-03-28T02:23:16.691 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:23:16 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6455.752945 secs 2024-03-28T02:23:17.621 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:23:17 smithi165 ceph-mon[36302]: pgmap v3659: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:23:17.644 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:23:17 smithi119 ceph-mon[31518]: pgmap v3659: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:23:17.691 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:23:17 smithi132 ceph-mon[35941]: pgmap v3659: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:23:19.621 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:23:19 smithi165 ceph-mon[36302]: pgmap v3660: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:23:19.644 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:23:19 smithi119 ceph-mon[31518]: pgmap v3660: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:23:19.691 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:23:19 smithi132 ceph-mon[35941]: pgmap v3660: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:23:21.621 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:23:21 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6460.753006 secs 2024-03-28T02:23:21.621 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:23:21 smithi165 ceph-mon[36302]: pgmap v3661: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:23:21.644 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:23:21 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6460.753006 secs 2024-03-28T02:23:21.644 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:23:21 smithi119 ceph-mon[31518]: pgmap v3661: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:23:21.691 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:23:21 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6460.753006 secs 2024-03-28T02:23:21.691 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:23:21 smithi132 ceph-mon[35941]: pgmap v3661: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:23:23.621 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:23:23 smithi165 ceph-mon[36302]: pgmap v3662: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T02:23:23.644 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:23:23 smithi119 ceph-mon[31518]: pgmap v3662: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T02:23:23.691 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:23:23 smithi132 ceph-mon[35941]: pgmap v3662: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T02:23:25.621 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:23:25 smithi165 ceph-mon[36302]: pgmap v3663: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-28T02:23:25.644 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:23:25 smithi119 ceph-mon[31518]: pgmap v3663: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-28T02:23:25.691 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:23:25 smithi132 ceph-mon[35941]: pgmap v3663: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-28T02:23:26.546 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:23:26 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6465.753069 secs 2024-03-28T02:23:26.621 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:23:26 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6465.753069 secs 2024-03-28T02:23:26.691 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:23:26 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6465.753069 secs 2024-03-28T02:23:27.621 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:23:27 smithi165 ceph-mon[36302]: pgmap v3664: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-03-28T02:23:27.644 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:23:27 smithi119 ceph-mon[31518]: pgmap v3664: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-03-28T02:23:27.691 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:23:27 smithi132 ceph-mon[35941]: pgmap v3664: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-03-28T02:23:29.394 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:23:29 smithi119 ceph-mon[31518]: pgmap v3665: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:23:29.621 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:23:29 smithi165 ceph-mon[36302]: pgmap v3665: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:23:29.691 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:23:29 smithi132 ceph-mon[35941]: pgmap v3665: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:23:31.621 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:23:31 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6470.753130 secs 2024-03-28T02:23:31.621 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:23:31 smithi165 ceph-mon[36302]: pgmap v3666: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:23:31.644 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:23:31 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6470.753130 secs 2024-03-28T02:23:31.644 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:23:31 smithi119 ceph-mon[31518]: pgmap v3666: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:23:31.691 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:23:31 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6470.753130 secs 2024-03-28T02:23:31.691 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:23:31 smithi132 ceph-mon[35941]: pgmap v3666: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:23:33.621 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:23:33 smithi165 ceph-mon[36302]: pgmap v3667: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:23:33.644 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:23:33 smithi119 ceph-mon[31518]: pgmap v3667: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:23:33.691 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:23:33 smithi132 ceph-mon[35941]: pgmap v3667: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:23:35.644 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:23:35 smithi119 ceph-mon[31518]: pgmap v3668: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-28T02:23:35.691 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:23:35 smithi132 ceph-mon[35941]: pgmap v3668: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-28T02:23:35.871 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:23:35 smithi165 ceph-mon[36302]: pgmap v3668: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-28T02:23:36.644 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:23:36 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6475.753221 secs 2024-03-28T02:23:36.691 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:23:36 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6475.753221 secs 2024-03-28T02:23:36.772 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:23:36 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6475.753221 secs 2024-03-28T02:23:37.644 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:23:37 smithi119 ceph-mon[31518]: pgmap v3669: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-03-28T02:23:37.691 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:23:37 smithi132 ceph-mon[35941]: pgmap v3669: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-03-28T02:23:37.871 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:23:37 smithi165 ceph-mon[36302]: pgmap v3669: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-03-28T02:23:39.644 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:23:39 smithi119 ceph-mon[31518]: pgmap v3670: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:23:39.691 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:23:39 smithi132 ceph-mon[35941]: pgmap v3670: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:23:39.871 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:23:39 smithi165 ceph-mon[36302]: pgmap v3670: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:23:41.692 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:23:41 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6480.753309 secs 2024-03-28T02:23:41.692 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:23:41 smithi132 ceph-mon[35941]: pgmap v3671: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:23:41.872 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:23:41 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6480.753309 secs 2024-03-28T02:23:41.872 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:23:41 smithi165 ceph-mon[36302]: pgmap v3671: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:23:41.894 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:23:41 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6480.753309 secs 2024-03-28T02:23:41.894 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:23:41 smithi119 ceph-mon[31518]: pgmap v3671: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:23:43.691 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:23:43 smithi132 ceph-mon[35941]: pgmap v3672: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:23:43.871 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:23:43 smithi165 ceph-mon[36302]: pgmap v3672: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:23:43.894 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:23:43 smithi119 ceph-mon[31518]: pgmap v3672: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:23:46.122 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:23:45 smithi165 ceph-mon[36302]: pgmap v3673: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-28T02:23:46.122 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:23:45 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6485.753369 secs 2024-03-28T02:23:46.144 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:23:45 smithi119 ceph-mon[31518]: pgmap v3673: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-28T02:23:46.145 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:23:45 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6485.753369 secs 2024-03-28T02:23:46.191 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:23:45 smithi132 ceph-mon[35941]: pgmap v3673: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-28T02:23:46.191 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:23:45 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6485.753369 secs 2024-03-28T02:23:48.121 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:23:47 smithi165 ceph-mon[36302]: pgmap v3674: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-03-28T02:23:48.144 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:23:47 smithi119 ceph-mon[31518]: pgmap v3674: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-03-28T02:23:48.191 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:23:47 smithi132 ceph-mon[35941]: pgmap v3674: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-03-28T02:23:50.122 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:23:49 smithi165 ceph-mon[36302]: pgmap v3675: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:23:50.144 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:23:49 smithi119 ceph-mon[31518]: pgmap v3675: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:23:50.191 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:23:49 smithi132 ceph-mon[35941]: pgmap v3675: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:23:51.121 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:23:50 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6490.753458 secs 2024-03-28T02:23:51.144 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:23:50 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6490.753458 secs 2024-03-28T02:23:51.191 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:23:50 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6490.753458 secs 2024-03-28T02:23:52.121 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:23:51 smithi165 ceph-mon[36302]: pgmap v3676: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:23:52.144 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:23:51 smithi119 ceph-mon[31518]: pgmap v3676: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:23:52.191 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:23:51 smithi132 ceph-mon[35941]: pgmap v3676: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:23:54.121 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:23:53 smithi165 ceph-mon[36302]: pgmap v3677: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:23:54.144 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:23:53 smithi119 ceph-mon[31518]: pgmap v3677: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:23:54.191 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:23:53 smithi132 ceph-mon[35941]: pgmap v3677: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:23:56.121 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:23:55 smithi165 ceph-mon[36302]: pgmap v3678: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-28T02:23:56.122 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:23:55 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6495.753544 secs 2024-03-28T02:23:56.145 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:23:55 smithi119 ceph-mon[31518]: pgmap v3678: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-28T02:23:56.145 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:23:55 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6495.753544 secs 2024-03-28T02:23:56.191 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:23:55 smithi132 ceph-mon[35941]: pgmap v3678: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-28T02:23:56.191 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:23:55 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6495.753544 secs 2024-03-28T02:23:57.994 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:23:57 smithi132 ceph-mon[35941]: pgmap v3679: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-03-28T02:23:58.122 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:23:57 smithi165 ceph-mon[36302]: pgmap v3679: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-03-28T02:23:58.144 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:23:57 smithi119 ceph-mon[31518]: pgmap v3679: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-03-28T02:23:59.828 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:23:59 smithi119 ceph-mon[31518]: pgmap v3680: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:24:00.122 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:23:59 smithi165 ceph-mon[36302]: pgmap v3680: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:24:00.192 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:23:59 smithi132 ceph-mon[35941]: pgmap v3680: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:24:01.122 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:24:00 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6500.753603 secs 2024-03-28T02:24:01.145 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:24:00 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6500.753603 secs 2024-03-28T02:24:01.192 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:24:00 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6500.753603 secs 2024-03-28T02:24:02.122 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:24:01 smithi165 ceph-mon[36302]: pgmap v3681: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:24:02.144 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:24:01 smithi119 ceph-mon[31518]: pgmap v3681: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:24:02.192 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:24:01 smithi132 ceph-mon[35941]: pgmap v3681: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:24:04.122 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:24:03 smithi165 ceph-mon[36302]: pgmap v3682: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:24:04.145 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:24:03 smithi119 ceph-mon[31518]: pgmap v3682: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:24:04.192 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:24:03 smithi132 ceph-mon[35941]: pgmap v3682: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:24:06.122 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:24:05 smithi165 ceph-mon[36302]: pgmap v3683: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-28T02:24:06.122 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:24:05 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6505.753692 secs 2024-03-28T02:24:06.145 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:24:05 smithi119 ceph-mon[31518]: pgmap v3683: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-28T02:24:06.145 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:24:05 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6505.753692 secs 2024-03-28T02:24:06.192 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:24:05 smithi132 ceph-mon[35941]: pgmap v3683: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-28T02:24:06.192 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:24:05 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6505.753692 secs 2024-03-28T02:24:08.122 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:24:07 smithi165 ceph-mon[36302]: pgmap v3684: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-28T02:24:08.145 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:24:07 smithi119 ceph-mon[31518]: pgmap v3684: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-28T02:24:08.192 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:24:07 smithi132 ceph-mon[35941]: pgmap v3684: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-28T02:24:10.122 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:24:09 smithi165 ceph-mon[36302]: pgmap v3685: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:24:10.145 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:24:09 smithi119 ceph-mon[31518]: pgmap v3685: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:24:10.192 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:24:09 smithi132 ceph-mon[35941]: pgmap v3685: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:24:11.122 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:24:10 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6510.753781 secs 2024-03-28T02:24:11.145 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:24:10 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6510.753781 secs 2024-03-28T02:24:11.192 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:24:10 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6510.753781 secs 2024-03-28T02:24:12.122 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:24:11 smithi165 ceph-mon[36302]: pgmap v3686: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:24:12.144 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:24:11 smithi119 ceph-mon[31518]: pgmap v3686: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:24:12.192 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:24:11 smithi132 ceph-mon[35941]: pgmap v3686: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:24:13.122 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:24:12 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:24:13.144 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:24:12 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:24:13.192 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:24:12 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:24:14.122 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:24:13 smithi165 ceph-mon[36302]: pgmap v3687: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:24:14.144 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:24:13 smithi119 ceph-mon[31518]: pgmap v3687: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:24:14.192 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:24:13 smithi132 ceph-mon[35941]: pgmap v3687: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:24:16.122 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:24:15 smithi165 ceph-mon[36302]: pgmap v3688: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-28T02:24:16.123 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:24:15 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6515.753872 secs 2024-03-28T02:24:16.145 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:24:15 smithi119 ceph-mon[31518]: pgmap v3688: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-28T02:24:16.145 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:24:15 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6515.753872 secs 2024-03-28T02:24:16.192 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:24:15 smithi132 ceph-mon[35941]: pgmap v3688: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-28T02:24:16.192 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:24:15 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6515.753872 secs 2024-03-28T02:24:17.872 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:24:17 smithi165 ceph-mon[36302]: pgmap v3689: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-28T02:24:17.872 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:24:17 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:24:17.873 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:24:17 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:24:17.873 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:24:17 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:24:17.873 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:24:17 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:24:17.873 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:24:17 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:24:18.145 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:24:17 smithi119 ceph-mon[31518]: pgmap v3689: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-28T02:24:18.145 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:24:17 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:24:18.145 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:24:17 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:24:18.145 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:24:17 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:24:18.145 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:24:17 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:24:18.146 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:24:17 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:24:18.192 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:24:17 smithi132 ceph-mon[35941]: pgmap v3689: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-28T02:24:18.192 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:24:17 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:24:18.192 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:24:17 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:24:18.192 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:24:17 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:24:18.193 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:24:17 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:24:18.193 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:24:17 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:24:19.122 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:24:18 smithi165 ceph-mon[36302]: pgmap v3690: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-03-28T02:24:19.122 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:24:18 smithi165 ceph-mon[36302]: pgmap v3691: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T02:24:19.145 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:24:18 smithi119 ceph-mon[31518]: pgmap v3690: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-03-28T02:24:19.145 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:24:18 smithi119 ceph-mon[31518]: pgmap v3691: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T02:24:19.192 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:24:18 smithi132 ceph-mon[35941]: pgmap v3690: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-03-28T02:24:19.192 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:24:18 smithi132 ceph-mon[35941]: pgmap v3691: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T02:24:21.122 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:24:20 smithi165 ceph-mon[36302]: pgmap v3692: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 4 op/s 2024-03-28T02:24:21.123 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:24:20 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6520.753903 secs 2024-03-28T02:24:21.145 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:24:20 smithi119 ceph-mon[31518]: pgmap v3692: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 4 op/s 2024-03-28T02:24:21.145 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:24:20 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6520.753903 secs 2024-03-28T02:24:21.192 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:24:20 smithi132 ceph-mon[35941]: pgmap v3692: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 4 op/s 2024-03-28T02:24:21.192 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:24:20 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6520.753903 secs 2024-03-28T02:24:23.122 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:24:22 smithi165 ceph-mon[36302]: pgmap v3693: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T02:24:23.145 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:24:22 smithi119 ceph-mon[31518]: pgmap v3693: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T02:24:23.192 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:24:22 smithi132 ceph-mon[35941]: pgmap v3693: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T02:24:25.122 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:24:24 smithi165 ceph-mon[36302]: pgmap v3694: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 3 op/s 2024-03-28T02:24:25.145 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:24:24 smithi119 ceph-mon[31518]: pgmap v3694: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 3 op/s 2024-03-28T02:24:25.192 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:24:24 smithi132 ceph-mon[35941]: pgmap v3694: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 3 op/s 2024-03-28T02:24:26.122 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:24:25 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6525.753918 secs 2024-03-28T02:24:26.145 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:24:25 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6525.753918 secs 2024-03-28T02:24:26.192 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:24:25 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6525.753918 secs 2024-03-28T02:24:27.093 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:24:26 smithi132 ceph-mon[35941]: pgmap v3695: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T02:24:27.122 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:24:26 smithi165 ceph-mon[36302]: pgmap v3695: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T02:24:27.145 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:24:26 smithi119 ceph-mon[31518]: pgmap v3695: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T02:24:29.122 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:24:28 smithi165 ceph-mon[36302]: pgmap v3696: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T02:24:29.145 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:24:28 smithi119 ceph-mon[31518]: pgmap v3696: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T02:24:29.192 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:24:28 smithi132 ceph-mon[35941]: pgmap v3696: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T02:24:31.122 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:24:30 smithi165 ceph-mon[36302]: pgmap v3697: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T02:24:31.122 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:24:30 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6530.754002 secs 2024-03-28T02:24:31.145 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:24:30 smithi119 ceph-mon[31518]: pgmap v3697: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T02:24:31.145 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:24:30 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6530.754002 secs 2024-03-28T02:24:31.192 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:24:30 smithi132 ceph-mon[35941]: pgmap v3697: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T02:24:31.192 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:24:30 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6530.754002 secs 2024-03-28T02:24:33.122 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:24:32 smithi165 ceph-mon[36302]: pgmap v3698: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-28T02:24:33.145 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:24:32 smithi119 ceph-mon[31518]: pgmap v3698: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-28T02:24:33.192 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:24:32 smithi132 ceph-mon[35941]: pgmap v3698: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-28T02:24:35.122 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:24:34 smithi165 ceph-mon[36302]: pgmap v3699: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:24:35.145 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:24:34 smithi119 ceph-mon[31518]: pgmap v3699: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:24:35.192 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:24:34 smithi132 ceph-mon[35941]: pgmap v3699: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:24:36.122 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:24:35 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6535.754092 secs 2024-03-28T02:24:36.145 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:24:35 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6535.754092 secs 2024-03-28T02:24:36.192 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:24:35 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6535.754092 secs 2024-03-28T02:24:37.122 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:24:36 smithi165 ceph-mon[36302]: pgmap v3700: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:24:37.145 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:24:36 smithi119 ceph-mon[31518]: pgmap v3700: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:24:37.192 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:24:36 smithi132 ceph-mon[35941]: pgmap v3700: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:24:39.122 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:24:38 smithi165 ceph-mon[36302]: pgmap v3701: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:24:39.145 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:24:38 smithi119 ceph-mon[31518]: pgmap v3701: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:24:39.192 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:24:38 smithi132 ceph-mon[35941]: pgmap v3701: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:24:41.122 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:24:40 smithi165 ceph-mon[36302]: pgmap v3702: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T02:24:41.122 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:24:40 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6540.754183 secs 2024-03-28T02:24:41.145 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:24:40 smithi119 ceph-mon[31518]: pgmap v3702: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T02:24:41.145 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:24:40 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6540.754183 secs 2024-03-28T02:24:41.192 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:24:40 smithi132 ceph-mon[35941]: pgmap v3702: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T02:24:41.192 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:24:40 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6540.754183 secs 2024-03-28T02:24:43.145 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:24:42 smithi119 ceph-mon[31518]: pgmap v3703: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-28T02:24:43.192 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:24:42 smithi132 ceph-mon[35941]: pgmap v3703: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-28T02:24:43.372 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:24:42 smithi165 ceph-mon[36302]: pgmap v3703: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-28T02:24:45.145 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:24:44 smithi119 ceph-mon[31518]: pgmap v3704: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:24:45.192 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:24:44 smithi132 ceph-mon[35941]: pgmap v3704: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:24:45.372 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:24:44 smithi165 ceph-mon[36302]: pgmap v3704: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:24:46.145 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:24:45 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6545.754243 secs 2024-03-28T02:24:46.192 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:24:45 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6545.754243 secs 2024-03-28T02:24:46.372 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:24:45 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6545.754243 secs 2024-03-28T02:24:47.192 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:24:46 smithi132 ceph-mon[35941]: pgmap v3705: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:24:47.372 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:24:46 smithi165 ceph-mon[36302]: pgmap v3705: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:24:47.395 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:24:46 smithi119 ceph-mon[31518]: pgmap v3705: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:24:49.192 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:24:48 smithi132 ceph-mon[35941]: pgmap v3706: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:24:49.372 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:24:48 smithi165 ceph-mon[36302]: pgmap v3706: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:24:49.395 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:24:48 smithi119 ceph-mon[31518]: pgmap v3706: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:24:51.192 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:24:50 smithi132 ceph-mon[35941]: pgmap v3707: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T02:24:51.192 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:24:50 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6550.754334 secs 2024-03-28T02:24:51.372 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:24:50 smithi165 ceph-mon[36302]: pgmap v3707: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T02:24:51.373 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:24:50 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6550.754334 secs 2024-03-28T02:24:51.395 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:24:50 smithi119 ceph-mon[31518]: pgmap v3707: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T02:24:51.395 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:24:50 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6550.754334 secs 2024-03-28T02:24:53.192 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:24:52 smithi132 ceph-mon[35941]: pgmap v3708: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-28T02:24:53.372 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:24:52 smithi165 ceph-mon[36302]: pgmap v3708: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-28T02:24:53.395 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:24:52 smithi119 ceph-mon[31518]: pgmap v3708: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-28T02:24:55.192 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:24:54 smithi132 ceph-mon[35941]: pgmap v3709: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:24:55.372 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:24:54 smithi165 ceph-mon[36302]: pgmap v3709: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:24:55.395 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:24:54 smithi119 ceph-mon[31518]: pgmap v3709: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:24:56.192 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:24:55 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6555.754350 secs 2024-03-28T02:24:56.372 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:24:55 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6555.754350 secs 2024-03-28T02:24:56.395 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:24:55 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6555.754350 secs 2024-03-28T02:24:57.192 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:24:56 smithi132 ceph-mon[35941]: pgmap v3710: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:24:57.372 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:24:56 smithi165 ceph-mon[36302]: pgmap v3710: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:24:57.395 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:24:56 smithi119 ceph-mon[31518]: pgmap v3710: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:24:59.192 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:24:58 smithi132 ceph-mon[35941]: pgmap v3711: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:24:59.372 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:24:58 smithi165 ceph-mon[36302]: pgmap v3711: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:24:59.395 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:24:58 smithi119 ceph-mon[31518]: pgmap v3711: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:25:01.192 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:25:00 smithi132 ceph-mon[35941]: pgmap v3712: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T02:25:01.193 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:25:00 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6560.754436 secs 2024-03-28T02:25:01.198 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:25:00 smithi119 ceph-mon[31518]: pgmap v3712: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T02:25:01.198 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:25:00 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6560.754436 secs 2024-03-28T02:25:01.372 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:25:00 smithi165 ceph-mon[36302]: pgmap v3712: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T02:25:01.373 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:25:00 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6560.754436 secs 2024-03-28T02:25:03.192 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:25:02 smithi132 ceph-mon[35941]: pgmap v3713: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-28T02:25:03.372 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:25:02 smithi165 ceph-mon[36302]: pgmap v3713: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-28T02:25:03.395 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:25:02 smithi119 ceph-mon[31518]: pgmap v3713: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-28T02:25:05.372 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:25:04 smithi165 ceph-mon[36302]: pgmap v3714: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:25:05.395 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:25:04 smithi119 ceph-mon[31518]: pgmap v3714: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:25:05.442 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:25:04 smithi132 ceph-mon[35941]: pgmap v3714: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:25:06.372 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:25:05 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6565.754498 secs 2024-03-28T02:25:06.395 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:25:05 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6565.754498 secs 2024-03-28T02:25:06.442 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:25:05 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6565.754498 secs 2024-03-28T02:25:07.274 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:25:06 smithi165 ceph-mon[36302]: pgmap v3715: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:25:07.297 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:25:06 smithi119 ceph-mon[31518]: pgmap v3715: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:25:07.442 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:25:06 smithi132 ceph-mon[35941]: pgmap v3715: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:25:09.373 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:25:08 smithi165 ceph-mon[36302]: pgmap v3716: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:25:09.396 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:25:08 smithi119 ceph-mon[31518]: pgmap v3716: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:25:09.442 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:25:08 smithi132 ceph-mon[35941]: pgmap v3716: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:25:11.373 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:25:10 smithi165 ceph-mon[36302]: pgmap v3717: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T02:25:11.373 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:25:10 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6570.754559 secs 2024-03-28T02:25:11.395 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:25:10 smithi119 ceph-mon[31518]: pgmap v3717: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T02:25:11.396 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:25:10 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6570.754559 secs 2024-03-28T02:25:11.442 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:25:10 smithi132 ceph-mon[35941]: pgmap v3717: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T02:25:11.443 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:25:10 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6570.754559 secs 2024-03-28T02:25:13.373 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:25:12 smithi165 ceph-mon[36302]: pgmap v3718: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-28T02:25:13.396 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:25:12 smithi119 ceph-mon[31518]: pgmap v3718: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-28T02:25:13.442 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:25:12 smithi132 ceph-mon[35941]: pgmap v3718: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-28T02:25:15.373 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:25:14 smithi165 ceph-mon[36302]: pgmap v3719: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:25:15.396 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:25:14 smithi119 ceph-mon[31518]: pgmap v3719: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:25:15.444 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:25:14 smithi132 ceph-mon[35941]: pgmap v3719: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:25:16.316 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:25:15 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6575.754621 secs 2024-03-28T02:25:16.373 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:25:15 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6575.754621 secs 2024-03-28T02:25:16.442 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:25:15 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6575.754621 secs 2024-03-28T02:25:17.373 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:25:16 smithi165 ceph-mon[36302]: pgmap v3720: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:25:17.395 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:25:16 smithi119 ceph-mon[31518]: pgmap v3720: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:25:17.442 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:25:16 smithi132 ceph-mon[35941]: pgmap v3720: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:25:18.373 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:25:18 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:25:18.395 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:25:18 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:25:18.443 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:25:18 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:25:19.373 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:25:19 smithi165 ceph-mon[36302]: pgmap v3721: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:25:19.373 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:25:19 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:25:19.373 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:25:19 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:25:19.373 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:25:19 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:25:19.396 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:25:19 smithi119 ceph-mon[31518]: pgmap v3721: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:25:19.396 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:25:19 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:25:19.396 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:25:19 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:25:19.396 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:25:19 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:25:19.443 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:25:19 smithi132 ceph-mon[35941]: pgmap v3721: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:25:19.443 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:25:19 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:25:19.443 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:25:19 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:25:19.443 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:25:19 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:25:20.373 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:25:20 smithi165 ceph-mon[36302]: pgmap v3722: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T02:25:20.373 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:25:20 smithi165 ceph-mon[36302]: pgmap v3723: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-03-28T02:25:20.395 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:25:20 smithi119 ceph-mon[31518]: pgmap v3722: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T02:25:20.396 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:25:20 smithi119 ceph-mon[31518]: pgmap v3723: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-03-28T02:25:20.443 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:25:20 smithi132 ceph-mon[35941]: pgmap v3722: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T02:25:20.443 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:25:20 smithi132 ceph-mon[35941]: pgmap v3723: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-03-28T02:25:21.373 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:25:21 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6580.754711 secs 2024-03-28T02:25:21.395 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:25:21 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6580.754711 secs 2024-03-28T02:25:21.443 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:25:21 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6580.754711 secs 2024-03-28T02:25:22.373 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:25:22 smithi165 ceph-mon[36302]: pgmap v3724: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:25:22.396 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:25:22 smithi119 ceph-mon[31518]: pgmap v3724: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:25:22.443 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:25:22 smithi132 ceph-mon[35941]: pgmap v3724: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:25:24.373 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:25:24 smithi165 ceph-mon[36302]: pgmap v3725: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:25:24.396 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:25:24 smithi119 ceph-mon[31518]: pgmap v3725: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:25:24.443 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:25:24 smithi132 ceph-mon[35941]: pgmap v3725: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:25:26.373 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:25:26 smithi165 ceph-mon[36302]: pgmap v3726: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:25:26.373 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:25:26 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6585.754802 secs 2024-03-28T02:25:26.396 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:25:26 smithi119 ceph-mon[31518]: pgmap v3726: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:25:26.396 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:25:26 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6585.754802 secs 2024-03-28T02:25:26.443 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:25:26 smithi132 ceph-mon[35941]: pgmap v3726: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:25:26.443 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:25:26 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6585.754802 secs 2024-03-28T02:25:28.373 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:25:28 smithi165 ceph-mon[36302]: pgmap v3727: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:25:28.396 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:25:28 smithi119 ceph-mon[31518]: pgmap v3727: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:25:28.443 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:25:28 smithi132 ceph-mon[35941]: pgmap v3727: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:25:30.373 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:25:30 smithi165 ceph-mon[36302]: pgmap v3728: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T02:25:30.396 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:25:30 smithi119 ceph-mon[31518]: pgmap v3728: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T02:25:30.443 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:25:30 smithi132 ceph-mon[35941]: pgmap v3728: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T02:25:31.373 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:25:31 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6590.754892 secs 2024-03-28T02:25:31.396 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:25:31 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6590.754892 secs 2024-03-28T02:25:31.443 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:25:31 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6590.754892 secs 2024-03-28T02:25:32.373 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:25:32 smithi165 ceph-mon[36302]: pgmap v3729: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:25:32.396 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:25:32 smithi119 ceph-mon[31518]: pgmap v3729: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:25:32.443 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:25:32 smithi132 ceph-mon[35941]: pgmap v3729: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:25:34.373 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:25:34 smithi165 ceph-mon[36302]: pgmap v3730: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:25:34.396 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:25:34 smithi119 ceph-mon[31518]: pgmap v3730: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:25:34.443 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:25:34 smithi132 ceph-mon[35941]: pgmap v3730: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:25:36.375 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:25:36 smithi165 ceph-mon[36302]: pgmap v3731: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:25:36.375 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:25:36 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6595.754976 secs 2024-03-28T02:25:36.396 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:25:36 smithi119 ceph-mon[31518]: pgmap v3731: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:25:36.397 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:25:36 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6595.754976 secs 2024-03-28T02:25:36.445 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:25:36 smithi132 ceph-mon[35941]: pgmap v3731: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:25:36.445 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:25:36 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6595.754976 secs 2024-03-28T02:25:38.373 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:25:38 smithi165 ceph-mon[36302]: pgmap v3732: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:25:38.396 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:25:38 smithi119 ceph-mon[31518]: pgmap v3732: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:25:38.443 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:25:38 smithi132 ceph-mon[35941]: pgmap v3732: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:25:40.373 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:25:40 smithi165 ceph-mon[36302]: pgmap v3733: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:25:40.396 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:25:40 smithi119 ceph-mon[31518]: pgmap v3733: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:25:40.443 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:25:40 smithi132 ceph-mon[35941]: pgmap v3733: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:25:41.373 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:25:41 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6600.755038 secs 2024-03-28T02:25:41.396 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:25:41 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6600.755038 secs 2024-03-28T02:25:41.443 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:25:41 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6600.755038 secs 2024-03-28T02:25:42.373 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:25:42 smithi165 ceph-mon[36302]: pgmap v3734: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:25:42.396 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:25:42 smithi119 ceph-mon[31518]: pgmap v3734: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:25:42.443 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:25:42 smithi132 ceph-mon[35941]: pgmap v3734: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:25:44.373 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:25:44 smithi165 ceph-mon[36302]: pgmap v3735: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:25:44.396 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:25:44 smithi119 ceph-mon[31518]: pgmap v3735: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:25:44.443 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:25:44 smithi132 ceph-mon[35941]: pgmap v3735: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:25:46.373 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:25:46 smithi165 ceph-mon[36302]: pgmap v3736: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:25:46.373 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:25:46 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6605.755123 secs 2024-03-28T02:25:46.396 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:25:46 smithi119 ceph-mon[31518]: pgmap v3736: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:25:46.396 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:25:46 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6605.755123 secs 2024-03-28T02:25:46.443 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:25:46 smithi132 ceph-mon[35941]: pgmap v3736: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:25:46.443 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:25:46 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6605.755123 secs 2024-03-28T02:25:48.373 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:25:48 smithi165 ceph-mon[36302]: pgmap v3737: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:25:48.396 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:25:48 smithi119 ceph-mon[31518]: pgmap v3737: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:25:48.443 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:25:48 smithi132 ceph-mon[35941]: pgmap v3737: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:25:50.373 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:25:50 smithi165 ceph-mon[36302]: pgmap v3738: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:25:50.396 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:25:50 smithi119 ceph-mon[31518]: pgmap v3738: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:25:50.443 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:25:50 smithi132 ceph-mon[35941]: pgmap v3738: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:25:51.373 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:25:51 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6610.755186 secs 2024-03-28T02:25:51.396 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:25:51 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6610.755186 secs 2024-03-28T02:25:51.444 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:25:51 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6610.755186 secs 2024-03-28T02:25:52.373 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:25:52 smithi165 ceph-mon[36302]: pgmap v3739: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:25:52.396 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:25:52 smithi119 ceph-mon[31518]: pgmap v3739: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:25:52.443 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:25:52 smithi132 ceph-mon[35941]: pgmap v3739: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:25:54.373 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:25:54 smithi165 ceph-mon[36302]: pgmap v3740: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:25:54.396 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:25:54 smithi119 ceph-mon[31518]: pgmap v3740: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:25:54.443 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:25:54 smithi132 ceph-mon[35941]: pgmap v3740: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:25:56.373 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:25:56 smithi165 ceph-mon[36302]: pgmap v3741: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:25:56.373 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:25:56 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6615.755219 secs 2024-03-28T02:25:56.396 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:25:56 smithi119 ceph-mon[31518]: pgmap v3741: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:25:56.396 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:25:56 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6615.755219 secs 2024-03-28T02:25:56.443 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:25:56 smithi132 ceph-mon[35941]: pgmap v3741: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:25:56.443 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:25:56 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6615.755219 secs 2024-03-28T02:25:57.443 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:25:57 smithi132 ceph-mon[35941]: pgmap v3742: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:25:57.623 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:25:57 smithi165 ceph-mon[36302]: pgmap v3742: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:25:57.646 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:25:57 smithi119 ceph-mon[31518]: pgmap v3742: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:26:00.123 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:25:59 smithi165 ceph-mon[36302]: pgmap v3743: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:26:00.145 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:25:59 smithi132 ceph-mon[35941]: pgmap v3743: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:26:00.146 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:25:59 smithi119 ceph-mon[31518]: pgmap v3743: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:26:01.123 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:26:00 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6620.755307 secs 2024-03-28T02:26:01.146 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:26:00 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6620.755307 secs 2024-03-28T02:26:01.193 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:26:00 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6620.755307 secs 2024-03-28T02:26:02.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:26:01 smithi165 ceph-mon[36302]: pgmap v3744: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:26:02.146 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:26:01 smithi119 ceph-mon[31518]: pgmap v3744: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:26:02.195 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:26:01 smithi132 ceph-mon[35941]: pgmap v3744: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:26:04.123 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:26:03 smithi165 ceph-mon[36302]: pgmap v3745: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:26:04.146 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:26:03 smithi119 ceph-mon[31518]: pgmap v3745: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:26:04.193 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:26:03 smithi132 ceph-mon[35941]: pgmap v3745: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:26:06.123 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:26:05 smithi165 ceph-mon[36302]: pgmap v3746: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:26:06.124 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:26:05 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6625.755391 secs 2024-03-28T02:26:06.146 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:26:05 smithi119 ceph-mon[31518]: pgmap v3746: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:26:06.146 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:26:05 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6625.755391 secs 2024-03-28T02:26:06.193 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:26:05 smithi132 ceph-mon[35941]: pgmap v3746: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:26:06.193 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:26:05 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6625.755391 secs 2024-03-28T02:26:08.123 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:26:07 smithi165 ceph-mon[36302]: pgmap v3747: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:26:08.146 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:26:07 smithi119 ceph-mon[31518]: pgmap v3747: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:26:08.193 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:26:07 smithi132 ceph-mon[35941]: pgmap v3747: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:26:10.123 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:26:09 smithi165 ceph-mon[36302]: pgmap v3748: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:26:10.146 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:26:09 smithi119 ceph-mon[31518]: pgmap v3748: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:26:10.193 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:26:09 smithi132 ceph-mon[35941]: pgmap v3748: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:26:11.123 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:26:10 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6630.755453 secs 2024-03-28T02:26:11.146 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:26:10 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6630.755453 secs 2024-03-28T02:26:11.193 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:26:10 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6630.755453 secs 2024-03-28T02:26:12.123 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:26:11 smithi165 ceph-mon[36302]: pgmap v3749: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:26:12.146 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:26:11 smithi119 ceph-mon[31518]: pgmap v3749: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:26:12.193 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:26:11 smithi132 ceph-mon[35941]: pgmap v3749: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:26:14.123 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:26:13 smithi165 ceph-mon[36302]: pgmap v3750: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:26:14.146 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:26:13 smithi119 ceph-mon[31518]: pgmap v3750: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:26:14.193 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:26:13 smithi132 ceph-mon[35941]: pgmap v3750: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:26:16.123 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:26:15 smithi165 ceph-mon[36302]: pgmap v3751: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:26:16.124 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:26:15 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6635.755542 secs 2024-03-28T02:26:16.146 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:26:15 smithi119 ceph-mon[31518]: pgmap v3751: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:26:16.147 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:26:15 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6635.755542 secs 2024-03-28T02:26:16.193 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:26:15 smithi132 ceph-mon[35941]: pgmap v3751: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:26:16.193 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:26:15 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6635.755542 secs 2024-03-28T02:26:18.123 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:26:17 smithi165 ceph-mon[36302]: pgmap v3752: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:26:18.146 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:26:17 smithi119 ceph-mon[31518]: pgmap v3752: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:26:18.193 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:26:17 smithi132 ceph-mon[35941]: pgmap v3752: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:26:20.123 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:26:19 smithi165 ceph-mon[36302]: pgmap v3753: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:26:20.124 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:26:19 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:26:20.146 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:26:19 smithi119 ceph-mon[31518]: pgmap v3753: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:26:20.146 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:26:19 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:26:20.193 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:26:19 smithi132 ceph-mon[35941]: pgmap v3753: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:26:20.193 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:26:19 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:26:21.123 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:26:20 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6640.755626 secs 2024-03-28T02:26:21.124 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:26:20 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:26:21.124 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:26:20 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:26:21.124 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:26:20 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:26:21.146 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:26:20 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6640.755626 secs 2024-03-28T02:26:21.146 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:26:20 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:26:21.146 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:26:20 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:26:21.147 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:26:20 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:26:21.193 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:26:20 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6640.755626 secs 2024-03-28T02:26:21.194 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:26:20 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:26:21.194 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:26:20 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:26:21.194 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:26:20 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:26:22.123 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:26:21 smithi165 ceph-mon[36302]: pgmap v3754: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T02:26:22.124 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:26:21 smithi165 ceph-mon[36302]: pgmap v3755: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:26:22.146 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:26:21 smithi119 ceph-mon[31518]: pgmap v3754: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T02:26:22.146 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:26:21 smithi119 ceph-mon[31518]: pgmap v3755: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:26:22.193 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:26:21 smithi132 ceph-mon[35941]: pgmap v3754: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T02:26:22.193 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:26:21 smithi132 ceph-mon[35941]: pgmap v3755: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:26:24.123 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:26:23 smithi165 ceph-mon[36302]: pgmap v3756: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:26:24.146 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:26:23 smithi119 ceph-mon[31518]: pgmap v3756: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:26:24.193 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:26:23 smithi132 ceph-mon[35941]: pgmap v3756: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:26:26.124 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:26:25 smithi165 ceph-mon[36302]: pgmap v3757: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 5 op/s 2024-03-28T02:26:26.124 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:26:25 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6645.755715 secs 2024-03-28T02:26:26.146 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:26:25 smithi119 ceph-mon[31518]: pgmap v3757: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 5 op/s 2024-03-28T02:26:26.146 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:26:25 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6645.755715 secs 2024-03-28T02:26:26.194 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:26:25 smithi132 ceph-mon[35941]: pgmap v3757: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 5 op/s 2024-03-28T02:26:26.194 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:26:25 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6645.755715 secs 2024-03-28T02:26:28.124 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:26:27 smithi165 ceph-mon[36302]: pgmap v3758: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:26:28.146 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:26:27 smithi119 ceph-mon[31518]: pgmap v3758: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:26:28.193 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:26:27 smithi132 ceph-mon[35941]: pgmap v3758: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:26:30.124 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:26:29 smithi165 ceph-mon[36302]: pgmap v3759: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:26:30.146 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:26:29 smithi119 ceph-mon[31518]: pgmap v3759: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:26:30.193 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:26:29 smithi132 ceph-mon[35941]: pgmap v3759: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:26:31.146 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:26:30 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6650.755800 secs 2024-03-28T02:26:31.193 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:26:30 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6650.755800 secs 2024-03-28T02:26:31.373 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:26:30 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6650.755800 secs 2024-03-28T02:26:32.147 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:26:31 smithi119 ceph-mon[31518]: pgmap v3760: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T02:26:32.193 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:26:31 smithi132 ceph-mon[35941]: pgmap v3760: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T02:26:32.373 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:26:31 smithi165 ceph-mon[36302]: pgmap v3760: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T02:26:34.141 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:26:33 smithi119 ceph-mon[31518]: pgmap v3761: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:26:34.194 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:26:33 smithi132 ceph-mon[35941]: pgmap v3761: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:26:34.374 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:26:33 smithi165 ceph-mon[36302]: pgmap v3761: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:26:36.147 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:26:35 smithi119 ceph-mon[31518]: pgmap v3762: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:26:36.147 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:26:35 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6655.755863 secs 2024-03-28T02:26:36.193 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:26:35 smithi132 ceph-mon[35941]: pgmap v3762: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:26:36.194 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:26:35 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6655.755863 secs 2024-03-28T02:26:36.374 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:26:35 smithi165 ceph-mon[36302]: pgmap v3762: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:26:36.374 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:26:35 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6655.755863 secs 2024-03-28T02:26:38.147 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:26:37 smithi119 ceph-mon[31518]: pgmap v3763: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-03-28T02:26:38.193 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:26:37 smithi132 ceph-mon[35941]: pgmap v3763: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-03-28T02:26:38.374 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:26:37 smithi165 ceph-mon[36302]: pgmap v3763: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-03-28T02:26:40.176 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:26:39 smithi165 ceph-mon[36302]: pgmap v3764: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:26:40.194 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:26:39 smithi132 ceph-mon[35941]: pgmap v3764: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:26:40.397 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:26:39 smithi119 ceph-mon[31518]: pgmap v3764: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:26:41.194 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:26:40 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6660.755927 secs 2024-03-28T02:26:41.374 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:26:40 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6660.755927 secs 2024-03-28T02:26:41.397 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:26:40 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6660.755927 secs 2024-03-28T02:26:42.194 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:26:41 smithi132 ceph-mon[35941]: pgmap v3765: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:26:42.374 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:26:41 smithi165 ceph-mon[36302]: pgmap v3765: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:26:42.396 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:26:41 smithi119 ceph-mon[31518]: pgmap v3765: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:26:44.194 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:26:43 smithi132 ceph-mon[35941]: pgmap v3766: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:26:44.374 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:26:43 smithi165 ceph-mon[36302]: pgmap v3766: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:26:44.397 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:26:43 smithi119 ceph-mon[31518]: pgmap v3766: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:26:46.194 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:26:45 smithi132 ceph-mon[35941]: pgmap v3767: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:26:46.194 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:26:45 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6665.755989 secs 2024-03-28T02:26:46.374 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:26:45 smithi165 ceph-mon[36302]: pgmap v3767: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:26:46.374 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:26:45 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6665.755989 secs 2024-03-28T02:26:46.397 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:26:45 smithi119 ceph-mon[31518]: pgmap v3767: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:26:46.397 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:26:45 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6665.755989 secs 2024-03-28T02:26:48.194 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:26:47 smithi132 ceph-mon[35941]: pgmap v3768: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:26:48.374 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:26:47 smithi165 ceph-mon[36302]: pgmap v3768: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:26:48.397 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:26:47 smithi119 ceph-mon[31518]: pgmap v3768: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:26:50.194 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:26:49 smithi132 ceph-mon[35941]: pgmap v3769: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:26:50.374 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:26:49 smithi165 ceph-mon[36302]: pgmap v3769: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:26:50.397 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:26:49 smithi119 ceph-mon[31518]: pgmap v3769: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:26:51.194 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:26:50 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6670.756074 secs 2024-03-28T02:26:51.374 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:26:50 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6670.756074 secs 2024-03-28T02:26:51.397 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:26:50 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6670.756074 secs 2024-03-28T02:26:52.374 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:26:51 smithi165 ceph-mon[36302]: pgmap v3770: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:26:52.397 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:26:51 smithi119 ceph-mon[31518]: pgmap v3770: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:26:52.444 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:26:51 smithi132 ceph-mon[35941]: pgmap v3770: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:26:54.374 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:26:53 smithi165 ceph-mon[36302]: pgmap v3771: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:26:54.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:26:53 smithi119 ceph-mon[31518]: pgmap v3771: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:26:54.444 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:26:53 smithi132 ceph-mon[35941]: pgmap v3771: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:26:56.374 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:26:55 smithi165 ceph-mon[36302]: pgmap v3772: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:26:56.374 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:26:55 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6675.756136 secs 2024-03-28T02:26:56.397 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:26:55 smithi119 ceph-mon[31518]: pgmap v3772: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:26:56.397 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:26:55 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6675.756136 secs 2024-03-28T02:26:56.444 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:26:55 smithi132 ceph-mon[35941]: pgmap v3772: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:26:56.444 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:26:55 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6675.756136 secs 2024-03-28T02:26:58.374 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:26:57 smithi165 ceph-mon[36302]: pgmap v3773: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:26:58.397 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:26:57 smithi119 ceph-mon[31518]: pgmap v3773: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:26:58.444 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:26:57 smithi132 ceph-mon[35941]: pgmap v3773: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:27:00.374 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:26:59 smithi165 ceph-mon[36302]: pgmap v3774: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:27:00.397 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:26:59 smithi119 ceph-mon[31518]: pgmap v3774: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:27:00.444 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:26:59 smithi132 ceph-mon[35941]: pgmap v3774: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:27:01.345 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:27:00 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6680.756196 secs 2024-03-28T02:27:01.374 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:27:00 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6680.756196 secs 2024-03-28T02:27:01.397 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:27:00 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6680.756196 secs 2024-03-28T02:27:02.374 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:27:01 smithi165 ceph-mon[36302]: pgmap v3775: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:27:02.397 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:27:01 smithi119 ceph-mon[31518]: pgmap v3775: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:27:02.444 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:27:01 smithi132 ceph-mon[35941]: pgmap v3775: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:27:04.374 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:27:03 smithi165 ceph-mon[36302]: pgmap v3776: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:27:04.397 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:27:03 smithi119 ceph-mon[31518]: pgmap v3776: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:27:04.444 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:27:03 smithi132 ceph-mon[35941]: pgmap v3776: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:27:06.374 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:27:05 smithi165 ceph-mon[36302]: pgmap v3777: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:27:06.374 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:27:05 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6685.756255 secs 2024-03-28T02:27:06.397 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:27:05 smithi119 ceph-mon[31518]: pgmap v3777: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:27:06.397 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:27:05 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6685.756255 secs 2024-03-28T02:27:06.444 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:27:05 smithi132 ceph-mon[35941]: pgmap v3777: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:27:06.444 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:27:05 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6685.756255 secs 2024-03-28T02:27:08.374 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:27:07 smithi165 ceph-mon[36302]: pgmap v3778: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:27:08.397 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:27:07 smithi119 ceph-mon[31518]: pgmap v3778: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:27:08.444 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:27:07 smithi132 ceph-mon[35941]: pgmap v3778: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:27:10.374 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:27:09 smithi165 ceph-mon[36302]: pgmap v3779: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:27:10.397 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:27:09 smithi119 ceph-mon[31518]: pgmap v3779: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:27:10.444 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:27:09 smithi132 ceph-mon[35941]: pgmap v3779: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:27:11.374 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:27:11 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6690.756314 secs 2024-03-28T02:27:11.397 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:27:11 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6690.756314 secs 2024-03-28T02:27:11.444 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:27:11 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6690.756314 secs 2024-03-28T02:27:12.374 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:27:12 smithi165 ceph-mon[36302]: pgmap v3780: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:27:12.397 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:27:12 smithi119 ceph-mon[31518]: pgmap v3780: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:27:12.444 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:27:12 smithi132 ceph-mon[35941]: pgmap v3780: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:27:14.374 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:27:14 smithi165 ceph-mon[36302]: pgmap v3781: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:27:14.397 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:27:14 smithi119 ceph-mon[31518]: pgmap v3781: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:27:14.444 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:27:14 smithi132 ceph-mon[35941]: pgmap v3781: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:27:16.374 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:27:16 smithi165 ceph-mon[36302]: pgmap v3782: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:27:16.374 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:27:16 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6695.756402 secs 2024-03-28T02:27:16.397 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:27:16 smithi119 ceph-mon[31518]: pgmap v3782: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:27:16.397 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:27:16 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6695.756402 secs 2024-03-28T02:27:16.444 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:27:16 smithi132 ceph-mon[35941]: pgmap v3782: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:27:16.444 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:27:16 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6695.756402 secs 2024-03-28T02:27:18.374 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:27:18 smithi165 ceph-mon[36302]: pgmap v3783: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:27:18.397 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:27:18 smithi119 ceph-mon[31518]: pgmap v3783: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:27:18.444 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:27:18 smithi132 ceph-mon[35941]: pgmap v3783: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:27:20.374 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:27:20 smithi165 ceph-mon[36302]: pgmap v3784: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:27:20.397 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:27:20 smithi119 ceph-mon[31518]: pgmap v3784: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:27:20.444 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:27:20 smithi132 ceph-mon[35941]: pgmap v3784: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:27:21.374 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:27:21 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6700.756492 secs 2024-03-28T02:27:21.375 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:27:21 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:27:21.397 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:27:21 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6700.756492 secs 2024-03-28T02:27:21.397 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:27:21 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:27:21.444 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:27:21 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6700.756492 secs 2024-03-28T02:27:21.444 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:27:21 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:27:22.374 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:27:22 smithi165 ceph-mon[36302]: pgmap v3785: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:27:22.374 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:27:22 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:27:22.375 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:27:22 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:27:22.375 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:27:22 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:27:22.375 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:27:22 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:27:22.397 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:27:22 smithi119 ceph-mon[31518]: pgmap v3785: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:27:22.397 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:27:22 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:27:22.397 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:27:22 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:27:22.397 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:27:22 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:27:22.397 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:27:22 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:27:22.444 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:27:22 smithi132 ceph-mon[35941]: pgmap v3785: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:27:22.444 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:27:22 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:27:22.444 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:27:22 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:27:22.444 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:27:22 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:27:22.444 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:27:22 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:27:23.874 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:27:23 smithi165 ceph-mon[36302]: pgmap v3786: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:27:23.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:27:23 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:27:23.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:27:23 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:27:23.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:27:23 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:27:23.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:27:23 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:27:23.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:27:23 smithi165 ceph-mon[36302]: pgmap v3787: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-28T02:27:23.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:27:23 smithi165 ceph-mon[36302]: pgmap v3788: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 3 op/s 2024-03-28T02:27:23.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:27:23 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:27:23.897 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:27:23 smithi119 ceph-mon[31518]: pgmap v3786: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:27:23.897 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:27:23 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:27:23.897 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:27:23 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:27:23.897 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:27:23 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:27:23.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:27:23 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:27:23.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:27:23 smithi119 ceph-mon[31518]: pgmap v3787: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-28T02:27:23.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:27:23 smithi119 ceph-mon[31518]: pgmap v3788: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 3 op/s 2024-03-28T02:27:23.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:27:23 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:27:23.944 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:27:23 smithi132 ceph-mon[35941]: pgmap v3786: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:27:23.944 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:27:23 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:27:23.945 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:27:23 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:27:23.945 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:27:23 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:27:23.945 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:27:23 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:27:23.945 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:27:23 smithi132 ceph-mon[35941]: pgmap v3787: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-28T02:27:23.945 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:27:23 smithi132 ceph-mon[35941]: pgmap v3788: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 3 op/s 2024-03-28T02:27:23.945 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:27:23 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:27:25.874 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:27:25 smithi165 ceph-mon[36302]: pgmap v3789: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-03-28T02:27:25.897 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:27:25 smithi119 ceph-mon[31518]: pgmap v3789: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-03-28T02:27:25.944 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:27:25 smithi132 ceph-mon[35941]: pgmap v3789: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-03-28T02:27:26.874 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:27:26 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6705.756576 secs 2024-03-28T02:27:26.897 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:27:26 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6705.756576 secs 2024-03-28T02:27:26.944 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:27:26 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6705.756576 secs 2024-03-28T02:27:27.874 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:27:27 smithi165 ceph-mon[36302]: pgmap v3790: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-03-28T02:27:27.897 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:27:27 smithi119 ceph-mon[31518]: pgmap v3790: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-03-28T02:27:27.944 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:27:27 smithi132 ceph-mon[35941]: pgmap v3790: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-03-28T02:27:29.874 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:27:29 smithi165 ceph-mon[36302]: pgmap v3791: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 3 op/s 2024-03-28T02:27:29.897 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:27:29 smithi119 ceph-mon[31518]: pgmap v3791: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 3 op/s 2024-03-28T02:27:29.944 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:27:29 smithi132 ceph-mon[35941]: pgmap v3791: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 3 op/s 2024-03-28T02:27:31.874 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:27:31 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6710.756637 secs 2024-03-28T02:27:31.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:27:31 smithi165 ceph-mon[36302]: pgmap v3792: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-03-28T02:27:31.897 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:27:31 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6710.756637 secs 2024-03-28T02:27:31.897 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:27:31 smithi119 ceph-mon[31518]: pgmap v3792: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-03-28T02:27:31.944 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:27:31 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6710.756637 secs 2024-03-28T02:27:31.944 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:27:31 smithi132 ceph-mon[35941]: pgmap v3792: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-03-28T02:27:33.874 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:27:33 smithi165 ceph-mon[36302]: pgmap v3793: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T02:27:33.897 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:27:33 smithi119 ceph-mon[31518]: pgmap v3793: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T02:27:33.944 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:27:33 smithi132 ceph-mon[35941]: pgmap v3793: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T02:27:35.874 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:27:35 smithi165 ceph-mon[36302]: pgmap v3794: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:27:35.897 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:27:35 smithi119 ceph-mon[31518]: pgmap v3794: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:27:35.944 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:27:35 smithi132 ceph-mon[35941]: pgmap v3794: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:27:36.874 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:27:36 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6715.756699 secs 2024-03-28T02:27:36.897 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:27:36 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6715.756699 secs 2024-03-28T02:27:36.944 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:27:36 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6715.756699 secs 2024-03-28T02:27:37.874 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:27:37 smithi165 ceph-mon[36302]: pgmap v3795: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:27:37.897 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:27:37 smithi119 ceph-mon[31518]: pgmap v3795: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:27:37.944 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:27:37 smithi132 ceph-mon[35941]: pgmap v3795: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:27:39.874 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:27:39 smithi165 ceph-mon[36302]: pgmap v3796: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:27:39.897 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:27:39 smithi119 ceph-mon[31518]: pgmap v3796: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:27:39.944 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:27:39 smithi132 ceph-mon[35941]: pgmap v3796: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:27:41.874 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:27:41 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6720.756761 secs 2024-03-28T02:27:41.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:27:41 smithi165 ceph-mon[36302]: pgmap v3797: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:27:41.897 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:27:41 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6720.756761 secs 2024-03-28T02:27:41.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:27:41 smithi119 ceph-mon[31518]: pgmap v3797: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:27:41.944 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:27:41 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6720.756761 secs 2024-03-28T02:27:41.944 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:27:41 smithi132 ceph-mon[35941]: pgmap v3797: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:27:43.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:27:43 smithi165 ceph-mon[36302]: pgmap v3798: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:27:43.897 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:27:43 smithi119 ceph-mon[31518]: pgmap v3798: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:27:43.944 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:27:43 smithi132 ceph-mon[35941]: pgmap v3798: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:27:45.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:27:45 smithi165 ceph-mon[36302]: pgmap v3799: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:27:45.897 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:27:45 smithi119 ceph-mon[31518]: pgmap v3799: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:27:45.944 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:27:45 smithi132 ceph-mon[35941]: pgmap v3799: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:27:46.874 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:27:46 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6725.756851 secs 2024-03-28T02:27:46.897 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:27:46 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6725.756851 secs 2024-03-28T02:27:46.944 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:27:46 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6725.756851 secs 2024-03-28T02:27:47.874 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:27:47 smithi165 ceph-mon[36302]: pgmap v3800: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:27:47.897 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:27:47 smithi119 ceph-mon[31518]: pgmap v3800: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:27:47.944 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:27:47 smithi132 ceph-mon[35941]: pgmap v3800: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:27:49.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:27:49 smithi165 ceph-mon[36302]: pgmap v3801: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:27:49.897 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:27:49 smithi119 ceph-mon[31518]: pgmap v3801: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:27:49.944 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:27:49 smithi132 ceph-mon[35941]: pgmap v3801: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:27:51.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:27:51 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6730.756914 secs 2024-03-28T02:27:51.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:27:51 smithi165 ceph-mon[36302]: pgmap v3802: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:27:51.897 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:27:51 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6730.756914 secs 2024-03-28T02:27:51.897 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:27:51 smithi119 ceph-mon[31518]: pgmap v3802: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:27:51.945 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:27:51 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6730.756914 secs 2024-03-28T02:27:51.945 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:27:51 smithi132 ceph-mon[35941]: pgmap v3802: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:27:53.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:27:53 smithi165 ceph-mon[36302]: pgmap v3803: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:27:53.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:27:53 smithi119 ceph-mon[31518]: pgmap v3803: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:27:53.944 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:27:53 smithi132 ceph-mon[35941]: pgmap v3803: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:27:55.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:27:55 smithi165 ceph-mon[36302]: pgmap v3804: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:27:55.897 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:27:55 smithi119 ceph-mon[31518]: pgmap v3804: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:27:55.944 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:27:55 smithi132 ceph-mon[35941]: pgmap v3804: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:27:56.897 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:27:56 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6735.756998 secs 2024-03-28T02:27:56.944 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:27:56 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6735.756998 secs 2024-03-28T02:27:57.125 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:27:56 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6735.756998 secs 2024-03-28T02:27:57.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:27:57 smithi119 ceph-mon[31518]: pgmap v3805: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T02:27:57.945 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:27:57 smithi132 ceph-mon[35941]: pgmap v3805: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T02:27:58.125 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:27:57 smithi165 ceph-mon[36302]: pgmap v3805: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T02:27:59.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:27:59 smithi119 ceph-mon[31518]: pgmap v3806: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T02:27:59.945 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:27:59 smithi132 ceph-mon[35941]: pgmap v3806: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T02:28:00.125 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:27:59 smithi165 ceph-mon[36302]: pgmap v3806: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T02:28:01.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:28:01 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6740.757059 secs 2024-03-28T02:28:01.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:28:01 smithi119 ceph-mon[31518]: pgmap v3807: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:28:01.944 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:28:01 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6740.757059 secs 2024-03-28T02:28:01.945 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:28:01 smithi132 ceph-mon[35941]: pgmap v3807: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:28:02.125 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:28:01 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6740.757059 secs 2024-03-28T02:28:02.125 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:28:01 smithi165 ceph-mon[36302]: pgmap v3807: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:28:03.945 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:28:03 smithi132 ceph-mon[35941]: pgmap v3808: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T02:28:04.125 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:28:03 smithi165 ceph-mon[36302]: pgmap v3808: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T02:28:04.148 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:28:03 smithi119 ceph-mon[31518]: pgmap v3808: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T02:28:05.945 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:28:05 smithi132 ceph-mon[35941]: pgmap v3809: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:28:06.124 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:28:05 smithi165 ceph-mon[36302]: pgmap v3809: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:28:06.147 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:28:05 smithi119 ceph-mon[31518]: pgmap v3809: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:28:06.945 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:28:06 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6745.757098 secs 2024-03-28T02:28:07.124 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:28:06 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6745.757098 secs 2024-03-28T02:28:07.148 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:28:06 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6745.757098 secs 2024-03-28T02:28:07.945 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:28:07 smithi132 ceph-mon[35941]: pgmap v3810: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:28:08.125 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:28:07 smithi165 ceph-mon[36302]: pgmap v3810: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:28:08.147 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:28:07 smithi119 ceph-mon[31518]: pgmap v3810: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:28:09.945 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:28:09 smithi132 ceph-mon[35941]: pgmap v3811: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:28:10.125 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:28:09 smithi165 ceph-mon[36302]: pgmap v3811: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:28:10.148 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:28:09 smithi119 ceph-mon[31518]: pgmap v3811: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:28:10.945 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:28:10 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6750.757201 secs 2024-03-28T02:28:11.026 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:28:10 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6750.757201 secs 2024-03-28T02:28:11.049 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:28:10 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6750.757201 secs 2024-03-28T02:28:11.945 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:28:11 smithi132 ceph-mon[35941]: pgmap v3812: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:28:12.125 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:28:11 smithi165 ceph-mon[36302]: pgmap v3812: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:28:12.148 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:28:11 smithi119 ceph-mon[31518]: pgmap v3812: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:28:13.945 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:28:13 smithi132 ceph-mon[35941]: pgmap v3813: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:28:14.125 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:28:13 smithi165 ceph-mon[36302]: pgmap v3813: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:28:14.148 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:28:13 smithi119 ceph-mon[31518]: pgmap v3813: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:28:15.945 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:28:15 smithi132 ceph-mon[35941]: pgmap v3814: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:28:15.945 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:28:15 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6755.757290 secs 2024-03-28T02:28:16.125 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:28:15 smithi165 ceph-mon[36302]: pgmap v3814: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:28:16.125 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:28:15 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6755.757290 secs 2024-03-28T02:28:16.148 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:28:15 smithi119 ceph-mon[31518]: pgmap v3814: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:28:16.148 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:28:15 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6755.757290 secs 2024-03-28T02:28:18.125 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:28:17 smithi165 ceph-mon[36302]: pgmap v3815: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:28:18.148 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:28:17 smithi119 ceph-mon[31518]: pgmap v3815: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:28:18.195 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:28:17 smithi132 ceph-mon[35941]: pgmap v3815: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:28:20.125 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:28:19 smithi165 ceph-mon[36302]: pgmap v3816: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:28:20.148 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:28:19 smithi119 ceph-mon[31518]: pgmap v3816: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:28:20.195 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:28:19 smithi132 ceph-mon[35941]: pgmap v3816: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:28:21.125 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:28:20 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6760.757352 secs 2024-03-28T02:28:21.148 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:28:20 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6760.757352 secs 2024-03-28T02:28:21.195 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:28:20 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6760.757352 secs 2024-03-28T02:28:22.125 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:28:21 smithi165 ceph-mon[36302]: pgmap v3817: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:28:22.148 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:28:21 smithi119 ceph-mon[31518]: pgmap v3817: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:28:22.195 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:28:21 smithi132 ceph-mon[35941]: pgmap v3817: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:28:24.125 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:28:23 smithi165 ceph-mon[36302]: pgmap v3818: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:28:24.125 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:28:23 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:28:24.148 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:28:23 smithi119 ceph-mon[31518]: pgmap v3818: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:28:24.148 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:28:23 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:28:24.195 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:28:23 smithi132 ceph-mon[35941]: pgmap v3818: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:28:24.195 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:28:23 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:28:25.125 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:28:24 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:28:25.125 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:28:24 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:28:25.125 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:28:24 smithi165 ceph-mon[36302]: pgmap v3819: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T02:28:25.125 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:28:24 smithi165 ceph-mon[36302]: pgmap v3820: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-28T02:28:25.125 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:28:24 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:28:25.148 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:28:24 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:28:25.148 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:28:24 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:28:25.148 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:28:24 smithi119 ceph-mon[31518]: pgmap v3819: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T02:28:25.148 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:28:24 smithi119 ceph-mon[31518]: pgmap v3820: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-28T02:28:25.148 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:28:24 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:28:25.195 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:28:24 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:28:25.195 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:28:24 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:28:25.195 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:28:24 smithi132 ceph-mon[35941]: pgmap v3819: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T02:28:25.195 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:28:24 smithi132 ceph-mon[35941]: pgmap v3820: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-28T02:28:25.195 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:28:24 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:28:26.125 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:28:25 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6765.757441 secs 2024-03-28T02:28:26.148 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:28:25 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6765.757441 secs 2024-03-28T02:28:26.195 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:28:25 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6765.757441 secs 2024-03-28T02:28:27.125 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:28:26 smithi165 ceph-mon[36302]: pgmap v3821: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:28:27.148 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:28:26 smithi119 ceph-mon[31518]: pgmap v3821: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:28:27.195 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:28:26 smithi132 ceph-mon[35941]: pgmap v3821: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:28:29.375 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:28:29 smithi165 ceph-mon[36302]: pgmap v3822: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:28:29.398 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:28:29 smithi119 ceph-mon[31518]: pgmap v3822: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:28:29.445 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:28:29 smithi132 ceph-mon[35941]: pgmap v3822: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:28:31.375 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:28:31 smithi165 ceph-mon[36302]: pgmap v3823: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:28:31.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:28:31 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6770.757532 secs 2024-03-28T02:28:31.398 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:28:31 smithi119 ceph-mon[31518]: pgmap v3823: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:28:31.398 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:28:31 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6770.757532 secs 2024-03-28T02:28:31.445 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:28:31 smithi132 ceph-mon[35941]: pgmap v3823: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:28:31.445 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:28:31 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6770.757532 secs 2024-03-28T02:28:33.375 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:28:33 smithi165 ceph-mon[36302]: pgmap v3824: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:28:33.398 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:28:33 smithi119 ceph-mon[31518]: pgmap v3824: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:28:33.445 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:28:33 smithi132 ceph-mon[35941]: pgmap v3824: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:28:35.375 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:28:35 smithi165 ceph-mon[36302]: pgmap v3825: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T02:28:35.398 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:28:35 smithi119 ceph-mon[31518]: pgmap v3825: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T02:28:35.445 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:28:35 smithi132 ceph-mon[35941]: pgmap v3825: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T02:28:36.375 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:28:36 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6775.757620 secs 2024-03-28T02:28:36.398 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:28:36 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6775.757620 secs 2024-03-28T02:28:36.445 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:28:36 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6775.757620 secs 2024-03-28T02:28:37.375 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:28:37 smithi165 ceph-mon[36302]: pgmap v3826: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:28:37.398 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:28:37 smithi119 ceph-mon[31518]: pgmap v3826: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:28:37.445 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:28:37 smithi132 ceph-mon[35941]: pgmap v3826: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:28:39.375 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:28:39 smithi165 ceph-mon[36302]: pgmap v3827: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:28:39.398 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:28:39 smithi119 ceph-mon[31518]: pgmap v3827: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:28:39.445 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:28:39 smithi132 ceph-mon[35941]: pgmap v3827: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:28:41.375 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:28:41 smithi165 ceph-mon[36302]: pgmap v3828: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:28:41.375 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:28:41 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6780.757705 secs 2024-03-28T02:28:41.398 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:28:41 smithi119 ceph-mon[31518]: pgmap v3828: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:28:41.398 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:28:41 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6780.757705 secs 2024-03-28T02:28:41.445 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:28:41 smithi132 ceph-mon[35941]: pgmap v3828: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:28:41.445 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:28:41 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6780.757705 secs 2024-03-28T02:28:43.375 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:28:43 smithi165 ceph-mon[36302]: pgmap v3829: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T02:28:43.398 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:28:43 smithi119 ceph-mon[31518]: pgmap v3829: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T02:28:43.445 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:28:43 smithi132 ceph-mon[35941]: pgmap v3829: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T02:28:45.375 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:28:45 smithi165 ceph-mon[36302]: pgmap v3830: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T02:28:45.398 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:28:45 smithi119 ceph-mon[31518]: pgmap v3830: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T02:28:45.445 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:28:45 smithi132 ceph-mon[35941]: pgmap v3830: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T02:28:46.375 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:28:46 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6785.757789 secs 2024-03-28T02:28:46.398 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:28:46 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6785.757789 secs 2024-03-28T02:28:46.445 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:28:46 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6785.757789 secs 2024-03-28T02:28:47.375 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:28:47 smithi165 ceph-mon[36302]: pgmap v3831: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:28:47.398 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:28:47 smithi119 ceph-mon[31518]: pgmap v3831: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:28:47.445 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:28:47 smithi132 ceph-mon[35941]: pgmap v3831: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:28:49.375 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:28:49 smithi165 ceph-mon[36302]: pgmap v3832: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T02:28:49.398 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:28:49 smithi119 ceph-mon[31518]: pgmap v3832: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T02:28:49.445 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:28:49 smithi132 ceph-mon[35941]: pgmap v3832: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T02:28:51.375 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:28:51 smithi165 ceph-mon[36302]: pgmap v3833: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:28:51.375 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:28:51 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6790.757852 secs 2024-03-28T02:28:51.398 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:28:51 smithi119 ceph-mon[31518]: pgmap v3833: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:28:51.398 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:28:51 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6790.757852 secs 2024-03-28T02:28:51.445 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:28:51 smithi132 ceph-mon[35941]: pgmap v3833: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:28:51.445 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:28:51 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6790.757852 secs 2024-03-28T02:28:53.375 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:28:53 smithi165 ceph-mon[36302]: pgmap v3834: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:28:53.398 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:28:53 smithi119 ceph-mon[31518]: pgmap v3834: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:28:53.445 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:28:53 smithi132 ceph-mon[35941]: pgmap v3834: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:28:55.375 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:28:55 smithi165 ceph-mon[36302]: pgmap v3835: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:28:55.398 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:28:55 smithi119 ceph-mon[31518]: pgmap v3835: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:28:55.445 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:28:55 smithi132 ceph-mon[35941]: pgmap v3835: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:28:56.398 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:28:56 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6795.757912 secs 2024-03-28T02:28:56.445 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:28:56 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6795.757912 secs 2024-03-28T02:28:56.625 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:28:56 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6795.757912 secs 2024-03-28T02:28:57.398 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:28:57 smithi119 ceph-mon[31518]: pgmap v3836: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:28:57.445 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:28:57 smithi132 ceph-mon[35941]: pgmap v3836: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:28:57.625 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:28:57 smithi165 ceph-mon[36302]: pgmap v3836: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:28:59.398 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:28:59 smithi119 ceph-mon[31518]: pgmap v3837: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:28:59.445 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:28:59 smithi132 ceph-mon[35941]: pgmap v3837: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:28:59.625 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:28:59 smithi165 ceph-mon[36302]: pgmap v3837: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:29:01.398 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:29:01 smithi119 ceph-mon[31518]: pgmap v3838: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:29:01.398 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:29:01 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6800.757975 secs 2024-03-28T02:29:01.445 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:29:01 smithi132 ceph-mon[35941]: pgmap v3838: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:29:01.445 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:29:01 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6800.757975 secs 2024-03-28T02:29:01.625 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:29:01 smithi165 ceph-mon[36302]: pgmap v3838: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:29:01.625 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:29:01 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6800.757975 secs 2024-03-28T02:29:03.398 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:29:03 smithi119 ceph-mon[31518]: pgmap v3839: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:29:03.445 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:29:03 smithi132 ceph-mon[35941]: pgmap v3839: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:29:03.625 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:29:03 smithi165 ceph-mon[36302]: pgmap v3839: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:29:05.445 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:29:05 smithi132 ceph-mon[35941]: pgmap v3840: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:29:05.625 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:29:05 smithi165 ceph-mon[36302]: pgmap v3840: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:29:05.648 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:29:05 smithi119 ceph-mon[31518]: pgmap v3840: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:29:06.445 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:29:06 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6805.757994 secs 2024-03-28T02:29:06.625 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:29:06 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6805.757994 secs 2024-03-28T02:29:06.648 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:29:06 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6805.757994 secs 2024-03-28T02:29:07.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:29:07 smithi132 ceph-mon[35941]: pgmap v3841: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:29:07.625 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:29:07 smithi165 ceph-mon[36302]: pgmap v3841: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:29:07.648 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:29:07 smithi119 ceph-mon[31518]: pgmap v3841: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:29:08.445 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:29:08 smithi132 ceph-mon[35941]: pgmap v3842: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:29:08.625 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:29:08 smithi165 ceph-mon[36302]: pgmap v3842: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:29:08.648 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:29:08 smithi119 ceph-mon[31518]: pgmap v3842: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:29:11.328 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:29:11 smithi165 ceph-mon[36302]: pgmap v3843: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:29:11.329 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:29:11 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6810.758056 secs 2024-03-28T02:29:11.398 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:29:11 smithi119 ceph-mon[31518]: pgmap v3843: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:29:11.398 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:29:11 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6810.758056 secs 2024-03-28T02:29:11.445 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:29:11 smithi132 ceph-mon[35941]: pgmap v3843: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:29:11.445 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:29:11 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6810.758056 secs 2024-03-28T02:29:13.375 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:29:13 smithi165 ceph-mon[36302]: pgmap v3844: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:29:13.398 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:29:13 smithi119 ceph-mon[31518]: pgmap v3844: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:29:13.445 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:29:13 smithi132 ceph-mon[35941]: pgmap v3844: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:29:15.375 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:29:15 smithi165 ceph-mon[36302]: pgmap v3845: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:29:15.398 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:29:15 smithi119 ceph-mon[31518]: pgmap v3845: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:29:15.445 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:29:15 smithi132 ceph-mon[35941]: pgmap v3845: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:29:16.375 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:29:16 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6815.758118 secs 2024-03-28T02:29:16.398 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:29:16 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6815.758118 secs 2024-03-28T02:29:16.445 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:29:16 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6815.758118 secs 2024-03-28T02:29:17.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:29:17 smithi165 ceph-mon[36302]: pgmap v3846: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:29:17.398 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:29:17 smithi119 ceph-mon[31518]: pgmap v3846: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:29:17.445 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:29:17 smithi132 ceph-mon[35941]: pgmap v3846: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:29:19.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:29:19 smithi165 ceph-mon[36302]: pgmap v3847: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:29:19.398 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:29:19 smithi119 ceph-mon[31518]: pgmap v3847: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:29:19.445 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:29:19 smithi132 ceph-mon[35941]: pgmap v3847: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:29:21.375 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:29:21 smithi165 ceph-mon[36302]: pgmap v3848: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:29:21.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:29:21 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6820.758132 secs 2024-03-28T02:29:21.398 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:29:21 smithi119 ceph-mon[31518]: pgmap v3848: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:29:21.398 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:29:21 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6820.758132 secs 2024-03-28T02:29:21.445 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:29:21 smithi132 ceph-mon[35941]: pgmap v3848: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:29:21.445 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:29:21 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6820.758132 secs 2024-03-28T02:29:23.375 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:29:23 smithi165 ceph-mon[36302]: pgmap v3849: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:29:23.398 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:29:23 smithi119 ceph-mon[31518]: pgmap v3849: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:29:23.445 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:29:23 smithi132 ceph-mon[35941]: pgmap v3849: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:29:25.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:29:25 smithi165 ceph-mon[36302]: pgmap v3850: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:29:25.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:29:25 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:29:25.398 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:29:25 smithi119 ceph-mon[31518]: pgmap v3850: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:29:25.398 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:29:25 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:29:25.445 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:29:25 smithi132 ceph-mon[35941]: pgmap v3850: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:29:25.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:29:25 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:29:26.375 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:29:26 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6825.758166 secs 2024-03-28T02:29:26.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:29:26 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:29:26.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:29:26 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:29:26.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:29:26 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:29:26.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:29:26 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6825.758166 secs 2024-03-28T02:29:26.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:29:26 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:29:26.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:29:26 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:29:26.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:29:26 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:29:26.445 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:29:26 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6825.758166 secs 2024-03-28T02:29:26.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:29:26 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:29:26.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:29:26 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:29:26.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:29:26 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:29:27.375 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:29:27 smithi165 ceph-mon[36302]: pgmap v3851: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T02:29:27.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:29:27 smithi165 ceph-mon[36302]: pgmap v3852: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:29:27.398 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:29:27 smithi119 ceph-mon[31518]: pgmap v3851: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T02:29:27.398 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:29:27 smithi119 ceph-mon[31518]: pgmap v3852: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:29:27.445 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:29:27 smithi132 ceph-mon[35941]: pgmap v3851: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T02:29:27.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:29:27 smithi132 ceph-mon[35941]: pgmap v3852: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:29:29.375 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:29:29 smithi165 ceph-mon[36302]: pgmap v3853: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:29:29.398 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:29:29 smithi119 ceph-mon[31518]: pgmap v3853: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:29:29.445 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:29:29 smithi132 ceph-mon[35941]: pgmap v3853: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:29:31.398 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:29:31 smithi119 ceph-mon[31518]: pgmap v3854: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:29:31.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:29:31 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6830.758242 secs 2024-03-28T02:29:31.445 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:29:31 smithi132 ceph-mon[35941]: pgmap v3854: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:29:31.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:29:31 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6830.758242 secs 2024-03-28T02:29:31.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:29:31 smithi165 ceph-mon[36302]: pgmap v3854: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:29:31.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:29:31 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6830.758242 secs 2024-03-28T02:29:33.398 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:29:33 smithi119 ceph-mon[31518]: pgmap v3855: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:29:33.445 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:29:33 smithi132 ceph-mon[35941]: pgmap v3855: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:29:33.625 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:29:33 smithi165 ceph-mon[36302]: pgmap v3855: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:29:35.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:29:35 smithi119 ceph-mon[31518]: pgmap v3856: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:29:35.445 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:29:35 smithi132 ceph-mon[35941]: pgmap v3856: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:29:35.625 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:29:35 smithi165 ceph-mon[36302]: pgmap v3856: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:29:36.445 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:29:36 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6835.758332 secs 2024-03-28T02:29:36.625 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:29:36 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6835.758332 secs 2024-03-28T02:29:36.648 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:29:36 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6835.758332 secs 2024-03-28T02:29:37.445 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:29:37 smithi132 ceph-mon[35941]: pgmap v3857: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T02:29:37.625 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:29:37 smithi165 ceph-mon[36302]: pgmap v3857: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T02:29:37.648 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:29:37 smithi119 ceph-mon[31518]: pgmap v3857: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T02:29:39.445 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:29:39 smithi132 ceph-mon[35941]: pgmap v3858: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:29:39.625 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:29:39 smithi165 ceph-mon[36302]: pgmap v3858: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:29:39.648 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:29:39 smithi119 ceph-mon[31518]: pgmap v3858: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:29:41.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:29:41 smithi119 ceph-mon[31518]: pgmap v3859: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:29:41.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:29:41 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6840.758392 secs 2024-03-28T02:29:41.428 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:29:41 smithi165 ceph-mon[36302]: pgmap v3859: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:29:41.428 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:29:41 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6840.758392 secs 2024-03-28T02:29:41.445 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:29:41 smithi132 ceph-mon[35941]: pgmap v3859: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:29:41.445 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:29:41 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6840.758392 secs 2024-03-28T02:29:42.625 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:29:42 smithi165 ceph-mon[36302]: pgmap v3860: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T02:29:42.648 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:29:42 smithi119 ceph-mon[31518]: pgmap v3860: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T02:29:42.695 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:29:42 smithi132 ceph-mon[35941]: pgmap v3860: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T02:29:44.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:29:44 smithi165 ceph-mon[36302]: pgmap v3861: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T02:29:44.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:29:44 smithi119 ceph-mon[31518]: pgmap v3861: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T02:29:44.945 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:29:44 smithi132 ceph-mon[35941]: pgmap v3861: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T02:29:46.875 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:29:46 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6845.758480 secs 2024-03-28T02:29:46.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:29:46 smithi165 ceph-mon[36302]: pgmap v3862: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:29:46.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:29:46 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6845.758480 secs 2024-03-28T02:29:46.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:29:46 smithi119 ceph-mon[31518]: pgmap v3862: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:29:46.945 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:29:46 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6845.758480 secs 2024-03-28T02:29:46.945 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:29:46 smithi132 ceph-mon[35941]: pgmap v3862: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:29:48.876 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:29:48 smithi165 ceph-mon[36302]: pgmap v3863: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T02:29:48.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:29:48 smithi119 ceph-mon[31518]: pgmap v3863: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T02:29:48.945 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:29:48 smithi132 ceph-mon[35941]: pgmap v3863: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T02:29:50.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:29:50 smithi119 ceph-mon[31518]: pgmap v3864: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:29:50.945 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:29:50 smithi132 ceph-mon[35941]: pgmap v3864: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:29:51.126 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:29:50 smithi165 ceph-mon[36302]: pgmap v3864: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:29:51.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:29:51 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6850.758522 secs 2024-03-28T02:29:51.945 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:29:51 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6850.758522 secs 2024-03-28T02:29:52.126 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:29:51 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6850.758522 secs 2024-03-28T02:29:52.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:29:52 smithi119 ceph-mon[31518]: pgmap v3865: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:29:52.945 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:29:52 smithi132 ceph-mon[35941]: pgmap v3865: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:29:53.126 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:29:52 smithi165 ceph-mon[36302]: pgmap v3865: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:29:54.899 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:29:54 smithi119 ceph-mon[31518]: pgmap v3866: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:29:54.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:29:54 smithi132 ceph-mon[35941]: pgmap v3866: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:29:55.126 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:29:54 smithi165 ceph-mon[36302]: pgmap v3866: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:29:56.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:29:56 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6855.758612 secs 2024-03-28T02:29:56.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:29:56 smithi132 ceph-mon[35941]: pgmap v3867: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:29:57.126 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:29:56 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6855.758612 secs 2024-03-28T02:29:57.126 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:29:56 smithi165 ceph-mon[36302]: pgmap v3867: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:29:57.148 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:29:56 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6855.758612 secs 2024-03-28T02:29:57.149 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:29:56 smithi119 ceph-mon[31518]: pgmap v3867: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:29:58.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:29:58 smithi132 ceph-mon[35941]: pgmap v3868: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:29:59.126 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:29:58 smithi165 ceph-mon[36302]: pgmap v3868: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:29:59.148 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:29:58 smithi119 ceph-mon[31518]: pgmap v3868: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:30:00.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:30:00 smithi132 ceph-mon[35941]: pgmap v3869: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:30:00.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:30:00 smithi132 ceph-mon[35941]: overall HEALTH_WARN 5 failed cephadm daemon(s); 1 filesystem is degraded; 1 MDSs report slow requests 2024-03-28T02:30:01.126 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:30:00 smithi165 ceph-mon[36302]: pgmap v3869: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:30:01.126 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:30:00 smithi165 ceph-mon[36302]: overall HEALTH_WARN 5 failed cephadm daemon(s); 1 filesystem is degraded; 1 MDSs report slow requests 2024-03-28T02:30:01.149 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:30:00 smithi119 ceph-mon[31518]: pgmap v3869: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:30:01.149 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:30:00 smithi119 ceph-mon[31518]: overall HEALTH_WARN 5 failed cephadm daemon(s); 1 filesystem is degraded; 1 MDSs report slow requests 2024-03-28T02:30:01.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:30:01 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6860.758697 secs 2024-03-28T02:30:02.126 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:30:01 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6860.758697 secs 2024-03-28T02:30:02.148 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:30:01 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6860.758697 secs 2024-03-28T02:30:02.946 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:30:02 smithi132 ceph-mon[35941]: pgmap v3870: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:30:03.126 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:30:02 smithi165 ceph-mon[36302]: pgmap v3870: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:30:03.149 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:30:02 smithi119 ceph-mon[31518]: pgmap v3870: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:30:05.196 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:30:04 smithi132 ceph-mon[35941]: pgmap v3871: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:30:05.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:30:04 smithi165 ceph-mon[36302]: pgmap v3871: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:30:05.398 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:30:04 smithi119 ceph-mon[31518]: pgmap v3871: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:30:06.196 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:30:05 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6865.758788 secs 2024-03-28T02:30:06.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:30:05 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6865.758788 secs 2024-03-28T02:30:06.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:30:05 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6865.758788 secs 2024-03-28T02:30:07.196 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:30:06 smithi132 ceph-mon[35941]: pgmap v3872: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:30:07.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:30:06 smithi165 ceph-mon[36302]: pgmap v3872: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:30:07.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:30:06 smithi119 ceph-mon[31518]: pgmap v3872: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:30:09.196 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:30:08 smithi132 ceph-mon[35941]: pgmap v3873: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:30:09.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:30:08 smithi165 ceph-mon[36302]: pgmap v3873: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:30:09.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:30:08 smithi119 ceph-mon[31518]: pgmap v3873: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:30:11.196 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:30:10 smithi132 ceph-mon[35941]: pgmap v3874: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:30:11.196 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:30:10 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6870.758878 secs 2024-03-28T02:30:11.231 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:30:10 smithi119 ceph-mon[31518]: pgmap v3874: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:30:11.231 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:30:10 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6870.758878 secs 2024-03-28T02:30:11.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:30:10 smithi165 ceph-mon[36302]: pgmap v3874: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:30:11.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:30:10 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6870.758878 secs 2024-03-28T02:30:13.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:30:12 smithi165 ceph-mon[36302]: pgmap v3875: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:30:13.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:30:12 smithi119 ceph-mon[31518]: pgmap v3875: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:30:13.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:30:12 smithi132 ceph-mon[35941]: pgmap v3875: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:30:15.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:30:14 smithi165 ceph-mon[36302]: pgmap v3876: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:30:15.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:30:14 smithi119 ceph-mon[31518]: pgmap v3876: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:30:15.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:30:14 smithi132 ceph-mon[35941]: pgmap v3876: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:30:16.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:30:15 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6875.758939 secs 2024-03-28T02:30:16.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:30:15 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6875.758939 secs 2024-03-28T02:30:16.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:30:15 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6875.758939 secs 2024-03-28T02:30:17.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:30:16 smithi165 ceph-mon[36302]: pgmap v3877: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:30:17.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:30:16 smithi119 ceph-mon[31518]: pgmap v3877: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:30:17.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:30:16 smithi132 ceph-mon[35941]: pgmap v3877: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:30:19.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:30:18 smithi165 ceph-mon[36302]: pgmap v3878: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:30:19.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:30:18 smithi119 ceph-mon[31518]: pgmap v3878: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:30:19.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:30:18 smithi132 ceph-mon[35941]: pgmap v3878: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:30:21.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:30:20 smithi165 ceph-mon[36302]: pgmap v3879: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:30:21.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:30:20 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6880.759000 secs 2024-03-28T02:30:21.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:30:20 smithi119 ceph-mon[31518]: pgmap v3879: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:30:21.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:30:20 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6880.759000 secs 2024-03-28T02:30:21.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:30:20 smithi132 ceph-mon[35941]: pgmap v3879: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:30:21.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:30:20 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6880.759000 secs 2024-03-28T02:30:23.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:30:22 smithi165 ceph-mon[36302]: pgmap v3880: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:30:23.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:30:22 smithi119 ceph-mon[31518]: pgmap v3880: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:30:23.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:30:22 smithi132 ceph-mon[35941]: pgmap v3880: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:30:25.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:30:24 smithi165 ceph-mon[36302]: pgmap v3881: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:30:25.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:30:24 smithi119 ceph-mon[31518]: pgmap v3881: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:30:25.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:30:24 smithi132 ceph-mon[35941]: pgmap v3881: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:30:26.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:30:25 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6885.759019 secs 2024-03-28T02:30:26.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:30:25 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:30:26.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:30:25 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6885.759019 secs 2024-03-28T02:30:26.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:30:25 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:30:26.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:30:25 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6885.759019 secs 2024-03-28T02:30:26.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:30:25 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:30:27.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:30:26 smithi165 ceph-mon[36302]: pgmap v3882: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:30:27.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:30:26 smithi119 ceph-mon[31518]: pgmap v3882: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:30:27.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:30:26 smithi132 ceph-mon[35941]: pgmap v3882: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:30:28.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:30:28 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:30:28.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:30:28 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:30:28.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:30:28 smithi165 ceph-mon[36302]: pgmap v3883: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T02:30:28.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:30:28 smithi165 ceph-mon[36302]: pgmap v3884: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:30:28.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:30:28 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:30:28.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:30:28 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:30:28.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:30:28 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:30:28.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:30:28 smithi119 ceph-mon[31518]: pgmap v3883: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T02:30:28.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:30:28 smithi119 ceph-mon[31518]: pgmap v3884: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:30:28.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:30:28 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:30:28.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:30:28 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:30:28.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:30:28 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:30:28.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:30:28 smithi132 ceph-mon[35941]: pgmap v3883: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T02:30:28.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:30:28 smithi132 ceph-mon[35941]: pgmap v3884: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:30:28.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:30:28 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:30:30.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:30:30 smithi119 ceph-mon[31518]: pgmap v3885: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 4 op/s 2024-03-28T02:30:30.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:30:30 smithi132 ceph-mon[35941]: pgmap v3885: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 4 op/s 2024-03-28T02:30:30.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:30:30 smithi165 ceph-mon[36302]: pgmap v3885: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 4 op/s 2024-03-28T02:30:31.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:30:31 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6890.759103 secs 2024-03-28T02:30:31.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:30:31 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6890.759103 secs 2024-03-28T02:30:31.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:30:31 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6890.759103 secs 2024-03-28T02:30:32.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:30:32 smithi119 ceph-mon[31518]: pgmap v3886: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:30:32.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:30:32 smithi132 ceph-mon[35941]: pgmap v3886: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:30:32.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:30:32 smithi165 ceph-mon[36302]: pgmap v3886: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:30:34.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:30:34 smithi132 ceph-mon[35941]: pgmap v3887: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:30:34.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:30:34 smithi165 ceph-mon[36302]: pgmap v3887: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:30:34.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:30:34 smithi119 ceph-mon[31518]: pgmap v3887: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:30:36.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:30:36 smithi132 ceph-mon[35941]: pgmap v3888: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:30:36.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:30:36 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6895.759178 secs 2024-03-28T02:30:36.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:30:36 smithi165 ceph-mon[36302]: pgmap v3888: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:30:36.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:30:36 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6895.759178 secs 2024-03-28T02:30:36.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:30:36 smithi119 ceph-mon[31518]: pgmap v3888: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:30:36.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:30:36 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6895.759178 secs 2024-03-28T02:30:38.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:30:38 smithi132 ceph-mon[35941]: pgmap v3889: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T02:30:38.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:30:38 smithi165 ceph-mon[36302]: pgmap v3889: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T02:30:38.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:30:38 smithi119 ceph-mon[31518]: pgmap v3889: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T02:30:40.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:30:40 smithi132 ceph-mon[35941]: pgmap v3890: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-28T02:30:40.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:30:40 smithi165 ceph-mon[36302]: pgmap v3890: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-28T02:30:40.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:30:40 smithi119 ceph-mon[31518]: pgmap v3890: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-28T02:30:41.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:30:41 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6900.759240 secs 2024-03-28T02:30:41.527 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:30:41 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6900.759240 secs 2024-03-28T02:30:41.550 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:30:41 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6900.759240 secs 2024-03-28T02:30:42.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:30:42 smithi132 ceph-mon[35941]: pgmap v3891: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-28T02:30:42.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:30:42 smithi165 ceph-mon[36302]: pgmap v3891: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-28T02:30:42.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:30:42 smithi119 ceph-mon[31518]: pgmap v3891: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-28T02:30:43.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:30:43 smithi119 ceph-mon[31518]: pgmap v3892: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:30:43.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:30:43 smithi132 ceph-mon[35941]: pgmap v3892: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:30:43.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:30:43 smithi165 ceph-mon[36302]: pgmap v3892: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:30:46.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:30:46 smithi119 ceph-mon[31518]: pgmap v3893: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:30:46.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:30:46 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6905.759300 secs 2024-03-28T02:30:46.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:30:46 smithi132 ceph-mon[35941]: pgmap v3893: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:30:46.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:30:46 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6905.759300 secs 2024-03-28T02:30:46.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:30:46 smithi165 ceph-mon[36302]: pgmap v3893: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:30:46.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:30:46 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6905.759300 secs 2024-03-28T02:30:48.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:30:48 smithi119 ceph-mon[31518]: pgmap v3894: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:30:48.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:30:48 smithi132 ceph-mon[35941]: pgmap v3894: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:30:48.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:30:48 smithi165 ceph-mon[36302]: pgmap v3894: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:30:50.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:30:50 smithi132 ceph-mon[35941]: pgmap v3895: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-28T02:30:50.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:30:50 smithi165 ceph-mon[36302]: pgmap v3895: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-28T02:30:50.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:30:50 smithi119 ceph-mon[31518]: pgmap v3895: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-28T02:30:51.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:30:51 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6910.759362 secs 2024-03-28T02:30:51.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:30:51 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6910.759362 secs 2024-03-28T02:30:51.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:30:51 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6910.759362 secs 2024-03-28T02:30:52.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:30:52 smithi132 ceph-mon[35941]: pgmap v3896: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T02:30:52.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:30:52 smithi165 ceph-mon[36302]: pgmap v3896: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T02:30:52.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:30:52 smithi119 ceph-mon[31518]: pgmap v3896: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T02:30:54.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:30:54 smithi132 ceph-mon[35941]: pgmap v3897: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:30:54.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:30:54 smithi165 ceph-mon[36302]: pgmap v3897: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:30:54.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:30:54 smithi119 ceph-mon[31518]: pgmap v3897: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:30:56.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:30:56 smithi132 ceph-mon[35941]: pgmap v3898: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:30:56.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:30:56 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6915.759422 secs 2024-03-28T02:30:56.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:30:56 smithi165 ceph-mon[36302]: pgmap v3898: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:30:56.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:30:56 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6915.759422 secs 2024-03-28T02:30:56.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:30:56 smithi119 ceph-mon[31518]: pgmap v3898: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:30:56.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:30:56 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6915.759422 secs 2024-03-28T02:30:58.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:30:58 smithi132 ceph-mon[35941]: pgmap v3899: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:30:58.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:30:58 smithi165 ceph-mon[36302]: pgmap v3899: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:30:58.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:30:58 smithi119 ceph-mon[31518]: pgmap v3899: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:30:59.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:30:59 smithi132 ceph-mon[35941]: pgmap v3900: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-28T02:30:59.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:30:59 smithi165 ceph-mon[36302]: pgmap v3900: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-28T02:30:59.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:30:59 smithi119 ceph-mon[31518]: pgmap v3900: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-28T02:31:01.126 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:31:00 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6920.759511 secs 2024-03-28T02:31:01.149 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:31:00 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6920.759511 secs 2024-03-28T02:31:01.196 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:31:00 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6920.759511 secs 2024-03-28T02:31:02.126 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:31:01 smithi165 ceph-mon[36302]: pgmap v3901: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T02:31:02.149 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:31:01 smithi119 ceph-mon[31518]: pgmap v3901: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T02:31:02.196 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:31:01 smithi132 ceph-mon[35941]: pgmap v3901: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T02:31:04.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:31:04 smithi132 ceph-mon[35941]: pgmap v3902: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:31:04.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:31:04 smithi165 ceph-mon[36302]: pgmap v3902: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:31:04.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:31:04 smithi119 ceph-mon[31518]: pgmap v3902: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:31:06.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:31:06 smithi132 ceph-mon[35941]: pgmap v3903: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:31:06.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:31:06 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6925.759601 secs 2024-03-28T02:31:06.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:31:06 smithi165 ceph-mon[36302]: pgmap v3903: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:31:06.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:31:06 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6925.759601 secs 2024-03-28T02:31:06.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:31:06 smithi119 ceph-mon[31518]: pgmap v3903: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:31:06.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:31:06 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6925.759601 secs 2024-03-28T02:31:08.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:31:08 smithi132 ceph-mon[35941]: pgmap v3904: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:31:08.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:31:08 smithi165 ceph-mon[36302]: pgmap v3904: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:31:08.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:31:08 smithi119 ceph-mon[31518]: pgmap v3904: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:31:10.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:31:10 smithi132 ceph-mon[35941]: pgmap v3905: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-28T02:31:10.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:31:10 smithi165 ceph-mon[36302]: pgmap v3905: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-28T02:31:10.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:31:10 smithi119 ceph-mon[31518]: pgmap v3905: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-28T02:31:11.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:31:11 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6930.759690 secs 2024-03-28T02:31:11.528 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:31:11 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6930.759690 secs 2024-03-28T02:31:11.551 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:31:11 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6930.759690 secs 2024-03-28T02:31:12.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:31:12 smithi132 ceph-mon[35941]: pgmap v3906: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T02:31:12.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:31:12 smithi165 ceph-mon[36302]: pgmap v3906: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T02:31:12.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:31:12 smithi119 ceph-mon[31518]: pgmap v3906: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T02:31:14.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:31:14 smithi132 ceph-mon[35941]: pgmap v3907: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:31:14.452 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:31:14 smithi119 ceph-mon[31518]: pgmap v3907: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:31:14.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:31:14 smithi165 ceph-mon[36302]: pgmap v3907: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:31:16.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:31:16 smithi132 ceph-mon[35941]: pgmap v3908: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:31:16.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:31:16 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6935.759780 secs 2024-03-28T02:31:16.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:31:16 smithi165 ceph-mon[36302]: pgmap v3908: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:31:16.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:31:16 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6935.759780 secs 2024-03-28T02:31:16.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:31:16 smithi119 ceph-mon[31518]: pgmap v3908: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:31:16.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:31:16 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6935.759780 secs 2024-03-28T02:31:17.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:31:17 smithi165 ceph-mon[36302]: pgmap v3909: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:31:17.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:31:17 smithi119 ceph-mon[31518]: pgmap v3909: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:31:17.696 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:31:17 smithi132 ceph-mon[35941]: pgmap v3909: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:31:20.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:31:20 smithi132 ceph-mon[35941]: pgmap v3910: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-28T02:31:20.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:31:20 smithi165 ceph-mon[36302]: pgmap v3910: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-28T02:31:20.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:31:20 smithi119 ceph-mon[31518]: pgmap v3910: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-28T02:31:21.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:31:21 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6940.759843 secs 2024-03-28T02:31:21.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:31:21 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6940.759843 secs 2024-03-28T02:31:21.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:31:21 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6940.759843 secs 2024-03-28T02:31:22.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:31:22 smithi132 ceph-mon[35941]: pgmap v3911: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T02:31:22.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:31:22 smithi165 ceph-mon[36302]: pgmap v3911: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T02:31:22.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:31:22 smithi119 ceph-mon[31518]: pgmap v3911: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T02:31:24.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:31:24 smithi132 ceph-mon[35941]: pgmap v3912: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:31:24.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:31:24 smithi165 ceph-mon[36302]: pgmap v3912: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:31:24.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:31:24 smithi119 ceph-mon[31518]: pgmap v3912: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:31:26.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:31:26 smithi132 ceph-mon[35941]: pgmap v3913: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:31:26.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:31:26 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6945.759904 secs 2024-03-28T02:31:26.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:31:26 smithi165 ceph-mon[36302]: pgmap v3913: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:31:26.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:31:26 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6945.759904 secs 2024-03-28T02:31:26.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:31:26 smithi119 ceph-mon[31518]: pgmap v3913: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:31:26.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:31:26 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6945.759904 secs 2024-03-28T02:31:28.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:31:28 smithi132 ceph-mon[35941]: pgmap v3914: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:31:28.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:31:28 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:31:28.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:31:28 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:31:28.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:31:28 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:31:28.476 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:31:28 smithi165 ceph-mon[36302]: pgmap v3914: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:31:28.476 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:31:28 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:31:28.476 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:31:28 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:31:28.476 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:31:28 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:31:28.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:31:28 smithi119 ceph-mon[31518]: pgmap v3914: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:31:28.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:31:28 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:31:28.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:31:28 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:31:28.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:31:28 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:31:29.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:31:29 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:31:29.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:31:29 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:31:29.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:31:29 smithi132 ceph-mon[35941]: pgmap v3915: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-28T02:31:29.598 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:31:29 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:31:29.598 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:31:29 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:31:29.598 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:31:29 smithi119 ceph-mon[31518]: pgmap v3915: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-28T02:31:29.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:31:29 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:31:29.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:31:29 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:31:29.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:31:29 smithi165 ceph-mon[36302]: pgmap v3915: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-28T02:31:30.947 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:31:30 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:31:30.947 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:31:30 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:31:30.948 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:31:30 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:31:30.948 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:31:30 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:31:30.948 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:31:30 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:31:30.948 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:31:30 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:31:30.948 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:31:30 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:31:30.948 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:31:30 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:31:30.948 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:31:30 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:31:30.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:31:30 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:31:30.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:31:30 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:31:30.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:31:30 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:31:30.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:31:30 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:31:30.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:31:30 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:31:30.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:31:30 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:31:30.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:31:30 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:31:30.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:31:30 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:31:30.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:31:30 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:31:30.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:31:30 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:31:30.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:31:30 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:31:31.127 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:31:30 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:31:31.127 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:31:30 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:31:31.127 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:31:30 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:31:31.127 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:31:30 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:31:31.127 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:31:30 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:31:31.127 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:31:30 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:31:31.127 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:31:30 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:31:31.127 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:31:30 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:31:31.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:31:30 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:31:31.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:31:30 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:31:31.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:31:30 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:31:31.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:31:30 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:31:31.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:31:30 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:31:31.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:31:30 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:31:31.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:31:30 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:31:31.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:31:30 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:31:31.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:31:30 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:31:31.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:31:30 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:31:31.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:31:30 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:31:31.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:31:30 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:31:31.149 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:31:30 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:31:31.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:31:30 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:31:31.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:31:30 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:31:31.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:31:30 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:31:31.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:31:30 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:31:31.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:31:30 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:31:31.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:31:30 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:31:31.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:31:30 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:31:31.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:31:30 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:31:31.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:31:30 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:31:31.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:31:30 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:31:31.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:31:30 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:31:31.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:31:30 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:31:31.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:31:30 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:31:31.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:31:30 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:31:31.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:31:30 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:31:31.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:31:30 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:31:31.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:31:30 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:31:31.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:31:30 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:31:31.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:31:30 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:31:31.947 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:31:31 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6950.759969 secs 2024-03-28T02:31:31.947 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:31:31 smithi132 ceph-mon[35941]: Adjusting osd_memory_target on smithi165 to 174.1M 2024-03-28T02:31:31.947 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:31:31 smithi132 ceph-mon[35941]: Unable to set osd_memory_target on smithi165 to 182635622: error parsing value: Value '182635622' is below minimum 939524096 2024-03-28T02:31:31.947 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:31:31 smithi132 ceph-mon[35941]: Adjusting osd_memory_target on smithi132 to 174.1M 2024-03-28T02:31:31.947 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:31:31 smithi132 ceph-mon[35941]: Unable to set osd_memory_target on smithi132 to 182640640: error parsing value: Value '182640640' is below minimum 939524096 2024-03-28T02:31:31.948 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:31:31 smithi132 ceph-mon[35941]: pgmap v3916: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T02:31:32.127 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:31:31 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6950.759969 secs 2024-03-28T02:31:32.127 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:31:31 smithi165 ceph-mon[36302]: Adjusting osd_memory_target on smithi165 to 174.1M 2024-03-28T02:31:32.127 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:31:31 smithi165 ceph-mon[36302]: Unable to set osd_memory_target on smithi165 to 182635622: error parsing value: Value '182635622' is below minimum 939524096 2024-03-28T02:31:32.127 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:31:31 smithi165 ceph-mon[36302]: Adjusting osd_memory_target on smithi132 to 174.1M 2024-03-28T02:31:32.127 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:31:31 smithi165 ceph-mon[36302]: Unable to set osd_memory_target on smithi132 to 182640640: error parsing value: Value '182640640' is below minimum 939524096 2024-03-28T02:31:32.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:31:31 smithi165 ceph-mon[36302]: pgmap v3916: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T02:31:32.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:31:31 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6950.759969 secs 2024-03-28T02:31:32.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:31:31 smithi119 ceph-mon[31518]: Adjusting osd_memory_target on smithi165 to 174.1M 2024-03-28T02:31:32.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:31:31 smithi119 ceph-mon[31518]: Unable to set osd_memory_target on smithi165 to 182635622: error parsing value: Value '182635622' is below minimum 939524096 2024-03-28T02:31:32.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:31:31 smithi119 ceph-mon[31518]: Adjusting osd_memory_target on smithi132 to 174.1M 2024-03-28T02:31:32.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:31:31 smithi119 ceph-mon[31518]: Unable to set osd_memory_target on smithi132 to 182640640: error parsing value: Value '182640640' is below minimum 939524096 2024-03-28T02:31:32.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:31:31 smithi119 ceph-mon[31518]: pgmap v3916: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T02:31:33.127 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:31:32 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:31:33.127 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:31:32 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:31:33.149 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:31:32 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:31:33.149 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:31:32 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:31:33.196 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:31:32 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:31:33.196 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:31:32 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:31:34.127 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:31:33 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:31:34.127 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:31:33 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:31:34.127 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:31:33 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:31:34.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:31:33 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:31:34.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:31:33 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:31:34.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:31:33 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:31:34.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:31:33 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:31:34.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:31:33 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:31:34.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:31:33 smithi165 ceph-mon[36302]: Adjusting osd_memory_target on smithi119 to 176.7M 2024-03-28T02:31:34.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:31:33 smithi165 ceph-mon[36302]: Unable to set osd_memory_target on smithi119 to 185372364: error parsing value: Value '185372364' is below minimum 939524096 2024-03-28T02:31:34.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:31:33 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:31:34.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:31:33 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:31:34.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:31:33 smithi165 ceph-mon[36302]: pgmap v3917: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T02:31:34.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:31:33 smithi165 ceph-mon[36302]: pgmap v3918: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T02:31:34.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:31:33 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:31:34.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:31:33 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:31:34.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:31:33 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:31:34.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:31:33 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:31:34.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:31:33 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:31:34.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:31:33 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:31:34.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:31:33 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:31:34.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:31:33 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:31:34.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:31:33 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:31:34.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:31:33 smithi119 ceph-mon[31518]: Adjusting osd_memory_target on smithi119 to 176.7M 2024-03-28T02:31:34.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:31:33 smithi119 ceph-mon[31518]: Unable to set osd_memory_target on smithi119 to 185372364: error parsing value: Value '185372364' is below minimum 939524096 2024-03-28T02:31:34.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:31:33 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:31:34.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:31:33 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:31:34.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:31:33 smithi119 ceph-mon[31518]: pgmap v3917: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T02:31:34.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:31:33 smithi119 ceph-mon[31518]: pgmap v3918: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T02:31:34.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:31:33 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:31:34.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:31:33 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:31:34.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:31:33 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:31:34.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:31:33 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:31:34.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:31:33 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:31:34.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:31:33 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:31:34.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:31:33 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:31:34.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:31:33 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:31:34.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:31:33 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:31:34.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:31:33 smithi132 ceph-mon[35941]: Adjusting osd_memory_target on smithi119 to 176.7M 2024-03-28T02:31:34.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:31:33 smithi132 ceph-mon[35941]: Unable to set osd_memory_target on smithi119 to 185372364: error parsing value: Value '185372364' is below minimum 939524096 2024-03-28T02:31:34.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:31:33 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:31:34.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:31:33 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:31:34.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:31:33 smithi132 ceph-mon[35941]: pgmap v3917: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T02:31:34.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:31:33 smithi132 ceph-mon[35941]: pgmap v3918: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T02:31:34.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:31:33 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:31:36.377 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:31:36 smithi165 ceph-mon[36302]: pgmap v3919: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T02:31:36.377 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:31:36 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6955.759986 secs 2024-03-28T02:31:36.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:31:36 smithi119 ceph-mon[31518]: pgmap v3919: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T02:31:36.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:31:36 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6955.759986 secs 2024-03-28T02:31:36.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:31:36 smithi132 ceph-mon[35941]: pgmap v3919: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T02:31:36.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:31:36 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6955.759986 secs 2024-03-28T02:31:38.377 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:31:38 smithi165 ceph-mon[36302]: pgmap v3920: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T02:31:38.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:31:38 smithi119 ceph-mon[31518]: pgmap v3920: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T02:31:38.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:31:38 smithi132 ceph-mon[35941]: pgmap v3920: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T02:31:40.377 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:31:40 smithi165 ceph-mon[36302]: pgmap v3921: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 4 op/s 2024-03-28T02:31:40.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:31:40 smithi119 ceph-mon[31518]: pgmap v3921: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 4 op/s 2024-03-28T02:31:40.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:31:40 smithi132 ceph-mon[35941]: pgmap v3921: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 4 op/s 2024-03-28T02:31:41.377 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:31:41 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6960.760051 secs 2024-03-28T02:31:41.400 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:31:41 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6960.760051 secs 2024-03-28T02:31:41.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:31:41 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6960.760051 secs 2024-03-28T02:31:42.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:31:42 smithi165 ceph-mon[36302]: pgmap v3922: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T02:31:42.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:31:42 smithi119 ceph-mon[31518]: pgmap v3922: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T02:31:42.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:31:42 smithi132 ceph-mon[35941]: pgmap v3922: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T02:31:44.377 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:31:44 smithi165 ceph-mon[36302]: pgmap v3923: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T02:31:44.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:31:44 smithi119 ceph-mon[31518]: pgmap v3923: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T02:31:44.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:31:44 smithi132 ceph-mon[35941]: pgmap v3923: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T02:31:46.377 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:31:46 smithi165 ceph-mon[36302]: pgmap v3924: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:31:46.377 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:31:46 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6965.760138 secs 2024-03-28T02:31:46.400 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:31:46 smithi119 ceph-mon[31518]: pgmap v3924: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:31:46.400 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:31:46 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6965.760138 secs 2024-03-28T02:31:46.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:31:46 smithi132 ceph-mon[35941]: pgmap v3924: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:31:46.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:31:46 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6965.760138 secs 2024-03-28T02:31:48.377 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:31:48 smithi165 ceph-mon[36302]: pgmap v3925: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:31:48.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:31:48 smithi119 ceph-mon[31518]: pgmap v3925: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:31:48.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:31:48 smithi132 ceph-mon[35941]: pgmap v3925: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:31:50.377 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:31:50 smithi165 ceph-mon[36302]: pgmap v3926: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:31:50.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:31:50 smithi119 ceph-mon[31518]: pgmap v3926: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:31:50.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:31:50 smithi132 ceph-mon[35941]: pgmap v3926: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:31:51.377 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:31:51 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6970.760226 secs 2024-03-28T02:31:51.400 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:31:51 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6970.760226 secs 2024-03-28T02:31:51.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:31:51 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6970.760226 secs 2024-03-28T02:31:52.377 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:31:52 smithi165 ceph-mon[36302]: pgmap v3927: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:31:52.399 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:31:52 smithi119 ceph-mon[31518]: pgmap v3927: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:31:52.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:31:52 smithi132 ceph-mon[35941]: pgmap v3927: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:31:54.377 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:31:54 smithi165 ceph-mon[36302]: pgmap v3928: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:31:54.400 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:31:54 smithi119 ceph-mon[31518]: pgmap v3928: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:31:54.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:31:54 smithi132 ceph-mon[35941]: pgmap v3928: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:31:56.400 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:31:56 smithi119 ceph-mon[31518]: pgmap v3929: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:31:56.400 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:31:56 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6975.760287 secs 2024-03-28T02:31:56.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:31:56 smithi132 ceph-mon[35941]: pgmap v3929: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:31:56.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:31:56 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6975.760287 secs 2024-03-28T02:31:56.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:31:56 smithi165 ceph-mon[36302]: pgmap v3929: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:31:56.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:31:56 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6975.760287 secs 2024-03-28T02:31:58.400 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:31:58 smithi119 ceph-mon[31518]: pgmap v3930: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:31:58.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:31:58 smithi132 ceph-mon[35941]: pgmap v3930: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:31:58.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:31:58 smithi165 ceph-mon[36302]: pgmap v3930: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:32:00.400 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:32:00 smithi119 ceph-mon[31518]: pgmap v3931: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:32:00.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:32:00 smithi132 ceph-mon[35941]: pgmap v3931: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:32:00.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:32:00 smithi165 ceph-mon[36302]: pgmap v3931: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:32:01.400 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:32:01 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6980.760346 secs 2024-03-28T02:32:01.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:32:01 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6980.760346 secs 2024-03-28T02:32:01.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:32:01 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6980.760346 secs 2024-03-28T02:32:02.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:32:02 smithi132 ceph-mon[35941]: pgmap v3932: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:32:02.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:32:02 smithi165 ceph-mon[36302]: pgmap v3932: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:32:02.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:32:02 smithi119 ceph-mon[31518]: pgmap v3932: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:32:04.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:32:04 smithi132 ceph-mon[35941]: pgmap v3933: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:32:04.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:32:04 smithi165 ceph-mon[36302]: pgmap v3933: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:32:04.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:32:04 smithi119 ceph-mon[31518]: pgmap v3933: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:32:06.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:32:06 smithi132 ceph-mon[35941]: pgmap v3934: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:32:06.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:32:06 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6985.760387 secs 2024-03-28T02:32:06.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:32:06 smithi165 ceph-mon[36302]: pgmap v3934: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:32:06.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:32:06 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6985.760387 secs 2024-03-28T02:32:06.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:32:06 smithi119 ceph-mon[31518]: pgmap v3934: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:32:06.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:32:06 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6985.760387 secs 2024-03-28T02:32:08.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:32:08 smithi132 ceph-mon[35941]: pgmap v3935: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:32:08.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:32:08 smithi165 ceph-mon[36302]: pgmap v3935: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:32:08.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:32:08 smithi119 ceph-mon[31518]: pgmap v3935: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:32:10.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:32:10 smithi132 ceph-mon[35941]: pgmap v3936: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:32:10.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:32:10 smithi165 ceph-mon[36302]: pgmap v3936: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:32:10.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:32:10 smithi119 ceph-mon[31518]: pgmap v3936: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:32:11.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:32:11 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6990.760403 secs 2024-03-28T02:32:11.529 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:32:11 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6990.760403 secs 2024-03-28T02:32:11.551 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:32:11 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6990.760403 secs 2024-03-28T02:32:12.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:32:12 smithi132 ceph-mon[35941]: pgmap v3937: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:32:12.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:32:12 smithi165 ceph-mon[36302]: pgmap v3937: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:32:12.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:32:12 smithi119 ceph-mon[31518]: pgmap v3937: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:32:14.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:32:14 smithi132 ceph-mon[35941]: pgmap v3938: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:32:14.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:32:14 smithi165 ceph-mon[36302]: pgmap v3938: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:32:14.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:32:14 smithi119 ceph-mon[31518]: pgmap v3938: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:32:16.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:32:16 smithi132 ceph-mon[35941]: pgmap v3939: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:32:16.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:32:16 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 6995.760491 secs 2024-03-28T02:32:16.452 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:32:16 smithi119 ceph-mon[31518]: pgmap v3939: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:32:16.453 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:32:16 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 6995.760491 secs 2024-03-28T02:32:16.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:32:16 smithi165 ceph-mon[36302]: pgmap v3939: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:32:16.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:32:16 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 6995.760491 secs 2024-03-28T02:32:18.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:32:18 smithi132 ceph-mon[35941]: pgmap v3940: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:32:18.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:32:18 smithi165 ceph-mon[36302]: pgmap v3940: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:32:18.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:32:18 smithi119 ceph-mon[31518]: pgmap v3940: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:32:19.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:32:19 smithi165 ceph-mon[36302]: pgmap v3941: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:32:19.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:32:19 smithi119 ceph-mon[31518]: pgmap v3941: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:32:19.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:32:19 smithi132 ceph-mon[35941]: pgmap v3941: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:32:21.127 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:32:20 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7000.760553 secs 2024-03-28T02:32:21.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:32:20 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7000.760553 secs 2024-03-28T02:32:21.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:32:20 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7000.760553 secs 2024-03-28T02:32:22.127 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:32:21 smithi165 ceph-mon[36302]: pgmap v3942: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:32:22.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:32:21 smithi119 ceph-mon[31518]: pgmap v3942: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:32:22.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:32:21 smithi132 ceph-mon[35941]: pgmap v3942: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:32:24.377 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:32:24 smithi165 ceph-mon[36302]: pgmap v3943: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:32:24.400 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:32:24 smithi119 ceph-mon[31518]: pgmap v3943: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:32:24.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:32:24 smithi132 ceph-mon[35941]: pgmap v3943: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:32:26.377 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:32:26 smithi165 ceph-mon[36302]: pgmap v3944: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:32:26.377 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:32:26 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7005.760640 secs 2024-03-28T02:32:26.400 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:32:26 smithi119 ceph-mon[31518]: pgmap v3944: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:32:26.400 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:32:26 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7005.760640 secs 2024-03-28T02:32:26.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:32:26 smithi132 ceph-mon[35941]: pgmap v3944: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:32:26.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:32:26 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7005.760640 secs 2024-03-28T02:32:28.377 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:32:28 smithi165 ceph-mon[36302]: pgmap v3945: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:32:28.400 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:32:28 smithi119 ceph-mon[31518]: pgmap v3945: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:32:28.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:32:28 smithi132 ceph-mon[35941]: pgmap v3945: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:32:30.377 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:32:30 smithi165 ceph-mon[36302]: pgmap v3946: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:32:30.400 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:32:30 smithi119 ceph-mon[31518]: pgmap v3946: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:32:30.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:32:30 smithi132 ceph-mon[35941]: pgmap v3946: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:32:31.377 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:32:31 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7010.760706 secs 2024-03-28T02:32:31.400 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:32:31 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7010.760706 secs 2024-03-28T02:32:31.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:32:31 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7010.760706 secs 2024-03-28T02:32:32.377 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:32:32 smithi165 ceph-mon[36302]: pgmap v3947: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:32:32.400 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:32:32 smithi119 ceph-mon[31518]: pgmap v3947: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:32:32.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:32:32 smithi132 ceph-mon[35941]: pgmap v3947: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:32:34.377 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:32:34 smithi165 ceph-mon[36302]: pgmap v3948: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:32:34.377 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:32:34 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:32:34.377 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:32:34 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:32:34.377 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:32:34 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:32:34.400 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:32:34 smithi119 ceph-mon[31518]: pgmap v3948: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:32:34.400 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:32:34 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:32:34.400 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:32:34 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:32:34.400 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:32:34 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:32:34.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:32:34 smithi132 ceph-mon[35941]: pgmap v3948: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:32:34.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:32:34 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:32:34.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:32:34 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:32:34.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:32:34 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:32:35.400 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:32:35 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:32:35.400 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:32:35 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:32:35.400 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:32:35 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:32:35.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:32:35 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:32:35.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:32:35 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:32:35.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:32:35 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:32:35.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:32:35 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:32:35.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:32:35 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:32:35.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:32:35 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:32:36.400 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:32:36 smithi119 ceph-mon[31518]: pgmap v3949: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T02:32:36.400 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:32:36 smithi119 ceph-mon[31518]: pgmap v3950: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:32:36.400 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:32:36 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7015.760793 secs 2024-03-28T02:32:36.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:32:36 smithi132 ceph-mon[35941]: pgmap v3949: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T02:32:36.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:32:36 smithi132 ceph-mon[35941]: pgmap v3950: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:32:36.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:32:36 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7015.760793 secs 2024-03-28T02:32:36.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:32:36 smithi165 ceph-mon[36302]: pgmap v3949: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T02:32:36.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:32:36 smithi165 ceph-mon[36302]: pgmap v3950: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:32:36.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:32:36 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7015.760793 secs 2024-03-28T02:32:38.400 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:32:38 smithi119 ceph-mon[31518]: pgmap v3951: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:32:38.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:32:38 smithi132 ceph-mon[35941]: pgmap v3951: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:32:38.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:32:38 smithi165 ceph-mon[36302]: pgmap v3951: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:32:40.400 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:32:40 smithi119 ceph-mon[31518]: pgmap v3952: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:32:40.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:32:40 smithi132 ceph-mon[35941]: pgmap v3952: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:32:40.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:32:40 smithi165 ceph-mon[36302]: pgmap v3952: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:32:41.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:32:41 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7020.760883 secs 2024-03-28T02:32:41.527 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:32:41 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7020.760883 secs 2024-03-28T02:32:41.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:32:41 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7020.760883 secs 2024-03-28T02:32:42.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:32:42 smithi132 ceph-mon[35941]: pgmap v3953: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:32:42.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:32:42 smithi165 ceph-mon[36302]: pgmap v3953: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:32:42.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:32:42 smithi119 ceph-mon[31518]: pgmap v3953: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:32:44.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:32:44 smithi132 ceph-mon[35941]: pgmap v3954: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:32:44.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:32:44 smithi165 ceph-mon[36302]: pgmap v3954: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:32:44.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:32:44 smithi119 ceph-mon[31518]: pgmap v3954: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:32:46.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:32:46 smithi132 ceph-mon[35941]: pgmap v3955: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T02:32:46.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:32:46 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7025.760945 secs 2024-03-28T02:32:46.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:32:46 smithi165 ceph-mon[36302]: pgmap v3955: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T02:32:46.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:32:46 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7025.760945 secs 2024-03-28T02:32:46.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:32:46 smithi119 ceph-mon[31518]: pgmap v3955: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T02:32:46.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:32:46 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7025.760945 secs 2024-03-28T02:32:48.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:32:48 smithi132 ceph-mon[35941]: pgmap v3956: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:32:48.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:32:48 smithi165 ceph-mon[36302]: pgmap v3956: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:32:48.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:32:48 smithi119 ceph-mon[31518]: pgmap v3956: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:32:50.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:32:50 smithi132 ceph-mon[35941]: pgmap v3957: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:32:50.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:32:50 smithi165 ceph-mon[36302]: pgmap v3957: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:32:50.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:32:50 smithi119 ceph-mon[31518]: pgmap v3957: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:32:51.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:32:51 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7030.761032 secs 2024-03-28T02:32:51.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:32:51 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7030.761032 secs 2024-03-28T02:32:51.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:32:51 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7030.761032 secs 2024-03-28T02:32:52.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:32:52 smithi132 ceph-mon[35941]: pgmap v3958: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:32:52.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:32:52 smithi165 ceph-mon[36302]: pgmap v3958: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:32:52.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:32:52 smithi119 ceph-mon[31518]: pgmap v3958: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:32:54.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:32:54 smithi132 ceph-mon[35941]: pgmap v3959: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:32:54.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:32:54 smithi165 ceph-mon[36302]: pgmap v3959: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:32:54.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:32:54 smithi119 ceph-mon[31518]: pgmap v3959: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:32:56.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:32:56 smithi132 ceph-mon[35941]: pgmap v3960: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:32:56.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:32:56 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7035.761122 secs 2024-03-28T02:32:56.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:32:56 smithi165 ceph-mon[36302]: pgmap v3960: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:32:56.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:32:56 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7035.761122 secs 2024-03-28T02:32:56.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:32:56 smithi119 ceph-mon[31518]: pgmap v3960: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:32:56.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:32:56 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7035.761122 secs 2024-03-28T02:32:58.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:32:58 smithi132 ceph-mon[35941]: pgmap v3961: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:32:58.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:32:58 smithi165 ceph-mon[36302]: pgmap v3961: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:32:58.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:32:58 smithi119 ceph-mon[31518]: pgmap v3961: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:33:00.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:33:00 smithi165 ceph-mon[36302]: pgmap v3962: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:33:00.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:33:00 smithi119 ceph-mon[31518]: pgmap v3962: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:33:00.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:33:00 smithi132 ceph-mon[35941]: pgmap v3962: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:33:01.598 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:33:01 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7040.761182 secs 2024-03-28T02:33:01.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:33:01 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7040.761182 secs 2024-03-28T02:33:01.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:33:01 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7040.761182 secs 2024-03-28T02:33:02.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:33:02 smithi165 ceph-mon[36302]: pgmap v3963: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:33:02.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:33:02 smithi119 ceph-mon[31518]: pgmap v3963: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:33:02.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:33:02 smithi132 ceph-mon[35941]: pgmap v3963: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:33:03.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:33:03 smithi165 ceph-mon[36302]: pgmap v3964: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:33:03.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:33:03 smithi119 ceph-mon[31518]: pgmap v3964: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:33:03.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:33:03 smithi132 ceph-mon[35941]: pgmap v3964: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:33:06.127 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:33:05 smithi165 ceph-mon[36302]: pgmap v3965: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:33:06.127 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:33:05 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7045.761242 secs 2024-03-28T02:33:06.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:33:05 smithi119 ceph-mon[31518]: pgmap v3965: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:33:06.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:33:05 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7045.761242 secs 2024-03-28T02:33:06.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:33:05 smithi132 ceph-mon[35941]: pgmap v3965: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:33:06.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:33:05 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7045.761242 secs 2024-03-28T02:33:08.127 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:33:07 smithi165 ceph-mon[36302]: pgmap v3966: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:33:08.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:33:07 smithi119 ceph-mon[31518]: pgmap v3966: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:33:08.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:33:07 smithi132 ceph-mon[35941]: pgmap v3966: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:33:10.127 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:33:09 smithi165 ceph-mon[36302]: pgmap v3967: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:33:10.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:33:09 smithi119 ceph-mon[31518]: pgmap v3967: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:33:10.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:33:09 smithi132 ceph-mon[35941]: pgmap v3967: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:33:11.028 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:33:10 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7050.761304 secs 2024-03-28T02:33:11.052 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:33:10 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7050.761304 secs 2024-03-28T02:33:11.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:33:10 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7050.761304 secs 2024-03-28T02:33:12.127 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:33:11 smithi165 ceph-mon[36302]: pgmap v3968: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:33:12.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:33:11 smithi119 ceph-mon[31518]: pgmap v3968: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:33:12.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:33:11 smithi132 ceph-mon[35941]: pgmap v3968: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:33:14.127 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:33:13 smithi165 ceph-mon[36302]: pgmap v3969: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:33:14.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:33:13 smithi119 ceph-mon[31518]: pgmap v3969: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:33:14.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:33:13 smithi132 ceph-mon[35941]: pgmap v3969: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:33:16.127 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:33:15 smithi165 ceph-mon[36302]: pgmap v3970: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:33:16.127 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:33:15 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7055.761395 secs 2024-03-28T02:33:16.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:33:15 smithi119 ceph-mon[31518]: pgmap v3970: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:33:16.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:33:15 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7055.761395 secs 2024-03-28T02:33:16.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:33:15 smithi132 ceph-mon[35941]: pgmap v3970: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:33:16.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:33:15 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7055.761395 secs 2024-03-28T02:33:18.112 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:33:17 smithi119 ceph-mon[31518]: pgmap v3971: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:33:18.127 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:33:17 smithi165 ceph-mon[36302]: pgmap v3971: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:33:18.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:33:17 smithi132 ceph-mon[35941]: pgmap v3971: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:33:20.127 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:33:19 smithi165 ceph-mon[36302]: pgmap v3972: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:33:20.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:33:19 smithi119 ceph-mon[31518]: pgmap v3972: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:33:20.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:33:19 smithi132 ceph-mon[35941]: pgmap v3972: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:33:21.127 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:33:20 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7060.761484 secs 2024-03-28T02:33:21.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:33:20 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7060.761484 secs 2024-03-28T02:33:21.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:33:20 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7060.761484 secs 2024-03-28T02:33:22.127 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:33:21 smithi165 ceph-mon[36302]: pgmap v3973: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:33:22.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:33:21 smithi119 ceph-mon[31518]: pgmap v3973: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:33:22.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:33:21 smithi132 ceph-mon[35941]: pgmap v3973: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:33:24.127 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:33:23 smithi165 ceph-mon[36302]: pgmap v3974: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:33:24.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:33:23 smithi119 ceph-mon[31518]: pgmap v3974: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:33:24.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:33:23 smithi132 ceph-mon[35941]: pgmap v3974: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:33:26.127 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:33:25 smithi165 ceph-mon[36302]: pgmap v3975: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:33:26.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:33:25 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7065.761573 secs 2024-03-28T02:33:26.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:33:25 smithi119 ceph-mon[31518]: pgmap v3975: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:33:26.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:33:25 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7065.761573 secs 2024-03-28T02:33:26.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:33:25 smithi132 ceph-mon[35941]: pgmap v3975: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:33:26.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:33:25 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7065.761573 secs 2024-03-28T02:33:28.127 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:33:27 smithi165 ceph-mon[36302]: pgmap v3976: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:33:28.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:33:27 smithi119 ceph-mon[31518]: pgmap v3976: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:33:28.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:33:27 smithi132 ceph-mon[35941]: pgmap v3976: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:33:30.127 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:33:29 smithi165 ceph-mon[36302]: pgmap v3977: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:33:30.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:33:29 smithi119 ceph-mon[31518]: pgmap v3977: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:33:30.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:33:29 smithi132 ceph-mon[35941]: pgmap v3977: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:33:31.099 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:33:30 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7070.761635 secs 2024-03-28T02:33:31.127 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:33:30 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7070.761635 secs 2024-03-28T02:33:31.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:33:30 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7070.761635 secs 2024-03-28T02:33:32.127 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:33:31 smithi165 ceph-mon[36302]: pgmap v3978: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:33:32.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:33:31 smithi119 ceph-mon[31518]: pgmap v3978: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:33:32.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:33:31 smithi132 ceph-mon[35941]: pgmap v3978: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:33:34.127 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:33:33 smithi165 ceph-mon[36302]: pgmap v3979: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:33:34.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:33:33 smithi119 ceph-mon[31518]: pgmap v3979: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:33:34.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:33:33 smithi132 ceph-mon[35941]: pgmap v3979: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:33:36.127 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:33:35 smithi165 ceph-mon[36302]: pgmap v3980: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:33:36.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:33:35 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:33:36.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:33:35 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7075.761707 secs 2024-03-28T02:33:36.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:33:35 smithi119 ceph-mon[31518]: pgmap v3980: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:33:36.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:33:35 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:33:36.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:33:35 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7075.761707 secs 2024-03-28T02:33:36.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:33:35 smithi132 ceph-mon[35941]: pgmap v3980: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:33:36.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:33:35 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:33:36.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:33:35 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7075.761707 secs 2024-03-28T02:33:37.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:33:36 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:33:37.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:33:36 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:33:37.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:33:36 smithi165 ceph-mon[36302]: pgmap v3981: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T02:33:37.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:33:36 smithi165 ceph-mon[36302]: pgmap v3982: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:33:37.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:33:36 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:33:37.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:33:36 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:33:37.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:33:36 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:33:37.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:33:36 smithi119 ceph-mon[31518]: pgmap v3981: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T02:33:37.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:33:36 smithi119 ceph-mon[31518]: pgmap v3982: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:33:37.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:33:36 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:33:37.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:33:36 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:33:37.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:33:36 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:33:37.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:33:36 smithi132 ceph-mon[35941]: pgmap v3981: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T02:33:37.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:33:36 smithi132 ceph-mon[35941]: pgmap v3982: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:33:37.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:33:36 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:33:39.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:33:39 smithi132 ceph-mon[35941]: pgmap v3983: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:33:39.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:33:39 smithi165 ceph-mon[36302]: pgmap v3983: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:33:39.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:33:39 smithi119 ceph-mon[31518]: pgmap v3983: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:33:41.430 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:33:41 smithi165 ceph-mon[36302]: pgmap v3984: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:33:41.430 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:33:41 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7080.761797 secs 2024-03-28T02:33:41.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:33:41 smithi132 ceph-mon[35941]: pgmap v3984: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:33:41.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:33:41 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7080.761797 secs 2024-03-28T02:33:41.453 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:33:41 smithi119 ceph-mon[31518]: pgmap v3984: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:33:41.453 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:33:41 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7080.761797 secs 2024-03-28T02:33:43.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:33:43 smithi132 ceph-mon[35941]: pgmap v3985: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:33:43.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:33:43 smithi165 ceph-mon[36302]: pgmap v3985: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:33:43.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:33:43 smithi119 ceph-mon[31518]: pgmap v3985: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:33:45.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:33:45 smithi132 ceph-mon[35941]: pgmap v3986: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 5 op/s 2024-03-28T02:33:45.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:33:45 smithi165 ceph-mon[36302]: pgmap v3986: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 5 op/s 2024-03-28T02:33:45.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:33:45 smithi119 ceph-mon[31518]: pgmap v3986: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 5 op/s 2024-03-28T02:33:46.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:33:46 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7085.761886 secs 2024-03-28T02:33:46.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:33:46 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7085.761886 secs 2024-03-28T02:33:46.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:33:46 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7085.761886 secs 2024-03-28T02:33:47.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:33:47 smithi165 ceph-mon[36302]: pgmap v3987: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T02:33:47.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:33:47 smithi119 ceph-mon[31518]: pgmap v3987: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T02:33:47.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:33:47 smithi132 ceph-mon[35941]: pgmap v3987: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T02:33:49.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:33:49 smithi165 ceph-mon[36302]: pgmap v3988: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:33:49.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:33:49 smithi119 ceph-mon[31518]: pgmap v3988: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:33:49.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:33:49 smithi132 ceph-mon[35941]: pgmap v3988: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:33:51.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:33:51 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7090.761973 secs 2024-03-28T02:33:51.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:33:51 smithi165 ceph-mon[36302]: pgmap v3989: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:33:51.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:33:51 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7090.761973 secs 2024-03-28T02:33:51.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:33:51 smithi119 ceph-mon[31518]: pgmap v3989: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:33:51.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:33:51 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7090.761973 secs 2024-03-28T02:33:51.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:33:51 smithi132 ceph-mon[35941]: pgmap v3989: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:33:53.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:33:53 smithi165 ceph-mon[36302]: pgmap v3990: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:33:53.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:33:53 smithi119 ceph-mon[31518]: pgmap v3990: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:33:53.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:33:53 smithi132 ceph-mon[35941]: pgmap v3990: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:33:55.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:33:55 smithi165 ceph-mon[36302]: pgmap v3991: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:33:55.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:33:55 smithi119 ceph-mon[31518]: pgmap v3991: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:33:55.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:33:55 smithi132 ceph-mon[35941]: pgmap v3991: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:33:56.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:33:56 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7095.762033 secs 2024-03-28T02:33:56.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:33:56 smithi165 ceph-mon[36302]: pgmap v3992: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T02:33:56.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:33:56 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7095.762033 secs 2024-03-28T02:33:56.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:33:56 smithi119 ceph-mon[31518]: pgmap v3992: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T02:33:56.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:33:56 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7095.762033 secs 2024-03-28T02:33:56.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:33:56 smithi132 ceph-mon[35941]: pgmap v3992: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T02:33:59.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:33:59 smithi132 ceph-mon[35941]: pgmap v3993: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:33:59.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:33:59 smithi165 ceph-mon[36302]: pgmap v3993: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:33:59.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:33:59 smithi119 ceph-mon[31518]: pgmap v3993: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:34:01.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:34:01 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7100.762095 secs 2024-03-28T02:34:01.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:34:01 smithi132 ceph-mon[35941]: pgmap v3994: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:34:01.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:34:01 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7100.762095 secs 2024-03-28T02:34:01.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:34:01 smithi165 ceph-mon[36302]: pgmap v3994: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:34:01.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:34:01 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7100.762095 secs 2024-03-28T02:34:01.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:34:01 smithi119 ceph-mon[31518]: pgmap v3994: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:34:03.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:34:03 smithi132 ceph-mon[35941]: pgmap v3995: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:34:03.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:34:03 smithi165 ceph-mon[36302]: pgmap v3995: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:34:03.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:34:03 smithi119 ceph-mon[31518]: pgmap v3995: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:34:05.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:34:05 smithi165 ceph-mon[36302]: pgmap v3996: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:34:05.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:34:05 smithi119 ceph-mon[31518]: pgmap v3996: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:34:05.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:34:05 smithi132 ceph-mon[35941]: pgmap v3996: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:34:06.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:34:06 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7105.762157 secs 2024-03-28T02:34:06.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:34:06 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7105.762157 secs 2024-03-28T02:34:06.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:34:06 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7105.762157 secs 2024-03-28T02:34:07.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:34:07 smithi165 ceph-mon[36302]: pgmap v3997: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:34:07.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:34:07 smithi119 ceph-mon[31518]: pgmap v3997: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:34:07.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:34:07 smithi132 ceph-mon[35941]: pgmap v3997: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:34:09.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:34:09 smithi165 ceph-mon[36302]: pgmap v3998: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:34:09.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:34:09 smithi119 ceph-mon[31518]: pgmap v3998: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:34:09.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:34:09 smithi132 ceph-mon[35941]: pgmap v3998: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:34:11.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:34:11 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7110.762219 secs 2024-03-28T02:34:11.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:34:11 smithi165 ceph-mon[36302]: pgmap v3999: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:34:11.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:34:11 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7110.762219 secs 2024-03-28T02:34:11.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:34:11 smithi119 ceph-mon[31518]: pgmap v3999: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:34:11.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:34:11 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7110.762219 secs 2024-03-28T02:34:11.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:34:11 smithi132 ceph-mon[35941]: pgmap v3999: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:34:13.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:34:13 smithi165 ceph-mon[36302]: pgmap v4000: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:34:13.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:34:13 smithi119 ceph-mon[31518]: pgmap v4000: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:34:13.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:34:13 smithi132 ceph-mon[35941]: pgmap v4000: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:34:14.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:34:14 smithi165 ceph-mon[36302]: pgmap v4001: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:34:14.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:34:14 smithi119 ceph-mon[31518]: pgmap v4001: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:34:14.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:34:14 smithi132 ceph-mon[35941]: pgmap v4001: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:34:16.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:34:15 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7115.762310 secs 2024-03-28T02:34:16.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:34:15 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7115.762310 secs 2024-03-28T02:34:16.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:34:15 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7115.762310 secs 2024-03-28T02:34:17.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:34:16 smithi165 ceph-mon[36302]: pgmap v4002: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:34:17.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:34:16 smithi119 ceph-mon[31518]: pgmap v4002: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:34:17.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:34:16 smithi132 ceph-mon[35941]: pgmap v4002: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:34:19.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:34:19 smithi132 ceph-mon[35941]: pgmap v4003: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:34:19.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:34:19 smithi165 ceph-mon[36302]: pgmap v4003: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:34:19.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:34:19 smithi119 ceph-mon[31518]: pgmap v4003: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:34:21.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:34:21 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7120.762372 secs 2024-03-28T02:34:21.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:34:21 smithi119 ceph-mon[31518]: pgmap v4004: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:34:21.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:34:21 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7120.762372 secs 2024-03-28T02:34:21.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:34:21 smithi165 ceph-mon[36302]: pgmap v4004: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:34:21.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:34:21 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7120.762372 secs 2024-03-28T02:34:21.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:34:21 smithi132 ceph-mon[35941]: pgmap v4004: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:34:23.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:34:23 smithi165 ceph-mon[36302]: pgmap v4005: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:34:23.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:34:23 smithi119 ceph-mon[31518]: pgmap v4005: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:34:23.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:34:23 smithi132 ceph-mon[35941]: pgmap v4005: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:34:25.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:34:25 smithi165 ceph-mon[36302]: pgmap v4006: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:34:25.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:34:25 smithi119 ceph-mon[31518]: pgmap v4006: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:34:25.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:34:25 smithi132 ceph-mon[35941]: pgmap v4006: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:34:26.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:34:26 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7125.762460 secs 2024-03-28T02:34:26.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:34:26 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7125.762460 secs 2024-03-28T02:34:26.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:34:26 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7125.762460 secs 2024-03-28T02:34:27.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:34:27 smithi165 ceph-mon[36302]: pgmap v4007: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:34:27.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:34:27 smithi119 ceph-mon[31518]: pgmap v4007: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:34:27.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:34:27 smithi132 ceph-mon[35941]: pgmap v4007: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:34:29.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:34:29 smithi165 ceph-mon[36302]: pgmap v4008: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:34:29.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:34:29 smithi119 ceph-mon[31518]: pgmap v4008: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:34:29.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:34:29 smithi132 ceph-mon[35941]: pgmap v4008: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:34:30.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:34:30 smithi165 ceph-mon[36302]: pgmap v4009: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:34:30.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:34:30 smithi119 ceph-mon[31518]: pgmap v4009: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:34:30.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:34:30 smithi132 ceph-mon[35941]: pgmap v4009: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:34:31.599 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:34:31 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7130.762549 secs 2024-03-28T02:34:31.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:34:31 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7130.762549 secs 2024-03-28T02:34:31.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:34:31 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7130.762549 secs 2024-03-28T02:34:32.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:34:32 smithi165 ceph-mon[36302]: pgmap v4010: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:34:32.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:34:32 smithi119 ceph-mon[31518]: pgmap v4010: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:34:32.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:34:32 smithi132 ceph-mon[35941]: pgmap v4010: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:34:35.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:34:35 smithi165 ceph-mon[36302]: pgmap v4011: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:34:35.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:34:35 smithi119 ceph-mon[31518]: pgmap v4011: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:34:35.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:34:35 smithi132 ceph-mon[35941]: pgmap v4011: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:34:36.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:34:36 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7135.762633 secs 2024-03-28T02:34:36.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:34:36 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7135.762633 secs 2024-03-28T02:34:36.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:34:36 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7135.762633 secs 2024-03-28T02:34:37.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:34:37 smithi165 ceph-mon[36302]: pgmap v4012: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:34:37.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:34:37 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:34:37.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:34:37 smithi119 ceph-mon[31518]: pgmap v4012: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:34:37.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:34:37 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:34:37.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:34:37 smithi132 ceph-mon[35941]: pgmap v4012: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:34:37.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:34:37 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:34:38.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:34:38 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:34:38.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:34:38 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:34:38.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:34:38 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:34:38.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:34:38 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:34:38.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:34:38 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:34:38.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:34:38 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:34:38.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:34:38 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:34:38.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:34:38 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:34:38.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:34:38 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:34:39.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:34:39 smithi165 ceph-mon[36302]: pgmap v4013: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T02:34:39.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:34:39 smithi165 ceph-mon[36302]: pgmap v4014: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:34:39.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:34:39 smithi119 ceph-mon[31518]: pgmap v4013: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T02:34:39.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:34:39 smithi119 ceph-mon[31518]: pgmap v4014: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:34:39.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:34:39 smithi132 ceph-mon[35941]: pgmap v4013: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T02:34:39.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:34:39 smithi132 ceph-mon[35941]: pgmap v4014: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:34:40.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:34:40 smithi165 ceph-mon[36302]: pgmap v4015: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:34:40.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:34:40 smithi119 ceph-mon[31518]: pgmap v4015: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:34:40.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:34:40 smithi132 ceph-mon[35941]: pgmap v4015: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:34:41.529 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:34:41 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7140.762695 secs 2024-03-28T02:34:41.552 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:34:41 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7140.762695 secs 2024-03-28T02:34:41.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:34:41 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7140.762695 secs 2024-03-28T02:34:42.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:34:42 smithi165 ceph-mon[36302]: pgmap v4016: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:34:42.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:34:42 smithi119 ceph-mon[31518]: pgmap v4016: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:34:42.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:34:42 smithi132 ceph-mon[35941]: pgmap v4016: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:34:45.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:34:44 smithi165 ceph-mon[36302]: pgmap v4017: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-28T02:34:45.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:34:44 smithi119 ceph-mon[31518]: pgmap v4017: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-28T02:34:45.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:34:44 smithi132 ceph-mon[35941]: pgmap v4017: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-28T02:34:46.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:34:45 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7145.762758 secs 2024-03-28T02:34:46.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:34:45 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7145.762758 secs 2024-03-28T02:34:46.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:34:45 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7145.762758 secs 2024-03-28T02:34:47.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:34:46 smithi165 ceph-mon[36302]: pgmap v4018: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:34:47.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:34:46 smithi119 ceph-mon[31518]: pgmap v4018: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:34:47.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:34:46 smithi132 ceph-mon[35941]: pgmap v4018: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:34:49.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:34:48 smithi165 ceph-mon[36302]: pgmap v4019: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T02:34:49.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:34:48 smithi119 ceph-mon[31518]: pgmap v4019: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T02:34:49.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:34:48 smithi132 ceph-mon[35941]: pgmap v4019: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T02:34:51.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:34:50 smithi165 ceph-mon[36302]: pgmap v4020: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:34:51.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:34:50 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7150.762845 secs 2024-03-28T02:34:51.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:34:50 smithi119 ceph-mon[31518]: pgmap v4020: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:34:51.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:34:50 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7150.762845 secs 2024-03-28T02:34:51.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:34:50 smithi132 ceph-mon[35941]: pgmap v4020: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:34:51.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:34:50 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7150.762845 secs 2024-03-28T02:34:53.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:34:52 smithi165 ceph-mon[36302]: pgmap v4021: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:34:53.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:34:52 smithi119 ceph-mon[31518]: pgmap v4021: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:34:53.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:34:52 smithi132 ceph-mon[35941]: pgmap v4021: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:34:55.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:34:54 smithi165 ceph-mon[36302]: pgmap v4022: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:34:55.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:34:54 smithi119 ceph-mon[31518]: pgmap v4022: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:34:55.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:34:54 smithi132 ceph-mon[35941]: pgmap v4022: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:34:56.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:34:55 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7155.762875 secs 2024-03-28T02:34:56.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:34:55 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7155.762875 secs 2024-03-28T02:34:56.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:34:55 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7155.762875 secs 2024-03-28T02:34:57.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:34:56 smithi165 ceph-mon[36302]: pgmap v4023: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:34:57.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:34:56 smithi119 ceph-mon[31518]: pgmap v4023: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:34:57.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:34:56 smithi132 ceph-mon[35941]: pgmap v4023: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:34:59.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:34:58 smithi165 ceph-mon[36302]: pgmap v4024: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:34:59.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:34:58 smithi119 ceph-mon[31518]: pgmap v4024: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:34:59.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:34:58 smithi132 ceph-mon[35941]: pgmap v4024: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:35:01.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:35:00 smithi165 ceph-mon[36302]: pgmap v4025: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:35:01.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:35:00 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7160.762939 secs 2024-03-28T02:35:01.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:35:00 smithi119 ceph-mon[31518]: pgmap v4025: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:35:01.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:35:00 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7160.762939 secs 2024-03-28T02:35:01.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:35:00 smithi132 ceph-mon[35941]: pgmap v4025: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:35:01.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:35:00 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7160.762939 secs 2024-03-28T02:35:03.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:35:02 smithi165 ceph-mon[36302]: pgmap v4026: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:35:03.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:35:02 smithi119 ceph-mon[31518]: pgmap v4026: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:35:03.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:35:02 smithi132 ceph-mon[35941]: pgmap v4026: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:35:05.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:35:05 smithi165 ceph-mon[36302]: pgmap v4027: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:35:05.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:35:05 smithi119 ceph-mon[31518]: pgmap v4027: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:35:05.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:35:05 smithi132 ceph-mon[35941]: pgmap v4027: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:35:06.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:35:06 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7165.762999 secs 2024-03-28T02:35:06.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:35:06 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7165.762999 secs 2024-03-28T02:35:06.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:35:06 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7165.762999 secs 2024-03-28T02:35:07.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:35:07 smithi165 ceph-mon[36302]: pgmap v4028: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:35:07.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:35:07 smithi119 ceph-mon[31518]: pgmap v4028: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:35:07.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:35:07 smithi132 ceph-mon[35941]: pgmap v4028: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:35:09.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:35:09 smithi165 ceph-mon[36302]: pgmap v4029: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:35:09.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:35:09 smithi119 ceph-mon[31518]: pgmap v4029: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:35:09.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:35:09 smithi132 ceph-mon[35941]: pgmap v4029: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:35:11.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:35:11 smithi165 ceph-mon[36302]: pgmap v4030: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:35:11.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:35:11 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7170.763063 secs 2024-03-28T02:35:11.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:35:11 smithi119 ceph-mon[31518]: pgmap v4030: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:35:11.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:35:11 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7170.763063 secs 2024-03-28T02:35:11.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:35:11 smithi132 ceph-mon[35941]: pgmap v4030: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:35:11.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:35:11 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7170.763063 secs 2024-03-28T02:35:13.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:35:13 smithi165 ceph-mon[36302]: pgmap v4031: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:35:13.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:35:13 smithi119 ceph-mon[31518]: pgmap v4031: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:35:13.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:35:13 smithi132 ceph-mon[35941]: pgmap v4031: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:35:15.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:35:15 smithi165 ceph-mon[36302]: pgmap v4032: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:35:15.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:35:15 smithi119 ceph-mon[31518]: pgmap v4032: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:35:15.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:35:15 smithi132 ceph-mon[35941]: pgmap v4032: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:35:16.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:35:16 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7175.763146 secs 2024-03-28T02:35:16.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:35:16 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7175.763146 secs 2024-03-28T02:35:16.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:35:16 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7175.763146 secs 2024-03-28T02:35:17.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:35:17 smithi119 ceph-mon[31518]: pgmap v4033: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:35:17.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:35:17 smithi132 ceph-mon[35941]: pgmap v4033: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:35:17.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:35:17 smithi165 ceph-mon[36302]: pgmap v4033: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:35:19.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:35:19 smithi119 ceph-mon[31518]: pgmap v4034: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:35:19.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:35:19 smithi132 ceph-mon[35941]: pgmap v4034: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:35:19.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:35:19 smithi165 ceph-mon[36302]: pgmap v4034: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:35:21.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:35:21 smithi119 ceph-mon[31518]: pgmap v4035: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:35:21.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:35:21 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7180.763209 secs 2024-03-28T02:35:21.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:35:21 smithi132 ceph-mon[35941]: pgmap v4035: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:35:21.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:35:21 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7180.763209 secs 2024-03-28T02:35:21.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:35:21 smithi165 ceph-mon[36302]: pgmap v4035: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:35:21.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:35:21 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7180.763209 secs 2024-03-28T02:35:23.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:35:23 smithi119 ceph-mon[31518]: pgmap v4036: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:35:23.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:35:23 smithi132 ceph-mon[35941]: pgmap v4036: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:35:23.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:35:23 smithi165 ceph-mon[36302]: pgmap v4036: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:35:25.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:35:25 smithi132 ceph-mon[35941]: pgmap v4037: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:35:25.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:35:25 smithi165 ceph-mon[36302]: pgmap v4037: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:35:25.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:35:25 smithi119 ceph-mon[31518]: pgmap v4037: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:35:26.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:35:26 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7185.763295 secs 2024-03-28T02:35:26.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:35:26 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7185.763295 secs 2024-03-28T02:35:26.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:35:26 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7185.763295 secs 2024-03-28T02:35:27.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:35:27 smithi132 ceph-mon[35941]: pgmap v4038: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:35:27.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:35:27 smithi165 ceph-mon[36302]: pgmap v4038: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:35:27.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:35:27 smithi119 ceph-mon[31518]: pgmap v4038: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:35:29.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:35:29 smithi132 ceph-mon[35941]: pgmap v4039: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T02:35:29.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:35:29 smithi165 ceph-mon[36302]: pgmap v4039: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T02:35:29.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:35:29 smithi119 ceph-mon[31518]: pgmap v4039: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T02:35:31.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:35:31 smithi132 ceph-mon[35941]: pgmap v4040: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:35:31.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:35:31 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7190.763355 secs 2024-03-28T02:35:31.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:35:31 smithi165 ceph-mon[36302]: pgmap v4040: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:35:31.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:35:31 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7190.763355 secs 2024-03-28T02:35:31.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:35:31 smithi119 ceph-mon[31518]: pgmap v4040: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:35:31.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:35:31 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7190.763355 secs 2024-03-28T02:35:33.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:35:33 smithi132 ceph-mon[35941]: pgmap v4041: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T02:35:33.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:35:33 smithi165 ceph-mon[36302]: pgmap v4041: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T02:35:33.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:35:33 smithi119 ceph-mon[31518]: pgmap v4041: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T02:35:35.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:35:35 smithi132 ceph-mon[35941]: pgmap v4042: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T02:35:35.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:35:35 smithi165 ceph-mon[36302]: pgmap v4042: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T02:35:35.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:35:35 smithi119 ceph-mon[31518]: pgmap v4042: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T02:35:36.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:35:36 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7195.763444 secs 2024-03-28T02:35:36.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:35:36 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7195.763444 secs 2024-03-28T02:35:36.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:35:36 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7195.763444 secs 2024-03-28T02:35:37.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:35:37 smithi165 ceph-mon[36302]: pgmap v4043: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:35:37.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:35:37 smithi119 ceph-mon[31518]: pgmap v4043: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:35:37.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:35:37 smithi132 ceph-mon[35941]: pgmap v4043: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:35:38.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:35:38 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:35:38.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:35:38 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:35:38.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:35:38 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:35:39.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:35:39 smithi165 ceph-mon[36302]: pgmap v4044: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:35:39.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:35:39 smithi119 ceph-mon[31518]: pgmap v4044: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:35:39.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:35:39 smithi132 ceph-mon[35941]: pgmap v4044: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:35:40.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:35:40 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:35:40.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:35:40 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:35:40.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:35:40 smithi165 ceph-mon[36302]: pgmap v4045: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T02:35:40.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:35:40 smithi165 ceph-mon[36302]: pgmap v4046: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:35:40.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:35:40 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:35:40.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:35:40 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:35:40.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:35:40 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:35:40.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:35:40 smithi119 ceph-mon[31518]: pgmap v4045: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T02:35:40.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:35:40 smithi119 ceph-mon[31518]: pgmap v4046: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:35:40.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:35:40 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:35:40.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:35:40 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:35:40.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:35:40 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:35:40.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:35:40 smithi132 ceph-mon[35941]: pgmap v4045: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T02:35:40.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:35:40 smithi132 ceph-mon[35941]: pgmap v4046: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:35:40.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:35:40 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:35:41.530 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:35:41 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7200.763533 secs 2024-03-28T02:35:41.553 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:35:41 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7200.763533 secs 2024-03-28T02:35:41.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:35:41 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7200.763533 secs 2024-03-28T02:35:42.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:35:42 smithi165 ceph-mon[36302]: pgmap v4047: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:35:42.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:35:42 smithi119 ceph-mon[31518]: pgmap v4047: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:35:42.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:35:42 smithi132 ceph-mon[35941]: pgmap v4047: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:35:44.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:35:44 smithi165 ceph-mon[36302]: pgmap v4048: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:35:44.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:35:44 smithi119 ceph-mon[31518]: pgmap v4048: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:35:44.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:35:44 smithi132 ceph-mon[35941]: pgmap v4048: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:35:45.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:35:45 smithi165 ceph-mon[36302]: pgmap v4049: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:35:45.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:35:45 smithi119 ceph-mon[31518]: pgmap v4049: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:35:45.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:35:45 smithi132 ceph-mon[35941]: pgmap v4049: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:35:46.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:35:46 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7205.763621 secs 2024-03-28T02:35:46.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:35:46 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7205.763621 secs 2024-03-28T02:35:46.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:35:46 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7205.763621 secs 2024-03-28T02:35:47.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:35:47 smithi165 ceph-mon[36302]: pgmap v4050: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:35:47.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:35:47 smithi119 ceph-mon[31518]: pgmap v4050: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:35:47.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:35:47 smithi132 ceph-mon[35941]: pgmap v4050: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:35:49.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:35:49 smithi165 ceph-mon[36302]: pgmap v4051: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T02:35:49.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:35:49 smithi119 ceph-mon[31518]: pgmap v4051: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T02:35:49.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:35:49 smithi132 ceph-mon[35941]: pgmap v4051: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T02:35:51.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:35:50 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7210.763711 secs 2024-03-28T02:35:51.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:35:50 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7210.763711 secs 2024-03-28T02:35:51.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:35:50 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7210.763711 secs 2024-03-28T02:35:52.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:35:51 smithi165 ceph-mon[36302]: pgmap v4052: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:35:52.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:35:51 smithi119 ceph-mon[31518]: pgmap v4052: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:35:52.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:35:51 smithi132 ceph-mon[35941]: pgmap v4052: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:35:53.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:35:53 smithi165 ceph-mon[36302]: pgmap v4053: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:35:53.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:35:53 smithi119 ceph-mon[31518]: pgmap v4053: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:35:53.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:35:53 smithi132 ceph-mon[35941]: pgmap v4053: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:35:55.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:35:55 smithi165 ceph-mon[36302]: pgmap v4054: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:35:55.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:35:55 smithi119 ceph-mon[31518]: pgmap v4054: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:35:55.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:35:55 smithi132 ceph-mon[35941]: pgmap v4054: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:35:56.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:35:56 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7215.763798 secs 2024-03-28T02:35:56.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:35:56 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7215.763798 secs 2024-03-28T02:35:56.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:35:56 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7215.763798 secs 2024-03-28T02:35:57.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:35:57 smithi165 ceph-mon[36302]: pgmap v4055: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:35:57.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:35:57 smithi119 ceph-mon[31518]: pgmap v4055: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:35:57.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:35:57 smithi132 ceph-mon[35941]: pgmap v4055: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:35:59.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:35:59 smithi165 ceph-mon[36302]: pgmap v4056: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:35:59.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:35:59 smithi119 ceph-mon[31518]: pgmap v4056: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:35:59.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:35:59 smithi132 ceph-mon[35941]: pgmap v4056: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:36:01.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:36:00 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7220.763882 secs 2024-03-28T02:36:01.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:36:00 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7220.763882 secs 2024-03-28T02:36:01.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:36:00 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7220.763882 secs 2024-03-28T02:36:02.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:36:01 smithi165 ceph-mon[36302]: pgmap v4057: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:36:02.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:36:01 smithi119 ceph-mon[31518]: pgmap v4057: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:36:02.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:36:01 smithi132 ceph-mon[35941]: pgmap v4057: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:36:03.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:36:03 smithi165 ceph-mon[36302]: pgmap v4058: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:36:03.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:36:03 smithi119 ceph-mon[31518]: pgmap v4058: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:36:03.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:36:03 smithi132 ceph-mon[35941]: pgmap v4058: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:36:05.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:36:05 smithi165 ceph-mon[36302]: pgmap v4059: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:36:05.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:36:05 smithi119 ceph-mon[31518]: pgmap v4059: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:36:05.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:36:05 smithi132 ceph-mon[35941]: pgmap v4059: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:36:06.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:36:06 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7225.763949 secs 2024-03-28T02:36:06.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:36:06 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7225.763949 secs 2024-03-28T02:36:06.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:36:06 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7225.763949 secs 2024-03-28T02:36:07.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:36:07 smithi165 ceph-mon[36302]: pgmap v4060: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:36:07.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:36:07 smithi119 ceph-mon[31518]: pgmap v4060: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:36:07.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:36:07 smithi132 ceph-mon[35941]: pgmap v4060: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:36:09.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:36:09 smithi165 ceph-mon[36302]: pgmap v4061: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:36:09.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:36:09 smithi119 ceph-mon[31518]: pgmap v4061: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:36:09.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:36:09 smithi132 ceph-mon[35941]: pgmap v4061: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:36:11.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:36:10 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7230.764010 secs 2024-03-28T02:36:11.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:36:10 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7230.764010 secs 2024-03-28T02:36:11.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:36:10 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7230.764010 secs 2024-03-28T02:36:12.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:36:11 smithi165 ceph-mon[36302]: pgmap v4062: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:36:12.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:36:11 smithi119 ceph-mon[31518]: pgmap v4062: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:36:12.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:36:11 smithi132 ceph-mon[35941]: pgmap v4062: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:36:13.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:36:13 smithi165 ceph-mon[36302]: pgmap v4063: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:36:13.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:36:13 smithi119 ceph-mon[31518]: pgmap v4063: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:36:13.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:36:13 smithi132 ceph-mon[35941]: pgmap v4063: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:36:15.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:36:15 smithi165 ceph-mon[36302]: pgmap v4064: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:36:15.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:36:15 smithi119 ceph-mon[31518]: pgmap v4064: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:36:15.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:36:15 smithi132 ceph-mon[35941]: pgmap v4064: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:36:16.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:36:16 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7235.764047 secs 2024-03-28T02:36:16.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:36:16 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7235.764047 secs 2024-03-28T02:36:16.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:36:16 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7235.764047 secs 2024-03-28T02:36:17.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:36:17 smithi165 ceph-mon[36302]: pgmap v4065: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:36:17.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:36:17 smithi119 ceph-mon[31518]: pgmap v4065: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:36:17.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:36:17 smithi132 ceph-mon[35941]: pgmap v4065: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:36:19.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:36:19 smithi165 ceph-mon[36302]: pgmap v4066: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:36:19.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:36:19 smithi119 ceph-mon[31518]: pgmap v4066: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:36:19.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:36:19 smithi132 ceph-mon[35941]: pgmap v4066: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:36:21.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:36:20 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7240.764086 secs 2024-03-28T02:36:21.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:36:20 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7240.764086 secs 2024-03-28T02:36:21.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:36:20 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7240.764086 secs 2024-03-28T02:36:22.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:36:21 smithi165 ceph-mon[36302]: pgmap v4067: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:36:22.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:36:21 smithi119 ceph-mon[31518]: pgmap v4067: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:36:22.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:36:21 smithi132 ceph-mon[35941]: pgmap v4067: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:36:23.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:36:23 smithi165 ceph-mon[36302]: pgmap v4068: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:36:23.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:36:23 smithi119 ceph-mon[31518]: pgmap v4068: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:36:23.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:36:23 smithi132 ceph-mon[35941]: pgmap v4068: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:36:25.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:36:25 smithi165 ceph-mon[36302]: pgmap v4069: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:36:25.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:36:25 smithi119 ceph-mon[31518]: pgmap v4069: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:36:25.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:36:25 smithi132 ceph-mon[35941]: pgmap v4069: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:36:26.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:36:26 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7245.764178 secs 2024-03-28T02:36:26.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:36:26 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7245.764178 secs 2024-03-28T02:36:26.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:36:26 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7245.764178 secs 2024-03-28T02:36:27.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:36:27 smithi165 ceph-mon[36302]: pgmap v4070: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:36:27.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:36:27 smithi119 ceph-mon[31518]: pgmap v4070: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:36:27.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:36:27 smithi132 ceph-mon[35941]: pgmap v4070: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:36:29.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:36:29 smithi165 ceph-mon[36302]: pgmap v4071: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:36:29.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:36:29 smithi119 ceph-mon[31518]: pgmap v4071: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:36:29.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:36:29 smithi132 ceph-mon[35941]: pgmap v4071: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:36:31.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:36:30 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7250.764250 secs 2024-03-28T02:36:31.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:36:30 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7250.764250 secs 2024-03-28T02:36:31.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:36:30 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7250.764250 secs 2024-03-28T02:36:33.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:36:32 smithi165 ceph-mon[36302]: pgmap v4072: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:36:33.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:36:32 smithi119 ceph-mon[31518]: pgmap v4072: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:36:33.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:36:32 smithi132 ceph-mon[35941]: pgmap v4072: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:36:35.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:36:34 smithi165 ceph-mon[36302]: pgmap v4073: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:36:35.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:36:34 smithi119 ceph-mon[31518]: pgmap v4073: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:36:35.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:36:34 smithi132 ceph-mon[35941]: pgmap v4073: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:36:36.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:36:35 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7255.764265 secs 2024-03-28T02:36:36.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:36:35 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7255.764265 secs 2024-03-28T02:36:36.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:36:35 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7255.764265 secs 2024-03-28T02:36:37.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:36:36 smithi165 ceph-mon[36302]: pgmap v4074: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:36:37.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:36:36 smithi119 ceph-mon[31518]: pgmap v4074: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:36:37.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:36:36 smithi132 ceph-mon[35941]: pgmap v4074: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:36:39.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:36:38 smithi165 ceph-mon[36302]: pgmap v4075: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:36:39.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:36:38 smithi119 ceph-mon[31518]: pgmap v4075: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:36:39.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:36:38 smithi132 ceph-mon[35941]: pgmap v4075: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:36:40.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:36:39 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:36:40.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:36:39 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:36:40.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:36:39 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:36:41.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:36:40 smithi165 ceph-mon[36302]: pgmap v4076: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:36:41.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:36:40 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7260.764443 secs 2024-03-28T02:36:41.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:36:40 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:36:41.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:36:40 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:36:41.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:36:40 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:36:41.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:36:40 smithi119 ceph-mon[31518]: pgmap v4076: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:36:41.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:36:40 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7260.764443 secs 2024-03-28T02:36:41.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:36:40 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:36:41.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:36:40 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:36:41.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:36:40 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:36:41.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:36:40 smithi132 ceph-mon[35941]: pgmap v4076: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:36:41.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:36:40 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7260.764443 secs 2024-03-28T02:36:41.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:36:40 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:36:41.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:36:40 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:36:41.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:36:40 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:36:42.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:36:41 smithi165 ceph-mon[36302]: pgmap v4077: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T02:36:42.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:36:41 smithi165 ceph-mon[36302]: pgmap v4078: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:36:42.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:36:41 smithi119 ceph-mon[31518]: pgmap v4077: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T02:36:42.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:36:41 smithi119 ceph-mon[31518]: pgmap v4078: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:36:42.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:36:41 smithi132 ceph-mon[35941]: pgmap v4077: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T02:36:42.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:36:41 smithi132 ceph-mon[35941]: pgmap v4078: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:36:44.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:36:43 smithi165 ceph-mon[36302]: pgmap v4079: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:36:44.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:36:43 smithi119 ceph-mon[31518]: pgmap v4079: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:36:44.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:36:43 smithi132 ceph-mon[35941]: pgmap v4079: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:36:46.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:36:45 smithi165 ceph-mon[36302]: pgmap v4080: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:36:46.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:36:45 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7265.764504 secs 2024-03-28T02:36:46.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:36:45 smithi119 ceph-mon[31518]: pgmap v4080: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:36:46.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:36:45 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7265.764504 secs 2024-03-28T02:36:46.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:36:45 smithi132 ceph-mon[35941]: pgmap v4080: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:36:46.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:36:45 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7265.764504 secs 2024-03-28T02:36:48.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:36:47 smithi165 ceph-mon[36302]: pgmap v4081: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:36:48.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:36:47 smithi119 ceph-mon[31518]: pgmap v4081: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:36:48.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:36:47 smithi132 ceph-mon[35941]: pgmap v4081: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:36:50.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:36:49 smithi165 ceph-mon[36302]: pgmap v4082: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-28T02:36:50.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:36:49 smithi119 ceph-mon[31518]: pgmap v4082: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-28T02:36:50.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:36:49 smithi132 ceph-mon[35941]: pgmap v4082: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-28T02:36:51.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:36:50 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7270.764593 secs 2024-03-28T02:36:51.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:36:50 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7270.764593 secs 2024-03-28T02:36:51.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:36:50 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7270.764593 secs 2024-03-28T02:36:52.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:36:51 smithi165 ceph-mon[36302]: pgmap v4083: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T02:36:52.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:36:51 smithi119 ceph-mon[31518]: pgmap v4083: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T02:36:52.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:36:51 smithi132 ceph-mon[35941]: pgmap v4083: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T02:36:54.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:36:53 smithi165 ceph-mon[36302]: pgmap v4084: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:36:54.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:36:53 smithi119 ceph-mon[31518]: pgmap v4084: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:36:54.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:36:53 smithi132 ceph-mon[35941]: pgmap v4084: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:36:56.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:36:55 smithi165 ceph-mon[36302]: pgmap v4085: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:36:56.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:36:55 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7275.764682 secs 2024-03-28T02:36:56.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:36:55 smithi119 ceph-mon[31518]: pgmap v4085: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:36:56.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:36:55 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7275.764682 secs 2024-03-28T02:36:56.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:36:55 smithi132 ceph-mon[35941]: pgmap v4085: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:36:56.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:36:55 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7275.764682 secs 2024-03-28T02:36:58.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:36:57 smithi165 ceph-mon[36302]: pgmap v4086: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:36:58.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:36:57 smithi119 ceph-mon[31518]: pgmap v4086: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:36:58.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:36:57 smithi132 ceph-mon[35941]: pgmap v4086: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:37:00.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:36:59 smithi165 ceph-mon[36302]: pgmap v4087: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:37:00.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:36:59 smithi119 ceph-mon[31518]: pgmap v4087: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:37:00.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:36:59 smithi132 ceph-mon[35941]: pgmap v4087: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:37:01.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:37:00 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7280.764768 secs 2024-03-28T02:37:01.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:37:00 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7280.764768 secs 2024-03-28T02:37:01.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:37:00 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7280.764768 secs 2024-03-28T02:37:02.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:37:01 smithi119 ceph-mon[31518]: pgmap v4088: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:37:02.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:37:01 smithi132 ceph-mon[35941]: pgmap v4088: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:37:02.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:37:01 smithi165 ceph-mon[36302]: pgmap v4088: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:37:04.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:37:03 smithi119 ceph-mon[31518]: pgmap v4089: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:37:04.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:37:03 smithi132 ceph-mon[35941]: pgmap v4089: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:37:04.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:37:03 smithi165 ceph-mon[36302]: pgmap v4089: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:37:06.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:37:05 smithi119 ceph-mon[31518]: pgmap v4090: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:37:06.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:37:05 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7285.764829 secs 2024-03-28T02:37:06.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:37:05 smithi132 ceph-mon[35941]: pgmap v4090: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:37:06.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:37:05 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7285.764829 secs 2024-03-28T02:37:06.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:37:05 smithi165 ceph-mon[36302]: pgmap v4090: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:37:06.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:37:05 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7285.764829 secs 2024-03-28T02:37:08.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:37:07 smithi119 ceph-mon[31518]: pgmap v4091: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:37:08.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:37:07 smithi132 ceph-mon[35941]: pgmap v4091: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:37:08.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:37:07 smithi165 ceph-mon[36302]: pgmap v4091: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:37:10.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:37:09 smithi119 ceph-mon[31518]: pgmap v4092: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:37:10.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:37:09 smithi132 ceph-mon[35941]: pgmap v4092: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:37:10.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:37:09 smithi165 ceph-mon[36302]: pgmap v4092: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:37:11.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:37:10 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7290.764917 secs 2024-03-28T02:37:11.280 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:37:10 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7290.764917 secs 2024-03-28T02:37:11.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:37:10 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7290.764917 secs 2024-03-28T02:37:12.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:37:11 smithi132 ceph-mon[35941]: pgmap v4093: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:37:12.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:37:11 smithi165 ceph-mon[36302]: pgmap v4093: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:37:12.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:37:11 smithi119 ceph-mon[31518]: pgmap v4093: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:37:14.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:37:13 smithi132 ceph-mon[35941]: pgmap v4094: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:37:14.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:37:13 smithi165 ceph-mon[36302]: pgmap v4094: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:37:14.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:37:13 smithi119 ceph-mon[31518]: pgmap v4094: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:37:16.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:37:16 smithi132 ceph-mon[35941]: pgmap v4095: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:37:16.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:37:16 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7295.765002 secs 2024-03-28T02:37:16.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:37:16 smithi165 ceph-mon[36302]: pgmap v4095: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:37:16.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:37:16 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7295.765002 secs 2024-03-28T02:37:16.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:37:16 smithi119 ceph-mon[31518]: pgmap v4095: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:37:16.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:37:16 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7295.765002 secs 2024-03-28T02:37:18.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:37:18 smithi132 ceph-mon[35941]: pgmap v4096: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:37:18.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:37:18 smithi165 ceph-mon[36302]: pgmap v4096: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:37:18.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:37:18 smithi119 ceph-mon[31518]: pgmap v4096: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:37:20.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:37:20 smithi132 ceph-mon[35941]: pgmap v4097: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:37:20.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:37:20 smithi165 ceph-mon[36302]: pgmap v4097: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:37:20.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:37:20 smithi119 ceph-mon[31518]: pgmap v4097: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:37:21.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:37:21 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7300.765088 secs 2024-03-28T02:37:21.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:37:21 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7300.765088 secs 2024-03-28T02:37:21.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:37:21 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7300.765088 secs 2024-03-28T02:37:22.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:37:22 smithi165 ceph-mon[36302]: pgmap v4098: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:37:22.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:37:22 smithi119 ceph-mon[31518]: pgmap v4098: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:37:22.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:37:22 smithi132 ceph-mon[35941]: pgmap v4098: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:37:24.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:37:24 smithi165 ceph-mon[36302]: pgmap v4099: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T02:37:24.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:37:24 smithi119 ceph-mon[31518]: pgmap v4099: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T02:37:24.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:37:24 smithi132 ceph-mon[35941]: pgmap v4099: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T02:37:26.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:37:26 smithi165 ceph-mon[36302]: pgmap v4100: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:37:26.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:37:26 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7305.765150 secs 2024-03-28T02:37:26.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:37:26 smithi119 ceph-mon[31518]: pgmap v4100: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:37:26.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:37:26 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7305.765150 secs 2024-03-28T02:37:26.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:37:26 smithi132 ceph-mon[35941]: pgmap v4100: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:37:26.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:37:26 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7305.765150 secs 2024-03-28T02:37:28.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:37:28 smithi119 ceph-mon[31518]: pgmap v4101: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T02:37:28.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:37:28 smithi165 ceph-mon[36302]: pgmap v4101: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T02:37:28.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:37:28 smithi132 ceph-mon[35941]: pgmap v4101: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T02:37:30.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:37:30 smithi165 ceph-mon[36302]: pgmap v4102: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T02:37:30.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:37:30 smithi119 ceph-mon[31518]: pgmap v4102: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T02:37:30.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:37:30 smithi132 ceph-mon[35941]: pgmap v4102: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T02:37:31.600 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:37:31 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7310.765252 secs 2024-03-28T02:37:31.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:37:31 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7310.765252 secs 2024-03-28T02:37:31.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:37:31 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7310.765252 secs 2024-03-28T02:37:32.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:37:32 smithi165 ceph-mon[36302]: pgmap v4103: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:37:32.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:37:32 smithi119 ceph-mon[31518]: pgmap v4103: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:37:32.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:37:32 smithi132 ceph-mon[35941]: pgmap v4103: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:37:34.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:37:34 smithi165 ceph-mon[36302]: pgmap v4104: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:37:34.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:37:34 smithi119 ceph-mon[31518]: pgmap v4104: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:37:34.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:37:34 smithi132 ceph-mon[35941]: pgmap v4104: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:37:36.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:37:36 smithi165 ceph-mon[36302]: pgmap v4105: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:37:36.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:37:36 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7315.765295 secs 2024-03-28T02:37:36.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:37:36 smithi119 ceph-mon[31518]: pgmap v4105: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:37:36.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:37:36 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7315.765295 secs 2024-03-28T02:37:36.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:37:36 smithi132 ceph-mon[35941]: pgmap v4105: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:37:36.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:37:36 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7315.765295 secs 2024-03-28T02:37:38.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:37:38 smithi165 ceph-mon[36302]: pgmap v4106: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:37:38.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:37:38 smithi119 ceph-mon[31518]: pgmap v4106: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:37:38.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:37:38 smithi132 ceph-mon[35941]: pgmap v4106: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:37:40.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:37:40 smithi165 ceph-mon[36302]: pgmap v4107: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:37:40.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:37:40 smithi119 ceph-mon[31518]: pgmap v4107: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:37:40.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:37:40 smithi132 ceph-mon[35941]: pgmap v4107: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:37:41.530 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:37:41 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7320.765385 secs 2024-03-28T02:37:41.530 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:37:41 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:37:41.554 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:37:41 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7320.765385 secs 2024-03-28T02:37:41.554 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:37:41 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:37:41.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:37:41 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7320.765385 secs 2024-03-28T02:37:41.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:37:41 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:37:42.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:37:42 smithi165 ceph-mon[36302]: pgmap v4108: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:37:42.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:37:42 smithi119 ceph-mon[31518]: pgmap v4108: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:37:42.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:37:42 smithi132 ceph-mon[35941]: pgmap v4108: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:37:43.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:37:43 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:37:43.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:37:43 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:37:43.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:37:43 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:37:43.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:37:43 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:37:43.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:37:43 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:37:43.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:37:43 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:37:43.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:37:43 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:37:43.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:37:43 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:37:43.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:37:43 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:37:44.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:37:44 smithi165 ceph-mon[36302]: pgmap v4109: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T02:37:44.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:37:44 smithi165 ceph-mon[36302]: pgmap v4110: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:37:44.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:37:44 smithi119 ceph-mon[31518]: pgmap v4109: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T02:37:44.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:37:44 smithi119 ceph-mon[31518]: pgmap v4110: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:37:44.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:37:44 smithi132 ceph-mon[35941]: pgmap v4109: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T02:37:44.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:37:44 smithi132 ceph-mon[35941]: pgmap v4110: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:37:45.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:37:45 smithi165 ceph-mon[36302]: pgmap v4111: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:37:45.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:37:45 smithi119 ceph-mon[31518]: pgmap v4111: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:37:45.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:37:45 smithi132 ceph-mon[35941]: pgmap v4111: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:37:46.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:37:46 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7325.765425 secs 2024-03-28T02:37:46.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:37:46 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7325.765425 secs 2024-03-28T02:37:46.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:37:46 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7325.765425 secs 2024-03-28T02:37:47.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:37:47 smithi165 ceph-mon[36302]: pgmap v4112: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:37:47.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:37:47 smithi119 ceph-mon[31518]: pgmap v4112: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:37:47.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:37:47 smithi132 ceph-mon[35941]: pgmap v4112: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:37:49.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:37:49 smithi165 ceph-mon[36302]: pgmap v4113: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:37:49.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:37:49 smithi119 ceph-mon[31518]: pgmap v4113: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:37:49.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:37:49 smithi132 ceph-mon[35941]: pgmap v4113: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:37:51.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:37:51 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7330.765514 secs 2024-03-28T02:37:51.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:37:51 smithi165 ceph-mon[36302]: pgmap v4114: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:37:51.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:37:51 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7330.765514 secs 2024-03-28T02:37:51.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:37:51 smithi119 ceph-mon[31518]: pgmap v4114: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:37:51.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:37:51 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7330.765514 secs 2024-03-28T02:37:51.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:37:51 smithi132 ceph-mon[35941]: pgmap v4114: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:37:53.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:37:53 smithi165 ceph-mon[36302]: pgmap v4115: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T02:37:53.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:37:53 smithi119 ceph-mon[31518]: pgmap v4115: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T02:37:53.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:37:53 smithi132 ceph-mon[35941]: pgmap v4115: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T02:37:55.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:37:55 smithi165 ceph-mon[36302]: pgmap v4116: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:37:55.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:37:55 smithi119 ceph-mon[31518]: pgmap v4116: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:37:55.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:37:55 smithi132 ceph-mon[35941]: pgmap v4116: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:37:56.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:37:56 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7335.765603 secs 2024-03-28T02:37:56.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:37:56 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7335.765603 secs 2024-03-28T02:37:56.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:37:56 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7335.765603 secs 2024-03-28T02:37:57.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:37:57 smithi165 ceph-mon[36302]: pgmap v4117: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:37:57.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:37:57 smithi119 ceph-mon[31518]: pgmap v4117: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:37:57.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:37:57 smithi132 ceph-mon[35941]: pgmap v4117: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:37:59.455 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:37:59 smithi119 ceph-mon[31518]: pgmap v4118: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:37:59.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:37:59 smithi165 ceph-mon[36302]: pgmap v4118: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:37:59.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:37:59 smithi132 ceph-mon[35941]: pgmap v4118: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:38:01.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:38:01 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7340.765693 secs 2024-03-28T02:38:01.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:38:01 smithi165 ceph-mon[36302]: pgmap v4119: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:38:01.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:38:01 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7340.765693 secs 2024-03-28T02:38:01.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:38:01 smithi119 ceph-mon[31518]: pgmap v4119: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:38:01.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:38:01 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7340.765693 secs 2024-03-28T02:38:01.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:38:01 smithi132 ceph-mon[35941]: pgmap v4119: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:38:03.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:38:03 smithi165 ceph-mon[36302]: pgmap v4120: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:38:03.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:38:03 smithi119 ceph-mon[31518]: pgmap v4120: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:38:03.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:38:03 smithi132 ceph-mon[35941]: pgmap v4120: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:38:06.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:38:06 smithi165 ceph-mon[36302]: pgmap v4121: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:38:06.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:38:06 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7345.765781 secs 2024-03-28T02:38:06.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:38:06 smithi119 ceph-mon[31518]: pgmap v4121: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:38:06.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:38:06 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7345.765781 secs 2024-03-28T02:38:06.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:38:06 smithi132 ceph-mon[35941]: pgmap v4121: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:38:06.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:38:06 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7345.765781 secs 2024-03-28T02:38:08.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:38:08 smithi165 ceph-mon[36302]: pgmap v4122: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:38:08.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:38:08 smithi119 ceph-mon[31518]: pgmap v4122: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:38:08.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:38:08 smithi132 ceph-mon[35941]: pgmap v4122: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:38:10.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:38:10 smithi165 ceph-mon[36302]: pgmap v4123: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:38:10.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:38:10 smithi119 ceph-mon[31518]: pgmap v4123: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:38:10.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:38:10 smithi132 ceph-mon[35941]: pgmap v4123: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:38:11.530 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:38:11 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7350.765842 secs 2024-03-28T02:38:11.554 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:38:11 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7350.765842 secs 2024-03-28T02:38:11.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:38:11 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7350.765842 secs 2024-03-28T02:38:12.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:38:12 smithi165 ceph-mon[36302]: pgmap v4124: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:38:12.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:38:12 smithi119 ceph-mon[31518]: pgmap v4124: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:38:12.701 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:38:12 smithi132 ceph-mon[35941]: pgmap v4124: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:38:14.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:38:14 smithi165 ceph-mon[36302]: pgmap v4125: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:38:14.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:38:14 smithi119 ceph-mon[31518]: pgmap v4125: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:38:14.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:38:14 smithi132 ceph-mon[35941]: pgmap v4125: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:38:16.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:38:16 smithi165 ceph-mon[36302]: pgmap v4126: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:38:16.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:38:16 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7355.765902 secs 2024-03-28T02:38:16.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:38:16 smithi119 ceph-mon[31518]: pgmap v4126: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:38:16.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:38:16 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7355.765902 secs 2024-03-28T02:38:16.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:38:16 smithi132 ceph-mon[35941]: pgmap v4126: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:38:16.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:38:16 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7355.765902 secs 2024-03-28T02:38:18.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:38:18 smithi165 ceph-mon[36302]: pgmap v4127: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:38:18.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:38:18 smithi119 ceph-mon[31518]: pgmap v4127: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:38:18.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:38:18 smithi132 ceph-mon[35941]: pgmap v4127: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:38:20.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:38:20 smithi165 ceph-mon[36302]: pgmap v4128: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:38:20.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:38:20 smithi119 ceph-mon[31518]: pgmap v4128: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:38:20.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:38:20 smithi132 ceph-mon[35941]: pgmap v4128: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:38:21.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:38:21 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7360.765994 secs 2024-03-28T02:38:21.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:38:21 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7360.765994 secs 2024-03-28T02:38:21.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:38:21 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7360.765994 secs 2024-03-28T02:38:22.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:38:22 smithi165 ceph-mon[36302]: pgmap v4129: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:38:22.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:38:22 smithi119 ceph-mon[31518]: pgmap v4129: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:38:22.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:38:22 smithi132 ceph-mon[35941]: pgmap v4129: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:38:24.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:38:24 smithi165 ceph-mon[36302]: pgmap v4130: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:38:24.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:38:24 smithi119 ceph-mon[31518]: pgmap v4130: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:38:24.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:38:24 smithi132 ceph-mon[35941]: pgmap v4130: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:38:26.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:38:26 smithi165 ceph-mon[36302]: pgmap v4131: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:38:26.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:38:26 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7365.766059 secs 2024-03-28T02:38:26.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:38:26 smithi119 ceph-mon[31518]: pgmap v4131: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:38:26.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:38:26 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7365.766059 secs 2024-03-28T02:38:26.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:38:26 smithi132 ceph-mon[35941]: pgmap v4131: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:38:26.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:38:26 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7365.766059 secs 2024-03-28T02:38:27.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:38:27 smithi165 ceph-mon[36302]: pgmap v4132: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:38:27.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:38:27 smithi119 ceph-mon[31518]: pgmap v4132: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:38:27.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:38:27 smithi132 ceph-mon[35941]: pgmap v4132: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:38:29.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:38:29 smithi165 ceph-mon[36302]: pgmap v4133: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:38:29.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:38:29 smithi119 ceph-mon[31518]: pgmap v4133: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:38:29.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:38:29 smithi132 ceph-mon[35941]: pgmap v4133: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:38:31.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:38:31 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7370.766123 secs 2024-03-28T02:38:31.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:38:31 smithi165 ceph-mon[36302]: pgmap v4134: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:38:31.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:38:31 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7370.766123 secs 2024-03-28T02:38:31.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:38:31 smithi119 ceph-mon[31518]: pgmap v4134: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:38:31.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:38:31 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7370.766123 secs 2024-03-28T02:38:31.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:38:31 smithi132 ceph-mon[35941]: pgmap v4134: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:38:33.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:38:33 smithi165 ceph-mon[36302]: pgmap v4135: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:38:33.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:38:33 smithi119 ceph-mon[31518]: pgmap v4135: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:38:33.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:38:33 smithi132 ceph-mon[35941]: pgmap v4135: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:38:35.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:38:35 smithi165 ceph-mon[36302]: pgmap v4136: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:38:35.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:38:35 smithi119 ceph-mon[31518]: pgmap v4136: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:38:35.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:38:35 smithi132 ceph-mon[35941]: pgmap v4136: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:38:36.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:38:36 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7375.766185 secs 2024-03-28T02:38:36.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:38:36 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7375.766185 secs 2024-03-28T02:38:36.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:38:36 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7375.766185 secs 2024-03-28T02:38:37.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:38:37 smithi119 ceph-mon[31518]: pgmap v4137: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:38:37.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:38:37 smithi132 ceph-mon[35941]: pgmap v4137: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:38:37.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:38:37 smithi165 ceph-mon[36302]: pgmap v4137: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:38:39.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:38:39 smithi119 ceph-mon[31518]: pgmap v4138: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:38:39.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:38:39 smithi132 ceph-mon[35941]: pgmap v4138: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:38:39.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:38:39 smithi165 ceph-mon[36302]: pgmap v4138: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:38:41.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:38:41 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7380.766245 secs 2024-03-28T02:38:41.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:38:41 smithi119 ceph-mon[31518]: pgmap v4139: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:38:41.680 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:38:41 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7380.766245 secs 2024-03-28T02:38:41.680 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:38:41 smithi165 ceph-mon[36302]: pgmap v4139: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:38:41.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:38:41 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7380.766245 secs 2024-03-28T02:38:41.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:38:41 smithi132 ceph-mon[35941]: pgmap v4139: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:38:43.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:38:43 smithi132 ceph-mon[35941]: pgmap v4140: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:38:43.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:38:43 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:38:43.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:38:43 smithi165 ceph-mon[36302]: pgmap v4140: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:38:43.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:38:43 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:38:43.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:38:43 smithi119 ceph-mon[31518]: pgmap v4140: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:38:43.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:38:43 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:38:44.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:38:44 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:38:44.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:38:44 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:38:44.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:38:44 smithi132 ceph-mon[35941]: pgmap v4141: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T02:38:44.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:38:44 smithi132 ceph-mon[35941]: pgmap v4142: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-28T02:38:44.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:38:44 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:38:44.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:38:44 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:38:44.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:38:44 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:38:44.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:38:44 smithi165 ceph-mon[36302]: pgmap v4141: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T02:38:44.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:38:44 smithi165 ceph-mon[36302]: pgmap v4142: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-28T02:38:44.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:38:44 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:38:44.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:38:44 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:38:44.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:38:44 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:38:44.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:38:44 smithi119 ceph-mon[31518]: pgmap v4141: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T02:38:44.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:38:44 smithi119 ceph-mon[31518]: pgmap v4142: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-28T02:38:44.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:38:44 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:38:46.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:38:45 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7385.766357 secs 2024-03-28T02:38:46.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:38:45 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7385.766357 secs 2024-03-28T02:38:46.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:38:45 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7385.766357 secs 2024-03-28T02:38:47.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:38:46 smithi165 ceph-mon[36302]: pgmap v4143: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:38:47.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:38:46 smithi119 ceph-mon[31518]: pgmap v4143: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:38:47.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:38:46 smithi132 ceph-mon[35941]: pgmap v4143: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:38:49.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:38:48 smithi132 ceph-mon[35941]: pgmap v4144: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:38:49.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:38:48 smithi165 ceph-mon[36302]: pgmap v4144: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:38:49.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:38:48 smithi119 ceph-mon[31518]: pgmap v4144: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:38:51.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:38:50 smithi132 ceph-mon[35941]: pgmap v4145: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:38:51.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:38:50 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7390.766419 secs 2024-03-28T02:38:51.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:38:50 smithi165 ceph-mon[36302]: pgmap v4145: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:38:51.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:38:50 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7390.766419 secs 2024-03-28T02:38:51.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:38:50 smithi119 ceph-mon[31518]: pgmap v4145: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:38:51.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:38:50 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7390.766419 secs 2024-03-28T02:38:53.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:38:52 smithi132 ceph-mon[35941]: pgmap v4146: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:38:53.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:38:52 smithi165 ceph-mon[36302]: pgmap v4146: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:38:53.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:38:52 smithi119 ceph-mon[31518]: pgmap v4146: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:38:55.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:38:54 smithi132 ceph-mon[35941]: pgmap v4147: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T02:38:55.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:38:54 smithi165 ceph-mon[36302]: pgmap v4147: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T02:38:55.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:38:54 smithi119 ceph-mon[31518]: pgmap v4147: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T02:38:56.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:38:55 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7395.766504 secs 2024-03-28T02:38:56.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:38:55 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7395.766504 secs 2024-03-28T02:38:56.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:38:55 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7395.766504 secs 2024-03-28T02:38:57.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:38:56 smithi132 ceph-mon[35941]: pgmap v4148: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:38:57.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:38:56 smithi165 ceph-mon[36302]: pgmap v4148: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:38:57.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:38:56 smithi119 ceph-mon[31518]: pgmap v4148: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:38:59.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:38:58 smithi165 ceph-mon[36302]: pgmap v4149: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:38:59.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:38:58 smithi119 ceph-mon[31518]: pgmap v4149: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:38:59.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:38:58 smithi132 ceph-mon[35941]: pgmap v4149: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:39:01.251 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:39:00 smithi132 ceph-mon[35941]: pgmap v4150: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:39:01.251 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:39:00 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7400.766569 secs 2024-03-28T02:39:01.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:39:00 smithi165 ceph-mon[36302]: pgmap v4150: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:39:01.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:39:00 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7400.766569 secs 2024-03-28T02:39:01.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:39:00 smithi119 ceph-mon[31518]: pgmap v4150: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:39:01.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:39:00 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7400.766569 secs 2024-03-28T02:39:03.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:39:02 smithi165 ceph-mon[36302]: pgmap v4151: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:39:03.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:39:02 smithi119 ceph-mon[31518]: pgmap v4151: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:39:03.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:39:02 smithi132 ceph-mon[35941]: pgmap v4151: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:39:05.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:39:05 smithi165 ceph-mon[36302]: pgmap v4152: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:39:05.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:39:05 smithi119 ceph-mon[31518]: pgmap v4152: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:39:05.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:39:05 smithi132 ceph-mon[35941]: pgmap v4152: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:39:06.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:39:06 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7405.766631 secs 2024-03-28T02:39:06.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:39:06 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7405.766631 secs 2024-03-28T02:39:06.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:39:06 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7405.766631 secs 2024-03-28T02:39:07.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:39:07 smithi165 ceph-mon[36302]: pgmap v4153: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:39:07.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:39:07 smithi119 ceph-mon[31518]: pgmap v4153: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:39:07.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:39:07 smithi132 ceph-mon[35941]: pgmap v4153: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:39:09.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:39:09 smithi165 ceph-mon[36302]: pgmap v4154: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:39:09.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:39:09 smithi119 ceph-mon[31518]: pgmap v4154: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:39:09.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:39:09 smithi132 ceph-mon[35941]: pgmap v4154: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:39:11.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:39:11 smithi165 ceph-mon[36302]: pgmap v4155: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:39:11.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:39:11 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7410.766692 secs 2024-03-28T02:39:11.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:39:11 smithi119 ceph-mon[31518]: pgmap v4155: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:39:11.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:39:11 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7410.766692 secs 2024-03-28T02:39:11.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:39:11 smithi132 ceph-mon[35941]: pgmap v4155: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:39:11.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:39:11 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7410.766692 secs 2024-03-28T02:39:13.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:39:13 smithi165 ceph-mon[36302]: pgmap v4156: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:39:13.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:39:13 smithi119 ceph-mon[31518]: pgmap v4156: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:39:13.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:39:13 smithi132 ceph-mon[35941]: pgmap v4156: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:39:14.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:39:14 smithi165 ceph-mon[36302]: pgmap v4157: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:39:14.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:39:14 smithi119 ceph-mon[31518]: pgmap v4157: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:39:14.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:39:14 smithi132 ceph-mon[35941]: pgmap v4157: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:39:16.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:39:15 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7415.766782 secs 2024-03-28T02:39:16.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:39:15 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7415.766782 secs 2024-03-28T02:39:16.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:39:15 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7415.766782 secs 2024-03-28T02:39:17.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:39:16 smithi165 ceph-mon[36302]: pgmap v4158: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:39:17.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:39:16 smithi119 ceph-mon[31518]: pgmap v4158: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:39:17.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:39:16 smithi132 ceph-mon[35941]: pgmap v4158: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:39:19.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:39:18 smithi132 ceph-mon[35941]: pgmap v4159: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:39:19.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:39:18 smithi165 ceph-mon[36302]: pgmap v4159: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:39:19.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:39:18 smithi119 ceph-mon[31518]: pgmap v4159: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:39:21.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:39:20 smithi132 ceph-mon[35941]: pgmap v4160: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:39:21.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:39:20 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7420.766845 secs 2024-03-28T02:39:21.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:39:20 smithi165 ceph-mon[36302]: pgmap v4160: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:39:21.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:39:20 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7420.766845 secs 2024-03-28T02:39:21.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:39:20 smithi119 ceph-mon[31518]: pgmap v4160: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:39:21.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:39:20 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7420.766845 secs 2024-03-28T02:39:23.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:39:22 smithi132 ceph-mon[35941]: pgmap v4161: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:39:23.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:39:22 smithi165 ceph-mon[36302]: pgmap v4161: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:39:23.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:39:22 smithi119 ceph-mon[31518]: pgmap v4161: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:39:25.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:39:24 smithi165 ceph-mon[36302]: pgmap v4162: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:39:25.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:39:24 smithi119 ceph-mon[31518]: pgmap v4162: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:39:25.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:39:24 smithi132 ceph-mon[35941]: pgmap v4162: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:39:26.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:39:25 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7425.766909 secs 2024-03-28T02:39:26.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:39:25 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7425.766909 secs 2024-03-28T02:39:26.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:39:25 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7425.766909 secs 2024-03-28T02:39:27.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:39:26 smithi165 ceph-mon[36302]: pgmap v4163: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:39:27.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:39:26 smithi119 ceph-mon[31518]: pgmap v4163: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:39:27.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:39:26 smithi132 ceph-mon[35941]: pgmap v4163: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:39:29.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:39:28 smithi165 ceph-mon[36302]: pgmap v4164: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:39:29.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:39:28 smithi119 ceph-mon[31518]: pgmap v4164: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:39:29.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:39:28 smithi132 ceph-mon[35941]: pgmap v4164: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:39:31.251 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:39:30 smithi132 ceph-mon[35941]: pgmap v4165: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:39:31.252 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:39:30 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7430.766971 secs 2024-03-28T02:39:31.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:39:30 smithi165 ceph-mon[36302]: pgmap v4165: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:39:31.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:39:30 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7430.766971 secs 2024-03-28T02:39:31.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:39:30 smithi119 ceph-mon[31518]: pgmap v4165: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:39:31.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:39:30 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7430.766971 secs 2024-03-28T02:39:33.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:39:32 smithi119 ceph-mon[31518]: pgmap v4166: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:39:33.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:39:32 smithi165 ceph-mon[36302]: pgmap v4166: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:39:33.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:39:32 smithi132 ceph-mon[35941]: pgmap v4166: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:39:35.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:39:34 smithi165 ceph-mon[36302]: pgmap v4167: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:39:35.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:39:34 smithi119 ceph-mon[31518]: pgmap v4167: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:39:35.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:39:34 smithi132 ceph-mon[35941]: pgmap v4167: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:39:36.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:39:35 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7435.767006 secs 2024-03-28T02:39:36.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:39:35 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7435.767006 secs 2024-03-28T02:39:36.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:39:35 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7435.767006 secs 2024-03-28T02:39:37.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:39:36 smithi165 ceph-mon[36302]: pgmap v4168: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:39:37.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:39:36 smithi119 ceph-mon[31518]: pgmap v4168: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:39:37.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:39:36 smithi132 ceph-mon[35941]: pgmap v4168: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:39:39.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:39:38 smithi165 ceph-mon[36302]: pgmap v4169: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:39:39.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:39:38 smithi119 ceph-mon[31518]: pgmap v4169: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:39:39.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:39:38 smithi132 ceph-mon[35941]: pgmap v4169: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:39:41.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:39:41 smithi165 ceph-mon[36302]: pgmap v4170: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:39:41.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:39:41 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7440.767046 secs 2024-03-28T02:39:41.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:39:41 smithi119 ceph-mon[31518]: pgmap v4170: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:39:41.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:39:41 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7440.767046 secs 2024-03-28T02:39:41.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:39:41 smithi132 ceph-mon[35941]: pgmap v4170: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:39:41.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:39:41 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7440.767046 secs 2024-03-28T02:39:43.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:39:43 smithi165 ceph-mon[36302]: pgmap v4171: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:39:43.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:39:43 smithi119 ceph-mon[31518]: pgmap v4171: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:39:43.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:39:43 smithi132 ceph-mon[35941]: pgmap v4171: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:39:45.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:39:45 smithi165 ceph-mon[36302]: pgmap v4172: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:39:45.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:39:45 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:39:45.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:39:45 smithi119 ceph-mon[31518]: pgmap v4172: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:39:45.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:39:45 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:39:45.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:39:45 smithi132 ceph-mon[35941]: pgmap v4172: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:39:45.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:39:45 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:39:46.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:39:46 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7445.767136 secs 2024-03-28T02:39:46.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:39:46 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:39:46.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:39:46 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:39:46.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:39:46 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:39:46.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:39:46 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7445.767136 secs 2024-03-28T02:39:46.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:39:46 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:39:46.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:39:46 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:39:46.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:39:46 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:39:46.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:39:46 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7445.767136 secs 2024-03-28T02:39:46.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:39:46 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:39:46.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:39:46 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:39:46.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:39:46 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:39:47.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:39:47 smithi165 ceph-mon[36302]: pgmap v4173: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T02:39:47.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:39:47 smithi165 ceph-mon[36302]: pgmap v4174: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:39:47.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:39:47 smithi119 ceph-mon[31518]: pgmap v4173: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T02:39:47.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:39:47 smithi119 ceph-mon[31518]: pgmap v4174: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:39:47.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:39:47 smithi132 ceph-mon[35941]: pgmap v4173: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T02:39:47.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:39:47 smithi132 ceph-mon[35941]: pgmap v4174: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:39:49.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:39:49 smithi165 ceph-mon[36302]: pgmap v4175: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:39:49.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:39:49 smithi119 ceph-mon[31518]: pgmap v4175: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:39:49.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:39:49 smithi132 ceph-mon[35941]: pgmap v4175: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:39:51.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:39:51 smithi165 ceph-mon[36302]: pgmap v4176: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:39:51.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:39:51 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7450.767227 secs 2024-03-28T02:39:51.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:39:51 smithi119 ceph-mon[31518]: pgmap v4176: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:39:51.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:39:51 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7450.767227 secs 2024-03-28T02:39:51.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:39:51 smithi132 ceph-mon[35941]: pgmap v4176: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:39:51.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:39:51 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7450.767227 secs 2024-03-28T02:39:53.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:39:53 smithi165 ceph-mon[36302]: pgmap v4177: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:39:53.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:39:53 smithi119 ceph-mon[31518]: pgmap v4177: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:39:53.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:39:53 smithi132 ceph-mon[35941]: pgmap v4177: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:39:55.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:39:55 smithi165 ceph-mon[36302]: pgmap v4178: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:39:55.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:39:55 smithi119 ceph-mon[31518]: pgmap v4178: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:39:55.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:39:55 smithi132 ceph-mon[35941]: pgmap v4178: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:39:56.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:39:56 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7455.767318 secs 2024-03-28T02:39:56.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:39:56 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7455.767318 secs 2024-03-28T02:39:56.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:39:56 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7455.767318 secs 2024-03-28T02:39:57.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:39:57 smithi165 ceph-mon[36302]: pgmap v4179: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T02:39:57.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:39:57 smithi119 ceph-mon[31518]: pgmap v4179: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T02:39:57.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:39:57 smithi132 ceph-mon[35941]: pgmap v4179: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T02:39:59.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:39:59 smithi165 ceph-mon[36302]: pgmap v4180: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:39:59.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:39:59 smithi119 ceph-mon[31518]: pgmap v4180: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:39:59.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:39:59 smithi132 ceph-mon[35941]: pgmap v4180: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:40:00.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:40:00 smithi165 ceph-mon[36302]: overall HEALTH_WARN 5 failed cephadm daemon(s); 1 filesystem is degraded; 1 MDSs report slow requests 2024-03-28T02:40:00.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:40:00 smithi119 ceph-mon[31518]: overall HEALTH_WARN 5 failed cephadm daemon(s); 1 filesystem is degraded; 1 MDSs report slow requests 2024-03-28T02:40:00.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:40:00 smithi132 ceph-mon[35941]: overall HEALTH_WARN 5 failed cephadm daemon(s); 1 filesystem is degraded; 1 MDSs report slow requests 2024-03-28T02:40:01.350 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:40:01 smithi132 ceph-mon[35941]: pgmap v4181: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:40:01.351 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:40:01 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7460.767409 secs 2024-03-28T02:40:01.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:40:01 smithi165 ceph-mon[36302]: pgmap v4181: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:40:01.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:40:01 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7460.767409 secs 2024-03-28T02:40:01.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:40:01 smithi119 ceph-mon[31518]: pgmap v4181: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:40:01.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:40:01 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7460.767409 secs 2024-03-28T02:40:03.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:40:03 smithi165 ceph-mon[36302]: pgmap v4182: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:40:03.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:40:03 smithi119 ceph-mon[31518]: pgmap v4182: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:40:03.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:40:03 smithi132 ceph-mon[35941]: pgmap v4182: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:40:05.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:40:05 smithi165 ceph-mon[36302]: pgmap v4183: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:40:05.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:40:05 smithi119 ceph-mon[31518]: pgmap v4183: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:40:05.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:40:05 smithi132 ceph-mon[35941]: pgmap v4183: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:40:06.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:40:06 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7465.767500 secs 2024-03-28T02:40:06.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:40:06 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7465.767500 secs 2024-03-28T02:40:06.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:40:06 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7465.767500 secs 2024-03-28T02:40:07.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:40:07 smithi165 ceph-mon[36302]: pgmap v4184: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:40:07.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:40:07 smithi119 ceph-mon[31518]: pgmap v4184: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:40:07.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:40:07 smithi132 ceph-mon[35941]: pgmap v4184: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:40:09.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:40:09 smithi165 ceph-mon[36302]: pgmap v4185: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:40:09.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:40:09 smithi119 ceph-mon[31518]: pgmap v4185: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:40:09.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:40:09 smithi132 ceph-mon[35941]: pgmap v4185: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:40:11.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:40:11 smithi165 ceph-mon[36302]: pgmap v4186: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:40:11.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:40:11 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7470.767586 secs 2024-03-28T02:40:11.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:40:11 smithi119 ceph-mon[31518]: pgmap v4186: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:40:11.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:40:11 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7470.767586 secs 2024-03-28T02:40:11.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:40:11 smithi132 ceph-mon[35941]: pgmap v4186: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:40:11.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:40:11 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7470.767586 secs 2024-03-28T02:40:13.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:40:13 smithi165 ceph-mon[36302]: pgmap v4187: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:40:13.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:40:13 smithi119 ceph-mon[31518]: pgmap v4187: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:40:13.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:40:13 smithi132 ceph-mon[35941]: pgmap v4187: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:40:15.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:40:15 smithi165 ceph-mon[36302]: pgmap v4188: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:40:15.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:40:15 smithi119 ceph-mon[31518]: pgmap v4188: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:40:15.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:40:15 smithi132 ceph-mon[35941]: pgmap v4188: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:40:16.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:40:16 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7475.767679 secs 2024-03-28T02:40:16.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:40:16 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7475.767679 secs 2024-03-28T02:40:16.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:40:16 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7475.767679 secs 2024-03-28T02:40:17.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:40:17 smithi165 ceph-mon[36302]: pgmap v4189: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:40:17.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:40:17 smithi119 ceph-mon[31518]: pgmap v4189: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:40:17.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:40:17 smithi132 ceph-mon[35941]: pgmap v4189: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:40:19.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:40:19 smithi165 ceph-mon[36302]: pgmap v4190: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:40:19.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:40:19 smithi119 ceph-mon[31518]: pgmap v4190: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:40:19.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:40:19 smithi132 ceph-mon[35941]: pgmap v4190: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:40:21.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:40:21 smithi119 ceph-mon[31518]: pgmap v4191: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:40:21.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:40:21 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7480.767763 secs 2024-03-28T02:40:21.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:40:21 smithi132 ceph-mon[35941]: pgmap v4191: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:40:21.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:40:21 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7480.767763 secs 2024-03-28T02:40:21.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:40:21 smithi165 ceph-mon[36302]: pgmap v4191: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:40:21.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:40:21 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7480.767763 secs 2024-03-28T02:40:23.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:40:23 smithi119 ceph-mon[31518]: pgmap v4192: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:40:23.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:40:23 smithi132 ceph-mon[35941]: pgmap v4192: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:40:23.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:40:23 smithi165 ceph-mon[36302]: pgmap v4192: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:40:25.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:40:25 smithi119 ceph-mon[31518]: pgmap v4193: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:40:25.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:40:25 smithi132 ceph-mon[35941]: pgmap v4193: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:40:25.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:40:25 smithi165 ceph-mon[36302]: pgmap v4193: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:40:26.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:40:26 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7485.767804 secs 2024-03-28T02:40:26.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:40:26 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7485.767804 secs 2024-03-28T02:40:26.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:40:26 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7485.767804 secs 2024-03-28T02:40:27.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:40:27 smithi132 ceph-mon[35941]: pgmap v4194: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:40:27.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:40:27 smithi165 ceph-mon[36302]: pgmap v4194: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:40:27.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:40:27 smithi119 ceph-mon[31518]: pgmap v4194: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:40:29.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:40:29 smithi132 ceph-mon[35941]: pgmap v4195: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:40:29.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:40:29 smithi165 ceph-mon[36302]: pgmap v4195: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:40:29.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:40:29 smithi119 ceph-mon[31518]: pgmap v4195: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:40:31.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:40:31 smithi132 ceph-mon[35941]: pgmap v4196: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:40:31.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:40:31 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7490.767907 secs 2024-03-28T02:40:31.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:40:31 smithi165 ceph-mon[36302]: pgmap v4196: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:40:31.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:40:31 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7490.767907 secs 2024-03-28T02:40:31.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:40:31 smithi119 ceph-mon[31518]: pgmap v4196: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:40:31.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:40:31 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7490.767907 secs 2024-03-28T02:40:33.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:40:33 smithi132 ceph-mon[35941]: pgmap v4197: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:40:33.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:40:33 smithi165 ceph-mon[36302]: pgmap v4197: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:40:33.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:40:33 smithi119 ceph-mon[31518]: pgmap v4197: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:40:35.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:40:35 smithi132 ceph-mon[35941]: pgmap v4198: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:40:35.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:40:35 smithi165 ceph-mon[36302]: pgmap v4198: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:40:35.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:40:35 smithi119 ceph-mon[31518]: pgmap v4198: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:40:36.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:40:36 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7495.767969 secs 2024-03-28T02:40:36.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:40:36 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7495.767969 secs 2024-03-28T02:40:36.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:40:36 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7495.767969 secs 2024-03-28T02:40:37.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:40:37 smithi132 ceph-mon[35941]: pgmap v4199: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:40:37.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:40:37 smithi165 ceph-mon[36302]: pgmap v4199: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:40:37.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:40:37 smithi119 ceph-mon[31518]: pgmap v4199: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:40:39.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:40:39 smithi132 ceph-mon[35941]: pgmap v4200: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:40:39.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:40:39 smithi165 ceph-mon[36302]: pgmap v4200: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:40:39.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:40:39 smithi119 ceph-mon[31518]: pgmap v4200: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:40:41.456 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:40:41 smithi119 ceph-mon[31518]: pgmap v4201: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:40:41.456 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:40:41 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7500.768058 secs 2024-03-28T02:40:41.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:40:41 smithi165 ceph-mon[36302]: pgmap v4201: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:40:41.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:40:41 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7500.768058 secs 2024-03-28T02:40:41.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:40:41 smithi132 ceph-mon[35941]: pgmap v4201: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:40:41.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:40:41 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7500.768058 secs 2024-03-28T02:40:43.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:40:43 smithi165 ceph-mon[36302]: pgmap v4202: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:40:43.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:40:43 smithi119 ceph-mon[31518]: pgmap v4202: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:40:43.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:40:43 smithi132 ceph-mon[35941]: pgmap v4202: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:40:45.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:40:45 smithi165 ceph-mon[36302]: pgmap v4203: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:40:45.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:40:45 smithi119 ceph-mon[31518]: pgmap v4203: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:40:45.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:40:45 smithi132 ceph-mon[35941]: pgmap v4203: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:40:46.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:40:46 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7505.768145 secs 2024-03-28T02:40:46.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:40:46 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:40:46.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:40:46 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7505.768145 secs 2024-03-28T02:40:46.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:40:46 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:40:46.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:40:46 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7505.768145 secs 2024-03-28T02:40:46.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:40:46 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:40:47.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:40:47 smithi165 ceph-mon[36302]: pgmap v4204: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:40:47.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:40:47 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:40:47.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:40:47 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:40:47.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:40:47 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:40:47.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:40:47 smithi119 ceph-mon[31518]: pgmap v4204: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:40:47.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:40:47 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:40:47.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:40:47 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:40:47.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:40:47 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:40:47.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:40:47 smithi132 ceph-mon[35941]: pgmap v4204: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:40:47.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:40:47 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:40:47.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:40:47 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:40:47.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:40:47 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:40:48.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:40:48 smithi165 ceph-mon[36302]: pgmap v4205: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T02:40:48.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:40:48 smithi165 ceph-mon[36302]: pgmap v4206: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:40:48.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:40:48 smithi119 ceph-mon[31518]: pgmap v4205: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T02:40:48.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:40:48 smithi119 ceph-mon[31518]: pgmap v4206: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:40:48.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:40:48 smithi132 ceph-mon[35941]: pgmap v4205: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T02:40:48.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:40:48 smithi132 ceph-mon[35941]: pgmap v4206: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:40:50.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:40:50 smithi165 ceph-mon[36302]: pgmap v4207: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-28T02:40:50.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:40:50 smithi119 ceph-mon[31518]: pgmap v4207: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-28T02:40:50.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:40:50 smithi132 ceph-mon[35941]: pgmap v4207: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-28T02:40:51.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:40:51 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7510.768234 secs 2024-03-28T02:40:51.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:40:51 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7510.768234 secs 2024-03-28T02:40:51.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:40:51 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7510.768234 secs 2024-03-28T02:40:52.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:40:52 smithi165 ceph-mon[36302]: pgmap v4208: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:40:52.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:40:52 smithi119 ceph-mon[31518]: pgmap v4208: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:40:52.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:40:52 smithi132 ceph-mon[35941]: pgmap v4208: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:40:54.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:40:54 smithi165 ceph-mon[36302]: pgmap v4209: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:40:54.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:40:54 smithi119 ceph-mon[31518]: pgmap v4209: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:40:54.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:40:54 smithi132 ceph-mon[35941]: pgmap v4209: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:40:56.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:40:56 smithi165 ceph-mon[36302]: pgmap v4210: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:40:56.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:40:56 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7515.768323 secs 2024-03-28T02:40:56.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:40:56 smithi119 ceph-mon[31518]: pgmap v4210: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:40:56.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:40:56 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7515.768323 secs 2024-03-28T02:40:56.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:40:56 smithi132 ceph-mon[35941]: pgmap v4210: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:40:56.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:40:56 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7515.768323 secs 2024-03-28T02:40:58.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:40:58 smithi165 ceph-mon[36302]: pgmap v4211: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T02:40:58.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:40:58 smithi119 ceph-mon[31518]: pgmap v4211: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T02:40:58.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:40:58 smithi132 ceph-mon[35941]: pgmap v4211: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T02:41:00.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:41:00 smithi165 ceph-mon[36302]: pgmap v4212: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:41:00.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:41:00 smithi119 ceph-mon[31518]: pgmap v4212: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:41:00.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:41:00 smithi132 ceph-mon[35941]: pgmap v4212: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:41:01.601 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:41:01 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7520.768414 secs 2024-03-28T02:41:01.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:41:01 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7520.768414 secs 2024-03-28T02:41:01.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:41:01 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7520.768414 secs 2024-03-28T02:41:02.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:41:02 smithi165 ceph-mon[36302]: pgmap v4213: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:41:02.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:41:02 smithi119 ceph-mon[31518]: pgmap v4213: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:41:02.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:41:02 smithi132 ceph-mon[35941]: pgmap v4213: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:41:04.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:41:04 smithi165 ceph-mon[36302]: pgmap v4214: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:41:04.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:41:04 smithi119 ceph-mon[31518]: pgmap v4214: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:41:04.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:41:04 smithi132 ceph-mon[35941]: pgmap v4214: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:41:06.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:41:06 smithi119 ceph-mon[31518]: pgmap v4215: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:41:06.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:41:06 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7525.768503 secs 2024-03-28T02:41:06.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:41:06 smithi165 ceph-mon[36302]: pgmap v4215: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:41:06.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:41:06 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7525.768503 secs 2024-03-28T02:41:06.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:41:06 smithi132 ceph-mon[35941]: pgmap v4215: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:41:06.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:41:06 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7525.768503 secs 2024-03-28T02:41:08.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:41:08 smithi165 ceph-mon[36302]: pgmap v4216: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:41:08.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:41:08 smithi119 ceph-mon[31518]: pgmap v4216: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:41:08.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:41:08 smithi132 ceph-mon[35941]: pgmap v4216: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:41:10.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:41:10 smithi165 ceph-mon[36302]: pgmap v4217: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:41:10.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:41:10 smithi119 ceph-mon[31518]: pgmap v4217: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:41:10.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:41:10 smithi132 ceph-mon[35941]: pgmap v4217: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:41:11.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:41:11 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7530.768566 secs 2024-03-28T02:41:11.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:41:11 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7530.768566 secs 2024-03-28T02:41:11.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:41:11 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7530.768566 secs 2024-03-28T02:41:12.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:41:12 smithi165 ceph-mon[36302]: pgmap v4218: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:41:12.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:41:12 smithi119 ceph-mon[31518]: pgmap v4218: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:41:12.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:41:12 smithi132 ceph-mon[35941]: pgmap v4218: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:41:13.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:41:13 smithi165 ceph-mon[36302]: pgmap v4219: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:41:13.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:41:13 smithi119 ceph-mon[31518]: pgmap v4219: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:41:13.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:41:13 smithi132 ceph-mon[35941]: pgmap v4219: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:41:16.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:41:16 smithi165 ceph-mon[36302]: pgmap v4220: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:41:16.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:41:16 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7535.768655 secs 2024-03-28T02:41:16.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:41:16 smithi119 ceph-mon[31518]: pgmap v4220: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:41:16.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:41:16 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7535.768655 secs 2024-03-28T02:41:16.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:41:16 smithi132 ceph-mon[35941]: pgmap v4220: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:41:16.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:41:16 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7535.768655 secs 2024-03-28T02:41:18.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:41:18 smithi165 ceph-mon[36302]: pgmap v4221: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T02:41:18.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:41:18 smithi119 ceph-mon[31518]: pgmap v4221: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T02:41:18.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:41:18 smithi132 ceph-mon[35941]: pgmap v4221: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T02:41:20.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:41:20 smithi165 ceph-mon[36302]: pgmap v4222: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T02:41:20.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:41:20 smithi119 ceph-mon[31518]: pgmap v4222: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T02:41:20.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:41:20 smithi132 ceph-mon[35941]: pgmap v4222: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T02:41:21.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:41:21 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7540.768718 secs 2024-03-28T02:41:21.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:41:21 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7540.768718 secs 2024-03-28T02:41:21.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:41:21 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7540.768718 secs 2024-03-28T02:41:22.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:41:22 smithi165 ceph-mon[36302]: pgmap v4223: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:41:22.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:41:22 smithi119 ceph-mon[31518]: pgmap v4223: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:41:22.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:41:22 smithi132 ceph-mon[35941]: pgmap v4223: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:41:24.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:41:24 smithi165 ceph-mon[36302]: pgmap v4224: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T02:41:24.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:41:24 smithi119 ceph-mon[31518]: pgmap v4224: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T02:41:24.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:41:24 smithi132 ceph-mon[35941]: pgmap v4224: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T02:41:26.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:41:26 smithi165 ceph-mon[36302]: pgmap v4225: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:41:26.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:41:26 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7545.768808 secs 2024-03-28T02:41:26.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:41:26 smithi119 ceph-mon[31518]: pgmap v4225: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:41:26.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:41:26 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7545.768808 secs 2024-03-28T02:41:26.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:41:26 smithi132 ceph-mon[35941]: pgmap v4225: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:41:26.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:41:26 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7545.768808 secs 2024-03-28T02:41:28.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:41:28 smithi165 ceph-mon[36302]: pgmap v4226: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:41:28.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:41:28 smithi119 ceph-mon[31518]: pgmap v4226: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:41:28.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:41:28 smithi132 ceph-mon[35941]: pgmap v4226: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:41:30.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:41:30 smithi165 ceph-mon[36302]: pgmap v4227: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:41:30.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:41:30 smithi119 ceph-mon[31518]: pgmap v4227: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:41:30.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:41:30 smithi132 ceph-mon[35941]: pgmap v4227: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:41:31.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:41:31 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7550.768889 secs 2024-03-28T02:41:31.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:41:31 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7550.768889 secs 2024-03-28T02:41:31.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:41:31 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7550.768889 secs 2024-03-28T02:41:32.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:41:32 smithi165 ceph-mon[36302]: pgmap v4228: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:41:32.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:41:32 smithi119 ceph-mon[31518]: pgmap v4228: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:41:32.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:41:32 smithi132 ceph-mon[35941]: pgmap v4228: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:41:34.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:41:34 smithi165 ceph-mon[36302]: pgmap v4229: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:41:34.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:41:34 smithi119 ceph-mon[31518]: pgmap v4229: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:41:34.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:41:34 smithi132 ceph-mon[35941]: pgmap v4229: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:41:36.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:41:36 smithi119 ceph-mon[31518]: pgmap v4230: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:41:36.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:41:36 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7555.768949 secs 2024-03-28T02:41:36.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:41:36 smithi132 ceph-mon[35941]: pgmap v4230: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:41:36.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:41:36 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7555.768949 secs 2024-03-28T02:41:36.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:41:36 smithi165 ceph-mon[36302]: pgmap v4230: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:41:36.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:41:36 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7555.768949 secs 2024-03-28T02:41:38.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:41:38 smithi119 ceph-mon[31518]: pgmap v4231: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:41:38.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:41:38 smithi132 ceph-mon[35941]: pgmap v4231: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:41:38.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:41:38 smithi165 ceph-mon[36302]: pgmap v4231: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:41:40.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:41:40 smithi119 ceph-mon[31518]: pgmap v4232: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T02:41:40.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:41:40 smithi132 ceph-mon[35941]: pgmap v4232: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T02:41:40.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:41:40 smithi165 ceph-mon[36302]: pgmap v4232: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T02:41:41.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:41:41 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7560.768965 secs 2024-03-28T02:41:41.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:41:41 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7560.768965 secs 2024-03-28T02:41:41.531 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:41:41 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7560.768965 secs 2024-03-28T02:41:42.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:41:42 smithi119 ceph-mon[31518]: pgmap v4233: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:41:42.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:41:42 smithi132 ceph-mon[35941]: pgmap v4233: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:41:42.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:41:42 smithi165 ceph-mon[36302]: pgmap v4233: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:41:44.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:41:44 smithi132 ceph-mon[35941]: pgmap v4234: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:41:44.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:41:44 smithi165 ceph-mon[36302]: pgmap v4234: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:41:44.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:41:44 smithi119 ceph-mon[31518]: pgmap v4234: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:41:46.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:41:46 smithi132 ceph-mon[35941]: pgmap v4235: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:41:46.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:41:46 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7565.769027 secs 2024-03-28T02:41:46.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:41:46 smithi165 ceph-mon[36302]: pgmap v4235: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:41:46.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:41:46 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7565.769027 secs 2024-03-28T02:41:46.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:41:46 smithi119 ceph-mon[31518]: pgmap v4235: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:41:46.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:41:46 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7565.769027 secs 2024-03-28T02:41:48.376 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:41:48 smithi165 ceph-mon[36302]: pgmap v4236: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:41:48.377 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:41:48 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:41:48.377 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:41:48 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:41:48.377 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:41:48 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:41:48.377 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:41:48 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:41:48.377 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:41:48 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:41:48.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:41:48 smithi132 ceph-mon[35941]: pgmap v4236: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:41:48.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:41:48 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:41:48.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:41:48 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:41:48.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:41:48 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:41:48.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:41:48 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:41:48.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:41:48 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:41:48.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:41:48 smithi119 ceph-mon[31518]: pgmap v4236: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:41:48.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:41:48 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:41:48.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:41:48 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:41:48.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:41:48 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:41:48.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:41:48 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:41:48.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:41:48 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:41:50.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:41:50 smithi132 ceph-mon[35941]: pgmap v4237: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T02:41:50.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:41:50 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:41:50.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:41:50 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:41:50.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:41:50 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:41:50.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:41:50 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:41:50.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:41:50 smithi165 ceph-mon[36302]: pgmap v4237: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T02:41:50.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:41:50 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:41:50.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:41:50 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:41:50.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:41:50 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:41:50.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:41:50 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:41:50.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:41:50 smithi119 ceph-mon[31518]: pgmap v4237: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T02:41:50.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:41:50 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:41:50.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:41:50 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:41:50.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:41:50 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:41:50.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:41:50 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:41:51.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:41:51 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7570.769095 secs 2024-03-28T02:41:51.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:41:51 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:41:51.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:41:51 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:41:51.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:41:51 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:41:51.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:41:51 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:41:51.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:41:51 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:41:51.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:41:51 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:41:51.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:41:51 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:41:51.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:41:51 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:41:51.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:41:51 smithi132 ceph-mon[35941]: Adjusting osd_memory_target on smithi132 to 174.1M 2024-03-28T02:41:51.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:41:51 smithi132 ceph-mon[35941]: Unable to set osd_memory_target on smithi132 to 182640640: error parsing value: Value '182640640' is below minimum 939524096 2024-03-28T02:41:51.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:41:51 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:41:51.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:41:51 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:41:51.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:41:51 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:41:51.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:41:51 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:41:51.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:41:51 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:41:51.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:41:51 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:41:51.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:41:51 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:41:51.452 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:41:51 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:41:51.452 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:41:51 smithi132 ceph-mon[35941]: Adjusting osd_memory_target on smithi165 to 174.1M 2024-03-28T02:41:51.452 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:41:51 smithi132 ceph-mon[35941]: Unable to set osd_memory_target on smithi165 to 182635622: error parsing value: Value '182635622' is below minimum 939524096 2024-03-28T02:41:51.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:41:51 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7570.769095 secs 2024-03-28T02:41:51.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:41:51 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:41:51.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:41:51 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:41:51.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:41:51 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:41:51.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:41:51 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:41:51.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:41:51 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:41:51.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:41:51 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:41:51.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:41:51 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:41:51.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:41:51 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:41:51.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:41:51 smithi165 ceph-mon[36302]: Adjusting osd_memory_target on smithi132 to 174.1M 2024-03-28T02:41:51.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:41:51 smithi165 ceph-mon[36302]: Unable to set osd_memory_target on smithi132 to 182640640: error parsing value: Value '182640640' is below minimum 939524096 2024-03-28T02:41:51.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:41:51 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:41:51.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:41:51 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:41:51.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:41:51 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:41:51.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:41:51 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:41:51.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:41:51 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:41:51.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:41:51 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:41:51.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:41:51 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:41:51.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:41:51 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:41:51.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:41:51 smithi165 ceph-mon[36302]: Adjusting osd_memory_target on smithi165 to 174.1M 2024-03-28T02:41:51.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:41:51 smithi165 ceph-mon[36302]: Unable to set osd_memory_target on smithi165 to 182635622: error parsing value: Value '182635622' is below minimum 939524096 2024-03-28T02:41:51.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:41:51 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7570.769095 secs 2024-03-28T02:41:51.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:41:51 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:41:51.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:41:51 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:41:51.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:41:51 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:41:51.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:41:51 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:41:51.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:41:51 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:41:51.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:41:51 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:41:51.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:41:51 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:41:51.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:41:51 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:41:51.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:41:51 smithi119 ceph-mon[31518]: Adjusting osd_memory_target on smithi132 to 174.1M 2024-03-28T02:41:51.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:41:51 smithi119 ceph-mon[31518]: Unable to set osd_memory_target on smithi132 to 182640640: error parsing value: Value '182640640' is below minimum 939524096 2024-03-28T02:41:51.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:41:51 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:41:51.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:41:51 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:41:51.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:41:51 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:41:51.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:41:51 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:41:51.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:41:51 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:41:51.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:41:51 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:41:51.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:41:51 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:41:51.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:41:51 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:41:51.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:41:51 smithi119 ceph-mon[31518]: Adjusting osd_memory_target on smithi165 to 174.1M 2024-03-28T02:41:51.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:41:51 smithi119 ceph-mon[31518]: Unable to set osd_memory_target on smithi165 to 182635622: error parsing value: Value '182635622' is below minimum 939524096 2024-03-28T02:41:52.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:41:52 smithi132 ceph-mon[35941]: pgmap v4238: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:41:52.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:41:52 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:41:52.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:41:52 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:41:52.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:41:52 smithi165 ceph-mon[36302]: pgmap v4238: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:41:52.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:41:52 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:41:52.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:41:52 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:41:52.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:41:52 smithi119 ceph-mon[31518]: pgmap v4238: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:41:52.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:41:52 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:41:52.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:41:52 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:41:53.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:41:53 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:41:53.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:41:53 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:41:53.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:41:53 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:41:53.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:41:53 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:41:53.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:41:53 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:41:53.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:41:53 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:41:53.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:41:53 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:41:53.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:41:53 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:41:53.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:41:53 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:41:53.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:41:53 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:41:53.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:41:53 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:41:53.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:41:53 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:41:53.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:41:53 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:41:53.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:41:53 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:41:53.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:41:53 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:41:53.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:41:53 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:41:53.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:41:53 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:41:53.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:41:53 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:41:53.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:41:53 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:41:53.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:41:53 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:41:53.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:41:53 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:41:53.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:41:53 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:41:53.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:41:53 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:41:53.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:41:53 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:41:53.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:41:53 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:41:53.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:41:53 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:41:53.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:41:53 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:41:53.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:41:53 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:41:53.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:41:53 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:41:53.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:41:53 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:41:53.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:41:53 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:41:53.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:41:53 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:41:53.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:41:53 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:41:54.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:41:54 smithi132 ceph-mon[35941]: Adjusting osd_memory_target on smithi119 to 176.7M 2024-03-28T02:41:54.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:41:54 smithi132 ceph-mon[35941]: Unable to set osd_memory_target on smithi119 to 185372364: error parsing value: Value '185372364' is below minimum 939524096 2024-03-28T02:41:54.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:41:54 smithi132 ceph-mon[35941]: pgmap v4239: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T02:41:54.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:41:54 smithi132 ceph-mon[35941]: pgmap v4240: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T02:41:54.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:41:54 smithi165 ceph-mon[36302]: Adjusting osd_memory_target on smithi119 to 176.7M 2024-03-28T02:41:54.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:41:54 smithi165 ceph-mon[36302]: Unable to set osd_memory_target on smithi119 to 185372364: error parsing value: Value '185372364' is below minimum 939524096 2024-03-28T02:41:54.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:41:54 smithi165 ceph-mon[36302]: pgmap v4239: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T02:41:54.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:41:54 smithi165 ceph-mon[36302]: pgmap v4240: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T02:41:54.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:41:54 smithi119 ceph-mon[31518]: Adjusting osd_memory_target on smithi119 to 176.7M 2024-03-28T02:41:54.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:41:54 smithi119 ceph-mon[31518]: Unable to set osd_memory_target on smithi119 to 185372364: error parsing value: Value '185372364' is below minimum 939524096 2024-03-28T02:41:54.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:41:54 smithi119 ceph-mon[31518]: pgmap v4239: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T02:41:54.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:41:54 smithi119 ceph-mon[31518]: pgmap v4240: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T02:41:56.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:41:56 smithi165 ceph-mon[36302]: pgmap v4241: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T02:41:56.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:41:56 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7575.769179 secs 2024-03-28T02:41:56.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:41:56 smithi119 ceph-mon[31518]: pgmap v4241: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T02:41:56.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:41:56 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7575.769179 secs 2024-03-28T02:41:56.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:41:56 smithi132 ceph-mon[35941]: pgmap v4241: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T02:41:56.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:41:56 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7575.769179 secs 2024-03-28T02:41:58.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:41:58 smithi165 ceph-mon[36302]: pgmap v4242: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T02:41:58.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:41:58 smithi119 ceph-mon[31518]: pgmap v4242: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T02:41:58.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:41:58 smithi132 ceph-mon[35941]: pgmap v4242: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T02:42:00.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:42:00 smithi165 ceph-mon[36302]: pgmap v4243: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T02:42:00.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:42:00 smithi119 ceph-mon[31518]: pgmap v4243: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T02:42:00.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:42:00 smithi132 ceph-mon[35941]: pgmap v4243: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T02:42:01.601 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:42:01 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7580.769271 secs 2024-03-28T02:42:01.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:42:01 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7580.769271 secs 2024-03-28T02:42:01.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:42:01 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7580.769271 secs 2024-03-28T02:42:02.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:42:02 smithi165 ceph-mon[36302]: pgmap v4244: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T02:42:02.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:42:02 smithi119 ceph-mon[31518]: pgmap v4244: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T02:42:02.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:42:02 smithi132 ceph-mon[35941]: pgmap v4244: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T02:42:04.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:42:04 smithi165 ceph-mon[36302]: pgmap v4245: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T02:42:04.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:42:04 smithi119 ceph-mon[31518]: pgmap v4245: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T02:42:04.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:42:04 smithi132 ceph-mon[35941]: pgmap v4245: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T02:42:06.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:42:06 smithi165 ceph-mon[36302]: pgmap v4246: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:42:06.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:42:06 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7585.769334 secs 2024-03-28T02:42:06.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:42:06 smithi119 ceph-mon[31518]: pgmap v4246: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:42:06.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:42:06 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7585.769334 secs 2024-03-28T02:42:06.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:42:06 smithi132 ceph-mon[35941]: pgmap v4246: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:42:06.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:42:06 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7585.769334 secs 2024-03-28T02:42:08.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:42:08 smithi165 ceph-mon[36302]: pgmap v4247: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:42:08.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:42:08 smithi119 ceph-mon[31518]: pgmap v4247: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:42:08.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:42:08 smithi132 ceph-mon[35941]: pgmap v4247: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:42:10.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:42:10 smithi165 ceph-mon[36302]: pgmap v4248: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:42:10.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:42:10 smithi119 ceph-mon[31518]: pgmap v4248: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:42:10.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:42:10 smithi132 ceph-mon[35941]: pgmap v4248: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:42:11.531 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:42:11 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7590.769396 secs 2024-03-28T02:42:11.555 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:42:11 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7590.769396 secs 2024-03-28T02:42:11.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:42:11 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7590.769396 secs 2024-03-28T02:42:12.543 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:42:12 smithi165 ceph-mon[36302]: pgmap v4249: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:42:12.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:42:12 smithi119 ceph-mon[31518]: pgmap v4249: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:42:12.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:42:12 smithi132 ceph-mon[35941]: pgmap v4249: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:42:14.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:42:14 smithi165 ceph-mon[36302]: pgmap v4250: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:42:14.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:42:14 smithi119 ceph-mon[31518]: pgmap v4250: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:42:14.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:42:14 smithi132 ceph-mon[35941]: pgmap v4250: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:42:16.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:42:16 smithi165 ceph-mon[36302]: pgmap v4251: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:42:16.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:42:16 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7595.769487 secs 2024-03-28T02:42:16.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:42:16 smithi119 ceph-mon[31518]: pgmap v4251: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:42:16.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:42:16 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7595.769487 secs 2024-03-28T02:42:16.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:42:16 smithi132 ceph-mon[35941]: pgmap v4251: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:42:16.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:42:16 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7595.769487 secs 2024-03-28T02:42:18.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:42:18 smithi165 ceph-mon[36302]: pgmap v4252: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:42:18.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:42:18 smithi119 ceph-mon[31518]: pgmap v4252: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:42:18.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:42:18 smithi132 ceph-mon[35941]: pgmap v4252: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:42:20.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:42:20 smithi165 ceph-mon[36302]: pgmap v4253: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:42:20.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:42:20 smithi119 ceph-mon[31518]: pgmap v4253: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:42:20.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:42:20 smithi132 ceph-mon[35941]: pgmap v4253: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:42:21.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:42:21 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7600.769575 secs 2024-03-28T02:42:21.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:42:21 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7600.769575 secs 2024-03-28T02:42:21.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:42:21 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7600.769575 secs 2024-03-28T02:42:22.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:42:22 smithi165 ceph-mon[36302]: pgmap v4254: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:42:22.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:42:22 smithi119 ceph-mon[31518]: pgmap v4254: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:42:22.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:42:22 smithi132 ceph-mon[35941]: pgmap v4254: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:42:24.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:42:24 smithi165 ceph-mon[36302]: pgmap v4255: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:42:24.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:42:24 smithi119 ceph-mon[31518]: pgmap v4255: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:42:24.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:42:24 smithi132 ceph-mon[35941]: pgmap v4255: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:42:26.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:42:26 smithi165 ceph-mon[36302]: pgmap v4256: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:42:26.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:42:26 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7605.769665 secs 2024-03-28T02:42:26.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:42:26 smithi119 ceph-mon[31518]: pgmap v4256: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:42:26.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:42:26 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7605.769665 secs 2024-03-28T02:42:26.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:42:26 smithi132 ceph-mon[35941]: pgmap v4256: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:42:26.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:42:26 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7605.769665 secs 2024-03-28T02:42:28.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:42:28 smithi165 ceph-mon[36302]: pgmap v4257: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:42:28.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:42:28 smithi119 ceph-mon[31518]: pgmap v4257: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:42:28.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:42:28 smithi132 ceph-mon[35941]: pgmap v4257: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:42:30.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:42:30 smithi165 ceph-mon[36302]: pgmap v4258: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:42:30.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:42:30 smithi119 ceph-mon[31518]: pgmap v4258: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:42:30.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:42:30 smithi132 ceph-mon[35941]: pgmap v4258: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:42:31.601 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:42:31 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7610.769751 secs 2024-03-28T02:42:31.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:42:31 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7610.769751 secs 2024-03-28T02:42:31.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:42:31 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7610.769751 secs 2024-03-28T02:42:32.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:42:32 smithi165 ceph-mon[36302]: pgmap v4259: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:42:32.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:42:32 smithi119 ceph-mon[31518]: pgmap v4259: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:42:32.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:42:32 smithi132 ceph-mon[35941]: pgmap v4259: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:42:34.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:42:34 smithi165 ceph-mon[36302]: pgmap v4260: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:42:34.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:42:34 smithi119 ceph-mon[31518]: pgmap v4260: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:42:34.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:42:34 smithi132 ceph-mon[35941]: pgmap v4260: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:42:36.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:42:36 smithi165 ceph-mon[36302]: pgmap v4261: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:42:36.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:42:36 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7615.769814 secs 2024-03-28T02:42:36.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:42:36 smithi119 ceph-mon[31518]: pgmap v4261: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:42:36.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:42:36 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7615.769814 secs 2024-03-28T02:42:36.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:42:36 smithi132 ceph-mon[35941]: pgmap v4261: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:42:36.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:42:36 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7615.769814 secs 2024-03-28T02:42:38.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:42:38 smithi165 ceph-mon[36302]: pgmap v4262: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:42:38.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:42:38 smithi119 ceph-mon[31518]: pgmap v4262: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:42:38.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:42:38 smithi132 ceph-mon[35941]: pgmap v4262: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:42:39.555 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:42:39 smithi119 ceph-mon[31518]: pgmap v4263: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:42:39.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:42:39 smithi165 ceph-mon[36302]: pgmap v4263: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:42:39.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:42:39 smithi132 ceph-mon[35941]: pgmap v4263: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:42:41.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:42:40 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7620.769891 secs 2024-03-28T02:42:41.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:42:40 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7620.769891 secs 2024-03-28T02:42:41.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:42:40 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7620.769891 secs 2024-03-28T02:42:42.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:42:41 smithi165 ceph-mon[36302]: pgmap v4264: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:42:42.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:42:41 smithi119 ceph-mon[31518]: pgmap v4264: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:42:42.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:42:41 smithi132 ceph-mon[35941]: pgmap v4264: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:42:44.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:42:44 smithi165 ceph-mon[36302]: pgmap v4265: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:42:44.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:42:44 smithi119 ceph-mon[31518]: pgmap v4265: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:42:44.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:42:44 smithi132 ceph-mon[35941]: pgmap v4265: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:42:46.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:42:46 smithi165 ceph-mon[36302]: pgmap v4266: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:42:46.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:42:46 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7625.769980 secs 2024-03-28T02:42:46.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:42:46 smithi119 ceph-mon[31518]: pgmap v4266: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:42:46.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:42:46 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7625.769980 secs 2024-03-28T02:42:46.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:42:46 smithi132 ceph-mon[35941]: pgmap v4266: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:42:46.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:42:46 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7625.769980 secs 2024-03-28T02:42:48.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:42:48 smithi165 ceph-mon[36302]: pgmap v4267: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:42:48.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:42:48 smithi119 ceph-mon[31518]: pgmap v4267: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:42:48.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:42:48 smithi132 ceph-mon[35941]: pgmap v4267: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:42:50.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:42:50 smithi165 ceph-mon[36302]: pgmap v4268: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:42:50.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:42:50 smithi119 ceph-mon[31518]: pgmap v4268: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:42:50.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:42:50 smithi132 ceph-mon[35941]: pgmap v4268: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:42:51.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:42:51 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7630.770042 secs 2024-03-28T02:42:51.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:42:51 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7630.770042 secs 2024-03-28T02:42:51.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:42:51 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7630.770042 secs 2024-03-28T02:42:52.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:42:52 smithi165 ceph-mon[36302]: pgmap v4269: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:42:52.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:42:52 smithi119 ceph-mon[31518]: pgmap v4269: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:42:52.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:42:52 smithi132 ceph-mon[35941]: pgmap v4269: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:42:54.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:42:54 smithi165 ceph-mon[36302]: pgmap v4270: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:42:54.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:42:54 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:42:54.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:42:54 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:42:54.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:42:54 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:42:54.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:42:54 smithi119 ceph-mon[31518]: pgmap v4270: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:42:54.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:42:54 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:42:54.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:42:54 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:42:54.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:42:54 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:42:54.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:42:54 smithi132 ceph-mon[35941]: pgmap v4270: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:42:54.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:42:54 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:42:54.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:42:54 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:42:54.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:42:54 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:42:55.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:42:55 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:42:55.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:42:55 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:42:55.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:42:55 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:42:55.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:42:55 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:42:55.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:42:55 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:42:55.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:42:55 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:42:55.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:42:55 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:42:55.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:42:55 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:42:55.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:42:55 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:42:56.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:42:56 smithi165 ceph-mon[36302]: pgmap v4271: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 7 op/s 2024-03-28T02:42:56.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:42:56 smithi165 ceph-mon[36302]: pgmap v4272: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T02:42:56.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:42:56 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7635.770130 secs 2024-03-28T02:42:56.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:42:56 smithi119 ceph-mon[31518]: pgmap v4271: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 7 op/s 2024-03-28T02:42:56.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:42:56 smithi119 ceph-mon[31518]: pgmap v4272: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T02:42:56.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:42:56 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7635.770130 secs 2024-03-28T02:42:56.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:42:56 smithi132 ceph-mon[35941]: pgmap v4271: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 7 op/s 2024-03-28T02:42:56.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:42:56 smithi132 ceph-mon[35941]: pgmap v4272: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T02:42:56.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:42:56 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7635.770130 secs 2024-03-28T02:42:58.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:42:58 smithi165 ceph-mon[36302]: pgmap v4273: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T02:42:58.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:42:58 smithi119 ceph-mon[31518]: pgmap v4273: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T02:42:58.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:42:58 smithi132 ceph-mon[35941]: pgmap v4273: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T02:43:00.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:43:00 smithi165 ceph-mon[36302]: pgmap v4274: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T02:43:00.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:43:00 smithi119 ceph-mon[31518]: pgmap v4274: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T02:43:00.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:43:00 smithi132 ceph-mon[35941]: pgmap v4274: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T02:43:01.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:43:01 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7640.770162 secs 2024-03-28T02:43:01.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:43:01 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7640.770162 secs 2024-03-28T02:43:01.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:43:01 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7640.770162 secs 2024-03-28T02:43:02.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:43:02 smithi165 ceph-mon[36302]: pgmap v4275: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T02:43:02.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:43:02 smithi119 ceph-mon[31518]: pgmap v4275: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T02:43:02.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:43:02 smithi132 ceph-mon[35941]: pgmap v4275: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T02:43:04.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:43:04 smithi165 ceph-mon[36302]: pgmap v4276: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T02:43:04.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:43:04 smithi119 ceph-mon[31518]: pgmap v4276: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T02:43:04.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:43:04 smithi132 ceph-mon[35941]: pgmap v4276: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T02:43:05.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:43:05 smithi119 ceph-mon[31518]: pgmap v4277: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T02:43:05.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:43:05 smithi132 ceph-mon[35941]: pgmap v4277: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T02:43:05.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:43:05 smithi165 ceph-mon[36302]: pgmap v4277: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T02:43:06.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:43:06 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7645.770252 secs 2024-03-28T02:43:06.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:43:06 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7645.770252 secs 2024-03-28T02:43:06.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:43:06 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7645.770252 secs 2024-03-28T02:43:07.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:43:07 smithi132 ceph-mon[35941]: pgmap v4278: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:43:07.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:43:07 smithi165 ceph-mon[36302]: pgmap v4278: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:43:07.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:43:07 smithi119 ceph-mon[31518]: pgmap v4278: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:43:10.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:43:09 smithi165 ceph-mon[36302]: pgmap v4279: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:43:10.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:43:09 smithi119 ceph-mon[31518]: pgmap v4279: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:43:10.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:43:09 smithi132 ceph-mon[35941]: pgmap v4279: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:43:12.102 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:43:11 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7650.770340 secs 2024-03-28T02:43:12.102 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:43:11 smithi132 ceph-mon[35941]: pgmap v4280: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:43:12.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:43:11 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7650.770340 secs 2024-03-28T02:43:12.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:43:11 smithi165 ceph-mon[36302]: pgmap v4280: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:43:12.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:43:11 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7650.770340 secs 2024-03-28T02:43:12.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:43:11 smithi119 ceph-mon[31518]: pgmap v4280: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:43:14.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:43:13 smithi165 ceph-mon[36302]: pgmap v4281: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:43:14.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:43:13 smithi119 ceph-mon[31518]: pgmap v4281: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:43:14.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:43:13 smithi132 ceph-mon[35941]: pgmap v4281: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:43:16.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:43:15 smithi165 ceph-mon[36302]: pgmap v4282: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:43:16.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:43:15 smithi119 ceph-mon[31518]: pgmap v4282: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:43:16.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:43:15 smithi132 ceph-mon[35941]: pgmap v4282: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:43:17.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:43:16 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7655.770431 secs 2024-03-28T02:43:17.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:43:16 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7655.770431 secs 2024-03-28T02:43:17.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:43:16 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7655.770431 secs 2024-03-28T02:43:18.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:43:17 smithi165 ceph-mon[36302]: pgmap v4283: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:43:18.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:43:17 smithi119 ceph-mon[31518]: pgmap v4283: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:43:18.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:43:17 smithi132 ceph-mon[35941]: pgmap v4283: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:43:20.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:43:19 smithi165 ceph-mon[36302]: pgmap v4284: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:43:20.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:43:19 smithi119 ceph-mon[31518]: pgmap v4284: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:43:20.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:43:19 smithi132 ceph-mon[35941]: pgmap v4284: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:43:22.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:43:21 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7660.770519 secs 2024-03-28T02:43:22.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:43:21 smithi165 ceph-mon[36302]: pgmap v4285: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:43:22.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:43:21 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7660.770519 secs 2024-03-28T02:43:22.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:43:21 smithi119 ceph-mon[31518]: pgmap v4285: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:43:22.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:43:21 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7660.770519 secs 2024-03-28T02:43:22.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:43:21 smithi132 ceph-mon[35941]: pgmap v4285: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:43:24.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:43:23 smithi165 ceph-mon[36302]: pgmap v4286: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:43:24.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:43:23 smithi119 ceph-mon[31518]: pgmap v4286: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:43:24.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:43:23 smithi132 ceph-mon[35941]: pgmap v4286: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:43:26.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:43:25 smithi165 ceph-mon[36302]: pgmap v4287: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:43:26.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:43:25 smithi119 ceph-mon[31518]: pgmap v4287: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:43:26.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:43:25 smithi132 ceph-mon[35941]: pgmap v4287: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:43:27.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:43:26 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7665.770581 secs 2024-03-28T02:43:27.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:43:26 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7665.770581 secs 2024-03-28T02:43:27.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:43:26 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7665.770581 secs 2024-03-28T02:43:28.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:43:27 smithi165 ceph-mon[36302]: pgmap v4288: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:43:28.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:43:27 smithi119 ceph-mon[31518]: pgmap v4288: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:43:28.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:43:27 smithi132 ceph-mon[35941]: pgmap v4288: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:43:30.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:43:29 smithi165 ceph-mon[36302]: pgmap v4289: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:43:30.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:43:29 smithi119 ceph-mon[31518]: pgmap v4289: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:43:30.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:43:29 smithi132 ceph-mon[35941]: pgmap v4289: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:43:31.101 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:43:30 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7670.770669 secs 2024-03-28T02:43:31.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:43:30 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7670.770669 secs 2024-03-28T02:43:31.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:43:30 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7670.770669 secs 2024-03-28T02:43:32.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:43:31 smithi165 ceph-mon[36302]: pgmap v4290: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:43:32.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:43:31 smithi119 ceph-mon[31518]: pgmap v4290: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:43:32.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:43:31 smithi132 ceph-mon[35941]: pgmap v4290: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:43:34.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:43:33 smithi165 ceph-mon[36302]: pgmap v4291: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:43:34.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:43:33 smithi119 ceph-mon[31518]: pgmap v4291: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:43:34.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:43:33 smithi132 ceph-mon[35941]: pgmap v4291: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:43:36.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:43:35 smithi165 ceph-mon[36302]: pgmap v4292: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:43:36.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:43:35 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7675.770730 secs 2024-03-28T02:43:36.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:43:35 smithi119 ceph-mon[31518]: pgmap v4292: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:43:36.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:43:35 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7675.770730 secs 2024-03-28T02:43:36.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:43:35 smithi132 ceph-mon[35941]: pgmap v4292: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:43:36.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:43:35 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7675.770730 secs 2024-03-28T02:43:38.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:43:37 smithi165 ceph-mon[36302]: pgmap v4293: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:43:38.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:43:37 smithi119 ceph-mon[31518]: pgmap v4293: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:43:38.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:43:37 smithi132 ceph-mon[35941]: pgmap v4293: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:43:40.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:43:39 smithi165 ceph-mon[36302]: pgmap v4294: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:43:40.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:43:39 smithi119 ceph-mon[31518]: pgmap v4294: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:43:40.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:43:39 smithi132 ceph-mon[35941]: pgmap v4294: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:43:41.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:43:40 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7680.770819 secs 2024-03-28T02:43:41.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:43:40 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7680.770819 secs 2024-03-28T02:43:41.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:43:40 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7680.770819 secs 2024-03-28T02:43:42.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:43:41 smithi165 ceph-mon[36302]: pgmap v4295: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:43:42.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:43:41 smithi119 ceph-mon[31518]: pgmap v4295: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:43:42.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:43:41 smithi132 ceph-mon[35941]: pgmap v4295: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:43:44.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:43:43 smithi165 ceph-mon[36302]: pgmap v4296: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:43:44.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:43:43 smithi119 ceph-mon[31518]: pgmap v4296: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:43:44.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:43:43 smithi132 ceph-mon[35941]: pgmap v4296: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:43:46.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:43:45 smithi165 ceph-mon[36302]: pgmap v4297: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:43:46.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:43:45 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7685.770867 secs 2024-03-28T02:43:46.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:43:45 smithi119 ceph-mon[31518]: pgmap v4297: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:43:46.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:43:45 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7685.770867 secs 2024-03-28T02:43:46.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:43:45 smithi132 ceph-mon[35941]: pgmap v4297: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:43:46.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:43:45 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7685.770867 secs 2024-03-28T02:43:48.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:43:47 smithi165 ceph-mon[36302]: pgmap v4298: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:43:48.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:43:47 smithi119 ceph-mon[31518]: pgmap v4298: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:43:48.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:43:47 smithi132 ceph-mon[35941]: pgmap v4298: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:43:50.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:43:49 smithi165 ceph-mon[36302]: pgmap v4299: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:43:50.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:43:49 smithi119 ceph-mon[31518]: pgmap v4299: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:43:50.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:43:49 smithi132 ceph-mon[35941]: pgmap v4299: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:43:51.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:43:50 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7690.770931 secs 2024-03-28T02:43:51.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:43:50 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7690.770931 secs 2024-03-28T02:43:51.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:43:50 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7690.770931 secs 2024-03-28T02:43:52.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:43:51 smithi165 ceph-mon[36302]: pgmap v4300: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:43:52.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:43:51 smithi119 ceph-mon[31518]: pgmap v4300: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:43:52.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:43:51 smithi132 ceph-mon[35941]: pgmap v4300: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:43:54.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:43:53 smithi165 ceph-mon[36302]: pgmap v4301: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:43:54.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:43:53 smithi119 ceph-mon[31518]: pgmap v4301: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:43:54.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:43:53 smithi132 ceph-mon[35941]: pgmap v4301: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:43:56.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:43:55 smithi165 ceph-mon[36302]: pgmap v4302: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:43:56.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:43:55 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:43:56.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:43:55 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7695.771004 secs 2024-03-28T02:43:56.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:43:55 smithi119 ceph-mon[31518]: pgmap v4302: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:43:56.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:43:55 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:43:56.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:43:55 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7695.771004 secs 2024-03-28T02:43:56.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:43:55 smithi132 ceph-mon[35941]: pgmap v4302: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:43:56.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:43:55 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:43:56.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:43:55 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7695.771004 secs 2024-03-28T02:43:57.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:43:56 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:43:57.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:43:56 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:43:57.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:43:56 smithi165 ceph-mon[36302]: pgmap v4303: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T02:43:57.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:43:56 smithi165 ceph-mon[36302]: pgmap v4304: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:43:57.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:43:56 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:43:57.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:43:56 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:43:57.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:43:56 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:43:57.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:43:56 smithi119 ceph-mon[31518]: pgmap v4303: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T02:43:57.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:43:56 smithi119 ceph-mon[31518]: pgmap v4304: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:43:57.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:43:56 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:43:57.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:43:56 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:43:57.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:43:56 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:43:57.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:43:56 smithi132 ceph-mon[35941]: pgmap v4303: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T02:43:57.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:43:56 smithi132 ceph-mon[35941]: pgmap v4304: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:43:57.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:43:56 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:43:59.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:43:59 smithi165 ceph-mon[36302]: pgmap v4305: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:43:59.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:43:59 smithi119 ceph-mon[31518]: pgmap v4305: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:43:59.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:43:59 smithi132 ceph-mon[35941]: pgmap v4305: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:44:01.503 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:44:01 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7700.771092 secs 2024-03-28T02:44:01.503 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:44:01 smithi132 ceph-mon[35941]: pgmap v4306: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:44:01.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:44:01 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7700.771092 secs 2024-03-28T02:44:01.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:44:01 smithi165 ceph-mon[36302]: pgmap v4306: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:44:01.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:44:01 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7700.771092 secs 2024-03-28T02:44:01.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:44:01 smithi119 ceph-mon[31518]: pgmap v4306: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:44:03.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:44:03 smithi165 ceph-mon[36302]: pgmap v4307: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:44:03.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:44:03 smithi119 ceph-mon[31518]: pgmap v4307: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:44:03.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:44:03 smithi132 ceph-mon[35941]: pgmap v4307: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:44:05.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:44:05 smithi165 ceph-mon[36302]: pgmap v4308: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 5 op/s 2024-03-28T02:44:05.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:44:05 smithi119 ceph-mon[31518]: pgmap v4308: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 5 op/s 2024-03-28T02:44:05.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:44:05 smithi132 ceph-mon[35941]: pgmap v4308: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 5 op/s 2024-03-28T02:44:06.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:44:06 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7705.771184 secs 2024-03-28T02:44:06.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:44:06 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7705.771184 secs 2024-03-28T02:44:06.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:44:06 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7705.771184 secs 2024-03-28T02:44:07.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:44:07 smithi165 ceph-mon[36302]: pgmap v4309: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T02:44:07.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:44:07 smithi119 ceph-mon[31518]: pgmap v4309: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T02:44:07.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:44:07 smithi132 ceph-mon[35941]: pgmap v4309: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T02:44:09.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:44:09 smithi165 ceph-mon[36302]: pgmap v4310: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:44:09.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:44:09 smithi119 ceph-mon[31518]: pgmap v4310: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:44:09.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:44:09 smithi132 ceph-mon[35941]: pgmap v4310: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:44:11.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:44:11 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7710.771276 secs 2024-03-28T02:44:11.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:44:11 smithi165 ceph-mon[36302]: pgmap v4311: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:44:11.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:44:11 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7710.771276 secs 2024-03-28T02:44:11.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:44:11 smithi119 ceph-mon[31518]: pgmap v4311: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:44:11.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:44:11 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7710.771276 secs 2024-03-28T02:44:11.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:44:11 smithi132 ceph-mon[35941]: pgmap v4311: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:44:13.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:44:13 smithi165 ceph-mon[36302]: pgmap v4312: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:44:13.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:44:13 smithi119 ceph-mon[31518]: pgmap v4312: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:44:13.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:44:13 smithi132 ceph-mon[35941]: pgmap v4312: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:44:15.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:44:15 smithi165 ceph-mon[36302]: pgmap v4313: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-28T02:44:15.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:44:15 smithi119 ceph-mon[31518]: pgmap v4313: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-28T02:44:15.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:44:15 smithi132 ceph-mon[35941]: pgmap v4313: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-28T02:44:16.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:44:16 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7715.771315 secs 2024-03-28T02:44:16.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:44:16 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7715.771315 secs 2024-03-28T02:44:16.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:44:16 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7715.771315 secs 2024-03-28T02:44:17.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:44:17 smithi165 ceph-mon[36302]: pgmap v4314: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-03-28T02:44:17.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:44:17 smithi119 ceph-mon[31518]: pgmap v4314: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-03-28T02:44:17.701 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:44:17 smithi132 ceph-mon[35941]: pgmap v4314: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-03-28T02:44:19.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:44:19 smithi165 ceph-mon[36302]: pgmap v4315: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:44:19.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:44:19 smithi119 ceph-mon[31518]: pgmap v4315: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:44:19.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:44:19 smithi132 ceph-mon[35941]: pgmap v4315: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:44:21.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:44:21 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7720.771405 secs 2024-03-28T02:44:21.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:44:21 smithi165 ceph-mon[36302]: pgmap v4316: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:44:21.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:44:21 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7720.771405 secs 2024-03-28T02:44:21.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:44:21 smithi119 ceph-mon[31518]: pgmap v4316: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:44:21.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:44:21 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7720.771405 secs 2024-03-28T02:44:21.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:44:21 smithi132 ceph-mon[35941]: pgmap v4316: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:44:23.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:44:23 smithi165 ceph-mon[36302]: pgmap v4317: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:44:23.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:44:23 smithi119 ceph-mon[31518]: pgmap v4317: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:44:23.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:44:23 smithi132 ceph-mon[35941]: pgmap v4317: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:44:25.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:44:25 smithi165 ceph-mon[36302]: pgmap v4318: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-28T02:44:25.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:44:25 smithi119 ceph-mon[31518]: pgmap v4318: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-28T02:44:25.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:44:25 smithi132 ceph-mon[35941]: pgmap v4318: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-28T02:44:26.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:44:26 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7725.771495 secs 2024-03-28T02:44:26.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:44:26 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7725.771495 secs 2024-03-28T02:44:26.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:44:26 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7725.771495 secs 2024-03-28T02:44:27.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:44:27 smithi165 ceph-mon[36302]: pgmap v4319: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-03-28T02:44:27.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:44:27 smithi119 ceph-mon[31518]: pgmap v4319: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-03-28T02:44:27.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:44:27 smithi132 ceph-mon[35941]: pgmap v4319: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-03-28T02:44:29.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:44:29 smithi165 ceph-mon[36302]: pgmap v4320: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:44:29.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:44:29 smithi119 ceph-mon[31518]: pgmap v4320: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:44:29.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:44:29 smithi132 ceph-mon[35941]: pgmap v4320: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:44:31.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:44:31 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7730.771584 secs 2024-03-28T02:44:31.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:44:31 smithi165 ceph-mon[36302]: pgmap v4321: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:44:31.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:44:31 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7730.771584 secs 2024-03-28T02:44:31.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:44:31 smithi119 ceph-mon[31518]: pgmap v4321: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:44:31.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:44:31 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7730.771584 secs 2024-03-28T02:44:31.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:44:31 smithi132 ceph-mon[35941]: pgmap v4321: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:44:33.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:44:33 smithi165 ceph-mon[36302]: pgmap v4322: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:44:33.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:44:33 smithi119 ceph-mon[31518]: pgmap v4322: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:44:33.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:44:33 smithi132 ceph-mon[35941]: pgmap v4322: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:44:35.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:44:35 smithi165 ceph-mon[36302]: pgmap v4323: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-28T02:44:35.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:44:35 smithi119 ceph-mon[31518]: pgmap v4323: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-28T02:44:35.701 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:44:35 smithi132 ceph-mon[35941]: pgmap v4323: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-28T02:44:36.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:44:36 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7735.771642 secs 2024-03-28T02:44:36.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:44:36 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7735.771642 secs 2024-03-28T02:44:36.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:44:36 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7735.771642 secs 2024-03-28T02:44:37.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:44:37 smithi165 ceph-mon[36302]: pgmap v4324: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-03-28T02:44:37.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:44:37 smithi119 ceph-mon[31518]: pgmap v4324: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-03-28T02:44:37.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:44:37 smithi132 ceph-mon[35941]: pgmap v4324: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-03-28T02:44:39.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:44:39 smithi165 ceph-mon[36302]: pgmap v4325: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:44:39.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:44:39 smithi119 ceph-mon[31518]: pgmap v4325: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:44:39.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:44:39 smithi132 ceph-mon[35941]: pgmap v4325: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:44:41.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:44:41 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7740.771733 secs 2024-03-28T02:44:41.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:44:41 smithi165 ceph-mon[36302]: pgmap v4326: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:44:41.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:44:41 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7740.771733 secs 2024-03-28T02:44:41.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:44:41 smithi119 ceph-mon[31518]: pgmap v4326: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:44:41.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:44:41 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7740.771733 secs 2024-03-28T02:44:41.701 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:44:41 smithi132 ceph-mon[35941]: pgmap v4326: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:44:43.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:44:43 smithi165 ceph-mon[36302]: pgmap v4327: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:44:43.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:44:43 smithi119 ceph-mon[31518]: pgmap v4327: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:44:43.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:44:43 smithi132 ceph-mon[35941]: pgmap v4327: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:44:44.558 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:44:44 smithi119 ceph-mon[31518]: pgmap v4328: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-28T02:44:44.701 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:44:44 smithi132 ceph-mon[35941]: pgmap v4328: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-28T02:44:44.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:44:44 smithi165 ceph-mon[36302]: pgmap v4328: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-28T02:44:46.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:44:45 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7745.771821 secs 2024-03-28T02:44:46.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:44:45 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7745.771821 secs 2024-03-28T02:44:46.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:44:45 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7745.771821 secs 2024-03-28T02:44:47.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:44:46 smithi165 ceph-mon[36302]: pgmap v4329: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-03-28T02:44:47.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:44:46 smithi119 ceph-mon[31518]: pgmap v4329: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-03-28T02:44:47.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:44:46 smithi132 ceph-mon[35941]: pgmap v4329: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-03-28T02:44:49.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:44:49 smithi165 ceph-mon[36302]: pgmap v4330: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:44:49.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:44:49 smithi119 ceph-mon[31518]: pgmap v4330: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:44:49.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:44:49 smithi132 ceph-mon[35941]: pgmap v4330: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:44:51.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:44:51 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7750.771890 secs 2024-03-28T02:44:51.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:44:51 smithi165 ceph-mon[36302]: pgmap v4331: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:44:51.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:44:51 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7750.771890 secs 2024-03-28T02:44:51.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:44:51 smithi119 ceph-mon[31518]: pgmap v4331: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:44:51.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:44:51 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7750.771890 secs 2024-03-28T02:44:51.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:44:51 smithi132 ceph-mon[35941]: pgmap v4331: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:44:53.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:44:53 smithi165 ceph-mon[36302]: pgmap v4332: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:44:53.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:44:53 smithi119 ceph-mon[31518]: pgmap v4332: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:44:53.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:44:53 smithi132 ceph-mon[35941]: pgmap v4332: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:44:55.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:44:55 smithi165 ceph-mon[36302]: pgmap v4333: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-28T02:44:55.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:44:55 smithi119 ceph-mon[31518]: pgmap v4333: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-28T02:44:55.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:44:55 smithi132 ceph-mon[35941]: pgmap v4333: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-28T02:44:56.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:44:56 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7755.771951 secs 2024-03-28T02:44:56.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:44:56 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7755.771951 secs 2024-03-28T02:44:56.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:44:56 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7755.771951 secs 2024-03-28T02:44:57.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:44:57 smithi165 ceph-mon[36302]: pgmap v4334: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-03-28T02:44:57.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:44:57 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:44:57.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:44:57 smithi119 ceph-mon[31518]: pgmap v4334: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-03-28T02:44:57.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:44:57 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:44:57.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:44:57 smithi132 ceph-mon[35941]: pgmap v4334: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-03-28T02:44:57.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:44:57 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:44:58.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:44:58 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:44:58.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:44:58 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:44:58.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:44:58 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:44:58.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:44:58 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:44:58.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:44:58 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:44:58.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:44:58 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:44:58.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:44:58 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:44:58.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:44:58 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:44:58.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:44:58 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:44:59.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:44:59 smithi165 ceph-mon[36302]: pgmap v4335: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T02:44:59.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:44:59 smithi165 ceph-mon[36302]: pgmap v4336: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:44:59.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:44:59 smithi119 ceph-mon[31518]: pgmap v4335: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T02:44:59.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:44:59 smithi119 ceph-mon[31518]: pgmap v4336: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:44:59.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:44:59 smithi132 ceph-mon[35941]: pgmap v4335: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T02:44:59.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:44:59 smithi132 ceph-mon[35941]: pgmap v4336: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:45:01.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:45:01 smithi165 ceph-mon[36302]: pgmap v4337: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:45:01.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:45:01 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7760.772011 secs 2024-03-28T02:45:01.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:45:01 smithi119 ceph-mon[31518]: pgmap v4337: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:45:01.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:45:01 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7760.772011 secs 2024-03-28T02:45:01.701 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:45:01 smithi132 ceph-mon[35941]: pgmap v4337: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:45:01.701 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:45:01 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7760.772011 secs 2024-03-28T02:45:03.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:45:03 smithi165 ceph-mon[36302]: pgmap v4338: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:45:03.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:45:03 smithi119 ceph-mon[31518]: pgmap v4338: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:45:03.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:45:03 smithi132 ceph-mon[35941]: pgmap v4338: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:45:05.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:45:05 smithi165 ceph-mon[36302]: pgmap v4339: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 4 op/s 2024-03-28T02:45:05.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:45:05 smithi119 ceph-mon[31518]: pgmap v4339: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 4 op/s 2024-03-28T02:45:05.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:45:05 smithi132 ceph-mon[35941]: pgmap v4339: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 4 op/s 2024-03-28T02:45:06.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:45:06 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7765.772071 secs 2024-03-28T02:45:06.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:45:06 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7765.772071 secs 2024-03-28T02:45:06.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:45:06 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7765.772071 secs 2024-03-28T02:45:07.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:45:07 smithi165 ceph-mon[36302]: pgmap v4340: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:45:07.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:45:07 smithi119 ceph-mon[31518]: pgmap v4340: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:45:07.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:45:07 smithi132 ceph-mon[35941]: pgmap v4340: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:45:09.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:45:09 smithi165 ceph-mon[36302]: pgmap v4341: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T02:45:09.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:45:09 smithi119 ceph-mon[31518]: pgmap v4341: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T02:45:09.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:45:09 smithi132 ceph-mon[35941]: pgmap v4341: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T02:45:11.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:45:11 smithi165 ceph-mon[36302]: pgmap v4342: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:45:11.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:45:11 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7770.772134 secs 2024-03-28T02:45:11.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:45:11 smithi119 ceph-mon[31518]: pgmap v4342: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:45:11.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:45:11 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7770.772134 secs 2024-03-28T02:45:11.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:45:11 smithi132 ceph-mon[35941]: pgmap v4342: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:45:11.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:45:11 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7770.772134 secs 2024-03-28T02:45:13.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:45:13 smithi165 ceph-mon[36302]: pgmap v4343: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:45:13.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:45:13 smithi119 ceph-mon[31518]: pgmap v4343: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:45:13.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:45:13 smithi132 ceph-mon[35941]: pgmap v4343: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:45:15.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:45:15 smithi165 ceph-mon[36302]: pgmap v4344: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:45:15.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:45:15 smithi119 ceph-mon[31518]: pgmap v4344: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:45:15.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:45:15 smithi132 ceph-mon[35941]: pgmap v4344: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:45:16.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:45:16 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7775.772195 secs 2024-03-28T02:45:16.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:45:16 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7775.772195 secs 2024-03-28T02:45:16.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:45:16 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7775.772195 secs 2024-03-28T02:45:17.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:45:17 smithi165 ceph-mon[36302]: pgmap v4345: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:45:17.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:45:17 smithi119 ceph-mon[31518]: pgmap v4345: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:45:17.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:45:17 smithi132 ceph-mon[35941]: pgmap v4345: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:45:19.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:45:19 smithi165 ceph-mon[36302]: pgmap v4346: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:45:19.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:45:19 smithi119 ceph-mon[31518]: pgmap v4346: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:45:19.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:45:19 smithi132 ceph-mon[35941]: pgmap v4346: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:45:21.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:45:21 smithi165 ceph-mon[36302]: pgmap v4347: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:45:21.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:45:21 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7780.772257 secs 2024-03-28T02:45:21.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:45:21 smithi119 ceph-mon[31518]: pgmap v4347: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:45:21.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:45:21 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7780.772257 secs 2024-03-28T02:45:21.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:45:21 smithi132 ceph-mon[35941]: pgmap v4347: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:45:21.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:45:21 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7780.772257 secs 2024-03-28T02:45:23.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:45:23 smithi165 ceph-mon[36302]: pgmap v4348: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:45:23.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:45:23 smithi119 ceph-mon[31518]: pgmap v4348: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:45:23.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:45:23 smithi132 ceph-mon[35941]: pgmap v4348: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:45:25.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:45:25 smithi165 ceph-mon[36302]: pgmap v4349: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:45:25.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:45:25 smithi119 ceph-mon[31518]: pgmap v4349: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:45:25.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:45:25 smithi132 ceph-mon[35941]: pgmap v4349: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:45:26.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:45:26 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7785.772317 secs 2024-03-28T02:45:26.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:45:26 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7785.772317 secs 2024-03-28T02:45:26.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:45:26 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7785.772317 secs 2024-03-28T02:45:27.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:45:27 smithi165 ceph-mon[36302]: pgmap v4350: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:45:27.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:45:27 smithi119 ceph-mon[31518]: pgmap v4350: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:45:27.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:45:27 smithi132 ceph-mon[35941]: pgmap v4350: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:45:29.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:45:29 smithi165 ceph-mon[36302]: pgmap v4351: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:45:29.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:45:29 smithi119 ceph-mon[31518]: pgmap v4351: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:45:29.701 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:45:29 smithi132 ceph-mon[35941]: pgmap v4351: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:45:30.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:45:30 smithi165 ceph-mon[36302]: pgmap v4352: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:45:30.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:45:30 smithi119 ceph-mon[31518]: pgmap v4352: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:45:30.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:45:30 smithi132 ceph-mon[35941]: pgmap v4352: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:45:31.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:45:31 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7790.772377 secs 2024-03-28T02:45:31.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:45:31 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7790.772377 secs 2024-03-28T02:45:31.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:45:31 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7790.772377 secs 2024-03-28T02:45:32.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:45:32 smithi119 ceph-mon[31518]: pgmap v4353: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:45:32.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:45:32 smithi132 ceph-mon[35941]: pgmap v4353: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:45:32.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:45:32 smithi165 ceph-mon[36302]: pgmap v4353: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:45:35.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:45:34 smithi165 ceph-mon[36302]: pgmap v4354: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:45:35.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:45:34 smithi119 ceph-mon[31518]: pgmap v4354: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:45:35.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:45:34 smithi132 ceph-mon[35941]: pgmap v4354: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:45:36.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:45:35 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7795.772466 secs 2024-03-28T02:45:36.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:45:35 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7795.772466 secs 2024-03-28T02:45:36.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:45:35 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7795.772466 secs 2024-03-28T02:45:37.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:45:36 smithi165 ceph-mon[36302]: pgmap v4355: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:45:37.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:45:36 smithi119 ceph-mon[31518]: pgmap v4355: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:45:37.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:45:36 smithi132 ceph-mon[35941]: pgmap v4355: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:45:39.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:45:38 smithi165 ceph-mon[36302]: pgmap v4356: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:45:39.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:45:38 smithi119 ceph-mon[31518]: pgmap v4356: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:45:39.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:45:38 smithi132 ceph-mon[35941]: pgmap v4356: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:45:41.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:45:40 smithi165 ceph-mon[36302]: pgmap v4357: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:45:41.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:45:40 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7800.772527 secs 2024-03-28T02:45:41.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:45:40 smithi119 ceph-mon[31518]: pgmap v4357: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:45:41.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:45:40 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7800.772527 secs 2024-03-28T02:45:41.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:45:40 smithi132 ceph-mon[35941]: pgmap v4357: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:45:41.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:45:40 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7800.772527 secs 2024-03-28T02:45:43.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:45:42 smithi165 ceph-mon[36302]: pgmap v4358: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:45:43.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:45:42 smithi119 ceph-mon[31518]: pgmap v4358: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:45:43.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:45:42 smithi132 ceph-mon[35941]: pgmap v4358: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:45:45.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:45:44 smithi165 ceph-mon[36302]: pgmap v4359: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:45:45.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:45:44 smithi119 ceph-mon[31518]: pgmap v4359: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:45:45.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:45:44 smithi132 ceph-mon[35941]: pgmap v4359: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:45:46.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:45:45 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7805.772613 secs 2024-03-28T02:45:46.148 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:45:45 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7805.772613 secs 2024-03-28T02:45:46.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:45:45 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7805.772613 secs 2024-03-28T02:45:47.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:45:46 smithi165 ceph-mon[36302]: pgmap v4360: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:45:47.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:45:46 smithi119 ceph-mon[31518]: pgmap v4360: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:45:47.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:45:46 smithi132 ceph-mon[35941]: pgmap v4360: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:45:49.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:45:48 smithi165 ceph-mon[36302]: pgmap v4361: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:45:49.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:45:48 smithi119 ceph-mon[31518]: pgmap v4361: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:45:49.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:45:48 smithi132 ceph-mon[35941]: pgmap v4361: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:45:51.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:45:50 smithi165 ceph-mon[36302]: pgmap v4362: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:45:51.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:45:50 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7810.772676 secs 2024-03-28T02:45:51.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:45:50 smithi119 ceph-mon[31518]: pgmap v4362: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:45:51.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:45:50 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7810.772676 secs 2024-03-28T02:45:51.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:45:50 smithi132 ceph-mon[35941]: pgmap v4362: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:45:51.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:45:50 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7810.772676 secs 2024-03-28T02:45:53.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:45:52 smithi165 ceph-mon[36302]: pgmap v4363: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T02:45:53.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:45:52 smithi119 ceph-mon[31518]: pgmap v4363: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T02:45:53.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:45:52 smithi132 ceph-mon[35941]: pgmap v4363: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T02:45:55.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:45:54 smithi165 ceph-mon[36302]: pgmap v4364: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T02:45:55.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:45:54 smithi119 ceph-mon[31518]: pgmap v4364: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T02:45:55.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:45:54 smithi132 ceph-mon[35941]: pgmap v4364: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T02:45:56.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:45:55 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7815.772765 secs 2024-03-28T02:45:56.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:45:55 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7815.772765 secs 2024-03-28T02:45:56.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:45:55 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7815.772765 secs 2024-03-28T02:45:57.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:45:56 smithi119 ceph-mon[31518]: pgmap v4365: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:45:57.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:45:56 smithi132 ceph-mon[35941]: pgmap v4365: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:45:57.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:45:56 smithi165 ceph-mon[36302]: pgmap v4365: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:45:59.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:45:58 smithi119 ceph-mon[31518]: pgmap v4366: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T02:45:59.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:45:58 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:45:59.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:45:58 smithi132 ceph-mon[35941]: pgmap v4366: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T02:45:59.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:45:58 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:45:59.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:45:58 smithi165 ceph-mon[36302]: pgmap v4366: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T02:45:59.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:45:58 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:46:00.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:45:59 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:46:00.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:45:59 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:46:00.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:45:59 smithi119 ceph-mon[31518]: pgmap v4367: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 7 op/s 2024-03-28T02:46:00.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:45:59 smithi119 ceph-mon[31518]: pgmap v4368: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:46:00.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:45:59 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:46:00.205 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:45:59 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:46:00.205 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:45:59 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:46:00.206 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:45:59 smithi132 ceph-mon[35941]: pgmap v4367: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 7 op/s 2024-03-28T02:46:00.206 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:45:59 smithi132 ceph-mon[35941]: pgmap v4368: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:46:00.206 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:45:59 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:46:00.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:45:59 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:46:00.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:45:59 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:46:00.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:45:59 smithi165 ceph-mon[36302]: pgmap v4367: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 7 op/s 2024-03-28T02:46:00.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:45:59 smithi165 ceph-mon[36302]: pgmap v4368: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:46:00.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:45:59 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:46:01.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:46:00 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7820.772851 secs 2024-03-28T02:46:01.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:46:00 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7820.772851 secs 2024-03-28T02:46:01.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:46:00 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7820.772851 secs 2024-03-28T02:46:02.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:46:01 smithi132 ceph-mon[35941]: pgmap v4369: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:46:02.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:46:01 smithi165 ceph-mon[36302]: pgmap v4369: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:46:02.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:46:01 smithi119 ceph-mon[31518]: pgmap v4369: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:46:04.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:46:04 smithi165 ceph-mon[36302]: pgmap v4370: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:46:04.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:46:04 smithi119 ceph-mon[31518]: pgmap v4370: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:46:04.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:46:04 smithi132 ceph-mon[35941]: pgmap v4370: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:46:06.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:46:06 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7825.772912 secs 2024-03-28T02:46:06.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:46:06 smithi165 ceph-mon[36302]: pgmap v4371: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:46:06.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:46:06 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7825.772912 secs 2024-03-28T02:46:06.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:46:06 smithi119 ceph-mon[31518]: pgmap v4371: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:46:06.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:46:06 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7825.772912 secs 2024-03-28T02:46:06.701 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:46:06 smithi132 ceph-mon[35941]: pgmap v4371: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:46:08.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:46:08 smithi165 ceph-mon[36302]: pgmap v4372: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:46:08.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:46:08 smithi119 ceph-mon[31518]: pgmap v4372: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:46:08.701 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:46:08 smithi132 ceph-mon[35941]: pgmap v4372: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:46:10.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:46:10 smithi165 ceph-mon[36302]: pgmap v4373: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T02:46:10.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:46:10 smithi119 ceph-mon[31518]: pgmap v4373: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T02:46:10.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:46:10 smithi132 ceph-mon[35941]: pgmap v4373: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T02:46:11.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:46:11 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7830.772974 secs 2024-03-28T02:46:11.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:46:11 smithi119 ceph-mon[31518]: pgmap v4374: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:46:11.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:46:11 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7830.772974 secs 2024-03-28T02:46:11.701 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:46:11 smithi132 ceph-mon[35941]: pgmap v4374: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:46:11.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:46:11 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7830.772974 secs 2024-03-28T02:46:11.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:46:11 smithi165 ceph-mon[36302]: pgmap v4374: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:46:14.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:46:14 smithi165 ceph-mon[36302]: pgmap v4375: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:46:14.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:46:14 smithi119 ceph-mon[31518]: pgmap v4375: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:46:14.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:46:14 smithi132 ceph-mon[35941]: pgmap v4375: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:46:16.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:46:16 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7835.773035 secs 2024-03-28T02:46:16.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:46:16 smithi165 ceph-mon[36302]: pgmap v4376: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:46:16.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:46:16 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7835.773035 secs 2024-03-28T02:46:16.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:46:16 smithi119 ceph-mon[31518]: pgmap v4376: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:46:16.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:46:16 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7835.773035 secs 2024-03-28T02:46:16.701 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:46:16 smithi132 ceph-mon[35941]: pgmap v4376: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:46:18.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:46:18 smithi165 ceph-mon[36302]: pgmap v4377: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:46:18.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:46:18 smithi119 ceph-mon[31518]: pgmap v4377: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:46:18.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:46:18 smithi132 ceph-mon[35941]: pgmap v4377: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:46:19.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:46:19 smithi119 ceph-mon[31518]: pgmap v4378: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:46:19.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:46:19 smithi132 ceph-mon[35941]: pgmap v4378: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:46:19.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:46:19 smithi165 ceph-mon[36302]: pgmap v4378: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:46:21.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:46:20 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7840.773097 secs 2024-03-28T02:46:21.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:46:20 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7840.773097 secs 2024-03-28T02:46:21.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:46:20 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7840.773097 secs 2024-03-28T02:46:22.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:46:21 smithi165 ceph-mon[36302]: pgmap v4379: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:46:22.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:46:21 smithi119 ceph-mon[31518]: pgmap v4379: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:46:22.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:46:21 smithi132 ceph-mon[35941]: pgmap v4379: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:46:24.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:46:24 smithi165 ceph-mon[36302]: pgmap v4380: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:46:24.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:46:24 smithi119 ceph-mon[31518]: pgmap v4380: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:46:24.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:46:24 smithi132 ceph-mon[35941]: pgmap v4380: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:46:26.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:46:26 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7845.773168 secs 2024-03-28T02:46:26.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:46:26 smithi165 ceph-mon[36302]: pgmap v4381: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:46:26.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:46:26 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7845.773168 secs 2024-03-28T02:46:26.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:46:26 smithi119 ceph-mon[31518]: pgmap v4381: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:46:26.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:46:26 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7845.773168 secs 2024-03-28T02:46:26.701 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:46:26 smithi132 ceph-mon[35941]: pgmap v4381: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:46:27.701 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:46:27 smithi132 ceph-mon[35941]: pgmap v4382: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:46:27.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:46:27 smithi165 ceph-mon[36302]: pgmap v4382: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:46:27.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:46:27 smithi119 ceph-mon[31518]: pgmap v4382: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:46:30.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:46:30 smithi165 ceph-mon[36302]: pgmap v4383: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:46:30.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:46:30 smithi119 ceph-mon[31518]: pgmap v4383: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:46:30.701 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:46:30 smithi132 ceph-mon[35941]: pgmap v4383: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:46:31.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:46:31 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7850.773229 secs 2024-03-28T02:46:31.701 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:46:31 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7850.773229 secs 2024-03-28T02:46:31.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:46:31 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7850.773229 secs 2024-03-28T02:46:32.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:46:32 smithi119 ceph-mon[31518]: pgmap v4384: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:46:32.701 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:46:32 smithi132 ceph-mon[35941]: pgmap v4384: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:46:32.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:46:32 smithi165 ceph-mon[36302]: pgmap v4384: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:46:33.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:46:33 smithi119 ceph-mon[31518]: pgmap v4385: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:46:33.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:46:33 smithi132 ceph-mon[35941]: pgmap v4385: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:46:33.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:46:33 smithi165 ceph-mon[36302]: pgmap v4385: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:46:36.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:46:36 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7855.773319 secs 2024-03-28T02:46:36.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:46:36 smithi165 ceph-mon[36302]: pgmap v4386: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:46:36.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:46:36 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7855.773319 secs 2024-03-28T02:46:36.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:46:36 smithi119 ceph-mon[31518]: pgmap v4386: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:46:36.701 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:46:36 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7855.773319 secs 2024-03-28T02:46:36.701 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:46:36 smithi132 ceph-mon[35941]: pgmap v4386: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:46:37.701 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:46:37 smithi132 ceph-mon[35941]: pgmap v4387: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:46:37.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:46:37 smithi165 ceph-mon[36302]: pgmap v4387: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:46:37.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:46:37 smithi119 ceph-mon[31518]: pgmap v4387: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:46:40.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:46:40 smithi165 ceph-mon[36302]: pgmap v4388: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:46:40.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:46:40 smithi119 ceph-mon[31518]: pgmap v4388: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:46:40.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:46:40 smithi132 ceph-mon[35941]: pgmap v4388: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:46:41.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:46:41 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7860.773357 secs 2024-03-28T02:46:41.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:46:41 smithi119 ceph-mon[31518]: pgmap v4389: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:46:41.701 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:46:41 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7860.773357 secs 2024-03-28T02:46:41.701 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:46:41 smithi132 ceph-mon[35941]: pgmap v4389: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:46:41.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:46:41 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7860.773357 secs 2024-03-28T02:46:41.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:46:41 smithi165 ceph-mon[36302]: pgmap v4389: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:46:44.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:46:44 smithi165 ceph-mon[36302]: pgmap v4390: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:46:44.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:46:44 smithi119 ceph-mon[31518]: pgmap v4390: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:46:44.701 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:46:44 smithi132 ceph-mon[35941]: pgmap v4390: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:46:45.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:46:45 smithi119 ceph-mon[31518]: pgmap v4391: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:46:45.701 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:46:45 smithi132 ceph-mon[35941]: pgmap v4391: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:46:45.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:46:45 smithi165 ceph-mon[36302]: pgmap v4391: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:46:46.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:46:46 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7865.773417 secs 2024-03-28T02:46:46.701 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:46:46 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7865.773417 secs 2024-03-28T02:46:46.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:46:46 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7865.773417 secs 2024-03-28T02:46:47.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:46:47 smithi119 ceph-mon[31518]: pgmap v4392: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:46:47.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:46:47 smithi132 ceph-mon[35941]: pgmap v4392: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:46:47.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:46:47 smithi165 ceph-mon[36302]: pgmap v4392: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:46:49.701 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:46:49 smithi132 ceph-mon[35941]: pgmap v4393: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:46:49.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:46:49 smithi165 ceph-mon[36302]: pgmap v4393: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:46:49.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:46:49 smithi119 ceph-mon[31518]: pgmap v4393: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:46:51.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:46:50 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7870.773505 secs 2024-03-28T02:46:51.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:46:50 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7870.773505 secs 2024-03-28T02:46:51.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:46:50 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7870.773505 secs 2024-03-28T02:46:52.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:46:51 smithi165 ceph-mon[36302]: pgmap v4394: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:46:52.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:46:51 smithi119 ceph-mon[31518]: pgmap v4394: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:46:52.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:46:51 smithi132 ceph-mon[35941]: pgmap v4394: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:46:53.701 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:46:53 smithi132 ceph-mon[35941]: pgmap v4395: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:46:53.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:46:53 smithi165 ceph-mon[36302]: pgmap v4395: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:46:53.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:46:53 smithi119 ceph-mon[31518]: pgmap v4395: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:46:55.701 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:46:55 smithi132 ceph-mon[35941]: pgmap v4396: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:46:55.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:46:55 smithi165 ceph-mon[36302]: pgmap v4396: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:46:55.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:46:55 smithi119 ceph-mon[31518]: pgmap v4396: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:46:56.701 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:46:56 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7875.773594 secs 2024-03-28T02:46:56.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:46:56 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7875.773594 secs 2024-03-28T02:46:56.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:46:56 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7875.773594 secs 2024-03-28T02:46:57.701 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:46:57 smithi132 ceph-mon[35941]: pgmap v4397: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:46:57.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:46:57 smithi165 ceph-mon[36302]: pgmap v4397: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:46:57.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:46:57 smithi119 ceph-mon[31518]: pgmap v4397: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:46:59.701 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:46:59 smithi132 ceph-mon[35941]: pgmap v4398: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:46:59.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:46:59 smithi165 ceph-mon[36302]: pgmap v4398: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:46:59.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:46:59 smithi119 ceph-mon[31518]: pgmap v4398: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:47:00.701 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:47:00 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:47:00.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:47:00 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:47:00.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:47:00 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:47:01.701 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:47:01 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7880.773665 secs 2024-03-28T02:47:01.701 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:47:01 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:47:01.701 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:47:01 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:47:01.701 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:47:01 smithi132 ceph-mon[35941]: pgmap v4399: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T02:47:01.701 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:47:01 smithi132 ceph-mon[35941]: pgmap v4400: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:47:01.701 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:47:01 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:47:01.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:47:01 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7880.773665 secs 2024-03-28T02:47:01.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:47:01 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:47:01.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:47:01 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:47:01.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:47:01 smithi165 ceph-mon[36302]: pgmap v4399: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T02:47:01.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:47:01 smithi165 ceph-mon[36302]: pgmap v4400: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:47:01.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:47:01 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:47:01.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:47:01 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7880.773665 secs 2024-03-28T02:47:01.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:47:01 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:47:01.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:47:01 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:47:01.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:47:01 smithi119 ceph-mon[31518]: pgmap v4399: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T02:47:01.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:47:01 smithi119 ceph-mon[31518]: pgmap v4400: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:47:01.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:47:01 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:47:04.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:47:03 smithi165 ceph-mon[36302]: pgmap v4401: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:47:04.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:47:03 smithi119 ceph-mon[31518]: pgmap v4401: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:47:04.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:47:03 smithi132 ceph-mon[35941]: pgmap v4401: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:47:05.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:47:05 smithi165 ceph-mon[36302]: pgmap v4402: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:47:05.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:47:05 smithi119 ceph-mon[31518]: pgmap v4402: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:47:05.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:47:05 smithi132 ceph-mon[35941]: pgmap v4402: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:47:06.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:47:06 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7885.773726 secs 2024-03-28T02:47:06.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:47:06 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7885.773726 secs 2024-03-28T02:47:06.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:47:06 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7885.773726 secs 2024-03-28T02:47:07.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:47:07 smithi165 ceph-mon[36302]: pgmap v4403: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:47:07.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:47:07 smithi119 ceph-mon[31518]: pgmap v4403: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:47:07.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:47:07 smithi132 ceph-mon[35941]: pgmap v4403: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:47:10.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:47:09 smithi165 ceph-mon[36302]: pgmap v4404: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-28T02:47:10.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:47:09 smithi119 ceph-mon[31518]: pgmap v4404: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-28T02:47:10.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:47:09 smithi132 ceph-mon[35941]: pgmap v4404: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-28T02:47:11.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:47:10 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7890.773787 secs 2024-03-28T02:47:11.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:47:10 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7890.773787 secs 2024-03-28T02:47:11.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:47:10 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7890.773787 secs 2024-03-28T02:47:12.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:47:11 smithi119 ceph-mon[31518]: pgmap v4405: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T02:47:12.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:47:11 smithi132 ceph-mon[35941]: pgmap v4405: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T02:47:12.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:47:11 smithi165 ceph-mon[36302]: pgmap v4405: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T02:47:14.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:47:13 smithi119 ceph-mon[31518]: pgmap v4406: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:47:14.183 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:47:13 smithi165 ceph-mon[36302]: pgmap v4406: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:47:14.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:47:13 smithi132 ceph-mon[35941]: pgmap v4406: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:47:16.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:47:15 smithi119 ceph-mon[31518]: pgmap v4407: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:47:16.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:47:15 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7895.773847 secs 2024-03-28T02:47:16.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:47:15 smithi132 ceph-mon[35941]: pgmap v4407: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:47:16.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:47:15 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7895.773847 secs 2024-03-28T02:47:16.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:47:15 smithi165 ceph-mon[36302]: pgmap v4407: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:47:16.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:47:15 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7895.773847 secs 2024-03-28T02:47:18.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:47:17 smithi132 ceph-mon[35941]: pgmap v4408: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:47:18.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:47:17 smithi165 ceph-mon[36302]: pgmap v4408: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:47:18.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:47:17 smithi119 ceph-mon[31518]: pgmap v4408: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:47:20.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:47:19 smithi132 ceph-mon[35941]: pgmap v4409: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:47:20.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:47:19 smithi165 ceph-mon[36302]: pgmap v4409: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:47:20.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:47:19 smithi119 ceph-mon[31518]: pgmap v4409: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:47:21.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:47:20 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7900.773887 secs 2024-03-28T02:47:21.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:47:20 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7900.773887 secs 2024-03-28T02:47:21.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:47:20 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7900.773887 secs 2024-03-28T02:47:22.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:47:21 smithi132 ceph-mon[35941]: pgmap v4410: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:47:22.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:47:21 smithi165 ceph-mon[36302]: pgmap v4410: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:47:22.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:47:21 smithi119 ceph-mon[31518]: pgmap v4410: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:47:24.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:47:23 smithi132 ceph-mon[35941]: pgmap v4411: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:47:24.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:47:23 smithi165 ceph-mon[36302]: pgmap v4411: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:47:24.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:47:23 smithi119 ceph-mon[31518]: pgmap v4411: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:47:26.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:47:25 smithi132 ceph-mon[35941]: pgmap v4412: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:47:26.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:47:25 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7905.773951 secs 2024-03-28T02:47:26.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:47:25 smithi165 ceph-mon[36302]: pgmap v4412: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:47:26.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:47:25 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7905.773951 secs 2024-03-28T02:47:26.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:47:25 smithi119 ceph-mon[31518]: pgmap v4412: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:47:26.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:47:25 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7905.773951 secs 2024-03-28T02:47:28.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:47:27 smithi132 ceph-mon[35941]: pgmap v4413: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:47:28.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:47:27 smithi165 ceph-mon[36302]: pgmap v4413: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:47:28.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:47:27 smithi119 ceph-mon[31518]: pgmap v4413: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:47:30.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:47:29 smithi132 ceph-mon[35941]: pgmap v4414: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:47:30.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:47:29 smithi165 ceph-mon[36302]: pgmap v4414: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:47:30.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:47:29 smithi119 ceph-mon[31518]: pgmap v4414: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:47:31.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:47:30 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7910.774012 secs 2024-03-28T02:47:31.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:47:30 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7910.774012 secs 2024-03-28T02:47:31.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:47:30 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7910.774012 secs 2024-03-28T02:47:32.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:47:31 smithi132 ceph-mon[35941]: pgmap v4415: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:47:32.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:47:31 smithi165 ceph-mon[36302]: pgmap v4415: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:47:32.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:47:31 smithi119 ceph-mon[31518]: pgmap v4415: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:47:34.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:47:33 smithi165 ceph-mon[36302]: pgmap v4416: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:47:34.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:47:33 smithi119 ceph-mon[31518]: pgmap v4416: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:47:34.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:47:33 smithi132 ceph-mon[35941]: pgmap v4416: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:47:36.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:47:35 smithi165 ceph-mon[36302]: pgmap v4417: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:47:36.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:47:35 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7915.774076 secs 2024-03-28T02:47:36.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:47:35 smithi119 ceph-mon[31518]: pgmap v4417: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:47:36.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:47:35 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7915.774076 secs 2024-03-28T02:47:36.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:47:35 smithi132 ceph-mon[35941]: pgmap v4417: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:47:36.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:47:35 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7915.774076 secs 2024-03-28T02:47:38.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:47:37 smithi165 ceph-mon[36302]: pgmap v4418: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:47:38.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:47:37 smithi119 ceph-mon[31518]: pgmap v4418: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:47:38.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:47:37 smithi132 ceph-mon[35941]: pgmap v4418: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:47:40.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:47:39 smithi165 ceph-mon[36302]: pgmap v4419: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:47:40.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:47:39 smithi119 ceph-mon[31518]: pgmap v4419: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:47:40.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:47:39 smithi132 ceph-mon[35941]: pgmap v4419: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:47:41.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:47:40 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7920.774155 secs 2024-03-28T02:47:41.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:47:40 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7920.774155 secs 2024-03-28T02:47:41.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:47:40 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7920.774155 secs 2024-03-28T02:47:42.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:47:41 smithi165 ceph-mon[36302]: pgmap v4420: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:47:42.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:47:41 smithi119 ceph-mon[31518]: pgmap v4420: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:47:42.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:47:41 smithi132 ceph-mon[35941]: pgmap v4420: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:47:44.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:47:43 smithi165 ceph-mon[36302]: pgmap v4421: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:47:44.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:47:43 smithi119 ceph-mon[31518]: pgmap v4421: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:47:44.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:47:43 smithi132 ceph-mon[35941]: pgmap v4421: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:47:46.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:47:45 smithi165 ceph-mon[36302]: pgmap v4422: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:47:46.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:47:45 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7925.774222 secs 2024-03-28T02:47:46.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:47:45 smithi119 ceph-mon[31518]: pgmap v4422: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:47:46.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:47:45 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7925.774222 secs 2024-03-28T02:47:46.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:47:45 smithi132 ceph-mon[35941]: pgmap v4422: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:47:46.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:47:45 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7925.774222 secs 2024-03-28T02:47:48.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:47:47 smithi165 ceph-mon[36302]: pgmap v4423: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:47:48.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:47:47 smithi119 ceph-mon[31518]: pgmap v4423: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:47:48.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:47:47 smithi132 ceph-mon[35941]: pgmap v4423: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:47:50.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:47:50 smithi165 ceph-mon[36302]: pgmap v4424: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:47:50.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:47:50 smithi119 ceph-mon[31518]: pgmap v4424: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:47:50.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:47:50 smithi132 ceph-mon[35941]: pgmap v4424: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:47:51.306 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:47:51 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7930.774285 secs 2024-03-28T02:47:51.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:47:51 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7930.774285 secs 2024-03-28T02:47:51.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:47:51 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7930.774285 secs 2024-03-28T02:47:52.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:47:52 smithi165 ceph-mon[36302]: pgmap v4425: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:47:52.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:47:52 smithi119 ceph-mon[31518]: pgmap v4425: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:47:52.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:47:52 smithi132 ceph-mon[35941]: pgmap v4425: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:47:54.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:47:54 smithi165 ceph-mon[36302]: pgmap v4426: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:47:54.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:47:54 smithi119 ceph-mon[31518]: pgmap v4426: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:47:54.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:47:54 smithi132 ceph-mon[35941]: pgmap v4426: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:47:56.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:47:56 smithi165 ceph-mon[36302]: pgmap v4427: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:47:56.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:47:56 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7935.774375 secs 2024-03-28T02:47:56.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:47:56 smithi119 ceph-mon[31518]: pgmap v4427: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:47:56.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:47:56 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7935.774375 secs 2024-03-28T02:47:56.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:47:56 smithi132 ceph-mon[35941]: pgmap v4427: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:47:56.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:47:56 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7935.774375 secs 2024-03-28T02:47:58.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:47:58 smithi165 ceph-mon[36302]: pgmap v4428: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:47:58.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:47:58 smithi119 ceph-mon[31518]: pgmap v4428: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:47:58.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:47:58 smithi132 ceph-mon[35941]: pgmap v4428: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:48:00.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:48:00 smithi165 ceph-mon[36302]: pgmap v4429: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:48:00.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:48:00 smithi119 ceph-mon[31518]: pgmap v4429: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:48:00.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:48:00 smithi132 ceph-mon[35941]: pgmap v4429: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:48:01.352 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:48:01 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7940.774465 secs 2024-03-28T02:48:01.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:48:01 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7940.774465 secs 2024-03-28T02:48:01.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:48:01 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7940.774465 secs 2024-03-28T02:48:02.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:48:02 smithi165 ceph-mon[36302]: pgmap v4430: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:48:02.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:48:02 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:48:02.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:48:02 smithi119 ceph-mon[31518]: pgmap v4430: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:48:02.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:48:02 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:48:02.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:48:02 smithi132 ceph-mon[35941]: pgmap v4430: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:48:02.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:48:02 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:48:03.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:48:03 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:48:03.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:48:03 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:48:03.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:48:03 smithi165 ceph-mon[36302]: pgmap v4431: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T02:48:03.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:48:03 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:48:03.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:48:03 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:48:03.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:48:03 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:48:03.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:48:03 smithi119 ceph-mon[31518]: pgmap v4431: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T02:48:03.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:48:03 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:48:03.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:48:03 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:48:03.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:48:03 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:48:03.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:48:03 smithi132 ceph-mon[35941]: pgmap v4431: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T02:48:03.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:48:03 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:48:04.701 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:48:04 smithi132 ceph-mon[35941]: pgmap v4432: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T02:48:04.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:48:04 smithi165 ceph-mon[36302]: pgmap v4432: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T02:48:04.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:48:04 smithi119 ceph-mon[31518]: pgmap v4432: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T02:48:06.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:48:05 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7945.774544 secs 2024-03-28T02:48:06.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:48:05 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7945.774544 secs 2024-03-28T02:48:06.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:48:05 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7945.774544 secs 2024-03-28T02:48:07.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:48:06 smithi165 ceph-mon[36302]: pgmap v4433: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T02:48:07.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:48:06 smithi119 ceph-mon[31518]: pgmap v4433: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T02:48:07.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:48:06 smithi132 ceph-mon[35941]: pgmap v4433: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T02:48:08.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:48:08 smithi132 ceph-mon[35941]: pgmap v4434: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T02:48:08.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:48:08 smithi165 ceph-mon[36302]: pgmap v4434: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T02:48:08.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:48:08 smithi119 ceph-mon[31518]: pgmap v4434: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T02:48:10.701 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:48:10 smithi132 ceph-mon[35941]: pgmap v4435: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T02:48:10.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:48:10 smithi165 ceph-mon[36302]: pgmap v4435: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T02:48:10.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:48:10 smithi119 ceph-mon[31518]: pgmap v4435: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T02:48:11.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:48:11 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7950.774604 secs 2024-03-28T02:48:11.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:48:11 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7950.774604 secs 2024-03-28T02:48:11.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:48:11 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7950.774604 secs 2024-03-28T02:48:12.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:48:12 smithi165 ceph-mon[36302]: pgmap v4436: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T02:48:12.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:48:12 smithi119 ceph-mon[31518]: pgmap v4436: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T02:48:12.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:48:12 smithi132 ceph-mon[35941]: pgmap v4436: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T02:48:14.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:48:14 smithi165 ceph-mon[36302]: pgmap v4437: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:48:14.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:48:14 smithi119 ceph-mon[31518]: pgmap v4437: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:48:14.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:48:14 smithi132 ceph-mon[35941]: pgmap v4437: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:48:16.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:48:15 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7955.774664 secs 2024-03-28T02:48:16.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:48:15 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7955.774664 secs 2024-03-28T02:48:16.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:48:15 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7955.774664 secs 2024-03-28T02:48:17.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:48:16 smithi165 ceph-mon[36302]: pgmap v4438: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:48:17.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:48:16 smithi119 ceph-mon[31518]: pgmap v4438: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:48:17.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:48:16 smithi132 ceph-mon[35941]: pgmap v4438: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:48:18.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:48:18 smithi165 ceph-mon[36302]: pgmap v4439: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:48:18.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:48:18 smithi119 ceph-mon[31518]: pgmap v4439: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:48:18.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:48:18 smithi132 ceph-mon[35941]: pgmap v4439: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:48:20.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:48:20 smithi165 ceph-mon[36302]: pgmap v4440: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:48:20.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:48:20 smithi119 ceph-mon[31518]: pgmap v4440: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:48:20.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:48:20 smithi132 ceph-mon[35941]: pgmap v4440: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:48:21.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:48:21 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7960.774726 secs 2024-03-28T02:48:21.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:48:21 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7960.774726 secs 2024-03-28T02:48:21.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:48:21 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7960.774726 secs 2024-03-28T02:48:22.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:48:22 smithi165 ceph-mon[36302]: pgmap v4441: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:48:22.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:48:22 smithi119 ceph-mon[31518]: pgmap v4441: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:48:22.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:48:22 smithi132 ceph-mon[35941]: pgmap v4441: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:48:24.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:48:24 smithi165 ceph-mon[36302]: pgmap v4442: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:48:24.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:48:24 smithi119 ceph-mon[31518]: pgmap v4442: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:48:24.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:48:24 smithi132 ceph-mon[35941]: pgmap v4442: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:48:26.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:48:25 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7965.774786 secs 2024-03-28T02:48:26.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:48:25 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7965.774786 secs 2024-03-28T02:48:26.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:48:25 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7965.774786 secs 2024-03-28T02:48:27.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:48:26 smithi165 ceph-mon[36302]: pgmap v4443: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:48:27.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:48:26 smithi119 ceph-mon[31518]: pgmap v4443: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:48:27.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:48:26 smithi132 ceph-mon[35941]: pgmap v4443: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:48:28.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:48:28 smithi165 ceph-mon[36302]: pgmap v4444: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:48:28.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:48:28 smithi119 ceph-mon[31518]: pgmap v4444: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:48:28.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:48:28 smithi132 ceph-mon[35941]: pgmap v4444: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:48:30.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:48:30 smithi165 ceph-mon[36302]: pgmap v4445: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:48:30.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:48:30 smithi119 ceph-mon[31518]: pgmap v4445: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:48:30.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:48:30 smithi132 ceph-mon[35941]: pgmap v4445: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:48:31.852 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:48:31 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7970.774848 secs 2024-03-28T02:48:31.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:48:31 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7970.774848 secs 2024-03-28T02:48:31.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:48:31 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7970.774848 secs 2024-03-28T02:48:32.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:48:32 smithi165 ceph-mon[36302]: pgmap v4446: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:48:32.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:48:32 smithi119 ceph-mon[31518]: pgmap v4446: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:48:32.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:48:32 smithi132 ceph-mon[35941]: pgmap v4446: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:48:35.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:48:35 smithi165 ceph-mon[36302]: pgmap v4447: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:48:35.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:48:35 smithi119 ceph-mon[31518]: pgmap v4447: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:48:35.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:48:35 smithi132 ceph-mon[35941]: pgmap v4447: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:48:36.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:48:36 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7975.774909 secs 2024-03-28T02:48:36.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:48:36 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7975.774909 secs 2024-03-28T02:48:36.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:48:36 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7975.774909 secs 2024-03-28T02:48:37.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:48:37 smithi165 ceph-mon[36302]: pgmap v4448: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:48:37.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:48:37 smithi119 ceph-mon[31518]: pgmap v4448: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:48:37.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:48:37 smithi132 ceph-mon[35941]: pgmap v4448: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:48:39.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:48:39 smithi165 ceph-mon[36302]: pgmap v4449: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:48:39.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:48:39 smithi119 ceph-mon[31518]: pgmap v4449: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:48:39.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:48:39 smithi132 ceph-mon[35941]: pgmap v4449: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:48:41.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:48:41 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7980.774969 secs 2024-03-28T02:48:41.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:48:41 smithi165 ceph-mon[36302]: pgmap v4450: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:48:41.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:48:41 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7980.774969 secs 2024-03-28T02:48:41.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:48:41 smithi119 ceph-mon[31518]: pgmap v4450: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:48:41.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:48:41 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7980.774969 secs 2024-03-28T02:48:41.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:48:41 smithi132 ceph-mon[35941]: pgmap v4450: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:48:43.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:48:43 smithi165 ceph-mon[36302]: pgmap v4451: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:48:43.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:48:43 smithi119 ceph-mon[31518]: pgmap v4451: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:48:43.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:48:43 smithi132 ceph-mon[35941]: pgmap v4451: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:48:45.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:48:45 smithi165 ceph-mon[36302]: pgmap v4452: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:48:45.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:48:45 smithi119 ceph-mon[31518]: pgmap v4452: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:48:45.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:48:45 smithi132 ceph-mon[35941]: pgmap v4452: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:48:46.806 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:48:46 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7985.775056 secs 2024-03-28T02:48:46.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:48:46 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7985.775056 secs 2024-03-28T02:48:46.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:48:46 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7985.775056 secs 2024-03-28T02:48:47.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:48:47 smithi165 ceph-mon[36302]: pgmap v4453: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:48:47.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:48:47 smithi119 ceph-mon[31518]: pgmap v4453: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:48:47.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:48:47 smithi132 ceph-mon[35941]: pgmap v4453: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:48:49.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:48:49 smithi165 ceph-mon[36302]: pgmap v4454: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:48:49.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:48:49 smithi119 ceph-mon[31518]: pgmap v4454: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:48:49.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:48:49 smithi132 ceph-mon[35941]: pgmap v4454: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:48:51.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:48:51 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7990.775145 secs 2024-03-28T02:48:51.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:48:51 smithi165 ceph-mon[36302]: pgmap v4455: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:48:51.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:48:51 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7990.775145 secs 2024-03-28T02:48:51.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:48:51 smithi119 ceph-mon[31518]: pgmap v4455: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:48:51.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:48:51 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7990.775145 secs 2024-03-28T02:48:51.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:48:51 smithi132 ceph-mon[35941]: pgmap v4455: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:48:53.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:48:53 smithi165 ceph-mon[36302]: pgmap v4456: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:48:53.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:48:53 smithi119 ceph-mon[31518]: pgmap v4456: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:48:53.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:48:53 smithi132 ceph-mon[35941]: pgmap v4456: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:48:55.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:48:55 smithi165 ceph-mon[36302]: pgmap v4457: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:48:55.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:48:55 smithi119 ceph-mon[31518]: pgmap v4457: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:48:55.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:48:55 smithi132 ceph-mon[35941]: pgmap v4457: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:48:56.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:48:56 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 7995.775205 secs 2024-03-28T02:48:56.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:48:56 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 7995.775205 secs 2024-03-28T02:48:56.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:48:56 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 7995.775205 secs 2024-03-28T02:48:57.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:48:57 smithi165 ceph-mon[36302]: pgmap v4458: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:48:57.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:48:57 smithi119 ceph-mon[31518]: pgmap v4458: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:48:57.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:48:57 smithi132 ceph-mon[35941]: pgmap v4458: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:48:59.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:48:59 smithi165 ceph-mon[36302]: pgmap v4459: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:48:59.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:48:59 smithi119 ceph-mon[31518]: pgmap v4459: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:48:59.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:48:59 smithi132 ceph-mon[35941]: pgmap v4459: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:49:01.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:49:01 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8000.775245 secs 2024-03-28T02:49:01.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:49:01 smithi165 ceph-mon[36302]: pgmap v4460: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:49:01.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:49:01 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8000.775245 secs 2024-03-28T02:49:01.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:49:01 smithi119 ceph-mon[31518]: pgmap v4460: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:49:01.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:49:01 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8000.775245 secs 2024-03-28T02:49:01.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:49:01 smithi132 ceph-mon[35941]: pgmap v4460: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:49:03.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:49:03 smithi165 ceph-mon[36302]: pgmap v4461: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:49:03.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:49:03 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:49:03.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:49:03 smithi119 ceph-mon[31518]: pgmap v4461: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:49:03.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:49:03 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:49:03.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:49:03 smithi132 ceph-mon[35941]: pgmap v4461: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:49:03.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:49:03 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:49:04.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:49:04 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:49:04.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:49:04 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:49:04.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:49:04 smithi165 ceph-mon[36302]: pgmap v4462: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T02:49:04.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:49:04 smithi165 ceph-mon[36302]: pgmap v4463: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-28T02:49:04.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:49:04 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:49:04.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:49:04 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:49:04.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:49:04 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:49:04.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:49:04 smithi119 ceph-mon[31518]: pgmap v4462: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T02:49:04.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:49:04 smithi119 ceph-mon[31518]: pgmap v4463: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-28T02:49:04.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:49:04 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:49:04.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:49:04 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:49:04.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:49:04 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:49:04.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:49:04 smithi132 ceph-mon[35941]: pgmap v4462: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T02:49:04.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:49:04 smithi132 ceph-mon[35941]: pgmap v4463: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-28T02:49:04.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:49:04 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:49:06.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:49:05 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8005.775286 secs 2024-03-28T02:49:06.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:49:05 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8005.775286 secs 2024-03-28T02:49:06.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:49:05 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8005.775286 secs 2024-03-28T02:49:07.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:49:06 smithi165 ceph-mon[36302]: pgmap v4464: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:49:07.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:49:06 smithi119 ceph-mon[31518]: pgmap v4464: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:49:07.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:49:06 smithi132 ceph-mon[35941]: pgmap v4464: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:49:09.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:49:08 smithi165 ceph-mon[36302]: pgmap v4465: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:49:09.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:49:08 smithi119 ceph-mon[31518]: pgmap v4465: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:49:09.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:49:08 smithi132 ceph-mon[35941]: pgmap v4465: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:49:11.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:49:10 smithi165 ceph-mon[36302]: pgmap v4466: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:49:11.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:49:10 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8010.775347 secs 2024-03-28T02:49:11.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:49:10 smithi119 ceph-mon[31518]: pgmap v4466: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:49:11.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:49:10 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8010.775347 secs 2024-03-28T02:49:11.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:49:10 smithi132 ceph-mon[35941]: pgmap v4466: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:49:11.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:49:10 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8010.775347 secs 2024-03-28T02:49:13.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:49:12 smithi165 ceph-mon[36302]: pgmap v4467: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:49:13.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:49:12 smithi119 ceph-mon[31518]: pgmap v4467: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:49:13.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:49:12 smithi132 ceph-mon[35941]: pgmap v4467: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T02:49:15.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:49:14 smithi165 ceph-mon[36302]: pgmap v4468: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T02:49:15.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:49:14 smithi119 ceph-mon[31518]: pgmap v4468: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T02:49:15.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:49:14 smithi132 ceph-mon[35941]: pgmap v4468: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T02:49:16.306 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:49:15 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8015.775436 secs 2024-03-28T02:49:16.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:49:15 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8015.775436 secs 2024-03-28T02:49:16.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:49:15 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8015.775436 secs 2024-03-28T02:49:17.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:49:16 smithi165 ceph-mon[36302]: pgmap v4469: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:49:17.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:49:16 smithi119 ceph-mon[31518]: pgmap v4469: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:49:17.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:49:16 smithi132 ceph-mon[35941]: pgmap v4469: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:49:19.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:49:18 smithi165 ceph-mon[36302]: pgmap v4470: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:49:19.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:49:18 smithi119 ceph-mon[31518]: pgmap v4470: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:49:19.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:49:18 smithi132 ceph-mon[35941]: pgmap v4470: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:49:21.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:49:21 smithi165 ceph-mon[36302]: pgmap v4471: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:49:21.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:49:21 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8020.775527 secs 2024-03-28T02:49:21.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:49:21 smithi119 ceph-mon[31518]: pgmap v4471: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:49:21.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:49:21 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8020.775527 secs 2024-03-28T02:49:21.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:49:21 smithi132 ceph-mon[35941]: pgmap v4471: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:49:21.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:49:21 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8020.775527 secs 2024-03-28T02:49:23.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:49:23 smithi165 ceph-mon[36302]: pgmap v4472: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:49:23.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:49:23 smithi119 ceph-mon[31518]: pgmap v4472: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:49:23.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:49:23 smithi132 ceph-mon[35941]: pgmap v4472: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:49:25.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:49:25 smithi165 ceph-mon[36302]: pgmap v4473: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:49:25.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:49:25 smithi119 ceph-mon[31518]: pgmap v4473: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:49:25.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:49:25 smithi132 ceph-mon[35941]: pgmap v4473: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:49:26.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:49:26 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8025.775613 secs 2024-03-28T02:49:26.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:49:26 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8025.775613 secs 2024-03-28T02:49:26.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:49:26 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8025.775613 secs 2024-03-28T02:49:27.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:49:27 smithi165 ceph-mon[36302]: pgmap v4474: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:49:27.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:49:27 smithi119 ceph-mon[31518]: pgmap v4474: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:49:27.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:49:27 smithi132 ceph-mon[35941]: pgmap v4474: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:49:29.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:49:29 smithi165 ceph-mon[36302]: pgmap v4475: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:49:29.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:49:29 smithi119 ceph-mon[31518]: pgmap v4475: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:49:29.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:49:29 smithi132 ceph-mon[35941]: pgmap v4475: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:49:31.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:49:31 smithi165 ceph-mon[36302]: pgmap v4476: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:49:31.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:49:31 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8030.775702 secs 2024-03-28T02:49:31.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:49:31 smithi119 ceph-mon[31518]: pgmap v4476: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:49:31.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:49:31 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8030.775702 secs 2024-03-28T02:49:31.452 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:49:31 smithi132 ceph-mon[35941]: pgmap v4476: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:49:31.452 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:49:31 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8030.775702 secs 2024-03-28T02:49:33.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:49:33 smithi165 ceph-mon[36302]: pgmap v4477: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:49:33.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:49:33 smithi119 ceph-mon[31518]: pgmap v4477: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:49:33.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:49:33 smithi132 ceph-mon[35941]: pgmap v4477: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:49:35.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:49:35 smithi165 ceph-mon[36302]: pgmap v4478: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:49:35.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:49:35 smithi119 ceph-mon[31518]: pgmap v4478: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:49:35.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:49:35 smithi132 ceph-mon[35941]: pgmap v4478: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:49:36.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:49:36 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8035.775792 secs 2024-03-28T02:49:36.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:49:36 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8035.775792 secs 2024-03-28T02:49:36.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:49:36 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8035.775792 secs 2024-03-28T02:49:37.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:49:37 smithi165 ceph-mon[36302]: pgmap v4479: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:49:37.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:49:37 smithi119 ceph-mon[31518]: pgmap v4479: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:49:37.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:49:37 smithi132 ceph-mon[35941]: pgmap v4479: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:49:39.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:49:39 smithi165 ceph-mon[36302]: pgmap v4480: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:49:39.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:49:39 smithi119 ceph-mon[31518]: pgmap v4480: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:49:39.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:49:39 smithi132 ceph-mon[35941]: pgmap v4480: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:49:41.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:49:41 smithi165 ceph-mon[36302]: pgmap v4481: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:49:41.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:49:41 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8040.775885 secs 2024-03-28T02:49:41.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:49:41 smithi119 ceph-mon[31518]: pgmap v4481: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:49:41.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:49:41 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8040.775885 secs 2024-03-28T02:49:41.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:49:41 smithi132 ceph-mon[35941]: pgmap v4481: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:49:41.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:49:41 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8040.775885 secs 2024-03-28T02:49:43.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:49:43 smithi165 ceph-mon[36302]: pgmap v4482: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:49:43.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:49:43 smithi119 ceph-mon[31518]: pgmap v4482: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:49:43.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:49:43 smithi132 ceph-mon[35941]: pgmap v4482: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:49:45.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:49:45 smithi165 ceph-mon[36302]: pgmap v4483: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:49:45.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:49:45 smithi119 ceph-mon[31518]: pgmap v4483: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:49:45.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:49:45 smithi132 ceph-mon[35941]: pgmap v4483: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:49:46.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:49:46 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8045.775949 secs 2024-03-28T02:49:46.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:49:46 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8045.775949 secs 2024-03-28T02:49:46.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:49:46 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8045.775949 secs 2024-03-28T02:49:47.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:49:47 smithi165 ceph-mon[36302]: pgmap v4484: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:49:47.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:49:47 smithi119 ceph-mon[31518]: pgmap v4484: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:49:47.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:49:47 smithi132 ceph-mon[35941]: pgmap v4484: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:49:49.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:49:49 smithi165 ceph-mon[36302]: pgmap v4485: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:49:49.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:49:49 smithi119 ceph-mon[31518]: pgmap v4485: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:49:49.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:49:49 smithi132 ceph-mon[35941]: pgmap v4485: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:49:51.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:49:51 smithi165 ceph-mon[36302]: pgmap v4486: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:49:51.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:49:51 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8050.775991 secs 2024-03-28T02:49:51.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:49:51 smithi119 ceph-mon[31518]: pgmap v4486: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:49:51.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:49:51 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8050.775991 secs 2024-03-28T02:49:51.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:49:51 smithi132 ceph-mon[35941]: pgmap v4486: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:49:51.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:49:51 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8050.775991 secs 2024-03-28T02:49:53.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:49:53 smithi165 ceph-mon[36302]: pgmap v4487: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:49:53.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:49:53 smithi119 ceph-mon[31518]: pgmap v4487: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:49:53.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:49:53 smithi132 ceph-mon[35941]: pgmap v4487: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:49:55.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:49:55 smithi165 ceph-mon[36302]: pgmap v4488: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:49:55.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:49:55 smithi119 ceph-mon[31518]: pgmap v4488: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:49:55.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:49:55 smithi132 ceph-mon[35941]: pgmap v4488: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:49:56.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:49:56 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8055.776055 secs 2024-03-28T02:49:56.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:49:56 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8055.776055 secs 2024-03-28T02:49:56.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:49:56 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8055.776055 secs 2024-03-28T02:49:57.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:49:57 smithi165 ceph-mon[36302]: pgmap v4489: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:49:57.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:49:57 smithi119 ceph-mon[31518]: pgmap v4489: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:49:57.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:49:57 smithi132 ceph-mon[35941]: pgmap v4489: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:49:59.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:49:59 smithi165 ceph-mon[36302]: pgmap v4490: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:49:59.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:49:59 smithi119 ceph-mon[31518]: pgmap v4490: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:49:59.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:49:59 smithi132 ceph-mon[35941]: pgmap v4490: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:50:00.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:50:00 smithi119 ceph-mon[31518]: overall HEALTH_WARN 5 failed cephadm daemon(s); 1 filesystem is degraded; 1 MDSs report slow requests 2024-03-28T02:50:00.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:50:00 smithi132 ceph-mon[35941]: overall HEALTH_WARN 5 failed cephadm daemon(s); 1 filesystem is degraded; 1 MDSs report slow requests 2024-03-28T02:50:00.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:50:00 smithi165 ceph-mon[36302]: overall HEALTH_WARN 5 failed cephadm daemon(s); 1 filesystem is degraded; 1 MDSs report slow requests 2024-03-28T02:50:01.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:50:01 smithi119 ceph-mon[31518]: pgmap v4491: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:50:01.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:50:01 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8060.776118 secs 2024-03-28T02:50:01.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:50:01 smithi132 ceph-mon[35941]: pgmap v4491: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:50:01.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:50:01 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8060.776118 secs 2024-03-28T02:50:01.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:50:01 smithi165 ceph-mon[36302]: pgmap v4491: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:50:01.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:50:01 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8060.776118 secs 2024-03-28T02:50:03.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:50:03 smithi119 ceph-mon[31518]: pgmap v4492: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:50:03.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:50:03 smithi132 ceph-mon[35941]: pgmap v4492: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:50:03.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:50:03 smithi165 ceph-mon[36302]: pgmap v4492: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:50:05.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:50:05 smithi119 ceph-mon[31518]: pgmap v4493: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:50:05.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:50:05 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:50:05.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:50:05 smithi132 ceph-mon[35941]: pgmap v4493: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:50:05.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:50:05 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:50:05.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:50:05 smithi165 ceph-mon[36302]: pgmap v4493: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:50:05.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:50:05 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:50:06.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:50:06 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8065.776182 secs 2024-03-28T02:50:06.452 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:50:06 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:50:06.452 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:50:06 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:50:06.452 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:50:06 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:50:06.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:50:06 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8065.776182 secs 2024-03-28T02:50:06.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:50:06 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:50:06.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:50:06 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:50:06.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:50:06 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:50:06.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:50:06 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8065.776182 secs 2024-03-28T02:50:06.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:50:06 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:50:06.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:50:06 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:50:06.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:50:06 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:50:07.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:50:07 smithi132 ceph-mon[35941]: pgmap v4494: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T02:50:07.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:50:07 smithi132 ceph-mon[35941]: pgmap v4495: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:50:07.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:50:07 smithi165 ceph-mon[36302]: pgmap v4494: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T02:50:07.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:50:07 smithi165 ceph-mon[36302]: pgmap v4495: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:50:07.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:50:07 smithi119 ceph-mon[31518]: pgmap v4494: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T02:50:07.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:50:07 smithi119 ceph-mon[31518]: pgmap v4495: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:50:09.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:50:09 smithi132 ceph-mon[35941]: pgmap v4496: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:50:09.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:50:09 smithi165 ceph-mon[36302]: pgmap v4496: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:50:09.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:50:09 smithi119 ceph-mon[31518]: pgmap v4496: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:50:11.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:50:11 smithi132 ceph-mon[35941]: pgmap v4497: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:50:11.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:50:11 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8070.776267 secs 2024-03-28T02:50:11.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:50:11 smithi165 ceph-mon[36302]: pgmap v4497: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:50:11.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:50:11 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8070.776267 secs 2024-03-28T02:50:11.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:50:11 smithi119 ceph-mon[31518]: pgmap v4497: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:50:11.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:50:11 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8070.776267 secs 2024-03-28T02:50:13.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:50:13 smithi132 ceph-mon[35941]: pgmap v4498: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:50:13.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:50:13 smithi165 ceph-mon[36302]: pgmap v4498: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:50:13.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:50:13 smithi119 ceph-mon[31518]: pgmap v4498: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:50:15.433 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:50:15 smithi165 ceph-mon[36302]: pgmap v4499: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:50:15.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:50:15 smithi132 ceph-mon[35941]: pgmap v4499: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:50:15.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:50:15 smithi119 ceph-mon[31518]: pgmap v4499: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:50:16.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:50:16 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8075.776331 secs 2024-03-28T02:50:16.556 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:50:16 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8075.776331 secs 2024-03-28T02:50:16.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:50:16 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8075.776331 secs 2024-03-28T02:50:17.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:50:17 smithi132 ceph-mon[35941]: pgmap v4500: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T02:50:17.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:50:17 smithi165 ceph-mon[36302]: pgmap v4500: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T02:50:17.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:50:17 smithi119 ceph-mon[31518]: pgmap v4500: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T02:50:19.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:50:19 smithi132 ceph-mon[35941]: pgmap v4501: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:50:19.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:50:19 smithi165 ceph-mon[36302]: pgmap v4501: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:50:19.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:50:19 smithi119 ceph-mon[31518]: pgmap v4501: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:50:21.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:50:21 smithi132 ceph-mon[35941]: pgmap v4502: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:50:21.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:50:21 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8080.776389 secs 2024-03-28T02:50:21.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:50:21 smithi165 ceph-mon[36302]: pgmap v4502: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:50:21.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:50:21 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8080.776389 secs 2024-03-28T02:50:21.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:50:21 smithi119 ceph-mon[31518]: pgmap v4502: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:50:21.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:50:21 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8080.776389 secs 2024-03-28T02:50:23.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:50:23 smithi165 ceph-mon[36302]: pgmap v4503: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:50:23.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:50:23 smithi119 ceph-mon[31518]: pgmap v4503: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:50:23.701 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:50:23 smithi132 ceph-mon[35941]: pgmap v4503: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:50:25.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:50:25 smithi165 ceph-mon[36302]: pgmap v4504: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:50:25.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:50:25 smithi119 ceph-mon[31518]: pgmap v4504: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:50:25.701 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:50:25 smithi132 ceph-mon[35941]: pgmap v4504: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:50:26.556 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:50:26 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8085.776433 secs 2024-03-28T02:50:26.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:50:26 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8085.776433 secs 2024-03-28T02:50:26.701 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:50:26 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8085.776433 secs 2024-03-28T02:50:27.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:50:27 smithi165 ceph-mon[36302]: pgmap v4505: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:50:27.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:50:27 smithi119 ceph-mon[31518]: pgmap v4505: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:50:27.701 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:50:27 smithi132 ceph-mon[35941]: pgmap v4505: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:50:29.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:50:29 smithi165 ceph-mon[36302]: pgmap v4506: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T02:50:29.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:50:29 smithi119 ceph-mon[31518]: pgmap v4506: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T02:50:29.701 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:50:29 smithi132 ceph-mon[35941]: pgmap v4506: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T02:50:31.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:50:31 smithi165 ceph-mon[36302]: pgmap v4507: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:50:31.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:50:31 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8090.776523 secs 2024-03-28T02:50:31.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:50:31 smithi119 ceph-mon[31518]: pgmap v4507: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:50:31.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:50:31 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8090.776523 secs 2024-03-28T02:50:31.701 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:50:31 smithi132 ceph-mon[35941]: pgmap v4507: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:50:31.701 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:50:31 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8090.776523 secs 2024-03-28T02:50:33.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:50:33 smithi165 ceph-mon[36302]: pgmap v4508: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T02:50:33.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:50:33 smithi119 ceph-mon[31518]: pgmap v4508: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T02:50:33.701 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:50:33 smithi132 ceph-mon[35941]: pgmap v4508: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T02:50:35.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:50:35 smithi165 ceph-mon[36302]: pgmap v4509: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T02:50:35.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:50:35 smithi119 ceph-mon[31518]: pgmap v4509: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T02:50:35.701 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:50:35 smithi132 ceph-mon[35941]: pgmap v4509: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T02:50:36.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:50:36 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8095.776616 secs 2024-03-28T02:50:36.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:50:36 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8095.776616 secs 2024-03-28T02:50:36.701 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:50:36 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8095.776616 secs 2024-03-28T02:50:37.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:50:37 smithi165 ceph-mon[36302]: pgmap v4510: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:50:37.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:50:37 smithi119 ceph-mon[31518]: pgmap v4510: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:50:37.701 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:50:37 smithi132 ceph-mon[35941]: pgmap v4510: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:50:39.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:50:39 smithi165 ceph-mon[36302]: pgmap v4511: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:50:39.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:50:39 smithi119 ceph-mon[31518]: pgmap v4511: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:50:39.701 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:50:39 smithi132 ceph-mon[35941]: pgmap v4511: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:50:41.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:50:41 smithi165 ceph-mon[36302]: pgmap v4512: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:50:41.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:50:41 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8100.776705 secs 2024-03-28T02:50:41.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:50:41 smithi119 ceph-mon[31518]: pgmap v4512: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:50:41.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:50:41 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8100.776705 secs 2024-03-28T02:50:41.701 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:50:41 smithi132 ceph-mon[35941]: pgmap v4512: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:50:41.701 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:50:41 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8100.776705 secs 2024-03-28T02:50:43.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:50:43 smithi165 ceph-mon[36302]: pgmap v4513: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:50:43.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:50:43 smithi119 ceph-mon[31518]: pgmap v4513: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:50:43.701 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:50:43 smithi132 ceph-mon[35941]: pgmap v4513: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:50:45.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:50:45 smithi165 ceph-mon[36302]: pgmap v4514: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:50:45.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:50:45 smithi119 ceph-mon[31518]: pgmap v4514: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:50:45.701 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:50:45 smithi132 ceph-mon[35941]: pgmap v4514: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:50:46.556 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:50:46 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8105.776793 secs 2024-03-28T02:50:46.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:50:46 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8105.776793 secs 2024-03-28T02:50:46.701 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:50:46 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8105.776793 secs 2024-03-28T02:50:47.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:50:47 smithi165 ceph-mon[36302]: pgmap v4515: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:50:47.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:50:47 smithi119 ceph-mon[31518]: pgmap v4515: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:50:47.701 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:50:47 smithi132 ceph-mon[35941]: pgmap v4515: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:50:49.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:50:49 smithi165 ceph-mon[36302]: pgmap v4516: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:50:49.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:50:49 smithi119 ceph-mon[31518]: pgmap v4516: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:50:49.701 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:50:49 smithi132 ceph-mon[35941]: pgmap v4516: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:50:51.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:50:51 smithi165 ceph-mon[36302]: pgmap v4517: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:50:51.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:50:51 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8110.776881 secs 2024-03-28T02:50:51.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:50:51 smithi119 ceph-mon[31518]: pgmap v4517: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:50:51.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:50:51 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8110.776881 secs 2024-03-28T02:50:51.701 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:50:51 smithi132 ceph-mon[35941]: pgmap v4517: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:50:51.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:50:51 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8110.776881 secs 2024-03-28T02:50:53.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:50:53 smithi165 ceph-mon[36302]: pgmap v4518: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:50:53.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:50:53 smithi119 ceph-mon[31518]: pgmap v4518: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:50:53.701 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:50:53 smithi132 ceph-mon[35941]: pgmap v4518: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:50:55.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:50:55 smithi165 ceph-mon[36302]: pgmap v4519: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:50:55.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:50:55 smithi119 ceph-mon[31518]: pgmap v4519: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:50:55.701 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:50:55 smithi132 ceph-mon[35941]: pgmap v4519: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:50:56.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:50:56 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8115.776943 secs 2024-03-28T02:50:56.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:50:56 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8115.776943 secs 2024-03-28T02:50:56.701 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:50:56 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8115.776943 secs 2024-03-28T02:50:57.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:50:57 smithi165 ceph-mon[36302]: pgmap v4520: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:50:57.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:50:57 smithi119 ceph-mon[31518]: pgmap v4520: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:50:57.701 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:50:57 smithi132 ceph-mon[35941]: pgmap v4520: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:50:59.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:50:59 smithi165 ceph-mon[36302]: pgmap v4521: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:50:59.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:50:59 smithi119 ceph-mon[31518]: pgmap v4521: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:50:59.701 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:50:59 smithi132 ceph-mon[35941]: pgmap v4521: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:51:01.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:51:01 smithi165 ceph-mon[36302]: pgmap v4522: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:51:01.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:51:01 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8120.777002 secs 2024-03-28T02:51:01.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:51:01 smithi119 ceph-mon[31518]: pgmap v4522: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:51:01.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:51:01 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8120.777002 secs 2024-03-28T02:51:01.701 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:51:01 smithi132 ceph-mon[35941]: pgmap v4522: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:51:01.701 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:51:01 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8120.777002 secs 2024-03-28T02:51:03.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:51:03 smithi165 ceph-mon[36302]: pgmap v4523: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:51:03.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:51:03 smithi119 ceph-mon[31518]: pgmap v4523: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:51:03.701 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:51:03 smithi132 ceph-mon[35941]: pgmap v4523: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:51:05.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:51:05 smithi165 ceph-mon[36302]: pgmap v4524: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:51:05.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:51:05 smithi119 ceph-mon[31518]: pgmap v4524: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:51:05.701 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:51:05 smithi132 ceph-mon[35941]: pgmap v4524: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:51:06.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:51:06 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8125.777062 secs 2024-03-28T02:51:06.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:51:06 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:51:06.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:51:06 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8125.777062 secs 2024-03-28T02:51:06.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:51:06 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:51:06.701 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:51:06 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8125.777062 secs 2024-03-28T02:51:06.701 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:51:06 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:51:07.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:51:07 smithi165 ceph-mon[36302]: pgmap v4525: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:51:07.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:51:07 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:51:07.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:51:07 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:51:07.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:51:07 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:51:07.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:51:07 smithi119 ceph-mon[31518]: pgmap v4525: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:51:07.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:51:07 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:51:07.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:51:07 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:51:07.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:51:07 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:51:07.701 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:51:07 smithi132 ceph-mon[35941]: pgmap v4525: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:51:07.701 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:51:07 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:51:07.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:51:07 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:51:07.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:51:07 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:51:08.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:51:08 smithi165 ceph-mon[36302]: pgmap v4526: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T02:51:08.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:51:08 smithi165 ceph-mon[36302]: pgmap v4527: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:51:08.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:51:08 smithi119 ceph-mon[31518]: pgmap v4526: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T02:51:08.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:51:08 smithi119 ceph-mon[31518]: pgmap v4527: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:51:08.701 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:51:08 smithi132 ceph-mon[35941]: pgmap v4526: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T02:51:08.701 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:51:08 smithi132 ceph-mon[35941]: pgmap v4527: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:51:10.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:51:10 smithi165 ceph-mon[36302]: pgmap v4528: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-28T02:51:10.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:51:10 smithi119 ceph-mon[31518]: pgmap v4528: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-28T02:51:10.701 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:51:10 smithi132 ceph-mon[35941]: pgmap v4528: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-28T02:51:11.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:51:11 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8130.777144 secs 2024-03-28T02:51:11.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:51:11 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8130.777144 secs 2024-03-28T02:51:11.701 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:51:11 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8130.777144 secs 2024-03-28T02:51:12.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:51:12 smithi165 ceph-mon[36302]: pgmap v4529: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:51:12.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:51:12 smithi119 ceph-mon[31518]: pgmap v4529: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:51:12.701 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:51:12 smithi132 ceph-mon[35941]: pgmap v4529: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:51:14.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:51:14 smithi165 ceph-mon[36302]: pgmap v4530: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:51:14.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:51:14 smithi119 ceph-mon[31518]: pgmap v4530: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:51:14.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:51:14 smithi132 ceph-mon[35941]: pgmap v4530: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:51:16.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:51:16 smithi119 ceph-mon[31518]: pgmap v4531: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:51:16.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:51:16 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8135.777232 secs 2024-03-28T02:51:16.701 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:51:16 smithi132 ceph-mon[35941]: pgmap v4531: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:51:16.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:51:16 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8135.777232 secs 2024-03-28T02:51:16.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:51:16 smithi165 ceph-mon[36302]: pgmap v4531: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:51:16.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:51:16 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8135.777232 secs 2024-03-28T02:51:18.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:51:18 smithi119 ceph-mon[31518]: pgmap v4532: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T02:51:18.701 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:51:18 smithi132 ceph-mon[35941]: pgmap v4532: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T02:51:18.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:51:18 smithi165 ceph-mon[36302]: pgmap v4532: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T02:51:20.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:51:20 smithi119 ceph-mon[31518]: pgmap v4533: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:51:20.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:51:20 smithi132 ceph-mon[35941]: pgmap v4533: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:51:20.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:51:20 smithi165 ceph-mon[36302]: pgmap v4533: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:51:21.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:51:21 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8140.777320 secs 2024-03-28T02:51:21.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:51:21 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8140.777320 secs 2024-03-28T02:51:21.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:51:21 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8140.777320 secs 2024-03-28T02:51:22.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:51:22 smithi132 ceph-mon[35941]: pgmap v4534: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:51:22.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:51:22 smithi165 ceph-mon[36302]: pgmap v4534: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:51:22.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:51:22 smithi119 ceph-mon[31518]: pgmap v4534: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:51:24.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:51:24 smithi132 ceph-mon[35941]: pgmap v4535: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:51:24.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:51:24 smithi165 ceph-mon[36302]: pgmap v4535: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:51:24.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:51:24 smithi119 ceph-mon[31518]: pgmap v4535: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:51:26.701 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:51:26 smithi132 ceph-mon[35941]: pgmap v4536: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:51:26.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:51:26 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8145.777409 secs 2024-03-28T02:51:26.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:51:26 smithi165 ceph-mon[36302]: pgmap v4536: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:51:26.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:51:26 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8145.777409 secs 2024-03-28T02:51:26.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:51:26 smithi119 ceph-mon[31518]: pgmap v4536: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:51:26.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:51:26 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8145.777409 secs 2024-03-28T02:51:28.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:51:28 smithi132 ceph-mon[35941]: pgmap v4537: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:51:28.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:51:28 smithi165 ceph-mon[36302]: pgmap v4537: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:51:28.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:51:28 smithi119 ceph-mon[31518]: pgmap v4537: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:51:30.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:51:30 smithi132 ceph-mon[35941]: pgmap v4538: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:51:30.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:51:30 smithi165 ceph-mon[36302]: pgmap v4538: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:51:30.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:51:30 smithi119 ceph-mon[31518]: pgmap v4538: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:51:31.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:51:31 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8150.777500 secs 2024-03-28T02:51:31.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:51:31 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8150.777500 secs 2024-03-28T02:51:31.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:51:31 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8150.777500 secs 2024-03-28T02:51:32.701 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:51:32 smithi132 ceph-mon[35941]: pgmap v4539: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:51:32.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:51:32 smithi165 ceph-mon[36302]: pgmap v4539: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:51:32.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:51:32 smithi119 ceph-mon[31518]: pgmap v4539: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:51:33.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:51:33 smithi165 ceph-mon[36302]: pgmap v4540: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:51:33.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:51:33 smithi119 ceph-mon[31518]: pgmap v4540: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:51:33.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:51:33 smithi132 ceph-mon[35941]: pgmap v4540: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:51:36.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:51:36 smithi165 ceph-mon[36302]: pgmap v4541: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:51:36.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:51:36 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8155.777541 secs 2024-03-28T02:51:36.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:51:36 smithi119 ceph-mon[31518]: pgmap v4541: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:51:36.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:51:36 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8155.777541 secs 2024-03-28T02:51:36.452 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:51:36 smithi132 ceph-mon[35941]: pgmap v4541: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:51:36.452 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:51:36 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8155.777541 secs 2024-03-28T02:51:38.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:51:38 smithi165 ceph-mon[36302]: pgmap v4542: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:51:38.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:51:38 smithi119 ceph-mon[31518]: pgmap v4542: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:51:38.452 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:51:38 smithi132 ceph-mon[35941]: pgmap v4542: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:51:40.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:51:40 smithi165 ceph-mon[36302]: pgmap v4543: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:51:40.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:51:40 smithi119 ceph-mon[31518]: pgmap v4543: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:51:40.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:51:40 smithi132 ceph-mon[35941]: pgmap v4543: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:51:41.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:51:41 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8160.777628 secs 2024-03-28T02:51:41.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:51:41 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8160.777628 secs 2024-03-28T02:51:41.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:51:41 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8160.777628 secs 2024-03-28T02:51:42.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:51:42 smithi119 ceph-mon[31518]: pgmap v4544: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:51:42.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:51:42 smithi132 ceph-mon[35941]: pgmap v4544: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:51:42.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:51:42 smithi165 ceph-mon[36302]: pgmap v4544: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:51:44.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:51:44 smithi119 ceph-mon[31518]: pgmap v4545: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:51:44.452 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:51:44 smithi132 ceph-mon[35941]: pgmap v4545: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:51:44.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:51:44 smithi165 ceph-mon[36302]: pgmap v4545: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:51:46.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:51:46 smithi119 ceph-mon[31518]: pgmap v4546: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:51:46.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:51:46 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8165.777716 secs 2024-03-28T02:51:46.433 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:51:46 smithi165 ceph-mon[36302]: pgmap v4546: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:51:46.434 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:51:46 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8165.777716 secs 2024-03-28T02:51:46.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:51:46 smithi132 ceph-mon[35941]: pgmap v4546: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:51:46.452 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:51:46 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8165.777716 secs 2024-03-28T02:51:48.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:51:48 smithi119 ceph-mon[31518]: pgmap v4547: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:51:48.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:51:48 smithi132 ceph-mon[35941]: pgmap v4547: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:51:48.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:51:48 smithi165 ceph-mon[36302]: pgmap v4547: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:51:50.452 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:51:50 smithi132 ceph-mon[35941]: pgmap v4548: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T02:51:50.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:51:50 smithi165 ceph-mon[36302]: pgmap v4548: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T02:51:50.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:51:50 smithi119 ceph-mon[31518]: pgmap v4548: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T02:51:51.452 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:51:51 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8170.777805 secs 2024-03-28T02:51:51.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:51:51 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8170.777805 secs 2024-03-28T02:51:51.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:51:51 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8170.777805 secs 2024-03-28T02:51:52.452 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:51:52 smithi132 ceph-mon[35941]: pgmap v4549: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:51:52.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:51:52 smithi165 ceph-mon[36302]: pgmap v4549: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:51:52.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:51:52 smithi119 ceph-mon[31518]: pgmap v4549: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:51:54.452 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:51:54 smithi132 ceph-mon[35941]: pgmap v4550: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:51:54.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:51:54 smithi165 ceph-mon[36302]: pgmap v4550: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:51:54.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:51:54 smithi119 ceph-mon[31518]: pgmap v4550: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:51:56.454 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:51:56 smithi132 ceph-mon[35941]: pgmap v4551: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:51:56.454 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:51:56 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8175.777869 secs 2024-03-28T02:51:56.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:51:56 smithi165 ceph-mon[36302]: pgmap v4551: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:51:56.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:51:56 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8175.777869 secs 2024-03-28T02:51:56.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:51:56 smithi119 ceph-mon[31518]: pgmap v4551: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:51:56.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:51:56 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8175.777869 secs 2024-03-28T02:51:58.452 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:51:58 smithi132 ceph-mon[35941]: pgmap v4552: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:51:58.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:51:58 smithi165 ceph-mon[36302]: pgmap v4552: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:51:58.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:51:58 smithi119 ceph-mon[31518]: pgmap v4552: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:52:00.452 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:52:00 smithi132 ceph-mon[35941]: pgmap v4553: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T02:52:00.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:52:00 smithi165 ceph-mon[36302]: pgmap v4553: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T02:52:00.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:52:00 smithi119 ceph-mon[31518]: pgmap v4553: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T02:52:01.452 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:52:01 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8180.777952 secs 2024-03-28T02:52:01.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:52:01 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8180.777952 secs 2024-03-28T02:52:01.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:52:01 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8180.777952 secs 2024-03-28T02:52:02.452 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:52:02 smithi132 ceph-mon[35941]: pgmap v4554: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:52:02.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:52:02 smithi165 ceph-mon[36302]: pgmap v4554: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:52:02.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:52:02 smithi119 ceph-mon[31518]: pgmap v4554: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:52:04.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:52:04 smithi165 ceph-mon[36302]: pgmap v4555: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:52:04.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:52:04 smithi119 ceph-mon[31518]: pgmap v4555: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:52:04.701 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:52:04 smithi132 ceph-mon[35941]: pgmap v4555: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:52:06.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:52:06 smithi165 ceph-mon[36302]: pgmap v4556: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:52:06.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:52:06 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8185.778042 secs 2024-03-28T02:52:06.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:52:06 smithi119 ceph-mon[31518]: pgmap v4556: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:52:06.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:52:06 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8185.778042 secs 2024-03-28T02:52:06.701 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:52:06 smithi132 ceph-mon[35941]: pgmap v4556: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:52:06.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:52:06 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8185.778042 secs 2024-03-28T02:52:08.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:52:08 smithi165 ceph-mon[36302]: pgmap v4557: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:52:08.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:52:08 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:52:08.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:52:08 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:52:08.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:52:08 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:52:08.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:52:08 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:52:08.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:52:08 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:52:08.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:52:08 smithi119 ceph-mon[31518]: pgmap v4557: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:52:08.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:52:08 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:52:08.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:52:08 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:52:08.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:52:08 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:52:08.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:52:08 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:52:08.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:52:08 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:52:08.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:52:08 smithi132 ceph-mon[35941]: pgmap v4557: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:52:08.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:52:08 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:52:08.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:52:08 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:52:08.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:52:08 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:52:08.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:52:08 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:52:08.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:52:08 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:52:10.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:52:10 smithi165 ceph-mon[36302]: pgmap v4558: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T02:52:10.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:52:10 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:52:10.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:52:10 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:52:10.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:52:10 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:52:10.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:52:10 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:52:10.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:52:10 smithi119 ceph-mon[31518]: pgmap v4558: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T02:52:10.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:52:10 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:52:10.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:52:10 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:52:10.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:52:10 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:52:10.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:52:10 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:52:10.701 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:52:10 smithi132 ceph-mon[35941]: pgmap v4558: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T02:52:10.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:52:10 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:52:10.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:52:10 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:52:10.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:52:10 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:52:10.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:52:10 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:52:11.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:52:11 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8190.778108 secs 2024-03-28T02:52:11.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:52:11 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:52:11.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:52:11 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:52:11.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:52:11 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:52:11.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:52:11 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:52:11.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:52:11 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:52:11.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:52:11 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:52:11.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:52:11 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:52:11.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:52:11 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:52:11.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:52:11 smithi165 ceph-mon[36302]: Adjusting osd_memory_target on smithi165 to 174.1M 2024-03-28T02:52:11.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:52:11 smithi165 ceph-mon[36302]: Unable to set osd_memory_target on smithi165 to 182635622: error parsing value: Value '182635622' is below minimum 939524096 2024-03-28T02:52:11.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:52:11 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:52:11.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:52:11 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:52:11.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:52:11 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:52:11.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:52:11 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:52:11.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:52:11 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:52:11.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:52:11 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:52:11.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:52:11 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:52:11.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:52:11 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:52:11.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:52:11 smithi165 ceph-mon[36302]: Adjusting osd_memory_target on smithi132 to 174.1M 2024-03-28T02:52:11.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:52:11 smithi165 ceph-mon[36302]: Unable to set osd_memory_target on smithi132 to 182640640: error parsing value: Value '182640640' is below minimum 939524096 2024-03-28T02:52:11.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:52:11 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8190.778108 secs 2024-03-28T02:52:11.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:52:11 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:52:11.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:52:11 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:52:11.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:52:11 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:52:11.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:52:11 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:52:11.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:52:11 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:52:11.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:52:11 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:52:11.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:52:11 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:52:11.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:52:11 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:52:11.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:52:11 smithi119 ceph-mon[31518]: Adjusting osd_memory_target on smithi165 to 174.1M 2024-03-28T02:52:11.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:52:11 smithi119 ceph-mon[31518]: Unable to set osd_memory_target on smithi165 to 182635622: error parsing value: Value '182635622' is below minimum 939524096 2024-03-28T02:52:11.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:52:11 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:52:11.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:52:11 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:52:11.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:52:11 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:52:11.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:52:11 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:52:11.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:52:11 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:52:11.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:52:11 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:52:11.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:52:11 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:52:11.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:52:11 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:52:11.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:52:11 smithi119 ceph-mon[31518]: Adjusting osd_memory_target on smithi132 to 174.1M 2024-03-28T02:52:11.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:52:11 smithi119 ceph-mon[31518]: Unable to set osd_memory_target on smithi132 to 182640640: error parsing value: Value '182640640' is below minimum 939524096 2024-03-28T02:52:11.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:52:11 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8190.778108 secs 2024-03-28T02:52:11.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:52:11 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:52:11.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:52:11 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:52:11.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:52:11 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:52:11.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:52:11 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:52:11.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:52:11 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:52:11.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:52:11 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:52:11.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:52:11 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:52:11.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:52:11 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:52:11.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:52:11 smithi132 ceph-mon[35941]: Adjusting osd_memory_target on smithi165 to 174.1M 2024-03-28T02:52:11.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:52:11 smithi132 ceph-mon[35941]: Unable to set osd_memory_target on smithi165 to 182635622: error parsing value: Value '182635622' is below minimum 939524096 2024-03-28T02:52:11.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:52:11 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:52:11.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:52:11 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:52:11.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:52:11 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:52:11.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:52:11 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:52:11.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:52:11 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:52:11.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:52:11 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:52:11.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:52:11 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:52:11.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:52:11 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:52:11.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:52:11 smithi132 ceph-mon[35941]: Adjusting osd_memory_target on smithi132 to 174.1M 2024-03-28T02:52:11.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:52:11 smithi132 ceph-mon[35941]: Unable to set osd_memory_target on smithi132 to 182640640: error parsing value: Value '182640640' is below minimum 939524096 2024-03-28T02:52:12.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:52:12 smithi165 ceph-mon[36302]: pgmap v4559: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:52:12.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:52:12 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:52:12.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:52:12 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:52:12.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:52:12 smithi119 ceph-mon[31518]: pgmap v4559: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:52:12.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:52:12 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:52:12.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:52:12 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:52:12.701 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:52:12 smithi132 ceph-mon[35941]: pgmap v4559: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:52:12.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:52:12 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:52:12.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:52:12 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:52:13.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:52:13 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:52:13.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:52:13 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:52:13.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:52:13 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:52:13.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:52:13 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:52:13.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:52:13 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:52:13.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:52:13 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:52:13.634 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:52:13 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:52:13.635 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:52:13 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:52:13.635 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:52:13 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:52:13.635 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:52:13 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:52:13.635 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:52:13 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:52:13.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:52:13 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:52:13.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:52:13 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:52:13.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:52:13 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:52:13.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:52:13 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:52:13.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:52:13 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:52:13.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:52:13 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:52:13.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:52:13 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:52:13.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:52:13 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:52:13.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:52:13 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:52:13.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:52:13 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:52:13.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:52:13 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:52:13.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:52:13 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:52:13.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:52:13 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:52:13.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:52:13 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:52:13.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:52:13 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:52:13.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:52:13 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:52:13.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:52:13 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:52:13.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:52:13 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:52:13.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:52:13 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T02:52:13.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:52:13 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:52:13.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:52:13 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:52:13.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:52:13 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:52:14.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:52:14 smithi165 ceph-mon[36302]: Adjusting osd_memory_target on smithi119 to 176.7M 2024-03-28T02:52:14.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:52:14 smithi165 ceph-mon[36302]: Unable to set osd_memory_target on smithi119 to 185372364: error parsing value: Value '185372364' is below minimum 939524096 2024-03-28T02:52:14.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:52:14 smithi165 ceph-mon[36302]: pgmap v4560: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:52:14.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:52:14 smithi165 ceph-mon[36302]: pgmap v4561: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T02:52:14.658 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:52:14 smithi119 ceph-mon[31518]: Adjusting osd_memory_target on smithi119 to 176.7M 2024-03-28T02:52:14.658 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:52:14 smithi119 ceph-mon[31518]: Unable to set osd_memory_target on smithi119 to 185372364: error parsing value: Value '185372364' is below minimum 939524096 2024-03-28T02:52:14.658 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:52:14 smithi119 ceph-mon[31518]: pgmap v4560: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:52:14.658 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:52:14 smithi119 ceph-mon[31518]: pgmap v4561: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T02:52:14.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:52:14 smithi132 ceph-mon[35941]: Adjusting osd_memory_target on smithi119 to 176.7M 2024-03-28T02:52:14.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:52:14 smithi132 ceph-mon[35941]: Unable to set osd_memory_target on smithi119 to 185372364: error parsing value: Value '185372364' is below minimum 939524096 2024-03-28T02:52:14.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:52:14 smithi132 ceph-mon[35941]: pgmap v4560: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:52:14.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:52:14 smithi132 ceph-mon[35941]: pgmap v4561: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T02:52:16.556 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:52:16 smithi119 ceph-mon[31518]: pgmap v4562: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3 KiB/s rd, 5 op/s 2024-03-28T02:52:16.556 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:52:16 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8195.778168 secs 2024-03-28T02:52:16.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:52:16 smithi165 ceph-mon[36302]: pgmap v4562: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3 KiB/s rd, 5 op/s 2024-03-28T02:52:16.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:52:16 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8195.778168 secs 2024-03-28T02:52:16.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:52:16 smithi132 ceph-mon[35941]: pgmap v4562: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3 KiB/s rd, 5 op/s 2024-03-28T02:52:16.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:52:16 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8195.778168 secs 2024-03-28T02:52:18.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:52:18 smithi165 ceph-mon[36302]: pgmap v4563: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3 KiB/s rd, 5 op/s 2024-03-28T02:52:18.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:52:18 smithi119 ceph-mon[31518]: pgmap v4563: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3 KiB/s rd, 5 op/s 2024-03-28T02:52:18.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:52:18 smithi132 ceph-mon[35941]: pgmap v4563: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3 KiB/s rd, 5 op/s 2024-03-28T02:52:20.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:52:20 smithi165 ceph-mon[36302]: pgmap v4564: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3 KiB/s rd, 5 op/s 2024-03-28T02:52:20.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:52:20 smithi119 ceph-mon[31518]: pgmap v4564: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3 KiB/s rd, 5 op/s 2024-03-28T02:52:20.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:52:20 smithi132 ceph-mon[35941]: pgmap v4564: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3 KiB/s rd, 5 op/s 2024-03-28T02:52:21.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:52:21 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8200.778231 secs 2024-03-28T02:52:21.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:52:21 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8200.778231 secs 2024-03-28T02:52:21.701 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:52:21 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8200.778231 secs 2024-03-28T02:52:22.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:52:22 smithi165 ceph-mon[36302]: pgmap v4565: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T02:52:22.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:52:22 smithi119 ceph-mon[31518]: pgmap v4565: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T02:52:22.701 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:52:22 smithi132 ceph-mon[35941]: pgmap v4565: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T02:52:24.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:52:24 smithi165 ceph-mon[36302]: pgmap v4566: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T02:52:24.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:52:24 smithi119 ceph-mon[31518]: pgmap v4566: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T02:52:24.701 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:52:24 smithi132 ceph-mon[35941]: pgmap v4566: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T02:52:26.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:52:26 smithi165 ceph-mon[36302]: pgmap v4567: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:52:26.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:52:26 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8205.778291 secs 2024-03-28T02:52:26.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:52:26 smithi119 ceph-mon[31518]: pgmap v4567: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:52:26.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:52:26 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8205.778291 secs 2024-03-28T02:52:26.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:52:26 smithi132 ceph-mon[35941]: pgmap v4567: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:52:26.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:52:26 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8205.778291 secs 2024-03-28T02:52:28.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:52:28 smithi165 ceph-mon[36302]: pgmap v4568: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:52:28.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:52:28 smithi119 ceph-mon[31518]: pgmap v4568: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:52:28.701 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:52:28 smithi132 ceph-mon[35941]: pgmap v4568: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:52:30.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:52:30 smithi165 ceph-mon[36302]: pgmap v4569: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T02:52:30.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:52:30 smithi119 ceph-mon[31518]: pgmap v4569: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T02:52:30.701 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:52:30 smithi132 ceph-mon[35941]: pgmap v4569: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T02:52:31.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:52:31 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8210.778352 secs 2024-03-28T02:52:31.603 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:52:31 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8210.778352 secs 2024-03-28T02:52:31.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:52:31 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8210.778352 secs 2024-03-28T02:52:32.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:52:32 smithi165 ceph-mon[36302]: pgmap v4570: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-28T02:52:32.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:52:32 smithi119 ceph-mon[31518]: pgmap v4570: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-28T02:52:32.701 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:52:32 smithi132 ceph-mon[35941]: pgmap v4570: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-28T02:52:34.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:52:34 smithi165 ceph-mon[36302]: pgmap v4571: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:52:34.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:52:34 smithi119 ceph-mon[31518]: pgmap v4571: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:52:34.701 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:52:34 smithi132 ceph-mon[35941]: pgmap v4571: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:52:36.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:52:36 smithi165 ceph-mon[36302]: pgmap v4572: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:52:36.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:52:36 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8215.778414 secs 2024-03-28T02:52:36.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:52:36 smithi119 ceph-mon[31518]: pgmap v4572: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:52:36.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:52:36 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8215.778414 secs 2024-03-28T02:52:36.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:52:36 smithi132 ceph-mon[35941]: pgmap v4572: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:52:36.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:52:36 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8215.778414 secs 2024-03-28T02:52:38.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:52:38 smithi165 ceph-mon[36302]: pgmap v4573: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:52:38.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:52:38 smithi119 ceph-mon[31518]: pgmap v4573: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:52:38.701 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:52:38 smithi132 ceph-mon[35941]: pgmap v4573: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:52:40.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:52:40 smithi165 ceph-mon[36302]: pgmap v4574: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T02:52:40.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:52:40 smithi119 ceph-mon[31518]: pgmap v4574: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T02:52:40.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:52:40 smithi132 ceph-mon[35941]: pgmap v4574: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T02:52:41.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:52:41 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8220.778474 secs 2024-03-28T02:52:41.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:52:41 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8220.778474 secs 2024-03-28T02:52:41.701 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:52:41 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8220.778474 secs 2024-03-28T02:52:42.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:52:42 smithi165 ceph-mon[36302]: pgmap v4575: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-28T02:52:42.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:52:42 smithi119 ceph-mon[31518]: pgmap v4575: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-28T02:52:42.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:52:42 smithi132 ceph-mon[35941]: pgmap v4575: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-28T02:52:44.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:52:44 smithi165 ceph-mon[36302]: pgmap v4576: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:52:44.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:52:44 smithi119 ceph-mon[31518]: pgmap v4576: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:52:44.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:52:44 smithi132 ceph-mon[35941]: pgmap v4576: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:52:46.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:52:46 smithi165 ceph-mon[36302]: pgmap v4577: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:52:46.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:52:46 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8225.778534 secs 2024-03-28T02:52:46.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:52:46 smithi119 ceph-mon[31518]: pgmap v4577: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:52:46.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:52:46 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8225.778534 secs 2024-03-28T02:52:46.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:52:46 smithi132 ceph-mon[35941]: pgmap v4577: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:52:46.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:52:46 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8225.778534 secs 2024-03-28T02:52:48.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:52:48 smithi165 ceph-mon[36302]: pgmap v4578: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:52:48.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:52:48 smithi119 ceph-mon[31518]: pgmap v4578: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:52:48.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:52:48 smithi132 ceph-mon[35941]: pgmap v4578: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:52:50.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:52:50 smithi165 ceph-mon[36302]: pgmap v4579: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T02:52:50.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:52:50 smithi119 ceph-mon[31518]: pgmap v4579: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T02:52:50.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:52:50 smithi132 ceph-mon[35941]: pgmap v4579: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T02:52:51.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:52:51 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8230.778620 secs 2024-03-28T02:52:51.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:52:51 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8230.778620 secs 2024-03-28T02:52:51.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:52:51 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8230.778620 secs 2024-03-28T02:52:52.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:52:52 smithi165 ceph-mon[36302]: pgmap v4580: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-28T02:52:52.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:52:52 smithi119 ceph-mon[31518]: pgmap v4580: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-28T02:52:52.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:52:52 smithi132 ceph-mon[35941]: pgmap v4580: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-28T02:52:54.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:52:54 smithi165 ceph-mon[36302]: pgmap v4581: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:52:54.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:52:54 smithi119 ceph-mon[31518]: pgmap v4581: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:52:54.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:52:54 smithi132 ceph-mon[35941]: pgmap v4581: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:52:56.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:52:56 smithi165 ceph-mon[36302]: pgmap v4582: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:52:56.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:52:56 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8235.778660 secs 2024-03-28T02:52:56.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:52:56 smithi119 ceph-mon[31518]: pgmap v4582: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:52:56.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:52:56 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8235.778660 secs 2024-03-28T02:52:56.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:52:56 smithi132 ceph-mon[35941]: pgmap v4582: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:52:56.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:52:56 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8235.778660 secs 2024-03-28T02:52:58.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:52:58 smithi165 ceph-mon[36302]: pgmap v4583: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:52:58.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:52:58 smithi119 ceph-mon[31518]: pgmap v4583: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:52:58.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:52:58 smithi132 ceph-mon[35941]: pgmap v4583: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:53:00.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:53:00 smithi165 ceph-mon[36302]: pgmap v4584: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T02:53:00.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:53:00 smithi119 ceph-mon[31518]: pgmap v4584: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T02:53:00.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:53:00 smithi132 ceph-mon[35941]: pgmap v4584: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T02:53:01.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:53:01 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8240.778698 secs 2024-03-28T02:53:01.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:53:01 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8240.778698 secs 2024-03-28T02:53:01.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:53:01 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8240.778698 secs 2024-03-28T02:53:02.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:53:02 smithi119 ceph-mon[31518]: pgmap v4585: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-28T02:53:02.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:53:02 smithi132 ceph-mon[35941]: pgmap v4585: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-28T02:53:02.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:53:02 smithi165 ceph-mon[36302]: pgmap v4585: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-28T02:53:04.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:53:04 smithi119 ceph-mon[31518]: pgmap v4586: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:53:04.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:53:04 smithi132 ceph-mon[35941]: pgmap v4586: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:53:04.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:53:04 smithi165 ceph-mon[36302]: pgmap v4586: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:53:06.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:53:06 smithi119 ceph-mon[31518]: pgmap v4587: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:53:06.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:53:06 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8245.778738 secs 2024-03-28T02:53:06.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:53:06 smithi132 ceph-mon[35941]: pgmap v4587: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:53:06.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:53:06 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8245.778738 secs 2024-03-28T02:53:06.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:53:06 smithi165 ceph-mon[36302]: pgmap v4587: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:53:06.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:53:06 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8245.778738 secs 2024-03-28T02:53:08.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:53:08 smithi132 ceph-mon[35941]: pgmap v4588: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:53:08.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:53:08 smithi165 ceph-mon[36302]: pgmap v4588: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:53:08.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:53:08 smithi119 ceph-mon[31518]: pgmap v4588: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:53:10.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:53:10 smithi132 ceph-mon[35941]: pgmap v4589: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T02:53:10.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:53:10 smithi165 ceph-mon[36302]: pgmap v4589: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T02:53:10.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:53:10 smithi119 ceph-mon[31518]: pgmap v4589: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T02:53:11.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:53:11 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8250.778827 secs 2024-03-28T02:53:11.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:53:11 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8250.778827 secs 2024-03-28T02:53:11.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:53:11 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8250.778827 secs 2024-03-28T02:53:12.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:53:12 smithi132 ceph-mon[35941]: pgmap v4590: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-28T02:53:12.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:53:12 smithi165 ceph-mon[36302]: pgmap v4590: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-28T02:53:12.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:53:12 smithi119 ceph-mon[31518]: pgmap v4590: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-28T02:53:13.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:53:13 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:53:13.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:53:13 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:53:13.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:53:13 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:53:14.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:53:14 smithi132 ceph-mon[35941]: pgmap v4591: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:53:14.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:53:14 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:53:14.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:53:14 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:53:14.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:53:14 smithi165 ceph-mon[36302]: pgmap v4591: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:53:14.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:53:14 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:53:14.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:53:14 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:53:14.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:53:14 smithi119 ceph-mon[31518]: pgmap v4591: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:53:14.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:53:14 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:53:14.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:53:14 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:53:16.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:53:16 smithi132 ceph-mon[35941]: pgmap v4592: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:53:16.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:53:16 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8255.778900 secs 2024-03-28T02:53:16.864 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:53:16 smithi119 ceph-mon[31518]: pgmap v4592: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:53:16.864 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:53:16 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8255.778900 secs 2024-03-28T02:53:16.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:53:16 smithi165 ceph-mon[36302]: pgmap v4592: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:53:16.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:53:16 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8255.778900 secs 2024-03-28T02:53:18.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:53:18 smithi132 ceph-mon[35941]: pgmap v4593: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T02:53:18.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:53:18 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:53:18.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:53:18 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:53:18.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:53:18 smithi119 ceph-mon[31518]: pgmap v4593: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T02:53:18.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:53:18 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:53:18.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:53:18 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:53:18.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:53:18 smithi165 ceph-mon[36302]: pgmap v4593: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T02:53:18.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:53:18 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:53:18.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:53:18 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:53:20.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:53:20 smithi132 ceph-mon[35941]: pgmap v4594: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-03-28T02:53:20.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:53:20 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:53:20.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:53:20 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:53:20.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:53:20 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:53:20.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:53:20 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:53:20.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:53:20 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:53:20.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:53:20 smithi165 ceph-mon[36302]: pgmap v4594: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-03-28T02:53:20.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:53:20 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:53:20.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:53:20 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:53:20.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:53:20 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:53:20.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:53:20 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:53:20.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:53:20 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:53:20.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:53:20 smithi119 ceph-mon[31518]: pgmap v4594: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-03-28T02:53:20.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:53:20 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:53:20.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:53:20 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:53:20.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:53:20 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:53:20.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:53:20 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:53:20.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:53:20 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:53:21.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:53:21 smithi132 ceph-mon[35941]: pgmap v4595: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T02:53:21.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:53:21 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8260.778962 secs 2024-03-28T02:53:21.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:53:21 smithi165 ceph-mon[36302]: pgmap v4595: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T02:53:21.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:53:21 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8260.778962 secs 2024-03-28T02:53:21.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:53:21 smithi119 ceph-mon[31518]: pgmap v4595: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T02:53:21.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:53:21 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8260.778962 secs 2024-03-28T02:53:23.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:53:23 smithi165 ceph-mon[36302]: pgmap v4596: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T02:53:23.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:53:23 smithi119 ceph-mon[31518]: pgmap v4596: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T02:53:23.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:53:23 smithi132 ceph-mon[35941]: pgmap v4596: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T02:53:25.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:53:25 smithi165 ceph-mon[36302]: pgmap v4597: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T02:53:25.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:53:25 smithi119 ceph-mon[31518]: pgmap v4597: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T02:53:25.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:53:25 smithi132 ceph-mon[35941]: pgmap v4597: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T02:53:26.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:53:26 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8265.779023 secs 2024-03-28T02:53:26.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:53:26 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8265.779023 secs 2024-03-28T02:53:26.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:53:26 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8265.779023 secs 2024-03-28T02:53:27.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:53:27 smithi165 ceph-mon[36302]: pgmap v4598: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-28T02:53:27.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:53:27 smithi119 ceph-mon[31518]: pgmap v4598: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-28T02:53:27.942 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:53:27 smithi132 ceph-mon[35941]: pgmap v4598: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-28T02:53:28.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:53:28 smithi165 ceph-mon[36302]: pgmap v4599: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-28T02:53:28.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:53:28 smithi119 ceph-mon[31518]: pgmap v4599: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-28T02:53:28.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:53:28 smithi132 ceph-mon[35941]: pgmap v4599: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-28T02:53:31.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:53:31 smithi165 ceph-mon[36302]: pgmap v4600: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 7 op/s 2024-03-28T02:53:31.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:53:31 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8270.779107 secs 2024-03-28T02:53:31.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:53:31 smithi119 ceph-mon[31518]: pgmap v4600: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 7 op/s 2024-03-28T02:53:31.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:53:31 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8270.779107 secs 2024-03-28T02:53:31.452 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:53:31 smithi132 ceph-mon[35941]: pgmap v4600: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 7 op/s 2024-03-28T02:53:31.452 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:53:31 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8270.779107 secs 2024-03-28T02:53:33.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:53:33 smithi165 ceph-mon[36302]: pgmap v4601: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:53:33.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:53:33 smithi119 ceph-mon[31518]: pgmap v4601: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:53:33.452 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:53:33 smithi132 ceph-mon[35941]: pgmap v4601: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:53:35.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:53:35 smithi165 ceph-mon[36302]: pgmap v4602: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:53:35.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:53:35 smithi119 ceph-mon[31518]: pgmap v4602: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:53:35.452 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:53:35 smithi132 ceph-mon[35941]: pgmap v4602: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:53:36.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:53:36 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8275.779169 secs 2024-03-28T02:53:36.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:53:36 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8275.779169 secs 2024-03-28T02:53:36.452 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:53:36 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8275.779169 secs 2024-03-28T02:53:37.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:53:37 smithi119 ceph-mon[31518]: pgmap v4603: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:53:37.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:53:37 smithi165 ceph-mon[36302]: pgmap v4603: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:53:37.452 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:53:37 smithi132 ceph-mon[35941]: pgmap v4603: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:53:39.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:53:39 smithi119 ceph-mon[31518]: pgmap v4604: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:53:39.452 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:53:39 smithi132 ceph-mon[35941]: pgmap v4604: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:53:39.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:53:39 smithi165 ceph-mon[36302]: pgmap v4604: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:53:41.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:53:41 smithi119 ceph-mon[31518]: pgmap v4605: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:53:41.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:53:41 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8280.779230 secs 2024-03-28T02:53:41.452 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:53:41 smithi132 ceph-mon[35941]: pgmap v4605: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:53:41.452 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:53:41 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8280.779230 secs 2024-03-28T02:53:41.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:53:41 smithi165 ceph-mon[36302]: pgmap v4605: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:53:41.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:53:41 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8280.779230 secs 2024-03-28T02:53:43.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:53:43 smithi119 ceph-mon[31518]: pgmap v4606: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:53:43.452 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:53:43 smithi132 ceph-mon[35941]: pgmap v4606: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:53:43.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:53:43 smithi165 ceph-mon[36302]: pgmap v4606: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:53:45.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:53:45 smithi119 ceph-mon[31518]: pgmap v4607: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:53:45.452 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:53:45 smithi132 ceph-mon[35941]: pgmap v4607: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:53:45.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:53:45 smithi165 ceph-mon[36302]: pgmap v4607: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:53:46.452 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:53:46 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8285.779292 secs 2024-03-28T02:53:46.533 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:53:46 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8285.779292 secs 2024-03-28T02:53:46.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:53:46 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8285.779292 secs 2024-03-28T02:53:47.405 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:53:47 smithi119 ceph-mon[31518]: pgmap v4608: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:53:47.452 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:53:47 smithi132 ceph-mon[35941]: pgmap v4608: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:53:47.545 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:53:47 smithi165 ceph-mon[36302]: pgmap v4608: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:53:49.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:53:49 smithi165 ceph-mon[36302]: pgmap v4609: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:53:49.452 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:53:49 smithi132 ceph-mon[35941]: pgmap v4609: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:53:49.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:53:49 smithi119 ceph-mon[31518]: pgmap v4609: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:53:51.452 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:53:51 smithi132 ceph-mon[35941]: pgmap v4610: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:53:51.452 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:53:51 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8290.779329 secs 2024-03-28T02:53:51.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:53:51 smithi165 ceph-mon[36302]: pgmap v4610: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:53:51.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:53:51 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8290.779329 secs 2024-03-28T02:53:51.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:53:51 smithi119 ceph-mon[31518]: pgmap v4610: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:53:51.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:53:51 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8290.779329 secs 2024-03-28T02:53:53.452 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:53:53 smithi132 ceph-mon[35941]: pgmap v4611: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:53:53.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:53:53 smithi165 ceph-mon[36302]: pgmap v4611: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:53:53.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:53:53 smithi119 ceph-mon[31518]: pgmap v4611: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:53:55.452 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:53:55 smithi132 ceph-mon[35941]: pgmap v4612: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:53:55.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:53:55 smithi165 ceph-mon[36302]: pgmap v4612: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:53:55.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:53:55 smithi119 ceph-mon[31518]: pgmap v4612: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:53:56.452 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:53:56 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8295.779389 secs 2024-03-28T02:53:56.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:53:56 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8295.779389 secs 2024-03-28T02:53:56.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:53:56 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8295.779389 secs 2024-03-28T02:53:57.452 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:53:57 smithi132 ceph-mon[35941]: pgmap v4613: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:53:57.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:53:57 smithi165 ceph-mon[36302]: pgmap v4613: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:53:57.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:53:57 smithi119 ceph-mon[31518]: pgmap v4613: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:53:59.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:53:59 smithi165 ceph-mon[36302]: pgmap v4614: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:53:59.452 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:53:59 smithi132 ceph-mon[35941]: pgmap v4614: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:53:59.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:53:59 smithi119 ceph-mon[31518]: pgmap v4614: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:54:01.452 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:54:01 smithi132 ceph-mon[35941]: pgmap v4615: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:54:01.452 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:54:01 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8300.779448 secs 2024-03-28T02:54:01.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:54:01 smithi165 ceph-mon[36302]: pgmap v4615: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:54:01.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:54:01 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8300.779448 secs 2024-03-28T02:54:01.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:54:01 smithi119 ceph-mon[31518]: pgmap v4615: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:54:01.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:54:01 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8300.779448 secs 2024-03-28T02:54:03.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:54:03 smithi165 ceph-mon[36302]: pgmap v4616: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:54:03.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:54:03 smithi119 ceph-mon[31518]: pgmap v4616: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:54:03.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:54:03 smithi132 ceph-mon[35941]: pgmap v4616: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:54:05.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:54:05 smithi165 ceph-mon[36302]: pgmap v4617: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:54:05.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:54:05 smithi119 ceph-mon[31518]: pgmap v4617: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:54:05.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:54:05 smithi132 ceph-mon[35941]: pgmap v4617: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:54:06.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:54:06 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8305.779537 secs 2024-03-28T02:54:06.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:54:06 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8305.779537 secs 2024-03-28T02:54:06.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:54:06 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8305.779537 secs 2024-03-28T02:54:07.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:54:07 smithi165 ceph-mon[36302]: pgmap v4618: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:54:07.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:54:07 smithi119 ceph-mon[31518]: pgmap v4618: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:54:07.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:54:07 smithi132 ceph-mon[35941]: pgmap v4618: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:54:09.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:54:09 smithi165 ceph-mon[36302]: pgmap v4619: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:54:09.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:54:09 smithi119 ceph-mon[31518]: pgmap v4619: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:54:09.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:54:09 smithi132 ceph-mon[35941]: pgmap v4619: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:54:11.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:54:11 smithi165 ceph-mon[36302]: pgmap v4620: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:54:11.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:54:11 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8310.779597 secs 2024-03-28T02:54:11.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:54:11 smithi119 ceph-mon[31518]: pgmap v4620: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:54:11.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:54:11 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8310.779597 secs 2024-03-28T02:54:11.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:54:11 smithi132 ceph-mon[35941]: pgmap v4620: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:54:11.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:54:11 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8310.779597 secs 2024-03-28T02:54:13.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:54:13 smithi165 ceph-mon[36302]: pgmap v4621: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:54:13.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:54:13 smithi119 ceph-mon[31518]: pgmap v4621: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:54:13.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:54:13 smithi132 ceph-mon[35941]: pgmap v4621: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:54:15.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:54:15 smithi165 ceph-mon[36302]: pgmap v4622: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:54:15.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:54:15 smithi119 ceph-mon[31518]: pgmap v4622: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:54:15.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:54:15 smithi132 ceph-mon[35941]: pgmap v4622: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:54:16.533 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:54:16 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8315.779682 secs 2024-03-28T02:54:16.557 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:54:16 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8315.779682 secs 2024-03-28T02:54:16.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:54:16 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8315.779682 secs 2024-03-28T02:54:17.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:54:17 smithi165 ceph-mon[36302]: pgmap v4623: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:54:17.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:54:17 smithi119 ceph-mon[31518]: pgmap v4623: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:54:17.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:54:17 smithi132 ceph-mon[35941]: pgmap v4623: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:54:19.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:54:19 smithi165 ceph-mon[36302]: pgmap v4624: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:54:19.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:54:19 smithi119 ceph-mon[31518]: pgmap v4624: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:54:19.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:54:19 smithi132 ceph-mon[35941]: pgmap v4624: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:54:21.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:54:21 smithi165 ceph-mon[36302]: pgmap v4625: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:54:21.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:54:21 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8320.779769 secs 2024-03-28T02:54:21.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:54:21 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:54:21.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:54:21 smithi119 ceph-mon[31518]: pgmap v4625: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:54:21.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:54:21 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8320.779769 secs 2024-03-28T02:54:21.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:54:21 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:54:21.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:54:21 smithi132 ceph-mon[35941]: pgmap v4625: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:54:21.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:54:21 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8320.779769 secs 2024-03-28T02:54:21.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:54:21 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:54:23.374 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:54:23 smithi165 ceph-mon[36302]: pgmap v4626: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:54:23.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:54:23 smithi119 ceph-mon[31518]: pgmap v4626: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:54:23.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:54:23 smithi132 ceph-mon[35941]: pgmap v4626: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:54:25.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:54:25 smithi165 ceph-mon[36302]: pgmap v4627: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:54:25.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:54:25 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:54:25.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:54:25 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:54:25.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:54:25 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:54:25.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:54:25 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:54:25.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:54:25 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:54:25.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:54:25 smithi119 ceph-mon[31518]: pgmap v4627: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:54:25.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:54:25 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:54:25.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:54:25 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:54:25.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:54:25 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:54:25.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:54:25 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:54:25.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:54:25 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:54:25.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:54:25 smithi132 ceph-mon[35941]: pgmap v4627: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:54:25.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:54:25 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:54:25.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:54:25 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:54:25.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:54:25 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:54:25.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:54:25 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:54:25.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:54:25 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:54:26.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:54:26 smithi165 ceph-mon[36302]: pgmap v4628: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T02:54:26.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:54:26 smithi165 ceph-mon[36302]: pgmap v4629: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-03-28T02:54:26.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:54:26 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8325.779856 secs 2024-03-28T02:54:26.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:54:26 smithi119 ceph-mon[31518]: pgmap v4628: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T02:54:26.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:54:26 smithi119 ceph-mon[31518]: pgmap v4629: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-03-28T02:54:26.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:54:26 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8325.779856 secs 2024-03-28T02:54:26.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:54:26 smithi132 ceph-mon[35941]: pgmap v4628: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T02:54:26.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:54:26 smithi132 ceph-mon[35941]: pgmap v4629: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-03-28T02:54:26.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:54:26 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8325.779856 secs 2024-03-28T02:54:28.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:54:28 smithi165 ceph-mon[36302]: pgmap v4630: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-03-28T02:54:28.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:54:28 smithi119 ceph-mon[31518]: pgmap v4630: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-03-28T02:54:28.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:54:28 smithi132 ceph-mon[35941]: pgmap v4630: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-03-28T02:54:30.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:54:30 smithi165 ceph-mon[36302]: pgmap v4631: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-28T02:54:30.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:54:30 smithi119 ceph-mon[31518]: pgmap v4631: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-28T02:54:30.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:54:30 smithi132 ceph-mon[35941]: pgmap v4631: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-28T02:54:31.603 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:54:31 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8330.779917 secs 2024-03-28T02:54:31.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:54:31 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8330.779917 secs 2024-03-28T02:54:31.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:54:31 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8330.779917 secs 2024-03-28T02:54:32.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:54:32 smithi165 ceph-mon[36302]: pgmap v4632: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T02:54:32.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:54:32 smithi119 ceph-mon[31518]: pgmap v4632: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T02:54:32.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:54:32 smithi132 ceph-mon[35941]: pgmap v4632: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T02:54:34.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:54:34 smithi165 ceph-mon[36302]: pgmap v4633: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T02:54:34.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:54:34 smithi119 ceph-mon[31518]: pgmap v4633: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T02:54:34.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:54:34 smithi132 ceph-mon[35941]: pgmap v4633: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T02:54:36.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:54:36 smithi165 ceph-mon[36302]: pgmap v4634: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-28T02:54:36.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:54:36 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8335.779979 secs 2024-03-28T02:54:36.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:54:36 smithi119 ceph-mon[31518]: pgmap v4634: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-28T02:54:36.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:54:36 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8335.779979 secs 2024-03-28T02:54:36.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:54:36 smithi132 ceph-mon[35941]: pgmap v4634: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-28T02:54:36.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:54:36 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8335.779979 secs 2024-03-28T02:54:38.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:54:38 smithi165 ceph-mon[36302]: pgmap v4635: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T02:54:38.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:54:38 smithi119 ceph-mon[31518]: pgmap v4635: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T02:54:38.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:54:38 smithi132 ceph-mon[35941]: pgmap v4635: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T02:54:40.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:54:40 smithi165 ceph-mon[36302]: pgmap v4636: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T02:54:40.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:54:40 smithi119 ceph-mon[31518]: pgmap v4636: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T02:54:40.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:54:40 smithi132 ceph-mon[35941]: pgmap v4636: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T02:54:41.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:54:41 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8340.780040 secs 2024-03-28T02:54:41.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:54:41 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8340.780040 secs 2024-03-28T02:54:41.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:54:41 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8340.780040 secs 2024-03-28T02:54:42.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:54:42 smithi165 ceph-mon[36302]: pgmap v4637: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:54:42.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:54:42 smithi119 ceph-mon[31518]: pgmap v4637: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:54:42.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:54:42 smithi132 ceph-mon[35941]: pgmap v4637: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:54:44.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:54:44 smithi165 ceph-mon[36302]: pgmap v4638: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:54:44.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:54:44 smithi119 ceph-mon[31518]: pgmap v4638: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:54:44.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:54:44 smithi132 ceph-mon[35941]: pgmap v4638: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:54:46.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:54:46 smithi165 ceph-mon[36302]: pgmap v4639: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:54:46.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:54:46 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8345.780116 secs 2024-03-28T02:54:46.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:54:46 smithi119 ceph-mon[31518]: pgmap v4639: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:54:46.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:54:46 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8345.780116 secs 2024-03-28T02:54:46.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:54:46 smithi132 ceph-mon[35941]: pgmap v4639: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:54:46.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:54:46 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8345.780116 secs 2024-03-28T02:54:48.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:54:48 smithi165 ceph-mon[36302]: pgmap v4640: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:54:48.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:54:48 smithi119 ceph-mon[31518]: pgmap v4640: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:54:48.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:54:48 smithi132 ceph-mon[35941]: pgmap v4640: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:54:50.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:54:50 smithi165 ceph-mon[36302]: pgmap v4641: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:54:50.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:54:50 smithi119 ceph-mon[31518]: pgmap v4641: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:54:50.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:54:50 smithi132 ceph-mon[35941]: pgmap v4641: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:54:51.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:54:51 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8350.780165 secs 2024-03-28T02:54:51.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:54:51 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8350.780165 secs 2024-03-28T02:54:51.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:54:51 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8350.780165 secs 2024-03-28T02:54:52.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:54:52 smithi165 ceph-mon[36302]: pgmap v4642: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:54:52.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:54:52 smithi119 ceph-mon[31518]: pgmap v4642: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:54:52.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:54:52 smithi132 ceph-mon[35941]: pgmap v4642: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:54:54.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:54:54 smithi165 ceph-mon[36302]: pgmap v4643: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:54:54.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:54:54 smithi119 ceph-mon[31518]: pgmap v4643: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:54:54.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:54:54 smithi132 ceph-mon[35941]: pgmap v4643: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:54:56.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:54:56 smithi165 ceph-mon[36302]: pgmap v4644: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:54:56.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:54:56 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8355.780203 secs 2024-03-28T02:54:56.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:54:56 smithi119 ceph-mon[31518]: pgmap v4644: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:54:56.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:54:56 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8355.780203 secs 2024-03-28T02:54:56.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:54:56 smithi132 ceph-mon[35941]: pgmap v4644: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:54:56.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:54:56 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8355.780203 secs 2024-03-28T02:54:58.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:54:58 smithi165 ceph-mon[36302]: pgmap v4645: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:54:58.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:54:58 smithi119 ceph-mon[31518]: pgmap v4645: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:54:58.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:54:58 smithi132 ceph-mon[35941]: pgmap v4645: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:55:00.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:55:00 smithi119 ceph-mon[31518]: pgmap v4646: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:55:00.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:55:00 smithi132 ceph-mon[35941]: pgmap v4646: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:55:00.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:55:00 smithi165 ceph-mon[36302]: pgmap v4646: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:55:01.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:55:01 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8360.780292 secs 2024-03-28T02:55:01.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:55:01 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8360.780292 secs 2024-03-28T02:55:01.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:55:01 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8360.780292 secs 2024-03-28T02:55:02.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:55:02 smithi119 ceph-mon[31518]: pgmap v4647: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:55:02.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:55:02 smithi132 ceph-mon[35941]: pgmap v4647: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:55:02.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:55:02 smithi165 ceph-mon[36302]: pgmap v4647: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:55:04.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:55:04 smithi119 ceph-mon[31518]: pgmap v4648: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:55:04.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:55:04 smithi132 ceph-mon[35941]: pgmap v4648: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:55:04.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:55:04 smithi165 ceph-mon[36302]: pgmap v4648: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:55:06.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:55:06 smithi132 ceph-mon[35941]: pgmap v4649: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:55:06.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:55:06 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8365.780353 secs 2024-03-28T02:55:06.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:55:06 smithi165 ceph-mon[36302]: pgmap v4649: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:55:06.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:55:06 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8365.780353 secs 2024-03-28T02:55:06.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:55:06 smithi119 ceph-mon[31518]: pgmap v4649: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:55:06.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:55:06 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8365.780353 secs 2024-03-28T02:55:08.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:55:08 smithi119 ceph-mon[31518]: pgmap v4650: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:55:08.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:55:08 smithi132 ceph-mon[35941]: pgmap v4650: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:55:08.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:55:08 smithi165 ceph-mon[36302]: pgmap v4650: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:55:10.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:55:10 smithi132 ceph-mon[35941]: pgmap v4651: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:55:10.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:55:10 smithi165 ceph-mon[36302]: pgmap v4651: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:55:10.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:55:10 smithi119 ceph-mon[31518]: pgmap v4651: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:55:11.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:55:11 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8370.780440 secs 2024-03-28T02:55:11.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:55:11 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8370.780440 secs 2024-03-28T02:55:11.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:55:11 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8370.780440 secs 2024-03-28T02:55:12.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:55:12 smithi132 ceph-mon[35941]: pgmap v4652: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:55:12.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:55:12 smithi165 ceph-mon[36302]: pgmap v4652: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:55:12.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:55:12 smithi119 ceph-mon[31518]: pgmap v4652: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:55:14.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:55:14 smithi132 ceph-mon[35941]: pgmap v4653: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:55:14.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:55:14 smithi165 ceph-mon[36302]: pgmap v4653: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:55:14.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:55:14 smithi119 ceph-mon[31518]: pgmap v4653: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:55:16.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:55:16 smithi132 ceph-mon[35941]: pgmap v4654: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:55:16.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:55:16 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8375.780528 secs 2024-03-28T02:55:16.708 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:55:16 smithi119 ceph-mon[31518]: pgmap v4654: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:55:16.708 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:55:16 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8375.780528 secs 2024-03-28T02:55:16.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:55:16 smithi165 ceph-mon[36302]: pgmap v4654: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:55:16.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:55:16 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8375.780528 secs 2024-03-28T02:55:18.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:55:18 smithi132 ceph-mon[35941]: pgmap v4655: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:55:18.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:55:18 smithi165 ceph-mon[36302]: pgmap v4655: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:55:18.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:55:18 smithi119 ceph-mon[31518]: pgmap v4655: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:55:20.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:55:20 smithi132 ceph-mon[35941]: pgmap v4656: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:55:20.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:55:20 smithi165 ceph-mon[36302]: pgmap v4656: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:55:20.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:55:20 smithi119 ceph-mon[31518]: pgmap v4656: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:55:21.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:55:21 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8380.780590 secs 2024-03-28T02:55:21.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:55:21 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8380.780590 secs 2024-03-28T02:55:21.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:55:21 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8380.780590 secs 2024-03-28T02:55:22.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:55:22 smithi165 ceph-mon[36302]: pgmap v4657: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:55:22.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:55:22 smithi119 ceph-mon[31518]: pgmap v4657: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:55:22.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:55:22 smithi132 ceph-mon[35941]: pgmap v4657: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:55:24.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:55:24 smithi165 ceph-mon[36302]: pgmap v4658: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:55:24.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:55:24 smithi119 ceph-mon[31518]: pgmap v4658: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:55:24.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:55:24 smithi132 ceph-mon[35941]: pgmap v4658: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:55:25.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:55:25 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:55:25.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:55:25 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:55:25.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:55:25 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:55:26.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:55:26 smithi165 ceph-mon[36302]: pgmap v4659: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:55:26.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:55:26 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8385.780655 secs 2024-03-28T02:55:26.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:55:26 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:55:26.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:55:26 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:55:26.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:55:26 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:55:26.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:55:26 smithi119 ceph-mon[31518]: pgmap v4659: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:55:26.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:55:26 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8385.780655 secs 2024-03-28T02:55:26.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:55:26 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:55:26.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:55:26 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:55:26.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:55:26 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:55:26.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:55:26 smithi132 ceph-mon[35941]: pgmap v4659: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:55:26.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:55:26 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8385.780655 secs 2024-03-28T02:55:26.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:55:26 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:55:26.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:55:26 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:55:26.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:55:26 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:55:27.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:55:27 smithi165 ceph-mon[36302]: pgmap v4660: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T02:55:27.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:55:27 smithi165 ceph-mon[36302]: pgmap v4661: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:55:27.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:55:27 smithi119 ceph-mon[31518]: pgmap v4660: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T02:55:27.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:55:27 smithi119 ceph-mon[31518]: pgmap v4661: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:55:27.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:55:27 smithi132 ceph-mon[35941]: pgmap v4660: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T02:55:27.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:55:27 smithi132 ceph-mon[35941]: pgmap v4661: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:55:29.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:55:29 smithi165 ceph-mon[36302]: pgmap v4662: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:55:29.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:55:29 smithi119 ceph-mon[31518]: pgmap v4662: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:55:29.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:55:29 smithi132 ceph-mon[35941]: pgmap v4662: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:55:30.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:55:30 smithi165 ceph-mon[36302]: pgmap v4663: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:55:30.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:55:30 smithi119 ceph-mon[31518]: pgmap v4663: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:55:30.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:55:30 smithi132 ceph-mon[35941]: pgmap v4663: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:55:31.853 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:55:31 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8390.780743 secs 2024-03-28T02:55:31.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:55:31 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8390.780743 secs 2024-03-28T02:55:31.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:55:31 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8390.780743 secs 2024-03-28T02:55:32.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:55:32 smithi165 ceph-mon[36302]: pgmap v4664: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:55:32.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:55:32 smithi119 ceph-mon[31518]: pgmap v4664: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:55:32.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:55:32 smithi132 ceph-mon[35941]: pgmap v4664: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:55:35.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:55:35 smithi165 ceph-mon[36302]: pgmap v4665: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 5 op/s 2024-03-28T02:55:35.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:55:35 smithi119 ceph-mon[31518]: pgmap v4665: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 5 op/s 2024-03-28T02:55:35.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:55:35 smithi132 ceph-mon[35941]: pgmap v4665: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 5 op/s 2024-03-28T02:55:36.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:55:36 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8395.780840 secs 2024-03-28T02:55:36.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:55:36 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8395.780840 secs 2024-03-28T02:55:36.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:55:36 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8395.780840 secs 2024-03-28T02:55:37.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:55:37 smithi165 ceph-mon[36302]: pgmap v4666: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T02:55:37.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:55:37 smithi119 ceph-mon[31518]: pgmap v4666: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T02:55:37.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:55:37 smithi132 ceph-mon[35941]: pgmap v4666: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T02:55:39.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:55:39 smithi165 ceph-mon[36302]: pgmap v4667: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:55:39.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:55:39 smithi119 ceph-mon[31518]: pgmap v4667: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:55:39.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:55:39 smithi132 ceph-mon[35941]: pgmap v4667: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:55:41.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:55:41 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8400.780877 secs 2024-03-28T02:55:41.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:55:41 smithi165 ceph-mon[36302]: pgmap v4668: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:55:41.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:55:41 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8400.780877 secs 2024-03-28T02:55:41.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:55:41 smithi119 ceph-mon[31518]: pgmap v4668: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:55:41.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:55:41 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8400.780877 secs 2024-03-28T02:55:41.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:55:41 smithi132 ceph-mon[35941]: pgmap v4668: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:55:43.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:55:43 smithi165 ceph-mon[36302]: pgmap v4669: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:55:43.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:55:43 smithi119 ceph-mon[31518]: pgmap v4669: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:55:43.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:55:43 smithi132 ceph-mon[35941]: pgmap v4669: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:55:45.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:55:45 smithi119 ceph-mon[31518]: pgmap v4670: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-28T02:55:45.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:55:45 smithi132 ceph-mon[35941]: pgmap v4670: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-28T02:55:45.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:55:45 smithi165 ceph-mon[36302]: pgmap v4670: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-28T02:55:46.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:55:46 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8405.780939 secs 2024-03-28T02:55:46.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:55:46 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8405.780939 secs 2024-03-28T02:55:46.783 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:55:46 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8405.780939 secs 2024-03-28T02:55:47.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:55:47 smithi119 ceph-mon[31518]: pgmap v4671: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-03-28T02:55:47.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:55:47 smithi132 ceph-mon[35941]: pgmap v4671: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-03-28T02:55:47.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:55:47 smithi165 ceph-mon[36302]: pgmap v4671: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-03-28T02:55:49.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:55:49 smithi119 ceph-mon[31518]: pgmap v4672: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:55:49.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:55:49 smithi132 ceph-mon[35941]: pgmap v4672: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:55:49.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:55:49 smithi165 ceph-mon[36302]: pgmap v4672: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:55:51.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:55:51 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8410.781001 secs 2024-03-28T02:55:51.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:55:51 smithi119 ceph-mon[31518]: pgmap v4673: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:55:51.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:55:51 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8410.781001 secs 2024-03-28T02:55:51.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:55:51 smithi132 ceph-mon[35941]: pgmap v4673: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:55:51.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:55:51 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8410.781001 secs 2024-03-28T02:55:51.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:55:51 smithi165 ceph-mon[36302]: pgmap v4673: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:55:53.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:55:53 smithi132 ceph-mon[35941]: pgmap v4674: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:55:53.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:55:53 smithi165 ceph-mon[36302]: pgmap v4674: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:55:53.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:55:53 smithi119 ceph-mon[31518]: pgmap v4674: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:55:55.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:55:55 smithi132 ceph-mon[35941]: pgmap v4675: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-28T02:55:55.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:55:55 smithi165 ceph-mon[36302]: pgmap v4675: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-28T02:55:55.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:55:55 smithi119 ceph-mon[31518]: pgmap v4675: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-28T02:55:56.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:55:56 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8415.781063 secs 2024-03-28T02:55:56.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:55:56 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8415.781063 secs 2024-03-28T02:55:56.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:55:56 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8415.781063 secs 2024-03-28T02:55:57.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:55:57 smithi132 ceph-mon[35941]: pgmap v4676: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-03-28T02:55:57.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:55:57 smithi165 ceph-mon[36302]: pgmap v4676: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-03-28T02:55:57.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:55:57 smithi119 ceph-mon[31518]: pgmap v4676: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-03-28T02:55:59.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:55:59 smithi132 ceph-mon[35941]: pgmap v4677: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:55:59.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:55:59 smithi165 ceph-mon[36302]: pgmap v4677: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:55:59.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:55:59 smithi119 ceph-mon[31518]: pgmap v4677: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:56:01.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:56:01 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8420.781151 secs 2024-03-28T02:56:01.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:56:01 smithi132 ceph-mon[35941]: pgmap v4678: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:56:01.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:56:01 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8420.781151 secs 2024-03-28T02:56:01.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:56:01 smithi165 ceph-mon[36302]: pgmap v4678: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:56:01.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:56:01 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8420.781151 secs 2024-03-28T02:56:01.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:56:01 smithi119 ceph-mon[31518]: pgmap v4678: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:56:03.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:56:03 smithi132 ceph-mon[35941]: pgmap v4679: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:56:03.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:56:03 smithi165 ceph-mon[36302]: pgmap v4679: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:56:03.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:56:03 smithi119 ceph-mon[31518]: pgmap v4679: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:56:05.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:56:05 smithi132 ceph-mon[35941]: pgmap v4680: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-28T02:56:05.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:56:05 smithi165 ceph-mon[36302]: pgmap v4680: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-28T02:56:05.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:56:05 smithi119 ceph-mon[31518]: pgmap v4680: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-28T02:56:06.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:56:06 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8425.781210 secs 2024-03-28T02:56:06.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:56:06 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8425.781210 secs 2024-03-28T02:56:06.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:56:06 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8425.781210 secs 2024-03-28T02:56:07.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:56:07 smithi165 ceph-mon[36302]: pgmap v4681: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-03-28T02:56:07.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:56:07 smithi119 ceph-mon[31518]: pgmap v4681: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-03-28T02:56:07.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:56:07 smithi132 ceph-mon[35941]: pgmap v4681: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-03-28T02:56:09.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:56:09 smithi165 ceph-mon[36302]: pgmap v4682: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:56:09.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:56:09 smithi119 ceph-mon[31518]: pgmap v4682: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:56:09.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:56:09 smithi132 ceph-mon[35941]: pgmap v4682: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:56:11.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:56:11 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8430.781270 secs 2024-03-28T02:56:11.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:56:11 smithi165 ceph-mon[36302]: pgmap v4683: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:56:11.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:56:11 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8430.781270 secs 2024-03-28T02:56:11.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:56:11 smithi119 ceph-mon[31518]: pgmap v4683: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:56:11.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:56:11 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8430.781270 secs 2024-03-28T02:56:11.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:56:11 smithi132 ceph-mon[35941]: pgmap v4683: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:56:13.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:56:13 smithi119 ceph-mon[31518]: pgmap v4684: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:56:13.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:56:13 smithi165 ceph-mon[36302]: pgmap v4684: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:56:13.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:56:13 smithi132 ceph-mon[35941]: pgmap v4684: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:56:15.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:56:15 smithi165 ceph-mon[36302]: pgmap v4685: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-28T02:56:15.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:56:15 smithi119 ceph-mon[31518]: pgmap v4685: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-28T02:56:15.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:56:15 smithi132 ceph-mon[35941]: pgmap v4685: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-28T02:56:16.782 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:56:16 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8435.781329 secs 2024-03-28T02:56:16.806 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:56:16 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8435.781329 secs 2024-03-28T02:56:16.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:56:16 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8435.781329 secs 2024-03-28T02:56:17.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:56:17 smithi165 ceph-mon[36302]: pgmap v4686: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-03-28T02:56:17.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:56:17 smithi119 ceph-mon[31518]: pgmap v4686: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-03-28T02:56:17.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:56:17 smithi132 ceph-mon[35941]: pgmap v4686: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-03-28T02:56:19.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:56:19 smithi165 ceph-mon[36302]: pgmap v4687: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:56:19.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:56:19 smithi119 ceph-mon[31518]: pgmap v4687: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:56:19.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:56:19 smithi132 ceph-mon[35941]: pgmap v4687: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:56:21.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:56:21 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8440.781390 secs 2024-03-28T02:56:21.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:56:21 smithi165 ceph-mon[36302]: pgmap v4688: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:56:21.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:56:21 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8440.781390 secs 2024-03-28T02:56:21.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:56:21 smithi119 ceph-mon[31518]: pgmap v4688: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:56:21.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:56:21 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8440.781390 secs 2024-03-28T02:56:21.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:56:21 smithi132 ceph-mon[35941]: pgmap v4688: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:56:23.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:56:23 smithi119 ceph-mon[31518]: pgmap v4689: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:56:23.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:56:23 smithi165 ceph-mon[36302]: pgmap v4689: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:56:23.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:56:23 smithi132 ceph-mon[35941]: pgmap v4689: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:56:24.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:56:24 smithi165 ceph-mon[36302]: pgmap v4690: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-28T02:56:24.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:56:24 smithi119 ceph-mon[31518]: pgmap v4690: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-28T02:56:24.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:56:24 smithi132 ceph-mon[35941]: pgmap v4690: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-28T02:56:26.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:56:25 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8445.781450 secs 2024-03-28T02:56:26.149 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:56:25 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8445.781450 secs 2024-03-28T02:56:26.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:56:25 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8445.781450 secs 2024-03-28T02:56:27.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:56:26 smithi119 ceph-mon[31518]: pgmap v4691: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-28T02:56:27.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:56:26 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:56:27.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:56:26 smithi132 ceph-mon[35941]: pgmap v4691: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-28T02:56:27.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:56:26 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:56:27.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:56:26 smithi165 ceph-mon[36302]: pgmap v4691: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-28T02:56:27.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:56:26 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:56:28.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:56:27 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:56:28.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:56:27 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:56:28.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:56:27 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:56:28.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:56:27 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:56:28.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:56:27 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:56:28.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:56:27 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:56:28.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:56:27 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:56:28.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:56:27 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:56:28.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:56:27 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:56:29.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:56:28 smithi119 ceph-mon[31518]: pgmap v4692: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T02:56:29.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:56:28 smithi119 ceph-mon[31518]: pgmap v4693: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:56:29.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:56:28 smithi132 ceph-mon[35941]: pgmap v4692: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T02:56:29.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:56:28 smithi132 ceph-mon[35941]: pgmap v4693: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:56:29.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:56:28 smithi165 ceph-mon[36302]: pgmap v4692: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T02:56:29.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:56:28 smithi165 ceph-mon[36302]: pgmap v4693: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:56:31.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:56:30 smithi119 ceph-mon[31518]: pgmap v4694: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:56:31.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:56:30 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8450.781539 secs 2024-03-28T02:56:31.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:56:30 smithi132 ceph-mon[35941]: pgmap v4694: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:56:31.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:56:30 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8450.781539 secs 2024-03-28T02:56:31.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:56:30 smithi165 ceph-mon[36302]: pgmap v4694: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:56:31.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:56:30 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8450.781539 secs 2024-03-28T02:56:33.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:56:32 smithi132 ceph-mon[35941]: pgmap v4695: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:56:33.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:56:32 smithi165 ceph-mon[36302]: pgmap v4695: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:56:33.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:56:32 smithi119 ceph-mon[31518]: pgmap v4695: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:56:35.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:56:34 smithi132 ceph-mon[35941]: pgmap v4696: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-28T02:56:35.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:56:34 smithi165 ceph-mon[36302]: pgmap v4696: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-28T02:56:35.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:56:34 smithi119 ceph-mon[31518]: pgmap v4696: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-28T02:56:36.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:56:35 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8455.781623 secs 2024-03-28T02:56:36.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:56:35 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8455.781623 secs 2024-03-28T02:56:36.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:56:35 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8455.781623 secs 2024-03-28T02:56:37.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:56:36 smithi132 ceph-mon[35941]: pgmap v4697: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T02:56:37.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:56:36 smithi165 ceph-mon[36302]: pgmap v4697: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T02:56:37.406 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:56:36 smithi119 ceph-mon[31518]: pgmap v4697: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T02:56:39.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:56:38 smithi132 ceph-mon[35941]: pgmap v4698: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-28T02:56:39.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:56:38 smithi165 ceph-mon[36302]: pgmap v4698: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-28T02:56:39.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:56:38 smithi119 ceph-mon[31518]: pgmap v4698: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-28T02:56:41.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:56:40 smithi132 ceph-mon[35941]: pgmap v4699: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:56:41.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:56:40 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8460.781683 secs 2024-03-28T02:56:41.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:56:40 smithi165 ceph-mon[36302]: pgmap v4699: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:56:41.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:56:40 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8460.781683 secs 2024-03-28T02:56:41.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:56:40 smithi119 ceph-mon[31518]: pgmap v4699: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:56:41.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:56:40 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8460.781683 secs 2024-03-28T02:56:43.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:56:42 smithi132 ceph-mon[35941]: pgmap v4700: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T02:56:43.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:56:42 smithi165 ceph-mon[36302]: pgmap v4700: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T02:56:43.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:56:42 smithi119 ceph-mon[31518]: pgmap v4700: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T02:56:45.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:56:44 smithi132 ceph-mon[35941]: pgmap v4701: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T02:56:45.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:56:44 smithi165 ceph-mon[36302]: pgmap v4701: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T02:56:45.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:56:44 smithi119 ceph-mon[31518]: pgmap v4701: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T02:56:46.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:56:45 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8465.781745 secs 2024-03-28T02:56:46.282 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:56:45 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8465.781745 secs 2024-03-28T02:56:46.306 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:56:45 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8465.781745 secs 2024-03-28T02:56:47.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:56:46 smithi165 ceph-mon[36302]: pgmap v4702: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:56:47.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:56:46 smithi119 ceph-mon[31518]: pgmap v4702: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:56:47.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:56:46 smithi132 ceph-mon[35941]: pgmap v4702: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:56:49.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:56:48 smithi165 ceph-mon[36302]: pgmap v4703: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:56:49.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:56:48 smithi119 ceph-mon[31518]: pgmap v4703: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:56:49.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:56:48 smithi132 ceph-mon[35941]: pgmap v4703: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:56:51.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:56:50 smithi165 ceph-mon[36302]: pgmap v4704: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:56:51.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:56:50 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8470.781806 secs 2024-03-28T02:56:51.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:56:50 smithi119 ceph-mon[31518]: pgmap v4704: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:56:51.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:56:50 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8470.781806 secs 2024-03-28T02:56:51.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:56:50 smithi132 ceph-mon[35941]: pgmap v4704: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:56:51.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:56:50 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8470.781806 secs 2024-03-28T02:56:53.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:56:52 smithi165 ceph-mon[36302]: pgmap v4705: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:56:53.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:56:52 smithi119 ceph-mon[31518]: pgmap v4705: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:56:53.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:56:52 smithi132 ceph-mon[35941]: pgmap v4705: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:56:55.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:56:54 smithi165 ceph-mon[36302]: pgmap v4706: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:56:55.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:56:54 smithi119 ceph-mon[31518]: pgmap v4706: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:56:55.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:56:54 smithi132 ceph-mon[35941]: pgmap v4706: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:56:56.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:56:55 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8475.781892 secs 2024-03-28T02:56:56.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:56:55 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8475.781892 secs 2024-03-28T02:56:56.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:56:55 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8475.781892 secs 2024-03-28T02:56:57.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:56:56 smithi119 ceph-mon[31518]: pgmap v4707: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:56:57.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:56:56 smithi165 ceph-mon[36302]: pgmap v4707: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:56:57.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:56:56 smithi132 ceph-mon[35941]: pgmap v4707: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:56:59.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:56:58 smithi165 ceph-mon[36302]: pgmap v4708: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:56:59.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:56:58 smithi119 ceph-mon[31518]: pgmap v4708: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:56:59.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:56:58 smithi132 ceph-mon[35941]: pgmap v4708: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:57:01.254 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:57:00 smithi132 ceph-mon[35941]: pgmap v4709: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:57:01.254 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:57:00 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8480.781956 secs 2024-03-28T02:57:01.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:57:00 smithi165 ceph-mon[36302]: pgmap v4709: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:57:01.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:57:00 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8480.781956 secs 2024-03-28T02:57:01.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:57:00 smithi119 ceph-mon[31518]: pgmap v4709: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:57:01.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:57:00 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8480.781956 secs 2024-03-28T02:57:03.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:57:02 smithi165 ceph-mon[36302]: pgmap v4710: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:57:03.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:57:02 smithi119 ceph-mon[31518]: pgmap v4710: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:57:03.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:57:02 smithi132 ceph-mon[35941]: pgmap v4710: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:57:05.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:57:04 smithi165 ceph-mon[36302]: pgmap v4711: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:57:05.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:57:04 smithi119 ceph-mon[31518]: pgmap v4711: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:57:05.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:57:04 smithi132 ceph-mon[35941]: pgmap v4711: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:57:06.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:57:06 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8485.781970 secs 2024-03-28T02:57:06.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:57:06 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8485.781970 secs 2024-03-28T02:57:06.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:57:06 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8485.781970 secs 2024-03-28T02:57:07.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:57:07 smithi165 ceph-mon[36302]: pgmap v4712: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:57:07.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:57:07 smithi119 ceph-mon[31518]: pgmap v4712: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:57:07.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:57:07 smithi132 ceph-mon[35941]: pgmap v4712: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:57:09.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:57:09 smithi165 ceph-mon[36302]: pgmap v4713: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:57:09.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:57:09 smithi119 ceph-mon[31518]: pgmap v4713: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:57:09.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:57:09 smithi132 ceph-mon[35941]: pgmap v4713: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:57:11.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:57:11 smithi165 ceph-mon[36302]: pgmap v4714: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:57:11.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:57:11 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8490.782033 secs 2024-03-28T02:57:11.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:57:11 smithi119 ceph-mon[31518]: pgmap v4714: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:57:11.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:57:11 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8490.782033 secs 2024-03-28T02:57:11.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:57:11 smithi132 ceph-mon[35941]: pgmap v4714: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:57:11.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:57:11 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8490.782033 secs 2024-03-28T02:57:13.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:57:13 smithi165 ceph-mon[36302]: pgmap v4715: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:57:13.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:57:13 smithi119 ceph-mon[31518]: pgmap v4715: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:57:13.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:57:13 smithi132 ceph-mon[35941]: pgmap v4715: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:57:15.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:57:15 smithi165 ceph-mon[36302]: pgmap v4716: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:57:15.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:57:15 smithi119 ceph-mon[31518]: pgmap v4716: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:57:15.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:57:15 smithi132 ceph-mon[35941]: pgmap v4716: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:57:16.306 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:57:16 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8495.782093 secs 2024-03-28T02:57:16.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:57:16 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8495.782093 secs 2024-03-28T02:57:16.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:57:16 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8495.782093 secs 2024-03-28T02:57:17.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:57:17 smithi165 ceph-mon[36302]: pgmap v4717: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:57:17.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:57:17 smithi119 ceph-mon[31518]: pgmap v4717: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:57:17.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:57:17 smithi132 ceph-mon[35941]: pgmap v4717: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:57:19.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:57:19 smithi165 ceph-mon[36302]: pgmap v4718: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:57:19.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:57:19 smithi119 ceph-mon[31518]: pgmap v4718: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:57:19.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:57:19 smithi132 ceph-mon[35941]: pgmap v4718: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:57:21.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:57:21 smithi165 ceph-mon[36302]: pgmap v4719: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:57:21.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:57:21 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8500.782182 secs 2024-03-28T02:57:21.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:57:21 smithi119 ceph-mon[31518]: pgmap v4719: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:57:21.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:57:21 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8500.782182 secs 2024-03-28T02:57:21.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:57:21 smithi132 ceph-mon[35941]: pgmap v4719: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:57:21.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:57:21 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8500.782182 secs 2024-03-28T02:57:23.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:57:23 smithi165 ceph-mon[36302]: pgmap v4720: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:57:23.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:57:23 smithi119 ceph-mon[31518]: pgmap v4720: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:57:23.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:57:23 smithi132 ceph-mon[35941]: pgmap v4720: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:57:25.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:57:25 smithi165 ceph-mon[36302]: pgmap v4721: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:57:25.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:57:25 smithi119 ceph-mon[31518]: pgmap v4721: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:57:25.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:57:25 smithi132 ceph-mon[35941]: pgmap v4721: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:57:26.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:57:26 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8505.782269 secs 2024-03-28T02:57:26.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:57:26 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8505.782269 secs 2024-03-28T02:57:26.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:57:26 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8505.782269 secs 2024-03-28T02:57:27.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:57:27 smithi165 ceph-mon[36302]: pgmap v4722: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:57:27.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:57:27 smithi119 ceph-mon[31518]: pgmap v4722: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:57:27.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:57:27 smithi132 ceph-mon[35941]: pgmap v4722: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:57:29.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:57:29 smithi165 ceph-mon[36302]: pgmap v4723: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:57:29.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:57:29 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:57:29.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:57:29 smithi119 ceph-mon[31518]: pgmap v4723: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:57:29.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:57:29 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:57:29.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:57:29 smithi132 ceph-mon[35941]: pgmap v4723: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:57:29.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:57:29 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:57:30.701 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:57:30 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:57:30.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:57:30 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:57:30.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:57:30 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:57:30.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:57:30 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:57:30.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:57:30 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:57:30.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:57:30 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:57:30.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:57:30 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:57:30.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:57:30 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:57:30.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:57:30 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:57:30.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:57:30 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:57:30.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:57:30 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:57:30.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:57:30 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:57:30.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:57:30 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:57:30.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:57:30 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:57:30.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:57:30 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:57:30.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:57:30 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:57:30.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:57:30 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:57:30.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:57:30 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:57:30.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:57:30 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:57:30.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:57:30 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:57:30.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:57:30 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:57:30.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:57:30 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:57:30.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:57:30 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:57:30.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:57:30 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:57:30.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:57:30 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:57:30.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:57:30 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:57:30.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:57:30 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:57:31.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:57:31 smithi165 ceph-mon[36302]: pgmap v4724: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:57:31.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:57:31 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8510.782330 secs 2024-03-28T02:57:31.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:57:31 smithi165 ceph-mon[36302]: pgmap v4725: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-28T02:57:31.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:57:31 smithi165 ceph-mon[36302]: pgmap v4726: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-03-28T02:57:31.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:57:31 smithi119 ceph-mon[31518]: pgmap v4724: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:57:31.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:57:31 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8510.782330 secs 2024-03-28T02:57:31.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:57:31 smithi119 ceph-mon[31518]: pgmap v4725: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-28T02:57:31.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:57:31 smithi119 ceph-mon[31518]: pgmap v4726: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-03-28T02:57:31.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:57:31 smithi132 ceph-mon[35941]: pgmap v4724: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:57:31.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:57:31 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8510.782330 secs 2024-03-28T02:57:31.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:57:31 smithi132 ceph-mon[35941]: pgmap v4725: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-28T02:57:31.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:57:31 smithi132 ceph-mon[35941]: pgmap v4726: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-03-28T02:57:33.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:57:33 smithi165 ceph-mon[36302]: pgmap v4727: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-03-28T02:57:33.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:57:33 smithi119 ceph-mon[31518]: pgmap v4727: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-03-28T02:57:33.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:57:33 smithi132 ceph-mon[35941]: pgmap v4727: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-03-28T02:57:35.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:57:35 smithi165 ceph-mon[36302]: pgmap v4728: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-03-28T02:57:35.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:57:35 smithi119 ceph-mon[31518]: pgmap v4728: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-03-28T02:57:35.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:57:35 smithi132 ceph-mon[35941]: pgmap v4728: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-03-28T02:57:36.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:57:36 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8515.782420 secs 2024-03-28T02:57:36.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:57:36 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8515.782420 secs 2024-03-28T02:57:36.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:57:36 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8515.782420 secs 2024-03-28T02:57:37.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:57:37 smithi165 ceph-mon[36302]: pgmap v4729: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-03-28T02:57:37.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:57:37 smithi119 ceph-mon[31518]: pgmap v4729: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-03-28T02:57:37.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:57:37 smithi132 ceph-mon[35941]: pgmap v4729: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-03-28T02:57:38.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:57:38 smithi165 ceph-mon[36302]: pgmap v4730: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 3 op/s 2024-03-28T02:57:38.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:57:38 smithi119 ceph-mon[31518]: pgmap v4730: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 3 op/s 2024-03-28T02:57:38.952 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:57:38 smithi132 ceph-mon[35941]: pgmap v4730: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 3 op/s 2024-03-28T02:57:41.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:57:40 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8520.782483 secs 2024-03-28T02:57:41.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:57:40 smithi132 ceph-mon[35941]: pgmap v4731: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T02:57:41.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:57:40 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8520.782483 secs 2024-03-28T02:57:41.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:57:40 smithi165 ceph-mon[36302]: pgmap v4731: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T02:57:41.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:57:40 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8520.782483 secs 2024-03-28T02:57:41.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:57:40 smithi119 ceph-mon[31518]: pgmap v4731: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T02:57:43.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:57:43 smithi165 ceph-mon[36302]: pgmap v4732: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:57:43.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:57:43 smithi119 ceph-mon[31518]: pgmap v4732: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:57:43.701 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:57:43 smithi132 ceph-mon[35941]: pgmap v4732: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:57:45.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:57:45 smithi165 ceph-mon[36302]: pgmap v4733: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:57:45.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:57:45 smithi119 ceph-mon[31518]: pgmap v4733: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:57:45.701 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:57:45 smithi132 ceph-mon[35941]: pgmap v4733: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:57:46.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:57:46 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8525.782569 secs 2024-03-28T02:57:46.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:57:46 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8525.782569 secs 2024-03-28T02:57:46.701 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:57:46 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8525.782569 secs 2024-03-28T02:57:47.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:57:47 smithi165 ceph-mon[36302]: pgmap v4734: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:57:47.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:57:47 smithi119 ceph-mon[31518]: pgmap v4734: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:57:47.701 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:57:47 smithi132 ceph-mon[35941]: pgmap v4734: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:57:49.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:57:49 smithi165 ceph-mon[36302]: pgmap v4735: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:57:49.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:57:49 smithi119 ceph-mon[31518]: pgmap v4735: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:57:49.701 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:57:49 smithi132 ceph-mon[35941]: pgmap v4735: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:57:51.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:57:51 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8530.782655 secs 2024-03-28T02:57:51.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:57:51 smithi165 ceph-mon[36302]: pgmap v4736: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:57:51.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:57:51 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8530.782655 secs 2024-03-28T02:57:51.679 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:57:51 smithi119 ceph-mon[31518]: pgmap v4736: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:57:51.701 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:57:51 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8530.782655 secs 2024-03-28T02:57:51.701 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:57:51 smithi132 ceph-mon[35941]: pgmap v4736: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:57:53.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:57:53 smithi165 ceph-mon[36302]: pgmap v4737: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:57:53.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:57:53 smithi119 ceph-mon[31518]: pgmap v4737: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:57:53.701 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:57:53 smithi132 ceph-mon[35941]: pgmap v4737: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:57:55.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:57:55 smithi165 ceph-mon[36302]: pgmap v4738: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:57:55.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:57:55 smithi119 ceph-mon[31518]: pgmap v4738: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:57:55.701 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:57:55 smithi132 ceph-mon[35941]: pgmap v4738: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:57:56.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:57:56 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8535.782696 secs 2024-03-28T02:57:56.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:57:56 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8535.782696 secs 2024-03-28T02:57:56.701 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:57:56 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8535.782696 secs 2024-03-28T02:57:57.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:57:57 smithi165 ceph-mon[36302]: pgmap v4739: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:57:57.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:57:57 smithi119 ceph-mon[31518]: pgmap v4739: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:57:57.701 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:57:57 smithi132 ceph-mon[35941]: pgmap v4739: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:57:59.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:57:59 smithi119 ceph-mon[31518]: pgmap v4740: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:57:59.701 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:57:59 smithi132 ceph-mon[35941]: pgmap v4740: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:57:59.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:57:59 smithi165 ceph-mon[36302]: pgmap v4740: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:58:01.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:58:01 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8540.782756 secs 2024-03-28T02:58:01.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:58:01 smithi119 ceph-mon[31518]: pgmap v4741: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:58:01.701 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:58:01 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8540.782756 secs 2024-03-28T02:58:01.701 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:58:01 smithi132 ceph-mon[35941]: pgmap v4741: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:58:01.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:58:01 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8540.782756 secs 2024-03-28T02:58:01.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:58:01 smithi165 ceph-mon[36302]: pgmap v4741: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:58:03.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:58:03 smithi119 ceph-mon[31518]: pgmap v4742: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:58:03.701 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:58:03 smithi132 ceph-mon[35941]: pgmap v4742: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:58:03.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:58:03 smithi165 ceph-mon[36302]: pgmap v4742: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:58:05.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:58:05 smithi119 ceph-mon[31518]: pgmap v4743: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:58:05.701 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:58:05 smithi132 ceph-mon[35941]: pgmap v4743: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:58:05.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:58:05 smithi165 ceph-mon[36302]: pgmap v4743: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:58:06.701 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:58:06 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8545.782847 secs 2024-03-28T02:58:06.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:58:06 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8545.782847 secs 2024-03-28T02:58:06.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:58:06 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8545.782847 secs 2024-03-28T02:58:07.701 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:58:07 smithi132 ceph-mon[35941]: pgmap v4744: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:58:07.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:58:07 smithi165 ceph-mon[36302]: pgmap v4744: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:58:07.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:58:07 smithi119 ceph-mon[31518]: pgmap v4744: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:58:09.701 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:58:09 smithi132 ceph-mon[35941]: pgmap v4745: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:58:09.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:58:09 smithi165 ceph-mon[36302]: pgmap v4745: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:58:09.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:58:09 smithi119 ceph-mon[31518]: pgmap v4745: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:58:11.701 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:58:11 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8550.782901 secs 2024-03-28T02:58:11.701 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:58:11 smithi132 ceph-mon[35941]: pgmap v4746: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:58:11.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:58:11 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8550.782901 secs 2024-03-28T02:58:11.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:58:11 smithi165 ceph-mon[36302]: pgmap v4746: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:58:11.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:58:11 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8550.782901 secs 2024-03-28T02:58:11.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:58:11 smithi119 ceph-mon[31518]: pgmap v4746: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:58:13.701 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:58:13 smithi132 ceph-mon[35941]: pgmap v4747: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:58:13.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:58:13 smithi165 ceph-mon[36302]: pgmap v4747: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:58:13.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:58:13 smithi119 ceph-mon[31518]: pgmap v4747: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:58:15.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:58:15 smithi119 ceph-mon[31518]: pgmap v4748: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:58:15.701 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:58:15 smithi132 ceph-mon[35941]: pgmap v4748: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:58:15.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:58:15 smithi165 ceph-mon[36302]: pgmap v4748: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:58:16.701 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:58:16 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8555.782938 secs 2024-03-28T02:58:16.782 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:58:16 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8555.782938 secs 2024-03-28T02:58:16.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:58:16 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8555.782938 secs 2024-03-28T02:58:17.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:58:17 smithi165 ceph-mon[36302]: pgmap v4749: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:58:17.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:58:17 smithi119 ceph-mon[31518]: pgmap v4749: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:58:17.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:58:17 smithi132 ceph-mon[35941]: pgmap v4749: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:58:19.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:58:19 smithi165 ceph-mon[36302]: pgmap v4750: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:58:19.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:58:19 smithi119 ceph-mon[31518]: pgmap v4750: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:58:19.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:58:19 smithi132 ceph-mon[35941]: pgmap v4750: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:58:21.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:58:21 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8560.783002 secs 2024-03-28T02:58:21.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:58:21 smithi165 ceph-mon[36302]: pgmap v4751: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:58:21.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:58:21 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8560.783002 secs 2024-03-28T02:58:21.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:58:21 smithi119 ceph-mon[31518]: pgmap v4751: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:58:21.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:58:21 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8560.783002 secs 2024-03-28T02:58:21.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:58:21 smithi132 ceph-mon[35941]: pgmap v4751: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:58:23.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:58:23 smithi165 ceph-mon[36302]: pgmap v4752: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:58:23.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:58:23 smithi119 ceph-mon[31518]: pgmap v4752: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:58:23.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:58:23 smithi132 ceph-mon[35941]: pgmap v4752: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:58:25.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:58:25 smithi165 ceph-mon[36302]: pgmap v4753: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:58:25.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:58:25 smithi119 ceph-mon[31518]: pgmap v4753: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:58:25.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:58:25 smithi132 ceph-mon[35941]: pgmap v4753: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:58:26.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:58:26 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8565.783062 secs 2024-03-28T02:58:26.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:58:26 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8565.783062 secs 2024-03-28T02:58:26.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:58:26 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8565.783062 secs 2024-03-28T02:58:27.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:58:27 smithi165 ceph-mon[36302]: pgmap v4754: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:58:27.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:58:27 smithi119 ceph-mon[31518]: pgmap v4754: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:58:27.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:58:27 smithi132 ceph-mon[35941]: pgmap v4754: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:58:29.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:58:29 smithi165 ceph-mon[36302]: pgmap v4755: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:58:29.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:58:29 smithi119 ceph-mon[31518]: pgmap v4755: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:58:29.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:58:29 smithi132 ceph-mon[35941]: pgmap v4755: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:58:31.753 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:58:31 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8570.783125 secs 2024-03-28T02:58:31.753 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:58:31 smithi132 ceph-mon[35941]: pgmap v4756: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:58:31.753 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:58:31 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:58:31.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:58:31 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8570.783125 secs 2024-03-28T02:58:31.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:58:31 smithi165 ceph-mon[36302]: pgmap v4756: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:58:31.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:58:31 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:58:31.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:58:31 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8570.783125 secs 2024-03-28T02:58:31.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:58:31 smithi119 ceph-mon[31518]: pgmap v4756: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:58:31.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:58:31 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:58:32.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:58:32 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:58:32.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:58:32 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:58:32.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:58:32 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:58:32.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:58:32 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:58:32.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:58:32 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:58:32.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:58:32 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:58:32.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:58:32 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:58:32.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:58:32 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:58:32.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:58:32 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:58:33.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:58:33 smithi165 ceph-mon[36302]: pgmap v4757: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T02:58:33.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:58:33 smithi165 ceph-mon[36302]: pgmap v4758: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:58:33.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:58:33 smithi119 ceph-mon[31518]: pgmap v4757: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T02:58:33.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:58:33 smithi119 ceph-mon[31518]: pgmap v4758: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:58:33.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:58:33 smithi132 ceph-mon[35941]: pgmap v4757: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T02:58:33.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:58:33 smithi132 ceph-mon[35941]: pgmap v4758: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:58:35.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:58:35 smithi165 ceph-mon[36302]: pgmap v4759: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-28T02:58:35.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:58:35 smithi119 ceph-mon[31518]: pgmap v4759: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-28T02:58:35.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:58:35 smithi132 ceph-mon[35941]: pgmap v4759: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-28T02:58:36.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:58:36 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8575.783188 secs 2024-03-28T02:58:36.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:58:36 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8575.783188 secs 2024-03-28T02:58:36.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:58:36 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8575.783188 secs 2024-03-28T02:58:37.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:58:37 smithi165 ceph-mon[36302]: pgmap v4760: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:58:37.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:58:37 smithi119 ceph-mon[31518]: pgmap v4760: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:58:37.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:58:37 smithi132 ceph-mon[35941]: pgmap v4760: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:58:39.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:58:39 smithi165 ceph-mon[36302]: pgmap v4761: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:58:39.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:58:39 smithi119 ceph-mon[31518]: pgmap v4761: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:58:39.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:58:39 smithi132 ceph-mon[35941]: pgmap v4761: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:58:41.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:58:41 smithi165 ceph-mon[36302]: pgmap v4762: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:58:41.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:58:41 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8580.783227 secs 2024-03-28T02:58:41.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:58:41 smithi119 ceph-mon[31518]: pgmap v4762: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:58:41.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:58:41 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8580.783227 secs 2024-03-28T02:58:41.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:58:41 smithi132 ceph-mon[35941]: pgmap v4762: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:58:41.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:58:41 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8580.783227 secs 2024-03-28T02:58:42.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:58:42 smithi165 ceph-mon[36302]: pgmap v4763: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T02:58:42.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:58:42 smithi119 ceph-mon[31518]: pgmap v4763: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T02:58:42.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:58:42 smithi132 ceph-mon[35941]: pgmap v4763: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T02:58:45.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:58:44 smithi132 ceph-mon[35941]: pgmap v4764: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:58:45.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:58:44 smithi165 ceph-mon[36302]: pgmap v4764: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:58:45.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:58:44 smithi119 ceph-mon[31518]: pgmap v4764: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:58:46.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:58:45 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8585.783290 secs 2024-03-28T02:58:46.282 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:58:45 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8585.783290 secs 2024-03-28T02:58:46.306 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:58:45 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8585.783290 secs 2024-03-28T02:58:47.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:58:46 smithi132 ceph-mon[35941]: pgmap v4765: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:58:47.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:58:46 smithi165 ceph-mon[36302]: pgmap v4765: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:58:47.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:58:46 smithi119 ceph-mon[31518]: pgmap v4765: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:58:49.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:58:48 smithi132 ceph-mon[35941]: pgmap v4766: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:58:49.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:58:48 smithi165 ceph-mon[36302]: pgmap v4766: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:58:49.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:58:48 smithi119 ceph-mon[31518]: pgmap v4766: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:58:51.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:58:50 smithi132 ceph-mon[35941]: pgmap v4767: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:58:51.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:58:50 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8590.783380 secs 2024-03-28T02:58:51.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:58:50 smithi165 ceph-mon[36302]: pgmap v4767: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:58:51.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:58:50 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8590.783380 secs 2024-03-28T02:58:51.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:58:50 smithi119 ceph-mon[31518]: pgmap v4767: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:58:51.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:58:50 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8590.783380 secs 2024-03-28T02:58:53.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:58:52 smithi132 ceph-mon[35941]: pgmap v4768: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:58:53.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:58:52 smithi165 ceph-mon[36302]: pgmap v4768: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:58:53.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:58:52 smithi119 ceph-mon[31518]: pgmap v4768: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:58:55.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:58:54 smithi132 ceph-mon[35941]: pgmap v4769: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:58:55.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:58:54 smithi165 ceph-mon[36302]: pgmap v4769: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:58:55.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:58:54 smithi119 ceph-mon[31518]: pgmap v4769: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:58:56.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:58:55 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8595.783441 secs 2024-03-28T02:58:56.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:58:55 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8595.783441 secs 2024-03-28T02:58:56.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:58:55 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8595.783441 secs 2024-03-28T02:58:57.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:58:56 smithi132 ceph-mon[35941]: pgmap v4770: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:58:57.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:58:56 smithi165 ceph-mon[36302]: pgmap v4770: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:58:57.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:58:56 smithi119 ceph-mon[31518]: pgmap v4770: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:58:59.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:58:58 smithi165 ceph-mon[36302]: pgmap v4771: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:58:59.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:58:58 smithi119 ceph-mon[31518]: pgmap v4771: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:58:59.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:58:58 smithi132 ceph-mon[35941]: pgmap v4771: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:59:01.253 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:59:00 smithi132 ceph-mon[35941]: pgmap v4772: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:59:01.253 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:59:00 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8600.783455 secs 2024-03-28T02:59:01.284 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:59:00 smithi119 ceph-mon[31518]: pgmap v4772: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:59:01.284 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:59:00 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8600.783455 secs 2024-03-28T02:59:01.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:59:00 smithi165 ceph-mon[36302]: pgmap v4772: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:59:01.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:59:00 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8600.783455 secs 2024-03-28T02:59:03.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:59:02 smithi165 ceph-mon[36302]: pgmap v4773: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:59:03.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:59:02 smithi119 ceph-mon[31518]: pgmap v4773: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:59:03.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:59:02 smithi132 ceph-mon[35941]: pgmap v4773: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:59:05.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:59:04 smithi165 ceph-mon[36302]: pgmap v4774: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:59:05.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:59:04 smithi119 ceph-mon[31518]: pgmap v4774: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:59:05.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:59:04 smithi132 ceph-mon[35941]: pgmap v4774: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:59:06.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:59:05 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8605.783541 secs 2024-03-28T02:59:06.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:59:05 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8605.783541 secs 2024-03-28T02:59:06.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:59:05 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8605.783541 secs 2024-03-28T02:59:07.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:59:06 smithi165 ceph-mon[36302]: pgmap v4775: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:59:07.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:59:06 smithi119 ceph-mon[31518]: pgmap v4775: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:59:07.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:59:06 smithi132 ceph-mon[35941]: pgmap v4775: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:59:09.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:59:08 smithi165 ceph-mon[36302]: pgmap v4776: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:59:09.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:59:08 smithi119 ceph-mon[31518]: pgmap v4776: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:59:09.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:59:08 smithi132 ceph-mon[35941]: pgmap v4776: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:59:11.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:59:10 smithi165 ceph-mon[36302]: pgmap v4777: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:59:11.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:59:10 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8610.783627 secs 2024-03-28T02:59:11.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:59:10 smithi119 ceph-mon[31518]: pgmap v4777: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:59:11.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:59:10 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8610.783627 secs 2024-03-28T02:59:11.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:59:10 smithi132 ceph-mon[35941]: pgmap v4777: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:59:11.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:59:10 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8610.783627 secs 2024-03-28T02:59:13.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:59:12 smithi165 ceph-mon[36302]: pgmap v4778: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:59:13.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:59:12 smithi119 ceph-mon[31518]: pgmap v4778: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:59:13.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:59:12 smithi132 ceph-mon[35941]: pgmap v4778: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:59:15.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:59:14 smithi165 ceph-mon[36302]: pgmap v4779: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:59:15.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:59:14 smithi119 ceph-mon[31518]: pgmap v4779: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:59:15.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:59:14 smithi132 ceph-mon[35941]: pgmap v4779: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:59:16.281 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:59:16 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8615.783716 secs 2024-03-28T02:59:16.305 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:59:16 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8615.783716 secs 2024-03-28T02:59:16.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:59:16 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8615.783716 secs 2024-03-28T02:59:17.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:59:17 smithi165 ceph-mon[36302]: pgmap v4780: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:59:17.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:59:17 smithi119 ceph-mon[31518]: pgmap v4780: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:59:17.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:59:17 smithi132 ceph-mon[35941]: pgmap v4780: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:59:19.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:59:19 smithi165 ceph-mon[36302]: pgmap v4781: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:59:19.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:59:19 smithi119 ceph-mon[31518]: pgmap v4781: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:59:19.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:59:19 smithi132 ceph-mon[35941]: pgmap v4781: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:59:21.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:59:21 smithi165 ceph-mon[36302]: pgmap v4782: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:59:21.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:59:21 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8620.783807 secs 2024-03-28T02:59:21.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:59:21 smithi119 ceph-mon[31518]: pgmap v4782: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:59:21.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:59:21 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8620.783807 secs 2024-03-28T02:59:21.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:59:21 smithi132 ceph-mon[35941]: pgmap v4782: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:59:21.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:59:21 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8620.783807 secs 2024-03-28T02:59:23.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:59:23 smithi165 ceph-mon[36302]: pgmap v4783: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:59:23.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:59:23 smithi119 ceph-mon[31518]: pgmap v4783: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:59:23.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:59:23 smithi132 ceph-mon[35941]: pgmap v4783: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:59:25.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:59:25 smithi165 ceph-mon[36302]: pgmap v4784: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:59:25.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:59:25 smithi119 ceph-mon[31518]: pgmap v4784: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:59:25.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:59:25 smithi132 ceph-mon[35941]: pgmap v4784: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:59:26.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:59:26 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8625.783898 secs 2024-03-28T02:59:26.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:59:26 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8625.783898 secs 2024-03-28T02:59:26.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:59:26 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8625.783898 secs 2024-03-28T02:59:27.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:59:27 smithi165 ceph-mon[36302]: pgmap v4785: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:59:27.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:59:27 smithi119 ceph-mon[31518]: pgmap v4785: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:59:27.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:59:27 smithi132 ceph-mon[35941]: pgmap v4785: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:59:29.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:59:29 smithi165 ceph-mon[36302]: pgmap v4786: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:59:29.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:59:29 smithi119 ceph-mon[31518]: pgmap v4786: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:59:29.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:59:29 smithi132 ceph-mon[35941]: pgmap v4786: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:59:31.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:59:31 smithi165 ceph-mon[36302]: pgmap v4787: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:59:31.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:59:31 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8630.783988 secs 2024-03-28T02:59:31.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:59:31 smithi119 ceph-mon[31518]: pgmap v4787: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:59:31.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:59:31 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8630.783988 secs 2024-03-28T02:59:31.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:59:31 smithi132 ceph-mon[35941]: pgmap v4787: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:59:31.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:59:31 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8630.783988 secs 2024-03-28T02:59:33.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:59:33 smithi165 ceph-mon[36302]: pgmap v4788: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:59:33.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:59:33 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:59:33.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:59:33 smithi119 ceph-mon[31518]: pgmap v4788: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:59:33.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:59:33 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:59:33.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:59:33 smithi132 ceph-mon[35941]: pgmap v4788: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:59:33.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:59:33 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T02:59:34.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:59:34 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:59:34.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:59:34 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:59:34.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:59:34 smithi165 ceph-mon[36302]: pgmap v4789: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T02:59:34.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:59:34 smithi165 ceph-mon[36302]: pgmap v4790: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:59:34.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:59:34 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:59:34.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:59:34 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:59:34.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:59:34 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:59:34.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:59:34 smithi119 ceph-mon[31518]: pgmap v4789: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T02:59:34.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:59:34 smithi119 ceph-mon[31518]: pgmap v4790: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:59:34.404 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:59:34 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:59:34.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:59:34 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T02:59:34.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:59:34 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T02:59:34.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:59:34 smithi132 ceph-mon[35941]: pgmap v4789: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T02:59:34.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:59:34 smithi132 ceph-mon[35941]: pgmap v4790: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:59:34.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:59:34 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T02:59:36.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:59:36 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8635.784048 secs 2024-03-28T02:59:36.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:59:36 smithi165 ceph-mon[36302]: pgmap v4791: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:59:36.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:59:36 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8635.784048 secs 2024-03-28T02:59:36.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:59:36 smithi119 ceph-mon[31518]: pgmap v4791: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:59:36.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:59:36 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8635.784048 secs 2024-03-28T02:59:36.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:59:36 smithi132 ceph-mon[35941]: pgmap v4791: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:59:38.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:59:38 smithi165 ceph-mon[36302]: pgmap v4792: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:59:38.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:59:38 smithi119 ceph-mon[31518]: pgmap v4792: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:59:38.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:59:38 smithi132 ceph-mon[35941]: pgmap v4792: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:59:40.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:59:40 smithi165 ceph-mon[36302]: pgmap v4793: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:59:40.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:59:40 smithi119 ceph-mon[31518]: pgmap v4793: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:59:40.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:59:40 smithi132 ceph-mon[35941]: pgmap v4793: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:59:41.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:59:41 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8640.784112 secs 2024-03-28T02:59:41.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:59:41 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8640.784112 secs 2024-03-28T02:59:41.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:59:41 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8640.784112 secs 2024-03-28T02:59:42.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:59:42 smithi165 ceph-mon[36302]: pgmap v4794: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:59:42.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:59:42 smithi119 ceph-mon[31518]: pgmap v4794: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:59:42.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:59:42 smithi132 ceph-mon[35941]: pgmap v4794: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T02:59:43.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:59:43 smithi165 ceph-mon[36302]: pgmap v4795: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T02:59:43.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:59:43 smithi119 ceph-mon[31518]: pgmap v4795: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T02:59:43.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:59:43 smithi132 ceph-mon[35941]: pgmap v4795: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T02:59:46.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:59:46 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8645.784175 secs 2024-03-28T02:59:46.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:59:46 smithi165 ceph-mon[36302]: pgmap v4796: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:59:46.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:59:46 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8645.784175 secs 2024-03-28T02:59:46.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:59:46 smithi119 ceph-mon[31518]: pgmap v4796: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:59:46.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:59:46 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8645.784175 secs 2024-03-28T02:59:46.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:59:46 smithi132 ceph-mon[35941]: pgmap v4796: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:59:48.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:59:48 smithi165 ceph-mon[36302]: pgmap v4797: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:59:48.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:59:48 smithi119 ceph-mon[31518]: pgmap v4797: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:59:48.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:59:48 smithi132 ceph-mon[35941]: pgmap v4797: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:59:50.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:59:50 smithi165 ceph-mon[36302]: pgmap v4798: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:59:50.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:59:50 smithi119 ceph-mon[31518]: pgmap v4798: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:59:50.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:59:50 smithi132 ceph-mon[35941]: pgmap v4798: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:59:51.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:59:51 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8650.784266 secs 2024-03-28T02:59:51.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:59:51 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8650.784266 secs 2024-03-28T02:59:51.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:59:51 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8650.784266 secs 2024-03-28T02:59:52.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:59:52 smithi165 ceph-mon[36302]: pgmap v4799: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:59:52.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:59:52 smithi119 ceph-mon[31518]: pgmap v4799: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:59:52.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:59:52 smithi132 ceph-mon[35941]: pgmap v4799: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:59:53.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:59:53 smithi165 ceph-mon[36302]: pgmap v4800: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:59:53.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:59:53 smithi119 ceph-mon[31518]: pgmap v4800: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:59:53.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:59:53 smithi132 ceph-mon[35941]: pgmap v4800: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:59:56.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:59:56 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8655.784354 secs 2024-03-28T02:59:56.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:59:56 smithi165 ceph-mon[36302]: pgmap v4801: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:59:56.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:59:56 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8655.784354 secs 2024-03-28T02:59:56.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:59:56 smithi119 ceph-mon[31518]: pgmap v4801: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:59:56.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:59:56 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8655.784354 secs 2024-03-28T02:59:56.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:59:56 smithi132 ceph-mon[35941]: pgmap v4801: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T02:59:58.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 02:59:58 smithi165 ceph-mon[36302]: pgmap v4802: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:59:58.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 02:59:58 smithi119 ceph-mon[31518]: pgmap v4802: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T02:59:58.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 02:59:58 smithi132 ceph-mon[35941]: pgmap v4802: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:00:00.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:00:00 smithi165 ceph-mon[36302]: pgmap v4803: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:00:00.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:00:00 smithi165 ceph-mon[36302]: overall HEALTH_WARN 5 failed cephadm daemon(s); 1 filesystem is degraded; 1 MDSs report slow requests 2024-03-28T03:00:00.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:00:00 smithi119 ceph-mon[31518]: pgmap v4803: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:00:00.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:00:00 smithi119 ceph-mon[31518]: overall HEALTH_WARN 5 failed cephadm daemon(s); 1 filesystem is degraded; 1 MDSs report slow requests 2024-03-28T03:00:00.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:00:00 smithi132 ceph-mon[35941]: pgmap v4803: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:00:00.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:00:00 smithi132 ceph-mon[35941]: overall HEALTH_WARN 5 failed cephadm daemon(s); 1 filesystem is degraded; 1 MDSs report slow requests 2024-03-28T03:00:01.852 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:00:01 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8660.784412 secs 2024-03-28T03:00:01.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:00:01 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8660.784412 secs 2024-03-28T03:00:01.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:00:01 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8660.784412 secs 2024-03-28T03:00:02.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:00:02 smithi165 ceph-mon[36302]: pgmap v4804: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:00:02.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:00:02 smithi119 ceph-mon[31518]: pgmap v4804: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:00:02.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:00:02 smithi132 ceph-mon[35941]: pgmap v4804: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:00:03.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:00:03 smithi165 ceph-mon[36302]: pgmap v4805: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:00:03.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:00:03 smithi119 ceph-mon[31518]: pgmap v4805: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:00:03.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:00:03 smithi132 ceph-mon[35941]: pgmap v4805: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:00:06.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:00:05 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8665.784455 secs 2024-03-28T03:00:06.381 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:00:05 smithi165 ceph-mon[36302]: pgmap v4806: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:00:06.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:00:05 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8665.784455 secs 2024-03-28T03:00:06.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:00:05 smithi119 ceph-mon[31518]: pgmap v4806: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:00:06.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:00:06 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8665.784455 secs 2024-03-28T03:00:06.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:00:06 smithi132 ceph-mon[35941]: pgmap v4806: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:00:08.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:00:08 smithi165 ceph-mon[36302]: pgmap v4807: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:00:08.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:00:08 smithi119 ceph-mon[31518]: pgmap v4807: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:00:08.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:00:08 smithi132 ceph-mon[35941]: pgmap v4807: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:00:10.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:00:10 smithi165 ceph-mon[36302]: pgmap v4808: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:00:10.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:00:10 smithi119 ceph-mon[31518]: pgmap v4808: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:00:10.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:00:10 smithi132 ceph-mon[35941]: pgmap v4808: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:00:11.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:00:11 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8670.784545 secs 2024-03-28T03:00:11.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:00:11 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8670.784545 secs 2024-03-28T03:00:11.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:00:11 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8670.784545 secs 2024-03-28T03:00:12.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:00:12 smithi165 ceph-mon[36302]: pgmap v4809: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:00:12.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:00:12 smithi119 ceph-mon[31518]: pgmap v4809: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:00:12.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:00:12 smithi132 ceph-mon[35941]: pgmap v4809: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:00:13.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:00:13 smithi165 ceph-mon[36302]: pgmap v4810: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:00:13.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:00:13 smithi119 ceph-mon[31518]: pgmap v4810: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:00:13.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:00:13 smithi132 ceph-mon[35941]: pgmap v4810: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:00:16.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:00:16 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8675.784632 secs 2024-03-28T03:00:16.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:00:16 smithi165 ceph-mon[36302]: pgmap v4811: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:00:16.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:00:16 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8675.784632 secs 2024-03-28T03:00:16.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:00:16 smithi119 ceph-mon[31518]: pgmap v4811: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:00:16.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:00:16 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8675.784632 secs 2024-03-28T03:00:16.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:00:16 smithi132 ceph-mon[35941]: pgmap v4811: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:00:18.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:00:18 smithi165 ceph-mon[36302]: pgmap v4812: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:00:18.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:00:18 smithi119 ceph-mon[31518]: pgmap v4812: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:00:18.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:00:18 smithi132 ceph-mon[35941]: pgmap v4812: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:00:19.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:00:19 smithi165 ceph-mon[36302]: pgmap v4813: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:00:19.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:00:19 smithi119 ceph-mon[31518]: pgmap v4813: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:00:19.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:00:19 smithi132 ceph-mon[35941]: pgmap v4813: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:00:21.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:00:20 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8680.784694 secs 2024-03-28T03:00:21.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:00:20 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8680.784694 secs 2024-03-28T03:00:21.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:00:20 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8680.784694 secs 2024-03-28T03:00:22.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:00:21 smithi119 ceph-mon[31518]: pgmap v4814: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:00:22.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:00:21 smithi132 ceph-mon[35941]: pgmap v4814: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:00:22.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:00:21 smithi165 ceph-mon[36302]: pgmap v4814: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:00:24.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:00:24 smithi165 ceph-mon[36302]: pgmap v4815: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:00:24.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:00:24 smithi119 ceph-mon[31518]: pgmap v4815: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:00:24.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:00:24 smithi132 ceph-mon[35941]: pgmap v4815: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:00:26.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:00:26 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8685.784782 secs 2024-03-28T03:00:26.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:00:26 smithi165 ceph-mon[36302]: pgmap v4816: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:00:26.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:00:26 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8685.784782 secs 2024-03-28T03:00:26.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:00:26 smithi119 ceph-mon[31518]: pgmap v4816: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:00:26.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:00:26 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8685.784782 secs 2024-03-28T03:00:26.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:00:26 smithi132 ceph-mon[35941]: pgmap v4816: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:00:27.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:00:27 smithi165 ceph-mon[36302]: pgmap v4817: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:00:27.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:00:27 smithi119 ceph-mon[31518]: pgmap v4817: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:00:27.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:00:27 smithi132 ceph-mon[35941]: pgmap v4817: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:00:30.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:00:30 smithi165 ceph-mon[36302]: pgmap v4818: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:00:30.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:00:30 smithi119 ceph-mon[31518]: pgmap v4818: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:00:30.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:00:30 smithi132 ceph-mon[35941]: pgmap v4818: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:00:31.852 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:00:31 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8690.784842 secs 2024-03-28T03:00:31.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:00:31 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8690.784842 secs 2024-03-28T03:00:31.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:00:31 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8690.784842 secs 2024-03-28T03:00:32.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:00:32 smithi165 ceph-mon[36302]: pgmap v4819: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:00:32.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:00:32 smithi119 ceph-mon[31518]: pgmap v4819: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:00:32.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:00:32 smithi132 ceph-mon[35941]: pgmap v4819: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:00:33.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:00:33 smithi165 ceph-mon[36302]: pgmap v4820: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:00:33.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:00:33 smithi119 ceph-mon[31518]: pgmap v4820: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:00:33.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:00:33 smithi132 ceph-mon[35941]: pgmap v4820: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:00:34.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:00:34 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T03:00:34.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:00:34 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T03:00:34.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:00:34 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T03:00:35.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:00:35 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T03:00:35.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:00:35 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T03:00:35.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:00:35 smithi165 ceph-mon[36302]: pgmap v4821: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T03:00:35.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:00:35 smithi165 ceph-mon[36302]: pgmap v4822: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:00:35.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:00:35 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:00:35.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:00:35 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T03:00:35.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:00:35 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T03:00:35.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:00:35 smithi119 ceph-mon[31518]: pgmap v4821: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T03:00:35.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:00:35 smithi119 ceph-mon[31518]: pgmap v4822: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:00:35.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:00:35 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:00:35.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:00:35 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T03:00:35.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:00:35 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T03:00:35.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:00:35 smithi132 ceph-mon[35941]: pgmap v4821: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T03:00:35.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:00:35 smithi132 ceph-mon[35941]: pgmap v4822: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:00:35.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:00:35 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:00:36.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:00:36 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8695.784907 secs 2024-03-28T03:00:36.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:00:36 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8695.784907 secs 2024-03-28T03:00:36.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:00:36 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8695.784907 secs 2024-03-28T03:00:37.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:00:37 smithi165 ceph-mon[36302]: pgmap v4823: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:00:37.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:00:37 smithi119 ceph-mon[31518]: pgmap v4823: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:00:37.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:00:37 smithi132 ceph-mon[35941]: pgmap v4823: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:00:40.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:00:40 smithi165 ceph-mon[36302]: pgmap v4824: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-28T03:00:40.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:00:40 smithi119 ceph-mon[31518]: pgmap v4824: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-28T03:00:40.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:00:40 smithi132 ceph-mon[35941]: pgmap v4824: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-28T03:00:41.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:00:41 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8700.784968 secs 2024-03-28T03:00:41.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:00:41 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8700.784968 secs 2024-03-28T03:00:41.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:00:41 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8700.784968 secs 2024-03-28T03:00:42.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:00:42 smithi165 ceph-mon[36302]: pgmap v4825: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:00:42.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:00:42 smithi119 ceph-mon[31518]: pgmap v4825: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:00:42.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:00:42 smithi132 ceph-mon[35941]: pgmap v4825: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:00:44.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:00:44 smithi165 ceph-mon[36302]: pgmap v4826: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:00:44.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:00:44 smithi119 ceph-mon[31518]: pgmap v4826: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:00:44.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:00:44 smithi132 ceph-mon[35941]: pgmap v4826: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:00:46.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:00:46 smithi165 ceph-mon[36302]: pgmap v4827: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-28T03:00:46.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:00:46 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8705.785057 secs 2024-03-28T03:00:46.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:00:46 smithi119 ceph-mon[31518]: pgmap v4827: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-28T03:00:46.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:00:46 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8705.785057 secs 2024-03-28T03:00:46.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:00:46 smithi132 ceph-mon[35941]: pgmap v4827: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-28T03:00:46.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:00:46 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8705.785057 secs 2024-03-28T03:00:48.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:00:48 smithi165 ceph-mon[36302]: pgmap v4828: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T03:00:48.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:00:48 smithi119 ceph-mon[31518]: pgmap v4828: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T03:00:48.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:00:48 smithi132 ceph-mon[35941]: pgmap v4828: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T03:00:50.340 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:00:50 smithi119 ceph-mon[31518]: pgmap v4829: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T03:00:50.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:00:50 smithi165 ceph-mon[36302]: pgmap v4829: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T03:00:50.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:00:50 smithi132 ceph-mon[35941]: pgmap v4829: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T03:00:51.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:00:51 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8710.785120 secs 2024-03-28T03:00:51.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:00:51 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8710.785120 secs 2024-03-28T03:00:51.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:00:51 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8710.785120 secs 2024-03-28T03:00:52.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:00:52 smithi165 ceph-mon[36302]: pgmap v4830: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:00:52.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:00:52 smithi119 ceph-mon[31518]: pgmap v4830: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:00:52.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:00:52 smithi132 ceph-mon[35941]: pgmap v4830: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:00:54.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:00:54 smithi165 ceph-mon[36302]: pgmap v4831: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:00:54.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:00:54 smithi119 ceph-mon[31518]: pgmap v4831: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:00:54.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:00:54 smithi132 ceph-mon[35941]: pgmap v4831: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:00:56.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:00:56 smithi165 ceph-mon[36302]: pgmap v4832: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:00:56.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:00:56 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8715.785137 secs 2024-03-28T03:00:56.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:00:56 smithi119 ceph-mon[31518]: pgmap v4832: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:00:56.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:00:56 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8715.785137 secs 2024-03-28T03:00:56.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:00:56 smithi132 ceph-mon[35941]: pgmap v4832: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:00:56.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:00:56 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8715.785137 secs 2024-03-28T03:00:58.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:00:58 smithi165 ceph-mon[36302]: pgmap v4833: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:00:58.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:00:58 smithi119 ceph-mon[31518]: pgmap v4833: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:00:58.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:00:58 smithi132 ceph-mon[35941]: pgmap v4833: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:01:00.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:01:00 smithi165 ceph-mon[36302]: pgmap v4834: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:01:00.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:01:00 smithi119 ceph-mon[31518]: pgmap v4834: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:01:00.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:01:00 smithi132 ceph-mon[35941]: pgmap v4834: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:01:01.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:01:01 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8720.785198 secs 2024-03-28T03:01:01.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:01:01 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8720.785198 secs 2024-03-28T03:01:01.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:01:01 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8720.785198 secs 2024-03-28T03:01:02.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:01:02 smithi165 ceph-mon[36302]: pgmap v4835: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:01:02.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:01:02 smithi119 ceph-mon[31518]: pgmap v4835: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:01:02.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:01:02 smithi132 ceph-mon[35941]: pgmap v4835: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:01:04.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:01:04 smithi165 ceph-mon[36302]: pgmap v4836: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:01:04.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:01:04 smithi119 ceph-mon[31518]: pgmap v4836: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:01:04.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:01:04 smithi132 ceph-mon[35941]: pgmap v4836: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:01:06.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:01:06 smithi119 ceph-mon[31518]: pgmap v4837: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:01:06.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:01:06 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8725.785234 secs 2024-03-28T03:01:06.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:01:06 smithi132 ceph-mon[35941]: pgmap v4837: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:01:06.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:01:06 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8725.785234 secs 2024-03-28T03:01:06.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:01:06 smithi165 ceph-mon[36302]: pgmap v4837: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:01:06.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:01:06 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8725.785234 secs 2024-03-28T03:01:08.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:01:08 smithi119 ceph-mon[31518]: pgmap v4838: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:01:08.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:01:08 smithi132 ceph-mon[35941]: pgmap v4838: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:01:08.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:01:08 smithi165 ceph-mon[36302]: pgmap v4838: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:01:10.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:01:10 smithi119 ceph-mon[31518]: pgmap v4839: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:01:10.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:01:10 smithi132 ceph-mon[35941]: pgmap v4839: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:01:10.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:01:10 smithi165 ceph-mon[36302]: pgmap v4839: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:01:11.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:01:11 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8730.785296 secs 2024-03-28T03:01:11.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:01:11 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8730.785296 secs 2024-03-28T03:01:11.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:01:11 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8730.785296 secs 2024-03-28T03:01:12.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:01:12 smithi119 ceph-mon[31518]: pgmap v4840: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:01:12.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:01:12 smithi132 ceph-mon[35941]: pgmap v4840: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:01:12.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:01:12 smithi165 ceph-mon[36302]: pgmap v4840: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:01:14.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:01:14 smithi132 ceph-mon[35941]: pgmap v4841: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:01:14.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:01:14 smithi165 ceph-mon[36302]: pgmap v4841: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:01:14.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:01:14 smithi119 ceph-mon[31518]: pgmap v4841: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:01:16.432 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:01:16 smithi165 ceph-mon[36302]: pgmap v4842: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:01:16.432 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:01:16 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8735.785358 secs 2024-03-28T03:01:16.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:01:16 smithi132 ceph-mon[35941]: pgmap v4842: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:01:16.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:01:16 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8735.785358 secs 2024-03-28T03:01:16.456 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:01:16 smithi119 ceph-mon[31518]: pgmap v4842: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:01:16.456 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:01:16 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8735.785358 secs 2024-03-28T03:01:18.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:01:18 smithi132 ceph-mon[35941]: pgmap v4843: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:01:18.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:01:18 smithi165 ceph-mon[36302]: pgmap v4843: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:01:18.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:01:18 smithi119 ceph-mon[31518]: pgmap v4843: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:01:20.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:01:20 smithi132 ceph-mon[35941]: pgmap v4844: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:01:20.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:01:20 smithi165 ceph-mon[36302]: pgmap v4844: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:01:20.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:01:20 smithi119 ceph-mon[31518]: pgmap v4844: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:01:21.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:01:21 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8740.785416 secs 2024-03-28T03:01:21.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:01:21 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8740.785416 secs 2024-03-28T03:01:21.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:01:21 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8740.785416 secs 2024-03-28T03:01:22.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:01:22 smithi119 ceph-mon[31518]: pgmap v4845: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:01:22.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:01:22 smithi132 ceph-mon[35941]: pgmap v4845: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:01:22.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:01:22 smithi165 ceph-mon[36302]: pgmap v4845: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:01:24.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:01:24 smithi132 ceph-mon[35941]: pgmap v4846: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:01:24.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:01:24 smithi165 ceph-mon[36302]: pgmap v4846: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:01:24.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:01:24 smithi119 ceph-mon[31518]: pgmap v4846: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:01:26.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:01:26 smithi132 ceph-mon[35941]: pgmap v4847: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:01:26.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:01:26 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8745.785500 secs 2024-03-28T03:01:26.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:01:26 smithi165 ceph-mon[36302]: pgmap v4847: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:01:26.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:01:26 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8745.785500 secs 2024-03-28T03:01:26.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:01:26 smithi119 ceph-mon[31518]: pgmap v4847: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:01:26.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:01:26 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8745.785500 secs 2024-03-28T03:01:28.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:01:28 smithi132 ceph-mon[35941]: pgmap v4848: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:01:28.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:01:28 smithi165 ceph-mon[36302]: pgmap v4848: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:01:28.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:01:28 smithi119 ceph-mon[31518]: pgmap v4848: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:01:30.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:01:30 smithi132 ceph-mon[35941]: pgmap v4849: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:01:30.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:01:30 smithi165 ceph-mon[36302]: pgmap v4849: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:01:30.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:01:30 smithi119 ceph-mon[31518]: pgmap v4849: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:01:31.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:01:31 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8750.785560 secs 2024-03-28T03:01:31.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:01:31 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8750.785560 secs 2024-03-28T03:01:31.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:01:31 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8750.785560 secs 2024-03-28T03:01:32.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:01:32 smithi165 ceph-mon[36302]: pgmap v4850: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:01:32.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:01:32 smithi119 ceph-mon[31518]: pgmap v4850: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:01:32.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:01:32 smithi132 ceph-mon[35941]: pgmap v4850: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:01:34.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:01:34 smithi165 ceph-mon[36302]: pgmap v4851: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:01:34.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:01:34 smithi119 ceph-mon[31518]: pgmap v4851: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:01:34.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:01:34 smithi132 ceph-mon[35941]: pgmap v4851: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:01:36.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:01:36 smithi165 ceph-mon[36302]: pgmap v4852: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:01:36.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:01:36 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8755.785622 secs 2024-03-28T03:01:36.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:01:36 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T03:01:36.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:01:36 smithi119 ceph-mon[31518]: pgmap v4852: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:01:36.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:01:36 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8755.785622 secs 2024-03-28T03:01:36.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:01:36 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T03:01:36.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:01:36 smithi132 ceph-mon[35941]: pgmap v4852: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:01:36.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:01:36 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8755.785622 secs 2024-03-28T03:01:36.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:01:36 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T03:01:37.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:01:37 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T03:01:37.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:01:37 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T03:01:37.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:01:37 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:01:37.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:01:37 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T03:01:37.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:01:37 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T03:01:37.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:01:37 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:01:37.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:01:37 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T03:01:37.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:01:37 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T03:01:37.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:01:37 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:01:38.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:01:38 smithi165 ceph-mon[36302]: pgmap v4853: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T03:01:38.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:01:38 smithi165 ceph-mon[36302]: pgmap v4854: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:01:38.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:01:38 smithi119 ceph-mon[31518]: pgmap v4853: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T03:01:38.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:01:38 smithi119 ceph-mon[31518]: pgmap v4854: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:01:38.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:01:38 smithi132 ceph-mon[35941]: pgmap v4853: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T03:01:38.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:01:38 smithi132 ceph-mon[35941]: pgmap v4854: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:01:40.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:01:40 smithi165 ceph-mon[36302]: pgmap v4855: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:01:40.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:01:40 smithi119 ceph-mon[31518]: pgmap v4855: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:01:40.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:01:40 smithi132 ceph-mon[35941]: pgmap v4855: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:01:41.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:01:41 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8760.785710 secs 2024-03-28T03:01:41.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:01:41 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8760.785710 secs 2024-03-28T03:01:41.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:01:41 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8760.785710 secs 2024-03-28T03:01:42.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:01:42 smithi165 ceph-mon[36302]: pgmap v4856: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:01:42.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:01:42 smithi119 ceph-mon[31518]: pgmap v4856: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:01:42.701 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:01:42 smithi132 ceph-mon[35941]: pgmap v4856: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:01:44.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:01:44 smithi165 ceph-mon[36302]: pgmap v4857: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:01:44.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:01:44 smithi119 ceph-mon[31518]: pgmap v4857: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:01:44.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:01:44 smithi132 ceph-mon[35941]: pgmap v4857: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:01:46.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:01:46 smithi165 ceph-mon[36302]: pgmap v4858: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:01:46.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:01:46 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8765.785751 secs 2024-03-28T03:01:46.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:01:46 smithi119 ceph-mon[31518]: pgmap v4858: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:01:46.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:01:46 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8765.785751 secs 2024-03-28T03:01:46.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:01:46 smithi132 ceph-mon[35941]: pgmap v4858: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:01:46.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:01:46 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8765.785751 secs 2024-03-28T03:01:48.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:01:48 smithi165 ceph-mon[36302]: pgmap v4859: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T03:01:48.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:01:48 smithi119 ceph-mon[31518]: pgmap v4859: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T03:01:48.701 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:01:48 smithi132 ceph-mon[35941]: pgmap v4859: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T03:01:50.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:01:50 smithi165 ceph-mon[36302]: pgmap v4860: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:01:50.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:01:50 smithi119 ceph-mon[31518]: pgmap v4860: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:01:50.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:01:50 smithi132 ceph-mon[35941]: pgmap v4860: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:01:51.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:01:51 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8770.785812 secs 2024-03-28T03:01:51.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:01:51 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8770.785812 secs 2024-03-28T03:01:51.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:01:51 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8770.785812 secs 2024-03-28T03:01:52.555 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:01:52 smithi119 ceph-mon[31518]: pgmap v4861: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:01:52.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:01:52 smithi165 ceph-mon[36302]: pgmap v4861: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:01:52.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:01:52 smithi132 ceph-mon[35941]: pgmap v4861: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:01:54.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:01:54 smithi165 ceph-mon[36302]: pgmap v4862: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:01:54.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:01:54 smithi119 ceph-mon[31518]: pgmap v4862: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:01:54.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:01:54 smithi132 ceph-mon[35941]: pgmap v4862: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:01:56.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:01:56 smithi165 ceph-mon[36302]: pgmap v4863: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:01:56.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:01:56 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8775.785915 secs 2024-03-28T03:01:56.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:01:56 smithi119 ceph-mon[31518]: pgmap v4863: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:01:56.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:01:56 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8775.785915 secs 2024-03-28T03:01:56.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:01:56 smithi132 ceph-mon[35941]: pgmap v4863: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:01:56.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:01:56 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8775.785915 secs 2024-03-28T03:01:58.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:01:58 smithi165 ceph-mon[36302]: pgmap v4864: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:01:58.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:01:58 smithi119 ceph-mon[31518]: pgmap v4864: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:01:58.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:01:58 smithi132 ceph-mon[35941]: pgmap v4864: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:02:00.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:02:00 smithi165 ceph-mon[36302]: pgmap v4865: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:02:00.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:02:00 smithi119 ceph-mon[31518]: pgmap v4865: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:02:00.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:02:00 smithi132 ceph-mon[35941]: pgmap v4865: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:02:01.601 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:02:01 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8780.785974 secs 2024-03-28T03:02:01.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:02:01 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8780.785974 secs 2024-03-28T03:02:01.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:02:01 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8780.785974 secs 2024-03-28T03:02:02.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:02:02 smithi165 ceph-mon[36302]: pgmap v4866: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:02:02.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:02:02 smithi119 ceph-mon[31518]: pgmap v4866: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:02:02.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:02:02 smithi132 ceph-mon[35941]: pgmap v4866: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:02:04.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:02:04 smithi165 ceph-mon[36302]: pgmap v4867: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:02:04.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:02:04 smithi119 ceph-mon[31518]: pgmap v4867: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:02:04.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:02:04 smithi132 ceph-mon[35941]: pgmap v4867: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:02:06.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:02:06 smithi165 ceph-mon[36302]: pgmap v4868: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:02:06.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:02:06 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8785.786062 secs 2024-03-28T03:02:06.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:02:06 smithi119 ceph-mon[31518]: pgmap v4868: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:02:06.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:02:06 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8785.786062 secs 2024-03-28T03:02:06.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:02:06 smithi132 ceph-mon[35941]: pgmap v4868: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:02:06.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:02:06 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8785.786062 secs 2024-03-28T03:02:08.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:02:08 smithi165 ceph-mon[36302]: pgmap v4869: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:02:08.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:02:08 smithi119 ceph-mon[31518]: pgmap v4869: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:02:08.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:02:08 smithi132 ceph-mon[35941]: pgmap v4869: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:02:10.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:02:10 smithi165 ceph-mon[36302]: pgmap v4870: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:02:10.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:02:10 smithi119 ceph-mon[31518]: pgmap v4870: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:02:10.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:02:10 smithi132 ceph-mon[35941]: pgmap v4870: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:02:11.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:02:11 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8790.786151 secs 2024-03-28T03:02:11.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:02:11 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8790.786151 secs 2024-03-28T03:02:11.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:02:11 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8790.786151 secs 2024-03-28T03:02:12.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:02:12 smithi165 ceph-mon[36302]: pgmap v4871: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:02:12.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:02:12 smithi119 ceph-mon[31518]: pgmap v4871: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:02:12.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:02:12 smithi132 ceph-mon[35941]: pgmap v4871: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:02:14.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:02:14 smithi165 ceph-mon[36302]: pgmap v4872: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:02:14.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:02:14 smithi119 ceph-mon[31518]: pgmap v4872: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:02:14.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:02:14 smithi132 ceph-mon[35941]: pgmap v4872: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:02:16.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:02:16 smithi165 ceph-mon[36302]: pgmap v4873: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:02:16.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:02:16 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8795.786211 secs 2024-03-28T03:02:16.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:02:16 smithi119 ceph-mon[31518]: pgmap v4873: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:02:16.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:02:16 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8795.786211 secs 2024-03-28T03:02:16.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:02:16 smithi132 ceph-mon[35941]: pgmap v4873: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:02:16.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:02:16 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8795.786211 secs 2024-03-28T03:02:18.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:02:18 smithi165 ceph-mon[36302]: pgmap v4874: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:02:18.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:02:18 smithi119 ceph-mon[31518]: pgmap v4874: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:02:18.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:02:18 smithi132 ceph-mon[35941]: pgmap v4874: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:02:20.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:02:20 smithi165 ceph-mon[36302]: pgmap v4875: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:02:20.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:02:20 smithi119 ceph-mon[31518]: pgmap v4875: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:02:20.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:02:20 smithi132 ceph-mon[35941]: pgmap v4875: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:02:21.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:02:21 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8800.786271 secs 2024-03-28T03:02:21.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:02:21 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8800.786271 secs 2024-03-28T03:02:21.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:02:21 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8800.786271 secs 2024-03-28T03:02:22.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:02:22 smithi119 ceph-mon[31518]: pgmap v4876: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:02:22.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:02:22 smithi132 ceph-mon[35941]: pgmap v4876: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:02:22.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:02:22 smithi165 ceph-mon[36302]: pgmap v4876: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:02:24.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:02:24 smithi119 ceph-mon[31518]: pgmap v4877: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:02:24.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:02:24 smithi132 ceph-mon[35941]: pgmap v4877: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:02:24.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:02:24 smithi165 ceph-mon[36302]: pgmap v4877: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:02:26.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:02:26 smithi119 ceph-mon[31518]: pgmap v4878: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:02:26.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:02:26 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8805.786332 secs 2024-03-28T03:02:26.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:02:26 smithi132 ceph-mon[35941]: pgmap v4878: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:02:26.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:02:26 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8805.786332 secs 2024-03-28T03:02:26.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:02:26 smithi165 ceph-mon[36302]: pgmap v4878: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:02:26.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:02:26 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8805.786332 secs 2024-03-28T03:02:28.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:02:28 smithi119 ceph-mon[31518]: pgmap v4879: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:02:28.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:02:28 smithi132 ceph-mon[35941]: pgmap v4879: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:02:28.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:02:28 smithi165 ceph-mon[36302]: pgmap v4879: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:02:30.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:02:30 smithi132 ceph-mon[35941]: pgmap v4880: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:02:30.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:02:30 smithi165 ceph-mon[36302]: pgmap v4880: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:02:30.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:02:30 smithi119 ceph-mon[31518]: pgmap v4880: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:02:31.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:02:31 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8810.786369 secs 2024-03-28T03:02:31.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:02:31 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8810.786369 secs 2024-03-28T03:02:31.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:02:31 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8810.786369 secs 2024-03-28T03:02:32.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:02:32 smithi132 ceph-mon[35941]: pgmap v4881: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:02:32.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:02:32 smithi165 ceph-mon[36302]: pgmap v4881: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:02:32.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:02:32 smithi119 ceph-mon[31518]: pgmap v4881: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:02:34.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:02:34 smithi132 ceph-mon[35941]: pgmap v4882: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:02:34.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:02:34 smithi165 ceph-mon[36302]: pgmap v4882: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:02:34.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:02:34 smithi119 ceph-mon[31518]: pgmap v4882: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:02:36.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:02:36 smithi132 ceph-mon[35941]: pgmap v4883: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:02:36.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:02:36 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8815.786459 secs 2024-03-28T03:02:36.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:02:36 smithi165 ceph-mon[36302]: pgmap v4883: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:02:36.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:02:36 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8815.786459 secs 2024-03-28T03:02:36.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:02:36 smithi119 ceph-mon[31518]: pgmap v4883: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:02:36.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:02:36 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8815.786459 secs 2024-03-28T03:02:37.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:02:37 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T03:02:37.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:02:37 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:02:37.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:02:37 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:02:37.874 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:02:37 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T03:02:37.874 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:02:37 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:02:37.874 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:02:37 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:02:37.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:02:37 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T03:02:37.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:02:37 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:02:37.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:02:37 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:02:38.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:02:38 smithi132 ceph-mon[35941]: pgmap v4884: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:02:38.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:02:38 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:02:38.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:02:38 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:02:38.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:02:38 smithi165 ceph-mon[36302]: pgmap v4884: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:02:38.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:02:38 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:02:38.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:02:38 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:02:38.883 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:02:38 smithi119 ceph-mon[31518]: pgmap v4884: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:02:38.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:02:38 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:02:38.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:02:38 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:02:40.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:02:40 smithi132 ceph-mon[35941]: pgmap v4885: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:02:40.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:02:40 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:02:40.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:02:40 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:02:40.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:02:40 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:02:40.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:02:40 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:02:40.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:02:40 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:02:40.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:02:40 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:02:40.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:02:40 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:02:40.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:02:40 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:02:40.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:02:40 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:02:40.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:02:40 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:02:40.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:02:40 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:02:40.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:02:40 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:02:40.452 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:02:40 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:02:40.452 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:02:40 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:02:40.452 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:02:40 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:02:40.452 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:02:40 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:02:40.452 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:02:40 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:02:40.452 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:02:40 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:02:40.452 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:02:40 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:02:40.452 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:02:40 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:02:40.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:02:40 smithi165 ceph-mon[36302]: pgmap v4885: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:02:40.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:02:40 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:02:40.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:02:40 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:02:40.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:02:40 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:02:40.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:02:40 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:02:40.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:02:40 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:02:40.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:02:40 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:02:40.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:02:40 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:02:40.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:02:40 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:02:40.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:02:40 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:02:40.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:02:40 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:02:40.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:02:40 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:02:40.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:02:40 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:02:40.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:02:40 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:02:40.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:02:40 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:02:40.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:02:40 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:02:40.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:02:40 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:02:40.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:02:40 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:02:40.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:02:40 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:02:40.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:02:40 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:02:40.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:02:40 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:02:40.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:02:40 smithi119 ceph-mon[31518]: pgmap v4885: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:02:40.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:02:40 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:02:40.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:02:40 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:02:40.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:02:40 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:02:40.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:02:40 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:02:40.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:02:40 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:02:40.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:02:40 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:02:40.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:02:40 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:02:40.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:02:40 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:02:40.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:02:40 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:02:40.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:02:40 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:02:40.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:02:40 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:02:40.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:02:40 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:02:40.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:02:40 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:02:40.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:02:40 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:02:40.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:02:40 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:02:40.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:02:40 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:02:40.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:02:40 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:02:40.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:02:40 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:02:40.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:02:40 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:02:40.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:02:40 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:02:41.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:02:41 smithi132 ceph-mon[35941]: Adjusting osd_memory_target on smithi132 to 174.1M 2024-03-28T03:02:41.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:02:41 smithi132 ceph-mon[35941]: Unable to set osd_memory_target on smithi132 to 182640640: error parsing value: Value '182640640' is below minimum 939524096 2024-03-28T03:02:41.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:02:41 smithi132 ceph-mon[35941]: Adjusting osd_memory_target on smithi165 to 174.1M 2024-03-28T03:02:41.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:02:41 smithi132 ceph-mon[35941]: Unable to set osd_memory_target on smithi165 to 182635622: error parsing value: Value '182635622' is below minimum 939524096 2024-03-28T03:02:41.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:02:41 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8820.786521 secs 2024-03-28T03:02:41.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:02:41 smithi165 ceph-mon[36302]: Adjusting osd_memory_target on smithi132 to 174.1M 2024-03-28T03:02:41.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:02:41 smithi165 ceph-mon[36302]: Unable to set osd_memory_target on smithi132 to 182640640: error parsing value: Value '182640640' is below minimum 939524096 2024-03-28T03:02:41.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:02:41 smithi165 ceph-mon[36302]: Adjusting osd_memory_target on smithi165 to 174.1M 2024-03-28T03:02:41.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:02:41 smithi165 ceph-mon[36302]: Unable to set osd_memory_target on smithi165 to 182635622: error parsing value: Value '182635622' is below minimum 939524096 2024-03-28T03:02:41.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:02:41 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8820.786521 secs 2024-03-28T03:02:41.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:02:41 smithi119 ceph-mon[31518]: Adjusting osd_memory_target on smithi132 to 174.1M 2024-03-28T03:02:41.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:02:41 smithi119 ceph-mon[31518]: Unable to set osd_memory_target on smithi132 to 182640640: error parsing value: Value '182640640' is below minimum 939524096 2024-03-28T03:02:41.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:02:41 smithi119 ceph-mon[31518]: Adjusting osd_memory_target on smithi165 to 174.1M 2024-03-28T03:02:41.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:02:41 smithi119 ceph-mon[31518]: Unable to set osd_memory_target on smithi165 to 182635622: error parsing value: Value '182635622' is below minimum 939524096 2024-03-28T03:02:41.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:02:41 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8820.786521 secs 2024-03-28T03:02:42.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:02:42 smithi165 ceph-mon[36302]: pgmap v4886: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:02:42.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:02:42 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:02:42.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:02:42 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:02:42.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:02:42 smithi119 ceph-mon[31518]: pgmap v4886: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:02:42.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:02:42 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:02:42.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:02:42 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:02:42.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:02:42 smithi132 ceph-mon[35941]: pgmap v4886: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:02:42.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:02:42 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:02:42.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:02:42 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:02:43.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:02:43 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:02:43.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:02:43 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:02:43.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:02:43 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:02:43.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:02:43 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:02:43.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:02:43 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:02:43.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:02:43 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:02:43.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:02:43 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:02:43.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:02:43 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:02:43.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:02:43 smithi165 ceph-mon[36302]: Adjusting osd_memory_target on smithi119 to 176.7M 2024-03-28T03:02:43.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:02:43 smithi165 ceph-mon[36302]: Unable to set osd_memory_target on smithi119 to 185372364: error parsing value: Value '185372364' is below minimum 939524096 2024-03-28T03:02:43.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:02:43 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T03:02:43.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:02:43 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T03:02:43.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:02:43 smithi165 ceph-mon[36302]: pgmap v4887: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T03:02:43.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:02:43 smithi165 ceph-mon[36302]: pgmap v4888: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T03:02:43.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:02:43 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:02:43.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:02:43 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:02:43.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:02:43 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:02:43.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:02:43 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:02:43.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:02:43 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:02:43.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:02:43 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:02:43.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:02:43 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:02:43.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:02:43 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:02:43.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:02:43 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:02:43.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:02:43 smithi119 ceph-mon[31518]: Adjusting osd_memory_target on smithi119 to 176.7M 2024-03-28T03:02:43.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:02:43 smithi119 ceph-mon[31518]: Unable to set osd_memory_target on smithi119 to 185372364: error parsing value: Value '185372364' is below minimum 939524096 2024-03-28T03:02:43.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:02:43 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T03:02:43.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:02:43 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T03:02:43.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:02:43 smithi119 ceph-mon[31518]: pgmap v4887: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T03:02:43.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:02:43 smithi119 ceph-mon[31518]: pgmap v4888: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T03:02:43.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:02:43 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:02:43.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:02:43 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:02:43.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:02:43 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:02:43.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:02:43 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:02:43.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:02:43 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:02:43.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:02:43 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:02:43.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:02:43 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:02:43.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:02:43 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:02:43.701 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:02:43 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:02:43.701 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:02:43 smithi132 ceph-mon[35941]: Adjusting osd_memory_target on smithi119 to 176.7M 2024-03-28T03:02:43.701 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:02:43 smithi132 ceph-mon[35941]: Unable to set osd_memory_target on smithi119 to 185372364: error parsing value: Value '185372364' is below minimum 939524096 2024-03-28T03:02:43.701 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:02:43 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T03:02:43.701 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:02:43 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T03:02:43.701 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:02:43 smithi132 ceph-mon[35941]: pgmap v4887: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T03:02:43.701 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:02:43 smithi132 ceph-mon[35941]: pgmap v4888: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T03:02:43.701 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:02:43 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:02:45.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:02:45 smithi165 ceph-mon[36302]: pgmap v4889: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T03:02:45.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:02:45 smithi119 ceph-mon[31518]: pgmap v4889: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T03:02:45.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:02:45 smithi132 ceph-mon[35941]: pgmap v4889: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T03:02:46.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:02:46 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8825.786610 secs 2024-03-28T03:02:46.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:02:46 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8825.786610 secs 2024-03-28T03:02:46.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:02:46 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8825.786610 secs 2024-03-28T03:02:47.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:02:47 smithi165 ceph-mon[36302]: pgmap v4890: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T03:02:47.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:02:47 smithi119 ceph-mon[31518]: pgmap v4890: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T03:02:47.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:02:47 smithi132 ceph-mon[35941]: pgmap v4890: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T03:02:49.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:02:49 smithi165 ceph-mon[36302]: pgmap v4891: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T03:02:49.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:02:49 smithi119 ceph-mon[31518]: pgmap v4891: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T03:02:49.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:02:49 smithi132 ceph-mon[35941]: pgmap v4891: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T03:02:51.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:02:51 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8830.786670 secs 2024-03-28T03:02:51.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:02:51 smithi165 ceph-mon[36302]: pgmap v4892: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T03:02:51.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:02:51 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8830.786670 secs 2024-03-28T03:02:51.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:02:51 smithi119 ceph-mon[31518]: pgmap v4892: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T03:02:51.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:02:51 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8830.786670 secs 2024-03-28T03:02:51.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:02:51 smithi132 ceph-mon[35941]: pgmap v4892: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T03:02:52.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:02:52 smithi165 ceph-mon[36302]: pgmap v4893: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T03:02:52.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:02:52 smithi119 ceph-mon[31518]: pgmap v4893: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T03:02:52.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:02:52 smithi132 ceph-mon[35941]: pgmap v4893: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T03:02:55.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:02:55 smithi165 ceph-mon[36302]: pgmap v4894: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:02:55.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:02:55 smithi119 ceph-mon[31518]: pgmap v4894: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:02:55.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:02:55 smithi132 ceph-mon[35941]: pgmap v4894: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:02:56.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:02:56 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8835.786729 secs 2024-03-28T03:02:56.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:02:56 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8835.786729 secs 2024-03-28T03:02:56.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:02:56 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8835.786729 secs 2024-03-28T03:02:57.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:02:57 smithi165 ceph-mon[36302]: pgmap v4895: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:02:57.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:02:57 smithi119 ceph-mon[31518]: pgmap v4895: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:02:57.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:02:57 smithi132 ceph-mon[35941]: pgmap v4895: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:02:59.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:02:59 smithi165 ceph-mon[36302]: pgmap v4896: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:02:59.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:02:59 smithi119 ceph-mon[31518]: pgmap v4896: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:02:59.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:02:59 smithi132 ceph-mon[35941]: pgmap v4896: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:03:01.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:03:01 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8840.786820 secs 2024-03-28T03:03:01.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:03:01 smithi165 ceph-mon[36302]: pgmap v4897: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:03:01.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:03:01 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8840.786820 secs 2024-03-28T03:03:01.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:03:01 smithi119 ceph-mon[31518]: pgmap v4897: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:03:01.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:03:01 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8840.786820 secs 2024-03-28T03:03:01.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:03:01 smithi132 ceph-mon[35941]: pgmap v4897: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:03:02.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:03:02 smithi165 ceph-mon[36302]: pgmap v4898: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:03:02.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:03:02 smithi119 ceph-mon[31518]: pgmap v4898: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:03:02.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:03:02 smithi132 ceph-mon[35941]: pgmap v4898: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:03:05.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:03:05 smithi165 ceph-mon[36302]: pgmap v4899: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:03:05.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:03:05 smithi119 ceph-mon[31518]: pgmap v4899: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:03:05.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:03:05 smithi132 ceph-mon[35941]: pgmap v4899: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:03:06.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:03:06 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8845.786916 secs 2024-03-28T03:03:06.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:03:06 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8845.786916 secs 2024-03-28T03:03:06.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:03:06 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8845.786916 secs 2024-03-28T03:03:07.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:03:07 smithi165 ceph-mon[36302]: pgmap v4900: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T03:03:07.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:03:07 smithi119 ceph-mon[31518]: pgmap v4900: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T03:03:07.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:03:07 smithi132 ceph-mon[35941]: pgmap v4900: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T03:03:09.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:03:09 smithi165 ceph-mon[36302]: pgmap v4901: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T03:03:09.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:03:09 smithi119 ceph-mon[31518]: pgmap v4901: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T03:03:09.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:03:09 smithi132 ceph-mon[35941]: pgmap v4901: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T03:03:10.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:03:10 smithi165 ceph-mon[36302]: pgmap v4902: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:03:10.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:03:10 smithi119 ceph-mon[31518]: pgmap v4902: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:03:10.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:03:10 smithi132 ceph-mon[35941]: pgmap v4902: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:03:11.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:03:11 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8850.786979 secs 2024-03-28T03:03:11.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:03:11 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8850.786979 secs 2024-03-28T03:03:11.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:03:11 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8850.786979 secs 2024-03-28T03:03:12.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:03:12 smithi165 ceph-mon[36302]: pgmap v4903: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T03:03:12.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:03:12 smithi119 ceph-mon[31518]: pgmap v4903: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T03:03:12.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:03:12 smithi132 ceph-mon[35941]: pgmap v4903: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T03:03:15.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:03:15 smithi165 ceph-mon[36302]: pgmap v4904: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:03:15.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:03:15 smithi119 ceph-mon[31518]: pgmap v4904: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:03:15.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:03:15 smithi132 ceph-mon[35941]: pgmap v4904: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:03:16.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:03:16 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8855.786996 secs 2024-03-28T03:03:16.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:03:16 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8855.786996 secs 2024-03-28T03:03:16.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:03:16 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8855.786996 secs 2024-03-28T03:03:17.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:03:17 smithi165 ceph-mon[36302]: pgmap v4905: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:03:17.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:03:17 smithi119 ceph-mon[31518]: pgmap v4905: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:03:17.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:03:17 smithi132 ceph-mon[35941]: pgmap v4905: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:03:19.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:03:19 smithi165 ceph-mon[36302]: pgmap v4906: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:03:19.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:03:19 smithi119 ceph-mon[31518]: pgmap v4906: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:03:19.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:03:19 smithi132 ceph-mon[35941]: pgmap v4906: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:03:21.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:03:21 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8860.787060 secs 2024-03-28T03:03:21.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:03:21 smithi165 ceph-mon[36302]: pgmap v4907: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:03:21.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:03:21 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8860.787060 secs 2024-03-28T03:03:21.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:03:21 smithi119 ceph-mon[31518]: pgmap v4907: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:03:21.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:03:21 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8860.787060 secs 2024-03-28T03:03:21.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:03:21 smithi132 ceph-mon[35941]: pgmap v4907: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:03:23.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:03:23 smithi165 ceph-mon[36302]: pgmap v4908: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:03:23.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:03:23 smithi119 ceph-mon[31518]: pgmap v4908: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:03:23.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:03:23 smithi132 ceph-mon[35941]: pgmap v4908: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:03:25.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:03:25 smithi165 ceph-mon[36302]: pgmap v4909: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:03:25.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:03:25 smithi119 ceph-mon[31518]: pgmap v4909: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:03:25.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:03:25 smithi132 ceph-mon[35941]: pgmap v4909: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:03:26.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:03:26 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8865.787119 secs 2024-03-28T03:03:26.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:03:26 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8865.787119 secs 2024-03-28T03:03:26.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:03:26 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8865.787119 secs 2024-03-28T03:03:27.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:03:27 smithi165 ceph-mon[36302]: pgmap v4910: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:03:27.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:03:27 smithi119 ceph-mon[31518]: pgmap v4910: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:03:27.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:03:27 smithi132 ceph-mon[35941]: pgmap v4910: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:03:28.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:03:28 smithi165 ceph-mon[36302]: pgmap v4911: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:03:28.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:03:28 smithi119 ceph-mon[31518]: pgmap v4911: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:03:28.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:03:28 smithi132 ceph-mon[35941]: pgmap v4911: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:03:31.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:03:31 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8870.787206 secs 2024-03-28T03:03:31.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:03:31 smithi165 ceph-mon[36302]: pgmap v4912: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:03:31.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:03:31 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8870.787206 secs 2024-03-28T03:03:31.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:03:31 smithi119 ceph-mon[31518]: pgmap v4912: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:03:31.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:03:31 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8870.787206 secs 2024-03-28T03:03:31.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:03:31 smithi132 ceph-mon[35941]: pgmap v4912: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:03:33.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:03:33 smithi165 ceph-mon[36302]: pgmap v4913: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T03:03:33.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:03:33 smithi119 ceph-mon[31518]: pgmap v4913: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T03:03:33.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:03:33 smithi132 ceph-mon[35941]: pgmap v4913: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T03:03:35.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:03:35 smithi165 ceph-mon[36302]: pgmap v4914: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:03:35.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:03:35 smithi119 ceph-mon[31518]: pgmap v4914: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:03:35.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:03:35 smithi132 ceph-mon[35941]: pgmap v4914: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:03:36.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:03:36 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8875.787295 secs 2024-03-28T03:03:36.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:03:36 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8875.787295 secs 2024-03-28T03:03:36.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:03:36 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8875.787295 secs 2024-03-28T03:03:37.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:03:37 smithi165 ceph-mon[36302]: pgmap v4915: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T03:03:37.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:03:37 smithi119 ceph-mon[31518]: pgmap v4915: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T03:03:37.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:03:37 smithi132 ceph-mon[35941]: pgmap v4915: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T03:03:38.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:03:38 smithi165 ceph-mon[36302]: pgmap v4916: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T03:03:38.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:03:38 smithi119 ceph-mon[31518]: pgmap v4916: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T03:03:38.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:03:38 smithi132 ceph-mon[35941]: pgmap v4916: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T03:03:41.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:03:41 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8880.787356 secs 2024-03-28T03:03:41.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:03:41 smithi119 ceph-mon[31518]: pgmap v4917: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:03:41.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:03:41 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8880.787356 secs 2024-03-28T03:03:41.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:03:41 smithi165 ceph-mon[36302]: pgmap v4917: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:03:41.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:03:41 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8880.787356 secs 2024-03-28T03:03:41.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:03:41 smithi132 ceph-mon[35941]: pgmap v4917: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:03:43.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:03:43 smithi165 ceph-mon[36302]: pgmap v4918: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:03:43.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:03:43 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T03:03:43.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:03:43 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:03:43.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:03:43 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:03:43.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:03:43 smithi119 ceph-mon[31518]: pgmap v4918: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:03:43.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:03:43 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T03:03:43.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:03:43 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:03:43.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:03:43 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:03:43.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:03:43 smithi132 ceph-mon[35941]: pgmap v4918: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:03:43.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:03:43 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T03:03:43.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:03:43 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:03:43.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:03:43 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:03:44.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:03:44 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T03:03:44.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:03:44 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T03:03:44.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:03:44 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:03:44.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:03:44 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T03:03:44.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:03:44 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T03:03:44.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:03:44 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:03:44.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:03:44 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T03:03:44.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:03:44 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T03:03:44.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:03:44 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:03:45.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:03:45 smithi165 ceph-mon[36302]: pgmap v4919: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T03:03:45.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:03:45 smithi165 ceph-mon[36302]: pgmap v4920: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 4 op/s 2024-03-28T03:03:45.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:03:45 smithi119 ceph-mon[31518]: pgmap v4919: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T03:03:45.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:03:45 smithi119 ceph-mon[31518]: pgmap v4920: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 4 op/s 2024-03-28T03:03:45.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:03:45 smithi132 ceph-mon[35941]: pgmap v4919: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T03:03:45.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:03:45 smithi132 ceph-mon[35941]: pgmap v4920: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 4 op/s 2024-03-28T03:03:46.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:03:46 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8885.787417 secs 2024-03-28T03:03:46.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:03:46 smithi165 ceph-mon[36302]: pgmap v4921: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:03:46.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:03:46 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8885.787417 secs 2024-03-28T03:03:46.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:03:46 smithi119 ceph-mon[31518]: pgmap v4921: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:03:46.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:03:46 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8885.787417 secs 2024-03-28T03:03:46.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:03:46 smithi132 ceph-mon[35941]: pgmap v4921: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:03:49.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:03:49 smithi132 ceph-mon[35941]: pgmap v4922: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:03:49.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:03:49 smithi165 ceph-mon[36302]: pgmap v4922: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:03:49.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:03:49 smithi119 ceph-mon[31518]: pgmap v4922: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:03:51.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:03:51 smithi132 ceph-mon[35941]: pgmap v4923: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:03:51.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:03:51 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8890.787504 secs 2024-03-28T03:03:51.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:03:51 smithi165 ceph-mon[36302]: pgmap v4923: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:03:51.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:03:51 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8890.787504 secs 2024-03-28T03:03:51.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:03:51 smithi119 ceph-mon[31518]: pgmap v4923: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:03:51.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:03:51 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8890.787504 secs 2024-03-28T03:03:53.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:03:53 smithi132 ceph-mon[35941]: pgmap v4924: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:03:53.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:03:53 smithi165 ceph-mon[36302]: pgmap v4924: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:03:53.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:03:53 smithi119 ceph-mon[31518]: pgmap v4924: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:03:55.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:03:55 smithi132 ceph-mon[35941]: pgmap v4925: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T03:03:55.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:03:55 smithi165 ceph-mon[36302]: pgmap v4925: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T03:03:55.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:03:55 smithi119 ceph-mon[31518]: pgmap v4925: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T03:03:56.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:03:56 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8895.787594 secs 2024-03-28T03:03:56.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:03:56 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8895.787594 secs 2024-03-28T03:03:56.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:03:56 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8895.787594 secs 2024-03-28T03:03:57.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:03:57 smithi132 ceph-mon[35941]: pgmap v4926: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-28T03:03:57.554 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:03:57 smithi119 ceph-mon[31518]: pgmap v4926: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-28T03:03:57.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:03:57 smithi165 ceph-mon[36302]: pgmap v4926: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-28T03:03:59.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:03:59 smithi165 ceph-mon[36302]: pgmap v4927: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:03:59.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:03:59 smithi119 ceph-mon[31518]: pgmap v4927: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:03:59.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:03:59 smithi132 ceph-mon[35941]: pgmap v4927: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:04:01.502 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:04:01 smithi132 ceph-mon[35941]: pgmap v4928: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:04:01.502 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:04:01 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8900.787632 secs 2024-03-28T03:04:01.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:04:01 smithi165 ceph-mon[36302]: pgmap v4928: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:04:01.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:04:01 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8900.787632 secs 2024-03-28T03:04:01.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:04:01 smithi119 ceph-mon[31518]: pgmap v4928: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:04:01.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:04:01 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8900.787632 secs 2024-03-28T03:04:03.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:04:03 smithi165 ceph-mon[36302]: pgmap v4929: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:04:03.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:04:03 smithi119 ceph-mon[31518]: pgmap v4929: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:04:03.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:04:03 smithi132 ceph-mon[35941]: pgmap v4929: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:04:05.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:04:05 smithi165 ceph-mon[36302]: pgmap v4930: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-28T03:04:05.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:04:05 smithi119 ceph-mon[31518]: pgmap v4930: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-28T03:04:05.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:04:05 smithi132 ceph-mon[35941]: pgmap v4930: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-28T03:04:06.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:04:06 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8905.787670 secs 2024-03-28T03:04:06.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:04:06 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8905.787670 secs 2024-03-28T03:04:06.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:04:06 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8905.787670 secs 2024-03-28T03:04:07.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:04:07 smithi165 ceph-mon[36302]: pgmap v4931: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-28T03:04:07.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:04:07 smithi119 ceph-mon[31518]: pgmap v4931: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-28T03:04:07.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:04:07 smithi132 ceph-mon[35941]: pgmap v4931: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-28T03:04:09.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:04:09 smithi165 ceph-mon[36302]: pgmap v4932: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:04:09.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:04:09 smithi119 ceph-mon[31518]: pgmap v4932: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:04:09.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:04:09 smithi132 ceph-mon[35941]: pgmap v4932: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:04:11.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:04:11 smithi165 ceph-mon[36302]: pgmap v4933: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:04:11.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:04:11 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8910.787758 secs 2024-03-28T03:04:11.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:04:11 smithi119 ceph-mon[31518]: pgmap v4933: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:04:11.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:04:11 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8910.787758 secs 2024-03-28T03:04:11.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:04:11 smithi132 ceph-mon[35941]: pgmap v4933: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:04:11.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:04:11 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8910.787758 secs 2024-03-28T03:04:13.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:04:13 smithi119 ceph-mon[31518]: pgmap v4934: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:04:13.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:04:13 smithi165 ceph-mon[36302]: pgmap v4934: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:04:13.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:04:13 smithi132 ceph-mon[35941]: pgmap v4934: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:04:15.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:04:15 smithi165 ceph-mon[36302]: pgmap v4935: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:04:15.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:04:15 smithi119 ceph-mon[31518]: pgmap v4935: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:04:15.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:04:15 smithi132 ceph-mon[35941]: pgmap v4935: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:04:16.531 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:04:16 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8915.787818 secs 2024-03-28T03:04:16.554 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:04:16 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8915.787818 secs 2024-03-28T03:04:16.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:04:16 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8915.787818 secs 2024-03-28T03:04:17.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:04:17 smithi165 ceph-mon[36302]: pgmap v4936: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T03:04:17.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:04:17 smithi119 ceph-mon[31518]: pgmap v4936: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T03:04:17.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:04:17 smithi132 ceph-mon[35941]: pgmap v4936: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T03:04:19.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:04:19 smithi165 ceph-mon[36302]: pgmap v4937: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:04:19.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:04:19 smithi119 ceph-mon[31518]: pgmap v4937: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:04:19.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:04:19 smithi132 ceph-mon[35941]: pgmap v4937: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:04:21.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:04:21 smithi165 ceph-mon[36302]: pgmap v4938: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:04:21.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:04:21 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8920.787889 secs 2024-03-28T03:04:21.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:04:21 smithi119 ceph-mon[31518]: pgmap v4938: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:04:21.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:04:21 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8920.787889 secs 2024-03-28T03:04:21.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:04:21 smithi132 ceph-mon[35941]: pgmap v4938: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:04:21.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:04:21 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8920.787889 secs 2024-03-28T03:04:23.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:04:23 smithi165 ceph-mon[36302]: pgmap v4939: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:04:23.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:04:23 smithi119 ceph-mon[31518]: pgmap v4939: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:04:23.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:04:23 smithi132 ceph-mon[35941]: pgmap v4939: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:04:25.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:04:25 smithi165 ceph-mon[36302]: pgmap v4940: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:04:25.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:04:25 smithi119 ceph-mon[31518]: pgmap v4940: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:04:25.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:04:25 smithi132 ceph-mon[35941]: pgmap v4940: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:04:26.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:04:26 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8925.787972 secs 2024-03-28T03:04:26.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:04:26 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8925.787972 secs 2024-03-28T03:04:26.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:04:26 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8925.787972 secs 2024-03-28T03:04:27.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:04:27 smithi165 ceph-mon[36302]: pgmap v4941: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:04:27.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:04:27 smithi119 ceph-mon[31518]: pgmap v4941: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:04:27.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:04:27 smithi132 ceph-mon[35941]: pgmap v4941: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:04:29.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:04:29 smithi165 ceph-mon[36302]: pgmap v4942: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:04:29.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:04:29 smithi119 ceph-mon[31518]: pgmap v4942: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:04:29.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:04:29 smithi132 ceph-mon[35941]: pgmap v4942: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:04:31.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:04:31 smithi165 ceph-mon[36302]: pgmap v4943: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:04:31.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:04:31 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8930.788034 secs 2024-03-28T03:04:31.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:04:31 smithi119 ceph-mon[31518]: pgmap v4943: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:04:31.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:04:31 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8930.788034 secs 2024-03-28T03:04:31.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:04:31 smithi132 ceph-mon[35941]: pgmap v4943: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:04:31.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:04:31 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8930.788034 secs 2024-03-28T03:04:33.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:04:33 smithi165 ceph-mon[36302]: pgmap v4944: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:04:33.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:04:33 smithi119 ceph-mon[31518]: pgmap v4944: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:04:33.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:04:33 smithi132 ceph-mon[35941]: pgmap v4944: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:04:35.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:04:35 smithi165 ceph-mon[36302]: pgmap v4945: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:04:35.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:04:35 smithi119 ceph-mon[31518]: pgmap v4945: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:04:35.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:04:35 smithi132 ceph-mon[35941]: pgmap v4945: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:04:36.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:04:36 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8935.788097 secs 2024-03-28T03:04:36.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:04:36 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8935.788097 secs 2024-03-28T03:04:36.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:04:36 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8935.788097 secs 2024-03-28T03:04:37.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:04:37 smithi165 ceph-mon[36302]: pgmap v4946: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:04:37.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:04:37 smithi119 ceph-mon[31518]: pgmap v4946: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:04:37.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:04:37 smithi132 ceph-mon[35941]: pgmap v4946: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:04:39.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:04:39 smithi165 ceph-mon[36302]: pgmap v4947: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:04:39.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:04:39 smithi119 ceph-mon[31518]: pgmap v4947: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:04:39.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:04:39 smithi132 ceph-mon[35941]: pgmap v4947: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:04:41.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:04:41 smithi165 ceph-mon[36302]: pgmap v4948: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:04:41.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:04:41 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8940.788160 secs 2024-03-28T03:04:41.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:04:41 smithi119 ceph-mon[31518]: pgmap v4948: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:04:41.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:04:41 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8940.788160 secs 2024-03-28T03:04:41.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:04:41 smithi132 ceph-mon[35941]: pgmap v4948: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:04:41.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:04:41 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8940.788160 secs 2024-03-28T03:04:43.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:04:43 smithi165 ceph-mon[36302]: pgmap v4949: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:04:43.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:04:43 smithi119 ceph-mon[31518]: pgmap v4949: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:04:43.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:04:43 smithi132 ceph-mon[35941]: pgmap v4949: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:04:45.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:04:45 smithi165 ceph-mon[36302]: pgmap v4950: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:04:45.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:04:45 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T03:04:45.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:04:45 smithi119 ceph-mon[31518]: pgmap v4950: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:04:45.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:04:45 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T03:04:45.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:04:45 smithi132 ceph-mon[35941]: pgmap v4950: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:04:45.701 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:04:45 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T03:04:46.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:04:46 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8945.788203 secs 2024-03-28T03:04:46.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:04:46 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T03:04:46.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:04:46 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T03:04:46.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:04:46 smithi165 ceph-mon[36302]: pgmap v4951: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T03:04:46.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:04:46 smithi165 ceph-mon[36302]: pgmap v4952: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:04:46.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:04:46 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:04:46.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:04:46 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8945.788203 secs 2024-03-28T03:04:46.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:04:46 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T03:04:46.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:04:46 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T03:04:46.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:04:46 smithi119 ceph-mon[31518]: pgmap v4951: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T03:04:46.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:04:46 smithi119 ceph-mon[31518]: pgmap v4952: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:04:46.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:04:46 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:04:46.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:04:46 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8945.788203 secs 2024-03-28T03:04:46.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:04:46 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T03:04:46.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:04:46 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T03:04:46.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:04:46 smithi132 ceph-mon[35941]: pgmap v4951: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T03:04:46.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:04:46 smithi132 ceph-mon[35941]: pgmap v4952: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:04:46.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:04:46 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:04:47.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:04:47 smithi132 ceph-mon[35941]: pgmap v4953: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:04:48.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:04:47 smithi165 ceph-mon[36302]: pgmap v4953: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:04:48.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:04:47 smithi119 ceph-mon[31518]: pgmap v4953: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:04:49.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:04:49 smithi132 ceph-mon[35941]: pgmap v4954: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:04:50.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:04:49 smithi165 ceph-mon[36302]: pgmap v4954: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:04:50.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:04:49 smithi119 ceph-mon[31518]: pgmap v4954: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:04:51.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:04:50 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8950.788291 secs 2024-03-28T03:04:51.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:04:50 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8950.788291 secs 2024-03-28T03:04:51.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:04:50 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8950.788291 secs 2024-03-28T03:04:52.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:04:51 smithi132 ceph-mon[35941]: pgmap v4955: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:04:52.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:04:51 smithi165 ceph-mon[36302]: pgmap v4955: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:04:52.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:04:51 smithi119 ceph-mon[31518]: pgmap v4955: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:04:53.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:04:53 smithi132 ceph-mon[35941]: pgmap v4956: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-28T03:04:54.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:04:53 smithi165 ceph-mon[36302]: pgmap v4956: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-28T03:04:54.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:04:53 smithi119 ceph-mon[31518]: pgmap v4956: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-28T03:04:55.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:04:55 smithi132 ceph-mon[35941]: pgmap v4957: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T03:04:56.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:04:55 smithi165 ceph-mon[36302]: pgmap v4957: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T03:04:56.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:04:55 smithi119 ceph-mon[31518]: pgmap v4957: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T03:04:56.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:04:56 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8955.788379 secs 2024-03-28T03:04:57.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:04:56 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8955.788379 secs 2024-03-28T03:04:57.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:04:56 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8955.788379 secs 2024-03-28T03:04:57.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:04:57 smithi132 ceph-mon[35941]: pgmap v4958: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:04:58.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:04:57 smithi165 ceph-mon[36302]: pgmap v4958: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:04:58.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:04:57 smithi119 ceph-mon[31518]: pgmap v4958: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:04:59.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:04:59 smithi132 ceph-mon[35941]: pgmap v4959: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:05:00.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:04:59 smithi165 ceph-mon[36302]: pgmap v4959: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:05:00.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:04:59 smithi119 ceph-mon[31518]: pgmap v4959: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:05:01.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:05:00 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8960.788468 secs 2024-03-28T03:05:01.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:05:00 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8960.788468 secs 2024-03-28T03:05:01.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:05:00 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8960.788468 secs 2024-03-28T03:05:02.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:05:01 smithi132 ceph-mon[35941]: pgmap v4960: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:05:02.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:05:01 smithi165 ceph-mon[36302]: pgmap v4960: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:05:02.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:05:01 smithi119 ceph-mon[31518]: pgmap v4960: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:05:03.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:05:03 smithi132 ceph-mon[35941]: pgmap v4961: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:05:04.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:05:03 smithi165 ceph-mon[36302]: pgmap v4961: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:05:04.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:05:03 smithi119 ceph-mon[31518]: pgmap v4961: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:05:05.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:05:05 smithi132 ceph-mon[35941]: pgmap v4962: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:05:06.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:05:05 smithi165 ceph-mon[36302]: pgmap v4962: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:05:06.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:05:05 smithi119 ceph-mon[31518]: pgmap v4962: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:05:06.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:05:06 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8965.788557 secs 2024-03-28T03:05:07.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:05:06 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8965.788557 secs 2024-03-28T03:05:07.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:05:06 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8965.788557 secs 2024-03-28T03:05:07.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:05:07 smithi132 ceph-mon[35941]: pgmap v4963: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:05:08.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:05:07 smithi165 ceph-mon[36302]: pgmap v4963: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:05:08.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:05:07 smithi119 ceph-mon[31518]: pgmap v4963: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:05:09.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:05:09 smithi132 ceph-mon[35941]: pgmap v4964: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:05:10.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:05:09 smithi165 ceph-mon[36302]: pgmap v4964: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:05:10.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:05:09 smithi119 ceph-mon[31518]: pgmap v4964: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:05:11.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:05:10 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8970.788642 secs 2024-03-28T03:05:11.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:05:10 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8970.788642 secs 2024-03-28T03:05:11.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:05:10 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8970.788642 secs 2024-03-28T03:05:12.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:05:11 smithi132 ceph-mon[35941]: pgmap v4965: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:05:12.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:05:11 smithi165 ceph-mon[36302]: pgmap v4965: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:05:12.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:05:11 smithi119 ceph-mon[31518]: pgmap v4965: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:05:13.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:05:13 smithi132 ceph-mon[35941]: pgmap v4966: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:05:14.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:05:13 smithi165 ceph-mon[36302]: pgmap v4966: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:05:14.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:05:13 smithi119 ceph-mon[31518]: pgmap v4966: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:05:16.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:05:16 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8975.788703 secs 2024-03-28T03:05:16.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:05:16 smithi132 ceph-mon[35941]: pgmap v4967: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:05:17.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:05:16 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8975.788703 secs 2024-03-28T03:05:17.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:05:16 smithi165 ceph-mon[36302]: pgmap v4967: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:05:17.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:05:16 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8975.788703 secs 2024-03-28T03:05:17.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:05:16 smithi119 ceph-mon[31518]: pgmap v4967: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:05:18.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:05:18 smithi132 ceph-mon[35941]: pgmap v4968: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:05:19.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:05:18 smithi165 ceph-mon[36302]: pgmap v4968: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:05:19.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:05:18 smithi119 ceph-mon[31518]: pgmap v4968: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:05:20.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:05:20 smithi132 ceph-mon[35941]: pgmap v4969: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:05:21.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:05:20 smithi165 ceph-mon[36302]: pgmap v4969: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:05:21.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:05:20 smithi119 ceph-mon[31518]: pgmap v4969: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:05:21.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:05:21 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8980.788786 secs 2024-03-28T03:05:22.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:05:21 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8980.788786 secs 2024-03-28T03:05:22.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:05:21 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8980.788786 secs 2024-03-28T03:05:22.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:05:22 smithi132 ceph-mon[35941]: pgmap v4970: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:05:23.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:05:22 smithi165 ceph-mon[36302]: pgmap v4970: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:05:23.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:05:22 smithi119 ceph-mon[31518]: pgmap v4970: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:05:24.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:05:24 smithi132 ceph-mon[35941]: pgmap v4971: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:05:25.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:05:24 smithi165 ceph-mon[36302]: pgmap v4971: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:05:25.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:05:24 smithi119 ceph-mon[31518]: pgmap v4971: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:05:26.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:05:26 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8985.788875 secs 2024-03-28T03:05:26.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:05:26 smithi132 ceph-mon[35941]: pgmap v4972: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:05:27.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:05:26 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8985.788875 secs 2024-03-28T03:05:27.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:05:26 smithi165 ceph-mon[36302]: pgmap v4972: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:05:27.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:05:26 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8985.788875 secs 2024-03-28T03:05:27.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:05:26 smithi119 ceph-mon[31518]: pgmap v4972: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:05:29.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:05:28 smithi165 ceph-mon[36302]: pgmap v4973: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:05:29.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:05:28 smithi119 ceph-mon[31518]: pgmap v4973: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:05:29.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:05:28 smithi132 ceph-mon[35941]: pgmap v4973: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:05:31.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:05:30 smithi165 ceph-mon[36302]: pgmap v4974: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:05:31.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:05:30 smithi119 ceph-mon[31518]: pgmap v4974: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:05:31.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:05:30 smithi132 ceph-mon[35941]: pgmap v4974: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:05:32.101 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:05:31 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8990.788960 secs 2024-03-28T03:05:32.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:05:31 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8990.788960 secs 2024-03-28T03:05:32.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:05:31 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8990.788960 secs 2024-03-28T03:05:33.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:05:32 smithi165 ceph-mon[36302]: pgmap v4975: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:05:33.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:05:32 smithi119 ceph-mon[31518]: pgmap v4975: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:05:33.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:05:32 smithi132 ceph-mon[35941]: pgmap v4975: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:05:35.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:05:34 smithi165 ceph-mon[36302]: pgmap v4976: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:05:35.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:05:34 smithi119 ceph-mon[31518]: pgmap v4976: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:05:35.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:05:34 smithi132 ceph-mon[35941]: pgmap v4976: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:05:37.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:05:36 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 8995.789043 secs 2024-03-28T03:05:37.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:05:36 smithi165 ceph-mon[36302]: pgmap v4977: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:05:37.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:05:36 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 8995.789043 secs 2024-03-28T03:05:37.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:05:36 smithi119 ceph-mon[31518]: pgmap v4977: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:05:37.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:05:36 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 8995.789043 secs 2024-03-28T03:05:37.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:05:36 smithi132 ceph-mon[35941]: pgmap v4977: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:05:39.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:05:38 smithi165 ceph-mon[36302]: pgmap v4978: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:05:39.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:05:38 smithi119 ceph-mon[31518]: pgmap v4978: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:05:39.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:05:38 smithi132 ceph-mon[35941]: pgmap v4978: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:05:41.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:05:40 smithi165 ceph-mon[36302]: pgmap v4979: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:05:41.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:05:40 smithi119 ceph-mon[31518]: pgmap v4979: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:05:41.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:05:40 smithi132 ceph-mon[35941]: pgmap v4979: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:05:42.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:05:41 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9000.789132 secs 2024-03-28T03:05:42.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:05:41 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9000.789132 secs 2024-03-28T03:05:42.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:05:41 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9000.789132 secs 2024-03-28T03:05:43.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:05:42 smithi165 ceph-mon[36302]: pgmap v4980: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:05:43.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:05:42 smithi119 ceph-mon[31518]: pgmap v4980: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:05:43.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:05:42 smithi132 ceph-mon[35941]: pgmap v4980: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:05:45.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:05:44 smithi165 ceph-mon[36302]: pgmap v4981: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:05:45.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:05:44 smithi119 ceph-mon[31518]: pgmap v4981: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:05:45.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:05:44 smithi132 ceph-mon[35941]: pgmap v4981: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:05:47.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:05:46 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9005.789194 secs 2024-03-28T03:05:47.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:05:46 smithi165 ceph-mon[36302]: pgmap v4982: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:05:47.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:05:46 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T03:05:47.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:05:46 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9005.789194 secs 2024-03-28T03:05:47.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:05:46 smithi119 ceph-mon[31518]: pgmap v4982: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:05:47.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:05:46 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T03:05:47.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:05:46 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9005.789194 secs 2024-03-28T03:05:47.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:05:46 smithi132 ceph-mon[35941]: pgmap v4982: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:05:47.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:05:46 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T03:05:48.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:05:47 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T03:05:48.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:05:47 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T03:05:48.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:05:47 smithi165 ceph-mon[36302]: pgmap v4983: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T03:05:48.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:05:47 smithi165 ceph-mon[36302]: pgmap v4984: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:05:48.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:05:47 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:05:48.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:05:47 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T03:05:48.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:05:47 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T03:05:48.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:05:47 smithi119 ceph-mon[31518]: pgmap v4983: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T03:05:48.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:05:47 smithi119 ceph-mon[31518]: pgmap v4984: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:05:48.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:05:47 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:05:48.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:05:47 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T03:05:48.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:05:47 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T03:05:48.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:05:47 smithi132 ceph-mon[35941]: pgmap v4983: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T03:05:48.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:05:47 smithi132 ceph-mon[35941]: pgmap v4984: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:05:48.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:05:47 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:05:50.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:05:50 smithi132 ceph-mon[35941]: pgmap v4985: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-28T03:05:50.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:05:50 smithi165 ceph-mon[36302]: pgmap v4985: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-28T03:05:50.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:05:50 smithi119 ceph-mon[31518]: pgmap v4985: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-28T03:05:51.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:05:51 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9010.789255 secs 2024-03-28T03:05:51.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:05:51 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9010.789255 secs 2024-03-28T03:05:51.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:05:51 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9010.789255 secs 2024-03-28T03:05:52.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:05:52 smithi132 ceph-mon[35941]: pgmap v4986: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:05:52.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:05:52 smithi165 ceph-mon[36302]: pgmap v4986: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:05:52.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:05:52 smithi119 ceph-mon[31518]: pgmap v4986: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:05:54.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:05:54 smithi132 ceph-mon[35941]: pgmap v4987: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:05:54.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:05:54 smithi165 ceph-mon[36302]: pgmap v4987: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:05:54.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:05:54 smithi119 ceph-mon[31518]: pgmap v4987: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:05:56.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:05:56 smithi132 ceph-mon[35941]: pgmap v4988: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:05:56.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:05:56 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9015.789344 secs 2024-03-28T03:05:56.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:05:56 smithi165 ceph-mon[36302]: pgmap v4988: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:05:56.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:05:56 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9015.789344 secs 2024-03-28T03:05:56.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:05:56 smithi119 ceph-mon[31518]: pgmap v4988: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:05:56.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:05:56 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9015.789344 secs 2024-03-28T03:05:58.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:05:58 smithi132 ceph-mon[35941]: pgmap v4989: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T03:05:58.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:05:58 smithi165 ceph-mon[36302]: pgmap v4989: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T03:05:58.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:05:58 smithi119 ceph-mon[31518]: pgmap v4989: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T03:06:00.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:06:00 smithi132 ceph-mon[35941]: pgmap v4990: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-28T03:06:00.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:06:00 smithi165 ceph-mon[36302]: pgmap v4990: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-28T03:06:00.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:06:00 smithi119 ceph-mon[31518]: pgmap v4990: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-28T03:06:01.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:06:01 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9020.789435 secs 2024-03-28T03:06:01.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:06:01 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9020.789435 secs 2024-03-28T03:06:01.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:06:01 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9020.789435 secs 2024-03-28T03:06:02.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:06:02 smithi119 ceph-mon[31518]: pgmap v4991: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T03:06:02.601 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:06:02 smithi132 ceph-mon[35941]: pgmap v4991: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T03:06:02.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:06:02 smithi165 ceph-mon[36302]: pgmap v4991: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T03:06:04.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:06:04 smithi165 ceph-mon[36302]: pgmap v4992: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:06:04.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:06:04 smithi119 ceph-mon[31518]: pgmap v4992: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:06:04.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:06:04 smithi132 ceph-mon[35941]: pgmap v4992: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:06:06.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:06:06 smithi165 ceph-mon[36302]: pgmap v4993: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:06:06.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:06:06 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9025.789523 secs 2024-03-28T03:06:06.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:06:06 smithi119 ceph-mon[31518]: pgmap v4993: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:06:06.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:06:06 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9025.789523 secs 2024-03-28T03:06:06.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:06:06 smithi132 ceph-mon[35941]: pgmap v4993: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:06:06.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:06:06 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9025.789523 secs 2024-03-28T03:06:08.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:06:08 smithi165 ceph-mon[36302]: pgmap v4994: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:06:08.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:06:08 smithi119 ceph-mon[31518]: pgmap v4994: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:06:08.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:06:08 smithi132 ceph-mon[35941]: pgmap v4994: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:06:10.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:06:10 smithi165 ceph-mon[36302]: pgmap v4995: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-28T03:06:10.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:06:10 smithi119 ceph-mon[31518]: pgmap v4995: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-28T03:06:10.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:06:10 smithi132 ceph-mon[35941]: pgmap v4995: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-28T03:06:11.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:06:11 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9030.789610 secs 2024-03-28T03:06:11.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:06:11 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9030.789610 secs 2024-03-28T03:06:11.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:06:11 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9030.789610 secs 2024-03-28T03:06:12.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:06:12 smithi165 ceph-mon[36302]: pgmap v4996: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T03:06:12.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:06:12 smithi119 ceph-mon[31518]: pgmap v4996: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T03:06:12.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:06:12 smithi132 ceph-mon[35941]: pgmap v4996: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T03:06:14.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:06:14 smithi165 ceph-mon[36302]: pgmap v4997: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:06:14.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:06:14 smithi119 ceph-mon[31518]: pgmap v4997: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:06:14.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:06:14 smithi132 ceph-mon[35941]: pgmap v4997: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:06:16.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:06:16 smithi165 ceph-mon[36302]: pgmap v4998: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:06:16.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:06:16 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9035.789698 secs 2024-03-28T03:06:16.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:06:16 smithi119 ceph-mon[31518]: pgmap v4998: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:06:16.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:06:16 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9035.789698 secs 2024-03-28T03:06:16.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:06:16 smithi132 ceph-mon[35941]: pgmap v4998: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:06:16.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:06:16 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9035.789698 secs 2024-03-28T03:06:18.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:06:18 smithi119 ceph-mon[31518]: pgmap v4999: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:06:18.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:06:18 smithi165 ceph-mon[36302]: pgmap v4999: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:06:18.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:06:18 smithi132 ceph-mon[35941]: pgmap v4999: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:06:20.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:06:20 smithi165 ceph-mon[36302]: pgmap v5000: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-28T03:06:20.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:06:20 smithi119 ceph-mon[31518]: pgmap v5000: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-28T03:06:20.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:06:20 smithi132 ceph-mon[35941]: pgmap v5000: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-28T03:06:21.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:06:21 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9040.789736 secs 2024-03-28T03:06:21.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:06:21 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9040.789736 secs 2024-03-28T03:06:21.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:06:21 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9040.789736 secs 2024-03-28T03:06:22.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:06:22 smithi165 ceph-mon[36302]: pgmap v5001: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T03:06:22.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:06:22 smithi119 ceph-mon[31518]: pgmap v5001: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T03:06:22.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:06:22 smithi132 ceph-mon[35941]: pgmap v5001: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T03:06:24.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:06:24 smithi165 ceph-mon[36302]: pgmap v5002: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:06:24.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:06:24 smithi119 ceph-mon[31518]: pgmap v5002: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:06:24.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:06:24 smithi132 ceph-mon[35941]: pgmap v5002: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:06:26.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:06:26 smithi165 ceph-mon[36302]: pgmap v5003: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:06:26.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:06:26 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9045.789825 secs 2024-03-28T03:06:26.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:06:26 smithi119 ceph-mon[31518]: pgmap v5003: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:06:26.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:06:26 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9045.789825 secs 2024-03-28T03:06:26.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:06:26 smithi132 ceph-mon[35941]: pgmap v5003: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:06:26.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:06:26 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9045.789825 secs 2024-03-28T03:06:28.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:06:28 smithi165 ceph-mon[36302]: pgmap v5004: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:06:28.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:06:28 smithi119 ceph-mon[31518]: pgmap v5004: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:06:28.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:06:28 smithi132 ceph-mon[35941]: pgmap v5004: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:06:30.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:06:30 smithi165 ceph-mon[36302]: pgmap v5005: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T03:06:30.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:06:30 smithi119 ceph-mon[31518]: pgmap v5005: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T03:06:30.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:06:30 smithi132 ceph-mon[35941]: pgmap v5005: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T03:06:31.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:06:31 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9050.789870 secs 2024-03-28T03:06:31.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:06:31 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9050.789870 secs 2024-03-28T03:06:31.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:06:31 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9050.789870 secs 2024-03-28T03:06:32.601 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:06:32 smithi132 ceph-mon[35941]: pgmap v5006: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T03:06:32.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:06:32 smithi165 ceph-mon[36302]: pgmap v5006: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T03:06:32.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:06:32 smithi119 ceph-mon[31518]: pgmap v5006: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T03:06:34.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:06:34 smithi165 ceph-mon[36302]: pgmap v5007: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:06:34.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:06:34 smithi119 ceph-mon[31518]: pgmap v5007: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:06:34.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:06:34 smithi132 ceph-mon[35941]: pgmap v5007: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:06:36.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:06:36 smithi165 ceph-mon[36302]: pgmap v5008: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:06:36.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:06:36 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9055.789958 secs 2024-03-28T03:06:36.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:06:36 smithi119 ceph-mon[31518]: pgmap v5008: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:06:36.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:06:36 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9055.789958 secs 2024-03-28T03:06:36.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:06:36 smithi132 ceph-mon[35941]: pgmap v5008: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:06:36.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:06:36 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9055.789958 secs 2024-03-28T03:06:38.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:06:38 smithi165 ceph-mon[36302]: pgmap v5009: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:06:38.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:06:38 smithi119 ceph-mon[31518]: pgmap v5009: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:06:38.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:06:38 smithi132 ceph-mon[35941]: pgmap v5009: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:06:40.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:06:40 smithi165 ceph-mon[36302]: pgmap v5010: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T03:06:40.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:06:40 smithi119 ceph-mon[31518]: pgmap v5010: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T03:06:40.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:06:40 smithi132 ceph-mon[35941]: pgmap v5010: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T03:06:41.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:06:41 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9060.790017 secs 2024-03-28T03:06:41.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:06:41 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9060.790017 secs 2024-03-28T03:06:41.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:06:41 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9060.790017 secs 2024-03-28T03:06:42.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:06:42 smithi165 ceph-mon[36302]: pgmap v5011: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-28T03:06:42.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:06:42 smithi119 ceph-mon[31518]: pgmap v5011: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-28T03:06:42.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:06:42 smithi132 ceph-mon[35941]: pgmap v5011: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-28T03:06:44.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:06:44 smithi165 ceph-mon[36302]: pgmap v5012: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:06:44.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:06:44 smithi119 ceph-mon[31518]: pgmap v5012: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:06:44.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:06:44 smithi132 ceph-mon[35941]: pgmap v5012: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:06:46.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:06:46 smithi165 ceph-mon[36302]: pgmap v5013: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:06:46.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:06:46 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9065.790078 secs 2024-03-28T03:06:46.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:06:46 smithi119 ceph-mon[31518]: pgmap v5013: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:06:46.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:06:46 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9065.790078 secs 2024-03-28T03:06:46.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:06:46 smithi132 ceph-mon[35941]: pgmap v5013: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:06:46.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:06:46 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9065.790078 secs 2024-03-28T03:06:48.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:06:48 smithi165 ceph-mon[36302]: pgmap v5014: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:06:48.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:06:48 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T03:06:48.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:06:48 smithi119 ceph-mon[31518]: pgmap v5014: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:06:48.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:06:48 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T03:06:48.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:06:48 smithi132 ceph-mon[35941]: pgmap v5014: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:06:48.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:06:48 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T03:06:49.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:06:49 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T03:06:49.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:06:49 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T03:06:49.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:06:49 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:06:49.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:06:49 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T03:06:49.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:06:49 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T03:06:49.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:06:49 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:06:49.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:06:49 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T03:06:49.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:06:49 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T03:06:49.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:06:49 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:06:50.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:06:50 smithi165 ceph-mon[36302]: pgmap v5015: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T03:06:50.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:06:50 smithi165 ceph-mon[36302]: pgmap v5016: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-28T03:06:50.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:06:50 smithi119 ceph-mon[31518]: pgmap v5015: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T03:06:50.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:06:50 smithi119 ceph-mon[31518]: pgmap v5016: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-28T03:06:50.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:06:50 smithi132 ceph-mon[35941]: pgmap v5015: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T03:06:50.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:06:50 smithi132 ceph-mon[35941]: pgmap v5016: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-28T03:06:51.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:06:51 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9070.790168 secs 2024-03-28T03:06:51.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:06:51 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9070.790168 secs 2024-03-28T03:06:51.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:06:51 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9070.790168 secs 2024-03-28T03:06:52.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:06:52 smithi165 ceph-mon[36302]: pgmap v5017: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:06:52.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:06:52 smithi119 ceph-mon[31518]: pgmap v5017: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:06:52.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:06:52 smithi132 ceph-mon[35941]: pgmap v5017: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:06:54.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:06:54 smithi165 ceph-mon[36302]: pgmap v5018: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:06:54.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:06:54 smithi119 ceph-mon[31518]: pgmap v5018: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:06:54.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:06:54 smithi132 ceph-mon[35941]: pgmap v5018: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:06:56.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:06:56 smithi165 ceph-mon[36302]: pgmap v5019: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:06:56.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:06:56 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9075.790255 secs 2024-03-28T03:06:56.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:06:56 smithi119 ceph-mon[31518]: pgmap v5019: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:06:56.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:06:56 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9075.790255 secs 2024-03-28T03:06:56.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:06:56 smithi132 ceph-mon[35941]: pgmap v5019: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:06:56.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:06:56 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9075.790255 secs 2024-03-28T03:06:58.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:06:58 smithi165 ceph-mon[36302]: pgmap v5020: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:06:58.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:06:58 smithi119 ceph-mon[31518]: pgmap v5020: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:06:58.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:06:58 smithi132 ceph-mon[35941]: pgmap v5020: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:07:00.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:07:00 smithi119 ceph-mon[31518]: pgmap v5021: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T03:07:00.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:07:00 smithi132 ceph-mon[35941]: pgmap v5021: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T03:07:00.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:07:00 smithi165 ceph-mon[36302]: pgmap v5021: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T03:07:01.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:07:01 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9080.790299 secs 2024-03-28T03:07:01.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:07:01 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9080.790299 secs 2024-03-28T03:07:01.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:07:01 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9080.790299 secs 2024-03-28T03:07:02.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:07:02 smithi119 ceph-mon[31518]: pgmap v5022: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:07:02.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:07:02 smithi132 ceph-mon[35941]: pgmap v5022: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:07:02.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:07:02 smithi165 ceph-mon[36302]: pgmap v5022: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:07:04.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:07:04 smithi119 ceph-mon[31518]: pgmap v5023: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:07:04.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:07:04 smithi132 ceph-mon[35941]: pgmap v5023: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:07:04.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:07:04 smithi165 ceph-mon[36302]: pgmap v5023: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:07:06.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:07:06 smithi132 ceph-mon[35941]: pgmap v5024: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:07:06.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:07:06 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9085.790363 secs 2024-03-28T03:07:06.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:07:06 smithi165 ceph-mon[36302]: pgmap v5024: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:07:06.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:07:06 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9085.790363 secs 2024-03-28T03:07:06.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:07:06 smithi119 ceph-mon[31518]: pgmap v5024: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:07:06.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:07:06 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9085.790363 secs 2024-03-28T03:07:08.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:07:08 smithi132 ceph-mon[35941]: pgmap v5025: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:07:08.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:07:08 smithi165 ceph-mon[36302]: pgmap v5025: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:07:08.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:07:08 smithi119 ceph-mon[31518]: pgmap v5025: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:07:10.701 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:07:10 smithi132 ceph-mon[35941]: pgmap v5026: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:07:10.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:07:10 smithi165 ceph-mon[36302]: pgmap v5026: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:07:10.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:07:10 smithi119 ceph-mon[31518]: pgmap v5026: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:07:11.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:07:11 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9090.790449 secs 2024-03-28T03:07:11.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:07:11 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9090.790449 secs 2024-03-28T03:07:11.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:07:11 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9090.790449 secs 2024-03-28T03:07:12.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:07:12 smithi132 ceph-mon[35941]: pgmap v5027: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:07:12.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:07:12 smithi165 ceph-mon[36302]: pgmap v5027: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:07:12.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:07:12 smithi119 ceph-mon[31518]: pgmap v5027: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:07:14.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:07:14 smithi132 ceph-mon[35941]: pgmap v5028: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:07:14.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:07:14 smithi165 ceph-mon[36302]: pgmap v5028: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:07:14.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:07:14 smithi119 ceph-mon[31518]: pgmap v5028: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:07:16.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:07:16 smithi132 ceph-mon[35941]: pgmap v5029: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:07:16.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:07:16 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9095.790511 secs 2024-03-28T03:07:16.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:07:16 smithi165 ceph-mon[36302]: pgmap v5029: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:07:16.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:07:16 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9095.790511 secs 2024-03-28T03:07:16.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:07:16 smithi119 ceph-mon[31518]: pgmap v5029: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:07:16.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:07:16 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9095.790511 secs 2024-03-28T03:07:18.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:07:18 smithi132 ceph-mon[35941]: pgmap v5030: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:07:18.706 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:07:18 smithi119 ceph-mon[31518]: pgmap v5030: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:07:18.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:07:18 smithi165 ceph-mon[36302]: pgmap v5030: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:07:20.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:07:20 smithi132 ceph-mon[35941]: pgmap v5031: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:07:20.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:07:20 smithi165 ceph-mon[36302]: pgmap v5031: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:07:20.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:07:20 smithi119 ceph-mon[31518]: pgmap v5031: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:07:21.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:07:21 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9100.790572 secs 2024-03-28T03:07:21.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:07:21 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9100.790572 secs 2024-03-28T03:07:21.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:07:21 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9100.790572 secs 2024-03-28T03:07:22.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:07:22 smithi165 ceph-mon[36302]: pgmap v5032: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:07:22.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:07:22 smithi119 ceph-mon[31518]: pgmap v5032: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:07:22.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:07:22 smithi132 ceph-mon[35941]: pgmap v5032: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:07:24.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:07:24 smithi165 ceph-mon[36302]: pgmap v5033: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:07:24.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:07:24 smithi119 ceph-mon[31518]: pgmap v5033: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:07:24.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:07:24 smithi132 ceph-mon[35941]: pgmap v5033: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:07:26.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:07:26 smithi165 ceph-mon[36302]: pgmap v5034: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:07:26.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:07:26 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9105.790656 secs 2024-03-28T03:07:26.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:07:26 smithi119 ceph-mon[31518]: pgmap v5034: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:07:26.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:07:26 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9105.790656 secs 2024-03-28T03:07:26.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:07:26 smithi132 ceph-mon[35941]: pgmap v5034: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:07:26.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:07:26 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9105.790656 secs 2024-03-28T03:07:28.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:07:28 smithi165 ceph-mon[36302]: pgmap v5035: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:07:28.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:07:28 smithi119 ceph-mon[31518]: pgmap v5035: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:07:28.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:07:28 smithi132 ceph-mon[35941]: pgmap v5035: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:07:30.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:07:30 smithi165 ceph-mon[36302]: pgmap v5036: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:07:30.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:07:30 smithi119 ceph-mon[31518]: pgmap v5036: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:07:30.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:07:30 smithi132 ceph-mon[35941]: pgmap v5036: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:07:31.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:07:31 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9110.790715 secs 2024-03-28T03:07:31.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:07:31 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9110.790715 secs 2024-03-28T03:07:31.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:07:31 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9110.790715 secs 2024-03-28T03:07:32.851 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:07:32 smithi132 ceph-mon[35941]: pgmap v5037: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:07:32.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:07:32 smithi165 ceph-mon[36302]: pgmap v5037: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:07:32.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:07:32 smithi119 ceph-mon[31518]: pgmap v5037: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:07:34.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:07:34 smithi165 ceph-mon[36302]: pgmap v5038: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:07:34.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:07:34 smithi119 ceph-mon[31518]: pgmap v5038: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:07:34.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:07:34 smithi132 ceph-mon[35941]: pgmap v5038: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:07:36.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:07:36 smithi165 ceph-mon[36302]: pgmap v5039: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:07:36.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:07:36 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9115.790805 secs 2024-03-28T03:07:36.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:07:36 smithi119 ceph-mon[31518]: pgmap v5039: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:07:36.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:07:36 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9115.790805 secs 2024-03-28T03:07:36.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:07:36 smithi132 ceph-mon[35941]: pgmap v5039: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:07:36.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:07:36 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9115.790805 secs 2024-03-28T03:07:38.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:07:38 smithi165 ceph-mon[36302]: pgmap v5040: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:07:38.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:07:38 smithi119 ceph-mon[31518]: pgmap v5040: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:07:38.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:07:38 smithi132 ceph-mon[35941]: pgmap v5040: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:07:40.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:07:40 smithi165 ceph-mon[36302]: pgmap v5041: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:07:40.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:07:40 smithi119 ceph-mon[31518]: pgmap v5041: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:07:40.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:07:40 smithi132 ceph-mon[35941]: pgmap v5041: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:07:41.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:07:41 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9120.790845 secs 2024-03-28T03:07:41.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:07:41 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9120.790845 secs 2024-03-28T03:07:41.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:07:41 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9120.790845 secs 2024-03-28T03:07:42.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:07:42 smithi165 ceph-mon[36302]: pgmap v5042: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:07:42.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:07:42 smithi119 ceph-mon[31518]: pgmap v5042: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:07:42.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:07:42 smithi132 ceph-mon[35941]: pgmap v5042: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:07:44.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:07:44 smithi165 ceph-mon[36302]: pgmap v5043: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:07:44.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:07:44 smithi119 ceph-mon[31518]: pgmap v5043: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:07:44.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:07:44 smithi132 ceph-mon[35941]: pgmap v5043: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:07:46.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:07:46 smithi165 ceph-mon[36302]: pgmap v5044: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:07:46.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:07:46 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9125.790906 secs 2024-03-28T03:07:46.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:07:46 smithi119 ceph-mon[31518]: pgmap v5044: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:07:46.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:07:46 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9125.790906 secs 2024-03-28T03:07:46.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:07:46 smithi132 ceph-mon[35941]: pgmap v5044: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:07:46.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:07:46 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9125.790906 secs 2024-03-28T03:07:48.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:07:48 smithi165 ceph-mon[36302]: pgmap v5045: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:07:48.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:07:48 smithi119 ceph-mon[31518]: pgmap v5045: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:07:48.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:07:48 smithi132 ceph-mon[35941]: pgmap v5045: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:07:49.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:07:49 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T03:07:49.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:07:49 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T03:07:49.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:07:49 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T03:07:50.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:07:50 smithi165 ceph-mon[36302]: pgmap v5046: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:07:50.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:07:50 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T03:07:50.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:07:50 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T03:07:50.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:07:50 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:07:50.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:07:50 smithi119 ceph-mon[31518]: pgmap v5046: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:07:50.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:07:50 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T03:07:50.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:07:50 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T03:07:50.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:07:50 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:07:50.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:07:50 smithi132 ceph-mon[35941]: pgmap v5046: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:07:50.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:07:50 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T03:07:50.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:07:50 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T03:07:50.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:07:50 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:07:51.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:07:51 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9130.790969 secs 2024-03-28T03:07:51.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:07:51 smithi165 ceph-mon[36302]: pgmap v5047: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T03:07:51.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:07:51 smithi165 ceph-mon[36302]: pgmap v5048: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:07:51.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:07:51 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9130.790969 secs 2024-03-28T03:07:51.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:07:51 smithi119 ceph-mon[31518]: pgmap v5047: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T03:07:51.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:07:51 smithi119 ceph-mon[31518]: pgmap v5048: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:07:51.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:07:51 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9130.790969 secs 2024-03-28T03:07:51.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:07:51 smithi132 ceph-mon[35941]: pgmap v5047: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T03:07:51.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:07:51 smithi132 ceph-mon[35941]: pgmap v5048: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:07:53.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:07:53 smithi165 ceph-mon[36302]: pgmap v5049: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:07:53.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:07:53 smithi119 ceph-mon[31518]: pgmap v5049: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:07:53.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:07:53 smithi132 ceph-mon[35941]: pgmap v5049: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:07:55.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:07:55 smithi165 ceph-mon[36302]: pgmap v5050: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:07:55.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:07:55 smithi119 ceph-mon[31518]: pgmap v5050: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:07:55.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:07:55 smithi132 ceph-mon[35941]: pgmap v5050: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:07:56.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:07:56 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9135.791054 secs 2024-03-28T03:07:56.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:07:56 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9135.791054 secs 2024-03-28T03:07:56.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:07:56 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9135.791054 secs 2024-03-28T03:07:57.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:07:57 smithi165 ceph-mon[36302]: pgmap v5051: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:07:57.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:07:57 smithi119 ceph-mon[31518]: pgmap v5051: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:07:57.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:07:57 smithi132 ceph-mon[35941]: pgmap v5051: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:07:59.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:07:59 smithi165 ceph-mon[36302]: pgmap v5052: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:07:59.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:07:59 smithi119 ceph-mon[31518]: pgmap v5052: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:07:59.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:07:59 smithi132 ceph-mon[35941]: pgmap v5052: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:08:01.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:08:01 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9140.791115 secs 2024-03-28T03:08:01.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:08:01 smithi165 ceph-mon[36302]: pgmap v5053: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T03:08:01.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:08:01 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9140.791115 secs 2024-03-28T03:08:01.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:08:01 smithi119 ceph-mon[31518]: pgmap v5053: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T03:08:01.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:08:01 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9140.791115 secs 2024-03-28T03:08:01.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:08:01 smithi132 ceph-mon[35941]: pgmap v5053: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T03:08:03.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:08:03 smithi165 ceph-mon[36302]: pgmap v5054: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:08:03.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:08:03 smithi119 ceph-mon[31518]: pgmap v5054: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:08:03.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:08:03 smithi132 ceph-mon[35941]: pgmap v5054: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:08:05.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:08:05 smithi165 ceph-mon[36302]: pgmap v5055: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:08:05.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:08:05 smithi119 ceph-mon[31518]: pgmap v5055: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:08:05.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:08:05 smithi132 ceph-mon[35941]: pgmap v5055: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:08:06.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:08:06 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9145.791199 secs 2024-03-28T03:08:06.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:08:06 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9145.791199 secs 2024-03-28T03:08:06.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:08:06 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9145.791199 secs 2024-03-28T03:08:07.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:08:07 smithi165 ceph-mon[36302]: pgmap v5056: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:08:07.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:08:07 smithi119 ceph-mon[31518]: pgmap v5056: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:08:07.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:08:07 smithi132 ceph-mon[35941]: pgmap v5056: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:08:09.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:08:09 smithi165 ceph-mon[36302]: pgmap v5057: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:08:09.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:08:09 smithi119 ceph-mon[31518]: pgmap v5057: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:08:09.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:08:09 smithi132 ceph-mon[35941]: pgmap v5057: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:08:11.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:08:11 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9150.791260 secs 2024-03-28T03:08:11.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:08:11 smithi165 ceph-mon[36302]: pgmap v5058: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:08:11.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:08:11 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9150.791260 secs 2024-03-28T03:08:11.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:08:11 smithi119 ceph-mon[31518]: pgmap v5058: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:08:11.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:08:11 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9150.791260 secs 2024-03-28T03:08:11.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:08:11 smithi132 ceph-mon[35941]: pgmap v5058: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:08:13.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:08:13 smithi165 ceph-mon[36302]: pgmap v5059: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:08:13.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:08:13 smithi119 ceph-mon[31518]: pgmap v5059: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:08:13.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:08:13 smithi132 ceph-mon[35941]: pgmap v5059: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:08:15.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:08:15 smithi165 ceph-mon[36302]: pgmap v5060: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:08:15.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:08:15 smithi119 ceph-mon[31518]: pgmap v5060: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:08:15.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:08:15 smithi132 ceph-mon[35941]: pgmap v5060: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:08:16.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:08:16 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9155.791321 secs 2024-03-28T03:08:16.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:08:16 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9155.791321 secs 2024-03-28T03:08:17.049 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:08:16 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9155.791321 secs 2024-03-28T03:08:17.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:08:17 smithi119 ceph-mon[31518]: pgmap v5061: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:08:17.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:08:17 smithi132 ceph-mon[35941]: pgmap v5061: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:08:17.986 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:08:17 smithi165 ceph-mon[36302]: pgmap v5061: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:08:19.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:08:19 smithi119 ceph-mon[31518]: pgmap v5062: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:08:19.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:08:19 smithi132 ceph-mon[35941]: pgmap v5062: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:08:20.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:08:19 smithi165 ceph-mon[36302]: pgmap v5062: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:08:21.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:08:21 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9160.791409 secs 2024-03-28T03:08:21.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:08:21 smithi119 ceph-mon[31518]: pgmap v5063: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:08:21.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:08:21 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9160.791409 secs 2024-03-28T03:08:21.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:08:21 smithi132 ceph-mon[35941]: pgmap v5063: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:08:22.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:08:21 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9160.791409 secs 2024-03-28T03:08:22.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:08:21 smithi165 ceph-mon[36302]: pgmap v5063: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:08:22.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:08:22 smithi132 ceph-mon[35941]: pgmap v5064: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:08:23.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:08:22 smithi165 ceph-mon[36302]: pgmap v5064: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:08:23.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:08:22 smithi119 ceph-mon[31518]: pgmap v5064: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:08:25.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:08:25 smithi165 ceph-mon[36302]: pgmap v5065: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:08:25.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:08:25 smithi119 ceph-mon[31518]: pgmap v5065: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:08:25.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:08:25 smithi132 ceph-mon[35941]: pgmap v5065: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:08:26.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:08:26 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9165.791423 secs 2024-03-28T03:08:26.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:08:26 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9165.791423 secs 2024-03-28T03:08:26.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:08:26 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9165.791423 secs 2024-03-28T03:08:27.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:08:27 smithi165 ceph-mon[36302]: pgmap v5066: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:08:27.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:08:27 smithi119 ceph-mon[31518]: pgmap v5066: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:08:27.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:08:27 smithi132 ceph-mon[35941]: pgmap v5066: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:08:29.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:08:29 smithi165 ceph-mon[36302]: pgmap v5067: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:08:29.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:08:29 smithi119 ceph-mon[31518]: pgmap v5067: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:08:29.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:08:29 smithi132 ceph-mon[35941]: pgmap v5067: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:08:31.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:08:31 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9170.791482 secs 2024-03-28T03:08:31.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:08:31 smithi165 ceph-mon[36302]: pgmap v5068: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:08:31.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:08:31 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9170.791482 secs 2024-03-28T03:08:31.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:08:31 smithi119 ceph-mon[31518]: pgmap v5068: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:08:31.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:08:31 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9170.791482 secs 2024-03-28T03:08:31.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:08:31 smithi132 ceph-mon[35941]: pgmap v5068: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:08:33.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:08:33 smithi165 ceph-mon[36302]: pgmap v5069: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:08:33.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:08:33 smithi119 ceph-mon[31518]: pgmap v5069: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:08:33.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:08:33 smithi132 ceph-mon[35941]: pgmap v5069: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:08:35.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:08:35 smithi165 ceph-mon[36302]: pgmap v5070: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:08:35.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:08:35 smithi119 ceph-mon[31518]: pgmap v5070: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:08:35.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:08:35 smithi132 ceph-mon[35941]: pgmap v5070: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:08:36.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:08:36 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9175.791570 secs 2024-03-28T03:08:36.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:08:36 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9175.791570 secs 2024-03-28T03:08:36.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:08:36 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9175.791570 secs 2024-03-28T03:08:37.554 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:08:37 smithi119 ceph-mon[31518]: pgmap v5071: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:08:37.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:08:37 smithi165 ceph-mon[36302]: pgmap v5071: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:08:37.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:08:37 smithi132 ceph-mon[35941]: pgmap v5071: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:08:39.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:08:39 smithi165 ceph-mon[36302]: pgmap v5072: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:08:39.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:08:39 smithi119 ceph-mon[31518]: pgmap v5072: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:08:39.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:08:39 smithi132 ceph-mon[35941]: pgmap v5072: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:08:41.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:08:41 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9180.791630 secs 2024-03-28T03:08:41.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:08:41 smithi165 ceph-mon[36302]: pgmap v5073: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:08:41.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:08:41 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9180.791630 secs 2024-03-28T03:08:41.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:08:41 smithi119 ceph-mon[31518]: pgmap v5073: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:08:41.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:08:41 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9180.791630 secs 2024-03-28T03:08:41.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:08:41 smithi132 ceph-mon[35941]: pgmap v5073: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:08:43.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:08:43 smithi165 ceph-mon[36302]: pgmap v5074: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:08:43.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:08:43 smithi119 ceph-mon[31518]: pgmap v5074: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:08:43.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:08:43 smithi132 ceph-mon[35941]: pgmap v5074: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:08:45.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:08:45 smithi165 ceph-mon[36302]: pgmap v5075: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:08:45.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:08:45 smithi119 ceph-mon[31518]: pgmap v5075: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:08:45.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:08:45 smithi132 ceph-mon[35941]: pgmap v5075: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:08:46.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:08:46 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9185.791718 secs 2024-03-28T03:08:46.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:08:46 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9185.791718 secs 2024-03-28T03:08:46.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:08:46 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9185.791718 secs 2024-03-28T03:08:47.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:08:47 smithi119 ceph-mon[31518]: pgmap v5076: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:08:47.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:08:47 smithi132 ceph-mon[35941]: pgmap v5076: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:08:47.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:08:47 smithi165 ceph-mon[36302]: pgmap v5076: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:08:49.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:08:49 smithi119 ceph-mon[31518]: pgmap v5077: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:08:49.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:08:49 smithi132 ceph-mon[35941]: pgmap v5077: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:08:49.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:08:49 smithi165 ceph-mon[36302]: pgmap v5077: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:08:51.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:08:51 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9190.791757 secs 2024-03-28T03:08:51.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:08:51 smithi119 ceph-mon[31518]: pgmap v5078: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:08:51.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:08:51 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T03:08:51.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:08:51 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9190.791757 secs 2024-03-28T03:08:51.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:08:51 smithi132 ceph-mon[35941]: pgmap v5078: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:08:51.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:08:51 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T03:08:51.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:08:51 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9190.791757 secs 2024-03-28T03:08:51.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:08:51 smithi165 ceph-mon[36302]: pgmap v5078: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:08:51.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:08:51 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T03:08:52.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:08:52 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T03:08:52.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:08:52 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T03:08:52.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:08:52 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:08:52.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:08:52 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T03:08:52.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:08:52 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T03:08:52.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:08:52 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:08:52.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:08:52 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T03:08:52.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:08:52 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T03:08:52.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:08:52 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:08:53.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:08:53 smithi119 ceph-mon[31518]: pgmap v5079: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T03:08:53.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:08:53 smithi119 ceph-mon[31518]: pgmap v5080: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:08:53.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:08:53 smithi132 ceph-mon[35941]: pgmap v5079: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T03:08:53.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:08:53 smithi132 ceph-mon[35941]: pgmap v5080: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:08:53.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:08:53 smithi165 ceph-mon[36302]: pgmap v5079: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T03:08:53.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:08:53 smithi165 ceph-mon[36302]: pgmap v5080: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:08:55.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:08:55 smithi132 ceph-mon[35941]: pgmap v5081: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-28T03:08:55.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:08:55 smithi165 ceph-mon[36302]: pgmap v5081: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-28T03:08:55.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:08:55 smithi119 ceph-mon[31518]: pgmap v5081: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-28T03:08:56.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:08:56 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9195.791820 secs 2024-03-28T03:08:56.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:08:56 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9195.791820 secs 2024-03-28T03:08:56.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:08:56 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9195.791820 secs 2024-03-28T03:08:57.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:08:57 smithi132 ceph-mon[35941]: pgmap v5082: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:08:57.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:08:57 smithi165 ceph-mon[36302]: pgmap v5082: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:08:57.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:08:57 smithi119 ceph-mon[31518]: pgmap v5082: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:08:59.701 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:08:59 smithi132 ceph-mon[35941]: pgmap v5083: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:08:59.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:08:59 smithi165 ceph-mon[36302]: pgmap v5083: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:08:59.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:08:59 smithi119 ceph-mon[31518]: pgmap v5083: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:09:01.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:09:01 smithi132 ceph-mon[35941]: pgmap v5084: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:09:01.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:09:01 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9200.791866 secs 2024-03-28T03:09:01.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:09:01 smithi165 ceph-mon[36302]: pgmap v5084: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:09:01.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:09:01 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9200.791866 secs 2024-03-28T03:09:01.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:09:01 smithi119 ceph-mon[31518]: pgmap v5084: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:09:01.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:09:01 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9200.791866 secs 2024-03-28T03:09:03.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:09:03 smithi132 ceph-mon[35941]: pgmap v5085: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T03:09:03.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:09:03 smithi165 ceph-mon[36302]: pgmap v5085: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T03:09:03.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:09:03 smithi119 ceph-mon[31518]: pgmap v5085: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T03:09:05.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:09:05 smithi132 ceph-mon[35941]: pgmap v5086: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:09:05.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:09:05 smithi165 ceph-mon[36302]: pgmap v5086: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:09:05.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:09:05 smithi119 ceph-mon[31518]: pgmap v5086: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:09:06.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:09:06 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9205.791942 secs 2024-03-28T03:09:06.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:09:06 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9205.791942 secs 2024-03-28T03:09:06.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:09:06 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9205.791942 secs 2024-03-28T03:09:07.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:09:07 smithi165 ceph-mon[36302]: pgmap v5087: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:09:07.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:09:07 smithi119 ceph-mon[31518]: pgmap v5087: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:09:07.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:09:07 smithi132 ceph-mon[35941]: pgmap v5087: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:09:09.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:09:09 smithi165 ceph-mon[36302]: pgmap v5088: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:09:09.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:09:09 smithi119 ceph-mon[31518]: pgmap v5088: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:09:09.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:09:09 smithi132 ceph-mon[35941]: pgmap v5088: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:09:11.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:09:11 smithi165 ceph-mon[36302]: pgmap v5089: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:09:11.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:09:11 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9210.791980 secs 2024-03-28T03:09:11.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:09:11 smithi119 ceph-mon[31518]: pgmap v5089: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:09:11.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:09:11 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9210.791980 secs 2024-03-28T03:09:11.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:09:11 smithi132 ceph-mon[35941]: pgmap v5089: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:09:11.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:09:11 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9210.791980 secs 2024-03-28T03:09:13.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:09:13 smithi165 ceph-mon[36302]: pgmap v5090: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:09:13.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:09:13 smithi119 ceph-mon[31518]: pgmap v5090: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:09:13.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:09:13 smithi132 ceph-mon[35941]: pgmap v5090: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:09:15.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:09:15 smithi165 ceph-mon[36302]: pgmap v5091: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:09:15.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:09:15 smithi119 ceph-mon[31518]: pgmap v5091: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:09:15.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:09:15 smithi132 ceph-mon[35941]: pgmap v5091: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:09:16.780 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:09:16 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9215.792066 secs 2024-03-28T03:09:16.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:09:16 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9215.792066 secs 2024-03-28T03:09:16.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:09:16 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9215.792066 secs 2024-03-28T03:09:17.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:09:17 smithi165 ceph-mon[36302]: pgmap v5092: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:09:17.905 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:09:17 smithi119 ceph-mon[31518]: pgmap v5092: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:09:17.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:09:17 smithi132 ceph-mon[35941]: pgmap v5092: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:09:19.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:09:19 smithi165 ceph-mon[36302]: pgmap v5093: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T03:09:19.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:09:19 smithi119 ceph-mon[31518]: pgmap v5093: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T03:09:19.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:09:19 smithi132 ceph-mon[35941]: pgmap v5093: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T03:09:21.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:09:21 smithi165 ceph-mon[36302]: pgmap v5094: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:09:21.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:09:21 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9220.792125 secs 2024-03-28T03:09:21.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:09:21 smithi119 ceph-mon[31518]: pgmap v5094: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:09:21.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:09:21 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9220.792125 secs 2024-03-28T03:09:21.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:09:21 smithi132 ceph-mon[35941]: pgmap v5094: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:09:21.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:09:21 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9220.792125 secs 2024-03-28T03:09:23.875 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:09:23 smithi119 ceph-mon[31518]: pgmap v5095: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T03:09:23.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:09:23 smithi165 ceph-mon[36302]: pgmap v5095: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T03:09:23.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:09:23 smithi132 ceph-mon[35941]: pgmap v5095: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T03:09:25.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:09:25 smithi165 ceph-mon[36302]: pgmap v5096: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T03:09:25.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:09:25 smithi119 ceph-mon[31518]: pgmap v5096: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T03:09:25.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:09:25 smithi132 ceph-mon[35941]: pgmap v5096: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T03:09:26.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:09:26 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9225.792141 secs 2024-03-28T03:09:26.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:09:26 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9225.792141 secs 2024-03-28T03:09:26.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:09:26 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9225.792141 secs 2024-03-28T03:09:27.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:09:27 smithi165 ceph-mon[36302]: pgmap v5097: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:09:27.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:09:27 smithi119 ceph-mon[31518]: pgmap v5097: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:09:27.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:09:27 smithi132 ceph-mon[35941]: pgmap v5097: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:09:29.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:09:29 smithi165 ceph-mon[36302]: pgmap v5098: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:09:29.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:09:29 smithi119 ceph-mon[31518]: pgmap v5098: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:09:29.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:09:29 smithi132 ceph-mon[35941]: pgmap v5098: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:09:31.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:09:31 smithi165 ceph-mon[36302]: pgmap v5099: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:09:31.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:09:31 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9230.792202 secs 2024-03-28T03:09:31.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:09:31 smithi119 ceph-mon[31518]: pgmap v5099: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:09:31.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:09:31 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9230.792202 secs 2024-03-28T03:09:31.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:09:31 smithi132 ceph-mon[35941]: pgmap v5099: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:09:31.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:09:31 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9230.792202 secs 2024-03-28T03:09:33.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:09:33 smithi165 ceph-mon[36302]: pgmap v5100: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:09:33.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:09:33 smithi119 ceph-mon[31518]: pgmap v5100: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:09:33.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:09:33 smithi132 ceph-mon[35941]: pgmap v5100: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:09:35.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:09:35 smithi165 ceph-mon[36302]: pgmap v5101: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:09:35.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:09:35 smithi119 ceph-mon[31518]: pgmap v5101: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:09:35.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:09:35 smithi132 ceph-mon[35941]: pgmap v5101: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:09:36.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:09:36 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9235.792290 secs 2024-03-28T03:09:36.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:09:36 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9235.792290 secs 2024-03-28T03:09:36.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:09:36 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9235.792290 secs 2024-03-28T03:09:37.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:09:37 smithi165 ceph-mon[36302]: pgmap v5102: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:09:37.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:09:37 smithi119 ceph-mon[31518]: pgmap v5102: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:09:37.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:09:37 smithi132 ceph-mon[35941]: pgmap v5102: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:09:39.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:09:39 smithi165 ceph-mon[36302]: pgmap v5103: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:09:39.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:09:39 smithi119 ceph-mon[31518]: pgmap v5103: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:09:39.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:09:39 smithi132 ceph-mon[35941]: pgmap v5103: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:09:41.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:09:41 smithi165 ceph-mon[36302]: pgmap v5104: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:09:41.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:09:41 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9240.792380 secs 2024-03-28T03:09:41.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:09:41 smithi119 ceph-mon[31518]: pgmap v5104: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:09:41.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:09:41 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9240.792380 secs 2024-03-28T03:09:41.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:09:41 smithi132 ceph-mon[35941]: pgmap v5104: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:09:41.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:09:41 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9240.792380 secs 2024-03-28T03:09:43.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:09:43 smithi165 ceph-mon[36302]: pgmap v5105: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:09:43.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:09:43 smithi119 ceph-mon[31518]: pgmap v5105: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:09:43.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:09:43 smithi132 ceph-mon[35941]: pgmap v5105: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:09:45.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:09:45 smithi165 ceph-mon[36302]: pgmap v5106: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:09:45.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:09:45 smithi119 ceph-mon[31518]: pgmap v5106: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:09:45.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:09:45 smithi132 ceph-mon[35941]: pgmap v5106: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:09:46.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:09:46 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9245.792470 secs 2024-03-28T03:09:46.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:09:46 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9245.792470 secs 2024-03-28T03:09:46.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:09:46 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9245.792470 secs 2024-03-28T03:09:47.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:09:47 smithi165 ceph-mon[36302]: pgmap v5107: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:09:47.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:09:47 smithi119 ceph-mon[31518]: pgmap v5107: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:09:47.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:09:47 smithi132 ceph-mon[35941]: pgmap v5107: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:09:49.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:09:49 smithi165 ceph-mon[36302]: pgmap v5108: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:09:49.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:09:49 smithi119 ceph-mon[31518]: pgmap v5108: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:09:49.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:09:49 smithi132 ceph-mon[35941]: pgmap v5108: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:09:51.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:09:51 smithi165 ceph-mon[36302]: pgmap v5109: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:09:51.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:09:51 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9250.792531 secs 2024-03-28T03:09:51.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:09:51 smithi119 ceph-mon[31518]: pgmap v5109: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:09:51.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:09:51 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9250.792531 secs 2024-03-28T03:09:51.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:09:51 smithi132 ceph-mon[35941]: pgmap v5109: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:09:51.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:09:51 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9250.792531 secs 2024-03-28T03:09:52.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:09:52 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T03:09:52.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:09:52 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T03:09:52.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:09:52 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T03:09:53.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:09:53 smithi165 ceph-mon[36302]: pgmap v5110: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:09:53.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:09:53 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T03:09:53.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:09:53 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T03:09:53.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:09:53 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:09:53.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:09:53 smithi119 ceph-mon[31518]: pgmap v5110: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:09:53.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:09:53 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T03:09:53.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:09:53 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T03:09:53.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:09:53 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:09:53.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:09:53 smithi132 ceph-mon[35941]: pgmap v5110: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:09:53.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:09:53 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T03:09:53.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:09:53 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T03:09:53.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:09:53 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:09:54.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:09:54 smithi165 ceph-mon[36302]: pgmap v5111: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T03:09:54.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:09:54 smithi165 ceph-mon[36302]: pgmap v5112: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:09:54.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:09:54 smithi119 ceph-mon[31518]: pgmap v5111: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T03:09:54.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:09:54 smithi119 ceph-mon[31518]: pgmap v5112: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:09:54.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:09:54 smithi132 ceph-mon[35941]: pgmap v5111: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T03:09:54.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:09:54 smithi132 ceph-mon[35941]: pgmap v5112: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:09:56.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:09:56 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9255.792621 secs 2024-03-28T03:09:56.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:09:56 smithi165 ceph-mon[36302]: pgmap v5113: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:09:56.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:09:56 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9255.792621 secs 2024-03-28T03:09:56.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:09:56 smithi119 ceph-mon[31518]: pgmap v5113: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:09:56.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:09:56 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9255.792621 secs 2024-03-28T03:09:56.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:09:56 smithi132 ceph-mon[35941]: pgmap v5113: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:09:58.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:09:58 smithi165 ceph-mon[36302]: pgmap v5114: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:09:58.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:09:58 smithi119 ceph-mon[31518]: pgmap v5114: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:09:58.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:09:58 smithi132 ceph-mon[35941]: pgmap v5114: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:10:00.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:10:00 smithi165 ceph-mon[36302]: pgmap v5115: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:10:00.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:10:00 smithi165 ceph-mon[36302]: overall HEALTH_WARN 5 failed cephadm daemon(s); 1 filesystem is degraded; 1 MDSs report slow requests 2024-03-28T03:10:00.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:10:00 smithi119 ceph-mon[31518]: pgmap v5115: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:10:00.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:10:00 smithi119 ceph-mon[31518]: overall HEALTH_WARN 5 failed cephadm daemon(s); 1 filesystem is degraded; 1 MDSs report slow requests 2024-03-28T03:10:00.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:10:00 smithi132 ceph-mon[35941]: pgmap v5115: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:10:00.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:10:00 smithi132 ceph-mon[35941]: overall HEALTH_WARN 5 failed cephadm daemon(s); 1 filesystem is degraded; 1 MDSs report slow requests 2024-03-28T03:10:01.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:10:01 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9260.792709 secs 2024-03-28T03:10:01.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:10:01 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9260.792709 secs 2024-03-28T03:10:02.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:10:01 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9260.792709 secs 2024-03-28T03:10:02.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:10:02 smithi119 ceph-mon[31518]: pgmap v5116: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:10:02.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:10:02 smithi132 ceph-mon[35941]: pgmap v5116: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:10:03.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:10:02 smithi165 ceph-mon[36302]: pgmap v5116: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:10:04.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:10:04 smithi119 ceph-mon[31518]: pgmap v5117: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T03:10:04.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:10:04 smithi132 ceph-mon[35941]: pgmap v5117: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T03:10:05.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:10:04 smithi165 ceph-mon[36302]: pgmap v5117: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T03:10:06.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:10:06 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9265.792798 secs 2024-03-28T03:10:06.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:10:06 smithi119 ceph-mon[31518]: pgmap v5118: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:10:06.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:10:06 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9265.792798 secs 2024-03-28T03:10:06.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:10:06 smithi132 ceph-mon[35941]: pgmap v5118: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:10:07.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:10:06 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9265.792798 secs 2024-03-28T03:10:07.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:10:06 smithi165 ceph-mon[36302]: pgmap v5118: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:10:08.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:10:08 smithi132 ceph-mon[35941]: pgmap v5119: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:10:09.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:10:08 smithi165 ceph-mon[36302]: pgmap v5119: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:10:09.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:10:08 smithi119 ceph-mon[31518]: pgmap v5119: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:10:10.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:10:10 smithi132 ceph-mon[35941]: pgmap v5120: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:10:10.979 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:10:10 smithi119 ceph-mon[31518]: pgmap v5120: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:10:11.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:10:10 smithi165 ceph-mon[36302]: pgmap v5120: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:10:11.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:10:11 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9270.792883 secs 2024-03-28T03:10:12.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:10:11 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9270.792883 secs 2024-03-28T03:10:12.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:10:11 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9270.792883 secs 2024-03-28T03:10:12.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:10:12 smithi132 ceph-mon[35941]: pgmap v5121: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:10:13.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:10:12 smithi165 ceph-mon[36302]: pgmap v5121: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:10:13.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:10:12 smithi119 ceph-mon[31518]: pgmap v5121: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:10:13.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:10:13 smithi132 ceph-mon[35941]: pgmap v5122: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:10:14.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:10:13 smithi165 ceph-mon[36302]: pgmap v5122: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:10:14.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:10:13 smithi119 ceph-mon[31518]: pgmap v5122: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:10:16.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:10:16 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9275.792900 secs 2024-03-28T03:10:16.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:10:16 smithi119 ceph-mon[31518]: pgmap v5123: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:10:16.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:10:16 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9275.792900 secs 2024-03-28T03:10:16.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:10:16 smithi132 ceph-mon[35941]: pgmap v5123: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:10:16.832 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:10:16 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9275.792900 secs 2024-03-28T03:10:16.832 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:10:16 smithi165 ceph-mon[36302]: pgmap v5123: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:10:18.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:10:18 smithi119 ceph-mon[31518]: pgmap v5124: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:10:18.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:10:18 smithi132 ceph-mon[35941]: pgmap v5124: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:10:18.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:10:18 smithi165 ceph-mon[36302]: pgmap v5124: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:10:20.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:10:20 smithi119 ceph-mon[31518]: pgmap v5125: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:10:20.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:10:20 smithi132 ceph-mon[35941]: pgmap v5125: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:10:20.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:10:20 smithi165 ceph-mon[36302]: pgmap v5125: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:10:21.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:10:21 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9280.792965 secs 2024-03-28T03:10:21.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:10:21 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9280.792965 secs 2024-03-28T03:10:21.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:10:21 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9280.792965 secs 2024-03-28T03:10:22.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:10:22 smithi132 ceph-mon[35941]: pgmap v5126: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:10:22.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:10:22 smithi165 ceph-mon[36302]: pgmap v5126: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:10:22.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:10:22 smithi119 ceph-mon[31518]: pgmap v5126: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:10:24.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:10:24 smithi132 ceph-mon[35941]: pgmap v5127: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:10:24.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:10:24 smithi165 ceph-mon[36302]: pgmap v5127: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:10:24.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:10:24 smithi119 ceph-mon[31518]: pgmap v5127: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:10:26.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:10:26 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9285.793025 secs 2024-03-28T03:10:26.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:10:26 smithi119 ceph-mon[31518]: pgmap v5128: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:10:26.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:10:26 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9285.793025 secs 2024-03-28T03:10:26.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:10:26 smithi132 ceph-mon[35941]: pgmap v5128: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:10:26.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:10:26 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9285.793025 secs 2024-03-28T03:10:26.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:10:26 smithi165 ceph-mon[36302]: pgmap v5128: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:10:28.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:10:28 smithi132 ceph-mon[35941]: pgmap v5129: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:10:28.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:10:28 smithi165 ceph-mon[36302]: pgmap v5129: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:10:28.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:10:28 smithi119 ceph-mon[31518]: pgmap v5129: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:10:30.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:10:30 smithi132 ceph-mon[35941]: pgmap v5130: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:10:30.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:10:30 smithi165 ceph-mon[36302]: pgmap v5130: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:10:30.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:10:30 smithi119 ceph-mon[31518]: pgmap v5130: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:10:31.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:10:31 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9290.793087 secs 2024-03-28T03:10:31.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:10:31 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9290.793087 secs 2024-03-28T03:10:31.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:10:31 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9290.793087 secs 2024-03-28T03:10:32.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:10:32 smithi132 ceph-mon[35941]: pgmap v5131: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T03:10:32.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:10:32 smithi165 ceph-mon[36302]: pgmap v5131: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T03:10:32.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:10:32 smithi119 ceph-mon[31518]: pgmap v5131: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T03:10:34.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:10:34 smithi132 ceph-mon[35941]: pgmap v5132: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T03:10:34.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:10:34 smithi165 ceph-mon[36302]: pgmap v5132: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T03:10:34.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:10:34 smithi119 ceph-mon[31518]: pgmap v5132: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T03:10:36.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:10:36 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9295.793163 secs 2024-03-28T03:10:36.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:10:36 smithi132 ceph-mon[35941]: pgmap v5133: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:10:36.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:10:36 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9295.793163 secs 2024-03-28T03:10:36.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:10:36 smithi165 ceph-mon[36302]: pgmap v5133: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:10:36.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:10:36 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9295.793163 secs 2024-03-28T03:10:36.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:10:36 smithi119 ceph-mon[31518]: pgmap v5133: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:10:38.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:10:38 smithi132 ceph-mon[35941]: pgmap v5134: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T03:10:38.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:10:38 smithi165 ceph-mon[36302]: pgmap v5134: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T03:10:38.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:10:38 smithi119 ceph-mon[31518]: pgmap v5134: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T03:10:40.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:10:40 smithi165 ceph-mon[36302]: pgmap v5135: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:10:40.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:10:40 smithi119 ceph-mon[31518]: pgmap v5135: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:10:40.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:10:40 smithi132 ceph-mon[35941]: pgmap v5135: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:10:41.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:10:41 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9300.793223 secs 2024-03-28T03:10:41.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:10:41 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9300.793223 secs 2024-03-28T03:10:41.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:10:41 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9300.793223 secs 2024-03-28T03:10:42.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:10:42 smithi165 ceph-mon[36302]: pgmap v5136: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:10:42.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:10:42 smithi119 ceph-mon[31518]: pgmap v5136: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:10:42.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:10:42 smithi132 ceph-mon[35941]: pgmap v5136: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:10:44.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:10:44 smithi165 ceph-mon[36302]: pgmap v5137: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:10:44.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:10:44 smithi119 ceph-mon[31518]: pgmap v5137: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:10:44.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:10:44 smithi132 ceph-mon[35941]: pgmap v5137: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:10:46.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:10:46 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9305.793313 secs 2024-03-28T03:10:46.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:10:46 smithi165 ceph-mon[36302]: pgmap v5138: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:10:46.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:10:46 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9305.793313 secs 2024-03-28T03:10:46.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:10:46 smithi119 ceph-mon[31518]: pgmap v5138: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:10:46.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:10:46 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9305.793313 secs 2024-03-28T03:10:46.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:10:46 smithi132 ceph-mon[35941]: pgmap v5138: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:10:48.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:10:48 smithi165 ceph-mon[36302]: pgmap v5139: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:10:48.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:10:48 smithi119 ceph-mon[31518]: pgmap v5139: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:10:48.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:10:48 smithi132 ceph-mon[35941]: pgmap v5139: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:10:50.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:10:50 smithi165 ceph-mon[36302]: pgmap v5140: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:10:50.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:10:50 smithi119 ceph-mon[31518]: pgmap v5140: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:10:50.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:10:50 smithi132 ceph-mon[35941]: pgmap v5140: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:10:51.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:10:51 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9310.793351 secs 2024-03-28T03:10:51.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:10:51 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9310.793351 secs 2024-03-28T03:10:51.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:10:51 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9310.793351 secs 2024-03-28T03:10:52.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:10:52 smithi165 ceph-mon[36302]: pgmap v5141: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:10:52.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:10:52 smithi119 ceph-mon[31518]: pgmap v5141: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:10:52.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:10:52 smithi132 ceph-mon[35941]: pgmap v5141: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:10:54.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:10:54 smithi165 ceph-mon[36302]: pgmap v5142: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:10:54.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:10:54 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T03:10:54.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:10:54 smithi119 ceph-mon[31518]: pgmap v5142: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:10:54.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:10:54 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T03:10:54.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:10:54 smithi132 ceph-mon[35941]: pgmap v5142: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:10:54.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:10:54 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T03:10:55.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:10:55 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T03:10:55.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:10:55 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T03:10:55.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:10:55 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:10:55.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:10:55 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T03:10:55.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:10:55 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T03:10:55.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:10:55 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:10:55.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:10:55 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T03:10:55.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:10:55 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T03:10:55.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:10:55 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:10:56.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:10:56 smithi165 ceph-mon[36302]: pgmap v5143: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T03:10:56.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:10:56 smithi165 ceph-mon[36302]: pgmap v5144: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:10:56.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:10:56 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9315.793440 secs 2024-03-28T03:10:56.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:10:56 smithi119 ceph-mon[31518]: pgmap v5143: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T03:10:56.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:10:56 smithi119 ceph-mon[31518]: pgmap v5144: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:10:56.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:10:56 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9315.793440 secs 2024-03-28T03:10:56.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:10:56 smithi132 ceph-mon[35941]: pgmap v5143: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T03:10:56.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:10:56 smithi132 ceph-mon[35941]: pgmap v5144: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:10:56.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:10:56 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9315.793440 secs 2024-03-28T03:10:58.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:10:58 smithi165 ceph-mon[36302]: pgmap v5145: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:10:58.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:10:58 smithi119 ceph-mon[31518]: pgmap v5145: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:10:58.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:10:58 smithi132 ceph-mon[35941]: pgmap v5145: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:11:00.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:11:00 smithi165 ceph-mon[36302]: pgmap v5146: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-28T03:11:00.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:11:00 smithi119 ceph-mon[31518]: pgmap v5146: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-28T03:11:00.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:11:00 smithi132 ceph-mon[35941]: pgmap v5146: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-28T03:11:01.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:11:01 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9320.793481 secs 2024-03-28T03:11:01.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:11:01 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9320.793481 secs 2024-03-28T03:11:01.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:11:01 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9320.793481 secs 2024-03-28T03:11:02.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:11:02 smithi165 ceph-mon[36302]: pgmap v5147: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:11:02.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:11:02 smithi119 ceph-mon[31518]: pgmap v5147: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:11:02.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:11:02 smithi132 ceph-mon[35941]: pgmap v5147: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:11:04.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:11:04 smithi165 ceph-mon[36302]: pgmap v5148: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:11:04.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:11:04 smithi119 ceph-mon[31518]: pgmap v5148: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:11:04.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:11:04 smithi132 ceph-mon[35941]: pgmap v5148: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:11:06.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:11:06 smithi165 ceph-mon[36302]: pgmap v5149: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T03:11:06.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:11:06 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9325.793563 secs 2024-03-28T03:11:06.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:11:06 smithi119 ceph-mon[31518]: pgmap v5149: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T03:11:06.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:11:06 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9325.793563 secs 2024-03-28T03:11:06.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:11:06 smithi132 ceph-mon[35941]: pgmap v5149: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T03:11:06.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:11:06 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9325.793563 secs 2024-03-28T03:11:08.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:11:08 smithi165 ceph-mon[36302]: pgmap v5150: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:11:08.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:11:08 smithi119 ceph-mon[31518]: pgmap v5150: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:11:08.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:11:08 smithi132 ceph-mon[35941]: pgmap v5150: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:11:10.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:11:10 smithi165 ceph-mon[36302]: pgmap v5151: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:11:10.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:11:10 smithi119 ceph-mon[31518]: pgmap v5151: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:11:10.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:11:10 smithi132 ceph-mon[35941]: pgmap v5151: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:11:11.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:11:11 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9330.793650 secs 2024-03-28T03:11:11.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:11:11 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9330.793650 secs 2024-03-28T03:11:11.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:11:11 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9330.793650 secs 2024-03-28T03:11:12.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:11:12 smithi165 ceph-mon[36302]: pgmap v5152: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:11:12.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:11:12 smithi119 ceph-mon[31518]: pgmap v5152: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:11:12.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:11:12 smithi132 ceph-mon[35941]: pgmap v5152: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:11:14.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:11:14 smithi165 ceph-mon[36302]: pgmap v5153: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:11:14.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:11:14 smithi119 ceph-mon[31518]: pgmap v5153: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:11:14.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:11:14 smithi132 ceph-mon[35941]: pgmap v5153: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:11:16.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:11:16 smithi165 ceph-mon[36302]: pgmap v5154: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:11:16.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:11:16 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9335.793685 secs 2024-03-28T03:11:16.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:11:16 smithi119 ceph-mon[31518]: pgmap v5154: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:11:16.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:11:16 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9335.793685 secs 2024-03-28T03:11:16.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:11:16 smithi132 ceph-mon[35941]: pgmap v5154: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:11:16.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:11:16 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9335.793685 secs 2024-03-28T03:11:18.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:11:18 smithi165 ceph-mon[36302]: pgmap v5155: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:11:18.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:11:18 smithi119 ceph-mon[31518]: pgmap v5155: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:11:18.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:11:18 smithi132 ceph-mon[35941]: pgmap v5155: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:11:20.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:11:20 smithi165 ceph-mon[36302]: pgmap v5156: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:11:20.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:11:20 smithi119 ceph-mon[31518]: pgmap v5156: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:11:20.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:11:20 smithi132 ceph-mon[35941]: pgmap v5156: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:11:21.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:11:21 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9340.793774 secs 2024-03-28T03:11:21.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:11:21 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9340.793774 secs 2024-03-28T03:11:21.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:11:21 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9340.793774 secs 2024-03-28T03:11:22.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:11:22 smithi165 ceph-mon[36302]: pgmap v5157: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:11:22.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:11:22 smithi119 ceph-mon[31518]: pgmap v5157: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:11:22.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:11:22 smithi132 ceph-mon[35941]: pgmap v5157: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:11:24.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:11:24 smithi165 ceph-mon[36302]: pgmap v5158: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:11:24.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:11:24 smithi119 ceph-mon[31518]: pgmap v5158: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:11:24.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:11:24 smithi132 ceph-mon[35941]: pgmap v5158: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:11:26.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:11:26 smithi165 ceph-mon[36302]: pgmap v5159: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:11:26.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:11:26 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9345.793812 secs 2024-03-28T03:11:26.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:11:26 smithi119 ceph-mon[31518]: pgmap v5159: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:11:26.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:11:26 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9345.793812 secs 2024-03-28T03:11:26.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:11:26 smithi132 ceph-mon[35941]: pgmap v5159: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:11:26.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:11:26 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9345.793812 secs 2024-03-28T03:11:28.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:11:28 smithi165 ceph-mon[36302]: pgmap v5160: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:11:28.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:11:28 smithi119 ceph-mon[31518]: pgmap v5160: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:11:28.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:11:28 smithi132 ceph-mon[35941]: pgmap v5160: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:11:30.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:11:30 smithi165 ceph-mon[36302]: pgmap v5161: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:11:30.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:11:30 smithi119 ceph-mon[31518]: pgmap v5161: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:11:30.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:11:30 smithi132 ceph-mon[35941]: pgmap v5161: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:11:31.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:11:31 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9350.793899 secs 2024-03-28T03:11:31.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:11:31 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9350.793899 secs 2024-03-28T03:11:31.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:11:31 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9350.793899 secs 2024-03-28T03:11:32.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:11:32 smithi165 ceph-mon[36302]: pgmap v5162: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:11:32.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:11:32 smithi119 ceph-mon[31518]: pgmap v5162: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:11:32.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:11:32 smithi132 ceph-mon[35941]: pgmap v5162: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:11:34.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:11:34 smithi119 ceph-mon[31518]: pgmap v5163: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:11:34.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:11:34 smithi132 ceph-mon[35941]: pgmap v5163: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:11:35.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:11:34 smithi165 ceph-mon[36302]: pgmap v5163: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:11:36.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:11:36 smithi119 ceph-mon[31518]: pgmap v5164: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:11:36.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:11:36 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9355.793917 secs 2024-03-28T03:11:36.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:11:36 smithi132 ceph-mon[35941]: pgmap v5164: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:11:36.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:11:36 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9355.793917 secs 2024-03-28T03:11:37.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:11:36 smithi165 ceph-mon[36302]: pgmap v5164: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:11:37.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:11:36 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9355.793917 secs 2024-03-28T03:11:38.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:11:38 smithi119 ceph-mon[31518]: pgmap v5165: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:11:38.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:11:38 smithi132 ceph-mon[35941]: pgmap v5165: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:11:39.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:11:38 smithi165 ceph-mon[36302]: pgmap v5165: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:11:40.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:11:40 smithi119 ceph-mon[31518]: pgmap v5166: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:11:40.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:11:40 smithi132 ceph-mon[35941]: pgmap v5166: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:11:41.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:11:40 smithi165 ceph-mon[36302]: pgmap v5166: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:11:41.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:11:41 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9360.793978 secs 2024-03-28T03:11:42.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:11:41 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9360.793978 secs 2024-03-28T03:11:42.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:11:41 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9360.793978 secs 2024-03-28T03:11:42.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:11:42 smithi132 ceph-mon[35941]: pgmap v5167: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:11:43.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:11:42 smithi165 ceph-mon[36302]: pgmap v5167: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:11:43.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:11:42 smithi119 ceph-mon[31518]: pgmap v5167: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:11:44.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:11:44 smithi132 ceph-mon[35941]: pgmap v5168: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:11:45.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:11:44 smithi165 ceph-mon[36302]: pgmap v5168: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:11:45.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:11:44 smithi119 ceph-mon[31518]: pgmap v5168: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:11:46.931 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:11:46 smithi165 ceph-mon[36302]: pgmap v5169: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:11:46.931 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:11:46 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9365.794040 secs 2024-03-28T03:11:46.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:11:46 smithi132 ceph-mon[35941]: pgmap v5169: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:11:46.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:11:46 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9365.794040 secs 2024-03-28T03:11:47.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:11:46 smithi119 ceph-mon[31518]: pgmap v5169: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:11:47.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:11:46 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9365.794040 secs 2024-03-28T03:11:48.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:11:48 smithi132 ceph-mon[35941]: pgmap v5170: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:11:49.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:11:48 smithi165 ceph-mon[36302]: pgmap v5170: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:11:49.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:11:48 smithi119 ceph-mon[31518]: pgmap v5170: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:11:50.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:11:50 smithi132 ceph-mon[35941]: pgmap v5171: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:11:50.954 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:11:50 smithi119 ceph-mon[31518]: pgmap v5171: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:11:51.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:11:50 smithi165 ceph-mon[36302]: pgmap v5171: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:11:51.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:11:51 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9370.794100 secs 2024-03-28T03:11:52.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:11:51 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9370.794100 secs 2024-03-28T03:11:52.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:11:51 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9370.794100 secs 2024-03-28T03:11:52.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:11:52 smithi132 ceph-mon[35941]: pgmap v5172: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:11:53.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:11:52 smithi165 ceph-mon[36302]: pgmap v5172: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:11:53.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:11:52 smithi119 ceph-mon[31518]: pgmap v5172: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:11:54.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:11:53 smithi165 ceph-mon[36302]: pgmap v5173: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:11:54.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:11:53 smithi119 ceph-mon[31518]: pgmap v5173: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:11:54.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:11:53 smithi132 ceph-mon[35941]: pgmap v5173: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:11:56.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:11:55 smithi165 ceph-mon[36302]: pgmap v5174: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:11:56.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:11:55 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9375.794201 secs 2024-03-28T03:11:56.383 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:11:55 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T03:11:56.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:11:55 smithi119 ceph-mon[31518]: pgmap v5174: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:11:56.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:11:55 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9375.794201 secs 2024-03-28T03:11:56.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:11:55 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T03:11:56.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:11:55 smithi132 ceph-mon[35941]: pgmap v5174: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:11:56.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:11:55 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9375.794201 secs 2024-03-28T03:11:56.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:11:55 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T03:11:57.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:11:56 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T03:11:57.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:11:56 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T03:11:57.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:11:56 smithi165 ceph-mon[36302]: pgmap v5175: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T03:11:57.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:11:56 smithi165 ceph-mon[36302]: pgmap v5176: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:11:57.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:11:56 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:11:57.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:11:56 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T03:11:57.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:11:56 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T03:11:57.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:11:56 smithi119 ceph-mon[31518]: pgmap v5175: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T03:11:57.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:11:56 smithi119 ceph-mon[31518]: pgmap v5176: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:11:57.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:11:56 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:11:57.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:11:56 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T03:11:57.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:11:56 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T03:11:57.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:11:56 smithi132 ceph-mon[35941]: pgmap v5175: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T03:11:57.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:11:56 smithi132 ceph-mon[35941]: pgmap v5176: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:11:57.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:11:56 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:11:59.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:11:59 smithi165 ceph-mon[36302]: pgmap v5177: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:11:59.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:11:59 smithi119 ceph-mon[31518]: pgmap v5177: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:11:59.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:11:59 smithi132 ceph-mon[35941]: pgmap v5177: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:12:01.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:12:01 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9380.794261 secs 2024-03-28T03:12:01.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:12:01 smithi165 ceph-mon[36302]: pgmap v5178: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:12:01.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:12:01 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9380.794261 secs 2024-03-28T03:12:01.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:12:01 smithi119 ceph-mon[31518]: pgmap v5178: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:12:01.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:12:01 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9380.794261 secs 2024-03-28T03:12:01.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:12:01 smithi132 ceph-mon[35941]: pgmap v5178: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:12:03.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:12:03 smithi165 ceph-mon[36302]: pgmap v5179: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:12:03.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:12:03 smithi119 ceph-mon[31518]: pgmap v5179: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:12:03.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:12:03 smithi132 ceph-mon[35941]: pgmap v5179: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:12:05.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:12:05 smithi165 ceph-mon[36302]: pgmap v5180: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:12:05.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:12:05 smithi119 ceph-mon[31518]: pgmap v5180: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:12:05.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:12:05 smithi132 ceph-mon[35941]: pgmap v5180: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:12:06.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:12:06 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9385.794322 secs 2024-03-28T03:12:06.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:12:06 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9385.794322 secs 2024-03-28T03:12:06.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:12:06 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9385.794322 secs 2024-03-28T03:12:07.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:12:07 smithi165 ceph-mon[36302]: pgmap v5181: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T03:12:07.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:12:07 smithi119 ceph-mon[31518]: pgmap v5181: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T03:12:07.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:12:07 smithi132 ceph-mon[35941]: pgmap v5181: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T03:12:09.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:12:09 smithi165 ceph-mon[36302]: pgmap v5182: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:12:09.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:12:09 smithi119 ceph-mon[31518]: pgmap v5182: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:12:09.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:12:09 smithi132 ceph-mon[35941]: pgmap v5182: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:12:11.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:12:11 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9390.794382 secs 2024-03-28T03:12:11.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:12:11 smithi165 ceph-mon[36302]: pgmap v5183: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:12:11.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:12:11 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9390.794382 secs 2024-03-28T03:12:11.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:12:11 smithi119 ceph-mon[31518]: pgmap v5183: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:12:11.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:12:11 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9390.794382 secs 2024-03-28T03:12:11.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:12:11 smithi132 ceph-mon[35941]: pgmap v5183: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:12:13.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:12:13 smithi165 ceph-mon[36302]: pgmap v5184: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:12:13.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:12:13 smithi119 ceph-mon[31518]: pgmap v5184: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:12:13.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:12:13 smithi132 ceph-mon[35941]: pgmap v5184: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:12:15.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:12:15 smithi165 ceph-mon[36302]: pgmap v5185: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:12:15.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:12:15 smithi119 ceph-mon[31518]: pgmap v5185: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:12:15.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:12:15 smithi132 ceph-mon[35941]: pgmap v5185: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:12:16.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:12:16 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9395.794443 secs 2024-03-28T03:12:16.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:12:16 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9395.794443 secs 2024-03-28T03:12:16.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:12:16 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9395.794443 secs 2024-03-28T03:12:17.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:12:17 smithi165 ceph-mon[36302]: pgmap v5186: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:12:17.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:12:17 smithi119 ceph-mon[31518]: pgmap v5186: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:12:17.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:12:17 smithi132 ceph-mon[35941]: pgmap v5186: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:12:19.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:12:19 smithi165 ceph-mon[36302]: pgmap v5187: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:12:19.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:12:19 smithi119 ceph-mon[31518]: pgmap v5187: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:12:19.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:12:19 smithi132 ceph-mon[35941]: pgmap v5187: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:12:21.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:12:21 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9400.794504 secs 2024-03-28T03:12:21.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:12:21 smithi165 ceph-mon[36302]: pgmap v5188: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:12:21.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:12:21 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9400.794504 secs 2024-03-28T03:12:21.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:12:21 smithi119 ceph-mon[31518]: pgmap v5188: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:12:21.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:12:21 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9400.794504 secs 2024-03-28T03:12:21.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:12:21 smithi132 ceph-mon[35941]: pgmap v5188: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:12:23.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:12:23 smithi165 ceph-mon[36302]: pgmap v5189: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:12:23.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:12:23 smithi119 ceph-mon[31518]: pgmap v5189: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:12:23.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:12:23 smithi132 ceph-mon[35941]: pgmap v5189: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:12:25.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:12:25 smithi165 ceph-mon[36302]: pgmap v5190: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:12:25.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:12:25 smithi119 ceph-mon[31518]: pgmap v5190: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:12:25.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:12:25 smithi132 ceph-mon[35941]: pgmap v5190: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:12:26.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:12:26 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9405.794563 secs 2024-03-28T03:12:26.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:12:26 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9405.794563 secs 2024-03-28T03:12:26.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:12:26 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9405.794563 secs 2024-03-28T03:12:27.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:12:27 smithi165 ceph-mon[36302]: pgmap v5191: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:12:27.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:12:27 smithi119 ceph-mon[31518]: pgmap v5191: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:12:27.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:12:27 smithi132 ceph-mon[35941]: pgmap v5191: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:12:29.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:12:29 smithi165 ceph-mon[36302]: pgmap v5192: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:12:29.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:12:29 smithi119 ceph-mon[31518]: pgmap v5192: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:12:29.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:12:29 smithi132 ceph-mon[35941]: pgmap v5192: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:12:31.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:12:31 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9410.794623 secs 2024-03-28T03:12:31.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:12:31 smithi165 ceph-mon[36302]: pgmap v5193: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:12:31.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:12:31 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9410.794623 secs 2024-03-28T03:12:31.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:12:31 smithi119 ceph-mon[31518]: pgmap v5193: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:12:31.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:12:31 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9410.794623 secs 2024-03-28T03:12:31.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:12:31 smithi132 ceph-mon[35941]: pgmap v5193: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:12:33.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:12:33 smithi165 ceph-mon[36302]: pgmap v5194: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:12:33.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:12:33 smithi119 ceph-mon[31518]: pgmap v5194: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:12:33.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:12:33 smithi132 ceph-mon[35941]: pgmap v5194: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:12:35.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:12:35 smithi165 ceph-mon[36302]: pgmap v5195: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:12:35.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:12:35 smithi119 ceph-mon[31518]: pgmap v5195: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:12:35.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:12:35 smithi132 ceph-mon[35941]: pgmap v5195: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:12:36.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:12:36 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9415.794637 secs 2024-03-28T03:12:36.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:12:36 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9415.794637 secs 2024-03-28T03:12:37.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:12:36 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9415.794637 secs 2024-03-28T03:12:37.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:12:37 smithi119 ceph-mon[31518]: pgmap v5196: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:12:37.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:12:37 smithi132 ceph-mon[35941]: pgmap v5196: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:12:38.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:12:37 smithi165 ceph-mon[36302]: pgmap v5196: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:12:39.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:12:39 smithi119 ceph-mon[31518]: pgmap v5197: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:12:39.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:12:39 smithi132 ceph-mon[35941]: pgmap v5197: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:12:40.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:12:39 smithi165 ceph-mon[36302]: pgmap v5197: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:12:41.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:12:41 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9420.794726 secs 2024-03-28T03:12:41.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:12:41 smithi119 ceph-mon[31518]: pgmap v5198: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:12:41.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:12:41 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9420.794726 secs 2024-03-28T03:12:41.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:12:41 smithi132 ceph-mon[35941]: pgmap v5198: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:12:42.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:12:41 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9420.794726 secs 2024-03-28T03:12:42.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:12:41 smithi165 ceph-mon[36302]: pgmap v5198: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:12:43.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:12:43 smithi119 ceph-mon[31518]: pgmap v5199: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:12:43.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:12:43 smithi132 ceph-mon[35941]: pgmap v5199: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:12:44.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:12:43 smithi165 ceph-mon[36302]: pgmap v5199: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:12:45.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:12:45 smithi119 ceph-mon[31518]: pgmap v5200: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:12:45.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:12:45 smithi132 ceph-mon[35941]: pgmap v5200: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:12:46.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:12:45 smithi165 ceph-mon[36302]: pgmap v5200: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:12:46.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:12:46 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9425.794789 secs 2024-03-28T03:12:47.030 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:12:46 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9425.794789 secs 2024-03-28T03:12:47.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:12:46 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9425.794789 secs 2024-03-28T03:12:47.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:12:47 smithi132 ceph-mon[35941]: pgmap v5201: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:12:48.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:12:47 smithi165 ceph-mon[36302]: pgmap v5201: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:12:48.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:12:47 smithi119 ceph-mon[31518]: pgmap v5201: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:12:49.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:12:49 smithi132 ceph-mon[35941]: pgmap v5202: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:12:50.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:12:49 smithi165 ceph-mon[36302]: pgmap v5202: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:12:50.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:12:49 smithi119 ceph-mon[31518]: pgmap v5202: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:12:51.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:12:51 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9430.794830 secs 2024-03-28T03:12:51.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:12:51 smithi132 ceph-mon[35941]: pgmap v5203: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:12:51.955 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:12:51 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9430.794830 secs 2024-03-28T03:12:51.955 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:12:51 smithi119 ceph-mon[31518]: pgmap v5203: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:12:52.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:12:51 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9430.794830 secs 2024-03-28T03:12:52.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:12:51 smithi165 ceph-mon[36302]: pgmap v5203: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:12:53.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:12:53 smithi132 ceph-mon[35941]: pgmap v5204: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:12:54.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:12:53 smithi165 ceph-mon[36302]: pgmap v5204: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:12:54.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:12:53 smithi119 ceph-mon[31518]: pgmap v5204: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:12:55.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:12:55 smithi132 ceph-mon[35941]: pgmap v5205: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:12:56.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:12:55 smithi165 ceph-mon[36302]: pgmap v5205: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:12:56.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:12:55 smithi119 ceph-mon[31518]: pgmap v5205: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:12:56.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:12:56 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9435.794894 secs 2024-03-28T03:12:57.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:12:56 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9435.794894 secs 2024-03-28T03:12:57.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:12:56 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9435.794894 secs 2024-03-28T03:12:57.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:12:57 smithi165 ceph-mon[36302]: pgmap v5206: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:12:57.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:12:57 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T03:12:57.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:12:57 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:12:57.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:12:57 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:12:57.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:12:57 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:12:57.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:12:57 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:12:57.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:12:57 smithi132 ceph-mon[35941]: pgmap v5206: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:12:57.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:12:57 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T03:12:57.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:12:57 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:12:57.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:12:57 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:12:57.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:12:57 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:12:57.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:12:57 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:12:58.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:12:57 smithi119 ceph-mon[31518]: pgmap v5206: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:12:58.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:12:57 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T03:12:58.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:12:57 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:12:58.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:12:57 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:12:58.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:12:57 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:12:58.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:12:57 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:13:00.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:12:59 smithi165 ceph-mon[36302]: pgmap v5207: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:13:00.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:12:59 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:13:00.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:12:59 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:13:00.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:12:59 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:13:00.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:12:59 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:13:00.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:12:59 smithi119 ceph-mon[31518]: pgmap v5207: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:13:00.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:12:59 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:13:00.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:12:59 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:13:00.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:12:59 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:13:00.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:12:59 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:13:00.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:12:59 smithi132 ceph-mon[35941]: pgmap v5207: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:13:00.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:12:59 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:13:00.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:12:59 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:13:00.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:12:59 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:13:00.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:12:59 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:13:01.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:13:00 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:13:01.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:13:00 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:13:01.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:13:00 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:13:01.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:13:00 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:13:01.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:13:00 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:13:01.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:13:00 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:13:01.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:13:00 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:13:01.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:13:00 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:13:01.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:13:00 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:13:01.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:13:00 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:13:01.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:13:00 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:13:01.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:13:00 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:13:01.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:13:00 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:13:01.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:13:00 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:13:01.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:13:00 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:13:01.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:13:00 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:13:01.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:13:00 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:13:01.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:13:00 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:13:01.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:13:00 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:13:01.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:13:00 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:13:01.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:13:00 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:13:01.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:13:00 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:13:01.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:13:00 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:13:01.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:13:00 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:13:01.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:13:00 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:13:01.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:13:00 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:13:01.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:13:00 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:13:01.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:13:00 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:13:01.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:13:00 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:13:01.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:13:00 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:13:01.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:13:00 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:13:01.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:13:00 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:13:01.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:13:00 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:13:01.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:13:00 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:13:01.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:13:00 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:13:01.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:13:00 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:13:01.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:13:00 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:13:01.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:13:00 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:13:01.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:13:00 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:13:01.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:13:00 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:13:01.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:13:00 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:13:01.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:13:00 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:13:01.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:13:00 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:13:01.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:13:00 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:13:01.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:13:00 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:13:01.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:13:00 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:13:01.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:13:00 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:13:01.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:13:00 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:13:01.988 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:13:01 smithi119 ceph-mon[31518]: Adjusting osd_memory_target on smithi165 to 174.1M 2024-03-28T03:13:01.988 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:13:01 smithi119 ceph-mon[31518]: Unable to set osd_memory_target on smithi165 to 182635622: error parsing value: Value '182635622' is below minimum 939524096 2024-03-28T03:13:01.988 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:13:01 smithi119 ceph-mon[31518]: Adjusting osd_memory_target on smithi132 to 174.1M 2024-03-28T03:13:01.988 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:13:01 smithi119 ceph-mon[31518]: Unable to set osd_memory_target on smithi132 to 182640640: error parsing value: Value '182640640' is below minimum 939524096 2024-03-28T03:13:01.988 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:13:01 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9440.794957 secs 2024-03-28T03:13:01.988 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:13:01 smithi119 ceph-mon[31518]: pgmap v5208: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:13:01.988 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:13:01 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:13:01.988 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:13:01 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:13:02.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:13:01 smithi165 ceph-mon[36302]: Adjusting osd_memory_target on smithi165 to 174.1M 2024-03-28T03:13:02.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:13:01 smithi165 ceph-mon[36302]: Unable to set osd_memory_target on smithi165 to 182635622: error parsing value: Value '182635622' is below minimum 939524096 2024-03-28T03:13:02.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:13:01 smithi165 ceph-mon[36302]: Adjusting osd_memory_target on smithi132 to 174.1M 2024-03-28T03:13:02.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:13:01 smithi165 ceph-mon[36302]: Unable to set osd_memory_target on smithi132 to 182640640: error parsing value: Value '182640640' is below minimum 939524096 2024-03-28T03:13:02.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:13:01 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9440.794957 secs 2024-03-28T03:13:02.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:13:01 smithi165 ceph-mon[36302]: pgmap v5208: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:13:02.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:13:01 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:13:02.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:13:01 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:13:02.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:13:01 smithi132 ceph-mon[35941]: Adjusting osd_memory_target on smithi165 to 174.1M 2024-03-28T03:13:02.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:13:01 smithi132 ceph-mon[35941]: Unable to set osd_memory_target on smithi165 to 182635622: error parsing value: Value '182635622' is below minimum 939524096 2024-03-28T03:13:02.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:13:01 smithi132 ceph-mon[35941]: Adjusting osd_memory_target on smithi132 to 174.1M 2024-03-28T03:13:02.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:13:01 smithi132 ceph-mon[35941]: Unable to set osd_memory_target on smithi132 to 182640640: error parsing value: Value '182640640' is below minimum 939524096 2024-03-28T03:13:02.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:13:01 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9440.794957 secs 2024-03-28T03:13:02.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:13:01 smithi132 ceph-mon[35941]: pgmap v5208: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:13:02.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:13:01 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:13:02.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:13:01 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:13:03.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:13:02 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:13:03.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:13:02 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:13:03.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:13:02 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:13:03.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:13:02 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:13:03.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:13:02 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:13:03.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:13:02 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:13:03.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:13:02 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:13:03.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:13:02 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:13:03.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:13:02 smithi165 ceph-mon[36302]: Adjusting osd_memory_target on smithi119 to 176.7M 2024-03-28T03:13:03.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:13:02 smithi165 ceph-mon[36302]: pgmap v5209: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:13:03.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:13:02 smithi165 ceph-mon[36302]: Unable to set osd_memory_target on smithi119 to 185372364: error parsing value: Value '185372364' is below minimum 939524096 2024-03-28T03:13:03.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:13:02 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T03:13:03.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:13:02 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T03:13:03.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:13:02 smithi165 ceph-mon[36302]: pgmap v5210: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T03:13:03.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:13:02 smithi165 ceph-mon[36302]: pgmap v5211: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 3 op/s 2024-03-28T03:13:03.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:13:02 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:13:03.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:13:02 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:13:03.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:13:02 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:13:03.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:13:02 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:13:03.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:13:02 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:13:03.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:13:02 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:13:03.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:13:02 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:13:03.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:13:02 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:13:03.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:13:02 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:13:03.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:13:02 smithi119 ceph-mon[31518]: Adjusting osd_memory_target on smithi119 to 176.7M 2024-03-28T03:13:03.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:13:02 smithi119 ceph-mon[31518]: pgmap v5209: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:13:03.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:13:02 smithi119 ceph-mon[31518]: Unable to set osd_memory_target on smithi119 to 185372364: error parsing value: Value '185372364' is below minimum 939524096 2024-03-28T03:13:03.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:13:02 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T03:13:03.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:13:02 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T03:13:03.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:13:02 smithi119 ceph-mon[31518]: pgmap v5210: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T03:13:03.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:13:02 smithi119 ceph-mon[31518]: pgmap v5211: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 3 op/s 2024-03-28T03:13:03.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:13:02 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:13:03.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:13:02 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:13:03.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:13:02 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:13:03.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:13:02 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:13:03.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:13:02 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:13:03.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:13:02 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:13:03.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:13:02 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:13:03.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:13:02 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:13:03.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:13:02 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:13:03.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:13:02 smithi132 ceph-mon[35941]: Adjusting osd_memory_target on smithi119 to 176.7M 2024-03-28T03:13:03.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:13:02 smithi132 ceph-mon[35941]: pgmap v5209: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:13:03.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:13:02 smithi132 ceph-mon[35941]: Unable to set osd_memory_target on smithi119 to 185372364: error parsing value: Value '185372364' is below minimum 939524096 2024-03-28T03:13:03.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:13:02 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T03:13:03.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:13:02 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T03:13:03.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:13:02 smithi132 ceph-mon[35941]: pgmap v5210: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T03:13:03.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:13:02 smithi132 ceph-mon[35941]: pgmap v5211: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 3 op/s 2024-03-28T03:13:03.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:13:02 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:13:05.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:13:05 smithi165 ceph-mon[36302]: pgmap v5212: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:13:05.901 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:13:05 smithi132 ceph-mon[35941]: pgmap v5212: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:13:05.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:13:05 smithi119 ceph-mon[31518]: pgmap v5212: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:13:06.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:13:06 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9445.795019 secs 2024-03-28T03:13:06.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:13:06 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9445.795019 secs 2024-03-28T03:13:06.914 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:13:06 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9445.795019 secs 2024-03-28T03:13:07.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:13:07 smithi165 ceph-mon[36302]: pgmap v5213: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:13:07.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:13:07 smithi119 ceph-mon[31518]: pgmap v5213: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:13:07.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:13:07 smithi132 ceph-mon[35941]: pgmap v5213: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:13:09.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:13:09 smithi165 ceph-mon[36302]: pgmap v5214: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 3 op/s 2024-03-28T03:13:09.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:13:09 smithi119 ceph-mon[31518]: pgmap v5214: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 3 op/s 2024-03-28T03:13:09.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:13:09 smithi132 ceph-mon[35941]: pgmap v5214: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 3 op/s 2024-03-28T03:13:11.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:13:11 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9450.795081 secs 2024-03-28T03:13:11.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:13:11 smithi165 ceph-mon[36302]: pgmap v5215: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:13:11.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:13:11 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9450.795081 secs 2024-03-28T03:13:11.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:13:11 smithi119 ceph-mon[31518]: pgmap v5215: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:13:11.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:13:11 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9450.795081 secs 2024-03-28T03:13:11.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:13:11 smithi132 ceph-mon[35941]: pgmap v5215: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:13:13.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:13:13 smithi165 ceph-mon[36302]: pgmap v5216: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T03:13:13.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:13:13 smithi119 ceph-mon[31518]: pgmap v5216: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T03:13:13.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:13:13 smithi132 ceph-mon[35941]: pgmap v5216: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T03:13:15.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:13:15 smithi165 ceph-mon[36302]: pgmap v5217: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:13:15.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:13:15 smithi119 ceph-mon[31518]: pgmap v5217: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:13:15.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:13:15 smithi132 ceph-mon[35941]: pgmap v5217: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:13:16.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:13:16 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9455.795157 secs 2024-03-28T03:13:16.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:13:16 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9455.795157 secs 2024-03-28T03:13:16.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:13:16 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9455.795157 secs 2024-03-28T03:13:17.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:13:17 smithi119 ceph-mon[31518]: pgmap v5218: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:13:17.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:13:17 smithi165 ceph-mon[36302]: pgmap v5218: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:13:17.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:13:17 smithi132 ceph-mon[35941]: pgmap v5218: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:13:19.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:13:19 smithi165 ceph-mon[36302]: pgmap v5219: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:13:19.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:13:19 smithi119 ceph-mon[31518]: pgmap v5219: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:13:19.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:13:19 smithi132 ceph-mon[35941]: pgmap v5219: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:13:21.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:13:21 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9460.795216 secs 2024-03-28T03:13:21.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:13:21 smithi165 ceph-mon[36302]: pgmap v5220: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:13:21.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:13:21 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9460.795216 secs 2024-03-28T03:13:21.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:13:21 smithi119 ceph-mon[31518]: pgmap v5220: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:13:21.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:13:21 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9460.795216 secs 2024-03-28T03:13:21.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:13:21 smithi132 ceph-mon[35941]: pgmap v5220: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:13:23.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:13:23 smithi165 ceph-mon[36302]: pgmap v5221: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:13:23.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:13:23 smithi119 ceph-mon[31518]: pgmap v5221: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:13:23.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:13:23 smithi132 ceph-mon[35941]: pgmap v5221: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:13:25.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:13:25 smithi165 ceph-mon[36302]: pgmap v5222: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:13:25.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:13:25 smithi119 ceph-mon[31518]: pgmap v5222: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:13:25.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:13:25 smithi132 ceph-mon[35941]: pgmap v5222: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:13:26.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:13:26 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9465.795278 secs 2024-03-28T03:13:26.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:13:26 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9465.795278 secs 2024-03-28T03:13:26.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:13:26 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9465.795278 secs 2024-03-28T03:13:27.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:13:27 smithi165 ceph-mon[36302]: pgmap v5223: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:13:27.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:13:27 smithi119 ceph-mon[31518]: pgmap v5223: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:13:27.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:13:27 smithi132 ceph-mon[35941]: pgmap v5223: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:13:29.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:13:29 smithi165 ceph-mon[36302]: pgmap v5224: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:13:29.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:13:29 smithi119 ceph-mon[31518]: pgmap v5224: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:13:29.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:13:29 smithi132 ceph-mon[35941]: pgmap v5224: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:13:31.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:13:31 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9470.795338 secs 2024-03-28T03:13:31.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:13:31 smithi165 ceph-mon[36302]: pgmap v5225: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:13:31.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:13:31 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9470.795338 secs 2024-03-28T03:13:31.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:13:31 smithi119 ceph-mon[31518]: pgmap v5225: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:13:31.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:13:31 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9470.795338 secs 2024-03-28T03:13:31.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:13:31 smithi132 ceph-mon[35941]: pgmap v5225: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:13:33.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:13:33 smithi165 ceph-mon[36302]: pgmap v5226: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:13:33.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:13:33 smithi119 ceph-mon[31518]: pgmap v5226: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:13:33.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:13:33 smithi132 ceph-mon[35941]: pgmap v5226: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:13:35.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:13:35 smithi165 ceph-mon[36302]: pgmap v5227: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:13:35.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:13:35 smithi119 ceph-mon[31518]: pgmap v5227: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:13:35.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:13:35 smithi132 ceph-mon[35941]: pgmap v5227: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:13:36.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:13:36 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9475.795425 secs 2024-03-28T03:13:36.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:13:36 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9475.795425 secs 2024-03-28T03:13:37.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:13:36 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9475.795425 secs 2024-03-28T03:13:37.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:13:37 smithi119 ceph-mon[31518]: pgmap v5228: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:13:37.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:13:37 smithi132 ceph-mon[35941]: pgmap v5228: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:13:38.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:13:37 smithi165 ceph-mon[36302]: pgmap v5228: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:13:39.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:13:39 smithi119 ceph-mon[31518]: pgmap v5229: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:13:39.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:13:39 smithi132 ceph-mon[35941]: pgmap v5229: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:13:40.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:13:39 smithi165 ceph-mon[36302]: pgmap v5229: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:13:41.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:13:41 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9480.795489 secs 2024-03-28T03:13:41.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:13:41 smithi119 ceph-mon[31518]: pgmap v5230: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:13:41.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:13:41 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9480.795489 secs 2024-03-28T03:13:41.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:13:41 smithi132 ceph-mon[35941]: pgmap v5230: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:13:42.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:13:41 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9480.795489 secs 2024-03-28T03:13:42.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:13:41 smithi165 ceph-mon[36302]: pgmap v5230: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:13:43.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:13:43 smithi132 ceph-mon[35941]: pgmap v5231: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:13:44.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:13:43 smithi165 ceph-mon[36302]: pgmap v5231: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:13:44.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:13:43 smithi119 ceph-mon[31518]: pgmap v5231: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:13:45.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:13:45 smithi132 ceph-mon[35941]: pgmap v5232: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:13:46.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:13:45 smithi165 ceph-mon[36302]: pgmap v5232: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:13:46.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:13:45 smithi119 ceph-mon[31518]: pgmap v5232: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:13:46.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:13:46 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9485.795575 secs 2024-03-28T03:13:47.030 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:13:46 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9485.795575 secs 2024-03-28T03:13:47.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:13:46 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9485.795575 secs 2024-03-28T03:13:47.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:13:47 smithi132 ceph-mon[35941]: pgmap v5233: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:13:48.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:13:47 smithi165 ceph-mon[36302]: pgmap v5233: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:13:48.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:13:47 smithi119 ceph-mon[31518]: pgmap v5233: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:13:49.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:13:49 smithi132 ceph-mon[35941]: pgmap v5234: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:13:50.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:13:49 smithi165 ceph-mon[36302]: pgmap v5234: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:13:50.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:13:49 smithi119 ceph-mon[31518]: pgmap v5234: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:13:51.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:13:51 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9490.795635 secs 2024-03-28T03:13:51.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:13:51 smithi132 ceph-mon[35941]: pgmap v5235: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:13:51.955 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:13:51 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9490.795635 secs 2024-03-28T03:13:51.955 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:13:51 smithi119 ceph-mon[31518]: pgmap v5235: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:13:52.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:13:51 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9490.795635 secs 2024-03-28T03:13:52.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:13:51 smithi165 ceph-mon[36302]: pgmap v5235: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:13:53.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:13:53 smithi132 ceph-mon[35941]: pgmap v5236: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:13:54.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:13:53 smithi165 ceph-mon[36302]: pgmap v5236: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:13:54.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:13:53 smithi119 ceph-mon[31518]: pgmap v5236: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:13:55.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:13:55 smithi132 ceph-mon[35941]: pgmap v5237: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:13:56.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:13:55 smithi165 ceph-mon[36302]: pgmap v5237: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:13:56.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:13:55 smithi119 ceph-mon[31518]: pgmap v5237: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:13:56.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:13:56 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9495.795673 secs 2024-03-28T03:13:57.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:13:56 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9495.795673 secs 2024-03-28T03:13:57.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:13:56 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9495.795673 secs 2024-03-28T03:13:58.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:13:57 smithi165 ceph-mon[36302]: pgmap v5238: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:13:58.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:13:57 smithi119 ceph-mon[31518]: pgmap v5238: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:13:58.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:13:57 smithi132 ceph-mon[35941]: pgmap v5238: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:14:00.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:13:59 smithi165 ceph-mon[36302]: pgmap v5239: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:14:00.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:13:59 smithi119 ceph-mon[31518]: pgmap v5239: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:14:00.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:13:59 smithi132 ceph-mon[35941]: pgmap v5239: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:14:02.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:14:01 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9500.795757 secs 2024-03-28T03:14:02.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:14:01 smithi165 ceph-mon[36302]: pgmap v5240: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:14:02.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:14:01 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9500.795757 secs 2024-03-28T03:14:02.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:14:01 smithi119 ceph-mon[31518]: pgmap v5240: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:14:02.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:14:01 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9500.795757 secs 2024-03-28T03:14:02.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:14:01 smithi132 ceph-mon[35941]: pgmap v5240: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:14:04.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:14:03 smithi165 ceph-mon[36302]: pgmap v5241: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:14:04.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:14:03 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T03:14:04.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:14:03 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:14:04.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:14:03 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:14:04.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:14:03 smithi119 ceph-mon[31518]: pgmap v5241: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:14:04.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:14:03 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T03:14:04.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:14:03 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:14:04.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:14:03 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:14:04.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:14:03 smithi132 ceph-mon[35941]: pgmap v5241: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:14:04.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:14:03 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T03:14:04.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:14:03 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:14:04.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:14:03 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:14:05.101 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:14:04 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T03:14:05.101 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:14:04 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T03:14:05.101 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:14:04 smithi132 ceph-mon[35941]: pgmap v5242: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T03:14:05.101 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:14:04 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:14:05.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:14:04 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T03:14:05.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:14:04 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T03:14:05.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:14:04 smithi165 ceph-mon[36302]: pgmap v5242: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T03:14:05.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:14:04 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:14:05.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:14:04 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T03:14:05.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:14:04 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T03:14:05.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:14:04 smithi119 ceph-mon[31518]: pgmap v5242: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T03:14:05.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:14:04 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:14:06.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:14:05 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9505.795818 secs 2024-03-28T03:14:06.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:14:05 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9505.795818 secs 2024-03-28T03:14:06.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:14:05 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9505.795818 secs 2024-03-28T03:14:07.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:14:06 smithi165 ceph-mon[36302]: pgmap v5243: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T03:14:07.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:14:06 smithi119 ceph-mon[31518]: pgmap v5243: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T03:14:07.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:14:06 smithi132 ceph-mon[35941]: pgmap v5243: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T03:14:09.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:14:09 smithi132 ceph-mon[35941]: pgmap v5244: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T03:14:09.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:14:09 smithi165 ceph-mon[36302]: pgmap v5244: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T03:14:09.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:14:09 smithi119 ceph-mon[31518]: pgmap v5244: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T03:14:11.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:14:11 smithi132 ceph-mon[35941]: pgmap v5245: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T03:14:11.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:14:11 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9510.795877 secs 2024-03-28T03:14:11.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:14:11 smithi165 ceph-mon[36302]: pgmap v5245: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T03:14:11.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:14:11 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9510.795877 secs 2024-03-28T03:14:11.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:14:11 smithi119 ceph-mon[31518]: pgmap v5245: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T03:14:11.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:14:11 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9510.795877 secs 2024-03-28T03:14:13.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:14:13 smithi132 ceph-mon[35941]: pgmap v5246: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T03:14:13.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:14:13 smithi165 ceph-mon[36302]: pgmap v5246: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T03:14:13.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:14:13 smithi119 ceph-mon[31518]: pgmap v5246: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T03:14:15.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:14:15 smithi132 ceph-mon[35941]: pgmap v5247: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T03:14:15.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:14:15 smithi165 ceph-mon[36302]: pgmap v5247: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T03:14:15.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:14:15 smithi119 ceph-mon[31518]: pgmap v5247: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T03:14:16.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:14:16 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9515.795900 secs 2024-03-28T03:14:16.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:14:16 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9515.795900 secs 2024-03-28T03:14:16.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:14:16 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9515.795900 secs 2024-03-28T03:14:17.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:14:17 smithi132 ceph-mon[35941]: pgmap v5248: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T03:14:17.530 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:14:17 smithi165 ceph-mon[36302]: pgmap v5248: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T03:14:17.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:14:17 smithi119 ceph-mon[31518]: pgmap v5248: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T03:14:19.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:14:19 smithi132 ceph-mon[35941]: pgmap v5249: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:14:19.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:14:19 smithi165 ceph-mon[36302]: pgmap v5249: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:14:19.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:14:19 smithi119 ceph-mon[31518]: pgmap v5249: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:14:21.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:14:21 smithi132 ceph-mon[35941]: pgmap v5250: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:14:21.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:14:21 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9520.795959 secs 2024-03-28T03:14:21.558 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:14:21 smithi119 ceph-mon[31518]: pgmap v5250: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:14:21.558 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:14:21 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9520.795959 secs 2024-03-28T03:14:21.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:14:21 smithi165 ceph-mon[36302]: pgmap v5250: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:14:21.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:14:21 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9520.795959 secs 2024-03-28T03:14:23.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:14:23 smithi132 ceph-mon[35941]: pgmap v5251: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:14:23.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:14:23 smithi165 ceph-mon[36302]: pgmap v5251: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:14:23.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:14:23 smithi119 ceph-mon[31518]: pgmap v5251: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:14:25.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:14:25 smithi165 ceph-mon[36302]: pgmap v5252: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 6 op/s 2024-03-28T03:14:25.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:14:25 smithi119 ceph-mon[31518]: pgmap v5252: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 6 op/s 2024-03-28T03:14:25.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:14:25 smithi132 ceph-mon[35941]: pgmap v5252: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 6 op/s 2024-03-28T03:14:26.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:14:26 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9525.796046 secs 2024-03-28T03:14:26.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:14:26 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9525.796046 secs 2024-03-28T03:14:26.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:14:26 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9525.796046 secs 2024-03-28T03:14:27.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:14:27 smithi165 ceph-mon[36302]: pgmap v5253: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:14:27.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:14:27 smithi119 ceph-mon[31518]: pgmap v5253: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:14:27.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:14:27 smithi132 ceph-mon[35941]: pgmap v5253: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:14:29.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:14:29 smithi165 ceph-mon[36302]: pgmap v5254: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T03:14:29.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:14:29 smithi119 ceph-mon[31518]: pgmap v5254: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T03:14:29.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:14:29 smithi132 ceph-mon[35941]: pgmap v5254: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T03:14:31.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:14:31 smithi165 ceph-mon[36302]: pgmap v5255: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:14:31.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:14:31 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9530.796132 secs 2024-03-28T03:14:31.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:14:31 smithi119 ceph-mon[31518]: pgmap v5255: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:14:31.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:14:31 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9530.796132 secs 2024-03-28T03:14:31.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:14:31 smithi132 ceph-mon[35941]: pgmap v5255: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:14:31.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:14:31 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9530.796132 secs 2024-03-28T03:14:33.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:14:33 smithi165 ceph-mon[36302]: pgmap v5256: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T03:14:33.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:14:33 smithi119 ceph-mon[31518]: pgmap v5256: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T03:14:33.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:14:33 smithi132 ceph-mon[35941]: pgmap v5256: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T03:14:35.502 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:14:35 smithi132 ceph-mon[35941]: pgmap v5257: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 6 op/s 2024-03-28T03:14:35.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:14:35 smithi165 ceph-mon[36302]: pgmap v5257: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 6 op/s 2024-03-28T03:14:35.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:14:35 smithi119 ceph-mon[31518]: pgmap v5257: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 6 op/s 2024-03-28T03:14:36.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:14:36 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9535.796221 secs 2024-03-28T03:14:36.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:14:36 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9535.796221 secs 2024-03-28T03:14:36.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:14:36 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9535.796221 secs 2024-03-28T03:14:37.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:14:37 smithi165 ceph-mon[36302]: pgmap v5258: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T03:14:37.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:14:37 smithi119 ceph-mon[31518]: pgmap v5258: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T03:14:37.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:14:37 smithi132 ceph-mon[35941]: pgmap v5258: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T03:14:39.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:14:39 smithi165 ceph-mon[36302]: pgmap v5259: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:14:39.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:14:39 smithi119 ceph-mon[31518]: pgmap v5259: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:14:39.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:14:39 smithi132 ceph-mon[35941]: pgmap v5259: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:14:41.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:14:41 smithi165 ceph-mon[36302]: pgmap v5260: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:14:41.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:14:41 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9540.796310 secs 2024-03-28T03:14:41.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:14:41 smithi119 ceph-mon[31518]: pgmap v5260: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:14:41.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:14:41 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9540.796310 secs 2024-03-28T03:14:41.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:14:41 smithi132 ceph-mon[35941]: pgmap v5260: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:14:41.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:14:41 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9540.796310 secs 2024-03-28T03:14:43.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:14:43 smithi165 ceph-mon[36302]: pgmap v5261: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:14:43.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:14:43 smithi119 ceph-mon[31518]: pgmap v5261: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:14:43.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:14:43 smithi132 ceph-mon[35941]: pgmap v5261: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:14:45.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:14:45 smithi165 ceph-mon[36302]: pgmap v5262: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-28T03:14:45.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:14:45 smithi119 ceph-mon[31518]: pgmap v5262: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-28T03:14:45.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:14:45 smithi132 ceph-mon[35941]: pgmap v5262: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-28T03:14:46.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:14:46 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9545.796328 secs 2024-03-28T03:14:46.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:14:46 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9545.796328 secs 2024-03-28T03:14:46.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:14:46 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9545.796328 secs 2024-03-28T03:14:47.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:14:47 smithi165 ceph-mon[36302]: pgmap v5263: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T03:14:47.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:14:47 smithi119 ceph-mon[31518]: pgmap v5263: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T03:14:47.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:14:47 smithi132 ceph-mon[35941]: pgmap v5263: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T03:14:49.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:14:49 smithi165 ceph-mon[36302]: pgmap v5264: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:14:49.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:14:49 smithi119 ceph-mon[31518]: pgmap v5264: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:14:49.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:14:49 smithi132 ceph-mon[35941]: pgmap v5264: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:14:51.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:14:51 smithi165 ceph-mon[36302]: pgmap v5265: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:14:51.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:14:51 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9550.796413 secs 2024-03-28T03:14:51.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:14:51 smithi119 ceph-mon[31518]: pgmap v5265: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:14:51.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:14:51 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9550.796413 secs 2024-03-28T03:14:51.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:14:51 smithi132 ceph-mon[35941]: pgmap v5265: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:14:51.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:14:51 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9550.796413 secs 2024-03-28T03:14:53.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:14:53 smithi165 ceph-mon[36302]: pgmap v5266: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:14:53.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:14:53 smithi119 ceph-mon[31518]: pgmap v5266: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:14:53.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:14:53 smithi132 ceph-mon[35941]: pgmap v5266: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:14:55.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:14:55 smithi165 ceph-mon[36302]: pgmap v5267: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-28T03:14:55.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:14:55 smithi119 ceph-mon[31518]: pgmap v5267: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-28T03:14:55.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:14:55 smithi132 ceph-mon[35941]: pgmap v5267: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-28T03:14:56.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:14:56 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9555.796503 secs 2024-03-28T03:14:56.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:14:56 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9555.796503 secs 2024-03-28T03:14:56.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:14:56 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9555.796503 secs 2024-03-28T03:14:57.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:14:57 smithi165 ceph-mon[36302]: pgmap v5268: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T03:14:57.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:14:57 smithi119 ceph-mon[31518]: pgmap v5268: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T03:14:57.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:14:57 smithi132 ceph-mon[35941]: pgmap v5268: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T03:14:59.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:14:59 smithi165 ceph-mon[36302]: pgmap v5269: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:14:59.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:14:59 smithi119 ceph-mon[31518]: pgmap v5269: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:14:59.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:14:59 smithi132 ceph-mon[35941]: pgmap v5269: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:15:01.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:15:01 smithi165 ceph-mon[36302]: pgmap v5270: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:15:01.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:15:01 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9560.796565 secs 2024-03-28T03:15:01.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:15:01 smithi119 ceph-mon[31518]: pgmap v5270: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:15:01.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:15:01 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9560.796565 secs 2024-03-28T03:15:01.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:15:01 smithi132 ceph-mon[35941]: pgmap v5270: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:15:01.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:15:01 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9560.796565 secs 2024-03-28T03:15:03.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:15:03 smithi165 ceph-mon[36302]: pgmap v5271: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:15:03.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:15:03 smithi119 ceph-mon[31518]: pgmap v5271: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:15:03.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:15:03 smithi132 ceph-mon[35941]: pgmap v5271: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:15:05.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:15:05 smithi165 ceph-mon[36302]: pgmap v5272: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-28T03:15:05.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:15:05 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T03:15:05.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:15:05 smithi119 ceph-mon[31518]: pgmap v5272: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-28T03:15:05.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:15:05 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T03:15:05.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:15:05 smithi132 ceph-mon[35941]: pgmap v5272: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-28T03:15:05.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:15:05 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T03:15:06.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:15:06 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9565.796653 secs 2024-03-28T03:15:06.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:15:06 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T03:15:06.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:15:06 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T03:15:06.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:15:06 smithi165 ceph-mon[36302]: pgmap v5273: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-03-28T03:15:06.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:15:06 smithi165 ceph-mon[36302]: pgmap v5274: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:15:06.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:15:06 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:15:06.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:15:06 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9565.796653 secs 2024-03-28T03:15:06.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:15:06 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T03:15:06.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:15:06 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T03:15:06.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:15:06 smithi119 ceph-mon[31518]: pgmap v5273: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-03-28T03:15:06.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:15:06 smithi119 ceph-mon[31518]: pgmap v5274: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:15:06.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:15:06 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:15:06.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:15:06 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9565.796653 secs 2024-03-28T03:15:06.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:15:06 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T03:15:06.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:15:06 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T03:15:06.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:15:06 smithi132 ceph-mon[35941]: pgmap v5273: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-03-28T03:15:06.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:15:06 smithi132 ceph-mon[35941]: pgmap v5274: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:15:06.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:15:06 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:15:09.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:15:08 smithi165 ceph-mon[36302]: pgmap v5275: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:15:09.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:15:08 smithi119 ceph-mon[31518]: pgmap v5275: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:15:09.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:15:08 smithi132 ceph-mon[35941]: pgmap v5275: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:15:10.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:15:09 smithi165 ceph-mon[36302]: pgmap v5276: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:15:10.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:15:09 smithi119 ceph-mon[31518]: pgmap v5276: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:15:10.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:15:09 smithi132 ceph-mon[35941]: pgmap v5276: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:15:11.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:15:10 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9570.796693 secs 2024-03-28T03:15:11.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:15:10 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9570.796693 secs 2024-03-28T03:15:11.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:15:10 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9570.796693 secs 2024-03-28T03:15:12.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:15:11 smithi165 ceph-mon[36302]: pgmap v5277: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:15:12.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:15:11 smithi119 ceph-mon[31518]: pgmap v5277: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:15:12.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:15:11 smithi132 ceph-mon[35941]: pgmap v5277: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:15:14.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:15:13 smithi165 ceph-mon[36302]: pgmap v5278: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 4 op/s 2024-03-28T03:15:14.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:15:13 smithi119 ceph-mon[31518]: pgmap v5278: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 4 op/s 2024-03-28T03:15:14.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:15:13 smithi132 ceph-mon[35941]: pgmap v5278: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 4 op/s 2024-03-28T03:15:16.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:15:15 smithi165 ceph-mon[36302]: pgmap v5279: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T03:15:16.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:15:15 smithi119 ceph-mon[31518]: pgmap v5279: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T03:15:16.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:15:15 smithi132 ceph-mon[35941]: pgmap v5279: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T03:15:17.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:15:16 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9575.796754 secs 2024-03-28T03:15:17.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:15:16 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9575.796754 secs 2024-03-28T03:15:17.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:15:16 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9575.796754 secs 2024-03-28T03:15:18.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:15:17 smithi165 ceph-mon[36302]: pgmap v5280: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:15:18.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:15:17 smithi119 ceph-mon[31518]: pgmap v5280: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:15:18.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:15:17 smithi132 ceph-mon[35941]: pgmap v5280: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:15:20.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:15:19 smithi165 ceph-mon[36302]: pgmap v5281: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:15:20.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:15:19 smithi119 ceph-mon[31518]: pgmap v5281: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:15:20.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:15:19 smithi132 ceph-mon[35941]: pgmap v5281: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:15:21.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:15:20 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9580.796845 secs 2024-03-28T03:15:21.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:15:20 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9580.796845 secs 2024-03-28T03:15:21.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:15:20 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9580.796845 secs 2024-03-28T03:15:22.275 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:15:21 smithi119 ceph-mon[31518]: pgmap v5282: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:15:22.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:15:21 smithi165 ceph-mon[36302]: pgmap v5282: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:15:22.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:15:21 smithi132 ceph-mon[35941]: pgmap v5282: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:15:24.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:15:23 smithi165 ceph-mon[36302]: pgmap v5283: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:15:24.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:15:23 smithi119 ceph-mon[31518]: pgmap v5283: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:15:24.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:15:23 smithi132 ceph-mon[35941]: pgmap v5283: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:15:26.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:15:25 smithi165 ceph-mon[36302]: pgmap v5284: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:15:26.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:15:25 smithi119 ceph-mon[31518]: pgmap v5284: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:15:26.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:15:25 smithi132 ceph-mon[35941]: pgmap v5284: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:15:27.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:15:26 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9585.796884 secs 2024-03-28T03:15:27.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:15:26 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9585.796884 secs 2024-03-28T03:15:27.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:15:26 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9585.796884 secs 2024-03-28T03:15:28.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:15:27 smithi165 ceph-mon[36302]: pgmap v5285: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:15:28.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:15:27 smithi119 ceph-mon[31518]: pgmap v5285: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:15:28.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:15:27 smithi132 ceph-mon[35941]: pgmap v5285: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:15:30.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:15:29 smithi165 ceph-mon[36302]: pgmap v5286: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:15:30.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:15:29 smithi119 ceph-mon[31518]: pgmap v5286: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:15:30.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:15:29 smithi132 ceph-mon[35941]: pgmap v5286: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:15:31.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:15:30 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9590.796947 secs 2024-03-28T03:15:31.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:15:30 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9590.796947 secs 2024-03-28T03:15:31.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:15:30 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9590.796947 secs 2024-03-28T03:15:32.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:15:31 smithi165 ceph-mon[36302]: pgmap v5287: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:15:32.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:15:31 smithi119 ceph-mon[31518]: pgmap v5287: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:15:32.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:15:31 smithi132 ceph-mon[35941]: pgmap v5287: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:15:34.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:15:33 smithi165 ceph-mon[36302]: pgmap v5288: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:15:34.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:15:33 smithi119 ceph-mon[31518]: pgmap v5288: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:15:34.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:15:33 smithi132 ceph-mon[35941]: pgmap v5288: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:15:36.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:15:35 smithi165 ceph-mon[36302]: pgmap v5289: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:15:36.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:15:35 smithi119 ceph-mon[31518]: pgmap v5289: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:15:36.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:15:35 smithi132 ceph-mon[35941]: pgmap v5289: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:15:37.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:15:36 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9595.796981 secs 2024-03-28T03:15:37.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:15:36 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9595.796981 secs 2024-03-28T03:15:37.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:15:36 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9595.796981 secs 2024-03-28T03:15:38.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:15:37 smithi165 ceph-mon[36302]: pgmap v5290: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:15:38.146 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:15:37 smithi119 ceph-mon[31518]: pgmap v5290: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:15:38.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:15:37 smithi132 ceph-mon[35941]: pgmap v5290: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:15:40.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:15:39 smithi165 ceph-mon[36302]: pgmap v5291: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:15:40.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:15:39 smithi119 ceph-mon[31518]: pgmap v5291: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:15:40.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:15:39 smithi132 ceph-mon[35941]: pgmap v5291: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:15:41.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:15:40 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9600.797066 secs 2024-03-28T03:15:41.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:15:40 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9600.797066 secs 2024-03-28T03:15:41.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:15:40 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9600.797066 secs 2024-03-28T03:15:42.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:15:41 smithi165 ceph-mon[36302]: pgmap v5292: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:15:42.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:15:41 smithi119 ceph-mon[31518]: pgmap v5292: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:15:42.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:15:41 smithi132 ceph-mon[35941]: pgmap v5292: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:15:44.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:15:43 smithi165 ceph-mon[36302]: pgmap v5293: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:15:44.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:15:43 smithi119 ceph-mon[31518]: pgmap v5293: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:15:44.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:15:43 smithi132 ceph-mon[35941]: pgmap v5293: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:15:46.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:15:45 smithi165 ceph-mon[36302]: pgmap v5294: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:15:46.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:15:45 smithi119 ceph-mon[31518]: pgmap v5294: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:15:46.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:15:45 smithi132 ceph-mon[35941]: pgmap v5294: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:15:47.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:15:46 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9605.797150 secs 2024-03-28T03:15:47.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:15:46 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9605.797150 secs 2024-03-28T03:15:47.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:15:46 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9605.797150 secs 2024-03-28T03:15:48.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:15:47 smithi165 ceph-mon[36302]: pgmap v5295: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:15:48.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:15:47 smithi119 ceph-mon[31518]: pgmap v5295: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:15:48.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:15:47 smithi132 ceph-mon[35941]: pgmap v5295: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:15:50.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:15:49 smithi165 ceph-mon[36302]: pgmap v5296: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:15:50.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:15:49 smithi119 ceph-mon[31518]: pgmap v5296: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:15:50.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:15:49 smithi132 ceph-mon[35941]: pgmap v5296: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:15:51.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:15:50 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9610.797211 secs 2024-03-28T03:15:51.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:15:50 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9610.797211 secs 2024-03-28T03:15:51.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:15:50 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9610.797211 secs 2024-03-28T03:15:52.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:15:51 smithi165 ceph-mon[36302]: pgmap v5297: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:15:52.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:15:51 smithi119 ceph-mon[31518]: pgmap v5297: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:15:52.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:15:51 smithi132 ceph-mon[35941]: pgmap v5297: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:15:54.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:15:53 smithi165 ceph-mon[36302]: pgmap v5298: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:15:54.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:15:53 smithi119 ceph-mon[31518]: pgmap v5298: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:15:54.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:15:53 smithi132 ceph-mon[35941]: pgmap v5298: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:15:56.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:15:55 smithi165 ceph-mon[36302]: pgmap v5299: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:15:56.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:15:55 smithi119 ceph-mon[31518]: pgmap v5299: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:15:56.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:15:55 smithi132 ceph-mon[35941]: pgmap v5299: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:15:57.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:15:56 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9615.797270 secs 2024-03-28T03:15:57.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:15:56 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9615.797270 secs 2024-03-28T03:15:57.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:15:56 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9615.797270 secs 2024-03-28T03:15:58.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:15:57 smithi165 ceph-mon[36302]: pgmap v5300: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:15:58.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:15:57 smithi119 ceph-mon[31518]: pgmap v5300: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:15:58.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:15:57 smithi132 ceph-mon[35941]: pgmap v5300: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:16:00.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:15:59 smithi165 ceph-mon[36302]: pgmap v5301: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:16:00.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:15:59 smithi119 ceph-mon[31518]: pgmap v5301: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:16:00.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:15:59 smithi132 ceph-mon[35941]: pgmap v5301: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:16:01.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:16:00 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9620.797357 secs 2024-03-28T03:16:01.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:16:00 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9620.797357 secs 2024-03-28T03:16:01.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:16:00 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9620.797357 secs 2024-03-28T03:16:02.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:16:02 smithi165 ceph-mon[36302]: pgmap v5302: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:16:02.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:16:02 smithi119 ceph-mon[31518]: pgmap v5302: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:16:02.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:16:02 smithi132 ceph-mon[35941]: pgmap v5302: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:16:04.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:16:03 smithi165 ceph-mon[36302]: pgmap v5303: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:16:04.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:16:03 smithi119 ceph-mon[31518]: pgmap v5303: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:16:04.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:16:03 smithi132 ceph-mon[35941]: pgmap v5303: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:16:06.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:16:05 smithi165 ceph-mon[36302]: pgmap v5304: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:16:06.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:16:05 smithi119 ceph-mon[31518]: pgmap v5304: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:16:06.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:16:05 smithi132 ceph-mon[35941]: pgmap v5304: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:16:07.101 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:16:06 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9625.797442 secs 2024-03-28T03:16:07.101 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:16:06 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T03:16:07.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:16:06 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9625.797442 secs 2024-03-28T03:16:07.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:16:06 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T03:16:07.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:16:06 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9625.797442 secs 2024-03-28T03:16:07.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:16:06 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T03:16:08.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:16:07 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T03:16:08.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:16:07 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T03:16:08.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:16:07 smithi165 ceph-mon[36302]: pgmap v5305: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T03:16:08.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:16:07 smithi165 ceph-mon[36302]: pgmap v5306: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:16:08.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:16:07 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:16:08.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:16:07 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T03:16:08.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:16:07 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T03:16:08.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:16:07 smithi119 ceph-mon[31518]: pgmap v5305: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T03:16:08.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:16:07 smithi119 ceph-mon[31518]: pgmap v5306: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:16:08.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:16:07 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:16:08.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:16:07 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T03:16:08.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:16:07 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T03:16:08.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:16:07 smithi132 ceph-mon[35941]: pgmap v5305: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T03:16:08.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:16:07 smithi132 ceph-mon[35941]: pgmap v5306: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:16:08.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:16:07 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:16:10.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:16:10 smithi165 ceph-mon[36302]: pgmap v5307: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:16:10.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:16:10 smithi119 ceph-mon[31518]: pgmap v5307: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:16:10.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:16:10 smithi132 ceph-mon[35941]: pgmap v5307: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:16:11.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:16:11 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9630.797532 secs 2024-03-28T03:16:11.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:16:11 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9630.797532 secs 2024-03-28T03:16:11.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:16:11 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9630.797532 secs 2024-03-28T03:16:12.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:16:12 smithi165 ceph-mon[36302]: pgmap v5308: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:16:12.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:16:12 smithi119 ceph-mon[31518]: pgmap v5308: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:16:12.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:16:12 smithi132 ceph-mon[35941]: pgmap v5308: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:16:14.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:16:14 smithi165 ceph-mon[36302]: pgmap v5309: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:16:14.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:16:14 smithi119 ceph-mon[31518]: pgmap v5309: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:16:14.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:16:14 smithi132 ceph-mon[35941]: pgmap v5309: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:16:16.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:16:16 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9635.797620 secs 2024-03-28T03:16:16.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:16:16 smithi165 ceph-mon[36302]: pgmap v5310: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:16:16.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:16:16 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9635.797620 secs 2024-03-28T03:16:16.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:16:16 smithi119 ceph-mon[31518]: pgmap v5310: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:16:16.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:16:16 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9635.797620 secs 2024-03-28T03:16:16.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:16:16 smithi132 ceph-mon[35941]: pgmap v5310: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:16:18.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:16:18 smithi165 ceph-mon[36302]: pgmap v5311: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T03:16:18.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:16:18 smithi119 ceph-mon[31518]: pgmap v5311: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T03:16:18.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:16:18 smithi132 ceph-mon[35941]: pgmap v5311: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T03:16:20.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:16:20 smithi165 ceph-mon[36302]: pgmap v5312: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:16:20.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:16:20 smithi119 ceph-mon[31518]: pgmap v5312: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:16:20.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:16:20 smithi132 ceph-mon[35941]: pgmap v5312: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:16:21.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:16:21 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9640.797709 secs 2024-03-28T03:16:21.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:16:21 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9640.797709 secs 2024-03-28T03:16:21.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:16:21 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9640.797709 secs 2024-03-28T03:16:22.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:16:22 smithi165 ceph-mon[36302]: pgmap v5313: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:16:22.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:16:22 smithi119 ceph-mon[31518]: pgmap v5313: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:16:22.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:16:22 smithi132 ceph-mon[35941]: pgmap v5313: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:16:24.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:16:24 smithi165 ceph-mon[36302]: pgmap v5314: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:16:24.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:16:24 smithi119 ceph-mon[31518]: pgmap v5314: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:16:24.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:16:24 smithi132 ceph-mon[35941]: pgmap v5314: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:16:26.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:16:26 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9645.797750 secs 2024-03-28T03:16:26.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:16:26 smithi165 ceph-mon[36302]: pgmap v5315: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:16:26.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:16:26 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9645.797750 secs 2024-03-28T03:16:26.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:16:26 smithi119 ceph-mon[31518]: pgmap v5315: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:16:26.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:16:26 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9645.797750 secs 2024-03-28T03:16:26.702 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:16:26 smithi132 ceph-mon[35941]: pgmap v5315: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:16:28.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:16:28 smithi165 ceph-mon[36302]: pgmap v5316: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:16:28.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:16:28 smithi119 ceph-mon[31518]: pgmap v5316: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:16:28.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:16:28 smithi132 ceph-mon[35941]: pgmap v5316: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:16:30.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:16:30 smithi165 ceph-mon[36302]: pgmap v5317: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:16:30.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:16:30 smithi119 ceph-mon[31518]: pgmap v5317: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:16:30.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:16:30 smithi132 ceph-mon[35941]: pgmap v5317: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:16:31.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:16:31 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9650.797839 secs 2024-03-28T03:16:31.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:16:31 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9650.797839 secs 2024-03-28T03:16:31.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:16:31 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9650.797839 secs 2024-03-28T03:16:32.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:16:32 smithi165 ceph-mon[36302]: pgmap v5318: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:16:32.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:16:32 smithi119 ceph-mon[31518]: pgmap v5318: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:16:32.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:16:32 smithi132 ceph-mon[35941]: pgmap v5318: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:16:34.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:16:34 smithi165 ceph-mon[36302]: pgmap v5319: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:16:34.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:16:34 smithi119 ceph-mon[31518]: pgmap v5319: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:16:34.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:16:34 smithi132 ceph-mon[35941]: pgmap v5319: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:16:36.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:16:36 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9655.797925 secs 2024-03-28T03:16:36.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:16:36 smithi119 ceph-mon[31518]: pgmap v5320: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:16:36.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:16:36 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9655.797925 secs 2024-03-28T03:16:36.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:16:36 smithi132 ceph-mon[35941]: pgmap v5320: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:16:36.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:16:36 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9655.797925 secs 2024-03-28T03:16:36.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:16:36 smithi165 ceph-mon[36302]: pgmap v5320: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:16:38.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:16:38 smithi119 ceph-mon[31518]: pgmap v5321: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:16:38.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:16:38 smithi132 ceph-mon[35941]: pgmap v5321: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:16:38.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:16:38 smithi165 ceph-mon[36302]: pgmap v5321: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:16:40.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:16:40 smithi119 ceph-mon[31518]: pgmap v5322: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:16:40.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:16:40 smithi132 ceph-mon[35941]: pgmap v5322: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:16:40.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:16:40 smithi165 ceph-mon[36302]: pgmap v5322: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:16:41.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:16:41 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9660.798008 secs 2024-03-28T03:16:41.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:16:41 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9660.798008 secs 2024-03-28T03:16:41.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:16:41 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9660.798008 secs 2024-03-28T03:16:42.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:16:42 smithi132 ceph-mon[35941]: pgmap v5323: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:16:42.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:16:42 smithi165 ceph-mon[36302]: pgmap v5323: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:16:42.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:16:42 smithi119 ceph-mon[31518]: pgmap v5323: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:16:44.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:16:44 smithi132 ceph-mon[35941]: pgmap v5324: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:16:44.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:16:44 smithi165 ceph-mon[36302]: pgmap v5324: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:16:44.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:16:44 smithi119 ceph-mon[31518]: pgmap v5324: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:16:46.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:16:46 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9665.798069 secs 2024-03-28T03:16:46.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:16:46 smithi132 ceph-mon[35941]: pgmap v5325: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:16:46.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:16:46 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9665.798069 secs 2024-03-28T03:16:46.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:16:46 smithi165 ceph-mon[36302]: pgmap v5325: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:16:46.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:16:46 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9665.798069 secs 2024-03-28T03:16:46.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:16:46 smithi119 ceph-mon[31518]: pgmap v5325: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:16:48.680 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:16:48 smithi165 ceph-mon[36302]: pgmap v5326: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:16:48.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:16:48 smithi132 ceph-mon[35941]: pgmap v5326: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:16:48.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:16:48 smithi119 ceph-mon[31518]: pgmap v5326: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:16:50.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:16:50 smithi132 ceph-mon[35941]: pgmap v5327: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:16:50.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:16:50 smithi165 ceph-mon[36302]: pgmap v5327: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:16:50.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:16:50 smithi119 ceph-mon[31518]: pgmap v5327: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:16:51.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:16:51 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9670.798131 secs 2024-03-28T03:16:51.804 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:16:51 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9670.798131 secs 2024-03-28T03:16:51.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:16:51 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9670.798131 secs 2024-03-28T03:16:52.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:16:52 smithi132 ceph-mon[35941]: pgmap v5328: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:16:52.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:16:52 smithi165 ceph-mon[36302]: pgmap v5328: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:16:52.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:16:52 smithi119 ceph-mon[31518]: pgmap v5328: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:16:54.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:16:54 smithi132 ceph-mon[35941]: pgmap v5329: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:16:54.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:16:54 smithi165 ceph-mon[36302]: pgmap v5329: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:16:54.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:16:54 smithi119 ceph-mon[31518]: pgmap v5329: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:16:56.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:16:56 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9675.798192 secs 2024-03-28T03:16:56.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:16:56 smithi132 ceph-mon[35941]: pgmap v5330: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:16:56.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:16:56 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9675.798192 secs 2024-03-28T03:16:56.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:16:56 smithi165 ceph-mon[36302]: pgmap v5330: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:16:56.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:16:56 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9675.798192 secs 2024-03-28T03:16:56.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:16:56 smithi119 ceph-mon[31518]: pgmap v5330: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:16:58.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:16:58 smithi132 ceph-mon[35941]: pgmap v5331: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:16:58.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:16:58 smithi165 ceph-mon[36302]: pgmap v5331: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:16:58.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:16:58 smithi119 ceph-mon[31518]: pgmap v5331: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:17:00.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:17:00 smithi132 ceph-mon[35941]: pgmap v5332: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:17:00.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:17:00 smithi165 ceph-mon[36302]: pgmap v5332: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:17:00.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:17:00 smithi119 ceph-mon[31518]: pgmap v5332: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:17:01.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:17:01 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9680.798231 secs 2024-03-28T03:17:01.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:17:01 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9680.798231 secs 2024-03-28T03:17:01.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:17:01 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9680.798231 secs 2024-03-28T03:17:02.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:17:02 smithi165 ceph-mon[36302]: pgmap v5333: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:17:02.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:17:02 smithi119 ceph-mon[31518]: pgmap v5333: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:17:02.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:17:02 smithi132 ceph-mon[35941]: pgmap v5333: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:17:04.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:17:04 smithi165 ceph-mon[36302]: pgmap v5334: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:17:04.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:17:04 smithi119 ceph-mon[31518]: pgmap v5334: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:17:04.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:17:04 smithi132 ceph-mon[35941]: pgmap v5334: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:17:06.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:17:06 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9685.798293 secs 2024-03-28T03:17:06.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:17:06 smithi165 ceph-mon[36302]: pgmap v5335: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:17:06.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:17:06 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9685.798293 secs 2024-03-28T03:17:06.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:17:06 smithi119 ceph-mon[31518]: pgmap v5335: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:17:06.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:17:06 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9685.798293 secs 2024-03-28T03:17:06.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:17:06 smithi132 ceph-mon[35941]: pgmap v5335: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:17:08.752 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:17:08 smithi132 ceph-mon[35941]: pgmap v5336: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:17:08.752 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:17:08 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T03:17:08.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:17:08 smithi165 ceph-mon[36302]: pgmap v5336: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:17:08.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:17:08 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T03:17:08.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:17:08 smithi119 ceph-mon[31518]: pgmap v5336: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:17:08.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:17:08 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T03:17:09.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:17:09 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T03:17:09.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:17:09 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T03:17:09.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:17:09 smithi165 ceph-mon[36302]: pgmap v5337: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T03:17:09.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:17:09 smithi165 ceph-mon[36302]: pgmap v5338: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-28T03:17:09.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:17:09 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:17:09.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:17:09 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T03:17:09.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:17:09 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T03:17:09.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:17:09 smithi119 ceph-mon[31518]: pgmap v5337: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T03:17:09.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:17:09 smithi119 ceph-mon[31518]: pgmap v5338: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-28T03:17:09.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:17:09 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:17:09.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:17:09 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T03:17:09.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:17:09 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T03:17:09.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:17:09 smithi132 ceph-mon[35941]: pgmap v5337: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T03:17:09.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:17:09 smithi132 ceph-mon[35941]: pgmap v5338: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-28T03:17:09.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:17:09 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:17:11.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:17:10 smithi165 ceph-mon[36302]: pgmap v5339: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:17:11.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:17:10 smithi119 ceph-mon[31518]: pgmap v5339: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:17:11.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:17:10 smithi132 ceph-mon[35941]: pgmap v5339: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:17:12.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:17:11 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9690.798354 secs 2024-03-28T03:17:12.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:17:11 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9690.798354 secs 2024-03-28T03:17:12.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:17:11 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9690.798354 secs 2024-03-28T03:17:13.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:17:12 smithi165 ceph-mon[36302]: pgmap v5340: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:17:13.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:17:12 smithi119 ceph-mon[31518]: pgmap v5340: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:17:13.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:17:12 smithi132 ceph-mon[35941]: pgmap v5340: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:17:15.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:17:14 smithi165 ceph-mon[36302]: pgmap v5341: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:17:15.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:17:14 smithi119 ceph-mon[31518]: pgmap v5341: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:17:15.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:17:14 smithi132 ceph-mon[35941]: pgmap v5341: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:17:16.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:17:15 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9695.798393 secs 2024-03-28T03:17:16.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:17:15 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9695.798393 secs 2024-03-28T03:17:16.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:17:15 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9695.798393 secs 2024-03-28T03:17:17.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:17:17 smithi165 ceph-mon[36302]: pgmap v5342: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:17:17.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:17:17 smithi119 ceph-mon[31518]: pgmap v5342: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:17:17.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:17:17 smithi132 ceph-mon[35941]: pgmap v5342: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:17:19.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:17:18 smithi165 ceph-mon[36302]: pgmap v5343: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T03:17:19.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:17:18 smithi119 ceph-mon[31518]: pgmap v5343: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T03:17:19.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:17:18 smithi132 ceph-mon[35941]: pgmap v5343: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T03:17:21.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:17:20 smithi165 ceph-mon[36302]: pgmap v5344: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:17:21.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:17:20 smithi119 ceph-mon[31518]: pgmap v5344: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:17:21.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:17:20 smithi132 ceph-mon[35941]: pgmap v5344: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:17:22.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:17:21 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9700.798431 secs 2024-03-28T03:17:22.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:17:21 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9700.798431 secs 2024-03-28T03:17:22.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:17:21 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9700.798431 secs 2024-03-28T03:17:23.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:17:22 smithi165 ceph-mon[36302]: pgmap v5345: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:17:23.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:17:22 smithi119 ceph-mon[31518]: pgmap v5345: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:17:23.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:17:22 smithi132 ceph-mon[35941]: pgmap v5345: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:17:25.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:17:24 smithi165 ceph-mon[36302]: pgmap v5346: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:17:25.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:17:24 smithi119 ceph-mon[31518]: pgmap v5346: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:17:25.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:17:24 smithi132 ceph-mon[35941]: pgmap v5346: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:17:26.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:17:25 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9705.798518 secs 2024-03-28T03:17:26.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:17:25 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9705.798518 secs 2024-03-28T03:17:26.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:17:25 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9705.798518 secs 2024-03-28T03:17:27.304 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:17:27 smithi119 ceph-mon[31518]: pgmap v5347: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:17:27.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:17:27 smithi165 ceph-mon[36302]: pgmap v5347: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:17:27.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:17:27 smithi132 ceph-mon[35941]: pgmap v5347: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:17:29.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:17:28 smithi165 ceph-mon[36302]: pgmap v5348: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:17:29.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:17:28 smithi119 ceph-mon[31518]: pgmap v5348: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:17:29.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:17:28 smithi132 ceph-mon[35941]: pgmap v5348: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:17:31.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:17:30 smithi165 ceph-mon[36302]: pgmap v5349: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:17:31.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:17:30 smithi119 ceph-mon[31518]: pgmap v5349: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:17:31.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:17:30 smithi132 ceph-mon[35941]: pgmap v5349: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:17:32.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:17:31 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9710.798604 secs 2024-03-28T03:17:32.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:17:31 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9710.798604 secs 2024-03-28T03:17:32.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:17:31 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9710.798604 secs 2024-03-28T03:17:33.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:17:32 smithi165 ceph-mon[36302]: pgmap v5350: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:17:33.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:17:32 smithi119 ceph-mon[31518]: pgmap v5350: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:17:33.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:17:32 smithi132 ceph-mon[35941]: pgmap v5350: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:17:35.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:17:34 smithi165 ceph-mon[36302]: pgmap v5351: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:17:35.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:17:34 smithi119 ceph-mon[31518]: pgmap v5351: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:17:35.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:17:34 smithi132 ceph-mon[35941]: pgmap v5351: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:17:36.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:17:35 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9715.798692 secs 2024-03-28T03:17:36.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:17:35 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9715.798692 secs 2024-03-28T03:17:36.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:17:35 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9715.798692 secs 2024-03-28T03:17:37.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:17:37 smithi165 ceph-mon[36302]: pgmap v5352: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:17:37.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:17:37 smithi119 ceph-mon[31518]: pgmap v5352: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:17:37.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:17:37 smithi132 ceph-mon[35941]: pgmap v5352: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:17:39.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:17:38 smithi165 ceph-mon[36302]: pgmap v5353: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:17:39.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:17:38 smithi119 ceph-mon[31518]: pgmap v5353: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:17:39.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:17:38 smithi132 ceph-mon[35941]: pgmap v5353: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:17:41.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:17:40 smithi165 ceph-mon[36302]: pgmap v5354: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:17:41.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:17:40 smithi119 ceph-mon[31518]: pgmap v5354: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:17:41.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:17:40 smithi132 ceph-mon[35941]: pgmap v5354: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:17:42.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:17:41 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9720.798781 secs 2024-03-28T03:17:42.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:17:41 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9720.798781 secs 2024-03-28T03:17:42.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:17:41 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9720.798781 secs 2024-03-28T03:17:43.078 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:17:42 smithi119 ceph-mon[31518]: pgmap v5355: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:17:43.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:17:42 smithi165 ceph-mon[36302]: pgmap v5355: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:17:43.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:17:42 smithi132 ceph-mon[35941]: pgmap v5355: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:17:45.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:17:44 smithi165 ceph-mon[36302]: pgmap v5356: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:17:45.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:17:44 smithi119 ceph-mon[31518]: pgmap v5356: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:17:45.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:17:44 smithi132 ceph-mon[35941]: pgmap v5356: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:17:46.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:17:45 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9725.798870 secs 2024-03-28T03:17:46.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:17:45 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9725.798870 secs 2024-03-28T03:17:46.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:17:45 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9725.798870 secs 2024-03-28T03:17:47.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:17:47 smithi165 ceph-mon[36302]: pgmap v5357: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:17:47.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:17:47 smithi119 ceph-mon[31518]: pgmap v5357: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:17:47.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:17:47 smithi132 ceph-mon[35941]: pgmap v5357: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:17:49.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:17:48 smithi165 ceph-mon[36302]: pgmap v5358: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:17:49.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:17:48 smithi119 ceph-mon[31518]: pgmap v5358: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:17:49.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:17:48 smithi132 ceph-mon[35941]: pgmap v5358: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:17:51.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:17:50 smithi165 ceph-mon[36302]: pgmap v5359: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:17:51.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:17:50 smithi119 ceph-mon[31518]: pgmap v5359: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:17:51.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:17:50 smithi132 ceph-mon[35941]: pgmap v5359: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:17:52.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:17:51 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9730.798957 secs 2024-03-28T03:17:52.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:17:51 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9730.798957 secs 2024-03-28T03:17:52.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:17:51 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9730.798957 secs 2024-03-28T03:17:53.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:17:52 smithi165 ceph-mon[36302]: pgmap v5360: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T03:17:53.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:17:52 smithi119 ceph-mon[31518]: pgmap v5360: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T03:17:53.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:17:52 smithi132 ceph-mon[35941]: pgmap v5360: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T03:17:55.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:17:54 smithi165 ceph-mon[36302]: pgmap v5361: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-03-28T03:17:55.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:17:54 smithi119 ceph-mon[31518]: pgmap v5361: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-03-28T03:17:55.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:17:54 smithi132 ceph-mon[35941]: pgmap v5361: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-03-28T03:17:56.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:17:56 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9735.799046 secs 2024-03-28T03:17:56.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:17:56 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9735.799046 secs 2024-03-28T03:17:56.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:17:56 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9735.799046 secs 2024-03-28T03:17:57.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:17:57 smithi119 ceph-mon[31518]: pgmap v5362: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-03-28T03:17:57.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:17:57 smithi132 ceph-mon[35941]: pgmap v5362: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-03-28T03:17:58.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:17:57 smithi165 ceph-mon[36302]: pgmap v5362: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-03-28T03:17:59.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:17:58 smithi165 ceph-mon[36302]: pgmap v5363: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-03-28T03:17:59.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:17:58 smithi119 ceph-mon[31518]: pgmap v5363: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-03-28T03:17:59.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:17:58 smithi132 ceph-mon[35941]: pgmap v5363: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-03-28T03:18:01.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:18:00 smithi165 ceph-mon[36302]: pgmap v5364: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-03-28T03:18:01.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:18:00 smithi119 ceph-mon[31518]: pgmap v5364: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-03-28T03:18:01.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:18:00 smithi132 ceph-mon[35941]: pgmap v5364: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-03-28T03:18:02.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:18:01 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9740.799107 secs 2024-03-28T03:18:02.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:18:01 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9740.799107 secs 2024-03-28T03:18:02.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:18:01 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9740.799107 secs 2024-03-28T03:18:03.120 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:18:02 smithi165 ceph-mon[36302]: pgmap v5365: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T03:18:03.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:18:02 smithi119 ceph-mon[31518]: pgmap v5365: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T03:18:03.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:18:02 smithi132 ceph-mon[35941]: pgmap v5365: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T03:18:05.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:18:04 smithi165 ceph-mon[36302]: pgmap v5366: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:18:05.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:18:04 smithi119 ceph-mon[31518]: pgmap v5366: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:18:05.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:18:04 smithi132 ceph-mon[35941]: pgmap v5366: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:18:06.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:18:05 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9745.799144 secs 2024-03-28T03:18:06.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:18:05 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9745.799144 secs 2024-03-28T03:18:06.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:18:05 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9745.799144 secs 2024-03-28T03:18:07.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:18:07 smithi165 ceph-mon[36302]: pgmap v5367: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:18:07.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:18:07 smithi119 ceph-mon[31518]: pgmap v5367: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:18:07.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:18:07 smithi132 ceph-mon[35941]: pgmap v5367: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:18:09.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:18:08 smithi165 ceph-mon[36302]: pgmap v5368: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:18:09.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:18:08 smithi119 ceph-mon[31518]: pgmap v5368: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:18:09.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:18:08 smithi132 ceph-mon[35941]: pgmap v5368: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:18:10.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:18:09 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T03:18:10.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:18:09 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T03:18:10.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:18:09 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T03:18:11.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:18:10 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T03:18:11.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:18:10 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T03:18:11.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:18:10 smithi165 ceph-mon[36302]: pgmap v5369: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T03:18:11.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:18:10 smithi165 ceph-mon[36302]: pgmap v5370: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:18:11.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:18:10 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:18:11.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:18:10 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T03:18:11.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:18:10 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T03:18:11.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:18:10 smithi119 ceph-mon[31518]: pgmap v5369: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T03:18:11.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:18:10 smithi119 ceph-mon[31518]: pgmap v5370: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:18:11.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:18:10 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:18:11.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:18:10 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T03:18:11.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:18:10 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T03:18:11.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:18:10 smithi132 ceph-mon[35941]: pgmap v5369: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T03:18:11.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:18:10 smithi132 ceph-mon[35941]: pgmap v5370: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:18:11.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:18:10 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:18:12.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:18:11 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9750.799204 secs 2024-03-28T03:18:12.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:18:11 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9750.799204 secs 2024-03-28T03:18:12.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:18:11 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9750.799204 secs 2024-03-28T03:18:13.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:18:12 smithi165 ceph-mon[36302]: pgmap v5371: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:18:13.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:18:12 smithi119 ceph-mon[31518]: pgmap v5371: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:18:13.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:18:12 smithi132 ceph-mon[35941]: pgmap v5371: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:18:15.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:18:15 smithi165 ceph-mon[36302]: pgmap v5372: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 5 op/s 2024-03-28T03:18:15.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:18:15 smithi119 ceph-mon[31518]: pgmap v5372: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 5 op/s 2024-03-28T03:18:15.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:18:15 smithi132 ceph-mon[35941]: pgmap v5372: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 5 op/s 2024-03-28T03:18:16.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:18:16 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9755.799290 secs 2024-03-28T03:18:16.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:18:16 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9755.799290 secs 2024-03-28T03:18:16.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:18:16 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9755.799290 secs 2024-03-28T03:18:17.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:18:17 smithi165 ceph-mon[36302]: pgmap v5373: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:18:17.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:18:17 smithi119 ceph-mon[31518]: pgmap v5373: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:18:17.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:18:17 smithi132 ceph-mon[35941]: pgmap v5373: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:18:19.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:18:19 smithi165 ceph-mon[36302]: pgmap v5374: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:18:19.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:18:19 smithi119 ceph-mon[31518]: pgmap v5374: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:18:19.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:18:19 smithi132 ceph-mon[35941]: pgmap v5374: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:18:21.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:18:21 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9760.799351 secs 2024-03-28T03:18:21.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:18:21 smithi165 ceph-mon[36302]: pgmap v5375: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T03:18:21.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:18:21 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9760.799351 secs 2024-03-28T03:18:21.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:18:21 smithi119 ceph-mon[31518]: pgmap v5375: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T03:18:21.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:18:21 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9760.799351 secs 2024-03-28T03:18:21.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:18:21 smithi132 ceph-mon[35941]: pgmap v5375: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T03:18:23.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:18:23 smithi165 ceph-mon[36302]: pgmap v5376: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:18:23.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:18:23 smithi119 ceph-mon[31518]: pgmap v5376: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:18:23.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:18:23 smithi132 ceph-mon[35941]: pgmap v5376: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:18:25.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:18:25 smithi165 ceph-mon[36302]: pgmap v5377: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-28T03:18:25.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:18:25 smithi119 ceph-mon[31518]: pgmap v5377: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-28T03:18:25.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:18:25 smithi132 ceph-mon[35941]: pgmap v5377: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-28T03:18:26.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:18:26 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9765.799439 secs 2024-03-28T03:18:26.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:18:26 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9765.799439 secs 2024-03-28T03:18:26.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:18:26 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9765.799439 secs 2024-03-28T03:18:27.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:18:27 smithi165 ceph-mon[36302]: pgmap v5378: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-03-28T03:18:27.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:18:27 smithi119 ceph-mon[31518]: pgmap v5378: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-03-28T03:18:27.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:18:27 smithi132 ceph-mon[35941]: pgmap v5378: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-03-28T03:18:29.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:18:29 smithi165 ceph-mon[36302]: pgmap v5379: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:18:29.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:18:29 smithi119 ceph-mon[31518]: pgmap v5379: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:18:29.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:18:29 smithi132 ceph-mon[35941]: pgmap v5379: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:18:31.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:18:31 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9770.799527 secs 2024-03-28T03:18:31.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:18:31 smithi165 ceph-mon[36302]: pgmap v5380: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:18:31.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:18:31 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9770.799527 secs 2024-03-28T03:18:31.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:18:31 smithi119 ceph-mon[31518]: pgmap v5380: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:18:31.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:18:31 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9770.799527 secs 2024-03-28T03:18:31.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:18:31 smithi132 ceph-mon[35941]: pgmap v5380: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:18:33.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:18:33 smithi165 ceph-mon[36302]: pgmap v5381: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:18:33.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:18:33 smithi119 ceph-mon[31518]: pgmap v5381: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:18:33.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:18:33 smithi132 ceph-mon[35941]: pgmap v5381: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:18:35.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:18:35 smithi165 ceph-mon[36302]: pgmap v5382: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-28T03:18:35.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:18:35 smithi119 ceph-mon[31518]: pgmap v5382: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-28T03:18:35.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:18:35 smithi132 ceph-mon[35941]: pgmap v5382: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-28T03:18:36.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:18:36 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9775.799613 secs 2024-03-28T03:18:36.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:18:36 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9775.799613 secs 2024-03-28T03:18:36.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:18:36 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9775.799613 secs 2024-03-28T03:18:37.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:18:37 smithi165 ceph-mon[36302]: pgmap v5383: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-28T03:18:37.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:18:37 smithi119 ceph-mon[31518]: pgmap v5383: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-28T03:18:37.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:18:37 smithi132 ceph-mon[35941]: pgmap v5383: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-28T03:18:39.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:18:39 smithi165 ceph-mon[36302]: pgmap v5384: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:18:39.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:18:39 smithi119 ceph-mon[31518]: pgmap v5384: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:18:39.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:18:39 smithi132 ceph-mon[35941]: pgmap v5384: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:18:41.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:18:41 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9780.799674 secs 2024-03-28T03:18:41.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:18:41 smithi165 ceph-mon[36302]: pgmap v5385: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:18:41.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:18:41 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9780.799674 secs 2024-03-28T03:18:41.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:18:41 smithi119 ceph-mon[31518]: pgmap v5385: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:18:41.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:18:41 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9780.799674 secs 2024-03-28T03:18:41.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:18:41 smithi132 ceph-mon[35941]: pgmap v5385: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:18:43.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:18:43 smithi165 ceph-mon[36302]: pgmap v5386: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:18:43.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:18:43 smithi119 ceph-mon[31518]: pgmap v5386: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:18:43.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:18:43 smithi132 ceph-mon[35941]: pgmap v5386: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:18:45.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:18:45 smithi119 ceph-mon[31518]: pgmap v5387: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-28T03:18:45.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:18:45 smithi132 ceph-mon[35941]: pgmap v5387: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-28T03:18:45.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:18:45 smithi165 ceph-mon[36302]: pgmap v5387: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-28T03:18:46.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:18:46 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9785.799761 secs 2024-03-28T03:18:46.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:18:46 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9785.799761 secs 2024-03-28T03:18:46.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:18:46 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9785.799761 secs 2024-03-28T03:18:47.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:18:47 smithi119 ceph-mon[31518]: pgmap v5388: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-28T03:18:47.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:18:47 smithi132 ceph-mon[35941]: pgmap v5388: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-28T03:18:47.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:18:47 smithi165 ceph-mon[36302]: pgmap v5388: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-28T03:18:49.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:18:49 smithi119 ceph-mon[31518]: pgmap v5389: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:18:49.682 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:18:49 smithi165 ceph-mon[36302]: pgmap v5389: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:18:49.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:18:49 smithi132 ceph-mon[35941]: pgmap v5389: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:18:51.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:18:51 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9790.799847 secs 2024-03-28T03:18:51.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:18:51 smithi119 ceph-mon[31518]: pgmap v5390: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:18:51.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:18:51 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9790.799847 secs 2024-03-28T03:18:51.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:18:51 smithi132 ceph-mon[35941]: pgmap v5390: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:18:51.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:18:51 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9790.799847 secs 2024-03-28T03:18:51.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:18:51 smithi165 ceph-mon[36302]: pgmap v5390: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:18:53.701 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:18:53 smithi132 ceph-mon[35941]: pgmap v5391: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:18:53.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:18:53 smithi165 ceph-mon[36302]: pgmap v5391: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:18:53.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:18:53 smithi119 ceph-mon[31518]: pgmap v5391: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:18:55.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:18:55 smithi132 ceph-mon[35941]: pgmap v5392: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-28T03:18:55.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:18:55 smithi165 ceph-mon[36302]: pgmap v5392: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-28T03:18:55.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:18:55 smithi119 ceph-mon[31518]: pgmap v5392: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-28T03:18:56.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:18:56 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9795.799936 secs 2024-03-28T03:18:56.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:18:56 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9795.799936 secs 2024-03-28T03:18:56.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:18:56 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9795.799936 secs 2024-03-28T03:18:57.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:18:57 smithi132 ceph-mon[35941]: pgmap v5393: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-28T03:18:57.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:18:57 smithi165 ceph-mon[36302]: pgmap v5393: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-28T03:18:57.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:18:57 smithi119 ceph-mon[31518]: pgmap v5393: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-28T03:18:59.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:18:59 smithi132 ceph-mon[35941]: pgmap v5394: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:18:59.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:18:59 smithi165 ceph-mon[36302]: pgmap v5394: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:18:59.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:18:59 smithi119 ceph-mon[31518]: pgmap v5394: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:19:01.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:19:01 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9800.800023 secs 2024-03-28T03:19:01.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:19:01 smithi132 ceph-mon[35941]: pgmap v5395: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:19:01.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:19:01 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9800.800023 secs 2024-03-28T03:19:01.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:19:01 smithi165 ceph-mon[36302]: pgmap v5395: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:19:01.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:19:01 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9800.800023 secs 2024-03-28T03:19:01.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:19:01 smithi119 ceph-mon[31518]: pgmap v5395: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:19:03.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:19:03 smithi132 ceph-mon[35941]: pgmap v5396: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:19:03.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:19:03 smithi165 ceph-mon[36302]: pgmap v5396: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:19:03.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:19:03 smithi119 ceph-mon[31518]: pgmap v5396: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:19:05.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:19:05 smithi132 ceph-mon[35941]: pgmap v5397: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-28T03:19:05.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:19:05 smithi165 ceph-mon[36302]: pgmap v5397: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-28T03:19:05.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:19:05 smithi119 ceph-mon[31518]: pgmap v5397: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-28T03:19:06.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:19:06 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9805.800086 secs 2024-03-28T03:19:07.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:19:06 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9805.800086 secs 2024-03-28T03:19:07.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:19:06 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9805.800086 secs 2024-03-28T03:19:07.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:19:07 smithi132 ceph-mon[35941]: pgmap v5398: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-28T03:19:08.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:19:07 smithi165 ceph-mon[36302]: pgmap v5398: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-28T03:19:08.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:19:07 smithi119 ceph-mon[31518]: pgmap v5398: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-28T03:19:09.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:19:09 smithi132 ceph-mon[35941]: pgmap v5399: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:19:10.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:19:09 smithi165 ceph-mon[36302]: pgmap v5399: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:19:10.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:19:09 smithi119 ceph-mon[31518]: pgmap v5399: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:19:10.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:19:10 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T03:19:11.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:19:10 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T03:19:11.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:19:10 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T03:19:11.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:19:11 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9810.800146 secs 2024-03-28T03:19:11.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:19:11 smithi132 ceph-mon[35941]: pgmap v5400: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:19:12.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:19:11 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9810.800146 secs 2024-03-28T03:19:12.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:19:11 smithi165 ceph-mon[36302]: pgmap v5400: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:19:12.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:19:11 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9810.800146 secs 2024-03-28T03:19:12.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:19:11 smithi119 ceph-mon[31518]: pgmap v5400: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:19:13.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:19:12 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T03:19:13.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:19:12 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T03:19:13.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:19:12 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:19:13.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:19:12 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T03:19:13.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:19:12 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T03:19:13.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:19:12 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:19:13.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:19:12 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T03:19:13.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:19:12 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T03:19:13.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:19:12 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:19:14.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:19:13 smithi165 ceph-mon[36302]: pgmap v5401: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T03:19:14.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:19:13 smithi165 ceph-mon[36302]: pgmap v5402: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:19:14.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:19:13 smithi119 ceph-mon[31518]: pgmap v5401: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T03:19:14.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:19:13 smithi119 ceph-mon[31518]: pgmap v5402: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:19:14.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:19:13 smithi132 ceph-mon[35941]: pgmap v5401: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T03:19:14.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:19:13 smithi132 ceph-mon[35941]: pgmap v5402: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:19:16.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:19:15 smithi165 ceph-mon[36302]: pgmap v5403: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-28T03:19:16.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:19:15 smithi119 ceph-mon[31518]: pgmap v5403: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-28T03:19:16.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:19:15 smithi132 ceph-mon[35941]: pgmap v5403: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-28T03:19:17.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:19:16 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9815.800206 secs 2024-03-28T03:19:17.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:19:16 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9815.800206 secs 2024-03-28T03:19:17.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:19:16 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9815.800206 secs 2024-03-28T03:19:18.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:19:17 smithi165 ceph-mon[36302]: pgmap v5404: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:19:18.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:19:17 smithi119 ceph-mon[31518]: pgmap v5404: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:19:18.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:19:17 smithi132 ceph-mon[35941]: pgmap v5404: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:19:20.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:19:19 smithi165 ceph-mon[36302]: pgmap v5405: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:19:20.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:19:19 smithi119 ceph-mon[31518]: pgmap v5405: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:19:20.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:19:19 smithi132 ceph-mon[35941]: pgmap v5405: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:19:22.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:19:21 smithi165 ceph-mon[36302]: pgmap v5406: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:19:22.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:19:21 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9820.800290 secs 2024-03-28T03:19:22.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:19:21 smithi119 ceph-mon[31518]: pgmap v5406: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:19:22.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:19:21 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9820.800290 secs 2024-03-28T03:19:22.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:19:21 smithi132 ceph-mon[35941]: pgmap v5406: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:19:22.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:19:21 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9820.800290 secs 2024-03-28T03:19:24.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:19:23 smithi165 ceph-mon[36302]: pgmap v5407: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T03:19:24.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:19:23 smithi119 ceph-mon[31518]: pgmap v5407: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T03:19:24.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:19:23 smithi132 ceph-mon[35941]: pgmap v5407: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T03:19:26.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:19:25 smithi165 ceph-mon[36302]: pgmap v5408: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:19:26.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:19:25 smithi119 ceph-mon[31518]: pgmap v5408: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:19:26.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:19:25 smithi132 ceph-mon[35941]: pgmap v5408: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:19:27.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:19:26 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9825.800373 secs 2024-03-28T03:19:27.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:19:26 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9825.800373 secs 2024-03-28T03:19:27.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:19:26 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9825.800373 secs 2024-03-28T03:19:28.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:19:27 smithi165 ceph-mon[36302]: pgmap v5409: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:19:28.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:19:27 smithi119 ceph-mon[31518]: pgmap v5409: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:19:28.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:19:27 smithi132 ceph-mon[35941]: pgmap v5409: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:19:30.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:19:29 smithi165 ceph-mon[36302]: pgmap v5410: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:19:30.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:19:29 smithi119 ceph-mon[31518]: pgmap v5410: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:19:30.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:19:29 smithi132 ceph-mon[35941]: pgmap v5410: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:19:32.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:19:31 smithi165 ceph-mon[36302]: pgmap v5411: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:19:32.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:19:31 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9830.800462 secs 2024-03-28T03:19:32.132 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:19:31 smithi119 ceph-mon[31518]: pgmap v5411: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:19:32.132 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:19:31 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9830.800462 secs 2024-03-28T03:19:32.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:19:31 smithi132 ceph-mon[35941]: pgmap v5411: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:19:32.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:19:31 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9830.800462 secs 2024-03-28T03:19:34.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:19:33 smithi165 ceph-mon[36302]: pgmap v5412: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:19:34.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:19:33 smithi119 ceph-mon[31518]: pgmap v5412: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:19:34.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:19:33 smithi132 ceph-mon[35941]: pgmap v5412: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:19:36.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:19:35 smithi165 ceph-mon[36302]: pgmap v5413: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:19:36.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:19:35 smithi119 ceph-mon[31518]: pgmap v5413: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:19:36.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:19:35 smithi132 ceph-mon[35941]: pgmap v5413: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:19:37.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:19:36 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9835.800549 secs 2024-03-28T03:19:37.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:19:36 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9835.800549 secs 2024-03-28T03:19:37.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:19:36 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9835.800549 secs 2024-03-28T03:19:38.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:19:37 smithi165 ceph-mon[36302]: pgmap v5414: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:19:38.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:19:37 smithi119 ceph-mon[31518]: pgmap v5414: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:19:38.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:19:37 smithi132 ceph-mon[35941]: pgmap v5414: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:19:39.101 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:19:38 smithi132 ceph-mon[35941]: pgmap v5415: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:19:39.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:19:38 smithi165 ceph-mon[36302]: pgmap v5415: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:19:39.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:19:38 smithi119 ceph-mon[31518]: pgmap v5415: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:19:41.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:19:40 smithi165 ceph-mon[36302]: pgmap v5416: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:19:41.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:19:40 smithi119 ceph-mon[31518]: pgmap v5416: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:19:41.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:19:40 smithi132 ceph-mon[35941]: pgmap v5416: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:19:42.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:19:41 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9840.800638 secs 2024-03-28T03:19:42.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:19:41 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9840.800638 secs 2024-03-28T03:19:42.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:19:41 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9840.800638 secs 2024-03-28T03:19:43.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:19:42 smithi165 ceph-mon[36302]: pgmap v5417: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:19:43.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:19:42 smithi119 ceph-mon[31518]: pgmap v5417: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:19:43.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:19:42 smithi132 ceph-mon[35941]: pgmap v5417: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:19:45.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:19:44 smithi119 ceph-mon[31518]: pgmap v5418: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:19:45.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:19:44 smithi132 ceph-mon[35941]: pgmap v5418: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:19:45.382 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:19:44 smithi165 ceph-mon[36302]: pgmap v5418: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:19:47.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:19:46 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9845.800726 secs 2024-03-28T03:19:47.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:19:46 smithi119 ceph-mon[31518]: pgmap v5419: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:19:47.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:19:46 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9845.800726 secs 2024-03-28T03:19:47.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:19:46 smithi132 ceph-mon[35941]: pgmap v5419: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:19:47.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:19:46 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9845.800726 secs 2024-03-28T03:19:47.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:19:46 smithi165 ceph-mon[36302]: pgmap v5419: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:19:49.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:19:48 smithi119 ceph-mon[31518]: pgmap v5420: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:19:49.181 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:19:48 smithi165 ceph-mon[36302]: pgmap v5420: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:19:49.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:19:48 smithi132 ceph-mon[35941]: pgmap v5420: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:19:51.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:19:50 smithi119 ceph-mon[31518]: pgmap v5421: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:19:51.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:19:50 smithi132 ceph-mon[35941]: pgmap v5421: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:19:51.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:19:50 smithi165 ceph-mon[36302]: pgmap v5421: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:19:52.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:19:51 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9850.800815 secs 2024-03-28T03:19:52.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:19:51 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9850.800815 secs 2024-03-28T03:19:52.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:19:51 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9850.800815 secs 2024-03-28T03:19:53.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:19:52 smithi132 ceph-mon[35941]: pgmap v5422: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:19:53.304 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:19:52 smithi119 ceph-mon[31518]: pgmap v5422: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:19:53.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:19:52 smithi165 ceph-mon[36302]: pgmap v5422: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:19:55.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:19:54 smithi132 ceph-mon[35941]: pgmap v5423: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:19:55.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:19:54 smithi165 ceph-mon[36302]: pgmap v5423: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:19:55.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:19:54 smithi119 ceph-mon[31518]: pgmap v5423: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:19:57.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:19:56 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9855.800915 secs 2024-03-28T03:19:57.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:19:56 smithi132 ceph-mon[35941]: pgmap v5424: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:19:57.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:19:56 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9855.800915 secs 2024-03-28T03:19:57.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:19:56 smithi165 ceph-mon[36302]: pgmap v5424: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:19:57.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:19:56 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9855.800915 secs 2024-03-28T03:19:57.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:19:56 smithi119 ceph-mon[31518]: pgmap v5424: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:19:59.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:19:58 smithi132 ceph-mon[35941]: pgmap v5425: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:19:59.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:19:58 smithi165 ceph-mon[36302]: pgmap v5425: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:19:59.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:19:58 smithi119 ceph-mon[31518]: pgmap v5425: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:20:01.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:20:00 smithi132 ceph-mon[35941]: pgmap v5426: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:20:01.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:20:00 smithi132 ceph-mon[35941]: overall HEALTH_WARN 5 failed cephadm daemon(s); 1 filesystem is degraded; 1 MDSs report slow requests 2024-03-28T03:20:01.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:20:00 smithi165 ceph-mon[36302]: pgmap v5426: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:20:01.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:20:00 smithi165 ceph-mon[36302]: overall HEALTH_WARN 5 failed cephadm daemon(s); 1 filesystem is degraded; 1 MDSs report slow requests 2024-03-28T03:20:01.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:20:00 smithi119 ceph-mon[31518]: pgmap v5426: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:20:01.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:20:00 smithi119 ceph-mon[31518]: overall HEALTH_WARN 5 failed cephadm daemon(s); 1 filesystem is degraded; 1 MDSs report slow requests 2024-03-28T03:20:02.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:20:01 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9860.801003 secs 2024-03-28T03:20:02.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:20:01 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9860.801003 secs 2024-03-28T03:20:02.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:20:01 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9860.801003 secs 2024-03-28T03:20:03.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:20:02 smithi119 ceph-mon[31518]: pgmap v5427: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:20:03.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:20:02 smithi132 ceph-mon[35941]: pgmap v5427: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:20:03.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:20:02 smithi165 ceph-mon[36302]: pgmap v5427: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:20:05.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:20:04 smithi132 ceph-mon[35941]: pgmap v5428: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:20:05.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:20:04 smithi165 ceph-mon[36302]: pgmap v5428: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:20:05.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:20:04 smithi119 ceph-mon[31518]: pgmap v5428: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:20:07.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:20:06 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9865.801066 secs 2024-03-28T03:20:07.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:20:06 smithi132 ceph-mon[35941]: pgmap v5429: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:20:07.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:20:06 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9865.801066 secs 2024-03-28T03:20:07.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:20:06 smithi165 ceph-mon[36302]: pgmap v5429: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:20:07.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:20:06 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9865.801066 secs 2024-03-28T03:20:07.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:20:06 smithi119 ceph-mon[31518]: pgmap v5429: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:20:09.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:20:08 smithi132 ceph-mon[35941]: pgmap v5430: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:20:09.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:20:08 smithi165 ceph-mon[36302]: pgmap v5430: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:20:09.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:20:08 smithi119 ceph-mon[31518]: pgmap v5430: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:20:11.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:20:10 smithi165 ceph-mon[36302]: pgmap v5431: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:20:11.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:20:10 smithi119 ceph-mon[31518]: pgmap v5431: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:20:11.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:20:10 smithi132 ceph-mon[35941]: pgmap v5431: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:20:12.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:20:11 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9870.801154 secs 2024-03-28T03:20:12.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:20:11 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9870.801154 secs 2024-03-28T03:20:12.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:20:11 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9870.801154 secs 2024-03-28T03:20:13.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:20:12 smithi165 ceph-mon[36302]: pgmap v5432: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:20:13.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:20:12 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T03:20:13.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:20:12 smithi119 ceph-mon[31518]: pgmap v5432: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:20:13.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:20:12 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T03:20:13.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:20:12 smithi132 ceph-mon[35941]: pgmap v5432: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:20:13.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:20:12 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T03:20:14.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:20:13 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T03:20:14.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:20:13 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T03:20:14.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:20:13 smithi165 ceph-mon[36302]: pgmap v5433: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T03:20:14.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:20:13 smithi165 ceph-mon[36302]: pgmap v5434: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:20:14.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:20:13 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:20:14.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:20:13 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T03:20:14.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:20:13 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T03:20:14.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:20:13 smithi119 ceph-mon[31518]: pgmap v5433: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T03:20:14.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:20:13 smithi119 ceph-mon[31518]: pgmap v5434: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:20:14.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:20:13 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:20:14.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:20:13 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T03:20:14.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:20:13 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T03:20:14.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:20:13 smithi132 ceph-mon[35941]: pgmap v5433: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T03:20:14.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:20:13 smithi132 ceph-mon[35941]: pgmap v5434: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:20:14.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:20:13 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:20:16.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:20:16 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9875.801214 secs 2024-03-28T03:20:16.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:20:16 smithi165 ceph-mon[36302]: pgmap v5435: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:20:16.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:20:16 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9875.801214 secs 2024-03-28T03:20:16.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:20:16 smithi119 ceph-mon[31518]: pgmap v5435: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:20:16.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:20:16 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9875.801214 secs 2024-03-28T03:20:16.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:20:16 smithi132 ceph-mon[35941]: pgmap v5435: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:20:18.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:20:18 smithi165 ceph-mon[36302]: pgmap v5436: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:20:18.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:20:18 smithi119 ceph-mon[31518]: pgmap v5436: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:20:18.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:20:18 smithi132 ceph-mon[35941]: pgmap v5436: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:20:20.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:20:20 smithi165 ceph-mon[36302]: pgmap v5437: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:20:20.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:20:20 smithi119 ceph-mon[31518]: pgmap v5437: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:20:20.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:20:20 smithi132 ceph-mon[35941]: pgmap v5437: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:20:21.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:20:21 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9880.801275 secs 2024-03-28T03:20:21.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:20:21 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9880.801275 secs 2024-03-28T03:20:21.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:20:21 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9880.801275 secs 2024-03-28T03:20:22.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:20:22 smithi119 ceph-mon[31518]: pgmap v5438: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:20:22.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:20:22 smithi132 ceph-mon[35941]: pgmap v5438: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:20:22.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:20:22 smithi165 ceph-mon[36302]: pgmap v5438: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:20:24.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:20:24 smithi119 ceph-mon[31518]: pgmap v5439: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T03:20:24.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:20:24 smithi132 ceph-mon[35941]: pgmap v5439: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T03:20:24.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:20:24 smithi165 ceph-mon[36302]: pgmap v5439: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T03:20:26.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:20:26 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9885.801335 secs 2024-03-28T03:20:26.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:20:26 smithi119 ceph-mon[31518]: pgmap v5440: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:20:26.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:20:26 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9885.801335 secs 2024-03-28T03:20:26.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:20:26 smithi132 ceph-mon[35941]: pgmap v5440: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:20:26.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:20:26 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9885.801335 secs 2024-03-28T03:20:26.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:20:26 smithi165 ceph-mon[36302]: pgmap v5440: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:20:28.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:20:28 smithi119 ceph-mon[31518]: pgmap v5441: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:20:28.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:20:28 smithi132 ceph-mon[35941]: pgmap v5441: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:20:28.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:20:28 smithi165 ceph-mon[36302]: pgmap v5441: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:20:30.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:20:30 smithi119 ceph-mon[31518]: pgmap v5442: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:20:30.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:20:30 smithi132 ceph-mon[35941]: pgmap v5442: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:20:30.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:20:30 smithi165 ceph-mon[36302]: pgmap v5442: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:20:31.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:20:31 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9890.801395 secs 2024-03-28T03:20:31.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:20:31 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9890.801395 secs 2024-03-28T03:20:31.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:20:31 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9890.801395 secs 2024-03-28T03:20:32.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:20:32 smithi132 ceph-mon[35941]: pgmap v5443: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:20:32.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:20:32 smithi165 ceph-mon[36302]: pgmap v5443: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:20:32.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:20:32 smithi119 ceph-mon[31518]: pgmap v5443: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:20:34.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:20:34 smithi132 ceph-mon[35941]: pgmap v5444: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:20:34.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:20:34 smithi165 ceph-mon[36302]: pgmap v5444: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:20:34.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:20:34 smithi119 ceph-mon[31518]: pgmap v5444: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:20:36.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:20:36 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9895.801456 secs 2024-03-28T03:20:36.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:20:36 smithi132 ceph-mon[35941]: pgmap v5445: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:20:36.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:20:36 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9895.801456 secs 2024-03-28T03:20:36.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:20:36 smithi165 ceph-mon[36302]: pgmap v5445: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:20:36.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:20:36 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9895.801456 secs 2024-03-28T03:20:36.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:20:36 smithi119 ceph-mon[31518]: pgmap v5445: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:20:38.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:20:38 smithi132 ceph-mon[35941]: pgmap v5446: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:20:38.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:20:38 smithi165 ceph-mon[36302]: pgmap v5446: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:20:38.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:20:38 smithi119 ceph-mon[31518]: pgmap v5446: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:20:40.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:20:40 smithi132 ceph-mon[35941]: pgmap v5447: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:20:40.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:20:40 smithi165 ceph-mon[36302]: pgmap v5447: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:20:40.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:20:40 smithi119 ceph-mon[31518]: pgmap v5447: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:20:41.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:20:41 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9900.801515 secs 2024-03-28T03:20:41.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:20:41 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9900.801515 secs 2024-03-28T03:20:41.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:20:41 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9900.801515 secs 2024-03-28T03:20:42.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:20:42 smithi132 ceph-mon[35941]: pgmap v5448: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:20:42.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:20:42 smithi165 ceph-mon[36302]: pgmap v5448: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:20:42.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:20:42 smithi119 ceph-mon[31518]: pgmap v5448: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:20:44.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:20:44 smithi132 ceph-mon[35941]: pgmap v5449: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:20:44.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:20:44 smithi165 ceph-mon[36302]: pgmap v5449: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:20:44.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:20:44 smithi119 ceph-mon[31518]: pgmap v5449: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:20:46.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:20:46 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9905.801605 secs 2024-03-28T03:20:46.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:20:46 smithi165 ceph-mon[36302]: pgmap v5450: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:20:46.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:20:46 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9905.801605 secs 2024-03-28T03:20:46.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:20:46 smithi119 ceph-mon[31518]: pgmap v5450: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:20:46.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:20:46 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9905.801605 secs 2024-03-28T03:20:46.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:20:46 smithi132 ceph-mon[35941]: pgmap v5450: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:20:48.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:20:48 smithi165 ceph-mon[36302]: pgmap v5451: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:20:48.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:20:48 smithi119 ceph-mon[31518]: pgmap v5451: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:20:48.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:20:48 smithi132 ceph-mon[35941]: pgmap v5451: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:20:50.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:20:50 smithi165 ceph-mon[36302]: pgmap v5452: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:20:50.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:20:50 smithi119 ceph-mon[31518]: pgmap v5452: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:20:50.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:20:50 smithi132 ceph-mon[35941]: pgmap v5452: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:20:51.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:20:51 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9910.801695 secs 2024-03-28T03:20:51.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:20:51 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9910.801695 secs 2024-03-28T03:20:51.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:20:51 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9910.801695 secs 2024-03-28T03:20:52.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:20:52 smithi165 ceph-mon[36302]: pgmap v5453: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:20:52.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:20:52 smithi119 ceph-mon[31518]: pgmap v5453: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:20:52.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:20:52 smithi132 ceph-mon[35941]: pgmap v5453: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:20:54.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:20:54 smithi165 ceph-mon[36302]: pgmap v5454: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:20:54.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:20:54 smithi119 ceph-mon[31518]: pgmap v5454: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:20:54.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:20:54 smithi132 ceph-mon[35941]: pgmap v5454: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:20:56.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:20:56 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9915.801756 secs 2024-03-28T03:20:56.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:20:56 smithi165 ceph-mon[36302]: pgmap v5455: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:20:56.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:20:56 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9915.801756 secs 2024-03-28T03:20:56.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:20:56 smithi119 ceph-mon[31518]: pgmap v5455: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:20:56.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:20:56 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9915.801756 secs 2024-03-28T03:20:56.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:20:56 smithi132 ceph-mon[35941]: pgmap v5455: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:20:58.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:20:58 smithi165 ceph-mon[36302]: pgmap v5456: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:20:58.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:20:58 smithi119 ceph-mon[31518]: pgmap v5456: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:20:58.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:20:58 smithi132 ceph-mon[35941]: pgmap v5456: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:21:00.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:21:00 smithi165 ceph-mon[36302]: pgmap v5457: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:21:00.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:21:00 smithi119 ceph-mon[31518]: pgmap v5457: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:21:00.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:21:00 smithi132 ceph-mon[35941]: pgmap v5457: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:21:01.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:21:01 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9920.801771 secs 2024-03-28T03:21:01.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:21:01 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9920.801771 secs 2024-03-28T03:21:01.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:21:01 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9920.801771 secs 2024-03-28T03:21:02.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:21:02 smithi165 ceph-mon[36302]: pgmap v5458: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:21:02.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:21:02 smithi119 ceph-mon[31518]: pgmap v5458: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:21:02.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:21:02 smithi132 ceph-mon[35941]: pgmap v5458: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:21:04.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:21:04 smithi165 ceph-mon[36302]: pgmap v5459: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:21:04.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:21:04 smithi119 ceph-mon[31518]: pgmap v5459: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:21:04.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:21:04 smithi132 ceph-mon[35941]: pgmap v5459: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:21:06.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:21:06 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9925.801856 secs 2024-03-28T03:21:06.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:21:06 smithi165 ceph-mon[36302]: pgmap v5460: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:21:06.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:21:06 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9925.801856 secs 2024-03-28T03:21:06.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:21:06 smithi119 ceph-mon[31518]: pgmap v5460: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:21:06.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:21:06 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9925.801856 secs 2024-03-28T03:21:06.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:21:06 smithi132 ceph-mon[35941]: pgmap v5460: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:21:08.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:21:08 smithi165 ceph-mon[36302]: pgmap v5461: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:21:08.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:21:08 smithi119 ceph-mon[31518]: pgmap v5461: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:21:08.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:21:08 smithi132 ceph-mon[35941]: pgmap v5461: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:21:10.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:21:10 smithi165 ceph-mon[36302]: pgmap v5462: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:21:10.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:21:10 smithi119 ceph-mon[31518]: pgmap v5462: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:21:10.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:21:10 smithi132 ceph-mon[35941]: pgmap v5462: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:21:11.851 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:21:11 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9930.801943 secs 2024-03-28T03:21:11.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:21:11 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9930.801943 secs 2024-03-28T03:21:11.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:21:11 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9930.801943 secs 2024-03-28T03:21:12.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:21:12 smithi165 ceph-mon[36302]: pgmap v5463: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:21:12.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:21:12 smithi119 ceph-mon[31518]: pgmap v5463: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:21:12.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:21:12 smithi132 ceph-mon[35941]: pgmap v5463: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:21:14.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:21:14 smithi165 ceph-mon[36302]: pgmap v5464: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:21:14.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:21:14 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T03:21:14.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:21:14 smithi119 ceph-mon[31518]: pgmap v5464: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:21:14.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:21:14 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T03:21:14.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:21:14 smithi132 ceph-mon[35941]: pgmap v5464: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:21:14.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:21:14 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T03:21:15.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:21:15 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T03:21:15.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:21:15 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T03:21:15.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:21:15 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:21:15.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:21:15 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T03:21:15.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:21:15 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T03:21:15.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:21:15 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:21:15.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:21:15 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T03:21:15.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:21:15 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T03:21:15.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:21:15 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:21:16.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:21:16 smithi165 ceph-mon[36302]: pgmap v5465: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T03:21:16.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:21:16 smithi165 ceph-mon[36302]: pgmap v5466: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:21:16.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:21:16 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9935.802005 secs 2024-03-28T03:21:16.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:21:16 smithi119 ceph-mon[31518]: pgmap v5465: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T03:21:16.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:21:16 smithi119 ceph-mon[31518]: pgmap v5466: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:21:16.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:21:16 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9935.802005 secs 2024-03-28T03:21:16.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:21:16 smithi132 ceph-mon[35941]: pgmap v5465: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T03:21:16.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:21:16 smithi132 ceph-mon[35941]: pgmap v5466: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:21:16.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:21:16 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9935.802005 secs 2024-03-28T03:21:18.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:21:18 smithi165 ceph-mon[36302]: pgmap v5467: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:21:18.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:21:18 smithi119 ceph-mon[31518]: pgmap v5467: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:21:18.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:21:18 smithi132 ceph-mon[35941]: pgmap v5467: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:21:20.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:21:20 smithi165 ceph-mon[36302]: pgmap v5468: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-28T03:21:20.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:21:20 smithi119 ceph-mon[31518]: pgmap v5468: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-28T03:21:20.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:21:20 smithi132 ceph-mon[35941]: pgmap v5468: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-28T03:21:21.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:21:21 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9940.802095 secs 2024-03-28T03:21:21.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:21:21 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9940.802095 secs 2024-03-28T03:21:21.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:21:21 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9940.802095 secs 2024-03-28T03:21:22.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:21:22 smithi165 ceph-mon[36302]: pgmap v5469: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:21:22.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:21:22 smithi119 ceph-mon[31518]: pgmap v5469: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:21:22.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:21:22 smithi132 ceph-mon[35941]: pgmap v5469: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:21:24.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:21:24 smithi165 ceph-mon[36302]: pgmap v5470: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:21:24.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:21:24 smithi119 ceph-mon[31518]: pgmap v5470: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:21:24.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:21:24 smithi132 ceph-mon[35941]: pgmap v5470: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:21:26.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:21:26 smithi165 ceph-mon[36302]: pgmap v5471: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T03:21:26.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:21:26 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9945.802157 secs 2024-03-28T03:21:26.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:21:26 smithi119 ceph-mon[31518]: pgmap v5471: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T03:21:26.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:21:26 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9945.802157 secs 2024-03-28T03:21:26.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:21:26 smithi132 ceph-mon[35941]: pgmap v5471: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T03:21:26.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:21:26 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9945.802157 secs 2024-03-28T03:21:28.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:21:28 smithi165 ceph-mon[36302]: pgmap v5472: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:21:28.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:21:28 smithi119 ceph-mon[31518]: pgmap v5472: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:21:28.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:21:28 smithi132 ceph-mon[35941]: pgmap v5472: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:21:30.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:21:30 smithi165 ceph-mon[36302]: pgmap v5473: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:21:30.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:21:30 smithi119 ceph-mon[31518]: pgmap v5473: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:21:30.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:21:30 smithi132 ceph-mon[35941]: pgmap v5473: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:21:31.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:21:31 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9950.802173 secs 2024-03-28T03:21:31.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:21:31 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9950.802173 secs 2024-03-28T03:21:31.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:21:31 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9950.802173 secs 2024-03-28T03:21:32.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:21:32 smithi165 ceph-mon[36302]: pgmap v5474: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:21:32.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:21:32 smithi119 ceph-mon[31518]: pgmap v5474: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:21:32.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:21:32 smithi132 ceph-mon[35941]: pgmap v5474: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:21:34.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:21:34 smithi165 ceph-mon[36302]: pgmap v5475: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:21:34.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:21:34 smithi119 ceph-mon[31518]: pgmap v5475: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:21:34.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:21:34 smithi132 ceph-mon[35941]: pgmap v5475: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:21:36.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:21:36 smithi165 ceph-mon[36302]: pgmap v5476: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:21:36.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:21:36 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9955.802235 secs 2024-03-28T03:21:36.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:21:36 smithi119 ceph-mon[31518]: pgmap v5476: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:21:36.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:21:36 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9955.802235 secs 2024-03-28T03:21:36.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:21:36 smithi132 ceph-mon[35941]: pgmap v5476: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:21:36.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:21:36 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9955.802235 secs 2024-03-28T03:21:38.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:21:38 smithi165 ceph-mon[36302]: pgmap v5477: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:21:38.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:21:38 smithi119 ceph-mon[31518]: pgmap v5477: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:21:38.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:21:38 smithi132 ceph-mon[35941]: pgmap v5477: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:21:40.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:21:40 smithi165 ceph-mon[36302]: pgmap v5478: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:21:40.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:21:40 smithi119 ceph-mon[31518]: pgmap v5478: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:21:40.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:21:40 smithi132 ceph-mon[35941]: pgmap v5478: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:21:41.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:21:41 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9960.802297 secs 2024-03-28T03:21:41.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:21:41 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9960.802297 secs 2024-03-28T03:21:41.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:21:41 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9960.802297 secs 2024-03-28T03:21:42.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:21:42 smithi165 ceph-mon[36302]: pgmap v5479: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:21:42.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:21:42 smithi119 ceph-mon[31518]: pgmap v5479: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:21:42.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:21:42 smithi132 ceph-mon[35941]: pgmap v5479: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:21:44.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:21:44 smithi119 ceph-mon[31518]: pgmap v5480: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:21:44.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:21:44 smithi132 ceph-mon[35941]: pgmap v5480: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:21:45.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:21:44 smithi165 ceph-mon[36302]: pgmap v5480: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:21:46.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:21:46 smithi119 ceph-mon[31518]: pgmap v5481: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:21:46.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:21:46 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9965.802360 secs 2024-03-28T03:21:46.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:21:46 smithi132 ceph-mon[35941]: pgmap v5481: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:21:46.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:21:46 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9965.802360 secs 2024-03-28T03:21:47.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:21:46 smithi165 ceph-mon[36302]: pgmap v5481: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:21:47.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:21:46 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9965.802360 secs 2024-03-28T03:21:48.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:21:48 smithi119 ceph-mon[31518]: pgmap v5482: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:21:48.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:21:48 smithi132 ceph-mon[35941]: pgmap v5482: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:21:49.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:21:48 smithi165 ceph-mon[36302]: pgmap v5482: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:21:50.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:21:50 smithi119 ceph-mon[31518]: pgmap v5483: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:21:50.931 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:21:50 smithi165 ceph-mon[36302]: pgmap v5483: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:21:50.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:21:50 smithi132 ceph-mon[35941]: pgmap v5483: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:21:51.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:21:51 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9970.802399 secs 2024-03-28T03:21:51.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:21:51 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9970.802399 secs 2024-03-28T03:21:52.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:21:51 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9970.802399 secs 2024-03-28T03:21:52.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:21:52 smithi132 ceph-mon[35941]: pgmap v5484: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:21:53.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:21:52 smithi165 ceph-mon[36302]: pgmap v5484: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:21:53.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:21:52 smithi119 ceph-mon[31518]: pgmap v5484: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:21:54.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:21:54 smithi132 ceph-mon[35941]: pgmap v5485: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:21:54.955 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:21:54 smithi119 ceph-mon[31518]: pgmap v5485: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:21:55.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:21:54 smithi165 ceph-mon[36302]: pgmap v5485: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:21:56.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:21:56 smithi132 ceph-mon[35941]: pgmap v5486: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:21:56.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:21:56 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9975.802483 secs 2024-03-28T03:21:57.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:21:56 smithi165 ceph-mon[36302]: pgmap v5486: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:21:57.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:21:56 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9975.802483 secs 2024-03-28T03:21:57.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:21:56 smithi119 ceph-mon[31518]: pgmap v5486: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:21:57.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:21:56 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9975.802483 secs 2024-03-28T03:21:58.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:21:58 smithi132 ceph-mon[35941]: pgmap v5487: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:21:59.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:21:58 smithi165 ceph-mon[36302]: pgmap v5487: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:21:59.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:21:58 smithi119 ceph-mon[31518]: pgmap v5487: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:22:00.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:22:00 smithi132 ceph-mon[35941]: pgmap v5488: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:22:01.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:22:00 smithi165 ceph-mon[36302]: pgmap v5488: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:22:01.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:22:00 smithi119 ceph-mon[31518]: pgmap v5488: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:22:01.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:22:01 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9980.802572 secs 2024-03-28T03:22:02.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:22:01 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9980.802572 secs 2024-03-28T03:22:02.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:22:01 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9980.802572 secs 2024-03-28T03:22:02.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:22:02 smithi132 ceph-mon[35941]: pgmap v5489: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:22:03.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:22:02 smithi165 ceph-mon[36302]: pgmap v5489: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:22:03.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:22:02 smithi119 ceph-mon[31518]: pgmap v5489: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:22:04.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:22:04 smithi132 ceph-mon[35941]: pgmap v5490: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:22:05.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:22:04 smithi165 ceph-mon[36302]: pgmap v5490: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:22:05.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:22:04 smithi119 ceph-mon[31518]: pgmap v5490: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:22:07.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:22:06 smithi165 ceph-mon[36302]: pgmap v5491: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:22:07.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:22:06 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9985.802661 secs 2024-03-28T03:22:07.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:22:06 smithi119 ceph-mon[31518]: pgmap v5491: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:22:07.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:22:06 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9985.802661 secs 2024-03-28T03:22:07.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:22:06 smithi132 ceph-mon[35941]: pgmap v5491: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:22:07.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:22:06 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9985.802661 secs 2024-03-28T03:22:09.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:22:08 smithi165 ceph-mon[36302]: pgmap v5492: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:22:09.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:22:08 smithi119 ceph-mon[31518]: pgmap v5492: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:22:09.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:22:08 smithi132 ceph-mon[35941]: pgmap v5492: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:22:11.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:22:10 smithi165 ceph-mon[36302]: pgmap v5493: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:22:11.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:22:10 smithi119 ceph-mon[31518]: pgmap v5493: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:22:11.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:22:10 smithi132 ceph-mon[35941]: pgmap v5493: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:22:12.101 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:22:11 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9990.802753 secs 2024-03-28T03:22:12.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:22:11 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9990.802753 secs 2024-03-28T03:22:12.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:22:11 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9990.802753 secs 2024-03-28T03:22:13.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:22:12 smithi165 ceph-mon[36302]: pgmap v5494: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:22:13.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:22:12 smithi119 ceph-mon[31518]: pgmap v5494: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:22:13.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:22:12 smithi132 ceph-mon[35941]: pgmap v5494: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:22:15.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:22:14 smithi165 ceph-mon[36302]: pgmap v5495: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:22:15.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:22:14 smithi119 ceph-mon[31518]: pgmap v5495: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:22:15.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:22:14 smithi132 ceph-mon[35941]: pgmap v5495: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:22:16.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:22:15 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T03:22:16.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:22:15 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T03:22:16.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:22:15 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T03:22:17.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:22:16 smithi165 ceph-mon[36302]: pgmap v5496: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:22:17.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:22:16 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 9995.802815 secs 2024-03-28T03:22:17.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:22:16 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T03:22:17.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:22:16 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T03:22:17.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:22:16 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:22:17.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:22:16 smithi119 ceph-mon[31518]: pgmap v5496: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:22:17.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:22:16 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 9995.802815 secs 2024-03-28T03:22:17.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:22:16 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T03:22:17.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:22:16 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T03:22:17.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:22:16 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:22:17.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:22:16 smithi132 ceph-mon[35941]: pgmap v5496: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:22:17.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:22:16 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 9995.802815 secs 2024-03-28T03:22:17.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:22:16 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T03:22:17.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:22:16 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T03:22:17.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:22:16 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:22:18.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:22:17 smithi165 ceph-mon[36302]: pgmap v5497: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T03:22:18.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:22:17 smithi119 ceph-mon[31518]: pgmap v5497: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T03:22:18.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:22:17 smithi132 ceph-mon[35941]: pgmap v5497: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T03:22:20.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:22:19 smithi165 ceph-mon[36302]: pgmap v5498: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T03:22:20.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:22:19 smithi119 ceph-mon[31518]: pgmap v5498: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T03:22:20.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:22:19 smithi132 ceph-mon[35941]: pgmap v5498: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T03:22:21.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:22:20 smithi165 ceph-mon[36302]: pgmap v5499: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T03:22:21.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:22:20 smithi119 ceph-mon[31518]: pgmap v5499: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T03:22:21.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:22:20 smithi132 ceph-mon[35941]: pgmap v5499: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T03:22:22.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:22:21 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10000.802916 secs 2024-03-28T03:22:22.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:22:21 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10000.802916 secs 2024-03-28T03:22:22.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:22:21 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10000.802916 secs 2024-03-28T03:22:23.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:22:22 smithi165 ceph-mon[36302]: pgmap v5500: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T03:22:23.138 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:22:22 smithi119 ceph-mon[31518]: pgmap v5500: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T03:22:23.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:22:22 smithi132 ceph-mon[35941]: pgmap v5500: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T03:22:25.843 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:22:25 smithi119 ceph-mon[31518]: pgmap v5501: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T03:22:25.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:22:25 smithi165 ceph-mon[36302]: pgmap v5501: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T03:22:25.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:22:25 smithi132 ceph-mon[35941]: pgmap v5501: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T03:22:26.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:22:26 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10005.802931 secs 2024-03-28T03:22:26.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:22:26 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10005.802931 secs 2024-03-28T03:22:26.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:22:26 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10005.802931 secs 2024-03-28T03:22:27.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:22:27 smithi165 ceph-mon[36302]: pgmap v5502: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T03:22:27.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:22:27 smithi119 ceph-mon[31518]: pgmap v5502: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T03:22:27.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:22:27 smithi132 ceph-mon[35941]: pgmap v5502: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T03:22:29.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:22:29 smithi165 ceph-mon[36302]: pgmap v5503: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:22:29.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:22:29 smithi119 ceph-mon[31518]: pgmap v5503: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:22:29.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:22:29 smithi132 ceph-mon[35941]: pgmap v5503: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:22:31.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:22:31 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10010.802991 secs 2024-03-28T03:22:31.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:22:31 smithi165 ceph-mon[36302]: pgmap v5504: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:22:31.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:22:31 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10010.802991 secs 2024-03-28T03:22:31.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:22:31 smithi119 ceph-mon[31518]: pgmap v5504: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:22:31.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:22:31 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10010.802991 secs 2024-03-28T03:22:31.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:22:31 smithi132 ceph-mon[35941]: pgmap v5504: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:22:33.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:22:33 smithi165 ceph-mon[36302]: pgmap v5505: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:22:33.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:22:33 smithi119 ceph-mon[31518]: pgmap v5505: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:22:33.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:22:33 smithi132 ceph-mon[35941]: pgmap v5505: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:22:35.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:22:35 smithi165 ceph-mon[36302]: pgmap v5506: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:22:35.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:22:35 smithi119 ceph-mon[31518]: pgmap v5506: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:22:35.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:22:35 smithi132 ceph-mon[35941]: pgmap v5506: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:22:36.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:22:36 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10015.803050 secs 2024-03-28T03:22:36.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:22:36 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10015.803050 secs 2024-03-28T03:22:36.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:22:36 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10015.803050 secs 2024-03-28T03:22:37.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:22:37 smithi165 ceph-mon[36302]: pgmap v5507: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:22:37.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:22:37 smithi119 ceph-mon[31518]: pgmap v5507: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:22:37.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:22:37 smithi132 ceph-mon[35941]: pgmap v5507: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:22:39.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:22:39 smithi165 ceph-mon[36302]: pgmap v5508: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:22:39.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:22:39 smithi119 ceph-mon[31518]: pgmap v5508: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:22:39.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:22:39 smithi132 ceph-mon[35941]: pgmap v5508: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:22:41.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:22:41 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10020.803110 secs 2024-03-28T03:22:41.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:22:41 smithi165 ceph-mon[36302]: pgmap v5509: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:22:41.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:22:41 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10020.803110 secs 2024-03-28T03:22:41.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:22:41 smithi119 ceph-mon[31518]: pgmap v5509: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:22:41.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:22:41 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10020.803110 secs 2024-03-28T03:22:41.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:22:41 smithi132 ceph-mon[35941]: pgmap v5509: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:22:43.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:22:43 smithi165 ceph-mon[36302]: pgmap v5510: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:22:43.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:22:43 smithi119 ceph-mon[31518]: pgmap v5510: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:22:43.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:22:43 smithi132 ceph-mon[35941]: pgmap v5510: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:22:45.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:22:45 smithi165 ceph-mon[36302]: pgmap v5511: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:22:45.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:22:45 smithi119 ceph-mon[31518]: pgmap v5511: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:22:45.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:22:45 smithi132 ceph-mon[35941]: pgmap v5511: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:22:46.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:22:46 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10025.803201 secs 2024-03-28T03:22:46.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:22:46 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10025.803201 secs 2024-03-28T03:22:46.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:22:46 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10025.803201 secs 2024-03-28T03:22:47.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:22:47 smithi165 ceph-mon[36302]: pgmap v5512: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:22:47.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:22:47 smithi119 ceph-mon[31518]: pgmap v5512: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:22:47.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:22:47 smithi132 ceph-mon[35941]: pgmap v5512: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:22:49.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:22:49 smithi165 ceph-mon[36302]: pgmap v5513: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:22:49.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:22:49 smithi119 ceph-mon[31518]: pgmap v5513: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:22:49.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:22:49 smithi132 ceph-mon[35941]: pgmap v5513: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:22:51.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:22:51 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10030.803292 secs 2024-03-28T03:22:51.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:22:51 smithi165 ceph-mon[36302]: pgmap v5514: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:22:51.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:22:51 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10030.803292 secs 2024-03-28T03:22:51.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:22:51 smithi119 ceph-mon[31518]: pgmap v5514: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:22:51.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:22:51 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10030.803292 secs 2024-03-28T03:22:51.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:22:51 smithi132 ceph-mon[35941]: pgmap v5514: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:22:53.859 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:22:53 smithi119 ceph-mon[31518]: pgmap v5515: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:22:53.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:22:53 smithi165 ceph-mon[36302]: pgmap v5515: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:22:53.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:22:53 smithi132 ceph-mon[35941]: pgmap v5515: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:22:55.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:22:55 smithi165 ceph-mon[36302]: pgmap v5516: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:22:55.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:22:55 smithi119 ceph-mon[31518]: pgmap v5516: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:22:55.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:22:55 smithi132 ceph-mon[35941]: pgmap v5516: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:22:56.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:22:56 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10035.803381 secs 2024-03-28T03:22:56.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:22:56 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10035.803381 secs 2024-03-28T03:22:56.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:22:56 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10035.803381 secs 2024-03-28T03:22:57.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:22:57 smithi165 ceph-mon[36302]: pgmap v5517: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:22:57.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:22:57 smithi119 ceph-mon[31518]: pgmap v5517: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:22:57.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:22:57 smithi132 ceph-mon[35941]: pgmap v5517: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:22:59.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:22:59 smithi165 ceph-mon[36302]: pgmap v5518: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:22:59.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:22:59 smithi119 ceph-mon[31518]: pgmap v5518: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:22:59.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:22:59 smithi132 ceph-mon[35941]: pgmap v5518: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:23:01.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:23:01 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10040.803469 secs 2024-03-28T03:23:01.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:23:01 smithi165 ceph-mon[36302]: pgmap v5519: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:23:01.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:23:01 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10040.803469 secs 2024-03-28T03:23:01.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:23:01 smithi119 ceph-mon[31518]: pgmap v5519: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:23:01.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:23:01 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10040.803469 secs 2024-03-28T03:23:01.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:23:01 smithi132 ceph-mon[35941]: pgmap v5519: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:23:03.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:23:03 smithi165 ceph-mon[36302]: pgmap v5520: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:23:03.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:23:03 smithi119 ceph-mon[31518]: pgmap v5520: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:23:03.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:23:03 smithi132 ceph-mon[35941]: pgmap v5520: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:23:05.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:23:05 smithi165 ceph-mon[36302]: pgmap v5521: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:23:05.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:23:05 smithi119 ceph-mon[31518]: pgmap v5521: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:23:05.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:23:05 smithi132 ceph-mon[35941]: pgmap v5521: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:23:07.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:23:06 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10045.803558 secs 2024-03-28T03:23:07.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:23:06 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10045.803558 secs 2024-03-28T03:23:07.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:23:06 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10045.803558 secs 2024-03-28T03:23:08.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:23:07 smithi165 ceph-mon[36302]: pgmap v5522: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:23:08.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:23:07 smithi119 ceph-mon[31518]: pgmap v5522: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:23:08.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:23:07 smithi132 ceph-mon[35941]: pgmap v5522: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:23:10.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:23:09 smithi165 ceph-mon[36302]: pgmap v5523: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:23:10.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:23:09 smithi119 ceph-mon[31518]: pgmap v5523: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:23:10.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:23:09 smithi132 ceph-mon[35941]: pgmap v5523: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:23:11.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:23:10 smithi165 ceph-mon[36302]: pgmap v5524: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:23:11.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:23:10 smithi119 ceph-mon[31518]: pgmap v5524: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:23:11.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:23:10 smithi132 ceph-mon[35941]: pgmap v5524: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:23:12.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:23:11 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10050.803599 secs 2024-03-28T03:23:12.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:23:11 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10050.803599 secs 2024-03-28T03:23:12.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:23:11 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10050.803599 secs 2024-03-28T03:23:13.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:23:12 smithi165 ceph-mon[36302]: pgmap v5525: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:23:13.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:23:12 smithi119 ceph-mon[31518]: pgmap v5525: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:23:13.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:23:12 smithi132 ceph-mon[35941]: pgmap v5525: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:23:15.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:23:15 smithi165 ceph-mon[36302]: pgmap v5526: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:23:15.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:23:15 smithi119 ceph-mon[31518]: pgmap v5526: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:23:15.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:23:15 smithi132 ceph-mon[35941]: pgmap v5526: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:23:16.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:23:16 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10055.803639 secs 2024-03-28T03:23:16.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:23:16 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10055.803639 secs 2024-03-28T03:23:16.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:23:16 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10055.803639 secs 2024-03-28T03:23:17.755 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:23:17 smithi165 ceph-mon[36302]: pgmap v5527: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:23:17.755 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:23:17 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T03:23:17.755 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:23:17 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:23:17.755 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:23:17 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:23:17.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:23:17 smithi119 ceph-mon[31518]: pgmap v5527: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:23:17.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:23:17 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T03:23:17.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:23:17 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:23:17.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:23:17 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:23:17.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:23:17 smithi132 ceph-mon[35941]: pgmap v5527: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:23:17.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:23:17 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T03:23:17.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:23:17 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:23:17.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:23:17 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:23:18.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:23:18 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:23:18.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:23:18 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:23:18.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:23:18 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:23:18.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:23:18 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:23:18.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:23:18 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:23:18.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:23:18 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:23:19.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:23:19 smithi165 ceph-mon[36302]: pgmap v5528: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:23:19.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:23:19 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:23:19.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:23:19 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:23:19.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:23:19 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:23:19.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:23:19 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:23:19.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:23:19 smithi119 ceph-mon[31518]: pgmap v5528: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:23:19.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:23:19 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:23:19.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:23:19 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:23:19.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:23:19 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:23:19.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:23:19 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:23:19.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:23:19 smithi132 ceph-mon[35941]: pgmap v5528: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:23:19.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:23:19 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:23:19.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:23:19 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:23:19.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:23:19 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:23:19.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:23:19 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:23:20.608 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:23:20 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:23:20.609 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:23:20 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:23:20.609 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:23:20 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:23:20.609 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:23:20 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:23:20.609 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:23:20 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:23:20.609 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:23:20 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:23:20.609 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:23:20 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:23:20.609 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:23:20 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:23:20.609 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:23:20 smithi132 ceph-mon[35941]: Adjusting osd_memory_target on smithi165 to 174.1M 2024-03-28T03:23:20.609 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:23:20 smithi132 ceph-mon[35941]: Unable to set osd_memory_target on smithi165 to 182635622: error parsing value: Value '182635622' is below minimum 939524096 2024-03-28T03:23:20.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:23:20 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:23:20.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:23:20 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:23:20.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:23:20 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:23:20.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:23:20 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:23:20.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:23:20 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:23:20.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:23:20 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:23:20.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:23:20 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:23:20.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:23:20 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:23:20.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:23:20 smithi165 ceph-mon[36302]: Adjusting osd_memory_target on smithi165 to 174.1M 2024-03-28T03:23:20.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:23:20 smithi165 ceph-mon[36302]: Unable to set osd_memory_target on smithi165 to 182635622: error parsing value: Value '182635622' is below minimum 939524096 2024-03-28T03:23:20.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:23:20 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:23:20.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:23:20 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:23:20.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:23:20 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:23:20.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:23:20 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:23:20.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:23:20 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:23:20.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:23:20 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:23:20.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:23:20 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:23:20.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:23:20 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:23:20.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:23:20 smithi119 ceph-mon[31518]: Adjusting osd_memory_target on smithi165 to 174.1M 2024-03-28T03:23:20.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:23:20 smithi119 ceph-mon[31518]: Unable to set osd_memory_target on smithi165 to 182635622: error parsing value: Value '182635622' is below minimum 939524096 2024-03-28T03:23:21.663 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:23:21 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10060.803728 secs 2024-03-28T03:23:21.663 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:23:21 smithi132 ceph-mon[35941]: pgmap v5529: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:23:21.663 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:23:21 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:23:21.663 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:23:21 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:23:21.780 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:23:21 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10060.803728 secs 2024-03-28T03:23:21.781 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:23:21 smithi165 ceph-mon[36302]: pgmap v5529: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:23:21.781 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:23:21 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:23:21.781 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:23:21 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:23:21.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:23:21 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10060.803728 secs 2024-03-28T03:23:21.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:23:21 smithi119 ceph-mon[31518]: pgmap v5529: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:23:21.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:23:21 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:23:21.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:23:21 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:23:23.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:23:23 smithi119 ceph-mon[31518]: pgmap v5530: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:23:23.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:23:23 smithi165 ceph-mon[36302]: pgmap v5530: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:23:23.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:23:23 smithi132 ceph-mon[35941]: pgmap v5530: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:23:24.862 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:23:24 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:23:24.862 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:23:24 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:23:24.862 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:23:24 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:23:24.862 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:23:24 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:23:24.862 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:23:24 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:23:24.862 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:23:24 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:23:24.862 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:23:24 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:23:24.863 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:23:24 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:23:24.863 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:23:24 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:23:24.863 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:23:24 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:23:24.863 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:23:24 smithi119 ceph-mon[31518]: Adjusting osd_memory_target on smithi132 to 174.1M 2024-03-28T03:23:24.863 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:23:24 smithi119 ceph-mon[31518]: Unable to set osd_memory_target on smithi132 to 182640640: error parsing value: Value '182640640' is below minimum 939524096 2024-03-28T03:23:24.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:23:24 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:23:24.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:23:24 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:23:24.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:23:24 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:23:24.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:23:24 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:23:24.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:23:24 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:23:24.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:23:24 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:23:24.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:23:24 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:23:24.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:23:24 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:23:24.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:23:24 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:23:24.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:23:24 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:23:24.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:23:24 smithi165 ceph-mon[36302]: Adjusting osd_memory_target on smithi132 to 174.1M 2024-03-28T03:23:24.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:23:24 smithi165 ceph-mon[36302]: Unable to set osd_memory_target on smithi132 to 182640640: error parsing value: Value '182640640' is below minimum 939524096 2024-03-28T03:23:24.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:23:24 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:23:24.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:23:24 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:23:24.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:23:24 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:23:24.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:23:24 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:23:24.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:23:24 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:23:24.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:23:24 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:23:24.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:23:24 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:23:24.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:23:24 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:23:24.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:23:24 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:23:24.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:23:24 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:23:24.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:23:24 smithi132 ceph-mon[35941]: Adjusting osd_memory_target on smithi132 to 174.1M 2024-03-28T03:23:24.951 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:23:24 smithi132 ceph-mon[35941]: Unable to set osd_memory_target on smithi132 to 182640640: error parsing value: Value '182640640' is below minimum 939524096 2024-03-28T03:23:25.777 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:23:25 smithi119 ceph-mon[31518]: pgmap v5531: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:23:25.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:23:25 smithi165 ceph-mon[36302]: pgmap v5531: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:23:25.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:23:25 smithi132 ceph-mon[35941]: pgmap v5531: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:23:26.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:23:26 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10065.803811 secs 2024-03-28T03:23:26.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:23:26 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10065.803811 secs 2024-03-28T03:23:26.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:23:26 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10065.803811 secs 2024-03-28T03:23:27.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:23:27 smithi165 ceph-mon[36302]: pgmap v5532: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:23:27.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:23:27 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:23:27.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:23:27 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:23:27.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:23:27 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:23:27.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:23:27 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:23:27.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:23:27 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:23:27.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:23:27 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:23:27.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:23:27 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:23:27.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:23:27 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:23:27.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:23:27 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:23:27.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:23:27 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:23:27.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:23:27 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T03:23:27.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:23:27 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T03:23:27.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:23:27 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:23:27.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:23:27 smithi119 ceph-mon[31518]: pgmap v5532: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:23:27.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:23:27 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:23:27.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:23:27 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:23:27.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:23:27 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:23:27.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:23:27 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:23:27.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:23:27 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:23:27.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:23:27 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:23:27.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:23:27 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:23:27.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:23:27 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:23:27.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:23:27 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:23:27.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:23:27 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:23:27.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:23:27 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T03:23:27.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:23:27 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T03:23:27.904 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:23:27 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:23:27.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:23:27 smithi132 ceph-mon[35941]: pgmap v5532: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:23:27.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:23:27 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:23:27.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:23:27 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:23:27.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:23:27 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:23:27.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:23:27 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:23:27.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:23:27 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:23:27.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:23:27 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:23:27.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:23:27 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:23:27.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:23:27 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:23:27.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:23:27 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:23:27.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:23:27 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:23:27.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:23:27 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T03:23:27.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:23:27 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T03:23:27.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:23:27 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:23:28.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:23:28 smithi165 ceph-mon[36302]: Adjusting osd_memory_target on smithi119 to 176.7M 2024-03-28T03:23:28.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:23:28 smithi165 ceph-mon[36302]: Unable to set osd_memory_target on smithi119 to 185372364: error parsing value: Value '185372364' is below minimum 939524096 2024-03-28T03:23:28.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:23:28 smithi165 ceph-mon[36302]: pgmap v5533: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-28T03:23:28.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:23:28 smithi165 ceph-mon[36302]: pgmap v5534: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T03:23:28.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:23:28 smithi119 ceph-mon[31518]: Adjusting osd_memory_target on smithi119 to 176.7M 2024-03-28T03:23:28.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:23:28 smithi119 ceph-mon[31518]: Unable to set osd_memory_target on smithi119 to 185372364: error parsing value: Value '185372364' is below minimum 939524096 2024-03-28T03:23:28.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:23:28 smithi119 ceph-mon[31518]: pgmap v5533: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-28T03:23:28.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:23:28 smithi119 ceph-mon[31518]: pgmap v5534: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T03:23:28.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:23:28 smithi132 ceph-mon[35941]: Adjusting osd_memory_target on smithi119 to 176.7M 2024-03-28T03:23:28.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:23:28 smithi132 ceph-mon[35941]: Unable to set osd_memory_target on smithi119 to 185372364: error parsing value: Value '185372364' is below minimum 939524096 2024-03-28T03:23:28.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:23:28 smithi132 ceph-mon[35941]: pgmap v5533: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-28T03:23:28.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:23:28 smithi132 ceph-mon[35941]: pgmap v5534: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T03:23:30.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:23:30 smithi165 ceph-mon[36302]: pgmap v5535: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T03:23:30.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:23:30 smithi119 ceph-mon[31518]: pgmap v5535: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T03:23:30.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:23:30 smithi132 ceph-mon[35941]: pgmap v5535: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T03:23:31.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:23:31 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10070.803872 secs 2024-03-28T03:23:31.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:23:31 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10070.803872 secs 2024-03-28T03:23:31.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:23:31 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10070.803872 secs 2024-03-28T03:23:32.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:23:32 smithi165 ceph-mon[36302]: pgmap v5536: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T03:23:32.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:23:32 smithi119 ceph-mon[31518]: pgmap v5536: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T03:23:32.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:23:32 smithi132 ceph-mon[35941]: pgmap v5536: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T03:23:34.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:23:34 smithi119 ceph-mon[31518]: pgmap v5537: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-28T03:23:34.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:23:34 smithi132 ceph-mon[35941]: pgmap v5537: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-28T03:23:35.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:23:34 smithi165 ceph-mon[36302]: pgmap v5537: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-28T03:23:36.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:23:36 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10075.803958 secs 2024-03-28T03:23:36.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:23:36 smithi119 ceph-mon[31518]: pgmap v5538: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T03:23:36.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:23:36 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10075.803958 secs 2024-03-28T03:23:36.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:23:36 smithi132 ceph-mon[35941]: pgmap v5538: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T03:23:37.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:23:36 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10075.803958 secs 2024-03-28T03:23:37.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:23:36 smithi165 ceph-mon[36302]: pgmap v5538: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T03:23:38.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:23:38 smithi119 ceph-mon[31518]: pgmap v5539: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T03:23:38.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:23:38 smithi132 ceph-mon[35941]: pgmap v5539: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T03:23:39.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:23:38 smithi165 ceph-mon[36302]: pgmap v5539: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T03:23:40.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:23:40 smithi119 ceph-mon[31518]: pgmap v5540: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:23:40.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:23:40 smithi132 ceph-mon[35941]: pgmap v5540: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:23:41.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:23:40 smithi165 ceph-mon[36302]: pgmap v5540: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:23:41.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:23:41 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10080.804020 secs 2024-03-28T03:23:42.104 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:23:41 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10080.804020 secs 2024-03-28T03:23:42.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:23:41 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10080.804020 secs 2024-03-28T03:23:42.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:23:42 smithi132 ceph-mon[35941]: pgmap v5541: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:23:43.013 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:23:42 smithi119 ceph-mon[31518]: pgmap v5541: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:23:43.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:23:42 smithi165 ceph-mon[36302]: pgmap v5541: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:23:44.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:23:44 smithi132 ceph-mon[35941]: pgmap v5542: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:23:45.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:23:44 smithi165 ceph-mon[36302]: pgmap v5542: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:23:45.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:23:44 smithi119 ceph-mon[31518]: pgmap v5542: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:23:46.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:23:46 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10085.804090 secs 2024-03-28T03:23:46.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:23:46 smithi132 ceph-mon[35941]: pgmap v5543: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:23:47.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:23:46 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10085.804090 secs 2024-03-28T03:23:47.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:23:46 smithi165 ceph-mon[36302]: pgmap v5543: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:23:47.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:23:46 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10085.804090 secs 2024-03-28T03:23:47.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:23:46 smithi119 ceph-mon[31518]: pgmap v5543: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:23:48.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:23:48 smithi132 ceph-mon[35941]: pgmap v5544: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:23:49.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:23:48 smithi165 ceph-mon[36302]: pgmap v5544: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:23:49.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:23:48 smithi119 ceph-mon[31518]: pgmap v5544: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:23:50.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:23:50 smithi132 ceph-mon[35941]: pgmap v5545: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:23:51.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:23:50 smithi165 ceph-mon[36302]: pgmap v5545: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:23:51.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:23:50 smithi119 ceph-mon[31518]: pgmap v5545: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:23:51.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:23:51 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10090.804101 secs 2024-03-28T03:23:52.030 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:23:51 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10090.804101 secs 2024-03-28T03:23:52.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:23:51 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10090.804101 secs 2024-03-28T03:23:52.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:23:52 smithi119 ceph-mon[31518]: pgmap v5546: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:23:52.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:23:52 smithi132 ceph-mon[35941]: pgmap v5546: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:23:53.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:23:52 smithi165 ceph-mon[36302]: pgmap v5546: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:23:54.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:23:54 smithi132 ceph-mon[35941]: pgmap v5547: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:23:55.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:23:54 smithi165 ceph-mon[36302]: pgmap v5547: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:23:55.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:23:54 smithi119 ceph-mon[31518]: pgmap v5547: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:23:56.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:23:56 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10095.804165 secs 2024-03-28T03:23:56.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:23:56 smithi132 ceph-mon[35941]: pgmap v5548: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:23:57.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:23:56 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10095.804165 secs 2024-03-28T03:23:57.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:23:56 smithi165 ceph-mon[36302]: pgmap v5548: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:23:57.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:23:56 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10095.804165 secs 2024-03-28T03:23:57.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:23:56 smithi119 ceph-mon[31518]: pgmap v5548: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:23:59.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:23:58 smithi165 ceph-mon[36302]: pgmap v5549: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:23:59.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:23:58 smithi119 ceph-mon[31518]: pgmap v5549: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:23:59.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:23:58 smithi132 ceph-mon[35941]: pgmap v5549: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:24:01.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:24:00 smithi165 ceph-mon[36302]: pgmap v5550: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:24:01.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:24:00 smithi119 ceph-mon[31518]: pgmap v5550: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:24:01.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:24:00 smithi132 ceph-mon[35941]: pgmap v5550: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:24:02.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:24:01 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10100.804225 secs 2024-03-28T03:24:02.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:24:01 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10100.804225 secs 2024-03-28T03:24:02.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:24:01 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10100.804225 secs 2024-03-28T03:24:03.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:24:02 smithi165 ceph-mon[36302]: pgmap v5551: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:24:03.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:24:02 smithi119 ceph-mon[31518]: pgmap v5551: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:24:03.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:24:02 smithi132 ceph-mon[35941]: pgmap v5551: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:24:05.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:24:04 smithi165 ceph-mon[36302]: pgmap v5552: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:24:05.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:24:04 smithi119 ceph-mon[31518]: pgmap v5552: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:24:05.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:24:04 smithi132 ceph-mon[35941]: pgmap v5552: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:24:07.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:24:06 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10105.804314 secs 2024-03-28T03:24:07.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:24:06 smithi165 ceph-mon[36302]: pgmap v5553: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:24:07.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:24:06 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10105.804314 secs 2024-03-28T03:24:07.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:24:06 smithi119 ceph-mon[31518]: pgmap v5553: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:24:07.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:24:06 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10105.804314 secs 2024-03-28T03:24:07.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:24:06 smithi132 ceph-mon[35941]: pgmap v5553: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:24:09.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:24:08 smithi165 ceph-mon[36302]: pgmap v5554: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:24:09.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:24:08 smithi119 ceph-mon[31518]: pgmap v5554: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:24:09.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:24:08 smithi132 ceph-mon[35941]: pgmap v5554: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:24:11.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:24:10 smithi165 ceph-mon[36302]: pgmap v5555: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:24:11.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:24:10 smithi119 ceph-mon[31518]: pgmap v5555: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:24:11.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:24:10 smithi132 ceph-mon[35941]: pgmap v5555: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:24:12.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:24:11 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10110.804405 secs 2024-03-28T03:24:12.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:24:11 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10110.804405 secs 2024-03-28T03:24:12.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:24:11 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10110.804405 secs 2024-03-28T03:24:13.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:24:12 smithi165 ceph-mon[36302]: pgmap v5556: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:24:13.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:24:12 smithi119 ceph-mon[31518]: pgmap v5556: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:24:13.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:24:12 smithi132 ceph-mon[35941]: pgmap v5556: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:24:15.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:24:14 smithi165 ceph-mon[36302]: pgmap v5557: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:24:15.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:24:14 smithi119 ceph-mon[31518]: pgmap v5557: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:24:15.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:24:14 smithi132 ceph-mon[35941]: pgmap v5557: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:24:17.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:24:16 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10115.804495 secs 2024-03-28T03:24:17.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:24:16 smithi165 ceph-mon[36302]: pgmap v5558: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:24:17.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:24:16 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10115.804495 secs 2024-03-28T03:24:17.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:24:16 smithi119 ceph-mon[31518]: pgmap v5558: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:24:17.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:24:16 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10115.804495 secs 2024-03-28T03:24:17.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:24:16 smithi132 ceph-mon[35941]: pgmap v5558: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:24:19.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:24:18 smithi165 ceph-mon[36302]: pgmap v5559: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:24:19.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:24:18 smithi119 ceph-mon[31518]: pgmap v5559: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:24:19.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:24:18 smithi132 ceph-mon[35941]: pgmap v5559: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:24:21.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:24:20 smithi165 ceph-mon[36302]: pgmap v5560: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:24:21.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:24:20 smithi119 ceph-mon[31518]: pgmap v5560: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:24:21.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:24:20 smithi132 ceph-mon[35941]: pgmap v5560: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:24:22.031 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:24:21 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10120.804554 secs 2024-03-28T03:24:22.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:24:21 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10120.804554 secs 2024-03-28T03:24:22.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:24:21 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10120.804554 secs 2024-03-28T03:24:23.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:24:22 smithi165 ceph-mon[36302]: pgmap v5561: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:24:23.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:24:22 smithi119 ceph-mon[31518]: pgmap v5561: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:24:23.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:24:22 smithi132 ceph-mon[35941]: pgmap v5561: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:24:25.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:24:24 smithi165 ceph-mon[36302]: pgmap v5562: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:24:25.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:24:24 smithi119 ceph-mon[31518]: pgmap v5562: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:24:25.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:24:24 smithi132 ceph-mon[35941]: pgmap v5562: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:24:27.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:24:26 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10125.804642 secs 2024-03-28T03:24:27.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:24:26 smithi165 ceph-mon[36302]: pgmap v5563: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:24:27.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:24:26 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10125.804642 secs 2024-03-28T03:24:27.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:24:26 smithi119 ceph-mon[31518]: pgmap v5563: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:24:27.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:24:26 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10125.804642 secs 2024-03-28T03:24:27.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:24:26 smithi132 ceph-mon[35941]: pgmap v5563: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:24:29.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:24:28 smithi165 ceph-mon[36302]: pgmap v5564: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:24:29.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:24:28 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T03:24:29.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:24:28 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:24:29.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:24:28 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:24:29.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:24:28 smithi119 ceph-mon[31518]: pgmap v5564: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:24:29.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:24:28 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T03:24:29.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:24:28 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:24:29.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:24:28 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:24:29.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:24:28 smithi132 ceph-mon[35941]: pgmap v5564: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:24:29.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:24:28 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T03:24:29.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:24:28 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:24:29.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:24:28 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:24:31.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:24:30 smithi165 ceph-mon[36302]: pgmap v5565: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:24:31.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:24:30 smithi119 ceph-mon[31518]: pgmap v5565: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:24:31.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:24:30 smithi132 ceph-mon[35941]: pgmap v5565: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:24:32.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:24:31 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10130.804703 secs 2024-03-28T03:24:32.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:24:31 smithi165 ceph-mon[36302]: pgmap v5566: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:24:32.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:24:31 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10130.804703 secs 2024-03-28T03:24:32.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:24:31 smithi119 ceph-mon[31518]: pgmap v5566: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:24:32.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:24:31 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10130.804703 secs 2024-03-28T03:24:32.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:24:31 smithi132 ceph-mon[35941]: pgmap v5566: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:24:34.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:24:33 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:24:34.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:24:33 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:24:34.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:24:33 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T03:24:34.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:24:33 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T03:24:34.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:24:33 smithi132 ceph-mon[35941]: pgmap v5567: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T03:24:34.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:24:33 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:24:34.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:24:33 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:24:34.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:24:33 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:24:34.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:24:33 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T03:24:34.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:24:33 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T03:24:34.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:24:33 smithi165 ceph-mon[36302]: pgmap v5567: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T03:24:34.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:24:33 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:24:34.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:24:33 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:24:34.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:24:33 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:24:34.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:24:33 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T03:24:34.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:24:33 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T03:24:34.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:24:33 smithi119 ceph-mon[31518]: pgmap v5567: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T03:24:34.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:24:33 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:24:36.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:24:35 smithi165 ceph-mon[36302]: pgmap v5568: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T03:24:36.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:24:35 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10135.804791 secs 2024-03-28T03:24:36.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:24:35 smithi119 ceph-mon[31518]: pgmap v5568: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T03:24:36.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:24:35 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10135.804791 secs 2024-03-28T03:24:36.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:24:35 smithi132 ceph-mon[35941]: pgmap v5568: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T03:24:36.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:24:35 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10135.804791 secs 2024-03-28T03:24:38.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:24:37 smithi165 ceph-mon[36302]: pgmap v5569: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T03:24:38.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:24:37 smithi119 ceph-mon[31518]: pgmap v5569: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T03:24:38.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:24:37 smithi132 ceph-mon[35941]: pgmap v5569: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T03:24:40.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:24:40 smithi165 ceph-mon[36302]: pgmap v5570: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T03:24:40.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:24:40 smithi119 ceph-mon[31518]: pgmap v5570: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T03:24:40.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:24:40 smithi132 ceph-mon[35941]: pgmap v5570: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T03:24:41.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:24:41 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10140.804852 secs 2024-03-28T03:24:41.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:24:41 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10140.804852 secs 2024-03-28T03:24:41.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:24:41 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10140.804852 secs 2024-03-28T03:24:42.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:24:42 smithi165 ceph-mon[36302]: pgmap v5571: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T03:24:42.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:24:42 smithi119 ceph-mon[31518]: pgmap v5571: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T03:24:42.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:24:42 smithi132 ceph-mon[35941]: pgmap v5571: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T03:24:44.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:24:44 smithi165 ceph-mon[36302]: pgmap v5572: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T03:24:44.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:24:44 smithi119 ceph-mon[31518]: pgmap v5572: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T03:24:44.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:24:44 smithi132 ceph-mon[35941]: pgmap v5572: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T03:24:46.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:24:46 smithi165 ceph-mon[36302]: pgmap v5573: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:24:46.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:24:46 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10145.804937 secs 2024-03-28T03:24:46.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:24:46 smithi119 ceph-mon[31518]: pgmap v5573: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:24:46.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:24:46 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10145.804937 secs 2024-03-28T03:24:46.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:24:46 smithi132 ceph-mon[35941]: pgmap v5573: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:24:46.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:24:46 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10145.804937 secs 2024-03-28T03:24:48.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:24:48 smithi165 ceph-mon[36302]: pgmap v5574: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:24:48.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:24:48 smithi119 ceph-mon[31518]: pgmap v5574: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:24:48.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:24:48 smithi132 ceph-mon[35941]: pgmap v5574: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:24:50.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:24:50 smithi165 ceph-mon[36302]: pgmap v5575: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:24:50.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:24:50 smithi119 ceph-mon[31518]: pgmap v5575: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:24:50.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:24:50 smithi132 ceph-mon[35941]: pgmap v5575: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:24:51.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:24:51 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10150.804999 secs 2024-03-28T03:24:51.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:24:51 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10150.804999 secs 2024-03-28T03:24:51.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:24:51 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10150.804999 secs 2024-03-28T03:24:52.296 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:24:52 smithi165 ceph-mon[36302]: pgmap v5576: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:24:52.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:24:52 smithi119 ceph-mon[31518]: pgmap v5576: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:24:52.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:24:52 smithi132 ceph-mon[35941]: pgmap v5576: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:24:54.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:24:54 smithi165 ceph-mon[36302]: pgmap v5577: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:24:54.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:24:54 smithi119 ceph-mon[31518]: pgmap v5577: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:24:54.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:24:54 smithi132 ceph-mon[35941]: pgmap v5577: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:24:56.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:24:56 smithi165 ceph-mon[36302]: pgmap v5578: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:24:56.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:24:56 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10155.805087 secs 2024-03-28T03:24:56.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:24:56 smithi119 ceph-mon[31518]: pgmap v5578: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:24:56.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:24:56 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10155.805087 secs 2024-03-28T03:24:56.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:24:56 smithi132 ceph-mon[35941]: pgmap v5578: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:24:56.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:24:56 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10155.805087 secs 2024-03-28T03:24:58.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:24:58 smithi165 ceph-mon[36302]: pgmap v5579: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:24:58.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:24:58 smithi119 ceph-mon[31518]: pgmap v5579: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:24:58.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:24:58 smithi132 ceph-mon[35941]: pgmap v5579: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:25:00.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:25:00 smithi165 ceph-mon[36302]: pgmap v5580: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:25:00.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:25:00 smithi119 ceph-mon[31518]: pgmap v5580: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:25:00.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:25:00 smithi132 ceph-mon[35941]: pgmap v5580: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:25:01.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:25:01 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10160.805165 secs 2024-03-28T03:25:01.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:25:01 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10160.805165 secs 2024-03-28T03:25:01.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:25:01 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10160.805165 secs 2024-03-28T03:25:02.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:25:02 smithi165 ceph-mon[36302]: pgmap v5581: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:25:02.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:25:02 smithi119 ceph-mon[31518]: pgmap v5581: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:25:02.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:25:02 smithi132 ceph-mon[35941]: pgmap v5581: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:25:04.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:25:04 smithi165 ceph-mon[36302]: pgmap v5582: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:25:04.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:25:04 smithi119 ceph-mon[31518]: pgmap v5582: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:25:04.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:25:04 smithi132 ceph-mon[35941]: pgmap v5582: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:25:06.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:25:06 smithi165 ceph-mon[36302]: pgmap v5583: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:25:06.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:25:06 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10165.805205 secs 2024-03-28T03:25:06.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:25:06 smithi119 ceph-mon[31518]: pgmap v5583: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:25:06.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:25:06 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10165.805205 secs 2024-03-28T03:25:06.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:25:06 smithi132 ceph-mon[35941]: pgmap v5583: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:25:06.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:25:06 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10165.805205 secs 2024-03-28T03:25:08.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:25:08 smithi165 ceph-mon[36302]: pgmap v5584: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:25:08.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:25:08 smithi119 ceph-mon[31518]: pgmap v5584: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:25:08.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:25:08 smithi132 ceph-mon[35941]: pgmap v5584: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:25:10.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:25:10 smithi165 ceph-mon[36302]: pgmap v5585: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:25:10.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:25:10 smithi119 ceph-mon[31518]: pgmap v5585: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:25:10.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:25:10 smithi132 ceph-mon[35941]: pgmap v5585: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:25:11.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:25:11 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10170.805266 secs 2024-03-28T03:25:11.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:25:11 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10170.805266 secs 2024-03-28T03:25:11.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:25:11 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10170.805266 secs 2024-03-28T03:25:12.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:25:12 smithi165 ceph-mon[36302]: pgmap v5586: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:25:12.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:25:12 smithi119 ceph-mon[31518]: pgmap v5586: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:25:12.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:25:12 smithi132 ceph-mon[35941]: pgmap v5586: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:25:14.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:25:14 smithi165 ceph-mon[36302]: pgmap v5587: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:25:14.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:25:14 smithi119 ceph-mon[31518]: pgmap v5587: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:25:14.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:25:14 smithi132 ceph-mon[35941]: pgmap v5587: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:25:16.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:25:16 smithi165 ceph-mon[36302]: pgmap v5588: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:25:16.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:25:16 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10175.805327 secs 2024-03-28T03:25:16.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:25:16 smithi119 ceph-mon[31518]: pgmap v5588: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:25:16.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:25:16 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10175.805327 secs 2024-03-28T03:25:16.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:25:16 smithi132 ceph-mon[35941]: pgmap v5588: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:25:16.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:25:16 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10175.805327 secs 2024-03-28T03:25:18.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:25:18 smithi165 ceph-mon[36302]: pgmap v5589: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T03:25:18.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:25:18 smithi119 ceph-mon[31518]: pgmap v5589: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T03:25:18.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:25:18 smithi132 ceph-mon[35941]: pgmap v5589: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T03:25:20.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:25:20 smithi165 ceph-mon[36302]: pgmap v5590: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T03:25:20.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:25:20 smithi119 ceph-mon[31518]: pgmap v5590: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T03:25:20.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:25:20 smithi132 ceph-mon[35941]: pgmap v5590: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T03:25:21.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:25:21 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10180.805389 secs 2024-03-28T03:25:21.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:25:21 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10180.805389 secs 2024-03-28T03:25:21.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:25:21 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10180.805389 secs 2024-03-28T03:25:22.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:25:22 smithi119 ceph-mon[31518]: pgmap v5591: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:25:22.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:25:22 smithi132 ceph-mon[35941]: pgmap v5591: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:25:22.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:25:22 smithi165 ceph-mon[36302]: pgmap v5591: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:25:24.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:25:24 smithi119 ceph-mon[31518]: pgmap v5592: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T03:25:24.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:25:24 smithi132 ceph-mon[35941]: pgmap v5592: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T03:25:24.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:25:24 smithi165 ceph-mon[36302]: pgmap v5592: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T03:25:26.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:25:26 smithi119 ceph-mon[31518]: pgmap v5593: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:25:26.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:25:26 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10185.805479 secs 2024-03-28T03:25:26.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:25:26 smithi132 ceph-mon[35941]: pgmap v5593: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:25:26.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:25:26 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10185.805479 secs 2024-03-28T03:25:26.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:25:26 smithi165 ceph-mon[36302]: pgmap v5593: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:25:26.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:25:26 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10185.805479 secs 2024-03-28T03:25:28.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:25:28 smithi119 ceph-mon[31518]: pgmap v5594: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:25:28.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:25:28 smithi132 ceph-mon[35941]: pgmap v5594: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:25:28.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:25:28 smithi165 ceph-mon[36302]: pgmap v5594: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:25:30.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:25:30 smithi119 ceph-mon[31518]: pgmap v5595: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:25:30.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:25:30 smithi132 ceph-mon[35941]: pgmap v5595: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:25:30.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:25:30 smithi165 ceph-mon[36302]: pgmap v5595: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:25:31.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:25:31 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10190.805564 secs 2024-03-28T03:25:31.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:25:31 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10190.805564 secs 2024-03-28T03:25:31.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:25:31 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10190.805564 secs 2024-03-28T03:25:32.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:25:32 smithi132 ceph-mon[35941]: pgmap v5596: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:25:32.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:25:32 smithi165 ceph-mon[36302]: pgmap v5596: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:25:32.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:25:32 smithi119 ceph-mon[31518]: pgmap v5596: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:25:34.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:25:34 smithi132 ceph-mon[35941]: pgmap v5597: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:25:34.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:25:34 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T03:25:34.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:25:34 smithi165 ceph-mon[36302]: pgmap v5597: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:25:34.641 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:25:34 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T03:25:34.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:25:34 smithi119 ceph-mon[31518]: pgmap v5597: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:25:34.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:25:34 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T03:25:35.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:25:35 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T03:25:35.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:25:35 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T03:25:35.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:25:35 smithi132 ceph-mon[35941]: pgmap v5598: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T03:25:35.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:25:35 smithi132 ceph-mon[35941]: pgmap v5599: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:25:35.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:25:35 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:25:35.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:25:35 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T03:25:35.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:25:35 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T03:25:35.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:25:35 smithi165 ceph-mon[36302]: pgmap v5598: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T03:25:35.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:25:35 smithi165 ceph-mon[36302]: pgmap v5599: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:25:35.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:25:35 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:25:35.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:25:35 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T03:25:35.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:25:35 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T03:25:35.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:25:35 smithi119 ceph-mon[31518]: pgmap v5598: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T03:25:35.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:25:35 smithi119 ceph-mon[31518]: pgmap v5599: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:25:35.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:25:35 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:25:36.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:25:36 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10195.805624 secs 2024-03-28T03:25:36.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:25:36 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10195.805624 secs 2024-03-28T03:25:36.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:25:36 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10195.805624 secs 2024-03-28T03:25:37.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:25:37 smithi132 ceph-mon[35941]: pgmap v5600: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:25:37.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:25:37 smithi165 ceph-mon[36302]: pgmap v5600: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:25:37.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:25:37 smithi119 ceph-mon[31518]: pgmap v5600: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:25:39.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:25:39 smithi165 ceph-mon[36302]: pgmap v5601: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:25:39.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:25:39 smithi119 ceph-mon[31518]: pgmap v5601: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:25:39.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:25:39 smithi132 ceph-mon[35941]: pgmap v5601: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:25:41.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:25:41 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10200.805711 secs 2024-03-28T03:25:41.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:25:41 smithi165 ceph-mon[36302]: pgmap v5602: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:25:41.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:25:41 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10200.805711 secs 2024-03-28T03:25:41.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:25:41 smithi119 ceph-mon[31518]: pgmap v5602: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:25:41.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:25:41 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10200.805711 secs 2024-03-28T03:25:41.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:25:41 smithi132 ceph-mon[35941]: pgmap v5602: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:25:43.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:25:43 smithi165 ceph-mon[36302]: pgmap v5603: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:25:43.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:25:43 smithi119 ceph-mon[31518]: pgmap v5603: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:25:43.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:25:43 smithi132 ceph-mon[35941]: pgmap v5603: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:25:45.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:25:45 smithi165 ceph-mon[36302]: pgmap v5604: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T03:25:45.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:25:45 smithi119 ceph-mon[31518]: pgmap v5604: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T03:25:45.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:25:45 smithi132 ceph-mon[35941]: pgmap v5604: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T03:25:46.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:25:46 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10205.805799 secs 2024-03-28T03:25:46.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:25:46 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10205.805799 secs 2024-03-28T03:25:46.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:25:46 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10205.805799 secs 2024-03-28T03:25:47.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:25:47 smithi165 ceph-mon[36302]: pgmap v5605: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:25:47.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:25:47 smithi119 ceph-mon[31518]: pgmap v5605: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:25:47.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:25:47 smithi132 ceph-mon[35941]: pgmap v5605: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:25:49.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:25:49 smithi119 ceph-mon[31518]: pgmap v5606: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:25:49.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:25:49 smithi132 ceph-mon[35941]: pgmap v5606: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:25:50.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:25:49 smithi165 ceph-mon[36302]: pgmap v5606: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:25:51.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:25:51 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10210.805886 secs 2024-03-28T03:25:51.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:25:51 smithi119 ceph-mon[31518]: pgmap v5607: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:25:51.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:25:51 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10210.805886 secs 2024-03-28T03:25:51.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:25:51 smithi132 ceph-mon[35941]: pgmap v5607: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:25:52.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:25:51 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10210.805886 secs 2024-03-28T03:25:52.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:25:51 smithi165 ceph-mon[36302]: pgmap v5607: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:25:53.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:25:53 smithi119 ceph-mon[31518]: pgmap v5608: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:25:53.931 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:25:53 smithi165 ceph-mon[36302]: pgmap v5608: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:25:53.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:25:53 smithi132 ceph-mon[35941]: pgmap v5608: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:25:55.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:25:55 smithi119 ceph-mon[31518]: pgmap v5609: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:25:55.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:25:55 smithi132 ceph-mon[35941]: pgmap v5609: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:25:56.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:25:55 smithi165 ceph-mon[36302]: pgmap v5609: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:25:56.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:25:56 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10215.805971 secs 2024-03-28T03:25:57.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:25:56 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10215.805971 secs 2024-03-28T03:25:57.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:25:56 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10215.805971 secs 2024-03-28T03:25:57.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:25:57 smithi132 ceph-mon[35941]: pgmap v5610: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:25:58.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:25:57 smithi165 ceph-mon[36302]: pgmap v5610: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:25:58.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:25:57 smithi119 ceph-mon[31518]: pgmap v5610: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:25:59.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:25:59 smithi132 ceph-mon[35941]: pgmap v5611: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:26:00.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:25:59 smithi165 ceph-mon[36302]: pgmap v5611: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:26:00.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:25:59 smithi119 ceph-mon[31518]: pgmap v5611: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:26:01.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:26:01 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10220.806033 secs 2024-03-28T03:26:01.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:26:01 smithi132 ceph-mon[35941]: pgmap v5612: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:26:02.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:26:01 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10220.806033 secs 2024-03-28T03:26:02.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:26:01 smithi165 ceph-mon[36302]: pgmap v5612: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:26:02.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:26:01 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10220.806033 secs 2024-03-28T03:26:02.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:26:01 smithi119 ceph-mon[31518]: pgmap v5612: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:26:03.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:26:03 smithi132 ceph-mon[35941]: pgmap v5613: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:26:04.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:26:03 smithi165 ceph-mon[36302]: pgmap v5613: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:26:04.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:26:03 smithi119 ceph-mon[31518]: pgmap v5613: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:26:05.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:26:05 smithi132 ceph-mon[35941]: pgmap v5614: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:26:06.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:26:05 smithi165 ceph-mon[36302]: pgmap v5614: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:26:06.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:26:05 smithi119 ceph-mon[31518]: pgmap v5614: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:26:06.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:26:06 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10225.806122 secs 2024-03-28T03:26:07.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:26:06 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10225.806122 secs 2024-03-28T03:26:07.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:26:06 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10225.806122 secs 2024-03-28T03:26:07.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:26:07 smithi132 ceph-mon[35941]: pgmap v5615: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:26:08.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:26:07 smithi165 ceph-mon[36302]: pgmap v5615: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:26:08.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:26:07 smithi119 ceph-mon[31518]: pgmap v5615: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:26:09.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:26:09 smithi132 ceph-mon[35941]: pgmap v5616: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:26:10.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:26:09 smithi165 ceph-mon[36302]: pgmap v5616: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:26:10.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:26:09 smithi119 ceph-mon[31518]: pgmap v5616: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:26:12.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:26:11 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10230.806185 secs 2024-03-28T03:26:12.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:26:11 smithi165 ceph-mon[36302]: pgmap v5617: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:26:12.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:26:11 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10230.806185 secs 2024-03-28T03:26:12.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:26:11 smithi119 ceph-mon[31518]: pgmap v5617: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:26:12.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:26:11 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10230.806185 secs 2024-03-28T03:26:12.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:26:11 smithi132 ceph-mon[35941]: pgmap v5617: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:26:14.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:26:13 smithi165 ceph-mon[36302]: pgmap v5618: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T03:26:14.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:26:13 smithi119 ceph-mon[31518]: pgmap v5618: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T03:26:14.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:26:13 smithi132 ceph-mon[35941]: pgmap v5618: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T03:26:16.099 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:26:15 smithi119 ceph-mon[31518]: pgmap v5619: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:26:16.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:26:15 smithi165 ceph-mon[36302]: pgmap v5619: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:26:16.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:26:15 smithi132 ceph-mon[35941]: pgmap v5619: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:26:17.101 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:26:16 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10235.806245 secs 2024-03-28T03:26:17.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:26:16 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10235.806245 secs 2024-03-28T03:26:17.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:26:16 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10235.806245 secs 2024-03-28T03:26:18.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:26:17 smithi165 ceph-mon[36302]: pgmap v5620: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T03:26:18.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:26:17 smithi119 ceph-mon[31518]: pgmap v5620: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T03:26:18.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:26:17 smithi132 ceph-mon[35941]: pgmap v5620: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T03:26:20.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:26:19 smithi165 ceph-mon[36302]: pgmap v5621: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T03:26:20.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:26:19 smithi119 ceph-mon[31518]: pgmap v5621: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T03:26:20.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:26:19 smithi132 ceph-mon[35941]: pgmap v5621: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T03:26:22.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:26:21 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10240.806305 secs 2024-03-28T03:26:22.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:26:21 smithi165 ceph-mon[36302]: pgmap v5622: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:26:22.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:26:21 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10240.806305 secs 2024-03-28T03:26:22.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:26:21 smithi119 ceph-mon[31518]: pgmap v5622: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:26:22.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:26:21 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10240.806305 secs 2024-03-28T03:26:22.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:26:21 smithi132 ceph-mon[35941]: pgmap v5622: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:26:24.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:26:23 smithi165 ceph-mon[36302]: pgmap v5623: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:26:24.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:26:23 smithi119 ceph-mon[31518]: pgmap v5623: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:26:24.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:26:23 smithi132 ceph-mon[35941]: pgmap v5623: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:26:26.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:26:25 smithi165 ceph-mon[36302]: pgmap v5624: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:26:26.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:26:25 smithi119 ceph-mon[31518]: pgmap v5624: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:26:26.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:26:25 smithi132 ceph-mon[35941]: pgmap v5624: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:26:27.054 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:26:26 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10245.806364 secs 2024-03-28T03:26:27.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:26:26 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10245.806364 secs 2024-03-28T03:26:27.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:26:26 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10245.806364 secs 2024-03-28T03:26:28.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:26:27 smithi165 ceph-mon[36302]: pgmap v5625: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:26:28.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:26:27 smithi119 ceph-mon[31518]: pgmap v5625: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:26:28.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:26:27 smithi132 ceph-mon[35941]: pgmap v5625: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:26:30.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:26:29 smithi165 ceph-mon[36302]: pgmap v5626: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:26:30.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:26:29 smithi119 ceph-mon[31518]: pgmap v5626: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:26:30.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:26:29 smithi132 ceph-mon[35941]: pgmap v5626: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:26:32.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:26:31 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10250.806424 secs 2024-03-28T03:26:32.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:26:31 smithi165 ceph-mon[36302]: pgmap v5627: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:26:32.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:26:31 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10250.806424 secs 2024-03-28T03:26:32.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:26:31 smithi119 ceph-mon[31518]: pgmap v5627: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:26:32.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:26:31 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10250.806424 secs 2024-03-28T03:26:32.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:26:31 smithi132 ceph-mon[35941]: pgmap v5627: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:26:34.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:26:33 smithi165 ceph-mon[36302]: pgmap v5628: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T03:26:34.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:26:33 smithi119 ceph-mon[31518]: pgmap v5628: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T03:26:34.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:26:33 smithi132 ceph-mon[35941]: pgmap v5628: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T03:26:36.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:26:35 smithi165 ceph-mon[36302]: pgmap v5629: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:26:36.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:26:35 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T03:26:36.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:26:35 smithi119 ceph-mon[31518]: pgmap v5629: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:26:36.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:26:35 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T03:26:36.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:26:35 smithi132 ceph-mon[35941]: pgmap v5629: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:26:36.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:26:35 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T03:26:37.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:26:36 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10255.806509 secs 2024-03-28T03:26:37.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:26:36 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T03:26:37.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:26:36 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T03:26:37.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:26:36 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:26:37.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:26:36 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10255.806509 secs 2024-03-28T03:26:37.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:26:36 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T03:26:37.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:26:36 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T03:26:37.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:26:36 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:26:37.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:26:36 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10255.806509 secs 2024-03-28T03:26:37.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:26:36 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T03:26:37.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:26:36 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T03:26:37.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:26:36 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:26:38.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:26:37 smithi165 ceph-mon[36302]: pgmap v5630: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T03:26:38.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:26:37 smithi165 ceph-mon[36302]: pgmap v5631: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T03:26:38.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:26:37 smithi119 ceph-mon[31518]: pgmap v5630: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T03:26:38.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:26:37 smithi119 ceph-mon[31518]: pgmap v5631: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T03:26:38.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:26:37 smithi132 ceph-mon[35941]: pgmap v5630: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T03:26:38.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:26:37 smithi132 ceph-mon[35941]: pgmap v5631: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T03:26:40.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:26:39 smithi165 ceph-mon[36302]: pgmap v5632: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T03:26:40.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:26:39 smithi119 ceph-mon[31518]: pgmap v5632: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T03:26:40.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:26:39 smithi132 ceph-mon[35941]: pgmap v5632: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T03:26:42.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:26:41 smithi165 ceph-mon[36302]: pgmap v5633: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:26:42.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:26:41 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10260.806598 secs 2024-03-28T03:26:42.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:26:41 smithi119 ceph-mon[31518]: pgmap v5633: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:26:42.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:26:41 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10260.806598 secs 2024-03-28T03:26:42.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:26:41 smithi132 ceph-mon[35941]: pgmap v5633: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:26:42.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:26:41 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10260.806598 secs 2024-03-28T03:26:44.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:26:43 smithi165 ceph-mon[36302]: pgmap v5634: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:26:44.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:26:43 smithi119 ceph-mon[31518]: pgmap v5634: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:26:44.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:26:43 smithi132 ceph-mon[35941]: pgmap v5634: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:26:46.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:26:45 smithi165 ceph-mon[36302]: pgmap v5635: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:26:46.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:26:45 smithi119 ceph-mon[31518]: pgmap v5635: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:26:46.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:26:45 smithi132 ceph-mon[35941]: pgmap v5635: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:26:47.101 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:26:46 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10265.806640 secs 2024-03-28T03:26:47.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:26:46 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10265.806640 secs 2024-03-28T03:26:47.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:26:46 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10265.806640 secs 2024-03-28T03:26:48.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:26:47 smithi165 ceph-mon[36302]: pgmap v5636: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T03:26:48.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:26:47 smithi119 ceph-mon[31518]: pgmap v5636: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T03:26:48.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:26:47 smithi132 ceph-mon[35941]: pgmap v5636: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T03:26:50.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:26:49 smithi165 ceph-mon[36302]: pgmap v5637: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:26:50.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:26:49 smithi119 ceph-mon[31518]: pgmap v5637: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:26:50.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:26:49 smithi132 ceph-mon[35941]: pgmap v5637: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:26:52.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:26:51 smithi165 ceph-mon[36302]: pgmap v5638: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:26:52.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:26:51 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10270.806729 secs 2024-03-28T03:26:52.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:26:51 smithi119 ceph-mon[31518]: pgmap v5638: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:26:52.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:26:51 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10270.806729 secs 2024-03-28T03:26:52.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:26:51 smithi132 ceph-mon[35941]: pgmap v5638: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:26:52.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:26:51 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10270.806729 secs 2024-03-28T03:26:54.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:26:53 smithi165 ceph-mon[36302]: pgmap v5639: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:26:54.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:26:53 smithi119 ceph-mon[31518]: pgmap v5639: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:26:54.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:26:53 smithi132 ceph-mon[35941]: pgmap v5639: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:26:56.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:26:55 smithi165 ceph-mon[36302]: pgmap v5640: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:26:56.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:26:55 smithi119 ceph-mon[31518]: pgmap v5640: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:26:56.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:26:55 smithi132 ceph-mon[35941]: pgmap v5640: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:26:57.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:26:56 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10275.806820 secs 2024-03-28T03:26:57.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:26:56 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10275.806820 secs 2024-03-28T03:26:57.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:26:56 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10275.806820 secs 2024-03-28T03:26:58.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:26:57 smithi165 ceph-mon[36302]: pgmap v5641: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:26:58.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:26:57 smithi119 ceph-mon[31518]: pgmap v5641: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:26:58.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:26:57 smithi132 ceph-mon[35941]: pgmap v5641: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:27:00.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:26:59 smithi165 ceph-mon[36302]: pgmap v5642: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:27:00.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:26:59 smithi119 ceph-mon[31518]: pgmap v5642: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:27:00.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:26:59 smithi132 ceph-mon[35941]: pgmap v5642: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:27:02.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:27:01 smithi165 ceph-mon[36302]: pgmap v5643: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:27:02.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:27:01 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10280.806896 secs 2024-03-28T03:27:02.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:27:01 smithi119 ceph-mon[31518]: pgmap v5643: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:27:02.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:27:01 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10280.806896 secs 2024-03-28T03:27:02.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:27:01 smithi132 ceph-mon[35941]: pgmap v5643: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:27:02.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:27:01 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10280.806896 secs 2024-03-28T03:27:04.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:27:03 smithi165 ceph-mon[36302]: pgmap v5644: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:27:04.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:27:03 smithi119 ceph-mon[31518]: pgmap v5644: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:27:04.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:27:03 smithi132 ceph-mon[35941]: pgmap v5644: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:27:06.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:27:05 smithi165 ceph-mon[36302]: pgmap v5645: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:27:06.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:27:05 smithi119 ceph-mon[31518]: pgmap v5645: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:27:06.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:27:05 smithi132 ceph-mon[35941]: pgmap v5645: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:27:07.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:27:06 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10285.806980 secs 2024-03-28T03:27:07.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:27:06 smithi132 ceph-mon[35941]: pgmap v5646: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:27:07.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:27:06 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10285.806980 secs 2024-03-28T03:27:07.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:27:06 smithi165 ceph-mon[36302]: pgmap v5646: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:27:07.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:27:06 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10285.806980 secs 2024-03-28T03:27:07.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:27:06 smithi119 ceph-mon[31518]: pgmap v5646: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:27:09.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:27:09 smithi132 ceph-mon[35941]: pgmap v5647: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:27:09.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:27:09 smithi165 ceph-mon[36302]: pgmap v5647: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:27:09.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:27:09 smithi119 ceph-mon[31518]: pgmap v5647: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:27:11.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:27:11 smithi132 ceph-mon[35941]: pgmap v5648: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:27:11.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:27:11 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10290.807066 secs 2024-03-28T03:27:11.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:27:11 smithi165 ceph-mon[36302]: pgmap v5648: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:27:11.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:27:11 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10290.807066 secs 2024-03-28T03:27:11.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:27:11 smithi119 ceph-mon[31518]: pgmap v5648: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:27:11.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:27:11 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10290.807066 secs 2024-03-28T03:27:13.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:27:13 smithi132 ceph-mon[35941]: pgmap v5649: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:27:13.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:27:13 smithi165 ceph-mon[36302]: pgmap v5649: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:27:13.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:27:13 smithi119 ceph-mon[31518]: pgmap v5649: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:27:15.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:27:15 smithi132 ceph-mon[35941]: pgmap v5650: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:27:15.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:27:15 smithi165 ceph-mon[36302]: pgmap v5650: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:27:15.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:27:15 smithi119 ceph-mon[31518]: pgmap v5650: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:27:16.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:27:16 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10295.807152 secs 2024-03-28T03:27:16.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:27:16 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10295.807152 secs 2024-03-28T03:27:16.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:27:16 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10295.807152 secs 2024-03-28T03:27:17.601 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:27:17 smithi132 ceph-mon[35941]: pgmap v5651: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:27:17.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:27:17 smithi165 ceph-mon[36302]: pgmap v5651: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:27:17.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:27:17 smithi119 ceph-mon[31518]: pgmap v5651: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:27:19.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:27:19 smithi119 ceph-mon[31518]: pgmap v5652: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:27:19.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:27:19 smithi165 ceph-mon[36302]: pgmap v5652: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:27:19.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:27:19 smithi132 ceph-mon[35941]: pgmap v5652: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:27:21.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:27:21 smithi165 ceph-mon[36302]: pgmap v5653: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:27:21.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:27:21 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10300.807212 secs 2024-03-28T03:27:21.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:27:21 smithi119 ceph-mon[31518]: pgmap v5653: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:27:21.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:27:21 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10300.807212 secs 2024-03-28T03:27:21.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:27:21 smithi132 ceph-mon[35941]: pgmap v5653: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:27:21.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:27:21 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10300.807212 secs 2024-03-28T03:27:23.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:27:23 smithi165 ceph-mon[36302]: pgmap v5654: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:27:23.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:27:23 smithi119 ceph-mon[31518]: pgmap v5654: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:27:23.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:27:23 smithi132 ceph-mon[35941]: pgmap v5654: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:27:25.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:27:25 smithi165 ceph-mon[36302]: pgmap v5655: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:27:25.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:27:25 smithi119 ceph-mon[31518]: pgmap v5655: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:27:25.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:27:25 smithi132 ceph-mon[35941]: pgmap v5655: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:27:26.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:27:26 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10305.807296 secs 2024-03-28T03:27:26.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:27:26 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10305.807296 secs 2024-03-28T03:27:26.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:27:26 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10305.807296 secs 2024-03-28T03:27:27.554 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:27:27 smithi119 ceph-mon[31518]: pgmap v5656: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:27:27.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:27:27 smithi165 ceph-mon[36302]: pgmap v5656: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:27:27.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:27:27 smithi132 ceph-mon[35941]: pgmap v5656: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:27:29.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:27:29 smithi165 ceph-mon[36302]: pgmap v5657: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:27:29.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:27:29 smithi119 ceph-mon[31518]: pgmap v5657: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:27:29.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:27:29 smithi132 ceph-mon[35941]: pgmap v5657: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:27:31.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:27:31 smithi165 ceph-mon[36302]: pgmap v5658: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:27:31.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:27:31 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10310.807356 secs 2024-03-28T03:27:31.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:27:31 smithi119 ceph-mon[31518]: pgmap v5658: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:27:31.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:27:31 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10310.807356 secs 2024-03-28T03:27:31.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:27:31 smithi132 ceph-mon[35941]: pgmap v5658: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:27:31.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:27:31 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10310.807356 secs 2024-03-28T03:27:33.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:27:33 smithi165 ceph-mon[36302]: pgmap v5659: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:27:33.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:27:33 smithi119 ceph-mon[31518]: pgmap v5659: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:27:33.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:27:33 smithi132 ceph-mon[35941]: pgmap v5659: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:27:35.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:27:35 smithi165 ceph-mon[36302]: pgmap v5660: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:27:35.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:27:35 smithi119 ceph-mon[31518]: pgmap v5660: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:27:35.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:27:35 smithi132 ceph-mon[35941]: pgmap v5660: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:27:36.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:27:36 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10315.807445 secs 2024-03-28T03:27:36.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:27:36 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10315.807445 secs 2024-03-28T03:27:36.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:27:36 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10315.807445 secs 2024-03-28T03:27:37.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:27:37 smithi165 ceph-mon[36302]: pgmap v5661: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:27:37.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:27:37 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T03:27:37.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:27:37 smithi119 ceph-mon[31518]: pgmap v5661: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:27:37.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:27:37 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T03:27:37.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:27:37 smithi132 ceph-mon[35941]: pgmap v5661: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:27:37.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:27:37 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T03:27:39.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:27:38 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:27:39.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:27:38 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:27:39.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:27:38 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:27:39.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:27:38 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:27:39.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:27:38 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:27:39.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:27:38 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:27:39.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:27:38 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T03:27:39.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:27:38 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T03:27:39.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:27:38 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:27:39.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:27:38 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:27:39.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:27:38 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:27:39.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:27:38 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:27:39.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:27:38 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:27:39.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:27:38 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:27:39.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:27:38 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:27:39.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:27:38 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T03:27:39.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:27:38 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T03:27:39.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:27:38 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:27:39.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:27:38 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:27:39.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:27:38 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:27:39.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:27:38 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:27:39.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:27:38 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:27:39.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:27:38 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:27:39.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:27:38 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:27:39.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:27:38 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T03:27:39.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:27:38 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T03:27:39.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:27:38 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:27:40.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:27:39 smithi165 ceph-mon[36302]: pgmap v5662: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:27:40.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:27:39 smithi165 ceph-mon[36302]: pgmap v5663: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-28T03:27:40.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:27:39 smithi165 ceph-mon[36302]: pgmap v5664: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 3 op/s 2024-03-28T03:27:40.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:27:39 smithi119 ceph-mon[31518]: pgmap v5662: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:27:40.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:27:39 smithi119 ceph-mon[31518]: pgmap v5663: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-28T03:27:40.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:27:39 smithi119 ceph-mon[31518]: pgmap v5664: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 3 op/s 2024-03-28T03:27:40.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:27:39 smithi132 ceph-mon[35941]: pgmap v5662: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:27:40.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:27:39 smithi132 ceph-mon[35941]: pgmap v5663: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-28T03:27:40.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:27:39 smithi132 ceph-mon[35941]: pgmap v5664: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 3 op/s 2024-03-28T03:27:42.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:27:41 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10320.807534 secs 2024-03-28T03:27:42.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:27:41 smithi165 ceph-mon[36302]: pgmap v5665: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-03-28T03:27:42.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:27:41 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10320.807534 secs 2024-03-28T03:27:42.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:27:41 smithi119 ceph-mon[31518]: pgmap v5665: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-03-28T03:27:42.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:27:41 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10320.807534 secs 2024-03-28T03:27:42.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:27:41 smithi132 ceph-mon[35941]: pgmap v5665: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-03-28T03:27:44.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:27:43 smithi165 ceph-mon[36302]: pgmap v5666: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 3 op/s 2024-03-28T03:27:44.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:27:43 smithi119 ceph-mon[31518]: pgmap v5666: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 3 op/s 2024-03-28T03:27:44.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:27:43 smithi132 ceph-mon[35941]: pgmap v5666: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 3 op/s 2024-03-28T03:27:46.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:27:45 smithi165 ceph-mon[36302]: pgmap v5667: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-03-28T03:27:46.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:27:45 smithi119 ceph-mon[31518]: pgmap v5667: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-03-28T03:27:46.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:27:45 smithi132 ceph-mon[35941]: pgmap v5667: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-03-28T03:27:47.101 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:27:46 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10325.807620 secs 2024-03-28T03:27:47.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:27:46 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10325.807620 secs 2024-03-28T03:27:47.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:27:46 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10325.807620 secs 2024-03-28T03:27:48.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:27:47 smithi165 ceph-mon[36302]: pgmap v5668: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-03-28T03:27:48.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:27:47 smithi119 ceph-mon[31518]: pgmap v5668: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-03-28T03:27:48.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:27:47 smithi132 ceph-mon[35941]: pgmap v5668: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-03-28T03:27:50.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:27:49 smithi165 ceph-mon[36302]: pgmap v5669: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T03:27:50.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:27:49 smithi119 ceph-mon[31518]: pgmap v5669: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T03:27:50.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:27:49 smithi132 ceph-mon[35941]: pgmap v5669: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T03:27:52.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:27:51 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10330.807704 secs 2024-03-28T03:27:52.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:27:51 smithi165 ceph-mon[36302]: pgmap v5670: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:27:52.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:27:51 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10330.807704 secs 2024-03-28T03:27:52.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:27:51 smithi119 ceph-mon[31518]: pgmap v5670: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:27:52.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:27:51 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10330.807704 secs 2024-03-28T03:27:52.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:27:51 smithi132 ceph-mon[35941]: pgmap v5670: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:27:54.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:27:53 smithi165 ceph-mon[36302]: pgmap v5671: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:27:54.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:27:53 smithi119 ceph-mon[31518]: pgmap v5671: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:27:54.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:27:53 smithi132 ceph-mon[35941]: pgmap v5671: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:27:56.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:27:55 smithi165 ceph-mon[36302]: pgmap v5672: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:27:56.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:27:55 smithi119 ceph-mon[31518]: pgmap v5672: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:27:56.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:27:55 smithi132 ceph-mon[35941]: pgmap v5672: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:27:57.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:27:56 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10335.807793 secs 2024-03-28T03:27:57.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:27:56 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10335.807793 secs 2024-03-28T03:27:57.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:27:56 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10335.807793 secs 2024-03-28T03:27:58.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:27:57 smithi165 ceph-mon[36302]: pgmap v5673: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:27:58.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:27:57 smithi119 ceph-mon[31518]: pgmap v5673: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:27:58.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:27:57 smithi132 ceph-mon[35941]: pgmap v5673: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:28:00.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:27:59 smithi165 ceph-mon[36302]: pgmap v5674: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:28:00.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:27:59 smithi119 ceph-mon[31518]: pgmap v5674: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:28:00.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:27:59 smithi132 ceph-mon[35941]: pgmap v5674: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:28:02.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:28:01 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10340.807836 secs 2024-03-28T03:28:02.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:28:01 smithi165 ceph-mon[36302]: pgmap v5675: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:28:02.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:28:01 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10340.807836 secs 2024-03-28T03:28:02.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:28:01 smithi119 ceph-mon[31518]: pgmap v5675: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:28:02.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:28:01 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10340.807836 secs 2024-03-28T03:28:02.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:28:01 smithi132 ceph-mon[35941]: pgmap v5675: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:28:04.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:28:03 smithi165 ceph-mon[36302]: pgmap v5676: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:28:04.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:28:03 smithi119 ceph-mon[31518]: pgmap v5676: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:28:04.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:28:03 smithi132 ceph-mon[35941]: pgmap v5676: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:28:06.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:28:05 smithi165 ceph-mon[36302]: pgmap v5677: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:28:06.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:28:05 smithi119 ceph-mon[31518]: pgmap v5677: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:28:06.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:28:05 smithi132 ceph-mon[35941]: pgmap v5677: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:28:07.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:28:06 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10345.807899 secs 2024-03-28T03:28:07.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:28:06 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10345.807899 secs 2024-03-28T03:28:07.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:28:06 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10345.807899 secs 2024-03-28T03:28:08.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:28:07 smithi165 ceph-mon[36302]: pgmap v5678: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:28:08.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:28:07 smithi119 ceph-mon[31518]: pgmap v5678: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:28:08.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:28:07 smithi132 ceph-mon[35941]: pgmap v5678: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:28:10.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:28:09 smithi165 ceph-mon[36302]: pgmap v5679: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:28:10.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:28:09 smithi119 ceph-mon[31518]: pgmap v5679: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:28:10.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:28:09 smithi132 ceph-mon[35941]: pgmap v5679: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:28:12.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:28:11 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10350.807959 secs 2024-03-28T03:28:12.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:28:11 smithi165 ceph-mon[36302]: pgmap v5680: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:28:12.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:28:11 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10350.807959 secs 2024-03-28T03:28:12.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:28:11 smithi119 ceph-mon[31518]: pgmap v5680: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:28:12.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:28:11 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10350.807959 secs 2024-03-28T03:28:12.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:28:11 smithi132 ceph-mon[35941]: pgmap v5680: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:28:14.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:28:13 smithi165 ceph-mon[36302]: pgmap v5681: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:28:14.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:28:13 smithi119 ceph-mon[31518]: pgmap v5681: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:28:14.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:28:13 smithi132 ceph-mon[35941]: pgmap v5681: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:28:16.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:28:15 smithi165 ceph-mon[36302]: pgmap v5682: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:28:16.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:28:15 smithi119 ceph-mon[31518]: pgmap v5682: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:28:16.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:28:15 smithi132 ceph-mon[35941]: pgmap v5682: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:28:17.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:28:16 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10355.808020 secs 2024-03-28T03:28:17.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:28:16 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10355.808020 secs 2024-03-28T03:28:17.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:28:16 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10355.808020 secs 2024-03-28T03:28:18.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:28:17 smithi119 ceph-mon[31518]: pgmap v5683: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:28:18.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:28:17 smithi132 ceph-mon[35941]: pgmap v5683: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:28:18.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:28:17 smithi165 ceph-mon[36302]: pgmap v5683: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:28:20.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:28:19 smithi119 ceph-mon[31518]: pgmap v5684: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:28:20.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:28:19 smithi132 ceph-mon[35941]: pgmap v5684: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:28:20.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:28:19 smithi165 ceph-mon[36302]: pgmap v5684: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:28:21.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:28:20 smithi119 ceph-mon[31518]: pgmap v5685: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:28:21.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:28:20 smithi132 ceph-mon[35941]: pgmap v5685: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:28:21.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:28:20 smithi165 ceph-mon[36302]: pgmap v5685: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:28:22.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:28:21 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10360.808082 secs 2024-03-28T03:28:22.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:28:21 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10360.808082 secs 2024-03-28T03:28:22.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:28:21 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10360.808082 secs 2024-03-28T03:28:23.280 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:28:22 smithi165 ceph-mon[36302]: pgmap v5686: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:28:23.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:28:22 smithi119 ceph-mon[31518]: pgmap v5686: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:28:23.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:28:22 smithi132 ceph-mon[35941]: pgmap v5686: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:28:25.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:28:25 smithi165 ceph-mon[36302]: pgmap v5687: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:28:25.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:28:25 smithi119 ceph-mon[31518]: pgmap v5687: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:28:25.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:28:25 smithi132 ceph-mon[35941]: pgmap v5687: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:28:26.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:28:26 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10365.808146 secs 2024-03-28T03:28:26.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:28:26 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10365.808146 secs 2024-03-28T03:28:26.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:28:26 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10365.808146 secs 2024-03-28T03:28:27.804 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:28:27 smithi119 ceph-mon[31518]: pgmap v5688: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:28:27.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:28:27 smithi165 ceph-mon[36302]: pgmap v5688: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:28:27.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:28:27 smithi132 ceph-mon[35941]: pgmap v5688: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:28:29.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:28:29 smithi165 ceph-mon[36302]: pgmap v5689: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:28:29.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:28:29 smithi119 ceph-mon[31518]: pgmap v5689: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:28:29.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:28:29 smithi132 ceph-mon[35941]: pgmap v5689: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:28:31.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:28:31 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10370.808189 secs 2024-03-28T03:28:31.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:28:31 smithi165 ceph-mon[36302]: pgmap v5690: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:28:31.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:28:31 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10370.808189 secs 2024-03-28T03:28:31.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:28:31 smithi119 ceph-mon[31518]: pgmap v5690: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:28:31.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:28:31 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10370.808189 secs 2024-03-28T03:28:31.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:28:31 smithi132 ceph-mon[35941]: pgmap v5690: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:28:33.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:28:33 smithi165 ceph-mon[36302]: pgmap v5691: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:28:33.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:28:33 smithi119 ceph-mon[31518]: pgmap v5691: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:28:33.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:28:33 smithi132 ceph-mon[35941]: pgmap v5691: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:28:35.875 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:28:35 smithi119 ceph-mon[31518]: pgmap v5692: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:28:35.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:28:35 smithi165 ceph-mon[36302]: pgmap v5692: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:28:35.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:28:35 smithi132 ceph-mon[35941]: pgmap v5692: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:28:36.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:28:36 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10375.808251 secs 2024-03-28T03:28:36.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:28:36 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10375.808251 secs 2024-03-28T03:28:36.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:28:36 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10375.808251 secs 2024-03-28T03:28:37.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:28:37 smithi165 ceph-mon[36302]: pgmap v5693: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:28:37.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:28:37 smithi119 ceph-mon[31518]: pgmap v5693: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:28:37.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:28:37 smithi132 ceph-mon[35941]: pgmap v5693: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:28:39.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:28:39 smithi165 ceph-mon[36302]: pgmap v5694: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:28:39.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:28:39 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T03:28:39.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:28:39 smithi119 ceph-mon[31518]: pgmap v5694: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:28:39.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:28:39 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T03:28:39.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:28:39 smithi132 ceph-mon[35941]: pgmap v5694: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:28:39.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:28:39 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T03:28:40.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:28:40 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T03:28:40.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:28:40 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T03:28:40.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:28:40 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:28:40.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:28:40 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T03:28:40.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:28:40 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T03:28:40.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:28:40 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:28:40.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:28:40 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T03:28:40.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:28:40 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T03:28:40.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:28:40 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:28:41.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:28:41 smithi165 ceph-mon[36302]: pgmap v5695: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T03:28:41.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:28:41 smithi165 ceph-mon[36302]: pgmap v5696: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:28:41.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:28:41 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10380.808292 secs 2024-03-28T03:28:41.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:28:41 smithi119 ceph-mon[31518]: pgmap v5695: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T03:28:41.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:28:41 smithi119 ceph-mon[31518]: pgmap v5696: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:28:41.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:28:41 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10380.808292 secs 2024-03-28T03:28:41.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:28:41 smithi132 ceph-mon[35941]: pgmap v5695: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T03:28:41.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:28:41 smithi132 ceph-mon[35941]: pgmap v5696: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:28:41.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:28:41 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10380.808292 secs 2024-03-28T03:28:43.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:28:43 smithi165 ceph-mon[36302]: pgmap v5697: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:28:43.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:28:43 smithi119 ceph-mon[31518]: pgmap v5697: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:28:43.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:28:43 smithi132 ceph-mon[35941]: pgmap v5697: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:28:45.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:28:45 smithi165 ceph-mon[36302]: pgmap v5698: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-28T03:28:45.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:28:45 smithi119 ceph-mon[31518]: pgmap v5698: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-28T03:28:45.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:28:45 smithi132 ceph-mon[35941]: pgmap v5698: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-28T03:28:46.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:28:46 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10385.808330 secs 2024-03-28T03:28:46.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:28:46 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10385.808330 secs 2024-03-28T03:28:46.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:28:46 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10385.808330 secs 2024-03-28T03:28:47.851 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:28:47 smithi132 ceph-mon[35941]: pgmap v5699: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:28:47.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:28:47 smithi165 ceph-mon[36302]: pgmap v5699: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:28:47.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:28:47 smithi119 ceph-mon[31518]: pgmap v5699: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:28:49.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:28:49 smithi165 ceph-mon[36302]: pgmap v5700: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:28:49.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:28:49 smithi119 ceph-mon[31518]: pgmap v5700: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:28:49.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:28:49 smithi132 ceph-mon[35941]: pgmap v5700: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:28:51.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:28:51 smithi165 ceph-mon[36302]: pgmap v5701: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T03:28:51.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:28:51 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10390.808419 secs 2024-03-28T03:28:51.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:28:51 smithi119 ceph-mon[31518]: pgmap v5701: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T03:28:51.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:28:51 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10390.808419 secs 2024-03-28T03:28:51.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:28:51 smithi132 ceph-mon[35941]: pgmap v5701: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T03:28:51.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:28:51 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10390.808419 secs 2024-03-28T03:28:53.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:28:53 smithi165 ceph-mon[36302]: pgmap v5702: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:28:53.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:28:53 smithi119 ceph-mon[31518]: pgmap v5702: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:28:53.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:28:53 smithi132 ceph-mon[35941]: pgmap v5702: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:28:55.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:28:55 smithi165 ceph-mon[36302]: pgmap v5703: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:28:55.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:28:55 smithi119 ceph-mon[31518]: pgmap v5703: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:28:55.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:28:55 smithi132 ceph-mon[35941]: pgmap v5703: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:28:56.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:28:56 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10395.808505 secs 2024-03-28T03:28:56.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:28:56 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10395.808505 secs 2024-03-28T03:28:56.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:28:56 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10395.808505 secs 2024-03-28T03:28:57.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:28:57 smithi165 ceph-mon[36302]: pgmap v5704: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:28:57.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:28:57 smithi119 ceph-mon[31518]: pgmap v5704: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:28:57.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:28:57 smithi132 ceph-mon[35941]: pgmap v5704: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:28:59.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:28:59 smithi165 ceph-mon[36302]: pgmap v5705: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:28:59.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:28:59 smithi119 ceph-mon[31518]: pgmap v5705: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:28:59.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:28:59 smithi132 ceph-mon[35941]: pgmap v5705: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:29:01.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:29:01 smithi165 ceph-mon[36302]: pgmap v5706: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:29:01.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:29:01 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10400.808592 secs 2024-03-28T03:29:01.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:29:01 smithi119 ceph-mon[31518]: pgmap v5706: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:29:01.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:29:01 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10400.808592 secs 2024-03-28T03:29:01.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:29:01 smithi132 ceph-mon[35941]: pgmap v5706: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:29:01.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:29:01 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10400.808592 secs 2024-03-28T03:29:03.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:29:03 smithi165 ceph-mon[36302]: pgmap v5707: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:29:03.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:29:03 smithi119 ceph-mon[31518]: pgmap v5707: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:29:03.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:29:03 smithi132 ceph-mon[35941]: pgmap v5707: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:29:05.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:29:05 smithi165 ceph-mon[36302]: pgmap v5708: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:29:05.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:29:05 smithi119 ceph-mon[31518]: pgmap v5708: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:29:05.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:29:05 smithi132 ceph-mon[35941]: pgmap v5708: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:29:06.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:29:06 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10405.808681 secs 2024-03-28T03:29:06.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:29:06 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10405.808681 secs 2024-03-28T03:29:06.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:29:06 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10405.808681 secs 2024-03-28T03:29:07.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:29:07 smithi165 ceph-mon[36302]: pgmap v5709: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:29:07.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:29:07 smithi119 ceph-mon[31518]: pgmap v5709: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:29:07.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:29:07 smithi132 ceph-mon[35941]: pgmap v5709: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:29:09.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:29:09 smithi165 ceph-mon[36302]: pgmap v5710: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:29:09.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:29:09 smithi119 ceph-mon[31518]: pgmap v5710: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:29:09.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:29:09 smithi132 ceph-mon[35941]: pgmap v5710: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:29:11.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:29:11 smithi165 ceph-mon[36302]: pgmap v5711: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:29:11.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:29:11 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10410.808743 secs 2024-03-28T03:29:11.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:29:11 smithi119 ceph-mon[31518]: pgmap v5711: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:29:11.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:29:11 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10410.808743 secs 2024-03-28T03:29:11.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:29:11 smithi132 ceph-mon[35941]: pgmap v5711: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:29:11.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:29:11 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10410.808743 secs 2024-03-28T03:29:13.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:29:13 smithi165 ceph-mon[36302]: pgmap v5712: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:29:13.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:29:13 smithi119 ceph-mon[31518]: pgmap v5712: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:29:13.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:29:13 smithi132 ceph-mon[35941]: pgmap v5712: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:29:15.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:29:15 smithi165 ceph-mon[36302]: pgmap v5713: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:29:15.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:29:15 smithi119 ceph-mon[31518]: pgmap v5713: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:29:15.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:29:15 smithi132 ceph-mon[35941]: pgmap v5713: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:29:16.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:29:16 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10415.808832 secs 2024-03-28T03:29:16.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:29:16 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10415.808832 secs 2024-03-28T03:29:16.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:29:16 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10415.808832 secs 2024-03-28T03:29:17.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:29:17 smithi165 ceph-mon[36302]: pgmap v5714: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:29:17.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:29:17 smithi119 ceph-mon[31518]: pgmap v5714: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:29:17.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:29:17 smithi132 ceph-mon[35941]: pgmap v5714: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:29:19.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:29:19 smithi119 ceph-mon[31518]: pgmap v5715: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:29:19.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:29:19 smithi132 ceph-mon[35941]: pgmap v5715: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:29:20.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:29:19 smithi165 ceph-mon[36302]: pgmap v5715: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:29:21.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:29:21 smithi119 ceph-mon[31518]: pgmap v5716: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:29:21.903 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:29:21 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10420.808895 secs 2024-03-28T03:29:21.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:29:21 smithi132 ceph-mon[35941]: pgmap v5716: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:29:21.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:29:21 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10420.808895 secs 2024-03-28T03:29:22.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:29:21 smithi165 ceph-mon[36302]: pgmap v5716: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:29:22.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:29:21 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10420.808895 secs 2024-03-28T03:29:23.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:29:23 smithi119 ceph-mon[31518]: pgmap v5717: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:29:23.931 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:29:23 smithi165 ceph-mon[36302]: pgmap v5717: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:29:23.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:29:23 smithi132 ceph-mon[35941]: pgmap v5717: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:29:25.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:29:25 smithi119 ceph-mon[31518]: pgmap v5718: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:29:25.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:29:25 smithi132 ceph-mon[35941]: pgmap v5718: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:29:26.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:29:25 smithi165 ceph-mon[36302]: pgmap v5718: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:29:26.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:29:26 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10425.808931 secs 2024-03-28T03:29:27.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:29:26 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10425.808931 secs 2024-03-28T03:29:27.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:29:26 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10425.808931 secs 2024-03-28T03:29:27.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:29:27 smithi132 ceph-mon[35941]: pgmap v5719: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:29:28.054 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:29:27 smithi119 ceph-mon[31518]: pgmap v5719: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:29:28.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:29:27 smithi165 ceph-mon[36302]: pgmap v5719: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:29:29.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:29:29 smithi132 ceph-mon[35941]: pgmap v5720: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:29:30.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:29:29 smithi165 ceph-mon[36302]: pgmap v5720: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:29:30.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:29:29 smithi119 ceph-mon[31518]: pgmap v5720: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:29:31.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:29:31 smithi132 ceph-mon[35941]: pgmap v5721: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:29:31.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:29:31 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10430.808968 secs 2024-03-28T03:29:32.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:29:31 smithi165 ceph-mon[36302]: pgmap v5721: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:29:32.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:29:31 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10430.808968 secs 2024-03-28T03:29:32.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:29:31 smithi119 ceph-mon[31518]: pgmap v5721: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:29:32.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:29:31 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10430.808968 secs 2024-03-28T03:29:33.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:29:33 smithi132 ceph-mon[35941]: pgmap v5722: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:29:34.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:29:33 smithi165 ceph-mon[36302]: pgmap v5722: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:29:34.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:29:33 smithi119 ceph-mon[31518]: pgmap v5722: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:29:34.578 DEBUG:teuthology.orchestra.run:got remote process result: 124 2024-03-28T03:29:34.579 INFO:tasks.workunit:Stopping ['fs/misc'] on client.0... 2024-03-28T03:29:34.579 DEBUG:teuthology.orchestra.run.smithi119:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-03-28T03:29:35.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:29:35 smithi132 ceph-mon[35941]: pgmap v5723: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:29:36.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:29:35 smithi165 ceph-mon[36302]: pgmap v5723: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:29:36.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:29:35 smithi119 ceph-mon[31518]: pgmap v5723: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:29:36.588 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_3be9a5c9bc793e11e2800a8c0c696e8b46742033/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph-c_3be9a5c9bc793e11e2800a8c0c696e8b46742033/qa/tasks/workunit.py", line 295, in _spawn_on_all_clients p.spawn(_run_tests, ctx, refspec, role, [unit], env, File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_3be9a5c9bc793e11e2800a8c0c696e8b46742033/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test fs/misc/dirfrag.sh) on smithi119 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=3be9a5c9bc793e11e2800a8c0c696e8b46742033 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/dirfrag.sh' 2024-03-28T03:29:36.833 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=3ee65905020244fa95eff32dd1081960 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_3be9a5c9bc793e11e2800a8c0c696e8b46742033/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph-c_3be9a5c9bc793e11e2800a8c0c696e8b46742033/qa/tasks/workunit.py", line 295, in _spawn_on_all_clients p.spawn(_run_tests, ctx, refspec, role, [unit], env, File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_3be9a5c9bc793e11e2800a8c0c696e8b46742033/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test fs/misc/dirfrag.sh) on smithi119 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=3be9a5c9bc793e11e2800a8c0c696e8b46742033 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/dirfrag.sh' 2024-03-28T03:29:36.836 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-03-28T03:29:36.873 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-03-28T03:29:36.873 INFO:teuthology.task.exec:Executing custom commands... 2024-03-28T03:29:36.873 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi119.front.sepia.ceph.com 2024-03-28T03:29:36.874 DEBUG:teuthology.orchestra.run.smithi119:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-03-28T03:29:36.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:29:36 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10435.809056 secs 2024-03-28T03:29:36.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:29:36 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10435.809056 secs 2024-03-28T03:29:36.979 DEBUG:teuthology.orchestra.run.smithi119:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-03-28T03:29:37.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:29:36 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10435.809056 secs 2024-03-28T03:29:37.137 DEBUG:teuthology.orchestra.run.smithi119:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-03-28T03:29:37.403 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:29:37.399+0000 7fc47a600640 1 -- 172.21.15.119:0/3106848506 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc474102010 msgr2=0x7fc47410e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T03:29:37.403 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:29:37.399+0000 7fc47a600640 1 --2- 172.21.15.119:0/3106848506 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc474102010 0x7fc47410e520 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fc46c009930 tx=0x7fc46c02bc30 comp rx=0 tx=0).stop 2024-03-28T03:29:37.403 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:29:37.400+0000 7fc47a600640 1 -- 172.21.15.119:0/3106848506 shutdown_connections 2024-03-28T03:29:37.404 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:29:37.400+0000 7fc47a600640 1 --2- 172.21.15.119:0/3106848506 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc474102010 0x7fc47410e520 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:29:37.404 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:29:37.400+0000 7fc47a600640 1 --2- 172.21.15.119:0/3106848506 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc474101650 0x7fc474101ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:29:37.404 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:29:37.400+0000 7fc47a600640 1 --2- 172.21.15.119:0/3106848506 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc474107660 0x7fc474107a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:29:37.404 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:29:37.400+0000 7fc47a600640 1 -- 172.21.15.119:0/3106848506 >> 172.21.15.119:0/3106848506 conn(0x7fc4740fd160 msgr2=0x7fc4740ff580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T03:29:37.404 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:29:37.400+0000 7fc47a600640 1 -- 172.21.15.119:0/3106848506 shutdown_connections 2024-03-28T03:29:37.404 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:29:37.400+0000 7fc47a600640 1 -- 172.21.15.119:0/3106848506 wait complete. 2024-03-28T03:29:37.406 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:29:37.401+0000 7fc47a600640 1 Processor -- start 2024-03-28T03:29:37.407 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:29:37.401+0000 7fc47a600640 1 -- start start 2024-03-28T03:29:37.407 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:29:37.401+0000 7fc47a600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc474101650 0x7fc4741085a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T03:29:37.407 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:29:37.401+0000 7fc47a600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc474102010 0x7fc474108ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T03:29:37.407 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:29:37.401+0000 7fc47a600640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc474107660 0x7fc474109020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T03:29:37.407 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:29:37.401+0000 7fc47a600640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc474111ac0 con 0x7fc474101650 2024-03-28T03:29:37.407 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:29:37.401+0000 7fc47a600640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc474111940 con 0x7fc474107660 2024-03-28T03:29:37.407 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:29:37.401+0000 7fc47a600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc474111c40 con 0x7fc474102010 2024-03-28T03:29:37.407 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:29:37.401+0000 7fc472a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc474102010 0x7fc474108ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T03:29:37.408 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:29:37.401+0000 7fc472a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc474102010 0x7fc474108ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.119:47574/0 (socket says 172.21.15.119:47574) 2024-03-28T03:29:37.408 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:29:37.401+0000 7fc472a00640 1 -- 172.21.15.119:0/210561562 learned_addr learned my addr 172.21.15.119:0/210561562 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T03:29:37.408 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:29:37.401+0000 7fc473400640 1 --2- 172.21.15.119:0/210561562 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc474101650 0x7fc4741085a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T03:29:37.408 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:29:37.401+0000 7fc473e00640 1 --2- 172.21.15.119:0/210561562 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc474107660 0x7fc474109020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T03:29:37.408 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:29:37.402+0000 7fc472a00640 1 -- 172.21.15.119:0/210561562 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc474107660 msgr2=0x7fc474109020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T03:29:37.408 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:29:37.402+0000 7fc472a00640 1 --2- 172.21.15.119:0/210561562 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc474107660 0x7fc474109020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:29:37.408 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:29:37.402+0000 7fc472a00640 1 -- 172.21.15.119:0/210561562 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc474101650 msgr2=0x7fc4741085a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T03:29:37.408 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:29:37.402+0000 7fc472a00640 1 --2- 172.21.15.119:0/210561562 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc474101650 0x7fc4741085a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:29:37.409 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:29:37.402+0000 7fc472a00640 1 -- 172.21.15.119:0/210561562 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4741aa6e0 con 0x7fc474102010 2024-03-28T03:29:37.409 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:29:37.402+0000 7fc473400640 1 --2- 172.21.15.119:0/210561562 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc474101650 0x7fc4741085a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T03:29:37.409 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:29:37.402+0000 7fc472a00640 1 --2- 172.21.15.119:0/210561562 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc474102010 0x7fc474108ae0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fc45c00e9f0 tx=0x7fc45c00edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T03:29:37.409 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:29:37.402+0000 7fc46be00640 1 -- 172.21.15.119:0/210561562 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc45c00cd80 con 0x7fc474102010 2024-03-28T03:29:37.409 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:29:37.402+0000 7fc46be00640 1 -- 172.21.15.119:0/210561562 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc45c005500 con 0x7fc474102010 2024-03-28T03:29:37.409 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:29:37.402+0000 7fc47a600640 1 -- 172.21.15.119:0/210561562 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4741aa9d0 con 0x7fc474102010 2024-03-28T03:29:37.409 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:29:37.402+0000 7fc46be00640 1 -- 172.21.15.119:0/210561562 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc45c010690 con 0x7fc474102010 2024-03-28T03:29:37.409 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:29:37.403+0000 7fc47a600640 1 -- 172.21.15.119:0/210561562 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4741aaf20 con 0x7fc474102010 2024-03-28T03:29:37.410 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:29:37.406+0000 7fc46be00640 1 -- 172.21.15.119:0/210561562 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 27) v1 ==== 137422+0+0 (secure 0 0 0) 0x7fc45c0046c0 con 0x7fc474102010 2024-03-28T03:29:37.410 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:29:37.407+0000 7fc46be00640 1 --2- 172.21.15.119:0/210561562 >> [v2:172.21.15.119:6800/4273482239,v1:172.21.15.119:6801/4273482239] conn(0x7fc4580a2a90 0x7fc4580a4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T03:29:37.410 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:29:37.407+0000 7fc46be00640 1 -- 172.21.15.119:0/210561562 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 11163+0+0 (secure 0 0 0) 0x7fc45c0cfce0 con 0x7fc474102010 2024-03-28T03:29:37.411 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:29:37.407+0000 7fc473400640 1 --2- 172.21.15.119:0/210561562 >> [v2:172.21.15.119:6800/4273482239,v1:172.21.15.119:6801/4273482239] conn(0x7fc4580a2a90 0x7fc4580a4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T03:29:37.415 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:29:37.404+0000 7fc47a600640 1 -- 172.21.15.119:0/210561562 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc474102d90 con 0x7fc474102010 2024-03-28T03:29:37.416 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:29:37.413+0000 7fc473400640 1 --2- 172.21.15.119:0/210561562 >> [v2:172.21.15.119:6800/4273482239,v1:172.21.15.119:6801/4273482239] conn(0x7fc4580a2a90 0x7fc4580a4f50 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fc464005fd0 tx=0x7fc464009500 comp rx=0 tx=0).ready entity=mgr.25277 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T03:29:37.425 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:29:37.422+0000 7fc46be00640 1 -- 172.21.15.119:0/210561562 <== mon.1 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fc45c09bff0 con 0x7fc474102010 2024-03-28T03:29:37.668 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:29:37.665+0000 7fc47a600640 1 -- 172.21.15.119:0/210561562 --> [v2:172.21.15.119:6800/4273482239,v1:172.21.15.119:6801/4273482239] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) v1 -- 0x7fc4740fe550 con 0x7fc4580a2a90 2024-03-28T03:29:37.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:29:37 smithi132 ceph-mon[35941]: pgmap v5724: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:29:38.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:29:37 smithi165 ceph-mon[36302]: pgmap v5724: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:29:38.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:29:37 smithi119 ceph-mon[31518]: pgmap v5724: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:29:38.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:29:38 smithi132 ceph-mon[35941]: from='client.25564 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T03:29:39.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:29:38 smithi165 ceph-mon[36302]: from='client.25564 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T03:29:39.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:29:38 smithi119 ceph-mon[31518]: from='client.25564 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T03:29:40.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:29:39 smithi165 ceph-mon[36302]: pgmap v5725: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:29:40.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:29:39 smithi119 ceph-mon[31518]: pgmap v5725: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:29:40.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:29:39 smithi132 ceph-mon[35941]: pgmap v5725: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:29:41.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:29:40 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T03:29:41.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:29:40 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T03:29:41.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:29:40 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T03:29:42.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:29:41 smithi165 ceph-mon[36302]: pgmap v5726: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:29:42.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:29:41 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10440.809145 secs 2024-03-28T03:29:42.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:29:41 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T03:29:42.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:29:41 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T03:29:42.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:29:41 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:29:42.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:29:41 smithi119 ceph-mon[31518]: pgmap v5726: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:29:42.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:29:41 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10440.809145 secs 2024-03-28T03:29:42.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:29:41 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T03:29:42.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:29:41 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T03:29:42.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:29:41 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:29:42.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:29:41 smithi132 ceph-mon[35941]: pgmap v5726: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:29:42.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:29:41 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10440.809145 secs 2024-03-28T03:29:42.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:29:41 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T03:29:42.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:29:41 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T03:29:42.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:29:41 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:29:43.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:29:42 smithi165 ceph-mon[36302]: pgmap v5727: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T03:29:43.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:29:42 smithi165 ceph-mon[36302]: pgmap v5728: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:29:43.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:29:42 smithi119 ceph-mon[31518]: pgmap v5727: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T03:29:43.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:29:42 smithi119 ceph-mon[31518]: pgmap v5728: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:29:43.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:29:42 smithi132 ceph-mon[35941]: pgmap v5727: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T03:29:43.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:29:42 smithi132 ceph-mon[35941]: pgmap v5728: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:29:45.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:29:44 smithi165 ceph-mon[36302]: pgmap v5729: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:29:45.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:29:44 smithi119 ceph-mon[31518]: pgmap v5729: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:29:45.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:29:44 smithi132 ceph-mon[35941]: pgmap v5729: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:29:47.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:29:46 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10445.809236 secs 2024-03-28T03:29:47.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:29:46 smithi165 ceph-mon[36302]: pgmap v5730: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:29:47.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:29:46 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10445.809236 secs 2024-03-28T03:29:47.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:29:46 smithi119 ceph-mon[31518]: pgmap v5730: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:29:47.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:29:46 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10445.809236 secs 2024-03-28T03:29:47.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:29:46 smithi132 ceph-mon[35941]: pgmap v5730: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:29:49.002 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:29:48 smithi132 ceph-mon[35941]: pgmap v5731: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:29:49.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:29:48 smithi165 ceph-mon[36302]: pgmap v5731: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:29:49.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:29:48 smithi119 ceph-mon[31518]: pgmap v5731: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:29:51.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:29:50 smithi165 ceph-mon[36302]: pgmap v5732: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:29:51.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:29:50 smithi119 ceph-mon[31518]: pgmap v5732: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:29:51.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:29:50 smithi132 ceph-mon[35941]: pgmap v5732: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:29:52.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:29:51 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10450.809296 secs 2024-03-28T03:29:52.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:29:51 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10450.809296 secs 2024-03-28T03:29:52.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:29:51 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10450.809296 secs 2024-03-28T03:29:53.031 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:29:52 smithi165 ceph-mon[36302]: pgmap v5733: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T03:29:53.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:29:52 smithi119 ceph-mon[31518]: pgmap v5733: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T03:29:53.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:29:52 smithi132 ceph-mon[35941]: pgmap v5733: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T03:29:55.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:29:54 smithi165 ceph-mon[36302]: pgmap v5734: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:29:55.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:29:54 smithi119 ceph-mon[31518]: pgmap v5734: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:29:55.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:29:54 smithi132 ceph-mon[35941]: pgmap v5734: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:29:57.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:29:56 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10455.809333 secs 2024-03-28T03:29:57.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:29:56 smithi165 ceph-mon[36302]: pgmap v5735: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:29:57.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:29:56 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10455.809333 secs 2024-03-28T03:29:57.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:29:56 smithi119 ceph-mon[31518]: pgmap v5735: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:29:57.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:29:56 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10455.809333 secs 2024-03-28T03:29:57.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:29:56 smithi132 ceph-mon[35941]: pgmap v5735: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:29:59.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:29:58 smithi165 ceph-mon[36302]: pgmap v5736: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:29:59.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:29:58 smithi119 ceph-mon[31518]: pgmap v5736: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:29:59.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:29:58 smithi132 ceph-mon[35941]: pgmap v5736: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:30:01.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:30:00 smithi165 ceph-mon[36302]: pgmap v5737: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:30:01.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:30:00 smithi165 ceph-mon[36302]: overall HEALTH_WARN 5 failed cephadm daemon(s); 1 filesystem is degraded; 1 MDSs report slow requests 2024-03-28T03:30:01.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:30:00 smithi119 ceph-mon[31518]: pgmap v5737: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:30:01.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:30:00 smithi119 ceph-mon[31518]: overall HEALTH_WARN 5 failed cephadm daemon(s); 1 filesystem is degraded; 1 MDSs report slow requests 2024-03-28T03:30:01.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:30:00 smithi132 ceph-mon[35941]: pgmap v5737: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:30:01.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:30:00 smithi132 ceph-mon[35941]: overall HEALTH_WARN 5 failed cephadm daemon(s); 1 filesystem is degraded; 1 MDSs report slow requests 2024-03-28T03:30:02.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:30:01 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10460.809393 secs 2024-03-28T03:30:02.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:30:01 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10460.809393 secs 2024-03-28T03:30:02.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:30:01 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10460.809393 secs 2024-03-28T03:30:03.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:30:02 smithi165 ceph-mon[36302]: pgmap v5738: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:30:03.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:30:02 smithi119 ceph-mon[31518]: pgmap v5738: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:30:03.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:30:02 smithi132 ceph-mon[35941]: pgmap v5738: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:30:05.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:30:04 smithi165 ceph-mon[36302]: pgmap v5739: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:30:05.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:30:04 smithi119 ceph-mon[31518]: pgmap v5739: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:30:05.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:30:04 smithi132 ceph-mon[35941]: pgmap v5739: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:30:07.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:30:06 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10465.809456 secs 2024-03-28T03:30:07.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:30:06 smithi165 ceph-mon[36302]: pgmap v5740: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:30:07.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:30:06 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10465.809456 secs 2024-03-28T03:30:07.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:30:06 smithi119 ceph-mon[31518]: pgmap v5740: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:30:07.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:30:06 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10465.809456 secs 2024-03-28T03:30:07.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:30:06 smithi132 ceph-mon[35941]: pgmap v5740: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:30:09.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:30:08 smithi165 ceph-mon[36302]: pgmap v5741: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:30:09.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:30:08 smithi119 ceph-mon[31518]: pgmap v5741: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:30:09.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:30:08 smithi132 ceph-mon[35941]: pgmap v5741: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:30:11.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:30:10 smithi165 ceph-mon[36302]: pgmap v5742: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:30:11.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:30:10 smithi119 ceph-mon[31518]: pgmap v5742: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:30:11.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:30:10 smithi132 ceph-mon[35941]: pgmap v5742: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:30:12.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:30:11 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10470.809517 secs 2024-03-28T03:30:12.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:30:11 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10470.809517 secs 2024-03-28T03:30:12.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:30:11 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10470.809517 secs 2024-03-28T03:30:13.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:30:12 smithi165 ceph-mon[36302]: pgmap v5743: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:30:13.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:30:12 smithi119 ceph-mon[31518]: pgmap v5743: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:30:13.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:30:12 smithi132 ceph-mon[35941]: pgmap v5743: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:30:15.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:30:14 smithi165 ceph-mon[36302]: pgmap v5744: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:30:15.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:30:14 smithi119 ceph-mon[31518]: pgmap v5744: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:30:15.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:30:14 smithi132 ceph-mon[35941]: pgmap v5744: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:30:17.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:30:16 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10475.809605 secs 2024-03-28T03:30:17.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:30:16 smithi165 ceph-mon[36302]: pgmap v5745: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:30:17.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:30:16 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10475.809605 secs 2024-03-28T03:30:17.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:30:16 smithi119 ceph-mon[31518]: pgmap v5745: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:30:17.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:30:16 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10475.809605 secs 2024-03-28T03:30:17.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:30:16 smithi132 ceph-mon[35941]: pgmap v5745: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:30:19.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:30:18 smithi165 ceph-mon[36302]: pgmap v5746: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:30:19.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:30:18 smithi119 ceph-mon[31518]: pgmap v5746: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:30:19.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:30:18 smithi132 ceph-mon[35941]: pgmap v5746: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:30:21.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:30:20 smithi165 ceph-mon[36302]: pgmap v5747: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:30:21.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:30:20 smithi119 ceph-mon[31518]: pgmap v5747: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:30:21.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:30:20 smithi132 ceph-mon[35941]: pgmap v5747: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:30:22.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:30:21 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10480.809692 secs 2024-03-28T03:30:22.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:30:21 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10480.809692 secs 2024-03-28T03:30:22.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:30:21 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10480.809692 secs 2024-03-28T03:30:23.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:30:22 smithi165 ceph-mon[36302]: pgmap v5748: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:30:23.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:30:22 smithi119 ceph-mon[31518]: pgmap v5748: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:30:23.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:30:22 smithi132 ceph-mon[35941]: pgmap v5748: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:30:25.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:30:24 smithi165 ceph-mon[36302]: pgmap v5749: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:30:25.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:30:24 smithi119 ceph-mon[31518]: pgmap v5749: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:30:25.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:30:24 smithi132 ceph-mon[35941]: pgmap v5749: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:30:27.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:30:26 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10485.809783 secs 2024-03-28T03:30:27.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:30:26 smithi165 ceph-mon[36302]: pgmap v5750: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:30:27.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:30:26 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10485.809783 secs 2024-03-28T03:30:27.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:30:26 smithi119 ceph-mon[31518]: pgmap v5750: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:30:27.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:30:26 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10485.809783 secs 2024-03-28T03:30:27.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:30:26 smithi132 ceph-mon[35941]: pgmap v5750: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:30:29.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:30:28 smithi165 ceph-mon[36302]: pgmap v5751: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:30:29.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:30:28 smithi119 ceph-mon[31518]: pgmap v5751: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:30:29.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:30:28 smithi132 ceph-mon[35941]: pgmap v5751: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:30:31.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:30:30 smithi165 ceph-mon[36302]: pgmap v5752: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:30:31.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:30:30 smithi119 ceph-mon[31518]: pgmap v5752: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:30:31.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:30:30 smithi132 ceph-mon[35941]: pgmap v5752: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:30:32.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:30:31 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10490.809847 secs 2024-03-28T03:30:32.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:30:31 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10490.809847 secs 2024-03-28T03:30:32.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:30:31 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10490.809847 secs 2024-03-28T03:30:33.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:30:32 smithi165 ceph-mon[36302]: pgmap v5753: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:30:33.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:30:32 smithi119 ceph-mon[31518]: pgmap v5753: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:30:33.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:30:32 smithi132 ceph-mon[35941]: pgmap v5753: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:30:35.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:30:34 smithi119 ceph-mon[31518]: pgmap v5754: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:30:35.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:30:34 smithi132 ceph-mon[35941]: pgmap v5754: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:30:35.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:30:34 smithi165 ceph-mon[36302]: pgmap v5754: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:30:37.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:30:36 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10495.809909 secs 2024-03-28T03:30:37.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:30:36 smithi119 ceph-mon[31518]: pgmap v5755: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:30:37.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:30:36 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10495.809909 secs 2024-03-28T03:30:37.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:30:36 smithi132 ceph-mon[35941]: pgmap v5755: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:30:37.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:30:36 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10495.809909 secs 2024-03-28T03:30:37.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:30:36 smithi165 ceph-mon[36302]: pgmap v5755: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:30:39.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:30:38 smithi119 ceph-mon[31518]: pgmap v5756: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:30:39.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:30:38 smithi132 ceph-mon[35941]: pgmap v5756: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:30:39.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:30:38 smithi165 ceph-mon[36302]: pgmap v5756: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:30:41.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:30:40 smithi119 ceph-mon[31518]: pgmap v5757: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:30:41.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:30:40 smithi132 ceph-mon[35941]: pgmap v5757: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:30:41.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:30:40 smithi165 ceph-mon[36302]: pgmap v5757: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:30:42.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:30:41 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10500.809993 secs 2024-03-28T03:30:42.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:30:41 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T03:30:42.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:30:41 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10500.809993 secs 2024-03-28T03:30:42.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:30:41 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T03:30:42.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:30:41 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10500.809993 secs 2024-03-28T03:30:42.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:30:41 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T03:30:43.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:30:42 smithi132 ceph-mon[35941]: pgmap v5758: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T03:30:43.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:30:42 smithi165 ceph-mon[36302]: pgmap v5758: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T03:30:43.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:30:42 smithi119 ceph-mon[31518]: pgmap v5758: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T03:30:44.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:30:43 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T03:30:44.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:30:43 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T03:30:44.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:30:43 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:30:44.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:30:43 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T03:30:44.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:30:43 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T03:30:44.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:30:43 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:30:44.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:30:43 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T03:30:44.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:30:43 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T03:30:44.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:30:43 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:30:45.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:30:44 smithi132 ceph-mon[35941]: pgmap v5759: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T03:30:45.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:30:44 smithi132 ceph-mon[35941]: pgmap v5760: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:30:45.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:30:44 smithi165 ceph-mon[36302]: pgmap v5759: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T03:30:45.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:30:44 smithi165 ceph-mon[36302]: pgmap v5760: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:30:45.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:30:44 smithi119 ceph-mon[31518]: pgmap v5759: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T03:30:45.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:30:44 smithi119 ceph-mon[31518]: pgmap v5760: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:30:46.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:30:45 smithi165 ceph-mon[36302]: pgmap v5761: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:30:46.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:30:45 smithi119 ceph-mon[31518]: pgmap v5761: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:30:46.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:30:45 smithi132 ceph-mon[35941]: pgmap v5761: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:30:47.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:30:46 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10505.810079 secs 2024-03-28T03:30:47.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:30:46 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10505.810079 secs 2024-03-28T03:30:47.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:30:46 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10505.810079 secs 2024-03-28T03:30:48.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:30:47 smithi165 ceph-mon[36302]: pgmap v5762: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:30:48.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:30:47 smithi119 ceph-mon[31518]: pgmap v5762: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:30:48.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:30:47 smithi132 ceph-mon[35941]: pgmap v5762: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:30:50.252 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:30:49 smithi132 ceph-mon[35941]: pgmap v5763: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-28T03:30:50.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:30:49 smithi165 ceph-mon[36302]: pgmap v5763: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-28T03:30:50.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:30:49 smithi119 ceph-mon[31518]: pgmap v5763: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-28T03:30:52.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:30:51 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10510.810154 secs 2024-03-28T03:30:52.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:30:51 smithi165 ceph-mon[36302]: pgmap v5764: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:30:52.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:30:51 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10510.810154 secs 2024-03-28T03:30:52.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:30:51 smithi119 ceph-mon[31518]: pgmap v5764: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:30:52.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:30:51 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10510.810154 secs 2024-03-28T03:30:52.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:30:51 smithi132 ceph-mon[35941]: pgmap v5764: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:30:54.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:30:53 smithi165 ceph-mon[36302]: pgmap v5765: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T03:30:54.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:30:53 smithi119 ceph-mon[31518]: pgmap v5765: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T03:30:54.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:30:53 smithi132 ceph-mon[35941]: pgmap v5765: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T03:30:56.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:30:55 smithi165 ceph-mon[36302]: pgmap v5766: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:30:56.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:30:55 smithi119 ceph-mon[31518]: pgmap v5766: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:30:56.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:30:55 smithi132 ceph-mon[35941]: pgmap v5766: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:30:57.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:30:57 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10515.810195 secs 2024-03-28T03:30:57.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:30:57 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10515.810195 secs 2024-03-28T03:30:57.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:30:57 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10515.810195 secs 2024-03-28T03:30:58.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:30:58 smithi165 ceph-mon[36302]: pgmap v5767: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:30:58.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:30:58 smithi119 ceph-mon[31518]: pgmap v5767: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:30:58.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:30:58 smithi132 ceph-mon[35941]: pgmap v5767: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:31:00.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:31:00 smithi165 ceph-mon[36302]: pgmap v5768: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:31:00.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:31:00 smithi119 ceph-mon[31518]: pgmap v5768: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:31:00.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:31:00 smithi132 ceph-mon[35941]: pgmap v5768: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:31:02.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:31:02 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10520.810256 secs 2024-03-28T03:31:02.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:31:02 smithi165 ceph-mon[36302]: pgmap v5769: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:31:02.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:31:02 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10520.810256 secs 2024-03-28T03:31:02.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:31:02 smithi119 ceph-mon[31518]: pgmap v5769: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:31:02.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:31:02 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10520.810256 secs 2024-03-28T03:31:02.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:31:02 smithi132 ceph-mon[35941]: pgmap v5769: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:31:04.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:31:04 smithi165 ceph-mon[36302]: pgmap v5770: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:31:04.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:31:04 smithi119 ceph-mon[31518]: pgmap v5770: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:31:04.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:31:04 smithi132 ceph-mon[35941]: pgmap v5770: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:31:06.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:31:06 smithi165 ceph-mon[36302]: pgmap v5771: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:31:06.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:31:06 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10525.810344 secs 2024-03-28T03:31:06.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:31:06 smithi119 ceph-mon[31518]: pgmap v5771: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:31:06.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:31:06 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10525.810344 secs 2024-03-28T03:31:06.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:31:06 smithi132 ceph-mon[35941]: pgmap v5771: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:31:06.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:31:06 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10525.810344 secs 2024-03-28T03:31:08.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:31:08 smithi165 ceph-mon[36302]: pgmap v5772: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:31:08.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:31:08 smithi119 ceph-mon[31518]: pgmap v5772: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:31:08.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:31:08 smithi132 ceph-mon[35941]: pgmap v5772: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:31:10.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:31:10 smithi165 ceph-mon[36302]: pgmap v5773: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:31:10.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:31:10 smithi119 ceph-mon[31518]: pgmap v5773: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:31:10.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:31:10 smithi132 ceph-mon[35941]: pgmap v5773: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:31:11.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:31:11 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10530.810432 secs 2024-03-28T03:31:11.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:31:11 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10530.810432 secs 2024-03-28T03:31:11.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:31:11 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10530.810432 secs 2024-03-28T03:31:12.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:31:12 smithi165 ceph-mon[36302]: pgmap v5774: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:31:12.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:31:12 smithi119 ceph-mon[31518]: pgmap v5774: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:31:12.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:31:12 smithi132 ceph-mon[35941]: pgmap v5774: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:31:14.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:31:14 smithi165 ceph-mon[36302]: pgmap v5775: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:31:14.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:31:14 smithi119 ceph-mon[31518]: pgmap v5775: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:31:14.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:31:14 smithi132 ceph-mon[35941]: pgmap v5775: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:31:16.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:31:16 smithi165 ceph-mon[36302]: pgmap v5776: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:31:16.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:31:16 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10535.810519 secs 2024-03-28T03:31:16.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:31:16 smithi119 ceph-mon[31518]: pgmap v5776: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:31:16.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:31:16 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10535.810519 secs 2024-03-28T03:31:16.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:31:16 smithi132 ceph-mon[35941]: pgmap v5776: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:31:16.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:31:16 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10535.810519 secs 2024-03-28T03:31:18.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:31:18 smithi165 ceph-mon[36302]: pgmap v5777: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:31:18.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:31:18 smithi119 ceph-mon[31518]: pgmap v5777: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:31:18.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:31:18 smithi132 ceph-mon[35941]: pgmap v5777: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:31:20.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:31:20 smithi165 ceph-mon[36302]: pgmap v5778: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:31:20.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:31:20 smithi119 ceph-mon[31518]: pgmap v5778: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:31:20.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:31:20 smithi132 ceph-mon[35941]: pgmap v5778: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:31:21.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:31:21 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10540.810607 secs 2024-03-28T03:31:21.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:31:21 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10540.810607 secs 2024-03-28T03:31:21.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:31:21 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10540.810607 secs 2024-03-28T03:31:22.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:31:22 smithi119 ceph-mon[31518]: pgmap v5779: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:31:22.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:31:22 smithi132 ceph-mon[35941]: pgmap v5779: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:31:22.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:31:22 smithi165 ceph-mon[36302]: pgmap v5779: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:31:24.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:31:24 smithi119 ceph-mon[31518]: pgmap v5780: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:31:24.430 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:31:24 smithi165 ceph-mon[36302]: pgmap v5780: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:31:24.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:31:24 smithi132 ceph-mon[35941]: pgmap v5780: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:31:26.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:31:26 smithi119 ceph-mon[31518]: pgmap v5781: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:31:26.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:31:26 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10545.810692 secs 2024-03-28T03:31:26.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:31:26 smithi132 ceph-mon[35941]: pgmap v5781: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:31:26.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:31:26 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10545.810692 secs 2024-03-28T03:31:26.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:31:26 smithi165 ceph-mon[36302]: pgmap v5781: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:31:26.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:31:26 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10545.810692 secs 2024-03-28T03:31:28.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:31:28 smithi119 ceph-mon[31518]: pgmap v5782: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:31:28.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:31:28 smithi132 ceph-mon[35941]: pgmap v5782: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:31:28.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:31:28 smithi165 ceph-mon[36302]: pgmap v5782: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:31:30.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:31:30 smithi132 ceph-mon[35941]: pgmap v5783: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:31:30.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:31:30 smithi165 ceph-mon[36302]: pgmap v5783: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:31:30.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:31:30 smithi119 ceph-mon[31518]: pgmap v5783: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:31:31.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:31:31 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10550.810780 secs 2024-03-28T03:31:31.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:31:31 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10550.810780 secs 2024-03-28T03:31:31.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:31:31 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10550.810780 secs 2024-03-28T03:31:32.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:31:32 smithi132 ceph-mon[35941]: pgmap v5784: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:31:32.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:31:32 smithi165 ceph-mon[36302]: pgmap v5784: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:31:32.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:31:32 smithi119 ceph-mon[31518]: pgmap v5784: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:31:34.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:31:34 smithi132 ceph-mon[35941]: pgmap v5785: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:31:34.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:31:34 smithi165 ceph-mon[36302]: pgmap v5785: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:31:34.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:31:34 smithi119 ceph-mon[31518]: pgmap v5785: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:31:36.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:31:36 smithi132 ceph-mon[35941]: pgmap v5786: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:31:36.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:31:36 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10555.810840 secs 2024-03-28T03:31:36.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:31:36 smithi165 ceph-mon[36302]: pgmap v5786: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:31:36.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:31:36 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10555.810840 secs 2024-03-28T03:31:36.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:31:36 smithi119 ceph-mon[31518]: pgmap v5786: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:31:36.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:31:36 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10555.810840 secs 2024-03-28T03:31:38.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:31:38 smithi132 ceph-mon[35941]: pgmap v5787: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:31:38.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:31:38 smithi165 ceph-mon[36302]: pgmap v5787: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:31:38.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:31:38 smithi119 ceph-mon[31518]: pgmap v5787: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:31:40.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:31:40 smithi132 ceph-mon[35941]: pgmap v5788: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:31:40.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:31:40 smithi165 ceph-mon[36302]: pgmap v5788: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:31:40.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:31:40 smithi119 ceph-mon[31518]: pgmap v5788: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:31:41.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:31:41 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10560.810884 secs 2024-03-28T03:31:41.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:31:41 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10560.810884 secs 2024-03-28T03:31:41.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:31:41 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10560.810884 secs 2024-03-28T03:31:42.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:31:42 smithi132 ceph-mon[35941]: pgmap v5789: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:31:42.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:31:42 smithi165 ceph-mon[36302]: pgmap v5789: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:31:42.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:31:42 smithi119 ceph-mon[31518]: pgmap v5789: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:31:44.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:31:44 smithi132 ceph-mon[35941]: pgmap v5790: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:31:44.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:31:44 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T03:31:44.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:31:44 smithi165 ceph-mon[36302]: pgmap v5790: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:31:44.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:31:44 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T03:31:44.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:31:44 smithi119 ceph-mon[31518]: pgmap v5790: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:31:44.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:31:44 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T03:31:45.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:31:45 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T03:31:45.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:31:45 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T03:31:45.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:31:45 smithi165 ceph-mon[36302]: pgmap v5791: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T03:31:45.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:31:45 smithi165 ceph-mon[36302]: pgmap v5792: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:31:45.630 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:31:45 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:31:45.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:31:45 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T03:31:45.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:31:45 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T03:31:45.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:31:45 smithi119 ceph-mon[31518]: pgmap v5791: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T03:31:45.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:31:45 smithi119 ceph-mon[31518]: pgmap v5792: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:31:45.653 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:31:45 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:31:45.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:31:45 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T03:31:45.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:31:45 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T03:31:45.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:31:45 smithi132 ceph-mon[35941]: pgmap v5791: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T03:31:45.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:31:45 smithi132 ceph-mon[35941]: pgmap v5792: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:31:45.700 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:31:45 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:31:46.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:31:46 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10565.810946 secs 2024-03-28T03:31:46.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:31:46 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10565.810946 secs 2024-03-28T03:31:46.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:31:46 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10565.810946 secs 2024-03-28T03:31:47.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:31:47 smithi165 ceph-mon[36302]: pgmap v5793: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:31:47.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:31:47 smithi119 ceph-mon[31518]: pgmap v5793: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:31:47.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:31:47 smithi132 ceph-mon[35941]: pgmap v5793: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:31:49.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:31:49 smithi165 ceph-mon[36302]: pgmap v5794: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:31:49.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:31:49 smithi119 ceph-mon[31518]: pgmap v5794: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:31:49.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:31:49 smithi132 ceph-mon[35941]: pgmap v5794: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:31:51.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:31:51 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10570.811030 secs 2024-03-28T03:31:51.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:31:51 smithi165 ceph-mon[36302]: pgmap v5795: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:31:51.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:31:51 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10570.811030 secs 2024-03-28T03:31:51.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:31:51 smithi119 ceph-mon[31518]: pgmap v5795: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:31:51.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:31:51 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10570.811030 secs 2024-03-28T03:31:51.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:31:51 smithi132 ceph-mon[35941]: pgmap v5795: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:31:53.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:31:53 smithi165 ceph-mon[36302]: pgmap v5796: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:31:53.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:31:53 smithi119 ceph-mon[31518]: pgmap v5796: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:31:53.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:31:53 smithi132 ceph-mon[35941]: pgmap v5796: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:31:55.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:31:55 smithi165 ceph-mon[36302]: pgmap v5797: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T03:31:55.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:31:55 smithi119 ceph-mon[31518]: pgmap v5797: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T03:31:55.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:31:55 smithi132 ceph-mon[35941]: pgmap v5797: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T03:31:56.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:31:56 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10575.811121 secs 2024-03-28T03:31:56.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:31:56 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10575.811121 secs 2024-03-28T03:31:56.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:31:56 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10575.811121 secs 2024-03-28T03:31:57.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:31:57 smithi165 ceph-mon[36302]: pgmap v5798: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:31:57.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:31:57 smithi119 ceph-mon[31518]: pgmap v5798: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:31:57.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:31:57 smithi132 ceph-mon[35941]: pgmap v5798: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:31:59.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:31:59 smithi165 ceph-mon[36302]: pgmap v5799: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:31:59.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:31:59 smithi119 ceph-mon[31518]: pgmap v5799: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:31:59.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:31:59 smithi132 ceph-mon[35941]: pgmap v5799: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:32:01.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:32:01 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10580.811209 secs 2024-03-28T03:32:01.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:32:01 smithi165 ceph-mon[36302]: pgmap v5800: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:32:01.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:32:01 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10580.811209 secs 2024-03-28T03:32:01.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:32:01 smithi119 ceph-mon[31518]: pgmap v5800: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:32:01.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:32:01 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10580.811209 secs 2024-03-28T03:32:01.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:32:01 smithi132 ceph-mon[35941]: pgmap v5800: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:32:03.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:32:03 smithi165 ceph-mon[36302]: pgmap v5801: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:32:03.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:32:03 smithi119 ceph-mon[31518]: pgmap v5801: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:32:03.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:32:03 smithi132 ceph-mon[35941]: pgmap v5801: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:32:05.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:32:05 smithi165 ceph-mon[36302]: pgmap v5802: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:32:05.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:32:05 smithi119 ceph-mon[31518]: pgmap v5802: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:32:05.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:32:05 smithi132 ceph-mon[35941]: pgmap v5802: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:32:06.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:32:06 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10585.811297 secs 2024-03-28T03:32:06.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:32:06 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10585.811297 secs 2024-03-28T03:32:06.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:32:06 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10585.811297 secs 2024-03-28T03:32:07.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:32:07 smithi165 ceph-mon[36302]: pgmap v5803: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:32:07.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:32:07 smithi119 ceph-mon[31518]: pgmap v5803: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:32:07.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:32:07 smithi132 ceph-mon[35941]: pgmap v5803: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:32:09.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:32:09 smithi165 ceph-mon[36302]: pgmap v5804: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:32:09.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:32:09 smithi119 ceph-mon[31518]: pgmap v5804: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:32:09.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:32:09 smithi132 ceph-mon[35941]: pgmap v5804: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:32:11.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:32:11 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10590.811313 secs 2024-03-28T03:32:11.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:32:11 smithi165 ceph-mon[36302]: pgmap v5805: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:32:11.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:32:11 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10590.811313 secs 2024-03-28T03:32:11.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:32:11 smithi119 ceph-mon[31518]: pgmap v5805: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:32:11.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:32:11 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10590.811313 secs 2024-03-28T03:32:11.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:32:11 smithi132 ceph-mon[35941]: pgmap v5805: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:32:13.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:32:13 smithi165 ceph-mon[36302]: pgmap v5806: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:32:13.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:32:13 smithi119 ceph-mon[31518]: pgmap v5806: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:32:13.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:32:13 smithi132 ceph-mon[35941]: pgmap v5806: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:32:15.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:32:15 smithi165 ceph-mon[36302]: pgmap v5807: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:32:15.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:32:15 smithi119 ceph-mon[31518]: pgmap v5807: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:32:15.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:32:15 smithi132 ceph-mon[35941]: pgmap v5807: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:32:16.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:32:16 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10595.811402 secs 2024-03-28T03:32:16.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:32:16 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10595.811402 secs 2024-03-28T03:32:16.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:32:16 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10595.811402 secs 2024-03-28T03:32:17.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:32:17 smithi165 ceph-mon[36302]: pgmap v5808: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:32:17.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:32:17 smithi119 ceph-mon[31518]: pgmap v5808: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:32:17.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:32:17 smithi132 ceph-mon[35941]: pgmap v5808: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:32:19.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:32:19 smithi165 ceph-mon[36302]: pgmap v5809: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:32:19.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:32:19 smithi119 ceph-mon[31518]: pgmap v5809: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:32:19.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:32:19 smithi132 ceph-mon[35941]: pgmap v5809: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:32:21.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:32:21 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10600.811486 secs 2024-03-28T03:32:21.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:32:21 smithi165 ceph-mon[36302]: pgmap v5810: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:32:21.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:32:21 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10600.811486 secs 2024-03-28T03:32:21.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:32:21 smithi119 ceph-mon[31518]: pgmap v5810: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:32:21.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:32:21 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10600.811486 secs 2024-03-28T03:32:21.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:32:21 smithi132 ceph-mon[35941]: pgmap v5810: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:32:23.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:32:23 smithi165 ceph-mon[36302]: pgmap v5811: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:32:23.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:32:23 smithi119 ceph-mon[31518]: pgmap v5811: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:32:23.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:32:23 smithi132 ceph-mon[35941]: pgmap v5811: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:32:25.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:32:25 smithi165 ceph-mon[36302]: pgmap v5812: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:32:25.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:32:25 smithi119 ceph-mon[31518]: pgmap v5812: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:32:25.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:32:25 smithi132 ceph-mon[35941]: pgmap v5812: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:32:26.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:32:26 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10605.811577 secs 2024-03-28T03:32:26.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:32:26 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10605.811577 secs 2024-03-28T03:32:26.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:32:26 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10605.811577 secs 2024-03-28T03:32:27.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:32:27 smithi165 ceph-mon[36302]: pgmap v5813: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:32:27.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:32:27 smithi119 ceph-mon[31518]: pgmap v5813: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:32:27.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:32:27 smithi132 ceph-mon[35941]: pgmap v5813: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:32:29.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:32:29 smithi165 ceph-mon[36302]: pgmap v5814: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:32:29.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:32:29 smithi119 ceph-mon[31518]: pgmap v5814: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:32:29.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:32:29 smithi132 ceph-mon[35941]: pgmap v5814: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:32:31.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:32:31 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10610.811662 secs 2024-03-28T03:32:31.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:32:31 smithi119 ceph-mon[31518]: pgmap v5815: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:32:31.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:32:31 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10610.811662 secs 2024-03-28T03:32:31.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:32:31 smithi132 ceph-mon[35941]: pgmap v5815: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:32:32.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:32:31 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10610.811662 secs 2024-03-28T03:32:32.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:32:31 smithi165 ceph-mon[36302]: pgmap v5815: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:32:33.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:32:33 smithi119 ceph-mon[31518]: pgmap v5816: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:32:33.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:32:33 smithi132 ceph-mon[35941]: pgmap v5816: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:32:34.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:32:33 smithi165 ceph-mon[36302]: pgmap v5816: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:32:35.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:32:35 smithi119 ceph-mon[31518]: pgmap v5817: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:32:35.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:32:35 smithi132 ceph-mon[35941]: pgmap v5817: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:32:36.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:32:35 smithi165 ceph-mon[36302]: pgmap v5817: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:32:36.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:32:36 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10615.811752 secs 2024-03-28T03:32:36.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:32:36 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10615.811752 secs 2024-03-28T03:32:37.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:32:36 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10615.811752 secs 2024-03-28T03:32:37.433 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:32:37.431+0000 7fc46aa00640 1 -- 172.21.15.119:0/210561562 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc44c0034c0 con 0x7fc474102010 2024-03-28T03:32:37.434 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:32:37.432+0000 7fc46be00640 1 -- 172.21.15.119:0/210561562 <== mon.1 v2:172.21.15.165:3300/0 7 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (secure 0 0 0) 0x7fc44c0034c0 con 0x7fc474102010 2024-03-28T03:32:37.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:32:37 smithi132 ceph-mon[35941]: pgmap v5818: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T03:32:38.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:32:37 smithi165 ceph-mon[36302]: pgmap v5818: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T03:32:38.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:32:37 smithi119 ceph-mon[31518]: pgmap v5818: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T03:32:39.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:32:39 smithi132 ceph-mon[35941]: pgmap v5819: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T03:32:40.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:32:39 smithi165 ceph-mon[36302]: pgmap v5819: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T03:32:40.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:32:39 smithi119 ceph-mon[31518]: pgmap v5819: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T03:32:41.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:32:41 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10620.811842 secs 2024-03-28T03:32:41.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:32:41 smithi132 ceph-mon[35941]: pgmap v5820: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:32:42.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:32:41 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10620.811842 secs 2024-03-28T03:32:42.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:32:41 smithi165 ceph-mon[36302]: pgmap v5820: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:32:42.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:32:41 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10620.811842 secs 2024-03-28T03:32:42.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:32:41 smithi119 ceph-mon[31518]: pgmap v5820: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:32:43.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:32:43 smithi132 ceph-mon[35941]: pgmap v5821: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T03:32:44.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:32:43 smithi165 ceph-mon[36302]: pgmap v5821: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T03:32:44.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:32:43 smithi119 ceph-mon[31518]: pgmap v5821: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T03:32:45.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:32:45 smithi132 ceph-mon[35941]: pgmap v5822: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:32:45.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:32:45 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T03:32:46.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:32:45 smithi165 ceph-mon[36302]: pgmap v5822: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:32:46.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:32:45 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T03:32:46.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:32:45 smithi119 ceph-mon[31518]: pgmap v5822: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:32:46.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:32:45 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T03:32:46.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:32:46 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10625.811899 secs 2024-03-28T03:32:46.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:32:46 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T03:32:46.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:32:46 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T03:32:46.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:32:46 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:32:47.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:32:46 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10625.811899 secs 2024-03-28T03:32:47.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:32:46 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T03:32:47.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:32:46 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T03:32:47.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:32:46 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:32:47.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:32:46 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10625.811899 secs 2024-03-28T03:32:47.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:32:46 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T03:32:47.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:32:46 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T03:32:47.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:32:46 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:32:47.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:32:47 smithi132 ceph-mon[35941]: pgmap v5823: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T03:32:47.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:32:47 smithi132 ceph-mon[35941]: pgmap v5824: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:32:48.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:32:47 smithi165 ceph-mon[36302]: pgmap v5823: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T03:32:48.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:32:47 smithi165 ceph-mon[36302]: pgmap v5824: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:32:48.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:32:47 smithi119 ceph-mon[31518]: pgmap v5823: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T03:32:48.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:32:47 smithi119 ceph-mon[31518]: pgmap v5824: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:32:49.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:32:49 smithi132 ceph-mon[35941]: pgmap v5825: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:32:50.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:32:49 smithi165 ceph-mon[36302]: pgmap v5825: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:32:50.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:32:49 smithi119 ceph-mon[31518]: pgmap v5825: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:32:51.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:32:51 smithi132 ceph-mon[35941]: pgmap v5826: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:32:51.950 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:32:51 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10630.811962 secs 2024-03-28T03:32:52.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:32:51 smithi165 ceph-mon[36302]: pgmap v5826: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:32:52.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:32:51 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10630.811962 secs 2024-03-28T03:32:52.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:32:51 smithi119 ceph-mon[31518]: pgmap v5826: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:32:52.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:32:51 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10630.811962 secs 2024-03-28T03:32:53.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:32:53 smithi132 ceph-mon[35941]: pgmap v5827: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:32:54.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:32:53 smithi165 ceph-mon[36302]: pgmap v5827: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:32:54.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:32:53 smithi119 ceph-mon[31518]: pgmap v5827: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:32:56.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:32:55 smithi165 ceph-mon[36302]: pgmap v5828: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-28T03:32:56.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:32:55 smithi119 ceph-mon[31518]: pgmap v5828: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-28T03:32:56.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:32:55 smithi132 ceph-mon[35941]: pgmap v5828: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-28T03:32:57.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:32:56 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10635.811980 secs 2024-03-28T03:32:57.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:32:56 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10635.811980 secs 2024-03-28T03:32:57.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:32:56 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10635.811980 secs 2024-03-28T03:32:58.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:32:57 smithi165 ceph-mon[36302]: pgmap v5829: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T03:32:58.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:32:57 smithi119 ceph-mon[31518]: pgmap v5829: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T03:32:58.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:32:57 smithi132 ceph-mon[35941]: pgmap v5829: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T03:33:00.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:32:59 smithi165 ceph-mon[36302]: pgmap v5830: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:33:00.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:32:59 smithi119 ceph-mon[31518]: pgmap v5830: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:33:00.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:32:59 smithi132 ceph-mon[35941]: pgmap v5830: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:33:02.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:33:01 smithi165 ceph-mon[36302]: pgmap v5831: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:33:02.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:33:01 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10640.811996 secs 2024-03-28T03:33:02.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:33:01 smithi119 ceph-mon[31518]: pgmap v5831: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:33:02.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:33:01 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10640.811996 secs 2024-03-28T03:33:02.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:33:01 smithi132 ceph-mon[35941]: pgmap v5831: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:33:02.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:33:01 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10640.811996 secs 2024-03-28T03:33:04.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:33:03 smithi165 ceph-mon[36302]: pgmap v5832: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:33:04.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:33:03 smithi119 ceph-mon[31518]: pgmap v5832: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:33:04.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:33:03 smithi132 ceph-mon[35941]: pgmap v5832: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:33:06.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:33:05 smithi165 ceph-mon[36302]: pgmap v5833: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:33:06.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:33:05 smithi119 ceph-mon[31518]: pgmap v5833: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:33:06.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:33:05 smithi132 ceph-mon[35941]: pgmap v5833: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:33:07.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:33:06 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10645.812059 secs 2024-03-28T03:33:07.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:33:06 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10645.812059 secs 2024-03-28T03:33:07.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:33:06 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10645.812059 secs 2024-03-28T03:33:08.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:33:07 smithi165 ceph-mon[36302]: pgmap v5834: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:33:08.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:33:07 smithi119 ceph-mon[31518]: pgmap v5834: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:33:08.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:33:07 smithi132 ceph-mon[35941]: pgmap v5834: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:33:10.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:33:09 smithi165 ceph-mon[36302]: pgmap v5835: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:33:10.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:33:09 smithi119 ceph-mon[31518]: pgmap v5835: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:33:10.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:33:09 smithi132 ceph-mon[35941]: pgmap v5835: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:33:12.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:33:11 smithi165 ceph-mon[36302]: pgmap v5836: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:33:12.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:33:11 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10650.812120 secs 2024-03-28T03:33:12.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:33:11 smithi119 ceph-mon[31518]: pgmap v5836: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:33:12.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:33:11 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10650.812120 secs 2024-03-28T03:33:12.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:33:11 smithi132 ceph-mon[35941]: pgmap v5836: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:33:12.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:33:11 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10650.812120 secs 2024-03-28T03:33:14.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:33:13 smithi165 ceph-mon[36302]: pgmap v5837: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:33:14.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:33:13 smithi119 ceph-mon[31518]: pgmap v5837: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:33:14.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:33:13 smithi132 ceph-mon[35941]: pgmap v5837: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:33:16.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:33:15 smithi165 ceph-mon[36302]: pgmap v5838: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:33:16.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:33:15 smithi119 ceph-mon[31518]: pgmap v5838: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:33:16.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:33:15 smithi132 ceph-mon[35941]: pgmap v5838: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:33:17.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:33:16 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10655.812211 secs 2024-03-28T03:33:17.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:33:16 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10655.812211 secs 2024-03-28T03:33:17.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:33:16 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10655.812211 secs 2024-03-28T03:33:18.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:33:17 smithi165 ceph-mon[36302]: pgmap v5839: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:33:18.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:33:17 smithi119 ceph-mon[31518]: pgmap v5839: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:33:18.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:33:17 smithi132 ceph-mon[35941]: pgmap v5839: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:33:20.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:33:19 smithi165 ceph-mon[36302]: pgmap v5840: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:33:20.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:33:19 smithi119 ceph-mon[31518]: pgmap v5840: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:33:20.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:33:19 smithi132 ceph-mon[35941]: pgmap v5840: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:33:22.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:33:21 smithi165 ceph-mon[36302]: pgmap v5841: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:33:22.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:33:21 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10660.812271 secs 2024-03-28T03:33:22.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:33:21 smithi119 ceph-mon[31518]: pgmap v5841: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:33:22.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:33:21 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10660.812271 secs 2024-03-28T03:33:22.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:33:21 smithi132 ceph-mon[35941]: pgmap v5841: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:33:22.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:33:21 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10660.812271 secs 2024-03-28T03:33:24.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:33:23 smithi165 ceph-mon[36302]: pgmap v5842: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:33:24.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:33:23 smithi119 ceph-mon[31518]: pgmap v5842: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:33:24.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:33:23 smithi132 ceph-mon[35941]: pgmap v5842: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:33:26.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:33:25 smithi165 ceph-mon[36302]: pgmap v5843: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:33:26.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:33:25 smithi119 ceph-mon[31518]: pgmap v5843: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:33:26.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:33:25 smithi132 ceph-mon[35941]: pgmap v5843: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:33:27.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:33:26 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10665.812309 secs 2024-03-28T03:33:27.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:33:26 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10665.812309 secs 2024-03-28T03:33:27.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:33:26 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10665.812309 secs 2024-03-28T03:33:28.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:33:27 smithi165 ceph-mon[36302]: pgmap v5844: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:33:28.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:33:27 smithi119 ceph-mon[31518]: pgmap v5844: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:33:28.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:33:27 smithi132 ceph-mon[35941]: pgmap v5844: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:33:30.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:33:29 smithi165 ceph-mon[36302]: pgmap v5845: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:33:30.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:33:29 smithi119 ceph-mon[31518]: pgmap v5845: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:33:30.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:33:29 smithi132 ceph-mon[35941]: pgmap v5845: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:33:32.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:33:31 smithi165 ceph-mon[36302]: pgmap v5846: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:33:32.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:33:31 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10670.812334 secs 2024-03-28T03:33:32.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:33:31 smithi119 ceph-mon[31518]: pgmap v5846: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:33:32.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:33:31 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10670.812334 secs 2024-03-28T03:33:32.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:33:31 smithi132 ceph-mon[35941]: pgmap v5846: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:33:32.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:33:31 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10670.812334 secs 2024-03-28T03:33:34.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:33:33 smithi165 ceph-mon[36302]: pgmap v5847: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:33:34.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:33:33 smithi119 ceph-mon[31518]: pgmap v5847: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:33:34.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:33:33 smithi132 ceph-mon[35941]: pgmap v5847: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:33:36.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:33:35 smithi165 ceph-mon[36302]: pgmap v5848: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:33:36.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:33:35 smithi119 ceph-mon[31518]: pgmap v5848: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:33:36.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:33:35 smithi132 ceph-mon[35941]: pgmap v5848: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:33:37.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:33:36 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10675.812393 secs 2024-03-28T03:33:37.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:33:36 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10675.812393 secs 2024-03-28T03:33:37.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:33:36 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10675.812393 secs 2024-03-28T03:33:38.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:33:37 smithi165 ceph-mon[36302]: pgmap v5849: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:33:38.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:33:37 smithi119 ceph-mon[31518]: pgmap v5849: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:33:38.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:33:37 smithi132 ceph-mon[35941]: pgmap v5849: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:33:40.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:33:39 smithi165 ceph-mon[36302]: pgmap v5850: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:33:40.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:33:39 smithi119 ceph-mon[31518]: pgmap v5850: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:33:40.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:33:39 smithi132 ceph-mon[35941]: pgmap v5850: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:33:42.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:33:41 smithi165 ceph-mon[36302]: pgmap v5851: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:33:42.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:33:41 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10680.812481 secs 2024-03-28T03:33:42.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:33:41 smithi119 ceph-mon[31518]: pgmap v5851: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:33:42.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:33:41 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10680.812481 secs 2024-03-28T03:33:42.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:33:41 smithi132 ceph-mon[35941]: pgmap v5851: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:33:42.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:33:41 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10680.812481 secs 2024-03-28T03:33:44.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:33:43 smithi165 ceph-mon[36302]: pgmap v5852: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:33:44.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:33:43 smithi119 ceph-mon[31518]: pgmap v5852: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:33:44.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:33:43 smithi132 ceph-mon[35941]: pgmap v5852: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:33:46.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:33:45 smithi165 ceph-mon[36302]: pgmap v5853: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:33:46.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:33:45 smithi119 ceph-mon[31518]: pgmap v5853: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:33:46.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:33:45 smithi132 ceph-mon[35941]: pgmap v5853: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:33:47.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:33:46 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10685.812569 secs 2024-03-28T03:33:47.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:33:46 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T03:33:47.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:33:46 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10685.812569 secs 2024-03-28T03:33:47.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:33:46 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T03:33:47.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:33:46 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10685.812569 secs 2024-03-28T03:33:47.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:33:46 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T03:33:48.195 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:33:47 smithi119 ceph-mon[31518]: pgmap v5854: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:33:48.195 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:33:47 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:33:48.195 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:33:47 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:33:48.196 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:33:47 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:33:48.196 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:33:47 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:33:48.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:33:47 smithi132 ceph-mon[35941]: pgmap v5854: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:33:48.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:33:47 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:33:48.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:33:47 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:33:48.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:33:47 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:33:48.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:33:47 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:33:48.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:33:47 smithi165 ceph-mon[36302]: pgmap v5854: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:33:48.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:33:47 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:33:48.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:33:47 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:33:48.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:33:47 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:33:48.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:33:47 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:33:50.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:33:49 smithi165 ceph-mon[36302]: pgmap v5855: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:33:50.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:33:49 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:33:50.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:33:49 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:33:50.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:33:49 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:33:50.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:33:49 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:33:50.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:33:49 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:33:50.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:33:49 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:33:50.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:33:49 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:33:50.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:33:49 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:33:50.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:33:49 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:33:50.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:33:49 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:33:50.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:33:49 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:33:50.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:33:49 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:33:50.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:33:49 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:33:50.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:33:49 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:33:50.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:33:49 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:33:50.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:33:49 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:33:50.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:33:49 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:33:50.131 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:33:49 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:33:50.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:33:49 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:33:50.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:33:49 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:33:50.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:33:49 smithi119 ceph-mon[31518]: pgmap v5855: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:33:50.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:33:49 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:33:50.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:33:49 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:33:50.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:33:49 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:33:50.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:33:49 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:33:50.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:33:49 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:33:50.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:33:49 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:33:50.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:33:49 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:33:50.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:33:49 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:33:50.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:33:49 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:33:50.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:33:49 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:33:50.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:33:49 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:33:50.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:33:49 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:33:50.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:33:49 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:33:50.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:33:49 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:33:50.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:33:49 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:33:50.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:33:49 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:33:50.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:33:49 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:33:50.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:33:49 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:33:50.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:33:49 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:33:50.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:33:49 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:33:50.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:33:49 smithi132 ceph-mon[35941]: pgmap v5855: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:33:50.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:33:49 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:33:50.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:33:49 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:33:50.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:33:49 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:33:50.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:33:49 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:33:50.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:33:49 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:33:50.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:33:49 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:33:50.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:33:49 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:33:50.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:33:49 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:33:50.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:33:49 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:33:50.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:33:49 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:33:50.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:33:49 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:33:50.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:33:49 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:33:50.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:33:49 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:33:50.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:33:49 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:33:50.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:33:49 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:33:50.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:33:49 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:33:50.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:33:49 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:33:50.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:33:49 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:33:50.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:33:49 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:33:50.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:33:49 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:33:51.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:33:50 smithi165 ceph-mon[36302]: Adjusting osd_memory_target on smithi132 to 174.1M 2024-03-28T03:33:51.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:33:50 smithi165 ceph-mon[36302]: Unable to set osd_memory_target on smithi132 to 182640640: error parsing value: Value '182640640' is below minimum 939524096 2024-03-28T03:33:51.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:33:50 smithi165 ceph-mon[36302]: Adjusting osd_memory_target on smithi165 to 174.1M 2024-03-28T03:33:51.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:33:50 smithi165 ceph-mon[36302]: Unable to set osd_memory_target on smithi165 to 182635622: error parsing value: Value '182635622' is below minimum 939524096 2024-03-28T03:33:51.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:33:50 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:33:51.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:33:50 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:33:51.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:33:50 smithi119 ceph-mon[31518]: Adjusting osd_memory_target on smithi132 to 174.1M 2024-03-28T03:33:51.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:33:50 smithi119 ceph-mon[31518]: Unable to set osd_memory_target on smithi132 to 182640640: error parsing value: Value '182640640' is below minimum 939524096 2024-03-28T03:33:51.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:33:50 smithi119 ceph-mon[31518]: Adjusting osd_memory_target on smithi165 to 174.1M 2024-03-28T03:33:51.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:33:50 smithi119 ceph-mon[31518]: Unable to set osd_memory_target on smithi165 to 182635622: error parsing value: Value '182635622' is below minimum 939524096 2024-03-28T03:33:51.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:33:50 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:33:51.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:33:50 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:33:51.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:33:50 smithi132 ceph-mon[35941]: Adjusting osd_memory_target on smithi132 to 174.1M 2024-03-28T03:33:51.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:33:50 smithi132 ceph-mon[35941]: Unable to set osd_memory_target on smithi132 to 182640640: error parsing value: Value '182640640' is below minimum 939524096 2024-03-28T03:33:51.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:33:50 smithi132 ceph-mon[35941]: Adjusting osd_memory_target on smithi165 to 174.1M 2024-03-28T03:33:51.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:33:50 smithi132 ceph-mon[35941]: Unable to set osd_memory_target on smithi165 to 182635622: error parsing value: Value '182635622' is below minimum 939524096 2024-03-28T03:33:51.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:33:50 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:33:51.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:33:50 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:33:52.100 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:33:51 smithi132 ceph-mon[35941]: pgmap v5856: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:33:52.100 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:33:51 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10690.812656 secs 2024-03-28T03:33:52.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:33:51 smithi165 ceph-mon[36302]: pgmap v5856: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:33:52.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:33:51 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10690.812656 secs 2024-03-28T03:33:52.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:33:51 smithi119 ceph-mon[31518]: pgmap v5856: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:33:52.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:33:51 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10690.812656 secs 2024-03-28T03:33:53.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:33:52 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:33:53.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:33:52 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:33:53.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:33:52 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:33:53.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:33:52 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:33:53.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:33:52 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:33:53.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:33:52 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:33:53.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:33:52 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:33:53.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:33:52 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:33:53.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:33:52 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T03:33:53.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:33:52 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T03:33:53.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:33:52 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:33:53.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:33:52 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:33:53.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:33:52 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:33:53.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:33:52 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:33:53.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:33:52 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:33:53.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:33:52 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:33:53.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:33:52 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:33:53.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:33:52 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:33:53.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:33:52 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:33:53.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:33:52 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T03:33:53.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:33:52 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T03:33:53.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:33:52 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:33:53.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:33:52 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:33:53.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:33:52 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:33:53.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:33:52 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:33:53.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:33:52 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:33:53.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:33:52 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:33:53.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:33:52 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:33:53.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:33:52 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:33:53.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:33:52 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:33:53.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:33:52 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T03:33:53.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:33:52 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T03:33:53.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:33:52 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:33:54.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:33:53 smithi165 ceph-mon[36302]: Adjusting osd_memory_target on smithi119 to 176.7M 2024-03-28T03:33:54.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:33:53 smithi165 ceph-mon[36302]: Unable to set osd_memory_target on smithi119 to 185372364: error parsing value: Value '185372364' is below minimum 939524096 2024-03-28T03:33:54.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:33:53 smithi165 ceph-mon[36302]: pgmap v5857: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T03:33:54.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:33:53 smithi165 ceph-mon[36302]: pgmap v5858: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T03:33:54.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:33:53 smithi119 ceph-mon[31518]: Adjusting osd_memory_target on smithi119 to 176.7M 2024-03-28T03:33:54.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:33:53 smithi119 ceph-mon[31518]: Unable to set osd_memory_target on smithi119 to 185372364: error parsing value: Value '185372364' is below minimum 939524096 2024-03-28T03:33:54.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:33:53 smithi119 ceph-mon[31518]: pgmap v5857: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T03:33:54.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:33:53 smithi119 ceph-mon[31518]: pgmap v5858: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T03:33:54.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:33:53 smithi132 ceph-mon[35941]: Adjusting osd_memory_target on smithi119 to 176.7M 2024-03-28T03:33:54.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:33:53 smithi132 ceph-mon[35941]: Unable to set osd_memory_target on smithi119 to 185372364: error parsing value: Value '185372364' is below minimum 939524096 2024-03-28T03:33:54.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:33:53 smithi132 ceph-mon[35941]: pgmap v5857: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T03:33:54.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:33:53 smithi132 ceph-mon[35941]: pgmap v5858: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T03:33:56.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:33:55 smithi165 ceph-mon[36302]: pgmap v5859: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T03:33:56.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:33:55 smithi119 ceph-mon[31518]: pgmap v5859: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T03:33:56.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:33:55 smithi132 ceph-mon[35941]: pgmap v5859: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T03:33:57.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:33:56 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10695.812744 secs 2024-03-28T03:33:57.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:33:56 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10695.812744 secs 2024-03-28T03:33:57.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:33:56 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10695.812744 secs 2024-03-28T03:33:58.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:33:57 smithi165 ceph-mon[36302]: pgmap v5860: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T03:33:58.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:33:57 smithi119 ceph-mon[31518]: pgmap v5860: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T03:33:58.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:33:57 smithi132 ceph-mon[35941]: pgmap v5860: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T03:34:00.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:33:59 smithi165 ceph-mon[36302]: pgmap v5861: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T03:34:00.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:33:59 smithi119 ceph-mon[31518]: pgmap v5861: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T03:34:00.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:33:59 smithi132 ceph-mon[35941]: pgmap v5861: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T03:34:02.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:34:01 smithi165 ceph-mon[36302]: pgmap v5862: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T03:34:02.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:34:01 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10700.812806 secs 2024-03-28T03:34:02.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:34:01 smithi119 ceph-mon[31518]: pgmap v5862: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T03:34:02.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:34:01 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10700.812806 secs 2024-03-28T03:34:02.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:34:01 smithi132 ceph-mon[35941]: pgmap v5862: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T03:34:02.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:34:01 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10700.812806 secs 2024-03-28T03:34:04.066 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:34:03 smithi119 ceph-mon[31518]: pgmap v5863: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T03:34:04.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:34:03 smithi165 ceph-mon[36302]: pgmap v5863: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T03:34:04.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:34:03 smithi132 ceph-mon[35941]: pgmap v5863: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T03:34:06.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:34:05 smithi165 ceph-mon[36302]: pgmap v5864: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:34:06.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:34:05 smithi119 ceph-mon[31518]: pgmap v5864: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:34:06.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:34:05 smithi132 ceph-mon[35941]: pgmap v5864: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:34:07.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:34:06 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10705.812896 secs 2024-03-28T03:34:07.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:34:06 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10705.812896 secs 2024-03-28T03:34:07.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:34:06 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10705.812896 secs 2024-03-28T03:34:08.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:34:07 smithi165 ceph-mon[36302]: pgmap v5865: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:34:08.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:34:07 smithi119 ceph-mon[31518]: pgmap v5865: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:34:08.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:34:07 smithi132 ceph-mon[35941]: pgmap v5865: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:34:10.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:34:09 smithi165 ceph-mon[36302]: pgmap v5866: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:34:10.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:34:09 smithi119 ceph-mon[31518]: pgmap v5866: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:34:10.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:34:09 smithi132 ceph-mon[35941]: pgmap v5866: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:34:12.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:34:11 smithi165 ceph-mon[36302]: pgmap v5867: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:34:12.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:34:11 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10710.812980 secs 2024-03-28T03:34:12.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:34:11 smithi119 ceph-mon[31518]: pgmap v5867: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:34:12.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:34:11 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10710.812980 secs 2024-03-28T03:34:12.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:34:11 smithi132 ceph-mon[35941]: pgmap v5867: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:34:12.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:34:11 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10710.812980 secs 2024-03-28T03:34:14.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:34:13 smithi165 ceph-mon[36302]: pgmap v5868: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:34:14.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:34:13 smithi119 ceph-mon[31518]: pgmap v5868: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:34:14.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:34:13 smithi132 ceph-mon[35941]: pgmap v5868: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:34:16.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:34:15 smithi165 ceph-mon[36302]: pgmap v5869: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:34:16.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:34:15 smithi119 ceph-mon[31518]: pgmap v5869: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:34:16.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:34:15 smithi132 ceph-mon[35941]: pgmap v5869: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:34:17.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:34:16 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10715.813018 secs 2024-03-28T03:34:17.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:34:16 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10715.813018 secs 2024-03-28T03:34:17.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:34:16 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10715.813018 secs 2024-03-28T03:34:18.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:34:17 smithi165 ceph-mon[36302]: pgmap v5870: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:34:18.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:34:17 smithi119 ceph-mon[31518]: pgmap v5870: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:34:18.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:34:17 smithi132 ceph-mon[35941]: pgmap v5870: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:34:20.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:34:19 smithi165 ceph-mon[36302]: pgmap v5871: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:34:20.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:34:19 smithi119 ceph-mon[31518]: pgmap v5871: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:34:20.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:34:19 smithi132 ceph-mon[35941]: pgmap v5871: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:34:22.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:34:21 smithi165 ceph-mon[36302]: pgmap v5872: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:34:22.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:34:21 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10720.813078 secs 2024-03-28T03:34:22.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:34:21 smithi119 ceph-mon[31518]: pgmap v5872: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:34:22.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:34:21 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10720.813078 secs 2024-03-28T03:34:22.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:34:21 smithi132 ceph-mon[35941]: pgmap v5872: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:34:22.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:34:21 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10720.813078 secs 2024-03-28T03:34:24.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:34:23 smithi165 ceph-mon[36302]: pgmap v5873: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:34:24.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:34:23 smithi119 ceph-mon[31518]: pgmap v5873: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:34:24.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:34:23 smithi132 ceph-mon[35941]: pgmap v5873: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:34:26.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:34:25 smithi165 ceph-mon[36302]: pgmap v5874: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:34:26.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:34:25 smithi119 ceph-mon[31518]: pgmap v5874: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:34:26.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:34:25 smithi132 ceph-mon[35941]: pgmap v5874: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:34:27.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:34:26 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10725.813139 secs 2024-03-28T03:34:27.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:34:26 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10725.813139 secs 2024-03-28T03:34:27.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:34:26 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10725.813139 secs 2024-03-28T03:34:28.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:34:27 smithi119 ceph-mon[31518]: pgmap v5875: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:34:28.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:34:27 smithi132 ceph-mon[35941]: pgmap v5875: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:34:28.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:34:27 smithi165 ceph-mon[36302]: pgmap v5875: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:34:30.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:34:29 smithi119 ceph-mon[31518]: pgmap v5876: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:34:30.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:34:29 smithi132 ceph-mon[35941]: pgmap v5876: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:34:30.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:34:29 smithi165 ceph-mon[36302]: pgmap v5876: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:34:32.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:34:31 smithi119 ceph-mon[31518]: pgmap v5877: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:34:32.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:34:31 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10730.813228 secs 2024-03-28T03:34:32.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:34:31 smithi132 ceph-mon[35941]: pgmap v5877: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:34:32.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:34:31 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10730.813228 secs 2024-03-28T03:34:32.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:34:31 smithi165 ceph-mon[36302]: pgmap v5877: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:34:32.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:34:31 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10730.813228 secs 2024-03-28T03:34:34.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:34:33 smithi132 ceph-mon[35941]: pgmap v5878: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:34:34.274 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:34:33 smithi119 ceph-mon[31518]: pgmap v5878: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:34:34.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:34:33 smithi165 ceph-mon[36302]: pgmap v5878: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:34:36.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:34:35 smithi132 ceph-mon[35941]: pgmap v5879: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:34:36.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:34:35 smithi165 ceph-mon[36302]: pgmap v5879: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:34:36.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:34:35 smithi119 ceph-mon[31518]: pgmap v5879: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:34:37.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:34:36 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10735.813317 secs 2024-03-28T03:34:37.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:34:36 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10735.813317 secs 2024-03-28T03:34:37.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:34:36 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10735.813317 secs 2024-03-28T03:34:38.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:34:37 smithi132 ceph-mon[35941]: pgmap v5880: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:34:38.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:34:37 smithi165 ceph-mon[36302]: pgmap v5880: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:34:38.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:34:37 smithi119 ceph-mon[31518]: pgmap v5880: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:34:40.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:34:39 smithi132 ceph-mon[35941]: pgmap v5881: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:34:40.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:34:39 smithi165 ceph-mon[36302]: pgmap v5881: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:34:40.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:34:39 smithi119 ceph-mon[31518]: pgmap v5881: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:34:42.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:34:41 smithi132 ceph-mon[35941]: pgmap v5882: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:34:42.200 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:34:41 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10740.813406 secs 2024-03-28T03:34:42.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:34:41 smithi165 ceph-mon[36302]: pgmap v5882: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:34:42.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:34:41 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10740.813406 secs 2024-03-28T03:34:42.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:34:41 smithi119 ceph-mon[31518]: pgmap v5882: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:34:42.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:34:41 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10740.813406 secs 2024-03-28T03:34:44.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:34:43 smithi132 ceph-mon[35941]: pgmap v5883: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:34:44.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:34:43 smithi165 ceph-mon[36302]: pgmap v5883: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:34:44.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:34:43 smithi119 ceph-mon[31518]: pgmap v5883: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:34:46.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:34:45 smithi132 ceph-mon[35941]: pgmap v5884: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:34:46.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:34:45 smithi165 ceph-mon[36302]: pgmap v5884: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:34:46.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:34:45 smithi119 ceph-mon[31518]: pgmap v5884: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:34:47.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:34:46 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10745.813468 secs 2024-03-28T03:34:47.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:34:46 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10745.813468 secs 2024-03-28T03:34:47.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:34:46 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10745.813468 secs 2024-03-28T03:34:48.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:34:47 smithi165 ceph-mon[36302]: pgmap v5885: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:34:48.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:34:47 smithi119 ceph-mon[31518]: pgmap v5885: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:34:48.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:34:47 smithi132 ceph-mon[35941]: pgmap v5885: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:34:50.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:34:49 smithi165 ceph-mon[36302]: pgmap v5886: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:34:50.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:34:49 smithi119 ceph-mon[31518]: pgmap v5886: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:34:50.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:34:49 smithi132 ceph-mon[35941]: pgmap v5886: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:34:52.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:34:51 smithi165 ceph-mon[36302]: pgmap v5887: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:34:52.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:34:51 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10750.813556 secs 2024-03-28T03:34:52.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:34:51 smithi119 ceph-mon[31518]: pgmap v5887: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:34:52.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:34:51 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10750.813556 secs 2024-03-28T03:34:52.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:34:51 smithi132 ceph-mon[35941]: pgmap v5887: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:34:52.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:34:51 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10750.813556 secs 2024-03-28T03:34:53.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:34:52 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T03:34:53.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:34:52 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:34:53.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:34:52 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:34:53.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:34:52 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T03:34:53.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:34:52 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:34:53.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:34:52 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:34:53.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:34:52 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T03:34:53.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:34:52 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:34:53.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:34:52 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:34:54.350 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:34:53 smithi132 ceph-mon[35941]: pgmap v5888: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:34:54.350 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:34:53 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T03:34:54.351 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:34:53 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T03:34:54.351 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:34:53 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:34:54.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:34:53 smithi165 ceph-mon[36302]: pgmap v5888: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:34:54.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:34:53 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T03:34:54.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:34:53 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T03:34:54.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:34:53 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:34:54.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:34:53 smithi119 ceph-mon[31518]: pgmap v5888: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:34:54.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:34:53 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T03:34:54.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:34:53 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T03:34:54.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:34:53 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:34:55.280 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:34:54 smithi165 ceph-mon[36302]: pgmap v5889: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T03:34:55.280 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:34:54 smithi165 ceph-mon[36302]: pgmap v5890: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:34:55.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:34:54 smithi119 ceph-mon[31518]: pgmap v5889: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T03:34:55.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:34:54 smithi119 ceph-mon[31518]: pgmap v5890: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:34:55.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:34:54 smithi132 ceph-mon[35941]: pgmap v5889: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T03:34:55.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:34:54 smithi132 ceph-mon[35941]: pgmap v5890: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:34:57.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:34:56 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10755.813573 secs 2024-03-28T03:34:57.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:34:56 smithi165 ceph-mon[36302]: pgmap v5891: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:34:57.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:34:56 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10755.813573 secs 2024-03-28T03:34:57.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:34:56 smithi119 ceph-mon[31518]: pgmap v5891: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:34:57.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:34:56 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10755.813573 secs 2024-03-28T03:34:57.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:34:56 smithi132 ceph-mon[35941]: pgmap v5891: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:34:58.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:34:57 smithi165 ceph-mon[36302]: pgmap v5892: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:34:58.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:34:57 smithi119 ceph-mon[31518]: pgmap v5892: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:34:58.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:34:57 smithi132 ceph-mon[35941]: pgmap v5892: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:35:00.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:35:00 smithi165 ceph-mon[36302]: pgmap v5893: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:35:00.901 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:35:00 smithi119 ceph-mon[31518]: pgmap v5893: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:35:00.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:35:00 smithi132 ceph-mon[35941]: pgmap v5893: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:35:01.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:35:01 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10760.813633 secs 2024-03-28T03:35:01.901 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:35:01 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10760.813633 secs 2024-03-28T03:35:01.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:35:01 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10760.813633 secs 2024-03-28T03:35:02.878 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:35:02 smithi165 ceph-mon[36302]: pgmap v5894: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:35:02.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:35:02 smithi119 ceph-mon[31518]: pgmap v5894: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:35:02.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:35:02 smithi132 ceph-mon[35941]: pgmap v5894: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:35:04.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:35:04 smithi119 ceph-mon[31518]: pgmap v5895: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T03:35:04.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:35:04 smithi132 ceph-mon[35941]: pgmap v5895: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T03:35:05.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:35:04 smithi165 ceph-mon[36302]: pgmap v5895: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T03:35:06.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:35:06 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10765.813674 secs 2024-03-28T03:35:06.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:35:06 smithi119 ceph-mon[31518]: pgmap v5896: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:35:06.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:35:06 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10765.813674 secs 2024-03-28T03:35:06.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:35:06 smithi132 ceph-mon[35941]: pgmap v5896: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:35:07.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:35:06 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10765.813674 secs 2024-03-28T03:35:07.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:35:06 smithi165 ceph-mon[36302]: pgmap v5896: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:35:08.901 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:35:08 smithi119 ceph-mon[31518]: pgmap v5897: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:35:08.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:35:08 smithi132 ceph-mon[35941]: pgmap v5897: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:35:09.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:35:08 smithi165 ceph-mon[36302]: pgmap v5897: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:35:10.902 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:35:10 smithi119 ceph-mon[31518]: pgmap v5898: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:35:10.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:35:10 smithi132 ceph-mon[35941]: pgmap v5898: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:35:11.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:35:10 smithi165 ceph-mon[36302]: pgmap v5898: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:35:11.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:35:11 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10770.813736 secs 2024-03-28T03:35:12.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:35:11 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10770.813736 secs 2024-03-28T03:35:12.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:35:11 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10770.813736 secs 2024-03-28T03:35:12.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:35:12 smithi132 ceph-mon[35941]: pgmap v5899: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:35:13.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:35:12 smithi165 ceph-mon[36302]: pgmap v5899: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:35:13.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:35:12 smithi119 ceph-mon[31518]: pgmap v5899: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:35:14.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:35:14 smithi132 ceph-mon[35941]: pgmap v5900: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:35:15.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:35:14 smithi165 ceph-mon[36302]: pgmap v5900: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:35:15.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:35:14 smithi119 ceph-mon[31518]: pgmap v5900: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:35:16.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:35:16 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10775.813823 secs 2024-03-28T03:35:16.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:35:16 smithi132 ceph-mon[35941]: pgmap v5901: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:35:17.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:35:16 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10775.813823 secs 2024-03-28T03:35:17.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:35:16 smithi165 ceph-mon[36302]: pgmap v5901: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:35:17.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:35:16 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10775.813823 secs 2024-03-28T03:35:17.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:35:16 smithi119 ceph-mon[31518]: pgmap v5901: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:35:18.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:35:18 smithi132 ceph-mon[35941]: pgmap v5902: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:35:19.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:35:18 smithi165 ceph-mon[36302]: pgmap v5902: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:35:19.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:35:18 smithi119 ceph-mon[31518]: pgmap v5902: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:35:20.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:35:20 smithi132 ceph-mon[35941]: pgmap v5903: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:35:21.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:35:20 smithi165 ceph-mon[36302]: pgmap v5903: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:35:21.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:35:20 smithi119 ceph-mon[31518]: pgmap v5903: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:35:21.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:35:21 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10780.813900 secs 2024-03-28T03:35:22.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:35:21 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10780.813900 secs 2024-03-28T03:35:22.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:35:21 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10780.813900 secs 2024-03-28T03:35:22.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:35:22 smithi132 ceph-mon[35941]: pgmap v5904: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:35:23.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:35:22 smithi165 ceph-mon[36302]: pgmap v5904: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:35:23.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:35:22 smithi119 ceph-mon[31518]: pgmap v5904: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:35:24.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:35:24 smithi132 ceph-mon[35941]: pgmap v5905: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:35:25.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:35:24 smithi165 ceph-mon[36302]: pgmap v5905: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:35:25.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:35:24 smithi119 ceph-mon[31518]: pgmap v5905: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:35:26.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:35:26 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10785.813985 secs 2024-03-28T03:35:26.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:35:26 smithi132 ceph-mon[35941]: pgmap v5906: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:35:27.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:35:26 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10785.813985 secs 2024-03-28T03:35:27.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:35:26 smithi165 ceph-mon[36302]: pgmap v5906: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:35:27.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:35:26 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10785.813985 secs 2024-03-28T03:35:27.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:35:26 smithi119 ceph-mon[31518]: pgmap v5906: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:35:29.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:35:28 smithi165 ceph-mon[36302]: pgmap v5907: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:35:29.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:35:28 smithi119 ceph-mon[31518]: pgmap v5907: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:35:29.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:35:28 smithi132 ceph-mon[35941]: pgmap v5907: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:35:31.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:35:30 smithi165 ceph-mon[36302]: pgmap v5908: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:35:31.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:35:30 smithi119 ceph-mon[31518]: pgmap v5908: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:35:31.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:35:30 smithi132 ceph-mon[35941]: pgmap v5908: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:35:32.053 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:35:31 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10790.814071 secs 2024-03-28T03:35:32.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:35:31 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10790.814071 secs 2024-03-28T03:35:32.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:35:31 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10790.814071 secs 2024-03-28T03:35:33.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:35:32 smithi165 ceph-mon[36302]: pgmap v5909: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:35:33.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:35:32 smithi119 ceph-mon[31518]: pgmap v5909: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:35:33.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:35:32 smithi132 ceph-mon[35941]: pgmap v5909: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:35:35.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:35:34 smithi165 ceph-mon[36302]: pgmap v5910: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:35:35.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:35:34 smithi119 ceph-mon[31518]: pgmap v5910: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:35:35.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:35:34 smithi132 ceph-mon[35941]: pgmap v5910: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:35:37.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:35:36 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10795.814134 secs 2024-03-28T03:35:37.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:35:36 smithi165 ceph-mon[36302]: pgmap v5911: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:35:37.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:35:36 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10795.814134 secs 2024-03-28T03:35:37.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:35:36 smithi119 ceph-mon[31518]: pgmap v5911: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:35:37.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:35:36 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10795.814134 secs 2024-03-28T03:35:37.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:35:36 smithi132 ceph-mon[35941]: pgmap v5911: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:35:37.462 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:35:37.460+0000 7fc46aa00640 1 -- 172.21.15.119:0/210561562 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc44c005850 con 0x7fc474102010 2024-03-28T03:35:37.463 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:35:37.461+0000 7fc46be00640 1 -- 172.21.15.119:0/210561562 <== mon.1 v2:172.21.15.165:3300/0 8 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (secure 0 0 0) 0x7fc44c005850 con 0x7fc474102010 2024-03-28T03:35:39.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:35:38 smithi165 ceph-mon[36302]: pgmap v5912: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:35:39.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:35:38 smithi119 ceph-mon[31518]: pgmap v5912: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:35:39.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:35:38 smithi132 ceph-mon[35941]: pgmap v5912: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:35:41.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:35:40 smithi165 ceph-mon[36302]: pgmap v5913: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:35:41.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:35:40 smithi119 ceph-mon[31518]: pgmap v5913: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:35:41.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:35:40 smithi132 ceph-mon[35941]: pgmap v5913: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:35:42.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:35:41 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10800.814218 secs 2024-03-28T03:35:42.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:35:41 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10800.814218 secs 2024-03-28T03:35:42.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:35:41 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10800.814218 secs 2024-03-28T03:35:43.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:35:42 smithi165 ceph-mon[36302]: pgmap v5914: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:35:43.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:35:42 smithi119 ceph-mon[31518]: pgmap v5914: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:35:43.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:35:42 smithi132 ceph-mon[35941]: pgmap v5914: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:35:45.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:35:44 smithi165 ceph-mon[36302]: pgmap v5915: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:35:45.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:35:44 smithi119 ceph-mon[31518]: pgmap v5915: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:35:45.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:35:44 smithi132 ceph-mon[35941]: pgmap v5915: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:35:47.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:35:46 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10805.814256 secs 2024-03-28T03:35:47.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:35:46 smithi165 ceph-mon[36302]: pgmap v5916: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:35:47.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:35:46 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10805.814256 secs 2024-03-28T03:35:47.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:35:46 smithi119 ceph-mon[31518]: pgmap v5916: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:35:47.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:35:46 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10805.814256 secs 2024-03-28T03:35:47.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:35:46 smithi132 ceph-mon[35941]: pgmap v5916: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:35:49.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:35:48 smithi165 ceph-mon[36302]: pgmap v5917: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:35:49.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:35:48 smithi119 ceph-mon[31518]: pgmap v5917: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:35:49.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:35:48 smithi132 ceph-mon[35941]: pgmap v5917: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:35:51.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:35:50 smithi165 ceph-mon[36302]: pgmap v5918: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:35:51.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:35:50 smithi119 ceph-mon[31518]: pgmap v5918: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:35:51.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:35:50 smithi132 ceph-mon[35941]: pgmap v5918: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:35:52.098 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:35:51 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10810.814345 secs 2024-03-28T03:35:52.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:35:51 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10810.814345 secs 2024-03-28T03:35:52.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:35:51 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10810.814345 secs 2024-03-28T03:35:53.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:35:52 smithi165 ceph-mon[36302]: pgmap v5919: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:35:53.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:35:52 smithi119 ceph-mon[31518]: pgmap v5919: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:35:53.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:35:52 smithi132 ceph-mon[35941]: pgmap v5919: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:35:55.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:35:54 smithi165 ceph-mon[36302]: pgmap v5920: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:35:55.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:35:54 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T03:35:55.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:35:54 smithi119 ceph-mon[31518]: pgmap v5920: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:35:55.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:35:54 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T03:35:55.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:35:54 smithi132 ceph-mon[35941]: pgmap v5920: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:35:55.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:35:54 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T03:35:56.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:35:55 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T03:35:56.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:35:55 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T03:35:56.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:35:55 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:35:56.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:35:55 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T03:35:56.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:35:55 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T03:35:56.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:35:55 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:35:56.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:35:55 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T03:35:56.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:35:55 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T03:35:56.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:35:55 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:35:57.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:35:56 smithi165 ceph-mon[36302]: pgmap v5921: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T03:35:57.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:35:56 smithi165 ceph-mon[36302]: pgmap v5922: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:35:57.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:35:56 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10815.814405 secs 2024-03-28T03:35:57.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:35:56 smithi119 ceph-mon[31518]: pgmap v5921: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T03:35:57.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:35:56 smithi119 ceph-mon[31518]: pgmap v5922: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:35:57.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:35:56 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10815.814405 secs 2024-03-28T03:35:57.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:35:56 smithi132 ceph-mon[35941]: pgmap v5921: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T03:35:57.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:35:56 smithi132 ceph-mon[35941]: pgmap v5922: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:35:57.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:35:56 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10815.814405 secs 2024-03-28T03:35:59.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:35:58 smithi165 ceph-mon[36302]: pgmap v5923: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:35:59.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:35:58 smithi119 ceph-mon[31518]: pgmap v5923: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:35:59.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:35:58 smithi132 ceph-mon[35941]: pgmap v5923: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:36:01.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:36:00 smithi165 ceph-mon[36302]: pgmap v5924: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-28T03:36:01.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:36:00 smithi119 ceph-mon[31518]: pgmap v5924: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-28T03:36:01.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:36:00 smithi132 ceph-mon[35941]: pgmap v5924: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-28T03:36:02.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:36:01 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10820.814493 secs 2024-03-28T03:36:02.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:36:01 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10820.814493 secs 2024-03-28T03:36:02.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:36:01 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10820.814493 secs 2024-03-28T03:36:03.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:36:02 smithi165 ceph-mon[36302]: pgmap v5925: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:36:03.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:36:02 smithi119 ceph-mon[31518]: pgmap v5925: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:36:03.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:36:02 smithi132 ceph-mon[35941]: pgmap v5925: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:36:05.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:36:04 smithi165 ceph-mon[36302]: pgmap v5926: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:36:05.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:36:04 smithi119 ceph-mon[31518]: pgmap v5926: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:36:05.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:36:04 smithi132 ceph-mon[35941]: pgmap v5926: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:36:07.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:36:06 smithi165 ceph-mon[36302]: pgmap v5927: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T03:36:07.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:36:06 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10825.814578 secs 2024-03-28T03:36:07.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:36:06 smithi119 ceph-mon[31518]: pgmap v5927: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T03:36:07.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:36:06 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10825.814578 secs 2024-03-28T03:36:07.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:36:06 smithi132 ceph-mon[35941]: pgmap v5927: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T03:36:07.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:36:06 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10825.814578 secs 2024-03-28T03:36:09.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:36:08 smithi165 ceph-mon[36302]: pgmap v5928: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:36:09.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:36:08 smithi119 ceph-mon[31518]: pgmap v5928: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:36:09.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:36:08 smithi132 ceph-mon[35941]: pgmap v5928: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:36:11.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:36:10 smithi165 ceph-mon[36302]: pgmap v5929: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:36:11.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:36:10 smithi119 ceph-mon[31518]: pgmap v5929: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:36:11.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:36:10 smithi132 ceph-mon[35941]: pgmap v5929: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:36:12.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:36:11 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10830.814640 secs 2024-03-28T03:36:12.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:36:11 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10830.814640 secs 2024-03-28T03:36:12.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:36:11 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10830.814640 secs 2024-03-28T03:36:13.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:36:12 smithi165 ceph-mon[36302]: pgmap v5930: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:36:13.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:36:12 smithi119 ceph-mon[31518]: pgmap v5930: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:36:13.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:36:12 smithi132 ceph-mon[35941]: pgmap v5930: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:36:15.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:36:14 smithi165 ceph-mon[36302]: pgmap v5931: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:36:15.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:36:14 smithi119 ceph-mon[31518]: pgmap v5931: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:36:15.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:36:14 smithi132 ceph-mon[35941]: pgmap v5931: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:36:17.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:36:16 smithi165 ceph-mon[36302]: pgmap v5932: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:36:17.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:36:16 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10835.814679 secs 2024-03-28T03:36:17.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:36:16 smithi119 ceph-mon[31518]: pgmap v5932: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:36:17.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:36:16 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10835.814679 secs 2024-03-28T03:36:17.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:36:16 smithi132 ceph-mon[35941]: pgmap v5932: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:36:17.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:36:16 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10835.814679 secs 2024-03-28T03:36:19.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:36:18 smithi165 ceph-mon[36302]: pgmap v5933: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:36:19.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:36:18 smithi119 ceph-mon[31518]: pgmap v5933: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:36:19.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:36:18 smithi132 ceph-mon[35941]: pgmap v5933: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:36:21.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:36:20 smithi165 ceph-mon[36302]: pgmap v5934: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:36:21.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:36:20 smithi119 ceph-mon[31518]: pgmap v5934: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:36:21.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:36:20 smithi132 ceph-mon[35941]: pgmap v5934: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:36:22.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:36:21 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10840.814763 secs 2024-03-28T03:36:22.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:36:21 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10840.814763 secs 2024-03-28T03:36:22.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:36:21 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10840.814763 secs 2024-03-28T03:36:23.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:36:22 smithi119 ceph-mon[31518]: pgmap v5935: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:36:23.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:36:22 smithi132 ceph-mon[35941]: pgmap v5935: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:36:23.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:36:22 smithi165 ceph-mon[36302]: pgmap v5935: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:36:25.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:36:24 smithi119 ceph-mon[31518]: pgmap v5936: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:36:25.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:36:24 smithi132 ceph-mon[35941]: pgmap v5936: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:36:25.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:36:24 smithi165 ceph-mon[36302]: pgmap v5936: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:36:27.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:36:26 smithi119 ceph-mon[31518]: pgmap v5937: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:36:27.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:36:26 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10845.814778 secs 2024-03-28T03:36:27.180 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:36:26 smithi165 ceph-mon[36302]: pgmap v5937: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:36:27.181 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:36:26 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10845.814778 secs 2024-03-28T03:36:27.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:36:26 smithi132 ceph-mon[35941]: pgmap v5937: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:36:27.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:36:26 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10845.814778 secs 2024-03-28T03:36:29.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:36:28 smithi119 ceph-mon[31518]: pgmap v5938: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:36:29.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:36:28 smithi132 ceph-mon[35941]: pgmap v5938: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:36:29.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:36:28 smithi165 ceph-mon[36302]: pgmap v5938: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:36:31.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:36:30 smithi119 ceph-mon[31518]: pgmap v5939: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T03:36:31.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:36:30 smithi132 ceph-mon[35941]: pgmap v5939: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T03:36:31.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:36:30 smithi165 ceph-mon[36302]: pgmap v5939: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T03:36:32.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:36:31 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10850.814838 secs 2024-03-28T03:36:32.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:36:31 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10850.814838 secs 2024-03-28T03:36:32.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:36:31 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10850.814838 secs 2024-03-28T03:36:33.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:36:32 smithi132 ceph-mon[35941]: pgmap v5940: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:36:33.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:36:32 smithi165 ceph-mon[36302]: pgmap v5940: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:36:33.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:36:32 smithi119 ceph-mon[31518]: pgmap v5940: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:36:35.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:36:34 smithi132 ceph-mon[35941]: pgmap v5941: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:36:35.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:36:34 smithi165 ceph-mon[36302]: pgmap v5941: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:36:35.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:36:34 smithi119 ceph-mon[31518]: pgmap v5941: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:36:37.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:36:36 smithi132 ceph-mon[35941]: pgmap v5942: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:36:37.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:36:36 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10855.814927 secs 2024-03-28T03:36:37.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:36:36 smithi165 ceph-mon[36302]: pgmap v5942: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:36:37.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:36:36 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10855.814927 secs 2024-03-28T03:36:37.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:36:36 smithi119 ceph-mon[31518]: pgmap v5942: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:36:37.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:36:36 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10855.814927 secs 2024-03-28T03:36:39.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:36:38 smithi132 ceph-mon[35941]: pgmap v5943: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:36:39.202 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:36:38 smithi119 ceph-mon[31518]: pgmap v5943: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:36:39.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:36:38 smithi165 ceph-mon[36302]: pgmap v5943: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:36:41.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:36:40 smithi132 ceph-mon[35941]: pgmap v5944: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T03:36:41.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:36:40 smithi165 ceph-mon[36302]: pgmap v5944: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T03:36:41.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:36:40 smithi119 ceph-mon[31518]: pgmap v5944: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T03:36:42.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:36:41 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10860.814987 secs 2024-03-28T03:36:42.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:36:41 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10860.814987 secs 2024-03-28T03:36:42.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:36:41 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10860.814987 secs 2024-03-28T03:36:43.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:36:42 smithi132 ceph-mon[35941]: pgmap v5945: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:36:43.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:36:42 smithi165 ceph-mon[36302]: pgmap v5945: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:36:43.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:36:42 smithi119 ceph-mon[31518]: pgmap v5945: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:36:45.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:36:44 smithi132 ceph-mon[35941]: pgmap v5946: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:36:45.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:36:44 smithi165 ceph-mon[36302]: pgmap v5946: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:36:45.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:36:44 smithi119 ceph-mon[31518]: pgmap v5946: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:36:47.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:36:46 smithi165 ceph-mon[36302]: pgmap v5947: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:36:47.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:36:46 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10865.815025 secs 2024-03-28T03:36:47.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:36:46 smithi119 ceph-mon[31518]: pgmap v5947: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:36:47.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:36:46 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10865.815025 secs 2024-03-28T03:36:47.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:36:46 smithi132 ceph-mon[35941]: pgmap v5947: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:36:47.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:36:46 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10865.815025 secs 2024-03-28T03:36:49.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:36:48 smithi165 ceph-mon[36302]: pgmap v5948: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:36:49.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:36:48 smithi119 ceph-mon[31518]: pgmap v5948: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:36:49.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:36:48 smithi132 ceph-mon[35941]: pgmap v5948: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:36:51.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:36:50 smithi165 ceph-mon[36302]: pgmap v5949: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T03:36:51.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:36:50 smithi119 ceph-mon[31518]: pgmap v5949: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T03:36:51.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:36:50 smithi132 ceph-mon[35941]: pgmap v5949: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T03:36:52.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:36:51 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10870.815115 secs 2024-03-28T03:36:52.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:36:51 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10870.815115 secs 2024-03-28T03:36:52.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:36:51 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10870.815115 secs 2024-03-28T03:36:53.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:36:52 smithi165 ceph-mon[36302]: pgmap v5950: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:36:53.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:36:52 smithi119 ceph-mon[31518]: pgmap v5950: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:36:53.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:36:52 smithi132 ceph-mon[35941]: pgmap v5950: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:36:55.251 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:36:54 smithi132 ceph-mon[35941]: pgmap v5951: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:36:55.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:36:54 smithi165 ceph-mon[36302]: pgmap v5951: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:36:55.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:36:54 smithi119 ceph-mon[31518]: pgmap v5951: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:36:56.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:36:55 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T03:36:56.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:36:55 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T03:36:56.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:36:55 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T03:36:57.280 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:36:56 smithi165 ceph-mon[36302]: pgmap v5952: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:36:57.280 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:36:56 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10875.815175 secs 2024-03-28T03:36:57.280 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:36:56 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T03:36:57.280 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:36:56 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T03:36:57.280 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:36:56 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:36:57.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:36:56 smithi119 ceph-mon[31518]: pgmap v5952: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:36:57.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:36:56 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10875.815175 secs 2024-03-28T03:36:57.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:36:56 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T03:36:57.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:36:56 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T03:36:57.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:36:56 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:36:57.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:36:56 smithi132 ceph-mon[35941]: pgmap v5952: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:36:57.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:36:56 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10875.815175 secs 2024-03-28T03:36:57.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:36:56 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T03:36:57.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:36:56 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T03:36:57.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:36:56 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:36:58.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:36:57 smithi165 ceph-mon[36302]: pgmap v5953: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T03:36:58.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:36:57 smithi165 ceph-mon[36302]: pgmap v5954: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:36:58.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:36:57 smithi119 ceph-mon[31518]: pgmap v5953: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T03:36:58.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:36:57 smithi119 ceph-mon[31518]: pgmap v5954: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:36:58.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:36:57 smithi132 ceph-mon[35941]: pgmap v5953: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T03:36:58.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:36:57 smithi132 ceph-mon[35941]: pgmap v5954: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:37:00.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:37:00 smithi165 ceph-mon[36302]: pgmap v5955: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 5 op/s 2024-03-28T03:37:00.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:37:00 smithi119 ceph-mon[31518]: pgmap v5955: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 5 op/s 2024-03-28T03:37:00.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:37:00 smithi132 ceph-mon[35941]: pgmap v5955: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 5 op/s 2024-03-28T03:37:01.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:37:01 smithi165 ceph-mon[36302]: pgmap v5956: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:37:01.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:37:01 smithi119 ceph-mon[31518]: pgmap v5956: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:37:01.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:37:01 smithi132 ceph-mon[35941]: pgmap v5956: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:37:02.303 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:37:02 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10880.815265 secs 2024-03-28T03:37:02.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:37:02 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10880.815265 secs 2024-03-28T03:37:02.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:37:02 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10880.815265 secs 2024-03-28T03:37:03.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:37:03 smithi165 ceph-mon[36302]: pgmap v5957: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:37:03.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:37:03 smithi119 ceph-mon[31518]: pgmap v5957: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:37:03.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:37:03 smithi132 ceph-mon[35941]: pgmap v5957: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:37:05.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:37:05 smithi132 ceph-mon[35941]: pgmap v5958: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:37:06.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:37:05 smithi165 ceph-mon[36302]: pgmap v5958: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:37:06.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:37:05 smithi119 ceph-mon[31518]: pgmap v5958: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:37:06.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:37:06 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10885.815354 secs 2024-03-28T03:37:07.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:37:06 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10885.815354 secs 2024-03-28T03:37:07.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:37:06 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10885.815354 secs 2024-03-28T03:37:07.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:37:07 smithi132 ceph-mon[35941]: pgmap v5959: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T03:37:08.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:37:07 smithi165 ceph-mon[36302]: pgmap v5959: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T03:37:08.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:37:07 smithi119 ceph-mon[31518]: pgmap v5959: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T03:37:09.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:37:09 smithi132 ceph-mon[35941]: pgmap v5960: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:37:09.954 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:37:09 smithi119 ceph-mon[31518]: pgmap v5960: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:37:10.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:37:09 smithi165 ceph-mon[36302]: pgmap v5960: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:37:11.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:37:11 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10890.815439 secs 2024-03-28T03:37:11.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:37:11 smithi132 ceph-mon[35941]: pgmap v5961: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:37:12.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:37:11 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10890.815439 secs 2024-03-28T03:37:12.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:37:11 smithi165 ceph-mon[36302]: pgmap v5961: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:37:12.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:37:11 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10890.815439 secs 2024-03-28T03:37:12.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:37:11 smithi119 ceph-mon[31518]: pgmap v5961: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:37:13.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:37:13 smithi132 ceph-mon[35941]: pgmap v5962: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:37:14.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:37:13 smithi165 ceph-mon[36302]: pgmap v5962: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:37:14.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:37:13 smithi119 ceph-mon[31518]: pgmap v5962: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:37:15.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:37:15 smithi132 ceph-mon[35941]: pgmap v5963: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:37:16.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:37:15 smithi165 ceph-mon[36302]: pgmap v5963: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:37:16.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:37:15 smithi119 ceph-mon[31518]: pgmap v5963: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:37:17.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:37:16 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10895.815481 secs 2024-03-28T03:37:17.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:37:16 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10895.815481 secs 2024-03-28T03:37:17.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:37:16 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10895.815481 secs 2024-03-28T03:37:18.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:37:17 smithi165 ceph-mon[36302]: pgmap v5964: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:37:18.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:37:17 smithi119 ceph-mon[31518]: pgmap v5964: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:37:18.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:37:17 smithi132 ceph-mon[35941]: pgmap v5964: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:37:20.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:37:19 smithi165 ceph-mon[36302]: pgmap v5965: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:37:20.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:37:19 smithi119 ceph-mon[31518]: pgmap v5965: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:37:20.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:37:19 smithi132 ceph-mon[35941]: pgmap v5965: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:37:22.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:37:21 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10900.815573 secs 2024-03-28T03:37:22.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:37:21 smithi165 ceph-mon[36302]: pgmap v5966: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:37:22.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:37:21 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10900.815573 secs 2024-03-28T03:37:22.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:37:21 smithi119 ceph-mon[31518]: pgmap v5966: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:37:22.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:37:21 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10900.815573 secs 2024-03-28T03:37:22.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:37:21 smithi132 ceph-mon[35941]: pgmap v5966: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:37:24.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:37:23 smithi165 ceph-mon[36302]: pgmap v5967: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:37:24.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:37:23 smithi119 ceph-mon[31518]: pgmap v5967: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:37:24.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:37:23 smithi132 ceph-mon[35941]: pgmap v5967: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:37:26.001 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:37:25 smithi132 ceph-mon[35941]: pgmap v5968: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:37:26.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:37:25 smithi165 ceph-mon[36302]: pgmap v5968: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:37:26.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:37:25 smithi119 ceph-mon[31518]: pgmap v5968: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:37:27.029 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:37:26 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10905.815656 secs 2024-03-28T03:37:27.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:37:26 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10905.815656 secs 2024-03-28T03:37:27.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:37:26 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10905.815656 secs 2024-03-28T03:37:28.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:37:27 smithi165 ceph-mon[36302]: pgmap v5969: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:37:28.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:37:27 smithi119 ceph-mon[31518]: pgmap v5969: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:37:28.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:37:27 smithi132 ceph-mon[35941]: pgmap v5969: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:37:30.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:37:29 smithi165 ceph-mon[36302]: pgmap v5970: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:37:30.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:37:29 smithi119 ceph-mon[31518]: pgmap v5970: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:37:30.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:37:29 smithi132 ceph-mon[35941]: pgmap v5970: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:37:32.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:37:31 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10910.815740 secs 2024-03-28T03:37:32.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:37:31 smithi165 ceph-mon[36302]: pgmap v5971: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:37:32.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:37:31 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10910.815740 secs 2024-03-28T03:37:32.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:37:31 smithi119 ceph-mon[31518]: pgmap v5971: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:37:32.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:37:31 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10910.815740 secs 2024-03-28T03:37:32.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:37:31 smithi132 ceph-mon[35941]: pgmap v5971: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:37:34.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:37:33 smithi165 ceph-mon[36302]: pgmap v5972: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:37:34.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:37:33 smithi119 ceph-mon[31518]: pgmap v5972: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:37:34.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:37:33 smithi132 ceph-mon[35941]: pgmap v5972: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:37:36.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:37:35 smithi165 ceph-mon[36302]: pgmap v5973: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:37:36.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:37:35 smithi119 ceph-mon[31518]: pgmap v5973: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:37:36.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:37:35 smithi132 ceph-mon[35941]: pgmap v5973: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:37:37.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:37:36 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10915.815837 secs 2024-03-28T03:37:37.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:37:36 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10915.815837 secs 2024-03-28T03:37:37.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:37:36 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10915.815837 secs 2024-03-28T03:37:38.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:37:37 smithi165 ceph-mon[36302]: pgmap v5974: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:37:38.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:37:37 smithi119 ceph-mon[31518]: pgmap v5974: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:37:38.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:37:37 smithi132 ceph-mon[35941]: pgmap v5974: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:37:40.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:37:39 smithi165 ceph-mon[36302]: pgmap v5975: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:37:40.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:37:39 smithi119 ceph-mon[31518]: pgmap v5975: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:37:40.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:37:39 smithi132 ceph-mon[35941]: pgmap v5975: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:37:42.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:37:41 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10920.815926 secs 2024-03-28T03:37:42.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:37:41 smithi165 ceph-mon[36302]: pgmap v5976: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:37:42.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:37:41 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10920.815926 secs 2024-03-28T03:37:42.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:37:41 smithi119 ceph-mon[31518]: pgmap v5976: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:37:42.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:37:41 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10920.815926 secs 2024-03-28T03:37:42.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:37:41 smithi132 ceph-mon[35941]: pgmap v5976: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:37:44.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:37:43 smithi165 ceph-mon[36302]: pgmap v5977: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:37:44.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:37:43 smithi119 ceph-mon[31518]: pgmap v5977: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:37:44.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:37:43 smithi132 ceph-mon[35941]: pgmap v5977: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:37:46.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:37:45 smithi165 ceph-mon[36302]: pgmap v5978: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:37:46.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:37:45 smithi119 ceph-mon[31518]: pgmap v5978: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:37:46.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:37:45 smithi132 ceph-mon[35941]: pgmap v5978: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:37:47.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:37:46 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10925.815988 secs 2024-03-28T03:37:47.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:37:46 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10925.815988 secs 2024-03-28T03:37:47.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:37:46 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10925.815988 secs 2024-03-28T03:37:48.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:37:47 smithi165 ceph-mon[36302]: pgmap v5979: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:37:48.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:37:47 smithi119 ceph-mon[31518]: pgmap v5979: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:37:48.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:37:47 smithi132 ceph-mon[35941]: pgmap v5979: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:37:50.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:37:49 smithi165 ceph-mon[36302]: pgmap v5980: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:37:50.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:37:49 smithi119 ceph-mon[31518]: pgmap v5980: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:37:50.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:37:49 smithi132 ceph-mon[35941]: pgmap v5980: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:37:52.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:37:51 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10930.816049 secs 2024-03-28T03:37:52.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:37:51 smithi165 ceph-mon[36302]: pgmap v5981: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:37:52.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:37:51 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10930.816049 secs 2024-03-28T03:37:52.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:37:51 smithi119 ceph-mon[31518]: pgmap v5981: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:37:52.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:37:51 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10930.816049 secs 2024-03-28T03:37:52.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:37:51 smithi132 ceph-mon[35941]: pgmap v5981: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:37:52.484 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:37:52.482+0000 7fc46aa00640 1 -- 172.21.15.119:0/210561562 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- auth(proto 2 132 bytes epoch 0) v1 -- 0x7fc44c005b70 con 0x7fc474102010 2024-03-28T03:37:52.484 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:37:52.482+0000 7fc46be00640 1 -- 172.21.15.119:0/210561562 <== mon.1 v2:172.21.15.165:3300/0 9 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 233+0+0 (secure 0 0 0) 0x7fc44c005b70 con 0x7fc474102010 2024-03-28T03:37:54.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:37:53 smithi165 ceph-mon[36302]: pgmap v5982: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:37:54.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:37:53 smithi119 ceph-mon[31518]: pgmap v5982: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:37:54.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:37:53 smithi132 ceph-mon[35941]: pgmap v5982: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:37:56.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:37:55 smithi165 ceph-mon[36302]: pgmap v5983: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:37:56.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:37:55 smithi119 ceph-mon[31518]: pgmap v5983: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:37:56.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:37:55 smithi132 ceph-mon[35941]: pgmap v5983: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:37:57.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:37:56 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10935.816109 secs 2024-03-28T03:37:57.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:37:56 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10935.816109 secs 2024-03-28T03:37:57.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:37:56 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10935.816109 secs 2024-03-28T03:37:58.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:37:57 smithi165 ceph-mon[36302]: pgmap v5984: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T03:37:58.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:37:57 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T03:37:58.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:37:57 smithi119 ceph-mon[31518]: pgmap v5984: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T03:37:58.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:37:57 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T03:37:58.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:37:57 smithi132 ceph-mon[35941]: pgmap v5984: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T03:37:58.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:37:57 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T03:37:59.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:37:58 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T03:37:59.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:37:58 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T03:37:59.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:37:58 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:37:59.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:37:58 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T03:37:59.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:37:58 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T03:37:59.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:37:58 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:37:59.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:37:58 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T03:37:59.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:37:58 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T03:37:59.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:37:58 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:38:00.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:37:59 smithi165 ceph-mon[36302]: pgmap v5985: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T03:38:00.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:37:59 smithi119 ceph-mon[31518]: pgmap v5985: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T03:38:00.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:37:59 smithi132 ceph-mon[35941]: pgmap v5985: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T03:38:02.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:38:01 smithi165 ceph-mon[36302]: pgmap v5986: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 7 op/s 2024-03-28T03:38:02.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:38:01 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10940.816170 secs 2024-03-28T03:38:02.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:38:01 smithi119 ceph-mon[31518]: pgmap v5986: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 7 op/s 2024-03-28T03:38:02.174 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:38:01 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10940.816170 secs 2024-03-28T03:38:02.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:38:01 smithi132 ceph-mon[35941]: pgmap v5986: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 7 op/s 2024-03-28T03:38:02.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:38:01 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10940.816170 secs 2024-03-28T03:38:04.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:38:03 smithi165 ceph-mon[36302]: pgmap v5987: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T03:38:04.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:38:03 smithi119 ceph-mon[31518]: pgmap v5987: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T03:38:04.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:38:03 smithi132 ceph-mon[35941]: pgmap v5987: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T03:38:06.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:38:05 smithi165 ceph-mon[36302]: pgmap v5988: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T03:38:06.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:38:05 smithi119 ceph-mon[31518]: pgmap v5988: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T03:38:06.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:38:05 smithi132 ceph-mon[35941]: pgmap v5988: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T03:38:07.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:38:06 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10945.816231 secs 2024-03-28T03:38:07.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:38:06 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10945.816231 secs 2024-03-28T03:38:07.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:38:06 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10945.816231 secs 2024-03-28T03:38:08.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:38:07 smithi165 ceph-mon[36302]: pgmap v5989: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T03:38:08.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:38:07 smithi119 ceph-mon[31518]: pgmap v5989: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T03:38:08.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:38:07 smithi132 ceph-mon[35941]: pgmap v5989: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T03:38:10.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:38:09 smithi165 ceph-mon[36302]: pgmap v5990: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T03:38:10.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:38:09 smithi119 ceph-mon[31518]: pgmap v5990: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T03:38:10.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:38:09 smithi132 ceph-mon[35941]: pgmap v5990: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T03:38:12.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:38:11 smithi165 ceph-mon[36302]: pgmap v5991: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:38:12.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:38:11 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10950.816293 secs 2024-03-28T03:38:12.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:38:11 smithi119 ceph-mon[31518]: pgmap v5991: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:38:12.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:38:11 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10950.816293 secs 2024-03-28T03:38:12.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:38:11 smithi132 ceph-mon[35941]: pgmap v5991: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:38:12.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:38:11 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10950.816293 secs 2024-03-28T03:38:14.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:38:13 smithi119 ceph-mon[31518]: pgmap v5992: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T03:38:14.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:38:13 smithi132 ceph-mon[35941]: pgmap v5992: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T03:38:14.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:38:13 smithi165 ceph-mon[36302]: pgmap v5992: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T03:38:16.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:38:15 smithi119 ceph-mon[31518]: pgmap v5993: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-03-28T03:38:16.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:38:15 smithi132 ceph-mon[35941]: pgmap v5993: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-03-28T03:38:16.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:38:15 smithi165 ceph-mon[36302]: pgmap v5993: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-03-28T03:38:17.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:38:16 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10955.816382 secs 2024-03-28T03:38:17.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:38:16 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10955.816382 secs 2024-03-28T03:38:17.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:38:16 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10955.816382 secs 2024-03-28T03:38:18.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:38:17 smithi119 ceph-mon[31518]: pgmap v5994: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-28T03:38:18.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:38:17 smithi132 ceph-mon[35941]: pgmap v5994: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-28T03:38:18.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:38:17 smithi165 ceph-mon[36302]: pgmap v5994: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-28T03:38:20.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:38:19 smithi119 ceph-mon[31518]: pgmap v5995: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:38:20.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:38:19 smithi132 ceph-mon[35941]: pgmap v5995: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:38:20.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:38:19 smithi165 ceph-mon[36302]: pgmap v5995: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:38:22.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:38:21 smithi132 ceph-mon[35941]: pgmap v5996: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:38:22.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:38:21 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10960.816472 secs 2024-03-28T03:38:22.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:38:21 smithi165 ceph-mon[36302]: pgmap v5996: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:38:22.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:38:21 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10960.816472 secs 2024-03-28T03:38:22.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:38:21 smithi119 ceph-mon[31518]: pgmap v5996: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:38:22.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:38:21 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10960.816472 secs 2024-03-28T03:38:24.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:38:23 smithi132 ceph-mon[35941]: pgmap v5997: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:38:24.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:38:23 smithi165 ceph-mon[36302]: pgmap v5997: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:38:24.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:38:23 smithi119 ceph-mon[31518]: pgmap v5997: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:38:26.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:38:25 smithi132 ceph-mon[35941]: pgmap v5998: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T03:38:26.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:38:25 smithi165 ceph-mon[36302]: pgmap v5998: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T03:38:26.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:38:25 smithi119 ceph-mon[31518]: pgmap v5998: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T03:38:27.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:38:26 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10965.816560 secs 2024-03-28T03:38:27.234 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:38:26 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10965.816560 secs 2024-03-28T03:38:27.279 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:38:26 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10965.816560 secs 2024-03-28T03:38:28.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:38:27 smithi132 ceph-mon[35941]: pgmap v5999: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:38:28.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:38:27 smithi165 ceph-mon[36302]: pgmap v5999: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:38:28.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:38:27 smithi119 ceph-mon[31518]: pgmap v5999: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:38:30.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:38:29 smithi132 ceph-mon[35941]: pgmap v6000: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:38:30.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:38:29 smithi165 ceph-mon[36302]: pgmap v6000: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:38:30.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:38:29 smithi119 ceph-mon[31518]: pgmap v6000: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:38:32.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:38:31 smithi132 ceph-mon[35941]: pgmap v6001: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:38:32.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:38:31 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10970.816620 secs 2024-03-28T03:38:32.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:38:31 smithi165 ceph-mon[36302]: pgmap v6001: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:38:32.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:38:31 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10970.816620 secs 2024-03-28T03:38:32.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:38:31 smithi119 ceph-mon[31518]: pgmap v6001: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:38:32.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:38:31 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10970.816620 secs 2024-03-28T03:38:34.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:38:33 smithi132 ceph-mon[35941]: pgmap v6002: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:38:34.204 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:38:33 smithi119 ceph-mon[31518]: pgmap v6002: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:38:34.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:38:33 smithi165 ceph-mon[36302]: pgmap v6002: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:38:36.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:38:35 smithi132 ceph-mon[35941]: pgmap v6003: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T03:38:36.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:38:35 smithi165 ceph-mon[36302]: pgmap v6003: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T03:38:36.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:38:35 smithi119 ceph-mon[31518]: pgmap v6003: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T03:38:37.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:38:36 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10975.816681 secs 2024-03-28T03:38:37.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:38:36 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10975.816681 secs 2024-03-28T03:38:37.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:38:36 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10975.816681 secs 2024-03-28T03:38:37.491 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:38:37.490+0000 7fc46aa00640 1 -- 172.21.15.119:0/210561562 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc44c005ed0 con 0x7fc474102010 2024-03-28T03:38:37.492 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:38:37.490+0000 7fc46be00640 1 -- 172.21.15.119:0/210561562 <== mon.1 v2:172.21.15.165:3300/0 10 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (secure 0 0 0) 0x7fc44c005ed0 con 0x7fc474102010 2024-03-28T03:38:38.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:38:37 smithi165 ceph-mon[36302]: pgmap v6004: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:38:38.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:38:37 smithi119 ceph-mon[31518]: pgmap v6004: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:38:38.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:38:37 smithi132 ceph-mon[35941]: pgmap v6004: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:38:40.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:38:39 smithi165 ceph-mon[36302]: pgmap v6005: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:38:40.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:38:39 smithi119 ceph-mon[31518]: pgmap v6005: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:38:40.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:38:39 smithi132 ceph-mon[35941]: pgmap v6005: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:38:42.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:38:41 smithi165 ceph-mon[36302]: pgmap v6006: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:38:42.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:38:41 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10980.816740 secs 2024-03-28T03:38:42.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:38:41 smithi119 ceph-mon[31518]: pgmap v6006: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:38:42.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:38:41 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10980.816740 secs 2024-03-28T03:38:42.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:38:41 smithi132 ceph-mon[35941]: pgmap v6006: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:38:42.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:38:41 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10980.816740 secs 2024-03-28T03:38:44.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:38:43 smithi165 ceph-mon[36302]: pgmap v6007: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:38:44.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:38:43 smithi119 ceph-mon[31518]: pgmap v6007: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:38:44.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:38:43 smithi132 ceph-mon[35941]: pgmap v6007: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:38:46.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:38:45 smithi165 ceph-mon[36302]: pgmap v6008: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T03:38:46.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:38:45 smithi119 ceph-mon[31518]: pgmap v6008: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T03:38:46.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:38:45 smithi132 ceph-mon[35941]: pgmap v6008: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T03:38:47.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:38:46 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10985.816802 secs 2024-03-28T03:38:47.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:38:46 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10985.816802 secs 2024-03-28T03:38:47.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:38:46 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10985.816802 secs 2024-03-28T03:38:48.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:38:47 smithi165 ceph-mon[36302]: pgmap v6009: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:38:48.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:38:47 smithi119 ceph-mon[31518]: pgmap v6009: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:38:48.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:38:47 smithi132 ceph-mon[35941]: pgmap v6009: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:38:50.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:38:49 smithi165 ceph-mon[36302]: pgmap v6010: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:38:50.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:38:49 smithi119 ceph-mon[31518]: pgmap v6010: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:38:50.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:38:49 smithi132 ceph-mon[35941]: pgmap v6010: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:38:52.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:38:51 smithi165 ceph-mon[36302]: pgmap v6011: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:38:52.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:38:51 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10990.816890 secs 2024-03-28T03:38:52.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:38:51 smithi119 ceph-mon[31518]: pgmap v6011: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:38:52.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:38:51 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10990.816890 secs 2024-03-28T03:38:52.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:38:51 smithi132 ceph-mon[35941]: pgmap v6011: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:38:52.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:38:51 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10990.816890 secs 2024-03-28T03:38:54.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:38:53 smithi165 ceph-mon[36302]: pgmap v6012: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:38:54.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:38:53 smithi119 ceph-mon[31518]: pgmap v6012: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:38:54.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:38:53 smithi132 ceph-mon[35941]: pgmap v6012: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:38:56.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:38:56 smithi165 ceph-mon[36302]: pgmap v6013: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T03:38:56.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:38:56 smithi119 ceph-mon[31518]: pgmap v6013: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T03:38:56.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:38:56 smithi132 ceph-mon[35941]: pgmap v6013: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T03:38:57.279 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:38:57 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 10995.816951 secs 2024-03-28T03:38:57.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:38:57 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 10995.816951 secs 2024-03-28T03:38:57.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:38:57 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 10995.816951 secs 2024-03-28T03:38:58.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:38:58 smithi165 ceph-mon[36302]: pgmap v6014: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:38:58.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:38:58 smithi119 ceph-mon[31518]: pgmap v6014: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:38:58.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:38:58 smithi132 ceph-mon[35941]: pgmap v6014: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:38:59.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:38:59 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T03:38:59.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:38:59 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T03:38:59.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:38:59 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T03:39:00.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:39:00 smithi165 ceph-mon[36302]: pgmap v6015: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:39:00.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:39:00 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T03:39:00.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:39:00 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T03:39:00.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:39:00 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:39:00.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:39:00 smithi119 ceph-mon[31518]: pgmap v6015: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:39:00.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:39:00 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T03:39:00.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:39:00 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T03:39:00.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:39:00 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:39:00.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:39:00 smithi132 ceph-mon[35941]: pgmap v6015: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:39:00.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:39:00 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T03:39:00.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:39:00 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T03:39:00.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:39:00 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:39:01.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:39:01 smithi165 ceph-mon[36302]: pgmap v6016: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T03:39:01.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:39:01 smithi165 ceph-mon[36302]: pgmap v6017: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:39:01.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:39:01 smithi119 ceph-mon[31518]: pgmap v6016: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T03:39:01.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:39:01 smithi119 ceph-mon[31518]: pgmap v6017: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:39:01.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:39:01 smithi132 ceph-mon[35941]: pgmap v6016: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T03:39:01.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:39:01 smithi132 ceph-mon[35941]: pgmap v6017: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:39:02.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:39:02 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 11000.817035 secs 2024-03-28T03:39:02.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:39:02 smithi165 ceph-mon[36302]: pgmap v6018: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:39:02.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:39:02 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 11000.817035 secs 2024-03-28T03:39:02.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:39:02 smithi119 ceph-mon[31518]: pgmap v6018: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:39:02.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:39:02 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 11000.817035 secs 2024-03-28T03:39:02.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:39:02 smithi132 ceph-mon[35941]: pgmap v6018: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:39:05.105 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:39:04 smithi119 ceph-mon[31518]: pgmap v6019: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-03-28T03:39:05.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:39:04 smithi165 ceph-mon[36302]: pgmap v6019: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-03-28T03:39:05.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:39:04 smithi132 ceph-mon[35941]: pgmap v6019: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-03-28T03:39:07.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:39:06 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 11005.817096 secs 2024-03-28T03:39:07.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:39:06 smithi165 ceph-mon[36302]: pgmap v6020: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:39:07.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:39:06 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 11005.817096 secs 2024-03-28T03:39:07.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:39:06 smithi119 ceph-mon[31518]: pgmap v6020: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:39:07.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:39:06 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 11005.817096 secs 2024-03-28T03:39:07.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:39:06 smithi132 ceph-mon[35941]: pgmap v6020: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:39:09.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:39:08 smithi165 ceph-mon[36302]: pgmap v6021: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:39:09.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:39:08 smithi119 ceph-mon[31518]: pgmap v6021: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:39:09.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:39:08 smithi132 ceph-mon[35941]: pgmap v6021: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:39:11.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:39:10 smithi165 ceph-mon[36302]: pgmap v6022: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T03:39:11.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:39:10 smithi119 ceph-mon[31518]: pgmap v6022: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T03:39:11.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:39:10 smithi132 ceph-mon[35941]: pgmap v6022: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T03:39:12.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:39:11 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 11010.817166 secs 2024-03-28T03:39:12.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:39:11 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 11010.817166 secs 2024-03-28T03:39:12.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:39:11 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 11010.817166 secs 2024-03-28T03:39:13.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:39:12 smithi165 ceph-mon[36302]: pgmap v6023: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:39:13.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:39:12 smithi119 ceph-mon[31518]: pgmap v6023: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:39:13.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:39:12 smithi132 ceph-mon[35941]: pgmap v6023: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:39:15.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:39:14 smithi165 ceph-mon[36302]: pgmap v6024: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:39:15.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:39:14 smithi119 ceph-mon[31518]: pgmap v6024: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:39:15.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:39:14 smithi132 ceph-mon[35941]: pgmap v6024: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:39:17.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:39:16 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 11015.817255 secs 2024-03-28T03:39:17.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:39:16 smithi165 ceph-mon[36302]: pgmap v6025: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:39:17.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:39:16 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 11015.817255 secs 2024-03-28T03:39:17.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:39:16 smithi119 ceph-mon[31518]: pgmap v6025: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:39:17.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:39:16 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 11015.817255 secs 2024-03-28T03:39:17.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:39:16 smithi132 ceph-mon[35941]: pgmap v6025: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:39:19.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:39:18 smithi165 ceph-mon[36302]: pgmap v6026: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:39:19.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:39:18 smithi119 ceph-mon[31518]: pgmap v6026: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:39:19.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:39:18 smithi132 ceph-mon[35941]: pgmap v6026: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:39:21.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:39:20 smithi165 ceph-mon[36302]: pgmap v6027: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:39:21.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:39:20 smithi119 ceph-mon[31518]: pgmap v6027: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:39:21.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:39:20 smithi132 ceph-mon[35941]: pgmap v6027: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:39:22.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:39:21 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 11020.817293 secs 2024-03-28T03:39:22.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:39:21 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 11020.817293 secs 2024-03-28T03:39:22.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:39:21 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 11020.817293 secs 2024-03-28T03:39:23.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:39:22 smithi165 ceph-mon[36302]: pgmap v6028: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:39:23.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:39:22 smithi119 ceph-mon[31518]: pgmap v6028: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:39:23.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:39:22 smithi132 ceph-mon[35941]: pgmap v6028: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:39:25.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:39:24 smithi165 ceph-mon[36302]: pgmap v6029: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:39:25.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:39:24 smithi119 ceph-mon[31518]: pgmap v6029: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:39:25.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:39:24 smithi132 ceph-mon[35941]: pgmap v6029: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:39:27.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:39:26 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 11025.817385 secs 2024-03-28T03:39:27.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:39:26 smithi165 ceph-mon[36302]: pgmap v6030: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:39:27.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:39:26 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 11025.817385 secs 2024-03-28T03:39:27.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:39:26 smithi119 ceph-mon[31518]: pgmap v6030: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:39:27.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:39:26 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 11025.817385 secs 2024-03-28T03:39:27.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:39:26 smithi132 ceph-mon[35941]: pgmap v6030: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:39:29.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:39:28 smithi165 ceph-mon[36302]: pgmap v6031: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:39:29.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:39:28 smithi119 ceph-mon[31518]: pgmap v6031: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:39:29.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:39:28 smithi132 ceph-mon[35941]: pgmap v6031: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:39:31.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:39:30 smithi165 ceph-mon[36302]: pgmap v6032: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:39:31.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:39:30 smithi119 ceph-mon[31518]: pgmap v6032: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:39:31.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:39:30 smithi132 ceph-mon[35941]: pgmap v6032: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:39:32.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:39:31 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 11030.817424 secs 2024-03-28T03:39:32.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:39:31 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 11030.817424 secs 2024-03-28T03:39:32.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:39:31 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 11030.817424 secs 2024-03-28T03:39:33.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:39:32 smithi165 ceph-mon[36302]: pgmap v6033: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:39:33.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:39:32 smithi119 ceph-mon[31518]: pgmap v6033: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:39:33.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:39:32 smithi132 ceph-mon[35941]: pgmap v6033: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:39:35.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:39:34 smithi165 ceph-mon[36302]: pgmap v6034: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:39:35.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:39:34 smithi119 ceph-mon[31518]: pgmap v6034: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:39:35.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:39:34 smithi132 ceph-mon[35941]: pgmap v6034: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:39:37.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:39:36 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 11035.817511 secs 2024-03-28T03:39:37.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:39:36 smithi165 ceph-mon[36302]: pgmap v6035: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:39:37.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:39:36 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 11035.817511 secs 2024-03-28T03:39:37.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:39:36 smithi119 ceph-mon[31518]: pgmap v6035: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:39:37.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:39:36 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 11035.817511 secs 2024-03-28T03:39:37.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:39:36 smithi132 ceph-mon[35941]: pgmap v6035: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:39:39.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:39:38 smithi165 ceph-mon[36302]: pgmap v6036: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:39:39.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:39:38 smithi119 ceph-mon[31518]: pgmap v6036: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:39:39.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:39:38 smithi132 ceph-mon[35941]: pgmap v6036: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:39:41.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:39:40 smithi165 ceph-mon[36302]: pgmap v6037: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:39:41.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:39:40 smithi119 ceph-mon[31518]: pgmap v6037: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:39:41.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:39:40 smithi132 ceph-mon[35941]: pgmap v6037: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:39:42.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:39:41 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 11040.817599 secs 2024-03-28T03:39:42.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:39:41 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 11040.817599 secs 2024-03-28T03:39:42.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:39:41 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 11040.817599 secs 2024-03-28T03:39:43.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:39:42 smithi165 ceph-mon[36302]: pgmap v6038: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:39:43.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:39:42 smithi119 ceph-mon[31518]: pgmap v6038: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:39:43.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:39:42 smithi132 ceph-mon[35941]: pgmap v6038: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:39:45.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:39:44 smithi165 ceph-mon[36302]: pgmap v6039: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:39:45.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:39:44 smithi119 ceph-mon[31518]: pgmap v6039: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:39:45.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:39:44 smithi132 ceph-mon[35941]: pgmap v6039: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:39:47.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:39:46 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 11045.817688 secs 2024-03-28T03:39:47.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:39:46 smithi165 ceph-mon[36302]: pgmap v6040: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:39:47.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:39:46 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 11045.817688 secs 2024-03-28T03:39:47.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:39:46 smithi119 ceph-mon[31518]: pgmap v6040: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:39:47.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:39:46 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 11045.817688 secs 2024-03-28T03:39:47.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:39:46 smithi132 ceph-mon[35941]: pgmap v6040: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:39:49.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:39:48 smithi165 ceph-mon[36302]: pgmap v6041: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:39:49.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:39:48 smithi119 ceph-mon[31518]: pgmap v6041: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:39:49.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:39:48 smithi132 ceph-mon[35941]: pgmap v6041: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:39:51.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:39:50 smithi165 ceph-mon[36302]: pgmap v6042: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:39:51.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:39:50 smithi119 ceph-mon[31518]: pgmap v6042: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:39:51.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:39:50 smithi132 ceph-mon[35941]: pgmap v6042: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:39:52.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:39:51 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 11050.817773 secs 2024-03-28T03:39:52.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:39:51 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 11050.817773 secs 2024-03-28T03:39:52.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:39:51 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 11050.817773 secs 2024-03-28T03:39:53.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:39:52 smithi165 ceph-mon[36302]: pgmap v6043: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:39:53.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:39:52 smithi119 ceph-mon[31518]: pgmap v6043: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:39:53.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:39:52 smithi132 ceph-mon[35941]: pgmap v6043: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:39:55.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:39:54 smithi165 ceph-mon[36302]: pgmap v6044: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:39:55.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:39:54 smithi119 ceph-mon[31518]: pgmap v6044: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:39:55.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:39:54 smithi132 ceph-mon[35941]: pgmap v6044: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:39:57.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:39:56 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 11055.817869 secs 2024-03-28T03:39:57.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:39:56 smithi165 ceph-mon[36302]: pgmap v6045: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:39:57.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:39:56 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 11055.817869 secs 2024-03-28T03:39:57.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:39:56 smithi119 ceph-mon[31518]: pgmap v6045: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:39:57.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:39:56 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 11055.817869 secs 2024-03-28T03:39:57.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:39:56 smithi132 ceph-mon[35941]: pgmap v6045: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:39:59.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:39:58 smithi165 ceph-mon[36302]: pgmap v6046: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:39:59.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:39:58 smithi119 ceph-mon[31518]: pgmap v6046: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:39:59.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:39:58 smithi132 ceph-mon[35941]: pgmap v6046: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:40:01.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:40:00 smithi165 ceph-mon[36302]: pgmap v6047: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:40:01.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:40:00 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T03:40:01.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:40:00 smithi165 ceph-mon[36302]: overall HEALTH_WARN 5 failed cephadm daemon(s); 1 filesystem is degraded; 1 MDSs report slow requests 2024-03-28T03:40:01.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:40:00 smithi119 ceph-mon[31518]: pgmap v6047: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:40:01.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:40:00 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T03:40:01.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:40:00 smithi119 ceph-mon[31518]: overall HEALTH_WARN 5 failed cephadm daemon(s); 1 filesystem is degraded; 1 MDSs report slow requests 2024-03-28T03:40:01.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:40:00 smithi132 ceph-mon[35941]: pgmap v6047: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:40:01.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:40:00 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T03:40:01.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:40:00 smithi132 ceph-mon[35941]: overall HEALTH_WARN 5 failed cephadm daemon(s); 1 filesystem is degraded; 1 MDSs report slow requests 2024-03-28T03:40:02.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:40:01 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 11060.817960 secs 2024-03-28T03:40:02.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:40:01 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T03:40:02.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:40:01 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T03:40:02.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:40:01 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:40:02.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:40:01 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 11060.817960 secs 2024-03-28T03:40:02.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:40:01 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T03:40:02.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:40:01 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T03:40:02.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:40:01 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:40:02.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:40:01 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 11060.817960 secs 2024-03-28T03:40:02.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:40:01 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T03:40:02.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:40:01 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T03:40:02.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:40:01 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:40:03.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:40:02 smithi119 ceph-mon[31518]: pgmap v6048: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T03:40:03.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:40:02 smithi119 ceph-mon[31518]: pgmap v6049: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:40:03.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:40:02 smithi132 ceph-mon[35941]: pgmap v6048: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T03:40:03.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:40:02 smithi132 ceph-mon[35941]: pgmap v6049: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:40:03.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:40:02 smithi165 ceph-mon[36302]: pgmap v6048: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T03:40:03.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:40:02 smithi165 ceph-mon[36302]: pgmap v6049: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:40:05.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:40:04 smithi119 ceph-mon[31518]: pgmap v6050: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:40:05.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:40:04 smithi132 ceph-mon[35941]: pgmap v6050: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:40:05.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:40:04 smithi165 ceph-mon[36302]: pgmap v6050: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:40:07.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:40:06 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 11065.818044 secs 2024-03-28T03:40:07.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:40:06 smithi132 ceph-mon[35941]: pgmap v6051: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:40:07.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:40:06 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 11065.818044 secs 2024-03-28T03:40:07.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:40:06 smithi165 ceph-mon[36302]: pgmap v6051: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:40:07.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:40:06 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 11065.818044 secs 2024-03-28T03:40:07.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:40:06 smithi119 ceph-mon[31518]: pgmap v6051: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:40:09.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:40:08 smithi132 ceph-mon[35941]: pgmap v6052: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:40:09.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:40:08 smithi165 ceph-mon[36302]: pgmap v6052: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:40:09.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:40:08 smithi119 ceph-mon[31518]: pgmap v6052: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:40:11.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:40:10 smithi132 ceph-mon[35941]: pgmap v6053: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T03:40:11.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:40:10 smithi165 ceph-mon[36302]: pgmap v6053: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T03:40:11.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:40:10 smithi119 ceph-mon[31518]: pgmap v6053: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T03:40:12.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:40:11 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 11070.818129 secs 2024-03-28T03:40:12.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:40:11 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 11070.818129 secs 2024-03-28T03:40:12.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:40:11 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 11070.818129 secs 2024-03-28T03:40:13.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:40:12 smithi132 ceph-mon[35941]: pgmap v6054: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T03:40:13.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:40:12 smithi165 ceph-mon[36302]: pgmap v6054: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T03:40:13.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:40:12 smithi119 ceph-mon[31518]: pgmap v6054: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T03:40:15.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:40:14 smithi132 ceph-mon[35941]: pgmap v6055: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:40:15.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:40:14 smithi165 ceph-mon[36302]: pgmap v6055: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:40:15.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:40:14 smithi119 ceph-mon[31518]: pgmap v6055: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:40:17.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:40:16 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 11075.818217 secs 2024-03-28T03:40:17.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:40:16 smithi132 ceph-mon[35941]: pgmap v6056: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:40:17.314 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:40:16 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 11075.818217 secs 2024-03-28T03:40:17.314 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:40:16 smithi119 ceph-mon[31518]: pgmap v6056: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:40:17.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:40:16 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 11075.818217 secs 2024-03-28T03:40:17.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:40:16 smithi165 ceph-mon[36302]: pgmap v6056: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:40:19.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:40:18 smithi132 ceph-mon[35941]: pgmap v6057: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:40:19.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:40:18 smithi165 ceph-mon[36302]: pgmap v6057: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:40:19.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:40:18 smithi119 ceph-mon[31518]: pgmap v6057: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:40:21.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:40:20 smithi132 ceph-mon[35941]: pgmap v6058: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-28T03:40:21.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:40:20 smithi165 ceph-mon[36302]: pgmap v6058: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-28T03:40:21.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:40:20 smithi119 ceph-mon[31518]: pgmap v6058: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-28T03:40:22.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:40:21 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 11080.818279 secs 2024-03-28T03:40:22.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:40:21 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 11080.818279 secs 2024-03-28T03:40:22.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:40:21 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 11080.818279 secs 2024-03-28T03:40:23.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:40:22 smithi165 ceph-mon[36302]: pgmap v6059: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-28T03:40:23.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:40:22 smithi119 ceph-mon[31518]: pgmap v6059: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-28T03:40:23.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:40:22 smithi132 ceph-mon[35941]: pgmap v6059: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-28T03:40:25.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:40:24 smithi165 ceph-mon[36302]: pgmap v6060: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:40:25.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:40:24 smithi119 ceph-mon[31518]: pgmap v6060: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:40:25.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:40:24 smithi132 ceph-mon[35941]: pgmap v6060: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:40:27.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:40:26 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 11085.818363 secs 2024-03-28T03:40:27.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:40:26 smithi165 ceph-mon[36302]: pgmap v6061: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:40:27.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:40:26 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 11085.818363 secs 2024-03-28T03:40:27.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:40:26 smithi119 ceph-mon[31518]: pgmap v6061: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:40:27.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:40:26 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 11085.818363 secs 2024-03-28T03:40:27.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:40:26 smithi132 ceph-mon[35941]: pgmap v6061: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:40:29.250 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:40:28 smithi132 ceph-mon[35941]: pgmap v6062: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:40:29.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:40:28 smithi165 ceph-mon[36302]: pgmap v6062: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:40:29.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:40:28 smithi119 ceph-mon[31518]: pgmap v6062: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:40:31.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:40:30 smithi165 ceph-mon[36302]: pgmap v6063: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:40:31.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:40:30 smithi119 ceph-mon[31518]: pgmap v6063: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:40:31.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:40:30 smithi132 ceph-mon[35941]: pgmap v6063: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:40:32.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:40:31 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 11090.818449 secs 2024-03-28T03:40:32.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:40:31 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 11090.818449 secs 2024-03-28T03:40:32.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:40:31 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 11090.818449 secs 2024-03-28T03:40:33.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:40:32 smithi165 ceph-mon[36302]: pgmap v6064: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-28T03:40:33.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:40:32 smithi119 ceph-mon[31518]: pgmap v6064: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-28T03:40:33.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:40:32 smithi132 ceph-mon[35941]: pgmap v6064: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-28T03:40:35.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:40:34 smithi165 ceph-mon[36302]: pgmap v6065: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:40:35.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:40:34 smithi119 ceph-mon[31518]: pgmap v6065: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:40:35.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:40:34 smithi132 ceph-mon[35941]: pgmap v6065: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:40:37.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:40:36 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 11095.818508 secs 2024-03-28T03:40:37.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:40:36 smithi165 ceph-mon[36302]: pgmap v6066: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:40:37.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:40:36 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 11095.818508 secs 2024-03-28T03:40:37.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:40:36 smithi119 ceph-mon[31518]: pgmap v6066: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:40:37.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:40:36 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 11095.818508 secs 2024-03-28T03:40:37.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:40:36 smithi132 ceph-mon[35941]: pgmap v6066: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:40:39.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:40:38 smithi165 ceph-mon[36302]: pgmap v6067: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:40:39.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:40:38 smithi119 ceph-mon[31518]: pgmap v6067: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:40:39.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:40:38 smithi132 ceph-mon[35941]: pgmap v6067: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:40:41.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:40:41 smithi165 ceph-mon[36302]: pgmap v6068: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:40:41.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:40:41 smithi119 ceph-mon[31518]: pgmap v6068: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:40:41.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:40:41 smithi132 ceph-mon[35941]: pgmap v6068: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:40:42.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:40:42 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 11100.818547 secs 2024-03-28T03:40:42.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:40:42 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 11100.818547 secs 2024-03-28T03:40:42.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:40:42 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 11100.818547 secs 2024-03-28T03:40:43.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:40:43 smithi165 ceph-mon[36302]: pgmap v6069: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:40:43.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:40:43 smithi119 ceph-mon[31518]: pgmap v6069: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:40:43.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:40:43 smithi132 ceph-mon[35941]: pgmap v6069: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:40:45.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:40:45 smithi165 ceph-mon[36302]: pgmap v6070: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:40:45.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:40:45 smithi119 ceph-mon[31518]: pgmap v6070: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:40:45.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:40:45 smithi132 ceph-mon[35941]: pgmap v6070: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:40:47.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:40:47 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 11105.818609 secs 2024-03-28T03:40:47.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:40:47 smithi165 ceph-mon[36302]: pgmap v6071: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:40:47.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:40:47 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 11105.818609 secs 2024-03-28T03:40:47.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:40:47 smithi119 ceph-mon[31518]: pgmap v6071: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:40:47.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:40:47 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 11105.818609 secs 2024-03-28T03:40:47.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:40:47 smithi132 ceph-mon[35941]: pgmap v6071: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:40:49.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:40:49 smithi165 ceph-mon[36302]: pgmap v6072: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:40:49.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:40:49 smithi119 ceph-mon[31518]: pgmap v6072: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:40:49.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:40:49 smithi132 ceph-mon[35941]: pgmap v6072: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:40:51.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:40:51 smithi165 ceph-mon[36302]: pgmap v6073: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:40:51.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:40:51 smithi119 ceph-mon[31518]: pgmap v6073: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:40:51.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:40:51 smithi132 ceph-mon[35941]: pgmap v6073: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:40:52.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:40:52 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 11110.818645 secs 2024-03-28T03:40:52.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:40:52 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 11110.818645 secs 2024-03-28T03:40:52.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:40:52 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 11110.818645 secs 2024-03-28T03:40:53.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:40:53 smithi165 ceph-mon[36302]: pgmap v6074: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:40:53.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:40:53 smithi119 ceph-mon[31518]: pgmap v6074: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:40:53.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:40:53 smithi132 ceph-mon[35941]: pgmap v6074: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:40:54.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:40:54 smithi165 ceph-mon[36302]: pgmap v6075: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:40:54.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:40:54 smithi119 ceph-mon[31518]: pgmap v6075: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:40:54.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:40:54 smithi132 ceph-mon[35941]: pgmap v6075: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:40:56.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:40:56 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 11115.818705 secs 2024-03-28T03:40:56.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:40:56 smithi165 ceph-mon[36302]: pgmap v6076: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:40:56.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:40:56 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 11115.818705 secs 2024-03-28T03:40:56.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:40:56 smithi119 ceph-mon[31518]: pgmap v6076: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:40:56.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:40:56 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 11115.818705 secs 2024-03-28T03:40:56.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:40:56 smithi132 ceph-mon[35941]: pgmap v6076: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:40:58.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:40:58 smithi165 ceph-mon[36302]: pgmap v6077: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:40:58.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:40:58 smithi119 ceph-mon[31518]: pgmap v6077: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:40:58.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:40:58 smithi132 ceph-mon[35941]: pgmap v6077: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:41:00.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:41:00 smithi165 ceph-mon[36302]: pgmap v6078: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:41:00.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:41:00 smithi119 ceph-mon[31518]: pgmap v6078: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:41:00.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:41:00 smithi132 ceph-mon[35941]: pgmap v6078: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:41:01.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:41:01 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 11120.818792 secs 2024-03-28T03:41:01.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:41:01 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 11120.818792 secs 2024-03-28T03:41:01.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:41:01 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 11120.818792 secs 2024-03-28T03:41:02.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:41:02 smithi165 ceph-mon[36302]: pgmap v6079: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:41:02.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:41:02 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T03:41:02.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:41:02 smithi119 ceph-mon[31518]: pgmap v6079: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:41:02.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:41:02 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T03:41:02.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:41:02 smithi132 ceph-mon[35941]: pgmap v6079: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:41:02.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:41:02 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T03:41:03.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:41:03 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T03:41:03.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:41:03 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T03:41:03.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:41:03 smithi165 ceph-mon[36302]: pgmap v6080: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T03:41:03.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:41:03 smithi165 ceph-mon[36302]: pgmap v6081: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:41:03.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:41:03 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:41:03.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:41:03 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T03:41:03.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:41:03 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T03:41:03.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:41:03 smithi119 ceph-mon[31518]: pgmap v6080: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T03:41:03.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:41:03 smithi119 ceph-mon[31518]: pgmap v6081: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:41:03.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:41:03 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:41:03.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:41:03 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T03:41:03.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:41:03 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T03:41:03.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:41:03 smithi132 ceph-mon[35941]: pgmap v6080: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T03:41:03.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:41:03 smithi132 ceph-mon[35941]: pgmap v6081: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:41:03.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:41:03 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:41:06.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:41:05 smithi165 ceph-mon[36302]: pgmap v6082: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:41:06.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:41:05 smithi119 ceph-mon[31518]: pgmap v6082: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:41:06.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:41:05 smithi132 ceph-mon[35941]: pgmap v6082: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:41:07.053 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:41:06 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 11125.818832 secs 2024-03-28T03:41:07.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:41:06 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 11125.818832 secs 2024-03-28T03:41:07.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:41:06 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 11125.818832 secs 2024-03-28T03:41:08.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:41:07 smithi165 ceph-mon[36302]: pgmap v6083: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:41:08.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:41:07 smithi119 ceph-mon[31518]: pgmap v6083: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:41:08.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:41:07 smithi132 ceph-mon[35941]: pgmap v6083: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:41:10.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:41:09 smithi165 ceph-mon[36302]: pgmap v6084: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 3 op/s 2024-03-28T03:41:10.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:41:09 smithi119 ceph-mon[31518]: pgmap v6084: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 3 op/s 2024-03-28T03:41:10.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:41:09 smithi132 ceph-mon[35941]: pgmap v6084: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 3 op/s 2024-03-28T03:41:12.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:41:11 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 11130.818903 secs 2024-03-28T03:41:12.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:41:11 smithi165 ceph-mon[36302]: pgmap v6085: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:41:12.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:41:11 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 11130.818903 secs 2024-03-28T03:41:12.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:41:11 smithi119 ceph-mon[31518]: pgmap v6085: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:41:12.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:41:11 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 11130.818903 secs 2024-03-28T03:41:12.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:41:11 smithi132 ceph-mon[35941]: pgmap v6085: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:41:14.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:41:13 smithi165 ceph-mon[36302]: pgmap v6086: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-28T03:41:14.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:41:13 smithi119 ceph-mon[31518]: pgmap v6086: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-28T03:41:14.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:41:13 smithi132 ceph-mon[35941]: pgmap v6086: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-28T03:41:16.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:41:15 smithi165 ceph-mon[36302]: pgmap v6087: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:41:16.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:41:15 smithi119 ceph-mon[31518]: pgmap v6087: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:41:16.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:41:15 smithi132 ceph-mon[35941]: pgmap v6087: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:41:17.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:41:16 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 11135.818916 secs 2024-03-28T03:41:17.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:41:16 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 11135.818916 secs 2024-03-28T03:41:17.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:41:16 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 11135.818916 secs 2024-03-28T03:41:18.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:41:17 smithi165 ceph-mon[36302]: pgmap v6088: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:41:18.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:41:17 smithi119 ceph-mon[31518]: pgmap v6088: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:41:18.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:41:17 smithi132 ceph-mon[35941]: pgmap v6088: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:41:20.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:41:19 smithi165 ceph-mon[36302]: pgmap v6089: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:41:20.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:41:19 smithi119 ceph-mon[31518]: pgmap v6089: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:41:20.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:41:19 smithi132 ceph-mon[35941]: pgmap v6089: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:41:22.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:41:21 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 11140.818979 secs 2024-03-28T03:41:22.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:41:21 smithi165 ceph-mon[36302]: pgmap v6090: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:41:22.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:41:21 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 11140.818979 secs 2024-03-28T03:41:22.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:41:21 smithi119 ceph-mon[31518]: pgmap v6090: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:41:22.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:41:21 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 11140.818979 secs 2024-03-28T03:41:22.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:41:21 smithi132 ceph-mon[35941]: pgmap v6090: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:41:24.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:41:23 smithi165 ceph-mon[36302]: pgmap v6091: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:41:24.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:41:23 smithi119 ceph-mon[31518]: pgmap v6091: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:41:24.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:41:23 smithi132 ceph-mon[35941]: pgmap v6091: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:41:26.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:41:25 smithi165 ceph-mon[36302]: pgmap v6092: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:41:26.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:41:25 smithi119 ceph-mon[31518]: pgmap v6092: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:41:26.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:41:25 smithi132 ceph-mon[35941]: pgmap v6092: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:41:27.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:41:26 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 11145.819041 secs 2024-03-28T03:41:27.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:41:26 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 11145.819041 secs 2024-03-28T03:41:27.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:41:26 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 11145.819041 secs 2024-03-28T03:41:28.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:41:27 smithi165 ceph-mon[36302]: pgmap v6093: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:41:28.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:41:27 smithi119 ceph-mon[31518]: pgmap v6093: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:41:28.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:41:27 smithi132 ceph-mon[35941]: pgmap v6093: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:41:30.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:41:29 smithi165 ceph-mon[36302]: pgmap v6094: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:41:30.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:41:29 smithi119 ceph-mon[31518]: pgmap v6094: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:41:30.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:41:29 smithi132 ceph-mon[35941]: pgmap v6094: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:41:32.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:41:31 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 11150.819055 secs 2024-03-28T03:41:32.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:41:31 smithi165 ceph-mon[36302]: pgmap v6095: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:41:32.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:41:31 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 11150.819055 secs 2024-03-28T03:41:32.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:41:31 smithi119 ceph-mon[31518]: pgmap v6095: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:41:32.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:41:31 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 11150.819055 secs 2024-03-28T03:41:32.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:41:31 smithi132 ceph-mon[35941]: pgmap v6095: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:41:34.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:41:33 smithi165 ceph-mon[36302]: pgmap v6096: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:41:34.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:41:33 smithi119 ceph-mon[31518]: pgmap v6096: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:41:34.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:41:33 smithi132 ceph-mon[35941]: pgmap v6096: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:41:36.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:41:35 smithi165 ceph-mon[36302]: pgmap v6097: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:41:36.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:41:35 smithi119 ceph-mon[31518]: pgmap v6097: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:41:36.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:41:35 smithi132 ceph-mon[35941]: pgmap v6097: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:41:37.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:41:36 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 11155.819115 secs 2024-03-28T03:41:37.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:41:36 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 11155.819115 secs 2024-03-28T03:41:37.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:41:36 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 11155.819115 secs 2024-03-28T03:41:37.520 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:41:37.518+0000 7fc46aa00640 1 -- 172.21.15.119:0/210561562 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc44c0060f0 con 0x7fc474102010 2024-03-28T03:41:37.521 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:41:37.519+0000 7fc46be00640 1 -- 172.21.15.119:0/210561562 <== mon.1 v2:172.21.15.165:3300/0 11 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (secure 0 0 0) 0x7fc44c0060f0 con 0x7fc474102010 2024-03-28T03:41:38.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:41:37 smithi165 ceph-mon[36302]: pgmap v6098: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:41:38.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:41:37 smithi119 ceph-mon[31518]: pgmap v6098: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:41:38.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:41:37 smithi132 ceph-mon[35941]: pgmap v6098: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:41:40.130 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:41:39 smithi165 ceph-mon[36302]: pgmap v6099: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:41:40.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:41:39 smithi119 ceph-mon[31518]: pgmap v6099: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:41:40.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:41:39 smithi132 ceph-mon[35941]: pgmap v6099: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:41:42.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:41:41 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 11160.819129 secs 2024-03-28T03:41:42.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:41:41 smithi165 ceph-mon[36302]: pgmap v6100: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:41:42.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:41:41 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 11160.819129 secs 2024-03-28T03:41:42.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:41:41 smithi119 ceph-mon[31518]: pgmap v6100: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:41:42.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:41:41 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 11160.819129 secs 2024-03-28T03:41:42.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:41:41 smithi132 ceph-mon[35941]: pgmap v6100: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:41:44.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:41:43 smithi165 ceph-mon[36302]: pgmap v6101: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:41:44.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:41:43 smithi119 ceph-mon[31518]: pgmap v6101: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:41:44.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:41:43 smithi132 ceph-mon[35941]: pgmap v6101: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:41:46.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:41:45 smithi165 ceph-mon[36302]: pgmap v6102: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:41:46.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:41:45 smithi119 ceph-mon[31518]: pgmap v6102: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:41:46.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:41:45 smithi132 ceph-mon[35941]: pgmap v6102: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:41:47.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:41:46 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 11165.819190 secs 2024-03-28T03:41:47.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:41:46 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 11165.819190 secs 2024-03-28T03:41:47.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:41:46 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 11165.819190 secs 2024-03-28T03:41:48.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:41:47 smithi165 ceph-mon[36302]: pgmap v6103: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:41:48.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:41:47 smithi119 ceph-mon[31518]: pgmap v6103: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:41:48.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:41:47 smithi132 ceph-mon[35941]: pgmap v6103: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:41:50.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:41:49 smithi165 ceph-mon[36302]: pgmap v6104: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:41:50.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:41:49 smithi119 ceph-mon[31518]: pgmap v6104: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:41:50.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:41:49 smithi132 ceph-mon[35941]: pgmap v6104: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:41:52.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:41:51 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 11170.819278 secs 2024-03-28T03:41:52.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:41:51 smithi165 ceph-mon[36302]: pgmap v6105: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:41:52.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:41:51 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 11170.819278 secs 2024-03-28T03:41:52.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:41:51 smithi119 ceph-mon[31518]: pgmap v6105: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:41:52.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:41:51 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 11170.819278 secs 2024-03-28T03:41:52.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:41:51 smithi132 ceph-mon[35941]: pgmap v6105: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:41:54.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:41:53 smithi165 ceph-mon[36302]: pgmap v6106: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:41:54.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:41:53 smithi119 ceph-mon[31518]: pgmap v6106: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:41:54.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:41:53 smithi132 ceph-mon[35941]: pgmap v6106: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:41:56.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:41:55 smithi165 ceph-mon[36302]: pgmap v6107: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:41:56.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:41:55 smithi119 ceph-mon[31518]: pgmap v6107: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:41:56.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:41:55 smithi132 ceph-mon[35941]: pgmap v6107: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:41:57.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:41:56 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 11175.819339 secs 2024-03-28T03:41:57.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:41:56 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 11175.819339 secs 2024-03-28T03:41:57.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:41:56 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 11175.819339 secs 2024-03-28T03:41:58.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:41:57 smithi165 ceph-mon[36302]: pgmap v6108: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:41:58.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:41:57 smithi119 ceph-mon[31518]: pgmap v6108: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:41:58.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:41:57 smithi132 ceph-mon[35941]: pgmap v6108: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:42:00.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:41:59 smithi119 ceph-mon[31518]: pgmap v6109: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:42:00.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:41:59 smithi132 ceph-mon[35941]: pgmap v6109: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:42:00.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:41:59 smithi165 ceph-mon[36302]: pgmap v6109: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:42:02.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:42:01 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 11180.819434 secs 2024-03-28T03:42:02.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:42:01 smithi119 ceph-mon[31518]: pgmap v6110: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:42:02.180 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:42:01 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 11180.819434 secs 2024-03-28T03:42:02.180 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:42:01 smithi165 ceph-mon[36302]: pgmap v6110: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:42:02.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:42:01 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 11180.819434 secs 2024-03-28T03:42:02.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:42:01 smithi132 ceph-mon[35941]: pgmap v6110: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:42:04.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:42:03 smithi119 ceph-mon[31518]: pgmap v6111: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:42:04.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:42:03 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T03:42:04.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:42:03 smithi132 ceph-mon[35941]: pgmap v6111: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:42:04.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:42:03 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T03:42:04.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:42:03 smithi165 ceph-mon[36302]: pgmap v6111: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:42:04.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:42:03 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T03:42:05.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:42:04 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T03:42:05.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:42:04 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T03:42:05.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:42:04 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:42:05.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:42:04 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T03:42:05.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:42:04 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T03:42:05.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:42:04 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:42:05.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:42:04 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T03:42:05.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:42:04 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T03:42:05.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:42:04 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:42:06.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:42:05 smithi119 ceph-mon[31518]: pgmap v6112: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T03:42:06.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:42:05 smithi119 ceph-mon[31518]: pgmap v6113: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:42:06.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:42:05 smithi132 ceph-mon[35941]: pgmap v6112: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T03:42:06.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:42:05 smithi132 ceph-mon[35941]: pgmap v6113: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:42:06.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:42:05 smithi165 ceph-mon[36302]: pgmap v6112: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T03:42:06.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:42:05 smithi165 ceph-mon[36302]: pgmap v6113: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:42:07.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:42:06 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 11185.819476 secs 2024-03-28T03:42:07.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:42:06 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 11185.819476 secs 2024-03-28T03:42:07.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:42:06 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 11185.819476 secs 2024-03-28T03:42:08.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:42:07 smithi132 ceph-mon[35941]: pgmap v6114: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:42:08.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:42:07 smithi165 ceph-mon[36302]: pgmap v6114: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:42:08.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:42:07 smithi119 ceph-mon[31518]: pgmap v6114: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:42:10.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:42:09 smithi132 ceph-mon[35941]: pgmap v6115: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:42:10.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:42:09 smithi165 ceph-mon[36302]: pgmap v6115: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:42:10.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:42:09 smithi119 ceph-mon[31518]: pgmap v6115: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:42:12.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:42:11 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 11190.819565 secs 2024-03-28T03:42:12.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:42:11 smithi132 ceph-mon[35941]: pgmap v6116: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:42:12.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:42:11 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 11190.819565 secs 2024-03-28T03:42:12.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:42:11 smithi165 ceph-mon[36302]: pgmap v6116: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:42:12.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:42:11 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 11190.819565 secs 2024-03-28T03:42:12.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:42:11 smithi119 ceph-mon[31518]: pgmap v6116: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:42:14.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:42:13 smithi132 ceph-mon[35941]: pgmap v6117: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:42:14.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:42:13 smithi165 ceph-mon[36302]: pgmap v6117: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:42:14.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:42:13 smithi119 ceph-mon[31518]: pgmap v6117: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:42:16.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:42:15 smithi132 ceph-mon[35941]: pgmap v6118: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T03:42:16.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:42:15 smithi165 ceph-mon[36302]: pgmap v6118: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T03:42:16.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:42:15 smithi119 ceph-mon[31518]: pgmap v6118: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T03:42:17.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:42:16 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 11195.819624 secs 2024-03-28T03:42:17.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:42:16 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 11195.819624 secs 2024-03-28T03:42:17.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:42:16 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 11195.819624 secs 2024-03-28T03:42:18.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:42:17 smithi132 ceph-mon[35941]: pgmap v6119: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:42:18.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:42:17 smithi165 ceph-mon[36302]: pgmap v6119: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:42:18.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:42:17 smithi119 ceph-mon[31518]: pgmap v6119: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:42:20.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:42:19 smithi132 ceph-mon[35941]: pgmap v6120: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:42:20.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:42:19 smithi165 ceph-mon[36302]: pgmap v6120: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:42:20.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:42:19 smithi119 ceph-mon[31518]: pgmap v6120: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:42:22.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:42:21 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 11200.819712 secs 2024-03-28T03:42:22.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:42:21 smithi165 ceph-mon[36302]: pgmap v6121: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:42:22.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:42:21 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 11200.819712 secs 2024-03-28T03:42:22.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:42:21 smithi119 ceph-mon[31518]: pgmap v6121: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:42:22.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:42:21 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 11200.819712 secs 2024-03-28T03:42:22.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:42:21 smithi132 ceph-mon[35941]: pgmap v6121: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:42:24.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:42:23 smithi165 ceph-mon[36302]: pgmap v6122: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:42:24.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:42:23 smithi119 ceph-mon[31518]: pgmap v6122: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:42:24.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:42:23 smithi132 ceph-mon[35941]: pgmap v6122: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:42:26.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:42:25 smithi165 ceph-mon[36302]: pgmap v6123: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:42:26.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:42:25 smithi119 ceph-mon[31518]: pgmap v6123: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:42:26.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:42:25 smithi132 ceph-mon[35941]: pgmap v6123: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:42:27.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:42:26 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 11205.819801 secs 2024-03-28T03:42:27.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:42:26 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 11205.819801 secs 2024-03-28T03:42:27.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:42:26 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 11205.819801 secs 2024-03-28T03:42:28.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:42:27 smithi165 ceph-mon[36302]: pgmap v6124: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:42:28.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:42:27 smithi119 ceph-mon[31518]: pgmap v6124: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:42:28.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:42:27 smithi132 ceph-mon[35941]: pgmap v6124: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:42:30.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:42:29 smithi165 ceph-mon[36302]: pgmap v6125: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:42:30.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:42:29 smithi119 ceph-mon[31518]: pgmap v6125: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:42:30.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:42:29 smithi132 ceph-mon[35941]: pgmap v6125: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:42:32.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:42:31 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 11210.819891 secs 2024-03-28T03:42:32.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:42:31 smithi165 ceph-mon[36302]: pgmap v6126: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:42:32.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:42:31 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 11210.819891 secs 2024-03-28T03:42:32.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:42:31 smithi119 ceph-mon[31518]: pgmap v6126: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:42:32.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:42:31 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 11210.819891 secs 2024-03-28T03:42:32.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:42:31 smithi132 ceph-mon[35941]: pgmap v6126: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:42:34.250 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:42:33 smithi132 ceph-mon[35941]: pgmap v6127: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:42:34.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:42:33 smithi165 ceph-mon[36302]: pgmap v6127: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:42:34.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:42:33 smithi119 ceph-mon[31518]: pgmap v6127: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:42:36.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:42:35 smithi165 ceph-mon[36302]: pgmap v6128: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:42:36.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:42:35 smithi119 ceph-mon[31518]: pgmap v6128: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:42:36.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:42:35 smithi132 ceph-mon[35941]: pgmap v6128: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:42:37.303 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:42:37 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 11215.819977 secs 2024-03-28T03:42:37.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:42:37 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 11215.819977 secs 2024-03-28T03:42:37.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:42:37 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 11215.819977 secs 2024-03-28T03:42:38.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:42:38 smithi165 ceph-mon[36302]: pgmap v6129: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:42:38.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:42:38 smithi119 ceph-mon[31518]: pgmap v6129: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:42:38.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:42:38 smithi132 ceph-mon[35941]: pgmap v6129: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:42:40.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:42:40 smithi165 ceph-mon[36302]: pgmap v6130: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:42:40.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:42:40 smithi119 ceph-mon[31518]: pgmap v6130: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:42:40.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:42:40 smithi132 ceph-mon[35941]: pgmap v6130: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:42:42.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:42:42 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 11220.820041 secs 2024-03-28T03:42:42.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:42:42 smithi165 ceph-mon[36302]: pgmap v6131: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:42:42.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:42:42 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 11220.820041 secs 2024-03-28T03:42:42.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:42:42 smithi119 ceph-mon[31518]: pgmap v6131: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:42:42.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:42:42 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 11220.820041 secs 2024-03-28T03:42:42.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:42:42 smithi132 ceph-mon[35941]: pgmap v6131: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:42:44.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:42:44 smithi165 ceph-mon[36302]: pgmap v6132: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:42:44.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:42:44 smithi119 ceph-mon[31518]: pgmap v6132: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:42:44.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:42:44 smithi132 ceph-mon[35941]: pgmap v6132: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:42:46.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:42:46 smithi165 ceph-mon[36302]: pgmap v6133: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:42:46.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:42:46 smithi119 ceph-mon[31518]: pgmap v6133: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:42:46.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:42:46 smithi132 ceph-mon[35941]: pgmap v6133: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:42:47.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:42:47 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 11225.820126 secs 2024-03-28T03:42:47.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:42:47 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 11225.820126 secs 2024-03-28T03:42:47.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:42:47 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 11225.820126 secs 2024-03-28T03:42:48.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:42:48 smithi165 ceph-mon[36302]: pgmap v6134: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:42:48.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:42:48 smithi119 ceph-mon[31518]: pgmap v6134: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:42:48.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:42:48 smithi132 ceph-mon[35941]: pgmap v6134: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:42:50.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:42:50 smithi165 ceph-mon[36302]: pgmap v6135: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:42:50.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:42:50 smithi119 ceph-mon[31518]: pgmap v6135: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:42:50.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:42:50 smithi132 ceph-mon[35941]: pgmap v6135: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:42:52.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:42:52 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 11230.820217 secs 2024-03-28T03:42:52.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:42:52 smithi165 ceph-mon[36302]: pgmap v6136: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:42:52.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:42:52 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 11230.820217 secs 2024-03-28T03:42:52.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:42:52 smithi119 ceph-mon[31518]: pgmap v6136: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:42:52.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:42:52 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 11230.820217 secs 2024-03-28T03:42:52.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:42:52 smithi132 ceph-mon[35941]: pgmap v6136: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:42:54.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:42:54 smithi165 ceph-mon[36302]: pgmap v6137: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:42:54.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:42:54 smithi119 ceph-mon[31518]: pgmap v6137: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:42:54.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:42:54 smithi132 ceph-mon[35941]: pgmap v6137: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:42:56.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:42:56 smithi165 ceph-mon[36302]: pgmap v6138: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:42:56.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:42:56 smithi119 ceph-mon[31518]: pgmap v6138: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:42:56.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:42:56 smithi132 ceph-mon[35941]: pgmap v6138: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:42:57.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:42:57 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 11235.820228 secs 2024-03-28T03:42:57.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:42:57 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 11235.820228 secs 2024-03-28T03:42:57.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:42:57 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 11235.820228 secs 2024-03-28T03:42:58.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:42:58 smithi165 ceph-mon[36302]: pgmap v6139: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:42:58.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:42:58 smithi119 ceph-mon[31518]: pgmap v6139: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:42:58.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:42:58 smithi132 ceph-mon[35941]: pgmap v6139: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:43:00.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:43:00 smithi165 ceph-mon[36302]: pgmap v6140: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:43:00.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:43:00 smithi119 ceph-mon[31518]: pgmap v6140: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:43:00.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:43:00 smithi132 ceph-mon[35941]: pgmap v6140: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:43:02.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:43:02 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 11240.820315 secs 2024-03-28T03:43:02.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:43:02 smithi165 ceph-mon[36302]: pgmap v6141: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:43:02.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:43:02 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 11240.820315 secs 2024-03-28T03:43:02.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:43:02 smithi119 ceph-mon[31518]: pgmap v6141: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:43:02.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:43:02 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 11240.820315 secs 2024-03-28T03:43:02.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:43:02 smithi132 ceph-mon[35941]: pgmap v6141: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:43:03.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:43:03 smithi119 ceph-mon[31518]: pgmap v6142: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:43:03.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:43:03 smithi132 ceph-mon[35941]: pgmap v6142: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:43:03.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:43:03 smithi165 ceph-mon[36302]: pgmap v6142: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:43:05.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:43:05 smithi165 ceph-mon[36302]: pgmap v6143: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:43:05.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:43:05 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T03:43:05.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:43:05 smithi119 ceph-mon[31518]: pgmap v6143: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:43:05.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:43:05 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T03:43:05.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:43:05 smithi132 ceph-mon[35941]: pgmap v6143: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:43:05.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:43:05 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T03:43:06.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:43:06 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 11245.820400 secs 2024-03-28T03:43:06.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:43:06 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T03:43:06.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:43:06 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T03:43:06.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:43:06 smithi165 ceph-mon[36302]: pgmap v6144: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T03:43:06.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:43:06 smithi165 ceph-mon[36302]: pgmap v6145: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:43:06.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:43:06 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:43:06.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:43:06 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 11245.820400 secs 2024-03-28T03:43:06.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:43:06 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T03:43:06.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:43:06 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T03:43:06.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:43:06 smithi119 ceph-mon[31518]: pgmap v6144: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T03:43:06.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:43:06 smithi119 ceph-mon[31518]: pgmap v6145: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:43:06.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:43:06 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:43:06.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:43:06 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 11245.820400 secs 2024-03-28T03:43:06.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:43:06 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T03:43:06.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:43:06 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T03:43:06.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:43:06 smithi132 ceph-mon[35941]: pgmap v6144: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T03:43:06.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:43:06 smithi132 ceph-mon[35941]: pgmap v6145: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:43:06.699 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:43:06 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:43:09.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:43:08 smithi165 ceph-mon[36302]: pgmap v6146: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:43:09.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:43:08 smithi119 ceph-mon[31518]: pgmap v6146: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:43:09.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:43:08 smithi132 ceph-mon[35941]: pgmap v6146: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:43:11.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:43:10 smithi165 ceph-mon[36302]: pgmap v6147: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:43:11.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:43:10 smithi119 ceph-mon[31518]: pgmap v6147: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:43:11.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:43:10 smithi132 ceph-mon[35941]: pgmap v6147: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:43:12.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:43:11 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 11250.820488 secs 2024-03-28T03:43:12.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:43:11 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 11250.820488 secs 2024-03-28T03:43:12.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:43:11 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 11250.820488 secs 2024-03-28T03:43:13.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:43:12 smithi165 ceph-mon[36302]: pgmap v6148: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:43:13.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:43:12 smithi119 ceph-mon[31518]: pgmap v6148: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:43:13.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:43:12 smithi132 ceph-mon[35941]: pgmap v6148: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:43:15.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:43:14 smithi165 ceph-mon[36302]: pgmap v6149: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-28T03:43:15.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:43:14 smithi119 ceph-mon[31518]: pgmap v6149: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-28T03:43:15.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:43:14 smithi132 ceph-mon[35941]: pgmap v6149: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-28T03:43:17.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:43:16 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 11255.820549 secs 2024-03-28T03:43:17.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:43:16 smithi165 ceph-mon[36302]: pgmap v6150: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T03:43:17.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:43:16 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 11255.820549 secs 2024-03-28T03:43:17.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:43:16 smithi119 ceph-mon[31518]: pgmap v6150: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T03:43:17.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:43:16 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 11255.820549 secs 2024-03-28T03:43:17.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:43:16 smithi132 ceph-mon[35941]: pgmap v6150: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T03:43:19.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:43:18 smithi165 ceph-mon[36302]: pgmap v6151: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:43:19.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:43:18 smithi119 ceph-mon[31518]: pgmap v6151: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:43:19.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:43:18 smithi132 ceph-mon[35941]: pgmap v6151: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:43:21.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:43:20 smithi165 ceph-mon[36302]: pgmap v6152: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:43:21.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:43:20 smithi119 ceph-mon[31518]: pgmap v6152: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:43:21.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:43:20 smithi132 ceph-mon[35941]: pgmap v6152: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:43:22.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:43:21 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 11260.820637 secs 2024-03-28T03:43:22.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:43:21 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 11260.820637 secs 2024-03-28T03:43:22.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:43:21 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 11260.820637 secs 2024-03-28T03:43:23.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:43:22 smithi165 ceph-mon[36302]: pgmap v6153: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:43:23.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:43:22 smithi119 ceph-mon[31518]: pgmap v6153: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:43:23.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:43:22 smithi132 ceph-mon[35941]: pgmap v6153: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:43:25.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:43:24 smithi165 ceph-mon[36302]: pgmap v6154: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:43:25.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:43:24 smithi119 ceph-mon[31518]: pgmap v6154: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:43:25.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:43:24 smithi132 ceph-mon[35941]: pgmap v6154: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:43:27.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:43:26 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 11265.820697 secs 2024-03-28T03:43:27.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:43:26 smithi165 ceph-mon[36302]: pgmap v6155: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:43:27.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:43:26 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 11265.820697 secs 2024-03-28T03:43:27.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:43:26 smithi119 ceph-mon[31518]: pgmap v6155: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:43:27.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:43:26 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 11265.820697 secs 2024-03-28T03:43:27.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:43:26 smithi132 ceph-mon[35941]: pgmap v6155: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:43:29.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:43:28 smithi165 ceph-mon[36302]: pgmap v6156: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:43:29.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:43:28 smithi119 ceph-mon[31518]: pgmap v6156: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:43:29.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:43:28 smithi132 ceph-mon[35941]: pgmap v6156: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:43:31.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:43:30 smithi165 ceph-mon[36302]: pgmap v6157: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:43:31.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:43:30 smithi119 ceph-mon[31518]: pgmap v6157: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:43:31.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:43:30 smithi132 ceph-mon[35941]: pgmap v6157: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:43:32.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:43:31 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 11270.820785 secs 2024-03-28T03:43:32.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:43:31 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 11270.820785 secs 2024-03-28T03:43:32.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:43:31 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 11270.820785 secs 2024-03-28T03:43:33.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:43:32 smithi165 ceph-mon[36302]: pgmap v6158: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:43:33.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:43:32 smithi119 ceph-mon[31518]: pgmap v6158: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:43:33.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:43:32 smithi132 ceph-mon[35941]: pgmap v6158: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:43:35.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:43:34 smithi165 ceph-mon[36302]: pgmap v6159: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:43:35.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:43:34 smithi119 ceph-mon[31518]: pgmap v6159: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:43:35.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:43:34 smithi132 ceph-mon[35941]: pgmap v6159: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:43:37.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:43:36 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 11275.820844 secs 2024-03-28T03:43:37.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:43:36 smithi165 ceph-mon[36302]: pgmap v6160: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:43:37.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:43:36 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 11275.820844 secs 2024-03-28T03:43:37.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:43:36 smithi119 ceph-mon[31518]: pgmap v6160: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:43:37.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:43:36 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 11275.820844 secs 2024-03-28T03:43:37.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:43:36 smithi132 ceph-mon[35941]: pgmap v6160: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:43:39.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:43:38 smithi165 ceph-mon[36302]: pgmap v6161: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:43:39.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:43:38 smithi119 ceph-mon[31518]: pgmap v6161: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:43:39.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:43:38 smithi132 ceph-mon[35941]: pgmap v6161: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:43:41.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:43:40 smithi165 ceph-mon[36302]: pgmap v6162: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:43:41.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:43:40 smithi119 ceph-mon[31518]: pgmap v6162: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:43:41.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:43:40 smithi132 ceph-mon[35941]: pgmap v6162: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:43:42.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:43:41 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 11280.820885 secs 2024-03-28T03:43:42.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:43:41 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 11280.820885 secs 2024-03-28T03:43:42.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:43:41 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 11280.820885 secs 2024-03-28T03:43:43.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:43:42 smithi119 ceph-mon[31518]: pgmap v6163: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:43:43.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:43:42 smithi132 ceph-mon[35941]: pgmap v6163: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:43:43.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:43:42 smithi165 ceph-mon[36302]: pgmap v6163: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:43:45.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:43:44 smithi119 ceph-mon[31518]: pgmap v6164: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:43:45.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:43:44 smithi132 ceph-mon[35941]: pgmap v6164: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:43:45.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:43:44 smithi165 ceph-mon[36302]: pgmap v6164: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:43:47.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:43:46 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 11285.820895 secs 2024-03-28T03:43:47.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:43:46 smithi119 ceph-mon[31518]: pgmap v6165: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:43:47.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:43:46 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 11285.820895 secs 2024-03-28T03:43:47.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:43:46 smithi132 ceph-mon[35941]: pgmap v6165: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:43:47.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:43:46 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 11285.820895 secs 2024-03-28T03:43:47.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:43:46 smithi165 ceph-mon[36302]: pgmap v6165: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:43:49.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:43:48 smithi132 ceph-mon[35941]: pgmap v6166: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:43:49.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:43:48 smithi165 ceph-mon[36302]: pgmap v6166: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:43:49.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:43:48 smithi119 ceph-mon[31518]: pgmap v6166: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:43:51.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:43:50 smithi132 ceph-mon[35941]: pgmap v6167: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:43:51.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:43:50 smithi165 ceph-mon[36302]: pgmap v6167: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:43:51.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:43:50 smithi119 ceph-mon[31518]: pgmap v6167: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:43:52.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:43:51 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 11290.820930 secs 2024-03-28T03:43:52.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:43:51 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 11290.820930 secs 2024-03-28T03:43:52.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:43:51 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 11290.820930 secs 2024-03-28T03:43:53.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:43:52 smithi132 ceph-mon[35941]: pgmap v6168: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:43:53.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:43:52 smithi165 ceph-mon[36302]: pgmap v6168: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:43:53.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:43:52 smithi119 ceph-mon[31518]: pgmap v6168: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:43:55.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:43:54 smithi132 ceph-mon[35941]: pgmap v6169: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:43:55.204 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:43:54 smithi119 ceph-mon[31518]: pgmap v6169: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:43:55.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:43:54 smithi165 ceph-mon[36302]: pgmap v6169: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:43:57.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:43:56 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 11295.821013 secs 2024-03-28T03:43:57.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:43:56 smithi132 ceph-mon[35941]: pgmap v6170: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:43:57.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:43:56 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 11295.821013 secs 2024-03-28T03:43:57.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:43:56 smithi165 ceph-mon[36302]: pgmap v6170: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:43:57.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:43:56 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 11295.821013 secs 2024-03-28T03:43:57.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:43:56 smithi119 ceph-mon[31518]: pgmap v6170: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:43:59.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:43:58 smithi132 ceph-mon[35941]: pgmap v6171: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:43:59.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:43:58 smithi165 ceph-mon[36302]: pgmap v6171: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:43:59.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:43:58 smithi119 ceph-mon[31518]: pgmap v6171: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:44:01.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:44:00 smithi132 ceph-mon[35941]: pgmap v6172: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:44:01.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:44:00 smithi165 ceph-mon[36302]: pgmap v6172: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:44:01.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:44:00 smithi119 ceph-mon[31518]: pgmap v6172: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:44:02.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:44:01 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 11300.821099 secs 2024-03-28T03:44:02.279 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:44:01 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 11300.821099 secs 2024-03-28T03:44:02.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:44:01 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 11300.821099 secs 2024-03-28T03:44:03.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:44:02 smithi165 ceph-mon[36302]: pgmap v6173: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:44:03.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:44:02 smithi119 ceph-mon[31518]: pgmap v6173: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:44:03.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:44:02 smithi132 ceph-mon[35941]: pgmap v6173: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:44:05.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:44:04 smithi165 ceph-mon[36302]: pgmap v6174: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:44:05.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:44:04 smithi119 ceph-mon[31518]: pgmap v6174: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:44:05.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:44:04 smithi132 ceph-mon[35941]: pgmap v6174: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:44:07.250 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:44:06 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 11305.821132 secs 2024-03-28T03:44:07.250 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:44:06 smithi132 ceph-mon[35941]: pgmap v6175: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:44:07.250 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:44:06 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T03:44:07.250 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:44:06 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:44:07.250 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:44:06 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:44:07.251 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:44:06 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:44:07.251 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:44:06 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:44:07.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:44:06 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 11305.821132 secs 2024-03-28T03:44:07.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:44:06 smithi165 ceph-mon[36302]: pgmap v6175: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:44:07.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:44:06 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T03:44:07.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:44:06 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:44:07.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:44:06 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:44:07.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:44:06 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:44:07.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:44:06 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:44:07.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:44:06 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 11305.821132 secs 2024-03-28T03:44:07.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:44:06 smithi119 ceph-mon[31518]: pgmap v6175: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:44:07.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:44:06 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T03:44:07.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:44:06 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:44:07.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:44:06 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:44:07.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:44:06 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:44:07.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:44:06 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:44:09.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:44:08 smithi165 ceph-mon[36302]: pgmap v6176: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:44:09.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:44:08 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:44:09.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:44:08 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:44:09.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:44:08 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:44:09.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:44:08 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:44:09.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:44:08 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:44:09.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:44:08 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:44:09.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:44:08 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:44:09.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:44:08 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:44:09.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:44:08 smithi119 ceph-mon[31518]: pgmap v6176: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:44:09.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:44:08 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:44:09.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:44:08 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:44:09.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:44:08 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:44:09.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:44:08 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:44:09.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:44:08 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:44:09.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:44:08 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:44:09.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:44:08 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:44:09.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:44:08 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:44:09.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:44:08 smithi132 ceph-mon[35941]: pgmap v6176: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:44:09.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:44:08 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:44:09.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:44:08 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:44:09.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:44:08 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:44:09.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:44:08 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:44:09.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:44:08 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:44:09.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:44:08 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:44:09.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:44:08 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:44:09.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:44:08 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:44:10.275 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:44:09 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:44:10.275 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:44:09 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:44:10.275 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:44:09 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:44:10.275 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:44:09 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:44:10.276 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:44:09 smithi119 ceph-mon[31518]: Adjusting osd_memory_target on smithi132 to 174.1M 2024-03-28T03:44:10.276 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:44:09 smithi119 ceph-mon[31518]: Unable to set osd_memory_target on smithi132 to 182640640: error parsing value: Value '182640640' is below minimum 939524096 2024-03-28T03:44:10.276 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:44:09 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:44:10.276 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:44:09 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:44:10.276 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:44:09 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:44:10.276 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:44:09 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:44:10.276 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:44:09 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:44:10.276 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:44:09 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:44:10.276 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:44:09 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:44:10.277 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:44:09 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:44:10.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:44:09 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:44:10.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:44:09 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:44:10.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:44:09 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:44:10.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:44:09 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:44:10.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:44:09 smithi165 ceph-mon[36302]: Adjusting osd_memory_target on smithi132 to 174.1M 2024-03-28T03:44:10.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:44:09 smithi165 ceph-mon[36302]: Unable to set osd_memory_target on smithi132 to 182640640: error parsing value: Value '182640640' is below minimum 939524096 2024-03-28T03:44:10.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:44:09 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:44:10.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:44:09 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:44:10.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:44:09 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:44:10.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:44:09 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:44:10.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:44:09 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:44:10.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:44:09 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:44:10.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:44:09 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:44:10.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:44:09 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:44:10.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:44:09 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:44:10.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:44:09 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:44:10.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:44:09 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:44:10.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:44:09 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:44:10.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:44:09 smithi132 ceph-mon[35941]: Adjusting osd_memory_target on smithi132 to 174.1M 2024-03-28T03:44:10.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:44:09 smithi132 ceph-mon[35941]: Unable to set osd_memory_target on smithi132 to 182640640: error parsing value: Value '182640640' is below minimum 939524096 2024-03-28T03:44:10.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:44:09 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:44:10.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:44:09 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:44:10.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:44:09 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:44:10.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:44:09 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:44:10.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:44:09 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:44:10.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:44:09 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:44:10.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:44:09 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:44:10.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:44:09 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:44:11.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:44:10 smithi165 ceph-mon[36302]: Adjusting osd_memory_target on smithi165 to 174.1M 2024-03-28T03:44:11.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:44:10 smithi165 ceph-mon[36302]: Unable to set osd_memory_target on smithi165 to 182635622: error parsing value: Value '182635622' is below minimum 939524096 2024-03-28T03:44:11.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:44:10 smithi165 ceph-mon[36302]: pgmap v6177: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:44:11.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:44:10 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:44:11.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:44:10 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:44:11.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:44:10 smithi119 ceph-mon[31518]: Adjusting osd_memory_target on smithi165 to 174.1M 2024-03-28T03:44:11.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:44:10 smithi119 ceph-mon[31518]: Unable to set osd_memory_target on smithi165 to 182635622: error parsing value: Value '182635622' is below minimum 939524096 2024-03-28T03:44:11.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:44:10 smithi119 ceph-mon[31518]: pgmap v6177: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:44:11.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:44:10 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:44:11.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:44:10 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:44:11.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:44:10 smithi132 ceph-mon[35941]: Adjusting osd_memory_target on smithi165 to 174.1M 2024-03-28T03:44:11.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:44:10 smithi132 ceph-mon[35941]: Unable to set osd_memory_target on smithi165 to 182635622: error parsing value: Value '182635622' is below minimum 939524096 2024-03-28T03:44:11.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:44:10 smithi132 ceph-mon[35941]: pgmap v6177: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:44:11.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:44:10 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:44:11.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:44:10 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:44:12.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:44:11 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 11310.821192 secs 2024-03-28T03:44:12.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:44:11 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:44:12.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:44:11 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:44:12.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:44:11 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:44:12.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:44:11 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:44:12.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:44:11 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:44:12.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:44:11 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:44:12.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:44:11 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:44:12.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:44:11 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:44:12.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:44:11 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T03:44:12.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:44:11 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T03:44:12.380 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:44:11 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:44:12.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:44:11 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 11310.821192 secs 2024-03-28T03:44:12.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:44:11 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:44:12.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:44:11 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:44:12.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:44:11 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:44:12.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:44:11 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:44:12.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:44:11 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:44:12.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:44:11 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:44:12.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:44:11 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:44:12.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:44:11 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:44:12.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:44:11 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T03:44:12.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:44:11 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T03:44:12.403 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:44:11 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:44:12.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:44:11 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 11310.821192 secs 2024-03-28T03:44:12.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:44:11 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:44:12.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:44:11 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:44:12.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:44:11 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:44:12.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:44:11 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:44:12.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:44:11 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:44:12.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:44:11 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:44:12.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:44:11 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:44:12.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:44:11 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T03:44:12.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:44:11 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T03:44:12.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:44:11 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T03:44:12.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:44:11 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:44:13.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:44:12 smithi165 ceph-mon[36302]: pgmap v6178: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:44:13.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:44:12 smithi165 ceph-mon[36302]: Adjusting osd_memory_target on smithi119 to 176.7M 2024-03-28T03:44:13.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:44:12 smithi165 ceph-mon[36302]: Unable to set osd_memory_target on smithi119 to 185372364: error parsing value: Value '185372364' is below minimum 939524096 2024-03-28T03:44:13.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:44:12 smithi165 ceph-mon[36302]: pgmap v6179: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T03:44:13.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:44:12 smithi165 ceph-mon[36302]: pgmap v6180: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:44:13.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:44:12 smithi119 ceph-mon[31518]: pgmap v6178: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:44:13.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:44:12 smithi119 ceph-mon[31518]: Adjusting osd_memory_target on smithi119 to 176.7M 2024-03-28T03:44:13.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:44:12 smithi119 ceph-mon[31518]: Unable to set osd_memory_target on smithi119 to 185372364: error parsing value: Value '185372364' is below minimum 939524096 2024-03-28T03:44:13.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:44:12 smithi119 ceph-mon[31518]: pgmap v6179: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T03:44:13.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:44:12 smithi119 ceph-mon[31518]: pgmap v6180: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:44:13.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:44:12 smithi132 ceph-mon[35941]: pgmap v6178: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:44:13.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:44:12 smithi132 ceph-mon[35941]: Adjusting osd_memory_target on smithi119 to 176.7M 2024-03-28T03:44:13.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:44:12 smithi132 ceph-mon[35941]: Unable to set osd_memory_target on smithi119 to 185372364: error parsing value: Value '185372364' is below minimum 939524096 2024-03-28T03:44:13.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:44:12 smithi132 ceph-mon[35941]: pgmap v6179: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T03:44:13.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:44:12 smithi132 ceph-mon[35941]: pgmap v6180: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:44:15.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:44:14 smithi165 ceph-mon[36302]: pgmap v6181: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 3 op/s 2024-03-28T03:44:15.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:44:14 smithi119 ceph-mon[31518]: pgmap v6181: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 3 op/s 2024-03-28T03:44:15.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:44:14 smithi132 ceph-mon[35941]: pgmap v6181: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 3 op/s 2024-03-28T03:44:17.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:44:16 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 11315.821281 secs 2024-03-28T03:44:17.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:44:16 smithi165 ceph-mon[36302]: pgmap v6182: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:44:17.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:44:16 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 11315.821281 secs 2024-03-28T03:44:17.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:44:16 smithi119 ceph-mon[31518]: pgmap v6182: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:44:17.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:44:16 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 11315.821281 secs 2024-03-28T03:44:17.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:44:16 smithi132 ceph-mon[35941]: pgmap v6182: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:44:19.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:44:19 smithi165 ceph-mon[36302]: pgmap v6183: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 3 op/s 2024-03-28T03:44:19.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:44:19 smithi119 ceph-mon[31518]: pgmap v6183: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 3 op/s 2024-03-28T03:44:19.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:44:19 smithi132 ceph-mon[35941]: pgmap v6183: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 3 op/s 2024-03-28T03:44:21.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:44:21 smithi165 ceph-mon[36302]: pgmap v6184: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:44:21.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:44:21 smithi119 ceph-mon[31518]: pgmap v6184: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:44:21.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:44:21 smithi132 ceph-mon[35941]: pgmap v6184: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:44:22.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:44:22 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 11320.821342 secs 2024-03-28T03:44:22.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:44:22 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 11320.821342 secs 2024-03-28T03:44:22.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:44:22 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 11320.821342 secs 2024-03-28T03:44:23.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:44:23 smithi165 ceph-mon[36302]: pgmap v6185: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T03:44:23.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:44:23 smithi119 ceph-mon[31518]: pgmap v6185: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T03:44:23.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:44:23 smithi132 ceph-mon[35941]: pgmap v6185: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T03:44:25.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:44:25 smithi165 ceph-mon[36302]: pgmap v6186: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:44:25.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:44:25 smithi119 ceph-mon[31518]: pgmap v6186: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:44:25.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:44:25 smithi132 ceph-mon[35941]: pgmap v6186: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:44:27.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:44:27 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 11325.821401 secs 2024-03-28T03:44:27.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:44:27 smithi165 ceph-mon[36302]: pgmap v6187: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:44:27.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:44:27 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 11325.821401 secs 2024-03-28T03:44:27.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:44:27 smithi119 ceph-mon[31518]: pgmap v6187: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:44:27.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:44:27 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 11325.821401 secs 2024-03-28T03:44:27.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:44:27 smithi132 ceph-mon[35941]: pgmap v6187: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:44:29.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:44:29 smithi165 ceph-mon[36302]: pgmap v6188: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:44:29.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:44:29 smithi119 ceph-mon[31518]: pgmap v6188: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:44:29.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:44:29 smithi132 ceph-mon[35941]: pgmap v6188: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:44:31.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:44:31 smithi165 ceph-mon[36302]: pgmap v6189: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:44:31.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:44:31 smithi119 ceph-mon[31518]: pgmap v6189: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:44:31.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:44:31 smithi132 ceph-mon[35941]: pgmap v6189: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:44:32.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:44:32 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 11330.821462 secs 2024-03-28T03:44:32.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:44:32 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 11330.821462 secs 2024-03-28T03:44:32.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:44:32 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 11330.821462 secs 2024-03-28T03:44:33.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:44:33 smithi165 ceph-mon[36302]: pgmap v6190: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:44:33.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:44:33 smithi119 ceph-mon[31518]: pgmap v6190: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:44:33.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:44:33 smithi132 ceph-mon[35941]: pgmap v6190: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:44:35.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:44:35 smithi165 ceph-mon[36302]: pgmap v6191: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:44:35.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:44:35 smithi119 ceph-mon[31518]: pgmap v6191: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:44:35.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:44:35 smithi132 ceph-mon[35941]: pgmap v6191: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:44:37.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:44:37 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 11335.821550 secs 2024-03-28T03:44:37.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:44:37 smithi165 ceph-mon[36302]: pgmap v6192: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:44:37.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:44:37 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 11335.821550 secs 2024-03-28T03:44:37.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:44:37 smithi119 ceph-mon[31518]: pgmap v6192: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:44:37.418 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:37.416+0000 7fc473400640 1 -- 172.21.15.119:0/210561562 >> [v2:172.21.15.119:6800/4273482239,v1:172.21.15.119:6801/4273482239] conn(0x7fc4580a2a90 msgr2=0x7fc4580a4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).tick idle (900002618) for more than 900000000 us, fault. 2024-03-28T03:44:37.418 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:37.416+0000 7fc473400640 1 --2- 172.21.15.119:0/210561562 >> [v2:172.21.15.119:6800/4273482239,v1:172.21.15.119:6801/4273482239] conn(0x7fc4580a2a90 0x7fc4580a4f50 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fc464005fd0 tx=0x7fc464009500 comp rx=0 tx=0).stop 2024-03-28T03:44:37.418 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:37.416+0000 7fc46be00640 1 -- 172.21.15.119:0/210561562 >> [v2:172.21.15.119:6800/4273482239,v1:172.21.15.119:6801/4273482239] conn(0x7fc4580a2a90 msgr2=0x7fc4580a4f50 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-03-28T03:44:37.419 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:37.416+0000 7fc46be00640 1 --2- 172.21.15.119:0/210561562 >> [v2:172.21.15.119:6800/4273482239,v1:172.21.15.119:6801/4273482239] conn(0x7fc4580a2a90 0x7fc4580a4f50 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:44:37.419 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:37.416+0000 7fc46be00640 1 --2- 172.21.15.119:0/210561562 >> [v2:172.21.15.119:6800/4273482239,v1:172.21.15.119:6801/4273482239] conn(0x7fc4580ae5e0 0x7fc4580b09d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T03:44:37.419 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:37.416+0000 7fc46be00640 1 -- 172.21.15.119:0/210561562 --> [v2:172.21.15.119:6800/4273482239,v1:172.21.15.119:6801/4273482239] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) v1 -- 0x7fc4580b0fb0 con 0x7fc4580ae5e0 2024-03-28T03:44:37.419 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:37.417+0000 7fc473400640 1 --2- 172.21.15.119:0/210561562 >> [v2:172.21.15.119:6800/4273482239,v1:172.21.15.119:6801/4273482239] conn(0x7fc4580ae5e0 0x7fc4580b09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T03:44:37.421 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:37.419+0000 7fc473400640 1 --2- 172.21.15.119:0/210561562 >> [v2:172.21.15.119:6800/4273482239,v1:172.21.15.119:6801/4273482239] conn(0x7fc4580ae5e0 0x7fc4580b09d0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fc464011010 tx=0x7fc464016000 comp rx=0 tx=0).ready entity=mgr.25277 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T03:44:37.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:44:37 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 11335.821550 secs 2024-03-28T03:44:37.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:44:37 smithi132 ceph-mon[35941]: pgmap v6192: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:44:37.549 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:37.548+0000 7fc46aa00640 1 -- 172.21.15.119:0/210561562 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc44c006360 con 0x7fc474102010 2024-03-28T03:44:37.550 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:37.548+0000 7fc46be00640 1 -- 172.21.15.119:0/210561562 <== mon.1 v2:172.21.15.165:3300/0 12 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (secure 0 0 0) 0x7fc44c006360 con 0x7fc474102010 2024-03-28T03:44:37.667 INFO:teuthology.orchestra.run.smithi119.stderr:Error ETIMEDOUT: Connection timed out. Please check the client's permission and connection. 2024-03-28T03:44:37.690 DEBUG:teuthology.orchestra.run:got remote process result: 110 2024-03-28T03:44:37.691 ERROR:teuthology.run_tasks:Manager failed: full_sequential_finally Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/task/full_sequential_finally.py", line 45, in task mgr = run_tasks.run_one_task(taskname, ctx=ctx, config=confg) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/task/exec.py", line 66, in task remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi119 with status 110: "sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/'" 2024-03-28T03:44:37.697 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-03-28T03:44:37.708 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T03:44:37.930 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:37.928+0000 7fc4f4e00640 1 -- 172.21.15.119:0/1114572310 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc4f010ded0 msgr2=0x7fc4f01102e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T03:44:37.931 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:37.928+0000 7fc4f4e00640 1 --2- 172.21.15.119:0/1114572310 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc4f010ded0 0x7fc4f01102e0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fc4e8009900 tx=0x7fc4e802bcc0 comp rx=0 tx=0).stop 2024-03-28T03:44:37.931 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:37.928+0000 7fc4f4e00640 1 -- 172.21.15.119:0/1114572310 shutdown_connections 2024-03-28T03:44:37.931 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:37.928+0000 7fc4f4e00640 1 --2- 172.21.15.119:0/1114572310 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc4f010ded0 0x7fc4f01102e0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:44:37.931 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:37.928+0000 7fc4f4e00640 1 --2- 172.21.15.119:0/1114572310 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc4f0074390 0x7fc4f010d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:44:37.931 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:37.928+0000 7fc4f4e00640 1 --2- 172.21.15.119:0/1114572310 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc4f0073a70 0x7fc4f0073e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:44:37.931 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:37.928+0000 7fc4f4e00640 1 -- 172.21.15.119:0/1114572310 >> 172.21.15.119:0/1114572310 conn(0x7fc4f00fd140 msgr2=0x7fc4f00ff560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T03:44:37.931 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:37.928+0000 7fc4f4e00640 1 -- 172.21.15.119:0/1114572310 shutdown_connections 2024-03-28T03:44:37.931 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:37.928+0000 7fc4f4e00640 1 -- 172.21.15.119:0/1114572310 wait complete. 2024-03-28T03:44:37.933 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:37.929+0000 7fc4f4e00640 1 Processor -- start 2024-03-28T03:44:37.934 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:37.929+0000 7fc4f4e00640 1 -- start start 2024-03-28T03:44:37.934 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:37.930+0000 7fc4f4e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc4f0073a70 0x7fc4f019a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T03:44:37.934 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:37.930+0000 7fc4f4e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc4f0074390 0x7fc4f019a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T03:44:37.934 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:37.930+0000 7fc4f4e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc4f010ded0 0x7fc4f019eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T03:44:37.934 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:37.930+0000 7fc4f4e00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4f0113c80 con 0x7fc4f0074390 2024-03-28T03:44:37.934 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:37.930+0000 7fc4f4e00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4f0113b00 con 0x7fc4f0073a70 2024-03-28T03:44:37.934 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:37.930+0000 7fc4f4e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4f0113e00 con 0x7fc4f010ded0 2024-03-28T03:44:37.935 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:37.930+0000 7fc4ede00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc4f0073a70 0x7fc4f019a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T03:44:37.935 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:37.930+0000 7fc4ede00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc4f0073a70 0x7fc4f019a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.119:38344/0 (socket says 172.21.15.119:38344) 2024-03-28T03:44:37.935 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:37.930+0000 7fc4ede00640 1 -- 172.21.15.119:0/205200466 learned_addr learned my addr 172.21.15.119:0/205200466 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T03:44:37.935 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:37.930+0000 7fc4ee800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc4f010ded0 0x7fc4f019eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T03:44:37.935 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:37.930+0000 7fc4e5400640 1 --2- 172.21.15.119:0/205200466 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc4f0074390 0x7fc4f019a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T03:44:37.935 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:37.930+0000 7fc4ede00640 1 -- 172.21.15.119:0/205200466 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc4f010ded0 msgr2=0x7fc4f019eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T03:44:37.935 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:37.930+0000 7fc4ede00640 1 --2- 172.21.15.119:0/205200466 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc4f010ded0 0x7fc4f019eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:44:37.935 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:37.930+0000 7fc4ede00640 1 -- 172.21.15.119:0/205200466 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc4f0074390 msgr2=0x7fc4f019a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T03:44:37.935 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:37.930+0000 7fc4ede00640 1 --2- 172.21.15.119:0/205200466 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc4f0074390 0x7fc4f019a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:44:37.936 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:37.931+0000 7fc4ede00640 1 -- 172.21.15.119:0/205200466 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4f019f220 con 0x7fc4f0073a70 2024-03-28T03:44:37.936 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:37.931+0000 7fc4ee800640 1 --2- 172.21.15.119:0/205200466 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc4f010ded0 0x7fc4f019eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T03:44:37.936 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:37.931+0000 7fc4ede00640 1 --2- 172.21.15.119:0/205200466 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc4f0073a70 0x7fc4f019a230 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc4e000ebc0 tx=0x7fc4e000c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T03:44:37.936 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:37.931+0000 7fc4e7400640 1 -- 172.21.15.119:0/205200466 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4e0010630 con 0x7fc4f0073a70 2024-03-28T03:44:37.936 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:37.931+0000 7fc4e7400640 1 -- 172.21.15.119:0/205200466 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc4e0010cb0 con 0x7fc4f0073a70 2024-03-28T03:44:37.936 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:37.931+0000 7fc4e7400640 1 -- 172.21.15.119:0/205200466 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4e0018bd0 con 0x7fc4f0073a70 2024-03-28T03:44:37.936 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:37.931+0000 7fc4f4e00640 1 -- 172.21.15.119:0/205200466 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4f019f510 con 0x7fc4f0073a70 2024-03-28T03:44:37.936 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:37.931+0000 7fc4f4e00640 1 -- 172.21.15.119:0/205200466 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4f01a6da0 con 0x7fc4f0073a70 2024-03-28T03:44:37.937 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:37.932+0000 7fc4f4e00640 1 -- 172.21.15.119:0/205200466 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4f00688e0 con 0x7fc4f0073a70 2024-03-28T03:44:37.937 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:37.934+0000 7fc4e7400640 1 -- 172.21.15.119:0/205200466 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 27) v1 ==== 137422+0+0 (secure 0 0 0) 0x7fc4e00107d0 con 0x7fc4f0073a70 2024-03-28T03:44:37.937 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:37.935+0000 7fc4e7400640 1 --2- 172.21.15.119:0/205200466 >> [v2:172.21.15.119:6800/4273482239,v1:172.21.15.119:6801/4273482239] conn(0x7fc4d40a2970 0x7fc4d40a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T03:44:37.937 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:37.935+0000 7fc4e7400640 1 -- 172.21.15.119:0/205200466 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 11163+0+0 (secure 0 0 0) 0x7fc4e00cff00 con 0x7fc4f0073a70 2024-03-28T03:44:37.938 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:37.936+0000 7fc4e5400640 1 --2- 172.21.15.119:0/205200466 >> [v2:172.21.15.119:6800/4273482239,v1:172.21.15.119:6801/4273482239] conn(0x7fc4d40a2970 0x7fc4d40a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T03:44:37.938 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:37.936+0000 7fc4e5400640 1 --2- 172.21.15.119:0/205200466 >> [v2:172.21.15.119:6800/4273482239,v1:172.21.15.119:6801/4273482239] conn(0x7fc4d40a2970 0x7fc4d40a4e30 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fc4f00ff880 tx=0x7fc4d8005f90 comp rx=0 tx=0).ready entity=mgr.25277 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T03:44:37.941 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:37.940+0000 7fc4e7400640 1 -- 172.21.15.119:0/205200466 <== mon.2 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fc4e009bb60 con 0x7fc4f0073a70 2024-03-28T03:44:38.142 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:38.140+0000 7fc4f4e00640 1 -- 172.21.15.119:0/205200466 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc4f0109080 con 0x7fc4f0073a70 2024-03-28T03:44:38.144 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:38.142+0000 7fc4e7400640 1 -- 172.21.15.119:0/205200466 <== mon.2 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v125) v1 ==== 74+0+27258 (secure 0 0 0) 0x7fc4e009fc00 con 0x7fc4f0073a70 2024-03-28T03:44:38.145 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-28T03:44:38.145 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":125,"fsid":"10f73b92-ec98-11ee-95d0-87774f69a715","created":"2024-03-28T00:14:57.563604+0000","modified":"2024-03-28T00:35:52.413057+0000","last_up_change":"2024-03-28T00:22:58.821662+0000","last_in_change":"2024-03-28T00:22:38.467270+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T00:18:46.253414+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-28T00:23:54.261981+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-28T00:23:55.685836+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":13,"snap_epoch":121,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c0b875d-a01e-4294-9c9c-5d0b048b8912","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":580396252},{"type":"v1","addr":"172.21.15.119:6803","nonce":580396252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":580396252},{"type":"v1","addr":"172.21.15.119:6805","nonce":580396252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":580396252},{"type":"v1","addr":"172.21.15.119:6809","nonce":580396252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":580396252},{"type":"v1","addr":"172.21.15.119:6807","nonce":580396252}]},"public_addr":"172.21.15.119:6803/580396252","cluster_addr":"172.21.15.119:6805/580396252","heartbeat_back_addr":"172.21.15.119:6809/580396252","heartbeat_front_addr":"172.21.15.119:6807/580396252","state":["exists","up"]},{"osd":1,"uuid":"5567f7c2-8ea4-4789-9ea1-467793324fb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":2500318888},{"type":"v1","addr":"172.21.15.132:6801","nonce":2500318888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":2500318888},{"type":"v1","addr":"172.21.15.132:6803","nonce":2500318888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":2500318888},{"type":"v1","addr":"172.21.15.132:6807","nonce":2500318888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":2500318888},{"type":"v1","addr":"172.21.15.132:6805","nonce":2500318888}]},"public_addr":"172.21.15.132:6801/2500318888","cluster_addr":"172.21.15.132:6803/2500318888","heartbeat_back_addr":"172.21.15.132:6807/2500318888","heartbeat_front_addr":"172.21.15.132:6805/2500318888","state":["exists","up"]},{"osd":2,"uuid":"ee54dbd0-24b4-4ee6-988c-ce71491199d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":2361434832},{"type":"v1","addr":"172.21.15.165:6801","nonce":2361434832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":2361434832},{"type":"v1","addr":"172.21.15.165:6803","nonce":2361434832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":2361434832},{"type":"v1","addr":"172.21.15.165:6807","nonce":2361434832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":2361434832},{"type":"v1","addr":"172.21.15.165:6805","nonce":2361434832}]},"public_addr":"172.21.15.165:6801/2361434832","cluster_addr":"172.21.15.165:6803/2361434832","heartbeat_back_addr":"172.21.15.165:6807/2361434832","heartbeat_front_addr":"172.21.15.165:6805/2361434832","state":["exists","up"]},{"osd":3,"uuid":"0464c40b-828f-4db4-96d9-3f7a6325e6da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":3732272210},{"type":"v1","addr":"172.21.15.119:6811","nonce":3732272210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":3732272210},{"type":"v1","addr":"172.21.15.119:6813","nonce":3732272210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":3732272210},{"type":"v1","addr":"172.21.15.119:6817","nonce":3732272210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":3732272210},{"type":"v1","addr":"172.21.15.119:6815","nonce":3732272210}]},"public_addr":"172.21.15.119:6811/3732272210","cluster_addr":"172.21.15.119:6813/3732272210","heartbeat_back_addr":"172.21.15.119:6817/3732272210","heartbeat_front_addr":"172.21.15.119:6815/3732272210","state":["exists","up"]},{"osd":4,"uuid":"8fd70b8e-5128-4173-aa0e-e735dfeb995d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":2803039505},{"type":"v1","addr":"172.21.15.132:6809","nonce":2803039505}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":2803039505},{"type":"v1","addr":"172.21.15.132:6811","nonce":2803039505}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":2803039505},{"type":"v1","addr":"172.21.15.132:6815","nonce":2803039505}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":2803039505},{"type":"v1","addr":"172.21.15.132:6813","nonce":2803039505}]},"public_addr":"172.21.15.132:6809/2803039505","cluster_addr":"172.21.15.132:6811/2803039505","heartbeat_back_addr":"172.21.15.132:6815/2803039505","heartbeat_front_addr":"172.21.15.132:6813/2803039505","state":["exists","up"]},{"osd":5,"uuid":"3224b4c1-733e-4424-a6da-870bbb5f94e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":2561252206},{"type":"v1","addr":"172.21.15.165:6809","nonce":2561252206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":2561252206},{"type":"v1","addr":"172.21.15.165:6811","nonce":2561252206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":2561252206},{"type":"v1","addr":"172.21.15.165:6815","nonce":2561252206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":2561252206},{"type":"v1","addr":"172.21.15.165:6813","nonce":2561252206}]},"public_addr":"172.21.15.165:6809/2561252206","cluster_addr":"172.21.15.165:6811/2561252206","heartbeat_back_addr":"172.21.15.165:6815/2561252206","heartbeat_front_addr":"172.21.15.165:6813/2561252206","state":["exists","up"]},{"osd":6,"uuid":"e435745d-59b5-439a-9091-0946ca0e9332","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":2545410230},{"type":"v1","addr":"172.21.15.119:6819","nonce":2545410230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":2545410230},{"type":"v1","addr":"172.21.15.119:6821","nonce":2545410230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":2545410230},{"type":"v1","addr":"172.21.15.119:6825","nonce":2545410230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":2545410230},{"type":"v1","addr":"172.21.15.119:6823","nonce":2545410230}]},"public_addr":"172.21.15.119:6819/2545410230","cluster_addr":"172.21.15.119:6821/2545410230","heartbeat_back_addr":"172.21.15.119:6825/2545410230","heartbeat_front_addr":"172.21.15.119:6823/2545410230","state":["exists","up"]},{"osd":7,"uuid":"79b97a12-f6b4-48c3-b517-2dd6b6e7c85f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":2877892376},{"type":"v1","addr":"172.21.15.132:6817","nonce":2877892376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":2877892376},{"type":"v1","addr":"172.21.15.132:6819","nonce":2877892376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":2877892376},{"type":"v1","addr":"172.21.15.132:6823","nonce":2877892376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":2877892376},{"type":"v1","addr":"172.21.15.132:6821","nonce":2877892376}]},"public_addr":"172.21.15.132:6817/2877892376","cluster_addr":"172.21.15.132:6819/2877892376","heartbeat_back_addr":"172.21.15.132:6823/2877892376","heartbeat_front_addr":"172.21.15.132:6821/2877892376","state":["exists","up"]},{"osd":8,"uuid":"2970524c-b99a-412d-85c2-6c29b54035e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":4115226150},{"type":"v1","addr":"172.21.15.165:6817","nonce":4115226150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":4115226150},{"type":"v1","addr":"172.21.15.165:6819","nonce":4115226150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":4115226150},{"type":"v1","addr":"172.21.15.165:6823","nonce":4115226150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":4115226150},{"type":"v1","addr":"172.21.15.165:6821","nonce":4115226150}]},"public_addr":"172.21.15.165:6817/4115226150","cluster_addr":"172.21.15.165:6819/4115226150","heartbeat_back_addr":"172.21.15.165:6823/4115226150","heartbeat_front_addr":"172.21.15.165:6821/4115226150","state":["exists","up"]},{"osd":9,"uuid":"2b1a08ed-cde9-4def-b3e8-838fff22709c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":2532245222},{"type":"v1","addr":"172.21.15.119:6827","nonce":2532245222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":2532245222},{"type":"v1","addr":"172.21.15.119:6829","nonce":2532245222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":2532245222},{"type":"v1","addr":"172.21.15.119:6833","nonce":2532245222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":2532245222},{"type":"v1","addr":"172.21.15.119:6831","nonce":2532245222}]},"public_addr":"172.21.15.119:6827/2532245222","cluster_addr":"172.21.15.119:6829/2532245222","heartbeat_back_addr":"172.21.15.119:6833/2532245222","heartbeat_front_addr":"172.21.15.119:6831/2532245222","state":["exists","up"]},{"osd":10,"uuid":"f019acd0-915a-4094-b12c-b53672740a5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":3454298492},{"type":"v1","addr":"172.21.15.132:6825","nonce":3454298492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":3454298492},{"type":"v1","addr":"172.21.15.132:6827","nonce":3454298492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":3454298492},{"type":"v1","addr":"172.21.15.132:6831","nonce":3454298492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":3454298492},{"type":"v1","addr":"172.21.15.132:6829","nonce":3454298492}]},"public_addr":"172.21.15.132:6825/3454298492","cluster_addr":"172.21.15.132:6827/3454298492","heartbeat_back_addr":"172.21.15.132:6831/3454298492","heartbeat_front_addr":"172.21.15.132:6829/3454298492","state":["exists","up"]},{"osd":11,"uuid":"2c1e59a8-8f21-45c0-8568-4b632ce020f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":2138771601},{"type":"v1","addr":"172.21.15.165:6825","nonce":2138771601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":2138771601},{"type":"v1","addr":"172.21.15.165:6827","nonce":2138771601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":2138771601},{"type":"v1","addr":"172.21.15.165:6831","nonce":2138771601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":2138771601},{"type":"v1","addr":"172.21.15.165:6829","nonce":2138771601}]},"public_addr":"172.21.15.165:6825/2138771601","cluster_addr":"172.21.15.165:6827/2138771601","heartbeat_back_addr":"172.21.15.165:6831/2138771601","heartbeat_front_addr":"172.21.15.165:6829/2138771601","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:17:54.791531+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:18:19.466638+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:18:43.435536+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:19:15.702685+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:19:41.767370+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:20:06.793003+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:20:39.952744+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:21:05.727262+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:21:30.735829+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:22:04.072142+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:22:30.894047+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:22:56.637728+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.119:6836/2109630829":"2024-03-29T00:35:52.413013+0000","172.21.15.165:6832/2013665520":"2024-03-29T00:32:07.607942+0000","172.21.15.165:6839/384483120":"2024-03-29T00:32:07.271819+0000","172.21.15.165:6838/384483120":"2024-03-29T00:32:07.271819+0000","172.21.15.119:6834/744950487":"2024-03-29T00:31:51.094723+0000","172.21.15.165:6839/768712672":"2024-03-29T00:33:20.841067+0000","172.21.15.119:0/87565965":"2024-03-29T00:16:03.582545+0000","172.21.15.119:6834/2793569574":"2024-03-29T00:35:34.679932+0000","172.21.15.165:6839/3004684592":"2024-03-29T00:31:32.429788+0000","172.21.15.165:6833/1389081045":"2024-03-29T00:31:20.067892+0000","172.21.15.165:6832/609963168":"2024-03-29T00:30:41.299818+0000","172.21.15.119:6834/2941089726":"2024-03-29T00:33:37.236449+0000","172.21.15.165:6839/2762859081":"2024-03-29T00:33:59.786505+0000","172.21.15.165:6833/609963168":"2024-03-29T00:30:41.299818+0000","172.21.15.165:6837/2717860763":"2024-03-29T00:33:02.702892+0000","172.21.15.165:6836/3170852275":"2024-03-29T00:32:25.939039+0000","172.21.15.119:6800/2056426384":"2024-03-29T00:15:30.472283+0000","172.21.15.119:6801/2575246519":"2024-03-29T00:16:03.582545+0000","172.21.15.165:6836/1531193032":"2024-03-29T00:30:24.146598+0000","172.21.15.119:6834/1659925849":"2024-03-29T00:34:17.464068+0000","172.21.15.119:6835/744950487":"2024-03-29T00:31:51.094723+0000","172.21.15.165:6836/1456402510":"2024-03-29T00:30:56.719761+0000","172.21.15.165:6837/1456402510":"2024-03-29T00:30:56.719761+0000","172.21.15.165:6839/1687390659":"2024-03-29T00:32:46.777861+0000","172.21.15.119:0/2635982178":"2024-03-29T00:16:03.582545+0000","172.21.15.119:0/2430401731":"2024-03-29T00:26:32.965486+0000","172.21.15.119:6837/482178170":"2024-03-29T00:35:17.974473+0000","172.21.15.119:6801/1401937096":"2024-03-29T00:26:32.965486+0000","172.21.15.119:0/4201699272":"2024-03-29T00:16:03.582545+0000","172.21.15.165:6838/3004684592":"2024-03-29T00:31:32.429788+0000","172.21.15.165:6836/1011925619":"2024-03-29T00:31:49.271822+0000","172.21.15.165:6837/1531193032":"2024-03-29T00:30:24.146598+0000","172.21.15.165:6833/2013665520":"2024-03-29T00:32:07.607942+0000","172.21.15.119:6834/2101147835":"2024-03-29T00:32:28.920424+0000","172.21.15.119:6835/3517971302":"2024-03-29T00:35:17.389381+0000","172.21.15.165:6832/1389081045":"2024-03-29T00:31:20.067892+0000","172.21.15.119:6835/2210305461":"2024-03-29T00:33:02.940231+0000","172.21.15.119:0/2891240357":"2024-03-29T00:15:30.472283+0000","172.21.15.119:6838/1595343240":"2024-03-29T00:34:16.710959+0000","172.21.15.119:6801/2056426384":"2024-03-29T00:15:30.472283+0000","172.21.15.119:0/357445067":"2024-03-29T00:26:32.965486+0000","172.21.15.165:6838/1687390659":"2024-03-29T00:32:46.777861+0000","172.21.15.119:6838/3947365264":"2024-03-29T00:34:59.827426+0000","172.21.15.165:6837/1011925619":"2024-03-29T00:31:49.271822+0000","172.21.15.119:6800/2575246519":"2024-03-29T00:16:03.582545+0000","172.21.15.165:6836/823338130":"2024-03-29T00:33:39.774934+0000","172.21.15.119:0/468478300":"2024-03-29T00:15:30.472283+0000","172.21.15.119:0/4119730435":"2024-03-29T00:26:32.965486+0000","172.21.15.119:6800/1401937096":"2024-03-29T00:26:32.965486+0000","172.21.15.119:0/524303043":"2024-03-29T00:26:32.965486+0000","172.21.15.119:6835/2101147835":"2024-03-29T00:32:28.920424+0000","172.21.15.165:6832/4070507195":"2024-03-29T00:32:46.507920+0000","172.21.15.119:6836/3988635716":"2024-03-29T00:34:37.192476+0000","172.21.15.165:6836/2717860763":"2024-03-29T00:33:02.702892+0000","172.21.15.119:6834/2210305461":"2024-03-29T00:33:02.940231+0000","172.21.15.165:6832/3417761142":"2024-03-29T00:33:20.267933+0000","172.21.15.165:6838/768712672":"2024-03-29T00:33:20.841067+0000","172.21.15.119:6835/2941089726":"2024-03-29T00:33:37.236449+0000","172.21.15.165:6837/3170852275":"2024-03-29T00:32:25.939039+0000","172.21.15.119:6839/1595343240":"2024-03-29T00:34:16.710959+0000","172.21.15.165:6833/4070507195":"2024-03-29T00:32:46.507920+0000","172.21.15.165:6837/823338130":"2024-03-29T00:33:39.774934+0000","172.21.15.119:6834/4197433095":"2024-03-29T00:34:37.186007+0000","172.21.15.165:6832/1344198130":"2024-03-29T00:33:59.786346+0000","172.21.15.119:6837/3511403986":"2024-03-29T00:35:35.303954+0000","172.21.15.165:6838/2762859081":"2024-03-29T00:33:59.786505+0000","172.21.15.119:6835/4197433095":"2024-03-29T00:34:37.186007+0000","172.21.15.119:6836/482178170":"2024-03-29T00:35:17.974473+0000","172.21.15.165:6833/3417761142":"2024-03-29T00:33:20.267933+0000","172.21.15.165:6833/1344198130":"2024-03-29T00:33:59.786346+0000","172.21.15.119:6835/1659925849":"2024-03-29T00:34:17.464068+0000","172.21.15.119:6837/3988635716":"2024-03-29T00:34:37.192476+0000","172.21.15.119:6836/3511403986":"2024-03-29T00:35:35.303954+0000","172.21.15.119:6834/3816731508":"2024-03-29T00:34:54.192187+0000","172.21.15.119:6837/2109630829":"2024-03-29T00:35:52.413013+0000","172.21.15.119:6835/3816731508":"2024-03-29T00:34:54.192187+0000","172.21.15.119:6839/3947365264":"2024-03-29T00:34:59.827426+0000","172.21.15.119:0/1639109126":"2024-03-29T00:15:30.472283+0000","172.21.15.119:6834/3517971302":"2024-03-29T00:35:17.389381+0000","172.21.15.119:6835/2793569574":"2024-03-29T00:35:34.679932+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T03:44:38.148 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:38.146+0000 7fc4f4e00640 1 -- 172.21.15.119:0/205200466 >> [v2:172.21.15.119:6800/4273482239,v1:172.21.15.119:6801/4273482239] conn(0x7fc4d40a2970 msgr2=0x7fc4d40a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T03:44:38.148 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:38.146+0000 7fc4f4e00640 1 --2- 172.21.15.119:0/205200466 >> [v2:172.21.15.119:6800/4273482239,v1:172.21.15.119:6801/4273482239] conn(0x7fc4d40a2970 0x7fc4d40a4e30 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fc4f00ff880 tx=0x7fc4d8005f90 comp rx=0 tx=0).stop 2024-03-28T03:44:38.149 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:38.146+0000 7fc4f4e00640 1 -- 172.21.15.119:0/205200466 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc4f0073a70 msgr2=0x7fc4f019a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T03:44:38.149 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:38.146+0000 7fc4f4e00640 1 --2- 172.21.15.119:0/205200466 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc4f0073a70 0x7fc4f019a230 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc4e000ebc0 tx=0x7fc4e000c6a0 comp rx=0 tx=0).stop 2024-03-28T03:44:38.149 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:38.146+0000 7fc4f4e00640 1 -- 172.21.15.119:0/205200466 shutdown_connections 2024-03-28T03:44:38.149 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:38.146+0000 7fc4f4e00640 1 --2- 172.21.15.119:0/205200466 >> [v2:172.21.15.119:6800/4273482239,v1:172.21.15.119:6801/4273482239] conn(0x7fc4d40a2970 0x7fc4d40a4e30 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:44:38.149 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:38.146+0000 7fc4f4e00640 1 --2- 172.21.15.119:0/205200466 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc4f010ded0 0x7fc4f019eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:44:38.150 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:38.146+0000 7fc4f4e00640 1 --2- 172.21.15.119:0/205200466 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc4f0074390 0x7fc4f019a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:44:38.150 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:38.146+0000 7fc4f4e00640 1 --2- 172.21.15.119:0/205200466 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc4f0073a70 0x7fc4f019a230 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:44:38.150 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:38.146+0000 7fc4f4e00640 1 -- 172.21.15.119:0/205200466 >> 172.21.15.119:0/205200466 conn(0x7fc4f00fd140 msgr2=0x7fc4f00fe9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T03:44:38.150 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:38.146+0000 7fc4f4e00640 1 -- 172.21.15.119:0/205200466 shutdown_connections 2024-03-28T03:44:38.150 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:38.146+0000 7fc4f4e00640 1 -- 172.21.15.119:0/205200466 wait complete. 2024-03-28T03:44:38.166 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-28T00:18:46.253414+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-28T00:23:54.261981+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-28T00:23:55.685836+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '121', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 13, 'snap_epoch': 121, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-28T03:44:38.167 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-28T03:44:38.376 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:38.374+0000 7f3907200640 1 -- 172.21.15.119:0/3289316807 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3900103720 msgr2=0x7f3900103ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T03:44:38.376 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:38.374+0000 7f3907200640 1 --2- 172.21.15.119:0/3289316807 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3900103720 0x7f3900103ba0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f38f0009930 tx=0x7f38f002bc30 comp rx=0 tx=0).stop 2024-03-28T03:44:38.376 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:38.374+0000 7f3907200640 1 -- 172.21.15.119:0/3289316807 shutdown_connections 2024-03-28T03:44:38.376 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:38.374+0000 7f3907200640 1 --2- 172.21.15.119:0/3289316807 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f39001040e0 0x7f39001105f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:44:38.376 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:38.374+0000 7f3907200640 1 --2- 172.21.15.119:0/3289316807 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3900103720 0x7f3900103ba0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:44:38.376 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:38.374+0000 7f3907200640 1 --2- 172.21.15.119:0/3289316807 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3900109730 0x7f3900109b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:44:38.377 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:38.374+0000 7f3907200640 1 -- 172.21.15.119:0/3289316807 >> 172.21.15.119:0/3289316807 conn(0x7f39000ff1b0 msgr2=0x7f39001015d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T03:44:38.377 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:38.374+0000 7f3907200640 1 -- 172.21.15.119:0/3289316807 shutdown_connections 2024-03-28T03:44:38.377 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:38.374+0000 7f3907200640 1 -- 172.21.15.119:0/3289316807 wait complete. 2024-03-28T03:44:38.379 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:38.375+0000 7f3907200640 1 Processor -- start 2024-03-28T03:44:38.379 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:38.375+0000 7f3907200640 1 -- start start 2024-03-28T03:44:38.379 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:38.376+0000 7f3907200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3900103720 0x7f390019a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T03:44:38.379 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:38.376+0000 7f3907200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f39001040e0 0x7f390019a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T03:44:38.379 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:38.376+0000 7f3907200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3900109730 0x7f390019ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T03:44:38.380 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:38.376+0000 7f3907200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3900113b50 con 0x7f3900103720 2024-03-28T03:44:38.380 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:38.376+0000 7f3907200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39001139d0 con 0x7f3900109730 2024-03-28T03:44:38.380 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:38.376+0000 7f3907200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3900113cd0 con 0x7f39001040e0 2024-03-28T03:44:38.380 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:38.376+0000 7f3905e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3900103720 0x7f390019a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T03:44:38.380 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:38.376+0000 7f3906800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3900109730 0x7f390019ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T03:44:38.380 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:38.376+0000 7f3905400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f39001040e0 0x7f390019a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T03:44:38.380 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:38.376+0000 7f3905e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3900103720 0x7f390019a390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46406/0 (socket says 172.21.15.119:46406) 2024-03-28T03:44:38.380 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:38.376+0000 7f3905e00640 1 -- 172.21.15.119:0/3672047464 learned_addr learned my addr 172.21.15.119:0/3672047464 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T03:44:38.381 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:38.376+0000 7f3905400640 1 -- 172.21.15.119:0/3672047464 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3900109730 msgr2=0x7f390019ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T03:44:38.381 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:38.376+0000 7f3905400640 1 --2- 172.21.15.119:0/3672047464 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3900109730 0x7f390019ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:44:38.381 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:38.376+0000 7f3905400640 1 -- 172.21.15.119:0/3672047464 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3900103720 msgr2=0x7f390019a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T03:44:38.381 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:38.376+0000 7f3905400640 1 --2- 172.21.15.119:0/3672047464 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3900103720 0x7f390019a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:44:38.381 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:38.376+0000 7f3905400640 1 -- 172.21.15.119:0/3672047464 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f390019f380 con 0x7f39001040e0 2024-03-28T03:44:38.381 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:38.376+0000 7f3905e00640 1 --2- 172.21.15.119:0/3672047464 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3900103720 0x7f390019a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T03:44:38.381 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:38.376+0000 7f3906800640 1 --2- 172.21.15.119:0/3672047464 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3900109730 0x7f390019ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T03:44:38.381 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:38.377+0000 7f3905400640 1 --2- 172.21.15.119:0/3672047464 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f39001040e0 0x7f390019a8d0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f38f0036040 tx=0x7f38f0004360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T03:44:38.381 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:38.377+0000 7f38faa00640 1 -- 172.21.15.119:0/3672047464 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38f0043020 con 0x7f39001040e0 2024-03-28T03:44:38.382 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:38.377+0000 7f38faa00640 1 -- 172.21.15.119:0/3672047464 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f38f00054e0 con 0x7f39001040e0 2024-03-28T03:44:38.382 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:38.377+0000 7f38faa00640 1 -- 172.21.15.119:0/3672047464 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38f0003b80 con 0x7f39001040e0 2024-03-28T03:44:38.382 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:38.377+0000 7f3907200640 1 -- 172.21.15.119:0/3672047464 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f390019f610 con 0x7f39001040e0 2024-03-28T03:44:38.382 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:38.377+0000 7f3907200640 1 -- 172.21.15.119:0/3672047464 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39001a6f00 con 0x7f39001040e0 2024-03-28T03:44:38.383 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:38.378+0000 7f3907200640 1 -- 172.21.15.119:0/3672047464 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3900104e60 con 0x7f39001040e0 2024-03-28T03:44:38.383 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:38.380+0000 7f38faa00640 1 -- 172.21.15.119:0/3672047464 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 27) v1 ==== 137422+0+0 (secure 0 0 0) 0x7f38f0046020 con 0x7f39001040e0 2024-03-28T03:44:38.383 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:38.381+0000 7f38faa00640 1 --2- 172.21.15.119:0/3672047464 >> [v2:172.21.15.119:6800/4273482239,v1:172.21.15.119:6801/4273482239] conn(0x7f38e80a29f0 0x7f38e80a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T03:44:38.383 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:38.381+0000 7f38faa00640 1 -- 172.21.15.119:0/3672047464 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 11163+0+0 (secure 0 0 0) 0x7f38f00f1710 con 0x7f39001040e0 2024-03-28T03:44:38.384 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:38.382+0000 7f3905e00640 1 --2- 172.21.15.119:0/3672047464 >> [v2:172.21.15.119:6800/4273482239,v1:172.21.15.119:6801/4273482239] conn(0x7f38e80a29f0 0x7f38e80a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T03:44:38.384 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:38.382+0000 7f3905e00640 1 --2- 172.21.15.119:0/3672047464 >> [v2:172.21.15.119:6800/4273482239,v1:172.21.15.119:6801/4273482239] conn(0x7f38e80a29f0 0x7f38e80a4eb0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f38f4011440 tx=0x7f38f4005f90 comp rx=0 tx=0).ready entity=mgr.25277 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T03:44:38.387 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:38.386+0000 7f38faa00640 1 -- 172.21.15.119:0/3672047464 <== mon.1 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f38f004d020 con 0x7f39001040e0 2024-03-28T03:44:38.571 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:38.570+0000 7f3907200640 1 -- 172.21.15.119:0/3672047464 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f390010bcb0 con 0x7f39001040e0 2024-03-28T03:44:38.572 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:38.570+0000 7f38faa00640 1 -- 172.21.15.119:0/3672047464 <== mon.1 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v125) v1 ==== 93+0+10 (secure 0 0 0) 0x7f38f00bdad0 con 0x7f39001040e0 2024-03-28T03:44:38.572 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 1 2024-03-28T03:44:38.575 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:38.573+0000 7f3907200640 1 -- 172.21.15.119:0/3672047464 >> [v2:172.21.15.119:6800/4273482239,v1:172.21.15.119:6801/4273482239] conn(0x7f38e80a29f0 msgr2=0x7f38e80a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T03:44:38.575 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:38.573+0000 7f3907200640 1 --2- 172.21.15.119:0/3672047464 >> [v2:172.21.15.119:6800/4273482239,v1:172.21.15.119:6801/4273482239] conn(0x7f38e80a29f0 0x7f38e80a4eb0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f38f4011440 tx=0x7f38f4005f90 comp rx=0 tx=0).stop 2024-03-28T03:44:38.575 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:38.573+0000 7f3907200640 1 -- 172.21.15.119:0/3672047464 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f39001040e0 msgr2=0x7f390019a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T03:44:38.575 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:38.573+0000 7f3907200640 1 --2- 172.21.15.119:0/3672047464 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f39001040e0 0x7f390019a8d0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f38f0036040 tx=0x7f38f0004360 comp rx=0 tx=0).stop 2024-03-28T03:44:38.575 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:38.573+0000 7f3907200640 1 -- 172.21.15.119:0/3672047464 shutdown_connections 2024-03-28T03:44:38.576 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:38.573+0000 7f3907200640 1 --2- 172.21.15.119:0/3672047464 >> [v2:172.21.15.119:6800/4273482239,v1:172.21.15.119:6801/4273482239] conn(0x7f38e80a29f0 0x7f38e80a4eb0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:44:38.576 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:38.573+0000 7f3907200640 1 --2- 172.21.15.119:0/3672047464 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3900109730 0x7f390019ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:44:38.576 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:38.573+0000 7f3907200640 1 --2- 172.21.15.119:0/3672047464 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f39001040e0 0x7f390019a8d0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:44:38.576 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:38.573+0000 7f3907200640 1 --2- 172.21.15.119:0/3672047464 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3900103720 0x7f390019a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:44:38.576 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:38.573+0000 7f3907200640 1 -- 172.21.15.119:0/3672047464 >> 172.21.15.119:0/3672047464 conn(0x7f39000ff1b0 msgr2=0x7f39000ffb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T03:44:38.576 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:38.573+0000 7f3907200640 1 -- 172.21.15.119:0/3672047464 shutdown_connections 2024-03-28T03:44:38.576 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:38.573+0000 7f3907200640 1 -- 172.21.15.119:0/3672047464 wait complete. 2024-03-28T03:44:38.594 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-28T03:44:38.793 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:38.791+0000 7f95d4a00640 1 -- 172.21.15.119:0/997549424 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f95d0103840 msgr2=0x7f95d0103cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T03:44:38.793 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:38.791+0000 7f95d4a00640 1 --2- 172.21.15.119:0/997549424 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f95d0103840 0x7f95d0103cc0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f95b8009900 tx=0x7f95b802bc30 comp rx=0 tx=0).stop 2024-03-28T03:44:38.793 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:38.791+0000 7f95d4a00640 1 -- 172.21.15.119:0/997549424 shutdown_connections 2024-03-28T03:44:38.794 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:38.791+0000 7f95d4a00640 1 --2- 172.21.15.119:0/997549424 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f95d0104200 0x7f95d0110710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:44:38.794 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:38.791+0000 7f95d4a00640 1 --2- 172.21.15.119:0/997549424 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f95d0103840 0x7f95d0103cc0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:44:38.794 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:38.791+0000 7f95d4a00640 1 --2- 172.21.15.119:0/997549424 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f95d0109850 0x7f95d0109c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:44:38.794 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:38.791+0000 7f95d4a00640 1 -- 172.21.15.119:0/997549424 >> 172.21.15.119:0/997549424 conn(0x7f95d00ff350 msgr2=0x7f95d0101770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T03:44:38.794 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:38.791+0000 7f95d4a00640 1 -- 172.21.15.119:0/997549424 shutdown_connections 2024-03-28T03:44:38.794 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:38.791+0000 7f95d4a00640 1 -- 172.21.15.119:0/997549424 wait complete. 2024-03-28T03:44:38.796 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:38.792+0000 7f95d4a00640 1 Processor -- start 2024-03-28T03:44:38.796 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:38.792+0000 7f95d4a00640 1 -- start start 2024-03-28T03:44:38.796 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:38.793+0000 7f95d4a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f95d0103840 0x7f95d019a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T03:44:38.796 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:38.793+0000 7f95d4a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f95d0104200 0x7f95d019a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T03:44:38.797 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:38.793+0000 7f95d4a00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f95d0109850 0x7f95d019ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T03:44:38.797 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:38.793+0000 7f95d4a00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95d0113c70 con 0x7f95d0104200 2024-03-28T03:44:38.797 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:38.793+0000 7f95d4a00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95d0113af0 con 0x7f95d0109850 2024-03-28T03:44:38.797 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:38.793+0000 7f95d4a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95d0113df0 con 0x7f95d0103840 2024-03-28T03:44:38.797 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:38.793+0000 7f95cde00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f95d0103840 0x7f95d019a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T03:44:38.797 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:38.793+0000 7f95cd400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f95d0104200 0x7f95d019a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T03:44:38.797 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:38.793+0000 7f95cde00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f95d0103840 0x7f95d019a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.119:47088/0 (socket says 172.21.15.119:47088) 2024-03-28T03:44:38.797 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:38.793+0000 7f95ce800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f95d0109850 0x7f95d019ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T03:44:38.798 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:38.793+0000 7f95cde00640 1 -- 172.21.15.119:0/844729617 learned_addr learned my addr 172.21.15.119:0/844729617 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T03:44:38.798 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:38.793+0000 7f95ce800640 1 -- 172.21.15.119:0/844729617 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f95d0103840 msgr2=0x7f95d019a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T03:44:38.798 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:38.793+0000 7f95ce800640 1 --2- 172.21.15.119:0/844729617 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f95d0103840 0x7f95d019a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:44:38.798 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:38.793+0000 7f95ce800640 1 -- 172.21.15.119:0/844729617 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f95d0104200 msgr2=0x7f95d019a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T03:44:38.798 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:38.793+0000 7f95ce800640 1 --2- 172.21.15.119:0/844729617 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f95d0104200 0x7f95d019a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:44:38.798 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:38.793+0000 7f95ce800640 1 -- 172.21.15.119:0/844729617 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95d019f160 con 0x7f95d0109850 2024-03-28T03:44:38.798 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:38.793+0000 7f95cde00640 1 --2- 172.21.15.119:0/844729617 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f95d0103840 0x7f95d019a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T03:44:38.798 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:38.793+0000 7f95cd400640 1 --2- 172.21.15.119:0/844729617 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f95d0104200 0x7f95d019a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T03:44:38.798 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:38.794+0000 7f95ce800640 1 --2- 172.21.15.119:0/844729617 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f95d0109850 0x7f95d019ea30 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f95c000e970 tx=0x7f95c000ee30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T03:44:38.799 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:38.794+0000 7f95c6a00640 1 -- 172.21.15.119:0/844729617 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95c000ccc0 con 0x7f95d0109850 2024-03-28T03:44:38.799 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:38.794+0000 7f95c6a00640 1 -- 172.21.15.119:0/844729617 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f95c0005020 con 0x7f95d0109850 2024-03-28T03:44:38.799 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:38.794+0000 7f95d4a00640 1 -- 172.21.15.119:0/844729617 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95d019f450 con 0x7f95d0109850 2024-03-28T03:44:38.799 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:38.794+0000 7f95c6a00640 1 -- 172.21.15.119:0/844729617 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95c0004a00 con 0x7f95d0109850 2024-03-28T03:44:38.799 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:38.794+0000 7f95d4a00640 1 -- 172.21.15.119:0/844729617 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95d01a6ce0 con 0x7f95d0109850 2024-03-28T03:44:38.800 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:38.795+0000 7f95d4a00640 1 -- 172.21.15.119:0/844729617 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95d00688e0 con 0x7f95d0109850 2024-03-28T03:44:38.800 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:38.798+0000 7f95c6a00640 1 -- 172.21.15.119:0/844729617 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 27) v1 ==== 137422+0+0 (secure 0 0 0) 0x7f95c00051d0 con 0x7f95d0109850 2024-03-28T03:44:38.801 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:38.799+0000 7f95c6a00640 1 --2- 172.21.15.119:0/844729617 >> [v2:172.21.15.119:6800/4273482239,v1:172.21.15.119:6801/4273482239] conn(0x7f95b40a29f0 0x7f95b40a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T03:44:38.801 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:38.799+0000 7f95c6a00640 1 -- 172.21.15.119:0/844729617 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 11163+0+0 (secure 0 0 0) 0x7f95c00cf790 con 0x7f95d0109850 2024-03-28T03:44:38.801 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:38.799+0000 7f95cde00640 1 --2- 172.21.15.119:0/844729617 >> [v2:172.21.15.119:6800/4273482239,v1:172.21.15.119:6801/4273482239] conn(0x7f95b40a29f0 0x7f95b40a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T03:44:38.801 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:38.800+0000 7f95cde00640 1 --2- 172.21.15.119:0/844729617 >> [v2:172.21.15.119:6800/4273482239,v1:172.21.15.119:6801/4273482239] conn(0x7f95b40a29f0 0x7f95b40a4eb0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f95c80117b0 tx=0x7f95c8009500 comp rx=0 tx=0).ready entity=mgr.25277 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T03:44:38.809 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:38.808+0000 7f95c6a00640 1 -- 172.21.15.119:0/844729617 <== mon.2 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f95c009baa0 con 0x7f95d0109850 2024-03-28T03:44:39.006 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:39.004+0000 7f95d4a00640 1 -- 172.21.15.119:0/844729617 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f95d002d070 con 0x7f95d0109850 2024-03-28T03:44:39.006 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:39.004+0000 7f95c6a00640 1 -- 172.21.15.119:0/844729617 <== mon.2 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v125) v1 ==== 104+0+11 (secure 0 0 0) 0x7f95c009b3a0 con 0x7f95d0109850 2024-03-28T03:44:39.006 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 32 2024-03-28T03:44:39.009 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:39.007+0000 7f95d4a00640 1 -- 172.21.15.119:0/844729617 >> [v2:172.21.15.119:6800/4273482239,v1:172.21.15.119:6801/4273482239] conn(0x7f95b40a29f0 msgr2=0x7f95b40a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T03:44:39.009 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:39.007+0000 7f95d4a00640 1 --2- 172.21.15.119:0/844729617 >> [v2:172.21.15.119:6800/4273482239,v1:172.21.15.119:6801/4273482239] conn(0x7f95b40a29f0 0x7f95b40a4eb0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f95c80117b0 tx=0x7f95c8009500 comp rx=0 tx=0).stop 2024-03-28T03:44:39.009 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:39.007+0000 7f95d4a00640 1 -- 172.21.15.119:0/844729617 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f95d0109850 msgr2=0x7f95d019ea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T03:44:39.009 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:39.007+0000 7f95d4a00640 1 --2- 172.21.15.119:0/844729617 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f95d0109850 0x7f95d019ea30 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f95c000e970 tx=0x7f95c000ee30 comp rx=0 tx=0).stop 2024-03-28T03:44:39.009 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:39.007+0000 7f95d4a00640 1 -- 172.21.15.119:0/844729617 shutdown_connections 2024-03-28T03:44:39.009 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:39.007+0000 7f95d4a00640 1 --2- 172.21.15.119:0/844729617 >> [v2:172.21.15.119:6800/4273482239,v1:172.21.15.119:6801/4273482239] conn(0x7f95b40a29f0 0x7f95b40a4eb0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:44:39.009 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:39.007+0000 7f95d4a00640 1 --2- 172.21.15.119:0/844729617 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f95d0109850 0x7f95d019ea30 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:44:39.009 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:39.007+0000 7f95d4a00640 1 --2- 172.21.15.119:0/844729617 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f95d0104200 0x7f95d019a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:44:39.010 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:39.007+0000 7f95d4a00640 1 --2- 172.21.15.119:0/844729617 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f95d0103840 0x7f95d019a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:44:39.010 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:39.007+0000 7f95d4a00640 1 -- 172.21.15.119:0/844729617 >> 172.21.15.119:0/844729617 conn(0x7f95d00ff350 msgr2=0x7f95d00ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T03:44:39.010 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:39.007+0000 7f95d4a00640 1 -- 172.21.15.119:0/844729617 shutdown_connections 2024-03-28T03:44:39.010 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:39.007+0000 7f95d4a00640 1 -- 172.21.15.119:0/844729617 wait complete. 2024-03-28T03:44:39.028 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-28T03:44:39.234 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:39.232+0000 7f2f90a00640 1 -- 172.21.15.119:0/355924346 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2f8c104200 msgr2=0x7f2f8c110710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T03:44:39.234 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:39.232+0000 7f2f90a00640 1 --2- 172.21.15.119:0/355924346 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2f8c104200 0x7f2f8c110710 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f2f84009900 tx=0x7f2f8402bc30 comp rx=0 tx=0).stop 2024-03-28T03:44:39.235 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:39.232+0000 7f2f90a00640 1 -- 172.21.15.119:0/355924346 shutdown_connections 2024-03-28T03:44:39.235 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:39.232+0000 7f2f90a00640 1 --2- 172.21.15.119:0/355924346 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2f8c104200 0x7f2f8c110710 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:44:39.235 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:39.232+0000 7f2f90a00640 1 --2- 172.21.15.119:0/355924346 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2f8c103840 0x7f2f8c103cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:44:39.235 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:39.232+0000 7f2f90a00640 1 --2- 172.21.15.119:0/355924346 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2f8c109850 0x7f2f8c109c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:44:39.235 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:39.232+0000 7f2f90a00640 1 -- 172.21.15.119:0/355924346 >> 172.21.15.119:0/355924346 conn(0x7f2f8c0ff310 msgr2=0x7f2f8c101730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T03:44:39.235 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:39.232+0000 7f2f90a00640 1 -- 172.21.15.119:0/355924346 shutdown_connections 2024-03-28T03:44:39.235 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:39.232+0000 7f2f90a00640 1 -- 172.21.15.119:0/355924346 wait complete. 2024-03-28T03:44:39.237 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:39.233+0000 7f2f90a00640 1 Processor -- start 2024-03-28T03:44:39.237 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:39.233+0000 7f2f90a00640 1 -- start start 2024-03-28T03:44:39.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:39.234+0000 7f2f90a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2f8c103840 0x7f2f8c19a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T03:44:39.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:39.234+0000 7f2f90a00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2f8c104200 0x7f2f8c19a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T03:44:39.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:39.234+0000 7f2f90a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2f8c109850 0x7f2f8c19e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T03:44:39.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:39.234+0000 7f2f90a00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f8c113cb0 con 0x7f2f8c109850 2024-03-28T03:44:39.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:39.234+0000 7f2f90a00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f8c113b30 con 0x7f2f8c104200 2024-03-28T03:44:39.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:39.234+0000 7f2f90a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f8c113e30 con 0x7f2f8c103840 2024-03-28T03:44:39.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:39.234+0000 7f2f89e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2f8c103840 0x7f2f8c19a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T03:44:39.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:39.234+0000 7f2f89e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2f8c103840 0x7f2f8c19a0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.119:47100/0 (socket says 172.21.15.119:47100) 2024-03-28T03:44:39.239 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:39.234+0000 7f2f89e00640 1 -- 172.21.15.119:0/3345590493 learned_addr learned my addr 172.21.15.119:0/3345590493 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T03:44:39.239 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:39.234+0000 7f2f81400640 1 --2- 172.21.15.119:0/3345590493 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2f8c104200 0x7f2f8c19a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T03:44:39.239 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:39.234+0000 7f2f8a800640 1 --2- 172.21.15.119:0/3345590493 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2f8c109850 0x7f2f8c19e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T03:44:39.239 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:39.234+0000 7f2f89e00640 1 -- 172.21.15.119:0/3345590493 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2f8c104200 msgr2=0x7f2f8c19a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T03:44:39.239 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:39.234+0000 7f2f89e00640 1 --2- 172.21.15.119:0/3345590493 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2f8c104200 0x7f2f8c19a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:44:39.239 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:39.234+0000 7f2f89e00640 1 -- 172.21.15.119:0/3345590493 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2f8c109850 msgr2=0x7f2f8c19e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T03:44:39.239 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:39.235+0000 7f2f89e00640 1 --2- 172.21.15.119:0/3345590493 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2f8c109850 0x7f2f8c19e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:44:39.239 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:39.235+0000 7f2f89e00640 1 -- 172.21.15.119:0/3345590493 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f8c19f0c0 con 0x7f2f8c103840 2024-03-28T03:44:39.239 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:39.235+0000 7f2f89e00640 1 --2- 172.21.15.119:0/3345590493 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2f8c103840 0x7f2f8c19a0d0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f2f7c00eb90 tx=0x7f2f7c00c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T03:44:39.240 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:39.235+0000 7f2f83400640 1 -- 172.21.15.119:0/3345590493 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f7c004d20 con 0x7f2f8c103840 2024-03-28T03:44:39.240 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:39.235+0000 7f2f83400640 1 -- 172.21.15.119:0/3345590493 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2f7c00ee90 con 0x7f2f8c103840 2024-03-28T03:44:39.240 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:39.235+0000 7f2f90a00640 1 -- 172.21.15.119:0/3345590493 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f8c19f3b0 con 0x7f2f8c103840 2024-03-28T03:44:39.240 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:39.235+0000 7f2f83400640 1 -- 172.21.15.119:0/3345590493 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f7c010830 con 0x7f2f8c103840 2024-03-28T03:44:39.240 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:39.235+0000 7f2f90a00640 1 -- 172.21.15.119:0/3345590493 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f8c1a6c40 con 0x7f2f8c103840 2024-03-28T03:44:39.241 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:39.236+0000 7f2f90a00640 1 -- 172.21.15.119:0/3345590493 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f8c0688e0 con 0x7f2f8c103840 2024-03-28T03:44:39.241 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:39.238+0000 7f2f83400640 1 -- 172.21.15.119:0/3345590493 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 27) v1 ==== 137422+0+0 (secure 0 0 0) 0x7f2f7c005020 con 0x7f2f8c103840 2024-03-28T03:44:39.241 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:39.239+0000 7f2f83400640 1 --2- 172.21.15.119:0/3345590493 >> [v2:172.21.15.119:6800/4273482239,v1:172.21.15.119:6801/4273482239] conn(0x7f2f700a2970 0x7f2f700a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T03:44:39.242 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:39.239+0000 7f2f83400640 1 -- 172.21.15.119:0/3345590493 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 11163+0+0 (secure 0 0 0) 0x7f2f7c0cff80 con 0x7f2f8c103840 2024-03-28T03:44:39.242 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:39.239+0000 7f2f81400640 1 --2- 172.21.15.119:0/3345590493 >> [v2:172.21.15.119:6800/4273482239,v1:172.21.15.119:6801/4273482239] conn(0x7f2f700a2970 0x7f2f700a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T03:44:39.242 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:39.240+0000 7f2f81400640 1 --2- 172.21.15.119:0/3345590493 >> [v2:172.21.15.119:6800/4273482239,v1:172.21.15.119:6801/4273482239] conn(0x7f2f700a2970 0x7f2f700a4e30 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f2f8c101910 tx=0x7f2f74005f90 comp rx=0 tx=0).ready entity=mgr.25277 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T03:44:39.245 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:39.243+0000 7f2f83400640 1 -- 172.21.15.119:0/3345590493 <== mon.1 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f2f7c09c230 con 0x7f2f8c103840 2024-03-28T03:44:39.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:44:39 smithi165 ceph-mon[36302]: from='client.25564 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T03:44:39.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:44:39 smithi165 ceph-mon[36302]: pgmap v6193: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:44:39.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:44:39 smithi165 ceph-mon[36302]: from='client.? 172.21.15.119:0/205200466' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-28T03:44:39.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:44:39 smithi165 ceph-mon[36302]: from='client.? 172.21.15.119:0/3672047464' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-28T03:44:39.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:44:39 smithi165 ceph-mon[36302]: from='client.? 172.21.15.119:0/844729617' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-28T03:44:39.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:44:39 smithi119 ceph-mon[31518]: from='client.25564 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T03:44:39.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:44:39 smithi119 ceph-mon[31518]: pgmap v6193: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:44:39.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:44:39 smithi119 ceph-mon[31518]: from='client.? 172.21.15.119:0/205200466' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-28T03:44:39.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:44:39 smithi119 ceph-mon[31518]: from='client.? 172.21.15.119:0/3672047464' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-28T03:44:39.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:44:39 smithi119 ceph-mon[31518]: from='client.? 172.21.15.119:0/844729617' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-28T03:44:39.445 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:39.443+0000 7f2f90a00640 1 -- 172.21.15.119:0/3345590493 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2f8c10be10 con 0x7f2f8c103840 2024-03-28T03:44:39.446 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:39.444+0000 7f2f83400640 1 -- 172.21.15.119:0/3345590493 <== mon.1 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v125) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2f7c09bb30 con 0x7f2f8c103840 2024-03-28T03:44:39.446 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 64 2024-03-28T03:44:39.449 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:39.447+0000 7f2f90a00640 1 -- 172.21.15.119:0/3345590493 >> [v2:172.21.15.119:6800/4273482239,v1:172.21.15.119:6801/4273482239] conn(0x7f2f700a2970 msgr2=0x7f2f700a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T03:44:39.449 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:39.447+0000 7f2f90a00640 1 --2- 172.21.15.119:0/3345590493 >> [v2:172.21.15.119:6800/4273482239,v1:172.21.15.119:6801/4273482239] conn(0x7f2f700a2970 0x7f2f700a4e30 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f2f8c101910 tx=0x7f2f74005f90 comp rx=0 tx=0).stop 2024-03-28T03:44:39.449 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:39.447+0000 7f2f90a00640 1 -- 172.21.15.119:0/3345590493 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2f8c103840 msgr2=0x7f2f8c19a0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T03:44:39.449 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:39.447+0000 7f2f90a00640 1 --2- 172.21.15.119:0/3345590493 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2f8c103840 0x7f2f8c19a0d0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f2f7c00eb90 tx=0x7f2f7c00c6a0 comp rx=0 tx=0).stop 2024-03-28T03:44:39.450 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:39.447+0000 7f2f90a00640 1 -- 172.21.15.119:0/3345590493 shutdown_connections 2024-03-28T03:44:39.450 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:39.447+0000 7f2f90a00640 1 --2- 172.21.15.119:0/3345590493 >> [v2:172.21.15.119:6800/4273482239,v1:172.21.15.119:6801/4273482239] conn(0x7f2f700a2970 0x7f2f700a4e30 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:44:39.450 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:39.447+0000 7f2f90a00640 1 --2- 172.21.15.119:0/3345590493 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2f8c109850 0x7f2f8c19e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:44:39.450 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:39.447+0000 7f2f90a00640 1 --2- 172.21.15.119:0/3345590493 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2f8c104200 0x7f2f8c19a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:44:39.450 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:39.447+0000 7f2f90a00640 1 --2- 172.21.15.119:0/3345590493 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2f8c103840 0x7f2f8c19a0d0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:44:39.450 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:39.447+0000 7f2f90a00640 1 -- 172.21.15.119:0/3345590493 >> 172.21.15.119:0/3345590493 conn(0x7f2f8c0ff310 msgr2=0x7f2f8c0ff8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T03:44:39.450 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:39.447+0000 7f2f90a00640 1 -- 172.21.15.119:0/3345590493 shutdown_connections 2024-03-28T03:44:39.450 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:39.447+0000 7f2f90a00640 1 -- 172.21.15.119:0/3345590493 wait complete. 2024-03-28T03:44:39.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:44:39 smithi132 ceph-mon[35941]: from='client.25564 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T03:44:39.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:44:39 smithi132 ceph-mon[35941]: pgmap v6193: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:44:39.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:44:39 smithi132 ceph-mon[35941]: from='client.? 172.21.15.119:0/205200466' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-28T03:44:39.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:44:39 smithi132 ceph-mon[35941]: from='client.? 172.21.15.119:0/3672047464' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-28T03:44:39.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:44:39 smithi132 ceph-mon[35941]: from='client.? 172.21.15.119:0/844729617' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-28T03:44:39.468 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-03-28T03:44:39.679 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:39.677+0000 7f7e42600640 1 -- 172.21.15.119:0/3162655074 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7e3c104210 msgr2=0x7f7e3c110720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T03:44:39.679 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:39.677+0000 7f7e42600640 1 --2- 172.21.15.119:0/3162655074 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7e3c104210 0x7f7e3c110720 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f7e34009cd0 tx=0x7f7e3402c0f0 comp rx=0 tx=0).stop 2024-03-28T03:44:39.679 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:39.677+0000 7f7e42600640 1 -- 172.21.15.119:0/3162655074 shutdown_connections 2024-03-28T03:44:39.680 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:39.677+0000 7f7e42600640 1 --2- 172.21.15.119:0/3162655074 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7e3c104210 0x7f7e3c110720 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:44:39.680 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:39.677+0000 7f7e42600640 1 --2- 172.21.15.119:0/3162655074 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7e3c103850 0x7f7e3c103cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:44:39.680 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:39.677+0000 7f7e42600640 1 --2- 172.21.15.119:0/3162655074 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e3c109860 0x7f7e3c109c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:44:39.680 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:39.677+0000 7f7e42600640 1 -- 172.21.15.119:0/3162655074 >> 172.21.15.119:0/3162655074 conn(0x7f7e3c0ff320 msgr2=0x7f7e3c101740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T03:44:39.680 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:39.677+0000 7f7e42600640 1 -- 172.21.15.119:0/3162655074 shutdown_connections 2024-03-28T03:44:39.680 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:39.677+0000 7f7e42600640 1 -- 172.21.15.119:0/3162655074 wait complete. 2024-03-28T03:44:39.683 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:39.678+0000 7f7e42600640 1 Processor -- start 2024-03-28T03:44:39.683 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:39.678+0000 7f7e42600640 1 -- start start 2024-03-28T03:44:39.683 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:39.679+0000 7f7e42600640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7e3c103850 0x7f7e3c19a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T03:44:39.683 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:39.679+0000 7f7e42600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e3c104210 0x7f7e3c19a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T03:44:39.683 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:39.679+0000 7f7e42600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7e3c109860 0x7f7e3c19e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T03:44:39.683 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:39.679+0000 7f7e42600640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e3c113c30 con 0x7f7e3c104210 2024-03-28T03:44:39.684 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:39.679+0000 7f7e42600640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e3c113ab0 con 0x7f7e3c103850 2024-03-28T03:44:39.684 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:39.679+0000 7f7e42600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e3c113db0 con 0x7f7e3c109860 2024-03-28T03:44:39.684 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:39.679+0000 7f7e3b400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7e3c103850 0x7f7e3c19a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T03:44:39.684 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:39.679+0000 7f7e3be00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7e3c109860 0x7f7e3c19e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T03:44:39.684 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:39.679+0000 7f7e33e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e3c104210 0x7f7e3c19a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T03:44:39.684 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:39.679+0000 7f7e3b400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7e3c103850 0x7f7e3c19a110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.119:38410/0 (socket says 172.21.15.119:38410) 2024-03-28T03:44:39.684 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:39.679+0000 7f7e3b400640 1 -- 172.21.15.119:0/3586976322 learned_addr learned my addr 172.21.15.119:0/3586976322 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T03:44:39.684 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:39.679+0000 7f7e3be00640 1 -- 172.21.15.119:0/3586976322 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7e3c103850 msgr2=0x7f7e3c19a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T03:44:39.684 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:39.679+0000 7f7e3be00640 1 --2- 172.21.15.119:0/3586976322 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7e3c103850 0x7f7e3c19a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:44:39.685 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:39.679+0000 7f7e3be00640 1 -- 172.21.15.119:0/3586976322 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e3c104210 msgr2=0x7f7e3c19a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T03:44:39.685 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:39.679+0000 7f7e3be00640 1 --2- 172.21.15.119:0/3586976322 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e3c104210 0x7f7e3c19a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:44:39.685 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:39.679+0000 7f7e3be00640 1 -- 172.21.15.119:0/3586976322 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e3c19f100 con 0x7f7e3c109860 2024-03-28T03:44:39.685 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:39.679+0000 7f7e3b400640 1 --2- 172.21.15.119:0/3586976322 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7e3c103850 0x7f7e3c19a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T03:44:39.685 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:39.680+0000 7f7e3be00640 1 --2- 172.21.15.119:0/3586976322 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7e3c109860 0x7f7e3c19e9d0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f7e34037040 tx=0x7f7e34032ea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T03:44:39.685 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:39.680+0000 7f7e38c00640 1 -- 172.21.15.119:0/3586976322 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e34041020 con 0x7f7e3c109860 2024-03-28T03:44:39.685 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:39.680+0000 7f7e38c00640 1 -- 172.21.15.119:0/3586976322 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7e34034cd0 con 0x7f7e3c109860 2024-03-28T03:44:39.685 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:39.680+0000 7f7e38c00640 1 -- 172.21.15.119:0/3586976322 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e34039040 con 0x7f7e3c109860 2024-03-28T03:44:39.685 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:39.680+0000 7f7e42600640 1 -- 172.21.15.119:0/3586976322 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e3c19f390 con 0x7f7e3c109860 2024-03-28T03:44:39.685 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:39.680+0000 7f7e42600640 1 -- 172.21.15.119:0/3586976322 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e3c19f800 con 0x7f7e3c109860 2024-03-28T03:44:39.686 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:39.681+0000 7f7e42600640 1 -- 172.21.15.119:0/3586976322 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e3c0688e0 con 0x7f7e3c109860 2024-03-28T03:44:39.686 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:39.683+0000 7f7e38c00640 1 -- 172.21.15.119:0/3586976322 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 27) v1 ==== 137422+0+0 (secure 0 0 0) 0x7f7e34044020 con 0x7f7e3c109860 2024-03-28T03:44:39.687 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:39.684+0000 7f7e38c00640 1 --2- 172.21.15.119:0/3586976322 >> [v2:172.21.15.119:6800/4273482239,v1:172.21.15.119:6801/4273482239] conn(0x7f7e1c0a2970 0x7f7e1c0a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T03:44:39.687 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:39.684+0000 7f7e38c00640 1 -- 172.21.15.119:0/3586976322 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 11163+0+0 (secure 0 0 0) 0x7f7e340f1640 con 0x7f7e3c109860 2024-03-28T03:44:39.687 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:39.685+0000 7f7e3b400640 1 --2- 172.21.15.119:0/3586976322 >> [v2:172.21.15.119:6800/4273482239,v1:172.21.15.119:6801/4273482239] conn(0x7f7e1c0a2970 0x7f7e1c0a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T03:44:39.687 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:39.685+0000 7f7e3b400640 1 --2- 172.21.15.119:0/3586976322 >> [v2:172.21.15.119:6800/4273482239,v1:172.21.15.119:6801/4273482239] conn(0x7f7e1c0a2970 0x7f7e1c0a4e30 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f7e2c011440 tx=0x7f7e2c005f90 comp rx=0 tx=0).ready entity=mgr.25277 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T03:44:39.688 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:39.686+0000 7f7e38c00640 1 -- 172.21.15.119:0/3586976322 <== mon.1 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f7e3c19f800 con 0x7f7e3c109860 2024-03-28T03:44:39.937 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:39.936+0000 7f7e42600640 1 -- 172.21.15.119:0/3586976322 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) v1 -- 0x7f7e3c19b460 con 0x7f7e3c109860 2024-03-28T03:44:39.946 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:39.944+0000 7f7e38c00640 1 -- 172.21.15.119:0/3586976322 <== mon.1 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v27) v1 ==== 81+0+740370 (secure 0 0 0) 0x7f7e3c19b460 con 0x7f7e3c109860 2024-03-28T03:44:39.959 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-28T03:44:39.960 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-03-28T03:44:39.960 INFO:teuthology.orchestra.run.smithi119.stdout: "epoch": 27, 2024-03-28T03:44:39.960 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:39.960 INFO:teuthology.orchestra.run.smithi119.stdout: "active_gid": 25277, 2024-03-28T03:44:39.960 INFO:teuthology.orchestra.run.smithi119.stdout: "active_name": "x", 2024-03-28T03:44:39.960 INFO:teuthology.orchestra.run.smithi119.stdout: "active_addrs": { 2024-03-28T03:44:39.960 INFO:teuthology.orchestra.run.smithi119.stdout: "addrvec": [ 2024-03-28T03:44:39.961 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-03-28T03:44:39.961 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v2", 2024-03-28T03:44:39.961 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "172.21.15.119:6800", 2024-03-28T03:44:39.961 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 4273482239 2024-03-28T03:44:39.961 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:39.961 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-03-28T03:44:39.961 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v1", 2024-03-28T03:44:39.961 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "172.21.15.119:6801", 2024-03-28T03:44:39.961 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 4273482239 2024-03-28T03:44:39.962 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:39.962 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-03-28T03:44:39.962 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:39.962 INFO:teuthology.orchestra.run.smithi119.stdout: "active_addr": "172.21.15.119:6801/4273482239", 2024-03-28T03:44:39.962 INFO:teuthology.orchestra.run.smithi119.stdout: "active_change": "2024-03-28T00:26:32.965890+0000", 2024-03-28T03:44:39.962 INFO:teuthology.orchestra.run.smithi119.stdout: "active_mgr_features": 4540701547738038271, 2024-03-28T03:44:39.962 INFO:teuthology.orchestra.run.smithi119.stdout: "available": true, 2024-03-28T03:44:39.962 INFO:teuthology.orchestra.run.smithi119.stdout: "standbys": [ 2024-03-28T03:44:39.963 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-03-28T03:44:39.963 INFO:teuthology.orchestra.run.smithi119.stdout: "gid": 25268, 2024-03-28T03:44:39.963 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "y", 2024-03-28T03:44:39.963 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr_features": 4540701547738038271, 2024-03-28T03:44:39.963 INFO:teuthology.orchestra.run.smithi119.stdout: "available_modules": [ 2024-03-28T03:44:39.963 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-03-28T03:44:39.963 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "alerts", 2024-03-28T03:44:39.963 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2024-03-28T03:44:39.963 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2024-03-28T03:44:39.964 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2024-03-28T03:44:39.964 INFO:teuthology.orchestra.run.smithi119.stdout: "interval": { 2024-03-28T03:44:39.964 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "interval", 2024-03-28T03:44:39.964 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "secs", 2024-03-28T03:44:39.964 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:39.964 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:39.964 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "60", 2024-03-28T03:44:39.964 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:39.965 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:39.965 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:39.965 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "How frequently to reexamine health status", 2024-03-28T03:44:39.965 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:39.965 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:39.965 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:39.965 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:39.965 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2024-03-28T03:44:39.965 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2024-03-28T03:44:39.966 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:39.966 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:39.966 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:39.966 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:39.966 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:39.966 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:39.966 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:39.966 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:39.967 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:39.967 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:39.967 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:39.967 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:39.967 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:39.967 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:39.967 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:39.967 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:39.967 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:39.968 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:39.968 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:39.968 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2024-03-28T03:44:39.968 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2024-03-28T03:44:39.968 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:39.968 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:39.968 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:39.968 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:39.968 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:39.969 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:39.969 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:39.969 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:39.969 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:39.969 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:39.969 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:39.969 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:39.969 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2024-03-28T03:44:39.969 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2024-03-28T03:44:39.970 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:39.970 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:39.970 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:39.970 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2024-03-28T03:44:39.970 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:39.970 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:39.970 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:39.970 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:39.970 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:39.971 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:39.971 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:39.971 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:39.971 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:39.971 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:39.971 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:39.971 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:39.971 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:39.972 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:39.972 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:39.972 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2024-03-28T03:44:39.972 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2024-03-28T03:44:39.972 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:39.972 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:39.972 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:39.972 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:39.972 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:39.973 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:39.973 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:39.973 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:39.973 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:39.973 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:39.973 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:39.973 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:39.973 INFO:teuthology.orchestra.run.smithi119.stdout: "smtp_destination": { 2024-03-28T03:44:39.973 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "smtp_destination", 2024-03-28T03:44:39.974 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:39.974 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:39.974 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:39.974 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:39.974 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:39.974 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:39.974 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:39.974 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Email address to send alerts to", 2024-03-28T03:44:39.975 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:39.975 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:39.975 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:39.975 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:39.975 INFO:teuthology.orchestra.run.smithi119.stdout: "smtp_from_name": { 2024-03-28T03:44:39.975 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "smtp_from_name", 2024-03-28T03:44:39.975 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:39.975 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:39.975 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:39.976 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "Ceph", 2024-03-28T03:44:39.976 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:39.976 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:39.976 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:39.976 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Email From: name", 2024-03-28T03:44:39.976 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:39.976 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:39.976 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:39.976 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:39.977 INFO:teuthology.orchestra.run.smithi119.stdout: "smtp_host": { 2024-03-28T03:44:39.977 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "smtp_host", 2024-03-28T03:44:39.977 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:39.977 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:39.977 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:39.977 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:39.977 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:39.977 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:39.977 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:39.978 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "SMTP server", 2024-03-28T03:44:39.978 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:39.978 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:39.978 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:39.978 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:39.978 INFO:teuthology.orchestra.run.smithi119.stdout: "smtp_password": { 2024-03-28T03:44:39.978 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "smtp_password", 2024-03-28T03:44:39.978 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:39.979 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:39.979 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:39.979 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:39.979 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:39.979 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:39.979 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:39.979 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Password to authenticate with", 2024-03-28T03:44:39.979 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:39.979 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:39.980 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:39.980 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:39.980 INFO:teuthology.orchestra.run.smithi119.stdout: "smtp_port": { 2024-03-28T03:44:39.980 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "smtp_port", 2024-03-28T03:44:39.980 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2024-03-28T03:44:39.980 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:39.980 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:39.980 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "465", 2024-03-28T03:44:39.980 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:39.981 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:39.981 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:39.981 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "SMTP port", 2024-03-28T03:44:39.981 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:39.981 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:39.981 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:39.981 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:39.981 INFO:teuthology.orchestra.run.smithi119.stdout: "smtp_sender": { 2024-03-28T03:44:39.982 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "smtp_sender", 2024-03-28T03:44:39.982 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:39.982 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:39.982 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:39.982 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:39.982 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:39.982 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:39.982 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:39.983 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "SMTP envelope sender", 2024-03-28T03:44:39.983 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:39.983 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:39.983 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:39.983 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:39.983 INFO:teuthology.orchestra.run.smithi119.stdout: "smtp_ssl": { 2024-03-28T03:44:39.983 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "smtp_ssl", 2024-03-28T03:44:39.983 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:39.984 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:39.984 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:39.984 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2024-03-28T03:44:39.984 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:39.984 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:39.984 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:39.984 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Use SSL to connect to SMTP server", 2024-03-28T03:44:39.984 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:39.984 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:39.985 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:39.985 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:39.985 INFO:teuthology.orchestra.run.smithi119.stdout: "smtp_user": { 2024-03-28T03:44:39.985 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "smtp_user", 2024-03-28T03:44:39.985 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:39.985 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:39.985 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:39.985 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:39.985 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:39.986 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:39.986 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:39.986 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "User to authenticate as", 2024-03-28T03:44:39.986 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:39.986 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:39.986 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:39.986 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:39.986 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:39.986 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:39.987 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-03-28T03:44:39.987 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "balancer", 2024-03-28T03:44:39.987 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2024-03-28T03:44:39.987 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2024-03-28T03:44:39.987 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2024-03-28T03:44:39.987 INFO:teuthology.orchestra.run.smithi119.stdout: "active": { 2024-03-28T03:44:39.987 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "active", 2024-03-28T03:44:39.987 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:39.987 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:39.988 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:39.988 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2024-03-28T03:44:39.988 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:39.988 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:39.988 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:39.988 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "automatically balance PGs across cluster", 2024-03-28T03:44:39.988 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:39.988 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:39.988 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:39.989 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:39.989 INFO:teuthology.orchestra.run.smithi119.stdout: "begin_time": { 2024-03-28T03:44:39.989 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "begin_time", 2024-03-28T03:44:39.989 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:39.989 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:39.989 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:39.989 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "0000", 2024-03-28T03:44:39.989 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:39.989 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:39.990 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:39.990 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "beginning time of day to automatically balance", 2024-03-28T03:44:39.990 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-28T03:44:39.990 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:39.990 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:39.990 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:39.990 INFO:teuthology.orchestra.run.smithi119.stdout: "begin_weekday": { 2024-03-28T03:44:39.990 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "begin_weekday", 2024-03-28T03:44:39.990 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "uint", 2024-03-28T03:44:39.991 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:39.991 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:39.991 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "0", 2024-03-28T03:44:39.991 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "0", 2024-03-28T03:44:39.991 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "6", 2024-03-28T03:44:39.991 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:39.991 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-03-28T03:44:39.991 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-28T03:44:39.991 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:39.992 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:39.992 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:39.992 INFO:teuthology.orchestra.run.smithi119.stdout: "crush_compat_max_iterations": { 2024-03-28T03:44:39.992 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "crush_compat_max_iterations", 2024-03-28T03:44:39.992 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "uint", 2024-03-28T03:44:39.992 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:39.992 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:39.992 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "25", 2024-03-28T03:44:39.992 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "1", 2024-03-28T03:44:39.993 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "250", 2024-03-28T03:44:39.993 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:39.993 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-03-28T03:44:39.993 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:39.993 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:39.993 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:39.993 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:39.993 INFO:teuthology.orchestra.run.smithi119.stdout: "crush_compat_metrics": { 2024-03-28T03:44:39.994 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "crush_compat_metrics", 2024-03-28T03:44:39.994 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:39.994 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:39.994 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:39.994 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "pgs,objects,bytes", 2024-03-28T03:44:39.994 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:39.994 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:39.994 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:39.995 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-03-28T03:44:39.995 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-03-28T03:44:39.995 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:39.995 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:39.995 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:39.995 INFO:teuthology.orchestra.run.smithi119.stdout: "crush_compat_step": { 2024-03-28T03:44:39.995 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "crush_compat_step", 2024-03-28T03:44:39.995 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "float", 2024-03-28T03:44:39.996 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:39.996 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:39.996 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "0.5", 2024-03-28T03:44:39.996 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "0.001", 2024-03-28T03:44:39.996 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "0.999", 2024-03-28T03:44:39.996 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:39.996 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "aggressiveness of optimization", 2024-03-28T03:44:39.996 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-03-28T03:44:39.996 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:39.997 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:39.997 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:39.997 INFO:teuthology.orchestra.run.smithi119.stdout: "end_time": { 2024-03-28T03:44:39.997 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "end_time", 2024-03-28T03:44:39.997 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:39.997 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:39.997 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:39.997 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "2359", 2024-03-28T03:44:39.998 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:39.998 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:39.998 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:39.998 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "ending time of day to automatically balance", 2024-03-28T03:44:39.998 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-28T03:44:39.998 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:39.998 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:39.998 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:39.998 INFO:teuthology.orchestra.run.smithi119.stdout: "end_weekday": { 2024-03-28T03:44:39.999 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "end_weekday", 2024-03-28T03:44:39.999 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "uint", 2024-03-28T03:44:39.999 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:39.999 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:39.999 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "0", 2024-03-28T03:44:39.999 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "0", 2024-03-28T03:44:39.999 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "6", 2024-03-28T03:44:39.999 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.000 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-03-28T03:44:40.000 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-28T03:44:40.000 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.000 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.000 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.000 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2024-03-28T03:44:40.000 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2024-03-28T03:44:40.000 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.000 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.001 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.001 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:40.001 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.001 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.001 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:40.001 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:40.001 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:40.001 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:40.001 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:40.002 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:40.002 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:40.002 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:40.002 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.002 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.002 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.002 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.002 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.003 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2024-03-28T03:44:40.003 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2024-03-28T03:44:40.003 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.003 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.003 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.003 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:40.003 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.004 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.004 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.004 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.004 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.004 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.004 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.004 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.004 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2024-03-28T03:44:40.004 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2024-03-28T03:44:40.005 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.005 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.005 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.005 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2024-03-28T03:44:40.005 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.005 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.005 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:40.005 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:40.005 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:40.006 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:40.006 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:40.006 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:40.006 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:40.006 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:40.006 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.006 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.006 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.007 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.007 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.007 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2024-03-28T03:44:40.007 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2024-03-28T03:44:40.007 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.007 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.007 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.007 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:40.007 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.008 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.008 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.008 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.008 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.008 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.008 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.008 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.008 INFO:teuthology.orchestra.run.smithi119.stdout: "min_score": { 2024-03-28T03:44:40.009 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "min_score", 2024-03-28T03:44:40.009 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "float", 2024-03-28T03:44:40.009 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.009 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.009 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "0", 2024-03-28T03:44:40.009 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.009 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.009 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.010 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-03-28T03:44:40.010 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.010 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.010 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.010 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.010 INFO:teuthology.orchestra.run.smithi119.stdout: "mode": { 2024-03-28T03:44:40.010 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "mode", 2024-03-28T03:44:40.010 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.010 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.011 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.011 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "upmap", 2024-03-28T03:44:40.011 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.011 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.011 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:40.011 INFO:teuthology.orchestra.run.smithi119.stdout: "crush-compat", 2024-03-28T03:44:40.011 INFO:teuthology.orchestra.run.smithi119.stdout: "none", 2024-03-28T03:44:40.011 INFO:teuthology.orchestra.run.smithi119.stdout: "read", 2024-03-28T03:44:40.012 INFO:teuthology.orchestra.run.smithi119.stdout: "upmap", 2024-03-28T03:44:40.012 INFO:teuthology.orchestra.run.smithi119.stdout: "upmap-read" 2024-03-28T03:44:40.012 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:40.012 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Balancer mode", 2024-03-28T03:44:40.012 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.012 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.012 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.012 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.013 INFO:teuthology.orchestra.run.smithi119.stdout: "pool_ids": { 2024-03-28T03:44:40.013 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "pool_ids", 2024-03-28T03:44:40.013 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.013 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.013 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.013 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:40.013 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.013 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.013 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.014 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-03-28T03:44:40.014 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.014 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.014 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.014 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.014 INFO:teuthology.orchestra.run.smithi119.stdout: "sleep_interval": { 2024-03-28T03:44:40.014 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "sleep_interval", 2024-03-28T03:44:40.014 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "secs", 2024-03-28T03:44:40.015 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.015 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.015 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "60", 2024-03-28T03:44:40.015 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.015 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.015 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.015 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-03-28T03:44:40.015 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.015 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.016 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.016 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.016 INFO:teuthology.orchestra.run.smithi119.stdout: "upmap_max_deviation": { 2024-03-28T03:44:40.016 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "upmap_max_deviation", 2024-03-28T03:44:40.016 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2024-03-28T03:44:40.016 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.016 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.016 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "5", 2024-03-28T03:44:40.017 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "1", 2024-03-28T03:44:40.017 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.017 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.017 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "deviation below which no optimization is attempted", 2024-03-28T03:44:40.017 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-03-28T03:44:40.017 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.017 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.017 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.017 INFO:teuthology.orchestra.run.smithi119.stdout: "upmap_max_optimizations": { 2024-03-28T03:44:40.018 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "upmap_max_optimizations", 2024-03-28T03:44:40.018 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "uint", 2024-03-28T03:44:40.018 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.018 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.018 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "10", 2024-03-28T03:44:40.018 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.018 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.019 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.019 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-03-28T03:44:40.019 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.019 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.019 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.019 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:40.019 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:40.019 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.019 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-03-28T03:44:40.020 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "cephadm", 2024-03-28T03:44:40.020 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2024-03-28T03:44:40.020 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2024-03-28T03:44:40.020 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2024-03-28T03:44:40.020 INFO:teuthology.orchestra.run.smithi119.stdout: "agent_down_multiplier": { 2024-03-28T03:44:40.020 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "agent_down_multiplier", 2024-03-28T03:44:40.020 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "float", 2024-03-28T03:44:40.020 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.020 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.021 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "3.0", 2024-03-28T03:44:40.021 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.021 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.021 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.021 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-03-28T03:44:40.021 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.021 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.022 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.022 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.022 INFO:teuthology.orchestra.run.smithi119.stdout: "agent_refresh_rate": { 2024-03-28T03:44:40.022 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "agent_refresh_rate", 2024-03-28T03:44:40.022 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "secs", 2024-03-28T03:44:40.022 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.022 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.022 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "20", 2024-03-28T03:44:40.022 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.023 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.023 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.023 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-03-28T03:44:40.023 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.023 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.023 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.023 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.023 INFO:teuthology.orchestra.run.smithi119.stdout: "agent_starting_port": { 2024-03-28T03:44:40.023 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "agent_starting_port", 2024-03-28T03:44:40.024 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2024-03-28T03:44:40.024 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.024 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.024 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "4721", 2024-03-28T03:44:40.024 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.024 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.024 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.024 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-03-28T03:44:40.025 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.025 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.025 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.025 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.025 INFO:teuthology.orchestra.run.smithi119.stdout: "allow_ptrace": { 2024-03-28T03:44:40.025 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "allow_ptrace", 2024-03-28T03:44:40.025 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.025 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.026 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.026 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:40.026 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.026 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.026 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.026 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-03-28T03:44:40.026 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-03-28T03:44:40.027 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.027 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.027 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.027 INFO:teuthology.orchestra.run.smithi119.stdout: "autotune_interval": { 2024-03-28T03:44:40.027 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "autotune_interval", 2024-03-28T03:44:40.027 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "secs", 2024-03-28T03:44:40.027 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.027 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.027 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "600", 2024-03-28T03:44:40.028 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.028 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.028 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.028 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "how frequently to autotune daemon memory", 2024-03-28T03:44:40.028 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.028 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.028 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.028 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.028 INFO:teuthology.orchestra.run.smithi119.stdout: "autotune_memory_target_ratio": { 2024-03-28T03:44:40.029 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "autotune_memory_target_ratio", 2024-03-28T03:44:40.029 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "float", 2024-03-28T03:44:40.029 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.029 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.029 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "0.7", 2024-03-28T03:44:40.029 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.029 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.029 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.029 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-03-28T03:44:40.030 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.030 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.030 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.030 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.030 INFO:teuthology.orchestra.run.smithi119.stdout: "cephadm_log_destination": { 2024-03-28T03:44:40.030 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "cephadm_log_destination", 2024-03-28T03:44:40.030 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.030 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.031 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.031 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:40.031 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.031 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.031 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:40.031 INFO:teuthology.orchestra.run.smithi119.stdout: "file", 2024-03-28T03:44:40.031 INFO:teuthology.orchestra.run.smithi119.stdout: "file,syslog", 2024-03-28T03:44:40.031 INFO:teuthology.orchestra.run.smithi119.stdout: "syslog" 2024-03-28T03:44:40.031 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:40.032 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-03-28T03:44:40.032 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.032 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.032 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.032 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.032 INFO:teuthology.orchestra.run.smithi119.stdout: "cgroups_split": { 2024-03-28T03:44:40.032 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "cgroups_split", 2024-03-28T03:44:40.032 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.033 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.033 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.033 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2024-03-28T03:44:40.033 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.033 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.033 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.033 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-03-28T03:44:40.033 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.033 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.034 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.034 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.034 INFO:teuthology.orchestra.run.smithi119.stdout: "config_checks_enabled": { 2024-03-28T03:44:40.034 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "config_checks_enabled", 2024-03-28T03:44:40.034 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.034 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.034 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.034 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:40.035 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.035 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.035 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.035 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-03-28T03:44:40.035 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.035 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.035 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.035 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.035 INFO:teuthology.orchestra.run.smithi119.stdout: "config_dashboard": { 2024-03-28T03:44:40.036 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "config_dashboard", 2024-03-28T03:44:40.036 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.036 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.036 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.036 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2024-03-28T03:44:40.036 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.036 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.036 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.037 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-03-28T03:44:40.037 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.037 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.037 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.037 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.037 INFO:teuthology.orchestra.run.smithi119.stdout: "container_image_alertmanager": { 2024-03-28T03:44:40.037 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "container_image_alertmanager", 2024-03-28T03:44:40.037 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.037 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.038 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.038 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-03-28T03:44:40.038 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.038 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.038 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.038 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Prometheus container image", 2024-03-28T03:44:40.038 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.038 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.039 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.039 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.039 INFO:teuthology.orchestra.run.smithi119.stdout: "container_image_base": { 2024-03-28T03:44:40.039 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "container_image_base", 2024-03-28T03:44:40.039 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.039 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.039 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.039 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "quay.io/ceph/ceph", 2024-03-28T03:44:40.039 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.040 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.040 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.040 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Container image name, without the tag", 2024-03-28T03:44:40.040 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.040 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.040 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.040 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.040 INFO:teuthology.orchestra.run.smithi119.stdout: "container_image_elasticsearch": { 2024-03-28T03:44:40.041 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "container_image_elasticsearch", 2024-03-28T03:44:40.041 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.041 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.041 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.041 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-03-28T03:44:40.041 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.041 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.041 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.042 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "elasticsearch container image", 2024-03-28T03:44:40.042 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.042 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.042 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.042 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.042 INFO:teuthology.orchestra.run.smithi119.stdout: "container_image_grafana": { 2024-03-28T03:44:40.042 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "container_image_grafana", 2024-03-28T03:44:40.042 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.042 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.043 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.043 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.12", 2024-03-28T03:44:40.043 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.043 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.043 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.043 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Prometheus container image", 2024-03-28T03:44:40.043 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.043 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.044 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.044 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.044 INFO:teuthology.orchestra.run.smithi119.stdout: "container_image_haproxy": { 2024-03-28T03:44:40.044 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "container_image_haproxy", 2024-03-28T03:44:40.044 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.044 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.044 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.044 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-03-28T03:44:40.045 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.045 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.045 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.045 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "HAproxy container image", 2024-03-28T03:44:40.045 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.045 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.045 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.045 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.046 INFO:teuthology.orchestra.run.smithi119.stdout: "container_image_jaeger_agent": { 2024-03-28T03:44:40.046 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "container_image_jaeger_agent", 2024-03-28T03:44:40.046 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.046 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.046 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.046 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-03-28T03:44:40.046 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.046 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.047 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.047 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Jaeger agent container image", 2024-03-28T03:44:40.047 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.047 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.047 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.047 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.047 INFO:teuthology.orchestra.run.smithi119.stdout: "container_image_jaeger_collector": { 2024-03-28T03:44:40.047 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "container_image_jaeger_collector", 2024-03-28T03:44:40.047 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.048 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.048 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.048 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-03-28T03:44:40.048 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.048 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.048 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.048 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Jaeger collector container image", 2024-03-28T03:44:40.048 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.049 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.049 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.049 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.049 INFO:teuthology.orchestra.run.smithi119.stdout: "container_image_jaeger_query": { 2024-03-28T03:44:40.049 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "container_image_jaeger_query", 2024-03-28T03:44:40.049 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.049 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.049 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.050 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-03-28T03:44:40.050 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.050 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.050 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.050 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Jaeger query container image", 2024-03-28T03:44:40.050 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.050 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.050 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.051 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.051 INFO:teuthology.orchestra.run.smithi119.stdout: "container_image_keepalived": { 2024-03-28T03:44:40.051 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "container_image_keepalived", 2024-03-28T03:44:40.051 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.051 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.051 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.051 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-03-28T03:44:40.051 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.052 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.052 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.052 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Keepalived container image", 2024-03-28T03:44:40.052 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.052 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.052 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.052 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.052 INFO:teuthology.orchestra.run.smithi119.stdout: "container_image_loki": { 2024-03-28T03:44:40.052 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "container_image_loki", 2024-03-28T03:44:40.053 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.053 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.053 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.053 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-03-28T03:44:40.053 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.053 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.053 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.053 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Loki container image", 2024-03-28T03:44:40.054 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.054 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.054 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.054 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.054 INFO:teuthology.orchestra.run.smithi119.stdout: "container_image_node_exporter": { 2024-03-28T03:44:40.054 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "container_image_node_exporter", 2024-03-28T03:44:40.054 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.054 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.055 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.055 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-03-28T03:44:40.055 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.055 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.055 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.055 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Prometheus container image", 2024-03-28T03:44:40.055 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.055 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.055 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.056 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.056 INFO:teuthology.orchestra.run.smithi119.stdout: "container_image_nvmeof": { 2024-03-28T03:44:40.056 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "container_image_nvmeof", 2024-03-28T03:44:40.056 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.056 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.056 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.056 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-03-28T03:44:40.056 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.056 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.057 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.057 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Nvme-of container image", 2024-03-28T03:44:40.057 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.057 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.057 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.057 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.057 INFO:teuthology.orchestra.run.smithi119.stdout: "container_image_prometheus": { 2024-03-28T03:44:40.057 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "container_image_prometheus", 2024-03-28T03:44:40.058 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.058 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.058 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.058 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-03-28T03:44:40.058 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.058 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.058 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.058 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Prometheus container image", 2024-03-28T03:44:40.058 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.059 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.059 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.059 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.059 INFO:teuthology.orchestra.run.smithi119.stdout: "container_image_promtail": { 2024-03-28T03:44:40.059 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "container_image_promtail", 2024-03-28T03:44:40.059 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.059 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.059 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.060 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-03-28T03:44:40.060 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.060 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.060 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.060 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Promtail container image", 2024-03-28T03:44:40.060 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.060 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.060 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.060 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.061 INFO:teuthology.orchestra.run.smithi119.stdout: "container_image_snmp_gateway": { 2024-03-28T03:44:40.061 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "container_image_snmp_gateway", 2024-03-28T03:44:40.061 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.061 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.061 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.061 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-03-28T03:44:40.061 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.061 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.061 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.062 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "SNMP Gateway container image", 2024-03-28T03:44:40.062 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.062 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.062 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.062 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.062 INFO:teuthology.orchestra.run.smithi119.stdout: "container_init": { 2024-03-28T03:44:40.062 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "container_init", 2024-03-28T03:44:40.062 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.063 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.063 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.063 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2024-03-28T03:44:40.063 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.063 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.063 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.063 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Run podman/docker with `--init`", 2024-03-28T03:44:40.063 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.063 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.064 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.064 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.064 INFO:teuthology.orchestra.run.smithi119.stdout: "daemon_cache_timeout": { 2024-03-28T03:44:40.064 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "daemon_cache_timeout", 2024-03-28T03:44:40.064 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "secs", 2024-03-28T03:44:40.064 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.064 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.064 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "600", 2024-03-28T03:44:40.064 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.065 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.065 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.065 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-03-28T03:44:40.065 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.065 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.065 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.065 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.066 INFO:teuthology.orchestra.run.smithi119.stdout: "default_cephadm_command_timeout": { 2024-03-28T03:44:40.066 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "default_cephadm_command_timeout", 2024-03-28T03:44:40.066 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2024-03-28T03:44:40.066 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.066 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.066 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "900", 2024-03-28T03:44:40.066 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.067 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.067 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.067 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-03-28T03:44:40.067 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.067 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.067 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.067 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.067 INFO:teuthology.orchestra.run.smithi119.stdout: "default_registry": { 2024-03-28T03:44:40.067 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "default_registry", 2024-03-28T03:44:40.068 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.068 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.068 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.068 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "docker.io", 2024-03-28T03:44:40.068 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.068 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.068 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.068 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-03-28T03:44:40.069 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.069 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.069 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.069 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.069 INFO:teuthology.orchestra.run.smithi119.stdout: "device_cache_timeout": { 2024-03-28T03:44:40.069 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "device_cache_timeout", 2024-03-28T03:44:40.069 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "secs", 2024-03-28T03:44:40.069 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.070 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.070 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "1800", 2024-03-28T03:44:40.070 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.070 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.070 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.070 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "seconds to cache device inventory", 2024-03-28T03:44:40.070 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.070 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.070 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.071 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.071 INFO:teuthology.orchestra.run.smithi119.stdout: "device_enhanced_scan": { 2024-03-28T03:44:40.071 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "device_enhanced_scan", 2024-03-28T03:44:40.071 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.071 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.071 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.071 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:40.071 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.071 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.072 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.072 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Use libstoragemgmt during device scans", 2024-03-28T03:44:40.072 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.072 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.072 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.072 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.072 INFO:teuthology.orchestra.run.smithi119.stdout: "facts_cache_timeout": { 2024-03-28T03:44:40.072 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "facts_cache_timeout", 2024-03-28T03:44:40.073 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "secs", 2024-03-28T03:44:40.073 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.073 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.073 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "60", 2024-03-28T03:44:40.073 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.073 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.073 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.073 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "seconds to cache host facts data", 2024-03-28T03:44:40.073 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.074 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.074 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.074 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.074 INFO:teuthology.orchestra.run.smithi119.stdout: "host_check_interval": { 2024-03-28T03:44:40.074 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "host_check_interval", 2024-03-28T03:44:40.074 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "secs", 2024-03-28T03:44:40.074 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.074 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.075 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "600", 2024-03-28T03:44:40.075 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.075 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.075 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.075 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "how frequently to perform a host check", 2024-03-28T03:44:40.075 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.075 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.075 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.076 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.076 INFO:teuthology.orchestra.run.smithi119.stdout: "hw_monitoring": { 2024-03-28T03:44:40.076 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "hw_monitoring", 2024-03-28T03:44:40.076 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.076 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.076 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.076 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:40.076 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.076 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.077 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.077 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-03-28T03:44:40.077 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.077 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.077 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.077 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.077 INFO:teuthology.orchestra.run.smithi119.stdout: "inventory_list_all": { 2024-03-28T03:44:40.077 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "inventory_list_all", 2024-03-28T03:44:40.077 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.078 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.078 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.078 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:40.078 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.078 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.078 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.078 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-03-28T03:44:40.078 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.079 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.079 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.079 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.079 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2024-03-28T03:44:40.079 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2024-03-28T03:44:40.079 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.079 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.079 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.079 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:40.080 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.080 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.080 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:40.080 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:40.080 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:40.080 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:40.080 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:40.080 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:40.080 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:40.081 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:40.081 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.081 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.081 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.081 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.081 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.081 INFO:teuthology.orchestra.run.smithi119.stdout: "log_refresh_metadata": { 2024-03-28T03:44:40.081 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_refresh_metadata", 2024-03-28T03:44:40.082 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.082 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.082 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.082 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:40.082 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.082 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.082 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.082 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-03-28T03:44:40.083 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.083 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.083 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.083 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.083 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2024-03-28T03:44:40.083 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2024-03-28T03:44:40.083 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.083 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.083 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.084 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2024-03-28T03:44:40.084 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.084 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.084 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.084 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-03-28T03:44:40.084 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.084 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.084 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.084 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.085 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2024-03-28T03:44:40.085 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2024-03-28T03:44:40.085 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.085 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.085 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.085 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2024-03-28T03:44:40.085 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.085 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.086 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:40.086 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:40.086 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:40.086 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:40.086 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:40.086 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:40.086 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:40.086 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:40.087 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.087 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.087 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.087 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.087 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.087 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2024-03-28T03:44:40.087 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2024-03-28T03:44:40.087 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.088 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.088 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.088 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:40.088 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.088 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.088 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.088 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.088 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.088 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.089 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.089 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.089 INFO:teuthology.orchestra.run.smithi119.stdout: "manage_etc_ceph_ceph_conf": { 2024-03-28T03:44:40.089 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-03-28T03:44:40.089 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.089 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.089 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.089 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:40.090 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.090 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.090 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.090 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-03-28T03:44:40.090 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.090 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.090 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.090 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.091 INFO:teuthology.orchestra.run.smithi119.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-03-28T03:44:40.091 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-03-28T03:44:40.091 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.091 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.091 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.091 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "*", 2024-03-28T03:44:40.091 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.091 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.092 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.092 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-03-28T03:44:40.092 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.092 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.092 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.092 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.092 INFO:teuthology.orchestra.run.smithi119.stdout: "max_count_per_host": { 2024-03-28T03:44:40.092 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "max_count_per_host", 2024-03-28T03:44:40.093 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2024-03-28T03:44:40.093 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.093 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.093 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "10", 2024-03-28T03:44:40.093 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.093 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.093 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.093 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "max number of daemons per service per host", 2024-03-28T03:44:40.093 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.093 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.094 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.094 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.094 INFO:teuthology.orchestra.run.smithi119.stdout: "max_osd_draining_count": { 2024-03-28T03:44:40.094 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "max_osd_draining_count", 2024-03-28T03:44:40.094 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2024-03-28T03:44:40.094 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.094 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.094 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "10", 2024-03-28T03:44:40.095 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.095 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.095 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.095 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-03-28T03:44:40.095 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.095 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.095 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.095 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.096 INFO:teuthology.orchestra.run.smithi119.stdout: "migration_current": { 2024-03-28T03:44:40.096 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "migration_current", 2024-03-28T03:44:40.096 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2024-03-28T03:44:40.096 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.096 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.096 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:40.096 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.096 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.096 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.097 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "internal - do not modify", 2024-03-28T03:44:40.097 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.097 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.097 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.097 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.097 INFO:teuthology.orchestra.run.smithi119.stdout: "mode": { 2024-03-28T03:44:40.097 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "mode", 2024-03-28T03:44:40.097 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.097 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.098 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.098 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "root", 2024-03-28T03:44:40.098 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.098 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.098 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:40.098 INFO:teuthology.orchestra.run.smithi119.stdout: "cephadm-package", 2024-03-28T03:44:40.098 INFO:teuthology.orchestra.run.smithi119.stdout: "root" 2024-03-28T03:44:40.098 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:40.098 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "mode for remote execution of cephadm", 2024-03-28T03:44:40.099 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.099 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.099 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.099 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.099 INFO:teuthology.orchestra.run.smithi119.stdout: "oob_default_addr": { 2024-03-28T03:44:40.099 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "oob_default_addr", 2024-03-28T03:44:40.099 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.099 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.099 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.100 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "169.254.1.1", 2024-03-28T03:44:40.100 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.100 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.100 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.100 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Default address for RedFish API (oob management).", 2024-03-28T03:44:40.100 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.100 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.100 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.100 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.101 INFO:teuthology.orchestra.run.smithi119.stdout: "prometheus_alerts_path": { 2024-03-28T03:44:40.101 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "prometheus_alerts_path", 2024-03-28T03:44:40.101 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.101 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.101 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.101 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-03-28T03:44:40.101 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.101 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.101 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.102 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-03-28T03:44:40.102 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.102 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.102 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.102 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.102 INFO:teuthology.orchestra.run.smithi119.stdout: "registry_insecure": { 2024-03-28T03:44:40.102 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "registry_insecure", 2024-03-28T03:44:40.102 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.103 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.103 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.103 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:40.103 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.103 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.103 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.103 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-03-28T03:44:40.103 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.104 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.104 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.104 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.104 INFO:teuthology.orchestra.run.smithi119.stdout: "registry_password": { 2024-03-28T03:44:40.104 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "registry_password", 2024-03-28T03:44:40.104 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.104 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.104 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.104 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:40.105 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.105 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.105 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.105 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-03-28T03:44:40.105 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.105 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.105 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.105 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.105 INFO:teuthology.orchestra.run.smithi119.stdout: "registry_url": { 2024-03-28T03:44:40.106 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "registry_url", 2024-03-28T03:44:40.106 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.106 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.106 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.106 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:40.106 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.106 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.107 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.107 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-03-28T03:44:40.107 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.107 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.107 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.107 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.107 INFO:teuthology.orchestra.run.smithi119.stdout: "registry_username": { 2024-03-28T03:44:40.107 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "registry_username", 2024-03-28T03:44:40.107 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.108 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.108 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.108 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:40.108 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.108 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.108 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.108 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-03-28T03:44:40.108 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.109 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.109 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.109 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.109 INFO:teuthology.orchestra.run.smithi119.stdout: "secure_monitoring_stack": { 2024-03-28T03:44:40.109 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "secure_monitoring_stack", 2024-03-28T03:44:40.109 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.109 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.109 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.109 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:40.110 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.110 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.110 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.110 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-03-28T03:44:40.110 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.110 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.110 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.110 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.111 INFO:teuthology.orchestra.run.smithi119.stdout: "service_discovery_port": { 2024-03-28T03:44:40.111 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "service_discovery_port", 2024-03-28T03:44:40.111 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2024-03-28T03:44:40.111 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.111 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.111 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "8765", 2024-03-28T03:44:40.111 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.111 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.111 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.112 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "cephadm service discovery port", 2024-03-28T03:44:40.112 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.112 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.112 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.112 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.112 INFO:teuthology.orchestra.run.smithi119.stdout: "ssh_config_file": { 2024-03-28T03:44:40.112 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "ssh_config_file", 2024-03-28T03:44:40.112 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.113 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.113 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.113 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:40.113 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.113 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.113 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.113 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-03-28T03:44:40.113 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.113 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.114 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.114 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.114 INFO:teuthology.orchestra.run.smithi119.stdout: "use_agent": { 2024-03-28T03:44:40.114 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "use_agent", 2024-03-28T03:44:40.114 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.114 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.114 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.114 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:40.115 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.115 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.115 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.115 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-03-28T03:44:40.115 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.115 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.115 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.115 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.115 INFO:teuthology.orchestra.run.smithi119.stdout: "use_repo_digest": { 2024-03-28T03:44:40.116 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "use_repo_digest", 2024-03-28T03:44:40.116 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.116 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.116 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.116 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2024-03-28T03:44:40.116 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.116 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.116 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.117 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-03-28T03:44:40.117 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.117 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.117 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.117 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.117 INFO:teuthology.orchestra.run.smithi119.stdout: "warn_on_failed_host_check": { 2024-03-28T03:44:40.117 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "warn_on_failed_host_check", 2024-03-28T03:44:40.117 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.117 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.118 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.118 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2024-03-28T03:44:40.118 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.118 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.118 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.118 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "raise a health warning if the host check fails", 2024-03-28T03:44:40.118 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.118 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.118 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.119 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.119 INFO:teuthology.orchestra.run.smithi119.stdout: "warn_on_stray_daemons": { 2024-03-28T03:44:40.119 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "warn_on_stray_daemons", 2024-03-28T03:44:40.119 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.119 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.119 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.119 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2024-03-28T03:44:40.119 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.120 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.120 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.120 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-03-28T03:44:40.120 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.120 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.120 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.120 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.120 INFO:teuthology.orchestra.run.smithi119.stdout: "warn_on_stray_hosts": { 2024-03-28T03:44:40.120 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "warn_on_stray_hosts", 2024-03-28T03:44:40.121 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.121 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.121 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.121 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2024-03-28T03:44:40.121 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.121 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.121 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.121 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-03-28T03:44:40.121 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.122 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.122 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.122 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:40.122 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:40.122 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.122 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-03-28T03:44:40.122 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "crash", 2024-03-28T03:44:40.122 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2024-03-28T03:44:40.123 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2024-03-28T03:44:40.123 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2024-03-28T03:44:40.123 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2024-03-28T03:44:40.123 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2024-03-28T03:44:40.123 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.123 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.123 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.123 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:40.123 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.124 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.124 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:40.124 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:40.124 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:40.124 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:40.124 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:40.124 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:40.124 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:40.125 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:40.125 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.125 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.125 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.125 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.125 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.125 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2024-03-28T03:44:40.125 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2024-03-28T03:44:40.125 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.126 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.126 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.126 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:40.126 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.126 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.126 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.126 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.126 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.127 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.127 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.127 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.127 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2024-03-28T03:44:40.127 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2024-03-28T03:44:40.127 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.127 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.127 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.128 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2024-03-28T03:44:40.128 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.128 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.128 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:40.128 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:40.128 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:40.128 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:40.128 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:40.128 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:40.129 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:40.129 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:40.129 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.129 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.129 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.129 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.129 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.129 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2024-03-28T03:44:40.129 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2024-03-28T03:44:40.130 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.130 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.130 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.130 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:40.130 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.130 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.130 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.130 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.131 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.131 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.131 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.131 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.131 INFO:teuthology.orchestra.run.smithi119.stdout: "retain_interval": { 2024-03-28T03:44:40.131 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "retain_interval", 2024-03-28T03:44:40.131 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "secs", 2024-03-28T03:44:40.131 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.131 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.132 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "31536000", 2024-03-28T03:44:40.132 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.132 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.132 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.132 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "how long to retain crashes before pruning them", 2024-03-28T03:44:40.132 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.132 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.132 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.132 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.133 INFO:teuthology.orchestra.run.smithi119.stdout: "warn_recent_interval": { 2024-03-28T03:44:40.133 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "warn_recent_interval", 2024-03-28T03:44:40.133 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "secs", 2024-03-28T03:44:40.133 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.133 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.133 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "1209600", 2024-03-28T03:44:40.133 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.133 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.133 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.134 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "time interval in which to warn about recent crashes", 2024-03-28T03:44:40.134 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.134 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.134 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.134 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:40.134 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:40.134 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.134 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-03-28T03:44:40.135 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "dashboard", 2024-03-28T03:44:40.135 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2024-03-28T03:44:40.135 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2024-03-28T03:44:40.135 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2024-03-28T03:44:40.135 INFO:teuthology.orchestra.run.smithi119.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-03-28T03:44:40.135 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-03-28T03:44:40.135 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2024-03-28T03:44:40.135 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.136 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.136 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "10", 2024-03-28T03:44:40.136 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.136 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.136 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.136 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.136 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.136 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.136 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.137 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.137 INFO:teuthology.orchestra.run.smithi119.stdout: "ALERTMANAGER_API_HOST": { 2024-03-28T03:44:40.137 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "ALERTMANAGER_API_HOST", 2024-03-28T03:44:40.137 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.137 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.137 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.137 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:40.137 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.137 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.138 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.138 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.138 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.138 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.138 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.138 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.138 INFO:teuthology.orchestra.run.smithi119.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-03-28T03:44:40.138 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-03-28T03:44:40.139 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.139 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.139 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.139 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2024-03-28T03:44:40.139 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.139 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.139 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.139 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.139 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.140 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.140 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.140 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.140 INFO:teuthology.orchestra.run.smithi119.stdout: "AUDIT_API_ENABLED": { 2024-03-28T03:44:40.140 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "AUDIT_API_ENABLED", 2024-03-28T03:44:40.140 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.140 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.140 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.140 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:40.141 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.141 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.141 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.141 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.141 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.141 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.141 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.141 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.142 INFO:teuthology.orchestra.run.smithi119.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-03-28T03:44:40.142 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-03-28T03:44:40.142 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.142 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.142 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.142 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2024-03-28T03:44:40.142 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.142 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.142 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.143 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.143 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.143 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.143 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.143 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.143 INFO:teuthology.orchestra.run.smithi119.stdout: "ENABLE_BROWSABLE_API": { 2024-03-28T03:44:40.143 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "ENABLE_BROWSABLE_API", 2024-03-28T03:44:40.143 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.144 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.144 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.144 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2024-03-28T03:44:40.144 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.144 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.144 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.144 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.144 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.144 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.145 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.145 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.145 INFO:teuthology.orchestra.run.smithi119.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-03-28T03:44:40.145 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-03-28T03:44:40.145 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.145 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.145 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.145 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2024-03-28T03:44:40.146 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.146 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.146 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.146 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.146 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.146 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.146 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.146 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.147 INFO:teuthology.orchestra.run.smithi119.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-03-28T03:44:40.147 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-03-28T03:44:40.147 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.147 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.147 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.147 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2024-03-28T03:44:40.147 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.147 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.147 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.148 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.148 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.148 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.148 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.148 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.148 INFO:teuthology.orchestra.run.smithi119.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-03-28T03:44:40.148 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-03-28T03:44:40.148 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.149 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.149 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.149 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2024-03-28T03:44:40.149 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.149 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.149 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.149 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.149 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.150 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.150 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.150 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.150 INFO:teuthology.orchestra.run.smithi119.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-03-28T03:44:40.150 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-03-28T03:44:40.150 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.150 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.150 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.150 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2024-03-28T03:44:40.151 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.151 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.151 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.151 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.151 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.151 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.151 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.151 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.152 INFO:teuthology.orchestra.run.smithi119.stdout: "FEATURE_TOGGLE_NFS": { 2024-03-28T03:44:40.152 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-03-28T03:44:40.152 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.152 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.152 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.152 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2024-03-28T03:44:40.152 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.152 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.152 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.153 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.153 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.153 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.153 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.153 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.153 INFO:teuthology.orchestra.run.smithi119.stdout: "FEATURE_TOGGLE_RBD": { 2024-03-28T03:44:40.153 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-03-28T03:44:40.153 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.153 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.154 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.154 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2024-03-28T03:44:40.154 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.154 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.154 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.154 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.154 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.154 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.155 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.155 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.155 INFO:teuthology.orchestra.run.smithi119.stdout: "FEATURE_TOGGLE_RGW": { 2024-03-28T03:44:40.155 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-03-28T03:44:40.155 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.155 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.155 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.155 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2024-03-28T03:44:40.155 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.156 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.156 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.156 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.156 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.156 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.156 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.156 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.156 INFO:teuthology.orchestra.run.smithi119.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-03-28T03:44:40.157 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-03-28T03:44:40.157 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.157 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.157 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.157 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:40.157 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.157 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.157 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.157 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.158 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.158 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.158 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.158 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.158 INFO:teuthology.orchestra.run.smithi119.stdout: "GRAFANA_API_PASSWORD": { 2024-03-28T03:44:40.158 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "GRAFANA_API_PASSWORD", 2024-03-28T03:44:40.158 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.158 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.159 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.159 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "admin", 2024-03-28T03:44:40.159 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.159 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.159 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.159 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.159 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.159 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.159 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.160 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.160 INFO:teuthology.orchestra.run.smithi119.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-03-28T03:44:40.160 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-03-28T03:44:40.160 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.160 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.160 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.160 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2024-03-28T03:44:40.160 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.160 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.161 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.161 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.161 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.161 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.161 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.161 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.161 INFO:teuthology.orchestra.run.smithi119.stdout: "GRAFANA_API_URL": { 2024-03-28T03:44:40.161 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "GRAFANA_API_URL", 2024-03-28T03:44:40.162 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.162 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.162 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.162 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:40.162 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.162 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.162 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.162 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.162 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.163 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.163 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.163 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.163 INFO:teuthology.orchestra.run.smithi119.stdout: "GRAFANA_API_USERNAME": { 2024-03-28T03:44:40.163 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "GRAFANA_API_USERNAME", 2024-03-28T03:44:40.163 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.163 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.163 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.164 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "admin", 2024-03-28T03:44:40.164 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.164 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.164 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.164 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.164 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.164 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.164 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.164 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.165 INFO:teuthology.orchestra.run.smithi119.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-03-28T03:44:40.165 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-03-28T03:44:40.165 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.165 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.165 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.165 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:40.165 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.165 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.166 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.166 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.166 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.166 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.166 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.166 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.166 INFO:teuthology.orchestra.run.smithi119.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-03-28T03:44:40.166 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-03-28T03:44:40.166 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.167 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.167 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.167 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:40.167 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.167 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.167 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.167 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.167 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.168 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.168 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.168 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.168 INFO:teuthology.orchestra.run.smithi119.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-03-28T03:44:40.168 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-03-28T03:44:40.168 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.168 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.168 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.168 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2024-03-28T03:44:40.169 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.169 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.169 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.169 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.169 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.169 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.169 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.169 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.169 INFO:teuthology.orchestra.run.smithi119.stdout: "ISSUE_TRACKER_API_KEY": { 2024-03-28T03:44:40.170 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-03-28T03:44:40.170 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.170 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.170 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.170 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:40.170 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.170 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.171 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.171 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.171 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.171 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.171 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.171 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.171 INFO:teuthology.orchestra.run.smithi119.stdout: "MULTICLUSTER_CONFIG": { 2024-03-28T03:44:40.171 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "MULTICLUSTER_CONFIG", 2024-03-28T03:44:40.171 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.172 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.172 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.172 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "{}", 2024-03-28T03:44:40.172 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.172 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.172 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.172 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.172 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.172 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.173 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.173 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.173 INFO:teuthology.orchestra.run.smithi119.stdout: "PROMETHEUS_API_HOST": { 2024-03-28T03:44:40.173 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "PROMETHEUS_API_HOST", 2024-03-28T03:44:40.173 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.173 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.173 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.173 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:40.174 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.174 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.174 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.174 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.174 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.174 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.174 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.174 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.174 INFO:teuthology.orchestra.run.smithi119.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-03-28T03:44:40.175 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-03-28T03:44:40.175 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.175 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.175 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.175 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2024-03-28T03:44:40.175 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.175 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.175 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.175 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.176 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.176 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.176 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.176 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.176 INFO:teuthology.orchestra.run.smithi119.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-03-28T03:44:40.176 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-03-28T03:44:40.176 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.176 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.177 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.177 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:40.177 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.177 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.177 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.177 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.177 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.177 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.177 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.178 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.178 INFO:teuthology.orchestra.run.smithi119.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-03-28T03:44:40.178 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-03-28T03:44:40.178 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.178 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.178 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.178 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:40.178 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.179 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.179 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.179 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.179 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.179 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.179 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.179 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.179 INFO:teuthology.orchestra.run.smithi119.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-03-28T03:44:40.179 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-03-28T03:44:40.180 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.180 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.180 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.180 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2024-03-28T03:44:40.180 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.180 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.180 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.180 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.181 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.181 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.181 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.181 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.181 INFO:teuthology.orchestra.run.smithi119.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-03-28T03:44:40.181 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-03-28T03:44:40.181 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.181 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.181 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.182 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2024-03-28T03:44:40.182 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.182 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.182 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.182 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.182 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.182 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.182 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.183 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.183 INFO:teuthology.orchestra.run.smithi119.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-03-28T03:44:40.183 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-03-28T03:44:40.183 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.183 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.183 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.183 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:40.183 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.184 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.184 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.184 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.184 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.184 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.184 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.184 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.184 INFO:teuthology.orchestra.run.smithi119.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-03-28T03:44:40.184 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-03-28T03:44:40.185 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.185 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.185 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.185 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:40.185 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.185 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.185 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.185 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.185 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.186 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.186 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.186 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.186 INFO:teuthology.orchestra.run.smithi119.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-03-28T03:44:40.186 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-03-28T03:44:40.186 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.186 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.186 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.187 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:40.187 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.187 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.187 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.187 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.187 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.187 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.187 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.187 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.188 INFO:teuthology.orchestra.run.smithi119.stdout: "PWD_POLICY_ENABLED": { 2024-03-28T03:44:40.188 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "PWD_POLICY_ENABLED", 2024-03-28T03:44:40.188 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.188 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.188 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.188 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2024-03-28T03:44:40.188 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.188 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.189 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.189 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.189 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.189 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.189 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.189 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.189 INFO:teuthology.orchestra.run.smithi119.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-03-28T03:44:40.189 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-03-28T03:44:40.189 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.190 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.190 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.190 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-03-28T03:44:40.190 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.190 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.190 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.190 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.190 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.191 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.191 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.191 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.191 INFO:teuthology.orchestra.run.smithi119.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-03-28T03:44:40.191 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-03-28T03:44:40.191 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2024-03-28T03:44:40.191 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.191 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.192 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "10", 2024-03-28T03:44:40.192 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.192 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.192 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.192 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.192 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.192 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.192 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.192 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.193 INFO:teuthology.orchestra.run.smithi119.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-03-28T03:44:40.193 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-03-28T03:44:40.193 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2024-03-28T03:44:40.193 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.193 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.193 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "8", 2024-03-28T03:44:40.193 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.193 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.193 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.194 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.194 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.194 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.194 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.194 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.194 INFO:teuthology.orchestra.run.smithi119.stdout: "REST_REQUESTS_TIMEOUT": { 2024-03-28T03:44:40.194 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-03-28T03:44:40.194 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2024-03-28T03:44:40.195 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.195 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.195 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "45", 2024-03-28T03:44:40.195 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.195 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.195 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.195 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.195 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.196 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.196 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.196 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.196 INFO:teuthology.orchestra.run.smithi119.stdout: "RGW_API_ACCESS_KEY": { 2024-03-28T03:44:40.196 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "RGW_API_ACCESS_KEY", 2024-03-28T03:44:40.196 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.196 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.196 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.196 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:40.197 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.197 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.197 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.197 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.197 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.197 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.197 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.197 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.198 INFO:teuthology.orchestra.run.smithi119.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-03-28T03:44:40.198 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-03-28T03:44:40.198 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.198 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.198 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.198 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "admin", 2024-03-28T03:44:40.198 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.198 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.198 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.199 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.199 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.199 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.199 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.199 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.199 INFO:teuthology.orchestra.run.smithi119.stdout: "RGW_API_SECRET_KEY": { 2024-03-28T03:44:40.199 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "RGW_API_SECRET_KEY", 2024-03-28T03:44:40.199 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.199 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.200 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.200 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:40.200 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.200 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.200 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.200 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.200 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.200 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.200 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.201 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.201 INFO:teuthology.orchestra.run.smithi119.stdout: "RGW_API_SSL_VERIFY": { 2024-03-28T03:44:40.201 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "RGW_API_SSL_VERIFY", 2024-03-28T03:44:40.201 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.201 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.201 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.201 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2024-03-28T03:44:40.201 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.202 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.202 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.202 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.202 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.202 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.202 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.202 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.202 INFO:teuthology.orchestra.run.smithi119.stdout: "UNSAFE_TLS_v1_2": { 2024-03-28T03:44:40.203 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "UNSAFE_TLS_v1_2", 2024-03-28T03:44:40.203 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.203 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.203 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.203 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:40.203 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.203 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.203 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.203 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.204 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.204 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.204 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.204 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.204 INFO:teuthology.orchestra.run.smithi119.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-03-28T03:44:40.204 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-03-28T03:44:40.204 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2024-03-28T03:44:40.204 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.204 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.205 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "0", 2024-03-28T03:44:40.205 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.205 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.205 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.205 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.205 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.205 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.205 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.205 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.206 INFO:teuthology.orchestra.run.smithi119.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-03-28T03:44:40.206 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-03-28T03:44:40.206 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2024-03-28T03:44:40.206 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.206 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.206 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "10", 2024-03-28T03:44:40.206 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.206 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.207 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.207 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.207 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.207 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.207 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.207 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.207 INFO:teuthology.orchestra.run.smithi119.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-03-28T03:44:40.207 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-03-28T03:44:40.207 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2024-03-28T03:44:40.208 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.208 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.208 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "5", 2024-03-28T03:44:40.208 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.208 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.208 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.208 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.208 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.209 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.209 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.209 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.209 INFO:teuthology.orchestra.run.smithi119.stdout: "cross_origin_url": { 2024-03-28T03:44:40.209 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "cross_origin_url", 2024-03-28T03:44:40.209 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.209 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.209 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.209 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:40.210 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.210 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.210 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.210 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.210 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.210 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.210 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.210 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.211 INFO:teuthology.orchestra.run.smithi119.stdout: "crt_file": { 2024-03-28T03:44:40.211 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "crt_file", 2024-03-28T03:44:40.211 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.211 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.211 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.211 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:40.211 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.211 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.211 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.212 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.212 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.212 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.212 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.212 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.212 INFO:teuthology.orchestra.run.smithi119.stdout: "debug": { 2024-03-28T03:44:40.212 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "debug", 2024-03-28T03:44:40.212 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.212 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.213 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.213 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:40.213 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.213 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.213 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.213 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Enable/disable debug options", 2024-03-28T03:44:40.213 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.213 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.214 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.214 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.214 INFO:teuthology.orchestra.run.smithi119.stdout: "jwt_token_ttl": { 2024-03-28T03:44:40.214 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "jwt_token_ttl", 2024-03-28T03:44:40.214 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2024-03-28T03:44:40.214 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.214 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.214 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "28800", 2024-03-28T03:44:40.215 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.215 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.215 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.215 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.215 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.215 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.215 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.215 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.215 INFO:teuthology.orchestra.run.smithi119.stdout: "key_file": { 2024-03-28T03:44:40.216 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "key_file", 2024-03-28T03:44:40.216 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.216 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.216 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.216 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:40.216 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.216 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.216 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.216 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.217 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.217 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.217 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.217 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.217 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2024-03-28T03:44:40.217 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2024-03-28T03:44:40.217 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.217 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.218 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.218 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:40.218 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.218 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.218 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:40.218 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:40.218 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:40.218 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:40.219 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:40.219 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:40.219 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:40.219 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:40.219 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.219 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.219 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.219 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.219 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.220 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2024-03-28T03:44:40.220 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2024-03-28T03:44:40.220 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.220 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.220 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.220 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:40.220 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.220 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.221 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.221 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.221 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.221 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.221 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.221 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.221 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2024-03-28T03:44:40.221 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2024-03-28T03:44:40.221 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.222 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.222 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.222 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2024-03-28T03:44:40.222 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.222 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.222 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:40.222 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:40.223 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:40.223 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:40.223 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:40.223 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:40.223 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:40.223 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:40.223 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.223 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.223 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.224 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.224 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.224 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2024-03-28T03:44:40.224 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2024-03-28T03:44:40.224 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.224 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.224 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.224 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:40.224 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.225 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.225 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.225 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.225 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.225 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.225 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.225 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.225 INFO:teuthology.orchestra.run.smithi119.stdout: "motd": { 2024-03-28T03:44:40.226 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "motd", 2024-03-28T03:44:40.226 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.226 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.226 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.226 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:40.226 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.226 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.226 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.227 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "The message of the day", 2024-03-28T03:44:40.227 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.227 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.227 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.227 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.227 INFO:teuthology.orchestra.run.smithi119.stdout: "redirect_resolve_ip_addr": { 2024-03-28T03:44:40.227 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "redirect_resolve_ip_addr", 2024-03-28T03:44:40.227 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.228 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.228 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.228 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:40.228 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.228 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.228 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.228 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.228 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.228 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.229 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.229 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.229 INFO:teuthology.orchestra.run.smithi119.stdout: "server_addr": { 2024-03-28T03:44:40.229 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "server_addr", 2024-03-28T03:44:40.229 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.229 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.229 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.229 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "::", 2024-03-28T03:44:40.229 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.230 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.230 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.230 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.230 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.230 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.230 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.230 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.230 INFO:teuthology.orchestra.run.smithi119.stdout: "server_port": { 2024-03-28T03:44:40.231 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "server_port", 2024-03-28T03:44:40.231 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2024-03-28T03:44:40.231 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.231 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.231 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "8080", 2024-03-28T03:44:40.231 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.231 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.231 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.231 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.232 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.232 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.232 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.232 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.232 INFO:teuthology.orchestra.run.smithi119.stdout: "ssl": { 2024-03-28T03:44:40.232 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "ssl", 2024-03-28T03:44:40.232 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.232 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.233 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.233 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2024-03-28T03:44:40.233 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.233 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.233 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.233 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.233 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.233 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.233 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.234 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.234 INFO:teuthology.orchestra.run.smithi119.stdout: "ssl_server_port": { 2024-03-28T03:44:40.234 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "ssl_server_port", 2024-03-28T03:44:40.234 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2024-03-28T03:44:40.234 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.234 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.234 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "8443", 2024-03-28T03:44:40.234 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.235 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.235 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.235 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.235 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.235 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.235 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.235 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.235 INFO:teuthology.orchestra.run.smithi119.stdout: "standby_behaviour": { 2024-03-28T03:44:40.236 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "standby_behaviour", 2024-03-28T03:44:40.236 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.236 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.236 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.236 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "redirect", 2024-03-28T03:44:40.236 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.236 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.236 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:40.236 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:40.237 INFO:teuthology.orchestra.run.smithi119.stdout: "redirect" 2024-03-28T03:44:40.237 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:40.237 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.237 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.237 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.237 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.237 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.237 INFO:teuthology.orchestra.run.smithi119.stdout: "standby_error_status_code": { 2024-03-28T03:44:40.238 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "standby_error_status_code", 2024-03-28T03:44:40.238 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2024-03-28T03:44:40.238 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.238 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.238 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "500", 2024-03-28T03:44:40.238 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "400", 2024-03-28T03:44:40.238 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "599", 2024-03-28T03:44:40.238 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.238 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.239 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.239 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.239 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.239 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.239 INFO:teuthology.orchestra.run.smithi119.stdout: "url_prefix": { 2024-03-28T03:44:40.239 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "url_prefix", 2024-03-28T03:44:40.239 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.239 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.239 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.240 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:40.240 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.240 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.240 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.240 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.240 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.240 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.240 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.240 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:40.241 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:40.241 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.241 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-03-28T03:44:40.241 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "devicehealth", 2024-03-28T03:44:40.241 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2024-03-28T03:44:40.241 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2024-03-28T03:44:40.241 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2024-03-28T03:44:40.241 INFO:teuthology.orchestra.run.smithi119.stdout: "enable_monitoring": { 2024-03-28T03:44:40.242 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "enable_monitoring", 2024-03-28T03:44:40.242 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.242 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.242 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.242 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2024-03-28T03:44:40.242 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.242 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.242 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.243 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "monitor device health metrics", 2024-03-28T03:44:40.243 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.243 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.243 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.243 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.243 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2024-03-28T03:44:40.243 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2024-03-28T03:44:40.243 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.243 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.244 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.244 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:40.244 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.244 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.244 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:40.244 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:40.244 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:40.244 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:40.244 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:40.245 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:40.245 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:40.245 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:40.245 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.245 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.245 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.245 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.245 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.246 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2024-03-28T03:44:40.246 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2024-03-28T03:44:40.246 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.246 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.246 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.246 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:40.246 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.246 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.247 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.247 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.247 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.247 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.247 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.247 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.247 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2024-03-28T03:44:40.247 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2024-03-28T03:44:40.247 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.248 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.248 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.248 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2024-03-28T03:44:40.248 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.248 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.248 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:40.248 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:40.248 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:40.249 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:40.249 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:40.249 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:40.249 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:40.249 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:40.249 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.249 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.249 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.249 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.250 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.250 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2024-03-28T03:44:40.250 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2024-03-28T03:44:40.250 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.250 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.250 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.250 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:40.250 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.251 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.251 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.251 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.251 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.251 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.251 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.251 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.251 INFO:teuthology.orchestra.run.smithi119.stdout: "mark_out_threshold": { 2024-03-28T03:44:40.251 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "mark_out_threshold", 2024-03-28T03:44:40.252 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "secs", 2024-03-28T03:44:40.252 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.252 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.252 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "2419200", 2024-03-28T03:44:40.252 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.252 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.252 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.252 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-03-28T03:44:40.252 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.253 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.253 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.253 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.253 INFO:teuthology.orchestra.run.smithi119.stdout: "pool_name": { 2024-03-28T03:44:40.253 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "pool_name", 2024-03-28T03:44:40.253 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.253 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.253 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.254 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "device_health_metrics", 2024-03-28T03:44:40.254 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.254 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.254 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.254 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "name of pool in which to store device health metrics", 2024-03-28T03:44:40.254 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.254 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.254 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.255 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.255 INFO:teuthology.orchestra.run.smithi119.stdout: "retention_period": { 2024-03-28T03:44:40.255 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "retention_period", 2024-03-28T03:44:40.255 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "secs", 2024-03-28T03:44:40.255 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.255 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.255 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "15552000", 2024-03-28T03:44:40.255 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.255 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.256 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.256 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "how long to retain device health metrics", 2024-03-28T03:44:40.256 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.256 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.256 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.256 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.256 INFO:teuthology.orchestra.run.smithi119.stdout: "scrape_frequency": { 2024-03-28T03:44:40.256 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "scrape_frequency", 2024-03-28T03:44:40.256 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "secs", 2024-03-28T03:44:40.257 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.257 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.257 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "86400", 2024-03-28T03:44:40.257 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.257 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.257 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.257 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "how frequently to scrape device health metrics", 2024-03-28T03:44:40.257 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.258 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.258 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.258 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.258 INFO:teuthology.orchestra.run.smithi119.stdout: "self_heal": { 2024-03-28T03:44:40.258 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "self_heal", 2024-03-28T03:44:40.258 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.258 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.258 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.259 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2024-03-28T03:44:40.259 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.259 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.259 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.259 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-03-28T03:44:40.259 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.259 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.259 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.260 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.260 INFO:teuthology.orchestra.run.smithi119.stdout: "sleep_interval": { 2024-03-28T03:44:40.260 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "sleep_interval", 2024-03-28T03:44:40.260 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "secs", 2024-03-28T03:44:40.260 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.260 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.260 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "600", 2024-03-28T03:44:40.260 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.261 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.261 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.261 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "how frequently to wake up and check device health", 2024-03-28T03:44:40.261 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.261 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.261 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.261 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.261 INFO:teuthology.orchestra.run.smithi119.stdout: "warn_threshold": { 2024-03-28T03:44:40.262 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "warn_threshold", 2024-03-28T03:44:40.262 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "secs", 2024-03-28T03:44:40.262 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.262 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.262 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "7257600", 2024-03-28T03:44:40.262 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.262 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.262 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.263 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-03-28T03:44:40.263 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.263 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.263 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.263 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:40.263 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:40.263 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.263 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-03-28T03:44:40.263 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "diskprediction_local", 2024-03-28T03:44:40.264 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2024-03-28T03:44:40.264 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2024-03-28T03:44:40.264 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2024-03-28T03:44:40.264 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2024-03-28T03:44:40.264 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2024-03-28T03:44:40.264 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.264 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.264 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.265 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:40.265 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.265 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.265 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:40.265 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:40.265 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:40.265 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:40.265 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:40.265 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:40.266 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:40.266 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:40.266 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.266 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.266 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.266 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.266 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.266 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2024-03-28T03:44:40.267 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2024-03-28T03:44:40.267 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.267 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.267 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.267 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:40.267 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.267 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.267 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.267 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.268 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.268 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.268 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.268 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.268 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2024-03-28T03:44:40.268 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2024-03-28T03:44:40.268 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.268 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.269 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.269 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2024-03-28T03:44:40.269 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.269 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.269 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:40.269 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:40.269 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:40.269 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:40.269 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:40.270 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:40.270 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:40.270 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:40.270 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.270 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.270 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.270 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.270 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.270 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2024-03-28T03:44:40.271 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2024-03-28T03:44:40.271 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.271 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.271 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.271 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:40.271 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.271 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.271 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.271 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.272 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.272 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.272 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.272 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.272 INFO:teuthology.orchestra.run.smithi119.stdout: "predict_interval": { 2024-03-28T03:44:40.272 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "predict_interval", 2024-03-28T03:44:40.272 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.272 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.273 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.273 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "86400", 2024-03-28T03:44:40.273 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.273 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.273 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.273 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.273 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.273 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.273 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.274 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.274 INFO:teuthology.orchestra.run.smithi119.stdout: "predictor_model": { 2024-03-28T03:44:40.274 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "predictor_model", 2024-03-28T03:44:40.274 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.274 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.274 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.274 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "prophetstor", 2024-03-28T03:44:40.274 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.275 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.275 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.275 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.275 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.275 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.275 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.275 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.275 INFO:teuthology.orchestra.run.smithi119.stdout: "sleep_interval": { 2024-03-28T03:44:40.275 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "sleep_interval", 2024-03-28T03:44:40.276 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.276 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.276 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.276 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "600", 2024-03-28T03:44:40.276 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.276 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.276 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.276 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.276 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.277 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.277 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.277 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:40.277 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:40.277 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.277 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-03-28T03:44:40.277 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "influx", 2024-03-28T03:44:40.277 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": false, 2024-03-28T03:44:40.277 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "influxdb python module not found", 2024-03-28T03:44:40.278 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2024-03-28T03:44:40.278 INFO:teuthology.orchestra.run.smithi119.stdout: "batch_size": { 2024-03-28T03:44:40.278 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "batch_size", 2024-03-28T03:44:40.278 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2024-03-28T03:44:40.278 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.278 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.278 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "5000", 2024-03-28T03:44:40.278 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.279 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.279 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.279 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-03-28T03:44:40.279 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.279 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.279 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.279 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.279 INFO:teuthology.orchestra.run.smithi119.stdout: "database": { 2024-03-28T03:44:40.279 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "database", 2024-03-28T03:44:40.280 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.280 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.280 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.280 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "ceph", 2024-03-28T03:44:40.280 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.280 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.280 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.280 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-03-28T03:44:40.281 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.281 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.281 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.281 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.281 INFO:teuthology.orchestra.run.smithi119.stdout: "hostname": { 2024-03-28T03:44:40.281 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "hostname", 2024-03-28T03:44:40.281 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.281 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.282 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.282 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:40.282 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.282 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.282 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.282 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "InfluxDB server hostname", 2024-03-28T03:44:40.282 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.282 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.283 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.283 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.283 INFO:teuthology.orchestra.run.smithi119.stdout: "interval": { 2024-03-28T03:44:40.283 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "interval", 2024-03-28T03:44:40.283 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "secs", 2024-03-28T03:44:40.283 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.283 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.283 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "30", 2024-03-28T03:44:40.284 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "5", 2024-03-28T03:44:40.284 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.284 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.284 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-03-28T03:44:40.284 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.284 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.284 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.284 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.284 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2024-03-28T03:44:40.285 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2024-03-28T03:44:40.285 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.285 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.285 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.285 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:40.285 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.285 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.285 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:40.286 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:40.286 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:40.286 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:40.286 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:40.286 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:40.286 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:40.286 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:40.286 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.286 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.287 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.287 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.287 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.287 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2024-03-28T03:44:40.287 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2024-03-28T03:44:40.287 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.287 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.287 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.288 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:40.288 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.288 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.288 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.288 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.288 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.288 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.288 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.289 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.289 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2024-03-28T03:44:40.289 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2024-03-28T03:44:40.289 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.289 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.289 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.289 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2024-03-28T03:44:40.289 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.289 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.290 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:40.290 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:40.290 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:40.290 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:40.290 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:40.290 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:40.290 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:40.290 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:40.291 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.291 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.291 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.291 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.291 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.291 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2024-03-28T03:44:40.291 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2024-03-28T03:44:40.291 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.292 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.292 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.292 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:40.292 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.292 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.292 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.292 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.292 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.292 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.293 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.293 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.293 INFO:teuthology.orchestra.run.smithi119.stdout: "password": { 2024-03-28T03:44:40.293 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "password", 2024-03-28T03:44:40.293 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.293 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.293 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.293 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:40.293 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.294 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.294 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.294 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "password of InfluxDB server user", 2024-03-28T03:44:40.294 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.294 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.294 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.294 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.294 INFO:teuthology.orchestra.run.smithi119.stdout: "port": { 2024-03-28T03:44:40.295 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "port", 2024-03-28T03:44:40.295 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2024-03-28T03:44:40.295 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.295 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.295 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "8086", 2024-03-28T03:44:40.295 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.295 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.295 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.295 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "InfluxDB server port", 2024-03-28T03:44:40.296 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.296 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.296 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.296 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.296 INFO:teuthology.orchestra.run.smithi119.stdout: "ssl": { 2024-03-28T03:44:40.296 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "ssl", 2024-03-28T03:44:40.296 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.296 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.296 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.297 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "false", 2024-03-28T03:44:40.297 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.297 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.297 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.297 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-03-28T03:44:40.297 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.297 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.297 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.298 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.298 INFO:teuthology.orchestra.run.smithi119.stdout: "threads": { 2024-03-28T03:44:40.298 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "threads", 2024-03-28T03:44:40.298 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2024-03-28T03:44:40.298 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.298 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.298 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "5", 2024-03-28T03:44:40.298 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "1", 2024-03-28T03:44:40.299 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "32", 2024-03-28T03:44:40.299 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.299 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-03-28T03:44:40.299 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.299 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.299 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.299 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.299 INFO:teuthology.orchestra.run.smithi119.stdout: "username": { 2024-03-28T03:44:40.299 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "username", 2024-03-28T03:44:40.300 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.300 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.300 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.300 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:40.300 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.300 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.300 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.300 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "username of InfluxDB server user", 2024-03-28T03:44:40.300 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.301 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.301 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.301 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.301 INFO:teuthology.orchestra.run.smithi119.stdout: "verify_ssl": { 2024-03-28T03:44:40.301 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "verify_ssl", 2024-03-28T03:44:40.301 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.301 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.301 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.302 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "true", 2024-03-28T03:44:40.302 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.302 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.302 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.302 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-03-28T03:44:40.302 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.302 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.302 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.302 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:40.303 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:40.303 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.303 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-03-28T03:44:40.303 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "insights", 2024-03-28T03:44:40.303 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2024-03-28T03:44:40.303 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2024-03-28T03:44:40.303 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2024-03-28T03:44:40.303 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2024-03-28T03:44:40.304 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2024-03-28T03:44:40.304 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.304 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.304 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.304 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:40.304 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.304 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.304 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:40.304 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:40.305 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:40.305 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:40.305 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:40.305 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:40.305 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:40.305 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:40.305 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.305 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.305 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.306 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.306 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.306 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2024-03-28T03:44:40.306 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2024-03-28T03:44:40.306 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.306 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.306 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.306 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:40.307 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.307 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.307 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.307 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.307 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.307 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.307 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.307 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.308 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2024-03-28T03:44:40.308 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2024-03-28T03:44:40.308 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.308 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.308 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.308 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2024-03-28T03:44:40.308 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.308 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.308 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:40.309 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:40.309 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:40.309 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:40.309 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:40.309 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:40.309 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:40.309 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:40.309 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.310 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.310 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.310 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.310 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.310 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2024-03-28T03:44:40.310 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2024-03-28T03:44:40.310 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.310 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.310 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.311 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:40.311 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.311 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.311 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.311 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.311 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.311 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.311 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.312 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:40.312 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:40.312 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.312 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-03-28T03:44:40.312 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "iostat", 2024-03-28T03:44:40.312 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2024-03-28T03:44:40.312 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2024-03-28T03:44:40.312 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2024-03-28T03:44:40.312 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2024-03-28T03:44:40.313 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2024-03-28T03:44:40.313 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.313 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.313 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.313 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:40.313 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.313 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.313 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:40.313 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:40.314 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:40.314 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:40.314 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:40.314 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:40.314 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:40.314 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:40.314 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.314 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.315 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.315 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.315 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.315 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2024-03-28T03:44:40.315 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2024-03-28T03:44:40.315 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.315 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.315 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.315 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:40.316 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.316 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.316 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.316 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.316 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.316 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.316 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.316 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.316 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2024-03-28T03:44:40.317 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2024-03-28T03:44:40.317 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.317 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.317 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.317 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2024-03-28T03:44:40.317 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.317 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.317 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:40.318 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:40.318 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:40.318 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:40.318 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:40.318 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:40.318 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:40.318 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:40.318 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.318 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.319 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.319 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.319 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.319 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2024-03-28T03:44:40.319 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2024-03-28T03:44:40.319 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.319 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.319 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.320 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:40.320 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.320 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.320 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.320 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.320 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.320 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.320 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.320 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:40.321 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:40.321 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.321 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-03-28T03:44:40.321 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "k8sevents", 2024-03-28T03:44:40.321 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2024-03-28T03:44:40.321 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2024-03-28T03:44:40.321 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2024-03-28T03:44:40.321 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph_event_retention_days": { 2024-03-28T03:44:40.321 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "ceph_event_retention_days", 2024-03-28T03:44:40.322 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2024-03-28T03:44:40.322 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.322 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.322 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "7", 2024-03-28T03:44:40.322 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.322 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.322 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.322 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Days to hold ceph event information within local cache", 2024-03-28T03:44:40.323 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.323 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.323 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.323 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.323 INFO:teuthology.orchestra.run.smithi119.stdout: "config_check_secs": { 2024-03-28T03:44:40.323 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "config_check_secs", 2024-03-28T03:44:40.323 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2024-03-28T03:44:40.323 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.323 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.324 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "10", 2024-03-28T03:44:40.324 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "10", 2024-03-28T03:44:40.324 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.324 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.324 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-03-28T03:44:40.324 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.324 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.324 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.325 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.325 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2024-03-28T03:44:40.325 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2024-03-28T03:44:40.325 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.325 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.325 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.325 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:40.325 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.325 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.326 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:40.326 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:40.326 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:40.326 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:40.326 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:40.326 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:40.326 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:40.326 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:40.327 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.327 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.327 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.327 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.327 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.327 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2024-03-28T03:44:40.327 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2024-03-28T03:44:40.327 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.328 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.328 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.328 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:40.328 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.328 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.328 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.328 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.328 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.328 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.329 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.329 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.329 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2024-03-28T03:44:40.329 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2024-03-28T03:44:40.329 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.329 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.329 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.329 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2024-03-28T03:44:40.330 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.330 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.330 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:40.330 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:40.330 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:40.330 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:40.330 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:40.330 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:40.330 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:40.331 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:40.331 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.331 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.331 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.331 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.331 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.331 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2024-03-28T03:44:40.331 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2024-03-28T03:44:40.332 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.332 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.332 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.332 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:40.332 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.332 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.332 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.332 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.332 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.333 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.333 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.333 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:40.333 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:40.333 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.333 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-03-28T03:44:40.333 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "localpool", 2024-03-28T03:44:40.333 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2024-03-28T03:44:40.334 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2024-03-28T03:44:40.334 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2024-03-28T03:44:40.334 INFO:teuthology.orchestra.run.smithi119.stdout: "failure_domain": { 2024-03-28T03:44:40.334 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "failure_domain", 2024-03-28T03:44:40.334 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.334 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.334 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.334 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "host", 2024-03-28T03:44:40.334 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.335 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.335 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.335 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "failure domain for any created local pool", 2024-03-28T03:44:40.335 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-03-28T03:44:40.335 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.335 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.335 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.335 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2024-03-28T03:44:40.336 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2024-03-28T03:44:40.336 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.336 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.336 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.336 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:40.336 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.336 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.336 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:40.337 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:40.337 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:40.337 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:40.337 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:40.337 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:40.337 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:40.337 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:40.337 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.337 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.338 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.338 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.338 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.338 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2024-03-28T03:44:40.338 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2024-03-28T03:44:40.338 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.338 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.338 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.339 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:40.339 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.339 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.339 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.339 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.339 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.339 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.339 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.339 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.340 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2024-03-28T03:44:40.340 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2024-03-28T03:44:40.340 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.340 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.340 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.340 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2024-03-28T03:44:40.340 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.340 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.340 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:40.341 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:40.341 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:40.341 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:40.341 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:40.341 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:40.341 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:40.341 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:40.341 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.341 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.342 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.342 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.342 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.342 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2024-03-28T03:44:40.342 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2024-03-28T03:44:40.342 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.342 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.342 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.343 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:40.343 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.343 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.343 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.343 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.343 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.343 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.343 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.343 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.344 INFO:teuthology.orchestra.run.smithi119.stdout: "min_size": { 2024-03-28T03:44:40.344 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "min_size", 2024-03-28T03:44:40.344 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2024-03-28T03:44:40.344 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.344 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.344 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:40.344 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.344 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.344 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.345 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "default min_size for any created local pool", 2024-03-28T03:44:40.345 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-03-28T03:44:40.345 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.345 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.345 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.345 INFO:teuthology.orchestra.run.smithi119.stdout: "num_rep": { 2024-03-28T03:44:40.345 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "num_rep", 2024-03-28T03:44:40.345 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2024-03-28T03:44:40.345 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.346 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.346 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "3", 2024-03-28T03:44:40.346 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.346 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.346 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.346 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "default replica count for any created local pool", 2024-03-28T03:44:40.346 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.346 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.347 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.347 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.347 INFO:teuthology.orchestra.run.smithi119.stdout: "pg_num": { 2024-03-28T03:44:40.347 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "pg_num", 2024-03-28T03:44:40.347 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2024-03-28T03:44:40.347 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.347 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.347 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "128", 2024-03-28T03:44:40.348 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.348 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.348 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.348 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "default pg_num for any created local pool", 2024-03-28T03:44:40.348 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.348 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.348 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.348 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.348 INFO:teuthology.orchestra.run.smithi119.stdout: "prefix": { 2024-03-28T03:44:40.349 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "prefix", 2024-03-28T03:44:40.349 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.349 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.349 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.349 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:40.349 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.349 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.349 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.350 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "name prefix for any created local pool", 2024-03-28T03:44:40.350 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.350 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.350 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.350 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.350 INFO:teuthology.orchestra.run.smithi119.stdout: "subtree": { 2024-03-28T03:44:40.350 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "subtree", 2024-03-28T03:44:40.350 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.351 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.351 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.351 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "rack", 2024-03-28T03:44:40.351 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.351 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.351 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.351 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "CRUSH level for which to create a local pool", 2024-03-28T03:44:40.351 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-03-28T03:44:40.351 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.352 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.352 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:40.352 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:40.352 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.352 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-03-28T03:44:40.352 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "mds_autoscaler", 2024-03-28T03:44:40.352 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2024-03-28T03:44:40.352 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2024-03-28T03:44:40.352 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2024-03-28T03:44:40.353 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2024-03-28T03:44:40.353 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2024-03-28T03:44:40.353 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.353 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.353 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.353 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:40.353 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.353 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.354 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:40.354 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:40.354 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:40.354 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:40.354 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:40.354 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:40.354 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:40.354 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:40.354 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.355 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.355 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.355 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.355 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.355 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2024-03-28T03:44:40.355 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2024-03-28T03:44:40.355 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.355 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.355 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.356 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:40.356 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.356 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.356 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.356 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.356 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.356 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.356 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.357 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.357 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2024-03-28T03:44:40.357 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2024-03-28T03:44:40.357 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.357 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.357 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.357 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2024-03-28T03:44:40.357 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.357 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.358 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:40.358 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:40.358 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:40.358 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:40.358 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:40.358 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:40.358 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:40.358 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:40.359 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.359 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.359 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.359 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.359 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.359 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2024-03-28T03:44:40.359 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2024-03-28T03:44:40.359 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.360 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.360 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.360 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:40.360 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.360 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.360 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.360 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.360 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.360 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.361 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.361 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:40.361 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:40.361 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.361 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-03-28T03:44:40.361 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "mirroring", 2024-03-28T03:44:40.361 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2024-03-28T03:44:40.361 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2024-03-28T03:44:40.362 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2024-03-28T03:44:40.362 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2024-03-28T03:44:40.362 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2024-03-28T03:44:40.362 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.362 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.362 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.362 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:40.362 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.363 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.363 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:40.363 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:40.363 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:40.363 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:40.363 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:40.363 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:40.363 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:40.364 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:40.364 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.364 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.364 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.364 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.364 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.364 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2024-03-28T03:44:40.364 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2024-03-28T03:44:40.364 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.365 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.365 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.365 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:40.365 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.365 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.365 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.365 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.365 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.365 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.366 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.366 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.366 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2024-03-28T03:44:40.366 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2024-03-28T03:44:40.366 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.366 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.366 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.366 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2024-03-28T03:44:40.367 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.367 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.367 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:40.367 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:40.367 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:40.367 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:40.367 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:40.367 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:40.368 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:40.368 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:40.368 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.368 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.368 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.368 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.368 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.368 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2024-03-28T03:44:40.368 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2024-03-28T03:44:40.369 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.369 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.369 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.369 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:40.369 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.369 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.369 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.369 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.370 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.370 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.370 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.370 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:40.370 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:40.370 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.370 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-03-28T03:44:40.370 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "nfs", 2024-03-28T03:44:40.371 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2024-03-28T03:44:40.371 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2024-03-28T03:44:40.371 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2024-03-28T03:44:40.371 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2024-03-28T03:44:40.371 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2024-03-28T03:44:40.371 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.371 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.371 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.371 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:40.372 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.372 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.372 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:40.372 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:40.372 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:40.372 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:40.372 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:40.372 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:40.372 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:40.373 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:40.373 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.373 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.373 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.373 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.373 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.373 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2024-03-28T03:44:40.373 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2024-03-28T03:44:40.374 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.374 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.374 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.374 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:40.374 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.374 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.374 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.374 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.374 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.375 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.375 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.375 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.375 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2024-03-28T03:44:40.375 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2024-03-28T03:44:40.375 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.375 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.375 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.376 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2024-03-28T03:44:40.376 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.376 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.376 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:40.376 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:40.376 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:40.376 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:40.376 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:40.376 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:40.377 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:40.377 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:40.377 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.377 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.377 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.377 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.377 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.377 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2024-03-28T03:44:40.377 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2024-03-28T03:44:40.378 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.378 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.378 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.378 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:40.378 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.378 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.378 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.378 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.379 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.379 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.379 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.379 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:40.379 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:40.379 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.379 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-03-28T03:44:40.379 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "orchestrator", 2024-03-28T03:44:40.379 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2024-03-28T03:44:40.380 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2024-03-28T03:44:40.380 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2024-03-28T03:44:40.380 INFO:teuthology.orchestra.run.smithi119.stdout: "fail_fs": { 2024-03-28T03:44:40.380 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "fail_fs", 2024-03-28T03:44:40.380 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.380 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.380 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.380 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:40.380 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.381 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.381 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.381 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-03-28T03:44:40.381 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.381 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.381 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.381 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.381 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2024-03-28T03:44:40.381 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2024-03-28T03:44:40.382 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.382 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.382 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.382 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:40.382 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.382 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.382 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:40.382 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:40.383 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:40.383 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:40.383 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:40.383 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:40.383 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:40.383 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:40.383 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.383 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.384 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.384 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.384 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.384 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2024-03-28T03:44:40.384 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2024-03-28T03:44:40.384 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.384 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.384 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.384 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:40.385 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.385 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.385 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.385 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.385 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.385 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.385 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.385 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.386 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2024-03-28T03:44:40.386 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2024-03-28T03:44:40.386 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.386 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.386 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.386 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2024-03-28T03:44:40.386 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.386 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.386 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:40.387 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:40.387 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:40.387 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:40.387 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:40.387 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:40.387 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:40.387 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:40.387 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.388 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.388 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.388 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.388 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.388 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2024-03-28T03:44:40.388 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2024-03-28T03:44:40.388 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.388 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.388 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.389 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:40.389 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.389 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.389 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.389 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.389 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.389 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.389 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.389 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.390 INFO:teuthology.orchestra.run.smithi119.stdout: "orchestrator": { 2024-03-28T03:44:40.390 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "orchestrator", 2024-03-28T03:44:40.390 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.390 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.390 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.390 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:40.390 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.390 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.391 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:40.391 INFO:teuthology.orchestra.run.smithi119.stdout: "cephadm", 2024-03-28T03:44:40.391 INFO:teuthology.orchestra.run.smithi119.stdout: "rook", 2024-03-28T03:44:40.391 INFO:teuthology.orchestra.run.smithi119.stdout: "test_orchestrator" 2024-03-28T03:44:40.391 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:40.391 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Orchestrator backend", 2024-03-28T03:44:40.391 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.391 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.391 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.392 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:40.392 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:40.392 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.392 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-03-28T03:44:40.392 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "osd_perf_query", 2024-03-28T03:44:40.392 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2024-03-28T03:44:40.392 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2024-03-28T03:44:40.392 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2024-03-28T03:44:40.393 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2024-03-28T03:44:40.393 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2024-03-28T03:44:40.393 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.393 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.393 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.393 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:40.393 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.393 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.394 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:40.394 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:40.394 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:40.394 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:40.394 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:40.394 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:40.394 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:40.394 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:40.395 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.395 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.395 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.395 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.395 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.395 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2024-03-28T03:44:40.395 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2024-03-28T03:44:40.395 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.395 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.396 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.396 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:40.396 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.396 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.396 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.396 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.396 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.396 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.397 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.397 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.397 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2024-03-28T03:44:40.397 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2024-03-28T03:44:40.397 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.397 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.397 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.397 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2024-03-28T03:44:40.397 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.398 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.398 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:40.398 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:40.398 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:40.398 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:40.398 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:40.398 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:40.398 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:40.399 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:40.399 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.399 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.399 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.399 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.399 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.399 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2024-03-28T03:44:40.399 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2024-03-28T03:44:40.399 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.400 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.400 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.400 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:40.400 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.400 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.400 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.400 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.400 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.400 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.401 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.401 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:40.401 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:40.401 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.401 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-03-28T03:44:40.401 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "osd_support", 2024-03-28T03:44:40.401 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2024-03-28T03:44:40.401 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2024-03-28T03:44:40.402 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2024-03-28T03:44:40.402 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2024-03-28T03:44:40.402 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2024-03-28T03:44:40.402 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.402 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.402 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.402 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:40.402 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.403 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.403 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:40.403 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:40.403 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:40.403 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:40.403 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:40.403 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:40.403 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:40.403 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:40.404 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.404 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.404 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.404 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.404 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.404 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2024-03-28T03:44:40.404 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2024-03-28T03:44:40.404 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.404 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.405 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.405 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:40.405 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.405 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.405 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.405 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.405 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.405 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.406 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.406 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.406 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2024-03-28T03:44:40.406 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2024-03-28T03:44:40.406 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.406 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.406 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.406 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2024-03-28T03:44:40.406 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.407 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.407 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:40.407 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:40.407 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:40.407 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:40.407 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:40.407 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:40.407 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:40.407 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:40.408 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.408 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.408 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.408 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.408 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.408 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2024-03-28T03:44:40.408 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2024-03-28T03:44:40.408 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.409 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.409 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.409 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:40.409 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.409 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.409 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.409 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.409 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.410 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.410 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.410 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:40.410 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:40.410 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.410 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-03-28T03:44:40.410 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "pg_autoscaler", 2024-03-28T03:44:40.410 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2024-03-28T03:44:40.410 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2024-03-28T03:44:40.411 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2024-03-28T03:44:40.411 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2024-03-28T03:44:40.411 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2024-03-28T03:44:40.411 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.411 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.411 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.411 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:40.411 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.412 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.412 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:40.412 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:40.412 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:40.412 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:40.412 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:40.412 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:40.412 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:40.412 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:40.413 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.413 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.413 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.413 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.413 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.413 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2024-03-28T03:44:40.413 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2024-03-28T03:44:40.413 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.413 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.414 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.414 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:40.414 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.414 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.414 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.414 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.414 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.414 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.415 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.415 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.415 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2024-03-28T03:44:40.415 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2024-03-28T03:44:40.415 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.415 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.415 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.415 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2024-03-28T03:44:40.415 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.416 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.416 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:40.416 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:40.416 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:40.416 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:40.416 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:40.416 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:40.416 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:40.417 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:40.417 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.417 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.417 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.417 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.417 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.417 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2024-03-28T03:44:40.417 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2024-03-28T03:44:40.418 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.418 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.418 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.418 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:40.418 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.418 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.418 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.418 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.418 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.419 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.419 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.419 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.419 INFO:teuthology.orchestra.run.smithi119.stdout: "sleep_interval": { 2024-03-28T03:44:40.419 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "sleep_interval", 2024-03-28T03:44:40.419 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "secs", 2024-03-28T03:44:40.419 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.419 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.419 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "60", 2024-03-28T03:44:40.420 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.420 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.420 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.420 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.420 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.420 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.420 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.420 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.420 INFO:teuthology.orchestra.run.smithi119.stdout: "threshold": { 2024-03-28T03:44:40.421 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "threshold", 2024-03-28T03:44:40.421 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "float", 2024-03-28T03:44:40.421 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.421 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.421 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "3.0", 2024-03-28T03:44:40.421 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "1.0", 2024-03-28T03:44:40.421 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.421 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.422 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "scaling threshold", 2024-03-28T03:44:40.422 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-03-28T03:44:40.422 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.422 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.422 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:40.422 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:40.422 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.422 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-03-28T03:44:40.423 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "progress", 2024-03-28T03:44:40.423 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2024-03-28T03:44:40.423 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2024-03-28T03:44:40.423 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2024-03-28T03:44:40.423 INFO:teuthology.orchestra.run.smithi119.stdout: "allow_pg_recovery_event": { 2024-03-28T03:44:40.423 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "allow_pg_recovery_event", 2024-03-28T03:44:40.423 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.423 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.424 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.424 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:40.424 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.424 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.424 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.424 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "allow the module to show pg recovery progress", 2024-03-28T03:44:40.424 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.424 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.424 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.425 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.425 INFO:teuthology.orchestra.run.smithi119.stdout: "enabled": { 2024-03-28T03:44:40.425 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "enabled", 2024-03-28T03:44:40.425 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.425 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.425 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.425 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2024-03-28T03:44:40.425 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.425 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.426 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.426 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.426 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.426 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.426 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.426 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.426 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2024-03-28T03:44:40.426 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2024-03-28T03:44:40.427 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.427 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.427 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.427 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:40.427 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.427 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.427 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:40.427 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:40.427 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:40.428 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:40.428 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:40.428 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:40.428 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:40.428 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:40.428 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.428 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.428 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.429 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.429 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.429 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2024-03-28T03:44:40.429 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2024-03-28T03:44:40.429 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.429 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.429 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.429 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:40.429 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.430 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.430 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.430 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.430 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.430 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.430 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.430 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.430 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2024-03-28T03:44:40.431 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2024-03-28T03:44:40.431 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.431 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.431 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.431 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2024-03-28T03:44:40.431 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.431 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.431 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:40.432 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:40.432 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:40.432 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:40.432 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:40.432 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:40.432 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:40.432 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:40.432 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.432 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.433 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.433 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.433 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.433 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2024-03-28T03:44:40.433 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2024-03-28T03:44:40.433 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.433 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.433 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.433 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:40.434 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.434 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.434 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.434 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.434 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.434 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.434 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.434 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.435 INFO:teuthology.orchestra.run.smithi119.stdout: "max_completed_events": { 2024-03-28T03:44:40.435 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "max_completed_events", 2024-03-28T03:44:40.435 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2024-03-28T03:44:40.435 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.435 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.435 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "50", 2024-03-28T03:44:40.435 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.435 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.436 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.436 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "number of past completed events to remember", 2024-03-28T03:44:40.436 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.436 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.436 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.436 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.436 INFO:teuthology.orchestra.run.smithi119.stdout: "sleep_interval": { 2024-03-28T03:44:40.436 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "sleep_interval", 2024-03-28T03:44:40.437 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "secs", 2024-03-28T03:44:40.437 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.437 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.437 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "5", 2024-03-28T03:44:40.437 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.437 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.437 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.437 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "how long the module is going to sleep", 2024-03-28T03:44:40.437 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.438 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.438 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.438 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:40.438 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:40.438 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.438 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-03-28T03:44:40.438 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "prometheus", 2024-03-28T03:44:40.438 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2024-03-28T03:44:40.439 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2024-03-28T03:44:40.439 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2024-03-28T03:44:40.439 INFO:teuthology.orchestra.run.smithi119.stdout: "cache": { 2024-03-28T03:44:40.439 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "cache", 2024-03-28T03:44:40.439 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.439 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.439 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.439 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2024-03-28T03:44:40.440 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.440 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.440 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.440 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.440 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.440 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.440 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.440 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.440 INFO:teuthology.orchestra.run.smithi119.stdout: "exclude_perf_counters": { 2024-03-28T03:44:40.441 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "exclude_perf_counters", 2024-03-28T03:44:40.441 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.441 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.441 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.441 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2024-03-28T03:44:40.441 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.441 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.441 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.441 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-03-28T03:44:40.442 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-03-28T03:44:40.442 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.442 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.442 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.442 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2024-03-28T03:44:40.442 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2024-03-28T03:44:40.442 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.442 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.443 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.443 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:40.443 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.443 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.443 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:40.443 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:40.443 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:40.443 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:40.443 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:40.444 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:40.444 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:40.444 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:40.444 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.444 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.444 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.444 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.444 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.445 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2024-03-28T03:44:40.445 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2024-03-28T03:44:40.445 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.445 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.445 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.445 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:40.445 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.445 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.445 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.446 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.446 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.446 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.446 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.446 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.446 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2024-03-28T03:44:40.446 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2024-03-28T03:44:40.446 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.447 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.447 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.447 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2024-03-28T03:44:40.447 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.447 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.447 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:40.447 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:40.447 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:40.447 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:40.448 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:40.448 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:40.448 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:40.448 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:40.448 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.448 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.448 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.448 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.449 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.449 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2024-03-28T03:44:40.449 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2024-03-28T03:44:40.449 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.449 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.449 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.449 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:40.449 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.449 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.450 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.450 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.450 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.450 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.450 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.450 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.450 INFO:teuthology.orchestra.run.smithi119.stdout: "rbd_stats_pools": { 2024-03-28T03:44:40.450 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "rbd_stats_pools", 2024-03-28T03:44:40.451 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.451 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.451 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.451 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:40.451 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.451 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.451 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.451 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.451 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.452 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.452 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.452 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.452 INFO:teuthology.orchestra.run.smithi119.stdout: "rbd_stats_pools_refresh_interval": { 2024-03-28T03:44:40.452 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-03-28T03:44:40.452 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2024-03-28T03:44:40.452 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.452 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.452 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "300", 2024-03-28T03:44:40.453 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.453 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.453 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.453 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.453 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.453 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.453 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.453 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.453 INFO:teuthology.orchestra.run.smithi119.stdout: "scrape_interval": { 2024-03-28T03:44:40.454 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "scrape_interval", 2024-03-28T03:44:40.454 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "float", 2024-03-28T03:44:40.454 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.454 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.454 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "15.0", 2024-03-28T03:44:40.454 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.454 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.454 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.455 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.455 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.455 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.455 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.455 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.455 INFO:teuthology.orchestra.run.smithi119.stdout: "server_addr": { 2024-03-28T03:44:40.455 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "server_addr", 2024-03-28T03:44:40.455 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.455 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.456 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.456 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "::", 2024-03-28T03:44:40.456 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.456 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.456 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.456 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-03-28T03:44:40.456 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.456 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.456 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.457 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.457 INFO:teuthology.orchestra.run.smithi119.stdout: "server_port": { 2024-03-28T03:44:40.457 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "server_port", 2024-03-28T03:44:40.457 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2024-03-28T03:44:40.457 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.457 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.457 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "9283", 2024-03-28T03:44:40.457 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.458 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.458 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.458 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-03-28T03:44:40.458 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.458 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.458 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.458 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.458 INFO:teuthology.orchestra.run.smithi119.stdout: "stale_cache_strategy": { 2024-03-28T03:44:40.458 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "stale_cache_strategy", 2024-03-28T03:44:40.459 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.459 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.459 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.459 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "log", 2024-03-28T03:44:40.459 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.459 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.459 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.459 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.460 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.460 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.460 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.460 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.460 INFO:teuthology.orchestra.run.smithi119.stdout: "standby_behaviour": { 2024-03-28T03:44:40.460 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "standby_behaviour", 2024-03-28T03:44:40.460 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.460 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.461 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.461 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "default", 2024-03-28T03:44:40.461 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.461 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.461 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:40.461 INFO:teuthology.orchestra.run.smithi119.stdout: "default", 2024-03-28T03:44:40.461 INFO:teuthology.orchestra.run.smithi119.stdout: "error" 2024-03-28T03:44:40.461 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:40.461 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.462 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.462 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.462 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.462 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.462 INFO:teuthology.orchestra.run.smithi119.stdout: "standby_error_status_code": { 2024-03-28T03:44:40.462 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "standby_error_status_code", 2024-03-28T03:44:40.462 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2024-03-28T03:44:40.462 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.463 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.463 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "500", 2024-03-28T03:44:40.463 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "400", 2024-03-28T03:44:40.463 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "599", 2024-03-28T03:44:40.463 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.463 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.463 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.463 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.464 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.464 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:40.464 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:40.464 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.464 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-03-28T03:44:40.464 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "rbd_support", 2024-03-28T03:44:40.464 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2024-03-28T03:44:40.464 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2024-03-28T03:44:40.465 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2024-03-28T03:44:40.465 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2024-03-28T03:44:40.465 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2024-03-28T03:44:40.465 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.465 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.465 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.465 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:40.465 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.465 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.466 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:40.466 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:40.466 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:40.466 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:40.466 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:40.466 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:40.466 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:40.466 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:40.466 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.467 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.467 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.467 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.467 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.467 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2024-03-28T03:44:40.467 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2024-03-28T03:44:40.467 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.467 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.468 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.468 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:40.468 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.468 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.468 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.468 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.468 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.468 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.468 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.469 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.469 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2024-03-28T03:44:40.469 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2024-03-28T03:44:40.469 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.469 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.469 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.469 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2024-03-28T03:44:40.469 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.470 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.470 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:40.470 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:40.470 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:40.470 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:40.470 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:40.470 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:40.470 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:40.471 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:40.471 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.471 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.471 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.471 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.471 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.471 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2024-03-28T03:44:40.471 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2024-03-28T03:44:40.471 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.472 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.472 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.472 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:40.472 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.472 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.472 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.472 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.472 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.473 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.473 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.473 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.473 INFO:teuthology.orchestra.run.smithi119.stdout: "max_concurrent_snap_create": { 2024-03-28T03:44:40.473 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "max_concurrent_snap_create", 2024-03-28T03:44:40.473 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2024-03-28T03:44:40.473 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.473 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.473 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "10", 2024-03-28T03:44:40.474 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.474 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.474 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.474 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.474 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.474 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.474 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.474 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.475 INFO:teuthology.orchestra.run.smithi119.stdout: "mirror_snapshot_schedule": { 2024-03-28T03:44:40.475 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "mirror_snapshot_schedule", 2024-03-28T03:44:40.475 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.475 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.475 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.475 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:40.475 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.475 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.475 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.476 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.476 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.476 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.476 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.476 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.476 INFO:teuthology.orchestra.run.smithi119.stdout: "trash_purge_schedule": { 2024-03-28T03:44:40.476 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "trash_purge_schedule", 2024-03-28T03:44:40.476 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.476 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.477 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.477 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:40.477 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.477 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.477 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.477 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.477 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.477 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.478 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.478 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:40.478 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:40.478 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.478 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-03-28T03:44:40.478 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "restful", 2024-03-28T03:44:40.478 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2024-03-28T03:44:40.478 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2024-03-28T03:44:40.479 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2024-03-28T03:44:40.479 INFO:teuthology.orchestra.run.smithi119.stdout: "enable_auth": { 2024-03-28T03:44:40.479 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "enable_auth", 2024-03-28T03:44:40.479 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.479 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.479 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.479 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2024-03-28T03:44:40.479 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.479 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.480 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.480 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.480 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.480 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.480 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.480 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.480 INFO:teuthology.orchestra.run.smithi119.stdout: "key_file": { 2024-03-28T03:44:40.480 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "key_file", 2024-03-28T03:44:40.481 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.481 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.481 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.481 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:40.481 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.481 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.481 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.481 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.481 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.482 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.482 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.482 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.482 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2024-03-28T03:44:40.482 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2024-03-28T03:44:40.482 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.483 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.483 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.483 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:40.483 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.483 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.483 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:40.483 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:40.483 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:40.484 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:40.484 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:40.484 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:40.484 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:40.484 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:40.484 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.484 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.484 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.485 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.485 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.485 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2024-03-28T03:44:40.485 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2024-03-28T03:44:40.485 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.485 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.485 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.485 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:40.485 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.486 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.486 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.486 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.486 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.486 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.486 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.486 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.486 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2024-03-28T03:44:40.487 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2024-03-28T03:44:40.487 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.487 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.487 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.487 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2024-03-28T03:44:40.487 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.487 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.487 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:40.487 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:40.488 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:40.488 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:40.488 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:40.488 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:40.488 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:40.488 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:40.488 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.488 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.489 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.489 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.489 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.489 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2024-03-28T03:44:40.489 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2024-03-28T03:44:40.489 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.489 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.489 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.489 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:40.490 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.490 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.490 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.490 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.490 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.490 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.490 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.490 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.491 INFO:teuthology.orchestra.run.smithi119.stdout: "server_addr": { 2024-03-28T03:44:40.491 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "server_addr", 2024-03-28T03:44:40.491 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.491 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.491 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.491 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:40.491 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.491 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.491 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.492 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.492 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.492 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.492 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.492 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.492 INFO:teuthology.orchestra.run.smithi119.stdout: "server_port": { 2024-03-28T03:44:40.492 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "server_port", 2024-03-28T03:44:40.492 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.492 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.493 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.493 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:40.493 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.493 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.493 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.493 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.493 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.493 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.494 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.494 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:40.494 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:40.494 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.494 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-03-28T03:44:40.494 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "rgw", 2024-03-28T03:44:40.494 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2024-03-28T03:44:40.494 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2024-03-28T03:44:40.494 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2024-03-28T03:44:40.495 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2024-03-28T03:44:40.495 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2024-03-28T03:44:40.495 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.495 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.495 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.495 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:40.495 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.495 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.496 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:40.496 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:40.496 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:40.496 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:40.496 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:40.496 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:40.496 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:40.496 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:40.496 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.497 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.497 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.497 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.497 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.497 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2024-03-28T03:44:40.497 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2024-03-28T03:44:40.497 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.497 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.498 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.498 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:40.498 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.498 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.498 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.498 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.498 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.498 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.498 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.499 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.499 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2024-03-28T03:44:40.499 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2024-03-28T03:44:40.499 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.499 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.499 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.499 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2024-03-28T03:44:40.499 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.500 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.500 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:40.500 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:40.500 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:40.500 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:40.500 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:40.500 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:40.500 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:40.500 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:40.501 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.501 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.501 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.501 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.501 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.501 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2024-03-28T03:44:40.501 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2024-03-28T03:44:40.501 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.502 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.502 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.502 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:40.502 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.502 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.502 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.502 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.502 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.503 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.503 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.503 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:40.503 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:40.503 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.503 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-03-28T03:44:40.503 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "rook", 2024-03-28T03:44:40.503 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2024-03-28T03:44:40.503 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2024-03-28T03:44:40.504 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2024-03-28T03:44:40.504 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2024-03-28T03:44:40.504 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2024-03-28T03:44:40.504 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.504 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.504 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.504 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:40.504 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.505 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.505 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:40.505 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:40.505 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:40.505 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:40.505 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:40.505 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:40.505 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:40.505 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:40.506 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.506 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.506 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.506 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.506 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.506 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2024-03-28T03:44:40.506 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2024-03-28T03:44:40.506 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.507 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.507 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.507 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:40.507 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.507 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.507 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.507 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.507 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.508 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.508 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.508 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.508 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2024-03-28T03:44:40.508 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2024-03-28T03:44:40.508 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.508 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.508 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.508 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2024-03-28T03:44:40.509 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.509 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.509 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:40.509 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:40.509 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:40.509 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:40.509 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:40.509 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:40.510 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:40.510 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:40.510 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.510 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.510 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.510 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.510 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.510 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2024-03-28T03:44:40.511 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2024-03-28T03:44:40.511 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.511 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.511 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.511 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:40.511 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.511 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.511 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.511 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.512 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.512 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.512 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.512 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.512 INFO:teuthology.orchestra.run.smithi119.stdout: "storage_class": { 2024-03-28T03:44:40.512 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "storage_class", 2024-03-28T03:44:40.512 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.512 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.513 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.513 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "local", 2024-03-28T03:44:40.513 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.513 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.513 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.513 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-03-28T03:44:40.513 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.513 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.514 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.514 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:40.514 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:40.514 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.514 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-03-28T03:44:40.514 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "selftest", 2024-03-28T03:44:40.514 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2024-03-28T03:44:40.514 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2024-03-28T03:44:40.515 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2024-03-28T03:44:40.515 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2024-03-28T03:44:40.515 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2024-03-28T03:44:40.515 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.515 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.515 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.515 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:40.515 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.515 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.516 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:40.516 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:40.516 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:40.516 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:40.516 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:40.516 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:40.516 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:40.516 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:40.516 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.517 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.517 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.517 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.517 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.517 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2024-03-28T03:44:40.517 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2024-03-28T03:44:40.517 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.517 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.518 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.518 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:40.518 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.518 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.518 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.518 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.518 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.518 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.519 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.519 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.519 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2024-03-28T03:44:40.519 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2024-03-28T03:44:40.519 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.519 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.519 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.519 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2024-03-28T03:44:40.519 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.520 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.520 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:40.520 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:40.520 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:40.520 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:40.520 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:40.520 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:40.520 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:40.520 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:40.521 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.521 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.521 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.521 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.521 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.521 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2024-03-28T03:44:40.521 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2024-03-28T03:44:40.521 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.522 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.522 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.522 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:40.522 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.522 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.522 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.522 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.522 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.522 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.523 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.523 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.523 INFO:teuthology.orchestra.run.smithi119.stdout: "roption1": { 2024-03-28T03:44:40.523 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "roption1", 2024-03-28T03:44:40.523 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.523 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.523 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.523 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:40.523 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.524 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.524 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.524 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.524 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.524 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.524 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.524 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.524 INFO:teuthology.orchestra.run.smithi119.stdout: "roption2": { 2024-03-28T03:44:40.525 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "roption2", 2024-03-28T03:44:40.525 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.525 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.525 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.525 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "xyz", 2024-03-28T03:44:40.525 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.525 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.525 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.525 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.526 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.526 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.526 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.526 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.526 INFO:teuthology.orchestra.run.smithi119.stdout: "rwoption1": { 2024-03-28T03:44:40.526 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "rwoption1", 2024-03-28T03:44:40.526 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.526 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.527 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.527 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:40.527 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.527 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.527 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.527 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.527 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.527 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.527 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.528 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.528 INFO:teuthology.orchestra.run.smithi119.stdout: "rwoption2": { 2024-03-28T03:44:40.528 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "rwoption2", 2024-03-28T03:44:40.528 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2024-03-28T03:44:40.528 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.528 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.528 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:40.528 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.528 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.529 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.529 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.529 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.529 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.529 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.529 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.529 INFO:teuthology.orchestra.run.smithi119.stdout: "rwoption3": { 2024-03-28T03:44:40.529 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "rwoption3", 2024-03-28T03:44:40.530 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "float", 2024-03-28T03:44:40.530 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.530 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.530 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:40.530 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.530 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.530 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.530 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.531 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.531 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.531 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.531 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.531 INFO:teuthology.orchestra.run.smithi119.stdout: "rwoption4": { 2024-03-28T03:44:40.531 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "rwoption4", 2024-03-28T03:44:40.531 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.531 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.531 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.532 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:40.532 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.532 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.532 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.532 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.532 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.532 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.532 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.532 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.533 INFO:teuthology.orchestra.run.smithi119.stdout: "rwoption5": { 2024-03-28T03:44:40.533 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "rwoption5", 2024-03-28T03:44:40.533 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.533 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.533 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.533 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:40.533 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.533 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.534 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.534 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.534 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.534 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.534 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.534 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.534 INFO:teuthology.orchestra.run.smithi119.stdout: "rwoption6": { 2024-03-28T03:44:40.534 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "rwoption6", 2024-03-28T03:44:40.535 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.535 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.535 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.535 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2024-03-28T03:44:40.535 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.535 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.535 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.535 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.535 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.536 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.536 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.536 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.536 INFO:teuthology.orchestra.run.smithi119.stdout: "rwoption7": { 2024-03-28T03:44:40.536 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "rwoption7", 2024-03-28T03:44:40.536 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2024-03-28T03:44:40.536 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.536 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.537 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:40.537 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "1", 2024-03-28T03:44:40.537 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "42", 2024-03-28T03:44:40.537 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.537 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.537 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.537 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.537 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.538 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.538 INFO:teuthology.orchestra.run.smithi119.stdout: "testkey": { 2024-03-28T03:44:40.538 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "testkey", 2024-03-28T03:44:40.538 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.538 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.538 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.538 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:40.538 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.539 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.539 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.539 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.539 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.539 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.539 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.539 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.539 INFO:teuthology.orchestra.run.smithi119.stdout: "testlkey": { 2024-03-28T03:44:40.539 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "testlkey", 2024-03-28T03:44:40.540 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.540 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.540 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.540 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:40.540 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.540 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.540 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.540 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.541 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.541 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.541 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.541 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.541 INFO:teuthology.orchestra.run.smithi119.stdout: "testnewline": { 2024-03-28T03:44:40.541 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "testnewline", 2024-03-28T03:44:40.541 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.541 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.541 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.542 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:40.542 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.542 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.542 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.542 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.542 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.542 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.542 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.543 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:40.543 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:40.543 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.543 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-03-28T03:44:40.543 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "snap_schedule", 2024-03-28T03:44:40.543 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2024-03-28T03:44:40.543 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2024-03-28T03:44:40.543 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2024-03-28T03:44:40.543 INFO:teuthology.orchestra.run.smithi119.stdout: "allow_m_granularity": { 2024-03-28T03:44:40.544 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "allow_m_granularity", 2024-03-28T03:44:40.544 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.544 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.544 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.544 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:40.544 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.544 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.544 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.544 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "allow minute scheduled snapshots", 2024-03-28T03:44:40.545 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.545 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.545 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.545 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.545 INFO:teuthology.orchestra.run.smithi119.stdout: "dump_on_update": { 2024-03-28T03:44:40.545 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "dump_on_update", 2024-03-28T03:44:40.545 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.545 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.546 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.546 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:40.546 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.546 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.546 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.546 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "dump database to debug log on update", 2024-03-28T03:44:40.546 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.546 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.547 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.547 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.547 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2024-03-28T03:44:40.547 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2024-03-28T03:44:40.547 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.547 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.547 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.547 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:40.547 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.548 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.548 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:40.548 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:40.548 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:40.548 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:40.548 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:40.548 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:40.548 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:40.548 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:40.549 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.549 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.549 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.549 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.549 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.549 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2024-03-28T03:44:40.549 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2024-03-28T03:44:40.549 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.550 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.550 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.550 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:40.550 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.550 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.550 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.550 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.550 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.550 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.551 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.551 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.551 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2024-03-28T03:44:40.551 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2024-03-28T03:44:40.551 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.551 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.551 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.551 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2024-03-28T03:44:40.552 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.552 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.552 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:40.552 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:40.552 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:40.552 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:40.552 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:40.552 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:40.552 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:40.553 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:40.553 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.553 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.553 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.553 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.553 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.553 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2024-03-28T03:44:40.553 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2024-03-28T03:44:40.553 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.554 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.554 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.554 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:40.554 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.554 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.554 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.554 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.554 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.555 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.555 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.555 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:40.555 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:40.555 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.555 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-03-28T03:44:40.555 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "stats", 2024-03-28T03:44:40.555 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2024-03-28T03:44:40.556 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2024-03-28T03:44:40.556 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2024-03-28T03:44:40.556 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2024-03-28T03:44:40.556 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2024-03-28T03:44:40.556 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.556 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.556 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.556 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:40.556 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.557 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.557 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:40.557 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:40.557 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:40.557 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:40.557 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:40.557 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:40.557 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:40.557 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:40.558 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.558 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.558 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.558 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.558 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.558 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2024-03-28T03:44:40.558 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2024-03-28T03:44:40.558 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.559 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.559 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.559 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:40.559 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.559 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.559 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.559 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.559 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.559 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.560 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.560 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.560 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2024-03-28T03:44:40.560 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2024-03-28T03:44:40.560 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.560 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.560 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.560 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2024-03-28T03:44:40.561 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.561 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.561 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:40.561 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:40.561 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:40.561 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:40.561 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:40.561 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:40.561 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:40.562 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:40.562 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.562 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.562 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.562 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.562 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.562 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2024-03-28T03:44:40.562 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2024-03-28T03:44:40.563 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.563 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.563 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.563 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:40.563 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.563 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.563 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.563 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.564 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.564 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.564 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.564 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:40.564 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:40.564 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.564 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-03-28T03:44:40.564 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "status", 2024-03-28T03:44:40.565 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2024-03-28T03:44:40.565 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2024-03-28T03:44:40.565 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2024-03-28T03:44:40.565 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2024-03-28T03:44:40.565 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2024-03-28T03:44:40.565 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.565 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.565 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.565 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:40.566 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.566 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.566 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:40.566 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:40.566 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:40.566 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:40.566 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:40.566 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:40.567 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:40.567 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:40.567 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.567 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.567 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.567 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.567 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.567 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2024-03-28T03:44:40.567 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2024-03-28T03:44:40.568 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.568 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.568 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.568 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:40.568 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.568 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.568 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.568 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.569 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.569 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.569 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.569 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.569 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2024-03-28T03:44:40.569 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2024-03-28T03:44:40.569 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.569 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.569 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.570 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2024-03-28T03:44:40.570 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.570 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.570 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:40.570 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:40.570 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:40.570 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:40.570 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:40.571 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:40.571 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:40.571 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:40.571 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.571 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.571 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.571 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.571 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.571 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2024-03-28T03:44:40.572 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2024-03-28T03:44:40.572 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.572 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.572 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.572 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:40.572 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.572 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.572 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.573 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.573 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.573 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.573 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.573 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:40.573 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:40.573 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.573 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-03-28T03:44:40.573 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "telegraf", 2024-03-28T03:44:40.574 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2024-03-28T03:44:40.574 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2024-03-28T03:44:40.574 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2024-03-28T03:44:40.574 INFO:teuthology.orchestra.run.smithi119.stdout: "address": { 2024-03-28T03:44:40.574 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "address", 2024-03-28T03:44:40.574 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.574 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.574 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.575 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-03-28T03:44:40.575 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.575 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.575 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.575 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.575 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.575 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.575 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.576 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.576 INFO:teuthology.orchestra.run.smithi119.stdout: "interval": { 2024-03-28T03:44:40.576 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "interval", 2024-03-28T03:44:40.576 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "secs", 2024-03-28T03:44:40.576 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.576 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.576 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "15", 2024-03-28T03:44:40.576 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.577 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.577 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.577 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.577 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.577 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.577 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.577 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.577 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2024-03-28T03:44:40.577 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2024-03-28T03:44:40.578 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.578 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.578 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.578 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:40.578 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.578 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.578 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:40.578 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:40.579 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:40.579 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:40.579 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:40.579 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:40.579 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:40.579 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:40.579 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.579 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.579 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.580 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.580 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.580 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2024-03-28T03:44:40.580 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2024-03-28T03:44:40.580 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.580 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.580 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.580 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:40.580 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.581 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.581 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.581 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.581 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.581 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.581 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.581 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.581 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2024-03-28T03:44:40.582 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2024-03-28T03:44:40.582 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.582 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.582 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.582 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2024-03-28T03:44:40.582 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.582 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.582 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:40.582 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:40.583 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:40.583 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:40.583 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:40.583 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:40.583 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:40.583 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:40.583 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.583 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.584 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.584 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.584 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.584 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2024-03-28T03:44:40.584 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2024-03-28T03:44:40.584 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.584 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.584 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.585 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:40.585 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.585 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.585 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.585 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.585 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.585 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.585 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.585 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:40.586 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:40.586 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.586 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-03-28T03:44:40.586 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "telemetry", 2024-03-28T03:44:40.586 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2024-03-28T03:44:40.586 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2024-03-28T03:44:40.586 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2024-03-28T03:44:40.586 INFO:teuthology.orchestra.run.smithi119.stdout: "channel_basic": { 2024-03-28T03:44:40.586 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "channel_basic", 2024-03-28T03:44:40.587 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.587 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.587 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.587 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2024-03-28T03:44:40.587 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.587 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.587 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.587 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Share basic cluster information (size, version)", 2024-03-28T03:44:40.588 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.588 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.588 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.588 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.588 INFO:teuthology.orchestra.run.smithi119.stdout: "channel_crash": { 2024-03-28T03:44:40.588 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "channel_crash", 2024-03-28T03:44:40.588 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.588 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.588 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.589 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2024-03-28T03:44:40.589 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.589 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.589 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.589 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-03-28T03:44:40.589 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.589 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.589 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.589 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.590 INFO:teuthology.orchestra.run.smithi119.stdout: "channel_device": { 2024-03-28T03:44:40.590 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "channel_device", 2024-03-28T03:44:40.590 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.590 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.590 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.590 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2024-03-28T03:44:40.590 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.590 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.591 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.591 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-03-28T03:44:40.591 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.591 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.591 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.591 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.591 INFO:teuthology.orchestra.run.smithi119.stdout: "channel_ident": { 2024-03-28T03:44:40.591 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "channel_ident", 2024-03-28T03:44:40.592 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.592 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.592 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.592 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:40.592 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.592 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.592 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.592 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-03-28T03:44:40.592 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.593 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.593 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.593 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.593 INFO:teuthology.orchestra.run.smithi119.stdout: "channel_perf": { 2024-03-28T03:44:40.593 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "channel_perf", 2024-03-28T03:44:40.593 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.593 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.593 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.593 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:40.594 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.594 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.594 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.594 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Share various performance metrics of a cluster", 2024-03-28T03:44:40.594 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.594 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.594 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.595 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.595 INFO:teuthology.orchestra.run.smithi119.stdout: "contact": { 2024-03-28T03:44:40.595 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "contact", 2024-03-28T03:44:40.595 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.595 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.595 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.595 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:40.595 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.595 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.596 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.596 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.596 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.596 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.596 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.596 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.596 INFO:teuthology.orchestra.run.smithi119.stdout: "description": { 2024-03-28T03:44:40.596 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "description", 2024-03-28T03:44:40.597 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.597 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.597 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.597 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:40.597 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.597 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.597 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.597 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.598 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.598 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.598 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.598 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.598 INFO:teuthology.orchestra.run.smithi119.stdout: "device_url": { 2024-03-28T03:44:40.598 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "device_url", 2024-03-28T03:44:40.598 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.598 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.598 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.599 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-03-28T03:44:40.599 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.599 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.599 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.599 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.599 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.599 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.599 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.600 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.600 INFO:teuthology.orchestra.run.smithi119.stdout: "enabled": { 2024-03-28T03:44:40.600 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "enabled", 2024-03-28T03:44:40.600 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.600 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.600 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.600 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:40.600 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.600 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.601 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.601 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.601 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.601 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.601 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.601 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.601 INFO:teuthology.orchestra.run.smithi119.stdout: "interval": { 2024-03-28T03:44:40.601 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "interval", 2024-03-28T03:44:40.602 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2024-03-28T03:44:40.602 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.602 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.602 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "24", 2024-03-28T03:44:40.602 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "8", 2024-03-28T03:44:40.602 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.602 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.602 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.603 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.603 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.603 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.603 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.603 INFO:teuthology.orchestra.run.smithi119.stdout: "last_opt_revision": { 2024-03-28T03:44:40.603 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "last_opt_revision", 2024-03-28T03:44:40.603 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2024-03-28T03:44:40.603 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.603 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.604 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "1", 2024-03-28T03:44:40.604 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.604 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.604 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.604 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.604 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.604 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.604 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.605 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.605 INFO:teuthology.orchestra.run.smithi119.stdout: "leaderboard": { 2024-03-28T03:44:40.605 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "leaderboard", 2024-03-28T03:44:40.605 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.605 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.605 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.605 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:40.605 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.605 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.606 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.606 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.606 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.606 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.606 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.606 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.606 INFO:teuthology.orchestra.run.smithi119.stdout: "leaderboard_description": { 2024-03-28T03:44:40.606 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "leaderboard_description", 2024-03-28T03:44:40.607 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.607 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.607 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.607 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:40.607 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.607 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.607 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.607 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.607 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.608 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.608 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.608 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.608 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2024-03-28T03:44:40.608 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2024-03-28T03:44:40.608 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.608 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.608 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.609 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:40.609 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.609 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.609 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:40.609 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:40.609 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:40.609 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:40.609 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:40.610 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:40.610 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:40.610 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:40.610 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.610 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.610 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.610 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.610 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.611 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2024-03-28T03:44:40.611 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2024-03-28T03:44:40.611 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.611 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.611 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.611 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:40.611 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.611 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.611 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.612 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.612 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.612 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.612 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.612 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.612 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2024-03-28T03:44:40.612 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2024-03-28T03:44:40.612 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.613 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.613 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.613 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2024-03-28T03:44:40.613 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.613 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.613 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:40.613 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:40.613 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:40.614 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:40.614 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:40.614 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:40.614 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:40.614 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:40.614 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.614 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.614 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.614 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.615 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.615 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2024-03-28T03:44:40.615 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2024-03-28T03:44:40.615 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.615 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.615 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.615 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:40.615 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.615 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.616 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.616 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.616 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.616 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.616 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.616 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.616 INFO:teuthology.orchestra.run.smithi119.stdout: "organization": { 2024-03-28T03:44:40.616 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "organization", 2024-03-28T03:44:40.617 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.617 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.617 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.617 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:40.617 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.617 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.617 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.617 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.618 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.618 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.618 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.618 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.618 INFO:teuthology.orchestra.run.smithi119.stdout: "proxy": { 2024-03-28T03:44:40.618 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "proxy", 2024-03-28T03:44:40.618 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.618 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.618 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.619 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:40.619 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.619 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.619 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.619 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.619 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.619 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.619 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.619 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.620 INFO:teuthology.orchestra.run.smithi119.stdout: "url": { 2024-03-28T03:44:40.620 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "url", 2024-03-28T03:44:40.620 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.620 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.620 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.620 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-03-28T03:44:40.620 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.620 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.621 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.621 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.621 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.621 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.621 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.621 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:40.621 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:40.621 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.622 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-03-28T03:44:40.622 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "test_orchestrator", 2024-03-28T03:44:40.622 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2024-03-28T03:44:40.622 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2024-03-28T03:44:40.622 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2024-03-28T03:44:40.622 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2024-03-28T03:44:40.622 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2024-03-28T03:44:40.622 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.622 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.623 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.623 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:40.623 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.623 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.623 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:40.623 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:40.623 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:40.623 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:40.624 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:40.624 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:40.624 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:40.624 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:40.624 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.624 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.624 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.624 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.624 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.625 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2024-03-28T03:44:40.625 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2024-03-28T03:44:40.625 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.625 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.625 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.625 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:40.625 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.625 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.626 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.626 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.626 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.626 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.626 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.626 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.626 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2024-03-28T03:44:40.626 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2024-03-28T03:44:40.627 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.627 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.627 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.627 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2024-03-28T03:44:40.627 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.627 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.627 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:40.627 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:40.628 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:40.628 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:40.628 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:40.628 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:40.628 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:40.628 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:40.628 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.628 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.628 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.629 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.629 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.629 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2024-03-28T03:44:40.629 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2024-03-28T03:44:40.629 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.629 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.629 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.629 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:40.630 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.630 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.630 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.630 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.630 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.630 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.630 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.630 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:40.631 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:40.631 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.631 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-03-28T03:44:40.631 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "volumes", 2024-03-28T03:44:40.631 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2024-03-28T03:44:40.631 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2024-03-28T03:44:40.631 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2024-03-28T03:44:40.631 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2024-03-28T03:44:40.631 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2024-03-28T03:44:40.632 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.632 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.632 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.632 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:40.632 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.632 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.632 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:40.632 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:40.633 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:40.633 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:40.633 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:40.633 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:40.633 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:40.633 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:40.633 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.633 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.633 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.634 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.634 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.634 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2024-03-28T03:44:40.634 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2024-03-28T03:44:40.634 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.634 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.634 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.634 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:40.635 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.635 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.635 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.635 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.635 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.635 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.635 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.635 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.635 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2024-03-28T03:44:40.636 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2024-03-28T03:44:40.636 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.636 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.636 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.636 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2024-03-28T03:44:40.636 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.636 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.636 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:40.637 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:40.637 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:40.637 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:40.637 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:40.637 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:40.637 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:40.637 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:40.637 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.638 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.638 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.638 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.638 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.638 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2024-03-28T03:44:40.638 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2024-03-28T03:44:40.638 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.639 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.639 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.639 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:40.639 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.639 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.639 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.639 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.639 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.639 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.640 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.640 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.640 INFO:teuthology.orchestra.run.smithi119.stdout: "max_concurrent_clones": { 2024-03-28T03:44:40.640 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "max_concurrent_clones", 2024-03-28T03:44:40.640 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2024-03-28T03:44:40.640 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.640 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.640 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "4", 2024-03-28T03:44:40.641 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.641 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.641 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.641 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Number of asynchronous cloner threads", 2024-03-28T03:44:40.641 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.641 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.641 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.641 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.641 INFO:teuthology.orchestra.run.smithi119.stdout: "periodic_async_work": { 2024-03-28T03:44:40.642 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "periodic_async_work", 2024-03-28T03:44:40.642 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.642 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.642 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.642 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:40.642 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.642 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.642 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.643 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Periodically check for async work", 2024-03-28T03:44:40.643 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.643 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.643 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.643 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.643 INFO:teuthology.orchestra.run.smithi119.stdout: "snapshot_clone_delay": { 2024-03-28T03:44:40.643 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "snapshot_clone_delay", 2024-03-28T03:44:40.643 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2024-03-28T03:44:40.644 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.644 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.644 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "0", 2024-03-28T03:44:40.644 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.644 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.644 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.644 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-03-28T03:44:40.644 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.645 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.645 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.645 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.645 INFO:teuthology.orchestra.run.smithi119.stdout: "snapshot_clone_no_wait": { 2024-03-28T03:44:40.645 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "snapshot_clone_no_wait", 2024-03-28T03:44:40.645 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.645 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.645 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.646 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2024-03-28T03:44:40.646 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.646 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.646 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.646 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-03-28T03:44:40.646 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.646 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.646 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.647 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:40.647 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:40.647 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.647 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-03-28T03:44:40.647 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "zabbix", 2024-03-28T03:44:40.647 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2024-03-28T03:44:40.647 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2024-03-28T03:44:40.647 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2024-03-28T03:44:40.647 INFO:teuthology.orchestra.run.smithi119.stdout: "discovery_interval": { 2024-03-28T03:44:40.648 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "discovery_interval", 2024-03-28T03:44:40.648 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "uint", 2024-03-28T03:44:40.648 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.648 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.648 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "100", 2024-03-28T03:44:40.648 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.648 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.648 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.649 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.649 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.649 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.649 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.649 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.649 INFO:teuthology.orchestra.run.smithi119.stdout: "identifier": { 2024-03-28T03:44:40.649 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "identifier", 2024-03-28T03:44:40.649 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.649 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.650 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.650 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:40.650 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.650 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.650 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.650 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.650 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.650 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.651 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.651 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.651 INFO:teuthology.orchestra.run.smithi119.stdout: "interval": { 2024-03-28T03:44:40.651 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "interval", 2024-03-28T03:44:40.651 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "secs", 2024-03-28T03:44:40.651 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.651 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.651 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "60", 2024-03-28T03:44:40.651 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.652 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.652 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.652 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.652 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.652 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.652 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.652 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.652 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2024-03-28T03:44:40.653 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2024-03-28T03:44:40.653 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.653 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.653 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.653 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:40.653 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.653 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.653 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:40.653 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:40.654 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:40.654 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:40.654 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:40.654 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:40.654 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:40.654 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:40.654 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.654 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.654 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.655 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.655 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.655 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2024-03-28T03:44:40.655 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2024-03-28T03:44:40.655 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.655 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.655 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.655 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:40.656 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.656 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.656 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.656 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.656 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.656 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.656 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.656 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.656 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2024-03-28T03:44:40.657 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2024-03-28T03:44:40.657 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.657 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.657 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.657 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2024-03-28T03:44:40.657 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.657 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.657 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:40.657 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:40.658 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:40.658 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:40.658 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:40.658 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:40.658 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:40.658 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:40.658 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.658 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.659 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.659 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.659 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.659 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2024-03-28T03:44:40.659 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2024-03-28T03:44:40.659 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.659 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.659 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.659 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:40.660 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.660 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.660 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.660 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.660 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.660 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.660 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.660 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.661 INFO:teuthology.orchestra.run.smithi119.stdout: "zabbix_host": { 2024-03-28T03:44:40.661 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "zabbix_host", 2024-03-28T03:44:40.661 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.661 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.661 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.661 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:40.661 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.661 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.661 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.662 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.662 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.662 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.662 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.662 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.662 INFO:teuthology.orchestra.run.smithi119.stdout: "zabbix_port": { 2024-03-28T03:44:40.662 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "zabbix_port", 2024-03-28T03:44:40.662 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2024-03-28T03:44:40.663 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.663 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.663 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "10051", 2024-03-28T03:44:40.663 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.663 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.663 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.663 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.663 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.663 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.664 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.664 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.664 INFO:teuthology.orchestra.run.smithi119.stdout: "zabbix_sender": { 2024-03-28T03:44:40.664 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "zabbix_sender", 2024-03-28T03:44:40.664 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.664 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.664 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.664 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-03-28T03:44:40.665 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.665 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.665 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.665 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.665 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.665 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.665 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.665 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:40.666 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:40.666 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:40.666 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-03-28T03:44:40.666 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.666 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-03-28T03:44:40.666 INFO:teuthology.orchestra.run.smithi119.stdout: "gid": 25288, 2024-03-28T03:44:40.666 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "z", 2024-03-28T03:44:40.666 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr_features": 4540701547738038271, 2024-03-28T03:44:40.667 INFO:teuthology.orchestra.run.smithi119.stdout: "available_modules": [ 2024-03-28T03:44:40.667 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-03-28T03:44:40.667 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "alerts", 2024-03-28T03:44:40.667 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2024-03-28T03:44:40.667 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2024-03-28T03:44:40.667 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2024-03-28T03:44:40.667 INFO:teuthology.orchestra.run.smithi119.stdout: "interval": { 2024-03-28T03:44:40.667 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "interval", 2024-03-28T03:44:40.667 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "secs", 2024-03-28T03:44:40.668 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.668 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.668 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "60", 2024-03-28T03:44:40.668 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.668 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.668 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.668 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "How frequently to reexamine health status", 2024-03-28T03:44:40.668 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.669 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.669 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.669 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.669 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2024-03-28T03:44:40.669 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2024-03-28T03:44:40.669 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.669 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.669 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.669 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:40.670 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.670 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.670 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:40.670 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:40.670 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:40.670 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:40.670 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:40.670 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:40.671 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:40.671 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:40.671 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.671 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.671 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.671 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.671 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.671 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2024-03-28T03:44:40.672 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2024-03-28T03:44:40.672 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.672 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.672 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.672 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:40.672 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.672 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.672 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.672 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.673 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.673 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.673 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.673 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.673 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2024-03-28T03:44:40.673 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2024-03-28T03:44:40.673 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.673 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.674 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.674 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2024-03-28T03:44:40.674 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.674 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.674 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:40.674 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:40.674 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:40.674 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:40.675 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:40.675 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:40.675 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:40.675 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:40.675 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.675 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.675 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.675 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.675 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.676 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2024-03-28T03:44:40.676 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2024-03-28T03:44:40.676 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.676 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.676 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.676 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:40.676 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.676 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.677 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.677 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.677 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.677 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.677 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.677 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.677 INFO:teuthology.orchestra.run.smithi119.stdout: "smtp_destination": { 2024-03-28T03:44:40.677 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "smtp_destination", 2024-03-28T03:44:40.677 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.678 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.678 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.678 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:40.678 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.678 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.678 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.678 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Email address to send alerts to", 2024-03-28T03:44:40.678 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.679 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.679 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.679 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.679 INFO:teuthology.orchestra.run.smithi119.stdout: "smtp_from_name": { 2024-03-28T03:44:40.679 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "smtp_from_name", 2024-03-28T03:44:40.679 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.679 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.679 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.679 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "Ceph", 2024-03-28T03:44:40.680 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.680 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.680 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.680 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Email From: name", 2024-03-28T03:44:40.680 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.680 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.680 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.680 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.681 INFO:teuthology.orchestra.run.smithi119.stdout: "smtp_host": { 2024-03-28T03:44:40.681 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "smtp_host", 2024-03-28T03:44:40.681 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.681 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.681 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.681 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:40.681 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.681 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.681 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.682 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "SMTP server", 2024-03-28T03:44:40.682 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.682 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.682 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.682 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.682 INFO:teuthology.orchestra.run.smithi119.stdout: "smtp_password": { 2024-03-28T03:44:40.682 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "smtp_password", 2024-03-28T03:44:40.682 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.683 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.683 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.683 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:40.683 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.683 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.683 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.683 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Password to authenticate with", 2024-03-28T03:44:40.683 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.684 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.684 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.684 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.684 INFO:teuthology.orchestra.run.smithi119.stdout: "smtp_port": { 2024-03-28T03:44:40.684 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "smtp_port", 2024-03-28T03:44:40.684 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2024-03-28T03:44:40.684 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.684 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.684 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "465", 2024-03-28T03:44:40.685 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.685 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.685 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.685 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "SMTP port", 2024-03-28T03:44:40.685 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.685 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.685 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.685 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.685 INFO:teuthology.orchestra.run.smithi119.stdout: "smtp_sender": { 2024-03-28T03:44:40.686 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "smtp_sender", 2024-03-28T03:44:40.686 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.686 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.686 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.686 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:40.686 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.686 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.686 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.687 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "SMTP envelope sender", 2024-03-28T03:44:40.687 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.687 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.687 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.687 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.687 INFO:teuthology.orchestra.run.smithi119.stdout: "smtp_ssl": { 2024-03-28T03:44:40.687 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "smtp_ssl", 2024-03-28T03:44:40.687 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.687 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.688 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.688 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2024-03-28T03:44:40.688 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.688 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.688 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.688 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Use SSL to connect to SMTP server", 2024-03-28T03:44:40.688 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.688 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.688 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.689 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.689 INFO:teuthology.orchestra.run.smithi119.stdout: "smtp_user": { 2024-03-28T03:44:40.689 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "smtp_user", 2024-03-28T03:44:40.689 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.689 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.689 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.689 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:40.689 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.690 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.690 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.690 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "User to authenticate as", 2024-03-28T03:44:40.690 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.690 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.690 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.690 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:40.690 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:40.691 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.691 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-03-28T03:44:40.691 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "balancer", 2024-03-28T03:44:40.691 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2024-03-28T03:44:40.691 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2024-03-28T03:44:40.691 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2024-03-28T03:44:40.691 INFO:teuthology.orchestra.run.smithi119.stdout: "active": { 2024-03-28T03:44:40.691 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "active", 2024-03-28T03:44:40.692 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.692 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.692 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.692 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2024-03-28T03:44:40.692 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.692 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.692 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.692 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "automatically balance PGs across cluster", 2024-03-28T03:44:40.692 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.693 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.693 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.693 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.693 INFO:teuthology.orchestra.run.smithi119.stdout: "begin_time": { 2024-03-28T03:44:40.693 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "begin_time", 2024-03-28T03:44:40.693 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.693 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.693 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.693 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "0000", 2024-03-28T03:44:40.694 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.694 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.694 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.694 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "beginning time of day to automatically balance", 2024-03-28T03:44:40.694 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-28T03:44:40.694 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.694 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.694 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.695 INFO:teuthology.orchestra.run.smithi119.stdout: "begin_weekday": { 2024-03-28T03:44:40.695 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "begin_weekday", 2024-03-28T03:44:40.695 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "uint", 2024-03-28T03:44:40.695 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.695 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.695 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "0", 2024-03-28T03:44:40.695 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "0", 2024-03-28T03:44:40.695 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "6", 2024-03-28T03:44:40.695 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.696 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-03-28T03:44:40.696 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-28T03:44:40.696 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.696 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.696 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.696 INFO:teuthology.orchestra.run.smithi119.stdout: "crush_compat_max_iterations": { 2024-03-28T03:44:40.696 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "crush_compat_max_iterations", 2024-03-28T03:44:40.696 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "uint", 2024-03-28T03:44:40.696 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.697 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.697 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "25", 2024-03-28T03:44:40.697 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "1", 2024-03-28T03:44:40.697 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "250", 2024-03-28T03:44:40.697 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.697 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-03-28T03:44:40.697 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.697 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.698 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.698 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.698 INFO:teuthology.orchestra.run.smithi119.stdout: "crush_compat_metrics": { 2024-03-28T03:44:40.698 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "crush_compat_metrics", 2024-03-28T03:44:40.698 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.698 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.698 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.698 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "pgs,objects,bytes", 2024-03-28T03:44:40.699 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.699 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.699 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.699 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-03-28T03:44:40.699 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-03-28T03:44:40.699 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.699 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.699 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.699 INFO:teuthology.orchestra.run.smithi119.stdout: "crush_compat_step": { 2024-03-28T03:44:40.700 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "crush_compat_step", 2024-03-28T03:44:40.700 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "float", 2024-03-28T03:44:40.700 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.700 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.700 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "0.5", 2024-03-28T03:44:40.700 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "0.001", 2024-03-28T03:44:40.700 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "0.999", 2024-03-28T03:44:40.700 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.701 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "aggressiveness of optimization", 2024-03-28T03:44:40.701 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-03-28T03:44:40.701 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.701 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.701 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.701 INFO:teuthology.orchestra.run.smithi119.stdout: "end_time": { 2024-03-28T03:44:40.701 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "end_time", 2024-03-28T03:44:40.701 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.701 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.702 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.702 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "2359", 2024-03-28T03:44:40.702 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.702 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.702 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.702 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "ending time of day to automatically balance", 2024-03-28T03:44:40.702 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-28T03:44:40.703 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.703 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.703 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.703 INFO:teuthology.orchestra.run.smithi119.stdout: "end_weekday": { 2024-03-28T03:44:40.703 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "end_weekday", 2024-03-28T03:44:40.703 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "uint", 2024-03-28T03:44:40.703 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.703 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.703 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "0", 2024-03-28T03:44:40.704 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "0", 2024-03-28T03:44:40.704 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "6", 2024-03-28T03:44:40.704 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.704 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-03-28T03:44:40.704 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-28T03:44:40.704 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.704 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.704 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.704 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2024-03-28T03:44:40.705 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2024-03-28T03:44:40.705 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.705 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.705 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.705 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:40.705 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.705 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.705 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:40.706 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:40.706 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:40.706 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:40.706 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:40.706 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:40.706 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:40.706 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:40.706 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.706 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.707 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.707 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.707 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.707 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2024-03-28T03:44:40.707 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2024-03-28T03:44:40.707 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.707 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.707 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.707 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:40.708 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.708 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.708 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.708 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.708 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.708 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.708 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.708 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.709 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2024-03-28T03:44:40.709 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2024-03-28T03:44:40.709 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.709 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.709 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.709 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2024-03-28T03:44:40.709 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.709 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.709 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:40.710 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:40.710 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:40.710 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:40.710 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:40.710 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:40.710 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:40.710 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:40.710 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.711 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.711 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.711 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.711 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.711 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2024-03-28T03:44:40.711 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2024-03-28T03:44:40.711 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.711 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.712 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.712 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:40.712 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.712 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.712 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.712 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.712 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.712 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.712 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.713 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.713 INFO:teuthology.orchestra.run.smithi119.stdout: "min_score": { 2024-03-28T03:44:40.713 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "min_score", 2024-03-28T03:44:40.713 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "float", 2024-03-28T03:44:40.713 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.713 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.713 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "0", 2024-03-28T03:44:40.713 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.714 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.714 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.714 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-03-28T03:44:40.714 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.714 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.714 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.714 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.714 INFO:teuthology.orchestra.run.smithi119.stdout: "mode": { 2024-03-28T03:44:40.715 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "mode", 2024-03-28T03:44:40.715 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.715 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.715 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.715 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "upmap", 2024-03-28T03:44:40.715 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.715 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.715 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:40.715 INFO:teuthology.orchestra.run.smithi119.stdout: "crush-compat", 2024-03-28T03:44:40.716 INFO:teuthology.orchestra.run.smithi119.stdout: "none", 2024-03-28T03:44:40.716 INFO:teuthology.orchestra.run.smithi119.stdout: "read", 2024-03-28T03:44:40.716 INFO:teuthology.orchestra.run.smithi119.stdout: "upmap", 2024-03-28T03:44:40.716 INFO:teuthology.orchestra.run.smithi119.stdout: "upmap-read" 2024-03-28T03:44:40.716 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:40.716 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Balancer mode", 2024-03-28T03:44:40.716 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.716 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.717 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.717 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.717 INFO:teuthology.orchestra.run.smithi119.stdout: "pool_ids": { 2024-03-28T03:44:40.717 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "pool_ids", 2024-03-28T03:44:40.717 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.717 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.717 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.717 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:40.717 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.718 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.718 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.718 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-03-28T03:44:40.718 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.718 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.718 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.718 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.718 INFO:teuthology.orchestra.run.smithi119.stdout: "sleep_interval": { 2024-03-28T03:44:40.719 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "sleep_interval", 2024-03-28T03:44:40.719 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "secs", 2024-03-28T03:44:40.719 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.719 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.719 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "60", 2024-03-28T03:44:40.719 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.719 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.719 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.720 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-03-28T03:44:40.720 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.720 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.720 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.720 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.720 INFO:teuthology.orchestra.run.smithi119.stdout: "upmap_max_deviation": { 2024-03-28T03:44:40.720 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "upmap_max_deviation", 2024-03-28T03:44:40.720 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2024-03-28T03:44:40.720 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.721 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.721 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "5", 2024-03-28T03:44:40.721 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "1", 2024-03-28T03:44:40.721 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.721 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.721 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "deviation below which no optimization is attempted", 2024-03-28T03:44:40.721 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-03-28T03:44:40.721 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.721 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.722 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.722 INFO:teuthology.orchestra.run.smithi119.stdout: "upmap_max_optimizations": { 2024-03-28T03:44:40.722 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "upmap_max_optimizations", 2024-03-28T03:44:40.722 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "uint", 2024-03-28T03:44:40.722 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.722 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.722 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "10", 2024-03-28T03:44:40.722 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.723 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.723 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.723 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-03-28T03:44:40.723 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.723 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.723 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.723 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:40.723 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:40.723 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.724 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-03-28T03:44:40.724 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "cephadm", 2024-03-28T03:44:40.724 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2024-03-28T03:44:40.724 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2024-03-28T03:44:40.724 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2024-03-28T03:44:40.724 INFO:teuthology.orchestra.run.smithi119.stdout: "agent_down_multiplier": { 2024-03-28T03:44:40.724 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "agent_down_multiplier", 2024-03-28T03:44:40.724 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "float", 2024-03-28T03:44:40.724 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.725 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.725 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "3.0", 2024-03-28T03:44:40.725 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.725 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.725 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.725 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-03-28T03:44:40.725 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.725 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.725 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.726 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.726 INFO:teuthology.orchestra.run.smithi119.stdout: "agent_refresh_rate": { 2024-03-28T03:44:40.726 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "agent_refresh_rate", 2024-03-28T03:44:40.726 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "secs", 2024-03-28T03:44:40.726 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.726 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.726 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "20", 2024-03-28T03:44:40.727 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.727 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.727 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.727 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-03-28T03:44:40.727 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.727 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.727 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.727 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.727 INFO:teuthology.orchestra.run.smithi119.stdout: "agent_starting_port": { 2024-03-28T03:44:40.728 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "agent_starting_port", 2024-03-28T03:44:40.728 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2024-03-28T03:44:40.728 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.728 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.728 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "4721", 2024-03-28T03:44:40.728 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.728 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.728 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.728 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-03-28T03:44:40.729 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.729 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.729 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.729 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.729 INFO:teuthology.orchestra.run.smithi119.stdout: "allow_ptrace": { 2024-03-28T03:44:40.729 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "allow_ptrace", 2024-03-28T03:44:40.729 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.729 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.729 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.730 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:40.730 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.730 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.730 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.730 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-03-28T03:44:40.730 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-03-28T03:44:40.730 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.731 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.731 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.731 INFO:teuthology.orchestra.run.smithi119.stdout: "autotune_interval": { 2024-03-28T03:44:40.731 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "autotune_interval", 2024-03-28T03:44:40.731 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "secs", 2024-03-28T03:44:40.731 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.731 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.731 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "600", 2024-03-28T03:44:40.731 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.732 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.732 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.732 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "how frequently to autotune daemon memory", 2024-03-28T03:44:40.732 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.732 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.732 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.732 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.732 INFO:teuthology.orchestra.run.smithi119.stdout: "autotune_memory_target_ratio": { 2024-03-28T03:44:40.732 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "autotune_memory_target_ratio", 2024-03-28T03:44:40.733 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "float", 2024-03-28T03:44:40.733 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.733 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.733 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "0.7", 2024-03-28T03:44:40.733 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.733 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.733 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.733 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-03-28T03:44:40.733 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.734 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.734 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.734 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.734 INFO:teuthology.orchestra.run.smithi119.stdout: "cephadm_log_destination": { 2024-03-28T03:44:40.734 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "cephadm_log_destination", 2024-03-28T03:44:40.734 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.734 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.734 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.735 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:40.735 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.735 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.735 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:40.735 INFO:teuthology.orchestra.run.smithi119.stdout: "file", 2024-03-28T03:44:40.735 INFO:teuthology.orchestra.run.smithi119.stdout: "file,syslog", 2024-03-28T03:44:40.735 INFO:teuthology.orchestra.run.smithi119.stdout: "syslog" 2024-03-28T03:44:40.735 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:40.735 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-03-28T03:44:40.736 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.736 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.736 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.736 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.736 INFO:teuthology.orchestra.run.smithi119.stdout: "cgroups_split": { 2024-03-28T03:44:40.736 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "cgroups_split", 2024-03-28T03:44:40.736 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.736 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.736 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.737 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2024-03-28T03:44:40.737 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.737 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.737 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.737 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-03-28T03:44:40.737 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.737 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.737 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.737 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.738 INFO:teuthology.orchestra.run.smithi119.stdout: "config_checks_enabled": { 2024-03-28T03:44:40.738 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "config_checks_enabled", 2024-03-28T03:44:40.738 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.738 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.738 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.738 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:40.738 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.738 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.739 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.739 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-03-28T03:44:40.739 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.739 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.739 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.739 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.739 INFO:teuthology.orchestra.run.smithi119.stdout: "config_dashboard": { 2024-03-28T03:44:40.739 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "config_dashboard", 2024-03-28T03:44:40.740 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.740 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.740 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.740 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2024-03-28T03:44:40.740 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.740 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.740 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.740 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-03-28T03:44:40.741 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.741 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.741 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.741 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.741 INFO:teuthology.orchestra.run.smithi119.stdout: "container_image_alertmanager": { 2024-03-28T03:44:40.741 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "container_image_alertmanager", 2024-03-28T03:44:40.741 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.741 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.741 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.742 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-03-28T03:44:40.742 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.742 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.742 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.742 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Prometheus container image", 2024-03-28T03:44:40.742 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.742 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.742 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.743 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.743 INFO:teuthology.orchestra.run.smithi119.stdout: "container_image_base": { 2024-03-28T03:44:40.743 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "container_image_base", 2024-03-28T03:44:40.743 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.743 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.743 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.743 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "quay.io/ceph/ceph", 2024-03-28T03:44:40.743 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.743 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.744 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.744 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Container image name, without the tag", 2024-03-28T03:44:40.744 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.744 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.744 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.744 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.744 INFO:teuthology.orchestra.run.smithi119.stdout: "container_image_elasticsearch": { 2024-03-28T03:44:40.744 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "container_image_elasticsearch", 2024-03-28T03:44:40.745 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.745 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.745 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.745 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-03-28T03:44:40.745 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.745 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.745 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.745 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "elasticsearch container image", 2024-03-28T03:44:40.745 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.746 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.746 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.746 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.746 INFO:teuthology.orchestra.run.smithi119.stdout: "container_image_grafana": { 2024-03-28T03:44:40.746 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "container_image_grafana", 2024-03-28T03:44:40.746 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.746 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.746 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.747 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.12", 2024-03-28T03:44:40.747 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.747 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.747 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.747 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Prometheus container image", 2024-03-28T03:44:40.747 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.747 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.747 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.747 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.748 INFO:teuthology.orchestra.run.smithi119.stdout: "container_image_haproxy": { 2024-03-28T03:44:40.748 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "container_image_haproxy", 2024-03-28T03:44:40.748 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.748 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.748 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.748 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-03-28T03:44:40.748 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.748 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.749 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.749 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "HAproxy container image", 2024-03-28T03:44:40.749 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.749 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.749 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.749 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.749 INFO:teuthology.orchestra.run.smithi119.stdout: "container_image_jaeger_agent": { 2024-03-28T03:44:40.749 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "container_image_jaeger_agent", 2024-03-28T03:44:40.749 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.750 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.750 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.750 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-03-28T03:44:40.750 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.750 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.750 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.750 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Jaeger agent container image", 2024-03-28T03:44:40.751 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.751 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.751 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.751 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.751 INFO:teuthology.orchestra.run.smithi119.stdout: "container_image_jaeger_collector": { 2024-03-28T03:44:40.751 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "container_image_jaeger_collector", 2024-03-28T03:44:40.751 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.751 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.751 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.752 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-03-28T03:44:40.752 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.752 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.752 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.752 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Jaeger collector container image", 2024-03-28T03:44:40.752 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.752 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.752 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.752 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.753 INFO:teuthology.orchestra.run.smithi119.stdout: "container_image_jaeger_query": { 2024-03-28T03:44:40.753 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "container_image_jaeger_query", 2024-03-28T03:44:40.753 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.753 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.753 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.753 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-03-28T03:44:40.753 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.753 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.753 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.754 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Jaeger query container image", 2024-03-28T03:44:40.754 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.754 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.754 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.754 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.754 INFO:teuthology.orchestra.run.smithi119.stdout: "container_image_keepalived": { 2024-03-28T03:44:40.754 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "container_image_keepalived", 2024-03-28T03:44:40.754 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.755 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.755 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.755 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-03-28T03:44:40.755 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.755 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.755 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.755 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Keepalived container image", 2024-03-28T03:44:40.755 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.755 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.756 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.756 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.756 INFO:teuthology.orchestra.run.smithi119.stdout: "container_image_loki": { 2024-03-28T03:44:40.756 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "container_image_loki", 2024-03-28T03:44:40.756 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.756 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.756 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.756 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-03-28T03:44:40.757 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.757 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.757 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.757 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Loki container image", 2024-03-28T03:44:40.757 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.757 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.757 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.757 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.757 INFO:teuthology.orchestra.run.smithi119.stdout: "container_image_node_exporter": { 2024-03-28T03:44:40.758 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "container_image_node_exporter", 2024-03-28T03:44:40.758 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.758 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.758 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.758 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-03-28T03:44:40.758 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.758 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.758 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.759 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Prometheus container image", 2024-03-28T03:44:40.759 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.759 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.759 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.759 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.759 INFO:teuthology.orchestra.run.smithi119.stdout: "container_image_nvmeof": { 2024-03-28T03:44:40.759 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "container_image_nvmeof", 2024-03-28T03:44:40.759 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.760 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.760 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.760 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-03-28T03:44:40.760 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.760 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.760 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.760 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Nvme-of container image", 2024-03-28T03:44:40.760 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.760 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.761 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.761 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.761 INFO:teuthology.orchestra.run.smithi119.stdout: "container_image_prometheus": { 2024-03-28T03:44:40.761 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "container_image_prometheus", 2024-03-28T03:44:40.761 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.761 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.761 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.761 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-03-28T03:44:40.761 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.762 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.762 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.762 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Prometheus container image", 2024-03-28T03:44:40.762 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.762 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.762 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.762 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.762 INFO:teuthology.orchestra.run.smithi119.stdout: "container_image_promtail": { 2024-03-28T03:44:40.763 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "container_image_promtail", 2024-03-28T03:44:40.763 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.763 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.763 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.763 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-03-28T03:44:40.763 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.763 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.763 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.763 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Promtail container image", 2024-03-28T03:44:40.764 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.764 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.764 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.764 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.764 INFO:teuthology.orchestra.run.smithi119.stdout: "container_image_snmp_gateway": { 2024-03-28T03:44:40.764 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "container_image_snmp_gateway", 2024-03-28T03:44:40.764 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.764 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.764 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.765 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-03-28T03:44:40.765 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.765 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.765 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.765 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "SNMP Gateway container image", 2024-03-28T03:44:40.765 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.765 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.765 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.766 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.766 INFO:teuthology.orchestra.run.smithi119.stdout: "container_init": { 2024-03-28T03:44:40.766 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "container_init", 2024-03-28T03:44:40.766 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.766 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.766 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.766 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2024-03-28T03:44:40.766 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.766 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.767 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.767 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Run podman/docker with `--init`", 2024-03-28T03:44:40.767 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.767 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.767 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.767 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.767 INFO:teuthology.orchestra.run.smithi119.stdout: "daemon_cache_timeout": { 2024-03-28T03:44:40.767 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "daemon_cache_timeout", 2024-03-28T03:44:40.767 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "secs", 2024-03-28T03:44:40.768 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.768 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.768 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "600", 2024-03-28T03:44:40.768 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.768 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.768 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.768 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-03-28T03:44:40.768 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.769 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.769 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.769 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.769 INFO:teuthology.orchestra.run.smithi119.stdout: "default_cephadm_command_timeout": { 2024-03-28T03:44:40.769 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "default_cephadm_command_timeout", 2024-03-28T03:44:40.769 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2024-03-28T03:44:40.769 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.769 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.769 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "900", 2024-03-28T03:44:40.770 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.770 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.770 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.770 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-03-28T03:44:40.770 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.770 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.770 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.771 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.771 INFO:teuthology.orchestra.run.smithi119.stdout: "default_registry": { 2024-03-28T03:44:40.771 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "default_registry", 2024-03-28T03:44:40.771 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.771 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.771 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.771 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "docker.io", 2024-03-28T03:44:40.771 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.771 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.772 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.772 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-03-28T03:44:40.772 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.772 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.772 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.772 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.772 INFO:teuthology.orchestra.run.smithi119.stdout: "device_cache_timeout": { 2024-03-28T03:44:40.772 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "device_cache_timeout", 2024-03-28T03:44:40.773 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "secs", 2024-03-28T03:44:40.773 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.773 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.773 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "1800", 2024-03-28T03:44:40.773 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.773 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.773 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.773 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "seconds to cache device inventory", 2024-03-28T03:44:40.773 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.774 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.774 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.774 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.774 INFO:teuthology.orchestra.run.smithi119.stdout: "device_enhanced_scan": { 2024-03-28T03:44:40.774 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "device_enhanced_scan", 2024-03-28T03:44:40.774 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.774 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.774 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.775 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:40.775 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.775 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.775 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.775 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Use libstoragemgmt during device scans", 2024-03-28T03:44:40.775 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.775 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.775 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.775 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.776 INFO:teuthology.orchestra.run.smithi119.stdout: "facts_cache_timeout": { 2024-03-28T03:44:40.776 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "facts_cache_timeout", 2024-03-28T03:44:40.776 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "secs", 2024-03-28T03:44:40.776 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.776 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.776 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "60", 2024-03-28T03:44:40.776 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.776 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.776 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.777 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "seconds to cache host facts data", 2024-03-28T03:44:40.777 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.777 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.777 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.777 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.777 INFO:teuthology.orchestra.run.smithi119.stdout: "host_check_interval": { 2024-03-28T03:44:40.777 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "host_check_interval", 2024-03-28T03:44:40.777 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "secs", 2024-03-28T03:44:40.777 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.778 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.778 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "600", 2024-03-28T03:44:40.778 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.778 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.778 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.778 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "how frequently to perform a host check", 2024-03-28T03:44:40.778 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.778 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.779 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.779 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.779 INFO:teuthology.orchestra.run.smithi119.stdout: "hw_monitoring": { 2024-03-28T03:44:40.779 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "hw_monitoring", 2024-03-28T03:44:40.779 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.779 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.779 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.779 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:40.779 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.780 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.780 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.780 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-03-28T03:44:40.780 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.780 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.780 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.780 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.780 INFO:teuthology.orchestra.run.smithi119.stdout: "inventory_list_all": { 2024-03-28T03:44:40.781 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "inventory_list_all", 2024-03-28T03:44:40.781 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.781 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.781 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.781 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:40.781 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.781 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.781 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.782 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-03-28T03:44:40.782 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.782 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.782 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.782 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.782 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2024-03-28T03:44:40.782 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2024-03-28T03:44:40.782 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.783 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.783 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.783 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:40.783 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.783 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.783 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:40.783 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:40.783 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:40.783 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:40.784 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:40.784 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:40.784 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:40.784 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:40.784 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.784 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.784 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.784 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.785 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.785 INFO:teuthology.orchestra.run.smithi119.stdout: "log_refresh_metadata": { 2024-03-28T03:44:40.785 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_refresh_metadata", 2024-03-28T03:44:40.785 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.785 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.785 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.785 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:40.785 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.785 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.786 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.786 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-03-28T03:44:40.786 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.786 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.786 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.786 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.786 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2024-03-28T03:44:40.786 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2024-03-28T03:44:40.787 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.787 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.787 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.787 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2024-03-28T03:44:40.787 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.787 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.787 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.787 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-03-28T03:44:40.788 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.788 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.788 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.788 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.788 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2024-03-28T03:44:40.788 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2024-03-28T03:44:40.788 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.788 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.788 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.789 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2024-03-28T03:44:40.789 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.789 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.789 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:40.789 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:40.789 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:40.789 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:40.789 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:40.790 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:40.790 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:40.790 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:40.790 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.790 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.790 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.790 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.790 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.791 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2024-03-28T03:44:40.791 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2024-03-28T03:44:40.791 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.791 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.791 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.791 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:40.791 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.791 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.791 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.792 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.792 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.792 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.792 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.792 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.792 INFO:teuthology.orchestra.run.smithi119.stdout: "manage_etc_ceph_ceph_conf": { 2024-03-28T03:44:40.792 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-03-28T03:44:40.792 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.793 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.793 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.793 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:40.793 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.793 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.793 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.793 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-03-28T03:44:40.793 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.793 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.794 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.794 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.794 INFO:teuthology.orchestra.run.smithi119.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-03-28T03:44:40.794 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-03-28T03:44:40.794 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.794 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.794 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.794 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "*", 2024-03-28T03:44:40.795 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.795 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.795 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.795 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-03-28T03:44:40.795 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.795 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.795 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.795 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.795 INFO:teuthology.orchestra.run.smithi119.stdout: "max_count_per_host": { 2024-03-28T03:44:40.796 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "max_count_per_host", 2024-03-28T03:44:40.796 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2024-03-28T03:44:40.796 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.796 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.796 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "10", 2024-03-28T03:44:40.796 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.796 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.796 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.797 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "max number of daemons per service per host", 2024-03-28T03:44:40.797 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.797 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.797 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.797 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.797 INFO:teuthology.orchestra.run.smithi119.stdout: "max_osd_draining_count": { 2024-03-28T03:44:40.797 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "max_osd_draining_count", 2024-03-28T03:44:40.797 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2024-03-28T03:44:40.797 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.798 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.798 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "10", 2024-03-28T03:44:40.798 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.798 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.798 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.798 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-03-28T03:44:40.798 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.798 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.798 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.799 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.799 INFO:teuthology.orchestra.run.smithi119.stdout: "migration_current": { 2024-03-28T03:44:40.799 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "migration_current", 2024-03-28T03:44:40.799 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2024-03-28T03:44:40.799 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.799 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.799 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:40.799 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.799 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.800 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.800 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "internal - do not modify", 2024-03-28T03:44:40.800 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.800 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.800 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.800 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.800 INFO:teuthology.orchestra.run.smithi119.stdout: "mode": { 2024-03-28T03:44:40.800 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "mode", 2024-03-28T03:44:40.801 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.801 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.801 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.801 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "root", 2024-03-28T03:44:40.801 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.801 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.801 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:40.801 INFO:teuthology.orchestra.run.smithi119.stdout: "cephadm-package", 2024-03-28T03:44:40.801 INFO:teuthology.orchestra.run.smithi119.stdout: "root" 2024-03-28T03:44:40.802 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:40.802 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "mode for remote execution of cephadm", 2024-03-28T03:44:40.802 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.802 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.802 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.802 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.802 INFO:teuthology.orchestra.run.smithi119.stdout: "oob_default_addr": { 2024-03-28T03:44:40.802 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "oob_default_addr", 2024-03-28T03:44:40.803 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.803 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.803 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.803 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "169.254.1.1", 2024-03-28T03:44:40.803 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.803 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.803 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.803 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Default address for RedFish API (oob management).", 2024-03-28T03:44:40.804 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.804 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.804 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.804 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.804 INFO:teuthology.orchestra.run.smithi119.stdout: "prometheus_alerts_path": { 2024-03-28T03:44:40.804 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "prometheus_alerts_path", 2024-03-28T03:44:40.804 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.804 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.805 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.805 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-03-28T03:44:40.805 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.805 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.805 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.805 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-03-28T03:44:40.805 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.805 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.805 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.806 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.806 INFO:teuthology.orchestra.run.smithi119.stdout: "registry_insecure": { 2024-03-28T03:44:40.806 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "registry_insecure", 2024-03-28T03:44:40.806 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.806 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.806 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.806 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:40.806 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.807 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.807 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.807 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-03-28T03:44:40.807 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.807 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.807 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.807 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.807 INFO:teuthology.orchestra.run.smithi119.stdout: "registry_password": { 2024-03-28T03:44:40.808 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "registry_password", 2024-03-28T03:44:40.808 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.808 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.808 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.808 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:40.808 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.808 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.808 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.808 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-03-28T03:44:40.809 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.809 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.809 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.809 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.809 INFO:teuthology.orchestra.run.smithi119.stdout: "registry_url": { 2024-03-28T03:44:40.809 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "registry_url", 2024-03-28T03:44:40.809 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.809 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.809 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.810 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:40.810 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.810 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.810 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.810 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-03-28T03:44:40.810 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.810 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.810 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.811 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.811 INFO:teuthology.orchestra.run.smithi119.stdout: "registry_username": { 2024-03-28T03:44:40.811 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "registry_username", 2024-03-28T03:44:40.811 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.811 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.811 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.811 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:40.811 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.812 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.812 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.812 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-03-28T03:44:40.812 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.812 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.812 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.812 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.812 INFO:teuthology.orchestra.run.smithi119.stdout: "secure_monitoring_stack": { 2024-03-28T03:44:40.813 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "secure_monitoring_stack", 2024-03-28T03:44:40.813 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.813 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.813 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.813 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:40.813 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.813 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.813 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.813 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-03-28T03:44:40.814 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.814 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.814 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.814 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.814 INFO:teuthology.orchestra.run.smithi119.stdout: "service_discovery_port": { 2024-03-28T03:44:40.814 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "service_discovery_port", 2024-03-28T03:44:40.814 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2024-03-28T03:44:40.814 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.815 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.815 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "8765", 2024-03-28T03:44:40.815 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.815 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.815 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.815 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "cephadm service discovery port", 2024-03-28T03:44:40.815 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.815 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.816 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.816 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.816 INFO:teuthology.orchestra.run.smithi119.stdout: "ssh_config_file": { 2024-03-28T03:44:40.816 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "ssh_config_file", 2024-03-28T03:44:40.816 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.816 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.816 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.816 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:40.816 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.817 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.817 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.817 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-03-28T03:44:40.817 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.817 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.817 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.817 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.817 INFO:teuthology.orchestra.run.smithi119.stdout: "use_agent": { 2024-03-28T03:44:40.817 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "use_agent", 2024-03-28T03:44:40.818 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.818 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.818 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.818 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:40.818 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.818 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.818 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.818 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-03-28T03:44:40.819 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.819 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.819 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.819 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.819 INFO:teuthology.orchestra.run.smithi119.stdout: "use_repo_digest": { 2024-03-28T03:44:40.819 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "use_repo_digest", 2024-03-28T03:44:40.819 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.819 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.820 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.820 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2024-03-28T03:44:40.820 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.820 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.820 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.820 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-03-28T03:44:40.820 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.820 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.820 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.821 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.821 INFO:teuthology.orchestra.run.smithi119.stdout: "warn_on_failed_host_check": { 2024-03-28T03:44:40.821 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "warn_on_failed_host_check", 2024-03-28T03:44:40.821 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.821 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.821 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.821 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2024-03-28T03:44:40.821 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.821 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.822 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.822 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "raise a health warning if the host check fails", 2024-03-28T03:44:40.822 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.822 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.822 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.822 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.822 INFO:teuthology.orchestra.run.smithi119.stdout: "warn_on_stray_daemons": { 2024-03-28T03:44:40.822 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "warn_on_stray_daemons", 2024-03-28T03:44:40.823 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.823 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.823 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.823 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2024-03-28T03:44:40.823 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.823 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.823 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.823 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-03-28T03:44:40.823 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.824 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.824 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.824 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.824 INFO:teuthology.orchestra.run.smithi119.stdout: "warn_on_stray_hosts": { 2024-03-28T03:44:40.824 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "warn_on_stray_hosts", 2024-03-28T03:44:40.824 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.824 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.824 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.825 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2024-03-28T03:44:40.825 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.825 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.825 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.825 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-03-28T03:44:40.825 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.825 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.825 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.825 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:40.826 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:40.826 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.826 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-03-28T03:44:40.826 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "crash", 2024-03-28T03:44:40.826 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2024-03-28T03:44:40.826 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2024-03-28T03:44:40.826 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2024-03-28T03:44:40.826 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2024-03-28T03:44:40.826 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2024-03-28T03:44:40.827 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.827 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.827 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.827 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:40.827 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.827 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.827 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:40.827 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:40.827 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:40.828 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:40.828 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:40.828 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:40.828 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:40.828 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:40.828 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.828 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.828 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.829 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.829 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.829 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2024-03-28T03:44:40.829 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2024-03-28T03:44:40.829 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.829 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.829 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.829 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:40.829 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.830 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.830 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.830 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.830 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.830 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.830 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.830 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.830 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2024-03-28T03:44:40.831 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2024-03-28T03:44:40.831 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.831 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.831 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.831 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2024-03-28T03:44:40.831 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.831 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.831 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:40.831 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:40.832 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:40.832 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:40.832 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:40.832 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:40.832 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:40.832 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:40.832 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.832 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.833 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.833 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.833 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.833 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2024-03-28T03:44:40.833 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2024-03-28T03:44:40.833 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.833 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.833 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.833 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:40.834 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.834 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.834 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.834 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.834 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.834 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.834 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.834 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.835 INFO:teuthology.orchestra.run.smithi119.stdout: "retain_interval": { 2024-03-28T03:44:40.835 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "retain_interval", 2024-03-28T03:44:40.835 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "secs", 2024-03-28T03:44:40.835 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.835 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.835 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "31536000", 2024-03-28T03:44:40.835 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.835 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.835 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.836 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "how long to retain crashes before pruning them", 2024-03-28T03:44:40.836 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.836 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.836 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.836 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.836 INFO:teuthology.orchestra.run.smithi119.stdout: "warn_recent_interval": { 2024-03-28T03:44:40.836 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "warn_recent_interval", 2024-03-28T03:44:40.836 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "secs", 2024-03-28T03:44:40.837 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.837 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.837 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "1209600", 2024-03-28T03:44:40.837 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.837 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.837 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.837 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "time interval in which to warn about recent crashes", 2024-03-28T03:44:40.837 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.837 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.838 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.838 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:40.838 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:40.838 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.838 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-03-28T03:44:40.838 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "dashboard", 2024-03-28T03:44:40.838 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2024-03-28T03:44:40.838 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2024-03-28T03:44:40.839 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2024-03-28T03:44:40.839 INFO:teuthology.orchestra.run.smithi119.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-03-28T03:44:40.839 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-03-28T03:44:40.839 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2024-03-28T03:44:40.839 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.839 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.839 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "10", 2024-03-28T03:44:40.839 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.840 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.840 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.840 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.840 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.840 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.840 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.840 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.840 INFO:teuthology.orchestra.run.smithi119.stdout: "ALERTMANAGER_API_HOST": { 2024-03-28T03:44:40.840 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "ALERTMANAGER_API_HOST", 2024-03-28T03:44:40.841 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.841 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.841 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.841 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:40.841 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.841 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.841 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.841 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.842 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.842 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.842 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.842 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.842 INFO:teuthology.orchestra.run.smithi119.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-03-28T03:44:40.842 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-03-28T03:44:40.842 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.842 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.843 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.843 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2024-03-28T03:44:40.843 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.843 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.843 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.843 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.843 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.843 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.843 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.844 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.844 INFO:teuthology.orchestra.run.smithi119.stdout: "AUDIT_API_ENABLED": { 2024-03-28T03:44:40.844 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "AUDIT_API_ENABLED", 2024-03-28T03:44:40.844 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.844 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.844 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.844 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:40.844 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.845 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.845 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.845 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.845 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.845 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.845 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.845 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.845 INFO:teuthology.orchestra.run.smithi119.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-03-28T03:44:40.845 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-03-28T03:44:40.846 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.846 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.846 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.846 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2024-03-28T03:44:40.846 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.846 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.846 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.846 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.847 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.847 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.847 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.847 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.847 INFO:teuthology.orchestra.run.smithi119.stdout: "ENABLE_BROWSABLE_API": { 2024-03-28T03:44:40.847 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "ENABLE_BROWSABLE_API", 2024-03-28T03:44:40.847 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.847 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.847 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.848 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2024-03-28T03:44:40.848 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.848 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.848 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.848 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.848 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.848 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.848 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.849 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.849 INFO:teuthology.orchestra.run.smithi119.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-03-28T03:44:40.849 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-03-28T03:44:40.849 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.849 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.849 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.849 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2024-03-28T03:44:40.849 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.849 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.850 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.850 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.850 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.850 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.850 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.850 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.850 INFO:teuthology.orchestra.run.smithi119.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-03-28T03:44:40.850 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-03-28T03:44:40.851 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.851 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.851 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.851 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2024-03-28T03:44:40.851 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.851 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.851 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.851 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.851 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.852 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.852 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.852 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.852 INFO:teuthology.orchestra.run.smithi119.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-03-28T03:44:40.852 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-03-28T03:44:40.852 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.852 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.852 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.853 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2024-03-28T03:44:40.853 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.853 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.853 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.853 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.853 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.853 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.853 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.853 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.854 INFO:teuthology.orchestra.run.smithi119.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-03-28T03:44:40.854 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-03-28T03:44:40.854 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.854 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.854 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.854 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2024-03-28T03:44:40.854 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.854 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.855 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.855 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.855 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.855 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.855 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.855 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.855 INFO:teuthology.orchestra.run.smithi119.stdout: "FEATURE_TOGGLE_NFS": { 2024-03-28T03:44:40.855 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-03-28T03:44:40.856 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.856 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.856 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.856 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2024-03-28T03:44:40.856 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.856 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.856 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.856 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.856 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.857 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.857 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.857 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.857 INFO:teuthology.orchestra.run.smithi119.stdout: "FEATURE_TOGGLE_RBD": { 2024-03-28T03:44:40.857 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-03-28T03:44:40.857 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.857 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.857 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.857 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2024-03-28T03:44:40.858 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.858 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.858 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.858 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.858 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.858 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.858 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.858 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.859 INFO:teuthology.orchestra.run.smithi119.stdout: "FEATURE_TOGGLE_RGW": { 2024-03-28T03:44:40.859 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-03-28T03:44:40.859 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.859 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.859 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.859 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2024-03-28T03:44:40.859 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.859 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.859 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.860 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.860 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.860 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.860 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.860 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.860 INFO:teuthology.orchestra.run.smithi119.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-03-28T03:44:40.860 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-03-28T03:44:40.860 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.861 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.861 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.861 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:40.861 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.861 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.861 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.861 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.861 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.861 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.862 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.862 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.862 INFO:teuthology.orchestra.run.smithi119.stdout: "GRAFANA_API_PASSWORD": { 2024-03-28T03:44:40.862 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "GRAFANA_API_PASSWORD", 2024-03-28T03:44:40.862 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.862 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.862 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.862 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "admin", 2024-03-28T03:44:40.863 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.863 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.863 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.863 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.863 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.863 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.863 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.863 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.864 INFO:teuthology.orchestra.run.smithi119.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-03-28T03:44:40.864 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-03-28T03:44:40.864 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.864 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.864 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.864 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2024-03-28T03:44:40.864 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.864 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.864 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.865 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.865 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.865 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.865 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.865 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.865 INFO:teuthology.orchestra.run.smithi119.stdout: "GRAFANA_API_URL": { 2024-03-28T03:44:40.865 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "GRAFANA_API_URL", 2024-03-28T03:44:40.865 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.866 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.866 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.866 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:40.866 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.866 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.866 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.866 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.866 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.866 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.867 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.867 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.867 INFO:teuthology.orchestra.run.smithi119.stdout: "GRAFANA_API_USERNAME": { 2024-03-28T03:44:40.867 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "GRAFANA_API_USERNAME", 2024-03-28T03:44:40.867 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.867 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.867 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.867 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "admin", 2024-03-28T03:44:40.868 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.868 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.868 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.868 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.868 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.868 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.868 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.868 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.868 INFO:teuthology.orchestra.run.smithi119.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-03-28T03:44:40.869 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-03-28T03:44:40.869 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.869 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.869 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.869 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:40.869 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.869 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.869 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.869 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.870 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.870 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.870 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.870 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.870 INFO:teuthology.orchestra.run.smithi119.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-03-28T03:44:40.870 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-03-28T03:44:40.870 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.870 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.871 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.871 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:40.871 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.871 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.871 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.871 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.871 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.871 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.872 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.872 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.872 INFO:teuthology.orchestra.run.smithi119.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-03-28T03:44:40.872 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-03-28T03:44:40.872 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.872 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.872 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.872 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2024-03-28T03:44:40.872 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.873 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.873 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.873 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.873 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.873 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.873 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.873 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.873 INFO:teuthology.orchestra.run.smithi119.stdout: "ISSUE_TRACKER_API_KEY": { 2024-03-28T03:44:40.873 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-03-28T03:44:40.874 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.874 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.874 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.874 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:40.874 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.874 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.874 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.874 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.875 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.875 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.875 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.875 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.875 INFO:teuthology.orchestra.run.smithi119.stdout: "MULTICLUSTER_CONFIG": { 2024-03-28T03:44:40.875 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "MULTICLUSTER_CONFIG", 2024-03-28T03:44:40.875 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.875 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.876 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.876 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "{}", 2024-03-28T03:44:40.876 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.876 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.876 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.876 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.876 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.876 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.877 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.877 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.877 INFO:teuthology.orchestra.run.smithi119.stdout: "PROMETHEUS_API_HOST": { 2024-03-28T03:44:40.877 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "PROMETHEUS_API_HOST", 2024-03-28T03:44:40.877 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.877 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.877 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.877 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:40.877 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.878 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.878 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.878 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.878 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.878 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.878 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.878 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.878 INFO:teuthology.orchestra.run.smithi119.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-03-28T03:44:40.879 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-03-28T03:44:40.879 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.879 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.879 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.879 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2024-03-28T03:44:40.879 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.879 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.879 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.879 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.880 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.880 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.880 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.880 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.880 INFO:teuthology.orchestra.run.smithi119.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-03-28T03:44:40.880 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-03-28T03:44:40.880 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.880 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.881 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.881 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:40.881 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.881 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.881 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.881 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.881 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.881 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.881 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.882 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.882 INFO:teuthology.orchestra.run.smithi119.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-03-28T03:44:40.882 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-03-28T03:44:40.882 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.882 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.882 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.882 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:40.882 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.883 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.883 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.883 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.883 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.883 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.883 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.883 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.883 INFO:teuthology.orchestra.run.smithi119.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-03-28T03:44:40.883 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-03-28T03:44:40.884 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.884 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.884 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.884 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2024-03-28T03:44:40.884 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.884 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.884 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.884 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.885 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.885 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.885 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.885 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.885 INFO:teuthology.orchestra.run.smithi119.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-03-28T03:44:40.885 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-03-28T03:44:40.885 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.885 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.886 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.886 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2024-03-28T03:44:40.886 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.886 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.886 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.886 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.886 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.886 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.887 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.887 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.887 INFO:teuthology.orchestra.run.smithi119.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-03-28T03:44:40.887 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-03-28T03:44:40.887 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.887 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.887 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.887 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:40.887 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.888 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.888 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.888 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.888 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.888 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.888 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.888 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.888 INFO:teuthology.orchestra.run.smithi119.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-03-28T03:44:40.889 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-03-28T03:44:40.889 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.889 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.889 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.889 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:40.889 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.889 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.889 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.889 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.890 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.890 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.890 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.890 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.890 INFO:teuthology.orchestra.run.smithi119.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-03-28T03:44:40.890 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-03-28T03:44:40.890 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.890 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.891 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.891 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:40.891 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.891 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.891 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.891 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.891 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.891 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.892 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.892 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.892 INFO:teuthology.orchestra.run.smithi119.stdout: "PWD_POLICY_ENABLED": { 2024-03-28T03:44:40.892 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "PWD_POLICY_ENABLED", 2024-03-28T03:44:40.892 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.892 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.892 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.892 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2024-03-28T03:44:40.893 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.893 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.893 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.893 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.893 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.893 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.893 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.893 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.893 INFO:teuthology.orchestra.run.smithi119.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-03-28T03:44:40.894 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-03-28T03:44:40.894 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.894 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.894 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.894 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-03-28T03:44:40.894 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.894 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.894 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.895 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.895 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.895 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.895 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.895 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.895 INFO:teuthology.orchestra.run.smithi119.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-03-28T03:44:40.895 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-03-28T03:44:40.895 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2024-03-28T03:44:40.895 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.896 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.896 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "10", 2024-03-28T03:44:40.896 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.896 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.896 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.896 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.896 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.896 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.897 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.897 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.897 INFO:teuthology.orchestra.run.smithi119.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-03-28T03:44:40.897 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-03-28T03:44:40.897 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2024-03-28T03:44:40.897 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.897 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.897 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "8", 2024-03-28T03:44:40.898 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.898 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.898 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.898 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.898 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.898 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.898 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.898 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.898 INFO:teuthology.orchestra.run.smithi119.stdout: "REST_REQUESTS_TIMEOUT": { 2024-03-28T03:44:40.899 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-03-28T03:44:40.899 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2024-03-28T03:44:40.899 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.899 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.899 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "45", 2024-03-28T03:44:40.899 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.899 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.899 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.900 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.900 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.900 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.900 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.900 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.900 INFO:teuthology.orchestra.run.smithi119.stdout: "RGW_API_ACCESS_KEY": { 2024-03-28T03:44:40.900 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "RGW_API_ACCESS_KEY", 2024-03-28T03:44:40.900 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.900 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.901 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.901 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:40.901 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.901 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.901 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.901 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.901 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.901 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.901 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.902 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.902 INFO:teuthology.orchestra.run.smithi119.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-03-28T03:44:40.902 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-03-28T03:44:40.902 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.902 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.902 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.902 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "admin", 2024-03-28T03:44:40.902 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.903 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.903 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.903 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.903 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.903 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.903 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.903 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.903 INFO:teuthology.orchestra.run.smithi119.stdout: "RGW_API_SECRET_KEY": { 2024-03-28T03:44:40.903 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "RGW_API_SECRET_KEY", 2024-03-28T03:44:40.904 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.904 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.904 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.904 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:40.904 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.904 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.904 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.904 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.905 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.905 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.905 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.905 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.905 INFO:teuthology.orchestra.run.smithi119.stdout: "RGW_API_SSL_VERIFY": { 2024-03-28T03:44:40.905 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "RGW_API_SSL_VERIFY", 2024-03-28T03:44:40.905 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.905 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.905 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.906 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2024-03-28T03:44:40.906 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.906 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.906 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.906 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.906 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.906 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.906 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.907 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.907 INFO:teuthology.orchestra.run.smithi119.stdout: "UNSAFE_TLS_v1_2": { 2024-03-28T03:44:40.907 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "UNSAFE_TLS_v1_2", 2024-03-28T03:44:40.907 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.907 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.907 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.907 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:40.907 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.907 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.908 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.908 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.908 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.908 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.908 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.908 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.908 INFO:teuthology.orchestra.run.smithi119.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-03-28T03:44:40.908 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-03-28T03:44:40.909 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2024-03-28T03:44:40.909 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.909 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.909 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "0", 2024-03-28T03:44:40.909 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.909 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.909 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.909 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.909 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.910 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.910 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.910 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.910 INFO:teuthology.orchestra.run.smithi119.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-03-28T03:44:40.910 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-03-28T03:44:40.910 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2024-03-28T03:44:40.910 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.910 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.911 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "10", 2024-03-28T03:44:40.911 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.911 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.911 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.911 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.911 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.911 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.911 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.912 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.912 INFO:teuthology.orchestra.run.smithi119.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-03-28T03:44:40.912 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-03-28T03:44:40.912 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2024-03-28T03:44:40.912 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.912 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.912 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "5", 2024-03-28T03:44:40.912 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.912 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.913 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.913 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.913 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.913 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.913 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.913 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.913 INFO:teuthology.orchestra.run.smithi119.stdout: "cross_origin_url": { 2024-03-28T03:44:40.913 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "cross_origin_url", 2024-03-28T03:44:40.914 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.914 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.914 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.914 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:40.914 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.914 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.914 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.914 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.914 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.915 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.915 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.915 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.915 INFO:teuthology.orchestra.run.smithi119.stdout: "crt_file": { 2024-03-28T03:44:40.915 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "crt_file", 2024-03-28T03:44:40.915 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.915 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.915 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.916 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:40.916 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.916 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.916 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.916 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.916 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.916 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.916 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.916 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.917 INFO:teuthology.orchestra.run.smithi119.stdout: "debug": { 2024-03-28T03:44:40.917 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "debug", 2024-03-28T03:44:40.917 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.917 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.917 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.917 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:40.917 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.917 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.917 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.918 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Enable/disable debug options", 2024-03-28T03:44:40.918 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.918 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.918 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.918 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.918 INFO:teuthology.orchestra.run.smithi119.stdout: "jwt_token_ttl": { 2024-03-28T03:44:40.918 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "jwt_token_ttl", 2024-03-28T03:44:40.918 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2024-03-28T03:44:40.919 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.919 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.919 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "28800", 2024-03-28T03:44:40.919 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.919 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.919 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.919 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.919 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.920 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.920 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.920 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.920 INFO:teuthology.orchestra.run.smithi119.stdout: "key_file": { 2024-03-28T03:44:40.920 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "key_file", 2024-03-28T03:44:40.920 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.920 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.920 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.920 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:40.921 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.921 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.921 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.921 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.921 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.921 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.921 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.921 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.922 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2024-03-28T03:44:40.922 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2024-03-28T03:44:40.922 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.922 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.922 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.922 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:40.922 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.922 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.922 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:40.923 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:40.923 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:40.923 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:40.923 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:40.923 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:40.923 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:40.923 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:40.923 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.924 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.924 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.924 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.924 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.924 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2024-03-28T03:44:40.924 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2024-03-28T03:44:40.924 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.924 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.924 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.925 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:40.925 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.925 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.925 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.925 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.925 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.925 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.925 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.926 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.926 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2024-03-28T03:44:40.926 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2024-03-28T03:44:40.926 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.926 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.926 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.926 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2024-03-28T03:44:40.926 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.927 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.927 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:40.927 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:40.927 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:40.927 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:40.927 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:40.927 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:40.927 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:40.927 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:40.928 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.928 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.928 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.928 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.928 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.928 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2024-03-28T03:44:40.928 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2024-03-28T03:44:40.928 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.929 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.929 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.929 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:40.929 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.929 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.929 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.929 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.929 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.929 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.930 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.930 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.930 INFO:teuthology.orchestra.run.smithi119.stdout: "motd": { 2024-03-28T03:44:40.930 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "motd", 2024-03-28T03:44:40.930 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.930 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.930 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.930 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:40.931 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.931 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.931 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.931 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "The message of the day", 2024-03-28T03:44:40.931 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.931 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.931 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.931 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.932 INFO:teuthology.orchestra.run.smithi119.stdout: "redirect_resolve_ip_addr": { 2024-03-28T03:44:40.932 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "redirect_resolve_ip_addr", 2024-03-28T03:44:40.932 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.932 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.932 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.932 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:40.932 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.932 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.932 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.933 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.933 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.933 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.933 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.933 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.933 INFO:teuthology.orchestra.run.smithi119.stdout: "server_addr": { 2024-03-28T03:44:40.933 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "server_addr", 2024-03-28T03:44:40.933 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.933 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.934 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.934 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "::", 2024-03-28T03:44:40.934 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.934 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.934 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.934 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.934 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.934 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.935 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.935 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.935 INFO:teuthology.orchestra.run.smithi119.stdout: "server_port": { 2024-03-28T03:44:40.935 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "server_port", 2024-03-28T03:44:40.935 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2024-03-28T03:44:40.935 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.935 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.935 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "8080", 2024-03-28T03:44:40.935 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.936 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.936 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.936 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.936 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.936 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.936 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.936 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.936 INFO:teuthology.orchestra.run.smithi119.stdout: "ssl": { 2024-03-28T03:44:40.936 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "ssl", 2024-03-28T03:44:40.937 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.937 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.937 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.937 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2024-03-28T03:44:40.937 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.937 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.937 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.937 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.937 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.938 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.938 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.938 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.938 INFO:teuthology.orchestra.run.smithi119.stdout: "ssl_server_port": { 2024-03-28T03:44:40.938 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "ssl_server_port", 2024-03-28T03:44:40.938 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2024-03-28T03:44:40.938 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.938 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.938 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "8443", 2024-03-28T03:44:40.939 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.939 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.939 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.939 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.939 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.939 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.939 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.939 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.940 INFO:teuthology.orchestra.run.smithi119.stdout: "standby_behaviour": { 2024-03-28T03:44:40.940 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "standby_behaviour", 2024-03-28T03:44:40.940 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.940 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.940 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.940 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "redirect", 2024-03-28T03:44:40.940 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.940 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.941 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:40.941 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:40.941 INFO:teuthology.orchestra.run.smithi119.stdout: "redirect" 2024-03-28T03:44:40.941 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:40.941 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.941 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.941 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.941 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.941 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.942 INFO:teuthology.orchestra.run.smithi119.stdout: "standby_error_status_code": { 2024-03-28T03:44:40.942 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "standby_error_status_code", 2024-03-28T03:44:40.942 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2024-03-28T03:44:40.942 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.942 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.942 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "500", 2024-03-28T03:44:40.942 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "400", 2024-03-28T03:44:40.942 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "599", 2024-03-28T03:44:40.943 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.943 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.943 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.943 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.943 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.943 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.943 INFO:teuthology.orchestra.run.smithi119.stdout: "url_prefix": { 2024-03-28T03:44:40.943 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "url_prefix", 2024-03-28T03:44:40.944 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.944 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.944 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.944 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:40.944 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.944 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.944 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.944 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.944 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.945 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.945 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.945 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:40.945 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:40.945 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.945 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-03-28T03:44:40.945 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "devicehealth", 2024-03-28T03:44:40.945 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2024-03-28T03:44:40.945 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2024-03-28T03:44:40.946 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2024-03-28T03:44:40.946 INFO:teuthology.orchestra.run.smithi119.stdout: "enable_monitoring": { 2024-03-28T03:44:40.946 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "enable_monitoring", 2024-03-28T03:44:40.946 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.946 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.946 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.946 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2024-03-28T03:44:40.946 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.947 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.947 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.947 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "monitor device health metrics", 2024-03-28T03:44:40.947 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.947 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.947 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.947 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.947 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2024-03-28T03:44:40.948 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2024-03-28T03:44:40.948 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.948 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.948 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.948 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:40.948 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.948 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.948 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:40.949 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:40.949 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:40.949 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:40.949 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:40.949 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:40.949 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:40.949 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:40.949 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.949 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.950 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.950 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.950 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.950 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2024-03-28T03:44:40.950 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2024-03-28T03:44:40.950 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.950 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.950 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.951 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:40.951 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.951 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.951 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.951 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.951 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.951 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.951 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.952 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.952 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2024-03-28T03:44:40.952 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2024-03-28T03:44:40.952 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.952 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.952 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.952 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2024-03-28T03:44:40.952 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.952 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.953 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:40.953 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:40.953 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:40.953 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:40.953 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:40.953 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:40.953 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:40.953 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:40.954 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.954 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.954 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.954 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.954 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.954 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2024-03-28T03:44:40.954 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2024-03-28T03:44:40.954 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.954 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.955 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.955 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:40.955 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.955 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.955 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.955 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.955 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.955 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.956 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.956 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.956 INFO:teuthology.orchestra.run.smithi119.stdout: "mark_out_threshold": { 2024-03-28T03:44:40.956 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "mark_out_threshold", 2024-03-28T03:44:40.956 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "secs", 2024-03-28T03:44:40.956 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.956 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.956 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "2419200", 2024-03-28T03:44:40.956 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.957 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.957 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.957 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-03-28T03:44:40.957 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.957 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.957 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.957 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.957 INFO:teuthology.orchestra.run.smithi119.stdout: "pool_name": { 2024-03-28T03:44:40.958 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "pool_name", 2024-03-28T03:44:40.958 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.958 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.958 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.958 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "device_health_metrics", 2024-03-28T03:44:40.958 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.958 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.958 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.958 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "name of pool in which to store device health metrics", 2024-03-28T03:44:40.959 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.959 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.959 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.959 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.959 INFO:teuthology.orchestra.run.smithi119.stdout: "retention_period": { 2024-03-28T03:44:40.959 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "retention_period", 2024-03-28T03:44:40.959 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "secs", 2024-03-28T03:44:40.959 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.960 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.960 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "15552000", 2024-03-28T03:44:40.960 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.960 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.960 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.960 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "how long to retain device health metrics", 2024-03-28T03:44:40.960 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.960 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.960 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.961 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.961 INFO:teuthology.orchestra.run.smithi119.stdout: "scrape_frequency": { 2024-03-28T03:44:40.961 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "scrape_frequency", 2024-03-28T03:44:40.961 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "secs", 2024-03-28T03:44:40.961 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.961 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.961 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "86400", 2024-03-28T03:44:40.961 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.961 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.962 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.962 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "how frequently to scrape device health metrics", 2024-03-28T03:44:40.962 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.962 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.962 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.962 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.962 INFO:teuthology.orchestra.run.smithi119.stdout: "self_heal": { 2024-03-28T03:44:40.962 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "self_heal", 2024-03-28T03:44:40.963 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.963 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.963 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.963 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2024-03-28T03:44:40.963 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.963 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.963 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.963 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-03-28T03:44:40.964 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.964 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.964 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.964 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.964 INFO:teuthology.orchestra.run.smithi119.stdout: "sleep_interval": { 2024-03-28T03:44:40.964 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "sleep_interval", 2024-03-28T03:44:40.964 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "secs", 2024-03-28T03:44:40.964 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.964 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.965 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "600", 2024-03-28T03:44:40.965 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.965 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.965 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.965 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "how frequently to wake up and check device health", 2024-03-28T03:44:40.965 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.965 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.965 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.965 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.966 INFO:teuthology.orchestra.run.smithi119.stdout: "warn_threshold": { 2024-03-28T03:44:40.966 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "warn_threshold", 2024-03-28T03:44:40.966 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "secs", 2024-03-28T03:44:40.966 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.966 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.966 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "7257600", 2024-03-28T03:44:40.966 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.966 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.967 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.967 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-03-28T03:44:40.967 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.967 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.967 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.967 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:40.967 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:40.967 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.968 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-03-28T03:44:40.968 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "diskprediction_local", 2024-03-28T03:44:40.968 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2024-03-28T03:44:40.968 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2024-03-28T03:44:40.968 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2024-03-28T03:44:40.968 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2024-03-28T03:44:40.968 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2024-03-28T03:44:40.968 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.968 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.969 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.969 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:40.969 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.969 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.969 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:40.969 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:40.969 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:40.969 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:40.969 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:40.970 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:40.970 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:40.970 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:40.970 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.970 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.970 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.970 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.970 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.971 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2024-03-28T03:44:40.971 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2024-03-28T03:44:40.971 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.971 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.971 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.971 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:40.971 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.971 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.971 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.972 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.972 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.972 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.972 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.972 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.972 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2024-03-28T03:44:40.972 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2024-03-28T03:44:40.972 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.972 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.973 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.973 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2024-03-28T03:44:40.973 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.973 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.973 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:40.973 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:40.973 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:40.973 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:40.974 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:40.974 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:40.974 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:40.974 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:40.974 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.974 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.974 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.974 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.974 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.975 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2024-03-28T03:44:40.975 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2024-03-28T03:44:40.975 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.975 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.975 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.975 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:40.975 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.975 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.975 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.976 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.976 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.976 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.976 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.976 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.976 INFO:teuthology.orchestra.run.smithi119.stdout: "predict_interval": { 2024-03-28T03:44:40.976 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "predict_interval", 2024-03-28T03:44:40.976 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.977 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.977 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.977 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "86400", 2024-03-28T03:44:40.977 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.977 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.977 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.977 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.977 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.977 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.978 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.978 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.978 INFO:teuthology.orchestra.run.smithi119.stdout: "predictor_model": { 2024-03-28T03:44:40.978 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "predictor_model", 2024-03-28T03:44:40.978 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.978 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.978 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.978 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "prophetstor", 2024-03-28T03:44:40.979 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.979 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.979 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.979 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.979 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.979 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.979 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.979 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.980 INFO:teuthology.orchestra.run.smithi119.stdout: "sleep_interval": { 2024-03-28T03:44:40.980 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "sleep_interval", 2024-03-28T03:44:40.980 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.980 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.980 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.980 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "600", 2024-03-28T03:44:40.980 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.980 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.980 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.981 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.981 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.981 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.981 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.981 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:40.981 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:40.981 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.981 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-03-28T03:44:40.981 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "influx", 2024-03-28T03:44:40.982 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": false, 2024-03-28T03:44:40.982 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "influxdb python module not found", 2024-03-28T03:44:40.982 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2024-03-28T03:44:40.982 INFO:teuthology.orchestra.run.smithi119.stdout: "batch_size": { 2024-03-28T03:44:40.982 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "batch_size", 2024-03-28T03:44:40.982 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2024-03-28T03:44:40.982 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.982 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.982 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "5000", 2024-03-28T03:44:40.983 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.983 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.983 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.983 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-03-28T03:44:40.983 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.983 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.983 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.983 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.984 INFO:teuthology.orchestra.run.smithi119.stdout: "database": { 2024-03-28T03:44:40.984 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "database", 2024-03-28T03:44:40.984 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.984 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.984 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.984 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "ceph", 2024-03-28T03:44:40.984 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.984 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.985 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.985 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-03-28T03:44:40.985 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.985 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.985 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.985 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.985 INFO:teuthology.orchestra.run.smithi119.stdout: "hostname": { 2024-03-28T03:44:40.985 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "hostname", 2024-03-28T03:44:40.985 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.986 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.986 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.986 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:40.986 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.986 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.986 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.986 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "InfluxDB server hostname", 2024-03-28T03:44:40.986 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.987 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.987 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.987 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.987 INFO:teuthology.orchestra.run.smithi119.stdout: "interval": { 2024-03-28T03:44:40.987 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "interval", 2024-03-28T03:44:40.987 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "secs", 2024-03-28T03:44:40.987 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.987 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.987 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "30", 2024-03-28T03:44:40.988 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "5", 2024-03-28T03:44:40.988 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.988 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.988 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-03-28T03:44:40.988 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.988 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.988 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.988 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.989 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2024-03-28T03:44:40.989 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2024-03-28T03:44:40.989 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.989 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.989 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.989 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:40.989 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.989 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.990 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:40.990 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:40.990 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:40.990 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:40.990 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:40.990 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:40.990 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:40.990 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:40.991 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.991 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.991 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.991 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.991 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.991 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2024-03-28T03:44:40.991 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2024-03-28T03:44:40.991 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.991 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.992 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.992 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:40.992 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.992 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.992 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.992 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.992 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.992 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.992 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.993 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.993 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2024-03-28T03:44:40.993 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2024-03-28T03:44:40.993 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.993 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.993 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.993 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2024-03-28T03:44:40.993 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.994 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.994 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:40.994 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:40.994 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:40.994 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:40.994 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:40.994 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:40.994 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:40.995 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:40.995 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.995 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.995 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.995 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.995 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.995 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2024-03-28T03:44:40.995 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2024-03-28T03:44:40.995 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:40.996 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.996 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:40.996 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:40.996 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.996 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.996 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.996 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:40.996 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.996 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.997 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.997 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.997 INFO:teuthology.orchestra.run.smithi119.stdout: "password": { 2024-03-28T03:44:40.997 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "password", 2024-03-28T03:44:40.997 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:40.997 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.997 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.997 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:40.998 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.998 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.998 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.998 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "password of InfluxDB server user", 2024-03-28T03:44:40.998 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:40.998 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:40.998 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:40.998 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:40.998 INFO:teuthology.orchestra.run.smithi119.stdout: "port": { 2024-03-28T03:44:40.999 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "port", 2024-03-28T03:44:40.999 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2024-03-28T03:44:40.999 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:40.999 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:40.999 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "8086", 2024-03-28T03:44:40.999 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:40.999 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:40.999 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:40.999 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "InfluxDB server port", 2024-03-28T03:44:41.000 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.000 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.000 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.000 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.000 INFO:teuthology.orchestra.run.smithi119.stdout: "ssl": { 2024-03-28T03:44:41.000 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "ssl", 2024-03-28T03:44:41.000 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.000 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.000 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.001 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "false", 2024-03-28T03:44:41.001 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.001 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.001 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.001 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-03-28T03:44:41.001 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.001 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.002 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.002 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.002 INFO:teuthology.orchestra.run.smithi119.stdout: "threads": { 2024-03-28T03:44:41.002 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "threads", 2024-03-28T03:44:41.002 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2024-03-28T03:44:41.002 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.002 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.002 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "5", 2024-03-28T03:44:41.002 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "1", 2024-03-28T03:44:41.003 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "32", 2024-03-28T03:44:41.003 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.003 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-03-28T03:44:41.003 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.003 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.003 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.003 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.003 INFO:teuthology.orchestra.run.smithi119.stdout: "username": { 2024-03-28T03:44:41.003 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "username", 2024-03-28T03:44:41.004 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.004 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.004 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.004 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:41.004 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.004 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.004 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.004 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "username of InfluxDB server user", 2024-03-28T03:44:41.004 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.005 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.005 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.005 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.005 INFO:teuthology.orchestra.run.smithi119.stdout: "verify_ssl": { 2024-03-28T03:44:41.005 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "verify_ssl", 2024-03-28T03:44:41.005 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.005 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.005 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.006 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "true", 2024-03-28T03:44:41.006 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.006 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.006 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.006 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-03-28T03:44:41.006 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.006 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.006 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.006 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:41.007 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:41.007 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.007 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-03-28T03:44:41.007 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "insights", 2024-03-28T03:44:41.007 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2024-03-28T03:44:41.007 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2024-03-28T03:44:41.007 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2024-03-28T03:44:41.007 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2024-03-28T03:44:41.008 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2024-03-28T03:44:41.008 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.008 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.008 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.008 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:41.008 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.008 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.008 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:41.008 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:41.009 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:41.009 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:41.009 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:41.009 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:41.009 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:41.009 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:41.009 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.009 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.010 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.010 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.010 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.010 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2024-03-28T03:44:41.010 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2024-03-28T03:44:41.010 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.010 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.010 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.010 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:41.011 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.011 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.011 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.011 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.011 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.011 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.011 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.011 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.012 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2024-03-28T03:44:41.012 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2024-03-28T03:44:41.012 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.012 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.012 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.012 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2024-03-28T03:44:41.012 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.012 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.012 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:41.013 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:41.013 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:41.013 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:41.013 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:41.013 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:41.013 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:41.013 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:41.013 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.013 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.014 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.014 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.014 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.014 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2024-03-28T03:44:41.014 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2024-03-28T03:44:41.014 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.014 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.014 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.015 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:41.015 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.015 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.015 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.015 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.015 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.015 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.015 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.015 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:41.016 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:41.016 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.016 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-03-28T03:44:41.016 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "iostat", 2024-03-28T03:44:41.016 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2024-03-28T03:44:41.016 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2024-03-28T03:44:41.016 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2024-03-28T03:44:41.016 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2024-03-28T03:44:41.017 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2024-03-28T03:44:41.017 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.017 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.017 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.017 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:41.017 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.017 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.017 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:41.017 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:41.018 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:41.018 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:41.018 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:41.018 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:41.018 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:41.018 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:41.018 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.018 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.019 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.019 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.019 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.019 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2024-03-28T03:44:41.019 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2024-03-28T03:44:41.019 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.019 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.019 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.020 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:41.020 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.020 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.020 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.020 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.020 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.020 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.020 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.020 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.021 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2024-03-28T03:44:41.021 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2024-03-28T03:44:41.021 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.021 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.021 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.021 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2024-03-28T03:44:41.021 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.021 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.021 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:41.022 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:41.022 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:41.022 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:41.022 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:41.022 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:41.022 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:41.022 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:41.022 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.023 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.023 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.023 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.023 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.023 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2024-03-28T03:44:41.023 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2024-03-28T03:44:41.023 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.023 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.023 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.024 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:41.024 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.024 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.024 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.024 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.024 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.024 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.024 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.025 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:41.025 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:41.025 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.025 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-03-28T03:44:41.025 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "k8sevents", 2024-03-28T03:44:41.025 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2024-03-28T03:44:41.025 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2024-03-28T03:44:41.025 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2024-03-28T03:44:41.025 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph_event_retention_days": { 2024-03-28T03:44:41.026 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "ceph_event_retention_days", 2024-03-28T03:44:41.026 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2024-03-28T03:44:41.026 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.026 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.026 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "7", 2024-03-28T03:44:41.026 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.026 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.026 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.026 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Days to hold ceph event information within local cache", 2024-03-28T03:44:41.027 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.027 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.027 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.027 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.027 INFO:teuthology.orchestra.run.smithi119.stdout: "config_check_secs": { 2024-03-28T03:44:41.027 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "config_check_secs", 2024-03-28T03:44:41.027 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2024-03-28T03:44:41.027 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.028 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.028 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "10", 2024-03-28T03:44:41.028 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "10", 2024-03-28T03:44:41.028 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.028 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.028 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-03-28T03:44:41.028 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.028 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.028 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.029 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.029 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2024-03-28T03:44:41.029 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2024-03-28T03:44:41.029 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.029 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.029 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.029 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:41.029 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.030 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.030 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:41.030 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:41.030 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:41.030 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:41.030 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:41.030 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:41.030 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:41.031 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:41.031 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.031 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.031 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.031 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.031 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.031 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2024-03-28T03:44:41.031 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2024-03-28T03:44:41.031 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.032 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.032 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.032 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:41.032 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.032 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.032 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.032 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.032 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.032 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.033 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.033 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.033 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2024-03-28T03:44:41.033 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2024-03-28T03:44:41.033 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.033 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.033 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.033 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2024-03-28T03:44:41.034 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.034 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.034 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:41.034 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:41.034 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:41.034 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:41.034 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:41.034 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:41.034 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:41.035 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:41.035 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.035 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.035 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.035 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.035 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.035 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2024-03-28T03:44:41.035 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2024-03-28T03:44:41.035 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.036 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.036 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.036 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:41.036 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.036 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.036 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.036 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.036 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.037 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.037 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.037 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:41.037 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:41.037 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.037 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-03-28T03:44:41.037 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "localpool", 2024-03-28T03:44:41.037 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2024-03-28T03:44:41.037 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2024-03-28T03:44:41.038 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2024-03-28T03:44:41.038 INFO:teuthology.orchestra.run.smithi119.stdout: "failure_domain": { 2024-03-28T03:44:41.038 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "failure_domain", 2024-03-28T03:44:41.038 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.038 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.038 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.038 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "host", 2024-03-28T03:44:41.038 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.039 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.039 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.039 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "failure domain for any created local pool", 2024-03-28T03:44:41.039 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-03-28T03:44:41.039 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.039 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.039 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.039 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2024-03-28T03:44:41.039 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2024-03-28T03:44:41.040 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.040 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.040 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.040 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:41.040 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.040 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.040 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:41.040 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:41.041 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:41.041 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:41.041 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:41.041 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:41.041 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:41.041 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:41.041 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.041 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.041 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.042 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.042 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.042 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2024-03-28T03:44:41.042 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2024-03-28T03:44:41.042 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.042 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.042 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.042 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:41.042 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.043 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.043 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.043 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.043 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.043 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.043 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.043 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.043 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2024-03-28T03:44:41.043 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2024-03-28T03:44:41.044 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.044 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.044 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.044 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2024-03-28T03:44:41.044 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.044 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.044 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:41.044 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:41.044 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:41.045 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:41.045 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:41.045 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:41.045 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:41.045 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:41.045 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.045 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.045 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.045 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.046 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.046 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2024-03-28T03:44:41.046 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2024-03-28T03:44:41.046 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.046 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.046 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.046 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:41.046 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.047 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.047 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.047 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.047 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.047 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.047 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.047 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.047 INFO:teuthology.orchestra.run.smithi119.stdout: "min_size": { 2024-03-28T03:44:41.047 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "min_size", 2024-03-28T03:44:41.048 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2024-03-28T03:44:41.048 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.048 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.048 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:41.048 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.048 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.048 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.048 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "default min_size for any created local pool", 2024-03-28T03:44:41.048 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-03-28T03:44:41.049 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.049 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.049 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.049 INFO:teuthology.orchestra.run.smithi119.stdout: "num_rep": { 2024-03-28T03:44:41.049 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "num_rep", 2024-03-28T03:44:41.049 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2024-03-28T03:44:41.049 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.049 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.049 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "3", 2024-03-28T03:44:41.050 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.050 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.050 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.050 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "default replica count for any created local pool", 2024-03-28T03:44:41.050 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.050 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.050 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.050 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.051 INFO:teuthology.orchestra.run.smithi119.stdout: "pg_num": { 2024-03-28T03:44:41.051 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "pg_num", 2024-03-28T03:44:41.051 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2024-03-28T03:44:41.051 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.051 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.051 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "128", 2024-03-28T03:44:41.051 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.051 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.051 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.052 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "default pg_num for any created local pool", 2024-03-28T03:44:41.052 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.052 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.052 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.052 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.052 INFO:teuthology.orchestra.run.smithi119.stdout: "prefix": { 2024-03-28T03:44:41.053 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "prefix", 2024-03-28T03:44:41.053 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.053 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.053 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.053 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:41.053 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.053 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.053 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.053 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "name prefix for any created local pool", 2024-03-28T03:44:41.054 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.054 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.054 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.054 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.054 INFO:teuthology.orchestra.run.smithi119.stdout: "subtree": { 2024-03-28T03:44:41.054 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "subtree", 2024-03-28T03:44:41.054 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.054 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.055 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.055 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "rack", 2024-03-28T03:44:41.055 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.055 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.055 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.055 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "CRUSH level for which to create a local pool", 2024-03-28T03:44:41.055 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-03-28T03:44:41.055 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.056 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.056 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:41.056 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:41.056 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.056 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-03-28T03:44:41.056 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "mds_autoscaler", 2024-03-28T03:44:41.056 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2024-03-28T03:44:41.056 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2024-03-28T03:44:41.056 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2024-03-28T03:44:41.057 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2024-03-28T03:44:41.057 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2024-03-28T03:44:41.057 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.057 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.057 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.057 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:41.057 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.057 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.057 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:41.058 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:41.058 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:41.058 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:41.058 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:41.058 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:41.058 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:41.058 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:41.058 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.059 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.059 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.059 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.059 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.059 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2024-03-28T03:44:41.059 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2024-03-28T03:44:41.059 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.059 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.060 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.060 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:41.060 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.060 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.060 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.060 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.060 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.060 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.060 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.061 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.061 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2024-03-28T03:44:41.061 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2024-03-28T03:44:41.061 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.061 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.061 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.061 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2024-03-28T03:44:41.061 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.062 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.062 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:41.062 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:41.062 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:41.062 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:41.062 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:41.062 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:41.062 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:41.062 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:41.063 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.063 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.063 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.063 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.063 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.063 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2024-03-28T03:44:41.063 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2024-03-28T03:44:41.063 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.064 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.064 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.064 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:41.064 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.064 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.064 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.064 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.064 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.064 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.065 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.065 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:41.065 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:41.065 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.065 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-03-28T03:44:41.065 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "mirroring", 2024-03-28T03:44:41.065 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2024-03-28T03:44:41.065 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2024-03-28T03:44:41.065 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2024-03-28T03:44:41.066 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2024-03-28T03:44:41.066 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2024-03-28T03:44:41.066 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.066 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.066 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.066 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:41.066 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.066 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.067 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:41.067 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:41.067 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:41.067 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:41.067 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:41.067 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:41.067 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:41.067 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:41.067 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.068 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.068 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.068 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.068 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.068 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2024-03-28T03:44:41.068 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2024-03-28T03:44:41.068 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.068 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.068 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.069 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:41.069 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.069 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.069 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.069 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.069 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.069 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.069 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.069 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.070 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2024-03-28T03:44:41.070 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2024-03-28T03:44:41.070 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.070 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.070 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.070 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2024-03-28T03:44:41.070 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.070 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.071 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:41.071 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:41.071 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:41.071 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:41.071 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:41.071 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:41.071 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:41.071 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:41.071 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.072 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.072 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.072 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.072 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.072 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2024-03-28T03:44:41.072 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2024-03-28T03:44:41.072 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.072 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.073 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.073 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:41.073 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.073 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.073 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.073 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.073 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.073 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.073 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.074 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:41.074 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:41.074 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.074 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-03-28T03:44:41.074 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "nfs", 2024-03-28T03:44:41.074 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2024-03-28T03:44:41.074 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2024-03-28T03:44:41.074 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2024-03-28T03:44:41.074 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2024-03-28T03:44:41.075 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2024-03-28T03:44:41.075 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.075 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.075 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.075 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:41.075 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.075 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.075 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:41.076 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:41.076 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:41.076 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:41.076 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:41.076 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:41.076 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:41.076 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:41.076 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.076 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.077 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.077 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.077 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.077 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2024-03-28T03:44:41.077 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2024-03-28T03:44:41.077 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.077 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.077 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.077 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:41.078 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.078 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.078 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.078 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.078 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.078 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.078 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.078 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.079 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2024-03-28T03:44:41.079 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2024-03-28T03:44:41.079 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.079 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.079 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.079 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2024-03-28T03:44:41.079 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.079 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.079 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:41.080 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:41.080 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:41.080 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:41.080 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:41.080 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:41.080 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:41.080 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:41.080 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.080 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.081 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.081 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.081 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.081 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2024-03-28T03:44:41.081 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2024-03-28T03:44:41.081 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.081 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.081 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.081 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:41.082 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.082 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.082 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.082 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.082 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.082 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.082 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.082 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:41.083 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:41.083 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.083 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-03-28T03:44:41.083 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "orchestrator", 2024-03-28T03:44:41.083 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2024-03-28T03:44:41.083 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2024-03-28T03:44:41.083 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2024-03-28T03:44:41.083 INFO:teuthology.orchestra.run.smithi119.stdout: "fail_fs": { 2024-03-28T03:44:41.083 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "fail_fs", 2024-03-28T03:44:41.084 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.084 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.084 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.084 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:41.084 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.084 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.084 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.084 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-03-28T03:44:41.085 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.085 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.085 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.085 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.085 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2024-03-28T03:44:41.085 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2024-03-28T03:44:41.085 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.085 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.086 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.086 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:41.086 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.086 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.086 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:41.086 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:41.086 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:41.086 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:41.086 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:41.087 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:41.087 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:41.087 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:41.087 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.087 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.087 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.087 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.087 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.088 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2024-03-28T03:44:41.088 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2024-03-28T03:44:41.088 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.088 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.088 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.088 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:41.088 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.088 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.088 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.089 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.089 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.089 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.089 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.089 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.089 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2024-03-28T03:44:41.089 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2024-03-28T03:44:41.089 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.090 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.090 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.090 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2024-03-28T03:44:41.090 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.090 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.090 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:41.090 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:41.090 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:41.090 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:41.091 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:41.091 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:41.091 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:41.091 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:41.091 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.091 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.091 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.091 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.092 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.092 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2024-03-28T03:44:41.092 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2024-03-28T03:44:41.092 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.092 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.092 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.092 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:41.092 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.092 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.093 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.093 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.093 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.093 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.093 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.093 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.093 INFO:teuthology.orchestra.run.smithi119.stdout: "orchestrator": { 2024-03-28T03:44:41.093 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "orchestrator", 2024-03-28T03:44:41.093 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.094 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.094 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.094 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:41.094 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.094 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.094 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:41.094 INFO:teuthology.orchestra.run.smithi119.stdout: "cephadm", 2024-03-28T03:44:41.094 INFO:teuthology.orchestra.run.smithi119.stdout: "rook", 2024-03-28T03:44:41.094 INFO:teuthology.orchestra.run.smithi119.stdout: "test_orchestrator" 2024-03-28T03:44:41.095 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:41.095 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Orchestrator backend", 2024-03-28T03:44:41.095 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.095 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.095 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.095 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:41.095 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:41.095 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.095 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-03-28T03:44:41.096 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "osd_perf_query", 2024-03-28T03:44:41.096 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2024-03-28T03:44:41.096 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2024-03-28T03:44:41.096 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2024-03-28T03:44:41.096 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2024-03-28T03:44:41.096 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2024-03-28T03:44:41.096 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.096 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.096 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.097 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:41.097 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.097 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.097 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:41.097 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:41.097 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:41.097 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:41.097 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:41.098 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:41.098 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:41.098 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:41.098 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.098 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.098 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.098 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.098 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.099 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2024-03-28T03:44:41.099 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2024-03-28T03:44:41.099 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.099 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.099 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.099 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:41.099 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.099 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.099 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.100 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.100 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.100 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.100 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.100 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.100 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2024-03-28T03:44:41.100 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2024-03-28T03:44:41.100 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.100 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.101 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.101 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2024-03-28T03:44:41.101 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.101 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.101 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:41.101 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:41.101 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:41.101 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:41.102 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:41.102 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:41.102 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:41.102 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:41.102 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.102 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.102 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.102 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.103 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.103 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2024-03-28T03:44:41.103 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2024-03-28T03:44:41.103 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.103 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.103 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.103 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:41.103 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.103 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.104 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.104 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.104 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.104 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.104 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.104 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:41.104 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:41.104 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.105 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-03-28T03:44:41.105 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "osd_support", 2024-03-28T03:44:41.105 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2024-03-28T03:44:41.105 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2024-03-28T03:44:41.105 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2024-03-28T03:44:41.105 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2024-03-28T03:44:41.105 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2024-03-28T03:44:41.105 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.105 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.106 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.106 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:41.106 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.106 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.106 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:41.106 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:41.106 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:41.106 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:41.107 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:41.107 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:41.107 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:41.107 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:41.107 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.107 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.107 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.107 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.108 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.108 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2024-03-28T03:44:41.108 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2024-03-28T03:44:41.108 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.108 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.108 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.108 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:41.108 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.108 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.109 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.109 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.109 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.109 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.109 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.109 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.109 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2024-03-28T03:44:41.109 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2024-03-28T03:44:41.109 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.110 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.110 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.110 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2024-03-28T03:44:41.110 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.110 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.110 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:41.110 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:41.110 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:41.111 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:41.111 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:41.111 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:41.111 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:41.111 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:41.111 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.111 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.111 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.111 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.112 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.112 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2024-03-28T03:44:41.112 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2024-03-28T03:44:41.112 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.112 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.112 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.112 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:41.112 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.113 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.113 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.113 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.113 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.113 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.113 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.113 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:41.113 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:41.113 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.114 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-03-28T03:44:41.114 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "pg_autoscaler", 2024-03-28T03:44:41.114 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2024-03-28T03:44:41.114 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2024-03-28T03:44:41.114 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2024-03-28T03:44:41.114 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2024-03-28T03:44:41.114 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2024-03-28T03:44:41.114 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.115 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.115 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.115 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:41.115 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.115 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.115 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:41.115 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:41.115 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:41.116 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:41.116 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:41.116 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:41.116 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:41.116 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:41.116 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.116 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.116 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.116 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.117 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.117 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2024-03-28T03:44:41.117 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2024-03-28T03:44:41.117 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.117 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.117 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.117 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:41.117 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.117 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.118 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.118 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.118 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.118 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.118 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.118 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.118 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2024-03-28T03:44:41.118 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2024-03-28T03:44:41.119 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.119 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.119 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.119 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2024-03-28T03:44:41.119 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.119 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.119 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:41.119 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:41.119 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:41.120 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:41.120 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:41.120 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:41.120 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:41.120 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:41.120 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.120 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.120 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.121 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.121 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.121 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2024-03-28T03:44:41.121 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2024-03-28T03:44:41.121 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.121 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.121 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.121 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:41.122 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.122 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.122 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.122 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.122 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.122 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.122 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.122 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.123 INFO:teuthology.orchestra.run.smithi119.stdout: "sleep_interval": { 2024-03-28T03:44:41.123 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "sleep_interval", 2024-03-28T03:44:41.123 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "secs", 2024-03-28T03:44:41.123 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.123 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.123 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "60", 2024-03-28T03:44:41.123 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.123 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.123 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.124 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.124 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.124 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.124 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.124 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.124 INFO:teuthology.orchestra.run.smithi119.stdout: "threshold": { 2024-03-28T03:44:41.124 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "threshold", 2024-03-28T03:44:41.124 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "float", 2024-03-28T03:44:41.125 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.125 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.125 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "3.0", 2024-03-28T03:44:41.125 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "1.0", 2024-03-28T03:44:41.125 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.125 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.125 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "scaling threshold", 2024-03-28T03:44:41.125 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-03-28T03:44:41.125 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.126 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.126 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:41.126 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:41.126 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.126 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-03-28T03:44:41.126 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "progress", 2024-03-28T03:44:41.126 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2024-03-28T03:44:41.126 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2024-03-28T03:44:41.127 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2024-03-28T03:44:41.127 INFO:teuthology.orchestra.run.smithi119.stdout: "allow_pg_recovery_event": { 2024-03-28T03:44:41.127 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "allow_pg_recovery_event", 2024-03-28T03:44:41.127 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.127 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.127 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.127 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:41.127 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.127 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.128 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.128 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "allow the module to show pg recovery progress", 2024-03-28T03:44:41.128 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.128 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.128 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.128 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.128 INFO:teuthology.orchestra.run.smithi119.stdout: "enabled": { 2024-03-28T03:44:41.128 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "enabled", 2024-03-28T03:44:41.129 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.129 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.129 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.129 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2024-03-28T03:44:41.129 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.129 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.129 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.129 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.130 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.130 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.130 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.130 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.130 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2024-03-28T03:44:41.130 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2024-03-28T03:44:41.130 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.130 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.131 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.131 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:41.131 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.131 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.131 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:41.131 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:41.131 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:41.131 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:41.131 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:41.132 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:41.132 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:41.132 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:41.132 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.132 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.132 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.132 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.132 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.132 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2024-03-28T03:44:41.133 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2024-03-28T03:44:41.133 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.133 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.133 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.133 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:41.133 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.133 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.133 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.134 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.134 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.134 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.134 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.134 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.134 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2024-03-28T03:44:41.134 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2024-03-28T03:44:41.134 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.134 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.135 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.135 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2024-03-28T03:44:41.135 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.135 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.135 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:41.135 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:41.135 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:41.135 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:41.135 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:41.136 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:41.136 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:41.136 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:41.136 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.136 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.136 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.136 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.136 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.137 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2024-03-28T03:44:41.137 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2024-03-28T03:44:41.137 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.137 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.137 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.137 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:41.137 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.137 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.137 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.138 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.138 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.138 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.138 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.138 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.138 INFO:teuthology.orchestra.run.smithi119.stdout: "max_completed_events": { 2024-03-28T03:44:41.138 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "max_completed_events", 2024-03-28T03:44:41.138 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2024-03-28T03:44:41.139 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.139 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.139 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "50", 2024-03-28T03:44:41.139 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.139 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.139 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.139 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "number of past completed events to remember", 2024-03-28T03:44:41.139 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.139 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.140 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.140 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.140 INFO:teuthology.orchestra.run.smithi119.stdout: "sleep_interval": { 2024-03-28T03:44:41.140 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "sleep_interval", 2024-03-28T03:44:41.140 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "secs", 2024-03-28T03:44:41.140 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.140 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.140 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "5", 2024-03-28T03:44:41.141 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.141 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.141 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.141 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "how long the module is going to sleep", 2024-03-28T03:44:41.141 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.141 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.141 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.141 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:41.141 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:41.142 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.142 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-03-28T03:44:41.142 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "prometheus", 2024-03-28T03:44:41.142 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2024-03-28T03:44:41.142 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2024-03-28T03:44:41.142 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2024-03-28T03:44:41.142 INFO:teuthology.orchestra.run.smithi119.stdout: "cache": { 2024-03-28T03:44:41.142 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "cache", 2024-03-28T03:44:41.142 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.143 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.143 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.143 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2024-03-28T03:44:41.143 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.143 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.143 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.143 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.143 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.144 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.144 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.144 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.144 INFO:teuthology.orchestra.run.smithi119.stdout: "exclude_perf_counters": { 2024-03-28T03:44:41.144 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "exclude_perf_counters", 2024-03-28T03:44:41.144 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.144 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.144 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.144 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2024-03-28T03:44:41.145 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.145 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.145 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.145 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-03-28T03:44:41.145 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-03-28T03:44:41.145 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.145 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.145 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.146 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2024-03-28T03:44:41.146 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2024-03-28T03:44:41.146 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.146 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.146 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.146 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:41.146 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.146 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.146 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:41.147 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:41.147 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:41.147 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:41.147 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:41.147 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:41.147 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:41.147 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:41.147 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.147 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.148 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.148 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.148 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.148 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2024-03-28T03:44:41.148 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2024-03-28T03:44:41.148 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.148 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.148 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.148 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:41.149 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.149 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.149 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.149 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.149 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.149 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.149 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.149 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.150 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2024-03-28T03:44:41.150 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2024-03-28T03:44:41.150 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.150 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.150 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.150 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2024-03-28T03:44:41.150 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.150 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.151 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:41.151 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:41.151 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:41.151 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:41.151 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:41.151 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:41.151 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:41.151 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:41.151 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.152 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.152 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.152 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.152 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.152 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2024-03-28T03:44:41.152 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2024-03-28T03:44:41.152 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.152 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.153 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.153 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:41.153 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.153 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.153 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.153 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.153 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.153 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.153 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.154 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.154 INFO:teuthology.orchestra.run.smithi119.stdout: "rbd_stats_pools": { 2024-03-28T03:44:41.154 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "rbd_stats_pools", 2024-03-28T03:44:41.154 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.154 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.154 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.154 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:41.154 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.155 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.155 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.155 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.155 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.155 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.155 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.155 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.155 INFO:teuthology.orchestra.run.smithi119.stdout: "rbd_stats_pools_refresh_interval": { 2024-03-28T03:44:41.155 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-03-28T03:44:41.156 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2024-03-28T03:44:41.156 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.156 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.156 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "300", 2024-03-28T03:44:41.156 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.156 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.156 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.156 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.157 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.157 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.157 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.157 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.157 INFO:teuthology.orchestra.run.smithi119.stdout: "scrape_interval": { 2024-03-28T03:44:41.157 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "scrape_interval", 2024-03-28T03:44:41.157 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "float", 2024-03-28T03:44:41.157 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.157 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.158 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "15.0", 2024-03-28T03:44:41.158 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.158 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.158 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.158 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.158 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.158 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.158 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.159 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.159 INFO:teuthology.orchestra.run.smithi119.stdout: "server_addr": { 2024-03-28T03:44:41.159 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "server_addr", 2024-03-28T03:44:41.159 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.159 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.159 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.159 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "::", 2024-03-28T03:44:41.159 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.160 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.160 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.160 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-03-28T03:44:41.160 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.160 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.160 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.160 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.160 INFO:teuthology.orchestra.run.smithi119.stdout: "server_port": { 2024-03-28T03:44:41.160 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "server_port", 2024-03-28T03:44:41.161 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2024-03-28T03:44:41.161 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.161 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.161 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "9283", 2024-03-28T03:44:41.161 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.161 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.161 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.161 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-03-28T03:44:41.162 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.162 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.162 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.162 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.162 INFO:teuthology.orchestra.run.smithi119.stdout: "stale_cache_strategy": { 2024-03-28T03:44:41.162 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "stale_cache_strategy", 2024-03-28T03:44:41.162 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.162 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.162 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.163 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "log", 2024-03-28T03:44:41.163 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.163 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.163 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.163 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.163 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.163 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.163 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.164 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.164 INFO:teuthology.orchestra.run.smithi119.stdout: "standby_behaviour": { 2024-03-28T03:44:41.164 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "standby_behaviour", 2024-03-28T03:44:41.164 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.164 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.164 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.164 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "default", 2024-03-28T03:44:41.164 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.164 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.165 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:41.165 INFO:teuthology.orchestra.run.smithi119.stdout: "default", 2024-03-28T03:44:41.165 INFO:teuthology.orchestra.run.smithi119.stdout: "error" 2024-03-28T03:44:41.165 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:41.165 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.165 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.165 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.165 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.166 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.166 INFO:teuthology.orchestra.run.smithi119.stdout: "standby_error_status_code": { 2024-03-28T03:44:41.166 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "standby_error_status_code", 2024-03-28T03:44:41.166 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2024-03-28T03:44:41.166 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.166 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.166 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "500", 2024-03-28T03:44:41.166 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "400", 2024-03-28T03:44:41.167 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "599", 2024-03-28T03:44:41.167 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.167 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.167 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.167 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.167 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.167 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:41.167 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:41.167 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.168 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-03-28T03:44:41.168 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "rbd_support", 2024-03-28T03:44:41.168 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2024-03-28T03:44:41.168 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2024-03-28T03:44:41.168 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2024-03-28T03:44:41.168 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2024-03-28T03:44:41.168 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2024-03-28T03:44:41.168 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.168 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.169 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.169 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:41.169 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.169 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.169 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:41.169 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:41.169 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:41.169 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:41.169 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:41.170 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:41.170 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:41.170 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:41.170 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.170 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.170 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.170 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.170 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.171 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2024-03-28T03:44:41.171 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2024-03-28T03:44:41.171 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.171 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.171 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.171 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:41.171 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.171 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.172 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.172 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.172 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.172 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.172 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.172 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.172 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2024-03-28T03:44:41.172 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2024-03-28T03:44:41.172 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.173 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.173 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.173 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2024-03-28T03:44:41.173 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.173 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.173 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:41.173 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:41.173 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:41.173 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:41.174 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:41.174 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:41.174 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:41.174 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:41.174 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.174 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.174 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.174 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.175 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.175 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2024-03-28T03:44:41.175 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2024-03-28T03:44:41.175 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.175 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.175 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.175 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:41.175 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.175 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.176 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.176 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.176 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.176 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.176 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.176 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.176 INFO:teuthology.orchestra.run.smithi119.stdout: "max_concurrent_snap_create": { 2024-03-28T03:44:41.176 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "max_concurrent_snap_create", 2024-03-28T03:44:41.176 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2024-03-28T03:44:41.177 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.177 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.177 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "10", 2024-03-28T03:44:41.177 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.177 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.177 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.177 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.177 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.178 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.178 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.178 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.178 INFO:teuthology.orchestra.run.smithi119.stdout: "mirror_snapshot_schedule": { 2024-03-28T03:44:41.178 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "mirror_snapshot_schedule", 2024-03-28T03:44:41.178 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.178 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.178 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.179 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:41.179 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.179 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.179 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.179 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.179 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.179 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.179 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.179 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.180 INFO:teuthology.orchestra.run.smithi119.stdout: "trash_purge_schedule": { 2024-03-28T03:44:41.180 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "trash_purge_schedule", 2024-03-28T03:44:41.180 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.180 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.180 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.180 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:41.180 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.180 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.180 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.181 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.181 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.181 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.181 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.181 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:41.181 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:41.181 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.181 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-03-28T03:44:41.181 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "restful", 2024-03-28T03:44:41.182 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2024-03-28T03:44:41.182 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2024-03-28T03:44:41.182 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2024-03-28T03:44:41.182 INFO:teuthology.orchestra.run.smithi119.stdout: "enable_auth": { 2024-03-28T03:44:41.182 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "enable_auth", 2024-03-28T03:44:41.182 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.182 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.182 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.182 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2024-03-28T03:44:41.183 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.183 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.183 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.183 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.183 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.183 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.183 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.183 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.184 INFO:teuthology.orchestra.run.smithi119.stdout: "key_file": { 2024-03-28T03:44:41.184 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "key_file", 2024-03-28T03:44:41.184 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.184 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.184 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.184 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:41.184 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.184 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.184 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.185 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.185 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.185 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.185 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.185 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.185 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2024-03-28T03:44:41.185 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2024-03-28T03:44:41.185 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.186 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.186 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.186 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:41.186 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.186 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.186 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:41.186 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:41.186 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:41.187 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:41.187 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:41.187 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:41.187 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:41.187 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:41.187 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.187 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.187 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.187 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.188 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.188 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2024-03-28T03:44:41.188 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2024-03-28T03:44:41.188 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.188 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.188 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.188 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:41.188 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.188 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.189 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.189 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.189 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.189 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.189 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.189 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.189 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2024-03-28T03:44:41.189 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2024-03-28T03:44:41.190 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.190 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.190 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.190 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2024-03-28T03:44:41.190 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.190 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.190 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:41.190 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:41.191 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:41.191 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:41.191 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:41.191 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:41.191 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:41.191 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:41.191 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.191 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.191 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.192 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.192 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.192 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2024-03-28T03:44:41.192 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2024-03-28T03:44:41.192 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.192 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.192 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.192 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:41.192 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.193 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.193 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.193 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.193 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.193 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.193 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.193 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.193 INFO:teuthology.orchestra.run.smithi119.stdout: "server_addr": { 2024-03-28T03:44:41.194 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "server_addr", 2024-03-28T03:44:41.194 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.194 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.194 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.194 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:41.194 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.194 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.194 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.194 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.195 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.195 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.195 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.195 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.195 INFO:teuthology.orchestra.run.smithi119.stdout: "server_port": { 2024-03-28T03:44:41.195 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "server_port", 2024-03-28T03:44:41.195 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.195 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.196 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.196 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:41.196 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.196 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.196 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.196 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.196 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.196 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.196 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.197 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:41.197 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:41.197 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.197 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-03-28T03:44:41.197 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "rgw", 2024-03-28T03:44:41.197 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2024-03-28T03:44:41.197 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2024-03-28T03:44:41.197 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2024-03-28T03:44:41.198 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2024-03-28T03:44:41.198 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2024-03-28T03:44:41.198 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.198 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.198 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.198 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:41.198 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.198 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.199 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:41.199 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:41.199 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:41.199 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:41.199 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:41.199 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:41.199 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:41.199 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:41.199 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.200 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.200 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.200 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.200 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.200 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2024-03-28T03:44:41.200 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2024-03-28T03:44:41.200 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.200 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.201 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.201 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:41.201 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.201 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.201 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.201 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.201 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.201 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.202 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.202 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.202 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2024-03-28T03:44:41.202 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2024-03-28T03:44:41.202 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.202 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.202 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.202 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2024-03-28T03:44:41.202 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.203 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.203 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:41.203 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:41.203 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:41.203 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:41.203 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:41.203 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:41.203 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:41.203 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:41.204 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.204 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.204 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.204 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.204 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.204 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2024-03-28T03:44:41.204 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2024-03-28T03:44:41.204 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.205 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.205 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.205 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:41.205 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.205 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.205 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.205 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.205 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.205 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.206 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.206 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:41.206 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:41.206 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.206 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-03-28T03:44:41.206 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "rook", 2024-03-28T03:44:41.206 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2024-03-28T03:44:41.206 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2024-03-28T03:44:41.206 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2024-03-28T03:44:41.207 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2024-03-28T03:44:41.207 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2024-03-28T03:44:41.207 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.207 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.207 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.207 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:41.207 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.207 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.208 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:41.208 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:41.208 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:41.208 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:41.208 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:41.208 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:41.208 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:41.208 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:41.208 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.209 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.209 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.209 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.209 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.209 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2024-03-28T03:44:41.209 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2024-03-28T03:44:41.209 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.209 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.209 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.210 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:41.210 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.210 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.210 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.210 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.210 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.210 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.210 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.211 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.211 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2024-03-28T03:44:41.211 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2024-03-28T03:44:41.211 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.211 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.211 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.211 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2024-03-28T03:44:41.211 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.211 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.212 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:41.212 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:41.212 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:41.212 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:41.212 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:41.212 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:41.212 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:41.212 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:41.213 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.213 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.213 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.213 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.213 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.213 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2024-03-28T03:44:41.213 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2024-03-28T03:44:41.213 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.213 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.214 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.214 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:41.214 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.214 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.214 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.214 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.214 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.214 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.215 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.215 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.215 INFO:teuthology.orchestra.run.smithi119.stdout: "storage_class": { 2024-03-28T03:44:41.215 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "storage_class", 2024-03-28T03:44:41.215 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.215 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.215 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.215 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "local", 2024-03-28T03:44:41.215 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.216 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.216 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.216 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-03-28T03:44:41.216 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.216 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.216 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.216 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:41.216 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:41.217 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.217 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-03-28T03:44:41.217 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "selftest", 2024-03-28T03:44:41.217 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2024-03-28T03:44:41.217 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2024-03-28T03:44:41.217 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2024-03-28T03:44:41.217 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2024-03-28T03:44:41.217 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2024-03-28T03:44:41.217 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.218 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.218 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.218 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:41.218 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.218 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.218 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:41.218 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:41.218 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:41.219 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:41.219 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:41.219 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:41.219 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:41.219 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:41.219 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.219 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.219 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.219 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.220 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.220 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2024-03-28T03:44:41.220 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2024-03-28T03:44:41.220 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.220 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.220 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.220 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:41.220 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.221 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.221 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.221 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.221 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.221 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.221 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.221 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.221 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2024-03-28T03:44:41.221 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2024-03-28T03:44:41.222 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.222 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.222 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.222 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2024-03-28T03:44:41.222 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.222 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.222 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:41.222 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:41.223 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:41.223 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:41.223 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:41.223 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:41.223 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:41.223 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:41.223 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.223 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.223 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.224 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.224 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.224 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2024-03-28T03:44:41.224 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2024-03-28T03:44:41.224 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.224 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.224 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.224 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:41.225 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.225 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.225 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.225 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.225 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.225 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.225 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.225 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.225 INFO:teuthology.orchestra.run.smithi119.stdout: "roption1": { 2024-03-28T03:44:41.226 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "roption1", 2024-03-28T03:44:41.226 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.226 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.226 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.226 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:41.226 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.226 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.226 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.227 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.227 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.227 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.227 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.227 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.227 INFO:teuthology.orchestra.run.smithi119.stdout: "roption2": { 2024-03-28T03:44:41.227 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "roption2", 2024-03-28T03:44:41.227 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.227 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.228 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.228 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "xyz", 2024-03-28T03:44:41.228 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.228 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.228 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.228 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.228 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.228 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.228 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.229 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.229 INFO:teuthology.orchestra.run.smithi119.stdout: "rwoption1": { 2024-03-28T03:44:41.229 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "rwoption1", 2024-03-28T03:44:41.229 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.229 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.229 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.229 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:41.229 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.229 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.230 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.230 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.230 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.230 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.230 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.230 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.230 INFO:teuthology.orchestra.run.smithi119.stdout: "rwoption2": { 2024-03-28T03:44:41.230 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "rwoption2", 2024-03-28T03:44:41.231 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2024-03-28T03:44:41.231 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.231 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.231 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:41.231 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.231 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.231 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.231 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.231 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.232 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.232 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.232 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.232 INFO:teuthology.orchestra.run.smithi119.stdout: "rwoption3": { 2024-03-28T03:44:41.232 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "rwoption3", 2024-03-28T03:44:41.232 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "float", 2024-03-28T03:44:41.232 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.232 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.232 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:41.233 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.233 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.233 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.233 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.233 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.233 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.233 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.233 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.234 INFO:teuthology.orchestra.run.smithi119.stdout: "rwoption4": { 2024-03-28T03:44:41.234 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "rwoption4", 2024-03-28T03:44:41.234 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.234 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.234 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.234 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:41.234 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.234 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.235 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.235 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.235 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.235 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.235 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.235 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.235 INFO:teuthology.orchestra.run.smithi119.stdout: "rwoption5": { 2024-03-28T03:44:41.235 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "rwoption5", 2024-03-28T03:44:41.235 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.236 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.236 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.236 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:41.236 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.236 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.236 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.236 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.236 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.237 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.237 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.237 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.237 INFO:teuthology.orchestra.run.smithi119.stdout: "rwoption6": { 2024-03-28T03:44:41.237 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "rwoption6", 2024-03-28T03:44:41.237 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.237 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.237 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.237 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2024-03-28T03:44:41.238 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.238 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.238 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.238 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.238 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.238 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.238 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.238 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.239 INFO:teuthology.orchestra.run.smithi119.stdout: "rwoption7": { 2024-03-28T03:44:41.239 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "rwoption7", 2024-03-28T03:44:41.239 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2024-03-28T03:44:41.239 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.239 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.239 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:41.239 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "1", 2024-03-28T03:44:41.239 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "42", 2024-03-28T03:44:41.239 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.240 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.240 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.240 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.240 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.240 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.240 INFO:teuthology.orchestra.run.smithi119.stdout: "testkey": { 2024-03-28T03:44:41.240 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "testkey", 2024-03-28T03:44:41.240 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.240 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.241 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.241 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:41.241 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.241 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.241 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.241 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.241 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.241 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.242 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.242 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.242 INFO:teuthology.orchestra.run.smithi119.stdout: "testlkey": { 2024-03-28T03:44:41.242 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "testlkey", 2024-03-28T03:44:41.242 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.242 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.242 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.242 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:41.243 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.243 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.243 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.243 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.243 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.243 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.243 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.243 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.243 INFO:teuthology.orchestra.run.smithi119.stdout: "testnewline": { 2024-03-28T03:44:41.244 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "testnewline", 2024-03-28T03:44:41.244 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.244 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.244 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.244 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:41.244 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.244 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.244 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.244 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.245 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.245 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.245 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.245 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:41.245 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:41.245 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.245 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-03-28T03:44:41.245 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "snap_schedule", 2024-03-28T03:44:41.245 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2024-03-28T03:44:41.246 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2024-03-28T03:44:41.246 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2024-03-28T03:44:41.246 INFO:teuthology.orchestra.run.smithi119.stdout: "allow_m_granularity": { 2024-03-28T03:44:41.246 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "allow_m_granularity", 2024-03-28T03:44:41.246 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.246 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.246 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.247 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:41.247 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.247 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.247 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.247 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "allow minute scheduled snapshots", 2024-03-28T03:44:41.247 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.247 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.247 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.247 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.248 INFO:teuthology.orchestra.run.smithi119.stdout: "dump_on_update": { 2024-03-28T03:44:41.248 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "dump_on_update", 2024-03-28T03:44:41.248 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.248 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.248 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.248 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:41.248 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.248 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.248 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.249 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "dump database to debug log on update", 2024-03-28T03:44:41.249 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.249 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.249 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.249 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.249 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2024-03-28T03:44:41.249 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2024-03-28T03:44:41.249 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.250 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.250 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.250 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:41.250 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.250 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.250 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:41.250 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:41.250 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:41.251 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:41.251 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:41.251 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:41.251 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:41.251 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:41.251 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.251 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.251 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.251 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.252 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.252 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2024-03-28T03:44:41.252 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2024-03-28T03:44:41.252 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.252 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.252 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.252 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:41.252 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.252 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.253 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.253 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.253 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.253 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.253 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.253 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.253 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2024-03-28T03:44:41.253 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2024-03-28T03:44:41.253 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.254 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.254 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.254 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2024-03-28T03:44:41.254 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.254 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.254 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:41.254 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:41.254 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:41.255 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:41.255 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:41.255 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:41.255 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:41.255 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:41.255 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.255 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.255 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.255 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.256 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.256 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2024-03-28T03:44:41.256 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2024-03-28T03:44:41.256 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.256 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.256 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.256 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:41.256 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.257 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.257 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.257 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.257 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.257 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.257 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.257 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:41.257 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:41.257 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.258 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-03-28T03:44:41.258 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "stats", 2024-03-28T03:44:41.258 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2024-03-28T03:44:41.258 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2024-03-28T03:44:41.258 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2024-03-28T03:44:41.258 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2024-03-28T03:44:41.258 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2024-03-28T03:44:41.258 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.259 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.259 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.259 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:41.259 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.259 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.259 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:41.259 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:41.259 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:41.260 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:41.260 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:41.260 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:41.260 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:41.260 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:41.260 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.260 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.260 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.260 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.261 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.261 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2024-03-28T03:44:41.261 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2024-03-28T03:44:41.261 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.261 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.261 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.261 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:41.261 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.262 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.262 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.262 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.262 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.262 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.262 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.262 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.263 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2024-03-28T03:44:41.263 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2024-03-28T03:44:41.263 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.263 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.263 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.263 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2024-03-28T03:44:41.263 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.263 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.263 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:41.264 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:41.264 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:41.264 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:41.264 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:41.264 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:41.264 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:41.264 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:41.264 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.265 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.265 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.265 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.265 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.265 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2024-03-28T03:44:41.265 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2024-03-28T03:44:41.265 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.265 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.265 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.266 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:41.266 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.266 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.266 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.266 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.266 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.266 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.266 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.267 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:41.267 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:41.267 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.267 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-03-28T03:44:41.267 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "status", 2024-03-28T03:44:41.267 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2024-03-28T03:44:41.267 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2024-03-28T03:44:41.267 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2024-03-28T03:44:41.267 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2024-03-28T03:44:41.268 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2024-03-28T03:44:41.268 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.268 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.268 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.268 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:41.268 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.268 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.268 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:41.268 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:41.269 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:41.269 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:41.269 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:41.269 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:41.269 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:41.269 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:41.269 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.269 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.270 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.270 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.270 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.270 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2024-03-28T03:44:41.270 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2024-03-28T03:44:41.270 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.270 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.270 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.271 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:41.271 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.271 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.271 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.271 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.271 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.271 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.271 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.271 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.272 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2024-03-28T03:44:41.272 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2024-03-28T03:44:41.272 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.272 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.272 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.272 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2024-03-28T03:44:41.272 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.272 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.273 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:41.273 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:41.273 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:41.273 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:41.273 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:41.273 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:41.273 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:41.273 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:41.273 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.274 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.274 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.274 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.274 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.274 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2024-03-28T03:44:41.274 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2024-03-28T03:44:41.274 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.274 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.274 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.275 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:41.275 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.275 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.275 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.275 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.275 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.275 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.275 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.276 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:41.276 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:41.276 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.276 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-03-28T03:44:41.276 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "telegraf", 2024-03-28T03:44:41.276 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2024-03-28T03:44:41.276 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2024-03-28T03:44:41.276 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2024-03-28T03:44:41.276 INFO:teuthology.orchestra.run.smithi119.stdout: "address": { 2024-03-28T03:44:41.277 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "address", 2024-03-28T03:44:41.277 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.277 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.277 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.277 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-03-28T03:44:41.277 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.277 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.277 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.277 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.278 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.278 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.278 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.278 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.278 INFO:teuthology.orchestra.run.smithi119.stdout: "interval": { 2024-03-28T03:44:41.278 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "interval", 2024-03-28T03:44:41.278 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "secs", 2024-03-28T03:44:41.278 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.279 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.279 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "15", 2024-03-28T03:44:41.279 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.279 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.279 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.279 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.279 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.279 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.280 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.280 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.280 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2024-03-28T03:44:41.280 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2024-03-28T03:44:41.280 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.280 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.280 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.280 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:41.280 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.281 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.281 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:41.281 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:41.281 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:41.281 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:41.281 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:41.281 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:41.281 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:41.281 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:41.282 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.282 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.282 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.282 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.282 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.282 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2024-03-28T03:44:41.282 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2024-03-28T03:44:41.282 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.282 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.283 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.283 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:41.283 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.283 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.283 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.283 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.283 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.283 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.284 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.284 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.284 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2024-03-28T03:44:41.284 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2024-03-28T03:44:41.284 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.284 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.284 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.284 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2024-03-28T03:44:41.284 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.285 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.285 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:41.285 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:41.285 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:41.285 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:41.285 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:41.285 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:41.285 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:41.286 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:41.286 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.286 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.286 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.286 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.286 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.286 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2024-03-28T03:44:41.286 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2024-03-28T03:44:41.286 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.287 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.287 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.287 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:41.287 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.287 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.287 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.287 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.287 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.288 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.288 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.288 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:41.288 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:41.288 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.288 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-03-28T03:44:41.288 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "telemetry", 2024-03-28T03:44:41.288 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2024-03-28T03:44:41.288 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2024-03-28T03:44:41.289 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2024-03-28T03:44:41.289 INFO:teuthology.orchestra.run.smithi119.stdout: "channel_basic": { 2024-03-28T03:44:41.289 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "channel_basic", 2024-03-28T03:44:41.289 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.289 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.289 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.289 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2024-03-28T03:44:41.289 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.289 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.290 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.290 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Share basic cluster information (size, version)", 2024-03-28T03:44:41.290 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.290 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.290 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.290 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.290 INFO:teuthology.orchestra.run.smithi119.stdout: "channel_crash": { 2024-03-28T03:44:41.290 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "channel_crash", 2024-03-28T03:44:41.291 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.291 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.291 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.291 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2024-03-28T03:44:41.291 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.291 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.291 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.291 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-03-28T03:44:41.291 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.292 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.292 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.292 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.292 INFO:teuthology.orchestra.run.smithi119.stdout: "channel_device": { 2024-03-28T03:44:41.292 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "channel_device", 2024-03-28T03:44:41.292 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.292 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.292 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.292 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2024-03-28T03:44:41.293 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.293 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.293 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.293 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-03-28T03:44:41.293 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.293 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.293 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.293 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.294 INFO:teuthology.orchestra.run.smithi119.stdout: "channel_ident": { 2024-03-28T03:44:41.294 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "channel_ident", 2024-03-28T03:44:41.294 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.294 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.294 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.294 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:41.294 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.294 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.294 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.295 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-03-28T03:44:41.295 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.295 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.295 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.295 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.295 INFO:teuthology.orchestra.run.smithi119.stdout: "channel_perf": { 2024-03-28T03:44:41.295 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "channel_perf", 2024-03-28T03:44:41.295 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.295 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.296 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.296 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:41.296 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.296 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.296 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.296 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Share various performance metrics of a cluster", 2024-03-28T03:44:41.296 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.296 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.296 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.297 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.297 INFO:teuthology.orchestra.run.smithi119.stdout: "contact": { 2024-03-28T03:44:41.297 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "contact", 2024-03-28T03:44:41.297 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.297 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.297 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.297 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:41.297 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.298 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.298 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.298 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.298 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.298 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.298 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.298 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.298 INFO:teuthology.orchestra.run.smithi119.stdout: "description": { 2024-03-28T03:44:41.298 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "description", 2024-03-28T03:44:41.299 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.299 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.299 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.299 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:41.299 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.299 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.299 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.299 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.299 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.300 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.300 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.300 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.300 INFO:teuthology.orchestra.run.smithi119.stdout: "device_url": { 2024-03-28T03:44:41.300 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "device_url", 2024-03-28T03:44:41.300 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.300 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.300 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.300 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-03-28T03:44:41.301 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.301 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.301 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.301 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.301 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.301 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.301 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.301 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.302 INFO:teuthology.orchestra.run.smithi119.stdout: "enabled": { 2024-03-28T03:44:41.302 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "enabled", 2024-03-28T03:44:41.302 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.302 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.302 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.302 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:41.302 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.302 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.302 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.303 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.303 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.303 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.303 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.303 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.303 INFO:teuthology.orchestra.run.smithi119.stdout: "interval": { 2024-03-28T03:44:41.303 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "interval", 2024-03-28T03:44:41.303 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2024-03-28T03:44:41.304 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.304 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.304 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "24", 2024-03-28T03:44:41.304 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "8", 2024-03-28T03:44:41.304 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.304 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.304 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.304 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.304 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.305 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.305 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.305 INFO:teuthology.orchestra.run.smithi119.stdout: "last_opt_revision": { 2024-03-28T03:44:41.305 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "last_opt_revision", 2024-03-28T03:44:41.305 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2024-03-28T03:44:41.305 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.305 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.305 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "1", 2024-03-28T03:44:41.305 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.306 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.306 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.306 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.306 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.306 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.306 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.306 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.306 INFO:teuthology.orchestra.run.smithi119.stdout: "leaderboard": { 2024-03-28T03:44:41.307 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "leaderboard", 2024-03-28T03:44:41.307 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.307 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.307 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.307 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:41.307 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.307 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.307 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.307 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.308 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.308 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.308 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.308 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.308 INFO:teuthology.orchestra.run.smithi119.stdout: "leaderboard_description": { 2024-03-28T03:44:41.308 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "leaderboard_description", 2024-03-28T03:44:41.308 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.308 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.309 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.309 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:41.309 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.309 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.309 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.309 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.309 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.309 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.310 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.310 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.310 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2024-03-28T03:44:41.310 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2024-03-28T03:44:41.310 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.310 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.310 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.310 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:41.310 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.311 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.311 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:41.311 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:41.311 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:41.311 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:41.311 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:41.311 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:41.311 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:41.312 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:41.312 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.312 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.312 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.312 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.312 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.312 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2024-03-28T03:44:41.312 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2024-03-28T03:44:41.312 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.313 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.313 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.313 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:41.313 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.313 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.313 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.313 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.313 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.314 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.314 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.314 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.314 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2024-03-28T03:44:41.314 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2024-03-28T03:44:41.314 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.314 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.314 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.314 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2024-03-28T03:44:41.315 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.315 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.315 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:41.315 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:41.315 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:41.315 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:41.315 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:41.315 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:41.315 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:41.316 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:41.316 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.316 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.316 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.316 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.316 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.316 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2024-03-28T03:44:41.316 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2024-03-28T03:44:41.317 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.317 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.317 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.317 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:41.317 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.317 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.317 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.317 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.317 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.318 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.318 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.318 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.318 INFO:teuthology.orchestra.run.smithi119.stdout: "organization": { 2024-03-28T03:44:41.318 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "organization", 2024-03-28T03:44:41.318 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.318 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.318 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.318 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:41.319 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.319 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.319 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.319 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.319 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.319 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.319 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.319 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.320 INFO:teuthology.orchestra.run.smithi119.stdout: "proxy": { 2024-03-28T03:44:41.320 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "proxy", 2024-03-28T03:44:41.320 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.320 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.320 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.320 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:41.320 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.320 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.320 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.320 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.321 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.321 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.321 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.321 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.321 INFO:teuthology.orchestra.run.smithi119.stdout: "url": { 2024-03-28T03:44:41.321 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "url", 2024-03-28T03:44:41.321 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.321 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.322 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.322 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-03-28T03:44:41.322 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.322 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.322 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.322 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.322 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.322 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.322 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.323 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:41.323 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:41.323 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.323 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-03-28T03:44:41.323 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "test_orchestrator", 2024-03-28T03:44:41.323 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2024-03-28T03:44:41.323 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2024-03-28T03:44:41.323 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2024-03-28T03:44:41.324 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2024-03-28T03:44:41.324 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2024-03-28T03:44:41.324 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.324 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.324 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.324 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:41.324 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.324 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.324 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:41.325 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:41.325 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:41.325 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:41.325 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:41.325 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:41.325 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:41.325 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:41.325 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.325 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.326 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.326 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.326 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.326 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2024-03-28T03:44:41.326 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2024-03-28T03:44:41.326 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.326 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.326 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.327 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:41.327 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.327 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.327 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.327 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.327 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.327 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.327 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.327 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.328 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2024-03-28T03:44:41.328 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2024-03-28T03:44:41.328 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.328 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.328 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.328 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2024-03-28T03:44:41.328 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.328 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.328 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:41.329 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:41.329 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:41.329 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:41.329 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:41.329 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:41.329 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:41.329 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:41.329 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.329 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.330 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.330 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.330 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.330 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2024-03-28T03:44:41.330 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2024-03-28T03:44:41.330 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.330 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.330 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.331 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:41.331 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.331 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.331 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.331 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.331 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.331 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.331 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.332 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:41.332 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:41.332 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.332 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-03-28T03:44:41.332 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "volumes", 2024-03-28T03:44:41.332 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2024-03-28T03:44:41.332 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2024-03-28T03:44:41.332 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2024-03-28T03:44:41.332 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2024-03-28T03:44:41.333 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2024-03-28T03:44:41.333 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.333 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.333 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.333 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:41.333 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.333 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.333 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:41.334 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:41.334 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:41.334 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:41.334 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:41.334 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:41.334 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:41.334 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:41.334 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.334 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.335 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.335 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.335 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.335 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2024-03-28T03:44:41.335 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2024-03-28T03:44:41.335 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.335 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.335 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.336 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:41.336 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.336 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.336 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.336 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.336 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.336 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.336 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.336 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.337 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2024-03-28T03:44:41.337 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2024-03-28T03:44:41.337 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.337 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.337 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.337 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2024-03-28T03:44:41.337 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.337 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.337 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:41.338 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:41.338 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:41.338 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:41.338 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:41.338 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:41.338 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:41.338 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:41.338 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.339 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.339 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.339 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.339 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.339 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2024-03-28T03:44:41.339 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2024-03-28T03:44:41.339 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.339 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.339 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.340 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:41.340 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.340 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.340 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.340 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.340 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.340 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.340 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.340 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.341 INFO:teuthology.orchestra.run.smithi119.stdout: "max_concurrent_clones": { 2024-03-28T03:44:41.341 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "max_concurrent_clones", 2024-03-28T03:44:41.341 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2024-03-28T03:44:41.341 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.341 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.341 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "4", 2024-03-28T03:44:41.341 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.341 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.342 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.342 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Number of asynchronous cloner threads", 2024-03-28T03:44:41.342 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.342 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.342 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.342 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.342 INFO:teuthology.orchestra.run.smithi119.stdout: "periodic_async_work": { 2024-03-28T03:44:41.342 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "periodic_async_work", 2024-03-28T03:44:41.343 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.343 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.343 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.343 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:41.343 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.343 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.343 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.343 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Periodically check for async work", 2024-03-28T03:44:41.343 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.344 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.344 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.344 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.344 INFO:teuthology.orchestra.run.smithi119.stdout: "snapshot_clone_delay": { 2024-03-28T03:44:41.344 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "snapshot_clone_delay", 2024-03-28T03:44:41.344 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2024-03-28T03:44:41.344 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.344 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.344 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "0", 2024-03-28T03:44:41.345 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.345 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.345 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.345 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-03-28T03:44:41.345 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.345 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.345 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.345 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.346 INFO:teuthology.orchestra.run.smithi119.stdout: "snapshot_clone_no_wait": { 2024-03-28T03:44:41.346 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "snapshot_clone_no_wait", 2024-03-28T03:44:41.346 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.346 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.346 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.346 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2024-03-28T03:44:41.346 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.346 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.346 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.347 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-03-28T03:44:41.347 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.347 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.347 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.347 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:41.347 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:41.347 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.347 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-03-28T03:44:41.348 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "zabbix", 2024-03-28T03:44:41.348 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2024-03-28T03:44:41.348 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2024-03-28T03:44:41.348 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2024-03-28T03:44:41.348 INFO:teuthology.orchestra.run.smithi119.stdout: "discovery_interval": { 2024-03-28T03:44:41.348 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "discovery_interval", 2024-03-28T03:44:41.348 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "uint", 2024-03-28T03:44:41.348 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.348 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.349 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "100", 2024-03-28T03:44:41.349 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.349 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.349 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.349 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.349 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.349 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.349 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.349 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.350 INFO:teuthology.orchestra.run.smithi119.stdout: "identifier": { 2024-03-28T03:44:41.350 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "identifier", 2024-03-28T03:44:41.350 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.350 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.350 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.350 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:41.350 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.350 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.351 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.351 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.351 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.351 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.351 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.351 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.351 INFO:teuthology.orchestra.run.smithi119.stdout: "interval": { 2024-03-28T03:44:41.351 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "interval", 2024-03-28T03:44:41.351 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "secs", 2024-03-28T03:44:41.352 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.352 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.352 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "60", 2024-03-28T03:44:41.352 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.352 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.352 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.352 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.352 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.353 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.353 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.353 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.353 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2024-03-28T03:44:41.353 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2024-03-28T03:44:41.353 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.353 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.353 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.353 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:41.354 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.354 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.354 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:41.354 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:41.354 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:41.354 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:41.354 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:41.354 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:41.354 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:41.355 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:41.355 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.355 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.355 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.355 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.355 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.355 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2024-03-28T03:44:41.355 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2024-03-28T03:44:41.356 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.356 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.356 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.356 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:41.356 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.356 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.356 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.356 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.356 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.357 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.357 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.357 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.357 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2024-03-28T03:44:41.357 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2024-03-28T03:44:41.357 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.357 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.357 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.357 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2024-03-28T03:44:41.358 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.358 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.358 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:41.358 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:41.358 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:41.358 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:41.358 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:41.358 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:41.359 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:41.359 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:41.359 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.359 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.359 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.359 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.359 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.359 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2024-03-28T03:44:41.359 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2024-03-28T03:44:41.360 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.360 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.360 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.360 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:41.360 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.360 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.360 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.360 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.360 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.361 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.361 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.361 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.361 INFO:teuthology.orchestra.run.smithi119.stdout: "zabbix_host": { 2024-03-28T03:44:41.361 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "zabbix_host", 2024-03-28T03:44:41.361 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.361 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.361 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.362 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:41.362 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.362 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.362 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.362 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.362 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.362 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.362 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.362 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.363 INFO:teuthology.orchestra.run.smithi119.stdout: "zabbix_port": { 2024-03-28T03:44:41.363 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "zabbix_port", 2024-03-28T03:44:41.363 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2024-03-28T03:44:41.363 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.363 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.363 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "10051", 2024-03-28T03:44:41.363 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.363 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.364 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.364 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.364 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.364 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.364 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.364 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.364 INFO:teuthology.orchestra.run.smithi119.stdout: "zabbix_sender": { 2024-03-28T03:44:41.364 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "zabbix_sender", 2024-03-28T03:44:41.364 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.365 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.365 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.365 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-03-28T03:44:41.365 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.365 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.365 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.365 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.365 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.366 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.366 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.366 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:41.366 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:41.366 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:41.366 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-03-28T03:44:41.366 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:41.366 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:41.366 INFO:teuthology.orchestra.run.smithi119.stdout: "modules": [ 2024-03-28T03:44:41.367 INFO:teuthology.orchestra.run.smithi119.stdout: "cephadm", 2024-03-28T03:44:41.367 INFO:teuthology.orchestra.run.smithi119.stdout: "dashboard", 2024-03-28T03:44:41.367 INFO:teuthology.orchestra.run.smithi119.stdout: "iostat", 2024-03-28T03:44:41.367 INFO:teuthology.orchestra.run.smithi119.stdout: "nfs", 2024-03-28T03:44:41.367 INFO:teuthology.orchestra.run.smithi119.stdout: "restful", 2024-03-28T03:44:41.367 INFO:teuthology.orchestra.run.smithi119.stdout: "snap_schedule" 2024-03-28T03:44:41.367 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:41.367 INFO:teuthology.orchestra.run.smithi119.stdout: "available_modules": [ 2024-03-28T03:44:41.367 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-03-28T03:44:41.368 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "alerts", 2024-03-28T03:44:41.368 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2024-03-28T03:44:41.368 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2024-03-28T03:44:41.368 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2024-03-28T03:44:41.368 INFO:teuthology.orchestra.run.smithi119.stdout: "interval": { 2024-03-28T03:44:41.368 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "interval", 2024-03-28T03:44:41.368 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "secs", 2024-03-28T03:44:41.368 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.368 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.369 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "60", 2024-03-28T03:44:41.369 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.369 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.369 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.369 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "How frequently to reexamine health status", 2024-03-28T03:44:41.369 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.369 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.369 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.370 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.370 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2024-03-28T03:44:41.370 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2024-03-28T03:44:41.370 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.370 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.370 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.370 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:41.370 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.371 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.371 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:41.371 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:41.371 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:41.371 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:41.371 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:41.371 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:41.371 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:41.371 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:41.372 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.372 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.372 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.372 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.372 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.372 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2024-03-28T03:44:41.372 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2024-03-28T03:44:41.372 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.372 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.373 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.373 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:41.373 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.373 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.373 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.373 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.373 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.373 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.374 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.374 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.374 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2024-03-28T03:44:41.374 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2024-03-28T03:44:41.374 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.374 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.374 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.374 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2024-03-28T03:44:41.375 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.375 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.375 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:41.375 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:41.375 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:41.375 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:41.375 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:41.375 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:41.375 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:41.376 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:41.376 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.376 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.376 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.376 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.376 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.376 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2024-03-28T03:44:41.376 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2024-03-28T03:44:41.376 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.377 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.377 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.377 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:41.377 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.377 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.377 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.377 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.377 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.377 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.378 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.378 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.378 INFO:teuthology.orchestra.run.smithi119.stdout: "smtp_destination": { 2024-03-28T03:44:41.378 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "smtp_destination", 2024-03-28T03:44:41.378 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.378 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.378 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.378 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:41.379 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.379 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.379 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.379 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Email address to send alerts to", 2024-03-28T03:44:41.379 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.379 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.379 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.379 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.379 INFO:teuthology.orchestra.run.smithi119.stdout: "smtp_from_name": { 2024-03-28T03:44:41.380 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "smtp_from_name", 2024-03-28T03:44:41.380 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.380 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.380 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.380 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "Ceph", 2024-03-28T03:44:41.380 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.380 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.380 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.380 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Email From: name", 2024-03-28T03:44:41.381 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.381 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.381 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.381 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.381 INFO:teuthology.orchestra.run.smithi119.stdout: "smtp_host": { 2024-03-28T03:44:41.381 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "smtp_host", 2024-03-28T03:44:41.381 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.381 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.381 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.382 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:41.382 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.382 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.382 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.382 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "SMTP server", 2024-03-28T03:44:41.382 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.382 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.382 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.383 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.383 INFO:teuthology.orchestra.run.smithi119.stdout: "smtp_password": { 2024-03-28T03:44:41.383 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "smtp_password", 2024-03-28T03:44:41.383 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.383 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.383 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.383 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:41.383 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.383 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.384 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.384 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Password to authenticate with", 2024-03-28T03:44:41.384 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.384 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.384 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.384 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.384 INFO:teuthology.orchestra.run.smithi119.stdout: "smtp_port": { 2024-03-28T03:44:41.384 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "smtp_port", 2024-03-28T03:44:41.384 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2024-03-28T03:44:41.385 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.385 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.385 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "465", 2024-03-28T03:44:41.385 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.385 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.385 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.385 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "SMTP port", 2024-03-28T03:44:41.385 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.386 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.386 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.386 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.386 INFO:teuthology.orchestra.run.smithi119.stdout: "smtp_sender": { 2024-03-28T03:44:41.386 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "smtp_sender", 2024-03-28T03:44:41.386 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.386 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.386 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.387 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:41.387 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.387 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.387 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.387 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "SMTP envelope sender", 2024-03-28T03:44:41.387 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.387 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.387 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.387 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.388 INFO:teuthology.orchestra.run.smithi119.stdout: "smtp_ssl": { 2024-03-28T03:44:41.388 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "smtp_ssl", 2024-03-28T03:44:41.388 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.388 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.388 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.388 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2024-03-28T03:44:41.388 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.388 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.389 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.389 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Use SSL to connect to SMTP server", 2024-03-28T03:44:41.389 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.389 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.389 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.389 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.389 INFO:teuthology.orchestra.run.smithi119.stdout: "smtp_user": { 2024-03-28T03:44:41.389 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "smtp_user", 2024-03-28T03:44:41.389 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.390 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.390 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.390 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:41.390 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.390 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.390 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.390 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "User to authenticate as", 2024-03-28T03:44:41.390 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.391 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.391 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.391 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:41.391 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:41.391 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.391 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-03-28T03:44:41.391 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "balancer", 2024-03-28T03:44:41.391 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2024-03-28T03:44:41.391 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2024-03-28T03:44:41.392 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2024-03-28T03:44:41.392 INFO:teuthology.orchestra.run.smithi119.stdout: "active": { 2024-03-28T03:44:41.392 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "active", 2024-03-28T03:44:41.392 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.392 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.392 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.392 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2024-03-28T03:44:41.392 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.392 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.392 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.393 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "automatically balance PGs across cluster", 2024-03-28T03:44:41.393 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.393 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.393 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.393 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.393 INFO:teuthology.orchestra.run.smithi119.stdout: "begin_time": { 2024-03-28T03:44:41.393 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "begin_time", 2024-03-28T03:44:41.393 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.394 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.394 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.394 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "0000", 2024-03-28T03:44:41.394 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.394 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.394 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.394 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "beginning time of day to automatically balance", 2024-03-28T03:44:41.394 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-28T03:44:41.394 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.395 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.395 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.395 INFO:teuthology.orchestra.run.smithi119.stdout: "begin_weekday": { 2024-03-28T03:44:41.395 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "begin_weekday", 2024-03-28T03:44:41.395 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "uint", 2024-03-28T03:44:41.395 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.395 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.395 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "0", 2024-03-28T03:44:41.395 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "0", 2024-03-28T03:44:41.396 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "6", 2024-03-28T03:44:41.396 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.396 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-03-28T03:44:41.396 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-28T03:44:41.396 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.396 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.396 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.396 INFO:teuthology.orchestra.run.smithi119.stdout: "crush_compat_max_iterations": { 2024-03-28T03:44:41.397 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "crush_compat_max_iterations", 2024-03-28T03:44:41.397 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "uint", 2024-03-28T03:44:41.397 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.397 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.397 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "25", 2024-03-28T03:44:41.397 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "1", 2024-03-28T03:44:41.397 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "250", 2024-03-28T03:44:41.397 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.397 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-03-28T03:44:41.398 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.398 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.398 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.398 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.398 INFO:teuthology.orchestra.run.smithi119.stdout: "crush_compat_metrics": { 2024-03-28T03:44:41.398 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "crush_compat_metrics", 2024-03-28T03:44:41.398 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.398 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.399 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.399 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "pgs,objects,bytes", 2024-03-28T03:44:41.399 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.399 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.399 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.399 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-03-28T03:44:41.399 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-03-28T03:44:41.399 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.399 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.400 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.400 INFO:teuthology.orchestra.run.smithi119.stdout: "crush_compat_step": { 2024-03-28T03:44:41.400 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "crush_compat_step", 2024-03-28T03:44:41.400 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "float", 2024-03-28T03:44:41.400 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.400 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.400 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "0.5", 2024-03-28T03:44:41.400 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "0.001", 2024-03-28T03:44:41.400 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "0.999", 2024-03-28T03:44:41.401 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.401 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "aggressiveness of optimization", 2024-03-28T03:44:41.401 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-03-28T03:44:41.401 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.401 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.401 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.401 INFO:teuthology.orchestra.run.smithi119.stdout: "end_time": { 2024-03-28T03:44:41.401 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "end_time", 2024-03-28T03:44:41.402 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.402 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.402 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.402 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "2359", 2024-03-28T03:44:41.402 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.402 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.402 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.402 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "ending time of day to automatically balance", 2024-03-28T03:44:41.402 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-28T03:44:41.403 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.403 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.403 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.403 INFO:teuthology.orchestra.run.smithi119.stdout: "end_weekday": { 2024-03-28T03:44:41.403 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "end_weekday", 2024-03-28T03:44:41.403 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "uint", 2024-03-28T03:44:41.403 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.403 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.404 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "0", 2024-03-28T03:44:41.404 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "0", 2024-03-28T03:44:41.404 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "6", 2024-03-28T03:44:41.404 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.404 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-03-28T03:44:41.404 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-28T03:44:41.404 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.404 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.404 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.405 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2024-03-28T03:44:41.405 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2024-03-28T03:44:41.405 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.405 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.405 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.405 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:41.405 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.405 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.406 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:41.406 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:41.406 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:41.406 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:41.406 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:41.406 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:41.406 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:41.406 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:41.406 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.407 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.407 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.407 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.407 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.407 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2024-03-28T03:44:41.407 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2024-03-28T03:44:41.407 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.407 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.407 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.408 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:41.408 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.408 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.408 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.408 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.408 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.408 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.408 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.409 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.409 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2024-03-28T03:44:41.409 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2024-03-28T03:44:41.409 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.409 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.409 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.409 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2024-03-28T03:44:41.409 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.409 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.410 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:41.410 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:41.410 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:41.410 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:41.410 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:41.410 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:41.410 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:41.410 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:41.411 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.411 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.411 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.411 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.411 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.411 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2024-03-28T03:44:41.411 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2024-03-28T03:44:41.411 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.411 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.412 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.412 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:41.412 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.412 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.412 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.412 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.412 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.412 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.413 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.413 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.413 INFO:teuthology.orchestra.run.smithi119.stdout: "min_score": { 2024-03-28T03:44:41.413 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "min_score", 2024-03-28T03:44:41.413 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "float", 2024-03-28T03:44:41.413 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.413 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.413 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "0", 2024-03-28T03:44:41.413 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.414 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.414 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.414 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-03-28T03:44:41.414 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.414 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.414 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.414 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.414 INFO:teuthology.orchestra.run.smithi119.stdout: "mode": { 2024-03-28T03:44:41.415 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "mode", 2024-03-28T03:44:41.415 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.415 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.415 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.415 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "upmap", 2024-03-28T03:44:41.415 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.415 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.415 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:41.416 INFO:teuthology.orchestra.run.smithi119.stdout: "crush-compat", 2024-03-28T03:44:41.416 INFO:teuthology.orchestra.run.smithi119.stdout: "none", 2024-03-28T03:44:41.416 INFO:teuthology.orchestra.run.smithi119.stdout: "read", 2024-03-28T03:44:41.416 INFO:teuthology.orchestra.run.smithi119.stdout: "upmap", 2024-03-28T03:44:41.416 INFO:teuthology.orchestra.run.smithi119.stdout: "upmap-read" 2024-03-28T03:44:41.416 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:41.416 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Balancer mode", 2024-03-28T03:44:41.416 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.416 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.417 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.417 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.417 INFO:teuthology.orchestra.run.smithi119.stdout: "pool_ids": { 2024-03-28T03:44:41.417 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "pool_ids", 2024-03-28T03:44:41.417 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.417 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.417 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.417 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:41.418 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.418 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.418 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.418 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-03-28T03:44:41.418 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.418 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.418 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.418 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.419 INFO:teuthology.orchestra.run.smithi119.stdout: "sleep_interval": { 2024-03-28T03:44:41.419 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "sleep_interval", 2024-03-28T03:44:41.419 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "secs", 2024-03-28T03:44:41.419 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.419 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.419 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "60", 2024-03-28T03:44:41.419 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.419 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.419 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.420 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-03-28T03:44:41.420 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.420 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.420 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.420 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.420 INFO:teuthology.orchestra.run.smithi119.stdout: "upmap_max_deviation": { 2024-03-28T03:44:41.420 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "upmap_max_deviation", 2024-03-28T03:44:41.420 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2024-03-28T03:44:41.420 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.421 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.421 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "5", 2024-03-28T03:44:41.421 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "1", 2024-03-28T03:44:41.421 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.421 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.421 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "deviation below which no optimization is attempted", 2024-03-28T03:44:41.421 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-03-28T03:44:41.421 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.422 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.422 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.422 INFO:teuthology.orchestra.run.smithi119.stdout: "upmap_max_optimizations": { 2024-03-28T03:44:41.422 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "upmap_max_optimizations", 2024-03-28T03:44:41.422 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "uint", 2024-03-28T03:44:41.422 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.422 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.423 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "10", 2024-03-28T03:44:41.423 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.423 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.423 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.423 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-03-28T03:44:41.423 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.423 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.423 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.423 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:41.424 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:41.424 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.424 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-03-28T03:44:41.424 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "cephadm", 2024-03-28T03:44:41.424 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2024-03-28T03:44:41.424 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2024-03-28T03:44:41.424 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2024-03-28T03:44:41.424 INFO:teuthology.orchestra.run.smithi119.stdout: "agent_down_multiplier": { 2024-03-28T03:44:41.424 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "agent_down_multiplier", 2024-03-28T03:44:41.425 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "float", 2024-03-28T03:44:41.425 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.425 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.425 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "3.0", 2024-03-28T03:44:41.425 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.425 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.425 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.425 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-03-28T03:44:41.425 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.426 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.426 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.426 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.426 INFO:teuthology.orchestra.run.smithi119.stdout: "agent_refresh_rate": { 2024-03-28T03:44:41.426 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "agent_refresh_rate", 2024-03-28T03:44:41.426 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "secs", 2024-03-28T03:44:41.426 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.426 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.427 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "20", 2024-03-28T03:44:41.427 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.427 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.427 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.427 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-03-28T03:44:41.427 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.427 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.427 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.428 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.428 INFO:teuthology.orchestra.run.smithi119.stdout: "agent_starting_port": { 2024-03-28T03:44:41.428 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "agent_starting_port", 2024-03-28T03:44:41.428 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2024-03-28T03:44:41.428 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.428 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.428 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "4721", 2024-03-28T03:44:41.428 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.428 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.429 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.429 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-03-28T03:44:41.429 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.429 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.429 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.429 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.429 INFO:teuthology.orchestra.run.smithi119.stdout: "allow_ptrace": { 2024-03-28T03:44:41.429 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "allow_ptrace", 2024-03-28T03:44:41.429 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.430 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.430 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.430 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:41.430 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.430 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.430 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.430 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-03-28T03:44:41.430 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-03-28T03:44:41.431 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.431 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.431 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.431 INFO:teuthology.orchestra.run.smithi119.stdout: "autotune_interval": { 2024-03-28T03:44:41.431 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "autotune_interval", 2024-03-28T03:44:41.431 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "secs", 2024-03-28T03:44:41.431 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.431 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.431 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "600", 2024-03-28T03:44:41.432 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.432 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.432 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.432 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "how frequently to autotune daemon memory", 2024-03-28T03:44:41.432 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.432 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.432 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.432 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.433 INFO:teuthology.orchestra.run.smithi119.stdout: "autotune_memory_target_ratio": { 2024-03-28T03:44:41.433 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "autotune_memory_target_ratio", 2024-03-28T03:44:41.433 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "float", 2024-03-28T03:44:41.433 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.433 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.433 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "0.7", 2024-03-28T03:44:41.433 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.433 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.433 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.434 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-03-28T03:44:41.434 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.434 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.434 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.434 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.434 INFO:teuthology.orchestra.run.smithi119.stdout: "cephadm_log_destination": { 2024-03-28T03:44:41.434 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "cephadm_log_destination", 2024-03-28T03:44:41.434 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.435 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.435 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.435 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:41.435 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.435 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.435 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:41.435 INFO:teuthology.orchestra.run.smithi119.stdout: "file", 2024-03-28T03:44:41.435 INFO:teuthology.orchestra.run.smithi119.stdout: "file,syslog", 2024-03-28T03:44:41.435 INFO:teuthology.orchestra.run.smithi119.stdout: "syslog" 2024-03-28T03:44:41.436 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:41.436 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-03-28T03:44:41.436 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.436 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.436 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.436 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.436 INFO:teuthology.orchestra.run.smithi119.stdout: "cgroups_split": { 2024-03-28T03:44:41.436 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "cgroups_split", 2024-03-28T03:44:41.436 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.437 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.437 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.437 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2024-03-28T03:44:41.437 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.437 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.437 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.437 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-03-28T03:44:41.437 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.438 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.438 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.438 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.438 INFO:teuthology.orchestra.run.smithi119.stdout: "config_checks_enabled": { 2024-03-28T03:44:41.438 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "config_checks_enabled", 2024-03-28T03:44:41.438 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.438 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.438 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.439 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:41.439 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.439 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.439 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.439 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-03-28T03:44:41.439 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.439 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.439 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.440 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.440 INFO:teuthology.orchestra.run.smithi119.stdout: "config_dashboard": { 2024-03-28T03:44:41.440 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "config_dashboard", 2024-03-28T03:44:41.440 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.440 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.440 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.440 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2024-03-28T03:44:41.440 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.440 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.441 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.441 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-03-28T03:44:41.441 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.441 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.441 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.441 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.441 INFO:teuthology.orchestra.run.smithi119.stdout: "container_image_alertmanager": { 2024-03-28T03:44:41.441 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "container_image_alertmanager", 2024-03-28T03:44:41.441 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.442 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.442 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.442 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-03-28T03:44:41.442 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.442 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.442 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.442 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Prometheus container image", 2024-03-28T03:44:41.442 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.443 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.443 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.443 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.443 INFO:teuthology.orchestra.run.smithi119.stdout: "container_image_base": { 2024-03-28T03:44:41.443 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "container_image_base", 2024-03-28T03:44:41.443 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.443 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.443 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.444 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "quay.io/ceph/ceph", 2024-03-28T03:44:41.444 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.444 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.444 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.444 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Container image name, without the tag", 2024-03-28T03:44:41.444 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.444 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.444 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.444 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.445 INFO:teuthology.orchestra.run.smithi119.stdout: "container_image_elasticsearch": { 2024-03-28T03:44:41.445 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "container_image_elasticsearch", 2024-03-28T03:44:41.445 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.445 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.445 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.445 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-03-28T03:44:41.445 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.445 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.446 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.446 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "elasticsearch container image", 2024-03-28T03:44:41.446 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.446 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.446 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.446 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.446 INFO:teuthology.orchestra.run.smithi119.stdout: "container_image_grafana": { 2024-03-28T03:44:41.446 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "container_image_grafana", 2024-03-28T03:44:41.446 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.447 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.447 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.447 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.12", 2024-03-28T03:44:41.447 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.447 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.447 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.447 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Prometheus container image", 2024-03-28T03:44:41.447 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.448 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.448 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.448 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.448 INFO:teuthology.orchestra.run.smithi119.stdout: "container_image_haproxy": { 2024-03-28T03:44:41.448 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "container_image_haproxy", 2024-03-28T03:44:41.448 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.448 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.448 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.448 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-03-28T03:44:41.449 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.449 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.449 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.449 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "HAproxy container image", 2024-03-28T03:44:41.449 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.449 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.449 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.449 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.450 INFO:teuthology.orchestra.run.smithi119.stdout: "container_image_jaeger_agent": { 2024-03-28T03:44:41.450 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "container_image_jaeger_agent", 2024-03-28T03:44:41.450 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.450 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.450 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.450 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-03-28T03:44:41.450 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.450 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.450 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.451 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Jaeger agent container image", 2024-03-28T03:44:41.451 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.451 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.451 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.451 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.451 INFO:teuthology.orchestra.run.smithi119.stdout: "container_image_jaeger_collector": { 2024-03-28T03:44:41.451 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "container_image_jaeger_collector", 2024-03-28T03:44:41.451 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.452 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.452 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.452 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-03-28T03:44:41.452 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.452 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.452 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.452 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Jaeger collector container image", 2024-03-28T03:44:41.452 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.453 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.453 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.453 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.453 INFO:teuthology.orchestra.run.smithi119.stdout: "container_image_jaeger_query": { 2024-03-28T03:44:41.453 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "container_image_jaeger_query", 2024-03-28T03:44:41.453 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.453 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.453 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.453 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-03-28T03:44:41.454 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.454 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.454 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.454 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Jaeger query container image", 2024-03-28T03:44:41.454 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.454 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.454 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.454 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.455 INFO:teuthology.orchestra.run.smithi119.stdout: "container_image_keepalived": { 2024-03-28T03:44:41.455 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "container_image_keepalived", 2024-03-28T03:44:41.455 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.455 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.455 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.455 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-03-28T03:44:41.455 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.455 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.455 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.456 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Keepalived container image", 2024-03-28T03:44:41.456 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.456 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.456 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.456 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.456 INFO:teuthology.orchestra.run.smithi119.stdout: "container_image_loki": { 2024-03-28T03:44:41.456 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "container_image_loki", 2024-03-28T03:44:41.456 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.457 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.457 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.457 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-03-28T03:44:41.457 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.457 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.457 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.457 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Loki container image", 2024-03-28T03:44:41.457 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.457 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.458 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.458 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.458 INFO:teuthology.orchestra.run.smithi119.stdout: "container_image_node_exporter": { 2024-03-28T03:44:41.458 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "container_image_node_exporter", 2024-03-28T03:44:41.458 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.458 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.458 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.458 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-03-28T03:44:41.459 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.459 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.459 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.459 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Prometheus container image", 2024-03-28T03:44:41.459 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.459 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.459 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.459 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.459 INFO:teuthology.orchestra.run.smithi119.stdout: "container_image_nvmeof": { 2024-03-28T03:44:41.460 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "container_image_nvmeof", 2024-03-28T03:44:41.460 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.460 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.460 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.460 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-03-28T03:44:41.460 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.460 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.460 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.460 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Nvme-of container image", 2024-03-28T03:44:41.461 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.461 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.461 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.461 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.461 INFO:teuthology.orchestra.run.smithi119.stdout: "container_image_prometheus": { 2024-03-28T03:44:41.461 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "container_image_prometheus", 2024-03-28T03:44:41.461 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.461 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.461 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.462 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-03-28T03:44:41.462 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.462 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.462 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.462 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Prometheus container image", 2024-03-28T03:44:41.462 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.462 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.462 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.463 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.463 INFO:teuthology.orchestra.run.smithi119.stdout: "container_image_promtail": { 2024-03-28T03:44:41.463 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "container_image_promtail", 2024-03-28T03:44:41.463 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.463 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.463 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.463 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-03-28T03:44:41.463 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.463 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.464 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.464 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Promtail container image", 2024-03-28T03:44:41.464 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.464 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.464 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.464 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.464 INFO:teuthology.orchestra.run.smithi119.stdout: "container_image_snmp_gateway": { 2024-03-28T03:44:41.464 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "container_image_snmp_gateway", 2024-03-28T03:44:41.465 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.465 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.465 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.465 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-03-28T03:44:41.465 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.465 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.465 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.465 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "SNMP Gateway container image", 2024-03-28T03:44:41.465 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.466 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.466 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.466 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.466 INFO:teuthology.orchestra.run.smithi119.stdout: "container_init": { 2024-03-28T03:44:41.466 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "container_init", 2024-03-28T03:44:41.466 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.466 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.466 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.467 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2024-03-28T03:44:41.467 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.467 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.467 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.467 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Run podman/docker with `--init`", 2024-03-28T03:44:41.467 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.467 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.467 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.467 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.468 INFO:teuthology.orchestra.run.smithi119.stdout: "daemon_cache_timeout": { 2024-03-28T03:44:41.468 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "daemon_cache_timeout", 2024-03-28T03:44:41.468 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "secs", 2024-03-28T03:44:41.468 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.468 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.468 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "600", 2024-03-28T03:44:41.468 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.468 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.468 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.469 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-03-28T03:44:41.469 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.469 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.469 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.469 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.469 INFO:teuthology.orchestra.run.smithi119.stdout: "default_cephadm_command_timeout": { 2024-03-28T03:44:41.469 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "default_cephadm_command_timeout", 2024-03-28T03:44:41.469 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2024-03-28T03:44:41.470 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.470 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.470 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "900", 2024-03-28T03:44:41.470 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.470 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.470 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.470 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-03-28T03:44:41.470 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.470 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.471 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.471 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.471 INFO:teuthology.orchestra.run.smithi119.stdout: "default_registry": { 2024-03-28T03:44:41.471 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "default_registry", 2024-03-28T03:44:41.471 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.471 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.471 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.471 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "docker.io", 2024-03-28T03:44:41.472 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.472 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.472 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.472 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-03-28T03:44:41.472 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.472 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.472 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.472 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.472 INFO:teuthology.orchestra.run.smithi119.stdout: "device_cache_timeout": { 2024-03-28T03:44:41.473 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "device_cache_timeout", 2024-03-28T03:44:41.473 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "secs", 2024-03-28T03:44:41.473 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.473 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.473 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "1800", 2024-03-28T03:44:41.473 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.473 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.473 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.474 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "seconds to cache device inventory", 2024-03-28T03:44:41.474 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.474 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.474 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.474 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.474 INFO:teuthology.orchestra.run.smithi119.stdout: "device_enhanced_scan": { 2024-03-28T03:44:41.474 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "device_enhanced_scan", 2024-03-28T03:44:41.474 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.475 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.475 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.475 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:41.475 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.475 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.475 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.475 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Use libstoragemgmt during device scans", 2024-03-28T03:44:41.475 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.475 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.476 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.476 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.476 INFO:teuthology.orchestra.run.smithi119.stdout: "facts_cache_timeout": { 2024-03-28T03:44:41.476 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "facts_cache_timeout", 2024-03-28T03:44:41.476 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "secs", 2024-03-28T03:44:41.476 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.476 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.476 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "60", 2024-03-28T03:44:41.476 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.477 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.477 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.477 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "seconds to cache host facts data", 2024-03-28T03:44:41.477 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.477 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.477 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.477 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.477 INFO:teuthology.orchestra.run.smithi119.stdout: "host_check_interval": { 2024-03-28T03:44:41.478 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "host_check_interval", 2024-03-28T03:44:41.478 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "secs", 2024-03-28T03:44:41.478 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.478 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.478 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "600", 2024-03-28T03:44:41.478 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.478 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.478 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.479 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "how frequently to perform a host check", 2024-03-28T03:44:41.479 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.479 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.479 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.479 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.479 INFO:teuthology.orchestra.run.smithi119.stdout: "hw_monitoring": { 2024-03-28T03:44:41.479 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "hw_monitoring", 2024-03-28T03:44:41.479 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.479 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.480 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.480 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:41.480 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.480 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.480 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.480 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-03-28T03:44:41.480 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.480 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.480 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.481 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.481 INFO:teuthology.orchestra.run.smithi119.stdout: "inventory_list_all": { 2024-03-28T03:44:41.481 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "inventory_list_all", 2024-03-28T03:44:41.481 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.481 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.481 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.481 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:41.481 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.482 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.482 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.482 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-03-28T03:44:41.482 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.482 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.482 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.482 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.483 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2024-03-28T03:44:41.483 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2024-03-28T03:44:41.483 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.483 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.483 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.483 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:41.483 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.483 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.483 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:41.484 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:41.484 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:41.484 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:41.484 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:41.484 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:41.484 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:41.484 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:41.484 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.485 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.485 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.485 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.485 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.485 INFO:teuthology.orchestra.run.smithi119.stdout: "log_refresh_metadata": { 2024-03-28T03:44:41.485 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_refresh_metadata", 2024-03-28T03:44:41.485 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.485 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.485 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.486 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:41.486 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.486 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.486 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.486 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-03-28T03:44:41.486 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.486 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.486 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.487 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.487 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2024-03-28T03:44:41.487 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2024-03-28T03:44:41.487 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.487 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.487 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.487 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2024-03-28T03:44:41.487 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.487 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.488 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.488 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-03-28T03:44:41.488 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.488 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.488 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.488 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.488 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2024-03-28T03:44:41.488 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2024-03-28T03:44:41.488 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.489 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.489 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.489 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2024-03-28T03:44:41.489 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.489 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.489 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:41.489 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:41.489 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:41.490 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:41.490 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:41.490 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:41.490 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:41.490 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:41.490 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.490 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.490 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.490 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.491 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.491 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2024-03-28T03:44:41.491 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2024-03-28T03:44:41.491 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.491 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.491 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.491 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:41.491 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.491 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.492 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.492 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.492 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.492 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.492 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.492 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.492 INFO:teuthology.orchestra.run.smithi119.stdout: "manage_etc_ceph_ceph_conf": { 2024-03-28T03:44:41.492 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-03-28T03:44:41.492 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.493 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.493 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.493 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:41.493 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.493 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.493 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.493 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-03-28T03:44:41.493 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.494 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.494 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.494 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.494 INFO:teuthology.orchestra.run.smithi119.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-03-28T03:44:41.494 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-03-28T03:44:41.494 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.494 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.494 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.495 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "*", 2024-03-28T03:44:41.495 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.495 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.495 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.495 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-03-28T03:44:41.495 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.495 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.495 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.495 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.496 INFO:teuthology.orchestra.run.smithi119.stdout: "max_count_per_host": { 2024-03-28T03:44:41.496 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "max_count_per_host", 2024-03-28T03:44:41.496 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2024-03-28T03:44:41.496 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.496 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.496 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "10", 2024-03-28T03:44:41.496 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.496 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.497 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.497 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "max number of daemons per service per host", 2024-03-28T03:44:41.497 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.497 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.497 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.497 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.497 INFO:teuthology.orchestra.run.smithi119.stdout: "max_osd_draining_count": { 2024-03-28T03:44:41.497 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "max_osd_draining_count", 2024-03-28T03:44:41.497 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2024-03-28T03:44:41.498 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.498 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.498 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "10", 2024-03-28T03:44:41.498 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.498 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.498 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.498 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-03-28T03:44:41.498 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.499 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.499 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.499 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.499 INFO:teuthology.orchestra.run.smithi119.stdout: "migration_current": { 2024-03-28T03:44:41.499 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "migration_current", 2024-03-28T03:44:41.499 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2024-03-28T03:44:41.499 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.499 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.499 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:41.500 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.500 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.500 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.500 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "internal - do not modify", 2024-03-28T03:44:41.500 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.500 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.500 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.500 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.501 INFO:teuthology.orchestra.run.smithi119.stdout: "mode": { 2024-03-28T03:44:41.501 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "mode", 2024-03-28T03:44:41.501 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.501 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.501 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.501 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "root", 2024-03-28T03:44:41.501 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.501 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.501 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:41.502 INFO:teuthology.orchestra.run.smithi119.stdout: "cephadm-package", 2024-03-28T03:44:41.502 INFO:teuthology.orchestra.run.smithi119.stdout: "root" 2024-03-28T03:44:41.502 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:41.502 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "mode for remote execution of cephadm", 2024-03-28T03:44:41.502 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.502 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.502 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.502 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.546 INFO:teuthology.orchestra.run.smithi119.stdout: "oob_default_addr": { 2024-03-28T03:44:41.546 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "oob_default_addr", 2024-03-28T03:44:41.547 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.547 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.547 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.547 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "169.254.1.1", 2024-03-28T03:44:41.547 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.547 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.547 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.547 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Default address for RedFish API (oob management).", 2024-03-28T03:44:41.548 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.548 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.548 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.548 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.548 INFO:teuthology.orchestra.run.smithi119.stdout: "prometheus_alerts_path": { 2024-03-28T03:44:41.548 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "prometheus_alerts_path", 2024-03-28T03:44:41.548 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.548 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.549 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.549 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-03-28T03:44:41.549 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.549 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.549 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.549 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-03-28T03:44:41.549 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.549 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.549 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.550 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.550 INFO:teuthology.orchestra.run.smithi119.stdout: "registry_insecure": { 2024-03-28T03:44:41.550 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "registry_insecure", 2024-03-28T03:44:41.550 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.550 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.550 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.550 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:41.551 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.551 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.551 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.551 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-03-28T03:44:41.551 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.551 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.551 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.551 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.551 INFO:teuthology.orchestra.run.smithi119.stdout: "registry_password": { 2024-03-28T03:44:41.552 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "registry_password", 2024-03-28T03:44:41.552 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.552 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.552 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.552 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:41.552 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.552 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.552 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.553 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-03-28T03:44:41.553 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.553 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.553 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.553 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.553 INFO:teuthology.orchestra.run.smithi119.stdout: "registry_url": { 2024-03-28T03:44:41.553 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "registry_url", 2024-03-28T03:44:41.553 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.554 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.554 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.554 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:41.554 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.554 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.554 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.554 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-03-28T03:44:41.554 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.555 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.555 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.555 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.555 INFO:teuthology.orchestra.run.smithi119.stdout: "registry_username": { 2024-03-28T03:44:41.555 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "registry_username", 2024-03-28T03:44:41.555 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.555 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.555 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.555 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:41.556 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.556 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.556 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.556 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-03-28T03:44:41.556 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.556 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.556 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.556 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.556 INFO:teuthology.orchestra.run.smithi119.stdout: "secure_monitoring_stack": { 2024-03-28T03:44:41.557 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "secure_monitoring_stack", 2024-03-28T03:44:41.557 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.557 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.557 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.557 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:41.557 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.557 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.557 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.558 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-03-28T03:44:41.558 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.558 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.558 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.558 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.558 INFO:teuthology.orchestra.run.smithi119.stdout: "service_discovery_port": { 2024-03-28T03:44:41.558 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "service_discovery_port", 2024-03-28T03:44:41.558 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2024-03-28T03:44:41.559 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.559 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.559 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "8765", 2024-03-28T03:44:41.559 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.559 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.559 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.559 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "cephadm service discovery port", 2024-03-28T03:44:41.559 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.559 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.560 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.560 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.560 INFO:teuthology.orchestra.run.smithi119.stdout: "ssh_config_file": { 2024-03-28T03:44:41.560 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "ssh_config_file", 2024-03-28T03:44:41.560 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.560 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.560 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.560 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:41.560 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.561 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.561 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.561 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-03-28T03:44:41.561 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.561 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.561 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.561 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.561 INFO:teuthology.orchestra.run.smithi119.stdout: "use_agent": { 2024-03-28T03:44:41.562 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "use_agent", 2024-03-28T03:44:41.562 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.562 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.562 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.562 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:41.562 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.562 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.562 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.563 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-03-28T03:44:41.563 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.563 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.563 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.563 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.563 INFO:teuthology.orchestra.run.smithi119.stdout: "use_repo_digest": { 2024-03-28T03:44:41.563 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "use_repo_digest", 2024-03-28T03:44:41.563 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.564 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.564 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.564 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2024-03-28T03:44:41.564 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.564 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.564 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.564 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-03-28T03:44:41.564 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.565 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.565 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.565 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.565 INFO:teuthology.orchestra.run.smithi119.stdout: "warn_on_failed_host_check": { 2024-03-28T03:44:41.565 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "warn_on_failed_host_check", 2024-03-28T03:44:41.565 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.565 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.565 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.565 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2024-03-28T03:44:41.566 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.566 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.566 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.566 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "raise a health warning if the host check fails", 2024-03-28T03:44:41.566 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.566 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.566 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.566 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.567 INFO:teuthology.orchestra.run.smithi119.stdout: "warn_on_stray_daemons": { 2024-03-28T03:44:41.567 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "warn_on_stray_daemons", 2024-03-28T03:44:41.567 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.567 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.567 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.567 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2024-03-28T03:44:41.567 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.567 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.567 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.568 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-03-28T03:44:41.568 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.568 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.568 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.568 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.568 INFO:teuthology.orchestra.run.smithi119.stdout: "warn_on_stray_hosts": { 2024-03-28T03:44:41.568 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "warn_on_stray_hosts", 2024-03-28T03:44:41.568 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.568 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.569 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.569 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2024-03-28T03:44:41.569 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.569 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.569 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.569 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-03-28T03:44:41.569 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.569 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.569 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.570 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:41.570 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:41.570 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.570 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-03-28T03:44:41.570 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "crash", 2024-03-28T03:44:41.570 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2024-03-28T03:44:41.570 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2024-03-28T03:44:41.570 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2024-03-28T03:44:41.571 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2024-03-28T03:44:41.571 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2024-03-28T03:44:41.571 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.571 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.571 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.571 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:41.571 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.571 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.571 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:41.572 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:41.572 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:41.572 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:41.572 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:41.572 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:41.572 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:41.572 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:41.572 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.572 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.573 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.573 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.573 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.573 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2024-03-28T03:44:41.573 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2024-03-28T03:44:41.573 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.573 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.573 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.573 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:41.574 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.574 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.574 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.574 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.574 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.574 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.574 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.574 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.575 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2024-03-28T03:44:41.575 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2024-03-28T03:44:41.575 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.575 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.575 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.575 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2024-03-28T03:44:41.575 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.575 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.575 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:41.576 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:41.576 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:41.576 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:41.576 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:41.576 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:41.576 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:41.576 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:41.576 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.576 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.577 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.577 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.577 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.577 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2024-03-28T03:44:41.577 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2024-03-28T03:44:41.577 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.577 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.577 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.577 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:41.578 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.578 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.578 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.578 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.578 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.578 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.578 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.578 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.579 INFO:teuthology.orchestra.run.smithi119.stdout: "retain_interval": { 2024-03-28T03:44:41.579 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "retain_interval", 2024-03-28T03:44:41.579 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "secs", 2024-03-28T03:44:41.579 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.579 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.579 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "31536000", 2024-03-28T03:44:41.579 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.579 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.579 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.580 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "how long to retain crashes before pruning them", 2024-03-28T03:44:41.580 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.580 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.580 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.580 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.580 INFO:teuthology.orchestra.run.smithi119.stdout: "warn_recent_interval": { 2024-03-28T03:44:41.580 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "warn_recent_interval", 2024-03-28T03:44:41.580 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "secs", 2024-03-28T03:44:41.580 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.581 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.581 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "1209600", 2024-03-28T03:44:41.581 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.581 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.581 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.581 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "time interval in which to warn about recent crashes", 2024-03-28T03:44:41.581 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.581 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.581 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.582 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:41.582 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:41.582 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.582 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-03-28T03:44:41.582 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "dashboard", 2024-03-28T03:44:41.582 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2024-03-28T03:44:41.582 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2024-03-28T03:44:41.582 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2024-03-28T03:44:41.582 INFO:teuthology.orchestra.run.smithi119.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-03-28T03:44:41.583 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-03-28T03:44:41.583 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2024-03-28T03:44:41.583 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.583 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.583 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "10", 2024-03-28T03:44:41.583 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.583 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.583 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.583 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.584 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.584 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.584 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.584 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.584 INFO:teuthology.orchestra.run.smithi119.stdout: "ALERTMANAGER_API_HOST": { 2024-03-28T03:44:41.584 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "ALERTMANAGER_API_HOST", 2024-03-28T03:44:41.584 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.584 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.584 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.585 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:41.585 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.585 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.585 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.585 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.585 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.585 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.585 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.586 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.586 INFO:teuthology.orchestra.run.smithi119.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-03-28T03:44:41.586 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-03-28T03:44:41.586 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.586 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.586 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.586 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2024-03-28T03:44:41.586 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.586 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.587 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.587 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.587 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.587 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.587 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.587 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.587 INFO:teuthology.orchestra.run.smithi119.stdout: "AUDIT_API_ENABLED": { 2024-03-28T03:44:41.587 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "AUDIT_API_ENABLED", 2024-03-28T03:44:41.587 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.588 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.588 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.588 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:41.588 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.588 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.588 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.588 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.588 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.589 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.589 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.589 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.589 INFO:teuthology.orchestra.run.smithi119.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-03-28T03:44:41.589 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-03-28T03:44:41.589 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.589 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.589 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.589 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2024-03-28T03:44:41.590 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.590 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.590 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.590 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.590 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.590 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.590 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.591 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.591 INFO:teuthology.orchestra.run.smithi119.stdout: "ENABLE_BROWSABLE_API": { 2024-03-28T03:44:41.591 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "ENABLE_BROWSABLE_API", 2024-03-28T03:44:41.591 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.591 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.591 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.591 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2024-03-28T03:44:41.591 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.591 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.592 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.592 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.592 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.592 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.592 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.592 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.592 INFO:teuthology.orchestra.run.smithi119.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-03-28T03:44:41.592 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-03-28T03:44:41.592 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.593 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.593 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.593 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2024-03-28T03:44:41.593 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.593 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.593 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.593 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.593 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.593 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.594 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.594 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.594 INFO:teuthology.orchestra.run.smithi119.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-03-28T03:44:41.594 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-03-28T03:44:41.594 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.594 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.594 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.594 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2024-03-28T03:44:41.595 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.595 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.595 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.595 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.595 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.595 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.595 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.595 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.595 INFO:teuthology.orchestra.run.smithi119.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-03-28T03:44:41.596 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-03-28T03:44:41.596 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.596 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.596 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.596 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2024-03-28T03:44:41.596 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.596 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.596 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.597 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.597 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.597 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.597 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.597 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.597 INFO:teuthology.orchestra.run.smithi119.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-03-28T03:44:41.597 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-03-28T03:44:41.597 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.598 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.598 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.598 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2024-03-28T03:44:41.598 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.598 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.598 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.598 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.598 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.599 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.599 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.599 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.599 INFO:teuthology.orchestra.run.smithi119.stdout: "FEATURE_TOGGLE_NFS": { 2024-03-28T03:44:41.599 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-03-28T03:44:41.599 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.599 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.599 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.600 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2024-03-28T03:44:41.600 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.600 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.600 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.600 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.600 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.600 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.600 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.600 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.601 INFO:teuthology.orchestra.run.smithi119.stdout: "FEATURE_TOGGLE_RBD": { 2024-03-28T03:44:41.601 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-03-28T03:44:41.601 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.601 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.601 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.601 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2024-03-28T03:44:41.601 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.601 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.602 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.602 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.602 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.602 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.602 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.602 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.602 INFO:teuthology.orchestra.run.smithi119.stdout: "FEATURE_TOGGLE_RGW": { 2024-03-28T03:44:41.602 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-03-28T03:44:41.602 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.603 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.603 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.603 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2024-03-28T03:44:41.603 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.603 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.603 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.603 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.603 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.604 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.604 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.604 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.604 INFO:teuthology.orchestra.run.smithi119.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-03-28T03:44:41.604 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-03-28T03:44:41.604 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.604 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.604 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.605 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:41.605 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.605 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.605 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.605 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.605 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.605 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.605 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.605 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.606 INFO:teuthology.orchestra.run.smithi119.stdout: "GRAFANA_API_PASSWORD": { 2024-03-28T03:44:41.606 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "GRAFANA_API_PASSWORD", 2024-03-28T03:44:41.606 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.606 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.606 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.606 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "admin", 2024-03-28T03:44:41.606 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.606 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.607 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.607 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.607 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.607 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.607 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.607 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.607 INFO:teuthology.orchestra.run.smithi119.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-03-28T03:44:41.607 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-03-28T03:44:41.607 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.608 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.608 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.608 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2024-03-28T03:44:41.608 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.608 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.608 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.608 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.608 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.609 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.609 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.609 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.609 INFO:teuthology.orchestra.run.smithi119.stdout: "GRAFANA_API_URL": { 2024-03-28T03:44:41.609 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "GRAFANA_API_URL", 2024-03-28T03:44:41.609 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.609 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.609 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.609 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:41.610 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.610 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.610 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.610 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.610 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.610 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.610 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.610 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.611 INFO:teuthology.orchestra.run.smithi119.stdout: "GRAFANA_API_USERNAME": { 2024-03-28T03:44:41.611 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "GRAFANA_API_USERNAME", 2024-03-28T03:44:41.611 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.611 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.611 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.611 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "admin", 2024-03-28T03:44:41.611 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.611 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.612 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.612 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.612 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.612 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.612 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.612 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.612 INFO:teuthology.orchestra.run.smithi119.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-03-28T03:44:41.612 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-03-28T03:44:41.612 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.613 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.613 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.613 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:41.613 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.613 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.613 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.613 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.613 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.614 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.614 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.614 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.614 INFO:teuthology.orchestra.run.smithi119.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-03-28T03:44:41.614 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-03-28T03:44:41.614 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.614 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.614 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.615 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:41.615 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.615 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.615 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.615 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.615 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.615 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.616 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.616 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.616 INFO:teuthology.orchestra.run.smithi119.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-03-28T03:44:41.616 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-03-28T03:44:41.616 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.616 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.616 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.616 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2024-03-28T03:44:41.616 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.617 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.617 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.617 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.617 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.617 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.617 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.617 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.617 INFO:teuthology.orchestra.run.smithi119.stdout: "ISSUE_TRACKER_API_KEY": { 2024-03-28T03:44:41.618 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-03-28T03:44:41.618 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.618 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.618 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.618 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:41.618 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.618 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.618 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.619 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.619 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.619 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.619 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.619 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.619 INFO:teuthology.orchestra.run.smithi119.stdout: "MULTICLUSTER_CONFIG": { 2024-03-28T03:44:41.619 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "MULTICLUSTER_CONFIG", 2024-03-28T03:44:41.619 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.620 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.620 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.620 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "{}", 2024-03-28T03:44:41.620 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.620 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.620 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.620 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.620 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.620 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.621 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.621 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.621 INFO:teuthology.orchestra.run.smithi119.stdout: "PROMETHEUS_API_HOST": { 2024-03-28T03:44:41.621 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "PROMETHEUS_API_HOST", 2024-03-28T03:44:41.621 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.621 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.621 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.621 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:41.622 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.622 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.622 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.622 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.622 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.622 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.622 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.622 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.623 INFO:teuthology.orchestra.run.smithi119.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-03-28T03:44:41.623 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-03-28T03:44:41.623 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.623 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.623 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.623 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2024-03-28T03:44:41.623 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.623 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.624 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.624 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.624 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.624 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.624 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.624 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.624 INFO:teuthology.orchestra.run.smithi119.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-03-28T03:44:41.624 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-03-28T03:44:41.625 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.625 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.625 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.625 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:41.625 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.625 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.625 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.625 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.626 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.626 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.626 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.626 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.626 INFO:teuthology.orchestra.run.smithi119.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-03-28T03:44:41.626 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-03-28T03:44:41.626 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.626 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.626 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.627 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:41.627 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.627 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.627 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.627 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.627 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.627 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.627 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.628 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.628 INFO:teuthology.orchestra.run.smithi119.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-03-28T03:44:41.628 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-03-28T03:44:41.628 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.628 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.628 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.628 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2024-03-28T03:44:41.628 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.629 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.629 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.629 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.629 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.629 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.629 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.629 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.630 INFO:teuthology.orchestra.run.smithi119.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-03-28T03:44:41.630 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-03-28T03:44:41.630 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.630 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.630 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.630 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2024-03-28T03:44:41.630 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.630 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.630 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.631 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.631 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.631 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.631 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.631 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.631 INFO:teuthology.orchestra.run.smithi119.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-03-28T03:44:41.631 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-03-28T03:44:41.631 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.632 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.632 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.632 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:41.632 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.632 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.632 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.632 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.632 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.633 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.633 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.633 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.633 INFO:teuthology.orchestra.run.smithi119.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-03-28T03:44:41.633 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-03-28T03:44:41.633 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.633 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.633 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.633 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:41.634 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.634 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.634 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.634 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.634 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.634 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.634 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.634 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.635 INFO:teuthology.orchestra.run.smithi119.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-03-28T03:44:41.635 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-03-28T03:44:41.635 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.635 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.635 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.635 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:41.635 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.635 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.636 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.636 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.636 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.636 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.636 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.636 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.636 INFO:teuthology.orchestra.run.smithi119.stdout: "PWD_POLICY_ENABLED": { 2024-03-28T03:44:41.636 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "PWD_POLICY_ENABLED", 2024-03-28T03:44:41.637 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.637 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.637 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.637 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2024-03-28T03:44:41.637 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.637 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.637 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.637 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.638 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.638 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.638 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.638 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.638 INFO:teuthology.orchestra.run.smithi119.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-03-28T03:44:41.638 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-03-28T03:44:41.638 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.638 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.639 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.639 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-03-28T03:44:41.639 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.639 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.639 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.639 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.639 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.639 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.640 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.640 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.640 INFO:teuthology.orchestra.run.smithi119.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-03-28T03:44:41.640 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-03-28T03:44:41.640 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2024-03-28T03:44:41.640 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.640 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.640 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "10", 2024-03-28T03:44:41.640 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.641 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.641 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.641 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.641 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.641 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.641 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.641 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.641 INFO:teuthology.orchestra.run.smithi119.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-03-28T03:44:41.642 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-03-28T03:44:41.642 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2024-03-28T03:44:41.642 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.642 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.642 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "8", 2024-03-28T03:44:41.642 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.642 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.642 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.643 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.643 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.643 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.643 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.643 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.643 INFO:teuthology.orchestra.run.smithi119.stdout: "REST_REQUESTS_TIMEOUT": { 2024-03-28T03:44:41.643 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-03-28T03:44:41.643 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2024-03-28T03:44:41.644 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.644 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.644 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "45", 2024-03-28T03:44:41.644 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.644 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.644 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.644 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.644 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.645 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.645 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.645 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.645 INFO:teuthology.orchestra.run.smithi119.stdout: "RGW_API_ACCESS_KEY": { 2024-03-28T03:44:41.645 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "RGW_API_ACCESS_KEY", 2024-03-28T03:44:41.645 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.645 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.645 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.646 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:41.646 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.646 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.646 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.646 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.646 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.646 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.646 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.647 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.647 INFO:teuthology.orchestra.run.smithi119.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-03-28T03:44:41.647 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-03-28T03:44:41.647 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.647 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.647 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.647 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "admin", 2024-03-28T03:44:41.647 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.648 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.648 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.648 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.648 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.648 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.648 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.648 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.648 INFO:teuthology.orchestra.run.smithi119.stdout: "RGW_API_SECRET_KEY": { 2024-03-28T03:44:41.648 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "RGW_API_SECRET_KEY", 2024-03-28T03:44:41.648 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.649 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.649 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.649 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:41.649 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.649 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.649 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.649 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.649 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.650 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.650 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.650 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.650 INFO:teuthology.orchestra.run.smithi119.stdout: "RGW_API_SSL_VERIFY": { 2024-03-28T03:44:41.650 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "RGW_API_SSL_VERIFY", 2024-03-28T03:44:41.650 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.650 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.650 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.651 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2024-03-28T03:44:41.651 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.651 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.651 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.651 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.651 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.651 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.651 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.651 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.652 INFO:teuthology.orchestra.run.smithi119.stdout: "UNSAFE_TLS_v1_2": { 2024-03-28T03:44:41.652 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "UNSAFE_TLS_v1_2", 2024-03-28T03:44:41.652 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.652 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.652 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.652 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:41.652 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.652 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.652 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.653 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.653 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.653 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.653 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.653 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.653 INFO:teuthology.orchestra.run.smithi119.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-03-28T03:44:41.653 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-03-28T03:44:41.653 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2024-03-28T03:44:41.654 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.654 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.654 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "0", 2024-03-28T03:44:41.654 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.654 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.654 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.654 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.654 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.654 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.655 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.655 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.655 INFO:teuthology.orchestra.run.smithi119.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-03-28T03:44:41.655 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-03-28T03:44:41.655 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2024-03-28T03:44:41.655 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.655 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.655 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "10", 2024-03-28T03:44:41.656 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.656 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.656 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.656 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.656 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.656 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.656 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.656 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.656 INFO:teuthology.orchestra.run.smithi119.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-03-28T03:44:41.657 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-03-28T03:44:41.657 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2024-03-28T03:44:41.657 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.657 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.657 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "5", 2024-03-28T03:44:41.657 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.657 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.657 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.658 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.658 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.658 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.658 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.658 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.658 INFO:teuthology.orchestra.run.smithi119.stdout: "cross_origin_url": { 2024-03-28T03:44:41.658 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "cross_origin_url", 2024-03-28T03:44:41.658 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.659 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.659 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.659 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:41.659 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.659 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.659 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.659 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.659 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.659 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.660 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.660 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.660 INFO:teuthology.orchestra.run.smithi119.stdout: "crt_file": { 2024-03-28T03:44:41.660 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "crt_file", 2024-03-28T03:44:41.660 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.660 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.660 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.660 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:41.661 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.661 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.661 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.661 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.661 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.661 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.661 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.661 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.661 INFO:teuthology.orchestra.run.smithi119.stdout: "debug": { 2024-03-28T03:44:41.662 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "debug", 2024-03-28T03:44:41.662 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.662 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.662 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.662 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:41.662 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.662 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.662 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.663 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Enable/disable debug options", 2024-03-28T03:44:41.663 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.663 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.663 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.663 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.663 INFO:teuthology.orchestra.run.smithi119.stdout: "jwt_token_ttl": { 2024-03-28T03:44:41.663 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "jwt_token_ttl", 2024-03-28T03:44:41.663 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2024-03-28T03:44:41.663 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.664 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.664 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "28800", 2024-03-28T03:44:41.664 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.664 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.664 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.664 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.664 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.665 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.665 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.665 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.665 INFO:teuthology.orchestra.run.smithi119.stdout: "key_file": { 2024-03-28T03:44:41.665 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "key_file", 2024-03-28T03:44:41.665 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.665 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.665 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.665 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:41.666 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.666 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.666 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.666 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.666 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.666 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.666 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.666 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.667 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2024-03-28T03:44:41.667 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2024-03-28T03:44:41.667 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.667 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.667 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.667 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:41.667 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.667 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.668 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:41.668 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:41.668 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:41.668 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:41.668 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:41.668 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:41.668 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:41.668 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:41.668 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.669 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.669 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.669 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.669 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.669 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2024-03-28T03:44:41.669 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2024-03-28T03:44:41.669 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.669 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.669 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.670 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:41.670 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.670 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.670 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.670 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.670 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.670 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.670 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.671 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.671 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2024-03-28T03:44:41.671 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2024-03-28T03:44:41.671 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.671 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.671 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.671 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2024-03-28T03:44:41.671 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.672 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.672 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:41.672 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:41.672 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:41.672 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:41.672 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:41.672 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:41.672 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:41.673 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:41.673 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.673 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.673 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.673 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.673 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.673 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2024-03-28T03:44:41.673 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2024-03-28T03:44:41.674 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.674 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.674 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.674 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:41.674 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.674 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.674 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.674 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.675 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.675 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.675 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.675 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.675 INFO:teuthology.orchestra.run.smithi119.stdout: "motd": { 2024-03-28T03:44:41.675 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "motd", 2024-03-28T03:44:41.675 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.675 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.675 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.676 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:41.676 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.676 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.676 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.676 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "The message of the day", 2024-03-28T03:44:41.676 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.676 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.676 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.677 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.677 INFO:teuthology.orchestra.run.smithi119.stdout: "redirect_resolve_ip_addr": { 2024-03-28T03:44:41.677 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "redirect_resolve_ip_addr", 2024-03-28T03:44:41.677 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.677 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.677 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.677 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:41.677 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.677 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.678 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.678 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.678 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.678 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.678 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.678 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.678 INFO:teuthology.orchestra.run.smithi119.stdout: "server_addr": { 2024-03-28T03:44:41.678 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "server_addr", 2024-03-28T03:44:41.679 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.679 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.679 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.679 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "::", 2024-03-28T03:44:41.679 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.679 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.679 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.679 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.680 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.680 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.680 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.680 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.680 INFO:teuthology.orchestra.run.smithi119.stdout: "server_port": { 2024-03-28T03:44:41.680 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "server_port", 2024-03-28T03:44:41.680 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2024-03-28T03:44:41.680 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.680 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.681 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "8080", 2024-03-28T03:44:41.681 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.681 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.681 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.681 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.681 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.681 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.681 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.681 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.682 INFO:teuthology.orchestra.run.smithi119.stdout: "ssl": { 2024-03-28T03:44:41.682 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "ssl", 2024-03-28T03:44:41.682 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.682 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.682 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.682 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2024-03-28T03:44:41.682 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.682 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.683 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.683 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.683 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.683 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.683 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.683 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.683 INFO:teuthology.orchestra.run.smithi119.stdout: "ssl_server_port": { 2024-03-28T03:44:41.683 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "ssl_server_port", 2024-03-28T03:44:41.683 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2024-03-28T03:44:41.684 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.684 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.684 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "8443", 2024-03-28T03:44:41.684 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.684 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.684 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.684 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.684 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.685 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.685 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.685 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.685 INFO:teuthology.orchestra.run.smithi119.stdout: "standby_behaviour": { 2024-03-28T03:44:41.685 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "standby_behaviour", 2024-03-28T03:44:41.685 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.685 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.685 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.686 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "redirect", 2024-03-28T03:44:41.686 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.686 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.686 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:41.686 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:41.686 INFO:teuthology.orchestra.run.smithi119.stdout: "redirect" 2024-03-28T03:44:41.686 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:41.686 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.687 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.687 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.687 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.687 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.687 INFO:teuthology.orchestra.run.smithi119.stdout: "standby_error_status_code": { 2024-03-28T03:44:41.687 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "standby_error_status_code", 2024-03-28T03:44:41.687 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2024-03-28T03:44:41.687 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.687 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.688 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "500", 2024-03-28T03:44:41.688 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "400", 2024-03-28T03:44:41.688 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "599", 2024-03-28T03:44:41.688 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.688 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.688 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.688 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.688 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.688 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.689 INFO:teuthology.orchestra.run.smithi119.stdout: "url_prefix": { 2024-03-28T03:44:41.689 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "url_prefix", 2024-03-28T03:44:41.689 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.689 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.689 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.689 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:41.689 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.689 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.689 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.690 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.690 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.690 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.690 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.690 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:41.690 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:41.690 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.690 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-03-28T03:44:41.691 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "devicehealth", 2024-03-28T03:44:41.691 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2024-03-28T03:44:41.691 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2024-03-28T03:44:41.691 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2024-03-28T03:44:41.691 INFO:teuthology.orchestra.run.smithi119.stdout: "enable_monitoring": { 2024-03-28T03:44:41.691 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "enable_monitoring", 2024-03-28T03:44:41.691 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.691 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.692 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.692 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2024-03-28T03:44:41.692 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.692 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.692 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.692 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "monitor device health metrics", 2024-03-28T03:44:41.692 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.692 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.692 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.693 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.693 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2024-03-28T03:44:41.693 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2024-03-28T03:44:41.693 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.693 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.693 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.693 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:41.693 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.693 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.694 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:41.694 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:41.694 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:41.694 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:41.694 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:41.694 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:41.694 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:41.694 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:41.695 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.695 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.695 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.695 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.695 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.695 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2024-03-28T03:44:41.695 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2024-03-28T03:44:41.695 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.695 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.696 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.696 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:41.696 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.696 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.696 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.696 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.696 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.696 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.696 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.697 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.697 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2024-03-28T03:44:41.697 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2024-03-28T03:44:41.697 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.697 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.697 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.697 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2024-03-28T03:44:41.697 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.698 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.698 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:41.698 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:41.698 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:41.698 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:41.698 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:41.698 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:41.698 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:41.699 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:41.699 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.699 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.699 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.699 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.699 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.699 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2024-03-28T03:44:41.699 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2024-03-28T03:44:41.699 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.700 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.700 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.700 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:41.700 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.700 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.700 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.700 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.700 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.700 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.701 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.701 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.701 INFO:teuthology.orchestra.run.smithi119.stdout: "mark_out_threshold": { 2024-03-28T03:44:41.701 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "mark_out_threshold", 2024-03-28T03:44:41.701 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "secs", 2024-03-28T03:44:41.701 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.701 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.701 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "2419200", 2024-03-28T03:44:41.701 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.702 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.702 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.702 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-03-28T03:44:41.702 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.702 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.702 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.702 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.702 INFO:teuthology.orchestra.run.smithi119.stdout: "pool_name": { 2024-03-28T03:44:41.703 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "pool_name", 2024-03-28T03:44:41.703 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.703 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.703 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.703 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "device_health_metrics", 2024-03-28T03:44:41.703 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.703 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.703 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.704 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "name of pool in which to store device health metrics", 2024-03-28T03:44:41.704 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.704 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.704 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.704 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.704 INFO:teuthology.orchestra.run.smithi119.stdout: "retention_period": { 2024-03-28T03:44:41.704 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "retention_period", 2024-03-28T03:44:41.704 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "secs", 2024-03-28T03:44:41.704 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.705 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.705 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "15552000", 2024-03-28T03:44:41.705 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.705 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.705 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.705 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "how long to retain device health metrics", 2024-03-28T03:44:41.705 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.705 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.706 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.706 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.706 INFO:teuthology.orchestra.run.smithi119.stdout: "scrape_frequency": { 2024-03-28T03:44:41.706 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "scrape_frequency", 2024-03-28T03:44:41.706 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "secs", 2024-03-28T03:44:41.706 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.706 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.706 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "86400", 2024-03-28T03:44:41.707 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.707 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.707 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.707 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "how frequently to scrape device health metrics", 2024-03-28T03:44:41.707 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.707 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.707 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.707 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.708 INFO:teuthology.orchestra.run.smithi119.stdout: "self_heal": { 2024-03-28T03:44:41.708 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "self_heal", 2024-03-28T03:44:41.708 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.708 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.708 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.708 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2024-03-28T03:44:41.708 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.708 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.708 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.709 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-03-28T03:44:41.709 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.709 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.709 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.709 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.709 INFO:teuthology.orchestra.run.smithi119.stdout: "sleep_interval": { 2024-03-28T03:44:41.709 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "sleep_interval", 2024-03-28T03:44:41.709 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "secs", 2024-03-28T03:44:41.709 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.710 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.710 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "600", 2024-03-28T03:44:41.710 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.710 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.710 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.710 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "how frequently to wake up and check device health", 2024-03-28T03:44:41.710 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.710 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.711 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.711 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.711 INFO:teuthology.orchestra.run.smithi119.stdout: "warn_threshold": { 2024-03-28T03:44:41.711 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "warn_threshold", 2024-03-28T03:44:41.711 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "secs", 2024-03-28T03:44:41.711 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.711 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.711 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "7257600", 2024-03-28T03:44:41.712 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.712 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.712 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.712 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-03-28T03:44:41.712 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.712 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.712 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.712 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:41.713 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:41.713 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.713 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-03-28T03:44:41.713 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "diskprediction_local", 2024-03-28T03:44:41.713 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2024-03-28T03:44:41.713 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2024-03-28T03:44:41.713 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2024-03-28T03:44:41.713 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2024-03-28T03:44:41.713 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2024-03-28T03:44:41.714 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.714 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.714 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.714 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:41.714 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.714 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.714 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:41.714 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:41.715 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:41.715 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:41.715 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:41.715 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:41.715 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:41.715 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:41.715 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.715 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.716 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.716 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.716 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.716 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2024-03-28T03:44:41.716 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2024-03-28T03:44:41.716 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.716 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.716 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.716 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:41.717 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.717 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.717 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.717 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.717 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.717 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.717 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.717 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.718 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2024-03-28T03:44:41.718 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2024-03-28T03:44:41.718 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.718 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.718 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.718 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2024-03-28T03:44:41.718 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.718 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.719 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:41.719 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:41.719 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:41.719 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:41.719 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:41.719 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:41.719 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:41.719 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:41.719 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.719 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.720 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.720 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.720 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.720 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2024-03-28T03:44:41.720 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2024-03-28T03:44:41.720 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.720 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.720 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.721 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:41.721 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.721 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.721 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.721 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.721 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.721 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.721 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.721 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.722 INFO:teuthology.orchestra.run.smithi119.stdout: "predict_interval": { 2024-03-28T03:44:41.722 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "predict_interval", 2024-03-28T03:44:41.722 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.722 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.722 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.722 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "86400", 2024-03-28T03:44:41.722 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.722 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.723 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.723 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.723 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.723 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.723 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.723 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.723 INFO:teuthology.orchestra.run.smithi119.stdout: "predictor_model": { 2024-03-28T03:44:41.723 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "predictor_model", 2024-03-28T03:44:41.723 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.724 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.724 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.724 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "prophetstor", 2024-03-28T03:44:41.724 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.724 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.724 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.725 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.725 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.725 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.725 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.725 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.725 INFO:teuthology.orchestra.run.smithi119.stdout: "sleep_interval": { 2024-03-28T03:44:41.725 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "sleep_interval", 2024-03-28T03:44:41.725 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.725 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.726 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.726 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "600", 2024-03-28T03:44:41.726 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.726 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.726 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.726 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.726 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.726 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.727 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.727 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:41.727 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:41.727 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.727 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-03-28T03:44:41.727 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "influx", 2024-03-28T03:44:41.727 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": false, 2024-03-28T03:44:41.727 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "influxdb python module not found", 2024-03-28T03:44:41.727 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2024-03-28T03:44:41.728 INFO:teuthology.orchestra.run.smithi119.stdout: "batch_size": { 2024-03-28T03:44:41.728 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "batch_size", 2024-03-28T03:44:41.728 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2024-03-28T03:44:41.728 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.728 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.728 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "5000", 2024-03-28T03:44:41.728 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.728 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.729 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.729 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-03-28T03:44:41.729 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.729 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.729 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.729 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.729 INFO:teuthology.orchestra.run.smithi119.stdout: "database": { 2024-03-28T03:44:41.729 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "database", 2024-03-28T03:44:41.729 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.730 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.730 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.730 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "ceph", 2024-03-28T03:44:41.730 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.730 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.730 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.730 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-03-28T03:44:41.730 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.731 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.731 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.731 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.731 INFO:teuthology.orchestra.run.smithi119.stdout: "hostname": { 2024-03-28T03:44:41.731 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "hostname", 2024-03-28T03:44:41.731 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.731 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.731 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.731 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:41.732 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.732 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.732 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.732 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "InfluxDB server hostname", 2024-03-28T03:44:41.732 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.732 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.732 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.732 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.732 INFO:teuthology.orchestra.run.smithi119.stdout: "interval": { 2024-03-28T03:44:41.733 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "interval", 2024-03-28T03:44:41.733 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "secs", 2024-03-28T03:44:41.733 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.733 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.733 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "30", 2024-03-28T03:44:41.733 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "5", 2024-03-28T03:44:41.733 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.733 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.733 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-03-28T03:44:41.734 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.734 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.734 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.734 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.734 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2024-03-28T03:44:41.734 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2024-03-28T03:44:41.734 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.734 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.735 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.735 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:41.735 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.735 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.735 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:41.735 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:41.735 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:41.735 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:41.735 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:41.735 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:41.736 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:41.736 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:41.736 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.736 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.736 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.736 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.736 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.736 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2024-03-28T03:44:41.736 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2024-03-28T03:44:41.737 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.737 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.737 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.737 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:41.737 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.737 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.737 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.737 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.737 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.738 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.738 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.738 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.738 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2024-03-28T03:44:41.738 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2024-03-28T03:44:41.738 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.738 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.738 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.738 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2024-03-28T03:44:41.739 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.739 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.739 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:41.739 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:41.739 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:41.739 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:41.739 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:41.739 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:41.739 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:41.740 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:41.740 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.740 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.740 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.740 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.740 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.740 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2024-03-28T03:44:41.740 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2024-03-28T03:44:41.740 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.741 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.741 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.741 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:41.741 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.741 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.741 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.741 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.741 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.741 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.742 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.742 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.742 INFO:teuthology.orchestra.run.smithi119.stdout: "password": { 2024-03-28T03:44:41.742 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "password", 2024-03-28T03:44:41.742 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.742 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.742 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.742 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:41.743 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.743 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.743 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.743 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "password of InfluxDB server user", 2024-03-28T03:44:41.743 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.743 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.743 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.743 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.743 INFO:teuthology.orchestra.run.smithi119.stdout: "port": { 2024-03-28T03:44:41.744 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "port", 2024-03-28T03:44:41.744 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2024-03-28T03:44:41.744 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.744 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.744 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "8086", 2024-03-28T03:44:41.744 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.744 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.744 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.744 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "InfluxDB server port", 2024-03-28T03:44:41.745 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.745 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.745 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.745 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.745 INFO:teuthology.orchestra.run.smithi119.stdout: "ssl": { 2024-03-28T03:44:41.745 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "ssl", 2024-03-28T03:44:41.745 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.745 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.745 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.746 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "false", 2024-03-28T03:44:41.746 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.746 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.746 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.746 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-03-28T03:44:41.746 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.746 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.746 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.746 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.747 INFO:teuthology.orchestra.run.smithi119.stdout: "threads": { 2024-03-28T03:44:41.747 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "threads", 2024-03-28T03:44:41.747 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2024-03-28T03:44:41.747 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.747 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.747 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "5", 2024-03-28T03:44:41.747 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "1", 2024-03-28T03:44:41.747 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "32", 2024-03-28T03:44:41.747 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.748 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-03-28T03:44:41.748 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.748 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.748 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.748 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.748 INFO:teuthology.orchestra.run.smithi119.stdout: "username": { 2024-03-28T03:44:41.748 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "username", 2024-03-28T03:44:41.748 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.748 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.748 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.749 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:41.749 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.749 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.749 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.749 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "username of InfluxDB server user", 2024-03-28T03:44:41.749 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.749 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.749 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.749 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.750 INFO:teuthology.orchestra.run.smithi119.stdout: "verify_ssl": { 2024-03-28T03:44:41.750 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "verify_ssl", 2024-03-28T03:44:41.750 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.750 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.750 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.750 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "true", 2024-03-28T03:44:41.750 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.750 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.751 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.751 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-03-28T03:44:41.751 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.751 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.751 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.751 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:41.751 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:41.751 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.751 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-03-28T03:44:41.752 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "insights", 2024-03-28T03:44:41.752 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2024-03-28T03:44:41.752 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2024-03-28T03:44:41.752 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2024-03-28T03:44:41.752 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2024-03-28T03:44:41.752 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2024-03-28T03:44:41.752 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.752 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.752 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.753 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:41.753 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.753 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.753 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:41.753 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:41.753 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:41.753 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:41.753 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:41.753 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:41.753 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:41.754 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:41.754 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.754 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.754 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.754 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.754 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.754 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2024-03-28T03:44:41.754 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2024-03-28T03:44:41.755 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.755 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.755 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.755 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:41.755 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.755 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.755 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.755 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.755 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.756 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.756 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.756 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.756 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2024-03-28T03:44:41.756 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2024-03-28T03:44:41.756 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.756 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.756 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.756 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2024-03-28T03:44:41.757 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.757 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.757 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:41.757 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:41.757 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:41.757 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:41.757 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:41.757 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:41.757 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:41.758 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:41.758 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.758 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.758 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.758 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.758 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.758 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2024-03-28T03:44:41.758 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2024-03-28T03:44:41.759 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.759 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.759 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.759 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:41.759 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.759 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.759 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.759 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.759 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.760 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.760 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.760 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:41.760 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:41.760 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.760 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-03-28T03:44:41.760 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "iostat", 2024-03-28T03:44:41.760 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2024-03-28T03:44:41.760 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2024-03-28T03:44:41.761 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2024-03-28T03:44:41.761 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2024-03-28T03:44:41.761 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2024-03-28T03:44:41.761 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.761 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.761 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.761 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:41.761 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.761 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.762 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:41.762 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:41.762 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:41.762 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:41.762 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:41.762 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:41.762 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:41.762 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:41.762 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.763 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.763 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.763 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.763 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.763 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2024-03-28T03:44:41.763 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2024-03-28T03:44:41.763 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.763 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.763 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.764 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:41.764 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.764 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.764 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.764 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.764 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.764 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.764 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.764 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.765 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2024-03-28T03:44:41.765 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2024-03-28T03:44:41.765 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.765 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.765 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.765 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2024-03-28T03:44:41.765 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.765 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.766 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:41.766 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:41.766 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:41.766 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:41.766 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:41.766 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:41.766 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:41.766 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:41.766 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.767 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.767 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.767 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.767 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.767 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2024-03-28T03:44:41.767 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2024-03-28T03:44:41.767 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.767 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.767 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.768 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:41.768 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.768 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.768 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.768 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.768 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.768 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.768 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.768 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:41.769 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:41.769 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.769 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-03-28T03:44:41.769 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "k8sevents", 2024-03-28T03:44:41.769 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2024-03-28T03:44:41.769 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2024-03-28T03:44:41.769 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2024-03-28T03:44:41.769 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph_event_retention_days": { 2024-03-28T03:44:41.769 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "ceph_event_retention_days", 2024-03-28T03:44:41.770 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2024-03-28T03:44:41.770 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.770 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.770 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "7", 2024-03-28T03:44:41.770 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.770 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.770 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.770 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Days to hold ceph event information within local cache", 2024-03-28T03:44:41.770 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.771 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.771 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.771 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.771 INFO:teuthology.orchestra.run.smithi119.stdout: "config_check_secs": { 2024-03-28T03:44:41.771 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "config_check_secs", 2024-03-28T03:44:41.771 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2024-03-28T03:44:41.771 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.771 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.771 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "10", 2024-03-28T03:44:41.772 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "10", 2024-03-28T03:44:41.772 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.772 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.772 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-03-28T03:44:41.772 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.772 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.772 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.772 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.772 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2024-03-28T03:44:41.773 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2024-03-28T03:44:41.773 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.773 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.773 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.773 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:41.773 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.773 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.773 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:41.773 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:41.774 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:41.774 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:41.774 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:41.774 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:41.774 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:41.774 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:41.774 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.774 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.774 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.775 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.775 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.775 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2024-03-28T03:44:41.775 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2024-03-28T03:44:41.775 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.775 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.775 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.775 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:41.775 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.776 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.776 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.776 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.776 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.776 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.776 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.776 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.776 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2024-03-28T03:44:41.776 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2024-03-28T03:44:41.776 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.777 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.777 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.777 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2024-03-28T03:44:41.777 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.777 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.777 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:41.777 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:41.777 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:41.777 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:41.778 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:41.778 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:41.778 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:41.778 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:41.778 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.778 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.778 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.778 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.779 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.779 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2024-03-28T03:44:41.779 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2024-03-28T03:44:41.779 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.779 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.779 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.779 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:41.779 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.779 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.779 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.780 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.780 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.780 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.780 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.780 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:41.780 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:41.780 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.780 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-03-28T03:44:41.780 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "localpool", 2024-03-28T03:44:41.781 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2024-03-28T03:44:41.781 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2024-03-28T03:44:41.781 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2024-03-28T03:44:41.781 INFO:teuthology.orchestra.run.smithi119.stdout: "failure_domain": { 2024-03-28T03:44:41.781 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "failure_domain", 2024-03-28T03:44:41.781 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.781 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.781 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.781 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "host", 2024-03-28T03:44:41.782 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.782 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.782 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.782 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "failure domain for any created local pool", 2024-03-28T03:44:41.782 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-03-28T03:44:41.782 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.782 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.782 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.782 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2024-03-28T03:44:41.783 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2024-03-28T03:44:41.783 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.783 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.783 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.783 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:41.783 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.783 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.783 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:41.783 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:41.784 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:41.784 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:41.784 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:41.784 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:41.784 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:41.784 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:41.784 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.784 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.784 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.784 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.785 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.785 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2024-03-28T03:44:41.785 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2024-03-28T03:44:41.785 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.785 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.785 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.785 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:41.785 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.785 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.786 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.786 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.786 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.786 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.786 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.786 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.786 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2024-03-28T03:44:41.786 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2024-03-28T03:44:41.786 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.787 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.787 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.787 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2024-03-28T03:44:41.787 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.787 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.787 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:41.787 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:41.787 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:41.787 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:41.788 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:41.788 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:41.788 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:41.788 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:41.788 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.788 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.788 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.788 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.788 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.788 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2024-03-28T03:44:41.789 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2024-03-28T03:44:41.789 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.789 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.789 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.789 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:41.789 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.789 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.789 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.789 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.790 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.790 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.790 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.790 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.790 INFO:teuthology.orchestra.run.smithi119.stdout: "min_size": { 2024-03-28T03:44:41.790 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "min_size", 2024-03-28T03:44:41.790 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2024-03-28T03:44:41.790 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.790 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.791 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:41.791 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.791 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.791 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.791 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "default min_size for any created local pool", 2024-03-28T03:44:41.791 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-03-28T03:44:41.791 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.791 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.791 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.792 INFO:teuthology.orchestra.run.smithi119.stdout: "num_rep": { 2024-03-28T03:44:41.792 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "num_rep", 2024-03-28T03:44:41.792 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2024-03-28T03:44:41.792 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.792 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.792 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "3", 2024-03-28T03:44:41.792 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.792 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.792 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.792 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "default replica count for any created local pool", 2024-03-28T03:44:41.793 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.793 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.793 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.793 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.793 INFO:teuthology.orchestra.run.smithi119.stdout: "pg_num": { 2024-03-28T03:44:41.793 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "pg_num", 2024-03-28T03:44:41.793 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2024-03-28T03:44:41.793 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.793 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.794 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "128", 2024-03-28T03:44:41.794 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.794 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.794 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.794 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "default pg_num for any created local pool", 2024-03-28T03:44:41.794 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.794 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.794 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.794 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.795 INFO:teuthology.orchestra.run.smithi119.stdout: "prefix": { 2024-03-28T03:44:41.795 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "prefix", 2024-03-28T03:44:41.795 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.795 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.795 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.795 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:41.795 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.795 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.795 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.795 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "name prefix for any created local pool", 2024-03-28T03:44:41.796 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.796 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.796 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.796 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.796 INFO:teuthology.orchestra.run.smithi119.stdout: "subtree": { 2024-03-28T03:44:41.796 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "subtree", 2024-03-28T03:44:41.796 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.796 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.796 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.797 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "rack", 2024-03-28T03:44:41.797 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.797 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.797 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.797 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "CRUSH level for which to create a local pool", 2024-03-28T03:44:41.797 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-03-28T03:44:41.797 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.797 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.797 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:41.798 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:41.798 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.798 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-03-28T03:44:41.798 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "mds_autoscaler", 2024-03-28T03:44:41.798 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2024-03-28T03:44:41.798 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2024-03-28T03:44:41.798 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2024-03-28T03:44:41.798 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2024-03-28T03:44:41.799 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2024-03-28T03:44:41.799 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.799 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.799 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.799 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:41.799 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.799 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.799 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:41.799 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:41.800 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:41.800 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:41.800 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:41.800 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:41.800 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:41.800 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:41.800 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.800 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.800 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.800 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.801 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.801 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2024-03-28T03:44:41.801 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2024-03-28T03:44:41.801 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.801 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.801 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.801 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:41.801 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.801 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.802 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.802 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.802 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.802 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.802 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.802 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.802 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2024-03-28T03:44:41.802 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2024-03-28T03:44:41.803 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.803 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.803 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.803 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2024-03-28T03:44:41.803 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.803 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.803 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:41.803 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:41.803 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:41.803 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:41.804 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:41.804 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:41.804 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:41.804 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:41.804 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.804 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.804 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.804 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.804 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.805 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2024-03-28T03:44:41.805 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2024-03-28T03:44:41.805 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.805 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.805 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.805 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:41.805 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.805 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.805 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.806 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.806 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.806 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.806 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.806 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:41.806 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:41.806 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.806 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-03-28T03:44:41.807 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "mirroring", 2024-03-28T03:44:41.807 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2024-03-28T03:44:41.807 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2024-03-28T03:44:41.807 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2024-03-28T03:44:41.807 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2024-03-28T03:44:41.807 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2024-03-28T03:44:41.807 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.807 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.807 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.808 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:41.808 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.808 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.808 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:41.808 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:41.808 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:41.808 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:41.808 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:41.808 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:41.809 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:41.809 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:41.809 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.809 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.809 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.809 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.809 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.809 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2024-03-28T03:44:41.809 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2024-03-28T03:44:41.809 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.810 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.810 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.810 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:41.810 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.810 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.810 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.810 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.810 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.810 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.811 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.811 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.811 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2024-03-28T03:44:41.811 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2024-03-28T03:44:41.811 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.811 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.811 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.811 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2024-03-28T03:44:41.812 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.812 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.812 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:41.812 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:41.812 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:41.812 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:41.812 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:41.812 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:41.812 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:41.813 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:41.813 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.813 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.813 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.813 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.813 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.813 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2024-03-28T03:44:41.813 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2024-03-28T03:44:41.813 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.814 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.814 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.814 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:41.814 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.814 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.814 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.814 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.814 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.815 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.815 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.815 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:41.815 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:41.815 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.815 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-03-28T03:44:41.815 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "nfs", 2024-03-28T03:44:41.815 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2024-03-28T03:44:41.815 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2024-03-28T03:44:41.816 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2024-03-28T03:44:41.816 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2024-03-28T03:44:41.816 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2024-03-28T03:44:41.816 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.816 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.816 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.816 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:41.816 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.816 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.816 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:41.817 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:41.817 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:41.817 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:41.817 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:41.817 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:41.817 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:41.817 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:41.817 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.818 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.818 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.818 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.818 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.818 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2024-03-28T03:44:41.818 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2024-03-28T03:44:41.818 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.818 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.818 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.819 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:41.819 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.819 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.819 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.819 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.819 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.819 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.819 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.819 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.820 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2024-03-28T03:44:41.820 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2024-03-28T03:44:41.820 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.820 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.820 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.820 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2024-03-28T03:44:41.820 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.820 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.820 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:41.821 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:41.821 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:41.821 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:41.821 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:41.821 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:41.821 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:41.821 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:41.821 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.821 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.822 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.822 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.822 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.822 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2024-03-28T03:44:41.822 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2024-03-28T03:44:41.822 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.822 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.822 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.822 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:41.823 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.823 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.823 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.823 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.823 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.823 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.823 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.823 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:41.823 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:41.823 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.824 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-03-28T03:44:41.824 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "orchestrator", 2024-03-28T03:44:41.824 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2024-03-28T03:44:41.824 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2024-03-28T03:44:41.824 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2024-03-28T03:44:41.824 INFO:teuthology.orchestra.run.smithi119.stdout: "fail_fs": { 2024-03-28T03:44:41.824 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "fail_fs", 2024-03-28T03:44:41.824 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.825 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.825 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.825 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:41.825 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.825 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.825 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.825 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-03-28T03:44:41.825 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.825 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.826 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.826 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.826 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2024-03-28T03:44:41.826 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2024-03-28T03:44:41.826 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.826 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.826 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.826 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:41.826 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.827 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.827 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:41.827 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:41.827 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:41.827 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:41.827 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:41.827 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:41.827 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:41.827 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:41.827 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.828 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.828 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.828 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.828 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.828 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2024-03-28T03:44:41.828 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2024-03-28T03:44:41.828 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.828 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.828 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.829 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:41.829 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.829 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.829 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.829 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.829 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.829 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.829 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.829 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.830 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2024-03-28T03:44:41.830 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2024-03-28T03:44:41.830 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.830 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.830 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.830 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2024-03-28T03:44:41.830 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.830 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.830 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:41.831 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:41.831 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:41.831 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:41.831 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:41.831 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:41.831 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:41.831 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:41.831 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.831 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.831 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.832 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.832 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.832 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2024-03-28T03:44:41.832 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2024-03-28T03:44:41.832 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.832 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.832 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.832 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:41.832 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.833 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.833 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.833 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.833 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.833 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.833 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.833 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.833 INFO:teuthology.orchestra.run.smithi119.stdout: "orchestrator": { 2024-03-28T03:44:41.833 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "orchestrator", 2024-03-28T03:44:41.834 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.834 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.834 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.834 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:41.834 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.834 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.834 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:41.834 INFO:teuthology.orchestra.run.smithi119.stdout: "cephadm", 2024-03-28T03:44:41.834 INFO:teuthology.orchestra.run.smithi119.stdout: "rook", 2024-03-28T03:44:41.835 INFO:teuthology.orchestra.run.smithi119.stdout: "test_orchestrator" 2024-03-28T03:44:41.835 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:41.835 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Orchestrator backend", 2024-03-28T03:44:41.835 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.835 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.835 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.835 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:41.835 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:41.835 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.836 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-03-28T03:44:41.836 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "osd_perf_query", 2024-03-28T03:44:41.836 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2024-03-28T03:44:41.836 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2024-03-28T03:44:41.836 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2024-03-28T03:44:41.836 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2024-03-28T03:44:41.836 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2024-03-28T03:44:41.836 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.836 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.837 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.837 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:41.837 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.837 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.837 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:41.837 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:41.837 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:41.837 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:41.837 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:41.838 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:41.838 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:41.838 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:41.838 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.838 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.838 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.838 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.838 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.838 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2024-03-28T03:44:41.838 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2024-03-28T03:44:41.839 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.839 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.839 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.839 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:41.839 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.839 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.839 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.839 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.839 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.840 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.840 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.840 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.840 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2024-03-28T03:44:41.840 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2024-03-28T03:44:41.840 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.840 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.840 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.840 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2024-03-28T03:44:41.841 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.841 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.841 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:41.841 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:41.841 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:41.841 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:41.841 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:41.841 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:41.841 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:41.842 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:41.842 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.842 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.842 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.842 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.842 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.842 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2024-03-28T03:44:41.843 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2024-03-28T03:44:41.843 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.843 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.843 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.843 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:41.843 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.843 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.843 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.843 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.844 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.844 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.844 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.844 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:41.844 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:41.844 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.844 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-03-28T03:44:41.844 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "osd_support", 2024-03-28T03:44:41.844 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2024-03-28T03:44:41.845 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2024-03-28T03:44:41.845 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2024-03-28T03:44:41.845 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2024-03-28T03:44:41.845 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2024-03-28T03:44:41.845 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.845 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.845 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.845 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:41.845 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.846 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.846 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:41.846 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:41.846 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:41.846 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:41.846 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:41.846 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:41.846 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:41.846 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:41.847 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.847 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.847 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.847 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.847 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.847 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2024-03-28T03:44:41.847 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2024-03-28T03:44:41.847 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.847 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.848 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.848 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:41.848 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.848 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.848 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.848 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.848 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.848 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.848 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.849 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.849 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2024-03-28T03:44:41.849 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2024-03-28T03:44:41.849 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.849 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.849 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.849 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2024-03-28T03:44:41.849 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.849 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.849 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:41.850 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:41.850 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:41.850 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:41.850 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:41.850 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:41.850 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:41.850 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:41.850 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.851 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.851 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.851 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.851 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.851 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2024-03-28T03:44:41.851 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2024-03-28T03:44:41.851 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.851 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.851 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.852 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:41.852 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.852 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.852 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.852 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.852 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.852 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.852 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.852 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:41.852 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:41.853 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.853 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-03-28T03:44:41.853 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "pg_autoscaler", 2024-03-28T03:44:41.853 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2024-03-28T03:44:41.853 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2024-03-28T03:44:41.853 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2024-03-28T03:44:41.853 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2024-03-28T03:44:41.853 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2024-03-28T03:44:41.854 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.854 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.854 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.854 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:41.854 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.854 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.854 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:41.854 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:41.854 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:41.854 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:41.855 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:41.855 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:41.855 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:41.855 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:41.855 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.855 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.855 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.855 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.856 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.856 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2024-03-28T03:44:41.856 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2024-03-28T03:44:41.856 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.856 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.856 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.856 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:41.856 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.856 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.857 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.857 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.857 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.857 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.857 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.857 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.857 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2024-03-28T03:44:41.857 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2024-03-28T03:44:41.857 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.858 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.858 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.858 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2024-03-28T03:44:41.858 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.858 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.858 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:41.858 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:41.858 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:41.858 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:41.859 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:41.859 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:41.859 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:41.859 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:41.859 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.859 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.859 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.859 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.859 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.860 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2024-03-28T03:44:41.860 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2024-03-28T03:44:41.860 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.860 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.860 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.860 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:41.860 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.860 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.860 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.861 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.861 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.861 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.861 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.861 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.861 INFO:teuthology.orchestra.run.smithi119.stdout: "sleep_interval": { 2024-03-28T03:44:41.861 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "sleep_interval", 2024-03-28T03:44:41.861 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "secs", 2024-03-28T03:44:41.861 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.862 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.862 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "60", 2024-03-28T03:44:41.862 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.862 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.862 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.862 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.862 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.862 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.862 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.862 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.863 INFO:teuthology.orchestra.run.smithi119.stdout: "threshold": { 2024-03-28T03:44:41.863 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "threshold", 2024-03-28T03:44:41.863 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "float", 2024-03-28T03:44:41.863 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.863 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.863 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "3.0", 2024-03-28T03:44:41.863 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "1.0", 2024-03-28T03:44:41.863 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.863 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.864 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "scaling threshold", 2024-03-28T03:44:41.864 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-03-28T03:44:41.864 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.864 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.864 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:41.864 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:41.864 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.864 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-03-28T03:44:41.864 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "progress", 2024-03-28T03:44:41.865 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2024-03-28T03:44:41.865 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2024-03-28T03:44:41.865 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2024-03-28T03:44:41.865 INFO:teuthology.orchestra.run.smithi119.stdout: "allow_pg_recovery_event": { 2024-03-28T03:44:41.865 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "allow_pg_recovery_event", 2024-03-28T03:44:41.865 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.865 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.865 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.866 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:41.866 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.866 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.866 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.866 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "allow the module to show pg recovery progress", 2024-03-28T03:44:41.866 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.866 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.866 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.866 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.867 INFO:teuthology.orchestra.run.smithi119.stdout: "enabled": { 2024-03-28T03:44:41.867 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "enabled", 2024-03-28T03:44:41.867 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.867 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.867 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.867 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2024-03-28T03:44:41.867 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.867 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.867 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.868 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.868 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.868 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.868 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.868 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.868 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2024-03-28T03:44:41.868 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2024-03-28T03:44:41.868 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.868 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.869 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.869 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:41.869 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.869 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.869 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:41.869 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:41.869 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:41.869 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:41.869 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:41.869 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:41.870 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:41.870 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:41.870 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.870 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.870 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.870 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.870 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.870 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2024-03-28T03:44:41.871 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2024-03-28T03:44:41.871 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.871 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.871 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.871 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:41.871 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.871 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.871 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.871 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.871 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.872 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.872 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.872 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.872 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2024-03-28T03:44:41.872 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2024-03-28T03:44:41.872 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.872 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.872 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.872 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2024-03-28T03:44:41.873 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.873 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.873 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:41.873 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:41.873 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:41.873 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:41.873 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:41.873 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:41.873 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:41.874 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:41.874 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.874 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.874 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.874 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.874 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.874 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2024-03-28T03:44:41.874 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2024-03-28T03:44:41.874 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.875 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.875 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.875 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:41.875 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.875 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.875 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.875 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.875 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.875 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.876 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.876 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.876 INFO:teuthology.orchestra.run.smithi119.stdout: "max_completed_events": { 2024-03-28T03:44:41.876 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "max_completed_events", 2024-03-28T03:44:41.876 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2024-03-28T03:44:41.876 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.876 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.876 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "50", 2024-03-28T03:44:41.876 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.877 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.877 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.877 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "number of past completed events to remember", 2024-03-28T03:44:41.877 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.877 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.877 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.877 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.877 INFO:teuthology.orchestra.run.smithi119.stdout: "sleep_interval": { 2024-03-28T03:44:41.877 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "sleep_interval", 2024-03-28T03:44:41.877 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "secs", 2024-03-28T03:44:41.878 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.878 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.878 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "5", 2024-03-28T03:44:41.878 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.878 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.878 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.878 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "how long the module is going to sleep", 2024-03-28T03:44:41.878 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.878 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.879 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.879 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:41.879 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:41.879 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.879 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-03-28T03:44:41.879 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "prometheus", 2024-03-28T03:44:41.879 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2024-03-28T03:44:41.879 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2024-03-28T03:44:41.879 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2024-03-28T03:44:41.880 INFO:teuthology.orchestra.run.smithi119.stdout: "cache": { 2024-03-28T03:44:41.880 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "cache", 2024-03-28T03:44:41.880 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.880 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.880 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.880 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2024-03-28T03:44:41.880 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.880 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.880 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.881 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.881 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.881 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.881 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.881 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.881 INFO:teuthology.orchestra.run.smithi119.stdout: "exclude_perf_counters": { 2024-03-28T03:44:41.881 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "exclude_perf_counters", 2024-03-28T03:44:41.881 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.881 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.882 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.882 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2024-03-28T03:44:41.882 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.882 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.882 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.882 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-03-28T03:44:41.882 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-03-28T03:44:41.882 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.882 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.883 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.883 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2024-03-28T03:44:41.883 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2024-03-28T03:44:41.883 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.883 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.883 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.883 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:41.883 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.883 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.884 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:41.884 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:41.884 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:41.884 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:41.884 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:41.884 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:41.884 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:41.884 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:41.884 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.884 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.885 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.885 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.885 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.885 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2024-03-28T03:44:41.885 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2024-03-28T03:44:41.885 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.885 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.885 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.885 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:41.886 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.886 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.886 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.886 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.886 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.886 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.886 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.886 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.886 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2024-03-28T03:44:41.887 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2024-03-28T03:44:41.887 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.887 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.887 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.887 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2024-03-28T03:44:41.887 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.887 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.887 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:41.887 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:41.888 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:41.888 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:41.888 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:41.888 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:41.888 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:41.888 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:41.888 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.888 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.888 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.889 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.889 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.889 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2024-03-28T03:44:41.889 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2024-03-28T03:44:41.889 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.889 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.889 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.889 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:41.889 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.890 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.890 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.890 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.890 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.890 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.890 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.890 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.890 INFO:teuthology.orchestra.run.smithi119.stdout: "rbd_stats_pools": { 2024-03-28T03:44:41.890 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "rbd_stats_pools", 2024-03-28T03:44:41.891 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.891 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.891 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.891 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:41.891 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.891 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.891 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.891 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.891 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.892 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.892 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.892 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.892 INFO:teuthology.orchestra.run.smithi119.stdout: "rbd_stats_pools_refresh_interval": { 2024-03-28T03:44:41.892 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-03-28T03:44:41.892 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2024-03-28T03:44:41.892 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.892 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.892 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "300", 2024-03-28T03:44:41.893 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.893 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.893 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.893 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.893 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.893 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.893 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.893 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.893 INFO:teuthology.orchestra.run.smithi119.stdout: "scrape_interval": { 2024-03-28T03:44:41.894 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "scrape_interval", 2024-03-28T03:44:41.894 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "float", 2024-03-28T03:44:41.894 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.894 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.894 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "15.0", 2024-03-28T03:44:41.894 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.894 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.894 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.894 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.894 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.895 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.895 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.895 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.895 INFO:teuthology.orchestra.run.smithi119.stdout: "server_addr": { 2024-03-28T03:44:41.895 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "server_addr", 2024-03-28T03:44:41.895 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.895 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.895 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.895 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "::", 2024-03-28T03:44:41.896 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.896 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.896 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.896 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-03-28T03:44:41.896 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.896 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.896 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.896 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.896 INFO:teuthology.orchestra.run.smithi119.stdout: "server_port": { 2024-03-28T03:44:41.896 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "server_port", 2024-03-28T03:44:41.897 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2024-03-28T03:44:41.897 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.897 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.897 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "9283", 2024-03-28T03:44:41.897 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.897 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.897 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.897 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-03-28T03:44:41.897 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.898 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.898 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.898 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.898 INFO:teuthology.orchestra.run.smithi119.stdout: "stale_cache_strategy": { 2024-03-28T03:44:41.898 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "stale_cache_strategy", 2024-03-28T03:44:41.898 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.898 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.898 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.898 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "log", 2024-03-28T03:44:41.899 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.899 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.899 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.899 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.899 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.899 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.899 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.899 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.899 INFO:teuthology.orchestra.run.smithi119.stdout: "standby_behaviour": { 2024-03-28T03:44:41.900 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "standby_behaviour", 2024-03-28T03:44:41.900 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.900 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.900 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.900 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "default", 2024-03-28T03:44:41.900 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.900 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.900 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:41.900 INFO:teuthology.orchestra.run.smithi119.stdout: "default", 2024-03-28T03:44:41.900 INFO:teuthology.orchestra.run.smithi119.stdout: "error" 2024-03-28T03:44:41.901 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:41.901 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.901 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.901 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.901 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.901 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.901 INFO:teuthology.orchestra.run.smithi119.stdout: "standby_error_status_code": { 2024-03-28T03:44:41.901 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "standby_error_status_code", 2024-03-28T03:44:41.901 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2024-03-28T03:44:41.902 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.902 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.902 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "500", 2024-03-28T03:44:41.902 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "400", 2024-03-28T03:44:41.902 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "599", 2024-03-28T03:44:41.902 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.902 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.902 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.902 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.903 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.903 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:41.903 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:41.903 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.903 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-03-28T03:44:41.903 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "rbd_support", 2024-03-28T03:44:41.903 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2024-03-28T03:44:41.903 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2024-03-28T03:44:41.903 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2024-03-28T03:44:41.903 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2024-03-28T03:44:41.904 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2024-03-28T03:44:41.904 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.904 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.904 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.904 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:41.904 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.904 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.904 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:41.904 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:41.905 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:41.905 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:41.905 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:41.905 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:41.905 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:41.905 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:41.905 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.905 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.905 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.905 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.906 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.906 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2024-03-28T03:44:41.906 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2024-03-28T03:44:41.906 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.906 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.906 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.906 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:41.906 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.906 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.907 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.907 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.907 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.907 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.907 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.907 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.907 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2024-03-28T03:44:41.907 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2024-03-28T03:44:41.908 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.908 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.908 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.908 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2024-03-28T03:44:41.908 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.908 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.908 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:41.908 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:41.908 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:41.908 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:41.909 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:41.909 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:41.909 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:41.909 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:41.909 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.909 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.909 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.909 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.909 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.910 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2024-03-28T03:44:41.910 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2024-03-28T03:44:41.910 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.910 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.910 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.910 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:41.910 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.910 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.910 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.911 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.911 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.911 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.911 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.911 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.911 INFO:teuthology.orchestra.run.smithi119.stdout: "max_concurrent_snap_create": { 2024-03-28T03:44:41.911 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "max_concurrent_snap_create", 2024-03-28T03:44:41.911 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2024-03-28T03:44:41.911 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.912 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.912 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "10", 2024-03-28T03:44:41.912 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.912 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.912 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.912 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.912 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.912 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.912 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.913 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.913 INFO:teuthology.orchestra.run.smithi119.stdout: "mirror_snapshot_schedule": { 2024-03-28T03:44:41.913 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "mirror_snapshot_schedule", 2024-03-28T03:44:41.913 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.913 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.913 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.913 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:41.913 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.913 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.914 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.914 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.914 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.914 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.914 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.914 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.914 INFO:teuthology.orchestra.run.smithi119.stdout: "trash_purge_schedule": { 2024-03-28T03:44:41.914 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "trash_purge_schedule", 2024-03-28T03:44:41.914 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.915 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.915 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.915 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:41.915 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.915 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.915 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.915 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.915 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.915 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.916 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.916 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:41.916 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:41.916 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.916 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-03-28T03:44:41.916 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "restful", 2024-03-28T03:44:41.916 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2024-03-28T03:44:41.916 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2024-03-28T03:44:41.916 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2024-03-28T03:44:41.917 INFO:teuthology.orchestra.run.smithi119.stdout: "enable_auth": { 2024-03-28T03:44:41.917 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "enable_auth", 2024-03-28T03:44:41.917 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.917 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.917 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.917 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2024-03-28T03:44:41.917 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.917 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.917 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.918 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.918 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.918 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.918 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.918 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.918 INFO:teuthology.orchestra.run.smithi119.stdout: "key_file": { 2024-03-28T03:44:41.918 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "key_file", 2024-03-28T03:44:41.918 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.919 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.919 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.919 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:41.919 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.919 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.919 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.919 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.919 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.919 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.920 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.920 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.920 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2024-03-28T03:44:41.920 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2024-03-28T03:44:41.920 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.920 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.920 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.920 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:41.920 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.921 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.921 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:41.921 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:41.921 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:41.921 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:41.921 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:41.921 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:41.921 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:41.921 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:41.922 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.922 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.922 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.922 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.922 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.922 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2024-03-28T03:44:41.922 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2024-03-28T03:44:41.922 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.922 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.923 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.923 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:41.923 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.923 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.923 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.923 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.923 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.923 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.923 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.923 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.924 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2024-03-28T03:44:41.924 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2024-03-28T03:44:41.924 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.924 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.924 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.924 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2024-03-28T03:44:41.924 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.924 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.924 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:41.925 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:41.925 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:41.925 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:41.925 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:41.925 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:41.925 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:41.925 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:41.925 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.925 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.926 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.926 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.926 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.926 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2024-03-28T03:44:41.926 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2024-03-28T03:44:41.926 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.926 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.926 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.926 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:41.927 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.927 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.927 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.927 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.927 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.927 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.927 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.927 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.927 INFO:teuthology.orchestra.run.smithi119.stdout: "server_addr": { 2024-03-28T03:44:41.928 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "server_addr", 2024-03-28T03:44:41.928 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.928 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.928 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.928 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:41.928 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.928 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.928 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.928 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.929 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.929 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.929 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.929 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.929 INFO:teuthology.orchestra.run.smithi119.stdout: "server_port": { 2024-03-28T03:44:41.929 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "server_port", 2024-03-28T03:44:41.929 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.929 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.929 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.929 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:41.930 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.930 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.930 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.930 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.930 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.930 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.930 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.930 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:41.931 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:41.931 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.931 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-03-28T03:44:41.931 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "rgw", 2024-03-28T03:44:41.931 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2024-03-28T03:44:41.931 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2024-03-28T03:44:41.931 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2024-03-28T03:44:41.931 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2024-03-28T03:44:41.931 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2024-03-28T03:44:41.931 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.932 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.932 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.932 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:41.932 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.932 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.932 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:41.932 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:41.932 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:41.932 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:41.932 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:41.933 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:41.933 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:41.933 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:41.933 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.933 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.933 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.933 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.933 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.933 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2024-03-28T03:44:41.934 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2024-03-28T03:44:41.934 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.934 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.934 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.934 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:41.934 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.934 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.934 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.934 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.935 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.935 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.935 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.935 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.935 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2024-03-28T03:44:41.935 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2024-03-28T03:44:41.935 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.935 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.936 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.936 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2024-03-28T03:44:41.936 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.936 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.936 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:41.936 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:41.936 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:41.936 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:41.936 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:41.936 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:41.937 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:41.937 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:41.937 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.937 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.937 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.937 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.937 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.937 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2024-03-28T03:44:41.937 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2024-03-28T03:44:41.938 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.938 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.938 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.938 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:41.938 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.938 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.938 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.938 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.938 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.938 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.939 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.939 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:41.939 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:41.939 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.939 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-03-28T03:44:41.939 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "rook", 2024-03-28T03:44:41.939 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2024-03-28T03:44:41.939 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2024-03-28T03:44:41.939 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2024-03-28T03:44:41.940 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2024-03-28T03:44:41.940 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2024-03-28T03:44:41.940 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.940 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.940 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.940 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:41.940 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.940 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.940 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:41.941 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:41.941 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:41.941 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:41.941 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:41.941 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:41.941 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:41.941 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:41.941 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.941 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.942 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.942 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.942 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.942 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2024-03-28T03:44:41.942 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2024-03-28T03:44:41.942 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.942 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.942 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.942 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:41.943 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.943 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.943 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.943 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.943 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.943 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.943 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.943 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.943 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2024-03-28T03:44:41.944 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2024-03-28T03:44:41.944 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.944 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.944 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.944 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2024-03-28T03:44:41.944 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.944 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.944 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:41.944 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:41.944 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:41.945 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:41.945 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:41.945 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:41.945 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:41.945 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:41.945 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.945 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.945 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.945 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.946 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.946 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2024-03-28T03:44:41.946 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2024-03-28T03:44:41.946 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.946 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.946 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.946 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:41.946 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.947 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.947 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.947 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.947 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.947 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.947 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.947 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.947 INFO:teuthology.orchestra.run.smithi119.stdout: "storage_class": { 2024-03-28T03:44:41.947 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "storage_class", 2024-03-28T03:44:41.948 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.948 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.948 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.948 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "local", 2024-03-28T03:44:41.948 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.948 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.948 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.948 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-03-28T03:44:41.948 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.948 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.949 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.949 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:41.949 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:41.949 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.949 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-03-28T03:44:41.949 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "selftest", 2024-03-28T03:44:41.949 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2024-03-28T03:44:41.949 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2024-03-28T03:44:41.949 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2024-03-28T03:44:41.950 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2024-03-28T03:44:41.950 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2024-03-28T03:44:41.950 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.950 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.950 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.950 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:41.950 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.950 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.950 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:41.950 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:41.951 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:41.951 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:41.951 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:41.951 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:41.951 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:41.951 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:41.951 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.951 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.951 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.952 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.952 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.952 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2024-03-28T03:44:41.952 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2024-03-28T03:44:41.952 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.952 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.952 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.952 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:41.952 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.953 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.953 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.953 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.953 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.953 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.953 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.953 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.953 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2024-03-28T03:44:41.954 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2024-03-28T03:44:41.954 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.954 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.954 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.954 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2024-03-28T03:44:41.954 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.954 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.954 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:41.955 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:41.955 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:41.955 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:41.955 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:41.955 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:41.955 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:41.955 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:41.955 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.955 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.955 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.956 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.956 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.956 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2024-03-28T03:44:41.956 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2024-03-28T03:44:41.956 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.956 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.956 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.956 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:41.956 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.957 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.957 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.957 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.957 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.957 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.957 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.957 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.957 INFO:teuthology.orchestra.run.smithi119.stdout: "roption1": { 2024-03-28T03:44:41.957 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "roption1", 2024-03-28T03:44:41.958 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.958 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.958 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.958 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:41.958 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.958 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.958 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.958 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.958 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.959 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.959 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.959 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.959 INFO:teuthology.orchestra.run.smithi119.stdout: "roption2": { 2024-03-28T03:44:41.959 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "roption2", 2024-03-28T03:44:41.959 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.959 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.959 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.959 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "xyz", 2024-03-28T03:44:41.960 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.960 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.960 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.960 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.960 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.960 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.960 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.960 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.960 INFO:teuthology.orchestra.run.smithi119.stdout: "rwoption1": { 2024-03-28T03:44:41.961 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "rwoption1", 2024-03-28T03:44:41.961 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.961 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.961 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.961 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:41.961 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.961 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.961 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.961 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.962 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.962 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.962 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.962 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.962 INFO:teuthology.orchestra.run.smithi119.stdout: "rwoption2": { 2024-03-28T03:44:41.962 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "rwoption2", 2024-03-28T03:44:41.962 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2024-03-28T03:44:41.962 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.962 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.963 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:41.963 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.963 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.963 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.963 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.963 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.963 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.963 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.963 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.963 INFO:teuthology.orchestra.run.smithi119.stdout: "rwoption3": { 2024-03-28T03:44:41.964 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "rwoption3", 2024-03-28T03:44:41.964 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "float", 2024-03-28T03:44:41.964 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.964 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.964 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:41.964 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.964 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.964 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.964 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.965 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.965 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.965 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.965 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.965 INFO:teuthology.orchestra.run.smithi119.stdout: "rwoption4": { 2024-03-28T03:44:41.965 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "rwoption4", 2024-03-28T03:44:41.965 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.965 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.966 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.966 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:41.966 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.966 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.966 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.966 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.966 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.966 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.967 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.967 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.967 INFO:teuthology.orchestra.run.smithi119.stdout: "rwoption5": { 2024-03-28T03:44:41.967 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "rwoption5", 2024-03-28T03:44:41.967 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.967 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.967 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.967 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:41.967 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.968 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.968 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.968 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.968 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.968 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.968 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.968 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.968 INFO:teuthology.orchestra.run.smithi119.stdout: "rwoption6": { 2024-03-28T03:44:41.968 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "rwoption6", 2024-03-28T03:44:41.969 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.969 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.969 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.969 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2024-03-28T03:44:41.969 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.969 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.969 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.969 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.969 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.969 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.970 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.970 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.970 INFO:teuthology.orchestra.run.smithi119.stdout: "rwoption7": { 2024-03-28T03:44:41.970 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "rwoption7", 2024-03-28T03:44:41.970 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2024-03-28T03:44:41.970 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.970 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.970 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:41.970 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "1", 2024-03-28T03:44:41.971 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "42", 2024-03-28T03:44:41.971 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.971 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.971 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.971 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.971 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.971 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.971 INFO:teuthology.orchestra.run.smithi119.stdout: "testkey": { 2024-03-28T03:44:41.971 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "testkey", 2024-03-28T03:44:41.972 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.972 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.972 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.972 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:41.972 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.972 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.972 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.972 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.972 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.973 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.973 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.973 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.973 INFO:teuthology.orchestra.run.smithi119.stdout: "testlkey": { 2024-03-28T03:44:41.973 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "testlkey", 2024-03-28T03:44:41.973 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.973 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.973 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.973 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:41.974 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.974 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.974 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.974 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.974 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.974 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.974 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.974 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.974 INFO:teuthology.orchestra.run.smithi119.stdout: "testnewline": { 2024-03-28T03:44:41.974 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "testnewline", 2024-03-28T03:44:41.975 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.975 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.975 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:41.975 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:41.975 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.975 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.975 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.975 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.975 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.976 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.976 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.976 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:41.976 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:41.976 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.976 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-03-28T03:44:41.976 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "snap_schedule", 2024-03-28T03:44:41.976 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2024-03-28T03:44:41.976 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2024-03-28T03:44:41.976 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2024-03-28T03:44:41.977 INFO:teuthology.orchestra.run.smithi119.stdout: "allow_m_granularity": { 2024-03-28T03:44:41.977 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "allow_m_granularity", 2024-03-28T03:44:41.977 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.977 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.977 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.977 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:41.977 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.977 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.977 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.978 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "allow minute scheduled snapshots", 2024-03-28T03:44:41.978 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.978 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.978 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.978 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.978 INFO:teuthology.orchestra.run.smithi119.stdout: "dump_on_update": { 2024-03-28T03:44:41.978 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "dump_on_update", 2024-03-28T03:44:41.978 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.978 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.979 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.979 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:41.979 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.979 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.979 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.979 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "dump database to debug log on update", 2024-03-28T03:44:41.979 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.979 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.979 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.979 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.980 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2024-03-28T03:44:41.980 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2024-03-28T03:44:41.980 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.980 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.980 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.980 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:41.980 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.980 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.980 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:41.981 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:41.981 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:41.981 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:41.981 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:41.981 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:41.981 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:41.981 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:41.981 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.981 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.981 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.982 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.982 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.982 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2024-03-28T03:44:41.982 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2024-03-28T03:44:41.982 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.982 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.982 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.982 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:41.982 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.983 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.983 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.983 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.983 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.983 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.983 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.983 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.983 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2024-03-28T03:44:41.983 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2024-03-28T03:44:41.984 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.984 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.984 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.984 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2024-03-28T03:44:41.984 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.984 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.984 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:41.984 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:41.984 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:41.984 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:41.985 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:41.985 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:41.985 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:41.985 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:41.985 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.985 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.985 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.985 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.985 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.986 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2024-03-28T03:44:41.986 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2024-03-28T03:44:41.986 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.986 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.986 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.986 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:41.986 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.986 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.986 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.987 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.987 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.987 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.987 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.987 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:41.987 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:41.987 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.987 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-03-28T03:44:41.987 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "stats", 2024-03-28T03:44:41.988 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2024-03-28T03:44:41.988 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2024-03-28T03:44:41.988 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2024-03-28T03:44:41.988 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2024-03-28T03:44:41.988 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2024-03-28T03:44:41.988 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.988 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.988 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.988 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:41.989 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.989 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.989 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:41.989 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:41.989 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:41.989 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:41.989 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:41.989 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:41.989 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:41.990 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:41.990 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.990 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.990 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.990 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.990 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.990 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2024-03-28T03:44:41.990 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2024-03-28T03:44:41.990 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.991 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.991 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.991 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:41.991 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.991 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.991 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.991 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.991 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.991 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.992 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.992 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.992 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2024-03-28T03:44:41.992 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2024-03-28T03:44:41.992 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.992 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.992 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.992 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2024-03-28T03:44:41.992 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.992 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.993 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:41.993 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:41.993 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:41.993 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:41.993 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:41.993 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:41.993 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:41.993 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:41.993 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.994 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.994 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.994 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.994 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.994 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2024-03-28T03:44:41.994 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2024-03-28T03:44:41.994 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.994 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.994 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.995 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:41.995 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.995 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.995 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.995 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.995 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.995 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.995 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.995 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:41.995 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:41.996 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.996 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-03-28T03:44:41.996 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "status", 2024-03-28T03:44:41.996 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2024-03-28T03:44:41.996 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2024-03-28T03:44:41.996 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2024-03-28T03:44:41.996 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2024-03-28T03:44:41.996 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2024-03-28T03:44:41.996 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:41.997 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.997 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.997 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:41.997 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.997 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.997 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:41.997 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:41.997 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:41.997 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:41.997 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:41.998 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:41.998 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:41.998 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:41.998 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:41.998 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:41.998 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:41.998 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:41.998 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:41.998 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2024-03-28T03:44:41.999 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2024-03-28T03:44:41.999 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:41.999 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:41.999 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:41.999 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:41.999 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:41.999 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:41.999 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:41.999 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:42.000 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:42.000 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:42.000 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:42.000 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:42.000 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2024-03-28T03:44:42.000 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2024-03-28T03:44:42.000 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:42.000 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:42.000 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:42.001 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2024-03-28T03:44:42.001 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:42.001 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:42.001 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:42.001 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:42.001 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:42.001 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:42.001 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:42.001 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:42.001 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:42.002 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:42.002 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:42.002 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:42.002 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:42.002 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:42.002 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:42.002 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2024-03-28T03:44:42.002 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2024-03-28T03:44:42.002 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:42.003 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:42.003 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:42.003 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:42.003 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:42.003 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:42.003 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:42.003 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:42.003 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:42.003 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:42.004 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:42.004 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:42.004 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:42.004 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:42.004 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-03-28T03:44:42.004 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "telegraf", 2024-03-28T03:44:42.004 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2024-03-28T03:44:42.004 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2024-03-28T03:44:42.004 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2024-03-28T03:44:42.004 INFO:teuthology.orchestra.run.smithi119.stdout: "address": { 2024-03-28T03:44:42.005 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "address", 2024-03-28T03:44:42.005 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:42.005 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:42.005 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:42.005 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-03-28T03:44:42.005 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:42.005 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:42.005 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:42.005 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:42.006 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:42.006 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:42.006 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:42.006 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:42.006 INFO:teuthology.orchestra.run.smithi119.stdout: "interval": { 2024-03-28T03:44:42.006 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "interval", 2024-03-28T03:44:42.006 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "secs", 2024-03-28T03:44:42.006 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:42.006 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:42.007 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "15", 2024-03-28T03:44:42.007 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:42.007 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:42.007 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:42.007 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:42.007 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:42.007 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:42.007 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:42.007 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:42.007 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2024-03-28T03:44:42.008 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2024-03-28T03:44:42.008 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:42.008 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:42.008 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:42.008 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:42.008 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:42.008 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:42.008 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:42.008 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:42.009 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:42.009 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:42.009 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:42.009 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:42.009 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:42.009 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:42.009 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:42.009 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:42.009 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:42.009 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:42.010 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:42.010 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2024-03-28T03:44:42.010 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2024-03-28T03:44:42.010 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:42.010 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:42.010 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:42.010 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:42.010 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:42.010 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:42.011 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:42.011 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:42.011 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:42.011 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:42.011 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:42.011 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:42.011 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2024-03-28T03:44:42.011 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2024-03-28T03:44:42.011 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:42.012 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:42.012 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:42.012 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2024-03-28T03:44:42.012 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:42.012 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:42.012 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:42.012 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:42.012 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:42.012 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:42.013 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:42.013 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:42.013 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:42.013 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:42.013 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:42.013 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:42.013 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:42.013 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:42.013 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:42.013 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2024-03-28T03:44:42.014 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2024-03-28T03:44:42.014 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:42.014 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:42.014 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:42.014 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:42.014 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:42.014 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:42.014 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:42.015 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:42.015 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:42.015 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:42.015 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:42.015 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:42.015 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:42.015 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:42.015 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-03-28T03:44:42.015 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "telemetry", 2024-03-28T03:44:42.016 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2024-03-28T03:44:42.016 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2024-03-28T03:44:42.016 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2024-03-28T03:44:42.016 INFO:teuthology.orchestra.run.smithi119.stdout: "channel_basic": { 2024-03-28T03:44:42.016 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "channel_basic", 2024-03-28T03:44:42.016 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:42.016 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:42.016 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:42.016 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2024-03-28T03:44:42.016 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:42.017 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:42.017 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:42.017 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Share basic cluster information (size, version)", 2024-03-28T03:44:42.017 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:42.017 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:42.017 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:42.017 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:42.017 INFO:teuthology.orchestra.run.smithi119.stdout: "channel_crash": { 2024-03-28T03:44:42.017 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "channel_crash", 2024-03-28T03:44:42.018 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:42.018 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:42.018 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:42.018 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2024-03-28T03:44:42.018 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:42.018 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:42.018 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:42.018 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-03-28T03:44:42.019 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:42.019 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:42.019 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:42.019 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:42.019 INFO:teuthology.orchestra.run.smithi119.stdout: "channel_device": { 2024-03-28T03:44:42.019 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "channel_device", 2024-03-28T03:44:42.019 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:42.019 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:42.019 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:42.020 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2024-03-28T03:44:42.020 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:42.020 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:42.020 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:42.020 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-03-28T03:44:42.020 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:42.020 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:42.020 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:42.020 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:42.021 INFO:teuthology.orchestra.run.smithi119.stdout: "channel_ident": { 2024-03-28T03:44:42.021 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "channel_ident", 2024-03-28T03:44:42.021 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:42.021 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:42.021 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:42.021 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:42.021 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:42.021 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:42.021 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:42.022 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-03-28T03:44:42.022 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:42.022 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:42.022 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:42.022 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:42.022 INFO:teuthology.orchestra.run.smithi119.stdout: "channel_perf": { 2024-03-28T03:44:42.022 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "channel_perf", 2024-03-28T03:44:42.022 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:42.022 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:42.023 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:42.023 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:42.023 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:42.023 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:42.023 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:42.023 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Share various performance metrics of a cluster", 2024-03-28T03:44:42.023 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:42.023 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:42.023 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:42.023 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:42.024 INFO:teuthology.orchestra.run.smithi119.stdout: "contact": { 2024-03-28T03:44:42.024 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "contact", 2024-03-28T03:44:42.024 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:42.024 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:42.024 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:42.024 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:42.024 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:42.024 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:42.024 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:42.025 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:42.025 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:42.025 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:42.025 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:42.025 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:42.025 INFO:teuthology.orchestra.run.smithi119.stdout: "description": { 2024-03-28T03:44:42.025 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "description", 2024-03-28T03:44:42.025 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:42.025 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:42.026 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:42.026 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:42.026 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:42.026 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:42.026 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:42.026 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:42.026 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:42.026 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:42.027 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:42.027 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:42.027 INFO:teuthology.orchestra.run.smithi119.stdout: "device_url": { 2024-03-28T03:44:42.027 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "device_url", 2024-03-28T03:44:42.027 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:42.027 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:42.027 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:42.027 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-03-28T03:44:42.027 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:42.028 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:42.028 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:42.028 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:42.028 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:42.028 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:42.028 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:42.028 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:42.028 INFO:teuthology.orchestra.run.smithi119.stdout: "enabled": { 2024-03-28T03:44:42.028 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "enabled", 2024-03-28T03:44:42.029 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:42.029 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:42.029 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:42.029 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:42.029 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:42.029 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:42.029 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:42.029 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:42.029 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:42.030 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:42.030 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:42.030 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:42.030 INFO:teuthology.orchestra.run.smithi119.stdout: "interval": { 2024-03-28T03:44:42.030 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "interval", 2024-03-28T03:44:42.030 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2024-03-28T03:44:42.030 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:42.030 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:42.030 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "24", 2024-03-28T03:44:42.031 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "8", 2024-03-28T03:44:42.031 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:42.031 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:42.031 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:42.031 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:42.031 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:42.031 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:42.031 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:42.031 INFO:teuthology.orchestra.run.smithi119.stdout: "last_opt_revision": { 2024-03-28T03:44:42.031 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "last_opt_revision", 2024-03-28T03:44:42.032 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2024-03-28T03:44:42.032 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:42.032 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:42.032 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "1", 2024-03-28T03:44:42.032 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:42.032 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:42.032 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:42.032 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:42.032 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:42.033 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:42.033 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:42.033 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:42.033 INFO:teuthology.orchestra.run.smithi119.stdout: "leaderboard": { 2024-03-28T03:44:42.033 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "leaderboard", 2024-03-28T03:44:42.033 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:42.033 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:42.033 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:42.033 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:42.034 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:42.034 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:42.034 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:42.034 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:42.034 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:42.034 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:42.034 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:42.034 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:42.034 INFO:teuthology.orchestra.run.smithi119.stdout: "leaderboard_description": { 2024-03-28T03:44:42.035 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "leaderboard_description", 2024-03-28T03:44:42.035 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:42.035 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:42.035 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:42.035 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:42.035 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:42.035 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:42.035 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:42.035 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:42.036 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:42.036 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:42.036 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:42.036 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:42.036 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2024-03-28T03:44:42.036 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2024-03-28T03:44:42.036 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:42.036 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:42.036 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:42.036 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:42.037 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:42.037 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:42.037 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:42.037 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:42.037 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:42.037 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:42.037 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:42.037 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:42.037 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:42.038 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:42.038 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:42.038 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:42.038 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:42.038 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:42.038 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:42.038 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2024-03-28T03:44:42.038 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2024-03-28T03:44:42.038 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:42.039 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:42.039 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:42.039 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:42.039 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:42.039 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:42.039 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:42.039 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:42.039 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:42.039 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:42.039 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:42.040 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:42.040 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2024-03-28T03:44:42.040 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2024-03-28T03:44:42.040 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:42.040 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:42.040 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:42.040 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2024-03-28T03:44:42.040 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:42.040 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:42.041 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:42.041 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:42.041 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:42.041 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:42.041 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:42.041 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:42.041 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:42.041 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:42.041 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:42.042 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:42.042 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:42.042 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:42.042 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:42.042 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2024-03-28T03:44:42.042 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2024-03-28T03:44:42.042 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:42.042 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:42.042 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:42.043 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:42.043 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:42.043 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:42.043 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:42.043 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:42.043 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:42.043 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:42.043 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:42.043 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:42.044 INFO:teuthology.orchestra.run.smithi119.stdout: "organization": { 2024-03-28T03:44:42.044 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "organization", 2024-03-28T03:44:42.044 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:42.044 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:42.044 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:42.044 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:42.044 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:42.044 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:42.044 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:42.045 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:42.045 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:42.045 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:42.045 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:42.045 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:42.045 INFO:teuthology.orchestra.run.smithi119.stdout: "proxy": { 2024-03-28T03:44:42.045 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "proxy", 2024-03-28T03:44:42.045 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:42.045 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:42.045 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:42.046 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:42.046 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:42.046 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:42.046 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:42.046 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:42.046 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:42.046 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:42.046 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:42.047 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:42.047 INFO:teuthology.orchestra.run.smithi119.stdout: "url": { 2024-03-28T03:44:42.047 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "url", 2024-03-28T03:44:42.047 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:42.047 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:42.047 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:42.047 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-03-28T03:44:42.047 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:42.047 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:42.048 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:42.048 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:42.048 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:42.048 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:42.048 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:42.048 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:42.048 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:42.048 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:42.048 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-03-28T03:44:42.049 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "test_orchestrator", 2024-03-28T03:44:42.049 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2024-03-28T03:44:42.049 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2024-03-28T03:44:42.049 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2024-03-28T03:44:42.049 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2024-03-28T03:44:42.049 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2024-03-28T03:44:42.049 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:42.049 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:42.049 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:42.050 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:42.050 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:42.050 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:42.050 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:42.050 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:42.050 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:42.050 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:42.050 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:42.050 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:42.051 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:42.051 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:42.051 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:42.051 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:42.051 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:42.051 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:42.051 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:42.051 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2024-03-28T03:44:42.051 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2024-03-28T03:44:42.051 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:42.052 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:42.052 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:42.052 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:42.052 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:42.052 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:42.052 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:42.052 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:42.052 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:42.052 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:42.053 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:42.053 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:42.053 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2024-03-28T03:44:42.053 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2024-03-28T03:44:42.053 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:42.053 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:42.053 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:42.053 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2024-03-28T03:44:42.053 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:42.054 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:42.054 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:42.054 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:42.054 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:42.054 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:42.054 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:42.054 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:42.054 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:42.054 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:42.055 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:42.055 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:42.055 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:42.055 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:42.055 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:42.055 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2024-03-28T03:44:42.055 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2024-03-28T03:44:42.055 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:42.055 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:42.056 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:42.056 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:42.056 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:42.056 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:42.056 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:42.056 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:42.056 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:42.056 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:42.056 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:42.056 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:42.057 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:42.057 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:42.057 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-03-28T03:44:42.057 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "volumes", 2024-03-28T03:44:42.057 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2024-03-28T03:44:42.057 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2024-03-28T03:44:42.057 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2024-03-28T03:44:42.057 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2024-03-28T03:44:42.057 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2024-03-28T03:44:42.058 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:42.058 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:42.058 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:42.058 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:42.058 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:42.058 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:42.058 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:42.058 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:42.058 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:42.059 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:42.059 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:42.059 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:42.059 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:42.059 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:42.059 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:42.059 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:42.059 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:42.059 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:42.059 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:42.060 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2024-03-28T03:44:42.060 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2024-03-28T03:44:42.060 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:42.060 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:42.060 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:42.060 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:42.060 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:42.060 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:42.060 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:42.061 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:42.061 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:42.061 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:42.061 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:42.061 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:42.061 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2024-03-28T03:44:42.061 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2024-03-28T03:44:42.061 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:42.061 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:42.061 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:42.062 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2024-03-28T03:44:42.062 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:42.062 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:42.062 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:42.062 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:42.062 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:42.062 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:42.062 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:42.062 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:42.063 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:42.063 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:42.063 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:42.063 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:42.063 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:42.063 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:42.063 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:42.063 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2024-03-28T03:44:42.063 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2024-03-28T03:44:42.064 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:42.064 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:42.064 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:42.064 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:42.064 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:42.064 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:42.064 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:42.064 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:42.064 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:42.065 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:42.065 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:42.065 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:42.065 INFO:teuthology.orchestra.run.smithi119.stdout: "max_concurrent_clones": { 2024-03-28T03:44:42.065 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "max_concurrent_clones", 2024-03-28T03:44:42.065 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2024-03-28T03:44:42.065 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:42.065 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:42.065 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "4", 2024-03-28T03:44:42.066 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:42.066 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:42.066 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:42.066 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Number of asynchronous cloner threads", 2024-03-28T03:44:42.066 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:42.066 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:42.066 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:42.066 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:42.066 INFO:teuthology.orchestra.run.smithi119.stdout: "periodic_async_work": { 2024-03-28T03:44:42.067 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "periodic_async_work", 2024-03-28T03:44:42.067 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:42.067 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:42.067 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:42.067 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:42.067 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:42.067 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:42.067 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:42.067 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Periodically check for async work", 2024-03-28T03:44:42.068 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:42.068 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:42.068 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:42.068 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:42.068 INFO:teuthology.orchestra.run.smithi119.stdout: "snapshot_clone_delay": { 2024-03-28T03:44:42.068 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "snapshot_clone_delay", 2024-03-28T03:44:42.068 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2024-03-28T03:44:42.068 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:42.068 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:42.069 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "0", 2024-03-28T03:44:42.069 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:42.069 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:42.069 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:42.069 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-03-28T03:44:42.069 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:42.069 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:42.069 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:42.069 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:42.070 INFO:teuthology.orchestra.run.smithi119.stdout: "snapshot_clone_no_wait": { 2024-03-28T03:44:42.070 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "snapshot_clone_no_wait", 2024-03-28T03:44:42.070 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:42.070 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:42.070 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:42.070 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2024-03-28T03:44:42.070 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:42.070 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:42.070 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:42.071 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-03-28T03:44:42.071 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:42.071 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:42.071 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:42.071 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:42.071 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:42.071 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:42.071 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-03-28T03:44:42.071 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "zabbix", 2024-03-28T03:44:42.071 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2024-03-28T03:44:42.072 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2024-03-28T03:44:42.072 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2024-03-28T03:44:42.072 INFO:teuthology.orchestra.run.smithi119.stdout: "discovery_interval": { 2024-03-28T03:44:42.072 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "discovery_interval", 2024-03-28T03:44:42.072 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "uint", 2024-03-28T03:44:42.072 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:42.072 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:42.072 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "100", 2024-03-28T03:44:42.072 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:42.073 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:42.073 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:42.073 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:42.073 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:42.073 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:42.073 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:42.073 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:42.073 INFO:teuthology.orchestra.run.smithi119.stdout: "identifier": { 2024-03-28T03:44:42.073 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "identifier", 2024-03-28T03:44:42.073 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:42.074 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:42.074 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:42.074 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:42.074 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:42.074 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:42.074 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:42.074 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:42.074 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:42.074 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:42.075 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:42.075 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:42.075 INFO:teuthology.orchestra.run.smithi119.stdout: "interval": { 2024-03-28T03:44:42.075 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "interval", 2024-03-28T03:44:42.075 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "secs", 2024-03-28T03:44:42.075 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:42.075 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:42.075 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "60", 2024-03-28T03:44:42.075 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:42.075 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:42.076 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:42.076 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:42.076 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:42.076 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:42.076 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:42.076 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:42.076 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2024-03-28T03:44:42.076 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2024-03-28T03:44:42.076 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:42.077 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:42.077 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:42.077 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:42.077 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:42.077 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:42.077 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:42.077 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:42.077 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:42.077 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:42.078 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:42.078 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:42.078 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:42.078 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:42.078 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:42.078 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:42.078 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:42.078 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:42.078 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:42.079 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2024-03-28T03:44:42.079 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2024-03-28T03:44:42.079 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:42.079 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:42.079 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:42.079 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:42.079 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:42.079 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:42.079 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:42.079 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:42.080 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:42.080 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:42.080 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:42.080 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:42.080 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2024-03-28T03:44:42.080 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2024-03-28T03:44:42.080 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:42.080 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:42.080 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:42.081 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2024-03-28T03:44:42.081 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:42.081 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:42.081 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2024-03-28T03:44:42.081 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2024-03-28T03:44:42.081 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2024-03-28T03:44:42.081 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2024-03-28T03:44:42.081 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2024-03-28T03:44:42.081 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2024-03-28T03:44:42.082 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2024-03-28T03:44:42.082 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:42.082 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:42.082 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:42.082 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:42.082 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:42.082 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:42.082 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2024-03-28T03:44:42.082 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2024-03-28T03:44:42.082 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2024-03-28T03:44:42.083 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:42.083 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-03-28T03:44:42.083 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2024-03-28T03:44:42.083 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:42.083 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:42.083 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:42.083 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:42.083 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:42.083 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:42.084 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:42.084 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:42.084 INFO:teuthology.orchestra.run.smithi119.stdout: "zabbix_host": { 2024-03-28T03:44:42.084 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "zabbix_host", 2024-03-28T03:44:42.084 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:42.084 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:42.084 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:42.084 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2024-03-28T03:44:42.084 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:42.084 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:42.085 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:42.085 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:42.085 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:42.085 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:42.085 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:42.085 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:42.085 INFO:teuthology.orchestra.run.smithi119.stdout: "zabbix_port": { 2024-03-28T03:44:42.085 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "zabbix_port", 2024-03-28T03:44:42.085 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2024-03-28T03:44:42.086 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:42.086 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:42.086 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "10051", 2024-03-28T03:44:42.086 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:42.086 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:42.086 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:42.086 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:42.086 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:42.086 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:42.087 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:42.087 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:42.087 INFO:teuthology.orchestra.run.smithi119.stdout: "zabbix_sender": { 2024-03-28T03:44:42.087 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "zabbix_sender", 2024-03-28T03:44:42.087 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2024-03-28T03:44:42.087 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2024-03-28T03:44:42.087 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2024-03-28T03:44:42.087 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-03-28T03:44:42.087 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2024-03-28T03:44:42.088 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2024-03-28T03:44:42.088 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2024-03-28T03:44:42.088 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2024-03-28T03:44:42.088 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2024-03-28T03:44:42.088 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2024-03-28T03:44:42.088 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2024-03-28T03:44:42.088 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:42.088 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:42.088 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:42.089 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:42.089 INFO:teuthology.orchestra.run.smithi119.stdout: "services": { 2024-03-28T03:44:42.089 INFO:teuthology.orchestra.run.smithi119.stdout: "dashboard": "https://172.21.15.119:8443/" 2024-03-28T03:44:42.089 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:42.089 INFO:teuthology.orchestra.run.smithi119.stdout: "always_on_modules": { 2024-03-28T03:44:42.089 INFO:teuthology.orchestra.run.smithi119.stdout: "octopus": [ 2024-03-28T03:44:42.089 INFO:teuthology.orchestra.run.smithi119.stdout: "balancer", 2024-03-28T03:44:42.089 INFO:teuthology.orchestra.run.smithi119.stdout: "crash", 2024-03-28T03:44:42.089 INFO:teuthology.orchestra.run.smithi119.stdout: "devicehealth", 2024-03-28T03:44:42.090 INFO:teuthology.orchestra.run.smithi119.stdout: "orchestrator", 2024-03-28T03:44:42.090 INFO:teuthology.orchestra.run.smithi119.stdout: "pg_autoscaler", 2024-03-28T03:44:42.090 INFO:teuthology.orchestra.run.smithi119.stdout: "progress", 2024-03-28T03:44:42.090 INFO:teuthology.orchestra.run.smithi119.stdout: "rbd_support", 2024-03-28T03:44:42.090 INFO:teuthology.orchestra.run.smithi119.stdout: "status", 2024-03-28T03:44:42.090 INFO:teuthology.orchestra.run.smithi119.stdout: "telemetry", 2024-03-28T03:44:42.090 INFO:teuthology.orchestra.run.smithi119.stdout: "volumes" 2024-03-28T03:44:42.090 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:42.091 INFO:teuthology.orchestra.run.smithi119.stdout: "pacific": [ 2024-03-28T03:44:42.091 INFO:teuthology.orchestra.run.smithi119.stdout: "balancer", 2024-03-28T03:44:42.091 INFO:teuthology.orchestra.run.smithi119.stdout: "crash", 2024-03-28T03:44:42.091 INFO:teuthology.orchestra.run.smithi119.stdout: "devicehealth", 2024-03-28T03:44:42.091 INFO:teuthology.orchestra.run.smithi119.stdout: "orchestrator", 2024-03-28T03:44:42.091 INFO:teuthology.orchestra.run.smithi119.stdout: "pg_autoscaler", 2024-03-28T03:44:42.091 INFO:teuthology.orchestra.run.smithi119.stdout: "progress", 2024-03-28T03:44:42.091 INFO:teuthology.orchestra.run.smithi119.stdout: "rbd_support", 2024-03-28T03:44:42.091 INFO:teuthology.orchestra.run.smithi119.stdout: "status", 2024-03-28T03:44:42.092 INFO:teuthology.orchestra.run.smithi119.stdout: "telemetry", 2024-03-28T03:44:42.092 INFO:teuthology.orchestra.run.smithi119.stdout: "volumes" 2024-03-28T03:44:42.092 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:42.092 INFO:teuthology.orchestra.run.smithi119.stdout: "quincy": [ 2024-03-28T03:44:42.092 INFO:teuthology.orchestra.run.smithi119.stdout: "balancer", 2024-03-28T03:44:42.092 INFO:teuthology.orchestra.run.smithi119.stdout: "crash", 2024-03-28T03:44:42.092 INFO:teuthology.orchestra.run.smithi119.stdout: "devicehealth", 2024-03-28T03:44:42.092 INFO:teuthology.orchestra.run.smithi119.stdout: "orchestrator", 2024-03-28T03:44:42.093 INFO:teuthology.orchestra.run.smithi119.stdout: "pg_autoscaler", 2024-03-28T03:44:42.093 INFO:teuthology.orchestra.run.smithi119.stdout: "progress", 2024-03-28T03:44:42.093 INFO:teuthology.orchestra.run.smithi119.stdout: "rbd_support", 2024-03-28T03:44:42.093 INFO:teuthology.orchestra.run.smithi119.stdout: "status", 2024-03-28T03:44:42.093 INFO:teuthology.orchestra.run.smithi119.stdout: "telemetry", 2024-03-28T03:44:42.093 INFO:teuthology.orchestra.run.smithi119.stdout: "volumes" 2024-03-28T03:44:42.093 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:42.093 INFO:teuthology.orchestra.run.smithi119.stdout: "reef": [ 2024-03-28T03:44:42.093 INFO:teuthology.orchestra.run.smithi119.stdout: "balancer", 2024-03-28T03:44:42.094 INFO:teuthology.orchestra.run.smithi119.stdout: "crash", 2024-03-28T03:44:42.094 INFO:teuthology.orchestra.run.smithi119.stdout: "devicehealth", 2024-03-28T03:44:42.094 INFO:teuthology.orchestra.run.smithi119.stdout: "orchestrator", 2024-03-28T03:44:42.094 INFO:teuthology.orchestra.run.smithi119.stdout: "pg_autoscaler", 2024-03-28T03:44:42.094 INFO:teuthology.orchestra.run.smithi119.stdout: "progress", 2024-03-28T03:44:42.094 INFO:teuthology.orchestra.run.smithi119.stdout: "rbd_support", 2024-03-28T03:44:42.094 INFO:teuthology.orchestra.run.smithi119.stdout: "status", 2024-03-28T03:44:42.094 INFO:teuthology.orchestra.run.smithi119.stdout: "telemetry", 2024-03-28T03:44:42.095 INFO:teuthology.orchestra.run.smithi119.stdout: "volumes" 2024-03-28T03:44:42.095 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-03-28T03:44:42.095 INFO:teuthology.orchestra.run.smithi119.stdout: "squid": [ 2024-03-28T03:44:42.095 INFO:teuthology.orchestra.run.smithi119.stdout: "balancer", 2024-03-28T03:44:42.095 INFO:teuthology.orchestra.run.smithi119.stdout: "crash", 2024-03-28T03:44:42.095 INFO:teuthology.orchestra.run.smithi119.stdout: "devicehealth", 2024-03-28T03:44:42.095 INFO:teuthology.orchestra.run.smithi119.stdout: "orchestrator", 2024-03-28T03:44:42.095 INFO:teuthology.orchestra.run.smithi119.stdout: "pg_autoscaler", 2024-03-28T03:44:42.095 INFO:teuthology.orchestra.run.smithi119.stdout: "progress", 2024-03-28T03:44:42.096 INFO:teuthology.orchestra.run.smithi119.stdout: "rbd_support", 2024-03-28T03:44:42.096 INFO:teuthology.orchestra.run.smithi119.stdout: "status", 2024-03-28T03:44:42.096 INFO:teuthology.orchestra.run.smithi119.stdout: "telemetry", 2024-03-28T03:44:42.096 INFO:teuthology.orchestra.run.smithi119.stdout: "volumes" 2024-03-28T03:44:42.096 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-03-28T03:44:42.096 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:42.096 INFO:teuthology.orchestra.run.smithi119.stdout: "last_failure_osd_epoch": 88, 2024-03-28T03:44:42.096 INFO:teuthology.orchestra.run.smithi119.stdout: "active_clients": [ 2024-03-28T03:44:42.096 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-03-28T03:44:42.097 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "devicehealth", 2024-03-28T03:44:42.097 INFO:teuthology.orchestra.run.smithi119.stdout: "addrvec": [ 2024-03-28T03:44:42.097 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-03-28T03:44:42.097 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v2", 2024-03-28T03:44:42.097 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "172.21.15.119:0", 2024-03-28T03:44:42.097 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 492327670 2024-03-28T03:44:42.097 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:42.097 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-03-28T03:44:42.098 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:42.098 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-03-28T03:44:42.098 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "libcephsqlite", 2024-03-28T03:44:42.098 INFO:teuthology.orchestra.run.smithi119.stdout: "addrvec": [ 2024-03-28T03:44:42.098 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-03-28T03:44:42.098 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v2", 2024-03-28T03:44:42.098 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "172.21.15.119:0", 2024-03-28T03:44:42.098 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 720778728 2024-03-28T03:44:42.099 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:42.099 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-03-28T03:44:42.099 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:42.099 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-03-28T03:44:42.099 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "rbd_support", 2024-03-28T03:44:42.099 INFO:teuthology.orchestra.run.smithi119.stdout: "addrvec": [ 2024-03-28T03:44:42.099 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-03-28T03:44:42.099 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v2", 2024-03-28T03:44:42.099 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "172.21.15.119:0", 2024-03-28T03:44:42.100 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 1583726902 2024-03-28T03:44:42.100 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:42.100 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-03-28T03:44:42.100 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:42.100 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-03-28T03:44:42.100 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "snap_schedule", 2024-03-28T03:44:42.100 INFO:teuthology.orchestra.run.smithi119.stdout: "addrvec": [ 2024-03-28T03:44:42.100 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-03-28T03:44:42.101 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v2", 2024-03-28T03:44:42.101 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "172.21.15.119:0", 2024-03-28T03:44:42.101 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 3251445222 2024-03-28T03:44:42.101 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:42.101 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-03-28T03:44:42.101 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:42.101 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-03-28T03:44:42.101 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "snap_schedule", 2024-03-28T03:44:42.101 INFO:teuthology.orchestra.run.smithi119.stdout: "addrvec": [ 2024-03-28T03:44:42.102 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-03-28T03:44:42.102 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v2", 2024-03-28T03:44:42.102 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "172.21.15.119:0", 2024-03-28T03:44:42.102 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 3071152482 2024-03-28T03:44:42.102 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:42.102 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-03-28T03:44:42.102 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:42.102 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-03-28T03:44:42.103 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "volumes", 2024-03-28T03:44:42.103 INFO:teuthology.orchestra.run.smithi119.stdout: "addrvec": [ 2024-03-28T03:44:42.103 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-03-28T03:44:42.103 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v2", 2024-03-28T03:44:42.103 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "172.21.15.119:0", 2024-03-28T03:44:42.103 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 630985334 2024-03-28T03:44:42.103 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:42.103 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-03-28T03:44:42.103 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:42.103 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-03-28T03:44:42.104 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-03-28T03:44:42.105 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:44:40 smithi132 ceph-mon[35941]: from='client.? 172.21.15.119:0/3345590493' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-28T03:44:42.105 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:44:40 smithi132 ceph-mon[35941]: from='client.? 172.21.15.119:0/3586976322' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-03-28T03:44:42.105 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:44:41 smithi132 ceph-mon[35941]: pgmap v6194: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:44:42.105 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:44:40 smithi165 ceph-mon[36302]: from='client.? 172.21.15.119:0/3345590493' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-28T03:44:42.105 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:44:40 smithi165 ceph-mon[36302]: from='client.? 172.21.15.119:0/3586976322' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-03-28T03:44:42.106 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:44:41 smithi165 ceph-mon[36302]: pgmap v6194: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:44:42.107 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:39.958+0000 7f7e42600640 1 -- 172.21.15.119:0/3586976322 >> [v2:172.21.15.119:6800/4273482239,v1:172.21.15.119:6801/4273482239] conn(0x7f7e1c0a2970 msgr2=0x7f7e1c0a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T03:44:42.107 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:39.958+0000 7f7e42600640 1 --2- 172.21.15.119:0/3586976322 >> [v2:172.21.15.119:6800/4273482239,v1:172.21.15.119:6801/4273482239] conn(0x7f7e1c0a2970 0x7f7e1c0a4e30 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f7e2c011440 tx=0x7f7e2c005f90 comp rx=0 tx=0).stop 2024-03-28T03:44:42.107 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:39.958+0000 7f7e42600640 1 -- 172.21.15.119:0/3586976322 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7e3c109860 msgr2=0x7f7e3c19e9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T03:44:42.107 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:39.958+0000 7f7e42600640 1 --2- 172.21.15.119:0/3586976322 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7e3c109860 0x7f7e3c19e9d0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f7e34037040 tx=0x7f7e34032ea0 comp rx=0 tx=0).stop 2024-03-28T03:44:42.107 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:39.958+0000 7f7e42600640 1 -- 172.21.15.119:0/3586976322 shutdown_connections 2024-03-28T03:44:42.107 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:39.958+0000 7f7e42600640 1 --2- 172.21.15.119:0/3586976322 >> [v2:172.21.15.119:6800/4273482239,v1:172.21.15.119:6801/4273482239] conn(0x7f7e1c0a2970 0x7f7e1c0a4e30 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:44:42.107 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:39.958+0000 7f7e42600640 1 --2- 172.21.15.119:0/3586976322 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7e3c109860 0x7f7e3c19e9d0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:44:42.107 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:39.958+0000 7f7e42600640 1 --2- 172.21.15.119:0/3586976322 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e3c104210 0x7f7e3c19a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:44:42.108 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:39.958+0000 7f7e42600640 1 --2- 172.21.15.119:0/3586976322 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7e3c103850 0x7f7e3c19a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:44:42.108 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:39.958+0000 7f7e42600640 1 -- 172.21.15.119:0/3586976322 >> 172.21.15.119:0/3586976322 conn(0x7f7e3c0ff320 msgr2=0x7f7e3c0ff9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T03:44:42.108 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:39.959+0000 7f7e42600640 1 -- 172.21.15.119:0/3586976322 shutdown_connections 2024-03-28T03:44:42.108 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:39.959+0000 7f7e42600640 1 -- 172.21.15.119:0/3586976322 wait complete. 2024-03-28T03:44:42.113 DEBUG:tasks.check_counter:Getting stats from a 2024-03-28T03:44:42.113 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 10f73b92-ec98-11ee-95d0-87774f69a715 -- ceph daemon mds.a perf dump 2024-03-28T03:44:42.114 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:44:40 smithi119 ceph-mon[31518]: from='client.? 172.21.15.119:0/3345590493' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-28T03:44:42.114 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:44:40 smithi119 ceph-mon[31518]: from='client.? 172.21.15.119:0/3586976322' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-03-28T03:44:42.114 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:44:41 smithi119 ceph-mon[31518]: pgmap v6194: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:44:42.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:44:42 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 11340.821639 secs 2024-03-28T03:44:42.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:44:42 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 11340.821639 secs 2024-03-28T03:44:42.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:44:42 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 11340.821639 secs 2024-03-28T03:44:42.608 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/mon.a/config 2024-03-28T03:44:43.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:44:43 smithi165 ceph-mon[36302]: pgmap v6195: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:44:43.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:44:43 smithi119 ceph-mon[31518]: pgmap v6195: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:44:43.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:44:43 smithi132 ceph-mon[35941]: pgmap v6195: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:44:44.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:44:44 smithi119 ceph-mon[31518]: pgmap v6196: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:44:44.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:44:44 smithi132 ceph-mon[35941]: pgmap v6196: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:44:44.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:44:44 smithi165 ceph-mon[36302]: pgmap v6196: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:44:44.994 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-03-28T03:44:44.994 INFO:teuthology.orchestra.run.smithi119.stdout: "AsyncMessenger::Worker-0": { 2024-03-28T03:44:44.994 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_messages": 12013, 2024-03-28T03:44:44.994 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_messages": 12015, 2024-03-28T03:44:44.994 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_bytes": 4036313, 2024-03-28T03:44:44.995 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_bytes": 3820135, 2024-03-28T03:44:44.995 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_created_connections": 19, 2024-03-28T03:44:44.995 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_active_connections": 2, 2024-03-28T03:44:44.995 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_total_time": 3.069268842, 2024-03-28T03:44:44.995 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_send_time": 1.382622823, 2024-03-28T03:44:44.995 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_recv_time": 0.666727247, 2024-03-28T03:44:44.995 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_fast_dispatch_time": 0.722648959, 2024-03-28T03:44:44.995 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_messages_queue_lat": { 2024-03-28T03:44:44.996 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 12013, 2024-03-28T03:44:44.996 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 1.253884108, 2024-03-28T03:44:44.996 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000104377 2024-03-28T03:44:44.996 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:44.996 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_handle_ack_lat": { 2024-03-28T03:44:44.996 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-28T03:44:44.996 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-28T03:44:44.996 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-28T03:44:44.996 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:44.997 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-28T03:44:44.997 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_encrypted_bytes": 0 2024-03-28T03:44:44.997 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:44.997 INFO:teuthology.orchestra.run.smithi119.stdout: "AsyncMessenger::Worker-1": { 2024-03-28T03:44:44.997 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_messages": 27253, 2024-03-28T03:44:44.997 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_messages": 27088, 2024-03-28T03:44:44.997 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_bytes": 9510043, 2024-03-28T03:44:44.997 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_bytes": 9522649, 2024-03-28T03:44:44.997 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_created_connections": 4, 2024-03-28T03:44:44.998 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_active_connections": 3, 2024-03-28T03:44:44.998 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_total_time": 4.855183241, 2024-03-28T03:44:44.998 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_send_time": 1.830172585, 2024-03-28T03:44:44.998 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_recv_time": 1.275476787, 2024-03-28T03:44:44.998 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_fast_dispatch_time": 1.369037912, 2024-03-28T03:44:44.998 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_messages_queue_lat": { 2024-03-28T03:44:44.998 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 27088, 2024-03-28T03:44:44.998 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 2.574984254, 2024-03-28T03:44:44.999 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000095059 2024-03-28T03:44:44.999 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:44.999 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_handle_ack_lat": { 2024-03-28T03:44:44.999 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-28T03:44:44.999 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-28T03:44:44.999 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-28T03:44:44.999 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:44.999 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_encrypted_bytes": 3441264, 2024-03-28T03:44:44.999 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_encrypted_bytes": 1928752 2024-03-28T03:44:44.999 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:45.000 INFO:teuthology.orchestra.run.smithi119.stdout: "AsyncMessenger::Worker-2": { 2024-03-28T03:44:45.000 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_messages": 16, 2024-03-28T03:44:45.000 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_messages": 2430, 2024-03-28T03:44:45.000 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_bytes": 3428, 2024-03-28T03:44:45.000 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_bytes": 5409477, 2024-03-28T03:44:45.000 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_created_connections": 43, 2024-03-28T03:44:45.000 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_active_connections": 5, 2024-03-28T03:44:45.000 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_total_time": 0.485189529, 2024-03-28T03:44:45.001 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_send_time": 0.374667167, 2024-03-28T03:44:45.001 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_recv_time": 0.043499074, 2024-03-28T03:44:45.001 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_fast_dispatch_time": 0.000072025, 2024-03-28T03:44:45.001 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_messages_queue_lat": { 2024-03-28T03:44:45.001 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 2427, 2024-03-28T03:44:45.001 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.295924962, 2024-03-28T03:44:45.001 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000121930 2024-03-28T03:44:45.001 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:45.001 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_handle_ack_lat": { 2024-03-28T03:44:45.002 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-28T03:44:45.002 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-28T03:44:45.002 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-28T03:44:45.002 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:45.002 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_encrypted_bytes": 2080, 2024-03-28T03:44:45.002 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_encrypted_bytes": 5408512 2024-03-28T03:44:45.002 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:45.002 INFO:teuthology.orchestra.run.smithi119.stdout: "cct": { 2024-03-28T03:44:45.003 INFO:teuthology.orchestra.run.smithi119.stdout: "total_workers": 1, 2024-03-28T03:44:45.003 INFO:teuthology.orchestra.run.smithi119.stdout: "unhealthy_workers": 0 2024-03-28T03:44:45.003 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:45.003 INFO:teuthology.orchestra.run.smithi119.stdout: "finisher-MDSRank": { 2024-03-28T03:44:45.003 INFO:teuthology.orchestra.run.smithi119.stdout: "queue_len": 0, 2024-03-28T03:44:45.003 INFO:teuthology.orchestra.run.smithi119.stdout: "complete_latency": { 2024-03-28T03:44:45.003 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 59223, 2024-03-28T03:44:45.003 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 3.036781939, 2024-03-28T03:44:45.003 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000051277 2024-03-28T03:44:45.004 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:45.004 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:45.004 INFO:teuthology.orchestra.run.smithi119.stdout: "finisher-PurgeQueue": { 2024-03-28T03:44:45.004 INFO:teuthology.orchestra.run.smithi119.stdout: "queue_len": 0, 2024-03-28T03:44:45.004 INFO:teuthology.orchestra.run.smithi119.stdout: "complete_latency": { 2024-03-28T03:44:45.004 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-28T03:44:45.004 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-28T03:44:45.004 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-28T03:44:45.004 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:45.005 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:45.005 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": { 2024-03-28T03:44:45.005 INFO:teuthology.orchestra.run.smithi119.stdout: "request": 0, 2024-03-28T03:44:45.005 INFO:teuthology.orchestra.run.smithi119.stdout: "reply": 0, 2024-03-28T03:44:45.005 INFO:teuthology.orchestra.run.smithi119.stdout: "reply_latency": { 2024-03-28T03:44:45.005 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-28T03:44:45.005 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-28T03:44:45.005 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-28T03:44:45.005 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:45.006 INFO:teuthology.orchestra.run.smithi119.stdout: "slow_reply": 0, 2024-03-28T03:44:45.006 INFO:teuthology.orchestra.run.smithi119.stdout: "forward": 0, 2024-03-28T03:44:45.006 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_fetch_complete": 0, 2024-03-28T03:44:45.006 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_fetch_keys": 0, 2024-03-28T03:44:45.006 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_commit": 0, 2024-03-28T03:44:45.006 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_split": 0, 2024-03-28T03:44:45.006 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_merge": 0, 2024-03-28T03:44:45.006 INFO:teuthology.orchestra.run.smithi119.stdout: "inodes": 0, 2024-03-28T03:44:45.007 INFO:teuthology.orchestra.run.smithi119.stdout: "inodes_top": 0, 2024-03-28T03:44:45.007 INFO:teuthology.orchestra.run.smithi119.stdout: "inodes_bottom": 0, 2024-03-28T03:44:45.007 INFO:teuthology.orchestra.run.smithi119.stdout: "inodes_pin_tail": 0, 2024-03-28T03:44:45.007 INFO:teuthology.orchestra.run.smithi119.stdout: "inodes_pinned": 0, 2024-03-28T03:44:45.007 INFO:teuthology.orchestra.run.smithi119.stdout: "inodes_expired": 0, 2024-03-28T03:44:45.007 INFO:teuthology.orchestra.run.smithi119.stdout: "inodes_with_caps": 0, 2024-03-28T03:44:45.007 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": 0, 2024-03-28T03:44:45.007 INFO:teuthology.orchestra.run.smithi119.stdout: "subtrees": 1, 2024-03-28T03:44:45.007 INFO:teuthology.orchestra.run.smithi119.stdout: "traverse": 0, 2024-03-28T03:44:45.008 INFO:teuthology.orchestra.run.smithi119.stdout: "traverse_hit": 0, 2024-03-28T03:44:45.008 INFO:teuthology.orchestra.run.smithi119.stdout: "traverse_forward": 0, 2024-03-28T03:44:45.008 INFO:teuthology.orchestra.run.smithi119.stdout: "traverse_discover": 0, 2024-03-28T03:44:45.008 INFO:teuthology.orchestra.run.smithi119.stdout: "traverse_dir_fetch": 0, 2024-03-28T03:44:45.008 INFO:teuthology.orchestra.run.smithi119.stdout: "traverse_remote_ino": 0, 2024-03-28T03:44:45.008 INFO:teuthology.orchestra.run.smithi119.stdout: "traverse_lock": 0, 2024-03-28T03:44:45.008 INFO:teuthology.orchestra.run.smithi119.stdout: "load_cent": 0, 2024-03-28T03:44:45.008 INFO:teuthology.orchestra.run.smithi119.stdout: "q": 0, 2024-03-28T03:44:45.008 INFO:teuthology.orchestra.run.smithi119.stdout: "exported": 0, 2024-03-28T03:44:45.009 INFO:teuthology.orchestra.run.smithi119.stdout: "exported_inodes": 0, 2024-03-28T03:44:45.009 INFO:teuthology.orchestra.run.smithi119.stdout: "imported": 0, 2024-03-28T03:44:45.009 INFO:teuthology.orchestra.run.smithi119.stdout: "imported_inodes": 0, 2024-03-28T03:44:45.009 INFO:teuthology.orchestra.run.smithi119.stdout: "openino_dir_fetch": 0, 2024-03-28T03:44:45.009 INFO:teuthology.orchestra.run.smithi119.stdout: "openino_backtrace_fetch": 0, 2024-03-28T03:44:45.009 INFO:teuthology.orchestra.run.smithi119.stdout: "openino_peer_discover": 0, 2024-03-28T03:44:45.009 INFO:teuthology.orchestra.run.smithi119.stdout: "root_rfiles": 0, 2024-03-28T03:44:45.009 INFO:teuthology.orchestra.run.smithi119.stdout: "root_rbytes": 0, 2024-03-28T03:44:45.009 INFO:teuthology.orchestra.run.smithi119.stdout: "root_rsnaps": 0, 2024-03-28T03:44:45.010 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_backtrace_fetch": 0, 2024-03-28T03:44:45.010 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_set_tag": 0, 2024-03-28T03:44:45.010 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_backtrace_repaired": 0, 2024-03-28T03:44:45.010 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_inotable_repaired": 0, 2024-03-28T03:44:45.010 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_dir_inodes": 0, 2024-03-28T03:44:45.010 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_dir_base_inodes": 0, 2024-03-28T03:44:45.010 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_dirfrag_rstats": 0, 2024-03-28T03:44:45.010 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_file_inodes": 0, 2024-03-28T03:44:45.011 INFO:teuthology.orchestra.run.smithi119.stdout: "handle_inode_file_caps": 0, 2024-03-28T03:44:45.011 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph_cap_op_revoke": 0, 2024-03-28T03:44:45.011 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph_cap_op_grant": 0, 2024-03-28T03:44:45.011 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph_cap_op_trunc": 0, 2024-03-28T03:44:45.011 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-28T03:44:45.011 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-28T03:44:45.011 INFO:teuthology.orchestra.run.smithi119.stdout: "handle_client_caps": 0, 2024-03-28T03:44:45.011 INFO:teuthology.orchestra.run.smithi119.stdout: "handle_client_caps_dirty": 0, 2024-03-28T03:44:45.011 INFO:teuthology.orchestra.run.smithi119.stdout: "handle_client_cap_release": 0, 2024-03-28T03:44:45.012 INFO:teuthology.orchestra.run.smithi119.stdout: "process_request_cap_release": 0 2024-03-28T03:44:45.012 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:45.012 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_cache": { 2024-03-28T03:44:45.012 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_update": 0, 2024-03-28T03:44:45.012 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_update_receipt": 0, 2024-03-28T03:44:45.012 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_try_discover": 0, 2024-03-28T03:44:45.012 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_send_discover": 0, 2024-03-28T03:44:45.012 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_handle_discover": 0, 2024-03-28T03:44:45.012 INFO:teuthology.orchestra.run.smithi119.stdout: "num_strays": 0, 2024-03-28T03:44:45.012 INFO:teuthology.orchestra.run.smithi119.stdout: "num_strays_delayed": 0, 2024-03-28T03:44:45.013 INFO:teuthology.orchestra.run.smithi119.stdout: "num_strays_enqueuing": 0, 2024-03-28T03:44:45.013 INFO:teuthology.orchestra.run.smithi119.stdout: "strays_created": 0, 2024-03-28T03:44:45.013 INFO:teuthology.orchestra.run.smithi119.stdout: "strays_enqueued": 0, 2024-03-28T03:44:45.013 INFO:teuthology.orchestra.run.smithi119.stdout: "strays_reintegrated": 0, 2024-03-28T03:44:45.013 INFO:teuthology.orchestra.run.smithi119.stdout: "strays_migrated": 0, 2024-03-28T03:44:45.013 INFO:teuthology.orchestra.run.smithi119.stdout: "num_recovering_processing": 0, 2024-03-28T03:44:45.013 INFO:teuthology.orchestra.run.smithi119.stdout: "num_recovering_enqueued": 0, 2024-03-28T03:44:45.013 INFO:teuthology.orchestra.run.smithi119.stdout: "num_recovering_prioritized": 0, 2024-03-28T03:44:45.014 INFO:teuthology.orchestra.run.smithi119.stdout: "recovery_started": 0, 2024-03-28T03:44:45.014 INFO:teuthology.orchestra.run.smithi119.stdout: "recovery_completed": 0, 2024-03-28T03:44:45.014 INFO:teuthology.orchestra.run.smithi119.stdout: "ireq_enqueue_scrub": 0, 2024-03-28T03:44:45.014 INFO:teuthology.orchestra.run.smithi119.stdout: "ireq_exportdir": 0, 2024-03-28T03:44:45.014 INFO:teuthology.orchestra.run.smithi119.stdout: "ireq_flush": 0, 2024-03-28T03:44:45.014 INFO:teuthology.orchestra.run.smithi119.stdout: "ireq_fragmentdir": 0, 2024-03-28T03:44:45.014 INFO:teuthology.orchestra.run.smithi119.stdout: "ireq_fragstats": 0, 2024-03-28T03:44:45.014 INFO:teuthology.orchestra.run.smithi119.stdout: "ireq_inodestats": 0 2024-03-28T03:44:45.014 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:45.015 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_log": { 2024-03-28T03:44:45.015 INFO:teuthology.orchestra.run.smithi119.stdout: "evlrg": 0, 2024-03-28T03:44:45.015 INFO:teuthology.orchestra.run.smithi119.stdout: "evadd": 0, 2024-03-28T03:44:45.015 INFO:teuthology.orchestra.run.smithi119.stdout: "evex": 0, 2024-03-28T03:44:45.015 INFO:teuthology.orchestra.run.smithi119.stdout: "evtrm": 0, 2024-03-28T03:44:45.015 INFO:teuthology.orchestra.run.smithi119.stdout: "ev": 7, 2024-03-28T03:44:45.015 INFO:teuthology.orchestra.run.smithi119.stdout: "evexg": 0, 2024-03-28T03:44:45.015 INFO:teuthology.orchestra.run.smithi119.stdout: "evexd": 0, 2024-03-28T03:44:45.015 INFO:teuthology.orchestra.run.smithi119.stdout: "segadd": 0, 2024-03-28T03:44:45.016 INFO:teuthology.orchestra.run.smithi119.stdout: "segex": 0, 2024-03-28T03:44:45.016 INFO:teuthology.orchestra.run.smithi119.stdout: "segtrm": 0, 2024-03-28T03:44:45.016 INFO:teuthology.orchestra.run.smithi119.stdout: "seg": 2, 2024-03-28T03:44:45.016 INFO:teuthology.orchestra.run.smithi119.stdout: "segmjr": 2, 2024-03-28T03:44:45.016 INFO:teuthology.orchestra.run.smithi119.stdout: "segexg": 0, 2024-03-28T03:44:45.016 INFO:teuthology.orchestra.run.smithi119.stdout: "segexd": 0, 2024-03-28T03:44:45.016 INFO:teuthology.orchestra.run.smithi119.stdout: "expos": 4194304, 2024-03-28T03:44:45.016 INFO:teuthology.orchestra.run.smithi119.stdout: "wrpos": 4196347, 2024-03-28T03:44:45.016 INFO:teuthology.orchestra.run.smithi119.stdout: "rdpos": 4196206, 2024-03-28T03:44:45.017 INFO:teuthology.orchestra.run.smithi119.stdout: "jlat": { 2024-03-28T03:44:45.017 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-28T03:44:45.017 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-28T03:44:45.017 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-28T03:44:45.017 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:45.017 INFO:teuthology.orchestra.run.smithi119.stdout: "replayed": 7 2024-03-28T03:44:45.017 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:45.017 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_mem": { 2024-03-28T03:44:45.017 INFO:teuthology.orchestra.run.smithi119.stdout: "ino": 3, 2024-03-28T03:44:45.018 INFO:teuthology.orchestra.run.smithi119.stdout: "ino+": 3, 2024-03-28T03:44:45.018 INFO:teuthology.orchestra.run.smithi119.stdout: "ino-": 0, 2024-03-28T03:44:45.018 INFO:teuthology.orchestra.run.smithi119.stdout: "dir": 1, 2024-03-28T03:44:45.018 INFO:teuthology.orchestra.run.smithi119.stdout: "dir+": 1, 2024-03-28T03:44:45.018 INFO:teuthology.orchestra.run.smithi119.stdout: "dir-": 0, 2024-03-28T03:44:45.018 INFO:teuthology.orchestra.run.smithi119.stdout: "dn": 0, 2024-03-28T03:44:45.018 INFO:teuthology.orchestra.run.smithi119.stdout: "dn+": 0, 2024-03-28T03:44:45.018 INFO:teuthology.orchestra.run.smithi119.stdout: "dn-": 0, 2024-03-28T03:44:45.019 INFO:teuthology.orchestra.run.smithi119.stdout: "cap": 0, 2024-03-28T03:44:45.019 INFO:teuthology.orchestra.run.smithi119.stdout: "cap+": 0, 2024-03-28T03:44:45.019 INFO:teuthology.orchestra.run.smithi119.stdout: "cap-": 0, 2024-03-28T03:44:45.019 INFO:teuthology.orchestra.run.smithi119.stdout: "rss": 49860, 2024-03-28T03:44:45.019 INFO:teuthology.orchestra.run.smithi119.stdout: "heap": 215420 2024-03-28T03:44:45.019 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:45.019 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_server": { 2024-03-28T03:44:45.019 INFO:teuthology.orchestra.run.smithi119.stdout: "dispatch_client_request": 0, 2024-03-28T03:44:45.019 INFO:teuthology.orchestra.run.smithi119.stdout: "dispatch_server_request": 0, 2024-03-28T03:44:45.019 INFO:teuthology.orchestra.run.smithi119.stdout: "handle_client_request": 0, 2024-03-28T03:44:45.020 INFO:teuthology.orchestra.run.smithi119.stdout: "handle_client_session": 0, 2024-03-28T03:44:45.020 INFO:teuthology.orchestra.run.smithi119.stdout: "handle_peer_request": 0, 2024-03-28T03:44:45.020 INFO:teuthology.orchestra.run.smithi119.stdout: "req_create_latency": { 2024-03-28T03:44:45.020 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-28T03:44:45.020 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-28T03:44:45.020 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-28T03:44:45.020 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:45.020 INFO:teuthology.orchestra.run.smithi119.stdout: "req_getattr_latency": { 2024-03-28T03:44:45.020 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-28T03:44:45.021 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-28T03:44:45.021 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-28T03:44:45.021 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:45.021 INFO:teuthology.orchestra.run.smithi119.stdout: "req_getfilelock_latency": { 2024-03-28T03:44:45.021 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-28T03:44:45.021 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-28T03:44:45.021 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-28T03:44:45.021 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:45.021 INFO:teuthology.orchestra.run.smithi119.stdout: "req_link_latency": { 2024-03-28T03:44:45.022 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-28T03:44:45.022 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-28T03:44:45.022 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-28T03:44:45.022 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:45.022 INFO:teuthology.orchestra.run.smithi119.stdout: "req_lookup_latency": { 2024-03-28T03:44:45.022 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-28T03:44:45.022 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-28T03:44:45.022 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-28T03:44:45.023 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:45.023 INFO:teuthology.orchestra.run.smithi119.stdout: "req_lookuphash_latency": { 2024-03-28T03:44:45.023 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-28T03:44:45.023 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-28T03:44:45.023 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-28T03:44:45.023 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:45.023 INFO:teuthology.orchestra.run.smithi119.stdout: "req_lookupino_latency": { 2024-03-28T03:44:45.023 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-28T03:44:45.023 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-28T03:44:45.023 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-28T03:44:45.024 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:45.024 INFO:teuthology.orchestra.run.smithi119.stdout: "req_lookupname_latency": { 2024-03-28T03:44:45.024 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-28T03:44:45.024 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-28T03:44:45.024 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-28T03:44:45.024 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:45.024 INFO:teuthology.orchestra.run.smithi119.stdout: "req_lookupparent_latency": { 2024-03-28T03:44:45.024 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-28T03:44:45.024 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-28T03:44:45.025 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-28T03:44:45.025 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:45.025 INFO:teuthology.orchestra.run.smithi119.stdout: "req_lookupsnap_latency": { 2024-03-28T03:44:45.025 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-28T03:44:45.025 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-28T03:44:45.025 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-28T03:44:45.025 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:45.025 INFO:teuthology.orchestra.run.smithi119.stdout: "req_lssnap_latency": { 2024-03-28T03:44:45.025 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-28T03:44:45.026 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-28T03:44:45.026 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-28T03:44:45.026 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:45.026 INFO:teuthology.orchestra.run.smithi119.stdout: "req_mkdir_latency": { 2024-03-28T03:44:45.026 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-28T03:44:45.026 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-28T03:44:45.026 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-28T03:44:45.027 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:45.027 INFO:teuthology.orchestra.run.smithi119.stdout: "req_mknod_latency": { 2024-03-28T03:44:45.027 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-28T03:44:45.027 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-28T03:44:45.027 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-28T03:44:45.027 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:45.028 INFO:teuthology.orchestra.run.smithi119.stdout: "req_mksnap_latency": { 2024-03-28T03:44:45.028 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-28T03:44:45.028 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-28T03:44:45.028 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-28T03:44:45.028 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:45.028 INFO:teuthology.orchestra.run.smithi119.stdout: "req_open_latency": { 2024-03-28T03:44:45.028 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-28T03:44:45.028 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-28T03:44:45.028 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-28T03:44:45.029 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:45.029 INFO:teuthology.orchestra.run.smithi119.stdout: "req_readdir_latency": { 2024-03-28T03:44:45.029 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-28T03:44:45.029 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-28T03:44:45.029 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-28T03:44:45.029 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:45.029 INFO:teuthology.orchestra.run.smithi119.stdout: "req_rename_latency": { 2024-03-28T03:44:45.029 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-28T03:44:45.029 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-28T03:44:45.030 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-28T03:44:45.030 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:45.030 INFO:teuthology.orchestra.run.smithi119.stdout: "req_renamesnap_latency": { 2024-03-28T03:44:45.030 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-28T03:44:45.030 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-28T03:44:45.030 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-28T03:44:45.030 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:45.030 INFO:teuthology.orchestra.run.smithi119.stdout: "req_snapdiff_latency": { 2024-03-28T03:44:45.031 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-28T03:44:45.031 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-28T03:44:45.031 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-28T03:44:45.031 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:45.031 INFO:teuthology.orchestra.run.smithi119.stdout: "req_rmdir_latency": { 2024-03-28T03:44:45.031 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-28T03:44:45.031 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-28T03:44:45.031 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-28T03:44:45.031 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:45.032 INFO:teuthology.orchestra.run.smithi119.stdout: "req_rmsnap_latency": { 2024-03-28T03:44:45.032 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-28T03:44:45.032 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-28T03:44:45.032 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-28T03:44:45.032 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:45.032 INFO:teuthology.orchestra.run.smithi119.stdout: "req_rmxattr_latency": { 2024-03-28T03:44:45.032 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-28T03:44:45.032 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-28T03:44:45.032 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-28T03:44:45.033 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:45.033 INFO:teuthology.orchestra.run.smithi119.stdout: "req_setattr_latency": { 2024-03-28T03:44:45.033 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-28T03:44:45.033 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-28T03:44:45.033 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-28T03:44:45.033 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:45.033 INFO:teuthology.orchestra.run.smithi119.stdout: "req_setdirlayout_latency": { 2024-03-28T03:44:45.033 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-28T03:44:45.033 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-28T03:44:45.034 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-28T03:44:45.034 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:45.034 INFO:teuthology.orchestra.run.smithi119.stdout: "req_setfilelock_latency": { 2024-03-28T03:44:45.034 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-28T03:44:45.034 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-28T03:44:45.034 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-28T03:44:45.034 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:45.034 INFO:teuthology.orchestra.run.smithi119.stdout: "req_setlayout_latency": { 2024-03-28T03:44:45.034 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-28T03:44:45.035 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-28T03:44:45.035 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-28T03:44:45.035 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:45.035 INFO:teuthology.orchestra.run.smithi119.stdout: "req_setxattr_latency": { 2024-03-28T03:44:45.035 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-28T03:44:45.035 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-28T03:44:45.035 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-28T03:44:45.035 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:45.035 INFO:teuthology.orchestra.run.smithi119.stdout: "req_symlink_latency": { 2024-03-28T03:44:45.036 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-28T03:44:45.036 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-28T03:44:45.036 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-28T03:44:45.036 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:45.036 INFO:teuthology.orchestra.run.smithi119.stdout: "req_unlink_latency": { 2024-03-28T03:44:45.036 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-28T03:44:45.036 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-28T03:44:45.036 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-28T03:44:45.036 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:45.037 INFO:teuthology.orchestra.run.smithi119.stdout: "cap_revoke_eviction": 0, 2024-03-28T03:44:45.037 INFO:teuthology.orchestra.run.smithi119.stdout: "cap_acquisition_throttle": 0, 2024-03-28T03:44:45.037 INFO:teuthology.orchestra.run.smithi119.stdout: "req_getvxattr_latency": { 2024-03-28T03:44:45.037 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-28T03:44:45.037 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-28T03:44:45.037 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-28T03:44:45.037 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:45.037 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:45.038 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_sessions": { 2024-03-28T03:44:45.038 INFO:teuthology.orchestra.run.smithi119.stdout: "session_count": 1, 2024-03-28T03:44:45.038 INFO:teuthology.orchestra.run.smithi119.stdout: "session_add": 3, 2024-03-28T03:44:45.038 INFO:teuthology.orchestra.run.smithi119.stdout: "session_remove": 2, 2024-03-28T03:44:45.038 INFO:teuthology.orchestra.run.smithi119.stdout: "sessions_open": 3, 2024-03-28T03:44:45.038 INFO:teuthology.orchestra.run.smithi119.stdout: "sessions_stale": 0, 2024-03-28T03:44:45.038 INFO:teuthology.orchestra.run.smithi119.stdout: "total_load": 0, 2024-03-28T03:44:45.038 INFO:teuthology.orchestra.run.smithi119.stdout: "average_load": 0, 2024-03-28T03:44:45.039 INFO:teuthology.orchestra.run.smithi119.stdout: "avg_session_uptime": 15827, 2024-03-28T03:44:45.039 INFO:teuthology.orchestra.run.smithi119.stdout: "mdthresh_evicted": 0 2024-03-28T03:44:45.039 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:45.039 INFO:teuthology.orchestra.run.smithi119.stdout: "mempool": { 2024-03-28T03:44:45.039 INFO:teuthology.orchestra.run.smithi119.stdout: "bloom_filter_bytes": 0, 2024-03-28T03:44:45.039 INFO:teuthology.orchestra.run.smithi119.stdout: "bloom_filter_items": 0, 2024-03-28T03:44:45.039 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_alloc_bytes": 0, 2024-03-28T03:44:45.039 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_alloc_items": 0, 2024-03-28T03:44:45.039 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_data_bytes": 0, 2024-03-28T03:44:45.039 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_data_items": 0, 2024-03-28T03:44:45.040 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-28T03:44:45.040 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_onode_items": 0, 2024-03-28T03:44:45.040 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-28T03:44:45.040 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_meta_items": 0, 2024-03-28T03:44:45.040 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_other_bytes": 0, 2024-03-28T03:44:45.040 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_other_items": 0, 2024-03-28T03:44:45.040 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-28T03:44:45.040 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_buffer_items": 0, 2024-03-28T03:44:45.041 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_extent_bytes": 0, 2024-03-28T03:44:45.041 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_extent_items": 0, 2024-03-28T03:44:45.041 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_blob_bytes": 0, 2024-03-28T03:44:45.041 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_blob_items": 0, 2024-03-28T03:44:45.041 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-28T03:44:45.041 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_shared_blob_items": 0, 2024-03-28T03:44:45.041 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-28T03:44:45.041 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_inline_bl_items": 0, 2024-03-28T03:44:45.041 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_fsck_bytes": 0, 2024-03-28T03:44:45.042 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_fsck_items": 0, 2024-03-28T03:44:45.042 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_txc_bytes": 0, 2024-03-28T03:44:45.042 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_txc_items": 0, 2024-03-28T03:44:45.042 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-28T03:44:45.042 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_writing_deferred_items": 0, 2024-03-28T03:44:45.042 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_writing_bytes": 0, 2024-03-28T03:44:45.042 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_writing_items": 0, 2024-03-28T03:44:45.042 INFO:teuthology.orchestra.run.smithi119.stdout: "bluefs_bytes": 0, 2024-03-28T03:44:45.042 INFO:teuthology.orchestra.run.smithi119.stdout: "bluefs_items": 0, 2024-03-28T03:44:45.043 INFO:teuthology.orchestra.run.smithi119.stdout: "bluefs_file_reader_bytes": 0, 2024-03-28T03:44:45.043 INFO:teuthology.orchestra.run.smithi119.stdout: "bluefs_file_reader_items": 0, 2024-03-28T03:44:45.043 INFO:teuthology.orchestra.run.smithi119.stdout: "bluefs_file_writer_bytes": 0, 2024-03-28T03:44:45.043 INFO:teuthology.orchestra.run.smithi119.stdout: "bluefs_file_writer_items": 0, 2024-03-28T03:44:45.043 INFO:teuthology.orchestra.run.smithi119.stdout: "buffer_anon_bytes": 198274, 2024-03-28T03:44:45.043 INFO:teuthology.orchestra.run.smithi119.stdout: "buffer_anon_items": 58, 2024-03-28T03:44:45.043 INFO:teuthology.orchestra.run.smithi119.stdout: "buffer_meta_bytes": 0, 2024-03-28T03:44:45.043 INFO:teuthology.orchestra.run.smithi119.stdout: "buffer_meta_items": 0, 2024-03-28T03:44:45.043 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_bytes": 0, 2024-03-28T03:44:45.044 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_items": 0, 2024-03-28T03:44:45.044 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_mapbl_bytes": 0, 2024-03-28T03:44:45.044 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_mapbl_items": 0, 2024-03-28T03:44:45.044 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_pglog_bytes": 0, 2024-03-28T03:44:45.044 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_pglog_items": 0, 2024-03-28T03:44:45.044 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_bytes": 13016, 2024-03-28T03:44:45.044 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_items": 374, 2024-03-28T03:44:45.044 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_mapping_bytes": 0, 2024-03-28T03:44:45.044 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_mapping_items": 0, 2024-03-28T03:44:45.044 INFO:teuthology.orchestra.run.smithi119.stdout: "pgmap_bytes": 0, 2024-03-28T03:44:45.045 INFO:teuthology.orchestra.run.smithi119.stdout: "pgmap_items": 0, 2024-03-28T03:44:45.045 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_co_bytes": 8320, 2024-03-28T03:44:45.045 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_co_items": 13, 2024-03-28T03:44:45.045 INFO:teuthology.orchestra.run.smithi119.stdout: "unittest_1_bytes": 0, 2024-03-28T03:44:45.045 INFO:teuthology.orchestra.run.smithi119.stdout: "unittest_1_items": 0, 2024-03-28T03:44:45.045 INFO:teuthology.orchestra.run.smithi119.stdout: "unittest_2_bytes": 0, 2024-03-28T03:44:45.045 INFO:teuthology.orchestra.run.smithi119.stdout: "unittest_2_items": 0 2024-03-28T03:44:45.045 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:45.045 INFO:teuthology.orchestra.run.smithi119.stdout: "objecter": { 2024-03-28T03:44:45.046 INFO:teuthology.orchestra.run.smithi119.stdout: "op_active": 0, 2024-03-28T03:44:45.046 INFO:teuthology.orchestra.run.smithi119.stdout: "op_laggy": 0, 2024-03-28T03:44:45.046 INFO:teuthology.orchestra.run.smithi119.stdout: "op_send": 35896, 2024-03-28T03:44:45.046 INFO:teuthology.orchestra.run.smithi119.stdout: "op_send_bytes": 21, 2024-03-28T03:44:45.046 INFO:teuthology.orchestra.run.smithi119.stdout: "op_resend": 3, 2024-03-28T03:44:45.046 INFO:teuthology.orchestra.run.smithi119.stdout: "op_reply": 35896, 2024-03-28T03:44:45.046 INFO:teuthology.orchestra.run.smithi119.stdout: "op_latency": { 2024-03-28T03:44:45.046 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 35896, 2024-03-28T03:44:45.047 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 29.078279170, 2024-03-28T03:44:45.047 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000810070 2024-03-28T03:44:45.047 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:45.047 INFO:teuthology.orchestra.run.smithi119.stdout: "op_inflight": 0, 2024-03-28T03:44:45.047 INFO:teuthology.orchestra.run.smithi119.stdout: "oplen_avg": { 2024-03-28T03:44:45.047 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 35896, 2024-03-28T03:44:45.047 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 35897 2024-03-28T03:44:45.047 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:45.047 INFO:teuthology.orchestra.run.smithi119.stdout: "op": 35896, 2024-03-28T03:44:45.047 INFO:teuthology.orchestra.run.smithi119.stdout: "op_r": 35896, 2024-03-28T03:44:45.048 INFO:teuthology.orchestra.run.smithi119.stdout: "op_w": 0, 2024-03-28T03:44:45.048 INFO:teuthology.orchestra.run.smithi119.stdout: "op_rmw": 0, 2024-03-28T03:44:45.048 INFO:teuthology.orchestra.run.smithi119.stdout: "op_pg": 0, 2024-03-28T03:44:45.048 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_stat": 23875, 2024-03-28T03:44:45.048 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_create": 0, 2024-03-28T03:44:45.048 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_read": 12019, 2024-03-28T03:44:45.048 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_write": 0, 2024-03-28T03:44:45.048 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_writefull": 0, 2024-03-28T03:44:45.048 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_writesame": 0, 2024-03-28T03:44:45.049 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_append": 0, 2024-03-28T03:44:45.049 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_zero": 0, 2024-03-28T03:44:45.049 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_truncate": 0, 2024-03-28T03:44:45.049 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_delete": 0, 2024-03-28T03:44:45.049 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_mapext": 0, 2024-03-28T03:44:45.049 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_sparse_read": 0, 2024-03-28T03:44:45.049 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_clonerange": 0, 2024-03-28T03:44:45.049 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_getxattr": 1, 2024-03-28T03:44:45.049 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_setxattr": 0, 2024-03-28T03:44:45.050 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_cmpxattr": 0, 2024-03-28T03:44:45.050 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_rmxattr": 0, 2024-03-28T03:44:45.050 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_resetxattrs": 0, 2024-03-28T03:44:45.050 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_call": 0, 2024-03-28T03:44:45.050 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_watch": 0, 2024-03-28T03:44:45.050 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_notify": 0, 2024-03-28T03:44:45.050 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_src_cmpxattr": 0, 2024-03-28T03:44:45.050 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_pgls": 0, 2024-03-28T03:44:45.050 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_pgls_filter": 0, 2024-03-28T03:44:45.051 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_other": 0, 2024-03-28T03:44:45.051 INFO:teuthology.orchestra.run.smithi119.stdout: "linger_active": 0, 2024-03-28T03:44:45.051 INFO:teuthology.orchestra.run.smithi119.stdout: "linger_send": 0, 2024-03-28T03:44:45.051 INFO:teuthology.orchestra.run.smithi119.stdout: "linger_resend": 0, 2024-03-28T03:44:45.051 INFO:teuthology.orchestra.run.smithi119.stdout: "linger_ping": 0, 2024-03-28T03:44:45.051 INFO:teuthology.orchestra.run.smithi119.stdout: "poolop_active": 0, 2024-03-28T03:44:45.051 INFO:teuthology.orchestra.run.smithi119.stdout: "poolop_send": 0, 2024-03-28T03:44:45.051 INFO:teuthology.orchestra.run.smithi119.stdout: "poolop_resend": 0, 2024-03-28T03:44:45.051 INFO:teuthology.orchestra.run.smithi119.stdout: "poolstat_active": 0, 2024-03-28T03:44:45.052 INFO:teuthology.orchestra.run.smithi119.stdout: "poolstat_send": 0, 2024-03-28T03:44:45.052 INFO:teuthology.orchestra.run.smithi119.stdout: "poolstat_resend": 0, 2024-03-28T03:44:45.052 INFO:teuthology.orchestra.run.smithi119.stdout: "statfs_active": 0, 2024-03-28T03:44:45.052 INFO:teuthology.orchestra.run.smithi119.stdout: "statfs_send": 0, 2024-03-28T03:44:45.052 INFO:teuthology.orchestra.run.smithi119.stdout: "statfs_resend": 0, 2024-03-28T03:44:45.052 INFO:teuthology.orchestra.run.smithi119.stdout: "command_active": 0, 2024-03-28T03:44:45.052 INFO:teuthology.orchestra.run.smithi119.stdout: "command_send": 0, 2024-03-28T03:44:45.052 INFO:teuthology.orchestra.run.smithi119.stdout: "command_resend": 0, 2024-03-28T03:44:45.052 INFO:teuthology.orchestra.run.smithi119.stdout: "map_epoch": 125, 2024-03-28T03:44:45.053 INFO:teuthology.orchestra.run.smithi119.stdout: "map_full": 0, 2024-03-28T03:44:45.053 INFO:teuthology.orchestra.run.smithi119.stdout: "map_inc": 40, 2024-03-28T03:44:45.053 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_sessions": 6, 2024-03-28T03:44:45.053 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_session_open": 44, 2024-03-28T03:44:45.053 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_session_close": 38, 2024-03-28T03:44:45.053 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_laggy": 0, 2024-03-28T03:44:45.053 INFO:teuthology.orchestra.run.smithi119.stdout: "omap_wr": 0, 2024-03-28T03:44:45.053 INFO:teuthology.orchestra.run.smithi119.stdout: "omap_rd": 2, 2024-03-28T03:44:45.053 INFO:teuthology.orchestra.run.smithi119.stdout: "omap_del": 0 2024-03-28T03:44:45.053 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:45.054 INFO:teuthology.orchestra.run.smithi119.stdout: "oft": { 2024-03-28T03:44:45.054 INFO:teuthology.orchestra.run.smithi119.stdout: "omap_total_objs": 0, 2024-03-28T03:44:45.054 INFO:teuthology.orchestra.run.smithi119.stdout: "omap_total_kv_pairs": 0, 2024-03-28T03:44:45.054 INFO:teuthology.orchestra.run.smithi119.stdout: "omap_total_updates": 0, 2024-03-28T03:44:45.054 INFO:teuthology.orchestra.run.smithi119.stdout: "omap_total_removes": 0 2024-03-28T03:44:45.054 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:45.054 INFO:teuthology.orchestra.run.smithi119.stdout: "osd-slow-ops": { 2024-03-28T03:44:45.054 INFO:teuthology.orchestra.run.smithi119.stdout: "slow_ops_count": 0 2024-03-28T03:44:45.054 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:45.055 INFO:teuthology.orchestra.run.smithi119.stdout: "purge_queue": { 2024-03-28T03:44:45.055 INFO:teuthology.orchestra.run.smithi119.stdout: "pq_executing_ops": 0, 2024-03-28T03:44:45.055 INFO:teuthology.orchestra.run.smithi119.stdout: "pq_executing_ops_high_water": 0, 2024-03-28T03:44:45.055 INFO:teuthology.orchestra.run.smithi119.stdout: "pq_executing": 0, 2024-03-28T03:44:45.055 INFO:teuthology.orchestra.run.smithi119.stdout: "pq_executing_high_water": 0, 2024-03-28T03:44:45.055 INFO:teuthology.orchestra.run.smithi119.stdout: "pq_executed_ops": 0, 2024-03-28T03:44:45.055 INFO:teuthology.orchestra.run.smithi119.stdout: "pq_executed": 0, 2024-03-28T03:44:45.055 INFO:teuthology.orchestra.run.smithi119.stdout: "pq_item_in_journal": 0 2024-03-28T03:44:45.055 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:45.056 INFO:teuthology.orchestra.run.smithi119.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-28T03:44:45.056 INFO:teuthology.orchestra.run.smithi119.stdout: "val": 0, 2024-03-28T03:44:45.056 INFO:teuthology.orchestra.run.smithi119.stdout: "max": 104857600, 2024-03-28T03:44:45.056 INFO:teuthology.orchestra.run.smithi119.stdout: "get_started": 0, 2024-03-28T03:44:45.056 INFO:teuthology.orchestra.run.smithi119.stdout: "get": 39278, 2024-03-28T03:44:45.056 INFO:teuthology.orchestra.run.smithi119.stdout: "get_sum": 9838663, 2024-03-28T03:44:45.056 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_fail": 0, 2024-03-28T03:44:45.056 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_success": 39278, 2024-03-28T03:44:45.056 INFO:teuthology.orchestra.run.smithi119.stdout: "take": 0, 2024-03-28T03:44:45.056 INFO:teuthology.orchestra.run.smithi119.stdout: "take_sum": 0, 2024-03-28T03:44:45.057 INFO:teuthology.orchestra.run.smithi119.stdout: "put": 39278, 2024-03-28T03:44:45.057 INFO:teuthology.orchestra.run.smithi119.stdout: "put_sum": 9838663, 2024-03-28T03:44:45.057 INFO:teuthology.orchestra.run.smithi119.stdout: "wait": { 2024-03-28T03:44:45.057 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-28T03:44:45.057 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-28T03:44:45.057 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-28T03:44:45.057 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:45.057 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:45.057 INFO:teuthology.orchestra.run.smithi119.stdout: "throttle-objecter_bytes": { 2024-03-28T03:44:45.058 INFO:teuthology.orchestra.run.smithi119.stdout: "val": 0, 2024-03-28T03:44:45.058 INFO:teuthology.orchestra.run.smithi119.stdout: "max": 104857600, 2024-03-28T03:44:45.058 INFO:teuthology.orchestra.run.smithi119.stdout: "get_started": 0, 2024-03-28T03:44:45.058 INFO:teuthology.orchestra.run.smithi119.stdout: "get": 0, 2024-03-28T03:44:45.058 INFO:teuthology.orchestra.run.smithi119.stdout: "get_sum": 0, 2024-03-28T03:44:45.058 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_fail": 0, 2024-03-28T03:44:45.058 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_success": 0, 2024-03-28T03:44:45.058 INFO:teuthology.orchestra.run.smithi119.stdout: "take": 35896, 2024-03-28T03:44:45.058 INFO:teuthology.orchestra.run.smithi119.stdout: "take_sum": 2048, 2024-03-28T03:44:45.059 INFO:teuthology.orchestra.run.smithi119.stdout: "put": 35896, 2024-03-28T03:44:45.059 INFO:teuthology.orchestra.run.smithi119.stdout: "put_sum": 2048, 2024-03-28T03:44:45.059 INFO:teuthology.orchestra.run.smithi119.stdout: "wait": { 2024-03-28T03:44:45.059 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-28T03:44:45.059 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-28T03:44:45.059 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-28T03:44:45.059 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:45.059 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:45.059 INFO:teuthology.orchestra.run.smithi119.stdout: "throttle-objecter_ops": { 2024-03-28T03:44:45.060 INFO:teuthology.orchestra.run.smithi119.stdout: "val": 0, 2024-03-28T03:44:45.060 INFO:teuthology.orchestra.run.smithi119.stdout: "max": 1024, 2024-03-28T03:44:45.060 INFO:teuthology.orchestra.run.smithi119.stdout: "get_started": 0, 2024-03-28T03:44:45.060 INFO:teuthology.orchestra.run.smithi119.stdout: "get": 0, 2024-03-28T03:44:45.060 INFO:teuthology.orchestra.run.smithi119.stdout: "get_sum": 0, 2024-03-28T03:44:45.060 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_fail": 0, 2024-03-28T03:44:45.060 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_success": 0, 2024-03-28T03:44:45.060 INFO:teuthology.orchestra.run.smithi119.stdout: "take": 35896, 2024-03-28T03:44:45.060 INFO:teuthology.orchestra.run.smithi119.stdout: "take_sum": 35896, 2024-03-28T03:44:45.061 INFO:teuthology.orchestra.run.smithi119.stdout: "put": 35896, 2024-03-28T03:44:45.061 INFO:teuthology.orchestra.run.smithi119.stdout: "put_sum": 35896, 2024-03-28T03:44:45.061 INFO:teuthology.orchestra.run.smithi119.stdout: "wait": { 2024-03-28T03:44:45.061 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-28T03:44:45.061 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-28T03:44:45.061 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-28T03:44:45.061 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:45.061 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:45.061 INFO:teuthology.orchestra.run.smithi119.stdout: "throttle-write_buf_throttle": { 2024-03-28T03:44:45.062 INFO:teuthology.orchestra.run.smithi119.stdout: "val": 0, 2024-03-28T03:44:45.062 INFO:teuthology.orchestra.run.smithi119.stdout: "max": 3758096384, 2024-03-28T03:44:45.062 INFO:teuthology.orchestra.run.smithi119.stdout: "get_started": 0, 2024-03-28T03:44:45.062 INFO:teuthology.orchestra.run.smithi119.stdout: "get": 0, 2024-03-28T03:44:45.062 INFO:teuthology.orchestra.run.smithi119.stdout: "get_sum": 0, 2024-03-28T03:44:45.062 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_fail": 0, 2024-03-28T03:44:45.062 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_success": 0, 2024-03-28T03:44:45.062 INFO:teuthology.orchestra.run.smithi119.stdout: "take": 0, 2024-03-28T03:44:45.063 INFO:teuthology.orchestra.run.smithi119.stdout: "take_sum": 0, 2024-03-28T03:44:45.063 INFO:teuthology.orchestra.run.smithi119.stdout: "put": 0, 2024-03-28T03:44:45.063 INFO:teuthology.orchestra.run.smithi119.stdout: "put_sum": 0, 2024-03-28T03:44:45.063 INFO:teuthology.orchestra.run.smithi119.stdout: "wait": { 2024-03-28T03:44:45.063 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-28T03:44:45.063 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-28T03:44:45.063 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-28T03:44:45.063 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:45.063 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:45.063 INFO:teuthology.orchestra.run.smithi119.stdout: "throttle-write_buf_throttle-0x555636358780": { 2024-03-28T03:44:45.064 INFO:teuthology.orchestra.run.smithi119.stdout: "val": 0, 2024-03-28T03:44:45.064 INFO:teuthology.orchestra.run.smithi119.stdout: "max": 3758096384, 2024-03-28T03:44:45.064 INFO:teuthology.orchestra.run.smithi119.stdout: "get_started": 0, 2024-03-28T03:44:45.064 INFO:teuthology.orchestra.run.smithi119.stdout: "get": 0, 2024-03-28T03:44:45.064 INFO:teuthology.orchestra.run.smithi119.stdout: "get_sum": 0, 2024-03-28T03:44:45.064 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_fail": 0, 2024-03-28T03:44:45.064 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_success": 0, 2024-03-28T03:44:45.064 INFO:teuthology.orchestra.run.smithi119.stdout: "take": 0, 2024-03-28T03:44:45.064 INFO:teuthology.orchestra.run.smithi119.stdout: "take_sum": 0, 2024-03-28T03:44:45.064 INFO:teuthology.orchestra.run.smithi119.stdout: "put": 0, 2024-03-28T03:44:45.065 INFO:teuthology.orchestra.run.smithi119.stdout: "put_sum": 0, 2024-03-28T03:44:45.065 INFO:teuthology.orchestra.run.smithi119.stdout: "wait": { 2024-03-28T03:44:45.065 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-28T03:44:45.065 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-28T03:44:45.065 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-28T03:44:45.065 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:45.065 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:45.065 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-03-28T03:44:45.944 INFO:tasks.check_counter:Daemon mds.a mds.exported=0 2024-03-28T03:44:45.944 INFO:tasks.check_counter:Daemon mds.a mds.imported=0 2024-03-28T03:44:45.944 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update=0 2024-03-28T03:44:45.944 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update_receipt=0 2024-03-28T03:44:45.944 INFO:tasks.check_counter:Daemon mds.a mds.root_rsnaps=0 2024-03-28T03:44:45.945 INFO:tasks.check_counter:Daemon mds.a mds_server.req_mksnap_latency.avgcount=0 2024-03-28T03:44:45.945 INFO:tasks.check_counter:Daemon mds.a mds_server.req_rmsnap_latency.avgcount=0 2024-03-28T03:44:45.945 INFO:tasks.check_counter:Daemon mds.a mds.dir_split=0 2024-03-28T03:44:45.945 DEBUG:tasks.check_counter:Getting stats from d 2024-03-28T03:44:45.945 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 10f73b92-ec98-11ee-95d0-87774f69a715 -- ceph daemon mds.d perf dump 2024-03-28T03:44:46.396 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/mon.a/config 2024-03-28T03:44:47.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:44:46 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 11345.821724 secs 2024-03-28T03:44:47.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:44:46 smithi119 ceph-mon[31518]: pgmap v6197: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:44:47.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:44:46 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 11345.821724 secs 2024-03-28T03:44:47.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:44:46 smithi132 ceph-mon[35941]: pgmap v6197: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:44:47.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:44:46 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 11345.821724 secs 2024-03-28T03:44:47.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:44:46 smithi165 ceph-mon[36302]: pgmap v6197: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:44:48.775 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-03-28T03:44:48.776 INFO:teuthology.orchestra.run.smithi119.stdout: "AsyncMessenger::Worker-0": { 2024-03-28T03:44:48.776 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_messages": 14, 2024-03-28T03:44:48.776 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_messages": 15, 2024-03-28T03:44:48.776 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_bytes": 249525, 2024-03-28T03:44:48.777 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_bytes": 8959, 2024-03-28T03:44:48.777 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_created_connections": 4166, 2024-03-28T03:44:48.777 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_active_connections": 18446744073709547488, 2024-03-28T03:44:48.777 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_total_time": 4.802037734, 2024-03-28T03:44:48.777 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_send_time": 0.000467558, 2024-03-28T03:44:48.778 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_recv_time": 3.411751470, 2024-03-28T03:44:48.778 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_fast_dispatch_time": 0.000689144, 2024-03-28T03:44:48.778 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_messages_queue_lat": { 2024-03-28T03:44:48.778 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 13, 2024-03-28T03:44:48.779 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.010085647, 2024-03-28T03:44:48.779 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000775819 2024-03-28T03:44:48.779 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:48.779 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_handle_ack_lat": { 2024-03-28T03:44:48.779 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 4, 2024-03-28T03:44:48.780 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000924, 2024-03-28T03:44:48.780 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000231 2024-03-28T03:44:48.780 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:48.780 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-28T03:44:48.780 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_encrypted_bytes": 0 2024-03-28T03:44:48.781 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:48.781 INFO:teuthology.orchestra.run.smithi119.stdout: "AsyncMessenger::Worker-1": { 2024-03-28T03:44:48.781 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_messages": 2990, 2024-03-28T03:44:48.781 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_messages": 2983, 2024-03-28T03:44:48.782 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_bytes": 1043715, 2024-03-28T03:44:48.782 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_bytes": 1813486, 2024-03-28T03:44:48.782 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_created_connections": 55, 2024-03-28T03:44:48.782 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_active_connections": 18446744073709551607, 2024-03-28T03:44:48.783 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_total_time": 0.907344098, 2024-03-28T03:44:48.783 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_send_time": 0.347910451, 2024-03-28T03:44:48.783 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_recv_time": 0.274692034, 2024-03-28T03:44:48.783 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_fast_dispatch_time": 0.162621047, 2024-03-28T03:44:48.783 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_messages_queue_lat": { 2024-03-28T03:44:48.784 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 2983, 2024-03-28T03:44:48.784 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.300984583, 2024-03-28T03:44:48.784 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000100899 2024-03-28T03:44:48.784 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:48.785 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_handle_ack_lat": { 2024-03-28T03:44:48.785 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 4, 2024-03-28T03:44:48.785 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000001108, 2024-03-28T03:44:48.785 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000277 2024-03-28T03:44:48.785 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:48.786 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_encrypted_bytes": 1037072, 2024-03-28T03:44:48.786 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_encrypted_bytes": 1806640 2024-03-28T03:44:48.786 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:48.786 INFO:teuthology.orchestra.run.smithi119.stdout: "AsyncMessenger::Worker-2": { 2024-03-28T03:44:48.787 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_messages": 13, 2024-03-28T03:44:48.787 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_messages": 2285, 2024-03-28T03:44:48.787 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_bytes": 7414, 2024-03-28T03:44:48.787 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_bytes": 4731592, 2024-03-28T03:44:48.787 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_created_connections": 413, 2024-03-28T03:44:48.788 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_active_connections": 18446744073709551259, 2024-03-28T03:44:48.788 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_total_time": 0.771090043, 2024-03-28T03:44:48.788 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_send_time": 0.357315326, 2024-03-28T03:44:48.788 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_recv_time": 0.300748569, 2024-03-28T03:44:48.788 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_fast_dispatch_time": 0.000109214, 2024-03-28T03:44:48.789 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_messages_queue_lat": { 2024-03-28T03:44:48.789 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 2281, 2024-03-28T03:44:48.789 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.261787819, 2024-03-28T03:44:48.789 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000114768 2024-03-28T03:44:48.790 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:48.790 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_handle_ack_lat": { 2024-03-28T03:44:48.790 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 3, 2024-03-28T03:44:48.790 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000789, 2024-03-28T03:44:48.791 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000263 2024-03-28T03:44:48.791 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:48.791 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_encrypted_bytes": 1280, 2024-03-28T03:44:48.791 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_encrypted_bytes": 4725184 2024-03-28T03:44:48.791 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:48.792 INFO:teuthology.orchestra.run.smithi119.stdout: "cct": { 2024-03-28T03:44:48.792 INFO:teuthology.orchestra.run.smithi119.stdout: "total_workers": 1, 2024-03-28T03:44:48.792 INFO:teuthology.orchestra.run.smithi119.stdout: "unhealthy_workers": 0 2024-03-28T03:44:48.792 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:48.792 INFO:teuthology.orchestra.run.smithi119.stdout: "finisher-MDSRank": { 2024-03-28T03:44:48.793 INFO:teuthology.orchestra.run.smithi119.stdout: "queue_len": 0, 2024-03-28T03:44:48.793 INFO:teuthology.orchestra.run.smithi119.stdout: "complete_latency": { 2024-03-28T03:44:48.793 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 19, 2024-03-28T03:44:48.793 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.001091672, 2024-03-28T03:44:48.794 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000057456 2024-03-28T03:44:48.794 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:48.794 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:48.794 INFO:teuthology.orchestra.run.smithi119.stdout: "finisher-PurgeQueue": { 2024-03-28T03:44:48.794 INFO:teuthology.orchestra.run.smithi119.stdout: "queue_len": 0, 2024-03-28T03:44:48.795 INFO:teuthology.orchestra.run.smithi119.stdout: "complete_latency": { 2024-03-28T03:44:48.795 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 4, 2024-03-28T03:44:48.795 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000162229, 2024-03-28T03:44:48.795 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000040557 2024-03-28T03:44:48.796 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:48.796 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:48.796 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": { 2024-03-28T03:44:48.796 INFO:teuthology.orchestra.run.smithi119.stdout: "request": 0, 2024-03-28T03:44:48.796 INFO:teuthology.orchestra.run.smithi119.stdout: "reply": 0, 2024-03-28T03:44:48.797 INFO:teuthology.orchestra.run.smithi119.stdout: "reply_latency": { 2024-03-28T03:44:48.797 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-28T03:44:48.797 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-28T03:44:48.797 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-28T03:44:48.797 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:48.798 INFO:teuthology.orchestra.run.smithi119.stdout: "slow_reply": 0, 2024-03-28T03:44:48.798 INFO:teuthology.orchestra.run.smithi119.stdout: "forward": 0, 2024-03-28T03:44:48.798 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_fetch_complete": 0, 2024-03-28T03:44:48.798 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_fetch_keys": 0, 2024-03-28T03:44:48.799 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_commit": 0, 2024-03-28T03:44:48.799 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_split": 0, 2024-03-28T03:44:48.799 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_merge": 0, 2024-03-28T03:44:48.799 INFO:teuthology.orchestra.run.smithi119.stdout: "inodes": 164, 2024-03-28T03:44:48.799 INFO:teuthology.orchestra.run.smithi119.stdout: "inodes_top": 8, 2024-03-28T03:44:48.800 INFO:teuthology.orchestra.run.smithi119.stdout: "inodes_bottom": 156, 2024-03-28T03:44:48.800 INFO:teuthology.orchestra.run.smithi119.stdout: "inodes_pin_tail": 0, 2024-03-28T03:44:48.800 INFO:teuthology.orchestra.run.smithi119.stdout: "inodes_pinned": 153, 2024-03-28T03:44:48.800 INFO:teuthology.orchestra.run.smithi119.stdout: "inodes_expired": 0, 2024-03-28T03:44:48.801 INFO:teuthology.orchestra.run.smithi119.stdout: "inodes_with_caps": 0, 2024-03-28T03:44:48.801 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": 0, 2024-03-28T03:44:48.801 INFO:teuthology.orchestra.run.smithi119.stdout: "subtrees": 7, 2024-03-28T03:44:48.801 INFO:teuthology.orchestra.run.smithi119.stdout: "traverse": 0, 2024-03-28T03:44:48.801 INFO:teuthology.orchestra.run.smithi119.stdout: "traverse_hit": 0, 2024-03-28T03:44:48.802 INFO:teuthology.orchestra.run.smithi119.stdout: "traverse_forward": 0, 2024-03-28T03:44:48.802 INFO:teuthology.orchestra.run.smithi119.stdout: "traverse_discover": 0, 2024-03-28T03:44:48.802 INFO:teuthology.orchestra.run.smithi119.stdout: "traverse_dir_fetch": 0, 2024-03-28T03:44:48.802 INFO:teuthology.orchestra.run.smithi119.stdout: "traverse_remote_ino": 0, 2024-03-28T03:44:48.803 INFO:teuthology.orchestra.run.smithi119.stdout: "traverse_lock": 0, 2024-03-28T03:44:48.803 INFO:teuthology.orchestra.run.smithi119.stdout: "load_cent": 0, 2024-03-28T03:44:48.803 INFO:teuthology.orchestra.run.smithi119.stdout: "q": 0, 2024-03-28T03:44:48.803 INFO:teuthology.orchestra.run.smithi119.stdout: "exported": 0, 2024-03-28T03:44:48.803 INFO:teuthology.orchestra.run.smithi119.stdout: "exported_inodes": 0, 2024-03-28T03:44:48.804 INFO:teuthology.orchestra.run.smithi119.stdout: "imported": 0, 2024-03-28T03:44:48.804 INFO:teuthology.orchestra.run.smithi119.stdout: "imported_inodes": 0, 2024-03-28T03:44:48.804 INFO:teuthology.orchestra.run.smithi119.stdout: "openino_dir_fetch": 0, 2024-03-28T03:44:48.804 INFO:teuthology.orchestra.run.smithi119.stdout: "openino_backtrace_fetch": 0, 2024-03-28T03:44:48.804 INFO:teuthology.orchestra.run.smithi119.stdout: "openino_peer_discover": 0, 2024-03-28T03:44:48.805 INFO:teuthology.orchestra.run.smithi119.stdout: "root_rfiles": 0, 2024-03-28T03:44:48.805 INFO:teuthology.orchestra.run.smithi119.stdout: "root_rbytes": 0, 2024-03-28T03:44:48.805 INFO:teuthology.orchestra.run.smithi119.stdout: "root_rsnaps": 0, 2024-03-28T03:44:48.805 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_backtrace_fetch": 0, 2024-03-28T03:44:48.806 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_set_tag": 0, 2024-03-28T03:44:48.806 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_backtrace_repaired": 0, 2024-03-28T03:44:48.806 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_inotable_repaired": 0, 2024-03-28T03:44:48.806 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_dir_inodes": 0, 2024-03-28T03:44:48.807 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_dir_base_inodes": 0, 2024-03-28T03:44:48.807 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_dirfrag_rstats": 0, 2024-03-28T03:44:48.807 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_file_inodes": 0, 2024-03-28T03:44:48.807 INFO:teuthology.orchestra.run.smithi119.stdout: "handle_inode_file_caps": 0, 2024-03-28T03:44:48.807 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph_cap_op_revoke": 0, 2024-03-28T03:44:48.808 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph_cap_op_grant": 0, 2024-03-28T03:44:48.808 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph_cap_op_trunc": 0, 2024-03-28T03:44:48.808 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-28T03:44:48.808 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-28T03:44:48.809 INFO:teuthology.orchestra.run.smithi119.stdout: "handle_client_caps": 0, 2024-03-28T03:44:48.809 INFO:teuthology.orchestra.run.smithi119.stdout: "handle_client_caps_dirty": 0, 2024-03-28T03:44:48.809 INFO:teuthology.orchestra.run.smithi119.stdout: "handle_client_cap_release": 0, 2024-03-28T03:44:48.809 INFO:teuthology.orchestra.run.smithi119.stdout: "process_request_cap_release": 0 2024-03-28T03:44:48.809 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:48.810 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_cache": { 2024-03-28T03:44:48.810 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_update": 0, 2024-03-28T03:44:48.810 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_update_receipt": 0, 2024-03-28T03:44:48.810 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_try_discover": 0, 2024-03-28T03:44:48.811 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_send_discover": 0, 2024-03-28T03:44:48.811 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_handle_discover": 0, 2024-03-28T03:44:48.811 INFO:teuthology.orchestra.run.smithi119.stdout: "num_strays": 0, 2024-03-28T03:44:48.811 INFO:teuthology.orchestra.run.smithi119.stdout: "num_strays_delayed": 0, 2024-03-28T03:44:48.811 INFO:teuthology.orchestra.run.smithi119.stdout: "num_strays_enqueuing": 0, 2024-03-28T03:44:48.812 INFO:teuthology.orchestra.run.smithi119.stdout: "strays_created": 0, 2024-03-28T03:44:48.812 INFO:teuthology.orchestra.run.smithi119.stdout: "strays_enqueued": 0, 2024-03-28T03:44:48.812 INFO:teuthology.orchestra.run.smithi119.stdout: "strays_reintegrated": 0, 2024-03-28T03:44:48.812 INFO:teuthology.orchestra.run.smithi119.stdout: "strays_migrated": 0, 2024-03-28T03:44:48.812 INFO:teuthology.orchestra.run.smithi119.stdout: "num_recovering_processing": 0, 2024-03-28T03:44:48.813 INFO:teuthology.orchestra.run.smithi119.stdout: "num_recovering_enqueued": 0, 2024-03-28T03:44:48.813 INFO:teuthology.orchestra.run.smithi119.stdout: "num_recovering_prioritized": 0, 2024-03-28T03:44:48.813 INFO:teuthology.orchestra.run.smithi119.stdout: "recovery_started": 0, 2024-03-28T03:44:48.813 INFO:teuthology.orchestra.run.smithi119.stdout: "recovery_completed": 0, 2024-03-28T03:44:48.814 INFO:teuthology.orchestra.run.smithi119.stdout: "ireq_enqueue_scrub": 0, 2024-03-28T03:44:48.814 INFO:teuthology.orchestra.run.smithi119.stdout: "ireq_exportdir": 0, 2024-03-28T03:44:48.814 INFO:teuthology.orchestra.run.smithi119.stdout: "ireq_flush": 0, 2024-03-28T03:44:48.814 INFO:teuthology.orchestra.run.smithi119.stdout: "ireq_fragmentdir": 0, 2024-03-28T03:44:48.815 INFO:teuthology.orchestra.run.smithi119.stdout: "ireq_fragstats": 0, 2024-03-28T03:44:48.815 INFO:teuthology.orchestra.run.smithi119.stdout: "ireq_inodestats": 0 2024-03-28T03:44:48.815 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:48.815 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_log": { 2024-03-28T03:44:48.815 INFO:teuthology.orchestra.run.smithi119.stdout: "evlrg": 0, 2024-03-28T03:44:48.816 INFO:teuthology.orchestra.run.smithi119.stdout: "evadd": 0, 2024-03-28T03:44:48.816 INFO:teuthology.orchestra.run.smithi119.stdout: "evex": 0, 2024-03-28T03:44:48.816 INFO:teuthology.orchestra.run.smithi119.stdout: "evtrm": 0, 2024-03-28T03:44:48.816 INFO:teuthology.orchestra.run.smithi119.stdout: "ev": 87, 2024-03-28T03:44:48.816 INFO:teuthology.orchestra.run.smithi119.stdout: "evexg": 0, 2024-03-28T03:44:48.817 INFO:teuthology.orchestra.run.smithi119.stdout: "evexd": 0, 2024-03-28T03:44:48.817 INFO:teuthology.orchestra.run.smithi119.stdout: "segadd": 0, 2024-03-28T03:44:48.817 INFO:teuthology.orchestra.run.smithi119.stdout: "segex": 0, 2024-03-28T03:44:48.817 INFO:teuthology.orchestra.run.smithi119.stdout: "segtrm": 0, 2024-03-28T03:44:48.817 INFO:teuthology.orchestra.run.smithi119.stdout: "seg": 2, 2024-03-28T03:44:48.818 INFO:teuthology.orchestra.run.smithi119.stdout: "segmjr": 2, 2024-03-28T03:44:48.818 INFO:teuthology.orchestra.run.smithi119.stdout: "segexg": 0, 2024-03-28T03:44:48.818 INFO:teuthology.orchestra.run.smithi119.stdout: "segexd": 0, 2024-03-28T03:44:48.818 INFO:teuthology.orchestra.run.smithi119.stdout: "expos": 4194304, 2024-03-28T03:44:48.819 INFO:teuthology.orchestra.run.smithi119.stdout: "wrpos": 4434484, 2024-03-28T03:44:48.819 INFO:teuthology.orchestra.run.smithi119.stdout: "rdpos": 4434316, 2024-03-28T03:44:48.819 INFO:teuthology.orchestra.run.smithi119.stdout: "jlat": { 2024-03-28T03:44:48.819 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-28T03:44:48.819 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-28T03:44:48.820 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-28T03:44:48.820 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:48.820 INFO:teuthology.orchestra.run.smithi119.stdout: "replayed": 87 2024-03-28T03:44:48.820 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:48.821 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_mem": { 2024-03-28T03:44:48.821 INFO:teuthology.orchestra.run.smithi119.stdout: "ino": 167, 2024-03-28T03:44:48.821 INFO:teuthology.orchestra.run.smithi119.stdout: "ino+": 167, 2024-03-28T03:44:48.821 INFO:teuthology.orchestra.run.smithi119.stdout: "ino-": 0, 2024-03-28T03:44:48.822 INFO:teuthology.orchestra.run.smithi119.stdout: "dir": 18, 2024-03-28T03:44:48.822 INFO:teuthology.orchestra.run.smithi119.stdout: "dir+": 18, 2024-03-28T03:44:48.822 INFO:teuthology.orchestra.run.smithi119.stdout: "dir-": 0, 2024-03-28T03:44:48.822 INFO:teuthology.orchestra.run.smithi119.stdout: "dn": 164, 2024-03-28T03:44:48.822 INFO:teuthology.orchestra.run.smithi119.stdout: "dn+": 164, 2024-03-28T03:44:48.823 INFO:teuthology.orchestra.run.smithi119.stdout: "dn-": 0, 2024-03-28T03:44:48.823 INFO:teuthology.orchestra.run.smithi119.stdout: "cap": 0, 2024-03-28T03:44:48.823 INFO:teuthology.orchestra.run.smithi119.stdout: "cap+": 0, 2024-03-28T03:44:48.823 INFO:teuthology.orchestra.run.smithi119.stdout: "cap-": 0, 2024-03-28T03:44:48.823 INFO:teuthology.orchestra.run.smithi119.stdout: "rss": 49664, 2024-03-28T03:44:48.824 INFO:teuthology.orchestra.run.smithi119.stdout: "heap": 215420 2024-03-28T03:44:48.824 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:48.824 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_server": { 2024-03-28T03:44:48.824 INFO:teuthology.orchestra.run.smithi119.stdout: "dispatch_client_request": 0, 2024-03-28T03:44:48.824 INFO:teuthology.orchestra.run.smithi119.stdout: "dispatch_server_request": 0, 2024-03-28T03:44:48.825 INFO:teuthology.orchestra.run.smithi119.stdout: "handle_client_request": 0, 2024-03-28T03:44:48.825 INFO:teuthology.orchestra.run.smithi119.stdout: "handle_client_session": 0, 2024-03-28T03:44:48.825 INFO:teuthology.orchestra.run.smithi119.stdout: "handle_peer_request": 0, 2024-03-28T03:44:48.825 INFO:teuthology.orchestra.run.smithi119.stdout: "req_create_latency": { 2024-03-28T03:44:48.826 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-28T03:44:48.826 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-28T03:44:48.826 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-28T03:44:48.826 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:48.827 INFO:teuthology.orchestra.run.smithi119.stdout: "req_getattr_latency": { 2024-03-28T03:44:48.827 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-28T03:44:48.827 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-28T03:44:48.827 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-28T03:44:48.827 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:48.828 INFO:teuthology.orchestra.run.smithi119.stdout: "req_getfilelock_latency": { 2024-03-28T03:44:48.828 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-28T03:44:48.828 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-28T03:44:48.828 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-28T03:44:48.828 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:48.829 INFO:teuthology.orchestra.run.smithi119.stdout: "req_link_latency": { 2024-03-28T03:44:48.829 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-28T03:44:48.829 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-28T03:44:48.829 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-28T03:44:48.830 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:48.830 INFO:teuthology.orchestra.run.smithi119.stdout: "req_lookup_latency": { 2024-03-28T03:44:48.830 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-28T03:44:48.830 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-28T03:44:48.830 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-28T03:44:48.831 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:48.831 INFO:teuthology.orchestra.run.smithi119.stdout: "req_lookuphash_latency": { 2024-03-28T03:44:48.831 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-28T03:44:48.831 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-28T03:44:48.832 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-28T03:44:48.832 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:48.832 INFO:teuthology.orchestra.run.smithi119.stdout: "req_lookupino_latency": { 2024-03-28T03:44:48.832 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-28T03:44:48.832 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-28T03:44:48.833 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-28T03:44:48.833 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:48.833 INFO:teuthology.orchestra.run.smithi119.stdout: "req_lookupname_latency": { 2024-03-28T03:44:48.833 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-28T03:44:48.834 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-28T03:44:48.834 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-28T03:44:48.834 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:48.834 INFO:teuthology.orchestra.run.smithi119.stdout: "req_lookupparent_latency": { 2024-03-28T03:44:48.835 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-28T03:44:48.835 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-28T03:44:48.835 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-28T03:44:48.835 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:48.835 INFO:teuthology.orchestra.run.smithi119.stdout: "req_lookupsnap_latency": { 2024-03-28T03:44:48.836 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-28T03:44:48.836 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-28T03:44:48.836 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-28T03:44:48.836 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:48.837 INFO:teuthology.orchestra.run.smithi119.stdout: "req_lssnap_latency": { 2024-03-28T03:44:48.837 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-28T03:44:48.837 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-28T03:44:48.837 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-28T03:44:48.837 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:48.838 INFO:teuthology.orchestra.run.smithi119.stdout: "req_mkdir_latency": { 2024-03-28T03:44:48.838 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-28T03:44:48.839 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-28T03:44:48.839 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-28T03:44:48.839 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:48.839 INFO:teuthology.orchestra.run.smithi119.stdout: "req_mknod_latency": { 2024-03-28T03:44:48.840 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-28T03:44:48.840 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-28T03:44:48.840 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-28T03:44:48.840 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:48.840 INFO:teuthology.orchestra.run.smithi119.stdout: "req_mksnap_latency": { 2024-03-28T03:44:48.840 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-28T03:44:48.840 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-28T03:44:48.840 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-28T03:44:48.841 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:48.841 INFO:teuthology.orchestra.run.smithi119.stdout: "req_open_latency": { 2024-03-28T03:44:48.841 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-28T03:44:48.841 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-28T03:44:48.841 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-28T03:44:48.841 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:48.841 INFO:teuthology.orchestra.run.smithi119.stdout: "req_readdir_latency": { 2024-03-28T03:44:48.841 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-28T03:44:48.841 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-28T03:44:48.841 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-28T03:44:48.842 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:48.842 INFO:teuthology.orchestra.run.smithi119.stdout: "req_rename_latency": { 2024-03-28T03:44:48.842 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-28T03:44:48.842 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-28T03:44:48.842 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-28T03:44:48.842 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:48.842 INFO:teuthology.orchestra.run.smithi119.stdout: "req_renamesnap_latency": { 2024-03-28T03:44:48.842 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-28T03:44:48.843 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-28T03:44:48.843 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-28T03:44:48.843 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:48.843 INFO:teuthology.orchestra.run.smithi119.stdout: "req_snapdiff_latency": { 2024-03-28T03:44:48.843 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-28T03:44:48.843 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-28T03:44:48.843 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-28T03:44:48.843 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:48.843 INFO:teuthology.orchestra.run.smithi119.stdout: "req_rmdir_latency": { 2024-03-28T03:44:48.844 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-28T03:44:48.844 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-28T03:44:48.844 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-28T03:44:48.844 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:48.844 INFO:teuthology.orchestra.run.smithi119.stdout: "req_rmsnap_latency": { 2024-03-28T03:44:48.844 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-28T03:44:48.844 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-28T03:44:48.844 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-28T03:44:48.844 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:48.845 INFO:teuthology.orchestra.run.smithi119.stdout: "req_rmxattr_latency": { 2024-03-28T03:44:48.845 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-28T03:44:48.845 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-28T03:44:48.845 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-28T03:44:48.845 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:48.845 INFO:teuthology.orchestra.run.smithi119.stdout: "req_setattr_latency": { 2024-03-28T03:44:48.845 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-28T03:44:48.845 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-28T03:44:48.845 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-28T03:44:48.846 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:48.846 INFO:teuthology.orchestra.run.smithi119.stdout: "req_setdirlayout_latency": { 2024-03-28T03:44:48.846 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-28T03:44:48.846 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-28T03:44:48.846 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-28T03:44:48.846 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:48.846 INFO:teuthology.orchestra.run.smithi119.stdout: "req_setfilelock_latency": { 2024-03-28T03:44:48.847 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-28T03:44:48.847 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-28T03:44:48.847 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-28T03:44:48.847 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:48.847 INFO:teuthology.orchestra.run.smithi119.stdout: "req_setlayout_latency": { 2024-03-28T03:44:48.847 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-28T03:44:48.847 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-28T03:44:48.847 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-28T03:44:48.848 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:48.848 INFO:teuthology.orchestra.run.smithi119.stdout: "req_setxattr_latency": { 2024-03-28T03:44:48.848 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-28T03:44:48.848 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-28T03:44:48.848 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-28T03:44:48.848 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:48.848 INFO:teuthology.orchestra.run.smithi119.stdout: "req_symlink_latency": { 2024-03-28T03:44:48.848 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-28T03:44:48.848 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-28T03:44:48.849 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-28T03:44:48.849 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:48.849 INFO:teuthology.orchestra.run.smithi119.stdout: "req_unlink_latency": { 2024-03-28T03:44:48.849 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-28T03:44:48.849 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-28T03:44:48.849 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-28T03:44:48.849 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:48.849 INFO:teuthology.orchestra.run.smithi119.stdout: "cap_revoke_eviction": 0, 2024-03-28T03:44:48.850 INFO:teuthology.orchestra.run.smithi119.stdout: "cap_acquisition_throttle": 0, 2024-03-28T03:44:48.850 INFO:teuthology.orchestra.run.smithi119.stdout: "req_getvxattr_latency": { 2024-03-28T03:44:48.850 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-28T03:44:48.850 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-28T03:44:48.850 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-28T03:44:48.850 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:48.850 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:48.850 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_sessions": { 2024-03-28T03:44:48.850 INFO:teuthology.orchestra.run.smithi119.stdout: "session_count": 2, 2024-03-28T03:44:48.851 INFO:teuthology.orchestra.run.smithi119.stdout: "session_add": 3, 2024-03-28T03:44:48.851 INFO:teuthology.orchestra.run.smithi119.stdout: "session_remove": 1, 2024-03-28T03:44:48.851 INFO:teuthology.orchestra.run.smithi119.stdout: "sessions_open": 2, 2024-03-28T03:44:48.851 INFO:teuthology.orchestra.run.smithi119.stdout: "sessions_stale": 0, 2024-03-28T03:44:48.851 INFO:teuthology.orchestra.run.smithi119.stdout: "total_load": 0, 2024-03-28T03:44:48.851 INFO:teuthology.orchestra.run.smithi119.stdout: "average_load": 0, 2024-03-28T03:44:48.851 INFO:teuthology.orchestra.run.smithi119.stdout: "avg_session_uptime": 13206, 2024-03-28T03:44:48.851 INFO:teuthology.orchestra.run.smithi119.stdout: "mdthresh_evicted": 0 2024-03-28T03:44:48.851 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:48.852 INFO:teuthology.orchestra.run.smithi119.stdout: "mempool": { 2024-03-28T03:44:48.852 INFO:teuthology.orchestra.run.smithi119.stdout: "bloom_filter_bytes": 0, 2024-03-28T03:44:48.852 INFO:teuthology.orchestra.run.smithi119.stdout: "bloom_filter_items": 0, 2024-03-28T03:44:48.852 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_alloc_bytes": 0, 2024-03-28T03:44:48.852 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_alloc_items": 0, 2024-03-28T03:44:48.852 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_data_bytes": 0, 2024-03-28T03:44:48.852 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_data_items": 0, 2024-03-28T03:44:48.852 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-28T03:44:48.852 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_onode_items": 0, 2024-03-28T03:44:48.853 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-28T03:44:48.853 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_meta_items": 0, 2024-03-28T03:44:48.853 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_other_bytes": 0, 2024-03-28T03:44:48.853 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_other_items": 0, 2024-03-28T03:44:48.853 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-28T03:44:48.853 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_buffer_items": 0, 2024-03-28T03:44:48.853 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_extent_bytes": 0, 2024-03-28T03:44:48.853 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_extent_items": 0, 2024-03-28T03:44:48.853 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_blob_bytes": 0, 2024-03-28T03:44:48.854 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_blob_items": 0, 2024-03-28T03:44:48.854 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-28T03:44:48.854 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_shared_blob_items": 0, 2024-03-28T03:44:48.854 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-28T03:44:48.854 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_inline_bl_items": 0, 2024-03-28T03:44:48.854 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_fsck_bytes": 0, 2024-03-28T03:44:48.854 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_fsck_items": 0, 2024-03-28T03:44:48.854 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_txc_bytes": 0, 2024-03-28T03:44:48.855 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_txc_items": 0, 2024-03-28T03:44:48.855 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-28T03:44:48.855 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_writing_deferred_items": 0, 2024-03-28T03:44:48.855 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_writing_bytes": 0, 2024-03-28T03:44:48.855 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_writing_items": 0, 2024-03-28T03:44:48.855 INFO:teuthology.orchestra.run.smithi119.stdout: "bluefs_bytes": 0, 2024-03-28T03:44:48.855 INFO:teuthology.orchestra.run.smithi119.stdout: "bluefs_items": 0, 2024-03-28T03:44:48.855 INFO:teuthology.orchestra.run.smithi119.stdout: "bluefs_file_reader_bytes": 0, 2024-03-28T03:44:48.855 INFO:teuthology.orchestra.run.smithi119.stdout: "bluefs_file_reader_items": 0, 2024-03-28T03:44:48.856 INFO:teuthology.orchestra.run.smithi119.stdout: "bluefs_file_writer_bytes": 0, 2024-03-28T03:44:48.856 INFO:teuthology.orchestra.run.smithi119.stdout: "bluefs_file_writer_items": 0, 2024-03-28T03:44:48.856 INFO:teuthology.orchestra.run.smithi119.stdout: "buffer_anon_bytes": 233288, 2024-03-28T03:44:48.856 INFO:teuthology.orchestra.run.smithi119.stdout: "buffer_anon_items": 240, 2024-03-28T03:44:48.856 INFO:teuthology.orchestra.run.smithi119.stdout: "buffer_meta_bytes": 0, 2024-03-28T03:44:48.856 INFO:teuthology.orchestra.run.smithi119.stdout: "buffer_meta_items": 0, 2024-03-28T03:44:48.856 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_bytes": 0, 2024-03-28T03:44:48.856 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_items": 0, 2024-03-28T03:44:48.857 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_mapbl_bytes": 0, 2024-03-28T03:44:48.857 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_mapbl_items": 0, 2024-03-28T03:44:48.857 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_pglog_bytes": 0, 2024-03-28T03:44:48.857 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_pglog_items": 0, 2024-03-28T03:44:48.857 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_bytes": 13016, 2024-03-28T03:44:48.857 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_items": 374, 2024-03-28T03:44:48.857 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_mapping_bytes": 0, 2024-03-28T03:44:48.857 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_mapping_items": 0, 2024-03-28T03:44:48.857 INFO:teuthology.orchestra.run.smithi119.stdout: "pgmap_bytes": 0, 2024-03-28T03:44:48.858 INFO:teuthology.orchestra.run.smithi119.stdout: "pgmap_items": 0, 2024-03-28T03:44:48.858 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_co_bytes": 514617, 2024-03-28T03:44:48.858 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_co_items": 4415, 2024-03-28T03:44:48.858 INFO:teuthology.orchestra.run.smithi119.stdout: "unittest_1_bytes": 0, 2024-03-28T03:44:48.858 INFO:teuthology.orchestra.run.smithi119.stdout: "unittest_1_items": 0, 2024-03-28T03:44:48.858 INFO:teuthology.orchestra.run.smithi119.stdout: "unittest_2_bytes": 0, 2024-03-28T03:44:48.858 INFO:teuthology.orchestra.run.smithi119.stdout: "unittest_2_items": 0 2024-03-28T03:44:48.858 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:48.859 INFO:teuthology.orchestra.run.smithi119.stdout: "objecter": { 2024-03-28T03:44:48.859 INFO:teuthology.orchestra.run.smithi119.stdout: "op_active": 0, 2024-03-28T03:44:48.859 INFO:teuthology.orchestra.run.smithi119.stdout: "op_laggy": 0, 2024-03-28T03:44:48.859 INFO:teuthology.orchestra.run.smithi119.stdout: "op_send": 18, 2024-03-28T03:44:48.859 INFO:teuthology.orchestra.run.smithi119.stdout: "op_send_bytes": 37, 2024-03-28T03:44:48.859 INFO:teuthology.orchestra.run.smithi119.stdout: "op_resend": 0, 2024-03-28T03:44:48.859 INFO:teuthology.orchestra.run.smithi119.stdout: "op_reply": 18, 2024-03-28T03:44:48.859 INFO:teuthology.orchestra.run.smithi119.stdout: "op_latency": { 2024-03-28T03:44:48.860 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 18, 2024-03-28T03:44:48.860 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.038999054, 2024-03-28T03:44:48.860 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.002166614 2024-03-28T03:44:48.860 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:48.860 INFO:teuthology.orchestra.run.smithi119.stdout: "op_inflight": 0, 2024-03-28T03:44:48.860 INFO:teuthology.orchestra.run.smithi119.stdout: "oplen_avg": { 2024-03-28T03:44:48.860 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 18, 2024-03-28T03:44:48.861 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 20 2024-03-28T03:44:48.861 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:48.861 INFO:teuthology.orchestra.run.smithi119.stdout: "op": 18, 2024-03-28T03:44:48.861 INFO:teuthology.orchestra.run.smithi119.stdout: "op_r": 12, 2024-03-28T03:44:48.861 INFO:teuthology.orchestra.run.smithi119.stdout: "op_w": 6, 2024-03-28T03:44:48.861 INFO:teuthology.orchestra.run.smithi119.stdout: "op_rmw": 0, 2024-03-28T03:44:48.861 INFO:teuthology.orchestra.run.smithi119.stdout: "op_pg": 0, 2024-03-28T03:44:48.862 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_stat": 3, 2024-03-28T03:44:48.862 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_create": 0, 2024-03-28T03:44:48.862 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_read": 6, 2024-03-28T03:44:48.862 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_write": 0, 2024-03-28T03:44:48.862 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_writefull": 0, 2024-03-28T03:44:48.862 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_writesame": 0, 2024-03-28T03:44:48.862 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_append": 0, 2024-03-28T03:44:48.863 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_zero": 1, 2024-03-28T03:44:48.863 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_truncate": 0, 2024-03-28T03:44:48.863 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_delete": 5, 2024-03-28T03:44:48.863 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_mapext": 0, 2024-03-28T03:44:48.863 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_sparse_read": 0, 2024-03-28T03:44:48.863 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_clonerange": 0, 2024-03-28T03:44:48.864 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_getxattr": 1, 2024-03-28T03:44:48.864 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_setxattr": 0, 2024-03-28T03:44:48.864 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_cmpxattr": 0, 2024-03-28T03:44:48.864 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_rmxattr": 0, 2024-03-28T03:44:48.864 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_resetxattrs": 0, 2024-03-28T03:44:48.864 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_call": 0, 2024-03-28T03:44:48.864 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_watch": 0, 2024-03-28T03:44:48.865 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_notify": 0, 2024-03-28T03:44:48.865 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_src_cmpxattr": 0, 2024-03-28T03:44:48.865 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_pgls": 0, 2024-03-28T03:44:48.865 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_pgls_filter": 0, 2024-03-28T03:44:48.865 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_other": 0, 2024-03-28T03:44:48.865 INFO:teuthology.orchestra.run.smithi119.stdout: "linger_active": 0, 2024-03-28T03:44:48.865 INFO:teuthology.orchestra.run.smithi119.stdout: "linger_send": 0, 2024-03-28T03:44:48.865 INFO:teuthology.orchestra.run.smithi119.stdout: "linger_resend": 0, 2024-03-28T03:44:48.865 INFO:teuthology.orchestra.run.smithi119.stdout: "linger_ping": 0, 2024-03-28T03:44:48.866 INFO:teuthology.orchestra.run.smithi119.stdout: "poolop_active": 0, 2024-03-28T03:44:48.866 INFO:teuthology.orchestra.run.smithi119.stdout: "poolop_send": 0, 2024-03-28T03:44:48.866 INFO:teuthology.orchestra.run.smithi119.stdout: "poolop_resend": 0, 2024-03-28T03:44:48.866 INFO:teuthology.orchestra.run.smithi119.stdout: "poolstat_active": 0, 2024-03-28T03:44:48.866 INFO:teuthology.orchestra.run.smithi119.stdout: "poolstat_send": 0, 2024-03-28T03:44:48.866 INFO:teuthology.orchestra.run.smithi119.stdout: "poolstat_resend": 0, 2024-03-28T03:44:48.866 INFO:teuthology.orchestra.run.smithi119.stdout: "statfs_active": 0, 2024-03-28T03:44:48.866 INFO:teuthology.orchestra.run.smithi119.stdout: "statfs_send": 0, 2024-03-28T03:44:48.867 INFO:teuthology.orchestra.run.smithi119.stdout: "statfs_resend": 0, 2024-03-28T03:44:48.867 INFO:teuthology.orchestra.run.smithi119.stdout: "command_active": 0, 2024-03-28T03:44:48.867 INFO:teuthology.orchestra.run.smithi119.stdout: "command_send": 0, 2024-03-28T03:44:48.867 INFO:teuthology.orchestra.run.smithi119.stdout: "command_resend": 0, 2024-03-28T03:44:48.867 INFO:teuthology.orchestra.run.smithi119.stdout: "map_epoch": 0, 2024-03-28T03:44:48.867 INFO:teuthology.orchestra.run.smithi119.stdout: "map_full": 0, 2024-03-28T03:44:48.867 INFO:teuthology.orchestra.run.smithi119.stdout: "map_inc": 0, 2024-03-28T03:44:48.867 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_sessions": 8, 2024-03-28T03:44:48.867 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_session_open": 101, 2024-03-28T03:44:48.868 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_session_close": 93, 2024-03-28T03:44:48.868 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_laggy": 0, 2024-03-28T03:44:48.868 INFO:teuthology.orchestra.run.smithi119.stdout: "omap_wr": 0, 2024-03-28T03:44:48.868 INFO:teuthology.orchestra.run.smithi119.stdout: "omap_rd": 4, 2024-03-28T03:44:48.868 INFO:teuthology.orchestra.run.smithi119.stdout: "omap_del": 0 2024-03-28T03:44:48.868 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:48.869 INFO:teuthology.orchestra.run.smithi119.stdout: "oft": { 2024-03-28T03:44:48.869 INFO:teuthology.orchestra.run.smithi119.stdout: "omap_total_objs": 0, 2024-03-28T03:44:48.869 INFO:teuthology.orchestra.run.smithi119.stdout: "omap_total_kv_pairs": 0, 2024-03-28T03:44:48.869 INFO:teuthology.orchestra.run.smithi119.stdout: "omap_total_updates": 0, 2024-03-28T03:44:48.869 INFO:teuthology.orchestra.run.smithi119.stdout: "omap_total_removes": 0 2024-03-28T03:44:48.869 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:48.869 INFO:teuthology.orchestra.run.smithi119.stdout: "osd-slow-ops": { 2024-03-28T03:44:48.869 INFO:teuthology.orchestra.run.smithi119.stdout: "slow_ops_count": 0 2024-03-28T03:44:48.870 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:48.870 INFO:teuthology.orchestra.run.smithi119.stdout: "purge_queue": { 2024-03-28T03:44:48.870 INFO:teuthology.orchestra.run.smithi119.stdout: "pq_executing_ops": 0, 2024-03-28T03:44:48.870 INFO:teuthology.orchestra.run.smithi119.stdout: "pq_executing_ops_high_water": 0, 2024-03-28T03:44:48.870 INFO:teuthology.orchestra.run.smithi119.stdout: "pq_executing": 0, 2024-03-28T03:44:48.870 INFO:teuthology.orchestra.run.smithi119.stdout: "pq_executing_high_water": 0, 2024-03-28T03:44:48.870 INFO:teuthology.orchestra.run.smithi119.stdout: "pq_executed_ops": 0, 2024-03-28T03:44:48.870 INFO:teuthology.orchestra.run.smithi119.stdout: "pq_executed": 0, 2024-03-28T03:44:48.871 INFO:teuthology.orchestra.run.smithi119.stdout: "pq_item_in_journal": 0 2024-03-28T03:44:48.871 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:48.871 INFO:teuthology.orchestra.run.smithi119.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-28T03:44:48.871 INFO:teuthology.orchestra.run.smithi119.stdout: "val": 0, 2024-03-28T03:44:48.871 INFO:teuthology.orchestra.run.smithi119.stdout: "max": 104857600, 2024-03-28T03:44:48.871 INFO:teuthology.orchestra.run.smithi119.stdout: "get_started": 0, 2024-03-28T03:44:48.871 INFO:teuthology.orchestra.run.smithi119.stdout: "get": 3013, 2024-03-28T03:44:48.871 INFO:teuthology.orchestra.run.smithi119.stdout: "get_sum": 872212, 2024-03-28T03:44:48.871 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_fail": 0, 2024-03-28T03:44:48.872 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_success": 3013, 2024-03-28T03:44:48.872 INFO:teuthology.orchestra.run.smithi119.stdout: "take": 0, 2024-03-28T03:44:48.872 INFO:teuthology.orchestra.run.smithi119.stdout: "take_sum": 0, 2024-03-28T03:44:48.872 INFO:teuthology.orchestra.run.smithi119.stdout: "put": 3013, 2024-03-28T03:44:48.872 INFO:teuthology.orchestra.run.smithi119.stdout: "put_sum": 872212, 2024-03-28T03:44:48.872 INFO:teuthology.orchestra.run.smithi119.stdout: "wait": { 2024-03-28T03:44:48.872 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-28T03:44:48.872 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-28T03:44:48.873 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-28T03:44:48.873 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:48.873 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:48.873 INFO:teuthology.orchestra.run.smithi119.stdout: "throttle-objecter_bytes": { 2024-03-28T03:44:48.873 INFO:teuthology.orchestra.run.smithi119.stdout: "val": 0, 2024-03-28T03:44:48.873 INFO:teuthology.orchestra.run.smithi119.stdout: "max": 104857600, 2024-03-28T03:44:48.873 INFO:teuthology.orchestra.run.smithi119.stdout: "get_started": 0, 2024-03-28T03:44:48.873 INFO:teuthology.orchestra.run.smithi119.stdout: "get": 0, 2024-03-28T03:44:48.873 INFO:teuthology.orchestra.run.smithi119.stdout: "get_sum": 0, 2024-03-28T03:44:48.874 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_fail": 0, 2024-03-28T03:44:48.874 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_success": 0, 2024-03-28T03:44:48.874 INFO:teuthology.orchestra.run.smithi119.stdout: "take": 18, 2024-03-28T03:44:48.874 INFO:teuthology.orchestra.run.smithi119.stdout: "take_sum": 240185, 2024-03-28T03:44:48.874 INFO:teuthology.orchestra.run.smithi119.stdout: "put": 18, 2024-03-28T03:44:48.874 INFO:teuthology.orchestra.run.smithi119.stdout: "put_sum": 240185, 2024-03-28T03:44:48.874 INFO:teuthology.orchestra.run.smithi119.stdout: "wait": { 2024-03-28T03:44:48.874 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-28T03:44:48.874 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-28T03:44:48.875 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-28T03:44:48.875 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:48.875 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:48.875 INFO:teuthology.orchestra.run.smithi119.stdout: "throttle-objecter_ops": { 2024-03-28T03:44:48.875 INFO:teuthology.orchestra.run.smithi119.stdout: "val": 0, 2024-03-28T03:44:48.875 INFO:teuthology.orchestra.run.smithi119.stdout: "max": 1024, 2024-03-28T03:44:48.875 INFO:teuthology.orchestra.run.smithi119.stdout: "get_started": 0, 2024-03-28T03:44:48.875 INFO:teuthology.orchestra.run.smithi119.stdout: "get": 0, 2024-03-28T03:44:48.876 INFO:teuthology.orchestra.run.smithi119.stdout: "get_sum": 0, 2024-03-28T03:44:48.876 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_fail": 0, 2024-03-28T03:44:48.876 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_success": 0, 2024-03-28T03:44:48.876 INFO:teuthology.orchestra.run.smithi119.stdout: "take": 18, 2024-03-28T03:44:48.876 INFO:teuthology.orchestra.run.smithi119.stdout: "take_sum": 18, 2024-03-28T03:44:48.876 INFO:teuthology.orchestra.run.smithi119.stdout: "put": 18, 2024-03-28T03:44:48.876 INFO:teuthology.orchestra.run.smithi119.stdout: "put_sum": 18, 2024-03-28T03:44:48.876 INFO:teuthology.orchestra.run.smithi119.stdout: "wait": { 2024-03-28T03:44:48.876 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-28T03:44:48.876 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-28T03:44:48.877 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-28T03:44:48.877 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:48.877 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:48.877 INFO:teuthology.orchestra.run.smithi119.stdout: "throttle-write_buf_throttle": { 2024-03-28T03:44:48.877 INFO:teuthology.orchestra.run.smithi119.stdout: "val": 0, 2024-03-28T03:44:48.877 INFO:teuthology.orchestra.run.smithi119.stdout: "max": 3758096384, 2024-03-28T03:44:48.877 INFO:teuthology.orchestra.run.smithi119.stdout: "get_started": 0, 2024-03-28T03:44:48.877 INFO:teuthology.orchestra.run.smithi119.stdout: "get": 0, 2024-03-28T03:44:48.878 INFO:teuthology.orchestra.run.smithi119.stdout: "get_sum": 0, 2024-03-28T03:44:48.878 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_fail": 0, 2024-03-28T03:44:48.878 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_success": 0, 2024-03-28T03:44:48.878 INFO:teuthology.orchestra.run.smithi119.stdout: "take": 0, 2024-03-28T03:44:48.878 INFO:teuthology.orchestra.run.smithi119.stdout: "take_sum": 0, 2024-03-28T03:44:48.878 INFO:teuthology.orchestra.run.smithi119.stdout: "put": 0, 2024-03-28T03:44:48.878 INFO:teuthology.orchestra.run.smithi119.stdout: "put_sum": 0, 2024-03-28T03:44:48.878 INFO:teuthology.orchestra.run.smithi119.stdout: "wait": { 2024-03-28T03:44:48.878 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-28T03:44:48.879 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-28T03:44:48.879 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-28T03:44:48.879 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:48.879 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-03-28T03:44:48.879 INFO:teuthology.orchestra.run.smithi119.stdout: "throttle-write_buf_throttle-0x555d1a9ec820": { 2024-03-28T03:44:48.879 INFO:teuthology.orchestra.run.smithi119.stdout: "val": 0, 2024-03-28T03:44:48.879 INFO:teuthology.orchestra.run.smithi119.stdout: "max": 3758096384, 2024-03-28T03:44:48.879 INFO:teuthology.orchestra.run.smithi119.stdout: "get_started": 0, 2024-03-28T03:44:48.879 INFO:teuthology.orchestra.run.smithi119.stdout: "get": 0, 2024-03-28T03:44:48.880 INFO:teuthology.orchestra.run.smithi119.stdout: "get_sum": 0, 2024-03-28T03:44:48.880 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_fail": 0, 2024-03-28T03:44:48.880 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_success": 0, 2024-03-28T03:44:48.880 INFO:teuthology.orchestra.run.smithi119.stdout: "take": 0, 2024-03-28T03:44:48.880 INFO:teuthology.orchestra.run.smithi119.stdout: "take_sum": 0, 2024-03-28T03:44:48.880 INFO:teuthology.orchestra.run.smithi119.stdout: "put": 0, 2024-03-28T03:44:48.880 INFO:teuthology.orchestra.run.smithi119.stdout: "put_sum": 0, 2024-03-28T03:44:48.880 INFO:teuthology.orchestra.run.smithi119.stdout: "wait": { 2024-03-28T03:44:48.881 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-03-28T03:44:48.881 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-03-28T03:44:48.881 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-03-28T03:44:48.881 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:48.881 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-03-28T03:44:48.881 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-03-28T03:44:49.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:44:48 smithi132 ceph-mon[35941]: pgmap v6198: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:44:49.328 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:44:48 smithi119 ceph-mon[31518]: pgmap v6198: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:44:49.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:44:48 smithi165 ceph-mon[36302]: pgmap v6198: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:44:49.754 INFO:tasks.check_counter:Daemon mds.d mds.exported=0 2024-03-28T03:44:49.755 INFO:tasks.check_counter:Daemon mds.d mds.imported=0 2024-03-28T03:44:49.755 INFO:tasks.check_counter:Daemon mds.d mds_cache.dir_update=0 2024-03-28T03:44:49.755 INFO:tasks.check_counter:Daemon mds.d mds_cache.dir_update_receipt=0 2024-03-28T03:44:49.755 INFO:tasks.check_counter:Daemon mds.d mds.root_rsnaps=0 2024-03-28T03:44:49.755 INFO:tasks.check_counter:Daemon mds.d mds_server.req_mksnap_latency.avgcount=0 2024-03-28T03:44:49.755 INFO:tasks.check_counter:Daemon mds.d mds_server.req_rmsnap_latency.avgcount=0 2024-03-28T03:44:49.755 INFO:tasks.check_counter:Daemon mds.d mds.dir_split=0 2024-03-28T03:44:49.755 DEBUG:tasks.check_counter:Getting stats from g 2024-03-28T03:44:49.756 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 10f73b92-ec98-11ee-95d0-87774f69a715 -- ceph daemon mds.g perf dump 2024-03-28T03:44:50.194 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/mon.a/config 2024-03-28T03:44:51.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:44:50 smithi132 ceph-mon[35941]: pgmap v6199: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:44:51.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:44:50 smithi165 ceph-mon[36302]: pgmap v6199: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:44:51.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:44:50 smithi119 ceph-mon[31518]: pgmap v6199: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:44:52.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:44:51 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 11350.821813 secs 2024-03-28T03:44:52.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:44:51 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 11350.821813 secs 2024-03-28T03:44:52.371 INFO:teuthology.orchestra.run.smithi119.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2024-03-28T03:44:52.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:44:51 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 11350.821813 secs 2024-03-28T03:44:53.149 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:44:52 smithi119 ceph-mon[31518]: pgmap v6200: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:44:53.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:44:52 smithi132 ceph-mon[35941]: pgmap v6200: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:44:53.225 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-03-28T03:44:53.226 ERROR:teuthology.run_tasks:Manager failed: check-counter Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/task/__init__.py", line 132, in __exit__ self.end() File "/home/teuthworker/src/git.ceph.com_ceph-c_3be9a5c9bc793e11e2800a8c0c696e8b46742033/qa/tasks/check_counter.py", line 84, in end proc = manager.admin_socket(daemon_type, daemon_id, ["perf", "dump"]) File "/home/teuthworker/src/git.ceph.com_ceph-c_3be9a5c9bc793e11e2800a8c0c696e8b46742033/qa/tasks/ceph_manager.py", line 1875, in admin_socket return shell( File "/home/teuthworker/src/git.ceph.com_ceph-c_3be9a5c9bc793e11e2800a8c0c696e8b46742033/qa/tasks/ceph_manager.py", line 43, in shell return remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi119 with status 22: 'sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 10f73b92-ec98-11ee-95d0-87774f69a715 -- ceph daemon mds.g perf dump' 2024-03-28T03:44:53.234 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-03-28T03:44:53.245 INFO:tasks.kclient:Unmounting kernel clients... 2024-03-28T03:44:53.245 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-03-28T03:44:53.245 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-28T03:44:53.315 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-03-28T03:44:53.315 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-28T03:44:53.369 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T03:44:53.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:44:52 smithi165 ceph-mon[36302]: pgmap v6200: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:44:53.588 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:53.586+0000 7ff40f600640 1 -- 172.21.15.119:0/3436321072 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff408109860 msgr2=0x7ff408109c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T03:44:53.588 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:53.586+0000 7ff40f600640 1 --2- 172.21.15.119:0/3436321072 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff408109860 0x7ff408109c40 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7ff3fc009930 tx=0x7ff3fc02bc30 comp rx=0 tx=0).stop 2024-03-28T03:44:53.588 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:53.586+0000 7ff40f600640 1 -- 172.21.15.119:0/3436321072 shutdown_connections 2024-03-28T03:44:53.588 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:53.586+0000 7ff40f600640 1 --2- 172.21.15.119:0/3436321072 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff408104210 0x7ff408110720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:44:53.588 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:53.586+0000 7ff40f600640 1 --2- 172.21.15.119:0/3436321072 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff408103850 0x7ff408103cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:44:53.588 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:53.586+0000 7ff40f600640 1 --2- 172.21.15.119:0/3436321072 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff408109860 0x7ff408109c40 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:44:53.589 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:53.586+0000 7ff40f600640 1 -- 172.21.15.119:0/3436321072 >> 172.21.15.119:0/3436321072 conn(0x7ff4080ff2e0 msgr2=0x7ff408101700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T03:44:53.589 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:53.586+0000 7ff40f600640 1 -- 172.21.15.119:0/3436321072 shutdown_connections 2024-03-28T03:44:53.589 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:53.586+0000 7ff40f600640 1 -- 172.21.15.119:0/3436321072 wait complete. 2024-03-28T03:44:53.591 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:53.587+0000 7ff40f600640 1 Processor -- start 2024-03-28T03:44:53.591 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:53.588+0000 7ff40f600640 1 -- start start 2024-03-28T03:44:53.592 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:53.588+0000 7ff40f600640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff408103850 0x7ff40819a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T03:44:53.592 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:53.588+0000 7ff40f600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff408104210 0x7ff40819a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T03:44:53.592 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:53.588+0000 7ff40f600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff408109860 0x7ff40819ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T03:44:53.592 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:53.588+0000 7ff40f600640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff408113c30 con 0x7ff408109860 2024-03-28T03:44:53.592 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:53.588+0000 7ff40f600640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff408113ab0 con 0x7ff408103850 2024-03-28T03:44:53.592 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:53.588+0000 7ff40f600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff408113db0 con 0x7ff408104210 2024-03-28T03:44:53.592 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:53.588+0000 7ff407e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff408104210 0x7ff40819a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T03:44:53.592 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:53.588+0000 7ff40cc00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff408103850 0x7ff40819a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T03:44:53.593 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:53.588+0000 7ff407e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff408104210 0x7ff40819a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.119:42206/0 (socket says 172.21.15.119:42206) 2024-03-28T03:44:53.593 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:53.588+0000 7ff40cc00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff408103850 0x7ff40819a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.119:42020/0 (socket says 172.21.15.119:42020) 2024-03-28T03:44:53.593 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:53.588+0000 7ff407e00640 1 -- 172.21.15.119:0/945026257 learned_addr learned my addr 172.21.15.119:0/945026257 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T03:44:53.593 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:53.588+0000 7ff40d600640 1 --2- 172.21.15.119:0/945026257 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff408109860 0x7ff40819ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T03:44:53.593 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:53.589+0000 7ff407e00640 1 -- 172.21.15.119:0/945026257 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff408103850 msgr2=0x7ff40819a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T03:44:53.593 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:53.589+0000 7ff407e00640 1 --2- 172.21.15.119:0/945026257 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff408103850 0x7ff40819a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:44:53.593 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:53.589+0000 7ff407e00640 1 -- 172.21.15.119:0/945026257 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff408109860 msgr2=0x7ff40819ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T03:44:53.593 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:53.589+0000 7ff407e00640 1 --2- 172.21.15.119:0/945026257 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff408109860 0x7ff40819ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:44:53.594 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:53.589+0000 7ff407e00640 1 -- 172.21.15.119:0/945026257 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff40819f150 con 0x7ff408104210 2024-03-28T03:44:53.594 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:53.589+0000 7ff40cc00640 1 --2- 172.21.15.119:0/945026257 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff408103850 0x7ff40819a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T03:44:53.594 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:53.589+0000 7ff40d600640 1 --2- 172.21.15.119:0/945026257 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff408109860 0x7ff40819ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T03:44:53.594 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:53.589+0000 7ff407e00640 1 --2- 172.21.15.119:0/945026257 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff408104210 0x7ff40819a6f0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7ff3f400d950 tx=0x7ff3f400de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T03:44:53.594 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:53.589+0000 7ff405600640 1 -- 172.21.15.119:0/945026257 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3f4017070 con 0x7ff408104210 2024-03-28T03:44:53.594 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:53.589+0000 7ff405600640 1 -- 172.21.15.119:0/945026257 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff3f4009c30 con 0x7ff408104210 2024-03-28T03:44:53.594 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:53.589+0000 7ff40f600640 1 -- 172.21.15.119:0/945026257 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff40819f440 con 0x7ff408104210 2024-03-28T03:44:53.594 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:53.589+0000 7ff405600640 1 -- 172.21.15.119:0/945026257 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3f400f6b0 con 0x7ff408104210 2024-03-28T03:44:53.594 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:53.589+0000 7ff40f600640 1 -- 172.21.15.119:0/945026257 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4081a6cd0 con 0x7ff408104210 2024-03-28T03:44:53.595 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:53.590+0000 7ff40f600640 1 -- 172.21.15.119:0/945026257 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff408068490 con 0x7ff408104210 2024-03-28T03:44:53.595 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:53.592+0000 7ff405600640 1 -- 172.21.15.119:0/945026257 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 27) v1 ==== 137422+0+0 (secure 0 0 0) 0x7ff3f40109d0 con 0x7ff408104210 2024-03-28T03:44:53.596 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:53.594+0000 7ff405600640 1 --2- 172.21.15.119:0/945026257 >> [v2:172.21.15.119:6800/4273482239,v1:172.21.15.119:6801/4273482239] conn(0x7ff3f00a2970 0x7ff3f00a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T03:44:53.596 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:53.594+0000 7ff405600640 1 -- 172.21.15.119:0/945026257 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 11163+0+0 (secure 0 0 0) 0x7ff3f40cf4e0 con 0x7ff408104210 2024-03-28T03:44:53.596 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:53.594+0000 7ff40cc00640 1 --2- 172.21.15.119:0/945026257 >> [v2:172.21.15.119:6800/4273482239,v1:172.21.15.119:6801/4273482239] conn(0x7ff3f00a2970 0x7ff3f00a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T03:44:53.597 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:53.595+0000 7ff40cc00640 1 --2- 172.21.15.119:0/945026257 >> [v2:172.21.15.119:6800/4273482239,v1:172.21.15.119:6801/4273482239] conn(0x7ff3f00a2970 0x7ff3f00a4e30 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7ff3fc005ab0 tx=0x7ff3fc005fd0 comp rx=0 tx=0).ready entity=mgr.25277 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T03:44:53.599 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:53.598+0000 7ff405600640 1 -- 172.21.15.119:0/945026257 <== mon.1 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff3f4009de0 con 0x7ff408104210 2024-03-28T03:44:53.798 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:53.796+0000 7ff40f600640 1 -- 172.21.15.119:0/945026257 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff40810b830 con 0x7ff408104210 2024-03-28T03:44:53.800 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:53.798+0000 7ff405600640 1 -- 172.21.15.119:0/945026257 <== mon.1 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v125) v1 ==== 74+0+27258 (secure 0 0 0) 0x7ff3f409b7f0 con 0x7ff408104210 2024-03-28T03:44:53.801 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-28T03:44:53.801 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":125,"fsid":"10f73b92-ec98-11ee-95d0-87774f69a715","created":"2024-03-28T00:14:57.563604+0000","modified":"2024-03-28T00:35:52.413057+0000","last_up_change":"2024-03-28T00:22:58.821662+0000","last_in_change":"2024-03-28T00:22:38.467270+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T00:18:46.253414+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-28T00:23:54.261981+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-28T00:23:55.685836+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":13,"snap_epoch":121,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c0b875d-a01e-4294-9c9c-5d0b048b8912","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":580396252},{"type":"v1","addr":"172.21.15.119:6803","nonce":580396252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":580396252},{"type":"v1","addr":"172.21.15.119:6805","nonce":580396252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":580396252},{"type":"v1","addr":"172.21.15.119:6809","nonce":580396252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":580396252},{"type":"v1","addr":"172.21.15.119:6807","nonce":580396252}]},"public_addr":"172.21.15.119:6803/580396252","cluster_addr":"172.21.15.119:6805/580396252","heartbeat_back_addr":"172.21.15.119:6809/580396252","heartbeat_front_addr":"172.21.15.119:6807/580396252","state":["exists","up"]},{"osd":1,"uuid":"5567f7c2-8ea4-4789-9ea1-467793324fb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":2500318888},{"type":"v1","addr":"172.21.15.132:6801","nonce":2500318888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":2500318888},{"type":"v1","addr":"172.21.15.132:6803","nonce":2500318888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":2500318888},{"type":"v1","addr":"172.21.15.132:6807","nonce":2500318888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":2500318888},{"type":"v1","addr":"172.21.15.132:6805","nonce":2500318888}]},"public_addr":"172.21.15.132:6801/2500318888","cluster_addr":"172.21.15.132:6803/2500318888","heartbeat_back_addr":"172.21.15.132:6807/2500318888","heartbeat_front_addr":"172.21.15.132:6805/2500318888","state":["exists","up"]},{"osd":2,"uuid":"ee54dbd0-24b4-4ee6-988c-ce71491199d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":2361434832},{"type":"v1","addr":"172.21.15.165:6801","nonce":2361434832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":2361434832},{"type":"v1","addr":"172.21.15.165:6803","nonce":2361434832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":2361434832},{"type":"v1","addr":"172.21.15.165:6807","nonce":2361434832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":2361434832},{"type":"v1","addr":"172.21.15.165:6805","nonce":2361434832}]},"public_addr":"172.21.15.165:6801/2361434832","cluster_addr":"172.21.15.165:6803/2361434832","heartbeat_back_addr":"172.21.15.165:6807/2361434832","heartbeat_front_addr":"172.21.15.165:6805/2361434832","state":["exists","up"]},{"osd":3,"uuid":"0464c40b-828f-4db4-96d9-3f7a6325e6da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":3732272210},{"type":"v1","addr":"172.21.15.119:6811","nonce":3732272210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":3732272210},{"type":"v1","addr":"172.21.15.119:6813","nonce":3732272210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":3732272210},{"type":"v1","addr":"172.21.15.119:6817","nonce":3732272210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":3732272210},{"type":"v1","addr":"172.21.15.119:6815","nonce":3732272210}]},"public_addr":"172.21.15.119:6811/3732272210","cluster_addr":"172.21.15.119:6813/3732272210","heartbeat_back_addr":"172.21.15.119:6817/3732272210","heartbeat_front_addr":"172.21.15.119:6815/3732272210","state":["exists","up"]},{"osd":4,"uuid":"8fd70b8e-5128-4173-aa0e-e735dfeb995d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":2803039505},{"type":"v1","addr":"172.21.15.132:6809","nonce":2803039505}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":2803039505},{"type":"v1","addr":"172.21.15.132:6811","nonce":2803039505}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":2803039505},{"type":"v1","addr":"172.21.15.132:6815","nonce":2803039505}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":2803039505},{"type":"v1","addr":"172.21.15.132:6813","nonce":2803039505}]},"public_addr":"172.21.15.132:6809/2803039505","cluster_addr":"172.21.15.132:6811/2803039505","heartbeat_back_addr":"172.21.15.132:6815/2803039505","heartbeat_front_addr":"172.21.15.132:6813/2803039505","state":["exists","up"]},{"osd":5,"uuid":"3224b4c1-733e-4424-a6da-870bbb5f94e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":2561252206},{"type":"v1","addr":"172.21.15.165:6809","nonce":2561252206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":2561252206},{"type":"v1","addr":"172.21.15.165:6811","nonce":2561252206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":2561252206},{"type":"v1","addr":"172.21.15.165:6815","nonce":2561252206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":2561252206},{"type":"v1","addr":"172.21.15.165:6813","nonce":2561252206}]},"public_addr":"172.21.15.165:6809/2561252206","cluster_addr":"172.21.15.165:6811/2561252206","heartbeat_back_addr":"172.21.15.165:6815/2561252206","heartbeat_front_addr":"172.21.15.165:6813/2561252206","state":["exists","up"]},{"osd":6,"uuid":"e435745d-59b5-439a-9091-0946ca0e9332","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":2545410230},{"type":"v1","addr":"172.21.15.119:6819","nonce":2545410230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":2545410230},{"type":"v1","addr":"172.21.15.119:6821","nonce":2545410230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":2545410230},{"type":"v1","addr":"172.21.15.119:6825","nonce":2545410230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":2545410230},{"type":"v1","addr":"172.21.15.119:6823","nonce":2545410230}]},"public_addr":"172.21.15.119:6819/2545410230","cluster_addr":"172.21.15.119:6821/2545410230","heartbeat_back_addr":"172.21.15.119:6825/2545410230","heartbeat_front_addr":"172.21.15.119:6823/2545410230","state":["exists","up"]},{"osd":7,"uuid":"79b97a12-f6b4-48c3-b517-2dd6b6e7c85f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":2877892376},{"type":"v1","addr":"172.21.15.132:6817","nonce":2877892376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":2877892376},{"type":"v1","addr":"172.21.15.132:6819","nonce":2877892376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":2877892376},{"type":"v1","addr":"172.21.15.132:6823","nonce":2877892376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":2877892376},{"type":"v1","addr":"172.21.15.132:6821","nonce":2877892376}]},"public_addr":"172.21.15.132:6817/2877892376","cluster_addr":"172.21.15.132:6819/2877892376","heartbeat_back_addr":"172.21.15.132:6823/2877892376","heartbeat_front_addr":"172.21.15.132:6821/2877892376","state":["exists","up"]},{"osd":8,"uuid":"2970524c-b99a-412d-85c2-6c29b54035e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":4115226150},{"type":"v1","addr":"172.21.15.165:6817","nonce":4115226150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":4115226150},{"type":"v1","addr":"172.21.15.165:6819","nonce":4115226150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":4115226150},{"type":"v1","addr":"172.21.15.165:6823","nonce":4115226150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":4115226150},{"type":"v1","addr":"172.21.15.165:6821","nonce":4115226150}]},"public_addr":"172.21.15.165:6817/4115226150","cluster_addr":"172.21.15.165:6819/4115226150","heartbeat_back_addr":"172.21.15.165:6823/4115226150","heartbeat_front_addr":"172.21.15.165:6821/4115226150","state":["exists","up"]},{"osd":9,"uuid":"2b1a08ed-cde9-4def-b3e8-838fff22709c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":2532245222},{"type":"v1","addr":"172.21.15.119:6827","nonce":2532245222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":2532245222},{"type":"v1","addr":"172.21.15.119:6829","nonce":2532245222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":2532245222},{"type":"v1","addr":"172.21.15.119:6833","nonce":2532245222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":2532245222},{"type":"v1","addr":"172.21.15.119:6831","nonce":2532245222}]},"public_addr":"172.21.15.119:6827/2532245222","cluster_addr":"172.21.15.119:6829/2532245222","heartbeat_back_addr":"172.21.15.119:6833/2532245222","heartbeat_front_addr":"172.21.15.119:6831/2532245222","state":["exists","up"]},{"osd":10,"uuid":"f019acd0-915a-4094-b12c-b53672740a5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":3454298492},{"type":"v1","addr":"172.21.15.132:6825","nonce":3454298492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":3454298492},{"type":"v1","addr":"172.21.15.132:6827","nonce":3454298492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":3454298492},{"type":"v1","addr":"172.21.15.132:6831","nonce":3454298492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":3454298492},{"type":"v1","addr":"172.21.15.132:6829","nonce":3454298492}]},"public_addr":"172.21.15.132:6825/3454298492","cluster_addr":"172.21.15.132:6827/3454298492","heartbeat_back_addr":"172.21.15.132:6831/3454298492","heartbeat_front_addr":"172.21.15.132:6829/3454298492","state":["exists","up"]},{"osd":11,"uuid":"2c1e59a8-8f21-45c0-8568-4b632ce020f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":2138771601},{"type":"v1","addr":"172.21.15.165:6825","nonce":2138771601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":2138771601},{"type":"v1","addr":"172.21.15.165:6827","nonce":2138771601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":2138771601},{"type":"v1","addr":"172.21.15.165:6831","nonce":2138771601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":2138771601},{"type":"v1","addr":"172.21.15.165:6829","nonce":2138771601}]},"public_addr":"172.21.15.165:6825/2138771601","cluster_addr":"172.21.15.165:6827/2138771601","heartbeat_back_addr":"172.21.15.165:6831/2138771601","heartbeat_front_addr":"172.21.15.165:6829/2138771601","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:17:54.791531+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:18:19.466638+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:18:43.435536+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:19:15.702685+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:19:41.767370+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:20:06.793003+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:20:39.952744+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:21:05.727262+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:21:30.735829+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:22:04.072142+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:22:30.894047+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:22:56.637728+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.119:6836/2109630829":"2024-03-29T00:35:52.413013+0000","172.21.15.165:6832/2013665520":"2024-03-29T00:32:07.607942+0000","172.21.15.165:6839/384483120":"2024-03-29T00:32:07.271819+0000","172.21.15.165:6838/384483120":"2024-03-29T00:32:07.271819+0000","172.21.15.119:6834/744950487":"2024-03-29T00:31:51.094723+0000","172.21.15.165:6839/768712672":"2024-03-29T00:33:20.841067+0000","172.21.15.119:0/87565965":"2024-03-29T00:16:03.582545+0000","172.21.15.119:6834/2793569574":"2024-03-29T00:35:34.679932+0000","172.21.15.165:6839/3004684592":"2024-03-29T00:31:32.429788+0000","172.21.15.165:6833/1389081045":"2024-03-29T00:31:20.067892+0000","172.21.15.165:6832/609963168":"2024-03-29T00:30:41.299818+0000","172.21.15.119:6834/2941089726":"2024-03-29T00:33:37.236449+0000","172.21.15.165:6839/2762859081":"2024-03-29T00:33:59.786505+0000","172.21.15.165:6833/609963168":"2024-03-29T00:30:41.299818+0000","172.21.15.165:6837/2717860763":"2024-03-29T00:33:02.702892+0000","172.21.15.165:6836/3170852275":"2024-03-29T00:32:25.939039+0000","172.21.15.119:6800/2056426384":"2024-03-29T00:15:30.472283+0000","172.21.15.119:6801/2575246519":"2024-03-29T00:16:03.582545+0000","172.21.15.165:6836/1531193032":"2024-03-29T00:30:24.146598+0000","172.21.15.119:6834/1659925849":"2024-03-29T00:34:17.464068+0000","172.21.15.119:6835/744950487":"2024-03-29T00:31:51.094723+0000","172.21.15.165:6836/1456402510":"2024-03-29T00:30:56.719761+0000","172.21.15.165:6837/1456402510":"2024-03-29T00:30:56.719761+0000","172.21.15.165:6839/1687390659":"2024-03-29T00:32:46.777861+0000","172.21.15.119:0/2635982178":"2024-03-29T00:16:03.582545+0000","172.21.15.119:0/2430401731":"2024-03-29T00:26:32.965486+0000","172.21.15.119:6837/482178170":"2024-03-29T00:35:17.974473+0000","172.21.15.119:6801/1401937096":"2024-03-29T00:26:32.965486+0000","172.21.15.119:0/4201699272":"2024-03-29T00:16:03.582545+0000","172.21.15.165:6838/3004684592":"2024-03-29T00:31:32.429788+0000","172.21.15.165:6836/1011925619":"2024-03-29T00:31:49.271822+0000","172.21.15.165:6837/1531193032":"2024-03-29T00:30:24.146598+0000","172.21.15.165:6833/2013665520":"2024-03-29T00:32:07.607942+0000","172.21.15.119:6834/2101147835":"2024-03-29T00:32:28.920424+0000","172.21.15.119:6835/3517971302":"2024-03-29T00:35:17.389381+0000","172.21.15.165:6832/1389081045":"2024-03-29T00:31:20.067892+0000","172.21.15.119:6835/2210305461":"2024-03-29T00:33:02.940231+0000","172.21.15.119:0/2891240357":"2024-03-29T00:15:30.472283+0000","172.21.15.119:6838/1595343240":"2024-03-29T00:34:16.710959+0000","172.21.15.119:6801/2056426384":"2024-03-29T00:15:30.472283+0000","172.21.15.119:0/357445067":"2024-03-29T00:26:32.965486+0000","172.21.15.165:6838/1687390659":"2024-03-29T00:32:46.777861+0000","172.21.15.119:6838/3947365264":"2024-03-29T00:34:59.827426+0000","172.21.15.165:6837/1011925619":"2024-03-29T00:31:49.271822+0000","172.21.15.119:6800/2575246519":"2024-03-29T00:16:03.582545+0000","172.21.15.165:6836/823338130":"2024-03-29T00:33:39.774934+0000","172.21.15.119:0/468478300":"2024-03-29T00:15:30.472283+0000","172.21.15.119:0/4119730435":"2024-03-29T00:26:32.965486+0000","172.21.15.119:6800/1401937096":"2024-03-29T00:26:32.965486+0000","172.21.15.119:0/524303043":"2024-03-29T00:26:32.965486+0000","172.21.15.119:6835/2101147835":"2024-03-29T00:32:28.920424+0000","172.21.15.165:6832/4070507195":"2024-03-29T00:32:46.507920+0000","172.21.15.119:6836/3988635716":"2024-03-29T00:34:37.192476+0000","172.21.15.165:6836/2717860763":"2024-03-29T00:33:02.702892+0000","172.21.15.119:6834/2210305461":"2024-03-29T00:33:02.940231+0000","172.21.15.165:6832/3417761142":"2024-03-29T00:33:20.267933+0000","172.21.15.165:6838/768712672":"2024-03-29T00:33:20.841067+0000","172.21.15.119:6835/2941089726":"2024-03-29T00:33:37.236449+0000","172.21.15.165:6837/3170852275":"2024-03-29T00:32:25.939039+0000","172.21.15.119:6839/1595343240":"2024-03-29T00:34:16.710959+0000","172.21.15.165:6833/4070507195":"2024-03-29T00:32:46.507920+0000","172.21.15.165:6837/823338130":"2024-03-29T00:33:39.774934+0000","172.21.15.119:6834/4197433095":"2024-03-29T00:34:37.186007+0000","172.21.15.165:6832/1344198130":"2024-03-29T00:33:59.786346+0000","172.21.15.119:6837/3511403986":"2024-03-29T00:35:35.303954+0000","172.21.15.165:6838/2762859081":"2024-03-29T00:33:59.786505+0000","172.21.15.119:6835/4197433095":"2024-03-29T00:34:37.186007+0000","172.21.15.119:6836/482178170":"2024-03-29T00:35:17.974473+0000","172.21.15.165:6833/3417761142":"2024-03-29T00:33:20.267933+0000","172.21.15.165:6833/1344198130":"2024-03-29T00:33:59.786346+0000","172.21.15.119:6835/1659925849":"2024-03-29T00:34:17.464068+0000","172.21.15.119:6837/3988635716":"2024-03-29T00:34:37.192476+0000","172.21.15.119:6836/3511403986":"2024-03-29T00:35:35.303954+0000","172.21.15.119:6834/3816731508":"2024-03-29T00:34:54.192187+0000","172.21.15.119:6837/2109630829":"2024-03-29T00:35:52.413013+0000","172.21.15.119:6835/3816731508":"2024-03-29T00:34:54.192187+0000","172.21.15.119:6839/3947365264":"2024-03-29T00:34:59.827426+0000","172.21.15.119:0/1639109126":"2024-03-29T00:15:30.472283+0000","172.21.15.119:6834/3517971302":"2024-03-29T00:35:17.389381+0000","172.21.15.119:6835/2793569574":"2024-03-29T00:35:34.679932+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T03:44:53.803 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:53.801+0000 7ff40f600640 1 -- 172.21.15.119:0/945026257 >> [v2:172.21.15.119:6800/4273482239,v1:172.21.15.119:6801/4273482239] conn(0x7ff3f00a2970 msgr2=0x7ff3f00a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T03:44:53.804 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:53.801+0000 7ff40f600640 1 --2- 172.21.15.119:0/945026257 >> [v2:172.21.15.119:6800/4273482239,v1:172.21.15.119:6801/4273482239] conn(0x7ff3f00a2970 0x7ff3f00a4e30 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7ff3fc005ab0 tx=0x7ff3fc005fd0 comp rx=0 tx=0).stop 2024-03-28T03:44:53.804 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:53.801+0000 7ff40f600640 1 -- 172.21.15.119:0/945026257 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff408104210 msgr2=0x7ff40819a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T03:44:53.804 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:53.801+0000 7ff40f600640 1 --2- 172.21.15.119:0/945026257 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff408104210 0x7ff40819a6f0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7ff3f400d950 tx=0x7ff3f400de10 comp rx=0 tx=0).stop 2024-03-28T03:44:53.804 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:53.801+0000 7ff40f600640 1 -- 172.21.15.119:0/945026257 shutdown_connections 2024-03-28T03:44:53.804 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:53.801+0000 7ff40f600640 1 --2- 172.21.15.119:0/945026257 >> [v2:172.21.15.119:6800/4273482239,v1:172.21.15.119:6801/4273482239] conn(0x7ff3f00a2970 0x7ff3f00a4e30 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:44:53.804 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:53.801+0000 7ff40f600640 1 --2- 172.21.15.119:0/945026257 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff408109860 0x7ff40819ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:44:53.804 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:53.801+0000 7ff40f600640 1 --2- 172.21.15.119:0/945026257 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff408104210 0x7ff40819a6f0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:44:53.804 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:53.801+0000 7ff40f600640 1 --2- 172.21.15.119:0/945026257 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff408103850 0x7ff40819a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:44:53.805 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:53.801+0000 7ff40f600640 1 -- 172.21.15.119:0/945026257 >> 172.21.15.119:0/945026257 conn(0x7ff4080ff2e0 msgr2=0x7ff4080ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T03:44:53.805 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:53.802+0000 7ff40f600640 1 -- 172.21.15.119:0/945026257 shutdown_connections 2024-03-28T03:44:53.805 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:53.802+0000 7ff40f600640 1 -- 172.21.15.119:0/945026257 wait complete. 2024-03-28T03:44:53.824 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-28T00:18:46.253414+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-28T00:23:54.261981+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-28T00:23:55.685836+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '121', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 13, 'snap_epoch': 121, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-28T03:44:53.824 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-28T03:44:54.041 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:54.038+0000 7fbb22e00640 1 -- 172.21.15.119:0/1538730083 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbb1c107e50 msgr2=0x7fbb1c108230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T03:44:54.041 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:54.038+0000 7fbb22e00640 1 --2- 172.21.15.119:0/1538730083 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbb1c107e50 0x7fbb1c108230 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fbb10009900 tx=0x7fbb1002bc40 comp rx=0 tx=0).stop 2024-03-28T03:44:54.041 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:54.038+0000 7fbb22e00640 1 -- 172.21.15.119:0/1538730083 shutdown_connections 2024-03-28T03:44:54.041 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:54.039+0000 7fbb22e00640 1 --2- 172.21.15.119:0/1538730083 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbb1c10cea0 0x7fbb1c114a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:44:54.041 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:54.039+0000 7fbb22e00640 1 --2- 172.21.15.119:0/1538730083 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbb1c108800 0x7fbb1c10c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:44:54.041 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:54.039+0000 7fbb22e00640 1 --2- 172.21.15.119:0/1538730083 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbb1c107e50 0x7fbb1c108230 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:44:54.042 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:54.039+0000 7fbb22e00640 1 -- 172.21.15.119:0/1538730083 >> 172.21.15.119:0/1538730083 conn(0x7fbb1c0774c0 msgr2=0x7fbb1c1064c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T03:44:54.042 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:54.039+0000 7fbb22e00640 1 -- 172.21.15.119:0/1538730083 shutdown_connections 2024-03-28T03:44:54.042 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:54.039+0000 7fbb22e00640 1 -- 172.21.15.119:0/1538730083 wait complete. 2024-03-28T03:44:54.044 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:54.040+0000 7fbb22e00640 1 Processor -- start 2024-03-28T03:44:54.044 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:54.040+0000 7fbb22e00640 1 -- start start 2024-03-28T03:44:54.044 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:54.040+0000 7fbb22e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbb1c107e50 0x7fbb1c19e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T03:44:54.044 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:54.040+0000 7fbb22e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbb1c10cea0 0x7fbb1c19e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T03:44:54.044 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:54.040+0000 7fbb22e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbb1c1a2cc0 0x7fbb1c1a3170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T03:44:54.044 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:54.040+0000 7fbb22e00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb1c117fc0 con 0x7fbb1c10cea0 2024-03-28T03:44:54.045 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:54.040+0000 7fbb22e00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb1c117e40 con 0x7fbb1c1a2cc0 2024-03-28T03:44:54.045 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:54.040+0000 7fbb22e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb1c118140 con 0x7fbb1c107e50 2024-03-28T03:44:54.045 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:54.040+0000 7fbb21a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbb1c107e50 0x7fbb1c19e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T03:44:54.045 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:54.040+0000 7fbb21a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbb1c107e50 0x7fbb1c19e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.119:50386/0 (socket says 172.21.15.119:50386) 2024-03-28T03:44:54.045 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:54.040+0000 7fbb21a00640 1 -- 172.21.15.119:0/1198147085 learned_addr learned my addr 172.21.15.119:0/1198147085 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T03:44:54.045 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:54.040+0000 7fbb21000640 1 --2- 172.21.15.119:0/1198147085 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbb1c10cea0 0x7fbb1c19e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T03:44:54.045 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:54.040+0000 7fbb22400640 1 --2- 172.21.15.119:0/1198147085 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbb1c1a2cc0 0x7fbb1c1a3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T03:44:54.045 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:54.041+0000 7fbb21a00640 1 -- 172.21.15.119:0/1198147085 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbb1c1a2cc0 msgr2=0x7fbb1c1a3170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T03:44:54.045 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:54.041+0000 7fbb21a00640 1 --2- 172.21.15.119:0/1198147085 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbb1c1a2cc0 0x7fbb1c1a3170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:44:54.046 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:54.041+0000 7fbb21a00640 1 -- 172.21.15.119:0/1198147085 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbb1c10cea0 msgr2=0x7fbb1c19e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T03:44:54.046 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:54.041+0000 7fbb21a00640 1 --2- 172.21.15.119:0/1198147085 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbb1c10cea0 0x7fbb1c19e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:44:54.046 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:54.041+0000 7fbb21a00640 1 -- 172.21.15.119:0/1198147085 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb1c1a37b0 con 0x7fbb1c107e50 2024-03-28T03:44:54.046 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:54.041+0000 7fbb21a00640 1 --2- 172.21.15.119:0/1198147085 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbb1c107e50 0x7fbb1c19e400 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fbb10005ab0 tx=0x7fbb10004360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T03:44:54.046 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:54.041+0000 7fbb1a000640 1 -- 172.21.15.119:0/1198147085 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb10041070 con 0x7fbb1c107e50 2024-03-28T03:44:54.046 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:54.041+0000 7fbb1a000640 1 -- 172.21.15.119:0/1198147085 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbb1002cca0 con 0x7fbb1c107e50 2024-03-28T03:44:54.046 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:54.041+0000 7fbb22e00640 1 -- 172.21.15.119:0/1198147085 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb1c1a3a40 con 0x7fbb1c107e50 2024-03-28T03:44:54.046 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:54.041+0000 7fbb1a000640 1 -- 172.21.15.119:0/1198147085 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb10003e50 con 0x7fbb1c107e50 2024-03-28T03:44:54.047 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:54.042+0000 7fbb22e00640 1 -- 172.21.15.119:0/1198147085 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb1c1ab200 con 0x7fbb1c107e50 2024-03-28T03:44:54.047 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:54.043+0000 7fbb22e00640 1 -- 172.21.15.119:0/1198147085 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb1c068800 con 0x7fbb1c107e50 2024-03-28T03:44:54.048 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:54.044+0000 7fbb1a000640 1 -- 172.21.15.119:0/1198147085 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 27) v1 ==== 137422+0+0 (secure 0 0 0) 0x7fbb10044070 con 0x7fbb1c107e50 2024-03-28T03:44:54.048 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:54.045+0000 7fbb1a000640 1 --2- 172.21.15.119:0/1198147085 >> [v2:172.21.15.119:6800/4273482239,v1:172.21.15.119:6801/4273482239] conn(0x7fbb040a2b90 0x7fbb040a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T03:44:54.048 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:54.046+0000 7fbb1a000640 1 -- 172.21.15.119:0/1198147085 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 11163+0+0 (secure 0 0 0) 0x7fbb100f1190 con 0x7fbb1c107e50 2024-03-28T03:44:54.048 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:54.046+0000 7fbb21000640 1 --2- 172.21.15.119:0/1198147085 >> [v2:172.21.15.119:6800/4273482239,v1:172.21.15.119:6801/4273482239] conn(0x7fbb040a2b90 0x7fbb040a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T03:44:54.048 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:54.046+0000 7fbb21000640 1 --2- 172.21.15.119:0/1198147085 >> [v2:172.21.15.119:6800/4273482239,v1:172.21.15.119:6801/4273482239] conn(0x7fbb040a2b90 0x7fbb040a5050 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fbb1c10e290 tx=0x7fbb0c009500 comp rx=0 tx=0).ready entity=mgr.25277 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T03:44:54.055 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:54.054+0000 7fbb1a000640 1 -- 172.21.15.119:0/1198147085 <== mon.1 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fbb100bcf00 con 0x7fbb1c107e50 2024-03-28T03:44:54.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:44:53 smithi119 ceph-mon[31518]: from='client.? 172.21.15.119:0/945026257' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-28T03:44:54.257 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:54.255+0000 7fbb22e00640 1 -- 172.21.15.119:0/1198147085 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbb1c1100a0 con 0x7fbb1c107e50 2024-03-28T03:44:54.257 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:54.255+0000 7fbb1a000640 1 -- 172.21.15.119:0/1198147085 <== mon.1 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v125) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbb100c0fa0 con 0x7fbb1c107e50 2024-03-28T03:44:54.257 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 1 2024-03-28T03:44:54.260 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:54.258+0000 7fbb22e00640 1 -- 172.21.15.119:0/1198147085 >> [v2:172.21.15.119:6800/4273482239,v1:172.21.15.119:6801/4273482239] conn(0x7fbb040a2b90 msgr2=0x7fbb040a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T03:44:54.260 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:54.258+0000 7fbb22e00640 1 --2- 172.21.15.119:0/1198147085 >> [v2:172.21.15.119:6800/4273482239,v1:172.21.15.119:6801/4273482239] conn(0x7fbb040a2b90 0x7fbb040a5050 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fbb1c10e290 tx=0x7fbb0c009500 comp rx=0 tx=0).stop 2024-03-28T03:44:54.261 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:54.258+0000 7fbb22e00640 1 -- 172.21.15.119:0/1198147085 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbb1c107e50 msgr2=0x7fbb1c19e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T03:44:54.261 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:54.258+0000 7fbb22e00640 1 --2- 172.21.15.119:0/1198147085 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbb1c107e50 0x7fbb1c19e400 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fbb10005ab0 tx=0x7fbb10004360 comp rx=0 tx=0).stop 2024-03-28T03:44:54.261 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:54.258+0000 7fbb22e00640 1 -- 172.21.15.119:0/1198147085 shutdown_connections 2024-03-28T03:44:54.261 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:54.258+0000 7fbb22e00640 1 --2- 172.21.15.119:0/1198147085 >> [v2:172.21.15.119:6800/4273482239,v1:172.21.15.119:6801/4273482239] conn(0x7fbb040a2b90 0x7fbb040a5050 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:44:54.261 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:54.258+0000 7fbb22e00640 1 --2- 172.21.15.119:0/1198147085 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbb1c1a2cc0 0x7fbb1c1a3170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:44:54.261 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:54.258+0000 7fbb22e00640 1 --2- 172.21.15.119:0/1198147085 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbb1c10cea0 0x7fbb1c19e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:44:54.261 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:54.258+0000 7fbb22e00640 1 --2- 172.21.15.119:0/1198147085 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbb1c107e50 0x7fbb1c19e400 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:44:54.261 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:54.258+0000 7fbb22e00640 1 -- 172.21.15.119:0/1198147085 >> 172.21.15.119:0/1198147085 conn(0x7fbb1c0774c0 msgr2=0x7fbb1c077ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T03:44:54.262 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:54.258+0000 7fbb22e00640 1 -- 172.21.15.119:0/1198147085 shutdown_connections 2024-03-28T03:44:54.262 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:54.258+0000 7fbb22e00640 1 -- 172.21.15.119:0/1198147085 wait complete. 2024-03-28T03:44:54.279 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-28T03:44:54.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:44:53 smithi165 ceph-mon[36302]: from='client.? 172.21.15.119:0/945026257' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-28T03:44:54.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:44:53 smithi132 ceph-mon[35941]: from='client.? 172.21.15.119:0/945026257' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-28T03:44:54.491 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:54.489+0000 7f353fe00640 1 -- 172.21.15.119:0/2603411731 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3538103840 msgr2=0x7f3538103cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T03:44:54.492 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:54.489+0000 7f353fe00640 1 --2- 172.21.15.119:0/2603411731 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3538103840 0x7f3538103cc0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f3524009860 tx=0x7f352402bb70 comp rx=0 tx=0).stop 2024-03-28T03:44:54.492 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:54.490+0000 7f353fe00640 1 -- 172.21.15.119:0/2603411731 shutdown_connections 2024-03-28T03:44:54.492 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:54.490+0000 7f353fe00640 1 --2- 172.21.15.119:0/2603411731 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3538104200 0x7f3538110710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:44:54.492 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:54.490+0000 7f353fe00640 1 --2- 172.21.15.119:0/2603411731 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3538103840 0x7f3538103cc0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:44:54.492 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:54.490+0000 7f353fe00640 1 --2- 172.21.15.119:0/2603411731 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3538109850 0x7f3538109c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:44:54.492 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:54.490+0000 7f353fe00640 1 -- 172.21.15.119:0/2603411731 >> 172.21.15.119:0/2603411731 conn(0x7f35380ff2f0 msgr2=0x7f3538101710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T03:44:54.492 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:54.490+0000 7f353fe00640 1 -- 172.21.15.119:0/2603411731 shutdown_connections 2024-03-28T03:44:54.492 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:54.490+0000 7f353fe00640 1 -- 172.21.15.119:0/2603411731 wait complete. 2024-03-28T03:44:54.494 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:54.491+0000 7f353fe00640 1 Processor -- start 2024-03-28T03:44:54.494 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:54.491+0000 7f353fe00640 1 -- start start 2024-03-28T03:44:54.494 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:54.491+0000 7f353fe00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3538103840 0x7f353819a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T03:44:54.494 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:54.491+0000 7f353fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3538104200 0x7f353819a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T03:44:54.494 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:54.492+0000 7f353fe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3538109850 0x7f353819ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T03:44:54.495 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:54.492+0000 7f353fe00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3538113c20 con 0x7f3538109850 2024-03-28T03:44:54.495 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:54.492+0000 7f353fe00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3538113aa0 con 0x7f3538103840 2024-03-28T03:44:54.495 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:54.492+0000 7f353fe00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3538113da0 con 0x7f3538104200 2024-03-28T03:44:54.495 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:54.492+0000 7f353d400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3538103840 0x7f353819a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T03:44:54.495 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:54.492+0000 7f353d400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3538103840 0x7f353819a140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.119:38404/0 (socket says 172.21.15.119:38404) 2024-03-28T03:44:54.495 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:54.492+0000 7f353d400640 1 -- 172.21.15.119:0/1515627306 learned_addr learned my addr 172.21.15.119:0/1515627306 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T03:44:54.495 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:54.492+0000 7f353ca00640 1 --2- 172.21.15.119:0/1515627306 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3538104200 0x7f353819a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T03:44:54.495 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:54.492+0000 7f353de00640 1 --2- 172.21.15.119:0/1515627306 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3538109850 0x7f353819ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T03:44:54.495 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:54.492+0000 7f353d400640 1 -- 172.21.15.119:0/1515627306 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3538104200 msgr2=0x7f353819a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T03:44:54.496 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:54.492+0000 7f353d400640 1 --2- 172.21.15.119:0/1515627306 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3538104200 0x7f353819a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:44:54.496 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:54.492+0000 7f353d400640 1 -- 172.21.15.119:0/1515627306 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3538109850 msgr2=0x7f353819ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T03:44:54.496 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:54.492+0000 7f353d400640 1 --2- 172.21.15.119:0/1515627306 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3538109850 0x7f353819ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:44:54.496 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:54.492+0000 7f353d400640 1 -- 172.21.15.119:0/1515627306 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f353819f130 con 0x7f3538103840 2024-03-28T03:44:54.496 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:54.492+0000 7f353d400640 1 --2- 172.21.15.119:0/1515627306 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3538103840 0x7f353819a140 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f352c00e9c0 tx=0x7f352c00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T03:44:54.497 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:54.493+0000 7f3536000640 1 -- 172.21.15.119:0/1515627306 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f352c00cde0 con 0x7f3538103840 2024-03-28T03:44:54.497 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:54.493+0000 7f3536000640 1 -- 172.21.15.119:0/1515627306 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f352c0035f0 con 0x7f3538103840 2024-03-28T03:44:54.497 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:54.493+0000 7f353fe00640 1 -- 172.21.15.119:0/1515627306 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f353819f420 con 0x7f3538103840 2024-03-28T03:44:54.497 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:54.493+0000 7f3536000640 1 -- 172.21.15.119:0/1515627306 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f352c010690 con 0x7f3538103840 2024-03-28T03:44:54.497 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:54.493+0000 7f353fe00640 1 -- 172.21.15.119:0/1515627306 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35381a6cb0 con 0x7f3538103840 2024-03-28T03:44:54.498 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:54.494+0000 7f353fe00640 1 -- 172.21.15.119:0/1515627306 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35380688e0 con 0x7f3538103840 2024-03-28T03:44:54.498 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:54.496+0000 7f3536000640 1 -- 172.21.15.119:0/1515627306 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 27) v1 ==== 137422+0+0 (secure 0 0 0) 0x7f352c0037a0 con 0x7f3538103840 2024-03-28T03:44:54.499 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:54.497+0000 7f3536000640 1 --2- 172.21.15.119:0/1515627306 >> [v2:172.21.15.119:6800/4273482239,v1:172.21.15.119:6801/4273482239] conn(0x7f35200a29f0 0x7f35200a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T03:44:54.499 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:54.497+0000 7f3536000640 1 -- 172.21.15.119:0/1515627306 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 11163+0+0 (secure 0 0 0) 0x7f352c0cfb00 con 0x7f3538103840 2024-03-28T03:44:54.499 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:54.497+0000 7f353ca00640 1 --2- 172.21.15.119:0/1515627306 >> [v2:172.21.15.119:6800/4273482239,v1:172.21.15.119:6801/4273482239] conn(0x7f35200a29f0 0x7f35200a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T03:44:54.500 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:54.498+0000 7f353ca00640 1 --2- 172.21.15.119:0/1515627306 >> [v2:172.21.15.119:6800/4273482239,v1:172.21.15.119:6801/4273482239] conn(0x7f35200a29f0 0x7f35200a4eb0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f352402c180 tx=0x7f35240056d0 comp rx=0 tx=0).ready entity=mgr.25277 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T03:44:54.507 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:54.506+0000 7f3536000640 1 -- 172.21.15.119:0/1515627306 <== mon.2 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f352c09be10 con 0x7f3538103840 2024-03-28T03:44:54.704 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:54.702+0000 7f353fe00640 1 -- 172.21.15.119:0/1515627306 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f353802d070 con 0x7f3538103840 2024-03-28T03:44:54.705 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:54.703+0000 7f3536000640 1 -- 172.21.15.119:0/1515627306 <== mon.2 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v125) v1 ==== 104+0+11 (secure 0 0 0) 0x7f352c09b710 con 0x7f3538103840 2024-03-28T03:44:54.705 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 32 2024-03-28T03:44:54.708 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:54.706+0000 7f353fe00640 1 -- 172.21.15.119:0/1515627306 >> [v2:172.21.15.119:6800/4273482239,v1:172.21.15.119:6801/4273482239] conn(0x7f35200a29f0 msgr2=0x7f35200a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T03:44:54.708 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:54.706+0000 7f353fe00640 1 --2- 172.21.15.119:0/1515627306 >> [v2:172.21.15.119:6800/4273482239,v1:172.21.15.119:6801/4273482239] conn(0x7f35200a29f0 0x7f35200a4eb0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f352402c180 tx=0x7f35240056d0 comp rx=0 tx=0).stop 2024-03-28T03:44:54.708 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:54.706+0000 7f353fe00640 1 -- 172.21.15.119:0/1515627306 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3538103840 msgr2=0x7f353819a140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T03:44:54.708 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:54.706+0000 7f353fe00640 1 --2- 172.21.15.119:0/1515627306 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3538103840 0x7f353819a140 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f352c00e9c0 tx=0x7f352c00ee80 comp rx=0 tx=0).stop 2024-03-28T03:44:54.708 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:54.706+0000 7f353fe00640 1 -- 172.21.15.119:0/1515627306 shutdown_connections 2024-03-28T03:44:54.709 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:54.706+0000 7f353fe00640 1 --2- 172.21.15.119:0/1515627306 >> [v2:172.21.15.119:6800/4273482239,v1:172.21.15.119:6801/4273482239] conn(0x7f35200a29f0 0x7f35200a4eb0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:44:54.709 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:54.706+0000 7f353fe00640 1 --2- 172.21.15.119:0/1515627306 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3538109850 0x7f353819ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:44:54.709 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:54.706+0000 7f353fe00640 1 --2- 172.21.15.119:0/1515627306 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3538104200 0x7f353819a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:44:54.709 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:54.706+0000 7f353fe00640 1 --2- 172.21.15.119:0/1515627306 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3538103840 0x7f353819a140 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:44:54.709 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:54.706+0000 7f353fe00640 1 -- 172.21.15.119:0/1515627306 >> 172.21.15.119:0/1515627306 conn(0x7f35380ff2f0 msgr2=0x7f35380ff9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T03:44:54.709 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:54.706+0000 7f353fe00640 1 -- 172.21.15.119:0/1515627306 shutdown_connections 2024-03-28T03:44:54.709 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:54.706+0000 7f353fe00640 1 -- 172.21.15.119:0/1515627306 wait complete. 2024-03-28T03:44:54.727 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-28T03:44:54.942 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:54.940+0000 7fc4f1800640 1 -- 172.21.15.119:0/420785874 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc4ec075bd0 msgr2=0x7fc4ec075fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T03:44:54.943 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:54.940+0000 7fc4f1800640 1 --2- 172.21.15.119:0/420785874 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc4ec075bd0 0x7fc4ec075fb0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fc4dc009930 tx=0x7fc4dc02bc20 comp rx=0 tx=0).stop 2024-03-28T03:44:54.943 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:54.940+0000 7fc4f1800640 1 -- 172.21.15.119:0/420785874 shutdown_connections 2024-03-28T03:44:54.943 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:54.940+0000 7fc4f1800640 1 --2- 172.21.15.119:0/420785874 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc4ec1124e0 0x7fc4ec1148d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:44:54.943 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:54.940+0000 7fc4f1800640 1 --2- 172.21.15.119:0/420785874 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc4ec0764f0 0x7fc4ec111fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:44:54.943 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:54.940+0000 7fc4f1800640 1 --2- 172.21.15.119:0/420785874 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc4ec075bd0 0x7fc4ec075fb0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:44:54.943 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:54.940+0000 7fc4f1800640 1 -- 172.21.15.119:0/420785874 >> 172.21.15.119:0/420785874 conn(0x7fc4ec0ff2f0 msgr2=0x7fc4ec101710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T03:44:54.943 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:54.941+0000 7fc4f1800640 1 -- 172.21.15.119:0/420785874 shutdown_connections 2024-03-28T03:44:54.943 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:54.941+0000 7fc4f1800640 1 -- 172.21.15.119:0/420785874 wait complete. 2024-03-28T03:44:54.945 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:54.942+0000 7fc4f1800640 1 Processor -- start 2024-03-28T03:44:54.946 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:54.942+0000 7fc4f1800640 1 -- start start 2024-03-28T03:44:54.946 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:54.942+0000 7fc4f1800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc4ec075bd0 0x7fc4ec19e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T03:44:54.946 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:54.942+0000 7fc4f1800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc4ec0764f0 0x7fc4ec19eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T03:44:54.946 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:54.942+0000 7fc4f1800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc4ec1124e0 0x7fc4ec1a2e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T03:44:54.946 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:54.942+0000 7fc4f1800640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4ec118090 con 0x7fc4ec0764f0 2024-03-28T03:44:54.946 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:54.942+0000 7fc4f1800640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4ec117f10 con 0x7fc4ec075bd0 2024-03-28T03:44:54.946 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:54.942+0000 7fc4f1800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4ec118210 con 0x7fc4ec1124e0 2024-03-28T03:44:54.947 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:54.942+0000 7fc4eaa00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc4ec075bd0 0x7fc4ec19e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T03:44:54.947 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:54.942+0000 7fc4eaa00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc4ec075bd0 0x7fc4ec19e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.119:38432/0 (socket says 172.21.15.119:38432) 2024-03-28T03:44:54.947 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:54.943+0000 7fc4eaa00640 1 -- 172.21.15.119:0/688084142 learned_addr learned my addr 172.21.15.119:0/688084142 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T03:44:54.947 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:54.943+0000 7fc4eb400640 1 --2- 172.21.15.119:0/688084142 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc4ec1124e0 0x7fc4ec1a2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T03:44:54.947 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:54.943+0000 7fc4e3e00640 1 --2- 172.21.15.119:0/688084142 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc4ec0764f0 0x7fc4ec19eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T03:44:54.947 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:54.943+0000 7fc4eaa00640 1 -- 172.21.15.119:0/688084142 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc4ec1124e0 msgr2=0x7fc4ec1a2e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T03:44:54.947 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:54.943+0000 7fc4eaa00640 1 --2- 172.21.15.119:0/688084142 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc4ec1124e0 0x7fc4ec1a2e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:44:54.947 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:54.943+0000 7fc4eaa00640 1 -- 172.21.15.119:0/688084142 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc4ec0764f0 msgr2=0x7fc4ec19eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T03:44:54.948 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:54.943+0000 7fc4eaa00640 1 --2- 172.21.15.119:0/688084142 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc4ec0764f0 0x7fc4ec19eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:44:54.948 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:54.943+0000 7fc4eaa00640 1 -- 172.21.15.119:0/688084142 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4ec1a3590 con 0x7fc4ec075bd0 2024-03-28T03:44:54.948 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:54.943+0000 7fc4eaa00640 1 --2- 172.21.15.119:0/688084142 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc4ec075bd0 0x7fc4ec19e5f0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fc4dc005ab0 tx=0x7fc4dc005e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T03:44:54.948 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:54.943+0000 7fc4e3400640 1 -- 172.21.15.119:0/688084142 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4dc043020 con 0x7fc4ec075bd0 2024-03-28T03:44:54.948 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:54.944+0000 7fc4e3400640 1 -- 172.21.15.119:0/688084142 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc4dc005000 con 0x7fc4ec075bd0 2024-03-28T03:44:54.948 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:54.944+0000 7fc4e3400640 1 -- 172.21.15.119:0/688084142 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4dc003a80 con 0x7fc4ec075bd0 2024-03-28T03:44:54.948 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:54.944+0000 7fc4f1800640 1 -- 172.21.15.119:0/688084142 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4ec1a3820 con 0x7fc4ec075bd0 2024-03-28T03:44:54.948 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:54.944+0000 7fc4f1800640 1 -- 172.21.15.119:0/688084142 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4ec1ab110 con 0x7fc4ec075bd0 2024-03-28T03:44:54.949 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:54.945+0000 7fc4f1800640 1 -- 172.21.15.119:0/688084142 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4ec077310 con 0x7fc4ec075bd0 2024-03-28T03:44:54.949 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:54.947+0000 7fc4e3400640 1 -- 172.21.15.119:0/688084142 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 27) v1 ==== 137422+0+0 (secure 0 0 0) 0x7fc4dc046020 con 0x7fc4ec075bd0 2024-03-28T03:44:54.949 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:54.948+0000 7fc4e3400640 1 --2- 172.21.15.119:0/688084142 >> [v2:172.21.15.119:6800/4273482239,v1:172.21.15.119:6801/4273482239] conn(0x7fc4d00a29f0 0x7fc4d00a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T03:44:54.950 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:54.948+0000 7fc4e3400640 1 -- 172.21.15.119:0/688084142 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 11163+0+0 (secure 0 0 0) 0x7fc4dc0f1bb0 con 0x7fc4ec075bd0 2024-03-28T03:44:54.950 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:54.948+0000 7fc4e3e00640 1 --2- 172.21.15.119:0/688084142 >> [v2:172.21.15.119:6800/4273482239,v1:172.21.15.119:6801/4273482239] conn(0x7fc4d00a29f0 0x7fc4d00a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T03:44:54.950 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:54.949+0000 7fc4e3e00640 1 --2- 172.21.15.119:0/688084142 >> [v2:172.21.15.119:6800/4273482239,v1:172.21.15.119:6801/4273482239] conn(0x7fc4d00a29f0 0x7fc4d00a4eb0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fc4d40095f0 tx=0x7fc4d4005f90 comp rx=0 tx=0).ready entity=mgr.25277 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T03:44:54.957 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:54.955+0000 7fc4e3400640 1 -- 172.21.15.119:0/688084142 <== mon.2 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fc4dc04d020 con 0x7fc4ec075bd0 2024-03-28T03:44:55.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:44:54 smithi119 ceph-mon[31518]: pgmap v6201: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:44:55.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:44:54 smithi119 ceph-mon[31518]: from='client.? 172.21.15.119:0/1198147085' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-28T03:44:55.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:44:54 smithi119 ceph-mon[31518]: from='client.? 172.21.15.119:0/1515627306' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-28T03:44:55.155 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:55.153+0000 7fc4f1800640 1 -- 172.21.15.119:0/688084142 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc4ec10d670 con 0x7fc4ec075bd0 2024-03-28T03:44:55.156 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:55.154+0000 7fc4e3400640 1 -- 172.21.15.119:0/688084142 <== mon.2 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v125) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc4dc0bdf70 con 0x7fc4ec075bd0 2024-03-28T03:44:55.157 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 64 2024-03-28T03:44:55.160 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:55.157+0000 7fc4f1800640 1 -- 172.21.15.119:0/688084142 >> [v2:172.21.15.119:6800/4273482239,v1:172.21.15.119:6801/4273482239] conn(0x7fc4d00a29f0 msgr2=0x7fc4d00a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T03:44:55.160 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:55.157+0000 7fc4f1800640 1 --2- 172.21.15.119:0/688084142 >> [v2:172.21.15.119:6800/4273482239,v1:172.21.15.119:6801/4273482239] conn(0x7fc4d00a29f0 0x7fc4d00a4eb0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fc4d40095f0 tx=0x7fc4d4005f90 comp rx=0 tx=0).stop 2024-03-28T03:44:55.160 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:55.157+0000 7fc4f1800640 1 -- 172.21.15.119:0/688084142 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc4ec075bd0 msgr2=0x7fc4ec19e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T03:44:55.160 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:55.157+0000 7fc4f1800640 1 --2- 172.21.15.119:0/688084142 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc4ec075bd0 0x7fc4ec19e5f0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fc4dc005ab0 tx=0x7fc4dc005e80 comp rx=0 tx=0).stop 2024-03-28T03:44:55.160 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:55.158+0000 7fc4f1800640 1 -- 172.21.15.119:0/688084142 shutdown_connections 2024-03-28T03:44:55.161 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:55.158+0000 7fc4f1800640 1 --2- 172.21.15.119:0/688084142 >> [v2:172.21.15.119:6800/4273482239,v1:172.21.15.119:6801/4273482239] conn(0x7fc4d00a29f0 0x7fc4d00a4eb0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:44:55.161 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:55.158+0000 7fc4f1800640 1 --2- 172.21.15.119:0/688084142 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc4ec1124e0 0x7fc4ec1a2e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:44:55.161 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:55.158+0000 7fc4f1800640 1 --2- 172.21.15.119:0/688084142 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc4ec0764f0 0x7fc4ec19eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:44:55.161 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:55.158+0000 7fc4f1800640 1 --2- 172.21.15.119:0/688084142 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc4ec075bd0 0x7fc4ec19e5f0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:44:55.161 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:55.158+0000 7fc4f1800640 1 -- 172.21.15.119:0/688084142 >> 172.21.15.119:0/688084142 conn(0x7fc4ec0ff2f0 msgr2=0x7fc4ec1016e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T03:44:55.161 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:55.158+0000 7fc4f1800640 1 -- 172.21.15.119:0/688084142 shutdown_connections 2024-03-28T03:44:55.161 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:55.158+0000 7fc4f1800640 1 -- 172.21.15.119:0/688084142 wait complete. 2024-03-28T03:44:55.180 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-28T03:44:55.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:44:54 smithi165 ceph-mon[36302]: pgmap v6201: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:44:55.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:44:54 smithi165 ceph-mon[36302]: from='client.? 172.21.15.119:0/1198147085' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-28T03:44:55.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:44:54 smithi165 ceph-mon[36302]: from='client.? 172.21.15.119:0/1515627306' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-28T03:44:55.392 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:55.389+0000 7fb214800640 1 -- 172.21.15.119:0/4186142110 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb20c109850 msgr2=0x7fb20c109c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T03:44:55.392 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:55.389+0000 7fb214800640 1 --2- 172.21.15.119:0/4186142110 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb20c109850 0x7fb20c109c30 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fb204009cd0 tx=0x7fb20402c0f0 comp rx=0 tx=0).stop 2024-03-28T03:44:55.392 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:55.390+0000 7fb214800640 1 -- 172.21.15.119:0/4186142110 shutdown_connections 2024-03-28T03:44:55.392 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:55.390+0000 7fb214800640 1 --2- 172.21.15.119:0/4186142110 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb20c104200 0x7fb20c110710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:44:55.392 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:55.390+0000 7fb214800640 1 --2- 172.21.15.119:0/4186142110 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb20c103840 0x7fb20c103cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:44:55.392 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:55.390+0000 7fb214800640 1 --2- 172.21.15.119:0/4186142110 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb20c109850 0x7fb20c109c30 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:44:55.393 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:55.390+0000 7fb214800640 1 -- 172.21.15.119:0/4186142110 >> 172.21.15.119:0/4186142110 conn(0x7fb20c0ff350 msgr2=0x7fb20c101770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T03:44:55.393 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:55.390+0000 7fb214800640 1 -- 172.21.15.119:0/4186142110 shutdown_connections 2024-03-28T03:44:55.393 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:55.390+0000 7fb214800640 1 -- 172.21.15.119:0/4186142110 wait complete. 2024-03-28T03:44:55.395 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:55.391+0000 7fb214800640 1 Processor -- start 2024-03-28T03:44:55.395 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:55.391+0000 7fb214800640 1 -- start start 2024-03-28T03:44:55.395 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:55.391+0000 7fb214800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb20c103840 0x7fb20c19a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T03:44:55.395 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:55.391+0000 7fb214800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb20c104200 0x7fb20c19a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T03:44:55.395 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:55.391+0000 7fb214800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb20c109850 0x7fb20c19ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T03:44:55.396 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:55.391+0000 7fb214800640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb20c113c60 con 0x7fb20c109850 2024-03-28T03:44:55.396 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:55.391+0000 7fb214800640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb20c113ae0 con 0x7fb20c103840 2024-03-28T03:44:55.396 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:55.391+0000 7fb214800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb20c113de0 con 0x7fb20c104200 2024-03-28T03:44:55.396 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:55.392+0000 7fb211e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb20c103840 0x7fb20c19a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T03:44:55.396 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:55.392+0000 7fb211e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb20c103840 0x7fb20c19a1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.119:38452/0 (socket says 172.21.15.119:38452) 2024-03-28T03:44:55.396 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:55.392+0000 7fb211e00640 1 -- 172.21.15.119:0/482524555 learned_addr learned my addr 172.21.15.119:0/482524555 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T03:44:55.396 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:55.392+0000 7fb212800640 1 --2- 172.21.15.119:0/482524555 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb20c109850 0x7fb20c19ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T03:44:55.396 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:55.392+0000 7fb211400640 1 --2- 172.21.15.119:0/482524555 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb20c104200 0x7fb20c19a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T03:44:55.396 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:55.392+0000 7fb211e00640 1 -- 172.21.15.119:0/482524555 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb20c104200 msgr2=0x7fb20c19a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T03:44:55.397 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:55.392+0000 7fb211e00640 1 --2- 172.21.15.119:0/482524555 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb20c104200 0x7fb20c19a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:44:55.397 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:55.392+0000 7fb211e00640 1 -- 172.21.15.119:0/482524555 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb20c109850 msgr2=0x7fb20c19ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T03:44:55.397 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:55.392+0000 7fb211e00640 1 --2- 172.21.15.119:0/482524555 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb20c109850 0x7fb20c19ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:44:55.397 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:55.392+0000 7fb211e00640 1 -- 172.21.15.119:0/482524555 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb20c19f190 con 0x7fb20c103840 2024-03-28T03:44:55.397 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:55.392+0000 7fb211400640 1 --2- 172.21.15.119:0/482524555 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb20c104200 0x7fb20c19a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T03:44:55.397 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:55.392+0000 7fb212800640 1 --2- 172.21.15.119:0/482524555 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb20c109850 0x7fb20c19ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T03:44:55.397 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:55.392+0000 7fb211e00640 1 --2- 172.21.15.119:0/482524555 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb20c103840 0x7fb20c19a1a0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fb204004af0 tx=0x7fb204032ea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T03:44:55.397 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:55.392+0000 7fb202a00640 1 -- 172.21.15.119:0/482524555 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb204041020 con 0x7fb20c103840 2024-03-28T03:44:55.397 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:55.392+0000 7fb214800640 1 -- 172.21.15.119:0/482524555 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb20c19f420 con 0x7fb20c103840 2024-03-28T03:44:55.398 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:55.392+0000 7fb202a00640 1 -- 172.21.15.119:0/482524555 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb204034cd0 con 0x7fb20c103840 2024-03-28T03:44:55.398 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:55.392+0000 7fb202a00640 1 -- 172.21.15.119:0/482524555 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb204039040 con 0x7fb20c103840 2024-03-28T03:44:55.398 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:55.393+0000 7fb214800640 1 -- 172.21.15.119:0/482524555 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb20c1a6d10 con 0x7fb20c103840 2024-03-28T03:44:55.399 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:55.394+0000 7fb214800640 1 -- 172.21.15.119:0/482524555 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb20c0688e0 con 0x7fb20c103840 2024-03-28T03:44:55.399 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:55.395+0000 7fb202a00640 1 -- 172.21.15.119:0/482524555 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 27) v1 ==== 137422+0+0 (secure 0 0 0) 0x7fb204044020 con 0x7fb20c103840 2024-03-28T03:44:55.399 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:55.397+0000 7fb202a00640 1 --2- 172.21.15.119:0/482524555 >> [v2:172.21.15.119:6800/4273482239,v1:172.21.15.119:6801/4273482239] conn(0x7fb1f80a2970 0x7fb1f80a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T03:44:55.399 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:55.397+0000 7fb202a00640 1 -- 172.21.15.119:0/482524555 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 11163+0+0 (secure 0 0 0) 0x7fb2040f1490 con 0x7fb20c103840 2024-03-28T03:44:55.399 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:55.397+0000 7fb211400640 1 --2- 172.21.15.119:0/482524555 >> [v2:172.21.15.119:6800/4273482239,v1:172.21.15.119:6801/4273482239] conn(0x7fb1f80a2970 0x7fb1f80a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T03:44:55.399 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:55.398+0000 7fb211400640 1 --2- 172.21.15.119:0/482524555 >> [v2:172.21.15.119:6800/4273482239,v1:172.21.15.119:6801/4273482239] conn(0x7fb1f80a2970 0x7fb1f80a4e30 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fb20c1019f0 tx=0x7fb1fc005f90 comp rx=0 tx=0).ready entity=mgr.25277 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T03:44:55.406 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:55.405+0000 7fb202a00640 1 -- 172.21.15.119:0/482524555 <== mon.2 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fb20404b020 con 0x7fb20c103840 2024-03-28T03:44:55.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:44:54 smithi132 ceph-mon[35941]: pgmap v6201: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:44:55.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:44:54 smithi132 ceph-mon[35941]: from='client.? 172.21.15.119:0/1198147085' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-28T03:44:55.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:44:54 smithi132 ceph-mon[35941]: from='client.? 172.21.15.119:0/1515627306' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-28T03:44:55.622 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:55.620+0000 7fb214800640 1 -- 172.21.15.119:0/482524555 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fb20c10bdb0 con 0x7fb20c103840 2024-03-28T03:44:55.623 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:55.622+0000 7fb202a00640 1 -- 172.21.15.119:0/482524555 <== mon.2 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 80 entries v125) v1 ==== 81+0+4913 (secure 0 0 0) 0x7fb2040bd850 con 0x7fb20c103840 2024-03-28T03:44:55.624 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:6836/2109630829 2024-03-29T00:35:52.413013+0000 2024-03-28T03:44:55.624 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.165:6832/2013665520 2024-03-29T00:32:07.607942+0000 2024-03-28T03:44:55.624 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.165:6839/384483120 2024-03-29T00:32:07.271819+0000 2024-03-28T03:44:55.624 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.165:6838/384483120 2024-03-29T00:32:07.271819+0000 2024-03-28T03:44:55.624 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:6834/744950487 2024-03-29T00:31:51.094723+0000 2024-03-28T03:44:55.624 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.165:6839/768712672 2024-03-29T00:33:20.841067+0000 2024-03-28T03:44:55.625 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:0/87565965 2024-03-29T00:16:03.582545+0000 2024-03-28T03:44:55.625 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:6834/2793569574 2024-03-29T00:35:34.679932+0000 2024-03-28T03:44:55.625 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.165:6839/3004684592 2024-03-29T00:31:32.429788+0000 2024-03-28T03:44:55.625 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.165:6833/1389081045 2024-03-29T00:31:20.067892+0000 2024-03-28T03:44:55.625 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.165:6832/609963168 2024-03-29T00:30:41.299818+0000 2024-03-28T03:44:55.625 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:6834/2941089726 2024-03-29T00:33:37.236449+0000 2024-03-28T03:44:55.625 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.165:6839/2762859081 2024-03-29T00:33:59.786505+0000 2024-03-28T03:44:55.625 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.165:6833/609963168 2024-03-29T00:30:41.299818+0000 2024-03-28T03:44:55.625 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.165:6837/2717860763 2024-03-29T00:33:02.702892+0000 2024-03-28T03:44:55.626 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.165:6836/3170852275 2024-03-29T00:32:25.939039+0000 2024-03-28T03:44:55.626 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:6800/2056426384 2024-03-29T00:15:30.472283+0000 2024-03-28T03:44:55.626 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:6801/2575246519 2024-03-29T00:16:03.582545+0000 2024-03-28T03:44:55.626 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.165:6836/1531193032 2024-03-29T00:30:24.146598+0000 2024-03-28T03:44:55.626 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:6834/1659925849 2024-03-29T00:34:17.464068+0000 2024-03-28T03:44:55.626 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:6835/744950487 2024-03-29T00:31:51.094723+0000 2024-03-28T03:44:55.626 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.165:6836/1456402510 2024-03-29T00:30:56.719761+0000 2024-03-28T03:44:55.626 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.165:6837/1456402510 2024-03-29T00:30:56.719761+0000 2024-03-28T03:44:55.626 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.165:6839/1687390659 2024-03-29T00:32:46.777861+0000 2024-03-28T03:44:55.627 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:0/2635982178 2024-03-29T00:16:03.582545+0000 2024-03-28T03:44:55.627 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:0/2430401731 2024-03-29T00:26:32.965486+0000 2024-03-28T03:44:55.627 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:6837/482178170 2024-03-29T00:35:17.974473+0000 2024-03-28T03:44:55.627 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:6801/1401937096 2024-03-29T00:26:32.965486+0000 2024-03-28T03:44:55.627 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:0/4201699272 2024-03-29T00:16:03.582545+0000 2024-03-28T03:44:55.627 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.165:6838/3004684592 2024-03-29T00:31:32.429788+0000 2024-03-28T03:44:55.627 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.165:6836/1011925619 2024-03-29T00:31:49.271822+0000 2024-03-28T03:44:55.627 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.165:6837/1531193032 2024-03-29T00:30:24.146598+0000 2024-03-28T03:44:55.627 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.165:6833/2013665520 2024-03-29T00:32:07.607942+0000 2024-03-28T03:44:55.628 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:6834/2101147835 2024-03-29T00:32:28.920424+0000 2024-03-28T03:44:55.628 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:6835/3517971302 2024-03-29T00:35:17.389381+0000 2024-03-28T03:44:55.628 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.165:6832/1389081045 2024-03-29T00:31:20.067892+0000 2024-03-28T03:44:55.628 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:6835/2210305461 2024-03-29T00:33:02.940231+0000 2024-03-28T03:44:55.628 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:0/2891240357 2024-03-29T00:15:30.472283+0000 2024-03-28T03:44:55.628 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:6838/1595343240 2024-03-29T00:34:16.710959+0000 2024-03-28T03:44:55.628 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:6801/2056426384 2024-03-29T00:15:30.472283+0000 2024-03-28T03:44:55.628 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:0/357445067 2024-03-29T00:26:32.965486+0000 2024-03-28T03:44:55.628 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.165:6838/1687390659 2024-03-29T00:32:46.777861+0000 2024-03-28T03:44:55.629 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:6838/3947365264 2024-03-29T00:34:59.827426+0000 2024-03-28T03:44:55.629 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.165:6837/1011925619 2024-03-29T00:31:49.271822+0000 2024-03-28T03:44:55.629 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:6800/2575246519 2024-03-29T00:16:03.582545+0000 2024-03-28T03:44:55.629 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.165:6836/823338130 2024-03-29T00:33:39.774934+0000 2024-03-28T03:44:55.629 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:0/468478300 2024-03-29T00:15:30.472283+0000 2024-03-28T03:44:55.629 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:0/4119730435 2024-03-29T00:26:32.965486+0000 2024-03-28T03:44:55.629 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:6800/1401937096 2024-03-29T00:26:32.965486+0000 2024-03-28T03:44:55.629 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:0/524303043 2024-03-29T00:26:32.965486+0000 2024-03-28T03:44:55.629 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:6835/2101147835 2024-03-29T00:32:28.920424+0000 2024-03-28T03:44:55.630 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.165:6832/4070507195 2024-03-29T00:32:46.507920+0000 2024-03-28T03:44:55.630 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:6836/3988635716 2024-03-29T00:34:37.192476+0000 2024-03-28T03:44:55.630 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.165:6836/2717860763 2024-03-29T00:33:02.702892+0000 2024-03-28T03:44:55.630 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:6834/2210305461 2024-03-29T00:33:02.940231+0000 2024-03-28T03:44:55.630 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.165:6832/3417761142 2024-03-29T00:33:20.267933+0000 2024-03-28T03:44:55.630 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.165:6838/768712672 2024-03-29T00:33:20.841067+0000 2024-03-28T03:44:55.630 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:6835/2941089726 2024-03-29T00:33:37.236449+0000 2024-03-28T03:44:55.630 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.165:6837/3170852275 2024-03-29T00:32:25.939039+0000 2024-03-28T03:44:55.631 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:6839/1595343240 2024-03-29T00:34:16.710959+0000 2024-03-28T03:44:55.631 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.165:6833/4070507195 2024-03-29T00:32:46.507920+0000 2024-03-28T03:44:55.631 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.165:6837/823338130 2024-03-29T00:33:39.774934+0000 2024-03-28T03:44:55.631 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:6834/4197433095 2024-03-29T00:34:37.186007+0000 2024-03-28T03:44:55.631 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.165:6832/1344198130 2024-03-29T00:33:59.786346+0000 2024-03-28T03:44:55.631 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:6837/3511403986 2024-03-29T00:35:35.303954+0000 2024-03-28T03:44:55.631 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.165:6838/2762859081 2024-03-29T00:33:59.786505+0000 2024-03-28T03:44:55.631 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:6835/4197433095 2024-03-29T00:34:37.186007+0000 2024-03-28T03:44:55.631 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:6836/482178170 2024-03-29T00:35:17.974473+0000 2024-03-28T03:44:55.632 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.165:6833/3417761142 2024-03-29T00:33:20.267933+0000 2024-03-28T03:44:55.632 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.165:6833/1344198130 2024-03-29T00:33:59.786346+0000 2024-03-28T03:44:55.632 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:6835/1659925849 2024-03-29T00:34:17.464068+0000 2024-03-28T03:44:55.632 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:6837/3988635716 2024-03-29T00:34:37.192476+0000 2024-03-28T03:44:55.632 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:6836/3511403986 2024-03-29T00:35:35.303954+0000 2024-03-28T03:44:55.632 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:6834/3816731508 2024-03-29T00:34:54.192187+0000 2024-03-28T03:44:55.632 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:6837/2109630829 2024-03-29T00:35:52.413013+0000 2024-03-28T03:44:55.632 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:6835/3816731508 2024-03-29T00:34:54.192187+0000 2024-03-28T03:44:55.632 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:6839/3947365264 2024-03-29T00:34:59.827426+0000 2024-03-28T03:44:55.633 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:0/1639109126 2024-03-29T00:15:30.472283+0000 2024-03-28T03:44:55.633 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:6834/3517971302 2024-03-29T00:35:17.389381+0000 2024-03-28T03:44:55.633 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:6835/2793569574 2024-03-29T00:35:34.679932+0000 2024-03-28T03:44:55.634 INFO:teuthology.orchestra.run.smithi119.stderr:listed 80 entries 2024-03-28T03:44:55.634 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:55.625+0000 7fb214800640 1 -- 172.21.15.119:0/482524555 >> [v2:172.21.15.119:6800/4273482239,v1:172.21.15.119:6801/4273482239] conn(0x7fb1f80a2970 msgr2=0x7fb1f80a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T03:44:55.634 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:55.625+0000 7fb214800640 1 --2- 172.21.15.119:0/482524555 >> [v2:172.21.15.119:6800/4273482239,v1:172.21.15.119:6801/4273482239] conn(0x7fb1f80a2970 0x7fb1f80a4e30 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fb20c1019f0 tx=0x7fb1fc005f90 comp rx=0 tx=0).stop 2024-03-28T03:44:55.634 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:55.625+0000 7fb214800640 1 -- 172.21.15.119:0/482524555 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb20c103840 msgr2=0x7fb20c19a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T03:44:55.634 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:55.625+0000 7fb214800640 1 --2- 172.21.15.119:0/482524555 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb20c103840 0x7fb20c19a1a0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fb204004af0 tx=0x7fb204032ea0 comp rx=0 tx=0).stop 2024-03-28T03:44:55.634 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:55.625+0000 7fb214800640 1 -- 172.21.15.119:0/482524555 shutdown_connections 2024-03-28T03:44:55.634 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:55.625+0000 7fb214800640 1 --2- 172.21.15.119:0/482524555 >> [v2:172.21.15.119:6800/4273482239,v1:172.21.15.119:6801/4273482239] conn(0x7fb1f80a2970 0x7fb1f80a4e30 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:44:55.635 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:55.625+0000 7fb214800640 1 --2- 172.21.15.119:0/482524555 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb20c109850 0x7fb20c19ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:44:55.635 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:55.625+0000 7fb214800640 1 --2- 172.21.15.119:0/482524555 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb20c104200 0x7fb20c19a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:44:55.635 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:55.625+0000 7fb214800640 1 --2- 172.21.15.119:0/482524555 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb20c103840 0x7fb20c19a1a0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:44:55.635 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:55.625+0000 7fb214800640 1 -- 172.21.15.119:0/482524555 >> 172.21.15.119:0/482524555 conn(0x7fb20c0ff350 msgr2=0x7fb20c0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T03:44:55.635 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:55.625+0000 7fb214800640 1 -- 172.21.15.119:0/482524555 shutdown_connections 2024-03-28T03:44:55.635 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:44:55.625+0000 7fb214800640 1 -- 172.21.15.119:0/482524555 wait complete. 2024-03-28T03:44:55.647 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-03-28T03:44:55.647 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T03:44:55.647 DEBUG:teuthology.orchestra.run.smithi119:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-03-28T03:44:56.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:44:55 smithi165 ceph-mon[36302]: from='client.? 172.21.15.119:0/688084142' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-28T03:44:56.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:44:55 smithi165 ceph-mon[36302]: from='client.? 172.21.15.119:0/482524555' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-28T03:44:56.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:44:55 smithi119 ceph-mon[31518]: from='client.? 172.21.15.119:0/688084142' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-28T03:44:56.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:44:55 smithi119 ceph-mon[31518]: from='client.? 172.21.15.119:0/482524555' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-28T03:44:56.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:44:55 smithi132 ceph-mon[35941]: from='client.? 172.21.15.119:0/688084142' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-28T03:44:56.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:44:55 smithi132 ceph-mon[35941]: from='client.? 172.21.15.119:0/482524555' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-28T03:44:57.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:44:56 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 11355.821904 secs 2024-03-28T03:44:57.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:44:56 smithi119 ceph-mon[31518]: pgmap v6202: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:44:57.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:44:56 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 11355.821904 secs 2024-03-28T03:44:57.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:44:56 smithi165 ceph-mon[36302]: pgmap v6202: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:44:57.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:44:56 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 11355.821904 secs 2024-03-28T03:44:57.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:44:56 smithi132 ceph-mon[35941]: pgmap v6202: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:44:59.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:44:58 smithi119 ceph-mon[31518]: pgmap v6203: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:44:59.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:44:58 smithi165 ceph-mon[36302]: pgmap v6203: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:44:59.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:44:58 smithi132 ceph-mon[35941]: pgmap v6203: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:45:01.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:45:00 smithi165 ceph-mon[36302]: pgmap v6204: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:45:01.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:45:00 smithi119 ceph-mon[31518]: pgmap v6204: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:45:01.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:45:00 smithi132 ceph-mon[35941]: pgmap v6204: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:45:02.279 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:45:01 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 11360.821964 secs 2024-03-28T03:45:02.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:45:01 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 11360.821964 secs 2024-03-28T03:45:02.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:45:01 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 11360.821964 secs 2024-03-28T03:45:03.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:45:02 smithi165 ceph-mon[36302]: pgmap v6205: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:45:03.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:45:02 smithi119 ceph-mon[31518]: pgmap v6205: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:45:03.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:45:02 smithi132 ceph-mon[35941]: pgmap v6205: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:45:05.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:45:04 smithi165 ceph-mon[36302]: pgmap v6206: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:45:05.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:45:04 smithi119 ceph-mon[31518]: pgmap v6206: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:45:05.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:45:04 smithi132 ceph-mon[35941]: pgmap v6206: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:45:07.250 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:45:06 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 11365.822025 secs 2024-03-28T03:45:07.251 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:45:06 smithi132 ceph-mon[35941]: pgmap v6207: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:45:07.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:45:06 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 11365.822025 secs 2024-03-28T03:45:07.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:45:06 smithi165 ceph-mon[36302]: pgmap v6207: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:45:07.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:45:06 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 11365.822025 secs 2024-03-28T03:45:07.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:45:06 smithi119 ceph-mon[31518]: pgmap v6207: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:45:09.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:45:09 smithi119 ceph-mon[31518]: pgmap v6208: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:45:09.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:45:09 smithi165 ceph-mon[36302]: pgmap v6208: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:45:09.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:45:09 smithi132 ceph-mon[35941]: pgmap v6208: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:45:11.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:45:11 smithi165 ceph-mon[36302]: pgmap v6209: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:45:11.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:45:11 smithi119 ceph-mon[31518]: pgmap v6209: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:45:11.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:45:11 smithi132 ceph-mon[35941]: pgmap v6209: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:45:12.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:45:12 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 11370.822093 secs 2024-03-28T03:45:12.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:45:12 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 11370.822093 secs 2024-03-28T03:45:12.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:45:12 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 11370.822093 secs 2024-03-28T03:45:13.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:45:13 smithi165 ceph-mon[36302]: pgmap v6210: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:45:13.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:45:13 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T03:45:13.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:45:13 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:45:13.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:45:13 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:45:13.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:45:13 smithi119 ceph-mon[31518]: pgmap v6210: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:45:13.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:45:13 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T03:45:13.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:45:13 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:45:13.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:45:13 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:45:13.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:45:13 smithi132 ceph-mon[35941]: pgmap v6210: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:45:13.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:45:13 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T03:45:13.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:45:13 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:45:13.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:45:13 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:45:14.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:45:14 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T03:45:14.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:45:14 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T03:45:14.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:45:14 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:45:14.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:45:14 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T03:45:14.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:45:14 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T03:45:14.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:45:14 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:45:14.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:45:14 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T03:45:14.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:45:14 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T03:45:14.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:45:14 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:45:15.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:45:15 smithi165 ceph-mon[36302]: pgmap v6211: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T03:45:15.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:45:15 smithi165 ceph-mon[36302]: pgmap v6212: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:45:15.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:45:15 smithi119 ceph-mon[31518]: pgmap v6211: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T03:45:15.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:45:15 smithi119 ceph-mon[31518]: pgmap v6212: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:45:15.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:45:15 smithi132 ceph-mon[35941]: pgmap v6211: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-28T03:45:15.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:45:15 smithi132 ceph-mon[35941]: pgmap v6212: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:45:17.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:45:17 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 11375.822172 secs 2024-03-28T03:45:17.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:45:17 smithi165 ceph-mon[36302]: pgmap v6213: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:45:17.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:45:17 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 11375.822172 secs 2024-03-28T03:45:17.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:45:17 smithi119 ceph-mon[31518]: pgmap v6213: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:45:17.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:45:17 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 11375.822172 secs 2024-03-28T03:45:17.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:45:17 smithi132 ceph-mon[35941]: pgmap v6213: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:45:19.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:45:19 smithi165 ceph-mon[36302]: pgmap v6214: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:45:19.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:45:19 smithi119 ceph-mon[31518]: pgmap v6214: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:45:19.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:45:19 smithi132 ceph-mon[35941]: pgmap v6214: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:45:21.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:45:21 smithi165 ceph-mon[36302]: pgmap v6215: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:45:21.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:45:21 smithi119 ceph-mon[31518]: pgmap v6215: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:45:21.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:45:21 smithi132 ceph-mon[35941]: pgmap v6215: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:45:22.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:45:22 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 11380.822211 secs 2024-03-28T03:45:22.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:45:22 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 11380.822211 secs 2024-03-28T03:45:22.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:45:22 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 11380.822211 secs 2024-03-28T03:45:23.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:45:23 smithi165 ceph-mon[36302]: pgmap v6216: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:45:23.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:45:23 smithi119 ceph-mon[31518]: pgmap v6216: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:45:23.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:45:23 smithi132 ceph-mon[35941]: pgmap v6216: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T03:45:25.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:45:25 smithi165 ceph-mon[36302]: pgmap v6217: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T03:45:25.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:45:25 smithi119 ceph-mon[31518]: pgmap v6217: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T03:45:25.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:45:25 smithi132 ceph-mon[35941]: pgmap v6217: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T03:45:27.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:45:27 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 11385.822301 secs 2024-03-28T03:45:27.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:45:27 smithi165 ceph-mon[36302]: pgmap v6218: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:45:27.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:45:27 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 11385.822301 secs 2024-03-28T03:45:27.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:45:27 smithi119 ceph-mon[31518]: pgmap v6218: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:45:27.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:45:27 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 11385.822301 secs 2024-03-28T03:45:27.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:45:27 smithi132 ceph-mon[35941]: pgmap v6218: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:45:29.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:45:29 smithi165 ceph-mon[36302]: pgmap v6219: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:45:29.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:45:29 smithi119 ceph-mon[31518]: pgmap v6219: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:45:29.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:45:29 smithi132 ceph-mon[35941]: pgmap v6219: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:45:31.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:45:31 smithi165 ceph-mon[36302]: pgmap v6220: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:45:31.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:45:31 smithi119 ceph-mon[31518]: pgmap v6220: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:45:31.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:45:31 smithi132 ceph-mon[35941]: pgmap v6220: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:45:32.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:45:32 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 11390.822389 secs 2024-03-28T03:45:32.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:45:32 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 11390.822389 secs 2024-03-28T03:45:32.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:45:32 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 11390.822389 secs 2024-03-28T03:45:33.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:45:33 smithi165 ceph-mon[36302]: pgmap v6221: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:45:33.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:45:33 smithi119 ceph-mon[31518]: pgmap v6221: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:45:33.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:45:33 smithi132 ceph-mon[35941]: pgmap v6221: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:45:35.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:45:35 smithi165 ceph-mon[36302]: pgmap v6222: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:45:35.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:45:35 smithi119 ceph-mon[31518]: pgmap v6222: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:45:35.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:45:35 smithi132 ceph-mon[35941]: pgmap v6222: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:45:37.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:45:37 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 11395.822451 secs 2024-03-28T03:45:37.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:45:37 smithi165 ceph-mon[36302]: pgmap v6223: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:45:37.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:45:37 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 11395.822451 secs 2024-03-28T03:45:37.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:45:37 smithi119 ceph-mon[31518]: pgmap v6223: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:45:37.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:45:37 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 11395.822451 secs 2024-03-28T03:45:37.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:45:37 smithi132 ceph-mon[35941]: pgmap v6223: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:45:39.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:45:39 smithi165 ceph-mon[36302]: pgmap v6224: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:45:39.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:45:39 smithi119 ceph-mon[31518]: pgmap v6224: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:45:39.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:45:39 smithi132 ceph-mon[35941]: pgmap v6224: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:45:41.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:45:41 smithi165 ceph-mon[36302]: pgmap v6225: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:45:41.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:45:41 smithi119 ceph-mon[31518]: pgmap v6225: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:45:41.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:45:41 smithi132 ceph-mon[35941]: pgmap v6225: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:45:42.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:45:42 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 11400.822511 secs 2024-03-28T03:45:42.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:45:42 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 11400.822511 secs 2024-03-28T03:45:42.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:45:42 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 11400.822511 secs 2024-03-28T03:45:43.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:45:43 smithi165 ceph-mon[36302]: pgmap v6226: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T03:45:43.796 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:45:43 smithi132 ceph-mon[35941]: pgmap v6226: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T03:45:43.796 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:45:43 smithi119 ceph-mon[31518]: pgmap v6226: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T03:45:44.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:45:44 smithi119 ceph-mon[31518]: pgmap v6227: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T03:45:44.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:45:44 smithi132 ceph-mon[35941]: pgmap v6227: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T03:45:44.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:45:44 smithi165 ceph-mon[36302]: pgmap v6227: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T03:45:46.878 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:45:46 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 11405.822546 secs 2024-03-28T03:45:46.878 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:45:46 smithi165 ceph-mon[36302]: pgmap v6228: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:45:46.901 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:45:46 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 11405.822546 secs 2024-03-28T03:45:46.901 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:45:46 smithi119 ceph-mon[31518]: pgmap v6228: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:45:46.948 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:45:46 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 11405.822546 secs 2024-03-28T03:45:46.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:45:46 smithi132 ceph-mon[35941]: pgmap v6228: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:45:48.878 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:45:48 smithi165 ceph-mon[36302]: pgmap v6229: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T03:45:48.901 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:45:48 smithi119 ceph-mon[31518]: pgmap v6229: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T03:45:48.948 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:45:48 smithi132 ceph-mon[35941]: pgmap v6229: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-28T03:45:50.878 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:45:50 smithi165 ceph-mon[36302]: pgmap v6230: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:45:50.901 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:45:50 smithi119 ceph-mon[31518]: pgmap v6230: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:45:50.948 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:45:50 smithi132 ceph-mon[35941]: pgmap v6230: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:45:51.901 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:45:51 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 11410.822635 secs 2024-03-28T03:45:51.948 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:45:51 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 11410.822635 secs 2024-03-28T03:45:52.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:45:51 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 11410.822635 secs 2024-03-28T03:45:52.901 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:45:52 smithi119 ceph-mon[31518]: pgmap v6231: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:45:52.948 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:45:52 smithi132 ceph-mon[35941]: pgmap v6231: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:45:53.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:45:52 smithi165 ceph-mon[36302]: pgmap v6231: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:45:54.901 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:45:54 smithi119 ceph-mon[31518]: pgmap v6232: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:45:54.948 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:45:54 smithi132 ceph-mon[35941]: pgmap v6232: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:45:55.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:45:54 smithi165 ceph-mon[36302]: pgmap v6232: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:45:56.901 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:45:56 smithi119 ceph-mon[31518]: 98 slow requests, 0 included below; oldest blocked for > 11415.822730 secs 2024-03-28T03:45:56.901 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:45:56 smithi119 ceph-mon[31518]: pgmap v6233: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:45:56.948 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:45:56 smithi132 ceph-mon[35941]: 98 slow requests, 0 included below; oldest blocked for > 11415.822730 secs 2024-03-28T03:45:56.948 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:45:56 smithi132 ceph-mon[35941]: pgmap v6233: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:45:57.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:45:56 smithi165 ceph-mon[36302]: 98 slow requests, 0 included below; oldest blocked for > 11415.822730 secs 2024-03-28T03:45:57.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:45:56 smithi165 ceph-mon[36302]: pgmap v6233: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:45:58.901 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:45:58 smithi119 ceph-mon[31518]: pgmap v6234: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:45:58.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:45:58 smithi132 ceph-mon[35941]: pgmap v6234: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:45:59.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:45:58 smithi165 ceph-mon[36302]: pgmap v6234: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:46:00.901 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:46:00 smithi119 ceph-mon[31518]: pgmap v6235: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 8 op/s 2024-03-28T03:46:00.948 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:46:00 smithi132 ceph-mon[35941]: pgmap v6235: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 8 op/s 2024-03-28T03:46:01.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:46:00 smithi165 ceph-mon[36302]: pgmap v6235: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 8 op/s 2024-03-28T03:46:01.948 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:46:01 smithi132 ceph-mon[35941]: 42 slow requests, 0 included below; oldest blocked for > 11420.822791 secs 2024-03-28T03:46:02.029 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:46:01 smithi165 ceph-mon[36302]: 42 slow requests, 0 included below; oldest blocked for > 11420.822791 secs 2024-03-28T03:46:02.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:46:01 smithi119 ceph-mon[31518]: 42 slow requests, 0 included below; oldest blocked for > 11420.822791 secs 2024-03-28T03:46:02.948 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:46:02 smithi132 ceph-mon[35941]: pgmap v6236: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-03-28T03:46:02.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:46:02 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.132:6838/3305822525,v1:172.21.15.132:6839/3305822525] up:active 2024-03-28T03:46:02.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:46:02 smithi132 ceph-mon[35941]: fsmap cephfs:5/5 3 up:standby-replay 1 up:resolve 4 up:active 2024-03-28T03:46:03.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:46:02 smithi165 ceph-mon[36302]: pgmap v6236: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-03-28T03:46:03.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:46:02 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.132:6838/3305822525,v1:172.21.15.132:6839/3305822525] up:active 2024-03-28T03:46:03.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:46:02 smithi165 ceph-mon[36302]: fsmap cephfs:5/5 3 up:standby-replay 1 up:resolve 4 up:active 2024-03-28T03:46:03.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:46:02 smithi119 ceph-mon[31518]: pgmap v6236: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-03-28T03:46:03.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:46:02 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.132:6838/3305822525,v1:172.21.15.132:6839/3305822525] up:active 2024-03-28T03:46:03.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:46:02 smithi119 ceph-mon[31518]: fsmap cephfs:5/5 3 up:standby-replay 1 up:resolve 4 up:active 2024-03-28T03:46:04.948 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:46:04 smithi132 ceph-mon[35941]: pgmap v6237: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-03-28T03:46:05.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:46:04 smithi165 ceph-mon[36302]: pgmap v6237: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-03-28T03:46:05.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:46:04 smithi119 ceph-mon[31518]: pgmap v6237: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-03-28T03:46:06.948 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:46:06 smithi132 ceph-mon[35941]: 42 slow requests, 0 included below; oldest blocked for > 11425.822882 secs 2024-03-28T03:46:06.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:46:06 smithi132 ceph-mon[35941]: pgmap v6238: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 8 op/s 2024-03-28T03:46:07.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:46:06 smithi165 ceph-mon[36302]: 42 slow requests, 0 included below; oldest blocked for > 11425.822882 secs 2024-03-28T03:46:07.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:46:06 smithi165 ceph-mon[36302]: pgmap v6238: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 8 op/s 2024-03-28T03:46:07.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:46:06 smithi119 ceph-mon[31518]: 42 slow requests, 0 included below; oldest blocked for > 11425.822882 secs 2024-03-28T03:46:07.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:46:06 smithi119 ceph-mon[31518]: pgmap v6238: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 8 op/s 2024-03-28T03:46:08.948 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:46:08 smithi132 ceph-mon[35941]: pgmap v6239: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-03-28T03:46:09.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:46:08 smithi165 ceph-mon[36302]: pgmap v6239: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-03-28T03:46:09.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:46:08 smithi119 ceph-mon[31518]: pgmap v6239: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-03-28T03:46:10.948 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:46:10 smithi132 ceph-mon[35941]: pgmap v6240: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 8 op/s 2024-03-28T03:46:11.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:46:10 smithi165 ceph-mon[36302]: pgmap v6240: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 8 op/s 2024-03-28T03:46:11.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:46:10 smithi119 ceph-mon[31518]: pgmap v6240: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 8 op/s 2024-03-28T03:46:11.948 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:46:11 smithi132 ceph-mon[35941]: 42 slow requests, 0 included below; oldest blocked for > 11430.822944 secs 2024-03-28T03:46:12.053 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:46:11 smithi119 ceph-mon[31518]: 42 slow requests, 0 included below; oldest blocked for > 11430.822944 secs 2024-03-28T03:46:12.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:46:11 smithi165 ceph-mon[36302]: 42 slow requests, 0 included below; oldest blocked for > 11430.822944 secs 2024-03-28T03:46:12.948 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:46:12 smithi132 ceph-mon[35941]: pgmap v6241: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:46:13.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:46:12 smithi165 ceph-mon[36302]: pgmap v6241: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:46:13.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:46:12 smithi119 ceph-mon[31518]: pgmap v6241: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:46:14.948 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:46:14 smithi132 ceph-mon[35941]: pgmap v6242: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:46:14.949 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:46:14 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T03:46:15.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:46:14 smithi165 ceph-mon[36302]: pgmap v6242: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:46:15.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:46:14 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T03:46:15.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:46:14 smithi119 ceph-mon[31518]: pgmap v6242: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:46:15.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:46:14 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T03:46:16.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:46:15 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T03:46:16.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:46:15 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T03:46:16.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:46:15 smithi165 ceph-mon[36302]: pgmap v6243: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T03:46:16.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:46:15 smithi165 ceph-mon[36302]: pgmap v6244: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:46:16.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:46:15 smithi165 ceph-mon[36302]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:46:16.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:46:15 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T03:46:16.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:46:15 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T03:46:16.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:46:15 smithi119 ceph-mon[31518]: pgmap v6243: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T03:46:16.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:46:15 smithi119 ceph-mon[31518]: pgmap v6244: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:46:16.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:46:15 smithi119 ceph-mon[31518]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:46:16.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:46:15 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T03:46:16.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:46:15 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T03:46:16.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:46:15 smithi132 ceph-mon[35941]: pgmap v6243: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T03:46:16.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:46:15 smithi132 ceph-mon[35941]: pgmap v6244: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:46:16.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:46:15 smithi132 ceph-mon[35941]: from='mgr.25277 ' entity='mgr.x' 2024-03-28T03:46:17.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:46:16 smithi165 ceph-mon[36302]: 42 slow requests, 0 included below; oldest blocked for > 11435.823005 secs 2024-03-28T03:46:17.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:46:16 smithi119 ceph-mon[31518]: 42 slow requests, 0 included below; oldest blocked for > 11435.823005 secs 2024-03-28T03:46:17.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:46:16 smithi132 ceph-mon[35941]: 42 slow requests, 0 included below; oldest blocked for > 11435.823005 secs 2024-03-28T03:46:18.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:46:17 smithi165 ceph-mon[36302]: pgmap v6245: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:46:18.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:46:17 smithi119 ceph-mon[31518]: pgmap v6245: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:46:18.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:46:17 smithi132 ceph-mon[35941]: pgmap v6245: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:46:20.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:46:20 smithi165 ceph-mon[36302]: pgmap v6246: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 4 op/s 2024-03-28T03:46:20.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:46:20 smithi119 ceph-mon[31518]: pgmap v6246: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 4 op/s 2024-03-28T03:46:20.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:46:20 smithi132 ceph-mon[35941]: pgmap v6246: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 4 op/s 2024-03-28T03:46:21.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:46:21 smithi165 ceph-mon[36302]: 42 slow requests, 0 included below; oldest blocked for > 11440.823067 secs 2024-03-28T03:46:21.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:46:21 smithi119 ceph-mon[31518]: 42 slow requests, 0 included below; oldest blocked for > 11440.823067 secs 2024-03-28T03:46:21.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:46:21 smithi132 ceph-mon[35941]: 42 slow requests, 0 included below; oldest blocked for > 11440.823067 secs 2024-03-28T03:46:22.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:46:22 smithi165 ceph-mon[36302]: pgmap v6247: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:46:22.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:46:22 smithi119 ceph-mon[31518]: pgmap v6247: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:46:22.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:46:22 smithi132 ceph-mon[35941]: pgmap v6247: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:46:23.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:46:23 smithi119 ceph-mon[31518]: pgmap v6248: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:46:23.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:46:23 smithi132 ceph-mon[35941]: pgmap v6248: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:46:23.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:46:23 smithi165 ceph-mon[36302]: pgmap v6248: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T03:46:26.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:46:26 smithi165 ceph-mon[36302]: pgmap v6249: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T03:46:26.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:46:26 smithi165 ceph-mon[36302]: 42 slow requests, 0 included below; oldest blocked for > 11445.823152 secs 2024-03-28T03:46:26.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:46:26 smithi119 ceph-mon[31518]: pgmap v6249: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T03:46:26.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:46:26 smithi119 ceph-mon[31518]: 42 slow requests, 0 included below; oldest blocked for > 11445.823152 secs 2024-03-28T03:46:26.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:46:26 smithi132 ceph-mon[35941]: pgmap v6249: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T03:46:26.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:46:26 smithi132 ceph-mon[35941]: 42 slow requests, 0 included below; oldest blocked for > 11445.823152 secs 2024-03-28T03:46:27.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:46:27 smithi119 ceph-mon[31518]: pgmap v6250: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:46:27.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:46:27 smithi132 ceph-mon[35941]: pgmap v6250: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:46:27.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:46:27 smithi165 ceph-mon[36302]: pgmap v6250: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:46:30.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:46:30 smithi165 ceph-mon[36302]: pgmap v6251: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T03:46:30.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:46:30 smithi119 ceph-mon[31518]: pgmap v6251: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T03:46:30.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:46:30 smithi132 ceph-mon[35941]: pgmap v6251: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-28T03:46:31.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:46:31 smithi165 ceph-mon[36302]: 42 slow requests, 0 included below; oldest blocked for > 11450.823242 secs 2024-03-28T03:46:31.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:46:31 smithi165 ceph-mon[36302]: pgmap v6252: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-28T03:46:31.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:46:31 smithi119 ceph-mon[31518]: 42 slow requests, 0 included below; oldest blocked for > 11450.823242 secs 2024-03-28T03:46:31.402 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:46:31 smithi119 ceph-mon[31518]: pgmap v6252: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-28T03:46:31.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:46:31 smithi132 ceph-mon[35941]: 42 slow requests, 0 included below; oldest blocked for > 11450.823242 secs 2024-03-28T03:46:31.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:46:31 smithi132 ceph-mon[35941]: pgmap v6252: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-28T03:46:33.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:46:33 smithi132 ceph-mon[35941]: pgmap v6253: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:46:33.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:46:33 smithi165 ceph-mon[36302]: pgmap v6253: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:46:33.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:46:33 smithi119 ceph-mon[31518]: pgmap v6253: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:46:35.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:46:35 smithi132 ceph-mon[35941]: pgmap v6254: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:46:35.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:46:35 smithi165 ceph-mon[36302]: pgmap v6254: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:46:35.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:46:35 smithi119 ceph-mon[31518]: pgmap v6254: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:46:36.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:46:36 smithi132 ceph-mon[35941]: 42 slow requests, 0 included below; oldest blocked for > 11455.823304 secs 2024-03-28T03:46:36.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:46:36 smithi165 ceph-mon[36302]: 42 slow requests, 0 included below; oldest blocked for > 11455.823304 secs 2024-03-28T03:46:36.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:46:36 smithi119 ceph-mon[31518]: 42 slow requests, 0 included below; oldest blocked for > 11455.823304 secs 2024-03-28T03:46:37.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:46:37 smithi132 ceph-mon[35941]: pgmap v6255: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:46:37.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:46:37 smithi165 ceph-mon[36302]: pgmap v6255: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:46:37.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:46:37 smithi119 ceph-mon[31518]: pgmap v6255: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:46:39.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:46:39 smithi132 ceph-mon[35941]: pgmap v6256: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-28T03:46:39.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:46:39 smithi165 ceph-mon[36302]: pgmap v6256: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-28T03:46:39.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:46:39 smithi119 ceph-mon[31518]: pgmap v6256: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-28T03:46:41.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:46:41 smithi132 ceph-mon[35941]: 42 slow requests, 0 included below; oldest blocked for > 11460.823362 secs 2024-03-28T03:46:41.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:46:41 smithi132 ceph-mon[35941]: pgmap v6257: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-28T03:46:41.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:46:41 smithi165 ceph-mon[36302]: 42 slow requests, 0 included below; oldest blocked for > 11460.823362 secs 2024-03-28T03:46:41.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:46:41 smithi165 ceph-mon[36302]: pgmap v6257: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-28T03:46:41.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:46:41 smithi119 ceph-mon[31518]: 42 slow requests, 0 included below; oldest blocked for > 11460.823362 secs 2024-03-28T03:46:41.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:46:41 smithi119 ceph-mon[31518]: pgmap v6257: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-28T03:46:43.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:46:43 smithi132 ceph-mon[35941]: pgmap v6258: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:46:43.454 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:46:43 smithi119 ceph-mon[31518]: pgmap v6258: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:46:43.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:46:43 smithi165 ceph-mon[36302]: pgmap v6258: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:46:45.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:46:45 smithi132 ceph-mon[35941]: pgmap v6259: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:46:45.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:46:45 smithi165 ceph-mon[36302]: pgmap v6259: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:46:45.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:46:45 smithi119 ceph-mon[31518]: pgmap v6259: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:46:46.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:46:46 smithi132 ceph-mon[35941]: 42 slow requests, 0 included below; oldest blocked for > 11465.823429 secs 2024-03-28T03:46:46.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:46:46 smithi165 ceph-mon[36302]: 42 slow requests, 0 included below; oldest blocked for > 11465.823429 secs 2024-03-28T03:46:46.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:46:46 smithi119 ceph-mon[31518]: 42 slow requests, 0 included below; oldest blocked for > 11465.823429 secs 2024-03-28T03:46:47.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:46:47 smithi119 ceph-mon[31518]: pgmap v6260: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:46:47.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:46:47 smithi132 ceph-mon[35941]: pgmap v6260: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:46:47.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:46:47 smithi165 ceph-mon[36302]: pgmap v6260: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:46:49.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:46:49 smithi132 ceph-mon[35941]: pgmap v6261: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-28T03:46:49.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:46:49 smithi165 ceph-mon[36302]: pgmap v6261: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-28T03:46:49.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:46:49 smithi119 ceph-mon[31518]: pgmap v6261: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-28T03:46:51.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:46:51 smithi132 ceph-mon[35941]: 42 slow requests, 0 included below; oldest blocked for > 11470.823515 secs 2024-03-28T03:46:51.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:46:51 smithi132 ceph-mon[35941]: pgmap v6262: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T03:46:51.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:46:51 smithi165 ceph-mon[36302]: 42 slow requests, 0 included below; oldest blocked for > 11470.823515 secs 2024-03-28T03:46:51.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:46:51 smithi165 ceph-mon[36302]: pgmap v6262: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T03:46:51.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:46:51 smithi119 ceph-mon[31518]: 42 slow requests, 0 included below; oldest blocked for > 11470.823515 secs 2024-03-28T03:46:51.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:46:51 smithi119 ceph-mon[31518]: pgmap v6262: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T03:46:53.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:46:53 smithi132 ceph-mon[35941]: pgmap v6263: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:46:53.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:46:53 smithi165 ceph-mon[36302]: pgmap v6263: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:46:53.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:46:53 smithi119 ceph-mon[31518]: pgmap v6263: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:46:55.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:46:55 smithi132 ceph-mon[35941]: pgmap v6264: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:46:55.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:46:55 smithi165 ceph-mon[36302]: pgmap v6264: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:46:55.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:46:55 smithi119 ceph-mon[31518]: pgmap v6264: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T03:46:56.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:46:56 smithi132 ceph-mon[35941]: 42 slow requests, 0 included below; oldest blocked for > 11475.823602 secs 2024-03-28T03:46:56.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:46:56 smithi165 ceph-mon[36302]: 42 slow requests, 0 included below; oldest blocked for > 11475.823602 secs 2024-03-28T03:46:56.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:46:56 smithi119 ceph-mon[31518]: 42 slow requests, 0 included below; oldest blocked for > 11475.823602 secs 2024-03-28T03:46:57.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:46:57 smithi132 ceph-mon[35941]: pgmap v6265: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:46:57.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:46:57 smithi165 ceph-mon[36302]: pgmap v6265: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:46:57.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:46:57 smithi119 ceph-mon[31518]: pgmap v6265: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-28T03:46:58.718 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi119.front.sepia.ceph.com 2024-03-28T03:46:58.719 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T03:46:58.719 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-28T03:46:58.884 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T03:46:58.884 DEBUG:teuthology.orchestra.run.smithi119:> ip netns list 2024-03-28T03:46:58.941 INFO:teuthology.orchestra.run.smithi119.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-28T03:46:58.942 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T03:46:58.942 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-28T03:46:59.039 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T03:46:59.039 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip link delete ceph-brx 2024-03-28T03:46:59.174 INFO:tasks.kclient:Unmounting kernel clients... 2024-03-28T03:46:59.174 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-03-28T03:46:59.174 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-28T03:46:59.219 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053418,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498348k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249172k,nr_inodes=812293,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/sda1 /var/lib/containers/storage/overlay ext4 rw,seclabel,relatime 0 0 overlay /var/lib/containers/storage/overlay/c59e097f2be57a86a669ed8c7e59c902fcd82e3a5d08852b2ee50a24b5d15ad7/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/IR5BIBMRL2GOVZPB7FJBHWLM4L:/var/lib/containers/storage/overlay/l/6HHTCAHEQVUOENA6A2MPSEFYS2,upperdir=/var/lib/containers/storage/overlay/c59e097f2be57a86a669ed8c7e59c902fcd82e3a5d08852b2ee50a24b5d15ad7/diff,workdir=/var/lib/containers/storage/overlay/c59e097f2be57a86a669ed8c7e59c902fcd82e3a5d08852b2ee50a24b5d15ad7/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/fe67626ff93cc6ef38ecc2b3c4a096bb3aa159961ea3e28acc8bf26e8cc60425/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/IR5BIBMRL2GOVZPB7FJBHWLM4L:/var/lib/containers/storage/overlay/l/6HHTCAHEQVUOENA6A2MPSEFYS2,upperdir=/var/lib/containers/storage/overlay/fe67626ff93cc6ef38ecc2b3c4a096bb3aa159961ea3e28acc8bf26e8cc60425/diff,workdir=/var/lib/containers/storage/overlay/fe67626ff93cc6ef38ecc2b3c4a096bb3aa159961ea3e28acc8bf26e8cc60425/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 tmpfs /run/user/0 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249172k,nr_inodes=812293,mode=700 0 0 binfmt_misc /proc/sys/fs/binfmt_misc binfmt_misc rw,nosuid,nodev,noexec,relatime 0 0 overlay /var/lib/containers/storage/overlay/fd0106dd5304ab7b07da9afe8f69ac5da80e2fb9e9d221638306937fcb24b42c/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/IR5BIBMRL2GOVZPB7FJBHWLM4L:/var/lib/containers/storage/overlay/l/6HHTCAHEQVUOENA6A2MPSEFYS2,upperdir=/var/lib/containers/storage/overlay/fd0106dd5304ab7b07da9afe8f69ac5da80e2fb9e9d221638306937fcb24b42c/diff,workdir=/var/lib/containers/storage/overlay/fd0106dd5304ab7b07da9afe8f69ac5da80e2fb9e9d221638306937fcb24b42c/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/4c3b8b4ca9ef357bfb102b19df961bb354fefbe8e93cee7e52a3be8a563b7253/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/IR5BIBMRL2GOVZPB7FJBHWLM4L:/var/lib/containers/storage/overlay/l/6HHTCAHEQVUOENA6A2MPSEFYS2,upperdir=/var/lib/containers/storage/overlay/4c3b8b4ca9ef357bfb102b19df961bb354fefbe8e93cee7e52a3be8a563b7253/diff,workdir=/var/lib/containers/storage/overlay/4c3b8b4ca9ef357bfb102b19df961bb354fefbe8e93cee7e52a3be8a563b7253/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/04c8598c15ee7faa71ce21e9e37fec769eb28caa2b50199eafec242aa28bba23/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/IR5BIBMRL2GOVZPB7FJBHWLM4L:/var/lib/containers/storage/overlay/l/6HHTCAHEQVUOENA6A2MPSEFYS2,upperdir=/var/lib/containers/storage/overlay/04c8598c15ee7faa71ce21e9e37fec769eb28caa2b50199eafec242aa28bba23/diff,workdir=/var/lib/containers/storage/overlay/04c8598c15ee7faa71ce21e9e37fec769eb28caa2b50199eafec242aa28bba23/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/56c6c876c0af0b0b8faece83c9a7320498a2160c123077484a3962e3b330e2e5/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/IR5BIBMRL2GOVZPB7FJBHWLM4L:/var/lib/containers/storage/overlay/l/6HHTCAHEQVUOENA6A2MPSEFYS2,upperdir=/var/lib/containers/storage/overlay/56c6c876c0af0b0b8faece83c9a7320498a2160c123077484a3962e3b330e2e5/diff,workdir=/var/lib/containers/storage/overlay/56c6c876c0af0b0b8faece83c9a7320498a2160c123077484a3962e3b330e2e5/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/2b96efa1a73cfba48819c2724d8e667ed910bdcbbe4aae3b7228d3103b17aae8/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/IR5BIBMRL2GOVZPB7FJBHWLM4L:/var/lib/containers/storage/overlay/l/6HHTCAHEQVUOENA6A2MPSEFYS2,upperdir=/var/lib/containers/storage/overlay/2b96efa1a73cfba48819c2724d8e667ed910bdcbbe4aae3b7228d3103b17aae8/diff,workdir=/var/lib/containers/storage/overlay/2b96efa1a73cfba48819c2724d8e667ed910bdcbbe4aae3b7228d3103b17aae8/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498348k,nr_inodes=819200,mode=755 0 0 overlay /var/lib/containers/storage/overlay/59620d0762bbbcb35477d80ef9f956507bff5ea80011279d7154852f9c55c6a5/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/IR5BIBMRL2GOVZPB7FJBHWLM4L:/var/lib/containers/storage/overlay/l/6HHTCAHEQVUOENA6A2MPSEFYS2,upperdir=/var/lib/containers/storage/overlay/59620d0762bbbcb35477d80ef9f956507bff5ea80011279d7154852f9c55c6a5/diff,workdir=/var/lib/containers/storage/overlay/59620d0762bbbcb35477d80ef9f956507bff5ea80011279d7154852f9c55c6a5/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 2024-03-28T03:46:59.219 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T03:46:59.220 DEBUG:teuthology.orchestra.run.smithi119:> ip netns list 2024-03-28T03:46:59.309 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:46:59 smithi119 ceph-mon[31518]: pgmap v6266: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-28T03:46:59.312 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T03:46:59.312 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip link delete ceph-brx 2024-03-28T03:46:59.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:46:59 smithi132 ceph-mon[35941]: pgmap v6266: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-28T03:46:59.486 INFO:teuthology.orchestra.run.smithi119.stderr:Cannot find device "ceph-brx" 2024-03-28T03:46:59.491 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-28T03:46:59.492 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-03-28T03:46:59.503 INFO:tasks.cephadm:Teardown begin 2024-03-28T03:46:59.503 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-28T03:46:59.503 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T03:46:59.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:46:59 smithi165 ceph-mon[36302]: pgmap v6266: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-28T03:46:59.724 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:46:59.722+0000 7f6ff0800640 1 -- 172.21.15.119:0/2739100835 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6fe8103850 msgr2=0x7f6fe8103cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T03:46:59.725 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:46:59.722+0000 7f6ff0800640 1 --2- 172.21.15.119:0/2739100835 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6fe8103850 0x7f6fe8103cd0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f6fd8009930 tx=0x7f6fd802bc20 comp rx=0 tx=0).stop 2024-03-28T03:46:59.725 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:46:59.722+0000 7f6ff0800640 1 -- 172.21.15.119:0/2739100835 shutdown_connections 2024-03-28T03:46:59.725 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:46:59.722+0000 7f6ff0800640 1 --2- 172.21.15.119:0/2739100835 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6fe8104210 0x7f6fe8110720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:46:59.725 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:46:59.722+0000 7f6ff0800640 1 --2- 172.21.15.119:0/2739100835 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6fe8103850 0x7f6fe8103cd0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:46:59.725 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:46:59.722+0000 7f6ff0800640 1 --2- 172.21.15.119:0/2739100835 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6fe8109860 0x7f6fe8109c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:46:59.725 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:46:59.722+0000 7f6ff0800640 1 -- 172.21.15.119:0/2739100835 >> 172.21.15.119:0/2739100835 conn(0x7f6fe80ff2e0 msgr2=0x7f6fe8101700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T03:46:59.725 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:46:59.722+0000 7f6ff0800640 1 -- 172.21.15.119:0/2739100835 shutdown_connections 2024-03-28T03:46:59.726 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:46:59.722+0000 7f6ff0800640 1 -- 172.21.15.119:0/2739100835 wait complete. 2024-03-28T03:46:59.728 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:46:59.723+0000 7f6ff0800640 1 Processor -- start 2024-03-28T03:46:59.728 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:46:59.724+0000 7f6ff0800640 1 -- start start 2024-03-28T03:46:59.728 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:46:59.724+0000 7f6ff0800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6fe8103850 0x7f6fe819a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T03:46:59.728 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:46:59.724+0000 7f6ff0800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6fe8104210 0x7f6fe819a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T03:46:59.728 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:46:59.724+0000 7f6ff0800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6fe8109860 0x7f6fe819ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T03:46:59.729 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:46:59.724+0000 7f6ff0800640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fe8113cc0 con 0x7f6fe8109860 2024-03-28T03:46:59.729 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:46:59.724+0000 7f6ff0800640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fe8113b40 con 0x7f6fe8104210 2024-03-28T03:46:59.729 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:46:59.724+0000 7f6ff0800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fe8113e40 con 0x7f6fe8103850 2024-03-28T03:46:59.729 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:46:59.724+0000 7f6fede00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6fe8103850 0x7f6fe819a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T03:46:59.729 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:46:59.724+0000 7f6fed400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6fe8104210 0x7f6fe819a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T03:46:59.729 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:46:59.724+0000 7f6fede00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6fe8103850 0x7f6fe819a220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.119:55744/0 (socket says 172.21.15.119:55744) 2024-03-28T03:46:59.729 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:46:59.724+0000 7f6fede00640 1 -- 172.21.15.119:0/3576896638 learned_addr learned my addr 172.21.15.119:0/3576896638 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T03:46:59.729 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:46:59.724+0000 7f6fed400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6fe8104210 0x7f6fe819a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.119:53066/0 (socket says 172.21.15.119:53066) 2024-03-28T03:46:59.729 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:46:59.725+0000 7f6fee800640 1 --2- 172.21.15.119:0/3576896638 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6fe8109860 0x7f6fe819ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T03:46:59.730 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:46:59.725+0000 7f6fed400640 1 -- 172.21.15.119:0/3576896638 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6fe8103850 msgr2=0x7f6fe819a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T03:46:59.730 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:46:59.725+0000 7f6fed400640 1 --2- 172.21.15.119:0/3576896638 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6fe8103850 0x7f6fe819a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:46:59.730 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:46:59.725+0000 7f6fed400640 1 -- 172.21.15.119:0/3576896638 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6fe8109860 msgr2=0x7f6fe819ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T03:46:59.730 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:46:59.725+0000 7f6fed400640 1 --2- 172.21.15.119:0/3576896638 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6fe8109860 0x7f6fe819ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:46:59.730 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:46:59.725+0000 7f6fed400640 1 -- 172.21.15.119:0/3576896638 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fe819f1c0 con 0x7f6fe8104210 2024-03-28T03:46:59.730 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:46:59.725+0000 7f6fede00640 1 --2- 172.21.15.119:0/3576896638 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6fe8103850 0x7f6fe819a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T03:46:59.730 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:46:59.725+0000 7f6fed400640 1 --2- 172.21.15.119:0/3576896638 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6fe8104210 0x7f6fe819a760 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f6fd8036040 tx=0x7f6fd8004340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T03:46:59.730 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:46:59.725+0000 7f6fe2a00640 1 -- 172.21.15.119:0/3576896638 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fd8043020 con 0x7f6fe8104210 2024-03-28T03:46:59.730 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:46:59.725+0000 7f6fe2a00640 1 -- 172.21.15.119:0/3576896638 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6fd80054e0 con 0x7f6fe8104210 2024-03-28T03:46:59.731 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:46:59.726+0000 7f6ff0800640 1 -- 172.21.15.119:0/3576896638 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fe819f450 con 0x7f6fe8104210 2024-03-28T03:46:59.731 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:46:59.726+0000 7f6fe2a00640 1 -- 172.21.15.119:0/3576896638 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fd8003b30 con 0x7f6fe8104210 2024-03-28T03:46:59.731 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:46:59.726+0000 7f6ff0800640 1 -- 172.21.15.119:0/3576896638 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fe81a6d40 con 0x7f6fe8104210 2024-03-28T03:46:59.732 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:46:59.727+0000 7f6ff0800640 1 -- 172.21.15.119:0/3576896638 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fe8068490 con 0x7f6fe8104210 2024-03-28T03:46:59.732 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:46:59.729+0000 7f6fe2a00640 1 -- 172.21.15.119:0/3576896638 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 27) v1 ==== 137422+0+0 (secure 0 0 0) 0x7f6fd8046020 con 0x7f6fe8104210 2024-03-28T03:46:59.732 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:46:59.730+0000 7f6fe2a00640 1 --2- 172.21.15.119:0/3576896638 >> [v2:172.21.15.119:6800/4273482239,v1:172.21.15.119:6801/4273482239] conn(0x7f6fd00a29f0 0x7f6fd00a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T03:46:59.732 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:46:59.730+0000 7f6fe2a00640 1 -- 172.21.15.119:0/3576896638 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 11163+0+0 (secure 0 0 0) 0x7f6fd80f1b70 con 0x7f6fe8104210 2024-03-28T03:46:59.732 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:46:59.730+0000 7f6fede00640 1 --2- 172.21.15.119:0/3576896638 >> [v2:172.21.15.119:6800/4273482239,v1:172.21.15.119:6801/4273482239] conn(0x7f6fd00a29f0 0x7f6fd00a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T03:46:59.733 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:46:59.731+0000 7f6fede00640 1 --2- 172.21.15.119:0/3576896638 >> [v2:172.21.15.119:6800/4273482239,v1:172.21.15.119:6801/4273482239] conn(0x7f6fd00a29f0 0x7f6fd00a4eb0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f6fdc011440 tx=0x7f6fdc005f90 comp rx=0 tx=0).ready entity=mgr.25277 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T03:46:59.734 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:46:59.732+0000 7f6fe2a00640 1 -- 172.21.15.119:0/3576896638 <== mon.2 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f6fd804d020 con 0x7f6fe8104210 2024-03-28T03:46:59.974 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:46:59.972+0000 7f6ff0800640 1 -- 172.21.15.119:0/3576896638 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6fe810bdf0 con 0x7f6fe8104210 2024-03-28T03:46:59.975 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:46:59.973+0000 7f6fe2a00640 1 -- 172.21.15.119:0/3576896638 <== mon.2 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 178 v178) v1 ==== 95+0+8339 (secure 0 0 0) 0x7f6fd80bdf30 con 0x7f6fe8104210 2024-03-28T03:46:59.975 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-28T03:46:59.976 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":178,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":178,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-28T00:23:56.215355+0000","modified":"2024-03-28T03:46:01.321729+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":125,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24581,"mds_1":25558,"mds_2":24530,"mds_3":25546,"mds_4":24511},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14466":{"gid":14466,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6841/1678451188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6840","nonce":1678451188},{"type":"v1","addr":"172.21.15.119:6841","nonce":1678451188}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24455":{"gid":24455,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.132:6833/1179016670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":1179016670},{"type":"v1","addr":"172.21.15.132:6833","nonce":1179016670}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24500":{"gid":24500,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.132:6835/356981946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":356981946},{"type":"v1","addr":"172.21.15.132:6835","nonce":356981946}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24511":{"gid":24511,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.165:6835/1947980102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":1947980102},{"type":"v1","addr":"172.21.15.165:6835","nonce":1947980102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24530":{"gid":24530,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.132:6837/3383082967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6836","nonce":3383082967},{"type":"v1","addr":"172.21.15.132:6837","nonce":3383082967}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3034,"addr":"172.21.15.132:6839/3305822525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6838","nonce":3305822525},{"type":"v1","addr":"172.21.15.132:6839","nonce":3305822525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_25546":{"gid":25546,"name":"g","rank":3,"incarnation":165,"state":"up:active","state_seq":5,"addr":"172.21.15.119:6835/1945411848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":1945411848},{"type":"v1","addr":"172.21.15.119:6835","nonce":1945411848}]},"join_fscid":-1,"laggy_since":"2024-03-28T00:35:53.436699+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_25558":{"gid":25558,"name":"d","rank":1,"incarnation":173,"state":"up:resolve","state_seq":2,"addr":"172.21.15.119:6835/1473458343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":1473458343},{"type":"v1","addr":"172.21.15.119:6835","nonce":1473458343}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[25546,24530,24581,24511]},"id":1}]} 2024-03-28T03:46:59.976 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 178 2024-03-28T03:46:59.978 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:46:59.976+0000 7f6ff0800640 1 -- 172.21.15.119:0/3576896638 >> [v2:172.21.15.119:6800/4273482239,v1:172.21.15.119:6801/4273482239] conn(0x7f6fd00a29f0 msgr2=0x7f6fd00a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T03:46:59.978 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:46:59.976+0000 7f6ff0800640 1 --2- 172.21.15.119:0/3576896638 >> [v2:172.21.15.119:6800/4273482239,v1:172.21.15.119:6801/4273482239] conn(0x7f6fd00a29f0 0x7f6fd00a4eb0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f6fdc011440 tx=0x7f6fdc005f90 comp rx=0 tx=0).stop 2024-03-28T03:46:59.979 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:46:59.976+0000 7f6ff0800640 1 -- 172.21.15.119:0/3576896638 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6fe8104210 msgr2=0x7f6fe819a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T03:46:59.979 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:46:59.976+0000 7f6ff0800640 1 --2- 172.21.15.119:0/3576896638 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6fe8104210 0x7f6fe819a760 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f6fd8036040 tx=0x7f6fd8004340 comp rx=0 tx=0).stop 2024-03-28T03:46:59.979 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:46:59.976+0000 7f6ff0800640 1 -- 172.21.15.119:0/3576896638 shutdown_connections 2024-03-28T03:46:59.979 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:46:59.976+0000 7f6ff0800640 1 --2- 172.21.15.119:0/3576896638 >> [v2:172.21.15.119:6800/4273482239,v1:172.21.15.119:6801/4273482239] conn(0x7f6fd00a29f0 0x7f6fd00a4eb0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:46:59.979 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:46:59.976+0000 7f6ff0800640 1 --2- 172.21.15.119:0/3576896638 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6fe8109860 0x7f6fe819ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:46:59.979 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:46:59.976+0000 7f6ff0800640 1 --2- 172.21.15.119:0/3576896638 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6fe8104210 0x7f6fe819a760 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:46:59.979 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:46:59.976+0000 7f6ff0800640 1 --2- 172.21.15.119:0/3576896638 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6fe8103850 0x7f6fe819a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:46:59.979 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:46:59.976+0000 7f6ff0800640 1 -- 172.21.15.119:0/3576896638 >> 172.21.15.119:0/3576896638 conn(0x7f6fe80ff2e0 msgr2=0x7f6fe80ffae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T03:46:59.980 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:46:59.976+0000 7f6ff0800640 1 -- 172.21.15.119:0/3576896638 shutdown_connections 2024-03-28T03:46:59.980 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:46:59.976+0000 7f6ff0800640 1 -- 172.21.15.119:0/3576896638 wait complete. 2024-03-28T03:46:59.998 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T03:47:00.203 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:00.201+0000 7f3cfa200640 1 -- 172.21.15.119:0/4187204367 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3cf4109850 msgr2=0x7f3cf4109c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T03:47:00.203 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:00.201+0000 7f3cfa200640 1 --2- 172.21.15.119:0/4187204367 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3cf4109850 0x7f3cf4109c30 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f3ce4009930 tx=0x7f3ce402bc30 comp rx=0 tx=0).stop 2024-03-28T03:47:00.203 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:00.201+0000 7f3cfa200640 1 -- 172.21.15.119:0/4187204367 shutdown_connections 2024-03-28T03:47:00.204 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:00.201+0000 7f3cfa200640 1 --2- 172.21.15.119:0/4187204367 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3cf4104200 0x7f3cf4110710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:47:00.204 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:00.201+0000 7f3cfa200640 1 --2- 172.21.15.119:0/4187204367 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3cf4103840 0x7f3cf4103cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:47:00.204 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:00.201+0000 7f3cfa200640 1 --2- 172.21.15.119:0/4187204367 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3cf4109850 0x7f3cf4109c30 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:47:00.204 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:00.201+0000 7f3cfa200640 1 -- 172.21.15.119:0/4187204367 >> 172.21.15.119:0/4187204367 conn(0x7f3cf40ff310 msgr2=0x7f3cf4101730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T03:47:00.204 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:00.201+0000 7f3cfa200640 1 -- 172.21.15.119:0/4187204367 shutdown_connections 2024-03-28T03:47:00.204 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:00.201+0000 7f3cfa200640 1 -- 172.21.15.119:0/4187204367 wait complete. 2024-03-28T03:47:00.206 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:00.202+0000 7f3cfa200640 1 Processor -- start 2024-03-28T03:47:00.206 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:00.202+0000 7f3cfa200640 1 -- start start 2024-03-28T03:47:00.207 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:00.203+0000 7f3cfa200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3cf4103840 0x7f3cf419a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T03:47:00.207 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:00.203+0000 7f3cfa200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3cf4104200 0x7f3cf419a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T03:47:00.207 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:00.203+0000 7f3cfa200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3cf4109850 0x7f3cf419ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T03:47:00.207 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:00.203+0000 7f3cfa200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cf4113cb0 con 0x7f3cf4104200 2024-03-28T03:47:00.207 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:00.203+0000 7f3cfa200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cf4113b30 con 0x7f3cf4103840 2024-03-28T03:47:00.207 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:00.203+0000 7f3cfa200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cf4113e30 con 0x7f3cf4109850 2024-03-28T03:47:00.207 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:00.203+0000 7f3cf3400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3cf4103840 0x7f3cf419a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T03:47:00.207 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:00.203+0000 7f3cf3400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3cf4103840 0x7f3cf419a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.119:53082/0 (socket says 172.21.15.119:53082) 2024-03-28T03:47:00.207 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:00.203+0000 7f3cf3400640 1 -- 172.21.15.119:0/3246622926 learned_addr learned my addr 172.21.15.119:0/3246622926 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T03:47:00.208 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:00.203+0000 7f3cf3e00640 1 --2- 172.21.15.119:0/3246622926 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3cf4109850 0x7f3cf419ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T03:47:00.208 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:00.203+0000 7f3cf2a00640 1 --2- 172.21.15.119:0/3246622926 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3cf4104200 0x7f3cf419a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T03:47:00.208 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:00.204+0000 7f3cf3400640 1 -- 172.21.15.119:0/3246622926 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3cf4109850 msgr2=0x7f3cf419ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T03:47:00.208 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:00.204+0000 7f3cf3400640 1 --2- 172.21.15.119:0/3246622926 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3cf4109850 0x7f3cf419ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:47:00.208 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:00.204+0000 7f3cf3400640 1 -- 172.21.15.119:0/3246622926 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3cf4104200 msgr2=0x7f3cf419a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T03:47:00.208 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:00.204+0000 7f3cf3400640 1 --2- 172.21.15.119:0/3246622926 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3cf4104200 0x7f3cf419a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:47:00.208 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:00.204+0000 7f3cf3400640 1 -- 172.21.15.119:0/3246622926 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cf419f160 con 0x7f3cf4103840 2024-03-28T03:47:00.208 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:00.204+0000 7f3cf3e00640 1 --2- 172.21.15.119:0/3246622926 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3cf4109850 0x7f3cf419ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T03:47:00.208 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:00.204+0000 7f3cf3400640 1 --2- 172.21.15.119:0/3246622926 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3cf4103840 0x7f3cf419a170 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f3ce4009900 tx=0x7f3ce4004360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T03:47:00.209 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:00.204+0000 7f3cebe00640 1 -- 172.21.15.119:0/3246622926 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ce4043020 con 0x7f3cf4103840 2024-03-28T03:47:00.209 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:00.204+0000 7f3cebe00640 1 -- 172.21.15.119:0/3246622926 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3ce40054e0 con 0x7f3cf4103840 2024-03-28T03:47:00.209 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:00.204+0000 7f3cebe00640 1 -- 172.21.15.119:0/3246622926 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ce4003bb0 con 0x7f3cf4103840 2024-03-28T03:47:00.209 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:00.204+0000 7f3cfa200640 1 -- 172.21.15.119:0/3246622926 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cf419f3f0 con 0x7f3cf4103840 2024-03-28T03:47:00.209 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:00.204+0000 7f3cfa200640 1 -- 172.21.15.119:0/3246622926 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cf41a6ce0 con 0x7f3cf4103840 2024-03-28T03:47:00.209 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:00.206+0000 7f3cfa200640 1 -- 172.21.15.119:0/3246622926 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cf4104f80 con 0x7f3cf4103840 2024-03-28T03:47:00.210 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:00.207+0000 7f3cebe00640 1 -- 172.21.15.119:0/3246622926 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 27) v1 ==== 137422+0+0 (secure 0 0 0) 0x7f3ce4046020 con 0x7f3cf4103840 2024-03-28T03:47:00.210 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:00.208+0000 7f3cebe00640 1 --2- 172.21.15.119:0/3246622926 >> [v2:172.21.15.119:6800/4273482239,v1:172.21.15.119:6801/4273482239] conn(0x7f3cd80a2970 0x7f3cd80a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T03:47:00.211 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:00.209+0000 7f3cebe00640 1 -- 172.21.15.119:0/3246622926 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 11163+0+0 (secure 0 0 0) 0x7f3ce40f1790 con 0x7f3cf4103840 2024-03-28T03:47:00.211 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:00.209+0000 7f3cf2a00640 1 --2- 172.21.15.119:0/3246622926 >> [v2:172.21.15.119:6800/4273482239,v1:172.21.15.119:6801/4273482239] conn(0x7f3cd80a2970 0x7f3cd80a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T03:47:00.211 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:00.209+0000 7f3cf2a00640 1 --2- 172.21.15.119:0/3246622926 >> [v2:172.21.15.119:6800/4273482239,v1:172.21.15.119:6801/4273482239] conn(0x7f3cd80a2970 0x7f3cd80a4e30 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f3cf4101950 tx=0x7f3ce0005f90 comp rx=0 tx=0).ready entity=mgr.25277 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T03:47:00.214 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:00.212+0000 7f3cebe00640 1 -- 172.21.15.119:0/3246622926 <== mon.2 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f3ce404d020 con 0x7f3cf4103840 2024-03-28T03:47:00.401 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:47:00 smithi119 ceph-mon[31518]: from='client.? 172.21.15.119:0/3576896638' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T03:47:00.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:47:00 smithi132 ceph-mon[35941]: from='client.? 172.21.15.119:0/3576896638' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T03:47:00.455 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:00.453+0000 7f3cfa200640 1 -- 172.21.15.119:0/3246622926 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3cf410bdb0 con 0x7f3cf4103840 2024-03-28T03:47:00.456 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:00.454+0000 7f3cebe00640 1 -- 172.21.15.119:0/3246622926 <== mon.2 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 178 v178) v1 ==== 95+0+8339 (secure 0 0 0) 0x7f3ce40bdbb0 con 0x7f3cf4103840 2024-03-28T03:47:00.457 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-28T03:47:00.457 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":178,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":178,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-28T00:23:56.215355+0000","modified":"2024-03-28T03:46:01.321729+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":125,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24581,"mds_1":25558,"mds_2":24530,"mds_3":25546,"mds_4":24511},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14466":{"gid":14466,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6841/1678451188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6840","nonce":1678451188},{"type":"v1","addr":"172.21.15.119:6841","nonce":1678451188}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24455":{"gid":24455,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.132:6833/1179016670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":1179016670},{"type":"v1","addr":"172.21.15.132:6833","nonce":1179016670}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24500":{"gid":24500,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.132:6835/356981946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":356981946},{"type":"v1","addr":"172.21.15.132:6835","nonce":356981946}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24511":{"gid":24511,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.165:6835/1947980102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":1947980102},{"type":"v1","addr":"172.21.15.165:6835","nonce":1947980102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24530":{"gid":24530,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.132:6837/3383082967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6836","nonce":3383082967},{"type":"v1","addr":"172.21.15.132:6837","nonce":3383082967}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3034,"addr":"172.21.15.132:6839/3305822525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6838","nonce":3305822525},{"type":"v1","addr":"172.21.15.132:6839","nonce":3305822525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_25546":{"gid":25546,"name":"g","rank":3,"incarnation":165,"state":"up:active","state_seq":5,"addr":"172.21.15.119:6835/1945411848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":1945411848},{"type":"v1","addr":"172.21.15.119:6835","nonce":1945411848}]},"join_fscid":-1,"laggy_since":"2024-03-28T00:35:53.436699+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_25558":{"gid":25558,"name":"d","rank":1,"incarnation":173,"state":"up:resolve","state_seq":2,"addr":"172.21.15.119:6835/1473458343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":1473458343},{"type":"v1","addr":"172.21.15.119:6835","nonce":1473458343}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[25546,24530,24581,24511]},"id":1}]} 2024-03-28T03:47:00.457 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 178 2024-03-28T03:47:00.459 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:00.456+0000 7f3cfa200640 1 -- 172.21.15.119:0/3246622926 >> [v2:172.21.15.119:6800/4273482239,v1:172.21.15.119:6801/4273482239] conn(0x7f3cd80a2970 msgr2=0x7f3cd80a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T03:47:00.460 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:00.456+0000 7f3cfa200640 1 --2- 172.21.15.119:0/3246622926 >> [v2:172.21.15.119:6800/4273482239,v1:172.21.15.119:6801/4273482239] conn(0x7f3cd80a2970 0x7f3cd80a4e30 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f3cf4101950 tx=0x7f3ce0005f90 comp rx=0 tx=0).stop 2024-03-28T03:47:00.460 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:00.456+0000 7f3cfa200640 1 -- 172.21.15.119:0/3246622926 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3cf4103840 msgr2=0x7f3cf419a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T03:47:00.460 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:00.456+0000 7f3cfa200640 1 --2- 172.21.15.119:0/3246622926 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3cf4103840 0x7f3cf419a170 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f3ce4009900 tx=0x7f3ce4004360 comp rx=0 tx=0).stop 2024-03-28T03:47:00.460 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:00.457+0000 7f3cfa200640 1 -- 172.21.15.119:0/3246622926 shutdown_connections 2024-03-28T03:47:00.461 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:00.457+0000 7f3cfa200640 1 --2- 172.21.15.119:0/3246622926 >> [v2:172.21.15.119:6800/4273482239,v1:172.21.15.119:6801/4273482239] conn(0x7f3cd80a2970 0x7f3cd80a4e30 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:47:00.461 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:00.457+0000 7f3cfa200640 1 --2- 172.21.15.119:0/3246622926 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3cf4109850 0x7f3cf419ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:47:00.461 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:00.457+0000 7f3cfa200640 1 --2- 172.21.15.119:0/3246622926 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3cf4104200 0x7f3cf419a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:47:00.461 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:00.457+0000 7f3cfa200640 1 --2- 172.21.15.119:0/3246622926 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3cf4103840 0x7f3cf419a170 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:47:00.461 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:00.457+0000 7f3cfa200640 1 -- 172.21.15.119:0/3246622926 >> 172.21.15.119:0/3246622926 conn(0x7f3cf40ff310 msgr2=0x7f3cf40ffae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T03:47:00.462 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:00.457+0000 7f3cfa200640 1 -- 172.21.15.119:0/3246622926 shutdown_connections 2024-03-28T03:47:00.462 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:00.457+0000 7f3cfa200640 1 -- 172.21.15.119:0/3246622926 wait complete. 2024-03-28T03:47:00.478 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T03:47:00.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:47:00 smithi165 ceph-mon[36302]: from='client.? 172.21.15.119:0/3576896638' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T03:47:00.689 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:00.687+0000 7f2498200640 1 -- 172.21.15.119:0/3373750164 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2490101ff0 msgr2=0x7f249010e500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T03:47:00.689 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:00.687+0000 7f2498200640 1 --2- 172.21.15.119:0/3373750164 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2490101ff0 0x7f249010e500 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f2488009cd0 tx=0x7f248802c0e0 comp rx=0 tx=0).stop 2024-03-28T03:47:00.689 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:00.687+0000 7f2498200640 1 -- 172.21.15.119:0/3373750164 shutdown_connections 2024-03-28T03:47:00.689 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:00.687+0000 7f2498200640 1 --2- 172.21.15.119:0/3373750164 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2490101ff0 0x7f249010e500 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:47:00.689 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:00.687+0000 7f2498200640 1 --2- 172.21.15.119:0/3373750164 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2490101630 0x7f2490101ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:47:00.690 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:00.687+0000 7f2498200640 1 --2- 172.21.15.119:0/3373750164 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2490107640 0x7f2490107a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:47:00.690 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:00.687+0000 7f2498200640 1 -- 172.21.15.119:0/3373750164 >> 172.21.15.119:0/3373750164 conn(0x7f2490078aa0 msgr2=0x7f2490078eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T03:47:00.690 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:00.687+0000 7f2498200640 1 -- 172.21.15.119:0/3373750164 shutdown_connections 2024-03-28T03:47:00.690 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:00.687+0000 7f2498200640 1 -- 172.21.15.119:0/3373750164 wait complete. 2024-03-28T03:47:00.692 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:00.688+0000 7f2498200640 1 Processor -- start 2024-03-28T03:47:00.692 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:00.688+0000 7f2498200640 1 -- start start 2024-03-28T03:47:00.692 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:00.689+0000 7f2498200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2490101630 0x7f24900739a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T03:47:00.692 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:00.689+0000 7f2498200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2490101ff0 0x7f2490073ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T03:47:00.693 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:00.689+0000 7f2498200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2490107640 0x7f2490074420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T03:47:00.693 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:00.689+0000 7f2498200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2490111a50 con 0x7f2490101ff0 2024-03-28T03:47:00.693 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:00.689+0000 7f2498200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24901118d0 con 0x7f2490101630 2024-03-28T03:47:00.693 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:00.689+0000 7f2498200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2490111bd0 con 0x7f2490107640 2024-03-28T03:47:00.693 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:00.689+0000 7f2495800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2490101630 0x7f24900739a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T03:47:00.693 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:00.689+0000 7f2494e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2490101ff0 0x7f2490073ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T03:47:00.693 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:00.689+0000 7f2495800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2490101630 0x7f24900739a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.119:53100/0 (socket says 172.21.15.119:53100) 2024-03-28T03:47:00.693 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:00.689+0000 7f2495800640 1 -- 172.21.15.119:0/3023550091 learned_addr learned my addr 172.21.15.119:0/3023550091 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T03:47:00.693 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:00.689+0000 7f2496200640 1 --2- 172.21.15.119:0/3023550091 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2490107640 0x7f2490074420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T03:47:00.694 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:00.689+0000 7f2495800640 1 -- 172.21.15.119:0/3023550091 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2490107640 msgr2=0x7f2490074420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T03:47:00.694 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:00.689+0000 7f2495800640 1 --2- 172.21.15.119:0/3023550091 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2490107640 0x7f2490074420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:47:00.694 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:00.690+0000 7f2495800640 1 -- 172.21.15.119:0/3023550091 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2490101ff0 msgr2=0x7f2490073ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T03:47:00.694 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:00.690+0000 7f2495800640 1 --2- 172.21.15.119:0/3023550091 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2490101ff0 0x7f2490073ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:47:00.694 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:00.690+0000 7f2495800640 1 -- 172.21.15.119:0/3023550091 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24901a63b0 con 0x7f2490101630 2024-03-28T03:47:00.694 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:00.690+0000 7f2495800640 1 --2- 172.21.15.119:0/3023550091 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2490101630 0x7f24900739a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f248400e9f0 tx=0x7f248400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T03:47:00.694 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:00.690+0000 7f248e000640 1 -- 172.21.15.119:0/3023550091 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f248400cd80 con 0x7f2490101630 2024-03-28T03:47:00.694 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:00.690+0000 7f248e000640 1 -- 172.21.15.119:0/3023550091 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2484010910 con 0x7f2490101630 2024-03-28T03:47:00.695 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:00.690+0000 7f248e000640 1 -- 172.21.15.119:0/3023550091 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2484018990 con 0x7f2490101630 2024-03-28T03:47:00.695 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:00.690+0000 7f2498200640 1 -- 172.21.15.119:0/3023550091 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24901a66a0 con 0x7f2490101630 2024-03-28T03:47:00.695 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:00.690+0000 7f2498200640 1 -- 172.21.15.119:0/3023550091 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24901a6c40 con 0x7f2490101630 2024-03-28T03:47:00.695 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:00.691+0000 7f2498200640 1 -- 172.21.15.119:0/3023550091 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24900688e0 con 0x7f2490101630 2024-03-28T03:47:00.695 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:00.693+0000 7f248e000640 1 -- 172.21.15.119:0/3023550091 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 27) v1 ==== 137422+0+0 (secure 0 0 0) 0x7f2484010ac0 con 0x7f2490101630 2024-03-28T03:47:00.696 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:00.694+0000 7f248e000640 1 --2- 172.21.15.119:0/3023550091 >> [v2:172.21.15.119:6800/4273482239,v1:172.21.15.119:6801/4273482239] conn(0x7f24780a2a20 0x7f24780a4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T03:47:00.696 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:00.695+0000 7f248e000640 1 -- 172.21.15.119:0/3023550091 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 11163+0+0 (secure 0 0 0) 0x7f24840cfaf0 con 0x7f2490101630 2024-03-28T03:47:00.696 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:00.695+0000 7f2494e00640 1 --2- 172.21.15.119:0/3023550091 >> [v2:172.21.15.119:6800/4273482239,v1:172.21.15.119:6801/4273482239] conn(0x7f24780a2a20 0x7f24780a4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T03:47:00.697 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:00.695+0000 7f2494e00640 1 --2- 172.21.15.119:0/3023550091 >> [v2:172.21.15.119:6800/4273482239,v1:172.21.15.119:6801/4273482239] conn(0x7f24780a2a20 0x7f24780a4ee0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f24900753e0 tx=0x7f247c00b390 comp rx=0 tx=0).ready entity=mgr.25277 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T03:47:00.700 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:00.698+0000 7f248e000640 1 -- 172.21.15.119:0/3023550091 <== mon.2 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f248409b750 con 0x7f2490101630 2024-03-28T03:47:00.901 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:00.899+0000 7f2498200640 1 -- 172.21.15.119:0/3023550091 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2490074e80 con 0x7f2490101630 2024-03-28T03:47:00.903 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:00.901+0000 7f248e000640 1 -- 172.21.15.119:0/3023550091 <== mon.2 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v125) v1 ==== 74+0+27258 (secure 0 0 0) 0x7f248409f7f0 con 0x7f2490101630 2024-03-28T03:47:00.904 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-28T03:47:00.904 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":125,"fsid":"10f73b92-ec98-11ee-95d0-87774f69a715","created":"2024-03-28T00:14:57.563604+0000","modified":"2024-03-28T00:35:52.413057+0000","last_up_change":"2024-03-28T00:22:58.821662+0000","last_in_change":"2024-03-28T00:22:38.467270+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T00:18:46.253414+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-28T00:23:54.261981+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-28T00:23:55.685836+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":13,"snap_epoch":121,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c0b875d-a01e-4294-9c9c-5d0b048b8912","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":580396252},{"type":"v1","addr":"172.21.15.119:6803","nonce":580396252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":580396252},{"type":"v1","addr":"172.21.15.119:6805","nonce":580396252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":580396252},{"type":"v1","addr":"172.21.15.119:6809","nonce":580396252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":580396252},{"type":"v1","addr":"172.21.15.119:6807","nonce":580396252}]},"public_addr":"172.21.15.119:6803/580396252","cluster_addr":"172.21.15.119:6805/580396252","heartbeat_back_addr":"172.21.15.119:6809/580396252","heartbeat_front_addr":"172.21.15.119:6807/580396252","state":["exists","up"]},{"osd":1,"uuid":"5567f7c2-8ea4-4789-9ea1-467793324fb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":2500318888},{"type":"v1","addr":"172.21.15.132:6801","nonce":2500318888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":2500318888},{"type":"v1","addr":"172.21.15.132:6803","nonce":2500318888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":2500318888},{"type":"v1","addr":"172.21.15.132:6807","nonce":2500318888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":2500318888},{"type":"v1","addr":"172.21.15.132:6805","nonce":2500318888}]},"public_addr":"172.21.15.132:6801/2500318888","cluster_addr":"172.21.15.132:6803/2500318888","heartbeat_back_addr":"172.21.15.132:6807/2500318888","heartbeat_front_addr":"172.21.15.132:6805/2500318888","state":["exists","up"]},{"osd":2,"uuid":"ee54dbd0-24b4-4ee6-988c-ce71491199d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":2361434832},{"type":"v1","addr":"172.21.15.165:6801","nonce":2361434832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":2361434832},{"type":"v1","addr":"172.21.15.165:6803","nonce":2361434832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":2361434832},{"type":"v1","addr":"172.21.15.165:6807","nonce":2361434832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":2361434832},{"type":"v1","addr":"172.21.15.165:6805","nonce":2361434832}]},"public_addr":"172.21.15.165:6801/2361434832","cluster_addr":"172.21.15.165:6803/2361434832","heartbeat_back_addr":"172.21.15.165:6807/2361434832","heartbeat_front_addr":"172.21.15.165:6805/2361434832","state":["exists","up"]},{"osd":3,"uuid":"0464c40b-828f-4db4-96d9-3f7a6325e6da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":3732272210},{"type":"v1","addr":"172.21.15.119:6811","nonce":3732272210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":3732272210},{"type":"v1","addr":"172.21.15.119:6813","nonce":3732272210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":3732272210},{"type":"v1","addr":"172.21.15.119:6817","nonce":3732272210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":3732272210},{"type":"v1","addr":"172.21.15.119:6815","nonce":3732272210}]},"public_addr":"172.21.15.119:6811/3732272210","cluster_addr":"172.21.15.119:6813/3732272210","heartbeat_back_addr":"172.21.15.119:6817/3732272210","heartbeat_front_addr":"172.21.15.119:6815/3732272210","state":["exists","up"]},{"osd":4,"uuid":"8fd70b8e-5128-4173-aa0e-e735dfeb995d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":2803039505},{"type":"v1","addr":"172.21.15.132:6809","nonce":2803039505}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":2803039505},{"type":"v1","addr":"172.21.15.132:6811","nonce":2803039505}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":2803039505},{"type":"v1","addr":"172.21.15.132:6815","nonce":2803039505}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":2803039505},{"type":"v1","addr":"172.21.15.132:6813","nonce":2803039505}]},"public_addr":"172.21.15.132:6809/2803039505","cluster_addr":"172.21.15.132:6811/2803039505","heartbeat_back_addr":"172.21.15.132:6815/2803039505","heartbeat_front_addr":"172.21.15.132:6813/2803039505","state":["exists","up"]},{"osd":5,"uuid":"3224b4c1-733e-4424-a6da-870bbb5f94e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":2561252206},{"type":"v1","addr":"172.21.15.165:6809","nonce":2561252206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":2561252206},{"type":"v1","addr":"172.21.15.165:6811","nonce":2561252206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":2561252206},{"type":"v1","addr":"172.21.15.165:6815","nonce":2561252206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":2561252206},{"type":"v1","addr":"172.21.15.165:6813","nonce":2561252206}]},"public_addr":"172.21.15.165:6809/2561252206","cluster_addr":"172.21.15.165:6811/2561252206","heartbeat_back_addr":"172.21.15.165:6815/2561252206","heartbeat_front_addr":"172.21.15.165:6813/2561252206","state":["exists","up"]},{"osd":6,"uuid":"e435745d-59b5-439a-9091-0946ca0e9332","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":2545410230},{"type":"v1","addr":"172.21.15.119:6819","nonce":2545410230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":2545410230},{"type":"v1","addr":"172.21.15.119:6821","nonce":2545410230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":2545410230},{"type":"v1","addr":"172.21.15.119:6825","nonce":2545410230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":2545410230},{"type":"v1","addr":"172.21.15.119:6823","nonce":2545410230}]},"public_addr":"172.21.15.119:6819/2545410230","cluster_addr":"172.21.15.119:6821/2545410230","heartbeat_back_addr":"172.21.15.119:6825/2545410230","heartbeat_front_addr":"172.21.15.119:6823/2545410230","state":["exists","up"]},{"osd":7,"uuid":"79b97a12-f6b4-48c3-b517-2dd6b6e7c85f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":2877892376},{"type":"v1","addr":"172.21.15.132:6817","nonce":2877892376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":2877892376},{"type":"v1","addr":"172.21.15.132:6819","nonce":2877892376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":2877892376},{"type":"v1","addr":"172.21.15.132:6823","nonce":2877892376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":2877892376},{"type":"v1","addr":"172.21.15.132:6821","nonce":2877892376}]},"public_addr":"172.21.15.132:6817/2877892376","cluster_addr":"172.21.15.132:6819/2877892376","heartbeat_back_addr":"172.21.15.132:6823/2877892376","heartbeat_front_addr":"172.21.15.132:6821/2877892376","state":["exists","up"]},{"osd":8,"uuid":"2970524c-b99a-412d-85c2-6c29b54035e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":4115226150},{"type":"v1","addr":"172.21.15.165:6817","nonce":4115226150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":4115226150},{"type":"v1","addr":"172.21.15.165:6819","nonce":4115226150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":4115226150},{"type":"v1","addr":"172.21.15.165:6823","nonce":4115226150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":4115226150},{"type":"v1","addr":"172.21.15.165:6821","nonce":4115226150}]},"public_addr":"172.21.15.165:6817/4115226150","cluster_addr":"172.21.15.165:6819/4115226150","heartbeat_back_addr":"172.21.15.165:6823/4115226150","heartbeat_front_addr":"172.21.15.165:6821/4115226150","state":["exists","up"]},{"osd":9,"uuid":"2b1a08ed-cde9-4def-b3e8-838fff22709c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":2532245222},{"type":"v1","addr":"172.21.15.119:6827","nonce":2532245222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":2532245222},{"type":"v1","addr":"172.21.15.119:6829","nonce":2532245222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":2532245222},{"type":"v1","addr":"172.21.15.119:6833","nonce":2532245222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":2532245222},{"type":"v1","addr":"172.21.15.119:6831","nonce":2532245222}]},"public_addr":"172.21.15.119:6827/2532245222","cluster_addr":"172.21.15.119:6829/2532245222","heartbeat_back_addr":"172.21.15.119:6833/2532245222","heartbeat_front_addr":"172.21.15.119:6831/2532245222","state":["exists","up"]},{"osd":10,"uuid":"f019acd0-915a-4094-b12c-b53672740a5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":3454298492},{"type":"v1","addr":"172.21.15.132:6825","nonce":3454298492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":3454298492},{"type":"v1","addr":"172.21.15.132:6827","nonce":3454298492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":3454298492},{"type":"v1","addr":"172.21.15.132:6831","nonce":3454298492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":3454298492},{"type":"v1","addr":"172.21.15.132:6829","nonce":3454298492}]},"public_addr":"172.21.15.132:6825/3454298492","cluster_addr":"172.21.15.132:6827/3454298492","heartbeat_back_addr":"172.21.15.132:6831/3454298492","heartbeat_front_addr":"172.21.15.132:6829/3454298492","state":["exists","up"]},{"osd":11,"uuid":"2c1e59a8-8f21-45c0-8568-4b632ce020f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":2138771601},{"type":"v1","addr":"172.21.15.165:6825","nonce":2138771601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":2138771601},{"type":"v1","addr":"172.21.15.165:6827","nonce":2138771601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":2138771601},{"type":"v1","addr":"172.21.15.165:6831","nonce":2138771601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":2138771601},{"type":"v1","addr":"172.21.15.165:6829","nonce":2138771601}]},"public_addr":"172.21.15.165:6825/2138771601","cluster_addr":"172.21.15.165:6827/2138771601","heartbeat_back_addr":"172.21.15.165:6831/2138771601","heartbeat_front_addr":"172.21.15.165:6829/2138771601","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:17:54.791531+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:18:19.466638+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:18:43.435536+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:19:15.702685+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:19:41.767370+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:20:06.793003+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:20:39.952744+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:21:05.727262+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:21:30.735829+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:22:04.072142+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:22:30.894047+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T00:22:56.637728+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.119:6836/2109630829":"2024-03-29T00:35:52.413013+0000","172.21.15.165:6832/2013665520":"2024-03-29T00:32:07.607942+0000","172.21.15.165:6839/384483120":"2024-03-29T00:32:07.271819+0000","172.21.15.165:6838/384483120":"2024-03-29T00:32:07.271819+0000","172.21.15.119:6834/744950487":"2024-03-29T00:31:51.094723+0000","172.21.15.165:6839/768712672":"2024-03-29T00:33:20.841067+0000","172.21.15.119:0/87565965":"2024-03-29T00:16:03.582545+0000","172.21.15.119:6834/2793569574":"2024-03-29T00:35:34.679932+0000","172.21.15.165:6839/3004684592":"2024-03-29T00:31:32.429788+0000","172.21.15.165:6833/1389081045":"2024-03-29T00:31:20.067892+0000","172.21.15.165:6832/609963168":"2024-03-29T00:30:41.299818+0000","172.21.15.119:6834/2941089726":"2024-03-29T00:33:37.236449+0000","172.21.15.165:6839/2762859081":"2024-03-29T00:33:59.786505+0000","172.21.15.165:6833/609963168":"2024-03-29T00:30:41.299818+0000","172.21.15.165:6837/2717860763":"2024-03-29T00:33:02.702892+0000","172.21.15.165:6836/3170852275":"2024-03-29T00:32:25.939039+0000","172.21.15.119:6800/2056426384":"2024-03-29T00:15:30.472283+0000","172.21.15.119:6801/2575246519":"2024-03-29T00:16:03.582545+0000","172.21.15.165:6836/1531193032":"2024-03-29T00:30:24.146598+0000","172.21.15.119:6834/1659925849":"2024-03-29T00:34:17.464068+0000","172.21.15.119:6835/744950487":"2024-03-29T00:31:51.094723+0000","172.21.15.165:6836/1456402510":"2024-03-29T00:30:56.719761+0000","172.21.15.165:6837/1456402510":"2024-03-29T00:30:56.719761+0000","172.21.15.165:6839/1687390659":"2024-03-29T00:32:46.777861+0000","172.21.15.119:0/2635982178":"2024-03-29T00:16:03.582545+0000","172.21.15.119:0/2430401731":"2024-03-29T00:26:32.965486+0000","172.21.15.119:6837/482178170":"2024-03-29T00:35:17.974473+0000","172.21.15.119:6801/1401937096":"2024-03-29T00:26:32.965486+0000","172.21.15.119:0/4201699272":"2024-03-29T00:16:03.582545+0000","172.21.15.165:6838/3004684592":"2024-03-29T00:31:32.429788+0000","172.21.15.165:6836/1011925619":"2024-03-29T00:31:49.271822+0000","172.21.15.165:6837/1531193032":"2024-03-29T00:30:24.146598+0000","172.21.15.165:6833/2013665520":"2024-03-29T00:32:07.607942+0000","172.21.15.119:6834/2101147835":"2024-03-29T00:32:28.920424+0000","172.21.15.119:6835/3517971302":"2024-03-29T00:35:17.389381+0000","172.21.15.165:6832/1389081045":"2024-03-29T00:31:20.067892+0000","172.21.15.119:6835/2210305461":"2024-03-29T00:33:02.940231+0000","172.21.15.119:0/2891240357":"2024-03-29T00:15:30.472283+0000","172.21.15.119:6838/1595343240":"2024-03-29T00:34:16.710959+0000","172.21.15.119:6801/2056426384":"2024-03-29T00:15:30.472283+0000","172.21.15.119:0/357445067":"2024-03-29T00:26:32.965486+0000","172.21.15.165:6838/1687390659":"2024-03-29T00:32:46.777861+0000","172.21.15.119:6838/3947365264":"2024-03-29T00:34:59.827426+0000","172.21.15.165:6837/1011925619":"2024-03-29T00:31:49.271822+0000","172.21.15.119:6800/2575246519":"2024-03-29T00:16:03.582545+0000","172.21.15.165:6836/823338130":"2024-03-29T00:33:39.774934+0000","172.21.15.119:0/468478300":"2024-03-29T00:15:30.472283+0000","172.21.15.119:0/4119730435":"2024-03-29T00:26:32.965486+0000","172.21.15.119:6800/1401937096":"2024-03-29T00:26:32.965486+0000","172.21.15.119:0/524303043":"2024-03-29T00:26:32.965486+0000","172.21.15.119:6835/2101147835":"2024-03-29T00:32:28.920424+0000","172.21.15.165:6832/4070507195":"2024-03-29T00:32:46.507920+0000","172.21.15.119:6836/3988635716":"2024-03-29T00:34:37.192476+0000","172.21.15.165:6836/2717860763":"2024-03-29T00:33:02.702892+0000","172.21.15.119:6834/2210305461":"2024-03-29T00:33:02.940231+0000","172.21.15.165:6832/3417761142":"2024-03-29T00:33:20.267933+0000","172.21.15.165:6838/768712672":"2024-03-29T00:33:20.841067+0000","172.21.15.119:6835/2941089726":"2024-03-29T00:33:37.236449+0000","172.21.15.165:6837/3170852275":"2024-03-29T00:32:25.939039+0000","172.21.15.119:6839/1595343240":"2024-03-29T00:34:16.710959+0000","172.21.15.165:6833/4070507195":"2024-03-29T00:32:46.507920+0000","172.21.15.165:6837/823338130":"2024-03-29T00:33:39.774934+0000","172.21.15.119:6834/4197433095":"2024-03-29T00:34:37.186007+0000","172.21.15.165:6832/1344198130":"2024-03-29T00:33:59.786346+0000","172.21.15.119:6837/3511403986":"2024-03-29T00:35:35.303954+0000","172.21.15.165:6838/2762859081":"2024-03-29T00:33:59.786505+0000","172.21.15.119:6835/4197433095":"2024-03-29T00:34:37.186007+0000","172.21.15.119:6836/482178170":"2024-03-29T00:35:17.974473+0000","172.21.15.165:6833/3417761142":"2024-03-29T00:33:20.267933+0000","172.21.15.165:6833/1344198130":"2024-03-29T00:33:59.786346+0000","172.21.15.119:6835/1659925849":"2024-03-29T00:34:17.464068+0000","172.21.15.119:6837/3988635716":"2024-03-29T00:34:37.192476+0000","172.21.15.119:6836/3511403986":"2024-03-29T00:35:35.303954+0000","172.21.15.119:6834/3816731508":"2024-03-29T00:34:54.192187+0000","172.21.15.119:6837/2109630829":"2024-03-29T00:35:52.413013+0000","172.21.15.119:6835/3816731508":"2024-03-29T00:34:54.192187+0000","172.21.15.119:6839/3947365264":"2024-03-29T00:34:59.827426+0000","172.21.15.119:0/1639109126":"2024-03-29T00:15:30.472283+0000","172.21.15.119:6834/3517971302":"2024-03-29T00:35:17.389381+0000","172.21.15.119:6835/2793569574":"2024-03-29T00:35:34.679932+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T03:47:00.906 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:00.904+0000 7f2498200640 1 -- 172.21.15.119:0/3023550091 >> [v2:172.21.15.119:6800/4273482239,v1:172.21.15.119:6801/4273482239] conn(0x7f24780a2a20 msgr2=0x7f24780a4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T03:47:00.906 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:00.904+0000 7f2498200640 1 --2- 172.21.15.119:0/3023550091 >> [v2:172.21.15.119:6800/4273482239,v1:172.21.15.119:6801/4273482239] conn(0x7f24780a2a20 0x7f24780a4ee0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f24900753e0 tx=0x7f247c00b390 comp rx=0 tx=0).stop 2024-03-28T03:47:00.907 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:00.904+0000 7f2498200640 1 -- 172.21.15.119:0/3023550091 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2490101630 msgr2=0x7f24900739a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T03:47:00.907 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:00.904+0000 7f2498200640 1 --2- 172.21.15.119:0/3023550091 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2490101630 0x7f24900739a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f248400e9f0 tx=0x7f248400eeb0 comp rx=0 tx=0).stop 2024-03-28T03:47:00.907 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:00.904+0000 7f2498200640 1 -- 172.21.15.119:0/3023550091 shutdown_connections 2024-03-28T03:47:00.907 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:00.904+0000 7f2498200640 1 --2- 172.21.15.119:0/3023550091 >> [v2:172.21.15.119:6800/4273482239,v1:172.21.15.119:6801/4273482239] conn(0x7f24780a2a20 0x7f24780a4ee0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:47:00.907 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:00.904+0000 7f2498200640 1 --2- 172.21.15.119:0/3023550091 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2490107640 0x7f2490074420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:47:00.907 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:00.904+0000 7f2498200640 1 --2- 172.21.15.119:0/3023550091 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2490101ff0 0x7f2490073ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:47:00.907 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:00.904+0000 7f2498200640 1 --2- 172.21.15.119:0/3023550091 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2490101630 0x7f24900739a0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:47:00.907 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:00.904+0000 7f2498200640 1 -- 172.21.15.119:0/3023550091 >> 172.21.15.119:0/3023550091 conn(0x7f2490078aa0 msgr2=0x7f2490104060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T03:47:00.907 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:00.904+0000 7f2498200640 1 -- 172.21.15.119:0/3023550091 shutdown_connections 2024-03-28T03:47:00.907 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:00.905+0000 7f2498200640 1 -- 172.21.15.119:0/3023550091 wait complete. 2024-03-28T03:47:00.926 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-28T03:47:01.136 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:01.134+0000 7fe168800640 1 -- 172.21.15.119:0/2268311811 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe160103840 msgr2=0x7fe160103cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T03:47:01.136 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:01.134+0000 7fe168800640 1 --2- 172.21.15.119:0/2268311811 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe160103840 0x7fe160103cc0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fe150009930 tx=0x7fe15002bc20 comp rx=0 tx=0).stop 2024-03-28T03:47:01.136 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:01.134+0000 7fe168800640 1 -- 172.21.15.119:0/2268311811 shutdown_connections 2024-03-28T03:47:01.136 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:01.134+0000 7fe168800640 1 --2- 172.21.15.119:0/2268311811 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe160104200 0x7fe160110710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:47:01.137 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:01.134+0000 7fe168800640 1 --2- 172.21.15.119:0/2268311811 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe160103840 0x7fe160103cc0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:47:01.137 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:01.134+0000 7fe168800640 1 --2- 172.21.15.119:0/2268311811 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe160109850 0x7fe160109c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:47:01.137 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:01.134+0000 7fe168800640 1 -- 172.21.15.119:0/2268311811 >> 172.21.15.119:0/2268311811 conn(0x7fe1600ff350 msgr2=0x7fe160101770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T03:47:01.137 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:01.134+0000 7fe168800640 1 -- 172.21.15.119:0/2268311811 shutdown_connections 2024-03-28T03:47:01.137 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:01.134+0000 7fe168800640 1 -- 172.21.15.119:0/2268311811 wait complete. 2024-03-28T03:47:01.139 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:01.135+0000 7fe168800640 1 Processor -- start 2024-03-28T03:47:01.139 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:01.135+0000 7fe168800640 1 -- start start 2024-03-28T03:47:01.139 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:01.136+0000 7fe168800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe160103840 0x7fe16019a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T03:47:01.139 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:01.136+0000 7fe168800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe160104200 0x7fe16019a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T03:47:01.139 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:01.136+0000 7fe168800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe160109850 0x7fe16019ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T03:47:01.140 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:01.136+0000 7fe168800640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe160113cb0 con 0x7fe160104200 2024-03-28T03:47:01.140 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:01.136+0000 7fe168800640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe160113b30 con 0x7fe160103840 2024-03-28T03:47:01.140 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:01.136+0000 7fe168800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe160113e30 con 0x7fe160109850 2024-03-28T03:47:01.140 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:01.136+0000 7fe166800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe160109850 0x7fe16019ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T03:47:01.140 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:01.136+0000 7fe165e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe160103840 0x7fe16019a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T03:47:01.140 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:01.136+0000 7fe165400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe160104200 0x7fe16019a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T03:47:01.140 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:01.136+0000 7fe165e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe160103840 0x7fe16019a210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.119:53116/0 (socket says 172.21.15.119:53116) 2024-03-28T03:47:01.140 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:01.136+0000 7fe166800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe160109850 0x7fe16019ead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.119:55788/0 (socket says 172.21.15.119:55788) 2024-03-28T03:47:01.141 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:01.136+0000 7fe165e00640 1 -- 172.21.15.119:0/3173542258 learned_addr learned my addr 172.21.15.119:0/3173542258 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T03:47:01.141 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:01.136+0000 7fe165e00640 1 -- 172.21.15.119:0/3173542258 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe160109850 msgr2=0x7fe16019ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T03:47:01.141 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:01.136+0000 7fe165e00640 1 --2- 172.21.15.119:0/3173542258 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe160109850 0x7fe16019ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:47:01.141 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:01.136+0000 7fe165e00640 1 -- 172.21.15.119:0/3173542258 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe160104200 msgr2=0x7fe16019a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T03:47:01.141 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:01.136+0000 7fe165e00640 1 --2- 172.21.15.119:0/3173542258 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe160104200 0x7fe16019a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:47:01.141 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:01.136+0000 7fe165e00640 1 -- 172.21.15.119:0/3173542258 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe16019f200 con 0x7fe160103840 2024-03-28T03:47:01.141 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:01.137+0000 7fe165e00640 1 --2- 172.21.15.119:0/3173542258 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe160103840 0x7fe16019a210 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fe15400d950 tx=0x7fe15400de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T03:47:01.141 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:01.137+0000 7fe15aa00640 1 -- 172.21.15.119:0/3173542258 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe154017070 con 0x7fe160103840 2024-03-28T03:47:01.141 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:01.137+0000 7fe15aa00640 1 -- 172.21.15.119:0/3173542258 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe154009c30 con 0x7fe160103840 2024-03-28T03:47:01.141 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:01.137+0000 7fe15aa00640 1 -- 172.21.15.119:0/3173542258 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe15400f6b0 con 0x7fe160103840 2024-03-28T03:47:01.142 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:01.137+0000 7fe168800640 1 -- 172.21.15.119:0/3173542258 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe16019f4f0 con 0x7fe160103840 2024-03-28T03:47:01.142 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:01.137+0000 7fe168800640 1 -- 172.21.15.119:0/3173542258 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1601a6d80 con 0x7fe160103840 2024-03-28T03:47:01.142 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:01.138+0000 7fe168800640 1 -- 172.21.15.119:0/3173542258 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1600688e0 con 0x7fe160103840 2024-03-28T03:47:01.142 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:01.140+0000 7fe15aa00640 1 -- 172.21.15.119:0/3173542258 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 27) v1 ==== 137422+0+0 (secure 0 0 0) 0x7fe1540109d0 con 0x7fe160103840 2024-03-28T03:47:01.144 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:01.142+0000 7fe15aa00640 1 --2- 172.21.15.119:0/3173542258 >> [v2:172.21.15.119:6800/4273482239,v1:172.21.15.119:6801/4273482239] conn(0x7fe1480a29f0 0x7fe1480a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T03:47:01.144 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:01.142+0000 7fe15aa00640 1 -- 172.21.15.119:0/3173542258 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 11163+0+0 (secure 0 0 0) 0x7fe1540cf3a0 con 0x7fe160103840 2024-03-28T03:47:01.144 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:01.142+0000 7fe165400640 1 --2- 172.21.15.119:0/3173542258 >> [v2:172.21.15.119:6800/4273482239,v1:172.21.15.119:6801/4273482239] conn(0x7fe1480a29f0 0x7fe1480a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T03:47:01.144 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:01.143+0000 7fe165400640 1 --2- 172.21.15.119:0/3173542258 >> [v2:172.21.15.119:6800/4273482239,v1:172.21.15.119:6801/4273482239] conn(0x7fe1480a29f0 0x7fe1480a4eb0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fe150009900 tx=0x7fe150005a20 comp rx=0 tx=0).ready entity=mgr.25277 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T03:47:01.148 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:01.146+0000 7fe15aa00640 1 -- 172.21.15.119:0/3173542258 <== mon.2 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fe154009de0 con 0x7fe160103840 2024-03-28T03:47:01.390 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:01.388+0000 7fe168800640 1 -- 172.21.15.119:0/3173542258 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fe16010bdd0 con 0x7fe160103840 2024-03-28T03:47:01.404 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:01.402+0000 7fe15aa00640 1 -- 172.21.15.119:0/3173542258 <== mon.2 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v179) v1 ==== 161+0+0 (secure 0 0 0) 0x7fe15409b000 con 0x7fe160103840 2024-03-28T03:47:01.405 INFO:teuthology.orchestra.run.smithi119.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-28T03:47:01.408 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:01.406+0000 7fe168800640 1 -- 172.21.15.119:0/3173542258 >> [v2:172.21.15.119:6800/4273482239,v1:172.21.15.119:6801/4273482239] conn(0x7fe1480a29f0 msgr2=0x7fe1480a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T03:47:01.408 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:01.406+0000 7fe168800640 1 --2- 172.21.15.119:0/3173542258 >> [v2:172.21.15.119:6800/4273482239,v1:172.21.15.119:6801/4273482239] conn(0x7fe1480a29f0 0x7fe1480a4eb0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fe150009900 tx=0x7fe150005a20 comp rx=0 tx=0).stop 2024-03-28T03:47:01.409 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:01.406+0000 7fe168800640 1 -- 172.21.15.119:0/3173542258 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe160103840 msgr2=0x7fe16019a210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T03:47:01.409 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:01.406+0000 7fe168800640 1 --2- 172.21.15.119:0/3173542258 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe160103840 0x7fe16019a210 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fe15400d950 tx=0x7fe15400de10 comp rx=0 tx=0).stop 2024-03-28T03:47:01.409 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:01.406+0000 7fe168800640 1 -- 172.21.15.119:0/3173542258 shutdown_connections 2024-03-28T03:47:01.409 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:01.406+0000 7fe168800640 1 --2- 172.21.15.119:0/3173542258 >> [v2:172.21.15.119:6800/4273482239,v1:172.21.15.119:6801/4273482239] conn(0x7fe1480a29f0 0x7fe1480a4eb0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:47:01.409 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:01.406+0000 7fe168800640 1 --2- 172.21.15.119:0/3173542258 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe160109850 0x7fe16019ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:47:01.409 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:01.406+0000 7fe168800640 1 --2- 172.21.15.119:0/3173542258 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe160104200 0x7fe16019a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:47:01.409 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:01.406+0000 7fe168800640 1 --2- 172.21.15.119:0/3173542258 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe160103840 0x7fe16019a210 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:47:01.409 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:01.406+0000 7fe168800640 1 -- 172.21.15.119:0/3173542258 >> 172.21.15.119:0/3173542258 conn(0x7fe1600ff350 msgr2=0x7fe1600ff9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T03:47:01.410 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:01.407+0000 7fe168800640 1 -- 172.21.15.119:0/3173542258 shutdown_connections 2024-03-28T03:47:01.410 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:01.407+0000 7fe168800640 1 -- 172.21.15.119:0/3173542258 wait complete. 2024-03-28T03:47:01.430 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-28T03:47:01.442 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:47:01 smithi132 ceph-mon[35941]: 42 slow requests, 0 included below; oldest blocked for > 11480.823665 secs 2024-03-28T03:47:01.442 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:47:01 smithi132 ceph-mon[35941]: from='client.? 172.21.15.119:0/3246622926' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T03:47:01.442 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:47:01 smithi132 ceph-mon[35941]: from='client.? 172.21.15.119:0/3023550091' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-28T03:47:01.442 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:47:01 smithi132 ceph-mon[35941]: pgmap v6267: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T03:47:01.484 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:47:01 smithi119 ceph-mon[31518]: 42 slow requests, 0 included below; oldest blocked for > 11480.823665 secs 2024-03-28T03:47:01.484 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:47:01 smithi119 ceph-mon[31518]: from='client.? 172.21.15.119:0/3246622926' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T03:47:01.484 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:47:01 smithi119 ceph-mon[31518]: from='client.? 172.21.15.119:0/3023550091' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-28T03:47:01.485 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:47:01 smithi119 ceph-mon[31518]: pgmap v6267: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T03:47:01.485 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:47:01 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mon-a[31514]: 2024-03-28T03:47:01.391+0000 7f1f64800700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-28T03:47:01.624 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:01.621+0000 7f06fa000640 1 -- 172.21.15.119:0/3861590836 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f06f40743f0 msgr2=0x7f06f410d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T03:47:01.624 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:01.621+0000 7f06fa000640 1 --2- 172.21.15.119:0/3861590836 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f06f40743f0 0x7f06f410d930 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f06e0009860 tx=0x7f06e002bc00 comp rx=0 tx=0).stop 2024-03-28T03:47:01.624 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:01.622+0000 7f06fa000640 1 -- 172.21.15.119:0/3861590836 shutdown_connections 2024-03-28T03:47:01.624 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:01.622+0000 7f06fa000640 1 --2- 172.21.15.119:0/3861590836 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f06f410de70 0x7f06f41102c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:47:01.625 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:01.622+0000 7f06fa000640 1 --2- 172.21.15.119:0/3861590836 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f06f40743f0 0x7f06f410d930 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:47:01.625 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:01.622+0000 7f06fa000640 1 --2- 172.21.15.119:0/3861590836 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f06f4073ad0 0x7f06f4073eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:47:01.625 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:01.622+0000 7f06fa000640 1 -- 172.21.15.119:0/3861590836 >> 172.21.15.119:0/3861590836 conn(0x7f06f4078aa0 msgr2=0x7f06f4078eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T03:47:01.625 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:01.622+0000 7f06fa000640 1 -- 172.21.15.119:0/3861590836 shutdown_connections 2024-03-28T03:47:01.625 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:01.622+0000 7f06fa000640 1 -- 172.21.15.119:0/3861590836 wait complete. 2024-03-28T03:47:01.627 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:01.623+0000 7f06fa000640 1 Processor -- start 2024-03-28T03:47:01.627 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:01.623+0000 7f06fa000640 1 -- start start 2024-03-28T03:47:01.627 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:01.623+0000 7f06fa000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f06f4073ad0 0x7f06f419a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T03:47:01.628 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:01.623+0000 7f06fa000640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f06f40743f0 0x7f06f419a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T03:47:01.628 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:01.623+0000 7f06fa000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f06f410de70 0x7f06f419e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T03:47:01.628 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:01.623+0000 7f06fa000640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06f4113c10 con 0x7f06f4073ad0 2024-03-28T03:47:01.628 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:01.623+0000 7f06fa000640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06f4113a90 con 0x7f06f40743f0 2024-03-28T03:47:01.628 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:01.623+0000 7f06fa000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06f4113d90 con 0x7f06f410de70 2024-03-28T03:47:01.628 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:01.623+0000 7f06f3400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f06f4073ad0 0x7f06f419a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T03:47:01.628 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:01.623+0000 7f06f3400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f06f4073ad0 0x7f06f419a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51462/0 (socket says 172.21.15.119:51462) 2024-03-28T03:47:01.628 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:01.623+0000 7f06f3400640 1 -- 172.21.15.119:0/1142311977 learned_addr learned my addr 172.21.15.119:0/1142311977 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T03:47:01.629 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:01.623+0000 7f06f3e00640 1 --2- 172.21.15.119:0/1142311977 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f06f410de70 0x7f06f419e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T03:47:01.629 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:01.623+0000 7f06f2a00640 1 --2- 172.21.15.119:0/1142311977 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f06f40743f0 0x7f06f419a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T03:47:01.629 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:01.624+0000 7f06f2a00640 1 -- 172.21.15.119:0/1142311977 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f06f410de70 msgr2=0x7f06f419e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T03:47:01.629 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:01.624+0000 7f06f2a00640 1 --2- 172.21.15.119:0/1142311977 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f06f410de70 0x7f06f419e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:47:01.629 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:01.624+0000 7f06f2a00640 1 -- 172.21.15.119:0/1142311977 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f06f4073ad0 msgr2=0x7f06f419a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T03:47:01.629 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:01.624+0000 7f06f2a00640 1 --2- 172.21.15.119:0/1142311977 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f06f4073ad0 0x7f06f419a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:47:01.629 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:01.624+0000 7f06f2a00640 1 -- 172.21.15.119:0/1142311977 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06f419f120 con 0x7f06f40743f0 2024-03-28T03:47:01.629 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:01.624+0000 7f06f3e00640 1 --2- 172.21.15.119:0/1142311977 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f06f410de70 0x7f06f419e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T03:47:01.629 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:01.624+0000 7f06f3400640 1 --2- 172.21.15.119:0/1142311977 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f06f4073ad0 0x7f06f419a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T03:47:01.630 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:01.624+0000 7f06f2a00640 1 --2- 172.21.15.119:0/1142311977 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f06f40743f0 0x7f06f419a670 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f06e0037040 tx=0x7f06e00344c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T03:47:01.630 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:01.625+0000 7f06ebe00640 1 -- 172.21.15.119:0/1142311977 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06e0041020 con 0x7f06f40743f0 2024-03-28T03:47:01.630 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:01.625+0000 7f06ebe00640 1 -- 172.21.15.119:0/1142311977 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f06e002ca80 con 0x7f06f40743f0 2024-03-28T03:47:01.630 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:01.625+0000 7f06fa000640 1 -- 172.21.15.119:0/1142311977 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06f419f3b0 con 0x7f06f40743f0 2024-03-28T03:47:01.630 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:01.625+0000 7f06ebe00640 1 -- 172.21.15.119:0/1142311977 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06e0039040 con 0x7f06f40743f0 2024-03-28T03:47:01.630 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:01.625+0000 7f06fa000640 1 -- 172.21.15.119:0/1142311977 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06f41a6ca0 con 0x7f06f40743f0 2024-03-28T03:47:01.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:47:01 smithi165 ceph-mon[36302]: 42 slow requests, 0 included below; oldest blocked for > 11480.823665 secs 2024-03-28T03:47:01.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:47:01 smithi165 ceph-mon[36302]: from='client.? 172.21.15.119:0/3246622926' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T03:47:01.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:47:01 smithi165 ceph-mon[36302]: from='client.? 172.21.15.119:0/3023550091' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-28T03:47:01.631 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:47:01 smithi165 ceph-mon[36302]: pgmap v6267: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T03:47:01.632 INFO:journalctl@ceph.mds.f.smithi165.stdout:Mar 28 03:47:01 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-f[67456]: ignoring --setuser ceph since I am not root 2024-03-28T03:47:01.632 INFO:journalctl@ceph.mds.f.smithi165.stdout:Mar 28 03:47:01 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-f[67456]: ignoring --setgroup ceph since I am not root 2024-03-28T03:47:01.632 INFO:journalctl@ceph.mds.f.smithi165.stdout:Mar 28 03:47:01 smithi165 ceph-mds[67460]: -- 172.21.15.165:0/3825993544 <== mon.1 v2:172.21.15.165:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55d0ba775a40 con 0x55d0ba748800 2024-03-28T03:47:01.632 INFO:journalctl@ceph.mds.f.smithi165.stdout:Mar 28 03:47:01 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-f[67456]: starting mds.f at 2024-03-28T03:47:01.633 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:01.626+0000 7f06fa000640 1 -- 172.21.15.119:0/1142311977 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06f40688e0 con 0x7f06f40743f0 2024-03-28T03:47:01.633 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:01.628+0000 7f06ebe00640 1 -- 172.21.15.119:0/1142311977 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 27) v1 ==== 137422+0+0 (secure 0 0 0) 0x7f06e0044020 con 0x7f06f40743f0 2024-03-28T03:47:01.633 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:01.630+0000 7f06ebe00640 1 --2- 172.21.15.119:0/1142311977 >> [v2:172.21.15.119:6800/4273482239,v1:172.21.15.119:6801/4273482239] conn(0x7f06d80a2970 0x7f06d80a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T03:47:01.633 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:01.630+0000 7f06ebe00640 1 -- 172.21.15.119:0/1142311977 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 11593+0+0 (secure 0 0 0) 0x7f06e00f1010 con 0x7f06f40743f0 2024-03-28T03:47:01.633 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:01.630+0000 7f06f3400640 1 --2- 172.21.15.119:0/1142311977 >> [v2:172.21.15.119:6800/4273482239,v1:172.21.15.119:6801/4273482239] conn(0x7f06d80a2970 0x7f06d80a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T03:47:01.633 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:01.631+0000 7f06f3400640 1 --2- 172.21.15.119:0/1142311977 >> [v2:172.21.15.119:6800/4273482239,v1:172.21.15.119:6801/4273482239] conn(0x7f06d80a2970 0x7f06d80a4e30 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f06ec00e8b0 tx=0x7f06ec0095c0 comp rx=0 tx=0).ready entity=mgr.25277 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T03:47:01.635 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:01.634+0000 7f06ebe00640 1 -- 172.21.15.119:0/1142311977 <== mon.2 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f06e004b020 con 0x7f06f40743f0 2024-03-28T03:47:01.699 INFO:journalctl@ceph.mds.h.smithi132.stdout:Mar 28 03:47:01 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-h[66129]: ignoring --setuser ceph since I am not root 2024-03-28T03:47:01.699 INFO:journalctl@ceph.mds.h.smithi132.stdout:Mar 28 03:47:01 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-h[66129]: ignoring --setgroup ceph since I am not root 2024-03-28T03:47:01.699 INFO:journalctl@ceph.mds.h.smithi132.stdout:Mar 28 03:47:01 smithi132 ceph-mds[66133]: -- 172.21.15.132:0/2728437972 <== mon.1 v2:172.21.15.165:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55662bf5ba40 con 0x55662bf2ec00 2024-03-28T03:47:01.699 INFO:journalctl@ceph.mds.h.smithi132.stdout:Mar 28 03:47:01 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-h[66129]: starting mds.h at 2024-03-28T03:47:01.699 INFO:journalctl@ceph.mds.k.smithi132.stdout:Mar 28 03:47:01 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-k[65498]: ignoring --setuser ceph since I am not root 2024-03-28T03:47:01.699 INFO:journalctl@ceph.mds.k.smithi132.stdout:Mar 28 03:47:01 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-k[65498]: ignoring --setgroup ceph since I am not root 2024-03-28T03:47:01.700 INFO:journalctl@ceph.mds.k.smithi132.stdout:Mar 28 03:47:01 smithi132 ceph-mds[65502]: -- 172.21.15.132:0/3512345611 <== mon.2 v2:172.21.15.132:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x555db73fda40 con 0x555db73d0c00 2024-03-28T03:47:01.700 INFO:journalctl@ceph.mds.k.smithi132.stdout:Mar 28 03:47:01 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-k[65498]: starting mds.k at 2024-03-28T03:47:01.700 INFO:journalctl@ceph.mds.b.smithi132.stdout:Mar 28 03:47:01 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-b[66443]: ignoring --setuser ceph since I am not root 2024-03-28T03:47:01.700 INFO:journalctl@ceph.mds.b.smithi132.stdout:Mar 28 03:47:01 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-b[66443]: ignoring --setgroup ceph since I am not root 2024-03-28T03:47:01.700 INFO:journalctl@ceph.mds.b.smithi132.stdout:Mar 28 03:47:01 smithi132 ceph-mds[66447]: -- 172.21.15.132:0/3210095346 <== mon.1 v2:172.21.15.165:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5594ca905a40 con 0x5594ca8d8c00 2024-03-28T03:47:01.700 INFO:journalctl@ceph.mds.b.smithi132.stdout:Mar 28 03:47:01 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-b[66443]: starting mds.b at 2024-03-28T03:47:01.701 INFO:journalctl@ceph.mds.e.smithi132.stdout:Mar 28 03:47:01 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-e[65818]: ignoring --setuser ceph since I am not root 2024-03-28T03:47:01.701 INFO:journalctl@ceph.mds.e.smithi132.stdout:Mar 28 03:47:01 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-e[65818]: ignoring --setgroup ceph since I am not root 2024-03-28T03:47:01.701 INFO:journalctl@ceph.mds.e.smithi132.stdout:Mar 28 03:47:01 smithi132 ceph-mds[65822]: -- 172.21.15.132:0/2494988510 <== mon.2 v2:172.21.15.132:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55a009a0fa40 con 0x55a0099e2800 2024-03-28T03:47:01.701 INFO:journalctl@ceph.mds.e.smithi132.stdout:Mar 28 03:47:01 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-e[65818]: starting mds.e at 2024-03-28T03:47:01.872 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:01.870+0000 7f06fa000640 1 -- 172.21.15.119:0/1142311977 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f06f4108a60 con 0x7f06f40743f0 2024-03-28T03:47:01.902 INFO:journalctl@ceph.mds.a.smithi119.stdout:Mar 28 03:47:01 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-a[70147]: ignoring --setuser ceph since I am not root 2024-03-28T03:47:01.902 INFO:journalctl@ceph.mds.a.smithi119.stdout:Mar 28 03:47:01 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-a[70147]: ignoring --setgroup ceph since I am not root 2024-03-28T03:47:01.902 INFO:journalctl@ceph.mds.a.smithi119.stdout:Mar 28 03:47:01 smithi119 ceph-mds[70153]: -- 172.21.15.119:0/3977798593 <== mon.1 v2:172.21.15.165:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x560489b73a40 con 0x560489b47000 2024-03-28T03:47:01.902 INFO:journalctl@ceph.mds.a.smithi119.stdout:Mar 28 03:47:01 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-a[70147]: starting mds.a at 2024-03-28T03:47:01.902 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 03:47:01 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[121385]: ignoring --setuser ceph since I am not root 2024-03-28T03:47:01.902 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 03:47:01 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[121385]: ignoring --setgroup ceph since I am not root 2024-03-28T03:47:01.902 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 03:47:01 smithi119 ceph-mds[121389]: -- 172.21.15.119:0/1784314598 <== mon.1 v2:172.21.15.165:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x556498b6da40 con 0x556498b40800 2024-03-28T03:47:01.903 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 03:47:01 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[121385]: starting mds.d at 2024-03-28T03:47:02.402 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:02.400+0000 7f06ebe00640 1 -- 172.21.15.119:0/1142311977 <== mon.2 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v180) v1 ==== 104+0+0 (secure 0 0 0) 0x7f06e00bd280 con 0x7f06f40743f0 2024-03-28T03:47:02.406 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:02.404+0000 7f06fa000640 1 -- 172.21.15.119:0/1142311977 >> [v2:172.21.15.119:6800/4273482239,v1:172.21.15.119:6801/4273482239] conn(0x7f06d80a2970 msgr2=0x7f06d80a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T03:47:02.406 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:02.404+0000 7f06fa000640 1 --2- 172.21.15.119:0/1142311977 >> [v2:172.21.15.119:6800/4273482239,v1:172.21.15.119:6801/4273482239] conn(0x7f06d80a2970 0x7f06d80a4e30 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f06ec00e8b0 tx=0x7f06ec0095c0 comp rx=0 tx=0).stop 2024-03-28T03:47:02.406 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:02.404+0000 7f06fa000640 1 -- 172.21.15.119:0/1142311977 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f06f40743f0 msgr2=0x7f06f419a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T03:47:02.406 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:02.404+0000 7f06fa000640 1 --2- 172.21.15.119:0/1142311977 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f06f40743f0 0x7f06f419a670 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f06e0037040 tx=0x7f06e00344c0 comp rx=0 tx=0).stop 2024-03-28T03:47:02.407 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:02.404+0000 7f06fa000640 1 -- 172.21.15.119:0/1142311977 shutdown_connections 2024-03-28T03:47:02.407 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:02.404+0000 7f06fa000640 1 --2- 172.21.15.119:0/1142311977 >> [v2:172.21.15.119:6800/4273482239,v1:172.21.15.119:6801/4273482239] conn(0x7f06d80a2970 0x7f06d80a4e30 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:47:02.407 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:02.404+0000 7f06fa000640 1 --2- 172.21.15.119:0/1142311977 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f06f410de70 0x7f06f419e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:47:02.407 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:02.404+0000 7f06fa000640 1 --2- 172.21.15.119:0/1142311977 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f06f40743f0 0x7f06f419a670 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:47:02.408 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:02.404+0000 7f06fa000640 1 --2- 172.21.15.119:0/1142311977 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f06f4073ad0 0x7f06f419a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:47:02.408 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:02.404+0000 7f06fa000640 1 -- 172.21.15.119:0/1142311977 >> 172.21.15.119:0/1142311977 conn(0x7f06f4078aa0 msgr2=0x7f06f40fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T03:47:02.408 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:02.405+0000 7f06fa000640 1 -- 172.21.15.119:0/1142311977 shutdown_connections 2024-03-28T03:47:02.408 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:02.405+0000 7f06fa000640 1 -- 172.21.15.119:0/1142311977 wait complete. 2024-03-28T03:47:02.429 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-28T03:47:02.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:47:02 smithi132 ceph-mon[35941]: from='client.? 172.21.15.119:0/3173542258' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-28T03:47:02.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:47:02 smithi132 ceph-mon[35941]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-28T03:47:02.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:47:02 smithi132 ceph-mon[35941]: Health check failed: 1 filesystem has a failed mds daemon (FS_WITH_FAILED_MDS) 2024-03-28T03:47:02.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:47:02 smithi132 ceph-mon[35941]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-28T03:47:02.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:47:02 smithi132 ceph-mon[35941]: Health check cleared: MDS_SLOW_REQUEST (was: 1 MDSs report slow requests) 2024-03-28T03:47:02.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:47:02 smithi132 ceph-mon[35941]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-03-28T03:47:02.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:47:02 smithi132 ceph-mon[35941]: osdmap e126: 12 total, 12 up, 12 in 2024-03-28T03:47:02.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:47:02 smithi132 ceph-mon[35941]: fsmap cephfs:0/5, 5 failed 2024-03-28T03:47:02.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:47:02 smithi132 ceph-mon[35941]: from='client.? 172.21.15.119:0/1142311977' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-03-28T03:47:02.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:47:02 smithi132 ceph-mon[35941]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-03-28T03:47:02.499 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:47:02 smithi119 ceph-mon[31518]: from='client.? 172.21.15.119:0/3173542258' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-28T03:47:02.499 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:47:02 smithi119 ceph-mon[31518]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-28T03:47:02.499 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:47:02 smithi119 ceph-mon[31518]: Health check failed: 1 filesystem has a failed mds daemon (FS_WITH_FAILED_MDS) 2024-03-28T03:47:02.499 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:47:02 smithi119 ceph-mon[31518]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-28T03:47:02.499 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:47:02 smithi119 ceph-mon[31518]: Health check cleared: MDS_SLOW_REQUEST (was: 1 MDSs report slow requests) 2024-03-28T03:47:02.500 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:47:02 smithi119 ceph-mon[31518]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-03-28T03:47:02.500 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:47:02 smithi119 ceph-mon[31518]: osdmap e126: 12 total, 12 up, 12 in 2024-03-28T03:47:02.500 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:47:02 smithi119 ceph-mon[31518]: fsmap cephfs:0/5, 5 failed 2024-03-28T03:47:02.500 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:47:02 smithi119 ceph-mon[31518]: from='client.? 172.21.15.119:0/1142311977' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-03-28T03:47:02.500 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:47:02 smithi119 ceph-mon[31518]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-03-28T03:47:02.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:47:02 smithi165 ceph-mon[36302]: from='client.? 172.21.15.119:0/3173542258' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-28T03:47:02.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:47:02 smithi165 ceph-mon[36302]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-28T03:47:02.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:47:02 smithi165 ceph-mon[36302]: Health check failed: 1 filesystem has a failed mds daemon (FS_WITH_FAILED_MDS) 2024-03-28T03:47:02.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:47:02 smithi165 ceph-mon[36302]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-28T03:47:02.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:47:02 smithi165 ceph-mon[36302]: Health check cleared: MDS_SLOW_REQUEST (was: 1 MDSs report slow requests) 2024-03-28T03:47:02.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:47:02 smithi165 ceph-mon[36302]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-03-28T03:47:02.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:47:02 smithi165 ceph-mon[36302]: osdmap e126: 12 total, 12 up, 12 in 2024-03-28T03:47:02.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:47:02 smithi165 ceph-mon[36302]: fsmap cephfs:0/5, 5 failed 2024-03-28T03:47:02.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:47:02 smithi165 ceph-mon[36302]: from='client.? 172.21.15.119:0/1142311977' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-03-28T03:47:02.629 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:47:02 smithi165 ceph-mon[36302]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-03-28T03:47:02.632 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:02.630+0000 7f2311e00640 1 -- 172.21.15.119:0/392307954 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f230c105dc0 msgr2=0x7f230c1061a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T03:47:02.633 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:02.630+0000 7f2311e00640 1 --2- 172.21.15.119:0/392307954 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f230c105dc0 0x7f230c1061a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f22fc009900 tx=0x7f22fc02bc30 comp rx=0 tx=0).stop 2024-03-28T03:47:02.633 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:02.630+0000 7f2311e00640 1 -- 172.21.15.119:0/392307954 shutdown_connections 2024-03-28T03:47:02.633 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:02.630+0000 7f2311e00640 1 --2- 172.21.15.119:0/392307954 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f230c1009c0 0x7f230c10cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:47:02.633 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:02.630+0000 7f2311e00640 1 --2- 172.21.15.119:0/392307954 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f230c100020 0x7f230c100480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:47:02.633 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:02.630+0000 7f2311e00640 1 --2- 172.21.15.119:0/392307954 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f230c105dc0 0x7f230c1061a0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:47:02.633 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:02.630+0000 7f2311e00640 1 -- 172.21.15.119:0/392307954 >> 172.21.15.119:0/392307954 conn(0x7f230c0766f0 msgr2=0x7f230c0fda80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T03:47:02.633 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:02.631+0000 7f2311e00640 1 -- 172.21.15.119:0/392307954 shutdown_connections 2024-03-28T03:47:02.634 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:02.631+0000 7f2311e00640 1 -- 172.21.15.119:0/392307954 wait complete. 2024-03-28T03:47:02.636 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:02.631+0000 7f2311e00640 1 Processor -- start 2024-03-28T03:47:02.636 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:02.632+0000 7f2311e00640 1 -- start start 2024-03-28T03:47:02.636 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:02.632+0000 7f2311e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f230c100020 0x7f230c19a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T03:47:02.636 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:02.632+0000 7f2311e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f230c1009c0 0x7f230c19a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T03:47:02.636 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:02.632+0000 7f2311e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f230c105dc0 0x7f230c19e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T03:47:02.636 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:02.632+0000 7f2311e00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f230c111a50 con 0x7f230c1009c0 2024-03-28T03:47:02.636 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:02.632+0000 7f2311e00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f230c1118d0 con 0x7f230c100020 2024-03-28T03:47:02.637 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:02.632+0000 7f2311e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f230c111bd0 con 0x7f230c105dc0 2024-03-28T03:47:02.637 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:02.632+0000 7f230a000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f230c1009c0 0x7f230c19a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T03:47:02.637 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:02.632+0000 7f230a000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f230c1009c0 0x7f230c19a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51468/0 (socket says 172.21.15.119:51468) 2024-03-28T03:47:02.637 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:02.632+0000 7f230a000640 1 -- 172.21.15.119:0/2488880046 learned_addr learned my addr 172.21.15.119:0/2488880046 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T03:47:02.637 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:02.632+0000 7f230b400640 1 --2- 172.21.15.119:0/2488880046 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f230c105dc0 0x7f230c19e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T03:47:02.637 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:02.632+0000 7f230aa00640 1 --2- 172.21.15.119:0/2488880046 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f230c100020 0x7f230c19a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T03:47:02.637 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:02.633+0000 7f230a000640 1 -- 172.21.15.119:0/2488880046 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f230c105dc0 msgr2=0x7f230c19e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T03:47:02.637 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:02.633+0000 7f230a000640 1 --2- 172.21.15.119:0/2488880046 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f230c105dc0 0x7f230c19e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:47:02.637 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:02.633+0000 7f230a000640 1 -- 172.21.15.119:0/2488880046 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f230c100020 msgr2=0x7f230c19a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T03:47:02.638 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:02.633+0000 7f230a000640 1 --2- 172.21.15.119:0/2488880046 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f230c100020 0x7f230c19a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:47:02.638 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:02.633+0000 7f230a000640 1 -- 172.21.15.119:0/2488880046 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f230c19f120 con 0x7f230c1009c0 2024-03-28T03:47:02.638 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:02.633+0000 7f230a000640 1 --2- 172.21.15.119:0/2488880046 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f230c1009c0 0x7f230c19a670 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f22f800e9f0 tx=0x7f22f800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T03:47:02.638 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:02.633+0000 7f2310a00640 1 -- 172.21.15.119:0/2488880046 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22f800ccc0 con 0x7f230c1009c0 2024-03-28T03:47:02.638 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:02.633+0000 7f2311e00640 1 -- 172.21.15.119:0/2488880046 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f230c19f410 con 0x7f230c1009c0 2024-03-28T03:47:02.638 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:02.633+0000 7f2310a00640 1 -- 172.21.15.119:0/2488880046 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f22f8010910 con 0x7f230c1009c0 2024-03-28T03:47:02.638 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:02.633+0000 7f2310a00640 1 -- 172.21.15.119:0/2488880046 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22f80188a0 con 0x7f230c1009c0 2024-03-28T03:47:02.638 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:02.633+0000 7f2311e00640 1 -- 172.21.15.119:0/2488880046 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f230c1a6ca0 con 0x7f230c1009c0 2024-03-28T03:47:02.639 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:02.635+0000 7f2311e00640 1 -- 172.21.15.119:0/2488880046 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f230c0688e0 con 0x7f230c1009c0 2024-03-28T03:47:02.639 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:02.636+0000 7f2310a00640 1 -- 172.21.15.119:0/2488880046 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 27) v1 ==== 137422+0+0 (secure 0 0 0) 0x7f22f8010ac0 con 0x7f230c1009c0 2024-03-28T03:47:02.641 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:02.638+0000 7f2310a00640 1 --2- 172.21.15.119:0/2488880046 >> [v2:172.21.15.119:6800/4273482239,v1:172.21.15.119:6801/4273482239] conn(0x7f22f00a2970 0x7f22f00a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T03:47:02.641 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:02.639+0000 7f2310a00640 1 -- 172.21.15.119:0/2488880046 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 11593+0+0 (secure 0 0 0) 0x7f22f80cff10 con 0x7f230c1009c0 2024-03-28T03:47:02.641 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:02.639+0000 7f230aa00640 1 --2- 172.21.15.119:0/2488880046 >> [v2:172.21.15.119:6800/4273482239,v1:172.21.15.119:6801/4273482239] conn(0x7f22f00a2970 0x7f22f00a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T03:47:02.641 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:02.639+0000 7f230aa00640 1 --2- 172.21.15.119:0/2488880046 >> [v2:172.21.15.119:6800/4273482239,v1:172.21.15.119:6801/4273482239] conn(0x7f22f00a2970 0x7f22f00a4e30 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f22fc006010 tx=0x7f22fc005dd0 comp rx=0 tx=0).ready entity=mgr.25277 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T03:47:02.645 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:02.644+0000 7f2310a00640 1 -- 172.21.15.119:0/2488880046 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f22f809c010 con 0x7f230c1009c0 2024-03-28T03:47:02.830 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:02.828+0000 7f2311e00640 1 -- 172.21.15.119:0/2488880046 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f230c1a6f40 con 0x7f230c1009c0 2024-03-28T03:47:03.404 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:03.401+0000 7f2310a00640 1 -- 172.21.15.119:0/2488880046 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v127) v1 ==== 181+0+0 (secure 0 0 0) 0x7f22f809b910 con 0x7f230c1009c0 2024-03-28T03:47:03.404 INFO:teuthology.orchestra.run.smithi119.stderr:pool 'cephfs_metadata' removed 2024-03-28T03:47:03.407 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:03.405+0000 7f2311e00640 1 -- 172.21.15.119:0/2488880046 >> [v2:172.21.15.119:6800/4273482239,v1:172.21.15.119:6801/4273482239] conn(0x7f22f00a2970 msgr2=0x7f22f00a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T03:47:03.407 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:03.405+0000 7f2311e00640 1 --2- 172.21.15.119:0/2488880046 >> [v2:172.21.15.119:6800/4273482239,v1:172.21.15.119:6801/4273482239] conn(0x7f22f00a2970 0x7f22f00a4e30 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f22fc006010 tx=0x7f22fc005dd0 comp rx=0 tx=0).stop 2024-03-28T03:47:03.407 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:03.405+0000 7f2311e00640 1 -- 172.21.15.119:0/2488880046 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f230c1009c0 msgr2=0x7f230c19a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T03:47:03.407 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:03.405+0000 7f2311e00640 1 --2- 172.21.15.119:0/2488880046 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f230c1009c0 0x7f230c19a670 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f22f800e9f0 tx=0x7f22f800edf0 comp rx=0 tx=0).stop 2024-03-28T03:47:03.408 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:03.405+0000 7f2311e00640 1 -- 172.21.15.119:0/2488880046 shutdown_connections 2024-03-28T03:47:03.408 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:03.405+0000 7f2311e00640 1 --2- 172.21.15.119:0/2488880046 >> [v2:172.21.15.119:6800/4273482239,v1:172.21.15.119:6801/4273482239] conn(0x7f22f00a2970 0x7f22f00a4e30 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:47:03.408 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:03.405+0000 7f2311e00640 1 --2- 172.21.15.119:0/2488880046 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f230c105dc0 0x7f230c19e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:47:03.408 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:03.405+0000 7f2311e00640 1 --2- 172.21.15.119:0/2488880046 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f230c1009c0 0x7f230c19a670 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:47:03.408 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:03.405+0000 7f2311e00640 1 --2- 172.21.15.119:0/2488880046 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f230c100020 0x7f230c19a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:47:03.408 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:03.405+0000 7f2311e00640 1 -- 172.21.15.119:0/2488880046 >> 172.21.15.119:0/2488880046 conn(0x7f230c0766f0 msgr2=0x7f230c0fdba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T03:47:03.408 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:03.405+0000 7f2311e00640 1 -- 172.21.15.119:0/2488880046 shutdown_connections 2024-03-28T03:47:03.408 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:03.405+0000 7f2311e00640 1 -- 172.21.15.119:0/2488880046 wait complete. 2024-03-28T03:47:03.432 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-28T03:47:03.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:47:03 smithi132 ceph-mon[35941]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-28T03:47:03.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:47:03 smithi132 ceph-mon[35941]: Health check cleared: FS_WITH_FAILED_MDS (was: 1 filesystem has a failed mds daemon) 2024-03-28T03:47:03.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:47:03 smithi132 ceph-mon[35941]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-28T03:47:03.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:47:03 smithi132 ceph-mon[35941]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-03-28T03:47:03.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:47:03 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-28T03:47:03.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:47:03 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-28T03:47:03.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:47:03 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-28T03:47:03.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:47:03 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-28T03:47:03.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:47:03 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-28T03:47:03.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:47:03 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-28T03:47:03.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:47:03 smithi132 ceph-mon[35941]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-28T03:47:03.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:47:03 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.165:6832/1722638690,v1:172.21.15.165:6833/1722638690] up:boot 2024-03-28T03:47:03.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:47:03 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.132:6834/2349242573,v1:172.21.15.132:6835/2349242573] up:boot 2024-03-28T03:47:03.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:47:03 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.132:6832/1407476002,v1:172.21.15.132:6833/1407476002] up:boot 2024-03-28T03:47:03.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:47:03 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.132:6838/1704684771,v1:172.21.15.132:6839/1704684771] up:boot 2024-03-28T03:47:03.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:47:03 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.132:6836/3612537000,v1:172.21.15.132:6837/3612537000] up:boot 2024-03-28T03:47:03.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:47:03 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.119:6834/1638843241,v1:172.21.15.119:6836/1638843241] up:boot 2024-03-28T03:47:03.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:47:03 smithi132 ceph-mon[35941]: mds.? [v2:172.21.15.119:6835/3153136403,v1:172.21.15.119:6837/3153136403] up:boot 2024-03-28T03:47:03.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:47:03 smithi132 ceph-mon[35941]: fsmap 7 up:standby 2024-03-28T03:47:03.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:47:03 smithi132 ceph-mon[35941]: from='client.? 172.21.15.119:0/2488880046' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-28T03:47:03.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:47:03 smithi132 ceph-mon[35941]: pgmap v6269: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T03:47:03.506 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:47:03 smithi119 ceph-mon[31518]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-28T03:47:03.506 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:47:03 smithi119 ceph-mon[31518]: Health check cleared: FS_WITH_FAILED_MDS (was: 1 filesystem has a failed mds daemon) 2024-03-28T03:47:03.507 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:47:03 smithi119 ceph-mon[31518]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-28T03:47:03.507 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:47:03 smithi119 ceph-mon[31518]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-03-28T03:47:03.507 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:47:03 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-28T03:47:03.507 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:47:03 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-28T03:47:03.507 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:47:03 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-28T03:47:03.507 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:47:03 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-28T03:47:03.507 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:47:03 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-28T03:47:03.507 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:47:03 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-28T03:47:03.508 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:47:03 smithi119 ceph-mon[31518]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-28T03:47:03.508 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:47:03 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.165:6832/1722638690,v1:172.21.15.165:6833/1722638690] up:boot 2024-03-28T03:47:03.508 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:47:03 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.132:6834/2349242573,v1:172.21.15.132:6835/2349242573] up:boot 2024-03-28T03:47:03.508 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:47:03 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.132:6832/1407476002,v1:172.21.15.132:6833/1407476002] up:boot 2024-03-28T03:47:03.508 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:47:03 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.132:6838/1704684771,v1:172.21.15.132:6839/1704684771] up:boot 2024-03-28T03:47:03.508 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:47:03 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.132:6836/3612537000,v1:172.21.15.132:6837/3612537000] up:boot 2024-03-28T03:47:03.508 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:47:03 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.119:6834/1638843241,v1:172.21.15.119:6836/1638843241] up:boot 2024-03-28T03:47:03.508 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:47:03 smithi119 ceph-mon[31518]: mds.? [v2:172.21.15.119:6835/3153136403,v1:172.21.15.119:6837/3153136403] up:boot 2024-03-28T03:47:03.508 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:47:03 smithi119 ceph-mon[31518]: fsmap 7 up:standby 2024-03-28T03:47:03.508 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:47:03 smithi119 ceph-mon[31518]: from='client.? 172.21.15.119:0/2488880046' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-28T03:47:03.509 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:47:03 smithi119 ceph-mon[31518]: pgmap v6269: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T03:47:03.530 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:47:03 smithi165 ceph-mon[36302]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-28T03:47:03.530 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:47:03 smithi165 ceph-mon[36302]: Health check cleared: FS_WITH_FAILED_MDS (was: 1 filesystem has a failed mds daemon) 2024-03-28T03:47:03.530 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:47:03 smithi165 ceph-mon[36302]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-28T03:47:03.530 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:47:03 smithi165 ceph-mon[36302]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-03-28T03:47:03.530 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:47:03 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-28T03:47:03.530 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:47:03 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-28T03:47:03.531 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:47:03 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-28T03:47:03.531 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:47:03 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-28T03:47:03.531 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:47:03 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-28T03:47:03.531 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:47:03 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-28T03:47:03.531 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:47:03 smithi165 ceph-mon[36302]: from='mgr.25277 172.21.15.119:0/1940729961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-28T03:47:03.531 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:47:03 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.165:6832/1722638690,v1:172.21.15.165:6833/1722638690] up:boot 2024-03-28T03:47:03.531 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:47:03 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.132:6834/2349242573,v1:172.21.15.132:6835/2349242573] up:boot 2024-03-28T03:47:03.531 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:47:03 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.132:6832/1407476002,v1:172.21.15.132:6833/1407476002] up:boot 2024-03-28T03:47:03.531 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:47:03 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.132:6838/1704684771,v1:172.21.15.132:6839/1704684771] up:boot 2024-03-28T03:47:03.532 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:47:03 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.132:6836/3612537000,v1:172.21.15.132:6837/3612537000] up:boot 2024-03-28T03:47:03.532 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:47:03 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.119:6834/1638843241,v1:172.21.15.119:6836/1638843241] up:boot 2024-03-28T03:47:03.532 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:47:03 smithi165 ceph-mon[36302]: mds.? [v2:172.21.15.119:6835/3153136403,v1:172.21.15.119:6837/3153136403] up:boot 2024-03-28T03:47:03.532 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:47:03 smithi165 ceph-mon[36302]: fsmap 7 up:standby 2024-03-28T03:47:03.532 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:47:03 smithi165 ceph-mon[36302]: from='client.? 172.21.15.119:0/2488880046' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-28T03:47:03.532 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:47:03 smithi165 ceph-mon[36302]: pgmap v6269: 97 pgs: 97 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-28T03:47:03.635 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:03.633+0000 7f5776c00640 1 -- 172.21.15.119:0/679548301 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5770107eb0 msgr2=0x7f5770108290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T03:47:03.635 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:03.633+0000 7f5776c00640 1 --2- 172.21.15.119:0/679548301 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5770107eb0 0x7f5770108290 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f5760009ca0 tx=0x7f576002c100 comp rx=0 tx=0).stop 2024-03-28T03:47:03.635 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:03.633+0000 7f5776c00640 1 -- 172.21.15.119:0/679548301 shutdown_connections 2024-03-28T03:47:03.635 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:03.633+0000 7f5776c00640 1 --2- 172.21.15.119:0/679548301 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f577010cf80 0x7f5770114b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:47:03.635 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:03.633+0000 7f5776c00640 1 --2- 172.21.15.119:0/679548301 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5770108860 0x7f577010c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:47:03.636 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:03.633+0000 7f5776c00640 1 --2- 172.21.15.119:0/679548301 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5770107eb0 0x7f5770108290 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:47:03.636 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:03.633+0000 7f5776c00640 1 -- 172.21.15.119:0/679548301 >> 172.21.15.119:0/679548301 conn(0x7f5770075be0 msgr2=0x7f5770078000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T03:47:03.636 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:03.633+0000 7f5776c00640 1 -- 172.21.15.119:0/679548301 shutdown_connections 2024-03-28T03:47:03.636 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:03.633+0000 7f5776c00640 1 -- 172.21.15.119:0/679548301 wait complete. 2024-03-28T03:47:03.638 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:03.634+0000 7f5776c00640 1 Processor -- start 2024-03-28T03:47:03.638 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:03.634+0000 7f5776c00640 1 -- start start 2024-03-28T03:47:03.638 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:03.634+0000 7f5776c00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5770107eb0 0x7f577019e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T03:47:03.639 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:03.634+0000 7f5776c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f577010cf80 0x7f577019ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T03:47:03.639 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:03.634+0000 7f5776c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f57701a2e50 0x7f57701a3300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T03:47:03.639 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:03.634+0000 7f5776c00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5770118030 con 0x7f577010cf80 2024-03-28T03:47:03.639 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:03.634+0000 7f5776c00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5770117eb0 con 0x7f5770107eb0 2024-03-28T03:47:03.639 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:03.634+0000 7f5776c00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57701181b0 con 0x7f57701a2e50 2024-03-28T03:47:03.639 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:03.635+0000 7f5767400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f577010cf80 0x7f577019ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T03:47:03.639 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:03.635+0000 7f576fe00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5770107eb0 0x7f577019e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T03:47:03.639 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:03.635+0000 7f5767400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f577010cf80 0x7f577019ead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51488/0 (socket says 172.21.15.119:51488) 2024-03-28T03:47:03.639 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:03.635+0000 7f5767400640 1 -- 172.21.15.119:0/3616858225 learned_addr learned my addr 172.21.15.119:0/3616858225 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T03:47:03.639 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:03.635+0000 7f5774c00640 1 --2- 172.21.15.119:0/3616858225 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f57701a2e50 0x7f57701a3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T03:47:03.640 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:03.635+0000 7f5767400640 1 -- 172.21.15.119:0/3616858225 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f57701a2e50 msgr2=0x7f57701a3300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T03:47:03.640 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:03.635+0000 7f5767400640 1 --2- 172.21.15.119:0/3616858225 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f57701a2e50 0x7f57701a3300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:47:03.640 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:03.635+0000 7f5767400640 1 -- 172.21.15.119:0/3616858225 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5770107eb0 msgr2=0x7f577019e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T03:47:03.640 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:03.635+0000 7f5767400640 1 --2- 172.21.15.119:0/3616858225 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5770107eb0 0x7f577019e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:47:03.640 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:03.635+0000 7f5767400640 1 -- 172.21.15.119:0/3616858225 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57701a3940 con 0x7f577010cf80 2024-03-28T03:47:03.640 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:03.635+0000 7f576fe00640 1 --2- 172.21.15.119:0/3616858225 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5770107eb0 0x7f577019e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T03:47:03.640 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:03.635+0000 7f5767400640 1 --2- 172.21.15.119:0/3616858225 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f577010cf80 0x7f577019ead0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f5758005fd0 tx=0x7f575800dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T03:47:03.640 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:03.636+0000 7f576d600640 1 -- 172.21.15.119:0/3616858225 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5758017070 con 0x7f577010cf80 2024-03-28T03:47:03.640 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:03.636+0000 7f576d600640 1 -- 172.21.15.119:0/3616858225 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5758009c30 con 0x7f577010cf80 2024-03-28T03:47:03.641 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:03.636+0000 7f576d600640 1 -- 172.21.15.119:0/3616858225 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5758010460 con 0x7f577010cf80 2024-03-28T03:47:03.641 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:03.636+0000 7f5776c00640 1 -- 172.21.15.119:0/3616858225 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57701a3c30 con 0x7f577010cf80 2024-03-28T03:47:03.641 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:03.636+0000 7f5776c00640 1 -- 172.21.15.119:0/3616858225 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57701ab420 con 0x7f577010cf80 2024-03-28T03:47:03.641 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:03.637+0000 7f5776c00640 1 -- 172.21.15.119:0/3616858225 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57700688e0 con 0x7f577010cf80 2024-03-28T03:47:03.641 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:03.639+0000 7f576d600640 1 -- 172.21.15.119:0/3616858225 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 27) v1 ==== 137422+0+0 (secure 0 0 0) 0x7f5758003e70 con 0x7f577010cf80 2024-03-28T03:47:03.642 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:03.640+0000 7f576d600640 1 --2- 172.21.15.119:0/3616858225 >> [v2:172.21.15.119:6800/4273482239,v1:172.21.15.119:6801/4273482239] conn(0x7f57540a2970 0x7f57540a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T03:47:03.642 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:03.640+0000 7f576d600640 1 -- 172.21.15.119:0/3616858225 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 11097+0+0 (secure 0 0 0) 0x7f57580cf540 con 0x7f577010cf80 2024-03-28T03:47:03.643 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:03.640+0000 7f576fe00640 1 --2- 172.21.15.119:0/3616858225 >> [v2:172.21.15.119:6800/4273482239,v1:172.21.15.119:6801/4273482239] conn(0x7f57540a2970 0x7f57540a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T03:47:03.643 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:03.641+0000 7f576fe00640 1 --2- 172.21.15.119:0/3616858225 >> [v2:172.21.15.119:6800/4273482239,v1:172.21.15.119:6801/4273482239] conn(0x7f57540a2970 0x7f57540a4e30 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f5760000f80 tx=0x7f5760005fd0 comp rx=0 tx=0).ready entity=mgr.25277 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T03:47:03.647 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:03.645+0000 7f576d600640 1 -- 172.21.15.119:0/3616858225 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f5758020030 con 0x7f577010cf80 2024-03-28T03:47:03.844 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:03.842+0000 7f5776c00640 1 -- 172.21.15.119:0/3616858225 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f5770108860 con 0x7f577010cf80 2024-03-28T03:47:04.420 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:04.417+0000 7f576d600640 1 -- 172.21.15.119:0/3616858225 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v128) v1 ==== 169+0+0 (secure 0 0 0) 0x7f575809b890 con 0x7f577010cf80 2024-03-28T03:47:04.420 INFO:teuthology.orchestra.run.smithi119.stderr:pool 'cephfs_data' removed 2024-03-28T03:47:04.428 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:04.426+0000 7f5776c00640 1 -- 172.21.15.119:0/3616858225 >> [v2:172.21.15.119:6800/4273482239,v1:172.21.15.119:6801/4273482239] conn(0x7f57540a2970 msgr2=0x7f57540a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T03:47:04.428 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:04.426+0000 7f5776c00640 1 --2- 172.21.15.119:0/3616858225 >> [v2:172.21.15.119:6800/4273482239,v1:172.21.15.119:6801/4273482239] conn(0x7f57540a2970 0x7f57540a4e30 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f5760000f80 tx=0x7f5760005fd0 comp rx=0 tx=0).stop 2024-03-28T03:47:04.428 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:04.426+0000 7f5776c00640 1 -- 172.21.15.119:0/3616858225 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f577010cf80 msgr2=0x7f577019ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T03:47:04.428 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:04.426+0000 7f5776c00640 1 --2- 172.21.15.119:0/3616858225 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f577010cf80 0x7f577019ead0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f5758005fd0 tx=0x7f575800dde0 comp rx=0 tx=0).stop 2024-03-28T03:47:04.429 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:04.427+0000 7f5776c00640 1 -- 172.21.15.119:0/3616858225 shutdown_connections 2024-03-28T03:47:04.429 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:04.427+0000 7f5776c00640 1 --2- 172.21.15.119:0/3616858225 >> [v2:172.21.15.119:6800/4273482239,v1:172.21.15.119:6801/4273482239] conn(0x7f57540a2970 0x7f57540a4e30 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:47:04.429 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:04.427+0000 7f5776c00640 1 --2- 172.21.15.119:0/3616858225 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f57701a2e50 0x7f57701a3300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:47:04.429 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:04.427+0000 7f5776c00640 1 --2- 172.21.15.119:0/3616858225 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f577010cf80 0x7f577019ead0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:47:04.430 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:04.427+0000 7f5776c00640 1 --2- 172.21.15.119:0/3616858225 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5770107eb0 0x7f577019e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T03:47:04.430 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:04.427+0000 7f5776c00640 1 -- 172.21.15.119:0/3616858225 >> 172.21.15.119:0/3616858225 conn(0x7f5770075be0 msgr2=0x7f5770077010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T03:47:04.430 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:04.428+0000 7f5776c00640 1 -- 172.21.15.119:0/3616858225 shutdown_connections 2024-03-28T03:47:04.430 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T03:47:04.428+0000 7f5776c00640 1 -- 172.21.15.119:0/3616858225 wait complete. 2024-03-28T03:47:04.458 DEBUG:teuthology.orchestra.run.smithi119:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-28T03:47:04.538 DEBUG:teuthology.orchestra.run.smithi132:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-28T03:47:04.571 DEBUG:teuthology.orchestra.run.smithi165:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-28T03:47:04.602 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-03-28T03:47:04.603 DEBUG:teuthology.orchestra.run.smithi119:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-28T03:47:04.658 DEBUG:teuthology.orchestra.run.smithi132:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-28T03:47:04.674 DEBUG:teuthology.orchestra.run.smithi165:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-28T03:47:04.691 INFO:tasks.cephadm:Stopping all daemons... 2024-03-28T03:47:04.691 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-03-28T03:47:04.691 DEBUG:teuthology.orchestra.run.smithi119:> sudo systemctl stop ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mon.a 2024-03-28T03:47:04.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:47:04 smithi132 ceph-mon[35941]: from='client.? 172.21.15.119:0/2488880046' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-03-28T03:47:04.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:47:04 smithi132 ceph-mon[35941]: osdmap e127: 12 total, 12 up, 12 in 2024-03-28T03:47:04.698 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:47:04 smithi132 ceph-mon[35941]: from='client.? 172.21.15.119:0/3616858225' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-28T03:47:04.768 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:47:04 smithi119 ceph-mon[31518]: from='client.? 172.21.15.119:0/2488880046' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-03-28T03:47:04.768 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:47:04 smithi119 ceph-mon[31518]: osdmap e127: 12 total, 12 up, 12 in 2024-03-28T03:47:04.768 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:47:04 smithi119 ceph-mon[31518]: from='client.? 172.21.15.119:0/3616858225' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-28T03:47:04.878 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:47:04 smithi165 ceph-mon[36302]: from='client.? 172.21.15.119:0/2488880046' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-03-28T03:47:04.878 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:47:04 smithi165 ceph-mon[36302]: osdmap e127: 12 total, 12 up, 12 in 2024-03-28T03:47:04.878 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:47:04 smithi165 ceph-mon[36302]: from='client.? 172.21.15.119:0/3616858225' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-28T03:47:05.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:47:04 smithi119 systemd[1]: Stopping Ceph mon.a for 10f73b92-ec98-11ee-95d0-87774f69a715... 2024-03-28T03:47:05.577 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:47:05 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mon-a[31514]: 2024-03-28T03:47:05.234+0000 7f1f6b600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-28T03:47:05.577 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:47:05 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mon-a[31514]: 2024-03-28T03:47:05.235+0000 7f1f6b600700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-03-28T03:47:05.890 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:47:05 smithi119 podman[172725]: 2024-03-28 03:47:05.575587218 +0000 UTC m=+0.492319092 container died 746fea8df4c21d73b15a5bde181da80e3c3ae150443b0a6041213fc1c0286b2c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mon-a, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-03-28T03:47:06.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:47:06 smithi119 podman[172725]: 2024-03-28 03:47:06.08890876 +0000 UTC m=+1.005640640 container cleanup 746fea8df4c21d73b15a5bde181da80e3c3ae150443b0a6041213fc1c0286b2c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mon-a, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240319) 2024-03-28T03:47:06.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:47:06 smithi119 bash[172725]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mon-a 2024-03-28T03:47:06.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Mar 28 03:47:06 smithi119 podman[172738]: 2024-03-28 03:47:06.357043128 +0000 UTC m=+0.765386502 container remove 746fea8df4c21d73b15a5bde181da80e3c3ae150443b0a6041213fc1c0286b2c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mon-a, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5) 2024-03-28T03:47:06.827 DEBUG:teuthology.orchestra.run.smithi119:> sudo pkill -f 'journalctl -f -n 0 -u ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mon.a.service' 2024-03-28T03:47:06.965 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-28T03:47:06.965 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-03-28T03:47:06.966 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-03-28T03:47:06.966 DEBUG:teuthology.orchestra.run.smithi132:> sudo systemctl stop ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mon.b 2024-03-28T03:47:07.294 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:47:07 smithi132 systemd[1]: Stopping Ceph mon.b for 10f73b92-ec98-11ee-95d0-87774f69a715... 2024-03-28T03:47:07.559 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:47:07 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mon-b[35937]: 2024-03-28T03:47:07.292+0000 7fda65574700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-28T03:47:07.559 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:47:07 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mon-b[35937]: 2024-03-28T03:47:07.292+0000 7fda65574700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-03-28T03:47:07.871 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:47:07 smithi132 podman[120245]: 2024-03-28 03:47:07.558415521 +0000 UTC m=+0.386396867 container died 5652a2d5e3de82235ce25daceca0f6cbea41215dc11d90392cf70b6767730265 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mon-b, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-03-28T03:47:08.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:47:08 smithi132 podman[120245]: 2024-03-28 03:47:08.052557742 +0000 UTC m=+0.880539081 container cleanup 5652a2d5e3de82235ce25daceca0f6cbea41215dc11d90392cf70b6767730265 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mon-b, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-03-28T03:47:08.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Mar 28 03:47:08 smithi132 bash[120245]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mon-b 2024-03-28T03:47:08.462 DEBUG:teuthology.orchestra.run.smithi132:> sudo pkill -f 'journalctl -f -n 0 -u ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mon.b.service' 2024-03-28T03:47:08.506 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-28T03:47:08.506 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-03-28T03:47:08.506 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-03-28T03:47:08.507 DEBUG:teuthology.orchestra.run.smithi165:> sudo systemctl stop ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mon.c 2024-03-28T03:47:08.859 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:47:08 smithi165 systemd[1]: Stopping Ceph mon.c for 10f73b92-ec98-11ee-95d0-87774f69a715... 2024-03-28T03:47:09.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:47:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mon-c[36294]: 2024-03-28T03:47:08.857+0000 7f4a696d9700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-28T03:47:09.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:47:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mon-c[36294]: 2024-03-28T03:47:08.857+0000 7f4a696d9700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-03-28T03:47:09.128 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:47:09 smithi165 podman[123435]: 2024-03-28 03:47:09.06252167 +0000 UTC m=+0.363888026 container died e52d038dbab0e33ddba2ad62b67119b0f9fac7e01509e2ba5dd786d2c122f555 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mon-c, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-03-28T03:47:09.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:47:09 smithi165 podman[123435]: 2024-03-28 03:47:09.550071621 +0000 UTC m=+0.851437970 container cleanup e52d038dbab0e33ddba2ad62b67119b0f9fac7e01509e2ba5dd786d2c122f555 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mon-c, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-03-28T03:47:09.628 INFO:journalctl@ceph.mon.c.smithi165.stdout:Mar 28 03:47:09 smithi165 bash[123435]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mon-c 2024-03-28T03:47:09.898 DEBUG:teuthology.orchestra.run.smithi165:> sudo pkill -f 'journalctl -f -n 0 -u ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mon.c.service' 2024-03-28T03:47:09.950 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-28T03:47:09.950 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-03-28T03:47:09.951 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-03-28T03:47:09.951 DEBUG:teuthology.orchestra.run.smithi119:> sudo systemctl stop ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mgr.x 2024-03-28T03:47:10.401 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 03:47:10 smithi119 systemd[1]: Stopping Ceph mgr.x for 10f73b92-ec98-11ee-95d0-87774f69a715... 2024-03-28T03:47:11.152 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 03:47:10 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-x[31752]: 2024-03-28T03:47:10.724+0000 7fac91e00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-28T03:47:11.153 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 03:47:10 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-x[31752]: 2024-03-28T03:47:10.724+0000 7fac91e00700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-03-28T03:47:11.153 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 03:47:10 smithi119 podman[172857]: 2024-03-28 03:47:10.828130393 +0000 UTC m=+0.442684364 container died f628950cf3c0735f866da143ab033c0cc9fb1e088fc1b45918c638f1a6fb596b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-x, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.license=GPLv2) 2024-03-28T03:47:11.597 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 03:47:11 smithi119 podman[172857]: 2024-03-28 03:47:11.285766884 +0000 UTC m=+0.900320911 container cleanup f628950cf3c0735f866da143ab033c0cc9fb1e088fc1b45918c638f1a6fb596b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-x, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-03-28T03:47:11.598 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 03:47:11 smithi119 bash[172857]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-x 2024-03-28T03:47:11.901 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Mar 28 03:47:11 smithi119 podman[172871]: 2024-03-28 03:47:11.595879909 +0000 UTC m=+0.755377860 container remove f628950cf3c0735f866da143ab033c0cc9fb1e088fc1b45918c638f1a6fb596b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-x, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True) 2024-03-28T03:47:12.101 DEBUG:teuthology.orchestra.run.smithi119:> sudo pkill -f 'journalctl -f -n 0 -u ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mgr.x.service' 2024-03-28T03:47:12.223 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-28T03:47:12.223 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-03-28T03:47:12.223 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-03-28T03:47:12.223 DEBUG:teuthology.orchestra.run.smithi132:> sudo systemctl stop ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mgr.y 2024-03-28T03:47:12.671 INFO:journalctl@ceph.mgr.y.smithi132.stdout:Mar 28 03:47:12 smithi132 systemd[1]: Stopping Ceph mgr.y for 10f73b92-ec98-11ee-95d0-87774f69a715... 2024-03-28T03:47:12.948 INFO:journalctl@ceph.mgr.y.smithi132.stdout:Mar 28 03:47:12 smithi132 podman[120381]: 2024-03-28 03:47:12.669744862 +0000 UTC m=+0.236065717 container died eba27e1c6ada57832c496f34a4822eec9bc5a05d2281dc77b74f4e61c8519f02 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-y, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True) 2024-03-28T03:47:13.336 INFO:journalctl@ceph.mgr.y.smithi132.stdout:Mar 28 03:47:13 smithi132 podman[120381]: 2024-03-28 03:47:13.0749653 +0000 UTC m=+0.641286143 container cleanup eba27e1c6ada57832c496f34a4822eec9bc5a05d2281dc77b74f4e61c8519f02 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-y, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-03-28T03:47:13.336 INFO:journalctl@ceph.mgr.y.smithi132.stdout:Mar 28 03:47:13 smithi132 bash[120381]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-y 2024-03-28T03:47:13.496 DEBUG:teuthology.orchestra.run.smithi132:> sudo pkill -f 'journalctl -f -n 0 -u ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mgr.y.service' 2024-03-28T03:47:13.536 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-28T03:47:13.536 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-03-28T03:47:13.536 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-03-28T03:47:13.536 DEBUG:teuthology.orchestra.run.smithi165:> sudo systemctl stop ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mgr.z 2024-03-28T03:47:13.878 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Mar 28 03:47:13 smithi165 systemd[1]: Stopping Ceph mgr.z for 10f73b92-ec98-11ee-95d0-87774f69a715... 2024-03-28T03:47:14.307 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Mar 28 03:47:13 smithi165 podman[123557]: 2024-03-28 03:47:13.91704252 +0000 UTC m=+0.188278958 container died 3ec565f6ab2b0c98e8ad23d6a00d8cf4111920693418a4ae1410c76760835df4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-z, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-03-28T03:47:14.628 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Mar 28 03:47:14 smithi165 podman[123557]: 2024-03-28 03:47:14.413007824 +0000 UTC m=+0.684244238 container cleanup 3ec565f6ab2b0c98e8ad23d6a00d8cf4111920693418a4ae1410c76760835df4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-z, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-03-28T03:47:14.628 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Mar 28 03:47:14 smithi165 bash[123557]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mgr-z 2024-03-28T03:47:14.825 DEBUG:teuthology.orchestra.run.smithi165:> sudo pkill -f 'journalctl -f -n 0 -u ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mgr.z.service' 2024-03-28T03:47:14.861 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-28T03:47:14.861 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-03-28T03:47:14.861 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-03-28T03:47:14.861 DEBUG:teuthology.orchestra.run.smithi119:> sudo systemctl stop ceph-10f73b92-ec98-11ee-95d0-87774f69a715@osd.0 2024-03-28T03:47:15.402 INFO:journalctl@ceph.osd.0.smithi119.stdout:Mar 28 03:47:14 smithi119 systemd[1]: Stopping Ceph osd.0 for 10f73b92-ec98-11ee-95d0-87774f69a715... 2024-03-28T03:47:15.901 INFO:journalctl@ceph.osd.0.smithi119.stdout:Mar 28 03:47:15 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-0[43761]: 2024-03-28T03:47:15.429+0000 7f65abe00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-28T03:47:15.901 INFO:journalctl@ceph.osd.0.smithi119.stdout:Mar 28 03:47:15 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-0[43761]: 2024-03-28T03:47:15.429+0000 7f65abe00700 -1 osd.0 128 *** Got signal Terminated *** 2024-03-28T03:47:15.901 INFO:journalctl@ceph.osd.0.smithi119.stdout:Mar 28 03:47:15 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-0[43761]: 2024-03-28T03:47:15.429+0000 7f65abe00700 -1 osd.0 128 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-28T03:47:21.151 INFO:journalctl@ceph.osd.0.smithi119.stdout:Mar 28 03:47:20 smithi119 podman[172977]: 2024-03-28 03:47:20.855488562 +0000 UTC m=+5.584069128 container died 23820adadea11bd04f5e6e7d5e8f9acdd141fad1b8705b74e9484b9bee88ea05 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-03-28T03:47:21.651 INFO:journalctl@ceph.osd.0.smithi119.stdout:Mar 28 03:47:21 smithi119 podman[172977]: 2024-03-28 03:47:21.407112942 +0000 UTC m=+6.135693485 container cleanup 23820adadea11bd04f5e6e7d5e8f9acdd141fad1b8705b74e9484b9bee88ea05 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-03-28T03:47:21.651 INFO:journalctl@ceph.osd.0.smithi119.stdout:Mar 28 03:47:21 smithi119 bash[172977]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-0 2024-03-28T03:47:22.151 INFO:journalctl@ceph.osd.0.smithi119.stdout:Mar 28 03:47:21 smithi119 podman[172998]: 2024-03-28 03:47:21.65855793 +0000 UTC m=+0.793370242 container remove 23820adadea11bd04f5e6e7d5e8f9acdd141fad1b8705b74e9484b9bee88ea05 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-0, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_CLEAN=True, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-03-28T03:47:22.783 INFO:journalctl@ceph.osd.0.smithi119.stdout:Mar 28 03:47:22 smithi119 podman[173091]: 2024-03-28 03:47:22.371687931 +0000 UTC m=+0.035059814 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-28T03:47:22.783 INFO:journalctl@ceph.osd.0.smithi119.stdout:Mar 28 03:47:22 smithi119 podman[173091]: 2024-03-28 03:47:22.484709605 +0000 UTC m=+0.148081518 container create 412cab2487e1173d775c17a21df382c8bbe42b03ab917ccc3d77cc17701dc5da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-0-deactivate, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c) 2024-03-28T03:47:22.783 INFO:journalctl@ceph.osd.0.smithi119.stdout:Mar 28 03:47:22 smithi119 podman[173091]: 2024-03-28 03:47:22.780108761 +0000 UTC m=+0.443480678 container init 412cab2487e1173d775c17a21df382c8bbe42b03ab917ccc3d77cc17701dc5da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-0-deactivate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-03-28T03:47:23.151 INFO:journalctl@ceph.osd.0.smithi119.stdout:Mar 28 03:47:22 smithi119 podman[173091]: 2024-03-28 03:47:22.792291757 +0000 UTC m=+0.455663647 container start 412cab2487e1173d775c17a21df382c8bbe42b03ab917ccc3d77cc17701dc5da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_CLEAN=True) 2024-03-28T03:47:23.151 INFO:journalctl@ceph.osd.0.smithi119.stdout:Mar 28 03:47:22 smithi119 podman[173091]: 2024-03-28 03:47:22.895692717 +0000 UTC m=+0.559064680 container attach 412cab2487e1173d775c17a21df382c8bbe42b03ab917ccc3d77cc17701dc5da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-0-deactivate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, ceph=True) 2024-03-28T03:47:23.589 INFO:journalctl@ceph.osd.0.smithi119.stdout:Mar 28 03:47:23 smithi119 podman[173091]: 2024-03-28 03:47:23.228246107 +0000 UTC m=+0.891618053 container died 412cab2487e1173d775c17a21df382c8bbe42b03ab917ccc3d77cc17701dc5da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-0-deactivate, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c) 2024-03-28T03:47:24.008 DEBUG:teuthology.orchestra.run.smithi119:> sudo pkill -f 'journalctl -f -n 0 -u ceph-10f73b92-ec98-11ee-95d0-87774f69a715@osd.0.service' 2024-03-28T03:47:24.130 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-28T03:47:24.130 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-03-28T03:47:24.130 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-03-28T03:47:24.130 DEBUG:teuthology.orchestra.run.smithi132:> sudo systemctl stop ceph-10f73b92-ec98-11ee-95d0-87774f69a715@osd.1 2024-03-28T03:47:24.439 INFO:journalctl@ceph.osd.1.smithi132.stdout:Mar 28 03:47:24 smithi132 systemd[1]: Stopping Ceph osd.1 for 10f73b92-ec98-11ee-95d0-87774f69a715... 2024-03-28T03:47:24.698 INFO:journalctl@ceph.osd.1.smithi132.stdout:Mar 28 03:47:24 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-1[40548]: 2024-03-28T03:47:24.437+0000 7f0b3a9a0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-28T03:47:24.699 INFO:journalctl@ceph.osd.1.smithi132.stdout:Mar 28 03:47:24 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-1[40548]: 2024-03-28T03:47:24.437+0000 7f0b3a9a0700 -1 osd.1 128 *** Got signal Terminated *** 2024-03-28T03:47:24.699 INFO:journalctl@ceph.osd.1.smithi132.stdout:Mar 28 03:47:24 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-1[40548]: 2024-03-28T03:47:24.437+0000 7f0b3a9a0700 -1 osd.1 128 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-28T03:47:30.049 INFO:journalctl@ceph.osd.1.smithi132.stdout:Mar 28 03:47:29 smithi132 podman[120508]: 2024-03-28 03:47:29.775709393 +0000 UTC m=+5.452631869 container died b006dc96dd4c3b7dc23e818bbc94151f40e3985e1b5eec8248bfc2ac40025916 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-1, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, CEPH_POINT_RELEASE=) 2024-03-28T03:47:30.432 INFO:journalctl@ceph.osd.1.smithi132.stdout:Mar 28 03:47:30 smithi132 podman[120508]: 2024-03-28 03:47:30.138201925 +0000 UTC m=+5.815124400 container cleanup b006dc96dd4c3b7dc23e818bbc94151f40e3985e1b5eec8248bfc2ac40025916 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-1, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-03-28T03:47:30.432 INFO:journalctl@ceph.osd.1.smithi132.stdout:Mar 28 03:47:30 smithi132 bash[120508]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-1 2024-03-28T03:47:30.698 INFO:journalctl@ceph.osd.1.smithi132.stdout:Mar 28 03:47:30 smithi132 podman[120525]: 2024-03-28 03:47:30.431545001 +0000 UTC m=+0.656394161 container remove b006dc96dd4c3b7dc23e818bbc94151f40e3985e1b5eec8248bfc2ac40025916 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-1, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-28T03:47:31.054 INFO:journalctl@ceph.osd.1.smithi132.stdout:Mar 28 03:47:30 smithi132 podman[120615]: 2024-03-28 03:47:30.656039467 +0000 UTC m=+0.017548055 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-28T03:47:31.054 INFO:journalctl@ceph.osd.1.smithi132.stdout:Mar 28 03:47:30 smithi132 podman[120615]: 2024-03-28 03:47:30.791179254 +0000 UTC m=+0.152687880 container create c66c2274060b58ad8c95fc37d5ca99919cb16e6a6fe1c311e2110b1a84f2e9ca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-1-deactivate, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-03-28T03:47:31.054 INFO:journalctl@ceph.osd.1.smithi132.stdout:Mar 28 03:47:31 smithi132 podman[120615]: 2024-03-28 03:47:31.05266988 +0000 UTC m=+0.414178484 container init c66c2274060b58ad8c95fc37d5ca99919cb16e6a6fe1c311e2110b1a84f2e9ca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-1-deactivate, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319) 2024-03-28T03:47:31.324 INFO:journalctl@ceph.osd.1.smithi132.stdout:Mar 28 03:47:31 smithi132 podman[120615]: 2024-03-28 03:47:31.056362097 +0000 UTC m=+0.417870684 container start c66c2274060b58ad8c95fc37d5ca99919cb16e6a6fe1c311e2110b1a84f2e9ca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-1-deactivate, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-03-28T03:47:31.325 INFO:journalctl@ceph.osd.1.smithi132.stdout:Mar 28 03:47:31 smithi132 podman[120615]: 2024-03-28 03:47:31.144063559 +0000 UTC m=+0.505572164 container attach c66c2274060b58ad8c95fc37d5ca99919cb16e6a6fe1c311e2110b1a84f2e9ca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-1-deactivate, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-28T03:47:31.698 INFO:journalctl@ceph.osd.1.smithi132.stdout:Mar 28 03:47:31 smithi132 podman[120615]: 2024-03-28 03:47:31.354616189 +0000 UTC m=+0.716124781 container died c66c2274060b58ad8c95fc37d5ca99919cb16e6a6fe1c311e2110b1a84f2e9ca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-1-deactivate, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-03-28T03:47:32.010 DEBUG:teuthology.orchestra.run.smithi132:> sudo pkill -f 'journalctl -f -n 0 -u ceph-10f73b92-ec98-11ee-95d0-87774f69a715@osd.1.service' 2024-03-28T03:47:32.050 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-28T03:47:32.050 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-03-28T03:47:32.051 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-03-28T03:47:32.051 DEBUG:teuthology.orchestra.run.smithi165:> sudo systemctl stop ceph-10f73b92-ec98-11ee-95d0-87774f69a715@osd.2 2024-03-28T03:47:32.378 INFO:journalctl@ceph.osd.2.smithi165.stdout:Mar 28 03:47:32 smithi165 systemd[1]: Stopping Ceph osd.2 for 10f73b92-ec98-11ee-95d0-87774f69a715... 2024-03-28T03:47:32.878 INFO:journalctl@ceph.osd.2.smithi165.stdout:Mar 28 03:47:32 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-2[41460]: 2024-03-28T03:47:32.531+0000 7fe3645a9700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-28T03:47:32.878 INFO:journalctl@ceph.osd.2.smithi165.stdout:Mar 28 03:47:32 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-2[41460]: 2024-03-28T03:47:32.531+0000 7fe3645a9700 -1 osd.2 128 *** Got signal Terminated *** 2024-03-28T03:47:32.879 INFO:journalctl@ceph.osd.2.smithi165.stdout:Mar 28 03:47:32 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-2[41460]: 2024-03-28T03:47:32.531+0000 7fe3645a9700 -1 osd.2 128 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-28T03:47:38.234 INFO:journalctl@ceph.osd.2.smithi165.stdout:Mar 28 03:47:37 smithi165 podman[123680]: 2024-03-28 03:47:37.917767616 +0000 UTC m=+5.676302174 container died e706fd3899de8c916f8e5615f158a5e18179b8f76b8bc8049844989b82d0de0a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-2, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c) 2024-03-28T03:47:38.568 INFO:journalctl@ceph.osd.2.smithi165.stdout:Mar 28 03:47:38 smithi165 podman[123680]: 2024-03-28 03:47:38.332483404 +0000 UTC m=+6.091017958 container cleanup e706fd3899de8c916f8e5615f158a5e18179b8f76b8bc8049844989b82d0de0a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-28T03:47:38.568 INFO:journalctl@ceph.osd.2.smithi165.stdout:Mar 28 03:47:38 smithi165 bash[123680]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-2 2024-03-28T03:47:38.878 INFO:journalctl@ceph.osd.2.smithi165.stdout:Mar 28 03:47:38 smithi165 podman[123697]: 2024-03-28 03:47:38.567211804 +0000 UTC m=+0.646124093 container remove e706fd3899de8c916f8e5615f158a5e18179b8f76b8bc8049844989b82d0de0a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_CLEAN=True) 2024-03-28T03:47:39.339 INFO:journalctl@ceph.osd.2.smithi165.stdout:Mar 28 03:47:38 smithi165 podman[123789]: 2024-03-28 03:47:38.897905159 +0000 UTC m=+0.017672687 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-28T03:47:39.339 INFO:journalctl@ceph.osd.2.smithi165.stdout:Mar 28 03:47:39 smithi165 podman[123789]: 2024-03-28 03:47:39.010029682 +0000 UTC m=+0.129797224 container create 5028f5b7f752b655c4054b193e32b2f78370b5b643d97c9bfe7d8eaee4e58369 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-2-deactivate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-03-28T03:47:39.339 INFO:journalctl@ceph.osd.2.smithi165.stdout:Mar 28 03:47:39 smithi165 podman[123789]: 2024-03-28 03:47:39.221007893 +0000 UTC m=+0.340775425 container init 5028f5b7f752b655c4054b193e32b2f78370b5b643d97c9bfe7d8eaee4e58369 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-2-deactivate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2) 2024-03-28T03:47:39.339 INFO:journalctl@ceph.osd.2.smithi165.stdout:Mar 28 03:47:39 smithi165 podman[123789]: 2024-03-28 03:47:39.224237058 +0000 UTC m=+0.344004584 container start 5028f5b7f752b655c4054b193e32b2f78370b5b643d97c9bfe7d8eaee4e58369 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-2-deactivate, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240319) 2024-03-28T03:47:39.339 INFO:journalctl@ceph.osd.2.smithi165.stdout:Mar 28 03:47:39 smithi165 podman[123789]: 2024-03-28 03:47:39.337816489 +0000 UTC m=+0.457584025 container attach 5028f5b7f752b655c4054b193e32b2f78370b5b643d97c9bfe7d8eaee4e58369 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-2-deactivate, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-28T03:47:39.628 INFO:journalctl@ceph.osd.2.smithi165.stdout:Mar 28 03:47:39 smithi165 podman[123789]: 2024-03-28 03:47:39.509174496 +0000 UTC m=+0.628942018 container died 5028f5b7f752b655c4054b193e32b2f78370b5b643d97c9bfe7d8eaee4e58369 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-2-deactivate, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2) 2024-03-28T03:47:40.143 DEBUG:teuthology.orchestra.run.smithi165:> sudo pkill -f 'journalctl -f -n 0 -u ceph-10f73b92-ec98-11ee-95d0-87774f69a715@osd.2.service' 2024-03-28T03:47:40.186 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-28T03:47:40.187 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-03-28T03:47:40.187 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-03-28T03:47:40.187 DEBUG:teuthology.orchestra.run.smithi119:> sudo systemctl stop ceph-10f73b92-ec98-11ee-95d0-87774f69a715@osd.3 2024-03-28T03:47:40.651 INFO:journalctl@ceph.osd.3.smithi119.stdout:Mar 28 03:47:40 smithi119 systemd[1]: Stopping Ceph osd.3 for 10f73b92-ec98-11ee-95d0-87774f69a715... 2024-03-28T03:47:41.151 INFO:journalctl@ceph.osd.3.smithi119.stdout:Mar 28 03:47:40 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-3[50436]: 2024-03-28T03:47:40.742+0000 7fc06aa00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-28T03:47:41.151 INFO:journalctl@ceph.osd.3.smithi119.stdout:Mar 28 03:47:40 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-3[50436]: 2024-03-28T03:47:40.742+0000 7fc06aa00700 -1 osd.3 128 *** Got signal Terminated *** 2024-03-28T03:47:41.152 INFO:journalctl@ceph.osd.3.smithi119.stdout:Mar 28 03:47:40 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-3[50436]: 2024-03-28T03:47:40.742+0000 7fc06aa00700 -1 osd.3 128 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-28T03:47:43.651 INFO:journalctl@ceph.osd.3.smithi119.stdout:Mar 28 03:47:43 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-3[50436]: 2024-03-28T03:47:43.292+0000 7fc066200700 -1 osd.3 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:17.698775+0000 front 2024-03-28T03:47:17.698711+0000 (oldest deadline 2024-03-28T03:47:42.398168+0000) 2024-03-28T03:47:43.699 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:47:43 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:47:43.361+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:18.649888+0000 front 2024-03-28T03:47:18.649830+0000 (oldest deadline 2024-03-28T03:47:42.749644+0000) 2024-03-28T03:47:44.651 INFO:journalctl@ceph.osd.3.smithi119.stdout:Mar 28 03:47:44 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-3[50436]: 2024-03-28T03:47:44.339+0000 7fc066200700 -1 osd.3 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:17.698775+0000 front 2024-03-28T03:47:17.698711+0000 (oldest deadline 2024-03-28T03:47:42.398168+0000) 2024-03-28T03:47:44.652 INFO:journalctl@ceph.osd.6.smithi119.stdout:Mar 28 03:47:44 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-6[57000]: 2024-03-28T03:47:44.527+0000 7f926a600700 -1 osd.6 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:18.788278+0000 front 2024-03-28T03:47:18.788347+0000 (oldest deadline 2024-03-28T03:47:44.088340+0000) 2024-03-28T03:47:44.698 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:47:44 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:47:44.391+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:18.649888+0000 front 2024-03-28T03:47:18.649830+0000 (oldest deadline 2024-03-28T03:47:42.749644+0000) 2024-03-28T03:47:45.540 INFO:journalctl@ceph.osd.3.smithi119.stdout:Mar 28 03:47:45 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-3[50436]: 2024-03-28T03:47:45.351+0000 7fc066200700 -1 osd.3 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:17.698775+0000 front 2024-03-28T03:47:17.698711+0000 (oldest deadline 2024-03-28T03:47:42.398168+0000) 2024-03-28T03:47:45.541 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:47:45 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:47:45.155+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:19.734645+0000 front 2024-03-28T03:47:19.734461+0000 (oldest deadline 2024-03-28T03:47:44.434325+0000) 2024-03-28T03:47:45.628 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:47:45 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:47:45.174+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:18.314545+0000 front 2024-03-28T03:47:18.314582+0000 (oldest deadline 2024-03-28T03:47:44.214430+0000) 2024-03-28T03:47:45.698 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:47:45 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:47:45.416+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:18.649888+0000 front 2024-03-28T03:47:18.649830+0000 (oldest deadline 2024-03-28T03:47:42.749644+0000) 2024-03-28T03:47:45.901 INFO:journalctl@ceph.osd.6.smithi119.stdout:Mar 28 03:47:45 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-6[57000]: 2024-03-28T03:47:45.538+0000 7f926a600700 -1 osd.6 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:18.788278+0000 front 2024-03-28T03:47:18.788347+0000 (oldest deadline 2024-03-28T03:47:44.088340+0000) 2024-03-28T03:47:46.535 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:47:46 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:47:46.187+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:19.734645+0000 front 2024-03-28T03:47:19.734461+0000 (oldest deadline 2024-03-28T03:47:44.434325+0000) 2024-03-28T03:47:46.535 INFO:journalctl@ceph.osd.3.smithi119.stdout:Mar 28 03:47:46 smithi119 podman[173195]: 2024-03-28 03:47:46.21667801 +0000 UTC m=+5.610879962 container died 9bcbcf8e146d56d2c0a79ae8aeca0b6dae123d7e5033e825c636ac2ebe7a4afa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-3, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-28T03:47:46.628 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:47:46 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:47:46.162+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:18.314545+0000 front 2024-03-28T03:47:18.314582+0000 (oldest deadline 2024-03-28T03:47:44.214430+0000) 2024-03-28T03:47:46.698 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:47:46 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:47:46.399+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:18.649888+0000 front 2024-03-28T03:47:18.649830+0000 (oldest deadline 2024-03-28T03:47:42.749644+0000) 2024-03-28T03:47:46.901 INFO:journalctl@ceph.osd.6.smithi119.stdout:Mar 28 03:47:46 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-6[57000]: 2024-03-28T03:47:46.575+0000 7f926a600700 -1 osd.6 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:18.788278+0000 front 2024-03-28T03:47:18.788347+0000 (oldest deadline 2024-03-28T03:47:44.088340+0000) 2024-03-28T03:47:46.902 INFO:journalctl@ceph.osd.3.smithi119.stdout:Mar 28 03:47:46 smithi119 podman[173195]: 2024-03-28 03:47:46.669855516 +0000 UTC m=+6.064057529 container cleanup 9bcbcf8e146d56d2c0a79ae8aeca0b6dae123d7e5033e825c636ac2ebe7a4afa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-3, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-03-28T03:47:46.902 INFO:journalctl@ceph.osd.3.smithi119.stdout:Mar 28 03:47:46 smithi119 bash[173195]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-3 2024-03-28T03:47:47.174 INFO:journalctl@ceph.osd.3.smithi119.stdout:Mar 28 03:47:46 smithi119 podman[173222]: 2024-03-28 03:47:46.937836014 +0000 UTC m=+0.706745823 container remove 9bcbcf8e146d56d2c0a79ae8aeca0b6dae123d7e5033e825c636ac2ebe7a4afa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-3, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-03-28T03:47:47.536 INFO:journalctl@ceph.osd.8.smithi165.stdout:Mar 28 03:47:47 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-8[55184]: 2024-03-28T03:47:47.321+0000 7fd543c54700 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:19.562620+0000 front 2024-03-28T03:47:19.563027+0000 (oldest deadline 2024-03-28T03:47:46.562998+0000) 2024-03-28T03:47:47.537 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:47:47 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:47:47.133+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:18.314545+0000 front 2024-03-28T03:47:18.314582+0000 (oldest deadline 2024-03-28T03:47:44.214430+0000) 2024-03-28T03:47:47.583 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:47:47 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:47:47.217+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:19.734645+0000 front 2024-03-28T03:47:19.734461+0000 (oldest deadline 2024-03-28T03:47:44.434325+0000) 2024-03-28T03:47:47.698 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:47:47 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:47:47.406+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:18.649888+0000 front 2024-03-28T03:47:18.649830+0000 (oldest deadline 2024-03-28T03:47:42.749644+0000) 2024-03-28T03:47:47.870 INFO:journalctl@ceph.osd.6.smithi119.stdout:Mar 28 03:47:47 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-6[57000]: 2024-03-28T03:47:47.581+0000 7f926a600700 -1 osd.6 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:18.788278+0000 front 2024-03-28T03:47:18.788347+0000 (oldest deadline 2024-03-28T03:47:44.088340+0000) 2024-03-28T03:47:47.870 INFO:journalctl@ceph.osd.3.smithi119.stdout:Mar 28 03:47:47 smithi119 podman[173311]: 2024-03-28 03:47:47.546942395 +0000 UTC m=+0.031644674 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-28T03:47:47.870 INFO:journalctl@ceph.osd.3.smithi119.stdout:Mar 28 03:47:47 smithi119 podman[173311]: 2024-03-28 03:47:47.697547518 +0000 UTC m=+0.182249835 container create 79fc97690126a8742146bb59de4daa3809a80cc896c4f5f4009005454f2afb3e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-03-28T03:47:47.878 INFO:journalctl@ceph.osd.5.smithi165.stdout:Mar 28 03:47:47 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-5[48453]: 2024-03-28T03:47:47.535+0000 7f5e2fe37700 -1 osd.5 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:15.457115+0000 front 2024-03-28T03:47:15.457379+0000 (oldest deadline 2024-03-28T03:47:46.657477+0000) 2024-03-28T03:47:48.151 INFO:journalctl@ceph.osd.3.smithi119.stdout:Mar 28 03:47:47 smithi119 podman[173311]: 2024-03-28 03:47:47.976407488 +0000 UTC m=+0.461109811 container init 79fc97690126a8742146bb59de4daa3809a80cc896c4f5f4009005454f2afb3e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-3-deactivate, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-03-28T03:47:48.152 INFO:journalctl@ceph.osd.3.smithi119.stdout:Mar 28 03:47:47 smithi119 podman[173311]: 2024-03-28 03:47:47.98783373 +0000 UTC m=+0.472536070 container start 79fc97690126a8742146bb59de4daa3809a80cc896c4f5f4009005454f2afb3e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, ceph=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-03-28T03:47:48.152 INFO:journalctl@ceph.osd.3.smithi119.stdout:Mar 28 03:47:48 smithi119 podman[173311]: 2024-03-28 03:47:48.083314085 +0000 UTC m=+0.568016406 container attach 79fc97690126a8742146bb59de4daa3809a80cc896c4f5f4009005454f2afb3e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-3-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_CLEAN=True, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True) 2024-03-28T03:47:48.378 INFO:journalctl@ceph.osd.8.smithi165.stdout:Mar 28 03:47:48 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-8[55184]: 2024-03-28T03:47:48.361+0000 7fd543c54700 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:19.562620+0000 front 2024-03-28T03:47:19.563027+0000 (oldest deadline 2024-03-28T03:47:46.562998+0000) 2024-03-28T03:47:48.379 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:47:48 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:47:48.113+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:18.314545+0000 front 2024-03-28T03:47:18.314582+0000 (oldest deadline 2024-03-28T03:47:44.214430+0000) 2024-03-28T03:47:48.631 INFO:journalctl@ceph.osd.3.smithi119.stdout:Mar 28 03:47:48 smithi119 podman[173311]: 2024-03-28 03:47:48.405304529 +0000 UTC m=+0.890006856 container died 79fc97690126a8742146bb59de4daa3809a80cc896c4f5f4009005454f2afb3e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-3-deactivate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-03-28T03:47:48.631 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:47:48 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:47:48.180+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:19.734645+0000 front 2024-03-28T03:47:19.734461+0000 (oldest deadline 2024-03-28T03:47:44.434325+0000) 2024-03-28T03:47:48.698 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:47:48 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:47:48.411+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:18.649888+0000 front 2024-03-28T03:47:18.649830+0000 (oldest deadline 2024-03-28T03:47:42.749644+0000) 2024-03-28T03:47:48.878 INFO:journalctl@ceph.osd.5.smithi165.stdout:Mar 28 03:47:48 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-5[48453]: 2024-03-28T03:47:48.522+0000 7f5e2fe37700 -1 osd.5 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:15.457115+0000 front 2024-03-28T03:47:15.457379+0000 (oldest deadline 2024-03-28T03:47:46.657477+0000) 2024-03-28T03:47:48.901 INFO:journalctl@ceph.osd.6.smithi119.stdout:Mar 28 03:47:48 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-6[57000]: 2024-03-28T03:47:48.628+0000 7f926a600700 -1 osd.6 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:18.788278+0000 front 2024-03-28T03:47:18.788347+0000 (oldest deadline 2024-03-28T03:47:44.088340+0000) 2024-03-28T03:47:49.151 DEBUG:teuthology.orchestra.run.smithi119:> sudo pkill -f 'journalctl -f -n 0 -u ceph-10f73b92-ec98-11ee-95d0-87774f69a715@osd.3.service' 2024-03-28T03:47:49.271 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-28T03:47:49.271 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-03-28T03:47:49.271 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-03-28T03:47:49.271 DEBUG:teuthology.orchestra.run.smithi132:> sudo systemctl stop ceph-10f73b92-ec98-11ee-95d0-87774f69a715@osd.4 2024-03-28T03:47:49.378 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:47:49 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:47:49.103+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:18.314545+0000 front 2024-03-28T03:47:18.314582+0000 (oldest deadline 2024-03-28T03:47:44.214430+0000) 2024-03-28T03:47:49.401 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:47:49 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:47:49.173+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:19.734645+0000 front 2024-03-28T03:47:19.734461+0000 (oldest deadline 2024-03-28T03:47:44.434325+0000) 2024-03-28T03:47:49.669 INFO:journalctl@ceph.osd.4.smithi132.stdout:Mar 28 03:47:49 smithi132 systemd[1]: Stopping Ceph osd.4 for 10f73b92-ec98-11ee-95d0-87774f69a715... 2024-03-28T03:47:49.669 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:47:49 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:47:49.409+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:18.649888+0000 front 2024-03-28T03:47:18.649830+0000 (oldest deadline 2024-03-28T03:47:42.749644+0000) 2024-03-28T03:47:49.878 INFO:journalctl@ceph.osd.5.smithi165.stdout:Mar 28 03:47:49 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-5[48453]: 2024-03-28T03:47:49.566+0000 7f5e2fe37700 -1 osd.5 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:15.457115+0000 front 2024-03-28T03:47:15.457379+0000 (oldest deadline 2024-03-28T03:47:46.657477+0000) 2024-03-28T03:47:49.878 INFO:journalctl@ceph.osd.8.smithi165.stdout:Mar 28 03:47:49 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-8[55184]: 2024-03-28T03:47:49.385+0000 7fd543c54700 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:19.562620+0000 front 2024-03-28T03:47:19.563027+0000 (oldest deadline 2024-03-28T03:47:46.562998+0000) 2024-03-28T03:47:49.901 INFO:journalctl@ceph.osd.6.smithi119.stdout:Mar 28 03:47:49 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-6[57000]: 2024-03-28T03:47:49.635+0000 7f926a600700 -1 osd.6 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:18.788278+0000 front 2024-03-28T03:47:18.788347+0000 (oldest deadline 2024-03-28T03:47:44.088340+0000) 2024-03-28T03:47:49.948 INFO:journalctl@ceph.osd.4.smithi132.stdout:Mar 28 03:47:49 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-4[47490]: 2024-03-28T03:47:49.666+0000 7f96430b3700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-28T03:47:49.948 INFO:journalctl@ceph.osd.4.smithi132.stdout:Mar 28 03:47:49 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-4[47490]: 2024-03-28T03:47:49.666+0000 7f96430b3700 -1 osd.4 128 *** Got signal Terminated *** 2024-03-28T03:47:49.949 INFO:journalctl@ceph.osd.4.smithi132.stdout:Mar 28 03:47:49 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-4[47490]: 2024-03-28T03:47:49.666+0000 7f96430b3700 -1 osd.4 128 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-28T03:47:50.378 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:47:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:47:50.103+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:18.314545+0000 front 2024-03-28T03:47:18.314582+0000 (oldest deadline 2024-03-28T03:47:44.214430+0000) 2024-03-28T03:47:50.401 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:47:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:47:50.142+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:19.734645+0000 front 2024-03-28T03:47:19.734461+0000 (oldest deadline 2024-03-28T03:47:44.434325+0000) 2024-03-28T03:47:50.698 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:47:50 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:47:50.418+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:18.649888+0000 front 2024-03-28T03:47:18.649830+0000 (oldest deadline 2024-03-28T03:47:42.749644+0000) 2024-03-28T03:47:50.699 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:47:50 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:47:50.418+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:29.151297+0000 front 2024-03-28T03:47:29.151274+0000 (oldest deadline 2024-03-28T03:47:50.152219+0000) 2024-03-28T03:47:50.842 INFO:journalctl@ceph.osd.5.smithi165.stdout:Mar 28 03:47:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-5[48453]: 2024-03-28T03:47:50.571+0000 7f5e2fe37700 -1 osd.5 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:15.457115+0000 front 2024-03-28T03:47:15.457379+0000 (oldest deadline 2024-03-28T03:47:46.657477+0000) 2024-03-28T03:47:50.842 INFO:journalctl@ceph.osd.5.smithi165.stdout:Mar 28 03:47:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-5[48453]: 2024-03-28T03:47:50.571+0000 7f5e2fe37700 -1 osd.5 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:26.658406+0000 front 2024-03-28T03:47:26.658220+0000 (oldest deadline 2024-03-28T03:47:50.158235+0000) 2024-03-28T03:47:50.842 INFO:journalctl@ceph.osd.8.smithi165.stdout:Mar 28 03:47:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-8[55184]: 2024-03-28T03:47:50.385+0000 7fd543c54700 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:19.562620+0000 front 2024-03-28T03:47:19.563027+0000 (oldest deadline 2024-03-28T03:47:46.562998+0000) 2024-03-28T03:47:50.842 INFO:journalctl@ceph.osd.8.smithi165.stdout:Mar 28 03:47:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-8[55184]: 2024-03-28T03:47:50.385+0000 7fd543c54700 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:26.563615+0000 front 2024-03-28T03:47:26.563737+0000 (oldest deadline 2024-03-28T03:47:50.063561+0000) 2024-03-28T03:47:50.901 INFO:journalctl@ceph.osd.6.smithi119.stdout:Mar 28 03:47:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-6[57000]: 2024-03-28T03:47:50.631+0000 7f926a600700 -1 osd.6 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:18.788278+0000 front 2024-03-28T03:47:18.788347+0000 (oldest deadline 2024-03-28T03:47:44.088340+0000) 2024-03-28T03:47:50.901 INFO:journalctl@ceph.osd.6.smithi119.stdout:Mar 28 03:47:50 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-6[57000]: 2024-03-28T03:47:50.631+0000 7f926a600700 -1 osd.6 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:28.691333+0000 front 2024-03-28T03:47:28.691283+0000 (oldest deadline 2024-03-28T03:47:49.791440+0000) 2024-03-28T03:47:51.199 INFO:journalctl@ceph.osd.7.smithi132.stdout:Mar 28 03:47:50 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-7[54273]: 2024-03-28T03:47:50.884+0000 7f75fa8b1700 -1 osd.7 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:25.970686+0000 front 2024-03-28T03:47:25.970656+0000 (oldest deadline 2024-03-28T03:47:50.571601+0000) 2024-03-28T03:47:51.378 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:47:51 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:47:51.111+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:18.314545+0000 front 2024-03-28T03:47:18.314582+0000 (oldest deadline 2024-03-28T03:47:44.214430+0000) 2024-03-28T03:47:51.379 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:47:51 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:47:51.111+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:27.715675+0000 front 2024-03-28T03:47:27.715645+0000 (oldest deadline 2024-03-28T03:47:50.615667+0000) 2024-03-28T03:47:51.401 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:47:51 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:47:51.138+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:19.734645+0000 front 2024-03-28T03:47:19.734461+0000 (oldest deadline 2024-03-28T03:47:44.434325+0000) 2024-03-28T03:47:51.698 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:47:51 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:47:51.381+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:18.649888+0000 front 2024-03-28T03:47:18.649830+0000 (oldest deadline 2024-03-28T03:47:42.749644+0000) 2024-03-28T03:47:51.699 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:47:51 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:47:51.381+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:29.151297+0000 front 2024-03-28T03:47:29.151274+0000 (oldest deadline 2024-03-28T03:47:50.152219+0000) 2024-03-28T03:47:51.878 INFO:journalctl@ceph.osd.8.smithi165.stdout:Mar 28 03:47:51 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-8[55184]: 2024-03-28T03:47:51.389+0000 7fd543c54700 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:19.562620+0000 front 2024-03-28T03:47:19.563027+0000 (oldest deadline 2024-03-28T03:47:46.562998+0000) 2024-03-28T03:47:51.879 INFO:journalctl@ceph.osd.8.smithi165.stdout:Mar 28 03:47:51 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-8[55184]: 2024-03-28T03:47:51.389+0000 7fd543c54700 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:26.563615+0000 front 2024-03-28T03:47:26.563737+0000 (oldest deadline 2024-03-28T03:47:50.063561+0000) 2024-03-28T03:47:51.879 INFO:journalctl@ceph.osd.5.smithi165.stdout:Mar 28 03:47:51 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-5[48453]: 2024-03-28T03:47:51.526+0000 7f5e2fe37700 -1 osd.5 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:15.457115+0000 front 2024-03-28T03:47:15.457379+0000 (oldest deadline 2024-03-28T03:47:46.657477+0000) 2024-03-28T03:47:51.879 INFO:journalctl@ceph.osd.5.smithi165.stdout:Mar 28 03:47:51 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-5[48453]: 2024-03-28T03:47:51.526+0000 7f5e2fe37700 -1 osd.5 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:26.658406+0000 front 2024-03-28T03:47:26.658220+0000 (oldest deadline 2024-03-28T03:47:50.158235+0000) 2024-03-28T03:47:51.901 INFO:journalctl@ceph.osd.6.smithi119.stdout:Mar 28 03:47:51 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-6[57000]: 2024-03-28T03:47:51.591+0000 7f926a600700 -1 osd.6 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:18.788278+0000 front 2024-03-28T03:47:18.788347+0000 (oldest deadline 2024-03-28T03:47:44.088340+0000) 2024-03-28T03:47:51.901 INFO:journalctl@ceph.osd.6.smithi119.stdout:Mar 28 03:47:51 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-6[57000]: 2024-03-28T03:47:51.591+0000 7f926a600700 -1 osd.6 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:28.691333+0000 front 2024-03-28T03:47:28.691283+0000 (oldest deadline 2024-03-28T03:47:49.791440+0000) 2024-03-28T03:47:52.198 INFO:journalctl@ceph.osd.7.smithi132.stdout:Mar 28 03:47:51 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-7[54273]: 2024-03-28T03:47:51.861+0000 7f75fa8b1700 -1 osd.7 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:25.970686+0000 front 2024-03-28T03:47:25.970656+0000 (oldest deadline 2024-03-28T03:47:50.571601+0000) 2024-03-28T03:47:52.401 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:47:52 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:47:52.128+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:19.734645+0000 front 2024-03-28T03:47:19.734461+0000 (oldest deadline 2024-03-28T03:47:44.434325+0000) 2024-03-28T03:47:52.401 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:47:52 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:47:52.128+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:25.537050+0000 front 2024-03-28T03:47:25.536933+0000 (oldest deadline 2024-03-28T03:47:51.437142+0000) 2024-03-28T03:47:52.489 INFO:journalctl@ceph.osd.8.smithi165.stdout:Mar 28 03:47:52 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-8[55184]: 2024-03-28T03:47:52.341+0000 7fd543c54700 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:19.562620+0000 front 2024-03-28T03:47:19.563027+0000 (oldest deadline 2024-03-28T03:47:46.562998+0000) 2024-03-28T03:47:52.489 INFO:journalctl@ceph.osd.8.smithi165.stdout:Mar 28 03:47:52 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-8[55184]: 2024-03-28T03:47:52.341+0000 7fd543c54700 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:26.563615+0000 front 2024-03-28T03:47:26.563737+0000 (oldest deadline 2024-03-28T03:47:50.063561+0000) 2024-03-28T03:47:52.490 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:47:52 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:47:52.155+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:18.314545+0000 front 2024-03-28T03:47:18.314582+0000 (oldest deadline 2024-03-28T03:47:44.214430+0000) 2024-03-28T03:47:52.490 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:47:52 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:47:52.155+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:27.715675+0000 front 2024-03-28T03:47:27.715645+0000 (oldest deadline 2024-03-28T03:47:50.615667+0000) 2024-03-28T03:47:52.698 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:47:52 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:47:52.354+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:18.649888+0000 front 2024-03-28T03:47:18.649830+0000 (oldest deadline 2024-03-28T03:47:42.749644+0000) 2024-03-28T03:47:52.699 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:47:52 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:47:52.354+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:29.151297+0000 front 2024-03-28T03:47:29.151274+0000 (oldest deadline 2024-03-28T03:47:50.152219+0000) 2024-03-28T03:47:52.878 INFO:journalctl@ceph.osd.5.smithi165.stdout:Mar 28 03:47:52 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-5[48453]: 2024-03-28T03:47:52.487+0000 7f5e2fe37700 -1 osd.5 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:15.457115+0000 front 2024-03-28T03:47:15.457379+0000 (oldest deadline 2024-03-28T03:47:46.657477+0000) 2024-03-28T03:47:52.879 INFO:journalctl@ceph.osd.5.smithi165.stdout:Mar 28 03:47:52 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-5[48453]: 2024-03-28T03:47:52.487+0000 7f5e2fe37700 -1 osd.5 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:26.658406+0000 front 2024-03-28T03:47:26.658220+0000 (oldest deadline 2024-03-28T03:47:50.158235+0000) 2024-03-28T03:47:52.901 INFO:journalctl@ceph.osd.6.smithi119.stdout:Mar 28 03:47:52 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-6[57000]: 2024-03-28T03:47:52.602+0000 7f926a600700 -1 osd.6 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:18.788278+0000 front 2024-03-28T03:47:18.788347+0000 (oldest deadline 2024-03-28T03:47:44.088340+0000) 2024-03-28T03:47:52.901 INFO:journalctl@ceph.osd.6.smithi119.stdout:Mar 28 03:47:52 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-6[57000]: 2024-03-28T03:47:52.602+0000 7f926a600700 -1 osd.6 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:28.691333+0000 front 2024-03-28T03:47:28.691283+0000 (oldest deadline 2024-03-28T03:47:49.791440+0000) 2024-03-28T03:47:53.198 INFO:journalctl@ceph.osd.7.smithi132.stdout:Mar 28 03:47:52 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-7[54273]: 2024-03-28T03:47:52.877+0000 7f75fa8b1700 -1 osd.7 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:25.970686+0000 front 2024-03-28T03:47:25.970656+0000 (oldest deadline 2024-03-28T03:47:50.571601+0000) 2024-03-28T03:47:53.517 INFO:journalctl@ceph.osd.8.smithi165.stdout:Mar 28 03:47:53 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-8[55184]: 2024-03-28T03:47:53.356+0000 7fd543c54700 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:19.562620+0000 front 2024-03-28T03:47:19.563027+0000 (oldest deadline 2024-03-28T03:47:46.562998+0000) 2024-03-28T03:47:53.518 INFO:journalctl@ceph.osd.8.smithi165.stdout:Mar 28 03:47:53 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-8[55184]: 2024-03-28T03:47:53.356+0000 7fd543c54700 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:26.563615+0000 front 2024-03-28T03:47:26.563737+0000 (oldest deadline 2024-03-28T03:47:50.063561+0000) 2024-03-28T03:47:53.518 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:47:53 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:47:53.148+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:18.314545+0000 front 2024-03-28T03:47:18.314582+0000 (oldest deadline 2024-03-28T03:47:44.214430+0000) 2024-03-28T03:47:53.518 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:47:53 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:47:53.148+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:27.715675+0000 front 2024-03-28T03:47:27.715645+0000 (oldest deadline 2024-03-28T03:47:50.615667+0000) 2024-03-28T03:47:53.554 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:47:53 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:47:53.152+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:19.734645+0000 front 2024-03-28T03:47:19.734461+0000 (oldest deadline 2024-03-28T03:47:44.434325+0000) 2024-03-28T03:47:53.555 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:47:53 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:47:53.152+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:25.537050+0000 front 2024-03-28T03:47:25.536933+0000 (oldest deadline 2024-03-28T03:47:51.437142+0000) 2024-03-28T03:47:53.698 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:47:53 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:47:53.339+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:18.649888+0000 front 2024-03-28T03:47:18.649830+0000 (oldest deadline 2024-03-28T03:47:42.749644+0000) 2024-03-28T03:47:53.699 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:47:53 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:47:53.339+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:29.151297+0000 front 2024-03-28T03:47:29.151274+0000 (oldest deadline 2024-03-28T03:47:50.152219+0000) 2024-03-28T03:47:53.878 INFO:journalctl@ceph.osd.5.smithi165.stdout:Mar 28 03:47:53 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-5[48453]: 2024-03-28T03:47:53.516+0000 7f5e2fe37700 -1 osd.5 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:15.457115+0000 front 2024-03-28T03:47:15.457379+0000 (oldest deadline 2024-03-28T03:47:46.657477+0000) 2024-03-28T03:47:53.878 INFO:journalctl@ceph.osd.5.smithi165.stdout:Mar 28 03:47:53 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-5[48453]: 2024-03-28T03:47:53.516+0000 7f5e2fe37700 -1 osd.5 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:26.658406+0000 front 2024-03-28T03:47:26.658220+0000 (oldest deadline 2024-03-28T03:47:50.158235+0000) 2024-03-28T03:47:53.901 INFO:journalctl@ceph.osd.6.smithi119.stdout:Mar 28 03:47:53 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-6[57000]: 2024-03-28T03:47:53.552+0000 7f926a600700 -1 osd.6 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:18.788278+0000 front 2024-03-28T03:47:18.788347+0000 (oldest deadline 2024-03-28T03:47:44.088340+0000) 2024-03-28T03:47:53.901 INFO:journalctl@ceph.osd.6.smithi119.stdout:Mar 28 03:47:53 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-6[57000]: 2024-03-28T03:47:53.552+0000 7f926a600700 -1 osd.6 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:28.691333+0000 front 2024-03-28T03:47:28.691283+0000 (oldest deadline 2024-03-28T03:47:49.791440+0000) 2024-03-28T03:47:54.198 INFO:journalctl@ceph.osd.4.smithi132.stdout:Mar 28 03:47:53 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-4[47490]: 2024-03-28T03:47:53.931+0000 7f963b69f700 -1 osd.4 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:28.392358+0000 front 2024-03-28T03:47:28.392411+0000 (oldest deadline 2024-03-28T03:47:52.993320+0000) 2024-03-28T03:47:54.199 INFO:journalctl@ceph.osd.7.smithi132.stdout:Mar 28 03:47:53 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-7[54273]: 2024-03-28T03:47:53.835+0000 7f75fa8b1700 -1 osd.7 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:25.970686+0000 front 2024-03-28T03:47:25.970656+0000 (oldest deadline 2024-03-28T03:47:50.571601+0000) 2024-03-28T03:47:54.515 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:47:54 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:47:54.187+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:19.734645+0000 front 2024-03-28T03:47:19.734461+0000 (oldest deadline 2024-03-28T03:47:44.434325+0000) 2024-03-28T03:47:54.515 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:47:54 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:47:54.187+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:25.537050+0000 front 2024-03-28T03:47:25.536933+0000 (oldest deadline 2024-03-28T03:47:51.437142+0000) 2024-03-28T03:47:54.567 INFO:journalctl@ceph.osd.8.smithi165.stdout:Mar 28 03:47:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-8[55184]: 2024-03-28T03:47:54.370+0000 7fd543c54700 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:19.562620+0000 front 2024-03-28T03:47:19.563027+0000 (oldest deadline 2024-03-28T03:47:46.562998+0000) 2024-03-28T03:47:54.567 INFO:journalctl@ceph.osd.8.smithi165.stdout:Mar 28 03:47:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-8[55184]: 2024-03-28T03:47:54.370+0000 7fd543c54700 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:26.563615+0000 front 2024-03-28T03:47:26.563737+0000 (oldest deadline 2024-03-28T03:47:50.063561+0000) 2024-03-28T03:47:54.567 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:47:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:47:54.143+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:18.314545+0000 front 2024-03-28T03:47:18.314582+0000 (oldest deadline 2024-03-28T03:47:44.214430+0000) 2024-03-28T03:47:54.567 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:47:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:47:54.143+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:27.715675+0000 front 2024-03-28T03:47:27.715645+0000 (oldest deadline 2024-03-28T03:47:50.615667+0000) 2024-03-28T03:47:54.698 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:47:54 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:47:54.380+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:18.649888+0000 front 2024-03-28T03:47:18.649830+0000 (oldest deadline 2024-03-28T03:47:42.749644+0000) 2024-03-28T03:47:54.698 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:47:54 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:47:54.380+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:29.151297+0000 front 2024-03-28T03:47:29.151274+0000 (oldest deadline 2024-03-28T03:47:50.152219+0000) 2024-03-28T03:47:54.878 INFO:journalctl@ceph.osd.5.smithi165.stdout:Mar 28 03:47:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-5[48453]: 2024-03-28T03:47:54.565+0000 7f5e2fe37700 -1 osd.5 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:15.457115+0000 front 2024-03-28T03:47:15.457379+0000 (oldest deadline 2024-03-28T03:47:46.657477+0000) 2024-03-28T03:47:54.878 INFO:journalctl@ceph.osd.5.smithi165.stdout:Mar 28 03:47:54 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-5[48453]: 2024-03-28T03:47:54.565+0000 7f5e2fe37700 -1 osd.5 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:26.658406+0000 front 2024-03-28T03:47:26.658220+0000 (oldest deadline 2024-03-28T03:47:50.158235+0000) 2024-03-28T03:47:54.901 INFO:journalctl@ceph.osd.6.smithi119.stdout:Mar 28 03:47:54 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-6[57000]: 2024-03-28T03:47:54.513+0000 7f926a600700 -1 osd.6 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:18.788278+0000 front 2024-03-28T03:47:18.788347+0000 (oldest deadline 2024-03-28T03:47:44.088340+0000) 2024-03-28T03:47:54.901 INFO:journalctl@ceph.osd.6.smithi119.stdout:Mar 28 03:47:54 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-6[57000]: 2024-03-28T03:47:54.513+0000 7f926a600700 -1 osd.6 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:28.691333+0000 front 2024-03-28T03:47:28.691283+0000 (oldest deadline 2024-03-28T03:47:49.791440+0000) 2024-03-28T03:47:55.093 INFO:journalctl@ceph.osd.7.smithi132.stdout:Mar 28 03:47:54 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-7[54273]: 2024-03-28T03:47:54.798+0000 7f75fa8b1700 -1 osd.7 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:25.970686+0000 front 2024-03-28T03:47:25.970656+0000 (oldest deadline 2024-03-28T03:47:50.571601+0000) 2024-03-28T03:47:55.378 INFO:journalctl@ceph.osd.4.smithi132.stdout:Mar 28 03:47:55 smithi132 podman[120737]: 2024-03-28 03:47:55.092124629 +0000 UTC m=+5.630547361 container died a3ea4b5b9d084ad72b83406198e43e2c9cd3549d370f5ad8a3dee6ffc1ed3064 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-4, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-28T03:47:55.378 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:47:55 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:47:55.330+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:18.649888+0000 front 2024-03-28T03:47:18.649830+0000 (oldest deadline 2024-03-28T03:47:42.749644+0000) 2024-03-28T03:47:55.378 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:47:55 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:47:55.330+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:29.151297+0000 front 2024-03-28T03:47:29.151274+0000 (oldest deadline 2024-03-28T03:47:50.152219+0000) 2024-03-28T03:47:55.504 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:47:55 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:47:55.204+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:19.734645+0000 front 2024-03-28T03:47:19.734461+0000 (oldest deadline 2024-03-28T03:47:44.434325+0000) 2024-03-28T03:47:55.505 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:47:55 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:47:55.204+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:25.537050+0000 front 2024-03-28T03:47:25.536933+0000 (oldest deadline 2024-03-28T03:47:51.437142+0000) 2024-03-28T03:47:55.557 INFO:journalctl@ceph.osd.8.smithi165.stdout:Mar 28 03:47:55 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-8[55184]: 2024-03-28T03:47:55.407+0000 7fd543c54700 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:19.562620+0000 front 2024-03-28T03:47:19.563027+0000 (oldest deadline 2024-03-28T03:47:46.562998+0000) 2024-03-28T03:47:55.557 INFO:journalctl@ceph.osd.8.smithi165.stdout:Mar 28 03:47:55 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-8[55184]: 2024-03-28T03:47:55.407+0000 7fd543c54700 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:26.563615+0000 front 2024-03-28T03:47:26.563737+0000 (oldest deadline 2024-03-28T03:47:50.063561+0000) 2024-03-28T03:47:55.557 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:47:55 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:47:55.181+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:18.314545+0000 front 2024-03-28T03:47:18.314582+0000 (oldest deadline 2024-03-28T03:47:44.214430+0000) 2024-03-28T03:47:55.558 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:47:55 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:47:55.181+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:27.715675+0000 front 2024-03-28T03:47:27.715645+0000 (oldest deadline 2024-03-28T03:47:50.615667+0000) 2024-03-28T03:47:55.698 INFO:journalctl@ceph.osd.4.smithi132.stdout:Mar 28 03:47:55 smithi132 podman[120737]: 2024-03-28 03:47:55.484489909 +0000 UTC m=+6.022912592 container cleanup a3ea4b5b9d084ad72b83406198e43e2c9cd3549d370f5ad8a3dee6ffc1ed3064 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-4, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, maintainer=Guillaume Abrioux ) 2024-03-28T03:47:55.698 INFO:journalctl@ceph.osd.4.smithi132.stdout:Mar 28 03:47:55 smithi132 bash[120737]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-4 2024-03-28T03:47:55.878 INFO:journalctl@ceph.osd.5.smithi165.stdout:Mar 28 03:47:55 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-5[48453]: 2024-03-28T03:47:55.555+0000 7f5e2fe37700 -1 osd.5 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:15.457115+0000 front 2024-03-28T03:47:15.457379+0000 (oldest deadline 2024-03-28T03:47:46.657477+0000) 2024-03-28T03:47:55.878 INFO:journalctl@ceph.osd.5.smithi165.stdout:Mar 28 03:47:55 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-5[48453]: 2024-03-28T03:47:55.555+0000 7f5e2fe37700 -1 osd.5 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:26.658406+0000 front 2024-03-28T03:47:26.658220+0000 (oldest deadline 2024-03-28T03:47:50.158235+0000) 2024-03-28T03:47:55.901 INFO:journalctl@ceph.osd.6.smithi119.stdout:Mar 28 03:47:55 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-6[57000]: 2024-03-28T03:47:55.503+0000 7f926a600700 -1 osd.6 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:18.788278+0000 front 2024-03-28T03:47:18.788347+0000 (oldest deadline 2024-03-28T03:47:44.088340+0000) 2024-03-28T03:47:55.901 INFO:journalctl@ceph.osd.6.smithi119.stdout:Mar 28 03:47:55 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-6[57000]: 2024-03-28T03:47:55.503+0000 7f926a600700 -1 osd.6 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:28.691333+0000 front 2024-03-28T03:47:28.691283+0000 (oldest deadline 2024-03-28T03:47:49.791440+0000) 2024-03-28T03:47:56.094 INFO:journalctl@ceph.osd.4.smithi132.stdout:Mar 28 03:47:55 smithi132 podman[120757]: 2024-03-28 03:47:55.761295964 +0000 UTC m=+0.666414115 container remove a3ea4b5b9d084ad72b83406198e43e2c9cd3549d370f5ad8a3dee6ffc1ed3064 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-4, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_BRANCH=HEAD) 2024-03-28T03:47:56.094 INFO:journalctl@ceph.osd.7.smithi132.stdout:Mar 28 03:47:55 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-7[54273]: 2024-03-28T03:47:55.810+0000 7f75fa8b1700 -1 osd.7 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:25.970686+0000 front 2024-03-28T03:47:25.970656+0000 (oldest deadline 2024-03-28T03:47:50.571601+0000) 2024-03-28T03:47:56.357 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:47:56 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:47:56.297+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:18.649888+0000 front 2024-03-28T03:47:18.649830+0000 (oldest deadline 2024-03-28T03:47:42.749644+0000) 2024-03-28T03:47:56.357 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:47:56 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:47:56.297+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:29.151297+0000 front 2024-03-28T03:47:29.151274+0000 (oldest deadline 2024-03-28T03:47:50.152219+0000) 2024-03-28T03:47:56.358 INFO:journalctl@ceph.osd.4.smithi132.stdout:Mar 28 03:47:56 smithi132 podman[120846]: 2024-03-28 03:47:55.99614291 +0000 UTC m=+0.018003113 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-28T03:47:56.358 INFO:journalctl@ceph.osd.4.smithi132.stdout:Mar 28 03:47:56 smithi132 podman[120846]: 2024-03-28 03:47:56.120536768 +0000 UTC m=+0.142396956 container create bd636154e781b5967962d761331ea7ba0e79f764490e0f6937f9464644f8056f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-4-deactivate, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-03-28T03:47:56.552 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:47:56 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:47:56.192+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:19.734645+0000 front 2024-03-28T03:47:19.734461+0000 (oldest deadline 2024-03-28T03:47:44.434325+0000) 2024-03-28T03:47:56.552 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:47:56 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:47:56.192+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:25.537050+0000 front 2024-03-28T03:47:25.536933+0000 (oldest deadline 2024-03-28T03:47:51.437142+0000) 2024-03-28T03:47:56.594 INFO:journalctl@ceph.osd.8.smithi165.stdout:Mar 28 03:47:56 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-8[55184]: 2024-03-28T03:47:56.401+0000 7fd543c54700 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:19.562620+0000 front 2024-03-28T03:47:19.563027+0000 (oldest deadline 2024-03-28T03:47:46.562998+0000) 2024-03-28T03:47:56.595 INFO:journalctl@ceph.osd.8.smithi165.stdout:Mar 28 03:47:56 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-8[55184]: 2024-03-28T03:47:56.401+0000 7fd543c54700 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:26.563615+0000 front 2024-03-28T03:47:26.563737+0000 (oldest deadline 2024-03-28T03:47:50.063561+0000) 2024-03-28T03:47:56.595 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:47:56 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:47:56.168+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:18.314545+0000 front 2024-03-28T03:47:18.314582+0000 (oldest deadline 2024-03-28T03:47:44.214430+0000) 2024-03-28T03:47:56.595 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:47:56 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:47:56.168+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:27.715675+0000 front 2024-03-28T03:47:27.715645+0000 (oldest deadline 2024-03-28T03:47:50.615667+0000) 2024-03-28T03:47:56.653 INFO:journalctl@ceph.osd.4.smithi132.stdout:Mar 28 03:47:56 smithi132 podman[120846]: 2024-03-28 03:47:56.356545463 +0000 UTC m=+0.378405660 container init bd636154e781b5967962d761331ea7ba0e79f764490e0f6937f9464644f8056f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-4-deactivate, GIT_CLEAN=True, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True) 2024-03-28T03:47:56.653 INFO:journalctl@ceph.osd.4.smithi132.stdout:Mar 28 03:47:56 smithi132 podman[120846]: 2024-03-28 03:47:56.359661863 +0000 UTC m=+0.381522061 container start bd636154e781b5967962d761331ea7ba0e79f764490e0f6937f9464644f8056f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-4-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-03-28T03:47:56.653 INFO:journalctl@ceph.osd.4.smithi132.stdout:Mar 28 03:47:56 smithi132 podman[120846]: 2024-03-28 03:47:56.481531985 +0000 UTC m=+0.503392184 container attach bd636154e781b5967962d761331ea7ba0e79f764490e0f6937f9464644f8056f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-4-deactivate, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0) 2024-03-28T03:47:56.878 INFO:journalctl@ceph.osd.5.smithi165.stdout:Mar 28 03:47:56 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-5[48453]: 2024-03-28T03:47:56.593+0000 7f5e2fe37700 -1 osd.5 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:15.457115+0000 front 2024-03-28T03:47:15.457379+0000 (oldest deadline 2024-03-28T03:47:46.657477+0000) 2024-03-28T03:47:56.878 INFO:journalctl@ceph.osd.5.smithi165.stdout:Mar 28 03:47:56 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-5[48453]: 2024-03-28T03:47:56.593+0000 7f5e2fe37700 -1 osd.5 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:26.658406+0000 front 2024-03-28T03:47:26.658220+0000 (oldest deadline 2024-03-28T03:47:50.158235+0000) 2024-03-28T03:47:56.901 INFO:journalctl@ceph.osd.6.smithi119.stdout:Mar 28 03:47:56 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-6[57000]: 2024-03-28T03:47:56.549+0000 7f926a600700 -1 osd.6 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:18.788278+0000 front 2024-03-28T03:47:18.788347+0000 (oldest deadline 2024-03-28T03:47:44.088340+0000) 2024-03-28T03:47:56.901 INFO:journalctl@ceph.osd.6.smithi119.stdout:Mar 28 03:47:56 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-6[57000]: 2024-03-28T03:47:56.549+0000 7f926a600700 -1 osd.6 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:28.691333+0000 front 2024-03-28T03:47:28.691283+0000 (oldest deadline 2024-03-28T03:47:49.791440+0000) 2024-03-28T03:47:56.948 INFO:journalctl@ceph.osd.7.smithi132.stdout:Mar 28 03:47:56 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-7[54273]: 2024-03-28T03:47:56.835+0000 7f75fa8b1700 -1 osd.7 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:25.970686+0000 front 2024-03-28T03:47:25.970656+0000 (oldest deadline 2024-03-28T03:47:50.571601+0000) 2024-03-28T03:47:56.949 INFO:journalctl@ceph.osd.4.smithi132.stdout:Mar 28 03:47:56 smithi132 podman[120846]: 2024-03-28 03:47:56.65279734 +0000 UTC m=+0.674657548 container died bd636154e781b5967962d761331ea7ba0e79f764490e0f6937f9464644f8056f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-4-deactivate, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-28T03:47:57.281 DEBUG:teuthology.orchestra.run.smithi132:> sudo pkill -f 'journalctl -f -n 0 -u ceph-10f73b92-ec98-11ee-95d0-87774f69a715@osd.4.service' 2024-03-28T03:47:57.321 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-28T03:47:57.321 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-03-28T03:47:57.322 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-03-28T03:47:57.322 DEBUG:teuthology.orchestra.run.smithi165:> sudo systemctl stop ceph-10f73b92-ec98-11ee-95d0-87774f69a715@osd.5 2024-03-28T03:47:57.568 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:47:57 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:47:57.210+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:19.734645+0000 front 2024-03-28T03:47:19.734461+0000 (oldest deadline 2024-03-28T03:47:44.434325+0000) 2024-03-28T03:47:57.569 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:47:57 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:47:57.210+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:25.537050+0000 front 2024-03-28T03:47:25.536933+0000 (oldest deadline 2024-03-28T03:47:51.437142+0000) 2024-03-28T03:47:57.621 INFO:journalctl@ceph.osd.5.smithi165.stdout:Mar 28 03:47:57 smithi165 systemd[1]: Stopping Ceph osd.5 for 10f73b92-ec98-11ee-95d0-87774f69a715... 2024-03-28T03:47:57.622 INFO:journalctl@ceph.osd.8.smithi165.stdout:Mar 28 03:47:57 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-8[55184]: 2024-03-28T03:47:57.390+0000 7fd543c54700 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:19.562620+0000 front 2024-03-28T03:47:19.563027+0000 (oldest deadline 2024-03-28T03:47:46.562998+0000) 2024-03-28T03:47:57.622 INFO:journalctl@ceph.osd.8.smithi165.stdout:Mar 28 03:47:57 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-8[55184]: 2024-03-28T03:47:57.390+0000 7fd543c54700 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:26.563615+0000 front 2024-03-28T03:47:26.563737+0000 (oldest deadline 2024-03-28T03:47:50.063561+0000) 2024-03-28T03:47:57.622 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:47:57 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:47:57.187+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:18.314545+0000 front 2024-03-28T03:47:18.314582+0000 (oldest deadline 2024-03-28T03:47:44.214430+0000) 2024-03-28T03:47:57.622 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:47:57 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:47:57.187+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:27.715675+0000 front 2024-03-28T03:47:27.715645+0000 (oldest deadline 2024-03-28T03:47:50.615667+0000) 2024-03-28T03:47:57.698 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:47:57 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:47:57.316+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:18.649888+0000 front 2024-03-28T03:47:18.649830+0000 (oldest deadline 2024-03-28T03:47:42.749644+0000) 2024-03-28T03:47:57.698 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:47:57 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:47:57.316+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:29.151297+0000 front 2024-03-28T03:47:29.151274+0000 (oldest deadline 2024-03-28T03:47:50.152219+0000) 2024-03-28T03:47:57.878 INFO:journalctl@ceph.osd.5.smithi165.stdout:Mar 28 03:47:57 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-5[48453]: 2024-03-28T03:47:57.620+0000 7f5e2fe37700 -1 osd.5 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:15.457115+0000 front 2024-03-28T03:47:15.457379+0000 (oldest deadline 2024-03-28T03:47:46.657477+0000) 2024-03-28T03:47:57.879 INFO:journalctl@ceph.osd.5.smithi165.stdout:Mar 28 03:47:57 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-5[48453]: 2024-03-28T03:47:57.620+0000 7f5e2fe37700 -1 osd.5 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:26.658406+0000 front 2024-03-28T03:47:26.658220+0000 (oldest deadline 2024-03-28T03:47:50.158235+0000) 2024-03-28T03:47:57.879 INFO:journalctl@ceph.osd.5.smithi165.stdout:Mar 28 03:47:57 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-5[48453]: 2024-03-28T03:47:57.655+0000 7f5e33843700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-28T03:47:57.879 INFO:journalctl@ceph.osd.5.smithi165.stdout:Mar 28 03:47:57 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-5[48453]: 2024-03-28T03:47:57.655+0000 7f5e33843700 -1 osd.5 128 *** Got signal Terminated *** 2024-03-28T03:47:57.879 INFO:journalctl@ceph.osd.5.smithi165.stdout:Mar 28 03:47:57 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-5[48453]: 2024-03-28T03:47:57.655+0000 7f5e33843700 -1 osd.5 128 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-28T03:47:57.901 INFO:journalctl@ceph.osd.6.smithi119.stdout:Mar 28 03:47:57 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-6[57000]: 2024-03-28T03:47:57.566+0000 7f926a600700 -1 osd.6 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:18.788278+0000 front 2024-03-28T03:47:18.788347+0000 (oldest deadline 2024-03-28T03:47:44.088340+0000) 2024-03-28T03:47:57.901 INFO:journalctl@ceph.osd.6.smithi119.stdout:Mar 28 03:47:57 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-6[57000]: 2024-03-28T03:47:57.566+0000 7f926a600700 -1 osd.6 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:28.691333+0000 front 2024-03-28T03:47:28.691283+0000 (oldest deadline 2024-03-28T03:47:49.791440+0000) 2024-03-28T03:47:58.198 INFO:journalctl@ceph.osd.7.smithi132.stdout:Mar 28 03:47:57 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-7[54273]: 2024-03-28T03:47:57.823+0000 7f75fa8b1700 -1 osd.7 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:25.970686+0000 front 2024-03-28T03:47:25.970656+0000 (oldest deadline 2024-03-28T03:47:50.571601+0000) 2024-03-28T03:47:58.601 INFO:journalctl@ceph.osd.8.smithi165.stdout:Mar 28 03:47:58 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-8[55184]: 2024-03-28T03:47:58.379+0000 7fd543c54700 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:19.562620+0000 front 2024-03-28T03:47:19.563027+0000 (oldest deadline 2024-03-28T03:47:46.562998+0000) 2024-03-28T03:47:58.602 INFO:journalctl@ceph.osd.8.smithi165.stdout:Mar 28 03:47:58 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-8[55184]: 2024-03-28T03:47:58.379+0000 7fd543c54700 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:26.563615+0000 front 2024-03-28T03:47:26.563737+0000 (oldest deadline 2024-03-28T03:47:50.063561+0000) 2024-03-28T03:47:58.602 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:47:58 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:47:58.161+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:18.314545+0000 front 2024-03-28T03:47:18.314582+0000 (oldest deadline 2024-03-28T03:47:44.214430+0000) 2024-03-28T03:47:58.602 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:47:58 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:47:58.161+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:27.715675+0000 front 2024-03-28T03:47:27.715645+0000 (oldest deadline 2024-03-28T03:47:50.615667+0000) 2024-03-28T03:47:58.607 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:47:58 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:47:58.232+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:19.734645+0000 front 2024-03-28T03:47:19.734461+0000 (oldest deadline 2024-03-28T03:47:44.434325+0000) 2024-03-28T03:47:58.607 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:47:58 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:47:58.232+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:25.537050+0000 front 2024-03-28T03:47:25.536933+0000 (oldest deadline 2024-03-28T03:47:51.437142+0000) 2024-03-28T03:47:58.698 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:47:58 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:47:58.353+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:18.649888+0000 front 2024-03-28T03:47:18.649830+0000 (oldest deadline 2024-03-28T03:47:42.749644+0000) 2024-03-28T03:47:58.699 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:47:58 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:47:58.353+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:29.151297+0000 front 2024-03-28T03:47:29.151274+0000 (oldest deadline 2024-03-28T03:47:50.152219+0000) 2024-03-28T03:47:58.699 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:47:58 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:47:58.353+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:34.853462+0000 front 2024-03-28T03:47:34.853609+0000 (oldest deadline 2024-03-28T03:47:58.353565+0000) 2024-03-28T03:47:58.878 INFO:journalctl@ceph.osd.5.smithi165.stdout:Mar 28 03:47:58 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-5[48453]: 2024-03-28T03:47:58.600+0000 7f5e2fe37700 -1 osd.5 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:15.457115+0000 front 2024-03-28T03:47:15.457379+0000 (oldest deadline 2024-03-28T03:47:46.657477+0000) 2024-03-28T03:47:58.878 INFO:journalctl@ceph.osd.5.smithi165.stdout:Mar 28 03:47:58 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-5[48453]: 2024-03-28T03:47:58.600+0000 7f5e2fe37700 -1 osd.5 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:26.658406+0000 front 2024-03-28T03:47:26.658220+0000 (oldest deadline 2024-03-28T03:47:50.158235+0000) 2024-03-28T03:47:58.878 INFO:journalctl@ceph.osd.5.smithi165.stdout:Mar 28 03:47:58 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-5[48453]: 2024-03-28T03:47:58.600+0000 7f5e2fe37700 -1 osd.5 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:32.458958+0000 front 2024-03-28T03:47:32.458985+0000 (oldest deadline 2024-03-28T03:47:58.359116+0000) 2024-03-28T03:47:58.901 INFO:journalctl@ceph.osd.6.smithi119.stdout:Mar 28 03:47:58 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-6[57000]: 2024-03-28T03:47:58.605+0000 7f926a600700 -1 osd.6 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:18.788278+0000 front 2024-03-28T03:47:18.788347+0000 (oldest deadline 2024-03-28T03:47:44.088340+0000) 2024-03-28T03:47:58.901 INFO:journalctl@ceph.osd.6.smithi119.stdout:Mar 28 03:47:58 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-6[57000]: 2024-03-28T03:47:58.605+0000 7f926a600700 -1 osd.6 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:28.691333+0000 front 2024-03-28T03:47:28.691283+0000 (oldest deadline 2024-03-28T03:47:49.791440+0000) 2024-03-28T03:47:59.199 INFO:journalctl@ceph.osd.7.smithi132.stdout:Mar 28 03:47:58 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-7[54273]: 2024-03-28T03:47:58.855+0000 7f75fa8b1700 -1 osd.7 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:25.970686+0000 front 2024-03-28T03:47:25.970656+0000 (oldest deadline 2024-03-28T03:47:50.571601+0000) 2024-03-28T03:47:59.199 INFO:journalctl@ceph.osd.7.smithi132.stdout:Mar 28 03:47:58 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-7[54273]: 2024-03-28T03:47:58.855+0000 7f75fa8b1700 -1 osd.7 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:34.072660+0000 front 2024-03-28T03:47:34.072689+0000 (oldest deadline 2024-03-28T03:47:58.772829+0000) 2024-03-28T03:47:59.628 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:47:59 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:47:59.133+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:18.314545+0000 front 2024-03-28T03:47:18.314582+0000 (oldest deadline 2024-03-28T03:47:44.214430+0000) 2024-03-28T03:47:59.628 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:47:59 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:47:59.133+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:27.715675+0000 front 2024-03-28T03:47:27.715645+0000 (oldest deadline 2024-03-28T03:47:50.615667+0000) 2024-03-28T03:47:59.629 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:47:59 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:47:59.133+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:32.916784+0000 front 2024-03-28T03:47:32.916834+0000 (oldest deadline 2024-03-28T03:47:58.816929+0000) 2024-03-28T03:47:59.629 INFO:journalctl@ceph.osd.8.smithi165.stdout:Mar 28 03:47:59 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-8[55184]: 2024-03-28T03:47:59.331+0000 7fd543c54700 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:19.562620+0000 front 2024-03-28T03:47:19.563027+0000 (oldest deadline 2024-03-28T03:47:46.562998+0000) 2024-03-28T03:47:59.629 INFO:journalctl@ceph.osd.8.smithi165.stdout:Mar 28 03:47:59 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-8[55184]: 2024-03-28T03:47:59.331+0000 7fd543c54700 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:26.563615+0000 front 2024-03-28T03:47:26.563737+0000 (oldest deadline 2024-03-28T03:47:50.063561+0000) 2024-03-28T03:47:59.629 INFO:journalctl@ceph.osd.8.smithi165.stdout:Mar 28 03:47:59 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-8[55184]: 2024-03-28T03:47:59.331+0000 7fd543c54700 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:34.764640+0000 front 2024-03-28T03:47:34.764564+0000 (oldest deadline 2024-03-28T03:47:58.864724+0000) 2024-03-28T03:47:59.636 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:47:59 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:47:59.216+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:19.734645+0000 front 2024-03-28T03:47:19.734461+0000 (oldest deadline 2024-03-28T03:47:44.434325+0000) 2024-03-28T03:47:59.636 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:47:59 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:47:59.216+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:25.537050+0000 front 2024-03-28T03:47:25.536933+0000 (oldest deadline 2024-03-28T03:47:51.437142+0000) 2024-03-28T03:47:59.637 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:47:59 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:47:59.216+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:32.539643+0000 front 2024-03-28T03:47:32.539558+0000 (oldest deadline 2024-03-28T03:47:58.439707+0000) 2024-03-28T03:47:59.698 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:47:59 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:47:59.394+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:18.649888+0000 front 2024-03-28T03:47:18.649830+0000 (oldest deadline 2024-03-28T03:47:42.749644+0000) 2024-03-28T03:47:59.698 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:47:59 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:47:59.394+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:29.151297+0000 front 2024-03-28T03:47:29.151274+0000 (oldest deadline 2024-03-28T03:47:50.152219+0000) 2024-03-28T03:47:59.698 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:47:59 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:47:59.394+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:34.853462+0000 front 2024-03-28T03:47:34.853609+0000 (oldest deadline 2024-03-28T03:47:58.353565+0000) 2024-03-28T03:47:59.901 INFO:journalctl@ceph.osd.6.smithi119.stdout:Mar 28 03:47:59 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-6[57000]: 2024-03-28T03:47:59.634+0000 7f926a600700 -1 osd.6 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:18.788278+0000 front 2024-03-28T03:47:18.788347+0000 (oldest deadline 2024-03-28T03:47:44.088340+0000) 2024-03-28T03:47:59.901 INFO:journalctl@ceph.osd.6.smithi119.stdout:Mar 28 03:47:59 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-6[57000]: 2024-03-28T03:47:59.634+0000 7f926a600700 -1 osd.6 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:28.691333+0000 front 2024-03-28T03:47:28.691283+0000 (oldest deadline 2024-03-28T03:47:49.791440+0000) 2024-03-28T03:48:00.125 INFO:journalctl@ceph.osd.5.smithi165.stdout:Mar 28 03:47:59 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-5[48453]: 2024-03-28T03:47:59.627+0000 7f5e2fe37700 -1 osd.5 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:15.457115+0000 front 2024-03-28T03:47:15.457379+0000 (oldest deadline 2024-03-28T03:47:46.657477+0000) 2024-03-28T03:48:00.126 INFO:journalctl@ceph.osd.5.smithi165.stdout:Mar 28 03:47:59 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-5[48453]: 2024-03-28T03:47:59.627+0000 7f5e2fe37700 -1 osd.5 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:26.658406+0000 front 2024-03-28T03:47:26.658220+0000 (oldest deadline 2024-03-28T03:47:50.158235+0000) 2024-03-28T03:48:00.126 INFO:journalctl@ceph.osd.5.smithi165.stdout:Mar 28 03:47:59 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-5[48453]: 2024-03-28T03:47:59.627+0000 7f5e2fe37700 -1 osd.5 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:32.458958+0000 front 2024-03-28T03:47:32.458985+0000 (oldest deadline 2024-03-28T03:47:58.359116+0000) 2024-03-28T03:48:00.198 INFO:journalctl@ceph.osd.7.smithi132.stdout:Mar 28 03:47:59 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-7[54273]: 2024-03-28T03:47:59.816+0000 7f75fa8b1700 -1 osd.7 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:25.970686+0000 front 2024-03-28T03:47:25.970656+0000 (oldest deadline 2024-03-28T03:47:50.571601+0000) 2024-03-28T03:48:00.198 INFO:journalctl@ceph.osd.7.smithi132.stdout:Mar 28 03:47:59 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-7[54273]: 2024-03-28T03:47:59.816+0000 7f75fa8b1700 -1 osd.7 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:34.072660+0000 front 2024-03-28T03:47:34.072689+0000 (oldest deadline 2024-03-28T03:47:58.772829+0000) 2024-03-28T03:48:00.378 INFO:journalctl@ceph.osd.8.smithi165.stdout:Mar 28 03:48:00 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-8[55184]: 2024-03-28T03:48:00.332+0000 7fd543c54700 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:19.562620+0000 front 2024-03-28T03:47:19.563027+0000 (oldest deadline 2024-03-28T03:47:46.562998+0000) 2024-03-28T03:48:00.378 INFO:journalctl@ceph.osd.8.smithi165.stdout:Mar 28 03:48:00 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-8[55184]: 2024-03-28T03:48:00.332+0000 7fd543c54700 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:26.563615+0000 front 2024-03-28T03:47:26.563737+0000 (oldest deadline 2024-03-28T03:47:50.063561+0000) 2024-03-28T03:48:00.378 INFO:journalctl@ceph.osd.8.smithi165.stdout:Mar 28 03:48:00 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-8[55184]: 2024-03-28T03:48:00.333+0000 7fd543c54700 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:34.764640+0000 front 2024-03-28T03:47:34.764564+0000 (oldest deadline 2024-03-28T03:47:58.864724+0000) 2024-03-28T03:48:00.379 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:00 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:00.124+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:18.314545+0000 front 2024-03-28T03:47:18.314582+0000 (oldest deadline 2024-03-28T03:47:44.214430+0000) 2024-03-28T03:48:00.379 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:00 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:00.124+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:27.715675+0000 front 2024-03-28T03:47:27.715645+0000 (oldest deadline 2024-03-28T03:47:50.615667+0000) 2024-03-28T03:48:00.379 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:00 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:00.124+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:32.916784+0000 front 2024-03-28T03:47:32.916834+0000 (oldest deadline 2024-03-28T03:47:58.816929+0000) 2024-03-28T03:48:00.651 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:00 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:48:00.259+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:19.734645+0000 front 2024-03-28T03:47:19.734461+0000 (oldest deadline 2024-03-28T03:47:44.434325+0000) 2024-03-28T03:48:00.652 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:00 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:48:00.259+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:25.537050+0000 front 2024-03-28T03:47:25.536933+0000 (oldest deadline 2024-03-28T03:47:51.437142+0000) 2024-03-28T03:48:00.652 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:00 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:48:00.259+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:32.539643+0000 front 2024-03-28T03:47:32.539558+0000 (oldest deadline 2024-03-28T03:47:58.439707+0000) 2024-03-28T03:48:00.698 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:00 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:00.410+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:18.649888+0000 front 2024-03-28T03:47:18.649830+0000 (oldest deadline 2024-03-28T03:47:42.749644+0000) 2024-03-28T03:48:00.698 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:00 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:00.410+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:29.151297+0000 front 2024-03-28T03:47:29.151274+0000 (oldest deadline 2024-03-28T03:47:50.152219+0000) 2024-03-28T03:48:00.698 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:00 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:00.410+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:34.853462+0000 front 2024-03-28T03:47:34.853609+0000 (oldest deadline 2024-03-28T03:47:58.353565+0000) 2024-03-28T03:48:01.128 INFO:journalctl@ceph.osd.5.smithi165.stdout:Mar 28 03:48:00 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-5[48453]: 2024-03-28T03:48:00.672+0000 7f5e2fe37700 -1 osd.5 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:15.457115+0000 front 2024-03-28T03:47:15.457379+0000 (oldest deadline 2024-03-28T03:47:46.657477+0000) 2024-03-28T03:48:01.128 INFO:journalctl@ceph.osd.5.smithi165.stdout:Mar 28 03:48:00 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-5[48453]: 2024-03-28T03:48:00.672+0000 7f5e2fe37700 -1 osd.5 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:26.658406+0000 front 2024-03-28T03:47:26.658220+0000 (oldest deadline 2024-03-28T03:47:50.158235+0000) 2024-03-28T03:48:01.129 INFO:journalctl@ceph.osd.5.smithi165.stdout:Mar 28 03:48:00 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-5[48453]: 2024-03-28T03:48:00.672+0000 7f5e2fe37700 -1 osd.5 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:32.458958+0000 front 2024-03-28T03:47:32.458985+0000 (oldest deadline 2024-03-28T03:47:58.359116+0000) 2024-03-28T03:48:01.151 INFO:journalctl@ceph.osd.6.smithi119.stdout:Mar 28 03:48:00 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-6[57000]: 2024-03-28T03:48:00.684+0000 7f926a600700 -1 osd.6 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:18.788278+0000 front 2024-03-28T03:47:18.788347+0000 (oldest deadline 2024-03-28T03:47:44.088340+0000) 2024-03-28T03:48:01.151 INFO:journalctl@ceph.osd.6.smithi119.stdout:Mar 28 03:48:00 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-6[57000]: 2024-03-28T03:48:00.684+0000 7f926a600700 -1 osd.6 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:28.691333+0000 front 2024-03-28T03:47:28.691283+0000 (oldest deadline 2024-03-28T03:47:49.791440+0000) 2024-03-28T03:48:01.151 INFO:journalctl@ceph.osd.6.smithi119.stdout:Mar 28 03:48:00 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-6[57000]: 2024-03-28T03:48:00.684+0000 7f926a600700 -1 osd.6 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:35.595041+0000 front 2024-03-28T03:47:35.594722+0000 (oldest deadline 2024-03-28T03:48:00.294699+0000) 2024-03-28T03:48:01.198 INFO:journalctl@ceph.osd.7.smithi132.stdout:Mar 28 03:48:00 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-7[54273]: 2024-03-28T03:48:00.842+0000 7f75fa8b1700 -1 osd.7 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:25.970686+0000 front 2024-03-28T03:47:25.970656+0000 (oldest deadline 2024-03-28T03:47:50.571601+0000) 2024-03-28T03:48:01.198 INFO:journalctl@ceph.osd.7.smithi132.stdout:Mar 28 03:48:00 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-7[54273]: 2024-03-28T03:48:00.842+0000 7f75fa8b1700 -1 osd.7 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:34.072660+0000 front 2024-03-28T03:47:34.072689+0000 (oldest deadline 2024-03-28T03:47:58.772829+0000) 2024-03-28T03:48:01.628 INFO:journalctl@ceph.osd.8.smithi165.stdout:Mar 28 03:48:01 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-8[55184]: 2024-03-28T03:48:01.333+0000 7fd543c54700 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:19.562620+0000 front 2024-03-28T03:47:19.563027+0000 (oldest deadline 2024-03-28T03:47:46.562998+0000) 2024-03-28T03:48:01.629 INFO:journalctl@ceph.osd.8.smithi165.stdout:Mar 28 03:48:01 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-8[55184]: 2024-03-28T03:48:01.333+0000 7fd543c54700 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:26.563615+0000 front 2024-03-28T03:47:26.563737+0000 (oldest deadline 2024-03-28T03:47:50.063561+0000) 2024-03-28T03:48:01.629 INFO:journalctl@ceph.osd.8.smithi165.stdout:Mar 28 03:48:01 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-8[55184]: 2024-03-28T03:48:01.333+0000 7fd543c54700 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:34.764640+0000 front 2024-03-28T03:47:34.764564+0000 (oldest deadline 2024-03-28T03:47:58.864724+0000) 2024-03-28T03:48:01.629 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:01 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:01.162+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:18.314545+0000 front 2024-03-28T03:47:18.314582+0000 (oldest deadline 2024-03-28T03:47:44.214430+0000) 2024-03-28T03:48:01.629 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:01 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:01.162+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:27.715675+0000 front 2024-03-28T03:47:27.715645+0000 (oldest deadline 2024-03-28T03:47:50.615667+0000) 2024-03-28T03:48:01.629 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:01 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:01.162+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:32.916784+0000 front 2024-03-28T03:47:32.916834+0000 (oldest deadline 2024-03-28T03:47:58.816929+0000) 2024-03-28T03:48:01.651 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:01 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:48:01.301+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:19.734645+0000 front 2024-03-28T03:47:19.734461+0000 (oldest deadline 2024-03-28T03:47:44.434325+0000) 2024-03-28T03:48:01.651 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:01 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:48:01.301+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:25.537050+0000 front 2024-03-28T03:47:25.536933+0000 (oldest deadline 2024-03-28T03:47:51.437142+0000) 2024-03-28T03:48:01.651 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:01 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:48:01.301+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:32.539643+0000 front 2024-03-28T03:47:32.539558+0000 (oldest deadline 2024-03-28T03:47:58.439707+0000) 2024-03-28T03:48:01.699 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:01 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:01.378+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:18.649888+0000 front 2024-03-28T03:47:18.649830+0000 (oldest deadline 2024-03-28T03:47:42.749644+0000) 2024-03-28T03:48:01.699 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:01 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:01.378+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:29.151297+0000 front 2024-03-28T03:47:29.151274+0000 (oldest deadline 2024-03-28T03:47:50.152219+0000) 2024-03-28T03:48:01.699 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:01 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:01.378+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:34.853462+0000 front 2024-03-28T03:47:34.853609+0000 (oldest deadline 2024-03-28T03:47:58.353565+0000) 2024-03-28T03:48:02.128 INFO:journalctl@ceph.osd.5.smithi165.stdout:Mar 28 03:48:01 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-5[48453]: 2024-03-28T03:48:01.710+0000 7f5e2fe37700 -1 osd.5 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:15.457115+0000 front 2024-03-28T03:47:15.457379+0000 (oldest deadline 2024-03-28T03:47:46.657477+0000) 2024-03-28T03:48:02.128 INFO:journalctl@ceph.osd.5.smithi165.stdout:Mar 28 03:48:01 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-5[48453]: 2024-03-28T03:48:01.710+0000 7f5e2fe37700 -1 osd.5 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:26.658406+0000 front 2024-03-28T03:47:26.658220+0000 (oldest deadline 2024-03-28T03:47:50.158235+0000) 2024-03-28T03:48:02.129 INFO:journalctl@ceph.osd.5.smithi165.stdout:Mar 28 03:48:01 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-5[48453]: 2024-03-28T03:48:01.710+0000 7f5e2fe37700 -1 osd.5 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:32.458958+0000 front 2024-03-28T03:47:32.458985+0000 (oldest deadline 2024-03-28T03:47:58.359116+0000) 2024-03-28T03:48:02.151 INFO:journalctl@ceph.osd.6.smithi119.stdout:Mar 28 03:48:01 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-6[57000]: 2024-03-28T03:48:01.651+0000 7f926a600700 -1 osd.6 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:18.788278+0000 front 2024-03-28T03:47:18.788347+0000 (oldest deadline 2024-03-28T03:47:44.088340+0000) 2024-03-28T03:48:02.151 INFO:journalctl@ceph.osd.6.smithi119.stdout:Mar 28 03:48:01 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-6[57000]: 2024-03-28T03:48:01.651+0000 7f926a600700 -1 osd.6 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:28.691333+0000 front 2024-03-28T03:47:28.691283+0000 (oldest deadline 2024-03-28T03:47:49.791440+0000) 2024-03-28T03:48:02.151 INFO:journalctl@ceph.osd.6.smithi119.stdout:Mar 28 03:48:01 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-6[57000]: 2024-03-28T03:48:01.651+0000 7f926a600700 -1 osd.6 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:35.595041+0000 front 2024-03-28T03:47:35.594722+0000 (oldest deadline 2024-03-28T03:48:00.294699+0000) 2024-03-28T03:48:02.198 INFO:journalctl@ceph.osd.7.smithi132.stdout:Mar 28 03:48:01 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-7[54273]: 2024-03-28T03:48:01.889+0000 7f75fa8b1700 -1 osd.7 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:25.970686+0000 front 2024-03-28T03:47:25.970656+0000 (oldest deadline 2024-03-28T03:47:50.571601+0000) 2024-03-28T03:48:02.198 INFO:journalctl@ceph.osd.7.smithi132.stdout:Mar 28 03:48:01 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-7[54273]: 2024-03-28T03:48:01.889+0000 7f75fa8b1700 -1 osd.7 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:34.072660+0000 front 2024-03-28T03:47:34.072689+0000 (oldest deadline 2024-03-28T03:47:58.772829+0000) 2024-03-28T03:48:02.628 INFO:journalctl@ceph.osd.8.smithi165.stdout:Mar 28 03:48:02 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-8[55184]: 2024-03-28T03:48:02.318+0000 7fd543c54700 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:19.562620+0000 front 2024-03-28T03:47:19.563027+0000 (oldest deadline 2024-03-28T03:47:46.562998+0000) 2024-03-28T03:48:02.628 INFO:journalctl@ceph.osd.8.smithi165.stdout:Mar 28 03:48:02 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-8[55184]: 2024-03-28T03:48:02.318+0000 7fd543c54700 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:26.563615+0000 front 2024-03-28T03:47:26.563737+0000 (oldest deadline 2024-03-28T03:47:50.063561+0000) 2024-03-28T03:48:02.629 INFO:journalctl@ceph.osd.8.smithi165.stdout:Mar 28 03:48:02 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-8[55184]: 2024-03-28T03:48:02.318+0000 7fd543c54700 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:34.764640+0000 front 2024-03-28T03:47:34.764564+0000 (oldest deadline 2024-03-28T03:47:58.864724+0000) 2024-03-28T03:48:02.629 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:02 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:02.177+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:18.314545+0000 front 2024-03-28T03:47:18.314582+0000 (oldest deadline 2024-03-28T03:47:44.214430+0000) 2024-03-28T03:48:02.629 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:02 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:02.177+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:27.715675+0000 front 2024-03-28T03:47:27.715645+0000 (oldest deadline 2024-03-28T03:47:50.615667+0000) 2024-03-28T03:48:02.629 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:02 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:02.177+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:32.916784+0000 front 2024-03-28T03:47:32.916834+0000 (oldest deadline 2024-03-28T03:47:58.816929+0000) 2024-03-28T03:48:02.651 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:02 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:48:02.296+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:19.734645+0000 front 2024-03-28T03:47:19.734461+0000 (oldest deadline 2024-03-28T03:47:44.434325+0000) 2024-03-28T03:48:02.651 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:02 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:48:02.296+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:25.537050+0000 front 2024-03-28T03:47:25.536933+0000 (oldest deadline 2024-03-28T03:47:51.437142+0000) 2024-03-28T03:48:02.651 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:02 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:48:02.296+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:32.539643+0000 front 2024-03-28T03:47:32.539558+0000 (oldest deadline 2024-03-28T03:47:58.439707+0000) 2024-03-28T03:48:02.698 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:02 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:02.388+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:18.649888+0000 front 2024-03-28T03:47:18.649830+0000 (oldest deadline 2024-03-28T03:47:42.749644+0000) 2024-03-28T03:48:02.698 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:02 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:02.388+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:29.151297+0000 front 2024-03-28T03:47:29.151274+0000 (oldest deadline 2024-03-28T03:47:50.152219+0000) 2024-03-28T03:48:02.698 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:02 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:02.388+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:34.853462+0000 front 2024-03-28T03:47:34.853609+0000 (oldest deadline 2024-03-28T03:47:58.353565+0000) 2024-03-28T03:48:03.151 INFO:journalctl@ceph.osd.6.smithi119.stdout:Mar 28 03:48:02 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-6[57000]: 2024-03-28T03:48:02.676+0000 7f926a600700 -1 osd.6 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:18.788278+0000 front 2024-03-28T03:47:18.788347+0000 (oldest deadline 2024-03-28T03:47:44.088340+0000) 2024-03-28T03:48:03.151 INFO:journalctl@ceph.osd.6.smithi119.stdout:Mar 28 03:48:02 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-6[57000]: 2024-03-28T03:48:02.676+0000 7f926a600700 -1 osd.6 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:28.691333+0000 front 2024-03-28T03:47:28.691283+0000 (oldest deadline 2024-03-28T03:47:49.791440+0000) 2024-03-28T03:48:03.151 INFO:journalctl@ceph.osd.6.smithi119.stdout:Mar 28 03:48:02 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-6[57000]: 2024-03-28T03:48:02.676+0000 7f926a600700 -1 osd.6 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:35.595041+0000 front 2024-03-28T03:47:35.594722+0000 (oldest deadline 2024-03-28T03:48:00.294699+0000) 2024-03-28T03:48:03.198 INFO:journalctl@ceph.osd.7.smithi132.stdout:Mar 28 03:48:02 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-7[54273]: 2024-03-28T03:48:02.856+0000 7f75fa8b1700 -1 osd.7 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:25.970686+0000 front 2024-03-28T03:47:25.970656+0000 (oldest deadline 2024-03-28T03:47:50.571601+0000) 2024-03-28T03:48:03.198 INFO:journalctl@ceph.osd.7.smithi132.stdout:Mar 28 03:48:02 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-7[54273]: 2024-03-28T03:48:02.856+0000 7f75fa8b1700 -1 osd.7 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:34.072660+0000 front 2024-03-28T03:47:34.072689+0000 (oldest deadline 2024-03-28T03:47:58.772829+0000) 2024-03-28T03:48:03.270 INFO:journalctl@ceph.osd.5.smithi165.stdout:Mar 28 03:48:03 smithi165 podman[123909]: 2024-03-28 03:48:03.009095683 +0000 UTC m=+5.494290595 container died 9ef3682a3653a596cf99eb6f4c04a86ccbc4cdcf91e86587a9ba6e8871a254e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.build-date=20240319, ceph=True, io.buildah.version=1.33.5) 2024-03-28T03:48:03.270 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:03 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:03.140+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:18.314545+0000 front 2024-03-28T03:47:18.314582+0000 (oldest deadline 2024-03-28T03:47:44.214430+0000) 2024-03-28T03:48:03.270 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:03 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:03.140+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:27.715675+0000 front 2024-03-28T03:47:27.715645+0000 (oldest deadline 2024-03-28T03:47:50.615667+0000) 2024-03-28T03:48:03.270 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:03 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:03.140+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:32.916784+0000 front 2024-03-28T03:47:32.916834+0000 (oldest deadline 2024-03-28T03:47:58.816929+0000) 2024-03-28T03:48:03.611 INFO:journalctl@ceph.osd.5.smithi165.stdout:Mar 28 03:48:03 smithi165 podman[123909]: 2024-03-28 03:48:03.368349383 +0000 UTC m=+5.853544295 container cleanup 9ef3682a3653a596cf99eb6f4c04a86ccbc4cdcf91e86587a9ba6e8871a254e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-5, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-03-28T03:48:03.612 INFO:journalctl@ceph.osd.5.smithi165.stdout:Mar 28 03:48:03 smithi165 bash[123909]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-5 2024-03-28T03:48:03.612 INFO:journalctl@ceph.osd.8.smithi165.stdout:Mar 28 03:48:03 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-8[55184]: 2024-03-28T03:48:03.331+0000 7fd543c54700 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:19.562620+0000 front 2024-03-28T03:47:19.563027+0000 (oldest deadline 2024-03-28T03:47:46.562998+0000) 2024-03-28T03:48:03.612 INFO:journalctl@ceph.osd.8.smithi165.stdout:Mar 28 03:48:03 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-8[55184]: 2024-03-28T03:48:03.331+0000 7fd543c54700 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:26.563615+0000 front 2024-03-28T03:47:26.563737+0000 (oldest deadline 2024-03-28T03:47:50.063561+0000) 2024-03-28T03:48:03.612 INFO:journalctl@ceph.osd.8.smithi165.stdout:Mar 28 03:48:03 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-8[55184]: 2024-03-28T03:48:03.331+0000 7fd543c54700 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:34.764640+0000 front 2024-03-28T03:47:34.764564+0000 (oldest deadline 2024-03-28T03:47:58.864724+0000) 2024-03-28T03:48:03.651 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:48:03.328+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:19.734645+0000 front 2024-03-28T03:47:19.734461+0000 (oldest deadline 2024-03-28T03:47:44.434325+0000) 2024-03-28T03:48:03.652 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:48:03.328+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:25.537050+0000 front 2024-03-28T03:47:25.536933+0000 (oldest deadline 2024-03-28T03:47:51.437142+0000) 2024-03-28T03:48:03.652 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:48:03.328+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:32.539643+0000 front 2024-03-28T03:47:32.539558+0000 (oldest deadline 2024-03-28T03:47:58.439707+0000) 2024-03-28T03:48:03.698 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:03 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:03.394+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:18.649888+0000 front 2024-03-28T03:47:18.649830+0000 (oldest deadline 2024-03-28T03:47:42.749644+0000) 2024-03-28T03:48:03.698 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:03 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:03.394+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:29.151297+0000 front 2024-03-28T03:47:29.151274+0000 (oldest deadline 2024-03-28T03:47:50.152219+0000) 2024-03-28T03:48:03.699 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:03 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:03.394+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:34.853462+0000 front 2024-03-28T03:47:34.853609+0000 (oldest deadline 2024-03-28T03:47:58.353565+0000) 2024-03-28T03:48:03.878 INFO:journalctl@ceph.osd.5.smithi165.stdout:Mar 28 03:48:03 smithi165 podman[123926]: 2024-03-28 03:48:03.610981393 +0000 UTC m=+0.595840377 container remove 9ef3682a3653a596cf99eb6f4c04a86ccbc4cdcf91e86587a9ba6e8871a254e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-5, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c) 2024-03-28T03:48:04.151 INFO:journalctl@ceph.osd.6.smithi119.stdout:Mar 28 03:48:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-6[57000]: 2024-03-28T03:48:03.725+0000 7f926a600700 -1 osd.6 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:18.788278+0000 front 2024-03-28T03:47:18.788347+0000 (oldest deadline 2024-03-28T03:47:44.088340+0000) 2024-03-28T03:48:04.151 INFO:journalctl@ceph.osd.6.smithi119.stdout:Mar 28 03:48:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-6[57000]: 2024-03-28T03:48:03.725+0000 7f926a600700 -1 osd.6 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:28.691333+0000 front 2024-03-28T03:47:28.691283+0000 (oldest deadline 2024-03-28T03:47:49.791440+0000) 2024-03-28T03:48:04.151 INFO:journalctl@ceph.osd.6.smithi119.stdout:Mar 28 03:48:03 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-6[57000]: 2024-03-28T03:48:03.725+0000 7f926a600700 -1 osd.6 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:35.595041+0000 front 2024-03-28T03:47:35.594722+0000 (oldest deadline 2024-03-28T03:48:00.294699+0000) 2024-03-28T03:48:04.177 INFO:journalctl@ceph.osd.5.smithi165.stdout:Mar 28 03:48:03 smithi165 podman[124021]: 2024-03-28 03:48:03.827451177 +0000 UTC m=+0.017864196 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-28T03:48:04.177 INFO:journalctl@ceph.osd.5.smithi165.stdout:Mar 28 03:48:03 smithi165 podman[124021]: 2024-03-28 03:48:03.928626714 +0000 UTC m=+0.119039781 container create 3f7e8aa755e1fb2f880e32db033d70272919012f8225b70964bf179195890e6a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-5-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-03-28T03:48:04.177 INFO:journalctl@ceph.osd.5.smithi165.stdout:Mar 28 03:48:04 smithi165 podman[124021]: 2024-03-28 03:48:04.147678983 +0000 UTC m=+0.338092005 container init 3f7e8aa755e1fb2f880e32db033d70272919012f8225b70964bf179195890e6a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-5-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-03-28T03:48:04.177 INFO:journalctl@ceph.osd.5.smithi165.stdout:Mar 28 03:48:04 smithi165 podman[124021]: 2024-03-28 03:48:04.150855405 +0000 UTC m=+0.341268416 container start 3f7e8aa755e1fb2f880e32db033d70272919012f8225b70964bf179195890e6a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-5-deactivate, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-03-28T03:48:04.198 INFO:journalctl@ceph.osd.7.smithi132.stdout:Mar 28 03:48:03 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-7[54273]: 2024-03-28T03:48:03.880+0000 7f75fa8b1700 -1 osd.7 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:25.970686+0000 front 2024-03-28T03:47:25.970656+0000 (oldest deadline 2024-03-28T03:47:50.571601+0000) 2024-03-28T03:48:04.198 INFO:journalctl@ceph.osd.7.smithi132.stdout:Mar 28 03:48:03 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-7[54273]: 2024-03-28T03:48:03.880+0000 7f75fa8b1700 -1 osd.7 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:34.072660+0000 front 2024-03-28T03:47:34.072689+0000 (oldest deadline 2024-03-28T03:47:58.772829+0000) 2024-03-28T03:48:04.438 INFO:journalctl@ceph.osd.5.smithi165.stdout:Mar 28 03:48:04 smithi165 podman[124021]: 2024-03-28 03:48:04.256237606 +0000 UTC m=+0.446650629 container attach 3f7e8aa755e1fb2f880e32db033d70272919012f8225b70964bf179195890e6a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-5-deactivate, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-03-28T03:48:04.439 INFO:journalctl@ceph.osd.8.smithi165.stdout:Mar 28 03:48:04 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-8[55184]: 2024-03-28T03:48:04.339+0000 7fd543c54700 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:19.562620+0000 front 2024-03-28T03:47:19.563027+0000 (oldest deadline 2024-03-28T03:47:46.562998+0000) 2024-03-28T03:48:04.439 INFO:journalctl@ceph.osd.8.smithi165.stdout:Mar 28 03:48:04 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-8[55184]: 2024-03-28T03:48:04.339+0000 7fd543c54700 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:26.563615+0000 front 2024-03-28T03:47:26.563737+0000 (oldest deadline 2024-03-28T03:47:50.063561+0000) 2024-03-28T03:48:04.439 INFO:journalctl@ceph.osd.8.smithi165.stdout:Mar 28 03:48:04 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-8[55184]: 2024-03-28T03:48:04.339+0000 7fd543c54700 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:34.764640+0000 front 2024-03-28T03:47:34.764564+0000 (oldest deadline 2024-03-28T03:47:58.864724+0000) 2024-03-28T03:48:04.439 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:04 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:04.175+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:18.314545+0000 front 2024-03-28T03:47:18.314582+0000 (oldest deadline 2024-03-28T03:47:44.214430+0000) 2024-03-28T03:48:04.439 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:04 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:04.175+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:27.715675+0000 front 2024-03-28T03:47:27.715645+0000 (oldest deadline 2024-03-28T03:47:50.615667+0000) 2024-03-28T03:48:04.439 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:04 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:04.175+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:32.916784+0000 front 2024-03-28T03:47:32.916834+0000 (oldest deadline 2024-03-28T03:47:58.816929+0000) 2024-03-28T03:48:04.651 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:48:04.314+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:19.734645+0000 front 2024-03-28T03:47:19.734461+0000 (oldest deadline 2024-03-28T03:47:44.434325+0000) 2024-03-28T03:48:04.651 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:48:04.314+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:25.537050+0000 front 2024-03-28T03:47:25.536933+0000 (oldest deadline 2024-03-28T03:47:51.437142+0000) 2024-03-28T03:48:04.651 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:48:04.314+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:32.539643+0000 front 2024-03-28T03:47:32.539558+0000 (oldest deadline 2024-03-28T03:47:58.439707+0000) 2024-03-28T03:48:04.698 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:04 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:04.390+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:18.649888+0000 front 2024-03-28T03:47:18.649830+0000 (oldest deadline 2024-03-28T03:47:42.749644+0000) 2024-03-28T03:48:04.698 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:04 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:04.390+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:29.151297+0000 front 2024-03-28T03:47:29.151274+0000 (oldest deadline 2024-03-28T03:47:50.152219+0000) 2024-03-28T03:48:04.699 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:04 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:04.390+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:34.853462+0000 front 2024-03-28T03:47:34.853609+0000 (oldest deadline 2024-03-28T03:47:58.353565+0000) 2024-03-28T03:48:04.726 INFO:journalctl@ceph.osd.5.smithi165.stdout:Mar 28 03:48:04 smithi165 podman[124021]: 2024-03-28 03:48:04.43787382 +0000 UTC m=+0.628286846 container died 3f7e8aa755e1fb2f880e32db033d70272919012f8225b70964bf179195890e6a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-5-deactivate, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-28T03:48:05.048 DEBUG:teuthology.orchestra.run.smithi165:> sudo pkill -f 'journalctl -f -n 0 -u ceph-10f73b92-ec98-11ee-95d0-87774f69a715@osd.5.service' 2024-03-28T03:48:05.088 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-28T03:48:05.088 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-03-28T03:48:05.088 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-03-28T03:48:05.088 DEBUG:teuthology.orchestra.run.smithi119:> sudo systemctl stop ceph-10f73b92-ec98-11ee-95d0-87774f69a715@osd.6 2024-03-28T03:48:05.151 INFO:journalctl@ceph.osd.6.smithi119.stdout:Mar 28 03:48:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-6[57000]: 2024-03-28T03:48:04.753+0000 7f926a600700 -1 osd.6 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:18.788278+0000 front 2024-03-28T03:47:18.788347+0000 (oldest deadline 2024-03-28T03:47:44.088340+0000) 2024-03-28T03:48:05.151 INFO:journalctl@ceph.osd.6.smithi119.stdout:Mar 28 03:48:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-6[57000]: 2024-03-28T03:48:04.753+0000 7f926a600700 -1 osd.6 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:28.691333+0000 front 2024-03-28T03:47:28.691283+0000 (oldest deadline 2024-03-28T03:47:49.791440+0000) 2024-03-28T03:48:05.151 INFO:journalctl@ceph.osd.6.smithi119.stdout:Mar 28 03:48:04 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-6[57000]: 2024-03-28T03:48:04.754+0000 7f926a600700 -1 osd.6 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:35.595041+0000 front 2024-03-28T03:47:35.594722+0000 (oldest deadline 2024-03-28T03:48:00.294699+0000) 2024-03-28T03:48:05.198 INFO:journalctl@ceph.osd.7.smithi132.stdout:Mar 28 03:48:04 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-7[54273]: 2024-03-28T03:48:04.917+0000 7f75fa8b1700 -1 osd.7 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:25.970686+0000 front 2024-03-28T03:47:25.970656+0000 (oldest deadline 2024-03-28T03:47:50.571601+0000) 2024-03-28T03:48:05.198 INFO:journalctl@ceph.osd.7.smithi132.stdout:Mar 28 03:48:04 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-7[54273]: 2024-03-28T03:48:04.917+0000 7f75fa8b1700 -1 osd.7 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:34.072660+0000 front 2024-03-28T03:47:34.072689+0000 (oldest deadline 2024-03-28T03:47:58.772829+0000) 2024-03-28T03:48:05.301 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:05 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:05.213+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:18.314545+0000 front 2024-03-28T03:47:18.314582+0000 (oldest deadline 2024-03-28T03:47:44.214430+0000) 2024-03-28T03:48:05.301 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:05 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:05.213+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:27.715675+0000 front 2024-03-28T03:47:27.715645+0000 (oldest deadline 2024-03-28T03:47:50.615667+0000) 2024-03-28T03:48:05.301 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:05 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:05.213+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:32.916784+0000 front 2024-03-28T03:47:32.916834+0000 (oldest deadline 2024-03-28T03:47:58.816929+0000) 2024-03-28T03:48:05.628 INFO:journalctl@ceph.osd.8.smithi165.stdout:Mar 28 03:48:05 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-8[55184]: 2024-03-28T03:48:05.300+0000 7fd543c54700 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:19.562620+0000 front 2024-03-28T03:47:19.563027+0000 (oldest deadline 2024-03-28T03:47:46.562998+0000) 2024-03-28T03:48:05.628 INFO:journalctl@ceph.osd.8.smithi165.stdout:Mar 28 03:48:05 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-8[55184]: 2024-03-28T03:48:05.300+0000 7fd543c54700 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:26.563615+0000 front 2024-03-28T03:47:26.563737+0000 (oldest deadline 2024-03-28T03:47:50.063561+0000) 2024-03-28T03:48:05.628 INFO:journalctl@ceph.osd.8.smithi165.stdout:Mar 28 03:48:05 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-8[55184]: 2024-03-28T03:48:05.300+0000 7fd543c54700 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:34.764640+0000 front 2024-03-28T03:47:34.764564+0000 (oldest deadline 2024-03-28T03:47:58.864724+0000) 2024-03-28T03:48:05.651 INFO:journalctl@ceph.osd.6.smithi119.stdout:Mar 28 03:48:05 smithi119 systemd[1]: Stopping Ceph osd.6 for 10f73b92-ec98-11ee-95d0-87774f69a715... 2024-03-28T03:48:05.651 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:05 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:48:05.269+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:19.734645+0000 front 2024-03-28T03:47:19.734461+0000 (oldest deadline 2024-03-28T03:47:44.434325+0000) 2024-03-28T03:48:05.651 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:05 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:48:05.269+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:25.537050+0000 front 2024-03-28T03:47:25.536933+0000 (oldest deadline 2024-03-28T03:47:51.437142+0000) 2024-03-28T03:48:05.651 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:05 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:48:05.269+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:32.539643+0000 front 2024-03-28T03:47:32.539558+0000 (oldest deadline 2024-03-28T03:47:58.439707+0000) 2024-03-28T03:48:05.698 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:05 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:05.402+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:18.649888+0000 front 2024-03-28T03:47:18.649830+0000 (oldest deadline 2024-03-28T03:47:42.749644+0000) 2024-03-28T03:48:05.699 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:05 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:05.402+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:29.151297+0000 front 2024-03-28T03:47:29.151274+0000 (oldest deadline 2024-03-28T03:47:50.152219+0000) 2024-03-28T03:48:05.699 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:05 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:05.402+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:34.853462+0000 front 2024-03-28T03:47:34.853609+0000 (oldest deadline 2024-03-28T03:47:58.353565+0000) 2024-03-28T03:48:06.151 INFO:journalctl@ceph.osd.6.smithi119.stdout:Mar 28 03:48:05 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-6[57000]: 2024-03-28T03:48:05.654+0000 7f9273e00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-28T03:48:06.152 INFO:journalctl@ceph.osd.6.smithi119.stdout:Mar 28 03:48:05 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-6[57000]: 2024-03-28T03:48:05.654+0000 7f9273e00700 -1 osd.6 128 *** Got signal Terminated *** 2024-03-28T03:48:06.152 INFO:journalctl@ceph.osd.6.smithi119.stdout:Mar 28 03:48:05 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-6[57000]: 2024-03-28T03:48:05.654+0000 7f9273e00700 -1 osd.6 128 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-28T03:48:06.152 INFO:journalctl@ceph.osd.6.smithi119.stdout:Mar 28 03:48:05 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-6[57000]: 2024-03-28T03:48:05.793+0000 7f926a600700 -1 osd.6 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:18.788278+0000 front 2024-03-28T03:47:18.788347+0000 (oldest deadline 2024-03-28T03:47:44.088340+0000) 2024-03-28T03:48:06.152 INFO:journalctl@ceph.osd.6.smithi119.stdout:Mar 28 03:48:05 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-6[57000]: 2024-03-28T03:48:05.793+0000 7f926a600700 -1 osd.6 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:28.691333+0000 front 2024-03-28T03:47:28.691283+0000 (oldest deadline 2024-03-28T03:47:49.791440+0000) 2024-03-28T03:48:06.152 INFO:journalctl@ceph.osd.6.smithi119.stdout:Mar 28 03:48:05 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-6[57000]: 2024-03-28T03:48:05.793+0000 7f926a600700 -1 osd.6 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:35.595041+0000 front 2024-03-28T03:47:35.594722+0000 (oldest deadline 2024-03-28T03:48:00.294699+0000) 2024-03-28T03:48:06.373 INFO:journalctl@ceph.osd.7.smithi132.stdout:Mar 28 03:48:05 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-7[54273]: 2024-03-28T03:48:05.957+0000 7f75fa8b1700 -1 osd.7 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:25.970686+0000 front 2024-03-28T03:47:25.970656+0000 (oldest deadline 2024-03-28T03:47:50.571601+0000) 2024-03-28T03:48:06.373 INFO:journalctl@ceph.osd.7.smithi132.stdout:Mar 28 03:48:05 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-7[54273]: 2024-03-28T03:48:05.957+0000 7f75fa8b1700 -1 osd.7 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:34.072660+0000 front 2024-03-28T03:47:34.072689+0000 (oldest deadline 2024-03-28T03:47:58.772829+0000) 2024-03-28T03:48:06.628 INFO:journalctl@ceph.osd.8.smithi165.stdout:Mar 28 03:48:06 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-8[55184]: 2024-03-28T03:48:06.305+0000 7fd543c54700 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:19.562620+0000 front 2024-03-28T03:47:19.563027+0000 (oldest deadline 2024-03-28T03:47:46.562998+0000) 2024-03-28T03:48:06.628 INFO:journalctl@ceph.osd.8.smithi165.stdout:Mar 28 03:48:06 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-8[55184]: 2024-03-28T03:48:06.305+0000 7fd543c54700 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:26.563615+0000 front 2024-03-28T03:47:26.563737+0000 (oldest deadline 2024-03-28T03:47:50.063561+0000) 2024-03-28T03:48:06.628 INFO:journalctl@ceph.osd.8.smithi165.stdout:Mar 28 03:48:06 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-8[55184]: 2024-03-28T03:48:06.305+0000 7fd543c54700 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:34.764640+0000 front 2024-03-28T03:47:34.764564+0000 (oldest deadline 2024-03-28T03:47:58.864724+0000) 2024-03-28T03:48:06.629 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:06 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:06.221+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:18.314545+0000 front 2024-03-28T03:47:18.314582+0000 (oldest deadline 2024-03-28T03:47:44.214430+0000) 2024-03-28T03:48:06.629 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:06 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:06.221+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:27.715675+0000 front 2024-03-28T03:47:27.715645+0000 (oldest deadline 2024-03-28T03:47:50.615667+0000) 2024-03-28T03:48:06.629 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:06 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:06.221+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:32.916784+0000 front 2024-03-28T03:47:32.916834+0000 (oldest deadline 2024-03-28T03:47:58.816929+0000) 2024-03-28T03:48:06.651 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:06 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:48:06.279+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:19.734645+0000 front 2024-03-28T03:47:19.734461+0000 (oldest deadline 2024-03-28T03:47:44.434325+0000) 2024-03-28T03:48:06.651 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:06 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:48:06.279+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:25.537050+0000 front 2024-03-28T03:47:25.536933+0000 (oldest deadline 2024-03-28T03:47:51.437142+0000) 2024-03-28T03:48:06.651 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:06 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:48:06.279+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:32.539643+0000 front 2024-03-28T03:47:32.539558+0000 (oldest deadline 2024-03-28T03:47:58.439707+0000) 2024-03-28T03:48:06.698 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:06 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:06.371+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:18.649888+0000 front 2024-03-28T03:47:18.649830+0000 (oldest deadline 2024-03-28T03:47:42.749644+0000) 2024-03-28T03:48:06.698 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:06 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:06.371+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:29.151297+0000 front 2024-03-28T03:47:29.151274+0000 (oldest deadline 2024-03-28T03:47:50.152219+0000) 2024-03-28T03:48:06.699 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:06 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:06.371+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:34.853462+0000 front 2024-03-28T03:47:34.853609+0000 (oldest deadline 2024-03-28T03:47:58.353565+0000) 2024-03-28T03:48:07.151 INFO:journalctl@ceph.osd.6.smithi119.stdout:Mar 28 03:48:06 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-6[57000]: 2024-03-28T03:48:06.835+0000 7f926a600700 -1 osd.6 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:18.788278+0000 front 2024-03-28T03:47:18.788347+0000 (oldest deadline 2024-03-28T03:47:44.088340+0000) 2024-03-28T03:48:07.152 INFO:journalctl@ceph.osd.6.smithi119.stdout:Mar 28 03:48:06 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-6[57000]: 2024-03-28T03:48:06.835+0000 7f926a600700 -1 osd.6 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:28.691333+0000 front 2024-03-28T03:47:28.691283+0000 (oldest deadline 2024-03-28T03:47:49.791440+0000) 2024-03-28T03:48:07.152 INFO:journalctl@ceph.osd.6.smithi119.stdout:Mar 28 03:48:06 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-6[57000]: 2024-03-28T03:48:06.835+0000 7f926a600700 -1 osd.6 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:35.595041+0000 front 2024-03-28T03:47:35.594722+0000 (oldest deadline 2024-03-28T03:48:00.294699+0000) 2024-03-28T03:48:07.152 INFO:journalctl@ceph.osd.6.smithi119.stdout:Mar 28 03:48:06 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-6[57000]: 2024-03-28T03:48:06.835+0000 7f926a600700 -1 osd.6 128 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2024-03-28T03:47:45.597049+0000 front 2024-03-28T03:47:45.596649+0000 (oldest deadline 2024-03-28T03:48:06.696547+0000) 2024-03-28T03:48:07.198 INFO:journalctl@ceph.osd.7.smithi132.stdout:Mar 28 03:48:06 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-7[54273]: 2024-03-28T03:48:06.909+0000 7f75fa8b1700 -1 osd.7 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:25.970686+0000 front 2024-03-28T03:47:25.970656+0000 (oldest deadline 2024-03-28T03:47:50.571601+0000) 2024-03-28T03:48:07.198 INFO:journalctl@ceph.osd.7.smithi132.stdout:Mar 28 03:48:06 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-7[54273]: 2024-03-28T03:48:06.909+0000 7f75fa8b1700 -1 osd.7 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:34.072660+0000 front 2024-03-28T03:47:34.072689+0000 (oldest deadline 2024-03-28T03:47:58.772829+0000) 2024-03-28T03:48:07.628 INFO:journalctl@ceph.osd.8.smithi165.stdout:Mar 28 03:48:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-8[55184]: 2024-03-28T03:48:07.328+0000 7fd543c54700 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:19.562620+0000 front 2024-03-28T03:47:19.563027+0000 (oldest deadline 2024-03-28T03:47:46.562998+0000) 2024-03-28T03:48:07.629 INFO:journalctl@ceph.osd.8.smithi165.stdout:Mar 28 03:48:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-8[55184]: 2024-03-28T03:48:07.328+0000 7fd543c54700 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:26.563615+0000 front 2024-03-28T03:47:26.563737+0000 (oldest deadline 2024-03-28T03:47:50.063561+0000) 2024-03-28T03:48:07.629 INFO:journalctl@ceph.osd.8.smithi165.stdout:Mar 28 03:48:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-8[55184]: 2024-03-28T03:48:07.328+0000 7fd543c54700 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:34.764640+0000 front 2024-03-28T03:47:34.764564+0000 (oldest deadline 2024-03-28T03:47:58.864724+0000) 2024-03-28T03:48:07.629 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:07.217+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:18.314545+0000 front 2024-03-28T03:47:18.314582+0000 (oldest deadline 2024-03-28T03:47:44.214430+0000) 2024-03-28T03:48:07.629 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:07.217+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:27.715675+0000 front 2024-03-28T03:47:27.715645+0000 (oldest deadline 2024-03-28T03:47:50.615667+0000) 2024-03-28T03:48:07.629 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:07 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:07.217+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:32.916784+0000 front 2024-03-28T03:47:32.916834+0000 (oldest deadline 2024-03-28T03:47:58.816929+0000) 2024-03-28T03:48:07.651 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:07 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:48:07.328+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:19.734645+0000 front 2024-03-28T03:47:19.734461+0000 (oldest deadline 2024-03-28T03:47:44.434325+0000) 2024-03-28T03:48:07.651 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:07 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:48:07.328+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:25.537050+0000 front 2024-03-28T03:47:25.536933+0000 (oldest deadline 2024-03-28T03:47:51.437142+0000) 2024-03-28T03:48:07.651 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:07 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:48:07.328+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:32.539643+0000 front 2024-03-28T03:47:32.539558+0000 (oldest deadline 2024-03-28T03:47:58.439707+0000) 2024-03-28T03:48:07.698 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:07 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:07.360+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:18.649888+0000 front 2024-03-28T03:47:18.649830+0000 (oldest deadline 2024-03-28T03:47:42.749644+0000) 2024-03-28T03:48:07.698 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:07 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:07.360+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:29.151297+0000 front 2024-03-28T03:47:29.151274+0000 (oldest deadline 2024-03-28T03:47:50.152219+0000) 2024-03-28T03:48:07.698 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:07 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:07.360+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:34.853462+0000 front 2024-03-28T03:47:34.853609+0000 (oldest deadline 2024-03-28T03:47:58.353565+0000) 2024-03-28T03:48:08.151 INFO:journalctl@ceph.osd.6.smithi119.stdout:Mar 28 03:48:07 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-6[57000]: 2024-03-28T03:48:07.786+0000 7f926a600700 -1 osd.6 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:18.788278+0000 front 2024-03-28T03:47:18.788347+0000 (oldest deadline 2024-03-28T03:47:44.088340+0000) 2024-03-28T03:48:08.151 INFO:journalctl@ceph.osd.6.smithi119.stdout:Mar 28 03:48:07 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-6[57000]: 2024-03-28T03:48:07.786+0000 7f926a600700 -1 osd.6 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:28.691333+0000 front 2024-03-28T03:47:28.691283+0000 (oldest deadline 2024-03-28T03:47:49.791440+0000) 2024-03-28T03:48:08.152 INFO:journalctl@ceph.osd.6.smithi119.stdout:Mar 28 03:48:07 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-6[57000]: 2024-03-28T03:48:07.786+0000 7f926a600700 -1 osd.6 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:35.595041+0000 front 2024-03-28T03:47:35.594722+0000 (oldest deadline 2024-03-28T03:48:00.294699+0000) 2024-03-28T03:48:08.152 INFO:journalctl@ceph.osd.6.smithi119.stdout:Mar 28 03:48:07 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-6[57000]: 2024-03-28T03:48:07.786+0000 7f926a600700 -1 osd.6 128 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2024-03-28T03:47:45.597049+0000 front 2024-03-28T03:47:45.596649+0000 (oldest deadline 2024-03-28T03:48:06.696547+0000) 2024-03-28T03:48:08.198 INFO:journalctl@ceph.osd.7.smithi132.stdout:Mar 28 03:48:07 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-7[54273]: 2024-03-28T03:48:07.926+0000 7f75fa8b1700 -1 osd.7 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:25.970686+0000 front 2024-03-28T03:47:25.970656+0000 (oldest deadline 2024-03-28T03:47:50.571601+0000) 2024-03-28T03:48:08.198 INFO:journalctl@ceph.osd.7.smithi132.stdout:Mar 28 03:48:07 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-7[54273]: 2024-03-28T03:48:07.926+0000 7f75fa8b1700 -1 osd.7 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:34.072660+0000 front 2024-03-28T03:47:34.072689+0000 (oldest deadline 2024-03-28T03:47:58.772829+0000) 2024-03-28T03:48:08.529 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:08.267+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:18.314545+0000 front 2024-03-28T03:47:18.314582+0000 (oldest deadline 2024-03-28T03:47:44.214430+0000) 2024-03-28T03:48:08.530 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:08.267+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:27.715675+0000 front 2024-03-28T03:47:27.715645+0000 (oldest deadline 2024-03-28T03:47:50.615667+0000) 2024-03-28T03:48:08.530 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:08.267+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:32.916784+0000 front 2024-03-28T03:47:32.916834+0000 (oldest deadline 2024-03-28T03:47:58.816929+0000) 2024-03-28T03:48:08.530 INFO:journalctl@ceph.osd.8.smithi165.stdout:Mar 28 03:48:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-8[55184]: 2024-03-28T03:48:08.296+0000 7fd543c54700 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:19.562620+0000 front 2024-03-28T03:47:19.563027+0000 (oldest deadline 2024-03-28T03:47:46.562998+0000) 2024-03-28T03:48:08.530 INFO:journalctl@ceph.osd.8.smithi165.stdout:Mar 28 03:48:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-8[55184]: 2024-03-28T03:48:08.296+0000 7fd543c54700 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:26.563615+0000 front 2024-03-28T03:47:26.563737+0000 (oldest deadline 2024-03-28T03:47:50.063561+0000) 2024-03-28T03:48:08.530 INFO:journalctl@ceph.osd.8.smithi165.stdout:Mar 28 03:48:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-8[55184]: 2024-03-28T03:48:08.296+0000 7fd543c54700 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:34.764640+0000 front 2024-03-28T03:47:34.764564+0000 (oldest deadline 2024-03-28T03:47:58.864724+0000) 2024-03-28T03:48:08.530 INFO:journalctl@ceph.osd.8.smithi165.stdout:Mar 28 03:48:08 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-8[55184]: 2024-03-28T03:48:08.296+0000 7fd543c54700 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2024-03-28T03:47:40.466520+0000 front 2024-03-28T03:47:40.466406+0000 (oldest deadline 2024-03-28T03:48:07.466815+0000) 2024-03-28T03:48:08.651 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:08 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:48:08.314+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:19.734645+0000 front 2024-03-28T03:47:19.734461+0000 (oldest deadline 2024-03-28T03:47:44.434325+0000) 2024-03-28T03:48:08.651 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:08 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:48:08.314+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:25.537050+0000 front 2024-03-28T03:47:25.536933+0000 (oldest deadline 2024-03-28T03:47:51.437142+0000) 2024-03-28T03:48:08.651 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:08 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:48:08.314+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:32.539643+0000 front 2024-03-28T03:47:32.539558+0000 (oldest deadline 2024-03-28T03:47:58.439707+0000) 2024-03-28T03:48:08.698 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:08 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:08.397+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:18.649888+0000 front 2024-03-28T03:47:18.649830+0000 (oldest deadline 2024-03-28T03:47:42.749644+0000) 2024-03-28T03:48:08.698 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:08 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:08.397+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:29.151297+0000 front 2024-03-28T03:47:29.151274+0000 (oldest deadline 2024-03-28T03:47:50.152219+0000) 2024-03-28T03:48:08.699 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:08 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:08.397+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:34.853462+0000 front 2024-03-28T03:47:34.853609+0000 (oldest deadline 2024-03-28T03:47:58.353565+0000) 2024-03-28T03:48:09.152 INFO:journalctl@ceph.osd.6.smithi119.stdout:Mar 28 03:48:08 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-6[57000]: 2024-03-28T03:48:08.813+0000 7f926a600700 -1 osd.6 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:18.788278+0000 front 2024-03-28T03:47:18.788347+0000 (oldest deadline 2024-03-28T03:47:44.088340+0000) 2024-03-28T03:48:09.152 INFO:journalctl@ceph.osd.6.smithi119.stdout:Mar 28 03:48:08 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-6[57000]: 2024-03-28T03:48:08.813+0000 7f926a600700 -1 osd.6 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:28.691333+0000 front 2024-03-28T03:47:28.691283+0000 (oldest deadline 2024-03-28T03:47:49.791440+0000) 2024-03-28T03:48:09.152 INFO:journalctl@ceph.osd.6.smithi119.stdout:Mar 28 03:48:08 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-6[57000]: 2024-03-28T03:48:08.813+0000 7f926a600700 -1 osd.6 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:35.595041+0000 front 2024-03-28T03:47:35.594722+0000 (oldest deadline 2024-03-28T03:48:00.294699+0000) 2024-03-28T03:48:09.152 INFO:journalctl@ceph.osd.6.smithi119.stdout:Mar 28 03:48:08 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-6[57000]: 2024-03-28T03:48:08.813+0000 7f926a600700 -1 osd.6 128 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2024-03-28T03:47:45.597049+0000 front 2024-03-28T03:47:45.596649+0000 (oldest deadline 2024-03-28T03:48:06.696547+0000) 2024-03-28T03:48:09.379 INFO:journalctl@ceph.osd.7.smithi132.stdout:Mar 28 03:48:08 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-7[54273]: 2024-03-28T03:48:08.953+0000 7f75fa8b1700 -1 osd.7 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:25.970686+0000 front 2024-03-28T03:47:25.970656+0000 (oldest deadline 2024-03-28T03:47:50.571601+0000) 2024-03-28T03:48:09.380 INFO:journalctl@ceph.osd.7.smithi132.stdout:Mar 28 03:48:08 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-7[54273]: 2024-03-28T03:48:08.953+0000 7f75fa8b1700 -1 osd.7 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:34.072660+0000 front 2024-03-28T03:47:34.072689+0000 (oldest deadline 2024-03-28T03:47:58.772829+0000) 2024-03-28T03:48:09.380 INFO:journalctl@ceph.osd.7.smithi132.stdout:Mar 28 03:48:08 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-7[54273]: 2024-03-28T03:48:08.953+0000 7f75fa8b1700 -1 osd.7 128 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2024-03-28T03:47:44.074061+0000 front 2024-03-28T03:47:44.074340+0000 (oldest deadline 2024-03-28T03:48:08.774058+0000) 2024-03-28T03:48:09.546 INFO:journalctl@ceph.osd.8.smithi165.stdout:Mar 28 03:48:09 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-8[55184]: 2024-03-28T03:48:09.329+0000 7fd543c54700 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:19.562620+0000 front 2024-03-28T03:47:19.563027+0000 (oldest deadline 2024-03-28T03:47:46.562998+0000) 2024-03-28T03:48:09.546 INFO:journalctl@ceph.osd.8.smithi165.stdout:Mar 28 03:48:09 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-8[55184]: 2024-03-28T03:48:09.329+0000 7fd543c54700 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:26.563615+0000 front 2024-03-28T03:47:26.563737+0000 (oldest deadline 2024-03-28T03:47:50.063561+0000) 2024-03-28T03:48:09.546 INFO:journalctl@ceph.osd.8.smithi165.stdout:Mar 28 03:48:09 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-8[55184]: 2024-03-28T03:48:09.329+0000 7fd543c54700 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:34.764640+0000 front 2024-03-28T03:47:34.764564+0000 (oldest deadline 2024-03-28T03:47:58.864724+0000) 2024-03-28T03:48:09.547 INFO:journalctl@ceph.osd.8.smithi165.stdout:Mar 28 03:48:09 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-8[55184]: 2024-03-28T03:48:09.329+0000 7fd543c54700 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2024-03-28T03:47:40.466520+0000 front 2024-03-28T03:47:40.466406+0000 (oldest deadline 2024-03-28T03:48:07.466815+0000) 2024-03-28T03:48:09.547 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:09 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:09.280+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:18.314545+0000 front 2024-03-28T03:47:18.314582+0000 (oldest deadline 2024-03-28T03:47:44.214430+0000) 2024-03-28T03:48:09.547 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:09 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:09.280+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:27.715675+0000 front 2024-03-28T03:47:27.715645+0000 (oldest deadline 2024-03-28T03:47:50.615667+0000) 2024-03-28T03:48:09.547 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:09 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:09.280+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:32.916784+0000 front 2024-03-28T03:47:32.916834+0000 (oldest deadline 2024-03-28T03:47:58.816929+0000) 2024-03-28T03:48:09.651 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:09 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:48:09.340+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:19.734645+0000 front 2024-03-28T03:47:19.734461+0000 (oldest deadline 2024-03-28T03:47:44.434325+0000) 2024-03-28T03:48:09.651 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:09 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:48:09.340+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:25.537050+0000 front 2024-03-28T03:47:25.536933+0000 (oldest deadline 2024-03-28T03:47:51.437142+0000) 2024-03-28T03:48:09.652 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:09 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:48:09.340+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:32.539643+0000 front 2024-03-28T03:47:32.539558+0000 (oldest deadline 2024-03-28T03:47:58.439707+0000) 2024-03-28T03:48:09.652 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:09 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:48:09.340+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2024-03-28T03:47:43.141824+0000 front 2024-03-28T03:47:43.142146+0000 (oldest deadline 2024-03-28T03:48:08.441984+0000) 2024-03-28T03:48:09.698 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:09 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:09.378+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:18.649888+0000 front 2024-03-28T03:47:18.649830+0000 (oldest deadline 2024-03-28T03:47:42.749644+0000) 2024-03-28T03:48:09.699 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:09 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:09.378+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:29.151297+0000 front 2024-03-28T03:47:29.151274+0000 (oldest deadline 2024-03-28T03:47:50.152219+0000) 2024-03-28T03:48:09.699 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:09 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:09.378+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:34.853462+0000 front 2024-03-28T03:47:34.853609+0000 (oldest deadline 2024-03-28T03:47:58.353565+0000) 2024-03-28T03:48:10.151 INFO:journalctl@ceph.osd.6.smithi119.stdout:Mar 28 03:48:09 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-6[57000]: 2024-03-28T03:48:09.784+0000 7f926a600700 -1 osd.6 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:18.788278+0000 front 2024-03-28T03:47:18.788347+0000 (oldest deadline 2024-03-28T03:47:44.088340+0000) 2024-03-28T03:48:10.152 INFO:journalctl@ceph.osd.6.smithi119.stdout:Mar 28 03:48:09 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-6[57000]: 2024-03-28T03:48:09.784+0000 7f926a600700 -1 osd.6 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:28.691333+0000 front 2024-03-28T03:47:28.691283+0000 (oldest deadline 2024-03-28T03:47:49.791440+0000) 2024-03-28T03:48:10.152 INFO:journalctl@ceph.osd.6.smithi119.stdout:Mar 28 03:48:09 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-6[57000]: 2024-03-28T03:48:09.784+0000 7f926a600700 -1 osd.6 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:35.595041+0000 front 2024-03-28T03:47:35.594722+0000 (oldest deadline 2024-03-28T03:48:00.294699+0000) 2024-03-28T03:48:10.152 INFO:journalctl@ceph.osd.6.smithi119.stdout:Mar 28 03:48:09 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-6[57000]: 2024-03-28T03:48:09.784+0000 7f926a600700 -1 osd.6 128 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2024-03-28T03:47:45.597049+0000 front 2024-03-28T03:47:45.596649+0000 (oldest deadline 2024-03-28T03:48:06.696547+0000) 2024-03-28T03:48:10.198 INFO:journalctl@ceph.osd.7.smithi132.stdout:Mar 28 03:48:09 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-7[54273]: 2024-03-28T03:48:09.919+0000 7f75fa8b1700 -1 osd.7 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:25.970686+0000 front 2024-03-28T03:47:25.970656+0000 (oldest deadline 2024-03-28T03:47:50.571601+0000) 2024-03-28T03:48:10.198 INFO:journalctl@ceph.osd.7.smithi132.stdout:Mar 28 03:48:09 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-7[54273]: 2024-03-28T03:48:09.919+0000 7f75fa8b1700 -1 osd.7 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:34.072660+0000 front 2024-03-28T03:47:34.072689+0000 (oldest deadline 2024-03-28T03:47:58.772829+0000) 2024-03-28T03:48:10.199 INFO:journalctl@ceph.osd.7.smithi132.stdout:Mar 28 03:48:09 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-7[54273]: 2024-03-28T03:48:09.919+0000 7f75fa8b1700 -1 osd.7 128 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2024-03-28T03:47:44.074061+0000 front 2024-03-28T03:47:44.074340+0000 (oldest deadline 2024-03-28T03:48:08.774058+0000) 2024-03-28T03:48:10.628 INFO:journalctl@ceph.osd.8.smithi165.stdout:Mar 28 03:48:10 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-8[55184]: 2024-03-28T03:48:10.366+0000 7fd543c54700 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:19.562620+0000 front 2024-03-28T03:47:19.563027+0000 (oldest deadline 2024-03-28T03:47:46.562998+0000) 2024-03-28T03:48:10.628 INFO:journalctl@ceph.osd.8.smithi165.stdout:Mar 28 03:48:10 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-8[55184]: 2024-03-28T03:48:10.366+0000 7fd543c54700 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:26.563615+0000 front 2024-03-28T03:47:26.563737+0000 (oldest deadline 2024-03-28T03:47:50.063561+0000) 2024-03-28T03:48:10.628 INFO:journalctl@ceph.osd.8.smithi165.stdout:Mar 28 03:48:10 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-8[55184]: 2024-03-28T03:48:10.366+0000 7fd543c54700 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:34.764640+0000 front 2024-03-28T03:47:34.764564+0000 (oldest deadline 2024-03-28T03:47:58.864724+0000) 2024-03-28T03:48:10.628 INFO:journalctl@ceph.osd.8.smithi165.stdout:Mar 28 03:48:10 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-8[55184]: 2024-03-28T03:48:10.366+0000 7fd543c54700 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2024-03-28T03:47:40.466520+0000 front 2024-03-28T03:47:40.466406+0000 (oldest deadline 2024-03-28T03:48:07.466815+0000) 2024-03-28T03:48:10.629 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:10 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:10.257+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:18.314545+0000 front 2024-03-28T03:47:18.314582+0000 (oldest deadline 2024-03-28T03:47:44.214430+0000) 2024-03-28T03:48:10.629 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:10 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:10.257+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:27.715675+0000 front 2024-03-28T03:47:27.715645+0000 (oldest deadline 2024-03-28T03:47:50.615667+0000) 2024-03-28T03:48:10.629 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:10 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:10.257+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:32.916784+0000 front 2024-03-28T03:47:32.916834+0000 (oldest deadline 2024-03-28T03:47:58.816929+0000) 2024-03-28T03:48:10.651 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:10 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:48:10.360+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:19.734645+0000 front 2024-03-28T03:47:19.734461+0000 (oldest deadline 2024-03-28T03:47:44.434325+0000) 2024-03-28T03:48:10.651 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:10 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:48:10.360+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:25.537050+0000 front 2024-03-28T03:47:25.536933+0000 (oldest deadline 2024-03-28T03:47:51.437142+0000) 2024-03-28T03:48:10.651 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:10 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:48:10.360+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:32.539643+0000 front 2024-03-28T03:47:32.539558+0000 (oldest deadline 2024-03-28T03:47:58.439707+0000) 2024-03-28T03:48:10.651 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:10 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:48:10.360+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2024-03-28T03:47:43.141824+0000 front 2024-03-28T03:47:43.142146+0000 (oldest deadline 2024-03-28T03:48:08.441984+0000) 2024-03-28T03:48:10.698 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:10 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:10.381+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:18.649888+0000 front 2024-03-28T03:47:18.649830+0000 (oldest deadline 2024-03-28T03:47:42.749644+0000) 2024-03-28T03:48:10.698 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:10 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:10.381+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:29.151297+0000 front 2024-03-28T03:47:29.151274+0000 (oldest deadline 2024-03-28T03:47:50.152219+0000) 2024-03-28T03:48:10.698 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:10 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:10.381+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:34.853462+0000 front 2024-03-28T03:47:34.853609+0000 (oldest deadline 2024-03-28T03:47:58.353565+0000) 2024-03-28T03:48:10.699 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:10 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:10.381+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2024-03-28T03:47:44.155643+0000 front 2024-03-28T03:47:44.155546+0000 (oldest deadline 2024-03-28T03:48:09.455316+0000) 2024-03-28T03:48:11.198 INFO:journalctl@ceph.osd.7.smithi132.stdout:Mar 28 03:48:10 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-7[54273]: 2024-03-28T03:48:10.935+0000 7f75fa8b1700 -1 osd.7 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:25.970686+0000 front 2024-03-28T03:47:25.970656+0000 (oldest deadline 2024-03-28T03:47:50.571601+0000) 2024-03-28T03:48:11.199 INFO:journalctl@ceph.osd.7.smithi132.stdout:Mar 28 03:48:10 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-7[54273]: 2024-03-28T03:48:10.935+0000 7f75fa8b1700 -1 osd.7 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:34.072660+0000 front 2024-03-28T03:47:34.072689+0000 (oldest deadline 2024-03-28T03:47:58.772829+0000) 2024-03-28T03:48:11.199 INFO:journalctl@ceph.osd.7.smithi132.stdout:Mar 28 03:48:10 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-7[54273]: 2024-03-28T03:48:10.935+0000 7f75fa8b1700 -1 osd.7 128 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2024-03-28T03:47:44.074061+0000 front 2024-03-28T03:47:44.074340+0000 (oldest deadline 2024-03-28T03:48:08.774058+0000) 2024-03-28T03:48:11.506 INFO:journalctl@ceph.osd.6.smithi119.stdout:Mar 28 03:48:11 smithi119 podman[173416]: 2024-03-28 03:48:11.227387515 +0000 UTC m=+5.720954959 container died ae3bdf70ec38bbf90da5280b71a914ecf2dae4df97185e7ac6eb06cb4b2309fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-6, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-28T03:48:11.506 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:11 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:48:11.400+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:19.734645+0000 front 2024-03-28T03:47:19.734461+0000 (oldest deadline 2024-03-28T03:47:44.434325+0000) 2024-03-28T03:48:11.506 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:11 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:48:11.400+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:25.537050+0000 front 2024-03-28T03:47:25.536933+0000 (oldest deadline 2024-03-28T03:47:51.437142+0000) 2024-03-28T03:48:11.506 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:11 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:48:11.400+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:32.539643+0000 front 2024-03-28T03:47:32.539558+0000 (oldest deadline 2024-03-28T03:47:58.439707+0000) 2024-03-28T03:48:11.506 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:11 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:48:11.400+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2024-03-28T03:47:43.141824+0000 front 2024-03-28T03:47:43.142146+0000 (oldest deadline 2024-03-28T03:48:08.441984+0000) 2024-03-28T03:48:11.628 INFO:journalctl@ceph.osd.8.smithi165.stdout:Mar 28 03:48:11 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-8[55184]: 2024-03-28T03:48:11.336+0000 7fd543c54700 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:19.562620+0000 front 2024-03-28T03:47:19.563027+0000 (oldest deadline 2024-03-28T03:47:46.562998+0000) 2024-03-28T03:48:11.628 INFO:journalctl@ceph.osd.8.smithi165.stdout:Mar 28 03:48:11 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-8[55184]: 2024-03-28T03:48:11.336+0000 7fd543c54700 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:26.563615+0000 front 2024-03-28T03:47:26.563737+0000 (oldest deadline 2024-03-28T03:47:50.063561+0000) 2024-03-28T03:48:11.628 INFO:journalctl@ceph.osd.8.smithi165.stdout:Mar 28 03:48:11 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-8[55184]: 2024-03-28T03:48:11.336+0000 7fd543c54700 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:34.764640+0000 front 2024-03-28T03:47:34.764564+0000 (oldest deadline 2024-03-28T03:47:58.864724+0000) 2024-03-28T03:48:11.628 INFO:journalctl@ceph.osd.8.smithi165.stdout:Mar 28 03:48:11 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-8[55184]: 2024-03-28T03:48:11.336+0000 7fd543c54700 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2024-03-28T03:47:40.466520+0000 front 2024-03-28T03:47:40.466406+0000 (oldest deadline 2024-03-28T03:48:07.466815+0000) 2024-03-28T03:48:11.629 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:11 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:11.236+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:18.314545+0000 front 2024-03-28T03:47:18.314582+0000 (oldest deadline 2024-03-28T03:47:44.214430+0000) 2024-03-28T03:48:11.629 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:11 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:11.236+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:27.715675+0000 front 2024-03-28T03:47:27.715645+0000 (oldest deadline 2024-03-28T03:47:50.615667+0000) 2024-03-28T03:48:11.629 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:11 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:11.236+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:32.916784+0000 front 2024-03-28T03:47:32.916834+0000 (oldest deadline 2024-03-28T03:47:58.816929+0000) 2024-03-28T03:48:11.698 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:11 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:11.339+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:18.649888+0000 front 2024-03-28T03:47:18.649830+0000 (oldest deadline 2024-03-28T03:47:42.749644+0000) 2024-03-28T03:48:11.698 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:11 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:11.339+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:29.151297+0000 front 2024-03-28T03:47:29.151274+0000 (oldest deadline 2024-03-28T03:47:50.152219+0000) 2024-03-28T03:48:11.699 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:11 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:11.339+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:34.853462+0000 front 2024-03-28T03:47:34.853609+0000 (oldest deadline 2024-03-28T03:47:58.353565+0000) 2024-03-28T03:48:11.699 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:11 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:11.339+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2024-03-28T03:47:44.155643+0000 front 2024-03-28T03:47:44.155546+0000 (oldest deadline 2024-03-28T03:48:09.455316+0000) 2024-03-28T03:48:11.816 INFO:journalctl@ceph.osd.6.smithi119.stdout:Mar 28 03:48:11 smithi119 podman[173416]: 2024-03-28 03:48:11.613586648 +0000 UTC m=+6.107154117 container cleanup ae3bdf70ec38bbf90da5280b71a914ecf2dae4df97185e7ac6eb06cb4b2309fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-6, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319) 2024-03-28T03:48:11.817 INFO:journalctl@ceph.osd.6.smithi119.stdout:Mar 28 03:48:11 smithi119 bash[173416]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-6 2024-03-28T03:48:12.151 INFO:journalctl@ceph.osd.6.smithi119.stdout:Mar 28 03:48:11 smithi119 podman[173436]: 2024-03-28 03:48:11.814584444 +0000 UTC m=+0.571639527 container remove ae3bdf70ec38bbf90da5280b71a914ecf2dae4df97185e7ac6eb06cb4b2309fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-6, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-28T03:48:12.199 INFO:journalctl@ceph.osd.7.smithi132.stdout:Mar 28 03:48:11 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-7[54273]: 2024-03-28T03:48:11.896+0000 7f75fa8b1700 -1 osd.7 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:25.970686+0000 front 2024-03-28T03:47:25.970656+0000 (oldest deadline 2024-03-28T03:47:50.571601+0000) 2024-03-28T03:48:12.199 INFO:journalctl@ceph.osd.7.smithi132.stdout:Mar 28 03:48:11 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-7[54273]: 2024-03-28T03:48:11.896+0000 7f75fa8b1700 -1 osd.7 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:34.072660+0000 front 2024-03-28T03:47:34.072689+0000 (oldest deadline 2024-03-28T03:47:58.772829+0000) 2024-03-28T03:48:12.199 INFO:journalctl@ceph.osd.7.smithi132.stdout:Mar 28 03:48:11 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-7[54273]: 2024-03-28T03:48:11.896+0000 7f75fa8b1700 -1 osd.7 128 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2024-03-28T03:47:44.074061+0000 front 2024-03-28T03:47:44.074340+0000 (oldest deadline 2024-03-28T03:48:08.774058+0000) 2024-03-28T03:48:12.466 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:12 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:12.373+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:18.649888+0000 front 2024-03-28T03:47:18.649830+0000 (oldest deadline 2024-03-28T03:47:42.749644+0000) 2024-03-28T03:48:12.466 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:12 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:12.373+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:29.151297+0000 front 2024-03-28T03:47:29.151274+0000 (oldest deadline 2024-03-28T03:47:50.152219+0000) 2024-03-28T03:48:12.466 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:12 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:12.373+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:34.853462+0000 front 2024-03-28T03:47:34.853609+0000 (oldest deadline 2024-03-28T03:47:58.353565+0000) 2024-03-28T03:48:12.466 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:12 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:12.373+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2024-03-28T03:47:44.155643+0000 front 2024-03-28T03:47:44.155546+0000 (oldest deadline 2024-03-28T03:48:09.455316+0000) 2024-03-28T03:48:12.628 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:12.264+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:18.314545+0000 front 2024-03-28T03:47:18.314582+0000 (oldest deadline 2024-03-28T03:47:44.214430+0000) 2024-03-28T03:48:12.628 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:12.264+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:27.715675+0000 front 2024-03-28T03:47:27.715645+0000 (oldest deadline 2024-03-28T03:47:50.615667+0000) 2024-03-28T03:48:12.628 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:12.264+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:32.916784+0000 front 2024-03-28T03:47:32.916834+0000 (oldest deadline 2024-03-28T03:47:58.816929+0000) 2024-03-28T03:48:12.629 INFO:journalctl@ceph.osd.8.smithi165.stdout:Mar 28 03:48:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-8[55184]: 2024-03-28T03:48:12.354+0000 7fd543c54700 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:19.562620+0000 front 2024-03-28T03:47:19.563027+0000 (oldest deadline 2024-03-28T03:47:46.562998+0000) 2024-03-28T03:48:12.629 INFO:journalctl@ceph.osd.8.smithi165.stdout:Mar 28 03:48:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-8[55184]: 2024-03-28T03:48:12.354+0000 7fd543c54700 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:26.563615+0000 front 2024-03-28T03:47:26.563737+0000 (oldest deadline 2024-03-28T03:47:50.063561+0000) 2024-03-28T03:48:12.629 INFO:journalctl@ceph.osd.8.smithi165.stdout:Mar 28 03:48:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-8[55184]: 2024-03-28T03:48:12.354+0000 7fd543c54700 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:34.764640+0000 front 2024-03-28T03:47:34.764564+0000 (oldest deadline 2024-03-28T03:47:58.864724+0000) 2024-03-28T03:48:12.629 INFO:journalctl@ceph.osd.8.smithi165.stdout:Mar 28 03:48:12 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-8[55184]: 2024-03-28T03:48:12.354+0000 7fd543c54700 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2024-03-28T03:47:40.466520+0000 front 2024-03-28T03:47:40.466406+0000 (oldest deadline 2024-03-28T03:48:07.466815+0000) 2024-03-28T03:48:12.651 INFO:journalctl@ceph.osd.6.smithi119.stdout:Mar 28 03:48:12 smithi119 podman[173527]: 2024-03-28 03:48:12.46702403 +0000 UTC m=+0.033924372 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-28T03:48:12.651 INFO:journalctl@ceph.osd.6.smithi119.stdout:Mar 28 03:48:12 smithi119 podman[173527]: 2024-03-28 03:48:12.590384862 +0000 UTC m=+0.157285257 container create fbeb266377cd6477c282788fd20d346cec7d5c7e57962e501fa2b2680d5206d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-6-deactivate, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True) 2024-03-28T03:48:12.652 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:12 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:48:12.361+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:19.734645+0000 front 2024-03-28T03:47:19.734461+0000 (oldest deadline 2024-03-28T03:47:44.434325+0000) 2024-03-28T03:48:12.652 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:12 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:48:12.361+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:25.537050+0000 front 2024-03-28T03:47:25.536933+0000 (oldest deadline 2024-03-28T03:47:51.437142+0000) 2024-03-28T03:48:12.652 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:12 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:48:12.361+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:32.539643+0000 front 2024-03-28T03:47:32.539558+0000 (oldest deadline 2024-03-28T03:47:58.439707+0000) 2024-03-28T03:48:12.652 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:12 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:48:12.361+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2024-03-28T03:47:43.141824+0000 front 2024-03-28T03:47:43.142146+0000 (oldest deadline 2024-03-28T03:48:08.441984+0000) 2024-03-28T03:48:13.152 INFO:journalctl@ceph.osd.6.smithi119.stdout:Mar 28 03:48:12 smithi119 podman[173527]: 2024-03-28 03:48:12.868937375 +0000 UTC m=+0.435837756 container init fbeb266377cd6477c282788fd20d346cec7d5c7e57962e501fa2b2680d5206d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-6-deactivate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-03-28T03:48:13.152 INFO:journalctl@ceph.osd.6.smithi119.stdout:Mar 28 03:48:12 smithi119 podman[173527]: 2024-03-28 03:48:12.879494372 +0000 UTC m=+0.446394754 container start fbeb266377cd6477c282788fd20d346cec7d5c7e57962e501fa2b2680d5206d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-6-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-03-28T03:48:13.152 INFO:journalctl@ceph.osd.6.smithi119.stdout:Mar 28 03:48:12 smithi119 podman[173527]: 2024-03-28 03:48:12.959369079 +0000 UTC m=+0.526269420 container attach fbeb266377cd6477c282788fd20d346cec7d5c7e57962e501fa2b2680d5206d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-6-deactivate, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_BRANCH=HEAD) 2024-03-28T03:48:13.198 INFO:journalctl@ceph.osd.7.smithi132.stdout:Mar 28 03:48:12 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-7[54273]: 2024-03-28T03:48:12.851+0000 7f75fa8b1700 -1 osd.7 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:25.970686+0000 front 2024-03-28T03:47:25.970656+0000 (oldest deadline 2024-03-28T03:47:50.571601+0000) 2024-03-28T03:48:13.198 INFO:journalctl@ceph.osd.7.smithi132.stdout:Mar 28 03:48:12 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-7[54273]: 2024-03-28T03:48:12.851+0000 7f75fa8b1700 -1 osd.7 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:34.072660+0000 front 2024-03-28T03:47:34.072689+0000 (oldest deadline 2024-03-28T03:47:58.772829+0000) 2024-03-28T03:48:13.199 INFO:journalctl@ceph.osd.7.smithi132.stdout:Mar 28 03:48:12 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-7[54273]: 2024-03-28T03:48:12.851+0000 7f75fa8b1700 -1 osd.7 128 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2024-03-28T03:47:44.074061+0000 front 2024-03-28T03:47:44.074340+0000 (oldest deadline 2024-03-28T03:48:08.774058+0000) 2024-03-28T03:48:13.628 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:13 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:13.254+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:18.314545+0000 front 2024-03-28T03:47:18.314582+0000 (oldest deadline 2024-03-28T03:47:44.214430+0000) 2024-03-28T03:48:13.628 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:13 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:13.254+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:27.715675+0000 front 2024-03-28T03:47:27.715645+0000 (oldest deadline 2024-03-28T03:47:50.615667+0000) 2024-03-28T03:48:13.629 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:13 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:13.254+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:32.916784+0000 front 2024-03-28T03:47:32.916834+0000 (oldest deadline 2024-03-28T03:47:58.816929+0000) 2024-03-28T03:48:13.629 INFO:journalctl@ceph.osd.8.smithi165.stdout:Mar 28 03:48:13 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-8[55184]: 2024-03-28T03:48:13.369+0000 7fd543c54700 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:19.562620+0000 front 2024-03-28T03:47:19.563027+0000 (oldest deadline 2024-03-28T03:47:46.562998+0000) 2024-03-28T03:48:13.629 INFO:journalctl@ceph.osd.8.smithi165.stdout:Mar 28 03:48:13 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-8[55184]: 2024-03-28T03:48:13.369+0000 7fd543c54700 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:26.563615+0000 front 2024-03-28T03:47:26.563737+0000 (oldest deadline 2024-03-28T03:47:50.063561+0000) 2024-03-28T03:48:13.629 INFO:journalctl@ceph.osd.8.smithi165.stdout:Mar 28 03:48:13 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-8[55184]: 2024-03-28T03:48:13.369+0000 7fd543c54700 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:34.764640+0000 front 2024-03-28T03:47:34.764564+0000 (oldest deadline 2024-03-28T03:47:58.864724+0000) 2024-03-28T03:48:13.629 INFO:journalctl@ceph.osd.8.smithi165.stdout:Mar 28 03:48:13 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-8[55184]: 2024-03-28T03:48:13.369+0000 7fd543c54700 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2024-03-28T03:47:40.466520+0000 front 2024-03-28T03:47:40.466406+0000 (oldest deadline 2024-03-28T03:48:07.466815+0000) 2024-03-28T03:48:13.651 INFO:journalctl@ceph.osd.6.smithi119.stdout:Mar 28 03:48:13 smithi119 podman[173527]: 2024-03-28 03:48:13.307829295 +0000 UTC m=+0.874729690 container died fbeb266377cd6477c282788fd20d346cec7d5c7e57962e501fa2b2680d5206d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-6-deactivate, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True) 2024-03-28T03:48:13.652 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:13 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:48:13.338+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:19.734645+0000 front 2024-03-28T03:47:19.734461+0000 (oldest deadline 2024-03-28T03:47:44.434325+0000) 2024-03-28T03:48:13.652 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:13 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:48:13.338+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:25.537050+0000 front 2024-03-28T03:47:25.536933+0000 (oldest deadline 2024-03-28T03:47:51.437142+0000) 2024-03-28T03:48:13.652 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:13 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:48:13.338+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:32.539643+0000 front 2024-03-28T03:47:32.539558+0000 (oldest deadline 2024-03-28T03:47:58.439707+0000) 2024-03-28T03:48:13.652 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:13 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:48:13.338+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2024-03-28T03:47:43.141824+0000 front 2024-03-28T03:47:43.142146+0000 (oldest deadline 2024-03-28T03:48:08.441984+0000) 2024-03-28T03:48:13.698 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:13 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:13.337+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:18.649888+0000 front 2024-03-28T03:47:18.649830+0000 (oldest deadline 2024-03-28T03:47:42.749644+0000) 2024-03-28T03:48:13.698 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:13 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:13.337+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:29.151297+0000 front 2024-03-28T03:47:29.151274+0000 (oldest deadline 2024-03-28T03:47:50.152219+0000) 2024-03-28T03:48:13.698 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:13 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:13.337+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:34.853462+0000 front 2024-03-28T03:47:34.853609+0000 (oldest deadline 2024-03-28T03:47:58.353565+0000) 2024-03-28T03:48:13.699 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:13 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:13.337+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2024-03-28T03:47:44.155643+0000 front 2024-03-28T03:47:44.155546+0000 (oldest deadline 2024-03-28T03:48:09.455316+0000) 2024-03-28T03:48:14.073 DEBUG:teuthology.orchestra.run.smithi119:> sudo pkill -f 'journalctl -f -n 0 -u ceph-10f73b92-ec98-11ee-95d0-87774f69a715@osd.6.service' 2024-03-28T03:48:14.197 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-28T03:48:14.198 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-03-28T03:48:14.198 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-03-28T03:48:14.198 DEBUG:teuthology.orchestra.run.smithi132:> sudo systemctl stop ceph-10f73b92-ec98-11ee-95d0-87774f69a715@osd.7 2024-03-28T03:48:14.199 INFO:journalctl@ceph.osd.7.smithi132.stdout:Mar 28 03:48:13 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-7[54273]: 2024-03-28T03:48:13.897+0000 7f75fa8b1700 -1 osd.7 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:25.970686+0000 front 2024-03-28T03:47:25.970656+0000 (oldest deadline 2024-03-28T03:47:50.571601+0000) 2024-03-28T03:48:14.199 INFO:journalctl@ceph.osd.7.smithi132.stdout:Mar 28 03:48:13 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-7[54273]: 2024-03-28T03:48:13.897+0000 7f75fa8b1700 -1 osd.7 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:34.072660+0000 front 2024-03-28T03:47:34.072689+0000 (oldest deadline 2024-03-28T03:47:58.772829+0000) 2024-03-28T03:48:14.199 INFO:journalctl@ceph.osd.7.smithi132.stdout:Mar 28 03:48:13 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-7[54273]: 2024-03-28T03:48:13.897+0000 7f75fa8b1700 -1 osd.7 128 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2024-03-28T03:47:44.074061+0000 front 2024-03-28T03:47:44.074340+0000 (oldest deadline 2024-03-28T03:48:08.774058+0000) 2024-03-28T03:48:14.497 INFO:journalctl@ceph.osd.7.smithi132.stdout:Mar 28 03:48:14 smithi132 systemd[1]: Stopping Ceph osd.7 for 10f73b92-ec98-11ee-95d0-87774f69a715... 2024-03-28T03:48:14.498 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:14 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:14.380+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:18.649888+0000 front 2024-03-28T03:47:18.649830+0000 (oldest deadline 2024-03-28T03:47:42.749644+0000) 2024-03-28T03:48:14.498 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:14 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:14.380+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:29.151297+0000 front 2024-03-28T03:47:29.151274+0000 (oldest deadline 2024-03-28T03:47:50.152219+0000) 2024-03-28T03:48:14.498 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:14 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:14.380+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:34.853462+0000 front 2024-03-28T03:47:34.853609+0000 (oldest deadline 2024-03-28T03:47:58.353565+0000) 2024-03-28T03:48:14.498 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:14 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:14.380+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2024-03-28T03:47:44.155643+0000 front 2024-03-28T03:47:44.155546+0000 (oldest deadline 2024-03-28T03:48:09.455316+0000) 2024-03-28T03:48:14.628 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:14.293+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:18.314545+0000 front 2024-03-28T03:47:18.314582+0000 (oldest deadline 2024-03-28T03:47:44.214430+0000) 2024-03-28T03:48:14.628 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:14.293+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:27.715675+0000 front 2024-03-28T03:47:27.715645+0000 (oldest deadline 2024-03-28T03:47:50.615667+0000) 2024-03-28T03:48:14.628 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:14.293+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:32.916784+0000 front 2024-03-28T03:47:32.916834+0000 (oldest deadline 2024-03-28T03:47:58.816929+0000) 2024-03-28T03:48:14.629 INFO:journalctl@ceph.osd.8.smithi165.stdout:Mar 28 03:48:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-8[55184]: 2024-03-28T03:48:14.389+0000 7fd543c54700 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:19.562620+0000 front 2024-03-28T03:47:19.563027+0000 (oldest deadline 2024-03-28T03:47:46.562998+0000) 2024-03-28T03:48:14.629 INFO:journalctl@ceph.osd.8.smithi165.stdout:Mar 28 03:48:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-8[55184]: 2024-03-28T03:48:14.389+0000 7fd543c54700 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:26.563615+0000 front 2024-03-28T03:47:26.563737+0000 (oldest deadline 2024-03-28T03:47:50.063561+0000) 2024-03-28T03:48:14.629 INFO:journalctl@ceph.osd.8.smithi165.stdout:Mar 28 03:48:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-8[55184]: 2024-03-28T03:48:14.389+0000 7fd543c54700 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:34.764640+0000 front 2024-03-28T03:47:34.764564+0000 (oldest deadline 2024-03-28T03:47:58.864724+0000) 2024-03-28T03:48:14.629 INFO:journalctl@ceph.osd.8.smithi165.stdout:Mar 28 03:48:14 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-8[55184]: 2024-03-28T03:48:14.389+0000 7fd543c54700 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2024-03-28T03:47:40.466520+0000 front 2024-03-28T03:47:40.466406+0000 (oldest deadline 2024-03-28T03:48:07.466815+0000) 2024-03-28T03:48:14.651 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:14 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:48:14.374+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:19.734645+0000 front 2024-03-28T03:47:19.734461+0000 (oldest deadline 2024-03-28T03:47:44.434325+0000) 2024-03-28T03:48:14.651 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:14 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:48:14.374+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:25.537050+0000 front 2024-03-28T03:47:25.536933+0000 (oldest deadline 2024-03-28T03:47:51.437142+0000) 2024-03-28T03:48:14.651 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:14 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:48:14.374+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:32.539643+0000 front 2024-03-28T03:47:32.539558+0000 (oldest deadline 2024-03-28T03:47:58.439707+0000) 2024-03-28T03:48:14.651 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:14 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:48:14.374+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2024-03-28T03:47:43.141824+0000 front 2024-03-28T03:47:43.142146+0000 (oldest deadline 2024-03-28T03:48:08.441984+0000) 2024-03-28T03:48:14.931 INFO:journalctl@ceph.osd.7.smithi132.stdout:Mar 28 03:48:14 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-7[54273]: 2024-03-28T03:48:14.496+0000 7f76022c5700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-28T03:48:14.931 INFO:journalctl@ceph.osd.7.smithi132.stdout:Mar 28 03:48:14 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-7[54273]: 2024-03-28T03:48:14.496+0000 7f76022c5700 -1 osd.7 128 *** Got signal Terminated *** 2024-03-28T03:48:14.931 INFO:journalctl@ceph.osd.7.smithi132.stdout:Mar 28 03:48:14 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-7[54273]: 2024-03-28T03:48:14.496+0000 7f76022c5700 -1 osd.7 128 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-28T03:48:15.198 INFO:journalctl@ceph.osd.7.smithi132.stdout:Mar 28 03:48:14 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-7[54273]: 2024-03-28T03:48:14.929+0000 7f75fa8b1700 -1 osd.7 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:25.970686+0000 front 2024-03-28T03:47:25.970656+0000 (oldest deadline 2024-03-28T03:47:50.571601+0000) 2024-03-28T03:48:15.199 INFO:journalctl@ceph.osd.7.smithi132.stdout:Mar 28 03:48:14 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-7[54273]: 2024-03-28T03:48:14.929+0000 7f75fa8b1700 -1 osd.7 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:34.072660+0000 front 2024-03-28T03:47:34.072689+0000 (oldest deadline 2024-03-28T03:47:58.772829+0000) 2024-03-28T03:48:15.199 INFO:journalctl@ceph.osd.7.smithi132.stdout:Mar 28 03:48:14 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-7[54273]: 2024-03-28T03:48:14.929+0000 7f75fa8b1700 -1 osd.7 128 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2024-03-28T03:47:44.074061+0000 front 2024-03-28T03:47:44.074340+0000 (oldest deadline 2024-03-28T03:48:08.774058+0000) 2024-03-28T03:48:15.628 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:15 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:15.326+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:18.314545+0000 front 2024-03-28T03:47:18.314582+0000 (oldest deadline 2024-03-28T03:47:44.214430+0000) 2024-03-28T03:48:15.628 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:15 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:15.326+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:27.715675+0000 front 2024-03-28T03:47:27.715645+0000 (oldest deadline 2024-03-28T03:47:50.615667+0000) 2024-03-28T03:48:15.628 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:15 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:15.326+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:32.916784+0000 front 2024-03-28T03:47:32.916834+0000 (oldest deadline 2024-03-28T03:47:58.816929+0000) 2024-03-28T03:48:15.628 INFO:journalctl@ceph.osd.8.smithi165.stdout:Mar 28 03:48:15 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-8[55184]: 2024-03-28T03:48:15.431+0000 7fd543c54700 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:19.562620+0000 front 2024-03-28T03:47:19.563027+0000 (oldest deadline 2024-03-28T03:47:46.562998+0000) 2024-03-28T03:48:15.629 INFO:journalctl@ceph.osd.8.smithi165.stdout:Mar 28 03:48:15 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-8[55184]: 2024-03-28T03:48:15.431+0000 7fd543c54700 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:26.563615+0000 front 2024-03-28T03:47:26.563737+0000 (oldest deadline 2024-03-28T03:47:50.063561+0000) 2024-03-28T03:48:15.629 INFO:journalctl@ceph.osd.8.smithi165.stdout:Mar 28 03:48:15 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-8[55184]: 2024-03-28T03:48:15.431+0000 7fd543c54700 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:34.764640+0000 front 2024-03-28T03:47:34.764564+0000 (oldest deadline 2024-03-28T03:47:58.864724+0000) 2024-03-28T03:48:15.629 INFO:journalctl@ceph.osd.8.smithi165.stdout:Mar 28 03:48:15 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-8[55184]: 2024-03-28T03:48:15.431+0000 7fd543c54700 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2024-03-28T03:47:40.466520+0000 front 2024-03-28T03:47:40.466406+0000 (oldest deadline 2024-03-28T03:48:07.466815+0000) 2024-03-28T03:48:15.698 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:15 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:15.393+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:18.649888+0000 front 2024-03-28T03:47:18.649830+0000 (oldest deadline 2024-03-28T03:47:42.749644+0000) 2024-03-28T03:48:15.699 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:15 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:15.393+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:29.151297+0000 front 2024-03-28T03:47:29.151274+0000 (oldest deadline 2024-03-28T03:47:50.152219+0000) 2024-03-28T03:48:15.699 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:15 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:15.393+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:34.853462+0000 front 2024-03-28T03:47:34.853609+0000 (oldest deadline 2024-03-28T03:47:58.353565+0000) 2024-03-28T03:48:15.699 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:15 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:15.393+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2024-03-28T03:47:44.155643+0000 front 2024-03-28T03:47:44.155546+0000 (oldest deadline 2024-03-28T03:48:09.455316+0000) 2024-03-28T03:48:15.901 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:15 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:48:15.406+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:19.734645+0000 front 2024-03-28T03:47:19.734461+0000 (oldest deadline 2024-03-28T03:47:44.434325+0000) 2024-03-28T03:48:15.901 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:15 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:48:15.406+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:25.537050+0000 front 2024-03-28T03:47:25.536933+0000 (oldest deadline 2024-03-28T03:47:51.437142+0000) 2024-03-28T03:48:15.901 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:15 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:48:15.406+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:32.539643+0000 front 2024-03-28T03:47:32.539558+0000 (oldest deadline 2024-03-28T03:47:58.439707+0000) 2024-03-28T03:48:15.901 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:15 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:48:15.406+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2024-03-28T03:47:43.141824+0000 front 2024-03-28T03:47:43.142146+0000 (oldest deadline 2024-03-28T03:48:08.441984+0000) 2024-03-28T03:48:16.199 INFO:journalctl@ceph.osd.7.smithi132.stdout:Mar 28 03:48:15 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-7[54273]: 2024-03-28T03:48:15.907+0000 7f75fa8b1700 -1 osd.7 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:25.970686+0000 front 2024-03-28T03:47:25.970656+0000 (oldest deadline 2024-03-28T03:47:50.571601+0000) 2024-03-28T03:48:16.199 INFO:journalctl@ceph.osd.7.smithi132.stdout:Mar 28 03:48:15 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-7[54273]: 2024-03-28T03:48:15.907+0000 7f75fa8b1700 -1 osd.7 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:34.072660+0000 front 2024-03-28T03:47:34.072689+0000 (oldest deadline 2024-03-28T03:47:58.772829+0000) 2024-03-28T03:48:16.199 INFO:journalctl@ceph.osd.7.smithi132.stdout:Mar 28 03:48:15 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-7[54273]: 2024-03-28T03:48:15.907+0000 7f75fa8b1700 -1 osd.7 128 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2024-03-28T03:47:44.074061+0000 front 2024-03-28T03:47:44.074340+0000 (oldest deadline 2024-03-28T03:48:08.774058+0000) 2024-03-28T03:48:16.628 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:16 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:16.303+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:18.314545+0000 front 2024-03-28T03:47:18.314582+0000 (oldest deadline 2024-03-28T03:47:44.214430+0000) 2024-03-28T03:48:16.628 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:16 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:16.303+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:27.715675+0000 front 2024-03-28T03:47:27.715645+0000 (oldest deadline 2024-03-28T03:47:50.615667+0000) 2024-03-28T03:48:16.628 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:16 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:16.303+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:32.916784+0000 front 2024-03-28T03:47:32.916834+0000 (oldest deadline 2024-03-28T03:47:58.816929+0000) 2024-03-28T03:48:16.629 INFO:journalctl@ceph.osd.8.smithi165.stdout:Mar 28 03:48:16 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-8[55184]: 2024-03-28T03:48:16.382+0000 7fd543c54700 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:19.562620+0000 front 2024-03-28T03:47:19.563027+0000 (oldest deadline 2024-03-28T03:47:46.562998+0000) 2024-03-28T03:48:16.629 INFO:journalctl@ceph.osd.8.smithi165.stdout:Mar 28 03:48:16 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-8[55184]: 2024-03-28T03:48:16.382+0000 7fd543c54700 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:26.563615+0000 front 2024-03-28T03:47:26.563737+0000 (oldest deadline 2024-03-28T03:47:50.063561+0000) 2024-03-28T03:48:16.629 INFO:journalctl@ceph.osd.8.smithi165.stdout:Mar 28 03:48:16 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-8[55184]: 2024-03-28T03:48:16.382+0000 7fd543c54700 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:34.764640+0000 front 2024-03-28T03:47:34.764564+0000 (oldest deadline 2024-03-28T03:47:58.864724+0000) 2024-03-28T03:48:16.629 INFO:journalctl@ceph.osd.8.smithi165.stdout:Mar 28 03:48:16 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-8[55184]: 2024-03-28T03:48:16.382+0000 7fd543c54700 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2024-03-28T03:47:40.466520+0000 front 2024-03-28T03:47:40.466406+0000 (oldest deadline 2024-03-28T03:48:07.466815+0000) 2024-03-28T03:48:16.651 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:48:16.394+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:19.734645+0000 front 2024-03-28T03:47:19.734461+0000 (oldest deadline 2024-03-28T03:47:44.434325+0000) 2024-03-28T03:48:16.651 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:48:16.394+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:25.537050+0000 front 2024-03-28T03:47:25.536933+0000 (oldest deadline 2024-03-28T03:47:51.437142+0000) 2024-03-28T03:48:16.652 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:48:16.394+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:32.539643+0000 front 2024-03-28T03:47:32.539558+0000 (oldest deadline 2024-03-28T03:47:58.439707+0000) 2024-03-28T03:48:16.652 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:16 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:48:16.394+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2024-03-28T03:47:43.141824+0000 front 2024-03-28T03:47:43.142146+0000 (oldest deadline 2024-03-28T03:48:08.441984+0000) 2024-03-28T03:48:16.698 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:16 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:16.421+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:18.649888+0000 front 2024-03-28T03:47:18.649830+0000 (oldest deadline 2024-03-28T03:47:42.749644+0000) 2024-03-28T03:48:16.698 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:16 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:16.421+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:29.151297+0000 front 2024-03-28T03:47:29.151274+0000 (oldest deadline 2024-03-28T03:47:50.152219+0000) 2024-03-28T03:48:16.699 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:16 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:16.421+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:34.853462+0000 front 2024-03-28T03:47:34.853609+0000 (oldest deadline 2024-03-28T03:47:58.353565+0000) 2024-03-28T03:48:16.699 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:16 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:16.421+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2024-03-28T03:47:44.155643+0000 front 2024-03-28T03:47:44.155546+0000 (oldest deadline 2024-03-28T03:48:09.455316+0000) 2024-03-28T03:48:17.199 INFO:journalctl@ceph.osd.7.smithi132.stdout:Mar 28 03:48:16 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-7[54273]: 2024-03-28T03:48:16.869+0000 7f75fa8b1700 -1 osd.7 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:25.970686+0000 front 2024-03-28T03:47:25.970656+0000 (oldest deadline 2024-03-28T03:47:50.571601+0000) 2024-03-28T03:48:17.199 INFO:journalctl@ceph.osd.7.smithi132.stdout:Mar 28 03:48:16 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-7[54273]: 2024-03-28T03:48:16.869+0000 7f75fa8b1700 -1 osd.7 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:34.072660+0000 front 2024-03-28T03:47:34.072689+0000 (oldest deadline 2024-03-28T03:47:58.772829+0000) 2024-03-28T03:48:17.199 INFO:journalctl@ceph.osd.7.smithi132.stdout:Mar 28 03:48:16 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-7[54273]: 2024-03-28T03:48:16.869+0000 7f75fa8b1700 -1 osd.7 128 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2024-03-28T03:47:44.074061+0000 front 2024-03-28T03:47:44.074340+0000 (oldest deadline 2024-03-28T03:48:08.774058+0000) 2024-03-28T03:48:17.628 INFO:journalctl@ceph.osd.8.smithi165.stdout:Mar 28 03:48:17 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-8[55184]: 2024-03-28T03:48:17.420+0000 7fd543c54700 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:19.562620+0000 front 2024-03-28T03:47:19.563027+0000 (oldest deadline 2024-03-28T03:47:46.562998+0000) 2024-03-28T03:48:17.629 INFO:journalctl@ceph.osd.8.smithi165.stdout:Mar 28 03:48:17 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-8[55184]: 2024-03-28T03:48:17.420+0000 7fd543c54700 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:26.563615+0000 front 2024-03-28T03:47:26.563737+0000 (oldest deadline 2024-03-28T03:47:50.063561+0000) 2024-03-28T03:48:17.629 INFO:journalctl@ceph.osd.8.smithi165.stdout:Mar 28 03:48:17 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-8[55184]: 2024-03-28T03:48:17.420+0000 7fd543c54700 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:34.764640+0000 front 2024-03-28T03:47:34.764564+0000 (oldest deadline 2024-03-28T03:47:58.864724+0000) 2024-03-28T03:48:17.629 INFO:journalctl@ceph.osd.8.smithi165.stdout:Mar 28 03:48:17 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-8[55184]: 2024-03-28T03:48:17.420+0000 7fd543c54700 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2024-03-28T03:47:40.466520+0000 front 2024-03-28T03:47:40.466406+0000 (oldest deadline 2024-03-28T03:48:07.466815+0000) 2024-03-28T03:48:17.629 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:17 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:17.254+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:18.314545+0000 front 2024-03-28T03:47:18.314582+0000 (oldest deadline 2024-03-28T03:47:44.214430+0000) 2024-03-28T03:48:17.629 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:17 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:17.254+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:27.715675+0000 front 2024-03-28T03:47:27.715645+0000 (oldest deadline 2024-03-28T03:47:50.615667+0000) 2024-03-28T03:48:17.629 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:17 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:17.254+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:32.916784+0000 front 2024-03-28T03:47:32.916834+0000 (oldest deadline 2024-03-28T03:47:58.816929+0000) 2024-03-28T03:48:17.629 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:17 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:17.254+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2024-03-28T03:47:52.919580+0000 front 2024-03-28T03:47:52.919549+0000 (oldest deadline 2024-03-28T03:48:17.019575+0000) 2024-03-28T03:48:17.899 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:17 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:17.470+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:18.649888+0000 front 2024-03-28T03:47:18.649830+0000 (oldest deadline 2024-03-28T03:47:42.749644+0000) 2024-03-28T03:48:17.899 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:17 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:17.470+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:29.151297+0000 front 2024-03-28T03:47:29.151274+0000 (oldest deadline 2024-03-28T03:47:50.152219+0000) 2024-03-28T03:48:17.899 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:17 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:17.470+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:34.853462+0000 front 2024-03-28T03:47:34.853609+0000 (oldest deadline 2024-03-28T03:47:58.353565+0000) 2024-03-28T03:48:17.899 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:17 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:17.470+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2024-03-28T03:47:44.155643+0000 front 2024-03-28T03:47:44.155546+0000 (oldest deadline 2024-03-28T03:48:09.455316+0000) 2024-03-28T03:48:17.899 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:17 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:17.470+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2024-03-28T03:47:52.356476+0000 front 2024-03-28T03:47:52.356457+0000 (oldest deadline 2024-03-28T03:48:17.056580+0000) 2024-03-28T03:48:17.901 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:17 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:48:17.429+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:19.734645+0000 front 2024-03-28T03:47:19.734461+0000 (oldest deadline 2024-03-28T03:47:44.434325+0000) 2024-03-28T03:48:17.901 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:17 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:48:17.429+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:25.537050+0000 front 2024-03-28T03:47:25.536933+0000 (oldest deadline 2024-03-28T03:47:51.437142+0000) 2024-03-28T03:48:17.901 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:17 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:48:17.429+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:32.539643+0000 front 2024-03-28T03:47:32.539558+0000 (oldest deadline 2024-03-28T03:47:58.439707+0000) 2024-03-28T03:48:17.902 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:17 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:48:17.429+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2024-03-28T03:47:43.141824+0000 front 2024-03-28T03:47:43.142146+0000 (oldest deadline 2024-03-28T03:48:08.441984+0000) 2024-03-28T03:48:18.198 INFO:journalctl@ceph.osd.7.smithi132.stdout:Mar 28 03:48:17 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-7[54273]: 2024-03-28T03:48:17.897+0000 7f75fa8b1700 -1 osd.7 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:25.970686+0000 front 2024-03-28T03:47:25.970656+0000 (oldest deadline 2024-03-28T03:47:50.571601+0000) 2024-03-28T03:48:18.199 INFO:journalctl@ceph.osd.7.smithi132.stdout:Mar 28 03:48:17 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-7[54273]: 2024-03-28T03:48:17.897+0000 7f75fa8b1700 -1 osd.7 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:34.072660+0000 front 2024-03-28T03:47:34.072689+0000 (oldest deadline 2024-03-28T03:47:58.772829+0000) 2024-03-28T03:48:18.199 INFO:journalctl@ceph.osd.7.smithi132.stdout:Mar 28 03:48:17 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-7[54273]: 2024-03-28T03:48:17.897+0000 7f75fa8b1700 -1 osd.7 128 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2024-03-28T03:47:44.074061+0000 front 2024-03-28T03:47:44.074340+0000 (oldest deadline 2024-03-28T03:48:08.774058+0000) 2024-03-28T03:48:18.628 INFO:journalctl@ceph.osd.8.smithi165.stdout:Mar 28 03:48:18 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-8[55184]: 2024-03-28T03:48:18.455+0000 7fd543c54700 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:19.562620+0000 front 2024-03-28T03:47:19.563027+0000 (oldest deadline 2024-03-28T03:47:46.562998+0000) 2024-03-28T03:48:18.628 INFO:journalctl@ceph.osd.8.smithi165.stdout:Mar 28 03:48:18 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-8[55184]: 2024-03-28T03:48:18.455+0000 7fd543c54700 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:26.563615+0000 front 2024-03-28T03:47:26.563737+0000 (oldest deadline 2024-03-28T03:47:50.063561+0000) 2024-03-28T03:48:18.628 INFO:journalctl@ceph.osd.8.smithi165.stdout:Mar 28 03:48:18 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-8[55184]: 2024-03-28T03:48:18.455+0000 7fd543c54700 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:34.764640+0000 front 2024-03-28T03:47:34.764564+0000 (oldest deadline 2024-03-28T03:47:58.864724+0000) 2024-03-28T03:48:18.629 INFO:journalctl@ceph.osd.8.smithi165.stdout:Mar 28 03:48:18 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-8[55184]: 2024-03-28T03:48:18.455+0000 7fd543c54700 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2024-03-28T03:47:40.466520+0000 front 2024-03-28T03:47:40.466406+0000 (oldest deadline 2024-03-28T03:48:07.466815+0000) 2024-03-28T03:48:18.629 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:18 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:18.225+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:18.314545+0000 front 2024-03-28T03:47:18.314582+0000 (oldest deadline 2024-03-28T03:47:44.214430+0000) 2024-03-28T03:48:18.629 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:18 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:18.225+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:27.715675+0000 front 2024-03-28T03:47:27.715645+0000 (oldest deadline 2024-03-28T03:47:50.615667+0000) 2024-03-28T03:48:18.629 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:18 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:18.225+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:32.916784+0000 front 2024-03-28T03:47:32.916834+0000 (oldest deadline 2024-03-28T03:47:58.816929+0000) 2024-03-28T03:48:18.629 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:18 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:18.225+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2024-03-28T03:47:52.919580+0000 front 2024-03-28T03:47:52.919549+0000 (oldest deadline 2024-03-28T03:48:17.019575+0000) 2024-03-28T03:48:18.863 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:18 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:18.494+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:18.649888+0000 front 2024-03-28T03:47:18.649830+0000 (oldest deadline 2024-03-28T03:47:42.749644+0000) 2024-03-28T03:48:18.863 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:18 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:18.494+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:29.151297+0000 front 2024-03-28T03:47:29.151274+0000 (oldest deadline 2024-03-28T03:47:50.152219+0000) 2024-03-28T03:48:18.863 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:18 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:18.494+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:34.853462+0000 front 2024-03-28T03:47:34.853609+0000 (oldest deadline 2024-03-28T03:47:58.353565+0000) 2024-03-28T03:48:18.864 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:18 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:18.494+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2024-03-28T03:47:44.155643+0000 front 2024-03-28T03:47:44.155546+0000 (oldest deadline 2024-03-28T03:48:09.455316+0000) 2024-03-28T03:48:18.864 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:18 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:18.494+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2024-03-28T03:47:52.356476+0000 front 2024-03-28T03:47:52.356457+0000 (oldest deadline 2024-03-28T03:48:17.056580+0000) 2024-03-28T03:48:18.901 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:18 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:48:18.450+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:19.734645+0000 front 2024-03-28T03:47:19.734461+0000 (oldest deadline 2024-03-28T03:47:44.434325+0000) 2024-03-28T03:48:18.902 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:18 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:48:18.450+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:25.537050+0000 front 2024-03-28T03:47:25.536933+0000 (oldest deadline 2024-03-28T03:47:51.437142+0000) 2024-03-28T03:48:18.902 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:18 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:48:18.450+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:32.539643+0000 front 2024-03-28T03:47:32.539558+0000 (oldest deadline 2024-03-28T03:47:58.439707+0000) 2024-03-28T03:48:18.902 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:18 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:48:18.450+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2024-03-28T03:47:43.141824+0000 front 2024-03-28T03:47:43.142146+0000 (oldest deadline 2024-03-28T03:48:08.441984+0000) 2024-03-28T03:48:19.199 INFO:journalctl@ceph.osd.7.smithi132.stdout:Mar 28 03:48:18 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-7[54273]: 2024-03-28T03:48:18.861+0000 7f75fa8b1700 -1 osd.7 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:25.970686+0000 front 2024-03-28T03:47:25.970656+0000 (oldest deadline 2024-03-28T03:47:50.571601+0000) 2024-03-28T03:48:19.199 INFO:journalctl@ceph.osd.7.smithi132.stdout:Mar 28 03:48:18 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-7[54273]: 2024-03-28T03:48:18.861+0000 7f75fa8b1700 -1 osd.7 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:34.072660+0000 front 2024-03-28T03:47:34.072689+0000 (oldest deadline 2024-03-28T03:47:58.772829+0000) 2024-03-28T03:48:19.199 INFO:journalctl@ceph.osd.7.smithi132.stdout:Mar 28 03:48:18 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-7[54273]: 2024-03-28T03:48:18.861+0000 7f75fa8b1700 -1 osd.7 128 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2024-03-28T03:47:44.074061+0000 front 2024-03-28T03:47:44.074340+0000 (oldest deadline 2024-03-28T03:48:08.774058+0000) 2024-03-28T03:48:19.199 INFO:journalctl@ceph.osd.7.smithi132.stdout:Mar 28 03:48:18 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-7[54273]: 2024-03-28T03:48:18.861+0000 7f75fa8b1700 -1 osd.7 128 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2024-03-28T03:47:54.575921+0000 front 2024-03-28T03:47:54.576045+0000 (oldest deadline 2024-03-28T03:48:18.676103+0000) 2024-03-28T03:48:19.628 INFO:journalctl@ceph.osd.8.smithi165.stdout:Mar 28 03:48:19 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-8[55184]: 2024-03-28T03:48:19.423+0000 7fd543c54700 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:19.562620+0000 front 2024-03-28T03:47:19.563027+0000 (oldest deadline 2024-03-28T03:47:46.562998+0000) 2024-03-28T03:48:19.628 INFO:journalctl@ceph.osd.8.smithi165.stdout:Mar 28 03:48:19 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-8[55184]: 2024-03-28T03:48:19.423+0000 7fd543c54700 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:26.563615+0000 front 2024-03-28T03:47:26.563737+0000 (oldest deadline 2024-03-28T03:47:50.063561+0000) 2024-03-28T03:48:19.628 INFO:journalctl@ceph.osd.8.smithi165.stdout:Mar 28 03:48:19 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-8[55184]: 2024-03-28T03:48:19.423+0000 7fd543c54700 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:34.764640+0000 front 2024-03-28T03:47:34.764564+0000 (oldest deadline 2024-03-28T03:47:58.864724+0000) 2024-03-28T03:48:19.629 INFO:journalctl@ceph.osd.8.smithi165.stdout:Mar 28 03:48:19 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-8[55184]: 2024-03-28T03:48:19.423+0000 7fd543c54700 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2024-03-28T03:47:40.466520+0000 front 2024-03-28T03:47:40.466406+0000 (oldest deadline 2024-03-28T03:48:07.466815+0000) 2024-03-28T03:48:19.629 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:19 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:19.232+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:18.314545+0000 front 2024-03-28T03:47:18.314582+0000 (oldest deadline 2024-03-28T03:47:44.214430+0000) 2024-03-28T03:48:19.629 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:19 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:19.232+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:27.715675+0000 front 2024-03-28T03:47:27.715645+0000 (oldest deadline 2024-03-28T03:47:50.615667+0000) 2024-03-28T03:48:19.629 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:19 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:19.232+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:32.916784+0000 front 2024-03-28T03:47:32.916834+0000 (oldest deadline 2024-03-28T03:47:58.816929+0000) 2024-03-28T03:48:19.629 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:19 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:19.232+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2024-03-28T03:47:52.919580+0000 front 2024-03-28T03:47:52.919549+0000 (oldest deadline 2024-03-28T03:48:17.019575+0000) 2024-03-28T03:48:19.901 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:19 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:48:19.421+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:19.734645+0000 front 2024-03-28T03:47:19.734461+0000 (oldest deadline 2024-03-28T03:47:44.434325+0000) 2024-03-28T03:48:19.901 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:19 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:48:19.421+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:25.537050+0000 front 2024-03-28T03:47:25.536933+0000 (oldest deadline 2024-03-28T03:47:51.437142+0000) 2024-03-28T03:48:19.902 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:19 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:48:19.421+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:32.539643+0000 front 2024-03-28T03:47:32.539558+0000 (oldest deadline 2024-03-28T03:47:58.439707+0000) 2024-03-28T03:48:19.902 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:19 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:48:19.421+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2024-03-28T03:47:43.141824+0000 front 2024-03-28T03:47:43.142146+0000 (oldest deadline 2024-03-28T03:48:08.441984+0000) 2024-03-28T03:48:19.913 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:19 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:19.514+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:18.649888+0000 front 2024-03-28T03:47:18.649830+0000 (oldest deadline 2024-03-28T03:47:42.749644+0000) 2024-03-28T03:48:19.913 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:19 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:19.514+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:29.151297+0000 front 2024-03-28T03:47:29.151274+0000 (oldest deadline 2024-03-28T03:47:50.152219+0000) 2024-03-28T03:48:19.913 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:19 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:19.514+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:34.853462+0000 front 2024-03-28T03:47:34.853609+0000 (oldest deadline 2024-03-28T03:47:58.353565+0000) 2024-03-28T03:48:19.914 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:19 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:19.514+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2024-03-28T03:47:44.155643+0000 front 2024-03-28T03:47:44.155546+0000 (oldest deadline 2024-03-28T03:48:09.455316+0000) 2024-03-28T03:48:19.914 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:19 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:19.514+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2024-03-28T03:47:52.356476+0000 front 2024-03-28T03:47:52.356457+0000 (oldest deadline 2024-03-28T03:48:17.056580+0000) 2024-03-28T03:48:20.198 INFO:journalctl@ceph.osd.7.smithi132.stdout:Mar 28 03:48:19 smithi132 podman[120965]: 2024-03-28 03:48:19.912545131 +0000 UTC m=+5.527888109 container died 0e1b3834426e6411cd75a36d734c13a1e15d05a57f92e7916b7db6cf2e877be8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-7, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-03-28T03:48:20.513 INFO:journalctl@ceph.osd.7.smithi132.stdout:Mar 28 03:48:20 smithi132 podman[120965]: 2024-03-28 03:48:20.29592364 +0000 UTC m=+5.911266596 container cleanup 0e1b3834426e6411cd75a36d734c13a1e15d05a57f92e7916b7db6cf2e877be8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-7, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c) 2024-03-28T03:48:20.513 INFO:journalctl@ceph.osd.7.smithi132.stdout:Mar 28 03:48:20 smithi132 bash[120965]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-7 2024-03-28T03:48:20.628 INFO:journalctl@ceph.osd.8.smithi165.stdout:Mar 28 03:48:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-8[55184]: 2024-03-28T03:48:20.463+0000 7fd543c54700 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:19.562620+0000 front 2024-03-28T03:47:19.563027+0000 (oldest deadline 2024-03-28T03:47:46.562998+0000) 2024-03-28T03:48:20.628 INFO:journalctl@ceph.osd.8.smithi165.stdout:Mar 28 03:48:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-8[55184]: 2024-03-28T03:48:20.463+0000 7fd543c54700 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:26.563615+0000 front 2024-03-28T03:47:26.563737+0000 (oldest deadline 2024-03-28T03:47:50.063561+0000) 2024-03-28T03:48:20.628 INFO:journalctl@ceph.osd.8.smithi165.stdout:Mar 28 03:48:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-8[55184]: 2024-03-28T03:48:20.464+0000 7fd543c54700 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:34.764640+0000 front 2024-03-28T03:47:34.764564+0000 (oldest deadline 2024-03-28T03:47:58.864724+0000) 2024-03-28T03:48:20.629 INFO:journalctl@ceph.osd.8.smithi165.stdout:Mar 28 03:48:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-8[55184]: 2024-03-28T03:48:20.464+0000 7fd543c54700 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2024-03-28T03:47:40.466520+0000 front 2024-03-28T03:47:40.466406+0000 (oldest deadline 2024-03-28T03:48:07.466815+0000) 2024-03-28T03:48:20.629 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:20.239+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:18.314545+0000 front 2024-03-28T03:47:18.314582+0000 (oldest deadline 2024-03-28T03:47:44.214430+0000) 2024-03-28T03:48:20.629 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:20.239+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:27.715675+0000 front 2024-03-28T03:47:27.715645+0000 (oldest deadline 2024-03-28T03:47:50.615667+0000) 2024-03-28T03:48:20.629 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:20.239+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:32.916784+0000 front 2024-03-28T03:47:32.916834+0000 (oldest deadline 2024-03-28T03:47:58.816929+0000) 2024-03-28T03:48:20.629 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:20 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:20.239+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2024-03-28T03:47:52.919580+0000 front 2024-03-28T03:47:52.919549+0000 (oldest deadline 2024-03-28T03:48:17.019575+0000) 2024-03-28T03:48:20.901 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:20 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:48:20.461+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:19.734645+0000 front 2024-03-28T03:47:19.734461+0000 (oldest deadline 2024-03-28T03:47:44.434325+0000) 2024-03-28T03:48:20.901 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:20 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:48:20.461+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:25.537050+0000 front 2024-03-28T03:47:25.536933+0000 (oldest deadline 2024-03-28T03:47:51.437142+0000) 2024-03-28T03:48:20.901 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:20 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:48:20.461+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:32.539643+0000 front 2024-03-28T03:47:32.539558+0000 (oldest deadline 2024-03-28T03:47:58.439707+0000) 2024-03-28T03:48:20.901 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:20 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:48:20.461+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2024-03-28T03:47:43.141824+0000 front 2024-03-28T03:47:43.142146+0000 (oldest deadline 2024-03-28T03:48:08.441984+0000) 2024-03-28T03:48:20.910 INFO:journalctl@ceph.osd.7.smithi132.stdout:Mar 28 03:48:20 smithi132 podman[120983]: 2024-03-28 03:48:20.555840753 +0000 UTC m=+0.638615810 container remove 0e1b3834426e6411cd75a36d734c13a1e15d05a57f92e7916b7db6cf2e877be8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True) 2024-03-28T03:48:20.911 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:20 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:20.512+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:18.649888+0000 front 2024-03-28T03:47:18.649830+0000 (oldest deadline 2024-03-28T03:47:42.749644+0000) 2024-03-28T03:48:20.911 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:20 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:20.512+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:29.151297+0000 front 2024-03-28T03:47:29.151274+0000 (oldest deadline 2024-03-28T03:47:50.152219+0000) 2024-03-28T03:48:20.911 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:20 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:20.512+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:34.853462+0000 front 2024-03-28T03:47:34.853609+0000 (oldest deadline 2024-03-28T03:47:58.353565+0000) 2024-03-28T03:48:20.911 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:20 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:20.512+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2024-03-28T03:47:44.155643+0000 front 2024-03-28T03:47:44.155546+0000 (oldest deadline 2024-03-28T03:48:09.455316+0000) 2024-03-28T03:48:20.911 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:20 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:20.512+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2024-03-28T03:47:52.356476+0000 front 2024-03-28T03:47:52.356457+0000 (oldest deadline 2024-03-28T03:48:17.056580+0000) 2024-03-28T03:48:21.169 INFO:journalctl@ceph.osd.7.smithi132.stdout:Mar 28 03:48:20 smithi132 podman[121070]: 2024-03-28 03:48:20.81289657 +0000 UTC m=+0.017567632 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-28T03:48:21.169 INFO:journalctl@ceph.osd.7.smithi132.stdout:Mar 28 03:48:20 smithi132 podman[121070]: 2024-03-28 03:48:20.940157945 +0000 UTC m=+0.144829013 container create 319af517f55d26849123d9274f33faaef7548095889ad83e52af7e9ecf3ac3a8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-7-deactivate, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True) 2024-03-28T03:48:21.169 INFO:journalctl@ceph.osd.7.smithi132.stdout:Mar 28 03:48:21 smithi132 podman[121070]: 2024-03-28 03:48:21.167896857 +0000 UTC m=+0.372567919 container init 319af517f55d26849123d9274f33faaef7548095889ad83e52af7e9ecf3ac3a8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-7-deactivate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.vendor=CentOS) 2024-03-28T03:48:21.448 INFO:journalctl@ceph.osd.7.smithi132.stdout:Mar 28 03:48:21 smithi132 podman[121070]: 2024-03-28 03:48:21.171568125 +0000 UTC m=+0.376239174 container start 319af517f55d26849123d9274f33faaef7548095889ad83e52af7e9ecf3ac3a8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-7-deactivate, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-03-28T03:48:21.449 INFO:journalctl@ceph.osd.7.smithi132.stdout:Mar 28 03:48:21 smithi132 podman[121070]: 2024-03-28 03:48:21.267917722 +0000 UTC m=+0.472588790 container attach 319af517f55d26849123d9274f33faaef7548095889ad83e52af7e9ecf3ac3a8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-7-deactivate, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, ceph=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-03-28T03:48:21.628 INFO:journalctl@ceph.osd.8.smithi165.stdout:Mar 28 03:48:21 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-8[55184]: 2024-03-28T03:48:21.454+0000 7fd543c54700 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:19.562620+0000 front 2024-03-28T03:47:19.563027+0000 (oldest deadline 2024-03-28T03:47:46.562998+0000) 2024-03-28T03:48:21.628 INFO:journalctl@ceph.osd.8.smithi165.stdout:Mar 28 03:48:21 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-8[55184]: 2024-03-28T03:48:21.454+0000 7fd543c54700 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:26.563615+0000 front 2024-03-28T03:47:26.563737+0000 (oldest deadline 2024-03-28T03:47:50.063561+0000) 2024-03-28T03:48:21.628 INFO:journalctl@ceph.osd.8.smithi165.stdout:Mar 28 03:48:21 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-8[55184]: 2024-03-28T03:48:21.454+0000 7fd543c54700 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:34.764640+0000 front 2024-03-28T03:47:34.764564+0000 (oldest deadline 2024-03-28T03:47:58.864724+0000) 2024-03-28T03:48:21.628 INFO:journalctl@ceph.osd.8.smithi165.stdout:Mar 28 03:48:21 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-8[55184]: 2024-03-28T03:48:21.454+0000 7fd543c54700 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2024-03-28T03:47:40.466520+0000 front 2024-03-28T03:47:40.466406+0000 (oldest deadline 2024-03-28T03:48:07.466815+0000) 2024-03-28T03:48:21.629 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:21 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:21.260+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:18.314545+0000 front 2024-03-28T03:47:18.314582+0000 (oldest deadline 2024-03-28T03:47:44.214430+0000) 2024-03-28T03:48:21.629 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:21 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:21.260+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:27.715675+0000 front 2024-03-28T03:47:27.715645+0000 (oldest deadline 2024-03-28T03:47:50.615667+0000) 2024-03-28T03:48:21.629 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:21 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:21.260+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:32.916784+0000 front 2024-03-28T03:47:32.916834+0000 (oldest deadline 2024-03-28T03:47:58.816929+0000) 2024-03-28T03:48:21.629 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:21 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:21.260+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2024-03-28T03:47:52.919580+0000 front 2024-03-28T03:47:52.919549+0000 (oldest deadline 2024-03-28T03:48:17.019575+0000) 2024-03-28T03:48:21.764 INFO:journalctl@ceph.osd.7.smithi132.stdout:Mar 28 03:48:21 smithi132 podman[121070]: 2024-03-28 03:48:21.459543471 +0000 UTC m=+0.664214536 container died 319af517f55d26849123d9274f33faaef7548095889ad83e52af7e9ecf3ac3a8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-7-deactivate, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.build-date=20240319, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-03-28T03:48:21.764 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:21 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:21.529+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:18.649888+0000 front 2024-03-28T03:47:18.649830+0000 (oldest deadline 2024-03-28T03:47:42.749644+0000) 2024-03-28T03:48:21.764 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:21 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:21.529+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:29.151297+0000 front 2024-03-28T03:47:29.151274+0000 (oldest deadline 2024-03-28T03:47:50.152219+0000) 2024-03-28T03:48:21.764 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:21 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:21.529+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:34.853462+0000 front 2024-03-28T03:47:34.853609+0000 (oldest deadline 2024-03-28T03:47:58.353565+0000) 2024-03-28T03:48:21.764 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:21 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:21.529+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2024-03-28T03:47:44.155643+0000 front 2024-03-28T03:47:44.155546+0000 (oldest deadline 2024-03-28T03:48:09.455316+0000) 2024-03-28T03:48:21.764 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:21 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:21.529+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2024-03-28T03:47:52.356476+0000 front 2024-03-28T03:47:52.356457+0000 (oldest deadline 2024-03-28T03:48:17.056580+0000) 2024-03-28T03:48:21.901 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:21 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:48:21.412+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:19.734645+0000 front 2024-03-28T03:47:19.734461+0000 (oldest deadline 2024-03-28T03:47:44.434325+0000) 2024-03-28T03:48:21.901 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:21 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:48:21.412+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:25.537050+0000 front 2024-03-28T03:47:25.536933+0000 (oldest deadline 2024-03-28T03:47:51.437142+0000) 2024-03-28T03:48:21.901 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:21 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:48:21.412+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:32.539643+0000 front 2024-03-28T03:47:32.539558+0000 (oldest deadline 2024-03-28T03:47:58.439707+0000) 2024-03-28T03:48:21.901 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:21 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:48:21.412+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2024-03-28T03:47:43.141824+0000 front 2024-03-28T03:47:43.142146+0000 (oldest deadline 2024-03-28T03:48:08.441984+0000) 2024-03-28T03:48:22.070 DEBUG:teuthology.orchestra.run.smithi132:> sudo pkill -f 'journalctl -f -n 0 -u ceph-10f73b92-ec98-11ee-95d0-87774f69a715@osd.7.service' 2024-03-28T03:48:22.111 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-28T03:48:22.112 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-03-28T03:48:22.112 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-03-28T03:48:22.112 DEBUG:teuthology.orchestra.run.smithi165:> sudo systemctl stop ceph-10f73b92-ec98-11ee-95d0-87774f69a715@osd.8 2024-03-28T03:48:22.413 INFO:journalctl@ceph.osd.8.smithi165.stdout:Mar 28 03:48:22 smithi165 systemd[1]: Stopping Ceph osd.8 for 10f73b92-ec98-11ee-95d0-87774f69a715... 2024-03-28T03:48:22.413 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:22 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:22.232+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:18.314545+0000 front 2024-03-28T03:47:18.314582+0000 (oldest deadline 2024-03-28T03:47:44.214430+0000) 2024-03-28T03:48:22.413 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:22 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:22.232+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:27.715675+0000 front 2024-03-28T03:47:27.715645+0000 (oldest deadline 2024-03-28T03:47:50.615667+0000) 2024-03-28T03:48:22.413 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:22 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:22.232+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:32.916784+0000 front 2024-03-28T03:47:32.916834+0000 (oldest deadline 2024-03-28T03:47:58.816929+0000) 2024-03-28T03:48:22.413 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:22 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:22.232+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2024-03-28T03:47:52.919580+0000 front 2024-03-28T03:47:52.919549+0000 (oldest deadline 2024-03-28T03:48:17.019575+0000) 2024-03-28T03:48:22.651 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:48:22.378+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:19.734645+0000 front 2024-03-28T03:47:19.734461+0000 (oldest deadline 2024-03-28T03:47:44.434325+0000) 2024-03-28T03:48:22.652 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:48:22.378+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:25.537050+0000 front 2024-03-28T03:47:25.536933+0000 (oldest deadline 2024-03-28T03:47:51.437142+0000) 2024-03-28T03:48:22.652 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:48:22.378+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:32.539643+0000 front 2024-03-28T03:47:32.539558+0000 (oldest deadline 2024-03-28T03:47:58.439707+0000) 2024-03-28T03:48:22.652 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:22 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:48:22.378+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2024-03-28T03:47:43.141824+0000 front 2024-03-28T03:47:43.142146+0000 (oldest deadline 2024-03-28T03:48:08.441984+0000) 2024-03-28T03:48:22.878 INFO:journalctl@ceph.osd.8.smithi165.stdout:Mar 28 03:48:22 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-8[55184]: 2024-03-28T03:48:22.411+0000 7fd54b668700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-28T03:48:22.878 INFO:journalctl@ceph.osd.8.smithi165.stdout:Mar 28 03:48:22 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-8[55184]: 2024-03-28T03:48:22.411+0000 7fd54b668700 -1 osd.8 128 *** Got signal Terminated *** 2024-03-28T03:48:22.878 INFO:journalctl@ceph.osd.8.smithi165.stdout:Mar 28 03:48:22 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-8[55184]: 2024-03-28T03:48:22.411+0000 7fd54b668700 -1 osd.8 128 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-28T03:48:22.879 INFO:journalctl@ceph.osd.8.smithi165.stdout:Mar 28 03:48:22 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-8[55184]: 2024-03-28T03:48:22.439+0000 7fd543c54700 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:19.562620+0000 front 2024-03-28T03:47:19.563027+0000 (oldest deadline 2024-03-28T03:47:46.562998+0000) 2024-03-28T03:48:22.879 INFO:journalctl@ceph.osd.8.smithi165.stdout:Mar 28 03:48:22 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-8[55184]: 2024-03-28T03:48:22.439+0000 7fd543c54700 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:26.563615+0000 front 2024-03-28T03:47:26.563737+0000 (oldest deadline 2024-03-28T03:47:50.063561+0000) 2024-03-28T03:48:22.879 INFO:journalctl@ceph.osd.8.smithi165.stdout:Mar 28 03:48:22 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-8[55184]: 2024-03-28T03:48:22.439+0000 7fd543c54700 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:34.764640+0000 front 2024-03-28T03:47:34.764564+0000 (oldest deadline 2024-03-28T03:47:58.864724+0000) 2024-03-28T03:48:22.879 INFO:journalctl@ceph.osd.8.smithi165.stdout:Mar 28 03:48:22 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-8[55184]: 2024-03-28T03:48:22.439+0000 7fd543c54700 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2024-03-28T03:47:40.466520+0000 front 2024-03-28T03:47:40.466406+0000 (oldest deadline 2024-03-28T03:48:07.466815+0000) 2024-03-28T03:48:22.949 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:22 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:22.554+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:18.649888+0000 front 2024-03-28T03:47:18.649830+0000 (oldest deadline 2024-03-28T03:47:42.749644+0000) 2024-03-28T03:48:22.949 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:22 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:22.554+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:29.151297+0000 front 2024-03-28T03:47:29.151274+0000 (oldest deadline 2024-03-28T03:47:50.152219+0000) 2024-03-28T03:48:22.949 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:22 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:22.554+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:34.853462+0000 front 2024-03-28T03:47:34.853609+0000 (oldest deadline 2024-03-28T03:47:58.353565+0000) 2024-03-28T03:48:22.949 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:22 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:22.554+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2024-03-28T03:47:44.155643+0000 front 2024-03-28T03:47:44.155546+0000 (oldest deadline 2024-03-28T03:48:09.455316+0000) 2024-03-28T03:48:22.949 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:22 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:22.554+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2024-03-28T03:47:52.356476+0000 front 2024-03-28T03:47:52.356457+0000 (oldest deadline 2024-03-28T03:48:17.056580+0000) 2024-03-28T03:48:23.628 INFO:journalctl@ceph.osd.8.smithi165.stdout:Mar 28 03:48:23 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-8[55184]: 2024-03-28T03:48:23.415+0000 7fd543c54700 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:19.562620+0000 front 2024-03-28T03:47:19.563027+0000 (oldest deadline 2024-03-28T03:47:46.562998+0000) 2024-03-28T03:48:23.629 INFO:journalctl@ceph.osd.8.smithi165.stdout:Mar 28 03:48:23 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-8[55184]: 2024-03-28T03:48:23.415+0000 7fd543c54700 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:26.563615+0000 front 2024-03-28T03:47:26.563737+0000 (oldest deadline 2024-03-28T03:47:50.063561+0000) 2024-03-28T03:48:23.629 INFO:journalctl@ceph.osd.8.smithi165.stdout:Mar 28 03:48:23 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-8[55184]: 2024-03-28T03:48:23.415+0000 7fd543c54700 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:34.764640+0000 front 2024-03-28T03:47:34.764564+0000 (oldest deadline 2024-03-28T03:47:58.864724+0000) 2024-03-28T03:48:23.629 INFO:journalctl@ceph.osd.8.smithi165.stdout:Mar 28 03:48:23 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-8[55184]: 2024-03-28T03:48:23.415+0000 7fd543c54700 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2024-03-28T03:47:40.466520+0000 front 2024-03-28T03:47:40.466406+0000 (oldest deadline 2024-03-28T03:48:07.466815+0000) 2024-03-28T03:48:23.629 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:23 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:23.231+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:18.314545+0000 front 2024-03-28T03:47:18.314582+0000 (oldest deadline 2024-03-28T03:47:44.214430+0000) 2024-03-28T03:48:23.629 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:23 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:23.231+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:27.715675+0000 front 2024-03-28T03:47:27.715645+0000 (oldest deadline 2024-03-28T03:47:50.615667+0000) 2024-03-28T03:48:23.629 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:23 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:23.231+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:32.916784+0000 front 2024-03-28T03:47:32.916834+0000 (oldest deadline 2024-03-28T03:47:58.816929+0000) 2024-03-28T03:48:23.629 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:23 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:23.231+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2024-03-28T03:47:52.919580+0000 front 2024-03-28T03:47:52.919549+0000 (oldest deadline 2024-03-28T03:48:17.019575+0000) 2024-03-28T03:48:23.651 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:23 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:48:23.384+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:19.734645+0000 front 2024-03-28T03:47:19.734461+0000 (oldest deadline 2024-03-28T03:47:44.434325+0000) 2024-03-28T03:48:23.651 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:23 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:48:23.384+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:25.537050+0000 front 2024-03-28T03:47:25.536933+0000 (oldest deadline 2024-03-28T03:47:51.437142+0000) 2024-03-28T03:48:23.651 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:23 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:48:23.384+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:32.539643+0000 front 2024-03-28T03:47:32.539558+0000 (oldest deadline 2024-03-28T03:47:58.439707+0000) 2024-03-28T03:48:23.651 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:23 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:48:23.384+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2024-03-28T03:47:43.141824+0000 front 2024-03-28T03:47:43.142146+0000 (oldest deadline 2024-03-28T03:48:08.441984+0000) 2024-03-28T03:48:23.948 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:23 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:23.529+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:18.649888+0000 front 2024-03-28T03:47:18.649830+0000 (oldest deadline 2024-03-28T03:47:42.749644+0000) 2024-03-28T03:48:23.949 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:23 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:23.529+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:29.151297+0000 front 2024-03-28T03:47:29.151274+0000 (oldest deadline 2024-03-28T03:47:50.152219+0000) 2024-03-28T03:48:23.949 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:23 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:23.529+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:34.853462+0000 front 2024-03-28T03:47:34.853609+0000 (oldest deadline 2024-03-28T03:47:58.353565+0000) 2024-03-28T03:48:23.949 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:23 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:23.529+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2024-03-28T03:47:44.155643+0000 front 2024-03-28T03:47:44.155546+0000 (oldest deadline 2024-03-28T03:48:09.455316+0000) 2024-03-28T03:48:23.949 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:23 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:23.529+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2024-03-28T03:47:52.356476+0000 front 2024-03-28T03:47:52.356457+0000 (oldest deadline 2024-03-28T03:48:17.056580+0000) 2024-03-28T03:48:23.949 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:23 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:23.529+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-03-28T03:47:57.557659+0000 front 2024-03-28T03:47:57.557559+0000 (oldest deadline 2024-03-28T03:48:23.457566+0000) 2024-03-28T03:48:24.628 INFO:journalctl@ceph.osd.8.smithi165.stdout:Mar 28 03:48:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-8[55184]: 2024-03-28T03:48:24.433+0000 7fd543c54700 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:19.562620+0000 front 2024-03-28T03:47:19.563027+0000 (oldest deadline 2024-03-28T03:47:46.562998+0000) 2024-03-28T03:48:24.628 INFO:journalctl@ceph.osd.8.smithi165.stdout:Mar 28 03:48:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-8[55184]: 2024-03-28T03:48:24.433+0000 7fd543c54700 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:26.563615+0000 front 2024-03-28T03:47:26.563737+0000 (oldest deadline 2024-03-28T03:47:50.063561+0000) 2024-03-28T03:48:24.629 INFO:journalctl@ceph.osd.8.smithi165.stdout:Mar 28 03:48:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-8[55184]: 2024-03-28T03:48:24.433+0000 7fd543c54700 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:34.764640+0000 front 2024-03-28T03:47:34.764564+0000 (oldest deadline 2024-03-28T03:47:58.864724+0000) 2024-03-28T03:48:24.629 INFO:journalctl@ceph.osd.8.smithi165.stdout:Mar 28 03:48:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-8[55184]: 2024-03-28T03:48:24.433+0000 7fd543c54700 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2024-03-28T03:47:40.466520+0000 front 2024-03-28T03:47:40.466406+0000 (oldest deadline 2024-03-28T03:48:07.466815+0000) 2024-03-28T03:48:24.629 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:24.219+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:18.314545+0000 front 2024-03-28T03:47:18.314582+0000 (oldest deadline 2024-03-28T03:47:44.214430+0000) 2024-03-28T03:48:24.629 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:24.219+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:27.715675+0000 front 2024-03-28T03:47:27.715645+0000 (oldest deadline 2024-03-28T03:47:50.615667+0000) 2024-03-28T03:48:24.629 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:24.219+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:32.916784+0000 front 2024-03-28T03:47:32.916834+0000 (oldest deadline 2024-03-28T03:47:58.816929+0000) 2024-03-28T03:48:24.629 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:24 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:24.219+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2024-03-28T03:47:52.919580+0000 front 2024-03-28T03:47:52.919549+0000 (oldest deadline 2024-03-28T03:48:17.019575+0000) 2024-03-28T03:48:24.651 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:48:24.398+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:19.734645+0000 front 2024-03-28T03:47:19.734461+0000 (oldest deadline 2024-03-28T03:47:44.434325+0000) 2024-03-28T03:48:24.651 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:48:24.398+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:25.537050+0000 front 2024-03-28T03:47:25.536933+0000 (oldest deadline 2024-03-28T03:47:51.437142+0000) 2024-03-28T03:48:24.651 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:48:24.398+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:32.539643+0000 front 2024-03-28T03:47:32.539558+0000 (oldest deadline 2024-03-28T03:47:58.439707+0000) 2024-03-28T03:48:24.651 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:24 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:48:24.398+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2024-03-28T03:47:43.141824+0000 front 2024-03-28T03:47:43.142146+0000 (oldest deadline 2024-03-28T03:48:08.441984+0000) 2024-03-28T03:48:24.949 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:24 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:24.541+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:18.649888+0000 front 2024-03-28T03:47:18.649830+0000 (oldest deadline 2024-03-28T03:47:42.749644+0000) 2024-03-28T03:48:24.949 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:24 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:24.541+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:29.151297+0000 front 2024-03-28T03:47:29.151274+0000 (oldest deadline 2024-03-28T03:47:50.152219+0000) 2024-03-28T03:48:24.949 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:24 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:24.541+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:34.853462+0000 front 2024-03-28T03:47:34.853609+0000 (oldest deadline 2024-03-28T03:47:58.353565+0000) 2024-03-28T03:48:24.949 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:24 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:24.541+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2024-03-28T03:47:44.155643+0000 front 2024-03-28T03:47:44.155546+0000 (oldest deadline 2024-03-28T03:48:09.455316+0000) 2024-03-28T03:48:24.949 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:24 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:24.541+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2024-03-28T03:47:52.356476+0000 front 2024-03-28T03:47:52.356457+0000 (oldest deadline 2024-03-28T03:48:17.056580+0000) 2024-03-28T03:48:24.949 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:24 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:24.541+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-03-28T03:47:57.557659+0000 front 2024-03-28T03:47:57.557559+0000 (oldest deadline 2024-03-28T03:48:23.457566+0000) 2024-03-28T03:48:25.628 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:25 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:25.210+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:18.314545+0000 front 2024-03-28T03:47:18.314582+0000 (oldest deadline 2024-03-28T03:47:44.214430+0000) 2024-03-28T03:48:25.628 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:25 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:25.210+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:27.715675+0000 front 2024-03-28T03:47:27.715645+0000 (oldest deadline 2024-03-28T03:47:50.615667+0000) 2024-03-28T03:48:25.629 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:25 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:25.210+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:32.916784+0000 front 2024-03-28T03:47:32.916834+0000 (oldest deadline 2024-03-28T03:47:58.816929+0000) 2024-03-28T03:48:25.629 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:25 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:25.210+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2024-03-28T03:47:52.919580+0000 front 2024-03-28T03:47:52.919549+0000 (oldest deadline 2024-03-28T03:48:17.019575+0000) 2024-03-28T03:48:25.629 INFO:journalctl@ceph.osd.8.smithi165.stdout:Mar 28 03:48:25 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-8[55184]: 2024-03-28T03:48:25.387+0000 7fd543c54700 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:19.562620+0000 front 2024-03-28T03:47:19.563027+0000 (oldest deadline 2024-03-28T03:47:46.562998+0000) 2024-03-28T03:48:25.629 INFO:journalctl@ceph.osd.8.smithi165.stdout:Mar 28 03:48:25 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-8[55184]: 2024-03-28T03:48:25.387+0000 7fd543c54700 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:26.563615+0000 front 2024-03-28T03:47:26.563737+0000 (oldest deadline 2024-03-28T03:47:50.063561+0000) 2024-03-28T03:48:25.629 INFO:journalctl@ceph.osd.8.smithi165.stdout:Mar 28 03:48:25 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-8[55184]: 2024-03-28T03:48:25.387+0000 7fd543c54700 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:34.764640+0000 front 2024-03-28T03:47:34.764564+0000 (oldest deadline 2024-03-28T03:47:58.864724+0000) 2024-03-28T03:48:25.629 INFO:journalctl@ceph.osd.8.smithi165.stdout:Mar 28 03:48:25 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-8[55184]: 2024-03-28T03:48:25.387+0000 7fd543c54700 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2024-03-28T03:47:40.466520+0000 front 2024-03-28T03:47:40.466406+0000 (oldest deadline 2024-03-28T03:48:07.466815+0000) 2024-03-28T03:48:25.901 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:25 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:48:25.411+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:19.734645+0000 front 2024-03-28T03:47:19.734461+0000 (oldest deadline 2024-03-28T03:47:44.434325+0000) 2024-03-28T03:48:25.901 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:25 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:48:25.411+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:25.537050+0000 front 2024-03-28T03:47:25.536933+0000 (oldest deadline 2024-03-28T03:47:51.437142+0000) 2024-03-28T03:48:25.902 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:25 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:48:25.411+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:32.539643+0000 front 2024-03-28T03:47:32.539558+0000 (oldest deadline 2024-03-28T03:47:58.439707+0000) 2024-03-28T03:48:25.902 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:25 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:48:25.411+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2024-03-28T03:47:43.141824+0000 front 2024-03-28T03:47:43.142146+0000 (oldest deadline 2024-03-28T03:48:08.441984+0000) 2024-03-28T03:48:25.902 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:25 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:48:25.411+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-03-28T03:48:01.048279+0000 front 2024-03-28T03:48:01.048186+0000 (oldest deadline 2024-03-28T03:48:24.548073+0000) 2024-03-28T03:48:25.948 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:25 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:25.551+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:18.649888+0000 front 2024-03-28T03:47:18.649830+0000 (oldest deadline 2024-03-28T03:47:42.749644+0000) 2024-03-28T03:48:25.948 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:25 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:25.551+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:29.151297+0000 front 2024-03-28T03:47:29.151274+0000 (oldest deadline 2024-03-28T03:47:50.152219+0000) 2024-03-28T03:48:25.949 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:25 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:25.551+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:34.853462+0000 front 2024-03-28T03:47:34.853609+0000 (oldest deadline 2024-03-28T03:47:58.353565+0000) 2024-03-28T03:48:25.949 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:25 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:25.551+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2024-03-28T03:47:44.155643+0000 front 2024-03-28T03:47:44.155546+0000 (oldest deadline 2024-03-28T03:48:09.455316+0000) 2024-03-28T03:48:25.949 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:25 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:25.551+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2024-03-28T03:47:52.356476+0000 front 2024-03-28T03:47:52.356457+0000 (oldest deadline 2024-03-28T03:48:17.056580+0000) 2024-03-28T03:48:25.949 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:25 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:25.551+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-03-28T03:47:57.557659+0000 front 2024-03-28T03:47:57.557559+0000 (oldest deadline 2024-03-28T03:48:23.457566+0000) 2024-03-28T03:48:26.628 INFO:journalctl@ceph.osd.8.smithi165.stdout:Mar 28 03:48:26 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-8[55184]: 2024-03-28T03:48:26.390+0000 7fd543c54700 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:19.562620+0000 front 2024-03-28T03:47:19.563027+0000 (oldest deadline 2024-03-28T03:47:46.562998+0000) 2024-03-28T03:48:26.628 INFO:journalctl@ceph.osd.8.smithi165.stdout:Mar 28 03:48:26 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-8[55184]: 2024-03-28T03:48:26.390+0000 7fd543c54700 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:26.563615+0000 front 2024-03-28T03:47:26.563737+0000 (oldest deadline 2024-03-28T03:47:50.063561+0000) 2024-03-28T03:48:26.628 INFO:journalctl@ceph.osd.8.smithi165.stdout:Mar 28 03:48:26 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-8[55184]: 2024-03-28T03:48:26.390+0000 7fd543c54700 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:34.764640+0000 front 2024-03-28T03:47:34.764564+0000 (oldest deadline 2024-03-28T03:47:58.864724+0000) 2024-03-28T03:48:26.629 INFO:journalctl@ceph.osd.8.smithi165.stdout:Mar 28 03:48:26 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-8[55184]: 2024-03-28T03:48:26.390+0000 7fd543c54700 -1 osd.8 128 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2024-03-28T03:47:40.466520+0000 front 2024-03-28T03:47:40.466406+0000 (oldest deadline 2024-03-28T03:48:07.466815+0000) 2024-03-28T03:48:26.629 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:26 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:26.180+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:18.314545+0000 front 2024-03-28T03:47:18.314582+0000 (oldest deadline 2024-03-28T03:47:44.214430+0000) 2024-03-28T03:48:26.629 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:26 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:26.180+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:27.715675+0000 front 2024-03-28T03:47:27.715645+0000 (oldest deadline 2024-03-28T03:47:50.615667+0000) 2024-03-28T03:48:26.629 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:26 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:26.180+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:32.916784+0000 front 2024-03-28T03:47:32.916834+0000 (oldest deadline 2024-03-28T03:47:58.816929+0000) 2024-03-28T03:48:26.629 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:26 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:26.180+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2024-03-28T03:47:52.919580+0000 front 2024-03-28T03:47:52.919549+0000 (oldest deadline 2024-03-28T03:48:17.019575+0000) 2024-03-28T03:48:26.901 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:26 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:48:26.448+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:19.734645+0000 front 2024-03-28T03:47:19.734461+0000 (oldest deadline 2024-03-28T03:47:44.434325+0000) 2024-03-28T03:48:26.902 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:26 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:48:26.448+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:25.537050+0000 front 2024-03-28T03:47:25.536933+0000 (oldest deadline 2024-03-28T03:47:51.437142+0000) 2024-03-28T03:48:26.902 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:26 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:48:26.448+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:32.539643+0000 front 2024-03-28T03:47:32.539558+0000 (oldest deadline 2024-03-28T03:47:58.439707+0000) 2024-03-28T03:48:26.902 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:26 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:48:26.448+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2024-03-28T03:47:43.141824+0000 front 2024-03-28T03:47:43.142146+0000 (oldest deadline 2024-03-28T03:48:08.441984+0000) 2024-03-28T03:48:26.902 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:26 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:48:26.448+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-03-28T03:48:01.048279+0000 front 2024-03-28T03:48:01.048186+0000 (oldest deadline 2024-03-28T03:48:24.548073+0000) 2024-03-28T03:48:26.948 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:26 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:26.520+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:18.649888+0000 front 2024-03-28T03:47:18.649830+0000 (oldest deadline 2024-03-28T03:47:42.749644+0000) 2024-03-28T03:48:26.948 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:26 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:26.520+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:29.151297+0000 front 2024-03-28T03:47:29.151274+0000 (oldest deadline 2024-03-28T03:47:50.152219+0000) 2024-03-28T03:48:26.949 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:26 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:26.520+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:34.853462+0000 front 2024-03-28T03:47:34.853609+0000 (oldest deadline 2024-03-28T03:47:58.353565+0000) 2024-03-28T03:48:26.949 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:26 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:26.520+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2024-03-28T03:47:44.155643+0000 front 2024-03-28T03:47:44.155546+0000 (oldest deadline 2024-03-28T03:48:09.455316+0000) 2024-03-28T03:48:26.949 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:26 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:26.520+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2024-03-28T03:47:52.356476+0000 front 2024-03-28T03:47:52.356457+0000 (oldest deadline 2024-03-28T03:48:17.056580+0000) 2024-03-28T03:48:26.949 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:26 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:26.520+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-03-28T03:47:57.557659+0000 front 2024-03-28T03:47:57.557559+0000 (oldest deadline 2024-03-28T03:48:23.457566+0000) 2024-03-28T03:48:27.628 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:27 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:27.151+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:18.314545+0000 front 2024-03-28T03:47:18.314582+0000 (oldest deadline 2024-03-28T03:47:44.214430+0000) 2024-03-28T03:48:27.628 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:27 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:27.151+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:27.715675+0000 front 2024-03-28T03:47:27.715645+0000 (oldest deadline 2024-03-28T03:47:50.615667+0000) 2024-03-28T03:48:27.629 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:27 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:27.151+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:32.916784+0000 front 2024-03-28T03:47:32.916834+0000 (oldest deadline 2024-03-28T03:47:58.816929+0000) 2024-03-28T03:48:27.629 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:27 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:27.151+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2024-03-28T03:47:52.919580+0000 front 2024-03-28T03:47:52.919549+0000 (oldest deadline 2024-03-28T03:48:17.019575+0000) 2024-03-28T03:48:27.629 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:27 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:27.151+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-03-28T03:48:02.320477+0000 front 2024-03-28T03:48:02.320512+0000 (oldest deadline 2024-03-28T03:48:27.020650+0000) 2024-03-28T03:48:27.652 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:27 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:48:27.403+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:19.734645+0000 front 2024-03-28T03:47:19.734461+0000 (oldest deadline 2024-03-28T03:47:44.434325+0000) 2024-03-28T03:48:27.652 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:27 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:48:27.403+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:25.537050+0000 front 2024-03-28T03:47:25.536933+0000 (oldest deadline 2024-03-28T03:47:51.437142+0000) 2024-03-28T03:48:27.652 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:27 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:48:27.403+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:32.539643+0000 front 2024-03-28T03:47:32.539558+0000 (oldest deadline 2024-03-28T03:47:58.439707+0000) 2024-03-28T03:48:27.652 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:27 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:48:27.403+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2024-03-28T03:47:43.141824+0000 front 2024-03-28T03:47:43.142146+0000 (oldest deadline 2024-03-28T03:48:08.441984+0000) 2024-03-28T03:48:27.652 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:27 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:48:27.403+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-03-28T03:48:01.048279+0000 front 2024-03-28T03:48:01.048186+0000 (oldest deadline 2024-03-28T03:48:24.548073+0000) 2024-03-28T03:48:27.948 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:27 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:27.566+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:18.649888+0000 front 2024-03-28T03:47:18.649830+0000 (oldest deadline 2024-03-28T03:47:42.749644+0000) 2024-03-28T03:48:27.948 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:27 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:27.566+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:29.151297+0000 front 2024-03-28T03:47:29.151274+0000 (oldest deadline 2024-03-28T03:47:50.152219+0000) 2024-03-28T03:48:27.949 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:27 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:27.566+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:34.853462+0000 front 2024-03-28T03:47:34.853609+0000 (oldest deadline 2024-03-28T03:47:58.353565+0000) 2024-03-28T03:48:27.949 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:27 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:27.566+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2024-03-28T03:47:44.155643+0000 front 2024-03-28T03:47:44.155546+0000 (oldest deadline 2024-03-28T03:48:09.455316+0000) 2024-03-28T03:48:27.949 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:27 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:27.566+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2024-03-28T03:47:52.356476+0000 front 2024-03-28T03:47:52.356457+0000 (oldest deadline 2024-03-28T03:48:17.056580+0000) 2024-03-28T03:48:27.949 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:27 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:27.566+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-03-28T03:47:57.557659+0000 front 2024-03-28T03:47:57.557559+0000 (oldest deadline 2024-03-28T03:48:23.457566+0000) 2024-03-28T03:48:28.072 INFO:journalctl@ceph.osd.8.smithi165.stdout:Mar 28 03:48:27 smithi165 podman[124140]: 2024-03-28 03:48:27.77943395 +0000 UTC m=+5.472490300 container died 345cf3a76e05bc123f01aeb1c83ab659e60cf5999598189a6cd0824d0ebd2c81 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-8, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-03-28T03:48:28.378 INFO:journalctl@ceph.osd.8.smithi165.stdout:Mar 28 03:48:28 smithi165 podman[124140]: 2024-03-28 03:48:28.161264077 +0000 UTC m=+5.854320428 container cleanup 345cf3a76e05bc123f01aeb1c83ab659e60cf5999598189a6cd0824d0ebd2c81 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-8, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-03-28T03:48:28.378 INFO:journalctl@ceph.osd.8.smithi165.stdout:Mar 28 03:48:28 smithi165 bash[124140]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-8 2024-03-28T03:48:28.379 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:28 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:28.174+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:18.314545+0000 front 2024-03-28T03:47:18.314582+0000 (oldest deadline 2024-03-28T03:47:44.214430+0000) 2024-03-28T03:48:28.379 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:28 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:28.174+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:27.715675+0000 front 2024-03-28T03:47:27.715645+0000 (oldest deadline 2024-03-28T03:47:50.615667+0000) 2024-03-28T03:48:28.379 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:28 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:28.174+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:32.916784+0000 front 2024-03-28T03:47:32.916834+0000 (oldest deadline 2024-03-28T03:47:58.816929+0000) 2024-03-28T03:48:28.379 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:28 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:28.174+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2024-03-28T03:47:52.919580+0000 front 2024-03-28T03:47:52.919549+0000 (oldest deadline 2024-03-28T03:48:17.019575+0000) 2024-03-28T03:48:28.379 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:28 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:28.174+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-03-28T03:48:02.320477+0000 front 2024-03-28T03:48:02.320512+0000 (oldest deadline 2024-03-28T03:48:27.020650+0000) 2024-03-28T03:48:28.769 INFO:journalctl@ceph.osd.8.smithi165.stdout:Mar 28 03:48:28 smithi165 podman[124157]: 2024-03-28 03:48:28.420734134 +0000 UTC m=+0.637747021 container remove 345cf3a76e05bc123f01aeb1c83ab659e60cf5999598189a6cd0824d0ebd2c81 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-8, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319) 2024-03-28T03:48:28.901 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:28 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:48:28.442+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:19.734645+0000 front 2024-03-28T03:47:19.734461+0000 (oldest deadline 2024-03-28T03:47:44.434325+0000) 2024-03-28T03:48:28.901 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:28 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:48:28.442+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:25.537050+0000 front 2024-03-28T03:47:25.536933+0000 (oldest deadline 2024-03-28T03:47:51.437142+0000) 2024-03-28T03:48:28.902 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:28 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:48:28.442+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:32.539643+0000 front 2024-03-28T03:47:32.539558+0000 (oldest deadline 2024-03-28T03:47:58.439707+0000) 2024-03-28T03:48:28.902 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:28 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:48:28.442+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2024-03-28T03:47:43.141824+0000 front 2024-03-28T03:47:43.142146+0000 (oldest deadline 2024-03-28T03:48:08.441984+0000) 2024-03-28T03:48:28.902 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:28 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:48:28.442+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-03-28T03:48:01.048279+0000 front 2024-03-28T03:48:01.048186+0000 (oldest deadline 2024-03-28T03:48:24.548073+0000) 2024-03-28T03:48:28.948 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:28 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:28.567+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:18.649888+0000 front 2024-03-28T03:47:18.649830+0000 (oldest deadline 2024-03-28T03:47:42.749644+0000) 2024-03-28T03:48:28.949 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:28 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:28.567+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:29.151297+0000 front 2024-03-28T03:47:29.151274+0000 (oldest deadline 2024-03-28T03:47:50.152219+0000) 2024-03-28T03:48:28.949 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:28 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:28.567+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:34.853462+0000 front 2024-03-28T03:47:34.853609+0000 (oldest deadline 2024-03-28T03:47:58.353565+0000) 2024-03-28T03:48:28.949 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:28 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:28.567+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2024-03-28T03:47:44.155643+0000 front 2024-03-28T03:47:44.155546+0000 (oldest deadline 2024-03-28T03:48:09.455316+0000) 2024-03-28T03:48:28.949 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:28 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:28.567+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2024-03-28T03:47:52.356476+0000 front 2024-03-28T03:47:52.356457+0000 (oldest deadline 2024-03-28T03:48:17.056580+0000) 2024-03-28T03:48:28.949 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:28 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:28.567+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-03-28T03:47:57.557659+0000 front 2024-03-28T03:47:57.557559+0000 (oldest deadline 2024-03-28T03:48:23.457566+0000) 2024-03-28T03:48:29.019 INFO:journalctl@ceph.osd.8.smithi165.stdout:Mar 28 03:48:28 smithi165 podman[124250]: 2024-03-28 03:48:28.671378753 +0000 UTC m=+0.017676500 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-28T03:48:29.020 INFO:journalctl@ceph.osd.8.smithi165.stdout:Mar 28 03:48:28 smithi165 podman[124250]: 2024-03-28 03:48:28.779850226 +0000 UTC m=+0.126147962 container create 6fc3fadd96cf373a41273466321978211389e3055c45f1145ba4493115fa6ebd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-8-deactivate, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-28T03:48:29.020 INFO:journalctl@ceph.osd.8.smithi165.stdout:Mar 28 03:48:29 smithi165 podman[124250]: 2024-03-28 03:48:29.015722977 +0000 UTC m=+0.362020724 container init 6fc3fadd96cf373a41273466321978211389e3055c45f1145ba4493115fa6ebd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-8-deactivate, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-03-28T03:48:29.020 INFO:journalctl@ceph.osd.8.smithi165.stdout:Mar 28 03:48:29 smithi165 podman[124250]: 2024-03-28 03:48:29.018909271 +0000 UTC m=+0.365207000 container start 6fc3fadd96cf373a41273466321978211389e3055c45f1145ba4493115fa6ebd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-8-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, maintainer=Guillaume Abrioux ) 2024-03-28T03:48:29.312 INFO:journalctl@ceph.osd.8.smithi165.stdout:Mar 28 03:48:29 smithi165 podman[124250]: 2024-03-28 03:48:29.123954671 +0000 UTC m=+0.470252418 container attach 6fc3fadd96cf373a41273466321978211389e3055c45f1145ba4493115fa6ebd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-8-deactivate, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0) 2024-03-28T03:48:29.313 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:29 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:29.136+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:18.314545+0000 front 2024-03-28T03:47:18.314582+0000 (oldest deadline 2024-03-28T03:47:44.214430+0000) 2024-03-28T03:48:29.313 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:29 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:29.136+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:27.715675+0000 front 2024-03-28T03:47:27.715645+0000 (oldest deadline 2024-03-28T03:47:50.615667+0000) 2024-03-28T03:48:29.313 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:29 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:29.136+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:32.916784+0000 front 2024-03-28T03:47:32.916834+0000 (oldest deadline 2024-03-28T03:47:58.816929+0000) 2024-03-28T03:48:29.313 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:29 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:29.136+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2024-03-28T03:47:52.919580+0000 front 2024-03-28T03:47:52.919549+0000 (oldest deadline 2024-03-28T03:48:17.019575+0000) 2024-03-28T03:48:29.313 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:29 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:29.136+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-03-28T03:48:02.320477+0000 front 2024-03-28T03:48:02.320512+0000 (oldest deadline 2024-03-28T03:48:27.020650+0000) 2024-03-28T03:48:29.628 INFO:journalctl@ceph.osd.8.smithi165.stdout:Mar 28 03:48:29 smithi165 podman[124250]: 2024-03-28 03:48:29.311951248 +0000 UTC m=+0.658248997 container died 6fc3fadd96cf373a41273466321978211389e3055c45f1145ba4493115fa6ebd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-8-deactivate, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS) 2024-03-28T03:48:29.901 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:29 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:48:29.485+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:19.734645+0000 front 2024-03-28T03:47:19.734461+0000 (oldest deadline 2024-03-28T03:47:44.434325+0000) 2024-03-28T03:48:29.902 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:29 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:48:29.485+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:25.537050+0000 front 2024-03-28T03:47:25.536933+0000 (oldest deadline 2024-03-28T03:47:51.437142+0000) 2024-03-28T03:48:29.902 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:29 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:48:29.485+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:32.539643+0000 front 2024-03-28T03:47:32.539558+0000 (oldest deadline 2024-03-28T03:47:58.439707+0000) 2024-03-28T03:48:29.902 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:29 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:48:29.485+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2024-03-28T03:47:43.141824+0000 front 2024-03-28T03:47:43.142146+0000 (oldest deadline 2024-03-28T03:48:08.441984+0000) 2024-03-28T03:48:29.902 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:29 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:48:29.485+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-03-28T03:48:01.048279+0000 front 2024-03-28T03:48:01.048186+0000 (oldest deadline 2024-03-28T03:48:24.548073+0000) 2024-03-28T03:48:29.948 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:29 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:29.588+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:18.649888+0000 front 2024-03-28T03:47:18.649830+0000 (oldest deadline 2024-03-28T03:47:42.749644+0000) 2024-03-28T03:48:29.948 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:29 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:29.588+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:29.151297+0000 front 2024-03-28T03:47:29.151274+0000 (oldest deadline 2024-03-28T03:47:50.152219+0000) 2024-03-28T03:48:29.949 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:29 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:29.588+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:34.853462+0000 front 2024-03-28T03:47:34.853609+0000 (oldest deadline 2024-03-28T03:47:58.353565+0000) 2024-03-28T03:48:29.949 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:29 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:29.588+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2024-03-28T03:47:44.155643+0000 front 2024-03-28T03:47:44.155546+0000 (oldest deadline 2024-03-28T03:48:09.455316+0000) 2024-03-28T03:48:29.949 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:29 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:29.588+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2024-03-28T03:47:52.356476+0000 front 2024-03-28T03:47:52.356457+0000 (oldest deadline 2024-03-28T03:48:17.056580+0000) 2024-03-28T03:48:29.949 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:29 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:29.588+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-03-28T03:47:57.557659+0000 front 2024-03-28T03:47:57.557559+0000 (oldest deadline 2024-03-28T03:48:23.457566+0000) 2024-03-28T03:48:29.973 DEBUG:teuthology.orchestra.run.smithi165:> sudo pkill -f 'journalctl -f -n 0 -u ceph-10f73b92-ec98-11ee-95d0-87774f69a715@osd.8.service' 2024-03-28T03:48:30.013 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-28T03:48:30.013 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-03-28T03:48:30.013 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-03-28T03:48:30.013 DEBUG:teuthology.orchestra.run.smithi119:> sudo systemctl stop ceph-10f73b92-ec98-11ee-95d0-87774f69a715@osd.9 2024-03-28T03:48:30.276 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:30 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:30.143+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:18.314545+0000 front 2024-03-28T03:47:18.314582+0000 (oldest deadline 2024-03-28T03:47:44.214430+0000) 2024-03-28T03:48:30.277 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:30 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:30.143+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:27.715675+0000 front 2024-03-28T03:47:27.715645+0000 (oldest deadline 2024-03-28T03:47:50.615667+0000) 2024-03-28T03:48:30.277 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:30 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:30.143+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:32.916784+0000 front 2024-03-28T03:47:32.916834+0000 (oldest deadline 2024-03-28T03:47:58.816929+0000) 2024-03-28T03:48:30.277 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:30 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:30.143+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2024-03-28T03:47:52.919580+0000 front 2024-03-28T03:47:52.919549+0000 (oldest deadline 2024-03-28T03:48:17.019575+0000) 2024-03-28T03:48:30.277 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:30 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:30.143+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-03-28T03:48:02.320477+0000 front 2024-03-28T03:48:02.320512+0000 (oldest deadline 2024-03-28T03:48:27.020650+0000) 2024-03-28T03:48:30.402 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:30 smithi119 systemd[1]: Stopping Ceph osd.9 for 10f73b92-ec98-11ee-95d0-87774f69a715... 2024-03-28T03:48:30.902 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:30 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:48:30.439+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:19.734645+0000 front 2024-03-28T03:47:19.734461+0000 (oldest deadline 2024-03-28T03:47:44.434325+0000) 2024-03-28T03:48:30.902 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:30 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:48:30.439+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:25.537050+0000 front 2024-03-28T03:47:25.536933+0000 (oldest deadline 2024-03-28T03:47:51.437142+0000) 2024-03-28T03:48:30.902 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:30 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:48:30.439+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:32.539643+0000 front 2024-03-28T03:47:32.539558+0000 (oldest deadline 2024-03-28T03:47:58.439707+0000) 2024-03-28T03:48:30.902 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:30 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:48:30.439+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2024-03-28T03:47:43.141824+0000 front 2024-03-28T03:47:43.142146+0000 (oldest deadline 2024-03-28T03:48:08.441984+0000) 2024-03-28T03:48:30.903 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:30 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:48:30.439+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-03-28T03:48:01.048279+0000 front 2024-03-28T03:48:01.048186+0000 (oldest deadline 2024-03-28T03:48:24.548073+0000) 2024-03-28T03:48:30.903 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:30 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:48:30.565+0000 7f6397e00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-28T03:48:30.903 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:30 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:48:30.565+0000 7f6397e00700 -1 osd.9 128 *** Got signal Terminated *** 2024-03-28T03:48:30.903 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:30 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:48:30.565+0000 7f6397e00700 -1 osd.9 128 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-28T03:48:30.948 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:30 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:30.606+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:18.649888+0000 front 2024-03-28T03:47:18.649830+0000 (oldest deadline 2024-03-28T03:47:42.749644+0000) 2024-03-28T03:48:30.949 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:30 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:30.606+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:29.151297+0000 front 2024-03-28T03:47:29.151274+0000 (oldest deadline 2024-03-28T03:47:50.152219+0000) 2024-03-28T03:48:30.949 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:30 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:30.606+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:34.853462+0000 front 2024-03-28T03:47:34.853609+0000 (oldest deadline 2024-03-28T03:47:58.353565+0000) 2024-03-28T03:48:30.949 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:30 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:30.606+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2024-03-28T03:47:44.155643+0000 front 2024-03-28T03:47:44.155546+0000 (oldest deadline 2024-03-28T03:48:09.455316+0000) 2024-03-28T03:48:30.949 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:30 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:30.606+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2024-03-28T03:47:52.356476+0000 front 2024-03-28T03:47:52.356457+0000 (oldest deadline 2024-03-28T03:48:17.056580+0000) 2024-03-28T03:48:30.949 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:30 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:30.606+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-03-28T03:47:57.557659+0000 front 2024-03-28T03:47:57.557559+0000 (oldest deadline 2024-03-28T03:48:23.457566+0000) 2024-03-28T03:48:31.378 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:31 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:31.122+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:18.314545+0000 front 2024-03-28T03:47:18.314582+0000 (oldest deadline 2024-03-28T03:47:44.214430+0000) 2024-03-28T03:48:31.379 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:31 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:31.122+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:27.715675+0000 front 2024-03-28T03:47:27.715645+0000 (oldest deadline 2024-03-28T03:47:50.615667+0000) 2024-03-28T03:48:31.379 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:31 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:31.122+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:32.916784+0000 front 2024-03-28T03:47:32.916834+0000 (oldest deadline 2024-03-28T03:47:58.816929+0000) 2024-03-28T03:48:31.379 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:31 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:31.122+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2024-03-28T03:47:52.919580+0000 front 2024-03-28T03:47:52.919549+0000 (oldest deadline 2024-03-28T03:48:17.019575+0000) 2024-03-28T03:48:31.379 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:31 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:31.122+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-03-28T03:48:02.320477+0000 front 2024-03-28T03:48:02.320512+0000 (oldest deadline 2024-03-28T03:48:27.020650+0000) 2024-03-28T03:48:31.901 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:31 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:48:31.416+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:19.734645+0000 front 2024-03-28T03:47:19.734461+0000 (oldest deadline 2024-03-28T03:47:44.434325+0000) 2024-03-28T03:48:31.901 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:31 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:48:31.416+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:25.537050+0000 front 2024-03-28T03:47:25.536933+0000 (oldest deadline 2024-03-28T03:47:51.437142+0000) 2024-03-28T03:48:31.902 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:31 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:48:31.416+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:32.539643+0000 front 2024-03-28T03:47:32.539558+0000 (oldest deadline 2024-03-28T03:47:58.439707+0000) 2024-03-28T03:48:31.902 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:31 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:48:31.416+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2024-03-28T03:47:43.141824+0000 front 2024-03-28T03:47:43.142146+0000 (oldest deadline 2024-03-28T03:48:08.441984+0000) 2024-03-28T03:48:31.902 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:31 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:48:31.416+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-03-28T03:48:01.048279+0000 front 2024-03-28T03:48:01.048186+0000 (oldest deadline 2024-03-28T03:48:24.548073+0000) 2024-03-28T03:48:31.948 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:31 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:31.564+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:18.649888+0000 front 2024-03-28T03:47:18.649830+0000 (oldest deadline 2024-03-28T03:47:42.749644+0000) 2024-03-28T03:48:31.949 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:31 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:31.564+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:29.151297+0000 front 2024-03-28T03:47:29.151274+0000 (oldest deadline 2024-03-28T03:47:50.152219+0000) 2024-03-28T03:48:31.949 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:31 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:31.564+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:34.853462+0000 front 2024-03-28T03:47:34.853609+0000 (oldest deadline 2024-03-28T03:47:58.353565+0000) 2024-03-28T03:48:31.949 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:31 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:31.564+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2024-03-28T03:47:44.155643+0000 front 2024-03-28T03:47:44.155546+0000 (oldest deadline 2024-03-28T03:48:09.455316+0000) 2024-03-28T03:48:31.949 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:31 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:31.564+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2024-03-28T03:47:52.356476+0000 front 2024-03-28T03:47:52.356457+0000 (oldest deadline 2024-03-28T03:48:17.056580+0000) 2024-03-28T03:48:31.949 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:31 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:31.564+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-03-28T03:47:57.557659+0000 front 2024-03-28T03:47:57.557559+0000 (oldest deadline 2024-03-28T03:48:23.457566+0000) 2024-03-28T03:48:32.628 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:32 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:32.130+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:18.314545+0000 front 2024-03-28T03:47:18.314582+0000 (oldest deadline 2024-03-28T03:47:44.214430+0000) 2024-03-28T03:48:32.628 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:32 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:32.130+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:27.715675+0000 front 2024-03-28T03:47:27.715645+0000 (oldest deadline 2024-03-28T03:47:50.615667+0000) 2024-03-28T03:48:32.629 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:32 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:32.130+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:32.916784+0000 front 2024-03-28T03:47:32.916834+0000 (oldest deadline 2024-03-28T03:47:58.816929+0000) 2024-03-28T03:48:32.629 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:32 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:32.130+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2024-03-28T03:47:52.919580+0000 front 2024-03-28T03:47:52.919549+0000 (oldest deadline 2024-03-28T03:48:17.019575+0000) 2024-03-28T03:48:32.629 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:32 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:32.130+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-03-28T03:48:02.320477+0000 front 2024-03-28T03:48:02.320512+0000 (oldest deadline 2024-03-28T03:48:27.020650+0000) 2024-03-28T03:48:32.901 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:32 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:48:32.464+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:19.734645+0000 front 2024-03-28T03:47:19.734461+0000 (oldest deadline 2024-03-28T03:47:44.434325+0000) 2024-03-28T03:48:32.902 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:32 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:48:32.464+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:25.537050+0000 front 2024-03-28T03:47:25.536933+0000 (oldest deadline 2024-03-28T03:47:51.437142+0000) 2024-03-28T03:48:32.902 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:32 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:48:32.464+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:32.539643+0000 front 2024-03-28T03:47:32.539558+0000 (oldest deadline 2024-03-28T03:47:58.439707+0000) 2024-03-28T03:48:32.902 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:32 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:48:32.464+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2024-03-28T03:47:43.141824+0000 front 2024-03-28T03:47:43.142146+0000 (oldest deadline 2024-03-28T03:48:08.441984+0000) 2024-03-28T03:48:32.902 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:32 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:48:32.464+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-03-28T03:48:01.048279+0000 front 2024-03-28T03:48:01.048186+0000 (oldest deadline 2024-03-28T03:48:24.548073+0000) 2024-03-28T03:48:32.948 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:32 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:32.543+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:18.649888+0000 front 2024-03-28T03:47:18.649830+0000 (oldest deadline 2024-03-28T03:47:42.749644+0000) 2024-03-28T03:48:32.949 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:32 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:32.543+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:29.151297+0000 front 2024-03-28T03:47:29.151274+0000 (oldest deadline 2024-03-28T03:47:50.152219+0000) 2024-03-28T03:48:32.949 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:32 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:32.543+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:34.853462+0000 front 2024-03-28T03:47:34.853609+0000 (oldest deadline 2024-03-28T03:47:58.353565+0000) 2024-03-28T03:48:32.949 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:32 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:32.543+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2024-03-28T03:47:44.155643+0000 front 2024-03-28T03:47:44.155546+0000 (oldest deadline 2024-03-28T03:48:09.455316+0000) 2024-03-28T03:48:32.949 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:32 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:32.543+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2024-03-28T03:47:52.356476+0000 front 2024-03-28T03:47:52.356457+0000 (oldest deadline 2024-03-28T03:48:17.056580+0000) 2024-03-28T03:48:32.949 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:32 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:32.543+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-03-28T03:47:57.557659+0000 front 2024-03-28T03:47:57.557559+0000 (oldest deadline 2024-03-28T03:48:23.457566+0000) 2024-03-28T03:48:32.949 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:32 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:32.543+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.119:6822 osd.6 since back 2024-03-28T03:48:06.958911+0000 front 2024-03-28T03:48:06.959040+0000 (oldest deadline 2024-03-28T03:48:32.258542+0000) 2024-03-28T03:48:33.628 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:33.162+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:18.314545+0000 front 2024-03-28T03:47:18.314582+0000 (oldest deadline 2024-03-28T03:47:44.214430+0000) 2024-03-28T03:48:33.628 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:33.162+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:27.715675+0000 front 2024-03-28T03:47:27.715645+0000 (oldest deadline 2024-03-28T03:47:50.615667+0000) 2024-03-28T03:48:33.629 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:33.162+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:32.916784+0000 front 2024-03-28T03:47:32.916834+0000 (oldest deadline 2024-03-28T03:47:58.816929+0000) 2024-03-28T03:48:33.629 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:33.162+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2024-03-28T03:47:52.919580+0000 front 2024-03-28T03:47:52.919549+0000 (oldest deadline 2024-03-28T03:48:17.019575+0000) 2024-03-28T03:48:33.629 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:33.162+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-03-28T03:48:02.320477+0000 front 2024-03-28T03:48:02.320512+0000 (oldest deadline 2024-03-28T03:48:27.020650+0000) 2024-03-28T03:48:33.629 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:33 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:33.162+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.119:6822 osd.6 since back 2024-03-28T03:48:07.021607+0000 front 2024-03-28T03:48:07.021738+0000 (oldest deadline 2024-03-28T03:48:32.821858+0000) 2024-03-28T03:48:33.901 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:33 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:48:33.511+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:19.734645+0000 front 2024-03-28T03:47:19.734461+0000 (oldest deadline 2024-03-28T03:47:44.434325+0000) 2024-03-28T03:48:33.902 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:33 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:48:33.511+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:25.537050+0000 front 2024-03-28T03:47:25.536933+0000 (oldest deadline 2024-03-28T03:47:51.437142+0000) 2024-03-28T03:48:33.902 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:33 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:48:33.511+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:32.539643+0000 front 2024-03-28T03:47:32.539558+0000 (oldest deadline 2024-03-28T03:47:58.439707+0000) 2024-03-28T03:48:33.902 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:33 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:48:33.511+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2024-03-28T03:47:43.141824+0000 front 2024-03-28T03:47:43.142146+0000 (oldest deadline 2024-03-28T03:48:08.441984+0000) 2024-03-28T03:48:33.902 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:33 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:48:33.511+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-03-28T03:48:01.048279+0000 front 2024-03-28T03:48:01.048186+0000 (oldest deadline 2024-03-28T03:48:24.548073+0000) 2024-03-28T03:48:33.948 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:33 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:33.541+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:18.649888+0000 front 2024-03-28T03:47:18.649830+0000 (oldest deadline 2024-03-28T03:47:42.749644+0000) 2024-03-28T03:48:33.949 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:33 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:33.541+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:29.151297+0000 front 2024-03-28T03:47:29.151274+0000 (oldest deadline 2024-03-28T03:47:50.152219+0000) 2024-03-28T03:48:33.949 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:33 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:33.541+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:34.853462+0000 front 2024-03-28T03:47:34.853609+0000 (oldest deadline 2024-03-28T03:47:58.353565+0000) 2024-03-28T03:48:33.949 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:33 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:33.541+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2024-03-28T03:47:44.155643+0000 front 2024-03-28T03:47:44.155546+0000 (oldest deadline 2024-03-28T03:48:09.455316+0000) 2024-03-28T03:48:33.949 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:33 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:33.541+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2024-03-28T03:47:52.356476+0000 front 2024-03-28T03:47:52.356457+0000 (oldest deadline 2024-03-28T03:48:17.056580+0000) 2024-03-28T03:48:33.949 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:33 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:33.541+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-03-28T03:47:57.557659+0000 front 2024-03-28T03:47:57.557559+0000 (oldest deadline 2024-03-28T03:48:23.457566+0000) 2024-03-28T03:48:33.949 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:33 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:33.541+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.119:6822 osd.6 since back 2024-03-28T03:48:06.958911+0000 front 2024-03-28T03:48:06.959040+0000 (oldest deadline 2024-03-28T03:48:32.258542+0000) 2024-03-28T03:48:34.629 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:34 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:34.187+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:18.314545+0000 front 2024-03-28T03:47:18.314582+0000 (oldest deadline 2024-03-28T03:47:44.214430+0000) 2024-03-28T03:48:34.629 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:34 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:34.187+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:27.715675+0000 front 2024-03-28T03:47:27.715645+0000 (oldest deadline 2024-03-28T03:47:50.615667+0000) 2024-03-28T03:48:34.629 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:34 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:34.187+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:32.916784+0000 front 2024-03-28T03:47:32.916834+0000 (oldest deadline 2024-03-28T03:47:58.816929+0000) 2024-03-28T03:48:34.629 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:34 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:34.187+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2024-03-28T03:47:52.919580+0000 front 2024-03-28T03:47:52.919549+0000 (oldest deadline 2024-03-28T03:48:17.019575+0000) 2024-03-28T03:48:34.629 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:34 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:34.187+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-03-28T03:48:02.320477+0000 front 2024-03-28T03:48:02.320512+0000 (oldest deadline 2024-03-28T03:48:27.020650+0000) 2024-03-28T03:48:34.629 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:34 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:34.187+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.119:6822 osd.6 since back 2024-03-28T03:48:07.021607+0000 front 2024-03-28T03:48:07.021738+0000 (oldest deadline 2024-03-28T03:48:32.821858+0000) 2024-03-28T03:48:34.901 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:34 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:48:34.474+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:19.734645+0000 front 2024-03-28T03:47:19.734461+0000 (oldest deadline 2024-03-28T03:47:44.434325+0000) 2024-03-28T03:48:34.902 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:34 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:48:34.474+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:25.537050+0000 front 2024-03-28T03:47:25.536933+0000 (oldest deadline 2024-03-28T03:47:51.437142+0000) 2024-03-28T03:48:34.902 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:34 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:48:34.474+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:32.539643+0000 front 2024-03-28T03:47:32.539558+0000 (oldest deadline 2024-03-28T03:47:58.439707+0000) 2024-03-28T03:48:34.902 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:34 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:48:34.474+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2024-03-28T03:47:43.141824+0000 front 2024-03-28T03:47:43.142146+0000 (oldest deadline 2024-03-28T03:48:08.441984+0000) 2024-03-28T03:48:34.902 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:34 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:48:34.474+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-03-28T03:48:01.048279+0000 front 2024-03-28T03:48:01.048186+0000 (oldest deadline 2024-03-28T03:48:24.548073+0000) 2024-03-28T03:48:34.948 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:34 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:34.546+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:18.649888+0000 front 2024-03-28T03:47:18.649830+0000 (oldest deadline 2024-03-28T03:47:42.749644+0000) 2024-03-28T03:48:34.948 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:34 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:34.546+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:29.151297+0000 front 2024-03-28T03:47:29.151274+0000 (oldest deadline 2024-03-28T03:47:50.152219+0000) 2024-03-28T03:48:34.949 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:34 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:34.546+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:34.853462+0000 front 2024-03-28T03:47:34.853609+0000 (oldest deadline 2024-03-28T03:47:58.353565+0000) 2024-03-28T03:48:34.949 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:34 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:34.546+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2024-03-28T03:47:44.155643+0000 front 2024-03-28T03:47:44.155546+0000 (oldest deadline 2024-03-28T03:48:09.455316+0000) 2024-03-28T03:48:34.949 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:34 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:34.546+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2024-03-28T03:47:52.356476+0000 front 2024-03-28T03:47:52.356457+0000 (oldest deadline 2024-03-28T03:48:17.056580+0000) 2024-03-28T03:48:34.949 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:34 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:34.546+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-03-28T03:47:57.557659+0000 front 2024-03-28T03:47:57.557559+0000 (oldest deadline 2024-03-28T03:48:23.457566+0000) 2024-03-28T03:48:34.949 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:34 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:34.546+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.119:6822 osd.6 since back 2024-03-28T03:48:06.958911+0000 front 2024-03-28T03:48:06.959040+0000 (oldest deadline 2024-03-28T03:48:32.258542+0000) 2024-03-28T03:48:35.628 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:35 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:35.214+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:18.314545+0000 front 2024-03-28T03:47:18.314582+0000 (oldest deadline 2024-03-28T03:47:44.214430+0000) 2024-03-28T03:48:35.628 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:35 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:35.214+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:27.715675+0000 front 2024-03-28T03:47:27.715645+0000 (oldest deadline 2024-03-28T03:47:50.615667+0000) 2024-03-28T03:48:35.629 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:35 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:35.214+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:32.916784+0000 front 2024-03-28T03:47:32.916834+0000 (oldest deadline 2024-03-28T03:47:58.816929+0000) 2024-03-28T03:48:35.629 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:35 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:35.214+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2024-03-28T03:47:52.919580+0000 front 2024-03-28T03:47:52.919549+0000 (oldest deadline 2024-03-28T03:48:17.019575+0000) 2024-03-28T03:48:35.629 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:35 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:35.214+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-03-28T03:48:02.320477+0000 front 2024-03-28T03:48:02.320512+0000 (oldest deadline 2024-03-28T03:48:27.020650+0000) 2024-03-28T03:48:35.629 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:35 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:35.214+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.119:6822 osd.6 since back 2024-03-28T03:48:07.021607+0000 front 2024-03-28T03:48:07.021738+0000 (oldest deadline 2024-03-28T03:48:32.821858+0000) 2024-03-28T03:48:35.902 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:35 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:48:35.506+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:19.734645+0000 front 2024-03-28T03:47:19.734461+0000 (oldest deadline 2024-03-28T03:47:44.434325+0000) 2024-03-28T03:48:35.902 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:35 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:48:35.506+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:25.537050+0000 front 2024-03-28T03:47:25.536933+0000 (oldest deadline 2024-03-28T03:47:51.437142+0000) 2024-03-28T03:48:35.902 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:35 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:48:35.506+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:32.539643+0000 front 2024-03-28T03:47:32.539558+0000 (oldest deadline 2024-03-28T03:47:58.439707+0000) 2024-03-28T03:48:35.902 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:35 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:48:35.506+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2024-03-28T03:47:43.141824+0000 front 2024-03-28T03:47:43.142146+0000 (oldest deadline 2024-03-28T03:48:08.441984+0000) 2024-03-28T03:48:35.902 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:35 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:48:35.506+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-03-28T03:48:01.048279+0000 front 2024-03-28T03:48:01.048186+0000 (oldest deadline 2024-03-28T03:48:24.548073+0000) 2024-03-28T03:48:35.902 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:35 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9[63602]: 2024-03-28T03:48:35.506+0000 7f638e600700 -1 osd.9 128 heartbeat_check: no reply from 172.21.15.119:6822 osd.6 since back 2024-03-28T03:48:09.151149+0000 front 2024-03-28T03:48:09.151346+0000 (oldest deadline 2024-03-28T03:48:35.050759+0000) 2024-03-28T03:48:35.948 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:35 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:35.550+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:18.649888+0000 front 2024-03-28T03:47:18.649830+0000 (oldest deadline 2024-03-28T03:47:42.749644+0000) 2024-03-28T03:48:35.948 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:35 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:35.550+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:29.151297+0000 front 2024-03-28T03:47:29.151274+0000 (oldest deadline 2024-03-28T03:47:50.152219+0000) 2024-03-28T03:48:35.949 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:35 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:35.550+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:34.853462+0000 front 2024-03-28T03:47:34.853609+0000 (oldest deadline 2024-03-28T03:47:58.353565+0000) 2024-03-28T03:48:35.949 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:35 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:35.550+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2024-03-28T03:47:44.155643+0000 front 2024-03-28T03:47:44.155546+0000 (oldest deadline 2024-03-28T03:48:09.455316+0000) 2024-03-28T03:48:35.949 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:35 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:35.550+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2024-03-28T03:47:52.356476+0000 front 2024-03-28T03:47:52.356457+0000 (oldest deadline 2024-03-28T03:48:17.056580+0000) 2024-03-28T03:48:35.949 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:35 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:35.550+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-03-28T03:47:57.557659+0000 front 2024-03-28T03:47:57.557559+0000 (oldest deadline 2024-03-28T03:48:23.457566+0000) 2024-03-28T03:48:35.949 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:35 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:35.550+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.119:6822 osd.6 since back 2024-03-28T03:48:06.958911+0000 front 2024-03-28T03:48:06.959040+0000 (oldest deadline 2024-03-28T03:48:32.258542+0000) 2024-03-28T03:48:36.297 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:36 smithi119 podman[173643]: 2024-03-28 03:48:36.011854036 +0000 UTC m=+5.584786701 container stop f3cd3d247249e026a5d430c978186a3b0665d02d3dc3f9c6b36252a8c51b3aeb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-28T03:48:36.298 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:36 smithi119 podman[173643]: 2024-03-28 03:48:36.037993196 +0000 UTC m=+5.610925932 container died f3cd3d247249e026a5d430c978186a3b0665d02d3dc3f9c6b36252a8c51b3aeb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-03-28T03:48:36.628 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:36 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:36.254+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:18.314545+0000 front 2024-03-28T03:47:18.314582+0000 (oldest deadline 2024-03-28T03:47:44.214430+0000) 2024-03-28T03:48:36.628 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:36 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:36.254+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:27.715675+0000 front 2024-03-28T03:47:27.715645+0000 (oldest deadline 2024-03-28T03:47:50.615667+0000) 2024-03-28T03:48:36.628 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:36 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:36.254+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:32.916784+0000 front 2024-03-28T03:47:32.916834+0000 (oldest deadline 2024-03-28T03:47:58.816929+0000) 2024-03-28T03:48:36.629 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:36 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:36.254+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2024-03-28T03:47:52.919580+0000 front 2024-03-28T03:47:52.919549+0000 (oldest deadline 2024-03-28T03:48:17.019575+0000) 2024-03-28T03:48:36.629 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:36 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:36.254+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-03-28T03:48:02.320477+0000 front 2024-03-28T03:48:02.320512+0000 (oldest deadline 2024-03-28T03:48:27.020650+0000) 2024-03-28T03:48:36.629 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:36 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:36.254+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.119:6822 osd.6 since back 2024-03-28T03:48:07.021607+0000 front 2024-03-28T03:48:07.021738+0000 (oldest deadline 2024-03-28T03:48:32.821858+0000) 2024-03-28T03:48:36.651 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:36 smithi119 podman[173643]: 2024-03-28 03:48:36.40760529 +0000 UTC m=+5.980537995 container cleanup f3cd3d247249e026a5d430c978186a3b0665d02d3dc3f9c6b36252a8c51b3aeb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS) 2024-03-28T03:48:36.652 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:36 smithi119 bash[173643]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9 2024-03-28T03:48:36.949 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:36 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:36.509+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:18.649888+0000 front 2024-03-28T03:47:18.649830+0000 (oldest deadline 2024-03-28T03:47:42.749644+0000) 2024-03-28T03:48:36.949 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:36 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:36.509+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:29.151297+0000 front 2024-03-28T03:47:29.151274+0000 (oldest deadline 2024-03-28T03:47:50.152219+0000) 2024-03-28T03:48:36.949 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:36 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:36.509+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:34.853462+0000 front 2024-03-28T03:47:34.853609+0000 (oldest deadline 2024-03-28T03:47:58.353565+0000) 2024-03-28T03:48:36.949 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:36 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:36.509+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2024-03-28T03:47:44.155643+0000 front 2024-03-28T03:47:44.155546+0000 (oldest deadline 2024-03-28T03:48:09.455316+0000) 2024-03-28T03:48:36.949 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:36 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:36.509+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2024-03-28T03:47:52.356476+0000 front 2024-03-28T03:47:52.356457+0000 (oldest deadline 2024-03-28T03:48:17.056580+0000) 2024-03-28T03:48:36.949 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:36 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:36.509+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-03-28T03:47:57.557659+0000 front 2024-03-28T03:47:57.557559+0000 (oldest deadline 2024-03-28T03:48:23.457566+0000) 2024-03-28T03:48:36.949 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:36 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:36.509+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.119:6822 osd.6 since back 2024-03-28T03:48:06.958911+0000 front 2024-03-28T03:48:06.959040+0000 (oldest deadline 2024-03-28T03:48:32.258542+0000) 2024-03-28T03:48:37.151 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:36 smithi119 podman[173661]: 2024-03-28 03:48:36.742414192 +0000 UTC m=+0.716327070 container remove f3cd3d247249e026a5d430c978186a3b0665d02d3dc3f9c6b36252a8c51b3aeb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c) 2024-03-28T03:48:37.607 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:37 smithi119 podman[173745]: 2024-03-28 03:48:37.252134542 +0000 UTC m=+0.036308987 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-28T03:48:37.608 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:37 smithi119 podman[173745]: 2024-03-28 03:48:37.359731322 +0000 UTC m=+0.143905790 container create 5a961f5d317a0188d0ae400f852a2c2302bf72c3fd712b2d896c8096340fe67c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9-deactivate, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-03-28T03:48:37.608 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:37 smithi119 podman[173745]: 2024-03-28 03:48:37.605513009 +0000 UTC m=+0.389687484 container init 5a961f5d317a0188d0ae400f852a2c2302bf72c3fd712b2d896c8096340fe67c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9-deactivate, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-28T03:48:37.628 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:37.235+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:18.314545+0000 front 2024-03-28T03:47:18.314582+0000 (oldest deadline 2024-03-28T03:47:44.214430+0000) 2024-03-28T03:48:37.628 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:37.235+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:27.715675+0000 front 2024-03-28T03:47:27.715645+0000 (oldest deadline 2024-03-28T03:47:50.615667+0000) 2024-03-28T03:48:37.628 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:37.235+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:32.916784+0000 front 2024-03-28T03:47:32.916834+0000 (oldest deadline 2024-03-28T03:47:58.816929+0000) 2024-03-28T03:48:37.628 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:37.235+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2024-03-28T03:47:52.919580+0000 front 2024-03-28T03:47:52.919549+0000 (oldest deadline 2024-03-28T03:48:17.019575+0000) 2024-03-28T03:48:37.629 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:37.235+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-03-28T03:48:02.320477+0000 front 2024-03-28T03:48:02.320512+0000 (oldest deadline 2024-03-28T03:48:27.020650+0000) 2024-03-28T03:48:37.629 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:37 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:37.235+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.119:6822 osd.6 since back 2024-03-28T03:48:07.021607+0000 front 2024-03-28T03:48:07.021738+0000 (oldest deadline 2024-03-28T03:48:32.821858+0000) 2024-03-28T03:48:37.901 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:37 smithi119 podman[173745]: 2024-03-28 03:48:37.616848338 +0000 UTC m=+0.401022813 container start 5a961f5d317a0188d0ae400f852a2c2302bf72c3fd712b2d896c8096340fe67c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9-deactivate, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-03-28T03:48:37.901 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:37 smithi119 podman[173745]: 2024-03-28 03:48:37.703749491 +0000 UTC m=+0.487923937 container attach 5a961f5d317a0188d0ae400f852a2c2302bf72c3fd712b2d896c8096340fe67c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9-deactivate, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-03-28T03:48:37.948 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:37 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:37.480+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:18.649888+0000 front 2024-03-28T03:47:18.649830+0000 (oldest deadline 2024-03-28T03:47:42.749644+0000) 2024-03-28T03:48:37.949 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:37 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:37.480+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:29.151297+0000 front 2024-03-28T03:47:29.151274+0000 (oldest deadline 2024-03-28T03:47:50.152219+0000) 2024-03-28T03:48:37.949 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:37 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:37.480+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:34.853462+0000 front 2024-03-28T03:47:34.853609+0000 (oldest deadline 2024-03-28T03:47:58.353565+0000) 2024-03-28T03:48:37.949 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:37 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:37.480+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2024-03-28T03:47:44.155643+0000 front 2024-03-28T03:47:44.155546+0000 (oldest deadline 2024-03-28T03:48:09.455316+0000) 2024-03-28T03:48:37.949 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:37 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:37.480+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2024-03-28T03:47:52.356476+0000 front 2024-03-28T03:47:52.356457+0000 (oldest deadline 2024-03-28T03:48:17.056580+0000) 2024-03-28T03:48:37.949 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:37 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:37.480+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-03-28T03:47:57.557659+0000 front 2024-03-28T03:47:57.557559+0000 (oldest deadline 2024-03-28T03:48:23.457566+0000) 2024-03-28T03:48:37.949 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:37 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:37.480+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.119:6822 osd.6 since back 2024-03-28T03:48:06.958911+0000 front 2024-03-28T03:48:06.959040+0000 (oldest deadline 2024-03-28T03:48:32.258542+0000) 2024-03-28T03:48:38.342 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:38 smithi119 podman[173745]: 2024-03-28 03:48:38.048165129 +0000 UTC m=+0.832339617 container died 5a961f5d317a0188d0ae400f852a2c2302bf72c3fd712b2d896c8096340fe67c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9-deactivate, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-03-28T03:48:38.530 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:38 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:38.207+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:18.314545+0000 front 2024-03-28T03:47:18.314582+0000 (oldest deadline 2024-03-28T03:47:44.214430+0000) 2024-03-28T03:48:38.530 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:38 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:38.207+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:27.715675+0000 front 2024-03-28T03:47:27.715645+0000 (oldest deadline 2024-03-28T03:47:50.615667+0000) 2024-03-28T03:48:38.530 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:38 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:38.207+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:32.916784+0000 front 2024-03-28T03:47:32.916834+0000 (oldest deadline 2024-03-28T03:47:58.816929+0000) 2024-03-28T03:48:38.530 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:38 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:38.207+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2024-03-28T03:47:52.919580+0000 front 2024-03-28T03:47:52.919549+0000 (oldest deadline 2024-03-28T03:48:17.019575+0000) 2024-03-28T03:48:38.531 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:38 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:38.207+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-03-28T03:48:02.320477+0000 front 2024-03-28T03:48:02.320512+0000 (oldest deadline 2024-03-28T03:48:27.020650+0000) 2024-03-28T03:48:38.531 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:38 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:38.207+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.119:6822 osd.6 since back 2024-03-28T03:48:07.021607+0000 front 2024-03-28T03:48:07.021738+0000 (oldest deadline 2024-03-28T03:48:32.821858+0000) 2024-03-28T03:48:38.600 INFO:journalctl@ceph.osd.9.smithi119.stdout:Mar 28 03:48:38 smithi119 podman[173771]: 2024-03-28 03:48:38.573056251 +0000 UTC m=+0.509966181 container remove 5a961f5d317a0188d0ae400f852a2c2302bf72c3fd712b2d896c8096340fe67c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-9-deactivate, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-03-28T03:48:38.615 DEBUG:teuthology.orchestra.run.smithi119:> sudo pkill -f 'journalctl -f -n 0 -u ceph-10f73b92-ec98-11ee-95d0-87774f69a715@osd.9.service' 2024-03-28T03:48:38.699 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:38 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:38.444+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:18.649888+0000 front 2024-03-28T03:47:18.649830+0000 (oldest deadline 2024-03-28T03:47:42.749644+0000) 2024-03-28T03:48:38.699 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:38 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:38.444+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:29.151297+0000 front 2024-03-28T03:47:29.151274+0000 (oldest deadline 2024-03-28T03:47:50.152219+0000) 2024-03-28T03:48:38.699 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:38 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:38.444+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:34.853462+0000 front 2024-03-28T03:47:34.853609+0000 (oldest deadline 2024-03-28T03:47:58.353565+0000) 2024-03-28T03:48:38.699 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:38 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:38.444+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2024-03-28T03:47:44.155643+0000 front 2024-03-28T03:47:44.155546+0000 (oldest deadline 2024-03-28T03:48:09.455316+0000) 2024-03-28T03:48:38.699 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:38 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:38.444+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2024-03-28T03:47:52.356476+0000 front 2024-03-28T03:47:52.356457+0000 (oldest deadline 2024-03-28T03:48:17.056580+0000) 2024-03-28T03:48:38.699 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:38 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:38.444+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-03-28T03:47:57.557659+0000 front 2024-03-28T03:47:57.557559+0000 (oldest deadline 2024-03-28T03:48:23.457566+0000) 2024-03-28T03:48:38.699 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:38 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:38.444+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.119:6822 osd.6 since back 2024-03-28T03:48:06.958911+0000 front 2024-03-28T03:48:06.959040+0000 (oldest deadline 2024-03-28T03:48:32.258542+0000) 2024-03-28T03:48:38.734 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-28T03:48:38.734 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-03-28T03:48:38.734 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-03-28T03:48:38.735 DEBUG:teuthology.orchestra.run.smithi132:> sudo systemctl stop ceph-10f73b92-ec98-11ee-95d0-87774f69a715@osd.10 2024-03-28T03:48:39.066 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:38 smithi132 systemd[1]: Stopping Ceph osd.10 for 10f73b92-ec98-11ee-95d0-87774f69a715... 2024-03-28T03:48:39.426 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:39 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:39.065+0000 7f0ea3f59700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-28T03:48:39.426 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:39 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:39.065+0000 7f0ea3f59700 -1 osd.10 128 *** Got signal Terminated *** 2024-03-28T03:48:39.426 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:39 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:39.065+0000 7f0ea3f59700 -1 osd.10 128 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-28T03:48:39.426 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:39 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:39.424+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:18.649888+0000 front 2024-03-28T03:47:18.649830+0000 (oldest deadline 2024-03-28T03:47:42.749644+0000) 2024-03-28T03:48:39.426 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:39 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:39.424+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:29.151297+0000 front 2024-03-28T03:47:29.151274+0000 (oldest deadline 2024-03-28T03:47:50.152219+0000) 2024-03-28T03:48:39.427 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:39 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:39.424+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:34.853462+0000 front 2024-03-28T03:47:34.853609+0000 (oldest deadline 2024-03-28T03:47:58.353565+0000) 2024-03-28T03:48:39.427 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:39 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:39.424+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2024-03-28T03:47:44.155643+0000 front 2024-03-28T03:47:44.155546+0000 (oldest deadline 2024-03-28T03:48:09.455316+0000) 2024-03-28T03:48:39.427 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:39 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:39.424+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2024-03-28T03:47:52.356476+0000 front 2024-03-28T03:47:52.356457+0000 (oldest deadline 2024-03-28T03:48:17.056580+0000) 2024-03-28T03:48:39.427 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:39 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:39.424+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-03-28T03:47:57.557659+0000 front 2024-03-28T03:47:57.557559+0000 (oldest deadline 2024-03-28T03:48:23.457566+0000) 2024-03-28T03:48:39.427 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:39 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:39.424+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.119:6822 osd.6 since back 2024-03-28T03:48:06.958911+0000 front 2024-03-28T03:48:06.959040+0000 (oldest deadline 2024-03-28T03:48:32.258542+0000) 2024-03-28T03:48:39.628 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:39 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:39.209+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:18.314545+0000 front 2024-03-28T03:47:18.314582+0000 (oldest deadline 2024-03-28T03:47:44.214430+0000) 2024-03-28T03:48:39.628 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:39 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:39.209+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:27.715675+0000 front 2024-03-28T03:47:27.715645+0000 (oldest deadline 2024-03-28T03:47:50.615667+0000) 2024-03-28T03:48:39.629 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:39 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:39.209+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:32.916784+0000 front 2024-03-28T03:47:32.916834+0000 (oldest deadline 2024-03-28T03:47:58.816929+0000) 2024-03-28T03:48:39.629 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:39 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:39.209+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2024-03-28T03:47:52.919580+0000 front 2024-03-28T03:47:52.919549+0000 (oldest deadline 2024-03-28T03:48:17.019575+0000) 2024-03-28T03:48:39.629 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:39 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:39.209+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-03-28T03:48:02.320477+0000 front 2024-03-28T03:48:02.320512+0000 (oldest deadline 2024-03-28T03:48:27.020650+0000) 2024-03-28T03:48:39.629 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:39 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:39.209+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.119:6822 osd.6 since back 2024-03-28T03:48:07.021607+0000 front 2024-03-28T03:48:07.021738+0000 (oldest deadline 2024-03-28T03:48:32.821858+0000) 2024-03-28T03:48:40.628 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:40 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:40.211+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:18.314545+0000 front 2024-03-28T03:47:18.314582+0000 (oldest deadline 2024-03-28T03:47:44.214430+0000) 2024-03-28T03:48:40.628 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:40 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:40.211+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:27.715675+0000 front 2024-03-28T03:47:27.715645+0000 (oldest deadline 2024-03-28T03:47:50.615667+0000) 2024-03-28T03:48:40.629 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:40 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:40.211+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:32.916784+0000 front 2024-03-28T03:47:32.916834+0000 (oldest deadline 2024-03-28T03:47:58.816929+0000) 2024-03-28T03:48:40.629 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:40 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:40.211+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2024-03-28T03:47:52.919580+0000 front 2024-03-28T03:47:52.919549+0000 (oldest deadline 2024-03-28T03:48:17.019575+0000) 2024-03-28T03:48:40.629 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:40 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:40.211+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-03-28T03:48:02.320477+0000 front 2024-03-28T03:48:02.320512+0000 (oldest deadline 2024-03-28T03:48:27.020650+0000) 2024-03-28T03:48:40.629 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:40 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:40.211+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.119:6822 osd.6 since back 2024-03-28T03:48:07.021607+0000 front 2024-03-28T03:48:07.021738+0000 (oldest deadline 2024-03-28T03:48:32.821858+0000) 2024-03-28T03:48:40.698 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:40 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:40.445+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:18.649888+0000 front 2024-03-28T03:47:18.649830+0000 (oldest deadline 2024-03-28T03:47:42.749644+0000) 2024-03-28T03:48:40.698 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:40 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:40.445+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:29.151297+0000 front 2024-03-28T03:47:29.151274+0000 (oldest deadline 2024-03-28T03:47:50.152219+0000) 2024-03-28T03:48:40.698 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:40 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:40.445+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:34.853462+0000 front 2024-03-28T03:47:34.853609+0000 (oldest deadline 2024-03-28T03:47:58.353565+0000) 2024-03-28T03:48:40.699 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:40 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:40.445+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2024-03-28T03:47:44.155643+0000 front 2024-03-28T03:47:44.155546+0000 (oldest deadline 2024-03-28T03:48:09.455316+0000) 2024-03-28T03:48:40.699 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:40 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:40.445+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2024-03-28T03:47:52.356476+0000 front 2024-03-28T03:47:52.356457+0000 (oldest deadline 2024-03-28T03:48:17.056580+0000) 2024-03-28T03:48:40.699 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:40 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:40.445+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-03-28T03:47:57.557659+0000 front 2024-03-28T03:47:57.557559+0000 (oldest deadline 2024-03-28T03:48:23.457566+0000) 2024-03-28T03:48:40.699 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:40 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:40.445+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.119:6822 osd.6 since back 2024-03-28T03:48:06.958911+0000 front 2024-03-28T03:48:06.959040+0000 (oldest deadline 2024-03-28T03:48:32.258542+0000) 2024-03-28T03:48:41.628 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:41 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:41.220+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:18.314545+0000 front 2024-03-28T03:47:18.314582+0000 (oldest deadline 2024-03-28T03:47:44.214430+0000) 2024-03-28T03:48:41.628 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:41 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:41.220+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:27.715675+0000 front 2024-03-28T03:47:27.715645+0000 (oldest deadline 2024-03-28T03:47:50.615667+0000) 2024-03-28T03:48:41.629 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:41 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:41.220+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:32.916784+0000 front 2024-03-28T03:47:32.916834+0000 (oldest deadline 2024-03-28T03:47:58.816929+0000) 2024-03-28T03:48:41.629 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:41 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:41.220+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2024-03-28T03:47:52.919580+0000 front 2024-03-28T03:47:52.919549+0000 (oldest deadline 2024-03-28T03:48:17.019575+0000) 2024-03-28T03:48:41.629 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:41 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:41.220+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-03-28T03:48:02.320477+0000 front 2024-03-28T03:48:02.320512+0000 (oldest deadline 2024-03-28T03:48:27.020650+0000) 2024-03-28T03:48:41.629 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:41 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:41.220+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.119:6822 osd.6 since back 2024-03-28T03:48:07.021607+0000 front 2024-03-28T03:48:07.021738+0000 (oldest deadline 2024-03-28T03:48:32.821858+0000) 2024-03-28T03:48:41.698 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:41 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:41.425+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:18.649888+0000 front 2024-03-28T03:47:18.649830+0000 (oldest deadline 2024-03-28T03:47:42.749644+0000) 2024-03-28T03:48:41.698 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:41 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:41.425+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:29.151297+0000 front 2024-03-28T03:47:29.151274+0000 (oldest deadline 2024-03-28T03:47:50.152219+0000) 2024-03-28T03:48:41.698 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:41 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:41.425+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:34.853462+0000 front 2024-03-28T03:47:34.853609+0000 (oldest deadline 2024-03-28T03:47:58.353565+0000) 2024-03-28T03:48:41.699 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:41 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:41.425+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2024-03-28T03:47:44.155643+0000 front 2024-03-28T03:47:44.155546+0000 (oldest deadline 2024-03-28T03:48:09.455316+0000) 2024-03-28T03:48:41.699 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:41 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:41.425+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2024-03-28T03:47:52.356476+0000 front 2024-03-28T03:47:52.356457+0000 (oldest deadline 2024-03-28T03:48:17.056580+0000) 2024-03-28T03:48:41.699 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:41 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:41.425+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-03-28T03:47:57.557659+0000 front 2024-03-28T03:47:57.557559+0000 (oldest deadline 2024-03-28T03:48:23.457566+0000) 2024-03-28T03:48:41.699 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:41 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:41.425+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.119:6822 osd.6 since back 2024-03-28T03:48:06.958911+0000 front 2024-03-28T03:48:06.959040+0000 (oldest deadline 2024-03-28T03:48:32.258542+0000) 2024-03-28T03:48:42.628 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:42 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:42.201+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:18.314545+0000 front 2024-03-28T03:47:18.314582+0000 (oldest deadline 2024-03-28T03:47:44.214430+0000) 2024-03-28T03:48:42.628 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:42 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:42.201+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:27.715675+0000 front 2024-03-28T03:47:27.715645+0000 (oldest deadline 2024-03-28T03:47:50.615667+0000) 2024-03-28T03:48:42.629 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:42 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:42.201+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:32.916784+0000 front 2024-03-28T03:47:32.916834+0000 (oldest deadline 2024-03-28T03:47:58.816929+0000) 2024-03-28T03:48:42.629 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:42 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:42.201+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2024-03-28T03:47:52.919580+0000 front 2024-03-28T03:47:52.919549+0000 (oldest deadline 2024-03-28T03:48:17.019575+0000) 2024-03-28T03:48:42.629 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:42 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:42.201+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-03-28T03:48:02.320477+0000 front 2024-03-28T03:48:02.320512+0000 (oldest deadline 2024-03-28T03:48:27.020650+0000) 2024-03-28T03:48:42.629 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:42 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:42.201+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.119:6822 osd.6 since back 2024-03-28T03:48:07.021607+0000 front 2024-03-28T03:48:07.021738+0000 (oldest deadline 2024-03-28T03:48:32.821858+0000) 2024-03-28T03:48:42.698 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:42 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:42.444+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:18.649888+0000 front 2024-03-28T03:47:18.649830+0000 (oldest deadline 2024-03-28T03:47:42.749644+0000) 2024-03-28T03:48:42.699 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:42 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:42.444+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:29.151297+0000 front 2024-03-28T03:47:29.151274+0000 (oldest deadline 2024-03-28T03:47:50.152219+0000) 2024-03-28T03:48:42.699 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:42 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:42.444+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:34.853462+0000 front 2024-03-28T03:47:34.853609+0000 (oldest deadline 2024-03-28T03:47:58.353565+0000) 2024-03-28T03:48:42.699 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:42 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:42.444+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2024-03-28T03:47:44.155643+0000 front 2024-03-28T03:47:44.155546+0000 (oldest deadline 2024-03-28T03:48:09.455316+0000) 2024-03-28T03:48:42.699 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:42 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:42.444+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2024-03-28T03:47:52.356476+0000 front 2024-03-28T03:47:52.356457+0000 (oldest deadline 2024-03-28T03:48:17.056580+0000) 2024-03-28T03:48:42.699 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:42 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:42.444+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-03-28T03:47:57.557659+0000 front 2024-03-28T03:47:57.557559+0000 (oldest deadline 2024-03-28T03:48:23.457566+0000) 2024-03-28T03:48:42.699 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:42 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:42.444+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.119:6822 osd.6 since back 2024-03-28T03:48:06.958911+0000 front 2024-03-28T03:48:06.959040+0000 (oldest deadline 2024-03-28T03:48:32.258542+0000) 2024-03-28T03:48:43.628 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:43.245+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:18.314545+0000 front 2024-03-28T03:47:18.314582+0000 (oldest deadline 2024-03-28T03:47:44.214430+0000) 2024-03-28T03:48:43.629 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:43.245+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:27.715675+0000 front 2024-03-28T03:47:27.715645+0000 (oldest deadline 2024-03-28T03:47:50.615667+0000) 2024-03-28T03:48:43.629 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:43.245+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:32.916784+0000 front 2024-03-28T03:47:32.916834+0000 (oldest deadline 2024-03-28T03:47:58.816929+0000) 2024-03-28T03:48:43.629 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:43.245+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2024-03-28T03:47:52.919580+0000 front 2024-03-28T03:47:52.919549+0000 (oldest deadline 2024-03-28T03:48:17.019575+0000) 2024-03-28T03:48:43.629 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:43.245+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-03-28T03:48:02.320477+0000 front 2024-03-28T03:48:02.320512+0000 (oldest deadline 2024-03-28T03:48:27.020650+0000) 2024-03-28T03:48:43.629 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:43 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:43.245+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.119:6822 osd.6 since back 2024-03-28T03:48:07.021607+0000 front 2024-03-28T03:48:07.021738+0000 (oldest deadline 2024-03-28T03:48:32.821858+0000) 2024-03-28T03:48:43.699 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:43 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:43.407+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:18.649888+0000 front 2024-03-28T03:47:18.649830+0000 (oldest deadline 2024-03-28T03:47:42.749644+0000) 2024-03-28T03:48:43.699 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:43 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:43.407+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:29.151297+0000 front 2024-03-28T03:47:29.151274+0000 (oldest deadline 2024-03-28T03:47:50.152219+0000) 2024-03-28T03:48:43.699 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:43 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:43.407+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:34.853462+0000 front 2024-03-28T03:47:34.853609+0000 (oldest deadline 2024-03-28T03:47:58.353565+0000) 2024-03-28T03:48:43.699 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:43 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:43.407+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2024-03-28T03:47:44.155643+0000 front 2024-03-28T03:47:44.155546+0000 (oldest deadline 2024-03-28T03:48:09.455316+0000) 2024-03-28T03:48:43.699 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:43 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:43.407+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2024-03-28T03:47:52.356476+0000 front 2024-03-28T03:47:52.356457+0000 (oldest deadline 2024-03-28T03:48:17.056580+0000) 2024-03-28T03:48:43.700 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:43 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:43.407+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-03-28T03:47:57.557659+0000 front 2024-03-28T03:47:57.557559+0000 (oldest deadline 2024-03-28T03:48:23.457566+0000) 2024-03-28T03:48:43.700 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:43 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10[61030]: 2024-03-28T03:48:43.407+0000 7f0ea054d700 -1 osd.10 128 heartbeat_check: no reply from 172.21.15.119:6822 osd.6 since back 2024-03-28T03:48:06.958911+0000 front 2024-03-28T03:48:06.959040+0000 (oldest deadline 2024-03-28T03:48:32.258542+0000) 2024-03-28T03:48:44.628 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:44.277+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:18.314545+0000 front 2024-03-28T03:47:18.314582+0000 (oldest deadline 2024-03-28T03:47:44.214430+0000) 2024-03-28T03:48:44.629 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:44.277+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:27.715675+0000 front 2024-03-28T03:47:27.715645+0000 (oldest deadline 2024-03-28T03:47:50.615667+0000) 2024-03-28T03:48:44.629 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:44.277+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:32.916784+0000 front 2024-03-28T03:47:32.916834+0000 (oldest deadline 2024-03-28T03:47:58.816929+0000) 2024-03-28T03:48:44.629 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:44.277+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2024-03-28T03:47:52.919580+0000 front 2024-03-28T03:47:52.919549+0000 (oldest deadline 2024-03-28T03:48:17.019575+0000) 2024-03-28T03:48:44.629 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:44.277+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-03-28T03:48:02.320477+0000 front 2024-03-28T03:48:02.320512+0000 (oldest deadline 2024-03-28T03:48:27.020650+0000) 2024-03-28T03:48:44.629 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:44.277+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.119:6822 osd.6 since back 2024-03-28T03:48:07.021607+0000 front 2024-03-28T03:48:07.021738+0000 (oldest deadline 2024-03-28T03:48:32.821858+0000) 2024-03-28T03:48:44.629 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:44 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:44.277+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.132:6820 osd.7 since back 2024-03-28T03:48:19.223458+0000 front 2024-03-28T03:48:19.223414+0000 (oldest deadline 2024-03-28T03:48:43.323381+0000) 2024-03-28T03:48:44.776 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:44 smithi132 podman[121176]: 2024-03-28 03:48:44.49773545 +0000 UTC m=+5.562787112 container died c7cc65528546a3d9e2de1393f4918011d615daa1992a243b44b20f09fb77a532 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-03-28T03:48:45.084 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:44 smithi132 podman[121176]: 2024-03-28 03:48:44.873439043 +0000 UTC m=+5.938490697 container cleanup c7cc65528546a3d9e2de1393f4918011d615daa1992a243b44b20f09fb77a532 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c) 2024-03-28T03:48:45.084 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:44 smithi132 bash[121176]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10 2024-03-28T03:48:45.399 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:45 smithi132 podman[121205]: 2024-03-28 03:48:45.082951822 +0000 UTC m=+0.585792524 container remove c7cc65528546a3d9e2de1393f4918011d615daa1992a243b44b20f09fb77a532 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-03-28T03:48:45.628 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:45 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:45.282+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:18.314545+0000 front 2024-03-28T03:47:18.314582+0000 (oldest deadline 2024-03-28T03:47:44.214430+0000) 2024-03-28T03:48:45.629 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:45 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:45.282+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:27.715675+0000 front 2024-03-28T03:47:27.715645+0000 (oldest deadline 2024-03-28T03:47:50.615667+0000) 2024-03-28T03:48:45.629 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:45 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:45.282+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:32.916784+0000 front 2024-03-28T03:47:32.916834+0000 (oldest deadline 2024-03-28T03:47:58.816929+0000) 2024-03-28T03:48:45.629 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:45 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:45.282+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2024-03-28T03:47:52.919580+0000 front 2024-03-28T03:47:52.919549+0000 (oldest deadline 2024-03-28T03:48:17.019575+0000) 2024-03-28T03:48:45.629 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:45 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:45.282+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-03-28T03:48:02.320477+0000 front 2024-03-28T03:48:02.320512+0000 (oldest deadline 2024-03-28T03:48:27.020650+0000) 2024-03-28T03:48:45.629 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:45 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:45.282+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.119:6822 osd.6 since back 2024-03-28T03:48:07.021607+0000 front 2024-03-28T03:48:07.021738+0000 (oldest deadline 2024-03-28T03:48:32.821858+0000) 2024-03-28T03:48:45.629 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:45 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:45.282+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.132:6820 osd.7 since back 2024-03-28T03:48:19.223458+0000 front 2024-03-28T03:48:19.223414+0000 (oldest deadline 2024-03-28T03:48:43.323381+0000) 2024-03-28T03:48:45.670 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:45 smithi132 podman[121297]: 2024-03-28 03:48:45.301372016 +0000 UTC m=+0.017383657 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-28T03:48:45.670 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:45 smithi132 podman[121297]: 2024-03-28 03:48:45.40026147 +0000 UTC m=+0.116273108 container create 0fb80d867204e025bdd4b68c6b743d7f0ed60e908e519b2b0b50a2b1d22dfb0d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10-deactivate, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c) 2024-03-28T03:48:45.670 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:45 smithi132 podman[121297]: 2024-03-28 03:48:45.594706516 +0000 UTC m=+0.310718156 container init 0fb80d867204e025bdd4b68c6b743d7f0ed60e908e519b2b0b50a2b1d22dfb0d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10-deactivate, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-03-28T03:48:45.671 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:45 smithi132 podman[121297]: 2024-03-28 03:48:45.598343014 +0000 UTC m=+0.314354657 container start 0fb80d867204e025bdd4b68c6b743d7f0ed60e908e519b2b0b50a2b1d22dfb0d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-03-28T03:48:45.671 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:45 smithi132 podman[121297]: 2024-03-28 03:48:45.669374795 +0000 UTC m=+0.385386444 container attach 0fb80d867204e025bdd4b68c6b743d7f0ed60e908e519b2b0b50a2b1d22dfb0d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10-deactivate, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-03-28T03:48:45.948 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:45 smithi132 podman[121297]: 2024-03-28 03:48:45.883663195 +0000 UTC m=+0.599674829 container died 0fb80d867204e025bdd4b68c6b743d7f0ed60e908e519b2b0b50a2b1d22dfb0d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10-deactivate, org.label-schema.license=GPLv2, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True) 2024-03-28T03:48:46.428 INFO:journalctl@ceph.osd.10.smithi132.stdout:Mar 28 03:48:46 smithi132 podman[121323]: 2024-03-28 03:48:46.406080407 +0000 UTC m=+0.517109301 container remove 0fb80d867204e025bdd4b68c6b743d7f0ed60e908e519b2b0b50a2b1d22dfb0d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-10-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-03-28T03:48:46.433 DEBUG:teuthology.orchestra.run.smithi132:> sudo pkill -f 'journalctl -f -n 0 -u ceph-10f73b92-ec98-11ee-95d0-87774f69a715@osd.10.service' 2024-03-28T03:48:46.474 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-28T03:48:46.474 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-03-28T03:48:46.474 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-03-28T03:48:46.474 DEBUG:teuthology.orchestra.run.smithi165:> sudo systemctl stop ceph-10f73b92-ec98-11ee-95d0-87774f69a715@osd.11 2024-03-28T03:48:46.628 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:46 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:46.280+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:18.314545+0000 front 2024-03-28T03:47:18.314582+0000 (oldest deadline 2024-03-28T03:47:44.214430+0000) 2024-03-28T03:48:46.629 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:46 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:46.280+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:27.715675+0000 front 2024-03-28T03:47:27.715645+0000 (oldest deadline 2024-03-28T03:47:50.615667+0000) 2024-03-28T03:48:46.629 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:46 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:46.280+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:32.916784+0000 front 2024-03-28T03:47:32.916834+0000 (oldest deadline 2024-03-28T03:47:58.816929+0000) 2024-03-28T03:48:46.629 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:46 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:46.280+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2024-03-28T03:47:52.919580+0000 front 2024-03-28T03:47:52.919549+0000 (oldest deadline 2024-03-28T03:48:17.019575+0000) 2024-03-28T03:48:46.629 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:46 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:46.280+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-03-28T03:48:02.320477+0000 front 2024-03-28T03:48:02.320512+0000 (oldest deadline 2024-03-28T03:48:27.020650+0000) 2024-03-28T03:48:46.629 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:46 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:46.280+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.119:6822 osd.6 since back 2024-03-28T03:48:07.021607+0000 front 2024-03-28T03:48:07.021738+0000 (oldest deadline 2024-03-28T03:48:32.821858+0000) 2024-03-28T03:48:46.629 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:46 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:46.280+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.132:6820 osd.7 since back 2024-03-28T03:48:19.223458+0000 front 2024-03-28T03:48:19.223414+0000 (oldest deadline 2024-03-28T03:48:43.323381+0000) 2024-03-28T03:48:46.629 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:46 smithi165 systemd[1]: Stopping Ceph osd.11 for 10f73b92-ec98-11ee-95d0-87774f69a715... 2024-03-28T03:48:47.128 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:46 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:46.786+0000 7f38ae519700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-28T03:48:47.128 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:46 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:46.786+0000 7f38ae519700 -1 osd.11 128 *** Got signal Terminated *** 2024-03-28T03:48:47.128 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:46 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:46.786+0000 7f38ae519700 -1 osd.11 128 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-28T03:48:47.628 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:47 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:47.283+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:18.314545+0000 front 2024-03-28T03:47:18.314582+0000 (oldest deadline 2024-03-28T03:47:44.214430+0000) 2024-03-28T03:48:47.628 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:47 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:47.283+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:27.715675+0000 front 2024-03-28T03:47:27.715645+0000 (oldest deadline 2024-03-28T03:47:50.615667+0000) 2024-03-28T03:48:47.629 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:47 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:47.283+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:32.916784+0000 front 2024-03-28T03:47:32.916834+0000 (oldest deadline 2024-03-28T03:47:58.816929+0000) 2024-03-28T03:48:47.629 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:47 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:47.283+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2024-03-28T03:47:52.919580+0000 front 2024-03-28T03:47:52.919549+0000 (oldest deadline 2024-03-28T03:48:17.019575+0000) 2024-03-28T03:48:47.629 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:47 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:47.283+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-03-28T03:48:02.320477+0000 front 2024-03-28T03:48:02.320512+0000 (oldest deadline 2024-03-28T03:48:27.020650+0000) 2024-03-28T03:48:47.629 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:47 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:47.283+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.119:6822 osd.6 since back 2024-03-28T03:48:07.021607+0000 front 2024-03-28T03:48:07.021738+0000 (oldest deadline 2024-03-28T03:48:32.821858+0000) 2024-03-28T03:48:47.629 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:47 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:47.283+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.132:6820 osd.7 since back 2024-03-28T03:48:19.223458+0000 front 2024-03-28T03:48:19.223414+0000 (oldest deadline 2024-03-28T03:48:43.323381+0000) 2024-03-28T03:48:48.580 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:48 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:48.298+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:18.314545+0000 front 2024-03-28T03:47:18.314582+0000 (oldest deadline 2024-03-28T03:47:44.214430+0000) 2024-03-28T03:48:48.581 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:48 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:48.298+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:27.715675+0000 front 2024-03-28T03:47:27.715645+0000 (oldest deadline 2024-03-28T03:47:50.615667+0000) 2024-03-28T03:48:48.581 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:48 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:48.298+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:32.916784+0000 front 2024-03-28T03:47:32.916834+0000 (oldest deadline 2024-03-28T03:47:58.816929+0000) 2024-03-28T03:48:48.581 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:48 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:48.298+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2024-03-28T03:47:52.919580+0000 front 2024-03-28T03:47:52.919549+0000 (oldest deadline 2024-03-28T03:48:17.019575+0000) 2024-03-28T03:48:48.581 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:48 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:48.298+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-03-28T03:48:02.320477+0000 front 2024-03-28T03:48:02.320512+0000 (oldest deadline 2024-03-28T03:48:27.020650+0000) 2024-03-28T03:48:48.581 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:48 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:48.298+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.119:6822 osd.6 since back 2024-03-28T03:48:07.021607+0000 front 2024-03-28T03:48:07.021738+0000 (oldest deadline 2024-03-28T03:48:32.821858+0000) 2024-03-28T03:48:48.581 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:48 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:48.298+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.132:6820 osd.7 since back 2024-03-28T03:48:19.223458+0000 front 2024-03-28T03:48:19.223414+0000 (oldest deadline 2024-03-28T03:48:43.323381+0000) 2024-03-28T03:48:49.628 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:49 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:49.317+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:18.314545+0000 front 2024-03-28T03:47:18.314582+0000 (oldest deadline 2024-03-28T03:47:44.214430+0000) 2024-03-28T03:48:49.629 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:49 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:49.317+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:27.715675+0000 front 2024-03-28T03:47:27.715645+0000 (oldest deadline 2024-03-28T03:47:50.615667+0000) 2024-03-28T03:48:49.629 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:49 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:49.317+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:32.916784+0000 front 2024-03-28T03:47:32.916834+0000 (oldest deadline 2024-03-28T03:47:58.816929+0000) 2024-03-28T03:48:49.629 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:49 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:49.317+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2024-03-28T03:47:52.919580+0000 front 2024-03-28T03:47:52.919549+0000 (oldest deadline 2024-03-28T03:48:17.019575+0000) 2024-03-28T03:48:49.629 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:49 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:49.317+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-03-28T03:48:02.320477+0000 front 2024-03-28T03:48:02.320512+0000 (oldest deadline 2024-03-28T03:48:27.020650+0000) 2024-03-28T03:48:49.629 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:49 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:49.317+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.119:6822 osd.6 since back 2024-03-28T03:48:07.021607+0000 front 2024-03-28T03:48:07.021738+0000 (oldest deadline 2024-03-28T03:48:32.821858+0000) 2024-03-28T03:48:49.629 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:49 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:49.317+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.132:6820 osd.7 since back 2024-03-28T03:48:19.223458+0000 front 2024-03-28T03:48:19.223414+0000 (oldest deadline 2024-03-28T03:48:43.323381+0000) 2024-03-28T03:48:49.629 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:49 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:49.317+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.165:6820 osd.8 since back 2024-03-28T03:48:23.323959+0000 front 2024-03-28T03:48:23.323986+0000 (oldest deadline 2024-03-28T03:48:49.223950+0000) 2024-03-28T03:48:50.628 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:50.330+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:18.314545+0000 front 2024-03-28T03:47:18.314582+0000 (oldest deadline 2024-03-28T03:47:44.214430+0000) 2024-03-28T03:48:50.629 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:50.330+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:27.715675+0000 front 2024-03-28T03:47:27.715645+0000 (oldest deadline 2024-03-28T03:47:50.615667+0000) 2024-03-28T03:48:50.629 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:50.330+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:32.916784+0000 front 2024-03-28T03:47:32.916834+0000 (oldest deadline 2024-03-28T03:47:58.816929+0000) 2024-03-28T03:48:50.629 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:50.330+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2024-03-28T03:47:52.919580+0000 front 2024-03-28T03:47:52.919549+0000 (oldest deadline 2024-03-28T03:48:17.019575+0000) 2024-03-28T03:48:50.629 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:50.330+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-03-28T03:48:02.320477+0000 front 2024-03-28T03:48:02.320512+0000 (oldest deadline 2024-03-28T03:48:27.020650+0000) 2024-03-28T03:48:50.629 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:50.330+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.119:6822 osd.6 since back 2024-03-28T03:48:07.021607+0000 front 2024-03-28T03:48:07.021738+0000 (oldest deadline 2024-03-28T03:48:32.821858+0000) 2024-03-28T03:48:50.629 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:50.330+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.132:6820 osd.7 since back 2024-03-28T03:48:19.223458+0000 front 2024-03-28T03:48:19.223414+0000 (oldest deadline 2024-03-28T03:48:43.323381+0000) 2024-03-28T03:48:50.629 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:50 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:50.330+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.165:6820 osd.8 since back 2024-03-28T03:48:23.323959+0000 front 2024-03-28T03:48:23.323986+0000 (oldest deadline 2024-03-28T03:48:49.223950+0000) 2024-03-28T03:48:51.628 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:51 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:51.367+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-03-28T03:47:18.314545+0000 front 2024-03-28T03:47:18.314582+0000 (oldest deadline 2024-03-28T03:47:44.214430+0000) 2024-03-28T03:48:51.629 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:51 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:51.367+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-03-28T03:47:27.715675+0000 front 2024-03-28T03:47:27.715645+0000 (oldest deadline 2024-03-28T03:47:50.615667+0000) 2024-03-28T03:48:51.629 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:51 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:51.367+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-03-28T03:47:32.916784+0000 front 2024-03-28T03:47:32.916834+0000 (oldest deadline 2024-03-28T03:47:58.816929+0000) 2024-03-28T03:48:51.629 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:51 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:51.367+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2024-03-28T03:47:52.919580+0000 front 2024-03-28T03:47:52.919549+0000 (oldest deadline 2024-03-28T03:48:17.019575+0000) 2024-03-28T03:48:51.629 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:51 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:51.367+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-03-28T03:48:02.320477+0000 front 2024-03-28T03:48:02.320512+0000 (oldest deadline 2024-03-28T03:48:27.020650+0000) 2024-03-28T03:48:51.629 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:51 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:51.367+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.119:6822 osd.6 since back 2024-03-28T03:48:07.021607+0000 front 2024-03-28T03:48:07.021738+0000 (oldest deadline 2024-03-28T03:48:32.821858+0000) 2024-03-28T03:48:51.629 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:51 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:51.367+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.132:6820 osd.7 since back 2024-03-28T03:48:19.223458+0000 front 2024-03-28T03:48:19.223414+0000 (oldest deadline 2024-03-28T03:48:43.323381+0000) 2024-03-28T03:48:51.629 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:51 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11[62293]: 2024-03-28T03:48:51.367+0000 7f38a6b05700 -1 osd.11 128 heartbeat_check: no reply from 172.21.15.165:6820 osd.8 since back 2024-03-28T03:48:23.323959+0000 front 2024-03-28T03:48:23.323986+0000 (oldest deadline 2024-03-28T03:48:49.223950+0000) 2024-03-28T03:48:52.628 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:52 smithi165 podman[124368]: 2024-03-28 03:48:52.325730148 +0000 UTC m=+5.649554046 container died ba1686891db3f1675f0faaa6d506b0a63b10840ee9ddfd4379b303ee661d3f80 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c) 2024-03-28T03:48:52.955 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:52 smithi165 podman[124368]: 2024-03-28 03:48:52.744921529 +0000 UTC m=+6.068745423 container cleanup ba1686891db3f1675f0faaa6d506b0a63b10840ee9ddfd4379b303ee661d3f80 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS) 2024-03-28T03:48:52.955 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:52 smithi165 bash[124368]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11 2024-03-28T03:48:53.347 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:52 smithi165 podman[124387]: 2024-03-28 03:48:52.954220399 +0000 UTC m=+0.627271631 container remove ba1686891db3f1675f0faaa6d506b0a63b10840ee9ddfd4379b303ee661d3f80 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-03-28T03:48:53.599 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:53 smithi165 podman[124476]: 2024-03-28 03:48:53.34668204 +0000 UTC m=+0.105253264 container create 3ea8f6b90b37dc7f3d2d1f3cf39a36dd34c8f5de1023e8d58569e9daca5031bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11-deactivate, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-03-28T03:48:53.600 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:53 smithi165 podman[124476]: 2024-03-28 03:48:53.259278606 +0000 UTC m=+0.017849788 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-28T03:48:53.600 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:53 smithi165 podman[124476]: 2024-03-28 03:48:53.52416377 +0000 UTC m=+0.282734951 container init 3ea8f6b90b37dc7f3d2d1f3cf39a36dd34c8f5de1023e8d58569e9daca5031bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11-deactivate, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-28T03:48:53.600 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:53 smithi165 podman[124476]: 2024-03-28 03:48:53.527365507 +0000 UTC m=+0.285936697 container start 3ea8f6b90b37dc7f3d2d1f3cf39a36dd34c8f5de1023e8d58569e9daca5031bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11-deactivate, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-03-28T03:48:53.600 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:53 smithi165 podman[124476]: 2024-03-28 03:48:53.59888333 +0000 UTC m=+0.357454521 container attach 3ea8f6b90b37dc7f3d2d1f3cf39a36dd34c8f5de1023e8d58569e9daca5031bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11-deactivate, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-28T03:48:53.878 INFO:journalctl@ceph.osd.11.smithi165.stdout:Mar 28 03:48:53 smithi165 podman[124476]: 2024-03-28 03:48:53.814264843 +0000 UTC m=+0.572836020 container died 3ea8f6b90b37dc7f3d2d1f3cf39a36dd34c8f5de1023e8d58569e9daca5031bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-osd-11-deactivate, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-03-28T03:48:54.378 DEBUG:teuthology.orchestra.run.smithi165:> sudo pkill -f 'journalctl -f -n 0 -u ceph-10f73b92-ec98-11ee-95d0-87774f69a715@osd.11.service' 2024-03-28T03:48:54.420 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-28T03:48:54.420 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-03-28T03:48:54.421 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-03-28T03:48:54.421 DEBUG:teuthology.orchestra.run.smithi119:> sudo systemctl stop ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.a 2024-03-28T03:48:54.902 INFO:journalctl@ceph.mds.a.smithi119.stdout:Mar 28 03:48:54 smithi119 systemd[1]: Stopping Ceph mds.a for 10f73b92-ec98-11ee-95d0-87774f69a715... 2024-03-28T03:48:55.401 INFO:journalctl@ceph.mds.a.smithi119.stdout:Mar 28 03:48:54 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-a[70147]: 2024-03-28T03:48:54.946+0000 7f389aa00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-28T03:48:55.401 INFO:journalctl@ceph.mds.a.smithi119.stdout:Mar 28 03:48:54 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-a[70147]: 2024-03-28T03:48:54.946+0000 7f389aa00700 -1 mds.a *** got signal Terminated *** 2024-03-28T03:48:57.789 INFO:journalctl@ceph.mds.a.smithi119.stdout:Mar 28 03:48:57 smithi119 podman[173849]: 2024-03-28 03:48:57.514189307 +0000 UTC m=+2.665578295 container died a8cf73f63bdb7aff54cf7961135851c82b6e20cdd59f7781d68590c388401d4f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-a, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2) 2024-03-28T03:48:58.110 INFO:journalctl@ceph.mds.a.smithi119.stdout:Mar 28 03:48:57 smithi119 podman[173849]: 2024-03-28 03:48:57.898942304 +0000 UTC m=+3.050331298 container cleanup a8cf73f63bdb7aff54cf7961135851c82b6e20cdd59f7781d68590c388401d4f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-a, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-03-28T03:48:58.110 INFO:journalctl@ceph.mds.a.smithi119.stdout:Mar 28 03:48:57 smithi119 bash[173849]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-a 2024-03-28T03:48:58.401 INFO:journalctl@ceph.mds.a.smithi119.stdout:Mar 28 03:48:58 smithi119 podman[173875]: 2024-03-28 03:48:58.108089833 +0000 UTC m=+0.575924303 container remove a8cf73f63bdb7aff54cf7961135851c82b6e20cdd59f7781d68590c388401d4f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-a, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-03-28T03:48:58.630 DEBUG:teuthology.orchestra.run.smithi119:> sudo pkill -f 'journalctl -f -n 0 -u ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.a.service' 2024-03-28T03:48:58.746 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-28T03:48:58.746 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-03-28T03:48:58.747 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-03-28T03:48:58.747 DEBUG:teuthology.orchestra.run.smithi119:> sudo systemctl stop ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.d 2024-03-28T03:48:58.901 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 03:48:58 smithi119 systemd[1]: Stopping Ceph mds.d for 10f73b92-ec98-11ee-95d0-87774f69a715... 2024-03-28T03:48:59.651 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 03:48:59 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[121385]: 2024-03-28T03:48:59.299+0000 7f9f4e000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-28T03:48:59.651 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 03:48:59 smithi119 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d[121385]: 2024-03-28T03:48:59.299+0000 7f9f4e000700 -1 mds.d *** got signal Terminated *** 2024-03-28T03:49:01.791 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 03:49:01 smithi119 podman[173981]: 2024-03-28 03:49:01.515846441 +0000 UTC m=+2.348524532 container died 49458320a2c67c738c4eb9535b3ccb34df887bf7469714ed67f6aa6e8aef6ff8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-03-28T03:49:02.070 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 03:49:01 smithi119 podman[173981]: 2024-03-28 03:49:01.901669769 +0000 UTC m=+2.734347833 container cleanup 49458320a2c67c738c4eb9535b3ccb34df887bf7469714ed67f6aa6e8aef6ff8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-03-28T03:49:02.071 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 03:49:01 smithi119 bash[173981]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d 2024-03-28T03:49:02.401 INFO:journalctl@ceph.mds.d.smithi119.stdout:Mar 28 03:49:02 smithi119 podman[173995]: 2024-03-28 03:49:02.069032898 +0000 UTC m=+0.534987848 container remove 49458320a2c67c738c4eb9535b3ccb34df887bf7469714ed67f6aa6e8aef6ff8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-d, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, GIT_CLEAN=True, ceph=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-03-28T03:49:02.516 DEBUG:teuthology.orchestra.run.smithi119:> sudo pkill -f 'journalctl -f -n 0 -u ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.d.service' 2024-03-28T03:49:02.627 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-28T03:49:02.627 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-03-28T03:49:02.627 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-03-28T03:49:02.627 DEBUG:teuthology.orchestra.run.smithi119:> sudo systemctl stop ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.g 2024-03-28T03:49:02.779 DEBUG:teuthology.orchestra.run.smithi119:> sudo pkill -f 'journalctl -f -n 0 -u ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.g.service' 2024-03-28T03:49:02.888 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-28T03:49:02.889 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-03-28T03:49:02.889 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-03-28T03:49:02.889 DEBUG:teuthology.orchestra.run.smithi119:> sudo systemctl stop ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.j 2024-03-28T03:49:03.036 DEBUG:teuthology.orchestra.run.smithi119:> sudo pkill -f 'journalctl -f -n 0 -u ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.j.service' 2024-03-28T03:49:03.144 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-28T03:49:03.145 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-03-28T03:49:03.145 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-03-28T03:49:03.145 DEBUG:teuthology.orchestra.run.smithi132:> sudo systemctl stop ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.b 2024-03-28T03:49:03.434 INFO:journalctl@ceph.mds.b.smithi132.stdout:Mar 28 03:49:03 smithi132 systemd[1]: Stopping Ceph mds.b for 10f73b92-ec98-11ee-95d0-87774f69a715... 2024-03-28T03:49:03.698 INFO:journalctl@ceph.mds.b.smithi132.stdout:Mar 28 03:49:03 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-b[66443]: 2024-03-28T03:49:03.432+0000 7f7cbc920700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-28T03:49:03.698 INFO:journalctl@ceph.mds.b.smithi132.stdout:Mar 28 03:49:03 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-b[66443]: 2024-03-28T03:49:03.432+0000 7f7cbc920700 -1 mds.b *** got signal Terminated *** 2024-03-28T03:49:05.783 INFO:journalctl@ceph.mds.b.smithi132.stdout:Mar 28 03:49:05 smithi132 podman[121401]: 2024-03-28 03:49:05.476316788 +0000 UTC m=+2.142024985 container died fe3fee15a0f94a3d3be8b3da321cffc54b745c71edf1b0c7685ea583d6dd0aa1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-b, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True) 2024-03-28T03:49:05.783 INFO:journalctl@ceph.mds.b.smithi132.stdout:Mar 28 03:49:05 smithi132 podman[121401]: 2024-03-28 03:49:05.782277929 +0000 UTC m=+2.447986118 container cleanup fe3fee15a0f94a3d3be8b3da321cffc54b745c71edf1b0c7685ea583d6dd0aa1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-b, GIT_CLEAN=True, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-03-28T03:49:06.068 INFO:journalctl@ceph.mds.b.smithi132.stdout:Mar 28 03:49:05 smithi132 bash[121401]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-b 2024-03-28T03:49:06.068 INFO:journalctl@ceph.mds.b.smithi132.stdout:Mar 28 03:49:05 smithi132 podman[121415]: 2024-03-28 03:49:05.958225317 +0000 UTC m=+0.480821523 container remove fe3fee15a0f94a3d3be8b3da321cffc54b745c71edf1b0c7685ea583d6dd0aa1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-b, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-03-28T03:49:06.087 DEBUG:teuthology.orchestra.run.smithi132:> sudo pkill -f 'journalctl -f -n 0 -u ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.b.service' 2024-03-28T03:49:06.122 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-28T03:49:06.122 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-03-28T03:49:06.122 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-03-28T03:49:06.123 DEBUG:teuthology.orchestra.run.smithi132:> sudo systemctl stop ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.e 2024-03-28T03:49:06.437 INFO:journalctl@ceph.mds.e.smithi132.stdout:Mar 28 03:49:06 smithi132 systemd[1]: Stopping Ceph mds.e for 10f73b92-ec98-11ee-95d0-87774f69a715... 2024-03-28T03:49:06.699 INFO:journalctl@ceph.mds.e.smithi132.stdout:Mar 28 03:49:06 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-e[65818]: 2024-03-28T03:49:06.435+0000 7f5a58261700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-28T03:49:06.700 INFO:journalctl@ceph.mds.e.smithi132.stdout:Mar 28 03:49:06 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-e[65818]: 2024-03-28T03:49:06.435+0000 7f5a58261700 -1 mds.e *** got signal Terminated *** 2024-03-28T03:49:09.730 INFO:journalctl@ceph.mds.e.smithi132.stdout:Mar 28 03:49:09 smithi132 podman[121521]: 2024-03-28 03:49:09.468237475 +0000 UTC m=+3.135178841 container died 6ff65ee54ee3f446aad493c093461f7d21f234459ef45191186beff56d381adb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-e, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-03-28T03:49:10.088 INFO:journalctl@ceph.mds.e.smithi132.stdout:Mar 28 03:49:09 smithi132 podman[121521]: 2024-03-28 03:49:09.819285086 +0000 UTC m=+3.486226446 container cleanup 6ff65ee54ee3f446aad493c093461f7d21f234459ef45191186beff56d381adb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, CEPH_POINT_RELEASE=) 2024-03-28T03:49:10.088 INFO:journalctl@ceph.mds.e.smithi132.stdout:Mar 28 03:49:09 smithi132 bash[121521]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-e 2024-03-28T03:49:10.220 DEBUG:teuthology.orchestra.run.smithi132:> sudo pkill -f 'journalctl -f -n 0 -u ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.e.service' 2024-03-28T03:49:10.254 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-28T03:49:10.255 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-03-28T03:49:10.255 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-03-28T03:49:10.255 DEBUG:teuthology.orchestra.run.smithi132:> sudo systemctl stop ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.h 2024-03-28T03:49:10.448 INFO:journalctl@ceph.mds.h.smithi132.stdout:Mar 28 03:49:10 smithi132 systemd[1]: Stopping Ceph mds.h for 10f73b92-ec98-11ee-95d0-87774f69a715... 2024-03-28T03:49:10.948 INFO:journalctl@ceph.mds.h.smithi132.stdout:Mar 28 03:49:10 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-h[66129]: 2024-03-28T03:49:10.563+0000 7f0508ee6700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-28T03:49:10.949 INFO:journalctl@ceph.mds.h.smithi132.stdout:Mar 28 03:49:10 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-h[66129]: 2024-03-28T03:49:10.563+0000 7f0508ee6700 -1 mds.h *** got signal Terminated *** 2024-03-28T03:49:13.773 INFO:journalctl@ceph.mds.h.smithi132.stdout:Mar 28 03:49:13 smithi132 podman[121643]: 2024-03-28 03:49:13.466231287 +0000 UTC m=+2.997481252 container died 2b9188b0f91b2effcad70e5d7bf2773db9ef9f79c6227489c54f214032e800ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-h, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-28T03:49:14.101 INFO:journalctl@ceph.mds.h.smithi132.stdout:Mar 28 03:49:13 smithi132 podman[121643]: 2024-03-28 03:49:13.77285958 +0000 UTC m=+3.304109547 container cleanup 2b9188b0f91b2effcad70e5d7bf2773db9ef9f79c6227489c54f214032e800ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-h, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-28T03:49:14.101 INFO:journalctl@ceph.mds.h.smithi132.stdout:Mar 28 03:49:13 smithi132 bash[121643]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-h 2024-03-28T03:49:14.101 INFO:journalctl@ceph.mds.h.smithi132.stdout:Mar 28 03:49:13 smithi132 podman[121657]: 2024-03-28 03:49:13.982256345 +0000 UTC m=+0.517199621 container remove 2b9188b0f91b2effcad70e5d7bf2773db9ef9f79c6227489c54f214032e800ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-h, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True) 2024-03-28T03:49:14.120 DEBUG:teuthology.orchestra.run.smithi132:> sudo pkill -f 'journalctl -f -n 0 -u ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.h.service' 2024-03-28T03:49:14.153 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-28T03:49:14.154 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-03-28T03:49:14.154 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-03-28T03:49:14.154 DEBUG:teuthology.orchestra.run.smithi132:> sudo systemctl stop ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.k 2024-03-28T03:49:14.448 INFO:journalctl@ceph.mds.k.smithi132.stdout:Mar 28 03:49:14 smithi132 systemd[1]: Stopping Ceph mds.k for 10f73b92-ec98-11ee-95d0-87774f69a715... 2024-03-28T03:49:14.948 INFO:journalctl@ceph.mds.k.smithi132.stdout:Mar 28 03:49:14 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-k[65498]: 2024-03-28T03:49:14.477+0000 7f7aabd26700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-28T03:49:14.948 INFO:journalctl@ceph.mds.k.smithi132.stdout:Mar 28 03:49:14 smithi132 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-k[65498]: 2024-03-28T03:49:14.477+0000 7f7aabd26700 -1 mds.k *** got signal Terminated *** 2024-03-28T03:49:17.728 INFO:journalctl@ceph.mds.k.smithi132.stdout:Mar 28 03:49:17 smithi132 podman[121763]: 2024-03-28 03:49:17.467361586 +0000 UTC m=+3.100923288 container died 90f8d0de7d8a42a54de9c0acf392e4eeaeaa3c3694c55603e96b844552c3d82e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-k, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.build-date=20240319, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-28T03:49:17.986 INFO:journalctl@ceph.mds.k.smithi132.stdout:Mar 28 03:49:17 smithi132 podman[121763]: 2024-03-28 03:49:17.809762186 +0000 UTC m=+3.443323893 container cleanup 90f8d0de7d8a42a54de9c0acf392e4eeaeaa3c3694c55603e96b844552c3d82e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-k, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-03-28T03:49:17.987 INFO:journalctl@ceph.mds.k.smithi132.stdout:Mar 28 03:49:17 smithi132 bash[121763]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-k 2024-03-28T03:49:18.237 INFO:journalctl@ceph.mds.k.smithi132.stdout:Mar 28 03:49:17 smithi132 podman[121790]: 2024-03-28 03:49:17.985827075 +0000 UTC m=+0.517265177 container remove 90f8d0de7d8a42a54de9c0acf392e4eeaeaa3c3694c55603e96b844552c3d82e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-k, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c) 2024-03-28T03:49:18.238 INFO:journalctl@ceph.mds.k.smithi132.stdout:Mar 28 03:49:18 smithi132 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.k.service: Deactivated successfully. 2024-03-28T03:49:18.238 INFO:journalctl@ceph.mds.k.smithi132.stdout:Mar 28 03:49:18 smithi132 systemd[1]: Stopped Ceph mds.k for 10f73b92-ec98-11ee-95d0-87774f69a715. 2024-03-28T03:49:18.238 INFO:journalctl@ceph.mds.k.smithi132.stdout:Mar 28 03:49:18 smithi132 systemd[1]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.k.service: Consumed 13.467s CPU time. 2024-03-28T03:49:18.239 DEBUG:teuthology.orchestra.run.smithi132:> sudo pkill -f 'journalctl -f -n 0 -u ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.k.service' 2024-03-28T03:49:18.275 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-28T03:49:18.275 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-03-28T03:49:18.275 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-03-28T03:49:18.275 DEBUG:teuthology.orchestra.run.smithi165:> sudo systemctl stop ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.c 2024-03-28T03:49:18.313 DEBUG:teuthology.orchestra.run.smithi165:> sudo pkill -f 'journalctl -f -n 0 -u ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.c.service' 2024-03-28T03:49:18.391 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-28T03:49:18.391 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-03-28T03:49:18.391 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-03-28T03:49:18.392 DEBUG:teuthology.orchestra.run.smithi165:> sudo systemctl stop ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.f 2024-03-28T03:49:18.628 INFO:journalctl@ceph.mds.f.smithi165.stdout:Mar 28 03:49:18 smithi165 systemd[1]: Stopping Ceph mds.f for 10f73b92-ec98-11ee-95d0-87774f69a715... 2024-03-28T03:49:18.894 INFO:journalctl@ceph.mds.f.smithi165.stdout:Mar 28 03:49:18 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-f[67456]: 2024-03-28T03:49:18.801+0000 7fd7c3680700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-28T03:49:18.894 INFO:journalctl@ceph.mds.f.smithi165.stdout:Mar 28 03:49:18 smithi165 ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-f[67456]: 2024-03-28T03:49:18.801+0000 7fd7c3680700 -1 mds.f *** got signal Terminated *** 2024-03-28T03:49:21.719 INFO:journalctl@ceph.mds.f.smithi165.stdout:Mar 28 03:49:21 smithi165 podman[124618]: 2024-03-28 03:49:21.45674006 +0000 UTC m=+2.848782436 container died 935245452bf88b28981fcd8b5cd9517d05d9147ac88ba8b31b6af1887d61e3f4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-f, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux ) 2024-03-28T03:49:22.078 INFO:journalctl@ceph.mds.f.smithi165.stdout:Mar 28 03:49:21 smithi165 podman[124618]: 2024-03-28 03:49:21.82570359 +0000 UTC m=+3.217745954 container cleanup 935245452bf88b28981fcd8b5cd9517d05d9147ac88ba8b31b6af1887d61e3f4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-f, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-03-28T03:49:22.078 INFO:journalctl@ceph.mds.f.smithi165.stdout:Mar 28 03:49:21 smithi165 bash[124618]: ceph-10f73b92-ec98-11ee-95d0-87774f69a715-mds-f 2024-03-28T03:49:22.242 DEBUG:teuthology.orchestra.run.smithi165:> sudo pkill -f 'journalctl -f -n 0 -u ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.f.service' 2024-03-28T03:49:22.279 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-28T03:49:22.279 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-03-28T03:49:22.280 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-03-28T03:49:22.280 DEBUG:teuthology.orchestra.run.smithi165:> sudo systemctl stop ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.i 2024-03-28T03:49:22.351 DEBUG:teuthology.orchestra.run.smithi165:> sudo pkill -f 'journalctl -f -n 0 -u ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.i.service' 2024-03-28T03:49:22.424 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-28T03:49:22.425 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-03-28T03:49:22.425 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-03-28T03:49:22.425 DEBUG:teuthology.orchestra.run.smithi165:> sudo systemctl stop ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.l 2024-03-28T03:49:22.495 DEBUG:teuthology.orchestra.run.smithi165:> sudo pkill -f 'journalctl -f -n 0 -u ceph-10f73b92-ec98-11ee-95d0-87774f69a715@mds.l.service' 2024-03-28T03:49:22.568 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-28T03:49:22.569 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-03-28T03:49:22.569 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 10f73b92-ec98-11ee-95d0-87774f69a715 --force --keep-logs 2024-03-28T03:49:22.897 INFO:teuthology.orchestra.run.smithi119.stdout:Deleting cluster with fsid: 10f73b92-ec98-11ee-95d0-87774f69a715 2024-03-28T03:49:28.505 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 10f73b92-ec98-11ee-95d0-87774f69a715 --force --keep-logs 2024-03-28T03:49:28.695 INFO:teuthology.orchestra.run.smithi132.stdout:Deleting cluster with fsid: 10f73b92-ec98-11ee-95d0-87774f69a715 2024-03-28T03:49:31.124 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 10f73b92-ec98-11ee-95d0-87774f69a715 --force --keep-logs 2024-03-28T03:49:31.314 INFO:teuthology.orchestra.run.smithi165.stdout:Deleting cluster with fsid: 10f73b92-ec98-11ee-95d0-87774f69a715 2024-03-28T03:49:33.816 DEBUG:teuthology.orchestra.run.smithi119:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-28T03:49:33.919 DEBUG:teuthology.orchestra.run.smithi132:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-28T03:49:33.954 DEBUG:teuthology.orchestra.run.smithi165:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-28T03:49:33.983 INFO:tasks.cephadm:Archiving crash dumps... 2024-03-28T03:49:33.986 DEBUG:teuthology.misc:Transferring archived files from smithi119:/var/lib/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/crash to /home/teuthworker/archive/mchangir-2024-03-27_03:45:23-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7625222/remote/smithi119/crash 2024-03-28T03:49:33.987 DEBUG:teuthology.orchestra.run.smithi119:> sudo tar c -f - -C /var/lib/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/crash -- . 2024-03-28T03:49:34.093 INFO:teuthology.orchestra.run.smithi119.stderr:tar: /var/lib/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/crash: Cannot open: No such file or directory 2024-03-28T03:49:34.093 INFO:teuthology.orchestra.run.smithi119.stderr:tar: Error is not recoverable: exiting now 2024-03-28T03:49:34.098 DEBUG:teuthology.misc:Transferring archived files from smithi132:/var/lib/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/crash to /home/teuthworker/archive/mchangir-2024-03-27_03:45:23-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7625222/remote/smithi132/crash 2024-03-28T03:49:34.099 DEBUG:teuthology.orchestra.run.smithi132:> sudo tar c -f - -C /var/lib/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/crash -- . 2024-03-28T03:49:34.140 INFO:teuthology.orchestra.run.smithi132.stderr:tar: /var/lib/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/crash: Cannot open: No such file or directory 2024-03-28T03:49:34.140 INFO:teuthology.orchestra.run.smithi132.stderr:tar: Error is not recoverable: exiting now 2024-03-28T03:49:34.142 DEBUG:teuthology.misc:Transferring archived files from smithi165:/var/lib/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/crash to /home/teuthworker/archive/mchangir-2024-03-27_03:45:23-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7625222/remote/smithi165/crash 2024-03-28T03:49:34.143 DEBUG:teuthology.orchestra.run.smithi165:> sudo tar c -f - -C /var/lib/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/crash -- . 2024-03-28T03:49:34.178 INFO:teuthology.orchestra.run.smithi165.stderr:tar: /var/lib/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/crash: Cannot open: No such file or directory 2024-03-28T03:49:34.178 INFO:teuthology.orchestra.run.smithi165.stderr:tar: Error is not recoverable: exiting now 2024-03-28T03:49:34.179 INFO:tasks.cephadm:Checking cluster log for badness... 2024-03-28T03:49:34.180 DEBUG:teuthology.orchestra.run.smithi119:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | head -n 1 2024-03-28T03:49:34.349 INFO:teuthology.orchestra.run.smithi119.stdout:2024-03-28T00:17:11.583395+0000 mon.a (mon.0) 187 : cluster [WRN] Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-03-28T03:49:34.357 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-03-28T03:49:34.358 INFO:tasks.cephadm:Compressing logs... 2024-03-28T03:49:34.358 DEBUG:teuthology.orchestra.run.smithi119:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-28T03:49:34.363 DEBUG:teuthology.orchestra.run.smithi132:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-28T03:49:34.366 DEBUG:teuthology.orchestra.run.smithi165:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-28T03:49:34.393 INFO:teuthology.orchestra.run.smithi165.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-03-28T03:49:34.393 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-03-28T03:49:34.393 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/ceph-mds.l.log 2024-03-28T03:49:34.396 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/ceph.audit.log 2024-03-28T03:49:34.396 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/cephadm.log: gzip/var/log/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/ceph-mds.l.log: -5 --verbose -- /var/log/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/ceph-mds.i.log 2024-03-28T03:49:34.396 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/ceph-osd.11.log 2024-03-28T03:49:34.396 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/ceph-mgr.z.log 2024-03-28T03:49:34.396 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/ceph-mds.i.log: /var/log/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/ceph-osd.8.log 2024-03-28T03:49:34.396 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/ceph-osd.5.log 2024-03-28T03:49:34.397 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/ceph-mds.f.log 2024-03-28T03:49:34.397 INFO:teuthology.orchestra.run.smithi165.stderr: 92.0% -- replaced with /var/log/ceph/cephadm.log.gz 2024-03-28T03:49:34.397 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/ceph.cephadm.log 2024-03-28T03:49:34.398 INFO:teuthology.orchestra.run.smithi132.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-03-28T03:49:34.398 INFO:teuthology.orchestra.run.smithi132.stderr:: No such file or directory 2024-03-28T03:49:34.398 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/ceph-osd.10.log 2024-03-28T03:49:34.399 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/ceph.audit.log 2024-03-28T03:49:34.399 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/ceph-mds.b.log 2024-03-28T03:49:34.402 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/ceph-osd.4.log 2024-03-28T03:49:34.403 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5/var/log/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/ceph-mds.b.log: --verbose -- /var/log/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/ceph-osd.1.log 2024-03-28T03:49:34.403 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/ceph-mds.k.log 2024-03-28T03:49:34.403 INFO:teuthology.orchestra.run.smithi132.stderr: 91.8% -- replaced with /var/log/ceph/cephadm.log.gz 2024-03-28T03:49:34.403 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/ceph-mds.h.log 2024-03-28T03:49:34.403 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/ceph-osd.7.log 2024-03-28T03:49:34.403 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/ceph-mds.e.log 2024-03-28T03:49:34.403 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/ceph-osd.7.log: 91.4% -- replaced with /var/log/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/ceph.audit.log.gz 2024-03-28T03:49:34.403 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/ceph-mgr.y.log 2024-03-28T03:49:34.405 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/ceph-mds.f.log: 91.4% -- replaced with /var/log/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/ceph.audit.log.gz 2024-03-28T03:49:34.406 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/ceph.log 2024-03-28T03:49:34.406 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/ceph.cephadm.log: 88.1% -- replaced with /var/log/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/ceph.cephadm.log.gz 2024-03-28T03:49:34.406 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/ceph-osd.2.log 2024-03-28T03:49:34.414 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/ceph.log: gzip -5 --verbose -- /var/log/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/ceph-mds.c.log 2024-03-28T03:49:34.415 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/ceph.cephadm.log 2024-03-28T03:49:34.422 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/ceph.log 2024-03-28T03:49:34.428 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/ceph-osd.2.log: 89.9% -- replaced with /var/log/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/ceph.log.gz 2024-03-28T03:49:34.429 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/ceph-volume.log 2024-03-28T03:49:34.430 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/ceph-volume.log 2024-03-28T03:49:34.431 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/ceph.log: 88.2% -- replaced with /var/log/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/ceph.cephadm.log.gz 2024-03-28T03:49:34.438 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/ceph-mds.c.log: 94.3% -- replaced with /var/log/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/ceph-mgr.z.log.gz 2024-03-28T03:49:34.442 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/ceph-mon.c.log 2024-03-28T03:49:34.443 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/ceph-mon.b.log 2024-03-28T03:49:34.459 INFO:teuthology.orchestra.run.smithi119.stderr:find: ‘/var/log/rbd-target-api’gzip: No such file or directory -5 2024-03-28T03:49:34.459 INFO:teuthology.orchestra.run.smithi119.stderr: --verbose -- /var/log/ceph/cephadm.log 2024-03-28T03:49:34.462 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/ceph-volume.log: /var/log/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/ceph-mon.c.log: 91.1% -- replaced with /var/log/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/ceph-volume.log.gz 2024-03-28T03:49:34.470 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/ceph-osd.6.log 2024-03-28T03:49:34.472 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/ceph-mon.a.log 2024-03-28T03:49:34.473 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/ceph.audit.log 2024-03-28T03:49:34.474 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/ceph-mds.g.log 2024-03-28T03:49:34.475 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/ceph-mds.j.log 2024-03-28T03:49:34.477 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/ceph-osd.3.log 2024-03-28T03:49:34.478 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/ceph-mgr.x.log 2024-03-28T03:49:34.479 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/ceph-osd.0.log 2024-03-28T03:49:34.480 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/ceph.cephadm.log 2024-03-28T03:49:34.481 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/ceph.log 2024-03-28T03:49:34.482 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/ceph-volume.log 2024-03-28T03:49:34.483 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/ceph.log: gzip -5 --verbose -- /var/log/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/ceph-osd.9.log 2024-03-28T03:49:34.484 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/ceph-volume.log: /var/log/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/ceph-mon.b.log: 89.9% 91.1% -- replaced with /var/log/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/ceph-volume.log.gz 2024-03-28T03:49:34.485 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/ceph-mds.d.log 2024-03-28T03:49:34.486 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/ceph-mds.a.log 2024-03-28T03:49:34.493 INFO:teuthology.orchestra.run.smithi132.stderr: 94.3% -- replaced with /var/log/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/ceph-mgr.y.log.gz 2024-03-28T03:49:34.496 INFO:teuthology.orchestra.run.smithi132.stderr: -- replaced with /var/log/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/ceph.log.gz 2024-03-28T03:49:34.507 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/ceph-mds.d.log: /var/log/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/ceph-mds.a.log: 89.9% -- replaced with /var/log/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/ceph.log.gz 2024-03-28T03:49:34.740 INFO:teuthology.orchestra.run.smithi119.stderr: 91.0% -- replaced with /var/log/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/ceph-volume.log.gz 2024-03-28T03:49:34.760 INFO:teuthology.orchestra.run.smithi119.stderr: 91.3% -- replaced with /var/log/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/ceph.audit.log.gz 2024-03-28T03:49:34.946 INFO:teuthology.orchestra.run.smithi119.stderr: 93.5% -- replaced with /var/log/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/ceph-mds.d.log.gz 2024-03-28T03:49:34.991 INFO:teuthology.orchestra.run.smithi119.stderr: 93.0% -- replaced with /var/log/ceph/cephadm.log.gz 2024-03-28T03:49:34.994 INFO:teuthology.orchestra.run.smithi119.stderr: 87.7% -- replaced with /var/log/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/ceph.cephadm.log.gz 2024-03-28T03:49:35.048 INFO:teuthology.orchestra.run.smithi165.stderr: 93.5% -- replaced with /var/log/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/ceph-mds.i.log.gz 2024-03-28T03:49:35.108 INFO:teuthology.orchestra.run.smithi165.stderr: 94.3% -- replaced with /var/log/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/ceph-mds.f.log.gz 2024-03-28T03:49:35.125 INFO:teuthology.orchestra.run.smithi119.stderr: 93.7% -- replaced with /var/log/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/ceph-mds.g.log.gz 2024-03-28T03:49:35.161 INFO:teuthology.orchestra.run.smithi165.stderr: 93.9% -- replaced with /var/log/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/ceph-mds.c.log.gz 2024-03-28T03:49:35.285 INFO:teuthology.orchestra.run.smithi165.stderr: 94.3% -- replaced with /var/log/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/ceph-mds.l.log.gz 2024-03-28T03:49:35.359 INFO:teuthology.orchestra.run.smithi132.stderr: 95.0% -- replaced with /var/log/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/ceph-mds.k.log.gz 2024-03-28T03:49:35.487 INFO:teuthology.orchestra.run.smithi119.stderr: 93.0% -- replaced with /var/log/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/ceph-mds.j.log.gz 2024-03-28T03:49:35.613 INFO:teuthology.orchestra.run.smithi132.stderr: 95.0% -- replaced with /var/log/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/ceph-mds.h.log.gz 2024-03-28T03:49:35.751 INFO:teuthology.orchestra.run.smithi119.stderr: 95.1% -- replaced with /var/log/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/ceph-mds.a.log.gz 2024-03-28T03:49:37.954 INFO:teuthology.orchestra.run.smithi165.stderr: 91.8% -- replaced with /var/log/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/ceph-mon.c.log.gz 2024-03-28T03:49:38.446 INFO:teuthology.orchestra.run.smithi119.stderr: 90.2% -- replaced with /var/log/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/ceph-mgr.x.log.gz 2024-03-28T03:49:38.850 INFO:teuthology.orchestra.run.smithi132.stderr: 92.0% -- replaced with /var/log/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/ceph-mon.b.log.gz 2024-03-28T03:49:42.015 INFO:teuthology.orchestra.run.smithi165.stderr: 92.8% -- replaced with /var/log/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/ceph-osd.11.log.gz 2024-03-28T03:49:42.558 INFO:teuthology.orchestra.run.smithi132.stderr: 95.2% -- replaced with /var/log/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/ceph-mds.e.log.gz 2024-03-28T03:49:42.968 INFO:teuthology.orchestra.run.smithi132.stderr: 92.7% -- replaced with /var/log/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/ceph-osd.4.log.gz 2024-03-28T03:49:45.263 INFO:teuthology.orchestra.run.smithi132.stderr: 92.9% -- replaced with /var/log/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/ceph-osd.1.log.gz 2024-03-28T03:49:45.507 INFO:teuthology.orchestra.run.smithi119.stderr: 92.8% -- replaced with /var/log/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/ceph-osd.0.log.gz 2024-03-28T03:49:45.985 INFO:teuthology.orchestra.run.smithi165.stderr: 92.0% -- replaced with /var/log/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/ceph-osd.5.log.gz 2024-03-28T03:49:46.528 INFO:teuthology.orchestra.run.smithi165.stderr: 92.5% -- replaced with /var/log/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/ceph-osd.2.log.gz 2024-03-28T03:49:46.794 INFO:teuthology.orchestra.run.smithi132.stderr: 92.6% -- replaced with /var/log/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/ceph-osd.7.log.gz 2024-03-28T03:49:47.057 INFO:teuthology.orchestra.run.smithi119.stderr: 90.0% -- replaced with /var/log/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/ceph-mon.a.log.gz 2024-03-28T03:49:47.254 INFO:teuthology.orchestra.run.smithi165.stderr: 92.1% -- replaced with /var/log/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/ceph-osd.8.log.gz 2024-03-28T03:49:47.256 INFO:teuthology.orchestra.run.smithi165.stderr: 2024-03-28T03:49:47.256 INFO:teuthology.orchestra.run.smithi165.stderr:real 0m12.876s 2024-03-28T03:49:47.256 INFO:teuthology.orchestra.run.smithi165.stderr:user 0m48.412s 2024-03-28T03:49:47.256 INFO:teuthology.orchestra.run.smithi165.stderr:sys 0m1.646s 2024-03-28T03:49:47.547 INFO:teuthology.orchestra.run.smithi132.stderr: 91.9% -- replaced with /var/log/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/ceph-osd.10.log.gz 2024-03-28T03:49:47.647 INFO:teuthology.orchestra.run.smithi119.stderr: 92.5% -- replaced with /var/log/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/ceph-osd.3.log.gz 2024-03-28T03:49:50.917 INFO:teuthology.orchestra.run.smithi119.stderr: 91.9% -- replaced with /var/log/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/ceph-osd.9.log.gz 2024-03-28T03:49:52.348 INFO:teuthology.orchestra.run.smithi119.stderr: 91.9% -- replaced with /var/log/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/ceph-osd.6.log.gz 2024-03-28T03:49:52.353 INFO:teuthology.orchestra.run.smithi119.stderr: 2024-03-28T03:49:52.353 INFO:teuthology.orchestra.run.smithi119.stderr:real 0m17.951s 2024-03-28T03:49:52.353 INFO:teuthology.orchestra.run.smithi119.stderr:user 0m52.212s 2024-03-28T03:49:52.353 INFO:teuthology.orchestra.run.smithi119.stderr:sys 0m4.542s 2024-03-28T03:50:45.789 INFO:teuthology.orchestra.run.smithi132.stderr: 93.4% -- replaced with /var/log/ceph/10f73b92-ec98-11ee-95d0-87774f69a715/ceph-mds.b.log.gz 2024-03-28T03:50:45.791 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-28T03:50:45.792 INFO:teuthology.orchestra.run.smithi132.stderr:real 1m11.411s 2024-03-28T03:50:45.792 INFO:teuthology.orchestra.run.smithi132.stderr:user 2m6.119s 2024-03-28T03:50:45.792 INFO:teuthology.orchestra.run.smithi132.stderr:sys 0m3.864s 2024-03-28T03:50:45.792 INFO:tasks.cephadm:Archiving logs... 2024-03-28T03:50:45.793 DEBUG:teuthology.misc:Transferring archived files from smithi119:/var/log/ceph to /home/teuthworker/archive/mchangir-2024-03-27_03:45:23-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7625222/remote/smithi119/log 2024-03-28T03:50:45.794 DEBUG:teuthology.orchestra.run.smithi119:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-28T03:50:52.893 DEBUG:teuthology.misc:Transferring archived files from smithi132:/var/log/ceph to /home/teuthworker/archive/mchangir-2024-03-27_03:45:23-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7625222/remote/smithi132/log 2024-03-28T03:50:52.894 DEBUG:teuthology.orchestra.run.smithi132:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-28T03:51:08.709 DEBUG:teuthology.misc:Transferring archived files from smithi165:/var/log/ceph to /home/teuthworker/archive/mchangir-2024-03-27_03:45:23-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7625222/remote/smithi165/log 2024-03-28T03:51:08.710 DEBUG:teuthology.orchestra.run.smithi165:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-28T03:51:15.665 INFO:tasks.cephadm:Removing cluster... 2024-03-28T03:51:15.665 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 10f73b92-ec98-11ee-95d0-87774f69a715 --force 2024-03-28T03:51:15.997 INFO:teuthology.orchestra.run.smithi119.stdout:Deleting cluster with fsid: 10f73b92-ec98-11ee-95d0-87774f69a715 2024-03-28T03:51:16.815 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 10f73b92-ec98-11ee-95d0-87774f69a715 --force 2024-03-28T03:51:17.005 INFO:teuthology.orchestra.run.smithi132.stdout:Deleting cluster with fsid: 10f73b92-ec98-11ee-95d0-87774f69a715 2024-03-28T03:51:17.419 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 10f73b92-ec98-11ee-95d0-87774f69a715 --force 2024-03-28T03:51:17.614 INFO:teuthology.orchestra.run.smithi165.stdout:Deleting cluster with fsid: 10f73b92-ec98-11ee-95d0-87774f69a715 2024-03-28T03:51:18.000 INFO:tasks.cephadm:Removing cephadm ... 2024-03-28T03:51:18.001 DEBUG:teuthology.orchestra.run.smithi119:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-28T03:51:18.052 DEBUG:teuthology.orchestra.run.smithi132:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-28T03:51:18.067 DEBUG:teuthology.orchestra.run.smithi165:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-28T03:51:18.083 INFO:tasks.cephadm:Teardown complete 2024-03-28T03:51:18.083 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-03-28T03:51:18.094 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-03-28T03:51:18.094 DEBUG:teuthology.orchestra.run.smithi119:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-28T03:51:18.101 DEBUG:teuthology.orchestra.run.smithi132:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-28T03:51:18.110 DEBUG:teuthology.orchestra.run.smithi165:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-28T03:51:18.244 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-03-28T03:51:18.254 INFO:teuthology.task.clock:Checking final clock skew... 2024-03-28T03:51:18.254 DEBUG:teuthology.orchestra.run.smithi119:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-28T03:51:18.261 DEBUG:teuthology.orchestra.run.smithi132:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-28T03:51:18.264 DEBUG:teuthology.orchestra.run.smithi165:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-28T03:51:18.275 INFO:teuthology.orchestra.run.smithi132.stderr:bash: line 1: ntpq: command not found 2024-03-28T03:51:18.277 INFO:teuthology.orchestra.run.smithi165.stderr:bash: line 1: ntpq: command not found 2024-03-28T03:51:18.280 INFO:teuthology.orchestra.run.smithi132.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-28T03:51:18.280 INFO:teuthology.orchestra.run.smithi132.stdout:=============================================================================== 2024-03-28T03:51:18.280 INFO:teuthology.orchestra.run.smithi132.stdout:^+ hv01.front.sepia.ceph.com 2 8 377 125 +91us[ +91us] +/- 86ms 2024-03-28T03:51:18.280 INFO:teuthology.orchestra.run.smithi132.stdout:^* hv02.front.sepia.ceph.com 3 8 377 242 -1185us[-1092us] +/- 67ms 2024-03-28T03:51:18.280 INFO:teuthology.orchestra.run.smithi132.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 58 +1365us[+1365us] +/- 65ms 2024-03-28T03:51:18.281 INFO:teuthology.orchestra.run.smithi132.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-03-28T03:51:18.282 INFO:teuthology.orchestra.run.smithi165.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-28T03:51:18.282 INFO:teuthology.orchestra.run.smithi165.stdout:=============================================================================== 2024-03-28T03:51:18.282 INFO:teuthology.orchestra.run.smithi165.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 56 -33us[+2572ns] +/- 87ms 2024-03-28T03:51:18.282 INFO:teuthology.orchestra.run.smithi165.stdout:^* hv02.front.sepia.ceph.com 3 7 377 4 -1187us[-1148us] +/- 70ms 2024-03-28T03:51:18.282 INFO:teuthology.orchestra.run.smithi165.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 50 +1271us[+1307us] +/- 65ms 2024-03-28T03:51:18.282 INFO:teuthology.orchestra.run.smithi165.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-03-28T03:51:18.302 INFO:teuthology.orchestra.run.smithi119.stderr:bash: line 1: ntpq: command not found 2024-03-28T03:51:18.311 INFO:teuthology.orchestra.run.smithi119.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-28T03:51:18.311 INFO:teuthology.orchestra.run.smithi119.stdout:=============================================================================== 2024-03-28T03:51:18.311 INFO:teuthology.orchestra.run.smithi119.stdout:^+ hv01.front.sepia.ceph.com 2 8 377 180 +61us[ +65us] +/- 85ms 2024-03-28T03:51:18.311 INFO:teuthology.orchestra.run.smithi119.stdout:^* hv02.front.sepia.ceph.com 3 7 377 127 -1197us[-1192us] +/- 68ms 2024-03-28T03:51:18.311 INFO:teuthology.orchestra.run.smithi119.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 41 +1317us[+1317us] +/- 65ms 2024-03-28T03:51:18.312 INFO:teuthology.orchestra.run.smithi119.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-03-28T03:51:18.312 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-03-28T03:51:18.322 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-03-28T03:51:18.323 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-03-28T03:51:18.344 DEBUG:teuthology.orchestra.run.smithi119:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-28T03:51:18.659 DEBUG:teuthology.orchestra.run.smithi132:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-28T03:51:18.743 DEBUG:teuthology.orchestra.run.smithi165:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-28T03:51:18.830 DEBUG:teuthology.orchestra.run.smithi119:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-28T03:51:18.934 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-28T03:51:18.935 DEBUG:teuthology.orchestra.run.smithi132:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-28T03:51:18.966 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-28T03:51:18.966 DEBUG:teuthology.orchestra.run.smithi165:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-28T03:51:18.998 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-28T03:51:18.998 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-03-28T03:51:19.009 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-03-28T03:51:19.044 INFO:teuthology.task.internal:Duration was 13553.548279 seconds 2024-03-28T03:51:19.045 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-03-28T03:51:19.070 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-03-28T03:51:19.070 DEBUG:teuthology.orchestra.run.smithi119:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-28T03:51:19.078 DEBUG:teuthology.orchestra.run.smithi132:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-28T03:51:19.080 DEBUG:teuthology.orchestra.run.smithi165:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-28T03:51:19.120 INFO:teuthology.orchestra.run.smithi132.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-28T03:51:19.123 INFO:teuthology.orchestra.run.smithi165.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-28T03:51:19.291 INFO:teuthology.orchestra.run.smithi119.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-28T03:51:19.831 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-03-28T03:51:19.831 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi119.front.sepia.ceph.com 2024-03-28T03:51:19.832 DEBUG:teuthology.orchestra.run.smithi119:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-28T03:51:19.898 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi132.front.sepia.ceph.com 2024-03-28T03:51:19.899 DEBUG:teuthology.orchestra.run.smithi132:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-28T03:51:19.929 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi165.front.sepia.ceph.com 2024-03-28T03:51:19.929 DEBUG:teuthology.orchestra.run.smithi165:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-28T03:51:19.959 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-03-28T03:51:19.959 DEBUG:teuthology.orchestra.run.smithi119:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-28T03:51:19.967 DEBUG:teuthology.orchestra.run.smithi132:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-28T03:51:19.972 DEBUG:teuthology.orchestra.run.smithi165:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-28T03:51:20.056 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-03-28T03:51:20.056 DEBUG:teuthology.orchestra.run.smithi119:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-28T03:51:20.062 DEBUG:teuthology.orchestra.run.smithi132:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-28T03:51:20.064 DEBUG:teuthology.orchestra.run.smithi165:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-28T03:51:20.948 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-03-28T03:51:20.959 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-03-28T03:51:20.960 DEBUG:teuthology.orchestra.run.smithi119:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-28T03:51:21.055 DEBUG:teuthology.orchestra.run.smithi132:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-28T03:51:21.084 DEBUG:teuthology.orchestra.run.smithi165:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-28T03:51:21.112 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-03-28T03:51:21.123 DEBUG:teuthology.orchestra.run.smithi119:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-28T03:51:21.131 DEBUG:teuthology.orchestra.run.smithi132:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-28T03:51:21.133 DEBUG:teuthology.orchestra.run.smithi165:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-28T03:51:21.156 INFO:teuthology.orchestra.run.smithi132.stdout:kernel.core_pattern = core 2024-03-28T03:51:21.179 INFO:teuthology.orchestra.run.smithi165.stdout:kernel.core_pattern = core 2024-03-28T03:51:21.233 INFO:teuthology.orchestra.run.smithi119.stdout:kernel.core_pattern = core 2024-03-28T03:51:21.304 DEBUG:teuthology.orchestra.run.smithi119:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-28T03:51:21.358 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-28T03:51:21.359 DEBUG:teuthology.orchestra.run.smithi132:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-28T03:51:21.372 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-28T03:51:21.373 DEBUG:teuthology.orchestra.run.smithi165:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-28T03:51:21.387 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-28T03:51:21.388 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-03-28T03:51:21.409 INFO:teuthology.task.internal:Transferring archived files... 2024-03-28T03:51:21.410 DEBUG:teuthology.misc:Transferring archived files from smithi119:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/mchangir-2024-03-27_03:45:23-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7625222/remote/smithi119 2024-03-28T03:51:21.411 DEBUG:teuthology.orchestra.run.smithi119:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-28T03:51:21.528 DEBUG:teuthology.misc:Transferring archived files from smithi132:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/mchangir-2024-03-27_03:45:23-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7625222/remote/smithi132 2024-03-28T03:51:21.528 DEBUG:teuthology.orchestra.run.smithi132:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-28T03:51:21.570 DEBUG:teuthology.misc:Transferring archived files from smithi165:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/mchangir-2024-03-27_03:45:23-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7625222/remote/smithi165 2024-03-28T03:51:21.570 DEBUG:teuthology.orchestra.run.smithi165:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-28T03:51:21.613 INFO:teuthology.task.internal:Removing archive directory... 2024-03-28T03:51:21.614 DEBUG:teuthology.orchestra.run.smithi119:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-28T03:51:21.621 DEBUG:teuthology.orchestra.run.smithi132:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-28T03:51:21.624 DEBUG:teuthology.orchestra.run.smithi165:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-28T03:51:21.671 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-03-28T03:51:21.682 INFO:teuthology.task.internal:Not uploading archives. 2024-03-28T03:51:21.682 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-03-28T03:51:21.703 INFO:teuthology.task.internal:Tidying up after the test... 2024-03-28T03:51:21.704 DEBUG:teuthology.orchestra.run.smithi119:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-28T03:51:21.711 DEBUG:teuthology.orchestra.run.smithi132:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-28T03:51:21.714 DEBUG:teuthology.orchestra.run.smithi165:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-28T03:51:21.726 INFO:teuthology.orchestra.run.smithi132.stdout: 262162 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 28 03:51 /home/ubuntu/cephtest 2024-03-28T03:51:21.727 INFO:teuthology.orchestra.run.smithi165.stdout: 262162 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 28 03:51 /home/ubuntu/cephtest 2024-03-28T03:51:21.761 INFO:teuthology.orchestra.run.smithi119.stdout: 395889 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 28 03:51 /home/ubuntu/cephtest 2024-03-28T03:51:21.764 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-03-28T03:51:21.775 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-03-28T03:51:21.820 INFO:teuthology.nuke:Checking targets against current locks 2024-03-28T03:51:21.846 DEBUG:teuthology.nuke:shortname: smithi119 2024-03-28T03:51:21.846 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-28T03:51:21.871 DEBUG:teuthology.nuke:shortname: smithi132 2024-03-28T03:51:21.872 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-28T03:51:21.876 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi119.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2024-03-27_03:45:23-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7625222', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-27 23:55:58.810273', 'locked_by': 'mchangir', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEd29ajJHtaj7DsIRisAwrNhnYTOIHHKP3komPhXBxj3BOJIQ5qd5WmnRRFVWc5g7t0POdHgNq4oOzNy6uYrIws='} 2024-03-28T03:51:21.891 DEBUG:teuthology.nuke:shortname: smithi165 2024-03-28T03:51:21.892 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-28T03:51:21.901 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi132.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2024-03-27_03:45:23-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7625222', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-27 23:55:58.813947', 'locked_by': 'mchangir', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAD8uE5unwtIM7H8gS8emVpmLElwB2ef1MrmO1+cnZryHOfIacQvr7hrIn8lM0o82Q5O3kinVH+4Ncr5A2yOMv4='} 2024-03-28T03:51:21.914 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi165.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2024-03-27_03:45:23-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7625222', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-27 23:55:58.812706', 'locked_by': 'mchangir', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLJ2qwb9kad4dYQAG+VSN9XqP+9Kl9k3R4j1cNPXAhwRYinunU5QtlJjgomp9+bRwSYpYLMKSmLGdK42XMlTBmU='} 2024-03-28T03:51:21.930 INFO:teuthology.orchestra.console.smithi119:Power off 2024-03-28T03:51:21.930 DEBUG:teuthology.orchestra.console.smithi119:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-28T03:51:21.954 DEBUG:teuthology.orchestra.console.smithi119:power off output: Chassis Power Control: Down/Off 2024-03-28T03:51:21.954 DEBUG:teuthology.orchestra.console.smithi119:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-28T03:51:21.975 INFO:teuthology.orchestra.console.smithi165:Power off 2024-03-28T03:51:21.975 DEBUG:teuthology.orchestra.console.smithi165:pexpect command: ipmitool -H smithi165.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-28T03:51:21.990 INFO:teuthology.orchestra.console.smithi132:Power off 2024-03-28T03:51:21.990 DEBUG:teuthology.orchestra.console.smithi132:pexpect command: ipmitool -H smithi132.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-28T03:51:21.999 DEBUG:teuthology.orchestra.console.smithi165:power off output: Chassis Power Control: Down/Off 2024-03-28T03:51:21.999 DEBUG:teuthology.orchestra.console.smithi165:pexpect command: ipmitool -H smithi165.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-28T03:51:22.008 DEBUG:teuthology.orchestra.console.smithi119:check power output: Chassis Power is on 2024-03-28T03:51:22.012 DEBUG:teuthology.orchestra.console.smithi132:power off output: Chassis Power Control: Down/Off 2024-03-28T03:51:22.012 DEBUG:teuthology.orchestra.console.smithi132:pexpect command: ipmitool -H smithi132.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-28T03:51:22.022 DEBUG:teuthology.orchestra.console.smithi165:check power output: Chassis Power is on 2024-03-28T03:51:22.034 DEBUG:teuthology.orchestra.console.smithi132:check power output: Chassis Power is on 2024-03-28T03:51:26.008 DEBUG:teuthology.orchestra.console.smithi119:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-28T03:51:26.023 DEBUG:teuthology.orchestra.console.smithi165:pexpect command: ipmitool -H smithi165.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-28T03:51:26.035 DEBUG:teuthology.orchestra.console.smithi132:pexpect command: ipmitool -H smithi132.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-28T03:51:26.118 DEBUG:teuthology.orchestra.console.smithi119:check power output: Chassis Power is on 2024-03-28T03:51:26.132 DEBUG:teuthology.orchestra.console.smithi165:check power output: Chassis Power is on 2024-03-28T03:51:26.144 DEBUG:teuthology.orchestra.console.smithi132:check power output: Chassis Power is on 2024-03-28T03:51:30.123 DEBUG:teuthology.orchestra.console.smithi119:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-28T03:51:30.135 DEBUG:teuthology.orchestra.console.smithi165:pexpect command: ipmitool -H smithi165.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-28T03:51:30.145 DEBUG:teuthology.orchestra.console.smithi132:pexpect command: ipmitool -H smithi132.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-28T03:51:30.236 DEBUG:teuthology.orchestra.console.smithi119:check power output: Chassis Power is off 2024-03-28T03:51:30.244 DEBUG:teuthology.orchestra.console.smithi165:check power output: Chassis Power is on 2024-03-28T03:51:30.254 DEBUG:teuthology.orchestra.console.smithi132:check power output: Chassis Power is on 2024-03-28T03:51:30.337 INFO:teuthology.orchestra.console.smithi119:Power off completed 2024-03-28T03:51:34.246 DEBUG:teuthology.orchestra.console.smithi165:pexpect command: ipmitool -H smithi165.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-28T03:51:34.255 DEBUG:teuthology.orchestra.console.smithi132:pexpect command: ipmitool -H smithi132.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-28T03:51:34.356 DEBUG:teuthology.orchestra.console.smithi165:check power output: Chassis Power is off 2024-03-28T03:51:34.365 DEBUG:teuthology.orchestra.console.smithi132:check power output: Chassis Power is off 2024-03-28T03:51:34.458 INFO:teuthology.orchestra.console.smithi165:Power off completed 2024-03-28T03:51:34.466 INFO:teuthology.orchestra.console.smithi132:Power off completed 2024-03-28T03:51:34.573 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 1df0c7e8a2354c42d738e5454ba83ea7332deac2 description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/no 5-workunit/fs/misc}} duration: 13553.54827928543 failure_reason: 'Command failed (workunit test fs/misc/dirfrag.sh) on smithi119 with status 124: ''mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=3be9a5c9bc793e11e2800a8c0c696e8b46742033 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/dirfrag.sh''' flavor: default owner: mchangir sentry_event: https://sentry.ceph.com/organizations/ceph/?query=3ee65905020244fa95eff32dd1081960 status: fail success: false 2024-03-28T03:51:34.573 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-28T03:51:34.665 INFO:teuthology.run:FAIL